Posts about 'Serendeputy'

Happy 2018

Friday, January 5th, 2018

Hey, it’s the new year.

This has not been the most productive week of work here. School was delayed two hours on Wednesday and closed on Thursday and Friday. The children are reasonably good about letting me work, but that’s a lot to ask. In the meanwhile, we had a pipe burst. We had a drawer break. Comcast demands its cable box must update, but then can’t connect to its own network. A week from heck.

On the plus side, Serendeputy v4 alpha is up and reasonably working. You can play around with it if you want. It doesn’t do much yet, but you can play with the catalog. I hope to be in full beta by the end of the month. We’ll see.

Ok, that’s all for now. I’ll write more when I have time (and something useful to say).

Hey, it’s October now.

Friday, October 13th, 2017

Things still are as they are. The world still spins, the soccer games still end with Italian Ice for everyone. The traffic on the road outside my window keeps getting worse as Waze routes people downtown through my neighborhood. This instance of WordPress is still slow as [molasses], but Cloudflare solves most problems and I still have higher priorities for my time than rewriting or moving this site.

I’m heads-down working on the next version of Serendeputy. The pixel mines are fertile, but excavating takes time. I’m lucky, though: it’s a lot easier to write a newsfeed engine when you’re not concerned with surveillance, stalking or spam, only usefulness.

The path is long, but I try to enjoy things keystroke by keystroke and day by day. Plus, I’m growing ever more expert at deciphering EXPLAIN ANALYZE and dancing with the OOM Killer.

I’ll write some more, at some point in the indeterminate future when I have something more useful to say. Or announce. Or demo. I still kick around Twitter a bit, though, if you want to say “Hi.”

How does one jump from v1 of a product to v4?

Monday, April 10th, 2017

By following a very weird road.

I wrote the first version of Serendeputy (my news engine and personal white whale) in 2008 after I left — launching the beta version in the middle of 2009. I kept improving it, but noticed soon that selling into media companies during the Great Recession did not have an especially-high close rate.

So, I joined up with Susan Hunt Stevens and co-founded Practically Green, now Wespire. I lived the startup life for a couple of years until family obligations drove me to bow out gracefully.

So I got back to product-strategy consulting and working on Serendeputy, building out what I was calling v2, a clean rebuild that was going to adjust to how the world was working in 2011.

Then, one of my consulting clients brought me on full-time and I put it aside.

A while later, our private-equity overlords shut us down.

So, I got back to product-strategy consulting and working on Serendeputy, building out what I was calling v3, a clean rebuild that was going to adjust to how the world was working in 2014.

Then, one of my consulting clients brought me on full-time and I put it aside.

A while later, my division got restructured, and my position was eliminated.

So, I got back to product-strategy consulting and working on Serendeputy, building out what I am calling v4, a clean rebuild that is adjusting to how the world works in 2017.

And, that’s where I am right now. It’s a good place to be, and I’m happy with the progress I’m making. Barring unforeseen circumstances, I should be able to have this version of the application out in the world by September.

I’ve been making good progress, but I was still a little haunted by v1. Out in the wild. Showing its age. Built before responsive design was a thing. Built before Twitter was a thing. Dated. Like gold-lamé dated.

Running Rails 2.3.2.

A tiny bit embarrassing, if I have to be totally honest.

So, after one more weekend spent debugging weird errors, I decided to mothball v1 and put up a coming-soon page for v4. That’s what’s up live at Serendeputy right now.

I would have loved to do a graceful transition from one version to the next, but the world was not leaning that way.

So, Onward!

Stay tuned. If you’re already one of the users of Serendeputy v1, I’ll send you note when the new version is ready to play with. For everyone else: if you’re interested in finding out when the new version goes live, drop me a note — jason at serendeputy — and I’ll make sure to let you know.

Now, back into work.

Quality is job 3.1

Friday, July 22nd, 2016

Ok, that’s an ‘90s-era Microsoft joke.

But, sometimes it’s true. During my unscheduled free time, I’ve been building the third version of my news engine.

V1 is Serendeputy which is still happily up and running. I wrote that in 2008-2009. Social was far less important then, and responsive design was not yet a thing; as a result, the app is showing its age.

I was working on v2 when NPR recruited me in 2014. I put it down when I took that job.

So now, it’s 2016, and it’s time for v3. The basic JTBD remain the same. I still have the tests from v2, so I’m not starting totally from scratch.

I’ve been living with this itch for the greater part of a decade. I have a pretty good idea of what it should look like. It’s time to realize that vision.

Serendeputy 2 Update, July edition

Wednesday, July 16th, 2014

Happy July, everyone!

I’m making progress, but I’ve been mostly heads-down with client work the last few weeks.

The biggest change is that I’m moving from database-driven querying to search-engine-driven querying. I set up the alpha version using Postgres and hstore, and ran some tests to see where it would fall over. It turns out that it falls over at smaller load levels than I’d like.

I’m glad I ran that set of experiments when it was just me and a couple of folks playing with it, instead of in front of the whole world.

So, I’m implementing a version using Amazon CloudSearch for some of the heavy indexing lifting. Depending on how things work out, I should be able to see if this will carry me through by the middle of August. Then, I can fire everything up again and see where the new constraint in the system will live.

It’s also giving me a good chance to go back and refactor a few of the pieces that I didn’t get right the first time. I need to tear everything down anyway, I might as well fix the other stuff while I have the tools out.

I’m hoping for beta one at some point between now and eventually. It’s not quite turning into Project Xanadu, but I worry that we’re trending that way.

Thrill-seekers prepare

Thursday, May 8th, 2014

Quick update: the alpha is close to the point where it will be available for thrill-seekers to test drive. Please ping me if you want an invite.

Progress continues.

Serendeputy 2 update for March

Tuesday, March 11th, 2014

I’m making good progress on this (increasingly) large project.

The production environment is sketched and spiked, using Amazon Web Services. I think I’ve solved the SSL issues with a combination of CloudFlare and elastic load balancer.

The crawler is up and running — I’d even say reasonably solid. It’s well-behaved, working with robots.txt files and following all the crawler best practices. Once it’s running on the AWS servers, I’ll get a better idea for how many sites just block traffic from the Amazon IPs on general principles. I have a feeling that I’ll be paying for other people’s sins.

The personalization math is done and the code is prototyped. As I’m using it, I’m tweaking the coefficients for the gestures to balance how much they can influence the profiles.

The front end is not egregiously ugly! No one would ever claim that I’m a designer, but I’m going to have that alpha in front of people to get feedback, and then bring in a real design person/team/agency for the beta version.

March is going to be an insane month in pretty much every corner of my life, but I’m still hoping to have an alpha version that I (and maybe you!) can use in the beginning of April.

Serendeputy 2 update for February

Saturday, February 1st, 2014

I’m still working diligently towards my rock-solid deadline of “Launch this thing at some point in the indeterminate future.”

I finished the gesture engine this week. Now that that’s solid, I’m spending most of my time over the next month working on the profile balancer and its complement, the homepage builder.

I’m looking at these like a chef composing a recipe: how can I balance the flavors just right? I’m working to strike the balance among recency, relevance (how it maps to your interest profile), and reputation (how important the page is on the worldwide Internet).

My AWS infrastructure is set up, and I’m running parts of the application on it right now. Making progress.

I should be able to get into alpha Real. Soon. Now. (I’m shooting for by Opening Day, but we’ll see…)

Thanks for staying tuned.

Serendeputy 2 notes

Thursday, September 5th, 2013

The children go back to school tomorrow, and the transition from Dad-mode to launch-mode begins.

After I took a week or two off to adjust to an abrupt transition, this has been an insanely productive summer. (Obviously not the “writing the blog” front, though.) I’m in the middle of rewriting Serendeputy, my personal news application, from the ground up, and I’m pretty excited by the direction it’s taken. A lot has changed since I wrote the first version in 2008-2009, and it’s been so much fun to be able to update it to match the world we’re in now.

It’s currently in private “pre-alpha” i.e., running on my laptop. I should have a thrill-seeker version available for early testing by the end of September. If you’re interested in playing with it, please drop me a line.

Tech notes for those who are interested in that sort of stuff:

  • It’s still a Rails (4.0) front end, but that’s almost entirely a thin wrapper over a series of services.
  • I’m playing with Clojure for the personalization engine. I’ve written the original logic in Ruby to make sure it works; I’m going to see if I can port it over to Clojure to get a performance and memory boost.
  • I’m pretty much all in on AWS. Serendeputy currently runs on a few Rackspace (formerly Slicehost) servers; the new version is entirely AWS-based.
  • I’m about to fire up RubyMotion to write the iOS client. I’d really rather have a native client, and I’m hoping that my using a Ruby-based environment can make that happen. If it works, then I should be able to contract out an Android version. If not, then I’ll fall back to the mobile-optimized site through the browser.

Your personal API

Tuesday, July 31st, 2012

This is geeky, but supremely interesting if you’re into that sort of thing: the Cambrian explosion of everything.

I’ve been doing a lot of thinking over the past few months about personal APIs, personal deputies and Freenon’s forkable nuggets of knowledge. Within a few years (maybe sooner), everything will have an API, and most of your personal interactions with the world will be API-mediated. I’m thinking through how Serendeputy fits into this model. Right now, it’s your personal intermediary for news, and your profile is its API. How does this extend beyond this one use case?

Mostly, I want to see what I can do to make sure that this future bends towards open, with individuals controlling their data and their life. (See the previous post on dystopian futures…)

Anyway, this is the type of stuff I’m thinking about. I want this to exist; now it’s just the small matter of implementation, distribution and paying the bills in the meantime 🙂

How to get cron to use the rbenv ruby

Friday, June 1st, 2012

With any luck, this will save people my hours of aggravation.

My problem:

I’m using .rbenv to manage the Ruby processes. When I run everything manually and through the tests, it all works great. Unfortunately, when I put it into cron, it fails.

My solution:

Step One: Find the rbenv binary:

jason:~$ which ruby

Step Two: Update the cronfile:

In my cronfile, I changed from:

* * * * * cd /home/jason/program-dir ; ruby program.rb


* * * * * cd /home/jason/program-dir ; /home/jason/.rbenv/versions/1.9.3-p194/bin/ruby program.rb


There’s probably a more Linux-fu method, but this worked. If you have a more elegant solution, please drop me a line so that I can update this post.

Merry Christmas!

Thursday, December 24th, 2009

Merry Christmas, everybody!

I’ve been a poor correspondent. Sadly, that’s unlikely to change at any time in the next couple of months. I have a major new release of Serendeputy in its final stages, and I’m also working a separate super-secret project. Lots to do. Blogging suffers.

But, I should be able to have some pictures up this weekend. I have a feeling that Sadie and Lucy are going to be happy with what Santa is going to bring them.

In the meantime, here are some sites I’ve recently added to the Serendeputy catalog which are really cool. It’s worth checking them out.

Fashion Served: I’ve been building out the fashion tree, and I found this site. It’s a showcase for some of the best fashion photography around.

(And yes, I understand the irony of someone who wears shorts and a t-shirt in his home office all day building out the fashion tree.)

Roger Ebert: Roger Ebert can’t talk anymore, but he sure can write well.

Wend Magazine: I’ve also been building out the green tree — we’ve become much more involved with green this year — and I found this site. All sorts of interesting links, articles and stories.

Clients from Hell: Freelance and design clients say the darndest things.

Special bonus, here are my top 25 Most Popular Sources at the moment: Fail Blog, Sadly No, Balloon Juice, Fashion Served, Techdirt, New York Times, Work it Mom, Consumerist, Roger Ebert, TechCrunch, Business Insider, Boston Globe, Wend Magazine, Steve Blank, Holliston Reporter, The Atlantic, Clients from Hell, ESPN Boston, Tracker Editor’s Blog, SEOMoz, Harvard Business School Publishing, The Baseline Scenario, MLB Trade Rumors, Entertainment Weekly, and Cracked.

Make of that list what you will.

Serendeputy doesn’t lie. That list is pretty much what I’m paying attention to right now…

What’s up with Serendeputy?

Wednesday, September 17th, 2008

As most of you know, I left a few months ago to work on a different project, called Serendeputy. I’m heads-down writing the software for the initial release (in case you’ve been wondering why it’s been a little quiet in these parts). I hope to have something useful to show by the end of the year. I’ll post updates here as I make progress.

I’ve also been writing some longer pieces over at the Serendeputy blog, which you might not have seen. Here are some of the better pieces:

Stack Overflow is pretty impressive
Progress Update — 9/15/08
Fan Mail: Twit Podcasts
Book Review: The Dip, by Seth Godin
SmugMug and Amazon Web Services

I’ll try to link over to that blog every once in a while for stuff that might be of general interest. Or, you can subscribe to the Serendeputy rss feed.

I hope to come up for air soon…