Posts about 'Serendeputy'

Serendeputy 2 Update, July edition

Wednesday, July 16th, 2014

Happy July, everyone!

I’m making progress, but I’ve been mostly heads-down with client work the last few weeks.

The biggest change is that I’m moving from database-driven querying to search-engine-driven querying. I set up the alpha version using Postgres and hstore, and ran some tests to see where it would fall over. It turns out that it falls over at smaller load levels than I’d like.

I’m glad I ran that set of experiments when it was just me and a couple of folks playing with it, instead of in front of the whole world.

So, I’m implementing a version using Amazon CloudSearch for some of the heavy indexing lifting. Depending on how things work out, I should be able to see if this will carry me through by the middle of August. Then, I can fire everything up again and see where the new constraint in the system will live.

It’s also giving me a good chance to go back and refactor a few of the pieces that I didn’t get right the first time. I need to tear everything down anyway, I might as well fix the other stuff while I have the tools out.

I’m hoping for beta one at some point between now and eventually. It’s not quite turning into Project Xanadu, but I worry that we’re trending that way.

Thrill-seekers prepare

Thursday, May 8th, 2014

Quick update: the alpha is close to the point where it will be available for thrill-seekers to test drive. Please ping me if you want an invite.

Progress continues.

Serendeputy 2 update for March

Tuesday, March 11th, 2014

I’m making good progress on this (increasingly) large project.

The production environment is sketched and spiked, using Amazon Web Services. I think I’ve solved the SSL issues with a combination of CloudFlare and elastic load balancer.

The crawler is up and running — I’d even say reasonably solid. It’s well-behaved, working with robots.txt files and following all the crawler best practices. Once it’s running on the AWS servers, I’ll get a better idea for how many sites just block traffic from the Amazon IPs on general principles. I have a feeling that I’ll be paying for other people’s sins.

The personalization math is done and the code is prototyped. As I’m using it, I’m tweaking the coefficients for the gestures to balance how much they can influence the profiles.

The front end is not egregiously ugly! No one would ever claim that I’m a designer, but I’m going to have that alpha in front of people to get feedback, and then bring in a real design person/team/agency for the beta version.

March is going to be an insane month in pretty much every corner of my life, but I’m still hoping to have an alpha version that I (and maybe you!) can use in the beginning of April.

Serendeputy 2 update for February

Saturday, February 1st, 2014

I’m still working diligently towards my rock-solid deadline of “Launch this thing at some point in the indeterminate future.”

I finished the gesture engine this week. Now that that’s solid, I’m spending most of my time over the next month working on the profile balancer and its complement, the homepage builder.

I’m looking at these like a chef composing a recipe: how can I balance the flavors just right? I’m working to strike the balance among recency, relevance (how it maps to your interest profile), and reputation (how important the page is on the worldwide Internet).

My AWS infrastructure is set up, and I’m running parts of the application on it right now. Making progress.

I should be able to get into alpha Real. Soon. Now. (I’m shooting for by Opening Day, but we’ll see…)

Thanks for staying tuned.

Serendeputy 2 notes

Thursday, September 5th, 2013

The children go back to school tomorrow, and the transition from Dad-mode to launch-mode begins.

After I took a week or two off to adjust to an abrupt transition, this has been an insanely productive summer. (Obviously not the “writing the blog” front, though.) I’m in the middle of rewriting Serendeputy, my personal news application, from the ground up, and I’m pretty excited by the direction it’s taken. A lot has changed since I wrote the first version in 2008-2009, and it’s been so much fun to be able to update it to match the world we’re in now.

It’s currently in private “pre-alpha” i.e., running on my laptop. I should have a thrill-seeker version available for early testing by the end of September. If you’re interested in playing with it, please drop me a line.

Tech notes for those who are interested in that sort of stuff:

  • It’s still a Rails (4.0) front end, but that’s almost entirely a thin wrapper over a series of services.
  • I’m playing with Clojure for the personalization engine. I’ve written the original logic in Ruby to make sure it works; I’m going to see if I can port it over to Clojure to get a performance and memory boost.
  • I’m pretty much all in on AWS. Serendeputy currently runs on a few Rackspace (formerly Slicehost) servers; the new version is entirely AWS-based.
  • I’m about to fire up RubyMotion to write the iOS client. I’d really rather have a native client, and I’m hoping that my using a Ruby-based environment can make that happen. If it works, then I should be able to contract out an Android version. If not, then I’ll fall back to the mobile-optimized site through the browser.

Your personal API

Tuesday, July 31st, 2012

This is geeky, but supremely interesting if you’re into that sort of thing: the Cambrian explosion of everything.

I’ve been doing a lot of thinking over the past few months about personal APIs, personal deputies and Freenon’s forkable nuggets of knowledge. Within a few years (maybe sooner), everything will have an API, and most of your personal interactions with the world will be API-mediated. I’m thinking through how Serendeputy fits into this model. Right now, it’s your personal intermediary for news, and your profile is its API. How does this extend beyond this one use case?

Mostly, I want to see what I can do to make sure that this future bends towards open, with individuals controlling their data and their life. (See the previous post on dystopian futures…)

Anyway, this is the type of stuff I’m thinking about. I want this to exist; now it’s just the small matter of implementation, distribution and paying the bills in the meantime :-)

How to get cron to use the rbenv ruby

Friday, June 1st, 2012

With any luck, this will save people my hours of aggravation.

My problem:

I’m using .rbenv to manage the Ruby processes. When I run everything manually and through the tests, it all works great. Unfortunately, when I put it into cron, it fails.

My solution:

Step One: Find the rbenv binary:

jason:~$ which ruby
/home/jason/.rbenv/shims/ruby

Step Two: Update the cronfile:

In my cronfile, I changed from:

* * * * * cd /home/jason/program-dir ; ruby program.rb

to

* * * * * cd /home/jason/program-dir ; /home/jason/.rbenv/versions/1.9.3-p194/bin/ruby program.rb

Voila.

There’s probably a more Linux-fu method, but this worked. If you have a more elegant solution, please drop me a line so that I can update this post.

Merry Christmas!

Thursday, December 24th, 2009

Merry Christmas, everybody!

I’ve been a poor correspondent. Sadly, that’s unlikely to change at any time in the next couple of months. I have a major new release of Serendeputy in its final stages, and I’m also working a separate super-secret project. Lots to do. Blogging suffers.

But, I should be able to have some pictures up this weekend. I have a feeling that Sadie and Lucy are going to be happy with what Santa is going to bring them.

In the meantime, here are some sites I’ve recently added to the Serendeputy catalog which are really cool. It’s worth checking them out.

Fashion Served: I’ve been building out the fashion tree, and I found this site. It’s a showcase for some of the best fashion photography around.

(And yes, I understand the irony of someone who wears shorts and a t-shirt in his home office all day building out the fashion tree.)

Roger Ebert: Roger Ebert can’t talk anymore, but he sure can write well.

Wend Magazine: I’ve also been building out the green tree — we’ve become much more involved with green this year — and I found this site. All sorts of interesting links, articles and stories.

Clients from Hell: Freelance and design clients say the darndest things.

Special bonus, here are my top 25 Most Popular Sources at the moment: Fail Blog, Sadly No, Balloon Juice, Fashion Served, Techdirt, New York Times, Work it Mom, Consumerist, Roger Ebert, TechCrunch, Business Insider, Boston Globe, Wend Magazine, Steve Blank, Holliston Reporter, The Atlantic, Clients from Hell, ESPN Boston, Tracker Editor’s Blog, SEOMoz, Harvard Business School Publishing, The Baseline Scenario, MLB Trade Rumors, Entertainment Weekly, and Cracked.

Make of that list what you will.

Serendeputy doesn’t lie. That list is pretty much what I’m paying attention to right now…

What’s up with Serendeputy?

Wednesday, September 17th, 2008

As most of you know, I left Boston.com a few months ago to work on a different project, called Serendeputy. I’m heads-down writing the software for the initial release (in case you’ve been wondering why it’s been a little quiet in these parts). I hope to have something useful to show by the end of the year. I’ll post updates here as I make progress.

I’ve also been writing some longer pieces over at the Serendeputy blog, which you might not have seen. Here are some of the better pieces:

Stack Overflow is pretty impressive
Progress Update — 9/15/08
Fan Mail: Twit Podcasts
Book Review: The Dip, by Seth Godin
SmugMug and Amazon Web Services

I’ll try to link over to that blog every once in a while for stuff that might be of general interest. Or, you can subscribe to the Serendeputy rss feed.

I hope to come up for air soon…