ruby: April 2008 Archives

I've been working on Kebima for several months now, using Firefox and Linux/OSX.  Chalk it up to not doing enough research, but I just figured transparent PNGs worked in IE.  Oh well.  They don't.  At least not in 6 and earlier.  So began my mission to get them to work.

I was already using a lightbox package that uses the technique mentioned in the MS support article, but I didn't want to have to apply a div-specific solution for every png on my page... and since i'm using the silk icon set, this would mean a lot a lot of specificity.

Googling gets you a lot of results.  The first hit is actually pretty good, in that it states that the script isn't maintained, but points you to 24 Ways, which has a pretty good solution.  This likely works out of the box for normal web development (I did run into one bug... for some reason the section of the script that sets root on line 17 failed... I took out the bit that allowed you to limit the div the script was applied to).  However rails likes to throw timestamps on the end of images, so instead of '/images/icon.png' you get '/images/icon.png?1209327623'

Because the 24 Ways script is looking for an img tag whose src attribute ends with '.png', this makes things not work.  My solution was to add a regex to match rails style image srcs

    var png_pattern        = /\.png(\?\d*)?$/i;

And then match against that in fnLoadPngs

            // background pngs
            if (obj.currentStyle.backgroundImage.match(png_pattern) !== null) {
                bg_fnFixPng(obj);
            }
            // image elements
            if (obj.tagName=='IMG' && obj.src.match(png_pattern) !== null){
                el_fnFixPng(obj);
            }

And it worked.  Plus I got a pretty decent workout running up and down the stairs between Mac and PC.

So I switched our Twitter integration for Kebima from using the HTTP interface to using the XMPP interface.  We really wanted real time updates, and polling just seems so barbaric.  I found some code on how to create a twitter bot and got the conversion made surprisingly fast.  It's still ugly because auto-following has to be done through HTTP, but in a few hours I had a pretty simple bot going.

Then we went to the Web 2.0 Expo and tried it out.  And it didn't work.  Turns out Twitter's Jabber replies were delayed or somesuch... probably fallout of Twitter's other greyout problems this week.  But it got me thinking...

If you had to build twitter from scratch, how thin a veneer over XMPP could you do it with?  At its heart, Twitter is a message router.  There are interfaces with SMS systems, HTTP, and Jabber.  But messages come in, messages go out.  I've heard they run XMPP under the hood to handle this, but I've also heard they run rails for a good chunk of functionality.  I'm a rails fan as well, but from what I can see, perhaps there's too much rails and not enough ejabberd in the mix.

Conceptually it seems like you could set this up as a set of processes that each act as an internal Jabber client for an individual twitter user.  The process is responsible for receiving messages

1. pushing them to SMS or the user's Jabber client
2. Building the web page that people visit
3. Handling API calls as they come in
4. Dispatching messages to followers
   

In fact conceptually you could set up three clients, each to handle one of these jobs.  Each of these processes could in essence be a fully functional twitter service for an individual user.

In addition to this you need gateways for SMS and the HTTP API, but it seems like those could be scaled out fairly easily as they're not user specific.  The SMS gateway is just going to build an XMPP message and dispatch it to the matching twitter client.  The HTTP gateway is doing the same thing.

The beauty of this is that it's naturally sharded.  No shared data between users.  You have to parse messages for @responses and route them to appropriate destinations, but it seems like that could easily be written as an ejabberd plugin. 

Google has proven this scales to a fairly large user base.  Is Twitter already beyond that scale, or am I missing something?

Regardless, we'll be pushing ahead with our plans to add direct GTalk integration for our app.  It may go down at times as well, but it seems more robust than Twitter's infrastructure at this point.