Aug 10

Easy Twitter Streaming API Testing with Mockingbird

My company makes two applications that focus on Twitter: CheapTweet and TweetReach. Lately, we’ve started transitioning parts of these apps to the Twitter Streaming API. To do that, we’ve been developing the flamingo project to encapsulate all the best practices of working with Twitter streams so that others won’t have to solve all the same problems. I’m not going to talk about flamingo today, though (I’ll do that soon). I’m going to talk about testing.

Testing code that talks to the Twitter Streaming API is hard. The are 8 documented failure HTTP response codes, random disconnections, and second-to-second rate changes. And, since this is Twitter we’re talking about, basically anything can happen at any time – including complete downtime.

So, let me introduce you to mockingbird. Mockingbird is a simple streaming HTTP server written in Ruby that you can use in place of the server at stream.twitter.com when you’re testing your Streaming API code. With mockingbird, all it takes is a few lines of code and you can simulate all the issues I described above and more. Here’s an example config:

Mockingbird.setup(:port=>8080) do
  wait 1
  5.times do
    send '{"foo":"bar"}'
    wait { rand }

So what does this do?

  1. Starts a mockingbird server on port 8080 on your localhost
  2. When there’s a connection, it starts by waiting for 1 second before sending any data
  3. Then, it sends ‘{“foo”:”bar”}’ 5 times with each time followed by randoom wait of between 0 and 1 seconds
  4. After that, it closes the connection and waits for more

You can do lots of other things with this simple configuration style. See the README and code examples at github.

Mockingbird provides a simple setup/teardown mechanism so it’s easy to start a server and shut it down during your unit tests, specs, or whatever. Here’s an example Test::Unit test:

class TestConnection < Test::Unit::TestCase
  def setup
    # Starts the server running a separate process
    Mockingbird.setup(:port=>8080) do
      status 500, "Error"

  def test_handle_500_errors
    # Your test code here for connecting to localhost:8080
    # and handling a 500 error here

  def teardown
    # Stops the server process

We’ve already found mockingbird to be extremely useful for testing flamingo. If you’re using the Streaming API and need to be confident your code can withstand the crazy stuff Twitter might throw at it, I think you’ll probably find it helpful, too. Of course if you have questions, comments, ideas or (even better) pull requests, hit me up.

Mar 09

Austin on Rails Talk: Building TweetReach with Sinatra, Tokyo Cabinet and Grackle

I gave a talk last night (3/24/2009) at Austin on Rails about building a new (yet-to-be-released) application called TweetReach using tools that are somewhat off the beaten path. These included Sinatra, Tokyo Cabinet and my new Twitter API library: Grackle (which I’ll talk about in more detail in a future blog post).

I referenced some code during the “Building TweetReach” part of the presentation. If you’re interested, you can download it.

By the way, I was the second speaker of the night. Mike Perham spoke before me about caching with Rails. Check his stuff out as well. Thanks to everyone who came out.

Update 4/2/2009: Launched TweetReach! Go check it out.

May 08

Adventures in Macro Blogging

I’ve been using Twitter seriously for several months now and I really enjoy the rapid information exchange of the micro blogging platform. I’ve never been able to sustain a “real” blog for very long but I’ve stuck with Twitter. That 140 character limit is a far lower barrier to entry than the blank HTML text area of WordPress admin which seems to call for so much tedious “organization” and “coherence”. All that said, I’m going to give macro blogging another try as I do occasionally wish to express myself in more than the length of a text message. Here’s hoping for relatively frequent, somewhat coherent posts longer than 140 characters.