28
Mar 10

Using MongoDB and a Relational Database Together

Below is the presentation I gave to the Austin MongoDB Day on 3/27/2010. I talked about the design decisions one needs to make when determining whether to introduce a document database like MongoDB into an existing application built on a relational database. I give a few examples of how we’ve made MongoDB and our MySQL database live together in harmony inside CheapTweet.

Thanks to Lynn Bender of GeekAustin, CoSpace, and the folks at 10Gen for putting on a great event.


29
Jul 09

Introduction to Sphinx and Thinking Sphinx: 10 Minute Talk at Austin on Rails

I did a quick talk at Austin on Rails last night about my experience using Sphinx and Thinking Sphinx for doing full-text search on CheapTweet. We did a series of short talks last night so this is only a 10 minute overview. Hope some of you will find it helpful.


25
Mar 09

Austin on Rails Talk: Building TweetReach with Sinatra, Tokyo Cabinet and Grackle

I gave a talk last night (3/24/2009) at Austin on Rails about building a new (yet-to-be-released) application called TweetReach using tools that are somewhat off the beaten path. These included Sinatra, Tokyo Cabinet and my new Twitter API library: Grackle (which I’ll talk about in more detail in a future blog post).

I referenced some code during the “Building TweetReach” part of the presentation. If you’re interested, you can download it.

By the way, I was the second speaker of the night. Mike Perham spoke before me about caching with Rails. Check his stuff out as well. Thanks to everyone who came out.

Update 4/2/2009: Launched TweetReach! Go check it out.


07
Sep 08

A recent Rubyist goes to the Lone Star Ruby Conference

I attended the Lone Star Ruby Conference (LSRC), held here in Austin over the past two days. As this was my first conference on the topic, I thought I would share some observations. First, a vigorous caveat emptor… I’m a former Java guy still relatively new to the Ruby world. At the very least I hope to provide the humble perspective of a transplant from the crowded streets of Javaopolis to the rolling hills of Ruby country. So here goes…

Rubyists aren’t afraid of change (and some churn)

I’ve long been an interested outsider to Ruby and Rails – following along on the fringes for a couple years now by reading articles, running irb occasionally and generating some scaffolds. At that distance, you basically see Ruby and Rails. What you don’t see is the constant churn of open source projects, plugins, development tools, GUI toolkits, ideologies and best practices. Gregg Pollack and Jason Seifer’s talk on innovations in the last year and Evan Phoenix‘s keynote about Ruby memes made the currents of change pretty clear. Java experienced a lot of change during my 8 years but it definitely seemed to be at a more staid pace. There were a few bigger, more corporate, entities driving it (Sun, Apache, IBM) and fewer mythologized personalities like why. In the Ruby community, a lot of energy seems to be expended reinventing various wheels (as Evan pointed out with his list of ARGV processors). There’s a fine distinction between a healthy competition of ideas and a naval-gazing, ego-stroking churn that wastes time. That said, I would never stand in the way of the free market of ideas. So, by all means, (with no apologies to Mao) let 1000 ARGV processors bloom and let’s see which stick.

There are some surprising technology holes – but they’re being filled

As a former Java programmer who worked deeply with concurrency, Ruby’s after-thought of a concurrency model frightened me a bit. Java, though described by Dave Thomas on the last night’s panel as “a blunt tool”, has a basement filled with extremely sharp swords known as threads and a well-defined concurrency model. But, you say, Ruby has Threads. Yes, it does, but it’s a shadow of what’s available in Java. There, you can wield the sharp tools of concurrency with great effect on difficult problems but you can certainly stab yourself without understanding memory consistency effects, atomicity concerns and everything else.

Until recently, it seems the Rubyist approach to most of these issues has been to ignore them by slapping in a Big Fat Mutex. As someone who’s dealt with connection thread pools for RDBMs access for a long time, I was surprised to learn how innovative the non-blocking MySQL connector being worked on by espace as part of NeverBlock was considered. Fibers, mentioned by both Matz and Bruce Williams in discussing Ruby 1.9, offer a light-weight cooperative approach to something resembling concurrency. It was also heartening to hear Matz talk about how scalability is a big concern for the future of Ruby. Of course, no mention of these issues in Ruby is complete without noting that Ruby (and Rails) has been able to achieve a great deal in this area by using processes. This is usually a pretty simple and straightforward approach but it’s fairly large-grained and could certainly stand to be augmented with more sophisticated fine-grained capabilities that live inside the Interpreter.

Joy matters

Despite the above grumbling about concurrency, Ruby as a language is a beautiful thing. It really is damn pleasant to write Ruby code. I think this is a direct extension of Matz’s personality and concerns. In his keynote, he spoke powerfully on the need to see a prgramming language as part of the human interface to computers and the need to make that interface as joyful as possible to use. I don’t think I’ve once read “Joy” as a chief requirement in a JSR submitted to the Java Community Process for inclusion into Sun’s Java spec. That alone is probably enough reason to switch to Ruby. I do wonder how Matz’s vision of a humane language will hold up under the predicted onslaught of mindless Java-drones such as me that will create the 3 million new Ruby programmers expected over the next few years. As for that…

Don’t fear the hordes

We’re not all mindless and we’re not all drones. The unspoken sentiment when talking about the future growth of Ruby was “things are ok now because we’re all smart and dedicated craftsmen, unlike people who are using Java but will eventually invade beautiful ruby-land and make it ‘enterprisey'”. Growth is stressful for any group, especially one that gleefully defines itself as a minority in opposition to the mainstream as I believe Rubyists do. However, I think it benefits no one to assume that new Rubyists who may come to it later than you did are any less smart, dedicated or concerned about their work than you are. A community that doesn’t grow is bound to become introverted and ultimately stagnant. It would be a great loss if the Ruby community did that. The good news is I don’t think it will. Personally, I’ve felt welcomed by the Austin Rails community and I’ve also found plenty of resources elsewhere to help me learn. The best thing thing for Ruby’s future is to continue to embrace the hordes and show them what they’ve been missing.

And, In conclusion…

There are an incredible number of smart people that truly seem to love Ruby and want to see it succeed. Though there are ideologies, squabbles and disagreements, the part of the community assembled at LSRC was as close to a meritocracy as I’ve personally seen. The power in Ruby seems to lie with those that have most demonstrated their technical abilities and not with those that have the right corporate affiliations. This focus on merit leads to the sometimes duplicative and anarchic approach to developing new technology as people compete with their ideas. Ultimately, though, this is probably Ruby’s greatest strength and will serve it well in the future. I’m looking forward to being a part of it.

By the way, thanks to the Lone Star Ruby Foundation for putting on the conference. Great job. See you all at LSRC 2009.


25
May 08

Surprisingly basic Rails performance tips – and the people that don’t love them

Antonio Cangiano offers up 10 Ruby on Rails Performance tips, some of which are really just good practice in any web application and aren’t specific to Rails. This includes gems like:

don’t be afraid of using the cool features provided by your database, even if they are not directly supported by Rails and doing so means bypassing ActiveRecord. For example define stored procedures and functions, knowing that you can use them by communicating directly with the database through driver calls, rather than ActiveRecord high level method.

And:

Retrieve only the information that you need. A lot of execution time can be wasted by running selects for data that is not really needed.

Shocking stuff!

What amazes me is the level of irritation evident in the comments from people decrying this as “premature optimization”. I agree that you shouldn’t completely reorganize your code to achieve some speculative performance increase before you really know what parts of your app are going to have issues. However, some things are just common sense. If I can pull back data from the database without doing O(n) queries in a loop, I should do that. If I need to run a report with lots of aggregated data, I should probably consider computing that in the DB via a function or stored procedure. Bottom line, there are things that are guaranteed to cause you problems. Sitting around and smugly saying, “I don’t want to optimize prematurely here” is no excuse for writing dumb code.

(Link to the performance tips via the FiveRuns Blog)