I had the following error whenever I tried to use ‘script/console’ (script/server and rake worked fine)

require ‘mysql’

MissingSourceFile: no such file to load — mysql
from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require’
from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `require’

Turns out I had multiple versions of RubyGems installed on my Macbook. The error message already spoils it by saying rubygems in /System/Library/Framework/ is being called, while I actually installed rubygems manually in a different folder.

To prevent these kind of problems in the future I simply deleted the entire Ruby.framework directory and use the manually installed versions instead.

I’ve been using my new Macbook for a whopping two days now, so I think it’s time to share some of my early impressions.

First of all, I’m not a typical computer user. I’m picky, opinionated and totally unreasonable when it comes to my requirements for an Operating System. Before my Macbook my setup consisted of:

• Desktop PC, Windows XP: Used for gaming mainly, also editting photo’s and as a ‘fat client‘ for Putty
• Laptop, Ubuntu: Used for work and midnight-coding. Purely a development tool for developing Ruby on Rails webapplications, accessing icanhazcheeseburger.com and other important work stuff.

The Windows XP of my desktop PC is a necessary evil. Games for Linux and Mac OS X are rare and my girlfriend still has an emergency Windows-account she can use whenever her Windows XP laptop refuses to work properly. Simply put: I’m stuck with XP for my desktop and I don’t really mind (a lot).

The laptop running Ubuntu belongs to my employer. Ubuntu is an excellent platform to develop Ruby on Rails applications, the huge Ubuntu repository contains recent version of all necessary tools and the apt package manager is simply unmatched on any platform. To isolate myself from outside influences during coding sessions I use a closed headphone and Amarok (a rather bloated KDE program, so I can’t bitch about iTunes without being an ever bigger hypocrit). Pidgin provides me with access to nearly all IM networks, while TwitterFox keeps me in touch with fellow twitterati.

My new Macbook is supposed to replace the Ubuntu laptop and serve as an alternative to my desktop at home. Here are some of my initial thoughts regarding this migration:

It just works

It doesn’t.

Ok to be fair, a lot of stuff just ‘works’, I am however spoilt by Ubuntu 8.10 which also ‘just works’, it just fails to work on different points compared to Mac OS X. OS X fails to ‘just work’ as a Ruby on Rails development environment, sure it comes preloaded with Ruby on Rails, but it has ancient version of Rails, Ruby and Rake and this completely screwed me over while trying to convert my fresh Macbook into the ultimate web-development machine.

Multimedia support is not as good out of the box as Ubuntu. In Ubuntu all the codecs you need are in the repository and are a easy to install. To be fair, this is only since the 8.x branch of Ubuntu. For OS X I had to find a bunch of codecs from questionable vendors (I’ll never trust the DivX corporation after their bloated Windows installers) before I could get my movies and series to play in Quicktime.
Speaking of horrible failures, I don’t see how anyone could enjoy Quicktime. It’s completely inadequate compared to Zoom Player on Windows or MPlayer on Linux. I’m planning to replace it by something decent ASAP. Preferbly something pretty like XBMC or the OS X specific port of this brilliant project.

Package management? What’s that?

Mac OS X understands package management just like this girl understands “PC’s”. Not a clue. Everything application you install has it’s own auto-update function and the built-in Software Update is about as useful as “Microsoft Update”. Which is not bad, it’s just not as great as Ubuntu’s software updater.

MacPorts < apt

MacPorts is similar to Debian/Ubuntu’s APT. It provides an easy way to install and update software for your Mac. Unfortunately it’s more like Gentoo’s ‘emerge‘ and less like Debian’s ‘apt-get’. Everything you install has to be compiled first, and if you want to install something with fancy graphics you’ll have to wait for an hour while the X window manager gets compiled (ok, this is only the first time).
Fortunately it does what it’s supposed to, it just takes quite a lot longer than APT and the repository of available software is tiny compared to Ubuntu’s insane heap of FOSS. For example, something like ‘htop‘ is not available on OS X, but luckily there’s the Activity Monitor GUI to offer some compensation.

OS X < GNU

Mac OS X implements all the usual Unix tools. Things like find and grep. These are kinda different to the GNU-tools I used on my Ubuntu laptop and Debian servers. As far as I can tell after these few days of use, the Mac OS X versions are inferior to their GNU-siblings. I’ll have to get used to this or find alternative tools

So much for the bitching, here are a few things I do like:

It just works

A lot of stuff works just fine out of the box. The prepackaged software gives Ubuntu a good run for it’s money. Also, it’s obvious OS X is more popular than Linux. Quite a few software products offer a reasonable up-to-date version for Mac. Skype for example has a Mac version which is nearly identical to the Windows version, while the Linux version is lacking a lot.

Basic hardware compatibility is not an issue

If you’re the only one making hardware for your OS (or is it the other way around?) it’s easy to make that hardware supported in the OS. While the linux kernel provides support for nearly every graphics card, sound card, IDE controller and toaster made in the history of electronics, OS X only has to support the very limited set of hardware used by Apple. And to Apple’s credit, it does this very well. Unlike Linux they don’t have to work around crappy vendors breaking standards that crash your laptop after a resume-from-standby. As a result, stuff like suspend-to-ram and connecting multiple monitors works really well and quickly.

It’s a lot like Linux

Overall I feel quite at home in OS X already. I was able to set up a proper development environment during a single evening, inspite of the great efforts of MacPorts to keep compiling very large bits of software. During the coming weeks I expect to find a lot more things that are different/worse/better in OS X compared to Ubuntu but the past few days have made me optimistic about the chances of OS X being the only OS on this Macbook. I originally intended to go for a triple boot setup; Windows XP, Ubuntu and OS X, but I think I’ll settle for installing Windows XP in a VirtualBox. Not because I like XP that much, but for testing web-application in the worst browser ever made and it’s younger brothers.

That’s it for now, I’ll probably write a new post in a couple of weeks, because I’m sure OS X has plenty of annoyances and amazements left in store for me.

The contestants

mod_rails (Apache 2.2.8)

Thin 0.8.0 (Cluster of 4 behind nginx 0.5.33, both TCP and Unix socket)

Ebb 0.1.0 (Cluster of 4 behind nginx 0.5.33)

Mongrel 1.1.4 (Cluster of 4 behind nginx 0.5.33)

The benchmark is the index page of Leef je doel, the hardware used is a Dell D830 laptop with a T7300 (speedstep disabled), 2GB of RAM and a 7200RPM harddisk. The page runs a few queries and also serves a lot of fragments from memcached.

The results

Performance

Like my previous test, Ebb comes out on top, closely followed by Thin.

Using the default Apache2 config supplied by Ubuntu, mod_rails shows performance equal to mongrel_cluster behind nginx. I’m sure that by tweaking the Apache config files, some performance can be gained.

Compared to setting up nginx and thin/ebb/mongrel, setting up mod_rails is a lot simpler. Have a look at their screencast to see how easy it is, if you have a working Apache2, you’ll have mod_rails up and running within minutes. There are some small issues with mod_rails at the moment, for a list, check mod_rails’ Google Code page.

During development we already prepared caching for all pages, mostly fragment caching. To expire cache that’s no longer current, we use sweepers that get called when something relevant is updated. These sweepers only sweep the caches that get outdated.

Because we were unfamiliar with caching and needed to see the result of our fragment caching, we used the file_store to store the generated caches. These files are stored on disk and this way you can easily see how many cache gets generated and what they contain.

Regex and file_store == FAIL

To sweep caches we used regular expressions, this way we could easily sweep all relevant caches at once. This was a bad idea, as you can read here and here

During the beta phase the size of the site steadily increased, more users, more goals, more groups. There was a noticeable delay whenever you updated/created something. A short investigation pointed to the cache sweepers as the culprit.

The file_store for cache isn’t exactly the fastest solution to store your cache, but when you combine it with regex sweepers, things really slow down. Whenever you do a regex sweep, all files in the cache directory get returned (not that surprising if you think of it), and the regex is run against it. So even if you do a sweep on /goals, it will also return files in /users and /groups. As a result, updating your profile could take 15 seconds.

MemCached

We’d been planning on moving the cache to memcached all along, so this seemed a good opportunity to do it. In the next few paragraphs I’ll describe how to install memcached, get the correct Rails plugin to make memcached play nice with fragment caching and how to configure your Rails application so it uses your memcache server.

Installation of MemCached on GNU/Linux

First of all, you’ll need to memcache daemon, assuming you have a nice linux distro you can:

sudo apt-get install memcached

This will work on Debian, Ubuntu and other Debian-based distro’s, on Gentoo you can

emerge memcached

The great thing about memcached is its simplicity, it requires no configuration after installation, just run it.

All about the gems baby

Now we’ll get the gem to allow Ruby to talk to memcache. There are two gems that do this, Ruby-MemCache and memcache-client. memcache-client is supposed to be faster, so I used that.

sudo gem install memcache-client

Plugin to play nice with rails

Rails’ fragment caching doesn’t work with memcached out of the box, you’ll need a plugin. This plugin also adds a nice bonus to the cache method in views.

script/plugin install svn://rubyforge.org/var/svn/zventstools/projects/extended_fragment_cache

Now we need to configure your Rails app to use the memcached server. You’ll need to edit your config/environments/production.rb

memcache_options = {
:c_threshold = 10_00,
:compression = true,
:debug = false,
:namespace = 'yourappname_or_anything_you_like',
:readonly = false,
:urlencode = false
}
CACHE = MemCache.new(memcache_options)
CACHE.servers = '127.0.0.1:11211'
config.action_controller.fragment_cache_store = CACHE, {}

That’s all folks!

That’s it, you’re all done, the ‘cache’ method in your views will now use the memcache server.

Oh wait, there’s an encore

But there’s more, using memcached you can set expiry times for caches. I edited the plugin for a default expiry time of 1 day. In vendor/plugins/extended_fragment_cache/lib/extended_fragment_cache.rb look for def write(key,content,options=nil)

def write(key,content,options=nil)
  expiry = options && options[:expire] || 1.day
  begin
    set(key,content,expiry)
    rescue ActiveRecord::Base.logger.error("MemCache Error: #{$!}")
    rescue MemCache::MemCacheError = err
    ActiveRecord::Base.logger.error("MemCache Error: #{$!}")
  end
end

You can change the 1.day to anything you want. To override this default behaviour, you can use the following code in your views

cache('goals/large_cloud', {:expire = 30.minutes.to_i})  do

This will make the cache called ‘goals/large_cloud’ to expire 30 minutes after it got created.

There are two important things to consider when you move to memcached

1. MemCached doesn’t support regex based expiry of caches. You need to manually enter every cache you want to expire. You can do this in some nice methods of course. Here’s ours for expiring the cache when a user gets updated.

def expire_user_fragments(user)
  fragments = %w[author_icon author_link side_block_friends ..snip...]
  fragments.each do |f|
    expire_fragment("user/#{user.id}/#{f}")
  end
end

2. Your application will fail when the MemCache server becomes unavailable. If you ever restart MemCache, or if it crashes (haven’t seen that happen yet), you need to restart your mongrel-cluster/thin/ebb.

3. When you restart MemCache, all cache is cleared, and you need to restart your mongrel-cluster/thin/ebb.

This guide only talks about fragment caching, over at Ben Curtis’ blog, you can read all about action caching.

Now what is Ebb?

The design is similar to the Evented Mongrel web server; except instead of using EventMachine (a ruby binding to libevent), the Ebb web server is written in C and uses the libev event loop library.

Connections are processed as follows:

1. libev loops and waits for incoming connections.
2. When Ebb receives a connection, it passes the request into the mongrel state machine which securely parses the headers.
3. When the request is complete, Ebb passes the information to a user supplied callback.
4. The Ruby binding supplying this callback transforms the request into a Rack compatible env hash and passes it on a Rack adapter.

The graph describes performance of a simple rack application and compares Ebb to mongrel, evented mongrel and thin. I’m more interested in performance with a Rails application and decided to do a benchmark for that.

In my benchmark I used the same application I used for my previous benchmark, only this time I benchmarked some extra pages.

Page 1 is a heavily cached page with few dynamic elements

Page 2 is a less cached page with a bit more dynamic elements

Page 3 is a non-cached page with an N+1 performance issue.

Ebb was tested using version 0.0.3, while Thin was on version 0.7.0. Both were run in a cluster of 4 behing nginx as a load balancer.

Interestingly Ebb managed to outperform Thin by about 10% on every page.

First, here’s a picture of the page that was used.

The page has a tagcloud, site statistics and a conditional menu in the upper right corner.

For the benchmark I’m using the following tools

ApacheBench 2.0.40. To fire off a large amount of requests into the webserver.

Mongrel 1.1.3. One of the Ruby web servers

Thin 0.6.4. An alternative to Mongrel.

nginx 0.5.26. A lightweight HTTP server, which will act as a proxy/balancer in front of the Ruby application servers

For the benchmarks I settled on the following scenario’s.

Single instance of Mongrel in development mode

Single instance of Mongrel in production mode

Cluster of 4 Mongrel instances behind nginx

Cluster of 4 Thin instances behind nginx

In a real production environment, you’d never see the single Mongrel instance, and certainly not in development mode. The graph will show you why.

The red bar is for 50 concurrent requests, with a total of 10000. The red blue is for 10 concurrent requests, with a total of 10000.

The raw numbers:

I’m pleased with the results, we still have a lot of stuff we could cache and database queries that could be optimized, so there’s room for improvement. The server that will eventually host the application runs an Apache proxy going to mongrel_cluster. I’m probably going to run some benchmarks later, to see how Apache proxy holds up against nginx.

My friend Iain was busy making syntax highlighting in VIm for rSpec’s plain text user stories. I thought it would be nice to modify the Google Syntax Highlighting plugin for WordPress, so you can have nicely highlighted stories on your blog.

First install the WordPress plugin and make sure it works. After, download my modified version of this plugin.

Whenever you want to post a plain-text user story, use the pre tag with ‘code’ as name and ‘story’ as class attribute to indicate it’s a user story.

This is how it looks:

Story: Arrangement

As a user
I want to invite people
So I have more chance of winning an arrangement

Scenario: Browsing the homepage
Given a clean database
When the user goes to /
Then the page should contain the text 'alledaagse'

Scenario: Choosing an arrangement
When the user chooses the godinfrankrijk arrangement
Then the page should contain the text 'God in Frankrijk'
And the page should contain the text 'voor te stellen'

Scenario: Inviting a person
Given a clean session
When the user chooses the ster arrangement
Then the page should contain the text 'leven als een ster'
When the user types 'His Name' into the invitee_name field
And the user types 'Street 13' into the invitee_address field
And the user types '1234AB' into the invitee_zip field
And the user types 'Rotterdam' into the invitee_city field
And the user types '0101234567' into the invitee_phone field
And the user clicks on the radiobutton labelled vrouw
And the user clicks the submit button
Then the page should contain the text 'mijn gegevens'

I’m very pleased with the end result, and I hope the customer will be as well.

We’ll find out next week

On my old laptop I tried using them before, but the poor thing couldn’t handle it.

Now with my new laptop I decided to give it another go.

Using compiz-fusion you can put these screenlets on a special ‘widget layer’, which you can summon by pressing a keyboard key. Whenever you need some information or need to make a small note, you press that keyboard key and the widget layer is displayed. Pressing the same key again and the widgets disappear.

There’s a nice screenlet that displays some sensor information, CPU and HDD temperatures. However, my laptop has another sensor on the graphics card that shows the GPU temperature. I just had to have this added to the screenlet.

I made a modified version of the Temperature2 script to generate a screenlet for nvidia GPU temperature. You’ll need the proprietary nvidia drivers installed and access to the ‘nvidia-settings’ utility. Download the file and place it in the ~/.screenlets/Temperature2/ directory, overwriting the old file.

Update: My modifications are now included into version 0.4 of Temperature2. You can get it at screenlets.org

I was browsing through this excellent presentation given by Dave Astels and David Chelimsky at RubyConf2007. Turns out that David Chelimsky and Aslak Hellesøy are working on a book about Behaviour Driven Development using RSpec.

The presentation is an excellent introduction for anyone thinking about moving to RSpec. It tells you a bit about the (rather short) history of the framework and the differences between BDD and TDD.

It also shows off the awesome plain-text stories I posted about earlier.