April2
Like any web 2.0 site, leefjedoel.nl is currently in beta. During this phase we’re trying to find bottlenecks, fix the last few bugs and optimize heavy parts of the site.
During development we already prepared caching for all pages, mostly fragment caching. To expire cache that’s no longer current, we use sweepers that get called when something relevant is updated. These sweepers only sweep the caches that get outdated.
Because we were unfamiliar with caching and needed to see the result of our fragment caching, we used the file_store to store the generated caches. These files are stored on disk and this way you can easily see how many cache gets generated and what they contain.
Regex and file_store == FAIL
To sweep caches we used regular expressions, this way we could easily sweep all relevant caches at once. This was a bad idea, as you can read here and here
During the beta phase the size of the site steadily increased, more users, more goals, more groups. There was a noticeable delay whenever you updated/created something. A short investigation pointed to the cache sweepers as the culprit.
The file_store for cache isn’t exactly the fastest solution to store your cache, but when you combine it with regex sweepers, things really slow down. Whenever you do a regex sweep, all files in the cache directory get returned (not that surprising if you think of it), and the regex is run against it. So even if you do a sweep on /goals, it will also return files in /users and /groups. As a result, updating your profile could take 15 seconds.
MemCached
We’d been planning on moving the cache to memcached all along, so this seemed a good opportunity to do it. In the next few paragraphs I’ll describe how to install memcached, get the correct Rails plugin to make memcached play nice with fragment caching and how to configure your Rails application so it uses your memcache server.
Installation of MemCached on GNU/Linux
First of all, you’ll need to memcache daemon, assuming you have a nice linux distro you can:
sudo apt-get install memcached
This will work on Debian, Ubuntu and other Debian-based distro’s, on Gentoo you can
emerge memcached
The great thing about memcached is its simplicity, it requires no configuration after installation, just run it.
All about the gems baby
Now we’ll get the gem to allow Ruby to talk to memcache. There are two gems that do this, Ruby-MemCache and memcache-client. memcache-client is supposed to be faster, so I used that.
sudo gem install memcache-client
Plugin to play nice with rails
Rails’ fragment caching doesn’t work with memcached out of the box, you’ll need a plugin. This plugin also adds a nice bonus to the cache method in views.
script/plugin install svn://rubyforge.org/var/svn/zventstools/projects/extended_fragment_cache
Environment setup
Now we need to configure your Rails app to use the memcached server. You’ll need to edit your config/environments/production.rb
memcache_options = {
:c_threshold = 10_00,
:compression = true,
:debug = false,
:namespace = 'yourappname_or_anything_you_like',
:readonly = false,
:urlencode = false
}
CACHE = MemCache.new(memcache_options)
CACHE.servers = '127.0.0.1:11211'
config.action_controller.fragment_cache_store = CACHE, {}
That’s all folks!
That’s it, you’re all done, the ‘cache’ method in your views will now use the memcache server.
Oh wait, there’s an encore
But there’s more, using memcached you can set expiry times for caches. I edited the plugin for a default expiry time of 1 day. In vendor/plugins/extended_fragment_cache/lib/extended_fragment_cache.rb look for def write(key,content,options=nil)
def write(key,content,options=nil)
expiry = options && options[:expire] || 1.day
begin
set(key,content,expiry)
rescue ActiveRecord::Base.logger.error("MemCache Error: #{$!}")
rescue MemCache::MemCacheError = err
ActiveRecord::Base.logger.error("MemCache Error: #{$!}")
end
end
You can change the 1.day to anything you want. To override this default behaviour, you can use the following code in your views
cache('goals/large_cloud', {:expire = 30.minutes.to_i}) do
This will make the cache called ‘goals/large_cloud’ to expire 30 minutes after it got created.
There are two important things to consider when you move to memcached
1. MemCached doesn’t support regex based expiry of caches. You need to manually enter every cache you want to expire. You can do this in some nice methods of course. Here’s ours for expiring the cache when a user gets updated.
def expire_user_fragments(user)
fragments = %w[author_icon author_link side_block_friends ..snip...]
fragments.each do |f|
expire_fragment("user/#{user.id}/#{f}")
end
end
2. Your application will fail when the MemCache server becomes unavailable. If you ever restart MemCache, or if it crashes (haven’t seen that happen yet), you need to restart your mongrel-cluster/thin/ebb.
3. When you restart MemCache, all cache is cleared, and you need to restart your mongrel-cluster/thin/ebb.
This guide only talks about fragment caching, over at Ben Curtis’ blog, you can read all about action caching.
