Luckily there are some nice tricks you can use to make a remote kernel upgrade as safe as possible.

First, make sure you have a valid kernel. Copy the .config of an old kernel and ‘make oldconfig’ to make it work on your new-to-be-compiled kernel.

Second, make the kernel using the ‘make-kpkg’ package, don’t forget the –initrd parameter if you need it.

Third, after installing the kernel package, don’t forget to check if the initrd image got generated. For some strange reason Debian won’t generate this even after building with make-kpkg –image, you can read more about that here. You basically need to copy a file from ‘/usr/share/kernel-package/examples/etc/kernel/postinst.d/initramfs’ to ‘/etc/kernel/postinst.d/initramfs’

Fourth, you need to configure Grub2 so it uses a kernel boot option which makes the kernel restart if it can’t boot for some reason. To do this with Grub2, open /etc/default/grub and add “panic=5″ to GRUB_CMDLINE_LINUX_DEFAULT, like this:

GRUB_CMDLINE_LINUX_DEFAULT=”quiet panic=5″

Fifth, Grub2 needs to be told to only boot the new kernel once, so in case the new kernel panics, it automatically restarts using a kernel you know works (your current one perhaps). Else the ‘panic=5′ we set earlier will only lead to a reboot loop.

To do this open /etc/default/grub and change GRUB_DEFAULT to saved, like this:

GRUB_DEFAULT=saved

To configure ‘reboot-once-into-new-kernel’, we need to do two more things, first we need to tell Grub2 what your working, safe kernel is. Check /boot/grub/grub.cfg for the name of this kernel, but don’t change anything in this file.

My safe kernel is called: ”Debian GNU/Linux, with Linux 2.6.32-2-686″, which I can set as the default, safe, kernel with this command:

grub-set-default ”Debian GNU/Linux, with Linux 2.6.32-2-686″

Now we have to tell Grub to reboot once into the new kernel, again we need the name of the new kernel, mine is ”Debian GNU/Linux, with Linux 2.6.32-example”, use this command.

grub-reboot ”Debian GNU/Linux, with Linux 2.6.32-example”

Don’t worry, this won’t reboot your machine.

The final step is to generate a new /boot/grub/grub.cfg, else all the things we just did won’t have an effect

update-grub

Now you’re ready to reboot, good luck and fingers crossed. After a succesful boot, you can set your new kernel as the default using the grub-set-default command

grub-set-default ”Debian GNU/Linux, with Linux 2.6.32-example”

Playing back 1080p content however, was still a problem. I was never able to get smooth playback with mplayer. Only the xbmc videoplayer could play 1080p smoothly on my system. XBMC is not an option due to the WAF, it has to be something that runs from MythTV.

Fooling around with coreavc-for-linux (I love CoreAVC on Windows) got me nowhere. Recompiling, tweaking and updating mplayer/nvidia/xorg, nothing seemed to work.

Until I tried a new version of mplayer from the Debian Multimedia repositories. The mplayer version in that repository has support for a new feature in the linux nvidia drivers that (finally) allows video decoding on the GPU.

The easy steps to get this to work:

• Download and install the new nvidia linux drivers (180+)
• Add the Debian Multimedia repository to your sources.list
• Install mplayer from the new Debian Multimedia repository
• Start mplayer with ‘-vo vdpau -vc ffh264vdpau’

More than a year of tweaking and tinkering made redundant by these easy steps, excellent!

I’ve been using my new Macbook for a whopping two days now, so I think it’s time to share some of my early impressions.

First of all, I’m not a typical computer user. I’m picky, opinionated and totally unreasonable when it comes to my requirements for an Operating System. Before my Macbook my setup consisted of:

• Desktop PC, Windows XP: Used for gaming mainly, also editting photo’s and as a ‘fat client‘ for Putty
• Laptop, Ubuntu: Used for work and midnight-coding. Purely a development tool for developing Ruby on Rails webapplications, accessing icanhazcheeseburger.com and other important work stuff.

The Windows XP of my desktop PC is a necessary evil. Games for Linux and Mac OS X are rare and my girlfriend still has an emergency Windows-account she can use whenever her Windows XP laptop refuses to work properly. Simply put: I’m stuck with XP for my desktop and I don’t really mind (a lot).

The laptop running Ubuntu belongs to my employer. Ubuntu is an excellent platform to develop Ruby on Rails applications, the huge Ubuntu repository contains recent version of all necessary tools and the apt package manager is simply unmatched on any platform. To isolate myself from outside influences during coding sessions I use a closed headphone and Amarok (a rather bloated KDE program, so I can’t bitch about iTunes without being an ever bigger hypocrit). Pidgin provides me with access to nearly all IM networks, while TwitterFox keeps me in touch with fellow twitterati.

My new Macbook is supposed to replace the Ubuntu laptop and serve as an alternative to my desktop at home. Here are some of my initial thoughts regarding this migration:

It just works

It doesn’t.

Ok to be fair, a lot of stuff just ‘works’, I am however spoilt by Ubuntu 8.10 which also ‘just works’, it just fails to work on different points compared to Mac OS X. OS X fails to ‘just work’ as a Ruby on Rails development environment, sure it comes preloaded with Ruby on Rails, but it has ancient version of Rails, Ruby and Rake and this completely screwed me over while trying to convert my fresh Macbook into the ultimate web-development machine.

Multimedia support is not as good out of the box as Ubuntu. In Ubuntu all the codecs you need are in the repository and are a easy to install. To be fair, this is only since the 8.x branch of Ubuntu. For OS X I had to find a bunch of codecs from questionable vendors (I’ll never trust the DivX corporation after their bloated Windows installers) before I could get my movies and series to play in Quicktime.
Speaking of horrible failures, I don’t see how anyone could enjoy Quicktime. It’s completely inadequate compared to Zoom Player on Windows or MPlayer on Linux. I’m planning to replace it by something decent ASAP. Preferbly something pretty like XBMC or the OS X specific port of this brilliant project.

Package management? What’s that?

Mac OS X understands package management just like this girl understands “PC’s”. Not a clue. Everything application you install has it’s own auto-update function and the built-in Software Update is about as useful as “Microsoft Update”. Which is not bad, it’s just not as great as Ubuntu’s software updater.

MacPorts < apt

MacPorts is similar to Debian/Ubuntu’s APT. It provides an easy way to install and update software for your Mac. Unfortunately it’s more like Gentoo’s ‘emerge‘ and less like Debian’s ‘apt-get’. Everything you install has to be compiled first, and if you want to install something with fancy graphics you’ll have to wait for an hour while the X window manager gets compiled (ok, this is only the first time).
Fortunately it does what it’s supposed to, it just takes quite a lot longer than APT and the repository of available software is tiny compared to Ubuntu’s insane heap of FOSS. For example, something like ‘htop‘ is not available on OS X, but luckily there’s the Activity Monitor GUI to offer some compensation.

OS X < GNU

Mac OS X implements all the usual Unix tools. Things like find and grep. These are kinda different to the GNU-tools I used on my Ubuntu laptop and Debian servers. As far as I can tell after these few days of use, the Mac OS X versions are inferior to their GNU-siblings. I’ll have to get used to this or find alternative tools

So much for the bitching, here are a few things I do like:

It just works

A lot of stuff works just fine out of the box. The prepackaged software gives Ubuntu a good run for it’s money. Also, it’s obvious OS X is more popular than Linux. Quite a few software products offer a reasonable up-to-date version for Mac. Skype for example has a Mac version which is nearly identical to the Windows version, while the Linux version is lacking a lot.

Basic hardware compatibility is not an issue

If you’re the only one making hardware for your OS (or is it the other way around?) it’s easy to make that hardware supported in the OS. While the linux kernel provides support for nearly every graphics card, sound card, IDE controller and toaster made in the history of electronics, OS X only has to support the very limited set of hardware used by Apple. And to Apple’s credit, it does this very well. Unlike Linux they don’t have to work around crappy vendors breaking standards that crash your laptop after a resume-from-standby. As a result, stuff like suspend-to-ram and connecting multiple monitors works really well and quickly.

It’s a lot like Linux

Overall I feel quite at home in OS X already. I was able to set up a proper development environment during a single evening, inspite of the great efforts of MacPorts to keep compiling very large bits of software. During the coming weeks I expect to find a lot more things that are different/worse/better in OS X compared to Ubuntu but the past few days have made me optimistic about the chances of OS X being the only OS on this Macbook. I originally intended to go for a triple boot setup; Windows XP, Ubuntu and OS X, but I think I’ll settle for installing Windows XP in a VirtualBox. Not because I like XP that much, but for testing web-application in the worst browser ever made and it’s younger brothers.

That’s it for now, I’ll probably write a new post in a couple of weeks, because I’m sure OS X has plenty of annoyances and amazements left in store for me.

The contestants

mod_rails (Apache 2.2.8)

Thin 0.8.0 (Cluster of 4 behind nginx 0.5.33, both TCP and Unix socket)

Ebb 0.1.0 (Cluster of 4 behind nginx 0.5.33)

Mongrel 1.1.4 (Cluster of 4 behind nginx 0.5.33)

The benchmark is the index page of Leef je doel, the hardware used is a Dell D830 laptop with a T7300 (speedstep disabled), 2GB of RAM and a 7200RPM harddisk. The page runs a few queries and also serves a lot of fragments from memcached.

The results

Performance

Like my previous test, Ebb comes out on top, closely followed by Thin.

Using the default Apache2 config supplied by Ubuntu, mod_rails shows performance equal to mongrel_cluster behind nginx. I’m sure that by tweaking the Apache config files, some performance can be gained.

Compared to setting up nginx and thin/ebb/mongrel, setting up mod_rails is a lot simpler. Have a look at their screencast to see how easy it is, if you have a working Apache2, you’ll have mod_rails up and running within minutes. There are some small issues with mod_rails at the moment, for a list, check mod_rails’ Google Code page.

During development we already prepared caching for all pages, mostly fragment caching. To expire cache that’s no longer current, we use sweepers that get called when something relevant is updated. These sweepers only sweep the caches that get outdated.

Because we were unfamiliar with caching and needed to see the result of our fragment caching, we used the file_store to store the generated caches. These files are stored on disk and this way you can easily see how many cache gets generated and what they contain.

Regex and file_store == FAIL

To sweep caches we used regular expressions, this way we could easily sweep all relevant caches at once. This was a bad idea, as you can read here and here

During the beta phase the size of the site steadily increased, more users, more goals, more groups. There was a noticeable delay whenever you updated/created something. A short investigation pointed to the cache sweepers as the culprit.

The file_store for cache isn’t exactly the fastest solution to store your cache, but when you combine it with regex sweepers, things really slow down. Whenever you do a regex sweep, all files in the cache directory get returned (not that surprising if you think of it), and the regex is run against it. So even if you do a sweep on /goals, it will also return files in /users and /groups. As a result, updating your profile could take 15 seconds.

MemCached

We’d been planning on moving the cache to memcached all along, so this seemed a good opportunity to do it. In the next few paragraphs I’ll describe how to install memcached, get the correct Rails plugin to make memcached play nice with fragment caching and how to configure your Rails application so it uses your memcache server.

Installation of MemCached on GNU/Linux

First of all, you’ll need to memcache daemon, assuming you have a nice linux distro you can:

sudo apt-get install memcached

This will work on Debian, Ubuntu and other Debian-based distro’s, on Gentoo you can

emerge memcached

The great thing about memcached is its simplicity, it requires no configuration after installation, just run it.

All about the gems baby

Now we’ll get the gem to allow Ruby to talk to memcache. There are two gems that do this, Ruby-MemCache and memcache-client. memcache-client is supposed to be faster, so I used that.

sudo gem install memcache-client

Plugin to play nice with rails

Rails’ fragment caching doesn’t work with memcached out of the box, you’ll need a plugin. This plugin also adds a nice bonus to the cache method in views.

script/plugin install svn://rubyforge.org/var/svn/zventstools/projects/extended_fragment_cache

Now we need to configure your Rails app to use the memcached server. You’ll need to edit your config/environments/production.rb

memcache_options = {
:c_threshold = 10_00,
:compression = true,
:debug = false,
:namespace = 'yourappname_or_anything_you_like',
:readonly = false,
:urlencode = false
}
CACHE = MemCache.new(memcache_options)
CACHE.servers = '127.0.0.1:11211'
config.action_controller.fragment_cache_store = CACHE, {}

That’s all folks!

That’s it, you’re all done, the ‘cache’ method in your views will now use the memcache server.

Oh wait, there’s an encore

But there’s more, using memcached you can set expiry times for caches. I edited the plugin for a default expiry time of 1 day. In vendor/plugins/extended_fragment_cache/lib/extended_fragment_cache.rb look for def write(key,content,options=nil)

def write(key,content,options=nil)
  expiry = options && options[:expire] || 1.day
  begin
    set(key,content,expiry)
    rescue ActiveRecord::Base.logger.error("MemCache Error: #{$!}")
    rescue MemCache::MemCacheError = err
    ActiveRecord::Base.logger.error("MemCache Error: #{$!}")
  end
end

You can change the 1.day to anything you want. To override this default behaviour, you can use the following code in your views

cache('goals/large_cloud', {:expire = 30.minutes.to_i})  do

This will make the cache called ‘goals/large_cloud’ to expire 30 minutes after it got created.

There are two important things to consider when you move to memcached

1. MemCached doesn’t support regex based expiry of caches. You need to manually enter every cache you want to expire. You can do this in some nice methods of course. Here’s ours for expiring the cache when a user gets updated.

def expire_user_fragments(user)
  fragments = %w[author_icon author_link side_block_friends ..snip...]
  fragments.each do |f|
    expire_fragment("user/#{user.id}/#{f}")
  end
end

2. Your application will fail when the MemCache server becomes unavailable. If you ever restart MemCache, or if it crashes (haven’t seen that happen yet), you need to restart your mongrel-cluster/thin/ebb.

3. When you restart MemCache, all cache is cleared, and you need to restart your mongrel-cluster/thin/ebb.

This guide only talks about fragment caching, over at Ben Curtis’ blog, you can read all about action caching.

To run torrents on my server I used Torrentflux. It’s a PHP-webinterface to BitTornado and it has a nice community and forum where they regularly post hacks to add new features. After a while I found out there was a different version (fork) of Torrentflux with more features and I switched.

TF-B4rt

This fork of Torrentflux is called Torrentflux-b4rt. It adds a bunch of new features, including some improvements to the interface and support for multiple bittorrent clients. TF-b4rt supports BitTornado, Mainline, Transmission and Azureus.
Azureus support is provided through a piece of middleman software called Fluazu, which talks in XML to Azureus. This (and being written in Python) causes Fluazu to use quite a bit of CPU cycles. About 10-30% on my 2.13Ghz Core2Duo server. It’s also a rather delicate system, if you manually delete a torrent through the Azureus-GUI/console, or disk, the webinterface starts to bug. It will show torrents running that aren’t, and you can’t delete them because it’s convinced they’re still active. But even if I didn’t do something like that, Fluazu got confused with my torrents.

The main reason I wanted to use Azureus (and thus, Fluazu), is the support for global download/upload rate-limits. However, the extra CPU usage for Fluazu was bothering me and I looked for alternatives.

Azureus Web Interface

Azureus has it’s own web interface. Two actually. The HTML Web-UI, and the Swing Web-UI. I tried the Swing Web-UI and it works rather well. You upload torrents, Azureus downloads them. Because it talks directly to Azureus it’s more reliable than Torrentflux-Fluazu.

Installing the Swing Web-UI breaks support for TF-B4rt/Fluazu.

Clutch

Another webinterface is called Clutch. It’s a webinterface that talks to the Tranmisson daemon. Transmission is a fast light-weight torrent client. Using the daemon version, you can have multiple uploads with a global upload/download limit.

After loading about 20 torrents in Transmission, I was using about 1% memory, while Azureus used 9.5%. CPU usage for Azureus was 2-4%, while Transmission rarely got above 1%.

Torrentflux-b4rt provides the most functuonality. It has some features you can’t find in the other two. For example you can enter RSS-feeds for torrents that get downloaded automatically and you can upload multiple files at a time. Also you can stop/start/delete all torrents and data at once. It also supports multiple users, multiple torrent-clients and has more extensive statistics. Because of all these extra feature, Torrentflux is also used in hosted solutions.

Unfortunately it doesn’t support global upload/download limits on its own. It needs the Fluazu/Azureus combination for this. And using this combination is rather CPU intensive. So all in all, Torrentflux-b4rt is a great solution if you don’t need the global upload/download limits.

The Azureus Web Interface is very nice if you’re a single user. The Azureus bittorrent client has a lot of features so having a web interface to this powerful client is great. It’s quite a bit more resource intensive compared to BitTornado or Transmission, but it does feature advanced features like the distributed tracker, which allows you to download torrents of which the original tracker is down.

Transmission+Clutch has a very clean interface and turned out to be the best choice for me. It doesn’t have all the advanced features of Torrentflux, but it does the basics just right. It requires few resources, allows you to set global upload/download rates and it has a feature to start/stop all torrents at once.

On my old laptop I tried using them before, but the poor thing couldn’t handle it.

Now with my new laptop I decided to give it another go.

Using compiz-fusion you can put these screenlets on a special ‘widget layer’, which you can summon by pressing a keyboard key. Whenever you need some information or need to make a small note, you press that keyboard key and the widget layer is displayed. Pressing the same key again and the widgets disappear.

There’s a nice screenlet that displays some sensor information, CPU and HDD temperatures. However, my laptop has another sensor on the graphics card that shows the GPU temperature. I just had to have this added to the screenlet.

I made a modified version of the Temperature2 script to generate a screenlet for nvidia GPU temperature. You’ll need the proprietary nvidia drivers installed and access to the ‘nvidia-settings’ utility. Download the file and place it in the ~/.screenlets/Temperature2/ directory, overwriting the old file.

Update: My modifications are now included into version 0.4 of Temperature2. You can get it at screenlets.org

We looked at RailsLiveCD.org but it wasn’t exactly the way we wanted. And whenever something isn’t exactly as you want in the world of OSS, you can fix it.

It took a great tutorial, some trial/error and a couple of failed CD’s, but finally I managed to get it just the way I wanted.

The failed CD’s were due to the fact that ‘qemu’, an emulation program that you can use to test bootable .iso files, ran very slowly on my laptop.

The live CD is based on Ubuntu Feisty. It includes MySQL, vim-ruby, Aptana, ruby, rails and the sample project we’ll be using at the workshop.

No other live cd would have our own project on the disk

I’ll post the ISO once we’ve given the workshop.

Update

You can now download the Ubuntu on Rails iso.

With 5 disks in a system, this would force me to create 3 graphs, or 1 graph with the temperatures of the 4 disks connected to the RAID array averaged into a single line.

I’ve always wanted to move from MRTG to RRDtool, because RRDtool can handle multiple datasources per graph, and has more options.

However, I found RRDtool very hard to configure compared to MRTG. So I went looking for an alternative.

Cacti is a front-end for RRDtool, which makes it easier to create nice graphs. If you just want to graph routers, it’s even easier to configure than MRTG. But if you like to have custom stats, like the Areca HDD temperatures, it takes a little more work.
Cacti has some nice guides on how to create graphs and new input methods on their site, but even with those it took me quite some time to create my first graph.

By now I’ve migrated all my graphs from my personal MRTG page to Cacti

Recently I re-enabled all desktop eyecandy by using the ‘compiz-fusion‘ project. I use compiz-fusion in combination with the closed source aTi fglrx driver. However, my laptop began running hot again. At first I thought this was because of all the 3D eyecandy that uses your videocard.

After a while I decided to check the powerstate of my videocard using the ‘aticonfig –lsp’ command and got the following error:

Error: Unable to obtain POWERplay information.

After some investigation I found out that the ‘aticonfig’ utility cannot be run from X display :1. Thanks to aTi’s craptastic linux drivers you’re forced to run 3D-eyecandy desktops on display :1 instead of display :0. So the solution was simple, kind of. I wrote the following snippet that can be run from display :1

DISPLAY=:0

aticonfig –set-powerstate=1

DISPLAY=:1

Put this in a small script and make it execute everytime you start your X session ( System, Prefences, Sessions). Hopefully aTi manages to catch up with Nvidia’s linux driver support and keep their promise about better drivers.