Playing back 1080p content however, was still a problem. I was never able to get smooth playback with mplayer. Only the xbmc videoplayer could play 1080p smoothly on my system. XBMC is not an option due to the WAF, it has to be something that runs from MythTV.

Fooling around with coreavc-for-linux (I love CoreAVC on Windows) got me nowhere. Recompiling, tweaking and updating mplayer/nvidia/xorg, nothing seemed to work.

Until I tried a new version of mplayer from the Debian Multimedia repositories. The mplayer version in that repository has support for a new feature in the linux nvidia drivers that (finally) allows video decoding on the GPU.

The easy steps to get this to work:

• Download and install the new nvidia linux drivers (180+)
• Add the Debian Multimedia repository to your sources.list
• Install mplayer from the new Debian Multimedia repository
• Start mplayer with ‘-vo vdpau -vc ffh264vdpau’

More than a year of tweaking and tinkering made redundant by these easy steps, excellent!

During development we already prepared caching for all pages, mostly fragment caching. To expire cache that’s no longer current, we use sweepers that get called when something relevant is updated. These sweepers only sweep the caches that get outdated.

Because we were unfamiliar with caching and needed to see the result of our fragment caching, we used the file_store to store the generated caches. These files are stored on disk and this way you can easily see how many cache gets generated and what they contain.

Regex and file_store == FAIL

To sweep caches we used regular expressions, this way we could easily sweep all relevant caches at once. This was a bad idea, as you can read here and here

During the beta phase the size of the site steadily increased, more users, more goals, more groups. There was a noticeable delay whenever you updated/created something. A short investigation pointed to the cache sweepers as the culprit.

The file_store for cache isn’t exactly the fastest solution to store your cache, but when you combine it with regex sweepers, things really slow down. Whenever you do a regex sweep, all files in the cache directory get returned (not that surprising if you think of it), and the regex is run against it. So even if you do a sweep on /goals, it will also return files in /users and /groups. As a result, updating your profile could take 15 seconds.

MemCached

We’d been planning on moving the cache to memcached all along, so this seemed a good opportunity to do it. In the next few paragraphs I’ll describe how to install memcached, get the correct Rails plugin to make memcached play nice with fragment caching and how to configure your Rails application so it uses your memcache server.

Installation of MemCached on GNU/Linux

First of all, you’ll need to memcache daemon, assuming you have a nice linux distro you can:

sudo apt-get install memcached

This will work on Debian, Ubuntu and other Debian-based distro’s, on Gentoo you can

emerge memcached

The great thing about memcached is its simplicity, it requires no configuration after installation, just run it.

All about the gems baby

Now we’ll get the gem to allow Ruby to talk to memcache. There are two gems that do this, Ruby-MemCache and memcache-client. memcache-client is supposed to be faster, so I used that.

sudo gem install memcache-client

Plugin to play nice with rails

Rails’ fragment caching doesn’t work with memcached out of the box, you’ll need a plugin. This plugin also adds a nice bonus to the cache method in views.

script/plugin install svn://rubyforge.org/var/svn/zventstools/projects/extended_fragment_cache

Now we need to configure your Rails app to use the memcached server. You’ll need to edit your config/environments/production.rb

memcache_options = {
:c_threshold = 10_00,
:compression = true,
:debug = false,
:namespace = 'yourappname_or_anything_you_like',
:readonly = false,
:urlencode = false
}
CACHE = MemCache.new(memcache_options)
CACHE.servers = '127.0.0.1:11211'
config.action_controller.fragment_cache_store = CACHE, {}

That’s all folks!

That’s it, you’re all done, the ‘cache’ method in your views will now use the memcache server.

Oh wait, there’s an encore

But there’s more, using memcached you can set expiry times for caches. I edited the plugin for a default expiry time of 1 day. In vendor/plugins/extended_fragment_cache/lib/extended_fragment_cache.rb look for def write(key,content,options=nil)

def write(key,content,options=nil)
  expiry = options && options[:expire] || 1.day
  begin
    set(key,content,expiry)
    rescue ActiveRecord::Base.logger.error("MemCache Error: #{$!}")
    rescue MemCache::MemCacheError = err
    ActiveRecord::Base.logger.error("MemCache Error: #{$!}")
  end
end

You can change the 1.day to anything you want. To override this default behaviour, you can use the following code in your views

cache('goals/large_cloud', {:expire = 30.minutes.to_i})  do

This will make the cache called ‘goals/large_cloud’ to expire 30 minutes after it got created.

There are two important things to consider when you move to memcached

1. MemCached doesn’t support regex based expiry of caches. You need to manually enter every cache you want to expire. You can do this in some nice methods of course. Here’s ours for expiring the cache when a user gets updated.

def expire_user_fragments(user)
  fragments = %w[author_icon author_link side_block_friends ..snip...]
  fragments.each do |f|
    expire_fragment("user/#{user.id}/#{f}")
  end
end

2. Your application will fail when the MemCache server becomes unavailable. If you ever restart MemCache, or if it crashes (haven’t seen that happen yet), you need to restart your mongrel-cluster/thin/ebb.

3. When you restart MemCache, all cache is cleared, and you need to restart your mongrel-cluster/thin/ebb.

This guide only talks about fragment caching, over at Ben Curtis’ blog, you can read all about action caching.

Areca has made some excellent tools available to monitor your array and attached disks. For example, Areca’s web interface can show you all the disk temperatures.

In Areca’s web interface you have total control over your RAID array. You can add/remove disks, create new arrays and do a check on existing arrays.

But it’s annoying to have to log in to this web interface just to check the temperatures of the disks. Also, it doesn’t come with nice graphs and history information. It just shows the current temperatures.

Areca has also made a CLI utility. Using this utility you can do the same things as in the web interface. I’ve used the CLI utility to generate the data needed by MRTG. MRTG can only handle 2 datasources in one graph. Having 5 disks total (1 bootdisk, 4 RAID member disks), I would need 3 graphs to monitor the disks. So I decided to create a little BASH script that would take the temperature of the bootdisk, and the average temperature of the 4 RAID disks and feed this to MRTG.

Third, admins of the match can request a player to record his actions to a so called ‘demo’. If the user would be cheating this would show on a demo. However, no sane player would still cheat while recording a demo.
A few years ago Valve released a piece of software that allows you to watch matches. This software was created due to the increasing popularity of Counter-Strike as an “e-Sports’ game.

The software, called HLTV, is basically a proxy server that allows people to watch a match without overloading the gameserver. It has a built-in delay, so that spectating people can’t give vital informatin, like enemy positions, to the players with tools as direct messaging and voice communication.

HLTV is also able to record a demo. The great advantage of demo’s recorded by HLTV is that the single HLTV demo records all the actions of all players, and players don’t have to do anything. If an admin suscpects someone is cheating, he would go into the HLTV console and record a demo. When the match is over he could send this file to him and watch for cheats.

However, with about 30 admins in #cs.gather.nl who normally stay for just a few weeks, it’s not possible to give them shell access to a Debian GNU/Linux machine and control the HLTV consoles. It would be much easier to communicate with the HLTV consoles from IRC and only allow operators from #cs.gather.nl (the admins) to execute commands.

To do this, you need an automated chat client, a bot, and a specially written program for ths bot, a script. I had seen IRC bots that appeared to be controlling HLTV’s, but never found a script that was freely available. It also appeared that most bots depended on so-called “remote console” connections to the HLTV’s.

The remote console, or rcon, is a way of remotely controlling consoles of gameservers. Unfortunately, the developer of Counter-Strike has changed this protocol in the past a few times, leading to non-functional scripts/bots.

I also wanted to try another method of talking to the HTLV consoles. I’ve decided to let the bot talk to the HLTV consoles using the “screen” command. Screen is a GNU tool for running programs in the background. Whenever you want you can bring it back to the foreground and execute commands. It’s also possible to send commands to a screen in the background and get information from a screen running in the background.

The downside of this approache compared to the remote console approach is that all HLTV proxy’s have to run on the same physical computer as the bot controlling them. But at that cost, it is more reliable and faster to talk to the HLTV proxy’s.

To setup your own HLTV’s which you control with a bot:

Prerequisites:

Eggdrop bot

Instructions for installing eggdrop can be found at www.eggheads.org

Screen

On Debian/Ubuntu simply apt-get install screen

Step 1 – Install HLTV

wget http://www.ariekanarie.nl/tutorials/hltv.tar.bz2

tar vfxj hltv.tar.bz2

Set up hltv.cfg and hltv.sh according to your configuration.
Step 2 – Runnng HLTV

You have to run HLTV in a screen. The TCL script will send commands to this screen. If you want to run multiple HLTV’s, simply copy the installation directory and make the hltv run in a different screenname. You have to run HLTV with the same username as the eggdrop bot, else the eggdrop script can’t send commands to the screen sessions.
Step 3 – Installing/Configuring the HLTV TCL script

First download the hltv script here and put it in your eggdrop/scripts directory.
Open hltv.tcl with your favorite text editor and change the following variables to what you need/like.

set hltv_channel “#cs.gather.nl”
set hltv_command “!hltv”

Put the names of the screen sessions in a space seperated list.

For example, my hltv screen names are hltv1, hltv2, hltv3, etc…

set hltv_screens {hltv1 hltv2 hltv3 hltv4 hltv5}

Step 4 – Loading the HLTV TCL script

Open the config file of your eggdrop, and scroll down to the part where all scripts are loaded. Add the following line:

source scripts/hltv.tcl

Save and exit the editor. Rehash the bot to load the script

Commands for the script

!hltv status – Detailed statusreport
!hltv stat – Basic statusreport
!hltv connect ip:port password (optional) – Connect to server, takes optional password argument
!hltv record demoname – Starts demorecord with name
!hltv stop – Disconnects from server
!hltv stoprecord – Stops demorecording on
!hltv status – Detailed statusreport for

Tutorial for installing MythTV frontend-backend on Debian with Hauppauge 350 capture card

Requirements:

• Hauppauge PVR350 capture card (most parts of this guide will work with other cards)
• Working Debian Stable/Testing/Unstable installation (Ubuntu will work as well)
• Patience
• Some experience with kernels, modules, nasty compilation problems
• More patience

Step 1 – Installing kernel 2.6.17
Kernel 2.6.17 has some of the drivers we need available in the kernel, as opposed to previous versions which were only available through the IVTV project (http://ivtvdriver.org)

Note that kernel 2.6.17 is currently only available in the Debian Unstable repository. But you can also get it from www.kernel.org
So, get your root-hat on by su root, sudo -s, logging in as root, or whatever you like.

Getting the kernel:

apt-get install linux-source-2.6.17

After downloading 40MB of kernel source code, comments and kernel jokes, we’re ready to extract these from the tarball it came in.

cd /usr/src
tar vfxj linux-source-2.6.17.tar.bz2
ln -s linux-source-2.6.17 linux

Before we can compile a kernel, we’ll need some stuff to make it all possible.

apt-get install build-essential kernel-package libncurses5-dev wget gcc-4.1

This should have gotten us all the tools we’ve need.

We’re now gonna steal your working kernel configuration, to save you a lot of headaches concerning the compilation of your first kernel.
The config file of your current kernel is located in /boot/config-”name-of-your-current-kernel”.
To make it easier, just type the following.

cp /boot/config-`uname -r` /usr/src/linux/.config

We’ve now copied your current kernel configuration to the 2.6.17 kernel source directory.

Let’s do some kernel configuration stuff, don’t worry, it’s a ton of fun.

cd /usr/src/linux
make menuconfig

You’ll now be presented with the glorious ncurses kernel configuration.

Navigate to “Device drivers” -> “Multimedia Support” and make sure you select “Video For Linux” by pressing the “Y” key.

Go to “Video Capture Adapters” and press the M key on the following lines:
BT848 Video For Linux
Conexant 2388x (bt878 successor) support

Go to “Encoders and Decoders” and press the M key on the following lines:
Conexant CX2584x audio/video decoders
Philips SAA7113/4/5 video decoders
Philips SAA7127/9 digital video encoders

Exit the kernel configuration menu and save the configuration when asked to do so.

Back at the prompt, you’ll have to type some stuff to start the kernel compilation.

make-kpkg kernel_image

This will take a while.

After compiling, you can install the kernel.

cd /usr/src
dpkg -i linux-image-2.6.17_i386.deb (the name might be slightly different)

After installing the kernel, it should have been put into your bootloader as the new default kernel to boot.
This is a good time to reboot your machine and prey it will work.

Assuming your machine is still working, you should now be in kernel 2.6.17. Congratulations!

Step 2. IVTV Installation

We’re now going to install IVTV.
This is a good time to make a user called mythtv where you can dump all this shit and run mythbackend from in the end.
But first we’re going to download and ‘install’ the firmware for your TV card.
The guys at IVTVdriver.org made this one very easy, so kudos to them.

cd /lib/firmware
wget http://dl.ivtvdriver.org/ivtv/firmware/firmware.tar.gz
tar zxvf firmware.tar.gz

adduser mythtv
cd /home/mythtv
mkdir mythsetup
cd mythsetup
wget http://dl.ivtvdriver.org/ivtv/archive/0.7.x/ivtv-0.7.0.tar.gz
tar zxvf ivtv-0.7.0.tar.gz
cd ivtv-0.7.0
make
make install

Because we’re running kernel 2.6.17, we need lirc (for the remote control) to be built from CVS.
To make life easy for you, you can grab a tarball and compile.

cd /home/mythtv/mythsetup
wget http://www.ariekanarie.nl/tutorials/lirc-cvs.tar.gz
tar zxvf lirc-cvs.tar.gz
cd lirc
./configure
make
make install

Open /etc/modules in your favourite editor and enter the following modulenames

ivtv
msp3400
saa7115
saa7127
tuner
lirc_i2c
ivtv_fb

Save and exit.Create a new file called /etc/modprobe.d/ivtv and enter the following in that file:

alias char-major-61 lirc_i2c
alias char-major-81 videodev
alias char-major-81-0 ivtv
alias char-major-81-1 bttv
options ivtv ivtv_debug=1
options tuner pal=B

Save the file and exit your favourite editor and type the following commands.

depmod -a
modprobe ivtv ivtv_fb lirc_i2c lirc_dev

This should fire up all the modules we need. Use the command ‘dmesg’ to check for any errors regarding the ivtv modules.

Step 3. MythTV installation

The current MythTV release is 0.19.1. For reasons beyond my comprehension, this version is not directly available from the MythTV website (just 0.19).

To get the latest and greatest MythTV, you need to get it from SVN. Don’t worry, the code is stable, it’s just not in a tarball.

How to get the good stuff. First we’ll try to satisfy a load of dependencies for compiling MythTV.

apt-get install subversion liblame-dev liblame0 libdvdnav-dev libdvdnav4 libiec61883-0 libiec61883-dev libavc1394-0 libavc1394-dev libasound2-dev libqt3-mt-dev en qt3-dev-tools libqt3-mt-mysql mysql-server
cd /home/mythtv/mythsetup

svn co http://svn.mythtv.org/svn/branches/release-0-19-fixes/mythtv

svn co http://svn.mythtv.org/svn/branches/release-0-19-fixes/mythplugins

svn co http://svn.mythtv.org/svn/branches/release-0-19-fixes/myththemes

This should have downloaded MythTV 0.19.1

Now on to compiling.

cd /home/mythtv/mythtv/mythsetup/mythtv

./configure

qmake mythtv.pro

make

make install