More and more now, I've noticed folks posting their talk slides on SlideShare. I've decided to give it a try and post one of my talks over there and see how it goes. If it sees decent traffic, I'll probably post all my slides over there too.
June 2008 Archives
In setting up a buildbot master, and, reading some documentation, noticed that it has tons of status plugins, one of which is an IRC bot.
Before anything else, you need TwistedWords, an instant-messaging library. So I had to go ahead, download and install it.
cd tmp
lftpget -c http://tmrc.mit.edu/mirror/twisted/Words/8.1/TwistedWords-8.1.0.tar.bz2
tar jxvf TwistedWords-8.1.0.tar.bz2
cd TwistedWords-8.1.0
python setup.py install
Once installed, all I had to do is to make a 2-line change to the buildbot master's master.cfg config file
from buildbot.status import words
c['status'].append(words.IRC('irc.mozilla.org', 'thunderbot', ['maildev']))
Next thing you know, you've got an IRC bot announcing builds, and you can even ask for new builds straight from an irc /msg, cool.
C'est une fète fabriquée de toute pièces, mais c'est une fète à célébrer quand même!
Et puis en plus, c'est en double pour les grand-pères!
Il y a pas personne de mieux au monde pour lire des histoire des Schtroumpfs! Il l'aime son grand-père a lunettes!
It's been a long little while, but things are finally reaching the point where I feel sufficiently satisfied to talk about it. (Plus, I've got time available now to do just that).
Life started with a rack full of brand-new Sun hardware, all wired in, mostly powered off. Thanks to mrz and Justin for all the heavy lifting!
When I was handed the 'keys', (actually, I handed my own ssh key) one of the Sun 4150s was pre-installed with CentOS. I was in!
I'll skip over an interesting learning experience with fancy Cisco switches and firewalls, maybe I'll talk about it some other time.
The first step was getting things to the point where they could boot and do something usefull. It's always possible to use remote management tools to trick the BIOS into booting from some disk image over the network, but it's very non-standard and usually requires unscriptable Java based tools, yuck.
This is where Cobbler comes in and saves the day. In short, it's an amazing tool, out of Redhat"s labs, that allows for the easy building and managment of a complete remote boot and installation solution. With it, you can easily configure systems by MAC addresses, pick what distribution you want to install on it, and you are done. Once setup, you just need to turn on the server you need, and within minutes, you have a freshly installed server, with all the specific installation instructions you wanted. The best with this is that it's 100% reproducible. You need 10 machines installed all the same, easy, just cut-n-paste their MAC addresses, turn them all on, go get a cofee, and you've got 10 identical machines. You want to re-install a system for a reason or another? Same thing. Configure it for a reinstall in cobbler and reboot the system.
It's very handy, and once you put all it's configuration in source-control, your entire server farm is easily reproductible.
Of course, having a bunch of servers all running a pristine Linux distribution isn't that usefull. You need to get them configured in various ways to actually get them to do the work you need them to do. In my particular case, first I needed basic configuration (networking/access control/logging/etc) and second, more importantly, I needed some virtual machines running to host some needed test services.
It's always possible to just login to servers and start installing packages, editing configuration files, starting services, etc. But it's such a bad idea. Even if you are a documentation fanatic, you'll always miss a step or two, and rebuilding a system exactly the way it was becomes impossible.
That's what tools like Puppet are for. They are an automated way to describe what you want on a system and perform the necessary tasks to get the system there. So, instead of installing the ntpd package, hand-editing /etc/ntp.conf and starting the ntp service yourself, you let Puppet do it for you.
Of course, it's also all in source-control, so every single change also gets tracked. The tricky part about systems like Puppet is that they are declarative. So, in the example above, you describe things much differently. You say, /etc/ntpd.conf needs to exist and needs this content. The package ntpd needs to be installed. The service ntpd : needs the ntpd package, the /etc/ntpd.conf file, and needs to be running.
Then Puppet figures what to do, in what order, and also will know that the running state of the ntpd service depends on the content of the /etc/ntpd.conf file, so if that changes, the service needs to be restarted.
There are many such systems out there, like cfengine and bcfg2. I've picked Puppet because it's very actively developped right now, plus supports Linux, Solaris & Mac OSX. The only downside to me is that it's all written in Ruby, and my Ruby-foo is pretty weak. Not a very big deal, because for most normal stuff, you deal with a configuration language, not the actual Ruby language.
At the end of last week, I had all the pieces in place, and I was finally able to directly help out Thunderbird development.
Somebody on #maildev needed access to some live IMAP servers that understood the CONDSTORE extension to test some stuff. With all these great tools already in place and working, it took very little time to get a new virtual machine configured, booted, installed and running some brand-new IMAP servers with support for CONDSTORE.
It's a small step, but it's really nice to finally get to directly contribute (even if in a small way) to the continuing success of Tunderbird.
There is still a lot of work to get done, things are not as automated as I would want them, and I've got some important missing pieces still dangling. For instance, there is no monitoring in place yet, so let's hope nothing goes down when I am not looking.
This coming week, the objective is to get a new mini-build-farm up and running, building Thunderbid continuously on Linux, Windows & OS X. While that's going on, I'll continue laying the missing bricks to the infrastructure itself.
That's enough for now.
Today, I ran into a very cool Thunderbird plugin, via David Ascher : ThreadVis
I remember having read the original IBM paper on this idea by Bernard J. Kerr
a while ago, and it was a very interesting take on thread visualization. I remember thinking at the time, bummer, I wish there was a Thunderbird plugin for this. Well, it took a little while, but it's finally here.
The idea is to use arcs to represent the relationship between the components of the thread. Why are most email threads displayed as trees, I am not sure. Looking at actual threads in my inbox with this addon confirmed to me that the idea has some merit. I especially love the fact that in this implementation, the arc length is relative to the time between 2 replies.
It's part of it's author's Master Thesis, so the extension also includes anonymized usage data extraction, to help him analyze how this is working out. It's optional, disabled by default, but do trun it on to help him out (I did).
If interested, install it directly from the official Mozilla Add-Ons site.
