Planet CompSoc

The problem

A word document (plain old .doc, not 2007) should be received by e-mail, fed to a script, turned into a PDF and published on a website.

At my disposal

My server running Debian ‘Lenny’, which does not have a display of any kind.

How hard can it be?

Harder than it should have been, as ever. Here are my steps:

# aptitude install python-uno xvfb openoffice.org-java-common openoffice.org-writer unoconv

You’ll note the inclusion of Xvfb there, because it turns out that “headless” mode in OpenOffice isn’t really headless at all. Sigh. Also sigh some more at the broken dependencies of the unoconv Debian package.

Now we can write our script to do the actual conversion. Shame it took twice as long as it should have…

In June/July 2010 I spent ten days travelling in Rajasthan, India with friends; this is my diary of the trip (full list of entries here).

Saturday 26 July

Landed at Indira Gandhi International Airport at 9.30AM local time. Walked out of the terminal into searing heat (over 40 degrees in the shade) and a medley of noise and traffic. Dodged our way through the traffic and found our taxi waiting. Saw our bags strapped to the roof and set off into the traffic. As we’d been warned, the only similarity between British driving and Indian is that both countries drive on the left – in India the horn means “get out of my way please, I want to pass” and flashing your lights means “I’m not going to stop, so get out of the way or I’ll ram you”. In the city, people, animals and motorbikes all compete for road space with the cars and lorries, there are no traffic lights to speak of, and the smallest gap is made to be squeezed through.

On the main highway – this first day we’re driving from Dheli to K’s grandparents’ house in Jaipur, approximately 200 miles to the south-east – the sheer volume of traffic and the two-lane road limits progress to a maximum of 50mph. Drove past Debenhams and M&S on the way out of Dheli; wondering just how westernized this country is…

Stopped for lunch at India’s answer to the motorway services, about 1pm. Stepped out of our air-conditioned taxi into searing, dusty mid-day heat and ran for cover in the building, which had just enough air conditioning and fans to reduce the temperature from ‘horrifically hot’ to ‘bearable’. Ate a nice samosa with a coke, then back to the car.

Arrived in Jaipur at about 4.30pm, glad to get out of the car. The drive down wasn’t particularly inspiring; mostly motorway.

Met K’s grandparents – great people, made us feel really welcome – settled into our hotel and had a really nice meal with them before going to bed (or trying to; despite the air conditioning, still pretty warm!).

Editor’s note: more to follow soon, with photos!

After a slightly too long hiatus, I've released the second bugfix release of the 0.7 series.There's not a whole lot new and exciting in this one, just lots of small fixes - bugfixes, removal of some unused variables, a few more command line arguments where they should have been already, that sort of thing. You can read more in the release notes. In other news, I'll be giving a talk at DjangoCon US 2010 which, for once, isn't about South, but instead covers a more broad spectrum of databases and schemas, in particular encompassing schemaless databases (which do still need migrations, in some form), so if you're going to be at the conference, come along, or just catch me outside of a session if you want to talk. I apologise for the slightly slow rate of work on South as of late, but I'm in the middle of moving cities. After next week, I'll be settled in London, and hopefully work will progress faster. I'll also be looking for contract/consulting work after next week, so if you'd like to hire me, see my 'corporate site'.

Early last month, Lassi Syrjälä released Telepathy-Ring, Nokia’s Telepathy connection manager for GSM telephony, under the LGPL. The version used on the N900 talks to a proprietary daemon to drive the cellular hardware, but this new 2.x.y series has been ported to oFono, Intel and Nokia’s Free cellular modem daemon. I was trying out Ring using oFono’s phone simulator backend, until it was pointed out that oFono also supports my laptop's built-in GSM chip. Oh really? Let’s see…

A few minutes of tweaking later, and I was looking at an apparently-unremarkable Empathy conversation window:

Ring needed a few little hacks to get this going, mostly because laptops’ GSM chips don’t generally support making GSM calls, which Ring expects to be able to do. But I didn’t have to touch any other Telepathy components’ source: I installed my Ring branch, opened the Empathy accounts dialog, created a new “tel” account, and here we are. +447771██████ in that screenshot is my real actual phone, and this conversation looks just how you’d expect.

Of course, right now this is a proof-of-concept; it’s not really ready for non-developers. I’m planning to clean up my Ring patches for submission upstream over the next few weeks, and will try to trick someone into writing a custom account configuration UI for Empathy; hopefully we can get this working properly pretty soon! Thanks to Lassi, Pekka Pessi (Ring’s original author), and others at Maemo; the oFono team; and other Telepathy and Empathy hackers for making this so straightforward!

I have a Grandstream GXP2000 SIP phone, courtesy of a special offer and a parent who likes SIP. It's kind of bulky, and is overkill for my uses of it: I don't actually care about it automatically downloading my address book from an arbitrary server, or running quizzes on its OSD. (Seriously, why would anyone do this?) But once I upgraded the firmware (to 1.1.2.27, which apparently you can get from Grandstream if you whine loudly enough, but I just grabbed it from, uh, GrandstreamSucks.com) to stop it crashing whenever I made or received a call (which only happened on one LAN I tried it on), it is pretty good at making and receiving calls, which is what I want it for.

You can stick custom ring tones on it, but they have to be in some bizarro format. Happily, Grandstream ship conversion tools, both for Windows and for Lunix. So I downloaded the latter; it turns out to be a modified SoX. And, oh look: they don't provide the source, or an offer to provide the source. *sigh* So, I requested the source, pointed out that they're violating section 3, and remarked that they could just throw a tarball into the same place they threw the binaries and never deal with any emails like this ever again.

I got a reply pretty quickly!

Attached per your request.

Attached was a zip file, expanding to:

% ls /tmp/sox
handlers.c  ring.c  sox.c  st_i.h

which is clearly not the complete source, and does not include build scripts.

You did not include the scripts used to control compilation. May I have the Makefile, please?

They sent me the Makefile (in a zip file).

% make
make: *** No rule to make target `Makefile.in', needed by `Makefile'.  Stop.

It appears that you have omitted the autoconf-related foo.

Please may I have complete working build scripts, please?

They sent me Makefile.in (in a zip file. This makes it much faster!!!). Of course, the rest of the build scripts and source still aren't there, so this still won't work. But let's play along:

% make
make: *** No rule to make target `configure.in', needed by `configure'.  Stop.

No double pleases this time:

Is it really too difficult to just zip up the entire directory and send it to me?

And apparently showing some anger did the trick:

We were using sox 12.17.4. You will need to use the patch based on that version.

Lo! A tarball containing the entire modified source tree, which when compiled works identically to the static binary they distribute. They still provide no source offer, but I got bored at this point. Here's a copy of the source in case you don't feel like repeating this process to get it.

(I meant to write this back in September, but never got around to it. I finally got sick of the thread sitting in my Inbox. It turns out that I never actually bothered making a custom ringtone...)

Two and a half years later

I edited this post earlier today to update the final link (having rearranged my own domain). This bumped the post on some less-than-awesome aggregators... and someone pointed out that some other links are broken. So I poked around a bit, and the .zip file offered by Grandstream does actually include the source code! Hooray. From the mtimes I suspect this took until June last year.

In June/July 2010 I spent ten days travelling in Rajasthan, India with friends; this is my diary of the trip (full list of entries here).

Friday 25 July

Flew out from Heathrow at 20.30 with Jet Airways, bound for New Delhi. A pretty decent (for 30,000 feet) Indian meal for dinner, then fell asleep over Russia after watching James Bond. India is 4.5 hours ahead of BST, so we’ll arive at about 9AM if all goes according to plan. All of us have had the relevant jabs and are carrying industrial quantities of suncream and insect repellent.

The trip was suggested by my friend K who was born in India and lived there for 15 years, and we plan to start by staying with his grandparents who live in Jaipur, the capital city of Rajasthan. Currently it’s reported to be 40 degrees centigrade in New Delhi; here’s hoping my system can cope…

Editor’s note: it gets a lot more exciting than the above, I promise. Parts 1 to 10 coming over the next week when I get time to write them up.

Five years after the cool kids first started jumping on the bandwagon, I’ve got myself a CurrentCost CC128 (Southern Electric send them to some customers for free, it seems – e.g. my granddad who didn’t want it).

So, with the addition of an eight quid data cable and the Linux box running in my lounge, may I present my electricity usage graphs. Bear in mind that these are (at the time of writing) for a five-bedroom house in central Oxford.

The parser for the XML output of the device I’m using is this one – just swap “COM20″ for “/dev/ttyUSB0″ in their testrun script and fix it to ignore empty lines read from the serial port, and you’re in business. I then hackdapted this RRDTool tutorial to plot the graphs.

After Transport For London released their new live departures API, and after Matthew Somerville made a Thing with it, I felt I had to have a go.Thus, I present Live Line, a visualisation of the trains on Tube lines overlaid on their schematic maps. It currently only does the non-forking lines - the line maps are drawn entirely with canvas, and there's a few code issues - but it still looks quite nice. The idea, and some of the logic, is borrowed from Matthew Somerville's idea, which uses geographic maps for a similar purpose.

For at least a year now, people have been suggesting to me that South should be in Django core.I've always resisted, for good reason - not only because of South's relative immaturity at the time, but also because forcing a single solution would, I think, not be a good thing in general. (South, admittedly, has a few design issues, but it's all about tradeoffs.) However, last week at djangocon.eu, I revealed, in rapid-fire English (apologies to non-native speakers for that, we're working on time dilation) my Grand Plan for the future of South and Django migrations in general, one that seems to have the approval of at least a few members of Django core. The proposal is not, as some expected, to put South completely into Django core; instead, South will be split into two parts. One, the part that implements database abstraction, dependency resolution, and history tracking, would be put into Django itself. The other part - autodetection, model freezing, and so forth, will remain as South. (If you'd like more in-depth details of the split, see my post to django-developers.) The idea behind this is simple - take the parts of South that nearly any migration solution would need (database abstraction especially), and move those into a place where they can be used easily by any migrations solution. More interestingly, because the proposal describes a common idea of dependencies and of what a migration actually is, it means that migrations can be written such that they need no supporting framework at all, and can be just pure Python calls to the database APIs (if Django ever ships any migrations, this is probably how they'd be done), raw SQL files (with a version per database backend if needs be), or a Python file managed by a supporting library (South 2.0 migrations), and that these different formats can all coexist, even on the same application. Some might argue that such a solution is slightly restrictive, but I see it as the job of a framework to provide some level of consistency, and by doing this, it means reusable apps can ship with migrations written using the library of their choice (or no supporting library, as the case may be). What does this mean for South? Well, in the short term, South 1.0 will be released, being mainly a polishing-off release of 0.7, and introducing a few new concepts (like the "rebase"). In the longer term, if my proposal meets with acceptance from the community (but, more importantly, those illustrious figures that make up Django's core developers), I'll start work on a branch to move the relevant parts over and firm up some APIs, and it can hopefully be finished and production-ready in time for 1.4. At the same time, I'll be cleaning up the South codebase to match, and will then release South 2.0 at the same time as the migration APIs are released with Django. South 2.0 will, necessarily, only support that version of Django - previous versions will have to stick with South 1.0, which will get maintenance fixes. I'll also ensure there's a reasonably clean upgrade path from 1.0 to 2.0. So, I'd love to hear your feedback and thoughts on my proposal. The mailing list thread is probably a good place for constructive comments or criticism. (Also, slides for my talk are available here, and there's also a video.)

The first bugfix release of South is now out, and there's plenty of fixes.It's been perhaps a little too long since the 0.7 release, so I'm pleased to announce that 0.7.1 has now arrived. It's really all small changes, as befits a point release - the release notes highlight the few slightly more major changes. In other news, I'll be at djangocon.eu next week, talking about both South up to now, as well as what I think the future holds. The jury's still out on what exactly will happen, but I have a few interesting proposals I'll be talking about, some of which are a bit more grand than others. If you're not going to be able to make it, don't worry, I'm sure you can all be subjected to me in two-dimensional video form later on. It's a once in a lifetime experience, mostly because few people are awake enough to try listening to it a second time. I'll also obviously be posting my proposals here and in other appropriate places, to gauge feedback.

Don't forget to consider the difference hardware can make to a systemSomewhat recently I built a new backups server at work as a proof of concept. Since it was only a proof of concept I just used the best machine available - a new Celeron desktop with 4 onboard SATA ports which I could use in software RAID. Pretty bad for a so-called server, but sufficient, I assumed, to keep up with 3 bonded 7-Mbit ADSL lines when running rdiff-backup. The proof of concept worked - the backups completed overnight when our bandwidth was cheap. Gradually, more backups were migrated to this server and the old backup server was retired. Then I noticed that backups were not finishing by 9am and costing us a considerable amount of money. I initially put this down to rdiff-backup and kicked myself for not using rsnapshot, a clever wrapper to the proven rsync binary. Closer examination, however, revealed that 80% of CPU time was allocated to disk I/O. iftop showed that the server was only pulling 2Mbit during backups. As a test, I bought a £35 4-port PCI SATA card and put it in the old backup server, an aging Pentium 4 with hyperthreading. I pulled the disks from the Celeron server and put 2 on the motherboard's only 2 SATA ports, and 2 on the PCI card. Amazingly, we then achieved a reduction of two thirds in the amount of time backups took to complete. It seems that the Northbridge in the new Celeron workstation actually couldn't keep up with 3Mbit of connectivity! Whilst I admit that rdiff-backup and rsync access the local files very heavily due to the remote binary diff algorithm, this result really surprised me. I assumed that local disk access will not be a performance issue for a task like this, but it evidently seems that this is not the case. No matter how low priority a server is, be sure to benchmark and get the right hardware for the job, because it'll only come back to haunt you later!

I’ve just read an interesting piece over at The Register on the bloated awkwardness of Visual Studio 2010, and another on the question of whether we need IDEs at all.

The latter is a difficult question for me – on the one hand, there’s a school of thought I have some sympathy with, which says that IDEs are a crutch of the feeble-minded, and allow bad programmers to kid themselves that they’re good, because they can generate lots of code automatically and hit ctrl-space if they run out of ideas.

However.

When I started programming, six years ago (!), the first three languages I used were Turbo Pascal for Windows, PHP and Visual Basic 6. TPW was a good basic language for teaching A-Level Computing, but the built-in editor was scarcely better than Notepad. I can’t remember if it had a compile-and-run button, but I seem to recall not. PHP was slightly better – once I’d worked out how to get Apache onto my Windows machine and sacrificed a chicken to get PHP talking to it – but again, no IDE out of the box, and even the relatively advanced capabilities built into the copy of DreamWeaver  (was I the only student in the country honest enough to cough up over £100 for it?) didn’t feel up to much.

VB6, though, felt magic. In retrospect, it was a horrid language, but not only could I drop controls onto a form and double-click to generate the outline of the method they’d activate, I could actually pause and resume the code while it was running! I could see the values of variables at a point in execution, and even go backwards and forwards. The completion facilities of the IDE were basic, but they were there, and they made things much faster. Writing code to automate Microsoft Office was a particular sweet spot – run the macro recorder to generate code containing roughly the API calls you were after, then drop some VB6 control flow round them, and off we go.

Later, reading Computer Science at Oxford, the practicals we did rarely stipulated an IDE, but we nearly always ended up using gedit + the relevant command-line compiler. Certainly, the existence of IDEs for Haskell was never alluded to – either the ultimate example of clever people thinking IDEs are the preserve of the feeble-minded, or the assumption that we’d be clever enough to go and look for one ourselves, depending on how silly I want to consider myself in retrospect.

I’m happy to say that Java hardly featured at all in our courses (I’m with Joel on that one), but when it did, we were told to use BlueJ, because, Mike Spivey explained, “it has only two buttons, and Eclipse has hundreds of others you don’t need”.

Given the short length of the practicals involved, I only paused to think “pish, how many buttons can it have?”, but I didn’t feel the need to find out for myself until I started writing Java for a living. He was right, there are hundreds of them. Despite which, I use Eclipse every day at work, and would never dream of trying to write code without it. It is, irrefutably, a big, bloated beast, but when you’re working on serious real-world Java, with version control, coding standards, complicated dependencies, hundreds of packages making up one program, and spend far more time reading and debugging code than writing it, you really do need the beast on your side (or so I believe).

So that’s it, then – I’ve converted to the world of IDEs? Well, not quite. The other language I use on a day-to-day basis – though mostly for pleasure rather than business – is Python. And I don’t usually use an IDE, simply bashing out code in Notepad++ on Windows, or KDevelop on Linux. OK, so KDevelop is sort-of an IDE, but it’s very lightweight.

Of course, Python being interpreted rather than compiled makes it easier to just fire up your Python program from the command line after editing it. And that, really, gives us a clue as to the only sane conclusion of the IDE debate: it’s the same as the programming language debate. There are tools (languages and IDEs) and there are jobs. Good programmers pick the best tool for the job, and for a compiled language as verbose as Java, an IDE arguably makes things faster. For Python, on the other hand, it’s not essential (IMO), but it depends on the tastes of the individual.

It’s amazing how many fewer afternoons I seem to spend hacking around on my servers these days. Perhaps I got a life; I certainly got a full-time job. I have however sorted a few long-standing bits and pieces out today…

dnorth.net is now available over IPv6

As are its various satelite sites and www.saintcolumbas.org. Sorry, no, there is no bouncing logo to reward those of you viewing them via such.

A backup system that doesn’t Totally Suck™

I’ve finally retired my creaking “run a shell script to rsync them onto my laptop when I remember (i.e. every six months)” manaul backup system in favour of an encrypted LVM partition on my home server, and rdiff-backup to make nice incremental backups of everything on a nightly basis. The instructions on how to do it are all out there on the interweb, and it’s not too difficult, fortunately. I’m a bit disappointed that backupninja doesn’t support remote rdiff-backup, but I guess I should submit a patch if it bothers me that much…meanwhile, my wrapper script seems to work just fine.

I love the Debian apitcron package, which emails you when there are updates to be installed. But what is the RHEL equivalent?For RHEL5 and above, this is really easy, just add a call to yum check-update -q in your crontab. This command will print a list of packages to be installed, and will return silently otherwise. For older versions of RHEL, which use up2date, you can use the following Python script (will need adapting not to use subprocess if you still use Python 2.3): http://wiki.maniacmartin.com/up2date This gets around the lack of a quiet parameter on up2date Simply change the top like so for Python 2.3: import os listing = os.popen("up2date -l", 'r').read()

After months of hard work, refactoring, blood, sweat, tears, and improvement, South 0.7 is ready.This release has been probably the biggest internal change in South since it first started. We've removed a lot of old code, and significantly refactored the migration-creating code to make it more extensible (as well as not being mostly in one 3000 line file) There are also a few new user-facing changes: New command names.The old startmigration was getting a little overloaded and wasn't too well-named, so it's been split into schemamigration and datamigration commands. There's also a new graphmigrations command, for those with complex dependency sets. No more issues with missing defaults. South realises you need a default for NOT NULL columns now, and prompts you if you forget to specify one as you're making the migration, instead of dying when you try to apply it. Django 1.2 and MultiDB support. As well as supporting the latest and greatest incarnation of the universe's best web framework, 0.7 also has some limited MultiDB support, in the form of a --database option (that works like the option of the same name on the new syncdb command in Django). Custom fields are no longer magically used. Instead, you now have to actually tell South about them. It's an ease-of-use regression, which I'm not too keen on - but it's needed to stop things mysteriously failing in your migrations a few weeks down the line, and there's a whole new tutorial section on it, as well as plenty of reference documentation (and IRC and the mailing list are always around to help). If you want more information, read the new long-form release notes, which goes into a bit more detail on the various changes, and mentions a few I haven't included here. There'll be a lull in development after the release as I rest, and then work will begin on the next release - 1.0*. Which features are going to appear have not yet been decided, but it's highly likely column rename support and a fixturemigration command will be among them. Anyway, why are you waiting? Go install South 0.7 now! * South has an octal numbering system.

It's that time of the year again, when a new South release rears its well-refactored, database-independent head.If you've been following the development of South at all, you'll know that 0.7 has been quite a while in the making. I've been busy working away at a big code refactor, along with Simon Law, who has gained the title of "Knowing Too Much About Migrations". However, there are some user-facing changes as well, some quite significant. While my release notes go into some detail on what's new, the major changes include: New command names.The old startmigration was getting a little overloaded and wasn't too well-named, so it's been split into schemamigration and datamigration commands. No more issues with missing defaults. South realises you need a default for NOT NULL columns now, and prompts you if you forget to specify one as you're making the migration, instead of dying when you try to apply it. Django 1.2 and MultiDB support. As well as supporting the latest and greatest incarnation of the universe's best web framework, 0.7 also has some limited MultiDB support, in the form of a --database option (that works like the option of the same name on the new syncdb command in Django). Custom fields are no longer magically used. Instead, you now have to actually tell South about them. We're working on docs to make this a bit easier - it's an ease-of-use regression, which I'm not too keen on - but it's needed to stop things mysteriously failing in your migrations a few weeks down the line. As I said above, read the new long-form release notes for much more on the release. If the release candidate proves to have sufficiently few bugs that I can fix them in time, 0.7 will be released next Thursday, the 25^th March. So, please, install the release candidate and try it out!

It’s been over a year since I deployed Django in production, and I wasn’t looking forward to it. Last time, I had a lot of trouble with mod_python, sessions and decimal objects refuising to pickle.

Thankfully, all this really seems to have grown up in the last year – mod_wsgi is now the recommended way of deploying Django in production, and following the mod_wsgi django instructions, I was in business in 20 minutes. No fuss, no mess, no drama, and best of all, using daemon mode, no noticeable performance hit when serving static files and PHP off the same Apache installation. The ability to run the django project as its own unprivileged user when using daemon mode is also real handy.

Hello internet! I am at FOSDEM 2010 in Brussels. I tried the fosdem-maemo schedule application for my Nokia N900, and decided to write an alternative app which is easier to use with my fingers, and looks more like a Maemo application.

The result is foschart. It's just something I knocked together in a few hours yesterday, but it's pretty usable already. It supports showing talks grouped by track, by room, and just in chronological order, and a list of favourites. It's all happily kinetic-scrollable, etc., and is very snappy once it's started.

There's no proper release or package yet; if you want to package it up properly, please do! But for now, apt-get install python-hildon, then copy foschart.py and schedule.xml to /opt/foschart, and foschart.desktop to /usr/share/applications/hildon. Then it should show up in your application list, and away you go. As ever, patches welcome. Enjoy!

Update:

The illustrious Jonny Lamb has made a package!

2010 will be a bad year for IPv4 – this is exactly why I designated native IPv6 support as fundamental, not merely a “nice-to-have”, when setting up Splice last July. Hats of to Bytemark for supplying IPv6 with their hosting. I’m sure the fact that it’s excluded from their SLA is something that won’t be the case in eighteen months’ time, and meanwhile, I and the rest of the crew are using it quite happily.

Footnote: those of you reaching for the comments button to sarcastically remark that this website appears to not be available over IPv6 will be pleased to hear that I intend to fix this in the immediate future.

Why I've IPv6 enabled my server and how I did itWe hear the horror stories every now and then. We're running out of IPv4 space and soon the world as we know it is going to end. One day we'll run out of addresses and computers will have to live behind the evil jail of NAT. There is another way, and that is of course IPv6. ISPs are mostly burying their heads in the sand, and won't implement it until there are more IPv6 enabled websites. Webmasters are waiting on IPv6 addresses to be handed dished out by hosting companies who are too lazy to do anything until ISPs dish out addresses to surfers. A vicious cycle indeed. Luckily, we host with Bytemark, who dish out native IPv6 addresses so we don't have to rely on hackish Sixxs tunnels. I have a /56 on my VPS. The first thing to do was to to bring up 2 addresses in /etc/network/interfaces. I tried putting a separate ipv6 stanza (the way you're supposed to do it), so I used post-up commands by placing this under iface ethe0 inet static

  #Bring up additional ipv6 addresses on same if
  up ip -6 addr add 2001:41c8:10a:200::1/56 dev eth0
  up ip -6 ro add default via fe80::1 dev eth0
  up ip -6 addr add 2001:41c8:10a:200::2/56 dev eth0

I saved myself a reboot by also running the up commands on the command line. Just as I did for v4, I planned to use the first address for everything, except lighttpd, because Apache is already using port 80. I added an extra Listen line to /etc/apache2/ports.conf

Listen [2001:41c8:10a:200::1]:80

All of my vhosts listen on *:80, so that's good. Lighttpd's server.bind syntax only supports one bind address and port, but you can bind to multiple addresses like so:

server.use-ipv6 = "enable"
server.bind = "::ffff:212.110.165.233"
$SERVER["socket"] == "[2001:41c8:10a:200::2]:80" { }

Next I turned on ejabberd's IPv6 support by adding the inet6 keyword to thgis stanza in /etc/ejabberd/ejabberd.cfg:

{listen,
[
{5222, ejabberd_c2s, [inet6, {access, c2s}, {shaper, c2s_shaper}]},
...
]}

Next up wasmy IRC bouncer, ZNC. I'd told it to explicitly bind to a certain IP address so I could have a vanity address. That needed disabling so I can connect to IPv6-only IRC servers (which to be honest isn't going to happen anytime soon.) Postfix has IPv6 support since 2.2, and i have 2.5.5 so that should just work, as it currently binds to all addresses. For good measure, i added inet_protocols=all to /etc/postfix/main.cf For Dovecot, I added listen = [::] to /etc/dovecot/dovecot.conf. Note that listen = * refers to all IPv4 only. Bytemark's hosted TinyDNS servers support IPv6 records (prefix 6 for automatic rDNS, prefix 3 otherwise), but I stupidly totally forgot about this and used this generator to cook up some AAAA records to match my A records. A little testing with the SixXS IPv6-IPv4 and IPv4-IPv6 Website Gateway, which is basically an IPv6 -> IPv4 web proxy that refuses to retrieve anything hosted on IPv4, and I confirmed everything was good to. Stop Press! Aren't we forgetting something? In keeping with the tradition set by kame, and followed by Google and many others, I needed a bouncing logo thats only shown to surfers that connect via IPv4. Lucky I had an animated gif that I'd made earlier. In Django, you can do something like

":" in request.META["REMOTE_ADDR"]

to work out if your surfer is an IPv6 surfer. IPv4 users can sneak a peak at using the SIXXS gateway See it's that easy. If your host gives you IPv6 space, then you have no excuse not to be leading the way to the move to adopting IPv6.

My e-mail still isn’t as shiny as I’d like. In particular, my use of Exim Filters to sort incoming mail into folders lacks the ability to mark messages as read (although it’s still miles ahead of the dreaded Procmail). This would be handy for high-traffic mailing lists which I don’t have time to read on a daily basis, but which I find it hard to ignore the “unread” icon next to the folders for.

One day, I should probably move to using the Dovecot LDA and its sieve implementation, which supports the “imap4flags” extension, thus allowing marking messages as read, making them turn purple in Thunderbird, and all sorts of other cool stuff. Sadly, life (or this afternoon) is too short.

In the meantime, I’ve solved the problem in the usual way I deal with life’s imperfections: gratuitious Python – http://pastebin.org/77199 run from a crojob every five minutes.

(Disclaimer: letting scratty little bits of Python anywhere near something as important as your e-mail is probably a Very Bad Idea.)

The camera/photo viewer on the N900 has a pretty nice tag cloud widget, which lets you quickly label your photos before you upload them to Flickr. (The novelty hasn't yet worn off!) But an autocompletion accident left me with a tag in the widget that I'd really prefer not to be there when I'm showing off my nice new phone to people.

I spent a happy¹ few hours trying to figure out where it gets the set of tags from. The viewer asks Tracker for the most commonly-used tags, but this tag wasn't used on any of my photos, so wasn't coming from there. In fact, it didn't appear in any of Tracker's database files! After a bit of investigation, I discovered that the photo viewer keeps its own independent set of recently-used tags, not in Tracker, but in GConf, at /apps/osso/image-viewer/recent_tags. Lest you should find yourself in my position, a quick

gconftool --set --type list --list-type=string /apps/osso/image-viewer/recent_tags '[]'

will expunged your undesired utterances from the cloud. Bug report time. Next stop: finding a tool that lets the user remove typos from the autocompletion database …

1. Grr.

My home machine, on an ADSL line, runs an OpenVPN server, so I can connect remotely and access resources (because otherwise I'd need to use IPv6, or something).  Now, the OpenVPN server pushes various routes to remote clients, because my home network is a little complex.  Today, I finally got my netbook (runing Ubuntu 9.10 Karmic (netbook remix)) connecting properly to the VPN (turns out things are happier if you set comp-lzo consistently), only to discover that network-manager was routing _everything_ over it - not ideal, given the VPN server is on the thin end of the ADSL connection...

Anyway, turns out you can fix this by modifying the VPN connection in Network Manager, going to IPv4 Settings, changing the method to "Automatic (VPN) addresses only", then going to "Routes..." and ticking the box marked "Use this connection only for resources on its network".

And now I have speedy internets again (and a working sixxs tunnel)

Earlier this year, I hacked together a stateless RSS reader called Bogroll.

It’s been doing sterling service for me at news.dnorth.net ever since. Today, I’ve sorted out a 0.2 release with the following improvements:

• Now caches etags/Last-Modified headers to avoid fetching a feed if it hasn’t changed since last time (thank you, Mark Pilgrim, for chapter 14 of Dive Into Python 3, which reminded me to be a good citizen in this regard). I was pleased to discover that the Universal Feed Parser it’s built on top of already supports gzip and deflate compression to save bandwidth.
• Now supports just one category per feed, because having articles appear in several categories just seems wrong to me
• Each category now really does contain the most recent X articles from the relevant feeds, because I’ve fixed the severely broken sort-by-date logic

A fair bit of refactoring has gone on under the hood, and the code now looks a bit more like an app and less like a ten-minute bodge. The next round will involve getting some proper unit tests in place, and possibly AJAX magic to load the articles lazily on the page.

You can download the 0.2 zip, or get the latest version from subversion if you like to live dangerously. The cool kids all seem to be using Git or Mercurial these days, but I haven’t found the need (or overcome the inertia) yet.

Enjoy. Feedback welcome to the usual address.

So the turkey is (at least partly) eaten, the mince pies are disappearing fast, the wrapping paper has been picked up off the floor and the presents played with. 2009 is done.

It’s hard to say what I’ll remember 2009 most for, because it’s been such a packed year for me. Perhaps I’ll remeber it as the last of my three happy years at Magdalen, the year I finished my degree, the year I set up a hosting co-operative with six friends, the year I moved away from home, the year I got a job or the year I took over as joint church treasurer.

It wasn’t a perfect year, of course – my personal TODO list is still 39 lines long – but never mind, I’m fairly sure there’s another year just around the corner. One item a week. What could possibly go wrong?

Happy New Year to all 3.5 of my readers (and all those following along via Facebook; do come and read the website this is all automatically imported from), and I look forward to seeing as many of you as possible in 2010.

During the planning of our Christmas card at work this year, a mad idea came up. Do we ignore mad ideas? No, we tackle them head-on.

The idea was to build an internet-controlled snow machine - you'd hit the button on the website, and watch a member of the Torchbox team get pelted with snow. When we first came up with the idea, we dismissed it as being "too complex", but after a while, we came around.

Cue three days of frantic development and phoning round to get the parts. I'll be posting a full build article, with all our source code, once we're done.

The brief summary is that we have a Django app which handles rate-limiting of snow, and tracking who has clicked the button, which then communicates with our snow machine using Artnet and DMX. Ustream is used to stream the video back to the internet.

Still, I imagine you want to see it in action, so head over to snow.torchbox.com, and have a go. It's only online 10 - 5 UK time, and only until Tuesday (we can't fill our office up with paper snow forever), but it's still very good fun, even if you don't know any of us!

Let’s be honest about this. Looking back, I should have known better. Nevertheless, as I describe the problem that ate far too many hours of my weekend, judge for yourself whether I was entirely to blame for How It Went…

The problem

A friend of mine has a laptop. It’s about five years old, and it runs Windows XP. This means, inevitably, that it’s a mess. My personal metric of measuring how rodgered a machine is by the number of icons in its system tray gave it a ten, and that’s pretty nasty. Nevertheless, until last Friday, there was nothing wrong with it that an uninstall fest followed by a defrag wouldn’t have fixed.

Enter the Internet Man

Last Friday, a chap called round to set up some broadband for my friend, who’d previously been on dial-up. I’m not going to name the ISP concerned, since it gives me more freedom to say nasty things about them, but suffice to say that they’re big enough that they really should have done better.

Their engineer fixed the simple(ish) problem with wiring which was stopping the broadband from working, and what he should have done at that point was connect the laptop to the router (less than six inches away on the same desk) using the supplied ethernet cable, and left. What he actually did was shove the supplied CD in the drive, which helpfully installed a few hundred megabytes of crapware onto the machine, then hooked it up to the broadband via wireless. Then he left.

Despite the above totally unnecessary shoving of stuff onto it, the laptop struggled manfully on (system tray count now up to 12) and seemed superficially fine.

Enter Dragon Naturally Speaking

My friend makes extensive use of Dragon NaturallySpeaking, a voice-recognition product which seems to knock the socks off everything else on the market when it comes to actually coping with different accents. And it was here that the problem first manifested itself: trying to use Dragon to voice-control Internet Explorer caused it to crash with an error message along the lines of the one described in this Microsoft knowledgebase article.

Enter the sucker, stage left

At this point, I fetched up, and agreed to see if I could fix the problem. Although the above KB article looked ideal, being the first hit on Google when I exercised the too-useful-to-be-documented Windows feature of “Ctrl-C copies the text of the active dialog box to the clipboard”, the hotfix it supplies claimed to be already present in service pack 3 of Windows XP. Just about the only virtue of this laptop was that it was fully up to date on patches and service packs, so what now?

Don’t press that button

Being rather short on ideas at this point, I decided to fire up the nearest thing broken Win32 boxes have to a magic bullet, namely System Restore. The machine refused to roll back to any of the restore points at first, but restarting into safe mode fixed that, and it was soon rolled back to the Friday, at a time before the problem occured.

Unfortunately, Dragon now seemed completely broken, giving the error message described in this support article. And no, of course there were no backups, I hadn’t taken one before I started, and my friend is no different to most of the non-geeky people I know in not backing up, except for dragging his holiday snaps onto CD once every six months.

To their credit (and they’re about the only players in this story to be awarded any), Nuance’s suggestion in the article of how to manually restore the user files for Dragon did work, after I rolled back the fatal system restore [or rather, didn't, because it didn't seem to have made a pre-restore restore point. Fortunately picking one from the Saturday seemed to work].

So, after two hours feeling my friend’s anxiety at the thought of having to retrain the speech recog from scratch – not a pleasant accompinement to the sick, swoopy feeling we get when we know we’ve just permanently erased some irreplaceable data – we were back at square one with the original problem. One last shot in  the dark, disabling the ISP’s nasty extensions to IE, seemed to fix the issue.

So who do we blame here? Laptop vendors, for selling machines so laden with rubbish before they even leave the factory that DLL hell seems assured the minute they meet with real life usage? Or, we could blame the idiots who seem to write the nasty unsigned drivers for most hardware on the market. We can definitely blame the ISP’s engineer for installing the crapware, but perhaps he’d been trained to, and anyway, why do ISPs think we need a CD full of crap to supplement the TCP/IP standard that’s been around for several decades? Is it really asking too much of Johnny User to plug in a cable or enter some Wifi passwords in to the applet that’s sodding well supplied with Windows, thus making the poorly writtten replacements from laptop manufacturers and ISPs alike completely superfluous? We could also blame Microsoft for making system restore not clever enough to cope with software like Dragon. Or possibly blame Nuance for not fixing or documenting what has apparently been a known incompatability for several versions of Dragon*.

Certainly, we can blame me. I clearly need to have “I will not agree to even slightly ‘fix’ someone else’s computer without taking a full disk-image of it first” tattooed across my forehead. I also clearly need to reimmerse myself in the happy world of properly written software which I’m lucky enough to earn a living in and try to forget the horrors of the last 48 hours.

* The most useful reference I could find on Google was this page. The Nuance KB doesn’t mention system restore. Then again, perhaps their customers simply don’t know this is the cause of the issue, or don’t get round to reporting it. It’s not like I have.

A couple of days ago, I released version 0.2.1 of Bustle, someone's favourite D-Bus profiler. As the version number suggests, there aren't really any big new features; most of the changes just make it a bit nicer to use, like showing you all the bus names a service owns, ellipsizing strings, a slightly less spartan UI, etc. Having finally gotten around to cutting a release, I've started wondering what to work on next. There are various small things I have in mind, such as searching, filtering, integrating the various statistic tools (bustle-time and friends) into the UI, and so on, but it'd be nice to have a larger goal to work towards.

One recurring feature request is the ability to see messages' arguments. This isn't currently possible because the simple plain-text logs produced by the monitor (which is a variation on the theme of dbus-monitor --profile) only includes the message header. I've thought for a while that the right thing to do would be to log the raw dbus messages, together with a timestamp, but wasn't sure what the files would look like. (Maybe shove the timestamps into the message headers?) Rob had a nice idea: why not log to pcap files? This avoids inventing a new format—the UI would just use libpcap and feed each message through the dbus parser—and would also let you look at the logs in WireShark, if you're into that kind of thing. I'm hoping to find some time to give this a shot soon. (Maybe on a cold Christmas evening, in front of a fire?)

In the meantime, have a peek at what your D-Bus-using applications are up to, and let me know what's missing!

Or "why your mail from hotmail may not reach me"Lately I've been having a major problem with spam sent from hotmail addresses using hotmail's own SMTP servers. This has evaded sanesecurity's signatures and SpamAssassin, even though I've fed about 100 of these messages to sa-learn and also given some to steveb at sanesecurity. Some of these new spams I've been getting even have a negative SpamAssassin score! They are mostly random waffle (not 411, viagra or sex), with a hyperlink embedded. My email address is never in the To: or Cc: fields.. I have tried contacting Hotmail in the past about issues such as this, and its like talking to a brick wall. Previously, the most obvious spam was rejected by our SMTP servers at SMTP-time, and that which was likely to be spam, but not certain, was delivered to a Junk folder. Now, I am going to configure my server to accept then silently drop all email which is routed via a hotmail.com SMTP server where my name is not in the To: or Cc: fields. (Technical issues with the way the SMTP servers are setup prevent me from rejecting it at SMTP-time at present.) This means that if you use Windows Live mail to Bcc me an email then I will no longer receive it. You will not receive an automated reply, in the interests of reducing backscatter.

I’ve long been of the opinion that Firefox and Thunderbird are a bit like democracy: aclaimed far and wide as major achievements and bastions of a civilised society, but actually, honestly, a bit crap in many ways. Sadly, we’re stuck with all three until someone manages to come up with some compelling alternatives.

If you’re reaching for your e-mail client at this point to tell me I’m being unduly harsh, look me in the eye and tell me you think the way Firefox cheerfully caches DNS lookups and ignores such things as TTLs is a good idea. Or, even harder, give me one good reason why Thunderbird doesn’t check all IMAP folders for new messages by default. You can’t; in both cases it’s a disgrace.

All this being the case, I wasn’t what you’d call hopeful when, last Sunday, I had to write some e-mails on a train. And to do so, I needed to refer to some other e-mails in my Inbox. Since said train lacked anything as useful as a wireless internet service*, I’d need some sort of offline IMAP facility**.

As I bashed Thunderbird offline IMAP into Google, I was expecting a half-baked plugin at best, and “can’t be done” at worst. What I was actually very pleasantly surprised to find is that this functionality is built into Thunderbird.

What astonished me even more is that it actually worked. Faultlessly. So perhaps democracy can be salvaged after all.

* And, let’s be honest, because I still haven’t got organised and bought a phone with internet capabilities.

** Nobody uses POP3 in the twenty-first century, right?

In my neverending quest to save the time of those using RDBMSen, South 0.6.2 is released.

It mostly contains bugfixes, but there's now support for extending introspection to third-party apps without having to edit their source.

I'll probably start working full steam on South 0.7 soon - there's a refactor of the migration engine (mostly done), of startmigration (not started), and a few changes to command names to make them nicer (the old ones will still work), and to the behaviour of default values.

I'm hoping 0.7 will be the last release before 1.0, at which point I will claim I've been working in an octal version numbering sequence all along.

An oft-requested feature in Empathy and Telepathy is support for OTR (Off The Record) encryption of messages, interoperating with the OTR plugin for Pidgin and other popular IM clients. We've been resisting implementing it so far, mainly because we think there are better ways to do end-to-end encryption of messages and audio and video calls over XMPP, which we hope to implement in the not too distant future.

However, a nice aspect of OTR as compared to other encryption solutions is that it allows you to plausibly deny having taken part in a conversation. We believe this to be an example of a wider trend towards deniability on the internet, a position which is backed up by the growing popularity of Tor, and by several modern browsers allowing you to cover your browsing tracks out of the box.

As a result, we've been working hard to help secure your privacy while you're using Empathy. We've had to do this quietly for various legal reasons, but we're proud to announce Empathy's new Private Mode. When enabled, your contact list will be anonymized, as will your entry on your contacts'. Thus, you can conduct conversations with anyone without fear of repercussions from their discovering your identity, or of anyone else knowing the conversation took place:

It's not obvious how to bring these privacy benefits to Jingle video calls. We came up with a technique we refer to as Kitten Secrecy (patents pending in all relevant countries), and managed to integrate it with Empathy with the help of our friends at Collabora Multimedia, who constructed a fantastic GStreamer element using only two leaky queues!

We think the results speak for themselves. The source is not quite ready for release yet, but (lawyers permitting) we hope it'll be public by the end of the month. Hope you can wait until then!

— the Telepathy and Empathy teams

---

If you're about to leave an angry comment:

At the Boston Gnome summit, Robert McQueen, Sjoerd Simons and I sketched out a plan for the API for end-to-end encryption of communications (implemented using XTLS, OTR or anything else) and how we'd implement this API for OTR. Work's just started on a challenge-response authentication API, which is a prerequisite. Stay tuned; or, jump onto the Telepathy list or #telepathy on Freenode if you're interested in helping out!

Dear lazyweb, does anyone else find that Spotify mysteriously hangs at the “logging in” stage when run under Wine? This suddenly happend on both my Linux boxes after it had been working fine for weeks.

Various posts on the Ubuntu forums suggested firewall issues, but nothing had changed, and Spotify continue to work on Windows on both machines.

In the end, I applied the nuclear fix of

$ mv ~/.wine ~/.wine-with-broken-spotify

and reinstalling Spotify from scratch, which worked. Might not be so handy for anyone with other apps under Wine, though.

Jason's hands were tired after typing two days of excellent notes on sessions at the Gnome Summit, so I took over writing up the Telepathy BOF (which was largely about Telepathy integration in Gnome Games).

Gnome Games's tubes code is broken because of Empathy moving to Mission Control 5, which broke the contact chooser they were using (which used Mission Control 4). A Canonical person (your scribe did not catch who it was) has written a contact selector in C which just uses telepathy-glib, which Gnome Games will use and then start working again. This widget could form the basis of the long-anticipated telepathy-gtk.

Rob pointed out that it's currently a bit of pain to request a channel for yourself: you can't just call one D-Bus method and get a channel back, you also have to implement a Client.Handler object on which MC will call the HandleChannels() method. Sjoerd noted that Empathy has helper code for doing this in simple cases, which could be moved to tp-glib (it's under the LGPL).

Jason wants a way to share high scores with your contacts. (Digression about a gnome-games high score server on gnome.org ensued, the notes of which your scribe lost in a kernel panic. One main point is that global high scores end up just featuring incredibly good scores and people setting their name to obscenities.) Jason wonders if g-g could publish your high scores to your contacts in the background?

• Rob and Will suggested that in principle this could use PEP on XMPP, which allows you to publish a blob of information to your XMPP server and be notified when your contacts' blobs change. (This is how Geolocation works in Empathy, for instance.) Unfortunately Google's server doesn't support PEP, so most people wouldn't be able to use this; plus, it would need an extension in Gabble.
• Another way to do it would be to initiate file transfers behind the scenes.
• Rob suggested using tubes. Gnome Games would come with a tiny daemon that handles tubes for the "org.gnome.games.highscores" service, serving up your high scores to anyone who asks. Then, when you open a high scores table, it would use ContactCapabilities to discover which of your online contacts support that service, and then ask them for their latest high scores (with a cache and a rate limit, obviously). It could show a throbber while it's doing this. Jason thinks this sounds like a good idea. Prerequisites: you need the latest MC and Gabble 0.9 in order for ContactCapabilities to work, but that's it.

J5 wondered if any g-g people are documenting how they're using tubes, because he was always confused by them, and he reckons this is a very important thing for app developers. Rob suggested pushing this into tp-book, and Sjoerd noted that Danielle has a helper which lets you say "give me a stream to this person" and get a GIOChannel back branch with methods to convert between telepathy address GValues and GSocketAddresses, which could conceivably be extended to set up a socket automatically. J5 thinks that if patterns for using tubes were really well documented, people would jump on the chance to use them.

Jason mentioned that people were discussing having a help option which jumps you to #gnome-games-$lang. J5 said that Ximian tried that, but found that people would just end up in empty chatrooms or paste goatse at each other. Integration with DevHelp would be nice to let people post examples etc.

J5 suggested that another good way to improve documentation is to make writing it a requirement for SoC. Rob noted that Telepathy hackers know that you need to use, eg., a Handler and a Tube, but it's hard for people really immersed in the stack to remember which prerequisites people need to learn in order to understand that stuff. Sandy said they'd been discussing documenting requirements and standards for SoC students, and thinks it's a great idea to ask students to blog stream-of-consciousness "this is what i did" updates. People have to make sure that they do this as they go along, because you forget the learning process after a few weeks.

Advantage of peer-to-peer high scores: you don't get the problem of one incredibly good person dominating. ajaxxx suggested that you could make high scores decay over time, or once you reach a particular level, to address this problem.

Tetrinet is latency-sensitive: will that be a problem with Telepathy, particularly using MUC tubes? Rob said just try it and if it's too slow it'll get faster as the implementation of Tubes improves. Sjoerd noted that for tetrinet you probably want to just export the Tetrinet server over a multi-user stream tube, rather than using d-tubes.

Telepathy should use UPnP to make FT and tubes fast in more cases. This is on the TODO list.

I'm just getting ready to fly away to Boston for the Gnome Summit. I'm looking forward to meeting people and seeing MIT, as well as getting the chance to spend more than a few hours in Boston (unlike every other time I've been there).

Inevitably, I haven't been organised enough to propose a Telepathy- or Empathy-related session, but Rob McQueen, Sjoerd Simons, Andres Salomon, Dafydd Harries, Shaun McCance (when he's not busy running a pair of interesting-sounding documentation sessions!) and myself will be around if people are interested; maybe something will coalesce. If window manager theming is more your kind of bag, Thomas Thurman's running a session on CSS in Metacity/Mutter. It'd be great to talk about integrating IM with the Gnome Shell; Moblin's people panel and many parts of Maemo make interesting use of Telepathy, and it'd be nice to have something similar on the desktop.

Speaking of Maemo, going to Boston means I'm not at the Maemo Summit in Amsterdam, which is a real shame: I'd love to meet more of the Maemo community, hear what people have up their sleeves for the N900, and discuss how Telepathy could help. Happily, Marco Barisione's giving a talk about how Telepathy's used on Maemo, and how you can use it too; relatedly, Travis Reitter and Mathias Hasselmann will speak about the address book, one of the heaviest users of Telepathy. Also, Marc Ordinas i Llopis is hosting a BoF on extending the (frankly stunning) Hildon desktop, and Ian Monroe is giving a talk with Sergiy Dubovik about preparing Qt4 applications for Fremantle and Harmattan. I hear Philippe Kalaf is also floating around somewhere. ☺

See you in (the wrong) Cambridge!

In my opinion, there's just not enough Subversion servers on the internet.

That's an opinion you'll probably never hear me express - I much rather everyone move over to a DVCS (*ahem* mercurial) - but Subversion has, some would say quite rightly, earned its place as the dominant VCS for nearly every part of the IT community (apart from big, lumbering, financial companies).

Recently, I've been slowly switching away from subversion for my personal projects into Mercurial. This, in itself, a painless process, but with my public projects - particularly South - this has the unintended effect of slightly alienating some of my users. Most are alright with adapting - after all, it's quickly becoming the norm to use a DVCS - but there's still a few people left out.

One of the particular issues I have is with svn:externals. A lot of apps - including some we have at work - rely on svn:externals to pull in external dependencies into a libs folder along with the project itself. Externals is one of the few features of subversion that I thought was pretty much perfect, and it was sad to see my move to Mercurial break it.

This week, at DjangoCon, Chris Wanstrath did a nice talk on DVCSen. One of the questions led him onto hg-git - the awesome git backend plugin for Mercurial, that the GitHub guys wrote - and how they first investigated the idea of an svn gateway to expose their repositories transparently to subversion users. From what I gathered, subversion's wire format proved too tricky to deal with, and so they turned elsewhere.

However, the idea of a subversion gateway intrigued me greatly. What better way to transparently serve South so people can still pull it using svn:externals while still developing in Mercurial? With that in mind, I started looking into Subversion's wire protocols and their delta format (svndiff) sometime yesterday afternoon.

After some digging, reading an obscure academic paper and liberal application of Wireshark, I grew confident enough that I could at least implement something. A day later, and I'd like to present the very first version of what I'm calling Heechee (if you get the pun, ten nerd points).

Heechee is a transparent mercurial-as-subversion gateway. It serves a Mercurial repository as a Subversion WebDAV-based repository. It's still in the early stages, but at the moment it will serve its own mercurial repository to subversion in such a way that you can check out the repository, and update to various revisions within it.

You can check it out at BitBucket. It's pretty alpha code, and make sure you have the dependencies mentioned in the README, but it works, which greatly surprises me. I plan to much improve the code to support more 'advanced' features, like being able to do more than checkout and update, as well as exposing tags and branches correctly. There's even the chance I'll stick Git support in, when I've had a play with Dulwich.

Still, hopefully I'll soon have it so it can serve South's code via svn, and thus restore harmony to my land of VCS users.

So, please, take the code and have a play with it. It's not formally licensed and released yet, but it will probably be Apache licensed. Feedback and suggestions for features, as well as how to linearise trees into svn histories, much welcomed!

Have verbal nagios alerts with festivalHere's a fragile script I cooked up to do it Verbal Nagios

South 0.6.1 is now available for public consumption; it fixes quite a few bugs.

If you want to see what's been fixed, have a gander at the Release Notes - there's a few significant bugs that have been fixed, including problems with SQLite, GeoDjango, and M2M fields.

Sorry for the short but sweet post - I'm in the process of writing my talk for DjangoCon! I'm giving a talk about South, with both a good grounding for those who haven't grokked migrations yet, as well as some tips on more advanced usage. Come watch it on Tuesday morning!

A script to automatically generate the shell prompt colour from the username and hostnameI like to have a different coloured shell prompt for every PC, so I don't end up executing commands on the wrong machine. Previously, I have edited .bashrc manually on every new machine, but it would be nice if this was done automatically when I check out my dot files from the repository onto the machine. If you make the following changes to your .bashrc, the colour and font-weight of the shell prompt will be chosen from a hash of your current username and the machines' hostname. My terminals have a black background. If yours have white, you might want to remove the white/yellow options and add black and brown instead, so the text shows up. Find if [ "$color_prompt" = yes ]; then change the next line to PS1='${debian_chroot:+($debian_chroot)}\[\033[`color_from_hostname`m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ ' Then, at the end of the file add: function color_from_hostname { hash=`(echo $USER; hostname) | md5sum | awk '{print $1}'` case ${hash#${hash%?}} in "0") c="1;34" ;; # Light blue "1") c="0;34" ;; # Dark blue "2") c="1;32" ;; # Light green "3") c="0;32" ;; # Dark green "4") c="1;36" ;; # Light cyan "5") c="0;36" ;; # Dark cyan "6") c="1;31" ;; # Light red "7") c="0;31" ;; # Dark red "8") c="1;35" ;; # Light purple "9") c="0;35" ;; # Dark purple "a") c="1;33" ;; # Light yellow "b") c="0;33" ;; # Dark yellow "c") c="1;37" ;; # Light white "d") c="0;37" ;; # Dark white # Run out of colours. Lets have some repeats "e") c="0;34" ;; "f") c="1;32" ;; esac echo $c }

Finally, Debian now does this when you /etc/init.d/apache2 reload, rather than silently failing and killing your websites.For a long time, I've been meaning to make a patch to the force-reload part of /etc/init.d/apache2, so that it performs an apache2 -t, to test if the new configuration files have valid syntax, and aborts the reload if there is an error, rather than silently killing your websites dead in their tracks. However, it seems that Debian or upstream have beaten me to it, as my init file (2.2.9-10+lenny2) seems to have this feature. However, I don't seem to test the syntax of a conf file without reloading Apache. I get this error when running apache2 -t: apache2: bad user name ${APACHE_RUN_USER} The following command does the right thing though: /usr/sbin/apache2ctl configtest

Some hints about using Backupninja to backup Debian servers to an Ubuntu boxAndrew and I currently use Backupninja for backing up our servers and laptops to our Ubuntu PVR, because this is the only machine at home thats guaranteed to be on 24-7. Backupninja was recommended to us by Michael and David. It runs on the servers that you want to back up, and uses a conf.d directory to store tasks, such including pgsql dumps which save to /var/backups, and then there is a final task which executes rdiff-backup to push over the files. As far as I can tell, rdiff-backup uses a combination of diffs, symlinks and rsync to have a snapshot history yet only use the bandwidth and disk space that a normal rsync would use. Backupninja insists that the version of rdiff-backup is the same at both ends, and although you can override it with the ignore-version conf file entry, I'd rather not as chaos and unrecoverable backups would probably follow. Since we had multiple Debian servers to be backed up, and only one one Ubuntu recipient, we chose to leave the Ubuntu boxes with the stock lenny version of rdiff-backup and change the Ubuntu version to match. (Also, rdiff-backup is not in lenny-backports) At the time of writing, Debian Lenny had v1.2.5 and Ubuntu Jaunty v1.2.7. We installed rdiff-backup_1.2.5-1build1_i386.deb on the Jaunty box but when the cron ran, got this lovely error: Fatal: rdiff-backup does not have the same version at the source and at the destination. After was not actually caused by the rdiff-backups having different versions, but by Ubuntu Bug 345086. Until v1.2.7, Ubuntu packaged rdiff-backup wrong. It installs to /usr/local/bin, not to /usr/bin as on Debian. As a result the Debian servers couldn't find the binary when connecting over ssh. I could have rebuilt the v1.2.5 deb from source so that it installs to /usr/bin, but in the end I settled on ln -s /usr/local/bin/rdiff-backup /usr/bin/rdiff-backup I've also filed a bug against Backupninja, with a patch so that it checks ssh's exit value and gives a more appropriate error if it can't find rdiff-backup in the right place on the remote server.

I've now made the move from svn to mercurial, so my repositories can now be found hereMercurial is the natural successor to svn - its commands are a superset so you don't have to relearn anything to get the same functioanlity. Also, it is distributed so you can check in changes offline to your local hg (Hg is the periodic table symbol for mercury) and then check them in remotely at a later time. I converted my svn projects to mercurial ones using this guide (I recommend you do it on your repo server or else it makes loads of connections and takes ages) and also installed hgweb using this guide

Need a Windows Vista DVD for its recovery mode, but your system manufacturor didn't give you one?Then nip over to Neosmart. They have torrents of the Windows Vista DVDs stripped of all the files that the installer uses, so the discs are only usable as recovery discs. Very useful if you mess up your Windows bootloader...

It is with great pleasure (and a measurable sense of relief) that I announce the release of South 0.6, a new release bringing quite a few new features, although most of them aren't immediately obvious.

Perhaps the biggest change is the move from parsing source files to get the field definitions (ugh, I know) to introspecting the fields to get their definitions. As well as being more stable and making my skin crawl less, this also means that dynamically-created models and fields are now fully supported.

South still falls back on the parser module for custom fields if it has to - since it doesn't know how to introspect them - but we encourage you to implement a South-readable interface for custom fields; see our wiki page explaining how to do that.

The other noticeable change is that the migrate command is much zippier, thanks mostly to the hard work and lazy-evaluation-patches of Juan and Simon from Akoha. Note that the SQL for migrations is no longer printed out as they're running any more - it was getting a bit verbose - but you can re-enable it with --verbosity=2. You can also completely shut South up using --verbosity=0, if you desire.

For easier conversion, there's a new one-shot convert_to_south command - take care to read the wiki page on that, however, as the caveat is that you must run a migrate --fake on any other computers running the same codebase. We're going to make that a bit more intuitive for the next release.

The release notes contain the rest of the changes - some big, some small - and remember, we've switched to Mercurial for revision control, so if you're still running off Subversion please change your checkouts.

After a long few months, during which I've increased my recommendations to run off trunk, South 0.6 is almost here.

This release contains a whole bunch of new features - more of which will be written up on the Release Notes page soon - but in the meantime, I've decided to adopt a more usual release process.

This means two things: First, I'm releasing a release candidate of South 0.6 for general testing and final critical-bug reporting; please see the Download page and follow the instructions to check out the 'stableish' tag, or if you already have a checkout please 'hg pull' and 'hg up' it.

The Git mirror is also, for once, up to date, and there's also a tar.gz available for those who don't fancy either DVCS.

The second change is that I'll actually maintain a stable branch from now on as well, much like Django. This means there will be a 0.6 branch for any fixes that are easy or trivial to backport, and in the unlikely case someone should manage to conceive of a security vulnerability in a migrations library fixes for that will go in too. This should hopefully mean you and your production servers can track a real release number and not miss out on bugfixes.

Note that a few of the planned features haven't made it into this release in time, but I felt I had to get something out there. My much-vaunted fixture migration didn't make it in, but introspection, the major feature, did.

Still, please, test the release! If things look good, I'll officially tag and release 0.6 early next week.

Just installed Ubuntu Jaunty fresh onto a new ext4 partition on my Advent 4211 netbook. Am delighted to report that WiFi, suspend and hibernate all work flawlessly out of the box, so I can finally stop using Windows on this machine.

It’s amazing what sort of shrinkage – both in size and price – you can miss if you’re not buying in a particular market. Yesterday, out for lunch with some friends, one of them showed off his latest toy, a 2GB USB drive. Which, as you can see even in these amateurish photos, is similar in size to an SD card:

It’s called the Tuff ‘n’ Tiny, apparently, and if you live in the UK, Ryman are selling the 2GB version online for £5.99 and two for a tenner on the high street.

I can also confirm that mine has partitioned quite happily into two FAT32 partitions, and, with prodding, boots Linux off one of them [the second. I had to use usb-creator, followed by install-mbr to get the bootloader to work]. I don’t think I’ll be following John’s suggestion of RAIDing two together just yet, but who knows…

People often waffle about whether Linux is ready for the desktop, so I thought I'd examine it the other way around. Be warned there is some humour intended here.I recently started working for a company which has a lot of Windows on its network, and although I could install any OS on my workstation, I decided to install Windows because Windows it allows Outlook and nicer integrating with the corporate Exchange server and other things such as easier remote management of Windows servers. Also, I thought that it would be less troublesome to use the same platform as my colleagues. Before I begin, you should note that I have used Windows only rarely in the past 3 years and am very used to the Linux way of doing things, but I don't think that this makes the examination unfair as most articles examining the suitability of Linux for the desktop are written by people who are used to Windows and assume that everyone else is. If they can be biased, then so can I :) No central repository: This makes installing software a pain as you have to visit each vendor's website or find their CD to install software. Dependencies aren't automatically installed, such as .net framework. Valve have proved with Steam that paid-for repositories can work for games, so why isn't there one for other software No middle click paste: Extremely annoying if you expect it to be there No SSH client out of the box: Linux distributions usually include some sort of program for accessing Windows servers over RDP, but Windows users are expected to find third-party software Random errors on clean installs: On a fresh install, Microsoft Virtual Server 2005 installed then moaned with a seemingly random error code. VirtualBox's Bridged Networking failed. Useless scripting languages out of the box: Windows machines don't include Perl or Python out of the box, only Batch scripting and VBScript (eeugh) More prone to viruses and crapware: Windows machines always need third-party resource-hogging virus scanners and firewalls Licensing overhead: Windows costs money and bugs you for a serial key on installation, which is an added hassle So, Windows fans, maybe you should consider looking in the mirror before laying into Linux. Windows felt much clunkier and awkward to use than Ubuntu to me, and as such I am declaring that from my point of view, Windows is not yet sufficiently polished to be deemed suitable for use on the Desktop

Just over 18 months ago, I signed up for a 256slice from Slicehost to host this website, my email, etc. Later this week, I shall be shutting the machine down and cancelling my account with them.

For the record, this has nothing to do with the level of service I’ve received from them – I’ve always found their support team quick to respond and helpful, and their articles site and wiki are both very handy.

However, a combination of fluctuations of the pound against the dollar, a surge in demand for RAM by my applications and sites, and my getting increasingly fed up with transatlantic ping times of 130ms meant the machine was becoming unfit for purpose and overloaded.

Thus, I have now found a reasonably cheap way to bring “my stuff” home to hosting in the UK. About which, I will be writing more shortly!

So goodbye, Slicehost, and keep up the good work. I’d certainly recommend you to anyone who lives in America and needs a VM.

And Ubuntu gets seven out of ten

Last week, it was clear my parents’ ageing XP box needed some attention – in fact, let’s be honest, it had been crying out for a zap-and-reload since last year.

This time, though, I was determined. With my sister safely moved off onto her new laptop, there should be no reason not to migrate the box to Ubuntu. The only things in use on it were Firefox, Thunderbird and Openoffice…so how hard could it be?

Five days on, I’m in a position to report that the answer is “harder than it should have been” – here’s what happened:

Problem 1: Audio CD autoplay

Dad wanted the machine to start playing his CDs automatically when he put them in, much like Windows does. Seemed perfectly reasonable to me, as I blithely assured him it’d be a matter of a few mouse clicks.

It wasn’t.

Much Googling and swearing later, this hack (fifth comment down) seems to do the necessary, though I had to bump the sleep interval to six seconds.

Problem 2: No matter how close KPatience comes, it’s not “real FreeCell”

…largely because, despite generating the same deals from the same numbers as the Windows version, it counts every loss of the same game number as a loss in the stats, whereas the Microsoft one counts them as one loss.

Fortunately, freecell.exe runs fine under wine, so copy it across and save yourself the bother.

In conclusion

Other than the above minor niggles, all seems well, and hopefully the machine should now do a few more years’ trouble-free duty.

So, as the title of this post says, Windows XP has now been deposed as my OS of choice for “real people”. Whoever would’a thought it?

Update, 18 July 2009

I forgot to say that probably the most impressive feature of the switch was that Ubuntu flawlessly detected and printed a test page on our trusty old Epson Stylus C62. That’s the sort of slickness we like to see.

Hello, internet! I'm at the Gran Canaria Desktop Summit. I'm a big fan of this "conference beside a beach" concept: interesting people, talks and projects, in the sun beside the Atlantic. There are a *lot* of Collabora people here; the head-count on the roof of the hotel last night exceeded 20, with still more people arriving today (and others stuck in Madrid). As ever, it's good to catch up with colleagues and others!

Bringing GUADEC and aKademy together seems to have been a partial success. It’s interesting to learn about projects from the other side of the fence, and I've had some productive conversations about the state of using Telepathy in KDE. That said, it’s a bit sad that so much of the cross-desktop schedule seemed to be a talk about a GNOME implementation of a concept, followed by a KDE implementation of the same idea. Maybe putting the duplicated effort in people's faces could help improve matters, but that seems unlikely.

Earlier today I gave a talk about profiling and optimizing D-Bus APIs (grab the slides!), including Bustle, a D-Bus visualization and profiling tool I've written to help visualize bus activity and find issues. I think it went quite well, and it lead to some interesting questions and discussions afterwards. In the talk, I mentioned a few examples of sub-ideal API design in other frameworks, but largely focused on the problems we've found in the Telepathy API, and how we’ve gone about fixing them. A couple of people mentioned that this might have reinforced the impression that Telepathy is really hard to use, which was not what I intended at all! On the contrary, the improvements I talked about have made Telepathy a lot more developer-friendly than it was when I first got involved: both the D-Bus API (if you ignore the deprecated bits) and the higher-level client-side convenience libraries (telepathy-glib and telepathy-qt4) have become significantly easier to use. It's telling that a lot of the changes that have made developers' lives easier have also made the D-Bus API more efficient, sane, and correct.

Of course, there's still room for improvement. As I mentioned, contact lists are quite awkward to work with, which is a shame because most Telepathy UIs would like to use contact lists. :-) libempathy-gtk has a couple of contact list widgets you could reuse, some of which I'd like to see in a smaller, API-stable libtelepathy-gtk; I hear that there's talk to extract the Python contact list widget that's being written for GNOME Games to a library that other Pythonic Telepathy applications (such as pyhallebard for which I can't find a website right now — thanks Anonymous!) could use.

In other news, Tracker 0.7 sounds (among other things) like a potential solution to the silly unfortunate situation where every music player uses its own library database, and thus your DAAP server either needs a separate database or to be built into your music player. Mojito looks pretty interesting; I'd love to see a UI for it on my desktop. It annoys me that I have to check so many disjoint sources at the moment. It might be nice if it could show me blog post titles from my LiveJournal or Facebook contacts; I suppose I should start hacking. :)

It's about time I posted an update on what's going on with South development, so here it is.

Firstly, there hasn't been much work done in the last few weeks, due to exams and there being a Ball to help run. Now that's over, development will crack down. 0.6 is going to focus on the following new features:

• Less parsing, more introspection. The default way of getting fields' definitions will be to introspect them; the implementation of this is pretty much complete already, and can be found in the 'noparsing' Mercurial branch. Custom fields (which cannot be introspected withour prior knowledge) have two options; provide a south_field_triple method to tell us their definition (preferred way), or as a fallback the parser will be used. This feature solves many issues, including dynamic models, and also doesn't lose the ability to work with most custom fields right from the start.
• Reverse dependencies. Just brought up by one of the Ella developers, this is a good idea that should have been in South a while back. You'll be able to specify 'this other migration depends on us' as well as 'we depend on that other migration' - this helps pluggable apps around a common core.
• Fixture migration. Since I know this is going to bite me in the ass at some point, I'll be rolling in the fixture-migration someone mentioned at EuroDjangoCon (uses the test runner to make a temporary database, loads an old fixture set, an old schema, migrates, dumps the new fixture). I haven't fully tested this yet, so it might not make it for reasons of impossibility, but I'm reasonably confident.
• Signals. There will be post-migrate and pre-migrate signals added to help developers add in custom code that needs to be run after migrations or before migrations (the latter helps people trying to carry out a deployable app rename, for example)

As well as those new features, there's also around 25 bugs that need fixing, and will be tackled soon, hopefully. I aim to get a beta release of these out in the next month, especially in the light of South's allegedly high penetration rate.

The last piece of news is that I'm moving my main development efforts to Mercurial (and specifically BitBucket), and only pushing mostly-stable trunk phases to Subversion. There's a github mirror of whatever hg-git deems to be main development, but that's for convenience only, and because I can do it in one command.

Fixing an ext3 partition that has corrupt superblocks and won't mount because the kernel thinks its ext4.

Yesterday, my USB ext3 500GB disk complained that it could not write any files because it was full. I deleted a few files, then had a look at df -h, which claimed i was using 16TB of a 16TB drive. I unmounted the drive, and then replugged it in. This time it did not auto-mount.

It turns out that some of the superblocks had become corrupted. fsck wouldn't run because it couldn't read the superblock. No worries, I thought. I'll just use dumpe2fs to tell me where the backup superblocks are and pass one of these as an argument with -b to e2fsck to force it to use one that was still intact. Only dumpe2fs failed with an error that I forgot to make a note of. Also, gparted claimed that the partition was ext4.

In the end I ran mkfs.ext3 -n /dev/sdb1. -n only displays what would be done if the disk was formatted, but does not actually format. One of the things it does display though, is the deterministic locations of where it would create the superblocks for a partition this size. Armed with this information, I could then try them one in turn with the -b switch on e2fsck until I stumbled across one that was intact. fsck found an awful lot of errors (probably because the disk was heavily in use when the problem occured), but I seem to have the majority of my data intact.

The moral of the story is to always have backups! You may not be as lucky as me.

I guess I might as well park these here as anywhere else:

Some hilarity courtesy of bash.org: #693046, #9214, #105199, #626249, #610952

Funny source code comments – “throw up”, snigger. Nothing like Java-based humour.

Batteries Feel Included: 307 – it’s a bit strange, but it made me nearly fall off my chair laughing

A genius article from 1995 explaining the trouble with the internet.

This post brought to you by the department of “I’ve finished finals and have absolutely naff all to do for the next week”.

I had a lot of interesting chats with people about South last week at EuroDjangoCon, and several eyebrows have been raised at me both parsing models and then storing their definitions as dicts.

While this is a design that is partially born of backwards compatibility, it was mostly a conscious design choice I had to make, and so I'd like to now describe why I made it.

Firstly, for those who aren't aware, South's magic autodetection and auto-writing-migration code is based off of a file called 'modelsparser.py', which does what it says on the tin; it opens models.py, parses it, and extracts definitions directly.

Now, my first response to this is also 'ew', but there's a reason for it: custom fields. If it wasn't for those, introspection of the existing fields would work just fine, given a long list of special cases (I in fact have a file called modelsinspector.py sitting around that does just that - if you're interested, it's in the 'noparsing' branch on BitBucket. It doesn't do much, but it proves the concept).

However, with custom fields in the game, you have two options: hope they follow Django conventions and try to introspect common options, or put the onus on custom field authors to add a hook that makes it spit out a definition. While I would obviously choose the second one (possibly added to some of the first), it simply doesn't provide the drop-in compatability I want South to provide.

However, as Alex Gaynor and several others have surmised, I could just pickle everything and stick that at the bottom. No introspection needed, no weird namespace errors because the models dictionary is simply a collection of eval-able strings.

However, I don't like pickle (the Python kind; the delicious food kind, on the other hand, is more than welcome in my household). The security issues don't matter here - after all, we're passing raw strings to eval instead - but it has two major features I don't like. Firstly, it's not at all human-readable - and while I don't imagine people editing their migrations by hand, I like to be able to see what is going on, and I like the fact that South still doesn't depend on you using startmigration - a design choice I'm very proud of.

Secondly, I've been burned by pickles in the past. If you remove a module that a pickled thing uses, pickle will crash and burn, and if you've not done them right, you won't know quite where the error happened. (For the purposes of argument, of course, I'm assuming I'd do it right, so this isn't really a point. Still...)

There's yet another option (see, my job isn't easy) - custom-pickling, which Alex suggested last Wednesday; it consists of roughly freezing the __dict__ of the field. This has some advantages - it's more human-readable - but you get the errors similar to my current approach (where datetime.datetime wasn't unfreezing if people didn't have datetime imported).

My main gripe, with both this and pickle, is that the internals of fields aren't guaranteed to be consistent between releases of Django, just the external API, and so South uses a method that lets it use the external API - in my mind, this is less hassle, since that's the thing that's really not going to change very much (and even if it does, you can easily see what South is trying to call, and run that in a shell yourself to see where the problem lies).

So, that's why models are frozen as a massive lump of dictionaries and triples. As with many things, it probably wouldn't be necessary if models had been done slightly differently (so there was a method that always spat out a nice, portable way to reincarnate a field), but with no reason for that previously, I would have been thoroughly impressed by the core developers' foresight if it existed.

As for parsing versus introspection... I'm still divided. Parsing always has, and still feels, wrong, but introspection might just be less reliable. If you're interested, go peek at the branch I mentioned above, and see what you make of it.

And now, I will stop blathering and return to my small hole at the base of the mighty Django Mountain, where I am attempting to fit square pegs into round holes. Good day.

Quick tip for anyone like me who’s foolish enough to cram Exim4, Spamassasin, Apache with Wordpress running on it, and a pile of other stuff on to a server with 256MB of RAM: Install one of the Wordpress caching plugins. Otherwise, your server is liable to start swapping frantically if you get a dozen or so hits on one of the Wordpress-powered sites.

Indeed, it was possible to nuke jasper just by mashing the F5 key whilst sitting on the front page of this site.

You can now sign in to leave comments here (yes, dear readers, both of you!) – using an OpenID if you have one, and in related news, I’ve got m’self a shiny new OpenID to go and do likewise on other people’s sites.

For those interested, the slides from my migrations talk at EuroDjangoCon are up. I believe the videos will be around soon.

Why settle for a doorbell when you could make a SIP-enabled intercom?

Revision for finals is boring me, so I thought I'd take a break and my mind started wondering... wouldn't it be cool to give door-to-door salesmen the runaround, analogous to that Asterisk script that sends known cold callers into menu hell. Of course this would be possible if your door entry system incorporated an intercom that was a SIP device.

By butchering a cheap £3 Argos phone which you wouldn't miss if it got vandalised/rained on and connecting it to an indoor ATA, the intercom would then have its own identity in the Asterisk PBX, and visitors could press some big green button which dialled '9' or something in the intercoms context. We could then ask visitors questions which they can answer using the touch-tone keypad, e.g. specifying who they want, if they're salesman etc. One could even change the extensions.conf on Halloween.

If you're not in (ie your RFID rabbit is not on the house mir:ror reader), the intercom could take a message which could be saved on the server, or even better, it could call your mobile phone and patch the extensions together if you really wanted. Obviously you'd combine this with my Asterical script so you don't get calls in the middle of something important.

If you had lots of time, and either owned your house or had an extremely nice landlord, you could make Asterisk call a script that controlled an arduino, which could physically open the door. This would almost certainly invalidate your insurance policies, but it would make it possible to let in callers from the comfort of your own chair, and even remotely by dialling a PIN number on the phone.

Of course this is all crazy and I don't really plan on doing it, but my blog needed a new post and this was all I could come up with.

As I write, the sun is up, the bells are ringing, and Oxford is dragging itself out of bed for another day. It’s slightly sobering to think that I’ve just listened to the college choir for my last May morning as an undergraduate…hopefully not my last time from inside college, though – time to start making friends with lower year groups.

I’m going to miss Magdalen. But meanwhile, eight weeks still to enjoy!