Planet CompSoc

The kind people at Packt Publishing have requested that I review their new book about lighttpd. Unfortunately, I cannot comment on how good the book is until it arrives and I have a chance to read it, but they do have a sample chapter_(pdf) online to look at. Watch this space

Also, Happy New Year to all my readers

Just a heads-up to any LastGraph users to let you know that, soon, I'll be implementing one-week graph storage expiry (but probably not for premium users). There's now 128GB of PDFs being stored on S3, and it's probably time I started not throwing money away.

What this means is, from around the middle of next week, possibly earlier, when you make a graph, the download links you get will only be valid for seven days. After that, the files on the other end will vanish into the aether, so if you want to show them off you should probably download them and host them on your own blog.

Still, I'd like to give S3 credit; when LastGraph originally launched, I had a server with 10GB of disk space and only a bit more bandwidth. These days, LastGraph is pulling 44GB of transfer a month at S3 alone, and I have 4.4GB of stored user data (and that's compressed, children), so I'd like to thank the guys at State51 and Last.fm for the resources they've donated towards it, and the Premium users for just meeting the S3 bills so far with their signups!

How to hide Received headers when relaying emails - handy when you trust an authenticated user who shares an IP with a known spammer.

I've been given a Nokia 6120, and have a three PAYG SIM card in it, with which I plan to use the £5 for 2GB within a month addon every month. However, SpamAssassin gave all emails from my phone a high spam rating. A closer look revealed that three's internet access NAT gateway IPs are on spam blacklists, therefore I needed to strip the Received: header containing three's gateway IP.

In Debian, first apt-get install postfix-pcre
Then add to the end of /etc/postfix/main.cf the line:
header_checks = pcre:/etc/postfix/header_checks

The file /etc/postfix/header_checks should contain Perl regular expressions and what they should be rewritten to. The following rule will satisfy SpamAssassin by changing the Received header to an X-Received:
/^(Received: .*mia\.three\.co\.uk.*)/ REPLACE X-$1

However, there may be badly implemented spam filtering software that just greps all headers for bad IPs, so to be on the safe side I stripped the recieved line totally:
/^(Received: .*mia\.three\.co\.uk.*)/ REPLACE X-Recieved: Authenticated phone or internet dongle on three network

This is a white lie though, as any email from three's mobile internet access gets this header, not just 'authenticated' ones, but since I'm not an open relay, the only people who can recieve these messages are users on my local box. A small price to pay to enable emails that I send on mobile to not me put in Junk. It might well be a good idea to implement this whatever your mobile internet provider is, a single bad customer could put your shared IP there on a blacklist

I’ve just finished a Python script to shuffle the names for Secret Santa and email each participant the name of the person they’re giving to. Thus if you’re not organized enough to get everybody together to draw names out of a hat, you can still do it without one person having to know who’s giving to whom.

Sadly, I didn’t get this done in time to give it a try last week, but hopefully somebody will find a use for it.

You can grab the latest version of the script here.

It depends on pytz, because in order to generate emails which stand half a chance of getting through spam filters, it needs to be able to generate proper timezone-aware dates. Using the raw email sending libraries in Python was a bit of an eye-opener - turns out Django takes care of a lot of twiddly bits behind the scenes, which you don’t notice until you have to do it all by hand.

Spotted this on Farcebook Facebook just now…

I’d laugh, but somebody probably sued them when the free pig didn’t turn up.

It generally takes a lot to get me to really like a piece of hardware, thoroughly. For example, my experience of Fujitsu-Siemens' customer service was nothing but excellent, but the only reason I had so much contact with them was that my laptop's inverter went three times (to their credit, it's been fine for the past two years or so now, and my brother is still happily using it, but that laptop still sheds screws at the rate of around one a year).

However, a bit over a year ago, I decided to invest in some bluetooth headphones, and picked a pair of Jabra BT620s (warning: flash music thingy) semi-randomly from the field of available devices. A year later, and I can honestly say they're one of the better things I've bought this decade.

They're well-designed, and have good build quality; they also take and recieve phone calls, with pretty decent sound quality; they also turn into a USB Audio device (headphones and mic) when plugged in and turned on, a feature that is nearly impossible to find out about, and more importantly, last for absolutely ages. I've left these things on in my bag for over a week sometimes, and they'll still be happily flashing blue (they do that when switched on, but you can disable it, else you look a little weird) when I get them out.

I only wish the software support for A2DP was as good. Under Linux, it will work via ALSA, but PulseAudio still segfaults whenever you take it near them, Maemo has okay support, but often stutters slightly and the driver uses a lot of CPU, and Symbian is the only thing that seems to deal with them rather well (except for the occasional confusion when it decides it really should start playing the music over the tinny phone speakers at full volume, instead).

My only complaint is that they're a little large and heavy, but given that this seems to be needed for the batteries and speakers, it's fine, and I have big ears, anyway.

I also, in a strange way, enjoy having both earphones and an alarm clock with mini-USB ports and upgradable firmware; it's obviously the future. In a sick way, I can't wait for the day when I need to spend 30 minutes reflashing my shower so there's more than a millimetre between the "scalding hot" and "arctic cold" settings.

Following the wild success of Telepathy, we're proud to announce our new project:

It's been a while since I last talked about South, so it's about time I did a short update, I think. There's been a fair few things going on since last time, which was, err, a long time ago.

Firstly, we now have SQLite support (apart from column renames/alters/deletes; I'll be implementing those with the appropriate workarounds soon). This was one of the features I most wanted to roll in, since SQLite is quite widely used for quick development. SQL Server support is also on the way soon!

Also in the news, TWiD's latest episode has some interesting poll results for schema migrations; of the respondents, over half are using one of the premade tools, and of those about 2/5 are using South - good news for the migration camp overall, and a surprisingly high amount for South, considering I have a limited picture of who's using it.

Other news includes startmigration getting support for multi-table inheritance (finally), some nice new work on indexes to flesh out the database API, and plenty of bugfixes and small tweaks to make things flow more smoothly.

Also in the pipeline is the possibility of adding an option to send post_syncdb signals at the end; several people have asked for this, and for some coding styles it will greatly reduce headaches about functions running too early in the migrations.

Finally, 0.4 will probably be released at some point soon, so keep a look out!

I've always not been one for phones; for years, I had a (relatively) old Nokia 6100. It does the job of phone calls and SMS messaging very well, and lasts absolutely ages on a single charge.

At the beginning of the year, though, I wanted 3G connectivity for my PDA and laptop (yes, I have a whole range of portable devices, and every one has its use, although not necessarily simultaneously), and so liberated a Nokia 6280 from eBay. The 6280 was pretty good - long battery life, a WAP browser, and that all-important connectivity. However, when its screen broke last week, and replacement LCDs were not too cheap, I went back to eBay with two rules: it had to be under £60, and should fit the two 6280 batteries and umpteen small Nokia chargers I have (the Internet Tablets use them too).

As you can imagine, this doesn't leave a great deal of choice. In fact, excluding the 6280, it leaves pretty much only the N73. Having experienced the 6280's wonderful affinity for dust (Nokia didn't quite get the sliding mechanism dustproof - it gets inside the screen...), I decided to go for an N73 and see what they were like; after all, how bad can crazy selection criteria be?

Not too bad at all, it turns out. The N73 is a pretty neat little device, but more importantly, it runs Symbian. I had presumed all the older smartphones were like the 6280, but I was pleased to discover that there's more S60 apps out there than I can shake a stick at, and that the version the N73 runs is still the most current, compatability-wise.

Three days later, and I have a whole set of things, including Google Maps, which gets impressive position accuracy from cell info alone, the QR code reader I would have needed a year ago at Barcamp Brighton, my Google Calendar synced up, an application that changes to and from silent mode depending on my location (using cell towers) and calendar events (i.e. anything marked as a lecture), and more importantly, the Python.

In short, it's lovely. The next step is getting a permanent, cheap data plan; Orange will sell you one for £5 a week (on Pay As You Go; I'm not shelling out hundreds on a contract), but Three have the suspiciously good offer of £5/month for 2GB, which is better than some ADSL packages I've seen, so I'm seriously considering migrating. The only thing holding me back is Orange's excellent customer service, and the fact I have a deal from them which means I get five free texts a day, forever, for £12 that I paid back in 2001 or so. I'll have to see if I can keep that under a new number.

So I've finally tracked down why Royal Mail's website looks horrible in Firefox on Linux. They sniff user agents and if it isn't one of their 'supported' browsers then they send different HTML that doesn't render properly.

Not only are they incapable of writing web pages that work across multiple browsers, they actually check for Firefox on Windows. If you have Firefox on Linux, which has the exact same Gecko engine, then you get sent the HTML which doesn't render properly. If you spoof your user agent, then all is fine.

Not only is this a big oversight, but it seems silly to go to all the trouble to sniff the user agents, if the HTML they send to unsupported browsers doesn't render correctly in any web browser anyway?

 I have sent an email, but don't expect a reply, nor that they'll heed my suggestions.

When working on Telepathy, I've often wanted to be see which D-Bus methods are being called on whom, when signals are emitted, and so on. Timing information is also handy: I'd like to figure out why cold-starting Empathy takes 12 seconds, and it'd be much easier if I could look at a diagram rather than staring at the unreadable output of dbus-monitor.

Previously, Alban wrote a tool that used a patched version of mscgen, and produced appropriate input with a dbus-monitor-like Python script. I wanted some more D-Bus-specific diagrams, and ended up reimplementing both the monitoring component (by forking dbus-monitor, as its --profile output did not contain quite enough information) and the diagram-drawing component (using Cairo). I'm happy to present an initial release of Bustle:

There's a Telepathy-specific hack in the tool to shorten object paths, but it shouldn't make the tool any less useful for looking at other D-Bus traffic.

I haven't made binary packages yet, I'm afraid, so you'll need to grab the source tarball and build it if you want to try it out. In Debian-land, the dependencies are libdbus-1-dev libglib2.0-dev libghc6-mtl-dev libghc6-cairo-dev libghc6-gtk-dev libghc6-parsec-dev; see README in the source tree for how to build and use it.

The astute among you may have noticed from the dependencies that the diagram-building component is implemented in Haskell, using the excellent bindings to Gtk+ and Cairo. I got a prototype going within a few hours, and the strong correctness guarantees that the type system provides meant that I could refactor it mercilessly with confidence. I'm sure that I would have spent many frustrating hours chasing type bugs had I written it in Python, which is a more conventional high-level language for prototyping and writing tools like this. Next time you're frustrated by such bugs, you should give Haskell a try. :-)

I’ve not been sleeping all that well lately. And let’s be honest, this probably has something to do with being in the middle of a term at Oxford University, with a lot on my mind and far too much to see to.

However.

I’ve noticed recently that my room contains a surprising number of light sources, which I suspect can’t be helping me get to sleep. Reading from left to right, we have:

• Light leaking under the door from the staircase (where the lights are apparently permanently on)
• Blue power LED flashing on my laptop
• Green power/charging LED on my other laptop
• Huge red light on the multiplug under my desk
• Flashing lights on my hub

I’ve therefore come up with a cunning plan:

• Block the base of the door with an old pillow
• Prop a book over the laptop light
• Put the other laptop in a drawer
• Tie an old sock round the multiplug
• Shove the hub into a cardboard box with holes for the wires

Whether or not this helps remains to be seen. But here’s a plea to manufacturers: please think twice about the brightness and location of your LEDs, especially if there’s a chance someone will be trying to sleep in the same room as the device they’re on.

I finally got round to putting Linux on my Advent 4211 yesterday. There are various ways and means of doing this; the ones which worked for me were:

• Use the installation image optimised for small screens
• Follow these instructions to make a bootable USB drive from the image (note: it’s ‘cat’, not ‘zcat’ as the ubuntu image is not zipped. I know they’re Debian instructions, but they seem to work)
• Use the script given in this forum post (run as root) to get the WiFi working.

The end results are really rather impressive; here’s a pretty picture:

Here’s one for all you progammers out there: what happens to the values of x and y if you type the following in, say, Python?


>>> x = 5
>>> y = 6
>>> x,y = y,x


I’d always half-thought, without really considering it, that you’d end up with x=6 and y=6, i.e. that the above was syntactic sugar for:


>>> x = y
>>> y = x


Actually, though, it turns out…


>>> x=5
>>> y=6
>>> x,y=y,x
>>> print x,y
6 5


In what onlookers are already calling “potentially unwise”, our robot overlord ordered various toys from ThinkGeek.

Today — being as it is Friday, always the most productive of all days — they arrived! Apparently appropriate tools to control the missile launchers are not yet packaged for Debian. :(

VMware do seem to be moving towards a whole "woo look at my management console" thing.
Turns out (despite my misgivings) that it's quite possible to do USB passthrough of mass storage devices to your guest OS - so, for example, running XP in a VM, you plug in your Stick O' USB, and have XP use it rather than your Debian Etch host machine.  This assumes you're running Server 2.x, and so have Webby Goodness.
Turns out that this is a really simple trick.
Step 1: Ensure that you're in the plugdev group (46).
Step 2: Modify the permissions with which /proc/bus/usb is mounted:Yep, that's right kids, on Etch /proc/bus/usb is no longer mounted by /etc/fstab.  So don't bother putting your entries there, it won't work.
Step 3: Reboot
Step 4: Profit (yeah, I know this should be step 3).

If it doesn't work - you do have a USB controller installed on your VM, don't you?

If you're anything like me, you've probably missed appointments due to your bad memory. Well now there's now excuse, you can use your trusty basic phone to recieve reminders with reminder calls from Asterisk.

My new script, AsteriCal, will poll your Google Calendar's iCal URL and call you to remind you of upcoming events. Find out more here

It's written in Python, and communicates with Asterisk by placisng call files in Asterisk's outgoing calls spool directory.

You may have heard of the site Say No To 0870. It's a UK database of mappings of Company Names and their 0870/0844/0845 numbers to their geographic equivalents. Using the geographic equivlalents is a good idea - they're cheaper and included in a lot of plans' free minutes. Some companies can detect people calling their geographical numbers. Prefixing the number with 141 to withhold your number sometimes beats the system; sometimes it doesn't.

I thought it'd be a good idea to integrate the Say No To 0870 database with my Asterisk PBX, so that when I dial a number beginning with 08, Asterisk will actually call the geographical number with my Trunk provider. I achieved this using Asterisk AGI and Python. I screenscraped the Say No To 0870 website (theres no API, nor did they reply to my email) and untangled the HTML with Beautiful Soup.

If an 08 number has no entry, it just gets dialled. If there is an entry, it presents the user with a looping menu asking them to press 1 for the geographical number or 2 to dial the 08 number (in case the destination is smart, or the Say No To 0870 database is wrong). The script only considers the Main (Verified) numbers in the site's blue search results box.

You can view the script or, in Debian, you can install like so:
(Other distros store agi-bin in different places):

cd /usr/share/asterisk/
mkdir agi-bin
cd agi-bin
wget http://www.crummy.com/software/BeautifulSoup/download/BeautifulSoup.py
wget http://www.maniacmartin.com/files/noto0870.agi
chmod +x noto0870.agi
cd ../sounds
wget http://www.maniacmartin.com/files/mm-foundacheapernumber.gsm

Now, you need to add the following to the contexts in your /etc/asterisk/extensions.conf where you wish to allow users to dial 08-prefixed numbers:

exten => _08[4578].,1,AGI(noto0870.agi|${EXTEN})          ; Lookup in saynoto0870
exten => _08[4578].,2,Macro(dial-uk,0044${EXTEN:1})       ; If no match or user rejects, use normal UK trunk

The line for other UK numbers shouldn't accept 08 now. Something like this will do:

exten => _0[1237]XXXXXXXXX,1,Macro(dial-uk,0044${EXTEN:1})  ; Normal UK numbers

If a user chooses to use a cheaper number, the script will change the extension and jump to it on exiting.

How to setup Festival with Asterisk for arbitrary text-to-speech in your PBX.

Whilst playing with asterisk, I wanted the feature of text-to-speech, so the PBX can read things such as caller IDs.

The information at voip-info.org would have been helpful, except they couldn't reproduce the conf file verbatim as their in-house wiki parses ((double-bracketed text)) as links, so to help you, I've got a verbatim config file here:

(define (tts_textasterisk string mode) 
"(tts_textasterisk STRING MODE) 
Apply tts to STRING. This function is specifically designed for 
use in server mode so a single function call may synthesize the string. 
This function name may be added to the server safe functions." 
(let ((wholeutt (utt.synth (eval (list 'Utterance 'Text string))))) 
(utt.wave.resample wholeutt 8000) 
(utt.wave.rescale wholeutt 5) 
(utt.send.wave.client wholeutt)))

In Debian, the Festival conf file lives in /etc/festival.scm. Paste this into it, then restart Festival using its init script. Now in Asterisk's extensions.conf, you can call it like so:

exten =>2,1,Festival(Hi\, you are being called by)
exten =>2,n,Festival($CALLERID)

Don't forget to escape commas and such. (OK, this was a bad example, as you'd just use SayDigits() in this case)

My normal approach is useless here.

Do you write Python (not very well?).

Ever wondered why your calls to os.execvp() and its ilk don’t work as expected?

The answer may be that you’re doing this:

os.execvp('ls', ['/home/fred'])

When you actually mean this:

os.execvp('ls', ['ls', '/home/fred'])

As ever, reading the Python docs (help(os.execvp)) is usually better than randomly googling for an answer.

I was recently reading Regulation of Investigatory Powers Bill -- Some Scenarios and scenario 2 about session keys perked my interest. I wondered if it could be done with GnuPG, and after researching for a while I discovered it can, and here's how: As you probably know, in semi-new UK law, RIPA means that the police can demand that you hand over the plaintext of an encrypted file, or in certain circumstances demand that you hand over a key of your choice that decrypts the file. Since public/private key encryption is several orders of magnitude slower than symmetric encryption, GnuPG generates a random symmetric key and encrypts the plaintext with this. The symmetric key is then encrypted with the public key of the recipient and tacked on the front of the output. So, if the police want a key to a file, using your private key you can retrieve the symmetric key used for this particular file, and by handing this one-time key over you have satisfied the requirements under RIPA without compromising your main private key and allowing law enforcement to decrypt further messages without your knowledge or sign messages as you. To get the one-time key, do this on an uncompromised machine:

$ gpg --show-session-key accomplises.asc

You need a passphrase to unlock the secret key for
user: "Martin Smith "
1024-bit ELG-E key, ID 941AAA41, created 2006-10-23 (main key ID 50CB07BC)

gpg: encrypted with 1024-bit ELG-E key, ID 941AAA41, created 2006-10-23
      "Martin Smith "
gpg: session key: `9:12D785497247CA3C929EC62EAB63BD8A8F0BB29E9B60CF11FC43788B197F6A11'

Now law enforcement can use this key to decrypt accomplises.asc without needing any public or private keys:

$gpg --override-session-key 9:12D785497247CA3C929EC62EAB63BD8A8F0BB29E9B60CF11FC43788B197F6A11 accomplises.asc
gpg: encrypted with 1024-bit ELG-E key, ID 941AAA41, created 2006-10-23
      "Martin Smith "

The cycle ride is over, and I'm in the process of collecting those pledged sponsorships. People have an amazing ability to not have any money on them when its time to pay up, which is pretty convenient for them. Anyway, I figured that I will not be updating the website any more, since all of the days are already blogged. Due to the heavy GPX caching and the fact it was my first Django (and web framework for that matter) usage, which I chopped and changed many times, keeping this dead site online was costing me 71MB of RAM, which is quite a sizable proportion of my 256MB VPS. I decided that the best way was to make a static snapshot of the site, and serve it with my regular httpd directly. It was basically a matter of getting the snapshot: wget -nH -m http://lejog.maniacmartin.com And renaming the GPX files and their references by running these commands from the appropriate directories: for i in $(ls|grep -v index); do sed -i s@/posts/gpx/$i/@/posts/gpx/$i.gpx@g $i/index.html; done for i in $(ls|grep -v index); do mv $i/index.html $i.gpx; rm -r $i/; done I then copied the static/ and media/ folders over. I also added a mime type for GPX files for cleanliness to /etc/mime.types: application/gpx+xml gpx (You could also use the more generic text/xml type) I know for a Django site, this leaves a mess of folders all containing a single index.html page, but since the site will never be updated again, it frees up RAM and CPU cycles (caching of parsed GPX is intensive), and means that the site will load faster. Given the heavy usage of Google Maps, any speed up is a bonus.

One of the topics that popped up repeatedly at DjangoCon last weekend was how bad purely normalised table structures are, and how denormalisation is good for many things, including making your database cry less.

To that end, during the gaps in PyConUK this weekend, I decided to see how easy it would be to write a new Django field that will automatically denormalise a field in a related table across to another model.

Let's take an example; imagine we have this kind of model:

class Picture(models.model):
    user = models.ForeignKey(User)
    user_username = DenormField(User, "username")
    image = models.ImageField(upload_to="foo")
    title = models.CharField(max_length=100)

The idea is that the user's username (which, let's face it, you're going to need almost every time you need the photo) is stored in the same table, saving you from doing that join to get it from users each time. While this may not seem like much, it can be useful, and besides, it was fun to write.

My DenormField simply listens to a trio of the relevant signals, and then updates this value whenever it detects that the related User object has been updated, so it only costs on write (which, let's face it, is much better than on every read). You can do this kind of copying in normal Django app code as well, but I think this solution is much cleaner. I would do, I guess, since I wrote it.

It's only really a little test exercise in writing Strange New Fields (and I wanted to see how much monkeypatching it would take - none, as it turns out, just some very nasty code tricks). You can get a copy of it if you feel like having a play (warning: only briefly tested; may attach captions to your cat).

There are not very many things that could tempt me to fly 10 hours twice for only a weekend, but DjangoCon was one of them, and I'm glad I went; it was one of the best weekends I've had for quite a while.

There were plenty of good talks; I can't mention them all here, since as we all know, bytes cost money, but in that vein I must at least mention Cal Henderson's talk 'Why I Hate Django' - a brilliantly done talk on the things Django's missing, as well as the fact we don't have a mascot (or do we?) and that we aren't smug enough.

Still, I enjoyed many of the other talks as well (what a shame it is not to be able to see them all in person; still, Google will hopefully release the videos soon), as well as our Schema Evolutions panel, which seems to point towards a bright future for Django and migrations. 

Talking of bright futures, judging by the final session with Adrian and Jacob, there's plenty of stuff to get working on for 1.1, some of which have me quite excited. Multiple database support is going to be one of those important ones, really, but then Malcolm seems to have snuck the bottom layer of it in already, which is quite handy.

All in all, an excellent weekend, and finally a chance to meet a subset of everyone involved in Django (and they really do come from many places), and I can now hardly wait to get over to Prague next year for the promised EuroDjangoCon. If you didn't make it, then I highly recommend watching the videos when Google release them (after editing them on their ridicously great AV system, no doubt).

The world of migrations in Django is definitely warming up now, and with that we're proud to announce the release of South 0.3, the 'intelligent django migrations app', available now from south.aeracode.org.

New in this release are the use of fields rather than dicts for specifying columns in migrations, so migrations now look cleaner, work with all the databases Django supports to a better extent, and even support custom fields.

Also new is dependencies between apps, for those situations where your forum app's Post depends on your accounts app's Profile; South will work out the right way to apply everything so foreign keys don't horribly break.

There's also a nice number of bugfixes and general tweaks Andy and I have been making as South moves towards being used in active development work. If you think you might be needing migrations, or even if you don't, give South a try!

(Also new is a very carefully worded "why we're best" page on our wiki. I'm trying not to step on people's toes, just say why we're using South rather than anything else, but I'm not sure how well that's working)

With the launch of its own site - south.aeracode.org - and Andy McCurdy jumping in to help out, South has been making some good progress. We've made a few backwards incompatable changes, which while annoying will hopefully make things much easier in future.

Firstly, the format of the create_table and add_column calls has been changed to use Fields instead of long dicts of parameters. As well as looking more Django-like, this allows us to reuse the SQL generating features Django already has for fields, as well as playing nicer with custom types.

In line with these changes, startmigration has also seen some changes; as well as generating things using Fields now, it can group models together in migrations (this proves useful for foreign keys referencing tables made after them).

We're still working on finishing this all off, and there'll hopefully be something stable to show off soon. Woop.

Also, if you're going to DjangoCon, may I recommend the Schema Evolutions panel; there'll be all manner of evolutiony and migrationary things discussed therein.

I bought myself an Advent 4211 last week (you might know it as the MSI Wind; they’re essentially the same thing). So far, it’s lived up to expectations; there are just a couple of things to say…

• Mine seems to have the decent Synaptics touchpad - read The Register’s review for the warning about the other one.
• You really need to use headphones with it if apps you’re listening to don’t have a very good sound level - inevitably in a device this size, the speakers aren’t great, especially at full volume.

How the Linux install on it goes is something I shall let y’all know shortly…

My laptop contains an ATI Radeon chip.  Yes, I know that this is Bad for Linux-compatability - I was young and inexperienced when I got it.  Said laptop also runs Debian testing/unstable (/experimental), and used the xserver-xorg-video-radeon driver.

Now, kernel versions > 2.6.22 have been causing it to freeze completely when starting X - no network response, no nothing (with a lack of serial port, the trick is to boot into rescue mode, start ssh, ssh in, then continue to boot while tailing logs).  This is Less Than Ideal, especially given the number of security holes that have been discovered that apply to my 2.6.22 install.

So, last night, I decided to see what the fglrx driver was up to these days.  Last time I tried it, it had a nice tendency to hang the machine if I did, erm, pretty much anything, especially switching away from VT7.  Imagine my surprise when, having installed the appropriate packages (apt-cache search fglrx IYF) and prodded xorg.conf, it Just Worked™ - well, apart from the fact that mouse and keyboard input didn't seem to be happening...  Turns out that the xserver-xorg-input-kbd, xserver-xorg-input-mouse (and in my case xserver-xorg-input-synaptics) are kinda useful.  So, a quick switch to VT1, install, restart gdm, and switch back (look Ma, no hanging!), and I have working X.  Hooray!

If only the random router/firewall issues at work would sort themselves out as simply...

I may also be missing the most fabulous Jude in the world, and so have to keep busy somehow...

For the next two weeks or so I'm helping Martin Smith do his Land's End to John O'Groats trip, and so I'll only have slow internets to communicate from. Still, you can read about our travels at the new, shiny, lejog blog.

While this means you won't be able to send me post (the old, snail-paced variety) for a while, I doubt any of you want so, so that's not really important.

Also, once we get back there'll be a load of GPS data I will set to work visualising, if you like that kind of thing, and burning questions (which I have yet to determine) will be answered.

After the initial release some time last week, and a generally positive reaction, South now has a 0.2 release.

The most important (and only major) change is the addition of MySQL support, so the other section of Django users can finally give it a try. There's also the ability to create all models' migrations at once.

The main South project page has also been slightly improved, and while I've added even more text it should hopefully be easier to grok quicky.

If you have any feedback or advice on South, remember to tell me! If I don't know about something, it might not happen...

Regular readers will know that I recently migrated to lighttpd from Apache 2, to try and enable my blog to handle traffic spikes better. Out of curiosity, I decided to benchmark the new setup. I used a dedicated server on a 100Mbit line at ovh, France to perform the tests on my 256mb Xen VPS in London. First I installed ApacheBench, which is in the Debian package apache2-utils. Then I made it send 1000 requests, 10 concurrently, (figures I chose off the top of my head) to each of the following sites I host:

• A static file
• Wordpress with wp-cache plugin on php5-cgi with 4 processes under fastcgi
• Django eshop running in threaded mode under fastcgi
• CherryPy webpwman running under fastcgi

Now I don't claim it was a fair test, but the results are pretty shocking.: Average time per request on ion.maniacmartin.com:

Ping time	12ms
Static	53ms
Django	113ms
CherryPy	238ms
Wordpress	1700ms (+539 failed requests)

When my blog was reddited, it was Wordpress without wp-cache on Apache workers, all of which contained unncessesary mod_python Django instances. It stood no chance. Even after the optimisations I have done, Wordpress still performs shockingly bad compared to the alternatives. I will probably be switching my blog to a Django-powered backend, such as the soon-to-be-released aeblog, that powers Aeracode. Update: You can get a second opinion, which also shows that Django is faster, but by a much smaller margin.

After learning that my 256mb Debian Xen VPS died instantly after being reddited, I decided to take some action to prevent the same thing happening again. My blog is currently powered by Wordpress, so I enabled the wp-cache plugin that I had installed some time ago. It is however, one of the few PHP-powered things on my server. Then I decided to migrate to lighttpd, because it uses less memory than Apache, and because I like changing things on my server for fun. There were some hurdles, but I got there in the end. Here's what I'd do now if I had to do the migration again. They are not necessarily the best settings for all sites. I chose these settings as each site is small, and I don't have much CPU or RAM to play with. I use vim as my text editor. If you prefer something else (nano, emacs etc), then obviously substitute where necessary:

• Install lighttpd - apt-get install lighttpd . Then edit /etc/lighttpd/lighttpd.conf and set the port to 81. This will let us test lighttpd whilst Apache is running. Run apt-get install lighttpd again and it will finish successfully this time.
• Enable lighttpd modules - I need fastcgi for PHP, Django and CherryPy. Enable it like so: cd /etc/lighttpd/conf-enabled ln -s ../conf-available/10-fastcgi.conf 10-fastcgi.conf aptitude install php5-cgi The Debian release has support for PHP4 by default. Change this to PHP5: sed s/php4/php5/g 10-fast-cgi.conf I also wanted ssl support: ln -s ../conf-available/10-ssl.conf ssl.conf vim ssl.conf Specify your pem file location. If you have only a key and crt, cat them together into a .pem. Note that you cannot test lighttpd with Apache running if you have the SSL module enabled, as they'll compete for port 443
• Specify vhosts - You probably host multiple websites, and want the equivalent of Apache virtualhosts: I have one file for web framework-powered sites, and one for sites that serve PHP, but its up to you: vim vhosts.conf This specifies a vhost's URL with a regex. In this case its middlerasenchurch.org.uk or any subdomain of it.

  $HTTP["host"] =~ "(^|\.)middlerasenchurch\.org\.uk" {
          server.document-root = "/var/www/middlerasenchurch.org.uk"
  }

• Rewrite .htaccess files - lighttpd doesn't understand Apache's .htaccess files, but you can specify rewrites if the rewrite module is enabled (uncommented) in /etc/lighttpd/lighttpd.conf. If you need it, go and uncomment it. These rules make Wordpress work on my site:

  $HTTP["host"] =~ "(^|www\.)maniacmartin\.com" {
          server.document-root = "/var/www/maniacmartin.com",
          alias.url = (
                  "/svn" => "/usr/share/websvn/"
          )
          url.rewrite = (
          "^/?$" => "/blog/index.php",
          "^/blog/(wp-.+)$" => "$0",
          "^/blog/xmlrpc.php" => "$0",
          "^/blog/sitemap.xml" => "$0",
          "^/blog/(.+)/?$" => "/blog/index.php/$1"
          )
  }
  

• Make lighttpd spawn CherryPy - I used fastcgi to link lighttpd and CherryPy. First, I added this code to my python project, making it serve on fastcgi if it got passed the "--fastcgi" switch:

  if "--fastcgi" in sys.argv:
  	app = cherrypy.tree.mount(WebPWMan())
  	# CherryPy autoreload must be disabled for the flup server to work
  	cherrypy.config.update({'engine.autoreload_on':False})
  	cherrypy.config.update({
  			"tools.sessions.on": True,
  			"tools.sessions.timeout": 5,
  			"log.screen": False,
  			"log.access_file": "/tmp/cherry_access.log",
  			"log.error_file": "/tmp/cherry_error.log",
  	})
  	from flup.server.fcgi import WSGIServer
  	cherrypy.config.update({'engine.autoreload_on':False})
  	WSGIServer(app).run()
  	
  else:
  	cherrypy.config.update({'server.socket_port':8051})
  	cherrypy.config.update({"tools.sessions.on": True, "tools.sessions.timeout": 5})
  	cherrypy.quickstart(WebPWMan())
  

  Its important to redirect the logs away from stdout, as stdout is whipped away from CherryPy when you quit SSH, and the process will die. Also, logs should go to a folder that www-data can write to. (/tmp is probably not a good idea, but I don't read these logs anyway) In a file in /etc/lighttpd/conf-enabled, add an entry like this:

  $HTTP["host"] =~ "webpwman\.maniacmartin\.com" {
    fastcgi.server = (
      "/" => (
        "script.fcgi" => (
          "bin-path" => "/home/martin/webpwman/webpwman.py --fastcgi",
          "socket"          => "/tmp/webpwman.sock",
          "check-local"     => "disable",
          "disable-time"    => 1,
          "min-procs"       => 1,
          "max-procs"       => 1, # adjust as needed
        ),
      ),
    )
  }
  

• Configure Django in lighttpd - I run Django as seperate processes, and only pass it dynamic URLs, as lighttpd is much faster at serving static files. You could also do this with CherryPy, but my CherryPy site (webpwman) only has one static file, so it wasn't worth it. My confs look something like:

  $HTTP["host"] =~ "(^|\.)littledoves\.co\.uk" {
          server.document-root = "/var/www/littledoves.co.uk"
          fastcgi.server = (
                  "/littledoves.fcgi" => (
                          "main" => (
                                  # Use host / port instead of socket for TCP fastcgi
                                  # "host" => "127.0.0.1",
                                  # "port" => 3033,
                                  "socket" => "/tmp/littledoves.sock",
                                  "check-local" => "disable",
                          )
                  ),
          )
          alias.url = (
                  "/media/" => "/usr/local/django/django_src/django/contrib/admin/media/",
                  # Change to match where you put Django
          )
  
          url.rewrite-once = (
                  "^(/media.*)$" => "$1",
                  "^(/static.*)$" => "$1",
                  "^/favicon\.ico$" => "/static/favicon.ico",
                  "^(/.*)$" => "/littledoves.fcgi$1",
          )
  }
  

• Make an init script for your Django projects I started out with the script at http://code.djangoproject.com/wiki/InitdScriptForLinux, saved in /etc/init.d/django-fastcgi I changed the central part to:

  start-stop-daemon --start --chuid $RUN_AS --pidfile $RUNFILES_PATH/$SITE.pid --startas /usr/bin/python $SITES_PATH/$SITE/manage.py runfcgi method=threaded socket=$RUNFILES_PATH/$SITE.sock pidfile=$RUNFILES_PATH/$SITE.pid
                         pidfile=$RUNFILES_PATH/$SITE.pid
              chmod 400 $RUNFILES_PATH/$SITE.pid
  

  and changed the parameters at the top to: #### SERVER SPECIFIC CONFIGURATION DJANGO_SITES="lejogger littledoves" SITES_PATH=/usr/local/django/ RUNFILES_PATH=/tmp RUN_AS=www-data #### DO NOT CHANGE ANYTHING AFTER THIS LINE! This makes it use sockets, not TCP/IP ports. I used /tmp, not /var/run, as it needs to be a folder that www-data (or whatever user you run your Django instances as), can write to. To make the init script run on startup: chmod +x /etc/init.d/django-fastcgi update-rc.d django-fastcgi
• Test everything - I can't stress this enough. You know what to do Dont forget to restart lighttpd though: /etc/init.d/lighttpd restart While you're at it, you might want to specify different log files per virtual host too.
• Go live - First we need to disable Apache from starting on boot: cd /etc/rc2.d mv S91apache2 K09apache2 Change lighttpd to port 80 and go: /etc/init.d/lighttpd stop vim /etc/lighttpd/lighttpd.conf /etc/init.d/apache2 stop; /etc/init.d/lighttpd reload

Its all going fine here, apart from that I've lost mod_svn, which lighttpd doesn't have. I've switched to svn+ssh, and plan to add some features to the PHP-based websvn and deploy it soon.

With the recent massive changes to Django trunk - newforms-admin being the biggest - I've found a need to run both oldforms-admin and newforms-admin versions in both development and production; here's a quick HOWTO on both.

Development is arguably easier, although at times more annoying. The easy way, and the one I used, is the one blogged about by Patrick Altman, where you have a small script to change what your Python sees as the django module, which you can run each time you change project to a different version. His uses subversion checkouts every time; I prefer the one below, which chooses one of a set of already-checked-out Djangos, allowing quicker (and offline) changing:

#!/bin/sh
sudo rm /usr/lib/python2.5/site-packages/django
sudo ln -s /home/andrew/Programs/django-$1/django /usr/lib/python2.5/site-packages/django

Obviously this needs changing for wherever you keep your code; however, I have two folders in ~/Programs, django-trunk and django-0.97, so I just run something like "djangoswitch 0.97".

Production is a little trickier, but cleaner. Since you need to run more than one site at once, you can't go around relinking the django directory. However, all you have to do instead is tell mod_python (if you're using that method of deployment) you want a different Django version with a line like this:

PythonPath "['/usr/local/django/django_1.0'] + sys.path"

where all your other PythonBlah lines are in your Apache config. (Note, on my server the global version is 0.97-pre-svn, so I have a special override to 1.0 for my one app that uses it currently - this blog).

FastCGI is somewhat harder, but LastGraph doesn't need upgrading yet. I imagine it might take something like virtualenv to do it (or edit the manage.py script to twiddle with sys.path at the start).

webpwman is an online password manager that I wrote in CherryPy, that can import from KDE pwmanager CSV exports, and run behind an SSL-enabled webserver (which also prevents MITM attacks). It asks 3 security questions, which it randomly rotates on every correct login and asks for a master password which is used to decrypt the password from a json file. The idea being, that if you're on a compromised public terminal, then the bad guys should only get the passwords you viewed that session. It needs no database engine and supports only a single user, as I don't want to endorse storing your passwords with third parties, although I need such a tool as I can't remember my randomly generated passwords when out and about. I'd like to take this moment to congratulate the CherryPy team - I really like it, but your deployment documentation wasn't thorough enough for me, and I had to do a lot of googling to get it to sit behind another web server. Overall I've tried to keep it simple, and I'd appreciate any feedback, criticism or feature requests, especially regarding vunerabilities, as this is my first CherryPy project. Download and install webpwman here »

Tired of having to drop tables to re-syncdb your django models? Django-evolution not working, or too magic? Then I have just the solution; my newest project, South; intelligent migrations for Django projects.

Invented out of a frustration with continuous syncdb taunting, South can deal with migrations that suddenly vanish or appear, is designed to have database-agnostic migrations, and can autogenerate migrations based on your models.

It's certainly not yet complete, with only one database in the so-called database abstraction layer, and not all the fields working yet, but it powers this site, so I have proof it works! Take a look at the project page, or have a glance through the tutorial.

And please, send me some feedback! Or even better, patches, but that's highly optimistic :)

While I have been a happy user of Wordpress for many years, it was about time I jumped ship and moved my blog to something that let me manage things like project lists more sanely, and so welcome to my new Django-powered blog.

It uses a blog application/framework/thing I built from the ground up; while that's kind of unremarkable, given how easy this kind of stuff is if you've got the power of Django, one nice feature is that the URLs are a superset of the old Wordpress URLs. I even have a script that tests all the URLs from my server logs to help prove this. Honestly. (I may have missed some in the output, though, after it started testing attempted hacks)

Also of note is the tumblelog-ish thing on the side of the homepage, which just combines my feeds from last.fm, twitter and delicious (for now; more might appear soon). Hopefully it will make my near-glacial blogging speed a little less obvious.

I might well be releasing the code at some point soon, although I have to release the migrations library it depends on first! In the meantime, enjoy marvelling, examining or criticising my three evenings' work.

I recently started writing an online password manager. The basic idea is that it would ask 3 questions from a bank of questions, and then prompt for a decryption password and the name of a service of which to get a password. The service would run over SSL, and the passwords would be exported from my laptops KDE pwmanager in my server backup script. That way I'd be able to get my unmemorable passwords from any internet-connected terminal, without fear of an eavesdropper or piece of malware getting all of my passwords, since the 3 questions would change, much like the digits asked for when using online banking. Django seemed a touch too heavy, and I was tempted to use PHP, but Andrew suggested I give web.py, the slim web framework behind reddit, a try. web.py is indeed very slim, and I soon realised I'd have to find a different database ORM, as web.py's involved writing raw SQL as arguments, which I didn't really want to do. So I downloaded SQLObject. Midway through the project I decided that I needed sessions, so I didn't have to authenticate on every page. The web.py sessions example code calls web.ctx.session, but that didn't seem to exist for me. The web.py cookbook's page on sessions calls a web.session.Session(..) but specifies that this is available on version 0.3 only, which is strange, since the Get It! link on the site goes to a file called web.py-0.23.tar.gz, which seems to imply a latest version of 0.23. Eventually, I tried to find this session code myself, but it didn't seem to exist.

martin@zapper:webpwman/web $ grep -R session * martin@zapper:webpwman/web $

Anyway, back to SQLObject. I already had a postgresql database hanging around from when I was developing lejogger, So I thought I'd use that for developing webpwman (yes, I'm that lazy). The database is set up so that the user 'lejogger' can access it without any password. (ie psql -Ulejogger lejogger works fine), so I defined its URI: "postgres://lejogger@localhost/lejogger" This only produced tracebacks complaining about the lack of a password, and eventually I realised it was probably trying to connect via TCP/IP rather than the socket file, so I changed the URI to: "postgres://lejogger@/var/run/postgresql/lejogger" But it seems the leading / on the socket's file path disappears (This happens no matter how many I insert into the URI). In fact, I can see no way to make SQLObject connect to a postgresql database file socket whilst overriding the default username of the UNIX user, and I don't really want to edit my database permissions. After consideration, I have decided to rewrite the application using CherryPy (whose sessions code hopefully exists), and use a json file to store the data, since there won't be that much of it anyway and the passwords will be encrypted. This will also make it easier to import data, as the backup script can make then rsync/scp the json file up to the server. If I have missed something glaringly obvious, please contact me and I will update the post accordingly.

I guess it's just fussy to want not-too-hot and not-too-wet - at least it's not flooding.

Also, while the following works for cable modems:



You are strongly advised not to do so to your custom firewall (appropriate for Enterprise, apparently), unless you feel like digging out an old machine, a handy Debian install disk, network cards and your iptables knowledge and missing lunch...
On the other hand it might solve an issue where new VMware ESX machines (and, in fact, new physical machines) are added to the network and can't talk outside the local subnet.  Had we not cannibalized our Virtual Centre machine to get a network card for the firewall, I'd test it...

(also, I miss compressedchaos :()

Still, it's Saturday, and I have internet again \o/  (Oh, and hello Planet OxIRC, which I may actually be on now - that has probably about tripled the readership of this...).

This post is by no means up to the standard of the official Django 1.0 alpha release notes; it merely details what I had to do to get a Django 0.96 site working under Django 1.0 alpha. First, update your Django code. Mine's under svn, but svn up spewed an error, so i deleted the folder and checked out Django again. Next, open the models.py file in each of your applications. Remove the Admin classes from each model. Also, remove the prepopulate_from parameters from fields. Now open urls.py, and remove the old entry: # Uncomment this for admin: # (r'^admin/', include('django.contrib.admin.urls')), and replace it with (r'^admin/doc/', include('django.contrib.admindocs.urls')), (r'^admin/(.*)', admin.site.root), (the first line enables admin documentation). Also, add to the top of the urls.py file: from django.contrib import admin admin.autodiscover() admin.autodiscover() will look for an admin.py module in each installed app in INSTALLED_APPS and incorperate it into an admin page. Now, for each app that we want admin functionality for, make its admin.py The bare minimum you can do is simply inherit from the class they provide. You can do it like so: Extract from appname/admin.py:

from django.contrib import admin from breeze.web.models import * class CountryAdmin(admin.ModelAdmin): pass admin.site.register(Country, CountryAdmin) class UserLevelAdmin(admin.ModelAdmin): pass admin.site.register(UserLevel, UserLevelAdmin) class UserProfileAdmin(admin.ModelAdmin): prepopulated_fields = {"slug": ("name",)} # Populates slug field from name field admin.site.register(UserProfile, UserProfileAdmin)

Now, when you type ./manage.py runserver it should just work. Of course you can do lots more magic , see the Django Admin Documentation for more things, such as inline displaying of related tables. You can even make multiple admin classes.

I have recently embarked on writing a simple open-source BitTorrent private tracker and community website, as a single Django project. I felt compelled to do this after seeing the maze of PHP code behind other BitTorrent tracker/community sites, such as TBDev and xbtit. I want breeze to be simple yet functional, and easy to understand. I'm currently porting what I've written to Django 1.0 alpha, as initial development was done in the more familiar 0.96 release. As usual, you can check out my code using subversion (David-style joke) with svn co https://www.maniacmartin.com/svn/breeze You might also be interested in the (amazingly short) at tracker specification

I'm going to the Cambridge Folk Festival this coming long weekend (Thursday 31st July–Sunday 3rd August). The person I was camping with is probably not going to make it, having just had an operation; does anyone want her ticket? The face value is a hundred British pounds; she'll take substantially less. Let me know by the medium of your choice if you're interested!

LastGraph has been fixed, at least as far as I can tell; the last.fm team fixed the API bug pretty quickly on Monday, so thanks to them. If your profile was affected (that is, if its timeline is flat), email me and I'll delete its data file. Finally, I would have ended with a cheap phrase like "Keep on LastGraphing!", but that would have been cheesy and silly.

It seems that the launch of last.fm's new site has strangely broken the older API that LastGraph uses. I knew I was going to have to update to the new API, but I didn't think it would be this soon! For the two eagle-eyed people who've contacted me about this, well done - I'm working on a fix now (the precise problem is that the old API now always returns the last week's data, no matter when you ask for, hence the very flat graphs). Update: After a quick three minute URL change, it seems the new one is ALSO broken, and other things such as the lovely Last.fm Spiral are broken too. Time to go make sure they know about this at last.fm HQ.

I've decided to cycle the length of the UK for fun (and charity as well). And what better way to get in the mood by trying to learn Django and make a semi-useful Python app. So, I've given it a shot with a GPS/Google Maps mashup thing to analyse blogged route parts from maemo mapper on one of Andrew's Nokia internet tablets. Obviously you won't be able to see all of that until after we have actually blogged something - so check back after Monday, 12th August to see live progress (if it all works that is). It also accepts pledges of sponsorship, you can donate, whilst browsing the source code in subversion. Before anyone tells me, I know I'm not the best programmer in the world and that I took corners. Its made available only in the hope it'll be interesting or useful. You can check out more details at Martin's Land's End to John O'Groats cycle ride.

It comes to my attention that Jonathan Coulton is putting his clogs back on and coming back to the UK in the autumn. His first show here was great fun; you should go out of your way to see him this time around! Tickets are on sale for a show at Shepherd's Bush Empire on October 30th, which is the city I'm most interested in going to. I'll buy some tickets in the next day or so, so if you want me to get you one (and thus sit with a mostly-known set of people) let me know!

(Oxford people might prefer to see him there two days later, but I'm moving out in 10 days. Sniff.)

I’ve been aware for some time that my DNS isn’t quite as securely configured as I’d like. http://crashrecovery.org/named/ looks pretty good, but the two main issues bugging me were:

1. Anyone could do a ‘dig @ns.dnorth.net dnorth.net AXFR’ to retrieve a listing of all my DNS records - not great from a security point of view. This is a capability that should only be turned on for secondary DNS servers which need to fetch from the master.
2. The server would perform arbitrary lookups [for any domain] on request. This means it’s operating in ‘recursive mode’, which is a Bad Thing for various reasons.

The solutions were:

1. Add “allow-transfer { “slaves”; };” (without the double quotes) to the section of the configuration beginning “zone ‘dnorth.net’”. Then add a section defining the “slaves” access control list to be the local server, plus the secondaries: “acl slaves { 127.0.0.1; 123.45.67.89; }” replacing 123.45.67.89 by the IP address(es) of your secondary nameserver(s).
2. Add “recursion: no;” to the “options” section of the configuration.

Then restart the BIND9 service - on Debian, this is “/etc/init.d/bind9 restart”.

Health warning: Don’t do (2) above if you rely on your server to do its own DNS resolution - follow the crashrecovery tutorial above instead.

Well, last night I gave a talk at one of the ever-brilliant Oxford Geek Nights, and in case you're baying for the slides I used (all seven of them), you can find them at this wonderful OGN7 LastGraph Slides link. If you live within punting distance of Oxford, you should really try to come to the next one on August 27th. If not; well, they put videos up on the site... In a related note, I am enjoying the improvement on my slide-making capabilities via Inkscape. Previously, PDFs exported to massive file sizes and tying the individual pages together wasn't easy. Now, the new Cairo export in Inkscape (the same PDF export library LastGraph uses, yay) makes reasonably-sized PDFs, and pdftk munges them together easily and in record time. One day I'll give in and use an actual presentation program.

I find the different onomatopoeias used for the same sound in different languages entertaining, not least for the ensuing hilarity when you mispronounce them as English words. When I was in Paris with two of my housemates in December, we found a postcard which serves as a perfect illustration of why I enjoy mispronunciation. According to the back of the card, it depicts “Le coming-out du cochon”.

In the same shop, we found another card featuring a painting of a startled-looking boy playing his accordion, with a monkey dancing on his shoulder. I can’t decide how best to interpret the monkey’s expression.

If you’re not a massive nerd, I was just shown a joke which you might enjoy more than the rest of this post:

What do you call the blood that comes out when teenagers cut themselves?

Emoglobin.

---

On Thursday, Compsoc had a lightning talk session. Four people spoke:

• Andrew, the President, spoke about LastGraph (slides);
• I talked about ikiwiki, which we use behind the scenes for Compsoc’s website;
• Luke gave a talk about virtual worlds in general and Second Life in particular, and gave a demo;
• David spoke on “Windows Vs. Linux: Which Is Better?”, featuring a hilarious sketched-and-scanned graph which I would like a copy of.

I’m more impressed with Second Life than I thought I’d be. If only my laptop were speedy enough to run the thing, I’d take a look. (I tried to run it on my i855; the CPU pegged itself then everything died. Hard reset time!) I like the idea of a scripting language where everything is a state machine. It’s a bit of a shame that the only way to get code into or out of Second Life is to copy-paste between the built-in Notepad-alike editor and your proper editor of choice, which presumably does new-fangled things like “version control”. Oh well.

In case you care what I said about ikiwiki, I stuck my slides on the ’tubes. I’ve figured a few things out since the talk, such as how to get post-commit hooks working in Mercurial. Dom has some plans to add branching to Ikiwiki, so that you can make a test instance of the wiki to play with a new plugin just by making a new branch in svn or whatever. I like this idea.

S5 is pretty slick. I’d not used it before. (I didn’t actually use it directly, though: I relied on Pandoc to convert a file written with [Markdown] into an S5 slide show.) Anyway, there’s a lot to be said for having a slide show that you can just fire up in any web browser. (Except Andrew’s Konqueror apparently didn’t understand it. I wonder whether this is S5’s fault or Pandoc’s fault.)

Pandoc is a highly slick piece of software. It converts from any of Markdown, reStructuredText, HTML or L^AT_Eχ to Markdown, rst, HTML, L^AT_Eχ, man pages, ConTeXt, DocBook, RTF, or S5. It does all kinds of nice things along the way like curlifying quotes, making proper en-dashes, rendering L^AT_Eχ expressions as images so you can use it inline in Markdown and have it exported properly as HTML, and so on. It turns out to be a better Markdown processor than markdown itself, and supports extra things like tables, footnotes and so on. It turns out that the output from a practical I finished today was very nearly valid Pandockian table markdown, so I got a well-formatted project report pretty much for free. Oh, and for extra winning, it’s written in Haskell. I like the idea of being able to write man pages and documentation in something other than raw groff or DocBook XML.

I think the lightning talk format worked pretty well for Compsoc; it strikes me as a good way to get people involved. Nice work, committee!

The camel has two humps^[pdf] was an interesting read; a bit wooly, but believable. One of my favourite paragraphs, tangential to the paper's findings, concerned IDEs:

Programmers, who on the whole like to point and click, often expect that if you make programming point-and-click, then novices will find it easier. The entire field can be summarised as saying “no, they don’t”.

Being a very visual person, I naturally wanted a nice way of visualising the traffic lastgraph gets (especially as I nurse it through the initial round of bugs, most of which have gone). The ever-wonderful gltail always helps, but the same author has gone one further and come up with gltrail: It draws paths between pages visited, and looks pretty nice when visualising the lastgraph logs. It's also incredibly tempting to put it on a large screen in my room, the only problem being that I lack a large screen. And enough room. In a related note, it would be nice if there were some sort of AJAXy, real-time, web-based statistics like gltail; anyone know of any? Someone must have put that <canvas> element to good use by now...

Yes, this is my new official term for the server going up and down. It seems that some part of the system causes 100% CPU usage on the box it's running on, and in fact is so nasty to the system that it even makes the serial console lag. This, for those who are not server geeks, is Bad. I'm on the case, and have brought the full force of monitoring tools to bear to see if I can track down the cause. In the meantime, please accept my apologies for any downtime.

Yes, this is my new official term for the server going up and down. It seems that some part of the system causes 100% CPU usage on the box it's running on, and in fact is so nasty to the system that it even makes the serial console lag. This, for those who are not server geeks, is Bad. I'm on the case, and have brought the full force of monitoring tools to bear to see if I can track down the cause. In the meantime, please accept my apologies for any downtime.

After much work, lastgraph3's beta is now out. Please be warned that it may not work, may explode, and may abduct your cat, dog or goldfish in the process of failing. Apart from that, please feel free to give it a try at lastgraph3.aeracode.org. Report any and all problems back to me, either here, or at my email or something similar. Oh, and it's also very slow at fetching until I re-negotiate fast fetching with last.fm, since I've moved IP. Update: I've decided to make this one of those public, neverending betas, so LastGraph is now basically open for anyone and everyone to use. The last feature I added was LastGraph Premium, my new way of soliciting donations (this time, you can donate, and in return get a few extra features). I'm still not sure about it, but I've had some emails before from people who would have liked it, so we'll see how it turns out.

Something that's been bugging me for a while is that when I updated my Wordpress install (using svn), file uploads ceased to work. Attempting to upload a file resulted in the message "An error occurred in the upload. Please try again later.". It seems that the latest Wordpress uses Flash to show a progress bar during the upload. This site details many possible solutions, but the easiest one that's guaranteed to work is downloading the No-Flash-Uploader plugin. Simply drop it in your wp-content/plugins folder, then activate it in the Dashboard, and your uploads are fixed.

Something that’s been bugging me for a while is that when I updated my Wordpress install (using svn), file uploads ceased to work. Attempting to upload a file resulted in the message “An error occurred in the upload. Please try again later.”.

It seems that the latest Wordpress uses Flash to show a progress bar during the upload. This site details many possible solutions, but the easiest one that’s guaranteed to work is downloading the No-Flash-Uploader plugin. Simply drop it in your wp-content/plugins folder, then activate it in the Dashboard, and your uploads are fixed.

Having just put Windows XP back (in a dual boot configuration) I wanted to share the password manager that I have in the KDE system tray with Windows. I looked on the internet, and the only thing that has been ported is the pwmanager_dump program. However, combined with some the Windows ext2 driver, GNUWin32 tools (grep, awk, xargs etc) and a program that copies its command-line argument to the Windows clipboard, I have a hacked-up working pwmanager that syncs with linux.

I’m not 100% certain, but I think all of the tools I use are under the GPL, except for CopytoClipboard.exe, whose license I do not know. To get it to work, simply edit the first line of the bat file to point to your pwd file - of course you’ll have used the Windows ext2 driver to mount your linux partition to a Windows drive letter.

PwManager in Windows

It really isn't, honest. I'm working on the final part - the graph scheduler, which replicates the previous queued graph functionality - and I'll hopefully be pushing it all live soon. I was tempted to just push the live artist browsing part live, which all works currently, but it's only interesting for, oh, the first few hours...

I did a lightning talk to CompSoc last night on whether 64bit is ready for use on the desktop - if you’re bored, you can download the slides and the second of my famous sketch and scanned graphs.