Government And Website Launches

On Tuesday the UK Police service launched a new website, with much fanfair, that allows you to see the crimes that have happened near where you live. This is part of the Government’s commitment to make data much more freely available than it has been in the past.

People are nosy. There’s no way around the fact that we just love to see what our neighbours are doing and to investigate the area around us. Every time a website like the crime maps, a new census, or noise maps launches the same sequence of events happen. The are news stories on TV, on radio and online and then the website crashes. We end up with stories like this and people will, in general, never go back.

If you investigate where this new site is hosted you’ll find it’s on RackSpace, who offer a cloud hosting service so really there’s no reason for the site to go down. Although the site is interactive there is no user generated data so the site should be easily scalable.

The site cost £300,000 to develop, and although 75,000 hits a minute is a lot surely will a little bit of thought the site should have scaled fairly easily?

DjangoDE 0.1 Released

I spend most of time at work building websites in Django. My editor of choice until now has been Kate with Chromium open on another screen. Most of my co-workers use VIM inside a PuTTY session to do their editing. With multicore machines with gigabytes of RAM, surely there’s a better way?

I investigated the current state of Django IDEs and came to conclusion that none of them are that great. Most are a plugin to a giant IDE that tries accommodate many different languages, so each feels like second best, or they are designed more for traditionally programming and not webdevelopment so they don’t integrate with Django’s in built server and don’t support editing Javascript, or provide a built in webbrowser. I also don’t want to have to pay for the editor, which limits my choice even more.

Having decided that none of the existing IDEs quite fit my requirements I did what any self respecting open source programmer with an itch would do, I headed to Google Code, created a new project and then got coding.

Can the entrance barrier ever be too low?

Yesterday Google announced a new feature for Google Code’s Project Hosting. You can now edit files directly in your browser and commit them straight into the repository, or, if you don’t have commit privileges, attach your changes as a patch in the issue tracker.

If you’re trying to run a successful open source project then the key thing you want is more contributors. The more people adding to your project the better and more useful it will become, and the more likely it is to rise out of the swamp of forgotten, unused projects to become something that is well known and respected.

It’s often been said that to encourage interaction you need to lower the barrier so that people can contribute with little or no effort on their part. Initially open source projects are run by people who are scratching their own itches, and producing something that is useful to themselves. Google’s intention with this feature is clearly to allow someone to think “Project X” has a bug, I’ll just modified the code and send the developers a patch. The edit feature is very easy to find, with a prominent “Edit File” link at the top of the screen when you’re browsing the source code so Google have clearly succeeded in that respect.

Rsync backups to Amazon S3

Having recently got married I wanted to make sure all the photos taken at the event are safely stored for the posterity. I thought I’d take the opportunity of making sure that all the rest of my photos are safely backed up, and that any new ones are also backed up without me needing to do anything.

One of the simplest places to keep your backups is Amazon S3. There is essentially an unlimited amount of space available, and it’s pretty cheap. rsync is a great tool to use when backing up because it only copies files, and parts of files that have changed so it will reduce the amount of data transferred to the lowest amount of possible. With S3 you not only pay for the data stored, but also for the data transferred so rsync is perfect. So, how do we use rsync to transfer data to S3?

I won’t go through setting up an Amazon S3 or creating a bucket, the Amazon documentation does that just fine.

Using Python Logging Effectively

If you’re writing a Python program that doesn’t have a text-based user interface (either it’s a GUI or runs as part of another program, e.g. a webserver) then you should avoid using the print statement. It’s tempting to use print to fill the console with information about what your program is up to. For code of any size though, this quickly devolves into a hard to navigate mess.

Python’s standard library contains a module, logging, that lets you write code to log as much information as you like and configure what you bits you are interested in at runtime.

There are two concepts that you need to understand with logging. Firstly there is the logging level. This is how you determine how important the message is. The levels range from debug as the least important, through info, warning, error, critical to exception, the most important. Secondly there is the logger. This allows you divide your messages into groups depending on the part of your code they relate to. For example, you might have a gui logger and a data logger.