Drawing up the Local Spending Data guidelines… and how Google Docs saved the dayPosted: September 13, 2010
Last Thursday, the Local Public Data Panel on which I sit approved the final draft of the guidelines for publishing by councils of their spending over £500 (version 1.0 if you like). These started back in June, with a document Will Perrin and I drew up in response to a request from Camden council, and attracted a huge number of really helpful comments.
Since then, things have moved on a bit. The loose guidelines were fine as a starting point, especially as at that time we were talking theoretically, and hadn’t really had any concrete situations or data to deal with, but from speaking to councils, and actually using the data it became clear the something much firmer was needed.
What followed then was the usual public sector drafting nightmare, with various Word documents being emailed around, people getting very territorial, offline conversations, and frankly something that wasn’t getting very far.
However, a week beforehand I’d successfully used a shared Google Spreadsheet to free up a similar problem. In that case there were a bunch of organisations (including OpenlyLocal, the Local Government Association and Department for Communities and Local Government) that needed an up-to-date list of councils publishing spending data, together with the licence, URL and whether it was machine-readable (Basically what Adrian Short was doing here at one time – I’d asked him if he wanted to do it, but he didn’t have the time to keep his up-to-date.) In addition, it was clear that we each knew about councils the others didn’t.
The answer could have been a dedicated web app, a Word document that was added to and emailed around (actually that’s what started to happen). In the end, it was something much simpler – a Google spreadsheet with edit access given to multiple people. I used the OpenlyLocal API to populate the basic structure (including OpenlyLocal URLs, which mean that anyone getting the data via the API, or as a CSV would have a place they could query for more data), and bingo, it was sorted.
So given this success, Jonathan Evans from the LGA and I agreed to use the Google Docs approach with the spending guidelines. There are multiple advantages to this, but some are particularly relevant for tackling such a problem:
- We can all work on the document at the same time, messaging each others as we go, avoiding the delays, arguments and territoriality of the document emailing approach.
- The version tracking means that all your changes, not just those of the saved version are visible to all participants (and to people who subsequently become participants). This seems to lead to a spirit of collaboration rather than position-taking, and at least on this occasion avoided edit-wars.
- The world can see the product of your work, without having to separately publish it (though see note below)
You can also automatically get the information as data, either through the Google Docs API or more likely in the case of a spreadsheet particularly, as a CSV file. Construct it with this in mind (i.e. 1 header row), and you’ve got something that can be instantly used in mashups and visualisations.
Important note 1: The biggest problem with this approach in central government is Internet Explorer 6, which the Department of Communities & Local Government are stuck on and have no plans to upgrade. This means the approach only works when people are prepared to make the additions at home, or some other place that have a browser less than 9 years old.
Important note 2: Despite having put together the spending scoreboard spreadsheet, we were hopeless at telling the wider world about it, meaning that Simon Rogers at the Guardian ended up duplicating much of the work. Interestingly he was missing some that we knew about, and vice versa, and I’ve offered him edit access to the main spreadsheet so we can all work together on the same one.
Important note 3: A smaller but nevertheless irritating problem with Google Documents (and this seems to be true of Word and OpenOffice too) is that when they contain tables you get a mess of inaccessible HTML, with the result that when the spending guidance was put on the Local Public Data Panel website, the HTML had to be largely rewritten from scratch (by one of the data.gov.uk stars late at night). So Google, if you’re listening, please allow an option to export as accessible HTML.