I am really proud of my government colleagues for getting the data together to enable us to publish the very first report on the costs, quality and usage of central government websites. You can find it, along with all the data, on www.coi.gov.uk/websitemetrics2009-10

 Local government has been comparing like for like to benchmark for many years.  It is more difficult in central government as the audiences and functions of websites vary considerably.  Some are for the public (Diectgov, NHS Choices), for businesses (businesslink.gov.uk) and for public sector workforces (civil servants, teachers, armed forces).  And others are for people to engage with a particular organisation in policy formation (corporate websites).  While yet others have a regulatory function (Ofgem).

It’s made harder by some public bodies doing lots of syndication, placing information onto other websites where people regularly go, and early adoption of re-usable information and data, so that people can present it in new ways.  Both these result in people using the information, but not being recorded as visiting the website to do so. So the cost per visit is not related in any way to the cost per use, nor indeed the value to the user.

The data makes for interesting reading if not immediate analysis.  The notes that reporting public bodies have attached should be read in conjunction with the data.  However, some things do jump out.

One that struck me was the variation in cost for hosting and infrastructure compared to usage.  One wouldn’t expect a bell-shaped curve, as some sites need to have much more resilience and security than others, but if a free competitive market was really in operation, then one would expect more clustering around a few price points.

The other areas of expenditure will vary more according to the stage of development.  This year you might spend a lot on design and build to bring it up to what people expect, next year perhaps very little. Per year websites tell you something, but are not the whole picture – but how much better than not having anything!  Having knowledge of the order of the cost is really helpful and, even taking out all those who didn’t get anything from their visit, the cost for each reach is very low compared to any alternative.

And we can begin to look at some other interesting aspects.  Does the spending made on testing and evaluation result in an increase in user satisfaction?  By releasing the data, we’re keen to see what use people make of it and how they can add insight into the variety of questions that arise. Ross Ferguson, for example,  has done a visualisation of the number of yearly unique visitors/browsers (a figure that not everyone could provide as it requires deduplicating across all the monthly data).

We know people are interested.  In the three and a bit days since publication, there have been 14,534 downloads of the PDF and 1,034 downloads of the CSV.  It’s been interesting too to follow the use of the official bit.ly tag that we tweeted and discover where people find where the data is.  We’re looking forward to seeing what people do with it.

In terms of numbers, 46 is a very small sample.  Next year all the open central government websites are due to report and it doesn’t seem sensible to issue a report simply listing the data as we have done this year.  (We will of course issue the data as a dataset.)  How people analyse the data will help shape the human-readable report.

Let us know what you’re doing, and what you’d like to see.

Read the full article (10 Comments)

In the current climate of open, transparent and accountable government, it is now mandatory for government websites to have stats audits. But how did this come about and why is it beneficial?

Policy background

Back in July ’06 the National Audit Office published the results of its survey of Government on the Internet. The results were pretty shocking:

Over a quarter of government organisations still do not know the costs of their websites, making it impossible to assess whether they are value for money

16% of government organisations have no data about how their websites are being used, inhibiting website improvements.

The quality of government websites has improved only slightly since 2002.

These findings were used as evidence before the UK Parliament’s Public Accounts Committee (PAC) hearing in November ’07. PAC recommended the development of a single set of measures for government website costs, quality and usage which were to be reported centrally. Government’s response to the PAC Sixteenth Report was laid before the House of Commons in September ‘08.

Consistent data

The single set of measures was developed and is now in place, but how can the data be collected reliably? Measuring website usage can be done in a number of ways with sites using different methods, tools, standards, filters and terminology. To get consistency is a real challenge.

The media industry has solved this problem. Advertising revenue is based on the number of Ad Impressions – like Page Impressions but for ads – and rates vary with volume of site usage. Advertisers need a reliable way to ensure return on investment. They need to know that the websites on which they are buying space and surfacing content measure usage accurately and consistently. The solution is to insist on a site audit certificate.

Government websites don’t tend to generate revenue from advertising – although the practice is not forbidden in principle – but they are accountable to the taxpayer. Surely taxpayers have the right to expect a decent return on their investment? If I visit a government website, how much does it cost me? Is it value for money? I want to know!

The ABCe audit

In May 2009, COI appointed ABCe to be the sole auditor of government websites. ABCe is the industry owned website auditor and is the standard for the media industry, both for media owners and media buyers. COI has negotiated cost savings for the taxpayer by centralising the spend. The average cost of an audit is approximately £2,500 compared to £4,000 if departments went to ABCe independently. By the end of the financial year, all websites run by central government departments will have had one month’s usage data audited by ABCe.

The bigger picture

Why go to all this trouble and is there any benefit to the government departments themselves? Aside from increased accountability to the taxpayer, departments do stand to benefit from the increased rigour in site measurement and evaluation. Website audits are the first step towards properly managed performance improvement. It is only with consistent and reliable data that performance metrics – or KPIs – can be developed. These are things like:

• Average number of Visits per Unique User which measures how often a user returns to a website (customer loyalty)
• Average number of Page Impressions per Visit which provides a measure of user engagement (sometimes referred to as stickiness)

When usage levels are considered alongside costs, we can also begin to consider value for money metrics such as Cost per Visit.

Central reporting of quality data also enables benchmarking of government websites against each other. For example, if I get an average Visit Satisfaction of 70% for my website, how do I know if that is good or bad compared to other websites in my sector? With a standard set of core survey questions, this is now possible. It is also worth mentioning that local government are ahead of central government in this respect. Because of initiatives like the SOCITM Website Take-up Service and Gov Metric, Local Authorities have integrated satisfaction benchmarking into their site performance management.

Monitoring KPIs over time is a key business tool for demonstrating performance improvement which is so important for getting the appropriate level of investment in government digital media.

Central reporting of Visit Duration is a contentious issue. While it is probably not useful to compare websites on this metric – a long time on site may indicate a high level of engagement or a site that is difficult to navigate – it does provide interesting census-level data. Measuring Visit Duration enables Government to calculate the total amount of time spent on its websites by citizens. We can begin to get a picture of the value delivered to citizens by government online. For example, if we compare the cost of delivery to the cost for the citizen then we can begin to address the cost-benefit of online services to the citizen. Now that would be interesting!

Read the full article (11 Comments)