A big thank you and well done to everyone who contributed to a great event last Thursday.
30 people from Government executive agencies and non-departmental public bodies (NDPBs) came to find out about website auditing.
Attendees were given an introduction by Alex Butler, COI’s Transformational Strategy Director. Alex told how this work came about as the result of the National Audit Office (NAO) report Government on the Internet (July 2007). The NAO report found that:
- a quarter of government organisations did not know the costs of their websites
- 16% had no data about how much their websites were being used
- quality standards had only improved slightly since 2002
Alex explained how the audits form part of a package of improvements offered by COI including standard measures for website quality, value for money and usability. These measures form part of the requirement by the Public Accounts Committee for greater accuracy and transparency in how the Government manages its investment in digital media.
Those present then watched a short clip of COI Chief Executive Mark Lund’s presentation at the Internet Advertising Bureau event IAB Engage 2009. Mark describes the work COI is doing with the Cabinet Office and other parts of Government in setting standards and evaluating the cost-effectiveness government websites.
Richard Foan, ABCe’s Managing Director, then talked through the details of website auditing. Richard emphasised the need for standards in an industry where there are so many different methods, metrics and tools to choose from. He explained that ABCe are an independent non-profit organisation, auditing to industry standards. ABCe certify that government websites comply with the standards and measure website usage in a consistent way.
The question and answer session that followed was lively and yielded some useful actions:
- We are planning an FAQ page on the COI website based on the questions asked
- Technical guidance will be available from the planned FAQ page but also from this page: TG116a ABCe Audit (PDF, 541 kB) and Guide to Google Analytics Mirror Method (PDF, 157 kB)
- Follow up meetings will be arranged with those who requested them
- Slides from the event are available from this page (PDF, 1.5 MB)
- Access to the Government Website Database is available by contacting graham.phillips@coi.gsi.gov.uk
Thanks to all those who attended, I hope you found it useful. And thanks particularly to Linda who worked the floor extremely well and made a star entrance!
For those who missed the event, we are planning to do similar events in the future. But in the meantime hopefully this blog article gives a flavour of what took place.
Posted in: Evaluation and measurement, Training and events
Tags: abce, audit, COI, evaluation, standards
Thanks for the write up and slides, which are informative.
On one hand, I applaud the effort to make reporting on cost and usage more standardised and transparent, as it’s very difficult to benchmark as things stand.
On the other, I can’t help but be disappointed that the ABCe approach seems a bit excessive for government sites. Frankly, I’d argue we don’t need the level of independent audit that an advertiser would demand, especially given this costs £2k x all government sites (total = £100k? £500k? more?).
Worse, from my perspective, users of low-cost, full-featured analytics tools like Google Analytics which collect data independently away from our platforms are penalised to the tune of another £150 per site, plus considerable technical costs to implement the mirroring solution (e.g. the guidance talks of additional dedicated servers and load balanced setups for high traffic sites, but even minor code changes and log extraction will probably double the audit costs for each site).
So, hurrah for the work and the sophistication of the approach you’ve developed. But is it worth stopping to check at this point maybe, that the tool you’ve found is really the best way to deliver the goal of trustworthy, comparable figures on government website usage, at a reasonable cost?
Steph Gray says:
January 28, 2010 at 8:25 amWas there any consideration of the proposal I put forward – namely, the use of a centralised installation of open-source analytics platform Piwik? A couple of lines of code inserted into every government page template had, I suggested, the potential to immediately – and at rock-bottom cost – guarantee standards-compliant data from all participating sites, addressing both key concerns.
http://puffbox.com/2009/03/31/coi-audit-open-source-analytics-piwik/
Simon Dickson says:
January 28, 2010 at 10:04 amSteph, thanks for the feedback. I would argue that the ABCe audit is a reasonable process to go through because it ensures that usage figures are being counted to industry standards. There are so many different tools and ways of implementing this stuff. Government needs an independent third party to enable any kind of benchmarking.
The cost of the audits at ~£2k per site represents good value for money for Government. COI negotiated a 50% reduction in costs compared to the market rate that advertisers would pay.
I take the point about Google Analytics users having to pay more but I guess the counter-argument is that the tool itself is free. In terms of the additional costs to implement mirroring, we are looking into the possibility of a central mirroring service that would make it cheaper and easier for GA users.
Simon, your proposal was considered however it deviates from the prinipcle that every site owner should be free to choose the analytics tool which best meets their needs. If a cheap solution to centralised data reporting is all that is required then Piwik may be a suitable choice, however I suggest that this isn’t the only consideration when selecting an analytics solution.
In any case and even with a single tool solution, separate audits would still be required to verify standards compliance… unless you are suggesting that web analytics for all government sites is actually managed centrally?
Adam Bailin says:
January 28, 2010 at 11:15 amMy proposal wasn’t that Piwik shouldn’t be used instead of a department’s own preferred analytics solution, but in addition to. By all means, let departments look elsewhere. In fact, industry guidance would often be to deploy more than one analytics tool anyway, to cover for any individual tool’s idiosyncracies and weaknesses.
Simon Dickson says:
January 28, 2010 at 11:52 amEven so, third party audits would still be necessary. Several sites run GA alongside their preferred solution. Installing another analytics tool on top of that starts to get quite onerous. We always look to work with what people are actually doing rather than imposing solutions centrally. In my experience, the latter rarely succeeds.
Adam Bailin says:
January 28, 2010 at 12:45 pmGreat seo information,I have Digged this site for future and will keep a eye on your other postings. Thank you
ubot studio says:
February 26, 2010 at 10:07 am