Today is World Usability Day and to mark the occasion, we here at Digigov have developed an introduction to low cost usability methods that might be useful to you in these austere times.

The Government announced its intention to cut spending on websites by up to 50%. But we need to deliver a consistently excellent user experience on the government web estate. So the question is how can we manage usability budgets most effectively and efficiently?

Clearly the most necessary aspect of usability is to test with users from the target audience.  Where the product or service is live, then feedback will be given through their use and feedback.  What can we do to get things right much earlier on?  To get it as close to effective for the user as possible? What can we do once the product or service is live and we want to check improvements?

People often associate usability only with user testing in laboratories, which can be expensive if there are many iterations.  In this article I will describe some alternative lower cost usability evaluation methods and look at the pros and cons of each method.  These don’t replace testing with users.  They are tools that offer getting to a usable product earlier with more efficiency.

Task analysis

Task analysis involves taking the tasks a user would do on your website and mapping out the individual steps required to complete each task. This type of analysis often yields a number of sub-tasks required to complete each individual step and each sub-task may be broken down further and so on. This approach is known as hierarchical task analysis. The purpose is to identify any steps that are unnecessary or that may slow the user down in achieving their desired goal. The process of stripping out unnecessary steps and working out the best way to design a given task can lead to a highly optimised and efficient user experience.

Pros

• Relatively cheap
• Focussed on user actions
• It can be used both on existing websites but especially when designing new services to identify any obvious extra steps that may be removed.

Cons

• Can be complex depending on the task in question
• Can be lengthy to complete a thorough task analysis, e.g. a complex website with many different functions
• Doesn’t identify any new tasks that may be appropriate for the site to deliver.

Task analysis assumes you are offering the right services and helps to optimise them but doesn’t provide any insight into whether you are designing the right thing.  That comes from user feedback.

Online surveys

Online surveys are an inexpensive method of eliciting user feedback. Crucially they allow people to say what they think about their online experience immediately after the event and therefore provide a good indication of people’s true feelings. Surveys are used to establish site ratings (e.g. satisfaction, ease of use etc) and to assess why people are coming to a site (i.e. the user goal) and the extent to which they achieve the intended purpose of their visit (i.e. goal completion).

Pros

• Low cost
• Close to the experience
• Provides insight into who your users are and why they are coming to your site

Cons

• Subjective and therefore not as reliable as observing users
• Can be annoying to users
• Takes time to analyse results and make actionable recommendations for improvement

There’s an art to questionnaire design, so we advise getting a professional to do it testing with a few people before going live. Don’t ask unneccessary questions, every question should have a reason for being asked. A useful starting point is the set of core questions defined in the COI guidance on survey design.

Heuristic evaluation

Heuristic evaluation is a kind of expert review where a website (or any user interface) is evaluated against a set of established design heuristics (or rules of thumb). The most commonly used set is Nielsen’s Ten Usability Heuristics. However there are others, for example Schneiderman’s Eight Golden Rules and  Don Norman’s principles of design. An experienced usability practitioner will evaluate against a the most appropriate set for the type of interface being evaluated and, dependent on experience, may have a bespoke set that works best for them.

Heuristic evaluation is most effective when carried out on selected user tasks. As the evaluator works through the individual steps to complete a task, usability problems are recorded along with the heuristic or design principle that it breaches. Positive points may also be noted but on the whole the purpose of usability evaluation is to find problems and make recommendations for improvement.

Pros

• Quickly identifies potential usability problems
• Useful to spot obvious issues early on in the design process
• Can save money spent fixing problems further down the line

Cons

• Effectiveness depends on the experience of the expert
• Less reliable than user testing because different experts interpret heuristics differently

Heuristic evaluation can be as effective as user testing in identifying usability problems and cost a fraction of the price. If using this method try to get the most experienced testers you can afford and agree up front how each heuristic will be interpreted to avoid any misunderstandings later on.

Cognitive and pluralistic walkthroughs

Cognitive walkthrough is where an expert walks through a task associated with your website, putting themselves in the shoes of a typical user. At each step, the following questions are posed:

• Will the user know what to do?
• Is the correct option available?
• Is the option is available, will the user expect that it will yield the intended result?
• Will the user know that the intended result has been yielded?

Examining each step in this way, the expert notes successes and failures along the way. As with heursitic evaluation, cognitive walkthroughs are used early in the design process.

Pros

• Low cost
• Task focussed
• Quickly identifies usability problems early in the design process

Cons

• Doesn’t test with real users
• Relies on expert’s ability to put themselves in user’s shoes

Cognitive walkthroughs assume a well defined set of tasks and steps are available to carry out the evaluation.

Walkthroughs can also be performed along with users,  domain experts and developers in a group scenario. This is know as a pluralistic wakthrough and can help to identify further usability problems due to the different types of participant involved.

Standards and guidelines

Standards and guidelines such as the W3C Web Content Accessibility Guidelines have been designed to take the hard work out of web design. Following established good practice can be an easy way to avoid many of the potential pitfalls that could lead to a poor user experience. There’s no point spending money on user testing or heuristic evaluation when the basic minimum standards haven’t been applied. In a government context the COI web standards and guidelines and the Usability Toolkit are a useful starting point.

Pros

• Low cost
• Someone has done the hard work for you

Cons

• Can lead to a tickbox mentality where standards compliance is valued above user needs
• Not necessarily user-centred

Compliance with standards and guidelines can be checked using automated tools but it usually requires an expert to do a full conformance inspection.

Summary

The methods introduced in this article are not intended as a replacement for user testing. User testing is still the most reliable way to get real user insight and generate improvements. Use these methods where appropriate and in support of other analysis, design and evaluation activities.

Read the full article (2 Comments)

The British Standard for Web Accessibility BS8878 has been published as a second draft and is available for comments until 30 June 2010.

As part of its ongoing involvement in the development of digital policies and standards, COI has worked with the likes of BBC, RNIB, IBM, Lloyds-TSB and Nomensa to develop this standard.

The Government has already adopted PAS 78, through TG102 Delivering inclusive websites, requiring websites to have and accessibility policy detailing the process for maintaining website accessibility over time. BS8878 builds on PAS 78 and provides a comprehensive approach to delivering web accessibility.

It’s not a technical standard with a strict set of criteria for compliance. It’s a process standard dealing with organisational responsibilities for ensuring web accessibility and looking at the entire production process including:

• Procurement
• Design and evaluation
• Maintenance
• Managing feedback

It also covers topics such as the UK Equality Act 2010 and the use of personalisation to deliver inclusive websites.

The standard is not a replacement of existing web standards, namely the W3C Web Content Accessibility Guidelines (WCAG). Rather, it is meant to work alongside WCAG 2.0 and link to it where appropriate.

There is still time to get your comments in on the BSI website, so please tell us what you think. We know there’s still room for improvement.

Read the full article (3 Comments)

The title comes from a recommendation of the Power of Information Taskforce.  They commissioned our team at COI to undertake a usability review of government websites, as our mission  is to improve the quality and consistency of government and the public sector online.  

The usability review found that there were many basic issues with the sites they looked at. Over 150 individual usability problems were identified including:

• Failure to let users know where they are in the site
• Inconsistency in page design putting undue stress on the user to recall rather than recognise (increasing cognitive load)
• Use of language that users couldn’t understand
• Linking to documents without clues to the content within

The Taskforce asked us to build a ‘usability toolkit’ setting out the basic usability criteria for government websites. Working with usability agency Bunnyfoot, we’ve developed a toolkit containing guidelines on best practice.  The toolkit has an emphasis on visual learning, making extensive use of graphics and video demonstrations and is designed around eight themes including page layout, navigation and writing content. Each theme has an interactive self-assessment exercise which tests knowledge and is intended to initiate discussion.

The toolkit is open to everyone to use, but there are advantages in registering as this helps users keep track what the content they’ve visited and the assessment tests they’ve completed.  So far 85 people have registered from over 20 different departments across government.

The most popular completed test to date is on page layout (123 completed tests).  It has guidelines about:

• Consistency between pages
• Devoting page space to content
• Screen resolution
• How to un-clutter content
• Styling text for readability
• Effective use of colour

It’s basic stuff but essential knowledge for anyone developing content for government, public sector, or inideed any other websites. There’s also a great video showing how we read text using eyetracking:

So, if you’re in the public sector, don’t waste another minute, register to use the toolkit! It’s easy, you just need to enter an email address and password and, if you work in central government, your department name.

Read the full article (5 Comments)

Government Digital Inclusion Champion Martha Lane Fox was the Big Thinker at COI last week. She announced the launch of her campaign to Race Online for 2012, which aims to get more people online particularly from socially excluded groups.

Research by Price Waterhouse Coopers shows that 10 million adults in the UK have never used the Internet. 4 million of those are socially excluded, of which 38% are over 65 and 39% unemployed, and it is these that people that Fox is targeting. Her speech was inspiring. I particularly liked her ‘user-centred’ approach. She told three stories from around the country:

• A teacher at a computer literacy centre in Lambeth revealed that many of the students couldn’t read.
• A builder in Birmingham gets the majority of his work by looking online.
• Children on a rough estate in Bristol learned about growing vegetables from older residents, imparting computer knowledge in return.

Martha stressed that it isn’t about IT training, it’s about using computers for relevant purposes such as looking at photos of your grandchildren living abroad, saving money by shopping online or looking for jobs. People aren’t inherently interested in computers but may want to go online if they see that it can improve their quality of life.

Later in the week, I went to the BBC to hear about their new Accessibility Toolkit 2.0 (ATK 2.0) from Jonathan Hassel, Head of User Experience Design. His aim is to improve online experiences for disabled people.

There are 11 million adults in the UK with a long standing health problem or disability that affects their daily activities including their ability to work – and therefore covered by the Disability Discrimination Act. Of these, according to research from the Office for Disability Issues, 47% are over 65 and 43% are unemployed. Startlingly, 58% have never used the Internet.

Jonathan talked about the various barriers to disabled people getting online including lack of interest, lack of means and lack of confidence. These are the same reasons as for the population at large. BBC research into encouraging broadband adoption echoes the experiences of Martha Lane Fox. They focussed on the 21% of UK adults who do not have the Internet at home or use elsewhere. The figures are similar: 10.5 million aged 15+ with and average age of 61 (over half were 65+) and 67% are C2DE compared with 45% of the UK population.

The BBC’s research found low levels of interest (68%), low intention to acquire (82%) and low knowledge (81% knew little or nothing). The primary barriers were low perceived benefits versus cost and low confidence and skills. Secondary barriers include basic affordability, literacy and social exclusion. One participant claimed that:

“There’s nothing on there that you couldn’t get from Teletext.”

The challenge to get people online was acknowledged as being very difficult. Previous messages have failed because messages were not targeted and therefore not perceived as relevant or beneficial. Four broad areas were identified to aid adoption:

• Families and friends
• Media skills
• Easier home internet access
• Affordability and cost

Interviews with recent adopters revealed the power of friends and families, with one man saying:

“I wasn’t confident on the computer until (my son) started showing me bits on his computer.”

They also revealed that people were scared computer courses and the possibility of public humiliation in front of peers. It’s not enough simply to communicate the availability of courses.

The focus then returned to disabled audiences and Jonathan developed an interesting argument around web accessibility. The most common approach to date has been to focus on delivering inclusive websites. That is, to try to build websites that work for as wide an audience as possible. (Note on terminology here, this isn’t inclusion in the same sense as previously described, which is more about getting people online.) Jonathan’s argument is that this doesn’t work because people have such diverse needs that one design will never work. Websites do allow personalisation through operating system and browser settings but most people don’t know how these work. There’s also the AAA approach which places personalisation controls directly on to the web page. Most people don’t know what “AAA” means either. So the BBC have developed a prototype solution that brings personalisation controls into the browser but through an intuitive user interface.

The idea is that you have preferences for a website, accessible via a link at the top of the site. Clicking on this reveals a set of default options recommended by people with different health problems or disabilities. For example, you can set the colours, text size, font size and weight and so on. This isn’t revolutionary but what’s different is the user interface that the BBC have designed. It’s the first time it’s been done intuitively. And anything that helps create a positive user experience for people online is a good thing. After all, if we’ve worked hard to convince people of the benefits, the last thing we want is for them to switch off because of a usability or accessibility issue. Lets make sure it works for people when they arrive.

Read the full article (5 Comments)

Today the official accessibility guidance was updated to allow a choice of either WCAG 1.0 or WCAG 2.0 as the minimum standard for public sector websites:

Compliance with the Web Content Accessibility Guidelines is acceptable at Level Double-A of version 1.0 or the equivalent level in version 2.0. Future policy and timetables for implementation will align with European Commission recommendations on the adoption of WCAG 2.0. Planned future updates to this guidance will include details of the specific conformance requirements for version 2.0.

- Paragraph 5, Delivering inclusive websites (TG102)

My last post on accessibility suggested it would be a good first step. The suggestion was generally supported, not least by Jack Pickard, who since his initial response has written a more detailed article. In this, he sets out what he thinks the appropriate level of WCAG 2.0 should be for public sector websites. It’s an excellent article and will certainly help COI to develop the accessibility guidance further and smooth the transition to version 2.0. If you have a view on this, I urge you to read Jack’s article and comment.

Read the full article (6 Comments)

In June 2008, COI published the guidance Delivering Inclusive Websites (TG102) with the aim of making government websites easier to use for everyone, including people with disabilities. This set the minimum standard of accessibility for all public sector websites at WCAG 1.0 Level AA in line with the The Riga eInclusion Declaration.

Since then WCAG 2.0 has become a recognised W3C standard and the general feeling is that it’s a better standard than version 1.0. There’s greater focus on the user and less focus on the technology. It’s not just about HTML now; it’s technology agnostic and based on user outcomes rather than technical specifications – which can only be a good thing.

There is certainly a lot of activity in the European Commission looking at adoption of WCAG 2.0, what Member States are doing and what a common approach might mean. Sadly, there doesn’t appear to be much in the way of clear direction from the Commission… so we might have to come with a plan on our own!

Having discussed this with several colleagues in government, agencies and user community representatives, it’s clear that we need a plan for transitioning from version 1.0 to version 2.0. Moreover, Government has committed to reviewing its current policy:

At such time that version 2.0 becomes a W3C Recommendation, this policy will be reviewed within six months. Consideration will be given to the adoption of version 2.0 as the minimum standard for public sector websites.

So what might a plan to transition from version 1.0 to version 2.0 look like?

Well, assuming that we want the transition to be as smooth as possible, a first step would be to update the current policy to allow conformance with version 1.0 Level AA or the version 2.0 equivalent. This is the current working position that we have been advising departments unofficially. Allowing a choice of standards for a period of time seems like a fair stance to take, particularly given that the current target for complying with version 1.0 Level AA is December 2009 for central government department websites. A sudden jump to version 2.0 would risk jeopardising a lot of the work that has already gone into reaching the current minimum standard.

The following steps also seem sensible:

1. Accurately define conformance criteria for version 2.0 that are as close to the current target of version 1.0 AA as possible.
2. Allow site owners the option of adopting either version for a transitional period.
3. Re-assess the conformance levels for version 2.0 and put in a timetable for moving to the appropriate level of conformance (e.g. version 2.0 Level AA).
4. Put deadlines in place for a move to a version 2.0-only system, allowing sufficient time for organisations to update their training, policy documents, accessibility statements etc. (e.g. December 2010 in line with Executive Agencies and NDPBs).

We need to manage the transition from version 1.0 to version 2.0 carefully, allowing sufficient time for the new standard to “bed in”.

We should also be careful not to focus solely on WCAG when addressing the accessibility debate. We should always talk about usability as well. The focus is on the user.

I welcome any views on the suggested approach, particularly on the appropriate level of WCAG 2.0 to aim for and on the length of time required to make the transition.

Read the full article (12 Comments)