What happens online is only part of the overall process. Due consideration should be made to the overall planning and fulfillment. This involves work on the aims, audience segmentation, preparatory materials, promotion and facilitation, registration, moderation, media handling, structuring the document, alerting senior staff, dispute handling, and communications as a result of the comments.
Audience segmentation
To assist in preparing ‘The aims’, an audience segmentation is useful to determine a clear articulation of the responders expected and the kind of response sought.
Preparatory materials:
It may be useful for members of the public to have preparatory materials available that they could use, for example a video introduction or background links to relevant material. Preparatory materials could range from basic concepts to more detailed information that could help the deliberative process to run in a more informed way.
Volume assessment
The audience segmentation will give a good feel for the response and whether it is likely to be low, high or high for a very short period, for example when hundreds of journalists may download the document as the commentable document is published. Good practice in the last case is for a separate opportunity to download the document in advance of reaching the commentable document itself.
Enable minimal registration
Enforcing registration before being able to make comments is much debated. It raises questions in some minds as to what the government will do with that information. The process is usually used to check that the responder is a citizen and not a device and, in general, it is considered reasonable to request and require a name and valid email. The latter can also be used to communicate to a commenter if needed.
Many contributors expect to find out what happened as a result of the commentable process. If, and only if, stated in the section on ‘What will happen to your comments’, it would be acceptable following closure to write to all, thanking them for contributions and including a summary of the outcome.
Facilitating a response
Responders may want to comment to documents at a number of different levels, including the overall approach and detailed proposals. Splitting into paragraphs or sections without thought may provide neither such an opportunity, nor fit into people’s behaviour. The following are some questions that may help guide the final form:
- Does the overall document need an introduction, perhaps a video, to explain purpose and aims?
- What are the key areas that would most help obtain a response?
- What activities could keep momentum? (for example, tweets, webinars with Ministers, online chat ).
- What are the different audience segmentations most likely to do?
- Is this a document people may want to print out and peruse before commenting?
- How would it appear to someone with only 5 minutes, or 15 minutes or 50 minutes to spare in which to make a comment?
Structuring
To ensure the document structure is mindful of the conventions and restrictions of web publishing, approaches to structure should be addressed at the earliest possible opportunity during its preparation.
- It is advisable to split documents into sections where appropriate. However, a balance between page length and number of sections should be sought.
- Users will often prefer to scroll than to wait, however, they may also be confused if disparate ideas are not compartmentalised.
- Maintain consistent semantic mark-up throughout the document, ensuring that it accurately reflects document structure. This should ensure that documents are accessible on a number of viewing platforms (small-screen rendering, screen readers etc)
- Disable commenting on pages where it is not beneficial or appropriate. This increases prominence of the comment form on pages where it is enabled.
- Provide a comprehensive contents listing, some users may only wish to comment on specific areas.
Separation between facilitation and moderation
The success or failure the number and range of comments depends critically on two factors: facilitation and moderation. These have different roles and skills that are required.
- Facilitation: Facilitation is about promoting citizen dialogue on certain topics and requires a set of skills (able to engage online, personal savvy, experience on deliberation etc). Facilitators should have some background on the topic and could come from policy areas of the organization. Their tasks are proactive and include summarising, responding and encouraging comments.
- Moderation is also critical for the success of the online participation experience. The task is to keep the “rules of the game” clear and enforce them, specifically what is laid out in the ‘Moderation policy’, dealing specifically with possible conflicts and behaviours that are not acceptable online. Moderators are responsive and should come from a neutral place.
Independent oversight and arbitration
In developing a ‘Moderation policy’, it should be borne in mind that there is a tension between the values of freedom of expression and good management that provides a space where people feel free to comment. An out of line comment can affect others’ desire to comment and there will be people wanting to test that line.
The practice of moderation may be contested in public. An appropriately independent arbitration process should be established to whom complaints can be made.
Inform your press office
Each and every time a commentable document is released, your press office should be informed as part of the preparation and progress during. Comments may arise that leads to other online activity, for example in blogs, or in the broadcast media. Your press office will be able to advise how to keep the process open and transparent.
Feedback to responders
Your process for feedback to responders will have been outlined in ‘What will happen to your comments’. One common practice is for the facilitator to make a final comment thanking contributers and summarising what has been taken from the comments. Unless following the Consultation Code, there is no reason to respond to each and every comment and idea. However, if a summary of the comments is reported back to the public it should represent all views submitted in a fairly and balanced way, unless indicated otherwise at the start (for example selecting only the most popular ideas).
Commentable time period
You may wish to determine in advance how long people may continue to make comments or to decide when comments fall sufficiently infrequently to close.
Evaluation
Commentable documents should be evaluated. Measuring and sharing evaluation can improve the effectiveness of all engagement activities.
It is best if evaluation is built in from the beginning of the process. When designing an evaluation framework it is helpful to consider:
- Outputs: Immediate indicators of visits such as, but not limited to, the number of visits, dwell indicators, bounce rates, page per view and search terms.
- Out-takes: Satisfaction of users and their understanding of the process. Indicators used could include rating, feedback and e-survey mechanisms.
- Outcomes: The overall aim sought through use of commentable documents. Proxies to this can be, but are not limited to, total number of ideas posted, number of comments, number of active contributors, and number of removed/invalid comments.
- Cost-Effectiveness: The resources allocated to the exercise including time of the staff in charge of running it relative to the outcome.
The four items are interrelated, for example dwell indicators do not in themselves show success because this depends on the aim of the website.
Next section: Technical considerations
It is excellent to see the form and structure of online commentable documents being considered in this consultation. However, it might be worth including some guidance on the use of images and tables, for example. With WriteToReply, we have found that the use of large tables in documents can be difficult to represent within the confines of commonly used web templates (i.e. the commentariat or digress.it themes for WordPress). Links to online spreadsheets or enlarged versions of detailed images are possible ways to overcome this, but if consideration can be made at the authoring stage of the documents, all the better.
Joss Winn says:
July 21, 2010 at 12:21 pmAmong the ‘preparatory materials’, a way to jump into the consultation at the main points is useful for readers and can help authors by directing comments at the key parts of the document. i.e. http://writetoreply.org/govictstrategy/quick-start/
Additionally, guidance on linking to the different sections of the document from other websites might also be useful in amplifying the discussion around the document to elsewhere on the web. i.e. http://writetoreply.org/govictstrategy/help/
Joss Winn says:
July 21, 2010 at 12:32 pm[Facilitation and the Press Office]
Where a document is a long one, consulting on different themes, it may be appropriate to try to engage different audiences with different sections of the document. Identifying very specific contentious issues or highlighting specific “headline” claims or “surprising facts” can be used in social media settings to try to pull readers in (in the first instance) to a very particular part of a document.
Tony Hirst says:
July 23, 2010 at 1:20 pm[Structuring]
Does the requirement to publish data alongside reports (e.g. as considered in “TG135 Underlying data publication”) include consultation documents?
It is likely that only simple data sets would be included in consultations, which may be achieved using simple HTML TABLEs and progressive enhancement to either style a table with sortable columns, for example, or render it as a simple chart.
Alternatively, it may make more sense to either link out to data elements, or pull them in from another service and embed them within the page. As Joss hints at, careful consideration of how to achieve this in practical terms given the workflow used to produce a document may be required.
Tony Hirst says:
July 23, 2010 at 1:25 pm[Feedback to responders]
Where comment threading is supported, or where commenters start to respond to each others’ comments, it may be fruitful to support a mechanism where interested parties can keep track of comments appearing on/discussion around a particular part of a document.
For documents hosted on blog like platfroms such as Wordpress, support is often built in for commenters to either subscribe to an RSS comments feed from the whole blog, or a particular page on the blog, or sign up for an email alert if anyone else posts a comment to a page you have commented on yourself.
One thing we have explored on WriteToReply is the use of “comments by commenter” feeds, which allow comments made by a particular individual to be tracked. One possible use case of this is for one or more commenters to use a commentable document as a tool for drafting a more formal/comprehensive response to a document that could be submitted by other means. A potential downside of this approach for the consulting party is duplicate submissions by different routes.
Tony Hirst says:
July 23, 2010 at 1:33 pmAgree strongly re considering different ‘layers’ or audiences with different degrees of interest, expertise and time – though perhaps describing this as audience segmentation is taking it a bit far.
It may be worth underlining the importance of ensuring that a channel remains for stakeholder/professional organisations to respond on behalf of a group of members without impeding this (e.g. an email address).
Formal consultations should also be marked up using RDFa, on the parent department site, and the commentable version would presumably be a link with relation arg:replyOnline ?
Steph Gray says:
August 2, 2010 at 4:23 pmShould you perhaps spell out what happens when the commentable period closes? Ideally, posted comments would remain visible for a period of time, with clear labels indicating that the period for comments has closed, and providing a link to any organisational response once this is complete.
Perhaps the point to emphasise is that the commentable document is a small cog in the digital engagement wheel, not really a project in itself. It represents an opportunity to maintain stakeholder and customer relationships, but needs to be sustained with follow-up using other channels, and be considered in the context of those channels (i.e. some stakeholders will be meeting ministers, receiving email alerts, and being invited to several commentable consultations a year)
Steph Gray says:
August 2, 2010 at 4:28 pmThis is quite useful, clearly, as promoting discussion encourages subject matter experts to contribute where they may not be directly involved initially. Provision of a ‘subscription to comments’ opt-in, where appropriate, is a great opportunity for allowing and encouraging people to join the debate thus enabling ‘digital conversations’.
Margaret Manning says:
August 3, 2010 at 5:20 pm