Les, winner of more IPA Effectiveness Awards than anyone in the history of the competition, and co-editor of the COI’s Payback and Return on Marketing Investment(ROMI)in the public sector (PDF1.2Mb), explained the ten steps outlined in the document.

He opened on the importance of setting clear objectives; and understanding how a campaign will work and whom it will affect. With this in mind he gave some top tips on measurement:

• Think about evaluation before the campaign.
• Allocate time and money for measurement – and analysis of data.
• Metrics should reflect objectives and strategy.
• Tracking for diagnosis, behaviour for evaluation.
• Direct response is not the whole picture – by a long way.
• Measure actual (not claimed) exposure to campaign
• Plan activity with evaluation in mind. Build in tests and controls if possible.
• Measure and take account of other factors.
• Track all measures pre and post, and look beyond the short term.

This led to the difficult bits: how do you isolate the effect of a campaign on behaviour change from the effects of other factors – and how do you put a financial value on success?

Les presented a number of techniques to deal with the former and an interesting discussion arose about the difficulties of isolating the effects of smaller-scale or non-campaign-based activities where there is too little data or budget. It was suggested that one way around this is to look at the overall payback from observed behaviour change. Given the cost of the campaign, what proportion of this would it have to claim in order to have paid for itself? How much to have paid for itself twice over?

On financial valuation, it was clear that although there are sometimes straightforward proxies (like the money saved on supply teaching by recruitment of permanent staff), policy and economics teams in departments should be consulted for the latest on the ‘value’ of behaviour change to the exchequer.

Finally, but most importantly, came the debate about what to do with the results. If campaign A pays for itself several times over but campaign B barely breaks even, should campaign B be axed? The answer was no – at least not on this evidence alone. It might be that more should be invested in campaign B so it can reach a ‘tipping point’. It might be that B is past the point of greatest financial return but is rightly seeking to influence a hardened minority as part of government policy.

In all cases, Net Payback (the absolute value of benefit delivered, less the cost of the campaign) and ROMI (the number of pounds of Net Payback for every pound spent) should be seen as just one part of the evaluation of government marketing communication.

So why try to calculate Return on Investment at all? Because there are real benefits from undertaking the analysis:

• It requires a dialogue with policy colleagues about the contribution of marketing communication alongside other policy levers – such as legislation and intervention;
• It leads to a greater understanding of how marketing communication delivers value and the need to quantify and isolate campaign impact on behaviour change drives better evaluation;
• By speaking the same language and working to the same rules as accountants and economists, it gives more credibility to marketing communication and allows it to be considered as an ‘investment’; and
• In an era of economic uncertainty, it is one way to demonstrate the value of marketing communication to the taxpayer and ensure that spending decisions are made ‘intelligently’.

The COI team are working to turn their beta version report into two guides; one simple and one technical; by mid-April – and they would love to hear your views – either on this blog or direct.

In a wide ranging, stimulating and challenging seminar we covered some important themes:-

How organisations can reconcile hierarchic control and open democracy,how learning new things involves chaos and uncertainty first before a new consensus emerges,how the internet is changing the culture ,how “doing” is the now the best way to learn what works,how setting up experiments can be better than spending lots on predictive research, how easy and quick it can be to set up experiments and how if you want to understand the new social media you have to leap in and have a go.

Via email, you receive a link to film from a friend who suggests you look at it. Clicking on it, you find a film that is viewed from the perspective of a driver looking through a windscreen. Your mobile phone rings as the film plays and, in that split second of being distracted whilst answering, you run into a cyclist who crashes sickeningly against your windscreen. You realize that you have caused the accident by answering the call.

The long view

Richard’s career started at a time when the print technology would be recognisable to a contemporary of Gutenberg and in the space of 35 years media has completely transformed and in particular what it means to be a journalist.

24/7 and multiplatform

When he started at the BBC he might have developed and revised a news story three times in a day–interspersed with breaks for refreshments at a local hostelry-for three broadcasts on radio.
Now the news is 24/7 and delivered across different platforms.

BBC restructured to reflect this change

Two examples

1. Single editors across platforms rather than a different editor for TV/radio/online.
   Which cut costs by 20 % as well.
2. Story communities now come together at the BBC to access the knowledge and expertise of the whole corporation on an issue.

It has only just begun

The BBC has moved from being a siloed linear broadcaster to something quite different. But in Richard’s view the process of change has only just begun.
For example what will it mean when the iPlayer is on your telly and not just your laptop?  And what will it mean when this is mobile on your phone or when iPad or son of iPad is in mass circulation? This could happen within a decade or less.

The process of change is continuous

When internal change is introduced it follows a recognisable pattern.
At first,Richard was thought to be mad and there was resistance-“the management are moving too fast”.  Followed by a period of acceptance and normalization (“of course this is the way to do things”).  And then protests that the management is not moving quickly enough to promote change.
Change will be continuous and often uncomfortable for leaders, but people do adapt and find new ways of working.

Social media and citizen journalists are bringing big changes to Journalism

Much of this change is positive and it does not obviate the need for good journalists sticking to journalistic principles. But the practice of journalism is different, in four ways:

1. Eyewitness views and pictures can be better than the official filmed version as it is truly on the spot live reporting. The moment when broadcasters really understood this was the Kings Cross bomb- the grainy live reports on video enabled mobile phones from underground beat anything that the BBC or ITV could put on air.This led the news that day.
   Reporting on Iran relies on video clips uploaded by people inside Iran- a huge volume of them, at one time 8 video clips a minute.
2. Opinion
   The idea of collecting opinion and news from the general public is not new – there have been radio phone-ins for years. But social media makes the scale and speed of this something different.  We need to be responsive to this with comment boards and blogs.
3. Breaking news can come from the web and predictably will do so during the election campaign.   Social networks are the new cities or neighbourhood bars.  It’s a place where we find news and where it breaks – e.g., the plane ditching in the Hudson, the use of White Phosphorous in Iraq broken by a journalist using his contacts and his blog.
4. Networked journalism: On any give subject the public will know more than the journalist. This is especially true of some areas such as technology where a journalist who fails to consult the experts in the public and business arena can get caught out. So stories will increasingly be put into the public domain for feedback before being broadcast. For example The BBC having created a consultation mechanism called City Diaries for unvarnished and expert feedback on finance and banking
   See Charlie Beckets book “Supermedia”
   This has brought changes in structure and staffing.  Now the BBC newsroom has a desk of 12 that monitors social media 24/7 and looks at material sent in by the public – they verify it or caveat it.

Does his mean we no longer need journalists?

Not a bit of it. This huge flow of information just needs a different journalism.
Never confuse information with good journalism. There is both demand and a need for verification and fact checking that is at the heart of journalism however the journalist collects his or her information.  Standards of journalism should be driven up by the expertise of the public.

Richard’s four soundbites

1. We (i.e. the broadcasters) don’t own the news anymore
   The public can and do set the agenda- an example is the UGC pictures during the G20 protests. This led the news.
2. Transparency is the new objectivity
   Balance used to be the watchword of the regulated UK news environment
   Now that is not enough. Trust is built through transparency.
   This can be liberating and it means “showing your workings”- as you did when doing maths at school- and admitting mistakes. The positive benefit of this is that it reduces the power of the spin doctor – people can smell their influence and won’t tolerate it.
3. Information is not journalism
   Volumes of information and opinion from people who may have an axe to grind create a profound need for verification, context and, accuracy – and a consistent demand for a framework you can trust.
   BBC now providing real value in the way it uses government data – for instance providing maps of government data such as crash maps or the recession tracker looking at regional differences.
4. If you find yourself in competition with the internet get out
   How journalism related to the internet is of course one of the most vital topics to the future business models of news media.
   The internet enables and supports journalism through openness.
   You wouldn’t bet against Rupert Murdoch but blocking people from viewing content through paywalls is unlikely to succeed. The cat is out of the bag.

See Alan Rusbridger’s Cudlipp Lecture on whether there is such a thing as journalism

Discussion points

Will British politicians follow Obama’s lead in using social media?
Obama was a special case in some respects. He was coming from behind with fewer resources and took more risks. He understood that he had to let go of control. Most British politicians don’t “get this” and see social media as a channel for putting out their message and then fail to respond authentically when they get comment and feedback. It makes them look stupid and they are better off not doing it
That said this next election will bring the biggest influx of new MPs since 1945 and it may well bring a change of mood and a generation of politicians who do understand that social media is about transparency and interaction not messaging.

The future of the world service- is it challenged buy new channels from the like of the French and Chinese sponsored channels (and Al–Jazeera)?
There is a difference- these are mouthpieces of state and project a national perspective. By contrast, the BBC is independent and guards its independence, which can bring it into conflict with the government of the day.
The BBC will maintain its leadership position as a global news provider if it sticks to this and maintains that all vital trust.

The seminar was chaired by COI’s Chief Executive, Mark Lund and we had a number of external speakers including the Permanent Secretary for Government Communications, Matt Tee, who spoke about how government communicators must get better at demonstrating to budget holders, the essential role that communications play in helping to achieve behaviour change – something particularly important in a period where budgets are being cut.

Other speakers at the seminar were:

-Andrew Darnton (Independent Researcher and author of the GSR Knowledge Review on Behaviour Change) who spoke about behavioural theory and how this gives us ‘a way in’ to better understanding human behaviour but that it is critically important to ensure we identify factors influencing behaviour at all ‘levels of scale’ be those personal, social or in the wider environment.

-Stuart Sullivan Martin (Chief Strategy Officer, MEC Global) who spoke about the implications of behavioural theory on how we approach comms planning and how in a world of increasingly complex behavioural challenges we need to start ‘lighting many bonfires rather than looking for the one silver bullet’ in terms of finding the best communications approaches.

-Rory Sutherland (Chair of the IPA and vice chair of Ogilvy) who spoke about behavioural economics and how ‘the channel, the interface and the design in which a decision takes place almost has more effect on the decision you make than the knowledge of the long-term impact of that decision’ and how more ‘direct intelligence and thought should be given to the little things that can make a big difference but can cost little’.

-Alison Hardy (Dept of Health), who spoke about the Change4Life campaign and how a deep understanding of the factors that contribute to obesity has informed the campaign strategy.

The guidance document, developed in collaboration with the GCN, draws on key sources from the disciplines of social psychology and behavioural economics.  We have sought to distil this information into some key factors that are important to consider for anyone developing communications that seek to influence behaviour.  The guidance also contains a five-step framework demonstrating how, by increasing out understanding of behaviour, behavioural theory can help to define the role for communications and build a communications model.

Join the debate online by leaving your comments below or email your thoughts directly to behaviourchange@coi.gsi.gov.uk

The paper has been developed by COI on behalf of the GCN, in collaboration with two leading practitioners in the field of marketing evaluation, DDB’s Les Binet and Sarah Carter. It is thought to be the first study of its kind and is meant to stimulate discussion and debate within government, the industry and academia. So, circulate it widely and let us know what you think.

The deadline for feedback is 6 January 2010. After this, COI plans plan to hold a COI Big Thinkers session led by Les Binet and a technical workshop for departmental evaluation leads. The intention is to then produce both a ‘quick guide’ and ‘technical guide’ by April 2010.

Join the debate online by leaving your comments below or email your thoughts directly to evaluation@coi.gsi.gov.uk

Martha Lane Fox (photograph courtesy of John Naughton)

A very inspiring session by Martha. Key themes below:

4m of these are the most socially deprived.

How do they break down?

39% 65+yrs, 38% LT unempolyed, 19% families with kids, 70% in social housing, 12% disabled.

Martha stated she strives to live in an equal society and that this is simply not the case if people do not have access to the internet and the advantages it brings.

She talked of both a social and an economic imperative.

1). The social imperative

She has set up a taskforce made up of people who have experienced the transformational effects of technology.

She talked about Mo teaching ICT skills to many for whom English is a 2nd, 3rd or even 4th language.

She told us about Paul, unemployed for 18months but who now finds work through technology, searching the net in the morning before the jobs go live elsewhere.

And we learnt about a project in Bristol where adults and kids are working together with technology to improve their living environment – growing veg in the gardens and taking photos, downloading them and sharing them via websites. Kids learning about how to grow veg and adults learning about technology. They now even sell the produce and it has helped galvanise the community.

2. The Economic Imperative

Martha has worked with PWC who have estimated that moving the 4m online would bring an increase of £22bn to the economy. This is via increased education and employment. She suggested people will shop more yet save money and that even the poorest familis could save £300 per yr.

But what are we going to do?

Martha has set up an ‘Intelligent Hub’ which has developed 3 themes:

1. Encourage peer to peer training. Learn from someone like you. Perhaps inter-generational. She asked if these trainers could be given qualifications and be paid.

2. Use intermediaries. Organisations are already working with this group in many ways, such as charities. We need to permeate into these areas and help.

3. Awareness raising. Only 46% of MPs think that digital inclusion is an important issue.

Martha also stated that we need cheap access to products and systems and mentioned BT, Microsoft and Intel as companies she is engaging with.

Additionally, Martha said that we seem to be very good at doing lots of little projects across the UK, but less good at joining them up to make them greater than the sum of the parts.

She said that entrepreneurs find networks, copy good ideas and plug into others  to scale up. This provides greater impact and we need to learn from this approach.

To conclude, Martha said that it is not fair or right that 4m people do not have access to the advantages (e.g. savings, info.) of the web and we need to put that right.

The big research issue comes up first and it will be interesting to track the professor as he seeks to link implicit attitude testing (IAT) to behaviour change using the Tesco database. If IAT is more predictive of behaviour change than Explicit attitude testing (EAT) then it may change the research that is commissioned.

Here are the notes

Professor Geoffrey Beattie – Feedback notes.

Overall Implications

-        There are big implications for research.  Should we be using more Implicit Attitude Testing than Explicit in understanding the audience and the issues, testing communications and evaluating success?  If so which tests are reliable, who and how can we implement this?

-        There are implications for background research into what are the correct behaviour triggers we should be working with in the advertising.  What should the content of the advertising be to stimulate replacement behaviour rather than triggering bad behaviour.  Decoding past communications, language and behaviour to understand this might necessitate a greater emphasis on upfront research into the subject.

-        There are implications for the execution of advertising.  How to get an authentic message across.  What are the correct gestures?  Getting the casting right.  Testing the implicit communication of films before release.

-        Is IAT (Implicit attitude test) more predictive of behaviour change than EAT (explicit attitude test? Are there case histories? Feedback from Prof. Beattie on his research with Tesco will be important.  Is there a research co. that we can enrol to help develop a methodology to test this?

-        The next phase of professor Beattie’s work will be interesting to follow. Working with Tesco he has access to the largest behavioural database in the UK so the potential exists to correlate IAT with behavioural change. If there is a better correlation between IAT and EAT and behaviour change it could have significant impact on the market research industry.

Group Feedback

Group 1 –  (PR) Dan, Rozanne, Ian T, Emma,

How could we incorporate “unconscious gesture” or replicas into ads?  Is it a bit “1984/big brother” in the conventional sense, to use?

How can we adapt our research techniques to incorporate analysis of body gestures?

Understanding implicit attitudes has real implications for public sector behaviour change campaigns.

How to you change implicit attitudes when they’re hiding behind explicit behaviour and attitudes?

Has anyone approached a campaign based on implicit attitudes and also on the power of non verbal comms?

Implications

Better understand the implicit attitudes – otherwise we base campaigns on preaching to the “explicitly converted”!

Should we add another target audience with different considerations to our brief?  How do we reach the “complacent ones” – the cognitively dissonant ones.

We’ve assumed that negative responses to communications motivate positive behaviour change, but need to consider an emotional response which can lead to regressive behaviour – eg., fear leading to smoking, drinking, crime.

Only use people as spokespeople who really believe in the campaign – otherwise their gestures will undermine communication.

Be mindful of gestures and non-verbal communications in any face to face settings – can help understand the audience (journalist).

In press conferences – try and make sure gestures are “on message” and match the verbal communications.

Group 2.   (Research) Fiona, Damien, Hilary, James F

No written response but verbal response which reinforces the need to:

-        Learn more about how to use IAT (Implicit Attitude Testing) in research. In particular ensure the research is good quality.

-        Look at how/whether IAT is more predictive of behaviour change.

-        Look at the implications for evaluation.

Group 3.   (Planning) Simon D, Chris T,  Andrew L

Performance isn’t spontaneous – how does one get a reliable “gesture” led communication when the “best-process” is not spontaneous.

Are the behaviour changes we try to campaign on so deeply held that it takes decades to elicit widespread change?

Group 4.  (Client Teams) Stephen D, Lisa, Paul T

Assess Client and supplier reactions by watching their gestures, and internal meetings

There are implications for the research we do.  Are quantitative attitude surveys measuring the wrong thing?

In TV advertising – can we use real people not actors?    We should look at rushes with this in mind.

Group 5.  (Integration). Matt, Gill, Tamsin, Emma, Andrew LF

Using actors and getting the gestures right for greater effectiveness.

Many government ads, not using actors, so choosing clips with the most believable gesturing.

Channels – what about the choice when there is a low budget, ie., no TV.

We should build implicit measures into evaluation.

Review evaluation methodology and tools to include explicit/implicit measures.

Which channel is best measure of implicit/explicit, does this correlate to messages best consumed on own/in crowds.

Implications

Record focus groups.  Plan on implicit behaviour rather than explicit behaviour.

Understand the implicit v explicit triggers to better know how to change behaviour.

When do we revert to implicit behaviour over explicit behaviour and vice versa.

Group 6 (Content). Juwon, David, Martin

Is acting always a lie? – should we avoid it and lean towards graphic/animation for example or work to improve it ?

Performance beyond comms.

Interpretation of data will always be key and is currently an important part of the process.

Challenges methodology of phone/questionnaire evaluation.

Quotes From Visible Thought by Geoffrey Beattie

( Routledge 2003)

“I will argue that (the movements of the arms and hands) provide us with a glimpse of our hidden unarticulated thoughts”

Of course there have been books on body language in the past but they tend to focus on slow behaviours”

How do we identify the very brief facial expressions, which flit across the human face in interaction? What about the very quick movement of the hands and arms?

Big Brother constitutes and rich source of multi layered interaction

Voluntary or deliberate facial, movements, like false smiles, are controlled by the cerebral hemispheres and show an asymmetry in their expression on the face as a result of this. Involuntary facial movements that reflect real emotion, such as genuine smiles are controlled by lower or more primitive areas of the brain.

The hands represent the human mind in action. They provide us with a window on the human mind, where we can glimpse some of the unarticulated thinking that goes along with speech.

The claim that human beings use two separate languages (verbal and gestural) each with its own function may simply not be correct

We can, potentially discriminate between unconscious movements produced naturally by the brain and those that are produced consciously or deliberately by television presenters or people ‘acting’ in every day life.

Sapir argues for the existence of a collective unconscious, that is a set of rules or grammar, which everyone applies in bodily expression without being able to make the rules explicit.

Hewes presents a coherent argument that the first form of language must have been gestural in form

Different types of gesture

Iconic-gestures whose form displays a close relationship to the meaning of the accompanying speech

Metaphoric-gestures that are essentially pictorial but the content depicted here is an abstract idea rather than a concrete object or event

The Beat-movements that look as though they are beating out musical time.

Some people seem to miss out on the gesture channel completely. Others are tuned to it and quite unconsciously the process this important information along with speech itself. The differences between the amounts of information received (between the two) are quite staggering.

Through hand movements people quite unconsciously and unwittingly display their inner thoughts and their ways of understanding events in the world

Gesture-speech communications (i.e. the combination of the two) are remembered better. In everyday natural communication speakers spontaneously generate images to accompany their talk. Thereby through this activity they are helping to encode the content of the speech into the memory of another person