Two months ago Martha Lane Fox published her review of DirectGov proposing that it was currently failing at its job of providing the best resources to the UK population. One of the reasons that it is failing is that it doesn't understand what its purpose is and has no measures of success.
One of the recommendations that Martha Lane Fox gave in her review was that there should be a CEO of Digital. Now that Chris Chant has been appointed as an interim, the first job he should do is work out how to measure success and imprint this across all stakeholders.
Here I am giving him a hand by making some recommendations...
It was actually a bit disappointing for me to read Martha Lane Fox's report (pdf) because it focussed on the things that should be done to reduce costs to Government.
As all marketers know, that is only half the way of calculating the ROI of a website. The other side is looking at the value that it obtains. With that in mind it is of vital importance that there is a clear strategy for what the website should be hoping to achieve.
With this in place it is possible to look at how you measure the success of those goals.
I was disappointed because whilst it is prudent to make sure costs are minimised, really the whole point of Government services is to provide services for the people.
Simply bringing all the services that Government has into one place isn't going to make it easier for the user to use and find the information or services they need. The real benefits come from interlinking and deduplication of content, which is something that you could do even if the content is on different domains.
Maybe with recommendation number three (a central DirectGov team to look after usability) Martha Lane Fox is suggesting that currently this programme isn't working and needs to be looked at.
If there is going to be one website for all of the Government services, then there needs to be one team controlling the structure and content of the website to ensure that content isn't duplicated and is linked to associated content. Just creating a website which copies the structures of the existing websites in silos may be beneficial from a cost point of view, but is no better to the user.
However this would be a bold mood by DirectGov, which up until this point has been content to allow the various Government departments to create and organise their own content. Taking control from departments is always going to be controversial in Government, where bureaucracy rules supreme.
Many of the recommendations that Martha Lane Fox gave were also in the Varney Report for transformational Government which was meant to be implemented by April 2011.
This makes you wonder why only four months from that point there are a whole new set of recommendations. Is that tantamount to saying that the Government failed in making the Transformational Government plan come true? Neil Williams has a good round up of the press response to the report.
As I said last time I wrote about BusinessLink, measuring Government websites is hard.
Fortunately we can use the same method of looking at outputs and outcomes as we did with BusinessLink with DirectGov. Outputs being the volume of people doing "stuff" on the site, outcomes being the stuff that they did because of looking at the site.
As I said last time, the most simplistic view of viewing the success of a non-profit website is to look at the number of visits it has. Clearly DirectGov already does this as it posts information about it on the website and it is audited by the ABCe.
Again, there is an argument for unique users, but this isn't particularly accurate because of the issues that there are with cookies. We need to ensure that it is known that this metric isn't the number of people, but the number of times there is interaction.
Also, as with BusinessLink, there is a large volume of information on the DirectGov website on how to interact with Government, as well as practical advice on how to live your life 'better'.
For example, as well as telling you which forms you need to fill in if you sell a car, there is also some practical advice on things you should do ("don't leave the buyer alone with your keys in the ignition"). You won't need to use all this information at the same time, so repeat visits is something that DirectGov should use too.
Essentially the point of DirectGov is to allow users to complete online the Government transactions that they normally had to do over the phone or in person. For this to happen a key metric should be the number of transactions completed online correctly. I add that last word in, because incorrectly completed transactions cause a hassle for both user and Government in time and effort.
Clearly everyone has to do complete these transactions, so any additional transactions completed online are just being bastardised from other channels.
The key to a successful website in this case is that users complete online correctly without the help of the other channels. This means that a key metric should be the volume of calls to contact centres and time spent in the face to face units.
A web-only percentage of all transactions would be nice, but I suspect that a better metric would be offline support cost. Not all metrics have to go up to show success.
This is where it gets difficult. What does the Government hope users will achieve by reading and using the website?
Well firstly the website hopes that by using the website they will save time over the old system of ringing someone up (or going to a post office or other location). So surely a good metric would be the time saved by individuals.
Technically speaking, most users will also be aware that the process they are going through could be longer and more convoluted (if you can get longer and more convoluted than a 30 page document you have to read, print out, sign and then send in the post).
Surely an additional metric should be how much more time could be saved if the website was better. Whilst the Martha Lane Fox example was of a transaction, you could also argue that spending ages navigating around the site to find something is just as important.
How you measure this is debateable. You could go for the BusinessLink method and ring them up to ask them via a telephone survey (or even by standing in the street and asking people). Or you can have the option of asking them on site as they complete the transaction. Both would be my suggestion.
If you are going to measure via both methods, it would be prudent to measure market penetration (ie how many of the target audience do). You could also measure a customer satisfaction measure, although I tend to think that may be linked to the 'how much more time' measure. One or the other would be better.