Measuring a government website
is very much different from measuring a commercial one, since a government website’s job isn’t to make
money for the Government (well not directly).
This lack of a bottom
line often makes it difficult to justify changes to the website because
of the lack of direct profit for the owner. One way to get over this is to ensure you have a robust measuring process for the website to monitor the changes over time and attribute them correctly.
Measuring the success of government websites is hard. To prove how hard it is, you only need to look as far as the UK Government’s web standards and guidelines.
However, you could argue that the COI guidelines are merely a way of allowing the Government to create a comparative view of all the websites and ensure that they are doing something to measure their website. It is also a very useful way of getting government departments (and of course the UK public) to think about whether government websites are performing or not.
The American Government looks like they might have cracked a system that works by looking at the differences between outcomes and outputs.
Outputs online are the direct effects of the website. These are the things that people do on the site, how long it takes them to do it and how much of it they do. Outcomes online are the effects of users using the website and what they go and do with it.
Businesslink.gov.uk have quite a robust method and they are quite nicely grouped into outputs and outcomes (possibly accidentally).
On a very basic level, one of things that you have to get people to do is actually come to the websites. That is why visits is one of the most important metrics. You could argue at this point that unique visitors would be a better metric, however unique visitors aren’t accurate and probably aren’t very precise either. Moreover, the trouble with unique visitors as a web metric is that the tendency is to think that unique visitors is the same as people, which we know not to be true.
Moving up a step, they want to make our visitors who have found the website come back and use other parts of the website. This means there needs to be a metric of repeat visits.
How you want to measure this metric is open to debate. If you use a percentage of your total visits and aim to increase it, there is a suggestion that you aren’t looking for new audiences, whereas if you use an absolute value then you sometimes end up ignoring large increases in actual visits. Usually the best option is to go for both to give the context.
The next thing you want to do is look at making sure that the people looking at your website are doing something. Given that quite a large portion of it is written content, you possibly don’t want to be looking at bounce rates (although ignoring them is a sin too). Users who find this content through Google are more likely to bounce (as they’ve found what they were looking for) than those coming into the home page (for example).
Therefore a different way of looking at it is definitive actions that the users can take. In the case of businesslink you need to look at how many of the interactive tools have been used successfully (which research has shown that people who use save more money than those who don’t) and the number of people who register to the site.
This is where it gets a bit more interesting. This is where you really have to start thinking about what the point of your website is and what your users do after using it.
In the case of commercial websites, this last point almost isn’t important. When you’ve bought your new Dell laptop, Dell don’t really care whether you spend your life working on it, playing games or writing guest posts for Econsultancy (unless, of course, they think that they can sell you something else).
For government websites, it is the entire point of them. They are only there so that the users will go and do different things.
These can be very difficult to measure, but one option is to go to your target audience and ask them. Obviously you can’t go to them all, but asking a significant enough volume that it becomes statistically sound is an option. It also enables you to ask your target audience other things, like whether they used interactive tools, etc.
So what do we ask our prospective audience? Well the first step is to ask them if they’ve used the website so that you get a market penetration metric. This metric cost me a lot of time whilst I worked at BusinessLink.gov.uk explaining why it is different from visits and why both are important. Well, it turns out that many people who use the website aren’t in the target audience. This also means that it can be calculated it in a different way by looking at the percentage of businesses (not people or even visits) who take advantage of the site’s content.
So the point of the website is that businesses (or the people in them as the mantra internally goes) are meant to save time and money when doing business (with the Government, with each other and with customers). So they were asked if they saved time or money and how much of it. It’s a very difficult question to answer, so you have to make sure that your calculations are sound. It’s also important that these calculations are sound so that when you do comparisons over time, you can see the increased benefits.
The other point of the website is that it is that in quite the opposite way to the previous point, that it is meant to give businesses ideas and methods of increasing sales and profits. This metric, along with the one immediately above, give the website a great monetary value. This means that if you make changes to the website you can quantify (over time) a monetary return on investment for the website in terms of money for users.
Finally, it would be a bit remiss if users weren’t asked if they were happy with it. If they aren’t happy with it, then they could possibly have saved, or made, more money. Therefore customer satisfaction tends to be quite key.
If you read the annual review for BusinessLink.gov.uk, you’ll find that there are some additional metrics how much content businesslink.gov.uk has taken from government departments and how satisfied they are. I’d argue that these aren’t metrics relating to the website, but more the organisation overseeing it.