Feedback on web analysis tools

Below is a response I gave to Melaney Smith with reference to her article requesting feedback from users of various web measurement / analysis tools. The original articles is at: http://www.clickz.com/res/analyze_data/article.php/1477921

If you've got additional comments it would be interesting to hear them....

*************
We run the e-consultancy site in the UK (http://www.e-consultancy.com) which gives information and advice to UK e-business professionals. We have 8,000+ registered users. We’ve experimented with a fair few number of measurement and analysis tools on our own site and I’ve also experienced a fair number with our consulting clients. Below is some brief feedback on those. 2 brief points first:

1. Just before I talk about the tools, I would make this point, however. A point which, incidentally, I make in much greater depth in my forthcoming book. Encouraging people to think about tools as the starting point is not the right approach. People should not start with a list of tools with accompanying features and benefits. They need to get the measurement approach and metrics framework sorted out first. If you can define what you really want to know then it is relatively easy to pick a tool that will do the job. This is the only way to avoid the information overload and data paralysis you refer to. I could go on, but I know you want to hear about tools….

2. (Before my own book comes out…) There are very few resources out there that do a good job of talking about web metrics and associated tools. However, I would recommend Hurol Inan’s book to your readers (Measuring the Success of Your Website: A Customer-centric Approach to Website Management - http://www.amazon.com/exec/obidos/ASIN/1740096487/qid=1034416786/sr=2-3/ref=sr_2_3/102-2430290-1636108) and the accompanying site which has a list of vendors, with brief review, at http://www.hurolinan.com/book/br_list.asp?ResourceType='Vendor'

Tools that we’ve used + comments:

1. Webtrends – right at the beginning we used this for basic log file analysis type activities analysing log file data offline. It’s good at what it does but for all the reports and paperwork and charts that it can produce we find there is limited real value there. This is not so much a limitation of Webtrends, more log file analysis in general.

2. LiveStats – we still run this live on our server. It superseded Webtrends (mainly because it was cheaper, partly because it offered live ‘who’s on the site now’ live reporting) but basically does the same job which is basic site usage and traffic analysis. Fine for that job but of limited real value still.

3. Webtraffiq – we’ve got this running on the site now. It’s a page-tag type solution rather than working off log files. We insert a piece of code in our templates allowing it to then log activity to their servers. Personally I much prefer this approach to log file analysis (more accurate, less data management hassle and so on). I like the weekly HTML e-mail reports the system generates. So it scores well, in my view, on technology approach, ease of implementation, accuracy, user interface. But ultimately the actual information we get out of it is not really any different from the 2 above. Over time we may stick with this solution and drop the other two for our basic traffic analysis requirements.

4. ClickTracks – we use this to understand how our interfaces are working. The most interesting (and differentiating) thing about this tool (apart from its client-side technology approach) is its interface and the way it displays how a page is being used. It works off log files, admittedly, and it can be a little fiddly integrating with templates and CMSs but it is really useful to visually show where people are clicking on each page. This works particularly well for designers and clients like it because it is so visual – really easy to understand what is going on. It also does the basic traffic-type analysis and – blessedly – does not offer you too many reports which you don’t need.

5. Nedstat – we don’t use this on our site but I’ve worked with the tool before and liked it. It is a ‘page-tag’ type solution (insert code into your pages) which I prefer to log file analysis. The best thing about Nedstat is the user interface, reporting and tools available. Easy to use, powerful (export to Excel etc.), intuitive. All important to marketers, in particular, actually getting value out of a measurement tool.

Given the above experiences I would make 3 points:

1. We still haven’t found a tool that does path analysis (‘Clickstream’) that well in my opinion. We’d like to really understand traffic flows around key parts of our site (homepage, registration etc.). We reckon we can do a much better job of this ourselves than the tools out there.

2. Despite advances in all these tools we still find that 90% of the value in the site measurement and reporting we do from a quantitative data-driven point of view comes from tools we have built ourselves. Our main objective is to acquire and retain registered users so our user profile database and associated tracking, segmentation, personalisation, e-mail marketing etc. is *far* more valuable than site usage analysis. We can create a click stream analysis tool that would tell us not only how the site was being used but by which customer segments. Admittedly this is possible because of our high level of integration of user and content data and because this is what we do for a living so we have an unfair advantage. This said, it must also be recognised that none of the above tools are very expensive so I reckon we’re getting fair value for their cost. Personally I wouldn’t buy into the high end solutions like E.piphany – too expensive and too many redundant features. I’d build in house.

3. I would urge people not to forget about things like usability and focus groups and customer feedback. The tools discussed are good at telling you *what* is happening but not *why*. I’d say that our site development decisions have been more influenced by talking to actual users than what our reporting tools tell us. It’s much cheaper and faster too.

Hope this helps

Ashley
********