Lou Rosenfeld is the founder of Rosenfeld Media, a publishing house focused on user experience design, and the co-author of Information Architecture for the World Wide Web (O’Reilly & Associates; 3rd edition, 2006).

He also co-founded the Information Architecture Institute and UXnet, the User Experience Network, and as an information architecture consultant, has helped numerous Fortune 500s and other large, messy, political enterprises make their information easier to find.

Here, we asked Lou a few questions about site search analytics (SSA) – it’s importance in site design and development, and how it is being used by different types of businesses online.

In a nutshell, why is it important to analyse site search usage on your website?

Every time someone uses your site’s search engine, he is telling you what he wants from you, in his own words – his query. We analyse query data to find interesting patterns and anomalies. These can help us identify and diagnose problems with our site’s content, search system, navigation, metadata, and other aspects of its design. Think of SSA as a user research tool that helps all kinds of designers make more informed, data-driven design decisions.

We in the user experience community often tend to be a bit uncomfortable when analysing quantitative data; that’s not the sort of thing that we learned in design school. But dipping a toe in SSA helps us understand the ‘what’ aspects of our site and its users. It also helps us generate better ‘why’ questions to explore further. So SSA helps to counter-balance the qualitative tilt of many designers’ research methodology, and it helps us use our old standby qualitative methods more effectively.

How many visitors typically use site search and how does that vary between sectors?

Search rates are highly variable, and have as much to with users’ information needs and the nature of the indexed content as much as it does with sectors. For example, it’s probably fair to assume that the information-hungry users of Amazon or your intranet’s staff directory search far more than they browse. And that the converse would be true of users who seek social encounters at Facebook or entertainment at game sites.

Some suggest that as many as half of all users are search-dominant – they automatically search before considering other options – but again, it really depends on the nature of the information need and the information itself.

How many sites are using the SSA functionality in their analytics tools, as well as making changes to their business based on that data?

Although many analytics tools and search engines now support at least a minimal degree of SSA functionality – and there are excellent free tools like Google Analytics that offer SSA – few organisations tap their query data. This is due to a variety of fairly predictable reasons; general ignorance about SSA, technical roadblocks that may have been thrown up by IT, political landmines thrown up by managers, and legal barriers due to overly-paranoid legal departments.

But the good news is that more organisations are figuring out that they can benefit from SSA, and not just the e-commerce players that live and die by analytics. We’ve talked with people at such organisations as the BBC, Vanguard, Reed Elsevier, Netflix, WW Norton, and Sandia Labs who are taking full advantage of what they learn from SSA.

And some have gone beyond using SSA simply as a diagnostic tool. For example, the Financial Times monitors its site’s recent search activity for spikes in queries for personal and company names. They check to see if these suddenly popular queries correlate with their recent editorial coverage; if not, a spike might indicate a breaking story, and the paper’s editors are informed. So, in effect, the Financial Times is using SSA as a predictive tool.

How should you start off with site search analytics? 

Start with some query data and some basic questions. The following questions are generic enough to pertain to just about any situation, and your analytics tool or search engine might already provide you with reports to help answer them:

  • What are the most frequent unique queries?
  • Are frequent queries retrieving quality results?
  • What are the click-through rates per frequent query?
  • What are the most frequently clicked results per query?
  • Which frequent queries retrieve zero results?
  • What are the referrer pages for frequent queries?
  • Which queries retrieve popular documents?
  • What interesting patterns emerge in general?

If you spent an hour a week poking around the data to answer these questions while equipped with even an inexpensive, rudimentary analysis tool (say, Microsoft Excel), you’ll start benefiting immediately.

You’ll quickly notice problems – like a very frequent query that’s not retrieving any results – that you can fix after a little follow-up (for example, do we not have the appropriate content? Is it there but not indexed, tagged, or titled properly?). And by demonstrating to your colleagues the benefits of just a smidgen of SSA, you can go a long way toward getting it permanently established in your organisation.

Can you use site search findings to improve your search marketing strategy? 

It’s not so obvious. There are many differences between the queries people use to find your site and the ones they use to search within your site. For example, your most frequent web search queries will likely include many variants of your organisation’s name; naturally, these are unlikely to occur among your top site search queries.

That said, there probably is a useful connection between web searches and site searches. Here’s a theory that I’d like to test: Let’s assume that, because searchers have more context, site search queries are more specific than web queries. Let’s also assume that as the web becomes a more crowded and competitive arena, bidding for more general keywords becomes more expensive. You’d rather market your content using more specific, less expensive keywords. So, you might find that your site search queries are the ideal place to look for candidates.

If you have a less than optimal site search tool and a tight budget, is it better to invest in a better search product or in SSA?

It’s much better to invest in analytics, as it will help you diagnose problems with your current search engine that might not be that difficult or expensive to fix. So you’d avoid ripping out the old and installing the new.

Really, any organisation considering investing in an expensive new search engine (and when is a new search engine not expensive?) should do their analytics homework first. It’ll help profile what users need from the search experience, leading to a set of functional requirements that is far more comprehensive and balanced than the ones your IT department will write on their own.

How has Google’s launch of site search boxes within its organic results impacted SSA? Is it making site search analysts’ job harder?

I’m not sure it’s all that common at this point; I encounter it mostly with very large sites. That said, it clearly does channel traffic away from site search engines, thereby reducing the quantity of data we have to play with.

Does that give analysts the right to get mad? I don’t think so: As they usually do, Google finds great opportunities to improve its service, its customer loyalty, and its bottom line where others have ignored the search experience. In other words, years of horrible site search implementations are coming home to roost.

What else are you up to at the moment?

For years I’ve made my living as an information architecture consultant for Fortune 500s and other large organisations. That work has increasingly led me to SSA; I now teach day-long workshops on the topic in five or six cities annually, and I’m also finishing up a book on site search analytics, co-authored by Marko Hurst. Our book will be available in 2009 from my publishing house, Rosenfeld Media, which produces short, practical books on user experience design.

Many thanks for the opportunity to talk site search analytics, Richard!

Related stories:
Ten site search tips