The web has made market research widely available but the conclusions drawn from different methods of research provide different and sometimes conflicting results.

The savvy manager will not just depend on one source of research about the market but several – and keyword research should be an essential part of any research project.

Survey results make great reading

There’s always a certain degree of scepticism about published market research but there is little doubt that the results make great reading.

Generations of public relations people have long known that one of the best ways to get media coverage is to publish a survey. Particularly if you can grab the space for regular research – a monthly or an annual survey can guarantee a lot of coverage and be effective link bait.

The web has made the collection and publication of survey information so easy.

However, there is a backlash against this flood of research studies. Marina Hyde in her Guardian blog writes:

“In various other recent polls, 72% of people thought the world will end in two to three generations, that half or more Aston Villa games are uninspiring, that global warming is not caused by human activity, that the smoking ban will result in a more pleasant atmosphere in pubs and bars, that George Bush is mishandling the Iraq war, that First Great Western train services are satisfactory, and that the US government is withholding UFO data.”

And comedian Jackie Mason’s takes a swipe at the granularity of research:

“It’s amazing, we are learning more and more about less and less. If we carry on like this, soon we’ll know everything about nothing.”

Some consumers are getting fed up with participating in market research, according to

“Just 0.25% of the population supplies 32% of responses to online surveys, said Simon Chadwick, former head of NOP Research in the U.K. and now principal of Cambiar, a Phoenix consultancy, citing research by ComScore Networks. More broadly, he said, 50% of all survey responses come from less than 5% of the population.”

The big assumption in surveys is that we can extrapolate the answers from people who participate to those who do not. But as reports:

“No one really knows whether people who don't answer surveys are similar to those who do, because they don't answer surveys.”

In her column, Is Anything Really Measurable Online?, Rebecca Lieb reports that two of the web’s leading research publishers, comScore and Nielsen//NetRatings, are currently undergoing an audit of their methods because:

“Internet Advertising Bureau (IAB) members are claiming massive
discrepancies between their own web stats and the panel-based numbers
provided by ComScore and Nielsen panels up to two or three fold”.

Conflicting results

And two research studies investigating the same area can provide conflicting results. According to the Market Research Society:

“Two reputable companies asking the same person the same question via
telephone and email may receive two different answers.”

There are several reasons why results amongst pollsters may differ:

  • Polls may have been conducted at a different time, even if they are published at the same time.
  • Questions may be worded differently between pollsters, which in turn can affect the responses.
  • The method by which the poll is undertaken can also affect responses. They suggest the increasing popularity of internet polling has come about largely because some people may be happier to give their true opinions and feelings to a machine than to a person.
Keyword research must play a role in decision making

There are also studies and debates on keyword research information – different keyword research tools give different scores. Why don’t they match up?

Well each uses different ways of sampling data and so their results will be different. That’s hardly surprising when the results of standard market research can also be contradictory.

Proper use of keyword research will tell you:

  • The most important words that people use when they are searching for your products or services online. For example, which is more important - ‘digital camera’ or ‘digital photography’?
  • The long tail of your important keywords. The exact keyword, 'home office' gets a predicted daily search about 700, but looking for longer phrases that include 'home office', e.g, 'home office design' will find over a thousand keywords with a predicted daily search of 22,000.
  • Related words for identifying particular niche opportunities. The related search on “online marketing” will bring up 300 terms including “blog marketing” and “PPC management”.
  • The potential size of the online market. By adding together the predicted scores of your most important keywords and how they appear in longer search terms, you’ll get a good idea of the number of people searching for those terms every day.
  • Who your real competition is online. It’s the websites that score well on the most important keywords.
At Wordtracker, we’ve recently taken on a company to manage our PPC campaigns and we’ve found this a very stimulating exercise. We did all the usual preparation and collated thousands of keywords across different market segments.

Yet, there were still surprises. What we thought was clever turned out to be a dead end and the long shot that we thought was peripheral turned out to be a goldmine.

To be able to trust and use keyword research, you need to assess it as you would any market research. That means that you need to:

1. Know the source of the raw data.
2. Know the methodology by which it is collected and analysed.
3. Know how conclusions are drawn.

And finally, you need to test and measure, and adjust your strategies accordingly.

Ken McGaffin is the chief marketing officer of

Ken McGaffin

Published 2 July, 2007 by Ken McGaffin

Ken McGaffin is chief marketing officer at Wordtracker and a contributor to Econsultancy.

6 more posts from this author

You might be interested in

Comments (3)


Jared Bothwell

As I learnt at school the best way to reduce non-responce error is to increase your
responce. Off course when you read the results of surveys in the media rarely do they mention the participation rate.

about 11 years ago



Great post :) Thanks for the tips.

Any Wordtracker work I do, I do first in the Keyword Research Engine (
It helps me build a much better keyword list for any given market. (if any of you have taken any internet marketing courses, you know that you need to find problem statements, so this helps)

Hope this helps others is their keyword research!

over 10 years ago



I came to your blog just when I was surfing on this topic. I am happy that I found your blog and information I wanted.

about 9 years ago

Save or Cancel

Enjoying this article?

Get more just like this, delivered to your inbox.

Keep up to date with the latest analysis, inspiration and learning from the Econsultancy blog with our free Digital Pulse newsletter. You will receive a hand-picked digest of the latest and greatest articles, as well as snippets of new market data, best practice guides and trends research.