This series aims to demystify analytics through looking at the practice in-depth and talking about the data, the methods, and the desired results of digital marketing analytics.

First, let’s take a look at how analytics can help with business problems. That is, how should analysts approach the wide variety of problems businesses face?

Before we start…

Econsultancy is holding an Advanced Mastering Analytics course in Singapore on Thursday, 27th October 2016.

Click here for more information and to book your place!

Research about analytics

A recent research paper sheds some light on this question. In a 2015 report, ‘Becoming an Analytics-driven Organization to Create Value‘, EY (formerly Ernst & Young) asked 270 senior executives to list the top drivers to implement data analytics.  

Two of the top reasons business executives implement analytics programmes are:

  1. To understand customers better.
  2. To create new revenue streams.

Both are worthwhile ambitions and they give us some idea of what sorts of problems businesses expect analytics to solve. 

But how exactly can analytics help solve these problems?

To answer that question, it’s useful to take a step back and determine what, exactly, are we doing when we ‘do’ analytics?

Sure, we want to help ‘understand the customer better’ and help ‘create new revenue streams’ but what does that mean to an analyst on a day-to-day basis?

One answer to this question comes from outside of the business world, entirely.

Puzzles and mysteries

In an acclaimed article, Gregory F. Treverton, chairperson of the US Government’s National Intelligence Council, wrote about how government intelligence analysts approach problems.

When confronted with something new, they put the problem into one of two categories.

One type of problem he calls a ‘puzzle’ and describes it like this:

Even when you can’t find the right answer [to a puzzle], you know it exists.

He gives the example of the military strength of the Soviet Union during the Cold War. The US didn’t know how many weapons the Soviets had, but they knew such an answer existed.

The other type of problem is a ‘mystery’.

A mystery cannot be answered; it can only be framed.

His example for a mystery is terrorism. Unlike the Soviet weapons, intelligence officials did not have a clear idea of the cause and goals of terrorism, and so it cannot be quantified, measured, or solved.  

So, according to Treverton, it had to be treated differently.

The difference between the two types of problems hinges on whether they can be solved. A puzzle, like a crossword puzzle, has answers. A mystery, however, does not.

Classifying the problems

Going back to the problems from the EY paper, are they puzzles or mysteries?

Business problem one is ‘To understand customers better‘.  Is this a puzzle or a mystery? In other words, could we find answers to the questions around not understanding our customers well enough?

Most likely, yes. In most cases, there is only a limited amount of data about a customer which is useful to our business. This particular problem, then, is a puzzle.

But what about the problem two: ‘To create new revenue streams‘. Puzzle or mystery?

This problem isn’t quite as clear-cut. What do we need to know to start a new business line? Is it even possible?

We don’t really know either at the outset. This one looks more like a mystery.

Solving puzzles and mysteries

So how does classifying our problems in this way help us solve them?

Treverton describes the issues that analysts face when trying to solve both types of problems (emphasis mine).

Puzzle-solving is frustrated by a lack of information.

Mysteries often grow out of too much information.

Let’s apply these to our problems.

Problem one, not understanding our customers, has the same issue Treverton’s analysts faced when confronting a puzzle.

We want to understand the customer better, but we do not have enough information about them.

But is our other problem, ‘to create new revenue streams’, a result of too much information?

Well, a standard way to ‘solve’ this problem is to collect data about the market, review the competitive landscape, and read some product launch case studies. But does this really help?

Possibly. Many would argue that research does help launch businesses. 

But, from another angle, maybe not. If readily-available data were useful to starting a new business line, then it’s likely that another company would have already found it, making it an unattractive solution.

Because of this, all of the existing data in the world would not necessarily help solve this problem.

Creating genuinely new revenue streams, from that perspective, then looks like a mystery.

How to solve a mystery

But what should we do? Do we solve the puzzles and discard the mysteries?

Let’s go back to Treverton’s article.

Puzzles can be solved; they have answers.

Treating [mysteries] as puzzles is like trying to solve the unsolvable – an impossible challenge. But approaching them as mysteries may make us more comfortable with the uncertainties.

There is the answer. We should work on both puzzles and mysteries, but in different ways.

To ‘do’ analytics, we need to decide what kind of problem we are solving – a puzzle or a mystery – and treat it appropriately.

If the problem is a puzzle, or one for which more data helps, we should collect more data.  

If the problem is a mystery, then we should start by defining the problem area first before even thinking about collecting data.

Going back to the first problem from the EY report, we know the data which would help us to understand customers better, so we solve this problem by collecting more data.

But for the second problem, we do not know what, if any, of our existing data will help us create new revenue streams. We should, therefore, avoid data analysis until we understand the problem better.   

A better approach to a mystery would be to ask questions:

  • How much revenue are we looking to gain?
  • How much is the company willing to invest in this project?
  • Are we able to cannibalize existing revenue to get there?
  • Does this project have executive-level buy-in? 

And so on. Answering these questions is likely to increase the range of potential solutions and, hopefully, present options for which data can play a greater role.

Back to digital marketing

The approach used by the intelligence community to solve problems can also be used for digital marketing analytics.

Here are some of the things digital marketing analysts are asked to do on a regular basis: 

  • Are the new ads doing better than the old ones?
  • Where are people dropping off on the website?
  • How did this month’s performance compare with last month’s?
  • What was our return on ad spending?

And we can answer these questions. They are puzzles and the solution lies in retrieving the right data.

But digital marketing analysts are also asked other, more complicated questions: 

  • How can we lower our customer acquisition costs?
  • How can we sell more to existing customers?
  • What can we do to prevent churn? 

It’s tempting to respond with historical ad costs, customer acquisition cost figures and future projections to offer insights. But this approach is limiting.  

As we know by now, these problems are mysteries, not puzzles, and so a definitive answer from our data may not even exist.

A better approach is to treat these problems like mysteries and respond with questions. 

Ask about the scope of the project:

  • What can be changed?
  • What has to remain the same? 
  • What’s the budget?
  • Are we limited to digital media?
  • Etc. 

These questions may not, yet, have answers. Once they do, though, it is likely that you will be much closer to a few potential solutions.

Then, the data you collect can be analysed with the solutions in mind and provide better insights to the business. 


So when approaching analytics, whether it be a report for your team or a strategy for the department, think of the National Intelligence Council and classify the problems first.

For day-to-day performance monitoring, the best approach is to collect data and organise in a sensible way. Our next Analytics Demystified post discusses how to do so in greater detail.

For business analysis, though, the first step is to ask questions.

Defining the scope of the solution is more important than collecting data and, in doing so, you will be moving your data analytics up the value chain.