What probability and loss aversion can teach us about reporting results

When things go wrong

As an SEO agency, when something goes very wrong with a client account, maybe Panda or Penguin hits and the client’s keyword rankings and traffic tumble, both parties tend to want much more contact to sort out the problem.

The client needs handholding and the agency wants to make sure they do not lose the client so, as an SEO agency, you balance the warm fuzzy glow the client gets from being able to speak to you every week with the fact that you can often provide no useful, statistically relevant data to them on that narrow a timescale.

I recently read Fooled by Randomness by Nassim Taleb. Taleb is a trader and academic, most famous for the book Black Swan, about the impact of improbable events link.

One chapter that stuck out for me was about the unobvious effect the quantity of reporting can have on the perception of results.

Like most interesting things in marketing, it concerns a little bit of statistics and a little bit of psychology.

The psychology of loss

Kahneman’s and Tversky’s work on prospect theory and loss aversion provides the first relevant insight: losses hurt more than gains feel good.

An interesting takeaway, given the respective curves, is that wins may be best reported in small frequent increments to achieve the best perceived value and losses may be best reported in a large chunk to reduce the overall amount of perceived value lost.

The Statistics of reporting

In the book, Taleb illustrates his point using a trader. Just like SEO, his goals are long term. Just like SEO, his progress (keyword ranking) over a year is volatile.

Over the course of a year, the trader expects a 15% return in funds with a 10% volatility. This translates into a 93% probability of success in a given year.

However, the more often he checks his positions within that year the more often he is likely to be disappointed with his performance.

Replicating the table from the book:

Probablility of success at different scales

If the trader were to check his positions every day, he would be up 54% of the time and down 46% of the time. So even though ultimately he is going to be ahead of the game, he has a significant number of loss events to cope with.

When we combine the psychology of loss with the statistics of reporting, the cumulative emotional impact of that 46% loss may lead him to feel unrealistically unhappy with his performance.

In the example above you could easily replace trading returns with keyword positions and the trader would be your client.

Even though overall you are doing a good job, because you report on too short a timescale, your client may emotionally, subjectively feel you are not doing a good enough job.

How to deal with the problems created by frequent reporting

If you find yourself in times of crisis having to report frequently, try reporting on actions rather than numbers, i.e. report on the things you did rather than the traffic you achieved.

Try holding reports on traffic numbers to much longer reporting cycles. I suggest that, on short reporting cycles (weeks and months), you should report on actions whereas on long reporting cycles (quarters and years) report on numbers.

Reporting on numbers less frequently may be a hard sell, do you think this would improve your relationships with your clients?