This is where the butterfly effect comes in. Let me explain.


First, I’ll start with some Wikipedia definitions…

The Butterfly Effect:

“In chaos theory, the butterfly effect is the sensitive dependence on initial conditions in which a small change in one state of a deterministic nonlinear system can result in large differences in a later state. … A very small change in initial conditions had created a significantly different outcome.”

Conversion Optimisation:

“In internet marketing, conversion optimisation, or conversion rate optimisation (CRO) is a system for increasing the percentage of visitors to a website that convert into customers, or more generally, take any desired action on a webpage. It is commonly referred to as CRO.”

True Conversion Optimisation:

“True conversion optimisation is the beginning of a mindset change within a business, maturing to becoming a customer-centric organisation that embraces experimentation to out-perform their competitors.”

This last definition is mine, and is yet to make it onto Wikipedia!

The initial “condition” – Conversion optimisation is misunderstood

Fundamentally, conversion optimisation is hugely misunderstood by the most influential people in businesses. It’s perplexing to me how the good old “button colour testing” is still very much in evidence within the “idea factories” of decision-makers’ minds.

“Quick wins. Low hanging fruit. Tweaking the design. Changing a headline. Changing an image. I want to test this. We don’t need our developers, you do everything through the tool.”

This is the level to which conversion optimisation is considered within a startling number of business.

Misunderstood = under-appreciated

When something is misunderstood, that something is very easily under-appreciated, however much potential value it can bring.

Conversion optimisation is cripplingly under-appreciated when it can and should be one of the most influential growth levers for a business.

In my experience conversion optimisation is seen as a simple, tactical element of marketing. It typically plays no important role in the growth strategy of businesses.

You don’t invest in something you don’t appreciate

Where there is little perceived value in something, it is to be expected that you are not going to invest energy, time and resources in it – particularly people.

In the early years of conversion optimisation, the most influential testing platforms positioned A/B testing as a simple and quick way to tweak your website and see if you could get more people to do what you wanted them to. Businesses were encouraged to provide their developers with a few lines of JavaScript, then you could leave them to it whilst you as the marketeer got on with using the WYSIWYG editor to set-up A/B tests.

This led to a big rise in investment in tools and technology for conversion optimisation – with little if any investment in a multi-disciplinary team to develop intelligent, insight driven A/B tests.

With the proliferation of talk, investment, excitement and worry about “AI” and machine-learning, what’s lost in the shuffle is the importance of investing in human beings.

Until businesses realise growth is about people, and they should be investing in Human Intelligence in advance of Artificial Intelligence, investing in the right level of skills and resources simply won’t happen.

You don’t have the people or skills to conduct intelligent, genuine user research

Investing in user research is fundamental to any business that is striving to become customer centric and develop an experimentation mindset.

With limited (if any) investment in people, there is limited (or non-existent) investment in conducting user research, in whichever form this may take.

I find it shocking just how many companies go years without stopping to think “is what we are doing really what our visitors are looking for?”.

Without behavioural insights there is a lack of intelligence behind changes

Coming up with tests based on egos, opinions, best practices and competitor activities is the norm for most businesses.

Test hypotheses are pretty much non-existent, and even if there is some form of hypothesis, it lacks genuine intelligence through behavioural research and understanding.

Based on my experiences, over 75% of A/B tests that are run lack a genuine “why?” behind the test variation.

Confirmation bias leads to incorrect conclusions

All of us, including optimisers, suffer from confirmation bias – if you work really hard for something, your mind wants to believe that it’s true:

Does this variation really increase conversion rate by 94%?


Is one week really enough to gather sufficient and appropriate levels of visitors to make a highly informed decision?


Did we increase the micro conversion rate but didn’t bother looking at what effect this had on the macro conversion rate?

A lack of statistical understanding coupled with a desire to want tests to be successful very often erodes the integrity behind the test results.

If you put garbage in, you get garbage out

Yes, never has this been more true than in A/B testing.

Come up with a test idea with little or no research, hypothesis, data and intelligence, and you are not going to make any learnings off the back of it.

I have previously shared how MVT (multivariate testing) should be renamed NHT – No Hypothesis Testing. When you run an experiment with multiple variations of different “things” mixed together, there is no intelligence or hypothesis there.

The most successful companies with a genuine experimentation mindset are learning all the time about how to influence the decision making of their prospects and customers.

There are very few successful companies in conversion optimisation.

Conversion optimisation doesn’t shift the needle

The harsh truth is that most companies don’t see that conversion optimisation has had a tangible, credible and trustworthy positive impact on their primary performance metrics.

The number of times I have been in a meeting with a potential client who has been “running tests for the last few years”, whom I ask “what commercial impact have you seen on your bottom line?” which results in a very flat, non-committal answer, tells its own story.

It’s what industry thought leader Craig Sullivan explains simply as “the trough of disillusionment”.

If it doesn’t deliver value then why bother?

In the trough of disillusionment, it’s understandable for the decision makers within a business to look at this investment in conversion optimisation and conclude “why are we bothering?”.

With acquisition long established as the easiest way for a business to get more sales, simultaneously lining Google’s big pockets, it’s easy to see why investment in conversion optimisation can be seriously questioned – particularly for businesses who have first-hand experience of the butterfly effect which comes from a lack of understanding.

In summary

“Yes but what about this ‘butterfly effect killing the conversion optimisation industry?” I hear you say.

Well, following on from my points above, what are we left with?

1. Conversion optimisation is fundamentally misunderstood

Which means that…

2. Conversion optimisation is underappreciated

Which means that…

3. Conversion optimisation gets little in the way of investment in people

Which means that…

4. Conversion optimisation is done with little (if any) intelligent user research insights

Which means that…

5. Conversion optimisation lacks any real form of intelligence behind what is being tested

Which means that…

6. Conversion optimisation is dominated by test “results” which are a lie

Which means that…

7. Conversion optimisation delivers little (if any) tangible & actionable visitor learnings

Which means that…

8. Conversion optimisation delivers little (if any) commercial impact

Which means that…

9. Conversion optimisation is considered irrelevant to the growth ambitions of the business

Which means that….

We are left with a CRO industry and way of doing business which, though it can prove a competitive advantage (see Amazon as one example), is actually the most under-utilised part in the whole of digital marketing.

Until conversion optimisation is understood for what it truly is, it will remain as the biggest missed opportunity in global marketing.

True conversion optimisation is the beginning of a mindset transformation within a business, maturing to becoming a truly customer centric organisation that embraces experimentation and long term thinking to out-perform their competitors.

More on conversion optimisation: