I have spent the last 15+ years working in the digital industry, primarily in user research, UX and “conversion optimisation”.

My experiences running a conversion optimisation agency have led me to conclude that the conversion optimisation industry is really, really messed up.

Why? 

This is where the butterfly effect comes in. Let me explain.

butterfly

First, I'll start with some Wikipedia definitions…

The Butterfly Effect:

“In chaos theory, the butterfly effect is the sensitive dependence on initial conditions in which a small change in one state of a deterministic nonlinear system can result in large differences in a later state. ... A very small change in initial conditions had created a significantly different outcome.”

Conversion Optimisation:

“In internet marketing, conversion optimisation, or conversion rate optimisation (CRO) is a system for increasing the percentage of visitors to a website that convert into customers, or more generally, take any desired action on a webpage. It is commonly referred to as CRO.”

True Conversion Optimisation:

“True conversion optimisation is the beginning of a mindset change within a business, maturing to becoming a customer-centric organisation that embraces experimentation to out-perform their competitors.”

This last definition is mine, and is yet to make it onto Wikipedia! 

The initial “condition” – Conversion optimisation is misunderstood 

Fundamentally, conversion optimisation is hugely misunderstood by the most influential people in businesses. It’s perplexing to me how the good old “button colour testing” is still very much in evidence within the “idea factories” of decision-makers' minds.

"Quick wins. Low hanging fruit. Tweaking the design. Changing a headline. Changing an image. I want to test this. We don’t need our developers, you do everything through the tool."

This is the level to which conversion optimisation is considered within a startling number of business.

Misunderstood = under-appreciated

When something is misunderstood, that something is very easily under-appreciated, however much potential value it can bring. 

Conversion optimisation is cripplingly under-appreciated when it can and should be one of the most influential growth levers for a business. 

In my experience conversion optimisation is seen as a simple, tactical element of marketing. It typically plays no important role in the growth strategy of businesses.

You don’t invest in something you don’t appreciate

Where there is little perceived value in something, it is to be expected that you are not going to invest energy, time and resources in it – particularly people.

In the early years of conversion optimisation, the most influential testing platforms positioned A/B testing as a simple and quick way to tweak your website and see if you could get more people to do what you wanted them to. Businesses were encouraged to provide their developers with a few lines of JavaScript, then you could leave them to it whilst you as the marketeer got on with using the WYSIWYG editor to set-up A/B tests.

This led to a big rise in investment in tools and technology for conversion optimisation – with little if any investment in a multi-disciplinary team to develop intelligent, insight driven A/B tests. 

With the proliferation of talk, investment, excitement and worry about “AI” and machine-learning, what’s lost in the shuffle is the importance of investing in human beings. 

Until businesses realise growth is about people, and they should be investing in Human Intelligence in advance of Artificial Intelligence, investing in the right level of skills and resources simply won’t happen.

You don’t have the people or skills to conduct intelligent, genuine user research

Investing in user research is fundamental to any business that is striving to become customer centric and develop an experimentation mindset.

With limited (if any) investment in people, there is limited (or non-existent) investment in conducting user research, in whichever form this may take.

I find it shocking just how many companies go years without stopping to think “is what we are doing really what our visitors are looking for?”.

Without behavioural insights there is a lack of intelligence behind changes

Coming up with tests based on egos, opinions, best practices and competitor activities is the norm for most businesses. 

Test hypotheses are pretty much non-existent, and even if there is some form of hypothesis, it lacks genuine intelligence through behavioural research and understanding.

Based on my experiences, over 75% of A/B tests that are run lack a genuine “why?” behind the test variation.

Confirmation bias leads to incorrect conclusions

All of us, including optimisers, suffer from confirmation bias – if you work really hard for something, your mind wants to believe that it's true: 

Does this variation really increase conversion rate by 94%? 

Is one week really enough to gather sufficient and appropriate levels of visitors to make a highly informed decision?

Did we increase the micro conversion rate but didn’t bother looking at what effect this had on the macro conversion rate? 

A lack of statistical understanding coupled with a desire to want tests to be successful very often erodes the integrity behind the test results.

If you put garbage in, you get garbage out 

Yes, never has this been more true than in A/B testing.

Come up with a test idea with little or no research, hypothesis, data and intelligence, and you are not going to make any learnings off the back of it.

I have previously shared how MVT (multivariate testing) should be renamed NHT – No Hypothesis Testing. When you run an experiment with multiple variations of different “things” mixed together, there is no intelligence or hypothesis there. 

The most successful companies with a genuine experimentation mindset are learning all the time about how to influence the decision making of their prospects and customers. 

There are very few successful companies in conversion optimisation.

Conversion optimisation doesn’t shift the needle

The harsh truth is that most companies don’t see that conversion optimisation has had a tangible, credible and trustworthy positive impact on their primary performance metrics. 

The number of times I have been in a meeting with a potential client who has been “running tests for the last few years”, whom I ask “what commercial impact have you seen on your bottom line?” which results in a very flat, non-committal answer, tells its own story. 

It's what industry thought leader Craig Sullivan explains simply as “the trough of disillusionment”.

If it doesn’t deliver value then why bother?

In the trough of disillusionment, it's understandable for the decision makers within a business to look at this investment in conversion optimisation and conclude “why are we bothering?”.

With acquisition long established as the easiest way for a business to get more sales, simultaneously lining Google’s big pockets, it's easy to see why investment in conversion optimisation can be seriously questioned – particularly for businesses who have first-hand experience of the butterfly effect which comes from a lack of understanding.

In summary

“Yes but what about this ‘butterfly effect killing the conversion optimisation industry?” I hear you say.

Well, following on from my points above, what are we left with? 

1. Conversion optimisation is fundamentally misunderstood

Which means that...

2. Conversion optimisation is underappreciated

Which means that...

3. Conversion optimisation gets little in the way of investment in people

Which means that...

4. Conversion optimisation is done with little (if any) intelligent user research insights

Which means that...

5. Conversion optimisation lacks any real form of intelligence behind what is being tested

Which means that...

6. Conversion optimisation is dominated by test “results” which are a lie

Which means that...

7. Conversion optimisation delivers little (if any) tangible & actionable visitor learnings

Which means that...

8. Conversion optimisation delivers little (if any) commercial impact

Which means that...

9. Conversion optimisation is considered irrelevant to the growth ambitions of the business

Which means that….  

We are left with a CRO industry and way of doing business which, though it can prove a competitive advantage (see Amazon as one example), is actually the most under-utilised part in the whole of digital marketing.

Until conversion optimisation is understood for what it truly is, it will remain as the biggest missed opportunity in global marketing.

True conversion optimisation is the beginning of a mindset transformation within a business, maturing to becoming a truly customer centric organisation that embraces experimentation and long term thinking to out-perform their competitors.

More on conversion optimisation:

Paul Rouke

Published 11 October, 2017 by Paul Rouke

Paul Rouke is Founder & CEO at PRWD, author, creator of the CRO Maturity Audit, and a contributor to Econsultancy. You can follow him on Twitter or hook up with him on LinkedIn.

40 more posts from this author

You might be interested in

Comments (2)

Pete Austin

Pete Austin, CINO at Fresh Relevance

Re: "I find it shocking just how many companies go years without stopping to think “is what we are doing really what our visitors are looking for?”."

Actually that's really easy to understand. Imagine the marketing domain as a sprawling landscape, where the height is a measure of marketing effectiveness. Each potentially successful technique looks like a little hill and most CRO is about finding the highest point of the hills that you already know about.

There are many other hills - potentially successful techniques - and some may be mountains representing "what our visitors are [really] looking for". But you won't find these mountains unless you *REDUCE* your budget on currently successful marketing and invest the money saved on trying random marketing that will probably fail.

If you can afford to keep on long enough, you'll eventually find those bigger hills and, by focusing your marketing there, can reap great rewards in future. But in the short term it's more likely your experiments will come up dry and you'll sell less, compared to doing more of the same old marketing, which is bad for your employers and disasterous for your personal job security. "I know it looks bad, but trust me" is not a great thing to have to say each month. So it's really not surprising that most companies don't search enough.

What's the solution? You need a real-time marketing system with really great ROI, so you can increase sales immediately and fund CRO out of your winnings, not your budget.

9 days ago

Michael Morgan

Michael Morgan, Chief Data and Marketing Scientist at Morgan Analytics Inc.

Paul, I think you're right in nearly everything. But you may be under-emphasizing the factors driving persuasion to click-through to the target landing page, which is a precondition to conversion.
- The probability of clicking through on an ad, link, search result, etc., is a function of appeal to the web surfer upon exposure, the design, messages and placement of ads or links. So, visitor traits and characteristics of the message or the intent of the search. are very relevant.
- This is why A/B testing can be biased or even misleading. There is simply not enough information in the convert/not convert results to explain the sequence of outcomes (landing and conversion). Explanatory variables are not accounted for.
- It is better to design experimental versions of the ad or linkage offered, distribute them randomly (with some good forethought) and predict the landing outcome with the conversion outcome given a successful landing.
- Eventually, perhaps, one can build a fully operational attribution model that distinguishes drivers of each. Interestingly, what drives click-through may be quite different from what drives conversion, so you need both, and you can learn a lot from comparing the drivers of each outcome (landing and conversion). The key is to predict the sequence of outcomes, not just the ending state.

8 days ago

Comment
No-profile-pic
Save or Cancel
Daily_pulse_signup_wide

Enjoying this article?

Get more just like this, delivered to your inbox.

Keep up to date with the latest analysis, inspiration and learning from the Econsultancy blog with our free Digital Pulse newsletter. You will receive a hand-picked digest of the latest and greatest articles, as well as snippets of new market data, best practice guides and trends research.