This comes from our brand new report The Past, Present and Future of Website Optimisation, produced in association with Qubit and written by Morag Cuddeford-Jones.

The report is based on interviews with senior executives working across the digital marketing and ecommerce world.

While conducting the interviews we discovered that although testing for optimisation of the customer experience is gaining traction thanks to more easily integrated technology, companies are still finding this is not without its challenges. There is an inherent reliance on data, analytics and customer information that many of the executives we interviewed for this report admitted was not as seamless as it could be.

Here are a few more more takeaways from the report.

How much testing is ‘just right’?

The majority of executives interviewed indicated that they started testing slowly. Handfuls of tests were carried out on non-business critical elements of the digital strategy such as webpage colourways.

These tests delivered marginal returns and were able to be undertaken largely because of low cost to perform as well as low risk to service delivery.

For one organisation a delicate approach might mean one or two tests a month initially, while for others their start point was much higher at 20.

Few of those canvassed indicated that they had begun seriously examining testing as an optimisation strategy more than two years ago, with most activity for all interviewees being in the last 12 months.

However, having proven their effectiveness with marginal implications to the bottom line, executives gained confidence and began integrating testing more deeply into the organisation, both in terms of volume and impact.

Working with a legacy

Being able to integrate testing into existing digital technologies and techniques is perhaps the biggest stumbling block for many of the executives interviewed for this report.

For some, the initial technology integration has been the greatest hurdle, for others challenges have arisen where testing has moved them further towards developing a more sophisticated customer experience.

40% of interviewees noted that testing integrated well with their CMS systems with only a few adjustments needed and a further third ran a separate system that worked well alongside with few issues.

However, a quarter of respondents stated that they had experienced or were continuing to experience project-limiting problems around integration.

Daniel Sale, Head of Digital Marketing at Investec Private Bank expanded on these challenges:

“We don’t have the testing functionality packaged within our CMS so if we want to unleash it we’d have to buy something in. There are some fabulous products available that would work with our systems but they can be expensive and unless you have an in-house design team, it’s going to suck up a lot of agency time and cost.”


There is evidence that a company’s increasing sophistication in testing cannot simply be determined by the frequency of the tests they perform. Far more important is the impact it’s had on their optimisation strategy.

Respondents agreed that initial tests can help make changes to site elements that deliver marginal improvements in conversion, but there has also been a swift realisation that testing can deliver far greater optimisation benefits.

For Tele2, increased testing sophistication has led the company to delve deeper and deeper into layers of customer behaviour to increase acquisition, conversion and retention.

Here’s Tele2’s path to optimisation through testing…

Source: Tele2

For lots more insight download the full 26-page report: The Past, Present and Future of Website Optimisation.