User testing is an integral part of web design as it gives impartial and quantifiable insights into the customer’s experience.

As discussed at our recent JUMP multichannel event, usability testing removes any ego or opinion from product design which helps improve customer satisfaction and ultimately increase revenue and growth.

There are a number of different methodologies for user testing, including A/B or multivariate tests, heatmapping, or videos of real customer journeys.

As these case studies show, each different methodology can identify small changes that lead to massive increases in CTR, conversions and sales.

For more information on this topic, checkout our new Conversion Rate Optimization Report 2012 in association with RedEye

The research looks at tools, strategies and processes employed for improving conversion rates, as well as examining different areas of best practice.

Sidelines increases landing page conversion from 5% to 55%

When sports website Sidelines first started out it found that, despite a steady flow of traffic, only 5% of visitors actually signed up to its service.

So to try and increase the conversion rate co-founder Vinay Kuruvila ran six A/B tests.

1. The original landing page showed three screenshots of the product and one sentence below each describing a benefit of using Sidelines.

Due to the poor conversion rate they experimented with a simplified version that just had a random picture from the 2012 Super Bowl and a couple of sentences about the product.

As a result conversions went from 5% to 17%. Kuruvila suggests this is because if a screenshot doesn’t immediately show a customer what your product does, then it’s better to go for a simple, eye-catching photo.

2. For the next test they changed the landing page so it had a large background picture that covered the entire screen and reduced the text to a single tagline and one sentence explaining the product.

This test saw conversions creep up to 25%.

3. After trying a number of different taglines, Sidelines found that the deliberately vague option of “Follow Sports Together” was the most successful.

The theory is that it intrigues people and convinces them to sign up so they can find out more.

4. Initially the landing page only highlighted the sharing functions of the site, but when they added the words “Follow your favourite teams” conversions jumped from 25% to 40%.

This is because it made new visitors aware of the brand’s value proposition, so they knew what they stood to gain from signing up to Sidelines.

5. Like many startups Sidelines originally required new users to signup using their Facebook or Twitter accounts.

However after adding an email sign up option the number of people signing up using their social account increased by 7%. Kuruvila says this is because email signup option increased trust for the brand.

6. Kuruvila initially thought that Sidelines would appeal mostly to men, but traffic from Facebook ads proved that assumption to be incorrect.

The highest conversion rate “by far” was women aged 26-40 with the messaging “share photos, videos, articles and opinions about your favourite teams.” In addition, these women were inviting 35% more friends to join Sidelines than men were.

Finally, men who received email invites from women converted about twice as well as men who received email invites from other men.

It was clear that women aged 26-40 loved to share, but Sidelines wanted to ensure the landing page message appealed to the vast majority of users who start off as pure content consumers.

The compromise was a message that started off with the consumer value proposition and then include the sharing aspect: “Follow your favourite teams, join the hottest discussions and share photos, videos and articles with other fans.” 

This helped landing page conversions increase to 55%.

DHL uses A/B testing to achieve a 98% uplift in conversion rate

DHL ran an A/B test to try and increase signups to a free Import Tool Kit. It created a new page that made the signup form the main focus by making it larger and moving it to the top right of the screen.

All the text remained the same, but the image was changed from a cityscape to a picture of a friendly courier.

The new form design

The test ran for around four weeks in July, and as a result DHL achieved a 98+% conversion rate increase in two countries at a 95% statistical confidence.

ASOS reduces abandonment rate by 50% by changing one CTA

ASOS ran A/B testing at the sign-in page of its checkout to try and reduce the number of abandoned baskets. The original page had a CTA for new customers saying ‘Create your account’ that it changed to simply ‘Continue’.

This simple change reduced basket abandonment by 50% at this stage as consumers associate creating an account with a long process of form filling.

The new checkout design

In fact ASOS left the checkout process for new customers largely unchanged and still required them to choose a password and create an account, but this simple change in copywriting had a massive impact on the customer’s perception of what they were being asked to do.

The old checkout design

Hyundai increased requests for test drive by 62% using multivariate testing

As with other case studies on this list, Hyundai set out to improve conversions on its product landing pages.

This actually included three different goals that it wanted to optimise:

  • Primary goal was a brochure request and/or request for a test drive.
  • Secondary goal was a clickthrough from the car page to the first step of the funnel.
  • As a check, engagement (inverse of bounce rate) was also measured.

To test these different criteria Hyundai used multivariate testing, which is different to A/B testing as each change you make creates a new variation to be tested.

Original landing page

This allows you to track how each single change impacts conversion rates so you can find the best combination of factors. Using Traffic4U it tested eight different iterations by changing the following sections of the page:

  • New (SEO friendly) text versus control text. The hypothesis was that if Hyundai changed its normal text to SEO friendly text and it didn’t impact conversion rate, it could permanently implement it for SEO benefits
  • Extra call-to-action buttons versus no extra buttons. The thinking was that an extra call-to-action would encourage the user to complete the desired action.
  • Large photo of the car versus thumbnails. Hyundai thought that larger photos would entice the visitor and also confirm to them that they are on the right page.

The results showed that the combination of SEO text, extra CTAs and larger images increased the conversion rate (request for test drive or brochure) by 62% and there was a 208% increase in CTR (step 1 to step 2).

New landing page

User testing videos help increase sales by 9.5%

Appliances Online commissioned 125 user testing videos to help identify potential improvements to its product pages, giving it 250 hours of footage of customers browsing the site before making a purchase.

The company used the videos in conjunction with other tools, such as Click Tale, which provided heatmaps showing which elements of product pages users were interacting with the most.

Thanks to this insight it uncovered several issues with its product pages. For example, 70% said that pages were too busy, 17% said service information needed to be clearer, while 13% thought the video experience could be improved.

As a result, Appliances Online made these changes to its product pages:

  • Banners touting special offers were incorporated into the copy on the page, allowing the buy button to be moved above the fold.
  • The buy button was changed from dark blue to a more eye-catching green colour, while the text was altered from ‘Buy’ to a more descriptive ‘Add to basket’.
  • Previously product videos opened in a pop-up screen that took too long to load. The solution was to embed the video into the product pages, which is less interruptive, while consumers can also scan up and down the page looking at reviews and product specs, and the video stays still.
  • Creative copywriters were employed to draft unique product descriptions.

These changes resulted in a 9.5% increase in sales, while 37% more visitors viewed the product videos. As viewers of these videos are 57% more likely to add items to the basket, this was a big improvement. 

In addition, the number of reviews left by customers increased by 11%, while there was a 33% reduction in calls about delivery, as the information was more clearly visible on the product page. 

Veeam changed a single word an increased CTR by 161%

In an effort to increase conversions Veeam Software asked all visitors to its product pages what other information they would like to see.

A number of visitors answered “pricing” however Veeam sells through partners so does not publish pricing information on its site. However it does have a ‘Request A Quote’ link that leads to a sales inquiry form.

Simply by changing the CTA from ‘Request A Quote’ to ‘Request Pricing’ Veeam achieved a 161.66% increase in CTR from 0.54% to 1.40% with 100% statistical confidence.

This is a great example of increasing conversions by using customer feedback to improve the website.

Amazon made $2.7bn by asking one simple question

Consumer reviews are a proven sales driver in e-commerce, with research showing that 63% of customers are more likely to make a purchase from a site that has user reviews.

Amazon receives hundreds of reviews so it was faced with the challenge of how to curate them into a useful order for its customers.

It struck upon the best system by adding the question: “Was this review helpful to you?” This allows users to endorse the best reviews, with the top three being featured on the product page.

Other touches, such as allowing users to easily view the best negative or positive reviews, as well as some handy charts that summarise review ratings, make the large number of reviews manageable for users and more useful for Amazon.

Displaying the most helpful reviews has increased sales in the media products category by 20%, meaning this feature was worth $2.7bn to Amazon.

VOIP telephone service increases quote requests by 262%

In this test an anonymous B2B VOIP telephone services company set about increasing its number of quote requests by performing an A/B test of different page designs.

The original signup page

The company redesigned the page to simplify the form and reduce friction in the sign up process. The number of fields that users had to fill in was reduced from six to three, and it removed any requests for personal information.

The new signup page

Other changes included:

  • Changing the CTA from ‘Get Quote’ to ‘Show Me My Instant Quote’.
  • Giving an interactive quote online rather than getting a salesperson to call the customer.
  • Adding third-party security logos.
  • Adding a chart comparing its service versus traditional phones.

The result was a 262% increase in conversion rate from 2.4% to 8.8%.

David Moth

Published 22 October, 2012 by David Moth

David Moth is Editor and Head of Social at Econsultancy. You can follow him on Twitter or connect via LinkedIn

1719 more posts from this author

You might be interested in

Comments (4)


Deri Jones, CEO at SciVisum Ltd

We're all in the game of trying to optimise sites.

So it's great to hear how various techniques can make a difference.

But having said that - taking an evidence based approach, we have to realise we shouldn't just copy what these sites did!

It's possible that some of these techniques used are proven by evidence across many sites and examples and are therefore reasonable to copy.

But just because it works in one situation for one site, may not mean it works for another, or even on another page on the same site.

There are always too many other factors also in flux, that a change can't be attributed entirely to the one factor tested.

Factors such as - overlaps with : the Olympics , weather, marketing campaigns online, offline campaigns, promotional messages changing, competitors messages and campaigns changing, other changes on the site, overall traffic levels effecting site speed and error rates: all those can change the profile of visitors to a site: and change their expectation of the site.

So although this sounds like a bucket of cold water ....!

... What I'm really saying is positive:

That we need to be continuously measuring our sites and continuously testing. What worked on our site last year, may not work today, unless it's based on much wider evidence from many other sites and is no longer anecdotal, but based on real user evidence.

almost 6 years ago

Tom Howlett

Tom Howlett, Digital Marketing Executive at Koozai

Interesting article, it just goes to show that companies should always be looking to improve the pages on their site to help increase conversions. Especially when the smallest changes can see a large impact.

almost 6 years ago

Paul Rouke

Paul Rouke, Founder & CEO at PRWD

Thanks for the article David, and thanks also for featuring my ASOS article on their checkout testing and improvement around account creation. In summary if there is one (or in this case 2) different words that end users have such a negative view of its when they see 'Register' or 'Create an account'.

I typically ask users in research sessions "what are your expecting to have to provide when you are asked to register/create an account." After their initial & quite typical response of "I've just no time for any of that, they will ask for pages of information..." when I then ask them to specify what information they expect they will need to provide, they themselves conclude that actually it probably isn't any difference to if they were having to register/create account. They then conclude that perhaps its just a password that they will need to provide extra.

And that summarises the whole issue - create account/register is a huge perceived barrier for consumers that as was seen with ASOS is highly toxic to conversion rates. The simple move to treating new customers as just that, with wording like New Customers > Continue completely removes this barrier to entry.

We do typically find that especially for our multi-channel retail clients some users at the start of checkout do like to know that they can create an account at the end of their 1st checkout, so following the example of a brand like Lakeland is recommended.

I hope this helps to add some more rationale and consumer insight to the ASOS case study.

almost 6 years ago


Daniel Lee

One thing that a lot of AB Testers dont do, which quite frankly amazes me, is run an A/A test alongside the A/B - to test for consistency and significance.

over 5 years ago

Save or Cancel

Enjoying this article?

Get more just like this, delivered to your inbox.

Keep up to date with the latest analysis, inspiration and learning from the Econsultancy blog with our free Digital Pulse newsletter. You will receive a hand-picked digest of the latest and greatest articles, as well as snippets of new market data, best practice guides and trends research.