It's one of the most powerful tools in an online publisher's arsenal but it's one that few make effective use of. I'm talking about multivariate testing.

As part of its effort to turn its business around, Yahoo is using multivariate testing extensively on its homepage to figure out which combinations of content and features produce the best results.

According to Yahoo's new CEO, Carol Bartz, Yahoo has tested 141 different versions of its homepage. As one of the most trafficked homepages on the web, Yahoo is hoping that it can benefit by figuring out which of the homepages it's testing keep users around longer, not only boosting the value of the homepage itself but potentially giving Yahoo a greater opportunity to get users engaging with all of its services.

What's truly amazing: according to, Yahoo's homepage currently uses 33 different code bases (that's something Bartz wants to change). That means that its testing may be quite complex depending on how different the homepages being tested are from one another.

Needless to say, that Yahoo is using multivariate testing on an overly complex homepage that is built using a less-than-ideal hodgepodge of code bases hints at just how important Yahoo feels getting its homepage 'right' is.

And that's a good lesson for those of us whose homepages are a bit simpler. Thanks to the wide availability of multivariate testing tools, some of which are free, there's no excuse not to get into the act. Increased user satisfaction, usage growth and increased revenue are just some of the benefits I've personally seen achieved with even the simplest of A/B testing and it will be interesting to see what the results of Yahoo's multivariate testing are on the Yahoo homepage, and its bottom line.

Photo credit: Yodel Anecdotal via Flickr.

Patricio Robles

Published 26 March, 2009 by Patricio Robles

Patricio Robles is a tech reporter at Econsultancy. Follow him on Twitter.

2642 more posts from this author

You might be interested in

Comments (6)

Save or Cancel


Multivariate testing is pretty amazing :)  Good to see them doing it.

Would be interesting to know which company's providing them with the test tools. I'd personally recommend Google Website Optimizer as it's easy and free (wrote about it here Drawbacks being no advanced reporting tools and being able to run multiple and really quite sophisticated big tests.

over 9 years ago


Bob Page

The technology was built internally.  We looked at commercial services but none of them would work for our purposes.

(I led the team that developed the experimentation system.)

over 9 years ago



It'll be interesting to see if they do any segmentation within the MVT.  That is, understanding that certain MVT combinations work better with some segements of their audience than others. 

Agree that MVT is underutilized and underappreciated.  It ranks up there with organic SEO and email marketing in terms of really good ROI.

GWO is a great tool which we use.  Starting to look at others to see if we can do MVT within segments (demographics, geographies, etc.) to further improve conversion rates.

over 9 years ago


Ian Tester

Actually, yahoo! have been doing multivariate testing for years using some pretty nifty in-house software, and it works well for them because with their volume of traffic, they get results quick.

But that's no reason for everybody else not to do it as well: it's now easy and free, it just requires some planning (but we all plan new feature and design releases propeorly, right?) ;-)

over 9 years ago

David Iwanow

David Iwanow, SEO Product Manager at

Im impressed, google uses mvt for its search results, so the website with the worst usability obviously should take notice and start reducing some of those damn popup ads, flash ads, and animated mouse overs...

over 9 years ago


Russell Sutton

Hi Bob

im fascianted to hear that Yahoo! developed their own in house software for multivariate testing. Would none of the off-the-shelf systems works for Yahoo?

Do you have any plans to release the software in much the same way as Google has done with the Website Optimiser?



about 9 years ago

Save or Cancel

Enjoying this article?

Get more just like this, delivered to your inbox.

Keep up to date with the latest analysis, inspiration and learning from the Econsultancy blog with our free Digital Pulse newsletter. You will receive a hand-picked digest of the latest and greatest articles, as well as snippets of new market data, best practice guides and trends research.