Multivariate testing is one of the most powerful tools available to online publishers. But many of them don't use it for various reasons, from lack of knowledge about multivariate testing to lack of simple testing solutions.
At a weekend hackathon event, a couple of developers decided to change that by building a Headline Split Tester WordPress plugin that gives WordPress publishers the ability to set up A/B testing of their post headlines.
One of the developers explained on his blog:
When you set the alternate headline, we will A/B test your original headline and your alternate headline until one is deemed the winner (most clicks out of X total impressions shown). Then we show that headline going forward.
Simple? Yes. But that doesn't mean WordPress publishers can't benefit from using the same sort of tools larger publishers increasingly use.
One of those large publishers, The Huffington Post, was actually the inspiration for the plugin. As the plugin's developers note, the average blogger or small online publisher may not have the resources, knowledge or desire to implement similar A/B testing using solutions like Google Website Optimizer, which may be free and relatively easy to work with, but require some technical skill and labor to implement.
If it becomes popular, the Headline Split Tester plugin will probably be the first introduction many publishers, particularly smaller ones, have to multivariate testing. But there's no reason the relationship should end there.
There's plenty of opportunity to extend the functionality currently offered. For instance, testing could be conducted across multiple groups, with each group consisting of traffic from unique sources (eg. organic search, social networks). There's the potential to integrate testing with data from third party services offering APIs (think analytics and advertising platforms). And, of course, split testing could even be applied to post content itself. As Joshua Benton at the Nieman Journalism Lab notes, there's another, more sophisticated WordPress multivariate testing plugin in development as well.
While there is some debate in old media circles about this sort of thing, at the end of the day, when done thoughtfully, multivariate testing has the potential to help publishers produce better, more relevant content. And that's good for everybody.
Patricio Robles is a tech reporter at Econsultancy. Follow him on Twitter.
Recommended reports
Online Measurement and Strategy Report 2011
An analysis of the online measurement industry based on a survey of digital marketers. The 67-page report and 29-slide summary presentation contain valuable insights into the use of paid-for and free web analytics tools. Beyond web analytics, the report also looks at a range of data sources and information requirements, and the extent to which companies are able to develop a coherent measurement strategy.
Multivariate Testing Buyer's Guide
Econsultancy's Multivariate Testing (MVT) Buyer's Guide is an invaluable resource for those investing in site testing technology, with profiles of 13 leading technology suppliers and detailed information about the trends and issues affecting the online testing sector. The 127-page report also contains best practice and tips for identifying the right MVT vendor to meet your needs and useful advice for those just starting to get involved in this area.
3:43PM on 25th November 2010
Not the first time I read about this plugin in the last couple of days, but there's a doubt I still have unresolved: is this gonna affect SEO negatively?
4:36PM on 25th November 2010
nah it'll be sound Gab. Take it easy.
Director at Browser Media
4:42PM on 25th November 2010
We are big fans of multivariate testing and massive fans of Wordpress but are not wholly convinced by the logic of split testing titles for posts - is it not going to cause all sorts of issues with the permalink? We can only assume that you will have to choose one permalink to use as it would be problematic to use different urls for the same content. It is also important to establish what you are actually testing - if it is ranking success, then having a post title that is different to the permalink (assuming the permalink stays the same) is surely likely to be less successful that a post that has the same keywords in both the title and the permalink? Would be interested in exploring it in more detail but can't help feel that this is going to potentially cause confusion. Happy, as always, to be proven wrong.
5:48PM on 25th November 2010
Manager at Conversionation
11:30AM on 26th November 2010
One way to know: test :) Will report soon.
Manager at Conversionation
11:49AM on 26th November 2010
Tested. It's quite simple and basic. It shows the two headlines and the one that gets most clikcs "wins" (you see the data of how many times each times each headline has been clicked (to see the clicks of headlines of older posts that you want to test, it will obviously count from then on, after installation). Permalink doesn't change. However, and this is reason why I uninstall now and won't test SEO impact: when the alternate headline shows, all "likes", tweets, stumbles etc. dissapear, for instance in Topsy and Digg Digg.
So: still work to do. Nice initiative though.
Client Services & Operations Director at These Days
11:57AM on 26th November 2010
Good thinking, why didn't we think about that before :-) Nice! @JP: Curious about your findings.
3:01PM on 29th November 2010
Thanks Jean for testing. Good to know that counting of tweets etc. starts from zero with alternate heading.