CAPTCHAs -- those computer-generated images commonly used with website forms that challenge users to prove they're human -- are a popular tool in the arsenal against web spam.

But when looked at from a cost-benefit analysis standpoint, do they help or hurt conversions?

A great post on the SEOmoz blog details how one person set out to answer the question. 'chenry' of the SEOmoz community conducting a six-month study across 50 websites that he owns or has access to. For the first three months of the study, half of these websites implemented CAPTCHA on their forms and the other half didn't. For the last three months, the settings were reversed.

The results:

  • With CAPTCHA enabled, there was an 88% reduction in spam.
  • With CAPTCHA disabled, 4.1% of the conversions recorded were spam.
  • With CAPTCHA enabled, 7.3% of the conversions were either detected as spam or failed.

The analysis: CAPTCHAs do reduce spam conversions but it also seems to significantly increase the number of conversions that fail.

While further study would be useful to quantify exactly how many legitimate conversions fail because of CAPTCHA, the implication here is that the use of CAPTCHAs should be evaluated carefully on a case-by-case basis. If you're using CAPTCHAs on your websites, it probably makes sense to consider whether the pros outweigh the cons.

In the case of 'chenry' and his websites, when CAPTCHA was disabled, there were 2,134 total conversions. Of those, 91 were spam. While CAPTCHA reduced the spam conversions to only 11, were the 159 total failed conversions that CAPTCHA caused a smart trade-off? In other words, would you rather deal with 91 spam conversions or lose 159 conversions, some of which were almost certainly legitimate? Obviously, the value of each conversion will play a big role in answering this.

While 'chenry' mentions that he's now avoiding traditional CAPTCHAs and is instead experimenting with the "Honeypot" CAPTCHA technique, you don't necessarily need to ditch CAPTCHAs. They can be effective, especially in high-volume environments in which the value of each form conversion is low.

Where you determine that CAPTCHAs can be of value, you can minimize failed legitimate conversions by:

  • Choosing a CAPTCHA implementation that is legible. This would seem like a common sense way of making sure your CAPTCHA doesn't turn legitimate users away but chances are you've come across a CAPTCHA implementation that looked more like a Rorschach test than a CAPTCHA. Don't make the same mistake.
  • Offer an audio CAPTCHA. This is a good idea anyway since graphic-based CAPTCHAs are problematic for visually impaired users but audio CAPTCHAs can also assist users who have trouble reading the letters/numbers they've been presented.
  • Make it easy to refresh. If for whatever reason someone can't figure out a CAPTCHA, make it easy for them to try a new one, ideally without reloading the page completely.
  • Provide instructional copy. Be sure your users understand what a CAPTCHA is and how to complete it; provide some text that makes the process simple for users who don't know what they're looking at.
  • Add a first layer of defense. You don't necessarily need to include a CAPTCHA on every form you display. You can use CAPTCHA selectively when a first layer of filters flags a suspicious conversion. For instance, you could require a CAPTCHA after a form has been submitted and the expected referrer is not present or when a form is loaded by a user with an IP address from geographic regions that are not typical users of your website.

Photo credit: cogdogblog via Flickr.

Patricio Robles

Published 27 July, 2009 by Patricio Robles

Patricio Robles is a tech reporter at Econsultancy. Follow him on Twitter.

2647 more posts from this author

You might be interested in

Comments (1)


Craig Mackay, Website Operations Manager at William Hill Online

Great article - but "Add a first layer of defense" should be top, this is the key.

Looked into this a few years ago and found removing CAPTCHA increased conversions by anything up to 15%.  However also implemented various layers of security such as IP address recognition and monitoring, switching them on if required.

This is the best balance, CAPTCHA is useful but without doubt an annoyance for customers, especially when the majority border on being unusable.  Start by assuming that a customer should never see it - but ensure it gets displayed when the form is suspicious based on whatever data you have at that point.

about 9 years ago

Save or Cancel

Enjoying this article?

Get more just like this, delivered to your inbox.

Keep up to date with the latest analysis, inspiration and learning from the Econsultancy blog with our free Digital Pulse newsletter. You will receive a hand-picked digest of the latest and greatest articles, as well as snippets of new market data, best practice guides and trends research.