Using a CAPTCHA is a way of announcing to the world that you’ve got a spam problem, that you don’t know how to deal with it, and that you’ve decided to offload the frustration of the problem onto your user-base. As statements go, that’s pretty lame.
If you ran a high street store, you wouldn’t force your customers to mop the floor before you serve them, on account of the people who came in earlier with muddy boots. That mud is your problem, not theirs. The same goes for spam. reCAPTCHA bothers me the most because it tries to sugar coat users’ frustration and make it palatable to site owners. Helping digitise and preserve literature is a worthy goal, but it’s a task that’s utterly at odds with what your users are trying to do at that very moment.
Sometimes site owners seem to think they really need CAPTCHAs, having been hurt by spam in the past. Without hard evidence, it can be difficult to persuade them otherwise. Well, here’s some good news – I recently got chatting to Chris Korhonen of Animoto, who’s kindly shared some data that could help you talk your clients around.
In case you don’t know, Animoto is a web app that allows users to create video compositions from their photos, video clips and music. According to their press releases, their registered user-base grew from 300k users in August 2009 to 2 million users in November 2010. Roughly speaking, that’s 2,400 new registrations a day – giving them plenty of data to run quantitative research.
In Q1 2009 they ran a simple experiment, looking at the impact of CAPTCHA on registration completion. This is what their signup form looked like at the beginning of the study:
Users were directed to the sign-up form direct from the homepage before they could interact with the product. As you can see, there was a CAPTCHA at the bottom of the form (powered by reCAPTCHA). With this design, they had a conversion rate of roughly 48%. They then removed the CAPTCHA, and it boosted the conversion rate up to 64%. In conversion rate lingo, that’s an uplift of 33.3%! They replaced the CAPTCHA with honeypot fields and timestamp analysis, which has apparently proven to be very effective at preventing spam while being completely invisible to the end user.
To quote Chris:
“We left the test running until the results were statistically significant to a 99% confidence level. We’ve followed the same testing methodology with other bits and pieces – removing demographic fields, moving things around, and so on, but nothing has moved things more than a couple of percent.”
Got any evidence of your own about CAPTCHAs and conversion rates? Comments, please…
————–
Edit #1: For some reason this article has hit the front page of Hacker News and is getting quite a lot of traffic. I should mention that yes, I acknowledge CAPTCHAs are of course sometimes unavoidable. That doesn’t mean, however, that we should ever feel good about using them, nor should we fool ourselves that users don’t mind them.
Edit #2: Links added to articles about honeypot fields and timestamp analysis.
