A recent edition of “Which Test Won” recounts an A/B test in which DIYthemes, a template system for WordPress, invited visitors to sign up for email newsletters. Version A used the headline “Get Email Updates (it’s free!),” and used social proof messaging to encourage registration: “Join 14,752 others and get free updates.” Below this was a box for an email address and a “join” button. Version B was identical, except that it omitted the line about 14,752 subscribers.
Voters at the site overwhelmingly chose Version A: 82% to 18%. But the majority was—in this case—wrong. “Version B, without the social proof messaging, got a 122% lift in email opt-ins,” Anne Holland notes.
So what happened? “Derek Halpern, Chief Persuasion Officer, DIYthemes, suggested that ‘joining 14,752 others’ just wasn’t compelling enough, or might have distracted visitors from the submit button below,” says Holland. “But we think another factor might be that the messaging didn’t clearly explain what kind of email updates the subscriber would receive—or the benefits of opting in.”
Whatever the case, it sparked a lively discussion in the comment section. Feedback from Holland’s readers included comments like these:
- “Maybe the ‘joining’ messaging misled people into believing they were signing up for a discussion list as opposed to the blog’s own updates?”
- “I think with the phrase ‘joining 14,752 others’ it seems to suggest that the sole reason to join is because 14,752 have joined.”
- “My own testing has shown that you should leave the reasons for joining to the Welcome Email. Every single extra character is one more reason to delay entering your email address and hitting Submit.”
There’s only one way to know for sure. Many experts had an opinion on this test—and most of them got it wrong. Even when the answer seems obvious, test and test and test again.
Source: Which Test Won.