Real Estate Lead Gen Box Test Results
| VERSION B | |
How site visitors voted:
VERSION A (52%)
VERSION B (48%)
|
Actual Test Results:
The cleanest design combined with the most benefit-focused copy won. Version B got 8.5% more people to click the search button. ZAAZ, Inc., a full-service interactive agency, used Microsoft’s Experimentation Platform to conduct a series of multivariate tests on the MSN Real Estate homepage to improve lead generation. The many test panels included variations on graphics, headlines, button copy, and layout. Layouts that were simpler, with no tabs or icon graphics tended to work better. On the other hand, uber-functional headlines such as ‘Find Existing Homes’ lost out to ‘Find Your Dream Home.’ We guess the old direct response aphorisms of “KISS” (Keep It Simple, Stupid) and “Sell the sizzle, not the steak” still ring true online today. This is a useful test to show any Web designer (or client) who thinks it’s a great idea to put cute icons on your page. Sometimes it is… but sometimes an icon can depress response rates perhaps because it distracts from the task at hand, rather than enhancing it. Don’t put an icon on a high-traffic, conversion page or form without testing it first. WhichTestWon.com is sponsored by: WiderFunnel Conversion Optimization, the A/B and multivariate testing firm that guarantees Conversion Rate Lift. Click here for info on improving your site’s results – risk free – today! |
Also, could you guys leave a shot of the losing image in the follow-up article so we can further analyze it without leaving this page?
Thanks.
I’m wondering how many users were experienced internet users/real estate seekers as opposed to less experienced users?
I liked Version B because it is much faster with no tabs to mess with, but I picked Version A for this test because it does a better job of “hand holding”.
Jason – definitely a good idea. I’ll see if it can be added to our next site upgrade. Anne
Version A seemed like the winner to me (for the same reasons as Richard mentioned), but the single-line form in Version B seems a lot easier to use than the cityANDstateORzip form in A. I wonder how A’s layout with B’s form would perform.
The reversed out headline helps catch the eye in B I think. B also looks more like a simple search utility and less promotional. Version A would require 4 clicks to search foreclosures it looks like where B would require only 3, perhaps 1 less if the focus is on the form field when the page loads.
I agree with Amy. That would be an interesting test to follow up with.
I didn’t see any mention of testing procedure. Was it a truly random test? What was the sample size? Was it large enough for statistical significance? If so, what is the confidence level and the margin of error? A footnote with this info might be a good way to give the interpretation more credibility.
Hi Simon – great suggestion! To answer some of your questions: Yes, it was a truly random test. Equal traffic was sent to six versions of this lead gen box. I will have to check with my contact at Microsoft’s Experimentation Platform to find out the exact sample size, confidence level, and margin of error, but I have already been assured that the sample size was large enough for statistical significance. Thanks, again, for your input!
Natalie, Senior Reporter, WhichTestWon.com