Online Game A/B Test - WhichTestWon

Results for: Online Game Test: Which Version Got More Players to Register? (Same Benefits, Different Images & Copy)

Winning Version

A

25 Comments

Reader Guesses:

  • VERSION B
  • VERSION A
Loading ... Loading ...

WhichTestWon's Analysis: (Click back to see versions A & B)

Version A’s ‘Free Town’ offer paired with an image of the virtual town got 40.1% more consumers to fill out and submit a non-required registration form after they installed the game they’d purchased offline.

WiderFunnel Marketing Optimization (see agency profile), a conversion optimization firm, used Google Website Optimizer (see tech profile) to conduct the A/B test on The Sims 3 game launcher page. All versions were designed to fit a screen at 600×800 resolution so that no buttons appeared below the fold. The test ran until it hit 99.9% conclusiveness.

Both pages had identical buttons and nav bars. The benefits for both offers were also identical, but the copy describing them differed. Version A’s headline emphasized the ‘Free Town’ offer whereas Version B’s headline emphasized the ‘Free Game Content’ offer. Images matched the narrative of the copy.

Gamers obviously valued the town over game content, like furniture and clothing. And with 12.2 million uniques per month, a 40.1% lift amounts to thousands of new registrants. Does your product or offering have multiple benefits? If so, do you know which is most appealing to prospects? It’s worth testing to find out.

> Tweet that you ‘Got it Right’ here.

> Tweet that you ‘Got it Wrong’ here.

Share

25 Comments: Post a comment

This one was easy for a fellow gamer like me.

We bend over backwards for free in-game items, especially if they might be exclusive.

Only 9% votes for A! This shows how important it is to understand your target audience (or that many people haven’t voted yet!).

When I saw the big FREE town and FREE points in the title in A, I was sold! Maybe I should register my copy and start playing again ;-)

David is correct. I totally was sucked in with the old rule of ‘people buy from people’. In this case, that was just downright dumb! You got me.

David - National Paid Search

David- nice call! Will keep that in mind next time. I didn’t realize the direct benefit of the free town + the color background seemed to make B more focused.

Version A has a couple of things going for it that I feel made it the clear winner with a 40% gain. First, the headline is much stronger than simply getting tips, tools and game content. The town has a higher value to the game play for the Sims line of games. Secondly, the image of the town is congruent and supports the value proposition of the headline. An interesting follow up test would be to test the copy in the button. I would suggest emphasizing the value of getting a New Town in the button vs. simply Free.

@David: Totally agree! “how important it is to understand your target audience”

@Yaj: you should *definitely* register your copy and start playing again :)

80% of us got it wrong (I’m not a gamer sound found myself in the 80%, great test!)

I think as a perhaps overly-trained marketer, I got sucked into the lure of the white space and white background on the other version. Every time I see it I want to vote for that one even though I *know* it’s wrong!

I confess to voting for the “best practices” version instead of the one that obviously resonated best with the audiences. Another great example of why testing is so critical!

Fascinating as usual. Not understanding the value of the town offered in A had me guessing wrong. As others have said, knowing your audience is key.

Shows that it helps to know the product. This is the first one I have gotten wrong since I have no familiarity with SIMS or similar games. Nice test though!

Funny. I’m a gamer too – actually used to play the Sims with my wife quite a bit – but I selected version b because we were always into the outfits more than the towns. Know thy audience yes, but also test, test, test. Great one!

What an interesting test! I’m not a gamer at all but I’ve heard of The Sims so I thought it must be like an online neighbourhood (yep – I’m totally ignorant of video games!) and having the pic of a (new?) town would be a huge draw for fans wanting to develop the game further. Since I don’t play the game, I don’t recognise the characters in Version B so they don’t mean anything to me. And I love the violet-coloured night sky, too!

I love white space. Take a look at how Google optimises every pixel-great example is Youtube and the multi-variate testing done there.

But this test is perhaps more a question of emotion. I love pretty pictures too!

With this test I was torn between the two versions and decided to first analyse, then choose. Based on the heatmaps I went for version B.
http://www.stott.nl/2010/05/picutures-and-words-best-conversion-lift.html

Would you have done the same?

I have to agree that the picture of the town is beautiful and drives a lot of emotion. I bet if you used the same picture in both A and B you’d still see A come out as the winner but I don’t think you’d see as much of a difference in results between the two.

Totally had me fooled!
I would have never guessed that one.
Yes, need to know your audience, but also any assumption and opinion can kill your marketing efforts.

Ha Ha. I lose. But I’m not the right demographic. I was sure B would win since I recognized the SIMs characters. The town meant nothing to me.

Amazing!

I lost :)

I was leaning towards A but settled on B because of the characters (which are important for other types of offers).This example was really illustrative: 40% difference is huge! and 80% of us got it wrong. It goes to show you the importance of testing. Thanks, I learned something.

Version A tells you what you get and how to get it in one short sentence. Version B is vague about how to get the free items until you read the second line.

Version B also has a lot more wording in the body. When meeting a wall of text I would suspect the player is even less likely to read past the headline.

Another benefit of A is the placement of the action button. It lines up with the headline and isn’t broken with a distracting picture like version B.

Many people here are taking the time to study each image and compare them. This is probably why version B gets more votes. However, in reality the players are in a hurry to play. Unless you can tell them the information fast and not distract them they won’t even read the content.

This page on this website would be much more useful if it actually had a link to the test to see the different versions!
http://whichtestwon.com/archives/3668

I would love to see version B, but I don’t. Can anyone help me find it please?

It’s almost too simple to have to mention, but nobody really has: When you change more than one variable (the copy content, the copy length, the benefit emphasis, AND the design) -what solid conclusions can you draw as to WHY A works better than B? None at all. Too many variables are different between the two versions to draw many useful conclusions.

GET OUR FREE, ADDICTIVE WEEKLY TEST RESULTS