Entries for the 2012 email testing awards were free, compliments of Awards Sponsor Marketo. Our goal was to encourage as many marketers as possible to enter their tests in order to inspire others. Judging took place in June.

Judging was non-biased. We did not favor any particular agency, testing platform, or brand. Our goal was to spotlight tests that would be inspirational for the marketing, analytics and Web design community. Each submission was judged by the following criteria:

#1. Best Practices & Conclusiveness
Was the test conducted properly? Were the results conclusive? Did the test meet the stated objective?

#2. What was measured?
Was the conversion activity measured by the test directly relevant to the objective? Did measurements take any KPIs (key performance indicators) into consideration beyond immediate clicks?

#3. Results Data
What were the actual results? Did the test cause a substantial conversion gain? Note: All entries had to reveal their data to the judges, but we did agree to keep some data private. Also, judges did not automatically award the top gaining tests, because gain is related to how well the control creative was previously optimized. It’s not fair to judge a bad original with huge gains against a great original with smaller gains.

#4. Goal
We looked at the quality of the goal and whether it was clearly defined. Tests that had hypotheses that were generated from previous test data or site analytics had the edge in this category.

#5. ‘E’ Factor
This was really the excitement factor and applicability of the test itself. Were the results surprising, proving that you can’t depend on your intuition or “best practices” to guarantee best conversions? Was the test one that many, many marketers could and should copy on their sites or landing pages? Would the lessons learned be a useful tool to help marketers convince the powers-that-be to let them run similar tests?

Meet Our Judges

	Shonal Narayan Integrated Marketing Manager, Marketo
	Justin Rondeau Editor & Evangelist, WhichTestWon.com
	Anne Holland Publisher, WhichTestWon.com

Campaign Monitor