Factorial Test Design : Which Test Won


Term: Factorial Test Design

WhichTestWon’s  Definition: A “factor” is any element of a Web page that might be affecting conversions. This would include the layout, images, headline, navigation, etc. If you run an A/B test looking for data on a particular factor, you are limited to changing just one factor on the page at a time. (You can also run an A/B test and test completely different pages, but you would only learn which page won, not how individual factors on the new page made the difference.)

Multivariate tests, on the other hand, allow you to test multiple factors on a page and get specific data on each one’s performance, all at the same time. There are two main type of multivariate tests. Full-factorial tests are, in essence, multi-celled A/B tests, with a test panel for each of the factors and all of its permutations. These can require a lot of traffic to obtain conclusive results, but the data may be worth it. Fractional factorial tests, on the other hand, use complex (often Taguchi-based) mathematics to extrapolate results from a smaller number of test panels. These tests may require less traffic, but some statisticians do not think the results are as conclusive as other types of tests might be.

> See list of all 125 Testing Terms
> Free PDF: Get 40-page booklet of all 125 Testing Terms
Looking for a full-service testing firm to improve your site's results more easily?
> Click here for info on our Sponsor WiderFunnel

Join the conversation:

Please note:
1. Comment moderation is enabled, I'll do my best to approve comments asap.
2. If you have any trouble submitting comments, please send me an email (anneh at whichtestwon dot com).

Free Glossary Offer!

Get your handy 40-page booklet of the Top 125 Testing Terms
(value $19.95)