FAQ – About This Guide
We contacted several conversion testing tech vendors for inclusion in this guide. They were mostly testing tech we’ve heard of or worked with in the past. Any and all testing tech vendors are welcome for inclusion.
Please fill out this form if you’d like your testing tech to be included in this guide.
Information was collected during hour-long phone conversations with representatives at each tech company and through follow-up emails.
Tech Launched: The year the testing technology was launched. Not the year the parent company was launched. If a testing technology was acquired under a different name, the launched date is the year the acquired testing technology was launched. Not the year the parent company purchased it.
Last Update: The month and year we last updated this listing.
Types of Testing Supported:
A/B/n Testing: (also known as A/B testing) Allows you to test different versions of a web page against each other at the same time to see which page gets more conversions.
Multivariate Testing: Allows you to test each page element version against other page element versions at the same time to see which combination of elements gets the highest conversion rate.
–Full Factorial: A multivariate test that allows you to test every combination of page elements.
–Fractional Factorial: A multivariate test that allows you to test a portion of pages created as a result of combining different page elements.
Mobile Testing: Allows you to test different versions of mobile web pages against each other at the same time to see which increased conversions most.
Online Display Ad Testing: Allows you to test multiple ad creative against each other at the same time to see which optimizes conversions most.
Segment Testing: Allows you to define segments (such as visitors coming from pay per click ads) to see which version of the creative appealed most to each segment. Once completed, you can then target a different winning page per each segment.
Video Testing: Testing that involves online video on your site or landing page. There are several tech firms that offer online video testing. Please note: these firms typically create the video and test it for you using their own proprietary tech platform. We’ve yet to find one that sells a separate, self-service platform that allows marketers to test video on their sites.
Email Testing: Allows you to test different versions of email creative against each other at the same time to see which outperformed the other. Sometimes offered as part of a web optimization platform, but more commonly included with an email service provider.
Advanced Testing Features:
Behavioral Targeting: Analyzes visitor behavior and automatically serves visitors relevant pages, offers, ads, etc., based on past activity, such as page-view history, traffic source, or time spent on a page.
Geotargeting: The ability to segment traffic based on location and optimize Web pages for visitors in a geographic location.
Heatmaps: A visual representation of aggregate activity on a page. Commonly it’s a mouse tracking or click tracking feature that maps hot spots of activity. Some technologies offer eyetracking simulator software that uses proprietary algorithms to predict where visitors’ eyes travel on a page and produce heatmaps based on those predictions.
SaaS: (an acronym for Software as a Service) Users typically obtain a license from the provider to use the software on demand. These are also called “self service” and are usually priced at on a monthly scale.
Installed: (sometimes called “On-Premise”) Users typically have to install and host the software on their servers. Typically used by large publically-traded companies or companies that want all to keep all testing data secure on their servers.
Open API Available: (also called Application Programming Interface) An API allows different technologies to interact with each other. For testing tech, typically it means the ability to export test results to another marketing technology’s dashboard such as a web analytics platform. Very few API’s allow testing tech to fully integrate with other platforms such as content management systems (CMS), so that users can set up and run tests from the CMS, for example.
Custom Programming Available: Any test that involves transferring data from an outside software or server to the testing tech may require some level of custom programming. For example, a landing page that has personalized content such as visitors’ first names might require customer programming. Even some form tests require customer programming. Custom programming generally isn’t free or cheap.
Live One-on-One Training: This means you get free one-on-one training with a live person via phone or in person. Often it’s offered as hour-blocks of training. The goal is to explain how to use the testing technology. Not to be confused with consulting services that train you how to set up a testing strategy (what pages/page elements to test, etc.)
Canned On-Site Training: This could take the form of an internal blog, video tutorials, or how-to articles. Some vendors even offer calculators to help you determine how long a test needs to run to get conclusive results.
Help Desk: A web page that contains contact info for the support team. It might also have links to a FAQs section or commonly asked questions.
Forum: A place where users can post questions and answer questions or start discussions. Often the vendor’s support team will answer questions posted by users.
Live Chat: Allows user to interact instantly with members of the support team and get their technical questions answered in real time.
Available by Phone: Means the support team is available by phone.
Available by Email: Means you can reach the support team via email.
Available by Form: Means the support team is available via an online form.