Resources · glossary
A/B testing
Showing two random groups of visitors two versions of a page and checking which one performs better. It settles arguments about taste with numbers instead of opinions.
- A/B test
- split testing
- split test
- conversion testing
- 01
What it needs to make sense
Traffic. To detect a difference of around twenty per cent you usually need several hundred conversions per variant. With five enquiries a month a test will settle nothing, even though the chart will show something.
- 02
One change at a time
A test of two completely different pages will tell you which is better, but not why, so nothing can be carried over. Changing a single element gives you knowledge that stays after the test ends.
- 03
When to stop
After the planned period, not the moment a variant starts winning. Peeking at the result and stopping the test on a favourable reading is the most common way to get a false result.
- 04
A cheaper alternative for low traffic
Talking to five customers and watching where they get stuck. At a small scale this gives you more than a test that will never reach significance anyway.
Related terms
- ConversionA visitor taking the action you care about: a purchase, an enquiry, signing up to a list. The conversion rate is the share of such people among all visitors.
- UX auditA review of a website for whatever makes it harder for visitors to reach their goal, ending with a list of problems ranked by impact. It is not a graphic design project.
15 minutes. No sales pitch.
A term in a proposal sounds foreign? In 15 minutes we explain what it means for your project.