Skip to content
dvnt.io

Resources · glossary

A/B testing

Showing two random groups of visitors two versions of a page and checking which one performs better. It settles arguments about taste with numbers instead of opinions.

Also called
  • A/B test
  • split testing
  • split test
  • conversion testing
  1. 01

    What it needs to make sense

    Traffic. To detect a difference of around twenty per cent you usually need several hundred conversions per variant. With five enquiries a month a test will settle nothing, even though the chart will show something.

  2. 02

    One change at a time

    A test of two completely different pages will tell you which is better, but not why, so nothing can be carried over. Changing a single element gives you knowledge that stays after the test ends.

  3. 03

    When to stop

    After the planned period, not the moment a variant starts winning. Peeking at the result and stopping the test on a favourable reading is the most common way to get a false result.

  4. 04

    A cheaper alternative for low traffic

    Talking to five customers and watching where they get stuck. At a small scale this gives you more than a test that will never reach significance anyway.

Related terms

  • ConversionA visitor taking the action you care about: a purchase, an enquiry, signing up to a list. The conversion rate is the share of such people among all visitors.
  • UX auditA review of a website for whatever makes it harder for visitors to reach their goal, ending with a list of problems ranked by impact. It is not a graphic design project.

← All terms

15 minutes. No sales pitch.

A term in a proposal sounds foreign? In 15 minutes we explain what it means for your project.