Definition
A/B tests are best for testing a single change: subject line A versus subject line B. Multivariate tests change multiple elements simultaneously — for example, testing three subject lines, two hero images, and two CTA colours at the same time — and analyse how the elements interact. Multivariate tests require significantly larger sample sizes because they must measure every combination. For most email marketers, A/B testing is more practical. Multivariate tests become viable when you have lists of 100,000+ and need to optimise multiple elements simultaneously.
Related Glossary Terms
A/B Testing
A/B testing in email marketing is the practice of sending two variations of an email to a small sample of your list to determine which version performs better before sending the winner to the remaining subscribers.
Abandoned Cart Email
An abandoned cart email is an automated message sent to customers who added items to their online shopping cart but left without completing the purchase. It is one of the highest-converting email types in ecommerce.
ARPU (Average Revenue Per User)
ARPU (Average Revenue Per User) is a metric that measures the average revenue generated per email subscriber over a specific period, used to evaluate list value and campaign effectiveness.
Attention Rate
Attention rate is the percentage of email opens that last longer than 5 seconds, distinguishing genuine reads from passive opens, preview-pane views, or Apple MPP auto-loads.
Bounce Rate Threshold
A bounce rate threshold is the percentage of hard or soft bounces at which email platforms and ISPs begin to penalise sender reputation, typically around 2-5%.
Bounce Rate
Email bounce rate is the percentage of emails that were rejected by the receiving server before reaching the recipient. It is a key indicator of list health and data quality.