Never Send a Broken Email Again.

Why Testing Separates Professionals from Amateurs

Testing is the dividing line between professional email programmes and amateur ones. Professionals test every variable they can measure. Amateurs send and hope. The difference in results is not marginal — it is decisive.

The cost of sending broken emails is substantial. A single campaign with a broken link, a rendering error, or an accidentally exposed variable can cost thousands of pounds in lost revenue, damaged reputation, and wasted production effort. When you are sending to tens of thousands of subscribers, a mistake that affects 10% of them might represent hundreds of hours of wasted production time and thousands of pounds in missed opportunity.

But testing is not just about preventing disasters. It is about optimisation. A well-tested email consistently outperforms an untested one. Subject line testing alone can improve open rates by 20 to 30% in many programmes. Send time testing can boost engagement by similar margins. Content testing reveals what your audience actually responds to, replacing assumptions with data.

Testing at scale requires systematic processes and tools. You cannot manually check every variable across every email client on every campaign. Professionals build testing frameworks that automate the boring parts, standardise the QA process, and free human attention for the things that matter most — strategy, creativity, and analysis.

A/B Testing: The Right Way

A/B testing, also known as split testing, is the practice of sending two or more variants of an email to a subset of your list and letting the winner go to the remainder. When done correctly, it is the most powerful optimisation tool available to email marketers. When done incorrectly, it produces misleading results that lead to worse decisions.

What to test depends on your goals. For open rate optimisation, test subject lines, preheader text, and sender names. For click-through rate optimisation, test CTAs, body copy length, image placement, and button design. For conversion rate optimisation, test landing page links, offer framing, and urgency messaging. Prioritise tests that align with your current programme goals rather than testing everything at once.

Statistical significance is where most A/B testing goes wrong. A result is statistically significant when you can be confident it is not the result of random chance. The standard threshold in email marketing is 95% confidence, meaning there is only a 5% chance that the observed difference is due to luck. Many ESPs declare a winner prematurely based on a small lead that has not reached significance, leading to false positives.

Sample sizes must be large enough to detect meaningful differences. For a small list, A/B testing may not be practical because the sample is too small to reach significance. A general rule is that each variant needs at least 1,000 recipients for open rate tests and more for click tests, which have lower baseline rates. The smaller the expected effect size, the larger the sample needed.

Test duration should be long enough to capture engagement patterns. Most tests should run for at least four to six hours, and ideally 24 hours, to account for timezone differences and delayed opens. Very short tests risk missing opens from subscribers in different timezones. Very long tests delay the winner send, which can affect results if the offer is time-sensitive.

Winner handling requires a consistent process. When a test reaches significance, the winner is sent to the remaining list. When no variant reaches significance after the maximum test duration, send the control variant — not because it is better, but because changing something for no measurable gain introduces unnecessary risk.

Common mistakes include testing too many variables at once (which makes it impossible to know what caused the result), declaring winners too early, running tests on too-small segments, and failing to document results for future reference. Professional programmes maintain a testing log that captures hypotheses, results, and learnings over time.

Subject Line and Send Time Optimisation

Subject lines and send times are the two variables that most directly affect whether your email gets opened. Both benefit enormously from systematic testing.

Subject line testing variables include length, personalisation, emoji usage, urgency language, question formats, and benefit framing. Test one variable at a time to isolate its effect. For example, test a personalised subject line against a non-personalised one while keeping everything else identical. After determining whether personalisation works, test the personalisation format (first name only vs first and last name vs company name) in a follow-up test.

Emoji testing deserves special attention. Emojis can increase open rates by making subject lines stand out in a crowded inbox, but they can also decrease them if they appear unprofessional to your audience. The effect is highly audience-dependent. Test emojis for your specific list rather than relying on industry averages.

Send time testing is more complex than subject line testing because it interacts with subscriber timezones. A send time that works for subscribers in London may be terrible for subscribers in Sydney. The best approach is to test send time by segmenting your list by timezone, then testing different send times within each segment.

Day-of-week analysis reveals which days produce the best engagement for your specific audience. Industry averages suggest Tuesday through Thursday tend to perform best, but your results may vary. Weekends can work well for B2C audiences. B2B audiences typically engage more during business hours. Test rather than assume.

Content and CTA Testing

Content testing determines what your audience actually wants to read and what motivates them to act. It moves your programme from assumption-driven to evidence-driven.

Long vs short copy depends on your audience and goal. For transactional emails, short and direct usually wins. For educational content, longer copy often performs better because it provides more value. For promotional emails, the optimal length depends on the complexity of the offer and the level of trust with your audience. Test both directions.

Single vs multiple CTAs is a classic testing question. Conventional wisdom favours a single CTA to reduce decision fatigue, but multiple CTAs can serve different segments of your audience with different needs. The best approach is to test single-CTA designs against multi-CTA designs and track which generates more total clicks and which generates more conversions.

Personalisation depth extends beyond the subject line. Testing personalised content blocks, personalised product recommendations, and behaviour-triggered content reveals whether personalisation drives meaningful engagement for your audience. Start simple — first name in the body — and progressively test deeper personalisation.

Image vs text-only tests are particularly important given the prevalence of image blocking. Many email clients block images by default, meaning your beautifully designed newsletter might appear as a blank canvas with missing image icons. Test image-heavy designs against text-heavy or balanced approaches to find the right mix for your audience.

QA and Proofing Workflows

A structured QA process catches mistakes before they reach your subscribers. The most effective QA workflows combine automated checks with human review.

Pre-send checklists should be standardised and followed for every campaign. The checklist should include sender name and address verification, subject line and preheader text confirmation, link validation (every link goes where it should), image rendering checks, spam score testing, and unsubscribe link visibility. A laminated checklist taped to the monitor is surprisingly effective.

Link validation is one of the most common failure points. Automated link checkers can verify that every URL in your email resolves to the correct destination and does not return a 404 error. This is especially important for emails with dynamic content, personalised links, or UTM parameters that could break URLs.

Spam score testing involves running your email through spam filter simulators before sending. Tools like Mail-Tester, SpamAssassin, and MXToolbox analyse your email against hundreds of filtering criteria and provide a score. A score below five out of ten typically indicates issues that need addressing before sending.

Accessibility checks ensure your email is usable by subscribers with disabilities. Check colour contrast ratios, alt text presence and quality, heading hierarchy, and semantic structure. Screen reader testing reveals how your email sounds rather than looks, catching issues that visual review misses.

Rendering and Client Testing

Email rendering varies dramatically across clients and devices. An email that looks perfect in Apple Mail might be completely broken in Outlook.

The email client landscape is fragmented. Gmail dominates with roughly 30% market share, followed by Apple Mail, Outlook (both desktop and web), and Yahoo. Each uses a different rendering engine with different CSS support. Gmail strips certain styles and does not support embedded stylesheets properly. Outlook uses Word's rendering engine, which has severe HTML and CSS limitations. Apple Mail is the most standards-compliant but still has quirks.

Testing across clients requires rendering previews for the major clients. Litmus, Email on Acid, and other testing tools provide screenshots of your email across dozens of clients and devices. Review each screenshot for layout breaks, missing images, font fallback issues, and CTA rendering problems.

Dark mode rendering adds another layer of complexity. Email clients handle dark mode differently — some invert colours automatically, others apply their own dark mode styles, and others do nothing. Dark mode breaks designs that use white backgrounds with no borders or shadows, because the entire email becomes invisible against a dark background. Test your emails in dark mode on Gmail, Apple Mail, and Outlook.

Mobile rendering is non-negotiable given that more than half of all email opens occur on mobile devices. Single-column layouts are safer for mobile. Font sizes should be at least 14px for body text and 44px tap targets for CTAs. Test on both iOS and Android devices.

Deliverability Testing

Deliverability testing verifies that your email actually reaches the inbox rather than the spam folder.

Seed list testing involves sending to a controlled set of test addresses at major ISPs and checking where each one lands. Seed lists provide a snapshot of your deliverability health across providers. Regular seed list testing reveals trends — improving or declining deliverability — before they affect your overall programme.

Inbox placement monitoring tracks the percentage of your emails that land in the inbox versus spam across ISPs. A dedicated monitoring tool like 250ok, Return Path, or Everest provides ongoing visibility into your deliverability performance and alerts you to sudden changes.

Spam filter testing goes beyond spam scores to check how your email behaves in actual ISP spam filters. Different ISPs weight factors differently, so an email that passes Gmail's filters may be filtered by Outlook. Test across multiple providers regularly.

What You'll Learn in the Full Guide

Our comprehensive campaign testing PDF guide includes a complete A/B testing framework with sample sizes, duration guidelines, and statistical significance calculators. It provides QA checklist templates that you can customise for your team's workflow. A rendering testing tools comparison helps you choose the right testing platform for your budget and requirements. Spam testing thresholds and guidance help you interpret test results and prioritise fixes. The full guide turns testing from an occasional activity into a repeatable process that continuously improves your campaign performance.

Who Needs This Guide

This guide is for email marketers who want to move from sending to optimising, campaign managers responsible for campaign performance, QA teams who need structured testing processes, and marketing operations professionals building scalable workflows. If you send email campaigns and want every one to be as effective as possible, this guide is for you.