How to Test Landing Page Variants With a Holdout

The short answer: the page changes on a Tuesday and conversions rise the next week. Nobody can say whether the copy, the season or a competitor did it. If the change is copy only, a new headline, body line and button, AdAlign's Variants is the quickest way to test it. It publishes the rewrite through the AdAlign tag, holds back 50% of matching visitors as a control by default, and reports lift once each arm has 100 visitors (how it works). It sits on the Scale plan, from €1,290 a month (pricing, checked on 27 October 2026). Find the copy to test by scoring the pair in the free audit first. Without Scale, a Google Ads custom experiment runs a free holdout for Search traffic. Either way: work out the sample, split the traffic, leave both arms alone until the sample is in, then ship the winner.

Disclosure: I earn a commission if you buy AdAlign through a link on this page, and this page recommends it. Every claim about AdAlign links to AdAlign's own pages.

That is a before-and-after comparison, the test most teams run by accident. Nobody chose it. It is what a page change produces when nothing else is set up. A holdout keeps part of the traffic on the old page at the same time, so the page is the only difference.

Do the sum before the split:

1. The sample, in weeks. For example, at a 3% conversion rate, detecting a rise to 3.6% at 80% power and 5% significance needs 13,050 visitors per arm (sample size calculator). With 2,000 paid visitors a week split 50/50, each arm gets 1,000 a week, so 13,050 / 1,000 = 13.05 weeks. Plan for 14.
2. The pieces you already have. The current page is the control, and the score from the free audit tells you which slot to rewrite. The conversion event you already track in Google Ads or your analytics is the measure, so nothing new has to be built for the free route.
3. One metric and one stop date, written down. Conversions per visitor on the page, and the date the planned sample is reached. AdAlign's own disclaimer asks you to test its recommendations against current performance before a full rollout (disclaimer). The stop date is what makes that test mean something.

The free holdout: a Google Ads custom experiment

Google lists landing pages among the things custom experiments are typically used to test (Experiments page). The experiment arm copies the campaign. You change the final URL in that arm to the rewritten page, and the two arms share the traffic and the budget.

1. Check the campaign is eligible. Custom experiments run on Search, Display, Video and Hotel campaigns. For Search, the base campaign has to be active, use Smart Bidding and carry exact or phrase match keywords, and only one experiment runs at a time (set up a custom experiment).
2. On AI Max campaigns, turn Final URL expansion off. Otherwise Google can send either arm to a third page, and the test compares nothing (Final URL expansion in Search).
3. Create the experiment. Experiments, then the plus button, then Custom. Choose the campaign, the success metric and the dates (set up a custom experiment). Point the treatment arm's ads at the new page.
4. Use a cookie-based split. With a cookie split, a person sees only one version however often they search. A search-based split can show the same person both (experiments methodology). For a page test: one person, one page.
5. Leave it alone. Google warns that changes to either arm while the test runs make the results harder to read (set up a custom experiment). Its monitoring page shows the difference with a confidence interval, 80% by default and adjustable, and marks significant results with a blue asterisk (monitor experiments). Read it on the stop date. Not before.

This covers Search traffic only. For Meta, an Ads Manager A/B test with each cell pointing at a different page does the same job, and this landing page test use case has the setup.

The AdAlign route: Variants with a built-in holdout

Variants, labelled beta, starts from the low score rather than from a blank page (how it works):

1. Install the AdAlign tag on the domain. Variants only touches pages that carry it.
2. Pick the scope: one ad, an ad set, a campaign, or all paid traffic on a path such as /sofas.
3. Review the draft. Variants rewrites three text slots (headline, body copy and call to action) and shows the original as control beside the personalised version. No images, no colours.
4. Confirm any price or discount claim. Those ask for an explicit confirmation before publishing.
5. Publish with the holdout. By default, half of matching visitors stay on the original.
6. Read the lift. It is reported once each arm has 100 visitors, including when the rewrite loses. A variant pauses itself if the page changes underneath it.

The pricing page lists landing-page variants and a holdout lift readout per variant on Scale (pricing). Once a variant wins, copy the winning text into your CMS as well, so the page keeps the result if the tag ever comes off.

Reading the lift without fooling yourself

An early lift figure is useful. It shows up at 100 visitors per arm, so the tool has something to show in its first week. At a 3% conversion rate, for example, it is also 100 x 0.03 = 3 conversions per arm. Treat the early number as proof the test is running, not as the answer. Keep both arms going until the planned sample is in, then decide once. If the rewrite loses, that is a result too. AdAlign reports losing variants as well, and a losing headline tells the next brief what not to promise.

Consent, if your visitors are in the EU

A split that stores an identifier in the visitor's browser needs consent unless the storage is strictly necessary for a service the visitor asked for, under Article 5(3) of the ePrivacy Directive (EUR-Lex). AdAlign's privacy policy says its own site tests keep a random visitor ID in local storage only after a visitor accepts cookies (privacy policy). That describes adalign.io, not your domain. Ask during the pilot how the tag assigns your visitors, and make sure your cookie banner covers it.

When a testing tool fits better

Variants tests words. To test a new layout, a different hero image or a multi-page flow, use a testing tool. Convert, for example, starts at $399 a month billed monthly or $299 a month billed annually, with a 15-day free trial (Convert pricing, checked on 27 October 2026). The two answer different questions. AdAlign says which words on the page fail the ad. The testing tool measures any change you can build.

Free first, then the built-in holdout

The Google Ads experiment costs nothing beyond the media you already buy, and it is enough on its own for a Search-heavy page. Score the pair in the free audit, rewrite the slot it names, and run the split.

When you want the rewrite drafted, the holdout set and the lift read for every low-scoring page, AdAlign, which pays me a commission, includes Variants on Scale. It starts after a €1,500 paid 30-day pilot that is credited to the first invoices if you continue (pricing, checked on 27 October 2026). Find the slot to test in the free audit, or see a variant drafted on your own account through the demo. The tool is described in the AdAlign entry. Writing the rewrite itself is covered in how to brief landing page copy from ad creative.

FAQ

Why does AdAlign hold back 50% by default?

An even split gives each arm the same traffic, so the planned sample arrives at the same time in both. You can hold back a different share (how it works), but a smaller control makes the test take longer to read.

What happens if the web team edits the page mid-test?

On Variants, the variant pauses itself when the page changes underneath it (how it works). In a Google Ads experiment nothing pauses, so agree a freeze on the page until the stop date.

Is a holdout worth it for a small change?

If the page cannot reach the sample in a reasonable time, no. Ship the matching copy, re-score the pair, and watch conversion rate. A test you cannot finish is not more rigorous than no test.