The Question Attribution Can't Answer
Attribution tells you which ads a buyer touched before purchasing. It can't tell you whether they would have bought anyway. Retargeting ads and branded search ads are classic examples: they often reach people already on their way to buying, so they collect credit they didn't fully earn.
Incrementality testing answers the missing question: how many sales did this activity actually cause?
How It Works
You create two comparable groups, show the marketing to one and withhold it from the other, then measure the difference. The extra sales in the exposed group are the incremental lift.
| Group | People | Purchases | Conversion rate |
|---|---|---|---|
| Shown retargeting ads | 10,000 | 300 | 3.0% |
| Holdout (no ads) | 10,000 | 240 | 2.4% |
The ads caused about 60 extra purchases, or 20% of the 300. If your attribution report credited retargeting with all 300, it was over-crediting the channel fivefold.
Types of Incrementality Test
Audience Holdout
Randomly withhold ads or emails from part of an audience you control. Simple for email and retargeting.
Geo Lift
Pause or reduce ads in some regions and compare with similar regions. Works across channels.
Platform Lift Studies
Ad platforms run randomised tests inside their own systems, where eligible.
Running a Simple First Test
- Pick a suspect channelRetargeting or branded search are good first candidates.
- Choose the holdoutA random share of your audience, or matched regions.
- Agree the metric in advanceTotal sales or new customers, measured the same way for both groups.
- Run a full buying cycleUsually several weeks; avoid big promotions mid-test.
- Compare and actCut or cap spend where lift is small; defend spend where lift is strong.
The Trade-Offs
- It costs money. Withholding marketing means giving up some sales during the test.
- It needs volume. Small businesses may struggle to detect small effects.
- It's a snapshot. Results change as creative, audiences and seasons change.
Attribution and Incrementality Together
We'd use attribution day to day to compare campaigns, ads and keywords, then run occasional incrementality tests to check the channels you're least sure about. Accurate first-party attribution makes both better, because you're testing against a reliable count of sales. See multi-touch attribution and our ClickMagick attribution models guide.
Frequently Asked Questions
What is incrementality testing?
It's an experiment that compares results with and without a marketing activity to measure the sales it actually caused, rather than the sales it was credited with.
How is incrementality different from attribution?
Attribution shares credit among the touchpoints a buyer interacted with. Incrementality asks whether those sales would have happened anyway without the ad.
What is a geo lift test?
You pause or reduce advertising in some regions and keep it running in similar regions, then compare results. The difference estimates the lift caused by the ads.
How long should an incrementality test run?
Long enough to cover your full purchase cycle and gather enough conversions to detect a difference. Several weeks is typical for geo tests.
Do small businesses need incrementality testing?
Not constantly. A simple holdout on one channel you suspect is over-credited, such as retargeting or branded search, can be worth running once or twice a year.