Incrementality testing, explained: how a causal lift is measured when the screen is outdoors

Incrementality testing measures the outcomes a campaign caused by comparing what happened with a counterfactual: what would have happened without it. Brand lift asks what people think and attribution assigns credit; incrementality asks whether the outcome would have occurred anyway. Out-of-home cannot randomise exposure, so the design starts with geography or with exposure inferred from device location.

Every incrementality test compares a number that was observed with a number that was not, and the discipline is in how the second number gets made. Digital channels can randomise people; a screen on a street shows the same thing to everyone who passes. Below: the designs that survive that constraint, the arithmetic that decides whether a test can see anything, and what a brief has to settle before the flight.

What is incrementality testing?

Incrementality testing measures the part of an outcome that a campaign caused, by comparing the observed result with a counterfactual. Meta’s GeoLift documentation puts it in one sentence: a campaign’s incremental effect is “the difference between what we observed as the results of that campaign and what would have happened in a world where it didn’t take place”. The outcome is a behaviour, a store visit, a sale, an install. The counterfactual is a control that did not get the campaign.

That separates it from the two measures it is confused with. A brand lift study measures a change in what people think, through a survey. Attribution assigns credit for an outcome that did happen to the touchpoints that preceded it. The OAAA’s 2020 measurement guide keeps measurement (plays, impressions, reach, frequency) and attribution (analytics that “demonstrate causality or correlation”) as separate disciplines; incrementality is the causal end of the second. Attribution can credit a channel with visits that would have happened regardless. An incrementality test is built so that it cannot.

How is incrementality calculated?

Incremental outcome equals observed outcome minus counterfactual outcome. In a two-group design incremental lift is exposed rate minus control rate, and relative lift divides that gap by the control rate. The OAAA guide defines lift as the “percent difference in visitation rates between exposed audience and unexposed audience” and adds a variant, incremental lift, in which “a known visiting quantity is removed from the calculated set” so only visits above baseline remain.

A published example: Clear Channel Outdoor’s RADARProof study of an Atlanta fast casual campaign (Cuebiq data, August 2021) reports that over 15% of exposed adults visited, 30.6% more than non-exposed adults. Read backwards, the control rate is about 11.5% and the absolute lift about 3.5 points; its second figure, 28% of exposed visits classed as incremental, is the OAAA’s variant. A result that cannot be converted both ways has left out a base rate.

Which test designs work for out-of-home?

Four designs cover the field. They differ in where the counterfactual comes from, and that decides which of them out-of-home can use.

Geo holdout and matched market tests

Vaver and Koehler’s 2011 Google paper describes the geo experiment: “non-overlapping geographic regions are randomly assigned to a control or treatment condition, and each region realizes its assigned condition through the use of geo-targeted advertising”. The matched market test is the same idea without randomisation; the IAB’s July 2025 DOOH Measurement Guide describes dividing markets into test and control on shared traits, prioritised by campaign goal. Neither needs an exposure log. What they need is markets: Measured’s July 2026 guidance is 10 to 15 matched markets with 95% or higher historical correlation before the test begins, most tests running 4 to 8 weeks with a holdout of 10 to 20% of the test region’s budget. A campaign in one city has nothing to hold out.

Synthetic control

With one treated market, or no randomisation at all, the counterfactual is modelled. GeoLift builds it from untreated locations with synthetic control methods (SCMs): “using historical information prior to the treatment SCMs find the combination of untreated units that most closely replicate the treated”. Haus describes the out-of-home version, a New York flight against a synthetic control built from the remaining DMAs, because out-of-home audiences “operate without individual-level tracking, pixels, or PII”. The device-level analogue is Reveal Mobile’s net lift study, which propensity-matches control users so they are “identical to the exposed group, except for the fact that they were not exposed”, reported at 95% confidence. The IAB lists synthetic control beside matched markets as the two primary DOOH methods, with a caveat: “since exposure records may be incomplete, blending synthetic models with geo-based holdouts can help reduce contamination of the control group”.

Ghost ads and PSA controls, and why they barely work outdoors

In a PSA test the ad server randomises individuals and shows the control group a public-service message in the brand ad’s slot. Johnson, Lewis and Nubbemeyer’s ghost ads paper (2016; Journal of Marketing Research, 2017) lists the problems: PSAs “are expensive and prone to errors because they require coordination among advertisers, publishers, and third-party charities”, and performance-optimised delivery breaks them by sending the PSA and the brand ad to different people. Ghost ads instead log, at the platform, the impressions a control user would have received, at “at least an order of magnitude less” cost. Both rest on a system that assigns and records exposure person by person. A screen has none: it cannot show the PSA to half the pavement, and a PSA on a public screen costs the full slot while producing a control group of everyone who passed. Holding out some screens is a geo holdout at the scale of a street corner, and the same person passes both.

Pre/post with a seasonality control

The OAAA guide permits lift “versus the control group or before the ad buy occurred”; the IAB describes comparing sales during the campaign with historical data “while accounting for seasonality or external factors”. That clause is the work. Seasonally adjusted, a pre/post read is a first estimate and a sanity check beside a proper control. On its own, everything else that changed in the period is inside the number.

Why can’t out-of-home randomise exposure?

Because nobody is assigned to a screen. A display faces a street and whoever walks along it is exposed, so an out-of-home study constructs its exposed group after the fact, from device location. The OAAA defines exposure as presence in a screen’s exposure zone, or viewshed, while content is viewable, and this “does not require that the content be viewed”. OUTFRONT’s May 2024 explainer gives the mechanics: measurement partners geofence the viewshed, and a phone passing through is marked present by its mobile advertising identifier. Reveal Mobile computes device-level exposure from media plans, ad rolls, experiential specs and moving waypoints, joined to a panel of 60 million US daily active users. Panel devices logged inside a viewshed are exposed; the control is drawn from the rest. Exposure is inferred, not served, and the inference is only as good as the location signal behind the panel.

The control is defined by absence of evidence: a device is in it because it was not logged in the viewshed, and a panel of 60 million is a sample. Some of the control passed the screen unrecorded, which drags measured lift toward zero. A Geopath impression, the number of people “likely to notice” an ad, is a modelled audience with no roster of devices; it sizes the audience and cannot populate a test cell. How that estimate is built, and why its inputs change for a vehicle, is on how out-of-home impressions are measured.

A moving screen sharpens both problems. A fixed viewshed is one polygon drawn once; a vehicle’s viewshed is a polygon per timestamp, and the exposed set is the union of every phone logged inside it along the route. The exposure record for mobile inventory is a route log joined to a panel, not a site list, which is why Reveal lists moving waypoints as an input. The IAB’s guide says exposure for exterior moving formats “is highly variable depending on vehicle speed, obstructions, and route patterns” and that fixed screens are “more compatible with geofencing”. For a test on vehicle-based inventory the route data is part of the exposure definition. Firefly, the operator behind the Foursquare-measured study in the results table below, lists foot traffic, web conversion and app conversion studies on its measurement page and describes its digital tops as GPS-equipped, which is where the route log comes from. What a moving fence does to the exposed panel is on geofencing advertising.

How large does an incrementality test need to be?

Large enough that the smallest lift worth acting on is bigger than the noise. That lift is the minimum detectable effect (MDE), fixed by the baseline rate, the confidence level and the statistical power. For a two-group visitation study the sample per group is

n = (z_α/2 + z_β)² × [p₀(1 − p₀) + p₁(1 − p₁)] ÷ (p₁ − p₀)²

where p₀ is the control visit rate, p₁ the exposed rate to detect, z_α/2 is 1.96 for a 95% two-sided test and z_β is 0.84 for 80% power. Worked once: a 5.0% control rate and a 10% relative lift, which is 5.5%. The bracket is 0.0475 + 0.0520 = 0.0995; the denominator is 0.005 squared, 0.000025; the z-terms sum to 2.80 and square to 7.85. So 7.85 × 0.0995 ÷ 0.000025: about 31,200 devices per group before a 10% lift on a 5% base is reliably visible.

Small samples can only see large lifts, one reason single-campaign case studies publish large percentages; 90% power pushes the 10% case to about 41,800 per group. Geo tests do not use this closed form, because their unit is a market and the noise is a time series. GeoLift ships power calculators to decide “which are the best test locations, how many you should include, investment, and even how long you should run the test”; Measured says a well-powered geo test targets an MDE of 2 to 5%; Haus’s rule of thumb is enough conversions per cell to detect a 10 to 20% lift at 80% power. Vendors state confidence differently, Clear Channel at 90% on its Atlanta study, Reveal at 95% on every study. A lift reported without one is a number, not a finding.

What do published out-of-home incrementality results look like?

Few, and rarely with base rates. The table lists the named, dated results this desk could verify on the publisher’s own page, with the method each states: datapoints on what the genre reports, not benchmarks to plan against.

All three are matched-device designs on a location panel, none is a geo experiment, and each publishes a relative figure. The Clear Channel row states a confidence level and separates total lift from the incremental share. The Lamar row matters for its lookback, 7 days, which caps how long after exposure a visit can count; the IAB says attribution windows “vary widely by category”, and two results with different windows are not the same measurement.

What should an incrementality test brief ask for?

The design, the exposure definition, the sample and the window, in writing, before the flight. A test designed afterwards has no pre-period and nothing left to hold out.

1. The outcome and its window. Visits, sales, installs or sessions, and the lookback in days inside which an outcome counts.
2. The design, and why. Geo holdout, matched market, synthetic control or pre/post. If markets are held out: how many, chosen how, with what pre-period correlation.
3. The exposure definition. Device observed in a viewshed, or market-level assignment. For vehicle-based inventory, the route log that defines the viewshed and who supplies it.
4. The control construction. The matching variables, and whether a synthetic control is blended with a geographic holdout as the IAB suggests.
5. MDE, power and confidence, and the sample per cell they imply. Ask for the arithmetic.
6. The pre-period and contamination. Weeks of baseline before the flight; how people who commute between control and test markets, or pass both a held-out and a live screen, are treated.
7. Base rates in the report, for every metric, so absolute and relative lift can both be computed; and who runs and pays for the study, on the record.

The OAAA’s 2020 checklist already asks most of this: how baseline and lift are determined, whether a control group is used, the seasonality considerations, how time of day and day of week are factored in, whether there is panel bias or normalisation to census. Six years on, still the right list.

Where to go next

The category these tests measure is laid out in the guide to out-of-home advertising. Programmatic buys report delivery by screen and hour, the exposure log a device-panel study needs; direct buys usually reconstruct one, covered on programmatic DOOH. Why a vehicle’s exposure record is a route rather than a site is on location-based advertising.