Incrementality vs. attribution
Attribution divides credit for a conversion among the touchpoints that preceded it — last click, first touch, some weighted curve. It answers "which of our activities was nearby when this happened?" Incrementality asks the harder, more useful question: "would this have happened anyway?" A branded-search click gets full credit under most attribution models and often near-zero incrementality, because the buyer was already coming. The two numbers can disagree by an order of magnitude, and the incremental one is the one the spend decision should follow.
Revenue incrementality, done properly
For website personalization the incrementality unit that matters is dollars per visitor: revenue in treated sessions versus held-out sessions. Conversion rate alone can mislead — an experience can convert more visitors into smaller orders, lifting CVR while revenue falls — so the revenue verdict has to be measured directly and decomposed into conversion rate times order value.
The statistics need care too. Session revenue is zero-inflated and heavy-tailed: most sessions are worth nothing and rare large orders dominate the mean, which breaks the assumptions behind a t-test. AXO uses a seeded percentile bootstrap — thousands of resamples, deterministic so the same inputs always yield the same confidence interval — and only reports significance when the interval's lower bound clears zero.