Three questions that separate real runtime decisioning from a rules engine wearing an AI badge — and why a holdout is the only honest proof of lift.
Your account rep is a lovely person. They are also quota-carrying, and the thing they are selling you this quarter has the word "AI" or "agent" bolted onto a feature that, six months ago, had a different name and did roughly the same thing.
This isn't a conspiracy. It's just incentives. But it means the burden of telling real capability from re-labeled roadmap falls on you. Here's how to carry it without becoming the cynic who says no to everything.
1. "Show me where the decision is made — and when."
A lot of "AI personalization" turns out to be a rules engine someone renamed. That's not necessarily bad, but you should know. Ask whether the system is deciding per-visitor at runtime, or executing rules a human wrote ahead of time. Both are legitimate. Only one is what the slide implied. If the rep gets vague here, you've learned something.
2. "What does the agent actually do without a human in the loop?"
"Agentic" has become the most abused word in the category — this is the "agent-washing" you've probably started to notice. A genuine agent takes actions and makes choices autonomously. A lot of what's marketed as agentic is a workflow with a generate-button that a person still has to approve at every step. Again: the workflow might be useful! But "agentic" should mean the thing acts. Ask what happens with nobody watching. The answer separates the noun from the adjective.
3. "How will I know it worked — and will you let me measure it that way?"
This is the one that ends the gaslighting. Any vendor can show you a number that went up. The number going up doesn't mean they caused it — your traffic, your season, your other campaigns all move the number too. The only honest way to know is a holdout: a randomly withheld slice of visitors who don't get the personalization, run at the same time, so you can compare. The gap between the treated group and the held-out group is the actual lift. Everything else is a story.
Watch what happens when you ask for a holdout. A confident tool says "yes, here's how we set one up." A nervous one explains why holdouts are unnecessary, or hard, or already baked in invisibly. The discomfort is the signal.
Because the alternative is paying for a thing that correlates with good outcomes and taking credit on its behalf, until a CFO asks you to prove it and you can't. A holdout isn't just due diligence on the vendor — it's the receipt that protects you when someone above you wants to know what the spend bought.
AXO is built around the holdout, not embarrassed by it. Lift is measured against a randomly held-out control group by default, so the question "did this actually do anything?" has a real answer instead of a dashboard vibe. And the decisioning is genuinely runtime and behavioral — not a rules engine wearing an AI badge — which you're welcome to verify by asking exactly the questions above.
We'd rather you interrogate us than take the deck on faith. The whole category gets healthier when buyers ask "prove it" — and the tools that flinch tell you everything.
So: be kind to your rep. Then ask the three questions anyway.
Ask three questions. First: where is the decision made, and when — is the system deciding per-visitor at runtime, or executing rules a human wrote ahead of time? Second: what does the agent do without a human in the loop — a genuine agent acts autonomously, while much of what is marketed as agentic is a workflow a person must approve at every step. Third: how will you measure that it worked — a confident vendor will help you set up a holdout; a nervous one will explain why holdouts are unnecessary.
A holdout is a randomly withheld slice of visitors who do not receive the personalization, run at the same time as the treated group. The difference in outcomes between the two groups is the actual lift the personalization caused. Without a holdout, a metric that went up could be seasonality, traffic mix, or another campaign — a vendor dashboard showing a number going up is not proof the vendor caused it.
Agent washing is relabeling existing product features as "agents" or "agentic" without adding autonomous capability. The test: ask what the agent actually does with nobody watching. If every step still requires a human to click approve, it is a workflow with a generate button, not an agent.