A language model in the live decision path demos beautifully and bills you three taxes in production — cost, confidence, and a standing eval burden. The fix is AI upstream, a deterministic hot path.
Here's a question worth asking any "AI personalization" vendor before you sign: when a visitor lands, does a language model decide what they see — right then, live, on that pageview?
If the answer is yes, you've just inherited three problems the demo didn't mention. If the answer is no, you should understand why that's the better architecture, not a less impressive one.
Putting a large language model directly in the live decision path is seductive because it demos beautifully. The model "reasons" about the visitor, it's flexible, it feels intelligent. But a language model is, by design, a probabilistic system. Ask it the same question twice and you can get two different answers. That's a feature when you're brainstorming copy. It's a liability when it's the thing choosing what every visitor sees, thousands of times an hour, with money attached.
When the model is the runtime decider, three taxes come with it:
The cost tax. Every decision is an inference call. Every pageview that triggers a choice is a token bill. At low traffic you won't notice. At real traffic, you're paying a model to re-decide the same things over and over, and the meter never stops.
The confidence tax. Probabilistic means non-reproducible. You can't fully predict what it'll do, you can't perfectly replay why it did what it did, and "the model decided" is not a satisfying answer when a hero swap goes sideways and someone asks what happened. Determinism — same inputs, same output, every time — is exactly the property you want in production and exactly the property an LLM-in-the-loop gives up.
The eval tax. This is the one teams underestimate. The moment you have a stochastic component making live decisions, you have to continuously test it — build evals, monitor for drift, catch the hallucination before a customer does, re-test every time the model updates underneath you. That's not a one-time setup. It's a standing operational burden that grows with your surface area. You've effectively hired a QA function for a coin you can't stop flipping.
The fix isn't "no AI." It's putting the AI where it belongs: upstream, offline, with a human nearby — helping author variants, draft copy, configure segments. Then a deterministic system does the actual live deciding.
That's the important move. Use the language model for the creative, generative, slow work where its flexibility is a gift and a person can sanity-check it. Keep it out of the millisecond-by-millisecond runtime path where its unpredictability is a tax. The decision that runs a thousand times a minute should be boring, fast, reproducible, and free of inference cost.
Boring, in production, is a compliment.
AXO's hot path is deterministic. When a visitor lands, the decision about what they see is made by scoring their in-session behavior in the tag itself — no model call per pageview, no token bill per decision, no language model improvising in the live path. The behavior goes in, the same decision comes out every time, in milliseconds.
The AI shows up where it's safe and useful: helping draft and shape the content variants upstream, where a human can review the output before it ever goes near a visitor. By the time the tag is deciding, there's nothing left to hallucinate — the choices are already authored, already reviewed, already deterministic.
So the three things you were promised actually hold:
Lower cost, because you're not paying for inference on every decision.
Higher confidence, because deterministic decisions are reproducible and explainable instead of probabilistic and hand-wavy.
No eval tax, because there's nothing stochastic in the live path to babysit.
If a vendor's architecture puts a model in the runtime loop, that can still be a fine tool — but make them name the cost, the confidence, and the eval burden out loud. The best place for a brilliant, occasionally-hallucinating language model is upstream, helping a human get the content right. Not in the chain that decides, live, what a real person sees.
No. A language model is probabilistic by design: the same input can produce different outputs, which is a liability when the system is choosing what every visitor sees thousands of times an hour. The better architecture uses AI upstream — drafting variants and configuring segments with a human reviewing — and a deterministic system for the live, per-pageview decision.
Three ongoing taxes. A cost tax: every decision is an inference call, so the token bill scales with traffic. A confidence tax: probabilistic decisions are not reproducible, so you cannot fully replay why a visitor saw what they saw. And an eval tax: any stochastic component making live decisions must be continuously tested for drift and hallucination, which is a standing operational burden rather than a one-time setup.
Same inputs, same output, every time. In AXO, the live decision is made by scoring in-session behavior in the tag itself — no model call per pageview. Decisions are reproducible, explainable, and carry no inference cost, while generative AI is used upstream where a human reviews its output before it reaches a visitor.