Vercel's AI Gateway can fall back when a model is not confident
Decision requests can now escalate to a second model when the first answer trips a confidence threshold you set, at the cost of billing both calls.
Vercel added confidence-based fallback to AI Gateway’s Decision API on 6 October. A Decision request asks a model a structured question in one of three shapes — Choice (pick from a set of options), Score (a numeric rating) or Boolean (yes/no) — and the new feature reroutes a low-confidence answer to a second model instead of returning it as-is.
You set the trigger on the fallback model itself, inside providerOptions.gateway.models:
models: [{ model: 'openai/gpt-6-astra', when: { question: 'intent', confidenceBelow: 0.6 } }]Choice and Score questions escalate on a confidence threshold; Boolean questions escalate on a probability range instead, since a yes/no answer does not carry the same kind of confidence score. Leaving out question applies the condition to every decision type, and conditions can combine more than one signal. Because a triggered fallback runs a second model call end to end, Vercel bills both stages.
Why it matters: anyone using Decisions for routing, moderation or classification can now add a safety net for uncertain answers without hand-rolling confidence checks around every call — cheap model first, expensive model only when it is unsure.
The caveat: it is in beta, and a conservative threshold on a high-traffic path will trigger the fallback often enough to roughly double your bill for those requests.