Skip to content

Vercel's AI Gateway can now call your own private models

Private Inference lets teams connect fine-tuned or self-hosted endpoints to AI Gateway, so switching between a private and catalog model is just a model ID change.

Source: Vercel

Vercel added Private Inference to AI Gateway on 16 November. It connects your own model endpoints, fine-tuned, self-hosted or prerelease, to the same gateway that already routes catalog models, so you stop running a separate authentication and logging setup just for the models nobody else should see.

Private models stay scoped to your team and never appear in the public catalog. They work through the same Chat Completions, Responses and Messages APIs as everything else, with a Zero Data Retention option available per provider, and you can set a catalog model as an automatic fallback if your endpoint goes down. Setup is a CLI call:

vercel ai-gateway private-inference providers create <name> --base-url <url>
vercel ai-gateway private-inference models create <slug> --provider <name> --api-model <id>

Once registered, a private model shows up in the same logs, trace drains and token or time-to-first-token analytics as catalog traffic, and it’s callable from Claude Code, Codex, Cursor and other agents already wired to the gateway.

Why it matters: if you’ve fine-tuned a model or self-host one for cost or data reasons, you can now point coding agents and app code at it with a model ID instead of hand-rolling a second auth path, and fall back to a hosted model automatically if your box falls over.

The caveat: Vercel doesn’t publish pricing for Private Inference in the announcement, pointing instead to its documentation for billing and supported model types.

Share

Related reading

More from The Wire

All briefs