Skip to content

Vercel Sandbox can now run Scale AI's AgentEnv evaluations

A new Vercel provider in AgentEnv 0.9.1334 runs an agent and its task environment in separate Sandboxes, so you can score agent behaviour without standing up your own infrastructure.

Source: Vercel

Vercel said on 26 October that its Sandbox product now has a provider for AgentEnv, Scale AI’s open framework for testing agents against tasks and scoring the results. The integration ships in AgentEnv 0.9.1334 and later, and it runs the agent and its task environment in two separate Sandboxes rather than one shared process.

You authenticate with an existing Vercel CLI login and run the provided example to kick off an evaluation:

./vercel-eval --scope YOUR_TEAM --project YOUR_PROJECT

That gives you back the task’s score and the agent’s recorded actions, without having to provision your own containers or isolation for either side.

Why it matters: evaluating a coding or task agent properly means letting it act somewhere it can’t contaminate its own scoring environment. Splitting agent and environment into separate Sandboxes is a small but real piece of plumbing that most teams building agent evals end up writing themselves. Here it comes for free if you’re already on Vercel.

Share

Related reading

ML for Web Devs

Tokens, context windows, and why your bill is what it is

The unit of billing for language models is not the request or the word. Once you understand tokens and how a context window fills up, the invoice stops being a surprise.

ML for Web Devs

RAG is just search with extra steps

Retrieval-augmented generation gets written up like an architecture. It is a search query, a string concatenation and one API call, and the search query is the part that decides whether it works.

ML for Web Devs

A prompt is a spec, so write it like one

Prompt engineering has very little to do with magic words. It is the same skill as writing a clear ticket for a capable contractor who will not ask you any questions.

More from The Wire

All briefs