Skip to content

Vercel's AI SDK lets coding agents take image input, across every harness

HarnessAgent now passes images to the vision model behind whichever coding agent you are running, from Claude Code to Cursor to Codex, through one shared message format.

Source: Vercel

Vercel added image input support to HarnessAgent, the part of the AI SDK that wraps coding-agent harnesses, on 27 October. You can now pass an image alongside text in the same message, and it reaches the vision model built into whichever harness you are running.

The feature works across every adapter HarnessAgent supports: Claude Code, Cline, Codex, Cursor, Deep Agents, fx, GitHub Copilot, Grok Build, OpenCode and Pi. The format is the same regardless of which one you pick, a content array with a text part and a file part:

const result = await agent.generate({
  session,
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'Describe the image in one sentence.' },
        {
          type: 'file',
          mediaType: 'image/png',
          data: readFileSync('image.png'),
        },
      ],
    },
  ],
});

Why it matters: if you are orchestrating coding agents through the AI SDK, this means a screenshot, a design mock or an error dialog can go straight into the agent’s context without writing adapter-specific code for each harness. Before this, feeding an agent a UI bug meant describing it in text or dropping out of the shared API entirely.

Share

Related reading

More from The Wire

All briefs