Together Compiler lets AI agents retune GPU kernels overnight
Together AI now has agents propose and test GPU kernel changes automatically, claiming up to 3.7x more tokens per GPU on some hardware. It runs by default on three serverless models today.
Together AI launched Together Compiler on 4 November, the first product under what it calls Polymorphic Inference: letting AI agents rewrite the GPU code behind the models it serves, instead of relying on researchers to hand-tune each deployment.
The system works by having swarms of agents propose kernel changes across a model’s forward pass. A persistent engine hot-reloads and tests each candidate in seconds, and only changes that prove both faster and correct reach production. Together AI says the approach delivers 1.3 to 3x higher throughput on Nvidia Blackwell GPUs within 24 hours of a model landing, up to 3.7x more tokens per GPU on AMD’s MI355X, up to 2.7x more tokens per chip on AWS Trainium2, and up to 1.8x faster responses for a single user. It is switched on by default for three serverless models: GLM 5.3, GLM 5.3 Flash and DeepSeek V4.1 Flash. Dedicated deployments can request it separately.
Why it matters: if you serve one of those models through Together AI, you get the throughput gain without changing any code, and it should keep improving as new hardware and models ship.
The caveat: every number above comes from Together AI’s own benchmarks against its own baseline. Treat the multipliers as a vendor’s best case, not an independent result.