This morning the vLLM blog published the number AMD and its collaborators have been climbing toward since Kimi K3's day-0 support in July: on AgentX, the benchmark that replays real coding-agent sessions, an eight-GPU AMD Instinct MI355X system serves K3 at 7,567 total tokens per second per GPU against the B300's 6,083, a 24.4% lead at the benchmark's 50 tokens/s/user P90 target, its per-user responsiveness constraint.
Under SemiAnalysis's ownership-cost model, at $1.50 versus $2.26 per GPU-hour, that works out to 18.16 million versus 9.69 million tokens per dollar: 87.4% more tokens per dollar, or 46.6% cheaper per token. On paper the B300 is the faster chip: 13.5 PFLOP/s FP4 against 10.1 MXFP4, with identical 288 GB of HBM3E and 8 TB/s of bandwidth on both. Software closed the gap and then some.
Why this benchmark is closer to your bill than most
AgentX does not fire fixed 8K-in/1K-out requests. It replays real coding-agent session traces with request lengths, timing, and prefix-sharing intact, and the workload offers over 96% theoretical prefix reuse. If your traffic is Claude Code, Codex, or any agent loop that resends a growing conversation every turn, this is the nearest public measurement to the shape of your invoice.
What actually moved the number
The receipts are the interesting part. Three of the changes, each with its pull request:
- Cache partitioning. Under TP8 every GPU stored the entire compressed KV history; with eight-way decode context parallelism, a 142,000-token conversation's MLA payload drops from about 1.96 GB to about 245 MB per GPU, in #51705.
- MoE dedup. The routed-expert output projection was running eight times, once per GPU. Output coordinates are now divided among GPUs and assembled by an all-reduce that already existed, via #51253; loading only the rows each rank uses frees another 3.85 GiB per GPU, headroom that goes straight back into holding more reusable cache, in #59591.
- Fusion and deleted copies. Attention-residual mixing fused with RMSNorm lifted output throughput 14.2% in the test reported with #50593; sixty-nine pointless 1,792-byte copies per decode step, one per KDA layer, were simply removed in #58566.
Two asterisks before anyone orders hardware. NVIDIA's submission dates from August 16 and AMD's from September 30, and every gain above came from code merged into a moving codebase, so the B300's number today is likely better than the one shown. The dollars are also SemiAnalysis's modeled TCO with some assumptions unpublished, not a quote. And the result is specific to cache-heavy agent loops: a short, zero-reuse prompt leans on raw compute, where the B300's paper advantage still stands. The direction is solid; the exact multiplier will keep moving.
What reaches your invoice, and when
If you buy tokens instead of GPUs, an AMD benchmark changes nothing this week. We don't own GPUs either: Wallaby Token sells API access to Kimi K3, and the path from a serving-cost result to your invoice runs through the official list price. Industry serving costs fall, list prices follow, and our rate stays a standing 10% under the official list, not a promo window.
What you can bank today is the shape of the workload. AgentX's own test harness reports prefix-cache hit rates around 75–80%, and at that hit rate a 142K-token agent turn, the size this benchmark treats as normal, costs about eleven cents on our rate card: 142,000 input tokens at $0.27/M cached and $2.70/M uncached, plus a few hundred output tokens at $13.50/M. Cache-heavy agent traffic is exactly where the optimization money is going, and per-request itemized billing is how that reaches you.
If you'd rather watch it on your own sessions than on ours: a trial key with $0.50 of credit covers a handful of real agent turns, itemized per request. Want the cross-provider picture first? Our 18-provider Kimi K3 price comparison is kept current.
A note on dates: benchmark figures come from the vLLM blog post of October 16, 2026, on a SemiAnalysis InferenceX snapshot retrieved October 9; our prices were checked October 16 against the official rate card.