Generic serving
Qwen3.6-27B
1× H200
Multi-tenant API serving with traffic patterns from the Alibaba Bailian trace.
Gated DeltaNet in Qwen3.6
Humanity's Last System
Steven Gao1, Rulin Shao1, Keisuke Kamahori1, Yuetai Li1, Yufei Gao1, Atindra Jha2, Junjie Oscar Yin1, Shihang Li1, Radha Poovendran1, Jure Leskovec2, Hannaneh Hajishirzi1, Pang Wei Koh1, Baris Kasikci1, Stephanie Wang1, Luke Zettlemoyer1
1University of Washington2Stanford University
Coding agents get 12 hours and up to eight H200 GPUs to build an LLM inference server from scratch. Servers that pass verification are scored by their speedup over a production serving framework.
Leaderboard
| 1 | Opus 5max | Claude Code | 71.4% | 81.0% | 1.91 | 0.99 | 1.83 | 1.89 | 25.9 | 2.40 | 4.92 |
| 2 | Kimi K3max | Claude Code | 23.8% | 33.3% | 0 | 0 | 0 | 1.01 | 12.4 | 1.24 | 1.04 |
| 3 | Opus 4.8max | Claude Code | 19.0% | 71.4% | 0.45 | 0.029 | 0 | 0.43 | 7.10 | 1.08 | 0.76 |
| 4 | GLM 5.3max | Claude Code | 14.3% | 28.6% | 0 | 0 | 0 | 0.80 | 13.2 | 0 | 1.20 |
| 5 | GPT 5.6 Solmax | Codex | 9.5% | 42.9% | 0 | 0.013 | 0 | 0.54 | 0 | 1.66 | 0.54 |
| 6 | GPT 5.5xhigh | Codex | 0.0% | 28.6% | 0 | 0 | 0 | 0 | 0.25 | 0 | 0.038 |
Task columns show speedup over the baseline: 1 matches it, 0 is a run that failed. Correct and Faster than baseline are shares of each model's 21 runs.