Humanity's Last System

Steven Gao¹, Rulin Shao¹, Keisuke Kamahori¹, Yuetai Li¹, Yufei Gao¹, Atindra Jha², Junjie Oscar Yin¹, Shihang Li¹, Radha Poovendran¹, Jure Leskovec², Hannaneh Hajishirzi¹, Pang Wei Koh¹, Baris Kasikci¹, Stephanie Wang¹, Luke Zettlemoyer¹

¹University of Washington²Stanford University

Coding agents get 12 hours and up to eight H200 GPUs to build an LLM inference server from scratch. Servers that pass verification are scored by their speedup over a production serving framework.

Leaderboard


1	Opus 5max	Claude Code	71.4%	81.0%	1.91	0.99	1.83	1.89	25.9	2.40	4.92
2	Kimi K3max	Claude Code	23.8%	33.3%	0	0	0	1.01	12.4	1.24	1.04
3	Opus 4.8max	Claude Code	19.0%	71.4%	0.45	0.029	0	0.43	7.10	1.08	0.76
4	GLM 5.3max	Claude Code	14.3%	28.6%	0	0	0	0.80	13.2	0	1.20
5	GPT 5.6 Solmax	Codex	9.5%	42.9%	0	0.013	0	0.54	0	1.66	0.54
6	GPT 5.5xhigh	Codex	0.0%	28.6%	0	0	0	0	0.25	0	0.038

Task columns show speedup over the baseline: 1 matches it, 0 is a run that failed. Correct and Faster than baseline are shares of each model's 21 runs.

Tasks

Generic serving

Qwen3.6-27B

1× H200

Multi-tenant API serving with traffic patterns from the Alibaba Bailian trace.

Gated DeltaNet in Qwen3.6