Sam Mausberg

My LLM engine runs on one GH200, in place of vLLM’s model runner. On October 20 it was 2.30x faster than vLLM on code, one request at a time, and 1.18x faster on mixed prompts with 32 at once; both used speculative decoding. On October 13 it was 1.49x faster than mistral.rs, the next fastest of five engines.

vLLM finished first at every prompt length I tested from 32K tokens up, and the 1.18x and 1.49x runs wrote 512 tokens per request, past the stop signal. The code is private, so these are my own measurements.

On a B200, NVIDIA’s own evaluator scored the kernels I submitted to SOL-ExecBench. All 110 workloads ran correctly and beat the baseline.

These numbers cover single kernels, not whole models. By November 4, NVIDIA’s automated code review had cleared ConvNeXtV2 and FP8. On the other five, both #1s included, it errored and gave no verdict.

On GPU MODE’s H100 practice boards, as of Oct 16, 2026: prefix sum 2nd of 27, histogram 3rd of 27.

CAIRN is an experimental systems language for CPUs and NVIDIA GPUs, meant for AI agents to write. The compiler refuses GPU kernels with shared-memory races or unreached barriers before they run.

Claude Code agents wrote most of the compiler, runtime, tests and documentation to my design. In two small evaluations with one model family, CAIRN cost agents 5.5 to 11.6 times as many tokens per task as C++.

Its first GH200 run exposed a miscompile in NVIDIA’s NVVM 7.0.1, which I reduced to eight lines and worked around.

I worked on this experimental LLM compiler and runtime from February to September 2026. It served Llama-3.2-1B with greedy tokens bit-exact to the reference.

The vLLM comparisons were never finished, so its speed is unproven.

KernelIndex matches a GPU workload to compatible kernels and their published benchmarks, through a website, REST API and CLI. Records come straight from their sources and aren’t rerun.