Optimizing Inference Performance for the Open-Weight LLM GLM-5.1
August 07, 2026
Whitepaper
This white paper shows how Fixstars optimized inference for GLM-5.1, a 754B-parameter open-weight MoE model, to run responsively on a single node of eight NVIDIA H200 GPUs.
Because MLA replicates the compressed KV cache across ranks, tensor parallelism left almost no room for concurrent requests; switching to a DP-EP layout ("DP Attention") with NVFP4 quantization, KV-cache-aware routing, and CPU cache offloading raised node KV-cache capacity about 3.5x. Measured on a benchmark built from real coding-agent workloads, these changes cut P90 time-to-first-token from 59 s to 3.8 s (~15x) and lifted stable concurrency from a target of 30 to 50+.
