The full verifier passes fewer patches
SWE-Serve builds 53 tasks from merged SGLang changes. On the 19 tasks that start a live server, the same patches pass 69.4 percent of the time when the live tests are removed and 45.9 percent under the full verifier. NVIDIA's post, dated 23 September, counts 147 patches that flip from fail to pass when the live tests are dropped.[1]
Twelve tasks run on CPU and 41 use a single H100. A pass means the hidden verifier accepts the patch. The authors write that this stands apart from SGLang's upstream review and stops short of calling the patch ready to merge. The evaluation runs 11 models with mini-swe-agent under closed-book conditions.[1]
The 69.4 percent figure counts the pre-server check
The 69.4 percent and 45.9 percent figures are the same patches under two verifiers. The higher one is the pass rate that appears when the evaluation stops before the server starts. That pushes me toward reading the live server as a separate check, and it may still be only a longer list of the same tests.[1]