SGLang PR #36649 ● merged cuda · gb10

New chip, new model, and a server that won't start.

Bring up a DGX Spark, pull one of the new sparse-attention Qwen models, hit go: the server dies on startup, before it serves a single token. The fast decode kernel already ran on this chip. A stale hardware check just wasn't letting it through.

sgl-project/sglang · qwen_sparse_attn_backend.py · +2 / −2

How we hit it

The DGX Spark is NVIDIA's new GB10 desktop box for running models locally. Point it at one of the new sparse-attention Qwen models (Qwen3.8-Flash-Next), start SGLang, and it aborts during startup. Not a slow model. No model at all: the server never reaches ready.

The fast sparse-decode kernel (trtllm-gen) was gated behind a check for one GPU family. GB10 (the Grace-Blackwell chip in the DGX Spark) is a different compute capability, sm_121, so it failed the check and fell through to a slow fallback path.

On sm_121 that fallback did not just run slow. It crashed the server on startup, during CUDA graph capture. Since it was the only decode path this backend had, the model would not serve at all. The fix admits sm_121 to the gate. One line.

crash

before: server never reaches ready on DGX Spark

ready

after: server starts and returns coherent output

~35%

faster gather + attention on the fast kernel vs the fallback

The setup

Two decode kernels, one allowed to run

This is the QSA backend: Qwen Sparse Attention. During decode, the model generates one token at a time, so each step touches only a few rows of work. That shape wants a kernel built for it.

The backend has two options for decode:

trtllm-gen paged decode is the fast path. It runs the gather-plus-attention pair over a page-aligned scratch buffer, and the resolver's own comment measures it about 35% faster than the alternative.
FA4 cute varlen is the fallback. As that same comment admits, it "runs a prefill-shaped kernel at decode row counts", the wrong shape for one-token-at-a-time work.

Which one you get is decided by a single gate: a hardware-capability check. And that check only said yes to one GPU family.

The mechanism

GB10 falls through, then aborts

The gate was is_sm100_supported(). It passes on sm_100 class hardware and returns the fast kernel. GB10 reports as sm_121, a newer compute capability, so it failed the check and got the fallback instead.

The dispatch, before and after. The kernel was never the problem: trtllm-gen imports and runs fine on sm_121. The gate just never offered it, so GB10 took the fallback and hit a crash. Widening the gate reroutes sm_121 onto the path that already worked. Tap to zoom.

The fallback's failure on sm_121 was not a slowdown. It aborted during CUDA graph capture, the step where the server records the kernel sequence before serving, with a layout error deep in FlashAttention's cute epilogue:

traceback · CUDA graph capture

File "flash_attn/cute/flash_fwd.py", line 393, in epilogue
  MLIRError: Operation creation failed:
  error: expects `coord` and shape of view are weakly
  congruent, but got '!cute.layout<"(?,?):(?{i64 div=8},1)">'

Because _forward_paged_attention is the only decode path QSA has, that crash was terminal: no decode meant no serving. The model (Qwen3.8-Flash-Next) was simply unavailable on a DGX Spark.

The fix

Admit sm_121 to the gate

The repo already ships an is_sm121() helper, added when GB10 support first landed. The fast kernel already imports on sm_121. So the fix is to widen the one gate that was excluding it.

qwen_sparse_attn_backend.py_resolve_trtllm_sparse_decode

- from sglang.srt.utils import is_sm100_supported
+ from sglang.srt.utils import is_sm100_supported, is_sm121

- if not is_sm100_supported():
+ if not (is_sm100_supported() or is_sm121()):
      return None

No new kernel, no new API. It selects a path that already existed and already worked; it just was not reachable from GB10.
Zero blast radius. sm_90 and sm_100 take the exact same branch as before. Only sm_121 changes, from the crashing fallback to the fast kernel.

Why it matters

Local Grace-Blackwell inference, off the slow path

The DGX Spark is a small Grace-Blackwell box built for local model work. A gate that predates sm_121 quietly locked it out of the fast sparse-decode kernel and, worse, onto a path that crashed on capture. One helper the repo already had, dropped into one condition, turns "will not start" into "starts and runs fast".

The pattern. New silicon reports a new compute capability, and every capability gate written before it now excludes it by default. The kernel worked; the allowlist was just stale. Widening a gate is often the whole fix, once you have proven the excluded path is actually safe.

Measured on 2× DGX Spark (GB10, sm_121) serving Qwen3.8-Flash-Next-NVFP4 at tensor-parallel 2 across 2 nodes: ~1318 tok/s prefill, ~24.8 tok/s decode. Before the change, no numbers exist, because the server did not start.