[codex] Add flashinfer TRTLLM backend for diffusion NVFP4 (#22717)

This commit is contained in:
Xiaoyu Zhang
2026-04-18 09:06:28 +08:00
committed by GitHub
parent 9c47bbad13
commit 615d6c93b2
6 changed files with 403 additions and 46 deletions
+3
View File
@@ -187,6 +187,9 @@ sglang generate \
over the compatibility `--model-path` flow.
- For local directories, SGLang first looks for `*-mixed.safetensors`, then
falls back to loading from the directory.
- To force the generic diffusion ModelOpt FP4 path onto a specific FlashInfer
backend, set `SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND`. Supported values
include `flashinfer_cudnn`, `flashinfer_cutlass`, and `flashinfer_trtllm`.
- On disk, the quantization config stays `quant_method=modelopt` with
`quant_algo=NVFP4`; the `modelopt-nvfp4` label here is again a documentation
family name rather than a serialized config key.