Commit Graph
20367 Commits
Author SHA1 Message Date
Cheng WanandClaude Fable 5 8d8f17e28c [refactor] Migrate the first override families: Mistral/Pixtral dtype, MiniMaxM2, MiMoV2 (stack 7/15) (#30069)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 02:21:32 -07:00
Cheng WanandClaude Fable 5 df6491d80c [refactor] Wire the config resolution pipeline (dispatch, stash, dual-apply, publish) (stack 6/15) (#30068)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 02:21:21 -07:00
Cheng WanandClaude Fable 5 c3d751b231 [refactor] Add the declarative model-override registry and resolution gate (stack 5/15) (#30067)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 02:21:09 -07:00
Cheng WanandClaude Fable 5 10f258257f [refactor] Add the resolved-flags tier + resolvable-field metadata (stack 4/15) (#30066)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 02:20:58 -07:00
Cheng WanandClaude Fable 5 ff57171f98 [refactor] Soft-deprecate the legacy global ServerArgs accessors + ratchet (stack 3/15) (#30065)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 02:20:46 -07:00
Cheng WanandClaude Fable 5 def20782cf [refactor] Move ServerArgs ownership into the runtime context (stack 2/15) (#30064)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 02:20:34 -07:00
Cheng WanandClaude Fable 5 6d662c9245 [refactor] Add a read-through server_args accessor to RuntimeContext (stack 1/15) (#30063)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 02:19:46 -07:00
Mick 03962d4238 [diffusion] feat: enable compile warmup for vae decode (#29306) 2026-07-04 15:25:26 +08:00
san-tian 576dc31e33 Avoid logits multimem all-gather on cross-node TP groups (#29881) 2026-07-04 00:24:58 -07:00
Cheng Wan 88db9e033a Adjust KL_THRESHOLD for log probability calculations (#30101) 2026-07-03 23:53:51 -07:00
Pranjal Shankhdharandpranjalssh ad744c6c6b [trtllm_mha] Fuse cuda-graph metadata rebuild into one triton kernel (#29843)
Co-authored-by: pranjalssh <pranjalssh@fb.com>
2026-07-03 22:24:28 -07:00
Cheng Wan 4028304579 [DSA] Disable indexer fusion by default to restore DeepSeek-V3.2 accuracy (#30088) 2026-07-03 20:50:37 -07:00
Mick 5af1f949ca [diffusion] CI: prefer official diffusion consistency GT (#29831) 2026-07-04 10:09:33 +08:00
Cheng Wan 5f623ad24e Revert "Fix wrong RMSNorm fallback to old Flashinfer CUDA kernel when in PCG" (#30083) 2026-07-03 19:05:20 -07:00
Cheng Wan c21f6f19cf [MoE] Fix moe_fused_gate out-of-range expert id on all-NaN rows (fixes eagle_dp_attention crash) (#30079) 2026-07-03 18:48:51 -07:00
Alison Shao b28bc1060f fix(mimo-vl): pass padded_context_dim to Qwen2_5_VisionPatchMerger (#29994) 2026-07-04 08:08:25 +08:00
Khoa PhamandKhoa Pham 6ce02b95ad fix: reorder DSA indexer dual-stream ops to avoid CUDA graph stream explosion (#30025)
Co-authored-by: Khoa Pham <264503018+kpham-sgl@users.noreply.github.com>
2026-07-03 13:53:16 -07:00
Baizhou Zhang 1f0f353d92 [CI] Add GLM52 NVFP4 MTP B200 tests (#30021) 2026-07-03 13:43:38 -07:00
0203c60fdf [CP] Consolidate decode-context-parallel (DCP) helpers into layers/dcp/ (#29365)
Co-authored-by: Hao Phan <htphan@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 12:25:39 -07:00
R0CKSTAR 7820dc60a7 [Apple Silicon] Add labeler config (#29908)
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
2026-07-03 09:43:00 -07:00
Mick 486bcb48e7 [diffusion] CI: fix AMD diffusion CI import (#30039) 2026-07-03 22:26:08 +08:00
Jzz1943 1058d00fe0 [diffusion] feat: support cache-dit for Ideogram 4 (#29631) 2026-07-03 19:58:48 +08:00
Mick 42acfd1550 [diffusion] feat: performance_mode=speed enables torch.compile by default (#30016) 2026-07-03 19:12:17 +08:00
Zhaoyi Li 0ab095eb3f [AMD]: hot-patch transformers dynamic_module_utils symlink bug (#29986) 2026-07-03 18:20:36 +08:00
khalilzhk 430418e218 [NPU] bugfix for dsv4 memory pool (#30001) 2026-07-03 16:41:39 +08:00
Baizhou Zhang e3258d3b54 [Fix] Turn off dsa indexer fusion by default (#30018) 2026-07-03 01:08:06 -07:00
Polisetty V R K Jyothendra VarmaandMa Mingfei 4dddb04325 [Intel GPU] DeepSeek V4 6/N: use sgl-kernel implemetation of flash_mla_with_kvcache on XPU (#27914)
Signed-off-by: P V R K Jyothendra Varma <polisetty.v.r.k.jyothendra.varma@intel.com>
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-07-03 15:59:38 +08:00
e90fec4868 [Intel GPU] DeepSeek V4 10/N : Add sqrtsoftplus support to fused_topk_torch_native (#28048)
Signed-off-by: Rahul Vijayaraghavan <rahul.vijayaraghavan@intel.com>
Signed-off-by: P V R K Jyothendra Varma <polisetty.v.r.k.jyothendra.varma@intel.com>
Co-authored-by: Rahul Vijayaraghavan <rahul.vijayaraghavan@intel.com>
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-07-03 15:59:12 +08:00
Cao E 9df16b5ba9 [XPU] Remove redundant xpu graph backend and make xpu graph opt-in by default (#29911) 2026-07-03 15:58:56 +08:00
YC Yen-Ching Tseng 67697fb891 [AMD] Temporarily disabled: every-6-hours rocm 7.2 test (#30014) 2026-07-03 15:38:07 +08:00
amote-i 2fe7182e75 [DOC] [NPU] update supported features on ascend npu (#30011) 2026-07-03 15:36:41 +08:00
McZyWu 8416544ab0 [NPU] bugfix for Base class add mamba_track_indices parameter (#29999) 2026-07-03 15:10:41 +08:00
Chandrakant Khandelwal fe60764f54 [diffusion] fix: add profiling support and fix VBench dataset handling in bench_offline_throughput (#27704) 2026-07-03 15:08:38 +08:00
Cheng Wan 76f7f7c006 [Spec] Remove the ServerArgs clone + global save/restore hack from DFlashWorkerV2 (#29995) 2026-07-02 23:37:04 -07:00
xutizhou d364cd8ead Support DSV4 shared expert fusion for DeepEP and MegaMOE (#27349) 2026-07-02 23:18:25 -07:00
YAMY e81f05cf4f [dsv4] Trigger MHC prenorm prewarm at weight-load time with rank sync (#29988) 2026-07-02 23:05:03 -07:00
YAMY a6ee64d237 [DeepSeek-V4] Add an opt-in non-paged indexer for long-context prefill (#29619) 2026-07-02 22:54:40 -07:00
san-tianandSan Tian 70b6c06793 fix: avoid DSA indexer CPU seq lens fallback (#29798)
Co-authored-by: San Tian <san-tian@users.noreply.github.com>
2026-07-02 22:50:24 -07:00
Yiqi YangandMick e878c6ebdd [diffusion] feat: shard qwen-image dit across tp ranks (#29774)
Co-authored-by: Mick <mickjagger19@icloud.com>
2026-07-03 13:17:53 +08:00
Yiqi Yang f011d8c2e5 [Anthropic] Fix missing cache_read_input_tokens in streaming responses (#29703) 2026-07-03 12:46:45 +08:00
Kangyan-ZhouandClaude Opus 4.8 860244d4b4 [CI] Add per-stage NVIDIA model inventory tool (#29447)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 20:38:04 -07:00
Ke Bao 372a893744 Move deferred mamba cow and clear (#29945) 2026-07-03 11:29:05 +08:00
Xiaoyu ZhangandClaude Opus 4.8 a2d7eb303e [MoE] Consolidate ungrouped + grouped gate/topk onto one Triton router (#26771) — faster than AOT on B200/H100/H200, at parity with flashinfer (#29771)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 11:18:59 +08:00
ming_wang d8a4f7a7aa add mimo-v2-flash model tutorial (#29932) 2026-07-03 11:02:24 +08:00
Michael bee0f34e68 [AMD] Accept ROCm tensors in JIT kernel TensorMatcher + register 4 kernel tests (#29822) 2026-07-02 19:29:47 -07:00
Peng Xingchen a6bc7fef90 glm5.2 on ascend doc (new version) (#29828) 2026-07-03 10:19:44 +08:00
amote-i 70a813493f [NPU] [DOC] add missing DEEP_NORMAL_MODE_USE_INT8_QUANT for w8a8+deepep scenarios (#29937) 2026-07-03 10:18:22 +08:00
YC Yen-Ching Tseng 05bc3f2aa7 [AMD] Gate broken CK block-FP8 GEMM shapes to aiter-triton-GEMM to fix ROCm 7.0 Qwen3.5 accuracy (#29918) 2026-07-02 18:18:37 -07:00
sglang-bot 91d7645aca docs: sync LMSYS SGLang blog cards (#29990) 2026-07-03 01:01:32 +00:00
ashwini rathiandClaude Opus 4.7 75cdaf432a Add XPU CI job monitor workflow (#29807)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-07-03 08:30:17 +08:00