Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• 50aa97da45 Feat/flexkv main connector (#29701) linhu-nv 2026-07-07 15:35:52 +08:00
• dabd4cfcfd [AMD] Cap DSV4 Flash max_total_num_tokens (#30313) Bingxu ChenandYC Yen-Ching Tseng 2026-07-07 15:33:05 +08:00
• 99db3b0fa5 ci: run jit-kernel tests on scheduled full runs (#30306) Alison Shao 2026-07-07 00:14:39 -07:00
• 9a6f8e5992 [AMD] Fix DeepSeek V4 MTP accuracy issue (#30333) Wang, FangYuan 2026-07-07 14:57:57 +08:00
• 669fd4b8a5 [PP] Fix start_layer_id with pp in get kv_buffer_shape (#29887) 3 people 2026-07-07 14:54:38 +08:00
• fefc1743a9 Cute-DSL FP8 MQA logits (#25220) 3 people 2026-07-06 23:34:07 -07:00
• 2d9f0b3317 [NPU] [DOC] Update arguments detail to NPU support features page (#30328) ZeyuanChen2000 2026-07-07 14:09:46 +08:00
• 1da7d3a50b [MoE] Retire the AOT moe_fused_gate / kimi_k2_moe_fused_gate gate kernels (#26771) (#29997) Xiaoyu ZhangandClaude Opus 4.8 2026-07-07 13:53:17 +08:00
• 9bd02dc5b9 feat(sgl-kernel): add InfLLM v2 attention kernels (#29383) 7 people 2026-07-07 13:46:54 +08:00
• be70bfbdbb [DSA] Fold page-table into fused top-k v2 (decode): drop page_size=1 expansion (#30274) DarkSharpness 2026-07-07 12:28:05 +08:00
• 36b449af19 [EPD] Optimize multimodal global cache with paged embedding pool (#28441) 3 people 2026-07-07 12:05:52 +08:00
• 541f9221da feat(metrics): add Prometheus metrics for the EPD encoder server (#27564) Zyann 2026-07-07 12:04:51 +08:00
• abafe0022e [Unified Radix Cache] Rename tree variables to cache in unittest (#30281) Jialin Ouyang 2026-07-06 20:48:37 -07:00
• 998acf7df8 [DOCS][NPU]update npu support features (#30324) loading66 2026-07-07 11:37:19 +08:00
• 16372b4c5f [Spec] Anchor GLM-5.2 MTP IndexShare topk on the draft-extend step (#29787) 5 people 2026-07-06 20:36:48 -07:00
• 4145e595cf [MLX] Add correctness tests for qwen2_moe and qwen3_moe (#29440) Siming Deng 2026-07-07 11:32:12 +08:00
• df06e03662 [MLX] Size the attention KV pool at the compute dtype for quantized models (#30097) Siming Dengandsiming-deng 2026-07-07 11:31:59 +08:00
• c3da0a2582 [MLX] Fix single-token chunked-prefill continuation misrouted as decode (#30181) NOOB 2026-07-07 09:01:33 +05:30
• 3a679459e5 [bench] Add agentic-trace multi-turn dataset to bench_serving (#29215) 3 people 2026-07-06 19:45:44 -07:00
• e85ef54877 Support Cutedsl BF16 GEMM JIT kernel (#30117) 3 people 2026-07-06 19:16:16 -07:00
• 267ff1b5f9 Fix LTX2 RoPE JIT kernel CI (#30278) Brayden ZhongandBrayden Zhong 2026-07-06 19:15:21 -07:00
• 6279805962 [DSv4] Loading Time Weight Dequant (#27867) laixinandPeng Zhang 2026-07-07 09:54:34 +08:00
• 3cbb7568bd [Experimental] Full Cuda Graph Support for Prefill (#27988) Yuwei An 2026-07-06 18:13:03 -07:00
• c861896721 [refactor] Resolve config declarations onto server_args at the end of __post_init__ (#30297) Cheng Wan 2026-07-06 18:04:44 -07:00
• cf4edda956 docs: sync LMSYS SGLang blog cards (#30311) sglang-botandsglang-bot 2026-07-06 18:04:21 -07:00
• 4b5c612257 Skip redundant moe_sum_reduce for single-expert routing on XPU (#22660) Rahul VijayaraghavanandMa Mingfei 2026-07-07 06:33:16 +05:30
• 7047afafec [diffusion] fix: slice img_shapes per-sample in rollout response extractor (#29989) Rockdu 2026-07-06 17:56:52 -07:00
• 30fb0dd851 [CPU] add fused_qk_gemma_norm and refactor norm kernel implementation (#30216) Ma Mingfei 2026-07-07 08:52:59 +08:00
• 6c1fb8a937 [diffusion] fix: fix ragged-caption dynamic-batching accuracy bug in ernie-Image (#30241) Mick 2026-07-07 08:41:51 +08:00
• b41552334d Fix disagg speculative decoding with NIXL connector (#30222) nvjullin 2026-07-07 06:51:33 +08:00
• 1c23954cb9 [NPU] Add new diffusion tests (#29331) Elizaveta MartirosianandElizaveta Martirosian 2026-07-07 00:40:26 +03:00
• 093256aa4b [Mamba] Fix long-prefill accuracy drop in radix prefix-cache state restore (#29368) Aditya Kamat 2026-07-06 13:57:44 -07:00
• ca73c77055 [Diffusion] cache cross-attn K/V across denoise steps for Helios (#29755) Thomasandronnie_zheng 2026-07-07 04:56:37 +08:00
• d8462f4961 Fixes for NVFP4 numerical accuracy for router GEMM output and wrong correction bias cast (#29783) Brayden ZhongandBrayden Zhong 2026-07-06 13:53:05 -07:00
• 1579a82d17 [MLX] Fix FakeOverlapScheduler test stub broken by forward_ct accounting (#30125) Lijuan Tang 2026-07-06 13:43:40 -07:00
• 8f40b5eb3f When attention TP for linear and full attention, use Flashinfer allreduce fusion (#29699) Brayden ZhongandBrayden Zhong 2026-07-06 13:01:53 -07:00
• 52c6e27e7e [AMD][diffusion] fix: disable layernorm torch.compile decorator in eager mode on ROCm to avoid memory-access fault (#29673) sushil Dubey 2026-07-06 23:45:57 +05:30
• 3abdbab9bb ✨ [llm][npu][quant] Add W4A8 MXFP quantization support for Qwen3 Dense on Ascend NPU (#23650) Junlin Wuandronnie_zheng 2026-07-07 00:23:26 +08:00
• 1b481deade feat: sync npu nightly test improvements from Ascend testcases (#29403) hhhh1252023 2026-07-06 22:41:15 +08:00
• 80decc78ec [AMD][DeepSeek V4] Set SGLANG_OPT_FLASHMLA_SPARSE_PREFILL to false on hip code path (#30237) Thomas Wang 2026-07-06 16:25:55 +08:00
• b3ab56545b Add Accuracy Benchmark for OCR models (#25364) Chetan Kumar VermaandMa Mingfei 2026-07-06 13:48:43 +05:30
• e2b55bdbab NUMA: probe numactl binding and fall back when --membind is rejected (#28401) chengcuipingandMohammad Miadh Angkad 2026-07-06 16:03:59 +08:00
• 7c9bb316cf docs(cookbook): total (input+output) throughput per GPU + percentile latency labels (#30214) zijiexiaandClaude Opus 4.8 2026-07-06 00:39:48 -07:00
• 5f98f62a8a [diffusion] perf: tp-shard every text/image encoder across the full DiT replica (any parallelism) (#30086) Mick 2026-07-06 14:48:07 +08:00
• de00b838c4 [diffusion] fix: pass progressive params through image API (#30148) Jzz1943 2026-07-06 14:45:45 +08:00
• 6bb2918938 Bugfix qwen prefix cache circumstances (#30047) McZyWu 2026-07-06 14:39:30 +08:00
• 24c42c90be Clean up ServerArgs post-init dispatch (#30186) Lianmin Zheng 2026-07-05 23:05:07 -07:00
• 5eb1b6a7ba Remove retired DSA env paths (#29912) Baizhou Zhang 2026-07-05 22:58:02 -07:00
• 6f22790943 cookbook: add Hunyuan 3 (Hy3) Day-0 page (#30201) Xinyuan Tong 2026-07-06 13:30:47 +08:00
• b1942fc3ea [Model] Support Qwen3.6 ModelOpt mixed NVFP4 (#27906) Mohammad Miadh Angkad 2026-07-06 12:31:15 +08:00
• cc7d7ba3dd [Intel XPU] Bump nightly per-file timeout to 120m and extend XPU CI monitor (#30100) ashwini rathi 2026-07-06 09:56:28 +05:30
• 850719ebd9 [XPU] Unbreak stage-b: re-add --disable-decode-cuda-graph, quarantine EAGLE3 parity (#30048) ashwini rathiandarathi-hlab 2026-07-06 09:55:46 +05:30
• 81735ecf80 [AMD ]Feat/dsv4 ep tbo prefill (#29362) 3 people 2026-07-06 12:18:34 +08:00
• c016c6f355 [JIT Kernel] DeepSeek-V4 DSA indexer: faster top-k + page-table transform (runtime k <= 2048) (#26788) DarkSharpnessandClaude 2026-07-06 11:23:39 +08:00
• c9ceab34cf [AMD] [Docker] Update MoRI to v1.2.1 (#29394) Duyi-Wang 2026-07-06 10:37:04 +08:00
• 9d00385b63 Cleanup: relocate temp_set_env and consolidate multi-device/CUDA helpers in common.py (#30180) Lianmin Zheng 2026-07-05 18:47:26 -07:00
• 8673e85e6c Remove # fmt: off from environ.py Envs class (#30153) Lianmin Zheng 2026-07-05 12:45:24 -07:00
• 92a1f6e06c [refactor] Reorder ServerArgs sections common-first; inline LLAMA4/MIMO_V2 arch tuples (#30151) Lianmin Zheng 2026-07-05 12:17:24 -07:00
• 48ba79c11e [BugFix] Release HiCache prefetch resources on disagg-prefill bootstrap-queue abort (#30053) Kevin Flansburg 2026-07-05 09:06:59 -07:00
• 602c8615a1 [fix] Reconcile the legacy-getter ratchet baseline after racing merges (#30154) Cheng WanandClaude Fable 5 2026-07-05 07:06:07 -07:00
• 931b00f1b0 [diffusion] Clean up duplicate helper definitions (#30159) Xiaoyu Zhang 2026-07-05 22:05:11 +08:00
• 3ea875fef4 [chore] Remove the stack-review placeholder file (#30149) Cheng Wan 2026-07-05 02:39:45 -07:00
• addffd7489 [Diffusion] Diffusion model support log-requests (#23049) 3 people 2026-07-05 17:01:26 +08:00
• b070cb2ae0 Fix Diffusion GT generation pipelines (#29926) 3 people 2026-07-05 11:43:50 +03:00
• 8fb99bbaf8 [refactor] Config resolution pipeline: full-stack review (10-PR series, review only) (#30137) Cheng Wan 2026-07-05 00:00:07 -07:00
• ce733f106b [AMD] Fix RMSNorm batch-invariance on ROCm under deterministic inference (#28787) Yuankai ChenandChun Fang 2026-07-05 14:49:06 +08:00
• 67361ff91b [AMD][DI][CI] 3/N Add Kimi K2.6 FP8 MI355X 1P1D nightly recipes (#29855) Zhaoyi Liandbingxche 2026-07-04 23:05:12 -05:00
• a37bc2456d [diffusion] refactor: consolidate diffusion weight load planning (#30118) Mick 2026-07-05 11:59:50 +08:00
• 5e6f49c986 [diffusion] feat: add LingBot realtime prompt, KV window, and lazy VAE controls (#30040) HuangJi 2026-07-05 11:11:54 +08:00
• fbe3110866 Make mem_fraction_static reserve disaggregation-mode aware (#29615) cctry 2026-07-04 18:44:06 -07:00
• 754524d8de [Fix] Skip cross-node probe in MultimemAllGatherer on single-node runs (fixes mooncake EP segfault) (#30139) Cheng Wan 2026-07-04 15:36:23 -07:00
• 00f088f6be [fix] Wrap the sp_shard test entry point in sys.exit so failures propagate (#30138) Cheng Wan 2026-07-04 14:23:40 -07:00
• 63c4996fef fix: populate batch req rids and per-request http_worker_ipc for mult… (#29882) ybyang 2026-07-05 03:09:41 +08:00
• 854b46be99 feat(parser): resolve special-token suffix at runtime for compatibility (#29920) Xinyuan Tong 2026-07-05 00:13:46 +08:00
• 763c6bf372 [diffusion] perf: add unified SP shard helpers and zero-copy tail-pad attention (#30107) Mick 2026-07-04 23:57:39 +08:00
• b7c3709f33 [diffusion] fix: fix z-Image online fp8 quantization crash with dit_cpu_offload (#29903) Praneth ParuchuriandMick 2026-07-04 21:10:43 +05:30
• 6dd0cefb2a [CI] Revert ModelOpt NVFP4 threshold relax (#29844) Xiaoyu Zhang 2026-07-04 21:17:32 +08:00
• 36fc0093d6 [diffusion] fix: shut down diffusion workers on serve exit (#30110) Mick 2026-07-04 20:42:39 +08:00
• e552f6ed75 [Fix] Fix DSA indexer fusion for NeoX RoPE (#30111) Baizhou Zhang 2026-07-04 03:20:55 -07:00
• 92b800c531 [DSA][GLM5.2] Index Share for MHA (#29959) Kaixi 2026-07-04 11:50:26 +02:00
• b941e337a4 [refactor] Rename Arg.model_overridable to Arg.resolvable (stack 15/15) (#30077) Cheng WanandClaude Fable 5 2026-07-04 02:23:01 -07:00
• 7ea2284551 [refactor] Migrate the DeepSeek family and the parallel-request chains (stack 14/15) (#30076) Cheng WanandClaude Fable 5 2026-07-04 02:22:50 -07:00
• 3836cba9ee [refactor] Migrate the moe_runner_backend / quantization resolution chains (stack 13/15) (#30075) Cheng WanandClaude Fable 5 2026-07-04 02:22:38 -07:00
• 276fbfe880 [refactor] Migrate the page_size resolution chain (stack 12/15) (#30074) Cheng WanandClaude Fable 5 2026-07-04 02:22:28 -07:00
• abbb41a214 [refactor] Migrate the attention_backend resolution chain (stack 11/15) (#30073) Cheng WanandClaude Fable 5 2026-07-04 02:22:17 -07:00
• 5c95bf15c8 [refactor] Add the post-process resolution stage; migrate sampling_backend (stack 10/15) (#30072) Cheng WanandClaude Fable 5 2026-07-04 02:22:05 -07:00
• ae29c5a1dc [refactor] Sweep disable_hybrid_swa_memory writers; close the dtype family (stack 9/15) (#30071) Cheng WanandClaude Fable 5 2026-07-04 02:21:54 -07:00
• 4bf4db09f6 [refactor] Add predicate-keyed registration; migrate the Step3p family (stack 8/15) (#30070) Cheng WanandClaude Fable 5 2026-07-04 02:21:43 -07:00
• 8d8f17e28c [refactor] Migrate the first override families: Mistral/Pixtral dtype, MiniMaxM2, MiMoV2 (stack 7/15) (#30069) Cheng WanandClaude Fable 5 2026-07-04 02:21:32 -07:00
• df6491d80c [refactor] Wire the config resolution pipeline (dispatch, stash, dual-apply, publish) (stack 6/15) (#30068) Cheng WanandClaude Fable 5 2026-07-04 02:21:21 -07:00
• c3d751b231 [refactor] Add the declarative model-override registry and resolution gate (stack 5/15) (#30067) Cheng WanandClaude Fable 5 2026-07-04 02:21:09 -07:00
• 10f258257f [refactor] Add the resolved-flags tier + resolvable-field metadata (stack 4/15) (#30066) Cheng WanandClaude Fable 5 2026-07-04 02:20:58 -07:00
• ff57171f98 [refactor] Soft-deprecate the legacy global ServerArgs accessors + ratchet (stack 3/15) (#30065) Cheng WanandClaude Fable 5 2026-07-04 02:20:46 -07:00
• def20782cf [refactor] Move ServerArgs ownership into the runtime context (stack 2/15) (#30064) Cheng WanandClaude Fable 5 2026-07-04 02:20:34 -07:00
• 6d662c9245 [refactor] Add a read-through server_args accessor to RuntimeContext (stack 1/15) (#30063) Cheng WanandClaude Fable 5 2026-07-04 02:19:46 -07:00
• 03962d4238 [diffusion] feat: enable compile warmup for vae decode (#29306) Mick 2026-07-04 15:25:26 +08:00
• 576dc31e33 Avoid logits multimem all-gather on cross-node TP groups (#29881) san-tian 2026-07-04 15:24:58 +08:00
• 88db9e033a Adjust KL_THRESHOLD for log probability calculations (#30101) Cheng Wan 2026-07-03 23:53:51 -07:00
• ad744c6c6b [trtllm_mha] Fuse cuda-graph metadata rebuild into one triton kernel (#29843) Pranjal Shankhdharandpranjalssh 2026-07-04 10:54:28 +05:30
• 4028304579 [DSA] Disable indexer fusion by default to restore DeepSeek-V3.2 accuracy (#30088) Cheng Wan 2026-07-03 20:50:37 -07:00