Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• bb48405c31 Unify NVTX annotation helpers and split the enable gate per subsystem (#28165) Liangsheng Yin 2026-06-14 00:04:59 -07:00
• 50993554d8 fix(health): make health-check rid unique across tokenizer workers (#28143) ybyang 2026-06-14 15:01:52 +08:00
• f79a6b5c33 Support GLM-4.7 function calling via structural tags (#28149) 3 people 2026-06-14 14:57:07 +08:00
• 37ed10bd24 [diffusion] UX: reduce attention backend log noise (#28169) Mick 2026-06-14 14:55:13 +08:00
• cc72e2bd8c [Docs] Fix outdated benchmark marker API in add-jit-kernel skill (#27773) DarkSharpnessandClaude 2026-06-14 14:47:30 +08:00
• b250bea994 fix(sampling): reject non-finite temperature in SamplingParams.verify (#28153) Ting SUN 2026-06-14 14:41:36 +08:00
• 171037c3e7 Fix Qwen3.5 deterministic batch-invariant logprobs (#27869) Yuzhen Zhou 2026-06-13 23:23:06 -07:00
• 1747b88c5e [LoRA] Support DSA indexer LoRA targets for GLM-5.1 / DeepSeek-V3.2-family models (#28110) Yanbin Jiang 2026-06-13 23:02:33 -07:00
• 31ac743484 [diffusion] chore: improve server warmup coverage (#28127) Mick 2026-06-14 13:35:42 +08:00
• 5fb4e2d02e [UnifiedTree] Use Qwen3-32B in unified radix pp kl tests and set KL threshold to 0.005 (#27931) Chao Shi 2026-06-14 13:15:35 +08:00
• 91c63aeb4d Fix stale CUDA graph benchmark and docs refs (#28041) Mohammad Miadh Angkad 2026-06-14 12:51:42 +08:00
• 5da3b37a9d [CI] add Precision Regression Test on Nightly Run CI (#26902) Jared Wen 2026-06-14 12:43:57 +08:00
• a3fd5c24be feat: add NVTX markers for the scheduler main loop (#27901) JoyFuture 2026-06-14 08:16:53 +08:00
• 93b402580c feat: add decode clear steps env var (#28160) Chang Min Bark 2026-06-13 20:15:42 -04:00
• 3f4a338212 [AMD][Quantization] Online MXFP4 quantization 2/N - FP8 to MXFP4 requantization on AMD GPUs (#18182) 3 people 2026-06-14 01:08:19 +02:00
• 10d3337048 [AMD] Support unified_kv_triton for disaggregation (#27935) billishyahao 2026-06-14 05:08:27 +08:00
• 8a9f2aa116 fix: prevent stale bitmask leakage in LLGuidance grammar backend (#23653) Oleg ZhelezniakandYorkSu 2026-06-13 23:27:08 +03:00
• 47fabb52ed docs(minimax-m3): add high-concurrency throughput tip for H200 bf16 (#28150) Xinyuan Tong 2026-06-13 21:09:59 +01:00
• 27ba13358e [Spec] Clear dead DRAFT_EXTEND objects left after EAGLE v1 removal (#28133) Cheng Wan 2026-06-13 13:04:51 -07:00
• bde6bccf39 [Spec] Remove deprecated EAGLE v1 DRAFT_EXTEND forward mode (#28129) Cheng Wan 2026-06-13 13:00:21 -07:00
• d988d5d681 feat(cookbook): generic config-declared flagSelects playground axis (#28128) zijiexiaandClaude Opus 4.8 2026-06-13 11:46:34 -07:00
• 29128f31fd [NPU] Best Practice Docs Splitting (#27663) jianzhao-xu 2026-06-13 22:16:20 +08:00
• 0e592395c7 [Apple Silicon] [MLX] Fuse SwiGLU activation into gate gather_qmv for SwitchGLU MoE blocks (#26188) 4 people 2026-06-13 09:57:57 -04:00
• aea0e30853 [4/N] Qwen3.5Opt: Overlap mamba verify update with draft extend (#26924) Yuhao Yang 2026-06-13 20:29:20 +08:00
• f7041c9dee step3.5 flash revise for graph mode and use triton activation (#27739) McZyWu 2026-06-13 16:34:20 +08:00
• 806365e778 feat: Support HiCache for MiMo-V2 models (1/N) (#27378) 3 people 2026-06-13 15:40:27 +08:00
• 60d4bd4c70 [AMD] fix moriep quant kernel not implemented issue (#27855) billishyahao 2026-06-13 15:35:41 +08:00
• 568aa5fcdb Fix missing draft KV pool transfers in HybridCacheController (#27953) Yongji Wu 2026-06-13 00:17:31 -07:00
• 8ce05e8a20 [UnifiedTree]: Pin host buffers across async H→D in UnifiedRadixCache.load_back (#27444) Zhangheng 2026-06-13 15:16:56 +08:00
• eb9483b5c2 [PD][AMD]: incremental KV transfer with decode radix cache (#26288) inkcherryandDuyi-Wang 2026-06-13 15:06:44 +08:00
• f4029d0fc0 [HiCache] fix: clear storage reset state (#26009) shuwenn 2026-06-13 14:51:26 +08:00
• cb4933b22e [diffusion] optimize: enable vae parallel decode with cfg-parallel (#27875) Mick 2026-06-13 13:52:27 +08:00
• bcd45d3ac7 [AMD][DFlash] Add DFlash test to AMD CI (#28112) Jonah Bernardandjonahbernard 2026-06-12 22:06:41 -07:00
• 8becb37519 [diffusion] warmup: improve diffusion server warmup (#28119) Mick 2026-06-13 13:04:10 +08:00
• eb18416f9f [jit-kernel] Support per token group quant 8bit v2 jit kernel (#27449) Yuan Luoandluoyuan.luo 2026-06-13 12:15:09 +08:00
• 053e153183 [AMD] ci: register 8 attention-backend unit tests to run on AMD CI (#27817) Michael 2026-06-12 20:52:45 -07:00
• 1800d7caa6 Bump ray minimum version to 2.55.1 (#27724) Xinyu Zhang 2026-06-12 20:49:11 -07:00
• f288283c07 [AMD] move shared expert check function to quark (#27057) mqhc2020 2026-06-13 11:45:59 +08:00
• d8f8e89ffb [Test] Add unit tests for srt/mem_cache/utils.py (#27913) EvanderFBoopandgemini-code-assist[bot] 2026-06-13 11:38:57 +08:00
• 32ef040618 [Spec] Move eagle verify prepare_for_verify/sample to eagle_utils free helpers (#28117) Liangsheng Yin 2026-06-12 20:37:33 -07:00
• e02f7ca482 [perf] remove several h2d sync (#28076) Qiaolin Yu 2026-06-12 20:30:17 -07:00
• b001d3e815 Add prajjwal1 to CI_PERMISSIONS.json (#28120) Lianmin Zheng 2026-06-12 20:26:38 -07:00
• a0c6e0b3a4 chore: bump tokenspeed_mla 0.1.1 -> 0.1.6 (#28116) Khoa Pham 2026-06-12 19:57:36 -07:00
• ade30fd494 fix(server): serialize nested dict config values as JSON (#28094) Ting SUN 2026-06-13 10:27:31 +08:00
• e9c3b262e4 [Bugfix][Spec] Fix multi-layer EAGLE DRAFT_EXTEND_V2 attn-TP logprob metadata capture (#28026) Ting SUN 2026-06-13 10:26:35 +08:00
• d1a39b0c74 [DeepSeek V3] Defer moe finalize and fused it with main stream add (#27720) Khoa PhamandQiaolin Yu 2026-06-12 19:00:51 -07:00
• a14d1a5656 Add DeepSeek V4 MTP acceptance length checks (#28098) Baizhou Zhang 2026-06-12 18:53:20 -07:00
• 45f8d48994 docs: add llm-d page under Advanced Features (#28078) IBRAHIM IBRAHIMandClaude Opus 4.8 2026-06-12 21:40:31 -04:00
• 1a19f66acb Fix DP attention + EP mode of Nemotron (#28102) Brayden ZhongandBrayden Zhong 2026-06-12 18:39:04 -07:00
• 6c3e429ba1 [Tiny] Cuda Graph Refactor Code Style Follow up (#28107) Yuwei AnandClaude Fable 5 2026-06-12 18:38:13 -07:00
• 29ac249be1 docs: add cookbook-migrate-model skill from the Qwen3.5 pilot (#27845) zijiexiaandClaude Fable 5 2026-06-12 18:31:52 -07:00
• 5633ca8599 [Spec] Move prepare_for_draft to EagleDraftWorkerBase (#28105) Liangsheng Yin 2026-06-12 17:42:43 -07:00
• 5f315c74da [CI] Enforce modern stage=/runner_config= form for dispatchable test suites (#28108) Liangsheng Yin 2026-06-12 17:32:54 -07:00
• 335a9c7837 fix(frontend): return HTTP 400 for out-of-vocabulary token_ids_logprob (#28088) Ting SUN 2026-06-13 08:18:26 +08:00
• 82eedd5bd0 [DeepEP] Enable fabric handles automatically when supported (#27107) Khoa PhamandMohammad Miadh Angkad 2026-06-12 17:17:44 -07:00
• c26669e37b [NPU] [DOC] Update server arguments to NPU support features page (#28083) amote-i 2026-06-13 08:16:52 +08:00
• fda7955890 [AMD][DFlash] Enable Fused KV Materialization (#27854) Jonah Bernardandjonahbernard 2026-06-12 16:57:46 -07:00
• 3a1417a0c1 [refactor] Fold FrozenKVMTPCudaGraphRunner onto the shared DecodeCudaGraphRunner base (#28081) Cheng Wan 2026-06-12 16:55:53 -07:00
• 87554c7855 [Spec] Move draft-extend prep to EagleDraftWorkerBase; unify prepare_for_* names (#28093) Liangsheng Yin 2026-06-12 16:44:04 -07:00
• f23f48df98 fix(moe): make FlashInfer A2A robust to collapsed global_num_tokens (moe_dense_tp_size NaN) (#27945) YAMY 2026-06-12 16:11:44 -07:00
• d601edab73 [Spec] Fix EagleDraftWorker draft-extend attn backend assignment (#28096) Cheng WanandClaude Opus 4.8 2026-06-12 15:47:05 -07:00
• fd977adbd6 Fix PR-close cancellation skipping workflows beyond the first 30 (#27785) Liangsheng Yin 2026-06-12 15:39:33 -07:00
• b0b8436f1c [Fix] Unquote ResponseTool annotation breaking lint on all PRs (#28095) Kangyan-ZhouandClaude Fable 5 2026-06-12 15:20:36 -07:00
• 54989b1fd0 [AMD] ci: add label-gated extra-a tier (kv_canary + mock_model unit tests) (#27822) Michael 2026-06-12 15:12:56 -07:00
• 95867f0932 [Doc] Fix some inconsistencies in the Nemotron Cookbook (#28087) Brayden ZhongandBrayden Zhong 2026-06-12 14:51:58 -07:00
• 85712fa5b0 Fix Responses API request handling (#25881) 9 people 2026-06-12 22:47:55 +01:00
• b3270264e4 Fix Anthropic Messages API compatibility (#25876) +313 people 2026-06-12 22:46:57 +01:00
• caf59759ea [Spec] Centralize dummy verify-input capture; add carries_draft_hidden_states (#28032) Liangsheng Yin 2026-06-12 14:23:29 -07:00
• 65d76bd3f6 [AMD] Fix CI base-a fwd_occupancy: disable SGLANG_SANITIZE_NAN_LOGITS in AMD CI (#28075) Michael 2026-06-12 14:14:34 -07:00
• 1e71c1a859 fix(pd): disable overlap for spec+grammar in disagg decode loop (#28039) ybyang 2026-06-13 05:08:36 +08:00
• cb9140ee61 Enable PDL for GPT-OSS tinygemm router (#27941) Mohammad Miadh Angkad 2026-06-13 04:51:50 +08:00
• 9d37e710b7 [Bench] Add consistent p90/p95/p99 percentiles for all latency metrics (#27662) shuwenn 2026-06-13 04:44:38 +08:00
• 627ed3476b Fix invalid KVFP4QuantizeUtil references (#28013) Zach Zhu 2026-06-13 04:44:11 +08:00
• 3be5a7ec89 Respect explicit --max-running-requests instead of clamping to heuristic (#27399) Vedant V JhaveriandVedant Jhaveri 2026-06-12 12:30:47 -07:00
• 6e0fa5afe1 Support Nemotron DP attention and MTP (#24955) 3 people 2026-06-12 12:21:12 -07:00
• bb33594c1a flashinfer swa kv pool fix (dflash gemma 4) (#27737) David Wang 2026-06-12 14:35:05 -04:00
• fa4273d2db [Docs] Add Kimi K2.7 Code cookbook (#28064) Mohammad Miadh Angkadandzijiexia 2026-06-13 02:19:44 +08:00
• 75998d0421 Fix --mem-fraction-static not accounting for EAGLE draft model KV cache (#23862) cctry 2026-06-12 10:35:48 -07:00
• 533b59d00c Add @alexnails as codeowner for srt/platforms (#28066) Alex NailsandClaude Opus 4.8 2026-06-13 02:28:58 +09:00
• c80d8fe78a [Perf] Skip per-call mat_a/scales_a padding in cutlass FP8 blockwise GEMM (#27896) Yuan Luoandluoyuan.luo 2026-06-13 00:01:53 +08:00
• 9f6b2339f9 docs(minimax-m3): warm-steady-state benchmark numbers (#28062) Xinyuan Tong 2026-06-12 16:42:23 +01:00
• 50815d54a7 docs (#28061) Yuhao YangandXinyuan Tong 2026-06-12 21:19:40 +08:00
• dba617f2ec doc: update docs for new model (#28060) Xinyuan Tong 2026-06-12 14:14:35 +01:00
• fcae6767d5 Add bucketed multi-dir layout for NIXL file storage (#27672) qiaozpandishandhanani 2026-06-12 20:34:55 +08:00
• 18989f3d48 [PD] Fix resource leak on prealloc/transfer abort and idle check (#28022) luoroger37 2026-06-12 18:20:24 +08:00
• f308abc052 Revise the mimo-v2-flash best practice (#28016) ming_wang 2026-06-12 17:12:33 +08:00
• 83c0007f8d examples: add Chain-of-Verification (CoVe) hallucination reduction demo (#27866) liuxproandunknown 2026-06-12 16:57:33 +08:00
• 371b96e210 [AMD] Fix DeepSeek-V4-Flash-FP8 on MI300 (#27972) Xinyi Song 2026-06-12 01:51:32 -07:00
• 694cea8656 Add EPD disaggregated encode tracing (#25994) Joectwm 2026-06-12 03:16:14 -05:00
• 60e4f14953 [NPU][Bugfix] Fix accuracy issue in no-graph with MTP (#27752) iridiumine 2026-06-12 16:05:06 +08:00
• a52ccd2179 [Spec] Install EagleDraftExtendInput as the V2 draft-extend spec_info (#24860) Liangsheng Yin 2026-06-12 00:46:16 -07:00
• 1cd5cb1220 [AMD] [CI] Add dsv4 accuracy PR gate to pr-test-amd-rocm720 (#27149) Bingxu Chen 2026-06-12 14:42:16 +08:00
• 66ab5c9c7a fix(gateway): make sgl-model-gateway a cargo workspace so maturin 1.14 accepts the parent README (#27997) Xinyuan Tong 2026-06-12 07:37:29 +01:00
• fe887e6935 test(xpu): add multi-feature and embedding stage-b tests (#27861) ashwini rathi 2026-06-12 12:06:28 +05:30
• 3c1f9eafa5 [sgl] proactively release out-of-window SWA slots after chunked prefill (#27402) Bi Xueandispobock 2026-06-11 23:17:18 -07:00
• 36d61613a1 [AMD] Cache unified_kv swa_loc once per step instead of per layer (#27978) Xinyi SongandThomas Wang 2026-06-11 23:13:15 -07:00
• ca17bd8347 perf: eliminate CUDA syncs in VLM embed path (#26082) Jumiar 2026-06-12 13:15:56 +08:00
• cce35ee2e5 Remove outdated patch (#27994) Alan Kao 2026-06-12 13:14:55 +08:00
• ddbe5ff2f1 [AMD] Pin maturin<1.14 to fix ROCm image build failure (#27999) Thomas Wang 2026-06-12 13:13:27 +08:00
• 8038806557 [diffusion] optimize: optimize flux1 tensor parallel sharding (#27826) Mick 2026-06-12 13:13:18 +08:00