This website requires JavaScript.
07a087bf45
Fix Inkling tool-call parsing recovery, content handling, and streaming (#32861 )
Ke Bao
2026-07-30 14:11:09 +08:00
f4e0ac382e
[misc] Remove unused multi_layer_draft_forward_cg module (#32881 )
Liangsheng Yin
2026-07-29 21:17:16 -07:00
3d6e1e6f81
[AMD] Revert ROCm AITER pin to 9127c94 (#32879 )
Bingxu Chen
2026-07-30 11:53:49 +08:00
ed361ae7f0
Fix attention backends for models with per-layer head counts (num_attention_heads_per_layer) (#32625 )
Jimmy Shong
2026-07-29 20:03:00 -07:00
22faf9fef8
embedding: centralize capabilities and complete OpenAI compatibility (#32481 )
Mick
2026-07-30 10:28:52 +08:00
313a518bee
[Spec] Emit step trace span for multi-layer draft-extend graph replays (#32850 )
Liangsheng Yin
2026-07-29 19:22:58 -07:00
2aa86e9130
[diffusion] docs: add diffusion cookbook model tags (#32836 )
Mick
2026-07-30 10:03:55 +08:00
62dfaaa0e0
[Nemotron] Fix decode track-save reading the stale tail of the CUDA-graph track buffer (#32555 )
Sam Shleifer and Claude Fable 5
2026-07-29 22:03:44 -04:00
20d5b91e5b
[NPU] [DOC] update feature name to follow the code changement (#32749 )
amote-i
2026-07-30 09:54:08 +08:00
d7a4c830e5
sglang rust server tokenizer manager, ring and runtime (#32358 )
Rain Jiang
2026-07-29 18:42:12 -07:00
8fbf960980
[MLX] Size request capacity by attention DP (#32115 )
Xuanyi Li and R0CKSTAR
2026-07-29 18:18:22 -07:00
1d9c292547
[Kernel] Add inventory guards and clean benchmark layout (#32788 )
Xiaoyu Zhang
2026-07-30 09:03:24 +08:00
5efbb18a6f
[docs] Rotate popular models on the landing pages, lead the Cookbook nav with Kimi (#32835 )
zijiexia and Claude Opus 5
2026-07-29 17:46:50 -07:00
3c9efaf3e1
[docs] Kimi-K3: widen the H200 High-Throughput recipe to 4x8 TP32/EP32 (#32834 )
zijiexia and Claude Opus 5
2026-07-29 17:18:56 -07:00
6e48c13497
sglang rust server egress message (#32342 )
Rain Jiang
2026-07-29 17:16:06 -07:00
a55e1764a2
Enable GPT-OSS FlashInfer MXFP4 on SM120 (#32668 )
Mohammad Miadh Angkad
2026-07-30 08:04:23 +08:00
e5c46ff07d
[Fix] Route asymmetric-KV models to fa4 on SM100 and pin MiMoV2 FP8 MoE to flashinfer_trtllm (#32818 )
Liangsheng Yin
2026-07-29 16:37:43 -07:00
3c1717d9b6
Follow up on #30157 post-merge review (#32672 )
cctry
2026-07-29 15:03:59 -07:00
8fc54d46ef
Fix MoE reduce-scatterv eligibility check (#32663 )
Sam (Kesen Li)
2026-07-30 05:58:55 +08:00
d24de56995
sglang rust server sampling message (#32343 )
Rain Jiang
2026-07-29 14:55:06 -07:00
ffd4705baa
fix(reasoning): honor Poolside template thinking defaults (#32540 )
Willow Lopez and Jiminator
2026-07-30 05:02:19 +08:00
d0e69d3881
[feat] Optional base64 encoding for the flat prompt top logprob arrays (#31960 )
Sam Shleifer
2026-07-29 15:15:56 -04:00
e4f7f7b380
fix(qwen3.5): restrict MoE weights to local PP layers (#32022 )
2026-07-30 03:01:09 +08:00
f73d6f2789
Update Inkling cookbook install command (#32799 )
Ke Bao
2026-07-30 02:23:23 +08:00
2ca2ca753a
sglang rust server request message (#32242 )
Rain Jiang
2026-07-29 10:54:56 -07:00
eefb434d17
[PD+PP] Honor PP consensus for bootstrap and prealloc (#31869 )
ziang663 and Chao Shi
2026-07-30 01:19:17 +08:00
62d0f81f16
[2/N] elastic-ep: Enable EPLB after scale-up (#30553 )
Yoray Zack
2026-07-29 20:06:56 +03:00
d19999b755
[LFM2] Wire Lfm2MoeForCausalLM into the LFM2 serving override tables (#30780 )
Piotr Mazurek
2026-07-29 19:02:14 +02:00
f69af7b7ad
[Bugfix] compressed-tensors: mixed-precision checkpoints silently load unquantized (#32736 )
Hert4 and Mohammad Miadh Angkad
2026-07-29 23:55:33 +07:00
fddfc1fb5e
[GDN] Support FlashInfer GDN prefill with extra-buffer radix cache (#29735 )
YAMY
2026-07-29 09:47:35 -07:00
50029f05a3
[HiCache] Merge HiCache event checks to reduce decode overhead (#30511 )
huangtingwei and alphabetc1
2026-07-29 23:48:48 +08:00
bfc450248e
[AMD] Replace MI325 with MI300 CI Runners (#31409 )
Bingxu Chen
2026-07-29 23:18:07 +08:00
e1f2f9d1fa
Disable extra NCCL CUDA event synchronization with symm mem (#27089 )
Nicolas Castet
2026-07-29 09:39:36 -05:00
3763d3fa8c
Add decode-lock skip to compute-mamba-ratio (#32789 )
Ke Bao
2026-07-29 22:38:34 +08:00
1c6a0e91e1
fix mqa preshuffle layout issue for deepseek v4 (#31563 )
Wu Jiangming
2026-07-29 22:36:48 +08:00
22151edca1
[diffusion] optimization: accelerate CUDA video output finalization (#32784 )
Mick
2026-07-29 22:04:20 +08:00
0ebbe43dbb
fix(diffusion): size VSA top-k from padded blocks (#32695 )
Xiaoyu Zhang
2026-07-29 21:58:41 +08:00
4f5b50c576
perf(diffusion): decode Wan VAE in BF16 (#32697 )
Xiaoyu Zhang
2026-07-29 21:57:50 +08:00
917e900d4d
feat(diffusion): add regional torch compile (#32696 )
Xiaoyu Zhang
2026-07-29 21:57:05 +08:00
50b029257f
Skip mamba lock during decoding (#32228 )
Ke Bao
2026-07-29 21:53:28 +08:00
d004a15a3e
Fix GLM4-7B-Flash accuracy test configuration, tune Qwen3.6-27B/35B performance test parameters, and harden Ascend NPU multi-node E2E test utilities against pod name format errors. (#32371 )
pllimax
2026-07-29 21:44:17 +08:00
977f04aafe
[PD] NIXL connector: shard by destination (#32025 )
Ilia Yastrebov
2026-07-29 15:02:13 +02:00
67c2258906
[diffusion] fix: fix dual-DiT models crash with (1,)-placeholder weights after compile-time offload (#32743 )
Mick and Claude Sonnet 5
2026-07-29 20:44:13 +08:00
ca6e0ff2c8
[NPU] fix dsv4 mtp condition on NPU graph (#32711 )
JiaruiChang5268
2026-07-29 20:13:31 +08:00
d254ec9ff8
Add LFM2.5 embedding model support (#28691 )
Yi (Vincent) Zhong
2026-07-29 04:58:47 -07:00
4c82bb3252
Add Mooncake tenant id support (#30256 )
LZW
2026-07-29 18:17:41 +08:00
8742a1a0f8
Add a benchmark script for the HPC-Ops bf16xfp32 router GEMM (#32642 )
Xiaoyu Zhang and Claude Fable 5
2026-07-29 18:17:08 +08:00
c151080d28
Add compute-mamba-ratio skill (#32763 )
Ke Bao
2026-07-29 18:15:48 +08:00
d12ea3e9ba
docker: add Kimi K3 images (#32760 )
Baizhou Zhang
2026-07-29 03:02:28 -07:00
983e4aa18d
Eliminate redundant DSA state transfers (Mooncake) (#32620 )
hunhokim and Hun-ho Kim
2026-07-29 18:58:48 +09:00
f5bcd00e16
[AMD] DSv4: bring HIP compress-state pool into the memory_saver KV_CACHE region (#31747 )
Xinyu Jiang and Zhiyao Jiang
2026-07-29 02:41:08 -07:00
c32c4ef79c
[Kernel] Move sgl-kernel under sglang.kernels.aot (#32648 )
Xiaoyu Zhang
2026-07-29 17:25:00 +08:00
1b9dfa14e6
Fix FlashInfer MNNVL workspace size check (#32318 )
Mohammad Miadh Angkad
2026-07-29 17:12:18 +08:00
9ab88380c1
👥 chore(codeowners): Update codeowners for NPU quantization (#31917 )
Junlin Wu
2026-07-29 17:10:01 +08:00
227dadd79a
[diffusion] feat: support resident layers for DiT (#31538 )
Yihao Wang
2026-07-29 01:52:55 -07:00
0caf0fc01d
[Diffusion][Docs] Ascend A2, A3 add basic usage and benchmark results in diffusion cookbook (#30614 )
Andrew Kuksa and ANDREW_K
2026-07-29 11:50:15 +03:00
7dcebca255
Fix nightly CI: NVFP4 cuda-graph crash, NVILA batching, CuTe paged-KV zero-size, Kimi-VL OOM (#32118 )
Brayden Zhong and Brayden Zhong
2026-07-29 01:39:55 -07:00
cce5873513
[CI] Fail lint when a registered file's TestCase classes never run (#32735 )
Liangsheng Yin
2026-07-29 00:45:01 -07:00
f05c92fb6d
✨ [llm][npu][quant] Add W8A8 MXFP8 quantization for Qwen3 MoE on Ascend NPU (#30768 )
2026-07-29 15:39:36 +08:00
da5528db30
fix(vlm): materialize Qwen3-VL features on the vision device (#31596 )
Mick
2026-07-29 15:25:04 +08:00
bd47ec97ff
[EAGLE] Handle NaNs in fused top-k=1 (#32396 )
weireweire and weireweire
2026-07-29 15:07:42 +08:00
7c248dde7f
[diffusion] fix: don't self-kill diffusion worker when PID 1 is the real parent (#31361 )
Jyothirmai Kottu and Mick
2026-07-28 23:35:56 -07:00
9bdbb180b1
[Disagg][NIXL] Fix heterogeneous attn-TP KV transfer for replicated GQA heads (NIXL_ERR_NOT_FOUND) (#31968 )
siweil
2026-07-28 23:13:23 -07:00
ef6c07008b
Support DCP for Kimi Linear model (#32612 )
2026-07-28 22:59:58 -07:00
c4fc241fd3
[Perf] Free KV pages by segment in the paged allocator without a device sync (#32701 )
Liangsheng Yin
2026-07-28 22:24:16 -07:00
bf6e80718a
[Elastic EP] fix previously flaky test of test_mooncake_ep_small.py (#31706 )
Xun Sun
2026-07-29 13:14:12 +08:00
f01a0c7f97
Fixing MXFP8 online quantization pipeline (#31510 )
Brayden Zhong
2026-07-28 21:26:13 -07:00
68673fe6c5
[AMD] add Gemma3RMSNorm.forward_hip to unbreak ROCm (#32613 )
YC Yen-Ching Tseng
2026-07-29 12:19:17 +08:00
9a03bebf13
docs(kimi-k3): clarify VLM compatibility (#32661 )
Mick
2026-07-29 11:53:04 +08:00
1af0167493
[EPD][VLM] Fix Kimi-VL 2D encoder grids (#32104 )
Xiaojun(Robin) Zhang
2026-07-28 20:46:17 -07:00
d6fcfe02d6
🐛 [llm][npu][quant] Fix ModelSlim MXFP4 packed weight loading (#32013 )
Junlin Wu
2026-07-29 11:34:41 +08:00
9ea964a535
[diffusion] fix: per-shard FP8 scale shape for single-GPU fused linears (#32157 )
Yihao Wang
2026-07-28 20:28:01 -07:00
b21cc8eb9b
[misc] Update CodeOwner (#32717 )
Baizhou Zhang
2026-07-28 19:55:14 -07:00
14bd315d6e
[Refactor] Remove dead allocator backup_state / restore_state (#32709 )
Liangsheng Yin
2026-07-28 19:49:47 -07:00
dac4325c0e
sgl-kernel-npu tag update to 2026.7.27 (#32596 )
monkeyLoveding
2026-07-29 10:22:13 +08:00
339bef7fad
[MLX] Fix overlap-loop request bookkeeping and graceful shutdown (#32447 )
2026-07-29 10:04:53 +08:00
580b1acbe6
[MLX] Move fused swiglu tests to test/registered so CI collects them (#32448 )
2026-07-29 10:04:24 +08:00
ee678910f7
[Kernel] Skip KV writes to reserved padding slots (#32477 )
Leon Gao and Andrew Gu
2026-07-28 18:58:18 -07:00
d86492fea0
[NPU] adapt dflash v2 on npu (#31739 )
gjsheu
2026-07-29 09:39:19 +08:00
cb12a1547b
[NPU] [DOC] update supported features on ascend npu (#32647 )
amote-i
2026-07-29 09:05:21 +08:00
16a52bff23
[Refactor] Move sampling tokenizer validation helper (#32694 )
Lianmin Zheng
2026-07-28 16:48:03 -07:00
86ee545388
docs(cookbook): update Kimi-K3 GB200 recipes from measured 4x4 runs (#32592 )
YAMY and Lee Nau
2026-07-28 16:26:40 -07:00
9ca4023b13
[Core] Clean up array-like msgspec structs (#32688 )
Lianmin Zheng
2026-07-28 16:25:22 -07:00
70ea37e7e0
vlm: reject moss vision metadata mismatches (#31957 )
Mick
2026-07-29 07:09:54 +08:00
c9947b087b
Enable multimodal prefill BCG for VL and audio models (#30872 )
Xiaoyu Zhang
2026-07-29 06:47:40 +08:00
85618cc798
docs: update sglang cookbook (#32654 )
Liangsheng Yin
2026-07-28 15:33:24 -07:00
7778dd23ea
[diffusion] refactor: remove stale kernels and dead code (#32651 )
Xiaoyu Zhang
2026-07-29 06:23:23 +08:00
7f438a6031
feat: SM120 (Blackwell Desktop) support for GLM-5.1 inference (#26928 )
Void
2026-07-29 05:52:34 +08:00
9c0dbf508f
[cookbook] add inkling dspark command (#32465 )
Qiaolin Yu and Zijie Xia
2026-07-28 14:33:38 -07:00
0a49226d19
[LoRA] 1/n Per-rank tensor serialization for load_lora_adapter_from_tensors under dp_size > 1 (#32580 )
Ethan (Yusheng) Su
2026-07-28 14:28:56 -07:00
d943636a48
support regex that compatible with python re lib however apply more l… (#32676 )
Rain Jiang
2026-07-28 14:25:31 -07:00
dd67452b4f
[Cleanup] Move mamba-max-states-per-path validation into _handle_mamba_backend (#32502 )
YAMY
2026-07-28 14:21:21 -07:00
b8b9f3c8f5
[misc] update CI_PERMISSIONS.json (#32679 )
Qiaolin Yu
2026-07-28 14:12:29 -07:00
4e5a05148a
[FullCG] Support chunked cached-prefix prefill (#30825 )
paulzhang-tm and Claude Fable 5
2026-07-28 14:20:25 -04:00
1eee8fbdcc
[PD] Drain NIXL completion notifications before enforcing the WaitingForInput timeout (#32267 )
IvanShan177 and Claude Opus 4.8
2026-07-29 00:20:03 +08:00
ee236086db
Fix invalid escape warnings in tool parsers (#28370 )
Xinyuan Tong and FAN YUCHEN
2026-07-28 23:31:47 +08:00
84cdfde5b2
[diffusion] fix: fix diffusion output stability on mps (#30017 )
Mick
2026-07-28 21:55:03 +08:00
32c30c0f96
Return 400 instead of 500 for unfetchable or unparseable multimodal inputs (#31417 )
2026-07-28 05:23:42 -07:00
161fffedfc
docs: clarify diffusion stage reuse guidance (#32639 )
Mick
2026-07-28 19:52:52 +08:00
75017c3fa0
[diffusion] fix: keep fused qk-norm-rope out of dynamo tracing (#31849 )
Mick
2026-07-28 19:51:00 +08:00