This website requires JavaScript.
5569b14d26
fix(benchmark): request token IDs when counting OpenAI prompts (#34511 )
Jiaqi Li and Zijie Xia
2026-10-14 05:31:45 +08:00
2341e647d3
[DeepSeek-V4.1] Run decoder SWA bounded replay inside the breakable prefill CUDA graph (#43736 )
Yizheng Jiao
2026-10-13 13:55:09 -07:00
79af3df346
[gRPC] Move runtime.v1 bindings to tonic 0.14, report OpenAI status codes, return text on request (#42673 )
2026-10-13 13:46:15 -07:00
18210ed849
[HiSparse] Add speculative decoding support with multi-step KV cache swapping (#37771 )
huangtingwei
2026-10-14 04:38:31 +08:00
7dc2930585
[Feature] Add Aaronson-Gumbel text watermarking (#37577 )
Xinyuan Tong and Nihar Patel
2026-10-14 03:47:50 +08:00
5eda6b7e67
docs: add MiniCPM-V 4.7 cookbook (#43829 )
Xinyuan Tong
2026-10-14 03:04:47 +08:00
ce5352a814
[Rust] Serve any model name on the OpenAI chat and completions routes (#43923 )
2026-10-13 11:58:57 -07:00
01f3ed18a5
[unified memory pool] Prefer SWA reclaim quotas with a HiCache storage backend or on PD decode (#43919 )
2026-10-13 11:58:24 -07:00
7e3da1b784
[HiCache] Extract the buffer-mode prefetch and load-back lifecycle into pipeline operations (#43870 )
2026-10-13 11:51:44 -07:00
5bcaf0f8ed
[Core] Add request-level customized_info keys and mark Full prefill capture batches (#43772 )
2026-10-13 11:45:52 -07:00
149ef7d1b7
[Rust] Let an embedding package wrap the frontend HTTP app (#43924 )
2026-10-13 11:44:43 -07:00
a8381d73be
[HiCache] Block-group pool hits, caller-owned load destinations, allocation-target FULL eviction (#43868 )
2026-10-13 11:39:48 -07:00
8bf145d119
[DCP] Decode context parallelism for DSA (GLM-5.2 / GLM-5.3) with MTP (#43735 )
Khoa Pham and vincentzed
2026-10-13 11:35:34 -07:00
6ade23696d
[Core] Keep request priority within the signed 64-bit range in radix caches (#43920 )
2026-10-13 11:33:26 -07:00
7f9f742535
[Rust] Refuse an unsupported multimodal model before the Rust listener binds (#43925 )
2026-10-13 11:30:43 -07:00
2a8e9db3b0
[lora] Support LoRA for the NemotronH_Nano_VL_V2 model family (#39798 )
2026-10-14 02:29:48 +08:00
ff3f8c348c
[Observability] Report per-request storage prefetch and staging waits in meta_info (#43946 )
2026-10-13 11:13:01 -07:00
e98c6d7b25
Rainj me/rust bench (#43950 )
Rain Jiang
2026-10-13 11:07:42 -07:00
c0584cf9af
[perf] [multi-lora] remove empty CTAs from fused LoRA expand and add split-K to shrink (#41951 )
Kailash Ranganathan and Yanbin Jiang
2026-10-13 10:16:14 -07:00
e9400c5a8f
[AMD][DI][CI] kimik26 2p1d-ep16: capture CUDA graph on decode (eager decode is slow) (#43985 )
Zhaoyi Li
2026-10-13 10:46:53 -05:00
96f519e4fa
[AMD][DI][CI] glm52 2p1d-ep16: capture CUDA graph on decode (eager decode is too slow) (#43977 )
Zhaoyi Li
2026-10-13 10:45:46 -05:00
11cd2701a8
fix(openai): preserve zero cached tokens when cache reporting is enabled (#43005 )
2026-10-13 08:29:19 -07:00
3f1e185f6e
fix(openai): keep chat stream created timestamp stable (#43026 )
2026-10-13 07:43:33 -07:00
4c5ed0597e
Skip fully-masked KV tiles in sliding-window extend attention (#39769 )
jiaqiang-dot-liu and Cursor
2026-10-13 22:40:53 +08:00
27b68303d0
fix(openai): omit absent tool_calls from chat responses (#43027 )
2026-10-13 05:56:58 -07:00
8e37c6cf37
[CI] Run test_pi05_modelopt_fp8_cuda in the kernel-unit lane (#44055 )
Po-Han Huang (NVIDIA) and Claude Opus 5.5
2026-10-13 20:39:02 +08:00
5100f8d11d
[DSA] Q8KV8 FP8 Sparse Prefill on GLM-5.3-Flash with BF16 KV Cache (#41790 )
Ho-Ren (Jack) Chuang and Xiaoyu Zhang
2026-10-13 05:30:13 -07:00
29974d543e
[NPU] [DOC] update kimi k3 cookbook and supported features on NPU (#44052 )
amote-i
2026-10-13 20:27:26 +08:00
74e0a47863
Fix/compressed tensors nvfp4 hopper (#40022 )
jlqibm and Claude Opus 5
2026-10-13 08:10:31 -04:00
5727f08932
[diffusion] feat: add configurable reference image resizing for MiniMax H3 Ref2VA (#35839 )
LittleNyima and Mick Qian
2026-10-13 19:48:18 +08:00
85254b5a8f
[diffusion] comfyui: run Qwen-Image 2.1 in SGLD integrated mode (#43581 )
WenhaoZhang
2026-10-13 19:47:02 +08:00
991a29d0ad
fix(multimodal): convert grayscale tensors to RGB in PIL backend (#42127 )
Yunhao Zhou and 周云昊
2026-10-13 19:03:48 +08:00
4b5b727145
Fix EVS for NemotronH_Omni_Reasoning_V3 (#42933 )
2026-10-13 18:59:00 +08:00
5df4f2c036
DEBUG-ONLY: use crc32 for rid fingerprint (builtin hash() is per-process randomized)
minke.yu
2026-10-13 18:12:56 +08:00
db1e8cdbe3
DEBUG-ONLY: ARDBG batch fingerprint + phase + hicache sync logs, sampled device AR (1/1024) + idle spin sampling (p2 wedge trap)
minke.yu
2026-10-13 18:01:36 +08:00
aa15b06abe
[AMD][CI] Run test_tracing.py in the extra tier, matching CUDA (#43928 )
Michael and Claude Opus 5.5
2026-10-13 02:45:58 -07:00
4e0fa52530
[Perf] Compact GDN causal-conv grids for skewed prefill batches (#43570 )
Xiaoyu Zhang
2026-10-13 17:45:06 +08:00
01f6840693
[FlashInfer] Add startup autotuning for MegaMoE (#43789 )
Ziang Li
2026-10-13 02:30:31 -07:00
195cb7afb4
[CI] Use sgl-eval for GLM-5.3-Flash GSM8K tests (#43975 )
Mohammad Miadh Angkad and Mohammad Angkad
2026-10-13 17:26:50 +08:00
e0c957e222
[HiCache][PD] Allow DSA index-K elision under HiCache and disaggregation (#41769 )
Shu Wang and Anish Shanbhag
2026-10-13 04:07:54 -05:00
27be99ae73
[NVIDIA][MegaMoE] Fix the default spec decoding moe backend when the target model uses flashinfer megamoe (#40772 )
Shu Wang and Po-Han Huang
2026-10-13 04:00:24 -05:00
f3ac513d92
DEBUG-ONLY: limit ARDBG stack logging to ONE_SHOT_PUSH calls (volume control)
minke.yu
2026-10-13 16:01:18 +08:00
0054c5b8fe
DEBUG-ONLY: SGLANG_AR_SEQ_DEBUG seq logging for TP device AR + host CP gloo AR (p2 wedge repro)
minke.yu
2026-10-13 15:52:17 +08:00
6da0a40b02
g9i shadow: fix temporary-borrow compile errors in new tests
minke.yu
2026-10-13 15:39:06 +08:00
caf3403086
g9i shadow: fix cached_tokens_details placement to sglext container
minke.yu
2026-10-13 15:33:03 +08:00
617f19ba61
gateway tests: fix PDRouter initializer missing g9i field (pre-existing test compile break)
minke.yu
2026-10-13 15:18:33 +08:00
7ff3a11284
[Perf] Enable fast-math for JIT activations on Blackwell (#43236 )
Chunan Zeng and Claude Opus 5.5
2026-10-13 00:18:05 -07:00
ba3318ffbd
gateway g9i: fix shadow compile errors — lookup returns owned snapshot (borrow across loop mutation), bind temp in test
minke.yu
2026-10-13 15:14:05 +08:00
8279d3d6df
gateway g9i: v2 P1 shadow mode — prefix-hash chain, bookkeeping table, shadow classifier + queue model
minke.yu
2026-10-13 15:05:56 +08:00
d8f8522348
[router] power_of_two: in-flight request load for --dp-aware decode (#43483 )
AMD-yanfeiwang
2026-10-13 14:50:54 +08:00
35a014c7e8
[PD][NIXL] Fix staging buffer startup by accepting slot_layer_ids (#43723 )
Michal Shalev and Po-Han Huang
2026-10-13 09:48:13 +03:00
dbad07a06f
[Docs] Cookbook: refresh SemiAnalysis AgentX launches (DeepSeek-V4.1 GB200/GB300, GLM-5.2 FP8 B300) (#43983 )
Po-Han Huang (NVIDIA) and Claude Opus 5.5
2026-10-13 14:39:45 +08:00
50a144057c
[AMD][PD] enable mori transfer for PD decode DCP (#37358 )
2026-10-13 14:35:09 +08:00
46b7357779
[Diffusion] Speed up LTX-2 guidance and video output (#43667 )
Xiaoyu Zhang
2026-10-13 14:23:33 +08:00
3eebf987ed
[PD] Support heterogeneous TP for hybrid SWA KV transfer (#40909 )
Jimmy Shong
2026-10-12 23:21:54 -07:00
603a05e6c0
[ROCm][Spec] Route NGRAM tree reconstruction to the JIT kernel on HIP (#43328 )
2026-10-13 14:17:49 +08:00
17c3b259f5
[router] Fix power_of_two with --dp-aware (#42027 )
AMD-yanfeiwang
2026-10-13 14:15:36 +08:00
fc404bbbb3
[AMD][DI][CI] dsv4pro-fp8 2p1d-ep16-mtp: raise prefill mem fraction for MORI EP v2 memory (#43944 )
Zhaoyi Li
2026-10-13 01:03:57 -05:00
a69e893b1a
[AMD] Fuse prefill index-Q RoPE fake-quant and fp4 quantize kernels in DSV4.1 (#42911 )
Xinyi Song
2026-10-12 22:33:53 -07:00
42e43cf33f
[AMD] Fix Flux2 kv-gather abort on non-contiguous K/V (#43793 )
jacky.cheng and Mick Qian
2026-10-13 13:20:55 +08:00
43356304b4
[AMD][Kimi-K3] Use public FlyDSL APIs in KDA decode kernels (#43968 )
kk and wunhuang
2026-10-13 13:09:55 +08:00
ea13359da9
[diffusion] perf: implement paint_use_remesh (decimate to 40k faces before UV unwrap) for Hunyuan3D (#42770 )
WenhaoZhang and Mick Qian
2026-10-13 12:28:32 +08:00
6a89181614
tokenizer perf optimization (#39771 )
2026-10-13 12:27:21 +08:00
feba04fa49
[diffusion] fix: avoid redundant LoRA scaling while preserving exact numerics (#43028 )
WenhaoZhang and Mick Qian
2026-10-13 12:27:04 +08:00
cda267ab0f
[diffusion] fix: keep MiniMax-H3 VAE fused norms off ROCm (#43350 )
WenhaoZhang
2026-10-13 12:26:16 +08:00
b4822bae03
[diffusion] comfyui: run MiniMax H3, FastH3 and VDN in SGLD integrated mode (#43504 )
WenhaoZhang
2026-10-13 12:01:48 +08:00
4d5692b6cd
fix(openai): honor tool_choice=none in DeepSeek encoders (#43006 )
2026-10-12 20:41:50 -07:00
40f5d9411f
Reuse Hugging Face LFM2-MoE configuration (#43939 )
Piotr Mazurek and Codex
2026-10-12 20:29:18 -07:00
7bd41aec12
[CI] Remove Mistral Small 4 e2e test (#43973 )
Mohammad Miadh Angkad and Mohammad Angkad
2026-10-13 11:14:22 +08:00
c2dfd5d3d6
[AMD] Update v4 amd cookbook 1012 (#43962 )
Thomas Wang
2026-10-13 10:57:55 +08:00
03416e8c0f
[Fix] Account for captured inputs in DeepGEMM masked workspace budget (#43563 )
Xiaoyu Zhang
2026-10-13 10:53:10 +08:00
43e4e84731
[PD] Recompute retracted DeepSeek-V4 requests on prefill instead of crashing decode (#43936 )
Liangsheng Yin
2026-10-12 19:32:03 -07:00
07566cbf0d
[DSv4.1] Temporarily disable B300 PD/CP accuracy test (#43948 )
Yuwei An
2026-10-12 19:31:10 -07:00
75a0d91c1f
[Fix] Preserve HTTP errors for non-streaming /v1/messages (#43796 )
2026-10-13 10:10:45 +08:00
45418d7dc6
[HiCache] Avoid full-store scans for uncached file prefix lookups (#42077 )
2026-10-13 09:52:50 +08:00
23150f7ae0
[DSv4.1] Reduce sparse-prefill workspace padding (#43921 )
Yuwei An
2026-10-12 18:51:36 -07:00
107cff72d3
[DeepSeek-V4.1] mHC: drop seam_open, narrow precomputed stats in take_rows (#43833 )
Ziyi Xu and Claude Opus 5.5
2026-10-13 09:45:16 +08:00
e775fa7253
[Build][CI] Drop Python 3.10: require Python 3.11+ and run CUDA CI on Python 3.12 (#43021 )
2026-10-13 09:40:16 +08:00
59445b3da1
[DSV4][CP] Write low-ratio page indices when CP prefill runs paged attention (#43935 )
YAMY
2026-10-12 18:36:11 -07:00
7224fce1b5
[AMD] Recalibrate test_quark_mxfp4 GSM8K thresholds to observed MI355X noise (#41980 )
Michael
2026-10-12 16:46:03 -07:00
6a7abcfc9b
[DeepSeek-V4.1] Admit the NIXL transfer backend for DSpark PD (#43734 )
Yizheng Jiao
2026-10-12 15:32:17 -07:00
93809202a3
[Kernel] Expose bundled FlashMLA MegaAttention kernels and APIs (#43918 )
Khoa Pham
2026-10-12 14:50:04 -07:00
6781da8e5b
[AMD] [Docker] Update MoRI to v1.2.4 (#43834 )
Duyi-Wang
2026-10-13 05:45:42 +08:00
64cffed139
[AMD] Fix test_compact_attention_hip.py CI (#43836 )
Thomas Wang
2026-10-13 05:43:19 +08:00
d6853f5066
[AMD] Add flydsl mxfp8 dense gemm kernel for dpsk v4.1 prefill (#43803 )
Thomas Wang
2026-10-13 05:42:02 +08:00
6c2a682b59
[AMD] Keep the FP4 varctx schedule test off AITER's 2048-row compile (#41793 )
Michael
2026-10-12 14:39:08 -07:00
6966c5d027
[ROCm] Eliminate HiCache migration stalls and use exact-size host allocations (#42553 )
Zhang, Jiejing and JohnQinAMD
2026-10-12 14:05:51 -07:00
b60444c621
Fix autograd retention in CUDA graph capture tail hooks (#43854 )
ix-48
2026-10-13 03:31:18 +08:00
0efbb3d2b9
[DFlash] Decide the Domino candidate pool in one place and apply simulated acceptance inside the accept (#43791 )
Liangsheng Yin
2026-10-12 11:35:57 -07:00
392039d752
[DSpark] Share draft params, logits dtype, confidence and the accept tail between graph and eager (#43790 )
Liangsheng Yin
2026-10-12 11:35:41 -07:00
ef4cccb6a0
gateway g9i: allow flow-control-only mode (no fast-P URL) — B case flat pool
minke.yu
2026-10-13 01:31:57 +08:00
f92199b569
[Unified Cache] Abort requests whose external linker load fails instead of crashing the scheduler (#40896 )
Shuwen Wang
2026-10-13 00:03:21 +08:00
98b950366d
mooncake PD transfer: mix bootstrap_room into lane shard hash (fixes 1/16 lane collapse with single decode instance)
minke.yu
2026-10-12 23:41:23 +08:00
d88d5c325a
Prebuild fixup_zero_kv JIT module at TRTLLM-MLA backend init (#37525 )
Sam Shleifer and Claude Fable 5
2026-10-12 11:27:21 -04:00
23a1f53fc3
[QSA] Bound the decode indexer's work by the context length (#43463 )
Yuan Luo and luoyuan.luo
2026-10-12 22:47:45 +08:00
1a553927b0
[CI][Diffusion] Add caching for npu GT images in CI (#37426 )
Alexandr and Артем Савкин
2026-10-12 17:18:01 +03:00
279f1fad90
[diffusion] chore: bind worker CPUs to the GPU's NUMA node when memory policy is not permitted (#43756 )
2026-10-12 21:09:41 +08:00
87ab6c0155
gateway g9i: probe fast-P only + L3-inclusive classification + exact-body probe cache (A案; removes accidentally reintroduced home_hit)
minke.yu
2026-10-12 13:03:54 +00:00
0308920bdb
gateway g9i: retry probe once on transport errors (stale keep-alive pool)
minke.yu
2026-10-12 12:02:49 +00:00
d1357bf87b
[NPU] Diffusion CI fixes (#43210 )
Артем Савкин
2026-10-12 14:53:51 +03:00