 
|
51ace48387
|
[Metrics] Add request-level TPOT histogram (sglang:request_time_per_output_token_seconds) (#40275)
Co-authored-by: Gilford Ting <gilford@modal.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-29 12:20:22 -07:00 |
|
 
|
8f2a9638f7
|
[metrics] Fix non-streaming TTFT by flushing the first output through detokenization (#38600)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
|
2026-09-29 11:24:52 -07:00 |
|
   
|
79cafec013
|
[Model] Add IQuest Q1 support and MTP draft (#41590)
Co-authored-by: zelong huang <yzhu@ubiquant.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: zelong518 <zelonghuang05@gmail.com>
Co-authored-by: chenwenxiaolive <cocoshirolive@gmail.com>
|
2026-09-28 17:21:31 -07:00 |
|
  
|
7889fbab9a
|
[PD] Keep EAGLE DP graph and token metadata consistent (#32196)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
|
2026-09-28 17:15:51 -07:00 |
|
Denji-kk
|
6caf0ff4ae
|
perf(multimodal): offload CPU feature hashing with bounded admission (#39539)
|
2026-09-28 17:51:05 +08:00 |
|
Rain Jiang
|
6c4f36e47a
|
Rust server unify datapath for mm and generate requests (#39679)
|
2026-09-27 23:36:07 -07:00 |
|
    
|
e75e3b8a98
|
[Bugfix] fix(hicache): wait for decode offload before retraction (#30899)
Co-authored-by: agent <agent@local>
Co-authored-by: Kevin Flansburg <6134007+kflansburg@users.noreply.github.com>
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-09-27 23:24:19 -07:00 |
|
 ShunkangzandZhangheng
|
df1fc6ab64
|
[kv-shard 3/4] Enable Control Plane B (#39964)
Co-authored-by: Zhangheng <hzh0425@apache.org>
|
2026-09-27 14:19:08 +08:00 |
|
Sundara Raman Ramachandran
|
3ed56a341b
|
[Score API] Setwise scoring: CausalLM support (batched + --enable-mis) (#41188)
|
2026-09-25 23:02:44 -07:00 |
|
 Lianmin ZhengandLu Fang
|
3ecb6cd729
|
[MemCache] Unify component eviction cursors and lock receipts (#41276)
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
|
2026-09-25 20:53:35 -07:00 |
|
Liangsheng Yin
|
f35ec18d8b
|
[Test] Remove more unit tests that mirror implementation or never run in CI (#41297)
|
2026-09-25 19:22:14 -07:00 |
|
 
|
50e50e13ab
|
[DCP] Use logical token capacity for PD admission and load reporting (#39731)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
|
2026-09-25 17:39:47 -07:00 |
|
Liangsheng Yin
|
1f6ce4b068
|
[Test] Remove unit tests that only mirror implementation or never run in CI (#41286)
|
2026-09-25 16:15:02 -07:00 |
|
 Yuwei AnandClaude Opus 5.5
|
95047d3464
|
[Fix] Fix cpu CI fail introduced by pr #38652 (#41287)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-09-25 15:55:21 -07:00 |
|
 Erik WijmansandEthan Su
|
c421d16563
|
[sglang][lora] Support DP attention in LoRA backends (#36389)
Co-authored-by: Ethan (Yusheng) Su <yushengsu@radixark.ai>
|
2026-09-25 12:03:47 -10:00 |
|
 
|
cf5df82680
|
[KVCache] Support lmcache unified radix cache (#38652)
Signed-off-by: chunxiaozheng <1179548172@qq.com>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-09-25 14:08:45 -07:00 |
|
Nan Jiang
|
24b6930076
|
[Sampling] Stream sampling masks as per-request arrays (#40986)
|
2026-09-25 12:20:25 -07:00 |
|
Richard Wang
|
16d1c93b3d
|
[feat] add a system one compatible /v1/systemone route (#41208)
|
2026-09-25 12:59:11 +08:00 |
|
 
|
7325b38412
|
[PD] Honor gracefully_exit in disaggregation event loops and keep non-zero-rank launchers alive on SIGTERM (#40793)
Co-authored-by: metamergebot <324680979+metamergebot@users.noreply.github.com>
Co-authored-by: Shiyan Deng <dsy842974287@meta.com>
|
2026-09-24 20:29:07 -07:00 |
|
jacky.cheng
|
9dc4c5d891
|
[AMD] Restore non-DCP Mamba checkpoint donation to fix agent-mode cache hit at high conc with HiCache (#40907)
|
2026-09-24 18:08:10 -07:00 |
|
Sundara Raman Ramachandran
|
2f2f9d12f8
|
[Score API] Setwise Scoring Support (#38965)
|
2026-09-24 16:50:58 -07:00 |
|
 cctryandcctry
|
efff836911
|
[PD] Add decode host receive for custom transfer backends (#40238)
Co-authored-by: cctry <cctry@fb.com>
|
2026-09-23 21:57:33 -07:00 |
|
Nan Jiang
|
c3685dff42
|
[Sampling] Add selected/support sampling logprob modes (#40932)
|
2026-09-23 21:38:19 -07:00 |
|
      
|
f4d9d2e670
|
[RL] Add RL weight-update sessions and support updating spec draft runners (#40777)
Co-authored-by: Jiajun Li <guapisolo@gmail.com>
Co-authored-by: Nan Jiang <59716405+nanjiangwill@users.noreply.github.com>
Co-authored-by: maocheng23 <maocheng@berkeley.edu>
Co-authored-by: Byron Hsu <byron+per@periodiclabs.ai>
Co-authored-by: JD-ETH <jaedon.guo@gmail.com>
Co-authored-by: Letian Ruan <ruanlt420@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
|
2026-09-23 20:04:57 -07:00 |
|
 
|
6fe4b66f5c
|
[RL] Keep pause_generation and weight updates from deadlocking each other (#40779)
Co-authored-by: maocheng23 <maocheng@berkeley.edu>
Co-authored-by: Zhichenzzz <zczeng@uw.edu>
|
2026-09-23 15:13:42 -07:00 |
|
Cheng Wan
|
79fec592a8
|
[Refactor] Read parallel placement in consumers (#40638)
|
2026-09-23 14:57:25 -07:00 |
|
 
|
7fef014d51
|
feat(kv-hints): add kv hint envelope to request transport (#38891)
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
|
2026-09-23 10:31:13 -07:00 |
|
 inkcherryandDuyi-Wang
|
58f622ebc4
|
Reduce decode bootstrap latency with request-owned speculative KV (#38978)
Signed-off-by: inkcherry <mingzhi.liu@amd.com>
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
|
2026-09-23 02:24:53 -07:00 |
|
Richard Wang
|
40048f6e51
|
[dLLM] feat: support DiffusionGemma serving (#34061)
|
2026-09-23 09:59:31 +08:00 |
|
Shunkangz
|
ddebc52f23
|
[kv-shard 3/4] Enable Control plane (#38468)
|
2026-09-22 19:57:44 +08:00 |
|
jthomson04
|
15ba54bd5d
|
perf(engine): avoid timed waits for Engine responses (#39486)
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
|
2026-09-21 20:42:03 -07:00 |
|
YAMY
|
0229025127
|
[Spec][PP] Launch extend microbatches before the spec output exchange (#40499)
|
2026-09-21 15:47:03 -07:00 |
|
Cheng Wan
|
acac4dd9d9
|
[Refactor] Clean up parallel runtime comments (#40632)
|
2026-09-21 14:32:22 -07:00 |
|
Cheng Wan
|
970e946e4f
|
Retire the per-runner parallel record (#40343)
|
2026-09-21 12:26:40 -07:00 |
|
Cheng Wan
|
73f071db52
|
Deprecate the parallel getters the context answers, and ratchet them shut (#40342)
|
2026-09-21 12:25:32 -07:00 |
|
Cheng Wan
|
0db1a93adb
|
State the draft's whole topology in its scope, and read the rest from the context (#40339)
|
2026-09-21 12:19:38 -07:00 |
|
 AMRUTHA MandMa Mingfei
|
d20cd9d77f
|
[XPU]Enable HiSparse hierarchical sparse KV cache on Intel XPU (#32792)
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
|
2026-09-21 14:07:49 +08:00 |
|
Liangsheng Yin
|
0024efa0de
|
[CI] Derive registered-test kind from the registry call instead of the path (#40294)
|
2026-09-20 02:01:22 -07:00 |
|
Qiaolin Yu
|
f4c256354c
|
[kimi k3][pd disagg] support pp prefill + dcp decode with dspark (#40045)
|
2026-09-20 00:15:28 -07:00 |
|
 Mohammad Miadh AngkadandMohammad Angkad
|
22f02cc339
|
[Test] Fix scheduler fixtures after prefill burst counting (#40411)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
|
2026-09-20 00:08:04 -07:00 |
|
 huangtingweiandChao Shi
|
020703923d
|
[PP + HiCache] Add PP Prefetch Tickets for eager cross-stage storage prefetch (#36700)
Co-authored-by: Chao Shi <stepinto@live.com>
|
2026-09-20 11:19:31 +08:00 |
|
  
|
3a64faa1f2
|
Fix disagg PP MTP for GLM-5.2 (#39378)
Co-authored-by: Julien Lin <jullin@nvidia.com>
Co-authored-by: YAMY1234 <74099316+YAMY1234@users.noreply.github.com>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
|
2026-09-19 13:55:07 -07:00 |
|
Xiaoyu Zhang
|
0b0d2c257a
|
[Fix] Repair CI fixtures and ROCm speculative tree device checks (#40325)
|
2026-09-19 18:16:05 +08:00 |
|
Cheng Wan
|
3a5f52e144
|
Record a process's placement at publish, not at group build (#40071)
|
2026-09-18 23:52:37 -07:00 |
|
 
|
677c1cbdc9
|
[Metrics] Propagate idle gaps across all scheduler loops (#40004)
Co-authored-by: metamergebot <324680979+metamergebot@users.noreply.github.com>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
|
2026-09-18 22:42:00 -07:00 |
|
 
|
8189e3896b
|
[PD] Skip singleton transfer-status all-reduces (#40003)
Co-authored-by: Pranjal Shankhdhar <pranjalssh@users.noreply.github.com>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
|
2026-09-18 22:41:50 -07:00 |
|
Vincent Liu
|
090263eff6
|
[VLM] avoid CUDA placement on non-CUDA platforms (#38750)
|
2026-09-19 11:13:22 +08:00 |
|
Xiaoyu Zhang
|
986959e3c4
|
[Refactor] Deduplicate kernel helpers and remove unused code (#40197)
|
2026-09-19 09:27:30 +08:00 |
|
   
|
5931fd60ee
|
Support unified memory page-envelope transfers in PD (#39477)
Co-authored-by: yhzhuang <yhzhuang@fb.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Yonghao Zhuang <yhzhuang@users.noreply.github.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
|
2026-09-18 17:39:50 -07:00 |
|
       
|
a6cf05817f
|
dsv4.1: remaining model and runtime integration (#38798)
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
|
2026-09-18 02:55:30 -07:00 |
|