 Jason Mancusoandhnyls2002
|
201c4b9b2a
|
[RL, Spec] Introduce top-p mask capture for spec and add DFlash/DSpark impl (#34201)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
|
2026-09-30 13:32:26 -07:00 |
|
Nan Jiang
|
24b6930076
|
[Sampling] Stream sampling masks as per-request arrays (#40986)
|
2026-09-25 12:20:25 -07:00 |
|
Ting SUN
|
752801e4d1
|
fix(sampling): validate sampling_seed is an int within int64 range (#28960)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-09-24 17:25:13 +02:00 |
|
Nan Jiang
|
c3685dff42
|
[Sampling] Add selected/support sampling logprob modes (#40932)
|
2026-09-23 21:38:19 -07:00 |
|
 
|
f31a7bd45c
|
Use pinned memory for asynchronous sampling metadata transfers (#39777)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
|
2026-09-20 15:31:31 -07:00 |
|
Lifan Shen
|
8ea0ee300d
|
perf(sampling): avoid GPU syncs when applying custom logit processors (#39234)
|
2026-09-18 17:09:30 -07:00 |
|
Jensen
|
5beb2fd552
|
[NPU] Avoid device synchronization in Ascend sampling (#39404)
|
2026-09-16 10:15:16 +03:00 |
|
 jasonjk-parkandjasonjk
|
073fdf97cc
|
Generalize auxiliary outputs (#39164)
Co-authored-by: jasonjk <jasonjk@twshared0085.36.lco2.facebook.com>
|
2026-09-15 12:05:13 -07:00 |
|
 Yuxuan ZhangandXinyuan Tong
|
17ba2c2e7c
|
Support non-strict GLM47 tool calls with EBNF constraints (#38890)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
|
2026-09-15 16:41:48 +08:00 |
|
  
|
fd7743e0e1
|
[Sampling] Support sampling masks with overlap scheduling (#36631)
Co-authored-by: ByronHsu <ByronHsu@users.noreply.github.com>
Co-authored-by: root <root@slurm-h200-208-179.slurm-compute.tenant-slurm.svc.cluster.local>
Co-authored-by: Byron Hsu <byron+per@periodiclabs.ai>
|
2026-09-10 14:52:44 -07:00 |
|
 Byron HsuandByron Hsu
|
a88e852fab
|
[Sampling] Allow sampling-mask replay with DisallowedTokensLogitsProcessor (#38279)
Co-authored-by: Byron Hsu <24364830+ByronHsu@users.noreply.github.com>
|
2026-09-06 23:56:16 -07:00 |
|
 
|
3bac084d4e
|
[Model] Add native IFM K2 Horizon serving support (#37654)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
|
2026-09-03 16:39:43 +08:00 |
|
 Alex NailsandAlison Shao
|
28262c20df
|
[CI][RFC] Replace black-jupyter with ruff-format (#37210)
Co-authored-by: Alison Shao <a.shao@wustl.edu>
|
2026-09-02 19:46:08 -07:00 |
|
 Byron HsuandByronHsu
|
046cdaabaa
|
[Sampling] Capture masks from sampler support (#36630)
Co-authored-by: ByronHsu <ByronHsu@users.noreply.github.com>
|
2026-09-02 17:13:28 -07:00 |
|
EchO
|
5b7c62d5d6
|
fix(frontend): bound stop strings and regex patterns (#37029)
|
2026-08-29 22:25:26 -07:00 |
|
   
|
ec4bdbfa4a
|
[Feature] Beam search support (#31626)
Co-authored-by: cswuyg <cswuyg@gmail.com>
Co-authored-by: cswuyg <496090217@qq.com>
Co-authored-by: Vedant Jhaveri <vedantjh2@gmail.com>
Co-authored-by: Vedant Jhaveri <vjhaveri@linkedin.com>
|
2026-08-26 16:56:15 -07:00 |
|
 Alec SandAlec Solder
|
fbafd1b123
|
Add sampling observer auxiliary output hooks (#35747)
Co-authored-by: Alec Solder <alecs@fb.com>
|
2026-08-21 19:28:18 -07:00 |
|
milesial
|
082aac8fce
|
[Bugfix] Fix min-new-token EOS handling (#31378)
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com>
|
2026-08-19 15:29:22 -07:00 |
|
 Lifan ShenandXinyuan Tong
|
a2161ce682
|
Support thinking budget for Inkling (#33146)
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
|
2026-08-11 02:01:08 +08:00 |
|
Cheng Wan
|
eda0ddc260
|
config: delete the dead get_server_args() bindings across the repo (#33888)
|
2026-08-07 22:41:53 -07:00 |
|
 zijiexiaandClaude Opus 4.8
|
b819d2fb5b
|
[Docs] Rename docs_new/ to docs/ (#32123)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-08-03 16:51:00 -07:00 |
|
Liangsheng Yin
|
f5f021672a
|
[Fix] Treat an empty grammar constraint as unset in SamplingParams (#33328)
|
2026-08-02 21:45:51 -07:00 |
|
Mark
|
741e33db81
|
fix(sampling): reject conflicting structural tag constraints (#32525)
|
2026-08-02 20:24:55 -07:00 |
|
Cheng Wan
|
55b6769b0e
|
config: read resolved config via namespace accessors (#33013)
|
2026-07-31 15:06:59 -07:00 |
|
Lianmin Zheng
|
16a52bff23
|
[Refactor] Move sampling tokenizer validation helper (#32694)
|
2026-07-28 16:48:03 -07:00 |
|
Lianmin Zheng
|
9ca4023b13
|
[Core] Clean up array-like msgspec structs (#32688)
|
2026-07-28 16:25:22 -07:00 |
|
 Lianmin ZhengandAlec S
|
9989077f24
|
Use native batched llguidance mask generation (#32412)
Co-authored-by: Alec S <10566873+alecsolder@users.noreply.github.com>
|
2026-07-25 16:36:32 -07:00 |
|
Liangsheng Yin
|
3da1071d56
|
[Spec] Hold the grammar bitmask in one GrammarMask type across all decode paths (#32409)
|
2026-07-25 15:27:43 -07:00 |
|
Raghavendra Vedula
|
a9497e8d73
|
Guard min_new_tokens penalizer against None eos_token_id (#31973)
|
2026-07-22 23:24:25 +08:00 |
|
hari-hm
|
a909077d22
|
Return top-p/top-k sampling mask/nucleas (#27408)
|
2026-07-13 17:58:51 -05:00 |
|
zijiexia
|
50ed4c011f
|
Remove legacy Sphinx docs/ and finish the Mintlify cutover (#28964)
|
2026-07-13 15:06:08 -07:00 |
|
    
|
6cc9352dfe
|
[Spec] Add DSpark: confidence-scheduled speculative decoding (#30261)
Co-authored-by: sglang-bot <232288953+sglang-bot@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
|
2026-07-12 17:25:26 -05:00 |
|
Cheng Wan
|
1f15308dca
|
[refactor] Retire the legacy config accessor and the remaining process singletons (#30493)
|
2026-07-09 02:10:47 -07:00 |
|
 Rain JiangandLianmin Zheng
|
be1930133a
|
Convert IPC dataclasses to msgspec.Struct with opt-in msgpack transport (#28688)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2026-06-26 12:04:03 -07:00 |
|
 Lianmin ZhengandRain Jiang
|
fd87a85388
|
Convert SamplingParams to msgspec Struct (#29198)
Co-authored-by: Rain Jiang <96632942+rainj-me@users.noreply.github.com>
|
2026-06-24 13:42:45 -07:00 |
|
Liangsheng Yin
|
556cf54d47
|
[Perf] Avoid per-decode-step host sync in min_new_tokens penalty (#28397)
|
2026-06-15 23:32:28 -07:00 |
|
 
|
0417951a86
|
[Bug Fix] Validate tokenizer-dependent features with skip_tokenizer_init (#27882)
Co-authored-by: Randall <randall@iterationlab.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-06-14 20:07:52 -07:00 |
|
Ting SUN
|
b250bea994
|
fix(sampling): reject non-finite temperature in SamplingParams.verify (#28153)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-06-13 23:41:36 -07:00 |
|
Qiaolin Yu
|
e02f7ca482
|
[perf] remove several h2d sync (#28076)
|
2026-06-12 20:30:17 -07:00 |
|
Cheng Wan
|
97a0031799
|
[lint] Enable Ruff UP037 to drop redundant quoted annotations (#27984)
|
2026-06-11 17:38:05 -07:00 |
|
 Mohammad Miadh AngkadandLianmin Zheng
|
52f221cce0
|
Fix Req array token-id concatenation (#26182)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2026-06-06 19:59:51 -07:00 |
|
fzyzcjy
|
268f4c82f1
|
Add rids/bootstrap-room int-hash plumbing for deterministic per-request identification (#26814)
|
2026-05-31 09:57:22 +08:00 |
|
ZeyuanChen2000
|
90eb894564
|
[NPU] fix model llava-onevision-qwen2-7b-ov torch compiles error in npu case (#26573)
|
2026-05-30 17:10:58 +08:00 |
|
 RulinJuiceandRulinJuice
|
3f048c80b8
|
Reject repetition_penalty=0 in SamplingParams.verify() (#24874)
Co-authored-by: RulinJuice <265952454+RulinJuice@users.noreply.github.com>
|
2026-05-12 21:25:23 -07:00 |
|
cctry
|
0addd185af
|
Fix /generate endpoint crash when sampling params contain null values (#23401)
|
2026-04-22 09:56:10 -07:00 |
|
KrishnanPrash
|
a937ec31be
|
fix: server crash when stop_token_ids contains null (#22175)
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
|
2026-04-10 11:42:23 -07:00 |
|
   
|
72d3d8f4cf
|
[Feature Restoration] repetition_penalty is essential for GLM-V models (#21258)
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
|
2026-03-31 23:29:49 -07:00 |
|
Aurick Qiao
|
a34e9ed64a
|
Add adjusted_filter_batch (#21260)
|
2026-03-26 10:59:05 +08:00 |
|
Liangsheng Yin
|
e6d40bff81
|
Revert "feat: reduce constrained-decoding overhead in TP" (#16845)
|
2026-01-10 11:39:38 +08:00 |
|
Raayan Dhar
|
76b3c698d6
|
feat: reduce constrained-decoding overhead in TP (#13947)
Signed-off-by: Raayan Dhar raayan.dhar@gmail.com <raayan.dhar@gmail.com>
Signed-off-by: raayandhar <raayan.dhar@gmail.com>
|
2026-01-10 00:38:32 +08:00 |
|