 Liangsheng Yin
|
cde5a6e30f
|
Abstraction for spec worker and code cleanup (#11643)
|
2025-10-17 23:31:36 +08:00 |
|
|
Liangsheng Yin
|
d88ac9bc9a
|
[overlap-spec] Make plan stream an option (#11724)
|
2025-10-17 15:48:57 +08:00 |
|
|
Liangsheng Yin
|
ce11dd82dc
|
[CI] Try fix broken event loop init (#11746)
|
2025-10-17 13:30:17 +08:00 |
|
|
Liangsheng Yin
|
8221f9ae8b
|
Tiny cleanup some eagle unused codes (#11660)
|
2025-10-15 17:24:08 +08:00 |
|
|
Liangsheng Yin
|
5ea96ac7cc
|
Reduce one step decode for draft model. (#11561)
|
2025-10-14 23:52:04 +08:00 |
|
|
Liangsheng Yin
|
5a33c3aae7
|
Optimize Triton Draft Backend (#11556)
|
2025-10-14 20:08:32 +08:00 |
|
|
Liangsheng Yin
|
acc2327bbd
|
Move deep gemm related arguments to sglang.srt.environ (#11547)
|
2025-10-14 00:34:35 +08:00 |
|
|
Liangsheng Yin
|
bfadb5ea5f
|
Adjust overlap event loop (#11507)
|
2025-10-14 00:33:19 +08:00 |
|
|
Liangsheng Yin
|
54a46a264d
|
Remove tp_worker.worker (#11548)
|
2025-10-13 22:38:48 +08:00 |
|
|
Liangsheng Yin
|
516738b096
|
Depreate global_server_args_dict (#11528)
|
2025-10-13 19:34:43 +08:00 |
|
|
Liangsheng Yin
|
1083e7e3df
|
Deprecate global_server_args_dict (#11331)
|
2025-10-13 01:20:47 +08:00 |
|
|
Liangsheng Yin
|
2157d12ae8
|
[CI] fix lint (#11509)
|
2025-10-13 01:07:21 +08:00 |
|
|
Liangsheng Yin
|
f49419061d
|
Move args from global_config to environ (#11332)
|
2025-10-12 21:29:31 +08:00 |
|
|
Liangsheng Yin
|
01e59e8247
|
Fix CI break by express-laned PRs. (#11499)
|
2025-10-12 21:06:06 +08:00 |
|
|
|
20a6c0a63d
|
Beta spec-overlap for EAGLE (#11398)
Co-authored-by: Lianmin Zheng <15100009+merrymercy@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
|
2025-10-12 11:02:22 +08:00 |
|
|
Liangsheng Yin
|
739daa63e4
|
Adjust logits metada init for target verify (#11467)
|
2025-10-11 21:17:04 +08:00 |
|
|
Liangsheng YinandShangming Cai
|
c882b5ae75
|
[CI] improve disaggregation CI. (#11264)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2025-10-08 21:40:56 +08:00 |
|
|
Liangsheng Yin
|
4b4dc132fa
|
Rename ngram_utils -> ngram_info (#11316)
|
2025-10-08 11:49:46 +08:00 |
|
|
Liangsheng Yin
|
3ddd7dc9f8
|
Introduce future indices (#11301)
|
2025-10-07 22:24:02 +08:00 |
|
|
Liangsheng Yin
|
501dfa6b42
|
Remove sampling info events and overlap thread file (#11300)
|
2025-10-07 21:34:25 +08:00 |
|
|
|
1519a89cfd
|
Remove overlap thread (#11210)
Co-authored-by: Lianmin Zheng <15100009+merrymercy@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
|
2025-10-07 20:12:12 +08:00 |
|
|
Liangsheng Yinandgemini-code-assist[bot]
|
8a8a608af9
|
[ci] fix pp test (#11294)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-10-07 14:20:04 +08:00 |
|
|
Liangsheng Yin
|
4cb5a5235e
|
Tiny skip_sample adjust (#11225)
|
2025-10-05 23:41:04 +08:00 |
|
|
Liangsheng Yin
|
4726c9197f
|
[minor] fix the lint (#11198)
|
2025-10-04 01:04:58 +08:00 |
|
|
Liangsheng Yin
|
04b86b3c5c
|
[hot-fix] Fix CI break which caused by adding thinking_mode in eval (#11192)
|
2025-10-03 18:29:27 +08:00 |
|
|
Liangsheng Yin
|
de89ef49da
|
[CI]] Tee server logs to both file and stdout/stderr using PIPE (#11185)
|
2025-10-03 12:31:13 +08:00 |
|
|
Liangsheng Yin
|
3c699772c9
|
Introduce naming convention in io_struct and base sglang io classes. (#10133)
|
2025-10-03 10:55:13 +08:00 |
|
|
Liangsheng Yin
|
7ff740a6ce
|
Remove dp balance metadata and minimul token balance. (#11170)
|
2025-10-03 01:48:15 +08:00 |
|
|
Liangsheng Yin
|
bfcd9b2433
|
[grpc] style fix for grpc compilation. (#11175)
|
2025-10-03 01:44:29 +08:00 |
|
|
Liangsheng Yin
|
458611de77
|
Unify forward output datastructure (#11124)
|
2025-10-03 00:28:57 +08:00 |
|
|
Liangsheng Yin
|
7d00479950
|
Clean up ascend allocator (#11152)
|
2025-10-02 20:34:26 +08:00 |
|
|
Liangsheng Yin
|
6a29003410
|
Remove unused pack .item() in paged allocator. (#11156)
|
2025-10-02 18:01:21 +08:00 |
|
|
Liangsheng Yin
|
25e7dbe8af
|
Fix ngram spec with page size > 1 (#11135)
|
2025-10-02 12:34:23 +08:00 |
|
|
Liangsheng Yin
|
73d4a5f879
|
Organize spec-related data structures (#10735)
|
2025-10-01 09:45:30 +08:00 |
|
|
Liangsheng Yin
|
9710f718fb
|
[Eval] Add --repeat in run_eval (#11101)
|
2025-09-30 23:35:54 +08:00 |
|
|
Liangsheng Yin
|
4a762041d7
|
move environ into sglang.srt to avoid break SRT auto sync. (#10791)
|
2025-09-23 02:04:20 -07:00 |
|
|
Liangsheng Yin
|
f5a2faf2b8
|
Introduce FutureMap (#10715)
|
2025-09-22 23:27:30 -07:00 |
|
|
Liangsheng Yin
|
632b7d8cc9
|
Use simulate acc len from sglang.environ (#10771)
|
2025-09-23 12:59:50 +08:00 |
|
|
Liangsheng Yin
|
5ccf0b03bd
|
[bench] Fix random seed in bench_one_batch_server (#10548)
|
2025-09-17 19:30:32 +08:00 |
|
|
Liangsheng Yin
|
fa5d0bf6a5
|
Remove wrong imports from sglang.python (#10493)
|
2025-09-15 22:12:21 -07:00 |
|
|
Liangsheng Yin
|
c3c26f76b3
|
[Env] minimal version for organizing envs (#10479)
|
2025-09-16 03:51:25 +08:00 |
|
|
Liangsheng Yin
|
2cf811a9da
|
Fix --dataset-path in bench_one_batch_server (#10475)
|
2025-09-16 02:55:02 +08:00 |
|
|
Liangsheng Yin
|
305c9e8c2d
|
[4/N]DP refactor: support watching mode get_load and shortest queue strategy (#10201)
|
2025-09-15 10:06:08 +08:00 |
|
|
Liangsheng Yin
|
55a6e644b0
|
[Hack] Add pd-disaggregation decode polling interval (#10411)
|
2025-09-14 10:18:23 +08:00 |
|
|
Liangsheng Yin
|
6897e06b69
|
Remove repeatedly lists adding in init_incremental_detokenization (#10412)
|
2025-09-14 10:05:52 +08:00 |
|
|
Liangsheng Yin
|
83d55ac51f
|
[1/N]DP refactor: Improve dp rank scheduling in PD disaggregation mode. (#10169)
|
2025-09-09 12:27:55 +08:00 |
|
|
Liangsheng Yin
|
2c2b19b18b
|
[CI] fix ambiguous argument in testing hybrid attentions. (#10161)
|
2025-09-08 18:16:52 +08:00 |
|
|
Liangsheng YinandHuang Long
|
72f9fc5f11
|
Monkey patch uvicorn multi worker is_alive timeout (#10159)
Co-authored-by: Huang Long <121648372+llll114@users.noreply.github.com>
|
2025-09-08 17:43:23 +08:00 |
|
|
Liangsheng Yin
|
78f139812a
|
[1/N] DP-Refactor: move communicators into tokenizer_communicator_mixin (#10028)
|
2025-09-08 16:27:37 +08:00 |
|
|
Liangsheng Yin
|
e719bb0e84
|
[1/2] Refactor multi-tokenizer manager (#10074)
|
2025-09-07 19:13:34 +08:00 |
|
|
Liangsheng Yin
|
6e95f5e5bd
|
Simplify Router arguments passing and build it in docker image (#9964)
|
2025-09-05 12:13:55 +08:00 |
|
|
Liangsheng Yin
|
27e8ffed37
|
[1/N] DP-refactor: move dp balance code into scheduler's mixin class (#10004)
|
2025-09-04 16:53:58 +08:00 |
|
|
Liangsheng Yinandgemini-code-assist[bot]
|
5dfcd6c207
|
add proctitle for tokenizers (#9952)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-09-03 13:31:38 +08:00 |
|
|
Liangsheng Yin
|
6d3c20cf5b
|
fix set_interal_state API (#9850)
|
2025-09-01 01:31:35 +08:00 |
|
|
Liangsheng YinandLianmin Zheng
|
836873b99f
|
Fix memory leak when aborting decode request in PD-Disagg (#9817)
Co-authored-by: Lianmin Zheng <15100009+merrymercy@users.noreply.github.com>
|
2025-08-30 14:36:03 +08:00 |
|
|
Liangsheng Yinandgemini-code-assist[bot]
|
a23c30205d
|
Raise error when topk>1 and page>1 for paged attention backends. (#9784)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-08-29 12:47:34 +08:00 |
|
|
Liangsheng Yin
|
d0934a5192
|
gpt-oss blog reproduction document (#9728)
|
2025-08-28 10:15:08 +08:00 |
|
|
Liangsheng Yinandgemini-code-assist[bot]
|
0ff7241995
|
Improve bench_one_batch_server script (#9608)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-08-26 10:38:37 +08:00 |
|
|
Liangsheng Yin
|
9b5f0f64f5
|
Fix tiny misalign with previous truncation setting in tokenizer_manager (#9430)
|
2025-08-21 14:05:35 +08:00 |
|
|
Liangsheng Yin
|
eb19ccadae
|
[bug] fix errors related to context length in SD (#9388)
|
2025-08-21 10:32:34 +08:00 |
|
|
Liangsheng Yinandgemini-code-assist[bot]
|
08ebdf79d0
|
Fix the --allow-auto-truncate argument in tokenizer manager. (#9391)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-08-20 16:56:47 +08:00 |
|
|
Liangsheng Yin
|
4d98e48649
|
Revert "[Misc] feat: Deepgemm update for sgl-kernel (#8790)" to fix kernel CI (#9260)
|
2025-08-17 22:59:50 +08:00 |
|
|
Liangsheng Yin
|
0c8594e67d
|
Optional extension for green context (#9231)
|
2025-08-15 21:33:52 +08:00 |
|
|
Liangsheng Yin
|
445f9dca6e
|
Runtime check CUDA driver version to avoid unresolved green context symbols (#9021)
|
2025-08-12 09:26:10 -07:00 |
|
|
Liangsheng Yin
|
f9afa7dceb
|
Fix docs for clip max new tokens (#9082)
|
2025-08-11 13:15:21 -07:00 |
|
|
Liangsheng Yin
|
7a27e798ca
|
[CI] Do not trigger pd-disaggregation CI in draft PR (#8737)
|
2025-08-04 05:12:20 +08:00 |
|
|
Liangsheng Yin
|
9f47d686e5
|
Fix fused MoE when routed_scaling_factor is None (#8709)
|
2025-08-03 12:42:01 +08:00 |
|
|
Liangsheng Yin
|
603f5ce020
|
[Bug] fix green context's incompatibility with cuda < 12.4 (#8701)
|
2025-08-02 15:23:11 -07:00 |
|
|
Liangsheng Yin
|
f9f0138f80
|
Revert "[1/2] sgl-kernel: Fuse routed scaling factor into select_experts" (#8706)
|
2025-08-02 20:14:30 +08:00 |
|
|
Liangsheng Yin
|
e7dc163f57
|
add SVG logo (#8603)
|
2025-07-31 15:56:26 +08:00 |
|
|
Liangsheng Yin
|
76139bfba0
|
update mooncake in dockerfile (#7480)
|
2025-06-24 02:29:30 +08:00 |
|
|
Liangsheng Yin
|
25549433e8
|
Fix prefill OOM due to wrong token calculation when page > 1 (#7397)
|
2025-06-24 02:12:29 +08:00 |
|
|
Liangsheng Yin
|
05c9bc8956
|
[minor] simplify the TokenToKVPoolAllocator (#7414)
|
2025-06-22 12:37:18 +08:00 |
|
|
Liangsheng Yin
|
5ea5d22170
|
Fix CPU offloading for MLA memory pool (#7409)
|
2025-06-22 02:39:05 +08:00 |
|
|
Liangsheng Yin
|
c494386728
|
minor fix (#7245)
|
2025-06-16 23:30:26 +08:00 |
|
|
Liangsheng Yin
|
930746d93c
|
Improve log status (#7115)
|
2025-06-12 14:38:24 +08:00 |
|
|
Liangsheng Yinandfzyzcjy
|
78689d3393
|
PD Rust LB (PO2) (#6437)
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
|
2025-05-29 20:50:10 +08:00 |
|
|
Liangsheng Yin
|
17d080b7ae
|
Remove Cargo.lock, add it into .gitignore (#6438)
|
2025-05-20 12:01:32 +08:00 |
|
|
Liangsheng Yin
|
a3e4e9bf9e
|
Better PD initialization (#5751)
|
2025-05-07 01:12:57 +08:00 |
|
|
Liangsheng Yin
|
6d4d3bc81d
|
Fix not "import os" (#6057)
|
2025-05-06 22:06:41 +08:00 |
|
|
Liangsheng Yin
|
40d9b8acce
|
Improve overlap scheduling (#5788)
|
2025-04-28 11:19:16 +08:00 |
|
|
Liangsheng Yin
|
beb65c7433
|
[PD]Reduce kv transfer threads (#5791)
|
2025-04-27 23:03:30 +08:00 |
|
|
Liangsheng Yin
|
c55550cbf0
|
[PD] Better logs (#5715)
|
2025-04-25 17:25:45 +08:00 |
|
|
Liangsheng Yin
|
e69a219074
|
Enhance GPU memory settings (#5604)
|
2025-04-21 15:15:00 -07:00 |
|
|
Liangsheng Yin
|
33b16ad178
|
Distinguish bootstrap key only in decode server (#5422)
|
2025-04-15 20:59:28 +08:00 |
|
|
Liangsheng Yin
|
44afde82d7
|
Fix PD disaggregation bugs (#5326)
|
2025-04-14 19:27:30 +08:00 |
|
|
Liangsheng Yin
|
f765579046
|
Fix typo: infight -> inflight (#5357)
|
2025-04-14 01:25:30 +08:00 |
|
|
Liangsheng Yin
|
8616357a97
|
Fix deepseek awq v3 (#3450)
|
2025-02-12 22:09:52 +08:00 |
|
|
Liangsheng Yin
|
e7ebecf82e
|
Fix cache hit rate when chunked prefill (#2555)
|
2024-12-26 03:14:28 -08:00 |
|
|
Liangsheng Yin
|
5f12f0e7af
|
Fix chunked prefill when ignore eos (#2290)
|
2024-12-01 00:37:53 -08:00 |
|
|
Liangsheng YinandLianmin Zheng
|
b9fd178f1b
|
Fix retraction + overlap (#1860)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2024-10-31 18:27:42 -07:00 |
|
|
Liangsheng Yin
|
07bf2e846a
|
Allow consecutive ports when launching multiple sglang servers. (#1802)
|
2024-10-26 06:43:24 +00:00 |
|
|
Liangsheng Yin
|
a628dd8e31
|
Set ZMQ buffer size heuristic (#1801)
|
2024-10-25 23:15:56 -07:00 |
|
|
Liangsheng Yin
|
1e8903414a
|
Fix possible ZMQ hanging (#1800)
|
2024-10-25 23:07:07 -07:00 |
|
|
Liangsheng Yin
|
a2f5e7555f
|
Fix memory leak when doing chunked prefill (#1787)
|
2024-10-25 08:01:17 -07:00 |
|
|
Liangsheng Yin
|
3f5ac88d02
|
Fix out of memory message. (#1771)
|
2024-10-23 15:20:39 -07:00 |
|
|
Liangsheng Yin
|
5e1558f1f2
|
Update max_req_len and max_req_input_len (#1748)
|
2024-10-21 16:12:04 -07:00 |
|
|
Liangsheng Yin
|
94cde10920
|
Llama3.2 vision model support (#1551)
|
2024-10-21 15:01:21 -07:00 |
|
|
Liangsheng Yin
|
efb099cdee
|
Fix prefill oom (#1743)
|
2024-10-21 03:54:35 -07:00 |
|
|
Liangsheng Yin
|
9e0dac1ad7
|
Fix regex and logprob conflicts when chunked prefilling (#1703)
|
2024-10-17 18:33:21 -07:00 |
|