 ybyang
|
9d4be40124
|
Fix DSpark + DeepSeek V4 prefill CP compatibility (#33865)
|
2026-08-10 23:26:29 -07:00 |
|
|
|
a76a167812
|
Fix/hisparse host backed max request length (#28753)
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
|
2026-08-10 16:58:13 +08:00 |
|
|
ybyang
|
63c4996fef
|
fix: populate batch req rids and per-request http_worker_ipc for mult… (#29882)
|
2026-07-04 12:09:41 -07:00 |
|
|
ybyang
|
349a6af6b8
|
[HiCache] Fix hicache host memory leak by bounding PP-sync work_list (#28916)
|
2026-06-23 16:39:03 +08:00 |
|
|
ybyang
|
50993554d8
|
fix(health): make health-check rid unique across tokenizer workers (#28143)
|
2026-06-14 00:01:52 -07:00 |
|
|
ybyang
|
1e71c1a859
|
fix(pd): disable overlap for spec+grammar in disagg decode loop (#28039)
|
2026-06-12 14:08:36 -07:00 |
|
|
ybyang
|
8077fb1df7
|
fix(deepgemm): align PP-parallel warmup bs to CP padding (#27922)
|
2026-06-11 20:52:45 +08:00 |
|
|
ybyangandLianmin Zheng
|
687baf9471
|
fix(load-snapshot): avoid duplicate zmq bind in multi-tokenizer mode (#27145)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2026-06-03 18:24:56 -07:00 |
|
|
ybyang
|
f790674ad8
|
fix(moe): avoid unpacking None from masked deep_gemm without overlap when sbo enabled (#26839)
|
2026-06-03 00:44:28 -07:00 |
|
|
ybyang
|
9fe8b72912
|
Speed up DeepGEMM JIT warmup with per-PP-rank parallel compile (#26567)
|
2026-06-01 19:51:27 -07:00 |
|
|
ybyang
|
1eadb7a173
|
Fix multi-tokenizer batch request output routing (health stuck at 503) (#26831)
|
2026-05-31 00:31:23 -07:00 |
|
|
ybyang
|
610f55040d
|
update npu codeowners (#26042)
|
2026-05-21 20:46:59 -07:00 |
|
|
ybyang
|
b5b9c809e1
|
fix(model-gateway): rustfmt nightly in conversations/handlers.rs (#25947)
|
2026-05-20 23:29:51 -07:00 |
|
|
ybyang
|
a8c82c652e
|
fix(dsv4): make pool configurator PP-aware (#25750)
|
2026-05-20 10:58:53 +08:00 |
|
|
ybyang
|
ca29c2b0e7
|
fix(dsv4): drop stale pp_size=1 guard for V4 PD disaggregation (#25771)
|
2026-05-20 10:57:23 +08:00 |
|
|
ybyang
|
8322fe09a7
|
fix(dsv4): upgrade forward metadata on main stream for large PP size (#25729)
|
2026-05-19 20:52:00 +00:00 |
|
|
ybyang
|
d1acd62d29
|
fix(disagg): unstuck decode aborts under prealloc pressure (#25561)
|
2026-05-18 22:57:22 +08:00 |
|
|
ybyang
|
0f50ed86c9
|
fix(pd): fix kv pools without end_layer (#25476)
|
2026-05-16 19:49:26 +08:00 |
|
|
|
162540e0a8
|
feat: add Pipeline Parallelism (PP) and PD support for DeepSeek-V4 (#24704)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: xuyongfei <xuyongfei.xyf@antgroup.com>
|
2026-05-15 22:54:32 -07:00 |
|
|
|
3da87902d7
|
[HiSparse] Support FP8 KV cache by routing to flashmla_kv backend (#23013)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
|
2026-05-06 03:18:30 +00:00 |
|
|
ybyang
|
fa8993111d
|
Fix: Add token heuristic increment in total_tokens load balancing (#22614)
|
2026-04-21 01:29:33 -07:00 |
|
|
ybyang
|
271c177443
|
[NPU]chore(docker): use editable install for sglang in npu.Dockerfile (#23040)
|
2026-04-17 17:08:39 +08:00 |
|
|
ybyang
|
41258f874d
|
[PD]feat(bench): add --fake-prefill flag for decode-only stress testing (#22973)
|
2026-04-16 13:57:55 -07:00 |
|
|
ybyang
|
03fef357a6
|
fix(loads): switch get_loads_communicator to watching mode (#22919)
|
2026-04-16 02:12:22 -07:00 |
|
|
ybyang
|
fbd6dc3565
|
fix: normalize tool message content for GLM5.1 chat template (#22595)
|
2026-04-16 16:48:38 +08:00 |
|
|
ybyang
|
9e84f53785
|
[PD] Add a fallback to bypass rust dep for mini_lb (#21982)
|
2026-04-15 22:34:36 +08:00 |
|
|
ybyang
|
289cbcf482
|
fix: support PP2+CP8+TP8 (PP with context parallelism) (#19548)
|
2026-03-16 16:51:47 +00:00 |
|
|
ybyang
|
3e8abc71ca
|
[Disagg] Skip health check enqueue when PD disagg queues have backlog (#20191)
|
2026-03-09 12:58:10 -07:00 |
|
|
ybyang
|
ac1f07487a
|
Fix triton alloc extend kernel (#19780)
|
2026-03-03 21:01:16 -08:00 |
|
|
ybyang
|
c8aa2a6534
|
Fix dsv32 encode_messages (#18126)
|
2026-02-14 16:44:13 +08:00 |
|
|
ybyangandLiangsheng Yin
|
5ab76ff220
|
Special logic for healthcheck (#17734)
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
|
2026-01-26 10:26:40 -08:00 |
|
|
ybyang
|
ebca5879a1
|
Fix v32 continue_final_message not work (#16567)
|
2026-01-19 21:45:39 +08:00 |
|
|
ybyang
|
2122fea3c4
|
Update deepseekV32 Cp doc (#17054)
|
2026-01-14 11:19:26 +08:00 |
|
|
ybyang
|
aab640c99f
|
add doc for dsv32 cp+pp (#16916)
|
2026-01-12 19:14:07 +08:00 |
|
|
ybyang
|
503c3d9566
|
[model-gateway]: add qwen coder tool parser support xml format for qwen3 coder and microthinker (#12909)
|
2026-01-11 16:09:41 -08:00 |
|
|
ybyang
|
2babf88f24
|
fix spec qwen3 pd error (#16708)
|
2026-01-09 19:20:17 +08:00 |
|
|
ybyang
|
766476f52a
|
[SMG-GO] implement a Go SGLang Model Gateway - OpenAI Compatible API Server (#14770)
|
2025-12-10 06:03:28 -08:00 |
|
|
ybyang
|
3b18fd4cf5
|
[router] bindings for go (#13384)
Signed-off-by: ybyang <ybyang7@iflytek.com>
|
2025-11-16 10:12:10 -08:00 |
|
|
ybyang
|
756ad9ceb1
|
Reduce docker image size. mount cache when use pip/cargo build (#12238)
Signed-off-by: ybyang <ybyang7@iflytek.com>
|
2025-11-01 14:58:04 -07:00 |
|
|
ybyang
|
9c6e25d2a6
|
doc for logit_bias (#12188)
|
2025-10-28 10:32:12 -07:00 |
|
|
ybyang
|
e15b63a182
|
[Fix] fix missing ipc_name of __getitem__ in some IO structs (#12053)
Signed-off-by: ybyang <ybyang7@iflytek.com>
|
2025-10-24 16:59:14 +08:00 |
|
|
ybyangandYorkSu
|
dbb16bedd5
|
Support Thinking Budget (via custom_logit_processor for OpenAI API) [Fix #6572] (#11416)
Signed-off-by: ybyang <ybyang7@iflytek.com>
Co-authored-by: YorkSu <york_su@qq.com>
|
2025-10-21 16:27:56 +08:00 |
|
|
ybyang
|
b5e14b2b78
|
[1/2][feature] support openai like classification api (#11618)
|
2025-10-18 19:32:48 -07:00 |
|
|
ybyang
|
d513ee93ef
|
[2/2] [feature] support openai like classification api in router (#11670)
|
2025-10-18 19:31:08 -07:00 |
|
|
|
5061b8fd3e
|
fix stop when stream (#11462)
Signed-off-by: ybyang <ybyang7@iflytek.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
|
2025-10-11 22:06:31 +08:00 |
|
|
ybyang
|
0abb41c70d
|
adjust import setuptools_rust (#10524)
|
2025-09-16 11:01:58 -04:00 |
|
|
ybyang
|
6d40308905
|
Revert add mainprocess's proctitle (#10351)
|
2025-09-12 16:48:30 +08:00 |
|
|
|
5f77e1292d
|
Support Multi Process Tokenizer Manager(#6555) (#8964)
Signed-off-by: ybyang <ybyang7@iflytek.com>
Signed-off-by: huanglong <huanglong@linux.alibaba.com>
Co-authored-by: Huang Long <121648372+LLLL114@users.noreply.github.com>
Co-authored-by: huanglong <huanglong@linux.alibaba.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2025-09-01 01:00:13 -07:00 |
|
|
ybyang
|
fd18995cf3
|
Fix get_ip when no external network (#9700)
|
2025-08-27 10:28:52 -07:00 |
|
|
|
7490e3f67d
|
Support Multi Process Tokenizer Manager (#6555)
Signed-off-by: ybyang <ybyang7@iflytek.com>
Signed-off-by: huanglong <huanglong@linux.alibaba.com>
Co-authored-by: lw9527 <952799980@qq.com>
Co-authored-by: huanglong <huanglong@linux.alibaba.com>
Co-authored-by: Huang Long <121648372+LLLL114@users.noreply.github.com>
|
2025-08-08 01:45:50 -07:00 |
|
|
|
6f9baf1002
|
[Improvements] Merge health check route (#8444)
Signed-off-by: ybyang <ybyang7@iflytek.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Kan Wu <wukanustc@gmail.com>
|
2025-08-03 01:59:06 -07:00 |
|
|
ybyang
|
4540a4666a
|
[Feature] Simple Improve Health Check Mechanism for Production-Grade Stability (#8115)
Signed-off-by: ybyang <ybyang7@iflytek.com>
|
2025-07-19 18:10:00 -07:00 |
|
|
ybyang
|
4986104618
|
Bump xgrammar's version to 0.1.20 (#7866)
|
2025-07-08 17:55:30 -07:00 |
|
|
ybyang
|
7349717e4b
|
[doc] update lws doc for pd (#7318)
|
2025-07-01 10:39:04 +08:00 |
|
|
ybyang
|
03c039c48e
|
[OAI] patch origin request_id logic (#7508)
|
2025-06-24 20:09:38 -07:00 |
|
|
ybyangandYineng Zhang
|
906dbc34f1
|
[Docker] optimize dockerfile remove deepep and blackwell merge it to… (#7343)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-06-19 17:42:40 -07:00 |
|
|
ybyangandzhyncs
|
712bf9ec9b
|
[pd] optimize dockerfile for pd disaggregation (#7319)
Co-authored-by: zhyncs <me@zhyncs.com>
|
2025-06-18 11:26:41 -07:00 |
|
|
ybyangandShangming Cai
|
6b231325b9
|
[PD Perf] replace Queue to FastQueue (#6649)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-28 01:37:51 -07:00 |
|
|
ybyang
|
c250939ecb
|
[Fix Chat API] add request id for chat/completion for tracing (#6364)
|
2025-05-17 18:58:22 -07:00 |
|
|
ybyang
|
cc4a80caf6
|
[PD] Fix Assertion failed: /DeepEP/csrc/kernels/internode.cu:483, condition: ibgda_get_state()->num_rc_per_pe >= num_channels #134 (#5830)
|
2025-04-29 19:38:54 -07:00 |
|
|
ybyang
|
74cb12a878
|
[config] qwen3moe_tune_h20 fp8 tp4 (#5846)
|
2025-04-28 10:21:06 -07:00 |
|
|
ybyang
|
c6c6264073
|
[PD] support pd fake transfer for warmup (#5726)
|
2025-04-29 00:33:20 +08:00 |
|
|
ybyang
|
59dd090f1c
|
[PD] Fix no cache connect for recevier (#5534)
|
2025-04-19 14:55:28 +08:00 |
|
|
ybyang
|
dd83e7e9c3
|
[Bug fix] need record start time in pd mode (#5425)
|
2025-04-16 10:11:16 +08:00 |
|
|
ybyang
|
e9fc2ac7b6
|
[PD Bug] fix MLA get_contiguous_buf_infos error (#5384)
|
2025-04-14 22:56:39 +08:00 |
|
|
ybyang
|
194eea1774
|
[doc] update sponsorship (#3903)
|
2025-02-26 16:28:15 -08:00 |
|
|
ybyang
|
c51dc2cc8d
|
Docs: Deploy multi-node inference (LWS method) using sglang in a K8s cluster (#3624)
|
2025-02-17 18:14:20 -08:00 |
|