 Baizhou Zhang
|
bb6a21cd99
|
[Fix]Tiny fix in Dockerfile (#12748)
|
2025-11-05 21:04:33 -08:00 |
|
|
Baizhou Zhang
|
3c0a6df82d
|
[chore] Fix triton installation for cu13 image (#12742)
|
2025-11-05 20:20:18 -08:00 |
|
|
Baizhou Zhang
|
9a954982de
|
[chore] SGLang tag management in Dockerfile (#12734)
|
2025-11-05 19:03:26 -08:00 |
|
|
Baizhou Zhang
|
9ec6031d70
|
[chore]Remove dockerfile from target file of bump kernel version (#12728)
|
2025-11-05 18:01:19 -08:00 |
|
|
Baizhou Zhang
|
7c45b8b4bb
|
[CI] Fix qwen3-vl lora nightly ci (#12708)
|
2025-11-05 11:00:13 -08:00 |
|
|
Baizhou Zhang
|
d22d044734
|
Revert "Enable memory saver for hybrid model" (#12648)
|
2025-11-04 16:22:06 -08:00 |
|
|
Baizhou Zhang
|
42889acbd0
|
[hotfix] Fix deepep w4a8 bug (#12642)
|
2025-11-04 13:55:59 -08:00 |
|
|
Baizhou Zhang
|
15efbcb4e7
|
[chore] Fix update_kernel_whl_index script for multiple cuda version (#12519)
|
2025-11-03 16:34:14 -08:00 |
|
|
Baizhou Zhang
|
6e29446e45
|
[hotfix] Remove flashinfer-jit-cache from pyproject (#12530)
|
2025-11-02 22:11:05 -08:00 |
|
|
Baizhou Zhang
|
9a512cf95b
|
[CI] Move some Lora/Deterministic CI tests to nightly (#12507)
|
2025-11-01 19:54:22 -07:00 |
|
|
Baizhou Zhang
|
2b7bf11bd2
|
[Hotfix] Remove extra comment in sgl-kernel README (#12500)
|
2025-11-01 12:22:45 -07:00 |
|
|
Baizhou Zhang
|
566ade0388
|
[CI] Build aarch64 kernels for sgl-kernel test (#12480)
|
2025-11-01 11:55:42 -07:00 |
|
|
Baizhou Zhang
|
5f98b7fe61
|
[CI] Fix kernel installation on aarch runners (#12475)
|
2025-10-31 14:25:27 -07:00 |
|
|
Baizhou Zhang
|
57cc5385c0
|
[CI] Add more bins for 1-gpu CI test (#12422)
|
2025-10-31 00:05:01 -07:00 |
|
|
Baizhou Zhang
|
b7fdde4bb4
|
[ci] Fix ci_install_deepep (#12375)
|
2025-10-30 11:39:14 -07:00 |
|
|
Baizhou Zhang
|
621dfb8886
|
Import flash_mla from sgl-kernel (#12135)
|
2025-10-29 23:54:21 -07:00 |
|
|
Baizhou Zhang
|
685c06451f
|
[ci] Try fixing broken CIs (#12317)
|
2025-10-29 01:13:51 -07:00 |
|
|
Baizhou Zhang
|
587deb15a7
|
[hotfix] Fix pytest not found in CI (#12311)
|
2025-10-29 11:07:36 +08:00 |
|
|
Baizhou Zhang
|
75c09e1ffe
|
[Fix] Fix cu130 sgl-kernel wheel renaming (#12173)
|
2025-10-26 22:44:05 -07:00 |
|
|
Baizhou Zhang
|
97828878d8
|
[Doc] Small update of DeepSeek v3.2 document (#12138)
|
2025-10-25 20:34:05 -07:00 |
|
|
Baizhou Zhang
|
bcecf27e7c
|
[Doc] Fix format for deepseek v3.2 document (#12130)
|
2025-10-25 15:07:50 -07:00 |
|
|
Baizhou Zhang
|
4b0ac1d52a
|
Update sgl-kernel version to 0.3.16.post4 (#12125)
|
2025-10-25 14:33:33 -07:00 |
|
|
Baizhou Zhang
|
8e987fa2a3
|
Update document index for DeepSeek-v32 docs (#12101)
|
2025-10-25 13:38:58 -07:00 |
|
|
Baizhou Zhang
|
ce86979355
|
[Fix] Set global args in cpu test (#12105)
|
2025-10-24 21:46:17 -07:00 |
|
|
|
729b242934
|
[Doc] Add documentation for DeepSeek V3.2 (#11877)
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: ybyang <ybyang7@iflytek.com>
|
2025-10-24 19:06:22 -07:00 |
|
|
Baizhou Zhang
|
4ef981e2b6
|
Revert "[Fix] Fix lint to pass CI" (#12042)
|
2025-10-23 19:44:58 -07:00 |
|
|
Baizhou Zhang
|
69ed8b67a8
|
[Fix] Fix lint to pass CI (#12037)
|
2025-10-23 19:39:38 -07:00 |
|
|
Baizhou Zhangandgemini-code-assist[bot]
|
983ef22cf3
|
[Doc] Update deterministic inference flag in server_arguments.md (#11978)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-10-22 14:12:15 -07:00 |
|
|
Baizhou Zhang
|
ef4a8097b8
|
Rename flashmla kernel options of nsa backend for better readability (#11876)
|
2025-10-21 13:14:16 -07:00 |
|
|
Baizhou Zhang
|
ebff4ee648
|
Update sgl-kernel and remove fast hadamard depedency (#11844)
|
2025-10-21 13:13:54 -07:00 |
|
|
Baizhou Zhang
|
44f0ece9fc
|
[Doc] Update documents for FA4 (#11778)
|
2025-10-19 17:40:38 -07:00 |
|
|
Baizhou Zhang
|
cbb5fc2edc
|
[CI] Add CI test for DeepSeek V3.2 MTP (#11835)
|
2025-10-19 17:00:25 -07:00 |
|
|
Baizhou Zhang
|
20b8d2306c
|
Cleaning indexer for DeepSeek V3.2 (#11682)
|
2025-10-17 13:47:21 -07:00 |
|
|
Baizhou Zhang
|
b0d1d717e1
|
Revert "make radix cache deterministic" (#11728)
|
2025-10-16 14:36:15 -07:00 |
|
|
Baizhou Zhang
|
c224a4c6cc
|
Fix log for chunked prefix cache (#11624)
|
2025-10-14 11:49:33 -07:00 |
|
|
Baizhou Zhang
|
9f1f699a7a
|
[CI] Add Basic Test for DeepSeek V3.2 (#11308)
|
2025-10-13 11:41:02 -07:00 |
|
|
Baizhou Zhang
|
8b85926a6e
|
Remove tilelang dependency in Dockerfile (#11455)
|
2025-10-10 23:17:53 -07:00 |
|
|
Baizhou Zhang
|
292a867ad9
|
Add flashmla and fast hadamard transform to Dockerfile (#11235)
|
2025-10-05 21:31:28 -07:00 |
|
|
Baizhou ZhangandYineng Zhang
|
aa1c5cf5bd
|
Add warnings and remove dependency for deterministic inference (#10724)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-09-22 10:56:02 -07:00 |
|
|
Baizhou Zhang
|
f111649580
|
Replace os.environ in layernorm.py (#10684)
|
2025-09-20 00:20:33 -07:00 |
|
|
|
8ecef73f12
|
[1/2] Support deterministic inference with flashinfer attention backend (#10645)
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
|
2025-09-19 23:34:29 -07:00 |
|
|
Baizhou Zhangandzhyncs
|
3fa3c22ae2
|
Fix fast decode plan for flashinfer v0.4.0rc1 and upgrade sgl-kernel 0.3.11 (#10634)
Co-authored-by: zhyncs <me@zhyncs.com>
|
2025-09-19 01:25:29 -07:00 |
|
|
Baizhou Zhang
|
8ad700f735
|
Cleaning codes for speculative attention mode (#10149)
|
2025-09-08 17:38:06 -07:00 |
|
|
Baizhou Zhang
|
beac202bfd
|
Add lora_path argument to bench_multiturn.py (#10092)
|
2025-09-05 19:20:42 -07:00 |
|
|
Baizhou Zhang
|
7de2ce45b2
|
Disable radix cache in test_lora_update.py for better stability (#9852)
|
2025-08-31 22:28:22 -07:00 |
|
|
Baizhou Zhang
|
75e6a7cde1
|
Support radix cache for Lora feature (#7216)
|
2025-08-11 10:14:11 -07:00 |
|
|
Baizhou Zhang
|
f2d68ded6d
|
Rename lora_path to lora_id in batches (#8437)
|
2025-08-03 21:08:28 -07:00 |
|
|
Baizhou Zhang
|
e7e5a3050a
|
Update batch size limitation of dsv3_router_gemm kernel to 16 (#8051)
|
2025-08-01 11:53:31 +08:00 |
|
|
Baizhou Zhang
|
91e3d1542e
|
Update Cutlass in sgl-kernel to v4.1 (#8392)
|
2025-07-27 00:36:15 -07:00 |
|
|
Baizhou Zhang
|
282eb59ff3
|
Add bf16 output option for dsv3_router_gemm kernel (#7999)
|
2025-07-20 09:49:37 +08:00 |
|
|
Baizhou Zhang
|
8cddfa56a1
|
Clean warning logs for gate_proj loading in Lora (#8172)
|
2025-07-19 15:56:50 -07:00 |
|
|
Baizhou Zhang
|
88f484ce4c
|
Apply dsv3 router gemm kernel for deepseek-r1 fp4 (#7677)
|
2025-07-02 12:30:18 -07:00 |
|
|
Baizhou Zhang
|
7248272ccc
|
Add dsv3 router gemm kernel (#7627)
|
2025-06-29 23:31:55 -07:00 |
|
|
Baizhou Zhang
|
d2679f5109
|
Fix ChunkCache object has no attribute 'disable' (#7217)
|
2025-06-15 20:55:15 -07:00 |
|
|
Baizhou Zhang
|
25a6a9aa22
|
Fix circular import in test_prefix_chunk_info.py (#7097)
|
2025-06-11 10:57:45 -07:00 |
|
|
Baizhou Zhang
|
2a5f0100e0
|
Fix GGuf and add back test_gguf.py (#7067)
|
2025-06-10 21:07:20 -07:00 |
|
|
Baizhou Zhang
|
3b014bc13d
|
Fix test_lora.py CI (#7061)
|
2025-06-10 12:24:46 -07:00 |
|
|
Baizhou Zhang
|
6716b41786
|
Update default settings for blackwell (#7023)
|
2025-06-09 20:37:47 -07:00 |
|
|
Baizhou Zhang
|
a979daac3b
|
Fallback to lower triton version for unfound fused moe configs (#7013)
|
2025-06-09 15:41:03 -07:00 |
|
|
Baizhou Zhang
|
971a0dfa32
|
Extend cuda graph capture bs for B200 (#6937)
|
2025-06-08 05:13:22 -07:00 |
|
|
Baizhou Zhang
|
c4ffbeca19
|
Add triton fused moe kernel config for E=257 on B200 (#6939)
|
2025-06-06 23:15:01 -07:00 |
|
|
Baizhou Zhang
|
6a47b73024
|
Remove contiguous before Flashinfer groupwise fp8 gemm (#6804)
|
2025-06-01 18:30:54 -07:00 |
|
|
Baizhou Zhang
|
73def253b5
|
Fix mem_fraction_static for AMD CI (#6748)
|
2025-05-29 12:37:30 -07:00 |
|
|
Baizhou Zhang
|
f2bd3515fb
|
Tune memory arguments on B200 (#6718)
|
2025-05-29 00:03:22 -07:00 |
|
|
Baizhou Zhang
|
791b3bfabb
|
[Feature] Support Flashinfer fp8 blockwise GEMM kernel on Blackwell (#6479)
|
2025-05-28 16:03:43 -07:00 |
|
|
Baizhou Zhang
|
bdaefbbfbd
|
Add environment flag for disabling message queue broadcaster (#6403)
|
2025-05-26 22:32:41 -07:00 |
|
|
Baizhou Zhang
|
d4c038daed
|
[Fix]Fix capture fail bug for DeepSeek (#6275)
|
2025-05-21 11:11:20 -07:00 |
|
|
Baizhou Zhang
|
299fd22f9e
|
Fix throughput threshold for amd ci test (#6414)
|
2025-05-19 14:17:41 -07:00 |
|
|
Baizhou Zhang
|
839fb31e5f
|
[Fix] Improve dependencies for Blackwell image (#6334)
|
2025-05-16 12:38:22 -07:00 |
|
|
Baizhou Zhang
|
cfca4e0ed2
|
adding Triton configs for DeepSeekV3 FusedMoE kernel on Blackwell (#6111)
|
2025-05-07 23:39:10 -07:00 |
|
|
Baizhou Zhang
|
73600673bb
|
Clean logs for DeepSeek-V3 launching (#6079)
|
2025-05-07 18:54:50 -07:00 |
|
|
Baizhou Zhang
|
8f508cc77f
|
Update doc for MLA attention backends (#6034)
|
2025-05-07 18:51:05 -07:00 |
|
|
Baizhou Zhang
|
fee37d9e8d
|
[Doc]Fix description for dp_size argument (#6063)
|
2025-05-08 00:04:22 +08:00 |
|
|
Baizhou Zhang
|
bdd17998e6
|
[Fix] Fix and rename flashmla CI test (#6045)
|
2025-05-06 13:25:15 -07:00 |
|
|
Baizhou Zhang
|
bf203cb7a2
|
[Fix] Suppress dynamo logging when using flashinfer backend with torch compile (#5992)
|
2025-05-04 09:49:13 -07:00 |
|
|
Baizhou ZhangandYuhao Chen
|
799789afed
|
Bump Flashinfer to 0.2.5 (#5870)
Co-authored-by: Yuhao Chen <yxckeis8@gmail.com>
|
2025-04-29 19:50:57 -07:00 |
|
|
Baizhou Zhang
|
f48b007c1d
|
[Doc] Recover history of server_arguments.md (#5851)
|
2025-04-28 10:48:21 -07:00 |
|
|
Baizhou Zhang
|
84022c0e56
|
Release v0.4.6 (#5795)
|
2025-04-27 14:07:05 -07:00 |
|
|
Baizhou ZhangandLianmin Zheng
|
f9fb33efc3
|
Add 8-GPU Test for Deepseek-V3 (#5691)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2025-04-27 12:46:12 -07:00 |
|
|
Baizhou Zhang
|
a45a4b239d
|
Split local attention test from fa3 test (#5774)
|
2025-04-27 01:03:31 -07:00 |
|
|
Baizhou Zhang
|
a14654dd68
|
Fix weight loading bug for Deepseek v3+nextn (#5684)
|
2025-04-24 21:29:56 +08:00 |
|
|
Baizhou Zhang
|
3f87f83116
|
Fuse q_a_proj and kv_a_proj (#5619)
|
2025-04-22 20:35:08 -07:00 |
|
|
Baizhou Zhang
|
ce5412b62e
|
Turn on DeepGemm By Default and Update Doc (#5628)
|
2025-04-22 16:10:08 -07:00 |
|
|
Baizhou Zhang
|
5156d5a413
|
Add test config yamls for Deepseek v3 (#5433)
|
2025-04-20 17:28:52 -07:00 |
|
|
Baizhou Zhang
|
072b4d0398
|
Add document for LoRA serving (#5521)
|
2025-04-20 14:37:57 -07:00 |
|
|
Baizhou Zhang
|
b54b5a96e4
|
[Doc]Add instruction for profiling with bench_one_batch (#5581)
|
2025-04-20 14:05:36 -07:00 |
|
|
Baizhou Zhang
|
6fb29ffd9e
|
Deprecate enable-flashinfer-mla and enable-flashmla (#5480)
|
2025-04-17 01:43:33 -07:00 |
|
|
Baizhou Zhang
|
4fb05583ef
|
Deprecate disable-mla (#5481)
|
2025-04-17 01:43:14 -07:00 |
|
|
Baizhou Zhang
|
81c891111f
|
Add test for flash_attn_varlen_func kernel (#5484)
|
2025-04-17 01:42:56 -07:00 |
|
|
Baizhou Zhang
|
a42736bbb8
|
Support MHA with chunked prefix cache for DeepSeek chunked prefill (#5113)
|
2025-04-15 22:01:22 -07:00 |
|
|
Baizhou Zhang
|
f6772f1497
|
[Fix] Turn off DeepGEMM by default (#5263)
|
2025-04-14 17:45:44 -07:00 |
|
|
Baizhou Zhang
|
e4155e96d0
|
Add flash_attn_varlen_func to sgl-kernel (#5315)
|
2025-04-11 23:36:36 -07:00 |
|
|
Baizhou Zhang
|
efbae697b3
|
[Revision] Replace enable_flashinfer_mla argument with attention_backend (#5052)
|
2025-04-05 01:23:02 -07:00 |
|
|
Baizhou Zhang
|
e8999b13b7
|
Replace enable_flashinfer_mla argument with attention_backend (#5005)
|
2025-04-03 02:53:58 -07:00 |
|
|
Baizhou Zhang
|
4a63bc32b7
|
[Fix] Add torch compile for torch.clamp back (#4936)
|
2025-03-30 20:46:07 -07:00 |
|
|
Baizhou Zhang
|
42873eac09
|
[Fix] Improve Lora tests and reduce CI runtime (#4925)
|
2025-03-30 19:40:14 -07:00 |
|
|
Baizhou Zhang
|
e62d60fe6d
|
[Fix] avoid stream sync and torch compile in prefill for fa3 backend (#4932)
|
2025-03-30 13:53:44 -07:00 |
|
|
Baizhou Zhang
|
20c90be23d
|
[Feature] Support FA3 backend for MLA (#4831)
|
2025-03-28 18:30:14 -07:00 |
|
|
Baizhou Zhang
|
34e07a65f1
|
[Fix] Fix unexpected idx bug of Phi-3-small (#4728)
|
2025-03-25 21:33:48 -07:00 |
|
|
Baizhou Zhang
|
90532b7627
|
[Fix] Fix raw_bs bug when using flashinfer mla and eagle (#4557)
|
2025-03-18 21:26:53 -07:00 |
|