 Lianmin Zheng
|
6c839368e0
|
[Cleanup] IPC struct renames, better typing, and SenderWrapper removal (#29214)
|
2026-06-24 17:25:24 -07:00 |
|
|
Lianmin ZhengandYinghai Lu
|
7e63feee6f
|
Add scheduler metrics extension hooks (#29207)
Co-authored-by: Yinghai Lu <yinghai@meta.com>
|
2026-06-24 15:50:05 -07:00 |
|
|
Lianmin ZhengandRain Jiang
|
fd87a85388
|
Convert SamplingParams to msgspec Struct (#29198)
Co-authored-by: Rain Jiang <96632942+rainj-me@users.noreply.github.com>
|
2026-06-24 13:42:45 -07:00 |
|
|
Lianmin Zheng
|
d5c566e59b
|
Extract profile request cleanups (#29098)
|
2026-06-24 11:22:58 -07:00 |
|
|
Lianmin ZhengandCheng Wan
|
d6aacd2801
|
Handle input-embed-only batches in eager runner (#29121)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
|
2026-06-24 10:33:52 -07:00 |
|
|
Lianmin Zheng
|
34dd9c28ca
|
[Refactor] Introduce sock_send/sock_recv wrappers for zmq IPC (#29012)
|
2026-06-23 15:54:36 -07:00 |
|
|
Lianmin Zheng
|
ecab3f322e
|
Revert "Improve MFU metrics for prefill and verify timing" (#29079)
|
2026-06-23 15:46:22 -07:00 |
|
|
Lianmin ZhengandPranjal Shankhdhar
|
b60185c41c
|
Improve MFU metrics for prefill and verify timing (#29000)
Co-authored-by: Pranjal Shankhdhar <pranjal.ssh@gmail.com>
|
2026-06-23 12:26:56 -07:00 |
|
|
Lianmin Zheng
|
b28e990161
|
Migrate all ServerArgs fields to Annotated style, reduce add_cli_args by ~2400 lines (#28919)
|
2026-06-22 08:34:37 -07:00 |
|
|
Lianmin Zheng
|
ad9723af03
|
Clean up CUDA graph capture logs (#28937)
|
2026-06-22 06:15:26 -07:00 |
|
|
Lianmin Zheng
|
886b96621d
|
Migrate more server args to annotated style (#28830)
|
2026-06-21 20:50:17 -07:00 |
|
|
Lianmin Zhengandhnyls2002
|
a4d0ff3def
|
[misc] Make NaN-logit sanitization opt-in (default off) (#28829)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
|
2026-06-21 14:31:35 -07:00 |
|
|
Lianmin Zheng
|
7942d546d1
|
Revert "[Spec] Split init_backends; account draft weights in --mem-fraction-static" (#28841)
|
2026-06-21 07:52:35 -07:00 |
|
|
Lianmin Zheng
|
3975ea5ac7
|
Fix H20 torch import reinstall fallback (#28818)
|
2026-06-21 04:50:46 -07:00 |
|
|
Lianmin Zheng
|
c9488241e9
|
[Refactor] Auto-derive CLI args from dataclass fields to eliminate duplication (#28814)
|
2026-06-21 00:51:08 -07:00 |
|
|
Lianmin Zheng
|
d331fdd2ba
|
Add project rule: prefer msgspec.Struct over dataclasses (#28816)
|
2026-06-20 18:43:16 -07:00 |
|
|
Lianmin Zheng
|
54b9b9d0c9
|
Remove threading atexit monkey patch (#28812)
|
2026-06-20 18:30:50 -07:00 |
|
|
Lianmin Zheng
|
8a3d6c3403
|
Sort pyproject dependency lists (#28811)
|
2026-06-20 17:24:57 -07:00 |
|
|
Lianmin Zheng
|
95fb1ef697
|
[CI] Remove deprecated test/srt legacy CI setup (#28810)
|
2026-06-20 15:09:33 -07:00 |
|
|
Lianmin Zheng
|
fe428dd845
|
Clean up startup log noise (#28807)
|
2026-06-20 15:02:52 -07:00 |
|
|
Lianmin ZhengandYinghai Lu
|
45d203fb08
|
Fix tokenizer state cleanup on dispatch failure (#28694)
Co-authored-by: Yinghai Lu <yinghai@meta.com>
|
2026-06-19 21:55:39 -07:00 |
|
|
Lianmin Zheng
|
7cead0fb8f
|
Add JonnyKong to CI_PERMISSIONS.json (#28550)
|
2026-06-17 11:42:45 -07:00 |
|
|
Lianmin ZhengandIan O'Connell
|
7e629a2f8c
|
Allow overriding tokenizer path in benchmark harness (#28280)
Co-authored-by: Ian O'Connell <ianoc@meta.com>
|
2026-06-15 13:07:50 -07:00 |
|
|
Lianmin Zhengandgemini-code-assist[bot]
|
3b419f66da
|
[JIT] Track angle-bracket includes in source hash (#28273)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-06-15 02:58:14 -07:00 |
|
|
Lianmin Zheng
|
f18d38d040
|
Revert "[AMD][Quantization] Online MXFP4 quantization 2/N - FP8 to MXFP4 requantization on AMD GPUs" (#28213)
|
2026-06-14 13:34:06 -07:00 |
|
|
Lianmin Zheng
|
b001d3e815
|
Add prajjwal1 to CI_PERMISSIONS.json (#28120)
|
2026-06-12 20:26:38 -07:00 |
|
|
Lianmin Zheng
|
165331a200
|
Share BCG output buffers across capture sizes (#27659)
|
2026-06-09 20:33:46 -07:00 |
|
|
Lianmin Zheng
|
ca716f4734
|
Add TP server GPU process regression test (#27721)
|
2026-06-09 16:25:27 -07:00 |
|
|
Lianmin Zhengandlmzheng
|
eb8dceda44
|
Defer DeepGEMM PDL setup to worker init (#27671)
Co-authored-by: lmzheng <lmzheng@fb.com>
|
2026-06-09 13:52:30 -07:00 |
|
|
Lianmin Zheng
|
fca4ef9d69
|
Fix SWA pool resolution for EAGLE draft workers (#27491)
|
2026-06-08 11:00:29 -07:00 |
|
|
Lianmin Zheng
|
bcb5645629
|
Fix TRTLLM target verify query metadata (#27495)
|
2026-06-08 10:58:48 -07:00 |
|
|
Lianmin Zheng
|
a07d813ec8
|
Revert "Fix TRTLLM target verify query metadata (#27473)" (#27494)
|
2026-06-07 02:48:52 -07:00 |
|
|
Lianmin ZhengandMing Yang
|
db58e76c33
|
Add all_to_all_single to GroupCoordinator (#27492)
Co-authored-by: Ming Yang <minos.future@gmail.com>
|
2026-06-07 02:48:47 -07:00 |
|
|
Lianmin Zheng
|
5be0b0c8c0
|
Fix TRTLLM target verify query metadata (#27473)
|
2026-06-07 02:46:01 -07:00 |
|
|
|
88a7b0fd30
|
Classify malformed-multimodal rejects as invalid_request (#27451)
Co-authored-by: cctry <cctry@meta.com>
Co-authored-by: cctry <cctry@fb.com>
|
2026-06-06 10:19:22 -07:00 |
|
|
Lianmin Zheng
|
448d3afb76
|
fix(spec): complete CustomSpecAlgo duck-typing interface and guard against drift (#27300)
|
2026-06-04 15:52:46 -07:00 |
|
|
Lianmin Zheng
|
ff5c4d7b57
|
Add zx3xyy to CI_PERMISSIONS.json (#27213)
|
2026-06-03 22:12:00 -07:00 |
|
|
|
14ed9b448e
|
Add ZMQ IPv6 support, bench_serving sampling params, and reduce routed_dp_rank log noise (#27180)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: Grigory Sizov <grisha.sizov@gmail.com>
|
2026-06-03 17:49:34 -07:00 |
|
|
Lianmin Zheng
|
53b8378307
|
Fix weights_checker checksum for 0-dim tensors and multi-GPU (#26863)
|
2026-05-31 21:27:03 -07:00 |
|
|
Lianmin ZhengandJaewon
|
a779791b3f
|
Add random-ids dataset, round-robin expert simulation, and kill_process_tree logging (#26862)
Co-authored-by: Jaewon <52840625+jaewonlee-fb@users.noreply.github.com>
|
2026-05-31 20:50:29 -07:00 |
|
|
Lianmin Zhengandcctry
|
4ff1296f5e
|
Optimize get load calls (/v1/loads) using shared-memory load snapshots (#26348)
Co-authored-by: cctry <cctry@meta.com>
|
2026-05-29 13:40:26 -07:00 |
|
|
Lianmin Zheng
|
dc4e7bc479
|
Fix TRTLLM MHA draft decode cache seqlens replay (#26655)
|
2026-05-28 20:58:16 -07:00 |
|
|
Lianmin Zheng
|
93fa577bb9
|
Clean up server startup log noise (#26205)
|
2026-05-24 14:35:15 -07:00 |
|
|
Lianmin ZhengandJaewon
|
8fa56a0ab1
|
Fix FlashInfer A2A token cap sizing (#25907)
Co-authored-by: Jaewon <52840625+jaewonlee-fb@users.noreply.github.com>
|
2026-05-20 23:01:28 -07:00 |
|
|
Lianmin Zheng
|
c4a7d12092
|
Enable breakable CUDA graph for eagle (#25795)
|
2026-05-20 18:02:04 -07:00 |
|
|
Lianmin Zheng
|
61ac6792e6
|
Add DevashishLal-CB to CI_PERMISSIONS.json (#25908)
|
2026-05-20 11:19:43 -07:00 |
|
|
Lianmin Zheng
|
b45b52ee8f
|
Add spec_verify_calls_total metric for speculative decoding (#25689)
|
2026-05-18 18:35:11 -07:00 |
|
|
|
314dedf7c6
|
Use SGLANG_CACHE_DIR env for gpu_p2p_access_cache path (#25686)
Co-authored-by: Ian O'Connell <ianoc@meta.com>
Co-authored-by: ianoc <ianoc@fb.com>
|
2026-05-18 16:00:54 -07:00 |
|
|
Lianmin ZhengandHanming Lu
|
745abd6cc0
|
Add no_combine support to cutlass_moe_fp4 (#25688)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
|
2026-05-18 15:59:55 -07:00 |
|
|
|
878e6b8886
|
[SP] Fix runtime_max_tokens_per_rank for sequence parallelism (#25685)
Co-authored-by: Ming Yang <minos.future@gmail.com>
Co-authored-by: Yinghai Lu <yinghai@meta.com>
|
2026-05-18 15:59:33 -07:00 |
|
|
Lianmin Zheng
|
f21fe6ad4d
|
Fix PD disaggregation warmup: set request_name and improve error logging (#25542)
|
2026-05-18 09:49:56 -07:00 |
|
|
Lianmin ZhengandHanming Lu
|
21e420b4c2
|
[Test] Set default temperature to 0.0 in kl_test_utils (#25429)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
|
2026-05-15 18:20:10 -07:00 |
|
|
Lianmin Zhengandcctry
|
5fb6bde6c0
|
Add sglang:get_loads_duration_seconds metric (#25163)
Co-authored-by: cctry <cctry@meta.com>
|
2026-05-13 18:38:49 -07:00 |
|
|
Lianmin Zheng
|
1ae3218d03
|
Add jasonjk-park and charlotte12l to CI_PERMISSIONS.json (#25104)
|
2026-05-13 01:51:05 -07:00 |
|
|
Lianmin Zheng
|
48fc26a814
|
Fix Eagle draft decode positions (#25015)
|
2026-05-12 14:04:58 -07:00 |
|
|
Lianmin Zheng
|
34555aee9c
|
Add minosfuture to CI_PERMISSIONS.json (#25019)
|
2026-05-11 16:54:32 -07:00 |
|
|
Lianmin Zheng
|
893dfb7b74
|
Add libertyeagle to CI_PERMISSIONS.json (#25011)
|
2026-05-11 14:44:07 -07:00 |
|
|
Lianmin Zheng
|
e40e339c72
|
Filter non-int token ids in benchmark and observe decode-side bootstrap/alloc metrics (#24684)
|
2026-05-08 11:45:37 -07:00 |
|
|
Lianmin Zheng
|
3c3f0bd55e
|
Cache empty MatchResult in RadixCache (#24470)
|
2026-05-07 17:13:20 -07:00 |
|
|
Lianmin Zheng
|
b859f7ffba
|
Improve metrics, observability, and PD deploy tooling (#24521)
|
2026-05-06 11:27:35 -07:00 |
|
|
Lianmin Zheng
|
d4d4b04d66
|
[PD] Fix missing update_status call in abort() across all KV backends (#24522)
|
2026-05-06 05:30:11 -07:00 |
|
|
Lianmin Zheng
|
b91b05ae27
|
Add --random-input-len to send_one.py (#24464)
|
2026-05-05 17:49:33 -07:00 |
|
|
Lianmin Zheng
|
64f80eabbe
|
Register aten::rms_norm and aten::mm.dtype in batch invariant mode (#24459)
|
2026-05-05 17:21:34 -07:00 |
|
|
Lianmin Zheng
|
46bde1f426
|
Add fwd_occupancy metric to SchedulerStats and Prometheus collector (#24458)
|
2026-05-05 17:04:34 -07:00 |
|
|
Lianmin Zheng
|
710fed10fb
|
Revert "[fix] /pause_generation and /continue_generation wrong for --tokenizer-worker-num > 1" (#24461)
|
2026-05-05 16:44:34 -07:00 |
|
|
Lianmin Zheng
|
29dd3a36c0
|
Refactor device timer installation and rename prefill prealloc to bootstrap (#24341)
|
2026-05-04 13:57:13 -07:00 |
|
|
Lianmin Zheng
|
c7566a860b
|
Add CI permissions for user luccafong, 842974287 (#24238)
|
2026-05-01 12:50:36 -07:00 |
|
|
Lianmin Zheng
|
ece8a1a788
|
Refactor device timer, clean up metrics collector, and add fwd occupancy metric (#24197)
|
2026-05-01 10:25:25 -07:00 |
|
|
|
d9e8a4a7f8
|
[SWA] Ensure we use pre-computed SWA cache location during prefill (#24138)
Co-authored-by: Xiaozhu Meng <mxz297@gmail.com>
Co-authored-by: Yinghai Lu <yinghai@meta.com>
|
2026-05-01 00:01:49 -07:00 |
|
|
Lianmin Zheng
|
b1ef99f65f
|
[CI] Remove orphaned test/srt/ascend and test/srt/configs (#24145)
|
2026-04-30 04:43:11 -07:00 |
|
|
Lianmin Zheng
|
e60c60eff0
|
[SWA] Fix missing mamba_indices parameter in cpu copy interface (#24026)
|
2026-04-29 17:33:38 -07:00 |
|
|
Lianmin Zheng
|
2a771a40ac
|
Add engine_type label to tokenizer manager metrics (#23978)
|
2026-04-28 19:52:58 -07:00 |
|
|
Lianmin Zheng
|
d66eb3a91b
|
docs: update contribution guide with coding style guidelines (#23977)
|
2026-04-28 19:51:58 -07:00 |
|
|
Lianmin Zheng
|
14b4e6fa69
|
Support --model as alias for --model-path in CLI (#23894)
|
2026-04-28 17:23:27 -07:00 |
|
|
Lianmin Zheng
|
8536d4b402
|
Clean up noisy startup warnings from third-party deps (#23669)
|
2026-04-27 03:10:46 -07:00 |
|
|
Lianmin Zheng
|
a4facdf3f6
|
[CI] Refactor ci_install_dependency.sh into standalone functions (#23592)
|
2026-04-24 17:39:39 -07:00 |
|
|
Lianmin Zheng
|
6344b546c8
|
Deprecate --collect-tokens-histogram, auto-collect with --enable-metrics (#23595)
|
2026-04-24 12:00:16 -07:00 |
|
|
Lianmin Zheng
|
95d021b523
|
Pre-set SWA cache location in CudaGraphRunner (#23552)
|
2026-04-23 16:51:29 -07:00 |
|
|
Lianmin Zheng
|
bb962b0046
|
Fix MoE no_combine: skip router weight in down projection (#23545)
|
2026-04-23 16:47:58 -07:00 |
|
|
Lianmin Zheng
|
6a3c070ee3
|
Add 'allready' to ignore words list in .codespellrc (#23465)
|
2026-04-22 02:39:04 -07:00 |
|
|
Lianmin Zheng
|
9c47bbad13
|
Clean up bench_one_batch warning and simplify norm dispatch (#23110)
|
2026-04-17 17:42:20 -07:00 |
|
|
Lianmin Zheng
|
44e67c6835
|
Remove deprecated double sparsity feature (#23009)
|
2026-04-17 13:33:12 -07:00 |
|
|
Lianmin Zheng
|
35da90cb76
|
[misc] Configure logging before ServerArgs.__post_init__ (#22926)
|
2026-04-15 23:53:15 -07:00 |
|
|
Lianmin Zheng
|
ccff59254c
|
Update .codespellrc (#22912)
|
2026-04-15 16:25:55 -07:00 |
|
|
Lianmin Zheng
|
43925d179d
|
[Speculative] Fix Eagle3/DFLASH aux hidden state capture during CUDA graph init (#22836)
|
2026-04-15 14:04:54 -07:00 |
|
|
Lianmin Zheng
|
adb310b976
|
Cleanup server_args.py and minor code tidying (#22820)
|
2026-04-14 18:52:41 -07:00 |
|
|
Lianmin Zheng
|
222eda1598
|
[Misc] Use cache_once for is_arch_support_pdl in sgl-kernel (#22725)
|
2026-04-14 15:22:10 -07:00 |
|
|
Lianmin Zheng
|
9fb00ede15
|
Clean up TokenizerManager and req_time_stats: reduce overhead and simplify (#21646)
|
2026-04-13 16:47:32 -07:00 |
|
|
Lianmin Zheng
|
8f9553bccb
|
[Misc] Migrate SGLANG_SET_CPU_AFFINITY to envs and refactor model config building (#22730)
|
2026-04-13 16:10:31 -07:00 |
|
|
Lianmin Zheng
|
f81b6e8f51
|
[Misc] Add @cache_once to is_arch_support_pdl in jit_kernel (#22724)
|
2026-04-13 14:42:49 -07:00 |
|
|
Lianmin Zheng
|
ba7bcca6b3
|
Use reshape instead of contiguous().view() in TRTLLMHAAttnBackend (#22517)
|
2026-04-13 14:29:12 -07:00 |
|
|
Lianmin Zheng
|
ddc8ef1038
|
Lazy import flash_attention_v4 to avoid loading flash_attn.cute at startup (#22306)
|
2026-04-08 20:40:25 -07:00 |
|
|
Lianmin ZhengandClaude Opus 4.6
|
494bb86169
|
Cache sub-objects in __getitem__ to ensure identity stability (#22184)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-04-06 18:53:38 -07:00 |
|
|
Lianmin Zheng
|
a80961333b
|
Clean up req_time_stats: reduce overhead and simplify (#22186)
|
2026-04-06 14:20:51 -07:00 |
|
|
Lianmin Zheng
|
e835601fb7
|
Cache gfx95 quant format detection in DeepseekV2DecoderLayer (#22143)
|
2026-04-05 20:20:54 -07:00 |
|
|
Lianmin Zheng
|
eb407b80f3
|
[Kernel] Make FA3/FA4 imports lazy in FlashAttentionBackend (#22028)
|
2026-04-03 13:49:00 -07:00 |
|
|
Lianmin Zheng
|
fe38410c3e
|
Remove logging for subprocess watchdog start (#21968)
|
2026-04-02 11:30:33 -07:00 |
|
|
Lianmin Zheng
|
afb32d7622
|
README: coding agent sponsorship for long-term contributors (#21642)
|
2026-03-29 16:02:51 -07:00 |
|
|
Lianmin ZhengandClaude Opus 4.6
|
9f7792415a
|
Clean up TokenizerManager: remove dead code and improve rid validation (#21639)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-03-29 15:12:49 -07:00 |
|
|
Lianmin ZhengandClaude Opus 4.6
|
f3970b17ef
|
[Cleanup] Remove unused BatchMultimodalOutput and BatchMultimodalDecodeReq (#21640)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-03-29 14:54:25 -07:00 |
|