Commit Graph
26 Commits
Author SHA1 Message Date
Cheng Wan a9c97c9f69 [Feature] Add --attn-dp-size and deprecate --enable-dp-attention (#41818) 2026-09-30 13:34:18 -07:00
Cheng WanandClaude Opus 5 fd40a331bf config: a parallel size has one spelling; a patched scope declares its own (#36621)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 12:56:42 -07:00
Cheng WanandClaude Opus 5 ca1d7ed8e6 config: a parallel leaf with no live counterpart is read bare (#36620)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 12:56:11 -07:00
Cheng Wan 937af8538b config: the runtime readers take the published bags (#36254) 2026-08-26 05:08:25 -07:00
Cheng WanandClaude Opus 5 64aa859da2 config: constructing a config no longer resolves it (#35907)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-23 01:18:53 -07:00
Cheng Wan 99cfc90658 config: retire ServerArgs.override in favour of derive()
`ServerArgs.override(source, **fields)` was the last way to change a resolved
`ServerArgs` in place. Every remaining call-site was one of two things, and
neither wanted an in-place write:

- **A config for someone else.** A draft worker's context length, an encode
  worker's device, the compile script's watchdog, the client's port pick, a test
  fixture's backends. These already deepcopied first — the write was on the copy.
- **A launcher-stage resolution.** `resolve_auto_parsers` detected the chat
  template's parsers and wrote them back, to be inherited by the schedulers it
  spawns.

Both are "one config becomes another", so `derive(source, **fields)` returns the
variant and leaves the receiver — and any bags projected from it — untouched. It
deliberately is not `dataclasses.replace`: resolution does not re-run, because
the values being set are decided after it, from inputs it never had. Provenance
and the resolvable-field stash work as before, on the copy.

`resolve_auto_parsers` now computes the parsers and returns the config to launch
with; the detection helpers stop taking a config to mutate. `HiMambaRadixCache`
re-applied a HiCache layout normalization `__post_init__` already performs (the
same duplicate removed from `UnifiedRadixCache` in ebb1c88d23) and just goes.

With no in-place mutation left, `ServerArgs.__setattr__` raising after
resolution *is* the guarantee, so the textual writer ratchet retires and
`test_server_args_derive.py` pins the contract instead: the receiver survives
deriving, the published instance still refuses assignment, and deriving does not
publish. `SGLANG_STRICT_CONFIG_MUTATION` was already unused — the guard has been
unconditional since the mutation sweep — and goes with it.

The detection tests drop their `SimpleNamespace` stand-in for a real
`ServerArgs`; the test kit and the MLA chunk-metadata fixture publish a derived
variant instead of writing the runner's published config.
2026-08-05 19:30:53 -07:00
Cheng Wan b14f7b4f75 [refactor] Move model-capability adjustments into the resolution pipeline (#30299) 2026-07-07 21:26:55 -07:00
2495c02c2c [Refactor] Cuda Graph Runner/Backend Refactor (#23906)
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
2026-06-09 21:36:57 -07:00
1cff871c67 [Bugfix] Fix DeepEP timeout when compiling DeepGeMM in EP+DP+TP (#23185)
Co-authored-by: Byron Hsu <byronhsu@Byrons-MacBook-Pro.local>
Co-authored-by: Cheng Wan <ch-wan@users.noreply.github.com>
2026-04-19 17:36:11 -07:00
vincentzed ddb3970eb4 Tiny env cleanup in deepgemm (#15706)
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
2025-12-24 00:58:46 -08:00
Huang Lin 6abdf73f4d [Fix] Environment variable SGL_* is deprecated (#14943) 2025-12-13 19:55:43 -08:00
Lianmin Zheng ca52ed425f Clean up imports and move files (#14317) 2025-12-02 16:31:54 -08:00
ishandhanani 07821352fb Revert "Skip weight loading in deepgemm compilation" (#14241) 2025-12-01 12:59:09 -08:00
ishandhanani 263eab9f5d fix: dummy health check server not accessible on non-zero rank nodes (#12297) 2025-10-31 11:34:57 -07:00
Liangsheng Yin acc2327bbd Move deep gemm related arguments to sglang.srt.environ (#11547) 2025-10-14 00:34:35 +08:00
Cheng Wan 97cd38e58d Skip weight loading in deepgemm compilation (#11312) 2025-10-07 21:52:46 -07:00
Baron Liu b89d37cb11 [bugfix] Add 'disaggregation_mode' parameter to warmup function when compile deep_gemm manually (#8618) 2025-08-01 16:02:53 -07:00
Lifu Huang 3cf1473a09 Use monotonic clock for interval measurement (#6211)
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
2025-05-17 16:49:18 -07:00
Lianmin Zheng e8e18dcdcc Revert "fix some typos" (#6244) 2025-05-12 12:53:26 -07:00
applesaucethebunandBrayden Zhong d738ab52f8 fix some typos (#6209)
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
2025-05-13 01:42:38 +08:00
applesaucethebunandBrayden Zhong 2ce8793519 Add typo checker in pre-commit (#6179)
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
2025-05-11 12:55:00 +08:00
Yuhong Guo f87a6ab359 Resolves the 404 Not Found error when running compile_deep_gemm.py in multi-node setups (#5720) 2025-04-26 18:13:13 -07:00
JieXin Liang eebfdb9459 [fix] fix potential bumpy throughtput with deepgemm (#5722) 2025-04-26 18:12:48 -07:00
JieXin Liang 506be6b892 [fix] fix compile_deep_gemm missing kv_b_proj (#5620) 2025-04-22 00:06:36 -07:00
JieXin Liang 2343d8df7d [fix] force use deepgemm in compile_deep_gemm (#5618) 2025-04-21 21:36:02 -07:00
JieXin Liang c2942907d5 [feature] enable pre compile jit deep_gemm (#5580) 2025-04-21 16:52:53 -07:00