Commit Graph
19 Commits
Author SHA1 Message Date
Cheng Wan 970e946e4f Retire the per-runner parallel record (#40343) 2026-09-21 12:26:40 -07:00
5931fd60ee Support unified memory page-envelope transfers in PD (#39477)
Co-authored-by: yhzhuang <yhzhuang@fb.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Yonghao Zhuang <yhzhuang@users.noreply.github.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-09-18 17:39:50 -07:00
iridiumine f86f60081d [NPU] Adapt hicache for K3 hybrid models (#39415) 2026-09-18 14:48:20 +08:00
Yanbin Jiang ebd37705e4 [PD][LoRA] Gate decode admission on adapter slots (#39332) 2026-09-15 13:11:04 +08:00
Cheng Wan b99175dc7d [Config] Round 6.4: the runtime reads the bags, not the record (#38049)
Last of four; stacked on #38048.

The record is the operator's input; the bags are what is in effect. A reader
that takes the record and reads a field off it gets the input, which is the
wrong one of the two whenever resolution decided something -- and the mistake is
silent, because for most fields and most launches the two agree. Several of
these files already read both ways, sometimes in the same expression:

```python
get_tokenizer(
    get_serving().tokenizer_path,
    tokenizer_mode=server_args.tokenizer_mode,   # the input, not the decision
    ...
)
```

Sixty-odd files convert. Record field reads in runtime code go from 199 to 11.
Nine parameters that the conversion emptied are dropped along with the argument
at every call site -- the dead-parameter ratchet is what names them.

### "Runs after its process publishes" is a per-entry-point claim

Most converted reads sit in the serving and model-executor layers, which only
exist after publication, or in the two subprocess entry points, which publish
first thing. Three places are not like that, and they keep reading the record
they were handed:

- **`HttpServerEngineAdapter`** launches the server as a *child*. The parent
  resolves the record and never publishes, so the adapter's own reads -- the
  launch banner, the API key in its readiness loop, the TP width in
  `update_weights_from_tensor` -- are of `self.server_args`. A bag read here
  fails closed in a bare process, or answers for an unrelated engine in one that
  happens to have published.
- **`serve_grpc`** reads its sidecar port before the integrated servicer builds
  the `Engine` that publishes. The comment above that line already said so and
  already bound `cfg = resolving_view(server_args)` for it; the sidecar port and
  the port it derives from read `cfg`.
- **`initialize_dp_attention`** runs from callers whose publish is not
  guaranteed, so its one predicate stays on the resolution view.

`ROLE_NAMESPACE_SETS["dp_controller"]` gains `observability` and `serving`,
because the controller's metrics gate, tracing setup and worker-port broadcast
now read those namespaces. Under `SGLANG_ROLE_NAMESPACES=enforce` that set is
what the process may read, so a conversion that reaches a new namespace has to
widen it in the same change.

## Three things worth a reviewer's attention

**Eleven reads were `getattr(record, "field", default)`.** An AST scan for
attribute access does not see those, so the census that said "43 readers" was
counting the shape it could match rather than the thing it was after.
`incremental_streaming_output` was read that way twice, and the transcription
tests were the only reason it surfaced.

**Not every record read is a bag read waiting to happen.** A multimodal
processor's `base_gpu_id` is the instance's, not the process's: two engines in
one process keep different ones, and
`test_publishing_another_config_does_not_move_the_device` exists to say so. It
stays on the record while `rl_on_policy_target` beside it moves.
`RequestMetricsExporter` is the same shape -- it is handed the directory it
writes to, and a test builds several with different ones. `configure_logger` is
a third: 17 call sites, one of which passes an `argparse.Namespace`, so it is
not a global-context reader at all. Those eleven remaining reads are the ones
with a reason.

**The fixtures move with the code.** Tests that hung config off a mock manager
now publish a record, which is what the serving layer reads; where a test states
a value it says so with `override_server_args` instead of assigning through the
mock. `test_hisparse_unit` is the last of them: it stubbed a `server_args` onto
a fake scheduler to say the decode radix cache was off, and the value it was
standing in for is the published default, so the stub goes and the class
publishes.

## Two things CI caught that a local sweep could not

**`unittest.TestCase.enterContext` is Python 3.11+.** The converted fixtures used
it at 18 sites; `requires-python` is `>=3.10` and CI runs 3.10, so every one of
them raised `AttributeError` there while passing on a newer local interpreter.
They call `enter_override(self, ...)` now -- a four-line helper in
`sglang/test/test_utils.py` over the override's own `install()` / `restore()`.

**A batched sweep cannot see a missing publish.** Three fixtures needed a
published config and did not have one; each *passed* inside a shard where some
other file had published, and failed when run alone. The affected cases are
`test_serving_completions` (which set `incremental_streaming_output` on the mock
manager's record, where nothing reads it now), `test_qwen3_vl_feature_materialization`
(same shape for `mm_enable_dp_encoder`), and the two Qwen Rust tests -- whose
fixture already carried the comment `# Non-auto: get_resolved_model_impl would
choke on a SimpleNamespace` next to the `model_impl` it sets, which is exactly
what happened once `get_mm_processor_cls` started reading that value from the
bag. Its `publish` mirrors `model_impl` now, like the four fields it already
mirrored.

## Verification

A full registered-unit sweep (648 files) against this stack's merge-base:
19 failures on both sides, the same 19, none of them config. That sweep is what
caught 23 failures the file-scoped runs missed -- and, later, that the narrower
139-file list did not even contain the files this change reaches. It is also
what caught the `test_hisparse_unit` fixture above: the file passes inside a
shard where something else published, and fails when it is run on its own,
which is why every failing file is re-run alone before it is counted.
2026-09-06 21:41:46 -07:00
sglang-botandsglang-bot 6252993afe chore: update CI test est_time values (#38238)
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
2026-09-06 17:49:41 -07:00
Yonghao Zhuangandyhzhuang 4dc9cda5f9 [PD] Gate deferred decode KV release on backend capability (#37454)
Co-authored-by: yhzhuang <yhzhuang@fb.com>
2026-09-03 15:42:21 -07:00
Alex NailsandAlison Shao 28262c20df [CI][RFC] Replace black-jupyter with ruff-format (#37210)
Co-authored-by: Alison Shao <a.shao@wustl.edu>
2026-09-02 19:46:08 -07:00
cctry 3a855b050a fix(disagg): poll receivers during decode preallocation (#37483) 2026-09-02 14:26:40 -07:00
ishandhananiandShangming Cai 978244d671 [P/D disagg] Decode-side radix cache for SWA hybrid models (unified radix tree) (#27770)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-08-21 09:25:06 +08:00
YAMY aa215e5523 [PD] Overlap prefill DP-rank bootstrap queries (#35071) 2026-08-19 17:25:52 +08:00
Shangming CaiandClaude Opus 4.8 97dedd1ce9 [PD] Deferred decode-side KV release for aborts mid-transfer (#35049)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-18 23:56:33 +08:00
cctryandcctry 2e7c85da68 [PD] Preserve decode KV across retraction in HiCache (#34801)
Co-authored-by: cctry <cctry@fb.com>
2026-08-17 08:49:11 -07:00
Hao Zhangandzhisbug af7c62e337 Fix paged SWA retraction resume accounting (#33794)
Co-authored-by: zhisbug <1654062+zhisbug@users.noreply.github.com>
2026-08-06 16:32:54 -07:00
ziang663andChao Shi eefb434d17 [PD+PP] Honor PP consensus for bootstrap and prealloc (#31869)
Co-authored-by: Chao Shi <chao.shi@alibaba-inc.com>
2026-07-30 01:19:17 +08:00
fzyzcjy 725920915f Introduce ModelRunner.ps ParallelState (#31161) 2026-07-14 16:01:14 +08:00
fzyzcjy 5be9c9f7c6 Localize cur_batch field in Scheduler to avoid field-based state access (#29407) 2026-07-10 08:55:06 +08:00
amote-i c9303a08da Fix scheduler crash on prefill-unreachable decode abort (#29834) 2026-07-08 14:06:37 +08:00
luoroger37 18989f3d48 [PD] Fix resource leak on prealloc/transfer abort and idle check (#28022) 2026-06-12 18:20:24 +08:00