Commit Graph
910 Commits
Author SHA1 Message Date
Liangsheng Yin 1c4b130bf7 [CI] Unify basic and speculative sanity accuracy checks with MMLU (#39906) 2026-09-16 22:19:05 -07:00
Liangsheng Yin 923e4a56d4 [CI] Fix sanity evaluation and diffusion test suite blockers (#39892) 2026-09-16 21:47:41 -07:00
pllimax 00a9a81b67 [NPU][CI] Fail fast and speed up long-running qwen3.6 accuracy cases (#39813) 2026-09-16 20:20:37 +08:00
+4 faaff1eca8 dsv4.1: Top-k kernels and candidate selection (#39648)
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Ziyi Xu <ziyi.xu@radixark.ai>
2026-09-15 23:54:35 -07:00
Liangsheng Yin 87c9f78e99 [Fix][PD] Give prefill and decode their own RDMA NICs in disaggregation tests (#39584) 2026-09-15 15:07:42 -07:00
Liangsheng Yin d58342deab [Fix] Release NCCL on scheduler exit and let the ASGI server own shutdown (#39560) 2026-09-15 13:39:15 -07:00
pllimax 1aeeb25e86 [NPU][CI] Scope NPU nightly artifact dirs by image; fix stale glm5_2 case (#39585) 2026-09-15 22:55:53 +08:00
Liangsheng Yin 860fa83a9f [CI] Wait for a killed test server's GPU memory before the next launch (#39545) 2026-09-14 23:24:37 -07:00
pllimax 7465e42b7a [NPU][CI] Add CANN 9.1.0 and Ascend a5 nightly suites (#38833) 2026-09-14 22:39:42 +08:00
pllimax 433c999dd0 [NPU][CI] Fix sglang.test.ascend import failure in multi-node e2e pods (#39403) 2026-09-14 22:23:47 +08:00
Chao ShiandZhangheng bf9773e1da HiCache: Add @rank_consensus to various functions (#37425)
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-09-14 10:44:21 +08:00
Baizhou ZhangandChunan Zeng 5ebb16005d [DeepSeek-V4.1] Bump FlashMLA to the fork's rebase head (v4.1 kernels) (#39171)
Co-authored-by: Chunan Zeng <zcnrex@gmail.com>
2026-09-13 15:43:19 -07:00
a66451c058 [GLM-5.3 Flash] Restore and enable KPool metadata fusion (#38845)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: zRzRzRzRzRzRzR <Yuxuan.Zhang2@liverpool.ac.uk>
Co-authored-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: zanes-ops <zanes@nvidia.com>
2026-09-12 16:01:11 -07:00
iridiumine a984c78330 [NPU] Support DFlash speculative decoding for MiMo-V2.5-Pro (mxfp4) (#37565) 2026-09-12 11:59:23 +08:00
Bingxu Chen a338a9a01c [AMD][CI] Skip failing Wave test and relax multi-LoRA output check (#38585) 2026-09-11 12:25:20 -07:00
pllimax 92dffebe16 [NPU] Set DEEPEP_HYBRID_DEPLOYMENT for new DeepEP tests; switch glm5_2 to w8a8; tune nightly timeouts (#38775) 2026-09-10 23:32:54 +08:00
Brayden ZhongandBrayden Zhong c0b790cf7f Delete cutlass_mla, non-Marlin GPTQ, AWQ AOT kernel, and Dual Chunk Flash Attention (#32114)
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
2026-09-10 15:12:01 +08:00
DAI0818andPo-Han Huang 03e4c06589 Fix Qwen3.5 GDN multi-item scoring (#33922)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-09-09 22:30:39 -07:00
Cheng Wan f1a512c51c [Config] msgspec.Struct for the config tier (#38753) 2026-09-09 19:41:19 -07:00
Zhiqiang Xie beaf3d9252 [HiCache] Replace skip_lock_node_ids with a segment lock protocol (#36848) 2026-09-09 14:55:51 -07:00
Alison ShaoandClaude Opus 5 95a88bfd69 Relax GSM8K thresholds for the GLM-5.2 DSA-MTP variants (#38725)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-09 12:46:10 -07:00
Cheng Wan db272201a2 [Config] Retire get_global_server_args, and clear the deprecated flags that have a replacement (#38375) 2026-09-08 16:42:12 -07:00
+3 52fecfdf09 support qwen 3.8 flash next (#37500)
Co-authored-by: ch-wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: ispobock <26454835+ispobock@users.noreply.github.com>
Co-authored-by: JustinTong0323 <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: samuellees <26428561+samuellees@users.noreply.github.com>
Co-authored-by: YAMY1234 <74099316+YAMY1234@users.noreply.github.com>
Co-authored-by: yhyang201 <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: yizhang2077 <25844240+yizhang2077@users.noreply.github.com>
Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
Co-authored-by: Shinto C V <cshintov@gmail.com>
Co-authored-by: Julian Huang <huangzhilin.hzl@antgroup.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
2026-09-08 13:56:21 -07:00
Anupa Sajikumar 91a45ea37e [Intel][XPU][LoRA] Enable LoRA on Intel XPU (#30345) 2026-09-08 12:46:41 +08:00
Xiaoyu ZhangandMick Qian 4d23a4fa6d [Test] Consolidate test cleanup and CI taxonomy (net -11.4K lines) (#37436)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-07 15:13:59 +08:00
Baizhou Zhang b6c31b155c [CP V1 Deprecation 3/5] Remove generic prefill CP v1 runtime (#36228) 2026-09-06 21:53:53 -07:00
Cheng Wan b99175dc7d [Config] Round 6.4: the runtime reads the bags, not the record (#38049)
Last of four; stacked on #38048.

The record is the operator's input; the bags are what is in effect. A reader
that takes the record and reads a field off it gets the input, which is the
wrong one of the two whenever resolution decided something -- and the mistake is
silent, because for most fields and most launches the two agree. Several of
these files already read both ways, sometimes in the same expression:

```python
get_tokenizer(
    get_serving().tokenizer_path,
    tokenizer_mode=server_args.tokenizer_mode,   # the input, not the decision
    ...
)
```

Sixty-odd files convert. Record field reads in runtime code go from 199 to 11.
Nine parameters that the conversion emptied are dropped along with the argument
at every call site -- the dead-parameter ratchet is what names them.

### "Runs after its process publishes" is a per-entry-point claim

Most converted reads sit in the serving and model-executor layers, which only
exist after publication, or in the two subprocess entry points, which publish
first thing. Three places are not like that, and they keep reading the record
they were handed:

- **`HttpServerEngineAdapter`** launches the server as a *child*. The parent
  resolves the record and never publishes, so the adapter's own reads -- the
  launch banner, the API key in its readiness loop, the TP width in
  `update_weights_from_tensor` -- are of `self.server_args`. A bag read here
  fails closed in a bare process, or answers for an unrelated engine in one that
  happens to have published.
- **`serve_grpc`** reads its sidecar port before the integrated servicer builds
  the `Engine` that publishes. The comment above that line already said so and
  already bound `cfg = resolving_view(server_args)` for it; the sidecar port and
  the port it derives from read `cfg`.
- **`initialize_dp_attention`** runs from callers whose publish is not
  guaranteed, so its one predicate stays on the resolution view.

`ROLE_NAMESPACE_SETS["dp_controller"]` gains `observability` and `serving`,
because the controller's metrics gate, tracing setup and worker-port broadcast
now read those namespaces. Under `SGLANG_ROLE_NAMESPACES=enforce` that set is
what the process may read, so a conversion that reaches a new namespace has to
widen it in the same change.

## Three things worth a reviewer's attention

**Eleven reads were `getattr(record, "field", default)`.** An AST scan for
attribute access does not see those, so the census that said "43 readers" was
counting the shape it could match rather than the thing it was after.
`incremental_streaming_output` was read that way twice, and the transcription
tests were the only reason it surfaced.

**Not every record read is a bag read waiting to happen.** A multimodal
processor's `base_gpu_id` is the instance's, not the process's: two engines in
one process keep different ones, and
`test_publishing_another_config_does_not_move_the_device` exists to say so. It
stays on the record while `rl_on_policy_target` beside it moves.
`RequestMetricsExporter` is the same shape -- it is handed the directory it
writes to, and a test builds several with different ones. `configure_logger` is
a third: 17 call sites, one of which passes an `argparse.Namespace`, so it is
not a global-context reader at all. Those eleven remaining reads are the ones
with a reason.

**The fixtures move with the code.** Tests that hung config off a mock manager
now publish a record, which is what the serving layer reads; where a test states
a value it says so with `override_server_args` instead of assigning through the
mock. `test_hisparse_unit` is the last of them: it stubbed a `server_args` onto
a fake scheduler to say the decode radix cache was off, and the value it was
standing in for is the published default, so the stub goes and the class
publishes.

## Two things CI caught that a local sweep could not

**`unittest.TestCase.enterContext` is Python 3.11+.** The converted fixtures used
it at 18 sites; `requires-python` is `>=3.10` and CI runs 3.10, so every one of
them raised `AttributeError` there while passing on a newer local interpreter.
They call `enter_override(self, ...)` now -- a four-line helper in
`sglang/test/test_utils.py` over the override's own `install()` / `restore()`.

**A batched sweep cannot see a missing publish.** Three fixtures needed a
published config and did not have one; each *passed* inside a shard where some
other file had published, and failed when run alone. The affected cases are
`test_serving_completions` (which set `incremental_streaming_output` on the mock
manager's record, where nothing reads it now), `test_qwen3_vl_feature_materialization`
(same shape for `mm_enable_dp_encoder`), and the two Qwen Rust tests -- whose
fixture already carried the comment `# Non-auto: get_resolved_model_impl would
choke on a SimpleNamespace` next to the `model_impl` it sets, which is exactly
what happened once `get_mm_processor_cls` started reading that value from the
bag. Its `publish` mirrors `model_impl` now, like the four fields it already
mirrored.

## Verification

A full registered-unit sweep (648 files) against this stack's merge-base:
19 failures on both sides, the same 19, none of them config. That sweep is what
caught 23 failures the file-scoped runs missed -- and, later, that the narrower
139-file list did not even contain the files this change reaches. It is also
what caught the `test_hisparse_unit` fixture above: the file passes inside a
shard where something else published, and fails when it is run on its own,
which is why every failing file is re-run alone before it is counted.
2026-09-06 21:41:46 -07:00
ashwini rathi 0afba909e7 [XPU][CI] Fix empty nightly dashboard (#37800) 2026-09-07 09:29:51 +08:00
hhhh1252023 0948e6ebed [CI] Remove metrics artifact mechanism from nightly NPU workflows (#35489) 2026-09-05 17:16:52 +08:00
44c786679f [sp] Make attention-TP sequence sharding a per-forward batch property (#37546)
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
2026-09-04 02:11:03 -07:00
ashwini rathi e787de5478 [XPU][CI] Move XPU tests to nightly and add per-subclass server launch timeout (#37532) 2026-09-04 12:51:26 +08:00
Liangsheng Yin 6147a54ddf [PD] Bound transfer engine init with SGLANG_DISAGGREGATION_ENGINE_INIT_TIMEOUT (#37874) 2026-09-03 17:16:55 -07:00
Zhanghengand晟海 abed680320 [Unified Cache][5/N]: Integrate external linker mode end to end (#37381)
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
2026-09-04 02:02:58 +08:00
pllimax 3239baef25 [CI][NPU] Fix kimi_k2_6 16p in64k perf test and dsv4-flash testcases (#37760) 2026-09-03 21:57:14 +08:00
Alex NailsandAlison Shao 28262c20df [CI][RFC] Replace black-jupyter with ruff-format (#37210)
Co-authored-by: Alison Shao <a.shao@wustl.edu>
2026-09-02 19:46:08 -07:00
Alison Shao db1eb48651 [CI] Graceful teardown for the PD and HiSparse server fixtures (#37485) 2026-09-02 17:22:18 -07:00
YAMY 3c9cea8f10 [EAGLE] Prune draft-extend logits to selected rows (#35546) 2026-09-02 15:10:08 -07:00
Liangsheng Yin ebfd8c60e5 [CI] Install sgl-eval from PyPI through the test extra (#37504) 2026-09-02 01:45:25 -07:00
Wes 2d9c64394f Fix reasoning metrics and add TPOT to bench_multiturn (#35443) 2026-09-02 11:28:47 +08:00
Xiaoyu Zhang 403a15c163 [CI] Batch CPU test workers (#37252) 2026-09-02 10:35:14 +08:00
cctryandcctry 9a05b470fa [Memory] Size the CUDA graph pool from warmup measurements and fix graph-pool borrowing (#36911)
Co-authored-by: cctry <cctry@fb.com>
2026-09-01 09:32:38 -07:00
3ae54c6ca2 test(npu): add DSV4-Flash / GLM-5.2 / Kimi-K3 gpqa accuracy cases (#37431)
Co-authored-by: Sugar920 <Sugar920@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-09-01 22:00:04 +08:00
hhhh1252023 60f881b40c [CI/NPU] Isolate multi-node tests by run_id to prevent concurrent-run… (#35500) 2026-09-01 11:08:27 +08:00
ashwini rathiandarathi-hlab 783af667fb xpu: record per-model metrics to jsonl for nightly dashboard (#36699)
Co-authored-by: arathi-hlab <arathi-hlab@users.noreply.github.com>
2026-09-01 10:20:56 +08:00
Liangsheng YinandBaizhou Zhang 95f0f41021 [CI] Move tests onto the right CI stages (#34074)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
2026-08-31 12:39:59 -07:00
9cf157c252 [Radix Cache] Add Rust TreeCore backend with shared parity tests (#32710)
Co-authored-by: alphabetc1 <2508695655@qq.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
2026-09-01 00:26:20 +08:00
pllimax 63b2adbeac [NPU] Fix evalscope accuracy parsing and add glm5_1 aime26 request timeout (#36459) 2026-08-31 17:19:03 +08:00
29578d5578 refactor(unified-memory): translate the KV write location once, at ForwardBatch construction (#35245)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-08-30 23:52:14 -07:00
Liangsheng Yin 5d12ad4fd7 [mem_cache] Move mamba state and retraction_backup into ReqKvInfo (#37164) 2026-08-30 22:03:01 -07:00
Liangsheng Yin 007ef5e23a [mem_cache] Move req_pool_idx into ReqKvInfo (#37094) 2026-08-30 14:46:21 -07:00