3c4653b6a3
[Metrics] Fix PD latency histogram accounting ( #39706 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com >
2026-09-29 13:12:09 -07:00
51ace48387
[Metrics] Add request-level TPOT histogram (sglang:request_time_per_output_token_seconds) ( #40275 )
...
Co-authored-by: Gilford Ting <gilford@modal.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-29 12:20:22 -07:00
Liangsheng Yin
f35ec18d8b
[Test] Remove more unit tests that mirror implementation or never run in CI ( #41297 )
2026-09-25 19:22:14 -07:00
cctry and cctry
efff836911
[PD] Add decode host receive for custom transfer backends ( #40238 )
...
Co-authored-by: cctry <cctry@fb.com >
2026-09-23 21:57:33 -07:00
Cheng Wan
79fec592a8
[Refactor] Read parallel placement in consumers ( #40638 )
2026-09-23 14:57:25 -07:00
Suraj
1815e490de
[observability] Fix negative queue_time for retracted requests ( #39312 )
2026-09-22 15:08:01 +08:00
Cheng Wan
acac4dd9d9
[Refactor] Clean up parallel runtime comments ( #40632 )
2026-09-21 14:32:22 -07:00
Cheng Wan
970e946e4f
Retire the per-runner parallel record ( #40343 )
2026-09-21 12:26:40 -07:00
Cheng Wan
3a5f52e144
Record a process's placement at publish, not at group build ( #40071 )
2026-09-18 23:52:37 -07:00
ymren
cc171fbad0
feat: support custom OTLP trace service name ( #35802 )
2026-09-16 22:37:42 +08:00
Zhiqiang Xie
4e9e407d37
[HiCache] Label radix-cache metrics per rank and split the "shrunk" prefetch reason ( #39280 )
2026-09-15 17:54:04 -07:00
4da5599e93
Fix first-token metadata and reused attention-layer indexing ( #39328 )
...
Co-authored-by: Jinghui Zhang <jinghui@meta.com >
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com >
2026-09-15 15:22:26 -07:00
Zhiqiang Xie
7f5dd19256
[HiCache] Rework the buffer-mode storage prefetch pipeline and retry bookkeeping ( #39283 )
2026-09-15 10:48:49 -07:00
Liu Ziming
beecfda314
[observability] Fix missing e2e/decode/inference latency span attributes ( #37789 )
2026-09-08 22:50:55 +08:00
bf68369a18
[ray] Support Ray metric backend for engine metrics ( #31415 )
...
Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: Qiaolin Yu <liin1211@outlook.com >
2026-09-07 12:43:19 -07:00
sglang-bot and sglang-bot
6252993afe
chore: update CI test est_time values ( #38238 )
...
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com >
2026-09-06 17:49:41 -07:00
Jialin Ouyang and Pranjal Shankhdhar
010dc955be
[Metrics] Export scheduler stage wall time ( #37636 )
...
Co-authored-by: Pranjal Shankhdhar <pranjal.ssh@gmail.com >
2026-09-04 10:45:53 -07:00
8b1d8c1703
[Metrics] Add rolling scheduler utilization counters ( #37461 )
...
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com >
Co-authored-by: pranjalssh <pranjalssh@fb.com >
2026-09-04 08:06:17 -07:00
jundu and Ma Mingfei
8770c1db1f
[CPU][CI]: rename Xeon CPU CI suites to stage-*-intel ( #37395 )
...
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com >
2026-09-04 10:08:41 +08:00
Zhiqiang Xie
a480f388b2
[HiCache] L3 storage prefetch lifecycle metrics and cross-tier attribution fixes ( #37503 )
2026-09-03 16:00:04 -07:00
Alex Nails and Alison Shao
28262c20df
[CI][RFC] Replace black-jupyter with ruff-format ( #37210 )
...
Co-authored-by: Alison Shao <a.shao@wustl.edu >
2026-09-02 19:46:08 -07:00
Cheng Wan and Claude Opus 5
1a3e152f03
config: six more runtime readers ask the bags ( #36973 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-08-29 04:19:14 -07:00
514b997e6c
Register CPU CI for 17 e2e tests and partition xeon base-c suite ( #35227 )
...
Co-authored-by: Zhang, Mingxu <mingxu.zhang@intel.com >
Co-authored-by: MingxuZh <109504044+MingxuZh@users.noreply.github.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-24 10:49:38 +08:00
Hanming Lu
526af15845
[Metrics] Discount queued prefill load by recent cache hits when waiting-queue matching is off ( #35248 )
2026-08-18 11:27:21 -07:00
Lennox Fu
b42abbb1ba
[metrics] Fix prefill FLOPs estimate to count prefix and per-request causal pairs ( #34316 )
2026-08-17 15:18:54 -07:00
Sam Shleifer and Claude Fable 5
a2e88279c2
metrics: don't clock-rebase unset time sentinels in ReqTimeStats deserialization ( #34335 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-08-12 14:27:47 +08:00
Lianmin Zheng
4794b401d5
[Observability] Add startup, memory, and hybrid SWA diagnostics ( #33375 )
2026-08-04 12:50:09 -07:00
Lianmin Zheng and Jialin Ouyang
5081c063c0
fix(metrics): clear forward occupancy on idle ( #33562 )
...
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com >
2026-08-04 12:49:17 -07:00
Cheng Wan
aa3bbbc6e8
observability: publish the generated forward-pass-metrics endpoint to the bags ( #33337 )
...
When --forward-pass-metrics-ipc-name is left unset the reporter generates an
endpoint and has to hand it to external consumers (the documented contract is
that they read it back from the server config). That readback is the scheduler's
get_internal_state, which already reports get_context().resolved_server_args_dict(),
so the write moves to get_context().override and the read alongside it to
get_observability() — the endpoint still shows up in /server_info's
internal_states, and the ServerArgs instance stops being a message bus.
The test's server_args stand-in (a SimpleNamespace with a hand-rolled override)
becomes a real published config, so the reporter exercises the same accessors as
production.
Writer ratchet 19 -> 18.
2026-08-02 21:24:11 -07:00
Hongkuan Zhou
d0b9689805
[Fix] Include disagg prefill waiting queue in FPM ( #32122 )
...
Signed-off-by: hongkuanz <hongkuanz@nvidia.com >
2026-07-23 09:30:27 -07:00
fzyzcjy
725920915f
Introduce ModelRunner.ps ParallelState ( #31161 )
2026-07-14 16:01:14 +08:00
Liangsheng Yin
23390589f7
[misc] Remove unit test cases that fail the admission criteria (round 3) ( #30713 )
2026-07-09 19:42:51 -07:00
Liangsheng Yin
b5e75b9423
[style] Extract init-static values in scheduler hot path ( #30707 )
2026-07-09 19:33:02 -07:00
Liangsheng Yin
a36c873147
[misc] Remove unit test cases that fail the admission criteria (round 2) ( #30703 )
2026-07-09 16:35:59 -07:00
Cheng Wan
1f15308dca
[refactor] Retire the legacy config accessor and the remaining process singletons ( #30493 )
2026-07-09 02:10:47 -07:00
Cheng Wan
b14f7b4f75
[refactor] Move model-capability adjustments into the resolution pipeline ( #30299 )
2026-07-07 21:26:55 -07:00
Lianmin Zheng
d5c566e59b
Extract profile request cleanups ( #29098 )
2026-06-24 11:22:58 -07:00
jundu
e04ed05193
[CI] reduce CPU CI scope with base-c suite ( #28623 )
2026-06-24 16:18:06 +08:00
Dongjun Na
462c01ea6b
[observability] add Ray metric backend wrappers ( #26252 )
...
Signed-off-by: Dongjun Na <kmu5544616@gmail.com >
2026-06-17 19:41:53 -07:00
Liangsheng Yin
578f232e5e
Fix trace_modules gate disabling default trace contexts ( #27173 )
2026-06-03 14:03:42 -04:00
e67810bea7
[SGLang Tracing] Add pd disaggregation mooncake backend tracing ( #23755 )
...
Co-authored-by: Mu Huai <tianbowen.tbw@antgroup.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-06-03 16:43:29 +08:00
MingxuZh
21ba329dac
[Xeon] CPU CI enhancement for Intel Xeon platforms ( #24649 )
2026-05-28 10:49:04 +08:00
Dongjun Na
9d50cd9742
[observability] add ServerArgs.stat_loggers for pluggable metrics backend ( #24610 )
...
Signed-off-by: Dongjun Na <kmu5544616@gmail.com >
2026-05-24 22:41:28 +08:00
fzyzcjy
fd97fbb096
Move metrics reporting to SchedulerMetricsReporter and retire metrics mixin ( #25630 )
2026-05-18 18:42:08 +08:00
Liangsheng Yin
b7d62bd724
[CI] Rename basic CI stage-a/b/c -> base-a/b/c for symmetry with extra CI ( #25420 )
2026-05-15 18:26:55 -07:00
fzyzcjy
43797cc804
Bundle Scheduler rank/size fields into a frozen ParallelState ( #25444 )
2026-05-16 09:23:51 +08:00
Mick
ff70aeac30
[diffusion] feat: add performance mode server args ( #24491 )
2026-05-14 00:57:46 +08:00
e86fb42736
feat: emit per-iteration forward pass metrics via ZMQ PUB ( #22789 )
...
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com >
2026-05-12 10:28:17 -07:00
Tianhe and Claude Sonnet 4.6
95985f983d
feat(trace): support SGLANG_TRACE_LEVEL env var for startup trace level ( #24716 )
...
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-05-11 10:31:03 -07:00
Yinzuo Jiang
71160e4ddb
feat(observability): add OpenTelemetry tracing for pipeline parallelism ( #23169 )
...
Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com >
2026-04-28 17:05:23 +08:00