Commit Graph
59 Commits
Author SHA1 Message Date
3c4653b6a3 [Metrics] Fix PD latency histogram accounting (#39706)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-09-29 13:12:09 -07:00
51ace48387 [Metrics] Add request-level TPOT histogram (sglang:request_time_per_output_token_seconds) (#40275)
Co-authored-by: Gilford Ting <gilford@modal.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-29 12:20:22 -07:00
Liangsheng Yin f35ec18d8b [Test] Remove more unit tests that mirror implementation or never run in CI (#41297) 2026-09-25 19:22:14 -07:00
cctryandcctry efff836911 [PD] Add decode host receive for custom transfer backends (#40238)
Co-authored-by: cctry <cctry@fb.com>
2026-09-23 21:57:33 -07:00
Cheng Wan 79fec592a8 [Refactor] Read parallel placement in consumers (#40638) 2026-09-23 14:57:25 -07:00
Suraj 1815e490de [observability] Fix negative queue_time for retracted requests (#39312) 2026-09-22 15:08:01 +08:00
Cheng Wan acac4dd9d9 [Refactor] Clean up parallel runtime comments (#40632) 2026-09-21 14:32:22 -07:00
Cheng Wan 970e946e4f Retire the per-runner parallel record (#40343) 2026-09-21 12:26:40 -07:00
Cheng Wan 3a5f52e144 Record a process's placement at publish, not at group build (#40071) 2026-09-18 23:52:37 -07:00
ymren cc171fbad0 feat: support custom OTLP trace service name (#35802) 2026-09-16 22:37:42 +08:00
Zhiqiang Xie 4e9e407d37 [HiCache] Label radix-cache metrics per rank and split the "shrunk" prefetch reason (#39280) 2026-09-15 17:54:04 -07:00
4da5599e93 Fix first-token metadata and reused attention-layer indexing (#39328)
Co-authored-by: Jinghui Zhang <jinghui@meta.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
2026-09-15 15:22:26 -07:00
Zhiqiang Xie 7f5dd19256 [HiCache] Rework the buffer-mode storage prefetch pipeline and retry bookkeeping (#39283) 2026-09-15 10:48:49 -07:00
Liu Ziming beecfda314 [observability] Fix missing e2e/decode/inference latency span attributes (#37789) 2026-09-08 22:50:55 +08:00
bf68369a18 [ray] Support Ray metric backend for engine metrics (#31415)
Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
2026-09-07 12:43:19 -07:00
sglang-botandsglang-bot 6252993afe chore: update CI test est_time values (#38238)
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
2026-09-06 17:49:41 -07:00
Jialin OuyangandPranjal Shankhdhar 010dc955be [Metrics] Export scheduler stage wall time (#37636)
Co-authored-by: Pranjal Shankhdhar <pranjal.ssh@gmail.com>
2026-09-04 10:45:53 -07:00
8b1d8c1703 [Metrics] Add rolling scheduler utilization counters (#37461)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: pranjalssh <pranjalssh@fb.com>
2026-09-04 08:06:17 -07:00
junduandMa Mingfei 8770c1db1f [CPU][CI]: rename Xeon CPU CI suites to stage-*-intel (#37395)
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-09-04 10:08:41 +08:00
Zhiqiang Xie a480f388b2 [HiCache] L3 storage prefetch lifecycle metrics and cross-tier attribution fixes (#37503) 2026-09-03 16:00:04 -07:00
Alex NailsandAlison Shao 28262c20df [CI][RFC] Replace black-jupyter with ruff-format (#37210)
Co-authored-by: Alison Shao <a.shao@wustl.edu>
2026-09-02 19:46:08 -07:00
Cheng WanandClaude Opus 5 1a3e152f03 config: six more runtime readers ask the bags (#36973)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-29 04:19:14 -07:00
514b997e6c Register CPU CI for 17 e2e tests and partition xeon base-c suite (#35227)
Co-authored-by: Zhang, Mingxu <mingxu.zhang@intel.com>
Co-authored-by: MingxuZh <109504044+MingxuZh@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-08-24 10:49:38 +08:00
Hanming Lu 526af15845 [Metrics] Discount queued prefill load by recent cache hits when waiting-queue matching is off (#35248) 2026-08-18 11:27:21 -07:00
Lennox Fu b42abbb1ba [metrics] Fix prefill FLOPs estimate to count prefix and per-request causal pairs (#34316) 2026-08-17 15:18:54 -07:00
Sam ShleiferandClaude Fable 5 a2e88279c2 metrics: don't clock-rebase unset time sentinels in ReqTimeStats deserialization (#34335)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 14:27:47 +08:00
Lianmin Zheng 4794b401d5 [Observability] Add startup, memory, and hybrid SWA diagnostics (#33375) 2026-08-04 12:50:09 -07:00
Lianmin ZhengandJialin Ouyang 5081c063c0 fix(metrics): clear forward occupancy on idle (#33562)
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
2026-08-04 12:49:17 -07:00
Cheng Wan aa3bbbc6e8 observability: publish the generated forward-pass-metrics endpoint to the bags (#33337)
When --forward-pass-metrics-ipc-name is left unset the reporter generates an
endpoint and has to hand it to external consumers (the documented contract is
that they read it back from the server config). That readback is the scheduler's
get_internal_state, which already reports get_context().resolved_server_args_dict(),
so the write moves to get_context().override and the read alongside it to
get_observability() — the endpoint still shows up in /server_info's
internal_states, and the ServerArgs instance stops being a message bus.

The test's server_args stand-in (a SimpleNamespace with a hand-rolled override)
becomes a real published config, so the reporter exercises the same accessors as
production.

Writer ratchet 19 -> 18.
2026-08-02 21:24:11 -07:00
Hongkuan Zhou d0b9689805 [Fix] Include disagg prefill waiting queue in FPM (#32122)
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
2026-07-23 09:30:27 -07:00
fzyzcjy 725920915f Introduce ModelRunner.ps ParallelState (#31161) 2026-07-14 16:01:14 +08:00
Liangsheng Yin 23390589f7 [misc] Remove unit test cases that fail the admission criteria (round 3) (#30713) 2026-07-09 19:42:51 -07:00
Liangsheng Yin b5e75b9423 [style] Extract init-static values in scheduler hot path (#30707) 2026-07-09 19:33:02 -07:00
Liangsheng Yin a36c873147 [misc] Remove unit test cases that fail the admission criteria (round 2) (#30703) 2026-07-09 16:35:59 -07:00
Cheng Wan 1f15308dca [refactor] Retire the legacy config accessor and the remaining process singletons (#30493) 2026-07-09 02:10:47 -07:00
Cheng Wan b14f7b4f75 [refactor] Move model-capability adjustments into the resolution pipeline (#30299) 2026-07-07 21:26:55 -07:00
Lianmin Zheng d5c566e59b Extract profile request cleanups (#29098) 2026-06-24 11:22:58 -07:00
jundu e04ed05193 [CI] reduce CPU CI scope with base-c suite (#28623) 2026-06-24 16:18:06 +08:00
Dongjun Na 462c01ea6b [observability] add Ray metric backend wrappers (#26252)
Signed-off-by: Dongjun Na <kmu5544616@gmail.com>
2026-06-17 19:41:53 -07:00
Liangsheng Yin 578f232e5e Fix trace_modules gate disabling default trace contexts (#27173) 2026-06-03 14:03:42 -04:00
e67810bea7 [SGLang Tracing] Add pd disaggregation mooncake backend tracing (#23755)
Co-authored-by: Mu Huai <tianbowen.tbw@antgroup.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-06-03 16:43:29 +08:00
MingxuZh 21ba329dac [Xeon] CPU CI enhancement for Intel Xeon platforms (#24649) 2026-05-28 10:49:04 +08:00
Dongjun Na 9d50cd9742 [observability] add ServerArgs.stat_loggers for pluggable metrics backend (#24610)
Signed-off-by: Dongjun Na <kmu5544616@gmail.com>
2026-05-24 22:41:28 +08:00
fzyzcjy fd97fbb096 Move metrics reporting to SchedulerMetricsReporter and retire metrics mixin (#25630) 2026-05-18 18:42:08 +08:00
Liangsheng Yin b7d62bd724 [CI] Rename basic CI stage-a/b/c -> base-a/b/c for symmetry with extra CI (#25420) 2026-05-15 18:26:55 -07:00
fzyzcjy 43797cc804 Bundle Scheduler rank/size fields into a frozen ParallelState (#25444) 2026-05-16 09:23:51 +08:00
Mick ff70aeac30 [diffusion] feat: add performance mode server args (#24491) 2026-05-14 00:57:46 +08:00
e86fb42736 feat: emit per-iteration forward pass metrics via ZMQ PUB (#22789)
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
2026-05-12 10:28:17 -07:00
TianheandClaude Sonnet 4.6 95985f983d feat(trace): support SGLANG_TRACE_LEVEL env var for startup trace level (#24716)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 10:31:03 -07:00
Yinzuo Jiang 71160e4ddb feat(observability): add OpenTelemetry tracing for pipeline parallelism (#23169)
Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
2026-04-28 17:05:23 +08:00