Commit Graph
18732 Commits
Author SHA1 Message Date
minke.yu 1f6ca2cf68 test: adopt upstream #42035 auxiliary-output asserts now that checkpoint_disagg_prefill shim exists
build-sglang-image / build (push) Successful in 25m55s
2026-10-09 11:32:33 +08:00
minke.yu bde1fcbf51 test: tolerate chunked kwarg in #42035 polling test override (branch shim keeps chunked flag) 2026-10-09 11:25:05 +08:00
minke.yu c4f3e3be33 refactor: route disagg prefill checkpoints through checkpoint_disagg_prefill shim
Upstream #42035's tests and call structure expect Scheduler.checkpoint_disagg_prefill
(introduced upstream with #41520's checkpoint_kv_cache / insert_req cache API).
This branch still uses maybe_cache_unfinished_req (old cache_unfinished_req API),
so add a thin shim with the upstream name and route the three call sites through it.
No behavior change.
2026-10-09 11:22:42 +08:00
Liangsheng Yin 9ef65b51c2 [mem_cache] Skip the release-time insert on optimistic prefill requeue; make refresh_fill_ids public (#42194)
build-sglang-image / build (push) Successful in 25m53s
(cherry picked from commit 07e1a4f082)
2026-10-09 11:18:17 +08:00
minke.yu 6ba34507f1 test: adapt #42035 polling test to pre-#41520 name maybe_cache_unfinished_req
build-sglang-image / build (push) Successful in 26m20s
2026-10-09 11:10:39 +08:00
metamergebot 93d610c1fd [PD] Keep ingesting requests while a prefill forward result is pending (#42035)
build-sglang-image / build (push) Successful in 26m6s
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
(cherry picked from commit 6acf2f4736)
2026-10-09 10:56:14 +08:00
minke.yu 4d0efcc921 deploy: bring k8s 3p1d RBG cw manifest into git; bump to 1068926cd (merged upstream batch)
Source of truth moves from D:\B300\build-kmctl (untracked) to
deploy/b300-ds41/ds41-3p1d-rbg-cw.yaml. Image carries A-batch (7) + B-batch (4)
upstream cherry-picks + is_hip_runtime import fix; bench-validated on compose
(3p1d 8-conc 160k 98903 tok/s, +50% vs 0928 baseline).
2026-10-08 17:41:06 +08:00
minke.yu f71fafa684 deploy: bump to 1068926cd (is_hip_runtime import fix) 2026-10-08 16:39:04 +08:00
minke.yu 1068926cd6 fix: import is_hip_runtime in dsv4 low_ratio_compress (cherry-pick #41660 lost it)
build-sglang-image / build (push) Successful in 32m18s
Upstream's file gets is_hip_runtime from sglang.kernels.jit.utils via an
intermediate refactor we don't have; our cherry-pick applied cleanly but
the resulting file references is_hip_runtime() without importing it,
crashing every prefill (NameError in c2_prefill_norm_rope_store).

Also unify Dockerfile.fast with the v3.1 full path (check_deps + --no-deps
delta installs) so fast builds after a kernel-wheel bump don't silently
revert to the base image's older kernels.
2026-10-08 16:31:34 +08:00
minke.yu 48fb49baef deploy: bump cpdspark-nr/b4-dp/3p1d-otlp to dsv41-pd-visioncp-2372cdc06-local-20261008-0808 (merged upstream fixes batch) 2026-10-08 16:21:30 +08:00
minke.yu 2372cdc06e build: light delta also --no-deps — any dep-graph walk over torch re-downloads pinned nccl==2.29.7 over base's 2.30.7 2026-10-08 16:07:50 +08:00
minke.yu 7b815cc98b build: COPY check_deps.py from sglang/ tree (context .dockerignore excludes build/) 2026-10-08 16:01:12 +08:00
minke.yu d06c5dde42 build: v3.1 three-class dep delta — kernel wheels (sglang-kernel/flashinfer/sgl-deep-gemm) install via pip --no-deps when pins move
Base ships sglang-kernel 0.4.6.post1 but pyproject pins ==0.4.7; kernel
wheels must track the sglang source, so they are an allowed delta (~370MB
wheel), unlike the torch/nvidia family which must always reuse the base.
Also confirmed 2026-09-24 full build silently downgraded NCCL 2.30.7->2.29.7;
v3 keeps base NCCL untouched.
2026-10-08 15:59:36 +08:00
minke.yu 8d0ee5e51e build: v3 full path reuses base torch/nvidia via local delta check
build-sglang-image / build (push) Successful in 28m50s
Full-resolution pip walks installed torch's metadata and would re-download
the pinned nvidia family (~6GB, throttled to 2h40m on 2026-09-23) and
downgrade the deliberately newer nvidia-nccl-cu13 2.30.7 to ==2.29.7.
v3 replaces pip full-resolution with check_deps.py: compare pyproject
requirements against base-installed versions locally, pip-install only the
lightweight delta via -r, and fail loudly if a heavy pin (torch*/nvidia-*/
sglang-kernel/flashinfer/cuda-*) becomes unsatisfied. Editable install keeps
--no-deps --no-build-isolation.
2026-10-08 15:54:58 +08:00
metamergebot 75758cb0a0 [HiCache] Fix cgroup page-cache accounting and the sizing fallback when cgroup discovery fails (#42420)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
Co-authored-by: nvjullin <nvjullin@users.noreply.github.com>
(cherry picked from commit 4ab720e655)
2026-10-08 15:41:51 +08:00
metamergebot 60e80ac7d1 [HiCache] Fix write-back SWA insert backups tripping the write-through pending-ack assert (#42264)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: 842974287 <842974287@users.noreply.github.com>
(cherry picked from commit 2bae12b9b3)
2026-10-08 15:37:43 +08:00
Sasha Sidorov 4ef799aaa6 [PD] Wait for prefill completion before Mooncake early KV transfer (#41395)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit c683190b40)
2026-10-08 15:36:21 +08:00
Aurick Qiao 1be762368b [Disagg] Validate state strides before Mooncake transfers (#41607)
Co-authored-by: Aurick Qiao <6137920+aurickq@users.noreply.github.com>
(cherry picked from commit 1a0250c5b6)
2026-10-08 15:35:05 +08:00
DarkSharpness f577b61b52 [DSv4.1] Trim prefill model-entry overhead (#41704)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: BBuf <1182563586@qq.com>
(cherry picked from commit 2f6e0ec45d)
2026-10-08 15:32:48 +08:00
DarkSharpness 4792b61332 [DSv4.1] Fused c1/c2 compress for eager extend, faster c2 decode (#41660)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: BBuf <1182563586@qq.com>
(cherry picked from commit 5916999afb)
2026-10-08 15:32:48 +08:00
DarkSharpness 62f1ed099e [DSv4.1] Faster fp4 index-K gather and combine_topk_swa_indices (#41658)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: BBuf <1182563586@qq.com>
(cherry picked from commit d75d5b33f7)
2026-10-08 15:32:47 +08:00
AMD-yanfeiwang bfd9bb2e03 fix(cuda-graph): remove unnecessary CP batch alignment (#40543)
(cherry picked from commit 3dc7f4b816)
2026-10-08 15:32:46 +08:00
metamergebot 6deac260bf [PD] Make the prebuilt last-token H2D copy non-blocking under overlap (#42113)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: bbus <bbus@users.noreply.github.com>
(cherry picked from commit 7dd93b01a5)
2026-10-08 15:32:46 +08:00
Yuwei An 842ad7c4d4 [Fix][DSV4.1] SWA page size with bounded replay (#42128)
(cherry picked from commit ef867fa40d)
2026-10-08 15:32:45 +08:00
Mohammad Miadh Angkad b605d2d370 [DSA] Stop forcing the per-step CPU seq_lens sync for the k-pool indexer (#41987)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
(cherry picked from commit 42f2e4fcb7)
2026-10-08 15:32:44 +08:00
minke.yu e403cede0a deploy: D mr320/cg256/mf0.83 (KV pool 41.8M tok/worker); d1 HTTP 30001->30030 fix port collision with p1 scheduler loopback bind 26.10.01 2026-09-28 20:00:11 +08:00
minke.yu 735434fcde deploy: k8s RBG 3p1d cw manifest; D cg512 + load-balance-method total_requests 2026-09-28 16:47:16 +08:00
minke.yu e17fbc9a2d deploy: 3p1d-otlp cs32k variant (chunked_prefill 32768 on P)
A/B vs dockerserve-3p1d-otlp.yml (chunk 16384), same image dsv41-pd-visioncp-db7d2cb7d-fix:
- bs1 160k prefill 61950 -> 66084 tok/s (+6.7%)
- bs16 4k prefill 47725 -> 70463 tok/s (+47.6%, fewer scheduler rounds)
- decode TPOT within dspark noise; trace: 160k sender 10->5 chunks, 3.16s->2.98s
Bench logs: bench_cs32k-20260927.log (round2) vs bench_otlptrace2-20260927.log
2026-09-27 14:31:33 +08:00
minke.yu 5507fff133 deploy: add 3p1d soak/prof compose + otlp tracing variant (jaeger badger 6h) 2026-09-27 12:10:00 +08:00
minke.yu 4f0ca95477 deploy: b300 build scripts with pip cache mount + no-build-isolation fast path
Base image already has torch 2.13, but pyproject build-system.requires
pulls it into pip's isolated build env on every build (~6GB via throttled
mirror, 47min). --no-build-isolation reuses the base env; build.sh picks
Dockerfile.fast (--no-deps) when python/pyproject.toml is unchanged.
2026-09-24 13:11:41 +08:00
minke.yu b91137ab98 scheduler: CP-symmetric idle check for health-check admission
build-sglang-image / build (push) Successful in 27m52s
Health-check admit/skip used is_fully_idle(), which includes rank-local
hicache drain queues; ranks diverge right after activity, so one rank
dispatched the health-check generate while others piggyback-skipped,
deadlocking CP (hicache drain all_reduce vs CP request broadcast).
Seen on cp2/cp4 + hicache L3 after router health checks.

Recovered from b300 /data/ymk/sglang working copy (uncommitted WIP).
2026-09-24 12:01:57 +08:00
minke.yu 67d8368a84 deploy: archive b300 ds41 compose files; skip CI for deploy/ and .gitea/ changes 2026-09-24 11:49:37 +08:00
minke.yu db7d2cb7db fix: vision check uses get_parallel().pp_group (get_pp_group undefined in this tree)
build-sglang-image / build (push) Successful in 27m58s
2026-09-23 18:05:57 +08:00
injet f23179ce99 更新 .gitea/workflows/build-image.yaml
build-sglang-image / build (push) Successful in 26m22s
2026-09-23 17:54:53 +08:00
injet 8266769b2d 更新 .gitea/workflows/build-image.yaml
build-sglang-image / build (push) Failing after 1m12s
2026-09-23 17:49:30 +08:00
injet 3b671d6086 更新 .gitea/workflows/build-image.yaml 2026-09-23 17:48:46 +08:00
minke.yu 7925735a3e ci: also trigger image build on dsv41-pd-visioncp pushes
build-sglang-image / build (push) Successful in 29m50s
2026-09-23 16:46:06 +08:00
Xinyuan Tong 6833498646 model: prune comments and redundant tests in dsv41 vision CP 2026-09-23 14:36:01 +08:00
Xinyuan Tong b48e2cb1eb model: TP-wide single-owner image encoding for DeepSeek V4.1
ViT and Aligner are replicated per TP rank, encoding each image eight
times with TP8 on both CP1 and CP8. Elect one owner per image and use
ordered full-span broadcasts with a six-phase agreement protocol.
Both CP1 and CP8 benefit while local cache hits preserve collective order.
2026-09-23 14:36:01 +08:00
Xinyuan Tong bfeb7cd9b2 model: support DeepSeek V4.1 vision with interleave prefill CP
The CP runner bypassed the vision merge and used bare text embeddings.
Merge image features before sharding so request-global offsets stay valid.
Canonicalize model IDs separately to preserve scheduler hash IDs.
Keep unsupported combinations guarded and isolate embedding overrides
from multimodal prefills without starving queued FCFS requests.
2026-09-23 14:36:00 +08:00
Yuwei An ddf5207630 [Fix] Handle chunked paged MQA metadata in DSV4.1 eager forwards (#40637)
build-sglang-image / build (push) Successful in 32m15s
2026-09-23 13:35:55 +08:00
minke.yu 104218d9ed ci: tag builds with -ci-<UTC datetime> suffix (naming rule: <branch>-<sha9>-<ci|local>-<UTC>)
build-sglang-image / build (push) Successful in 30m44s
2026-09-22 20:37:57 +08:00
minke.yu 92632a60ba ci: drop gha cache backend (runner cache server 404s on this instance)
build-sglang-image / build (push) Successful in 50m51s
2026-09-22 18:49:20 +08:00
minke.yu 4b3b367b63 ci: hardcode Gitea registry (vars context falls back to docker.io on this instance)
build-sglang-image / build (push) Failing after 24m13s
2026-09-22 18:23:35 +08:00
minke.yu a78da9b524 ci: Gitea Actions workflow to build sglang image on push to dsv41-pd
build-sglang-image / build (push) Failing after 27m21s
Overseas node: original docker.io base (lmsysorg/sglang:dev-dsv41) and
default pypi. Tags follow the ymkymx convention <branch>-<sha9>-<datetime>
plus a moving <branch>-latest. Registry defaults to the Gitea instance's
container registry, override with the REGISTRY variable.
2026-09-22 16:59:13 +08:00
minke.yu b081dd3d23 Merge branch 'main' into dsv41-pd 2026-09-22 14:56:35 +08:00
Kevin Mi 8ac19cc19f [AMD][Kimi-K3] Fix deferred KDA gate projection and update DCP cookbook (#39066)
PR Test (Arm64) / check-changes (push) Successful in 10s
PR Test (NPU) / set-image-config (push) Successful in 1s
PR Test (NPU) / Recommend tests from coverage (push) Skipped
PR Test (NPU) / check-changes (push) Successful in 12s
PR Test (sgl-router) / gate (push) Successful in 8s
PR Test (Xeon) / check-changes (push) Successful in 9s
PR Test (XPU) / check-changes (push) Successful in 16s
pr-test-arm64.yml / pr-gate (push) Successful in 3s
PR Test (Arm64) / pr-gate (push) Successful in 3s
pr-test-npu.yml / pr-gate (push) Successful in 2s
PR Test (NPU) / pr-gate (push) Successful in 2s
PR Test (sgl-router) / tier-1 — lint (push) Failing after 33s
PR Test (sgl-router) / tier-2 — build + test (push) Skipped
PR Test (sgl-router) / tier-3 — docker (placeholder) (push) Skipped
PR Test (sgl-router) / tier-3 — k8s integration (push) Skipped
PR Test (sgl-router) / tier-3 — e2e (push) Skipped
pr-test-xpu.yml / pr-gate (push) Successful in 2s
PR Test (XPU) / pr-gate (push) Successful in 2s
pr-test-xeon.yml / pr-gate (push) Successful in 3s
PR Test (Xeon) / pr-gate (push) Successful in 3s
PR Test (sgl-router) / finish (push) Successful in 1s
Lint / lint (push) Failing after 2m51s
PR Test (NPU) / base-a-test-1-npu-a2 (push) Canceled after 0s
PR Test (NPU) / base-b-test-1-npu-a3 (push) Canceled after 0s
PR Test (NPU) / base-b-test-2-npu-a3 (push) Canceled after 0s
PR Test (NPU) / base-b-test-4-npu-a3 (push) Canceled after 0s
PR Test (NPU) / base-b-test-8-npu-a3 (push) Canceled after 0s
PR Test (NPU) / base-b-test-16-npu-a3 (push) Canceled after 0s
PR Test (NPU) / multimodal-gen-test-1-npu-a3 (0) (push) Canceled after 0s
PR Test (NPU) / multimodal-gen-test-1-npu-a3 (1) (push) Canceled after 0s
PR Test (NPU) / multimodal-gen-test-4-npu-a3 (0) (push) Canceled after 0s
PR Test (NPU) / multimodal-gen-test-4-npu-a3 (1) (push) Canceled after 0s
PR Test (NPU) / base-c-test-acc-2-npu-a3 (push) Canceled after 0s
PR Test (NPU) / base-c-test-acc-16-npu-a3 (push) Canceled after 0s
PR Test (NPU) / base-c-test-perf-2-npu-a3 (push) Canceled after 0s
PR Test (NPU) / base-c-test-perf-16-npu-a3 (push) Canceled after 0s
PR Test (NPU) / Analyze failure report (push) Canceled after 0s
PR Test (NPU) / setup-covstub (push) Canceled after 0s
PR Test (NPU) / pr-test-npu-finish (push) Canceled after 0s
pr-test-npu.yml / run (${{ fromJson(inputs.partitions).arr }}) (push) Canceled after 0s
PR Test (XPU) / finish (push) Canceled after 0s
PR Test (Arm64) / build-test (push) Canceled after 0s
PR Test (XPU) / stage-a-test-1-gpu-xpu (push) Canceled after 0s
PR Test (XPU) / multimodal-gen-test-1-gpu-xpu (push) Canceled after 0s
PR Test (Xeon) / build-test (gnr, gnr, xeon-gnr, stage-a-tp-test-cpu-intel) (push) Canceled after 0s
PR Test (Xeon) / build-test (spr1, 0, 3, spr, xeon-spr, stage-a-test-cpu-intel,stage-b-test-cpu-intel) (push) Canceled after 0s
PR Test (Xeon) / build-test (spr2, 1, 3, spr, xeon-spr, stage-a-test-cpu-intel,stage-b-test-cpu-intel) (push) Canceled after 0s
PR Test (Xeon) / build-test (spr3, 2, 3, spr, xeon-spr, stage-a-test-cpu-intel,stage-b-test-cpu-intel) (push) Canceled after 0s
2026-09-22 06:35:12 +00:00
Mohammad Miadh AngkadandMohammad Angkad 4c81cd1b09 [KDA] Fix missing beta sigmoid in PTX prefill (#40685)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-09-21 23:24:38 -07:00
Khoa Pham bc22e1de9e [DSpark] Fix draft CUDA graph stream explosion (#40658) 2026-09-21 23:05:45 -07:00
Brayden ZhongandXinyuan Tong a0781f2714 [Docs] GLM-5.3/5.3-Flash cookbooks: enable reasoning/tool-call parsers by default via auto (#40497)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-09-22 13:54:08 +08:00