12 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
minke.yu	4f0ca95477	deploy: b300 build scripts with pip cache mount + no-build-isolation fast path ... Base image already has torch 2.13, but pyproject build-system.requires pulls it into pip's isolated build env on every build (~6GB via throttled mirror, 47min). --no-build-isolation reuses the base env; build.sh picks Dockerfile.fast (--no-deps) when python/pyproject.toml is unchanged.	2026-09-24 13:11:41 +08:00
minke.yu	b91137ab98	scheduler: CP-symmetric idle check for health-check admission ... Health-check admit/skip used is_fully_idle(), which includes rank-local hicache drain queues; ranks diverge right after activity, so one rank dispatched the health-check generate while others piggyback-skipped, deadlocking CP (hicache drain all_reduce vs CP request broadcast). Seen on cp2/cp4 + hicache L3 after router health checks. Recovered from b300 /data/ymk/sglang working copy (uncommitted WIP).	2026-09-24 12:01:57 +08:00
minke.yu	67d8368a84	deploy: archive b300 ds41 compose files; skip CI for deploy/ and .gitea/ changes	2026-09-24 11:49:37 +08:00
minke.yu	db7d2cb7db	fix: vision check uses get_parallel().pp_group (get_pp_group undefined in this tree)	2026-09-23 18:05:57 +08:00
minke.yu	7925735a3e	ci: also trigger image build on dsv41-pd-visioncp pushes	2026-09-23 16:46:06 +08:00
minke.yu	104218d9ed	ci: tag builds with -ci-<UTC datetime> suffix (naming rule: <branch>-<sha9>-<ci|local>-<UTC>)	2026-09-22 20:37:57 +08:00
minke.yu	92632a60ba	ci: drop gha cache backend (runner cache server 404s on this instance)	2026-09-22 18:49:20 +08:00
minke.yu	4b3b367b63	ci: hardcode Gitea registry (vars context falls back to docker.io on this instance)	2026-09-22 18:23:35 +08:00
minke.yu	a78da9b524	ci: Gitea Actions workflow to build sglang image on push to dsv41-pd ... Overseas node: original docker.io base (lmsysorg/sglang:dev-dsv41) and default pypi. Tags follow the ymkymx convention <branch>-<sha9>-<datetime> plus a moving <branch>-latest. Registry defaults to the Gitea instance's container registry, override with the REGISTRY variable.	2026-09-22 16:59:13 +08:00
minke.yu	b081dd3d23	Merge branch 'main' into dsv41-pd	2026-09-22 14:56:35 +08:00
minke.yu	c74a4037fb	mooncake: probe pointers via cuda.bindings instead of ctypes libcudart ... The ctypes-based probing loaded a second libcudart copy into the process, which corrupted CUDA/runtime state and segfaulted the scheduler processes shortly after registration (reproducible on prefill ranks, intermittent on decode). cuda.bindings.runtime.cudaPointerGetAttributes is a properly typed binding and boots cleanly on both roles.	2026-09-21 12:46:25 +08:00
minke.yu	b963295489	mooncake: support host-memory payloads with intra-node NVLink transport ... IntraNodeNvlinkTransport can only register/reach device memory. sglang's register list includes host regions (aux buffers, some state components); one host region failed the whole registerLocalMemoryBatch and the engine rolled back every region, leaving the segment descriptor empty so all KV transfers failed with 'Requested address ... not found'. When the intra-node NVLink transport is active (MC_INTRANODE_NVLINK etc.): - register only device-memory regions (probe via cudaPointerGetAttributes) - route transfer blocks whose local source is host memory over the ordered per-endpoint zmq channel (new STATE_DATA message) instead of the engine; the receiver validates and writes the block into its buffer, relying on the same ordering guarantee as the aux TCP path - force aux over the TCP socket path automatically	2026-09-21 11:53:26 +08:00