da343ba5d1 (#42604 cherry-pick) bumped sglang-kernel to 0.4.9, whose
wheels are compiled against torch 2.14 (requires_dist torch==2.14.1 and
references c10::cuda::CUDAErrorLogCapture, absent in torch 2.13).
Import died with undefined symbol on this torch-2.13 branch.
sgl-deep-gemm 0.2.2 has no torch pin and stays.
The vision-CP cherry-pick stored (idx, item, token_count) 3-tuples into
PerImageRequestInfo.overlapping, but the field and its consumers
(_assemble_per_image_chunk, phase-1a) use (idx, item, start, end) with
cache key (hash, end-start+1). First owner-mode image request died with
'not enough values to unpack (expected 4, got 3)'.
fd86688428 (vision CP single-owner encoding) cherry-pick used
_item_overlap in _collect_image_span_requests without the definition;
first multimodal prefill request killed every scheduler with NameError.
fp4_index_logits_paged padded rows to 8 floats (32B) and
fp4_index_logits_candidates did not pad at all, but the tvm-ffi
deep_select topk kernel enforces kInputAlignmentBytes=128 on the input
row stride. bs>1 decode/CUDA-graph capture died with
'Tensor stride for dimension 0 is not aligned to 128 bytes'.
Candidates kernel now takes a separate OUT_STRIDE constexpr.
Health-check admit/skip used is_fully_idle(), which includes rank-local
hicache drain queues; ranks diverge right after activity, so one rank
dispatched the health-check generate while others piggyback-skipped,
deadlocking CP (hicache drain all_reduce vs CP request broadcast).
Seen on cp2/cp4 + hicache L3 after router health checks.
Recovered from b300 /data/ymk/sglang working copy (uncommitted WIP).
(cherry picked from commit b91137ab98)
- unified_radix_cache.query_storage_hit_length: build PREFETCH-phase
pool_transfers (comp prepare_prefetch + build_hicache_transfers +
sidecars) so DeepSeek V4's logical-anchor KV goes through
batch_exists_v2 side-pool existence instead of v1 batch_exists which
always misses (no physical KV objects are stored for V4).
- scheduler.handle_query_cache_hit: allow L3 query when hicache storage
backend is attached (not only decode-hicache mode; SWA models forbid
decode radix+hicache so P nodes could never query L3); construct
RadixKey from array('q') to match tree node key types.
- disaggregation/prefill: record_prefill_drain on final chunk in the
disagg-prefill result path (it bypasses batch_result_processor, so
drain_rate_tokens_per_s stayed 0 on P nodes).
- deploy: 3p1d l3test compose against the real mooncake store stack.
(cherry picked from commit e7dc175e94)
Source: https://github.com/sgl-project/sglang/pull/40219 (applied via GitHub .diff, 3way)
Adaptations for v0.5.21 base:
- attn_dp_enabled_of(cfg) -> cfg.enable_dp_attention (#41818 not in base)
- dropped _validate_joint_schema_request call (#42721 Clef, not in base)
- moved Backend import before the TRT-LLM prefill-graph check
Source: https://github.com/sgl-project/sglang/pull/40945 (applied via GitHub .diff, 3way)
Adaptation: PR base predates #41659 which removed the Selection struct;
ported sm90_decode.py and call sites to the inputs.out_page_indices interface
(raw_indices=None on decode paths, reset via inputs.reset_outputs()).
Source: https://github.com/sgl-project/sglang/pull/40860 (applied via GitHub .diff, 3way)
Conflicts resolved: kept #41658 padded-return trim in fp4_index_logits_paged,
took PR side for candidate_blocks branches, SM90 graph gate, deep_select test param.
Skipped: test_dsv41_candidate_graph_variants.py (file not in v0.5.21 base).