Files
sglang/python/sglang
minke.yu fa8ad18b5c
build-sglang-image / build (push) Failing after 1s
手工移植上游 #38348:hicache SWA prefetch fail-soft 降级
SWA 池 L3 取回短缺时不再整批丢弃 prefetch 结果:保留已取回的 Full KV
前缀,仅丢弃失败的 SWA 池(释放其 host buffer、从 comp_xfers 摘除),
load-back 时经既有 SWARebuild/RecoverSWAWithLockedFull 路径从 Full KV
重建窗口。Mamba 等不可重建池短缺时仍走原 all-or-nothing 整批丢弃。

配套改动(同 PR):
- swa.py validator:SWA 双层缺失但节点已 backuped(Full KV 在 host)时
  放行作为匹配边界;LOAD_BACK 收集遇双层缺失空洞时 break 而非 assert。
- prefetch_from_storage:用完整匹配前缀重算 last_hash,修复 anchor 节点
  无 host_value(异步 backup 未完成/降级嫁接)时哈希链锚错位。
- 适配差异:本树有 sidecar_hash_values 重构,belief 自愈沿用
  invalidation_hashes + trimmed_pages 语义;ongoing_prefetch 以
  CacheRequestHandle 为键,comp_xfers 改为显式参数传入检查函数。
- 单测同步更新为 fail-soft 语义(TP 一致性降级、失败池释放无泄漏、
  LOAD_BACK 锚定空洞停止收集)。
2026-10-10 07:23:15 +08:00
..
2026-09-22 14:56:35 +08:00

Code Structure

Folders

  • benchmark/: Benchmark implementations and dataset utilities.
  • cli/: Command-line interface commands and entrypoints.
  • kernels/: Kernel interfaces, implementations, selection, and debugging utilities shared by the runtimes.
  • lang/: Deprecated language frontend that is no longer actively maintained.
  • multimodal_gen/: Core runtime for image, video, and audio generation models, most of which are diffusion models.
  • srt/: Core runtime for autoregressive language models. (SRT = SGLang Runtime.)
  • test/: Shared test and evaluation utilities.

Files

  • README.md: This package structure overview.
  • __init__.py: Package initialization and public Python APIs.
  • bench_offline_throughput.py: Deprecated wrapper for the offline throughput benchmark.
  • bench_one_batch.py: Deprecated wrapper for the one-batch benchmark.
  • bench_one_batch_server.py: Deprecated wrapper for the server-based one-batch benchmark.
  • bench_serving.py: Deprecated wrapper for the online serving benchmark.
  • check_env.py: Environment and dependency diagnostics.
  • compile_deep_gemm.py: DeepGEMM kernel precompilation entrypoint.
  • launch_server.py: Compatibility entrypoint for launching an inference server.
  • profiler.py: Client entrypoint for collecting server profiling traces.
  • utils.py: Common package utilities.
  • version.py: Public package version resolution.