Files

T

Lianmin ZhengandJialin Ouyang f5a1434700 [HiCache] Document transfer arguments (#40239 )

Clarify the legacy host_indices argument and label Mamba test arguments,
including the current staging_tokens parameter. Executable code is unchanged.

Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>

2026-09-18 14:08:19 -07:00

src

[HiCache] Document transfer arguments (#40239 )

2026-09-18 14:08:19 -07:00

.gitignore

[Rust] Rename mem-cache to sglang-radix-tree (#37290 )

2026-09-01 03:03:44 -07:00

Cargo.lock

[Rust] Rename mem-cache to sglang-radix-tree (#37290 )

2026-09-01 03:03:44 -07:00

Cargo.toml

[Rust] Rename mem-cache to sglang-radix-tree (#37290 )

2026-09-01 03:03:44 -07:00

README.md

[Rust] Rename mem-cache to sglang-radix-tree (#37290 )

2026-09-01 03:03:44 -07:00

torch_2_13_compat.h

[Rust] Rename mem-cache to sglang-radix-tree (#37290 )

2026-09-01 03:03:44 -07:00

README.md

sglang-radix-tree

Rust tree core for the Unified Radix Cache, covering Full attention, sliding window attention, and Mamba components. It implements the tree side of the UnifiedTreeCoreInterface split — match/insert walks, node arena, locks, eviction walks, HiCache backup/load-back specs, and KV events — behind a PyO3 binding, while the cache orchestration stays in Python.

Usage

Select the backend with:

SGLANG_UNIFIED_RADIX_TREE_CORE_BACKEND=rust

SGLang wheels bundle the production extension. A source checkout falls back to the shared fingerprinted Rust-extension cache; it never writes a shared object into the Python package. LibTorch and the Python headers come from the running interpreter's PyTorch install. PyTorch 2.11 through 2.13 are accepted explicitly, and torch_2_13_compat.h covers two alignment APIs removed in PyTorch 2.13.

Development

# Build (libtorch from the installed torch package):
cd rust/sglang-radix-tree
LIBTORCH_USE_PYTORCH=1 \
  LIBTORCH_BYPASS_VERSION_CHECK=1 \
  CXXFLAGS="-include $PWD/torch_2_13_compat.h" \
  cargo build --release --locked --features python-extension

# Native tests do not enable pyo3's extension-module feature:
TORCH_ROOT=$(python3 -c 'import pathlib, torch; print(pathlib.Path(torch.__file__).parent)')
LIBTORCH_USE_PYTORCH=1 LIBTORCH_BYPASS_VERSION_CHECK=1 \
  CXXFLAGS="-include $PWD/torch_2_13_compat.h" \
  LD_LIBRARY_PATH="$TORCH_ROOT/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" \
  cargo test --locked

The inspection Cargo feature adds white-box methods for the shared Python/Rust cache suite. Production wheels do not enable it.

Unit tests live in src/tests/, mirroring the source layout one file per module (wired via #[cfg(test)] #[path = ...]), so implementation files stay free of inline test blocks.

Supported component sets are [Full], [Full, SWA], [Full, Mamba], and [Full, SWA, Mamba].