Build Rust extensions on demand in source checkouts (#34994)

This commit is contained in:
Lianmin Zheng
2026-08-16 14:58:06 -07:00
committed by GitHub
parent 0e231d365a
commit 67e12131df
39 changed files with 880 additions and 109 deletions
+7 -6
View File
@@ -6,9 +6,9 @@ GIL-released.
Built two ways:
- **PyO3 extension** `sglang.srt.multimodal._core` (features `python,parallel`,
requested by the wheel build) via setuptools-rust when installing sglang —
used by Python processors and parity tests.
- **PyO3 extension** `sglang.srt.rust_extensions._multimodal` (features
`python,parallel`, requested by the wheel build) via setuptools-rust when
installing sglang — used by Python processors and parity tests.
- **Pure-Rust `rlib`** (default features, i.e. neither) linked by
`sglang-server`'s MM worker path — that copy needs no pyo3, no libpython, and
no rayon: it spawns no threads and runs inline on the calling thread, because
@@ -19,7 +19,7 @@ Built two ways:
```
src/
├── lib.rs # module root; PyO3 module (_core) feature-gated
├── lib.rs # module root; PyO3 module (_multimodal) feature-gated
├── pipeline.rs # the server-pipeline contract: MmFamilyProcessor
│ # trait + the carriers (Tensor, TokenLayout, ...)
├── driver.rs # model-independent request driver (fetch →
@@ -115,7 +115,7 @@ un-stripped URL to `open()`).
## Python API
```python
from sglang.srt.multimodal._core import common, inkling
from sglang.srt.rust_extensions._multimodal import common, inkling
# Common (model-agnostic)
common.resize_rgb(arr, out_w, out_h)
@@ -174,7 +174,8 @@ impl ImageProcessorSpec for MyModelProcessor {
4. Wire up in `src/lib.rs`: `mod my_model;` and `my_model::register(m)?;`.
5. Add Python processor class that calls `from sglang.srt.multimodal._core import my_model`.
5. Add Python processor class that calls
`from sglang.srt.rust_extensions._multimodal import my_model`.
## Available transform primitives (`common::transforms`)