Build Rust extensions on demand in source checkouts (#34994)
This commit is contained in:
@@ -6,9 +6,9 @@ GIL-released.
|
||||
|
||||
Built two ways:
|
||||
|
||||
- **PyO3 extension** `sglang.srt.multimodal._core` (features `python,parallel`,
|
||||
requested by the wheel build) via setuptools-rust when installing sglang —
|
||||
used by Python processors and parity tests.
|
||||
- **PyO3 extension** `sglang.srt.rust_extensions._multimodal` (features
|
||||
`python,parallel`, requested by the wheel build) via setuptools-rust when
|
||||
installing sglang — used by Python processors and parity tests.
|
||||
- **Pure-Rust `rlib`** (default features, i.e. neither) linked by
|
||||
`sglang-server`'s MM worker path — that copy needs no pyo3, no libpython, and
|
||||
no rayon: it spawns no threads and runs inline on the calling thread, because
|
||||
@@ -19,7 +19,7 @@ Built two ways:
|
||||
|
||||
```
|
||||
src/
|
||||
├── lib.rs # module root; PyO3 module (_core) feature-gated
|
||||
├── lib.rs # module root; PyO3 module (_multimodal) feature-gated
|
||||
├── pipeline.rs # the server-pipeline contract: MmFamilyProcessor
|
||||
│ # trait + the carriers (Tensor, TokenLayout, ...)
|
||||
├── driver.rs # model-independent request driver (fetch →
|
||||
@@ -115,7 +115,7 @@ un-stripped URL to `open()`).
|
||||
## Python API
|
||||
|
||||
```python
|
||||
from sglang.srt.multimodal._core import common, inkling
|
||||
from sglang.srt.rust_extensions._multimodal import common, inkling
|
||||
|
||||
# Common (model-agnostic)
|
||||
common.resize_rgb(arr, out_w, out_h)
|
||||
@@ -174,7 +174,8 @@ impl ImageProcessorSpec for MyModelProcessor {
|
||||
|
||||
4. Wire up in `src/lib.rs`: `mod my_model;` and `my_model::register(m)?;`.
|
||||
|
||||
5. Add Python processor class that calls `from sglang.srt.multimodal._core import my_model`.
|
||||
5. Add Python processor class that calls
|
||||
`from sglang.srt.rust_extensions._multimodal import my_model`.
|
||||
|
||||
## Available transform primitives (`common::transforms`)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user