Remove legacy Sphinx docs/ and finish the Mintlify cutover (#28964)
This commit is contained in:
zijiexia
@@ -97,7 +97,7 @@ def gen(
|
||||
regex: Optional[str] = None,
|
||||
json_schema: Optional[str] = None,
|
||||
):
|
||||
"""Call the model to generate. See the meaning of the arguments in docs/backend/sampling_params.md"""
|
||||
"""Call the model to generate. See the meaning of the arguments in docs_new/docs/basic_usage/sampling_params.mdx"""
|
||||
|
||||
if choices:
|
||||
return SglSelect(
|
||||
|
||||
@@ -473,7 +473,7 @@ class SglGen(SglExpr):
|
||||
regex: Optional[str] = None,
|
||||
json_schema: Optional[str] = None,
|
||||
):
|
||||
"""Call the model to generate. See the meaning of the arguments in docs/backend/sampling_params.md"""
|
||||
"""Call the model to generate. See the meaning of the arguments in docs_new/docs/basic_usage/sampling_params.mdx"""
|
||||
super().__init__()
|
||||
self.name = name
|
||||
self.sampling_params = SglSamplingParams(
|
||||
|
||||
@@ -65,7 +65,7 @@ See existing Modular examples: `QwenImagePipeline` (uses `add_standard_t2i_stage
|
||||
| Model/VAE/DiT configs | `python/sglang/multimodal_gen/configs/models/dits/`, `vaes/`, `encoders/` |
|
||||
| Central registry | `python/sglang/multimodal_gen/registry.py` |
|
||||
| Model component registry | `python/sglang/multimodal_gen/runtime/models/registry.py` |
|
||||
| Current support list | `docs/diffusion/compatibility_matrix.md` |
|
||||
| Current support list | `docs_new/docs/sglang-diffusion/compatibility_matrix.mdx` |
|
||||
|
||||
---
|
||||
|
||||
|
||||
+2
-2
@@ -32,7 +32,7 @@ framework-specific optimization workflow.
|
||||
- `python/sglang/jit_kernel/norm.py`
|
||||
- `python/sglang/multimodal_gen/runtime/platforms/cuda.py`
|
||||
- `python/sglang/multimodal_gen/runtime/layers/attention/selector.py`
|
||||
- `docs/diffusion/performance/attention_backends.md` (repo root)
|
||||
- `docs_new/docs/sglang-diffusion/attention_backends.mdx` (repo root)
|
||||
|
||||
**Core Fusion Patterns**
|
||||
|
||||
@@ -121,7 +121,7 @@ framework-specific optimization workflow.
|
||||
- ROCm falls back to native.
|
||||
|
||||
2. Attention backend selection (FlashAttention, Sage, SDPA)
|
||||
- Locations: `platforms/cuda.py`, `attention/selector.py`, `docs/diffusion/performance/attention_backends.md`
|
||||
- Locations: `platforms/cuda.py`, `attention/selector.py`, `docs_new/docs/sglang-diffusion/attention_backends.mdx`
|
||||
- Behavior: CUDA prefers FlashAttention (FA3/FA4) when supported, otherwise Torch SDPA. Force via `--attention-backend` or `global_force_attn_backend`.
|
||||
|
||||
3. FlashInfer RoPE (Q/K inplace)
|
||||
|
||||
+5
-5
@@ -25,7 +25,7 @@ This skill owns the ModelOpt-to-SGLang bridge. It is not a generic kernel-tuning
|
||||
- For multi-transformer pipelines, use per-component overrides when different components need different checkpoints.
|
||||
- For B200 NVFP4 validation, keep backend-sensitive environment variables explicit. Wan2.2 NVFP4 is commonly validated with `SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND=cudnn`; benchmark the default CUTLASS path separately if that is what you are evaluating.
|
||||
- When a branch is missing the validated helper tools, refresh `python/sglang/multimodal_gen/tools/build_modelopt_fp8_transformer.py`, `python/sglang/multimodal_gen/tools/build_modelopt_nvfp4_transformer.py`, and `python/sglang/multimodal_gen/tools/compare_diffusion_trajectory_similarity.py` instead of inventing one-off scripts elsewhere.
|
||||
- After validating a new ModelOpt quant path, update the ModelOpt support matrix in `docs/diffusion/quantization.md` before closing the task.
|
||||
- After validating a new ModelOpt quant path, update the ModelOpt support matrix in `docs_new/docs/sglang-diffusion/quantization.mdx` before closing the task.
|
||||
|
||||
## Read First
|
||||
|
||||
@@ -71,9 +71,9 @@ Treat a new family, a new precision, or a new checkpoint layout as unsupported u
|
||||
Current B200 CI also contains an Ideogram4 NVFP4 native load case
|
||||
(`ideogram4_nvfp4_t2i` via `Comfy-Org/Ideogram-4`). Treat that as source
|
||||
evidence for an existing NVFP4 path, but do not expand the ModelOpt support
|
||||
matrix to Ideogram4 unless `docs/diffusion/quantization.md` is updated with the
|
||||
matrix to Ideogram4 unless `docs_new/docs/sglang-diffusion/quantization.mdx` is updated with the
|
||||
exact checkpoint, loader path, quality check, and benchmark scope.
|
||||
Before writing CLI examples, re-read the active branch's `docs/diffusion/quantization.md`: FLUX.2 NVFP4 is an official `black-forest-labs/*` repo rather than a `lmsys/*` converted repo, and its preferred flag depends on the current documented loader flow. Use `--transformer-path` for a component override directory with `config.json`; use `--transformer-weights-path` when the repo or path should be probed as raw weights.
|
||||
Before writing CLI examples, re-read the active branch's `docs_new/docs/sglang-diffusion/quantization.mdx`: FLUX.2 NVFP4 is an official `black-forest-labs/*` repo rather than a `lmsys/*` converted repo, and its preferred flag depends on the current documented loader flow. Use `--transformer-path` for a component override directory with `config.json`; use `--transformer-weights-path` when the repo or path should be probed as raw weights.
|
||||
|
||||
B200 CI coverage can include loose BF16-vs-quantized quality checks. Inspect the active branch's `run_suite.py` before assuming they are part of the suite; mainline and feature branches may differ. Those checks are intended to catch blank, corrupted, or obviously divergent images, not exact image parity.
|
||||
|
||||
@@ -100,7 +100,7 @@ accuracy check, and benchmark scope are validated on the active branch.
|
||||
|
||||
## Documentation Maintenance
|
||||
|
||||
- Keep the validated ModelOpt support matrix in `docs/diffusion/quantization.md`.
|
||||
- Keep the validated ModelOpt support matrix in `docs_new/docs/sglang-diffusion/quantization.mdx`.
|
||||
- Each row should record the validated scope, the Hugging Face repo or path for the quantized DiT weights, and the key caveats.
|
||||
- If the quantized DiT weights are not published yet, write `unpublished` explicitly instead of leaving the field blank.
|
||||
|
||||
@@ -402,6 +402,6 @@ When documenting results:
|
||||
| `tools/build_modelopt_fp8_transformer.py` | Build an SGLang-loadable FP8 transformer from a ModelOpt export |
|
||||
| `tools/build_modelopt_nvfp4_transformer.py` | Build mixed BF16+NVFP4 transformer directories when a family needs preserved BF16 layers |
|
||||
| `tools/compare_diffusion_trajectory_similarity.py` | reduced deterministic BF16-vs-quantized validation |
|
||||
| `docs/diffusion/quantization.md` | public ModelOpt support matrix and CLI examples |
|
||||
| `docs_new/docs/sglang-diffusion/quantization.mdx` | public ModelOpt support matrix and CLI examples |
|
||||
| `test/server/testcase_configs.py` | reusable ModelOpt testcase constants, thresholds, and helpers |
|
||||
| `test/server/gpu_cases.py` | concrete GPU and B200 ModelOpt CI case lists |
|
||||
|
||||
@@ -42,7 +42,7 @@ These options are intended to preserve output quality. In practice, some paths (
|
||||
| **FSDP Inference** | `--use-fsdp-inference` | Uses PyTorch FSDP to shard model weights across GPUs with prefetch. Low latency, low VRAM. | Reduces per-GPU VRAM | Mutually exclusive with `--dit-layerwise-offload`. More overhead than SP on high-bandwidth interconnects. |
|
||||
| **CPU Offload (components)** | `--text-encoder-cpu-offload`, `--image-encoder-cpu-offload`, `--vae-cpu-offload`, `--dit-cpu-offload` | Offloads specific pipeline components to CPU when not in use. | Reduces peak VRAM | Adds H2D transfer latency when the component is needed. Auto-enabled for low-VRAM GPUs (<30 GB). **Tip:** after the first request completes, the console prints a peak VRAM analysis with suggestions on which offload flags can be safely disabled — look for the `"Components that could stay resident"` log line. |
|
||||
| **Pin CPU Memory** | `--pin-cpu-memory` | Uses pinned (page-locked) memory for CPU offload transfers. | Faster H2D transfers | Slightly higher host memory usage. Enabled by default; disable only as workaround for CUDA errors. |
|
||||
| **Attention Backend (lossless)** | `--attention-backend fa` | Selects a lossless attention kernel for SGLang-native pipelines: `fa` (FlashAttention 2/3/4 alias) or `torch_sdpa`. | FA is usually faster than SDPA on long sequences | FA requires compatible GPU (Ampere+). For `--backend diffusers`, valid backend names differ; use the names documented in `docs/diffusion/performance/attention_backends.md`. |
|
||||
| **Attention Backend (lossless)** | `--attention-backend fa` | Selects a lossless attention kernel for SGLang-native pipelines: `fa` (FlashAttention 2/3/4 alias) or `torch_sdpa`. | FA is usually faster than SDPA on long sequences | FA requires compatible GPU (Ampere+). For `--backend diffusers`, valid backend names differ; use the names documented in `docs_new/docs/sglang-diffusion/attention_backends.mdx`. |
|
||||
| **Parallel Folding** | *(automatic when SP > 1)* | Reuses the SP process group as TP for the T5 text encoder, so text encoding is parallelized "for free". | Faster text encoding on multi-GPU | Automatic; no user action needed. Only applies to T5-based pipelines. |
|
||||
|
||||
---
|
||||
|
||||
@@ -4,7 +4,7 @@ A ComfyUI plugin for integrating with SGLang Diffusion server, supporting image
|
||||
|
||||
## Installation
|
||||
|
||||
1. **Install SGLang**: Follow the [Installation Guide](../../../../../docs/diffusion/installation.md) to install `sglang[diffusion]`.
|
||||
1. **Install SGLang**: Follow the [Installation Guide](https://docs.sglang.io/docs/sglang-diffusion/installation) to install `sglang[diffusion]`.
|
||||
2. **Install Plugin**: Copy this entire directory (`ComfyUI_SGLDiffusion`) to your ComfyUI `custom_nodes/` folder.
|
||||
3. **Restart ComfyUI**: Restart ComfyUI to load the plugin.
|
||||
|
||||
|
||||
@@ -31,7 +31,7 @@ class AscendTransferEngine(MooncakeTransferEngine):
|
||||
):
|
||||
if import_error is not None:
|
||||
logger.warning(
|
||||
"Please install memfabric_hybrid, for details, see docs/backend/pd_disaggregation.md"
|
||||
"Please install memfabric_hybrid, for details, see docs_new/docs/advanced_features/pd_disaggregation.mdx"
|
||||
)
|
||||
raise import_error
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ Ollama API compatibility for SGLang, plus a Smart Router for intelligent routing
|
||||
|
||||
## Ollama API
|
||||
|
||||
For basic Ollama API usage with SGLang (CLI and Python examples), see the [Ollama API documentation](https://sgl-project.github.io/basic_usage/ollama_api.html).
|
||||
For basic Ollama API usage with SGLang (CLI and Python examples), see the [Ollama API documentation](https://docs.sglang.io/docs/basic_usage/ollama_api).
|
||||
|
||||
## Smart Router
|
||||
|
||||
|
||||
@@ -76,8 +76,7 @@ class SamplingParams(msgspec.Struct, kw_only=True, omit_defaults=True):
|
||||
"""
|
||||
The sampling parameters.
|
||||
|
||||
See docs/backend/sampling_params.md or
|
||||
https://docs.sglang.io/backend/sampling_params.html
|
||||
See docs_new/docs/basic_usage/sampling_params.mdx
|
||||
for the documentation.
|
||||
"""
|
||||
|
||||
|
||||
Reference in New Issue
Block a user