From c98d31143d7e017cac3199fe4b4e52b435a99ba8 Mon Sep 17 00:00:00 2001 From: Yaochen Han <48639761+Alisehen@users.noreply.github.com> Date: Sat, 27 Jun 2026 00:40:33 +0800 Subject: [PATCH] update quantization code owner and document quantization contributions (#26784) Co-authored-by: ronnie_zheng --- .github/CODEOWNERS | 5 +- docs_new/docs.json | 3 +- docs_new/docs/developer_guide/overview.mdx | 1 + .../quantization_contribution_guide.mdx | 96 +++++++++++++++++++ 4 files changed, 102 insertions(+), 3 deletions(-) create mode 100644 docs_new/docs/developer_guide/quantization_contribution_guide.mdx diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 49fe82690..48df903ff 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -32,7 +32,8 @@ /python/sglang/srt/hardware_backend/mlx @yeahdongcn /python/sglang/srt/hardware_backend/musa @yeahdongcn /python/sglang/srt/hardware_backend/npu @ping1jing2 @iforgetmyname @whybeyoung -/python/sglang/srt/hardware_backend/npu/quantization @OrangeRedeng @TamirBaydasov @iforgetmyname +/python/sglang/srt/hardware_backend/gpu/quantization @Alisehen +/python/sglang/srt/hardware_backend/npu/quantization @OrangeRedeng @TamirBaydasov @Alisehen /python/sglang/srt/layers @merrymercy @Ying1123 @Fridge003 @ispobock @HaiShaw @ch-wan @BBuf @Edwardf0t1 /python/sglang/srt/layers/attention @merrymercy @Fridge003 @ispobock @Qiaolin-Yu @hebiao064 @HaiShaw /python/sglang/srt/layers/attention/fla @yizhang2077 @hebiao064 @yuan-luo @@ -40,7 +41,7 @@ /python/sglang/srt/layers/attention/mamba @yizhang2077 @hebiao064 /python/sglang/srt/layers/attention/dsa @1am9trash @hubertlu-tw @kkHuang-amd @HaiShaw @Fridge003 @YAMY1234 @rainj-me /python/sglang/srt/layers/attention/vision.py @mickqian @yuan-luo @yhyang201 -/python/sglang/srt/layers/quantization @ch-wan @BBuf @Edwardf0t1 @FlamingoPg @AniZpZ @HaiShaw @b8zhong +/python/sglang/srt/layers/quantization @ch-wan @BBuf @Edwardf0t1 @FlamingoPg @AniZpZ @HaiShaw @b8zhong @OrangeRedeng @TamirBaydasov @Alisehen /python/sglang/srt/layers/quantization/quark @kkHuang-amd @yichiche @hubertlu-tw @1am9trash @BowenBao /python/sglang/srt/lora @Ying1123 @Fridge003 @lifuhuang @yushengsu-thu @jybsuper /python/sglang/srt/managers @merrymercy @Ying1123 @hnyls2002 @xiezhq-hermann diff --git a/docs_new/docs.json b/docs_new/docs.json index 856dc534b..703dc4236 100644 --- a/docs_new/docs.json +++ b/docs_new/docs.json @@ -809,7 +809,8 @@ "group": "Development", "pages": [ "docs/developer_guide/development_guide_using_docker", - "docs/developer_guide/development_jit_kernel_guide" + "docs/developer_guide/development_jit_kernel_guide", + "docs/developer_guide/quantization_contribution_guide" ] }, { diff --git a/docs_new/docs/developer_guide/overview.mdx b/docs_new/docs/developer_guide/overview.mdx index 1ad72a0c4..e39d72621 100644 --- a/docs_new/docs/developer_guide/overview.mdx +++ b/docs_new/docs/developer_guide/overview.mdx @@ -6,6 +6,7 @@ description: Contributing to SGLang — development setup, benchmarking, and eva - [Contribution Guide](./contribution_guide) - [Development Guide (Docker)](./development_guide_using_docker) - [JIT Kernels](./development_jit_kernel_guide) +- [Quantization Contribution Guide](./quantization_contribution_guide) - [Benchmark and Profiling](./benchmark_and_profiling) - [Bench Serving](./bench_serving) - [Evaluating New Models](./evaluating_new_models) diff --git a/docs_new/docs/developer_guide/quantization_contribution_guide.mdx b/docs_new/docs/developer_guide/quantization_contribution_guide.mdx new file mode 100644 index 000000000..8637608ed --- /dev/null +++ b/docs_new/docs/developer_guide/quantization_contribution_guide.mdx @@ -0,0 +1,96 @@ +--- +title: "Quantization Contribution Guide" +metatags: + description: "Guidance for contributing SGLang quantization methods, schemes, backend kernels, tests, and validation results." +--- + +This guide describes how to add or refactor quantization support in SGLang. It focuses on the common structure used by weight-only and weight-activation quantization methods such as AWQ, GPTQ, compressed-tensors, ModelSlim, Quark, and related backend kernels. + +## Design Goals + +Quantization code should keep the quantization format semantics separate from hardware-specific execution. This makes it easier to add new formats, reuse kernels across formats, and review platform-specific changes independently. + +Follow the architecture proposed in [Quantization Modifications](https://github.com/sgl-project/sglang/issues/15194): + +- **Config**: parses model and runtime quantization parameters, validates supported options, and selects the proper scheme. +- **Scheme**: owns quantized weight creation, weight loading, post-processing, and quantized layer wiring for Linear, MoE, embedding, or other module types. +- **Backend kernel**: wraps hardware-specific execution, layout conversion, kernel selection, and kernel calls for GPU (CUDA/HIP/XPU), NPU, or other backends. + +Avoid putting config parsing, weight loading, and backend kernel calls in a single monolithic file. If a method needs multiple formats or backends, add a package under `python/sglang/srt/layers/quantization//` and split schemes into `schemes/`. + +## Recommended File Layout + +Use this layout for a quantization method that has multiple schemes or backend-specific execution paths: + +```text +python/sglang/srt/layers/quantization// + __init__.py + .py + schemes/ + __init__.py + _scheme.py + _linear.py + _moe.py + _.py +``` + +Backend kernels should live under the hardware backend they target: + +```text +python/sglang/srt/hardware_backend/gpu/quantization/_kernels.py +python/sglang/srt/hardware_backend/npu/quantization/_kernels.py +``` + +Keep shared method selection in the quantization package and keep backend imports narrow. This prevents circular imports and keeps non-target platforms from importing unavailable kernel dependencies. + +## Adding or Refactoring a Quantization Method + +1. Define the config entry point and register it through `python/sglang/srt/layers/quantization/__init__.py` when needed. +2. Add explicit scheme selection helpers such as `get_linear_scheme` and `get_moe_scheme`. +3. Move layer-specific weight creation and weight loading into scheme classes. +4. Move GPU (CUDA/HIP/XPU), NPU, or other hardware kernel calls into backend kernel modules. +5. Keep Linear, MoE, embedding, and non-linear module handling explicit. Do not assign a Linear quantization method to a module type that needs different semantics. +6. Preserve compatibility for existing quantized checkpoints and runtime flags. +7. Add tests that cover both config parsing and execution paths touched by the change. + +For examples, see the AWQ and GPTQ refactors: + +- [PR #21126](https://github.com/sgl-project/sglang/pull/21126): splits AWQ schemes, weight initialization, and backend kernel calls. +- [PR #26402](https://github.com/sgl-project/sglang/pull/26402): applies the same scheme/kernel split to GPTQ. + +## Tests and Validation + +Quantization changes can affect both accuracy and performance. Include validation that matches the blast radius of the change. + +For Python-only structure changes: + +```bash +ruff check +git diff --check +``` + +For quantized model behavior: + +- Launch at least one representative model for each touched quantization method. +- Send a `/generate` request and confirm the output path succeeds. +- Run an accuracy sanity test if the change can affect numerics. +- Include warmup-aware benchmark results when the change affects kernel calls, layout conversion, or dispatch. + +For backend-specific changes: + +- Validate GPU changes on a supported GPU environment (NVIDIA, AMD, or Intel). +- Validate NPU changes on a supported Ascend environment. +- Include the exact model, quantization flag, backend flag, hardware, and command used in the PR description. + +## PR Checklist + +Before requesting review, make sure the PR description includes: + +- The quantization method and backend paths changed. +- The issue, design proposal, or roadmap item the PR follows. +- Any compatibility notes for existing checkpoints or flags. +- Accuracy results when model outputs can change. +- Benchmark or profiling results when runtime performance can change. +- The exact local checks and model launch tests that were run. + +Use the general [Contribution Guide](./contribution_guide) for source setup, formatting, unit tests, CI triggering, and review process details.