--- title: Qwen3.6 metatags: description: "Deploy Qwen3.6 with SGLang - open-weight multimodal series with a 35B MoE (3B active) variant and a 27B dense variant, hybrid reasoning, tool calling, MTP, and long-context support." --- import { Qwen36Deployment } from '/src/snippets/autoregressive/qwen36-deployment.jsx'; ## 1. Model Introduction The Qwen3.6 series is developed by Alibaba. Built on direct feedback from the community, Qwen3.6 prioritizes stability and real-world utility, delivering substantial upgrades in agentic coding and thinking preservation. Two size/sparsity variants are released: - [Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) — **Sparse MoE** (35B total, 3B active) on a Gated Delta Networks backbone. - [Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) — **Dense** hybrid GDN; smaller weights footprint, single-GPU friendly. Both variants share the same hybrid reasoning, tool-calling, and multimodal interface and natively handle context lengths of up to 262,144 tokens, extensible to over 1M tokens. **Key Features:** - **Agentic Coding**: Handles frontend workflows and repository-level reasoning with greater fluency and precision - **Thinking Preservation**: New option to retain reasoning context from historical messages, streamlining iterative development - **Efficient Hybrid Architecture**: Gated Delta Networks backbone; sparse MoE (35B / 3B active) or dense 27B variant - **Hybrid Reasoning**: Thinking mode enabled by default with step-by-step reasoning, can be disabled for direct responses - **Tool Calling**: Built-in tool calling support with `qwen3_coder` parser - **Multi-Token Prediction (MTP)**: Speculative decoding support for lower latency; both MoE and Dense variants ship `mtp.safetensors` - **Multimodal**: Unified vision-language model supporting text, image, and video inputs **Available Models:**

Model	Architecture	Weights
Qwen3.6-35B-A3B (BF16)	MoE 35B / 3B active	[Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B)
Qwen3.6-35B-A3B (FP8)	MoE 35B / 3B active	[Qwen/Qwen3.6-35B-A3B-FP8](https://huggingface.co/Qwen/Qwen3.6-35B-A3B-FP8)
Qwen3.6-35B-A3B (NVFP4)	MoE 35B / 3B active (Blackwell)	[nvidia/Qwen3.6-35B-A3B-NVFP4](https://huggingface.co/nvidia/Qwen3.6-35B-A3B-NVFP4)
Qwen3.6-27B (BF16)	Dense 27B	[Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B)
Qwen3.6-27B (FP8)	Dense 27B	[Qwen/Qwen3.6-27B-FP8](https://huggingface.co/Qwen/Qwen3.6-27B-FP8)
Qwen3.6-27B (NVFP4)	Dense 27B (Blackwell)	[nvidia/Qwen3.6-27B-NVFP4](https://huggingface.co/nvidia/Qwen3.6-27B-NVFP4)

**License:** Apache 2.0 ## 2. SGLang Installation SGLang `>=0.5.10` is required for Qwen3.6. You can install from PyPI, from source, or use a Docker image: ```bash Command # Install from PyPI uv pip install --prerelease=allow sglang # Or install from source uv pip install --prerelease=allow 'git+https://github.com/sgl-project/sglang.git#subdirectory=python' # Or use Docker (NVIDIA GPUs; also serves the NVFP4 variants) docker pull lmsysorg/sglang:latest ``` For the full Docker setup and other installation methods, please refer to the [official SGLang installation guide](../../../docs/get-started/install). For SGLang CPU installation, please refer to the [CPU version installation guide](../../../docs/hardware-platforms/cpu_server#installation). ## 3. Model Deployment This section provides deployment configurations optimized for different hardware platforms and use cases. ### 3.1 Basic Configuration **Interactive Command Generator**: Use the configuration selector below to automatically generate the appropriate deployment command for your hardware platform and capabilities. ### 3.2 Configuration Tips - Speculative decoding (MTP) can significantly reduce latency for interactive use cases. - **Mamba Radix Cache**: Qwen3.6's hybrid Gated Delta Networks architecture supports two mamba scheduling strategies via `--mamba-radix-cache-strategy`: - **V1 (`no_buffer`)**: Default. No overlap scheduler, lower memory usage. - **V2 (`extra_buffer`)**: Enables overlap scheduling and branching point caching with `--mamba-radix-cache-strategy extra_buffer --page-size 64`. Requires FLA kernel backend (NVIDIA GPUs only). Trades higher mamba state memory for better throughput. - The `--mem-fraction-static` flag is recommended for optimal memory utilization, adjust it based on your hardware and workload. - Context length defaults to 262,144 tokens. If you encounter OOM errors, consider reducing it, but maintain at least 128K to preserve thinking capabilities. - **CUDA IPC Transport**: Add `SGLANG_USE_CUDA_IPC_TRANSPORT=1` as an environment variable to use CUDA IPC for transferring multimodal features, significantly improving TTFT (Time To First Token). Note: this consumes additional memory proportional to image size, so you may need to lower `--mem-fraction-static` or `--max-running-requests`. - **Multimodal Attention Backend**: Use `--mm-attention-backend fa3` on H100/H200 for better vision performance, or `--mm-attention-backend fa4` on B200/B300. - For processing large images or videos, you may need to lower `--mem-fraction-static` to leave room for image feature tensors. - Hardware requirements: - **35B-A3B BF16**: ~70GB for weights. TP=1 fits on all supported hardware. - **35B-A3B FP8**: ~35GB for weights. TP=1 fits on all supported hardware. - **35B-A3B NVFP4**: ~23GB for weights. TP=1 fits on B200/B300. - **27B BF16**: ~54GB for weights. TP=1 fits on all supported hardware. - **27B FP8**: ~27GB for weights. TP=1 fits on all supported hardware. - **27B NVFP4**: ~22GB for weights. TP=1 fits on B200/B300. All Qwen3.6 variants (MoE 35B-A3B and Dense 27B) fit on a single supported GPU. NVFP4 is available on B200/B300:

Hardware	Memory	BF16 TP	FP8 TP	NVFP4 TP
H100	80GB	1	1	—
H200	141GB	1	1	—
B200	183GB	1	1	1
B300	275GB	1	1	1

- **Xeon CPU service configuration:** Please refer to the `Notes` part in the serving engine launching section in [the SGLang CPU server document](../../../docs/hardware-platforms/cpu_server#launch-of-the-serving-engine) to better understand how to configure the arguments, especially for TP (tensor parallel) and NUMA binding settings. ## 4. Model Invocation Deploy Qwen3.6 with the following command (H200, all features enabled). Swap `--model-path` to `Qwen/Qwen3.6-27B-FP8` for the dense 27B variant — all other flags carry over: ```shell Command sglang serve \ --model-path Qwen/Qwen3.6-35B-A3B-FP8 \ --reasoning-parser qwen3 \ --tool-call-parser qwen3_coder \ --speculative-algorithm EAGLE \ --speculative-num-steps 3 \ --speculative-eagle-topk 1 \ --speculative-num-draft-tokens 4 \ --mem-fraction-static 0.8 \ --host 0.0.0.0 \ --port 30000 ``` ### 4.1 Basic Usage For basic API usage and request examples, please refer to: - [SGLang Basic Usage Guide](../../../docs/basic_usage/send_request) ### 4.2 Vision Input Qwen3.6 supports image and video inputs as a unified vision-language model. **Image Input Example:** ```python Example from openai import OpenAI client = OpenAI( base_url="http://localhost:30000/v1", api_key="EMPTY" ) response = client.chat.completions.create( model="Qwen/Qwen3.6-35B-A3B-FP8", messages=[ { "role": "user", "content": [ { "type": "image_url", "image_url": { "url": "https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5/demo/CI_Demo/mathv-1327.jpg" } }, { "type": "text", "text": "Describe this image in detail." } ] } ], max_tokens=2048, stream=True ) thinking_started = False has_thinking = False has_answer = False for chunk in response: if chunk.choices and len(chunk.choices) > 0: delta = chunk.choices[0].delta if hasattr(delta, 'reasoning_content') and delta.reasoning_content: if not thinking_started: print("=============== Thinking =================", flush=True) thinking_started = True has_thinking = True print(delta.reasoning_content, end="", flush=True) if delta.content: if has_thinking and not has_answer: print("\n=============== Content =================", flush=True) has_answer = True print(delta.content, end="", flush=True) print() ``` **Video Input Example:** ```python Example from openai import OpenAI client = OpenAI( base_url="http://localhost:30000/v1", api_key="EMPTY" ) response = client.chat.completions.create( model="Qwen/Qwen3.6-35B-A3B-FP8", messages=[ { "role": "user", "content": [ { "type": "video_url", "video_url": { "url": "https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5/demo/video/N1cdUjctpG8.mp4" } }, { "type": "text", "text": "Describe what happens in this video." } ] } ], max_tokens=2048, stream=True ) thinking_started = False has_thinking = False has_answer = False for chunk in response: if chunk.choices and len(chunk.choices) > 0: delta = chunk.choices[0].delta if hasattr(delta, 'reasoning_content') and delta.reasoning_content: if not thinking_started: print("=============== Thinking =================", flush=True) thinking_started = True has_thinking = True print(delta.reasoning_content, end="", flush=True) if delta.content: if has_thinking and not has_answer: print("\n=============== Content =================", flush=True) has_answer = True print(delta.content, end="", flush=True) print() ``` ### 4.3 Advanced Usage #### 4.3.1 Reasoning Parser Qwen3.6 supports Thinking mode **by default**. Enable the reasoning parser during deployment to separate the thinking and content sections. The thinking process is returned via `reasoning_content` in the streaming response. To disable thinking and use Instruct mode, pass `chat_template_kwargs` at request time: - **Thinking mode** (default): The model performs step-by-step reasoning before answering. No extra parameters needed. - **Instruct mode** (`{"enable_thinking": false}`): The model responds directly without a thinking process. **Example 1: Thinking Mode (Default)** ```python Example from openai import OpenAI client = OpenAI( base_url="http://localhost:30000/v1", api_key="EMPTY" ) response = client.chat.completions.create( model="Qwen/Qwen3.6-35B-A3B-FP8", messages=[ {"role": "user", "content": "Solve this problem step by step: What is 15% of 240?"} ], max_tokens=2048, stream=True ) has_thinking = False has_answer = False thinking_started = False for chunk in response: if chunk.choices and len(chunk.choices) > 0: delta = chunk.choices[0].delta if hasattr(delta, 'reasoning_content') and delta.reasoning_content: if not thinking_started: print("=============== Thinking =================", flush=True) thinking_started = True has_thinking = True print(delta.reasoning_content, end="", flush=True) if delta.content: if has_thinking and not has_answer: print("\n=============== Content =================", flush=True) has_answer = True print(delta.content, end="", flush=True) print() ``` **Example 2: Instruct Mode (Thinking Off)** To disable thinking and get a direct response, pass `{"enable_thinking": false}` via `chat_template_kwargs`: ```python Example from openai import OpenAI client = OpenAI( base_url="http://localhost:30000/v1", api_key="EMPTY" ) response = client.chat.completions.create( model="Qwen/Qwen3.6-35B-A3B-FP8", messages=[ {"role": "user", "content": "What is 15% of 240?"} ], extra_body={"chat_template_kwargs": {"enable_thinking": False}}, max_tokens=2048, stream=True ) for chunk in response: if chunk.choices and len(chunk.choices) > 0: delta = chunk.choices[0].delta if delta.content: print(delta.content, end="", flush=True) print() ``` #### 4.3.2 Thinking Preservation Qwen3.6 has been trained to preserve and leverage thinking traces from historical messages. Enable this for agent scenarios where maintaining full reasoning context improves decision consistency: ```python Example from openai import OpenAI client = OpenAI( base_url="http://localhost:30000/v1", api_key="EMPTY" ) response = client.chat.completions.create( model="Qwen/Qwen3.6-35B-A3B-FP8", messages=[ {"role": "user", "content": "Help me plan a web app architecture."} ], extra_body={"chat_template_kwargs": {"preserve_thinking": True}}, max_tokens=2048, stream=True ) thinking_started = False has_thinking = False has_answer = False for chunk in response: if chunk.choices and len(chunk.choices) > 0: delta = chunk.choices[0].delta if hasattr(delta, 'reasoning_content') and delta.reasoning_content: if not thinking_started: print("=============== Thinking =================", flush=True) thinking_started = True has_thinking = True print(delta.reasoning_content, end="", flush=True) if delta.content: if has_thinking and not has_answer: print("\n=============== Content =================", flush=True) has_answer = True print(delta.content, end="", flush=True) print() ``` #### 4.3.3 Tool Calling Qwen3.6 supports tool calling capabilities. Enable the tool call parser during deployment. ```python Example from openai import OpenAI client = OpenAI( base_url="http://localhost:30000/v1", api_key="EMPTY" ) tools = [ { "type": "function", "function": { "name": "get_weather", "description": "Get the current weather for a location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city name" }, "unit": { "type": "string", "enum": ["celsius", "fahrenheit"], "description": "Temperature unit" } }, "required": ["location"] } } } ] response = client.chat.completions.create( model="Qwen/Qwen3.6-35B-A3B-FP8", messages=[ {"role": "user", "content": "What's the weather in Beijing?"} ], tools=tools, stream=True ) thinking_started = False has_thinking = False for chunk in response: if chunk.choices and len(chunk.choices) > 0: delta = chunk.choices[0].delta if hasattr(delta, 'reasoning_content') and delta.reasoning_content: if not thinking_started: print("=============== Thinking =================", flush=True) thinking_started = True has_thinking = True print(delta.reasoning_content, end="", flush=True) if hasattr(delta, 'tool_calls') and delta.tool_calls: if has_thinking and thinking_started: print("\n=============== Content =================", flush=True) thinking_started = False for tool_call in delta.tool_calls: if tool_call.function: print(f"Tool Call: {tool_call.function.name}") print(f" Arguments: {tool_call.function.arguments}") if delta.content: print(delta.content, end="", flush=True) print() ```