[Docs] Rename docs_new/ to docs/ (#32123)

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

docs/cookbook/autoregressive/InclusionAI/LLaDA-2.1.mdx

@@ -0,0 +1,700 @@
+---
+title: LLaDA 2.1
+metatags:
+    description: "Deploy LLaDA 2.1 with SGLang - large-scale discrete diffusion language model with parallel token generation, iterative denoising, MoE architecture, and reinforcement learning for reasoning."
+---
+
+import { LLaDA21Deployment } from '/src/snippets/autoregressive/llada-21-deployment.jsx';
+
+## 1. Model Introduction
+
+[LLaDA 2.1](https://github.com/inclusionAI/LLaDA2.X) is a series of large-scale discrete diffusion language models (dLLMs) developed by the InclusionAI team at Ant Group. Unlike traditional autoregressive models that generate text left-to-right one token at a time, LLaDA 2.1 uses a diffusion-based approach — drafting tokens in parallel and refining them through iterative denoising, enabling self-correction during generation.
+
+**Key Features:**
+
+- **Token Editing (T2T + M2T)**: Combines Mask-to-Token (M2T) and Token-to-Token (T2T) editing, allowing the model to not only unmask tokens but also revise already-generated tokens mid-flight
+- **Dual Decoding Modes**: Speed Mode (S) for maximum throughput with T2T refinement, and Quality Mode (Q) for conservative thresholds and higher benchmark scores
+- **MoE Architecture**: Both variants use Mixture-of-Experts architecture for efficient scaling
+- **First Large-Scale RL for dLLMs**: Implements the first reinforcement learning framework specifically designed for diffusion language models, improving reasoning and instruction-following
+- **Lightning-Fast Decoding**: Up to 892 tokens/s on HumanEval+ for the 100B model
+
+**Available Models:**
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <colgroup>
+    <col style={{width: "20.0%"}} />
+    <col style={{width: "20.0%"}} />
+    <col style={{width: "20.0%"}} />
+    <col style={{width: "20.0%"}} />
+    <col style={{width: "20.0%"}} />
+  </colgroup>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Model</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Parameters</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Architecture</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Context Length</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>HuggingFace</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>**LLaDA2.1-mini**</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>16B</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>MoE (20 layers, 16 attention heads)</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>32,768 tokens</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>[inclusionAI/LLaDA2.1-mini](https://huggingface.co/inclusionAI/LLaDA2.1-mini)</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>**LLaDA2.1-flash**</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>100B</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>MoE</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>32,768 tokens</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>[inclusionAI/LLaDA2.1-flash](https://huggingface.co/inclusionAI/LLaDA2.1-flash)</td>
+    </tr>
+  </tbody>
+</table>
+
+**License:**
+
+Apache 2.0. Please refer to the [official LLaDA2.X repository](https://github.com/inclusionAI/LLaDA2.X) for details.
+
+## 2. SGLang Installation
+
+SGLang offers multiple installation methods. You can choose the most suitable installation method based on your hardware platform and requirements.
+
+Please refer to the [official SGLang installation guide](../../../docs/get-started/install) for installation instructions.
+
+## 3. Model Deployment
+
+This section provides deployment configurations optimized for different hardware platforms and use cases.
+
+### 3.1 Basic Configuration
+
+**Interactive Command Generator**: Use the configuration selector below to automatically generate the appropriate deployment command for your hardware platform, model size, and decoding mode. SGLang supports serving LLaDA-2.1 on NVIDIA H100, H200, B200, and AMD MI300X, MI325X, MI355X GPUs.
+
+<LLaDA21Deployment />
+
+### 3.2 Configuration Tips
+
+**dLLM-Specific Parameters:**
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Parameter</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Description</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Recommended Value</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`--dllm-algorithm`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Diffusion decoding algorithm</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>`JointThreshold`</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`--trust-remote-code`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Required for LLaDA model loading</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Always enabled</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`--mem-fraction-static`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Static memory fraction for KV cache</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>`0.8`</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`--max-running-requests`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Maximum concurrent requests</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>`1` (for best quality)</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`--attention-backend`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Attention computation backend</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>`flashinfer`</td>
+    </tr>
+  </tbody>
+</table>
+
+**Decoding Mode Comparison:**
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Mode</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Threshold</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Speed</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Quality</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Best For</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>**Quality Mode (Q)**</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Conservative</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Moderate</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Higher benchmark scores</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Accuracy-critical tasks</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>**Speed Mode (S)**</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Aggressive</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Very fast, relies on T2T editing</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Slightly lower</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Throughput-critical tasks</td>
+    </tr>
+  </tbody>
+</table>
+
+**Hardware Requirements:**
+
+- **LLaDA2.1-mini (16B)**: ~47 GB VRAM, runs on a single GPU (TP=1)
+- **LLaDA2.1-flash (100B)**: Requires multi-GPU setup (TP=4 on H100/H200, TP=2 on B200)
+
+## 4. Model Invocation
+
+### 4.1 Deployment
+
+Start the server using the command generated above, for example:
+
+```shell Command
+python -m sglang.launch_server \
+  --model-path inclusionAI/LLaDA2.1-mini \
+  --dllm-algorithm JointThreshold \
+  --tp 1 \
+  --trust-remote-code \
+  --mem-fraction-static 0.8 \
+  --max-running-requests 1 \
+  --attention-backend flashinfer \
+  --host 0.0.0.0 \
+  --port 8000
+```
+
+### 4.2 Basic Usage
+
+For basic API usage and request examples, please refer to:
+
+- [SGLang Basic Usage Guide](../../../docs/basic_usage/send_request)
+
+**Simple Completion Example:**
+
+```python Example
+from openai import OpenAI
+
+client = OpenAI(
+    base_url="http://localhost:8000/v1",
+    api_key="EMPTY"
+)
+
+response = client.chat.completions.create(
+    model="inclusionAI/LLaDA2.1-mini",
+    messages=[
+        {"role": "user", "content": "Explain what a diffusion language model is in simple terms."}
+    ],
+    max_tokens=1024
+)
+
+print(response.choices[0].message.content)
+```
+
+**Output Example:**
+
+```text Output
+Sure! Let's break it down in simple terms.
+
+A **diffusion language model** is a type of artificial intelligence that learns to generate text—like sentences, stories, or emails—by studying a lot of written text.
+
+Here’s how it works, using a simple real-life analogy:
+
+Imagine you have a big book full of stories. A diffusion language model is trying to learn how to write a new story. Instead of being told the rules, it starts by looking at all the words in the book and trying to understand how words usually go together.
+
+Now, think of the process like this:
+
+1. **Start with random noise**: The model begins with a completely random set of words (like a scribble on paper).
+2. ** ** "clean up" the noise**: It gradually "denoises" the noise by turning it into meaningful text, word by word, based on what it learned learned from the book.
+3. **Learn from patterns**: As it does this, it learns patterns—like how words often follow each other, or how sentences start.
+4. **Generate new text**: Once it’s learned the patterns, it can create new, coherent sentences or stories by starting from a and and building it up word by word.
+
+So, the "diffusion" part comes from the idea of going from random noise to clear, meaningful text—like turning a scribble into a full story.
+
+In short:
+A diffusion language model is an AI that learns to write text by reading lots of books and gradually turning random noise into coherent, meaningful sentences based on what it learned.
+```
+
+### 4.3 Advanced Usage
+
+#### 4.3.1 Streaming
+
+```python Example
+from openai import OpenAI
+
+client = OpenAI(
+    base_url="http://localhost:8000/v1",
+    api_key="EMPTY"
+)
+
+response = client.chat.completions.create(
+    model="inclusionAI/LLaDA2.1-mini",
+    messages=[
+        {"role": "user", "content": "Write a Python function to compute the Fibonacci sequence."}
+    ],
+    max_tokens=2048,
+    stream=True
+)
+
+for chunk in response:
+    if chunk.choices and len(chunk.choices) > 0:
+        delta = chunk.choices[0].delta
+        if delta.content:
+            print(delta.content, end="", flush=True)
+
+print()
+```
+
+**Output Example:**
+
+````text Output
+Here are several ways to implement the Fibonacci sequence in Python:
+
+## 1. Recursive Approach (Simple but Inefficient)
+
+```python
+def fibonacci_recursive(n):
+    """
+    Compute the nth Fibonacci number using recursion.
+
+    Args:
+        n (int): The position in the Fibonacci sequence (0-indexed)
+
+    Returns:
+        int: The nth Fibonacci number
+
+    Raises:
+        ValueError: If n is negative
+    """
+    if n < 0:
+        raise ValueError("n must be non-negative")
+
+    if n <= 1:
+        return n
+
+    return fibonacci_recursive(n - 1) + fibonacci_recursive(n - 2)
+
+# Example usage
+print(fibonacci_recursive(10))  # Output: 55
+```
+
+## 2. Iterative Approach (Efficient)
+...
+````
+
+#### 4.3.2 Code Generation
+
+```python Example
+from openai import OpenAI
+
+client = OpenAI(
+    base_url="http://localhost:8000/v1",
+    api_key="EMPTY"
+)
+
+response = client.chat.completions.create(
+    model="inclusionAI/LLaDA2.1-mini",
+    messages=[
+        {"role": "user", "content": "Write a Python function that checks if a string is a palindrome. Include docstring and test cases."}
+    ],
+    max_tokens=2048
+)
+
+print(response.choices[0].message.content)
+```
+
+**Output Example:**
+
+````text Output
+```python
+def is_palindrome(s):
+    """
+    Check if a string is a palindrome.
+
+    A palindrome is a word, phrase, or sequence that reads the same backward as forward.
+    This function ignores case, spaces, punctuation, and non characters characters.
+
+    Args:
+        s (str): The string to check
+
+    Returns:
+        bool: True if the string is a palindrome, False otherwise
+
+    Examples:
+        >>> is_palindrome("racecar")
+        True
+        >>> is_palindrome("A man a plan a canal Panama")
+        True
+        >>> is_palindrome("race a car")
+        False
+        >>> is_palindrome("")
+        True
+        >>> is_palindrome("a")
+        True
+    """
+    # Remove non-alphanumeric characters and convert to lowercase
+    cleaned = ''.join(char.lower() for char in s if char.isalnum())
+
+    # Check if the cleaned string reads the same forwards and backwards
+    return cleaned == cleaned[::-1]
+
+# Test cases
+def test_is_palindrome():
+    """Test the is_palindrome function with various inputs."""
+
+    # Test basic palindromes
+    assert is_palindrome("racecar") == True
+    assert is_palindrome("level") == True
+    assert is_palindrome("madam") == True
+    assert is_palindrome("radar") == True
+
+    # Test palindromes with spaces and punctuation
+    assert is_palindrome("A man a plan a canal Panama") == True
+    assert is_palindrome("race a car") == False
+    assert is_palindrome("Was it a car or a cat I saw?") == True
+    assert is_palindrome("Madam, I'm Adam") == True
+
+    # Test edge cases
+    assert is_palindrome("") == True
+    assert is_palindrome("a") == True
+    assert is_palindrome("A") == True
+    assert is_palindrome("Aa") == True
+
+    # Test non-palindromes
+    assert is_palindrome("hello") == False
+    assert is_palindrome("world") == False
+    assert is_palindrome("python") == False
+
+    # Test single characters
+    assert is_palindrome("1") == True
+    assert is_palindrome("1") == True
+
+    print("All tests passed!")
+
+# Run the tests
+if __name__ == "__main__":
+    # Example usage
+    print("Testing isalindrome function:")
+    print(f"'racecar' {is_palindrome('racecar')}")
+    print(f"'A man a plan a canal Panama': {is_palindrome('A man a plan a canal Panama')}")
+    print(f"'race a car': {is_palindrome('race a car')}")
+    print(f"'hello': {is_palindrome('hello')}")
+
+    # Run tests
+    test_is_palindrome()
+```
+
+This implementation includes:
+
+1. **Comprehensive function** `is_palindrome()` that:
+   - Ignores case by converting to lowercase
+   - Removes all non-alphanumeric characters (spaces, punctuation, etc.)
+   - Uses string slicing (`[::-1]`) to reverse the string
+
+2. **Detailed docstring** explaining:
+   - What the function does
+   - How it works
+   - Return value
+   - Examples of usage
+
+3. **Extensive test cases** covering:
+   - Basic palindromes
+   - Palindromes with spaces and punctuation
+   - Edge cases (empty string, single character)
+   - Non-palindromes
+   - Mixed case scenarios
+
+4. **Test function** that uses assertions to verify the function works correctly
+
+The function efficiently handles real-world palindrome checking by ignoring case, spaces, and punctuation, making it suitable for phrases like "A man a plan a canal Panama".
+````
+
+## 5. Benchmark
+
+This section uses **industry-standard configurations** for comparable benchmark results.
+
+### 5.1 Speed Benchmark
+
+**Test Environment:**
+
+- Hardware: NVIDIA B200 (4x)
+- SGLang Version: 0.5.8+
+
+#### 5.1.1 LLaDA2.1-mini
+
+**Model Deployment:**
+
+```bash Command
+python -m sglang.launch_server \
+  --model-path inclusionAI/LLaDA2.1-mini \
+  --dllm-algorithm JointThreshold \
+  --tp 1 \
+  --trust-remote-code \
+  --mem-fraction-static 0.8 \
+  --max-running-requests 1 \
+  --attention-backend flashinfer
+```
+
+- Latency Benchmark
+
+```bash Command
+python -m sglang.bench_serving \
+  --backend sglang \
+  --model inclusionAI/LLaDA2.1-mini \
+  --dataset-name random \
+  --random-input-len 1000 \
+  --random-output-len 1000 \
+  --num-prompts 10 \
+  --max-concurrency 1 \
+  --request-rate inf
+```
+
+- **Latency Result**:
+
+```text Output
+============ Serving Benchmark Result ============
+Backend:                                 sglang
+Traffic request rate:                    inf
+Max request concurrency:                 1
+Successful requests:                     10
+Benchmark duration (s):                  9.90
+Total input tokens:                      6101
+Total input text tokens:                 6101
+Total generated tokens:                  4220
+Total generated tokens (retokenized):    3433
+Request throughput (req/s):              1.01
+Input token throughput (tok/s):          616.26
+Output token throughput (tok/s):         426.26
+Peak output token throughput (tok/s):    1010.00
+Peak concurrent requests:                3
+Total token throughput (tok/s):          1042.53
+Concurrency:                             1.00
+----------------End-to-End Latency----------------
+Mean E2E Latency (ms):                   988.87
+Median E2E Latency (ms):                 655.27
+P90 E2E Latency (ms):                    1952.50
+P99 E2E Latency (ms):                    2932.19
+---------------Time to First Token----------------
+Mean TTFT (ms):                          152.74
+Median TTFT (ms):                        150.37
+P99 TTFT (ms):                           229.78
+-----Time per Output Token (excl. 1st token)------
+Mean TPOT (ms):                          2.16
+Median TPOT (ms):                        2.08
+P99 TPOT (ms):                           3.72
+---------------Inter-Token Latency----------------
+Mean ITL (ms):                           2.10
+Median ITL (ms):                         1.99
+P95 ITL (ms):                            4.03
+P99 ITL (ms):                            6.34
+Max ITL (ms):                            26.59
+==================================================
+```
+
+- Throughput Benchmark
+
+```bash Command
+python -m sglang.bench_serving \
+  --backend sglang \
+  --model inclusionAI/LLaDA2.1-mini \
+  --dataset-name random \
+  --random-input-len 1000 \
+  --random-output-len 1000 \
+  --num-prompts 500 \
+  --max-concurrency 100 \
+  --request-rate inf
+```
+
+- **Throughput Result**:
+
+```text Output
+============ Serving Benchmark Result ============
+Backend:                                 sglang
+Traffic request rate:                    inf
+Max request concurrency:                 100
+Successful requests:                     500
+Benchmark duration (s):                  467.74
+Total input tokens:                      249831
+Total input text tokens:                 249831
+Total generated tokens:                  252662
+Total generated tokens (retokenized):    189717
+Request throughput (req/s):              1.07
+Input token throughput (tok/s):          534.12
+Output token throughput (tok/s):         540.17
+Peak output token throughput (tok/s):    1753.00
+Peak concurrent requests:                105
+Total token throughput (tok/s):          1074.30
+Concurrency:                             90.77
+----------------End-to-End Latency----------------
+Mean E2E Latency (ms):                   84912.27
+Median E2E Latency (ms):                 86564.26
+P90 E2E Latency (ms):                    110567.26
+P99 E2E Latency (ms):                    114303.38
+---------------Time to First Token----------------
+Mean TTFT (ms):                          83920.39
+Median TTFT (ms):                        85669.54
+P99 TTFT (ms):                           112969.91
+-----Time per Output Token (excl. 1st token)------
+Mean TPOT (ms):                          2.67
+Median TPOT (ms):                        1.65
+P99 TPOT (ms):                           4.43
+---------------Inter-Token Latency----------------
+Mean ITL (ms):                           1.69
+Median ITL (ms):                         1.46
+P95 ITL (ms):                            3.96
+P99 ITL (ms):                            4.84
+Max ITL (ms):                            92.08
+==================================================
+```
+
+#### 5.1.2 LLaDA2.1-flash
+
+**Model Deployment:**
+
+```bash Command
+python -m sglang.launch_server \
+  --model-path inclusionAI/LLaDA2.1-flash \
+  --dllm-algorithm JointThreshold \
+  --tp 4 \
+  --trust-remote-code \
+  --mem-fraction-static 0.8 \
+  --max-running-requests 1 \
+  --attention-backend flashinfer
+```
+
+- Latency Benchmark
+
+```bash Command
+python -m sglang.bench_serving \
+  --backend sglang \
+  --model inclusionAI/LLaDA2.1-flash \
+  --dataset-name random \
+  --random-input-len 1000 \
+  --random-output-len 1000 \
+  --num-prompts 10 \
+  --max-concurrency 1 \
+  --request-rate inf
+```
+
+- **Latency Result**:
+
+```text Output
+============ Serving Benchmark Result ============
+Backend:                                 sglang
+Traffic request rate:                    inf
+Max request concurrency:                 1
+Successful requests:                     10
+Benchmark duration (s):                  14.46
+Total input tokens:                      6101
+Total input text tokens:                 6101
+Total generated tokens:                  4220
+Total generated tokens (retokenized):    3276
+Request throughput (req/s):              0.69
+Input token throughput (tok/s):          421.79
+Output token throughput (tok/s):         291.75
+Peak output token throughput (tok/s):    676.00
+Peak concurrent requests:                3
+Total token throughput (tok/s):          713.53
+Concurrency:                             1.00
+----------------End-to-End Latency----------------
+Mean E2E Latency (ms):                   1445.16
+Median E2E Latency (ms):                 968.06
+P90 E2E Latency (ms):                    3101.86
+P99 E2E Latency (ms):                    4208.49
+---------------Time to First Token----------------
+Mean TTFT (ms):                          231.63
+Median TTFT (ms):                        242.67
+P99 TTFT (ms):                           341.33
+-----Time per Output Token (excl. 1st token)------
+Mean TPOT (ms):                          3.04
+Median TPOT (ms):                        2.79
+P99 TPOT (ms):                           5.33
+---------------Inter-Token Latency----------------
+Mean ITL (ms):                           3.05
+Median ITL (ms):                         2.41
+P95 ITL (ms):                            7.25
+P99 ITL (ms):                            8.27
+Max ITL (ms):                            29.27
+==================================================
+```
+
+- Throughput Benchmark
+
+```bash Command
+python -m sglang.bench_serving \
+  --backend sglang \
+  --model inclusionAI/LLaDA2.1-flash \
+  --dataset-name random \
+  --random-input-len 1000 \
+  --random-output-len 1000 \
+  --num-prompts 500 \
+  --max-concurrency 100 \
+  --request-rate inf
+```
+
+- **Throughput Result**:
+
+```text Output
+============ Serving Benchmark Result ============
+Backend:                                 sglang
+Traffic request rate:                    inf
+Max request concurrency:                 100
+Successful requests:                     500
+Benchmark duration (s):                  671.85
+Total input tokens:                      249831
+Total input text tokens:                 249831
+Total generated tokens:                  252662
+Total generated tokens (retokenized):    177961
+Request throughput (req/s):              0.74
+Input token throughput (tok/s):          371.85
+Output token throughput (tok/s):         376.07
+Peak output token throughput (tok/s):    1521.00
+Peak concurrent requests:                103
+Total token throughput (tok/s):          747.92
+Concurrency:                             91.28
+----------------End-to-End Latency----------------
+Mean E2E Latency (ms):                   122658.36
+Median E2E Latency (ms):                 125265.55
+P90 E2E Latency (ms):                    159554.07
+P99 E2E Latency (ms):                    165174.88
+---------------Time to First Token----------------
+Mean TTFT (ms):                          121009.17
+Median TTFT (ms):                        124437.80
+P99 TTFT (ms):                           163579.29
+-----Time per Output Token (excl. 1st token)------
+Mean TPOT (ms):                          4.73
+Median TPOT (ms):                        2.16
+P99 TPOT (ms):                           7.13
+---------------Inter-Token Latency----------------
+Mean ITL (ms):                           2.38
+Median ITL (ms):                         1.40
+P95 ITL (ms):                            6.89
+P99 ITL (ms):                            8.60
+Max ITL (ms):                            176.78
+==================================================
+```
+
+### 5.2 Accuracy Benchmark
+
+#### 5.2.1 GSM8K Benchmark
+
+```bash Command
+python -m sglang.test.few_shot_gsm8k \
+  --num-questions 200 \
+  --port 8000
+```
+
+**Results:**
+
+```text Output
+Accuracy: 0.895
+Invalid: 0.000
+Latency: 100.552 s
+Output throughput: 262.094 token/s
+```

docs/cookbook/autoregressive/InclusionAI/Ling-2.5-1T.mdx

@@ -0,0 +1,217 @@
+---
+title: Ling-2.5-1T
+metatags:
+    description: "Deploy Ling-2.5-1T with SGLang - 1T parameter MoE model with 63B active parameters, trillion-scale context length up to 1M tokens, and agentic tool calling capabilities."
+---
+
+## 1. Model Introduction
+
+[Ling-2.5-1T](https://huggingface.co/inclusionAI/Ling-2.5-1T) is the latest flagship instant model in the Ling family. Thinking models raise the ceiling of intelligence, while instant models expand its reach by balancing efficiency and performance—making AGI not only more powerful, but also more accessible. Ling-2.5-1T delivers comprehensive upgrades across model architecture, token efficiency, and preference alignment, designed to bring universally accessible AI to a new level of quality.
+
+**Key Features:**
+
+- **Trillion-Scale Model**: 1T total parameters with 63B active parameters (up from 51B in the previous generation). Pre-training corpus expanded from 20T to 29T tokens. Leveraging an efficient hybrid linear attention architecture (1:7 MLA + Lightning Linear Attention), the model delivers exceptionally high throughput while processing context lengths of up to 1M tokens.
+- **Token Efficiency**: By introducing a composite reward mechanism combining "Correctness" and "Process Redundancy", Ling-2.5-1T further pushes the frontier of efficiency-performance balance in instant models. At comparable token efficiency levels, Ling-2.5-1T's reasoning capabilities significantly outperform its predecessor, approaching the level of frontier "thinking models" that typically consume ~4x the output tokens.
+- **Preference Alignment**: Through refined alignment strategies—such as bidirectional RL feedback and Agent-based instruction constraint verification—Ling-2.5-1T achieves substantial improvements over the previous generation in preference alignment tasks, including creative writing and instruction following.
+- **Agentic Capabilities**: Trained with Agentic RL in large-scale high-fidelity interactive environments, Ling-2.5-1T is compatible with mainstream agent platforms such as Claude Code, OpenCode, and OpenClaw. It achieves leading open-source performance on the general tool-calling benchmark, BFCL-V4.
+- **Context Length**: 256K -> 1M (YaRN)
+
+**Available Models:**
+
+- **BF16**: [inclusionAI/Ling-2.5-1T](https://huggingface.co/inclusionAI/Ling-2.5-1T)
+
+**License:** MIT
+
+## 2. SGLang Installation
+
+Ling-2.5-1T runs on the standard SGLang Docker image:
+
+```bash Command
+# NVIDIA (H200 / B200 / GB200 / GB300)
+docker pull lmsysorg/sglang:latest
+```
+
+For other installation methods, please refer to the [official SGLang installation guide](../../../docs/get-started/install).
+
+Ling-2.5-1T is also supported via the **nightly PyPI builds**. See the [SGLang Installation (PyPI)](../../../docs/get-started/install) guide for setup instructions.
+
+## 3. Model Deployment
+
+Ling-2.5-1T is a trillion-parameter BF16 model that requires multi-node deployment (at least 2 nodes). Use the configuration selector below to generate the deployment command for your hardware platform.
+
+import { Ling251TDeployment } from '/src/snippets/autoregressive/ling-25-1t-deployment.jsx'
+
+<Ling251TDeployment />
+
+### Configuration Tips
+
+- The `--trust-remote-code` flag is required for this model due to custom modeling code.
+- `--tp-size` can be set to a maximum of 8 for this model. If you have more GPUs available, increase `--pp-size` to scale across additional nodes.
+- Adding `--model-loader-extra-config '{"enable_multithread_load": "true","num_threads": 64}'` enables faster model loading.
+- On H200/GB200/GB300 with 2-node deployment, `--mem-frac 0.95` is required to avoid OOM since the model occupies most of the GPU memory. For better throughput, consider 4-node deployment (ref [model card](https://huggingface.co/inclusionAI/Ling-2.5-1T#run-inference) for more details).
+
+## 4. Model Invocation
+
+### 4.1 Basic Usage
+
+For example, launch the server on 2 H200 nodes:
+
+```bash Command
+export MASTER_IP=10.10.0.1 # The IP of Node 0
+export PORT=30000
+export DIST_PORT=50000
+
+# Node 0:
+python3 -m sglang.launch_server \
+--model-path inclusionAI/Ling-2.5-1T \
+--trust-remote-code \
+--tp-size 8 \
+--pp-size 2 \
+--nnodes 2 \
+--node-rank 0 \
+--host 0.0.0.0 \
+--port ${PORT} \
+--dist-init-addr ${MASTER_IP}:${DIST_PORT} \
+--tool-call-parser qwen \
+--model-loader-extra-config '{"enable_multithread_load": "true","num_threads": 64}' \
+--mem-frac 0.95
+
+
+# Node 1:
+python3 -m sglang.launch_server \
+--model-path inclusionAI/Ling-2.5-1T \
+--trust-remote-code \
+--tp-size 8 \
+--pp-size 2 \
+--nnodes 2 \
+--node-rank 1 \
+--dist-init-addr ${MASTER_IP}:${DIST_PORT} \
+--tool-call-parser qwen \
+--model-loader-extra-config '{"enable_multithread_load": "true","num_threads": 64}' \
+--mem-frac 0.95
+```
+
+Once the server is running, send requests to the master node:
+
+```bash Command
+curl -s http://${MASTER_IP}:${PORT}/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{"model": "auto", "messages": [{"role": "user", "content": "What is the capital of France?"}]}'
+```
+Output:
+```json Config
+{
+  "id": "e82af153da844ee6aed7a27a3187f2f4",
+  "object": "chat.completion",
+  "created": 1771216764,
+  "model": "auto",
+  "choices": [
+    {
+      "index": 0,
+      "message": {
+        "role": "assistant",
+        "content": "The capital of France is **Paris**.\n\n**Additional details:**\n*   It is the largest city in France.\n*   It is located in the north-central part of the country along the Seine River.\n*   Paris is often referred to as \"The City of Light\" (*La Ville Lumière*).",
+        "reasoning_content": null,
+        "tool_calls": null
+      },
+      "logprobs": null,
+      "finish_reason": "stop",
+      "matched_stop": 156895
+    }
+  ],
+  "usage": {
+    "prompt_tokens": 25,
+    "total_tokens": 93,
+    "completion_tokens": 68,
+    "prompt_tokens_details": null,
+    "reasoning_tokens": 0
+  }
+}
+```
+
+For more API usage examples, please refer to:
+
+- [SGLang Basic Usage Guide](../../../docs/basic_usage/send_request)
+
+### 4.2 Tool Calling Example
+
+```bash Command
+curl -s http://${MASTER_IP}:${PORT}/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "inclusionAI/Ling-2.5-1T",
+    "messages": [{"role": "user", "content": "Search for the latest news about AI"}],
+    "tools": [{
+      "type": "function",
+      "function": {
+        "name": "search",
+        "description": "Search for information on the internet",
+        "parameters": {
+          "type": "object",
+          "properties": {
+            "query": {"type": "string", "description": "The search query"}
+          },
+          "required": ["query"]
+        }
+      }
+    }],
+    "tool_choice": "auto"
+  }'
+```
+Output:
+```json Config
+{
+  "id": "b968e45c7d414f7482c8ffc0f9c6b688",
+  "object": "chat.completion",
+  "created": 1771216520,
+  "model": "inclusionAI/Ling-2.5-1T",
+  "choices": [
+    {
+      "index": 0,
+      "message": {
+        "role": "assistant",
+        "content": null,
+        "reasoning_content": null,
+        "tool_calls": [
+          {
+            "id": "call_e75f711d8ad840ed9d382c9e",
+            "index": 0,
+            "type": "function",
+            "function": {
+              "name": "search",
+              "arguments": "{\"query\": \"latest news about AI\"}"
+            }
+          }
+        ]
+      },
+      "logprobs": null,
+      "finish_reason": "tool_calls",
+      "matched_stop": null
+    }
+  ],
+  "usage": {
+    "prompt_tokens": 173,
+    "total_tokens": 196,
+    "completion_tokens": 23,
+    "prompt_tokens_details": null,
+    "reasoning_tokens": 0
+  }
+}
+```
+
+## 5. Benchmark
+
+### GSM8K
+
+- Benchmark Command
+```bash Command
+python3 benchmark/gsm8k/bench_sglang.py
+```
+
+- Test Result
+```text Output
+Accuracy: 0.960
+Invalid: 0.000
+Latency: 45.410 s
+Output throughput: 560.642 token/s
+```

docs/cookbook/autoregressive/InclusionAI/Ling-2.6.mdx

@@ -0,0 +1,223 @@
+---
+title: Ling-2.6
+metatags:
+    description: "Deploy the Ling-2.6 family with SGLang - Ling-2.6-flash (104B total / 7.4B active BF16 MoE) and Ling-2.6-1T (~1T FP8 MoE) with hybrid linear attention and agentic tool calling."
+---
+
+## 1. Model Introduction
+
+The **Ling-2.6** family from inclusionAI is the next iteration of the Ling instant-model series. Continuing the architectural direction set by Ling-2.5, Ling-2.6 doubles down on **inference efficiency**, **token efficiency**, and **agent performance** — staying competitive with frontier instant models while being faster, leaner, and better suited for production agent workloads.
+
+**Key Features:**
+
+- **Hybrid Linear Attention**: A `1:7 MLA + Lightning Linear` hybrid built on top of a highly sparse MoE backbone. Compared with same-class SOTA models, Ling-2.6-flash shows up to ~4× higher prefill and decode throughput in long-context scenarios; Ling-2.6-1T is shipped in FP8 so it fits a single GB300 node with `--tp 4`.
+- **Token Efficiency**: Trained with explicit token-efficiency objectives. On the full Artificial Analysis suite, Ling-2.6-flash uses only ~15M output tokens while remaining competitive — a meaningfully stronger intelligence-per-token profile than long-reasoning peers.
+- **Agentic Capabilities**: Refined for tool use, multi-step planning, and long-horizon execution. Reaches SOTA-class results on **BFCL-V4**, **TAU2-bench**, **SWE-bench Verified**, **Claw-Eval**, and **PinchBench**, and is validated against Claude Code, Kilo Code, Qwen Code, Hermes Agent, and OpenClaw.
+- **Long Context**: Native 128K, extendable to **256K (Ling-2.6-flash)** and **256K → 1M (Ling-2.6-1T via YaRN)**.
+
+**Available Models:**
+
+- **BF16**: [inclusionAI/Ling-2.6-flash](https://huggingface.co/inclusionAI/Ling-2.6-flash) — 104B total / 7.4B active
+- **FP8 (E4M3)**: [inclusionAI/Ling-2.6-1T](https://huggingface.co/inclusionAI/Ling-2.6-1T) — ~1T total
+
+**License:** MIT
+
+## 2. SGLang Installation
+
+SGLang offers multiple installation methods. You can choose the most suitable installation method based on your hardware platform and requirements.
+
+Please refer to the [official SGLang installation guide](../../../docs/get-started/install) for installation instructions.
+
+## 3. Model Deployment
+
+### 3.1 Ling-2.6-flash
+
+Ling-2.6-flash is a 104B/7.4B-active MoE that runs comfortably on a single 4-GPU node. Use the selector below to generate the launch command for your hardware.
+
+import { Ling26FlashDeployment } from '/src/snippets/autoregressive/ling-26-flash-deployment.jsx'
+
+<Ling26FlashDeployment />
+
+#### Configuration Tips
+
+- `--trust-remote-code` is required (custom `BailingMoeV2_5ForCausalLM` modeling code).
+- `--tp-size 4` is the reference layout. On 4× H20-3e the model reaches ~340 tokens/s decode at TP=4, batch 32.
+- Native context is 128K. Enable YaRN (`--json-model-override-args '{"rope_scaling": {"rope_type": "yarn", "factor": 2.0, ...}}'`) to extend to 256K — the snippet does this for you.
+- `--tool-call-parser qwen25` matches the model's `<tool_call>...</tool_call>` schema.
+- The recommended baseline does **not** include `--reasoning-parser qwen3`. Ling-2.6 is a controllable-reasoning model whose chat template defaults to `detailed thinking off`; the SGLang `qwen3` reasoning parser, in contrast, assumes default-thinking semantics and would mis-route normal output into `reasoning_content`. Only enable it if you specifically want `<think>...</think>` blocks split out — see [§4.3 Thinking Mode](#4-3-thinking-mode).
+- **MTP (multi-token prediction)** is supported. Add `--speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --mamba-radix-cache-strategy extra_buffer` to enable it — see the [model card](https://huggingface.co/inclusionAI/Ling-2.6-flash#run-inference) for the full example.
+
+### 3.2 Ling-2.6-1T
+
+Ling-2.6-1T ships in **FP8 (E4M3)**, so unlike Ling-2.5-1T it fits a **single GB300 node with `--tp 4`**. On smaller GPUs (H200/B200), a 2-node deployment with `--pp-size 2` is required.
+
+import { Ling261TDeployment } from '/src/snippets/autoregressive/ling-26-1t-deployment.jsx'
+
+<Ling261TDeployment />
+
+#### Configuration Tips
+
+- `--trust-remote-code` is required for the custom modeling code.
+- `--model-loader-extra-config '{"enable_multithread_load":"true","num_threads":64}'` significantly speeds up the multi-shard FP8 weight load (26 safetensors shards + an MTP layer).
+- Use `--tool-call-parser qwen` for tool calling.
+- The recommended baseline does **not** include `--reasoning-parser qwen3`. Ling-2.6's chat template defaults to `detailed thinking off`, while SGLang's `qwen3` reasoning parser assumes default-thinking semantics — combining the two requires a per-request workaround for tool calls (see [§4.3 Thinking Mode](#4-3-thinking-mode)). Only enable `--reasoning-parser qwen3` if you specifically want `<think>...</think>` blocks split into `reasoning_content`.
+- For 2-node deployments, set `MASTER_IP`, `PORT`, and `DIST_PORT` consistently across both nodes.
+
+## 4. Model Invocation
+
+For example, launch a Ling-2.6-1T server on a single GB300 node:
+
+```bash Command
+sglang serve \
+  --model-path inclusionAI/Ling-2.6-1T \
+  --tp-size 4 \
+  --trust-remote-code \
+  --host 0.0.0.0 \
+  --port 30000 \
+  --tool-call-parser qwen \
+  --model-loader-extra-config '{"enable_multithread_load":"true","num_threads":64}'
+```
+
+### 4.1 Basic Usage
+
+```bash Command
+curl -s http://${MASTER_IP}:${PORT}/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{"model": "auto", "messages": [{"role": "user", "content": "What is the capital of France?"}]}'
+```
+
+Output:
+```json Config
+{
+  "id": "...",
+  "object": "chat.completion",
+  "model": "auto",
+  "choices": [
+    {
+      "index": 0,
+      "message": {
+        "role": "assistant",
+        "content": "The capital of France is **Paris**.",
+        "reasoning_content": null,
+        "tool_calls": null
+      },
+      "finish_reason": "stop"
+    }
+  ]
+}
+```
+
+### 4.2 Tool Calling Example
+
+```bash Command
+curl -s http://${MASTER_IP}:${PORT}/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "auto",
+    "messages": [{"role": "user", "content": "Search for the latest news about AI"}],
+    "tools": [{
+      "type": "function",
+      "function": {
+        "name": "search",
+        "description": "Search for information on the internet",
+        "parameters": {
+          "type": "object",
+          "properties": {
+            "query": {"type": "string", "description": "The search query"}
+          },
+          "required": ["query"]
+        }
+      }
+    }],
+    "tool_choice": "auto"
+  }'
+```
+
+Output:
+```json Config
+{
+  "choices": [
+    {
+      "message": {
+        "role": "assistant",
+        "content": null,
+        "tool_calls": [
+          {
+            "id": "call_...",
+            "type": "function",
+            "function": {
+              "name": "search",
+              "arguments": "{\"query\": \"latest news about AI\"}"
+            }
+          }
+        ]
+      },
+      "finish_reason": "tool_calls"
+    }
+  ]
+}
+```
+
+### 4.3 Thinking Mode
+
+Both Ling-2.6-flash and Ling-2.6-1T are **controllable-reasoning** models. Their chat template uses textual directives in the system message — `detailed thinking on` or `detailed thinking off` — to toggle thinking. The template **defaults to `detailed thinking off`** when neither phrase is present, and it does **not** read the Qwen3-style `enable_thinking` template variable.
+
+#### Enabling thinking
+
+Include `detailed thinking on` in the first system message:
+
+```bash Command
+curl -s http://${MASTER_IP}:${PORT}/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "auto",
+    "messages": [
+      {"role": "system", "content": "detailed thinking on"},
+      {"role": "user", "content": "If a box has 12 red balls and 8 blue balls, then 5 red balls are removed, how many balls remain?"}
+    ]
+  }'
+```
+
+If you already have a system prompt, append the directive on its own line:
+
+```json
+{"role": "system", "content": "You are a helpful assistant.\ndetailed thinking on"}
+```
+
+When thinking is on, the model emits `<think>...</think>` blocks before its final answer. To get those split into `message.reasoning_content` automatically, also launch the server with `--reasoning-parser qwen3`.
+
+#### Caveat: `--reasoning-parser qwen3` + tool calling
+
+The SGLang `qwen3` reasoning parser was written for Qwen3, where models are **default-thinking** and clients opt out via `chat_template_kwargs.enable_thinking=false`. Ling-2.6 is the opposite — default-non-thinking, with toggling done in the system message. As a result, when the server is launched with **both** `--tool-call-parser qwen` and `--reasoning-parser qwen3`, every tool-call request must include `chat_template_kwargs.enable_thinking=false`, otherwise the parser routes the `<tool_call>...</tool_call>` block into `reasoning_content` instead of `message.tool_calls`:
+
+```bash Command
+curl -s http://${MASTER_IP}:${PORT}/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "auto",
+    "messages": [{"role": "user", "content": "Search for the latest news about AI"}],
+    "tools": [...],
+    "tool_choice": "auto",
+    "chat_template_kwargs": {"enable_thinking": false}
+  }'
+```
+
+`enable_thinking` here is consumed by the SGLang reasoning parser, **not** by the chat template — Ling-2.6's template ignores it. For the simplest configuration, just omit `--reasoning-parser qwen3` and toggle thinking via the system message.
+
+For more API examples, see the [SGLang Basic Usage Guide](../../../docs/basic_usage/send_request).
+
+## 5. Benchmark
+
+### GSM8K (Ling-2.6-1T, GB300 × 4)
+
+Reference run on a single GB300 node with `--tp 4`:
+
+```bash Command
+python3 benchmark/gsm8k/bench_sglang.py
+```
+
+```text Output
+Accuracy: 0.9621 (1269 / 1319)
+```
+
+For Ling-2.6-flash, see the official numbers on the [model card](https://huggingface.co/inclusionAI/Ling-2.6-flash) (BFCL-V4, TAU2-bench, SWE-bench Verified, Claw-Eval, PinchBench, Artificial Analysis).

docs/cookbook/autoregressive/InclusionAI/Ring-2.5-1T.mdx

@@ -0,0 +1,262 @@
+---
+title: Ring-2.5-1T
+metatags:
+    description: "Deploy Ring-2.5-1T with SGLang - world's first open-source 1T parameter reasoning model with hybrid linear attention, deep reasoning, and agentic tool calling capabilities."
+---
+
+## 1. Model Introduction
+
+[Ring-2.5-1T](https://huggingface.co/inclusionAI/Ring-2.5-1T) is the world's first open-source trillion-parameter reasoning model based on hybrid linear attention architecture, developed by InclusionAI. Building on Ring-1T, Ring-2.5-1T demonstrates substantial improvements in generation efficiency, reasoning depth, and long-horizon task execution capabilities.
+
+**Key Features:**
+
+- **Trillion-Scale Model**: ~1T total parameters with 63B activation parameters using a hybrid linear attention architecture (1:7 MLA + Lightning Linear Attention)
+- **Generation Efficiency**: Reduces memory access overhead by over 10x and increases generation throughput by more than 3x for sequences exceeding 32K tokens
+- **Deep Reasoning**: Achieves gold medal level for both IMO 2025 and CMO 2025, with dense rewards for rigorous reasoning process feedback
+- **Long-horizon Task Execution**: Enhanced autonomous execution capability through large-scale fully-async agentic RL training
+- **Tool Calling**: Supports function calling with XML-style tool call format
+- **Context Length**: 128K -> 256K (YaRN)
+
+**Available Models:**
+
+- **FP8 (8-bit quantized)**: [inclusionAI/Ring-2.5-1T](https://huggingface.co/inclusionAI/Ring-2.5-1T)
+
+**License:** MIT
+
+## 2. SGLang Installation
+
+Ring-2.5-1T runs on the standard SGLang Docker image:
+
+```bash Command
+# NVIDIA (H200 / B200 / GB200 / GB300)
+docker pull lmsysorg/sglang:latest
+
+# For MI300X/325X
+docker pull lmsysorg/sglang:v0.5.9-rocm700-mi30x
+
+# For MI355X
+docker pull lmsysorg/sglang:v0.5.9-rocm700-mi35x
+```
+
+For other installation methods, please refer to the [official SGLang installation guide](../../../docs/get-started/install).
+
+## 3. Model Deployment
+
+This section provides deployment configurations optimized for different hardware platforms.
+
+### 3.1 Basic Configuration
+
+**Interactive Command Generator**: Use the configuration selector below to automatically generate the appropriate deployment command for your hardware platform.
+
+import { Ring251TDeployment } from '/src/snippets/autoregressive/ring-25-1t-deployment.jsx'
+
+<Ring251TDeployment />
+
+### 3.2 Configuration Tips
+
+- The `--trust-remote-code` flag is required for this model due to custom modeling code.
+- The model uses FP8 quantization (compressed-tensors format).
+
+## 4. Model Invocation
+
+Deploy Ring-2.5-1T with the following command (on H200, all features enabled):
+
+```shell Command
+sglang serve \
+  --model-path inclusionAI/Ring-2.5-1T \
+  --tp 8 \
+  --trust-remote-code \
+  --host 0.0.0.0 \
+  --port 30000
+```
+
+### 4.1 Basic Usage
+
+For basic API usage and request examples, please refer to:
+
+- [SGLang Basic Usage Guide](../../../docs/basic_usage/send_request)
+
+### 4.2 Advanced Usage
+
+#### 4.2.1 Reasoning Parser
+
+To enable reasoning output separation, add `--reasoning-parser deepseek-r1` when launching the server. The thinking process is returned via `reasoning_content` in the streaming response.
+
+```shell Command
+sglang serve \
+  --model-path inclusionAI/Ring-2.5-1T \
+  --tp 8 \
+  --trust-remote-code \
+  --reasoning-parser deepseek-r1 \
+  --host 0.0.0.0 \
+  --port 30000
+```
+
+```python Example
+from openai import OpenAI
+
+client = OpenAI(
+    base_url="http://localhost:30000/v1",
+    api_key="EMPTY"
+)
+
+response = client.chat.completions.create(
+    model="inclusionAI/Ring-2.5-1T",
+    messages=[
+        {"role": "user", "content": "Solve this problem step by step: What is 15% of 240?"}
+    ],
+    max_tokens=2048,
+    stream=True
+)
+
+for chunk in response:
+    if chunk.choices and len(chunk.choices) > 0:
+        delta = chunk.choices[0].delta
+
+        if hasattr(delta, 'reasoning_content') and delta.reasoning_content:
+            print(delta.reasoning_content, end="", flush=True)
+
+        if delta.content:
+            print(delta.content, end="", flush=True)
+
+print()
+```
+
+<details>
+<summary>Output Example</summary>
+
+````text Output
+We are asked: "Solve this problem step by step: What is 15% of 240?" This is a straightforward percentage calculation. We need to show step-by-step solution.
+
+We can compute 15% of 240 as (15/100)*240 = 0.15 * 240 = 36.
+
+But we need to present step by step. Also ensure it's clear.
+
+We could also break down: 10% of 240 = 24, then 5% = 12, so 15% = 36.
+
+But any method is fine.
+
+We'll produce a solution with explanation: "To find 15% of 240, multiply 240 by 0.15 (or 15/100)."
+
+We'll show:
+
+15% = 15/100 = 0.15
+
+Then 0.15 × 240 = 36.
+
+Alternatively: (15/100) × 240 = (15 × 240) / 100 = 3600/100 = 36.
+
+Finally, answer: 36.
+
+We can also illustrate stepwise: "First, convert the percentage to a decimal: 15% = 0.15. Then multiply by the number: 0.15 × 240 = 36."
+
+We'll present as a final answer: \boxed{36}.
+
+However, we need to provide step-by-step solution as per instructions. We'll write a full explanation.
+
+We can also use the fraction method: 15% of 240 = (15/100)*240 = (15*240)/100 = 3600/100 = 36.
+
+Alr.
+
+I think that's it.
+
+
+**Step 1:** Write 15% as a fraction or decimal.
+\[ 15\% = \frac{15}{100} = 0.15\]
+
+**Step 2:** Multiply the number (240) by this fraction/decimal.
+\[ 240 \times 0.15 = 36\]
+
+Alternatively, using the fraction:
+\[ \frac{15}{100} \times 240 = \frac{15 \times 240}{100} = \frac{3600}{100} = 36\]
+
+**Conclusion:** 15% of 240 is 36.
+
+\[ \boxed{36} \]
+````
+
+</details>
+
+#### 4.2.2 Tool Calling
+
+To enable tool calling, add `--tool-call-parser qwen` when launching the server.
+
+```shell Command
+sglang serve \
+  --model-path inclusionAI/Ring-2.5-1T \
+  --tp 8 \
+  --trust-remote-code \
+  --tool-call-parser qwen \
+  --host 0.0.0.0 \
+  --port 30000
+```
+
+```python Example
+from openai import OpenAI
+
+client = OpenAI(
+    base_url="http://localhost:30000/v1",
+    api_key="EMPTY"
+)
+
+tools = [
+    {
+        "type": "function",
+        "function": {
+            "name": "get_weather",
+            "description": "Get the current weather for a location",
+            "parameters": {
+                "type": "object",
+                "properties": {
+                    "location": {
+                        "type": "string",
+                        "description": "The city name"
+                    }
+                },
+                "required": ["location"]
+            }
+        }
+    }
+]
+
+response = client.chat.completions.create(
+    model="inclusionAI/Ring-2.5-1T",
+    messages=[
+        {"role": "user", "content": "What's the weather in Beijing?"}
+    ],
+    tools=tools
+)
+
+print(response.choices[0].message.tool_calls)
+```
+
+**Output Example:**
+
+```text Output
+[ChatCompletionMessageFunctionToolCall(id='call_770360e31d194ed79d32cd8c', function=Function(arguments='{"location": "Beijing"}', name='get_weather'), type='function', index=0)]
+```
+
+## 5. Benchmark
+
+### GSM8K
+
+- Deployment Command
+```bash Command
+sglang serve \
+  --model-path inclusionAI/Ring-2.5-1T \
+  --tp-size 8 \
+  --trust-remote-code
+```
+
+- Benchmark Command
+```bash Command
+python3 benchmark/gsm8k/bench_sglang.py --temperature 1.2 --top-p 0.8 --max-new-tokens 32768 --num-questions 200 --tokenizer-path inclusionAI/Ring-2.5-1T --enable-thinking
+```
+
+- Test Result
+```text Output
+Accuracy: 0.955
+Invalid: 0.010
+Latency: 615.833 s
+Output throughput: 412.360 token/s
+```

docs/cookbook/autoregressive/InclusionAI/Ring-2.6-1T.mdx

@@ -0,0 +1,374 @@
+---
+title: Ring-2.6-1T
+metatags:
+    description: "Deploy Ring-2.6-1T with SGLang - a trillion-parameter InclusionAI reasoning model for agent workflows, high/xhigh reasoning effort, and tool use."
+tag: NEW
+---
+
+## 1. Model Introduction
+
+[Ring-2.6-1T](https://huggingface.co/inclusionAI/Ring-2.6-1T) is InclusionAI's trillion-parameter flagship reasoning model for real-world complex task execution. It targets agent workflows, engineering development, scientific research analysis, enterprise automation, and other long-horizon settings where the model must plan, use tools, recover from intermediate errors, and keep context across multiple steps.
+
+**Key Features:**
+
+- **Trillion-Scale Reasoning Model**: `BailingMoeV2_5ForCausalLM` with a `bailing_hybrid` architecture, 80 hidden layers, 256 routed experts, 8 selected experts per token, and FP8 compressed-tensors weights.
+- **Agent Execution**: Designed for multi-step task decomposition, tool collaboration, context continuation, and long-horizon execution. The model card reports 87.60 on PinchBench, 63.82 on ClawEval, and 95.32 on Tau2-Bench Telecom for the `high` setting.
+- **Reasoning Effort**: The model card describes `high` and `xhigh` reasoning-effort modes. In SGLang's OpenAI-compatible chat API, use top-level `reasoning_effort: "high"` for production agent workflows. To request the model-card `xhigh` prompt path, pass it through `chat_template_kwargs.reasoning_effort`.
+- **Hybrid Attention**: Uses the Bailing hybrid stack with MLA plus Lightning linear attention kernels in SGLang.
+- **Context Length**: Native 128K in the released config. Configure YaRN separately if you need a 256K deployment.
+
+**Available Models:**
+
+- **FP8 (E4M3 compressed-tensors)**: [inclusionAI/Ring-2.6-1T](https://huggingface.co/inclusionAI/Ring-2.6-1T)
+
+**License:** MIT
+
+## 2. SGLang Installation
+
+Ring-2.6-1T requires recent SGLang builds with Bailing hybrid model support. Start with the latest SGLang Docker image when validating this cookbook:
+
+```bash Command
+docker pull lmsysorg/sglang:latest
+```
+
+For other installation methods, please refer to the [official SGLang installation guide](../../../docs/get-started/install).
+
+## 3. Model Deployment
+
+Use the selector below to generate a single-node command for the tested hardware targets.
+
+import { Ring261TDeployment } from '/src/snippets/autoregressive/ring-26-1t-deployment.jsx'
+
+<Ring261TDeployment />
+
+### Configuration Tips
+
+- `--trust-remote-code` is required for the model's custom Bailing hybrid implementation.
+- Use `--tp-size 4` on a single 4-GPU GB300 node.
+- Use `--tp-size 8` on a single 8-GPU B200 node.
+- Use `--tp-size 8` on a single 8-GPU H200 node.
+- Use `--mem-fraction-static 0.95` on GB300 x4. The model uses about 238.5GB/GPU after loading, so lower values can fail during KV-pool initialization.
+- Use `--mem-fraction-static 0.8` on B200 x8.
+- Use `--mem-fraction-static 0.95` on H200 x8.
+- `--model-loader-extra-config '{"enable_multithread_load":"true","num_threads":64}'` is recommended because the model has 175 large safetensors shards.
+- Keep `--tool-call-parser glm` enabled by default for OpenAI-compatible tool calls. Ring's template emits XML `<arg_key>/<arg_value>` tool calls, which the `qwen` parser does not convert into `message.tool_calls`.
+- Keep `--reasoning-parser deepseek-r1` enabled by default so `<think>...</think>` content is split into `message.reasoning_content`.
+
+## 4. Model Invocation
+
+### 4.1 Basic Usage
+
+For example, launch the server on a single 4-GPU GB300 node:
+
+```bash Command
+export PORT=30000
+
+sglang serve \
+  --model-path inclusionAI/Ring-2.6-1T \
+  --tp-size 4 \
+  --trust-remote-code \
+  --host 0.0.0.0 \
+  --port ${PORT} \
+  --mem-fraction-static 0.95 \
+  --model-loader-extra-config '{"enable_multithread_load":"true","num_threads":64}' \
+  --tool-call-parser glm \
+  --reasoning-parser deepseek-r1
+```
+
+Send a basic chat request:
+
+```bash Command
+curl -s http://localhost:${PORT}/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "auto",
+    "messages": [{"role": "user", "content": "What is the capital of France?"}],
+    "max_tokens": 128
+  }'
+```
+
+### 4.2 Reasoning Effort
+
+Ring-2.6-1T exposes two reasoning-effort levels in the model card: `high` and `xhigh`. In SGLang's OpenAI-compatible chat API, start with top-level `reasoning_effort: "high"` for agent and production workflows:
+
+```bash Command
+curl -s http://localhost:${PORT}/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "auto",
+    "messages": [{"role": "user", "content": "Solve: if 3x + 7 = 52, what is x?"}],
+    "reasoning_effort": "high",
+    "max_tokens": 512
+}'
+```
+
+For the model-card `xhigh` path, pass the template value explicitly:
+
+```bash Command
+curl -s http://localhost:${PORT}/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "auto",
+    "messages": [{"role": "user", "content": "Solve: if 3x + 7 = 52, what is x?"}],
+    "chat_template_kwargs": {"reasoning_effort": "xhigh"},
+    "max_tokens": 512
+  }'
+```
+
+With the default deployment command, thinking text is separated into `message.reasoning_content` when the model emits `<think>...</think>` blocks.
+
+### 4.3 Tool Calling Example
+
+```bash Command
+curl -s http://localhost:${PORT}/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "auto",
+    "messages": [{"role": "user", "content": "What is the weather in Beijing?"}],
+    "tools": [{
+      "type": "function",
+      "function": {
+        "name": "get_weather",
+        "description": "Get the current weather for a location",
+        "parameters": {
+          "type": "object",
+          "properties": {
+            "location": {"type": "string", "description": "The city name"}
+          },
+          "required": ["location"]
+        }
+      }
+    }],
+    "tool_choice": "auto",
+    "max_tokens": 512
+  }'
+```
+
+For more API examples, see the [SGLang Basic Usage Guide](../../../docs/basic_usage/send_request).
+
+## 5. Benchmark
+
+### 5.1 Speed Benchmark
+
+- Hardware: NVIDIA B200 GPU (8x), NVIDIA H200 GPU (8x), and NVIDIA GB300 GPU (4x)
+- Model: `inclusionAI/Ring-2.6-1T`
+- Docker image: `lmsysorg/sglang:latest`
+- SGLang version tested: `0.5.11`
+- Tensor Parallelism: 8 on B200 x8 and H200 x8, 4 on GB300 x4
+
+Use the deployment command from [Section 3](#3-model-deployment), then confirm that the server is healthy before running benchmarks:
+
+```bash Command
+curl -s http://localhost:${PORT}/health
+curl -s http://localhost:${PORT}/v1/models
+```
+
+#### 5.1.1 Latency-Sensitive Benchmark
+
+- Test Command:
+
+```bash Command
+python3 -m sglang.bench_serving \
+  --backend sglang \
+  --host 127.0.0.1 \
+  --port ${PORT} \
+  --model inclusionAI/Ring-2.6-1T \
+  --dataset-name random \
+  --random-input-len 1024 \
+  --random-output-len 1024 \
+  --num-prompts 10 \
+  --max-concurrency 1 \
+  --request-rate inf
+```
+
+- Test Results (B200 x8):
+
+```text Output
+============ Serving Benchmark Result ============
+Backend:                                 sglang
+Traffic request rate:                    inf
+Max request concurrency:                 1
+Successful requests:                     10
+Benchmark duration (s):                  207.18
+Total input tokens:                      6101
+Total generated tokens:                  4220
+Request throughput (req/s):              0.05
+Input token throughput (tok/s):          29.45
+Output token throughput (tok/s):         20.37
+Total token throughput (tok/s):          49.82
+Mean E2E Latency (ms):                   20715.16
+Mean TTFT (ms):                          187.86
+Mean TPOT (ms):                          44.65
+Mean ITL (ms):                           48.76
+==================================================
+```
+
+- Test Results (GB300 x4):
+
+```text Output
+============ Serving Benchmark Result ============
+Backend:                                 sglang
+Traffic request rate:                    inf
+Max request concurrency:                 1
+Successful requests:                     10
+Benchmark duration (s):                  62.21
+Total input tokens:                      6101
+Total generated tokens:                  4220
+Request throughput (req/s):              0.16
+Input token throughput (tok/s):          98.07
+Output token throughput (tok/s):         67.83
+Total token throughput (tok/s):          165.91
+Mean E2E Latency (ms):                   6218.57
+Mean TTFT (ms):                          233.04
+Mean TPOT (ms):                          14.21
+Mean ITL (ms):                           14.22
+==================================================
+```
+
+- Test Results (H200 x8):
+
+```text Output
+============ Serving Benchmark Result ============
+Backend:                                 sglang
+Traffic request rate:                    inf
+Max request concurrency:                 1
+Successful requests:                     10
+Benchmark duration (s):                  57.10
+Total input tokens:                      6101
+Total generated tokens:                  4220
+Request throughput (req/s):              0.18
+Input token throughput (tok/s):          106.85
+Output token throughput (tok/s):         73.91
+Total token throughput (tok/s):          180.76
+Mean E2E Latency (ms):                   5707.72
+Mean TTFT (ms):                          163.35
+Mean TPOT (ms):                          13.17
+Mean ITL (ms):                           13.17
+==================================================
+```
+
+#### 5.1.2 Throughput-Sensitive Benchmark
+
+- Test Command:
+
+```bash Command
+python3 -m sglang.bench_serving \
+  --backend sglang \
+  --host 127.0.0.1 \
+  --port ${PORT} \
+  --model inclusionAI/Ring-2.6-1T \
+  --dataset-name random \
+  --random-input-len 1024 \
+  --random-output-len 1024 \
+  --num-prompts 100 \
+  --max-concurrency 100 \
+  --request-rate inf
+```
+
+- Test Results (B200 x8):
+
+```text Output
+============ Serving Benchmark Result ============
+Backend:                                 sglang
+Traffic request rate:                    inf
+Max request concurrency:                 100
+Successful requests:                     100
+Benchmark duration (s):                  46.30
+Total input tokens:                      50561
+Total generated tokens:                  52444
+Request throughput (req/s):              2.16
+Input token throughput (tok/s):          1092.10
+Output token throughput (tok/s):         1132.77
+Total token throughput (tok/s):          2224.86
+Mean E2E Latency (ms):                   27581.74
+Mean TTFT (ms):                          1710.53
+Mean TPOT (ms):                          51.27
+Mean ITL (ms):                           49.43
+==================================================
+```
+
+- Test Results (GB300 x4):
+
+```text Output
+============ Serving Benchmark Result ============
+Backend:                                 sglang
+Traffic request rate:                    inf
+Max request concurrency:                 100
+Successful requests:                     100
+Benchmark duration (s):                  55.80
+Total input tokens:                      50561
+Total generated tokens:                  52444
+Request throughput (req/s):              1.79
+Input token throughput (tok/s):          906.10
+Output token throughput (tok/s):         939.84
+Total token throughput (tok/s):          1845.94
+Mean E2E Latency (ms):                   33736.85
+Mean TTFT (ms):                          2156.40
+Mean TPOT (ms):                          63.09
+Mean ITL (ms):                           60.33
+==================================================
+```
+
+- Test Results (H200 x8):
+
+```text Output
+============ Serving Benchmark Result ============
+Backend:                                 sglang
+Traffic request rate:                    inf
+Max request concurrency:                 100
+Successful requests:                     100
+Benchmark duration (s):                  44.51
+Total input tokens:                      50561
+Total generated tokens:                  52444
+Request throughput (req/s):              2.25
+Input token throughput (tok/s):          1135.88
+Output token throughput (tok/s):         1178.18
+Total token throughput (tok/s):          2314.06
+Mean E2E Latency (ms):                   27177.14
+Mean TTFT (ms):                          2173.08
+Mean TPOT (ms):                          51.11
+Mean ITL (ms):                           47.77
+==================================================
+```
+
+### 5.2 Accuracy Benchmark
+
+#### 5.2.1 GSM8K Benchmark
+
+- Benchmark Command:
+
+```bash Command
+python3 -m sglang.test.run_eval \
+  --eval-name gsm8k \
+  --host 127.0.0.1 \
+  --port ${PORT} \
+  --model auto \
+  --num-examples 200 \
+  --num-threads 64 \
+  --max-tokens 2048 \
+  --reasoning-effort high
+```
+
+- Test Results (B200 x8):
+
+```text Output
+Total latency: 100.378 s
+Score: 0.990
+Output throughput: 627.401 token/s
+```
+
+- Test Results (GB300 x4):
+
+```text Output
+Total latency: 98.386 s
+Score: 0.990
+Output throughput: 621.469 token/s
+```
+
+- Test Results (H200 x8):
+
+```text Output
+Total latency: 76.849 s
+Score: 0.990
+Output throughput: 793.125 token/s
+```