[Doc]: refactor Intern-S2-Preview cookbook with interactive command generator (#25134)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.7
parent
adae4042a7
commit
b0018ad015
@@ -2,13 +2,20 @@
|
|||||||
title: Intern-S2-Preview
|
title: Intern-S2-Preview
|
||||||
metatags:
|
metatags:
|
||||||
description: "Deploy Intern-S2-Preview with SGLang"
|
description: "Deploy Intern-S2-Preview with SGLang"
|
||||||
|
tag: NEW
|
||||||
---
|
---
|
||||||
|
|
||||||
## 1. Model introduction
|
## 1. Model Introduction
|
||||||
|
|
||||||
[Intern-S2-Preview](https://huggingface.co/internLM/Intern-S2-Preview) is an efficient 35B scientific multimodal foundation model. Beyond conventional parameter and data scaling, Intern-S2-Preview explores task scaling: increasing the difficulty, diversity, and coverage of scientific tasks to further unlock model capabilities.
|
**Intern-S2-Preview** is an efficient 35B scientific multimodal foundation model. Beyond conventional parameter and data scaling, Intern-S2-Preview explores task scaling: increasing the difficulty, diversity, and coverage of scientific tasks to further unlock model capabilities.
|
||||||
|
|
||||||
## 2. SGLang installation
|
**Resources:**
|
||||||
|
|
||||||
|
- HuggingFace: [internLM/Intern-S2-Preview](https://huggingface.co/internLM/Intern-S2-Preview)
|
||||||
|
|
||||||
|
## 2. SGLang Installation
|
||||||
|
|
||||||
|
SGLang offers multiple installation methods. Please refer to the [official SGLang installation guide](../../../docs/get-started/install) for installation instructions.
|
||||||
|
|
||||||
Install SGLang from source or use an NVIDIA Docker image:
|
Install SGLang from source or use an NVIDIA Docker image:
|
||||||
|
|
||||||
@@ -19,63 +26,48 @@ uv pip install 'git+https://github.com/sgl-project/sglang.git#subdirectory=pytho
|
|||||||
# Or use Docker for NVIDIA GPUs
|
# Or use Docker for NVIDIA GPUs
|
||||||
docker pull lmsysorg/sglang:latest
|
docker pull lmsysorg/sglang:latest
|
||||||
```
|
```
|
||||||
|
For how to actually launch a docker image, see [Install → Method 3: Using Docker](../../../docs/get-started/install#method-3-using-docker). A minimal example (substitute the inner `sglang serve ...` with whatever the [command generator](#3-model-deployment) below produces):
|
||||||
|
|
||||||
For full installation details, see the [SGLang installation guide](/docs/get-started/install).
|
```bash Command
|
||||||
|
docker run --gpus all \
|
||||||
## 3. Model deployment
|
--shm-size 32g \
|
||||||
|
-p 30000:30000 \
|
||||||
|
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
||||||
**NVIDIA:**
|
--env "HF_TOKEN=<your-hf-token>" \
|
||||||
|
--ipc=host \
|
||||||
Deploy internLM/Intern-S2-Preview with the following commands:
|
lmsysorg/sglang:latest \
|
||||||
|
sglang serve <use args below>
|
||||||
### Standard Version
|
|
||||||
|
|
||||||
```shell Command
|
|
||||||
sglang serve \
|
|
||||||
--model-path internLM/Intern-S2-Preview \
|
|
||||||
--tp 8 \
|
|
||||||
--reasoning-parser qwen3 \
|
|
||||||
--tool-call-parser qwen3_coder \
|
|
||||||
--mem-fraction-static 0.8 \
|
|
||||||
--host 0.0.0.0 \
|
|
||||||
--port 30000
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Multi-Token Prediction (MTP)
|
## 3. Model Deployment
|
||||||
|
|
||||||
```shell Command
|
### 3.1 Basic Configuration
|
||||||
SGLANG_ENABLE_SPEC_V2=1 \
|
|
||||||
sglang serve \
|
|
||||||
--model-path internLM/Intern-S2-Preview \
|
|
||||||
--tp 8 \
|
|
||||||
--reasoning-parser qwen3 \
|
|
||||||
--tool-call-parser qwen3_coder \
|
|
||||||
--mamba-scheduler-strategy extra_buffer \
|
|
||||||
--speculative-algo 'NEXTN' \
|
|
||||||
--speculative-eagle-topk 1 \
|
|
||||||
--speculative-num-steps 3 \
|
|
||||||
--speculative-num-draft-tokens 4 \
|
|
||||||
--mem-fraction-static 0.8 \
|
|
||||||
--host 0.0.0.0 \
|
|
||||||
--port 30000
|
|
||||||
```
|
|
||||||
|
|
||||||
### Configuration tips
|
**Interactive Command Generator**: Use the selector below to generate the deployment command for your hardware and parser configuration.
|
||||||
|
|
||||||
|
import { InternS2PreviewDeployment } from "/src/snippets/autoregressive/intern-s2-preview-deployment.jsx";
|
||||||
|
|
||||||
|
<InternS2PreviewDeployment />
|
||||||
|
|
||||||
|
### 3.2 Configuration Tips
|
||||||
|
|
||||||
- Use `tp>=2` for the NVIDIA deployment commands.
|
- Use `tp>=2` for the NVIDIA deployment commands.
|
||||||
- Use `--reasoning-parser qwen3` to separate reasoning content from final content in streaming responses.
|
- Use `--reasoning-parser qwen3` to separate reasoning content from final content in streaming responses.
|
||||||
- Use `--tool-call-parser qwen3_coder` when serving tool-calling workloads.
|
- Use `--tool-call-parser qwen3_coder` when serving tool-calling workloads.
|
||||||
- Add `--mamba-scheduler-strategy extra_buffer with `--speculative-algo 'NEXTN'` to enable MTP.
|
- Add `--mamba-scheduler-strategy extra_buffer` with `--speculative-algo 'NEXTN'` to enable MTP.
|
||||||
- If weight loading is slow, add `--model-loader-extra-config='{"enable_multithread_load": "true","num_threads": 64}'`.
|
- If weight loading is slow, add `--model-loader-extra-config='{"enable_multithread_load": "true", "num_threads": 64}'`.
|
||||||
|
|
||||||
## 4. Model invocation
|
## 4. Model Invocation
|
||||||
|
|
||||||
### 4.1 Basic usage
|
### 4.1 Basic Usage
|
||||||
|
|
||||||
For basic API usage and request examples, see the [SGLang basic usage guide](/docs/basic_usage/send_request).
|
For basic API usage and request examples, see:
|
||||||
|
|
||||||
### 4.2 Vision input
|
- [Basic API Usage](../../../docs/basic_usage/send_request)
|
||||||
|
|
||||||
|
### 4.2 Advanced Usage
|
||||||
|
|
||||||
|
#### 4.2.1 Vision Input
|
||||||
|
|
||||||
Intern-S2-Preview supports image inputs. Here is an example with an image:
|
Intern-S2-Preview supports image inputs. Here is an example with an image:
|
||||||
|
|
||||||
@@ -134,7 +126,7 @@ for chunk in response:
|
|||||||
print()
|
print()
|
||||||
```
|
```
|
||||||
|
|
||||||
### 4.3 Reasoning parser
|
#### 4.2.2 Reasoning Parser
|
||||||
|
|
||||||
Enable streaming to read reasoning content separately from the final answer:
|
Enable streaming to read reasoning content separately from the final answer:
|
||||||
|
|
||||||
@@ -179,7 +171,7 @@ for chunk in response:
|
|||||||
print()
|
print()
|
||||||
```
|
```
|
||||||
|
|
||||||
### 4.4 Tool calling
|
#### 4.2.3 Tool Calling
|
||||||
|
|
||||||
Serve with `--tool-call-parser qwen3_coder` enabled, then send OpenAI-compatible tool requests:
|
Serve with `--tool-call-parser qwen3_coder` enabled, then send OpenAI-compatible tool requests:
|
||||||
|
|
||||||
|
|||||||
@@ -82,7 +82,6 @@ metatags:
|
|||||||
<Card
|
<Card
|
||||||
title="InternLM"
|
title="InternLM"
|
||||||
mode="card"
|
mode="card"
|
||||||
href="/cookbook/autoregressive/InternLM/Intern-S1"
|
|
||||||
href="/cookbook/autoregressive/InternLM/Intern-S2-Preview"
|
href="/cookbook/autoregressive/InternLM/Intern-S2-Preview"
|
||||||
img="/cards/logos/internlm.png"
|
img="/cards/logos/internlm.png"
|
||||||
/>
|
/>
|
||||||
|
|||||||
+2
-2
@@ -1042,8 +1042,8 @@
|
|||||||
{
|
{
|
||||||
"group": "InternLM",
|
"group": "InternLM",
|
||||||
"pages": [
|
"pages": [
|
||||||
"cookbook/autoregressive/InternLM/Intern-S1",
|
"cookbook/autoregressive/InternLM/Intern-S2-Preview",
|
||||||
"cookbook/autoregressive/InternLM/Intern-S2-Preview"
|
"cookbook/autoregressive/InternLM/Intern-S1"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -0,0 +1,179 @@
|
|||||||
|
export const InternS2PreviewDeployment = () => {
|
||||||
|
const options = {
|
||||||
|
hardware: {
|
||||||
|
name: 'hardware',
|
||||||
|
title: 'Hardware Platform',
|
||||||
|
items: [
|
||||||
|
{ id: 'h200', label: 'H200', default: true },
|
||||||
|
],
|
||||||
|
},
|
||||||
|
reasoning: {
|
||||||
|
name: 'reasoning',
|
||||||
|
title: 'Reasoning Parser',
|
||||||
|
items: [
|
||||||
|
{ id: 'disabled', label: 'Disabled', default: false },
|
||||||
|
{ id: 'enabled', label: 'Enabled', default: true },
|
||||||
|
],
|
||||||
|
},
|
||||||
|
toolcall: {
|
||||||
|
name: 'toolcall',
|
||||||
|
title: 'Tool Call Parser',
|
||||||
|
items: [
|
||||||
|
{ id: 'disabled', label: 'Disabled', default: false },
|
||||||
|
{ id: 'enabled', label: 'Enabled', default: true },
|
||||||
|
],
|
||||||
|
},
|
||||||
|
mtp: {
|
||||||
|
name: 'mtp',
|
||||||
|
title: 'Multi-Token Prediction',
|
||||||
|
items: [
|
||||||
|
{ id: 'disabled', label: 'Disabled', default: true },
|
||||||
|
{ id: 'enabled', label: 'Enabled', default: false },
|
||||||
|
],
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
const getInitialState = () => {
|
||||||
|
const initialState = {};
|
||||||
|
Object.entries(options).forEach(([key, option]) => {
|
||||||
|
const defaultItem = option.items.find((item) => item.default);
|
||||||
|
initialState[key] = defaultItem ? defaultItem.id : option.items[0].id;
|
||||||
|
});
|
||||||
|
return initialState;
|
||||||
|
};
|
||||||
|
|
||||||
|
const [values, setValues] = useState(getInitialState);
|
||||||
|
const [isDark, setIsDark] = useState(false);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
const checkDarkMode = () => {
|
||||||
|
const html = document.documentElement;
|
||||||
|
const isDarkMode =
|
||||||
|
html.classList.contains('dark') ||
|
||||||
|
html.getAttribute('data-theme') === 'dark' ||
|
||||||
|
html.style.colorScheme === 'dark';
|
||||||
|
setIsDark(isDarkMode);
|
||||||
|
};
|
||||||
|
checkDarkMode();
|
||||||
|
const observer = new MutationObserver(checkDarkMode);
|
||||||
|
observer.observe(document.documentElement, {
|
||||||
|
attributes: true,
|
||||||
|
attributeFilter: ['class', 'data-theme', 'style'],
|
||||||
|
});
|
||||||
|
return () => observer.disconnect();
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
const handleRadioChange = (optionName, value) => {
|
||||||
|
setValues((prev) => ({ ...prev, [optionName]: value }));
|
||||||
|
};
|
||||||
|
|
||||||
|
const generateCommand = () => {
|
||||||
|
const { reasoning, toolcall, mtp } = values;
|
||||||
|
const tpValue = 8;
|
||||||
|
|
||||||
|
const flags = [];
|
||||||
|
flags.push(' --model-path internLM/Intern-S2-Preview');
|
||||||
|
flags.push(` --tp ${tpValue}`);
|
||||||
|
if (reasoning === 'enabled') flags.push(' --reasoning-parser qwen3');
|
||||||
|
if (toolcall === 'enabled') flags.push(' --tool-call-parser qwen3_coder');
|
||||||
|
if (mtp === 'enabled') {
|
||||||
|
flags.push(' --mamba-scheduler-strategy extra_buffer');
|
||||||
|
flags.push(" --speculative-algo 'NEXTN'");
|
||||||
|
flags.push(' --speculative-eagle-topk 1');
|
||||||
|
flags.push(' --speculative-num-steps 3');
|
||||||
|
flags.push(' --speculative-num-draft-tokens 4');
|
||||||
|
}
|
||||||
|
flags.push(' --mem-fraction-static 0.8');
|
||||||
|
flags.push(' --host 0.0.0.0');
|
||||||
|
flags.push(' --port 30000');
|
||||||
|
|
||||||
|
const envPrefix = mtp === 'enabled' ? 'SGLANG_ENABLE_SPEC_V2=1 \\\n' : '';
|
||||||
|
return `${envPrefix}sglang serve \\\n${flags.join(' \\\n')}`;
|
||||||
|
};
|
||||||
|
|
||||||
|
const containerStyle = { maxWidth: '900px', margin: '0 auto', display: 'flex', flexDirection: 'column', gap: '4px' };
|
||||||
|
const cardStyle = {
|
||||||
|
padding: '8px 12px',
|
||||||
|
border: `1px solid ${isDark ? '#374151' : '#e5e7eb'}`,
|
||||||
|
borderLeft: `3px solid ${isDark ? '#E85D4D' : '#D45D44'}`,
|
||||||
|
borderRadius: '4px',
|
||||||
|
display: 'flex',
|
||||||
|
alignItems: 'center',
|
||||||
|
gap: '12px',
|
||||||
|
background: isDark ? '#1f2937' : '#fff',
|
||||||
|
};
|
||||||
|
const titleStyle = { fontSize: '13px', fontWeight: '600', minWidth: '140px', flexShrink: 0, color: isDark ? '#e5e7eb' : 'inherit' };
|
||||||
|
const itemsStyle = { display: 'flex', rowGap: '2px', columnGap: '6px', flexWrap: 'wrap', alignItems: 'center', flex: 1 };
|
||||||
|
const labelBaseStyle = {
|
||||||
|
padding: '4px 10px',
|
||||||
|
border: `1px solid ${isDark ? '#9ca3af' : '#d1d5db'}`,
|
||||||
|
borderRadius: '3px',
|
||||||
|
cursor: 'pointer',
|
||||||
|
display: 'inline-flex',
|
||||||
|
flexDirection: 'column',
|
||||||
|
alignItems: 'center',
|
||||||
|
justifyContent: 'center',
|
||||||
|
fontWeight: '500',
|
||||||
|
fontSize: '13px',
|
||||||
|
transition: 'all 0.2s',
|
||||||
|
userSelect: 'none',
|
||||||
|
minWidth: '45px',
|
||||||
|
textAlign: 'center',
|
||||||
|
flex: 1,
|
||||||
|
background: isDark ? '#374151' : '#fff',
|
||||||
|
color: isDark ? '#e5e7eb' : 'inherit',
|
||||||
|
};
|
||||||
|
const checkedStyle = { background: '#D45D44', color: 'white', borderColor: '#D45D44' };
|
||||||
|
const subtitleStyle = { display: 'block', fontSize: '9px', marginTop: '1px', lineHeight: '1.1', opacity: 0.7 };
|
||||||
|
const commandDisplayStyle = {
|
||||||
|
flex: 1,
|
||||||
|
padding: '12px 16px',
|
||||||
|
background: isDark ? '#111827' : '#f5f5f5',
|
||||||
|
borderRadius: '6px',
|
||||||
|
fontFamily: "'Menlo', 'Monaco', 'Courier New', monospace",
|
||||||
|
fontSize: '12px',
|
||||||
|
lineHeight: '1.5',
|
||||||
|
color: isDark ? '#e5e7eb' : '#374151',
|
||||||
|
whiteSpace: 'pre-wrap',
|
||||||
|
overflowX: 'auto',
|
||||||
|
margin: 0,
|
||||||
|
border: `1px solid ${isDark ? '#374151' : '#e5e7eb'}`,
|
||||||
|
};
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div style={containerStyle} className="not-prose">
|
||||||
|
{Object.entries(options).map(([key, option]) => (
|
||||||
|
<div key={key} style={cardStyle}>
|
||||||
|
<div style={titleStyle}>{option.title}</div>
|
||||||
|
<div style={itemsStyle}>
|
||||||
|
{option.items.map((item) => {
|
||||||
|
const isChecked = values[option.name] === item.id;
|
||||||
|
return (
|
||||||
|
<label key={item.id} style={{ ...labelBaseStyle, ...(isChecked ? checkedStyle : {}) }}>
|
||||||
|
<input
|
||||||
|
type="radio"
|
||||||
|
name={option.name}
|
||||||
|
value={item.id}
|
||||||
|
checked={isChecked}
|
||||||
|
onChange={() => handleRadioChange(option.name, item.id)}
|
||||||
|
style={{ display: 'none' }}
|
||||||
|
/>
|
||||||
|
{item.label}
|
||||||
|
{item.subtitle && (
|
||||||
|
<small style={{ ...subtitleStyle, color: isChecked ? 'rgba(255,255,255,0.85)' : 'inherit' }}>
|
||||||
|
{item.subtitle}
|
||||||
|
</small>
|
||||||
|
)}
|
||||||
|
</label>
|
||||||
|
);
|
||||||
|
})}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
))}
|
||||||
|
<div style={cardStyle}>
|
||||||
|
<div style={titleStyle}>Run this Command:</div>
|
||||||
|
<pre style={commandDisplayStyle}>{generateCommand()}</pre>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
};
|
||||||
Reference in New Issue
Block a user