minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[Docs] Update Nemotron3-Nano-Omni cookbook to reflect new model paths (#25198)

Browse Source
This commit is contained in:
Yihao Wang
2026-06-03 14:55:30 -07:00
committed by GitHub
parent 61aa3293d3
commit 8980eb82de
2 changed files with 25 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/cookbook/autoregressive/NVIDIA/Nemotron3-Nano-Omni.mdx
+20 -21
View File
@@ -30,10 +30,9 @@ Architecture and key features:
**Supported GPUs:** NVIDIA B200, H100, H200, A100, L40S, DGX Spark, RTX 6000
Available model variants on HuggingFace:
- [`nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning`](https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning)
- [`nvidia/Nemotron-3-Nano-Omni-30B-A3B-BF16`](https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-BF16)
- [`nvidia/Nemotron-3-Nano-Omni-30B-A3B-FP8`](https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-FP8)
- [`nvidia/Nemotron-3-Nano-Omni-30B-A3B-NVFP4`](https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-NVFP4)
- [`nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16`](https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16)
- [`nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-FP8`](https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-FP8)
- [`nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-NVFP4`](https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-NVFP4)
**Agentic workloads this model enables:**
- **Computer Use Agent:** Perception loop for agents navigating GUIs — reads screens, understands UI state over time, validates outcomes. Collapses vision and reasoning into a single loop.
@@ -76,7 +75,7 @@ This section provides a progressive guide from quick deployment to performance t
- **TP support:**
To set tensor parallelism, use `--tp <1|2|4|8>`. A 4×H100 setup is recommended for the BF16/Reasoning variant.
To set tensor parallelism, use `--tp <1|2|4|8>`. A 4×H100 setup is recommended for the BF16 variant.
- **FP8 KV cache:**
@@ -96,7 +95,7 @@ The command below launches the server for a 4×H100 setup with reasoning and too
```shell Command
sglang serve \
--model-path nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning \
--model-path nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 \
--host 0.0.0.0 \
--port 30000 \
--tp 4 \
@@ -112,7 +111,7 @@ SGLang provides an OpenAI-compatible endpoint. Example with the OpenAI Python cl
```python Example
from openai import OpenAI
SERVED_MODEL_NAME = "nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning"
SERVED_MODEL_NAME = "nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16"
client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
resp = client.chat.completions.create(
@@ -142,7 +141,7 @@ Streaming chat completion:
```python Example
from openai import OpenAI
SERVED_MODEL_NAME = "nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning"
SERVED_MODEL_NAME = "nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16"
client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
stream = client.chat.completions.create(
@@ -168,7 +167,7 @@ Pass image inputs using the OpenAI vision format. Supports both URLs and base64-
```python Example
from openai import OpenAI
SERVED_MODEL_NAME = "nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning"
SERVED_MODEL_NAME = "nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16"
client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
# From URL
@@ -199,7 +198,7 @@ For local images, encode as base64:
import base64
from openai import OpenAI
SERVED_MODEL_NAME = "nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning"
SERVED_MODEL_NAME = "nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16"
client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
with open("screenshot.png", "rb") as f:
@@ -233,7 +232,7 @@ Nemotron 3 Nano Omni uses Conv3D layers and Efficient Video Sampling (EVS) for t
import base64
from openai import OpenAI
SERVED_MODEL_NAME = "nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning"
SERVED_MODEL_NAME = "nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16"
client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
with open("video.mp4", "rb") as f:
@@ -268,7 +267,7 @@ Pass audio inputs as base64-encoded WAV or MP3 data:
import base64
from openai import OpenAI
SERVED_MODEL_NAME = "nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning"
SERVED_MODEL_NAME = "nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16"
client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
with open("audio.wav", "rb") as f:
@@ -302,7 +301,7 @@ Combine modalities in a single request. For example, an image alongside an audio
import base64
from openai import OpenAI
SERVED_MODEL_NAME = "nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning"
SERVED_MODEL_NAME = "nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16"
client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
with open("chart.png", "rb") as f:
@@ -336,7 +335,7 @@ The model supports two modes — Reasoning ON (default) vs OFF. Toggle per-reque
```python Example
from openai import OpenAI
SERVED_MODEL_NAME = "nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning"
SERVED_MODEL_NAME = "nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16"
client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
# Reasoning ON (default)
@@ -389,7 +388,7 @@ Call functions using the OpenAI Tools schema. The server must be launched with `
```python Example
from openai import OpenAI
SERVED_MODEL_NAME = "nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning"
SERVED_MODEL_NAME = "nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16"
client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
TOOLS = [
@@ -445,7 +444,7 @@ The user is asking about weather in Santa Clara, CA. I have a get_weather functi
```shell Command
sglang serve \
--model-path nvidia/Nemotron-3-Nano-Omni-30B-A3B-FP8 \
--model-path nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-FP8 \
--host 0.0.0.0 \
--port 30000 \
--tp 4 \
@@ -458,10 +457,10 @@ sglang serve \
```shell Command
sglang serve \
--model-path nvidia/Nemotron-3-Nano-Omni-30B-A3B-NVFP4 \
--model-path nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-NVFP4 \
--host 0.0.0.0 \
--port 30000 \
--tp 2 \
--tp 4 \
--trust-remote-code \
--tool-call-parser qwen3_coder \
--reasoning-parser deepseek-r1
@@ -487,7 +486,7 @@ Model Deployment Command:
```shell Command
sglang serve \
--model-path nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning \
--model-path nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 \
--trust-remote-code \
--tp 4 \
--max-running-requests 1024 \
@@ -503,7 +502,7 @@ python3 -m sglang.bench_serving \
--backend sglang \
--host 127.0.0.1 \
--port 30000 \
--model nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning \
--model nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 \
--dataset-name random \
--random-input-len 1024 \
--random-output-len 1024 \
@@ -564,7 +563,7 @@ Max ITL (ms): 2171.93
**Launch Model**
```shell Command
sglang serve \
--model-path nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning \
--model-path nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 \
--trust-remote-code \
--tp 4 \
--attention-backend flashinfer \
docs_new/src/snippets/autoregressive/nemotron3-nano-omni-deployment.jsx
+5 -7
View File
@@ -1,9 +1,8 @@
export const Nemotron3NanoOmniDeployment = () => {
const MODEL_PATHS = {
reasoning: 'nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning',
bf16: 'nvidia/Nemotron-3-Nano-Omni-30B-A3B-BF16',
fp8: 'nvidia/Nemotron-3-Nano-Omni-30B-A3B-FP8',
nvfp4: 'nvidia/Nemotron-3-Nano-Omni-30B-A3B-NVFP4',
bf16: 'nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16',
fp8: 'nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-FP8',
nvfp4: 'nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-NVFP4',
};
const options = {
@@ -11,8 +10,7 @@ export const Nemotron3NanoOmniDeployment = () => {
name: 'model',
title: 'Model',
items: [
{ id: 'reasoning', label: 'Reasoning', default: true },
{ id: 'bf16', label: 'BF16', default: false },
{ id: 'bf16', label: 'BF16', default: true },
{ id: 'fp8', label: 'FP8', default: false },
{ id: 'nvfp4', label: 'NVFP4', default: false },
],
@@ -77,7 +75,7 @@ export const Nemotron3NanoOmniDeployment = () => {
return '# TP=1 is not supported on L40S for this model. Please use TP=2 or higher.';
}
const modelPath = MODEL_PATHS[model] || MODEL_PATHS.reasoning;
const modelPath = MODEL_PATHS[model] || MODEL_PATHS.bf16;
let cmd = 'sglang serve \\\n';
cmd += ` --model-path ${modelPath} \\\n`;