Liangsheng Yin
f35ec18d8b
[Test] Remove more unit tests that mirror implementation or never run in CI ( #41297 )
2026-09-25 19:22:14 -07:00
Liangsheng Yin
1f6ce4b068
[Test] Remove unit tests that only mirror implementation or never run in CI ( #41286 )
2026-09-25 16:15:02 -07:00
Jackey Hua and Claude Fable 5.1
5bb24e399d
[Quant] ModelOpt mixed precision: dispatch block-FP8 MoE experts and derive the block size ( #38726 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-24 12:30:01 -04:00
Cheng Wan
acac4dd9d9
[Refactor] Clean up parallel runtime comments ( #40632 )
2026-09-21 14:32:22 -07:00
Cheng Wan
73f071db52
Deprecate the parallel getters the context answers, and ratchet them shut ( #40342 )
2026-09-21 12:25:32 -07:00
Cheng Wan
2d0e94e3a3
Check the topology identities where the layout is written, and build at the published widths ( #40340 )
2026-09-21 12:22:59 -07:00
Cheng Wan
afe71f4b9e
Read process groups through the runtime context ( #40068 )
2026-09-18 17:40:32 -07:00
Yang Liu and yangliu991
276663a79d
[model-loader] Split weight loading from postprocessing ( #34981 )
...
Co-authored-by: yangliu991 <yangliu991@fb.com >
2026-09-14 17:10:19 -07:00
cebca698e2
[Qwen3.8] Enable NVIDIA NVFP4 on DGX Spark with file-backed PLE and PDL router fix ( #39126 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: rdxa <rdxa@rdxa-int-spark-01.yvb.moe >
Co-authored-by: Yangmin Li <yangminl@nvidia.com >
Co-authored-by: Manrique <nanomlm@gmail.com >
Co-authored-by: yhyang201 <yhyang201@gmail.com >
2026-09-13 16:23:41 +08:00
Aurick Qiao
c389dd0863
Fix RunAI object-storage checkpoint index filtering ( #38988 )
2026-09-12 12:25:57 +08:00
fae8cd84cb
Support NemotronH_Omni_Reasoning_V3 in SGLang ( #35599 )
...
Signed-off-by: Ryan Stewart <rystewart@nvidia.com >
Signed-off-by: rystewart-nvidia <rystewart@nvidia.com >
Co-authored-by: elvischenv <219235043+elvischenv@users.noreply.github.com >
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
2026-09-10 16:57:22 -07:00
sglang-bot and sglang-bot
6252993afe
chore: update CI test est_time values ( #38238 )
...
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com >
2026-09-06 17:49:41 -07:00
Po-Han Huang (NVIDIA)
991368d880
Fix Muse Glimmer ModelOpt mixed weight mapping ( #37510 )
2026-09-04 20:53:55 -07:00
92a4d8b5ee
Clean logging under --weight-loader-prefetch-checkpoints ( #33930 )
...
Co-authored-by: Brayden Zhong <brayden@radixark.ai >
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com >
2026-09-04 20:05:53 -07:00
55bf3380e0
Support Hy4-preview ( #36805 )
...
Co-authored-by: BBuf <1182563586@qq.com >
Co-authored-by: alphabetc1 <2508695655@qq.com >
2026-09-04 18:03:49 -07:00
4349538c02
[model] add cosmos3 reasoner to llm only inference ( #33572 )
...
Signed-off-by: joeltg <joel@reflection.ai >
Signed-off-by: Joe Rowell <joe@poolside.ai >
Co-authored-by: Dawid Majchrowski <dmajchrowski@nvidia.com >
Co-authored-by: Kedi Wu <kediw@nvidia.com >
Co-authored-by: Kedi Wu <31940276+kediwu0331@users.noreply.github.com >
Co-authored-by: Joel Gustafson <joelgustafson@protonmail.com >
2026-09-04 22:11:58 +08:00
Alex Nails and Alison Shao
28262c20df
[CI][RFC] Replace black-jupyter with ruff-format ( #37210 )
...
Co-authored-by: Alison Shao <a.shao@wustl.edu >
2026-09-02 19:46:08 -07:00
Mick
0dd66def7c
[chore] harden checkpoint quantization metadata parsing ( #36922 )
2026-09-03 09:27:22 +08:00
Ziang Li
9a85473a89
[FlashInfer v0.6.18] add FlashInfer CuTe DSL NVFP4 W4A16 mode ( #35120 )
2026-08-31 18:47:30 -07:00
6580d5cd9a
weight cache: key daemon paths by GPU UUID ( #36101 )
...
Co-authored-by: siyu <liusy58@linux.alibaba.com >
Co-authored-by: Alex Nails <alex.nails@radixark.ai >
2026-08-31 01:44:45 -07:00
Pavan Sivaram Girijala and YanbingJiang
4dc7dc8518
[Fix] Transformers-fallback (GPT-NeoX) + KV pool config (DeepSeek-VL2) ( #35244 )
...
Co-authored-by: YanbingJiang <yanbing.jiang@intel.com >
2026-08-31 11:05:50 +08:00
Joel Gustafson and mmangkad
0a585d5bb1
[BugFix] Allow model_loader_extra_config with remote_instance + modelexpress backend ( #34639 )
...
Signed-off-by: joeltg <joel@reflection.ai >
Co-authored-by: mmangkad <176301910+mmangkad@users.noreply.github.com >
2026-08-29 00:43:10 -07:00
Cheng Wan and Claude Opus 5
ca1d7ed8e6
config: a parallel leaf with no live counterpart is read bare ( #36620 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-08-27 12:56:11 -07:00
Schwinn Saereesitthipitak
08315c56df
[Fix] Account resident weight memory in KV sizing ( #34053 )
...
Signed-off-by: Schwinn Saereesitthipitak <schwinns@nvidia.com >
2026-08-27 02:43:04 -07:00
Xun Sun
adcf73d7f7
[Weight Cache] Enhance test and support EPLB ( #36198 )
2026-08-26 23:11:53 -07:00
Cheng Wan
8005df61d3
config: spell the parallel config tier at the call site ( #36250 )
2026-08-26 03:00:28 -07:00
Xun Sun and siyu
a90d770c40
[Weight Cache] Support static DP/EP layouts ( #33684 )
...
Co-authored-by: siyu <liusy58@linux.alibaba.com >
2026-08-23 19:57:52 -07:00
siyu and liusy58
cb10ca16dd
[FEAT] Weight Daemon abstraction ( #33279 )
...
Co-authored-by: liusy58 <liusy58@smail.nju.edu.cn >
2026-08-22 02:13:46 -07:00
YAMY
5f12839591
[Fix] Support Kimi-K3 ModelOpt mixed NVFP4/FP8 checkpoint ( #35077 )
2026-08-19 08:13:45 -07:00
Mick
ef490853bb
quant: extract shared checkpoint quant metadata resolver ( #35172 )
2026-08-19 08:26:41 +08:00
6b94d39f13
[Model Loading] Overlap checkpoint staging with CUDA graph capture during startup ( #32017 )
...
Co-authored-by: Wenhui Zhu <wzhu59@asu.edu >
Co-authored-by: Alex Nails <alex.nails@radixark.ai >
2026-08-13 12:26:25 -07:00
Mohammad Miadh Angkad
8c5d5f75bf
Fix DSV4 DSpark shared expert loading ( #33312 )
2026-08-11 07:36:50 +08:00
Cheng Wan
63833f8034
config: business code no longer reads the published ServerArgs ( #34081 )
2026-08-09 14:44:08 -07:00
Ziang Li and Brayden Zhong
4ad990ba7d
[ModelOpt FP4] Support online MoE weight quantization ( #33115 )
...
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca >
2026-08-06 11:01:55 -07:00
Mohammad Miadh Angkad
65d5a0ec25
Support ModelOpt MXFP8 checkpoints ( #32538 )
2026-08-05 17:44:32 -07:00
Cheng Wan
47d8b5b749
config: route parallel config-leaf reads through get_parallel() ( #33170 )
...
The parallel namespace joins the accessor migration: 106 config-leaf reads
(enable_dp_lm_head, enable_dp_attention, pp_async_batch_depth, dp_size,
ep_join_rank_offset, dwdp_size, ...) flip from get_server_args()/
self.server_args to get_parallel(), which serves config leaves from the
published parallel bag via __getattr__.
- ParallelContext.__getattr__ is restructured to stay dynamo-traceable
(object.__getattribute__ graph-breaks): gate helpers such as
enable_moe_dense_fully_dp() run inside compiled model forwards. A
fullgraph regression test pins the pattern.
- The five live-shadowed topology sizes (tp/pp/dcp/attn_cp/moe_dp_size)
keep their server_args reads: the live @property wins on the accessor,
and conditionally-initialized groups would fail loud at unconditional
call sites.
- Elastic-EP scale writers (ep_size/dp_size x4 in model_runner) reroute
to get_context().override together with their remaining instance
readers (expert_location gpus-per-node paths); the ServerArgs.override
ratchet drops 39 -> 35.
- The expert placement helpers (compute_logical_to_rank_dispatch_
physical_map, _compute_logical_to_all_physical_map,
_prefer_same_node_experts) now read everything from the bags and drop
their server_args parameter; their unit tests publish the config they
need instead of stubbing it.
2026-08-01 08:58:39 -07:00
Cheng Wan
55b6769b0e
config: read resolved config via namespace accessors ( #33013 )
2026-07-31 15:06:59 -07:00
Nan Jiang
89f4a80c1f
Support fastsafetensors no-GDS loading and page-cache release ( #31859 )
2026-07-31 23:12:32 +08:00
Danila Shtan and Danila Shtan
5f9b0db18c
Fix async loading of RunAI-streamed tensors ( #32896 )
...
Co-authored-by: Danila Shtan <dan@nebius.com >
2026-07-31 21:46:33 +08:00
Trevor Morris
a6221d776f
feat: Support nvidia/MiniMax-M3-NVFP4 ( #31989 )
2026-07-30 14:32:03 -07:00
Broduker
b61cb5f9de
Fix DeepSeek V4 loading with RunAI Model Streamer. ( #30240 )
2026-07-30 23:03:34 +08:00
Cheng Wan and Shu Wang
659d349b61
[core/loader] Add presharded load format ( #24256 )
...
Co-authored-by: Shu Wang <shuwanguc@google.com >
2026-07-25 13:03:39 -07:00
f9c14e6bd4
[FEAT] Support fast engine recovery through weight cache ( #27139 )
...
Signed-off-by: Michael Qiu <qiudayu.qdy@antgroup.com >
Co-authored-by: liusy58 <liusy58@linux.alibaba.com >
Co-authored-by: Alex Nails <alex.nails@radixark.ai >
2026-07-25 14:31:21 +08:00
Liangsheng Yin
f4f15162bc
[Fix] Fail fast when a safetensors index references missing shard files ( #32279 )
2026-07-24 02:18:56 -07:00
Fardin Hoque
b54adced46
[Quant] Support NVFP4_AWQ checkpoints in ModelOpt FP4 path ( #31825 )
2026-07-22 08:46:10 +08:00
Liangsheng Yin
a36c873147
[misc] Remove unit test cases that fail the admission criteria (round 2) ( #30703 )
2026-07-09 16:35:59 -07:00
Cheng Wan
c02b032da0
[fix] Repoint the prefetch-dispatch test at the loader's current config binding ( #30631 )
2026-07-09 02:21:04 -07:00
Mohammad Miadh Angkad
666a09fe2a
Disable multi-threaded load by default when prefetch is on ( #30146 )
2026-07-09 00:28:53 -07:00
Mohammad Miadh Angkad
b1942fc3ea
[Model] Support Qwen3.6 ModelOpt mixed NVFP4 ( #27906 )
2026-07-05 21:31:15 -07:00
Mohammad Miadh Angkad
6c018eb4d1
Fix bounded checkpoint prefetching and buffered drop-cache handling ( #29156 )
2026-06-29 21:49:16 +00:00