Commit Graph

Select branches

Hide Pull Requests

bot/update-est-time-20260928

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• 37367da639 [fix CI] Fix logical condition in fused MoE layer for compressed tensor quantization (#10299) Xiaoyu Zhang 2025-09-11 14:54:09 +08:00
• ef959d7b85 [CPU] fix OOM when mem-fraction is not set (#9090) Zaili Wang 2025-09-11 14:52:22 +08:00
• 4aa1e69bc7 [chore]Add sgl-router to npu images (#10229) BourneSun0527 2025-09-11 14:51:16 +08:00
• dc491b399d add flash linear attention triton kernel (#10239) Yi Zhang 2025-09-11 12:47:20 +08:00
• 5b64f006ec [Feature] Support DeepEP normal & Redundant Experts on NPU (#9881) Even Zhou 2025-09-11 11:35:26 +08:00
• 5b7448de77 chore: bump sgl-kernel 0.3.9.post1 (#10294) Yineng Zhang 2025-09-10 18:26:34 -07:00
• 6d55f60e77 Revert "[1/2] Optimizations and refactors about quant kernel (#9534)" (#10292) Yineng Zhang 2025-09-10 18:24:23 -07:00
• 033b75f559 [Auto Sync] Update serving_base.py, serving_chat.py, servin... (20250910) (#10282) 3 people 2025-09-10 16:58:59 -07:00
• f3b5db6ee8 Feat: support disable tool parser (#10184) Xinyuan Tong 2025-09-10 14:03:55 -07:00
• 2286e85e77 pass a_scale from fp8 quant result instead of hard code to 1.0f (#10241) 3 people 2025-09-10 12:56:05 -07:00
• 91b3555d2d Add tests to AMD CI for MI35x (#9662) Hubert LuandSai Enduri 2025-09-10 12:50:05 -07:00
• 9e2f7252db add dual stream for qwen2_moe (#10252) Yi Zhang 2025-09-11 03:49:43 +08:00
• 21176b0093 [Bugfix] Fix Weightloading for the original nvidia/Deepseek-R1-FP4 checkpoint (#9940) 3 people 2025-09-10 12:00:23 -07:00
• 941002945b [1/2] Refactor LoRA to support backend-specific batch preprocessing. (#10251) Lifu Huang 2025-09-10 09:58:37 -07:00
• cda7e47ce7 [router] Add PD router mmlu test (#10256) Keyang Ru 2025-09-10 08:47:24 -07:00
• e903f695c8 Fix potential flakiness in test_lora_qwen3 (#10250) Lifu Huang 2025-09-10 01:04:39 -07:00
• 27760fc1b6 [Auto Sync] Update io_struct.py (20250910) (#10262) 3 people 2025-09-10 00:16:37 -07:00
• 0ac809de33 Fix assertion typo in tp_worker.py (#9954) Seunggeun Cho 2025-09-10 14:43:50 +09:00
• 4efe2c57c9 support vlm model spec bench (#10173) Lzhang-hub 2025-09-10 13:37:04 +08:00
• 5be8c2f7f7 Page first direct IO kernel (#10060) huangtingweiandZhiqiang Xie 2025-09-10 13:35:34 +08:00
• 737d73ed5b Fix: the default choice is wrong for flashinfer mxfp4 moe precision (#10253) Yiyu Liu 2025-09-10 00:10:38 -04:00
• ebd0e1c18b [doc] add walkthrough for implementing and hosting a simple llama wrapper m… (#10093) Glen Liu 2025-09-10 00:05:06 -04:00
• a1d038924b [Benchmark] Prefil-only benchmark scripts (#10240) Sundara Raman Ramachandran 2025-09-09 19:59:07 -07:00
• dccf52f9c8 [UT for RL] Add UT to cover release/resume memory case for moe model (#8803) ryang 2025-09-10 10:25:12 +08:00
• 676a7b51bd make --speculative-draft-model an alias of --speculative-draft-model-path (#10246) Lianmin Zheng 2025-09-09 19:12:24 -07:00
• 15f993472c refactor(InternVL): Use gpu to preprocess the input image (#9795) Kevin Tuan 2025-09-10 10:09:04 +08:00
• bcf1955f7e Revert "chore: upgrade v0.3.9 sgl-kernel" (#10245) Lianmin Zheng 2025-09-09 19:05:20 -07:00
• a06bf66425 [Auto Sync] Update collector.py, startup_func_log_and_timer... (20250910) (#10242) 3 people 2025-09-09 18:05:16 -07:00
• bf72b80122 [Auto Sync] Update io_struct.py (20250909) (#10236) 3 people 2025-09-09 14:15:21 -07:00
• 8cbe1538ef Add mamba kernel (#10234) Yi Zhang 2025-09-10 03:58:43 +08:00
• 8471e5e616 [HiCache] feat: add mooncake backend extra config (#10213) Teng Ma 2025-09-10 03:50:00 +08:00
• 4582931ac3 Revert "Revert the changes on NCCL symmetric memory" (#10238) Lianmin Zheng 2025-09-09 12:11:49 -07:00
• d352c29aa0 Revert the changes on NCCL symmetric memory (#10210) Lianmin ZhengandYineng Zhang 2025-09-09 11:01:33 -07:00
• d3ee70985f chore: upgrade v0.3.9 sgl-kernel (#10220) Yineng Zhang 2025-09-09 03:16:25 -07:00
• 71fc7b7fad [Fix] KV-cache eviction mismatch across PP ranks in DeepSeek V3/R1 (#10214) Rain H 2025-09-09 17:07:13 +08:00
• 9ab72f9895 add variable TP Decode > Prefill size support (#9960) shaharmor98 2025-09-09 11:47:26 +03:00
• f3817cb0b2 chore: bump v0.3.9 sgl-kernel (#10208) Yineng Zhang 2025-09-09 01:40:05 -07:00
• 71133a0426 [Auto Sync] Update sampling_batch_info.py (20250909) (#10212) 3 people 2025-09-09 01:29:52 -07:00
• 2cd94dd07e tool-call(dsv3): Fixed a parse problem when there are multiple function definitions in tool_calls (#10209) Yiming 2025-09-09 15:47:28 +08:00
• f5f6b3b4b5 Refactor fused_add_rmsnorm import logic (#10207) Shangming Cai 2025-09-09 15:23:58 +08:00
• 94fb4e9e54 feat: support fa cute in sgl-kernel (#10205) Yineng Zhangandcicirori 2025-09-09 00:14:39 -07:00
• d1d4074c4e [CPU] Add gelu_and_mul kernel in sgl-kernel and add ut (#9300) blzheng 2025-09-09 14:23:13 +08:00
• 718f25ae6e Explicitly export CMAKE_BUILD_PARALLEL_LEVEL (#10193) Keyang Ru 2025-09-08 22:35:27 -07:00
• 948b01a04c [Refactor] Remove Hicache Load & Write threads (#10127) DarkSharpnessandZhiqiang Xie 2025-09-08 22:18:50 -07:00
• cdc56ef6c1 feat: use sgl-kernel cu129 as default (#10188) Yineng Zhang 2025-09-08 22:01:17 -07:00
• 16ff3d4b05 Support opt model (#10165) wenhuipeng 2025-09-09 12:45:00 +08:00
• 83d55ac51f [1/N]DP refactor: Improve dp rank scheduling in PD disaggregation mode. (#10169) Liangsheng Yin 2025-09-09 12:27:55 +08:00
• 2fe17735a6 Updated Nvidia Jetson docs (#4422) Shakhizat Nurgaliyev 2025-09-09 08:41:21 +05:00
• 97fff98c68 [CPU] Fix phi4-mm prompt issue in bench_serving (#9900) blzheng 2025-09-09 11:12:32 +08:00
• ba066ca02f Update link for EAGLE speculative decoding (#10191) geray 2025-09-09 11:09:50 +08:00
• 96784a65fd [Fix] Orphan process in data parallel (#7995) Caproni 2025-09-09 11:09:09 +08:00
• df5407fb53 Revert "feat: add fused moe config for Qwen3-30B-A3B on B200" (#10185) Rain Jiang 2025-09-08 18:11:15 -07:00
• 8ad700f735 Cleaning codes for speculative attention mode (#10149) Baizhou Zhang 2025-09-08 17:38:06 -07:00
• 148022fc36 gb200: update dockerfile to latest kernel (#9522) ishandhanani 2025-09-08 17:32:36 -07:00
• 7a40e4f4a6 fix the cutlass moe tests (#10182) Rain Jiang 2025-09-08 16:24:55 -07:00
• 19d64f2b72 fix: resolve lint issue (#10181) Yineng Zhang 2025-09-08 15:09:55 -07:00
• a02071a12c [Bench] feat: mooncake trace integration (#9839) Teng MaandXuchun Shang 2025-09-09 02:50:54 +08:00
• 45b3a6a256 Revert "[ModelOpt] Fix Weight Loading for DSR1-FP4 Quantization (#9712)" (#10176) Yineng Zhang 2025-09-08 11:28:15 -07:00
• 9a18aa54c2 [fix] Relax white space rules in EBNFComposer (#9595) LukasBluebaum 2025-09-08 19:47:19 +02:00
• 91f0fd95a4 pref: Add H20 fp8 fused MoE kernel configs for Qwen3 (#10166) Zhiy-Zhangandqiufan.zzy 2025-09-09 00:57:21 +08:00
• 8085aca791 [Bug fix] Fix ascend mla in aclgraph (#9925) alanhe151220037 2025-09-09 00:49:43 +08:00
• 0096798ed6 [1/2] Speed up prefill mla attention (#10156) fzyzcjy 2025-09-09 00:00:33 +08:00
• 2c2b19b18b [CI] fix ambiguous argument in testing hybrid attentions. (#10161) Liangsheng Yin 2025-09-08 18:16:52 +08:00
• 72f9fc5f11 Monkey patch uvicorn multi worker is_alive timeout (#10159) Liangsheng YinandHuang Long 2025-09-08 17:43:23 +08:00
• ec99668ab7 [Hicache]: Add E2E CI For 3FS-KVStore (#10131) hzh0425 2025-09-08 16:54:50 +08:00
• 78f139812a [1/N] DP-Refactor: move communicators into tokenizer_communicator_mixin (#10028) Liangsheng Yin 2025-09-08 16:27:37 +08:00
• bfd7a18d8d update xgrammar 0.1.24 and transformers 4.56.1 (#10155) Swipe4057 2025-09-08 11:20:31 +03:00
• 5dd8c6444b [Bug fix] Fix Gemma 2 and fix Gemma 3 multimodal with bs > 1 on NPU (#9871) ssshinigamiandMaksim 2025-09-08 11:19:40 +03:00
• ee21817c6b enable llama3.1-8B on xpu (#9434) Huaiyu, Zheng 2025-09-08 13:34:20 +08:00
• b7d1f17b8d Revert "enable auto-round quantization model (#6226)" (#10148) Yineng Zhang 2025-09-07 22:31:11 -07:00
• c8295d2353 enable auto-round quantization model (#6226) Weiwei 2025-09-08 13:05:35 +08:00
• b67c277f86 [Bugfix] Qwen3MoE aclrtMemcpy failed with NPUGraph (#10013) Even Zhou 2025-09-08 12:50:49 +08:00
• 8116804e4f Fix: (glm4v) Add missing field (#10147) Xinyuan Tong 2025-09-08 04:47:14 +00:00
• 8c5930f08a Add speculator attention backend switch (#9981) cicirori 2025-09-08 06:44:36 +02:00
• 3b99f23c44 [Bugfix] Retract not releasing enough memory when page size > 1 (#9989) Zhiqiang Xie 2025-09-07 21:41:50 -07:00
• ee0b3c5bad [1/N][Bug] Fix w4afp8 MoE NaN issue (sgl-kernel, fixed) (#10108) Yuhao Yao 2025-09-08 12:39:07 +08:00
• 6049ca209e move compile threads to an option to avoid OOM on low memory host (#10123) Rain Jiang 2025-09-07 21:36:14 -07:00
• 7577f0e40f Add graph runner support with torch compile on CPU (#7843) Cao E 2025-09-08 12:33:58 +08:00
• 8cda5a622c Standalone speculative decoding (#10090) Qiaolin Yu 2025-09-07 20:55:09 -07:00
• 400d3b97ae Fix run time error in dsv3-fp8 model on mi35x (#10104) 4 people 2025-09-08 11:45:17 +08:00
• 37d83c6e6d Qwen2.5-VL eagle3 infer (#8801) Lzhang-hub 2025-09-08 11:44:34 +08:00
• 7802586cab fix the fp8 topk_config.correction_bias is none bug (#10040) Rain Jiang 2025-09-07 20:28:14 -07:00
• bc5fc332f7 Fix slow fused add RMSNorm (#10141) fzyzcjy 2025-09-08 11:20:39 +08:00
• f3440adcb5 vlm: enable GLM4.1V server testing & fix video processing (#10095) Xinyuan TongandBinyao Jiang 2025-09-08 02:53:08 +00:00
• 5a7e10fe4c [MoE] fix: incorrect weight initialization for cutlass_fused_experts_fp8 (#10144) Cheng Wan 2025-09-07 19:43:59 -07:00
• 33467c05a4 [BUG FIX] add fail check when get fail in case wait complete block (#9971) 3 people 2025-09-08 09:34:04 +08:00
• b0fcbb74d0 [DOC]: some minor updates (#10134) eigen 2025-09-07 17:58:15 -04:00
• 76a2c86b88 Fix flashinfer version in sgl-kernel (#10135) Lianmin Zheng 2025-09-07 12:54:07 -07:00
• e719bb0e84 [1/2] Refactor multi-tokenizer manager (#10074) Liangsheng Yin 2025-09-07 19:13:34 +08:00
• 067246830d [Minor] fix lint in main (#10128) DarkSharpness 2025-09-07 02:36:46 -07:00
• 617aa2b248 [Auto Sync] Update parallel_state.py (20250907) (#10126) 3 people 2025-09-07 02:12:32 -07:00
• 111b137964 add dataset_path for bench_one_batch_server.py (#10113) miterandlinhuang 2025-09-07 14:07:09 +08:00
• 41628dc1b1 [HiCache] fix: check clear() method for storage backend (#10096) Teng Maandhzh0425 2025-09-07 13:59:58 +08:00
• a12061df4c Fix cuda graph mode in flashinfer attn backend (#10056) Ben Barsdell 2025-09-07 15:59:48 +10:00
• 85ed8e0a5e Optimize nvfp4 block scaled gemm kernel when M is small. (#10101) Qi Yuhang 2025-09-07 13:31:00 +08:00
• dd1e268938 CUTLASS fp8 blockwise gemm support of sm120 (#9969) Jianying 2025-09-07 13:28:54 +08:00
• 9a7ced4e4d [Feature] LMCache Connector Integration (#9741) Yuwei AnandZhiqiang Xie 2025-09-06 20:14:55 -07:00
• cb3918a091 Optimize moe_sum_reduce_kernel (#9477) 3 people 2025-09-07 09:16:18 +08:00
• f3b6760213 [Auto Sync] Update server_args.py (20250906) (#10117) 3 people 2025-09-06 16:59:36 -07:00
• 9eb50ecc9c [router] Improve the router e2e tests (#10102) Keyang Ru 2025-09-06 16:19:28 -07:00