Commit Graph

Select branches

Hide Pull Requests

bot/update-est-time-20260928

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• 8305dc1718 [Diffusion] Disable packed QKV for FLUX & Z-Image (#16038) Xiaoyu Zhang 2025-12-29 14:33:05 +08:00
• ec8c831d5a [model-gateway]: optimize metrics for minimal CPU and memory overhead (#16041) Simo Lin 2025-12-28 22:28:08 -08:00
• f13949e5a9 [model-gateway] perf: optimize observability logging for minimal CPU/memory overhead (#16039) Simo Lin 2025-12-28 21:20:46 -08:00
• c236a3fdf0 [model-gateway][CI] Display benchmark results in GitHub Actions summary (#16037) Simo Lin 2025-12-28 20:57:27 -08:00
• 41a1d16b02 [model-gateway] Organize CLI arguments into logical groups for better --help output (#16035) Simo Linandgemini-code-assist[bot] 2025-12-28 20:25:53 -08:00
• 9884c9fd4c [model-gateway] Organize Rust CLI arguments into logical groups for better --help output (#16036) Simo Lin 2025-12-28 20:25:38 -08:00
• a435f55d18 Tiny print launch command with shlex (#16010) Liangsheng Yin 2025-12-29 11:26:46 +08:00
• 2ec6fa3c54 feat PD: add eagle3 support for DeepSeek V3 in EP mode (#14280) Mike QiuandMike_Qiu 2025-12-29 11:12:34 +08:00
• c58a573a40 [Docs] Improve documentation index page (#16028) Lianmin Zheng 2025-12-28 18:44:52 -08:00
• b840d6aaeb [diffusion] webui: reference to content task and better visualization capabilities (#16017) Li Jinliang 2025-12-29 10:22:21 +08:00
• ef4b3c0e96 Add host tensor allocator for memory_pool_host and support Mooncake standalone storage (#14873) EkiRuiandTeng Ma 2025-12-29 08:28:50 +08:00
• 6f9d0a89a0 [scheduler] fix: correcting extend_logprob_start_len calculation (#15922) Cheng Wan 2025-12-28 14:57:04 -08:00
• d7a3336ebe [diffusion] fix: fix stages not logged when perf_dump_path is provided (#16016) Mick 2025-12-28 23:17:43 +08:00
• 7d02c8e59f [JIT kernel] CI support jit kernel tests (#15939) Xiaoyu Zhang 2025-12-28 23:11:02 +08:00
• e6d5a213ad Fix metrics (#15998) Lianmin Zheng 2025-12-28 05:03:49 -08:00
• 9f8e23071a [diffusion] chore: fix default offload setting for image generation model (#15928) Mick 2025-12-28 20:45:33 +08:00
• d90f9bfc4e Temporarily disable temp_prefill_info assertion to unblock CI (#16008) fzyzcjy 2025-12-28 20:13:06 +08:00
• 3881bc8d0b [diffusion] CI: relax threshold by supporting different profiles (#16002) Mick 2025-12-28 20:05:07 +08:00
• 208e6a9dac [Doc]Update MTP moe backends for EP document (#16013) Baizhou Zhang 2025-12-28 19:36:23 +08:00
• be3828a13b Tiny cleanup duplicate code for multi-layer eagle worker. (#16004) Liangsheng Yin 2025-12-28 18:08:20 +08:00
• 5969be2f06 Apply fixture-kit mode to MMMUVLMMixin (#15615) lif 2025-12-28 17:22:05 +08:00
• 8fab48952e Tiny fix WASM test errors on machines with many cores (#15992) fzyzcjy 2025-12-28 17:12:04 +08:00
• 7ccaec6467 Add micro benchmarks for manual policy (#15991) fzyzcjy 2025-12-28 17:11:56 +08:00
• c457aad54a Update test parameters for deepep_large test (#16001) Cheng Wan 2025-12-28 00:58:19 -08:00
• c7e7bfa32c Add EAGLE3 test with MMLU dataset. (#15945) Liangsheng Yin 2025-12-28 16:07:58 +08:00
• bf90ea9c5b Unify spec v2's naming manner. (#15990) Liangsheng Yin 2025-12-28 14:14:52 +08:00
• 26c5091217 Tiny extract PeriodicTask in router (#15988) fzyzcjy 2025-12-28 14:07:13 +08:00
• 325a4c1945 Tiny add smg_manual_policy_cache_entries metric (#15987) fzyzcjy 2025-12-28 14:07:07 +08:00
• 0294844f04 [fix]deepgemm precompile when warmup (#15891) TZHelloWorld 2025-12-28 13:47:39 +08:00
• 0e536600e8 Refactor: separate CI-specific weight validation into dedicated module (#15216) Alison ShaoandKangyan-Zhou 2025-12-27 20:50:39 -08:00
• d70c265533 SGLang Tracing: fix attribute errors (header extraction & bootstrap span closing) (#15693) Vladislav Nosivskoyandishandhanani 2025-12-28 07:44:31 +03:00
• 656f4d69a1 Refactor fp8 nextn layer for DeepSeek nvfp4 checkpoint (#15353) Baizhou Zhang 2025-12-28 11:57:09 +08:00
• 8e43980ebb [Feature] JIT Fused QK norm + qk norm clean up (#15835) DarkSharpness 2025-12-28 11:53:50 +08:00
• 474a4699c5 Add correctness validation for decode_attention test (#15806) Hudson Xing 2025-12-28 11:46:17 +08:00
• b4a00ed2d9 [diffusion] chore: clean ComposedPipelineBase (#15937) Mick 2025-12-28 11:43:25 +08:00
• f55d608c89 Tiny fix cannot launch nvfp4 checkpoint with bf16 kv cache (#15986) fzyzcjy 2025-12-28 11:22:20 +08:00
• 349ce2dd19 Support kv8 (FP8) with torch_native attention backend (#12596) Ho-Ren (Jack) Chuang 2025-12-27 18:48:30 -08:00
• 183b65190a Clean up logging (#15919) Lianmin Zheng 2025-12-27 15:27:12 -08:00
• 2af955e16c [model-gateway]: remove unnecessary comment (#15947) Ratish P 2025-12-27 22:33:02 +04:00
• 0cd2b719a5 [diffusion] chore: remove useless params (#15925) Yuhao Yang 2025-12-28 01:01:08 +08:00
• 39d56196a0 [diffusion] logging: log available gpu mem while loading and generating (#15936) Mick 2025-12-28 00:34:58 +08:00
• 41addd2e08 chore: bump mooncake version to 0.3.8 (#15886) Shangming Cai 2025-12-27 23:34:24 +08:00
• 5c393e8153 Fix temp_prefill_info assertion error in PP disaggregation mode (#15943) Hudson Xing 2025-12-27 22:46:55 +08:00
• 3645ed0f73 [model-gateway] Add PrefixHash load balancing policy for KV cache-aware routing (#15935) Simo Lin 2025-12-27 05:58:05 -05:00
• ca740a41f3 [model-gateway]: fix grpc embedding test (#15934) Ratish P 2025-12-27 14:56:54 +04:00
• 0e25aa439b [model-gateway] optimize radix tree memory and reduce allocations (#15933) Simo Lin 2025-12-27 05:47:30 -05:00
• 60a230b1fd [NPU] Support w4a8 with activation clip (#14736) jiaming1130andZhengdQin 2025-12-27 16:19:46 +08:00
• aa89c6a7e2 [diffusion] refactor: unify model loading and offloading behavior (#15923) Mick 2025-12-27 16:18:24 +08:00
• 171912a9e3 [model-gateway] Add consistent hashing for ManualPolicy routing (#15907) Simo Lin 2025-12-27 01:53:03 -05:00
• faecd37ed4 Add Mimo-v2-flash model to ci test (#15887) Ke Bao 2025-12-27 14:18:08 +08:00
• 9ad546d7e8 Tiny cleanup the models' name in test_utils (#15920) Liangsheng Yin 2025-12-27 14:13:23 +08:00
• 29ce7b3612 [diffusion] chore: remove stepvideo code (#15918) Yuhao Yang 2025-12-27 13:25:05 +08:00
• a8380ded71 Add a test case for crash dump (#15905) Lianmin Zheng 2025-12-26 19:39:28 -08:00
• 4edee6954a [model-gateway] add JWT/OIDC authentication for control plane APIs (#15850) Simo Lin 2025-12-26 21:30:00 -05:00
• 67caea6fe4 fix: adding deepseek base tests to b200 (#15915) Douglas Yang 2025-12-26 18:04:25 -08:00
• cd3289c7a4 [BugFix][VLM] Correct weight loading with tie_word_embeddings == False (#15398) Zheng Wengang 2025-12-27 09:30:50 +08:00
• acddb8e0ec Revert embedding integration tests from 5f3a47d8a (#15914) Simo Lin 2025-12-26 20:29:03 -05:00
• 2ec57cefd9 hotfix: add type hints to scheduler mixins (#15916) Cheng Wan 2025-12-26 17:08:11 -08:00
• 988b14ca0e refactor: add type hints to scheduler mixins (#15913) Cheng Wan 2025-12-26 16:50:07 -08:00
• 93495dcac9 Revert "[VLM] Refactor load_mm_data to improve performance" (#15911) Lianmin Zheng 2025-12-26 13:43:18 -08:00
• 886e038329 [model-gateway]: fix crash in embedding worker health check (#15910) Ratish P 2025-12-27 00:57:13 +04:00
• 01bd0d3e8b [Tool Call][DSV32] Streamline function call parameters (#14750) Muqi Li 2025-12-27 03:35:30 +08:00
• 43e1bbc0d5 Revert "[feat] Init support for webui-I2I" (#15906) Lianmin Zheng 2025-12-26 09:53:03 -08:00
• 59b12996fb [diffusion] apps: support I2I tasks in webui (#15778) Li Jinliang 2025-12-26 23:45:11 +08:00
• b70914969b [diffusion] CI: support returning request id from endpoint (#15844) Mickandgemini-code-assist[bot] 2025-12-26 23:38:03 +08:00
• 51dbdb2202 [diffusion] improve: improve qwen-image-edit performance to align with LightX2V (#15812) Xiaoyu ZhangandMick 2025-12-26 22:25:10 +08:00
• 8dc6f0fc4d [diffusion] refactor: centralize hardware platform detection and streamline environment variable management (#15842) Mick 2025-12-26 22:16:18 +08:00
• cf34d0ab32 [Fix] assert error in log_prefill_stats (#15881) Huaixin ChangandJunjie Mao 2025-12-26 19:54:35 +08:00
• 3778c2fc6d [diffusion] CI: fix CI test case skip problem (#15874) Prozac614andMick 2025-12-26 19:42:20 +08:00
• 5d421db883 Tiny log warn users when tracing is automatically disabled (#15889) fzyzcjy 2025-12-26 19:00:37 +08:00
• fe3d47fc9d fix: warn once per env var key (#15846) shuwenn 2025-12-26 18:41:15 +08:00
• ef92b4eb88 [BUGFIX] fix edge case for qwen3-next (#14209) Yi Zhang 2025-12-26 17:14:37 +08:00
• 73c0c66ff8 [diffusion] ZImage support Tensor Parallel (#15849) Annis 2025-12-26 14:58:14 +08:00
• a1e9b4edfa diffusion: support Qwen-Image-Layered (#15817) chhnbandyhyang201 2025-12-26 14:57:01 +08:00
• e75657c839 [Bug] fix piggyback load report return None bug (#15870) Liangsheng Yin 2025-12-26 14:48:45 +08:00
• c28c536c91 Fix swa available memory check (#15867) Ke Bao 2025-12-26 13:19:01 +08:00
• 086813ae8a [VLM] refactor: refactor load_mm_data to improve performance (#14644) Yuan Luoandluoyuan.luo 2025-12-26 13:17:53 +08:00
• 3fd232ad96 [NPU] update Mixed chunk op to FIA (#15518) Hexq0210 2025-12-26 12:17:40 +08:00
• cb1812954a Introduce ModelRunnerKVCacheMixin to simplify the code. (#15821) Liangsheng Yin 2025-12-26 11:17:33 +08:00
• a91e072f33 Use allow auto truncate in the OpenAI API endpoint (#15369) Kangyan-Zhou 2025-12-25 18:45:57 -08:00
• 0271fc3456 Fix prefill num tokens metrics (#15858) fzyzcjy 2025-12-26 10:26:42 +08:00
• 68bece8caa Super tiny move last_prefill_tokens to metrics mixin (#15857) fzyzcjy 2025-12-26 10:26:10 +08:00
• 7b7e357f61 Separate swa and local attention chunk cache eviction (#15820) Ke Bao 2025-12-26 09:34:22 +08:00
• 2f66b0671b Fix chunk_kda_fwd missing argument (#15851) Ke Bao 2025-12-26 09:32:35 +08:00
• 4805118122 ci: add continue-on-error for scheduled PR tests (#15701) Alison Shao 2025-12-25 13:26:14 -08:00
• f2ccc44240 fix: improving format and design (#15791) Douglas Yang 2025-12-25 13:13:27 -08:00
• caa95c7eb4 [model-gateway] update ManualPolicy with header-based routing (#15847) Simo Lin 2025-12-25 16:04:43 -05:00
• 9d878c1f3e Optimize FP8 MLA KV cache writes with Triton kernel (#15522) Hudson Xing 2025-12-25 12:35:39 -08:00
• 8087ef126f Clean up the __init__ of TokenizerManager and DetokenizerManager (#15796) Lianmin Zheng 2025-12-25 10:30:27 -08:00
• 6ef543f99d Use X-SMG-Routing-Key header instead of json body and add tests (#15826) fzyzcjy 2025-12-26 01:58:14 +08:00
• a355911909 [diffusion] improve: improve post-processing by moving compute-intensive tasks to GPU (#15822) Mick 2025-12-26 01:29:04 +08:00
• f3ba711662 fix: change class name of GLM-ASR (#15772) Yuxuan Zhang 2025-12-26 00:15:16 +08:00
• 5c243ba588 Custom All Reduce for Piecewise Cuda Graph (#15356) Yuwei An 2025-12-25 07:55:15 -08:00
• b6702d72cf feat: log request when e2e latency exceeds the specified value (#15759) 3 people 2025-12-25 23:12:35 +08:00
• c12567274a Tiny change bench-serving to use routing key header (#15827) fzyzcjy 2025-12-25 21:29:49 +08:00
• bb9e6cdf9e [MiMoV2Flash] fix: respect --swa-full-tokens-ratio arg (#15488) Yingchun Lai 2025-12-25 21:02:56 +08:00
• de03b0cd30 [Nemotron 3 Nano] Add triton MoE configs (#15815) roikoren755 2025-12-25 13:31:20 +02:00
• 2a8a785634 [diffusion] log: avoid logging in hot path if unnecessary (#15818) Mick 2025-12-25 18:42:52 +08:00
• f4e835af2f Cleanup ModelRunner (#15802) Liangsheng Yin 2025-12-25 18:13:30 +08:00
• e6ce16a4c2 [diffusion] feat: support TP for Flux.1.dev (#15666) Xiaoyu ZhangandMick 2025-12-25 18:00:44 +08:00