Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• 0ae1e9a755 refine fused_moe benchmark (#7221) Xiaoyu Zhang 2025-06-16 12:21:32 +08:00
• e07d064729 Support LoRA in MMMU benchmark script. (#7218) Lifu Huang 2025-06-15 21:17:57 -07:00
• 3c2274fbee Implement gather before attn (#6378) Cheng Wan 2025-06-15 21:08:56 -07:00
• d2679f5109 Fix ChunkCache object has no attribute 'disable' (#7217) Baizhou Zhang 2025-06-15 20:55:15 -07:00
• 96be97bfff Minor PD style fix (#7215) Byron Hsu 2025-06-15 16:12:12 -07:00
• 88f9c347b2 [PD] use int32 for kv indices & get num_reserved_decode_tokens from server_args (#7214) Byron Hsu 2025-06-15 11:51:03 -07:00
• fff10809bf Revert "[EAGLE] Refactor code for page size > 1 & more simplifications" (#7210) Lianmin Zheng 2025-06-15 02:48:00 -07:00
• 5f1ab32717 [EAGLE] Refactor code for page size > 1 & more simplifications (#7163) Lianmin Zheng 2025-06-14 23:16:23 -07:00
• 7df7c679b6 feat: use zstd for docker (#7205) Yineng Zhang 2025-06-14 23:13:29 -07:00
• 38af4f68a9 Fix grammar abort & Minor style fixes (#7204) Lianmin Zheng 2025-06-14 22:49:41 -07:00
• a6305c7d50 Lianmin/simplify memory pool (#7202) Lianmin Zheng 2025-06-14 22:25:37 -07:00
• a023856b12 Move host memory pools into a separate file (#7200) Lianmin Zheng 2025-06-14 21:31:42 -07:00
• db0cc57e75 [PD] Support decode retract and update decode.py (#7196) Byron Hsu 2025-06-14 19:48:05 -07:00
• 349bb2c92a Fix error when disabling new DeepGEMM (#7198) fzyzcjy 2025-06-15 10:24:54 +08:00
• 0b8939bc2c Fix NCCL 2.27.3 not in docker image (#7195) fzyzcjy 2025-06-15 09:36:11 +08:00
• ed89837cf4 chore: upgrade sgl-kernel v0.1.8.post2 (#7186) JieXin Liangandzhyncs 2025-06-15 09:26:18 +08:00
• 55561e2553 [fix] fix determine_num_fused_shared_experts (#7180) JieXin Liang 2025-06-15 08:41:22 +08:00
• 4473320380 chore: bump v0.1.8.post2 (#7189) Yineng Zhang 2025-06-14 17:01:48 -07:00
• 0bd67ba2bd Fix a minor bug related to DeepGEMM upgrade (#7191) Zhijian Liu 2025-06-14 16:54:40 -07:00
• 7d316991b2 [PD] Update prefill.py (#7190) Byron Hsu 2025-06-14 15:59:54 -07:00
• ab1a4fa5cb [fix] fix cutlass_mla_backend with cuda_graph and add sm_scale for sgl-kernel cutlass_mla (#7184) JieXin Liang 2025-06-15 03:45:41 +08:00
• ed54bf9d19 [fix] fix dsv3 weight loader tqdm and simplify shared experts fusion (#7181) JieXin Liang 2025-06-15 02:56:29 +08:00
• b57d87c297 Fix shared experts fusion + weight requant (#7177) fzyzcjy 2025-06-14 17:35:18 +08:00
• 98538822d5 Add Phi-4-mm to supported VLM supported model list. (#7178) Lifu Huang 2025-06-13 23:17:40 -07:00
• f47a1b1d0f Increase timeout in test/srt/test_disaggregation.py (#7175) Lianmin Zheng 2025-06-13 23:12:14 -07:00
• 93cec4335f Support new DeepGEMM (#7172) fzyzcjy 2025-06-14 14:00:17 +08:00
• ba589b88fc Improve test cases for eagle infer (#7173) Lianmin Zheng 2025-06-13 22:25:13 -07:00
• 50876abc47 Add test for refactored openai server (#7161) Jinn 2025-06-13 22:42:57 -05:00
• b4c41f7276 Refactor DeepGEMM integration (#7150) fzyzcjy 2025-06-14 11:41:03 +08:00
• 8b8f2e7463 Support new DeepGEMM input format in silu_and_mul_masked_post_quant_fwd (#7153) fzyzcjy 2025-06-14 11:40:24 +08:00
• 0fc3d992bb Split the eagle test into two files (#7170) Lianmin Zheng 2025-06-13 20:14:26 -07:00
• be2d985df8 Minor style change of triton backend (#7165) Lianmin Zheng 2025-06-13 16:01:23 -07:00
• 5b1afa7814 Re-quantize DeepSeek model weights to support DeepGEMM new input format (#7156) fzyzcjy 2025-06-14 06:57:45 +08:00
• c49c1d9226 Remove 200us slow concat kernel (part 2: srt) (#7020) fzyzcjy 2025-06-14 06:19:31 +08:00
• 0f1dfa1efe Tiny add sanity checks for DeepGEMM inputs (#7157) fzyzcjy 2025-06-14 05:36:27 +08:00
• e3ec6bf4b6 Minor speed up block_quant_dequant (#6814) fzyzcjy 2025-06-14 05:32:46 +08:00
• b04df75acd Fix DeepEP error in some environments (#7154) fzyzcjy 2025-06-14 05:27:25 +08:00
• bec3e48402 Support new DeepGEMM format in per token group quant (part 2: srt) (#7155) fzyzcjy 2025-06-14 05:25:40 +08:00
• 8ab7d93c2e chore: bump v0.1.8.post1 (#7152) Yineng Zhang 2025-06-13 03:14:26 -07:00
• 5c66c4424f Support new DeepGEMM format in per token group quant (#7146) fzyzcjy 2025-06-13 17:00:22 +08:00
• aa46ed34d2 Remove 200us slow concat kernel (part 1: kernel) (#7145) fzyzcjy 2025-06-13 16:58:29 +08:00
• 2f4ec752bc filter by num_hidden_layers (#7056) pansicheng 2025-06-13 15:53:09 +08:00
• da47621ccc Minor speedup topk postprocessing (#7058) fzyzcjy 2025-06-13 15:50:18 +08:00
• 22a6b9fc05 Remove unnecessary metadata_expand.max_seq_len_k operations in fa3 to… (#7140) Binyao Jiang 2025-06-12 23:25:52 -07:00
• b02df20a8d Minor add utility to read expert distribution recorder output (#7134) fzyzcjy 2025-06-13 07:50:05 +08:00
• bd7cfbd2f8 [Fix] Reduce busy polling when scheduler is idle (#6026) Povilas Kanapickas 2025-06-13 00:58:22 +03:00
• 4b9971e401 Add gfx950 support for sgl-kernel. (#7092) 3 people 2025-06-13 02:07:48 +08:00
• dcc79d325b Update CODEOWNERS (#7126) Lianmin Zheng 2025-06-12 02:12:49 -07:00
• 7046e0fab7 feat: update blackwell setup (#7119) Yineng Zhang 2025-06-12 01:54:40 -07:00
• 930746d93c Improve log status (#7115) Liangsheng Yin 2025-06-12 14:38:24 +08:00
• 84727a5139 [sgl-kernel] Add cuda kernel for moe_ep_silu_and_mul (#6919) Yuan Luoandluoyuan.luo 2025-06-12 11:43:08 +08:00
• ef32677444 Fix positional argument (#7093) Quanfeng Li 2025-06-12 09:31:13 +08:00
• 021f76e4f4 [Perf] Refactor LoRAManager to eliminate stream syncs and redundant computations (#6994) Lifu Huang 2025-06-11 16:18:57 -07:00
• 777688b892 [feat]: Emit fixed-size KV blocks events (#6824) Faradawn Yang 2025-06-11 15:07:58 -05:00
• 0ca594eda9 [FIX]remove redundant code in logits_processor.py (#7079) Neo 2025-06-12 02:49:30 +08:00
• 31d6dee5c4 Support VILA models (#6106) Zijian 2025-06-12 02:47:25 +08:00
• 02543b545c Fix misusing the "_is_cuda". (#7091) sogalin 2025-06-12 02:21:31 +08:00
• 25a6a9aa22 Fix circular import in test_prefix_chunk_info.py (#7097) Baizhou Zhang 2025-06-11 10:57:45 -07:00
• 83d87685c5 vlm: adapt internvl to VisionAttention (#6870) Mick 2025-06-11 16:16:04 +08:00
• 2a5f0100e0 Fix GGuf and add back test_gguf.py (#7067) Baizhou Zhang 2025-06-10 21:07:20 -07:00
• dbdf76ca98 Clean up docs for server args and sampling parameters (generated by grok) (#7076) Lianmin Zheng 2025-06-10 19:55:42 -07:00
• f2a75a66c4 update doc (#7046) Ximingwang-09andximing.wxm 2025-06-11 10:02:01 +08:00
• 6b12d6a8d5 Simplify the heuristics for setting --mem-fraction-static (#7054) Lianmin Zheng 2025-06-10 19:01:39 -07:00
• 0f218731e3 Do not run frontend_reasoning.ipynb to reduce the CI load (#7073) Lianmin Zheng 2025-06-10 17:15:31 -07:00
• 14c18d25df Frontend language separate reasoning support (#6031) Yudi Xue 2025-06-10 17:11:29 -07:00
• 90bd3e32d6 Improve perf tuning docs (#7071) Lianmin Zheng 2025-06-10 16:55:04 -07:00
• ca9291181d [Feature] Add Logit Bias (#6579) Brayden ZhongandCinjon Resnick 2025-06-10 18:39:25 -04:00
• 344adb00ec fix arm sgl-kernel link issue (#7066) Yineng Zhang 2025-06-10 15:23:22 -07:00
• b56de8f943 Open AI API hidden states (#6716) kyle-pena-kuzco 2025-06-10 17:37:29 -04:00
• ce5ee3bdf0 chore: update dev docker (#7064) Yineng Zhang 2025-06-10 12:57:58 -07:00
• a0e4d4eb53 Fix missing tool call id if tool call index >0 in streaming tool call output. (#7049) Xu Wenqing 2025-06-11 03:56:43 +08:00
• 2f58445531 Revert "Add sanity checks when a test file is not added to CI (#6947)" (#7063) Yineng Zhang 2025-06-10 12:43:25 -07:00
• fe55947acd Add sanity checks when a test file is not added to CI (#6947) fzyzcjy 2025-06-11 03:34:57 +08:00
• 19995dd78e Tiny fix cutlass_mla_get_workspace_size stub incorrect signature (#7057) fzyzcjy 2025-06-11 03:27:57 +08:00
• 3b014bc13d Fix test_lora.py CI (#7061) Baizhou Zhang 2025-06-10 12:24:46 -07:00
• d7c3e8e93d [fix] libmlx5.so already in base image (#7060) Hank Han 2025-06-11 03:22:24 +08:00
• 8ea7df6114 [WA] fix output data is nan in CI test "test_moe_eval_accuracy_large.py" (#7021) 3 people 2025-06-11 00:08:10 +08:00
• 4a102a2b02 Minor style fix in cuda_graph_runner.py (#7053) Lianmin Zheng 2025-06-10 06:32:41 -07:00
• 6406408a70 Clean up server_args.py (#7037) Lianmin Zheng 2025-06-10 05:34:29 -07:00
• 019851d099 Fix eagle on AMD (#7051) Lianmin Zheng 2025-06-10 05:22:40 -07:00
• 2dae104dca Minor cleanup of fa3 backend (#6999) Lianmin Zheng 2025-06-10 03:58:44 -07:00
• cef6655b26 fix 24.12 docker (#7045) Yineng Zhang 2025-06-10 02:43:22 -07:00
• 27196d4148 [Docker] Upgrading base image from 24.04 to 24.12 (#7043) Swipe4057 2025-06-10 12:23:05 +03:00
• bb185b0e92 Update README.md (#7040) Lianmin Zheng 2025-06-10 01:59:14 -07:00
• 4f723edd3b chore: bump v0.4.7 (#7038) Yineng Zhang 2025-06-10 01:56:20 -07:00
• fcde67b016 CPU: map changes from developing branch in sgl-kernel (#6833) YanbingJiangandmingfeima 2025-06-10 16:08:15 +08:00
• 81372f3bef Fix fused_moe triton configs (#7029) yudian0504 2025-06-10 14:23:03 +08:00
• baa6624d7c [CI] Add CI workflow for sgl-router docker build (#7027) Arthur Cheng 2025-06-09 23:16:44 -07:00
• c2b16795b5 Add decode req pool (#6980) Byron Hsu 2025-06-09 21:23:36 -07:00
• f6ebba537a Support both approximate and exact expert distribution collection (#6964) fzyzcjy 2025-06-10 11:56:17 +08:00
• 6716b41786 Update default settings for blackwell (#7023) Baizhou Zhang 2025-06-09 20:37:47 -07:00
• 1c8b42c84c chore: update pr test xeon (#7018) Yineng Zhang 2025-06-09 17:36:25 -07:00
• f20f70003d Fix torch version in blackwell dockerfile (#7017) Qiaolin Yu 2025-06-09 20:06:50 -04:00
• f40942ad63 Migrate to assertEqual (#6741) Emmanuel Ferdman 2025-06-10 02:47:39 +03:00
• dc0705a504 Simplify prepare_extend_after_decode (#6987) Lianmin Zheng 2025-06-09 16:39:21 -07:00
• a968c888c0 Fix torchvision version for Blackwell (#7015) Wenxuan Tan 2025-06-09 17:50:19 -05:00
• a979daac3b Fallback to lower triton version for unfound fused moe configs (#7013) Baizhou Zhang 2025-06-09 15:41:03 -07:00
• f1569876d5 feat: add direct routing strategy to DP worker (#6884) ishandhanani 2025-06-09 11:44:05 -07:00
• 3465d7ae78 Update amd nightly models CI. (#6992) Sai Enduri 2025-06-09 10:54:08 -07:00
• e58423b2b9 Fix cutlass MLA gets almost zero accuracy (#6998) fzyzcjy 2025-06-10 01:16:29 +08:00