Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• d5d80ab477 [Bugfix] Fix scores mask for moe topk (#3705) chenxiaobing 2025-02-21 02:17:23 +08:00
• ddcf9fe3be Optimize triton attention custom mask (#3731) Ke Bao 2025-02-21 00:54:41 +08:00
• 6252ade985 revert BLOCK and num_warps on HIP (#3722) HAI 2025-02-20 07:30:18 -08:00
• 1eb8eade2b add control for cutlass fp8 blockwise gemm (#3727) yizhang2077 2025-02-20 16:10:35 +08:00
• 3c7bfd7eab Docs: Fix layout with sub-section (#3710) Chayenne 2025-02-19 15:44:30 -08:00
• bb121214c2 Variance measure for reasoning benchmark (#3677) simveit 2025-02-19 20:49:49 +01:00
• 55de40f782 [Docs]: Fix Multi-User Port Allocation Conflicts (#3601) 3 people 2025-02-19 19:15:44 +00:00
• 6b0aeb58fd [moe] optim: reduce memory consumption in fused_moe (#3692) Cheng Wan 2025-02-19 13:25:05 -05:00
• bb3e526823 [k8s] remove unnecessary hostIPC for security concern (#3700) Peter Pan 2025-02-20 02:11:21 +08:00
• f93e915817 [Docs] Add SkyPilot DeepSeek example (#3706) Zhanghao Wu 2025-02-19 10:10:23 -08:00
• 55a7ec388f use warp shuffle style reduce and flashinfer vectorize (#3628) Xiaoyu Zhang 2025-02-19 20:53:51 +08:00
• fe0673f1cc set NCCL_IB_GID_INDEX=3 for multi node NVIDIA InfiniBand if needed (#3698) Yineng Zhang 2025-02-19 20:50:22 +08:00
• 99c1b9d2ee fix: apply cache size limit of attention mask for VisionAttention (#3657) Mick 2025-02-19 20:16:48 +08:00
• 634a3561ac AMD Prefill optimize (#3665) 3 people 2025-02-19 01:35:58 +08:00
• 424848d26f fix: remove dependency on latest transformers impl (#3635) Mick 2025-02-19 01:14:11 +08:00
• e5ce395a6c Fix draft decode max batch size (#3676) Ke Bao 2025-02-18 23:03:26 +08:00
• f983213a1f update pr-test (#3663) Yineng Zhang 2025-02-18 17:23:43 +08:00
• 67fc595bb8 [Feature] Apply Cublas Grouped Gemm kernel (#3629) Baizhou Zhang 2025-02-17 23:18:31 -08:00
• 07ab4d4a2d fix #3654 Yineng Zhang 2025-02-18 15:16:16 +08:00
• 522e18eaeb Update amd docker image. (#3654) saienduri 2025-02-17 20:12:55 -08:00
• c51dc2cc8d Docs: Deploy multi-node inference (LWS method) using sglang in a K8s cluster (#3624) ybyang 2025-02-18 10:14:20 +08:00
• ddf39d3fce [ROCm] Optimal MOE Tuning for AMD Radeon Graphics (#3567) yigex 2025-02-18 09:54:10 +08:00
• 2eab113206 [ROCm] Add additional block quant GEMM tuning configs for AMD GPUs. (#3616) Wen-Heng (Jack) ChungandHAI 2025-02-17 17:54:18 -06:00
• 058d199d4e use transformers 4.48.3 (#3650) Yineng Zhang 2025-02-18 04:40:47 +08:00
• a5375adc3a chore: bump v0.4.3.post2 (#3645) Yineng Zhangandpankajroark 2025-02-18 02:48:30 +08:00
• 75d171a9c5 chore: update flashinfer v0.2.1.post2 (#3644) Yineng Zhang 2025-02-18 02:47:42 +08:00
• 714f3e6362 feat: support flashinfer mla with prefix cache (#3643) Yineng Zhang 2025-02-18 02:06:43 +08:00
• c38f3aed24 support multi-gpu block-gemm tuning (#3639) Xiaoyu Zhang 2025-02-18 00:00:35 +08:00
• 2e6be53e7d fix Dockerfile.rocm Yineng Zhang 2025-02-17 22:13:03 +08:00
• e782eb7e6a chore: bump v0.4.3.post1 (#3638) Yineng Zhang 2025-02-17 21:58:19 +08:00
• e319153be8 update unit test (#3636) Yineng Zhang 2025-02-17 21:06:10 +08:00
• 32b44d2fca add mtp unit test (#3634) Yineng Zhang 2025-02-17 19:04:07 +08:00
• 5f1a485d9e Revert "[ROCm] Use tl.range() in block GEMM kernels with `num_stage… (#3632) Yineng Zhang 2025-02-17 18:01:21 +08:00
• c9565e49e7 [docker] added rdma support (#3619) Shenggui Li 2025-02-17 15:36:16 +08:00
• d03c4c25a7 [docs] Update sampling_params.md (#3617) Shi Shuai 2025-02-17 02:52:30 +00:00
• 8f13377dea Draft of updated doc for sampling params. (#3260) simveitandshuaills 2025-02-16 23:28:22 +01:00
• 3d4a8f9bc0 Benchmark for reasoning models (#3532) simveitandChayenne 2025-02-16 20:07:30 +01:00
• 7474bed883 Update to latest amd image. (#3597) saienduri 2025-02-16 08:29:47 -08:00
• 03caefeb51 [ROCm] Use tl.range() in block GEMM kernels with num_stages set by host. (#3535) Wen-Heng (Jack) ChungandHAI 2025-02-16 03:40:38 -06:00
• bcc213df61 Model: Support Qwen 2.5 vl (#3258) Mick 2025-02-16 16:58:53 +08:00
• 39416e394a fix lockfile and port_registry file permission error (#3598) 3 people 2025-02-15 22:14:45 -05:00
• 231c40d859 [docs] added favicon to sphinx html (#3564) Shenggui Li 2025-02-16 02:21:21 +08:00
• bbc47c348f fix apply_token_bitmask_inplace_cuda (#3594) Yineng Zhang 2025-02-15 23:55:08 +08:00
• dfce926921 fix high qps crash when enable mtp (#3592) Yineng Zhangandispobock 2025-02-15 23:11:28 +08:00
• 6718b10996 fix eagle unit test (#3591) Yineng Zhang 2025-02-15 23:10:48 +08:00
• 7711ac6ed0 doc: emphasize and notify the usage of chat_template (#3589) MickandChayenne 2025-02-15 16:10:32 +08:00
• 7443197a63 [CI] Improve Docs CI Efficiency (#3587) Shi Shuaiandzhaochenyang20 2025-02-15 03:57:00 +00:00
• 862dd76c76 Support NextN (MTP) speculative decoding for DeepSeek-V3/R1 (#3582) Ke Bao 2025-02-15 05:28:34 +08:00
• fb4c9c3a30 [fix] added support for vlm in offline inference (#3548) Shenggui Li 2025-02-15 05:27:29 +08:00
• d973c78e79 ROCm docker: triton update (#3584) HAI 2025-02-14 10:26:32 -08:00
• 6ce6eabbcc Copy config files for MI300X to support in virtualized environments (#3505) Jesse Lopez 2025-02-14 09:23:32 -08:00
• 4e23c961e8 docs: update install (#3581) Yineng Zhang 2025-02-14 18:54:50 +08:00
• 3efbdf68b9 fix sgl-kernel codestyle (#3563) Xiaoyu Zhang 2025-02-14 18:05:52 +08:00
• 6cc309557a Add support for OpenAI API o1 model (#3363) Chuyue SunandShan Yu 2025-02-13 19:43:00 -08:00
• 31eec35ba8 fix doc (#3558) Yineng Zhang 2025-02-14 10:11:31 +08:00
• ac963be234 update flashinfer-python (#3557) Yineng Zhang 2025-02-14 09:52:56 +08:00
• e0b9a423c8 chore: bump v0.4.3 (#3556) Yineng Zhang 2025-02-14 09:43:14 +08:00
• e082142519 chore: bump 0.0.3.post6 sgl-kernel (#3555) Yineng Zhang 2025-02-14 08:55:15 +08:00
• 70f894b810 feat: support flashinfer mla attention for deepseek v3 (#3550) Yineng Zhang 2025-02-14 08:50:14 +08:00
• 368de3661e Update install docs (#3553) simveitandChayenne 2025-02-13 22:42:51 +01:00
• 20de05a753 update README (#3543) Yineng Zhang 2025-02-13 17:22:11 +08:00
• f076328bb7 fix moe_align_kernel shm init not sync bug (#3534) Xiaoyu Zhang 2025-02-13 16:47:00 +08:00
• bf2a70872e Update DeepSeek V3 Doc (#3541) Jhin 2025-02-13 01:15:37 -06:00
• 871a4aa1bf [ROCm] Add ROCm tuning configs for AMD Instinct MI325X. (#3536) Wen-Heng (Jack) Chung 2025-02-12 22:09:36 -06:00
• 98eecbda54 integrate blockwise fp8 kernel (#3529) yizhang2077 2025-02-13 04:39:33 +08:00
• 4430c0a513 chore: bump 0.0.3.post5 sgl-kernel (#3530) Yineng Zhang 2025-02-13 01:51:46 +08:00
• 640363ad20 support blockwise fp8 matmul kernel (#3267) yizhang2077 2025-02-13 01:49:33 +08:00
• 8616357a97 Fix deepseek awq v3 (#3450) Liangsheng Yin 2025-02-12 22:09:52 +08:00
• 8adbc78b30 added llama and cleaned up (#3503) Zachary Streeter 2025-02-12 04:48:30 -06:00
• 45e3a7bc41 use sgl_per_token_group_quant_fp8 kernel (#3493) Xiaoyu Zhang 2025-02-12 18:40:42 +08:00
• b96e92e6e6 chore: bump 0.0.3.post4 sgl-kernel (#3523) Yineng Zhang 2025-02-12 17:28:36 +08:00
• 693c2600e0 refine deepseek_v3 launch server doc (#3522) Xiaoyu Zhang 2025-02-12 17:27:07 +08:00
• ced680663c doc: Support a new vLM (#3405) Mickandzhaochenyang20 2025-02-12 16:43:14 +08:00
• b8318aec48 Make NCCL NVLS configurable (#3502) Ata Fatahi 2025-02-11 14:25:06 -05:00
• 2f48221033 docs: update install Yineng Zhang 2025-02-12 03:13:31 +08:00
• d81ac4434e MI30x: More graph captures for larger batch sizes and concurrencies (#3420) HAI 2025-02-11 11:04:38 -08:00
• 2491cc928d add deepseek-v3 amd docker command (#3495) Zachary Streeter 2025-02-11 13:03:08 -06:00
• 67c5de9286 fix router typo (#3496) Didier Durand 2025-02-11 20:00:57 +01:00
• 1e2cf2b541 fix server_arguments typo (#3499) Didier Durand 2025-02-11 19:59:53 +01:00
• 9490d15772 fix supported_models Qwen typo (#3498) Didier Durand 2025-02-11 19:59:18 +01:00
• eefcbdd353 fix deepseek_v3 typo (#3497) Didier Durand 2025-02-11 19:58:36 +01:00
• 7e6d5fc694 Support Eagle cuda graph for Triton backend (#3500) Ke Bao 2025-02-12 02:27:45 +08:00
• cadd5dbe6a Tune MI300X fused MoE Triton kernel JSON config. (#3492) Wen-Heng (Jack) Chung 2025-02-11 12:27:25 -06:00
• bb418ced80 optimize per token group quant fp8 (#3490) Xiaoyu Zhang 2025-02-11 22:19:05 +08:00
• fdf04a1426 [ROCm] Add ROCm tuning config to block gemm and Re-tune for AMD Radeon Graphics (#3418) 3 people 2025-02-11 15:55:04 +08:00
• 5f0e7de339 [Feat] Return hidden states (experimental) (#3364) Jackmin801andChayenne 2025-02-10 15:54:37 -08:00
• 2f47d710ae refine some typo (#3473) Xiaoyu Zhang 2025-02-10 23:35:44 +08:00
• 4fe92bfca5 fix mla test (#3469) Yineng Zhang 2025-02-10 21:12:00 +08:00
• d23cb9a01e [Eagle] reduce one draft forward (#3468) Ying Sheng 2025-02-10 04:21:49 -08:00
• 2d61132374 Support Eagle2 for Triton backend (#3466) Ke Bao 2025-02-10 20:00:42 +08:00
• cddb1cdf8f chore: bump v0.4.2.post4 (#3459) Yineng Zhang 2025-02-10 14:12:16 +08:00
• fa1b40e00d use nvcr.io/nvidia/tritonserver:24.04-py3-min as base image (#3457) Yineng Zhang 2025-02-10 13:52:33 +08:00
• c45cab1c00 [Fix] Fix accuracy bug and refactor codes for lora (#3413) Baizhou Zhang 2025-02-09 21:29:00 -08:00
• 27c4c9cf52 remove _grouped_size_compiled_for_decode_kernels (#3453) Yineng Zhang 2025-02-10 13:01:21 +08:00
• 52a492a16e Update contribution_guide.md (#3452) Ying Sheng 2025-02-09 20:53:47 -08:00
• 36f6fc5093 feat: enable ragged fa3 by default on hopper 12.4+ (#3442) Yineng Zhang 2025-02-10 07:43:01 +08:00
• d87272750b fix ci (#3441) Yineng Zhang 2025-02-10 04:22:28 +08:00
• 6239d0b2e7 chore: bump sgl-kernel v0.0.3.post3 (#3440) Yineng Zhang 2025-02-10 04:00:52 +08:00
• 4cfd3add6d support version in sgl-kernel (#3439) Yineng Zhang 2025-02-10 03:49:52 +08:00
• 20cf910d8f [docs] Update quantization documentation (#3437) 3 people 2025-02-09 18:39:49 +00:00