minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 10 Packages Projects Releases Wiki Activity
17,490 Commits 2 Branches 0 Tags
afbca97f74bf8a2afa67d15b193076843b50af3b
Commit Graph
33 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Alex Nails 78fa921189 [ROCm] Fix QuickReduce fp16 saturation corrupting bf16 all-reduces (106M non-finite -> 0, +0.3%) (#34484) 2026-08-29 22:08:17 -07:00
Alex NailsandClaude Opus 5 00fbb6e8ac [Perf] Tune the W4AFP8 DeepEP low-latency requant launch geometry (#35760) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-29 16:03:01 -07:00
Alex NailsandClaude Opus 5 998eeda0a5 [CI] Register test_dflash_logits at its real cost (#36242) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-24 18:22:49 -07:00
Alex NailsandClaude Opus 5 5ffdb02d0c [CI] Cut repeated tokenizer loads, serial subprocesses and a double scan (#36241) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-24 18:22:43 -07:00
Alex NailsandClaude Opus 5 68575b23d0 [CI] Stop the config ratchets re-parsing the package on every scan (#36240) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-24 18:22:32 -07:00
Alex NailsandClaude Opus 5 1ec20fd25d [CI] Stop the resolution ratchets re-parsing the whole package (#36235) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-24 16:55:35 -07:00
Alex NailsandClaude Opus 5 2070927e00 [Triton] Bound the sliding-window extend-attention KV loop: -86.6% on SWA layers, -9.4% prefill GPU, bit-identical (#34462) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-24 00:58:52 -07:00
Alex NailsandClaude Opus 5 666b08b4a5 [ROCm] Extend the gfx950 extend-attention tile to head_dim <= 128: -43% kernel, -14% TTFT, bit-identical (#34461) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-24 00:29:39 -07:00
Alex NailsandClaude Opus 5 b6dcd393d6 [Fix] Support 128-aligned hidden sizes in the W4AFP8 DeepEP low-latency requant kernel (#35593) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-19 22:18:17 -07:00
Alex NailsSiming DengR0CKSTARJiminatordamahuaClaude Opus 5
 		2969ab3d41 [MLX] Window-bounded SWA KV storage and in-graph sampling (#34166) 
Co-authored-by: Siming Deng <siming_deng_stat@163.com>
Co-authored-by: R0CKSTAR <yeahdongcn@gmail.com>
Co-authored-by: Jiminator <Jiminator@users.noreply.github.com>
Co-authored-by: damahua <damahua@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-09 21:28:51 -07:00
Alex NailsandClaude Fable 5 6fa3f9df11 [Bugfix] Treat unsharded model.safetensors as HF weights in Mistral-native format detection (#33671) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-05 01:54:46 -07:00
Alex NailsandClaude Opus 5 bfa4e4a57b [Nemotron] Hoist mamba track-mask host syncs out of the per-layer prefill path (#32589) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-03 23:58:28 -07:00
Alex Nails ec4a7fa2b7 codeowners update (#32608) 2026-07-27 21:21:20 -07:00
Alex NailsandClaude Opus 4.8 b83041c3cc Migrate CompressedTensorsW4A4Nvfp4MoE TRT-LLM path onto MoeRunner (#32248) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-24 20:57:38 -07:00
Alex NailsandClaude Opus 4.7 3d2e7cc601 [gRPC] Native server: launcher + HTTP + server args wiring (3/4) (#23508) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-07-07 14:57:25 -07:00
Alex NailsandClaude Opus 4.7 b5e4e289b1 [gRPC] Native server: Python bridge entrypoint (2/4) (#23507) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-06-22 13:08:33 -07:00
Alex NailsandClaude Opus 4.8 533b59d00c Add @alexnails as codeowner for srt/platforms (#28066) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-12 10:28:58 -07:00
Alex Nails 1d19721394 [gRPC] Native server: Rust crate (1/N) (#23506) 2026-05-18 22:31:38 -07:00
Alex Nails 4df42da658 Introduce CudaDeviceMixin and CudaSRTPlatform (#24096) 2026-05-15 10:59:02 -07:00
Alex NailsandClaude Opus 4.7 c3ab5bec7d ci: consolidate rust + protoc install across workflows (#23700) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 13:39:02 -07:00
Alex NailsandClaude Opus 4.7 345fecc547 fix(bench): wire request_func in bench_long_context ContextWorkloadGenerator (#23898) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-28 14:45:51 -07:00
Alex NailsandClaude Opus 4.7 579bd0b152 [bug fix] has_fp8_weights_in_checkpoint: handle HF repo IDs, not just local paths (#23542) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-23 12:56:49 -07:00
Alex Nails 332ec5e5ee [release] install rust toolchain in main dockerfile (#23014) 2026-04-20 09:50:08 -07:00
Alex NailsandClaude Opus 4.7 bea4c895c1 [gRPC] Pass --experimental_allow_proto3_optional to protoc in build.rs (#23226) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-20 17:20:14 +08:00
Alex Nails 10e17cc55e [gRPC] Native gRPC server: proto + Rust crate scaffold + server args (#22736) 2026-04-20 12:39:35 +08:00
Alex Nails 43eb66028f ci: install rust toolchain in ci_install_dependency.sh (#23017) 2026-04-16 23:18:22 -07:00
Alex NailsClaude Opus 4.6gemini-code-assist[bot]
 		8092431316 [serving] replace O(n²) stream_buffer string concat with integer offset (#22606) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-14 14:48:44 -07:00
Alex NailsandClaude Opus 4.6 c6fd9a00c7 [tokenizer] eliminate O(n²) copy in non-incremental streaming (#22567) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-11 23:05:36 -07:00
Alex NailsandClaude Opus 4.6 8eac618a8d [tokenizer] lazy text accumulation + use deltas directly for streaming (#22548) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-10 21:26:04 -07:00
Alex Nails 0af9166474 [tokenizer] improve non streaming request processing + some small fixes. (#20310) 2026-04-10 15:46:12 -07:00
Alex NailsandClaude Opus 4.6 931dbceadc [CI] Set RUNAI_STREAMER_MEMORY_LIMIT=0 for stage-b-test-1-gpu-small (#22346) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-08 02:23:35 -07:00
Alex NailsandClaude Opus 4.6 493ec91cbe [CI] Fix stage-b-test-1-gpu-large (0) timeout by reordering LoRA tests and using tokenizer from cache (#22292) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-07 20:00:44 -07:00
Alex Nails 912494f596 [CI] Fix lint that was not applied in #21458 (#21818) 2026-03-31 23:58:12 -07:00