|
5a6a1bb883
|
[mxfp8-kv] Skip writes to the reserved CUDA-graph padding slot (#35351)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Sam Shleifer <sam@thinkingmachines.ai>
|
2026-09-22 01:21:25 +08:00 |
|
|
|
8ae4a39b50
|
Gate mamba extra-buffer predicates on uses_mamba_radix_cache (#37474)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Sam Shleifer <sam@thinkingmachines.ai>
|
2026-09-17 14:39:12 +08:00 |
|
|
|
76e06febab
|
[CI] Add e2e test for dp-attention local control broadcast (#39437)
Co-authored-by: Sam Shleifer <sam@thinkingmachines.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-16 18:11:40 +08:00 |
|
|
|
6afb5e1771
|
Gate the idle-loop tree-cache sanity check behind a default-off env (#36205)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: hzh0425 <hzh0425@apache.org>
|
2026-08-30 00:10:28 +08:00 |
|
|
Sam ShleiferandClaude Fable 5
|
a2e88279c2
|
metrics: don't clock-rebase unset time sentinels in ReqTimeStats deserialization (#34335)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-08-12 14:27:47 +08:00 |
|
|
Sam ShleiferandClaude Fable 5
|
afb4f37ca5
|
[Inkling] silu_and_mul: replace helion kernels with plain Triton (#33903)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-08-08 15:00:23 +08:00 |
|
|
Sam ShleiferandClaude Fable 5
|
d238e36b24
|
[Fix] Restore data_parallel_rank alias on native /generate (dp-aware gateway routing is silently dropped) (#33565)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-08-08 14:59:16 +08:00 |
|
|
Sam ShleiferandClaude Fable 5
|
62a28197c0
|
Autotune flashinfer extend buckets at warmup (#32556)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-08-07 14:06:05 -07:00 |
|
|
Sam Shleifer
|
8e7d361def
|
[perf] Compute input logprobs without materializing the full-vocab log-softmax (#31958)
|
2026-08-07 14:01:08 -07:00 |
|
|
Sam Shleifer
|
58974ca16c
|
[perf] Assemble flat prompt top logprobs scheduler-side as numpy arrays (#32223)
|
2026-07-31 18:00:51 -07:00 |
|
|
Sam ShleiferandClaude Fable 5
|
62dfaaa0e0
|
[Nemotron] Fix decode track-save reading the stale tail of the CUDA-graph track buffer (#32555)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-07-29 19:03:44 -07:00 |
|
|
Sam Shleifer
|
d0e69d3881
|
[feat] Optional base64 encoding for the flat prompt top logprob arrays (#31960)
|
2026-07-29 12:15:56 -07:00 |
|
|
Sam Shleifer
|
5cc273a780
|
[feat] Opt-in flat response format for prompt top logprobs (#32078)
|
2026-07-26 23:44:34 -07:00 |
|
|
Sam Shleifer
|
b8bb1b4e5a
|
[lora] Fix WAR race: never write MoE runner output into hidden_states in place (#31870)
|
2026-07-24 00:58:55 -07:00 |
|
|
Sam Shleifer
|
1f9d778d1b
|
Skip dist_init/nccl port prechecks when the dist init method is overridden (#31410)
|
2026-07-23 12:40:41 -07:00 |
|
|
Sam Shleifer
|
b3a0185cab
|
model_runner: extract post-memory-pool wiring into _init_post_memory (#31601)
|
2026-07-19 00:57:54 +08:00 |
|
|
Sam Shleifer
|
d86ae51fcf
|
metrics: allow extra labels on HTTP request/response Prometheus metrics (#31530)
|
2026-07-19 00:57:26 +08:00 |
|
|
Sam Shleifer
|
0675d3033f
|
Fix mamba track-boundary seqlen under overlap scheduler (#31369)
|
2026-07-17 10:22:07 +08:00 |
|
|
Sam ShleiferandAlex Nails
|
dec0836302
|
Fix processor config loading for object-storage model paths (#31211)
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
|
2026-07-15 00:21:31 -07:00 |
|
|
Sam Shleifer
|
133254086b
|
Plug mamba_extra_buffer ping-pong slot leaks (#26941)
|
2026-06-04 21:46:36 +08:00 |
|
|
Sam Shleifer
|
ce2506e1c6
|
Deprecate record_nolora_graph dual MoE CUDA graph capture (#24314)
|
2026-05-15 19:20:58 -07:00 |
|
|
Sam Shleifer
|
e6f252e9b8
|
Cache FlashInfer autotune configs (#24156)
|
2026-05-05 02:00:40 +08:00 |
|
|
Sam Shleifer
|
63f225ca2e
|
[session] fix mamba pool leak in StreamingSession.release_session + plumb idle leak check (#23496)
|
2026-05-02 11:38:08 +08:00 |
|
|
|
d41e8c459d
|
Support RunAI loading for quantized checkpoints (#23850)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Sam Shleifer <sam@thinkingmachines.ai>
|
2026-05-02 11:11:40 +08:00 |
|
|
Sam Shleifer
|
b9e33d6a5b
|
Dual MoE CUDA graph capture for lora/nolora batches (#22809)
|
2026-04-22 14:11:11 -07:00 |
|
|
Sam Shleifer
|
1100b9865c
|
Fix MxInt4 MoE returning wrong output variable (#21348)
|
2026-03-26 10:57:09 +08:00 |
|
|
Sam Shleifer
|
0d49b13fdd
|
Fix circular import in quantization modules (#17372)
|
2026-01-21 15:47:09 +08:00 |
|