Commit Graph
26 Commits
Author SHA1 Message Date
84523d6785 [Spec] Reuse K3 auxiliary outputs across decode CUDA graph sizes (#40159)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-09-29 14:24:51 -07:00
Cheng Wan 49e7bd179d [Refactor] Group layer boundary unit tests (#41556) 2026-09-28 20:18:08 -07:00
Cheng Wan de9f850b36 [Refactor] Rename the module to layer_boundary (#41555) 2026-09-28 20:17:50 -07:00
Cheng Wan 701213c867 [Refactor] Retire the layer facade and simplify boundary internals (#41554) 2026-09-28 20:17:33 -07:00
Cheng Wan 9b89b96230 [Refactor] Migrate specialized decoder and overlap boundaries (#41553) 2026-09-28 20:17:17 -07:00
Cheng Wan 42f58b4da7 [Refactor] Carry residual state across stage boundaries (#41549) 2026-09-28 20:15:03 -07:00
Cheng Wan 0e907b755d [Refactor] Split the layer communicator into a package (move only) (#41439) 2026-09-27 16:54:41 -07:00
Cheng Wan deaa7f1463 [Refactor] Run two-batch-overlap layers on the declared boundaries (#41426) 2026-09-27 16:50:03 -07:00
Cheng Wan 963e9fb42d [Refactor] Leave the FFN reduction to the next layer under attention DP (#41197) 2026-09-25 01:47:03 -07:00
Cheng Wan 9d7f44bdbb [Refactor] Carry a deferred FFN all-reduce as UnreducedOutput and complete it in the next layer without the fused kernel (#41196) 2026-09-25 01:46:03 -07:00
f404db94d4 [DP attention] Publish DP buffer sizes from a ForwardBatch (#40858)
Co-authored-by: hanminglu <hanminglu@fb.com>
Co-authored-by: oss-sync bot (pranjalssh) <pranjalssh@users.noreply.github.com>
2026-09-24 11:34:19 -07:00
Xiaozhu Mengandmxz 111aeedd37 [Runtime] Add decode CUDA graph hooks for eager logits processing (#40222)
Co-authored-by: mxz <mxz@fb.com>
2026-09-18 23:27:03 -07:00
sglang-botandsglang-bot 6252993afe chore: update CI test est_time values (#38238)
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
2026-09-06 17:49:41 -07:00
44c786679f [sp] Make attention-TP sequence sharding a per-forward batch property (#37546)
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
2026-09-04 02:11:03 -07:00
Alex NailsandAlison Shao 28262c20df [CI][RFC] Replace black-jupyter with ruff-format (#37210)
Co-authored-by: Alison Shao <a.shao@wustl.edu>
2026-09-02 19:46:08 -07:00
Promisewjx 9d07b9e227 [Performance] Reduce idle DP work in breakable prefill CUDA graphs (#33871) 2026-08-27 10:09:54 +08:00
Cheng Wan 9b44695713 test: recover the config-namespace-migration deferrals (#33171)
The module-skipped tests injected config by faking get_server_args (a
SimpleNamespace stand-in patched onto the module) or by writing fields
onto a ServerArgs instance post-publish — both invisible to the namespace
accessors the production code now reads. Re-enable them by publishing the
config they need (get_context().override_server_args seeding, scoped per
test), asserting bag state where the old assertions checked instance
write-through (declare_load_time_override is bag-only), and extending the
per-runner stubs the code genuinely reads (kv_cache_dtype_str,
max_total_tokens, context_len).

The unified-radix-cache file (which grew a large hicache/insert-walk suite
while skipped) is recovered in the same change:

- test_cache_finished_req_strips_thinking (19 parametrized classes) wrote
  strip_thinking_cache onto the ServerArgs instance; the cache reads
  get_serving().strip_thinking_cache — use the serving bag's scoped
  override.
- test_shallower_crossing_backs_up_above_backuped_middle staged its
  broken-backup-continuity setup through insert_host, which now
  deliberately drops refills below an un-backed-up node under
  write-through (host_insert_dropped). Build the same tree state through
  an explicit backup + device eviction.

Every config-namespace-migration deferral is recovered, so the deferral
ratchet (test_migration_deferral_ratchet.py) has done its job and is
retired.
2026-08-01 08:59:27 -07:00
Yuzhen Zhou b954e9cf3d [6/6][kimi-deterministic] Use deterministic seeded coins for EAGLE rejection sampling (#30822) 2026-07-24 02:11:21 -07:00
Cheng Wan e1479cc966 config: route runtime config adjustments through the namespace bags (#31812) 2026-07-22 01:17:22 -07:00
Cheng Wan 1f15308dca [refactor] Retire the legacy config accessor and the remaining process singletons (#30493) 2026-07-09 02:10:47 -07:00
Cheng Wan be32c57598 [refactor] Read resolved config from server_args fields; retire the flags mirror tier (#30346) 2026-07-07 21:28:34 -07:00
Cheng Wan 8fb99bbaf8 [refactor] Config resolution pipeline: full-stack review (10-PR series, review only) (#30137) 2026-07-05 00:00:07 -07:00
karverma-amd 2552b860a3 [AMD][bugfix] Place TBO cuda-graph num_token_non_padded buffer on model devices (#28337) 2026-06-20 18:06:22 -07:00
Cheng Wan 53318911ca Add get_parallel(): a structured accessor for parallel-topology state (#28567) 2026-06-17 20:23:43 -07:00
Cheng WanandClaude Opus 4.8 9097647090 Route the eager forward path through the CUDA graph input-buffer registry (#27407)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 14:35:53 -07:00
Cheng WanandClaude Opus 4.8 0aa72a9e76 Replace skip_attn_backend_init with a batch-carried attention plan marker (+ staleness re-plan) (#27193)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 17:13:18 -07:00