minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Pass per-forward overrides to ForwardBatch.init_new as explicit arguments (#30670)

Browse Source
This commit is contained in:
fzyzcjy
2026-07-15 14:25:59 +08:00
committed by GitHub
parent 861d97d24d
commit e77d95c3d5
16 changed files with 147 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude/rules/forward-batch-init-new-purity.md
+17
View File
@@ -0,0 +1,17 @@
---
paths:
- "**/*.py"
---
# `ForwardBatch.init_new` must not mutate the ScheduleBatch
`init_new` (and any ForwardBatch factory) treats the input `ScheduleBatch` as
read-only. Per-forward overrides go through the kw-only params of `init_new` /
`TpModelWorker.forward_batch_generation`, never a ScheduleBatch field. Batch-prep
writes (`out_cache_loc`, `seq_lens_*`, ...) before `init_new` are out of scope.
Tolerated exceptions (don't add new ones):
- `seq_lens_sum` backfill — the `seq_lens` family is slated for removal (→ kv-committed lengths).
- `sampling_info` sub-object writes (grammars, canary ids) — shared object, until the sampling forward-copy op.
- `_expand_mrope_from_input` memoizing `mrope_position_delta_repeated_cache` — pre-existing.