 fzyzcjy
|
e586a6f2c5
|
Report the whole server's world size in the scheduler's internal state (#35929)
|
2026-08-24 20:21:45 +08:00 |
|
|
fzyzcjy
|
6dd79576cd
|
Expose the declared sglang env vars of a scheduler in its internal state (#35928)
|
2026-08-24 20:20:29 +08:00 |
|
|
fzyzcjy
|
c56cee0f80
|
Support gated launch to defer startup memory allocation (#35927)
|
2026-08-24 20:19:45 +08:00 |
|
|
fzyzcjy
|
3b24d8981b
|
Report per-token weight-version spans in generation meta info (#35926)
|
2026-08-24 20:18:52 +08:00 |
|
|
fzyzcjy
|
981dfa2b83
|
Make the scheduler track the published weight version (#35925)
|
2026-08-24 20:17:50 +08:00 |
|
|
fzyzcjy
|
02a3dca738
|
Extract _make_abort_req from the scheduler's abort paths (#35924)
|
2026-08-24 20:16:38 +08:00 |
|
|
fzyzcjy
|
a37fdae562
|
Extract collect_inflight_reqs from abort_request for reusing (#35923)
|
2026-08-24 20:15:09 +08:00 |
|
|
fzyzcjy
|
1afab30577
|
Fix bookkeeping fields not encapsulated with real allocations in normal alloc, PD pre-alloc, DFlash and EAGLE (#29432)
|
2026-07-15 14:52:21 +08:00 |
|
|
fzyzcjy
|
e789ca24a7
|
Lightweight extract allocation logic from mem_cache/common.py to more clearly show nearly parallel variants (#29431)
|
2026-07-15 14:49:37 +08:00 |
|
|
fzyzcjy
|
c315df49bb
|
Fix abusing presence of req.req_pool_idx to indicate the presence of req.kv resources (#29430)
|
2026-07-15 14:48:39 +08:00 |
|
|
fzyzcjy
|
2d979f1d8c
|
Let the presence of req.kv indicate the existence of owned kv resources (#29429)
|
2026-07-15 14:47:16 +08:00 |
|
|
fzyzcjy
|
27256aee5b
|
Let cache backend do not couple with owned committed kv details and avoid kv_committed_freed/kv_overallocated_freed fields (#29428)
|
2026-07-15 14:43:10 +08:00 |
|
|
fzyzcjy
|
d8d76c4d12
|
Introduce req.kv container for coupled owned kv field lifecycle (#29427)
|
2026-07-15 14:40:38 +08:00 |
|
|
fzyzcjy
|
201ddeaba1
|
Avoid relaying per-step outputs through ScheduleBatch fields in disagg prefill and PP (#30677)
|
2026-07-15 14:33:37 +08:00 |
|
|
fzyzcjy
|
01343d2759
|
Avoid implicit running_batch access in dllm and pdmux scheduling (#30676)
|
2026-07-15 14:33:00 +08:00 |
|
|
fzyzcjy
|
21c62b9830
|
Rewrite pause_generation retract path as req-level release and requeue for clarity (#30675)
|
2026-07-15 14:32:12 +08:00 |
|
|
fzyzcjy
|
1967b9ec99
|
Fix missed hisparse release and stale field cleanup in pause retract (#30674)
|
2026-07-15 14:31:11 +08:00 |
|
|
fzyzcjy
|
b6cc897fea
|
Fix non-existent abort mode in Scheduler.pause_generation and inline retract_all (#30673)
|
2026-07-15 14:27:48 +08:00 |
|
|
fzyzcjy
|
52a88fb212
|
Avoid mutating ScheduleBatch fields in place (#30672)
|
2026-07-15 14:27:08 +08:00 |
|
|
fzyzcjy
|
e77d95c3d5
|
Pass per-forward overrides to ForwardBatch.init_new as explicit arguments (#30670)
|
2026-07-15 14:25:59 +08:00 |
|
|
fzyzcjy
|
861d97d24d
|
Remove dead ScheduleBatch fields and avoid inplace seq_lens bump (#30669)
|
2026-07-15 14:23:31 +08:00 |
|
|
fzyzcjy
|
a5a71c6c26
|
Enhance mechanical-refactor-verify skill with a whole-chain verifier, new relocation primitives, and generator inference (#30585)
|
2026-07-14 16:45:48 +08:00 |
|
|
fzyzcjy
|
0fe2dbd42c
|
Split initialize() into orchestration helpers (#31169)
|
2026-07-14 16:09:00 +08:00 |
|
|
fzyzcjy
|
bf04cc9b14
|
Extract cuda-graph setup into a module (#31168)
|
2026-07-14 16:04:45 +08:00 |
|
|
fzyzcjy
|
ed2fcd3201
|
Extract attention-backend setup into a module (#31167)
|
2026-07-14 16:03:48 +08:00 |
|
|
fzyzcjy
|
54f99a21d5
|
Narrow component dependencies to injected fields instead of ModelRunner (#31166)
|
2026-07-14 16:03:07 +08:00 |
|
|
fzyzcjy
|
6999007a13
|
Drop ModelRunner's duplicated parallel-degree fields and read them via self.ps (#31165)
|
2026-07-14 16:02:40 +08:00 |
|
|
fzyzcjy
|
cfd17301a8
|
Extract per-architecture KV-cache pool builders into KVCacheConfigurator (#31163)
|
2026-07-14 16:02:09 +08:00 |
|
|
fzyzcjy
|
d6cf2908ce
|
Introduce KVCacheConfigurator and migrate KV-cache config logic (#31162)
|
2026-07-14 16:01:45 +08:00 |
|
|
fzyzcjy
|
725920915f
|
Introduce ModelRunner.ps ParallelState (#31161)
|
2026-07-14 16:01:14 +08:00 |
|
|
fzyzcjy
|
1dc48c2c3b
|
Absorb capturer setup and extract the shared-mooncake gate (#31160)
|
2026-07-14 16:00:51 +08:00 |
|
|
fzyzcjy
|
08798dba0d
|
Extract MoE/EP setup into a moe_ep_setup module (#31159)
|
2026-07-14 16:00:24 +08:00 |
|
|
fzyzcjy
|
440aebdfe0
|
Extract small single-function helpers into modules (#31158)
|
2026-07-14 16:00:01 +08:00 |
|
|
fzyzcjy
|
17c04602c6
|
Extract spec aux-hidden-state resolution into a module (#31157)
|
2026-07-14 15:59:35 +08:00 |
|
|
fzyzcjy
|
39e508b7fc
|
Extract layer-index setup into a module (#31156)
|
2026-07-14 15:59:07 +08:00 |
|
|
fzyzcjy
|
5b540b16de
|
Extract load_model helpers into a load_model_utils module (#31155)
|
2026-07-14 15:58:40 +08:00 |
|
|
fzyzcjy
|
d15f6a9ac3
|
Introduce NgramEmbeddingManager component (#31154)
|
2026-07-14 15:58:08 +08:00 |
|
|
fzyzcjy
|
0f20f52e5e
|
Introduce RemoteInstanceWeightTransporter component (#31153)
|
2026-07-14 15:57:43 +08:00 |
|
|
fzyzcjy
|
caa85ea022
|
Extract init_torch_distributed and refactor into functions (#31152)
|
2026-07-14 15:56:57 +08:00 |
|
|
fzyzcjy
|
205a2f2de4
|
Move LoRA cuda-graph buffers and logging into LoRAManager (#31151)
|
2026-07-14 15:56:18 +08:00 |
|
|
fzyzcjy
|
e20c346541
|
Extract hybrid-arch helpers into configs.hybrid_arch and ModelConfig (#31150)
|
2026-07-14 15:55:32 +08:00 |
|
|
fzyzcjy
|
c9b4081016
|
Extract expert location updating into EPLBManager (#31149)
|
2026-07-14 15:54:40 +08:00 |
|
|
fzyzcjy
|
6507d4a090
|
Introduce WeightUpdater and WeightExporter components (#31148)
|
2026-07-14 15:53:33 +08:00 |
|
|
fzyzcjy
|
45dfa318fb
|
Extract kv cache dtype configuration into mem_cache (#31147)
|
2026-07-14 15:52:39 +08:00 |
|
|
fzyzcjy
|
cad8fe7a66
|
Extract leaf helpers out of ModelRunner into utility modules (#31146)
|
2026-07-14 15:52:03 +08:00 |
|
|
fzyzcjy
|
b1a60ad00d
|
Clean up ModelRunner by renaming effective-token property and remove dead code (#31145)
|
2026-07-14 15:50:53 +08:00 |
|
|
fzyzcjy
|
2cf753c4fe
|
Clarify ModelRunner.dp_size into attn_dp_size (#31142)
|
2026-07-14 15:49:05 +08:00 |
|
|
fzyzcjy
|
1e75ba236e
|
Avoid implicit field-based side channel in Scheduler planning (#29408)
|
2026-07-10 08:55:51 +08:00 |
|
|
fzyzcjy
|
5be9c9f7c6
|
Localize cur_batch field in Scheduler to avoid field-based state access (#29407)
|
2026-07-10 08:55:06 +08:00 |
|
|
fzyzcjy
|
69368d7593
|
Stop reading cur_batch in is_fully_idle and abort_request (#29406)
|
2026-07-10 08:54:31 +08:00 |
|
|
|
2e66707399
|
Fix pipeline-parallel abort missing in-flight requests in non-current microbatch slots (#29405)
Co-authored-by: burling <3637497+burling@users.noreply.github.com>
Co-authored-by: zhaotyer <89376832+zhaotyer@users.noreply.github.com>
|
2026-07-10 08:53:18 +08:00 |
|
|
fzyzcjy
|
6ab7a65d94
|
Support grad injection and step override in the dumper's model dump (#30657)
|
2026-07-09 20:26:05 +08:00 |
|
|
fzyzcjy
|
5b28465eb9
|
Cap diagnostic detail computation for failing tensors (#30656)
|
2026-07-09 20:19:51 +08:00 |
|
|
fzyzcjy
|
287291c232
|
Fix rel_diff being nan for bitwise-identical tensors (#30655)
|
2026-07-09 20:16:18 +08:00 |
|
|
fzyzcjy
|
0d7e8cfb85
|
Support per-regex diff-threshold predicates in the tensor comparator (#30654)
|
2026-07-09 20:15:36 +08:00 |
|
|
fzyzcjy
|
bc607ff650
|
Enhance mechanical refactor proof construction and verification skill (#30483)
|
2026-07-09 07:39:11 +08:00 |
|
|
fzyzcjy
|
6af1d5ff2d
|
Enhance large class styles and code styles (#28978)
|
2026-07-09 07:32:03 +08:00 |
|
|
fzyzcjy
|
2b64fc7a2c
|
Remove Req.extend_logprob_start_len field and make it pure (#27625)
|
2026-06-25 09:06:14 +08:00 |
|
|
fzyzcjy
|
7002a37ea1
|
Inline extend_range accessors and remove the extend_input_len/fill_len properties (#27611)
|
2026-06-25 08:56:44 +08:00 |
|
|
fzyzcjy
|
d0524d6433
|
Avoid scattered assignment of extend_input_len and fill_len by merging them into Req.extend_range (#27610)
|
2026-06-25 08:33:28 +08:00 |
|
|
fzyzcjy
|
563c3418a7
|
Avoid dual semantics of extend_input_len by computing the candidate on the fly (#27616)
|
2026-06-25 08:17:37 +08:00 |
|
|
fzyzcjy
|
1368717248
|
Add more testing for chunked prefill (#27506)
|
2026-06-09 20:19:30 +08:00 |
|
|
fzyzcjy
|
609f5f549c
|
Add mixed-prefix gsm8k eval and its CPU unit test (#27502)
|
2026-06-09 20:17:41 +08:00 |
|
|
fzyzcjy
|
71a0b10462
|
Fix the _chunked_req_scheduled_last_iter flag with a content-based stash gate (#26938)
|
2026-06-08 14:55:41 +08:00 |
|
|
fzyzcjy
|
f746e4a608
|
Fix fill_len asymmetric assignment statement in ignore-eos branch (#26999)
|
2026-06-08 14:53:21 +08:00 |
|
|
fzyzcjy
|
9034c2f9ae
|
Fix Req fill_len (fill_ids) having dual semantics by restricting to truncated/committed semantics (#26659)
|
2026-06-08 14:52:48 +08:00 |
|
|
fzyzcjy
|
259a2da3e0
|
Refactor Req.fill_ids into full_untruncated_fill_ids + fill_len with equivalence (#26637)
|
2026-06-08 14:52:18 +08:00 |
|
|
fzyzcjy
|
4201de11de
|
Extract release_req and retract_all as module-level free functions (#26548)
|
2026-06-08 14:51:45 +08:00 |
|
|
fzyzcjy
|
3197808283
|
Avoid calling filter_batch with chunked_req_to_exclude being things unrelated to chunked reqs (#26547)
|
2026-06-08 14:51:21 +08:00 |
|
|
fzyzcjy
|
f5fdf9c5d8
|
Speed up dump comparator percentile computation using numpy (#26874)
|
2026-06-08 14:50:31 +08:00 |
|
|
fzyzcjy
|
995e649190
|
Add parallel-rank dump filenames and pipeline-global layer remapping to dumper (#26850)
|
2026-06-08 14:49:22 +08:00 |
|
|
fzyzcjy
|
0a190d1c97
|
Fix PP is_fully_idle missing in-flight microbatches (#27446)
|
2026-06-07 17:38:26 +08:00 |
|
|
fzyzcjy
|
14b8f98a21
|
Complete server warmup before scripted runtime scripts start (#27445)
|
2026-06-07 17:37:37 +08:00 |
|
|
fzyzcjy
|
99cf0a4399
|
Fix flaky test_self_e2e_pd_perturb (#27426)
|
2026-06-06 19:31:08 +08:00 |
|
|
fzyzcjy
|
bf4f2ccc78
|
Add scripted-runtime unit, core integration, and chunked-prefill tests (#27413)
|
2026-06-06 09:08:35 +08:00 |
|
|
fzyzcjy
|
5a82db85f0
|
Add scripted-runtime KV-pool and lock-ref exhauster primitives (#27412)
|
2026-06-06 09:07:40 +08:00 |
|
|
fzyzcjy
|
dd176387c6
|
Add scripted-runtime harness core and wire scheduler/IPC hooks (#27411)
|
2026-06-06 09:07:01 +08:00 |
|
|
fzyzcjy
|
21201ef718
|
Add kv_canary PP self-test fixture and SWA divergence coverage (#27410)
|
2026-06-06 09:06:19 +08:00 |
|
|
fzyzcjy
|
8cea0473ea
|
Fix dp-attention token alignment in the dumper comparator e2e test (#26996)
|
2026-06-02 00:50:45 -07:00 |
|
|
fzyzcjy
|
f220c72929
|
Add periodic KV-canary stats logging and kernel-run-counter health check (#26821)
|
2026-05-31 10:00:19 +08:00 |
|
|
fzyzcjy
|
7dd19ae3d8
|
Add a sliding-window-attention divergence reporter for the KV-canary (#26820)
|
2026-05-31 09:59:28 +08:00 |
|
|
fzyzcjy
|
ae9db7ff4b
|
Add the KV-canary perturb modes and PD-disaggregation e2e tests (#26819)
|
2026-05-31 09:59:09 +08:00 |
|
|
fzyzcjy
|
6be4b32d8d
|
Add token-id verification to the KV-canary (#26818)
|
2026-05-31 09:58:51 +08:00 |
|
|
fzyzcjy
|
0ca610a6df
|
Add real-data KV verification to the KV-canary (#26817)
|
2026-05-31 09:58:32 +08:00 |
|
|
fzyzcjy
|
cdee16e144
|
Add the KV-canary perturb framework for fault-injection self-tests (#26816)
|
2026-05-31 09:58:14 +08:00 |
|
|
fzyzcjy
|
678e73a9ee
|
Add a deterministic token oracle and production write-input assertion (#26815)
|
2026-05-31 09:57:43 +08:00 |
|
|
fzyzcjy
|
268f4c82f1
|
Add rids/bootstrap-room int-hash plumbing for deterministic per-request identification (#26814)
|
2026-05-31 09:57:22 +08:00 |
|
|
fzyzcjy
|
9c43c3719f
|
Support EAGLE speculative decoding in the KV-canary (#26813)
|
2026-05-31 09:57:01 +08:00 |
|
|
fzyzcjy
|
30a22cc360
|
Add a periodic full-radix-tree KV-canary sweep (#26812)
|
2026-05-31 09:56:42 +08:00 |
|
|
fzyzcjy
|
27eb139ef7
|
Add the KV-canary mock-model end-to-end test harness (#26811)
|
2026-05-31 09:56:00 +08:00 |
|
|
fzyzcjy
|
e188745ee0
|
Add KV-canary SWA + DeepSeek-V4 pool support (#26810)
|
2026-05-31 09:55:34 +08:00 |
|
|
fzyzcjy
|
9ecf314970
|
Add the KV-canary install API and forward-path wiring (#26809)
|
2026-05-31 09:55:03 +08:00 |
|
|
fzyzcjy
|
11391b2a1c
|
Add the KV-canary core: data layer, MHA KV-pool patcher, and per-forward runner (#26808)
|
2026-05-31 09:54:24 +08:00 |
|
|
fzyzcjy
|
736ad1f32a
|
Add the KV-canary plan JIT kernels (#26807)
|
2026-05-31 09:53:56 +08:00 |
|
|
fzyzcjy
|
70e983a2da
|
Add the KV-canary write JIT kernel and reference implementation (#26806)
|
2026-05-31 09:53:34 +08:00 |
|
|
fzyzcjy
|
16950954c6
|
Add the KV-canary verify JIT kernel and reference implementation (#26805)
|
2026-05-31 09:52:52 +08:00 |
|
|
fzyzcjy
|
8a20c58252
|
Pull test_utils server-launch boilerplate into reusable helpers (#26804)
|
2026-05-31 09:52:15 +08:00 |
|
|
fzyzcjy
|
ce647cbcfd
|
Add a SimplePhaseChecker for execution-phase assertions (#26803)
|
2026-05-31 09:51:39 +08:00 |
|
|
fzyzcjy
|
7f93952f79
|
Add a debug toggle for selectively reverting PR fixes (#26802)
|
2026-05-31 09:50:12 +08:00 |
|
|
fzyzcjy
|
656e75b798
|
Add a nullcontext placeholder in the forward path for KV-canary (#26801)
|
2026-05-31 09:49:32 +08:00 |
|