 fzyzcjy
|
ef5ac6f01e
|
Support shared prefix (gsp) in schedule simulator (#16353)
|
2026-01-04 08:03:16 +08:00 |
|
|
fzyzcjy
|
c88aaf22c7
|
Support in-flight request age metrics for router (#16341)
|
2026-01-04 07:38:53 +08:00 |
|
|
fzyzcjy
|
e139d2aa76
|
Super tiny fix CI (#16351)
|
2026-01-04 07:24:46 +08:00 |
|
|
fzyzcjy
|
2337b1bbb0
|
Refactor and fix prefill delayer (scheduler enhancer) (#16269)
|
2026-01-04 07:15:06 +08:00 |
|
|
fzyzcjy
|
877c8e3a96
|
Tiny refactor router test contexts (#16340)
|
2026-01-03 09:44:10 -08:00 |
|
|
fzyzcjyandgemini-code-assist[bot]
|
66dfb8c156
|
Tiny fix non-PD router http header missing whitelist (#16339)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-01-03 09:43:02 -08:00 |
|
|
fzyzcjy
|
87ef05e2e1
|
Support simple schedule simulator (#16344)
|
2026-01-03 22:05:40 +08:00 |
|
|
fzyzcjy
|
b65c9889a0
|
Fix incorrect running batch size in prefill stats (#15941)
|
2026-01-03 20:49:54 +08:00 |
|
|
fzyzcjy
|
c7c0d97fc6
|
Tiny support whitelisted headers in request logging (#16342)
|
2026-01-03 20:10:58 +08:00 |
|
|
fzyzcjy
|
7f2fa2167b
|
Tiny add --log-requests-target (#16338)
|
2026-01-03 17:28:27 +08:00 |
|
|
fzyzcjy
|
9821fae5c6
|
Tiny add explanations to realtime token metric labels (#16009)
|
2026-01-03 12:36:55 +08:00 |
|
|
fzyzcjy
|
749736ba95
|
Tiny fix single-gpu dumper and add tests for dumper (#16285)
|
2026-01-02 13:30:09 +08:00 |
|
|
fzyzcjy
|
00562ee14a
|
Support cache eviction for Manual Policy (#16263)
|
2026-01-02 08:52:01 +08:00 |
|
|
fzyzcjy
|
db499e1889
|
Tiny add filter, support duplications, add visualizations, fix error and robustness for dump comparator (#16262)
|
2026-01-01 17:47:00 +08:00 |
|
|
fzyzcjy
|
6cf3a6dd69
|
Support HTTP control for dumper (#16261)
|
2026-01-01 17:40:17 +08:00 |
|
|
fzyzcjy
|
90e24f5c31
|
Tiny add filter, dump dict, failable save to dumper (#16260)
|
2026-01-01 17:35:38 +08:00 |
|
|
fzyzcjy
|
d90f9bfc4e
|
Temporarily disable temp_prefill_info assertion to unblock CI (#16008)
|
2025-12-28 20:13:06 +08:00 |
|
|
fzyzcjy
|
8fab48952e
|
Tiny fix WASM test errors on machines with many cores (#15992)
|
2025-12-28 17:12:04 +08:00 |
|
|
fzyzcjy
|
7ccaec6467
|
Add micro benchmarks for manual policy (#15991)
|
2025-12-28 17:11:56 +08:00 |
|
|
fzyzcjy
|
26c5091217
|
Tiny extract PeriodicTask in router (#15988)
|
2025-12-28 14:07:13 +08:00 |
|
|
fzyzcjy
|
325a4c1945
|
Tiny add smg_manual_policy_cache_entries metric (#15987)
|
2025-12-28 14:07:07 +08:00 |
|
|
fzyzcjy
|
f55d608c89
|
Tiny fix cannot launch nvfp4 checkpoint with bf16 kv cache (#15986)
|
2025-12-28 11:22:20 +08:00 |
|
|
fzyzcjy
|
5d421db883
|
Tiny log warn users when tracing is automatically disabled (#15889)
|
2025-12-26 19:00:37 +08:00 |
|
|
fzyzcjy
|
0271fc3456
|
Fix prefill num tokens metrics (#15858)
|
2025-12-26 10:26:42 +08:00 |
|
|
fzyzcjy
|
68bece8caa
|
Super tiny move last_prefill_tokens to metrics mixin (#15857)
|
2025-12-26 10:26:10 +08:00 |
|
|
fzyzcjy
|
6ef543f99d
|
Use X-SMG-Routing-Key header instead of json body and add tests (#15826)
|
2025-12-25 09:58:14 -08:00 |
|
|
fzyzcjy
|
c12567274a
|
Tiny change bench-serving to use routing key header (#15827)
|
2025-12-25 21:29:49 +08:00 |
|
|
fzyzcjy
|
ccf2330b28
|
Tiny fix missing record_router_upstream_response (#15811)
|
2025-12-25 15:20:43 +08:00 |
|
|
fzyzcjy
|
45adad37d0
|
Add manual routing policy for router (#15586)
|
2025-12-25 11:27:36 +08:00 |
|
|
fzyzcjy
|
1ba897f330
|
Tiny refactor select_workers API for future passing more information (#15596)
|
2025-12-25 10:31:10 +08:00 |
|
|
fzyzcjy
|
9338f63f7d
|
Add request counter in addition to existing response counter (#15768)
|
2025-12-24 22:22:38 +08:00 |
|
|
fzyzcjy
|
8196998a03
|
Tiny add num retracted tokens metric (#15653)
|
2025-12-24 22:15:23 +08:00 |
|
|
fzyzcjy
|
aa21c6e3f1
|
Super tiny code cleanup (#15652)
|
2025-12-24 22:15:00 +08:00 |
|
|
fzyzcjy
|
fd4a558e71
|
Add metrics for having prefill and decode in different ranks (#15752)
|
2025-12-24 21:35:35 +08:00 |
|
|
fzyzcjy
|
b3b818fd86
|
Tiny unify realtime_tokens_total metric (#15747)
|
2025-12-24 21:29:43 +08:00 |
|
|
fzyzcjy
|
e245cac0c5
|
Support JSON format request logging for easier parsing (#15743)
|
2025-12-24 19:52:43 +08:00 |
|
|
fzyzcjy
|
c6a6ba4368
|
Tiny refactor request logger (#15740)
|
2025-12-24 19:51:26 +08:00 |
|
|
fzyzcjy
|
d6108166bc
|
Fix smg_http_requests_total semantics (#15655)
|
2025-12-24 17:41:01 +08:00 |
|
|
fzyzcjy
|
99d3bcdfed
|
[model-gateway] add back router worker health metric and fix init state (#15622)
|
2025-12-23 18:50:38 -08:00 |
|
|
fzyzcjy
|
4d64f15086
|
[mode;-gateway] add back fixes of incorrect metrics after worker removal (#15624)
|
2025-12-23 18:50:03 -08:00 |
|
|
fzyzcjy
|
6a5764a719
|
Super tiny add test_soft_watchdog to nightly (#15692)
|
2025-12-23 22:46:15 +08:00 |
|
|
fzyzcjy
|
b736a1525a
|
Tiny add back missing router per attempt response metric (#15621)
|
2025-12-22 09:59:10 -08:00 |
|
|
fzyzcjy
|
6984837d82
|
Fix router gRPC mode launch error caused by async loading (#15368)
|
2025-12-22 09:39:01 -08:00 |
|
|
fzyzcjy
|
e62e17442e
|
Tiny enable soft watchdog in CI for stuck without logs (#15616)
|
2025-12-22 17:01:04 +08:00 |
|
|
fzyzcjy
|
d5431ff894
|
Tiny add stuck simulation (#15613)
|
2025-12-22 17:00:18 +08:00 |
|
|
fzyzcjy
|
454a2544f2
|
Support soft watchdog for tokenizer/detokenizer/dp-controller processes (#15607)
|
2025-12-22 16:59:16 +08:00 |
|
|
fzyzcjy
|
89619a9904
|
Tiny avoid EnvField misuse (#15612)
|
2025-12-22 16:20:47 +08:00 |
|
|
fzyzcjy
|
ad7f35fb9d
|
Support gsp send routing id in bench serving (#15588)
|
2025-12-22 11:32:24 +08:00 |
|
|
fzyzcjy
|
f4100732b8
|
Tiny fix bench serving GSP mode cache file strategy (#15587)
|
2025-12-22 11:27:55 +08:00 |
|
|
fzyzcjy
|
88a405cc10
|
Support EPLB balancedness prometheus metric without GPU->CPU synchronize (#15401)
|
2025-12-18 22:24:23 +08:00 |
|
|
fzyzcjy
|
602fe3b296
|
Super tiny add moe_ep_rank to prometheus labels (#15407)
|
2025-12-18 22:23:20 +08:00 |
|
|
fzyzcjy
|
ad9616f13a
|
Tiny extract ModelRunnerOutput (#15400)
|
2025-12-18 22:18:45 +08:00 |
|
|
fzyzcjy
|
c5f4e20f2f
|
Support GPU execution time breakdown by forward mode metrics (#15396)
|
2025-12-18 22:18:17 +08:00 |
|
|
fzyzcjyandgemini-code-assist[bot]
|
2c196f95a7
|
Tiny fix unknown route in prometheus metrics (#15404)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-12-18 22:17:46 +08:00 |
|
|
fzyzcjy
|
c4aed389da
|
Super tiny rename failure_count for consistency (#15186)
|
2025-12-17 21:21:07 +08:00 |
|
|
fzyzcjy
|
9d04b570bc
|
Tiny dump native stacktraces in watchdog (#15222)
|
2025-12-17 21:20:41 +08:00 |
|
|
fzyzcjy
|
3e690cce53
|
Add realtime token counter metrics (#15198)
|
2025-12-17 21:20:24 +08:00 |
|
|
fzyzcjy
|
b12b40dea4
|
Add cuda_graph_forward_passes_total and num_retracted_reqs_total (#15189)
|
2025-12-17 21:19:42 +08:00 |
|
|
fzyzcjy
|
89ad390843
|
Fix num running requests (load) wrong cleared for ongoing requests (#15116)
|
2025-12-15 17:40:04 +08:00 |
|
|
fzyzcjy
|
62b3fdae43
|
Fix cache aware wrong routing caused by incorrect load tracking (#15101)
|
2025-12-15 10:03:54 +08:00 |
|
|
fzyzcjyandSimo Lin
|
4449c17011
|
[model-gateway] fix circuit breaker metrics (#15099)
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
|
2025-12-14 15:43:04 -08:00 |
|
|
fzyzcjy
|
5ca962ce7f
|
[model-gateway] extract circuit breaker state struct (#15098)
|
2025-12-14 15:40:23 -08:00 |
|
|
fzyzcjy
|
54df514bac
|
Avoid confusing zero value metric when worker is removed (#15096)
|
2025-12-14 13:57:20 +08:00 |
|
|
fzyzcjyandgemini-code-assist[bot]
|
681c68cfbf
|
Fix issue not reported when load decrement is incorrect (#15061)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-12-14 13:17:35 +08:00 |
|
|
fzyzcjy
|
168a31eb00
|
Support prefill max requests limitation (#14993)
|
2025-12-14 10:52:10 +08:00 |
|
|
fzyzcjy
|
fdc93b019c
|
Add sglang:decode_sum_seq_lens metric (#15066)
|
2025-12-14 08:42:45 +08:00 |
|
|
fzyzcjy
|
993278b488
|
Fix double decrease load (#15060)
|
2025-12-13 22:12:35 +08:00 |
|
|
fzyzcjy
|
9e9d910744
|
Fix load metric not updated when using guard (#15059)
|
2025-12-13 22:11:52 +08:00 |
|
|
fzyzcjy
|
bf17e769fe
|
Add sgl_router_attempt_http_responses_total for single attempt information (#15037)
|
2025-12-13 13:29:18 +08:00 |
|
|
fzyzcjy
|
9a5d6a84ab
|
Add error code in prometheus metrics and add X-SMG-Error-Code header (#15036)
|
2025-12-13 13:28:32 +08:00 |
|
|
fzyzcjy
|
31c23e5fe3
|
Provide more fine grained error reason for reqwest error (#15032)
|
2025-12-13 13:27:06 +08:00 |
|
|
fzyzcjy
|
06617a9ec8
|
Tiny change http router response format to unify (#15031)
|
2025-12-13 13:25:45 +08:00 |
|
|
fzyzcjy
|
e79ca95961
|
Tiny unify grpc existing error responses into new format (#15030)
|
2025-12-13 13:25:10 +08:00 |
|
|
fzyzcjy
|
9d3b411c56
|
Add code field and unify error responses for router (#15028)
|
2025-12-13 13:21:34 +08:00 |
|
|
fzyzcjy
|
05325db349
|
Super tiny remove unused log_request (#15035)
|
2025-12-13 13:21:24 +08:00 |
|
|
fzyzcjy
|
313f59ad80
|
Add soft watchdogs to debug soft hangs (#15023)
|
2025-12-13 10:41:35 +08:00 |
|
|
fzyzcjy
|
487cf81a68
|
Tiny extract SchedulerWatchdog (#15021)
|
2025-12-13 10:40:32 +08:00 |
|
|
fzyzcjy
|
df111bc0fe
|
Super tiny add gsp-fast-prepare (#14992)
|
2025-12-13 09:45:21 +08:00 |
|
|
fzyzcjy
|
6d2b3324ef
|
Super tiny fix confusing slash_command_handler hint (#14976)
|
2025-12-13 09:42:50 +08:00 |
|
|
fzyzcjy
|
8cc77261ec
|
Super tiny remove unused argument (#14966)
|
2025-12-13 09:41:01 +08:00 |
|
|
fzyzcjy
|
ec242f516e
|
Super tiny extract route_typed_request_once (#14951)
|
2025-12-12 11:10:10 -08:00 |
|
|
fzyzcjy
|
edb172e927
|
Super tiny refactor error.rs logic (#14949)
|
2025-12-12 14:48:18 +08:00 |
|
|
fzyzcjy
|
a7fa31ff17
|
Super tiny move error.rs (#14944)
|
2025-12-12 14:25:30 +08:00 |
|
|
fzyzcjy
|
bd91f8824a
|
Super tiny remove non-updated sgl_router_worker_load (#14888)
|
2025-12-12 13:19:30 +08:00 |
|
|
fzyzcjy
|
b47adb807e
|
Tiny add e2e http request arrival metric (#14893)
|
2025-12-12 13:18:23 +08:00 |
|
|
fzyzcjy
|
b2b5bdb06e
|
Tiny add router e2e duration histogram (#14892)
|
2025-12-12 13:18:13 +08:00 |
|
|
fzyzcjy
|
e4873d0439
|
Super tiny remove sgl_router_active_workers (#14891)
|
2025-12-11 20:00:55 -08:00 |
|
|
fzyzcjy
|
ba88f1cace
|
Tiny clean router load report logic (#14889)
|
2025-12-11 06:50:32 -08:00 |
|
|
fzyzcjy
|
60560c07a2
|
[model-gateway] refactor cleanup WorkflowContext.get_or_err (#14890)
|
2025-12-11 06:48:30 -08:00 |
|
|
fzyzcjy
|
45eeeb9a3c
|
Tiny support range ratio in GSP in bench serving (#14828)
|
2025-12-11 17:26:07 +08:00 |
|
|
fzyzcjy
|
a368df2818
|
Tiny add more error info for bench_serving (#14827)
|
2025-12-11 17:25:49 +08:00 |
|
|
fzyzcjy
|
3d82c0f17f
|
[model-gateway] support engine response http status statistics in router (#14712)
|
2025-12-10 08:30:53 -08:00 |
|
|
fzyzcjy
|
617e9b3bf8
|
[model-gateway] support customizing Prometheus duration buckets (#14716)
|
2025-12-10 08:30:03 -08:00 |
|
|
fzyzcjy
|
d85fecb536
|
Tiny fix incorrect worker removal command (#14822)
|
2025-12-10 22:29:39 +08:00 |
|
|
fzyzcjy
|
6634f67bd5
|
Fix router keep nonzero metrics after worker is deleted (#14819)
|
2025-12-10 22:29:30 +08:00 |
|
|
fzyzcjy
|
5eccaf7737
|
Support HTTP response status code prometheus metrics (#14710)
|
2025-12-10 22:28:57 +08:00 |
|
|
fzyzcjy
|
da3dc497b0
|
Fix dp-aware incompatible with completions and chat completions APIs (#14647)
|
2025-12-09 13:24:53 +08:00 |
|
|
fzyzcjy
|
817daba062
|
Tiny extract select_worker_min_load (#14648)
|
2025-12-08 21:22:10 -08:00 |
|
|
fzyzcjy
|
af20657cd4
|
Tiny support sgl-router http response status code metrics (#14689)
|
2025-12-08 19:54:35 -08:00 |
|
|
fzyzcjy
|
119fd956fb
|
Tiny support printing requests in bench_serving for observability (#14652)
|
2025-12-09 08:27:58 +08:00 |
|