 ishandhananiandKangyan-Zhou
|
d5fdab7022
|
chore: add NIXL owners and CI access (#40602)
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
|
2026-09-21 12:21:30 -07:00 |
|
|
ishandhanani
|
3f871a246c
|
feat(agent sessions): attribute stored KV cache blocks to sessions (#37482)
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
|
2026-09-14 21:34:49 -07:00 |
|
|
ishandhanani
|
ddc1df1203
|
fix(router): simplify bucket context limit check (#39259)
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
|
2026-09-13 12:27:59 -07:00 |
|
|
ishandhanani
|
cf51650335
|
fix(config): retain pre-engine resolution declarations (#37195)
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
|
2026-08-31 12:24:29 -07:00 |
|
|
ishandhanani
|
4d7d2ebb44
|
[Docker] Install AI Dynamo nightly (#36946)
|
2026-08-28 23:27:52 -07:00 |
|
|
ishandhananiandShangming Cai
|
978244d671
|
[P/D disagg] Decode-side radix cache for SWA hybrid models (unified radix tree) (#27770)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-08-21 09:25:06 +08:00 |
|
|
ishandhananiandAlex Nails
|
0f744b6848
|
feat: make mm_inputs msgpack-native (#29656)
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
|
2026-08-20 14:30:07 -07:00 |
|
|
ishandhanani
|
c84ddc0e76
|
docker: pin nightly image source to workflow commit (#33738)
|
2026-08-05 23:26:18 -07:00 |
|
|
ishandhananiandConnor Carpenter
|
21065bc862
|
feat: add native gRPC sidecar module launcher (#31076)
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: Connor Carpenter <connorc@nvidia.com>
|
2026-07-22 06:39:08 -07:00 |
|
|
ishandhanani
|
92fc692411
|
fix: include OpenSSL headers in runtime image (#31064)
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
|
2026-07-13 13:09:41 -07:00 |
|
|
|
b6907d9664
|
docker: install dynamo nightly in the dev image for rapid iteration/testing (#29634)
Co-authored-by: Kangyan Zhou <zky314343421@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-06-29 22:58:19 -07:00 |
|
|
ishandhanani
|
aaa31eb0a1
|
feat: first-class session identity in SGLang (#29436)
|
2026-06-28 07:16:38 -07:00 |
|
|
|
b030b1a5f3
|
hisparse: support NIXL DRAM KV destinations for HiSparse (#27563)
Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-06-27 22:32:29 +08:00 |
|
|
ishandhanani
|
e67b228d4c
|
feat: session radix cache (#27058)
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
|
2026-06-23 01:51:41 -07:00 |
|
|
ishandhanani
|
978fb6ed1a
|
hicache kv events: publish split write-through fragments (#27072)
|
2026-06-03 14:52:25 -07:00 |
|
|
ishandhananiandShangming Cai
|
87c3c96bc8
|
[Bug][PD][NIXL] always send aux on is_last; only expects_state when truthy (#25699)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-05-19 10:04:04 +08:00 |
|
|
ishandhanani
|
5b7ce417d0
|
[P/D disagg] - support decode side radix cache (#19746)
|
2026-05-01 21:55:34 +08:00 |
|
|
ishandhananiandBaizhou Zhang
|
0d224e5053
|
update: b300 container for dsv4 (#23697)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2026-04-24 22:59:20 -07:00 |
|
|
ishandhananiandjthomson04
|
3c007ee5d4
|
fix(hicache): emit KV events for L2 host cache insertions (#22894)
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Co-authored-by: jthomson04 <jwillthomson19@gmail.com>
|
2026-04-20 19:07:03 -07:00 |
|
|
ishandhanani
|
6f6843c582
|
[Docker] Move Rust toolchain install to torch_deps stage (#23278)
|
2026-04-20 13:13:10 -07:00 |
|
|
ishandhananiandClaude Opus 4.7
|
90d527195b
|
[CI] Fix nightly docker builds failing on root-owned workspace leftovers (#23279)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-04-20 11:56:33 -07:00 |
|
|
ishandhanani
|
b5d9a86e4c
|
fix: add back priorty as radix cache policy (#23275)
|
2026-04-20 10:04:35 -07:00 |
|
|
ishandhananiandClaude Opus 4.6
|
761259448d
|
ci: re-enable fp8 nightly benchmark configs (#22910)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-04-15 15:57:49 -07:00 |
|
|
ishandhananiandClaude Opus 4.6
|
2b0f349927
|
ci: clarify srt-slurm issue filing for incompatible flag combos (#22903)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-04-15 15:02:26 -07:00 |
|
|
ishandhananiandClaude Opus 4.6
|
9497001b0c
|
ci: add issue filing and suspect PR identification to log analyzer (#22899)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-04-15 14:27:14 -07:00 |
|
|
ishandhanani
|
f61c332cba
|
ci: log analyzer (#22859)
|
2026-04-15 14:10:00 -07:00 |
|
|
ishandhanani
|
2c9e76d333
|
ci: skip approval for nightly gb200 runs, keep for manual triggers (#22768)
|
2026-04-14 16:34:57 -07:00 |
|
|
ishandhanani
|
520ce526b9
|
Restore Qwen3 rope config fallback (#22739)
|
2026-04-13 21:47:37 -07:00 |
|
|
ishandhanani
|
cc449ac4e5
|
feat(metrics): expose raw KV cache pool token counts as prometheus gauges (#22726)
|
2026-04-13 18:30:36 -07:00 |
|
|
ishandhanani
|
6b2bf66cd9
|
fix[glm4.7 flash]: properly detect gfx95_quant_format (#22720)
|
2026-04-13 13:10:07 -07:00 |
|
|
|
c1ab68b45e
|
fix: streaming session race condition + some metrics (#21875)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
|
2026-04-12 18:05:23 -07:00 |
|
|
ishandhanani
|
3aaaf53f59
|
[Docker] Fix CI docker target after Dockerfile restructure (#22478)
|
2026-04-09 18:53:42 -07:00 |
|
|
ishandhanani
|
aa103eab8d
|
[Docker] Optimize Dockerfile for BuildKit layer caching (#22160)
|
2026-04-09 15:34:57 -07:00 |
|
|
ishandhanani
|
1081a25983
|
revert: remove TTL-based hard pin from HiRadixCache (#21884)
|
2026-04-01 16:51:15 -07:00 |
|
|
|
8f0f36c64b
|
[1/2] Add ModelExpress coordination for remote instance weight loading - matching TP (#19920)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Ishan Dhanani <ishan@dhanani.dev>
|
2026-03-18 13:38:32 -07:00 |
|
|
ishandhananiandClaude
|
f7da379b61
|
feat: TTL-based prefix pinning with refresh-on-hit for HiRadixCache (#18941)
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-03-01 23:27:22 -08:00 |
|
|
ishandhanani
|
6a1480ce45
|
Fix HiCacheNixl TypeError: mem_pool_host passed as file_path (#19517)
|
2026-02-27 10:59:32 -08:00 |
|
|
ishandhanani
|
01e3f4682e
|
feat(kv-events): Add medium field to KV event types for storage tier tracking (#18205)
|
2026-02-09 12:39:15 -08:00 |
|
|
ishandhanani
|
8f8c1724ae
|
docker: add patch to increase GPU deepep timeout (#18298)
|
2026-02-05 18:26:15 +08:00 |
|
|
ishandhanani
|
0a6925639b
|
ci: improve docker for cu13 builds (#18194)
|
2026-02-03 11:09:38 -08:00 |
|
|
ishandhanani
|
820df545f2
|
fix: add cu13 dev container to our release (#18192)
|
2026-02-03 10:42:05 -08:00 |
|
|
ishandhanani
|
1e309030e3
|
update urllib3 and gpgv Dockerfile (#17439)
|
2026-01-20 14:47:20 -08:00 |
|
|
ishandhanani
|
0500fea965
|
fix editable install (#16241)
|
2025-12-31 14:34:54 -08:00 |
|
|
ishandhanani
|
0e4d8790d7
|
feat(ci): add framework target to release-docker workflows (#14559)
|
2025-12-07 13:25:24 +08:00 |
|
|
ishandhanani
|
498ea41ca6
|
dockerfile: add runtime stage + ubuntu 24.04 (#13861)
|
2025-12-05 00:28:36 -08:00 |
|
|
ishandhanani
|
07821352fb
|
Revert "Skip weight loading in deepgemm compilation" (#14241)
|
2025-12-01 12:59:09 -08:00 |
|
|
ishandhanani
|
b537ac0d1f
|
Fix ZMQ bind error on non-zero rank nodes when using SGLANG_BLOCK_NONZERO_RANK_CHILDREN=0 (#13686)
|
2025-11-20 22:54:41 -08:00 |
|
|
ishandhanani
|
1a5c313f97
|
feat(engine): add rid parameter to methods in Engine class (#13095)
|
2025-11-11 22:37:54 -08:00 |
|
|
ishandhanani
|
885cfca273
|
ci: try to fix gpg error during kernel build (#12928)
|
2025-11-09 11:45:29 -08:00 |
|
|
ishandhanani
|
93be7e863e
|
fix: respect --ignore-eos in PD case for benchmarking (#12597)
|
2025-11-03 21:44:14 -08:00 |
|
|
ishandhanani
|
263eab9f5d
|
fix: dummy health check server not accessible on non-zero rank nodes (#12297)
|
2025-10-31 11:34:57 -07:00 |
|
|
ishandhananiandBaizhou Zhang
|
285a8e6986
|
docker: add CUDA13 support in dockerfile and update GDRCopy/NVSHMEM for blackwell support (#11517)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2025-10-27 22:00:54 -07:00 |
|
|
ishandhanani
|
cadfae666d
|
fix broken deepep/flashmla install in container by adding --no-build-isolation (#12170)
|
2025-10-26 16:39:51 -07:00 |
|
|
ishandhanani
|
3865afc5df
|
fix: release workflow should work on both archs (#12110)
|
2025-10-24 23:22:31 -07:00 |
|
|
ishandhanani
|
14203432b4
|
fix(compile_utils, ep_moe): update environment variable and dtype check (#12034)
|
2025-10-24 11:00:12 -07:00 |
|
|
ishandhanani
|
296f689242
|
fix(server_args): handle tokenizer init conflicts (#11776)
|
2025-10-20 00:27:19 -07:00 |
|
|
ishandhanani
|
73ea484af1
|
docker: add manifest to versioned docker releases (#11268)
|
2025-10-06 14:53:40 -07:00 |
|
|
ishandhanani
|
47488cc353
|
docker: x86 dev builds for hopper and blackwell (#11075)
|
2025-10-01 00:06:38 -07:00 |
|
|
ishandhananiandzhyncs
|
adba172fd1
|
ci: free space on workers for build (#10786)
Co-authored-by: zhyncs <me@zhyncs.com>
|
2025-09-24 02:58:22 -07:00 |
|
|
ishandhanani
|
b06db198ba
|
followup: clean up dockerfiles and release yamls (#10783)
|
2025-09-23 00:19:46 -07:00 |
|
|
ishandhananiandYineng Zhang
|
1c82d9db28
|
feat: unify dockerfiles (#10705)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-09-22 23:23:48 -07:00 |
|
|
ishandhanani
|
662393f27d
|
fix: kv events with tp > 1 (#10541)
|
2025-09-22 15:55:44 -07:00 |
|
|
ishandhanani
|
148022fc36
|
gb200: update dockerfile to latest kernel (#9522)
|
2025-09-08 17:32:36 -07:00 |
|
|
ishandhanani
|
e52c3866eb
|
chore(docker): update sgl_kernel version to 0.3.6 in Dockerfile.gb200 (#9243)
|
2025-08-15 12:06:52 -07:00 |
|
|
ishandhanani
|
1f9ec65374
|
fix(docker): update sgl_kernel version to 0.3.4 in Dockerfile.gb200 (#9118)
|
2025-08-12 13:12:33 -07:00 |
|
|
ishandhanani
|
9f24dfefd1
|
chore(gb200): remove ToT flashinfer installation (#9079)
|
2025-08-11 11:02:15 -07:00 |
|
|
ishandhanani
|
de8b8b6e5c
|
chore(deps): update minimum python to 3.10 (#8984)
|
2025-08-09 00:30:23 -07:00 |
|
|
ishandhanani
|
7d3af603e7
|
chore(ci): update Python version from 3.9 to 3.10 in sgl-kernel workflow (#8981)
|
2025-08-08 14:03:17 -07:00 |
|
|
ishandhanani
|
4e7f025219
|
chore(gb200): update to CUDA 12.9 and improve build process (#8772)
|
2025-08-08 13:42:47 -07:00 |
|
|
ishandhanani
|
b27b11919c
|
chore(gb200): update dockerfile to handle fp4 disaggregation (#8694)
|
2025-08-01 18:58:00 -07:00 |
|
|
ishandhanani
|
31fccf5a4f
|
chore: change logs fromINFO to DEBUG for dp and add force quit for tokenizer manager (#7251)
|
2025-06-18 01:36:43 -07:00 |
|
|
ishandhanani
|
f1569876d5
|
feat: add direct routing strategy to DP worker (#6884)
|
2025-06-09 11:44:05 -07:00 |
|
|
ishandhanani
|
f0f84975f4
|
feat: add dp-rank to KV events (#6852)
|
2025-06-04 15:29:34 -07:00 |
|
|
ishandhanani
|
b6909aa223
|
fix: allow launch_dummy_health_check_server to start inside of running asyncio loop (#6330)
|
2025-05-17 18:32:41 -07:00 |
|
|
ishandhanani
|
e444c13fb4
|
feat(engine): add bootstrap parameters to generate methods (dynamo) (#6075)
|
2025-05-07 10:33:58 -07:00 |
|