Commit Graph

  • aa23c29d52 deploy(b300/dynamo): publish default thinking mode from the workers b300/dynamo minke.yu 2026-11-17 22:55:08 +08:00
  • 99603451b9 docs(b300/dynamo): record tokenizer/chat-processor fixes and first-run results minke.yu 2026-11-17 22:42:19 +08:00
  • 81428f230c deploy(b300/dynamo): restore disaggregation lanes and mount frontend patches minke.yu 2026-11-17 22:29:38 +08:00
  • 2372edd73d observability: skip batch-span links for TraceNullContext requests minke.yu 2026-11-17 22:27:55 +08:00
  • 34d2224b7f dynamo: add encoding_dsv4 shim for the frontend SGLang chat processor minke.yu 2026-11-17 22:22:18 +08:00
  • 36f6a9cdb3 fix(b300/dynamo): restore FE_PORT definition minke.yu 2026-11-17 22:11:31 +08:00
  • 2b75283daa deploy(b300/dynamo): run frontend chat processing on SGLang processor minke.yu 2026-11-17 22:10:57 +08:00
  • 0fbd87d739 docs(b300/dynamo): Dynamo does not remove SGLang's bootstrap/prealloc tail minke.yu 2026-11-17 21:51:15 +08:00
  • bcfdd8a1f7 docs(b300/dynamo): spell out Dynamo PD control plane vs SGLang KV data plane minke.yu 2026-11-17 21:42:19 +08:00
  • 6f546386ca deploy(b300/dynamo): add Dynamo 1.6.2 image recipe and 4P4D PD startup minke.yu 2026-11-17 21:01:52 +08:00
  • 8b67636010 Merge b300/integration into main: 2026-11-17 c580 baseline main minke.yu 2026-11-17 20:08:08 +08:00
  • 310ad35430 deploy: record 2026-11-17 c580 baseline (D16+fastokens) and experiments b300/integration minke.yu 2026-11-17 20:05:11 +08:00
  • bdbd8dffb8 deploy(ds41-gist-exp): P buffer_only + optimistic, D tokenizer 16 minke.yu 2026-11-16 22:03:20 +08:00
  • f363a6ab7b deploy: record 11/16 pre-optimistic stable config minke.yu 2026-11-16 22:03:11 +08:00
  • bc35fd859e chore: update CI test est_time from recent run data bot/update-est-time-20261116 sglang-bot 2026-11-16 00:00:30 +00:00
  • e4c7e540be deploy(g9i): T26 qps20 + P queue32 production candidate minke.yu 2026-11-16 00:42:44 +08:00
  • b8b205b367 deploy(g9i): merged ns+td gateway, 32-request P queue cap (p95 5.70s) minke.yu 2026-11-15 22:25:33 +08:00
  • a493fad77e gateway: flatten Chat Completions namespace tools to ns__function gw-ns-est-td-20261115 minke.yu 2026-11-14 14:11:10 +08:00
  • 38aff6c842 gateway: fix raw responses helper placement minke.yu 2026-11-14 13:10:51 +08:00
  • f60160bce6 gateway: forward /v1/responses raw bodies so unknown tool types pass through minke.yu 2026-11-14 13:03:58 +08:00
  • b3d6d19c7d deploy(g9i): set regular P queue reject 32s (experiment) minke.yu 2026-11-15 21:45:51 +08:00
  • 8f53c1fb73 deploy(l3ssd): offload_on_evict=false, 4TB cap, L3 reset script minke.yu 2026-11-15 20:33:38 +08:00
  • 42f228bfb8 docs(deploy): fresh-instance multi-round tokenizer A/B (B2 TTFT p95 < 5s) minke.yu 2026-11-15 18:20:53 +08:00
  • 242a7ad23a docs(deploy): record L3 SSD tokenizer A/B (fastokens vs huggingface) minke.yu 2026-11-15 15:49:56 +08:00
  • 2eb163ad73 fix(deploy): point sglang cache at the shared warm cache minke.yu 2026-11-15 14:23:52 +08:00
  • d3240c745e deploy(b300): L3 Mooncake SSD baseline with mooncake 0.3.14 minke.yu 2026-11-15 14:10:30 +08:00
  • 433257c7b0 fix(recipe): drop the pre-48928 _fast_prompt_tokenizer fallback stub minke.yu 2026-11-15 10:53:15 +08:00
  • 7980a09c0a [Perf] DeepSeek-V4.1-Flash: Use segmented fastokens for prompt encoding (#48928) leon 2026-11-11 01:33:02 +08:00
  • ce1b587fb9 gateway: flatten Chat Completions namespace tools to ns__function gw-ns-g9i-20261114 minke.yu 2026-11-14 14:11:10 +08:00
  • a7c668a936 fix(chat): move namespace-tool flattening validator into ChatCompletionRequest minke.yu 2026-11-14 14:10:07 +08:00
  • 0c6849e378 chat: flatten namespace tools to ns__function for Chat Completions minke.yu 2026-11-14 14:09:36 +08:00
  • e879bd0e07 gateway: fix raw responses helper placement minke.yu 2026-11-14 13:10:51 +08:00
  • 996db4b60e gateway: forward /v1/responses raw bodies so unknown tool types pass through minke.yu 2026-11-14 13:03:58 +08:00
  • 85605b8c72 fix(protocol): ResponseTool accepts extra fields (tool_search execution) minke.yu 2026-11-13 23:25:50 +08:00
  • b6a5d70cba fix(responses): tolerate tools without model_extra in native validation minke.yu 2026-11-13 23:24:00 +08:00
  • cac60f7c2e fix(protocol): effective_tool_choice preserves namespace/type minke.yu 2026-11-13 23:22:21 +08:00
  • 68c2af7957 fix(protocol): Function.strict is Optional[bool] for recipe tool schemas minke.yu 2026-11-13 23:21:04 +08:00
  • 497aeff5f6 fix(recipe): define _fast_prompt_tokenizer fallback on this base minke.yu 2026-11-13 22:44:35 +08:00
  • 5fa8c23227 vendor: DeepSeek-recipe namespace tools (#46370 + #49625) minke.yu 2026-11-13 22:43:37 +08:00
  • f4de9f2141 observability: ignore sub-tolerance CPU>wall skew in accounting anomaly b300-pick-46884-async-20261107 minke.yu 2026-11-12 12:30:58 +08:00
  • 7f42a0806a observability: per-rank request counter for P/D RPS minke.yu 2026-11-12 10:30:56 +08:00
  • 17b922c84f test: prefill_batch_init detail and eviction semantics minke.yu 2026-11-12 10:30:47 +08:00
  • 5ad9ff6a5f observability: prefill_batch_init wall+CPU, sub-phases and eviction coverage minke.yu 2026-11-12 10:30:38 +08:00
  • fec10023e0 observability: optional thread-CPU recording for detail phases minke.yu 2026-11-12 10:30:29 +08:00
  • 9f9dce1ff6 test: restore _emit_pass_span residual default minke.yu 2026-11-12 10:30:17 +08:00
  • c401c614fa fix(dp): rank uncached candidates by published backlog only minke.yu 2026-11-11 02:06:59 +08:00
  • 24b284d4e7 feat(dp): uncached_tokens load-balance method minke.yu 2026-11-10 23:58:08 +08:00
  • fef023b64f [Fix] scheduler: Count all pending work in DP load snapshots QNan0513 2026-10-16 01:01:17 +08:00
  • 15218b2f65 deploy: bake the live hostPath overrides into the tree ymk 2026-11-09 06:31:48 +00:00
  • 883f7d08e1 fix(mooncake): use enable_client_http_server (not enable_http_server) minke.yu 2026-11-09 11:04:01 +08:00
  • e40c6dfe83 observability(mooncake): opt-in client metrics HTTP server per rank minke.yu 2026-11-09 10:55:17 +08:00
  • 606cf41603 chore: update CI test est_time from recent run data bot/update-est-time-20261109 sglang-bot 2026-11-09 00:00:30 +00:00
  • 32e5e0f30f observability: GPU-window probe + forward/prefill sub-phase timing minke.yu 2026-11-08 01:19:09 +08:00
  • a5fbe834e8 observability: split get_next residual into its four phases minke.yu 2026-11-08 00:20:55 +08:00
  • d6f1e1efbd rebase(hicache): per-op async backup on top of #46884 + #48782 minke.yu 2026-11-07 22:04:33 +08:00
  • 520ca548ea [Fix] hicache: Wait for overlapping write-through acknowledgements alphabetc1 2026-11-06 10:14:25 +08:00
  • 5f7be7fab4 [Perf] hicache: Publish prefill read-done when offload copies are fenced on the forward stream metamergebot 2026-10-27 12:53:17 -07:00
  • 97c087778b observability: harden flush span attributes minke.yu 2026-11-07 11:35:03 +08:00
  • 53267ee18f observability: time loop-other inflight poll and release/insert minke.yu 2026-11-07 09:26:46 +08:00
  • aa53670fcd observability: split flush_pending_backups host chain (off by default) minke.yu 2026-11-07 09:23:44 +08:00
  • 7b3f34cbc0 test: cover async SWA state-payload call chain; clarify CP gate ymk 2026-11-07 00:22:57 +08:00
  • 3b0baa3455 prefill: async SWA state-payload D2H (first version) ymk 2026-11-06 23:00:22 +08:00
  • fe9c6574fe observability: trace KV transfer chunks end-to-end (M2) ymk 2026-11-06 18:28:48 +08:00
  • 72e556ec81 observability: split prefill run_batch/process_result into host phases (M1) ymk 2026-11-06 18:28:26 +08:00
  • 23053730fc observability: count SWA gate rejections per prefill admission minke.yu 2026-11-06 10:54:28 +08:00
  • bf8cf7f1dc hicache: pipeline prefetch/backup IO lanes + stage timing ymk 2026-11-05 08:37:29 +00:00
  • c911d1b572 observability: expose SWA prefix-match gate rejections minke.yu 2026-11-06 10:33:52 +08:00
  • 145e05c44d observability: fix scheduler no-batch accounting minke.yu 2026-11-05 16:42:39 +08:00
  • 5d5870f3ea observability: add scheduler loop span and no-batch accounting minke.yu 2026-11-05 15:00:23 +08:00
  • 309af6e6e1 observability: record prefetch ready latency on the L3 hit success path minke.yu 2026-11-04 19:14:02 +08:00
  • 1a274df27e observability: fix prefetch ready latency to cover fast lookups minke.yu 2026-11-04 18:04:34 +08:00
  • cb7988f849 observability: measure storage prefetch ready latency (hit/miss) minke.yu 2026-11-04 17:26:41 +08:00
  • 0866012b80 observability: split scheduler recv_requests into pull/broadcast/unwrap minke.yu 2026-11-04 16:11:13 +08:00
  • 85f4f9576c scheduler: add opt-in per-pass prefill stage timing (SGLANG_BATCH_STAGE_TIMING) minke.yu 2026-11-04 14:48:46 +08:00
  • 2434d09848 observability: commit the in-image per-batch telemetry baseline (btelem4-6) minke.yu 2026-11-04 14:48:38 +08:00
  • d47a99b2ad test: import SWA eviction helper directly b300-pick-261102-v3 Florian Wunderlich 2026-10-23 09:44:06 +02:00
  • 2ba9b8431a Fix SWA eviction behind the replay-trimmed prefix frontier Florian Wunderlich 2026-10-23 09:39:49 +02:00
  • d5a998f346 test: import SWA eviction helper directly b300-pick-261102 Florian Wunderlich 2026-10-23 09:44:06 +02:00
  • a3401b3e65 Fix SWA eviction behind the replay-trimmed prefix frontier Florian Wunderlich 2026-10-23 09:39:49 +02:00
  • 01087e689c [DSv4.1] Carry decoder tail counts through DP scheduler synchronization Yangmin Li 2026-10-09 23:25:00 -07:00
  • 0b45c07637 perf: dsv41 decoder tail runs only the prompt's last window rows of each prefill chunk (opt-in) alphabetc1 2026-10-24 00:52:20 +08:00
  • 216880034a gateway: accept reasoning.effort=none and flat Responses tool_choice gw-schema-compat ymk 2026-11-02 06:13:02 +00:00
  • fa0fd7104e chore: update CI test est_time from recent run data bot/update-est-time-20261102 sglang-bot 2026-11-02 00:00:31 +00:00
  • ee9170e7bb fix(mooncake): bind IntraNode batches to the bootstrap CUDA device B300261102 gist-nvlink zicai.pu 2026-10-30 11:38:00 +08:00
  • 17505e6cf6 hicache: align prefetch and host restore boundaries zicai.pu 2026-10-30 11:37:01 +08:00
  • 81f7398855 fix(hicache): reject unproven trailing restore endpoints zicai.pu 2026-10-30 11:42:19 +08:00
  • 0a56a5f6e7 fix(hicache): preserve sparse restore checkpoints under host pressure zicai.pu 2026-10-30 11:35:29 +08:00
  • 7f6fe22445 mem_cache: fail-soft on SWA load-back gaps instead of asserting minke.yu 2026-10-26 13:23:17 +00:00
  • f6fc6ac3e7 mem_cache: allow disabling the DSv4.1 SWA prefix-match gate via env minke.yu 2026-10-26 04:06:25 +00:00
  • 2a75014390 chore: update CI test est_time from recent run data bot/update-est-time-20261026 sglang-bot 2026-10-26 00:00:28 +00:00
  • 0a3c370451 mooncake: probe pointers via cuda.bindings instead of ctypes libcudart minke.yu 2026-09-21 12:46:25 +08:00
  • b1b303c9f2 mooncake: support host-memory payloads with intra-node NVLink transport minke.yu 2026-09-21 11:53:26 +08:00
  • 7d996300b9 [Perf] dsv41: Use the fused ratio-2 compressor in the prefill graph (#46230) Liangsheng Yin 2026-10-23 21:33:42 -07:00
  • 54df642fe6 [Perf] dsv41: Translate SWA locations once per prefill graph step (#46229) Liangsheng Yin 2026-10-23 21:33:15 -07:00
  • 564aec61c8 [Perf] dsv41: Feed wo_b the fused MXFP8 attention output in the prefill graph (#46231) Liangsheng Yin 2026-10-23 21:31:33 -07:00
  • a7183a8246 [Feat] spec: Let out-of-tree platforms provide EAGLE draft graph runners (#45145) Yihao WangandClaude Opus 5.5 2026-10-23 21:28:56 -07:00
  • af3f9e4c8f [Feat] spec: Let out-of-tree platforms provide their own speculative worker (#45144) Yihao WangandClaude Opus 5.5 2026-10-23 21:24:58 -07:00
  • f09cec65e9 [Fix] diffusion/storage: Join uploads before request cleanup (#46068) MickandMick Qian 2026-10-24 12:19:10 +08:00
  • f4ecaae888 [Fix] diffusion/mesh: Guard asynchronous job publication and defaults (#46081) MickandMick Qian 2026-10-24 12:18:25 +08:00
  • cdab6c1bcc [Fix] diffusion/loader: Preserve checkpoints on read failures (#46095) MickandMick Qian 2026-10-24 12:17:46 +08:00