feat: add coordinated checkpoint prefetch for network filesystem loading (#20843)

2026-04-16 20:08:19 -07:00
parent a77abbe005
commit 04a53955b9
8 changed files with 265 additions and 6 deletions
@@ -20,6 +20,7 @@ SGLang supports various environment variables that can be used to configure its
 | `SGLANG_REQ_WAITING_TIMEOUT`              | Timeout (in seconds) for requests waiting in the queue before being scheduled                                                    | `-1`                         |
 | `SGLANG_REQ_RUNNING_TIMEOUT`              | Timeout (in seconds) for requests running in the decode batch                                                                    | `-1`                         |
 | `SGLANG_CACHE_DIR`                        | Cache directory for model weights and other data | `~/.cache/sglang` |
+| `SGLANG_PREFETCH_BLOCK_SIZE_MB`           | Block size (in MB) for sequential checkpoint prefetch reads that warm the OS page cache before workers load weights via mmap | `16` |

 ## Performance Tuning