Add SGLANG_CRASH_ON_JIT_COMPILE to forbid on-the-fly JIT compilation (#36615)
Signed-off-by: Shiyan Deng <dsy842974287@meta.com> Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
This commit is contained in:
co-authored by
Lianmin Zheng
parent
87d60a2229
commit
80e8302d03
@@ -1095,6 +1095,10 @@ class Envs:
|
||||
# is what makes reverting an edit an instant hit instead of a rebuild; set
|
||||
# it to trade that away for disk (1 keeps only the most recent build).
|
||||
SGLANG_JIT_CACHE_KEEP = EnvInt(None)
|
||||
# Raise instead of compiling when a module misses the cache, so a
|
||||
# deployment that expects a pre-seeded cache fails loudly at startup
|
||||
# rather than silently eating a cold compile.
|
||||
SGLANG_CRASH_ON_JIT_COMPILE = EnvBool(False)
|
||||
|
||||
# ===================================================================
|
||||
# Expert-parallel dispatch and MoE execution
|
||||
|
||||
Reference in New Issue
Block a user