deploy(b300/dynamo): publish default thinking mode from the workers
The AgentX baseline runs with thinking=true/high by default; the Dynamo frontend only applies a default when the worker publishes default_thinking_mode. Pass --dyn-default-thinking-mode enabled on both P and D so the SGLang chat processor matches the sglang baseline.
This commit is contained in:
@@ -118,6 +118,7 @@ docker run -d --name dyn-p --gpus all -e CUDA_VISIBLE_DEVICES=0,1,2,3 \
|
||||
--trust-remote-code --tokenizer-backend fastokens \
|
||||
--model-path "$MODEL" --served-model-name "$SERVED" \
|
||||
--reasoning-parser auto --tool-call-parser auto \
|
||||
--dyn-default-thinking-mode enabled \
|
||||
--default-chat-template-kwargs '{"thinking": true, "reasoning_effort": "high"}' \
|
||||
--tp 4 --dp-size 4 --ep-size 4 --enable-dp-attention --enable-dp-lm-head \
|
||||
--moe-a2a-backend megamoe --dsv4-attn-backend flashmla_mega \
|
||||
@@ -157,6 +158,7 @@ docker run -d --name dyn-d --gpus all -e CUDA_VISIBLE_DEVICES=4,5,6,7 \
|
||||
--trust-remote-code --tokenizer-backend fastokens \
|
||||
--model-path "$MODEL" --served-model-name "$SERVED" \
|
||||
--reasoning-parser auto --tool-call-parser auto \
|
||||
--dyn-default-thinking-mode enabled \
|
||||
--tp 4 --dp-size 4 --ep-size 4 --enable-dp-attention --enable-dp-lm-head \
|
||||
--enable-fp32-lm-head --enable-dp-attention-local-control-broadcast \
|
||||
--moe-a2a-backend megamoe --dsv4-attn-backend flashmla_mega \
|
||||
|
||||
Reference in New Issue
Block a user