@@ -151,7 +151,7 @@ After tuning, configuration files will be generated:
|
||||
- **Standard tuning**: `E=64,N=640,device_name=NVIDIA_GeForce_RTX_4090,dtype=fp8_w8a8.json`
|
||||
- **Separate kernel tuning**: Two files for up/down kernels with TMA optimization flags
|
||||
|
||||
Move these files to `sglang/srt/layers/moe/fused_moe_triton/configs/triton_version/` directory to use them in SGLang.
|
||||
Move these files to `sglang/srt/layers/moe/moe_runner/triton_utils/configs/triton_version/` directory to use them in SGLang.
|
||||
|
||||
### Supported Models
|
||||
|
||||
|
||||
@@ -147,7 +147,7 @@ spec:
|
||||
name: model
|
||||
- mountPath: /dev/infiniband
|
||||
name: ib
|
||||
- mountPath: /sgl-workspace/sglang/python/sglang/srt/layers/moe/fused_moe_triton/configs
|
||||
- mountPath: /sgl-workspace/sglang/python/sglang/srt/layers/moe/moe_runner/triton_utils/configs
|
||||
name: cf
|
||||
- mountPath: /root/.cache
|
||||
name: sgl-cache
|
||||
@@ -305,7 +305,7 @@ spec:
|
||||
name: model
|
||||
- mountPath: /dev/infiniband
|
||||
name: ib
|
||||
- mountPath: /sgl-workspace/sglang/python/sglang/srt/layers/moe/fused_moe_triton/configs
|
||||
- mountPath: /sgl-workspace/sglang/python/sglang/srt/layers/moe/moe_runner/triton_utils/configs
|
||||
name: cf
|
||||
dnsPolicy: ClusterFirstWithHostNet
|
||||
hostIPC: true
|
||||
@@ -458,7 +458,7 @@ spec:
|
||||
name: model
|
||||
- mountPath: /dev/infiniband
|
||||
name: ib
|
||||
- mountPath: /sgl-workspace/sglang/python/sglang/srt/layers/moe/fused_moe_triton/configs
|
||||
- mountPath: /sgl-workspace/sglang/python/sglang/srt/layers/moe/moe_runner/triton_utils/configs
|
||||
name: cf
|
||||
dnsPolicy: ClusterFirstWithHostNet
|
||||
hostIPC: true
|
||||
@@ -592,7 +592,7 @@ spec:
|
||||
name: model
|
||||
- mountPath: /dev/infiniband
|
||||
name: ib
|
||||
- mountPath: /sgl-workspace/sglang/python/sglang/srt/layers/moe/fused_moe_triton/configs
|
||||
- mountPath: /sgl-workspace/sglang/python/sglang/srt/layers/moe/moe_runner/triton_utils/configs
|
||||
name: cf
|
||||
dnsPolicy: ClusterFirstWithHostNet
|
||||
hostIPC: true
|
||||
|
||||
Reference in New Issue
Block a user