@@ -151,7 +151,7 @@ After tuning, configuration files will be generated:
|
|||||||
- **Standard tuning**: `E=64,N=640,device_name=NVIDIA_GeForce_RTX_4090,dtype=fp8_w8a8.json`
|
- **Standard tuning**: `E=64,N=640,device_name=NVIDIA_GeForce_RTX_4090,dtype=fp8_w8a8.json`
|
||||||
- **Separate kernel tuning**: Two files for up/down kernels with TMA optimization flags
|
- **Separate kernel tuning**: Two files for up/down kernels with TMA optimization flags
|
||||||
|
|
||||||
Move these files to `sglang/srt/layers/moe/fused_moe_triton/configs/triton_version/` directory to use them in SGLang.
|
Move these files to `sglang/srt/layers/moe/moe_runner/triton_utils/configs/triton_version/` directory to use them in SGLang.
|
||||||
|
|
||||||
### Supported Models
|
### Supported Models
|
||||||
|
|
||||||
|
|||||||
@@ -147,7 +147,7 @@ spec:
|
|||||||
name: model
|
name: model
|
||||||
- mountPath: /dev/infiniband
|
- mountPath: /dev/infiniband
|
||||||
name: ib
|
name: ib
|
||||||
- mountPath: /sgl-workspace/sglang/python/sglang/srt/layers/moe/fused_moe_triton/configs
|
- mountPath: /sgl-workspace/sglang/python/sglang/srt/layers/moe/moe_runner/triton_utils/configs
|
||||||
name: cf
|
name: cf
|
||||||
- mountPath: /root/.cache
|
- mountPath: /root/.cache
|
||||||
name: sgl-cache
|
name: sgl-cache
|
||||||
@@ -305,7 +305,7 @@ spec:
|
|||||||
name: model
|
name: model
|
||||||
- mountPath: /dev/infiniband
|
- mountPath: /dev/infiniband
|
||||||
name: ib
|
name: ib
|
||||||
- mountPath: /sgl-workspace/sglang/python/sglang/srt/layers/moe/fused_moe_triton/configs
|
- mountPath: /sgl-workspace/sglang/python/sglang/srt/layers/moe/moe_runner/triton_utils/configs
|
||||||
name: cf
|
name: cf
|
||||||
dnsPolicy: ClusterFirstWithHostNet
|
dnsPolicy: ClusterFirstWithHostNet
|
||||||
hostIPC: true
|
hostIPC: true
|
||||||
@@ -458,7 +458,7 @@ spec:
|
|||||||
name: model
|
name: model
|
||||||
- mountPath: /dev/infiniband
|
- mountPath: /dev/infiniband
|
||||||
name: ib
|
name: ib
|
||||||
- mountPath: /sgl-workspace/sglang/python/sglang/srt/layers/moe/fused_moe_triton/configs
|
- mountPath: /sgl-workspace/sglang/python/sglang/srt/layers/moe/moe_runner/triton_utils/configs
|
||||||
name: cf
|
name: cf
|
||||||
dnsPolicy: ClusterFirstWithHostNet
|
dnsPolicy: ClusterFirstWithHostNet
|
||||||
hostIPC: true
|
hostIPC: true
|
||||||
@@ -592,7 +592,7 @@ spec:
|
|||||||
name: model
|
name: model
|
||||||
- mountPath: /dev/infiniband
|
- mountPath: /dev/infiniband
|
||||||
name: ib
|
name: ib
|
||||||
- mountPath: /sgl-workspace/sglang/python/sglang/srt/layers/moe/fused_moe_triton/configs
|
- mountPath: /sgl-workspace/sglang/python/sglang/srt/layers/moe/moe_runner/triton_utils/configs
|
||||||
name: cf
|
name: cf
|
||||||
dnsPolicy: ClusterFirstWithHostNet
|
dnsPolicy: ClusterFirstWithHostNet
|
||||||
hostIPC: true
|
hostIPC: true
|
||||||
|
|||||||
Reference in New Issue
Block a user