[fix] fix dsv3_router_gemm filter (#7750)

Alcanderian · web-flow · commit ac49dac00946 · 2025-07-03T09:25:32.000-07:00
diff --git a/python/sglang/srt/models/deepseek_v2.py b/python/sglang/srt/models/deepseek_v2.py
@@ -233,7 +233,8 @@ def forward(self, hidden_states):
             )
 
         if (
-            hidden_states.shape[0] < 4
+            _is_cuda
+            and hidden_states.shape[0] < 4
             and hidden_states.shape[1] == 7168
             and self.weight.shape[0] == 256
             and _device_sm >= 90

Original file line number	Diff line number	Diff line change
`@@ -233,7 +233,8 @@ def forward(self, hidden_states):`
`233`	`233`	`)`
`234`	`234`
`235`	`235`	`if (`
`236`		`- hidden_states.shape[0] < 4`
	`236`	`+ _is_cuda`
	`237`	`+ and hidden_states.shape[0] < 4`
`237`	`238`	`and hidden_states.shape[1] == 7168`
`238`	`239`	`and self.weight.shape[0] == 256`
`239`	`240`	`and _device_sm >= 90`