Skip to content

Commit a38f70d

Browse files
committed
Fix Bug vllm-project#4: Add debug logging for nucleus top-p check
Add diagnostic logging to verify draft_top_p value and whether nucleus will execute. This will help diagnose why nucleus shows 32000 survivors (full vocab) instead of filtered set. Expected log output: [NUCLEUS_DEBUG] draft_top_p from config: 0.95, will run nucleus: True If we see 'will run nucleus: False', we'll know the config isn't loaded or there's a logic bug in the condition.
1 parent fcbd61a commit a38f70d

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

vllm/v1/spec_decode/eagle.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -262,6 +262,8 @@ def _sample_draft_tokens(
262262

263263
# --- top-p (nucleus) ---
264264
tp = float(getattr(self.opt_config, "draft_top_p", 0.95) or 0.95)
265+
print(f"[NUCLEUS_DEBUG] draft_top_p from config: {tp}, will run nucleus: {0.0 < tp < 1.0}",
266+
file=sys.stderr, flush=True)
265267

266268
if 0.0 < tp < 1.0:
267269
p = torch.softmax(x, dim=-1)

0 commit comments

Comments
 (0)