Skip to content

Commit f99f188

Browse files
authored
Hotfix of benchmark script (#12467)
1 parent c911026 commit f99f188

File tree

1 file changed

+1
-1
lines changed
  • python/llm/dev/benchmark/all-in-one

1 file changed

+1
-1
lines changed

python/llm/dev/benchmark/all-in-one/run.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -641,7 +641,7 @@ def transformers_int4_npu_win(repo_id,
641641
model = AutoModelForCausalLM.from_pretrained(model_path, load_in_low_bit=low_bit, trust_remote_code=True, torch_dtype=torch.float16,
642642
optimize_model=optimize_model, max_context_len=max_context_len, max_prompt_len=int(in_out_len[0]),
643643
quantization_group_size=npu_group_size, transpose_value_cache=transpose_value_cache,
644-
save_directory=save_directory, use_cache=True, attn_implementation="eager").eval()
644+
mixed_precision=True, save_directory=save_directory, use_cache=True, attn_implementation="eager").eval()
645645
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
646646
end = time.perf_counter()
647647
load_time = end - st

0 commit comments

Comments
 (0)