-
Notifications
You must be signed in to change notification settings - Fork 212
Description
I'm using a remote server and I'm encountering an error when using the "Qwen/Qwen2.5-32B-Instruct" tokenizer with my LMQL model. The error message is as follows:
lmql.runtime.tokenizer.TokenizerNotAvailableError: Failed to locate a suitable tokenizer implementation for 'Qwen/Qwen2.5-32B-Instruct' (Make sure your current environment provides a tokenizer backend like 'transformers', 'tiktoken' or 'llama.cpp' for this model)
Below is the code I'm using:
import lmql
model = lmql.model(
"openai/qwen2.5-32b",
endpoint="http://XXX.XX.XX.XX:XXXXX/v1/chat/completions",
api_key="test",
tokenizer="Qwen/Qwen2.5-32B-Instruct"
)
@lmql.query(model=model)
def tell_a_joke():
'''lmql
"""A great good dad joke. A indicates the punchline
Q:[JOKE]
A:[PUNCHLINE]""" where STOPS_AT(JOKE, "?") and
STOPS_AT(PUNCHLINE, "\n")
'''
tell_a_joke()
Could someone please advise on the correct way to configure the tokenizer parameter? Should I be using a different tokenizer string or is there a specific tokenizer backend that I need to install/configure in my environment?
Any help or guidance would be greatly appreciated!