diff --git a/samples/python/image_generation/README.md b/samples/python/image_generation/README.md index be53a9126d..950c0519fa 100644 --- a/samples/python/image_generation/README.md +++ b/samples/python/image_generation/README.md @@ -30,12 +30,29 @@ Users can change the sample code and play with the following generation paramete The `--upgrade-strategy eager` option is needed to ensure `optimum-intel` is upgraded to the latest version. Install [../../export-requirements.txt](../../export-requirements.txt) to convert a model. - ```sh pip install --upgrade-strategy eager -r ../../export-requirements.txt +``` + +Then, run the export with Optimum CLI: + +```sh optimum-cli export openvino --model dreamlike-art/dreamlike-anime-1.0 --task stable-diffusion --weight-format fp16 dreamlike_anime_1_0_ov/FP16 ``` +Alternatively, do it in Python code (FP16 is used by default). If NNCF is installed, the model will be compressed to INT8 automatically. + +```python +from optimum.exporters.openvino.convert import export_tokenizer +from optimum.intel import OVPipelineForText2Image + +output_dir = "dreamlike_anime_1_0_ov/FP16" + +pipeline = OVPipelineForText2Image.from_pretrained("dreamlike-art/dreamlike-anime-1.0", export=True) +pipeline.save_pretrained(output_dir) +export_tokenizer(pipeline.tokenizer, output_dir + "/tokenizer") +``` + ## Run text to image Install [deployment-requirements.txt](../../deployment-requirements.txt) via `pip install -r ../../deployment-requirements.txt` and then, run a sample: diff --git a/samples/python/rag/README.md b/samples/python/rag/README.md index 13b6de7bbc..d450be8624 100644 --- a/samples/python/rag/README.md +++ b/samples/python/rag/README.md @@ -10,9 +10,30 @@ Install [../../export-requirements.txt](../../export-requirements.txt) to conver ```sh pip install --upgrade-strategy eager -r ../../export-requirements.txt +``` + +Then, run the export with Optimum CLI: + +```sh optimum-cli export openvino --trust-remote-code --model BAAI/bge-small-en-v1.5 BAAI/bge-small-en-v1.5 ``` +Alternatively, do it in Python code: + +```python +from optimum.exporters.openvino.convert import export_tokenizer +from optimum.intel import OVModelForFeatureExtraction +from transformers import AutoTokenizer + +output_dir = "embedding_model" + +model = OVModelForFeatureExtraction.from_pretrained("BAAI/bge-small-en-v1.5", export=True, trust_remote_code=True) +model.save_pretrained(output_dir) + +tokenizer = AutoTokenizer.from_pretrained("BAAI/bge-small-en-v1.5") +export_tokenizer(tokenizer, output_dir) +``` + ## Run Install [deployment-requirements.txt](../../deployment-requirements.txt) via `pip install -r ../../deployment-requirements.txt` and then, run a sample: @@ -24,7 +45,6 @@ See [SUPPORTED_MODELS.md](../../../SUPPORTED_MODELS.md#text-embeddings-models) f # Text Embedding Pipeline Usage ```python -import argparse import openvino_genai pipeline = openvino_genai.TextEmbeddingPipeline(model_dir, "CPU") diff --git a/samples/python/speech_generation/README.md b/samples/python/speech_generation/README.md index 1b2adcfb1e..7a87002406 100644 --- a/samples/python/speech_generation/README.md +++ b/samples/python/speech_generation/README.md @@ -12,10 +12,31 @@ Install [../../export-requirements.txt](../../export-requirements.txt) to conver ```sh pip install --upgrade-strategy eager -r ../../export-requirements.txt +``` + +Then, run the export with Optimum CLI: + +```sh optimum-cli export openvino --model microsoft/speecht5_tts --model-kwargs "{\"vocoder\": \"microsoft/speecht5_hifigan\"}" speecht5_tts ``` -**Note:** Currently, text-to-speech in OpenVINO GenAI supports the `SpeechT5 TTS` model. +Alternatively, you can do it in Python code: + +```python +from optimum.exporters.openvino.convert import export_tokenizer +from optimum.intel import OVModelForTextToSpeechSeq2Seq +from transformers import AutoTokenizer + +output_dir = "speecht5_tts" + +model = OVModelForTextToSpeechSeq2Seq.from_pretrained("microsoft/speecht5_tts", vocoder="microsoft/speecht5_hifigan", export=True) +model.save_pretrained(output_dir) + +tokenizer = AutoTokenizer.from_pretrained("microsoft/speecht5_tts") +export_tokenizer(tokenizer, output_dir) +``` + +**Note:** Currently, text-to-speech in OpenVINO GenAI supports the `SpeechT5 TTS` model. When exporting the model, you must specify a vocoder using the `--model-kwargs` option in JSON format. ## Prepare speaker embedding file diff --git a/samples/python/text_generation/README.md b/samples/python/text_generation/README.md index 7d334df29e..e12d254f82 100644 --- a/samples/python/text_generation/README.md +++ b/samples/python/text_generation/README.md @@ -13,10 +13,35 @@ There are also Jupyter notebooks for some samples. You can find links to them in ## Download and convert the model and tokenizers The `--upgrade-strategy eager` option is needed to ensure `optimum-intel` is upgraded to the latest version. Install [../../export-requirements.txt](../../export-requirements.txt) if model conversion is required. + ```sh pip install --upgrade-strategy eager -r ../../export-requirements.txt +``` + +Then, run the export with Optimum CLI: + +```sh optimum-cli export openvino --model ``` + +Alternatively, do it in Python code (e.g. TinyLlama_v1.1). If NNCF is installed, the model will be compressed to INT8 automatically. + +```python +from optimum.exporters.openvino.convert import export_tokenizer +from optimum.intel import OVModelForCausalLM +from transformers import AutoTokenizer + +output_dir = "chat_model" + +model = OVModelForCausalLM.from_pretrained("TinyLlama/TinyLlama_v1.1", export=True) +model.save_pretrained(output_dir) + +tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama_v1.1") +tokenizer.save_pretrained(output_dir) +export_tokenizer(tokenizer, output_dir) +``` +[//]: # "tokenizer.save_pretrained(output_dir) is required above to mitigate runtime errors" + If a converted model in OpenVINO IR format is already available in the collection of [OpenVINO optimized LLMs](https://huggingface.co/collections/OpenVINO/llm-6687aaa2abca3bbcec71a9bd) on Hugging Face, it can be downloaded directly via huggingface-cli. ```sh pip install huggingface-hub diff --git a/samples/python/visual_language_chat/README.md b/samples/python/visual_language_chat/README.md index 098cbac630..c5669e5a84 100644 --- a/samples/python/visual_language_chat/README.md +++ b/samples/python/visual_language_chat/README.md @@ -14,9 +14,30 @@ Install [../../export-requirements.txt](../../export-requirements.txt) to conver ```sh pip install --upgrade-strategy eager -r ../../export-requirements.txt +``` + +Then, run the export with Optimum CLI: + +```sh optimum-cli export openvino --model openbmb/MiniCPM-V-2_6 --trust-remote-code MiniCPM-V-2_6 ``` +Alternatively, you can do it in Python code: + +```python +from optimum.exporters.openvino.convert import export_tokenizer +from optimum.intel import OVModelForVisualCausalLM +from transformers import AutoTokenizer + +output_dir = "MiniCPM-V-2_6" + +model = OVModelForVisualCausalLM.from_pretrained("openbmb/MiniCPM-V-2_6", export=True, trust_remote_code=True) +model.save_pretrained(output_dir) + +tokenizer = AutoTokenizer.from_pretrained("openbmb/MiniCPM-V-2_6") +export_tokenizer(tokenizer, output_dir) +``` + ## Run: [This image](https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/d5fbbd1a-d484-415c-88cb-9986625b7b11) can be used as a sample image. diff --git a/samples/python/whisper_speech_recognition/README.md b/samples/python/whisper_speech_recognition/README.md index ccd4a00499..24a9ba50ef 100644 --- a/samples/python/whisper_speech_recognition/README.md +++ b/samples/python/whisper_speech_recognition/README.md @@ -10,9 +10,30 @@ Install [../../export-requirements.txt](../../export-requirements.txt) to conver ```sh pip install --upgrade-strategy eager -r ../../export-requirements.txt +``` + +Then, run the export with Optimum CLI: + +```sh optimum-cli export openvino --trust-remote-code --model openai/whisper-base whisper-base ``` +Alternatively, you can do it in Python code: + +```python +from optimum.exporters.openvino.convert import export_tokenizer +from optimum.intel import OVModelForSpeechSeq2Seq +from transformers import AutoTokenizer + +output_dir = "whisper-base" + +model = OVModelForSpeechSeq2Seq.from_pretrained("openai/whisper-base", export=True, trust_remote_code=True) +model.save_pretrained(output_dir) + +tokenizer = AutoTokenizer.from_pretrained("openai/whisper-base") +export_tokenizer(tokenizer, output_dir) +``` + ## Prepare audio file Download example audio file: https://storage.openvinotoolkit.org/models_contrib/speech/2021.2/librispeech_s5/how_are_you_doing_today.wav