Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion samples/python/image_generation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,29 @@ Users can change the sample code and play with the following generation paramete
The `--upgrade-strategy eager` option is needed to ensure `optimum-intel` is upgraded to the latest version.

Install [../../export-requirements.txt](../../export-requirements.txt) to convert a model.

```sh
pip install --upgrade-strategy eager -r ../../export-requirements.txt
```

Then, run the export with Optimum CLI:

```sh
optimum-cli export openvino --model dreamlike-art/dreamlike-anime-1.0 --task stable-diffusion --weight-format fp16 dreamlike_anime_1_0_ov/FP16
```

Alternatively, do it in Python code (FP16 is used by default). If NNCF is installed, the model will be compressed to INT8 automatically.

```python
from optimum.exporters.openvino.convert import export_tokenizer
from optimum.intel import OVPipelineForText2Image

output_dir = "dreamlike_anime_1_0_ov/FP16"

pipeline = OVPipelineForText2Image.from_pretrained("dreamlike-art/dreamlike-anime-1.0", export=True)
pipeline.save_pretrained(output_dir)
export_tokenizer(pipeline.tokenizer, output_dir + "/tokenizer")
```

## Run text to image

Install [deployment-requirements.txt](../../deployment-requirements.txt) via `pip install -r ../../deployment-requirements.txt` and then, run a sample:
Expand Down
22 changes: 21 additions & 1 deletion samples/python/rag/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,30 @@ Install [../../export-requirements.txt](../../export-requirements.txt) to conver

```sh
pip install --upgrade-strategy eager -r ../../export-requirements.txt
```

Then, run the export with Optimum CLI:

```sh
optimum-cli export openvino --trust-remote-code --model BAAI/bge-small-en-v1.5 BAAI/bge-small-en-v1.5
```

Alternatively, do it in Python code:

```python
from optimum.exporters.openvino.convert import export_tokenizer
from optimum.intel import OVModelForFeatureExtraction
from transformers import AutoTokenizer

output_dir = "embedding_model"

model = OVModelForFeatureExtraction.from_pretrained("BAAI/bge-small-en-v1.5", export=True, trust_remote_code=True)
model.save_pretrained(output_dir)

tokenizer = AutoTokenizer.from_pretrained("BAAI/bge-small-en-v1.5")
export_tokenizer(tokenizer, output_dir)
```

## Run

Install [deployment-requirements.txt](../../deployment-requirements.txt) via `pip install -r ../../deployment-requirements.txt` and then, run a sample:
Expand All @@ -24,7 +45,6 @@ See [SUPPORTED_MODELS.md](../../../SUPPORTED_MODELS.md#text-embeddings-models) f
# Text Embedding Pipeline Usage

```python
import argparse
import openvino_genai

pipeline = openvino_genai.TextEmbeddingPipeline(model_dir, "CPU")
Expand Down
23 changes: 22 additions & 1 deletion samples/python/speech_generation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,31 @@ Install [../../export-requirements.txt](../../export-requirements.txt) to conver

```sh
pip install --upgrade-strategy eager -r ../../export-requirements.txt
```

Then, run the export with Optimum CLI:

```sh
optimum-cli export openvino --model microsoft/speecht5_tts --model-kwargs "{\"vocoder\": \"microsoft/speecht5_hifigan\"}" speecht5_tts
```

**Note:** Currently, text-to-speech in OpenVINO GenAI supports the `SpeechT5 TTS` model.
Alternatively, you can do it in Python code:

```python
from optimum.exporters.openvino.convert import export_tokenizer
from optimum.intel import OVModelForTextToSpeechSeq2Seq
from transformers import AutoTokenizer

output_dir = "speecht5_tts"

model = OVModelForTextToSpeechSeq2Seq.from_pretrained("microsoft/speecht5_tts", vocoder="microsoft/speecht5_hifigan", export=True)
model.save_pretrained(output_dir)

tokenizer = AutoTokenizer.from_pretrained("microsoft/speecht5_tts")
export_tokenizer(tokenizer, output_dir)
```

**Note:** Currently, text-to-speech in OpenVINO GenAI supports the `SpeechT5 TTS` model.
When exporting the model, you must specify a vocoder using the `--model-kwargs` option in JSON format.

## Prepare speaker embedding file
Expand Down
25 changes: 25 additions & 0 deletions samples/python/text_generation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,35 @@ There are also Jupyter notebooks for some samples. You can find links to them in
## Download and convert the model and tokenizers
The `--upgrade-strategy eager` option is needed to ensure `optimum-intel` is upgraded to the latest version.
Install [../../export-requirements.txt](../../export-requirements.txt) if model conversion is required.

```sh
pip install --upgrade-strategy eager -r ../../export-requirements.txt
```

Then, run the export with Optimum CLI:

```sh
optimum-cli export openvino --model <model> <output_folder>
```

Alternatively, do it in Python code (e.g. TinyLlama_v1.1). If NNCF is installed, the model will be compressed to INT8 automatically.

```python
from optimum.exporters.openvino.convert import export_tokenizer
from optimum.intel import OVModelForCausalLM
from transformers import AutoTokenizer

output_dir = "chat_model"

model = OVModelForCausalLM.from_pretrained("TinyLlama/TinyLlama_v1.1", export=True)
model.save_pretrained(output_dir)

tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama_v1.1")
tokenizer.save_pretrained(output_dir)
export_tokenizer(tokenizer, output_dir)
```
[//]: # "tokenizer.save_pretrained(output_dir) is required above to mitigate runtime errors"

If a converted model in OpenVINO IR format is already available in the collection of [OpenVINO optimized LLMs](https://huggingface.co/collections/OpenVINO/llm-6687aaa2abca3bbcec71a9bd) on Hugging Face, it can be downloaded directly via huggingface-cli.
```sh
pip install huggingface-hub
Expand Down
21 changes: 21 additions & 0 deletions samples/python/visual_language_chat/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,30 @@ Install [../../export-requirements.txt](../../export-requirements.txt) to conver

```sh
pip install --upgrade-strategy eager -r ../../export-requirements.txt
```

Then, run the export with Optimum CLI:

```sh
optimum-cli export openvino --model openbmb/MiniCPM-V-2_6 --trust-remote-code MiniCPM-V-2_6
```

Alternatively, you can do it in Python code:

```python
from optimum.exporters.openvino.convert import export_tokenizer
from optimum.intel import OVModelForVisualCausalLM
from transformers import AutoTokenizer

output_dir = "MiniCPM-V-2_6"

model = OVModelForVisualCausalLM.from_pretrained("openbmb/MiniCPM-V-2_6", export=True, trust_remote_code=True)
model.save_pretrained(output_dir)

tokenizer = AutoTokenizer.from_pretrained("openbmb/MiniCPM-V-2_6")
export_tokenizer(tokenizer, output_dir)
```

## Run:

[This image](https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/d5fbbd1a-d484-415c-88cb-9986625b7b11) can be used as a sample image.
Expand Down
21 changes: 21 additions & 0 deletions samples/python/whisper_speech_recognition/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,30 @@ Install [../../export-requirements.txt](../../export-requirements.txt) to conver

```sh
pip install --upgrade-strategy eager -r ../../export-requirements.txt
```

Then, run the export with Optimum CLI:

```sh
optimum-cli export openvino --trust-remote-code --model openai/whisper-base whisper-base
```

Alternatively, you can do it in Python code:

```python
from optimum.exporters.openvino.convert import export_tokenizer
from optimum.intel import OVModelForSpeechSeq2Seq
from transformers import AutoTokenizer

output_dir = "whisper-base"

model = OVModelForSpeechSeq2Seq.from_pretrained("openai/whisper-base", export=True, trust_remote_code=True)
model.save_pretrained(output_dir)

tokenizer = AutoTokenizer.from_pretrained("openai/whisper-base")
export_tokenizer(tokenizer, output_dir)
```

## Prepare audio file

Download example audio file: https://storage.openvinotoolkit.org/models_contrib/speech/2021.2/librispeech_s5/how_are_you_doing_today.wav
Expand Down
Loading