Olive Recipes For AI Model Optimization Toolkit

This repository compliments Olive, the AI model optimization toolkit, and includes recipes demonstrating its extensive features and use cases. Users of Olive can use these recipes as a reference to either optimize publicly available AI models or to optimize their own proprietary models.

Supported models, architectures, devices and execution providers

Below are list of available recipes grouped by different criteria. Click the link to expand.

Models grouped by model architecture

bert	clip	deepseek	llama	llama3	mistral	phi3	phi4	qwen2	resnet	vit
google-bert-bert-base-multilingual-cased	laion-CLIP-ViT-B-32-laion2B-s34B-b79K	deepseek-ai-DeepSeek-R1-Distill-Llama-8B	meta-llama-Llama-3.2-1B-Instruct	meta-llama-Llama-3.2-1B-Instruct	mistralai-Mistral-7B-Instruct-v0.2	microsoft-Phi-3-mini-128k-instruct	microsoft-Phi-4-mini-instruct	Qwen-Qwen2.5-0.5B-Instruct	microsoft-resnet-50	google-vit-base-patch16-224
intel-bert-base-uncased-mrpc	openai-clip-vit-base-patch16	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B			mistralai-Mistral-7B-Instruct-v0.2	microsoft-Phi-3-mini-128k-instruct	microsoft-Phi-4-mini-reasoning	Qwen-Qwen2.5-0.5B-Instruct		google-vit-base-patch16-224
	openai-clip-vit-base-patch32	deepseek-ai-DeepSeek-R1-Distill-Qwen-14B			mistralai-Mistral-7B-Instruct-v0.3	microsoft-Phi-3-mini-128k-instruct	microsoft-Phi-4-reasoning-plus	Qwen-Qwen2.5-0.5B		google-vit-base-patch16-224
		deepseek-ai-DeepSeek-R1-Distill-Qwen-7B				microsoft-Phi-3-mini-4k-instruct	microsoft-Phi-4-reasoning	Qwen-Qwen2.5-1.5B-Instruct
						microsoft-Phi-3-mini-4k-instruct	microsoft-Phi-4	Qwen-Qwen2.5-1.5B-Instruct
						microsoft-Phi-3-mini-4k-instruct	microsoft-Phi-4	Qwen-Qwen2.5-14B-Instruct
						microsoft-Phi-3.5-mini-instruct		Qwen-Qwen2.5-14B-Instruct
						microsoft-Phi-3.5-mini-instruct		Qwen-Qwen2.5-3B-Instruct
						microsoft-Phi-3.5-mini-instruct		Qwen-Qwen2.5-7B-Instruct
						microsoft-Phi-4		Qwen-Qwen2.5-7B-Instruct
								Qwen-Qwen2.5-Coder-0.5B-Instruct
								Qwen-Qwen2.5-Coder-0.5B-Instruct
								Qwen-Qwen2.5-Coder-1.5B-Instruct
								Qwen-Qwen2.5-Coder-1.5B-Instruct
								Qwen-Qwen2.5-Coder-14B-Instruct
								Qwen-Qwen2.5-Coder-14B-Instruct
								Qwen-Qwen2.5-Coder-3B-Instruct
								Qwen-Qwen2.5-Coder-7B-Instruct
								Qwen-Qwen2.5-Coder-7B-Instruct
								deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B
								deepseek-ai-DeepSeek-R1-Distill-Qwen-14B
								deepseek-ai-DeepSeek-R1-Distill-Qwen-7B

Models grouped by device

cpu	gpu	npu
Qwen-Qwen2.5-1.5B-Instruct	DeepSeek-R1-Distill-Qwen-1.5B_Model_Builder_FP16	Qwen-Qwen2.5-0.5B-Instruct
deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B	DeepSeek-R1-Distill-Qwen-14B_NVMO_INT4_AWQ	Qwen-Qwen2.5-1.5B-Instruct
google-bert-bert-base-multilingual-cased	DeepSeek-R1-Distill-Qwen-7B_NVMO_INT4_RTN	Qwen-Qwen2.5-1.5B-Instruct
google-gemma	Llama-3.2-1B-Instruct_Model_Builder_FP16	Qwen-Qwen2.5-1.5B-Instruct
google-vit-base-patch16-224	Mistral-7B-Instruct-v0.2_Model_Builder_INT4	Qwen-Qwen2.5-7B-Instruct
intel-bert-base-uncased-mrpc (ov)	Phi-3-mini-128k-instruct_NVMO_INT4_RTN	Qwen-Qwen2.5-Coder-0.5B-Instruct
laion-CLIP-ViT-B-32-laion2B-s34B-b79K	Phi-3-mini-4k-instruct_Model_Builder_INT4	Qwen-Qwen2.5-Coder-1.5B-Instruct
meta-llama-Llama-3.2-1B-Instruct	Phi3.5_Mini_Instruct_Model_Builder_INT4	Qwen-Qwen2.5-Coder-7B-Instruct
microsoft-Phi-3.5-mini-instruct	Qwen-Qwen2.5-0.5B-Instruct	deepseek-ai-DeepSeek-R1-Distill-Llama-8B
microsoft-resnet-50	Qwen-Qwen2.5-0.5B	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B
openai-clip-vit-base-patch16	Qwen-Qwen2.5-1.5B-Instruct	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B
openai-clip-vit-base-patch32	Qwen-Qwen2.5-1.5B-Instruct	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B
timm-mobilenetv3_small_100.lamb_in1k	Qwen-Qwen2.5-1.5B-Instruct	deepseek-ai-DeepSeek-R1-Distill-Qwen-7B
	Qwen-Qwen2.5-14B-Instruct	google-bert-bert-base-multilingual-cased
	Qwen-Qwen2.5-3B-Instruct	google-bert-bert-base-multilingual-cased
	Qwen-Qwen2.5-7B-Instruct	google-bert-bert-base-multilingual-cased
	Qwen-Qwen2.5-Coder-0.5B-Instruct	google-vit-base-patch16-224
	Qwen-Qwen2.5-Coder-1.5B-Instruct	google-vit-base-patch16-224
	Qwen-Qwen2.5-Coder-14B-Instruct	google-vit-base-patch16-224
	Qwen-Qwen2.5-Coder-3B-Instruct	google-vit-base-patch16-224
	Qwen-Qwen2.5-Coder-7B-Instruct	google-vit-base-patch16-224
	Qwen2.5-0.5B-Instruct_Model_Builder_FP16	intel-bert-base-uncased-mrpc (AMD)
	Qwen2.5-14B-Instruct_Model_Builder_INT4	intel-bert-base-uncased-mrpc (ov)
	Qwen2.5-7B-Instruct_Model_Builder_INT4	intel-bert-base-uncased-mrpc
	Qwen2.5-Coder-0.5B-Instruct_Model_Builder_FP16	laion-CLIP-ViT-B-32-laion2B-s34B-b79K
	Qwen2.5-Coder-1.5B-Instruct_Model_Builder_FP16	laion-CLIP-ViT-B-32-laion2B-s34B-b79K
	Qwen2.5-Coder-14B-Instruct_Model_Builder_INT4	laion-CLIP-ViT-B-32-laion2B-s34B-b79K
	Qwen2.5-Coder-7B-Instruct_Model_Builder_INT4	meta-llama-Llama-3.2-1B-Instruct
	Qwen2.5_1.5B_Instruct_Model_Builder_FP16	meta-llama-Llama-3.2-1B-Instruct
	deepseek-ai-DeepSeek-R1-Distill-Llama-8B	meta-llama-Llama-3.2-1B-Instruct
	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B	microsoft-Phi-3-mini-128k-instruct
	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B	microsoft-Phi-3-mini-4k-instruct
	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B	microsoft-Phi-3-mini-4k-instruct
	deepseek-ai-DeepSeek-R1-Distill-Qwen-14B	microsoft-Phi-3.5-mini-instruct
	deepseek-ai-DeepSeek-R1-Distill-Qwen-7B	microsoft-Phi-3.5-mini-instruct
	google-bert-bert-base-multilingual-cased	microsoft-Phi-3.5-mini-instruct
	google-bert-bert-base-multilingual-cased	microsoft-Phi-3.5-mini-instruct
	google-bert-bert-base-multilingual-cased	microsoft-Phi-3.5-mini-instruct
	google-gemma	microsoft-Phi-4-mini-instruct
	google-vit-base-patch16-224	microsoft-Phi-4-mini-instruct
	google-vit-base-patch16-224	microsoft-Phi-4-mini-reasoning
	google-vit-base-patch16-224	microsoft-Phi-4-reasoning-plus
	intel-bert-base-uncased-mrpc (ov)	microsoft-Phi-4-reasoning
	intel-bert-base-uncased-mrpc	microsoft-resnet-50
	intel-bert-base-uncased-mrpc	microsoft-resnet-50
	laion-CLIP-ViT-B-32-laion2B-s34B-b79K	microsoft-resnet-50
	laion-CLIP-ViT-B-32-laion2B-s34B-b79K	microsoft-table-transformer-detection
	laion-CLIP-ViT-B-32-laion2B-s34B-b79K	mistralai-Mistral-7B-Instruct-v0.2
	meta-llama-Llama-3.2-1B-Instruct	openai-clip-vit-base-patch16
	meta-llama-Llama-3.2-1B-Instruct	openai-clip-vit-base-patch16
	meta-llama-Llama-3.2-1B-Instruct	openai-clip-vit-base-patch16
	microsoft-Phi-3-mini-128k-instruct	openai-clip-vit-base-patch32
	microsoft-Phi-3-mini-4k-instruct	openai-clip-vit-base-patch32
	microsoft-Phi-3.5-mini-instruct	openai-clip-vit-base-patch32
	microsoft-Phi-3.5-mini-instruct	timm-mobilenetv3_small_100.lamb_in1k
	microsoft-Phi-3.5-mini-instruct
	microsoft-Phi-4-mini-instruct
	microsoft-Phi-4-mini-instruct
	microsoft-Phi-4-mini-instruct
	microsoft-Phi-4-mini-reasoning
	microsoft-Phi-4-mini-reasoning
	microsoft-Phi-4-mini-reasoning
	microsoft-Phi-4-reasoning-plus
	microsoft-Phi-4-reasoning
	microsoft-Phi-4
	microsoft-Phi-4
	microsoft-resnet-50
	microsoft-resnet-50
	microsoft-resnet-50
	mistralai-Mistral-7B-Instruct-v0.2
	mistralai-Mistral-7B-Instruct-v0.3
	openai-clip-vit-base-patch16
	openai-clip-vit-base-patch16
	openai-clip-vit-base-patch16
	openai-clip-vit-base-patch32
	openai-clip-vit-base-patch32
	openai-clip-vit-base-patch32
	phi-4_Model_Builder_INT4

Models grouped by EP

CPU	CUDA	Dml	NvTensorRTRTX	OpenVINO	QNN	VitisAI
google-gemma	google-gemma	Qwen-Qwen2.5-1.5B-Instruct	DeepSeek-R1-Distill-Qwen-1.5B_Model_Builder_FP16	Qwen-Qwen2.5-0.5B-Instruct	Qwen-Qwen2.5-1.5B-Instruct	Qwen-Qwen2.5-1.5B-Instruct
timm-mobilenetv3_small_100.lamb_in1k		deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B	DeepSeek-R1-Distill-Qwen-14B_NVMO_INT4_AWQ	Qwen-Qwen2.5-0.5B-Instruct	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B
		google-bert-bert-base-multilingual-cased	DeepSeek-R1-Distill-Qwen-7B_NVMO_INT4_RTN	Qwen-Qwen2.5-0.5B	google-bert-bert-base-multilingual-cased	google-bert-bert-base-multilingual-cased
		google-vit-base-patch16-224	Llama-3.2-1B-Instruct_Model_Builder_FP16	Qwen-Qwen2.5-1.5B-Instruct	google-vit-base-patch16-224	google-vit-base-patch16-224
		intel-bert-base-uncased-mrpc	Mistral-7B-Instruct-v0.2_Model_Builder_INT4	Qwen-Qwen2.5-1.5B-Instruct	google-vit-base-patch16-224	intel-bert-base-uncased-mrpc (AMD)
		laion-CLIP-ViT-B-32-laion2B-s34B-b79K	Phi-3-mini-128k-instruct_NVMO_INT4_RTN	Qwen-Qwen2.5-14B-Instruct	intel-bert-base-uncased-mrpc	laion-CLIP-ViT-B-32-laion2B-s34B-b79K
		meta-llama-Llama-3.2-1B-Instruct	Phi-3-mini-4k-instruct_Model_Builder_INT4	Qwen-Qwen2.5-3B-Instruct	laion-CLIP-ViT-B-32-laion2B-s34B-b79K	meta-llama-Llama-3.2-1B-Instruct
		microsoft-Phi-3.5-mini-instruct	Phi3.5_Mini_Instruct_Model_Builder_INT4	Qwen-Qwen2.5-7B-Instruct	meta-llama-Llama-3.2-1B-Instruct	microsoft-Phi-3.5-mini-instruct
		microsoft-resnet-50	Qwen-Qwen2.5-1.5B-Instruct	Qwen-Qwen2.5-7B-Instruct	microsoft-Phi-3-mini-128k-instruct	microsoft-resnet-50
		openai-clip-vit-base-patch16	Qwen2.5-0.5B-Instruct_Model_Builder_FP16	Qwen-Qwen2.5-Coder-0.5B-Instruct	microsoft-Phi-3-mini-4k-instruct	openai-clip-vit-base-patch16
		openai-clip-vit-base-patch32	Qwen2.5-14B-Instruct_Model_Builder_INT4	Qwen-Qwen2.5-Coder-0.5B-Instruct	microsoft-Phi-3.5-mini-instruct	openai-clip-vit-base-patch32
			Qwen2.5-7B-Instruct_Model_Builder_INT4	Qwen-Qwen2.5-Coder-1.5B-Instruct	microsoft-Phi-3.5-mini-instruct
			Qwen2.5-Coder-0.5B-Instruct_Model_Builder_FP16	Qwen-Qwen2.5-Coder-1.5B-Instruct	microsoft-Phi-3.5-mini-instruct
			Qwen2.5-Coder-1.5B-Instruct_Model_Builder_FP16	Qwen-Qwen2.5-Coder-14B-Instruct	microsoft-Phi-4-mini-instruct
			Qwen2.5-Coder-14B-Instruct_Model_Builder_INT4	Qwen-Qwen2.5-Coder-3B-Instruct	microsoft-resnet-50
			Qwen2.5-Coder-7B-Instruct_Model_Builder_INT4	Qwen-Qwen2.5-Coder-7B-Instruct	microsoft-table-transformer-detection
			Qwen2.5_1.5B_Instruct_Model_Builder_FP16	Qwen-Qwen2.5-Coder-7B-Instruct	openai-clip-vit-base-patch16
			deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B	deepseek-ai-DeepSeek-R1-Distill-Llama-8B	openai-clip-vit-base-patch32
			google-bert-bert-base-multilingual-cased	deepseek-ai-DeepSeek-R1-Distill-Llama-8B	timm-mobilenetv3_small_100.lamb_in1k
			google-vit-base-patch16-224	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B
			intel-bert-base-uncased-mrpc	deepseek-ai-DeepSeek-R1-Distill-Qwen-1.5B
			laion-CLIP-ViT-B-32-laion2B-s34B-b79K	deepseek-ai-DeepSeek-R1-Distill-Qwen-14B
			meta-llama-Llama-3.2-1B-Instruct	deepseek-ai-DeepSeek-R1-Distill-Qwen-7B
			microsoft-Phi-3.5-mini-instruct	deepseek-ai-DeepSeek-R1-Distill-Qwen-7B
			microsoft-resnet-50	google-bert-bert-base-multilingual-cased
			openai-clip-vit-base-patch16	google-vit-base-patch16-224
			openai-clip-vit-base-patch32	google-vit-base-patch16-224
			phi-4_Model_Builder_INT4	intel-bert-base-uncased-mrpc (ov)
				laion-CLIP-ViT-B-32-laion2B-s34B-b79K
				meta-llama-Llama-3.2-1B-Instruct
				microsoft-Phi-3-mini-128k-instruct
				microsoft-Phi-3-mini-4k-instruct
				microsoft-Phi-3-mini-4k-instruct
				microsoft-Phi-3.5-mini-instruct
				microsoft-Phi-3.5-mini-instruct
				microsoft-Phi-4-mini-instruct
				microsoft-Phi-4-mini-instruct
				microsoft-Phi-4-mini-instruct
				microsoft-Phi-4-mini-instruct
				microsoft-Phi-4-mini-reasoning
				microsoft-Phi-4-mini-reasoning
				microsoft-Phi-4-mini-reasoning
				microsoft-Phi-4-mini-reasoning
				microsoft-Phi-4-reasoning-plus
				microsoft-Phi-4-reasoning-plus
				microsoft-Phi-4-reasoning
				microsoft-Phi-4-reasoning
				microsoft-Phi-4
				microsoft-Phi-4
				microsoft-resnet-50
				mistralai-Mistral-7B-Instruct-v0.2
				mistralai-Mistral-7B-Instruct-v0.2
				mistralai-Mistral-7B-Instruct-v0.3
				openai-clip-vit-base-patch16
				openai-clip-vit-base-patch32

Learn more

• Olive Github Repository
• Getting Started
• Olive Documentation

🤝 Contributions and Feedback

• We welcome contributions! Please read the contribution guidelines for more details on how to contribute to the Olive project.
• For feature requests or bug reports, file a GitHub Issue.
• For general discussion or questions, use GitHub Discussions.

⚖️ License

Copyright (c) Microsoft Corporation. All rights reserved.

Licensed under the MIT License.