Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions docs/api/profiles.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# `pydantic_ai.profiles`

::: pydantic_ai.profiles.ModelProfile

::: pydantic_ai.profiles.openai

::: pydantic_ai.profiles.anthropic

::: pydantic_ai.profiles.google

::: pydantic_ai.profiles.meta

::: pydantic_ai.profiles.amazon

::: pydantic_ai.profiles.deepseek

::: pydantic_ai.profiles.grok

::: pydantic_ai.profiles.mistral

::: pydantic_ai.profiles.qwen
28 changes: 17 additions & 11 deletions docs/models/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,21 @@
PydanticAI is model-agnostic and has built-in support for multiple model providers:

* [OpenAI](openai.md)
* [DeepSeek](openai.md#openai-compatible-models)
* [Anthropic](anthropic.md)
* [Gemini](gemini.md) (via two different APIs: Generative Language API and VertexAI API)
* [Ollama](openai.md#ollama)
* [Groq](groq.md)
* [Mistral](mistral.md)
* [Cohere](cohere.md)
* [Bedrock](bedrock.md)

## OpenAI-compatible Providers

Many models are compatible with the OpenAI API, and can be used with `OpenAIModel` in PydanticAI:
In addition, many providers are compatible with the OpenAI API, and can be used with `OpenAIModel` in PydanticAI:

* [OpenRouter](openai.md#openrouter)
* [DeepSeek](openai.md#deepseek)
* [Grok (xAI)](openai.md#grok-xai)
* [Ollama](openai.md#ollama)
* [OpenRouter](openai.md#openrouter)
* [Perplexity](openai.md#perplexity)
* [Fireworks AI](openai.md#fireworks-ai)
* [Together AI](openai.md#together-ai)
Expand All @@ -40,27 +40,33 @@ PydanticAI uses a few key terms to describe how it interacts with different LLMs
roughly in the format `<VendorSdk>Model`, for example, we have `OpenAIModel`, `AnthropicModel`, `GeminiModel`,
etc. When using a Model class, you specify the actual LLM model name (e.g., `gpt-4o`,
`claude-3-5-sonnet-latest`, `gemini-1.5-flash`) as a parameter.
* **Provider**: This refers to Model-specific classes which handle the authentication and connections
* **Provider**: This refers to provider-specific classes which handle the authentication and connections
to an LLM vendor. Passing a non-default _Provider_ as a parameter to a Model is how you can ensure
that your agent will make requests to a specific endpoint, or make use of a specific approach to
authentication (e.g., you can use Vertex-specific auth with the `GeminiModel` by way of the `VertexProvider`).
In particular, this is how you can make use of an AI gateway, or an LLM vendor that offers API compatibility
with the vendor SDK used by an existing Model (such as `OpenAIModel`).
* **Profile**: This refers to a description of how requests to a specific model or family of models need to be
constructed to get the best results, independent of the model and provider classes used.
For example, different models have different restrictions on the JSON schemas that can be used for tools,
and the same schema transformer needs to be used for Gemini models whether you're using `GoogleModel`
with model name `gemini-2.5-pro-preview`, or `OpenAIModel` with `OpenRouterProvider` and model name `google/gemini-2.5-pro-preview`.

In short, you select a specific model name (like `gpt-4o`), PydanticAI uses the appropriate Model class (like `OpenAIModel`), and the provider handles the connection and authentication to the underlying service.
When you instantiate an [`Agent`][pydantic_ai.Agent] with just a name formatted as `<provider>:<model>`, e.g. `openai:gpt-4o` or `openrouter:google/gemini-2.5-pro-preview`,
PydanticAI will automatically select the appropriate model class, provider, and profile.
If you want to use a different provider or profile, you can instantiate a model class directly and pass in `provider` and/or `profile` arguments.

## Custom Models

To implement support for models not already supported, you will need to subclass the [`Model`][pydantic_ai.models.Model] abstract base class.

For streaming, you'll also need to implement the following abstract base class:

* [`StreamedResponse`][pydantic_ai.models.StreamedResponse]
To implement support for a model API that's not already supported, you will need to subclass the [`Model`][pydantic_ai.models.Model] abstract base class.
For streaming, you'll also need to implement the [`StreamedResponse`][pydantic_ai.models.StreamedResponse] abstract base class.

The best place to start is to review the source code for existing implementations, e.g. [`OpenAIModel`](https://github.com/pydantic/pydantic-ai/blob/main/pydantic_ai_slim/pydantic_ai/models/openai.py).

For details on when we'll accept contributions adding new models to PydanticAI, see the [contributing guidelines](../contributing.md#new-model-rules).

If a model API is compatible with the OpenAI API, you do not need a custom model class and can provide your own [custom provider](openai.md#openai-compatible-models) instead.

<!-- TODO(Marcelo): We need to create a section in the docs about reliability. -->
## Fallback Model

Expand Down
43 changes: 38 additions & 5 deletions docs/models/openai.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@

## Install

To use OpenAI models, you need to either install `pydantic-ai`, or install `pydantic-ai-slim` with the `openai` optional group:
To use OpenAI models or OpenAI-compatible APIs, you need to either install `pydantic-ai`, or install `pydantic-ai-slim` with the `openai` optional group:

```bash
pip/uv-add "pydantic-ai-slim[openai]"
```

## Configuration

To use `OpenAIModel` through their main API, go to [platform.openai.com](https://platform.openai.com/) and follow your nose until you find the place to generate an API key.
To use `OpenAIModel` with the OpenAI API, go to [platform.openai.com](https://platform.openai.com/) and follow your nose until you find the place to generate an API key.

## Environment variable

Expand Down Expand Up @@ -130,7 +130,7 @@ You can learn more about the differences between the Responses API and Chat Comp

## OpenAI-compatible Models

Many models are compatible with the OpenAI API, and can be used with `OpenAIModel` in PydanticAI.
Many providers and models are compatible with the OpenAI API, and can be used with `OpenAIModel` in PydanticAI.
Before getting started, check the [installation and configuration](#install) instructions above.

To use another OpenAI-compatible API, you can make use of the `base_url` and `api_key` arguments from `OpenAIProvider`:
Expand All @@ -150,7 +150,40 @@ agent = Agent(model)
...
```

You can also use the `provider` argument with a custom provider class like the `DeepSeekProvider`:
Various providers also have their own provider classes so that you don't need to specify the base URL yourself and you can use the standard `<PROVIDER>_API_KEY` environment variable to set the API key.
When a provider has its own provider class, you can use the `Agent("<provider>:<model>")` shorthand, e.g. `Agent("deepseek:deepseek-chat")` or `Agent("openrouter:google/gemini-2.5-pro-preview")`, instead of building the `OpenAIModel` explicitly. Similarly, you can pass the provider name as a string to the `provider` argument on `OpenAIModel` instead of building instantiating the provider class explicitly.

#### Model Profile

Sometimes, the provider or model you're using will have slightly different requirements than OpenAI's API or models, like having different restrictions on JSON schemas for tool definitions, or not supporting tool definitions to be marked as strict.

When using an alternative provider class provided by PydanticAI, an appropriate model profile is typically selected automatically based on the model name.
If the model you're using is not working correctly out of the box, you can tweak various aspects of how model requests are constructed by providing your own [`ModelProfile`][pydantic_ai.profiles.ModelProfile] (for behaviors shared among all model classes) or [`OpenAIModelProfile`][pydantic_ai.profiles.openai.OpenAIModelProfile] (for behaviors specific to `OpenAIModel`):

```py
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIModel
from pydantic_ai.profiles._json_schema import InlineDefsJsonSchemaTransformer
from pydantic_ai.profiles.openai import OpenAIModelProfile
from pydantic_ai.providers.openai import OpenAIProvider

model = OpenAIModel(
'model_name',
provider=OpenAIProvider(
base_url='https://<openai-compatible-api-endpoint>.com', api_key='your-api-key'
),
profile=OpenAIModelProfile(
json_schema_transformer=InlineDefsJsonSchemaTransformer, # Supported by any model class on a plain ModelProfile
openai_supports_strict_tool_definition=False # Supported by OpenAIModel only, requires OpenAIModelProfile
)
)
agent = Agent(model)
```

### DeepSeek

To use the [DeepSeek](https://deepseek.com) provider, first create an API key by following the [Quick Start guide](https://api-docs.deepseek.com/).
Once you have the API key, you can use it with the `DeepSeekProvider`:

```python
from pydantic_ai import Agent
Expand Down Expand Up @@ -285,7 +318,7 @@ agent = Agent(model)

To use [OpenRouter](https://openrouter.ai), first create an API key at [openrouter.ai/keys](https://openrouter.ai/keys).

Once you have the API key, you can use it with the `OpenAIProvider`:
Once you have the API key, you can use it with the `OpenRouterProvider`:

```python
from pydantic_ai import Agent
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ nav:
- api/models/function.md
- api/models/fallback.md
- api/models/wrapper.md
- api/profiles.md
- api/providers.md
- api/pydantic_graph/graph.md
- api/pydantic_graph/nodes.md
Expand Down
2 changes: 1 addition & 1 deletion pydantic_ai_slim/pydantic_ai/messages.py
Original file line number Diff line number Diff line change
Expand Up @@ -378,7 +378,7 @@ def model_response_object(self) -> dict[str, Any]:
"""Return a dictionary representation of the content, wrapping non-dict types appropriately."""
# gemini supports JSON dict return values, but no other JSON types, hence we wrap anything else in a dict
if isinstance(self.content, dict):
return tool_return_ta.dump_python(self.content, mode='json') # pyright: ignore[reportUnknownMemberType] # pragma: no cover
return tool_return_ta.dump_python(self.content, mode='json') # pyright: ignore[reportUnknownMemberType]
else:
return {'return_value': tool_return_ta.dump_python(self.content, mode='json')}

Expand Down
36 changes: 34 additions & 2 deletions pydantic_ai_slim/pydantic_ai/models/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,19 @@
from abc import ABC, abstractmethod
from collections.abc import AsyncIterator, Iterator
from contextlib import asynccontextmanager, contextmanager
from dataclasses import dataclass, field
from dataclasses import dataclass, field, replace
from datetime import datetime
from functools import cache
from functools import cache, cached_property

import httpx
from typing_extensions import Literal, TypeAliasType

from pydantic_ai.profiles import DEFAULT_PROFILE, ModelProfile, ModelProfileSpec

from .._parts_manager import ModelResponsePartsManager
from ..exceptions import UserError
from ..messages import ModelMessage, ModelRequest, ModelResponse, ModelResponseStreamEvent
from ..profiles._json_schema import JsonSchemaTransformer
from ..settings import ModelSettings
from ..tools import ToolDefinition
from ..usage import Usage
Expand Down Expand Up @@ -292,6 +295,8 @@ class ModelRequestParameters:
class Model(ABC):
"""Abstract class for a model."""

_profile: ModelProfileSpec | None = None

@abstractmethod
async def request(
self,
Expand Down Expand Up @@ -323,6 +328,13 @@ def customize_request_parameters(self, model_request_parameters: ModelRequestPar
In particular, this method can be used to make modifications to the generated tool JSON schemas if necessary
for vendor/model-specific reasons.
"""
if transformer := self.profile.json_schema_transformer:
model_request_parameters = replace(
model_request_parameters,
function_tools=[_customize_tool_def(transformer, t) for t in model_request_parameters.function_tools],
output_tools=[_customize_tool_def(transformer, t) for t in model_request_parameters.output_tools],
)

return model_request_parameters

@property
Expand All @@ -331,6 +343,18 @@ def model_name(self) -> str:
"""The model name."""
raise NotImplementedError()

@cached_property
def profile(self) -> ModelProfile:
"""The model profile."""
_profile = self._profile
if callable(_profile):
_profile = _profile(self.model_name)

if _profile is None:
return DEFAULT_PROFILE

return _profile

@property
@abstractmethod
def system(self) -> str:
Expand Down Expand Up @@ -584,3 +608,11 @@ def get_user_agent() -> str:
from .. import __version__

return f'pydantic-ai/{__version__}'


def _customize_tool_def(transformer: type[JsonSchemaTransformer], t: ToolDefinition):
schema_transformer = transformer(t.parameters_json_schema, strict=t.strict)
parameters_json_schema = schema_transformer.walk()
if t.strict is None:
t = replace(t, strict=schema_transformer.is_strict_compatible)
return replace(t, parameters_json_schema=parameters_json_schema)
4 changes: 4 additions & 0 deletions pydantic_ai_slim/pydantic_ai/models/anthropic.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
ToolReturnPart,
UserPromptPart,
)
from ..profiles import ModelProfileSpec
from ..providers import Provider, infer_provider
from ..settings import ModelSettings
from ..tools import ToolDefinition
Expand Down Expand Up @@ -118,6 +119,7 @@ def __init__(
model_name: AnthropicModelName,
*,
provider: Literal['anthropic'] | Provider[AsyncAnthropic] = 'anthropic',
profile: ModelProfileSpec | None = None,
):
"""Initialize an Anthropic model.

Expand All @@ -126,12 +128,14 @@ def __init__(
[here](https://docs.anthropic.com/en/docs/about-claude/models).
provider: The provider to use for the Anthropic API. Can be either the string 'anthropic' or an
instance of `Provider[AsyncAnthropic]`. If not provided, the other parameters will be used.
profile: The model profile to use. Defaults to a profile picked by the provider based on the model name.
"""
self._model_name = model_name

if isinstance(provider, str):
provider = infer_provider(provider)
self.client = provider.client
self._profile = profile or provider.model_profile

@property
def base_url(self) -> str:
Expand Down
Loading