Amazon Nova (Bedrock) limitations with tool schema

[chasewalden]

Initial Checks

• I confirm that I'm using the latest version of Pydantic AI
• I confirm that I searched for my issue in https://github.com/pydantic/pydantic-ai/issues before opening this issue

Description

Was trying to use one of the Nova foundation models on Bedrock, but ran into issues with tool-calling and structured output.

The Nova models seem to be a bit more strict (or less capable, depending who you ask) with regards to tool schemas. The top-level schema must be "type": "object" and the only fields allowed are "type", "properties", and "required". All other fields are disallowed and result in a hard API error.

See the second NOTE on the Nova docs page

Its a pretty annoying limitation, but means that many of the fields that Pydantic generates with the JSON Schema are not allowed. Most critically, `"$defs" are not allowed, so the schema would need to be transformed somehow to allow tool calling.

I know Nova is probably low-priority (and I seem to recall reading somewhere that Anthropic Bedrock was the main use-case for now), but there is nothing that would indicate that this wouldn't work. This might need to be called out in the docs and/or generate a warning/error when trying to use tools and structured output with a Nova model.

Example Code

class Something(BaseModel):
   foo: str
   bar: list[int]

class SomethingElse(BaseModel):
    something: Something
    extra: dict[str, str]

agent = Agent(
    model="bedrock:us.amazon.nova-pro-v1:0", 
    # model="bedrock:us.amazon.nova-lite-v1:0",   # or this
    # model="bedrock:us.amazon.nova-micro-v1:0",  # or this
    instructions="You are a helpful assistant.",
    output_type=Something,
)

@agent.tool_plain
def frobnicate(something_else: SomethingElse):
   print({something_else=!r})

_ = agent.run_sync("frobnicate, and then do what you need to do...")

Python, Pydantic AI & LLM client version

`python>=3.13`
`pydantic-ai==0.1.8`
`pydantic==2.11.3`
`boto3==1.38.4` (Bedrock)

[DouweM]

We already implement something similar for OpenAI at https://github.com/pydantic/pydantic-ai/blob/main/pydantic_ai_slim/pydantic_ai/models/openai.py#L967 and Gemini at https://github.com/pydantic/pydantic-ai/blob/main/pydantic_ai_slim/pydantic_ai/models/gemini.py#L764, so I'm not surprised other models need something similar.

We'd be open to a PR that performs a similar transformation for Bedrock, without changing the behavior of the schema, as we shouldn't cripple the schema just for Nova if other models do behave fine with the current complete schemas.

If that's not going to be possible since this is very model-specific, you can see if you can use the schema_generator attribute on Tool (and argument to @agent.tool_plain) to point at a custom generator, that can wrap the default GenerateToolJsonSchema. Once #1474 lands, you'd also be able to modify the ToolDefinition's parameters_json_schema in one place. If you don't want to pass schema_generator manually on all agent.tool_plain calls, you could define your own decorator which calls that one with the additional property.

[aristideubertas]

@DouweM I have also experienced this problem, would you be able to give a snippet of your reccomended approach? Or if something could be added to the docs.

This is also model specific, not Bedrock-wide, because Claude works fine.

[chasewalden]

@DouweM , overriding the schema generator seems like it'll work fine for tools but will not work for structured output.

[tamir-alltrue-ai]

Can confirm that this also affects llama models served on bedrock.

With Llama-3.3-70B I'm able to get structured output, but not tool calls.

[DouweM]

@aristideubertas @chasewalden @tamir-alltrue-ai I've shared an example of a similar JSON schema modification to make Qwen on Together.ai work with the OpenAIModel at #1659 (comment). Let me know if you're having trouble adjusting it for BedrockModel. You may need to add additional transformations besides everything _OpenAIJsonSchema does + prefer_inlined_defs. I suggest having a look at the classes for OpenAI and Gemini I linked in my previous comment.

If you get something to work, please share it here. As I mentioned in #1649 (comment), it may be time to start having model-specific logic in the BedrockModel class, possibly by separating the concept of the API from that of the underlying model. We've seen a similar need with OpenAIModel, which is used for many different models on OpenAI-compatible APIs, that don't always support exactly the same schemas as OpenAI's models, or features of OpenAI's API.

[DouweM]

@chasewalden With the changes in #1835, which include automatic JSON schema transformer selection based on the us.amazon. prefix in the model name, you should be able to drop any workarounds you implemented, and use just this as you had it before:

agent = Agent("bedrock:us.amazon.nova-pro-v1:0",)

If you have a chance, could you verify that that works as expected?

[chasewalden]

@chasewalden With the changes in #1835, which include automatic JSON schema transformer selection based on the us.amazon. prefix in the model name, you should be able to drop any workarounds you implemented, and use just this as you had it before:

agent = Agent("bedrock:us.amazon.nova-pro-v1:0",)
If you have a chance, could you verify that that works as expected?

I'll give it a look. We've stopped using Nova currently due to the extremely small output token limit (5k tokens), but I can try to see if this behaves once #1835 lands