Skip to content

Conversation

dmontagu
Copy link
Contributor

@dmontagu dmontagu commented Dec 18, 2024

Add UsageLimits and raise errors when they are exceeded.

This could use more tests before merging, I wouldn't say I'm confident it is working properly in all cases yet. But I'm also not confident enough in the approach to really want to put a lot more effort into testing yet. (Though I'll note I did already add one simple test, and updated the FunctionModel and TestModel to track token counts in the usage.)

Also needs docs.

Copy link

cloudflare-workers-and-pages bot commented Dec 18, 2024

Deploying pydantic-ai with  Cloudflare Pages  Cloudflare Pages

Latest commit: 849cbfe
Status: ✅  Deploy successful!
Preview URL: https://3553c835.pydantic-ai.pages.dev
Branch Preview URL: https://usage-limits.pydantic-ai.pages.dev

View logs

Copy link
Member

@samuelcolvin samuelcolvin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise I think this looks great.

Base automatically changed from rename-cost-to-usage to main December 18, 2024 16:32
Copy link
Contributor Author

@dmontagu dmontagu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not done reviewing, but here's a few copy edits on the docs..

@dmontagu
Copy link
Contributor Author

Once this is merged let's create an issue talking about related improvements we might make:

  • Automatically setting max_tokens in requests based on the limits and current usage
  • Automatically removing function tools from the list of choices provided to the model if you are on the last allowed request

I don't think we want to do either of those things unconditionally, but it's probably worth creating a tracking issue for discussion about whether/how to make such changes

Copy link
Contributor

@hyperlint-ai hyperlint-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The style guide flagged several spelling errors that seemed like false positives. We skipped posting inline suggestions for the following words:

  • [Ee]rroring

@dmontagu dmontagu enabled auto-merge (squash) December 18, 2024 20:22
@dmontagu dmontagu merged commit c23a286 into main Dec 18, 2024
15 checks passed
@dmontagu dmontagu deleted the usage-limits branch December 18, 2024 20:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants