8000 Add `to_chat_completions` method by Kludex · Pull Request #2041 · pydantic/pydantic-ai · GitHub
[go: up one dir, main page]
Skip to content

Add to_chat_completions method #2041

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Add to_chat_completions method #2041

wants to merge 1 commit into from
+250 −0

Conversation

Kludex
Copy link
Member
@Kludex Kludex commented Jun 20, 2025 •
edited
Loading

The idea is to be able to expose a PydanticAI agent in a Chat Completions endpoint.

from pydantic_ai import Agent

agent = Agent('openai:gpt-4.1', instructions='You always reply in Italian.')
app = agent.to_chat_completions()

Then you run this as uvicorn main:app.

In this PR, we will not support tools as input on the endpoint.

@Kludex Kludex marked this pull request as draft June 20, 2025 11:53
@github-actions GitHub Actions
Copy link

Docs Preview

commit: 4ac2edd
Preview URL: https://55c90f2e-pydantic-ai-previews.pydantic.workers.dev

@DouweM DouweM mentioned this pull request Jun 20, 2025
Draft
17 tasks
@st3fan
Copy link
st3fan commented Jun 20, 2025 •
edited
Loading

@Kludex this is great! I think it this will help in a big way with testing and integrating with exising chat ui frameworks.

I have two thoughts:

Is it possible to support streaming? It is a much better user experience when responses that can take a while come in gradually, typewriters style.

How do you feel about exposing the inline chat_completions() function in to_chat_completions () as a public api?

The reason I am asking this is because I would like to use this code in a bigger app. Right now to_chat_completions() returns a full FastAPI application, but I think what would be more useful is something smaller that I can include in an existing application.

Exposing the function may be good enough for that. Another option would be to expose it as a fastapi.APIRouter instead.

This would allow to more easily include it in a bigger app and for example configure middleware, telemetry, different routes, etc.

@Wh1isper
Copy link
Contributor

If I understand correctly, in the future we can convert any model endpoints to openai-compatible APIs via pydantic-ai, by way of official support?

@physicsrob
Copy link

I'd love to see functionality similar to this in pydantic-AI, but I do worry about the very real limitations imposed by the constrains of the completions API requiring all state to be maintained on the client side. It seems like a big limitation on what can be expressed in the completions API, and I imagine that's a big part of why OpenAI has been moving away from it (most recently to the Responses API).

How do you plan on handling, for instance, dependencies? I often have state stored in my Agent's dependencies object that evolves over the turns of a conversation. That seems impossible to support with the completions API, right? (Wouldn't any dependencies be impossible with the completions API?)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development
3B05

Successfully merging this pull request may close these issues.
4 participants
@Kludex @st3fan @Wh1isper @physicsrob
0