Changes to get reasoning to work with the responses API

[mwildehahn]

No description provided.

[hyperlint-ai]

PR Change Summary

Updated the responses API to utilize new message handling for improved clarity and functionality.

• Replaced all_messages() with new_messages() in WriteEmail class
• Updated message handling in ask_agent and evaluate_agent processes
• Enhanced state management for agent messages

Modified Files

• docs/graph.md

---

How can I customize these reviews?

Check out the Hyperlint AI Reviewer docs for more information on how to customize the review.

If you just want to ignore it on this PR, you can add the hyperlint-ignore label to the PR. Future changes won't trigger a Hyperlint review.

Note specifically for link checks, we only check the first 30 links in a file and we cache the results for several hours (for instance, if you just added a page, you might experience this). Our recommendation is to add hyperlint-ignore to the PR to ignore the link check for this PR.

[Kludex]

Why was this changed?

[mwildehahn]

#1142 (comment)

[mwildehahn]

feel free to rip whatever from this -- i honestly haven't used the responses api before this, was just hacking things to get it working. having id and tool_call_id is very confusing. maybe this is better as signature 🤷

[Kludex]

I mean, this doesn't seem related to the idea of the ThinkingPart PR. Is it? Or it's just something you noticed while looking at it?

[mwildehahn]

Without this the openai responses api was failing when sending back the ResponseReasoningItemParam.

My interpretation was:

• reasoning part has to precede the "tool call" (not totally sure why)
• this is linked together via these id properties
• the rs_<idval> and fc_<idval> have the same <idval>
• you then also need to provide call_id or you get an error about not returning the result

[mwildehahn]

ResponseCompletedEvent(response=Response(id='resp_680bb98fe86c8191bd35ca119209aaf60541f9d603665d09', created_at=1745598863.0, error=None, incomplete_details=None, instructions=None, metadata={}, model='o4-mini-2025-04-16', object='response', output=[ResponseReasoningItem(id='rs_680bb990ecd48191b2a5649341121d580541f9d603665d09', summary=[], type='reasoning', status=None), ResponseOutputMessage(id='msg_680bb992641481918ad33854dce4c94c0541f9d603665d09', content=[ResponseOutputText(annotations=[], text='Hello! How can I help you today?', type='output_text')], role='assistant', status='completed', type='message')],

i'm also seeing it fail if i try and provide reasoning back to the model via ResponseReasoningItemParam and there is no tool call. I think it's required that whenever we provide reasoning with rs_680bb990ecd48191b2a5649341121d580541f9d603665d09 we have to then provide msg_680bb992641481918ad33854dce4c94c0541f9d603665d09 or fc_680bb992641481918ad33854dce4c94c0541f9d603665d09 to associate the reasoning with.

I had to add id to TextPart as well to get this to work.

[Kludex]

Can you give me the snippet you used to test the things you are saying things fail? 🙏

It would help me to move things faster.

[mwildehahn]

Simple example with no tool calls:

import asyncio
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIResponsesModel, OpenAIResponsesModelSettings

agent = Agent(
    model=OpenAIResponsesModel("o4-mini-2025-04-16"),
    model_settings=OpenAIResponsesModelSettings(
        openai_reasoning_summary="detailed",
        openai_reasoning_effort="high",
    ),
    system_prompt="You are a helpful assistant that answers questions. Think carefully about each response before responding to the user.",
    output_type=str,
)


#     raise ModelHTTPError(status_code=status_code, model_name=self.model_name,
#     body=e.body) from epydantic_ai.exceptions.ModelHTTPError: status_code:
#     400, model_name: o4-mini-2025-04-16, body: {'message': "Invalid
#     'input[3].id': empty string. Expected a string with minimum length 1, but
#     got an empty string instead.", 'type': 'invalid_request_error', 'param':
#     'input[3].id', 'code': 'empty_string'}
async def run():
    inputs = ["What is 4 * 4 + 5 * 3 / 5?", "What's the previous answer * 5 * 6?"]
    history = []
    for i in inputs:
        async with agent.iter(user_prompt=i, message_history=history) as run:
            async for node in run:
                if Agent.is_model_request_node(node):
                    async with node.stream(ctx=run.ctx) as stream:
                        async for chunk in stream:
                            print(chunk)
        if run.result:
            history.extend(run.result.new_messages())


def main():
    asyncio.run(run())


if __name__ == "__main__":
    main()

If you fix this, then you'll get an error about the reasoning not being followed by the correct item (which is the missing id on the response message). Then if you fix that and add a tool call, it will fail with missing id and call_id on the tool call.

From debugging it:

• all items in the responses API have an id/item_id while streaming in from the server
• sending these back as history requires the id to be present

It seems like TextPart / ToolCallPart / ThinkingPart all need to have an id attribute added to them.

It's also more helpful if you print the raw chunks from openai since the current pydantic chunks don't represent what is being returned.

LMK if I can help more!

[mwildehahn]

If you run that with uv add "git+https://github.com/mwildehahn/pydantic-ai.git@0b0f0e2d3831e83e11a34401d0b4f9f33513ea01", it will work

[Kludex]

I see... 😞

I think we can make this change before we do the following. Let me see.