-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Add builtin_tools
to Agent
#2102
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Co-authored-by: Marcelo Trylesinski <[email protected]>
…dantic#1752) Co-authored-by: Marcelo Trylesinski <[email protected]>
- Added builtin_tools field to ModelRequestParameters - Merged new output_mode and output_object fields from main - Updated test snapshots to include all fields - Resolved import conflicts to include both builtin tools and profiles
|
||
city: str | ||
country: str | ||
region: str |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what is region
?
More generally, are the required contents (of this field and the others on this class) vendor-specific in any way? Should we include examples?
part: ServerToolCallPart | ||
"""The server tool call to make.""" | ||
|
||
event_kind: Literal['server_tool_call'] = 'server_tool_call' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it makes me sad that some of our discriminators are snake_case (here) and some are kebab-case (part_kind). I guess you probably didn't introduce this inconsistency in this PR, but it feels bad. Maybe we should change it in v1 and just do value normalization during validation (i.e., replace any _
with -
or vice versa).
if part.executable_code is not None: | ||
items.append(ServerToolCallPart(args=part.executable_code.model_dump(), tool_name='code_execution')) | ||
elif part.code_execution_result is not None: | ||
# TODO(Marcelo): Is the idea to generate the tool_call_id on the `executable_code`, and then pass it here? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I feel like we can/should answer this question before merging?
@Kludex should we include code interpreter from openai as its accessible on the responses API or save that for a follow up given its a fairly complex set of types |
Co-authored-by: David Montague <[email protected]>
c4aa1ef
to
4376fc2
Compare
@Kludex unless we want to break these out into individual classes this feels like its in a pretty good spot for a first pass and to prevent it already feels like a giant PR |
@@ -259,7 +262,7 @@ def __init__( | |||
history_processors: Sequence[HistoryProcessor[AgentDepsT]] | None = None, | |||
) -> None: ... | |||
|
|||
def __init__( | |||
def __init__( # noqa: C901 adding builtin tools |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
def __init__( # noqa: C901 adding builtin tools | |
def __init__( # noqa: C901 |
if tool == 'web-search': | ||
self._builtin_tools.append(WebSearchTool()) | ||
else: | ||
self._builtin_tools.append(tool) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should either support only passing an AbstractBuiltinTool
, or include all built-in (to Pydantic AI) built-in tools as strings, meaning also code-execution
. If we support strings, I think we should have a dict of name to class in builtin_tools.py
, and use that here.
""" | ||
|
||
|
||
class UserLocation(TypedDict, total=False): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we call this WebSearchUserLocation
or something so it's clear it belongs to that class?
|
||
@dataclass(repr=False) | ||
class ServerToolReturnPart(BaseToolReturnPart): | ||
"""A tool return message from a server tool.""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a bit confusing we're calling them "built-in tools" where the user passes them, but "server tools" here. Is there a specific reason for that, or can we standardize on one name? I prefer BuiltinToolReturnPart
.
tool_call_id=item.id, | ||
) | ||
) | ||
elif isinstance(item, BetaCodeExecutionToolResultBlock): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we merge this with the elif isinstance(item, BetaWebSearchToolResultBlock):
above?
@@ -307,6 +311,8 @@ def __init__( | |||
output_retries: The maximum number of retries to allow for output validation, defaults to `retries`. | |||
tools: Tools to register with the agent, you can also register tools via the decorators | |||
[`@agent.tool`][pydantic_ai.Agent.tool] and [`@agent.tool_plain`][pydantic_ai.Agent.tool_plain]. | |||
builtin_tools: The builtin tools that the agent will use. This depends on the model, as some models may not | |||
support certain tools. On models that don't support certain tools, the tool will be ignored. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I feel like silently ignoring built-in tools is unexpected, because the model is not going to behave as intended if it can't search the web or execute code. I'd prefer to raise an error from the model class if it sees an unsupported built-in tool, similarly to how we do for unsupported file/binary content types.
tool_call_id = generate_tool_call_id() | ||
items.append( | ||
ServerToolCallPart( | ||
tool_name=tool.type, args=tool.arguments, model_name='groq', tool_call_id=tool_call_id |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do we need the model_name
for?
@@ -734,6 +754,7 @@ async def _responses_create( | |||
) -> responses.Response | AsyncStream[responses.ResponseStreamEvent]: | |||
tools = self._get_tools(model_request_parameters) | |||
tools = list(model_settings.get('openai_builtin_tools', [])) + tools |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we deprecate this setting and push people to use the new builtin_tools
?
This setting currently supports FileSearchToolParam
and ComputerToolParam
as well, should we implement those as AbstractBuiltinTool
s?
Should we have a way to build an AbstractBuiltinTool
with arbitrary built-in tool JSON supported by a given model, so users can start using them without having to wait for us to add support to the model class?
@@ -286,4 +286,4 @@ skip = '.git*,*.svg,*.lock,*.css,*.yaml' | |||
check-hidden = true | |||
# Ignore "formatting" like **L**anguage | |||
ignore-regex = '\*\*[A-Z]\*\*[a-z]+\b' | |||
ignore-words-list = 'asend,aci' | |||
ignore-words-list = 'asend,aci,Hemishpere,synchonizing' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need those? Both of those clearly look like typos
Fixes test and merge conflicts for #1722
Closes #840