Add support for GPT-5 Free-Form Function Calling and Context Free Grammar constraints over tools

docs/models/openai.md

@@ -202,6 +202,131 @@ print(result2.output)
 #> This is an excellent joke invented by Samuel Colvin, it needs no explanation.
 ```
 
+### Freeform Function Calling
+
+GPT‑5 can now send raw text payloads - anything from Python scripts to SQL queries - to your custom tool without wrapping the data in JSON using freeform function calling. This differs from classic structured function calls, giving you greater flexibility when interacting with external runtimes such as:
+
+* code execution with sandboxes (Python, C++, Java, …)
+* SQL databases
+* Shell environments
+* Configuration generators
+
+Note that freeform function calling does NOT support parallel tool calling.
+
+You can enable freeform function calling for a tool using the `text_format` parameter when creating your tool. To use this the tool must take a single string argument (other than the runtime context) and the model must be one of the GPT-5 responses models. For example:
+
+```python
+from pydantic_ai import Agent
+from pydantic_ai.models.openai import OpenAIResponsesModel
+
+model = OpenAIResponsesModel('gpt-5')  # (1)!
+agent = Agent(model)
+
+@agent.tool_plain(text_format='text')  # (2)!
+def freeform_tool(sql: str): ...
+```
+
+1. The GPT-5 family (`gpt-5`, `gpt-5-mini`, `gpt-5-nano`) all support freeform function calling.
+2. If the tool or model cannot be used with freeform function calling then it will be invoked in the normal way.
+
+You can read more about this function calling style in the [OpenAI documentation](https://cookbook.openai.com/examples/gpt-5/gpt-5_new_params_and_tools#2-freeform-function-calling).
+
+#### Context Free Grammar
+
+A tool that queries an SQL database can only accept valid SQL. The freeform function calling of GPT-5 supports generation of valid SQL for this situation by constraining the generated text using a context free grammar.
+
+A context‑free grammar is a collection of production rules that define which strings belong to a language. Each rule rewrites a non‑terminal symbol into a sequence of terminals (literal tokens) and/or other non‑terminals, independent of surrounding context—hence context‑free. CFGs can capture the syntax of most programming languages and, in OpenAI custom tools, serve as contracts that force the model to emit only strings that the grammar accepts.
+
+##### Regular Expression
+
+The grammar can be written as either a regular expression:
+
+
+```python
+from pydantic_ai import Agent, FunctionTextFormat
+from pydantic_ai.models.openai import OpenAIResponsesModel
+
+model = OpenAIResponsesModel('gpt-5')  # (1)!
+agent = Agent(model)
+
+timestamp_grammar_definition = r'^\d{4}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01]) (?:[01]\d|2[0-3]):[0-5]\d$'
+
+@agent.tool_plain(text_format=FunctionTextFormat(syntax='regex', grammar=timestamp_grammar_definition))  # (2)!
+def timestamp_accepting_tool(timestamp: str): ...
+```
+
+1. The GPT-5 family (`gpt-5`, `gpt-5-mini`, `gpt-5-nano`) all support freeform function calling with context free grammar constraints. Unfortunately `gpt-5-nano` often struggles with these calls.
+2. If the tool or model cannot be used with freeform function calling then it will be invoked in the normal way, which may lead to invalid input.
+
+##### LARK
+
+Or as a [LARK](https://lark-parser.readthedocs.io/en/latest/how_to_use.html) grammar:
+
+```python
+from pydantic_ai import Agent, FunctionTextFormat
+from pydantic_ai.models.openai import OpenAIResponsesModel
+
+model = OpenAIResponsesModel('gpt-5')  # (1)!
+agent = Agent(model)
+
+timestamp_grammar_definition = r'''
+start: timestamp
+
+timestamp: YEAR "-" MONTH "-" DAY " " HOUR ":" MINUTE
+
+%import common.DIGIT
+
+YEAR: DIGIT DIGIT DIGIT DIGIT
+MONTH: /(0[1-9]|1[0-2])/
+DAY: /(0[1-9]|[12]\d|3[01])/
+HOUR: /([01]\d|2[0-3])/
+MINUTE: /[0-5]\d/
+'''
+
+@agent.tool_plain(text_format=FunctionTextFormat(syntax='lark', grammar=timestamp_grammar_definition))  # (2)!
+def i_like_iso_dates(date: str): ...
+```
+
+1. The GPT-5 family (`gpt-5`, `gpt-5-mini`, `gpt-5-nano`) all support freeform function calling with context free grammar constraints. Unfortunately `gpt-5-nano` often struggles with these calls.
+2. If the tool or model cannot be used with freeform function calling then it will be invoked in the normal way, which may lead to invalid input.
+
+There is a limit to the grammar complexity that GPT-5 supports, as such it is important to test your grammar.
+
+Freeform function calling, with or without a context free grammar, can be used with the output tool for the agent:
+
+```python
+from pydantic_ai import Agent, FunctionTextFormat
+from pydantic_ai.models.openai import OpenAIResponsesModel
+from pydantic_ai.output import ToolOutput
+
+sql_grammar_definition = r'''
+start: select_stmt
+select_stmt: "SELECT" select_list "FROM" table ("WHERE" condition ("AND" condition)*)?
+select_list: "*" | column ("," column)*
+table: "users" | "orders"
+column: "id" | "user_id" | "name" | "age"
+condition: column ("=" | ">" | "<") (NUMBER | STRING)
+%import common.NUMBER
+%import common.ESCAPED_STRING -> STRING
+%import common.WS
+%ignore WS
+''' # (1)!
+
+output_tool = ToolOutput(str, text_format=FunctionTextFormat(syntax='lark', grammar=sql_grammar_definition))
+model = OpenAIResponsesModel('gpt-5')
+agent = Agent(model, output_type=output_tool)
+```
+
+1. An inline SQL grammar definition would be quite extensive and so this simplified version has been written, you can find an example SQL grammar [in the openai example](https://cookbook.openai.com/examples/gpt-5/gpt-5_new_params_and_tools#33-example---sql-dialect--ms-sql-vs-postgresql). There are also example grammars in the [lark repo](https://github.com/lark-parser/lark/blob/master/examples/composition/json.lark). Remember that a simpler grammar that matches your DDL will be easier for GPT-5 to work with and will result in fewer semantically invalid results.
+
+##### Best Practices
+
+You can find recommended best practices in the [OpenAI Cookbook](https://cookbook.openai.com/examples/gpt-5/gpt-5_new_params_and_tools#35-best-practices).
+
+* [Lark Docs](https://lark-parser.readthedocs.io/en/stable/)
+* [Lark IDE](https://www.lark-parser.org/ide/)
+* [OpenAI Cookbook on CFG](https://cookbook.openai.com/examples/gpt-5/gpt-5_new_params_and_tools#3-contextfree-grammar-cfg)
+
 ## OpenAI-compatible Models
 
 Many providers and models are compatible with the OpenAI API, and can be used with `OpenAIChatModel` in Pydantic AI.

pydantic_ai_slim/pydantic_ai/__init__.py

@@ -90,7 +90,18 @@
 )
 from .run import AgentRun, AgentRunResult, AgentRunResultEvent
 from .settings import ModelSettings
-from .tools import DeferredToolRequests, DeferredToolResults, RunContext, Tool, ToolApproved, ToolDefinition, ToolDenied
+from .tools import (
+    DeferredToolRequests,
+    DeferredToolResults,
+    LarkTextFormat,
+    RegexTextFormat,
+    RunContext,
+    TextFormat,
+    Tool,
+    ToolApproved,
+    ToolDefinition,
+    ToolDenied,
+)
 from .toolsets import (
     AbstractToolset,
     ApprovalRequiredToolset,
@@ -191,6 +202,9 @@
     'DeferredToolResults',
     'ToolApproved',
     'ToolDenied',
+    'TextFormat',
+    'RegexTextFormat',
+    'LarkTextFormat',
     # toolsets
     'AbstractToolset',
     'ApprovalRequiredToolset',

pydantic_ai_slim/pydantic_ai/_output.py

@@ -31,7 +31,7 @@
     ToolOutput,
     _OutputSpecItem,  # type: ignore[reportPrivateUsage]
 )
-from .tools import GenerateToolJsonSchema, ObjectJsonSchema, ToolDefinition
+from .tools import GenerateToolJsonSchema, ObjectJsonSchema, TextFormat, ToolDefinition
 from .toolsets.abstract import AbstractToolset, ToolsetTool
 
 if TYPE_CHECKING:
@@ -656,6 +656,7 @@ def __init__(
         name: str | None = None,
         description: str | None = None,
         strict: bool | None = None,
+        text_format: Literal['plain'] | TextFormat | None = None,
     ):
         if inspect.isfunction(output) or inspect.ismethod(output):
             self._function_schema = _function_schema.function_schema(output, GenerateToolJsonSchema)
@@ -711,6 +712,7 @@ def __init__(
                 description=description,
                 json_schema=json_schema,
                 strict=strict,
+                text_format=text_format,
             )
         )
 
@@ -979,19 +981,23 @@ def build(
             name = None
             description = None
             strict = None
+            text_format = None
             if isinstance(output, ToolOutput):
                 # do we need to error on conflicts here? (DavidM): If this is internal maybe doesn't matter, if public, use overloads
                 name = output.name
                 description = output.description
                 strict = output.strict
+                text_format = output.text_format
 
                 output = output.output
 
             description = description or default_description
             if strict is None:
                 strict = default_strict
 
-            processor = ObjectOutputProcessor(output=output, description=description, strict=strict)
+            processor = ObjectOutputProcessor(
+                output=output, description=description, strict=strict, text_format=text_format
+            )
             object_def = processor.object_def
 
             if name is None:
@@ -1016,6 +1022,7 @@ def build(
                 description=description,
                 parameters_json_schema=object_def.json_schema,
                 strict=object_def.strict,
+                text_format=object_def.text_format,
                 outer_typed_dict_key=processor.outer_typed_dict_key,
                 kind='output',
             )

pydantic_ai_slim/pydantic_ai/agent/__init__.py

@@ -8,7 +8,7 @@
 from collections.abc import AsyncIterator, Awaitable, Callable, Iterator, Sequence
 from contextlib import AbstractAsyncContextManager, AsyncExitStack, asynccontextmanager, contextmanager
 from contextvars import ContextVar
-from typing import TYPE_CHECKING, Any, ClassVar, cast, overload
+from typing import TYPE_CHECKING, Any, ClassVar, Literal, cast, overload
 
 from opentelemetry.trace import NoOpTracer, use_span
 from pydantic.json_schema import GenerateJsonSchema
@@ -50,6 +50,7 @@
     DocstringFormat,
     GenerateToolJsonSchema,
     RunContext,
+    TextFormat,
     Tool,
     ToolFuncContext,
     ToolFuncEither,
@@ -1012,6 +1013,7 @@ def tool(
         require_parameter_descriptions: bool = False,
         schema_generator: type[GenerateJsonSchema] = GenerateToolJsonSchema,
         strict: bool | None = None,
+        text_format: Literal['plain'] | TextFormat | None = None,
         sequential: bool = False,
         requires_approval: bool = False,
         metadata: dict[str, Any] | None = None,
@@ -1029,6 +1031,7 @@ def tool(
         require_parameter_descriptions: bool = False,
         schema_generator: type[GenerateJsonSchema] = GenerateToolJsonSchema,
         strict: bool | None = None,
+        text_format: Literal['plain'] | TextFormat | None = None,
         sequential: bool = False,
         requires_approval: bool = False,
         metadata: dict[str, Any] | None = None,
@@ -1076,6 +1079,8 @@ async def spam(ctx: RunContext[str], y: float) -> float:
             schema_generator: The JSON schema generator class to use for this tool. Defaults to `GenerateToolJsonSchema`.
             strict: Whether to enforce JSON schema compliance (only affects OpenAI).
                 See [`ToolDefinition`][pydantic_ai.tools.ToolDefinition] for more info.
+            text_format: Used to invoke the function using freeform function calling (only affects OpenAI).
+                See [`ToolDefinition`][pydantic_ai.tools.ToolDefinition] for more info.
             sequential: Whether the function requires a sequential/serial execution environment. Defaults to False.
             requires_approval: Whether this tool requires human-in-the-loop approval. Defaults to False.
                 See the [tools documentation](../deferred-tools.md#human-in-the-loop-tool-approval) for more info.
@@ -1096,6 +1101,7 @@ def tool_decorator(
                 require_parameter_descriptions=require_parameter_descriptions,
                 schema_generator=schema_generator,
                 strict=strict,
+                text_format=text_format,
                 sequential=sequential,
                 requires_approval=requires_approval,
                 metadata=metadata,
@@ -1119,6 +1125,7 @@ def tool_plain(
         require_parameter_descriptions: bool = False,
         schema_generator: type[GenerateJsonSchema] = GenerateToolJsonSchema,
         strict: bool | None = None,
+        text_format: Literal['plain'] | TextFormat | None = None,
         sequential: bool = False,
         requires_approval: bool = False,
         metadata: dict[str, Any] | None = None,
@@ -1136,6 +1143,7 @@ def tool_plain(
         require_parameter_descriptions: bool = False,
         schema_generator: type[GenerateJsonSchema] = GenerateToolJsonSchema,
         strict: bool | None = None,
+        text_format: Literal['plain'] | TextFormat | None = None,
         sequential: bool = False,
         requires_approval: bool = False,
         metadata: dict[str, Any] | None = None,
@@ -1183,6 +1191,8 @@ async def spam(ctx: RunContext[str]) -> float:
             schema_generator: The JSON schema generator class to use for this tool. Defaults to `GenerateToolJsonSchema`.
             strict: Whether to enforce JSON schema compliance (only affects OpenAI).
                 See [`ToolDefinition`][pydantic_ai.tools.ToolDefinition] for more info.
+            text_format: Used to invoke the function using freeform function calling (only affects OpenAI).
+                See [`ToolDefinition`][pydantic_ai.tools.ToolDefinition] for more info.
             sequential: Whether the function requires a sequential/serial execution environment. Defaults to False.
             requires_approval: Whether this tool requires human-in-the-loop approval. Defaults to False.
                 See the [tools documentation](../deferred-tools.md#human-in-the-loop-tool-approval) for more info.
@@ -1201,6 +1211,7 @@ def tool_decorator(func_: ToolFuncPlain[ToolParams]) -> ToolFuncPlain[ToolParams
                 require_parameter_descriptions=require_parameter_descriptions,
                 schema_generator=schema_generator,
                 strict=strict,
+                text_format=text_format,
                 sequential=sequential,
                 requires_approval=requires_approval,
                 metadata=metadata,