add support for background responses

[SergeyMenshykh]

This PR adds a model for supporting background responses (long-running operations), updates the chat completion model to use it, and integrates this functionality into OpenAIResponsesChatClient.

Background responses use a continuation token returned by the GetResponseAsync and GetStreamingResponseAsync methods in the ChatResponse and ChatResponseUpdate classes, respectively, when background responses are enabled and supported by the chat client.

The continuation token contains all necessary details to enable polling for a background response using the non-streaming GetResponseAsync method. It also allows resuming a streamed background response with the GetStreamingResponseAsync method if the stream is interrupted. In both cases, the continuation token obtained from the initial chat result or the streamed update received before the interruption should be supplied as input for follow-up calls.

When a background response has completed, failed, or cannot proceed further (for example, when user input is required), the continuation token returned by either method will be null, signaling to consumers that processing is complete and there is nothing to poll or resume. The result returned by either method can then be used for further processing.

Non-streaming API:

var chatOptions = new ChatOptions
{
    BackgroundResponsesOptions = new() { Allow = true }
};

// Get initial response with continuation token
var response = await chatClient.GetResponseAsync("What's the biggest animal?", chatOptions);

// Continue to poll until the final response is received
while (response.ContinuationToken is not null)
{
    response = await chatClient.GetResponseAsync(response.ContinuationToken, chatOptions);
}

Console.WriteLine(response);

Streaming API:

ChatOptions chatOptions = new()
{
    BackgroundResponsesOptions = new BackgroundResponsesOptions { Allow = true },
};

string response = "";
ResumptionToken? continuationToken = null;

await foreach (var update in chatClient.GetStreamingResponseAsync("What is the capital of France?", chatOptions))
{
    response += update;

    // Simulate an interruption
    continuationToken = update.ContinuationToken;
    break;
}

// Resume streaming from the point captured by the continuation token
await foreach (var update in chatClient.GetStreamingResponseAsync(continuationToken!, chatOptions))
{
    response += update;
}

Console.WriteLine(response);

Microsoft Reviewers: Open in CodeFlow

CC: @westey-m

[SergeyMenshykh]

@dotnet-policy-service agree company="Microsoft"

[Copilot]

Pull Request Overview

This PR introduces support for background responses (long-running operations) in the AI chat framework. The feature enables clients to initiate operations that can be polled or resumed from interruption points using continuation tokens.

• Adds ResumptionToken base class and BackgroundResponsesOptions for continuation token support
• Updates ChatOptions, ChatResponse, and ChatResponseUpdate with background response properties
• Implements background response functionality in OpenAIResponsesChatClient with polling and stream resumption

Reviewed Changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
ResumptionToken.cs	Introduces base class for continuation tokens with byte serialization
BackgroundResponsesOptions.cs	Defines options for enabling background responses
ChatOptions.cs	Adds continuation token and background response options properties
ChatResponse.cs	Adds continuation token property for background response polling
ChatResponseUpdate.cs	Adds continuation token property for stream resumption
ChatClientExtensions.cs	Provides extension methods for background response operations
OpenAIResponsesContinuationToken.cs	Implements OpenAI-specific continuation token with JSON serialization
OpenAIResponsesChatClient.cs	Core implementation of background response support with polling and streaming
FunctionInvokingChatClient.cs	Updates to handle continuation tokens in function calling scenarios

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[stephentoub]

Should it be Enabled instead of Allow?

[SergeyMenshykh]

It could be, though there is an opinion that it may mislead consumers into thinking that setting that property should enable or disable background responses. In reality, it may not have any effect because the chat client implementation may not support background responses, or the underlying SDK does not allow for controlling the behavior. Allow is more permissive than Enable and is intended to communicate that it can enable or disable the feature if supported; otherwise, it will have no effect. On the other hand, the Enable semantic presumes that the values should be respected once set.

[stephentoub]

What is the purpose of the options being accepted here?

[SergeyMenshykh]

Good question. We need to supply functions for subsequent calls if they were advertised in the initial call; otherwise, the called tools won't be found by FICC.