FDP Evals AOAI compliant Sync API #37790
Conversation
Next Steps to Merge✅ All automated merging requirements have been met! To get your PR merged, see aka.ms/azsdk/specreview/merge.Comment generated by summarize-checks workflow run. |
API Change CheckAPIView identified API level changes in this PR and created the following API reviews
|
kseager
force-pushed
the
observability/oai-sync-api
branch
from
f090db3 to
bfc7ec8
Compare
|
wait for BYOD |
| context?: string; | ||
|
|
||
| /** Optional list of tools or resources utilized during the evaluation or generation of the response. */ | ||
| tools?: string[]; |
There was a problem hiding this comment.
Do we really need this?
I mean if we will support tools then probably
- it will not be string[]
- we need to add ToolDefinitions
- ToolCalls
each with complex object, I mean not a string
There was a problem hiding this comment.
yes its expected from copilot
There was a problem hiding this comment.
Could we make tools: {}[]? Agent tools will be objects.
There was a problem hiding this comment.
Then we need to align on the schema of that object, ideally it would align with the toolcall passed to EvaluationEngine https://msdata.visualstudio.com/Vienna/_git/vienna?path=/src/azureml-api/src/RAISvc/Contracts/AgentsV2/Messages/Content/ToolCallContent.cs
There was a problem hiding this comment.
for this one because anything related to Agents is highly variable - can we leave this one as object and internally you assume the contract you pointed to?
| context?: string; | ||
|
|
||
| /** Optional list of tools or resources utilized during the evaluation or generation of the response. */ | ||
| tools?: string[]; |
There was a problem hiding this comment.
Then we need to align on the schema of that object, ideally it would align with the toolcall passed to EvaluationEngine https://msdata.visualstudio.com/Vienna/_git/vienna?path=/src/azureml-api/src/RAISvc/Contracts/AgentsV2/Messages/Content/ToolCallContent.cs
Data Plane API Specification Update Pull Request
Tip
Overwhelmed by all this guidance? See the
Getting helpsection at the bottom of this PR description.PR review workflow diagram
Please understand this diagram before proceeding. It explains how to get your PR approved & merged.
API Info: The Basics
Most of the information about your service should be captured in the issue that serves as your API Spec engagement record.
Is this review for (select one):
note for the last API
azure-rest-api-specs/specification/ai/Azure.AI.Projects/evaluations/models.tsp
Line 353 in ff1ec65
Change Scope
This section will help us focus on the specific parts of your API that are new or have been modified.
Please share a link to the design document for the new APIs, a link to the previous API Spec document (if applicable), and the root paths that have been updated.
Viewing API changes
For convenient view of the API changes made by this PR, refer to the URLs provided in the table
in the
Generated ApiViewcomment added to this PR. You can use ApiView to show API versions diff.Suppressing failures
If one or multiple validation error/warning suppression(s) is detected in your PR, please follow the
Swagger-Suppression-Process
to get approval.
Release planner
A release plan should have been created. If not, please create one as it will help guide you through the REST API and SDK creation process.
❔Got questions? Need additional info?? We are here to help!
Contact us!
The Azure API Review Board is dedicated to helping you create amazing APIs. You can read about our mission and learn more about our process on our wiki.
Click here for links to tools, specs, guidelines & other good stuff
Tooling
Guidelines & Specifications
Helpful Links
Getting help
write accessper aka.ms/azsdk/access#request-access-to-rest-api-or-sdk-repositoriesNext Steps to Mergecomment. It will appear within few minutes of submitting this PR and will continue to be up-to-date with current PR state.and https://aka.ms/ci-fix.
queuedstate, please add a comment with contents/azp run.This should result in a new comment denoting a
PR validation pipelinehas started and the checks should be updated after few minutes.