[WIP] Update google genai instrumentation to work with latest semantic convention, allow for uploading content. #3772
Conversation
Co-authored-by: Aaron Abbott <[email protected]>
|
| ]: | ||
| attributes.update(completion_details_attributes) | ||
| event = Event(name="gen_ai.completion.details", attributes=attributes) | ||
| hook = load_upload_hook() |
There was a problem hiding this comment.
I think the instrumentor class should do a DI pattern for this. It probably should accept an optional upload hook as a parameter to instrument(), otherwise use load_upload_hook().
| ContentCapturingMode.SPAN_ONLY, | ||
| ContentCapturingMode.SPAN_AND_EVENT, | ||
| ]: | ||
| span = trace.get_current_span() | ||
| span.set_attributes(completion_details_attributes) | ||
| if self._content_recording_enabled in [ | ||
| ContentCapturingMode.EVENT_ONLY, | ||
| ContentCapturingMode.SPAN_AND_EVENT, | ||
| ]: |
There was a problem hiding this comment.
@DylanRussell should we still do this if there is an upload hook? One practical concern for GCP logging exporter is that if you include the JSON in the log directly and it exceeds the size limit (256kB) the whole log gets dropped. Since the upload hook works on the same log, that would mean everything gets dropped.
Alternatively, maybe we could emit two separate logs to avoid this problem
There was a problem hiding this comment.
The user can both configure whether they want the upload and whether they want the content in the log right ? So maybe lets just leave it to the user and recommend one way or the other ? Probably should discuss it more internally..
| from .dict_util import flatten_dict | ||
| from .flags import is_content_recording_enabled | ||
| from .message import ( | ||
| InputMessage, |
There was a problem hiding this comment.
I would import these from the gen ai utils package where they are defined (from opentelemetry.util.genai.types import ...)
| system_instruction: Union[InputMessage, None], | ||
| ): | ||
| attributes = { | ||
| "gen_ai.input.messages": json.dumps( |
There was a problem hiding this comment.
these strings can be imported as constants (from opentelemetry.semconv._incubating.attributes import (
gen_ai_attributes as GenAIAttributes,
))
| system_instruction: Union[InputMessage, None], | ||
| ): | ||
| attributes = { | ||
| "gen_ai.input.messages": json.dumps( |
There was a problem hiding this comment.
we only want to json.dumps stuff if it's going into a span. if it goes into an event (log) we shouldn't
| ContentCapturingMode.SPAN_ONLY, | ||
| ContentCapturingMode.SPAN_AND_EVENT, | ||
| ]: | ||
| span = trace.get_current_span() | ||
| span.set_attributes(completion_details_attributes) | ||
| if self._content_recording_enabled in [ | ||
| ContentCapturingMode.EVENT_ONLY, | ||
| ContentCapturingMode.SPAN_AND_EVENT, | ||
| ]: |
There was a problem hiding this comment.
The user can both configure whether they want the upload and whether they want the content in the log right ? So maybe lets just leave it to the user and recommend one way or the other ? Probably should discuss it more internally..
Description
Accomodation of semantic conventions changes made in open-telemetry/semantic-conventions#2179.
Also, when logging the completion details, the upload hook is called, so if user has configured
OTEL_INSTRUMENTATION_GENAI_UPLOAD_BASE_PATHandOTEL_INSTRUMENTATION_GENAI_UPLOAD_HOOKenv vars, completion details will also be logged as refs (if appropriate capture content env var is set).Type of change
How Has This Been Tested?
WIP, only some manual tests for now.
Does This PR Require a Core Repo Change?
No?
Checklist:
See contributing.md for styleguide, changelog guidelines, and more.