Skip to content

Commit 73233d2

Browse files
lievansabrenner
andauthored
fix(llmobs): fix parsing nontype chunk.delta in openai streamed responses (#14046)
In openai streamed responses chunk.delta can be set to `None`, so if not hasattr(chunk, "delta") evaluates to true but then this line throws since delta is None: if getattr(chunk.delta, "role") and not message.get("role"): This PR changes the logic to use uniformly use our `_get_attr` util function while parsing streamed chunks ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) --------- Co-authored-by: Sam Brenner <[email protected]>
1 parent 8423d8f commit 73233d2

File tree

4 files changed

+302
-8
lines changed

4 files changed

+302
-8
lines changed

ddtrace/llmobs/_integrations/utils.py

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -645,24 +645,24 @@ def openai_construct_message_from_streamed_chunks(streamed_chunks: List[Any]) ->
645645
"""
646646
message: Dict[str, Any] = {"content": "", "tool_calls": []}
647647
for chunk in streamed_chunks:
648-
if getattr(chunk, "usage", None):
648+
if _get_attr(chunk, "usage", None):
649649
message["usage"] = chunk.usage
650-
if not hasattr(chunk, "delta"):
650+
if not _get_attr(chunk, "delta", None):
651651
continue
652-
if getattr(chunk, "index", None) and not message.get("index"):
652+
if _get_attr(chunk, "index", None) and not message.get("index"):
653653
message["index"] = chunk.index
654-
if getattr(chunk.delta, "role") and not message.get("role"):
654+
if _get_attr(chunk.delta, "role", None) and not message.get("role"):
655655
message["role"] = chunk.delta.role
656-
if getattr(chunk, "finish_reason", None) and not message.get("finish_reason"):
656+
if _get_attr(chunk, "finish_reason", None) and not message.get("finish_reason"):
657657
message["finish_reason"] = chunk.finish_reason
658-
chunk_content = getattr(chunk.delta, "content", "")
658+
chunk_content = _get_attr(chunk.delta, "content", "")
659659
if chunk_content:
660660
message["content"] += chunk_content
661661
continue
662-
function_call = getattr(chunk.delta, "function_call", None)
662+
function_call = _get_attr(chunk.delta, "function_call", None)
663663
if function_call:
664664
openai_construct_tool_call_from_streamed_chunk(message["tool_calls"], function_call_chunk=function_call)
665-
tool_calls = getattr(chunk.delta, "tool_calls", None)
665+
tool_calls = _get_attr(chunk.delta, "tool_calls", None)
666666
if not tool_calls:
667667
continue
668668
for tool_call in tool_calls:
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
---
2+
fixes:
3+
- |
4+
LLM Observability, openai: This fix resolves an issue where openai tracing caused an ``AttributeError`` while parsing ``NoneType`` streamed chunk deltas.

0 commit comments

Comments
 (0)