-
Notifications
You must be signed in to change notification settings - Fork 1.7k
GH-3657: Fix DeepSeek tool call content null issue #3817
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
GH-3657: Fix DeepSeek tool call content null issue #3817
Conversation
Closes spring-projects#3657 * Add content fallback for DeepSeek when only tool calls are present * Ensures AssistantMessage has proper output for downstream processing * Add test case for AssistantMessage with tool calls only Signed-off-by: Dongha Koo <[email protected]>
...ai-client-chat/src/test/java/org/springframework/ai/chat/client/ChatClientResponseTests.java
Outdated
Show resolved
Hide resolved
Please update the copyright end year from 2024 to 2025. |
…t/client/ChatClientResponseTests.java Co-authored-by: jonghoonpark <[email protected]> Signed-off-by: 구동하 <[email protected]>
Closes spring-projects#3657 * Set content = "__tool_call__" when content is null and toolCalls are present * Prevents schema validation errors in downstream clients * Add unit test: ChatClientResponseTests.whenAssistantMessageHasOnlyToolCalls_thenContentIsToolCallMarker * Apply code style: import formatting, copyright year Signed-off-by: Dongha Koo <[email protected]>
55067cc
to
4b054b0
Compare
@dev-jonghoonpark |
I don’t think this is an issue with |
I'm not very familiar with But maybe changing this line in |
@mxsl-gr You're right to consider that — but in fact, the field is already defined as However, this doesn’t resolve the issue, because the validation error doesn’t originate from the type annotation itself — |
Just to clarify why this is still relevant for Spring AI: Even if the root cause originates in sglang, the validation failure (e.g., content: null causing OpenAPI schema errors or FastAPI crashes) is surfaced through Spring AI's API layer — such as .getOutput().getText() or generated client interfaces. But more importantly, Spring AI is not just a passive proxy of the upstream response — it actively maps, serializes, and exposes model output to downstream clients. This includes constructing OpenAPI schemas, wrapping tool call messages, and integrating with frameworks like Spring Web or WebFlux. Therefore, Spring AI shares responsibility for ensuring that the responses it produces are schema-compliant and consumer-friendly — regardless of upstream behavior. Even if upstream systems are improved later, Spring AI should guard against breakage in the current ecosystem. This PR addresses the issue at the precise point of failure and control, and does so in a backward-compatible, non-invasive way — stabilizing behavior for all downstream consumers that rely on consistent content output. |
✨ Summary
This PR resolves the issue where
DeepSeek
chat completions with tool calls return an assistant message withcontent = null
, leading to schema validation errors or downstream failures (e.g., insglang
or FastAPI with pydantic).To address this, we explicitly set the assistant message content to
"__tool_call__"
when tool calls are present but content isnull
.🛠 Changes
DeepSeekChatModel
, updatedbuildGeneration(...)
logic:content == null && toolCalls != empty
, setcontent = "__tool_call__"
to maintain compatibility.content
to be a non-null string.AssistantMessage#getText()
logic, keeping it backward compatible for other models.💡 Context
DeepSeek sometimes returns:
This results in downstream consumers (e.g.,
sglang
, FastAPI with pydantic) throwing errors:By ensuring a placeholder content (
"__tool_call__"
), we:🧪 Tests & Verification
ChatClientResponseTests.whenAssistantMessageHasOnlyToolCalls_thenContentIsToolCallMarker
getText()
returns"__tool_call__"
when expectedGeneration.getOutput().getText()
./mvnw clean verify
)-Ddisable.checks=false
)git log --oneline --graph
confirmed)main
📘 Documentation
🔗 Closes #3657
Please review and let me know if you’d like any refinements.
Happy to iterate!