ChatModel streaming chat response data is too large.

[192902649]

When using Flux<ChatClientResponse>, the context is returned with every response, which results in large data volume. This can be optimized — for example, only include the context in the response when chatResponse.result.metadata.finishReason == "stop".

[192902649]

public Flux<ChatClientResponse> chatV3(@PathVariable("chatId") String chatId,@RequestParam("message") String message) {
    Flux<ChatClientResponse> flux = ollamaChatClient.prompt()
            .user(message)
            .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, chatId))
            .stream().chatClientResponse();

    return flux;
}

[192902649]

[vinupreethi]

@ilayaperumalg i would like to work on this.

[ilayaperumalg]

@vinupreethi sure, please. thank you for your interest!