Skip to content

Fix exception when estimating token count of tool execution request #85

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 29, 2025

Conversation

jpenilla
Copy link
Contributor

@jpenilla jpenilla commented Jun 2, 2025

Issue

Closes langchain4j/langchain4j#3409

The HuggingFaceTokenCountEstimator does not properly check for null text in AiMessages which may be tool execution requests. This results in a stacktrace similar to the following:

java.util.concurrent.CompletionException: java.lang.NullPointerException: text cannot be null
        at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)
        at java.base/java.util.concurrent.CompletableFuture.uniApplyNow(CompletableFuture.java:687)
        at java.base/java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:662)
        at java.base/java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:2200)
        ...
Caused by: java.lang.NullPointerException: text cannot be null
        at ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.encode(HuggingFaceTokenizer.java:237)
        at dev.langchain4j.model.embedding.onnx.HuggingFaceTokenCountEstimator.estimateTokenCountInText(HuggingFaceTokenCountEstimator.java:99)
        at dev.langchain4j.model.embedding.onnx.HuggingFaceTokenCountEstimator.estimateTokenCountInMessage(HuggingFaceTokenCountEstimator.java:110)
        at dev.langchain4j.model.embedding.onnx.HuggingFaceTokenCountEstimator.estimateTokenCountInMessages(HuggingFaceTokenCountEstimator.java:122)
        at dev.langchain4j.memory.chat.TokenWindowChatMemory.ensureCapacity(TokenWindowChatMemory.java:95)
        at dev.langchain4j.memory.chat.TokenWindowChatMemory.add(TokenWindowChatMemory.java:71)
        at dev.langchain4j.service.tool.ToolService.executeInferenceAndToolsLoop(ToolService.java:128)
        ...

Change

We check for null text and return 0 for the estimated token count.

General checklist

  • There are no breaking changes
  • I have added unit and/or integration tests for my change
  • The tests cover both positive and negative cases
  • I have manually run all the unit and integration tests in the module I have added/changed, and they are all green
  • I have added/updated the documentation
  • I have added an example in the examples repo (only for "big" features)
  • I have added/updated Spring Boot starter(s) (if applicable)

@dliubarskyi dliubarskyi marked this pull request as ready for review July 29, 2025 09:50
Copy link
Member

@dliubarskyi dliubarskyi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpenilla thank you!

@dliubarskyi dliubarskyi merged commit 53ebad7 into langchain4j:main Jul 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] TokenWindowChatMemory fails with NPE when a tools is requested
2 participants