Possibilities to Enhance Vector Store Retrieval to Minimize Token Usage

[OscarAgreda]

Pull Request Title:

Enhance Vector Store Retrieval to Minimize Token Usage

Pull Request Description:

Summary

This pull request enhances the current vector store retrieval mechanism by introducing an additional method to minimize token usage when querying the language model. The new approach focuses on pre-processing and filtering relevant data locally, ensuring efficient query processing and reducing the overall token count sent to the LLM.

Key Changes

1. Added a New Function get_results_minimized_tokens:

   • This function utilizes the Neo4jVector's similarity_search method to filter and retrieve only the most relevant data based on the user's query.
   • Constructs the context locally, reducing the size of the context passed to the language model.

2. Updated Retrieval Mechanism:

   • Ensures that the language model receives a concise and precise context, minimizing the token count while maintaining response quality.
   • Adds citations for sources in the response to provide clarity and references.

3. Retry Mechanism:

   • Included a retry mechanism to handle transient errors and ensure robustness in the retrieval process.

Benefits

• Reduced Token Usage: By filtering and summarizing relevant data locally, this approach significantly reduces the number of tokens sent to the LLM, making the process more efficient.
• Improved Performance: Enhances the speed and efficiency of generating responses by minimizing unnecessary context.
• Robust and Reliable: The retry mechanism ensures the process is robust against transient errors.

Example Usage

# Using the vector store directly with minimized token usage
response = get_results_minimized_tokens("What are the key points in the recent SEC filing?")
print(response)

This enhancement will help in managing token limits effectively while providing accurate and concise responses based on the vector store's context.