Skip to content

Conversation

ManilShrestha
Copy link
Contributor

Issue
Currently, if an exception occurs for a request, the server does not respond back to the client, causing the client to wait for the response back without ever receiving anything.

Fix
I have created a new function, handle_exception, which can be invoked in the catch part of a try-catch block. This function does the following:

  1. Creates a packet response that is added to the responses and partial_responses queues.
  2. Removes the prompt from the prompt_ids2jobs and prompt_length dictionaries, if they exist.

Result
This ensures that the client receives the exception message in the form of a response.

Note: Since this PR is created on top of PR #21, it also addresses Issue #18. Therefore, I will close PR #21 as this PR covers the code changes included there as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant