Skip to content

Conversation

ManilShrestha
Copy link
Contributor

Issue
The current codebase has a critical issue where the server hangs whenever an exception is caught in the try block of the process_prompts() function. This causes any subsequent requests to be hung until the server is restarted.

Solution
The root cause was identified as the while True logic being inside the try block. Since the thread needs to be running regardless of exceptions, the continuous run logic was moved outside of the try block. This ensures that the server continues to handle new requests even if an exception occurs within the try block.

Test Performed

  1. Normal Generation
  2. Outlines Generation: Tested Regex, JSON, choice, and stop_at
  3. Exception Handling
    • Recreated an error scenario where the regex generation with 'R' (recursion) causes an exception in the outlines generation.
    • Verified that the server can process new requests after the exception, without hanging.
    • Cancelled the error-prone code cell (which continues to run indefinitely). After cancellation, made another request to confirm that the server can process the new request from the same instance without needing a restart.

@ManilShrestha
Copy link
Contributor Author

PS. The additional lines in the PR is due to the removal of extra white spaces. The only change is swapping while True and try:

@ManilShrestha
Copy link
Contributor Author

Since PR #22 is created on top of this PR, I am closing it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant