-
Notifications
You must be signed in to change notification settings - Fork 1.2k
fix: Avoid BadRequestError due to invalid max_tokens #3667
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hold on this pending discord discussion
43fb189
to
ea42cf6
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good find!
@luis5tb see CI failure:
Thanks! |
The error in the unittest is weird
I trigger the recording workflow and now is green |
@luis5tb the amount of changes files seems off? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please use one of -
- record-if-missing, or
- record + git restore + git add
the restore will remove all the minor changes like call id and token counts
I used the remote recording option (scripts/github/schedule-record-workflow.sh), there is no option for record-if-missing there. From what I see, using that option the "record" is hardcoded: https://github.com/llamastack/llama-stack/blob/main/.github/workflows/record-integration-tests.yml#L70 |
edf6e86
to
4c68892
Compare
This patch ensures if max tokens is not defined it is set to None. This avoid some providers to fail, as they don't have protection for it being set to 0 Issue: llamastack#3666
Removed the modification, just adding the new content
To try to workaround this I created it with remote, fetch the commit, reset it and only add the new files (similar to the record + git restore + git add). It is still adding a lot of files though |
This patch ensures if max tokens is not defined, then is set to None instead of 0 when calling openai_chat_completion. This way some providers (like gemini) that cannot handle the
max_tokens = 0
will not failIssue: #3666