Really struggling, latest version of llama-server, Mistral Nemo Instruct 2407, adding tool capability #16652
Unanswered
linuxmagic-mp
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Okay, after three days.. now probably a .jinja file expert, but still struggling. Start the server with a custom .jinja file, modeled on the on mistralai-Mistral-Nemo-Instruct-2407.jinja, tools successfully load, the LLM can run the tool, but it won't do it in inline, instead I keep getting the one turn problem, it outputs the TOOL_CALL JSON only, instead of represssing that and responding based on TOOL_RESULTS. Whatever input I add, then it replies with the user friendly response of the previous query. Anyone get this working?
Beta Was this translation helpful? Give feedback.
All reactions