Model: Seed OSS thinking + tool call support

[pwilkin]

This one has been an absolute nightmare to implement (Seed OSS tool calling is basically Qwen Coder all over again), so I hope this actually works (testing this on my Q2_K_S which fails to call the tools properly every second time is a nightmare as well).

[bfroemel]

fyi, small jinja template update (mostly comment translation, but one functional difference at the end): https://huggingface.co/ByteDance-Seed/Seed-OSS-36B-Instruct/commit/497f1dca95ebdec98e41d517b9f060ee753c902f#d2h-526183

[pwilkin]

fyi, small jinja template update (mostly comment translation, but one functional difference at the end): https://huggingface.co/ByteDance-Seed/Seed-OSS-36B-Instruct/commit/497f1dca95ebdec98e41d517b9f060ee753c902f#d2h-526183

Thanks, I'll retest with new template.

[pwilkin]

Okay, verified, both the tests and the real life tool calling test run on my trusty Q2_K_S quant pass fine.

@CISC any chance you could take a quick look at this? Apparently the model has gotten popular and some people want to get it merged :>

[CISC]

LGTM

[pwilkin]

Aight, should be good to go.

[ExtReMLapin]

I could be missing something but your builder.add_rule("root", string_join(tool_rules, " | ")); seems to lack the reasoning grammar.

Which means using tool_choice = "required" disables thinking.

[ExtReMLapin]

./build/bin/llama-server -ngl 999 -fa -m Seed-OSS-36B-Instruct-Q5_K_M.gguf -c 131071 --jinja --reasoning-format deepseek --slots --n-predict 131071 --no-context-shift -ctk q8_0 -ctv q8_0 --parallel 1 --port 2483 --host 192.168.25.25 --chat-template-file models/templates/ByteDance-Seed-OSS.jinja

---

Gave a quick try, tool calling results are terrible with very simple example, but it's probably the model's fault.

---

Gave another try with a tool we use , long context and it's really doing something off :

1. I feel like it's not streaming correctly, it's sending the whole tool call at once
2. there is something weird going on with reasoning (after tool call ???)

With tool_choice at required we should have a reasoning_content, no content, and BEFORE the tool call

to run the same query you can use node and this file, please note you might need multiple tries :
oss_js_node_debug.js

[pwilkin]

Why do the meaningful testing comments always start just as the PR gets merged? 😆

@ExtReMLapin I noticed this type of behavior too, but I'm not sure if this is a mistake or whether this is an intended feature of tool calls happening within the reasoning, which would mean we'd have to rework the grammar.

I've only tried tool calling on my Q2_K_S, but it works without any problems, at least on the simple tool calls I've tried (web search / execute shell command).

[ExtReMLapin]

Why do the meaningful testing comments always start just as the PR gets merged? 😆

Because i'm a lazy man and pulling it on my own branch is already too much effort !

As for the reasoning, I'm already surprised it happens, here with qwen, right now on master it doesn't happen because it's not allowed by grammar (in required tool calling mode).
If you want an example I opened a PR to support thinking AND tool calling @ required

#15248

I've only tried tool calling on my Q2_K_S, but it works without any problems,

I don't see why everyone seems so happy with this model, from my tests it's VERY VERY meh, nothing close to Qwen3

Maybe i'm just doing something wrong, but either way we should NOT see the thinking tags, it should either not think or parse the thinking tags and shove the text into reasoning_content

[pwilkin]

Maybe i'm just doing something wrong, but either way we should NOT see the thinking tags, it should either not think or parse the thinking tags and shove the text into reasoning_content

True - are you using the official updated template? (from https://huggingface.co/ByteDance-Seed/Seed-OSS-36B-Instruct/blob/main/chat_template.jinja)?

[ExtReMLapin]

As you can see in the command I posted I'm using the one you pushed in your PR --chat-template-file models/templates/ByteDance-Seed-OSS.jinja