⚠️ 🔴 Add ministral model

[manueldeprada]

Coming from #39799, adding Ministral model to support its interleaved attention.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[manueldeprada]

still have to check the slow tests, but rest is ready @ArthurZucker

[ArthurZucker]

Nice!

[manueldeprada]

run-slow: ministral

[github-actions]

This comment contains run-slow, running the specified jobs:

models: ['models/ministral']
quantizations: [] ...

[manueldeprada]

run-slow: ministral

[manueldeprada]

@ArthurZucker tests are ready. One note: after this merges, if i open a pr in mistralai/Ministral-8B-Instruct-2410 to change the architecture to MinistralForCausalLM, older transformers versions won’t load the model. Are we ok with this? Is there a preferred way of handling these situations? maybe a warning in the readme?

[ArthurZucker]

Okay!
Regarding BC, we are kinda obliged to break with a proper ⚠️ 🔴 on PR title should be alirght

[manueldeprada]

run-slow: auto, ministral

[github-actions]

This comment contains run-slow, running the specified jobs:

models: ['models/auto', 'models/ministral']
quantizations: [] ...

[ArthurZucker]

maybe a warning in the readme?

Yep we can have that!
Older version will load Mistral instead but should be fine it's not breaking

[manueldeprada]

Yep we can have that! Older version will load Mistral instead but should be fine it's not breaking

It is breaking: once we change mistralai/Ministral-8B-Instruct-2410/config.json to be

{
"architectures": [
"MiNIstralForCausalLM"
], ...

old versions will not recognize that architecture. Thats why I was worried 😅

[ArthurZucker]

No but you don't have to change them online IMO

[ArthurZucker]

What I meant by not breaking is that the old mistral does not support sliding non sliding so we are fixing something, the best way IMO is not to update the old model but to have some kind of filter for the models on the hub if possible!

[ArthurZucker]

Adding a layer type attribute will not break older version, and we can check if model type is mistral but it has layer_type -> we map to ministral

[ArthurZucker]

instead of chaging the model type

[manueldeprada]

great, I get it now! 🙏 will ping you back

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: auto, ministral, mistral

[manueldeprada]

@ArthurZucker should be ready now! Let me know if it is the correct place to map mistral -> ministral

Also, if the user explicitly instantiates a Mistral class, I added a warning telling them that they might want to change