👩‍🦯 Fix usage of VLM using text only

[SamuelBarryCS]

What

• Fixes sft_gemma3 example doesn't work #3957 by checking if there is image and adapting the call to the processor accordingly
• Fixes VLM config tests that were failing (mismatch in between config.text_config.num_hidden_layers and layer_types was causing the tests to fail)

How to review

• Read diff

Test performed

• Issue reproduced with a small script (pushed, but will be deleted before merging). Before the fix, the script yielded [rank0]: KeyError: 'images' like in sft_gemma3 example doesn't work #3957 but after the fix it is working just fine
• No major logic change so existing tests still passing

[SamuelBarryCS]

Will be deleted before merging

[SamuelBarryCS]

cc @qgallouedec for review when you get the time.
Merci!

[sergiopaniego]

Thanks for the fix!
We could add a modified version of the script as a test in test_sft_trainer.py and remove the script

[SamuelBarryCS]

Thanks for the fix! We could add a modified version of the script as a test in test_sft_trainer.py and remove the script

Thanks for your quick reply @sergiopaniego !
Done in 6465594 :), and the test is passing fine:

Good to merge on my side (but it looks like we still need approval for the workflows to run)

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[SamuelBarryCS]

@sergiopaniego do you understand why the tests are failing ? It doesn't look related to my changes ... :x
Otherwise ready to merge on my side (once this test issue is solved).

[qgallouedec]

Hi, thanks for your contribution. I opted for a different approach: when the dataset does not contain images, it can be pre-processed, and there is no point in doing it on the fly.

[SamuelBarryCS]

Hi, thanks for your contribution. I opted for a different approach: when the dataset does not contain images, it can be pre-processed, and there is no point in doing it on the fly.

This works as well! Thanks for the edit @qgallouedec.