Supporting huggingface-tei version 1.8.1 #5279
Description
The current version of sagemaker (2.251.1) does not support huggingface-tei version 1.8.1 but only the version 1.8.0.
Hugging face TEI version 1.8.1 (https://github.com/huggingface/text-embeddings-inference/releases/tag/v1.8.1) was released 2 weeks ago and provides support for Google's new embedding model, Gemma 300m.
Currently the sagemaker deployment of this model (https://huggingface.co/google/embeddinggemma-300m?sagemaker_deploy=true) is failing because hugging-face tei 1.8.0 does not take into account this model architecture:
"Could not start Candle backend: Could not start backend: Model is not supported\n\nCaused by:\n unknown variant `gemma3_text`, expected one of `bert`, `xlm-roberta`, `camembert`, `roberta`, `distilbert`, `nomic_bert`, `mistral`, `gte`, `new`, `qwen2`, `qwen3`, `mpnet`, `modernbert` at line 45 column 29"