-
Notifications
You must be signed in to change notification settings - Fork 290
[OpenVINO backend] supporting inference for Gemma, Mistral and GPT2 with ov backend #2310
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
[OpenVINO backend] supporting inference for Gemma, Mistral and GPT2 with ov backend #2310
Conversation
6576b03
to
074f0c2
Compare
d748dd5
to
f5470cd
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request introduces support for the OpenVINO backend for several models (Gemma, Mistral, GPT-2). The changes are extensive and well-structured, including adding OpenVINO as a dependency, implementing a robust test-skipping mechanism, and adding OpenVINO-specific code paths to handle backend differences. The new openvino_utils.py
module is a great way to encapsulate backend-specific logic.
My review focuses on improving robustness and configurability. I've identified a few issues:
- A hardcoded "CPU" device for OpenVINO compilation, which limits hardware acceleration options.
- A minor inefficiency in the test configuration file.
- Potential data issues in the test exclusion lists (trailing whitespace, missing newline).
Overall, this is a great contribution. The changes are thoughtful and address the complexities of adding a new backend.
openvino_excluded_tests.txt
Outdated
keras_hub/src/samplers/greedy_sampler_test.py | ||
keras_hub/src/samplers/top_k_sampler_test.py | ||
keras_hub/src/samplers/top_p_sampler_test.py | ||
keras_hub/src/utils/pipeline_model_test.py |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
c36b8e5
to
3d6a09b
Compare
cbd322e
to
e59d313
Compare
e59d313
to
41f9a0f
Compare
8a51525
to
cb6f1e9
Compare
381ac68
to
6390270
Compare
6390270
to
b798a4f
Compare
9f91fdb
to
5003c9e
Compare
5003c9e
to
e6ef629
Compare
Description of the change
As a part of my GSoC25 project to support inference with the openvino backend for
Gemma
,Mistral
andGPT-2
,This is my PR for supporting
Gemma
,Mistral
andGPT-2
pipelines.Reference
https://docs.openvino.ai/2025/index.html
https://keras.io/api/
https://keras.io/keras_hub/
Colab Notebook
Checklist