Skip to content

[OpenVINO backend] supporting inference for Gemma, Mistral and GPT2 with ov backend #2310

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 18 commits into
base: master
Choose a base branch
from

Conversation

Mohamed-Ashraf273
Copy link

@Mohamed-Ashraf273 Mohamed-Ashraf273 commented Jun 22, 2025

Description of the change

As a part of my GSoC25 project to support inference with the openvino backend for Gemma , Mistral and GPT-2,
This is my PR for supporting Gemma , Mistral and GPT-2 pipelines.

import os
os.environ["KERAS_BACKEND"] = "openvino"
import keras_hub

model = keras_hub.models.GPT2CausalLM.from_preset(
    "gpt2_large_en", dtype="float16"
)
model.summary()
output = model.generate("Keras is ", max_length=20)
print("Generated text:", output)

Reference

https://docs.openvino.ai/2025/index.html
https://keras.io/api/
https://keras.io/keras_hub/

Colab Notebook

Checklist

  • I have added all the necessary unit tests for my change.
  • I have verified that my change does not break existing code and works with all backends (TensorFlow, JAX, and PyTorch).
  • My PR is based on the latest changes of the main branch (if unsure, rebase the code).
  • I have followed the Keras Hub Model contribution guidelines in making these changes.
  • I have followed the Keras Hub API design guidelines in making these changes.
  • I have signed the Contributor License Agreement.

@github-actions github-actions bot added the Gemma Gemma model specific issues label Jun 22, 2025
@Mohamed-Ashraf273 Mohamed-Ashraf273 force-pushed the supporting_gemma_inference_with_ov_backend branch 26 times, most recently from 6576b03 to 074f0c2 Compare June 23, 2025 17:26
@Mohamed-Ashraf273 Mohamed-Ashraf273 force-pushed the supporting_gemma_inference_with_ov_backend branch 2 times, most recently from d748dd5 to f5470cd Compare June 24, 2025 13:36
@Mohamed-Ashraf273
Copy link
Author

@Mohamed-Ashraf273 Mohamed-Ashraf273 force-pushed the supporting_gemma_inference_with_ov_backend branch from c36b8e5 to 3d6a09b Compare July 15, 2025 20:44
@Mohamed-Ashraf273 Mohamed-Ashraf273 force-pushed the supporting_gemma_inference_with_ov_backend branch from cbd322e to e59d313 Compare July 16, 2025 11:58
@Mohamed-Ashraf273 Mohamed-Ashraf273 force-pushed the supporting_gemma_inference_with_ov_backend branch from e59d313 to 41f9a0f Compare July 16, 2025 12:01
@Mohamed-Ashraf273 Mohamed-Ashraf273 force-pushed the supporting_gemma_inference_with_ov_backend branch from 8a51525 to cb6f1e9 Compare July 18, 2025 11:27
@Mohamed-Ashraf273 Mohamed-Ashraf273 force-pushed the supporting_gemma_inference_with_ov_backend branch from 6390270 to b798a4f Compare July 20, 2025 11:53
@Mohamed-Ashraf273 Mohamed-Ashraf273 marked this pull request as draft July 20, 2025 12:45
@Mohamed-Ashraf273 Mohamed-Ashraf273 force-pushed the supporting_gemma_inference_with_ov_backend branch 4 times, most recently from 9f91fdb to 5003c9e Compare July 21, 2025 11:26
@Mohamed-Ashraf273 Mohamed-Ashraf273 force-pushed the supporting_gemma_inference_with_ov_backend branch from 5003c9e to e6ef629 Compare July 21, 2025 12:05
@Mohamed-Ashraf273 Mohamed-Ashraf273 marked this pull request as ready for review July 21, 2025 14:00
@Mohamed-Ashraf273
Copy link
Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Gemma Gemma model specific issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants