[OpenVINO][Draft]Add Qwen3VL and Qwen3VL-MOE #1452
Conversation
This reverts commit 8e7cdd2.
openvino-dev-samples
force-pushed
the
qwen3vl
branch
from
055dd56 to
3cb4e20
Compare
| return super().generate(input_name, framework, int_dtype, float_dtype) | ||
|
|
||
| @register_in_tasks_manager( | ||
| "qwen3_vl_text", |
There was a problem hiding this comment.
Thanks for the addition @openvino-dev-samples, qwen3_vl will only be available in the next release of transformers (not available in the latest release), and we will need to support this version in optimum-intel before we can merge this PR
There was a problem hiding this comment.
@echarlaix, what is the timeline we are looking at to upgrade to transformers 4.57 and merge this PR?
| "optimum==1.27.*", | ||
| "transformers>=4.45,<4.56", | ||
| "optimum", | ||
| "transformers>=4.36", |
There was a problem hiding this comment.
Since optimum and optimum-intel do not support transformers 4.57 currently, this PR is a Draft, and it only works with my personal repo of Transformers and Optimum.
You can reproduce the model export and inference with following steps:
pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
pip install git+https://github.com/openvino-dev-samples/optimum.git@qwen3vl
pip install git+https://github.com/openvino-dev-samples/transformers.git@qwen3vl
pip install git+https://github.com/openvino-dev-samples/optimum-intel.git@qwen3vl
There was a problem hiding this comment.
@openvino-dev-samples, model export fails with the following error:
ImportError: cannot import name '_attention_scale' from 'torch.onnx.symbolic_opset14' (/home/optimum-intel/.env/lib/python3.12/site-packages/torch/onnx/symbolic_opset14.py)
Steps to reproduce:
python3 -m venv .env
source .env/bin/activate
pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
pip install git+https://github.com/openvino-dev-samples/optimum.git@qwen3vl
pip install git+https://github.com/openvino-dev-samples/transformers.git@qwen3vl
pip install git+https://github.com/openvino-dev-samples/optimum-intel.git@qwen3vl
optimum-cli export openvino --model Qwen/Qwen3-VL-4B-Instruct ov_qwen3vl_instruct/
There was a problem hiding this comment.
Please do not using pytorch2.9x
There was a problem hiding this comment.
Worked with the following packages in virtual env:
python3 -m venv .env
source .env/bin/activate
pip install torch==2.8.0 torchvision --index-url https://download.pytorch.org/whl/cpu
pip install git+https://github.com/openvino-dev-samples/optimum.git@qwen3vl
pip install git+https://github.com/openvino-dev-samples/transformers.git@qwen3vl
pip install git+https://github.com/openvino-dev-samples/optimum-intel.git@qwen3vl
pip install openvino nncf openvino-tokenizers --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/
optimum-cli export openvino --model Qwen/Qwen3-VL-4B-Instruct ov_qwen3vl_instruct
4 participants
No description provided.