Skip to content

[none][feat] Support nano-v2-vlm with multiple PRs #8395

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 7 commits into from
Closed

[none][feat] Support nano-v2-vlm with multiple PRs #8395

wants to merge 7 commits into from

Conversation

@Wanli-Jiang
Copy link
Collaborator

@Wanli-Jiang Wanli-Jiang commented Oct 15, 2025 •
edited
Loading

PRs

Known issues

  • chunked_prefill for video will raise error.
  • chunked_prefill + EVS for video will raise error.
  • num_frames are fixed as 8 for video input.
  • No metadata for video input, so the nano-v2-vlm video prompt is bit different from model training.

Wanli-Jiang and others added 7 commits October 15, 2025 00:34
@Wanli-Jiang
[TRTLLM-8579][feat] Support quantized model for nano-v2-vlm
afa755b 
* Support for FP8 model.

Signed-off-by: Wanli Jiang <[email protected]>
@Wanli-Jiang
[None][feat] Add fmha_v2 kernel for head_dim=80 and sm=100 to support…
b942377 
… VLM

Signed-off-by: Wanli Jiang <[email protected]>
@Wanli-Jiang
[TRTLLM-8238][feat] Add EVS support for nano-v2-vlm
d5c5211 
Signed-off-by: Wanli Jiang <[email protected]>
@chang-l @Wanli-Jiang
Fix chunked prefill for video and add get_video_token support for EVS
1348172 
Signed-off-by: Chang Liu <[email protected]>
@Wanli-Jiang
[TRTLLM-8238][feat] Add EVS support for nano-v2-vlm
dc19ce1 
* Update EVS codes.

Signed-off-by: Wanli Jiang <[email protected]>
@Wanli-Jiang
[none][fix] Fix get_nunm_tokens_per_image for nano-v2-vlm
cfd8392 
Signed-off-by: Wanli Jiang <[email protected]>
@Wanli-Jiang
[None][feat] Support base64 video input
43b398e 
Signed-off-by: Wanli Jiang <[email protected]>
@Wanli-Jiang
Copy link
Collaborator Author

The latest rebase PR is #8578, please check it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Wanli-Jiang @chang-l