if video_len longer than num_frames random select frames from video

[ruyijidan]

we train video'lens may longer than 81 frames. this can random use sample 81 frames from video

[gemini-code-assist]

Summary of Changes

Hello @ruyijidan, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request modifies the video loading utility to robustly handle input videos that are longer than the required number of frames for training. Previously, longer videos might have caused issues or been truncated. With this change, the system now intelligently samples a contiguous segment of the required frame length by randomly selecting a starting point within the video, ensuring consistent data input for training while accommodating varied video lengths.

Highlights

• Video Frame Sampling: The load_video function in diffsynth/trainers/utils.py has been updated to support random sampling of frames from videos.
• Dynamic Start Index: If a video's total frame count exceeds the target num_frames, a random starting index is calculated to ensure a contiguous block of num_frames can be extracted.
• Improved Data Handling: This change allows the training process to utilize longer videos more effectively by sampling relevant segments, preventing data loss or errors due to length mismatches.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request adds functionality to randomly sample a segment of frames from a video if it's longer than the required number of frames. The overall logic is correct, but I've identified a few areas for improvement related to Python coding standards and style. Specifically, I've suggested moving the import statement to the top of the file, translating a comment to English for consistency, and addressing some minor style issues to improve code readability. Please see the detailed comments.

[gemini-code-assist]

Imports should be at the top of the file, not inside a function. This follows PEP 8 style guidelines and makes dependencies clear. Please move import random to the top of diffsynth/trainers/utils.py.

[gemini-code-assist]

There are a few style improvements that can be made here to align with PEP 8:

• The comment should be in English for consistency.
• The blank line after the if block is unnecessary.
• Comparison operators should have a single space on either side (e.g., total_frames > num_frames).

Suggested change

	if total_frames>num_frames:
	# 计算随机起始位置（确保能截取到81帧）
	max_start_idx = total_frames - num_frames
	start_idx = random.randint(0, max_start_idx)
	if total_frames > num_frames:
	# Calculate a random start index to ensure `num_frames` can be sampled.
	max_start_idx = total_frames - num_frames
	start_idx = random.randint(0, max_start_idx)

[Artiprocher]

@ruyijidan Thank you very much for your suggestion, but we do not plan to add this feature.

We prefer to guide users to handle video data slicing externally, rather than performing slicing during training, as this makes the training process more controllable.

Additionally, the slicing functionality during training is incompatible with our ongoing development of staged training (which processes computations such as the text encoder and VAE offline, then directly reads these precomputed results).

[ruyijidan]

@ruyijidan Thank you very much for your suggestion, but we do not plan to add this feature.

We prefer to guide users to handle video data slicing externally, rather than performing slicing during training, as this makes the training process more controllable.

Additionally, the slicing functionality during training is incompatible with our ongoing development of staged training (which processes computations such as the text encoder and VAE offline, then directly reads these precomputed results).

Thank you for relpy!That' alright.Because the Wanvideo trainning's dataset changed TensorDataset to VideoDataset I can genarate many tensors.pth and use TensorDataset to read.VideoDataset's version make all pipline in model forward.So I do this. Looking forward your new development so that training will needs lower vram and more faster.Thank you again