Sage Attention vs Flash Attention Speed Comparison with Wan 2.1 - Sage Attention is 37% Faster - 720p - 14b model - tested on Windows Python VENV - no WSL

[FurkanGozukara]

I am developing a Wan 2.1 Gradio App

Using DiffSynth-Studio as a backend - I am using slightly modified back end though you can see my pull request here : #463

I just implemented Sage Attention to my app since Windows pre-compiled wheels published by a hero

Here comparison. I feel like just a little bit quality degrade but huge speed up. Tested on RTX 5090 on Windows with native Python 3.10 VENV - (no WSL)

sage_attention.mp4

Here my app link : https://www.patreon.com/posts/123105403

The app is 1-click to install with all pre-compiled wheels

Here my app interface

Prompt

Close-up shot of a smiling young boy with a joyful expression, sitting comfortably in a cozy room. The boy has tousled brown hair and wears a colorful t-shirt. Bright, soft lighting highlights his happy face. Medium close-up, slightly tilted camera angle.

Negative Prompt

Overexposure, static, blurred details, subtitles, paintings, pictures, still, overall gray, worst quality, low quality, JPEG compression residue, ugly, mutilated, redundant fingers, poorly painted hands, poorly painted faces, deformed, disfigured, deformed limbs, fused fingers, cluttered background, three legs, a lot of people in the background, upside down

[Artiprocher]

@FurkanGozukara Thank you for providing the experimental results. However, we will still not set Sage Attention as the default option because it has compatibility issues on certain GPUs. If users install it themselves, we will automatically invoke it.

Additionally, we will integrate the 1.3B image-to-video model and the ControlNet model within two weeks. You can start thinking about how to design the UI now, haha. Models are here.

[FurkanGozukara]

@FurkanGozukara Thank you for providing the experimental results. However, we will still not set Sage Attention as the default option because it has compatibility issues on certain GPUs. If users install it themselves, we will automatically invoke it.

Additionally, we will integrate the 1.3B image-to-video model and the ControlNet model within two weeks. You can start thinking about how to design the UI now, haha. Models are here.

so awesome

Currently if Flash Attention installed it uses that even though Sage Attention installed as well

So I uninstall flash Attention then it uses Sage. If we could choose between could be good

These new models are amazing. Actually Wan official team also announced soon they will publish official Wan model with image to video with begin and ending

It is also super important that can we get GGUF support? This is asked of me so many times

Also your LoRA integration didnt fix previous slowness issue and you need to implemented cancel feature as i did . I made pull request for them

If you could further improve lora loading like this would be great

as we know LoRA is just addition of weights

so if we subtract back those weights we get original model

so loaded loras can be removed back from loaded pipeline quickly to prevent pipeline reload - because pipeline reload is slow on not very fast computers

[Artiprocher]

@FurkanGozukara We sincerely apologize, as our resources are very limited. There have been many new models related to Wan 2.1 recently, and we need to prioritize integrating these models. As a result, support for GGUF will be delayed.

[FurkanGozukara]

@FurkanGozukara We sincerely apologize, as our resources are very limited. There have been many new models related to Wan 2.1 recently, and we need to prioritize integrating these models. As a result, support for GGUF will be delayed.

True. I hope you can add GGUF asap so many people awaiting it :(

[chappie90]

@FurkanGozukara Out of curiosity, did you try with Sage Attention 2? It's supposed to be much faster?

[FurkanGozukara]

@FurkanGozukara Out of curiosity, did you try with Sage Attention 2? It's supposed to be much faster?

i am already using Sage Attention 2 i compiled myself (for linux users) and using pre compiled for windows

I couldnt compile flash attention 3 though it fails for windows : Dao-AILab/flash-attention#1524