feat: add a recipe on regional aot. #3543
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/tutorials/3543
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New FailureAs of commit e6b1474 with merge base 4696f66 ( NEW FAILURE - The following job has failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
recipes_source/regional_aot.py
Outdated
|
|
||
| **Author:** `Sayak Paul <https://github.com/sayakpaul>`, `Charles Bensimon <https://github.com/cbensimon>`, `Angela Yi <https://github.com/angelay>` | ||
|
|
||
| In our [regional compilation recipe](https://docs.pytorch.org/tutorials/recipes/regional_compilation.html), we showed |
There was a problem hiding this comment.
Maybe we can add to this tutorial saying something like, "if you want to learn how to do it using the AOT technologies, check out this page"
Co-authored-by: Angela Yi <[email protected]>
| _ = model(input) | ||
| return end - start | ||
|
|
||
| def aot_compile_load_model(regional=False) -> torch.nn.Module: |
There was a problem hiding this comment.
I think it's better to benchmark the compilation time in this case as there's some form of CUDA warming up already present from AoT.
|
@angelayi I have fixed a couple bugs and things are looking as expected on my end. Could you check? |
Co-authored-by: Svetlana Karslioglu <[email protected]>
|
@svekars your comments have been addressed. LMK if there's anything else. |
|
Can you fix lint? You can run locally |
Co-authored-by: Svetlana Karslioglu <[email protected]>
Co-authored-by: Svetlana Karslioglu <[email protected]>
|
@svekars anything else pending from a merge? |
|
tutorial build error seems to be unrelated |
3 participants
I don't see an issue for this tutorial. I discussed this with @angelayi and we agreed that having a recipe like is beneficial to the AoT workflow.
I ran the Python script introduced in this PR on a single RTX 4090, and I got:
cc @avikchaudhuri @gmagogsfm @zhxchen17 @tugsbayasgalan @angelayi @suo @ydwu4