Validate exhaustive autotuning for FP8 Inductor templates #355
+4
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jananisriram
temporarily deployed
to
docker-s3-upload
GitHub Actions
Inactive
jananisriram
temporarily deployed
to
docker-s3-upload
GitHub Actions
Inactive
|
This pull request was exported from Phabricator. Differential Revision: D80958642 |
jananisriram
added a commit
to jananisriram/pytorch
that referenced
this pull request
Aug 25, 2025
…ates (pytorch#161442) Summary: X-link: meta-pytorch/tritonbench#355 Pull Request resolved: pytorch#161442 Validate exhaustive autotuning for FP8 Inductor templates: scaled MM templates require `block_k >= 32`. Before, exhaustive autotuning defaulted to a limited set of autotuning configs, as limitations for exhaustively autotuning on FP8 shapes had not been tested. Test Plan: ``` CUDA_VISIBLE_DEVICES=0 TRITON_PRINT_AUTOTUNING=1 TRITON_ALWAYS_COMPILE=1 TORCH_LOGS=+inductor TORCHINDUCTOR_FORCE_DISABLE_CACHES=1 ENABLE_PERSISTENT_TMA_MATMUL=1 TORCHINDUCTOR_MAX_AUTOTUNE_GEMM=1 TORCHINDUCTOR_MAX_AUTOTUNE_GEMM_SEARCH_SPACE=DEFAULT buck2 run mode/{opt,inplace} pytorch/t ritonbench:run -- --op fp8_gemm --only torch_fp8_gemm,pt2_fp8_gemm --metrics tflops,accuracy --input-loader=/home/jananisriram/personal/exhaustive_autotune_rowwise_persistent_tma/json_fi les/rowwise_ptma_0.json --output="/home/jananisriram/personal/exhaustive_autotune_rowwise_persistent_tma/autotune/gpu0_bench.csv" --atol=1e-2 --rtol=0.5 2>&1 | tee ~/personal/exhaustive_ autotune_rowwise_persistent_tma/autotune/gpu0.log ``` autotunes on the maximum configs available, rather than the defaults, and skips configs not compatible with TMA. Rollback Plan: Differential Revision: D80958642
jananisriram
added a commit
to jananisriram/pytorch
that referenced
this pull request
Aug 26, 2025
…ates (pytorch#161442) Summary: X-link: meta-pytorch/tritonbench#355 Validate exhaustive autotuning for FP8 Inductor templates: scaled MM templates require `block_k >= 32`. Before, exhaustive autotuning defaulted to a limited set of autotuning configs, as limitations for exhaustively autotuning on FP8 shapes had not been tested. Test Plan: ``` CUDA_VISIBLE_DEVICES=0 TRITON_PRINT_AUTOTUNING=1 TRITON_ALWAYS_COMPILE=1 TORCH_LOGS=+inductor TORCHINDUCTOR_FORCE_DISABLE_CACHES=1 ENABLE_PERSISTENT_TMA_MATMUL=1 TORCHINDUCTOR_MAX_AUTOTUNE_GEMM=1 TORCHINDUCTOR_MAX_AUTOTUNE_GEMM_SEARCH_SPACE=DEFAULT buck2 run mode/{opt,inplace} pytorch/t ritonbench:run -- --op fp8_gemm --only torch_fp8_gemm,pt2_fp8_gemm --metrics tflops,accuracy --input-loader=/home/jananisriram/personal/exhaustive_autotune_rowwise_persistent_tma/json_fi les/rowwise_ptma_0.json --output="/home/jananisriram/personal/exhaustive_autotune_rowwise_persistent_tma/autotune/gpu0_bench.csv" --atol=1e-2 --rtol=0.5 2>&1 | tee ~/personal/exhaustive_ autotune_rowwise_persistent_tma/autotune/gpu0.log ``` autotunes on the maximum configs available, rather than the defaults, and skips configs not compatible with TMA. Rollback Plan: Differential Revision: D80958642
|
This pull request was exported from Phabricator. Differential Revision: D80958642 |
jananisriram
added a commit
that referenced
this pull request
Aug 26, 2025
Summary: Pull Request resolved: #355 X-link: pytorch/pytorch#161442 Validate exhaustive autotuning for FP8 Inductor templates: scaled MM templates require `block_k >= 32`. Before, exhaustive autotuning defaulted to a limited set of autotuning configs, as limitations for exhaustively autotuning on FP8 shapes had not been tested. Differential Revision: D80958642
jananisriram
force-pushed
the
export-D80958642
branch
from
2bd053d to
2912432
Compare
jananisriram
temporarily deployed
to
docker-s3-upload
GitHub Actions
Inactive
jananisriram
temporarily deployed
to
docker-s3-upload
GitHub Actions
Inactive
jananisriram
added a commit
to jananisriram/pytorch
that referenced
this pull request
Aug 26, 2025
…ates (pytorch#161442) Summary: X-link: meta-pytorch/tritonbench#355 Pull Request resolved: pytorch#161442 Validate exhaustive autotuning for FP8 Inductor templates: scaled MM templates require `block_k >= 32`. Before, exhaustive autotuning defaulted to a limited set of autotuning configs, as limitations for exhaustively autotuning on FP8 shapes had not been tested. Test Plan: ``` CUDA_VISIBLE_DEVICES=0 TRITON_PRINT_AUTOTUNING=1 TRITON_ALWAYS_COMPILE=1 TORCH_LOGS=+inductor TORCHINDUCTOR_FORCE_DISABLE_CACHES=1 ENABLE_PERSISTENT_TMA_MATMUL=1 TORCHINDUCTOR_MAX_AUTOTUNE_GEMM=1 TORCHINDUCTOR_MAX_AUTOTUNE_GEMM_SEARCH_SPACE=DEFAULT buck2 run mode/{opt,inplace} pytorch/t ritonbench:run -- --op fp8_gemm --only torch_fp8_gemm,pt2_fp8_gemm --metrics tflops,accuracy --input-loader=/home/jananisriram/personal/exhaustive_autotune_rowwise_persistent_tma/json_fi les/rowwise_ptma_0.json --output="/home/jananisriram/personal/exhaustive_autotune_rowwise_persistent_tma/autotune/gpu0_bench.csv" --atol=1e-2 --rtol=0.5 2>&1 | tee ~/personal/exhaustive_ autotune_rowwise_persistent_tma/autotune/gpu0.log ``` autotunes on the maximum configs available, rather than the defaults, and skips configs not compatible with TMA. Rollback Plan: Differential Revision: D80958642
facebook-github-bot
pushed a commit
that referenced
this pull request
Aug 27, 2025
Summary: X-link: pytorch/pytorch#161442 Validate exhaustive autotuning for FP8 Inductor templates: scaled MM templates require `block_k >= 32`. Before, exhaustive autotuning defaulted to a limited set of autotuning configs, as limitations for exhaustively autotuning on FP8 shapes had not been tested. Reviewed By: coconutruben Differential Revision: D80958642
facebook-github-bot
force-pushed
the
export-D80958642
branch
from
2912432 to
2c9a2f4
Compare
facebook-github-bot
had a problem deploying
to
docker-s3-upload
GitHub Actions
Failure
facebook-github-bot
temporarily deployed
to
docker-s3-upload
GitHub Actions
Inactive
|
This pull request was exported from Phabricator. Differential Revision: D80958642 |
jananisriram
added a commit
to jananisriram/pytorch
that referenced
this pull request
Aug 27, 2025
…ates (pytorch#161442) Summary: X-link: meta-pytorch/tritonbench#355 Validate exhaustive autotuning for FP8 Inductor templates: scaled MM templates require `block_k >= 32`. Before, exhaustive autotuning defaulted to a limited set of autotuning configs, as limitations for exhaustively autotuning on FP8 shapes had not been tested. Test Plan: ``` CUDA_VISIBLE_DEVICES=0 TRITON_PRINT_AUTOTUNING=1 TRITON_ALWAYS_COMPILE=1 TORCH_LOGS=+inductor TORCHINDUCTOR_FORCE_DISABLE_CACHES=1 ENABLE_PERSISTENT_TMA_MATMUL=1 TORCHINDUCTOR_MAX_AUTOTUNE_GEMM=1 TORCHINDUCTOR_MAX_AUTOTUNE_GEMM_SEARCH_SPACE=DEFAULT buck2 run mode/{opt,inplace} pytorch/t ritonbench:run -- --op fp8_gemm --only torch_fp8_gemm,pt2_fp8_gemm --metrics tflops,accuracy --input-loader=/home/jananisriram/personal/exhaustive_autotune_rowwise_persistent_tma/json_fi les/rowwise_ptma_0.json --output="/home/jananisriram/personal/exhaustive_autotune_rowwise_persistent_tma/autotune/gpu0_bench.csv" --atol=1e-2 --rtol=0.5 2>&1 | tee ~/personal/exhaustive_ autotune_rowwise_persistent_tma/autotune/gpu0.log ``` autotunes on the maximum configs available, rather than the defaults, and skips configs not compatible with TMA. Rollback Plan: Reviewed By: coconutruben Differential Revision: D80958642
facebook-github-bot
pushed a commit
that referenced
this pull request
Aug 27, 2025
Summary: X-link: pytorch/pytorch#161442 Validate exhaustive autotuning for FP8 Inductor templates: scaled MM templates require `block_k >= 32`. Before, exhaustive autotuning defaulted to a limited set of autotuning configs, as limitations for exhaustively autotuning on FP8 shapes had not been tested. Reviewed By: coconutruben Differential Revision: D80958642
facebook-github-bot
force-pushed
the
export-D80958642
branch
from
2c9a2f4 to
83d742c
Compare
facebook-github-bot
had a problem deploying
to
docker-s3-upload
GitHub Actions
Failure
facebook-github-bot
temporarily deployed
to
docker-s3-upload
GitHub Actions
Inactive
|
This pull request was exported from Phabricator. Differential Revision: D80958642 |
jananisriram
added a commit
to jananisriram/pytorch
that referenced
this pull request
Aug 27, 2025
…ates (pytorch#161442) Summary: X-link: meta-pytorch/tritonbench#355 Validate exhaustive autotuning for FP8 Inductor templates: scaled MM templates require `block_k >= 32`. Before, exhaustive autotuning defaulted to a limited set of autotuning configs, as limitations for exhaustively autotuning on FP8 shapes had not been tested. Test Plan: ``` CUDA_VISIBLE_DEVICES=0 TRITON_PRINT_AUTOTUNING=1 TRITON_ALWAYS_COMPILE=1 TORCH_LOGS=+inductor TORCHINDUCTOR_FORCE_DISABLE_CACHES=1 ENABLE_PERSISTENT_TMA_MATMUL=1 TORCHINDUCTOR_MAX_AUTOTUNE_GEMM=1 TORCHINDUCTOR_MAX_AUTOTUNE_GEMM_SEARCH_SPACE=DEFAULT buck2 run mode/{opt,inplace} pytorch/t ritonbench:run -- --op fp8_gemm --only torch_fp8_gemm,pt2_fp8_gemm --metrics tflops,accuracy --input-loader=/home/jananisriram/personal/exhaustive_autotune_rowwise_persistent_tma/json_fi les/rowwise_ptma_0.json --output="/home/jananisriram/personal/exhaustive_autotune_rowwise_persistent_tma/autotune/gpu0_bench.csv" --atol=1e-2 --rtol=0.5 2>&1 | tee ~/personal/exhaustive_ autotune_rowwise_persistent_tma/autotune/gpu0.log ``` autotunes on the maximum configs available, rather than the defaults, and skips configs not compatible with TMA. Rollback Plan: Reviewed By: coconutruben Differential Revision: D80958642
Summary: X-link: pytorch/pytorch#161442 Validate exhaustive autotuning for FP8 Inductor templates: scaled MM templates require `block_k >= 32`. Before, exhaustive autotuning defaulted to a limited set of autotuning configs, as limitations for exhaustively autotuning on FP8 shapes had not been tested. Reviewed By: coconutruben Differential Revision: D80958642
facebook-github-bot
force-pushed
the
export-D80958642
branch
from
83d742c to
d637a66
Compare
facebook-github-bot
had a problem deploying
to
docker-s3-upload
GitHub Actions
Failure
facebook-github-bot
temporarily deployed
to
docker-s3-upload
GitHub Actions
Inactive
|
This pull request was exported from Phabricator. Differential Revision: D80958642 |
pytorch-bot bot
pushed a commit
to pytorch/pytorch
that referenced
this pull request
Aug 28, 2025
…ates (#161442) Summary: X-link: meta-pytorch/tritonbench#355 Validate exhaustive autotuning for FP8 Inductor templates: scaled MM templates require `block_k >= 32`. Before, exhaustive autotuning defaulted to a limited set of autotuning configs, as limitations for exhaustively autotuning on FP8 shapes had not been tested. Test Plan: ``` CUDA_VISIBLE_DEVICES=0 TRITON_PRINT_AUTOTUNING=1 TRITON_ALWAYS_COMPILE=1 TORCH_LOGS=+inductor TORCHINDUCTOR_FORCE_DISABLE_CACHES=1 ENABLE_PERSISTENT_TMA_MATMUL=1 TORCHINDUCTOR_MAX_AUTOTUNE_GEMM=1 TORCHINDUCTOR_MAX_AUTOTUNE_GEMM_SEARCH_SPACE=DEFAULT buck2 run mode/{opt,inplace} pytorch/t ritonbench:run -- --op fp8_gemm --only torch_fp8_gemm,pt2_fp8_gemm --metrics tflops,accuracy --input-loader=/home/jananisriram/personal/exhaustive_autotune_rowwise_persistent_tma/json_fi les/rowwise_ptma_0.json --output="/home/jananisriram/personal/exhaustive_autotune_rowwise_persistent_tma/autotune/gpu0_bench.csv" --atol=1e-2 --rtol=0.5 2>&1 | tee ~/personal/exhaustive_ autotune_rowwise_persistent_tma/autotune/gpu0.log ``` autotunes on the maximum configs available, rather than the defaults, and skips configs not compatible with TMA. Rollback Plan: Reviewed By: coconutruben Differential Revision: D80958642
pytorch-bot bot
pushed a commit
to pytorch/pytorch
that referenced
this pull request
Sep 2, 2025
…ates (#161442) Summary: X-link: meta-pytorch/tritonbench#355 Pull Request resolved: #161442 Validate exhaustive autotuning for FP8 Inductor templates: scaled MM templates require `block_k >= 32`. Before, exhaustive autotuning defaulted to a limited set of autotuning configs, as limitations for exhaustively autotuning on FP8 shapes had not been tested. Test Plan: ``` CUDA_VISIBLE_DEVICES=0 TRITON_PRINT_AUTOTUNING=1 TRITON_ALWAYS_COMPILE=1 TORCH_LOGS=+inductor TORCHINDUCTOR_FORCE_DISABLE_CACHES=1 ENABLE_PERSISTENT_TMA_MATMUL=1 TORCHINDUCTOR_MAX_AUTOTUNE_GEMM=1 TORCHINDUCTOR_MAX_AUTOTUNE_GEMM_SEARCH_SPACE=DEFAULT buck2 run mode/{opt,inplace} pytorch/t ritonbench:run -- --op fp8_gemm --only torch_fp8_gemm,pt2_fp8_gemm --metrics tflops,accuracy --input-loader=/home/jananisriram/personal/exhaustive_autotune_rowwise_persistent_tma/json_fi les/rowwise_ptma_0.json --output="/home/jananisriram/personal/exhaustive_autotune_rowwise_persistent_tma/autotune/gpu0_bench.csv" --atol=1e-2 --rtol=0.5 2>&1 | tee ~/personal/exhaustive_ autotune_rowwise_persistent_tma/autotune/gpu0.log ``` autotunes on the maximum configs available, rather than the defaults, and skips configs not compatible with TMA. Rollback Plan: Reviewed By: coconutruben Differential Revision: D80958642
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
X-link: pytorch/pytorch#161442
Validate exhaustive autotuning for FP8 Inductor templates: scaled MM templates require
block_k >= 32. Before, exhaustive autotuning defaulted to a limited set of autotuning configs, as limitations for exhaustively autotuning on FP8 shapes had not been tested.Differential Revision: D80958642