Commit 3cf9b5d
[Inductor][FP8] Validate exhaustive autotuning for FP8 Inductor templates (pytorch#161442)
Summary:
X-link: meta-pytorch/tritonbench#355
Pull Request resolved: pytorch#161442
Validate exhaustive autotuning for FP8 Inductor templates: scaled MM templates require `block_k >= 32`. Before, exhaustive autotuning defaulted to a limited set of autotuning configs, as limitations for exhaustively autotuning on FP8 shapes had not been tested.
Test Plan:
```
CUDA_VISIBLE_DEVICES=0 TRITON_PRINT_AUTOTUNING=1 TRITON_ALWAYS_COMPILE=1 TORCH_LOGS=+inductor TORCHINDUCTOR_FORCE_DISABLE_CACHES=1 ENABLE_PERSISTENT_TMA_MATMUL=1 TORCHINDUCTOR_MAX_AUTOTUNE_GEMM=1 TORCHINDUCTOR_MAX_AUTOTUNE_GEMM_SEARCH_SPACE=DEFAULT buck2 run mode/{opt,inplace} pytorch/t
ritonbench:run -- --op fp8_gemm --only torch_fp8_gemm,pt2_fp8_gemm --metrics tflops,accuracy --input-loader=/home/jananisriram/personal/exhaustive_autotune_rowwise_persistent_tma/json_fi
les/rowwise_ptma_0.json --output="/home/jananisriram/personal/exhaustive_autotune_rowwise_persistent_tma/autotune/gpu0_bench.csv" --atol=1e-2 --rtol=0.5 2>&1 | tee ~/personal/exhaustive_
autotune_rowwise_persistent_tma/autotune/gpu0.log
```
autotunes on the maximum configs available, rather than the defaults, and skips configs not compatible with TMA.
Rollback Plan:
Differential Revision: D809586421 parent f192dc8 commit 3cf9b5d
1 file changed
+6
-13
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1522 | 1522 | | |
1523 | 1523 | | |
1524 | 1524 | | |
1525 | | - | |
| 1525 | + | |
1526 | 1526 | | |
1527 | | - | |
| 1527 | + | |
1528 | 1528 | | |
1529 | 1529 | | |
1530 | 1530 | | |
| |||
1571 | 1571 | | |
1572 | 1572 | | |
1573 | 1573 | | |
1574 | | - | |
1575 | 1574 | | |
1576 | 1575 | | |
1577 | 1576 | | |
| |||
1596 | 1595 | | |
1597 | 1596 | | |
1598 | 1597 | | |
1599 | | - | |
1600 | | - | |
1601 | | - | |
1602 | | - | |
1603 | | - | |
| 1598 | + | |
| 1599 | + | |
| 1600 | + | |
| 1601 | + | |
1604 | 1602 | | |
1605 | 1603 | | |
1606 | 1604 | | |
| |||
1614 | 1612 | | |
1615 | 1613 | | |
1616 | 1614 | | |
1617 | | - | |
1618 | | - | |
1619 | | - | |
1620 | | - | |
1621 | | - | |
1622 | 1615 | | |
1623 | 1616 | | |
1624 | 1617 | | |
| |||
0 commit comments