You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Check numerical equivalence / closeness between different kernel preferences
Summary:
This PR checks different kernel preferences for Float8Tensor are similar in numerics
(AUTO, TORCH and FBGEMM)
triton implementation and torchao implementation are a bit different right now actually, need to decide if we should fix it or not
main difference seems to be the triton implementation is using:
```
a_scale = MAX_FP8 / max_abs
then do
a_scale = 1.0 / a_scale
a_fp8 = a * a_scale
```
while torch is doing:
```
a_scale = max_abs / MAX_FP8
a_fp8 = a / a_scale
```
Also the hp_value_lb and hp_value_ub settings are slightly different
Test Plan:
python test/quantization/quantize_/workflows/float8/test_float8_tensor.py -k test_kernel_preference_numerical_equivalence
Reviewers:
Subscribers:
Tasks:
Tags:
stack-info: PR: #2651, branch: jerryzh168/stack/15
0 commit comments