Skip to content

Commit 2841bb0

Browse files
Kyle1668Quentin-Anthony
authored andcommitted
Resolved potential transformer engine bug
1 parent c60d595 commit 2841bb0

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

megatron/model/transformer_engine.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -646,8 +646,8 @@ def __init__(self, neox_args):
646646
override_linear_precision = (False, False, not neox_args.te_fp8_wgrad)
647647

648648
super().__init__(
649-
margin=neox_args.fp8_margin,
650-
fp8_format=te_fp8_format,
649+
margin=neox_args.te_fp8_margin,
650+
fp8_format=neox_args.te_fp8_format,
651651
amax_compute_algo=neox_args.te_fp8_amax_compute_algo,
652652
amax_history_len=neox_args.te_fp8_amax_history_len,
653653
override_linear_precision=override_linear_precision,

0 commit comments

Comments
 (0)