fix(layernorm): allow normalized_size==1 instead of error (#25896) #26034
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
norm_size == 1; only reject zero-size (and less than zero).norm_size <= 1→norm_size < 1.onnxruntime/core/providers/cpu/nn/layer_norm_helper.hLayerNorm_NormSizeEqualsOne_Succeeds: with bias → output equals bias.LayerNorm_NormSizeEqualsOne_NoBias: without bias → output all zeros.onnxruntime/test/contrib_ops/layer_norm_op_test.ccnorm_size == 1, yielding bias (if provided) as final output.Motivation and Context
norm_size == 1:norm_size == 1, and PyTorch returns zeros in this case; ONNX Runtime should align for backward compatibility with older models.norm_size == 1:mean == x,var == 0→normalized == 0→ output equals bias if provided, else0.norm_size > 1.Resolves: #25896
Files Changed
onnxruntime/core/providers/cpu/nn/layer_norm_helper.hnorm_size == 1; only rejectnorm_size < 1.onnxruntime/test/contrib_ops/layer_norm_op_test.cc