Skip to content

Commit 0d51dfb

Browse files
byungilmbyungilm
authored
Add condition for Matmul and Select in ClampFP16OutputSoftmaxMatcher (#31798)
### Description of the issue - symptom: audio output file is invalid. (Empty audio file) Output file of fp32 inference sounds fine. - root-cause: Matmul layer generated 'inf' value to output <img width="1500" height="598" alt="image" src="https://github.com/user-attachments/assets/17b13ca8-0f47-47c4-8c3c-6076635241ce" /> intermediate buffer comparison with fp32 inference : <html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns="http://www.w3.org/TR/REC-html40"> <head> <meta name=ProgId content=OneNote.File> <meta name=Generator content="Microsoft OneNote 15"> </head> <body lang=en-US style='font-family:Calibri;font-size:11.0pt'> <!--StartFragment--> <div style='direction:ltr'> Dst0 | data1 shape: [1, 2, 1, 411, 411] data2 shape: [1, 2, 1, 411, 411] Number of items: 337842 / 337842 Max value: inf / 738612.875 Min value: -inf / -1369326.500 Max per-pixel diff: inf Total per-pixel diff: inf Average per-pixel diff: inf Total sign diff: 232 -- | -- Src1 | data1 shape: [1, 2, 1, 96, 411] data2 shape: [1, 2, 1, 96, 411] Number of items: 78912 / 78912 Max value: 529.000 / 527.687 Min value: -523.500 / -522.751 Max per-pixel diff: 3.043 Total per-pixel diff: 18146.697 Average per-pixel diff: 0.230 Total sign diff: 19 Src2 | data1 shape: [1, 2, 1, 411, 411] data2 shape: [1, 2, 1, 411, 411] Number of items: 337842 / 337842 Max value: 295.500 / 296.001 Min value: -281.250 / -281.503 Max per-pixel diff: 0.920 Total per-pixel diff: 611.090 Average per-pixel diff: 0.002 Total sign diff: 1 Src0 | data1 shape: [1, 2, 1, 411, 96] data2 shape: [1, 2, 1, 411, 96] Number of items: 78912 / 78912 Max value: 58.031 / 58.034 Min value: -58.031 / -57.976 Max per-pixel diff: 0.251 Total per-pixel diff: 1893.853 Average per-pixel diff: 0.024 Total sign diff: 44 </div> <!--EndFragment--> </body> </html> ACTIVATIONS_SCALE_FACTOR did not work for this case. - how resolved: Add matching case of Select layer to a matcher of ClampFP16OutputSoftmaxMatcher to prevent 'inf' value propagated through softmax #### The code and line that caused this issue - Accuracy issue : Missing logic to prevent 'inf' value passed to softmax #### Reproduction step and snapshot - Execute openvino notebooks/openvoice2-and-melotts #### Checklist - [x] Is it a proper fix? (not a workaround) - [x] Did you include test case for this fix, if necessary? - [x] Did you review existing test that can be extended to cover this scenario? Which test did you review? : reviewed unittests with "ClampFp16OutputTest" ### Tickets: - CVS-169046 Signed-off-by: Min, Byungil <[email protected]>
1 parent 8653c14 commit 0d51dfb

File tree

2 files changed

+36
-2
lines changed

2 files changed

+36
-2
lines changed

‎src/plugins/intel_gpu/src/plugin/transformations/clamp_fp16_output.cpp‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
#include "openvino/op/multiply.hpp"
1515
#include "openvino/op/subtract.hpp"
1616
#include "openvino/op/divide.hpp"
17+
#include "openvino/op/select.hpp"
1718
#include "openvino/core/rt_info.hpp"
1819
#include "openvino/pass/pattern/op/pattern.hpp"
1920
#include "openvino/pass/pattern/op/wrap_type.hpp"
@@ -38,10 +39,12 @@ ClampFP16OutputSoftmaxMatcher::ClampFP16OutputSoftmaxMatcher() {
3839
auto in1 = any_input(class_other_than<v0::Constant>());
3940
auto matmul_m = wrap_type<v0::MatMul>({in0, in1}, type_matches(ov::element::f16) && consumers_count(1));
4041
auto reshape_m = wrap_type<v1::Reshape>({matmul_m, any_input()}, type_matches(ov::element::f16) && consumers_count(1));
41-
auto add_m = wrap_type<v1::Add>({matmul_m, any_input()}, type_matches(ov::element::f16) && consumers_count(1));
4242
auto eltwise_m = wrap_type<v1::Divide, v1::Add, v1::Multiply, v1::Subtract>({matmul_m, any_input()},
4343
type_matches(ov::element::f16) && consumers_count(1));
44-
auto softmax_input_m = std::make_shared<Or>(ov::OutputVector{eltwise_m, reshape_m, matmul_m});
44+
auto input_m = std::make_shared<Or>(ov::OutputVector{matmul_m, eltwise_m, reshape_m});
45+
auto select_then_m = wrap_type<v1::Select>({any_input(), input_m, any_input()}, type_matches(ov::element::f16) && consumers_count(1));
46+
auto select_else_m = wrap_type<v1::Select>({any_input(), any_input(), input_m}, type_matches(ov::element::f16) && consumers_count(1));
47+
auto softmax_input_m = std::make_shared<Or>(ov::OutputVector{select_then_m, select_else_m, input_m});
4548
auto softmax_m = wrap_type<v8::Softmax>({softmax_input_m}, type_matches(ov::element::f16));
4649

4750
ov::matcher_pass_callback callback = [OV_CAPTURE_CPY_AND_THIS](ov::pass::pattern::Matcher& m) {

‎src/plugins/intel_gpu/tests/unit/transformations/clamp_fp16_output_test.cpp‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@
2020
#include "openvino/op/maximum.hpp"
2121
#include "openvino/op/matmul.hpp"
2222
#include "openvino/op/softmax.hpp"
23+
#include "openvino/op/select.hpp"
2324
#include <plugin/transformations/clamp_fp16_output.hpp>
2425
#include <transformations/init_node_info.hpp>
2526
#include <transformations/utils/utils.hpp>
@@ -159,6 +160,36 @@ TEST_F(TransformationTestsF, ClampFp16OutputTest6) {
159160
comparator.enable(FunctionsComparator::CmpValues::ATTRIBUTES);
160161
}
161162

163+
TEST_F(TransformationTestsF, ClampFp16OutputTest7) {
164+
{
165+
auto input1 = std::make_shared<ov::op::v0::Parameter>(ov::element::f16, ov::Shape{ 3, 2, 2 });
166+
auto input2 = std::make_shared<ov::op::v0::Parameter>(ov::element::f16, ov::Shape{ 1, 2, 2 });
167+
auto matmul = std::make_shared<ov::op::v0::MatMul>(input1, input2, true, false);
168+
auto data = std::make_shared<ov::op::v0::Parameter>(ov::element::f16, ov::Shape{ 3, 2, 2 });
169+
auto cond = std::make_shared<ov::op::v0::Parameter>(ov::element::boolean, ov::Shape{ 3, 2, 2 });
170+
auto select = std::make_shared<ov::op::v1::Select>(cond, data, matmul);
171+
auto softmax = std::make_shared<ov::op::v8::Softmax>(select, 1);
172+
173+
model = std::make_shared<ov::Model>(ov::OutputVector{softmax}, ov::ParameterVector{input1, input2, data, cond});
174+
manager.register_pass<ClampFP16Output>();
175+
}
176+
{
177+
auto input1 = std::make_shared<ov::op::v0::Parameter>(ov::element::f16, ov::Shape{ 3, 2, 2 });
178+
auto input2 = std::make_shared<ov::op::v0::Parameter>(ov::element::f16, ov::Shape{ 1, 2, 2 });
179+
auto matmul = std::make_shared<ov::op::v0::MatMul>(input1, input2, true, false);
180+
auto data = std::make_shared<ov::op::v0::Parameter>(ov::element::f16, ov::Shape{ 3, 2, 2 });
181+
auto cond = std::make_shared<ov::op::v0::Parameter>(ov::element::boolean, ov::Shape{ 3, 2, 2 });
182+
auto select = std::make_shared<ov::op::v1::Select>(cond, data, matmul);
183+
auto min = static_cast<double>(std::numeric_limits<ov::float16>::lowest());
184+
auto max = static_cast<double>(std::numeric_limits<ov::float16>::max());
185+
auto clamp = std::make_shared<ov::op::v0::Clamp>(select, min, max);
186+
auto softmax = std::make_shared<ov::op::v8::Softmax>(clamp, 1);
187+
188+
model_ref = std::make_shared<ov::Model>(ov::OutputVector{softmax}, ov::ParameterVector{input1, input2, data, cond});
189+
}
190+
comparator.enable(FunctionsComparator::CmpValues::ATTRIBUTES);
191+
}
192+
162193
TEST_F(TransformationTestsF, ClampFp16OutputRMS) {
163194
{
164195
auto input1 = std::make_shared<ov::op::v0::Parameter>(ov::element::f16, ov::PartialShape{ -1, -1, 2560 });

0 commit comments

Comments
 (0)