[AUTOCUT] Gradle Check Flaky Test Report for ClientYamlTestSuiteIT

[opensearch-ci-bot]

Flaky Test Report for ClientYamlTestSuiteIT

Noticed the ClientYamlTestSuiteIT has some flaky, failing tests that failed during post-merge actions and timer trigerred run on main branch.

Details

Git Reference	Merged Pull Request	Build Details	Test Name
main	null	61047 62309 62314 62323 62327 62335 49451 57351 60802 62063 62124 62524 62838 62900 63398 63488 63855 64209	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=index/111_false_allow_templates/Index documents with setting dynamic parameter to false_allow_templates in the mapping of the index} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/numeric profiler} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search/400_combined_fields/Combined_fields query with OR operator favors high-frequency single-term docs} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/10_histogram/histogram profiler} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals} org.opensearch.test.rest.ClientYamlTestSuiteIT.classMethod org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=indices.refresh/10_basic/Indices refresh test no-match wildcard} org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=indices.stats/13_fields/Fields - pattern}
9f28ef8	18825	62305	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=index/111_false_allow_templates/Index documents with setting dynamic parameter to false_allow_templates in the mapping of the index}
bf5b43d	18910	61893	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search/400_combined_fields/Combined_fields query with OR operator favors high-frequency single-term docs}
2e027ee	18939	61825	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search/400_combined_fields/Combined_fields query with OR operator favors high-frequency single-term docs}
9b79f19	18887	61740	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}
46a0045	18636	59978	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/330_auto_date_histogram/profile at top level}
0e2ec6f	18541	59739	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/330_auto_date_histogram/profile at top level}
3c3d380	18072	57136	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/10_histogram/date_histogram profiler}
481fd38	16098	48529	org.opensearch.test.rest.ClientYamlTestSuiteIT.classMethod org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=indices.segments/10_basic/basic segments test}
c85ee68	15988	48303	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/360_date_histogram/Date histogram aggregation w/ shared field range test}
d64c5f8	15866	47801	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/360_date_histogram/Date histogram aggregation w/ shared field range test}
2742e74	15700	47023	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/360_date_histogram/Date histogram aggregation w/ shared field range test}
0df6e9e	15098	43940	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=indices.delete_index_template/10_basic/Delete index template which is not used by data stream but index pattern matches}
fdd3ea7	15023	43647	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}
ed24a14	14881	43113	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}
7a54f71	14707	42236	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}
8b62948	13953	39797	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}

The other pull requests, besides those involved in post-merge actions, that contain failing tests with the ClientYamlTestSuiteIT class are:

• 18814
• 15024
• 17329
• 16366
• 16803
• 18631
• 14443
• 19209
• 18842
• 15124
• 15568
• 17383
• 17718
• 16294
• 15131
• 18683
• 19009
• 13853
• 13829
• 18568
• 19304
• 18424
• 15333
• 16884
• 18277
• 15258
• 18632
• 18901
• 15922
• 19008
• 13865
• 17756
• 13788
• 18523
• 17772
• 13821
• 13620
• 17638
• 17009
• 14446
• 13232
• 18195
• 18777
• 13461
• 18693
• 17740
• 17023
• 14383
• 15039
• 17978
• 18673
• 15012
• 15365
• 18455
• 18707
• 18996
• 19033
• 18504
• 19152
• 14069
• 14464
• 15545
• 18161
• 18215
• 19115
• 14454
• 15289
• 18618
• 14718
• 14774
• 15882
• 17039
• 18597
• 18879
• 14613
• 18929
• 14711
• 19054
• 13573
• 14486
• 15194
• 15410
• 15509
• 15607
• 16297
• 16731
• 16960
• 17447
• 17919
• 17951
• 18718
• 18936
• 19046
• 19057
• 14817
• 15145
• 15256
• 15557
• 15603
• 18498
• 18601
• 18874
• 13190
• 13929
• 14611
• 14630
• 15100
• 15153
• 15358
• 15381
• 15578
• 16038
• 16726
• 17273
• 17712
• 17750
• 18106
• 18194
• 18491
• 18664
• 18896
• 18924
• 18966
• 19076
• 19396
• 14513
• 14415
• 17855
• 18544
• 12782
• 13772
• 13888
• 14118
• 14195
• 14684
• 15128
• 15326
• 15430
• 15649
• 15679
• 16021
• 16904
• 18184
• 18346
• 18467
• 18479
• 18511
• 18668
• 18889
• 18923
• 18961
• 18978
• 19136
• 19283
• 19307
• 19403

For more details on the failed tests refer to OpenSearch Gradle Check Metrics dashboard.

[getsaurabh02]

@prudhvigodithi can we confirm if these failures are pointing to an individual test or the overall test class?

[getsaurabh02]

Also, can we highlight the branch the test is failing on, in the issue description?

[prudhvigodithi]

Hey @getsaurabh02 today the gradle check runs on a commit ref, the issue description, the markdown table created has the commit that has the flaky failure. Here for this issue the commit 8b62948 has a flaky failure for the test class ClientYamlTestSuiteIT and the failing test is org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}.

Sometimes there are can be more than one test failing for the same commit or for the different one, here is the example #14381, in that case we group them under one issue created for a specific test class name.