[AUTOCUT] Gradle Check Flaky Test Report for KeywordTermsAggregatorTests

[opensearch-ci-bot]

Flaky Test Report for KeywordTermsAggregatorTests

Noticed the KeywordTermsAggregatorTests has some flaky, failing tests that failed during post-merge actions and timer trigerred run on main branch.

Details

Git Reference	Merged Pull Request	Build Details	Test Name
main	null	60434 60616	org.opensearch.search.aggregations.startree.KeywordTermsAggregatorTests.testStarTreeKeywordTerms
5ac8177	18663	60240	org.opensearch.search.aggregations.startree.KeywordTermsAggregatorTests.testStarTreeKeywordTerms
397a5e7	18469	58976	org.opensearch.search.aggregations.startree.KeywordTermsAggregatorTests.testStarTreeKeywordTerms
bf8cd25	18103	57786	org.opensearch.search.aggregations.startree.KeywordTermsAggregatorTests.testStarTreeKeywordTerms

The other pull requests, besides those involved in post-merge actions, that contain failing tests with the KeywordTermsAggregatorTests class are:

• 17165
• 17855
• 18161
• 17370
• 17712
• 17919
• 17925
• 18229
• 18467
• 18483
• 18494
• 18565
• 18702
• 18732
• 18766
• 18874
• 18921
• 19001

For more details on the failed tests refer to OpenSearch Gradle Check Metrics dashboard.

[andrross]

FYI @sandeshkr419

[andrross]

This is reproducible:

REPRODUCE WITH: ./gradlew ':server:test' --tests "org.opensearch.search.aggregations.startree.KeywordTermsAggregatorTests.testStarTreeKeywordTerms" -Dtests.seed=DAC644BCC974366C -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=cgg-Latn-UG -Dtests.timezone=Asia/Thimphu -Druntime.java=21

KeywordTermsAggregatorTests > testStarTreeKeywordTerms FAILED
    java.lang.AssertionError: expected:<0> but was:<6>
        at __randomizedtesting.SeedInfo.seed([DAC644BCC974366C:E37AAA48F3F907FB]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.failNotEquals(Assert.java:835)
        at org.junit.Assert.assertEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:633)
        at org.opensearch.search.aggregations.AggregatorTestCase.searchAndReduceStarTree(AggregatorTestCase.java:810)
        at org.opensearch.search.aggregations.startree.KeywordTermsAggregatorTests.testCase(KeywordTermsAggregatorTests.java:207)
        at org.opensearch.search.aggregations.startree.KeywordTermsAggregatorTests.testStarTreeKeywordTerms(KeywordTermsAggregatorTests.java:160)

[sandeshkr419]

Did a bit of dive-deep on it, couldn't get to the exact reason yet.

So basically this is happening when multiple segments are present for the index and for some cases the decision to use star-tree is false (meaning it is reverting to the default flow).

Next step can be to isolate the case for a segment when the existing conditions are not suffice to force it to use star-tree.

I don't want to just force the segment count to 1 to fix the test before knowing the exact reason why star-tree is (or cannot) be used.

---

Basically I forced merge segment count to 1 to get the test passing to come up with the above hypothesis.

expected:<0> but was:<6>

The assertion is on the number of docValues visited - collect() invoked which should be 0 for star-tree case as doc-values need not be visited once the result is pre-computed.

The final aggregation result (buckets) is however correct - checked from the final bucket comparison assertions.