Grok processor supports capturing multiple values for same field name

[gaobinlong]

Description

Add an option capture_all_matches for the grok ingest processor, when set to true, all matched values for same field name will be collected into an array, by default, only the first the matched value will be captured.

By checking the code:

OpenSearch/libs/grok/src/main/java/org/opensearch/grok/GrokCaptureType.java

Line 113 in 89edd4c

return; // Capture only the first value.

, found that maybe the current behavior is by design, but I also checked other ingestion tool like Logstash, it captures all matched values by default, so I think we can let the users decide the behavior by adding an extra option.

Related Issues

#18790

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

✅ Gradle check result for 6b39893: SUCCESS

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.96%. Comparing base (1ca590a) to head (b6f29e8).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##               main   #18799   +/-   ##
=========================================
  Coverage     72.95%   72.96%           
- Complexity    69804    69816   +12     
=========================================
  Files          5667     5667           
  Lines        320532   320582   +50     
  Branches      46397    46405    +8     
=========================================
+ Hits         233847   233914   +67     
+ Misses        67768    67706   -62     
- Partials      18917    18962   +45     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[varunbharadwaj]

Looks good to me. We call it capture_all_matches, but if we have a pattern matching multiple groups (for example word1,word2,word3, with pattern (%{WORD:word},)+), it looks like we only retrieve word3. We could highlight this in the docs later on.

[opensearch-trigger-bot]

This PR is stalled because it has been open for 30 days with no activity.

[github-actions]

✅ Gradle check result for b6f29e8: SUCCESS