[Repository S3] Move async http client to CRT from Netty and add configurability to chose client via repo settings

[pranikum]

Description

As part of this PR we are adding support for AwsCrtAsyncHttpClient in the S3AsyncService . Currently the default Http client used was NettyAsyncHttpClient. However as per our perf testing we have seen an improvement of approximately 5-7% when using AwsCrtAsyncHttpClient instead of NettyAsyncHttpClient. We will be making AwsCrtAsyncHttpClient as default. NettyAsyncHttpClient will still be supported through setting

Related Issues

Resolves #18535

Check List

• [Done] Functionality includes testing.
• [N/A] API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for 21d84e2: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 838c875: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[RamakrishnaChilaka]

As part of this PR we are adding support for S3 CRT client for S3 async client.

@pranikum Just to clarify, we’re not adding the S3 CRT client in addition to the existing S3 async client, but rather giving an option to replace the Netty HTTP client used in the existing S3 async client with the CRT HTTP client.

Resolves #18535

Therefore, with this change, we can't say that we are resolving the above issue, which explicitly calls to explore S3 based CRT client. (see ref)

[pranikum]

S3 CRT client for S3 async client.

@pranikum Just to clarify, we’re not adding the S3 CRT client in addition to the existing S3 async client, but rather giving an option to replace the Netty HTTP client used in the existing S3 async client with the CRT HTTP client.

Resolves #18535

Therefore, with this change, we can't say that we are resolving the above issue, which explicitly calls to explore S3 based CRT client. (see ref)

Not sure if I understood this. We are making "crt" as the default based on setting. The value can be overridden based on setting to "netty"

[pranikum]

S3 CRT client for S3 async client.

@pranikum Just to clarify, we’re not adding the S3 CRT client in addition to the existing S3 async client, but rather giving an option to replace the Netty HTTP client used in the existing S3 async client with the CRT HTTP client.

Resolves #18535

Therefore, with this change, we can't say that we are resolving the above issue, which explicitly calls to explore S3 based CRT client. (see ref)

Not sure if I understood this. We are making "crt" as the default based on setting. The value can be overridden based on setting to "netty"

Got the comment. Thanks for pointing the same. Updating the Issue/PR description

[github-actions]

✅ Gradle check result for 095fb23: SUCCESS

[codecov]

Codecov Report

❌ Patch coverage is 79.56989% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.87%. Comparing base (00beca1) to head (799c5f6).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
...org/opensearch/repositories/s3/S3AsyncService.java	80.24%	9 Missing and 7 partials ⚠️
...opensearch/repositories/s3/S3RepositoryPlugin.java	50.00%	0 Missing and 3 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #18800      +/-   ##
============================================
+ Coverage     72.80%   72.87%   +0.07%     
+ Complexity    69399    69394       -5     
============================================
  Files          5648     5648              
  Lines        319230   319329      +99     
  Branches      46174    46194      +20     
============================================
+ Hits         232421   232723     +302     
+ Misses        68003    67690     -313     
- Partials      18806    18916     +110     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

❌ Gradle check result for 149a42b: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[gbbafna]

Looks good overall .

[github-actions]

❌ Gradle check result for e1347ad: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for f75436d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for f75436d: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for b4a21c5: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for b4a21c5: SUCCESS

[github-actions]

❕ Gradle check result for 799c5f6: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.