[DRAFT PR]: [No Need to review]: S3 sse support final #2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Pranit Kumar <[email protected]>
Signed-off-by: Pranit Kumar <[email protected]>
gbbafna
reviewed
Aug 28, 2025
| } | ||
|
|
||
| private boolean isCompositeRepository(RepositoryMetadata repositoryMetadata) { | ||
| return repositoryMetadata.settings().hasValue(REPOSITORY_SETTINGS_SSE_ENABLED_ATTRIBUTE_KEY); |
There was a problem hiding this comment.
we can see if the cryptoMetadata is present . If yes, we can directly use that .
There was a problem hiding this comment.
Also we need it for remote store segment and translog .
gbbafna
reviewed
Aug 28, 2025
| }, | ||
| Property.IndexScope, | ||
| Property.PrivateIndex, | ||
| Property.Dynamic |
| /** | ||
| * Composite Repository for the ServerSideEncryption support. | ||
| */ | ||
| public class CompositeRemoteRepository { |
There was a problem hiding this comment.
updating repo settings should just it for both the repositories .
There was a problem hiding this comment.
The alternate is we can update it manually .
…opensearch-project#19113) * Fix access specifier for FieldMapper method to allow usage by plugins Signed-off-by: Mohit Godwani <[email protected]> * Apply spotless Signed-off-by: Mohit Godwani <[email protected]> --------- Signed-off-by: Mohit Godwani <[email protected]>
…igurability to chose client via repo settings (opensearch-project#18800) Signed-off-by: Pranit Kumar <[email protected]>
….38.0 in /plugins/repository-gcs (opensearch-project#19144) * Bump com.google.auth:google-auth-library-oauth2-http Bumps com.google.auth:google-auth-library-oauth2-http from 1.37.1 to 1.38.0. --- updated-dependencies: - dependency-name: com.google.auth:google-auth-library-oauth2-http dependency-version: 1.38.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…dfs-fixture (opensearch-project#19146) * Bump com.squareup.okio:okio in /test/fixtures/hdfs-fixture Bumps [com.squareup.okio:okio](https://github.com/square/okio) from 3.15.0 to 3.16.0. - [Release notes](https://github.com/square/okio/releases) - [Changelog](https://github.com/square/okio/blob/master/CHANGELOG.md) - [Commits](square/okio@parent-3.15.0...parent-3.16.0) --- updated-dependencies: - dependency-name: com.squareup.okio:okio dependency-version: 3.16.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Craig Perkins <[email protected]>
…rch-project#19151) Signed-off-by: Craig Perkins <[email protected]>
…ns/repository-azure (opensearch-project#19145) * Bump com.azure:azure-storage-common in /plugins/repository-azure Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.30.1 to 12.30.2. - [Release notes](https://github.com/Azure/azure-sdk-for-java/releases) - [Commits](Azure/azure-sdk-for-java@azure-storage-blob_12.30.1...azure-storage-common_12.30.2) --- updated-dependencies: - dependency-name: com.azure:azure-storage-common dependency-version: 12.30.2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Updating SHAs Signed-off-by: dependabot[bot] <[email protected]> * Update changelog Signed-off-by: dependabot[bot] <[email protected]> * Upgrade com.azure:azure-storage-blob to 12.31.2 Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Craig Perkins <[email protected]>
Signed-off-by: Andrey Pleskach <[email protected]>
…arch-project#19060) * Add query rewriting infrastructure to reduce query complexity Implements three query optimizations that work together: - Boolean flattening: removes unnecessary nested boolean queries - Terms merging: combines multiple term queries on same field in filter/should contexts - Match-all removal: eliminates redundant match_all queries Key features: - 60-70% reduction in query nodes for typical filtered queries - Feature flag: search.query_rewriting.enabled (default: true) - Preserves exact query semantics and results Signed-off-by: Atri Sharma <[email protected]> * Fix forbidden api issues Signed-off-by: Atri Sharma <[email protected]> * Update writers and get tests to pass Signed-off-by: Atri Sharma <[email protected]> * Update per CI Signed-off-by: Atri Sharma <[email protected]> * Fix term merging threshold and update comments Signed-off-by: Atri Sharma <[email protected]> * Expose setting and update per comments Signed-off-by: Atri Sharma <[email protected]> * Update CHANGELOG Signed-off-by: Atri Sharma <[email protected]> * Fix tests and ensure scoring MATCH ALL query is preserved Signed-off-by: Atri Sharma <[email protected]> * Migrate must to filter and must not to should optimizations to query rewriting infrastructure This commit migrates two existing query optimizations from BoolQueryBuilder to the new query rewriting infrastructure: 1. **MustToFilterRewriter**: Moves non scoring queries (range, geo, numeric term/terms/match) from must to filter clauses to avoid unnecessary scoring calculations (from PR opensearch-project#18541) 2. **MustNotToShouldRewriter**: Transforms negative queries into positive complements for better performance on single valued numeric fields (from PRs opensearch-project#17655 and opensearch-project#18498) Changes: Add MustToFilterRewriter with priority 150 (runs after boolean flattening) Add MustNotToShouldRewriter with priority 175 (runs after must to filter) Register both rewriters in QueryRewriterRegistry Add comprehensive test suites (15 tests for must to filter, 14 for must not to should) Disable legacy implementations in BoolQueryBuilder Comment out BoolQueryBuilder tests that relied on the old implementations The new rewriters maintain full backward compatibility while providing: Better separation of concerns Recursive rewriting for nested boolean queries Proper error handling and logging Consistent priority based execution order Signed-off-by: Atri Sharma <[email protected]> * Handle fields with missing fields Signed-off-by: Atri Sharma <[email protected]> --------- Signed-off-by: Atri Sharma <[email protected]>
* Fix tika CVE Signed-off-by: Prudhvi Godithi <[email protected]> * Update CHANGELOG.md Signed-off-by: Prudhvi Godithi <[email protected]> * fix html parser Signed-off-by: Prudhvi Godithi <[email protected]> * fix html parser Signed-off-by: Prudhvi Godithi <[email protected]> * fix html parser Signed-off-by: Prudhvi Godithi <[email protected]> * Add license Signed-off-by: Prudhvi Godithi <[email protected]> * Add license Signed-off-by: Prudhvi Godithi <[email protected]> * Update checksums Signed-off-by: Prudhvi Godithi <[email protected]> * Update shas Signed-off-by: Prudhvi Godithi <[email protected]> * Add pdf box license Signed-off-by: Prudhvi Godithi <[email protected]> * Fix tests Signed-off-by: Prudhvi Godithi <[email protected]> * Update security fonts permission Signed-off-by: Prudhvi Godithi <[email protected]> * Add dummy fonts Signed-off-by: Prudhvi Godithi <[email protected]> * Upstream fetch Signed-off-by: Prudhvi Godithi <[email protected]> * Fix license check error Signed-off-by: Prudhvi Godithi <[email protected]> * Fix license check error Signed-off-by: Prudhvi Godithi <[email protected]> --------- Signed-off-by: Prudhvi Godithi <[email protected]>
…ecated (opensearch-project#19154) * Replace centos:8 with almalinux:8 since centos docker images are deprecated Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Update Dockerfile Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]>
…oject#19153) * Add query in QueryCollectorContextSpecFactory Signed-off-by: vibrantvarun <[email protected]> * Add javadoc Signed-off-by: vibrantvarun <[email protected]> --------- Signed-off-by: vibrantvarun <[email protected]>
…nux:8 (opensearch-project#19159) Signed-off-by: Simon Marty <[email protected]>
* Add overload for channelFactory Signed-off-by: Rajat Gupta <[email protected]> * Fix tests Signed-off-by: Rajat Gupta <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Fix conflicts Signed-off-by: Rajat Gupta <[email protected]> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <[email protected]> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <[email protected]> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Switch to CompletionStage Signed-off-by: Craig Perkins <[email protected]> * Update CHANGELOG entry Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Add secondary constructor Signed-off-by: Rajat Gupta <[email protected]> * Modify changelog Signed-off-by: Rajat Gupta <[email protected]> * Update changelog Signed-off-by: Rajat Gupta <[email protected]> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <[email protected]> --------- Signed-off-by: Rajat Gupta <[email protected]> Signed-off-by: Atri Sharma <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Co-authored-by: Rajat Gupta <[email protected]> Co-authored-by: Atri Sharma <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Anurag Rai <[email protected]>
…-project#18389) The deprecated versions of this method take a type parameter, support for which was removed back in 2.0. The parameter is not used. I have kept the deprecated methods so as to not break downstream components that may be using them but changed all the code in server to stop passing in a type parameter. Signed-off-by: Andrew Ross <[email protected]>
* Add bwc version 2.19.4 Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update libs/core/src/main/java/org/opensearch/Version.java Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Craig Perkins <[email protected]> Co-authored-by: opensearch-ci-bot <[email protected]> Co-authored-by: Craig Perkins <[email protected]>
…to ActionListener (opensearch-project#19161) * Add CompletionStage variants to IndicesAdminClient as an alternative to ActionListener Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]>
…ches (opensearch-project#18981) Signed-off-by: Riley Jerger <[email protected]>
…arch-project#19171) The cancellation tests could deadlock when threads are delayed by OS scheduling. If cancellation triggers before all threads start, late threads may hit a code path where batchReduceSize causes the latch callback to be deferred to a MergeTask. Under certain timing conditions, these callbacks never execute, causing latch.await() to hang indefinitely. Ensure latch.countDown() is always called by wrapping consumeResult in try-catch. This guarantees test completion regardless of cancellation timing or exceptions. Signed-off-by: Atri Sharma <[email protected]>
…#19163) There was a check in the forbidden APIs plugin to cap the Java version at 14, presumably from ages ago when the plugin did not support Java 15 or newer. That means we have not been enforcing any Java deprecations in the past 5 years. This removes that check, updates the plugin to 3.9, and fixes all the resulting deprecation failures. Signed-off-by: Andrew Ross <[email protected]>
…pensearch-project#19053) * Do not set shardIndex of top docs in CollapsingTopDocsCollectorContext Signed-off-by: Binlong Gao <[email protected]> * Modify change log Signed-off-by: Binlong Gao <[email protected]> * Format code Signed-off-by: Binlong Gao <[email protected]> * Remove setShardIndex parameter in CollapseTopFieldDocs.merge() Signed-off-by: Binlong Gao <[email protected]> * Modify change log Signed-off-by: Binlong Gao <[email protected]> * tiny change Signed-off-by: Binlong Gao <[email protected]> --------- Signed-off-by: Binlong Gao <[email protected]>
…earch-project#19177) Partial writes can corrupt translog files during simulated failures. When recovering these files, TranslogCorruptedException is thrown but wasn't caught since it extends OpenSearchException, not TranslogException. Add TranslogCorruptedException to catch blocks alongside other expected exceptions during translog recovery attempts. Signed-off-by: Atri Sharma <[email protected]>
…arch-project#19164) Signed-off-by: Ankit Jain <[email protected]>
Signed-off-by: Pranit Kumar <[email protected]>
Signed-off-by: Pranit Kumar <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
[Describe what this change achieves]
Related Issues
Resolves #[Issue number to be closed when this PR is merged]
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.