Skip to content

Add false_allow_templates as a dynamic mapping option #18825

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

bruce-hong-glean
Copy link

@bruce-hong-glean bruce-hong-glean commented Jul 23, 2025

Description

This PR adds the dynamic: false_allow_templates option to the schema. This creates mappings for any field that matches properties or dynamic_templates, and ignores any fields that do not (as opposed to rejecting altogether).

Example from linked issue, the usage is:

  1. Create the index:
{
    "properties": {
        "url": { "type": "keyword" },
    },
    "dynamic": "false_allow_templates",
    "dynamic_templates": [
        {
            "dates": {
                "match": "date_*",
                "mapping": {
                    "type": "date",
                },
            },
        }
    ]
}
  1. url matches an explicit property and the fields date_timestamp and date_timezone math the dynamic template:
POST index1/_doc/1
{
  "url": "www.example.com",
  "date_timestamp": "...",
  "date_timezone": "...",
  "misc_field": "..."
}
  1. Produces the mapping:
{
    "properties": {
        "url": { "type": "keyword" },
        "date_timestamp": { "type": "date" },
        "date_timezone": { "type": "date" }
    }
}

Related Issues

Resolves #18617

Check List

  • Functionality includes testing.
  • API changes companion pull request created
  • Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions github-actions bot added enhancement Enhancement or improvement to existing feature or request Indexing Indexing, Bulk Indexing and anything related to indexing labels Jul 23, 2025
Copy link
Contributor

❌ Gradle check result for 4f37b48: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 0a6476a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@gaobinlong
Copy link
Contributor

The DCO check and precommit check is failed, please check the failure message to make them pass.

Update tests

Tests

Tests

Cleanup

Tested, cleanup

Cleanup

Signed-off-by: Bruce Hong <[email protected]>
@bruce-hong-glean bruce-hong-glean force-pushed the false-allow-dynamic-templates branch from 0a6476a to 82ee0eb Compare July 24, 2025 17:02
Copy link
Contributor

❌ Gradle check result for 249f59c: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 7a2535e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 13408ce: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@bruce-hong-glean
Copy link
Author

bruce-hong-glean commented Jul 24, 2025

Known flaky test failed, listed here

> Task :server:internalClusterTest

Tests with failures:
 - org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart_withNoRecoveryPayloadAndSource {p0={"cluster.indices.replication.strategy":"SEGMENT"}}

5436 tests completed, 1 failed, 56 skipped

> Task :server:internalClusterTest FAILED
build complete, generating: /var/jenkins/workspace/gradle-check/search/build/60998.tar.bz2

Signed-off-by: Bruce Hong <[email protected]>
@bruce-hong-glean bruce-hong-glean force-pushed the false-allow-dynamic-templates branch from 13408ce to 528e8ac Compare July 24, 2025 19:28
Copy link
Contributor

✅ Gradle check result for 528e8ac: SUCCESS

Copy link

codecov bot commented Jul 24, 2025

Codecov Report

❌ Patch coverage is 81.94444% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.83%. Comparing base (c1dfa6a) to head (217a993).
⚠️ Report is 9 commits behind head on main.

Files with missing lines Patch % Lines
...va/org/opensearch/index/mapper/DocumentParser.java 80.88% 9 Missing and 4 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #18825      +/-   ##
============================================
+ Coverage     72.75%   72.83%   +0.07%     
- Complexity    68520    68593      +73     
============================================
  Files          5570     5570              
  Lines        314998   315051      +53     
  Branches      45697    45706       +9     
============================================
+ Hits         229185   229465     +280     
+ Misses        67260    66953     -307     
- Partials      18553    18633      +80     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Bruce Hong <[email protected]>
Copy link
Contributor

❌ Gradle check result for d759498: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Bruce Hong <[email protected]>
@bruce-hong-glean bruce-hong-glean force-pushed the false-allow-dynamic-templates branch from 2f3bca2 to 99b9365 Compare July 24, 2025 23:18
Copy link
Contributor

✅ Gradle check result for 99b9365: SUCCESS

Signed-off-by: Bruce Hong <[email protected]>
Copy link
Contributor

❌ Gradle check result for 9a6762f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 22acae4: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@bruce-hong-glean bruce-hong-glean force-pushed the false-allow-dynamic-templates branch from 22acae4 to 12043cc Compare July 28, 2025 23:35
Copy link
Contributor

❌ Gradle check result for 12043cc: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Bruce Hong <[email protected]>
@bruce-hong-glean bruce-hong-glean force-pushed the false-allow-dynamic-templates branch from 12043cc to 217a993 Compare July 29, 2025 16:05
Copy link
Contributor

✅ Gradle check result for 217a993: SUCCESS

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request Indexing Indexing, Bulk Indexing and anything related to indexing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request] dynamic mapping setting called "false_allow_templates"
2 participants