adding scripted metric aggs docs #10211

AntonEliatra · 2025-07-11T12:28:44Z

Description

adding scripted metric aggs docs

Version

all

Checklist

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Anton Rubin <[email protected]>

github-actions · 2025-07-11T12:28:54Z

Thank you for submitting your PR. The PR states are In progress (or Draft) -> Tech review -> Doc review -> Editorial review -> Merged.

Before you submit your PR for doc review, make sure the content is technically accurate. If you need help finding a tech reviewer, tag a maintainer.

When you're ready for doc review, tag the assignee of this PR. The doc reviewer may push edits to the PR directly or leave comments and editorial suggestions for you to address (let us know in a comment if you have a preference). The doc reviewer will arrange for an editorial review.

Signed-off-by: Anton Rubin <[email protected]>

kolchfa-aws · 2025-07-11T19:55:13Z

@sandeshkr419 Could you please review this PR? Thanks!

Signed-off-by: Anton Rubin <[email protected]>

sandeshkr419

These scripts within script stages are painless scripts, right?

Should we mention it somewhere within the doc - like these script stages should contain painless scripts (and then maybe link it).

Basically, what I am looking for are the rules/syntax to define these script but at the same time, explaining painless script might be out of scope for this page.

So if we can link it and mention it briefly, that might be good.

_aggregations/metric/scripted-metric.md

sandeshkr419 · 2025-07-29T21:37:38Z

_aggregations/metric/scripted-metric.md

+
+## Handling empty buckets (no documents scenario)
+
+When using a `scripted_metric` aggregation as a sub-aggregation within a bucket aggregation (such as terms), it is important to account for buckets that contain no documents on certain shards. In such cases, those shards return a `null` value for the aggregation state. During the `reduce_script` phase, the states array may therefore include `null` entries corresponding to these shards. To ensure reliable execution, the `reduce_script` must be designed to handle `null` values gracefully. A common approach is to include a conditional check, such as `if (state != null)`, before accessing or operating on each state. Failure to implement such checks can result in runtime errors when processing empty buckets across shards.


Not related to this documentation - but do you think that handling null can be another improvement in code base by passing some paramater like ignore_null_results.

I'm wondering that might even make scripted aggs faster if the null checks are part of code-flow via some param rather than run as part of script.

Signed-off-by: Anton Rubin <[email protected]>

AntonEliatra · 2025-07-31T11:53:05Z

@sandeshkr419 thank you for the review, The changes have been pushed

sandeshkr419

LGTM!
Thanks for addressing the comments.

kolchfa-aws

Thank you, @AntonEliatra! Please see my comments and let me know if you have any questions.

_aggregations/metric/scripted-metric.md

Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: AntonEliatra <[email protected]>

AntonEliatra · 2025-08-08T11:51:44Z

@kolchfa-aws that's updated now

natebower

@AntonEliatra Please see my comments and changes and tag me for approval when addressed. Thanks!

_aggregations/metric/scripted-metric.md

natebower · 2025-08-08T15:22:09Z

_aggregations/metric/scripted-metric.md

+
+| Parameter        | Data type | Required/Optional | Description                                                                                        |
+| ---------------- | --------- | ----------------- | -------------------------------------------------------------------------------------------------- |
+| `init_script`    | String    | Optional          | A script that executes once per shard before any documents are processed. Used to set up an initial state (for example, initialize counters or lists in a state object). If not provided, the state starts as an empty object on each shard.       |


Should any/all instances of "state" be in code font on lines 22, 23, and 24?

natebower · 2025-08-08T15:22:58Z

_aggregations/metric/scripted-metric.md

+
+## Allowed return types
+
+Scripts can use any valid operation and object internally. However, the data you store in `state` or return from any script must be of one of the allowed types. This restriction exists because the intermediate state needs to be sent between nodes. The following types are allowed:


Should "state" be in code font here?

natebower · 2025-08-08T15:37:10Z

_aggregations/metric/scripted-metric.md

@@ -55,18 +231,38 @@ GET opensearch_dashboards_sample_data_logs/_search
 ```
 {% include copy-curl.html %}

-#### Example response
+
+The response returns three values in the `value` object, demonstrating how a scripted metric can return multiple metrics at once by using a map in the state:


"scripted metric" => "scripted_metric aggregration"? Should "state" be in code font?

_aggregations/metric/scripted-metric.md

natebower · 2025-08-08T15:39:27Z

_aggregations/metric/scripted-metric.md

+
+## Handling empty buckets (no documents scenario)
+
+When using a `scripted_metric` aggregation as a subaggregation within a bucket aggregation (such as `terms`), it is important to account for buckets that contain no documents on certain shards. In such cases, those shards return a `null` value for the aggregation state. During the `reduce_script` phase, the `states` array may therefore include `null` entries corresponding to these shards. To ensure reliable execution, the `reduce_script` must be designed to handle `null` values gracefully. A common approach is to include a conditional check, such as `if (state != null)`, before accessing or operating on each state. Failure to implement such checks can result in runtime errors when processing empty buckets across shards.


End of 2nd sentence: Should "state" be in code font?

_aggregations/metric/scripted-metric.md

Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: AntonEliatra <[email protected]>

Signed-off-by: Anton Rubin <[email protected]>

_aggregations/metric/scripted-metric.md

Signed-off-by: Nathan Bower <[email protected]>

natebower

Thanks @AntonEliatra! LGTM

* adding scripted metric aggs docs Signed-off-by: Anton Rubin <[email protected]> * fixing vale errors Signed-off-by: Anton Rubin <[email protected]> * addressing the PR comments Signed-off-by: Anton Rubin <[email protected]> * addressing the PR comments Signed-off-by: Anton Rubin <[email protected]> * Apply suggestions from code review Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: AntonEliatra <[email protected]> * Apply suggestions from code review Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: AntonEliatra <[email protected]> * addressing the PR comments Signed-off-by: Anton Rubin <[email protected]> * addressing the PR comments Signed-off-by: Anton Rubin <[email protected]> * Apply suggestions from code review Signed-off-by: Nathan Bower <[email protected]> --------- Signed-off-by: Anton Rubin <[email protected]> Signed-off-by: AntonEliatra <[email protected]> Signed-off-by: Nathan Bower <[email protected]> Co-authored-by: kolchfa-aws <[email protected]> Co-authored-by: Nathan Bower <[email protected]> (cherry picked from commit 489b888) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* adding scripted metric aggs docs Signed-off-by: Anton Rubin <[email protected]> * fixing vale errors Signed-off-by: Anton Rubin <[email protected]> * addressing the PR comments Signed-off-by: Anton Rubin <[email protected]> * addressing the PR comments Signed-off-by: Anton Rubin <[email protected]> * Apply suggestions from code review Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: AntonEliatra <[email protected]> * Apply suggestions from code review Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: AntonEliatra <[email protected]> * addressing the PR comments Signed-off-by: Anton Rubin <[email protected]> * addressing the PR comments Signed-off-by: Anton Rubin <[email protected]> * Apply suggestions from code review Signed-off-by: Nathan Bower <[email protected]> --------- Signed-off-by: Anton Rubin <[email protected]> Signed-off-by: AntonEliatra <[email protected]> Signed-off-by: Nathan Bower <[email protected]> Co-authored-by: kolchfa-aws <[email protected]> Co-authored-by: Nathan Bower <[email protected]>

adding scripted metric aggs docs

84c6e36

Signed-off-by: Anton Rubin <[email protected]>

AntonEliatra requested review from kolchfa-aws, AMoo-Miki, natebower, dlvenable, epugh and sumobrian as code owners July 11, 2025 12:28

github-actions bot assigned kolchfa-aws Jul 11, 2025

fixing vale errors

45dc651

Signed-off-by: Anton Rubin <[email protected]>

kolchfa-aws added Tech review PR: Tech review in progress Content gap backport 3.1 labels Jul 11, 2025

sandeshkr419 self-assigned this Jul 15, 2025

addressing the PR comments

324eb20

Signed-off-by: Anton Rubin <[email protected]>

sandeshkr419 suggested changes Jul 29, 2025

View reviewed changes

addressing the PR comments

69e4d7a

Signed-off-by: Anton Rubin <[email protected]>

sandeshkr419 approved these changes Aug 2, 2025

View reviewed changes

kolchfa-aws added Doc review PR: Doc review in progress and removed Tech review PR: Tech review in progress labels Aug 7, 2025

kolchfa-aws reviewed Aug 7, 2025

View reviewed changes

Apply suggestions from code review

f325f20

Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: AntonEliatra <[email protected]>

kolchfa-aws added Editorial review PR: Editorial review in progress and removed Doc review PR: Doc review in progress labels Aug 8, 2025

kolchfa-aws assigned natebower Aug 8, 2025

natebower reviewed Aug 8, 2025

View reviewed changes

Apply suggestions from code review

b5209b0

Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: AntonEliatra <[email protected]>

AntonEliatra added 2 commits August 8, 2025 17:57

addressing the PR comments

678ed71

Signed-off-by: Anton Rubin <[email protected]>

addressing the PR comments

4178183

Signed-off-by: Anton Rubin <[email protected]>

natebower reviewed Aug 8, 2025

View reviewed changes

_aggregations/metric/scripted-metric.md Outdated Show resolved Hide resolved

natebower reviewed Aug 8, 2025

View reviewed changes

_aggregations/metric/scripted-metric.md Outdated Show resolved Hide resolved

Apply suggestions from code review

c81d6b4

Signed-off-by: Nathan Bower <[email protected]>

natebower approved these changes Aug 8, 2025

View reviewed changes

natebower removed the Editorial review PR: Editorial review in progress label Aug 8, 2025

natebower merged commit 489b888 into opensearch-project:main Aug 8, 2025
6 checks passed

opensearch-trigger-bot bot mentioned this pull request Aug 8, 2025

[Backport 3.1] adding scripted metric aggs docs #10688

Merged

github-actions bot pushed a commit that referenced this pull request Aug 8, 2025

adding scripted metric aggs docs (#10211) (#10688)

c8e3c79


		## Handling empty buckets (no documents scenario)

		When using a `scripted_metric` aggregation as a sub-aggregation within a bucket aggregation (such as terms), it is important to account for buckets that contain no documents on certain shards. In such cases, those shards return a `null` value for the aggregation state. During the `reduce_script` phase, the states array may therefore include `null` entries corresponding to these shards. To ensure reliable execution, the `reduce_script` must be designed to handle `null` values gracefully. A common approach is to include a conditional check, such as `if (state != null)`, before accessing or operating on each state. Failure to implement such checks can result in runtime errors when processing empty buckets across shards.


		## Allowed return types

		Scripts can use any valid operation and object internally. However, the data you store in `state` or return from any script must be of one of the allowed types. This restriction exists because the intermediate state needs to be sent between nodes. The following types are allowed:

adding scripted metric aggs docs #10211

adding scripted metric aggs docs #10211

Uh oh!

Conversation

AntonEliatra commented Jul 11, 2025

Description

Version

Checklist

Uh oh!

github-actions bot commented Jul 11, 2025

Uh oh!

kolchfa-aws commented Jul 11, 2025

Uh oh!

sandeshkr419 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sandeshkr419 Jul 29, 2025

Choose a reason for hiding this comment

Uh oh!

AntonEliatra commented Jul 31, 2025

Uh oh!

sandeshkr419 left a comment

Choose a reason for hiding this comment

Uh oh!

kolchfa-aws left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AntonEliatra commented Aug 8, 2025

Uh oh!

natebower left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

natebower Aug 8, 2025

Choose a reason for hiding this comment

Uh oh!

natebower Aug 8, 2025

Choose a reason for hiding this comment

Uh oh!

natebower Aug 8, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

natebower Aug 8, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

natebower left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!