feat: tedge flows context

[didier-wenzek]

Proposed changes

• Deprecate flow script onConfigUpdate method
• Add a context argument to onMessage and onInterval to give the scripts access to the shared data
• context.mapper gives the scripts a read access to a store shared by all the flows
• context.mapper is populated by regular flows which output is configured to be the context.
• context.flow is shared by all the scripts of a flow
• context.script is private to a script instance
• context.config is the config provided by the flow to a script instance
• the keys of a context can be listed with the keys method.
• context can be updated using get, set, remove methods.

Types of changes

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
• Documentation Update (if none of the other choices apply)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue

#3850

Checklist

• I have read the CONTRIBUTING doc
• I have signed the CLA (in all commits with git commit -s. You can activate automatic signing by running just prepare-dev once)
• I ran just format as mentioned in CODING_GUIDELINES
• I used just check as mentioned in CODING_GUIDELINES
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)

Further comments

[codecov]

Codecov Report

❌ Patch coverage is 83.45588% with 45 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
crates/extensions/tedge_flows/src/runtime.rs	21.21%	26 Missing ⚠️
...ates/extensions/tedge_flows/src/js_lib/kv_store.rs	84.21%	6 Missing and 6 partials ⚠️
crates/extensions/tedge_flows/src/js_value.rs	33.33%	4 Missing ⚠️
crates/extensions/tedge_flows/src/actor.rs	0.00%	2 Missing ⚠️
crates/extensions/tedge_flows/src/config.rs	50.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

Robot Results

✅ Passed	❌ Failed	⏭️ Skipped	Total	Pass %	⏱️ Duration
744	0	3	744	100	2h22m40.638140999s

[albinsuresh]

That || {} would mask a possible race condition between this flow and the load_metadata.toml flow, right? Since both the flows could be loaded independently, this required context may not have been populated by the other flow, before this step is executed by this flow as it could theoretically receive the measurement message first, before the other flow receives and processes the metadata message for the same.

[didier-wenzek]

That || {} would mask a possible race condition between this flow and the load_metadata.toml flow, right?

As long as the flow mapper received the metadata first and the measurement second, then the mapper guarantees this line to be processed where the context metadata is available. But yes, a race can happen at the MQTT level - exactly as for any other mapper or agent logic.

[reubenmiller]

Though wouldn't the race conditional only occur for the first message? Any message delivered in the future will then be processed after the store has been populated.

[albinsuresh]

Though wouldn't the race conditional only occur for the first message? Any message delivered in the future will then be processed after the store has been populated.

As discussed offline, the race condition would happen not only when the flow is loaded for the very first time, but everytime the flow is reloaded as well, on mapper restarts. Even though the subscriptions made by these flows are persistent after the flow is loaded for the first time, the race between messages from two different topics can still happen whenever the MQTT connection is re-established. This is a fundamental issue with the MQTT protocol itself that we've had to workaround for the entity store, by caching telemetry messages for an entity before it is registered. The flow scripts will also have to take a similar approach by caching the data in the flow context.

[albinsuresh]

Though I'm fully aligned with tedge taking over the extract and store responsibilities of the meta messages, one additional "feature" that we've lost in the process is the flows awareness of any dynamic meta changes. For example, if there was a tick based flow doing some processing with an interval of 1h, if the meta definition changed after 30 mins, the flow should have been aware of this change. The flow can choose whether to react to that change or not for the current batch of messages. But, at least getting notified would have been useful.

This is probably not a big deal in most fleets where the telemetry metadata is pretty much static.

[didier-wenzek]

Correct

[reubenmiller]

Do we allow for multiple outputs? If so, then technically users could also publish the context to an MQTT topic as well, and the flow could subscribe to both the original flow messages, and the context updates (which are also published on some local MQTT topic)

[albinsuresh]

Happy to approve once the doc is updated.

A side note, unrelated to this PR: When console.log() outputs are logged in the mapper log, it appears directly as follows:

Nov 28 06:37:49 f6641df9de15 tedge-mapper[530]: JavaScript.Console: "test log"

It would have been better if the context(module_name) of that log is also included in that log statement.

[albinsuresh]

While trying to access a non-exported context namespace like context.abc.get() from the script, it was a silent failure with no trace of that error in the log. But, testing the same broken script with the tedge flows test utility was giving the proper error message:

2025-11-28T06:29:00.114082517Z ERROR tedge::cli::flows::test: Error in /etc/tedge/flows/measurements.toml: No messages can be processed due to an incorrect setting: JS raised exception: Error: cannot read property 'get' of undefined
    at onMessage (/etc/tedge/flows/measurements.toml|0|/etc/tedge/flows/add_timestamp.js:2:11)

Was it not feasible to get the same error message during live message processing?

[didier-wenzek]

no trace of that error in the log.

This is surprising. Both tedge-mapper flows and tedge flows test should log the same. I will check.

[didier-wenzek]

So there is indeed a difference between tedge-mapper flows and tedge flows test.

• tedge flows test ignores all the input, output and error configuration: all the messages are consumed from stdin and published to stdout or stderr.
• tedge-mapper flows uses the configured input, output and error. And the default is to send errors to te/errors.

One can improve flow error logging with an option to tee error messages to the mapper log.

[albinsuresh]

Approving with minor doc suggestions. The inconsistency with the error reporting and the logging of console.log() log statements, reported in the previous review, are independent issues. So, it can be handled separately as well.

[reubenmiller]

Approved