v3.10.0
What's Changed
💎 Window closing strategies
Previously, windows used the "key" closing strategy.
With this strategy, messages advance time and close only windows with the same message key.
It helps to capture more data when it's not aligned in time (e.g. some keys are produced irregularly), but the latest windows can remain unprocessed until the message with the same key is received.
In this release, we added a new "partition" strategy, and an API to configure the strategy for tumbling and hopping windows (sliding windows don't support it yet).
With "partition" closing strategy, messages advance time and close windows for the whole partition to which this key belongs.
It helps to close windows faster because different keys advance time at the cost of potentially skipping more out-of-order messages.
Example:
from datetime import timedelta
from quixstreams import Application
app = Application(...)
sdf = app.dataframe(...)
# Define a window with the "partition" closing strategy.
sdf = sdf.tumbling_window(timedelta(seconds=10)).sum().final(closing_strategy="partition")
Learn more about closing strategies in the docs - https://quix.io/docs/quix-streams/windowing.html#closing-strategies
Added by @quentin-quix in #747
💎 Connectors status callbacks
Sinks and Sources now accept optional on_client_connect_success and on_client_connect_failure callbacks and can trigger them to inform about the Connector status during setup.
🦠 Bugfixes
- fix bad timestamps in test_app by @tim-quix in #768
- Bound protobuf<6.0 in tests by @quentin-quix in #773
- Bugfix for recovering from exactly 1 changelog message by @tim-quix in #769
- print_table method handles non-dict values by @gwaramadze in #767
🛠️ Other changes
- Create topics eagerly the moment they are defined by @daniil-quix in #763
- Increase default timeout and retries for
Producer.produceby @quentin-quix in #771 - Add rich license by @gwaramadze in #776
- update docs and tutorials based on connect callback addition by @tim-quix in #775
- typing: make state protocols and ABCs generic by @quentin-quix in #777
Full Changelog: v3.9.0...v3.10.0