Skip to content

v3.10.0

Choose a tag to compare

released this 07 Mar 14:53
· 219 commits to main since this release
0115ebb

What's Changed

💎 Window closing strategies

Previously, windows used the "key" closing strategy.
With this strategy, messages advance time and close only windows with the same message key.
It helps to capture more data when it's not aligned in time (e.g. some keys are produced irregularly), but the latest windows can remain unprocessed until the message with the same key is received.

In this release, we added a new "partition" strategy, and an API to configure the strategy for tumbling and hopping windows (sliding windows don't support it yet).

With "partition" closing strategy, messages advance time and close windows for the whole partition to which this key belongs.
It helps to close windows faster because different keys advance time at the cost of potentially skipping more out-of-order messages.

Example:

from datetime import timedelta
from quixstreams import Application

app = Application(...)
sdf = app.dataframe(...)

# Define a window with the "partition" closing strategy.
sdf = sdf.tumbling_window(timedelta(seconds=10)).sum().final(closing_strategy="partition")

Learn more about closing strategies in the docs - https://quix.io/docs/quix-streams/windowing.html#closing-strategies

Added by @quentin-quix in #747

💎 Connectors status callbacks

Sinks and Sources now accept optional on_client_connect_success and on_client_connect_failure callbacks and can trigger them to inform about the Connector status during setup.

By @tim-quix in #708 #775

🦠 Bugfixes

  • fix bad timestamps in test_app by @tim-quix in #768
  • Bound protobuf<6.0 in tests by @quentin-quix in #773
  • Bugfix for recovering from exactly 1 changelog message by @tim-quix in #769
  • print_table method handles non-dict values by @gwaramadze in #767

🛠️ Other changes

  • Create topics eagerly the moment they are defined by @daniil-quix in #763
  • Increase default timeout and retries for Producer.produce by @quentin-quix in #771
  • Add rich license by @gwaramadze in #776
  • update docs and tutorials based on connect callback addition by @tim-quix in #775
  • typing: make state protocols and ABCs generic by @quentin-quix in #777

Full Changelog: v3.9.0...v3.10.0