feat: Updating producer to configure RoutingPolicy - adding partition_key as the default for Round Robin if set #366
Conversation
|
@BewareMyPower when you get a chance, please take another look. Updated with some additional tests and slightly changed the behavior to work with a stable vec of producers. |
|
Fixed the round robin unit test |
| pub enum RoutingPolicy { | ||
| #[default] | ||
| RoundRobin, | ||
| Single(usize), |
There was a problem hiding this comment.
This strategy is different from other clients. The index should be generated once rather than provided by the application side. Because the application cannot know the actual number of partitions in advance, the index could exceed the actual number of partitions
| let mut producers = Vec::new(); | ||
| for i in 0..partition_count { | ||
| let producer = TopicProducer::new( | ||
| pulsar.clone(), | ||
| addr.clone(), | ||
| format!("{}-{}", topic.clone(), i), | ||
| Some(i.to_string()), | ||
| options.clone(), | ||
| ) | ||
| .await | ||
| .unwrap(); | ||
| producers.push(producer); | ||
| } |
There was a problem hiding this comment.
It's better to leverage the REST API to create a partitioned topic. See example here:
Line 816 in d1f18e6
https://pulsar.apache.org/docs/next/admin-api-topics/#create-1
Then you won't need to create the PartitionedProducer directly for tests, whose visibility is only within the crate.
2 participants
Should address: #322
https://pulsar.apache.org/docs/next/client-libraries-producers/#choose-partitions-when-using-a-key
Copying the behavior in the Java Client, where the producer takes in a configuration for a RoutingPolicy (RoundRobin, Single or Custom) for publishing to a partitioned topic.
Non breaking behavior since if you don't specify a routing policy, it will automtically be Round Robin. Similarly to the Java Client, if you explicitly have a partition_key set in your message, it will deterministically hash the key and route to a partition that way