-
Notifications
You must be signed in to change notification settings - Fork 21
Enable partition decorator syntax for Storage Write API #69
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable partition decorator syntax for Storage Write API #69
Conversation
kcbq-connector/src/main/java/com/wepay/kafka/connect/bigquery/RecordTableResolver.java
Show resolved
Hide resolved
kcbq-connector/src/main/java/com/wepay/kafka/connect/bigquery/RecordTableResolver.java
Outdated
Show resolved
Hide resolved
kcbq-connector/src/main/java/com/wepay/kafka/connect/bigquery/RecordTableResolver.java
Outdated
Show resolved
Hide resolved
2b987ed
to
f5809d8
Compare
kcbq-connector/src/main/java/com/wepay/kafka/connect/bigquery/RecordTableResolver.java
Show resolved
Hide resolved
kcbq-connector/src/test/java/com/wepay/kafka/connect/bigquery/RecordTableResolverTest.java
Show resolved
Hide resolved
.../test/java/com/wepay/kafka/connect/bigquery/integration/utils/TimePartitioningTestUtils.java
Show resolved
Hide resolved
kcbq-connector/src/main/java/com/wepay/kafka/connect/bigquery/RecordTableResolver.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add boiler plate headers to the new files as @agrawal-siddharth has pointed out.
Once the code looks good, I'll run the tests and if they pass merge it.
Please copy the header from one of the other source files and do not modify it. The verification test looks for exact text and format. I hope to switch to a different tool in the future but for now we do not allow any changes to the license block. Please fix the headers in the following files:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please change the date to be one Copyright date of 2025 in the 3 new files. It should only have the original date.
kcbq-connector/src/main/java/com/wepay/kafka/connect/bigquery/RecordTableResolver.java
Show resolved
Hide resolved
@Claudenw could you please re-run the tests? Not sure why the failure. It passes locally fine |
@podgaietska please rebase |
9c4d93c
to
4329d7a
Compare
4329d7a
to
906d562
Compare
Hi @Claudenw. Just curious what is happening with this PR? Is there anything else needed from me? |
...ctor/src/main/java/com/wepay/kafka/connect/bigquery/write/storage/StorageWriteApiWriter.java
Show resolved
Hide resolved
...nector/src/main/java/com/wepay/kafka/connect/bigquery/write/storage/StorageWriteApiBase.java
Show resolved
Hide resolved
kcbq-connector/src/main/java/com/wepay/kafka/connect/bigquery/BigQuerySinkTask.java
Show resolved
Hide resolved
f280c01
to
79e2066
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As of v2.9.0 the generated documentation tries to reflect dependencies and interactions between the options. This request is to update the documentation to reflect the changes.
...onnector/src/main/java/com/wepay/kafka/connect/bigquery/config/StorageWriteApiValidator.java
Show resolved
Hide resolved
…riteApi Batch Load
79e2066
to
71e572c
Compare
Overview
Storage Write API recently added support for partition decorator syntax for default streams. This PR enables that feature in the connector’s Storage Write API path.
What Changed