Skip to content

Conversation

@serramatutu
Copy link

@serramatutu serramatutu commented Oct 30, 2025

Which issue does this PR close?

(TODO: create Github Issue if the [VOTE] succeeds)
(TODO: does arrow-rs have a way to serialize this as RFC3339 instead of struct when going to/from JSON?)

No issue yet. This is still being discussed in the mailing list, and this PR is here for reference.

FORMAT Pull request: apache/arrow#48002

Rationale for this change

Please refer to the mailing list discussion (we'll send an e-mail with the proposal soon).

What changes are included in this PR?

This commit adds a new TimestampWithOffset extension type. This type represents a timestamp column that stores potentially different timezone offsets per value. The timestamp is stored in UTC alongside the original timezone offset in minutes.

Are these changes tested?

Yes.

Are there any user-facing changes?

Yes, this is a new proposed canonical extension type.

This commit adds a new `TimestampWithOffset` extension type.

This type represents a timestamp column that stores potentially
different timezone offsets per value. The timestamp is stored in
UTC alongside the original timezone offset in minutes.
Copy link
Member

@westonpace westonpace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems pretty straightforward and reasonable

@felipecrv felipecrv marked this pull request as ready for review November 5, 2025 03:52
@alamb alamb changed the title [DRAFT] Add TimestampWithOffset extension type Add TimestampWithOffset extension type Nov 5, 2025
@alamb
Copy link
Contributor

alamb commented Nov 5, 2025

There are several TODOs in this PRs description. Is that intended?

@serramatutu
Copy link
Author

serramatutu commented Nov 6, 2025

@alamb We're currently still discussing the FORMAT in the arrow repo and in the mailing list before talking too much about implementation. Not sure why @felipecrv marked this as ready for review?

For arrow-rs, I still need to

  • Add run-end and dict encoding to the offset column
  • Figure how to do JSON de/encoding using RFC3339
  • Test more

Realistically, even after the FORMAT PR goes into the spec I think this will take me a few more weeks to fully flesh out.

@alamb
Copy link
Contributor

alamb commented Nov 7, 2025

Thanks for the clarification @serramatutu -- I'll mark it as a draft again then

@alamb alamb marked this pull request as draft November 7, 2025 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arrow Changes to the arrow crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants