feat: Add CLI/Environment variable support for PostgreSQL partition intervals

[kylorend3r]

The Definition and Basic Details

• Introduced configurable partition intervals via the --partition-interval command-line option. It prevents year-based and minute-based intervals for optimal performance
• Implemented a partition naming strategy. For example,
  • Daily partitions: metric_dbname_yyyymmdd
  • Weekly partitions: metric_dbname_y2024w01
  • Monthly partitions: metric_dbname_202401
  • Custom intervals: metric_dbname_20240101_1200_to_20240102_1200
• Added a new configuration to admin.config table to manage partition intervals for PostgreSQL type of schema in sink.

1 Week (168h) Example

3 Days (72h) Example

Possible Advantages

• Reduced lock contention during maintenance and more efficient partition dropping for data retention
• Ability to changing partition strategy via a config and a flexible approach to start with a custom interval depending on your requirements and environment.

Created this PR for #958

[coveralls]

Pull Request Test Coverage Report for Build 18586761341

Details

• 5 of 44 (11.36%) changed or added relevant lines in 2 files are covered.
• 1 unchanged line in 1 file lost coverage.
• Overall coverage decreased (-0.5%) to 70.855%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
internal/cmdopts/cmdoptions.go	5	11	45.45%
internal/sinks/postgres.go	0	33	0.0%

Files with Coverage Reduction	New Missed Lines	%
internal/sinks/postgres.go	1	41.77%

Totals
Change from base Build 18559469876:	-0.5%
Covered Lines:	3547
Relevant Lines:	5006

---

💛 - Coveralls

[pashagolub]

Thanks for proposal but I don't like over engineering.
We have time.Duration type in Go that perfectly describes intervals.

[kylorend3r]

My goal was to introduce weekly,daily or monthly partitioned tables because currently we only create weekly partitions. For some cases single partitions can be too big or small and people may want to store them daily or monthly partitions. Thereby I wanted to introduce this option to be able to start with different partition intervals at the beginning.

Let me know your opinions.

[pashagolub]

What I am trying to say is if we want to provide custom intervals then why do you want to have predefined options? Let user decide what exactly interval they want. This way you don't need to bother with predefined options, just use time arithmetic and we're done

[kylorend3r]

Hi @pashagolub ,

I had a chance to work on this MR according to your feedback and introduced the partition-interval for PostgreSQL schema type.

Basically, the user will be able to define its own custom interval and partition naming will be handled in admin sql functions. I hope, thanks to this enhancements, people will be able to define custom partition intervals and change them for the new partitions to be created.

Thanks for the feedback!

[pashagolub]

Thanks for your work! I was working on the same issue. I will check what we have in 2 weeks. Until then Postgres conferences are in a full run

[pashagolub]

Please check my approach to sql functions here f4cd404. I'd prefer to use mine version because:

• there is no need in admin.config changes
• there are less checks all over the code for allowed/disallowed intervals
• names are simple and predictable
• to_regclass usage instead of complex queries
• the last partition bound used, which automatically solves Partitions overlap when changing time zone of PostgreSQL #931
• timescale partition handling improved and simplified a lot