Partitioned stac-geoparquet writes

[gadomski]

We should provide a way for folks to write partitioned stac-geoparquet. Since there's an infinite number of ways folks might want to partition, we have to limit our scope by either:

• Only providing dials to partition a few, "best practice" ways, and/or
• Provide some generic mechanism where folks can provide their own partitioning schemes

[betolink]

This would be really useful to improve the search performance and help with living ongoing collections that need to keep appending (e.g. to the most recent partition only, or only to the correct spatial partition) .

I found some articles where this is explored a bit:
https://github.com/kylebarron/spatially-partitioned-geoparquet

https://dewey.dunnington.ca/post/2024/partitioning-strategies-for-bigger-than-memory-spatial-data/

In our use case, partitioning spatially first and then by time makes the most sense so maybe this rustac feature could implement just a few well known things like using a spatial partition (discrete grid?):

rustac.write(store,  partition_by=[ {"geometry": { "schema": "h3", "resolution": 2 }},  "datetime": "year"], items, format="geoparquet")

or we could pass a function to get the partition for any given field and use more elaborated schemas.

partition_by=["field", lambda: get_partition()]

If the partitioning is in the metadata, will rustac/duckdb do a more efficient search? e.g. if we use spatial partitions.