Skip to content

Enhance Spark's Register Table procedure to accept metadata folder as input #13889

@saitharun15

Description

@saitharun15

Feature Request / Improvement

Currently Spark's Register Table Procedure requires a full, direct path to a specific metadata file as input to register a table in a catalog.

This can be improved by allowing a metadata folder path as input, where the procedure automatically identifies and uses the latest metadata.json file within that directory to register the table, making the process more user-friendly.

This behavior is already implemented in Presto Iceberg (PR / Implementation]), and adding the same support in Spark would simplify table registration.

Query engine

Spark

Willingness to contribute

  • I can contribute this improvement/feature independently
  • I would be willing to contribute this improvement/feature with guidance from the Iceberg community
  • I cannot contribute this improvement/feature at this time

Metadata

Metadata

Assignees

No one assigned

    Labels

    improvementPR that improves existing functionality

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions