Skip to content

Conversation

raunaqmorarka
Copy link
Member

@raunaqmorarka raunaqmorarka commented Sep 18, 2025

Description

alter table lineitem execute remove_orphan_files(retention_threshold => '0d');

            metric_name         | metric_value
    ----------------------------+--------------
     processed_manifests_count  |            2
     active_files_count         |           98
     scanned_files_count        |           97
     deleted_files_count        |            0

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

## Iceberg
* Produce metrics about execution in the output of remove_orphan_files procedure. ({issue}`26661`)

Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR enhances the Iceberg connector's remove_orphan_files procedure to output execution metrics. Instead of returning no information, the procedure now returns a table with metrics including manifest file processing count, valid files found, files scanned, and files deleted.

  • Modified the executeTableExecute method signature across the SPI to return Map<String, Long> instead of void
  • Enhanced the remove_orphan_files implementation to collect and return execution metrics
  • Updated query result handling to display metrics as a two-column table with metric names and values

Reviewed Changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
core/trino-spi/src/main/java/io/trino/spi/connector/ConnectorMetadata.java Changed executeTableExecute return type from void to Map<String, Long>
plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/IcebergMetadata.java Implemented metrics collection and return for remove_orphan_files procedure
core/trino-main/src/main/java/io/trino/operator/SimpleTableExecuteOperator.java Updated operator to build result pages from returned metrics
core/trino-main/src/main/java/io/trino/sql/planner/plan/SimpleTableExecuteNode.java Changed from single output symbol to list of symbols for metric name/value columns
docs/src/main/sphinx/connector/iceberg.md Added documentation for the new metrics output format

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

List<MaterializedRow> expectedRows = expectedResults.getMaterializedRows();

if (compareUpdate) {
if (compareUpdate && !actualResults.getUpdateType().equals(Optional.of("ALTER TABLE EXECUTE"))) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of skipping here, can we introduce something assertAlter or a dedicated method - which would set compareUpdate to false ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be no need for a special assertAlter method. We should be able to assert a more detailed output with assertUpdate itself.
This branch is written with the assumption that an update query can only produce a row count output and nothing else. Since ALTER commands did not output anything before, this branch was never executed before.
Now with the more detailed output, we need to skip over this branch to allow the assertion on detailed results.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add boolean compareExecute boolean flag and special assertExecute method?

@ebyhr ebyhr requested a review from martint September 18, 2025 13:19
@raunaqmorarka raunaqmorarka force-pushed the remove-cli branch 2 times, most recently from c75a156 to 0a72e6b Compare September 26, 2025 11:41
Copy link
Member

@sopel39 sopel39 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm % comments

List<MaterializedRow> expectedRows = expectedResults.getMaterializedRows();

if (compareUpdate) {
if (compareUpdate && !actualResults.getUpdateType().equals(Optional.of("ALTER TABLE EXECUTE"))) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add boolean compareExecute boolean flag and special assertExecute method?

alter table lineitem execute remove_orphan_files(retention_threshold => '0d');

        metric_name         | metric_value
----------------------------+--------------
 processed_manifests_count  |            2
 active_files_count         |           98
 scanned_files_count        |           97
 deleted_files_count        |            0
@raunaqmorarka raunaqmorarka merged commit b8217e5 into trinodb:master Oct 13, 2025
98 of 101 checks passed
@raunaqmorarka raunaqmorarka deleted the remove-cli branch October 13, 2025 18:04
@github-actions github-actions bot added this to the 478 milestone Oct 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

6 participants