Skip to content

Conversation

Huginn-kio
Copy link
Contributor

HBASE-29572 Clean up output directory when copying the manifest fails during snapshot

@Huginn-kio
Copy link
Contributor Author

Huginn-kio commented Sep 4, 2025

If an export snapshot fails in copying .snapshotInfo and manifest when we use export snapshot tool to export snapshot to other cluster, which may leave behind an empty snapshot directory under .snapshot (snapshot.export.skip.tmp = true) or .snapshot/.tmp (snapshot.export.skip.tmp = false) without .snapshotInfo file and manifest.

SnapshotFileCache periodically refreshes the cache by traversing all snapshot directories under .snapshot and reading the .snapshotInfo file and manifest to update the cache. If it can not read .snapshotInfo from a snapshot directory, it will throw CorruptedSnapshotException. getSnapshotsInProgress may also result in such a situation when it attempts to retrieve the .snapshotInfo under each snapshot directory under .snapshot/.tmp.

So we need to clean up the output snapshot dir before throwing exception and aborting this job if copying manifest fails.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 30s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
_ master Compile Tests _
+1 💚 mvninstall 3m 24s master passed
+1 💚 compile 0m 38s master passed
+1 💚 checkstyle 0m 12s master passed
+1 💚 spotbugs 0m 32s master passed
+1 💚 spotless 0m 49s branch has no errors when running spotless:check.
_ Patch Compile Tests _
+1 💚 mvninstall 3m 8s the patch passed
+1 💚 compile 0m 36s the patch passed
+1 💚 javac 0m 36s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 11s the patch passed
+1 💚 spotbugs 0m 37s the patch passed
+1 💚 hadoopcheck 12m 14s Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 💚 spotless 1m 7s patch has no errors when running spotless:check.
_ Other Tests _
+1 💚 asflicense 0m 12s The patch does not generate ASF License warnings.
32m 21s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7275/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #7275
Optional Tests dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname Linux fb9ff19b6d06 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 5fef090
Default Java Eclipse Adoptium-17.0.11+9
Max. process+thread count 84 (vs. ulimit of 30000)
modules C: hbase-mapreduce U: hbase-mapreduce
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7275/1/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 31s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 24s master passed
+1 💚 compile 0m 21s master passed
+1 💚 javadoc 0m 15s master passed
+1 💚 shadedjars 6m 16s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 3m 12s the patch passed
+1 💚 compile 0m 20s the patch passed
+1 💚 javac 0m 20s the patch passed
+1 💚 javadoc 0m 14s the patch passed
+1 💚 shadedjars 6m 11s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚 unit 21m 36s hbase-mapreduce in the patch passed.
43m 39s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7275/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR #7275
Optional Tests javac javadoc unit compile shadedjars
uname Linux ce3160a6137f 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 5fef090
Default Java Eclipse Adoptium-17.0.11+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7275/1/testReport/
Max. process+thread count 2566 (vs. ulimit of 30000)
modules C: hbase-mapreduce U: hbase-mapreduce
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7275/1/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Huginn-kio Huginn-kio changed the base branch from master to branch-2.4 September 21, 2025 09:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants