Skip to content

Cherry-pick necessary mirror PRs from trunk branch to Branch-3.4 branch #8105

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 9 commits into
base: branch-3.4
Choose a base branch
from
Open

Cherry-pick necessary mirror PRs from trunk branch to Branch-3.4 branch #8105

wants to merge 9 commits into from
+372 −390

Conversation

@slfan1989
Copy link
Contributor

@slfan1989 slfan1989 commented Nov 27, 2025 •
edited
Loading

Description of PR

The purpose of this PR is to maintain consistency in compilation between the trunk and branch-3.4. It mainly involves cherry-picking compilation-related PRs from the trunk branch to address the EOL (End of Life) issues of CentOS 7, CentOS 8, and Debian 10.

How was this patch tested?

CI.

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

stoty and others added 9 commits November 27, 2025 08:39
@stoty @slfan1989
HADOOP-19495. Add JDK 24 to Ubuntu 20.04 docker development images.
8ef2aef
@vinayakumarb @slfan1989
HADOOP-19540. [JDK17] Add ubuntu:noble as a build platform with JDK-1…
894d5c7 
…7 as default (apache#7608)

1. Added "ubuntu:noble" to supported docker platforms.
2. Modified `start-build-env.sh` to use expected Dockerfiles dynamically.
   ex: `bash start-build-env.sh ubuntu_24` where `ubuntu_24` is the extra suffix in the name of the Dockerfile.
3. Added `jdk17++` profile activated when jdk17 is available.

Successfully built the entire Hadoop tar with native support on env created using `bash start-build-env.sh ubuntu_24`
@GauthamBanasandra @slfan1989
HADOOP-19570. Upgrade libxxhash to 0.8.3 in Windows 10 (apache#7689)
cf79452 
* The current version of libxxhash - 0.8.1 isn't
  available on the msys repo.
* This is causing the Hadoop Jenkins CI for
  Windows to fail while building the docker image
  for Windows 10.
* This PR upgrades libxxhash to 0.8.3 to fix this
  issue.
@slfan1989 @pan3793 @stoty
HADOOP-19636. [JDK17] Remove CentOS 7 Support and Clean Up Dockerfile. (
3fa7884 
apache#7822)

* HADOOP-19636. [JDK17] Remove CentOS 7 Support and Clean Up Dockerfile.

Co-authored-by: Cheng Pan <[email protected]>
Co-authored-by: Istvan Toth <[email protected]>
Reviewed-by: Xiaoqiao He <[email protected]>
Reviewed-by: Steve Loughran <[email protected]>
Reviewed-by: Chris Nauroth <[email protected]>
Reviewed-by: Cheng Pan <[email protected]>
Reviewed-by: Istvan Toth <[email protected]>
Signed-off-by: Shilun Fan <[email protected]>
@pan3793 @slfan1989
HADOOP-19661. Migrate CentOS 8 to Rocky Linux 8 in build env Dockerfile
e0cc64c 
Closes apache#7900

Signed-off-by: Shilun Fan <[email protected]>
Signed-off-by: Chris Nauroth <[email protected]>
@pan3793 @slfan1989
HADOOP-19661. (Follow-up) Rename Dockerfile_centos_8 to Dockerfile_ro…
bd71ba2 
…ckylinux_8 (apache#7917)

HADOOP-19661. (Follow-up) Rename Dockerfile_centos_8 to Dockerfile_rockylinux_8.

Signed-off-by: Shilun Fan <[email protected]>
@pan3793 @slfan1989
HADOOP-19659. Upgrade Debian 10 to 11 in build env Dockerfile (apache…
c6fd02b 
…#7898) Contributed by Cheng Pan.

* HADOOP-19659. Upgrade Debian 10 to 11 in build env Dockerfile

Signed-off-by: Shilun Fan <[email protected]>
@pan3793 @slfan1989
HADOOP-19661. (Follow-up) Install missing deps for Rocky Linux 8 dock…
52c88de 
…er image (apache#7931) Contributed by Cheng Pan.

* HADOOP-19661. (Follow-up) Install missing deps for Rocky Linux 8 docker image

Reviewed-by: Shilun Fan <[email protected]>
Signed-off-by: Shilun Fan <[email protected]>
@pan3793 @slfan1989
HADOOP-19659. (Follow-up) Install missing procps for Debian 11 docker…
820bc02 
… image (apache#7932) Contributed by Cheng Pan.

* HADOOP-19659. (Follow-up) Install missing procps for Debian 11 docker image.

Signed-off-by: Shilun Fan <[email protected]>
@slfan1989 slfan1989 marked this pull request as draft November 27, 2025 00:50
@hadoop-yetus
Copy link

hadoop-yetus commented Nov 27, 2025

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8105/1/console in case of problems.

@hadoop-yetus
Copy link

hadoop-yetus commented Nov 27, 2025

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 14m 24s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+0 🆗 jsonlint 0m 0s jsonlint was not available.
+0 🆗 xmllint 0m 1s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ branch-3.4 Compile Tests _
+0 🆗 mvndep 2m 45s Maven dependency ordering for branch
+1 💚 mvninstall 35m 12s branch-3.4 passed
+1 💚 compile 16m 59s branch-3.4 passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 compile 15m 32s branch-3.4 passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 mvnsite 21m 22s branch-3.4 passed
+1 💚 javadoc 9m 14s branch-3.4 passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 7m 35s branch-3.4 passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 shadedclient 48m 14s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 34s Maven dependency ordering for patch
+1 💚 mvninstall 29m 5s the patch passed
+1 💚 compile 16m 26s the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javac 16m 26s the patch passed
+1 💚 compile 15m 14s the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 javac 15m 14s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 hadolint 0m 7s No new issues.
+1 💚 mvnsite 18m 2s the patch passed
-1 ❌ shellcheck 0m 2s /results-shellcheck.txt The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1)
+1 💚 javadoc 9m 2s the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 7m 36s the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 shadedclient 50m 11s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 740m 57s /patch-unit-root.txt root in the patch passed.
+1 💚 asflicense 1m 50s The patch does not generate ASF License warnings.
1031m 32s
Reason Tests
Failed junit tests hadoop.mapred.gridmix.TestGridmixSubmission
hadoop.mapred.gridmix.TestLoadJob
hadoop.fs.s3a.TestS3AUnbuffer
hadoop.fs.s3a.impl.streams.TestStreamFactories
Subsystem Report/Notes
Docker ClientAPI=1.52 ServerAPI=1.52 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8105/1/artifact/out/Dockerfile
GITHUB PR #8105
Optional Tests dupname asflicense codespell detsecrets hadolint shellcheck shelldocs mvnsite unit jsonlint compile javac javadoc mvninstall shadedclient xmllint
uname Linux aa7bf47c230c 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision branch-3.4 / 820bc02
Default Java Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8105/1/testReport/
Max. process+thread count 3710 (vs. ulimit of 5500)
modules C: hadoop-project . U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8105/1/console
versions git=2.25.1 maven=3.6.3 hadolint=1.11.1-0-g0e692dd shellcheck=0.7.0
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@slfan1989 slfan1989 marked this pull request as ready for review November 29, 2025 01:21
@slfan1989 slfan1989 mentioned this pull request Nov 29, 2025
Open
@slfan1989
Copy link
Contributor Author

slfan1989 commented Nov 29, 2025

@steveloughran Could you please help review this PR? Thank you very much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@steveloughran steveloughran Awaiting requested review from steveloughran

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@slfan1989 @hadoop-yetus @stoty @vinayakumarb @GauthamBanasandra @pan3793