Skip to content

HIVE-29261: Clean up GitHub Actions for Docker Release #6127

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

HIVE-29261: Clean up GitHub Actions for Docker Release #6127

wants to merge 3 commits into from

Conversation

okumin
Copy link
Contributor

@okumin okumin commented Oct 14, 2025 •
edited
Loading

What changes were proposed in this pull request?

This PR would unify buildFromArchive and buildFromSource, which have a large common part.

https://issues.apache.org/jira/browse/HIVE-29261

Why are the changes needed?

I plan to add smoke tests(HIVE-29260) for Docker images and add Helm support. Before adding those additional steps, I'd like to clean up the current GitHub actions. Otherwise, we always have to replicate new testing or some steps.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

I tested the action with my personal fork and Docker Hub account.

Manual trigger(buildFromArchive)

Releaes Tag(buildFromSource)

Diff between buildFromArchive and buildFromSource

1,2c1,2
<   buildFromArchive:
<     if: github.event_name == 'workflow_dispatch'
---
>   buildFromSource:
>     if: github.event_name == 'create' && github.event.ref_type == 'tag' && startsWith(github.event.ref, 'rel/')
14c14
<         run:  echo "HIVE_VERSION=${{ github.event.inputs.hiveVersion }}"  >> $GITHUB_ENV
---
>         run: echo "HIVE_VERSION=$(mvn -f "pom.xml" -q help:evaluate -Dexpression=project.version -DforceStdout)" >> $GITHUB_ENV
20c20
<         run: echo "HADOOP_VERSION=${{ github.event.inputs.hadoopVersion }}" >> $GITHUB_ENV
---
>         run: echo "HADOOP_VERSION=$(mvn -f "pom.xml" -q help:evaluate -Dexpression=hadoop.version -DforceStdout)" >> $GITHUB_ENV
23c23
<         run: echo "TEZ_VERSION=${{ github.event.inputs.tezVersion }}" >> $GITHUB_ENV
---
>         run: echo "TEZ_VERSION=$(mvn -f "pom.xml" -q help:evaluate -Dexpression=tez.version -DforceStdout)" >> $GITHUB_ENV
24a25,36
>       - name: Build Hive project
>         run: mvn clean package -DskipTests -Pdist
> 
>       - name: Check for hive tar.gz
>         run: ls ./packaging/target/
> 
>       - name: Copy to docker directory
>         run: cp ./packaging/target/apache-hive-*-bin.tar.gz ./packaging/src/docker/
> 
>       - name: Confirm the copy of hive tar.gz
>         run: ls ./packaging/src/docker/
> 
47c59
<             BUILD_ENV=archive
---
>             BUILD_ENV=buildarchive
48a61,70
>       - name: Check for hive-standalone-metastore tar.gz
>         run: ls ./standalone-metastore/packaging/target/
> 
>       - name: Copy to docker directory
>         run: cp ./target/hive-standalone-metastore-*-bin.tar.gz ./src/docker/
>         working-directory: ./standalone-metastore/packaging
> 
>       - name: Confirm the copy of hive-standalone-metastore tar.gz
>         run: ls ./standalone-metastore/packaging/src/docker/
> 
61c83
<             BUILD_ENV=archive      
---
>             BUILD_ENV=buildarchive

okumin
okumin commented Oct 14, 2025
View reviewed changes
.github/workflows/docker-GA-images.yml
default: '0.10.2'

env:
DEVELOCITY_ACCESS_KEY: ${{ secrets.DEVELOCITY_ACCESS_KEY }}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably not used.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @simhadri-g

Copy link
Member

@deniskuzZ deniskuzZ Oct 17, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's also present in build.yml: HIVE-28303: Develocity integration (#5279)

Copy link
Member

@simhadri-g simhadri-g Oct 18, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is used by - https://develocity.apache.org/scans?search.rootProjectNames=Hive

ASF doc : https://infra.apache.org/gradle.html

But i am not sure if anyone is actually using the dashboard.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uh-huh. It is likely useful. Let me restore it back.
5f91fa9

.github/workflows/docker-GA-images.yml
- name: Prepare common environment variables
run: |
echo "namespace=${{ vars.DOCKER_NAMESPACE || 'apache' }}" >> $GITHUB_ENV
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I parameterized this part so that a contributor can test the action on their fork.

.github/workflows/docker-GA-images.yml
mvn clean package -DskipTests -Pdist
ls ./packaging/target/
mv ./packaging/target/apache-hive-*-bin.tar.gz ./packaging/src/docker/
rm -rf ./packaging/target
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just because I encountered a disk full. We might have a smarter way though this is enough.
https://github.com/okumin/hive/actions/runs/18424087084

@okumin okumin marked this pull request as ready for review October 14, 2025 07:25
deniskuzZ
deniskuzZ approved these changes Oct 17, 2025
View reviewed changes
Copy link
Member

@deniskuzZ deniskuzZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm +1
have you tried to trigger updated workflow?

simhadri-g
simhadri-g reviewed Oct 18, 2025
View reviewed changes
.github/workflows/docker-GA-images.yml
platforms: linux/amd64,linux/arm64
push: true
tags: apache/hive:${{ env.tag }}
tags: ${{ env.namespace }}/hive:${{ env.tag }}
Copy link
Member

@simhadri-g simhadri-g Oct 18, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check this,we i implemented this last time ${{ env.namespace }} ended up returning the wrong value like apachejenkins or something like that and the docker image was pushed to wrong repo. As a result, i had to hard code it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate on that issue? When each contributor tests it on their own repo, when the configuration is wrong, pushing an image would fail with an authorization error like this. In case that a release manager builds it in apache/hive, the apache repo is correctly used.

@okumin
Copy link
Contributor Author

okumin commented Oct 19, 2025

@deniskuzZ Yes, I have. I've attempted the following ones.

Manual trigger(buildFromArchive)

Releaes Tag(buildFromSource)

@sonarqubecloud SonarQubeCloud
Copy link

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@deniskuzZ deniskuzZ deniskuzZ approved these changes

@simhadri-g simhadri-g simhadri-g left review comments

Assignees

No one assigned

Labels

tests passed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@okumin @deniskuzZ @simhadri-g @asf-ci-hive