Skip to content

Conversation

mbg
Copy link
Member

@mbg mbg commented Aug 14, 2025

This PR adds a new workflow that can be manually triggered to create a rollback release. I.e. the workflow creates a new release which mirrors an older, previous release. Since we publish immutable releases, this is necessary to "revert" a broken release.

The process is:

  • We checkout the repository at main. (For testing on a PR only: we checkout the repository at the PR head instead.)
  • We create a new branch where we update the changelog for the "new" version to include a message indicating that it is a rollback. This is implemented by a new rollback_changelog.py script. There is some duplication among the Python scripts that could be neater, but I didn't want to tackle big improvements there as part of this PR.
  • We then tag the resulting commit with the new release version and update the major version tag.
  • We push both tags. (For testing on a PR only: we skip this.)
  • We then generate a partial changelog for the new version using the existing prepare_changelog.py script.
  • We then create the release as a draft. (Skipped for PR testing.)
  • Like in the normal release process, we then create a mergeback PR for the changelog and package.json changes targeting main. (For testing on a PR only: we skip creating the PR, but still push the branch.)

Once the process has completed:

  • Since the release is a draft, we have an opportunity to review this to check that we are happy (which is still mutable).
  • If we are happy, we can then publish the draft release and then merge the mergeback PR.

This is best reviewed commit-by-commit.

Risk assessment

For internal use only. Please select the risk level of this change:

  • Low risk: Changes are fully under feature flags, or have been fully tested and validated in pre-production environments and are highly observable, or are documentation or test only.

Merge / deployment checklist

  • Confirm this change is backwards compatible with existing workflows.
  • Consider adding a changelog entry for this change.
  • Confirm the readme and docs have been updated if necessary.

@mbg mbg force-pushed the mbg/ci/rollback-release branch 2 times, most recently from f4f6c63 to 3376c74 Compare August 15, 2025 13:32
@mbg mbg force-pushed the mbg/ci/rollback-release branch from 7c91b1c to 67e42fe Compare August 27, 2025 11:23
@mbg mbg marked this pull request as ready for review August 27, 2025 11:44
@mbg mbg requested a review from a team as a code owner August 27, 2025 11:44
@mbg mbg requested review from Copilot and henrymercer August 27, 2025 11:44
runs-on: ubuntu-latest
timeout-minutes: 45

# Don't set the deployment environment for test runs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Don't set the deployment environment for test runs
# Don't set the deployment environment for test runs
# The Actions token does not have permissions to push changes to workflow files.
# Since workflow files may change as part of a backport PR, we use the "Automation" environment for real runs to authenticate as a GitHub App and push these changes.


uses: ./.github/workflows/prepare-release.yml

rollback:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll soon be needing to release CodeQL Action v4 to run on Node 24. I think the backport automation in update-release-branch.yml would create necessary backport releases, but I think we might want a way to more quickly rollback a change across all active release branches (without waiting for the full set of PR checks). Perhaps it makes sense to matrix this job across all active release branches?

git tag --annotate "${MAJOR_VERSION_TAG}" --message "${MAJOR_VERSION_TAG}" --force

- name: Push tags
if: github.event_name == 'workflow_dispatch'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if: github.event_name == 'workflow_dispatch'
# skip when testing
if: github.event_name == 'workflow_dispatch'

run: |
git push origin --atomic --force refs/tags/"${RELEASE_TAG}" refs/tags/"${MAJOR_VERSION_TAG}"

- name: Prepare partial Changelog
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it could be clearer what we mean here by "partial changelog". It's still not fully clear, but we could call it "release description" for instance?

VERSION: "${{ needs.prepare.outputs.version }}"
GH_TOKEN: ${{ steps.app-token.outputs.token }}
run: |
# Do not mark this release as latest. The most recent CLI release must be marked as latest.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Do not mark this release as latest. The most recent CLI release must be marked as latest.
# Do not mark this release as latest. The most recent bundle release must be marked as latest.

run: |
# Do not mark this release as latest. The most recent CLI release must be marked as latest.
# Set as a draft to give us an opportunity to review the rollback release.
gh release create \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this print a link to the new release? It might be nice to add a link to the job summary to make it easier to move forwards to the next step of reviewing and publishing the release.

on:
# You can trigger this workflow via workflow dispatch to start a rollback.
# This will create a draft release that mirrors the release for `rollback-tag`.
workflow_dispatch:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this is intended to be triggered on main?

uses: ./.github/actions/prepare-mergeback-branch
with:
base: "main"
head: ""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this blank?

# - rollback_version: the version that we are rolling back, typically the one that followed `target_version`
# - new_version: the new version that we are releasing `target_version` as, typically the one that follows `rollback_version`
#
# Example: python3 .github/workflows/script/rollback_changelog.py "1.2.3" "1.2.4" "1.2.5"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's an easy change to make (perhaps Copilot can do it) named arguments via something like argparse would make this easier to read

Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants