ci: Ensure e2e setup errors fail tests, add retries during e2e setup #9082
+36
−17
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Setting up the correct configuration values like `enable-api-fields` is not optional since many tests depend on these values at runtime. If a test requires non-default config values which failed to be set, the failure may not cause an e2e test to fail for some time and the test failure message will cause confusion; the configuration being correct during setup is a given. By ensuring a failure in any of the configuration setup functions results in the test suite failing, we ensure the tests will only run against the expected configuration. This change ensures that if the e2e test configuration fails, the tests will not continue. Fixing that improves the clarity of some flaky failures, but it doesn't reduce the flakiness itself. To reduce failures during e2e environment configuration, this change also implements a retry mechanism to each of the set configuration functions. This reduces flakiness due to network connectivity or race conditions during environment setup. If all config retries are exhausted with failure, then the tests will fail as above.
tekton-robot
added
the
release-note-none
tekton-robot
added
the
size/M
|
/kind flake |
tekton-robot
added
the
kind/flake
vdemeester
approved these changes
Oct 17, 2025
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: vdemeester The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
tekton-robot
added
the
approved
|
/retest |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
Setting up the correct configuration values like
enable-api-fieldsis not optional since many tests depend on these values at runtime. If a test requires non-default config values which failed to be set, the failure may not cause an e2e test to fail for some time and the test failure message will cause confusion; the configuration being correct during setup is a given. By ensuring a failure in any of the configuration setup functions results in the test suite failing, we ensure the tests will only run against the expected configuration. This change ensures that if the e2e test configuration fails, the tests will not continue.Fixing that improves the clarity of some flaky failures, but it doesn't reduce the flakiness itself. To reduce failures during e2e environment configuration, this change also implements a retry mechanism to each of the set configuration functions. This reduces flakiness due to network connectivity or race conditions during environment setup. If all config retries are exhausted with failure, then the tests will fail as above.
Submitter Checklist
As the author of this PR, please check off the items in this checklist:
/kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tepRelease Notes