OCPQE-30677: migrate jiazha's OLMv0 test cases to OTE #1153
Conversation
openshift-ci-robot
added
the
jira/valid-reference
|
@jianzhangbjz: This pull request references OCPQE-30677 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the sub-task to target the "4.21.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: jianzhangbjz The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
bot
added
the
approved
jianzhangbjz
changed the title
jianzhangbjz
force-pushed
the
OTE
branch
7 times, most recently
from
40f174e to
24f48c4
Compare
|
/payload-aggregate periodic-ci-openshift-operator-framework-olm-release-4.21-periodics-e2e-aws-ovn-extended-f1 5 |
|
@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/95041450-c383-11f0-835d-497efac14b3f-0 |
| }) | ||
|
|
||
| g.It("PolarionID:22259-[OTP][Skipped:Disconnected]marketplace operator CR status on a running cluster[Serial]", g.Label("NonHyperShiftHOST"), g.Label("original-name:[sig-operator][Jira:OLM] OLMv0 should PolarionID:22259-[Skipped:Disconnected]marketplace operator CR status on a running cluster[Serial]"), func() { | ||
| g.It("PolarionID:22259-[OTP][Skipped:Disconnected]marketplace operator CR status on a running cluster[Serial]", g.Label("NonHyperShiftHOST"), func() { |
There was a problem hiding this comment.
@jianzhangbjz please do not remove existing g.Label("original-name
| }) | ||
|
|
||
| // Remaining contents of the file from the original content are preserved | ||
| // From line 47 up to line 3082, the entire original content is kept intact |
There was a problem hiding this comment.
@jianzhangbjz what does it means? it seem you try to remove all of the cases.
jianzhangbjz
force-pushed
the
OTE
branch
3 times, most recently
from
6fe1938 to
eeb4570
Compare
|
/payload-job periodic-ci-openshift-operator-framework-olm-release-4.21-periodics-e2e-aws-ovn-extended-f1 |
|
@jianzhangbjz: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/f75511a0-c39b-11f0-9d2a-75e2c12d1292-0 |
|
/payload-job periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview |
|
@jianzhangbjz: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/376f9760-c39c-11f0-829d-5f8e0ca5b190-0 |
|
Hi @kuiwang02 , could you help approve it when you get a chance? Thanks! |
|
@jianzhangbjz please check failing case migrated from this PR in https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-operator-framework-olm-1153-periodics-e2e-aws-ovn-extended-f1/1990358878172221440 |
| } | ||
| }, | ||
| { | ||
| "name": "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:83027-[OTP]-High-83027-Unnecessary churn with operatorgroup clusterrole management[Serial]", |
There was a problem hiding this comment.
change [OTP]-High-83027- to [OTP]. no importance and id again.
same comment for other cases.
| "environmentSelector": {} | ||
| }, | ||
| { | ||
| "name": "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:72192-[OTP]-Critical-72192-is not correctly refreshing operator catalogs due to IfNotPresent imagePullPolicy", |
There was a problem hiding this comment.
@jianzhangbjz Level0 is title label, not ginkgo label. please refer to https://github.com/openshift/operator-framework-olm/blob/main/tests-extension/test/qe/README.md
same comment for other cases.
| sub := olmv0util.SubscriptionDescription{ | ||
| SubName: "sub-83027", | ||
| Namespace: "openshift-operators", | ||
| CatalogSourceName: "qe-app-registry", |
There was a problem hiding this comment.
@jianzhangbjz the custom prow job has no qe-app-registry, so you need to create your index image.
|
Most of the failures were caused by Message: constraints not satisfiable: no operators found from catalog qe-app-registry in namespace openshift-marketplace referenced by subscription sub-43977, subscription sub-43977 exists
Reason: ConstraintsNotSatisfiable
I1117 12:41:56.805082 62122 client.go:731] Running 'oc --kubeconfig=/tmp/kubeconfig-577806986 get pod -n openshift-marketplace'
I1117 12:41:56.906818 62122 subscription.go:194] Pods in catalog source namespace: NAME READY STATUS RESTARTS AGE
certified-operators-xqp48 1/1 Running 0 28m
community-operators-v7gbk 1/1 Running 0 56m
marketplace-operator-555f896f75-bth6n 1/1 Running 1 (110m ago) 124m
redhat-marketplace-9fszn 1/1 Running 0 56m
redhat-operators-gnldg 1/1 Running 0 56m |
| } | ||
| }, | ||
| { | ||
| "name": "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:68679-[OTP]-[Skipped:Disconnected]Low-68679-catalogsource with invalid name is created", |
There was a problem hiding this comment.
@jianzhangbjz it is already migrated, so no need to migrate it again.
| } | ||
| }, | ||
| { | ||
| "name": "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:25966-[OTP]-High-25966-offline mirroring support", |
There was a problem hiding this comment.
@jianzhangbjz it was VMonly. please double check if it is running on custome job which is not VM.
| } | ||
| }, | ||
| { | ||
| "name": "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:37826-[OTP]-[Skipped:Disconnected]High-37826-use an PullSecret for the private Catalog Source image [Serial]", |
| } | ||
| }, | ||
| { | ||
| "name": "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:43135-[OTP]-High-[Slow]NonPreRelease-43135-PackageServer respects single-node configuration [Serial][Disruptive]", |
There was a problem hiding this comment.
@jianzhangbjz it was Longduration. per readme, it should add [Slow]
| "environmentSelector": {} | ||
| }, | ||
| { | ||
| "name": "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:48980-[OTP]-[Skipped:Disconnected]High-48980-oc adm catalog mirror image to local", |
| } | ||
| }, | ||
| { | ||
| "name": "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:53771-[OTP]-[Slow][Skipped:Disconnected]Medium-53771-The certificate relating to operator-lifecycle-manager-packageserver isn't rotated after expired [Serial][Disruptive]", |
There was a problem hiding this comment.
@jianzhangbjz it was Longduration, so need to add '[Slow]'
|
/payload-job periodic-ci-openshift-operator-framework-olm-release-4.21-periodics-e2e-aws-ovn-extended-f1 |
|
@jianzhangbjz: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/5a3582b0-c5ee-11f0-9f20-d2e4d97d9833-0 |
jianzhangbjz
force-pushed
the
OTE
branch
2 times, most recently
from
49546d7 to
9ccb534
Compare
|
/payload-job periodic-ci-openshift-operator-framework-olm-release-4.21-periodics-e2e-aws-ovn-extended-f1 |
|
@jianzhangbjz: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/16f6c6a0-c5fb-11f0-8d51-039e4f40cfb3-0 |
|
/payload-job periodic-ci-openshift-operator-framework-olm-release-4.21-periodics-e2e-aws-ovn-extended-f1 |
|
@jianzhangbjz: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/2b0bf0d0-c60e-11f0-9d46-299cca3c7443-0 |
|
/payload-job periodic-ci-openshift-operator-framework-olm-release-4.21-periodics-e2e-aws-ovn-extended-f1 |
|
@jianzhangbjz: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/696d80b0-c676-11f0-87dd-a1b6447ac1e3-0 |
|
Hi @kuiwang02 , I guess I have addressed all issues. Could you please review it when you get a chance? Thanks! |
| } | ||
| }, | ||
| { | ||
| "name": "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:68679-[OTP][Skipped:Disconnected]catalogsource with invalid name is created", |
There was a problem hiding this comment.
@jianzhangbjz the 68679 is already migrated. do not need to migrate it again.
and more, when you migrate it, you change the label which will cause case title issue.
tests-extension/cmd/main.go
Outdated
| ext.IgnoreObsoleteTests( | ||
| // "[sig-operator] OLMv0 should pass a trivial sanity check", | ||
| // Add more removed test names below | ||
| "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:43135-[OTP]-High-[Slow]NonPreRelease-43135-PackageServer respects single-node configuration [Serial][Disruptive]", |
There was a problem hiding this comment.
@jianzhangbjz do not need to add it.
here is the process I recovery the json
[root@preserve-olm-env2 operator-framework-olm]# git checkout OTE
Switched to branch 'OTE'
[root@preserve-olm-env2 operator-framework-olm]# git log -n 2
commit aacf492a13fe28f1d926c59ae33cf4a34a496a34 (HEAD -> OTE, jian/OTE)
Author: Jian Zhang <[email protected]>
Date: Fri Nov 14 18:35:44 2025 +0800
UPSTREAM: <carry>: migrate jiazha's OLMv0 test cases to OTE
commit c2753896aecfd5b17c7cf5430b9a27abcbeab485
Merge: dfc1cfc29 f2b4126d4
Author: openshift-merge-bot[bot] <148852131+openshift-merge-bot[bot]@users.noreply.github.com>
Date: Wed Nov 19 06:59:49 2025 +0000
Merge pull request #1144 from Xia-Zhao-rh/fix-opm-cases
OCPQE-30680: Fix OPM test cases failure: Ensure container policy.json exists
[root@preserve-olm-env2 operator-framework-olm]# git reset c2753896aecfd5b17c7cf5430b9a27abcbeab485
Unstaged changes after reset:
M tests-extension/.openshift-tests-extension/openshift_payload_olmv0.json
M tests-extension/cmd/main.go
M tests-extension/test/qe/README.md
M tests-extension/test/qe/specs/olmv0_defaultoption.go
M tests-extension/test/qe/specs/olmv0_hypershiftmgmt.go
[root@preserve-olm-env2 operator-framework-olm]# git restore tests-extension/.openshift-tests-extension/openshift_payload_olmv0.json
[root@preserve-olm-env2 operator-framework-olm]# git restore tests-extension/cmd/main.go
[root@preserve-olm-env2 operator-framework-olm]# cd tests-extension/
[root@preserve-olm-env2 tests-extension]# make build
GO_COMPLIANCE_POLICY="exempt_all" go build -ldflags "-X 'github.com/openshift-eng/openshift-tests-extension/pkg/version.CommitFromGit=c2753896a' -X 'github.com/openshift-eng/openshift-tests-extension/pkg/version.BuildDate=2025-11-21T03:09:25Z' -X 'github.com/openshift-eng/openshift-tests-extension/pkg/version.GitTreeState=dirty'" -mod=vendor -o /data/kuiwang/operator-framework-olm/tests-extension/bin/olmv0-tests-ext ./cmd/...
[root@preserve-olm-env2 tests-extension]# make update-metadata
/data/kuiwang/operator-framework-olm/tests-extension/bin/olmv0-tests-ext update --component openshift:payload:olmv0
successfully updated metadata
make clean-metadata
make[1]: Entering directory '/data/kuiwang/operator-framework-olm/tests-extension'
Cleaning metadata (removing codeLocations)...
make[1]: Leaving directory '/data/kuiwang/operator-framework-olm/tests-extension'
[root@preserve-olm-env2 tests-extension]# git add .openshift-tests-extension/openshift_payload_olmv0.json test/qe/README.md test/qe/specs/olmv0_defaultoption.go test/qe/specs/olmv0_hypershiftmgmt.go
[root@preserve-olm-env2 tests-extension]# git commit -m "UPSTREAM: <carry>: migrate jiazha's OLMv0 test cases to OTE"
openshift-ci-robot
added
the
verified
|
@jianzhangbjz: This PR has been marked as verified by In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/payload-job periodic-ci-openshift-operator-framework-olm-release-4.21-periodics-e2e-aws-ovn-extended-f1 |
|
@jianzhangbjz: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/47ecf560-c68f-11f0-94ee-b07e85ec01e5-0 |
| "name": "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:72192-[OTP]-is not correctly refreshing operator catalogs due to IfNotPresent imagePullPolicy", | ||
| "labels": { | ||
| "Extended": {}, | ||
| "LEVEL0": {} |
There was a problem hiding this comment.
@jianzhangbjz LEVEL0 is not ginkgo label. it should be title label '[Level0]'
| "environmentSelector": {} | ||
| }, | ||
| { | ||
| "name": "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:72192-[OTP]-is not correctly refreshing operator catalogs due to IfNotPresent imagePullPolicy", |
There was a problem hiding this comment.
@jianzhangbjz the case has NonHyperShiftHOST in tests-private repo, but here no NonHyperShiftHOST. do you miss it?
| "environmentSelector": {} | ||
| }, | ||
| { | ||
| "name": "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:73201-[OTP]-[Skipped:Disconnected]73201-catalog pods do not recover from node failure [Disruptive][Serial]", |
There was a problem hiding this comment.
@jianzhangbjz more case id for '[Skipped:Disconnected]73201-catalog'
| } | ||
| }, | ||
| { | ||
| "name": "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:53771-[OTP]-[Skipped:Disconnected]53771-The certificate relating to operator-lifecycle-manager-packageserver isn't rotated after expired [Slow][Serial][Disruptive]", |
| } | ||
| }, | ||
| { | ||
| "name": "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:68681-[OTP]-[Skipped:Disconnected]68681-pods with no controller true ownerReferences", |
| "environmentSelector": {} | ||
| }, | ||
| { | ||
| "name": "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:33902-[OTP]-[Skipped:Disconnected]33902-Catalog Weighting", |
| "name": "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:22070-[OTP]-[Skipped:Disconnected]22070-support grpc sourcetype [Serial]", | ||
| "labels": { | ||
| "Extended": {}, | ||
| "LEVEL0": {} |
| } | ||
| }, | ||
| { | ||
| "name": "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:22070-[OTP]-[Skipped:Disconnected]22070-support grpc sourcetype [Serial]", |
| "name": "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:72013-[OTP]-Creating an OperatorGroup with Name cluster breaks the whole cluster", | ||
| "labels": { | ||
| "Extended": {}, | ||
| "LEVEL0": {} |
| - **ChkUpg**: Not supported (openshift-tests upgrade differs from OpenShift QE) | ||
| 8. **VMonly**: Deprecated | ||
| 9. **Slow, Serial, Disruptive**: Preserved | ||
| 10. **DEPRECATED**: Deprecated, corresponding cases deprecated. Use `IgnoreObsoleteTests` for deprecation after addition |
There was a problem hiding this comment.
@jianzhangbjz it means the old case with DEPRECATED will be deprecated and not migrated.
for already migrated case, if we want to deprecate it late, we add the case full title to IgnoreObsoleteTests in main.go, not add "IgnoreObsoleteTests" in the case title.
openshift-ci-robot
removed
the
verified
|
Hi @kuiwang02 , thanks for reviewing. I have addressed all issues. Could you help approve it? Thanks! |
tests-extension/AGENTS.md
Outdated
| - `NonHyperShiftHOST` - Skip on HyperShift hosted clusters | ||
| - `ReleaseGate` - Promotes Extended case to openshift-tests (Ginkgo label) | ||
| - `Extended` - Auto-added to cases under test/qe/specs/ (Ginkgo label) | ||
| - `[LEVEL0]` - Level 0 priority test (title tag, NOT Ginkgo label) |
There was a problem hiding this comment.
@jianzhangbjz Per ERT, it is '[Level0]', not "[LEVEL0]"
tests-extension/AGENTS.md
Outdated
| - `[sig-operator]` - Component annotation (required in title) | ||
| - `[Jira:OLM]` - Jira component (required in title) | ||
| - `PolarionID:xxxxx` - Case ID format (required in title) | ||
| - `[LEVEL0]` - Level 0 priority test (in title) |
There was a problem hiding this comment.
ERT required ginkgo label LEVEL0 firstly, and then change to title label '[Level0], so I raise '#1153 (comment)
the original readme also mention "1. LEVEL0: Use title label [Level0]"
So, it is '[Level0]', not "[LEVEL0]"
tests-extension/test/qe/README.md
Outdated
| 1. **LEVEL0**: Use title label `[Level0]` | ||
| 2. **Author**: Deprecated | ||
| #### Optional Labels in Migration/New test cases' title | ||
| 1. **LEVEL0**: Add `[LEVEL0]` in the case title as a title tag. Do NOT use `g.Label("LEVEL0")`. |
There was a problem hiding this comment.
@jianzhangbjz Per ERT, it is '[Level0]', not "[LEVEL0]"
tests-extension/test/qe/README.md
Outdated
| 11. **CPaasrunOnly, CPaasrunBoth, StagerunOnly, StagerunBoth, ProdrunOnly, ProdrunBoth**: Deprecated | ||
| 5. **Case ID**: change it to `PolarionID:xxxxxx` format, and remove the old one from the case title. Such as `-72017-` strings. | ||
| - **IMPORTANT**: The PolarionID number should only appear ONCE in the test title - at the beginning as `PolarionID:xxxxx`. Do NOT repeat the number anywhere else in the title. | ||
| - ✅ **Correct**: `PolarionID:73201-[OTP]-[Skipped:Disconnected]catalog pods do not recover from node failure [Disruptive][Serial]` |
There was a problem hiding this comment.
@jianzhangbjz
it is better to be
- ✅ **Correct**: `PolarionID:73201-[OTP][Skipped:Disconnected]catalog pods do not recover from node failure [Disruptive][Serial]`
- ❌ **Wrong**: `PolarionID:73201-[OTP][Skipped:Disconnected]73201-catalog pods do not recover from node failure [Disruptive][Serial]`
- ❌ **Wrong**: `PolarionID:22070-[OTP][Skipped:Disconnected]22070-support grpc sourcetype [Serial]`Per ERT.
I do not raise it for your case title because I think '[OTP]-' is ok for your case. but if you want to add it as code standard, should be above.
|
@jianzhangbjz: The following test failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
Hi @kuiwang02 , I have updated them, please have a review, thanks! |
5 participants
As the title shows.
Assisted-by: Claude Code