MCO-1888: MCO: update bootimage enhancement for marketplace images #1876
Conversation
openshift-ci-robot
added
the
jira/valid-reference
|
@yuqi-zhang: This pull request references MCO-1888 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.21.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
djoshy
left a comment
djoshy
left a comment
There was a problem hiding this comment.
Overall makes sense to me, thanks for putting this together!
| - zone: us-east4-b | ||
| - zone: us-east4-c | ||
| platform: GCP | ||
| - zone: us-east4-ain that stream |
| **Detection fallback:** | ||
| If the boot image is not found in any historical list: | ||
| - Use AWS SDK to check publisher and RHCOS version: | ||
| - **Deregistered/Not Found**: Out of skew, immediately raise `Upgradeable=False` with remediation steps |
There was a problem hiding this comment.
This should be possible to add; currently the boot image controller can only cause a degrade via conditions on the MachineConfiguration object.
EDIT: Thinking a bit more, I suppose this could also happen through the proposed skew management mechanism....but it would not be immediate, the current estimated boot image should be old enough to have a skew issue, as the controller would skip over these machinesets.
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: djoshy The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Update the bootimage enhancement to account for AWS/GCP/Azure marketplace images, as we as ARO/ROSA offerings.
yuqi-zhang
force-pushed
the
bootimage-marketplace-updates
branch
from
3387ede to
c5dec1c
Compare
|
@yuqi-zhang: all tests passed! Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
patrickdillon
left a comment
patrickdillon
left a comment
There was a problem hiding this comment.
By and large this looks good. I left comments to correct some of the details. LMK if there are particular areas you are interested in that I may be able to help.
|
|
||
| **Azure Streams:** | ||
| - IPI/ARO - Standard installations and Azure Red Hat OpenShift (both HyperV Gen1 and Gen2) | ||
| - Marketplace - Azure Marketplace published images (paid offerings with OCP, OPP, OKE variants) |
There was a problem hiding this comment.
For paid marketplace images (I believe in all clouds--not just Azure), there is a variation between customers in the EMEA and non-EMEA regions.
There was a problem hiding this comment.
That's correct, we do have NA and EMEA offerings for AWS and Azure which contains their own plans/pricing. The images are the same for both regions, what may differ are the prices and other particular agreements for consuming the offer.
There was a problem hiding this comment.
Yes, this is covered in the later sections. I can add a note here as well to clarify, but hopefully the Azure section below has the correct details.
| - marketplace-ocp | ||
| - marketplace-oke | ||
| - marketplace-opp |
There was a problem hiding this comment.
All of the marketplace variations are included in a single marketplace json file, and those are included in the RHCOS stream in the cluster. See https://github.com/coreos/stream-metadata-go/blob/main/stream/rhcos/rhcos.go#L57-L78
There was a problem hiding this comment.
ack, so I suppose we'd add an AWS field under https://github.com/coreos/stream-metadata-go/blob/main/stream/rhcos/rhcos.go#L45?
There was a problem hiding this comment.
Just to clarify, since this is the AWS section, are you proposing that AWS will eventually have the same setup as the Azure marketplace images?
| **GCP UPI:** | ||
| - UPI installations typically upload their own images | ||
| - These fall into the non-managed case and require manual updates |
There was a problem hiding this comment.
I think for GCP UPI, users for the most part would still be using the RHCOS published images (this would not be the case in Azure). Checking on this, I see that our in-repo docs are actually confusing: the section is called "Create the cluster image" but if you examine the commands being run, the image is just being pulled from the rhcos stream.
Users could indeed create their own images, but I suspect for most cases UPI will be the same as IPI.
There was a problem hiding this comment.
Ah, in that case, should we update UPI clusters' images for customers as well? I guess there's no difference for us when we see the machineset object, since they would be referring to the same image? Or should we differentiate between the types of clusters and require manual action just in case.
| **Installer changes:** | ||
|
|
||
| Create stream metadata files for: | ||
| - rhcos (GCP IPI) |
There was a problem hiding this comment.
Is there a specific ask for GCP? The images should already be included inthe rhcos stream, for example:
% ./openshift-install coreos print-stream-json | jq '.architectures.x86_64.images.gcp'
{
"release": "9.6.20251023-0",
"project": "rhcos-cloud",
"name": "rhcos-9-6-20251023-0-gcp-x86-64"
}And the marketplace images are as well, as I linked above.
There was a problem hiding this comment.
The marketplace streams seem only to reference Azure for now, unless I misunderstood. I was under the impression that AWS and GCP marketplace variants have not yet been included in the streams or have the necessary tagging to properly differentiate them.
| version: '' | ||
| ``` | ||
| If `offer`, `publisher`, `sku`, and `version` are already set, this is already using the new unpaid marketplace image for IPI. We can check `publisher` for `azureopenshift`. |
There was a problem hiding this comment.
This first sentence was slightly misleading when I read it at first. If publisher == azureopenshift then it is an unpaid marketplace image, publisher and offer will still be set for both paid and unpaid images. (All marketplace images need the complete URN).
There was a problem hiding this comment.
Will clarify this point. Basically what I tried to say is that publisher == azureopenshift is unpaid and the below publisher == redhat or redhat-limited is paid
| - All variants are tagged with `variantType` metadata to enable deterministic stream detection | ||
| - The tagging will be done with AWS's [tagSet](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_Image.html) field, with `variantType as the Key and stream name as the value |
There was a problem hiding this comment.
Any suggestions for the images already present in AWS? Should we somehow tag them as well?
There was a problem hiding this comment.
Currently we don't plan on tagging existing images. Parsing of existing images that are referenced will be a mix between hard-coded lists we have today of images we know we've shipped, as well as the publisher metadata (see History Bootimage Tracking (Existing Clusters) section above).
If for some reason we don't know where an image came from, we hope to do that as a one off thing and ask the customer to take some manual action to update to the right image for their cluster.
5 participants
Update the bootimage enhancement to account for AWS/GCP/Azure marketplace images, as we as ARO/ROSA offerings.
cc @djoshy @patrickdillon