bib: only use org.osbild.rhsm and drop support for container only mtls

[mvo5]

[draft as a) this needs test b) its potentially breaking use-cases so we
need consensus first]

(TL;DR) This is the result of looking into how to make ibli/the service
support ISO building. The cleanest solution I can come up with
is to limit the scope to support ISO building only for containers
that either need no secrets or if they need secrets that get them
via the podman host /run/secrets mapping. This means user
with bootc containers that have hardcoded secrets in their container
to access their repos to build ISOs will break (if there are any).

---

bib: use org.osbild.rhsm for secrets instead of mtls

We currently run depsolve-dnf against the container rootdir.
For subscribed containers this means that we find a bunch
of repositories with ssl keys. For dnfjson this means it
switches the secrets provider to "org.osbuild.mtls" because
it (rightfully) assumes that while the secrets are in this
alternative rootdir they are not generally discoverable on
the system.

However this assumption is mostly untrue in our container world
because for RHEL containers podman will map the host secrets into
the container. So for RHEL containers we can just use the host
secrets and can drop the mTLS setup. This commit does that
and switches the secrets provider from mtls to rhsm (in a
rather hackish way right now).

This is great because the mTLS setup is very disconnected from
the manifest which means on a subscribed machine one cannot
create a rhel-bootc container manifest and then just run
it manually via osbuild. The workaround is to manually create
the mtls env (which is a bit clunky). The same is (probably)
true when moving image-mode into the service. Here manifest
generation and building is decoupled so we would hit the
same issue.

Note that this change is also bad because the existing code
was more general - a bootc container with its own set of
secrets inside the container (but not on the host) will no
longer work with this change applied.

Note that the disconnect between manifest/mtls setup is
also causing issues when moving this code into images.

So far the assumption in images/ibcli is that a manifest is
enough to be able to run a build with osbuild. However for
bootc manifests without something like this change this is
no longer true, we would have to carry the extra mTLS env
around and also add some lifecycle rules to delete the
temporary mTLS dir when its no longer needed.

---

bib: error for any repo secrets not mapped in from
the host

This commit addresses the issue that there could be repository
secrets inside the container that are not mapped from the host.

Such a setup is an error now. Its unclear if there are any
containers out there that uses the (accidental) support for
repository secrets directly from the container to auth against
mtls enabled repos to build ISOs for a bootc image.

If people need this there are two options:
a) put secrets under /etc/pki/entitlement and /etc/rhsm/ca so
that podman automatically maps it, then things should just
work as before
b) we could write a new secrets provider org.osbuild.container or
something that would take a container id and a path and then
during manifest build we would have to mount the container and
make the secrets available for curl/librepo this way.

For now lets go with (a). Ultimately ISO installers should come
as bootc containers and should not be constructed via RPMs. Once
that happens this issue goes away entirely.

[cgwalters]

Thanks for looking at this - quite a mega commit message! The whole situation is quite mind bending.

My intuition on this is that while it would be a CLI break, what if we just forced people to do a container build for ISOs? As you all know that's how I think it should work by default anyways.

Basically that would force things so that all package installations just work the same way podman build works, without having to think through all the osbuild-wrapping-dnf-in-containers.

The iso type here would simply be a as-simple-as-possible transform of a bootable container into ISO form, without performing mutations of the target operating system (including initramfs or kargs) unless specifically asked to do so.

And yes, this would mean we would need to document something like

FROM <image>
RUN dnf -y install anaconda-environment && dnf clean all

[achilleas-k]

what if we just forced people to do a container build for ISOs? As you all know that's how I think it should work by default anyways.

We all agree on this I think, but the question is how realistic is it to become reality soon? By soon I mean the next few weeks. If you think it's doable right now, I'd love to get started on it and start stripping out all the weird stuff we need to do for package resolving and installation.

Is installing anaconda-environment sufficient to prepare a container to run as an installer? We can manage the ISO building and putting all the bits from the container in the right places, I'm just not sure how much information and documentation we have ready for teaching users how to configure Anaconda for their needs.

[cgwalters]

I definitely get these sense that quite a bit of workflows are blocked by ISO problems and this PR seems like it will help.

Is installing anaconda-environment sufficient to prepare a container to run as an installer?

I'm sure it's not; we have further logic duplicated in both lorax and the osbuild pipelines right? The argument here is that that logic should be executable in a container build. I have opinions on the implementation of that but it doesn't really matter in the end what it runs as long as it boils down to something like /usr/libexec/anaconda-install that could do whatever. This is a lot like the design choices that drove bootc-base-imagectl.

[mvo5]

[..]

Basically that would force things so that all package installations just work the same way podman build works, without having to think through all the osbuild-wrapping-dnf-in-containers.

The iso type here would simply be a as-simple-as-possible transform of a bootable container into ISO form, without performing mutations of the target operating system (including initramfs or kargs) unless specifically asked to do so.
[..]

I did a spike for this in #1053 (hopefully) doing what you suggested here.