-
Notifications
You must be signed in to change notification settings - Fork 10
Use a minimal initrd to switch to the full initrd stored in /usr #110
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: flatcar-master
Are you sure you want to change the base?
Conversation
Pulls in flatcar/bootengine#110 Co-authored-by: James Le Cuirot <[email protected]> Signed-off-by: Kai Lueke <[email protected]>
0316d71
to
6794e4b
Compare
Pulls in flatcar/bootengine#110 Co-authored-by: James Le Cuirot <[email protected]> Signed-off-by: Kai Lueke <[email protected]>
0dc77a3
to
4dd3627
Compare
Pulls in flatcar/bootengine#110 Co-authored-by: James Le Cuirot <[email protected]> Signed-off-by: Kai Lueke <[email protected]>
e42505c
to
e8672f8
Compare
Pulls in flatcar/bootengine#110 Co-authored-by: James Le Cuirot <[email protected]> Signed-off-by: Kai Lueke <[email protected]>
Pulls in flatcar/bootengine#110 Co-authored-by: James Le Cuirot <[email protected]> Signed-off-by: Kai Lueke <[email protected]>
3172cf1
to
91da72b
Compare
Pulls in flatcar/bootengine#110 Co-authored-by: James Le Cuirot <[email protected]> Signed-off-by: Kai Lueke <[email protected]>
Pulls in flatcar/bootengine#110 Co-authored-by: James Le Cuirot <[email protected]> Signed-off-by: Kai Lueke <[email protected]>
91da72b
to
ce52950
Compare
Pulls in flatcar/bootengine#110 Co-authored-by: James Le Cuirot <[email protected]> Signed-off-by: Kai Lueke <[email protected]>
Pulls in flatcar/bootengine#110 Co-authored-by: James Le Cuirot <[email protected]> Signed-off-by: Kai Lueke <[email protected]>
ce52950
to
b187cfc
Compare
The growth of binaries over time and the inclusion of new features filled the available boot partition space, so that the kernel+initrd almost couldn't fit twice anymore as required for updates. We employed workarounds such as wrapper scripts for ignition, afterburn and other binaries so that they are loaded from /usr. However, this was still not enough and we would have to do the same for (network) kernel modules and firmware. To avoid making this ever more complex we can use a dedicated initrd focused on loading the full initrd from /usr and then this full initrd can use dracut as before and even drop all the workarounds we accumulated. Generate a minimal initrd to use instead of the full bootengine initrd. The bootengine initrd gets stored as squashfs on /usr. The minimal initrd still includes the early_cpio for amd64 microcode updates. We have a fixed list of modules or module directories to include, only focused on loading /usr and any emergency console interaction. This requires also checking for module dependencies to copy over. The busybox, veritysetup, and kmod binaries are needed and get their required libraries resolved and copied over. They are not static and use shared libraries which should be ok for now. The resulting vmlinuz file is 27 MB for amd64, down from ~60 MB, so we have enough room to include more kernel modules and so on for the next years while we also grow the boot partition and wait for users to redeploy until we can rely on a larger boot partition and eventually drop the minimal initrd again. Pulls in flatcar/bootengine#110 for the minimal initrd script and flatcar/seismograph#12 for making the device mapper discovery for the "rootdev" command more reliable. This also requied a backport of a kernel patch from 2017 that exposes the PARTUUID in the /sys uevent file. Co-authored-by: James Le Cuirot <[email protected]> Signed-off-by: Kai Lueke <[email protected]>
b187cfc
to
39654dd
Compare
The growth of binaries over time and the inclusion of new features filled the available boot partition space, so that the kernel+initrd almost couldn't fit twice anymore as required for updates. We employed workarounds such as wrapper scripts for ignition, afterburn and other binaries so that they are loaded from /usr. However, this was still not enough and we would have to do the same for (network) kernel modules and firmware. To avoid making this ever more complex we can use a dedicated initrd focused on loading the full initrd from /usr and then this full initrd can use dracut as before and even drop all the workarounds we accumulated. Generate a minimal initrd to use instead of the full bootengine initrd. The bootengine initrd gets stored as squashfs on /usr. The minimal initrd still includes the early_cpio for amd64 microcode updates. We have a fixed list of modules or module directories to include, only focused on loading /usr and any emergency console interaction. This requires also checking for module dependencies to copy over. The busybox, veritysetup, and kmod binaries are needed and get their required libraries resolved and copied over. They are not static and use shared libraries which should be ok for now. The resulting vmlinuz file is 27 MB for amd64, down from ~60 MB, so we have enough room to include more kernel modules and so on for the next years while we also grow the boot partition and wait for users to redeploy until we can rely on a larger boot partition and eventually drop the minimal initrd again. Pulls in flatcar/bootengine#110 for the minimal initrd script and flatcar/seismograph#12 for making the device mapper discovery for the "rootdev" command more reliable. This also requied a backport of a kernel patch from 2017 that exposes the PARTUUID in the /sys uevent file. Co-authored-by: James Le Cuirot <[email protected]> Signed-off-by: Kai Lueke <[email protected]>
39654dd
to
0d8c0af
Compare
The growth of binaries over time and the inclusion of new features filled the available boot partition space, so that the kernel+initrd almost couldn't fit twice anymore as required for updates. We employed workarounds such as wrapper scripts for ignition, afterburn and other binaries so that they are loaded from /usr. However, this was still not enough and we would have to do the same for (network) kernel modules and firmware. To avoid making this ever more complex we can use a dedicated initrd focused on loading the full initrd from /usr and then this full initrd can use dracut as before and even drop all the workarounds we accumulated. Generate a minimal initrd to use instead of the full bootengine initrd. The bootengine initrd gets stored as squashfs on /usr. The minimal initrd still includes the early_cpio for amd64 microcode updates. We have a fixed list of modules or module directories to include, only focused on loading /usr and any emergency console interaction. This requires also checking for module dependencies to copy over. The busybox, veritysetup, and kmod binaries are needed and get their required libraries resolved and copied over. They are not static and use shared libraries which should be ok for now. The resulting vmlinuz file is 27 MB for amd64, down from ~60 MB, so we have enough room to include more kernel modules and so on for the next years while we also grow the boot partition and wait for users to redeploy until we can rely on a larger boot partition and eventually drop the minimal initrd again. Pulls in flatcar/bootengine#110 for the minimal initrd script and flatcar/seismograph#12 for making the device mapper discovery for the "rootdev" command more reliable. This also requied a backport of a kernel patch from 2017 that exposes the PARTUUID in the /sys uevent file. Co-authored-by: James Le Cuirot <[email protected]> Signed-off-by: Kai Lueke <[email protected]>
The growth of binaries over time and the inclusion of new features filled the available boot partition space, so that the kernel+initrd almost couldn't fit twice anymore as required for updates. We employed workarounds such as wrapper scripts for ignition, afterburn and other binaries so that they are loaded from /usr. However, this was still not enough and we would have to do the same for (network) kernel modules and firmware. To avoid making this ever more complex we can use a dedicated initrd focused on loading the full initrd from /usr and then this full initrd can use dracut as before and even drop all the workarounds we accumulated. Generate a minimal initrd to use instead of the full bootengine initrd. The bootengine initrd gets stored as squashfs on /usr. The minimal initrd still includes the early_cpio for amd64 microcode updates. We have a fixed list of modules or module directories to include, only focused on loading /usr and any emergency console interaction. This requires also checking for module dependencies to copy over. The busybox, veritysetup, and kmod binaries are needed and get their required libraries resolved and copied over. They are not static and use shared libraries which should be ok for now. The resulting vmlinuz file is 27 MB for amd64, down from ~60 MB, so we have enough room to include more kernel modules and so on for the next years while we also grow the boot partition and wait for users to redeploy until we can rely on a larger boot partition and eventually drop the minimal initrd again. Pulls in flatcar/bootengine#110 for the minimal initrd script and flatcar/seismograph#12 for making the device mapper discovery for the "rootdev" command more reliable. This also requied a backport of a kernel patch from 2017 that exposes the PARTUUID in the /sys uevent file. Co-authored-by: James Le Cuirot <[email protected]> Signed-off-by: Kai Lueke <[email protected]>
The growth of binaries over time and the inclusion of new features filled the available boot partition space, so that the kernel+initrd almost couldn't fit twice anymore as required for updates. We employed workarounds such as wrapper scripts for ignition, afterburn and other binaries so that they are loaded from /usr. However, this was still not enough and we would have to do the same for (network) kernel modules and firmware. To avoid making this ever more complex we can use a dedicated initrd focused on loading the full initrd from /usr and then this full initrd can use dracut as before and even drop all the workarounds we accumulated. Generate a minimal initrd to use instead of the full bootengine initrd. The bootengine initrd gets stored as squashfs on /usr. The minimal initrd still includes the early_cpio for amd64 microcode updates. We have a fixed list of modules or module directories to include, only focused on loading /usr and any emergency console interaction. This requires also checking for module dependencies to copy over. The busybox, veritysetup, and kmod binaries are needed and get their required libraries resolved and copied over. They are not static and use shared libraries which should be ok for now. The resulting vmlinuz file is 27 MB for amd64, down from ~60 MB, so we have enough room to include more kernel modules and so on for the next years while we also grow the boot partition and wait for users to redeploy until we can rely on a larger boot partition and eventually drop the minimal initrd again. Pulls in flatcar/bootengine#110 for the minimal initrd script and flatcar/seismograph#12 for making the device mapper discovery for the "rootdev" command more reliable. This also requied a backport of a kernel patch from 2017 that exposes the PARTUUID in the /sys uevent file. Co-authored-by: James Le Cuirot <[email protected]> Signed-off-by: Kai Lueke <[email protected]>
The growth of binaries over time and the inclusion of new features filled the available boot partition space, so that the kernel+initrd almost couldn't fit twice anymore as required for updates. We employed workarounds such as wrapper scripts for ignition, afterburn and other binaries so that they are loaded from /usr. However, this was still not enough and we would have to do the same for (network) kernel modules and firmware. To avoid making this ever more complex we can use a dedicated initrd focused on loading the full initrd from /usr and then this full initrd can use dracut as before and even drop all the workarounds we accumulated. Generate a minimal initrd to use instead of the full bootengine initrd. The bootengine initrd gets stored as squashfs on /usr. The minimal initrd still includes the early_cpio for amd64 microcode updates. We have a fixed list of modules or module directories to include, only focused on loading /usr and any emergency console interaction. This requires also checking for module dependencies to copy over. The busybox, veritysetup, and kmod binaries are needed and get their required libraries resolved and copied over. They are not static and use shared libraries which should be ok for now. The resulting vmlinuz file is 27 MB for amd64, down from ~60 MB, so we have enough room to include more kernel modules and so on for the next years while we also grow the boot partition and wait for users to redeploy until we can rely on a larger boot partition and eventually drop the minimal initrd again. Pulls in flatcar/bootengine#110 for the minimal initrd script and flatcar/seismograph#12 for making the device mapper discovery for the "rootdev" command more reliable. This also requied a backport of a kernel patch from 2017 that exposes the PARTUUID in the /sys uevent file. Co-authored-by: James Le Cuirot <[email protected]> Signed-off-by: Kai Lueke <[email protected]>
The growth of binaries over time and the inclusion of new features filled the available boot partition space, so that the kernel+initrd almost couldn't fit twice anymore as required for updates. We employed workarounds such as wrapper scripts for ignition, afterburn and other binaries so that they are loaded from /usr. However, this was still not enough and we would have to do the same for (network) kernel modules and firmware. To avoid making this ever more complex we can use a dedicated initrd focused on loading the full initrd from /usr and then this full initrd can use dracut as before and even drop all the workarounds we accumulated. Generate a minimal initrd to use instead of the full bootengine initrd. The bootengine initrd gets stored as squashfs on /usr. The minimal initrd still includes the early_cpio for amd64 microcode updates. We have a fixed list of modules or module directories to include, only focused on loading /usr and any emergency console interaction. This requires also checking for module dependencies to copy over. The busybox, veritysetup, and kmod binaries are needed and get their required libraries resolved and copied over. They are not static and use shared libraries which should be ok for now. The resulting vmlinuz file is 27 MB for amd64, down from ~60 MB, so we have enough room to include more kernel modules and so on for the next years while we also grow the boot partition and wait for users to redeploy until we can rely on a larger boot partition and eventually drop the minimal initrd again. Pulls in flatcar/bootengine#110 for the minimal initrd script and flatcar/seismograph#12 for making the device mapper discovery for the "rootdev" command more reliable. This also requied a backport of a kernel patch from 2017 that exposes the PARTUUID in the /sys uevent file. Co-authored-by: James Le Cuirot <[email protected]> Signed-off-by: Kai Lueke <[email protected]>
02ee33f
to
8b330a4
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, but I think Chewi will want to have a look too.
The growth of binaries over time and the inclusion of new features filled the available boot partition space, so that the kernel+initrd almost couldn't fit twice anymore as required for updates. We employed workarounds such as wrapper scripts for ignition, afterburn and other binaries so that they are loaded from /usr. However, this was still not enough and we would have to do the same for (network) kernel modules and firmware. To avoid making this ever more complex we can use a dedicated initrd focused on loading the full initrd from /usr and then this full initrd can use dracut as before and even drop all the workarounds we accumulated. Generate a minimal initrd to use instead of the full bootengine initrd. The bootengine initrd gets stored as squashfs on /usr. The minimal initrd still includes the early_cpio for amd64 microcode updates. We have a fixed list of modules or module directories to include, only focused on loading /usr and any emergency console interaction. This requires also checking for module dependencies to copy over. The busybox, veritysetup, and kmod binaries are needed and get their required libraries resolved and copied over. They are not static and use shared libraries which should be ok for now. The resulting vmlinuz file is 27 MB for amd64, down from ~60 MB, so we have enough room to include more kernel modules and so on for the next years while we also grow the boot partition and wait for users to redeploy until we can rely on a larger boot partition and eventually drop the minimal initrd again. Pulls in flatcar/bootengine#110 for the minimal initrd script and flatcar/seismograph#12 for making the device mapper discovery for the "rootdev" command more reliable. This also requied a backport of a kernel patch from 2017 that exposes the PARTUUID in the /sys uevent file. Co-authored-by: James Le Cuirot <[email protected]> Signed-off-by: Kai Lueke <[email protected]>
f90e799
to
fbd693b
Compare
The growth of binaries over time and the inclusion of new features filled the available boot partition space, so that the kernel+initrd almost couldn't fit twice anymore as required for updates. We employed workarounds such as wrapper scripts for ignition, afterburn and other binaries so that they are loaded from /usr. However, this was still not enough and we would have to do the same for (network) kernel modules and firmware. To avoid making this ever more complex we can use a dedicated initrd focused on loading the full initrd from /usr and then this full initrd can use dracut as before and even drop all the workarounds we accumulated. Generate a minimal initrd to use instead of the full bootengine initrd. The bootengine initrd gets stored as squashfs on /usr. The minimal initrd still includes the early_cpio for amd64 microcode updates. We have a fixed list of modules or module directories to include, only focused on loading /usr and any emergency console interaction. This requires also checking for module dependencies to copy over. The busybox, veritysetup, and kmod binaries are needed and get their required libraries resolved and copied over. They are not static and use shared libraries which should be ok for now. The resulting vmlinuz file is 27 MB for amd64, down from ~60 MB, so we have enough room to include more kernel modules and so on for the next years while we also grow the boot partition and wait for users to redeploy until we can rely on a larger boot partition and eventually drop the minimal initrd again. Pulls in flatcar/bootengine#110 for the minimal initrd script and flatcar/seismograph#12 for making the device mapper discovery for the "rootdev" command more reliable. This also requied a backport of a kernel patch from 2017 that exposes the PARTUUID in the /sys uevent file. Co-authored-by: James Le Cuirot <[email protected]> Signed-off-by: Kai Lueke <[email protected]>
The growth of binaries over time and the inclusion of new features filled the available boot partition space, so that the kernel+initrd almost couldn't fit twice anymore as required for updates. We employed workarounds such as wrapper scripts for ignition, afterburn and other binaries so that they are loaded from /usr. However, this was still not enough and we would have to do the same for (network) kernel modules and firmware. To avoid making this ever more complex we can use a dedicated initrd focused on loading the full initrd from /usr and then this full initrd can use dracut as before and even drop all the workarounds we accumulated. Generate a minimal initrd to use instead of the full bootengine initrd. The bootengine initrd gets stored as squashfs on /usr. The minimal initrd still includes the early_cpio for amd64 microcode updates. We have a fixed list of modules or module directories to include, only focused on loading /usr and any emergency console interaction. This requires also checking for module dependencies to copy over. The busybox, veritysetup, and kmod binaries are needed and get their required libraries resolved and copied over. They are not static and use shared libraries which should be ok for now. The resulting vmlinuz file is 27 MB for amd64, down from ~60 MB, so we have enough room to include more kernel modules and so on for the next years while we also grow the boot partition and wait for users to redeploy until we can rely on a larger boot partition and eventually drop the minimal initrd again. Pulls in flatcar/bootengine#110 for the minimal initrd script and flatcar/seismograph#12 for making the device mapper discovery for the "rootdev" command more reliable. This also requied a backport of a kernel patch from 2017 that exposes the PARTUUID in the /sys uevent file. Co-authored-by: James Le Cuirot <[email protected]> Signed-off-by: Kai Lueke <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, it's far more comprehensive than what I was shooting for. I guess I shouldn't assume that it will always just work!
Was there any particular reason why you went with squashfs over erofs in the end, other than mkfs.erofs not being in the SDK? It probably should be.
Even without these changes, the verity-generator module and much of the usr-generator module is redundant anyway following changing in systemd. I have already dropped that code in my sd-veritysetup branch, which was originally going to be part of my boot rework, but I was also planning to apply it regardless. Generally speaking, I don't like leaving dead code around. That's what git is for. 😉
I recall someone saying something about our existing boot process being susceptible to duplicate disks, where it might pick USR-A from a disk other than the one you actually booted from. Maybe we could address that here (later?) although I'm not sure how you would tell which was the boot disk.
The growth of binaries over time and the inclusion of new features filled the available boot partition space, so that the kernel+initrd almost couldn't fit twice anymore as required for updates. We employed workarounds such as wrapper scripts for ignition, afterburn and other binaries so that they are loaded from /usr. However, this was still not enough and we would have to do the same for (network) kernel modules and firmware. To avoid making this ever more complex we can use a dedicated initrd focused on loading the full initrd from /usr and then this full initrd can use dracut as before and even drop all the workarounds we accumulated. Introduce a busybox init script that prepares a minimal environment, has debug toggles and an emergency shell, and only loads the real initrd from /usr to switch over to it. Because mdev is not a proper udev replacement, some additional scripting is needed. Busybox's modprobe can't work with dependencies well and we need the real kmod for that (which is also good to guarantee have the same modprobe options set). Also, some other busybox commands are often lacking things such as loading a kernel module automatically and this has to be done explicitly. We still set up dm-verity for /usr so that we have the same security properties (The code comes from the bootengine systemd generators we have and also covers the PXE boot with a squashfs /usr passed from an additional cpio). The real initrd then reuses the mount point for /usr, and loads any kernel modules and firmware that wasn't loaded already. We also have to make the dependencies for parse-ip-for-networkd.service a bit more explicit because the removal of the /sysusr mount in the full initrd exposed a race condition. Signed-off-by: Kai Lueke <[email protected]>
4d5ac98
to
0432e3a
Compare
So far we use squashfs for everything and I kept using it here for consistency. If we do it, I would prefer a follow-up PR.
Yes, good point. The generators don't handle the missing |
Okay. I thought we were using erofs in the bakery now, which we are, but it's still not the default.
Sorry, I don't know what you're referring to there. |
The systemd generator uses |
Ah! See this wrapper. This is academic in the case of the verity-generator though since you're now handling it here. I just meant that it's redundant either way. |
The minimal initrd now does the verity setup and /sysusr mounting and we can delete parts of the generator code. Delete the unused code but still keep support for "usr=" instead of "mount.usr=" cmdline entries. We could decide to remove this later to get rid of a few more things in the generators. Still, /sysroot(/usr) setup will be needed. For "mount.usr=" the upstream systemd generator handles this but there's still a bit of code to customize the mount args for it. The verity generator gets deleted fully and if we later want to use it again, we can use the upstream one with some cmdline preprocessing. Signed-off-by: Kai Lueke <[email protected]>
I've removed the unused code now and agree that we can probably make use of systemd upstream stuff in the future. |
The growth of binaries over time and the inclusion of new features
filled the available boot partition space, so that the kernel+initrd
almost couldn't fit twice anymore as required for updates. We employed
workarounds such as wrapper scripts for ignition, afterburn and other
binaries so that they are loaded from /usr. However, this was still not
enough and we would have to do the same for (network) kernel modules and
firmware. To avoid making this ever more complex we can use a dedicated
initrd focused on loading the full initrd from /usr and then this full
initrd can use dracut as before and even drop all the workarounds we
accumulated.
Introduce a busybox init script that prepares a minimal environment,
has debug toggles and an emergency shell, and only loads the real initrd
from /usr to switch over to it. Because mdev is not a proper udev
replacement, some additional scripting is needed. Busybox's modprobe
can't work with dependencies well and we need the real kmod for that
(which is also good to guarantee have the same modprobe options set).
Also, some other busybox commands are often lacking things such as
loading a kernel module automatically and this has to be done
explicitly. We still set up dm-verity for /usr so that we have the same
security properties (The code comes from the bootengine systemd
generators we have and also covers the PXE boot with a squashfs /usr
passed from an additional cpio). The real initrd then reuses the mount
point for /usr, and loads any kernel modules and firmware that wasn't
loaded already.
We also have to make the dependencies for parse-ip-for-networkd.service
a bit more explicit because the removal of the /sysusr mount in the full
initrd exposed a race condition.
How to use
With flatcar/scripts#3241
Testing done
See above