6ed2e09f3a
- This idea originates from very early in the project and was, at the time, a very easy way to categorise things. - Now, it doesn't really make much sense - it is fairly arbitary, often occuring when there is a change in kernel, but not from builder-hex0 to fiwix, and sysb is in reality completely unnecessary. - In short, the sys* stuff is a bit of a mess that makes the project more difficult to understand. - This puts everything down into one folder and has a manifest file that is used to generate the build scripts on the fly rather than using coded scripts. - This is created in the "seed" stage. stage0-posix -- (calls) --> seed -- (generates) --> main steps Alongside this change there are a variety of other smaller fixups to the general structure of the live-bootstrap rootfs. - Creating a rootfs has become much simpler and is defined as code in go.sh. The new structure, for an about-to-be booted system, is / -- /steps (direct copy of steps/) -- /distfiles (direct copy of distfiles/) -- all files from seed/* -- all files from seed/stage0-posix/* - There is no longer such a thing as /usr/include/musl, this didn't really make any sense, as musl is the final libc used. Rather, to separate musl and mes, we have /usr/include/mes, which is much easier to work with. - This also makes mes easier to blow away later. - A few things that weren't properly in packages have been changed; checksum-transcriber, simple-patch, kexec-fiwix have all been given fully qualified package names. - Highly breaking change, scripts now exist in their package directory but NOT WITH THE packagename.sh. Rather, they use pass1.sh, pass2.sh, etc. This avoids manual definition of passes. - Ditto with patches; default directory is patches, but then any patch series specific to a pass are named patches-passX. |
||
---|---|---|
.github/workflows | ||
.reuse | ||
LICENSES | ||
kernel-bootstrap | ||
lib | ||
seed | ||
steps | ||
.gitignore | ||
.gitmodules | ||
DEVEL.md | ||
README.rst | ||
download-distfiles.sh | ||
parts.rst | ||
rootfs.py | ||
source_manifest.py | ||
sysa.py |
README.rst
live-bootstrap
An attempt to provide a reproducible, automatic, complete end-to-end bootstrap from a minimal number of binary seeds to a supported fully functioning operating system.
Get me started!
- git clone https://github.com/fosslinux/live-bootstrap
- git submodule update --init --recursive
- Provide a kernel (vmlinuz file) as the name kernel in the root of the repository. This must be a 32-bit kernel.
- ./rootfs.py --qemu - ensure your account has kvm privileges and qemu
installed.
- Alternatively, run ./rootfs.py --chroot to run it in a chroot.
- Alternatively, run ./rootfs.py --bwrap to run it in a bubblewrap sandbox. When user namespaces are supported, this mode is rootless.
- Alternatively, run ./rootfs.py but don’t run the actual virtualization and instead copy sysa/tmp/initramfs to a USB or some other device and boot from bare metal. NOTE: we now require a hard drive. This is currently hardcoded as sda. You also need to put sysc/tmp/disk.img onto your sda on the bootstrapping machine.
- Alternatively, do not use python at all, see "Python-less build" below.
- Wait.
- If you can, observe the many binaries in /usr/bin! When the bootstrap is completed bash is launched providing a shell to explore the system.
Background
This project is a part of the bootstrappable project, a project that aims to be able to build complete computing platforms through the use of source code. When you build a compiler like GCC, you need another C compiler to compile the compiler - turtles all the way down. Even the first GCC compiler was written in C. There has to be a way to break the chain…
There has been significant work on this over the last 5 years, from Jeremiah Orians’ stage0, hex2 and M2-Planet to janneke’s Mes. We have a currently, fully-functioning chain of bootstrapping from the 357-byte hex0 seed to a complete GCC compiler and hence a full Linux operating system. From there, it is trivial to move to other UNIXes. However, there is only currently one vector through which this can be automatically done, GNU Guix.
While the primary author of this project does not believe Guix is a bad project, the great reliance on Guile, the complexity of many of the scripts and the rather steep learning curve to install and run Guix make it a very non plug-and-play solution. Furthermore, there is currently (Jan 2021) no possible way to run the bootstrap from outside of a pre-existing Linux environment. Additionally, Guix uses many scripts and distributed files that cannot be considered source code.
(NOTE: Guix is working on a Full Source Bootstrap, but I’m not completely sure what that entails).
Furthermore, having an alternative bootstrap automation tool allows people to have greater trust in the bootstrap procedure.
Comparison between GNU Guix and live-bootstrap
Item | Guix | live-bootstrap |
---|---|---|
Total size of seeds [1] | ~30MB (Reduced Source Bootstrap) [2] | ~1KB |
Use of kernel | Linux-Libre Kernel | Any Linux Kernel (2.6+) [3] |
Implementation complete | Yes | No (in development) |
Automation | Almost fully automatic | Optional user customization |
[1]: Both projects only use software licensed under a FSF-approved free software license. Kernel is excluded from seed. [2]: Reiterating that Guix is working on a full source bootstrap, although that still uses guile (~12 MB). [3]: Work is ongoing to use other, smaller POSIX kernels.
Why would I want bootstrapping?
That is outside of the scope of this README. Here’s a few things you can look at:
- https://bootstrappable.org
- Trusting Trust Attack (as described by Ken Thompson)
- https://guix.gnu.org/manual/en/html_node/Bootstrapping.html
- Collapse of the Internet (eg CollapseOS)
Specific things to be bootstrapped
GNU Guix is currently the furthest along project to automate bootstrapping. However, there are a number of non-auditable files used in many of their packages. Here is a list of file types that we deem unsuitable for bootstrapping.
- Binaries (apart from seed hex0, kaem, kernel).
- Any pre-generated configure scripts, or Makefile.in’s from autotools.
- Pre-generated bison/flex parsers (identifiable through a .y file).
- Any source code/binaries downloaded within a software’s build system that is outside of our control to verify before use in the build system.
- Any non-free software. (Must be FSF-approved license).
How does this work?
For a more in-depth discussion, see parts.rst.
sysa
sysa is the first ‘system’ used in live-bootstrap. We move to a new system after a reboot, which often occurs after the movement to a new kernel. It is run by the seed Linux kernel provided by the user. It compiles everything we need to be able to compile our own Linux kernel. It runs fully in an initramfs and does not rely on disk support in the seed Linux kernel.
sysb
sysb is the second 'system' of live-bootstrap. This uses the Linux 4.9.10 kernel compiled within sysa. As we do not rely on disk support in sysa, we need this intermediate system to be able to add the missing binaries to sysc before moving into it. This is executed through kexec from sysa. At this point, a SATA disk IS required.
sysc
sysc is the (current) last 'system' of live-bootstrap. This is a continuation from sysb, executed through util-linux's switch_root command which moves the entire rootfs without a reboot. Every package from here on out is compiled under this system, taking binaries from sysa. Chroot and bubblewrap modes skip sysb, as it is obviously irrelevant to them.
Python-less build
Python is no longer a requirement to set up the build system. The repository is almost completely in a form where it can be used as the source of a build.
- Download required tarballs into sysa/distfiles and sysc/distfiles. You can use the download-distfiles.sh script.
- Copy sysa/stage0-posix/src/* to the root of the repository.
- Copy sysa/stage0-posix/src/bootstrap-seeds/POSIX/x86/kaem-optional-seed to init in the root of the repository.
- Copy sysa/after.kaem to after.kaem
- Create a CPIO archive (eg, cpio --format newc --create --directory . > ../initramfs).
- Boot your initramfs and kernel.
chroot builds
For chroot based bootstraps you can skip creation of initramfs and instead start bootstrap with
sudo chroot . bootstrap-seeds/POSIX/x86/kaem-optional-seed
It is also recommended to copy everything to a new directory as bootstrapping messes up with files in git repository and cannot be re-run again.