e9787b55c8
These are implemented as dummy jumps with a script that just exits with success. Since this script will be sourced, rather than called, this causes the bootstrap process to exit at that point. The breakpoints are conditional on INTERNAL_CI (we check for "pass1" because that's the only pass when script-generator runs, but the resulting effect is that each pass only bootstraps its own part of the manifest). |
||
|---|---|---|
| .github/workflows | ||
| .reuse | ||
| LICENSES | ||
| kernel-bootstrap | ||
| lib | ||
| seed | ||
| steps | ||
| .gitignore | ||
| .gitmodules | ||
| DEVEL.md | ||
| README.rst | ||
| download-distfiles.sh | ||
| parts.rst | ||
| rootfs.py | ||
| source_manifest.py | ||
README.rst
live-bootstrap
An attempt to provide a reproducible, automatic, complete end-to-end bootstrap from a minimal number of binary seeds to a supported fully functioning operating system.
How do I use this?
Quick start:
See ./rootfs.py --help and follow the instructions given there. This uses a variety of userland tools to prepare the bootstrap.
(Currently, there is no way to perform the bootstrap without external preparations! This is a currently unsolved problem.)
Without using Python:
git submodule update --init --recursive
Consider whether you are going to run this in a chroot, in QEMU, or on bare metal. (All of this can be automated, but not in a trustable way. See further below.) a. chroot: Create a directory where the chroot will reside, run ./download-distfiles.sh, and copy:
System Message: ERROR/3 (<stdin>, line 35)
Unexpected indentation.
The entire contents of seed/stage0-posix into that directory.
All other files in seed into that directory.
steps/ and distfiles/ into that directory. * At least all files listed in steps/pre-network-sources must be
System Message: ERROR/3 (<stdin>, line 39)
Unexpected indentation.
copied in. All other files will be obtained from the network.
Run /bootstrap-seeds/POSIX/x86/kaem-optional-seed in the chroot. (Eg, chroot rootfs /bootstrap-seeds/POSIX/x86/kaem-optional-seed).
System Message: WARNING/2 (<stdin>, line 42)
Block quote ends without a blank line; unexpected unindent.
QEMU: Create two blank disk images. * On the first image, write
System Message: ERROR/3 (<stdin>, line 44)
Unexpected indentation.
seed/stage0-posix/bootstrap-seeds/NATIVE/x86/builder-hex0-x86-stage1.img to it, followed by kernel-bootstrap/builder-hex0-x86-stage2.hex0, followed by zeros padding the disk to the next sector.
System Message: WARNING/2 (<stdin>, line 47)
Block quote ends without a blank line; unexpected unindent.
distfiles can be obtained using ./download-distfiles.sh.
See the list in part a. For every file within that list, write a line to the disk src <size-of-file> <path-to-file>, followed by the contents of the file. * Only copy distfiles listed in steps/pre-network-sources into
System Message: ERROR/3 (<stdin>, line 52)
Unexpected indentation.
this disk.
Optionally (if you don't do this, distfiles will be network downloaded): * On the second image, create an MSDOS partition table and one ext3
System Message: ERROR/3 (<stdin>, line 55)
Unexpected indentation.
partition.
System Message: WARNING/2 (<stdin>, line 56)
Block quote ends without a blank line; unexpected unindent.
- Copy distfiles/ into this disk.
Run QEMU, with 4+G RAM, optionally SMP (multicore), both drives (in the order introduced above), a NIC with model E1000 (-nic user,model=e1000), and -machine kernel-irqchip=split.
System Message: WARNING/2 (<stdin>, line 60)
Enumerated list ends without a blank line; unexpected unindent.
c. Bare metal: Follow the same steps as QEMU, but the disks need to be two different physical disks, and boot from the first disk.
Background
Problem statement
live-bootstrap's overarching problem statement is;
> How can a usable Linux system be created with only human-auditable, and wherever possible, human-written, source code?
Clarifications:
- "usable" means a modern toolchain, with appropriate utilities, that can be used to expand the amount of software on the system, interactively, or non-interactively.
- "human-auditable" is discretionary, but is usually fairly strict. See "Specific things to be bootstrapped" below.
Why is this difficult?
The core of a modern Linux system is primarily written in C and C++. C and C++ are self-hosting, ie, nearly every single C compiler is written in C.
Every single version of GCC was written in C. To avoid using an existing toolchain, we need some way to be able to compile a GCC version without C. We can use a less well-featured compiler, TCC, to do this. And so forth, until we get to a fairly primitive C compiler written in assembly, cc_x86.
Going up through this process requires a bunch of other utilities as well; the autotools suite, guile and autogen, etc. These also have to be matched appropriately to the toolchain available.
Why should I care?
That is outside of the scope of this README. Here’s a few things you can look at:
- https://bootstrappable.org
- Trusting Trust Attack (as described by Ken Thompson)
- https://guix.gnu.org/manual/en/html_node/Bootstrapping.html
- Collapse of the Internet (eg CollapseOS)
Specific things to be bootstrapped
GNU Guix is currently the furthest along project to automate bootstrapping. However, there are a number of non-auditable files used in many of their packages. Here is a list of file types that we deem unsuitable for bootstrapping.
- Binaries (apart from seed hex0, kaem, builder-hex0).
- Any pre-generated configure scripts, or Makefile.in’s from autotools.
- Pre-generated bison/flex parsers (identifiable through a .y file).
- Any source code/binaries downloaded within a software’s build system that is outside of our control to verify before use in the build system.
- Any non-free software. (Must be FSF-approved license).
How does this work?
For a more in-depth discussion, see parts.rst.
Firstly, builder-hex0 is launched. builder-hex0 is a minimal kernel that is written in hex0, existing in 3 self-bootstrapping stages.
This is capable of executing the entirety of stage0-posix, (see seed/stage0-posix), which produces a variety of useful utilities and a basic C language, M2-Planet.
stage0-posix runs a file called after.kaem. This is a shell script that builds and runs a small program called script-generator. This program reads steps/manifest and converts it into a series of shell scripts that can be executed in sequence to complete the bootstrap.
From this point forward, steps/manifest is effectively self documenting. Each package built exists in steps/<pkg>, and the build scripts can be seen there.