- This idea originates from very early in the project and was, at the
time, a very easy way to categorise things.
- Now, it doesn't really make much sense - it is fairly arbitary, often
occuring when there is a change in kernel, but not from builder-hex0
to fiwix, and sysb is in reality completely unnecessary.
- In short, the sys* stuff is a bit of a mess that makes the project
more difficult to understand.
- This puts everything down into one folder and has a manifest file that
is used to generate the build scripts on the fly rather than using
coded scripts.
- This is created in the "seed" stage.
stage0-posix -- (calls) --> seed -- (generates) --> main steps
Alongside this change there are a variety of other smaller fixups to the
general structure of the live-bootstrap rootfs.
- Creating a rootfs has become much simpler and is defined as code in
go.sh. The new structure, for an about-to-be booted system, is
/
-- /steps (direct copy of steps/)
-- /distfiles (direct copy of distfiles/)
-- all files from seed/*
-- all files from seed/stage0-posix/*
- There is no longer such a thing as /usr/include/musl, this didn't
really make any sense, as musl is the final libc used. Rather, to
separate musl and mes, we have /usr/include/mes, which is much easier
to work with.
- This also makes mes easier to blow away later.
- A few things that weren't properly in packages have been changed;
checksum-transcriber, simple-patch, kexec-fiwix have all been given
fully qualified package names.
- Highly breaking change, scripts now exist in their package directory
but NOT WITH THE packagename.sh. Rather, they use pass1.sh, pass2.sh,
etc. This avoids manual definition of passes.
- Ditto with patches; default directory is patches, but then any patch
series specific to a pass are named patches-passX.
All expected environment variables are passed through the .env file.
Any other variable passed from /init and the environment in which it
is executed only "pollutes" the build environment.
No change in package hashes.
A tiny bootloader bootstrap has been added to compile the builder-hex0 kernel from hex0 source.
The boot compiler is builder-hex0-x86-stage1.hex0 and builder-hex0-x86-stage1.bin.
The builder-hex0 kernel is now named builder-hex0-x86-stage2.hex0.
The inclusion of a binary seed resolves the problem with the previous strategy which used an
architecture-specific hex0 compiler.
If sysb detects a full disk (i.e. DISK=sda) it now partitions the disk unconditionally because
previously fdisk was reporting existing but empty partitions which resulted in no
parititions being created.
e2fsprogs is now built with --disable-tls because musl was built on Fiwix without full threading
support and mkfs.ext4 was crashing without disabling thread local storage.
kexec-linux writes the linux kernel and initramfs to a RAM drive on Fiwix which ensure
a pre-allocated contiguous memory block. The following is written to the ram drive:
a 32-bit number which is the size of the kernel in bytes, a 32-bit number which is the size
of the initramfs in bytes, followed by the Linux kernel image, followed by the initramfs.
kexec-fiwix invokes a sync syscall to ensure all writes are flushed to
the ram drive and then initiates the kexec by shutting down Fiwix with a reboot syscall.
Fiwix knows whether and how to perform the kexec based on kernel parameters passed to it.
Environment variables "pollute" the build environment of packages and
can affect their output.
This change results in the removal of some files from packages that
were not meant to be packaged. It also removes the need for a
workaround in automake 1.10.3 to manually remove such files.
Variables are now saved in an .env file for each system and included
in scripts that need them using the dot operation.
- disk to be created (blank disk given to live-bootstrap) (default)
- disk to already exist but sources downloaded within live-bootstrap
- sources to be downloaded outside live-bootstrap (non-blank disk given
to live-bootstrap)
Also migrate sysb to use sys_transfer in QEMU mode also.
Note that this means copy_sysc is now irrelevant. sysc is *always*
sourced from sysa.
These variables should only affect live-bootstrap's scripts, yet they
currently "pollute" the build environment of most packages during the
bootstrap unnecessarily.
This change also makes bootstrap.cfg keep the same format between the
different bootstrap stages, which simplifies the input to each step.
This improves the readablility of the scripts. It also helps avoid
cases where a variable could be set twice in bootstrap.cfg, such as
the DISK variable.
Utilising previously introduced DESTDIR support, everything is installed
to aformentioned DESTDIR.
Prior to the building of XBPS, we use gzip compressed tarballs as
packages. This requires a lot of strange hacks especially for old tar +
gzip (timestamps, etc causing reproducibility problems). Then we use
XBPS once built.
- Add parts.rst documentation for Linux kernel.
- Completely fix problems caused by new bootstrap, update checksums for
/usr.
- Globalise populate_device_nodes.
- Enable deblobbing.
- We do not use latest 4.9.x because it relies on a new version of
binutils, while older versions do not. (Note: we should be able to go
a bit newer but I didn't bother testing >50 versions to figure this
out).
- We do not use newer kernel versions because they require one or more
of (new perl, new binutils, new make, new gcc, new bison, new tar).
- sysb and sysc are updated to use the SATA (libata) subsystem (aka sda)
instead of IDE-emulating SATA subsystem (aka hda) which is now
available to us.
- While theoretically according to docs 4.9 should work OOTB with our
version of binutils this is not the case, so we have to do a bit of
(interesting) patching. But this does not break anything.
- Thankfully serial support in 4.9 is not screwed over like it is in 2.6
so we can revert to that.
- 4.9 has the linux-libre project at our disposal, instead of gNewSense.
So we use this. Unfortunatley that takes forever because we have to
use sed because our version of gawk is too old/buggy. :( I plan to
introduce very shortly 1. parallelism 2. 'sysc snapshot' which will
start from sysc to avoid this. I do not want to use linux-libre
tarballs because they make modificiations directly from this script
(aka not easily verifiable, use the source!) and this script allows
for much greater flexibility.
- We compile the initramfs ahead-of-build using the in-tree cpio
generator instead of also building cpio to use less packages. We do
NOT build the initramfs into the kernel like 2.6 (unsupported).
- Oh and fix a kexec-tools checksum.
Now that we have the Linux Kernel built, we move to a full-disk (rather
than initramfs) setup in sysc. However, we cannot assume the seed kernel
has support for mounting hard drives. So, first we need to kexec into
sysb, which is used as a jumping off point to create the hard drive for
sysc.
Additionally, since 2.6.16 does not have support for on-demand initramfs
(initramfs must be built into kernel), we will have to rebuild the linux
kernel within sysb without the initramfs.
All of this process is not performed for chroot mode. Instead, we skip
sysb and jump straight to sysc, copying over appropriate data.
The python scripts have been changed slightly. Each sys* inherits
SysGeneral, which contains various functions which are not specific to
any sys* and simplifies those files. rootfs now also handles sysb and
sysc.
bootstrap.cfg also gives an indication whether we are running in a
chroot to avoid attempting to kexec/mount within a chroot.
fosslinux
Andrius Štikonas
Dor Askayo
rick-masters