* Bzip2 was manually installed directly into filesystem, so bzip2 package was empty.
Fixed by installing it to destdir. bzip2 moves out its binary before installing
its own package.
* sha256sum from stage0-posix was not checking any checksums because it does not
accept piped input. Fixed by using temporary file.
* grep was broken for a short time (but with fixed bzip2 package this caused failures)
due to touch creating grep file of zero size (egrep symlink was touched).
Fixed by implementing touch -h
Resolves#156, resolves#166, resolves#167
These variables should only affect live-bootstrap's scripts, yet they
currently "pollute" the build environment of most packages during the
bootstrap unnecessarily.
This change also makes bootstrap.cfg keep the same format between the
different bootstrap stages, which simplifies the input to each step.
This improves the readablility of the scripts. It also helps avoid
cases where a variable could be set twice in bootstrap.cfg, such as
the DISK variable.
This makes the order of objects in archives created by libtool
consistent.
It is known to affect cases where the *_LIBADD automake variable is
used to add extra objects from a separate archive.
Fixing this allows us to remove a few workarounds.
This allows to use makefile rather than kaem script for building sed.
Makefiles for musl and mes are unified into one makefile with some
conditional code.
- Rename sources to distfiles for clarity.
- Per sys(a/c) distfiles to reduce rootfs.py processing and reduce RAM
usage in sysa.
- Canonicalise early kaem mes/tcc files to kaem script conventions.
- Cleanup unused setup in python.
Include:
- Not regenerated man pages.
- Automake tarball with bad time (and completely useless).
- Ordering of files within tarballs.
- Resetting timestamp various fixes.
- Older tars not properly overwriting files leading to many issues.
- Weird lack of reproducibility in libtool scripts paths to tools.
- and more
Utilising previously introduced DESTDIR support, everything is installed
to aformentioned DESTDIR.
Prior to the building of XBPS, we use gzip compressed tarballs as
packages. This requires a lot of strange hacks especially for old tar +
gzip (timestamps, etc causing reproducibility problems). Then we use
XBPS once built.
There is no way for gzip 1.2.4 and tar 1.12 to disable the inclusion of
timestamps into the tarball/gzip, which creates non-reproducible
tarballs and hence packages.
While it is theoretically possible to set the timestamps to unix time 0
using touch, in reality this is not possible because mes libc does not
support utime() which sets the timestamp of a file from userspace. So we
need to ignore it rather than (re)set it.
When not running as root, the shebang workaround requires write
permissions to modify help2man. Since the original file is read-only,
permission must be granted explicitly.
Also fix the double '/' in the /usr prefix while at it.
delete --no-auto-compile : useless in this case (MES doesn't compile)
delete -- : this starts an interactive scheme session (which is not the case here)
Generally, this is bad, because reduces featureset of kernel.
However, we don't use any blobbed features anyway.
1. This allows much lower RAM usage.
2. Speeds up deblobbing from hours -> seconds.
This nukes blobbed files instead of replacing blobs.
For further RAM savings, we want to create sysb -without- having linux
tarball extracted at the same time. To accomplish this, we move the
gen_init_cpio + related script out of the tree and create it afterward.
Also use hard links to free up some more space.
- Add parts.rst documentation for Linux kernel.
- Completely fix problems caused by new bootstrap, update checksums for
/usr.
- Globalise populate_device_nodes.
- Enable deblobbing.
- We do not use latest 4.9.x because it relies on a new version of
binutils, while older versions do not. (Note: we should be able to go
a bit newer but I didn't bother testing >50 versions to figure this
out).
- We do not use newer kernel versions because they require one or more
of (new perl, new binutils, new make, new gcc, new bison, new tar).
- sysb and sysc are updated to use the SATA (libata) subsystem (aka sda)
instead of IDE-emulating SATA subsystem (aka hda) which is now
available to us.
- While theoretically according to docs 4.9 should work OOTB with our
version of binutils this is not the case, so we have to do a bit of
(interesting) patching. But this does not break anything.
- Thankfully serial support in 4.9 is not screwed over like it is in 2.6
so we can revert to that.
- 4.9 has the linux-libre project at our disposal, instead of gNewSense.
So we use this. Unfortunatley that takes forever because we have to
use sed because our version of gawk is too old/buggy. :( I plan to
introduce very shortly 1. parallelism 2. 'sysc snapshot' which will
start from sysc to avoid this. I do not want to use linux-libre
tarballs because they make modificiations directly from this script
(aka not easily verifiable, use the source!) and this script allows
for much greater flexibility.
- We compile the initramfs ahead-of-build using the in-tree cpio
generator instead of also building cpio to use less packages. We do
NOT build the initramfs into the kernel like 2.6 (unsupported).
- Oh and fix a kexec-tools checksum.
- This is much more standard and replaces /image in sysa and is the
standard in sysc (avoids many issues).
- GCC needs to have a file created for some unknown reason.
- Checksums updated.
Now that we have the Linux Kernel built, we move to a full-disk (rather
than initramfs) setup in sysc. However, we cannot assume the seed kernel
has support for mounting hard drives. So, first we need to kexec into
sysb, which is used as a jumping off point to create the hard drive for
sysc.
Additionally, since 2.6.16 does not have support for on-demand initramfs
(initramfs must be built into kernel), we will have to rebuild the linux
kernel within sysb without the initramfs.
All of this process is not performed for chroot mode. Instead, we skip
sysb and jump straight to sysc, copying over appropriate data.
The python scripts have been changed slightly. Each sys* inherits
SysGeneral, which contains various functions which are not specific to
any sys* and simplifies those files. rootfs now also handles sysb and
sysc.
bootstrap.cfg also gives an indication whether we are running in a
chroot to avoid attempting to kexec/mount within a chroot.
It seems that there is some instability of git snapshot packages.
I think newer version of git on the remote server packages them
in a different directory layout.
* Properly define coreutils versions, so that it is available in --version
* Move after tar. Later we can move this after gettext/texinfo too.
* There are no prebuilt files in git archive, remove function to delete info files.
fixes: https://github.com/fosslinux/live-bootstrap/issues/99
in my usecase (qemu, kernel 3.18.140) around 50% of
all builds are failing with a wrong binary hash.
the culprit is a varying generation of 'pipesize.h' during:
```
/bin/sh ./psize.sh > pipesize.h
```
normally generated 'pipesize.h' looks like:
```
/*
* pipesize.h
*
* This file is automatically generated by psize.sh
* Do not edit!
*/
#define PIPESIZE 65536
```
in the error case the last line has 512 instead of 65536.
With this change we use a fixed output of the large PIPESIZE.
Dor Askayo
Andrius Štikonas
fosslinux
bauen1
Gabriel Wicki
Melg Eight
Bastian Bittorf