Commit Graph

38 Commits

Author SHA1 Message Date
Eduardo Sánchez Muñoz c143e65154 Pass `--unshare-net` to bwrap when building with `--external-sources`
Ensures that when `--external-sources` is used, the build does not require a working internet connection.

`--unshare-net` is always passed for the sysa phase.
2023-04-13 18:52:37 +02:00
fosslinux 58b8c1768c Clarify custom kernel logic 2023-04-11 16:36:25 +10:00
rick-masters 3c817913c7 Fix criteria for kernel bootstrapping. 2023-04-03 18:07:24 +00:00
rick-masters ecf99ab08b Implement kernel bootstrap up to building Linux. 2023-03-31 18:08:50 +00:00
Seppo Yli-Olli 7167610125 Split sysa and sysc when using bwrap
Using separate bwrap invocations allows dropping chroot permission
2023-02-07 18:03:20 +02:00
Seppo Yli-Olli 56cab8bb71 Support building only sysa with chroot mode
This is intended primarily for external build systems that directly
bind into sysa and sysc rather than using rootfs Python wrapper.
2023-02-02 09:12:49 +02:00
Dor Askayo 74c852b166 Add /dev/shm for sysc
This allows named semaphores to be created. See sem_overview(7).
2023-02-02 01:55:33 +02:00
fosslinux ac010bc1a0
Merge pull request #201 from doraskayo/musl-libdir-triplet
Rename /usr/lib/musl to /usr/lib/i386-unknown-linux-musl
2023-01-30 08:52:17 +11:00
fosslinux 5458a3e6aa Quick fix to tmpdir rework
QEMU no --external-sources broken due to bootstrap.cfg
2023-01-29 21:07:56 +11:00
Dor Askayo 46a5773f66 Fix a typo in rootfs.py 2023-01-28 19:14:49 +02:00
fosslinux 51b0bf8405 Rework tmpdir & associated a bit.
- Split out tmpdir logic into a separate entity & add the appropriate
  arguments and checks.
- sysb can be removed since there is now no associated logic.
- Move disk/etc logic into tmpdir.py.
2023-01-28 16:51:00 +11:00
Dor Askayo ce2276583a Mention bubblewrap mode in error messages 2023-01-13 10:52:12 +02:00
fosslinux 6f76fd2071 Add early preseed
Speeds up development cycle, allows for the pre-kaem stages to be
extracted as a tarball
2023-01-05 20:44:19 +11:00
Dor Askayo 66b1ba21a6 Create tmpdir before bootstrap 2022-09-07 23:37:08 +03:00
fosslinux 5ad055d2d3 Fix bubblewrap mode 2022-06-18 09:10:11 +10:00
fosslinux 25f37ba926 Various fixes from rebase
This took a while so a bunch of stuff got broken in rebase.
2022-06-18 09:10:11 +10:00
fosslinux 0ce50a6393 Add options for;
- disk to be created (blank disk given to live-bootstrap) (default)
- disk to already exist but sources downloaded within live-bootstrap
- sources to be downloaded outside live-bootstrap (non-blank disk given
  to live-bootstrap)

Also migrate sysb to use sys_transfer in QEMU mode also.

Note that this means copy_sysc is now irrelevant. sysc is *always*
sourced from sysa.
2022-06-10 13:33:16 +10:00
Andrius Štikonas 28986a4a74 Fix remaining repo_path merge issues. 2022-06-05 20:59:35 +01:00
Andrius Štikonas 1a4e83e2fb Fix another mismerge that broke rootless bootstrap. 2022-06-02 20:51:15 +01:00
Andrius Štikonas d84bb236a4 Add an argument to rootfs.py to preseed bootstrap with binary packages. 2022-05-29 00:04:57 +01:00
Dor Askayo 1e67477ff2 Add a rootless bootstrap mode using bubblewrap
Set up a bubblewrap sandbox suitable for bootstrap. Since bubblewrap
can operate without root permissions when user namespaces are
allowed, this effectively adds a rootless bootstrap mode.
2022-05-27 11:46:14 +03:00
Dor Askayo 8330ab4504 Add an option to avoid creating a tmpfs in prepare()
Root access is required for creating tmpfs mounts in the context of
the current mount namespace, and creating a tmpfs in the context of a
new mount namespace is less useful because a process in the parent
namespace can't easily access it.

So add an option to avoid creating tmpfs mounts, which will be used by
the rootless bootstrap mode for now.

In addition, when tmp directories aren't mounted as tmpfs, their
contents can't be removed using os.umount(). So instead remove them
recursively using shutil.rmtree().
2022-05-27 11:45:09 +03:00
Dor Askayo 6d357226a9 Call prepare() externally to the sysa and sysc classes
This keeps the prepartion and bootstrap initiation logic in the same
place for each bootstrap mode, and allows each mode to specify its
own requirements and expectations from the different bootstrap steps.
2022-05-23 17:45:53 +03:00
Dor Askayo c429cf9dd7 Keep chroot indication only in prepare()
It's not actually needed by other methods.
2022-05-23 17:45:53 +03:00
fosslinux f1600467a7 Variety of improvements
- Rename sources to distfiles for clarity.
- Per sys(a/c) distfiles to reduce rootfs.py processing and reduce RAM
  usage in sysa.
- Canonicalise early kaem mes/tcc files to kaem script conventions.
- Cleanup unused setup in python.
2022-05-05 17:44:47 +10:00
Andrius Štikonas 7a8567dc78 Rename /after to /sysa. 2022-04-11 18:53:35 +01:00
Andrius Štikonas 32ec57d62d Unhardcode x86 from initial steps. 2021-10-20 21:16:05 +01:00
fosslinux c8998e9e71 Highlight need for 32-bit kernel; use 4G ram by default
It will always fit within 4G RAM.
2021-10-13 17:57:50 +01:00
fosslinux 306dac7ba6 Add prompts when particular options are not given in config files 2021-10-13 00:42:21 +01:00
Andrius Štikonas c6c8a1eb7f Move chroot transition script inside bootstrap. 2021-10-10 16:38:52 +01:00
fosslinux a8a9056f1d Change some formats to f-strings
As pylint recommends.
2021-09-20 14:31:10 +10:00
fosslinux 04180f5672 Various fixes + cleanup.
- Add parts.rst documentation for Linux kernel.
- Completely fix problems caused by new bootstrap, update checksums for
  /usr.
- Globalise populate_device_nodes.
- Enable deblobbing.
2021-09-14 14:59:08 +10:00
fosslinux d429c48d76 Update the linux kernel for sysb/c to 4.9.10.
- We do not use latest 4.9.x because it relies on a new version of
  binutils, while older versions do not. (Note: we should be able to go
  a bit newer but I didn't bother testing >50 versions to figure this
  out).
- We do not use newer kernel versions because they require one or more
  of (new perl, new binutils, new make, new gcc, new bison, new tar).
- sysb and sysc are updated to use the SATA (libata) subsystem (aka sda)
  instead of IDE-emulating SATA subsystem (aka hda) which is now
  available to us.
- While theoretically according to docs 4.9 should work OOTB with our
  version of binutils this is not the case, so we have to do a bit of
  (interesting) patching. But this does not break anything.
- Thankfully serial support in 4.9 is not screwed over like it is in 2.6
  so we can revert to that.
- 4.9 has the linux-libre project at our disposal, instead of gNewSense.
  So we use this. Unfortunatley that takes forever because we have to
  use sed because our version of gawk is too old/buggy. :( I plan to
  introduce very shortly 1. parallelism 2. 'sysc snapshot' which will
  start from sysc to avoid this. I do not want to use linux-libre
  tarballs because they make modificiations directly from this script
  (aka not easily verifiable, use the source!) and this script allows
  for much greater flexibility.
- We compile the initramfs ahead-of-build using the in-tree cpio
  generator instead of also building cpio to use less packages. We do
  NOT build the initramfs into the kernel like 2.6 (unsupported).
- Oh and fix a kexec-tools checksum.
2021-09-13 13:43:36 +10:00
fosslinux 5c88f1c87f Add sysb and sysc scaffolding.
Now that we have the Linux Kernel built, we move to a full-disk (rather
than initramfs) setup in sysc. However, we cannot assume the seed kernel
has support for mounting hard drives. So, first we need to kexec into
sysb, which is used as a jumping off point to create the hard drive for
sysc.

Additionally, since 2.6.16 does not have support for on-demand initramfs
(initramfs must be built into kernel), we will have to rebuild the linux
kernel within sysb without the initramfs.

All of this process is not performed for chroot mode. Instead, we skip
sysb and jump straight to sysc, copying over appropriate data.

The python scripts have been changed slightly. Each sys* inherits
SysGeneral, which contains various functions which are not specific to
any sys* and simplifies those files. rootfs now also handles sysb and
sysc.

bootstrap.cfg also gives an indication whether we are running in a
chroot to avoid attempting to kexec/mount within a chroot.
2021-08-27 14:54:08 +10:00
Melg Eight cf4ce56603
Add config file and force timestamp conditional option 2021-05-20 02:04:39 +03:00
Andrius Štikonas 8406399f47 Do not require chroot to be in PATH of non-root user. 2021-04-22 21:55:52 +01:00
Bastian Bittorf 7e69dcc926 rootfs.py: minikernel:
use branch v0.7 from kritis-linux and change settings
to be more i386-alike and add essential kernel feature
'flock' using latest kernel 3.18.140 from 3.18.x-branch

(+minor indentation fixes)
2021-04-21 08:04:56 +00:00
Andrius Štikonas 713fd9143f Port rootfs.sh to Python 2021-04-11 00:06:24 +01:00