live-bootstrap/rootfs.py

137 lines
5.2 KiB
Python
Raw Normal View History

2021-04-06 00:12:02 +01:00
#!/usr/bin/env python3
"""
A helper application used to start bootstrapping process.
It has a few modes of operation, you can create initramfs with
binary seeds and sources that you can boot into or alternatively
you can run bootstap inside chroot.
"""
# SPDX-License-Identifier: GPL-3.0-or-later
# SPDX-FileCopyrightText: 2021 Andrius Štikonas <andrius@stikonas.eu>
# SPDX-FileCopyrightText: 2021 Bastian Bittorf <bb@npl.de>
# SPDX-FileCopyrightText: 2021 Melg Eight <public.melg8@gmail.com>
# SPDX-FileCopyrightText: 2021 fosslinux <fosslinux@aussies.space>
2021-04-06 00:12:02 +01:00
import argparse
import os
import shutil
2021-04-06 00:12:02 +01:00
from sysa import SysA
from sysb import SysB
from sysc import SysC
from lib.utils import run
def create_configuration_file(args):
"""
Creates bootstrap.cfg file which would contain options used to
customize bootstrap.
"""
config_path = os.path.join('sysglobal', 'bootstrap.cfg')
with open(config_path, "w", encoding="utf_8") as config:
config.write("FORCE_TIMESTAMPS=" + str(args.force_timestamps) + "\n")
config.write("CHROOT=" + str(args.chroot) + "\n")
Update the linux kernel for sysb/c to 4.9.10. - We do not use latest 4.9.x because it relies on a new version of binutils, while older versions do not. (Note: we should be able to go a bit newer but I didn't bother testing >50 versions to figure this out). - We do not use newer kernel versions because they require one or more of (new perl, new binutils, new make, new gcc, new bison, new tar). - sysb and sysc are updated to use the SATA (libata) subsystem (aka sda) instead of IDE-emulating SATA subsystem (aka hda) which is now available to us. - While theoretically according to docs 4.9 should work OOTB with our version of binutils this is not the case, so we have to do a bit of (interesting) patching. But this does not break anything. - Thankfully serial support in 4.9 is not screwed over like it is in 2.6 so we can revert to that. - 4.9 has the linux-libre project at our disposal, instead of gNewSense. So we use this. Unfortunatley that takes forever because we have to use sed because our version of gawk is too old/buggy. :( I plan to introduce very shortly 1. parallelism 2. 'sysc snapshot' which will start from sysc to avoid this. I do not want to use linux-libre tarballs because they make modificiations directly from this script (aka not easily verifiable, use the source!) and this script allows for much greater flexibility. - We compile the initramfs ahead-of-build using the in-tree cpio generator instead of also building cpio to use less packages. We do NOT build the initramfs into the kernel like 2.6 (unsupported). - Oh and fix a kexec-tools checksum.
2021-08-04 03:56:07 +01:00
config.write("DISK=sda1\n")
2021-04-06 00:12:02 +01:00
def main():
"""
A few command line arguments to customize bootstrap.
This function also creates SysA object which prepares directory
structure with bootstrap seeds and all sources.
"""
parser = argparse.ArgumentParser()
parser.add_argument("-a", "--arch", help="Bootstrap architecture",
default="x86")
parser.add_argument("-c", "--chroot", help="Run inside chroot",
action="store_true")
parser.add_argument("-p", "--preserve", help="Do not unmount temporary dir",
action="store_true")
parser.add_argument("-t", "--tmpdir", help="Temporary directory")
parser.add_argument("--force_timestamps",
help="Force all files timestamps to be 0 unix time",
action="store_true")
2021-04-06 00:12:02 +01:00
# QEMU arguments
parser.add_argument("-q", "--qemu-cmd", help="QEMU command",
default="qemu-system-x86_64")
parser.add_argument("-r", "--qemu-ram", help="Memory (in megabytes) allocated to QEMU VM",
default=8000)
parser.add_argument("-k", "--kernel", help="Kernel to use (default is ./kernel)",
default="kernel")
parser.add_argument("-m", "--minikernel", help="Use minikernel",
action="store_true")
args = parser.parse_args()
if args.chroot and args.minikernel:
raise ValueError("chroot and minikernel options cannot be used simultaneously.")
if args.arch != "x86":
raise ValueError("Only x86 is supported at the moment.")
create_configuration_file(args)
2021-04-06 00:12:02 +01:00
system_b = SysB(arch=args.arch, preserve_tmp=args.preserve,
tmpdir=args.tmpdir, chroot=args.chroot)
system_a = SysA(arch=args.arch, preserve_tmp=args.preserve,
tmpdir=args.tmpdir, chroot=args.chroot, sysb_tmp=system_b.tmp_dir)
system_c = SysC(arch=args.arch, preserve_tmp=args.preserve,
tmpdir=args.tmpdir, chroot=args.chroot)
2021-04-06 00:12:02 +01:00
bootstrap(args, system_a, system_b, system_c)
2021-04-06 00:12:02 +01:00
def bootstrap(args, system_a, system_b, system_c):
2021-04-06 00:12:02 +01:00
"""Kick off bootstrap process."""
print("Bootstrapping %s -- SysA" % (args.arch))
2021-04-06 00:12:02 +01:00
if args.chroot:
find_chroot = """
import shutil
print(shutil.which('chroot'))
"""
chroot_binary = run('sudo', 'python3', '-c', find_chroot,
capture_output=True).stdout.decode().strip()
# sysa
2021-04-06 00:12:02 +01:00
init = os.path.join(os.sep, 'bootstrap-seeds', 'POSIX', args.arch, 'kaem-optional-seed')
run('sudo', 'env', '-i', 'PATH=/bin', chroot_binary, system_a.tmp_dir, init)
# Perform the steps for sysa -> sysc transition that would occur within
# qemu if we were running not in chroot
# We skip sysb as that is only pertinent to "hardware" (not chroot)
system_c.chroot_transition(system_a.tmp_dir)
# sysc
print("Bootstrapping %s -- SysC" % (args.arch))
init = os.path.join(os.sep, 'init')
run('sudo', chroot_binary, system_c.tmp_dir, init)
elif args.minikernel:
if os.path.isdir('kritis-linux'):
shutil.rmtree('kritis-linux')
run('git', 'clone',
'--depth', '1', '--branch', 'v0.7',
2021-04-06 00:12:02 +01:00
'https://github.com/bittorf/kritis-linux.git')
run('kritis-linux/ci_helper.sh',
'--private',
'--multi', '1',
'--repeat', '1',
'--arch', args.arch,
'--qemucpu', '486',
'--kernel', '3.18.140',
'--features', 'kflock,highrestimers',
# Hack to add -hda /dev/blah
'--ramsize', str(args.qemu_ram) + 'M -hda ' + system_b.dev_name,
'--initrd', system_a.initramfs_path,
'--log', '/tmp/bootstrap.log')
else:
run(args.qemu_cmd,
'-enable-kvm',
'-m', str(args.qemu_ram) + 'M',
'-no-reboot',
'-hda', system_c.dev_name,
'-kernel', args.kernel,
'-initrd', system_a.initramfs_path,
Update the linux kernel for sysb/c to 4.9.10. - We do not use latest 4.9.x because it relies on a new version of binutils, while older versions do not. (Note: we should be able to go a bit newer but I didn't bother testing >50 versions to figure this out). - We do not use newer kernel versions because they require one or more of (new perl, new binutils, new make, new gcc, new bison, new tar). - sysb and sysc are updated to use the SATA (libata) subsystem (aka sda) instead of IDE-emulating SATA subsystem (aka hda) which is now available to us. - While theoretically according to docs 4.9 should work OOTB with our version of binutils this is not the case, so we have to do a bit of (interesting) patching. But this does not break anything. - Thankfully serial support in 4.9 is not screwed over like it is in 2.6 so we can revert to that. - 4.9 has the linux-libre project at our disposal, instead of gNewSense. So we use this. Unfortunatley that takes forever because we have to use sed because our version of gawk is too old/buggy. :( I plan to introduce very shortly 1. parallelism 2. 'sysc snapshot' which will start from sysc to avoid this. I do not want to use linux-libre tarballs because they make modificiations directly from this script (aka not easily verifiable, use the source!) and this script allows for much greater flexibility. - We compile the initramfs ahead-of-build using the in-tree cpio generator instead of also building cpio to use less packages. We do NOT build the initramfs into the kernel like 2.6 (unsupported). - Oh and fix a kexec-tools checksum.
2021-08-04 03:56:07 +01:00
'-nographic',
'-append', 'console=ttyS0')
2021-04-06 00:12:02 +01:00
if __name__ == "__main__":
2021-04-06 00:12:02 +01:00
main()