Commit Graph

1243 Commits

Author SHA1 Message Date
Gábor Stefanik 97f31072ce Add license information to break.sh
Not that it's exactly copyrightable, being just "exit 0", but linter complains about it.
2023-12-26 13:01:24 +11:00
Gábor Stefanik 2823e16325 Update CI workflows to match changes from simplify 2023-12-26 13:01:24 +11:00
Gábor Stefanik e9787b55c8 Add breakpoints between passes for INTERNAL_CI
These are implemented as dummy jumps with a script that just exits
with success. Since this script will be sourced, rather than called,
this causes the bootstrap process to exit at that point.
The breakpoints are conditional on INTERNAL_CI (we check for "pass1"
because that's the only pass when script-generator runs, but the
resulting effect is that each pass only bootstraps its own part of
the manifest).
2023-12-26 13:01:24 +11:00
Gábor Stefanik f66f9efa2c Update bwrap bootstrap code to match the simplify refactor 2023-12-26 13:01:24 +11:00
Gábor Stefanik 1481ad0d20 Do not try to create /dev nodes under chroot/bwrap 2023-12-26 13:01:24 +11:00
Gábor Stefanik 9ed4ae9a59 Support reusing existing directory for multipass bootstrapping 2023-12-26 13:01:24 +11:00
fosslinux 3fd101e70a
Merge pull request #359 from fosslinux/binutils-2.41
Update binutils to 2.41
2023-12-26 01:53:39 +00:00
fosslinux 6905852107
Merge pull request #354 from Googulator/script-fixes
Python script fixes and improvements
2023-12-26 00:59:34 +00:00
fosslinux 7d0e1c0e75
Merge pull request #366 from Googulator/reproducible-linux
Fix non-reproducible Linux kernel build due to timestamps
2023-12-26 00:22:32 +00:00
Gábor Stefanik 47711cfea9 Fix non-reproducible Linux kernel build due to timestamps
Linux's KBUILD doesn't follow SOURCE_DATE_EPOCH, but rather it
uses its own variable, KBUILD_BUILD_TIMESTAMP.

While we are at it, also update kexec-linux's checksum, which
didn't match either. (This one was reproducible, it was just
out of date.)
2023-12-25 10:16:17 +01:00
Gábor Stefanik 529ea4cae1 Explain significance of build-bash and improve-network directives 2023-12-22 13:12:40 +01:00
Gábor Stefanik ab9455f918 Fix warnings in existing code revealed by newer pylint 2023-12-22 13:08:23 +01:00
Gábor Stefanik be1333ee8b Make tmpdir a method parameter of prepare()
Fixes pylint errors.
2023-12-22 13:08:20 +01:00
Gábor Stefanik b45e1f81ae Fix failure with --external-sources 2023-12-22 13:07:45 +01:00
Gábor Stefanik 32dc4c702b More verbose error message on HTTP errors 2023-12-22 13:07:45 +01:00
Gábor Stefanik 55d3c36e09 Move kernel-bootstrap source image out of the directory it's generated from
This way, an incomplete version of the image itself won't get included
in srcfs anymore.
2023-12-22 13:07:45 +01:00
Gábor Stefanik 3305f2a41b Use manifest to deduce pre-network sources list
No need to maintain a separate pre-network-sources file anymore,
the list is instead derived from the bootstrap manifest via the
source manifest.
2023-12-22 13:07:04 +01:00
Gábor Stefanik a68ae62f9e Download distfiles only when needed, based on manifest
Unless --external-sources is given, only download distfiles that
need to be included in srcfs. The rest will be downloaded anyway
by the bootstrap system once it gets network access.

To accomplish this, instead of searching steps for sources files,
we now parse steps/manifest. As a side effect, source_manifest.py
now outputs source files in the order they appear in the manifest.
2023-12-22 13:07:04 +01:00
Gábor Stefanik 8d193df823 Fix printed image paths for bare-metal bootstrap 2023-12-22 13:06:59 +01:00
Gábor Stefanik faad907fba Fix source_manifest.py in light of the recent refactor
There's no sysa or sysc anymore.
2023-12-22 13:06:19 +01:00
Gábor Stefanik c188185ad4 Remove residual references to sysa/sysb/sysc from rootfs.py 2023-12-22 13:06:19 +01:00
Gábor Stefanik b4d9c5e7cb Disk creation improvements
* Support specifying the size of the target disk image for qemu
* For bare metal, only pad the image to the next megabyte
* Use truncate() to extend images, instead of writing zeros (faster)
* Return None from get_disk() with nonexistent name
* Leave 1MiB on non-boot disks, or 1GiB on boot disks, unpartitioned
  (for proper 4K alignment and to help preserve the srcfs or boot
  partition creation)
* Fix qemu invocation when an external.img is not used
* Make -qr work with kernel bootstrap (will need kexec-fiwix fix)
2023-12-22 13:06:19 +01:00
Gábor Stefanik dc62d8242c Create external.img only when needed 2023-12-22 12:10:09 +01:00
fosslinux 191c013e23 Update binutils to 2.41 2023-12-21 11:03:15 +11:00
Andrius Štikonas 55ad47acd7
Merge pull request #357 from Googulator/amd64-archdir-fix
AMD64 archdir fix
2023-12-19 21:11:28 +00:00
Gábor Stefanik abca1a4b15 Fix ARCH vs ARCH_DIR confusion causing amd64 breakage 2023-12-19 18:32:59 +01:00
fosslinux 545bb42ca8
Merge pull request #334 from fosslinux/simplify
Simplify + Sustainability
2023-12-15 11:54:29 +00:00
fosslinux ac4a8c7e79 Install heirloom-devtools utility files to /usr/lib
Before they were just in / for no real reason
2023-12-15 22:51:00 +11:00
fosslinux ab47483a51 Linters & CI updates 2023-12-15 21:44:43 +11:00
fosslinux ff4f97ab8f Update all checksums 2023-12-15 21:44:27 +11:00
fosslinux ee77ef837d Update docs with new changes 2023-12-15 21:43:21 +11:00
fosslinux e06a19f9e2 Create a proper Linux kernel package 2023-12-15 21:43:21 +11:00
fosslinux 05c13dd64e Modify rootfs.py infrastructure to support the new layout 2023-12-15 21:43:21 +11:00
fosslinux 6ed2e09f3a Remove the notion of "sys*"
- This idea originates from very early in the project and was, at the
  time, a very easy way to categorise things.
- Now, it doesn't really make much sense - it is fairly arbitary, often
  occuring when there is a change in kernel, but not from builder-hex0
  to fiwix, and sysb is in reality completely unnecessary.
- In short, the sys* stuff is a bit of a mess that makes the project
  more difficult to understand.
- This puts everything down into one folder and has a manifest file that
  is used to generate the build scripts on the fly rather than using
  coded scripts.
- This is created in the "seed" stage.

stage0-posix -- (calls) --> seed -- (generates) --> main steps

Alongside this change there are a variety of other smaller fixups to the
general structure of the live-bootstrap rootfs.

- Creating a rootfs has become much simpler and is defined as code in
  go.sh. The new structure, for an about-to-be booted system, is

/
-- /steps (direct copy of steps/)
-- /distfiles (direct copy of distfiles/)
-- all files from seed/*
-- all files from seed/stage0-posix/*

- There is no longer such a thing as /usr/include/musl, this didn't
  really make any sense, as musl is the final libc used. Rather, to
  separate musl and mes, we have /usr/include/mes, which is much easier
  to work with.
- This also makes mes easier to blow away later.
- A few things that weren't properly in packages have been changed;
  checksum-transcriber, simple-patch, kexec-fiwix have all been given
  fully qualified package names.
- Highly breaking change, scripts now exist in their package directory
  but NOT WITH THE packagename.sh. Rather, they use pass1.sh, pass2.sh,
  etc. This avoids manual definition of passes.
  - Ditto with patches; default directory is patches, but then any patch
    series specific to a pass are named patches-passX.
2023-12-15 21:43:19 +11:00
fosslinux 0907cfd073 Add script-generator 2023-12-15 21:41:42 +11:00
fosslinux d13320dbfa
Merge pull request #344 from fosslinux/simplify-kexec-prepare
Simplify - kexec preparations
2023-12-15 10:40:04 +00:00
fosslinux 93682cbb92 Expand kexec fiwix initramfs 2023-12-15 21:38:09 +11:00
fosslinux 08bd0ef495 Support command execution in kexec linux arg
! means execute the argument as a command
2023-12-15 21:37:27 +11:00
fosslinux cef781a9f6
Merge pull request #343 from fosslinux/coreutils-stat
Build stat for coreutils
2023-12-15 10:36:16 +00:00
fosslinux d4e0fc9911
Merge pull request #341 from fosslinux/early-tools
Early tools in subdirectories
2023-12-15 10:35:59 +00:00
fosslinux 829f18d604 Rename sysa.ext2 to fiwix.ext2 2023-12-15 21:35:12 +11:00
fosslinux a0d4598028 Move checksum-transcriber to a subdirectory 2023-12-15 21:35:12 +11:00
fosslinux ddd8c8d380 Move simple-patch into a subdirectory
Add a checksum as well.

Closes #323
2023-12-15 21:35:12 +11:00
fosslinux 341a4681f5 Give kexec-fiwix a version number 2023-12-15 21:35:12 +11:00
fosslinux 73c4985aba
Merge pull request #340 from fosslinux/kaem-improv
Kaem-era improvements
2023-12-15 10:34:55 +00:00
fosslinux acddacbe6d
Merge pull request #337 from fosslinux/tarball-unify
Use Linux headers 4.9.10
2023-12-15 10:34:41 +00:00
fosslinux bb6fda569f Build stat for coreutils as well 2023-12-05 15:53:14 +11:00
Andrius Štikonas 70b7bc7f83
Merge pull request #342 from MaxHearnden/sysa-comma 2023-11-29 22:58:05 +00:00
MaxHearnden 773cc5405c add comma to list in sysa.py 2023-11-28 22:50:47 +00:00
fosslinux fc5fc39569 Use (/usr/lib/mes, /usr/include/mes), (/usr/lib, /usr/include)
Mes is now self-contained in;
/usr/lib/mes
/usr/include/mes
/usr/lib/x86-mes

musl adheres to FHS
/usr/include
/usr/lib
2023-11-28 16:56:04 +11:00