live-bootstrap/download-distfiles.sh

62 lines
1.6 KiB
Bash
Raw Permalink Normal View History

#!/usr/bin/env bash
# SPDX-FileCopyrightText: 2022 fosslinux <fosslinux@aussies.space>
#
# SPDX-License-Identifier: GPL-3.0-or-later
download_source() {
local distfiles="${1}"
local url="${2}"
local checksum="${3}"
local fname="${4}"
# Default to basename of url if not given
fname="${fname:-$(basename "${url}")}"
local dest_path="${distfiles}/${fname}"
if ! [ -e "${dest_path}" ]; then
echo "Downloading ${fname}"
curl --fail --location "${url}" --output "${dest_path}" || true
fi
}
check_source() {
local distfiles="${1}"
local url="${2}"
local checksum="${3}"
local fname="${4}"
# Default to basename of url if not given
fname="${fname:-$(basename "${url}")}"
local dest_path="${distfiles}/${fname}"
echo "${checksum} ${dest_path}" | sha256sum -c
}
set -e
Remove the notion of "sys*" - This idea originates from very early in the project and was, at the time, a very easy way to categorise things. - Now, it doesn't really make much sense - it is fairly arbitary, often occuring when there is a change in kernel, but not from builder-hex0 to fiwix, and sysb is in reality completely unnecessary. - In short, the sys* stuff is a bit of a mess that makes the project more difficult to understand. - This puts everything down into one folder and has a manifest file that is used to generate the build scripts on the fly rather than using coded scripts. - This is created in the "seed" stage. stage0-posix -- (calls) --> seed -- (generates) --> main steps Alongside this change there are a variety of other smaller fixups to the general structure of the live-bootstrap rootfs. - Creating a rootfs has become much simpler and is defined as code in go.sh. The new structure, for an about-to-be booted system, is / -- /steps (direct copy of steps/) -- /distfiles (direct copy of distfiles/) -- all files from seed/* -- all files from seed/stage0-posix/* - There is no longer such a thing as /usr/include/musl, this didn't really make any sense, as musl is the final libc used. Rather, to separate musl and mes, we have /usr/include/mes, which is much easier to work with. - This also makes mes easier to blow away later. - A few things that weren't properly in packages have been changed; checksum-transcriber, simple-patch, kexec-fiwix have all been given fully qualified package names. - Highly breaking change, scripts now exist in their package directory but NOT WITH THE packagename.sh. Rather, they use pass1.sh, pass2.sh, etc. This avoids manual definition of passes. - Ditto with patches; default directory is patches, but then any patch series specific to a pass are named patches-passX.
2023-11-06 23:51:23 +00:00
cd "$(dirname "$(readlink -f "$0")")"
Remove the notion of "sys*" - This idea originates from very early in the project and was, at the time, a very easy way to categorise things. - Now, it doesn't really make much sense - it is fairly arbitary, often occuring when there is a change in kernel, but not from builder-hex0 to fiwix, and sysb is in reality completely unnecessary. - In short, the sys* stuff is a bit of a mess that makes the project more difficult to understand. - This puts everything down into one folder and has a manifest file that is used to generate the build scripts on the fly rather than using coded scripts. - This is created in the "seed" stage. stage0-posix -- (calls) --> seed -- (generates) --> main steps Alongside this change there are a variety of other smaller fixups to the general structure of the live-bootstrap rootfs. - Creating a rootfs has become much simpler and is defined as code in go.sh. The new structure, for an about-to-be booted system, is / -- /steps (direct copy of steps/) -- /distfiles (direct copy of distfiles/) -- all files from seed/* -- all files from seed/stage0-posix/* - There is no longer such a thing as /usr/include/musl, this didn't really make any sense, as musl is the final libc used. Rather, to separate musl and mes, we have /usr/include/mes, which is much easier to work with. - This also makes mes easier to blow away later. - A few things that weren't properly in packages have been changed; checksum-transcriber, simple-patch, kexec-fiwix have all been given fully qualified package names. - Highly breaking change, scripts now exist in their package directory but NOT WITH THE packagename.sh. Rather, they use pass1.sh, pass2.sh, etc. This avoids manual definition of passes. - Ditto with patches; default directory is patches, but then any patch series specific to a pass are named patches-passX.
2023-11-06 23:51:23 +00:00
mkdir -p distfiles
# First, try to download anything missing - ignore failing mirrors
Remove the notion of "sys*" - This idea originates from very early in the project and was, at the time, a very easy way to categorise things. - Now, it doesn't really make much sense - it is fairly arbitary, often occuring when there is a change in kernel, but not from builder-hex0 to fiwix, and sysb is in reality completely unnecessary. - In short, the sys* stuff is a bit of a mess that makes the project more difficult to understand. - This puts everything down into one folder and has a manifest file that is used to generate the build scripts on the fly rather than using coded scripts. - This is created in the "seed" stage. stage0-posix -- (calls) --> seed -- (generates) --> main steps Alongside this change there are a variety of other smaller fixups to the general structure of the live-bootstrap rootfs. - Creating a rootfs has become much simpler and is defined as code in go.sh. The new structure, for an about-to-be booted system, is / -- /steps (direct copy of steps/) -- /distfiles (direct copy of distfiles/) -- all files from seed/* -- all files from seed/stage0-posix/* - There is no longer such a thing as /usr/include/musl, this didn't really make any sense, as musl is the final libc used. Rather, to separate musl and mes, we have /usr/include/mes, which is much easier to work with. - This also makes mes easier to blow away later. - A few things that weren't properly in packages have been changed; checksum-transcriber, simple-patch, kexec-fiwix have all been given fully qualified package names. - Highly breaking change, scripts now exist in their package directory but NOT WITH THE packagename.sh. Rather, they use pass1.sh, pass2.sh, etc. This avoids manual definition of passes. - Ditto with patches; default directory is patches, but then any patch series specific to a pass are named patches-passX.
2023-11-06 23:51:23 +00:00
for entry in steps/*; do
[ -e "${entry}/sources" ] || continue
# shellcheck disable=SC2162
while read line; do
# This is intentional - we want to split out ${line} into separate arguments.
# shellcheck disable=SC2086
download_source distfiles ${line}
done < "${entry}/sources"
done
# Then, check if everything has been obtained at least once
for entry in steps/*; do
[ -e "${entry}/sources" ] || continue
# shellcheck disable=SC2162
while read line; do
# This is intentional - we want to split out ${line} into separate arguments.
# shellcheck disable=SC2086
check_source distfiles ${line}
done < "${entry}/sources"
done