doc: Release update.

* BLURB: Release update.
* HACKING: Remove roadmap.
* INSTALL: Release update.
* NEWS: Add 0.21 section.
* README: Release update.
* ROADMAP: New file.
* configure.sh: Release update.
* doc/announce/ANNOUNCE-0.21: New file.
* doc/mes.texi: Release update.
This commit is contained in:
Jan Nieuwenhuizen 2019-11-04 19:59:13 +01:00
parent 594b2859b1
commit ac9c18f8b8
No known key found for this signature in database
GPG Key ID: F3C1A0D9C1D65273
13 changed files with 395 additions and 177 deletions

13
BLURB
View File

@ -1,4 +1,7 @@
GNU Mes is a Scheme interpreter and C compiler for bootstrapping the GNU
system.
GNU Mes was created to address the security concerns that arise from
bootstrapping an operating system using binary blobs, which is common
practice for all software distributions. Mes is a Scheme interpreter
@ -8,13 +11,13 @@ halved the size of opaque, uninspectable binaries that were needed to
bootstrap GNU Guix, a functional GNU/Linux distribution that focusses on
user freedom, reproducibility and security. That reduction was achieved
by replacing GNU Binutils, GNU GCC and the GNU C Library with Mes. The
final goal is to help create a full source bootstrap for any interested
UNIX-like operating system. After three years of volunteer work funding
will enable us to make another big step forward and reach an important
new milestone.
final goal is to help create a full source bootstrap for UNIX-like
operating systems. After three years of volunteer work funding will
enable us to make another big step forward and reach an important new
milestone.
GNU Mes - https://gnu.org/software/mes
GNU Guix - https://gnu.org/software/guix
Bootstrappable Builds - https://bootstrappable.org
Reproducible Builds - https://reproducible-builds.org
Reduced Binary Seed bootstrap - http://joyofsource.com/reduced-binary-seed-bootstrap.html
Reduced Binary Seed bootstrap - https://guix.gnu.org/blog/2019/guix-reduces-bootstrap-seed-by-50/

View File

@ -1,22 +1,22 @@
-*- org -*-
#+TITLE: bootstrappable.org project
Copyright © 2016, 2017, 2018 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
Copying and distribution of this file, with or without modification,
are permitted in any medium without royalty provided the copyright
notice and this notice are preserved.
* What?
** Full source bootstrapping for GuixSD
** Full source bootstrapping for GNU Guix
A package in GuixSD is can be uniquely identified by the hash of its
source code, its dependencies and its build recipe.
A package in Guix is uniquely identified by the hash of its source code,
its dependencies, and its build recipe.
Every package can be built from source, except for the bootstrap
binaries.
*** From the GuixSD manual
*** From the Guix manual
The distribution is fully “bootstrapped” and “self-contained”: each
package is built based solely on other packages in the distribution.
@ -25,7 +25,7 @@ The root of this dependency graph is a small set of “bootstrap
binaries”, provided by the (gnu packages bootstrap) module. For more
information on bootstrapping, *note Bootstrapping::.
**** GuixSD bootstrap tarballs
**** Guix bootstrap tarballs
$ du -schx $(readlink $(guix build bootstrap-tarballs)/*)
2.1M /gnu/store/9623n4bq6iq5c8cwwdq99qb7d0xj93ym-binutils-static-stripped-tarball-2.28.1/binutils-static-stripped-2.28.1-x86_64-linux.tar.xz
@ -66,7 +66,7 @@ about the bootstrap binaries and compilers?
** Software: MesCC-tools, M2-Planet, GNU Mes and MesCC
** MesCC-tools
https://github.com/oriansj/mescc-tools
https://savannah.gnu.org/projects/mescc-tools
*** hex.0: ~500 byte well-documented, self-hosting hex assembler
@ -101,13 +101,13 @@ be source.
* TODO
** remove or upstream patches from tcc-boot
** prepare src/mes.c for M2-Planet transpiler.
** arm, the Hurd
** ARM, the Hurd
** fix bootstrap-loops: (Nyacc?, mes.M2, psyntax.pp?)
** make GNU gcc (8.0?) bootstrappable again, remove [need for] tcc stage
* DONE
** upstream mes-boot to GuixSD.
** replace GuixSD bootstrap for x86_64.
** replace GuixSD bootstrap for x86.
** upstream mes-boot to Guix.
** replace Guix bootstrap for x86_64.
** replace Guix bootstrap for x86.
** add full source gcc-4.7 package build.
** bootstrap gcc+glibc-built binutils-20.1, gcc-4.1.0.
** have tcc-boot compile gnutools triplet: binutils-2.14, gcc-2.95.3, glibc-2.2.5.

56
HACKING
View File

@ -60,62 +60,6 @@ we have no guile (GUILE)...but we should have hex2, M1, and mes.M1.
That's a bootstrap problem which is currently ignored by using the
mes-seed package. mes.M1 will be produced by M2-Planet from mes.c.
* ROADMAP
** TODO
*** release 0.x, unsorted
- mes: prepare src/mes.c for M2-Planet transpiler,
Jeremiah branched-out from mes; see https://github.com/oriansj/mes-m2.
- mes/mescc: proper docstrings, api reference documentation.
- replace bootstrap-binaries with Gash: bash, coreutils, grep, gzip,
sed, tar.
- mes: real module support, Guile compatible (define-module,
define-public, export).
- mescc: ARMv7/AArch64 support.
*** after release 1.0
- replace initial gcc-2.95.3 with gcc-3.x or 4.x
- use 3rd party libc (ucLibc, dietlibc, ...) after Mes and reduce
need for bootstrappably-rich Mes C Library?
- mes/mescc: bootstrap a `bootstrap-Guile' before bootstrapping tcc?
- tcc: remove or upstream patches from tcc-boot.
- tcc: build 0.9.27 directly instead of via 0.9.26, see tinycc
wip-bootstrappable@0.9.27 branch
- mes/mescc: bootstrap a minimal-Guile?
+ libguile/{eval,init,list,strings,values,..}.
+ ice-9/eval.scm
- mescc: have mes-tcc pass all scaffold/tests, scaffold/tinycc tests.
- syntax-case bootstrap problem
+ resolve portable syntax-case bootstrap, or
+ get full source syntax-case up (Andre van Tonder?)
https://srfi.schemers.org/srfi-72/srfi-72.html, or
+ ... drop it?
- mescc: the Hurd support.
** DONE
*** 0.19 GNU mes now compiles TinyCC in ~8min and supports building Bash and GNU Tar.
*** 0.18 GNU mes now supports GuixSD bootstrap (x86,x86_64) and has native x86_64 support.
*** 0.17.1 GNU Mes now allows removing glibc, binutils and gcc from the GuixSD bootstrap.
*** 0.17 GNU Mes is now an official GNU project and bootstraps gcc-4.7.4.
*** 0.16.1 Mes now has info docs and installs ootb on Debian buster/testing.
*** 0.16 Mes Lib C now bootstraps glibc-2.2.5, binutils-2.20.1, gcc-4.1.0.
*** 0.15: MesCC now has a libc+gnu that supports compiling binutils-2.14, gcc-2.95.3 and glibc-2.2.5.
*** 0.14: Mes+MesCC now compiles a slightly patched self-hosting tcc.
*** 0.13: Mes+MesCC now compiles a modified, functional tcc.c (~25,000LOC) in 1h30'.
*** 0.12: Mes+MesCC now compiles mes.c (~3000LOC) in ~4min.
*** 0.11: MesCC now compiles a mes-tcc that passes 26/69 of mescc's C tests.
*** 0.10: Mescc now compiles a mes-tcc that compiles a trivial C to a running a.out.
*** 0.9: Mescc now writes M1 macro assembly files and compiles tcc.
*** 0.8: Mescc now writes object files in stage0's labeled hex2 format.
*** 0.7: Mescc supports -E, -c, -o options, include more complete set of header files,
enough to work on compiling tinycc's tcc.c albeit a somewhat modified version.
*** 0.6: Work with unmodified, unbundled Nyacc; compile 33/55 tinycc's tests/test2 suite.
*** 0.5: Mutual self-hosting Scheme interpreter and C compiler: mes.c and mescc,
Support call-with-current-continuation, refactor catch/throw
*** 0.4: Support Nyacc, Gcc-compiled Mes compiles minimal main.c using nyacc
*** 0.3: Garbage collector
*** 0.2: Support psyntax
*** 0.1: Mes eval/apply feature complete; support syntax-rules, compile main.c using LALR, dump ELF
* DEBUG
MES_DEBUG=<level> mes
** Levels

51
INSTALL
View File

@ -1,7 +1,7 @@
-*- org -*-
#+TITLE: Building and Installing GNU Mes
Copyright © 2016, 2017,2018 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
Copying and distribution of this file, with or without modification,
are permitted in any medium without royalty provided the copyright
@ -19,12 +19,14 @@ similar way.
* Regular build
** Prerequisites
*** Guix
guix environment -l guix.scm #64 bit + 32bit
guix environment --system=i686-linux -l guix.scm #32 bit only
#+BEGIN_SRC bash
guix environment -l guix.scm #64 bit + 32bit
guix environment --system=i686-linux -l guix.scm #32 bit only
#+END_SRC
*** Other GNU/Linux
- [[https://github.com/oriansj/mescc-tools][mescc-tools]], 0.5 is known to work.
- [[https://savannah.gnu.org/projects/nyacc][Nyacc]], 0.86.0 is known to work.
- [[https://savannah.gnu.org/projects/mescc-tools/][mescc-tools]], 0.6.1 is known to work.
- [[https://savannah.gnu.org/projects/nyacc][Nyacc]], 0.93.0 is known to work.
- GNU Gcc, 4.9.3 is known to work.
- GNU Guile, version 2.0.12 or is know to work.
- SH, /bin/sh, GNU Bash 4.3 is known to work.
@ -34,37 +36,54 @@ similar way.
- Perl, 5.22 is known to work.
** Build it
#+BEGIN_SRC bash
./configure
make
#+END_SRC
** Check it
#+BEGIN_SRC bash
make check
#+END_SRC
** Install it
#+BEGIN_SRC bash
make install
#+END_SRC
* Guix it
guix package -f guix.scm
#+BEGIN_SRC bash
guix build -f guix.scm
#+END_SRC
* Bootstrap build
The bootstrap build is being developed in Guix, on wip-bootstrap, see
http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/mes.scm?h=wip-bootstrap
The bootstrap build is part of Guix, see
https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/commencement.scm
** Prerequisites
- [[https://github.com/oriansj/mescc-tools][mescc-tools]], 0.5 is known to work.
- [[https://savannah.gnu.org/projects/nyacc][Nyacc]], 0.86.0 is known to work.
- [[https://gitlab.com/janneke/mes-seed][mes-seed]], 0.17.1 is known to work.
- [[https://savannah.gnu.org/projects/mescc-tools/][mescc-tools]], 0.6.1 is known to work.
- [[https://savannah.gnu.org/projects/nyacc][Nyacc]], 0.93.0 is known to work.
** Build it
<setup environment, e.g. prefix, MES_SEED>
./configure.sh [--prefix=<prefix>]
./build.sh
#+BEGIN_SRC bash
sh configure.sh [--prefix=<prefix>]
sh bootstrap.sh
#+END_SRC
** Check it
./check.sh
#+BEGIN_SRC bash
sh check.sh
#+END_SRC
** Install it
./install.sh
#+BEGIN_SRC bash
sh install.sh
#+END_SRC

19
NEWS
View File

@ -2,7 +2,7 @@
#+TITLE: GNU Mes NEWS history of user-visible changes
#+STARTUP: content hidestars
Copyright © 2016,2017,2018 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
Copying and distribution of this file, with or without modification,
are permitted in any medium without royalty provided the copyright
@ -10,6 +10,23 @@ Copyright © 2016,2017,2018 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
Please send Mes bug reports to bug-mes@gnu.org.
* Changes in 0.21 since 0.20
** Core
*** Mes can now be bootstrapped with Gash and Gash Core Utils.
*** Mes now supports a Scheme-only bootstrap.
*** Mes now supports -c EXPR.
** MesCC
*** Mes C Library now supports bootstrapping GNU Awk, GNU Bash, GNU SED, and GNU Tar.
*** Mes C Library now has limited float support in vfprintf, vsnprintf, vsscanf.
**** 7 new functions
abtod, atof, creat, dtoab, execlp, isgraph, mknod, readlink, strtod,
symblink.
**** 5 new stubs
getgrgid, getgrnam, getpgid, getpgrp, mktime, setgrent.
** Noteworthy bug fixes
*** A bug with `mes -c EXPR' has been fixed.
*** The REPL now works again on x86_64.
*** --with-system-libc now works again.
* Changes in 0.20 since 0.19
** Core
*** The build system has been simplified, again.

45
README
View File

@ -3,21 +3,21 @@
#+SUBTITLE: Maxwell Equations of Software
[[https://www.gnu.org/software/mes][GNU Mes]] brings a [[http://joyofsource.com/reduced-binary-seed-bootstrap.html][Reduced Binary Seed bootstrap]] to [[https://www.gnu.org/software/guix][GNU Guix]]. This
bootstrap has halved the size of opaque, uninspectable binaries that
were needed to bootstrap Guix. The final goal is to help create a full
source bootstrap as part of the [[http://bootstrappable.org][bootstrappable builds]] effort for any
interested UNIX-like operating system.
[[https://www.gnu.org/software/mes][GNU Mes]] is a Scheme interpreter and C compiler for bootstrapping the GNU
system. Since version 0.20 it brings a [[https://guix.gnu.org/blog/2019/guix-reduces-bootstrap-seed-by-50/][Reduced Binary Seed bootstrap]] to
[[https://www.gnu.org/software/guix][GNU Guix]]. This bootstrap has halved the size of opaque, uninspectable
binaries that were needed to bootstrap Guix 1.0. The final goal is to
help create a full source bootstrap as part of the [[http://bootstrappable.org][bootstrappable builds]]
effort for UNIX-like operating systems.
Mes consists of a mutual self-hosting Scheme interpreter written in
~5,000 LOC of simple C and a Nyacc-based C compiler written in Scheme.
This mes.c is [[https://github.com/oriansj/mes-m2][being simplified]] to be transpiled by [[https://github.com/oriansj/m2-planet][M2-Planet]].
~5,000 LOC of simple C, and a C compiler written in Scheme. This mes.c
is [[https://github.com/oriansj/mes-m2][being simplified]] to be transpiled by [[https://github.com/oriansj/m2-planet][M2-Planet]].
The Scheme interpreter (mes.c) has a Garbage Collector, a library of
loadable Scheme modules-- notably Dominique Boucher's [[https://github.com/schemeway/lalr-scm][LALR]], Pre-R6RS
[[https://www.cs.indiana.edu/chezscheme/syntax-case/old-psyntax.html][portable syntax-case]] with R7RS ellipsis, Matt Wette's [[https://www.nongnu.org/nyacc][Nyacc]] --and test
suite just barely enough to support a simple REPL and simple
C-compiler: MesCC.
The Scheme interpreter has a Garbage Collector, a library of loadable
Scheme modules-- notably Dominique Boucher's [[https://github.com/schemeway/lalr-scm][LALR]], Pre-R6RS [[https://www.cs.indiana.edu/chezscheme/syntax-case/old-psyntax.html][portable
syntax-case]] with R7RS ellipsis, Matt Wette's [[https://www.nongnu.org/nyacc][Nyacc]] --and test suite just
enough to support a REPL and a C99 compiler: MesCC.
Mes+MesCC can compile an only [[http://gitlab.com/janneke/tinycc][lightly patched TinyCC]] that is
self-hosting. Using this tcc and the Mes C library we now have a
@ -64,16 +64,17 @@ General Public Licence version 3 or later. See the file [[file:COPYING][COPYING
* Links
[0] https://www.gnu.org/software/mes
[1] https://www.gnu.org/software/guix
[2] http://bootstrappable.org
[3] https://github.com/oriansj/mes-m2
[4] https://github.com/oriansj/m2-planet
[5] https://github.com/schemeway/lalr-scm
[6] https://www.cs.indiana.edu/chezscheme/syntax-case/old-psyntax.html
[7] https://www.nongnu.org/nyacc
[8] http://gitlab.com/janneke/tinycc
[9] [[http://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf]]
[10] https://github.com/oriansj/stage0
[1] https://guix.gnu.org/blog/2019/guix-reduces-bootstrap-seed-by-50/
[2] https://www.gnu.org/software/guix
[3] https://bootstrappable.org
[4] https://github.com/oriansj/mes-m2
[5] https://github.com/oriansj/m2-planet
[6] https://github.com/schemeway/lalr-scm
[7] https://www.cs.indiana.edu/chezscheme/syntax-case/old-psyntax.html
[8] https://www.nongnu.org/nyacc
[9] https://gitlab.com/janneke/tinycc
[10] [[http://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf]]
[11] https://github.com/oriansj/stage0
* Legalese
Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <[[mailto:janneke@gnu.org][janneke@gnu.org]]>

91
ROADMAP Normal file
View File

@ -0,0 +1,91 @@
-*- org -*-
#+TITLE: Tentative GNU Mes Road Map
Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
Copying and distribution of this file, with or without modification,
are permitted in any medium without royalty provided the copyright
notice and this notice are preserved.
The goals of the GNU Mes project are two-fold:
- to bootstrap the GNU system purely from source, without using any binary
seeds.
- to aid the Reproducible-Builds.org effort; demonstrate the impact and
importance of GNU and Software Freedom.
Since its inception, the project has gone a long way towards that goal. Below
is a list of items we want for version "1.0" of GNU Mes.
You're welcome to discuss this road map on bug-mes@gnu.org or #bootstrappable
on Freenode!
* TODO
** release 0.x, unsorted
- Introduce the Reduced Binaries Seed bootstrap to NixOS (Debian,
Gentoo, ...?)
- Full Source Bootstrap: compile Mes.c using M2-Planet. Jeremiah
branched-out from mes; see https://github.com/oriansj/mes-m2, also see the
branches: wip-m2-merge and wip-m2.
- Scheme-only bootstrap bootstrap-binaries with Gash and Gash Core
Utilities: Awk, Bash, Core Utilities, Grep, Gzip, Make, Sed, Tar.
- Full Guile-compatible module support: define-module, define-public,
export, use-modules, ...
- ARMv7/AArch64 support: Mes C Library and MesCC.
- Simplify eval-apply and other core functionality (see mes-m2).
** after release 1.0
- Replace the initial gcc-2.95.3 with gcc-4.6.4.
- tcc: remove or upstream patches from tcc-boot.
- tcc: build 0.9.27 directly instead of via 0.9.26, see tinycc
wip-bootstrappable@0.9.27 branch
- Bootstrap a `bootstrap-Guile' before bootstrapping tcc?
- Skip tcc: Build gcc using MesCC.
- better garbage collector.
- mes/mescc: proper docstrings, api reference documentation.
- mes/mescc: bootstrap a minimal Guile?
+ libguile/{eval,init,list,strings,values,..}.
+ ice-9/eval.scm
- syntax-case bootstrap problem
+ resolve portable syntax-case bootstrap, or
+ get full source syntax-case up (Andre van Tonder?)
https://srfi.schemers.org/srfi-72/srfi-72.html, or
+ ... drop it?
- Support the Hurd. There is a wip-hurd branch; most needed now are fork
and exec.
* DONE
- 0.21 GNU Mes now supports a Scheme-only bootstrap and is packaged in
Debian GNU/Linux.
- 0.20 GNU Mes brings the Reduced Binary Source bootstrap to Guix.
- 0.19 GNU Mes now compiles TinyCC in ~8min and supports building Bash and
GNU Tar.
- 0.18 GNU Mes now supports GuixSD bootstrap (x86,x86_64) and has native
x86_64 support.
- 0.17.1 GNU Mes now allows removing glibc, binutils and gcc from the GuixSD
bootstrap.
- 0.17 GNU Mes is now an official GNU project and bootstraps gcc-4.7.4.
- 0.16.1 Mes now has info docs and installs ootb on Debian buster/testing.
- 0.16 Mes Lib C now bootstraps glibc-2.2.5, binutils-2.20.1, gcc-4.1.0.
- 0.15: MesCC now has a libc+gnu that supports compiling binutils-2.14,
gcc-2.95.3 and glibc-2.2.5.
- 0.14: Mes+MesCC now compiles a slightly patched self-hosting tcc.
- 0.13: Mes+MesCC now compiles a modified, functional tcc.c (~25,000LOC) in
1h30'.
- 0.12: Mes+MesCC now compiles mes.c (~3000LOC) in ~4min.
- 0.11: MesCC now compiles a mes-tcc that passes 26/69 of mescc's C tests.
- 0.10: Mescc now compiles a mes-tcc that compiles a trivial C to a running
a.out.
- 0.9: Mescc now writes M1 macro assembly files and compiles tcc.
- 0.8: Mescc now writes object files in stage0's labeled hex2 format.
- 0.7: Mescc supports -E, -c, -o options, include more complete set of
header files, enough to work on compiling tinycc's tcc.c albeit a somewhat
modified version.
- 0.6: Work with unmodified, unbundled Nyacc; compile 33/55 tinycc's
tests/test2 suite.
- 0.5: Mutual self-hosting Scheme interpreter and C compiler: mes.c and
mescc, Support call-with-current-continuation, refactor catch/throw
- 0.4: Support Nyacc, Gcc-compiled Mes compiles minimal main.c using nyacc
- 0.3: Garbage collector
- 0.2: Support psyntax
- 0.1: Mes eval/apply feature complete; support syntax-rules, compile main.c
using LALR, dump ELF

View File

@ -67,9 +67,9 @@ cp\
BOOTSTRAP\
COPYING\
HACKING\
INSTALL\
NEWS\
README\
ROADMAP\
${DESTDIR}${docdir}
if test -f ChangeLog+; then

View File

@ -254,7 +254,7 @@ GNU Mes is configured for
system: $mes_system
Run:
./build.sh to build mes
./check.sh to check mes
./install.sh to install mes
sh build.sh to build mes
sh check.sh to check mes
sh install.sh to install mes
EOF

View File

@ -129,5 +129,5 @@ janneke and Danny.
[8] https://www.nongnu.org/nyacc
[9] https://gitlab.com/janneke/tinycc
[10] http://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf
[11] https://github.com/oriansj/stage0
[11] https://savannah.nongnu.org/projects/stage0
[12] https://nlnet.nl/project/GNUMes

132
doc/announce/ANNOUNCE-0.21 Normal file
View File

@ -0,0 +1,132 @@
Subject: GNU Mes 0.21 released
<#secure method=pgpmime mode=sign>
We are pleased to announce the release of GNU Mes 0.21, representing
54 commits over 10 weeks.
Mes has now brought the Reduced Binary Seed bootstrap to Guix (bootstrap
a GNU/Linux system without binary GNU toolchain or equivalent). See
https://guix.gnu.org/blog/2019/guix-reduces-bootstrap-seed-by-50/
This release supports a Scheme-only bootstrap: Mes can now be built with
Gash and the experimental Gash Core Utils instead of using GNU Awk, GNU
Bash, the GNU Core Utilities, GNU Grep, GNU Gzip, GNU Make, GNU Sed, and
GNU Tar. Also, the Mes C Library now supports bootstrapping those.
Finally, this release brings Mes as a package to Debian GNU/Linux.
We are excited that the Nlnet Foundation[12] is now sponsoring this
work!
Next targets:
- Introduce the Reduced Binaries Seed bootstrap to NixOS (Debian,
Gentoo, ...?)
- Scheme-only bootstrap: use Guile, Gash and Gash Core Utils to remove
awk, bash, core utilities, grep, gzip, make, sed, tar, etc. from the
Guix bootstrap binaries
- ARM support
- Full Source Bootstrap: compile Mes.c using M2-Planet
- Reduced Binary Seed bootstrap for ARM
- the Hurd
Packages are available in Guix master.
* About
GNU Mes[0] brings a Reduced Binary Seed bootstrap[1] to GNU Guix[2].
This bootstrap has halved the size of opaque, uninspectable binaries
that were needed to bootstrap Guix. The final goal is to help create
a full source bootstrap as part of the bootstrappable builds[3] effort
for any interested UNIX-like operating system.
It consists of a mutual self-hosting Scheme interpreter written in
~5,000 LOC of simple C and a Nyacc-based C compiler written in Scheme.
This mes.c is being simplified[4] to be transpiled by M2-Planet[5].
The Scheme interpreter has a Garbage Collector, a library of loadable
Scheme modules-- notably Dominique Boucher's LALR[6], Pre-R6RS
[portable syntax-case[7] with R7RS ellipsis, Matt Wette's Nyacc[8]
--and test suite just enough to support a REPL and a C99 compiler:
MesCC.
Mes+MesCC can compile an only lightly patched TinyCC[9] that is
self-hosting. Using this tcc and the Mes C library we now have a
Reduced Binary Seed bootstrap for the gnutools triplet: glibc-2.2.5,
binutils-2.20.1, gcc-2.95.3. This is enough to bootstrap Guix for
i686-linux and x86_64-linux.
Mes is inspired by The Maxwell Equations of Software: LISP-1.5[10] -- John
McCarthy page 13, GNU Guix's source/binary packaging transparency and
Jeremiah Orians's stage0[11] ~500 byte self-hosting hex assembler.
* Download
git clone git://git.savannah.gnu.org/mes.git
Here are the compressed sources and a GPG detached signature[*]:
https://ftp.gnu.org/gnu/mes/mes-0.21.tar.gz
https://ftp.gnu.org/gnu/mes/mes-0.21.tar.gz.sig
Use a mirror for higher download bandwidth:
https://ftpmirror.gnu.org/mes/mes-0.21.tar.gz
https://ftpmirror.gnu.org/mes/mes-0.21.tar.gz.sig
Here are the MD5 and SHA1 checksums:
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx mes-0.21.tar.gz
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx mes-0.21.tar.gz
[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact. First, be sure to download both the .sig file
and the corresponding tarball. Then, run a command like this:
gpg --verify mes-0.21.tar.gz.sig
If that command fails because you don't have the required public key,
then run this command to import it:
gpg --keyserver keys.gnupg.net --recv-keys 1A858392E331EAFDB8C27FFBF3C1A0D9C1D65273
and rerun the 'gpg --verify' command.
* Get informed, get involved
See https://bootstrappable.org
Join #bootstrappable on irc.freenode.net.
* Changes in 0.21 since 0.20
** Core
*** Mes can now be bootstrapped with Gash and Gash Core Utils.
*** Mes now supports a Scheme-only bootstrap.
*** Mes now supports -c EXPR.
** MesCC
*** Mes C Library now supports bootstrapping GNU Awk, GNU Bash, GNU SED, and GNU Tar.
*** Mes C Library now has limited float support in vfprintf, vsnprintf, vsscanf.
**** 7 new functions
abtod, atof, creat, dtoab, execlp, isgraph, mknod, readlink, strtod,
symblink.
**** 5 new stubs
getgrgid, getgrnam, getpgid, getpgrp, mktime, setgrent.
** Noteworthy bug fixes
*** A bug with `mes -c EXPR' has been fixed.
*** The REPL now works again on x86_64.
*** --with-system-libc now works again.
Greetings,
janneke and Danny.
[0] https://www.gnu.org/software/mes
[1] https://guix.gnu.org/blog/2019/guix-reduces-bootstrap-seed-by-50/
[2] https://www.gnu.org/software/guix
[3] https://bootstrappable.org
[4] https://github.com/oriansj/mes-m2
[5] https://github.com/oriansj/m2-planet
[6] https://github.com/schemeway/lalr-scm
[7] https://www.cs.indiana.edu/chezscheme/syntax-case/old-psyntax.html
[8] https://www.nongnu.org/nyacc
[9] https://gitlab.com/janneke/tinycc
[10] http://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf
[11] https://savannah.nongnu.org/projects/stage0
[12] https://nlnet.nl/project/GNUMes

View File

@ -13,7 +13,7 @@
@set OPENPGP-SIGNING-KEY-ID 1A858392E331EAFDB8C27FFBF3C1A0D9C1D65273
@copying
Copyright @copyright{} 2018 Jan (janneke) Nieuwenhuizen@*
Copyright @copyright{} 2018,2019 Jan (janneke) Nieuwenhuizen@*
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
@ -34,7 +34,7 @@ Documentation License''.
@titlepage
@title GNU Mes Reference Manual
@subtitle Full Source Bootstrapping of the GNU Guix Operating System
@subtitle Full Source Bootstrapping for the GNU system
@author Jan (janneke) Nieuwenhuizen
@page
@ -124,8 +124,9 @@ that we can trust.
Mes consists of a mutual self-hosting Scheme interpreter written in C
and a Nyacc-based (see @pxref{NYACC User's Guide,,, nyacc-ug, NYACC
User's Guide}) C compiler written in Scheme. The Scheme interpreter
@file{mes.c} is about 5000LOC of restricted C and intended to be
bootstrappable using a very simple C compiler.
@file{mes.c} is about 5,000LOC of restricted C, to be compiled with
M2-Planet@footnote{See @url{https://github.com/oriansj/m2-planet}}, a
very simple C compiler.
If we want to trust our computers to do what we instructed them to do
then we need to be able to inspect all instructions---all
@ -255,10 +256,10 @@ binary seed that all software distributions inject are the so called
@emph{bootstrap binaries}. Bootstrap binaries are the initial binary
seeds that are used to start building the distribution.
The GNU Guix operating system has a relatively small closure of
bootstrap binaries: GNU binutils, GNU gcc, GNU Libc, GNU Guile, and
``Static binaries'' (think: bash, bzip2, coreutils, gawk, grep, gzip,
patch, sed, tar, xz).
The GNU Guix operating system, version 1.0 had a relatively small
closure of bootstrap binaries: GNU binutils, GNU gcc, GNU Libc, GNU
Guile, and ``Static binaries'' (think: bash, bzip2, coreutils, gawk,
grep, gzip, patch, sed, tar, xz).
@example
$ du -schx $(readlink $(guix build bootstrap-tarballs)/*)
@ -270,8 +271,8 @@ $ du -schx $(readlink $(guix build bootstrap-tarballs)/*)
33M total
@end example
only a 33MB download that unpacks to 252BM of opaque binaries, that we
most probably have the source of, shall we review these together? ;-)
only a 33MB download that unpacks to a 252MB @emph{seed} of opaque
binary code.
@example
$ for i in $(readlink $(guix build bootstrap-tarballs)/*);\
@ -285,6 +286,13 @@ $ du -schx *
252M total
@end example
During the Guix 1.1 development series we managed to create the first
reduction by 50% of the Guix @emph{bootstrap binaries}@footnote{See
@url{https://guix.gnu.org/blog/2019/guix-reduces-bootstrap-seed-by-50/}}.
This was a very important step because the ~250MB @emph{seed} of binary
code was practically non-auditable, which makes it hard to establish
what source code produced them.
@node Full Source Bootstrap
@section Full Source Bootstrap
@ -433,7 +441,7 @@ enjoy software Freedom 1.
@cindex installing Mes
Mes is available for download from its website at
@url{http://www.gnu.org/pub/gnu/mes/}. This section describes the
@url{https://www.gnu.org/pub/gnu/mes/}. This section describes the
software requirements of Mes, as well as how to install it and get ready
to use it.
@ -458,21 +466,14 @@ GNU Mes depends on the following packages:
@item @url{http://gnu.org/software/guile/, GNU Guile}, version 2.0.13 or
later, including 2.2.x;
@item @url{http://www.gnu.org/software/make/, GNU Make}.
@item @url{https://savannah.gnu.org/projects/nyacc/, NYACC}, 0.86.0 is known to work.
@item @url{https://savannah.gnu.org/projects/nyacc/, NYACC}, 0.93.0 or later, including 0.99.0.
@item @url{http://gcc.gnu.org, GCC's gcc}, version 2.95.3 or later.
@end itemize
The following dependencies are optional:
@itemize
@item
Installing @url{https://github.com/oriansj/mescc-tools/, mescc-tools},
version 0.5 or later, will allow you to have MesCC assemble and link.
@item @url{https://savannah.gnu.org/projects/mescc-tools/, mescc-tools}, version 0.6.1 or later,
@end itemize
@cindex Guile, compatibility
Mes is compatible with GNU Guile, so it is possible to share the same
Scheme code between both. Of course, Mes only supports the minimal
Scheme code between both. Currently Mes only supports the minimal
subset of R5RS and Guile extensions to run MesCC.
@node Bootstrap Requirements
@ -483,23 +484,21 @@ package. The bootstrap build procedure for Mes is similar to building
GNU software and goes like this
@example
export prefix=/usr/local # for example
export MES_SEED=../mes # for example
# optionally set some other environment variables
sh build.sh
sh configure.sh --prefix=/your/prefix/here
sh bootstrap.sh
sh check.sh
sh install.sh
@end example
See @file{build.sh} for inspiration on what environment variables to
set.
See @file{configure.sh} and @file{bootstrap.sh} for inspiration on what
environment variables to set.
Bootstrapping Mes depends on the following packages:
@itemize
@item a POSIX-compatible shell
@item @url{https://github.com/oriansj/mescc-tools/, mescc-tools}, version 0.5 or later.
@item @url{http://gitlab.com/janneke/mes-seed/, mes-seed}, version 0.17 or later.
@item @url{https://github.com/oriansj/mescc-tools/, mescc-tools}, version 0.6.1 or later.
@item @url{https://savannah.gnu.org/projects/nyacc/, NYACC}, 0.93.0 or later, including 0.99.0.
@end itemize
@node Running the Test Suites
@ -593,34 +592,51 @@ responsibility.
@node The Mes Bootstrap Process
@section The Mes Bootstrap Process
The Mes full source bootstrap process@footnote{TBH, the current state of
affairs demands to label this a `Reduced Binary Seed bootstrap'} is
currently being developed in Guix@footnote{See
@file{gnu/packages/commencement.scm} in the @var{core-updates} branch in Guix
The Reduced Binary Seed bootstrap currently adopted by Guix@footnote{See
@file{gnu/packages/commencement.scm} in the @var{master} branch in Guix
git
@url{http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/mes.scm?h=wip-bootstrap}}.
In it's intiial form it is only available for x86-linux.
@url{http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/commencement.scm}}.
In its intiial form it is only available for x86-linux.
Currently, it goes like this:
@verbatim
mescc-tools-source + mescc-tools-seed => mescc-tools
mes-source + mescc-tools + mescc-seed => mes
tcc-source + mes + tinycc-seed => tcc
binutils-source + mes + tcc => binutils0
gcc-source + tcc + binutils0 => gcc-core
glibc-source + kernel-headers-source + binutils0 + gcc => glibc0
binutils-source + binutils0 + gcc + glibc => binutils
gcc-source + binutils + gcc-core + glibc => gcc
glibc-source + binutils + gcc + glibc0 => glibc
gcc-mesboot (4.9.4)
^
|
glibc-mesboot (2.16.0)
^
|
gcc-mesboot1 (4.7.4)
^
|
binutils-mesboot (2.20.1a)
^
|
gcc-mesboot0 (2.95.3)
^
|
glibc-mesboot0 (2.2.5)
^
|
gcc-core-mesboot (2.95.3)
^
|
make-mesboot0, diffutils-mesboot, binutils-mesboot0 (2.20.1a)
^
|
tcc-boot
^
|
tcc-boot0
^
|
mes-boot
^
|
*
bootstrap-mescc-tools, bootstrap-mes (~10MB)
bootstrap-bash, bootstrap-coreutils&co, bootstrap-guile (~120MB)
@end verbatim
@c This graph is generated from wip-bootstrap, doing:
@ -636,14 +652,14 @@ that builds the rest of Guix.
Work is ongoing to remove these binary seeds that were intentionally
injected by our own doing as temporary shortcut
@example
mescc-tools-seed, mes-seed, tinycc-seed
bootstrap-mescc-tools (seed), bootstrap-mes (seed)
@end example
For now, these additional non-bootstrapped dependencies (i.e., binary
seeds) are taken for granted
@example
BOOTSTRAP-GUILE, bash, bzip2, coreutils, gawk, grep, gzip, patch, sed,
bootstrap-guile, bash, bzip2, coreutils, gawk, grep, gzip, patch, sed,
tar, xz
@end example
@ -659,7 +675,7 @@ The @command{mes} command is the Scheme interpreter whose prime
directive is to run the @command{MesCC} program.
For convenience and testing purposes, @command{mes} tries to mimic
Guile.
guile.
@example
mes @var{option}@dots{} @file{FILE}@dots{}
@ -704,9 +720,6 @@ variable.
@item -C,--compiled-path=@var{directory}
Accepted and ignored for Guile compatibility.
@item ---dump
dump binary program to stdout
@item -l @var{file}
Load Scheme source code from @var{file}, and continue processing the
command line.
@ -721,9 +734,6 @@ and the command-line arguments---the list provided by the
@item -h@r{, }--help
Display help on invoking Mes, and then exit.
@item ---load
load binary program [module/mes/boot-0.32-mo]
@item -v@r{, }--version
Display the current version of Mes, and then exit.

View File

@ -30,6 +30,7 @@
(define welcome
(string-append "GNU Mes " %version "
Copyright (C) 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
Copyright (C) 2019 Danny Milosavljevic <dannym@scratchpost.org>
GNU Mes comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it