diff --git a/AUTHORS b/AUTHORS index 4e716b8f..ef97cc39 100644 --- a/AUTHORS +++ b/AUTHORS @@ -18,6 +18,13 @@ scaffold/tests/98-fopen.c Han-Wen Nienhuys lib/string/memmem.c (_memmem, memmem) +Peter de Wachter +Small fixes and additions to +lib/x86-mes/x86.M1 +lib/x86_64-mes/x86_64.M1 +include/stdint.h +module/mescc/compile.mes + rain1 scaffold/tests/90-goto-var.c scaffold/tests/91-goto-array.c diff --git a/HACKING b/HACKING index a6b18d9f..047e2e3a 100644 --- a/HACKING +++ b/HACKING @@ -1,7 +1,7 @@ -*- org -*- #+TITLE: Hacking GNU Mes -Copyright © 2016, 2017,2018 Jan (janneke) Nieuwenhuizen +Copyright © 2016,2017,2018 Jan (janneke) Nieuwenhuizen Copying and distribution of this file, with or without modification, are permitted in any medium without royalty provided the copyright @@ -26,9 +26,7 @@ development. ** DEVELOPMENT BUILD To help development we assume ./configure sets these variables for make - CC -- gcc - CC32 -- i686-unknown-linux-gnu-gcc (or on x86, also gcc) - CC64 -- gcc for use with Mes C Library: -nostdinc, -nostdlib + CC -- gcc (or i686-unknown-linux-gnu-gcc sans libc) GUILE -- guile HEX2 -- hex2 MES -- unset @@ -36,65 +34,64 @@ To help development we assume ./configure sets these variables for make prefix -- "" Mes is supposed to serve as a full equivalent for Guile, however Mes -~30 times slower than Guile. That's why we usually don't use Mes -during development. +is still about 2 to 10 times slower than Guile. That's why we usually +don't use Mes during development. Gcc is used to verify the sanity of our C sources. i686-unknown-linux-gnu-gcc is used to compare hex/assembly, to test the gcc variant of Mes C Libirary. -Target prefix: mes-gcc. +Target prefix: x86-mes-gcc. -gcc (CC64) -nostdinc,-nostdlib is used to compare hex/assembly, to test -the 64bit variant of Mes C Library. -Target prefix: x86_64-mes-gcc. +gcc -nostdinc,-nostdlib is used to compare hex/assembly, to test the +64bit variant of Mes C Library. Target prefix: x86_64-mes-gcc. Guile is used to develop MesCC, the C compiler in Scheme that during bootstrapping will be executed by Mes. -mes is built from src/*.c and some out/src/*.h files that are snarfed from -src/*.c by build-aux/mes-snarf.scm. - -Running ./make.scm produces a `script' file. - ** BOOTSTRAP BUILD + ./configure.sh [--prefix=PREFIX] ./build.sh + ./install.sh -In bootstrap mode, we don't have gcc (CC), we don't have a 32 bit gcc -(CC32), we have no guile (GUILE)...but we should have hex2, M1, and -mes.M1. That's a bootstrap problem which is currently ignored by -using the mes-seed package. mes.M1 will be produced by M2-Planet from -mes.c. +In bootstrap mode, we don't have gcc (CC), we don't have a 32 bit gcc, +we have no guile (GUILE)...but we should have hex2, M1, and mes.M1. +That's a bootstrap problem which is currently ignored by using the +mes-seed package. mes.M1 will be produced by M2-Planet from mes.c. * ROADMAP ** TODO *** release 0.x, unsorted - - replace bootstrap utils with Gash: bash, coreutils, grep, gzip, + - mes: prepare src/mes.c for M2-Planet transpiler, + Jeremiah branched-out from mes; see https://github.com/oriansj/mes-m2. + - mes/mescc: proper docstrings, api reference documentation. + - replace bootstrap-binaries with Gash: bash, coreutils, grep, gzip, sed, tar. + - mes: real module support, Guile compatible (define-module, + define-public, export). + - mescc: ARMv7/AArch64 support. +*** after release 1.0 + - replace initial gcc-2.95.3 with gcc-3.x or 4.x + - use 3rd party libc (ucLibc, dietlibc, ...) after Mes and reduce + need for bootstrappably-rich Mes C Library? + - mes/mescc: bootstrap a `bootstrap-Guile' before bootstrapping tcc? - tcc: remove or upstream patches from tcc-boot. - tcc: build 0.9.27 directly instead of via 0.9.26, see tinycc wip-bootstrappable@0.9.27 branch - - mes: performance! - - mes: prepare src/mes.c for M2-Planet transpiler, - Jeremiah branched-out from mes; see https://github.com/oriansj/mes-m2. - - mes: real module support, bonus for supporting Guile's define-module/define-public syntax. - mes/mescc: bootstrap a minimal-Guile? - + libguile/{eval,init,list,strings,values,..}.c + + libguile/{eval,init,list,strings,values,..}. + ice-9/eval.scm - mescc: have mes-tcc pass all scaffold/tests, scaffold/tinycc tests. - - mescc: support long long? - - mescc: full support for floats? - - mescc: some success with 8cc,pcc,guile/libguile/eval.c? - - build: guile/guix/make.scm: add file-types, intermediate, hash all dependencies - - get full source syntax-case up (Andre van Tonder?) OR drop it. - https://srfi.schemers.org/srfi-72/srfi-72.html - psyntax/syntax-case and rewrite Nyacc without syntax-case+R7RS Ellipsis. - - mescc: split-off MesCC as as standalone Guile C compiler project. -*** release 1.0 - - more architectures (does GuixSD require this, i.e. before 1.0?). + - syntax-case bootstrap problem + + resolve portable syntax-case bootstrap, or + + get full source syntax-case up (Andre van Tonder?) + https://srfi.schemers.org/srfi-72/srfi-72.html, or + + ... drop it? + - mescc: the Hurd support. ** DONE +*** 0.19 GNU mes now compiles TinyCC in ~8min and supports building Bash and GNU Tar. *** 0.18 GNU mes now supports GuixSD bootstrap (x86,x86_64) and has native x86_64 support. *** 0.17.1 GNU Mes now allows removing glibc, binutils and gcc from the GuixSD bootstrap. *** 0.17 GNU Mes is now an official GNU project and bootstraps gcc-4.7.4. @@ -139,6 +136,9 @@ enough to work on compiling tinycc's tcc.c albeit a somewhat modified version. * Bugs +** mes: performance, Mes is now 2-10x slower than Guile. +** mes/mescc lack support for the Hurd. +** mes: gcc-x86_64 compiled mes segfaults with small arena, or gc_up_arena. ** mes: gcc-x86 compiled, tests/srfi-13.test number->string INT-MIN fails: test: number->string INT-MIN: fail expected: -2147483648 @@ -149,7 +149,6 @@ actual: -./,),(-*,( ** mes+mescc: parse tcc.c->tcc.E works, compile tcc.E -> tcc.M1 segfaults. time GUILE_LOAD_PATH=/home/janneke/src/nyacc/module:$GUILE_LOAD_PATH ../mes/scripts/mescc -E -o tcc.E -I . -I ../mes/lib -I ../mes/include -D 'CONFIG_TCCDIR="usr/lib/tcc"' -D 'CONFIG_TCC_CRTPREFIX="usr/lib:{B}/lib:."' -D 'CONFIG_TCC_ELFINTERP="/gnu/store/70jxsnpffkl7fdb7qv398n8yi1a3w5nx-glibc-2.26.105-g0890d5379c/lib/ld-linux.so.2"' -D 'CONFIG_TCC_LIBPATHS="/home/janneke/src/tinycc/usr/lib:{B}/lib:."' -D 'CONFIG_TCC_SYSINCLUDEPATHS="../mes/include:usr/include:{B}/include"' -D CONFIG_USE_LIBGCC=1 -D 'TCC_LIBGCC="/home/janneke/src/tinycc/usr/lib/libc+tcc-gcc.mlibc-o"' -D CONFIG_TCC_STATIC=1 -D ONE_SOURCE=yes -D TCC_TARGET_I386=1 -D BOOTSTRAP=1 tcc.c time GUILE_LOAD_PATH=/home/janneke/src/nyacc/module:$GUILE_LOAD_PATH MES_ARENA=200000000 ../mes/scripts/mescc -c -o tcc.M1 tcc.E -** mes: with-fluids: tests/fluids.test test 7 fails with Mes. ** mescc: 7n-struct-struct-array.c: struct file f = {"first.h"}; ** test/match.test ("nyacc-simple"): hygiene problem in match * OLD: Booting from LISP-1.5 into Mes diff --git a/NEWS b/NEWS index 8735c97e..c3a536e5 100644 --- a/NEWS +++ b/NEWS @@ -10,6 +10,48 @@ Copyright © 2016,2017,2018 Jan (janneke) Nieuwenhuizen Please send Mes bug reports to bug-mes@gnu.org. +* Changes in 0.19 since 0.18 +** Core +*** The build system has been simplified. +*** Mes now prints a backtrace upon error. +*** Performance has been improved 2-8 times, making Mes 2-10 times slower than Guile. +*** Mes now supports a module type and uses a `boot-module'. +*** Mes now supports a hash_table type. +*** Mes now supports a struct type. +*** Mes now supports building a %bootstrap-mes seed from Guix. +** Language +*** Records are now implemented using struct (WAS: vector). +*** 44 new functions +ceil, char-downcase, char-set-adjoin, char-set-complement, +char-upcase, current-time, delete-file, dup, dup2, file-exists?, +floor, frame-printer, get-internal-run-time, getcwd, gettimeofday, +hash, hash-ref, hash-set!, hash-table-printer, hashq, +hashq-get-handle, hashq-ref, hashq-set, inexact->exact, +make-hash-table, make-stack, make-struct, module-define!, +module-printer, module-ref, module-variable, read-line, round, +stack-length, stack-ref, string-downcase, string-tokenize, +string-upcase, struct-length, struct-ref, struct-set! struct-vtable, +struct-vtable, with-error-to-file. +** MesCC +*** Assembly defines have been cleaned-up: duplicates deleted, missing added, wrong fixed. +*** MesCC now supports compiling GNU Bash and GNU Tar. +**** 6 New functions +getegid, geteuid, getppid, setgid, setuid, sigdelset, sigprocmask. +**** 22 New macros +EACCES, ENOSPC, ESPIPE, INT16_MAX, INT16_MIN, INT32_MAX, INT32_MIN, +INT64_MAX, INT64_MIN, INT8_MAX, INT8_MIN, LLONG_MAX, LLONG_MIN, +SIZE_MAX SYS_getegid, SYS_geteuid, SYS_setgid SYS_setuid, S_IRGRP, +S_IROTH, S_IRWXG, S_IRWXO S_ISGID, S_ISUID, S_IWGRP, S_IWOTH, S_IXGRP, +S_IXOTH, UINT16_MAX, UINT32_MAX, UINT64_MAX, UINT8_MAX, +_POSIX_VERSION. +** Noteworthy bug fixes +*** Mes now supports characters #\xNN. +*** Mes now supports assq-ref and assoc-ref with alist == #f. +*** Mes now support \xNN in strings. Fixes using Nyacc-0.86.0. +*** MesCC now supports the unary plus operator. +*** MesCC now supports the `U' integer suffix. +*** MesCC now comes with INTnn_MIN/MAX, UINTnn defines in stdint.h. +*** MesCC now always exits non-zero when assembler or linker fail. * Changes in 0.18 since 0.17.1 ** Core *** Mes/MesCC now supports x86_64. diff --git a/README b/README index e952479f..62a08b25 100644 --- a/README +++ b/README @@ -3,10 +3,10 @@ #+SUBTITLE: Maxwell Equations of Software -[[https://www.gnu.org/software/mes][GNU Mes]] brings a [[http://joyofsource.com/reduced-binary-seed-bootstrap.html][Reduced Binary Seed bootstrap]] to [[https://www.gnu.org/software/guix][GuixSD]] and -potentially to any other interested GNU/Linux distribution, and aims -to help create a full source bootstrap as part of the -[[http://bootstrappable.org][bootstrappable builds]] effort. +[[https://www.gnu.org/software/mes][GNU Mes]] brings a [[http://joyofsource.com/reduced-binary-seed-bootstrap.html][Reduced Binary Seed bootstrap]] to [[https://www.gnu.org/software/guix][Guix]] and potentially +to any other interested GNU/Linux distribution, and aims to help +create a full source bootstrap as part of the [[http://bootstrappable.org][bootstrappable builds]] +effort. It consists of a mutual self-hosting Scheme interpreter written in ~5,000 LOC of simple C and a Nyacc-based C compiler written in Scheme. @@ -21,7 +21,7 @@ C-compiler: MesCC. Mes+MesCC can compile an only [[http://gitlab.com/janneke/tinycc][lightly patched TinyCC]] that is self-hosting. Using this tcc and the Mes C library we now have a Reduced Binary Seed bootstrap for the gnutools triplet: glibc-2.2.5, -binutils-2.20.1, gcc-2.95.3. This is enough to bootstrap GuixSD for +binutils-2.20.1, gcc-2.95.3. This is enough to bootstrap Guix for i686-linux and x86_64-linux. Mes is inspired by The Maxwell Equations of Software: [[http://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf][LISP-1.5]] -- John diff --git a/build-aux/GNUmakefile.in b/build-aux/GNUmakefile.in index ddc16a00..3678b71e 100644 --- a/build-aux/GNUmakefile.in +++ b/build-aux/GNUmakefile.in @@ -293,7 +293,7 @@ gen-announce: ../gnulib/build-aux/announce-gen\ --release-type=beta\ --package-name=mes\ - --previous-version='0.17 '\ + --previous-version='0.18 '\ --current-version=$(VERSION)\ --gpg-key-id=$(GPG_KEY_ID)\ --url-directory=https://ftp.gnu.org/gnu/mes\ diff --git a/doc/announce/ANNOUNCE-0.19 b/doc/announce/ANNOUNCE-0.19 new file mode 100644 index 00000000..7ee5f9ca --- /dev/null +++ b/doc/announce/ANNOUNCE-0.19 @@ -0,0 +1,154 @@ + +Subject: GNU Mes 0.19 released + +<#secure method=pgpmime mode=sign> +We are pleased to announce the release of GNU Mes 0.19, representing +100 commits over 10 weeks. + +Mes has now brought the Reduced Binary Seed bootstrap to Guix and work +is ongoing to audit and verify this bootstrap path in NixOS. + +This release introduces strings as byte-array, hash-tables and native +structs. While that does increase the footprint somewhat, it fixes +our performance issue; tinycc is now compiled in ~8min (WAS: ~1h30). + +Next targets: + + - translate mes.c into unsnarfed mes.M2 + - use Gash to remove bash, coreutils&co, grep, sed, tar from the Guix + bootstrap binaries + - replace the NixOS bootstrap + - use dietlibc, uClibc, ... for bootstrapping GNU (bash, binutils, + gcc, tar) and remove Mes C lib+gnu? + - bootstrap gcc-3.x or 4.x directly, drop initial gcc-2.95.3 target? + - have M1+hex2 create gcc/tcc-usable object files? archives? + - Debian? + - ARM, the Hurd? + +Packages are available from Guix's core-updates branch. + +* About + + GNU Mes[0] brings a Reduced Binary Seed bootstrap[1] to Guix[2] and + potentially to any other interested GNU/Linux distribution, and aims + to help create a full source bootstrap as part of the + bootstrappable builds[3] effort. + + It consists of a mutual self-hosting Scheme interpreter written in + ~5,000 LOC of simple C and a Nyacc-based C compiler written in Scheme. + This mes.c is being simplified[4] to be transpiled by M2-Planet[5]. + + The Scheme interpreter (mes.c) has a Garbage Collector, a library of + loadable Scheme modules-- notably Dominique Boucher's LALR[6], Pre-R6RS + [portable syntax-case[7] with R7RS ellipsis, Matt Wette's Nyacc[8] --and test + suite just barely enough to support a simple REPL and simple + C-compiler: MesCC. + + Mes+MesCC can compile an only lightly patched TinyCC[9] that is + self-hosting. Using this tcc and the Mes C library we now have a + Reduced Binary Seed bootstrap for the gnutools triplet: glibc-2.2.5, + binutils-2.20.1, gcc-2.95.3. This is enough to bootstrap Guix for + i686-linux and x86_64-linux. + + Mes is inspired by The Maxwell Equations of Software: LISP-1.5[10] -- John + McCarthy page 13, GNU Guix's source/binary packaging transparency and + Jeremiah Orians's stage0[11] ~500 byte self-hosting hex assembler. + +* Download + + git clone git://git.savannah.gnu.org/mes.git + + Here are the compressed sources and a GPG detached signature[*]: + https://ftp.gnu.org/gnu/mes/mes-0.19.tar.gz + https://ftp.gnu.org/gnu/mes/mes-0.19.tar.gz.sig + + Use a mirror for higher download bandwidth: + https://ftpmirror.gnu.org/mes/mes-0.19.tar.gz + https://ftpmirror.gnu.org/mes/mes-0.19.tar.gz.sig + + Here are the MD5 and SHA1 checksums: + + f9f901f175fbc8a5a3d90c9c551ccc8c mes-0.19.tar.gz + 4f7612731a745ebb806548186453d55e0d0bf217 mes-0.19.tar.gz + + [*] Use a .sig file to verify that the corresponding file (without the + .sig suffix) is intact. First, be sure to download both the .sig file + and the corresponding tarball. Then, run a command like this: + + gpg --verify mes-0.19.tar.gz.sig + + If that command fails because you don't have the required public key, + then run this command to import it: + + gpg --keyserver keys.gnupg.net --recv-keys 1A858392E331EAFDB8C27FFBF3C1A0D9C1D65273 + + and rerun the 'gpg --verify' command. + +Mes runs from the source tree and can also be built, packaged and +installed in Guix from a git checkout by running + + guix package -f .guix.scm + +* Get informed, get involved + + See https://bootstrappable.org + Join #bootstrappable on irc.freenode.net. + +* Changes in 0.19 since 0.18 + ** Core + *** The build system has been simplified. + *** Mes now prints a backtrace upon error. + *** Performance has been improved 2-8 times, making Mes 2-10 times slower than Guile. + *** Mes now supports a module type and uses a `boot-module'. + *** Mes now supports a hash_table type. + *** Mes now supports a struct type. + *** Mes now supports building a %bootstrap-mes seed from Guix. + ** Language + *** Records are now implemented using struct (WAS: vector). + *** 44 new functions + ceil, char-downcase, char-set-adjoin, char-set-complement, + char-upcase, current-time, delete-file, dup, dup2, file-exists?, + floor, frame-printer, get-internal-run-time, getcwd, gettimeofday, + hash, hash-ref, hash-set!, hash-table-printer, hashq, + hashq-get-handle, hashq-ref, hashq-set, inexact->exact, + make-hash-table, make-stack, make-struct, module-define!, + module-printer, module-ref, module-variable, read-line, round, + stack-length, stack-ref, string-downcase, string-tokenize, + string-upcase, struct-length, struct-ref, struct-set! struct-vtable, + struct-vtable, with-error-to-file. + ** MesCC + *** Assembly defines have been cleaned-up: duplicates deleted, missing added, wrong fixed. + *** MesCC now supports compiling GNU Bash and GNU Tar. + **** 6 New functions + getegid, geteuid, getppid, setgid, setuid, sigdelset, sigprocmask. + **** 22 New macros + EACCES, ENOSPC, ESPIPE, INT16_MAX, INT16_MIN, INT32_MAX, INT32_MIN, + INT64_MAX, INT64_MIN, INT8_MAX, INT8_MIN, LLONG_MAX, LLONG_MIN, + SIZE_MAX SYS_getegid, SYS_geteuid, SYS_setgid SYS_setuid, S_IRGRP, + S_IROTH, S_IRWXG, S_IRWXO S_ISGID, S_ISUID, S_IWGRP, S_IWOTH, S_IXGRP, + S_IXOTH, UINT16_MAX, UINT32_MAX, UINT64_MAX, UINT8_MAX, + _POSIX_VERSION. + ** Noteworthy bug fixes + *** Mes now supports characters #\xNN. + *** Mes now supports assq-ref and assoc-ref with alist == #f. + *** Mes now support \xNN in strings. Fixes using Nyacc-0.86.0. + *** MesCC now supports the unary plus operator. + *** MesCC now supports the `U' integer suffix. + *** MesCC now comes with INTnn_MIN/MAX, UINTnn defines in stdint.h. + *** MesCC now always exits non-zero when assembler or linker fail. + +Greetings, +janneke + +[0] https://www.gnu.org/software/mes +[1] http://joyofsource.com/reduced-binary-seed-bootstrap.html +[2] https://www.gnu.org/software/guix +[3] https://bootstrappable.org +[4] https://github.com/oriansj/mes-m2 +[5] https://github.com/oriansj/m2-planet +[6] https://github.com/schemeway/lalr-scm +[7] https://www.cs.indiana.edu/chezscheme/syntax-case/old-psyntax.html +[8] https://www.nongnu.org/nyacc +[9] https://gitlab.com/janneke/tinycc +[10] http://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf +[11] https://github.com/oriansj/stage0 diff --git a/doc/mes.texi b/doc/mes.texi index 2f4feb1c..1b0a72cd 100644 --- a/doc/mes.texi +++ b/doc/mes.texi @@ -33,7 +33,7 @@ Documentation License''. @end direntry @titlepage -@title Mes Reference Manual +@title GNU Mes Reference Manual @subtitle Full Source Bootstrapping of the GNU GuixSD Operating System @author Jan (janneke) Nieuwenhuizen @@ -49,7 +49,7 @@ Edition @value{EDITION} @* @c ********************************************************************* @node Top -@top Mes +@top GNU Mes This document describes GNU Mes version @value{VERSION}, a bootstrappable Scheme interpreter and C compiler written for bootstrapping the GNU system. @@ -1039,7 +1039,7 @@ Please send bug reports with full details to @email{bug-mes@@gnu.org}. @chapter Acknowledgments We would like to thank the following people for their help: Jeremiah -Orians, pdewacht, rain1, Ricardo Wurmus, Rutger van Beusekom. +Orians, Peter de Wachter, rain1, Ricardo Wurmus, Rutger van Beusekom. We also thank Ludovic Courtès for creating GuixSD and making the bootstrap problem so painfully visible, John McCarthy for creating diff --git a/mes/module/mes/boot-0.scm.in b/mes/module/mes/boot-0.scm.in index a5d7a275..68847f77 100644 --- a/mes/module/mes/boot-0.scm.in +++ b/mes/module/mes/boot-0.scm.in @@ -264,6 +264,10 @@ Ignored for Guile compatibility: --fresh-auto-compile --no-auto-compile -C,--compiled-path=DIR + +Report bugs to: bug-mes@gnu.org +GNU Mes home page: +General help using GNU software: " (or (and usage? (current-error-port)) (current-output-port))) (exit (or (and usage? 2) 0))) options) diff --git a/module/mescc.scm b/module/mescc.scm index de9b02e1..e75d5089 100644 --- a/module/mescc.scm +++ b/module/mescc.scm @@ -105,6 +105,10 @@ Environment variables: MES=BINARY run on mes-executable BINARY {mes,guile} MES_DEBUG=LEVEL show debug output with verbosity LEVEL {0..5} NYACC_TRACE=1 show Nyacc progress + +Report bugs to: bug-mes@gnu.org +GNU Mes home page: +General help using GNU software: ") (exit (or (and usage? 2) 0))) options))))