Commit b807b72a7c changed the AST format
when reading single commands by mistake. It caused 'read-sh' to
return a singleton list.
* geesh/parser.scm (make-parser): Restore old behaviour.
(read-sh-all): Convert the output from the parser into a list.
* tests/parser.scm: Add tests to make sure 'read-sh-all' always
returns a list.
* geesh/parser.scm (read-sh-all): New public function for reading all
commands from a port.
(make-parser): Simplify AST forms when reading multiple commands.
* tests/parser.scm: Test it.
* geesh/parser.scm (map+fold): New function.
(merge-here-docs): New function.
(remove-quotes): New function.
(read-here-docs): New function.
(make-lexer): Handle newlines and here-document operators specially,
and emit special 'HERE-DOC' and 'HERE-DOC-SEP' tokens.
(make-parser): Use new tokens to support here-documents.
* tests/parser.scm: Add tests for here-documents.
When lexing a here-end word, expansions should be ignored. That is,
"$x" should be treated as the string "$x" and not a reference to the
parameter named "x".
* geesh/lexer.scm (expansions?): New parameter to enable or disable
treating expansions specially.
(get-double-quotation): Use it.
(get-word): Ditto.
(get-here-end): New public function that reads a token without
treating expansions specially.
* tests/lexer.scm: Test it.
* geesh/lexer.scm (get-quoted-here-doc): New function.
(get-unquoted-here-doc): New function.
(wrap-port-with-tab-trimming): New function.
(get-here-doc): New public function.
* tests/lexer.scm: Test it.
If a dollar sign does not precede a name, it gets treated normally.
* geesh/lexer.scm (get-parameter): Return '#f' if the first character
is not a name character.
(get-parameter-expansion): Propagate the '#f' value.
(get-word): Replace the '#f' value with last character read.
(get-parameter-word): Ditto.
(get-double-quotation): Ditto.
* tests/lexer.scm: Test this.
The right-hand side of a dup redirect ("<&" and ">&") may contain
expansions, so we cannot process it during parsing.
* geesh/parser.scm (process-dup-or-close-word): Remove function.
(make-parser): Return the raw word from a dup redirect instead of
trying to convert it into a number or a symbol.
* tests/parser.scm: Fix tests accordingly.
A reserved word should not be special when it is an argument to a
command. This commit makes the parser treat reserved words as normal
arguments. Note that this change exposed problems in many of the
parser tests, which relied on reserved words delimiting commands where
they should not. Those are now fixed.
* geesh/parser.scm (make-parser): Add reserved words to the default
'WORD*' rule, and use a new rule without reserved words for command
names.
* tests/parser.scm: Add a test for reserved words as arguments, and
fix old tests that relied on the old, incorrect behaviour.
* geesh/parser.scm (make-parser): Instead of using a nested list for
assignments, use a flat list.
* tests/parser.scm: Adjust existing test for this (which checks a
single assignment) and add a second test which checks multiple
assignments.
* geesh/lexer.scm (get-parameter-expression): Use '#f' to signal an
omitted optional word instead of just omitting the word in the
resulting form.
* tests/lexer.scm: Add a test for omitted optional words.
* geesh/parser.scm: New file.
* tests/parser.scm: New file.
* Makefile.am: Add them.
* .dir-locals.el: New file. Include indenting rules for Shell AST
forms and 'call-with-backquoted-input-port'.
Timothy Sample