* geesh/word.scm (split-fields): Make an empty string yield zero
fields (an empty list).
* tests/word.scm: Update tests and add tests to make sure that quoted
empty strings are preserved.
* geesh/word.scm (split-fields): Handle a special '<sh-at>' qword
form, so that a quoted '$@' reference can still be split.
(argument-separator): New function.
(remove-quotes): Handle the '<sh-at>' form when it should not be
split. This requires knowing how to concatenate arguments, so add an
'ifs' parameter.
(expand-word): Pass 'ifs' to 'remove-quotes'.
We used to model 'IFS' as a character set, but when handling special
parameters, the order of the characters matters. Since it is also an
environment variable, it makes sense to treat as a string (rather than
a list).
* geesh/word.scm (split-fields): Treat 'ifs' as a string instead of a
character set.
(expand-word): Get 'ifs' from the environment if available or use a
default string otherwise.
* geesh/environment.scm (<environment>): Add an 'arguments' field.
(make-environment): Add an optional 'arguments' parameter.
(with-environment-arguments): New public function.
.dir-locals.el: Indent it nicely.
* geesh/environment.scm (<environment>): Add a 'functions' field.
(make-environment): Initialize it to the empty list.
(environment-function-ref): New public function.
(define-environment-function!): New public function.
Instead of using an environment variable named '?', we will use a
special environment field called 'status'. This lets us get rid of a
lot of number-string conversions (since an environment variable has to
have a string value).
* geesh/environment.scm (<environment>): Add a 'status' field.
(make-environment): Set it to 0 by default.
* geesh/repl.scm (run-repl): Use new field in place of '?' variable.
* geesh/shell.scm (exec-utility, sh:and, sh:exec-let, sh:for, sh:not,
sh:or, sh:pipeline, sh:subshell, sh:substitute-command): Ditto.
* geesh/word.scm (parameter-ref): New function that handles both
special parameters (e.g., '?') and variables.
(parameter-ref*): Like 'var-ref*', but for 'parameter-ref'.
(word->qword): Replace 'var-ref' and 'var-ref*' with 'parameter-ref'
and 'parameter-ref*' respectively.
* geesh/shell.scm (swap-and-shift-pairs): New function.
(make-pipes): New function.
(plumb): New function.
(sh:pipeline): New public function.
* tests/shell.scm: Test it.
Before, we were using raw file descriptors. This worked okay, but
there was interference caused by port buffering. Doing everything at
the port level avoids this problem, and has the added benefit of
allowing one to use the 'current-*-port' parameters with the shell
module (in a limited way, but there is room for improvement).
* geesh/shell.scm (*fd-count*): New variable.
(fd->current-port): New function.
(install-current-ports!): New function.
(exec-utility): Use it to set up file descriptors.
(save-and-install-redirect!): Rename this...
(save-and-set-redirect): ...to this and modify the current port
parameters instead of raw file descriptors.
(restore-saved-fdes!): Rename this...
(restore-saved-port): ...to this.
(sh:with-redirects): Adjust to use the renamed functions.
(sh:substitute-command): Parameterize the current ports directly
instead of using redirects.
* tests/shell.scm: Add tests to check if port buffers cause problems.
* geesh/shell.scm (%subshell): New function factored out from 'subshell'.
(subshell): Use it.
(substitute-command): New public function.
* tests/shell.scm: Test it.
* .dir-locals.el: Indent it nicely.
* geesh/shell.scm (save-and-install-redirect!): New function.
(restore-saved-fdes!): New function.
(with-redirects): New public function.
* tests/shell.scm: Test it.
* .dir-locals.el: Indent it nicely.
* geesh/built-ins/echo.scm: New file.
* geesh/built-ins.scm: New file.
* Makefile.scm: Add them.
* geesh/shell.scm (exec-let): Include built-ins in command search.
* tests/shell.scm: Add a test for this.
* geesh/shell.scm (slashless?) New function.
(split-search-path): New function.
(find-utility): New function.
(exec-let): Use them to search PATH for a utility when appropriate.
(exec): Update doc string.
* tests/shell.scm: Test it.
* geesh/parser.scm (make-parser): When parsing for loops, make all
word-lists actual lists, and make an empty word-list with the 'in'
keyword yield an empty list rather than a reference to '$@'.
* tests/parser.scm: Update tests.
We used to loop forever here because of the lack of 'eof-object?'
test!
* geesh/lexer.scm (get-single-quotation): Throw an error on unexpected
end-of-file.
This should have been in 2b05199562, but
it was overlooked.
* geesh/parser.scm (make-parser): Splice in arguments in the redirect
case (to be consistent with the non-redirect case).
* tests/parser.scm: Add a test for this.
* geesh/lexer.scm (get-double-quotation): Handle empty quotation.
* tests/lexer.scm: Add a test for this and for empty single
quotes (which already worked correctly).
This change is already in the syntax document.
* geesh/parser.scm (make-parser): Rename '<sh-define>' to '<sh-defun>'
and use the function name directly instead of making it a singleton
list.
* tests/parser.scm: Update tests.
* .dir-locals.el: Update indentation.
The AST generated by the parser did not match the syntax document.
However, neither were right. This commit updates the syntax document
to describe a more consistent form, and fixes the code to follow it.
* doc/syntax.txt (word): Make the children of '<sh-cmd-sub>' a
possibly empty list of 'list's.
* geesh/lexer.scm (get-bracketed-command): Splice in children for
'<sh-cmd-sub>'.
(get-backquoted-command): Ditto.
* geesh/parser.scm (read-sh/bracketed): Make sure that result is
always a list.
(read-sh/backquoted): Ditto.
* tests/lexer.scm: Update tests accordingly.
* tests/parser.scm: Ditto.
* geesh/lexer.scm (get-double-quotation): Make sure that '<sh-quote>'
forms only have one sub-word.
(get-unquoted-here-doc): Ditto.
* tests/lexer.scm: Update tests accordingly.
Commands with prefix redirects and no arguments or assignments would
generate '<sh-exec-let>' forms with an empty list for bindings.
* geesh/parser.scm (make-parser): Check for and handle an empty
assignment list.
* tests/parser.scm: Add test.
Commit b807b72a7c changed the AST format
when reading single commands by mistake. It caused 'read-sh' to
return a singleton list.
* geesh/parser.scm (make-parser): Restore old behaviour.
(read-sh-all): Convert the output from the parser into a list.
* tests/parser.scm: Add tests to make sure 'read-sh-all' always
returns a list.
* geesh/parser.scm (read-sh-all): New public function for reading all
commands from a port.
(make-parser): Simplify AST forms when reading multiple commands.
* tests/parser.scm: Test it.
* geesh/parser.scm (map+fold): New function.
(merge-here-docs): New function.
(remove-quotes): New function.
(read-here-docs): New function.
(make-lexer): Handle newlines and here-document operators specially,
and emit special 'HERE-DOC' and 'HERE-DOC-SEP' tokens.
(make-parser): Use new tokens to support here-documents.
* tests/parser.scm: Add tests for here-documents.
When lexing a here-end word, expansions should be ignored. That is,
"$x" should be treated as the string "$x" and not a reference to the
parameter named "x".
* geesh/lexer.scm (expansions?): New parameter to enable or disable
treating expansions specially.
(get-double-quotation): Use it.
(get-word): Ditto.
(get-here-end): New public function that reads a token without
treating expansions specially.
* tests/lexer.scm: Test it.
* geesh/lexer.scm (get-quoted-here-doc): New function.
(get-unquoted-here-doc): New function.
(wrap-port-with-tab-trimming): New function.
(get-here-doc): New public function.
* tests/lexer.scm: Test it.
If a dollar sign does not precede a name, it gets treated normally.
* geesh/lexer.scm (get-parameter): Return '#f' if the first character
is not a name character.
(get-parameter-expansion): Propagate the '#f' value.
(get-word): Replace the '#f' value with last character read.
(get-parameter-word): Ditto.
(get-double-quotation): Ditto.
* tests/lexer.scm: Test this.
The right-hand side of a dup redirect ("<&" and ">&") may contain
expansions, so we cannot process it during parsing.
* geesh/parser.scm (process-dup-or-close-word): Remove function.
(make-parser): Return the raw word from a dup redirect instead of
trying to convert it into a number or a symbol.
* tests/parser.scm: Fix tests accordingly.
A reserved word should not be special when it is an argument to a
command. This commit makes the parser treat reserved words as normal
arguments. Note that this change exposed problems in many of the
parser tests, which relied on reserved words delimiting commands where
they should not. Those are now fixed.
* geesh/parser.scm (make-parser): Add reserved words to the default
'WORD*' rule, and use a new rule without reserved words for command
names.
* tests/parser.scm: Add a test for reserved words as arguments, and
fix old tests that relied on the old, incorrect behaviour.
* geesh/parser.scm (make-parser): Instead of using a nested list for
assignments, use a flat list.
* tests/parser.scm: Adjust existing test for this (which checks a
single assignment) and add a second test which checks multiple
assignments.
* geesh/lexer.scm (get-parameter-expression): Use '#f' to signal an
omitted optional word instead of just omitting the word in the
resulting form.
* tests/lexer.scm: Add a test for omitted optional words.
* geesh/parser.scm: New file.
* tests/parser.scm: New file.
* Makefile.am: Add them.
* .dir-locals.el: New file. Include indenting rules for Shell AST
forms and 'call-with-backquoted-input-port'.
Timothy Sample