This commit fixes a problem with the old method of handling
here-documents. We used to ignore here-documents in many places where
they are valid (e.g., between Boolean operators). They are now valid
after any 'NEWLINE' token. To achieve this, the handling logic was
moved out of the parser and into the lexer. We now use a hook
mechanism to manage the necessary communication between the parser and
lexer.
* geesh/parser.scm (read-here-docs): Return strings instead of tokens.
(make-lexer): Add here-docs-hook as a keyword argument, and call it
after reading here-documents.
(make-parser): Let command-hooks transform commands, and remove
here-document handling rules.
(parse): Handle merging here-documents into commands.
* tests/parser.scm: Add tests for many less-intuitive here-document
locations.
* geesh/built-ins/set.scm (option?): New function.
(option-letter?): New function.
(set-option!): New function.
(main): Set or print options as specified by the given arguments.
* tests/spec/oil.scm: Filter out a redirect test that was only passing
because set did not work.
Instead of passing around references to the environment, just treat it
as a global. The old way was just the remains of an idea to make the
environment immutable and keep the interpreter from manipulating any
global state. By making everything global and mutable, we will have
less impedance mismatch with POSIX going forward.
The following changelog is only a sketch, since nearly every function
has changed.
* geesh/environment.scm: Replace this module with one that treats the
environment as a global resource.
* tests/environment.scm: Delete file.
* Makefile.am: Remove it from the list of tests.
* geesh/shell.scm, geesh/eval.scm, geesh/repl.scm, geesh/word.scm,
geesh/built-ins/break.scm, geesh/built-ins/continue.scm,
geesh/built-ins/echo.scm, geesh/built-ins/export.scm,
geesh/built-ins/false.scm, geesh/built-ins/read.scm,
geesh/built-ins/readonly.scm, geesh/built-ins/set.scm,
geesh/built-ins/true.scm, geesh/built-ins/unset.scm: Remove 'env'
parameters and use the new environment module.
* .dir-locals.el: Update indentation of functions that no longer take
an 'env' parameter and add with-arguments, with-environ, and
with-variables from the new environment module.
* tests/shell.scm, tests/word.scm: Update environment creation and
manipulation in tests.
Sometimes we need to make minor changes to the tests to make them work
for us. This patch enables doing search and replace on a test-by-test
basis.
* tests/spec/oil.scm: Allow for tests to be modified.
This was simply a mistake.
* geesh/shell.scm (sh:substitute-command): Leave 'current-input-port'
and 'current-error-port' alone.
* tests/shell.scm: Remove tests that check that input and error gets
ignored during substitutions.
* geesh/word.scm (split-fields): Move handling of list words into
'wedge-apart' so that they get handled in recursive calls.
* tests/word.scm: Add a test for this.
* geesh/word.scm (split-fields): Make an empty string yield zero
fields (an empty list).
* tests/word.scm: Update tests and add tests to make sure that quoted
empty strings are preserved.
tests/spec/oil.scm: New file.
tests/spec/check-spec: New file.
tests/spec/Makefile.am: New file.
configure.ac: Add the new Makefile.
Makefile.am (check-spec): New rule for running Oil shell
specification tests.
(clean-local): New rule that tells Automake how to clean up the
specification tests.
.gitignore: Add tests/spec/oil and tests/spec/oil-link.
* geesh/shell.scm (swap-and-shift-pairs): New function.
(make-pipes): New function.
(plumb): New function.
(sh:pipeline): New public function.
* tests/shell.scm: Test it.
Before, we were using raw file descriptors. This worked okay, but
there was interference caused by port buffering. Doing everything at
the port level avoids this problem, and has the added benefit of
allowing one to use the 'current-*-port' parameters with the shell
module (in a limited way, but there is room for improvement).
* geesh/shell.scm (*fd-count*): New variable.
(fd->current-port): New function.
(install-current-ports!): New function.
(exec-utility): Use it to set up file descriptors.
(save-and-install-redirect!): Rename this...
(save-and-set-redirect): ...to this and modify the current port
parameters instead of raw file descriptors.
(restore-saved-fdes!): Rename this...
(restore-saved-port): ...to this.
(sh:with-redirects): Adjust to use the renamed functions.
(sh:substitute-command): Parameterize the current ports directly
instead of using redirects.
* tests/shell.scm: Add tests to check if port buffers cause problems.
* geesh/shell.scm (%subshell): New function factored out from 'subshell'.
(subshell): Use it.
(substitute-command): New public function.
* tests/shell.scm: Test it.
* .dir-locals.el: Indent it nicely.
* geesh/shell.scm (save-and-install-redirect!): New function.
(restore-saved-fdes!): New function.
(with-redirects): New public function.
* tests/shell.scm: Test it.
* .dir-locals.el: Indent it nicely.
In order to test the REPL in isolation, we would have to mock out the
parser and interpreter. This is possible, but too much work for the
benefit (at least for now).
* tests/repl.scm: Deleted.
* Makefile.am: Remove it.
* geesh/built-ins/echo.scm: New file.
* geesh/built-ins.scm: New file.
* Makefile.scm: Add them.
* geesh/shell.scm (exec-let): Include built-ins in command search.
* tests/shell.scm: Add a test for this.
* geesh/shell.scm (slashless?) New function.
(split-search-path): New function.
(find-utility): New function.
(exec-let): Use them to search PATH for a utility when appropriate.
(exec): Update doc string.
* tests/shell.scm: Test it.
* geesh/parser.scm (make-parser): When parsing for loops, make all
word-lists actual lists, and make an empty word-list with the 'in'
keyword yield an empty list rather than a reference to '$@'.
* tests/parser.scm: Update tests.
This should have been in 2b05199562, but
it was overlooked.
* geesh/parser.scm (make-parser): Splice in arguments in the redirect
case (to be consistent with the non-redirect case).
* tests/parser.scm: Add a test for this.
* geesh/lexer.scm (get-double-quotation): Handle empty quotation.
* tests/lexer.scm: Add a test for this and for empty single
quotes (which already worked correctly).
This change is already in the syntax document.
* geesh/parser.scm (make-parser): Rename '<sh-define>' to '<sh-defun>'
and use the function name directly instead of making it a singleton
list.
* tests/parser.scm: Update tests.
* .dir-locals.el: Update indentation.
The AST generated by the parser did not match the syntax document.
However, neither were right. This commit updates the syntax document
to describe a more consistent form, and fixes the code to follow it.
* doc/syntax.txt (word): Make the children of '<sh-cmd-sub>' a
possibly empty list of 'list's.
* geesh/lexer.scm (get-bracketed-command): Splice in children for
'<sh-cmd-sub>'.
(get-backquoted-command): Ditto.
* geesh/parser.scm (read-sh/bracketed): Make sure that result is
always a list.
(read-sh/backquoted): Ditto.
* tests/lexer.scm: Update tests accordingly.
* tests/parser.scm: Ditto.
* geesh/lexer.scm (get-double-quotation): Make sure that '<sh-quote>'
forms only have one sub-word.
(get-unquoted-here-doc): Ditto.
* tests/lexer.scm: Update tests accordingly.
Commands with prefix redirects and no arguments or assignments would
generate '<sh-exec-let>' forms with an empty list for bindings.
* geesh/parser.scm (make-parser): Check for and handle an empty
assignment list.
* tests/parser.scm: Add test.
Timothy Sample