3 Jan
2020
3 Jan
'20
12:45 p.m.
Hoi,
in a computer forum I came across a very long command line,
including `xargs' and `sh -c'. Anyways, throughout the thread
it was modified several times, when accidently a pipe symbol
appeared between the command and the output redirection. The
command line did nothing; it ran successful. I was confused,
because I expected to see a syntax error in case of
``cmd|>file''. This made me wonder ...
With help of Sven Mascheck, I was able to clear my understanding.
The POSIX shell grammer provided the answer:
pipeline : pipe_sequence
...
pipe_sequence : command
| pipe_sequence '|' linebreak command
;
command : simple_command
...
simple_command : cmd_prefix cmd_word cmd_suffix
| cmd_prefix cmd_word
| cmd_prefix <--- HIER!
| cmd_name cmd_suffix
| cmd_name
;
cmd_prefix : io_redirect
...
io_redirect : io_file
...
io_file : '<' filename
| LESSAND filename
| '>' filename
...
A redirection is a (full) simple_command ... and because
``simple_command | simple_command'' is allowed, so is
``io_file | io_file''. This can lead to such strange (but
valid) command lines like:
<a | >b
b | <a
Sven liked this one:
:|>:
Here some further fun variants:
:|:>:
<:|:>:
They would provide nice puzzles. ;-)
My understanding was helped most by detaching from the
semantics and focussing on syntax. This one is obviously
valid, no matter it has no effect:
:|:|:
From there it was easier to grasp:
a | >a | >a
Which is valid, because ``>a'' is a (complete) simple_command.
Thus, no bug but consistent grammer. ;-)
If one would have liked to forbid such a corner case,
additional special case handling would have been necessary
... which is in contrast to the Unix way.
Sven checked the syntax against various shells with these
results:
- Syntax ok in these shells:
SVR2 sh (Ultrix), SVR4 sh (Heirloom)
ksh93
bash-1.05, bash-aktuell
pdksh-5.2.14
ash-0.4.26, dash-0.5.6.1
posh-0.3.7, posh-0.12.3
mksh-R24, mksh-R52b
yash-2.29
zsh-3.0.8, zsh-4.3.17
- Exception to the rule:
7thEd sh:
# pwd|>>file
# echo $?
141
On first sight ok, but with a silent error ... SIGPIPE (128+13).
I'd be interested in any stories and information around this
topic.
What about 7thEd sh?
meillo
3 Jan
3 Jan
2 p.m.
markus schnalke wrote in <1inMKv-0Km-00(a)marmaro.de>de>:
|Hoi,
Morsche.
|in a computer forum I came across a very long command line,
|including `xargs' and `sh -c'. Anyways, throughout the thread
|it was modified several times, when accidently a pipe symbol
|appeared between the command and the output redirection. The
|command line did nothing; it ran successful. I was confused,
|because I expected to see a syntax error in case of
|``cmd|>file''. This made me wonder ...
|
|
|With help of Sven Mascheck, I was able to clear my understanding.
|The POSIX shell grammer provided the answer:
...
|A redirection is a (full) simple_command ... and because
|``simple_command | simple_command'' is allowed, so is
|``io_file | io_file''. This can lead to such strange (but
|valid) command lines like:
|
| <a | >b
|>b | <a
|
|Sven liked this one:
|
|:|>:
|
|Here some further fun variants:
|
|:|:>:
|
| <:|:>:
|
|They would provide nice puzzles. ;-)
...
|- Syntax ok in these shells:
|
|SVR2 sh (Ultrix), SVR4 sh (Heirloom)
Be aware of spurious :> errors in Heirloom shell, and maybe its
origin (never had those). It is why the unit test of my little
web site builder thing has been switched to "printf '' > FILE".
|ksh93
|bash-1.05, bash-aktuell
|pdksh-5.2.14
|ash-0.4.26, dash-0.5.6.1
|posh-0.3.7, posh-0.12.3
|mksh-R24, mksh-R52b
|yash-2.29
|zsh-3.0.8, zsh-4.3.17
|
|- Exception to the rule:
|
|7thEd sh:
|
| # pwd|>>file
| # echo $?
| 141
|
|On first sight ok, but with a silent error ... SIGPIPE (128+13).
|
|I'd be interested in any stories and information around this
|topic.
|
|What about 7thEd sh?
Beware of bugs in the above code; I have only proved it correct,
not tried it.
|meillo
--End of <1inMKv-0Km-00(a)marmaro.de>
--steffen
|
|Der Kragenbaer, The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
5:03 p.m.
I'm curious to know if you have a script you used to prove these correct?
B
On Fri, Jan 3, 2020, at 8:00 AM, Steffen Nurpmeso wrote:
markus schnalke wrote in
<1inMKv-0Km-00(a)marmaro.de>de>:
|Hoi,
Morsche.
|in a computer forum I came across a very long command line,
|including `xargs' and `sh -c'. Anyways, throughout the thread
|it was modified several times, when accidently a pipe symbol
|appeared between the command and the output redirection. The
|command line did nothing; it ran successful. I was confused,
|because I expected to see a syntax error in case of
|``cmd|>file''. This made me wonder ...
|
|
|With help of Sven Mascheck, I was able to clear my understanding.
|The POSIX shell grammer provided the answer:
...
|A redirection is a (full) simple_command ... and because
|``simple_command | simple_command'' is allowed, so is
|``io_file | io_file''. This can lead to such strange (but
|valid) command lines like:
|
| <a | >b
|>b | <a
|
|Sven liked this one:
|
|:|>:
|
|Here some further fun variants:
|
|:|:>:
|
| <:|:>:
|
|They would provide nice puzzles. ;-)
...
|- Syntax ok in these shells:
|
|SVR2 sh (Ultrix), SVR4 sh (Heirloom)
Be aware of spurious :> errors in Heirloom shell, and maybe its
origin (never had those). It is why the unit test of my little
web site builder thing has been switched to "printf '' > FILE".
|ksh93
|bash-1.05, bash-aktuell
|pdksh-5.2.14
|ash-0.4.26, dash-0.5.6.1
|posh-0.3.7, posh-0.12.3
|mksh-R24, mksh-R52b
|yash-2.29
|zsh-3.0.8, zsh-4.3.17
|
|- Exception to the rule:
|
|7thEd sh:
|
| # pwd|>>file
| # echo $?
| 141
|
|On first sight ok, but with a silent error ... SIGPIPE (128+13).
|
|I'd be interested in any stories and information around this
|topic.
|
|What about 7thEd sh?
Beware of bugs in the above code; I have only proved it correct,
not tried it.
|meillo
--End of <1inMKv-0Km-00(a)marmaro.de>
--steffen
|
|Der Kragenbaer, The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
4 Jan
4 Jan
12:53 a.m.
New subject: sh: cmd | >file
On Fri, Jan 03, 2020 at 06:18:00PM +0100, markus schnalke wrote:
I guess Brian meant Steffen. I have no idea what Steffen meant with "proved
correct".
(and I believe Gunnar was not motivated to guarantee his heirloom port
to compete in production environments against modern shell variants with
continued maintenance, yet tried hard that time to find and fix as many bugs
as possible; I tried to help him a little bit that time. Just consider how
much still had been fixed by Sun even after their SVR4 sh. more corner case
bugs to be expected...)
BTW, I just typed ":|>file; echo $?" in a running shell and I was
particularly interested in the early Bourne shell variants. Unfortunately,
I was not able yet to check why the 7thEd sh yields SIGPIPE.
And ":|>>" (append) was just motivated by system call tracing (resulting
in
less file IO), while I tried to find out what was happening at all, before
you pointed me to the solution "simple command".
Like you I'm interested in the possible motivations for a redirection to be
a simple command. Thus Warner and Michael really have a point about using
plain redirection only, or in connection with exec, respectively.
Sven
[2020-01-03 11:03] "Brian Zick"
<brian(a)zick.io>
I'm
curious to know if you have a script you used to prove these correct?
We have to wait for Sven Mascheck to answer this question. He tested
the different shells.
8:41 p.m.
New subject: sh: cmd | >file
Sven Mascheck via TUHS wrote in <20200104005350.GA156384(a)lisa.in-ulm.de>de>:
|On Fri, Jan 03, 2020 at 06:18:00PM +0100, markus schnalke wrote:
|> [2020-01-03 11:03] "Brian Zick" <brian(a)zick.io>
|>> I'm curious to know if you have a script you used to prove these \
|>> correct?
|>
|> We have to wait for Sven Mascheck to answer this question. He tested
|> the different shells.
|
|I guess Brian meant Steffen. I have no idea what Steffen meant with "proved
|correct".
It was a citation, maybe too sloppy.
|(and I believe Gunnar was not motivated to guarantee his heirloom port
|to compete in production environments against modern shell variants with
|continued maintenance, yet tried hard that time to find and fix as \
|many bugs
|as possible; I tried to help him a little bit that time. Just consider how
That is what "is flowing out behind me", overall.
|much still had been fixed by Sun even after their SVR4 sh. more corner case
|bugs to be expected...)
I did not throw doubt on this. I think he achieved a lot in
Heirloom tools and especially roff in only four years i think,
starting in 2004. The sh has this ": > FILE" bug, which sometimes
shows up.
--steffen
|
|Der Kragenbaer, The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
3 Jan
3 Jan
7:38 p.m.
New subject: sh: cmd | >file
Hoi.
[2020-01-03 13:45] markus schnalke <meillo(a)marmaro.de>
I'd be interested in any stories and information around this
topic.
Especially I'd like to know since when and why ``>file'' is a
full command? Was it clever design or by accident?
meillo
7:44 p.m.
New subject: sh: cmd | >file
On Fri, Jan 3, 2020, 1:39 PM markus schnalke <meillo(a)marmaro.de> wrote:
Hoi.
[2020-01-03 13:45] markus schnalke <meillo(a)marmaro.de
It's used to open files in shell scripts. Iirc, exec >foo 5>& and the
like...
Warner
I'd be interested in any stories and information around this
topic.
Especially I'd like to know since when and why ``>file'' is a
full command? Was it clever design or by accident? 
10:49 p.m.
New subject: sh: cmd | >file
On 2020-01-03 13:44, Warner Losh wrote:
On Fri, Jan 3, 2020, 1:39 PM markus schnalke
<meillo(a)marmaro.de> wrote:
I've also used it to truncate files:
$ dd if=/dev/zero of=file count=10
10+0 records in
10+0 records out
5120 bytes (5.1 kB, 5.0 KiB) copied, 0.000686599 s, 7.5 MB/s
$ ls -l file
-rw-rw-r-- 1 mparson mparson 5120 Jan 3 16:45 file
$ >file
$ ls -l file
-rw-rw-r-- 1 mparson mparson 0 Jan 3 16:46 file
A more likely use case is to truncate a log file then HUP syslogd.
--
Michael Parson
Pflugerville, TX
KF5LGQ
Hoi.
[2020-01-03 13:45] markus schnalke <meillo(a)marmaro.de>
It's used to open files in shell scripts. Iirc, exec >foo 5>& and the
like...
I'd be interested in any stories and information around this
topic.
Especially I'd like to know since when and why ``>file'' is a
full command? Was it clever design or by accident? 
11:32 p.m.
New subject: sh: cmd | >file
On Fri, 3 Jan 2020, markus schnalke wrote:
Especially I'd like to know since when and why
``>file'' is a full
command? Was it clever design or by accident?
I/O redirection occurs before command execution (in this case, the null
command).
First, it creates (or truncates) the file, then runs the command; oh, it's
the null command...
-- Dave
1573
days inactive
1574
days old
9 comments
7 participants
participants (7)
-
Brian Zick -
Dave Horsfall -
markus schnalke -
Michael Parson -
Steffen Nurpmeso -
Sven Mascheck -
Warner Losh