ksh vs bash: setting variable in piped loops are lost

[Georges]

I'm a long time unix (not linux) programmer, mainly shell scripts.
Unix does not know about bash, but rather ksh or sh.

So I often build stuff like this:

n=0
du | sort -n | while read size dir
do
  if [ "$size" -gt 100000 ]
  then
    n=$((n+1))
  fi
done
echo "Found $n too big files"

the question is not how to reformulate this differently using awk or perl.
The question is:

in ksh this script returns the correct value
in bash it always returns 0

This is because in ksh the last command in the pipe runs in the current process, whereas in bash the first command runs in the current proces. Result: the modified variables are lost.

I'm still puzzled that his is the case and that nobody really cares about.
Maybe I'm missing the point and there is some simple environment variable or other setting to change to enable ksh compatible behaviour.

Georges

[mssever]

Originally Posted by Georges I'm a long time unix (not linux) programmer, mainly shell scripts. Unix does not know about bash, but rather ksh or sh. So I often build stuff like this: n=0 du | sort -n | while read size dir do if [ "$size" -gt 100000 ] then n=$((n+1)) fi done echo "Found $n too big files" the question is not how to reformulate this differently using awk or perl. The question is: in ksh this script returns the correct value in bash it always returns 0 This is because in ksh the last command in the pipe runs in the current process, whereas in bash the first command runs in the current proces. Result: the modified variables are lost. I'm still puzzled that his is the case and that nobody really cares about. Maybe I'm missing the point and there is some simple environment variable or other setting to change to enable ksh compatible behaviour. Georges

I guess that's something I've never really tried before--maybe because I've always used bash. I don't think that the bash devs care a whole lot about ksh, but I understand that zsh is nearly 100% compatible with ksh--plus a bunch of added features. You might be interested in it. It's available from the main repo. I haven't used it much, but then, bash does everything I need...

I suppose a bash kludge could be to use tmp files rather than a pipeline, but that isn't very elegant.

[Georges]

yes, "apt-get install ksh" is a solution too.

So you never use longer pipe constructs with loops etc?
That's the real power of shell programming, else one could as well use perl.

zsh also behaves correctly.

Only bash is different. The developers probably think they can reinvent the wheel in square form. Do you think there is any chance that a bug report will help?

[Arndt]

Originally Posted by Georges yes, "apt-get install ksh" is a solution too. So you never use longer pipe constructs with loops etc? That's the real power of shell programming, else one could as well use perl. zsh also behaves correctly. Only bash is different. The developers probably think they can reinvent the wheel in square form. Do you think there is any chance that a bug report will help?

Googling for "bash pipe subprocesses order" shows that the question has been asked before, but I couldn't find what the bash developers' official stand is.

[malagant]

I know this is a dead thread but I wanted to add something in case someone comes looking at this for a solution. My thanks as always to the mksh team. The solution involves (m)ksh co-processes. Sounds like you may be able to do something similar with bash using named pipe pairs (one for read, one for write) if you do a little research in that area.

This works in mksh 35b:

n=0
du | sort -n |& 
while read -p size dir
 
do
  if [ "$size" -gt 1000 ]
  then
    n=$((n+1))
  fi
done
echo "Found $n too big files"

Basically this is straight from the mksh man page. mksh is the successor to pdksh and several times smaller and faster than bash or zsh. Think of it as a step up from dash at almost 100% POSIX compliance. If you come from a Solaris, HP-UX or AIX shop and are used to ksh but miss the nice touches of the newer shells then you should give mksh a try. As a bonus it is UTF aware (zsh isn't nor is bash without add-ons, I believe) and is also actively being developed.

[Georges]

that's like a kludge with temporary files we mentioned earlier.
And there are no co-processes in bash either.
How can this be the default shell in linux. It lacks several great inventions which make unix stand out.

Thanks for the hint on mksh, I will give it a try (setting it as default shell).

Georges

[mirabilos]

Originally Posted by malagant My thanks as always to the mksh team.

That would be me thanks.

Originally Posted by malagant du | sort -n |& Basically this is straight from the mksh man page.

Actually, in your case, this would be better:

(du | sort -n) |&

I seem to recall, though, that these parenthesēs are implicit in recent versions of mksh. (These also build on a fair larger number of platforms – ULTRIX, OSF/1, QNX, … – and with much more compilers than just gcc.)