Re: Changes to job handling cause hangs in wait

[Harald van Dijk <harald@gigawatt.nl>]

On 01/12/2020 10:34, Herbert Xu wrote:
> On Tue, Dec 01, 2020 at 10:14:04AM +0000, Harald van Dijk wrote:
>>
>> POSIX says:
>>
>>    "If  the  wait utility is invoked with no operands, it shall wait until
>> all process IDs known to the invoking shell have terminated and exit with a
>> zero exit status."
>>
>> I would say that child processes that were created before dash was started
>> do not have process IDs known to dash.
> 
> Well I disagree

Then dash still has a bug regardless:

   bash -c 'dash -c "wait" <(sleep 1d)'

Here, dash does not wait for the child process.

The sleep process is either known to dash, in which case dash must wait 
in both examples, or not known to dash, in which case dash must wait in 
neither example.

>                 and the fact that the original ksh does the same
> thing is an important fact.

The original ksh is well-known not to conform to the current POSIX 
requirements, in some cases because POSIX changed requirements, in some 
cases because ksh's behaviour was clearly buggy and not worth standardising.

>>> bash -c set -e; export USER=`id -nu`; . /etc/profile >/dev/null 2>&1 || true;  . ~/.profile >/dev/null 2>&1 || true; buildtree="/tmp/autopkgtest-lxc.is4n6xxr/downtmp/build.f2G/real-tree"; mkdir -p -m 1777 -- "/tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-artifacts"; export AUTOPKGTEST_ARTIFACTS="/tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-artifacts"; export ADT_ARTIFACTS="$AUTOPKGTEST_ARTIFACTS"; mkdir -p -m 755 "/tmp/autopkgtest-lxc.is4n6xxr/downtmp/autopkgtest_tmp"; export AUTOPKGTEST_TMP="/tmp/autopkgtest-lxc.is4n6xxr/downtmp/autopkgtest_tmp"; export ADTTMP="$AUTOPKGTEST_TMP"; export DEBIAN_FRONTEND=noninteractive; export LANG=C.UTF-8; export DEB_BUILD_OPTIONS=parallel=2; unset LANGUAGE LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE   LC_MONETARY LC_MESSAGES LC_PAPER LC_NAME LC_ADDRESS   LC_TELEPHONE LC_MEASUREMENT LC_IDENTIFICATION LC_ALL;rm -f /tmp/autopkgtest_script_pid; set -C; echo $$ > /tmp/autopkgtest_script_pid; set +C; trap "rm -f /tmp/autopkgtest_script_pid" EXIT INT QUIT PIPE; cd "$buildtree"; export AUTOPKGTEST_NORMAL_USER=; export ADT_NORMAL_USER=; chmod +x /tmp/autopkgtest-lxc.is4n6xxr/downtmp/build.f2G/real-tree/debian/tests/timedated; touch /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stdout /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stderr; /tmp/autopkgtest-lxc.is4n6xxr/downtmp/build.f2G/real-tree/debian/tests/timedated 2> >(tee -a /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stderr >&2) > >(tee -a /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stdout);
>>>
>>> For some reason this is causing the final two tee's to be created
>>> as children of debian/tests/timedated rather than the bash shell.
>>
>> This is because of the same optimisation that dash also has, where it tries
>> to avoid creating a subshell for the last command in a list when it can just
>> exec() without a fork() instead. A minimal example without an explicit exec
>> is
>>
>>    bash -c 'dash -c ": & wait" <(sleep 1d)'
> 
> I'm not sure about that because bash itself is still hanging around,
> if it were really the -c optimisation then bash should not appear in
> the ps output at all.

Good point, it's a bit more complicated. Can take a more in-depth look 
at that example later.

Here's a simpler example without bash at all, then, where it is far 
clearer that the -c optimisation is responsible:

   dash -c 'sleep 1d & dash -c ": & wait"'

This used to exit immediately, leaving the sleep process running in the 
background without waiting for it. On the dash that's currently provided 
by Ubuntu, based on 0.5.10.2, it still behaves that way. With current 
dash from Git, it does not. This is clearly a change in behaviour in 
dash not in response to any bug (real or not) in bash.

Cheers,
Harald van Dijk