Re: Changes to job handling cause hangs in wait

[Harald van Dijk <harald@gigawatt.nl>]

On 01/12/2020 06:06, Herbert Xu wrote:
> On Tue, Dec 01, 2020 at 04:42:03PM +1100, Herbert Xu wrote:
>>
>> Nevermind, I see that the script has been modified to use bash.
>>
>> I can reproduce the problem now so it's all good.
> 
> OK the problem is this:
> 
> 	sh -c 'sleep 1d& exec $MYSHELL -c "sleep 1& wait"'
> 
> You can replace MYSHELL with whatever shell you want to use.
> 
> Essentially dash will now wait for all children, even ones that
> were created prior to its existence, however, bash only waits for
> children that it created directly.
> 
> FWIW ksh exhibits the same behaviour as dash and I think there
> is nothing wrong with this.

POSIX says:

   "If  the  wait utility is invoked with no operands, it shall wait 
until all process IDs known to the invoking shell have terminated and 
exit with a zero exit status."

I would say that child processes that were created before dash was 
started do not have process IDs known to dash.

> So the problem is really in the parent of this shell, which appears
> to be bash:
> 
> bash -c set -e; export USER=`id -nu`; . /etc/profile >/dev/null 2>&1 || true;  . ~/.profile >/dev/null 2>&1 || true; buildtree="/tmp/autopkgtest-lxc.is4n6xxr/downtmp/build.f2G/real-tree"; mkdir -p -m 1777 -- "/tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-artifacts"; export AUTOPKGTEST_ARTIFACTS="/tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-artifacts"; export ADT_ARTIFACTS="$AUTOPKGTEST_ARTIFACTS"; mkdir -p -m 755 "/tmp/autopkgtest-lxc.is4n6xxr/downtmp/autopkgtest_tmp"; export AUTOPKGTEST_TMP="/tmp/autopkgtest-lxc.is4n6xxr/downtmp/autopkgtest_tmp"; export ADTTMP="$AUTOPKGTEST_TMP"; export DEBIAN_FRONTEND=noninteractive; export LANG=C.UTF-8; export DEB_BUILD_OPTIONS=parallel=2; unset LANGUAGE LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE   LC_MONETARY LC_MESSAGES LC_PAPER LC_NAME LC_ADDRESS   LC_TELEPHONE LC_MEASUREMENT LC_IDENTIFICATION LC_ALL;rm -f /tmp/autopkgtest_script_pid; set -C; echo $$ > /tmp/autopkgtest_script_pid; set +C; trap "rm -f /tmp/autopkgtest_script_pid" EXIT INT QUIT PIPE; cd "$buildtree"; export AUTOPKGTEST_NORMAL_USER=; export ADT_NORMAL_USER=; chmod +x /tmp/autopkgtest-lxc.is4n6xxr/downtmp/build.f2G/real-tree/debian/tests/timedated; touch /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stdout /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stderr; /tmp/autopkgtest-lxc.is4n6xxr/downtmp/build.f2G/real-tree/debian/tests/timedated 2> >(tee -a /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stderr >&2) > >(tee -a /tmp/autopkgtest-lxc.is4n6xxr/downtmp/timedated-stdout);
> 
> For some reason this is causing the final two tee's to be created
> as children of debian/tests/timedated rather than the bash shell.

This is because of the same optimisation that dash also has, where it 
tries to avoid creating a subshell for the last command in a list when 
it can just exec() without a fork() instead. A minimal example without 
an explicit exec is

   bash -c 'dash -c ": & wait" <(sleep 1d)'

Cheers,
Harald van Dijk