race condition in tests/fp/Makefile

race condition in tests/fp/Makefile

[Peter Maydell]

I've just spotted this issue with tests/fp/Makefile which I think
is causing my tests/vm/netbsd builds to fail:

tests/Makefile.include has some rules that invoke a fresh make
process on tests/fp/Makefile:

tests/fp/%:
        $(MAKE) -C $(dir $@) $(notdir $@)

tests/fp/Makefile has some rules that invoke a fresh make process
in the parent build directory:

BUILD_DIR := $(CURDIR)/../..
$(LIBQEMUUTIL):
        $(MAKE) -C $(BUILD_DIR) libqemuutil.a

$(BUILD_DIR)/config-host.h:
        $(MAKE) -C $(BUILD_DIR) config-host.h

This means that we can end up with two 'make' processes
(the original top level one, and the one invoked by the
rules in tests/fp/Makefile) both trying to build things in
the top level build dir simultaneously. They then step on
each others toes and the build can fail.

In the most usual case where "make" and "make check"
run as separate steps, this doesn't happen, because
libqemuutil.a and config-host.h will both be built by
the "make" step, and then the second make invoked in
"make check" will fairly harmlessly see it has nothing
to do. But the tests/vm scripts all directly run
"make check" without a preceding "make", so they can
hit this.

I guess the best fix here is to move the dependencies
on libqemuutil.a and config-host.h into tests/Makefile.include
(though then you wouldn't be able to stand-alone run
tests/fp/Makefile -- does anybody do that?)

Also, should we change tests/vm to separately invoke
'make' and 'make check' ? I think that doing a single
'make check' is a bit fragile because we don't
really test it and there's no rule that says
"check depends on all" or similar AFAIK.

thanks
-- PMM

Re: race condition in tests/fp/Makefile

[Alex Bennée]

Peter Maydell <peter.maydell@linaro.org> writes:

> I've just spotted this issue with tests/fp/Makefile which I think
> is causing my tests/vm/netbsd builds to fail:
>
> tests/Makefile.include has some rules that invoke a fresh make
> process on tests/fp/Makefile:
>
> tests/fp/%:
>         $(MAKE) -C $(dir $@) $(notdir $@)
>
> tests/fp/Makefile has some rules that invoke a fresh make process
> in the parent build directory:
>
> BUILD_DIR := $(CURDIR)/../..
> $(LIBQEMUUTIL):
>         $(MAKE) -C $(BUILD_DIR) libqemuutil.a
>
> $(BUILD_DIR)/config-host.h:
>         $(MAKE) -C $(BUILD_DIR) config-host.h
>
> This means that we can end up with two 'make' processes
> (the original top level one, and the one invoked by the
> rules in tests/fp/Makefile) both trying to build things in
> the top level build dir simultaneously. They then step on
> each others toes and the build can fail.
>
> In the most usual case where "make" and "make check"
> run as separate steps, this doesn't happen, because
> libqemuutil.a and config-host.h will both be built by
> the "make" step, and then the second make invoked in
> "make check" will fairly harmlessly see it has nothing
> to do. But the tests/vm scripts all directly run
> "make check" without a preceding "make", so they can
> hit this.
>
> I guess the best fix here is to move the dependencies
> on libqemuutil.a and config-host.h into tests/Makefile.include
> (though then you wouldn't be able to stand-alone run
> tests/fp/Makefile -- does anybody do that?)

Not really - it needs bits to build. I guess when testing you might
invoke directly in the tree. I'll make it error out if the parent build
bits aren't there.

> Also, should we change tests/vm to separately invoke
> 'make' and 'make check' ? I think that doing a single
> 'make check' is a bit fragile because we don't
> really test it and there's no rule that says
> "check depends on all" or similar AFAIK.

OK - although shouldn't our rules be robust enough for this. 

>
> thanks
> -- PMM


-- 
Alex Bennée

Re: race condition in tests/fp/Makefile

[Peter Maydell]

On Tue, 7 Jan 2020 at 17:04, Alex Bennée <alex.bennee@linaro.org> wrote:
> Peter Maydell <peter.maydell@linaro.org> writes:
> > Also, should we change tests/vm to separately invoke
> > 'make' and 'make check' ? I think that doing a single
> > 'make check' is a bit fragile because we don't
> > really test it and there's no rule that says
> > "check depends on all" or similar AFAIK.
>
> OK - although shouldn't our rules be robust enough for this.

Ideally, you might hope that -- but I'm a bit
sceptical that we really correctly express all the
dependencies that every test has on the build having
actually built the things it needs to build...

-- PMM