Re: [Qemu-devel] [Qemu-trivial] [PATCH] Makefile: fix out-of-tree builds

Re: [Qemu-devel] [Qemu-trivial] [PATCH] Makefile: fix out-of-tree builds

[Stefan Hajnoczi]

On Thu, Jul 21, 2011 at 5:41 AM, Alexandre Raymond <cerbere@gmail.com> wrote:
> This patch fixes a minor bugs which prevented QEMU from being built
> out of tree.
>
> Signed-off-by: Alexandre Raymond <cerbere@gmail.com>
> ---
>  Makefile |    2 +-
>  1 files changed, 1 insertions(+), 1 deletions(-)

I don't normally use --source-path but it still seems broken to me
after applying your patch?

$ cd /tmp; mkdir out; cd out
$ ~/qemu/configure --source-path=$HOME/qemu
$ make
  GEN   config-all-devices.mak
cat: i386-softmmu/config-devices.mak: No such file or directory
cat: x86_64-softmmu/config-devices.mak: No such file or directory
cat: alpha-softmmu/config-devices.mak: No such file or directory

Stefan

Re: [Qemu-devel] [Qemu-trivial] [PATCH] Makefile: fix out-of-tree builds

[Michael Roth]

On 07/25/2011 05:15 AM, Stefan Hajnoczi wrote:
> On Thu, Jul 21, 2011 at 5:41 AM, Alexandre Raymond<cerbere@gmail.com>  wrote:
>> This patch fixes a minor bugs which prevented QEMU from being built
>> out of tree.
>>
>> Signed-off-by: Alexandre Raymond<cerbere@gmail.com>
>> ---
>>   Makefile |    2 +-
>>   1 files changed, 1 insertions(+), 1 deletions(-)
>
> I don't normally use --source-path but it still seems broken to me
> after applying your patch?
>
> $ cd /tmp; mkdir out; cd out
> $ ~/qemu/configure --source-path=$HOME/qemu
> $ make
>    GEN   config-all-devices.mak
> cat: i386-softmmu/config-devices.mak: No such file or directory
> cat: x86_64-softmmu/config-devices.mak: No such file or directory
> cat: alpha-softmmu/config-devices.mak: No such file or directory
>
> Stefan
>

Works okay for me with and without the patch if I do a `make distclean` 
in $HOME/qemu beforehand.

Not sure what the trigger is for the breakage Alexandre is trying to 
address.

Re: [Qemu-devel] [Qemu-trivial] [PATCH] Makefile: fix out-of-tree builds

[Stefan Hajnoczi]

On Mon, Jul 25, 2011 at 1:16 PM, Michael Roth <mdroth@linux.vnet.ibm.com> wrote:
> On 07/25/2011 05:15 AM, Stefan Hajnoczi wrote:
>>
>> On Thu, Jul 21, 2011 at 5:41 AM, Alexandre Raymond<cerbere@gmail.com>
>>  wrote:
>>>
>>> This patch fixes a minor bugs which prevented QEMU from being built
>>> out of tree.
>>>
>>> Signed-off-by: Alexandre Raymond<cerbere@gmail.com>
>>> ---
>>>  Makefile |    2 +-
>>>  1 files changed, 1 insertions(+), 1 deletions(-)
>>
>> I don't normally use --source-path but it still seems broken to me
>> after applying your patch?
>>
>> $ cd /tmp; mkdir out; cd out
>> $ ~/qemu/configure --source-path=$HOME/qemu
>> $ make
>>   GEN   config-all-devices.mak
>> cat: i386-softmmu/config-devices.mak: No such file or directory
>> cat: x86_64-softmmu/config-devices.mak: No such file or directory
>> cat: alpha-softmmu/config-devices.mak: No such file or directory
>>
>> Stefan
>>
>
> Works okay for me with and without the patch if I do a `make distclean` in
> $HOME/qemu beforehand.
>
> Not sure what the trigger is for the breakage Alexandre is trying to
> address.

You are right that make distclean in the source directory solves the issue.

Intuitively I expect ./configure to re-wire things, make distclean
should not be necessary.

Alexandre: Can you describe the case where you hit a build issue in more detail?

Stefan

Re: [Qemu-devel] [Qemu-trivial] [PATCH] Makefile: fix out-of-tree builds

[Michael Roth]

On 07/25/2011 07:43 AM, Stefan Hajnoczi wrote:
> On Mon, Jul 25, 2011 at 1:16 PM, Michael Roth<mdroth@linux.vnet.ibm.com>  wrote:
>> On 07/25/2011 05:15 AM, Stefan Hajnoczi wrote:
>>>
>>> On Thu, Jul 21, 2011 at 5:41 AM, Alexandre Raymond<cerbere@gmail.com>
>>>   wrote:
>>>>
>>>> This patch fixes a minor bugs which prevented QEMU from being built
>>>> out of tree.
>>>>
>>>> Signed-off-by: Alexandre Raymond<cerbere@gmail.com>
>>>> ---
>>>>   Makefile |    2 +-
>>>>   1 files changed, 1 insertions(+), 1 deletions(-)
>>>
>>> I don't normally use --source-path but it still seems broken to me
>>> after applying your patch?
>>>
>>> $ cd /tmp; mkdir out; cd out
>>> $ ~/qemu/configure --source-path=$HOME/qemu
>>> $ make
>>>    GEN   config-all-devices.mak
>>> cat: i386-softmmu/config-devices.mak: No such file or directory
>>> cat: x86_64-softmmu/config-devices.mak: No such file or directory
>>> cat: alpha-softmmu/config-devices.mak: No such file or directory
>>>
>>> Stefan
>>>
>>
>> Works okay for me with and without the patch if I do a `make distclean` in
>> $HOME/qemu beforehand.
>>
>> Not sure what the trigger is for the breakage Alexandre is trying to
>> address.
>
> You are right that make distclean in the source directory solves the issue.
>
> Intuitively I expect ./configure to re-wire things, make distclean
> should not be necessary.
>
> Alexandre: Can you describe the case where you hit a build issue in more detail?
>
> Stefan

The root problem seems to be that by including $(SRC_DIR) in VPATH (via 
$(call set-vpath, $(SRC_PATH):$(SRC_PATH)/hw)), $(SRC_DIR) ends up being 
searched for source files as well as target dependencies. So any crud 
left in there can still satisfy dependencies when building outside 
$SRC_PATH. I'm not sure there's a simple way around this except to 
prefix all source files with $(SRC_PATH) and remove $(SRC_PATH) from 
VPATH...I'm not even sure that would work though..

Perhaps just a friendly error message if we detect the $(SRC_PATH) 
directory needs a distclean? Once you know that's the magic fix it's not 
terribly inconvenient....alternatively we could automatically do the 
distclean in $SRC_PATH but that might be considered overstepping our bounds.

Consequently, it seems like this patch would be a noop...default-configs 
should never exist in an external build directory, so 
$(SRC_PATH)/default-configs and default-configs end up being equivalent 
when make eventually find it in $(SRC_PATH).

Re: [Qemu-devel] [Qemu-trivial] [PATCH] Makefile: fix out-of-tree builds

[Stefan Hajnoczi]

On Mon, Jul 25, 2011 at 2:57 PM, Michael Roth <mdroth@linux.vnet.ibm.com> wrote:
> On 07/25/2011 07:43 AM, Stefan Hajnoczi wrote:
>>
>> On Mon, Jul 25, 2011 at 1:16 PM, Michael Roth<mdroth@linux.vnet.ibm.com>
>>  wrote:
>>>
>>> On 07/25/2011 05:15 AM, Stefan Hajnoczi wrote:
>>>>
>>>> On Thu, Jul 21, 2011 at 5:41 AM, Alexandre Raymond<cerbere@gmail.com>
>>>>  wrote:
>>>>>
>>>>> This patch fixes a minor bugs which prevented QEMU from being built
>>>>> out of tree.
>>>>>
>>>>> Signed-off-by: Alexandre Raymond<cerbere@gmail.com>
>>>>> ---
>>>>>  Makefile |    2 +-
>>>>>  1 files changed, 1 insertions(+), 1 deletions(-)
>>>>
>>>> I don't normally use --source-path but it still seems broken to me
>>>> after applying your patch?
>>>>
>>>> $ cd /tmp; mkdir out; cd out
>>>> $ ~/qemu/configure --source-path=$HOME/qemu
>>>> $ make
>>>>   GEN   config-all-devices.mak
>>>> cat: i386-softmmu/config-devices.mak: No such file or directory
>>>> cat: x86_64-softmmu/config-devices.mak: No such file or directory
>>>> cat: alpha-softmmu/config-devices.mak: No such file or directory
>>>>
>>>> Stefan
>>>>
>>>
>>> Works okay for me with and without the patch if I do a `make distclean`
>>> in
>>> $HOME/qemu beforehand.
>>>
>>> Not sure what the trigger is for the breakage Alexandre is trying to
>>> address.
>>
>> You are right that make distclean in the source directory solves the
>> issue.
>>
>> Intuitively I expect ./configure to re-wire things, make distclean
>> should not be necessary.
>>
>> Alexandre: Can you describe the case where you hit a build issue in more
>> detail?
>>
>> Stefan
>
> The root problem seems to be that by including $(SRC_DIR) in VPATH (via
> $(call set-vpath, $(SRC_PATH):$(SRC_PATH)/hw)), $(SRC_DIR) ends up being
> searched for source files as well as target dependencies. So any crud left
> in there can still satisfy dependencies when building outside $SRC_PATH. I'm
> not sure there's a simple way around this except to prefix all source files
> with $(SRC_PATH) and remove $(SRC_PATH) from VPATH...I'm not even sure that
> would work though..
>
> Perhaps just a friendly error message if we detect the $(SRC_PATH) directory
> needs a distclean? Once you know that's the magic fix it's not terribly
> inconvenient....alternatively we could automatically do the distclean in
> $SRC_PATH but that might be considered overstepping our bounds.

Yes, ./configure could detect the case where --source-path= is used
but the source tree has build outputs.

> Consequently, it seems like this patch would be a noop...default-configs
> should never exist in an external build directory, so
> $(SRC_PATH)/default-configs and default-configs end up being equivalent when
> make eventually find it in $(SRC_PATH).

Thanks for finding out what is happening.  My understanding is that
$(SRC_PATH) must be used when invoking external commands during the
build, since they don't perform vpath search.  For make targets we
don't need to use $(SRC_PATH) since the vpath is in effect.

Stefan

Re: [Qemu-devel] [Qemu-trivial] [PATCH] Makefile: fix out-of-tree builds

[Michael Roth]

On 07/25/2011 09:43 AM, Stefan Hajnoczi wrote:
> On Mon, Jul 25, 2011 at 2:57 PM, Michael Roth<mdroth@linux.vnet.ibm.com>  wrote:
>> On 07/25/2011 07:43 AM, Stefan Hajnoczi wrote:
>>>
>>> On Mon, Jul 25, 2011 at 1:16 PM, Michael Roth<mdroth@linux.vnet.ibm.com>
>>>   wrote:
>>>>
>>>> On 07/25/2011 05:15 AM, Stefan Hajnoczi wrote:
>>>>>
>>>>> On Thu, Jul 21, 2011 at 5:41 AM, Alexandre Raymond<cerbere@gmail.com>
>>>>>   wrote:
>>>>>>
>>>>>> This patch fixes a minor bugs which prevented QEMU from being built
>>>>>> out of tree.
>>>>>>
>>>>>> Signed-off-by: Alexandre Raymond<cerbere@gmail.com>
>>>>>> ---
>>>>>>   Makefile |    2 +-
>>>>>>   1 files changed, 1 insertions(+), 1 deletions(-)
>>>>>
>>>>> I don't normally use --source-path but it still seems broken to me
>>>>> after applying your patch?
>>>>>
>>>>> $ cd /tmp; mkdir out; cd out
>>>>> $ ~/qemu/configure --source-path=$HOME/qemu
>>>>> $ make
>>>>>    GEN   config-all-devices.mak
>>>>> cat: i386-softmmu/config-devices.mak: No such file or directory
>>>>> cat: x86_64-softmmu/config-devices.mak: No such file or directory
>>>>> cat: alpha-softmmu/config-devices.mak: No such file or directory
>>>>>
>>>>> Stefan
>>>>>
>>>>
>>>> Works okay for me with and without the patch if I do a `make distclean`
>>>> in
>>>> $HOME/qemu beforehand.
>>>>
>>>> Not sure what the trigger is for the breakage Alexandre is trying to
>>>> address.
>>>
>>> You are right that make distclean in the source directory solves the
>>> issue.
>>>
>>> Intuitively I expect ./configure to re-wire things, make distclean
>>> should not be necessary.
>>>
>>> Alexandre: Can you describe the case where you hit a build issue in more
>>> detail?
>>>
>>> Stefan
>>
>> The root problem seems to be that by including $(SRC_DIR) in VPATH (via
>> $(call set-vpath, $(SRC_PATH):$(SRC_PATH)/hw)), $(SRC_DIR) ends up being
>> searched for source files as well as target dependencies. So any crud left
>> in there can still satisfy dependencies when building outside $SRC_PATH. I'm
>> not sure there's a simple way around this except to prefix all source files
>> with $(SRC_PATH) and remove $(SRC_PATH) from VPATH...I'm not even sure that
>> would work though..
>>
>> Perhaps just a friendly error message if we detect the $(SRC_PATH) directory
>> needs a distclean? Once you know that's the magic fix it's not terribly
>> inconvenient....alternatively we could automatically do the distclean in
>> $SRC_PATH but that might be considered overstepping our bounds.
>
> Yes, ./configure could detect the case where --source-path= is used
> but the source tree has build outputs.
>
>> Consequently, it seems like this patch would be a noop...default-configs
>> should never exist in an external build directory, so
>> $(SRC_PATH)/default-configs and default-configs end up being equivalent when
>> make eventually find it in $(SRC_PATH).
>
> Thanks for finding out what is happening.  My understanding is that
> $(SRC_PATH) must be used when invoking external commands during the
> build, since they don't perform vpath search.  For make targets we
> don't need to use $(SRC_PATH) since the vpath is in effect.

Ah, yah my mistake, patch would still have some effect.

>
> Stefan

Re: [Qemu-devel] [Qemu-trivial] [PATCH] Makefile: fix out-of-tree builds

[Alexandre Raymond]

Hi guys,

Thank you for investigating this problem.

I finally found out why this problem occured in the first place. It is
a combination of the bug you just described and the fact that "make
distclean" only deletes the build directories of the targets specified
*in the last call to configure*.

e.g.: ./configure && make && ./configure --target-list=i386-softmmu &&
make distclean
will only delete the build directories related to that i386 target
(and not the others).

This is caused by the fact that "make distclean" cleans the targets
specified by $(TARGET_DIRS) in the configure script, which is in turn
affected by --target-list.

I guess it would need an "ALL_TARGET_DIRS" variable instead to do its
cleanup, or rm -rf *-user and *-softmmu (although this sounds less
safe).

Alexandre



On Mon, Jul 25, 2011 at 11:06 AM, Michael Roth
<mdroth@linux.vnet.ibm.com> wrote:
> On 07/25/2011 09:43 AM, Stefan Hajnoczi wrote:
>>
>> On Mon, Jul 25, 2011 at 2:57 PM, Michael Roth<mdroth@linux.vnet.ibm.com>
>>  wrote:
>>>
>>> On 07/25/2011 07:43 AM, Stefan Hajnoczi wrote:
>>>>
>>>> On Mon, Jul 25, 2011 at 1:16 PM, Michael Roth<mdroth@linux.vnet.ibm.com>
>>>>  wrote:
>>>>>
>>>>> On 07/25/2011 05:15 AM, Stefan Hajnoczi wrote:
>>>>>>
>>>>>> On Thu, Jul 21, 2011 at 5:41 AM, Alexandre Raymond<cerbere@gmail.com>
>>>>>>  wrote:
>>>>>>>
>>>>>>> This patch fixes a minor bugs which prevented QEMU from being built
>>>>>>> out of tree.
>>>>>>>
>>>>>>> Signed-off-by: Alexandre Raymond<cerbere@gmail.com>
>>>>>>> ---
>>>>>>>  Makefile |    2 +-
>>>>>>>  1 files changed, 1 insertions(+), 1 deletions(-)
>>>>>>
>>>>>> I don't normally use --source-path but it still seems broken to me
>>>>>> after applying your patch?
>>>>>>
>>>>>> $ cd /tmp; mkdir out; cd out
>>>>>> $ ~/qemu/configure --source-path=$HOME/qemu
>>>>>> $ make
>>>>>>   GEN   config-all-devices.mak
>>>>>> cat: i386-softmmu/config-devices.mak: No such file or directory
>>>>>> cat: x86_64-softmmu/config-devices.mak: No such file or directory
>>>>>> cat: alpha-softmmu/config-devices.mak: No such file or directory
>>>>>>
>>>>>> Stefan
>>>>>>
>>>>>
>>>>> Works okay for me with and without the patch if I do a `make distclean`
>>>>> in
>>>>> $HOME/qemu beforehand.
>>>>>
>>>>> Not sure what the trigger is for the breakage Alexandre is trying to
>>>>> address.
>>>>
>>>> You are right that make distclean in the source directory solves the
>>>> issue.
>>>>
>>>> Intuitively I expect ./configure to re-wire things, make distclean
>>>> should not be necessary.
>>>>
>>>> Alexandre: Can you describe the case where you hit a build issue in more
>>>> detail?
>>>>
>>>> Stefan
>>>
>>> The root problem seems to be that by including $(SRC_DIR) in VPATH (via
>>> $(call set-vpath, $(SRC_PATH):$(SRC_PATH)/hw)), $(SRC_DIR) ends up being
>>> searched for source files as well as target dependencies. So any crud
>>> left
>>> in there can still satisfy dependencies when building outside $SRC_PATH.
>>> I'm
>>> not sure there's a simple way around this except to prefix all source
>>> files
>>> with $(SRC_PATH) and remove $(SRC_PATH) from VPATH...I'm not even sure
>>> that
>>> would work though..
>>>
>>> Perhaps just a friendly error message if we detect the $(SRC_PATH)
>>> directory
>>> needs a distclean? Once you know that's the magic fix it's not terribly
>>> inconvenient....alternatively we could automatically do the distclean in
>>> $SRC_PATH but that might be considered overstepping our bounds.
>>
>> Yes, ./configure could detect the case where --source-path= is used
>> but the source tree has build outputs.
>>
>>> Consequently, it seems like this patch would be a noop...default-configs
>>> should never exist in an external build directory, so
>>> $(SRC_PATH)/default-configs and default-configs end up being equivalent
>>> when
>>> make eventually find it in $(SRC_PATH).
>>
>> Thanks for finding out what is happening.  My understanding is that
>> $(SRC_PATH) must be used when invoking external commands during the
>> build, since they don't perform vpath search.  For make targets we
>> don't need to use $(SRC_PATH) since the vpath is in effect.
>
> Ah, yah my mistake, patch would still have some effect.
>
>>
>> Stefan
>
>