* deprecation of in-tree builds @ 2020-03-21 20:50 Peter Maydell 2020-03-21 22:50 ` BALATON Zoltan ` (3 more replies) 0 siblings, 4 replies; 53+ messages in thread From: Peter Maydell @ 2020-03-21 20:50 UTC (permalink / raw) To: QEMU Developers, Paolo Bonzini AIUI from Paolo, the intention is to deprecate and eventually stop supporting "in-tree" builds, so that the only option is building in a separate build directory. I thought we should probably mention that in the 5.0 changelog, so I wrote up some text: https://wiki.qemu.org/ChangeLog/5.0#Build_Information Suggestions for changes/comments etc welcome. (One thing we will need to fix before we can do separate build tree is the Coverity Scan build process, which (a) does an in-tree build (b) can't be easily switched to a builddir because all the source paths get baked into the scan results and moving to a builddir changes them all...) We could also make configure actively warn if used in the source tree. thanks -- PMM ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-21 20:50 deprecation of in-tree builds Peter Maydell @ 2020-03-21 22:50 ` BALATON Zoltan 2020-03-23 13:32 ` Stefan Hajnoczi [not found] ` <CAL1e-=gKB0qRxGntXrU0im2gjMh1tq_SLGTm+HsmgBRGXQ9+Bg@mail.gmail.com> ` (2 subsequent siblings) 3 siblings, 1 reply; 53+ messages in thread From: BALATON Zoltan @ 2020-03-21 22:50 UTC (permalink / raw) To: Peter Maydell; +Cc: Paolo Bonzini, QEMU Developers On Sat, 21 Mar 2020, Peter Maydell wrote: > AIUI from Paolo, the intention is to deprecate and eventually > stop supporting "in-tree" builds, so that the only option is > building in a separate build directory. I thought we should > probably mention that in the 5.0 changelog, so I wrote up some > text: > > https://wiki.qemu.org/ChangeLog/5.0#Build_Information > > Suggestions for changes/comments etc welcome. > > (One thing we will need to fix before we can do separate build > tree is the Coverity Scan build process, which (a) does an > in-tree build (b) can't be easily switched to a builddir because > all the source paths get baked into the scan results and moving > to a builddir changes them all...) > > We could also make configure actively warn if used in > the source tree. This was discussed before. I think instead of annoying people with a warning, rather configure should be changed to create a build dir if run from source and have a Makefile in top dir that runs make -C builddir so people don't have to care about this or change their ways and can continue to run configure && make from source dir but you don't have to support in-tree build. Then you can deprecate in-tree builds but supporting only out-of-tree without this convenience would not just unnecessarily annoy those who prefer working in a single tree but people (and apparently some tools) expect sources to build with usual configure; make; make install so that should be the minimum to support. Regards, BALATON Zoltan ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-21 22:50 ` BALATON Zoltan @ 2020-03-23 13:32 ` Stefan Hajnoczi 2020-03-24 13:54 ` Eric Blake 2020-08-18 19:14 ` Peter Maydell 0 siblings, 2 replies; 53+ messages in thread From: Stefan Hajnoczi @ 2020-03-23 13:32 UTC (permalink / raw) To: BALATON Zoltan; +Cc: Peter Maydell, QEMU Developers, Paolo Bonzini [-- Attachment #1: Type: text/plain, Size: 1893 bytes --] On Sat, Mar 21, 2020 at 11:50:23PM +0100, BALATON Zoltan wrote: > On Sat, 21 Mar 2020, Peter Maydell wrote: > > AIUI from Paolo, the intention is to deprecate and eventually > > stop supporting "in-tree" builds, so that the only option is > > building in a separate build directory. I thought we should > > probably mention that in the 5.0 changelog, so I wrote up some > > text: > > > > https://wiki.qemu.org/ChangeLog/5.0#Build_Information > > > > Suggestions for changes/comments etc welcome. > > > > (One thing we will need to fix before we can do separate build > > tree is the Coverity Scan build process, which (a) does an > > in-tree build (b) can't be easily switched to a builddir because > > all the source paths get baked into the scan results and moving > > to a builddir changes them all...) > > > > We could also make configure actively warn if used in > > the source tree. > > This was discussed before. I think instead of annoying people with a > warning, rather configure should be changed to create a build dir if run > from source and have a Makefile in top dir that runs make -C builddir so > people don't have to care about this or change their ways and can continue > to run configure && make from source dir but you don't have to support > in-tree build. Then you can deprecate in-tree builds but supporting only > out-of-tree without this convenience would not just unnecessarily annoy > those who prefer working in a single tree but people (and apparently some > tools) expect sources to build with usual configure; make; make install so > that should be the minimum to support. Yes, please! I use in-tree builds and find it tedious to cd into a build dir manually. Also, many build scripts (packaging, etc) we'll break if we simply remove in-tree builds. I think make && make install should continue to work. Stefan [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-23 13:32 ` Stefan Hajnoczi @ 2020-03-24 13:54 ` Eric Blake 2020-08-18 19:14 ` Peter Maydell 1 sibling, 0 replies; 53+ messages in thread From: Eric Blake @ 2020-03-24 13:54 UTC (permalink / raw) To: Stefan Hajnoczi, BALATON Zoltan Cc: Peter Maydell, QEMU Developers, Paolo Bonzini On 3/23/20 8:32 AM, Stefan Hajnoczi wrote: >>> We could also make configure actively warn if used in >>> the source tree. >> >> This was discussed before. I think instead of annoying people with a >> warning, rather configure should be changed to create a build dir if run >> from source and have a Makefile in top dir that runs make -C builddir so >> people don't have to care about this or change their ways and can continue >> to run configure && make from source dir but you don't have to support >> in-tree build. Then you can deprecate in-tree builds but supporting only >> out-of-tree without this convenience would not just unnecessarily annoy >> those who prefer working in a single tree but people (and apparently some >> tools) expect sources to build with usual configure; make; make install so >> that should be the minimum to support. > > Yes, please! I use in-tree builds and find it tedious to cd into a > build dir manually. > > Also, many build scripts (packaging, etc) we'll break if we simply > remove in-tree builds. I think make && make install should continue to > work. Here's what I'm using locally to let my finger memory of building in-tree work in spite of having done 'mkdir build; cd build; ../configure ...'. We'd still need glue code in configure itself to detect when it is being invoked in-tree to create a subdir build/ and such a forwarding file, but if we want in-tree to FEEL like it works (even though it uses a build dir under the hood), it's certainly worth considering. $ cat GNUmakefile # Hack for redirecting while reminding myself to use distinct builddir ifeq ($(MAKECMDGOALS),) recurse: all endif .NOTPARALLEL: % %: force @echo 'changing dir to build for $(MAKE) "$(MAKECMDGOALS)"...' @$(MAKE) -C build -f Makefile $(MAKECMDGOALS) force: ; .PHONY: force GNUmakefile: ; -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-23 13:32 ` Stefan Hajnoczi 2020-03-24 13:54 ` Eric Blake @ 2020-08-18 19:14 ` Peter Maydell 2020-08-20 10:54 ` Kevin Wolf 2020-08-20 16:05 ` Daniel P. Berrangé 1 sibling, 2 replies; 53+ messages in thread From: Peter Maydell @ 2020-08-18 19:14 UTC (permalink / raw) To: Stefan Hajnoczi; +Cc: Kevin Wolf, Paolo Bonzini, QEMU Developers On Mon, 23 Mar 2020 at 13:32, Stefan Hajnoczi <stefanha@gmail.com> wrote: > On Sat, Mar 21, 2020 at 11:50:23PM +0100, BALATON Zoltan wrote: > > This was discussed before. I think instead of annoying people with a > > warning, rather configure should be changed to create a build dir if run > > from source and have a Makefile in top dir that runs make -C builddir so > > people don't have to care about this or change their ways and can continue > > to run configure && make from source dir but you don't have to support > > in-tree build. Then you can deprecate in-tree builds but supporting only > > out-of-tree without this convenience would not just unnecessarily annoy > > those who prefer working in a single tree but people (and apparently some > > tools) expect sources to build with usual configure; make; make install so > > that should be the minimum to support. > > Yes, please! I use in-tree builds and find it tedious to cd into a > build dir manually. > > Also, many build scripts (packaging, etc) we'll break if we simply > remove in-tree builds. I think make && make install should continue to > work. Paolo's conversion-to-Meson patchseries is about to land, so now is the time for people who would like this "automatically create a build directory and use it" behaviour to write the necessary patches. Any volunteers ? My current plan is to land the Meson series first, because it is really painful for Paolo to try to keep rebasing it as other changes to the old build system occur. This would break in-tree builds temporarily until the "automatic creation and use of a builddir" patches go in on top of it. thanks -- PMM ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-08-18 19:14 ` Peter Maydell @ 2020-08-20 10:54 ` Kevin Wolf 2020-08-20 11:14 ` Michael Tokarev 2020-08-20 11:56 ` Paolo Bonzini 2020-08-20 16:05 ` Daniel P. Berrangé 1 sibling, 2 replies; 53+ messages in thread From: Kevin Wolf @ 2020-08-20 10:54 UTC (permalink / raw) To: Peter Maydell; +Cc: Stefan Hajnoczi, Paolo Bonzini, QEMU Developers Am 18.08.2020 um 21:14 hat Peter Maydell geschrieben: > On Mon, 23 Mar 2020 at 13:32, Stefan Hajnoczi <stefanha@gmail.com> wrote: > > On Sat, Mar 21, 2020 at 11:50:23PM +0100, BALATON Zoltan wrote: > > > This was discussed before. I think instead of annoying people with a > > > warning, rather configure should be changed to create a build dir if run > > > from source and have a Makefile in top dir that runs make -C builddir so > > > people don't have to care about this or change their ways and can continue > > > to run configure && make from source dir but you don't have to support > > > in-tree build. Then you can deprecate in-tree builds but supporting only > > > out-of-tree without this convenience would not just unnecessarily annoy > > > those who prefer working in a single tree but people (and apparently some > > > tools) expect sources to build with usual configure; make; make install so > > > that should be the minimum to support. > > > > Yes, please! I use in-tree builds and find it tedious to cd into a > > build dir manually. > > > > Also, many build scripts (packaging, etc) we'll break if we simply > > remove in-tree builds. I think make && make install should continue to > > work. > > Paolo's conversion-to-Meson patchseries is about to land, so now > is the time for people who would like this "automatically create > a build directory and use it" behaviour to write the necessary > patches. Any volunteers ? > > My current plan is to land the Meson series first, because it is > really painful for Paolo to try to keep rebasing it as other > changes to the old build system occur. This would break > in-tree builds temporarily until the "automatic creation and > use of a builddir" patches go in on top of it. Usually, our requirement is that patch series don't break anything. And if something slips through, whoever broke it is supposed to fix it, not whoever is affected. Kevin ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-08-20 10:54 ` Kevin Wolf @ 2020-08-20 11:14 ` Michael Tokarev 2020-08-20 11:56 ` Paolo Bonzini 1 sibling, 0 replies; 53+ messages in thread From: Michael Tokarev @ 2020-08-20 11:14 UTC (permalink / raw) To: Kevin Wolf, Peter Maydell; +Cc: Stefan Hajnoczi, QEMU Developers, Paolo Bonzini 20.08.2020 13:54, Kevin Wolf wrote: [] >> My current plan is to land the Meson series first, because it is >> really painful for Paolo to try to keep rebasing it as other >> changes to the old build system occur. This would break >> in-tree builds temporarily until the "automatic creation and >> use of a builddir" patches go in on top of it. > > Usually, our requirement is that patch series don't break anything. And > if something slips through, whoever broke it is supposed to fix it, not > whoever is affected. In this case it isn't really broken per se - there's a simple workaround (build in a subdir). This rule exist to keep the bisectability, and this one is trivial by invoking this very workaround. /mjt ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-08-20 10:54 ` Kevin Wolf 2020-08-20 11:14 ` Michael Tokarev @ 2020-08-20 11:56 ` Paolo Bonzini 2020-08-20 12:00 ` Peter Maydell 2020-08-20 13:30 ` Kevin Wolf 1 sibling, 2 replies; 53+ messages in thread From: Paolo Bonzini @ 2020-08-20 11:56 UTC (permalink / raw) To: Kevin Wolf, Peter Maydell; +Cc: Stefan Hajnoczi, QEMU Developers On 20/08/20 12:54, Kevin Wolf wrote: >> Paolo's conversion-to-Meson patchseries is about to land, so now >> is the time for people who would like this "automatically create >> a build directory and use it" behaviour to write the necessary >> patches. Any volunteers ? >> >> My current plan is to land the Meson series first, because it is >> really painful for Paolo to try to keep rebasing it as other >> changes to the old build system occur. This would break >> in-tree builds temporarily until the "automatic creation and >> use of a builddir" patches go in on top of it. > > Usually, our requirement is that patch series don't break anything. And > if something slips through, whoever broke it is supposed to fix it, not > whoever is affected. The Meson conversion was announced in October 2019 as breaking in-tree builds, and the deprecation request is from March 2020. So I don't think this is a breakage but rather a widely-announced change. Paolo ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-08-20 11:56 ` Paolo Bonzini @ 2020-08-20 12:00 ` Peter Maydell 2020-08-20 13:30 ` Kevin Wolf 1 sibling, 0 replies; 53+ messages in thread From: Peter Maydell @ 2020-08-20 12:00 UTC (permalink / raw) To: Paolo Bonzini; +Cc: Kevin Wolf, Stefan Hajnoczi, QEMU Developers On Thu, 20 Aug 2020 at 12:56, Paolo Bonzini <pbonzini@redhat.com> wrote: > > On 20/08/20 12:54, Kevin Wolf wrote: > >> Paolo's conversion-to-Meson patchseries is about to land, so now > >> is the time for people who would like this "automatically create > >> a build directory and use it" behaviour to write the necessary > >> patches. Any volunteers ? > >> > >> My current plan is to land the Meson series first, because it is > >> really painful for Paolo to try to keep rebasing it as other > >> changes to the old build system occur. This would break > >> in-tree builds temporarily until the "automatic creation and > >> use of a builddir" patches go in on top of it. > > > > Usually, our requirement is that patch series don't break anything. And > > if something slips through, whoever broke it is supposed to fix it, not > > whoever is affected. > > The Meson conversion was announced in October 2019 as breaking in-tree > builds, and the deprecation request is from March 2020. So I don't > think this is a breakage but rather a widely-announced change. No, we need to put in the back-compat for making basic in-source-tree configure/make/make install work. There was a fair amount of discussion of this on-list and the clear consensus was that it was widely enough used, that it wouldn't be a huge amount of work to make it do a behind-the-scenes creation of the buildir, and that we should retain it. thanks -- PMM ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-08-20 11:56 ` Paolo Bonzini 2020-08-20 12:00 ` Peter Maydell @ 2020-08-20 13:30 ` Kevin Wolf 2020-08-20 14:10 ` Peter Maydell 2020-08-20 15:50 ` Paolo Bonzini 1 sibling, 2 replies; 53+ messages in thread From: Kevin Wolf @ 2020-08-20 13:30 UTC (permalink / raw) To: Paolo Bonzini; +Cc: Peter Maydell, QEMU Developers, Stefan Hajnoczi Am 20.08.2020 um 13:56 hat Paolo Bonzini geschrieben: > On 20/08/20 12:54, Kevin Wolf wrote: > >> Paolo's conversion-to-Meson patchseries is about to land, so now > >> is the time for people who would like this "automatically create > >> a build directory and use it" behaviour to write the necessary > >> patches. Any volunteers ? > >> > >> My current plan is to land the Meson series first, because it is > >> really painful for Paolo to try to keep rebasing it as other > >> changes to the old build system occur. This would break > >> in-tree builds temporarily until the "automatic creation and > >> use of a builddir" patches go in on top of it. > > > > Usually, our requirement is that patch series don't break anything. And > > if something slips through, whoever broke it is supposed to fix it, not > > whoever is affected. > > The Meson conversion was announced in October 2019 as breaking in-tree > builds, and the deprecation request is from March 2020. So I don't > think this is a breakage but rather a widely-announced change. Wasn't the decision after that discussion that we do _not_ want to deprecate './configure; make' from the source directory? I seem to remember that we wanted to merge a message to make a recommendation for out-of-tree builds, but looking at configure, I can't find even that. So without that, and also without a mention in deprecated.rst, I don't think having mentioned a wish to break things a while ago means that we should just follow through with that despite the objections. At least this isn't how it has worked for other patch series. If it is the new standard, I can remove -drive tomorrow. I've been mentioning for years that I don't like it and want to remove it, so people can just deal with it. Kevin ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-08-20 13:30 ` Kevin Wolf @ 2020-08-20 14:10 ` Peter Maydell 2020-08-20 15:50 ` Paolo Bonzini 1 sibling, 0 replies; 53+ messages in thread From: Peter Maydell @ 2020-08-20 14:10 UTC (permalink / raw) To: Kevin Wolf; +Cc: Paolo Bonzini, QEMU Developers, Stefan Hajnoczi On Thu, 20 Aug 2020 at 14:30, Kevin Wolf <kwolf@redhat.com> wrote: > > Am 20.08.2020 um 13:56 hat Paolo Bonzini geschrieben: > > The Meson conversion was announced in October 2019 as breaking in-tree > > builds, and the deprecation request is from March 2020. So I don't > > think this is a breakage but rather a widely-announced change. > > Wasn't the decision after that discussion that we do _not_ want to > deprecate './configure; make' from the source directory? > > I seem to remember that we wanted to merge a message to make a > recommendation for out-of-tree builds, but looking at configure, I can't > find even that. We opted not to do that or to mark out-of-tree builds as deprecated in the release notes, because the outcome of the discussion was that we wanted to retain the support for 'run configure and make from the source tree'. thanks -- PMM ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-08-20 13:30 ` Kevin Wolf 2020-08-20 14:10 ` Peter Maydell @ 2020-08-20 15:50 ` Paolo Bonzini 2020-08-20 16:14 ` Kevin Wolf 1 sibling, 1 reply; 53+ messages in thread From: Paolo Bonzini @ 2020-08-20 15:50 UTC (permalink / raw) To: Kevin Wolf; +Cc: Peter Maydell, QEMU Developers, Stefan Hajnoczi On 20/08/20 15:30, Kevin Wolf wrote: > So without that, and also without a mention in deprecated.rst, I don't > think having mentioned a wish to break things a while ago means that we > should just follow through with that despite the objections. > > At least this isn't how it has worked for other patch series. If it is > the new standard, I can remove -drive tomorrow. I've been mentioning for > years that I don't like it and want to remove it, so people can just > deal with it. Come on, you're too smart to say that seriously. The deprecation policy only applies to user-visible features, users don't care if ./configure && make works. (There is precedent for that: the switch from "make subdir-*" to "make */all" was never included in user-visible documentation). I hate to say this but if the Meson conversion does not land in the next few days (I will disappear again on Saturday) I'm just going to drop it on the floor and let people have fun with their Makefiles. I have already spent the last 3 weeks on it almost full time and I cannot really justify, to both myself and my employer, spending much more time on a build system rather than fixing real bugs[1] or doing shiny new stuff. Paolo [1] Though there's plenty of known bugs in the Makefiles that are now fixed. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-08-20 15:50 ` Paolo Bonzini @ 2020-08-20 16:14 ` Kevin Wolf 2020-08-20 17:39 ` Paolo Bonzini 0 siblings, 1 reply; 53+ messages in thread From: Kevin Wolf @ 2020-08-20 16:14 UTC (permalink / raw) To: Paolo Bonzini; +Cc: Peter Maydell, QEMU Developers, Stefan Hajnoczi Am 20.08.2020 um 17:50 hat Paolo Bonzini geschrieben: > On 20/08/20 15:30, Kevin Wolf wrote: > > So without that, and also without a mention in deprecated.rst, I don't > > think having mentioned a wish to break things a while ago means that we > > should just follow through with that despite the objections. > > > > At least this isn't how it has worked for other patch series. If it is > > the new standard, I can remove -drive tomorrow. I've been mentioning for > > years that I don't like it and want to remove it, so people can just > > deal with it. > > Come on, you're too smart to say that seriously. The deprecation policy > only applies to user-visible features, users don't care if ./configure > && make works. (There is precedent for that: the switch from "make > subdir-*" to "make */all" was never included in user-visible documentation). > > I hate to say this but if the Meson conversion does not land in the next > few days (I will disappear again on Saturday) I'm just going to drop it > on the floor and let people have fun with their Makefiles. > > I have already spent the last 3 weeks on it almost full time and I > cannot really justify, to both myself and my employer, spending much > more time on a build system rather than fixing real bugs[1] or doing > shiny new stuff. I just don't understand why 99.9% of it were okay to do, but the final bit that would make the switch a lot more seamless to people is asking too much. You're familiar with the state after your patches, I'm not. I assume you're also the one who sends the pull request, and anything developed after that pull request will mean that people will already be impacted. If you ask me to do pause my work, instead familiarise myself with your work and do that final bit so that you can then include it in your pull request, I'm sure your employer will pay for more time being spent rather than less. Kevin ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-08-20 16:14 ` Kevin Wolf @ 2020-08-20 17:39 ` Paolo Bonzini 2020-08-21 9:55 ` Gerd Hoffmann 0 siblings, 1 reply; 53+ messages in thread From: Paolo Bonzini @ 2020-08-20 17:39 UTC (permalink / raw) To: Kevin Wolf; +Cc: Peter Maydell, QEMU Developers, Stefan Hajnoczi On 20/08/20 18:14, Kevin Wolf wrote: > I just don't understand why 99.9% of it were okay to do, but the final > bit that would make the switch a lot more seamless to people is asking > too much. You're familiar with the state after your patches, I'm not. I > assume you're also the one who sends the pull request, and anything > developed after that pull request will mean that people will already > be impacted. Because that 99.9% was done according to a plan of avoiding half-baked solutions, and only make decisions during the initial conversion that are maintainable in the long term. All the design and the work came from that one principle. Since total parity just cannot be attained (because Meson just hollers and exits if you try to do an in-tree build, or because Meson just places binaries mirroring the layout of source tree), anything that tries to mimic of the previous buildsystem in unsupported ways is half-baked by definition. Some half-baked solutions are easy, and can be accomodated. For example take "Meson produces qemu-system-arm and tutorials says the binary is arm-softmmu/qemu-system-arm", which Peter requested. I would have gladly done without it if it was just for Peter's muscle memory, :) but the issue with tutorials is a fair one. It's literally 3 lines of code and I can add them easily. Dropping in-tree builds is also an issue with respect to tutorials. However, dropping support for in-tree builds simply requires a good-enough warning and it can be done just before the release because people using tutorials rarely build from git master. If it's a matter of "I don't want my workflow to change too much", I'm not familiar with everybody's setups and requirements and it's not a precise-enough requirement for me to write the code (and block the inclusion of the pull request). What you're asking is not the final 0.1%; it's a completely different thing, because it _must_ be by definition half-baked. For example here are all the tradeoffs that come to mind: - given the argument about tutorials, should the in-tree "./configure && make" support extend to linking binaries back from the build tree? All binaries or only binaries that existed at the time of the conversion? Should the linked binaries be placed in arm-softmmu/qemu-system-arm or should it be in ./qemu-system-arm? Any of this choices introduces potential confusion to whoever follows a tutorial so, arguably, a warning would be better than a half-baked solution inasmuch as tutorials are concerned. - if binaries are linked back, should they be kept in .gitignore? - should distclean also be kept in GNUMakefile? (probably, since distclean is _especially_ useful for in-tree builds) - who is responsible to fix it if the support bitrots? (it will, especially since with Meson things like adding a binary only require touching one place: nobody will remember to add the binary to GNUMakefile) - should I also add a CI job that tests it? I'm rather happier to spend my time explaining details of the conversion if they have to hack on the build system in the near future, than to gather a detailed answer to these questions and any other that didn't come to mind. > If you ask me to do pause my work, instead familiarise myself with your > work and do that final bit so that you can then include it in your pull > request, I'm sure your employer will pay for more time being spent > rather than less. You don't need to familiarize with my work, you only need to familiarize yourself with out-of-tree builds which almost everyone else is using[1]. With out-of-tree builds, there is arguably no change whatsoever for day-to-day development (as long as you don't touch the build system of course). Thanks, Paolo [1] For example, among the people in this thread who were using in-tree builds, Stefan is using meson for his new project so presumably has switched to out-of-tree builds, and Eric has already switched to out-of-tree builds. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-08-20 17:39 ` Paolo Bonzini @ 2020-08-21 9:55 ` Gerd Hoffmann 0 siblings, 0 replies; 53+ messages in thread From: Gerd Hoffmann @ 2020-08-21 9:55 UTC (permalink / raw) To: Paolo Bonzini; +Cc: Kevin Wolf, Peter Maydell, QEMU Developers, Stefan Hajnoczi Hi, > If it's a matter of "I don't want my workflow to change too much", I'm > not familiar with everybody's setups and requirements and it's not a > precise-enough requirement for me to write the code (and block the > inclusion of the pull request). It's almost zero change to my workflow. I only had to fix modular builds (patch below). I'm doing out-of-tree builds for ages. First, because I consider it good practice. Second, because it allows me to easily test multiple different configurations (modular vs. non-module, with/without spice, ...) without "make distclean; ./configure $otheropts", by simply having multiple build trees. Third, because I can just "rm -rf $builddir" and start over in case I trap into one of the bugs our Makefiles have (not rebuilding objects on cflags changes for example). > What you're asking is not the final 0.1%; it's a completely different > thing, because it _must_ be by definition half-baked. For example here > are all the tradeoffs that come to mind: [ list snipped ] > I'm rather happier to spend my time explaining details of the conversion > if they have to hack on the build system in the near future, than to > gather a detailed answer to these questions and any other that didn't > come to mind. Fully agree. My personal take on this is we should just have configure catch in-tree builds, print instructions for out-of-tree builds and throw an error. Switching from in-tree to out-of-tree builds isn't that much of an issue after all, and given that our current build system supports it too even "git bisect" should be painless. > (as long as you don't touch the build system of course). And *this* should be easier once the series landed. Also I'm looking forward to have less build system bugs. take care, Gerd ------------------------------------------------------------------- diff --git a/util/module.c b/util/module.c index c956ef096b3b..6e63006a8fb2 100644 --- a/util/module.c +++ b/util/module.c @@ -113,7 +113,7 @@ static int module_load_file(const char *fname) { GModule *g_module; void (*sym)(void); - const char *dsosuf = HOST_DSOSUF; + const char *dsosuf = CONFIG_HOST_DSOSUF; int len = strlen(fname); int suf_len = strlen(dsosuf); ModuleEntry *e, *next; @@ -221,7 +221,7 @@ bool module_load_one(const char *prefix, const char *lib_name) for (i = 0; i < n_dirs; i++) { fname = g_strdup_printf("%s/%s%s", - dirs[i], module_name, HOST_DSOSUF); + dirs[i], module_name, CONFIG_HOST_DSOSUF); ret = module_load_file(fname); g_free(fname); fname = NULL; ^ permalink raw reply related [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-08-18 19:14 ` Peter Maydell 2020-08-20 10:54 ` Kevin Wolf @ 2020-08-20 16:05 ` Daniel P. Berrangé 1 sibling, 0 replies; 53+ messages in thread From: Daniel P. Berrangé @ 2020-08-20 16:05 UTC (permalink / raw) To: Peter Maydell; +Cc: Kevin Wolf, Stefan Hajnoczi, QEMU Developers, Paolo Bonzini On Tue, Aug 18, 2020 at 08:14:33PM +0100, Peter Maydell wrote: > On Mon, 23 Mar 2020 at 13:32, Stefan Hajnoczi <stefanha@gmail.com> wrote: > > On Sat, Mar 21, 2020 at 11:50:23PM +0100, BALATON Zoltan wrote: > > > This was discussed before. I think instead of annoying people with a > > > warning, rather configure should be changed to create a build dir if run > > > from source and have a Makefile in top dir that runs make -C builddir so > > > people don't have to care about this or change their ways and can continue > > > to run configure && make from source dir but you don't have to support > > > in-tree build. Then you can deprecate in-tree builds but supporting only > > > out-of-tree without this convenience would not just unnecessarily annoy > > > those who prefer working in a single tree but people (and apparently some > > > tools) expect sources to build with usual configure; make; make install so > > > that should be the minimum to support. > > > > Yes, please! I use in-tree builds and find it tedious to cd into a > > build dir manually. > > > > Also, many build scripts (packaging, etc) we'll break if we simply > > remove in-tree builds. I think make && make install should continue to > > work. > > Paolo's conversion-to-Meson patchseries is about to land, so now > is the time for people who would like this "automatically create > a build directory and use it" behaviour to write the necessary > patches. Any volunteers ? I'll try to come up with some GNUmakefile that isn't entirely awful as a starting point to at least get ./configure make make test to work, leaving symlinks to the biult ELF binaries in $PWD. We can then iterate on it if desired. > My current plan is to land the Meson series first, because it is > really painful for Paolo to try to keep rebasing it as other > changes to the old build system occur. This would break > in-tree builds temporarily until the "automatic creation and > use of a builddir" patches go in on top of it. > > thanks > -- PMM > Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 53+ messages in thread
[parent not found: <CAL1e-=gKB0qRxGntXrU0im2gjMh1tq_SLGTm+HsmgBRGXQ9+Bg@mail.gmail.com>]
* Re: deprecation of in-tree builds [not found] ` <CAL1e-=gKB0qRxGntXrU0im2gjMh1tq_SLGTm+HsmgBRGXQ9+Bg@mail.gmail.com> @ 2020-03-22 17:20 ` Peter Maydell 2020-03-22 19:51 ` Aleksandar Markovic 0 siblings, 1 reply; 53+ messages in thread From: Peter Maydell @ 2020-03-22 17:20 UTC (permalink / raw) To: Aleksandar Markovic; +Cc: Paolo Bonzini, QEMU Developers On Sun, 22 Mar 2020 at 15:29, Aleksandar Markovic <aleksandar.m.mail@gmail.com> wrote: > If the "progress" (in the form of deprecation) is so impotrant, than the authors should devise it so that there is no dammage to existing features, and no adverse effects. > > In this light, perhaps in-tree builds deorecation is 5.0 is little premature. The idea of deprecation is to give advance warning. So it's better for our users if we announce it earlier, rather than later. Strictly speaking our deprecation policy is for user-facing features, not build-time stuff, where we are less strict about how much notice we give people. But it seems to me that if it's easy to give some advance notice then why shouldn't we do so? I agree that we should obviously make sure that everything that currently assumes an in-tree build also works with an out-of-tree build before we drop the support... (Also, if we don't announce that we're planning to drop support, nobody's going to report to us issues which we need to fix :-)) thanks -- PMM ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-22 17:20 ` Peter Maydell @ 2020-03-22 19:51 ` Aleksandar Markovic 2020-03-22 20:14 ` Peter Maydell 0 siblings, 1 reply; 53+ messages in thread From: Aleksandar Markovic @ 2020-03-22 19:51 UTC (permalink / raw) To: Peter Maydell; +Cc: Paolo Bonzini, QEMU Developers, Aleksandar Markovic [-- Attachment #1: Type: text/plain, Size: 2150 bytes --] 18:21 Ned, 22.03.2020. Peter Maydell <peter.maydell@linaro.org> је написао/ла: > > On Sun, 22 Mar 2020 at 15:29, Aleksandar Markovic > <aleksandar.m.mail@gmail.com> wrote: > > If the "progress" (in the form of deprecation) is so impotrant, than the authors should devise it so that there is no dammage to existing features, and no adverse effects. > > > > In this light, perhaps in-tree builds deorecation is 5.0 is little premature. > > The idea of deprecation is to give advance warning. So it's > better for our users if we announce it earlier, rather than > later. Strictly speaking our deprecation policy is for > user-facing features, not build-time stuff, where we are... If an end-user feature works only in in-tree builds (so, explitely said: not in out-of-tree builds), this is not a build-time stuff, but user-facing feature issue. If someone is keen on removing any feature (as is truly in this case), I expect at least some moderate investigation being done on what could be affected (prior to announcing deprecation), rather than attitude "ok, let's announce deprecation, see if someone start clamoring, and, if not, we are good to go with removing". For me, this slightly disappointing. I haven't seen anyone doing a sufficiently thourough analysis on what happens without in-tree builds, and doesn't work in out-of-tree builds in a proper way. But, ok, this is just my opinion, probably unpopular within dev comunity, since it requires additional effort by us. Still, we should guide ourself with "what is right to do", and not "what is easy to do" principles. Regards, Aleksandar > less strict about how much notice we give people. But it > seems to me that if it's easy to give some advance notice > then why shouldn't we do so? > > I agree that we should obviously make sure that everything > that currently assumes an in-tree build also works with an > out-of-tree build before we drop the support... > > (Also, if we don't announce that we're planning to drop > support, nobody's going to report to us issues which > we need to fix :-)) > > thanks > -- PMM > [-- Attachment #2: Type: text/html, Size: 2650 bytes --] ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-22 19:51 ` Aleksandar Markovic @ 2020-03-22 20:14 ` Peter Maydell 2020-03-22 20:45 ` Aleksandar Markovic ` (2 more replies) 0 siblings, 3 replies; 53+ messages in thread From: Peter Maydell @ 2020-03-22 20:14 UTC (permalink / raw) To: Aleksandar Markovic; +Cc: Paolo Bonzini, QEMU Developers, Aleksandar Markovic On Sun, 22 Mar 2020 at 19:52, Aleksandar Markovic <aleksandar.qemu.devel@gmail.com> wrote: > If an end-user feature works only in in-tree builds (so, > explitely said: not in out-of-tree builds), this is not > a build-time stuff, but user-facing feature issue. gprof is a developer feature, not an end-user-facing feature. By the latter I mean "some feature that users who have installed a built binary might be using": command line stuff, actual functionality in the QEMU binary, QMP protocol, that kind of thing. > If someone is keen on removing any feature (as is truly in this case), I expect at least some moderate investigation being done on what could be affected (prior to announcing deprecation), rather than attitude "ok, let's announce deprecation, see if someone start clamoring, and, if not, we are good to go with removing". For me, this slightly disappointing. Before you told me about the gprof issue, the *only* thing I was aware of that might break was the coverity scan build, which is a purely project internal bit of infrastructure. From my point of view, we did the investigation, in the sense that for years we have had out-of-tree as our standard recommended way of building QEMU and the thing we test in our CI. Anything that breaks out-of-tree is by definition something that fell through the gaps in our CI and which we couldn't know about unless somebody tells us about it. The "announce deprecation" part is the final part of the process, and sometimes it does, yes, result in somebody saying "you missed this thing", because we know our CI is far from perfect. > I haven't seen anyone doing a sufficiently thourough analysis >on what happens without in-tree builds, and doesn't work in >out-of-tree builds in a proper way. *Everything* is supposed to work in out of tree builds. If it doesn't that's a bug -- unless people report bugs we'll never know to fix them. Most developers use out of tree builds and all our CI is out of tree builds, so they actually get better ad-hoc and CI coverage than in-tree. Out-of-tree is overwhelmingly what we build and what we test, so it's in-tree that breaks more often and where I'd expect to find more things we didn't realise were broken. To be clear, I'm not saying we should pull the rug out from anybody. I'm saying: * we should clearly say what our plans are, with a long warning if we can reasonably give longer warning * if there's anything that we would accidentally be breaking with those plans, we should adjust the plans so we don't break things we didn't mean to break This doesn't seem controversial to me... thanks -- PMM ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-22 20:14 ` Peter Maydell @ 2020-03-22 20:45 ` Aleksandar Markovic 2020-03-22 20:46 ` BALATON Zoltan 2020-03-23 10:20 ` Daniel P. Berrangé 2 siblings, 0 replies; 53+ messages in thread From: Aleksandar Markovic @ 2020-03-22 20:45 UTC (permalink / raw) To: Peter Maydell; +Cc: Paolo Bonzini, QEMU Developers, Aleksandar Markovic [-- Attachment #1: Type: text/plain, Size: 1431 bytes --] 21:14 Ned, 22.03.2020. Peter Maydell > *Everything* is supposed to work in out of tree builds. > If it doesn't that's a bug -- unless people report bugs > we'll never know to fix them. Most developers use out > of tree builds and all our CI is out of tree builds, so > they actually get better ad-hoc and CI coverage than > in-tree. Out-of-tree is overwhelmingly what we build and > what we test, so it's in-tree that breaks more often and > where I'd expect to find more things we didn't realise > were broken. > > To be clear, I'm not saying we should pull the rug out > from anybody. I'm saying: > * we should clearly say what our plans are, with a > long warning if we can reasonably give longer warning > * if there's anything that we would accidentally > be breaking with those plans, we should adjust the > plans so we don't break things we didn't mean to break > > This doesn't seem controversial to me... > OK, given all info you presented in last paragraphs and elsewhere - that seems to have more emphasis on potential adjustments, and the obligatory condition that nothing breaks - I agree with the approach you spelled out, or, in other words, agree with introducing deprecation note. I hope that we all perceive occasional differences in opinions as our value (and, I even claim, a key to success of any dev community), and not a nuisance or a danger. Thanks! :) Regards, Aleksandar > thanks > -- PMM [-- Attachment #2: Type: text/html, Size: 1773 bytes --] ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-22 20:14 ` Peter Maydell 2020-03-22 20:45 ` Aleksandar Markovic @ 2020-03-22 20:46 ` BALATON Zoltan 2020-03-22 21:15 ` Peter Maydell 2020-03-23 10:20 ` Daniel P. Berrangé 2 siblings, 1 reply; 53+ messages in thread From: BALATON Zoltan @ 2020-03-22 20:46 UTC (permalink / raw) To: Peter Maydell Cc: Paolo Bonzini, Aleksandar Markovic, QEMU Developers, Aleksandar Markovic On Sun, 22 Mar 2020, Peter Maydell wrote: > On Sun, 22 Mar 2020 at 19:52, Aleksandar Markovic > <aleksandar.qemu.devel@gmail.com> wrote: >> If an end-user feature works only in in-tree builds (so, >> explitely said: not in out-of-tree builds), this is not >> a build-time stuff, but user-facing feature issue. > > gprof is a developer feature, not an end-user-facing > feature. By the latter I mean "some feature that users > who have installed a built binary might be using": > command line stuff, actual functionality in the QEMU > binary, QMP protocol, that kind of thing. > >> If someone is keen on removing any feature (as is truly in this case), I expect at least some moderate investigation being done on what could be affected (prior to announcing deprecation), rather than attitude "ok, let's announce deprecation, see if someone start clamoring, and, if not, we are good to go with removing". For me, this slightly disappointing. > > Before you told me about the gprof issue, the *only* thing Was that gprof or gcov? > I was aware of that might break was the coverity scan build, > which is a purely project internal bit of infrastructure. Plus potentially any scripts people might use to build stuff and distro packagers that might use in-tree build. They would suddently find their previously working scripts are now broken and they need to adapt. While making sure running configure; make; make install in source tree even if it actually does a build in a new build dir it creates automatically would be less annoying change than having to manually manage out-of-tree build dirs by those who did not do that so far. Is it really that difficult to add a CI job to do a git clone then configure; make; make install in it to make sure it breaks less often? And to make sure this still works after in-tree builds are deprecated and removed? I think we can't check every distro packager or don't know what users do but supporting the usual way of building packages used by many may worth the little extra effort to not annoy users/developers unnecessarily. Regards, BALATON Zoltan ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-22 20:46 ` BALATON Zoltan @ 2020-03-22 21:15 ` Peter Maydell 2020-03-23 0:35 ` BALATON Zoltan 0 siblings, 1 reply; 53+ messages in thread From: Peter Maydell @ 2020-03-22 21:15 UTC (permalink / raw) To: BALATON Zoltan Cc: Paolo Bonzini, Aleksandar Markovic, QEMU Developers, Aleksandar Markovic On Sun, 22 Mar 2020 at 20:46, BALATON Zoltan <balaton@eik.bme.hu> wrote: > On Sun, 22 Mar 2020, Peter Maydell wrote: > > Before you told me about the gprof issue, the *only* thing > > Was that gprof or gcov? Sorry, gcov; I always get those two mixed up in my head. > Plus potentially any scripts people might use to build stuff and distro > packagers that might use in-tree build. They would suddently find their > previously working scripts are now broken and they need to adapt. It is to avoid the "suddenly" part that we announce in advance that features are going away :-) More generally, distro packagers must adapt for any new QEMU release -- new features appear that they may need to update their dependency lists to handle, old features are sometimes removed and the corresponding configure --enable-foo options stop working, existing features need new dependencies. Also, we've been recommending out-of-tree builds in our README since 2015. They're hardly a new thing. > While > making sure running configure; make; make install in source tree even if > it actually does a build in a new build dir it creates automatically would > be less annoying change than having to manually manage out-of-tree build > dirs by those who did not do that so far. > > Is it really that difficult to add a CI job to do a git clone then > configure; make; make install in it to make sure it breaks less often? To be honest, I don't feel very strongly here, except that I didn't want us to drop in-tree builds without noting it in the release notes, and my impression from previous list discussions was that that was the way the project was heading. If somebody wants to write patches to cause 'configure' to create a new build tree that's OK I guess (though I'd be dubious because I think that hidden magic like that is overall often going to confuse people, and it's still extra machinery in the makefile and the configure script). But I don't really see much point in maintaining two different mechanisms which add complication and where one of them is just not overall as useful as the other. I fairly often see posts from people on eg stackoverflow who are trying to compile and modify QEMU, and they're usually using in-tree build and I usually mention in a PS to answering their question that they'd really be better off with an out-of-tree build. I think we should stop making it easy to default to a setup that we don't recommend. thanks -- PMM ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-22 21:15 ` Peter Maydell @ 2020-03-23 0:35 ` BALATON Zoltan 0 siblings, 0 replies; 53+ messages in thread From: BALATON Zoltan @ 2020-03-23 0:35 UTC (permalink / raw) To: Peter Maydell Cc: Paolo Bonzini, Aleksandar Markovic, QEMU Developers, Aleksandar Markovic On Sun, 22 Mar 2020, Peter Maydell wrote: > On Sun, 22 Mar 2020 at 20:46, BALATON Zoltan <balaton@eik.bme.hu> wrote: >> On Sun, 22 Mar 2020, Peter Maydell wrote: >>> Before you told me about the gprof issue, the *only* thing >> >> Was that gprof or gcov? > > Sorry, gcov; I always get those two mixed up in my head. > >> Plus potentially any scripts people might use to build stuff and distro >> packagers that might use in-tree build. They would suddently find their >> previously working scripts are now broken and they need to adapt. > > It is to avoid the "suddenly" part that we announce in advance > that features are going away :-) More generally, distro packagers People usually don't read docs so they'll find out "suddenly" anyway... > must adapt for any new QEMU release -- new features appear that > they may need to update their dependency lists to handle, old > features are sometimes removed and the corresponding configure > --enable-foo options stop working, existing features need new > dependencies. It's true they'll occasionally have to adapt and probably most packagers already use out-of-tree builds but if there's something that can make their life easier without putting too much burden on QEMU I think it's a good thing to make it convenient for people compiling QEMU. > If somebody wants to write patches to cause 'configure' to create > a new build tree that's OK I guess (though I'd be dubious because > I think that hidden magic like that is overall often going > to confuse people, and it's still extra machinery in the > makefile and the configure script). But I don't really see > much point in maintaining two different mechanisms which add > complication and where one of them is just not overall as useful > as the other. A convenience Makefile in top level to call make -C builddir and maybe a few lines in configure to create it does not seem to be too much extra machinery but I don't know the build system that well. Also it does not have to be hidden, it can print a message to user to tell it created a build dir and that the build results are found there. It's probably less confusing to people who never used out-of-tree builds before and relieves them from having to learn something new which a lot of people like to avoid if possible. > I fairly often see posts from people on eg stackoverflow > who are trying to compile and modify QEMU, and they're > usually using in-tree build and I usually mention in a > PS to answering their question that they'd really be > better off with an out-of-tree build. I think we should > stop making it easy to default to a setup that we don't > recommend. I think this proves my point that a lot of people expect this to work and the answer should not be to annoy them and force them to change their ways but to support it in some way. If this is a problem with the make system then auto-creating build dir could avoid this problem without imposing things on people so if it's not too much effort it's probably worth doing. Regards, BALATON Zoltan ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-22 20:14 ` Peter Maydell 2020-03-22 20:45 ` Aleksandar Markovic 2020-03-22 20:46 ` BALATON Zoltan @ 2020-03-23 10:20 ` Daniel P. Berrangé 2 siblings, 0 replies; 53+ messages in thread From: Daniel P. Berrangé @ 2020-03-23 10:20 UTC (permalink / raw) To: Peter Maydell Cc: Paolo Bonzini, Aleksandar Markovic, QEMU Developers, Aleksandar Markovic On Sun, Mar 22, 2020 at 08:14:24PM +0000, Peter Maydell wrote: > To be clear, I'm not saying we should pull the rug out > from anybody. I'm saying: > * we should clearly say what our plans are, with a > long warning if we can reasonably give longer warning > * if there's anything that we would accidentally > be breaking with those plans, we should adjust the > plans so we don't break things we didn't mean to break > > This doesn't seem controversial to me... IIUC, all of our CI platofrms are testing out of tree builds. This gives us confidence that all the important aspects of QEMU build process are operating as expected. It also means that anyone relying on in-tree builds is already using a process that is a second-class citizen in terms of supportability. We've already seen bit-rot in in-tree builds due to this lack of formal testing. If there are things that we discover to be broken with out of tree builds, this is highlighting gaps in our CI coverage we'll want to address. This will bring said broken feature into a situation where we have confidence in its usage going forward. IOW, I see no downside to deprecating in-tree builds - it is just a reflection of how we have already considered them to be untested and undesirable to use. We still have a few places in docs/ that are illustrating "./configure" that we should fix too... Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-21 20:50 deprecation of in-tree builds Peter Maydell 2020-03-21 22:50 ` BALATON Zoltan [not found] ` <CAL1e-=gKB0qRxGntXrU0im2gjMh1tq_SLGTm+HsmgBRGXQ9+Bg@mail.gmail.com> @ 2020-03-30 13:26 ` Markus Armbruster 2020-03-30 13:31 ` Peter Maydell 2020-03-31 10:38 ` Daniel P. Berrangé 3 siblings, 1 reply; 53+ messages in thread From: Markus Armbruster @ 2020-03-30 13:26 UTC (permalink / raw) To: Peter Maydell; +Cc: Paolo Bonzini, QEMU Developers Peter Maydell <peter.maydell@linaro.org> writes: > AIUI from Paolo, the intention is to deprecate and eventually > stop supporting "in-tree" builds, so that the only option is > building in a separate build directory. I thought we should > probably mention that in the 5.0 changelog, so I wrote up some > text: > > https://wiki.qemu.org/ChangeLog/5.0#Build_Information Overdue. Thanks for doing this! > Suggestions for changes/comments etc welcome. Looks fine to me. > (One thing we will need to fix before we can do separate build > tree is the Coverity Scan build process, which (a) does an > in-tree build (b) can't be easily switched to a builddir because > all the source paths get baked into the scan results and moving > to a builddir changes them all...) > > We could also make configure actively warn if used in > the source tree. Unless we commit to automatically creating a default out-of-tree build directory for this usage. I wouldn't bother myself. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-30 13:26 ` Markus Armbruster @ 2020-03-30 13:31 ` Peter Maydell 2020-03-30 13:42 ` Daniel P. Berrangé ` (2 more replies) 0 siblings, 3 replies; 53+ messages in thread From: Peter Maydell @ 2020-03-30 13:31 UTC (permalink / raw) To: Markus Armbruster; +Cc: Paolo Bonzini, QEMU Developers On Mon, 30 Mar 2020 at 14:26, Markus Armbruster <armbru@redhat.com> wrote: > > Peter Maydell <peter.maydell@linaro.org> writes: > > > AIUI from Paolo, the intention is to deprecate and eventually > > stop supporting "in-tree" builds, so that the only option is > > building in a separate build directory. I thought we should > > probably mention that in the 5.0 changelog, so I wrote up some > > text: > > > > https://wiki.qemu.org/ChangeLog/5.0#Build_Information > > Overdue. Thanks for doing this! > > > Suggestions for changes/comments etc welcome. > > Looks fine to me. Consensus in the thread seemed to lean towards having the 'configure/make' runes auto-create a build directory; if we want to do that we should probably not say anything in the release notes, because we'll cause people to change unnecessarily. Or at least have them say "We recommend out-of-tree builds. In future we might make the commands that currently do an in-tree build automatically create and use a build directory for you." rather than a blanket "we're going to drop this and you should change what you do now". Thoughts? thanks -- PMM ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-30 13:31 ` Peter Maydell @ 2020-03-30 13:42 ` Daniel P. Berrangé 2020-03-30 14:37 ` Kevin Wolf 2020-03-30 16:25 ` Aleksandar Markovic 2020-03-31 7:15 ` Markus Armbruster 2 siblings, 1 reply; 53+ messages in thread From: Daniel P. Berrangé @ 2020-03-30 13:42 UTC (permalink / raw) To: Peter Maydell; +Cc: Paolo Bonzini, Markus Armbruster, QEMU Developers On Mon, Mar 30, 2020 at 02:31:52PM +0100, Peter Maydell wrote: > On Mon, 30 Mar 2020 at 14:26, Markus Armbruster <armbru@redhat.com> wrote: > > > > Peter Maydell <peter.maydell@linaro.org> writes: > > > > > AIUI from Paolo, the intention is to deprecate and eventually > > > stop supporting "in-tree" builds, so that the only option is > > > building in a separate build directory. I thought we should > > > probably mention that in the 5.0 changelog, so I wrote up some > > > text: > > > > > > https://wiki.qemu.org/ChangeLog/5.0#Build_Information > > > > Overdue. Thanks for doing this! > > > > > Suggestions for changes/comments etc welcome. > > > > Looks fine to me. > > Consensus in the thread seemed to lean towards having > the 'configure/make' runes auto-create a build directory; > if we want to do that we should probably not say anything in > the release notes, because we'll cause people to change > unnecessarily. Or at least have them say "We recommend > out-of-tree builds. In future we might make the commands > that currently do an in-tree build automatically create > and use a build directory for you." rather than a blanket > "we're going to drop this and you should change what you > do now". > > Thoughts? When the time comes to switch to Meson, we'll be forcing a separate build directory & that will also potentially involve a change in build instructions for users. So if we expect Meson will arrive in QEMU soon (say before end of this year), then I'd try to keep changes related to make to the bare minimum, and have Meson switch as the "flag day" when developers are forced to adapt their build process. IOW, merely warn people that in-tree builds are deprecated and not tested by the project and enourage switching to out of tree builds. Dont try to magically create a separate build dir now. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-30 13:42 ` Daniel P. Berrangé @ 2020-03-30 14:37 ` Kevin Wolf 2020-03-30 17:29 ` Michal Suchánek 2020-03-31 7:48 ` Paolo Bonzini 0 siblings, 2 replies; 53+ messages in thread From: Kevin Wolf @ 2020-03-30 14:37 UTC (permalink / raw) To: Daniel P. Berrangé Cc: QEMU Developers, Peter Maydell, Markus Armbruster, Paolo Bonzini Am 30.03.2020 um 15:42 hat Daniel P. Berrangé geschrieben: > On Mon, Mar 30, 2020 at 02:31:52PM +0100, Peter Maydell wrote: > > On Mon, 30 Mar 2020 at 14:26, Markus Armbruster <armbru@redhat.com> wrote: > > > > > > Peter Maydell <peter.maydell@linaro.org> writes: > > > > > > > AIUI from Paolo, the intention is to deprecate and eventually > > > > stop supporting "in-tree" builds, so that the only option is > > > > building in a separate build directory. I thought we should > > > > probably mention that in the 5.0 changelog, so I wrote up some > > > > text: > > > > > > > > https://wiki.qemu.org/ChangeLog/5.0#Build_Information > > > > > > Overdue. Thanks for doing this! > > > > > > > Suggestions for changes/comments etc welcome. > > > > > > Looks fine to me. > > > > Consensus in the thread seemed to lean towards having > > the 'configure/make' runes auto-create a build directory; > > if we want to do that we should probably not say anything in > > the release notes, because we'll cause people to change > > unnecessarily. Or at least have them say "We recommend > > out-of-tree builds. In future we might make the commands > > that currently do an in-tree build automatically create > > and use a build directory for you." rather than a blanket > > "we're going to drop this and you should change what you > > do now". > > > > Thoughts? > > When the time comes to switch to Meson, we'll be forcing a > separate build directory & that will also potentially involve > a change in build instructions for users. > > So if we expect Meson will arrive in QEMU soon (say before end > of this year), then I'd try to keep changes related to make to > the bare minimum, and have Meson switch as the "flag day" when > developers are forced to adapt their build process. > > IOW, merely warn people that in-tree builds are deprecated and > not tested by the project and enourage switching to out of tree > builds. Dont try to magically create a separate build dir now. If manually dealing with separate build directories is inconvenient today, it will still be inconvenient with Meson, so this would mean introducing the automatic directly creation together with the other changes to enable Meson. Which is fine by me, as long as it is really done when the external directory becomes mandatory, so that people won't have to switch back and forth between directories. Kevin ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-30 14:37 ` Kevin Wolf @ 2020-03-30 17:29 ` Michal Suchánek 2020-03-30 17:36 ` Daniel P. Berrangé 2020-03-30 20:15 ` BALATON Zoltan 2020-03-31 7:48 ` Paolo Bonzini 1 sibling, 2 replies; 53+ messages in thread From: Michal Suchánek @ 2020-03-30 17:29 UTC (permalink / raw) To: Kevin Wolf Cc: Peter Maydell, Paolo Bonzini, Daniel P. Berrangé, QEMU Developers, Markus Armbruster On Mon, Mar 30, 2020 at 04:37:59PM +0200, Kevin Wolf wrote: > Am 30.03.2020 um 15:42 hat Daniel P. Berrangé geschrieben: > > On Mon, Mar 30, 2020 at 02:31:52PM +0100, Peter Maydell wrote: > > > On Mon, 30 Mar 2020 at 14:26, Markus Armbruster <armbru@redhat.com> wrote: > > > > > > > > Peter Maydell <peter.maydell@linaro.org> writes: > > > > > > > > > AIUI from Paolo, the intention is to deprecate and eventually > > > > > stop supporting "in-tree" builds, so that the only option is > > > > > building in a separate build directory. I thought we should > > > > > probably mention that in the 5.0 changelog, so I wrote up some > > > > > text: > > > > > > > > > > https://wiki.qemu.org/ChangeLog/5.0#Build_Information > > > > > > > > Overdue. Thanks for doing this! > > > > > > > > > Suggestions for changes/comments etc welcome. > > > > > > > > Looks fine to me. > > > > > > Consensus in the thread seemed to lean towards having > > > the 'configure/make' runes auto-create a build directory; > > > if we want to do that we should probably not say anything in > > > the release notes, because we'll cause people to change > > > unnecessarily. Or at least have them say "We recommend > > > out-of-tree builds. In future we might make the commands > > > that currently do an in-tree build automatically create > > > and use a build directory for you." rather than a blanket > > > "we're going to drop this and you should change what you > > > do now". > > > > > > Thoughts? > > > > When the time comes to switch to Meson, we'll be forcing a > > separate build directory & that will also potentially involve > > a change in build instructions for users. > > > > So if we expect Meson will arrive in QEMU soon (say before end > > of this year), then I'd try to keep changes related to make to > > the bare minimum, and have Meson switch as the "flag day" when > > developers are forced to adapt their build process. > > > > IOW, merely warn people that in-tree builds are deprecated and > > not tested by the project and enourage switching to out of tree > > builds. Dont try to magically create a separate build dir now. > > If manually dealing with separate build directories is inconvenient > today, it will still be inconvenient with Meson, so this would mean > introducing the automatic directly creation together with the other > changes to enable Meson. Which is fine by me, as long as it is really > done when the external directory becomes mandatory, so that people won't > have to switch back and forth between directories. From packaging point of view if something has auto* files the default is IIRC to do in-tree build with most package managers. You have to adjust manually for out-of-tree build. I am not sure what is state of Meson support in packaging scripts across distributions. Most likely it will not be picked automagically so you have to provide a custom build script anyway. Then it is the time to look up the canonical build instructions. In other words dealing with Meson is inconvenient either way. Building in-tree or out-of-tree becomes a non-issue whan you switch to exotic build system anyway. That's not to say that auto* tools are great. It's just the replacements have issues that are not well understood by average Joe Packager while the auto* tools have good old issues people know how to deal with. Thanks Michal ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-30 17:29 ` Michal Suchánek @ 2020-03-30 17:36 ` Daniel P. Berrangé 2020-03-30 20:15 ` BALATON Zoltan 1 sibling, 0 replies; 53+ messages in thread From: Daniel P. Berrangé @ 2020-03-30 17:36 UTC (permalink / raw) To: Michal Suchánek Cc: Kevin Wolf, Peter Maydell, Paolo Bonzini, QEMU Developers, Markus Armbruster On Mon, Mar 30, 2020 at 07:29:55PM +0200, Michal Suchánek wrote: > On Mon, Mar 30, 2020 at 04:37:59PM +0200, Kevin Wolf wrote: > > Am 30.03.2020 um 15:42 hat Daniel P. Berrangé geschrieben: > > > On Mon, Mar 30, 2020 at 02:31:52PM +0100, Peter Maydell wrote: > > > > On Mon, 30 Mar 2020 at 14:26, Markus Armbruster <armbru@redhat.com> wrote: > > > > > > > > > > Peter Maydell <peter.maydell@linaro.org> writes: > > > > > > > > > > > AIUI from Paolo, the intention is to deprecate and eventually > > > > > > stop supporting "in-tree" builds, so that the only option is > > > > > > building in a separate build directory. I thought we should > > > > > > probably mention that in the 5.0 changelog, so I wrote up some > > > > > > text: > > > > > > > > > > > > https://wiki.qemu.org/ChangeLog/5.0#Build_Information > > > > > > > > > > Overdue. Thanks for doing this! > > > > > > > > > > > Suggestions for changes/comments etc welcome. > > > > > > > > > > Looks fine to me. > > > > > > > > Consensus in the thread seemed to lean towards having > > > > the 'configure/make' runes auto-create a build directory; > > > > if we want to do that we should probably not say anything in > > > > the release notes, because we'll cause people to change > > > > unnecessarily. Or at least have them say "We recommend > > > > out-of-tree builds. In future we might make the commands > > > > that currently do an in-tree build automatically create > > > > and use a build directory for you." rather than a blanket > > > > "we're going to drop this and you should change what you > > > > do now". > > > > > > > > Thoughts? > > > > > > When the time comes to switch to Meson, we'll be forcing a > > > separate build directory & that will also potentially involve > > > a change in build instructions for users. > > > > > > So if we expect Meson will arrive in QEMU soon (say before end > > > of this year), then I'd try to keep changes related to make to > > > the bare minimum, and have Meson switch as the "flag day" when > > > developers are forced to adapt their build process. > > > > > > IOW, merely warn people that in-tree builds are deprecated and > > > not tested by the project and enourage switching to out of tree > > > builds. Dont try to magically create a separate build dir now. > > > > If manually dealing with separate build directories is inconvenient > > today, it will still be inconvenient with Meson, so this would mean > > introducing the automatic directly creation together with the other > > changes to enable Meson. Which is fine by me, as long as it is really > > done when the external directory becomes mandatory, so that people won't > > have to switch back and forth between directories. > > From packaging point of view if something has auto* files the default is > IIRC to do in-tree build with most package managers. You have to adjust > manually for out-of-tree build. I am not sure what is state of Meson > support in packaging scripts across distributions. Most likely it will > not be picked automagically so you have to provide a custom build script > anyway. Then it is the time to look up the canonical build instructions. Many big projects already use Meson, including SystemD and GNOME for example. So I'd expect all modern distros would have standard "best practices" defined for building projects using Meson by now. Doing an out of tree build for autotools based projects doesn't take much adaption by distros either from what we got in terms of feedback when libvirt made the switch to force out-of-tree. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-30 17:29 ` Michal Suchánek 2020-03-30 17:36 ` Daniel P. Berrangé @ 2020-03-30 20:15 ` BALATON Zoltan 1 sibling, 0 replies; 53+ messages in thread From: BALATON Zoltan @ 2020-03-30 20:15 UTC (permalink / raw) To: Michal Suchánek Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrangé, QEMU Developers, Markus Armbruster, Paolo Bonzini [-- Attachment #1: Type: text/plain, Size: 3791 bytes --] On Mon, 30 Mar 2020, Michal Suchánek wrote: > On Mon, Mar 30, 2020 at 04:37:59PM +0200, Kevin Wolf wrote: >> Am 30.03.2020 um 15:42 hat Daniel P. Berrangé geschrieben: >>> On Mon, Mar 30, 2020 at 02:31:52PM +0100, Peter Maydell wrote: >>>> On Mon, 30 Mar 2020 at 14:26, Markus Armbruster <armbru@redhat.com> wrote: >>>>> >>>>> Peter Maydell <peter.maydell@linaro.org> writes: >>>>> >>>>>> AIUI from Paolo, the intention is to deprecate and eventually >>>>>> stop supporting "in-tree" builds, so that the only option is >>>>>> building in a separate build directory. I thought we should >>>>>> probably mention that in the 5.0 changelog, so I wrote up some >>>>>> text: >>>>>> >>>>>> https://wiki.qemu.org/ChangeLog/5.0#Build_Information >>>>> >>>>> Overdue. Thanks for doing this! >>>>> >>>>>> Suggestions for changes/comments etc welcome. >>>>> >>>>> Looks fine to me. >>>> >>>> Consensus in the thread seemed to lean towards having >>>> the 'configure/make' runes auto-create a build directory; >>>> if we want to do that we should probably not say anything in >>>> the release notes, because we'll cause people to change >>>> unnecessarily. Or at least have them say "We recommend >>>> out-of-tree builds. In future we might make the commands >>>> that currently do an in-tree build automatically create >>>> and use a build directory for you." rather than a blanket >>>> "we're going to drop this and you should change what you >>>> do now". >>>> >>>> Thoughts? >>> >>> When the time comes to switch to Meson, we'll be forcing a >>> separate build directory & that will also potentially involve >>> a change in build instructions for users. >>> >>> So if we expect Meson will arrive in QEMU soon (say before end >>> of this year), then I'd try to keep changes related to make to >>> the bare minimum, and have Meson switch as the "flag day" when >>> developers are forced to adapt their build process. >>> >>> IOW, merely warn people that in-tree builds are deprecated and >>> not tested by the project and enourage switching to out of tree >>> builds. Dont try to magically create a separate build dir now. >> >> If manually dealing with separate build directories is inconvenient >> today, it will still be inconvenient with Meson, so this would mean >> introducing the automatic directly creation together with the other >> changes to enable Meson. Which is fine by me, as long as it is really >> done when the external directory becomes mandatory, so that people won't >> have to switch back and forth between directories. > > From packaging point of view if something has auto* files the default is > IIRC to do in-tree build with most package managers. You have to adjust > manually for out-of-tree build. I am not sure what is state of Meson > support in packaging scripts across distributions. Most likely it will > not be picked automagically so you have to provide a custom build script > anyway. Then it is the time to look up the canonical build instructions. > > In other words dealing with Meson is inconvenient either way. Building > in-tree or out-of-tree becomes a non-issue whan you switch to exotic > build system anyway. > > That's not to say that auto* tools are great. It's just the replacements > have issues that are not well understood by average Joe Packager while > the auto* tools have good old issues people know how to deal with. Actually QEMU does not use auto* tools, it just has a configure script that mimics that. This may not be the best way to do it but this is convenient for people trying to compile it and works as expected so I think this behaviour should be preserved if possible just to not upset developers and packagers unnecessarily. Regards, BALATON Zoltan ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-30 14:37 ` Kevin Wolf 2020-03-30 17:29 ` Michal Suchánek @ 2020-03-31 7:48 ` Paolo Bonzini 2020-03-31 9:20 ` Liviu Ionescu ` (2 more replies) 1 sibling, 3 replies; 53+ messages in thread From: Paolo Bonzini @ 2020-03-31 7:48 UTC (permalink / raw) To: Kevin Wolf, Daniel P. Berrangé Cc: Peter Maydell, Markus Armbruster, QEMU Developers On 30/03/20 16:37, Kevin Wolf wrote: > If manually dealing with separate build directories is inconvenient > today, it will still be inconvenient with Meson, so this would mean > introducing the automatic directly creation together with the other > changes to enable Meson. Which is fine by me, as long as it is really > done when the external directory becomes mandatory, so that people won't > have to switch back and forth between directories. Serious question: why is automatic directly creation more convenient for developers? Even if "./configure" generates a "build" directory for you, you would still have to invoke the QEMU binary as "build/x86_64-softmmu/qemu-system-x86_64". That is less convenient than doing "mkdir build" in the first place. I can see why it's more convenient for packaging, as they just invoke "make" and "make install", but as far as developers are concerned it seems to add complexity for little or no gain. Thanks, Paolo ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-31 7:48 ` Paolo Bonzini @ 2020-03-31 9:20 ` Liviu Ionescu 2020-03-31 10:19 ` Peter Maydell 2020-03-31 12:02 ` Kevin Wolf 2020-03-31 12:54 ` BALATON Zoltan 2 siblings, 1 reply; 53+ messages in thread From: Liviu Ionescu @ 2020-03-31 9:20 UTC (permalink / raw) To: Paolo Bonzini Cc: Kevin Wolf, Peter Maydell, "Daniel P. Berrangé", Markus Armbruster, QEMU Developers > On 31 Mar 2020, at 10:48, Paolo Bonzini <pbonzini@redhat.com> wrote: > > On 30/03/20 16:37, Kevin Wolf wrote: >> If manually dealing with separate build directories is inconvenient >> today, It may be for some, but it isn't for all, perhaps we should not generalise and get the wrong conclusions. In the xPack Build Box (a set of Docker images) that I use for the binary xPack builds, I probably have some tens of libraries and tools, and most of them use separate build folders, no problem with this. With these docker images I build, on the same 64-bit linux machine, multi-platform QEMU binaries (and other complicated things like toolchains), and for them I use separate build folders for each target platform (win32, win64, linux32, linux64) and a common source folder. >> it will still be inconvenient with Meson, so this would mean >> introducing the automatic directly creation together with the other >> changes to enable Meson. Which is fine by me, as long as it is really >> done when the external directory becomes mandatory, so that people won't >> have to switch back and forth between directories. > > Serious question: why is automatic directly creation more convenient for > developers? For my use case it isn't, automatic creation will break my folder structure and I'll not be able to use the common source folder and multiple target build folders structure that I use now. > Even if "./configure" generates a "build" directory for > you, you would still have to invoke the QEMU binary as > "build/x86_64-softmmu/qemu-system-x86_64". That is less convenient than > doing "mkdir build" in the first place. I agree, if 'build' can be anywhere, not necessarily inside the source tree. I just tested, and meson creates the build folder where it is instructed to, it can be inside or outside the source folder, so I think that the QEMU build procedure should preserve his freedom, and not enforced the use of a specific folder, otherwise performing multi-platform builds will be more complicated than necessary. Regards, Liviu ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-31 9:20 ` Liviu Ionescu @ 2020-03-31 10:19 ` Peter Maydell 2020-03-31 11:46 ` Liviu Ionescu 0 siblings, 1 reply; 53+ messages in thread From: Peter Maydell @ 2020-03-31 10:19 UTC (permalink / raw) To: Liviu Ionescu Cc: Kevin Wolf, Paolo Bonzini, Daniel P. Berrangé, Markus Armbruster, QEMU Developers On Tue, 31 Mar 2020 at 10:20, Liviu Ionescu <ilg@livius.net> wrote: > > On 31 Mar 2020, at 10:48, Paolo Bonzini <pbonzini@redhat.com> wrote: > > Serious question: why is automatic directly creation more convenient for > > developers? > > For my use case it isn't, automatic creation will break my folder structure and I'll not be able to use the common source folder and multiple target build folders structure that I use now. I think that's a setup that a lot of us use, and we don't plan to change it. The proposed "automatic" directory creation would only happen for people who run 'configure' and 'make' directly in the source tree (which currently does an in-tree build) -- the behaviour if they're run from some other (manually created) build directory would remain as it is today. thanks -- PMM ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-31 10:19 ` Peter Maydell @ 2020-03-31 11:46 ` Liviu Ionescu 2020-03-31 12:07 ` Kevin Wolf 0 siblings, 1 reply; 53+ messages in thread From: Liviu Ionescu @ 2020-03-31 11:46 UTC (permalink / raw) To: Peter Maydell Cc: Kevin Wolf, Paolo Bonzini, "Daniel P. Berrangé", Markus Armbruster, QEMU Developers > On 31 Mar 2020, at 13:19, Peter Maydell <peter.maydell@linaro.org> wrote: > > > ... the behaviour > if they're run from some other (manually created) build directory > would remain as it is today. Great! Then for now a strongly worded warning displayed at the end of the build (to be clearly visible) should do it, and after a while a possible change to meson or any other solution that uses a separate build folder would not surprise anyone. The latest version of the xPack Build Box (v3.1) includes Python 3, meson, ninja, cmake, gcc 9, and generally all modern tools, so I guess building an updated QEMU would not be a problem. Regards, Liviu ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-31 11:46 ` Liviu Ionescu @ 2020-03-31 12:07 ` Kevin Wolf 2020-03-31 15:23 ` Liviu Ionescu 0 siblings, 1 reply; 53+ messages in thread From: Kevin Wolf @ 2020-03-31 12:07 UTC (permalink / raw) To: Liviu Ionescu Cc: QEMU Developers, Peter Maydell, "Daniel P. Berrangé", Markus Armbruster, Paolo Bonzini Am 31.03.2020 um 13:46 hat Liviu Ionescu geschrieben: > > On 31 Mar 2020, at 13:19, Peter Maydell <peter.maydell@linaro.org> wrote: > > > > > > ... the behaviour > > if they're run from some other (manually created) build directory > > would remain as it is today. > > Great! > > Then for now a strongly worded warning displayed at the end of the > build (to be clearly visible) should do it, and after a while a > possible change to meson or any other solution that uses a separate > build folder would not surprise anyone. Thanks for declaring my workflow irrelevant, I guess... It's interesting how you complain about a hypothetical situation where you would have to change workflow, but are completely happy to make the same thing happen to other people. Kevin ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-31 12:07 ` Kevin Wolf @ 2020-03-31 15:23 ` Liviu Ionescu 0 siblings, 0 replies; 53+ messages in thread From: Liviu Ionescu @ 2020-03-31 15:23 UTC (permalink / raw) To: Kevin Wolf Cc: QEMU Developers, Peter Maydell, "Daniel P. Berrangé", Markus Armbruster, Paolo Bonzini > On 31 Mar 2020, at 15:07, Kevin Wolf <kwolf@redhat.com> wrote: > > Thanks for declaring my workflow irrelevant, I guess... Sorry if this happened, it was not intended. The intention was to point out that there are also other use cases of running the QEMU builds, different from those specific to building Linux distribution packages. Given the large number of libraries and tools that the xPack Build Box workflow supports, it is no surprise that it accepts both out of tree and in tree builds. Just that for a large project like QEMU, since I usually run a multi-platform build at once, having multiple copies of QEMU in each platform build folder would be less space efficient than having a single copy in the folder with common sources. Apart from this, I actually don't mind, you can choose whatever solution you want, I guess I can accommodate it somehow. Regards, Liviu ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-31 7:48 ` Paolo Bonzini 2020-03-31 9:20 ` Liviu Ionescu @ 2020-03-31 12:02 ` Kevin Wolf 2020-03-31 12:05 ` Peter Maydell 2020-03-31 15:08 ` Eric Blake 2020-03-31 12:54 ` BALATON Zoltan 2 siblings, 2 replies; 53+ messages in thread From: Kevin Wolf @ 2020-03-31 12:02 UTC (permalink / raw) To: Paolo Bonzini Cc: Peter Maydell, Daniel P. Berrangé, Markus Armbruster, QEMU Developers Am 31.03.2020 um 09:48 hat Paolo Bonzini geschrieben: > On 30/03/20 16:37, Kevin Wolf wrote: > > If manually dealing with separate build directories is inconvenient > > today, it will still be inconvenient with Meson, so this would mean > > introducing the automatic directly creation together with the other > > changes to enable Meson. Which is fine by me, as long as it is really > > done when the external directory becomes mandatory, so that people won't > > have to switch back and forth between directories. > > Serious question: why is automatic directly creation more convenient for > developers? Even if "./configure" generates a "build" directory for > you, you would still have to invoke the QEMU binary as > "build/x86_64-softmmu/qemu-system-x86_64". That is less convenient than > doing "mkdir build" in the first place. Mainly because it allows me to start everything (most importantly: my editor, git and make) from the same directory. I guess the automatic directory creation is the less important part compared to a Makefile in the source tree that calls the Makefile in the build directory, because creation the directory is a one-time thing, but I call make all the time. Creating it automatically is nice especially for those who build QEMU for the first time and expect that the ./configure; make; make install sequence they are used to just works. Prefixing build/ when using the binaries is a change, too, but I guess tab completion means that it's not much worse than prefixing ./ > I can see why it's more convenient for packaging, as they just invoke > "make" and "make install", but as far as developers are concerned it > seems to add complexity for little or no gain. I often use the same terminal for building/testing and git, or sometimes even a second editor for source code. Kevin ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-31 12:02 ` Kevin Wolf @ 2020-03-31 12:05 ` Peter Maydell 2020-03-31 12:24 ` Kevin Wolf 2020-03-31 15:08 ` Eric Blake 1 sibling, 1 reply; 53+ messages in thread From: Peter Maydell @ 2020-03-31 12:05 UTC (permalink / raw) To: Kevin Wolf Cc: Paolo Bonzini, Daniel P. Berrangé, Markus Armbruster, QEMU Developers On Tue, 31 Mar 2020 at 13:02, Kevin Wolf <kwolf@redhat.com> wrote: > Mainly because it allows me to start everything (most importantly: my > editor, git and make) from the same directory. You can use 'make -C build/whatever' to avoid having to change directory. The odd one out here is configure, where you have to say (cd build/whatever && ../../configure ...) I have occasionally wondered if we should add an equivalent to make's -C option to configure, but AFAIX autoconf-configures don't have it, so it would be an odd non-standardism. thanks -- PMM ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-31 12:05 ` Peter Maydell @ 2020-03-31 12:24 ` Kevin Wolf 2020-03-31 12:32 ` Peter Maydell 0 siblings, 1 reply; 53+ messages in thread From: Kevin Wolf @ 2020-03-31 12:24 UTC (permalink / raw) To: Peter Maydell Cc: Paolo Bonzini, Daniel P. Berrangé, Markus Armbruster, QEMU Developers Am 31.03.2020 um 14:05 hat Peter Maydell geschrieben: > On Tue, 31 Mar 2020 at 13:02, Kevin Wolf <kwolf@redhat.com> wrote: > > Mainly because it allows me to start everything (most importantly: my > > editor, git and make) from the same directory. > > You can use 'make -C build/whatever' to avoid having to change directory. Which is about five times as long as 'make'. I'm not sure what the problem would be with a Makefile in the source tree that just calls make in the build directory? It's convenient and doesn't hurt anyone who prefers to work directly in the build directory. > The odd one out here is configure, where you have to say > (cd build/whatever && ../../configure ...) > I have occasionally wondered if we should add an equivalent to > make's -C option to configure, but AFAIX autoconf-configures don't > have it, so it would be an odd non-standardism. Well, not allowing in-tree builds is an odd non-standardism, too, so I think some more non-standardisms to make it more bearable wouldn't be too bad. Kevin ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-31 12:24 ` Kevin Wolf @ 2020-03-31 12:32 ` Peter Maydell 0 siblings, 0 replies; 53+ messages in thread From: Peter Maydell @ 2020-03-31 12:32 UTC (permalink / raw) To: Kevin Wolf Cc: Paolo Bonzini, Daniel P. Berrangé, Markus Armbruster, QEMU Developers On Tue, 31 Mar 2020 at 13:24, Kevin Wolf <kwolf@redhat.com> wrote: > > Am 31.03.2020 um 14:05 hat Peter Maydell geschrieben: > > On Tue, 31 Mar 2020 at 13:02, Kevin Wolf <kwolf@redhat.com> wrote: > > > Mainly because it allows me to start everything (most importantly: my > > > editor, git and make) from the same directory. > > > > You can use 'make -C build/whatever' to avoid having to change directory. > > Which is about five times as long as 'make'. True; but once you've done it once then the second time is usually C-r <some letters from the last command> RET to pull it out of the shell history. (Usually I run something like 'make -C build/foo -j8 && make -C build/foo -j8 check && make -C build/foo check-acceptance', which I'm definitely not typing out every time.) > I'm not sure what the problem would be with a Makefile in the source > tree that just calls make in the build directory? It's convenient and > doesn't hurt anyone who prefers to work directly in the build directory. I don't object to it particularly; I'm just mentioning that you don't need to cd all over the place if you do decide to switch to using build dirs. (I think the main benefit of build dirs is that you get to have more than one at once, at which point you need to specify which one you meant somehow.) thanks -- PMM ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-31 12:02 ` Kevin Wolf 2020-03-31 12:05 ` Peter Maydell @ 2020-03-31 15:08 ` Eric Blake 2020-03-31 15:20 ` BALATON Zoltan 1 sibling, 1 reply; 53+ messages in thread From: Eric Blake @ 2020-03-31 15:08 UTC (permalink / raw) To: Kevin Wolf, Paolo Bonzini Cc: Peter Maydell, Daniel P. Berrangé, Markus Armbruster, QEMU Developers On 3/31/20 7:02 AM, Kevin Wolf wrote: > Am 31.03.2020 um 09:48 hat Paolo Bonzini geschrieben: >> On 30/03/20 16:37, Kevin Wolf wrote: >>> If manually dealing with separate build directories is inconvenient >>> today, it will still be inconvenient with Meson, so this would mean >>> introducing the automatic directly creation together with the other >>> changes to enable Meson. Which is fine by me, as long as it is really >>> done when the external directory becomes mandatory, so that people won't >>> have to switch back and forth between directories. >> >> Serious question: why is automatic directly creation more convenient for >> developers? Even if "./configure" generates a "build" directory for >> you, you would still have to invoke the QEMU binary as >> "build/x86_64-softmmu/qemu-system-x86_64". That is less convenient than >> doing "mkdir build" in the first place. > > Mainly because it allows me to start everything (most importantly: my > editor, git and make) from the same directory. > > I guess the automatic directory creation is the less important part > compared to a Makefile in the source tree that calls the Makefile in the > build directory, because creation the directory is a one-time thing, but > I call make all the time. If we make in-tree ./configure create a GNUmakefile shim that auto-forwards to build/, it would also be possible to create symlinks to the various targets that will live in build. I recently switched my qemu playground to use VPATH builds, but with symlinks such as qemu-img -> build/qemu-img, I can still use my muscle memory of an in-tree build for normal development. > > Creating it automatically is nice especially for those who build QEMU > for the first time and expect that the ./configure; make; make install > sequence they are used to just works. > > Prefixing build/ when using the binaries is a change, too, but I guess > tab completion means that it's not much worse than prefixing ./ With symlinks, it is possible to give much more than 'make' the illusion of working in-tree. It then boils down to a question of how many symlinks are worth creating. > >> I can see why it's more convenient for packaging, as they just invoke >> "make" and "make install", but as far as developers are concerned it >> seems to add complexity for little or no gain. > > I often use the same terminal for building/testing and git, or sometimes > even a second editor for source code. > > Kevin > > -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-31 15:08 ` Eric Blake @ 2020-03-31 15:20 ` BALATON Zoltan 2020-03-31 15:44 ` Kevin Wolf 0 siblings, 1 reply; 53+ messages in thread From: BALATON Zoltan @ 2020-03-31 15:20 UTC (permalink / raw) To: Eric Blake Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrangé, QEMU Developers, Markus Armbruster, Paolo Bonzini On Tue, 31 Mar 2020, Eric Blake wrote: > On 3/31/20 7:02 AM, Kevin Wolf wrote: >> Am 31.03.2020 um 09:48 hat Paolo Bonzini geschrieben: >>> On 30/03/20 16:37, Kevin Wolf wrote: >>>> If manually dealing with separate build directories is inconvenient >>>> today, it will still be inconvenient with Meson, so this would mean >>>> introducing the automatic directly creation together with the other >>>> changes to enable Meson. Which is fine by me, as long as it is really >>>> done when the external directory becomes mandatory, so that people won't >>>> have to switch back and forth between directories. >>> >>> Serious question: why is automatic directly creation more convenient for >>> developers? Even if "./configure" generates a "build" directory for >>> you, you would still have to invoke the QEMU binary as >>> "build/x86_64-softmmu/qemu-system-x86_64". That is less convenient than >>> doing "mkdir build" in the first place. >> >> Mainly because it allows me to start everything (most importantly: my >> editor, git and make) from the same directory. >> >> I guess the automatic directory creation is the less important part >> compared to a Makefile in the source tree that calls the Makefile in the >> build directory, because creation the directory is a one-time thing, but >> I call make all the time. > > If we make in-tree ./configure create a GNUmakefile shim that auto-forwards > to build/, it would also be possible to create symlinks to the various > targets that will live in build. I recently switched my qemu playground to > use VPATH builds, but with symlinks such as qemu-img -> build/qemu-img, I can > still use my muscle memory of an in-tree build for normal development. > >> >> Creating it automatically is nice especially for those who build QEMU >> for the first time and expect that the ./configure; make; make install >> sequence they are used to just works. >> >> Prefixing build/ when using the binaries is a change, too, but I guess >> tab completion means that it's not much worse than prefixing ./ > > With symlinks, it is possible to give much more than 'make' the illusion of > working in-tree. It then boils down to a question of how many symlinks are > worth creating. Also creating symlinks, while nice, is probably added maintainance burden because it's something that only needs to be done in in-tree runs so it will likely get broken by those using only out of tree. I think getting results in a build dir is acceptable (most results are already in some subdir at least for system emulation) and just adding convenience Makefile is not much complexity while preserving symlinks for build results would be so that's probably not much maintenance improvement over current situarion. Regards, BALATON Zoltan ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-31 15:20 ` BALATON Zoltan @ 2020-03-31 15:44 ` Kevin Wolf 0 siblings, 0 replies; 53+ messages in thread From: Kevin Wolf @ 2020-03-31 15:44 UTC (permalink / raw) To: BALATON Zoltan Cc: Peter Maydell, Daniel P. Berrangé, QEMU Developers, Markus Armbruster, Paolo Bonzini Am 31.03.2020 um 17:20 hat BALATON Zoltan geschrieben: > On Tue, 31 Mar 2020, Eric Blake wrote: > > On 3/31/20 7:02 AM, Kevin Wolf wrote: > > > Am 31.03.2020 um 09:48 hat Paolo Bonzini geschrieben: > > > > On 30/03/20 16:37, Kevin Wolf wrote: > > > > > If manually dealing with separate build directories is inconvenient > > > > > today, it will still be inconvenient with Meson, so this would mean > > > > > introducing the automatic directly creation together with the other > > > > > changes to enable Meson. Which is fine by me, as long as it is really > > > > > done when the external directory becomes mandatory, so that people won't > > > > > have to switch back and forth between directories. > > > > > > > > Serious question: why is automatic directly creation more convenient for > > > > developers? Even if "./configure" generates a "build" directory for > > > > you, you would still have to invoke the QEMU binary as > > > > "build/x86_64-softmmu/qemu-system-x86_64". That is less convenient than > > > > doing "mkdir build" in the first place. > > > > > > Mainly because it allows me to start everything (most importantly: my > > > editor, git and make) from the same directory. > > > > > > I guess the automatic directory creation is the less important part > > > compared to a Makefile in the source tree that calls the Makefile in the > > > build directory, because creation the directory is a one-time thing, but > > > I call make all the time. > > > > If we make in-tree ./configure create a GNUmakefile shim that > > auto-forwards to build/, it would also be possible to create symlinks to > > the various targets that will live in build. I recently switched my > > qemu playground to use VPATH builds, but with symlinks such as qemu-img > > -> build/qemu-img, I can still use my muscle memory of an in-tree build > > for normal development. I like this idea. We could even make the emulator binaries available on the top level then. > > > Creating it automatically is nice especially for those who build QEMU > > > for the first time and expect that the ./configure; make; make install > > > sequence they are used to just works. > > > > > > Prefixing build/ when using the binaries is a change, too, but I guess > > > tab completion means that it's not much worse than prefixing ./ > > > > With symlinks, it is possible to give much more than 'make' the illusion > > of working in-tree. It then boils down to a question of how many > > symlinks are worth creating. > > Also creating symlinks, while nice, is probably added maintainance > burden because it's something that only needs to be done in in-tree > runs so it will likely get broken by those using only out of tree. To me this sounds like something that will be created once and then never touched again except maybe for adding new binaries. If some binary is missing, this is easily fixed by those who use pseudo-in-tree builds. Kevin ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-31 7:48 ` Paolo Bonzini 2020-03-31 9:20 ` Liviu Ionescu 2020-03-31 12:02 ` Kevin Wolf @ 2020-03-31 12:54 ` BALATON Zoltan 2 siblings, 0 replies; 53+ messages in thread From: BALATON Zoltan @ 2020-03-31 12:54 UTC (permalink / raw) To: Paolo Bonzini Cc: Kevin Wolf, Peter Maydell, Daniel P. Berrangé, Markus Armbruster, QEMU Developers On Tue, 31 Mar 2020, Paolo Bonzini wrote: > On 30/03/20 16:37, Kevin Wolf wrote: >> If manually dealing with separate build directories is inconvenient >> today, it will still be inconvenient with Meson, so this would mean >> introducing the automatic directly creation together with the other >> changes to enable Meson. Which is fine by me, as long as it is really >> done when the external directory becomes mandatory, so that people won't >> have to switch back and forth between directories. > > Serious question: why is automatic directly creation more convenient for > developers? Even if "./configure" generates a "build" directory for > you, you would still have to invoke the QEMU binary as > "build/x86_64-softmmu/qemu-system-x86_64". That is less convenient than > doing "mkdir build" in the first place. I already have to put qemu commands in scripts due to insanely long options which cannot be conveniently typed so I'd just need to add automatically created build dir to that script once and be able to continue using ./configure and make from source dir. This is more convenient than having to manage the build dir by hand and add it to every make command as well. If it can be automated then why should I do it? > I can see why it's more convenient for packaging, as they just invoke > "make" and "make install", but as far as developers are concerned it > seems to add complexity for little or no gain. It's not much complexity and makes life easier for those who do not want to know about separate build dirs, Meson or the QEMU build system. Just make sure it builds with usual configure; make; make install and maybe print a line of text saying you find the build results in whatever dir and that's it. As long as this works most people will be happy. It you want people to contribute to QEMU then try to make their life easier not annoy them unnecesseraily. The way of submitting patches is already difficult enough to scare away most people, changing to an exotic build system would also result even less people being able to make any contribution. Regards, BALATON Zoltan ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-30 13:31 ` Peter Maydell 2020-03-30 13:42 ` Daniel P. Berrangé @ 2020-03-30 16:25 ` Aleksandar Markovic 2020-03-31 7:15 ` Markus Armbruster 2 siblings, 0 replies; 53+ messages in thread From: Aleksandar Markovic @ 2020-03-30 16:25 UTC (permalink / raw) To: Peter Maydell; +Cc: Paolo Bonzini, Markus Armbruster, QEMU Developers [-- Attachment #1: Type: text/plain, Size: 1335 bytes --] 15:32 Pon, 30.03.2020. Peter Maydell <peter.maydell@linaro.org> је написао/ла: > > Consensus in the thread seemed to lean towards having > the 'configure/make' runes auto-create a build directory; > if we want to do that we should probably not say anything in > the release notes, because we'll cause people to change > unnecessarily. Or at least have them say "We recommend > out-of-tree builds. In future we might make the commands > that currently do an in-tree build automatically create > and use a build directory for you." rather than a blanket > "we're going to drop this and you should change what you > do now". > My vote is for the former, rather than for the latter. I fact, if I may suggest, I would use some longer and stronger wording, just for the simple reason that the warning is less likely to be ignored (as Zoltan, I think rightfully, noted to potentially be quite frequent case in reality): "We strongly recommend out-of-tree builds. In future we might make the commands that currently do an in-tree build automatically create and use a separate build directory for you. Still, our recommentation is that you change all your build procedures to use only out-of-tree builds as soon as it is possible for you." Regards, Aleksandar > Thoughts? > > thanks > -- PMM > [-- Attachment #2: Type: text/html, Size: 1655 bytes --] ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-30 13:31 ` Peter Maydell 2020-03-30 13:42 ` Daniel P. Berrangé 2020-03-30 16:25 ` Aleksandar Markovic @ 2020-03-31 7:15 ` Markus Armbruster 2020-03-31 12:33 ` BALATON Zoltan 2 siblings, 1 reply; 53+ messages in thread From: Markus Armbruster @ 2020-03-31 7:15 UTC (permalink / raw) To: Peter Maydell; +Cc: Paolo Bonzini, QEMU Developers Peter Maydell <peter.maydell@linaro.org> writes: > On Mon, 30 Mar 2020 at 14:26, Markus Armbruster <armbru@redhat.com> wrote: >> >> Peter Maydell <peter.maydell@linaro.org> writes: >> >> > AIUI from Paolo, the intention is to deprecate and eventually >> > stop supporting "in-tree" builds, so that the only option is >> > building in a separate build directory. I thought we should >> > probably mention that in the 5.0 changelog, so I wrote up some >> > text: >> > >> > https://wiki.qemu.org/ChangeLog/5.0#Build_Information >> >> Overdue. Thanks for doing this! >> >> > Suggestions for changes/comments etc welcome. >> >> Looks fine to me. > > Consensus in the thread seemed to lean towards having > the 'configure/make' runes auto-create a build directory; > if we want to do that we should probably not say anything in > the release notes, because we'll cause people to change > unnecessarily. Or at least have them say "We recommend > out-of-tree builds. In future we might make the commands > that currently do an in-tree build automatically create > and use a build directory for you." rather than a blanket > "we're going to drop this and you should change what you > do now". > > Thoughts? I'm wary of complicating the build system to save developers a minor change of habits. We will have to ask developers to change habits anyway when we switch to Meson. I agree with Daniel's recommendation to delay changes requiring habit-changes until then. However, telling people to stay clear of the unloved and brittle in-tree build is simply good advice we should not withhold. When you can't make the obvious way to do things the way that has the best chance of actually working, at least have the courtesy to point people to the way that has the best chance of actually working. If some developers want to avoid a change of habits badly enough, surely one or more of them will volunteer to contribute / maintain the complications needed for that with Meson. Whatever it takes to get rid of in-tree-only build breakages :) ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-31 7:15 ` Markus Armbruster @ 2020-03-31 12:33 ` BALATON Zoltan 2020-03-31 12:50 ` Daniel P. Berrangé 0 siblings, 1 reply; 53+ messages in thread From: BALATON Zoltan @ 2020-03-31 12:33 UTC (permalink / raw) To: Markus Armbruster; +Cc: Peter Maydell, QEMU Developers, Paolo Bonzini On Tue, 31 Mar 2020, Markus Armbruster wrote: > Peter Maydell <peter.maydell@linaro.org> writes: >> On Mon, 30 Mar 2020 at 14:26, Markus Armbruster <armbru@redhat.com> wrote: >>> >>> Peter Maydell <peter.maydell@linaro.org> writes: >>> >>>> AIUI from Paolo, the intention is to deprecate and eventually >>>> stop supporting "in-tree" builds, so that the only option is >>>> building in a separate build directory. I thought we should >>>> probably mention that in the 5.0 changelog, so I wrote up some >>>> text: >>>> >>>> https://wiki.qemu.org/ChangeLog/5.0#Build_Information >>> >>> Overdue. Thanks for doing this! >>> >>>> Suggestions for changes/comments etc welcome. >>> >>> Looks fine to me. >> >> Consensus in the thread seemed to lean towards having >> the 'configure/make' runes auto-create a build directory; >> if we want to do that we should probably not say anything in >> the release notes, because we'll cause people to change >> unnecessarily. Or at least have them say "We recommend >> out-of-tree builds. In future we might make the commands >> that currently do an in-tree build automatically create >> and use a build directory for you." rather than a blanket >> "we're going to drop this and you should change what you >> do now". >> >> Thoughts? > > I'm wary of complicating the build system to save developers a minor > change of habits. How much complication is it? Looks like a few lines in configure (like the one submitted to print warning but instead of printing a long warning just run the commands (which is even shorter then the text). Then a Makefile in top level dir to do make -C build like someone has also posted in another email. This does not seem too much burden to make life of some people easier so I don't see why some of you insist to force everyone to change their ways even if it could be solved with some effort. > We will have to ask developers to change habits anyway when we switch to > Meson. I agree with Daniel's recommendation to delay changes requiring > habit-changes until then. However, telling people to stay clear of the > unloved and brittle in-tree build is simply good advice we should not > withhold. Can someone please explain why is it brittle and cannot be supported? It has worked well so far apart from some breakage due to being untested which is also not a techincal necessity just a decision by some maintiners to not test it. Adding a CI job to keep it working would also not be difficult or much complexity. Regards, BALATON Zoltan ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-31 12:33 ` BALATON Zoltan @ 2020-03-31 12:50 ` Daniel P. Berrangé 2020-03-31 15:04 ` BALATON Zoltan 0 siblings, 1 reply; 53+ messages in thread From: Daniel P. Berrangé @ 2020-03-31 12:50 UTC (permalink / raw) To: BALATON Zoltan Cc: Peter Maydell, Paolo Bonzini, Markus Armbruster, QEMU Developers On Tue, Mar 31, 2020 at 02:33:46PM +0200, BALATON Zoltan wrote: > > We will have to ask developers to change habits anyway when we switch to > > Meson. I agree with Daniel's recommendation to delay changes requiring > > habit-changes until then. However, telling people to stay clear of the > > unloved and brittle in-tree build is simply good advice we should not > > withhold. > > Can someone please explain why is it brittle and cannot be supported? It has > worked well so far apart from some breakage due to being untested which is > also not a techincal necessity just a decision by some maintiners to not > test it. Adding a CI job to keep it working would also not be difficult or > much complexity. Writing make rules to correctly handle both src-dir and build-dir scenarios is a non-negligible maint burden. If you look back through QEMU's history we have a steady stream of patches which have broken one or the other build scenarios. Developers will often not test both scenarios, just the one they prefer to use. This results in a maint burden on the subsystem maintainers who merge patches and then find they break & have to back them out. Sometimes even the subsystem maintainer gets it wrong and burden falls on Peter to find & reject it. Even if we have CI to test both, it is still a burden on developers to debug failures reported by the CI and figure out what needs fixing. With the number of builds we do & the time for a single CI cycle it gets very time consuming. I've personally wasted many many hours debugging src-dir vs build-dir problems in QEMU's makefiles - probably more than an entire day was lost when I did the patches to split the trace.h header file. This all serves to divert time away from useful work on QEMU. If there was some critically important functional thing that src-dir builds offer that can't be achieved by build-dir builds, then the extra maint work could be justified. I don't think that's the case though. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-31 12:50 ` Daniel P. Berrangé @ 2020-03-31 15:04 ` BALATON Zoltan 2020-03-31 15:23 ` Daniel P. Berrangé 0 siblings, 1 reply; 53+ messages in thread From: BALATON Zoltan @ 2020-03-31 15:04 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Peter Maydell, Paolo Bonzini, Markus Armbruster, QEMU Developers [-- Attachment #1: Type: text/plain, Size: 3207 bytes --] On Tue, 31 Mar 2020, Daniel P. Berrangé wrote: > On Tue, Mar 31, 2020 at 02:33:46PM +0200, BALATON Zoltan wrote: >>> We will have to ask developers to change habits anyway when we switch to >>> Meson. I agree with Daniel's recommendation to delay changes requiring >>> habit-changes until then. However, telling people to stay clear of the >>> unloved and brittle in-tree build is simply good advice we should not >>> withhold. >> >> Can someone please explain why is it brittle and cannot be supported? It has >> worked well so far apart from some breakage due to being untested which is >> also not a techincal necessity just a decision by some maintiners to not >> test it. Adding a CI job to keep it working would also not be difficult or >> much complexity. > > Writing make rules to correctly handle both src-dir and build-dir scenarios > is a non-negligible maint burden. If you look back through QEMU's history > we have a steady stream of patches which have broken one or the other > build scenarios. That's probably becuase there are no clear rules (such as always prefix files in source dir with $(SRC_PATH), generated files with some BUILDDIR, etc.) and these are not documented so every time someone touches it has to explore and debug it again. This could be avoided if these were written down once but instead of trying to document and clean up the build system the chosen direction is to just throw it out and replace it with something more complex and with more dependencies (Meson) and then to save "additional complexity" it also breaks people's workflow and demand them to adapt themselves. I'm not saying there should be no changes but if there's a way to make them less painful it could be considered if it's not much extra work and in this case it does not seem to be. > Developers will often not test both scenarios, just the one they prefer > to use. This results in a maint burden on the subsystem maintainers who > merge patches and then find they break & have to back them out. Sometimes > even the subsystem maintainer gets it wrong and burden falls on Peter to > find & reject it. > > Even if we have CI to test both, it is still a burden on developers to > debug failures reported by the CI and figure out what needs fixing. With > the number of builds we do & the time for a single CI cycle it gets very > time consuming. I've personally wasted many many hours debugging src-dir > vs build-dir problems in QEMU's makefiles - probably more than an entire > day was lost when I did the patches to split the trace.h header file. > > This all serves to divert time away from useful work on QEMU. If there > was some critically important functional thing that src-dir builds offer > that can't be achieved by build-dir builds, then the extra maint work > could be justified. I don't think that's the case though. OK, so then only supporting out-of-tree builds but adding convenience function and Makefile to still allow people to run configure; make from source dir would solve this without also needing people to change what they always did so why is that solution not acceptable? Regards, BALATON Zoltan ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-31 15:04 ` BALATON Zoltan @ 2020-03-31 15:23 ` Daniel P. Berrangé 2020-03-31 15:55 ` BALATON Zoltan 0 siblings, 1 reply; 53+ messages in thread From: Daniel P. Berrangé @ 2020-03-31 15:23 UTC (permalink / raw) To: BALATON Zoltan Cc: Peter Maydell, Paolo Bonzini, Markus Armbruster, QEMU Developers On Tue, Mar 31, 2020 at 05:04:16PM +0200, BALATON Zoltan wrote: > On Tue, 31 Mar 2020, Daniel P. Berrangé wrote: > > On Tue, Mar 31, 2020 at 02:33:46PM +0200, BALATON Zoltan wrote: > > > > We will have to ask developers to change habits anyway when we switch to > > > > Meson. I agree with Daniel's recommendation to delay changes requiring > > > > habit-changes until then. However, telling people to stay clear of the > > > > unloved and brittle in-tree build is simply good advice we should not > > > > withhold. > > > > > > Can someone please explain why is it brittle and cannot be supported? It has > > > worked well so far apart from some breakage due to being untested which is > > > also not a techincal necessity just a decision by some maintiners to not > > > test it. Adding a CI job to keep it working would also not be difficult or > > > much complexity. > > > > Writing make rules to correctly handle both src-dir and build-dir scenarios > > is a non-negligible maint burden. If you look back through QEMU's history > > we have a steady stream of patches which have broken one or the other > > build scenarios. > > That's probably becuase there are no clear rules (such as always prefix > files in source dir with $(SRC_PATH), generated files with some BUILDDIR, > etc.) and these are not documented so every time someone touches it has to > explore and debug it again. This could be avoided if these were written down > once but instead of trying to document and clean up the build system the > chosen direction is to just throw it out and replace it with something more > complex and with more dependencies (Meson) and then to save "additional > complexity" it also breaks people's workflow and demand them to adapt > themselves. I'm not saying there should be no changes but if there's a way > to make them less painful it could be considered if it's not much extra work > and in this case it does not seem to be. I did indeed write the docs on QEMU's build system found in docs/devel/build-system.txt but it is not enough to eliminate the maint burden of supporting both. QEMU's makefiles are just too complex and full of unexpected effects to make it easy to support both srcdir & builddir IMHO. > > Developers will often not test both scenarios, just the one they prefer > > to use. This results in a maint burden on the subsystem maintainers who > > merge patches and then find they break & have to back them out. Sometimes > > even the subsystem maintainer gets it wrong and burden falls on Peter to > > find & reject it. > > > > Even if we have CI to test both, it is still a burden on developers to > > debug failures reported by the CI and figure out what needs fixing. With > > the number of builds we do & the time for a single CI cycle it gets very > > time consuming. I've personally wasted many many hours debugging src-dir > > vs build-dir problems in QEMU's makefiles - probably more than an entire > > day was lost when I did the patches to split the trace.h header file. > > > > This all serves to divert time away from useful work on QEMU. If there > > was some critically important functional thing that src-dir builds offer > > that can't be achieved by build-dir builds, then the extra maint work > > could be justified. I don't think that's the case though. > > OK, so then only supporting out-of-tree builds but adding convenience > function and Makefile to still allow people to run configure; make from > source dir would solve this without also needing people to change what they > always did so why is that solution not acceptable? There's several scenarios discussed. Currently we don't do anything to prevent src-dir builds, so there's no need to extra make magic described I proposed a patch to add a warning, but it still allows src-dir builds, so nothing functional has changed. We could forbid src-dir builds now and add the magic to redirct to a build-dir. That would work, but I don't think we should block src-dir builds in make, assuming we have a switch to meson arriving in the near-ish future. Once we switch to Meson there will inevitably be some disruption in build system ussage as it is a completely new tool. That makes it a good flag day for forbidding src-dir builds. Given that it is a flag day, I don't think there's any need to create makefile magic at that time. IMHO we shouldn't be so afraid of change that we need to try to preserve every precise way of working we've had in the past. In the long term this frame of mind holds back the project and promotes a negative impression of QEMU as an old, outdated project resistant to any kind of change. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-31 15:23 ` Daniel P. Berrangé @ 2020-03-31 15:55 ` BALATON Zoltan 0 siblings, 0 replies; 53+ messages in thread From: BALATON Zoltan @ 2020-03-31 15:55 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Peter Maydell, Paolo Bonzini, Markus Armbruster, QEMU Developers [-- Attachment #1: Type: text/plain, Size: 3146 bytes --] On Tue, 31 Mar 2020, Daniel P. Berrangé wrote: > On Tue, Mar 31, 2020 at 05:04:16PM +0200, BALATON Zoltan wrote: >> OK, so then only supporting out-of-tree builds but adding convenience >> function and Makefile to still allow people to run configure; make from >> source dir would solve this without also needing people to change what they >> always did so why is that solution not acceptable? > > There's several scenarios discussed. Currently we don't do anything to > prevent src-dir builds, so there's no need to extra make magic described > I proposed a patch to add a warning, but it still allows src-dir builds, > so nothing functional has changed. > > We could forbid src-dir builds now and add the magic to redirct to a > build-dir. That would work, but I don't think we should block src-dir > builds in make, assuming we have a switch to meson arriving in the > near-ish future. So then why print a warning when it will change again when the build system is replaced? Isn't it enough to worry about it at that point? If it works now as before then no need to constantly bug people at every cofigure invocation. Simply put a warning in changelog that current build system is deprecated and will be replaced with something else like Meson in some future release and that should be enough to notify people about the coming change. You don't print warnings for every other deprecated features either. > Once we switch to Meson there will inevitably be some disruption in > build system ussage as it is a completely new tool. That makes it a > good flag day for forbidding src-dir builds. Given that it is a flag > day, I don't think there's any need to create makefile magic at that > time. Ideally a change in build system should make it more convenient to work with not more inconvenient otherwise it's questionable if we want to introduce it at all but we'll see then. But then why bug people with a 14 line warning now if they'll have to change their ways again shortly? I thought you wanted to get rid of in-tree builds now to ease the maintenance of current make system and the convenience makefile would help that. > IMHO we shouldn't be so afraid of change that we need to try to preserve > every precise way of working we've had in the past. In the long term this > frame of mind holds back the project and promotes a negative impression > of QEMU as an old, outdated project resistant to any kind of change. I did not say that. What I said was that you can change or replace the build system with whatever you want as long as I can still run make (or equivalently simple command) in source dir and it will build the project. I can live with build results moved to a subdir to ease your task of managing build system so you don't have to support two ways of building now. But if you say this will still work for now and don't mind continue to support it as before at least until a new build system is introduced I'm happy with that too. I'll raise same complaints against new build system then when it's about to be merged. Regards, BALATON Zoltan ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: deprecation of in-tree builds 2020-03-21 20:50 deprecation of in-tree builds Peter Maydell ` (2 preceding siblings ...) 2020-03-30 13:26 ` Markus Armbruster @ 2020-03-31 10:38 ` Daniel P. Berrangé 3 siblings, 0 replies; 53+ messages in thread From: Daniel P. Berrangé @ 2020-03-31 10:38 UTC (permalink / raw) To: Peter Maydell; +Cc: Paolo Bonzini, QEMU Developers On Sat, Mar 21, 2020 at 08:50:34PM +0000, Peter Maydell wrote: > AIUI from Paolo, the intention is to deprecate and eventually > stop supporting "in-tree" builds, so that the only option is > building in a separate build directory. I thought we should > probably mention that in the 5.0 changelog, so I wrote up some > text: > > https://wiki.qemu.org/ChangeLog/5.0#Build_Information > > Suggestions for changes/comments etc welcome. > > (One thing we will need to fix before we can do separate build > tree is the Coverity Scan build process, which (a) does an > in-tree build (b) can't be easily switched to a builddir because > all the source paths get baked into the scan results and moving > to a builddir changes them all...) > > We could also make configure actively warn if used in > the source tree. I've submitted a patch to add such a warning, copying everyone who has responded in this thread. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 53+ messages in thread
end of thread, other threads:[~2020-08-21 9:56 UTC | newest] Thread overview: 53+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-03-21 20:50 deprecation of in-tree builds Peter Maydell 2020-03-21 22:50 ` BALATON Zoltan 2020-03-23 13:32 ` Stefan Hajnoczi 2020-03-24 13:54 ` Eric Blake 2020-08-18 19:14 ` Peter Maydell 2020-08-20 10:54 ` Kevin Wolf 2020-08-20 11:14 ` Michael Tokarev 2020-08-20 11:56 ` Paolo Bonzini 2020-08-20 12:00 ` Peter Maydell 2020-08-20 13:30 ` Kevin Wolf 2020-08-20 14:10 ` Peter Maydell 2020-08-20 15:50 ` Paolo Bonzini 2020-08-20 16:14 ` Kevin Wolf 2020-08-20 17:39 ` Paolo Bonzini 2020-08-21 9:55 ` Gerd Hoffmann 2020-08-20 16:05 ` Daniel P. Berrangé [not found] ` <CAL1e-=gKB0qRxGntXrU0im2gjMh1tq_SLGTm+HsmgBRGXQ9+Bg@mail.gmail.com> 2020-03-22 17:20 ` Peter Maydell 2020-03-22 19:51 ` Aleksandar Markovic 2020-03-22 20:14 ` Peter Maydell 2020-03-22 20:45 ` Aleksandar Markovic 2020-03-22 20:46 ` BALATON Zoltan 2020-03-22 21:15 ` Peter Maydell 2020-03-23 0:35 ` BALATON Zoltan 2020-03-23 10:20 ` Daniel P. Berrangé 2020-03-30 13:26 ` Markus Armbruster 2020-03-30 13:31 ` Peter Maydell 2020-03-30 13:42 ` Daniel P. Berrangé 2020-03-30 14:37 ` Kevin Wolf 2020-03-30 17:29 ` Michal Suchánek 2020-03-30 17:36 ` Daniel P. Berrangé 2020-03-30 20:15 ` BALATON Zoltan 2020-03-31 7:48 ` Paolo Bonzini 2020-03-31 9:20 ` Liviu Ionescu 2020-03-31 10:19 ` Peter Maydell 2020-03-31 11:46 ` Liviu Ionescu 2020-03-31 12:07 ` Kevin Wolf 2020-03-31 15:23 ` Liviu Ionescu 2020-03-31 12:02 ` Kevin Wolf 2020-03-31 12:05 ` Peter Maydell 2020-03-31 12:24 ` Kevin Wolf 2020-03-31 12:32 ` Peter Maydell 2020-03-31 15:08 ` Eric Blake 2020-03-31 15:20 ` BALATON Zoltan 2020-03-31 15:44 ` Kevin Wolf 2020-03-31 12:54 ` BALATON Zoltan 2020-03-30 16:25 ` Aleksandar Markovic 2020-03-31 7:15 ` Markus Armbruster 2020-03-31 12:33 ` BALATON Zoltan 2020-03-31 12:50 ` Daniel P. Berrangé 2020-03-31 15:04 ` BALATON Zoltan 2020-03-31 15:23 ` Daniel P. Berrangé 2020-03-31 15:55 ` BALATON Zoltan 2020-03-31 10:38 ` Daniel P. Berrangé
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.