* [Qemu-devel] Yet another git submodule rant @ 2017-11-08 9:57 Thomas Huth 2017-11-08 10:25 ` Daniel P. Berrange ` (3 more replies) 0 siblings, 4 replies; 21+ messages in thread From: Thomas Huth @ 2017-11-08 9:57 UTC (permalink / raw) To: QEMU Developers, Daniel P. Berrange; +Cc: Gerd Hoffmann, Peter Maydell That automatic git submodule stuff now broke my workflow again. I usually keep the git repository on my laptop and then simply rsync the sources (without .git directories) to my target machine to compile it there. Used to work great for years. Now it's broken, the build process complains: GIT submodule checkout is out of date. Please run scripts/git-submodule.sh update from the source directory checkout /home/thuth/devel/qemu Running "scripts/git-submodule.sh update" did not fix the issue at all - I first had to tinker with it for a while to find out that I simply have to delete ".git-submodule-status" in my git tree to fix the issue. I've got the feeling that all this submodule crap is constantly causing pain ... do we really need this? Can't we find another solution instead? Or at least stop modifying files automatically in the $SRC_PATH ? ... ok, sorry for the harsh words, ... but now that I've written this down, I feel at least a little bit better ... Thomas ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-08 9:57 [Qemu-devel] Yet another git submodule rant Thomas Huth @ 2017-11-08 10:25 ` Daniel P. Berrange 2017-11-08 12:05 ` Michal Suchánek 2017-11-08 10:53 ` Gerd Hoffmann ` (2 subsequent siblings) 3 siblings, 1 reply; 21+ messages in thread From: Daniel P. Berrange @ 2017-11-08 10:25 UTC (permalink / raw) To: Thomas Huth; +Cc: QEMU Developers, Gerd Hoffmann, Peter Maydell On Wed, Nov 08, 2017 at 10:57:21AM +0100, Thomas Huth wrote: > > That automatic git submodule stuff now broke my workflow again. I > usually keep the git repository on my laptop and then simply rsync the > sources (without .git directories) to my target machine to compile it > there. Used to work great for years. Now it's broken, the build process > complains: Excluding the .git dir will be a problem - that needs to exist, unless you are building from tar.xz (where we will bundled all the submodule sources together in the tar.xz). If you really want to exclude the .git directories, then instead of doing an rsync, then use the scripts/archive-source.sh tool to create an archive of all the sources (which bundles required submodules) and unpack that on your target machine. Then all submodule handling on the target will be skipped. > > GIT submodule checkout is out of date. Please run > scripts/git-submodule.sh update > from the source directory checkout /home/thuth/devel/qemu > > Running "scripts/git-submodule.sh update" did not fix the issue at all - > I first had to tinker with it for a while to find out that I simply have > to delete ".git-submodule-status" in my git tree to fix the issue. I think it got confused from you deleting the .git directory > I've got the feeling that all this submodule crap is constantly causing > pain ... do we really need this? Can't we find another solution instead? > Or at least stop modifying files automatically in the $SRC_PATH ? We *have* to modify SRC_PATH, as that's where git submodule checkouts live. If we didn't do that, then people would end up having failed build due to submodules not existing, or even worse, having a silently incorrect build due to using the wrong checked out version. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-08 10:25 ` Daniel P. Berrange @ 2017-11-08 12:05 ` Michal Suchánek 2017-11-08 12:34 ` Daniel P. Berrange 0 siblings, 1 reply; 21+ messages in thread From: Michal Suchánek @ 2017-11-08 12:05 UTC (permalink / raw) To: Daniel P. Berrange Cc: Thomas Huth, Peter Maydell, QEMU Developers, Gerd Hoffmann On Wed, 8 Nov 2017 10:25:29 +0000 "Daniel P. Berrange" <berrange@redhat.com> wrote: > On Wed, Nov 08, 2017 at 10:57:21AM +0100, Thomas Huth wrote: > > > > That automatic git submodule stuff now broke my workflow again. I > > usually keep the git repository on my laptop and then simply rsync > > the sources (without .git directories) to my target machine to > > compile it there. Used to work great for years. Now it's broken, > > the build process complains: > > Excluding the .git dir will be a problem - that needs to exist, unless > you are building from tar.xz (where we will bundled all the submodule > sources together in the tar.xz). > > If you really want to exclude the .git directories, then instead of > doing an rsync, then use the scripts/archive-source.sh tool to create > an archive of all the sources (which bundles required submodules) > and unpack that on your target machine. Then all submodule handling > on the target will be skipped. This probably does not work because it archives committed sources and not actual sources. > > > > > GIT submodule checkout is out of date. Please run > > scripts/git-submodule.sh update > > from the source directory checkout /home/thuth/devel/qemu > > > > Running "scripts/git-submodule.sh update" did not fix the issue at > > all - I first had to tinker with it for a while to find out that I > > simply have to delete ".git-submodule-status" in my git tree to fix > > the issue. > > I think it got confused from you deleting the .git directory > > > I've got the feeling that all this submodule crap is constantly > > causing pain ... do we really need this? Can't we find another > > solution instead? Or at least stop modifying files automatically in > > the $SRC_PATH ? > > We *have* to modify SRC_PATH, as that's where git submodule checkouts > live. If we didn't do that, then people would end up having failed > build due to submodules not existing, or even worse, having a silently > incorrect build due to using the wrong checked out version. And that's another reason why submodules are bad thing. So again, can this be done without submodules? Let somebody else do the testing and polishing of this pre-alpha git feature. Thanks Michal ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-08 12:05 ` Michal Suchánek @ 2017-11-08 12:34 ` Daniel P. Berrange 2017-11-08 12:43 ` Michal Suchánek 0 siblings, 1 reply; 21+ messages in thread From: Daniel P. Berrange @ 2017-11-08 12:34 UTC (permalink / raw) To: Michal Suchánek Cc: Thomas Huth, Peter Maydell, QEMU Developers, Gerd Hoffmann On Wed, Nov 08, 2017 at 01:05:27PM +0100, Michal Suchánek wrote: > On Wed, 8 Nov 2017 10:25:29 +0000 > "Daniel P. Berrange" <berrange@redhat.com> wrote: > > > On Wed, Nov 08, 2017 at 10:57:21AM +0100, Thomas Huth wrote: > > > > > > That automatic git submodule stuff now broke my workflow again. I > > > usually keep the git repository on my laptop and then simply rsync > > > the sources (without .git directories) to my target machine to > > > compile it there. Used to work great for years. Now it's broken, > > > the build process complains: > > > > Excluding the .git dir will be a problem - that needs to exist, unless > > you are building from tar.xz (where we will bundled all the submodule > > sources together in the tar.xz). > > > > If you really want to exclude the .git directories, then instead of > > doing an rsync, then use the scripts/archive-source.sh tool to create > > an archive of all the sources (which bundles required submodules) > > and unpack that on your target machine. Then all submodule handling > > on the target will be skipped. > > This probably does not work because it archives committed sources and > not actual sources. Actually that's not quite correct - if it sees you have non-comitted changes, and will use 'git stash create' to capture them in the archive it creates. So it would only miss brand new files which have not yet been 'git add'ed > > > I've got the feeling that all this submodule crap is constantly > > > causing pain ... do we really need this? Can't we find another > > > solution instead? Or at least stop modifying files automatically in > > > the $SRC_PATH ? > > > > We *have* to modify SRC_PATH, as that's where git submodule checkouts > > live. If we didn't do that, then people would end up having failed > > build due to submodules not existing, or even worse, having a silently > > incorrect build due to using the wrong checked out version. > > And that's another reason why submodules are bad thing. > > So again, can this be done without submodules? > > Let somebody else do the testing and polishing of this pre-alpha git > feature. Honestly I think the submodule stuff is working pretty well - a handful of people have mentioned problems and we've addressed most of them or illustrated alternative approaches to deal with it. For most developers it has been pretty much transparent and just works, or we would have seen alot more noise. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-08 12:34 ` Daniel P. Berrange @ 2017-11-08 12:43 ` Michal Suchánek 2017-11-08 12:55 ` Daniel P. Berrange 0 siblings, 1 reply; 21+ messages in thread From: Michal Suchánek @ 2017-11-08 12:43 UTC (permalink / raw) To: Daniel P. Berrange Cc: Peter Maydell, Thomas Huth, QEMU Developers, Gerd Hoffmann On Wed, 8 Nov 2017 12:34:17 +0000 "Daniel P. Berrange" <berrange@redhat.com> wrote: > On Wed, Nov 08, 2017 at 01:05:27PM +0100, Michal Suchánek wrote: > > On Wed, 8 Nov 2017 10:25:29 +0000 > > "Daniel P. Berrange" <berrange@redhat.com> wrote: > > > > > On Wed, Nov 08, 2017 at 10:57:21AM +0100, Thomas Huth wrote: > > > > > > > > That automatic git submodule stuff now broke my workflow again. > > > > I usually keep the git repository on my laptop and then simply > > > > rsync the sources (without .git directories) to my target > > > > machine to compile it there. Used to work great for years. Now > > > > it's broken, the build process complains: > > > > > > Excluding the .git dir will be a problem - that needs to exist, > > > unless you are building from tar.xz (where we will bundled all > > > the submodule sources together in the tar.xz). > > > > > > If you really want to exclude the .git directories, then instead > > > of doing an rsync, then use the scripts/archive-source.sh tool to > > > create an archive of all the sources (which bundles required > > > submodules) and unpack that on your target machine. Then all > > > submodule handling on the target will be skipped. > > > > This probably does not work because it archives committed sources > > and not actual sources. > > Actually that's not quite correct - if it sees you have non-comitted > changes, and will use 'git stash create' to capture them in the > archive it creates. So it would only miss brand new files which have > not yet been 'git add'ed So there really isn't a good solution. > > > > > I've got the feeling that all this submodule crap is constantly > > > > causing pain ... do we really need this? Can't we find another > > > > solution instead? Or at least stop modifying files > > > > automatically in the $SRC_PATH ? > > > > > > We *have* to modify SRC_PATH, as that's where git submodule > > > checkouts live. If we didn't do that, then people would end up > > > having failed build due to submodules not existing, or even > > > worse, having a silently incorrect build due to using the wrong > > > checked out version. > > > > And that's another reason why submodules are bad thing. > > > > So again, can this be done without submodules? > > > > Let somebody else do the testing and polishing of this pre-alpha git > > feature. > > Honestly I think the submodule stuff is working pretty well - a > handful of people have mentioned problems and we've addressed most of > them or illustrated alternative approaches to deal with it. For most > developers it has been pretty much transparent and just works, or we > would have seen alot more noise. > On 11/08/2017 06:57 AM, Thomas Huth wrote: > > Also yesterday on IRC: > > <RaV3N> [...] I downloaded the qemu source from git and tried to > compile it. I am getting this: > > ./configure --static && make && sudo make install > CC ui/input-keymap.o > ui/input-keymap.c:8:10: fatal error: ui/input-keymap-linux-to-qcode.c: > No such file or directory > > <cota> [...] you might want to do "git submodule init && git submodule > update" > .. you call that transparent. Yeah, right. Please don't pretend that something is working when it clearly isn't. Repositories using submodules are far from first class citizens as far as support from git is concerned. Therefore projects that don't have git development at least as secondary focus should not use them until that changes. Thanks Michal ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-08 12:43 ` Michal Suchánek @ 2017-11-08 12:55 ` Daniel P. Berrange 0 siblings, 0 replies; 21+ messages in thread From: Daniel P. Berrange @ 2017-11-08 12:55 UTC (permalink / raw) To: Michal Suchánek Cc: Peter Maydell, Thomas Huth, QEMU Developers, Gerd Hoffmann On Wed, Nov 08, 2017 at 01:43:15PM +0100, Michal Suchánek wrote: > On Wed, 8 Nov 2017 12:34:17 +0000 > "Daniel P. Berrange" <berrange@redhat.com> wrote: > > > On Wed, Nov 08, 2017 at 01:05:27PM +0100, Michal Suchánek wrote: > > > On Wed, 8 Nov 2017 10:25:29 +0000 > > > "Daniel P. Berrange" <berrange@redhat.com> wrote: > > > > > > > On Wed, Nov 08, 2017 at 10:57:21AM +0100, Thomas Huth wrote: > > > > > > > > > > That automatic git submodule stuff now broke my workflow again. > > > > > I usually keep the git repository on my laptop and then simply > > > > > rsync the sources (without .git directories) to my target > > > > > machine to compile it there. Used to work great for years. Now > > > > > it's broken, the build process complains: > > > > > > > > Excluding the .git dir will be a problem - that needs to exist, > > > > unless you are building from tar.xz (where we will bundled all > > > > the submodule sources together in the tar.xz). > > > > > > > > If you really want to exclude the .git directories, then instead > > > > of doing an rsync, then use the scripts/archive-source.sh tool to > > > > create an archive of all the sources (which bundles required > > > > submodules) and unpack that on your target machine. Then all > > > > submodule handling on the target will be skipped. > > > > > > This probably does not work because it archives committed sources > > > and not actual sources. > > > > Actually that's not quite correct - if it sees you have non-comitted > > changes, and will use 'git stash create' to capture them in the > > archive it creates. So it would only miss brand new files which have > > not yet been 'git add'ed > > So there really isn't a good solution. > > > > > > > > I've got the feeling that all this submodule crap is constantly > > > > > causing pain ... do we really need this? Can't we find another > > > > > solution instead? Or at least stop modifying files > > > > > automatically in the $SRC_PATH ? > > > > > > > > We *have* to modify SRC_PATH, as that's where git submodule > > > > checkouts live. If we didn't do that, then people would end up > > > > having failed build due to submodules not existing, or even > > > > worse, having a silently incorrect build due to using the wrong > > > > checked out version. > > > > > > And that's another reason why submodules are bad thing. > > > > > > So again, can this be done without submodules? > > > > > > Let somebody else do the testing and polishing of this pre-alpha git > > > feature. > > > > Honestly I think the submodule stuff is working pretty well - a > > handful of people have mentioned problems and we've addressed most of > > them or illustrated alternative approaches to deal with it. For most > > developers it has been pretty much transparent and just works, or we > > would have seen alot more noise. > > > On 11/08/2017 06:57 AM, Thomas Huth wrote: > > > > > Also yesterday on IRC: > > > > <RaV3N> [...] I downloaded the qemu source from git and tried to > > compile it. I am getting this: > > > > ./configure --static && make && sudo make install > > CC ui/input-keymap.o > > ui/input-keymap.c:8:10: fatal error: ui/input-keymap-linux-to-qcode.c: > > No such file or directory > > > > <cota> [...] you might want to do "git submodule init && git submodule > > update" > > > > .. you call that transparent. > > Yeah, right. > > Please don't pretend that something is working when it clearly isn't. > > Repositories using submodules are far from first class citizens as far > as support from git is concerned. > > Therefore projects that don't have git development at least as > secondary focus should not use them until that changes. That's really overstating the problem. There are some caveats with use of submodules, but they are hardly showstoppers that prevent their usage entirely. There are plenty of projects using GIT with submodules for *years* (libvirt has used them for 8 years). Some of the problems pointed out have been clear bugs in the way we've integrated into QEMU's build system, which I've been addressing. Others have been due to usage scenarios I didn't anticipate, which I've attempted to provide solutions for, either by improving the build system, or suggesting alternative approaches. We'll continue to try to improve this, as we would for any other major change to the buildsystem we've been through, which caused problems for some workflows. I none the less believe this is working for the vast majority of developers of QEMU, given the size of our community vs the number of problems reported. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-08 9:57 [Qemu-devel] Yet another git submodule rant Thomas Huth 2017-11-08 10:25 ` Daniel P. Berrange @ 2017-11-08 10:53 ` Gerd Hoffmann 2017-11-08 11:48 ` Thomas Huth 2017-11-08 16:09 ` Laszlo Ersek 2017-11-08 12:26 ` Philippe Mathieu-Daudé 2017-11-09 3:02 ` Alexey Kardashevskiy 3 siblings, 2 replies; 21+ messages in thread From: Gerd Hoffmann @ 2017-11-08 10:53 UTC (permalink / raw) To: Thomas Huth; +Cc: QEMU Developers, Daniel P. Berrange, Peter Maydell On Wed, Nov 08, 2017 at 10:57:21AM +0100, Thomas Huth wrote: > > That automatic git submodule stuff now broke my workflow again. I > usually keep the git repository on my laptop and then simply rsync the > sources (without .git directories) to my target machine to compile it > there. How about excluding all files listed in .gitignore from rsync? Which I guess would be a good idea anyway because you don't want sync over the build artifacts to the target machine if you want compile there. Might be as simple as "rsync --exclude-from .gitignore" (didn't test though). Or syncronize using git instead of rsync. This is what I am doing. cheers, Gerd ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-08 10:53 ` Gerd Hoffmann @ 2017-11-08 11:48 ` Thomas Huth 2017-11-08 13:30 ` Philippe Mathieu-Daudé 2017-11-08 16:09 ` Laszlo Ersek 1 sibling, 1 reply; 21+ messages in thread From: Thomas Huth @ 2017-11-08 11:48 UTC (permalink / raw) To: Gerd Hoffmann; +Cc: QEMU Developers, Daniel P. Berrange, Peter Maydell On 08.11.2017 11:53, Gerd Hoffmann wrote: > On Wed, Nov 08, 2017 at 10:57:21AM +0100, Thomas Huth wrote: >> >> That automatic git submodule stuff now broke my workflow again. I >> usually keep the git repository on my laptop and then simply rsync the >> sources (without .git directories) to my target machine to compile it >> there. > > How about excluding all files listed in .gitignore from rsync? Which I > guess would be a good idea anyway because you don't want sync over the > build artifacts to the target machine if you want compile there. Might > be as simple as "rsync --exclude-from .gitignore" (didn't test though). That's a neat idea, thanks, I'll give it a try! > Or syncronize using git instead of rsync. This is what I am doing. Tried that in the past already, but it didn't really feel right (I don't want to check in each time before syncing my files)... Thomas ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-08 11:48 ` Thomas Huth @ 2017-11-08 13:30 ` Philippe Mathieu-Daudé 0 siblings, 0 replies; 21+ messages in thread From: Philippe Mathieu-Daudé @ 2017-11-08 13:30 UTC (permalink / raw) To: Thomas Huth, Gerd Hoffmann; +Cc: Peter Maydell, QEMU Developers On 11/08/2017 08:48 AM, Thomas Huth wrote: > On 08.11.2017 11:53, Gerd Hoffmann wrote: >> On Wed, Nov 08, 2017 at 10:57:21AM +0100, Thomas Huth wrote: >>> >>> That automatic git submodule stuff now broke my workflow again. I >>> usually keep the git repository on my laptop and then simply rsync the >>> sources (without .git directories) to my target machine to compile it >>> there. >> >> How about excluding all files listed in .gitignore from rsync? Which I >> guess would be a good idea anyway because you don't want sync over the >> build artifacts to the target machine if you want compile there. Might >> be as simple as "rsync --exclude-from .gitignore" (didn't test though). > > That's a neat idea, thanks, I'll give it a try! > >> Or syncronize using git instead of rsync. This is what I am doing. > > Tried that in the past already, but it didn't really feel right (I don't > want to check in each time before syncing my files)... TIL: git push --recurse-submodules=on-demand GIT-PUSH(1) --recurse-submodules=check|on-demand|only|no [...] If check is used Git will verify that all submodule commits that changed in the revisions to be pushed are available on at least one remote of the submodule. If any commits are missing the push will be aborted and exit with non-zero status. If on-demand is used all submodules that changed in the revisions to be pushed will be pushed. If on-demand was not able to push all necessary revisions it will also be aborted and exit with non-zero status. [...] GIT-CONFIG(1) push.recurseSubmodules Make sure all submodule commits used by the revisions to be pushed are available on a remote-tracking branch. If the value is check then Git will verify that all submodule commits that changed in the revisions to be pushed are available on at least one remote of the submodule. If any commits are missing, the push will be aborted and exit with non-zero status. If the value is on-demand then all submodules that changed in the revisions to be pushed will be pushed. If on-demand was not able to push all necessary revisions it will also be aborted and exit with non-zero status. [...] ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-08 10:53 ` Gerd Hoffmann 2017-11-08 11:48 ` Thomas Huth @ 2017-11-08 16:09 ` Laszlo Ersek 1 sibling, 0 replies; 21+ messages in thread From: Laszlo Ersek @ 2017-11-08 16:09 UTC (permalink / raw) To: Gerd Hoffmann, Thomas Huth; +Cc: Peter Maydell, QEMU Developers On 11/08/17 11:53, Gerd Hoffmann wrote: > On Wed, Nov 08, 2017 at 10:57:21AM +0100, Thomas Huth wrote: >> >> That automatic git submodule stuff now broke my workflow again. I >> usually keep the git repository on my laptop and then simply rsync the >> sources (without .git directories) to my target machine to compile it >> there. > > How about excluding all files listed in .gitignore from rsync? Which I > guess would be a good idea anyway because you don't want sync over the > build artifacts to the target machine if you want compile there. Might > be as simple as "rsync --exclude-from .gitignore" (didn't test though). > > Or syncronize using git instead of rsync. This is what I am doing. Me too, $ git push --mirror host:dir (I don't mind having to commit first.) Thanks, Laszlo ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-08 9:57 [Qemu-devel] Yet another git submodule rant Thomas Huth 2017-11-08 10:25 ` Daniel P. Berrange 2017-11-08 10:53 ` Gerd Hoffmann @ 2017-11-08 12:26 ` Philippe Mathieu-Daudé 2017-11-08 13:01 ` Daniel P. Berrange 2017-11-09 3:02 ` Alexey Kardashevskiy 3 siblings, 1 reply; 21+ messages in thread From: Philippe Mathieu-Daudé @ 2017-11-08 12:26 UTC (permalink / raw) To: Thomas Huth, QEMU Developers, Daniel P. Berrange Cc: Peter Maydell, Gerd Hoffmann, Emilio G. Cota, Michal Suchánek On 11/08/2017 06:57 AM, Thomas Huth wrote: > > That automatic git submodule stuff now broke my workflow again. I > usually keep the git repository on my laptop and then simply rsync the > sources (without .git directories) to my target machine to compile it > there. Used to work great for years. Now it's broken, the build process > complains: > > GIT submodule checkout is out of date. Please run > scripts/git-submodule.sh update > from the source directory checkout /home/thuth/devel/qemu > > Running "scripts/git-submodule.sh update" did not fix the issue at all - > I first had to tinker with it for a while to find out that I simply have > to delete ".git-submodule-status" in my git tree to fix the issue. > > I've got the feeling that all this submodule crap is constantly causing > pain ... do we really need this? Can't we find another solution instead? > Or at least stop modifying files automatically in the $SRC_PATH ? Also yesterday on IRC: <RaV3N> [...] I downloaded the qemu source from git and tried to compile it. I am getting this: ./configure --static && make && sudo make install CC ui/input-keymap.o ui/input-keymap.c:8:10: fatal error: ui/input-keymap-linux-to-qcode.c: No such file or directory <cota> [...] you might want to do "git submodule init && git submodule update" ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-08 12:26 ` Philippe Mathieu-Daudé @ 2017-11-08 13:01 ` Daniel P. Berrange 2017-11-10 10:35 ` Alexey Kardashevskiy 0 siblings, 1 reply; 21+ messages in thread From: Daniel P. Berrange @ 2017-11-08 13:01 UTC (permalink / raw) To: Philippe Mathieu-Daudé Cc: Thomas Huth, QEMU Developers, Peter Maydell, Gerd Hoffmann, Emilio G. Cota, Michal Suchánek On Wed, Nov 08, 2017 at 09:26:01AM -0300, Philippe Mathieu-Daudé wrote: > On 11/08/2017 06:57 AM, Thomas Huth wrote: > > > > That automatic git submodule stuff now broke my workflow again. I > > usually keep the git repository on my laptop and then simply rsync the > > sources (without .git directories) to my target machine to compile it > > there. Used to work great for years. Now it's broken, the build process > > complains: > > > > GIT submodule checkout is out of date. Please run > > scripts/git-submodule.sh update > > from the source directory checkout /home/thuth/devel/qemu > > > > Running "scripts/git-submodule.sh update" did not fix the issue at all - > > I first had to tinker with it for a while to find out that I simply have > > to delete ".git-submodule-status" in my git tree to fix the issue. > > > > I've got the feeling that all this submodule crap is constantly causing > > pain ... do we really need this? Can't we find another solution instead? > > Or at least stop modifying files automatically in the $SRC_PATH ? > > Also yesterday on IRC: > > <RaV3N> [...] I downloaded the qemu source from git and tried to compile > it. I am getting this: > > ./configure --static && make && sudo make install > CC ui/input-keymap.o > ui/input-keymap.c:8:10: fatal error: ui/input-keymap-linux-to-qcode.c: > No such file or directory I had a pull request merged yesterday later afternoon which possibly would address that problem, though hard hard to say for certain. > <cota> [...] you might want to do "git submodule init && git submodule > update" This certainly should not be required - the build system should always take care of that, so it would be considered a bug if it didn't happen. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-08 13:01 ` Daniel P. Berrange @ 2017-11-10 10:35 ` Alexey Kardashevskiy 2017-11-10 10:41 ` Daniel P. Berrange 0 siblings, 1 reply; 21+ messages in thread From: Alexey Kardashevskiy @ 2017-11-10 10:35 UTC (permalink / raw) To: Daniel P. Berrange, Philippe Mathieu-Daudé Cc: Peter Maydell, Thomas Huth, QEMU Developers, Emilio G. Cota, Gerd Hoffmann, Michal Suchánek On 09/11/17 00:01, Daniel P. Berrange wrote: > On Wed, Nov 08, 2017 at 09:26:01AM -0300, Philippe Mathieu-Daudé wrote: >> On 11/08/2017 06:57 AM, Thomas Huth wrote: >>> >>> That automatic git submodule stuff now broke my workflow again. I >>> usually keep the git repository on my laptop and then simply rsync the >>> sources (without .git directories) to my target machine to compile it >>> there. Used to work great for years. Now it's broken, the build process >>> complains: >>> >>> GIT submodule checkout is out of date. Please run >>> scripts/git-submodule.sh update >>> from the source directory checkout /home/thuth/devel/qemu >>> >>> Running "scripts/git-submodule.sh update" did not fix the issue at all - >>> I first had to tinker with it for a while to find out that I simply have >>> to delete ".git-submodule-status" in my git tree to fix the issue. >>> >>> I've got the feeling that all this submodule crap is constantly causing >>> pain ... do we really need this? Can't we find another solution instead? >>> Or at least stop modifying files automatically in the $SRC_PATH ? >> >> Also yesterday on IRC: >> >> <RaV3N> [...] I downloaded the qemu source from git and tried to compile >> it. I am getting this: >> >> ./configure --static && make && sudo make install >> CC ui/input-keymap.o >> ui/input-keymap.c:8:10: fatal error: ui/input-keymap-linux-to-qcode.c: >> No such file or directory > > I had a pull request merged yesterday later afternoon which possibly > would address that problem, though hard hard to say for certain. wow, already? :( I still wonder why do not we checkout submodules into the build directory and why .git-submodule-status is not there too... -- Alexey ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-10 10:35 ` Alexey Kardashevskiy @ 2017-11-10 10:41 ` Daniel P. Berrange 2017-11-10 13:46 ` Alexey Kardashevskiy 0 siblings, 1 reply; 21+ messages in thread From: Daniel P. Berrange @ 2017-11-10 10:41 UTC (permalink / raw) To: Alexey Kardashevskiy Cc: Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth, QEMU Developers, Emilio G. Cota, Gerd Hoffmann, Michal Suchánek On Fri, Nov 10, 2017 at 09:35:54PM +1100, Alexey Kardashevskiy wrote: > On 09/11/17 00:01, Daniel P. Berrange wrote: > > On Wed, Nov 08, 2017 at 09:26:01AM -0300, Philippe Mathieu-Daudé wrote: > >> On 11/08/2017 06:57 AM, Thomas Huth wrote: > >>> > >>> That automatic git submodule stuff now broke my workflow again. I > >>> usually keep the git repository on my laptop and then simply rsync the > >>> sources (without .git directories) to my target machine to compile it > >>> there. Used to work great for years. Now it's broken, the build process > >>> complains: > >>> > >>> GIT submodule checkout is out of date. Please run > >>> scripts/git-submodule.sh update > >>> from the source directory checkout /home/thuth/devel/qemu > >>> > >>> Running "scripts/git-submodule.sh update" did not fix the issue at all - > >>> I first had to tinker with it for a while to find out that I simply have > >>> to delete ".git-submodule-status" in my git tree to fix the issue. > >>> > >>> I've got the feeling that all this submodule crap is constantly causing > >>> pain ... do we really need this? Can't we find another solution instead? > >>> Or at least stop modifying files automatically in the $SRC_PATH ? > >> > >> Also yesterday on IRC: > >> > >> <RaV3N> [...] I downloaded the qemu source from git and tried to compile > >> it. I am getting this: > >> > >> ./configure --static && make && sudo make install > >> CC ui/input-keymap.o > >> ui/input-keymap.c:8:10: fatal error: ui/input-keymap-linux-to-qcode.c: > >> No such file or directory > > > > I had a pull request merged yesterday later afternoon which possibly > > would address that problem, though hard hard to say for certain. > > wow, already? :( > > I still wonder why do not we checkout submodules into the build directory > and why .git-submodule-status is not there too... That simply isn't the way submodules work, they are inherently part of the source tree, and the status file reflects that too. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-10 10:41 ` Daniel P. Berrange @ 2017-11-10 13:46 ` Alexey Kardashevskiy 2017-11-10 14:01 ` Peter Maydell 2017-11-10 14:22 ` Daniel P. Berrange 0 siblings, 2 replies; 21+ messages in thread From: Alexey Kardashevskiy @ 2017-11-10 13:46 UTC (permalink / raw) To: Daniel P. Berrange Cc: Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth, QEMU Developers, Emilio G. Cota, Gerd Hoffmann, Michal Suchánek On 10/11/17 21:41, Daniel P. Berrange wrote: > On Fri, Nov 10, 2017 at 09:35:54PM +1100, Alexey Kardashevskiy wrote: >> On 09/11/17 00:01, Daniel P. Berrange wrote: >>> On Wed, Nov 08, 2017 at 09:26:01AM -0300, Philippe Mathieu-Daudé wrote: >>>> On 11/08/2017 06:57 AM, Thomas Huth wrote: >>>>> >>>>> That automatic git submodule stuff now broke my workflow again. I >>>>> usually keep the git repository on my laptop and then simply rsync the >>>>> sources (without .git directories) to my target machine to compile it >>>>> there. Used to work great for years. Now it's broken, the build process >>>>> complains: >>>>> >>>>> GIT submodule checkout is out of date. Please run >>>>> scripts/git-submodule.sh update >>>>> from the source directory checkout /home/thuth/devel/qemu >>>>> >>>>> Running "scripts/git-submodule.sh update" did not fix the issue at all - >>>>> I first had to tinker with it for a while to find out that I simply have >>>>> to delete ".git-submodule-status" in my git tree to fix the issue. >>>>> >>>>> I've got the feeling that all this submodule crap is constantly causing >>>>> pain ... do we really need this? Can't we find another solution instead? >>>>> Or at least stop modifying files automatically in the $SRC_PATH ? >>>> >>>> Also yesterday on IRC: >>>> >>>> <RaV3N> [...] I downloaded the qemu source from git and tried to compile >>>> it. I am getting this: >>>> >>>> ./configure --static && make && sudo make install >>>> CC ui/input-keymap.o >>>> ui/input-keymap.c:8:10: fatal error: ui/input-keymap-linux-to-qcode.c: >>>> No such file or directory >>> >>> I had a pull request merged yesterday later afternoon which possibly >>> would address that problem, though hard hard to say for certain. >> >> wow, already? :( >> >> I still wonder why do not we checkout submodules into the build directory >> and why .git-submodule-status is not there too... > > That simply isn't the way submodules work, they are inherently part of > the source tree, and the status file reflects that too. Sorry, I am missing the point here. What precisely does prevent us from checking out the required modules to the build directory and build them there? git provides a submodule repository url and sha1 for the current qemu branch. Yes, the submodule tree inside the qemu tree might be different from the one in the build directory but the purpose of all of this is to always build the correct code - so this requirement is met. And it will still be better than changing the $SRC_PATH when a user specifically asked not to do that by calling "./configure --source-path='. -- Alexey ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-10 13:46 ` Alexey Kardashevskiy @ 2017-11-10 14:01 ` Peter Maydell 2017-11-11 0:15 ` Alexey Kardashevskiy 2017-11-10 14:22 ` Daniel P. Berrange 1 sibling, 1 reply; 21+ messages in thread From: Peter Maydell @ 2017-11-10 14:01 UTC (permalink / raw) To: Alexey Kardashevskiy Cc: Daniel P. Berrange, Philippe Mathieu-Daudé, Thomas Huth, QEMU Developers, Emilio G. Cota, Gerd Hoffmann, Michal Suchánek On 10 November 2017 at 13:46, Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > And it will still be > better than changing the $SRC_PATH when a user specifically asked not to do > that by calling "./configure --source-path='. I'm not terribly happy with the submodule stuff either, but that configure rune is not making use of a documented or supported feature. --source-path is for telling configure where the source code is. Passing it an empty string should probably be rejected as an error. thanks -- PMM ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-10 14:01 ` Peter Maydell @ 2017-11-11 0:15 ` Alexey Kardashevskiy 0 siblings, 0 replies; 21+ messages in thread From: Alexey Kardashevskiy @ 2017-11-11 0:15 UTC (permalink / raw) To: Peter Maydell Cc: Daniel P. Berrange, Philippe Mathieu-Daudé, Thomas Huth, QEMU Developers, Emilio G. Cota, Gerd Hoffmann, Michal Suchánek On 11/11/17 01:01, Peter Maydell wrote: > On 10 November 2017 at 13:46, Alexey Kardashevskiy <aik@ozlabs.ru> wrote: >> And it will still be >> better than changing the $SRC_PATH when a user specifically asked not to do >> that by calling "./configure --source-path='. > > I'm not terribly happy with the submodule stuff either, but > that configure rune is not making use of a documented > or supported feature. --source-path is for telling configure > where the source code is. Passing it an empty string should > probably be rejected as an error. It is not empty, I just cut an example too much :) The script looks like: cd /home/aik/pbuild/qemu-garrison2-ppc64/ /home/aik/p/qemu/configure --target-list=ppc64-softmmu --source-path=/home/aik/p/qemu/ -- Alexey ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-10 13:46 ` Alexey Kardashevskiy 2017-11-10 14:01 ` Peter Maydell @ 2017-11-10 14:22 ` Daniel P. Berrange 2017-11-11 1:10 ` Alexey Kardashevskiy 1 sibling, 1 reply; 21+ messages in thread From: Daniel P. Berrange @ 2017-11-10 14:22 UTC (permalink / raw) To: Alexey Kardashevskiy Cc: Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth, QEMU Developers, Emilio G. Cota, Gerd Hoffmann, Michal Suchánek On Sat, Nov 11, 2017 at 12:46:36AM +1100, Alexey Kardashevskiy wrote: > On 10/11/17 21:41, Daniel P. Berrange wrote: > > On Fri, Nov 10, 2017 at 09:35:54PM +1100, Alexey Kardashevskiy wrote: > >> On 09/11/17 00:01, Daniel P. Berrange wrote: > >>> On Wed, Nov 08, 2017 at 09:26:01AM -0300, Philippe Mathieu-Daudé wrote: > >>>> On 11/08/2017 06:57 AM, Thomas Huth wrote: > >>>>> > >>>>> That automatic git submodule stuff now broke my workflow again. I > >>>>> usually keep the git repository on my laptop and then simply rsync the > >>>>> sources (without .git directories) to my target machine to compile it > >>>>> there. Used to work great for years. Now it's broken, the build process > >>>>> complains: > >>>>> > >>>>> GIT submodule checkout is out of date. Please run > >>>>> scripts/git-submodule.sh update > >>>>> from the source directory checkout /home/thuth/devel/qemu > >>>>> > >>>>> Running "scripts/git-submodule.sh update" did not fix the issue at all - > >>>>> I first had to tinker with it for a while to find out that I simply have > >>>>> to delete ".git-submodule-status" in my git tree to fix the issue. > >>>>> > >>>>> I've got the feeling that all this submodule crap is constantly causing > >>>>> pain ... do we really need this? Can't we find another solution instead? > >>>>> Or at least stop modifying files automatically in the $SRC_PATH ? > >>>> > >>>> Also yesterday on IRC: > >>>> > >>>> <RaV3N> [...] I downloaded the qemu source from git and tried to compile > >>>> it. I am getting this: > >>>> > >>>> ./configure --static && make && sudo make install > >>>> CC ui/input-keymap.o > >>>> ui/input-keymap.c:8:10: fatal error: ui/input-keymap-linux-to-qcode.c: > >>>> No such file or directory > >>> > >>> I had a pull request merged yesterday later afternoon which possibly > >>> would address that problem, though hard hard to say for certain. > >> > >> wow, already? :( > >> > >> I still wonder why do not we checkout submodules into the build directory > >> and why .git-submodule-status is not there too... > > > > That simply isn't the way submodules work, they are inherently part of > > the source tree, and the status file reflects that too. > > Sorry, I am missing the point here. What precisely does prevent us from > checking out the required modules to the build directory and build them > there? git provides a submodule repository url and sha1 for the current > qemu branch. The build directory should never contain any of your version controlled source, as that will get irretrievably lost when the build dir is purged. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-10 14:22 ` Daniel P. Berrange @ 2017-11-11 1:10 ` Alexey Kardashevskiy 2017-11-20 1:06 ` Alexey Kardashevskiy 0 siblings, 1 reply; 21+ messages in thread From: Alexey Kardashevskiy @ 2017-11-11 1:10 UTC (permalink / raw) To: Daniel P. Berrange Cc: Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth, QEMU Developers, Emilio G. Cota, Gerd Hoffmann, Michal Suchánek On 11/11/17 01:22, Daniel P. Berrange wrote: > On Sat, Nov 11, 2017 at 12:46:36AM +1100, Alexey Kardashevskiy wrote: >> On 10/11/17 21:41, Daniel P. Berrange wrote: >>> On Fri, Nov 10, 2017 at 09:35:54PM +1100, Alexey Kardashevskiy wrote: >>>> On 09/11/17 00:01, Daniel P. Berrange wrote: >>>>> On Wed, Nov 08, 2017 at 09:26:01AM -0300, Philippe Mathieu-Daudé wrote: >>>>>> On 11/08/2017 06:57 AM, Thomas Huth wrote: >>>>>>> >>>>>>> That automatic git submodule stuff now broke my workflow again. I >>>>>>> usually keep the git repository on my laptop and then simply rsync the >>>>>>> sources (without .git directories) to my target machine to compile it >>>>>>> there. Used to work great for years. Now it's broken, the build process >>>>>>> complains: >>>>>>> >>>>>>> GIT submodule checkout is out of date. Please run >>>>>>> scripts/git-submodule.sh update >>>>>>> from the source directory checkout /home/thuth/devel/qemu >>>>>>> >>>>>>> Running "scripts/git-submodule.sh update" did not fix the issue at all - >>>>>>> I first had to tinker with it for a while to find out that I simply have >>>>>>> to delete ".git-submodule-status" in my git tree to fix the issue. >>>>>>> >>>>>>> I've got the feeling that all this submodule crap is constantly causing >>>>>>> pain ... do we really need this? Can't we find another solution instead? >>>>>>> Or at least stop modifying files automatically in the $SRC_PATH ? >>>>>> >>>>>> Also yesterday on IRC: >>>>>> >>>>>> <RaV3N> [...] I downloaded the qemu source from git and tried to compile >>>>>> it. I am getting this: >>>>>> >>>>>> ./configure --static && make && sudo make install >>>>>> CC ui/input-keymap.o >>>>>> ui/input-keymap.c:8:10: fatal error: ui/input-keymap-linux-to-qcode.c: >>>>>> No such file or directory >>>>> >>>>> I had a pull request merged yesterday later afternoon which possibly >>>>> would address that problem, though hard hard to say for certain. >>>> >>>> wow, already? :( >>>> >>>> I still wonder why do not we checkout submodules into the build directory >>>> and why .git-submodule-status is not there too... >>> >>> That simply isn't the way submodules work, they are inherently part of >>> the source tree, and the status file reflects that too. >> >> Sorry, I am missing the point here. What precisely does prevent us from >> checking out the required modules to the build directory and build them >> there? git provides a submodule repository url and sha1 for the current >> qemu branch. > > The build directory should never contain any of your version controlled > source, as that will get irretrievably lost when the build dir is purged. I am not suggesting editing submodule sources from the build directory, I am suggesting not building them from the source directory, in order to keep them in sync, Makefile can rsync or "git push" them to the build directory. Yeah, fairly ugly but still more correct than the current solution. And again, .git-submodule-status is not a source file, what is it doing in $SRC_PATH? Now I cannot compile qemu pretty much 50% of time because of that enhancement. In the today's episode: ssh aikhostos2 cd /home/aik/pbuild/qemu-aikhostos2-ppc64/ ; /home/aik/p/qemu/configure --target-list=ppc64-softmmu --source-path=/home/aik/p/qemu/ --enable-debug --enable-debug-info --disable-werror --disable-git-update --enable-trace-backend=log All good. One detail: capstone git ssh aikhostos2 make -C /home/aik/pbuild/qemu-aikhostos2-ppc64/ -j24 make: Entering directory `/home/aik/pbuild/qemu-aikhostos2-ppc64' GEN ppc64-softmmu/config-devices.mak.tmp mkdir -p dtc/libfdt mkdir -p dtc/tests GEN config-host.h GEN qemu-options.def make[1]: *** No rule to make target `/home/aik/pbuild/qemu-aikhostos2-ppc64/capstone/libcapstone.a'. Stop. make: *** [subdir-capstone] Error 2 make: *** Waiting for unfinished jobs.... GEN ppc64-softmmu/config-devices.mak How, why, where are all these nice warning messages? Ah, there they are (added 'set -x' to the script): + substat=.git-submodule-status + command=status + shift + maybe_modules='ui/keycodemapdb dtc capstone' + test -z git + modules= + for m in '$maybe_modules' + git submodule status ui/keycodemapdb + test 128 = 0 + echo 'warn: ignoring non-existent submodule ui/keycodemapdb' + for m in '$maybe_modules' + git submodule status dtc + test 128 = 0:m + echo 'warn: ignoring non-existent submodule dtc' + for m in '$maybe_modules' + git submodule status capstone + test 128 = 0 + echo 'warn: ignoring non-existent submodule capstone' + test -n 'ui/keycodemapdb dtc capstone' + test -e .git + case "$command" in + test -z 'ui/keycodemapdb dtc capstone' + test -f .git-submodule-status + exit 1 I assume that 'warn' is not printed because of Makefile's 'quiet-command'. And this fails because on the server the source directory is created with 'git worktree' which points to bare git repo outside of the source tree and the git on that build machine has no idea about worktrees (the repo is visible to the build machine). [vpl1 ~]$ ssh -x aikhostos2 'cd p/qemu && pwd && git --version && ls -d ../qemu.git && git submodule status' /home/aik/p/qemu git version 1.8.3.1 ../qemu.git fatal: Not a git repository: /home/aik/p/qemu.git/worktrees/qemu Again - ok, when I know what is the problem, I know how to fix it - simply move .git away when running ./configure and make (as it also runs ./configure some time) - but the messages... Firstly ./configure decides that 'capstone' comes from 'git', then it simply fails with no indication why it went wrong. Ha. I just found another workaround - I can do ./configure --with-git=false and silently disable git updates. Pretty much pointing a gun to my leg, ready to shoot. We could just check git submodule status every build and print a warning if it is out of sync but continue. This way I would look at the submodules list and decide what to do. -- Alexey ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-11 1:10 ` Alexey Kardashevskiy @ 2017-11-20 1:06 ` Alexey Kardashevskiy 0 siblings, 0 replies; 21+ messages in thread From: Alexey Kardashevskiy @ 2017-11-20 1:06 UTC (permalink / raw) To: Daniel P. Berrange Cc: Philippe Mathieu-Daudé, Peter Maydell, Thomas Huth, QEMU Developers, Emilio G. Cota, Gerd Hoffmann, Michal Suchánek On 11/11/17 12:10, Alexey Kardashevskiy wrote: > On 11/11/17 01:22, Daniel P. Berrange wrote: >> On Sat, Nov 11, 2017 at 12:46:36AM +1100, Alexey Kardashevskiy wrote: >>> On 10/11/17 21:41, Daniel P. Berrange wrote: >>>> On Fri, Nov 10, 2017 at 09:35:54PM +1100, Alexey Kardashevskiy wrote: >>>>> On 09/11/17 00:01, Daniel P. Berrange wrote: >>>>>> On Wed, Nov 08, 2017 at 09:26:01AM -0300, Philippe Mathieu-Daudé wrote: >>>>>>> On 11/08/2017 06:57 AM, Thomas Huth wrote: >>>>>>>> >>>>>>>> That automatic git submodule stuff now broke my workflow again. I >>>>>>>> usually keep the git repository on my laptop and then simply rsync the >>>>>>>> sources (without .git directories) to my target machine to compile it >>>>>>>> there. Used to work great for years. Now it's broken, the build process >>>>>>>> complains: >>>>>>>> >>>>>>>> GIT submodule checkout is out of date. Please run >>>>>>>> scripts/git-submodule.sh update >>>>>>>> from the source directory checkout /home/thuth/devel/qemu >>>>>>>> >>>>>>>> Running "scripts/git-submodule.sh update" did not fix the issue at all - >>>>>>>> I first had to tinker with it for a while to find out that I simply have >>>>>>>> to delete ".git-submodule-status" in my git tree to fix the issue. >>>>>>>> >>>>>>>> I've got the feeling that all this submodule crap is constantly causing >>>>>>>> pain ... do we really need this? Can't we find another solution instead? >>>>>>>> Or at least stop modifying files automatically in the $SRC_PATH ? >>>>>>> >>>>>>> Also yesterday on IRC: >>>>>>> >>>>>>> <RaV3N> [...] I downloaded the qemu source from git and tried to compile >>>>>>> it. I am getting this: >>>>>>> >>>>>>> ./configure --static && make && sudo make install >>>>>>> CC ui/input-keymap.o >>>>>>> ui/input-keymap.c:8:10: fatal error: ui/input-keymap-linux-to-qcode.c: >>>>>>> No such file or directory >>>>>> >>>>>> I had a pull request merged yesterday later afternoon which possibly >>>>>> would address that problem, though hard hard to say for certain. >>>>> >>>>> wow, already? :( >>>>> >>>>> I still wonder why do not we checkout submodules into the build directory >>>>> and why .git-submodule-status is not there too... >>>> >>>> That simply isn't the way submodules work, they are inherently part of >>>> the source tree, and the status file reflects that too. >>> >>> Sorry, I am missing the point here. What precisely does prevent us from >>> checking out the required modules to the build directory and build them >>> there? git provides a submodule repository url and sha1 for the current >>> qemu branch. >> >> The build directory should never contain any of your version controlled >> source, as that will get irretrievably lost when the build dir is purged. > > I am not suggesting editing submodule sources from the build directory, I > am suggesting not building them from the source directory, in order to keep > them in sync, Makefile can rsync or "git push" them to the build directory. > Yeah, fairly ugly but still more correct than the current solution. > > And again, .git-submodule-status is not a source file, what is it doing in > $SRC_PATH? > > Now I cannot compile qemu pretty much 50% of time because of that > enhancement. In the today's episode: > > ssh aikhostos2 cd /home/aik/pbuild/qemu-aikhostos2-ppc64/ ; > /home/aik/p/qemu/configure --target-list=ppc64-softmmu > --source-path=/home/aik/p/qemu/ --enable-debug --enable-debug-info > --disable-werror --disable-git-update --enable-trace-backend=log > > All good. One detail: > capstone git > > ssh aikhostos2 make -C /home/aik/pbuild/qemu-aikhostos2-ppc64/ -j24 > > make: Entering directory `/home/aik/pbuild/qemu-aikhostos2-ppc64' > GEN ppc64-softmmu/config-devices.mak.tmp > mkdir -p dtc/libfdt > mkdir -p dtc/tests > GEN config-host.h > GEN qemu-options.def > make[1]: *** No rule to make target > `/home/aik/pbuild/qemu-aikhostos2-ppc64/capstone/libcapstone.a'. Stop. > make: *** [subdir-capstone] Error 2 > make: *** Waiting for unfinished jobs.... > GEN ppc64-softmmu/config-devices.mak > > > How, why, where are all these nice warning messages? Ah, there they are > (added 'set -x' to the script): > > + substat=.git-submodule-status > + command=status > + shift > + maybe_modules='ui/keycodemapdb dtc capstone' > + test -z git > + modules= > + for m in '$maybe_modules' > + git submodule status ui/keycodemapdb > + test 128 = 0 > + echo 'warn: ignoring non-existent submodule ui/keycodemapdb' > + for m in '$maybe_modules' > + git submodule status dtc > + test 128 = 0:m > + echo 'warn: ignoring non-existent submodule dtc' > + for m in '$maybe_modules' > + git submodule status capstone > + test 128 = 0 > + echo 'warn: ignoring non-existent submodule capstone' > + test -n 'ui/keycodemapdb dtc capstone' > + test -e .git > + case "$command" in > + test -z 'ui/keycodemapdb dtc capstone' > + test -f .git-submodule-status > + exit 1 > > > I assume that 'warn' is not printed because of Makefile's 'quiet-command'. > > And this fails because on the server the source directory is created with > 'git worktree' which points to bare git repo outside of the source tree and > the git on that build machine has no idea about worktrees (the repo is > visible to the build machine). > > [vpl1 ~]$ ssh -x aikhostos2 'cd p/qemu && pwd && git --version && ls -d > ../qemu.git && git submodule status' > /home/aik/p/qemu > git version 1.8.3.1 > ../qemu.git > fatal: Not a git repository: /home/aik/p/qemu.git/worktrees/qemu > > > Again - ok, when I know what is the problem, I know how to fix it - simply > move .git away when running ./configure and make (as it also runs > ./configure some time) - but the messages... Firstly ./configure decides > that 'capstone' comes from 'git', then it simply fails with no indication > why it went wrong. > > Ha. I just found another workaround - I can do ./configure --with-git=false > and silently disable git updates. Pretty much pointing a gun to my leg, > ready to shoot. > > We could just check git submodule status every build and print a warning if > it is out of sync but continue. This way I would look at the submodules > list and decide what to do. I assume that no response to this means my proposal is somehow worse that what we got now, correct? Thanks. -- Alexey ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [Qemu-devel] Yet another git submodule rant 2017-11-08 9:57 [Qemu-devel] Yet another git submodule rant Thomas Huth ` (2 preceding siblings ...) 2017-11-08 12:26 ` Philippe Mathieu-Daudé @ 2017-11-09 3:02 ` Alexey Kardashevskiy 3 siblings, 0 replies; 21+ messages in thread From: Alexey Kardashevskiy @ 2017-11-09 3:02 UTC (permalink / raw) To: Thomas Huth, QEMU Developers, Daniel P. Berrange Cc: Peter Maydell, Gerd Hoffmann On 08/11/17 20:57, Thomas Huth wrote: > > That automatic git submodule stuff now broke my workflow again. I > usually keep the git repository on my laptop and then simply rsync the > sources (without .git directories) to my target machine to compile it > there. Used to work great for years. Now it's broken, the build process > complains: > > GIT submodule checkout is out of date. Please run > scripts/git-submodule.sh update > from the source directory checkout /home/thuth/devel/qemu > > Running "scripts/git-submodule.sh update" did not fix the issue at all - > I first had to tinker with it for a while to find out that I simply have > to delete ".git-submodule-status" in my git tree to fix the issue. > > I've got the feeling that all this submodule crap is constantly causing > pain ... do we really need this? Can't we find another solution instead? > Or at least stop modifying files automatically in the $SRC_PATH ? > > ... ok, sorry for the harsh words, ... but now that I've written this > down, I feel at least a little bit better ... Since there is resistance to add a way to disable changing $SRC_PATH, I guess the way to go is simply move .git folder outside of source tree and use $GIT_DIR or --work-dir only when you want git to manage the sources - this should make $git_submodules empty in ./configure and git-submodule.sh not to touch anything. -- Alexey ^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2017-11-20 1:06 UTC | newest] Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-11-08 9:57 [Qemu-devel] Yet another git submodule rant Thomas Huth 2017-11-08 10:25 ` Daniel P. Berrange 2017-11-08 12:05 ` Michal Suchánek 2017-11-08 12:34 ` Daniel P. Berrange 2017-11-08 12:43 ` Michal Suchánek 2017-11-08 12:55 ` Daniel P. Berrange 2017-11-08 10:53 ` Gerd Hoffmann 2017-11-08 11:48 ` Thomas Huth 2017-11-08 13:30 ` Philippe Mathieu-Daudé 2017-11-08 16:09 ` Laszlo Ersek 2017-11-08 12:26 ` Philippe Mathieu-Daudé 2017-11-08 13:01 ` Daniel P. Berrange 2017-11-10 10:35 ` Alexey Kardashevskiy 2017-11-10 10:41 ` Daniel P. Berrange 2017-11-10 13:46 ` Alexey Kardashevskiy 2017-11-10 14:01 ` Peter Maydell 2017-11-11 0:15 ` Alexey Kardashevskiy 2017-11-10 14:22 ` Daniel P. Berrange 2017-11-11 1:10 ` Alexey Kardashevskiy 2017-11-20 1:06 ` Alexey Kardashevskiy 2017-11-09 3:02 ` Alexey Kardashevskiy
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.