* check-tcg errors (build-user, build-user-plugins) again @ 2020-12-02 9:32 Claudio Fontana 2020-12-02 11:16 ` Alex Bennée 2021-01-10 21:17 ` check-tcg HOWTO? Claudio Fontana 0 siblings, 2 replies; 9+ messages in thread From: Claudio Fontana @ 2020-12-02 9:32 UTC (permalink / raw) To: Alex Bennee; +Cc: Philippe Mathieu-Daudé, qemu-devel Hi Alex and all, when trying to use check-tcg (master), I am getting often these errors: $ ../configure --disable-system --disable-tools $ make -j12 check-tcg ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid Trying to pull registry.gitlab.com/qemu-project/qemu/qemu/debian11... Trying to pull registry.gitlab.com/qemu-project/qemu/qemu/fedora-cris-cross... Trying to pull registry.gitlab.com/qemu-project/qemu/qemu/debian10... ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid [...] TEST linux-test on x86_64 timeout: failed to run command ‘/home/claudio/git/qemu/build/qemu-x86_64’timeout: : No such file or directoryfailed to run command ‘/home/claudio/git/qemu/build/qemu-x86_64’ [...] Is there some pre-configuration on the host necessary to be able to run check-tcg? I see these errors in gitlab also for build-user build-user-plugin Maybe this is what Philippe mentioned before though, that this is expected at the moment due to a temporary Meson shortcoming? Ciao, Claudio -- Claudio Fontana Engineering Manager Virtualization, SUSE Labs Core SUSE Software Solutions Italy Srl ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: check-tcg errors (build-user, build-user-plugins) again 2020-12-02 9:32 check-tcg errors (build-user, build-user-plugins) again Claudio Fontana @ 2020-12-02 11:16 ` Alex Bennée 2020-12-02 11:25 ` Claudio Fontana 2021-01-10 21:17 ` check-tcg HOWTO? Claudio Fontana 1 sibling, 1 reply; 9+ messages in thread From: Alex Bennée @ 2020-12-02 11:16 UTC (permalink / raw) To: Claudio Fontana Cc: Marc-André Lureau, Philippe Mathieu-Daudé, qemu-devel Claudio Fontana <cfontana@suse.de> writes: > Hi Alex and all, > > when trying to use check-tcg (master), I am getting often these errors: > > $ ../configure --disable-system --disable-tools > > $ make -j12 check-tcg > > ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid > ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid > ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid > Trying to pull registry.gitlab.com/qemu-project/qemu/qemu/debian11... > Trying to pull registry.gitlab.com/qemu-project/qemu/qemu/fedora-cris-cross... > Trying to pull registry.gitlab.com/qemu-project/qemu/qemu/debian10... > ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid > > [...] > TEST linux-test on x86_64 > timeout: failed to run command ‘/home/claudio/git/qemu/build/qemu-x86_64’timeout: : No such file or directoryfailed to run command ‘/home/claudio/git/qemu/build/qemu-x86_64’ > > [...] > > > Is there some pre-configuration on the host necessary to be able to > run check-tcg? There shouldn't be but those errors remind me of some of the tweaks I had to make to me Gentoo system when using podman (instead of docker). In the end I think I just ended up adding the lines: alex:100000:65536 to /etc/subgid and /etc/subgid- Marc-André may have some better pointers as he added podman support to the builder scripts. The main difference between the images on the registry and the local versions is most add the current user so there is a clean mapping between the container user and the host file-system. It's the last step of the build so we still use the cached layers from the registry versions. > I see these errors in gitlab also for > > build-user > build-user-plugin > > Maybe this is what Philippe mentioned before though, that this is > expected at the moment due to a temporary Meson shortcoming? That is odd - I'm not seeing anything like that on the master builds: https://gitlab.com/qemu-project/qemu/-/jobs/883985106 https://gitlab.com/qemu-project/qemu/-/jobs/883985113 AFAIK GitLab is still using Docker to build it's containers (albeit with BUILDKIT enabled). > > Ciao, > > Claudio -- Alex Bennée ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: check-tcg errors (build-user, build-user-plugins) again 2020-12-02 11:16 ` Alex Bennée @ 2020-12-02 11:25 ` Claudio Fontana 2020-12-02 12:52 ` Alex Bennée 2020-12-02 14:25 ` Philippe Mathieu-Daudé 0 siblings, 2 replies; 9+ messages in thread From: Claudio Fontana @ 2020-12-02 11:25 UTC (permalink / raw) To: Alex Bennée Cc: Marc-André Lureau, Philippe Mathieu-Daudé, qemu-devel On 12/2/20 12:16 PM, Alex Bennée wrote: > > Claudio Fontana <cfontana@suse.de> writes: > >> Hi Alex and all, >> >> when trying to use check-tcg (master), I am getting often these errors: >> >> $ ../configure --disable-system --disable-tools >> >> $ make -j12 check-tcg >> >> ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid >> ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid >> ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid >> Trying to pull registry.gitlab.com/qemu-project/qemu/qemu/debian11... >> Trying to pull registry.gitlab.com/qemu-project/qemu/qemu/fedora-cris-cross... >> Trying to pull registry.gitlab.com/qemu-project/qemu/qemu/debian10... >> ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid >> >> [...] >> TEST linux-test on x86_64 >> timeout: failed to run command ‘/home/claudio/git/qemu/build/qemu-x86_64’timeout: : No such file or directoryfailed to run command ‘/home/claudio/git/qemu/build/qemu-x86_64’ >> >> [...] >> >> >> Is there some pre-configuration on the host necessary to be able to >> run check-tcg? > > There shouldn't be but those errors remind me of some of the tweaks I > had to make to me Gentoo system when using podman (instead of docker). > In the end I think I just ended up adding the lines: > > alex:100000:65536 > > to /etc/subgid and /etc/subgid- > > Marc-André may have some better pointers as he added podman support to > the builder scripts. I did that and things seem a bit better, but still a lot of errors: 63 ../sysdeps/x86_64/start.S: No such file or directory. Error: error creating build container: The following failures happened while trying to pull image specified by "debian:bullseye-slim" based on search registries in /etc/containers/registries.conf: * "localhost/debian:bullseye-slim": Error initializing source docker://localhost/debian:bullseye-slim: error pinging docker registry localhost: Get https://localhost/v2/: dial tcp [::1]:443: connect: connection refused * "docker.io/library/debian:bullseye-slim": Error committing the finished image: error adding layer with blob "sha256:ae63fcbbc3b289e425e4c8840ccde4314f4a060cbc0345e6871a28bdc72f6fe8": Error processing tar file(exit status 1): there might not be enough IDs available in the namespace (requested 0:42 for /etc/gshadow): lchown /etc/gshadow: invalid argument Traceback (most recent call last): File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 709, in <module> sys.exit(main()) File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 705, in main return args.cmdobj.run(args, argv) File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 501, in run extra_files_cksum=cksum) File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 354, in build_image quiet=quiet) File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 244, in _do_check return subprocess.check_call(self._command + cmd, **kwargs) File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call raise CalledProcessError(retcode, cmd) [...] Error: error pulling image "registry.gitlab.com/qemu-project/qemu/qemu/fedora-cris-cross": unable to pull registry.gitlab.com/qemu-project/ [...] > > The main difference between the images on the registry and the local > versions is most add the current user so there is a clean mapping > between the container user and the host file-system. It's the last step > of the build so we still use the cached layers from the registry > versions. > >> I see these errors in gitlab also for >> >> build-user >> build-user-plugin >> >> Maybe this is what Philippe mentioned before though, that this is >> expected at the moment due to a temporary Meson shortcoming? > > That is odd - I'm not seeing anything like that on the master builds: > > https://gitlab.com/qemu-project/qemu/-/jobs/883985106 > https://gitlab.com/qemu-project/qemu/-/jobs/883985113 > > AFAIK GitLab is still using Docker to build it's containers (albeit with > BUILDKIT enabled). I am running again on gitlab the master branch, maybe there is something I need to fix, but to do that I need to enable check-tcg successfully I think. Thanks! Claudio > >> >> Ciao, >> >> Claudio > > ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: check-tcg errors (build-user, build-user-plugins) again 2020-12-02 11:25 ` Claudio Fontana @ 2020-12-02 12:52 ` Alex Bennée 2020-12-02 14:24 ` Claudio Fontana 2020-12-02 14:25 ` Philippe Mathieu-Daudé 1 sibling, 1 reply; 9+ messages in thread From: Alex Bennée @ 2020-12-02 12:52 UTC (permalink / raw) To: Claudio Fontana Cc: Marc-André Lureau, Philippe Mathieu-Daudé, qemu-devel Claudio Fontana <cfontana@suse.de> writes: > On 12/2/20 12:16 PM, Alex Bennée wrote: >> >> Claudio Fontana <cfontana@suse.de> writes: >> >>> Hi Alex and all, >>> >>> when trying to use check-tcg (master), I am getting often these errors: >>> >>> $ ../configure --disable-system --disable-tools >>> >>> $ make -j12 check-tcg >>> >>> ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid >>> ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid >>> ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid >>> Trying to pull registry.gitlab.com/qemu-project/qemu/qemu/debian11... >>> Trying to pull registry.gitlab.com/qemu-project/qemu/qemu/fedora-cris-cross... >>> Trying to pull registry.gitlab.com/qemu-project/qemu/qemu/debian10... >>> ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid >>> >>> [...] >>> TEST linux-test on x86_64 >>> timeout: failed to run command ‘/home/claudio/git/qemu/build/qemu-x86_64’timeout: : No such file or directoryfailed to run command ‘/home/claudio/git/qemu/build/qemu-x86_64’ >>> >>> [...] >>> >>> >>> Is there some pre-configuration on the host necessary to be able to >>> run check-tcg? >> >> There shouldn't be but those errors remind me of some of the tweaks I >> had to make to me Gentoo system when using podman (instead of docker). >> In the end I think I just ended up adding the lines: >> >> alex:100000:65536 >> >> to /etc/subgid and /etc/subgid- >> >> Marc-André may have some better pointers as he added podman support to >> the builder scripts. > > > I did that and things seem a bit better, but still a lot of errors: > > > 63 ../sysdeps/x86_64/start.S: No such file or directory. > > Error: error creating build container: The following failures happened while trying to pull image specified by "debian:bullseye-slim" based on search registries in /etc/containers/registries.conf: > * "localhost/debian:bullseye-slim": Error initializing source docker://localhost/debian:bullseye-slim: error pinging docker registry localhost: Get https://localhost/v2/: dial tcp [::1]:443: connect: connection refused > * "docker.io/library/debian:bullseye-slim": Error committing the finished image: error adding layer with blob "sha256:ae63fcbbc3b289e425e4c8840ccde4314f4a060cbc0345e6871a28bdc72f6fe8": Error processing tar file(exit status 1): there might not be enough IDs available in the namespace (requested 0:42 for /etc/gshadow): lchown /etc/gshadow: invalid argument > Traceback (most recent call last): > File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 709, in <module> > sys.exit(main()) > File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 705, in main > return args.cmdobj.run(args, argv) > File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 501, in run > extra_files_cksum=cksum) > File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 354, in build_image > quiet=quiet) > File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 244, in _do_check > return subprocess.check_call(self._command + cmd, **kwargs) > File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call > raise CalledProcessError(retcode, cmd) > > > [...] > Error: error pulling image "registry.gitlab.com/qemu-project/qemu/qemu/fedora-cris-cross": unable to pull registry.gitlab.com/qemu-project/ > I'm guessing this can be fixed by adding gitlab to /etc/containers/registries.conf I'll see if I can resurrect my podman setup because it was working before we added the caching from gitlab. > [...] > > > >> >> The main difference between the images on the registry and the local >> versions is most add the current user so there is a clean mapping >> between the container user and the host file-system. It's the last step >> of the build so we still use the cached layers from the registry >> versions. >> >>> I see these errors in gitlab also for >>> >>> build-user >>> build-user-plugin >>> >>> Maybe this is what Philippe mentioned before though, that this is >>> expected at the moment due to a temporary Meson shortcoming? >> >> That is odd - I'm not seeing anything like that on the master builds: >> >> https://gitlab.com/qemu-project/qemu/-/jobs/883985106 >> https://gitlab.com/qemu-project/qemu/-/jobs/883985113 >> >> AFAIK GitLab is still using Docker to build it's containers (albeit with >> BUILDKIT enabled). > > > I am running again on gitlab the master branch, maybe there is something I need to fix, but to do that I need to enable check-tcg successfully I think. > > Thanks! > > Claudio > >> >>> >>> Ciao, >>> >>> Claudio >> >> -- Alex Bennée ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: check-tcg errors (build-user, build-user-plugins) again 2020-12-02 12:52 ` Alex Bennée @ 2020-12-02 14:24 ` Claudio Fontana 0 siblings, 0 replies; 9+ messages in thread From: Claudio Fontana @ 2020-12-02 14:24 UTC (permalink / raw) To: Alex Bennée Cc: Marc-André Lureau, Philippe Mathieu-Daudé, qemu-devel On 12/2/20 1:52 PM, Alex Bennée wrote: > > Claudio Fontana <cfontana@suse.de> writes: > >> On 12/2/20 12:16 PM, Alex Bennée wrote: >>> >>> Claudio Fontana <cfontana@suse.de> writes: >>> >>>> Hi Alex and all, >>>> >>>> when trying to use check-tcg (master), I am getting often these errors: >>>> >>>> $ ../configure --disable-system --disable-tools >>>> >>>> $ make -j12 check-tcg >>>> >>>> ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid >>>> ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid >>>> ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid >>>> Trying to pull registry.gitlab.com/qemu-project/qemu/qemu/debian11... >>>> Trying to pull registry.gitlab.com/qemu-project/qemu/qemu/fedora-cris-cross... >>>> Trying to pull registry.gitlab.com/qemu-project/qemu/qemu/debian10... >>>> ERRO[0000] cannot find mappings for user claudio: No subgid ranges found for group "claudio" in /etc/subgid >>>> >>>> [...] >>>> TEST linux-test on x86_64 >>>> timeout: failed to run command ‘/home/claudio/git/qemu/build/qemu-x86_64’timeout: : No such file or directoryfailed to run command ‘/home/claudio/git/qemu/build/qemu-x86_64’ >>>> >>>> [...] >>>> >>>> >>>> Is there some pre-configuration on the host necessary to be able to >>>> run check-tcg? >>> >>> There shouldn't be but those errors remind me of some of the tweaks I >>> had to make to me Gentoo system when using podman (instead of docker). >>> In the end I think I just ended up adding the lines: >>> >>> alex:100000:65536 >>> >>> to /etc/subgid and /etc/subgid- >>> >>> Marc-André may have some better pointers as he added podman support to >>> the builder scripts. >> >> >> I did that and things seem a bit better, but still a lot of errors: >> >> >> 63 ../sysdeps/x86_64/start.S: No such file or directory. >> >> Error: error creating build container: The following failures happened while trying to pull image specified by "debian:bullseye-slim" based on search registries in /etc/containers/registries.conf: >> * "localhost/debian:bullseye-slim": Error initializing source docker://localhost/debian:bullseye-slim: error pinging docker registry localhost: Get https://localhost/v2/: dial tcp [::1]:443: connect: connection refused >> * "docker.io/library/debian:bullseye-slim": Error committing the finished image: error adding layer with blob "sha256:ae63fcbbc3b289e425e4c8840ccde4314f4a060cbc0345e6871a28bdc72f6fe8": Error processing tar file(exit status 1): there might not be enough IDs available in the namespace (requested 0:42 for /etc/gshadow): lchown /etc/gshadow: invalid argument >> Traceback (most recent call last): >> File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 709, in <module> >> sys.exit(main()) >> File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 705, in main >> return args.cmdobj.run(args, argv) >> File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 501, in run >> extra_files_cksum=cksum) >> File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 354, in build_image >> quiet=quiet) >> File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 244, in _do_check >> return subprocess.check_call(self._command + cmd, **kwargs) >> File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call >> raise CalledProcessError(retcode, cmd) >> >> >> [...] >> Error: error pulling image "registry.gitlab.com/qemu-project/qemu/qemu/fedora-cris-cross": unable to pull registry.gitlab.com/qemu-project/ >> > > I'm guessing this can be fixed by adding gitlab to /etc/containers/registries.conf Hi Alex, I added: [registries.search] registries = ["docker.io", "registry.gitlab.com"] I get: Storing signatures Error processing tar file(exit status 1): there might not be enough IDs available in the namespace (requested 0:42 for /etc/gshadow): lchown /etc/gshadow: invalid argument Error: error pulling image "registry.gitlab.com/qemu-project/qemu/qemu/debian11": unable to pull registry.gitlab.com/qemu-project/qemu/qemu/debian11: unable to pull image: Error committing the finished image: error adding layer with blob "sha256:ae63fcbbc3b289e425e4c8840ccde4314f4a060cbc0345e6871a28bdc72f6fe8": Error processing tar file(exit status 1): there might not be enough IDs available in the namespace (requested 0:42 for /etc/gshadow): lchown /etc/gshadow: invalid argument Any idea? > > I'll see if I can resurrect my podman setup because it was working before > we added the caching from gitlab. > >> [...] >> >> >> >>> >>> The main difference between the images on the registry and the local >>> versions is most add the current user so there is a clean mapping >>> between the container user and the host file-system. It's the last step >>> of the build so we still use the cached layers from the registry >>> versions. >>> >>>> I see these errors in gitlab also for >>>> >>>> build-user >>>> build-user-plugin >>>> >>>> Maybe this is what Philippe mentioned before though, that this is >>>> expected at the moment due to a temporary Meson shortcoming? >>> >>> That is odd - I'm not seeing anything like that on the master builds: >>> >>> https://gitlab.com/qemu-project/qemu/-/jobs/883985106 >>> https://gitlab.com/qemu-project/qemu/-/jobs/883985113 >>> >>> AFAIK GitLab is still using Docker to build it's containers (albeit with >>> BUILDKIT enabled). >> >> >> I am running again on gitlab the master branch, maybe there is something I need to fix, but to do that I need to enable check-tcg successfully I think. >> >> Thanks! >> >> Claudio >> >>> >>>> >>>> Ciao, >>>> >>>> Claudio >>> >>> > > ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: check-tcg errors (build-user, build-user-plugins) again 2020-12-02 11:25 ` Claudio Fontana 2020-12-02 12:52 ` Alex Bennée @ 2020-12-02 14:25 ` Philippe Mathieu-Daudé 1 sibling, 0 replies; 9+ messages in thread From: Philippe Mathieu-Daudé @ 2020-12-02 14:25 UTC (permalink / raw) To: Claudio Fontana, Alex Bennée Cc: Marc-André Lureau, Daniel P . Berrange, qemu-devel On 12/2/20 12:25 PM, Claudio Fontana wrote: > On 12/2/20 12:16 PM, Alex Bennée wrote: >> Claudio Fontana <cfontana@suse.de> writes: >>> Is there some pre-configuration on the host necessary to be able to >>> run check-tcg? >> >> There shouldn't be but those errors remind me of some of the tweaks I >> had to make to me Gentoo system when using podman (instead of docker). >> In the end I think I just ended up adding the lines: >> >> alex:100000:65536 >> >> to /etc/subgid and /etc/subgid- >> >> Marc-André may have some better pointers as he added podman support to >> the builder scripts. Not sure if helpful, but what worked for me is remove podman and use docker... For sure I'm not testing podman, enough to deal with docker. > I did that and things seem a bit better, but still a lot of errors: > > > 63 ../sysdeps/x86_64/start.S: No such file or directory. > > Error: error creating build container: The following failures happened while trying to pull image specified by "debian:bullseye-slim" based on search registries in /etc/containers/registries.conf: > * "localhost/debian:bullseye-slim": Error initializing source docker://localhost/debian:bullseye-slim: error pinging docker registry localhost: Get https://localhost/v2/: dial tcp [::1]:443: connect: connection refused > * "docker.io/library/debian:bullseye-slim": Error committing the finished image: error adding layer with blob "sha256:ae63fcbbc3b289e425e4c8840ccde4314f4a060cbc0345e6871a28bdc72f6fe8": Error processing tar file(exit status 1): there might not be enough IDs available in the namespace (requested 0:42 for /etc/gshadow): lchown /etc/gshadow: invalid argument > Traceback (most recent call last): > File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 709, in <module> > sys.exit(main()) > File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 705, in main > return args.cmdobj.run(args, argv) > File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 501, in run > extra_files_cksum=cksum) > File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 354, in build_image > quiet=quiet) > File "/home/claudio/git/qemu-pristine/qemu/tests/docker/docker.py", line 244, in _do_check > return subprocess.check_call(self._command + cmd, **kwargs) > File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call > raise CalledProcessError(retcode, cmd) > > > [...] > Error: error pulling image "registry.gitlab.com/qemu-project/qemu/qemu/fedora-cris-cross": unable to pull registry.gitlab.com/qemu-project/ Maybe you need "use explicit docker.io registry" from Daniel: https://www.mail-archive.com/qemu-devel@nongnu.org/msg763484.html [...] ^ permalink raw reply [flat|nested] 9+ messages in thread
* check-tcg HOWTO? 2020-12-02 9:32 check-tcg errors (build-user, build-user-plugins) again Claudio Fontana 2020-12-02 11:16 ` Alex Bennée @ 2021-01-10 21:17 ` Claudio Fontana 2021-01-11 13:35 ` Alex Bennée 1 sibling, 1 reply; 9+ messages in thread From: Claudio Fontana @ 2021-01-10 21:17 UTC (permalink / raw) To: Alex Bennee; +Cc: Paolo Bonzini, Philippe Mathieu-Daudé, qemu-devel Hi Alex, happy new year, I am trying to get check-tcg to run reliably, as I am doing some substantial refactoring of tcg cpu operations, so I need to verify that TCG is fine. This is an overall getting started question, is there a how-to on how to use check-tcg and how to fix things when things don't go smoothly? I get different results on different machines for check-tcg, although the runs are containerized, on one machine the tests for aarch64 tcg are SKIPPED completely (so no errors), on the other machine I get: qemu-system-aarch64: terminating on signal 15 from pid 18583 (timeout) qemu-system-aarch64: terminating on signal 15 from pid 18584 (timeout) qemu-system-aarch64: terminating on signal 15 from pid 18585 (timeout) make[2]: *** [../Makefile.target:162: run-hello] Error 124 make[2]: *** Waiting for unfinished jobs.... make[2]: *** [../Makefile.target:162: run-pauth-3] Error 124 make[2]: *** [../Makefile.target:162: run-memory] Error 124 Both are configured with configure --enable-tcg Anything more than V=1 to get more output? How do I debug and get logs and cores out of containers? in tests/tcg/ there is: a README (with no hint unfortunately) , Makefile.qemu Makefile.prereqs Makefile.target There are a bunch of variables in these files, which seem to be possible to configure, am I expected to set some of those? I think that it would be beneficial to have either more documentation or more immediately actionable information out of make check failures; Any help you could give me to make some progess? Thanks, Claudio ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: check-tcg HOWTO? 2021-01-10 21:17 ` check-tcg HOWTO? Claudio Fontana @ 2021-01-11 13:35 ` Alex Bennée 2021-01-11 14:47 ` Claudio Fontana 0 siblings, 1 reply; 9+ messages in thread From: Alex Bennée @ 2021-01-11 13:35 UTC (permalink / raw) To: Claudio Fontana; +Cc: Paolo Bonzini, Philippe Mathieu-Daudé, qemu-devel Claudio Fontana <cfontana@suse.de> writes: > Hi Alex, > > happy new year, > > I am trying to get check-tcg to run reliably, > as I am doing some substantial refactoring of tcg cpu operations, so I need to verify that TCG is fine. > > This is an overall getting started question, is there a how-to on how > to use check-tcg and how to fix things when things don't go smoothly? Not really but I could certainly add something. Generally I just run the tests manually in gdb when I'm trying to debug stuff. > I get different results on different machines for check-tcg, although the runs are containerized, > on one machine the tests for aarch64 tcg are SKIPPED completely (so no > errors), The compiles *may* be containerized - the runs are always in your host environment. It's one of the reasons the binaries are built as static images so you don't need to mess about with dynamic linking and libraries. The only reason some tests get skipped is if you have a locally installed cross compiler which doesn't support some architecture features (e.g. CROSS_CC_HAS_SVE). > on the other machine I get: > > qemu-system-aarch64: terminating on signal 15 from pid 18583 (timeout) > qemu-system-aarch64: terminating on signal 15 from pid 18584 (timeout) > qemu-system-aarch64: terminating on signal 15 from pid 18585 (timeout) > make[2]: *** [../Makefile.target:162: run-hello] Error 124 > make[2]: *** Waiting for unfinished jobs.... > make[2]: *** [../Makefile.target:162: run-pauth-3] Error 124 > make[2]: *** [../Makefile.target:162: run-memory] Error 124 Given it's timing out on hello I guess it's the shutdown deadlocking. Running with V=1 will give you the command line but the semihosting config is setup for redirect. So I usually build my own command line. e.g.: ./qemu-system-aarch64 -monitor none -display none \ -chardev stdio,id=output \ -M virt -cpu max -display none \ -semihosting-config enable=on,target=native,chardev=output \ -kernel tests/tcg/aarch64-softmmu/hello There is nothing particularly special apart from making sure semihosting is wired up for the output. Apart from some special cases like test-mmap-XXXX most tests don't take any arguments. > > Both are configured with > > configure --enable-tcg > > Anything more than V=1 to get more output? The output is normally dumped in $TESTNAME.out in the appropriate $BUILDDIR/tests/tcg/$TARGET/ directory. > How do I debug and get logs and cores out of containers? As I mentioned above the tests are not run in containers, just the compiles (if local compilers are missing). > > in tests/tcg/ there is: > > a README (with no hint unfortunately) , Woefully out of date I'm afraid. What docs we have are in docs/devel/testing.rst > Makefile.qemu Links into the main tests/Makefile.include - invoked for each target > Makefile.prereqs This ensures docker images are built (if required) for each set of tests. > Makefile.target This is the main (rather simple) makefile which provides the build and run targets. You can run directly if you are in the right build dir, eg: 13:58:10 [alex@zen:~/l/q/b/a/t/t/aarch64-softmmu] |✔ + pwd /home/alex/lsrc/qemu.git/builds/arm.all/tests/tcg/aarch64-softmmu 13:58:57 [alex@zen:~/l/q/b/a/t/t/aarch64-softmmu] |✔ + make -f ~/lsrc/qemu.git/tests/tcg/Makefile.target TARGET="aarch64-softmmu" SRC_PATH="/home/alex/lsrc/qemu.git" run But TBH this is functionally equivalent to calling: make run-tcg-tests-aarch64-softmmu in your main build directory. > There are a bunch of variables in these files, which seem to be > possible to configure, am I expected to set some of those? Not really. Most of the magic variables from: tests/tcg/config-$TARGET.mak which is built by tests/tcg/configure.sh during the configure step. > I think that it would be beneficial to have either more documentation > or more immediately actionable information out of make check failures; V=1 should show the command lines run and then you should be able to run the tests directly yourself. > Any help you could give me to make some progess? Hope that helps. > > Thanks, > > Claudio -- Alex Bennée ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: check-tcg HOWTO? 2021-01-11 13:35 ` Alex Bennée @ 2021-01-11 14:47 ` Claudio Fontana 0 siblings, 0 replies; 9+ messages in thread From: Claudio Fontana @ 2021-01-11 14:47 UTC (permalink / raw) To: Alex Bennée; +Cc: Paolo Bonzini, Philippe Mathieu-Daudé, qemu-devel Ciao Alex, thanks for your answer, On 1/11/21 2:35 PM, Alex Bennée wrote: > > Claudio Fontana <cfontana@suse.de> writes: > >> Hi Alex, >> >> happy new year, >> >> I am trying to get check-tcg to run reliably, >> as I am doing some substantial refactoring of tcg cpu operations, so I need to verify that TCG is fine. >> >> This is an overall getting started question, is there a how-to on how >> to use check-tcg and how to fix things when things don't go smoothly? > > Not really but I could certainly add something. Generally I just run the > tests manually in gdb when I'm trying to debug stuff. Right, I plan to do the same, if I get to the command line to run. I think it does make sense to add something similar to what was explained here in documentation, with a pointer maybe from README? > >> I get different results on different machines for check-tcg, although the runs are containerized, >> on one machine the tests for aarch64 tcg are SKIPPED completely (so no >> errors), > > The compiles *may* be containerized - the runs are always in your host > environment. It's one of the reasons the binaries are built as static > images so you don't need to mess about with dynamic linking and > libraries. > Ah good to know, thanks. So everything is actually run in the host environment in the end. > The only reason some tests get skipped is if you have a locally > installed cross compiler which doesn't support some architecture > features (e.g. CROSS_CC_HAS_SVE). hmm I will have to check then how to make sure that the test does not see these cross compilers...? > >> on the other machine I get: >> >> qemu-system-aarch64: terminating on signal 15 from pid 18583 (timeout) >> qemu-system-aarch64: terminating on signal 15 from pid 18584 (timeout) >> qemu-system-aarch64: terminating on signal 15 from pid 18585 (timeout) >> make[2]: *** [../Makefile.target:162: run-hello] Error 124 >> make[2]: *** Waiting for unfinished jobs.... >> make[2]: *** [../Makefile.target:162: run-pauth-3] Error 124 >> make[2]: *** [../Makefile.target:162: run-memory] Error 124 > > Given it's timing out on hello I guess it's the shutdown deadlocking. > Running with V=1 will give you the command line but the semihosting > config is setup for redirect. So I usually build my own command line. e.g.: > > ./qemu-system-aarch64 -monitor none -display none \ > -chardev stdio,id=output \ > -M virt -cpu max -display none \ > -semihosting-config enable=on,target=native,chardev=output \ > -kernel tests/tcg/aarch64-softmmu/hello > Would it be possible for check-tcg (and possibly even make check in general where applicable) to automatically spew the command line to reproduce the error, similar to what you have shown here? I think this is would be of great value for anyone to be able to act on the errors reported. > There is nothing particularly special apart from making sure semihosting > is wired up for the output. Apart from some special cases like > test-mmap-XXXX most tests don't take any arguments. > >> >> Both are configured with >> >> configure --enable-tcg >> >> Anything more than V=1 to get more output? > > The output is normally dumped in $TESTNAME.out in the appropriate > $BUILDDIR/tests/tcg/$TARGET/ directory. > >> How do I debug and get logs and cores out of containers? > > As I mentioned above the tests are not run in containers, just the > compiles (if local compilers are missing). Thanks for clearing this up! > >> >> in tests/tcg/ there is: >> >> a README (with no hint unfortunately) , > > Woefully out of date I'm afraid. What docs we have are in docs/devel/testing.rst maybe a + Please see docs/devel/testing.rst for hints on how to run make-tcg and reproduce its results from the cmdline. > >> Makefile.qemu > > Links into the main tests/Makefile.include - invoked for each target > >> Makefile.prereqs > > This ensures docker images are built (if required) for each set of tests. > >> Makefile.target > > This is the main (rather simple) makefile which provides the build and > run targets. You can run directly if you are in the right build dir, eg: > > 13:58:10 [alex@zen:~/l/q/b/a/t/t/aarch64-softmmu] |✔ + pwd > /home/alex/lsrc/qemu.git/builds/arm.all/tests/tcg/aarch64-softmmu > 13:58:57 [alex@zen:~/l/q/b/a/t/t/aarch64-softmmu] |✔ + > make -f ~/lsrc/qemu.git/tests/tcg/Makefile.target TARGET="aarch64-softmmu" SRC_PATH="/home/alex/lsrc/qemu.git" run > > But TBH this is functionally equivalent to calling: > > make run-tcg-tests-aarch64-softmmu > > in your main build directory. Thanks, that's helpful. > >> There are a bunch of variables in these files, which seem to be >> possible to configure, am I expected to set some of those? > > Not really. Most of the magic variables from: > > tests/tcg/config-$TARGET.mak > > which is built by tests/tcg/configure.sh during the configure step. > >> I think that it would be beneficial to have either more documentation >> or more immediately actionable information out of make check failures; > > V=1 should show the command lines run and then you should be able to run > the tests directly yourself. Hmm V=1 did not show the command line to me, but maybe I just missed it somehow? Or it contained some sockets that cannot be manually connected? > >> Any help you could give me to make some progess? > > Hope that helps. It does, I wonder if this could be fed into docs/ with a pointer to it from README, and also if this new feature for the tests could be developed, ie, producing a command line useful for reproducing the error, something that can be run directly without further editing, sanitizing etc... Thanks a lot, Claudio ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2021-01-11 14:48 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-12-02 9:32 check-tcg errors (build-user, build-user-plugins) again Claudio Fontana 2020-12-02 11:16 ` Alex Bennée 2020-12-02 11:25 ` Claudio Fontana 2020-12-02 12:52 ` Alex Bennée 2020-12-02 14:24 ` Claudio Fontana 2020-12-02 14:25 ` Philippe Mathieu-Daudé 2021-01-10 21:17 ` check-tcg HOWTO? Claudio Fontana 2021-01-11 13:35 ` Alex Bennée 2021-01-11 14:47 ` Claudio Fontana
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.