* [PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu) @ 2020-07-24 6:44 Alex Bennée 2020-07-24 6:44 ` [PATCH v3 01/16] shippable: add one more qemu to registry url Alex Bennée ` (15 more replies) 0 siblings, 16 replies; 24+ messages in thread From: Alex Bennée @ 2020-07-24 6:44 UTC (permalink / raw) To: qemu-devel Cc: fam, berrange, Alex Bennée, richard.henderson, f4bug, cota, aurelien Hi, This is the current state of my candidate fixes for 5.1. Apart from the revisions following reviews for v2: https://patchew.org/QEMU/20200722062902.24509-1-alex.bennee@linaro.org/ this includes a new version of Laurent's nanosleep fix and some clean-ups to the python console drain code which should sort out some of the 100% cpu spins Peter was seeing. Other minor tweaks noted bellow -- in the patches themselves. The following are still un-reviewed: - tests/docker: fix update command due to python3 str/bytes distinction - accel/tcg: better handle memory constrained systems - util/oslib-win32: add qemu_get_host_physmem implementation Alex Bennée (8): shippable: add one more qemu to registry url util: add qemu_get_host_physmem utility function util/oslib-win32: add qemu_get_host_physmem implementation accel/tcg: better handle memory constrained systems linux-user: don't use MAP_FIXED in pgd_find_hole_fallback tests/docker: fix update command due to python3 str/bytes distinction tests/docker: fix binfmt_misc image building tests/docker: add support for DEB_KEYRING KONRAD Frederic (2): semihosting: defer connect_chardevs a little more to use serialx semihosting: don't send the trailing '\0' Laszlo Ersek (1): target/i386: floatx80: avoid compound literals in static initializers Laurent Vivier (2): linux-user: fix clock_nanosleep() linux-user,ppc: fix clock_nanosleep() for linux-user-ppc Robert Foley (3): python/qemu: Cleanup changes to ConsoleSocket python/qemu: Change ConsoleSocket to optionally drain socket. tests/vm: add shutdown timeout in basevm.py include/fpu/softfloat.h | 1 + include/qemu/osdep.h | 15 + accel/tcg/translate-all.c | 7 +- hw/semihosting/console.c | 4 +- linux-user/elfload.c | 10 +- linux-user/syscall.c | 15 +- softmmu/vl.c | 5 +- target/i386/fpu_helper.c | 426 +++++++++--------- util/oslib-posix.c | 15 + util/oslib-win32.c | 11 + .shippable.yml | 2 +- python/qemu/console_socket.py | 137 +++--- python/qemu/machine.py | 14 +- python/qemu/pylintrc | 2 +- tests/docker/Makefile.include | 2 +- tests/docker/docker.py | 13 +- tests/docker/dockerfiles/debian-bootstrap.pre | 7 + tests/vm/basevm.py | 15 +- 18 files changed, 390 insertions(+), 311 deletions(-) -- 2.20.1 ^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH v3 01/16] shippable: add one more qemu to registry url 2020-07-24 6:44 [PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu) Alex Bennée @ 2020-07-24 6:44 ` Alex Bennée 2020-07-24 6:44 ` [PATCH v3 02/16] semihosting: defer connect_chardevs a little more to use serialx Alex Bennée ` (14 subsequent siblings) 15 siblings, 0 replies; 24+ messages in thread From: Alex Bennée @ 2020-07-24 6:44 UTC (permalink / raw) To: qemu-devel Cc: fam, berrange, Alex Bennée, richard.henderson, f4bug, Philippe Mathieu-Daudé, cota, aurelien The registry url is <project>/<repo>/qemu/<image> Perhaps we should rationalise that some day but for now. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-Id: <20200717105139.25293-2-alex.bennee@linaro.org> Message-Id: <20200722062902.24509-2-alex.bennee@linaro.org> --- .shippable.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.shippable.yml b/.shippable.yml index f6b742432e5..89d8be4291b 100644 --- a/.shippable.yml +++ b/.shippable.yml @@ -27,7 +27,7 @@ env: TARGET_LIST=ppc64-softmmu,ppc64-linux-user,ppc64abi32-linux-user build: pre_ci_boot: - image_name: registry.gitlab.com/qemu-project/qemu/${IMAGE} + image_name: registry.gitlab.com/qemu-project/qemu/qemu/${IMAGE} image_tag: latest pull: true options: "-e HOME=/root" -- 2.20.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v3 02/16] semihosting: defer connect_chardevs a little more to use serialx 2020-07-24 6:44 [PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu) Alex Bennée 2020-07-24 6:44 ` [PATCH v3 01/16] shippable: add one more qemu to registry url Alex Bennée @ 2020-07-24 6:44 ` Alex Bennée 2020-07-24 6:44 ` [PATCH v3 03/16] semihosting: don't send the trailing '\0' Alex Bennée ` (13 subsequent siblings) 15 siblings, 0 replies; 24+ messages in thread From: Alex Bennée @ 2020-07-24 6:44 UTC (permalink / raw) To: qemu-devel Cc: fam, berrange, Alex Bennée, richard.henderson, f4bug, KONRAD Frederic, cota, Paolo Bonzini, aurelien From: KONRAD Frederic <frederic.konrad@adacore.com> With that we can just use -semihosting-config chardev=serial0. [AJB: tweak commit message] Signed-off-by: KONRAD Frederic <frederic.konrad@adacore.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-Id: <1592215252-26742-1-git-send-email-frederic.konrad@adacore.com> Message-Id: <20200717105139.25293-3-alex.bennee@linaro.org> Message-Id: <20200722062902.24509-3-alex.bennee@linaro.org> --- softmmu/vl.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/softmmu/vl.c b/softmmu/vl.c index 34162415579..295459adc95 100644 --- a/softmmu/vl.c +++ b/softmmu/vl.c @@ -4127,8 +4127,6 @@ void qemu_init(int argc, char **argv, char **envp) qemu_opts_foreach(qemu_find_opts("chardev"), chardev_init_func, NULL, &error_fatal); - /* now chardevs have been created we may have semihosting to connect */ - qemu_semihosting_connect_chardevs(); #ifdef CONFIG_VIRTFS qemu_opts_foreach(qemu_find_opts("fsdev"), @@ -4277,6 +4275,9 @@ void qemu_init(int argc, char **argv, char **envp) if (foreach_device_config(DEV_DEBUGCON, debugcon_parse) < 0) exit(1); + /* now chardevs have been created we may have semihosting to connect */ + qemu_semihosting_connect_chardevs(); + /* If no default VGA is requested, the default is "none". */ if (default_vga) { vga_model = get_default_vga_model(machine_class); -- 2.20.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v3 03/16] semihosting: don't send the trailing '\0' 2020-07-24 6:44 [PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu) Alex Bennée 2020-07-24 6:44 ` [PATCH v3 01/16] shippable: add one more qemu to registry url Alex Bennée 2020-07-24 6:44 ` [PATCH v3 02/16] semihosting: defer connect_chardevs a little more to use serialx Alex Bennée @ 2020-07-24 6:44 ` Alex Bennée 2020-07-24 6:44 ` [PATCH v3 04/16] util: add qemu_get_host_physmem utility function Alex Bennée ` (12 subsequent siblings) 15 siblings, 0 replies; 24+ messages in thread From: Alex Bennée @ 2020-07-24 6:44 UTC (permalink / raw) To: qemu-devel Cc: fam, berrange, Alex Bennée, richard.henderson, f4bug, Philippe Mathieu-Daudé, KONRAD Frederic, cota, aurelien From: KONRAD Frederic <frederic.konrad@adacore.com> Don't send the trailing 0 from the string. Signed-off-by: KONRAD Frederic <frederic.konrad@adacore.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-Id: <1592215252-26742-2-git-send-email-frederic.konrad@adacore.com> Message-Id: <20200717105139.25293-4-alex.bennee@linaro.org> Message-Id: <20200722062902.24509-4-alex.bennee@linaro.org> --- hw/semihosting/console.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/hw/semihosting/console.c b/hw/semihosting/console.c index 22e7827824a..9b4fee92602 100644 --- a/hw/semihosting/console.c +++ b/hw/semihosting/console.c @@ -52,7 +52,9 @@ static GString *copy_user_string(CPUArchState *env, target_ulong addr) do { if (cpu_memory_rw_debug(cpu, addr++, &c, 1, 0) == 0) { - s = g_string_append_c(s, c); + if (c) { + s = g_string_append_c(s, c); + } } else { qemu_log_mask(LOG_GUEST_ERROR, "%s: passed inaccessible address " TARGET_FMT_lx, -- 2.20.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v3 04/16] util: add qemu_get_host_physmem utility function 2020-07-24 6:44 [PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu) Alex Bennée ` (2 preceding siblings ...) 2020-07-24 6:44 ` [PATCH v3 03/16] semihosting: don't send the trailing '\0' Alex Bennée @ 2020-07-24 6:44 ` Alex Bennée 2020-07-24 6:44 ` [PATCH v3 05/16] util/oslib-win32: add qemu_get_host_physmem implementation Alex Bennée ` (11 subsequent siblings) 15 siblings, 0 replies; 24+ messages in thread From: Alex Bennée @ 2020-07-24 6:44 UTC (permalink / raw) To: qemu-devel Cc: fam, berrange, Alex Bennée, richard.henderson, f4bug, Christian Ehrhardt, cota, Stefan Weil, Paolo Bonzini, aurelien This will be used in a future patch. For POSIX systems _SC_PHYS_PAGES isn't standardised but at least appears in the man pages for Open/FreeBSD. The result is advisory so any users of it shouldn't just fail if we can't work it out. The win32 stub currently returns 0 until someone with a Windows system can develop and test a patch. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Cc: BALATON Zoltan <balaton@eik.bme.hu> Cc: Christian Ehrhardt <christian.ehrhardt@canonical.com> Message-Id: <20200722062902.24509-5-alex.bennee@linaro.org> --- include/qemu/osdep.h | 12 ++++++++++++ util/oslib-posix.c | 15 +++++++++++++++ util/oslib-win32.c | 6 ++++++ 3 files changed, 33 insertions(+) diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h index 45c217aa280..0b1298b3c91 100644 --- a/include/qemu/osdep.h +++ b/include/qemu/osdep.h @@ -668,4 +668,16 @@ static inline void qemu_reset_optind(void) */ char *qemu_get_host_name(Error **errp); +/** + * qemu_get_host_physmem: + * + * Operating system agnostic way of querying host memory. + * + * Returns amount of physical memory on the system. This is purely + * advisery and may return 0 if we can't work it out. At the other + * end we saturate to SIZE_MAX if you are lucky enough to have that + * much memory. + */ +size_t qemu_get_host_physmem(void); + #endif diff --git a/util/oslib-posix.c b/util/oslib-posix.c index d9236746248..ad8001a4ad8 100644 --- a/util/oslib-posix.c +++ b/util/oslib-posix.c @@ -841,3 +841,18 @@ char *qemu_get_host_name(Error **errp) return g_steal_pointer(&hostname); } + +size_t qemu_get_host_physmem(void) +{ +#ifdef _SC_PHYS_PAGES + long pages = sysconf(_SC_PHYS_PAGES); + if (pages > 0) { + if (pages > SIZE_MAX / qemu_real_host_page_size) { + return SIZE_MAX; + } else { + return pages * qemu_real_host_page_size; + } + } +#endif + return 0; +} diff --git a/util/oslib-win32.c b/util/oslib-win32.c index 7eedbe5859a..31030463cc9 100644 --- a/util/oslib-win32.c +++ b/util/oslib-win32.c @@ -828,3 +828,9 @@ char *qemu_get_host_name(Error **errp) return g_utf16_to_utf8(tmp, size, NULL, NULL, NULL); } + +size_t qemu_get_host_physmem(void) +{ + /* currently unimplemented */ + return 0; +} -- 2.20.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v3 05/16] util/oslib-win32: add qemu_get_host_physmem implementation 2020-07-24 6:44 [PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu) Alex Bennée ` (3 preceding siblings ...) 2020-07-24 6:44 ` [PATCH v3 04/16] util: add qemu_get_host_physmem utility function Alex Bennée @ 2020-07-24 6:44 ` Alex Bennée 2020-07-24 6:51 ` Stefan Weil 2020-07-24 6:44 ` [PATCH v3 06/16] accel/tcg: better handle memory constrained systems Alex Bennée ` (10 subsequent siblings) 15 siblings, 1 reply; 24+ messages in thread From: Alex Bennée @ 2020-07-24 6:44 UTC (permalink / raw) To: qemu-devel Cc: fam, berrange, Alex Bennée, richard.henderson, f4bug, cota, Stefan Weil, aurelien Compile tested only. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Cc: Stefan Weil <sw@weilnetz.de> --- v2 - used more widely available GlobalMemoryStatusEx - dropped Phillipe's r-b due to the change --- util/oslib-win32.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/util/oslib-win32.c b/util/oslib-win32.c index 31030463cc9..a74ee5a3aa7 100644 --- a/util/oslib-win32.c +++ b/util/oslib-win32.c @@ -831,6 +831,11 @@ char *qemu_get_host_name(Error **errp) size_t qemu_get_host_physmem(void) { - /* currently unimplemented */ + MEMORYSTATUSEX statex; + statex.dwLength = sizeof(statex); + + if (!GlobalMemoryStatusEx(&statex)) { + return statex.ullTotalPhys; + } return 0; } -- 2.20.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH v3 05/16] util/oslib-win32: add qemu_get_host_physmem implementation 2020-07-24 6:44 ` [PATCH v3 05/16] util/oslib-win32: add qemu_get_host_physmem implementation Alex Bennée @ 2020-07-24 6:51 ` Stefan Weil 0 siblings, 0 replies; 24+ messages in thread From: Stefan Weil @ 2020-07-24 6:51 UTC (permalink / raw) To: Alex Bennée, qemu-devel Cc: fam, berrange, richard.henderson, f4bug, cota, aurelien Am 24.07.20 um 08:44 schrieb Alex Bennée: > Compile tested only. > > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > Cc: Stefan Weil <sw@weilnetz.de> > > --- > v2 > - used more widely available GlobalMemoryStatusEx > - dropped Phillipe's r-b due to the change > --- > util/oslib-win32.c | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/util/oslib-win32.c b/util/oslib-win32.c > index 31030463cc9..a74ee5a3aa7 100644 > --- a/util/oslib-win32.c > +++ b/util/oslib-win32.c > @@ -831,6 +831,11 @@ char *qemu_get_host_name(Error **errp) > > size_t qemu_get_host_physmem(void) > { > - /* currently unimplemented */ > + MEMORYSTATUSEX statex; > + statex.dwLength = sizeof(statex); > + > + if (!GlobalMemoryStatusEx(&statex)) { > + return statex.ullTotalPhys; > + } > return 0; > } Thanks. The logic must be reversed, as GlobalMemoryStatusEx returns a nonzero value when it succeeded. When that is fixed, you may add Reviewed-by: Stefan Weil <sw@weilnetz.de> ^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH v3 06/16] accel/tcg: better handle memory constrained systems 2020-07-24 6:44 [PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu) Alex Bennée ` (4 preceding siblings ...) 2020-07-24 6:44 ` [PATCH v3 05/16] util/oslib-win32: add qemu_get_host_physmem implementation Alex Bennée @ 2020-07-24 6:44 ` Alex Bennée 2020-07-24 13:41 ` Robert Foley 2020-07-24 18:22 ` Richard Henderson 2020-07-24 6:45 ` [PATCH v3 07/16] target/i386: floatx80: avoid compound literals in static initializers Alex Bennée ` (9 subsequent siblings) 15 siblings, 2 replies; 24+ messages in thread From: Alex Bennée @ 2020-07-24 6:44 UTC (permalink / raw) To: qemu-devel Cc: fam, berrange, Alex Bennée, richard.henderson, f4bug, Christian Ehrhardt, cota, Paolo Bonzini, aurelien, Richard Henderson It turns out there are some 64 bit systems that have relatively low amounts of physical memory available to them (typically CI system). Even with swapping available a 1GB translation buffer that fills up can put the machine under increased memory pressure. Detect these low memory situations and reduce tb_size appropriately. Fixes: 600e17b2615 ("accel/tcg: increase default code gen buffer size for 64 bit") Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Cc: BALATON Zoltan <balaton@eik.bme.hu> Cc: Christian Ehrhardt <christian.ehrhardt@canonical.com> --- v3 - use slightly tweak the logic to taper off more nicely --- accel/tcg/translate-all.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c index 2afa46bd2b1..2d83013633b 100644 --- a/accel/tcg/translate-all.c +++ b/accel/tcg/translate-all.c @@ -976,7 +976,12 @@ static inline size_t size_code_gen_buffer(size_t tb_size) { /* Size the buffer. */ if (tb_size == 0) { - tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE; + size_t phys_mem = qemu_get_host_physmem(); + if (phys_mem == 0) { + tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE; + } else { + tb_size = MIN(DEFAULT_CODE_GEN_BUFFER_SIZE, phys_mem / 8); + } } if (tb_size < MIN_CODE_GEN_BUFFER_SIZE) { tb_size = MIN_CODE_GEN_BUFFER_SIZE; -- 2.20.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH v3 06/16] accel/tcg: better handle memory constrained systems 2020-07-24 6:44 ` [PATCH v3 06/16] accel/tcg: better handle memory constrained systems Alex Bennée @ 2020-07-24 13:41 ` Robert Foley 2020-07-24 18:22 ` Richard Henderson 1 sibling, 0 replies; 24+ messages in thread From: Robert Foley @ 2020-07-24 13:41 UTC (permalink / raw) To: Alex Bennée Cc: Fam Zheng, Daniel P. Berrangé, Richard Henderson, QEMU Developers, Christian Ehrhardt, f4bug, Emilio G. Cota, Paolo Bonzini, Aurelien Jarno, Richard Henderson Reviewed-by: Robert Foley <robert.foley@linaro.org> On Fri, 24 Jul 2020 at 02:49, Alex Bennée <alex.bennee@linaro.org> wrote: > > It turns out there are some 64 bit systems that have relatively low > amounts of physical memory available to them (typically CI system). > Even with swapping available a 1GB translation buffer that fills up > can put the machine under increased memory pressure. Detect these low > memory situations and reduce tb_size appropriately. > > Fixes: 600e17b2615 ("accel/tcg: increase default code gen buffer size for 64 bit") > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > Cc: BALATON Zoltan <balaton@eik.bme.hu> > Cc: Christian Ehrhardt <christian.ehrhardt@canonical.com> > > --- > v3 > - use slightly tweak the logic to taper off more nicely > --- > accel/tcg/translate-all.c | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c > index 2afa46bd2b1..2d83013633b 100644 > --- a/accel/tcg/translate-all.c > +++ b/accel/tcg/translate-all.c > @@ -976,7 +976,12 @@ static inline size_t size_code_gen_buffer(size_t tb_size) > { > /* Size the buffer. */ > if (tb_size == 0) { > - tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE; > + size_t phys_mem = qemu_get_host_physmem(); > + if (phys_mem == 0) { > + tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE; > + } else { > + tb_size = MIN(DEFAULT_CODE_GEN_BUFFER_SIZE, phys_mem / 8); > + } > } > if (tb_size < MIN_CODE_GEN_BUFFER_SIZE) { > tb_size = MIN_CODE_GEN_BUFFER_SIZE; > -- > 2.20.1 > > ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v3 06/16] accel/tcg: better handle memory constrained systems 2020-07-24 6:44 ` [PATCH v3 06/16] accel/tcg: better handle memory constrained systems Alex Bennée 2020-07-24 13:41 ` Robert Foley @ 2020-07-24 18:22 ` Richard Henderson 1 sibling, 0 replies; 24+ messages in thread From: Richard Henderson @ 2020-07-24 18:22 UTC (permalink / raw) To: Alex Bennée, qemu-devel Cc: fam, berrange, f4bug, Christian Ehrhardt, cota, Paolo Bonzini, aurelien, Richard Henderson On 7/23/20 11:44 PM, Alex Bennée wrote: > It turns out there are some 64 bit systems that have relatively low > amounts of physical memory available to them (typically CI system). > Even with swapping available a 1GB translation buffer that fills up > can put the machine under increased memory pressure. Detect these low > memory situations and reduce tb_size appropriately. > > Fixes: 600e17b2615 ("accel/tcg: increase default code gen buffer size for 64 bit") > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > Cc: BALATON Zoltan <balaton@eik.bme.hu> > Cc: Christian Ehrhardt <christian.ehrhardt@canonical.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> r~ ^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH v3 07/16] target/i386: floatx80: avoid compound literals in static initializers 2020-07-24 6:44 [PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu) Alex Bennée ` (5 preceding siblings ...) 2020-07-24 6:44 ` [PATCH v3 06/16] accel/tcg: better handle memory constrained systems Alex Bennée @ 2020-07-24 6:45 ` Alex Bennée 2020-07-24 18:23 ` Richard Henderson 2020-07-24 6:45 ` [PATCH v3 08/16] linux-user: don't use MAP_FIXED in pgd_find_hole_fallback Alex Bennée ` (8 subsequent siblings) 15 siblings, 1 reply; 24+ messages in thread From: Alex Bennée @ 2020-07-24 6:45 UTC (permalink / raw) To: qemu-devel Cc: fam, Peter Maydell, berrange, Eduardo Habkost, Laszlo Ersek, richard.henderson, f4bug, Philippe Mathieu-Daudé, Alex Bennée, cota, Joseph Myers, Paolo Bonzini, aurelien, Richard Henderson From: Laszlo Ersek <lersek@redhat.com> Quoting ISO C99 6.7.8p4, "All the expressions in an initializer for an object that has static storage duration shall be constant expressions or string literals". The compound literal produced by the make_floatx80() macro is not such a constant expression, per 6.6p7-9. (An implementation may accept it, according to 6.6p10, but is not required to.) Therefore using "floatx80_zero" and make_floatx80() for initializing "f2xm1_table" and "fpatan_table" is not portable. And gcc-4.8 in RHEL-7.6 actually chokes on them: > target/i386/fpu_helper.c:871:5: error: initializer element is not constant > { make_floatx80(0xbfff, 0x8000000000000000ULL), > ^ We've had the make_floatx80_init() macro for this purpose since commit 3bf7e40ab914 ("softfloat: fix for C99", 2012-03-17), so let's use that macro again. Fixes: eca30647fc07 Fixes: ff57bb7b6326 Signed-off-by: Laszlo Ersek <lersek@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Cc: Aurelien Jarno <aurelien@aurel32.net> Cc: Eduardo Habkost <ehabkost@redhat.com> Cc: Joseph Myers <joseph@codesourcery.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Peter Maydell <peter.maydell@linaro.org> Cc: Richard Henderson <rth@twiddle.net> Link: https://lists.gnu.org/archive/html/qemu-devel/2017-08/msg06566.html Link: https://lists.gnu.org/archive/html/qemu-devel/2020-07/msg04714.html Message-Id: <20200716144251.23004-1-lersek@redhat.com> Message-Id: <20200722062902.24509-8-alex.bennee@linaro.org> --- include/fpu/softfloat.h | 1 + target/i386/fpu_helper.c | 426 +++++++++++++++++++-------------------- 2 files changed, 214 insertions(+), 213 deletions(-) diff --git a/include/fpu/softfloat.h b/include/fpu/softfloat.h index f1a19df066b..659218b5c78 100644 --- a/include/fpu/softfloat.h +++ b/include/fpu/softfloat.h @@ -822,6 +822,7 @@ static inline bool floatx80_invalid_encoding(floatx80 a) } #define floatx80_zero make_floatx80(0x0000, 0x0000000000000000LL) +#define floatx80_zero_init make_floatx80_init(0x0000, 0x0000000000000000LL) #define floatx80_one make_floatx80(0x3fff, 0x8000000000000000LL) #define floatx80_ln2 make_floatx80(0x3ffe, 0xb17217f7d1cf79acLL) #define floatx80_pi make_floatx80(0x4000, 0xc90fdaa22168c235LL) diff --git a/target/i386/fpu_helper.c b/target/i386/fpu_helper.c index f5e6c4b88d4..4ea73874d83 100644 --- a/target/i386/fpu_helper.c +++ b/target/i386/fpu_helper.c @@ -868,201 +868,201 @@ struct f2xm1_data { }; static const struct f2xm1_data f2xm1_table[65] = { - { make_floatx80(0xbfff, 0x8000000000000000ULL), - make_floatx80(0x3ffe, 0x8000000000000000ULL), - make_floatx80(0xbffe, 0x8000000000000000ULL) }, - { make_floatx80(0xbffe, 0xf800000000002e7eULL), - make_floatx80(0x3ffe, 0x82cd8698ac2b9160ULL), - make_floatx80(0xbffd, 0xfa64f2cea7a8dd40ULL) }, - { make_floatx80(0xbffe, 0xefffffffffffe960ULL), - make_floatx80(0x3ffe, 0x85aac367cc488345ULL), - make_floatx80(0xbffd, 0xf4aa7930676ef976ULL) }, - { make_floatx80(0xbffe, 0xe800000000006f10ULL), - make_floatx80(0x3ffe, 0x88980e8092da5c14ULL), - make_floatx80(0xbffd, 0xeecfe2feda4b47d8ULL) }, - { make_floatx80(0xbffe, 0xe000000000008a45ULL), - make_floatx80(0x3ffe, 0x8b95c1e3ea8ba2a5ULL), - make_floatx80(0xbffd, 0xe8d47c382ae8bab6ULL) }, - { make_floatx80(0xbffe, 0xd7ffffffffff8a9eULL), - make_floatx80(0x3ffe, 0x8ea4398b45cd8116ULL), - make_floatx80(0xbffd, 0xe2b78ce97464fdd4ULL) }, - { make_floatx80(0xbffe, 0xd0000000000019a0ULL), - make_floatx80(0x3ffe, 0x91c3d373ab11b919ULL), - make_floatx80(0xbffd, 0xdc785918a9dc8dceULL) }, - { make_floatx80(0xbffe, 0xc7ffffffffff14dfULL), - make_floatx80(0x3ffe, 0x94f4efa8fef76836ULL), - make_floatx80(0xbffd, 0xd61620ae02112f94ULL) }, - { make_floatx80(0xbffe, 0xc000000000006530ULL), - make_floatx80(0x3ffe, 0x9837f0518db87fbbULL), - make_floatx80(0xbffd, 0xcf901f5ce48f008aULL) }, - { make_floatx80(0xbffe, 0xb7ffffffffff1723ULL), - make_floatx80(0x3ffe, 0x9b8d39b9d54eb74cULL), - make_floatx80(0xbffd, 0xc8e58c8c55629168ULL) }, - { make_floatx80(0xbffe, 0xb00000000000b5e1ULL), - make_floatx80(0x3ffe, 0x9ef5326091a0c366ULL), - make_floatx80(0xbffd, 0xc2159b3edcbe7934ULL) }, - { make_floatx80(0xbffe, 0xa800000000006f8aULL), - make_floatx80(0x3ffe, 0xa27043030c49370aULL), - make_floatx80(0xbffd, 0xbb1f79f9e76d91ecULL) }, - { make_floatx80(0xbffe, 0x9fffffffffff816aULL), - make_floatx80(0x3ffe, 0xa5fed6a9b15171cfULL), - make_floatx80(0xbffd, 0xb40252ac9d5d1c62ULL) }, - { make_floatx80(0xbffe, 0x97ffffffffffb621ULL), - make_floatx80(0x3ffe, 0xa9a15ab4ea7c30e6ULL), - make_floatx80(0xbffd, 0xacbd4a962b079e34ULL) }, - { make_floatx80(0xbffe, 0x8fffffffffff162bULL), - make_floatx80(0x3ffe, 0xad583eea42a1b886ULL), - make_floatx80(0xbffd, 0xa54f822b7abc8ef4ULL) }, - { make_floatx80(0xbffe, 0x87ffffffffff4d34ULL), - make_floatx80(0x3ffe, 0xb123f581d2ac7b51ULL), - make_floatx80(0xbffd, 0x9db814fc5aa7095eULL) }, - { make_floatx80(0xbffe, 0x800000000000227dULL), - make_floatx80(0x3ffe, 0xb504f333f9de539dULL), - make_floatx80(0xbffd, 0x95f619980c4358c6ULL) }, - { make_floatx80(0xbffd, 0xefffffffffff3978ULL), - make_floatx80(0x3ffe, 0xb8fbaf4762fbd0a1ULL), - make_floatx80(0xbffd, 0x8e08a1713a085ebeULL) }, - { make_floatx80(0xbffd, 0xe00000000000df81ULL), - make_floatx80(0x3ffe, 0xbd08a39f580bfd8cULL), - make_floatx80(0xbffd, 0x85eeb8c14fe804e8ULL) }, - { make_floatx80(0xbffd, 0xd00000000000bccfULL), - make_floatx80(0x3ffe, 0xc12c4cca667062f6ULL), - make_floatx80(0xbffc, 0xfb4eccd6663e7428ULL) }, - { make_floatx80(0xbffd, 0xc00000000000eff0ULL), - make_floatx80(0x3ffe, 0xc5672a1155069abeULL), - make_floatx80(0xbffc, 0xea6357baabe59508ULL) }, - { make_floatx80(0xbffd, 0xb000000000000fe6ULL), - make_floatx80(0x3ffe, 0xc9b9bd866e2f234bULL), - make_floatx80(0xbffc, 0xd91909e6474372d4ULL) }, - { make_floatx80(0xbffd, 0x9fffffffffff2172ULL), - make_floatx80(0x3ffe, 0xce248c151f84bf00ULL), - make_floatx80(0xbffc, 0xc76dcfab81ed0400ULL) }, - { make_floatx80(0xbffd, 0x8fffffffffffafffULL), - make_floatx80(0x3ffe, 0xd2a81d91f12afb2bULL), - make_floatx80(0xbffc, 0xb55f89b83b541354ULL) }, - { make_floatx80(0xbffc, 0xffffffffffff81a3ULL), - make_floatx80(0x3ffe, 0xd744fccad69d7d5eULL), - make_floatx80(0xbffc, 0xa2ec0cd4a58a0a88ULL) }, - { make_floatx80(0xbffc, 0xdfffffffffff1568ULL), - make_floatx80(0x3ffe, 0xdbfbb797daf25a44ULL), - make_floatx80(0xbffc, 0x901121a0943696f0ULL) }, - { make_floatx80(0xbffc, 0xbfffffffffff68daULL), - make_floatx80(0x3ffe, 0xe0ccdeec2a94f811ULL), - make_floatx80(0xbffb, 0xf999089eab583f78ULL) }, - { make_floatx80(0xbffc, 0x9fffffffffff4690ULL), - make_floatx80(0x3ffe, 0xe5b906e77c83657eULL), - make_floatx80(0xbffb, 0xd237c8c41be4d410ULL) }, - { make_floatx80(0xbffb, 0xffffffffffff8aeeULL), - make_floatx80(0x3ffe, 0xeac0c6e7dd24427cULL), - make_floatx80(0xbffb, 0xa9f9c8c116ddec20ULL) }, - { make_floatx80(0xbffb, 0xbfffffffffff2d18ULL), - make_floatx80(0x3ffe, 0xefe4b99bdcdb06ebULL), - make_floatx80(0xbffb, 0x80da33211927c8a8ULL) }, - { make_floatx80(0xbffa, 0xffffffffffff8ccbULL), - make_floatx80(0x3ffe, 0xf5257d152486d0f4ULL), - make_floatx80(0xbffa, 0xada82eadb792f0c0ULL) }, - { make_floatx80(0xbff9, 0xffffffffffff11feULL), - make_floatx80(0x3ffe, 0xfa83b2db722a0846ULL), - make_floatx80(0xbff9, 0xaf89a491babef740ULL) }, - { floatx80_zero, - make_floatx80(0x3fff, 0x8000000000000000ULL), - floatx80_zero }, - { make_floatx80(0x3ff9, 0xffffffffffff2680ULL), - make_floatx80(0x3fff, 0x82cd8698ac2b9f6fULL), - make_floatx80(0x3ff9, 0xb361a62b0ae7dbc0ULL) }, - { make_floatx80(0x3ffb, 0x800000000000b500ULL), - make_floatx80(0x3fff, 0x85aac367cc488345ULL), - make_floatx80(0x3ffa, 0xb5586cf9891068a0ULL) }, - { make_floatx80(0x3ffb, 0xbfffffffffff4b67ULL), - make_floatx80(0x3fff, 0x88980e8092da7cceULL), - make_floatx80(0x3ffb, 0x8980e8092da7cce0ULL) }, - { make_floatx80(0x3ffb, 0xffffffffffffff57ULL), - make_floatx80(0x3fff, 0x8b95c1e3ea8bd6dfULL), - make_floatx80(0x3ffb, 0xb95c1e3ea8bd6df0ULL) }, - { make_floatx80(0x3ffc, 0x9fffffffffff811fULL), - make_floatx80(0x3fff, 0x8ea4398b45cd4780ULL), - make_floatx80(0x3ffb, 0xea4398b45cd47800ULL) }, - { make_floatx80(0x3ffc, 0xbfffffffffff9980ULL), - make_floatx80(0x3fff, 0x91c3d373ab11b919ULL), - make_floatx80(0x3ffc, 0x8e1e9b9d588dc8c8ULL) }, - { make_floatx80(0x3ffc, 0xdffffffffffff631ULL), - make_floatx80(0x3fff, 0x94f4efa8fef70864ULL), - make_floatx80(0x3ffc, 0xa7a77d47f7b84320ULL) }, - { make_floatx80(0x3ffc, 0xffffffffffff2499ULL), - make_floatx80(0x3fff, 0x9837f0518db892d4ULL), - make_floatx80(0x3ffc, 0xc1bf828c6dc496a0ULL) }, - { make_floatx80(0x3ffd, 0x8fffffffffff80fbULL), - make_floatx80(0x3fff, 0x9b8d39b9d54e3a79ULL), - make_floatx80(0x3ffc, 0xdc69cdceaa71d3c8ULL) }, - { make_floatx80(0x3ffd, 0x9fffffffffffbc23ULL), - make_floatx80(0x3fff, 0x9ef5326091a10313ULL), - make_floatx80(0x3ffc, 0xf7a993048d081898ULL) }, - { make_floatx80(0x3ffd, 0xafffffffffff20ecULL), - make_floatx80(0x3fff, 0xa27043030c49370aULL), - make_floatx80(0x3ffd, 0x89c10c0c3124dc28ULL) }, - { make_floatx80(0x3ffd, 0xc00000000000fd2cULL), - make_floatx80(0x3fff, 0xa5fed6a9b15171cfULL), - make_floatx80(0x3ffd, 0x97fb5aa6c545c73cULL) }, - { make_floatx80(0x3ffd, 0xd0000000000093beULL), - make_floatx80(0x3fff, 0xa9a15ab4ea7c30e6ULL), - make_floatx80(0x3ffd, 0xa6856ad3a9f0c398ULL) }, - { make_floatx80(0x3ffd, 0xe00000000000c2aeULL), - make_floatx80(0x3fff, 0xad583eea42a17876ULL), - make_floatx80(0x3ffd, 0xb560fba90a85e1d8ULL) }, - { make_floatx80(0x3ffd, 0xefffffffffff1e3fULL), - make_floatx80(0x3fff, 0xb123f581d2abef6cULL), - make_floatx80(0x3ffd, 0xc48fd6074aafbdb0ULL) }, - { make_floatx80(0x3ffd, 0xffffffffffff1c23ULL), - make_floatx80(0x3fff, 0xb504f333f9de2cadULL), - make_floatx80(0x3ffd, 0xd413cccfe778b2b4ULL) }, - { make_floatx80(0x3ffe, 0x8800000000006344ULL), - make_floatx80(0x3fff, 0xb8fbaf4762fbd0a1ULL), - make_floatx80(0x3ffd, 0xe3eebd1d8bef4284ULL) }, - { make_floatx80(0x3ffe, 0x9000000000005d67ULL), - make_floatx80(0x3fff, 0xbd08a39f580c668dULL), - make_floatx80(0x3ffd, 0xf4228e7d60319a34ULL) }, - { make_floatx80(0x3ffe, 0x9800000000009127ULL), - make_floatx80(0x3fff, 0xc12c4cca6670e042ULL), - make_floatx80(0x3ffe, 0x82589994cce1c084ULL) }, - { make_floatx80(0x3ffe, 0x9fffffffffff06f9ULL), - make_floatx80(0x3fff, 0xc5672a11550655c3ULL), - make_floatx80(0x3ffe, 0x8ace5422aa0cab86ULL) }, - { make_floatx80(0x3ffe, 0xa7fffffffffff80dULL), - make_floatx80(0x3fff, 0xc9b9bd866e2f234bULL), - make_floatx80(0x3ffe, 0x93737b0cdc5e4696ULL) }, - { make_floatx80(0x3ffe, 0xafffffffffff1470ULL), - make_floatx80(0x3fff, 0xce248c151f83fd69ULL), - make_floatx80(0x3ffe, 0x9c49182a3f07fad2ULL) }, - { make_floatx80(0x3ffe, 0xb800000000000e0aULL), - make_floatx80(0x3fff, 0xd2a81d91f12aec5cULL), - make_floatx80(0x3ffe, 0xa5503b23e255d8b8ULL) }, - { make_floatx80(0x3ffe, 0xc00000000000b7faULL), - make_floatx80(0x3fff, 0xd744fccad69dd630ULL), - make_floatx80(0x3ffe, 0xae89f995ad3bac60ULL) }, - { make_floatx80(0x3ffe, 0xc800000000003aa6ULL), - make_floatx80(0x3fff, 0xdbfbb797daf25a44ULL), - make_floatx80(0x3ffe, 0xb7f76f2fb5e4b488ULL) }, - { make_floatx80(0x3ffe, 0xd00000000000a6aeULL), - make_floatx80(0x3fff, 0xe0ccdeec2a954685ULL), - make_floatx80(0x3ffe, 0xc199bdd8552a8d0aULL) }, - { make_floatx80(0x3ffe, 0xd800000000004165ULL), - make_floatx80(0x3fff, 0xe5b906e77c837155ULL), - make_floatx80(0x3ffe, 0xcb720dcef906e2aaULL) }, - { make_floatx80(0x3ffe, 0xe00000000000582cULL), - make_floatx80(0x3fff, 0xeac0c6e7dd24713aULL), - make_floatx80(0x3ffe, 0xd5818dcfba48e274ULL) }, - { make_floatx80(0x3ffe, 0xe800000000001a5dULL), - make_floatx80(0x3fff, 0xefe4b99bdcdb06ebULL), - make_floatx80(0x3ffe, 0xdfc97337b9b60dd6ULL) }, - { make_floatx80(0x3ffe, 0xefffffffffffc1efULL), - make_floatx80(0x3fff, 0xf5257d152486a2faULL), - make_floatx80(0x3ffe, 0xea4afa2a490d45f4ULL) }, - { make_floatx80(0x3ffe, 0xf800000000001069ULL), - make_floatx80(0x3fff, 0xfa83b2db722a0e5cULL), - make_floatx80(0x3ffe, 0xf50765b6e4541cb8ULL) }, - { make_floatx80(0x3fff, 0x8000000000000000ULL), - make_floatx80(0x4000, 0x8000000000000000ULL), - make_floatx80(0x3fff, 0x8000000000000000ULL) }, + { make_floatx80_init(0xbfff, 0x8000000000000000ULL), + make_floatx80_init(0x3ffe, 0x8000000000000000ULL), + make_floatx80_init(0xbffe, 0x8000000000000000ULL) }, + { make_floatx80_init(0xbffe, 0xf800000000002e7eULL), + make_floatx80_init(0x3ffe, 0x82cd8698ac2b9160ULL), + make_floatx80_init(0xbffd, 0xfa64f2cea7a8dd40ULL) }, + { make_floatx80_init(0xbffe, 0xefffffffffffe960ULL), + make_floatx80_init(0x3ffe, 0x85aac367cc488345ULL), + make_floatx80_init(0xbffd, 0xf4aa7930676ef976ULL) }, + { make_floatx80_init(0xbffe, 0xe800000000006f10ULL), + make_floatx80_init(0x3ffe, 0x88980e8092da5c14ULL), + make_floatx80_init(0xbffd, 0xeecfe2feda4b47d8ULL) }, + { make_floatx80_init(0xbffe, 0xe000000000008a45ULL), + make_floatx80_init(0x3ffe, 0x8b95c1e3ea8ba2a5ULL), + make_floatx80_init(0xbffd, 0xe8d47c382ae8bab6ULL) }, + { make_floatx80_init(0xbffe, 0xd7ffffffffff8a9eULL), + make_floatx80_init(0x3ffe, 0x8ea4398b45cd8116ULL), + make_floatx80_init(0xbffd, 0xe2b78ce97464fdd4ULL) }, + { make_floatx80_init(0xbffe, 0xd0000000000019a0ULL), + make_floatx80_init(0x3ffe, 0x91c3d373ab11b919ULL), + make_floatx80_init(0xbffd, 0xdc785918a9dc8dceULL) }, + { make_floatx80_init(0xbffe, 0xc7ffffffffff14dfULL), + make_floatx80_init(0x3ffe, 0x94f4efa8fef76836ULL), + make_floatx80_init(0xbffd, 0xd61620ae02112f94ULL) }, + { make_floatx80_init(0xbffe, 0xc000000000006530ULL), + make_floatx80_init(0x3ffe, 0x9837f0518db87fbbULL), + make_floatx80_init(0xbffd, 0xcf901f5ce48f008aULL) }, + { make_floatx80_init(0xbffe, 0xb7ffffffffff1723ULL), + make_floatx80_init(0x3ffe, 0x9b8d39b9d54eb74cULL), + make_floatx80_init(0xbffd, 0xc8e58c8c55629168ULL) }, + { make_floatx80_init(0xbffe, 0xb00000000000b5e1ULL), + make_floatx80_init(0x3ffe, 0x9ef5326091a0c366ULL), + make_floatx80_init(0xbffd, 0xc2159b3edcbe7934ULL) }, + { make_floatx80_init(0xbffe, 0xa800000000006f8aULL), + make_floatx80_init(0x3ffe, 0xa27043030c49370aULL), + make_floatx80_init(0xbffd, 0xbb1f79f9e76d91ecULL) }, + { make_floatx80_init(0xbffe, 0x9fffffffffff816aULL), + make_floatx80_init(0x3ffe, 0xa5fed6a9b15171cfULL), + make_floatx80_init(0xbffd, 0xb40252ac9d5d1c62ULL) }, + { make_floatx80_init(0xbffe, 0x97ffffffffffb621ULL), + make_floatx80_init(0x3ffe, 0xa9a15ab4ea7c30e6ULL), + make_floatx80_init(0xbffd, 0xacbd4a962b079e34ULL) }, + { make_floatx80_init(0xbffe, 0x8fffffffffff162bULL), + make_floatx80_init(0x3ffe, 0xad583eea42a1b886ULL), + make_floatx80_init(0xbffd, 0xa54f822b7abc8ef4ULL) }, + { make_floatx80_init(0xbffe, 0x87ffffffffff4d34ULL), + make_floatx80_init(0x3ffe, 0xb123f581d2ac7b51ULL), + make_floatx80_init(0xbffd, 0x9db814fc5aa7095eULL) }, + { make_floatx80_init(0xbffe, 0x800000000000227dULL), + make_floatx80_init(0x3ffe, 0xb504f333f9de539dULL), + make_floatx80_init(0xbffd, 0x95f619980c4358c6ULL) }, + { make_floatx80_init(0xbffd, 0xefffffffffff3978ULL), + make_floatx80_init(0x3ffe, 0xb8fbaf4762fbd0a1ULL), + make_floatx80_init(0xbffd, 0x8e08a1713a085ebeULL) }, + { make_floatx80_init(0xbffd, 0xe00000000000df81ULL), + make_floatx80_init(0x3ffe, 0xbd08a39f580bfd8cULL), + make_floatx80_init(0xbffd, 0x85eeb8c14fe804e8ULL) }, + { make_floatx80_init(0xbffd, 0xd00000000000bccfULL), + make_floatx80_init(0x3ffe, 0xc12c4cca667062f6ULL), + make_floatx80_init(0xbffc, 0xfb4eccd6663e7428ULL) }, + { make_floatx80_init(0xbffd, 0xc00000000000eff0ULL), + make_floatx80_init(0x3ffe, 0xc5672a1155069abeULL), + make_floatx80_init(0xbffc, 0xea6357baabe59508ULL) }, + { make_floatx80_init(0xbffd, 0xb000000000000fe6ULL), + make_floatx80_init(0x3ffe, 0xc9b9bd866e2f234bULL), + make_floatx80_init(0xbffc, 0xd91909e6474372d4ULL) }, + { make_floatx80_init(0xbffd, 0x9fffffffffff2172ULL), + make_floatx80_init(0x3ffe, 0xce248c151f84bf00ULL), + make_floatx80_init(0xbffc, 0xc76dcfab81ed0400ULL) }, + { make_floatx80_init(0xbffd, 0x8fffffffffffafffULL), + make_floatx80_init(0x3ffe, 0xd2a81d91f12afb2bULL), + make_floatx80_init(0xbffc, 0xb55f89b83b541354ULL) }, + { make_floatx80_init(0xbffc, 0xffffffffffff81a3ULL), + make_floatx80_init(0x3ffe, 0xd744fccad69d7d5eULL), + make_floatx80_init(0xbffc, 0xa2ec0cd4a58a0a88ULL) }, + { make_floatx80_init(0xbffc, 0xdfffffffffff1568ULL), + make_floatx80_init(0x3ffe, 0xdbfbb797daf25a44ULL), + make_floatx80_init(0xbffc, 0x901121a0943696f0ULL) }, + { make_floatx80_init(0xbffc, 0xbfffffffffff68daULL), + make_floatx80_init(0x3ffe, 0xe0ccdeec2a94f811ULL), + make_floatx80_init(0xbffb, 0xf999089eab583f78ULL) }, + { make_floatx80_init(0xbffc, 0x9fffffffffff4690ULL), + make_floatx80_init(0x3ffe, 0xe5b906e77c83657eULL), + make_floatx80_init(0xbffb, 0xd237c8c41be4d410ULL) }, + { make_floatx80_init(0xbffb, 0xffffffffffff8aeeULL), + make_floatx80_init(0x3ffe, 0xeac0c6e7dd24427cULL), + make_floatx80_init(0xbffb, 0xa9f9c8c116ddec20ULL) }, + { make_floatx80_init(0xbffb, 0xbfffffffffff2d18ULL), + make_floatx80_init(0x3ffe, 0xefe4b99bdcdb06ebULL), + make_floatx80_init(0xbffb, 0x80da33211927c8a8ULL) }, + { make_floatx80_init(0xbffa, 0xffffffffffff8ccbULL), + make_floatx80_init(0x3ffe, 0xf5257d152486d0f4ULL), + make_floatx80_init(0xbffa, 0xada82eadb792f0c0ULL) }, + { make_floatx80_init(0xbff9, 0xffffffffffff11feULL), + make_floatx80_init(0x3ffe, 0xfa83b2db722a0846ULL), + make_floatx80_init(0xbff9, 0xaf89a491babef740ULL) }, + { floatx80_zero_init, + make_floatx80_init(0x3fff, 0x8000000000000000ULL), + floatx80_zero_init }, + { make_floatx80_init(0x3ff9, 0xffffffffffff2680ULL), + make_floatx80_init(0x3fff, 0x82cd8698ac2b9f6fULL), + make_floatx80_init(0x3ff9, 0xb361a62b0ae7dbc0ULL) }, + { make_floatx80_init(0x3ffb, 0x800000000000b500ULL), + make_floatx80_init(0x3fff, 0x85aac367cc488345ULL), + make_floatx80_init(0x3ffa, 0xb5586cf9891068a0ULL) }, + { make_floatx80_init(0x3ffb, 0xbfffffffffff4b67ULL), + make_floatx80_init(0x3fff, 0x88980e8092da7cceULL), + make_floatx80_init(0x3ffb, 0x8980e8092da7cce0ULL) }, + { make_floatx80_init(0x3ffb, 0xffffffffffffff57ULL), + make_floatx80_init(0x3fff, 0x8b95c1e3ea8bd6dfULL), + make_floatx80_init(0x3ffb, 0xb95c1e3ea8bd6df0ULL) }, + { make_floatx80_init(0x3ffc, 0x9fffffffffff811fULL), + make_floatx80_init(0x3fff, 0x8ea4398b45cd4780ULL), + make_floatx80_init(0x3ffb, 0xea4398b45cd47800ULL) }, + { make_floatx80_init(0x3ffc, 0xbfffffffffff9980ULL), + make_floatx80_init(0x3fff, 0x91c3d373ab11b919ULL), + make_floatx80_init(0x3ffc, 0x8e1e9b9d588dc8c8ULL) }, + { make_floatx80_init(0x3ffc, 0xdffffffffffff631ULL), + make_floatx80_init(0x3fff, 0x94f4efa8fef70864ULL), + make_floatx80_init(0x3ffc, 0xa7a77d47f7b84320ULL) }, + { make_floatx80_init(0x3ffc, 0xffffffffffff2499ULL), + make_floatx80_init(0x3fff, 0x9837f0518db892d4ULL), + make_floatx80_init(0x3ffc, 0xc1bf828c6dc496a0ULL) }, + { make_floatx80_init(0x3ffd, 0x8fffffffffff80fbULL), + make_floatx80_init(0x3fff, 0x9b8d39b9d54e3a79ULL), + make_floatx80_init(0x3ffc, 0xdc69cdceaa71d3c8ULL) }, + { make_floatx80_init(0x3ffd, 0x9fffffffffffbc23ULL), + make_floatx80_init(0x3fff, 0x9ef5326091a10313ULL), + make_floatx80_init(0x3ffc, 0xf7a993048d081898ULL) }, + { make_floatx80_init(0x3ffd, 0xafffffffffff20ecULL), + make_floatx80_init(0x3fff, 0xa27043030c49370aULL), + make_floatx80_init(0x3ffd, 0x89c10c0c3124dc28ULL) }, + { make_floatx80_init(0x3ffd, 0xc00000000000fd2cULL), + make_floatx80_init(0x3fff, 0xa5fed6a9b15171cfULL), + make_floatx80_init(0x3ffd, 0x97fb5aa6c545c73cULL) }, + { make_floatx80_init(0x3ffd, 0xd0000000000093beULL), + make_floatx80_init(0x3fff, 0xa9a15ab4ea7c30e6ULL), + make_floatx80_init(0x3ffd, 0xa6856ad3a9f0c398ULL) }, + { make_floatx80_init(0x3ffd, 0xe00000000000c2aeULL), + make_floatx80_init(0x3fff, 0xad583eea42a17876ULL), + make_floatx80_init(0x3ffd, 0xb560fba90a85e1d8ULL) }, + { make_floatx80_init(0x3ffd, 0xefffffffffff1e3fULL), + make_floatx80_init(0x3fff, 0xb123f581d2abef6cULL), + make_floatx80_init(0x3ffd, 0xc48fd6074aafbdb0ULL) }, + { make_floatx80_init(0x3ffd, 0xffffffffffff1c23ULL), + make_floatx80_init(0x3fff, 0xb504f333f9de2cadULL), + make_floatx80_init(0x3ffd, 0xd413cccfe778b2b4ULL) }, + { make_floatx80_init(0x3ffe, 0x8800000000006344ULL), + make_floatx80_init(0x3fff, 0xb8fbaf4762fbd0a1ULL), + make_floatx80_init(0x3ffd, 0xe3eebd1d8bef4284ULL) }, + { make_floatx80_init(0x3ffe, 0x9000000000005d67ULL), + make_floatx80_init(0x3fff, 0xbd08a39f580c668dULL), + make_floatx80_init(0x3ffd, 0xf4228e7d60319a34ULL) }, + { make_floatx80_init(0x3ffe, 0x9800000000009127ULL), + make_floatx80_init(0x3fff, 0xc12c4cca6670e042ULL), + make_floatx80_init(0x3ffe, 0x82589994cce1c084ULL) }, + { make_floatx80_init(0x3ffe, 0x9fffffffffff06f9ULL), + make_floatx80_init(0x3fff, 0xc5672a11550655c3ULL), + make_floatx80_init(0x3ffe, 0x8ace5422aa0cab86ULL) }, + { make_floatx80_init(0x3ffe, 0xa7fffffffffff80dULL), + make_floatx80_init(0x3fff, 0xc9b9bd866e2f234bULL), + make_floatx80_init(0x3ffe, 0x93737b0cdc5e4696ULL) }, + { make_floatx80_init(0x3ffe, 0xafffffffffff1470ULL), + make_floatx80_init(0x3fff, 0xce248c151f83fd69ULL), + make_floatx80_init(0x3ffe, 0x9c49182a3f07fad2ULL) }, + { make_floatx80_init(0x3ffe, 0xb800000000000e0aULL), + make_floatx80_init(0x3fff, 0xd2a81d91f12aec5cULL), + make_floatx80_init(0x3ffe, 0xa5503b23e255d8b8ULL) }, + { make_floatx80_init(0x3ffe, 0xc00000000000b7faULL), + make_floatx80_init(0x3fff, 0xd744fccad69dd630ULL), + make_floatx80_init(0x3ffe, 0xae89f995ad3bac60ULL) }, + { make_floatx80_init(0x3ffe, 0xc800000000003aa6ULL), + make_floatx80_init(0x3fff, 0xdbfbb797daf25a44ULL), + make_floatx80_init(0x3ffe, 0xb7f76f2fb5e4b488ULL) }, + { make_floatx80_init(0x3ffe, 0xd00000000000a6aeULL), + make_floatx80_init(0x3fff, 0xe0ccdeec2a954685ULL), + make_floatx80_init(0x3ffe, 0xc199bdd8552a8d0aULL) }, + { make_floatx80_init(0x3ffe, 0xd800000000004165ULL), + make_floatx80_init(0x3fff, 0xe5b906e77c837155ULL), + make_floatx80_init(0x3ffe, 0xcb720dcef906e2aaULL) }, + { make_floatx80_init(0x3ffe, 0xe00000000000582cULL), + make_floatx80_init(0x3fff, 0xeac0c6e7dd24713aULL), + make_floatx80_init(0x3ffe, 0xd5818dcfba48e274ULL) }, + { make_floatx80_init(0x3ffe, 0xe800000000001a5dULL), + make_floatx80_init(0x3fff, 0xefe4b99bdcdb06ebULL), + make_floatx80_init(0x3ffe, 0xdfc97337b9b60dd6ULL) }, + { make_floatx80_init(0x3ffe, 0xefffffffffffc1efULL), + make_floatx80_init(0x3fff, 0xf5257d152486a2faULL), + make_floatx80_init(0x3ffe, 0xea4afa2a490d45f4ULL) }, + { make_floatx80_init(0x3ffe, 0xf800000000001069ULL), + make_floatx80_init(0x3fff, 0xfa83b2db722a0e5cULL), + make_floatx80_init(0x3ffe, 0xf50765b6e4541cb8ULL) }, + { make_floatx80_init(0x3fff, 0x8000000000000000ULL), + make_floatx80_init(0x4000, 0x8000000000000000ULL), + make_floatx80_init(0x3fff, 0x8000000000000000ULL) }, }; void helper_f2xm1(CPUX86State *env) @@ -1275,24 +1275,24 @@ struct fpatan_data { }; static const struct fpatan_data fpatan_table[9] = { - { floatx80_zero, - floatx80_zero }, - { make_floatx80(0x3ffb, 0xfeadd4d5617b6e33ULL), - make_floatx80(0xbfb9, 0xdda19d8305ddc420ULL) }, - { make_floatx80(0x3ffc, 0xfadbafc96406eb15ULL), - make_floatx80(0x3fbb, 0xdb8f3debef442fccULL) }, - { make_floatx80(0x3ffd, 0xb7b0ca0f26f78474ULL), - make_floatx80(0xbfbc, 0xeab9bdba460376faULL) }, - { make_floatx80(0x3ffd, 0xed63382b0dda7b45ULL), - make_floatx80(0x3fbc, 0xdfc88bd978751a06ULL) }, - { make_floatx80(0x3ffe, 0x8f005d5ef7f59f9bULL), - make_floatx80(0x3fbd, 0xb906bc2ccb886e90ULL) }, - { make_floatx80(0x3ffe, 0xa4bc7d1934f70924ULL), - make_floatx80(0x3fbb, 0xcd43f9522bed64f8ULL) }, - { make_floatx80(0x3ffe, 0xb8053e2bc2319e74ULL), - make_floatx80(0xbfbc, 0xd3496ab7bd6eef0cULL) }, - { make_floatx80(0x3ffe, 0xc90fdaa22168c235ULL), - make_floatx80(0xbfbc, 0xece675d1fc8f8cbcULL) }, + { floatx80_zero_init, + floatx80_zero_init }, + { make_floatx80_init(0x3ffb, 0xfeadd4d5617b6e33ULL), + make_floatx80_init(0xbfb9, 0xdda19d8305ddc420ULL) }, + { make_floatx80_init(0x3ffc, 0xfadbafc96406eb15ULL), + make_floatx80_init(0x3fbb, 0xdb8f3debef442fccULL) }, + { make_floatx80_init(0x3ffd, 0xb7b0ca0f26f78474ULL), + make_floatx80_init(0xbfbc, 0xeab9bdba460376faULL) }, + { make_floatx80_init(0x3ffd, 0xed63382b0dda7b45ULL), + make_floatx80_init(0x3fbc, 0xdfc88bd978751a06ULL) }, + { make_floatx80_init(0x3ffe, 0x8f005d5ef7f59f9bULL), + make_floatx80_init(0x3fbd, 0xb906bc2ccb886e90ULL) }, + { make_floatx80_init(0x3ffe, 0xa4bc7d1934f70924ULL), + make_floatx80_init(0x3fbb, 0xcd43f9522bed64f8ULL) }, + { make_floatx80_init(0x3ffe, 0xb8053e2bc2319e74ULL), + make_floatx80_init(0xbfbc, 0xd3496ab7bd6eef0cULL) }, + { make_floatx80_init(0x3ffe, 0xc90fdaa22168c235ULL), + make_floatx80_init(0xbfbc, 0xece675d1fc8f8cbcULL) }, }; void helper_fpatan(CPUX86State *env) -- 2.20.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH v3 07/16] target/i386: floatx80: avoid compound literals in static initializers 2020-07-24 6:45 ` [PATCH v3 07/16] target/i386: floatx80: avoid compound literals in static initializers Alex Bennée @ 2020-07-24 18:23 ` Richard Henderson 0 siblings, 0 replies; 24+ messages in thread From: Richard Henderson @ 2020-07-24 18:23 UTC (permalink / raw) To: Alex Bennée, qemu-devel Cc: fam, Peter Maydell, berrange, Eduardo Habkost, Laszlo Ersek, f4bug, Philippe Mathieu-Daudé, cota, Joseph Myers, Paolo Bonzini, aurelien, Richard Henderson On 7/23/20 11:45 PM, Alex Bennée wrote: > From: Laszlo Ersek <lersek@redhat.com> > > Quoting ISO C99 6.7.8p4, "All the expressions in an initializer for an > object that has static storage duration shall be constant expressions or > string literals". > > The compound literal produced by the make_floatx80() macro is not such a > constant expression, per 6.6p7-9. (An implementation may accept it, > according to 6.6p10, but is not required to.) > > Therefore using "floatx80_zero" and make_floatx80() for initializing > "f2xm1_table" and "fpatan_table" is not portable. And gcc-4.8 in RHEL-7.6 > actually chokes on them: > >> target/i386/fpu_helper.c:871:5: error: initializer element is not constant >> { make_floatx80(0xbfff, 0x8000000000000000ULL), >> ^ > We've had the make_floatx80_init() macro for this purpose since commit > 3bf7e40ab914 ("softfloat: fix for C99", 2012-03-17), so let's use that > macro again. > > Fixes: eca30647fc07 > Fixes: ff57bb7b6326 > Signed-off-by: Laszlo Ersek <lersek@redhat.com> > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > Reviewed-by: Alex Bennée <alex.bennee@linaro.org> > Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> > Cc: Aurelien Jarno <aurelien@aurel32.net> > Cc: Eduardo Habkost <ehabkost@redhat.com> > Cc: Joseph Myers <joseph@codesourcery.com> > Cc: Paolo Bonzini <pbonzini@redhat.com> > Cc: Peter Maydell <peter.maydell@linaro.org> > Cc: Richard Henderson <rth@twiddle.net> > Link: https://lists.gnu.org/archive/html/qemu-devel/2017-08/msg06566.html > Link: https://lists.gnu.org/archive/html/qemu-devel/2020-07/msg04714.html > Message-Id: <20200716144251.23004-1-lersek@redhat.com> > Message-Id: <20200722062902.24509-8-alex.bennee@linaro.org> > --- > include/fpu/softfloat.h | 1 + > target/i386/fpu_helper.c | 426 +++++++++++++++++++-------------------- > 2 files changed, 214 insertions(+), 213 deletions(-) Reviewed-by: Richard Henderson <richard.henderson@linaro.org> r~ ^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH v3 08/16] linux-user: don't use MAP_FIXED in pgd_find_hole_fallback 2020-07-24 6:44 [PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu) Alex Bennée ` (6 preceding siblings ...) 2020-07-24 6:45 ` [PATCH v3 07/16] target/i386: floatx80: avoid compound literals in static initializers Alex Bennée @ 2020-07-24 6:45 ` Alex Bennée 2020-07-24 6:45 ` [PATCH v3 09/16] tests/docker: fix update command due to python3 str/bytes distinction Alex Bennée ` (7 subsequent siblings) 15 siblings, 0 replies; 24+ messages in thread From: Alex Bennée @ 2020-07-24 6:45 UTC (permalink / raw) To: qemu-devel Cc: fam, berrange, Alex Bennée, richard.henderson, f4bug, Laurent Vivier, cota, aurelien Plain MAP_FIXED has the undesirable behaviour of splatting exiting maps so we don't actually achieve what we want when looking for gaps. We should be using MAP_FIXED_NOREPLACE. As this isn't always available we need to potentially check the returned address to see if the kernel gave us what we asked for. Fixes: ad592e37dfc ("linux-user: provide fallback pgd_find_hole for bare chroots") Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-Id: <20200722062902.24509-9-alex.bennee@linaro.org> --- include/qemu/osdep.h | 3 +++ linux-user/elfload.c | 10 ++++++---- 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h index 0b1298b3c91..20872e793e4 100644 --- a/include/qemu/osdep.h +++ b/include/qemu/osdep.h @@ -173,6 +173,9 @@ extern int daemon(int, int); #ifndef MAP_ANONYMOUS #define MAP_ANONYMOUS MAP_ANON #endif +#ifndef MAP_FIXED_NOREPLACE +#define MAP_FIXED_NOREPLACE 0 +#endif #ifndef ENOMEDIUM #define ENOMEDIUM ENODEV #endif diff --git a/linux-user/elfload.c b/linux-user/elfload.c index 7e7f642332d..fe9dfe795dd 100644 --- a/linux-user/elfload.c +++ b/linux-user/elfload.c @@ -2134,12 +2134,15 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, /* we have run out of space */ return -1; } else { - int flags = MAP_ANONYMOUS | MAP_PRIVATE | MAP_NORESERVE | MAP_FIXED; + int flags = MAP_ANONYMOUS | MAP_PRIVATE | MAP_NORESERVE | + MAP_FIXED_NOREPLACE; void * mmap_start = mmap((void *) align_start, guest_size, PROT_NONE, flags, -1, 0); if (mmap_start != MAP_FAILED) { munmap((void *) align_start, guest_size); - return (uintptr_t) mmap_start + offset; + if (MAP_FIXED_NOREPLACE || mmap_start == (void *) align_start) { + return (uintptr_t) mmap_start + offset; + } } base += qemu_host_page_size; } @@ -2307,9 +2310,8 @@ static void pgb_reserved_va(const char *image_name, abi_ulong guest_loaddr, /* Widen the "image" to the entire reserved address space. */ pgb_static(image_name, 0, reserved_va, align); -#ifdef MAP_FIXED_NOREPLACE + /* osdep.h defines this as 0 if it's missing */ flags |= MAP_FIXED_NOREPLACE; -#endif /* Reserve the memory on the host. */ assert(guest_base != 0); -- 2.20.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v3 09/16] tests/docker: fix update command due to python3 str/bytes distinction 2020-07-24 6:44 [PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu) Alex Bennée ` (7 preceding siblings ...) 2020-07-24 6:45 ` [PATCH v3 08/16] linux-user: don't use MAP_FIXED in pgd_find_hole_fallback Alex Bennée @ 2020-07-24 6:45 ` Alex Bennée 2020-07-24 6:45 ` [PATCH v3 10/16] tests/docker: fix binfmt_misc image building Alex Bennée ` (6 subsequent siblings) 15 siblings, 0 replies; 24+ messages in thread From: Alex Bennée @ 2020-07-24 6:45 UTC (permalink / raw) To: qemu-devel Cc: fam, berrange, Alex Bennée, richard.henderson, f4bug, Philippe Mathieu-Daudé, cota, aurelien Does this seem convoluted to you? It feels a little complicated to me. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200722062902.24509-10-alex.bennee@linaro.org> --- tests/docker/docker.py | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/tests/docker/docker.py b/tests/docker/docker.py index c9f20d8d093..356d7618f1a 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -24,7 +24,7 @@ import tempfile import re import signal from tarfile import TarFile, TarInfo -from io import StringIO +from io import StringIO, BytesIO from shutil import copy, rmtree from pwd import getpwuid from datetime import datetime, timedelta @@ -541,13 +541,14 @@ class UpdateCommand(SubCommand): # Create a Docker buildfile df = StringIO() - df.write("FROM %s\n" % args.tag) - df.write("ADD . /\n") - df.seek(0) + df.write(u"FROM %s\n" % args.tag) + df.write(u"ADD . /\n") + + df_bytes = BytesIO(bytes(df.getvalue(), "UTF-8")) df_tar = TarInfo(name="Dockerfile") - df_tar.size = len(df.buf) - tmp_tar.addfile(df_tar, fileobj=df) + df_tar.size = df_bytes.getbuffer().nbytes + tmp_tar.addfile(df_tar, fileobj=df_bytes) tmp_tar.close() -- 2.20.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v3 10/16] tests/docker: fix binfmt_misc image building 2020-07-24 6:44 [PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu) Alex Bennée ` (8 preceding siblings ...) 2020-07-24 6:45 ` [PATCH v3 09/16] tests/docker: fix update command due to python3 str/bytes distinction Alex Bennée @ 2020-07-24 6:45 ` Alex Bennée 2020-07-24 6:45 ` [PATCH v3 11/16] tests/docker: add support for DEB_KEYRING Alex Bennée ` (5 subsequent siblings) 15 siblings, 0 replies; 24+ messages in thread From: Alex Bennée @ 2020-07-24 6:45 UTC (permalink / raw) To: qemu-devel Cc: fam, berrange, Alex Bennée, richard.henderson, f4bug, Philippe Mathieu-Daudé, cota, aurelien When we updated the arguments for docker.py we missed a bit. Fixes: dfae6284 Fixes: dfae628459 ("docker.py/build: support -t and -f arguments") Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-Id: <20200722062902.24509-11-alex.bennee@linaro.org> --- tests/docker/Makefile.include | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include index a104e9df281..9119dff97de 100644 --- a/tests/docker/Makefile.include +++ b/tests/docker/Makefile.include @@ -78,7 +78,7 @@ docker-binfmt-image-debian-%: $(DOCKER_FILES_DIR)/debian-bootstrap.docker DEB_ARCH=$(DEB_ARCH) \ DEB_TYPE=$(DEB_TYPE) \ $(if $(DEB_URL),DEB_URL=$(DEB_URL),) \ - $(DOCKER_SCRIPT) build qemu/debian-$* $< \ + $(DOCKER_SCRIPT) build -t qemu/debian-$* -f $< \ $(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \ $(if $(NOUSER),,--add-current-user) \ $(if $(EXTRA_FILES),--extra-files $(EXTRA_FILES)) \ -- 2.20.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v3 11/16] tests/docker: add support for DEB_KEYRING 2020-07-24 6:44 [PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu) Alex Bennée ` (9 preceding siblings ...) 2020-07-24 6:45 ` [PATCH v3 10/16] tests/docker: fix binfmt_misc image building Alex Bennée @ 2020-07-24 6:45 ` Alex Bennée 2020-07-24 6:45 ` [PATCH v3 12/16] linux-user: fix clock_nanosleep() Alex Bennée ` (4 subsequent siblings) 15 siblings, 0 replies; 24+ messages in thread From: Alex Bennée @ 2020-07-24 6:45 UTC (permalink / raw) To: qemu-devel Cc: fam, berrange, Alex Bennée, richard.henderson, f4bug, Philippe Mathieu-Daudé, cota, aurelien For installing stuff from sid or ports you may need to manually specify the location of the keyring. You can even import keys into your personal keyring and point it there, e.g.: gpg --keyserver keyring.debian.org --recv-keys 84C573CD4E1AFD6C make docker-binfmt-image-debian-sid-hppa DEB_TYPE=sid DEB_ARCH=hppa \ DEB_URL=http://ftp.ports.debian.org/debian-ports/ \ EXECUTABLE=./hppa-linux-user/qemu-hppa V=1 \ DEB_KEYRING=${HOME}/.gnupg/pubring.kbx Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-Id: <20200722062902.24509-12-alex.bennee@linaro.org> --- tests/docker/dockerfiles/debian-bootstrap.pre | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tests/docker/dockerfiles/debian-bootstrap.pre b/tests/docker/dockerfiles/debian-bootstrap.pre index c164778c302..35c85f7db8a 100755 --- a/tests/docker/dockerfiles/debian-bootstrap.pre +++ b/tests/docker/dockerfiles/debian-bootstrap.pre @@ -79,6 +79,13 @@ else fi fi +# +# Add optional args +# +if [ -n "${DEB_KEYRING}" ]; then + DEBOOTSTRAP="${DEBOOTSTRAP} --keyring=${DEB_KEYRING}" +fi + # # Finally check to see if any qemu's are installed # -- 2.20.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v3 12/16] linux-user: fix clock_nanosleep() 2020-07-24 6:44 [PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu) Alex Bennée ` (10 preceding siblings ...) 2020-07-24 6:45 ` [PATCH v3 11/16] tests/docker: add support for DEB_KEYRING Alex Bennée @ 2020-07-24 6:45 ` Alex Bennée 2020-07-24 18:24 ` Richard Henderson 2020-07-24 6:45 ` [PATCH v3 13/16] linux-user, ppc: fix clock_nanosleep() for linux-user-ppc Alex Bennée ` (3 subsequent siblings) 15 siblings, 1 reply; 24+ messages in thread From: Alex Bennée @ 2020-07-24 6:45 UTC (permalink / raw) To: qemu-devel Cc: fam, berrange, Alex Bennée, richard.henderson, f4bug, Laurent Vivier, cota, aurelien From: Laurent Vivier <laurent@vivier.eu> If the call is interrupted by a signal handler, it fails with error EINTR and if "remain" is not NULL and "flags" is not TIMER_ABSTIME, it returns the remaining unslept time in "remain". Update linux-user to not overwrite the "remain" structure if there is no error. Found with "make check-tcg", linux-test fails on nanosleep test: TEST linux-test on x86_64 .../tests/tcg/multiarch/linux-test.c:242: nanosleep Reported-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Laurent Vivier <laurent@vivier.eu> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200722174612.2917566-2-laurent@vivier.eu> --- linux-user/syscall.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/linux-user/syscall.c b/linux-user/syscall.c index 1211e759c26..43a6e283961 100644 --- a/linux-user/syscall.c +++ b/linux-user/syscall.c @@ -11831,8 +11831,14 @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1, target_to_host_timespec(&ts, arg3); ret = get_errno(safe_clock_nanosleep(arg1, arg2, &ts, arg4 ? &ts : NULL)); - if (arg4) + /* + * if the call is interrupted by a signal handler, it fails + * with error -TARGET_EINTR and if arg4 is not NULL and arg2 is not + * TIMER_ABSTIME, it returns the remaining unslept time in arg4. + */ + if (ret == -TARGET_EINTR && arg4 && arg2 != TIMER_ABSTIME) { host_to_target_timespec(arg4, &ts); + } #if defined(TARGET_PPC) /* clock_nanosleep is odd in that it returns positive errno values. -- 2.20.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH v3 12/16] linux-user: fix clock_nanosleep() 2020-07-24 6:45 ` [PATCH v3 12/16] linux-user: fix clock_nanosleep() Alex Bennée @ 2020-07-24 18:24 ` Richard Henderson 0 siblings, 0 replies; 24+ messages in thread From: Richard Henderson @ 2020-07-24 18:24 UTC (permalink / raw) To: Alex Bennée, qemu-devel Cc: fam, berrange, f4bug, Laurent Vivier, cota, aurelien On 7/23/20 11:45 PM, Alex Bennée wrote: > From: Laurent Vivier <laurent@vivier.eu> > > If the call is interrupted by a signal handler, it fails with error EINTR > and if "remain" is not NULL and "flags" is not TIMER_ABSTIME, it returns > the remaining unslept time in "remain". > > Update linux-user to not overwrite the "remain" structure if there is no > error. > > Found with "make check-tcg", linux-test fails on nanosleep test: > > TEST linux-test on x86_64 > .../tests/tcg/multiarch/linux-test.c:242: nanosleep > > Reported-by: Philippe Mathieu-Daudé <f4bug@amsat.org> > Signed-off-by: Laurent Vivier <laurent@vivier.eu> > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > Message-Id: <20200722174612.2917566-2-laurent@vivier.eu> > --- > linux-user/syscall.c | 8 +++++++- > 1 file changed, 7 insertions(+), 1 deletion(-) Reviewed-by: Richard Henderson <richard.henderson@linaro.org> r~ ^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH v3 13/16] linux-user, ppc: fix clock_nanosleep() for linux-user-ppc 2020-07-24 6:44 [PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu) Alex Bennée ` (11 preceding siblings ...) 2020-07-24 6:45 ` [PATCH v3 12/16] linux-user: fix clock_nanosleep() Alex Bennée @ 2020-07-24 6:45 ` Alex Bennée 2020-07-24 18:24 ` [PATCH v3 13/16] linux-user,ppc: " Richard Henderson 2020-07-24 6:45 ` [PATCH v3 14/16] python/qemu: Cleanup changes to ConsoleSocket Alex Bennée ` (2 subsequent siblings) 15 siblings, 1 reply; 24+ messages in thread From: Alex Bennée @ 2020-07-24 6:45 UTC (permalink / raw) To: qemu-devel Cc: fam, berrange, Alex Bennée, richard.henderson, f4bug, Laurent Vivier, cota, aurelien From: Laurent Vivier <laurent@vivier.eu> Our safe_clock_nanosleep() returns -1 and updates errno. We don't need to update the CRF bit in syscall.c because it will be updated in ppc/cpu_loop.c as the return value is negative. Signed-off-by: Laurent Vivier <laurent@vivier.eu> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200722174612.2917566-3-laurent@vivier.eu> --- linux-user/syscall.c | 7 ------- 1 file changed, 7 deletions(-) diff --git a/linux-user/syscall.c b/linux-user/syscall.c index 43a6e283961..f5c4f6b95db 100644 --- a/linux-user/syscall.c +++ b/linux-user/syscall.c @@ -11840,13 +11840,6 @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1, host_to_target_timespec(arg4, &ts); } -#if defined(TARGET_PPC) - /* clock_nanosleep is odd in that it returns positive errno values. - * On PPC, CR0 bit 3 should be set in such a situation. */ - if (ret && ret != -TARGET_ERESTARTSYS) { - ((CPUPPCState *)cpu_env)->crf[0] |= 1; - } -#endif return ret; } #endif -- 2.20.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH v3 13/16] linux-user,ppc: fix clock_nanosleep() for linux-user-ppc 2020-07-24 6:45 ` [PATCH v3 13/16] linux-user, ppc: fix clock_nanosleep() for linux-user-ppc Alex Bennée @ 2020-07-24 18:24 ` Richard Henderson 0 siblings, 0 replies; 24+ messages in thread From: Richard Henderson @ 2020-07-24 18:24 UTC (permalink / raw) To: Alex Bennée, qemu-devel Cc: fam, berrange, f4bug, Laurent Vivier, cota, aurelien On 7/23/20 11:45 PM, Alex Bennée wrote: > From: Laurent Vivier <laurent@vivier.eu> > > Our safe_clock_nanosleep() returns -1 and updates errno. > > We don't need to update the CRF bit in syscall.c because it will > be updated in ppc/cpu_loop.c as the return value is negative. > > Signed-off-by: Laurent Vivier <laurent@vivier.eu> > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > Message-Id: <20200722174612.2917566-3-laurent@vivier.eu> > --- > linux-user/syscall.c | 7 ------- > 1 file changed, 7 deletions(-) Reviewed-by: Richard Henderson <richard.henderson@linaro.org> r~ ^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH v3 14/16] python/qemu: Cleanup changes to ConsoleSocket 2020-07-24 6:44 [PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu) Alex Bennée ` (12 preceding siblings ...) 2020-07-24 6:45 ` [PATCH v3 13/16] linux-user, ppc: fix clock_nanosleep() for linux-user-ppc Alex Bennée @ 2020-07-24 6:45 ` Alex Bennée 2020-07-24 7:25 ` Philippe Mathieu-Daudé 2020-07-24 6:45 ` [PATCH v3 15/16] python/qemu: Change ConsoleSocket to optionally drain socket Alex Bennée 2020-07-24 6:45 ` [PATCH v3 16/16] tests/vm: add shutdown timeout in basevm.py Alex Bennée 15 siblings, 1 reply; 24+ messages in thread From: Alex Bennée @ 2020-07-24 6:45 UTC (permalink / raw) To: qemu-devel Cc: fam, berrange, Robert Foley, Alex Bennée, richard.henderson, f4bug, cota, Cleber Rosa, aurelien, Eduardo Habkost From: Robert Foley <robert.foley@linaro.org> The changes to console_socket.py and machine.py are to cleanup for pylint and flake8. Signed-off-by: Robert Foley <robert.foley@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200717203041.9867-2-robert.foley@linaro.org> --- python/qemu/console_socket.py | 57 ++++++++++++++++++----------------- python/qemu/machine.py | 7 +++-- python/qemu/pylintrc | 2 +- 3 files changed, 34 insertions(+), 32 deletions(-) diff --git a/python/qemu/console_socket.py b/python/qemu/console_socket.py index 830cb7c6282..09986bc2152 100644 --- a/python/qemu/console_socket.py +++ b/python/qemu/console_socket.py @@ -1,12 +1,9 @@ -#!/usr/bin/env python3 -# -# This python module implements a ConsoleSocket object which is -# designed always drain the socket itself, and place -# the bytes into a in memory buffer for later processing. -# -# Optionally a file path can be passed in and we will also -# dump the characters to this file for debug. -# +""" +QEMU Console Socket Module: + +This python module implements a ConsoleSocket object, +which can drain a socket and optionally dump the bytes to file. +""" # Copyright 2020 Linaro # # Authors: @@ -15,20 +12,27 @@ # This code is licensed under the GPL version 2 or later. See # the COPYING file in the top-level directory. # + import asyncore import socket import threading -import io -import os -import sys from collections import deque import time -import traceback + class ConsoleSocket(asyncore.dispatcher): + """ + ConsoleSocket represents a socket attached to a char device. + Drains the socket and places the bytes into an in memory buffer + for later processing. + + Optionally a file path can be passed in and we will also + dump the characters to this file for debugging purposes. + """ def __init__(self, address, file=None): self._recv_timeout_sec = 300 + self._sleep_time = 0.5 self._buffer = deque() self._asyncore_thread = None self._sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) @@ -70,31 +74,28 @@ class ConsoleSocket(asyncore.dispatcher): def handle_read(self): """process arriving characters into in memory _buffer""" - try: - data = asyncore.dispatcher.recv(self, 1) - # latin1 is needed since there are some chars - # we are receiving that cannot be encoded to utf-8 - # such as 0xe2, 0x80, 0xA6. - string = data.decode("latin1") - except: - print("Exception seen.") - traceback.print_exc() - return + data = asyncore.dispatcher.recv(self, 1) + # latin1 is needed since there are some chars + # we are receiving that cannot be encoded to utf-8 + # such as 0xe2, 0x80, 0xA6. + string = data.decode("latin1") if self._logfile: self._logfile.write("{}".format(string)) self._logfile.flush() for c in string: self._buffer.extend(c) - def recv(self, n=1, sleep_delay_s=0.1): - """Return chars from in memory buffer""" + def recv(self, buffer_size=1): + """Return chars from in memory buffer. + Maintains the same API as socket.socket.recv. + """ start_time = time.time() - while len(self._buffer) < n: - time.sleep(sleep_delay_s) + while len(self._buffer) < buffer_size: + time.sleep(self._sleep_time) elapsed_sec = time.time() - start_time if elapsed_sec > self._recv_timeout_sec: raise socket.timeout - chars = ''.join([self._buffer.popleft() for i in range(n)]) + chars = ''.join([self._buffer.popleft() for i in range(buffer_size)]) # We choose to use latin1 to remain consistent with # handle_read() and give back the same data as the user would # receive if they were reading directly from the diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 80c4d4a8b6e..9956360a792 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -27,7 +27,7 @@ import socket import tempfile from typing import Optional, Type from types import TracebackType -from qemu.console_socket import ConsoleSocket +from . import console_socket from . import qmp @@ -674,8 +674,9 @@ class QEMUMachine: """ if self._console_socket is None: if self._drain_console: - self._console_socket = ConsoleSocket(self._console_address, - file=self._console_log_path) + self._console_socket = console_socket.ConsoleSocket( + self._console_address, + file=self._console_log_path) else: self._console_socket = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) diff --git a/python/qemu/pylintrc b/python/qemu/pylintrc index 5d6ae7367d8..3f69205000d 100644 --- a/python/qemu/pylintrc +++ b/python/qemu/pylintrc @@ -33,7 +33,7 @@ good-names=i, Run, _, fd, - + c, [VARIABLES] [STRING] -- 2.20.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH v3 14/16] python/qemu: Cleanup changes to ConsoleSocket 2020-07-24 6:45 ` [PATCH v3 14/16] python/qemu: Cleanup changes to ConsoleSocket Alex Bennée @ 2020-07-24 7:25 ` Philippe Mathieu-Daudé 0 siblings, 0 replies; 24+ messages in thread From: Philippe Mathieu-Daudé @ 2020-07-24 7:25 UTC (permalink / raw) To: Alex Bennée, qemu-devel Cc: fam, berrange, Robert Foley, richard.henderson, cota, Cleber Rosa, John Snow, aurelien, Eduardo Habkost +John who requested the changes. On 7/24/20 8:45 AM, Alex Bennée wrote: > From: Robert Foley <robert.foley@linaro.org> > > The changes to console_socket.py and machine.py are to > cleanup for pylint and flake8. > > Signed-off-by: Robert Foley <robert.foley@linaro.org> > Reviewed-by: Alex Bennée <alex.bennee@linaro.org> > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > Message-Id: <20200717203041.9867-2-robert.foley@linaro.org> > --- > python/qemu/console_socket.py | 57 ++++++++++++++++++----------------- > python/qemu/machine.py | 7 +++-- > python/qemu/pylintrc | 2 +- > 3 files changed, 34 insertions(+), 32 deletions(-) > > diff --git a/python/qemu/console_socket.py b/python/qemu/console_socket.py > index 830cb7c6282..09986bc2152 100644 > --- a/python/qemu/console_socket.py > +++ b/python/qemu/console_socket.py > @@ -1,12 +1,9 @@ > -#!/usr/bin/env python3 > -# > -# This python module implements a ConsoleSocket object which is > -# designed always drain the socket itself, and place > -# the bytes into a in memory buffer for later processing. > -# > -# Optionally a file path can be passed in and we will also > -# dump the characters to this file for debug. > -# > +""" > +QEMU Console Socket Module: > + > +This python module implements a ConsoleSocket object, > +which can drain a socket and optionally dump the bytes to file. > +""" > # Copyright 2020 Linaro > # > # Authors: > @@ -15,20 +12,27 @@ > # This code is licensed under the GPL version 2 or later. See > # the COPYING file in the top-level directory. > # > + > import asyncore > import socket > import threading > -import io > -import os > -import sys > from collections import deque > import time > -import traceback > + > > class ConsoleSocket(asyncore.dispatcher): > + """ > + ConsoleSocket represents a socket attached to a char device. > > + Drains the socket and places the bytes into an in memory buffer > + for later processing. > + > + Optionally a file path can be passed in and we will also > + dump the characters to this file for debugging purposes. > + """ > def __init__(self, address, file=None): > self._recv_timeout_sec = 300 > + self._sleep_time = 0.5 > self._buffer = deque() > self._asyncore_thread = None > self._sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) > @@ -70,31 +74,28 @@ class ConsoleSocket(asyncore.dispatcher): > > def handle_read(self): > """process arriving characters into in memory _buffer""" > - try: > - data = asyncore.dispatcher.recv(self, 1) > - # latin1 is needed since there are some chars > - # we are receiving that cannot be encoded to utf-8 > - # such as 0xe2, 0x80, 0xA6. > - string = data.decode("latin1") > - except: > - print("Exception seen.") > - traceback.print_exc() > - return > + data = asyncore.dispatcher.recv(self, 1) A bit more than the commit description, but I don't mind. Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> > + # latin1 is needed since there are some chars > + # we are receiving that cannot be encoded to utf-8 > + # such as 0xe2, 0x80, 0xA6. > + string = data.decode("latin1") > if self._logfile: > self._logfile.write("{}".format(string)) > self._logfile.flush() > for c in string: > self._buffer.extend(c) > > - def recv(self, n=1, sleep_delay_s=0.1): > - """Return chars from in memory buffer""" > + def recv(self, buffer_size=1): > + """Return chars from in memory buffer. > + Maintains the same API as socket.socket.recv. > + """ > start_time = time.time() > - while len(self._buffer) < n: > - time.sleep(sleep_delay_s) > + while len(self._buffer) < buffer_size: > + time.sleep(self._sleep_time) > elapsed_sec = time.time() - start_time > if elapsed_sec > self._recv_timeout_sec: > raise socket.timeout > - chars = ''.join([self._buffer.popleft() for i in range(n)]) > + chars = ''.join([self._buffer.popleft() for i in range(buffer_size)]) > # We choose to use latin1 to remain consistent with > # handle_read() and give back the same data as the user would > # receive if they were reading directly from the > diff --git a/python/qemu/machine.py b/python/qemu/machine.py > index 80c4d4a8b6e..9956360a792 100644 > --- a/python/qemu/machine.py > +++ b/python/qemu/machine.py > @@ -27,7 +27,7 @@ import socket > import tempfile > from typing import Optional, Type > from types import TracebackType > -from qemu.console_socket import ConsoleSocket > +from . import console_socket > > from . import qmp > > @@ -674,8 +674,9 @@ class QEMUMachine: > """ > if self._console_socket is None: > if self._drain_console: > - self._console_socket = ConsoleSocket(self._console_address, > - file=self._console_log_path) > + self._console_socket = console_socket.ConsoleSocket( > + self._console_address, > + file=self._console_log_path) > else: > self._console_socket = socket.socket(socket.AF_UNIX, > socket.SOCK_STREAM) > diff --git a/python/qemu/pylintrc b/python/qemu/pylintrc > index 5d6ae7367d8..3f69205000d 100644 > --- a/python/qemu/pylintrc > +++ b/python/qemu/pylintrc > @@ -33,7 +33,7 @@ good-names=i, > Run, > _, > fd, > - > + c, > [VARIABLES] > > [STRING] > ^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH v3 15/16] python/qemu: Change ConsoleSocket to optionally drain socket. 2020-07-24 6:44 [PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu) Alex Bennée ` (13 preceding siblings ...) 2020-07-24 6:45 ` [PATCH v3 14/16] python/qemu: Cleanup changes to ConsoleSocket Alex Bennée @ 2020-07-24 6:45 ` Alex Bennée 2020-07-24 6:45 ` [PATCH v3 16/16] tests/vm: add shutdown timeout in basevm.py Alex Bennée 15 siblings, 0 replies; 24+ messages in thread From: Alex Bennée @ 2020-07-24 6:45 UTC (permalink / raw) To: qemu-devel Cc: fam, berrange, Robert Foley, Alex Bennée, richard.henderson, f4bug, cota, Cleber Rosa, aurelien, Eduardo Habkost From: Robert Foley <robert.foley@linaro.org> The primary purpose of this change is to clean up machine.py's console_socket property to return a single type, a ConsoleSocket. ConsoleSocket now derives from a socket, which means that in the default case (of not draining), machine.py will see the same behavior as it did prior to ConsoleSocket. Signed-off-by: Robert Foley <robert.foley@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200717203041.9867-3-robert.foley@linaro.org> --- python/qemu/console_socket.py | 92 +++++++++++++++++++++-------------- python/qemu/machine.py | 13 ++--- 2 files changed, 59 insertions(+), 46 deletions(-) diff --git a/python/qemu/console_socket.py b/python/qemu/console_socket.py index 09986bc2152..70869fbbdc4 100644 --- a/python/qemu/console_socket.py +++ b/python/qemu/console_socket.py @@ -13,68 +13,75 @@ which can drain a socket and optionally dump the bytes to file. # the COPYING file in the top-level directory. # -import asyncore import socket import threading from collections import deque import time -class ConsoleSocket(asyncore.dispatcher): +class ConsoleSocket(socket.socket): """ ConsoleSocket represents a socket attached to a char device. - Drains the socket and places the bytes into an in memory buffer - for later processing. + Optionally (if drain==True), drains the socket and places the bytes + into an in memory buffer for later processing. Optionally a file path can be passed in and we will also dump the characters to this file for debugging purposes. """ - def __init__(self, address, file=None): + def __init__(self, address, file=None, drain=False): self._recv_timeout_sec = 300 self._sleep_time = 0.5 self._buffer = deque() - self._asyncore_thread = None - self._sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) - self._sock.connect(address) + socket.socket.__init__(self, socket.AF_UNIX, socket.SOCK_STREAM) + self.connect(address) self._logfile = None if file: self._logfile = open(file, "w") - asyncore.dispatcher.__init__(self, sock=self._sock) self._open = True - self._thread_start() + if drain: + self._drain_thread = self._thread_start() + else: + self._drain_thread = None - def _thread_start(self): - """Kick off a thread to wait on the asyncore.loop""" - if self._asyncore_thread is not None: - return - self._asyncore_thread = threading.Thread(target=asyncore.loop, - kwargs={'timeout':1}) - self._asyncore_thread.daemon = True - self._asyncore_thread.start() + def _drain_fn(self): + """Drains the socket and runs while the socket is open.""" + while self._open: + try: + self._drain_socket() + except socket.timeout: + # The socket is expected to timeout since we set a + # short timeout to allow the thread to exit when + # self._open is set to False. + time.sleep(self._sleep_time) - def handle_close(self): - """redirect close to base class""" - # Call the base class close, but not self.close() since - # handle_close() occurs in the context of the thread which - # self.close() attempts to join. - asyncore.dispatcher.close(self) + def _thread_start(self): + """Kick off a thread to drain the socket.""" + # Configure socket to not block and timeout. + # This allows our drain thread to not block + # on recieve and exit smoothly. + socket.socket.setblocking(self, False) + socket.socket.settimeout(self, 1) + drain_thread = threading.Thread(target=self._drain_fn) + drain_thread.daemon = True + drain_thread.start() + return drain_thread def close(self): """Close the base object and wait for the thread to terminate""" if self._open: self._open = False - asyncore.dispatcher.close(self) - if self._asyncore_thread is not None: - thread, self._asyncore_thread = self._asyncore_thread, None + if self._drain_thread is not None: + thread, self._drain_thread = self._drain_thread, None thread.join() + socket.socket.close(self) if self._logfile: self._logfile.close() self._logfile = None - def handle_read(self): + def _drain_socket(self): """process arriving characters into in memory _buffer""" - data = asyncore.dispatcher.recv(self, 1) + data = socket.socket.recv(self, 1) # latin1 is needed since there are some chars # we are receiving that cannot be encoded to utf-8 # such as 0xe2, 0x80, 0xA6. @@ -85,27 +92,38 @@ class ConsoleSocket(asyncore.dispatcher): for c in string: self._buffer.extend(c) - def recv(self, buffer_size=1): + def recv(self, bufsize=1): """Return chars from in memory buffer. Maintains the same API as socket.socket.recv. """ + if self._drain_thread is None: + # Not buffering the socket, pass thru to socket. + return socket.socket.recv(self, bufsize) start_time = time.time() - while len(self._buffer) < buffer_size: + while len(self._buffer) < bufsize: time.sleep(self._sleep_time) elapsed_sec = time.time() - start_time if elapsed_sec > self._recv_timeout_sec: raise socket.timeout - chars = ''.join([self._buffer.popleft() for i in range(buffer_size)]) + chars = ''.join([self._buffer.popleft() for i in range(bufsize)]) # We choose to use latin1 to remain consistent with # handle_read() and give back the same data as the user would # receive if they were reading directly from the # socket w/o our intervention. return chars.encode("latin1") - def set_blocking(self): - """Maintain compatibility with socket API""" - pass + def setblocking(self, value): + """When not draining we pass thru to the socket, + since when draining we control socket blocking. + """ + if self._drain_thread is None: + socket.socket.setblocking(self, value) def settimeout(self, seconds): - """Set current timeout on recv""" - self._recv_timeout_sec = seconds + """When not draining we pass thru to the socket, + since when draining we control the timeout. + """ + if seconds is not None: + self._recv_timeout_sec = seconds + if self._drain_thread is None: + socket.socket.settimeout(self, seconds) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 9956360a792..c5f777b9e72 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -23,7 +23,6 @@ import os import subprocess import shutil import signal -import socket import tempfile from typing import Optional, Type from types import TracebackType @@ -673,12 +672,8 @@ class QEMUMachine: Returns a socket connected to the console """ if self._console_socket is None: - if self._drain_console: - self._console_socket = console_socket.ConsoleSocket( - self._console_address, - file=self._console_log_path) - else: - self._console_socket = socket.socket(socket.AF_UNIX, - socket.SOCK_STREAM) - self._console_socket.connect(self._console_address) + self._console_socket = console_socket.ConsoleSocket( + self._console_address, + file=self._console_log_path, + drain=self._drain_console) return self._console_socket -- 2.20.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v3 16/16] tests/vm: add shutdown timeout in basevm.py 2020-07-24 6:44 [PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu) Alex Bennée ` (14 preceding siblings ...) 2020-07-24 6:45 ` [PATCH v3 15/16] python/qemu: Change ConsoleSocket to optionally drain socket Alex Bennée @ 2020-07-24 6:45 ` Alex Bennée 15 siblings, 0 replies; 24+ messages in thread From: Alex Bennée @ 2020-07-24 6:45 UTC (permalink / raw) To: qemu-devel Cc: fam, berrange, Robert Foley, Alex Bennée, richard.henderson, f4bug, Philippe Mathieu-Daudé, cota, aurelien From: Robert Foley <robert.foley@linaro.org> We are adding the shutdown timeout to solve an issue we now see where the aarch64 VMs timeout on shutdown under TCG. There is a new 3 second timeout in machine.py, which we override in basevm.py when shutting down. Signed-off-by: Robert Foley <robert.foley@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200717203041.9867-4-robert.foley@linaro.org> --- tests/vm/basevm.py | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index 7acb48b8761..3fac20e929a 100644 --- a/tests/vm/basevm.py +++ b/tests/vm/basevm.py @@ -80,6 +80,8 @@ class BaseVM(object): arch = "#arch" # command to halt the guest, can be overridden by subclasses poweroff = "poweroff" + # Time to wait for shutdown to finish. + shutdown_timeout_default = 30 # enable IPv6 networking ipv6 = True # This is the timeout on the wait for console bytes. @@ -87,7 +89,7 @@ class BaseVM(object): # Scale up some timeouts under TCG. # 4 is arbitrary, but greater than 2, # since we found we need to wait more than twice as long. - tcg_ssh_timeout_multiplier = 4 + tcg_timeout_multiplier = 4 def __init__(self, args, config=None): self._guest = None self._genisoimage = args.genisoimage @@ -141,9 +143,12 @@ class BaseVM(object): if args.jobs and args.jobs > 1: self._args += ["-smp", "%d" % args.jobs] if kvm_available(self.arch): + self._shutdown_timeout = self.shutdown_timeout_default self._args += ["-enable-kvm"] else: logging.info("KVM not available, not using -enable-kvm") + self._shutdown_timeout = \ + self.shutdown_timeout_default * self.tcg_timeout_multiplier self._data_args = [] if self._config['qemu_args'] != None: @@ -423,7 +428,7 @@ class BaseVM(object): def wait_ssh(self, wait_root=False, seconds=300, cmd="exit 0"): # Allow more time for VM to boot under TCG. if not kvm_available(self.arch): - seconds *= self.tcg_ssh_timeout_multiplier + seconds *= self.tcg_timeout_multiplier starttime = datetime.datetime.now() endtime = starttime + datetime.timedelta(seconds=seconds) cmd_success = False @@ -441,14 +446,14 @@ class BaseVM(object): raise Exception("Timeout while waiting for guest ssh") def shutdown(self): - self._guest.shutdown() + self._guest.shutdown(timeout=self._shutdown_timeout) def wait(self): - self._guest.wait() + self._guest.wait(timeout=self._shutdown_timeout) def graceful_shutdown(self): self.ssh_root(self.poweroff) - self._guest.wait() + self._guest.wait(timeout=self._shutdown_timeout) def qmp(self, *args, **kwargs): return self._guest.qmp(*args, **kwargs) -- 2.20.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
end of thread, other threads:[~2020-07-24 18:27 UTC | newest] Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-07-24 6:44 [PATCH v3 00/16] candidate fixes for 5.1-rc1 (testing, semihosting, OOM tcg, x86 fpu) Alex Bennée 2020-07-24 6:44 ` [PATCH v3 01/16] shippable: add one more qemu to registry url Alex Bennée 2020-07-24 6:44 ` [PATCH v3 02/16] semihosting: defer connect_chardevs a little more to use serialx Alex Bennée 2020-07-24 6:44 ` [PATCH v3 03/16] semihosting: don't send the trailing '\0' Alex Bennée 2020-07-24 6:44 ` [PATCH v3 04/16] util: add qemu_get_host_physmem utility function Alex Bennée 2020-07-24 6:44 ` [PATCH v3 05/16] util/oslib-win32: add qemu_get_host_physmem implementation Alex Bennée 2020-07-24 6:51 ` Stefan Weil 2020-07-24 6:44 ` [PATCH v3 06/16] accel/tcg: better handle memory constrained systems Alex Bennée 2020-07-24 13:41 ` Robert Foley 2020-07-24 18:22 ` Richard Henderson 2020-07-24 6:45 ` [PATCH v3 07/16] target/i386: floatx80: avoid compound literals in static initializers Alex Bennée 2020-07-24 18:23 ` Richard Henderson 2020-07-24 6:45 ` [PATCH v3 08/16] linux-user: don't use MAP_FIXED in pgd_find_hole_fallback Alex Bennée 2020-07-24 6:45 ` [PATCH v3 09/16] tests/docker: fix update command due to python3 str/bytes distinction Alex Bennée 2020-07-24 6:45 ` [PATCH v3 10/16] tests/docker: fix binfmt_misc image building Alex Bennée 2020-07-24 6:45 ` [PATCH v3 11/16] tests/docker: add support for DEB_KEYRING Alex Bennée 2020-07-24 6:45 ` [PATCH v3 12/16] linux-user: fix clock_nanosleep() Alex Bennée 2020-07-24 18:24 ` Richard Henderson 2020-07-24 6:45 ` [PATCH v3 13/16] linux-user, ppc: fix clock_nanosleep() for linux-user-ppc Alex Bennée 2020-07-24 18:24 ` [PATCH v3 13/16] linux-user,ppc: " Richard Henderson 2020-07-24 6:45 ` [PATCH v3 14/16] python/qemu: Cleanup changes to ConsoleSocket Alex Bennée 2020-07-24 7:25 ` Philippe Mathieu-Daudé 2020-07-24 6:45 ` [PATCH v3 15/16] python/qemu: Change ConsoleSocket to optionally drain socket Alex Bennée 2020-07-24 6:45 ` [PATCH v3 16/16] tests/vm: add shutdown timeout in basevm.py Alex Bennée
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.