All of lore.kernel.org
 help / color / mirror / Atom feed
* [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4
@ 2021-08-14  6:09 Thomas Huth
  2021-08-14  6:09 ` [PULL 01/11] gitlab: exclude sparc-softmmu and riscv32-softmmu from cross builds Thomas Huth
                   ` (11 more replies)
  0 siblings, 12 replies; 21+ messages in thread
From: Thomas Huth @ 2021-08-14  6:09 UTC (permalink / raw)
  To: qemu-devel, peter.maydell

 Hi Peter,

in case we're going to have an -rc4, here's a pull request that contains
the fixes for getting the gitlab-CI green again. I also added some doc
updates since they should be completely riskless. But if we won't have an
rc4 due to other reasons, this pull request here certainly also does not
justify another RC, so please ignore this PR in that case.

The following changes since commit 703e8cd6189cf699c8d5c094bc68b5f3afa6ad71:

  Update version for v6.1.0-rc3 release (2021-08-10 19:08:09 +0100)

are available in the Git repository at:

  https://gitlab.com/thuth/qemu.git tags/pull-request-2021-08-11

for you to fetch changes up to 36b508993c4dcc6b3ef4b5c00e293ee9560926ee:

  docs/about/removed-features: Document removed machines from older QEMU versions (2021-08-11 15:39:09 +0200)

CI run can be seen here:

 https://gitlab.com/thuth/qemu/-/pipelines/351602605

----------------------------------------------------------------
* Fixes for the gitlab-CI (fix the hanging  build-oss-fuzz pipeline)
* Add documentation about features that have been removed in older versions

----------------------------------------------------------------
Alexander Bulekov (1):
      fuzz: avoid building twice, when running on gitlab

Daniel P. Berrangé (2):
      gitlab: exclude sparc-softmmu and riscv32-softmmu from cross builds
      gitlab: skip many more targets in windows cross builds

Thomas Huth (8):
      storage-daemon: Add missing build dependency to the vhost-user-blk-test
      tests/qtest/vhost-user-blk-test: Check whether qemu-storage-daemon is available
      docs/about/removed-features: Document removed CLI options from QEMU v2.12
      docs/about/removed-features: Document removed CLI options from QEMU v3.0
      docs/about/removed-features: Document removed CLI options from QEMU v3.1
      docs/about/removed-features: Document removed HMP commands from QEMU v2.12
      docs/about/removed-features: Document removed devices from older QEMU versions
      docs/about/removed-features: Document removed machines from older QEMU versions

 .gitlab-ci.d/crossbuild-template.yml |   3 +-
 .gitlab-ci.d/crossbuilds.yml         |   4 +
 docs/about/removed-features.rst      | 183 ++++++++++++++++++++++++++++++++++-
 scripts/oss-fuzz/build.sh            |  24 ++---
 storage-daemon/meson.build           |   8 +-
 tests/qtest/meson.build              |   7 +-
 tests/qtest/vhost-user-blk-test.c    |   8 ++
 7 files changed, 215 insertions(+), 22 deletions(-)



^ permalink raw reply	[flat|nested] 21+ messages in thread

* [PULL 01/11] gitlab: exclude sparc-softmmu and riscv32-softmmu from cross builds
  2021-08-14  6:09 [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4 Thomas Huth
@ 2021-08-14  6:09 ` Thomas Huth
  2021-08-14  6:09 ` [PULL 02/11] gitlab: skip many more targets in windows " Thomas Huth
                   ` (10 subsequent siblings)
  11 siblings, 0 replies; 21+ messages in thread
From: Thomas Huth @ 2021-08-14  6:09 UTC (permalink / raw)
  To: qemu-devel, peter.maydell

From: Daniel P. Berrangé <berrange@redhat.com>

We need to cut down compile time by excluding more targets. Both these
targets still have their 64-bit variant enabled, so the loss of coverage
is mitigated to some degree.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210810140653.3969823-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/crossbuild-template.yml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
index 7d3ad00a1e..cfb576b54c 100644
--- a/.gitlab-ci.d/crossbuild-template.yml
+++ b/.gitlab-ci.d/crossbuild-template.yml
@@ -9,7 +9,8 @@
       ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
         --disable-user --target-list-exclude="arm-softmmu cris-softmmu
           i386-softmmu microblaze-softmmu mips-softmmu mipsel-softmmu
-          mips64-softmmu ppc-softmmu sh4-softmmu xtensa-softmmu"
+          mips64-softmmu ppc-softmmu riscv32-softmmu sh4-softmmu
+          sparc-softmmu xtensa-softmmu"
     - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
     - if grep -q "EXESUF=.exe" config-host.mak;
       then make installer;
-- 
2.27.0



^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PULL 02/11] gitlab: skip many more targets in windows cross builds
  2021-08-14  6:09 [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4 Thomas Huth
  2021-08-14  6:09 ` [PULL 01/11] gitlab: exclude sparc-softmmu and riscv32-softmmu from cross builds Thomas Huth
@ 2021-08-14  6:09 ` Thomas Huth
  2021-08-15 15:47   ` Peter Maydell
  2021-08-14  6:09 ` [PULL 03/11] storage-daemon: Add missing build dependency to the vhost-user-blk-test Thomas Huth
                   ` (9 subsequent siblings)
  11 siblings, 1 reply; 21+ messages in thread
From: Thomas Huth @ 2021-08-14  6:09 UTC (permalink / raw)
  To: qemu-devel, peter.maydell

From: Daniel P. Berrangé <berrange@redhat.com>

The windows cross builds still take way too long in gitlab CI, so need
more targets to be skipped. We don't want to hurt coverage of other
cross builds more though, so we let jobs fine tune with a new env
variale $CROSS_SKIP_TARGETS.

We take the set of targets that are considered relatively niche or
very old architectures, and skip approx half of them in win32 builds
and the other half of them in win64.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210810140653.3969823-3-berrange@redhat.com>
[thuth: Swapped the "CROSS_SKIP_TARGETS:" lines as suggested by philmd]
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/crossbuild-template.yml | 2 +-
 .gitlab-ci.d/crossbuilds.yml         | 4 ++++
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
index cfb576b54c..10d22dcf6c 100644
--- a/.gitlab-ci.d/crossbuild-template.yml
+++ b/.gitlab-ci.d/crossbuild-template.yml
@@ -10,7 +10,7 @@
         --disable-user --target-list-exclude="arm-softmmu cris-softmmu
           i386-softmmu microblaze-softmmu mips-softmmu mipsel-softmmu
           mips64-softmmu ppc-softmmu riscv32-softmmu sh4-softmmu
-          sparc-softmmu xtensa-softmmu"
+          sparc-softmmu xtensa-softmmu $CROSS_SKIP_TARGETS"
     - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
     - if grep -q "EXESUF=.exe" config-host.mak;
       then make installer;
diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
index 4ff3aa3cfc..f10168db2e 100644
--- a/.gitlab-ci.d/crossbuilds.yml
+++ b/.gitlab-ci.d/crossbuilds.yml
@@ -160,6 +160,8 @@ cross-win32-system:
     job: win32-fedora-cross-container
   variables:
     IMAGE: fedora-win32-cross
+    CROSS_SKIP_TARGETS: alpha-softmmu avr-softmmu hppa-softmmu m68k-softmmu
+                        microblazeel-softmmu mips64el-softmmu nios2-softmmu
   artifacts:
     paths:
       - build/qemu-setup*.exe
@@ -170,6 +172,8 @@ cross-win64-system:
     job: win64-fedora-cross-container
   variables:
     IMAGE: fedora-win64-cross
+    CROSS_SKIP_TARGETS: or1k-softmmu rx-softmmu sh4eb-softmmu sparc64-softmmu
+                        tricore-softmmu xtensaeb-softmmu
   artifacts:
     paths:
       - build/qemu-setup*.exe
-- 
2.27.0



^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PULL 03/11] storage-daemon: Add missing build dependency to the vhost-user-blk-test
  2021-08-14  6:09 [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4 Thomas Huth
  2021-08-14  6:09 ` [PULL 01/11] gitlab: exclude sparc-softmmu and riscv32-softmmu from cross builds Thomas Huth
  2021-08-14  6:09 ` [PULL 02/11] gitlab: skip many more targets in windows " Thomas Huth
@ 2021-08-14  6:09 ` Thomas Huth
  2021-08-14  6:09 ` [PULL 04/11] tests/qtest/vhost-user-blk-test: Check whether qemu-storage-daemon is available Thomas Huth
                   ` (8 subsequent siblings)
  11 siblings, 0 replies; 21+ messages in thread
From: Thomas Huth @ 2021-08-14  6:09 UTC (permalink / raw)
  To: qemu-devel, peter.maydell

vhost-user-blk-test needs the qemu-storage-daemon, otherwise it
currently hangs. So make sure that we build the daemon before running
the tests.

Message-Id: <20210811094705.131314-1-thuth@redhat.com>
Tested-by: Alexander Bulekov <alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 storage-daemon/meson.build | 8 ++++----
 tests/qtest/meson.build    | 7 +++++--
 2 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/storage-daemon/meson.build b/storage-daemon/meson.build
index 68852f3d25..49c9d2eac9 100644
--- a/storage-daemon/meson.build
+++ b/storage-daemon/meson.build
@@ -6,8 +6,8 @@ subdir('qapi')
 
 if have_tools
   qsd_ss = qsd_ss.apply(config_host, strict: false)
-  executable('qemu-storage-daemon',
-             qsd_ss.sources(),
-             dependencies: qsd_ss.dependencies(),
-             install: true)
+  qsd = executable('qemu-storage-daemon',
+                   qsd_ss.sources(),
+                   dependencies: qsd_ss.dependencies(),
+                   install: true)
 endif
diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
index e22a0792c5..2bc3efd49f 100644
--- a/tests/qtest/meson.build
+++ b/tests/qtest/meson.build
@@ -276,8 +276,11 @@ foreach dir : target_dirs
   endif
   qtest_env.set('G_TEST_DBUS_DAEMON', meson.source_root() / 'tests/dbus-vmstate-daemon.sh')
   qtest_env.set('QTEST_QEMU_BINARY', './qemu-system-' + target_base)
-  qtest_env.set('QTEST_QEMU_STORAGE_DAEMON_BINARY', './storage-daemon/qemu-storage-daemon')
-  
+  if have_tools and have_vhost_user_blk_server
+    qtest_env.set('QTEST_QEMU_STORAGE_DAEMON_BINARY', './storage-daemon/qemu-storage-daemon')
+    test_deps += [qsd]
+  endif
+
   foreach test : target_qtests
     # Executables are shared across targets, declare them only the first time we
     # encounter them
-- 
2.27.0



^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PULL 04/11] tests/qtest/vhost-user-blk-test: Check whether qemu-storage-daemon is available
  2021-08-14  6:09 [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4 Thomas Huth
                   ` (2 preceding siblings ...)
  2021-08-14  6:09 ` [PULL 03/11] storage-daemon: Add missing build dependency to the vhost-user-blk-test Thomas Huth
@ 2021-08-14  6:09 ` Thomas Huth
  2021-08-14  6:09 ` [PULL 05/11] fuzz: avoid building twice, when running on gitlab Thomas Huth
                   ` (7 subsequent siblings)
  11 siblings, 0 replies; 21+ messages in thread
From: Thomas Huth @ 2021-08-14  6:09 UTC (permalink / raw)
  To: qemu-devel, peter.maydell

The vhost-user-blk-test currently hangs if QTEST_QEMU_STORAGE_DAEMON_BINARY
points to a non-existing binary. Let's improve this situation by checking
for the availability of the binary first, so we can fail gracefully if
it is not accessible.

Message-Id: <20210811095949.133462-1-thuth@redhat.com>
Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Tested-by: Alexander Bulekov <alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qtest/vhost-user-blk-test.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/tests/qtest/vhost-user-blk-test.c b/tests/qtest/vhost-user-blk-test.c
index 8796c74ca4..6f108a1b62 100644
--- a/tests/qtest/vhost-user-blk-test.c
+++ b/tests/qtest/vhost-user-blk-test.c
@@ -789,6 +789,14 @@ static const char *qtest_qemu_storage_daemon_binary(void)
         exit(0);
     }
 
+    /* If we've got a path to the binary, check whether we can access it */
+    if (strchr(qemu_storage_daemon_bin, '/') &&
+        access(qemu_storage_daemon_bin, X_OK) != 0) {
+        fprintf(stderr, "ERROR: '%s' is not accessible\n",
+                qemu_storage_daemon_bin);
+        exit(1);
+    }
+
     return qemu_storage_daemon_bin;
 }
 
-- 
2.27.0



^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PULL 05/11] fuzz: avoid building twice, when running on gitlab
  2021-08-14  6:09 [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4 Thomas Huth
                   ` (3 preceding siblings ...)
  2021-08-14  6:09 ` [PULL 04/11] tests/qtest/vhost-user-blk-test: Check whether qemu-storage-daemon is available Thomas Huth
@ 2021-08-14  6:09 ` Thomas Huth
  2021-08-14  6:09 ` [PULL 06/11] docs/about/removed-features: Document removed CLI options from QEMU v2.12 Thomas Huth
                   ` (6 subsequent siblings)
  11 siblings, 0 replies; 21+ messages in thread
From: Thomas Huth @ 2021-08-14  6:09 UTC (permalink / raw)
  To: qemu-devel, peter.maydell

From: Alexander Bulekov <alxndr@bu.edu>

On oss-fuzz, we build twice, to put together a build that is portable to
the runner containers. On gitlab ci, this is wasteful and contributes to
timeouts on the build-oss-fuzz job. Avoid building twice on gitlab, at
the remote cost of potentially missing some cases that break oss-fuzz
builds.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Message-Id: <20210809111621.54454-1-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 scripts/oss-fuzz/build.sh | 24 +++++++++++++-----------
 1 file changed, 13 insertions(+), 11 deletions(-)

diff --git a/scripts/oss-fuzz/build.sh b/scripts/oss-fuzz/build.sh
index c1af43fded..98b56e0521 100755
--- a/scripts/oss-fuzz/build.sh
+++ b/scripts/oss-fuzz/build.sh
@@ -73,17 +73,19 @@ if ! make "-j$(nproc)" qemu-fuzz-i386; then
           "\nFor example: CC=clang CXX=clang++ $0"
 fi
 
-for i in $(ldd ./qemu-fuzz-i386 | cut -f3 -d' '); do
-    cp "$i" "$DEST_DIR/lib/"
-done
-rm qemu-fuzz-i386
-
-# Build a second time to build the final binary with correct rpath
-../configure --disable-werror --cc="$CC" --cxx="$CXX" --enable-fuzzing \
-    --prefix="$DEST_DIR" --bindir="$DEST_DIR" --datadir="$DEST_DIR/data/" \
-    --extra-cflags="$EXTRA_CFLAGS" --extra-ldflags="-Wl,-rpath,\$ORIGIN/lib" \
-    --target-list="i386-softmmu"
-make "-j$(nproc)" qemu-fuzz-i386 V=1
+if [ "$GITLAB_CI" != "true" ]; then
+    for i in $(ldd ./qemu-fuzz-i386 | cut -f3 -d' '); do
+        cp "$i" "$DEST_DIR/lib/"
+    done
+    rm qemu-fuzz-i386
+
+    # Build a second time to build the final binary with correct rpath
+    ../configure --disable-werror --cc="$CC" --cxx="$CXX" --enable-fuzzing \
+        --prefix="$DEST_DIR" --bindir="$DEST_DIR" --datadir="$DEST_DIR/data/" \
+        --extra-cflags="$EXTRA_CFLAGS" --extra-ldflags="-Wl,-rpath,\$ORIGIN/lib" \
+        --target-list="i386-softmmu"
+    make "-j$(nproc)" qemu-fuzz-i386 V=1
+fi
 
 # Copy over the datadir
 cp  -r ../pc-bios/ "$DEST_DIR/pc-bios"
-- 
2.27.0



^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PULL 06/11] docs/about/removed-features: Document removed CLI options from QEMU v2.12
  2021-08-14  6:09 [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4 Thomas Huth
                   ` (4 preceding siblings ...)
  2021-08-14  6:09 ` [PULL 05/11] fuzz: avoid building twice, when running on gitlab Thomas Huth
@ 2021-08-14  6:09 ` Thomas Huth
  2021-08-14  6:09 ` [PULL 07/11] docs/about/removed-features: Document removed CLI options from QEMU v3.0 Thomas Huth
                   ` (5 subsequent siblings)
  11 siblings, 0 replies; 21+ messages in thread
From: Thomas Huth @ 2021-08-14  6:09 UTC (permalink / raw)
  To: qemu-devel, peter.maydell

These CLI options had been removed/replaced in QEMU v2.12. Still, some
people might want to update from older versions to the recent QEMU version,
so we should give some recommendations for the replacements in our
documentation.

Message-Id: <20210811084103.74832-2-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 docs/about/removed-features.rst | 39 +++++++++++++++++++++++++++++++--
 1 file changed, 37 insertions(+), 2 deletions(-)

diff --git a/docs/about/removed-features.rst b/docs/about/removed-features.rst
index 07d597847c..c18af3c76f 100644
--- a/docs/about/removed-features.rst
+++ b/docs/about/removed-features.rst
@@ -9,8 +9,43 @@ trouble after a recent upgrade.
 System emulator command line arguments
 --------------------------------------
 
-``-net ...,name=``\ *name* (removed in 5.1)
-'''''''''''''''''''''''''''''''''''''''''''
+``-hdachs`` (removed in 2.12)
+'''''''''''''''''''''''''''''
+
+The geometry defined by ``-hdachs c,h,s,t`` should now be specified via
+``-device ide-hd,drive=dr,cyls=c,heads=h,secs=s,bios-chs-trans=t``
+(together with ``-drive if=none,id=dr,...``).
+
+``-net channel`` (removed in 2.12)
+''''''''''''''''''''''''''''''''''
+
+This option has been replaced by ``-net user,guestfwd=...``.
+
+``-net dump`` (removed in 2.12)
+'''''''''''''''''''''''''''''''
+
+``-net dump[,vlan=n][,file=filename][,len=maxlen]`` has been replaced by
+``-object filter-dump,id=id,netdev=dev[,file=filename][,maxlen=maxlen]``.
+Note that the new syntax works with netdev IDs instead of the old "vlan" hubs.
+
+``-no-kvm-pit`` (removed in 2.12)
+'''''''''''''''''''''''''''''''''
+
+This was just a dummy option that has been ignored, since the in-kernel PIT
+cannot be disabled separately from the irqchip anymore. A similar effect
+(which also disables the KVM IOAPIC) can be obtained with
+``-M kernel_irqchip=split``.
+
+``-tdf`` (removed in 2.12)
+''''''''''''''''''''''''''
+
+There is no replacement, the ``-tdf`` option has just been ignored since the
+behaviour that could be changed by this option in qemu-kvm is now the default
+when using the KVM PIT. It still can be requested explicitly using
+``-global kvm-pit.lost_tick_policy=delay``.
+
+``-net ...,name=...`` (removed in 5.1)
+''''''''''''''''''''''''''''''''''''''
 
 The ``name`` parameter of the ``-net`` option was a synonym
 for the ``id`` parameter, which should now be used instead.
-- 
2.27.0



^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PULL 07/11] docs/about/removed-features: Document removed CLI options from QEMU v3.0
  2021-08-14  6:09 [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4 Thomas Huth
                   ` (5 preceding siblings ...)
  2021-08-14  6:09 ` [PULL 06/11] docs/about/removed-features: Document removed CLI options from QEMU v2.12 Thomas Huth
@ 2021-08-14  6:09 ` Thomas Huth
  2021-08-14  6:09 ` [PULL 08/11] docs/about/removed-features: Document removed CLI options from QEMU v3.1 Thomas Huth
                   ` (4 subsequent siblings)
  11 siblings, 0 replies; 21+ messages in thread
From: Thomas Huth @ 2021-08-14  6:09 UTC (permalink / raw)
  To: qemu-devel, peter.maydell

These CLI options had been removed/replaced in QEMU v3.0. Still, some
people might want to update from older versions to the recent QEMU version,
so we should give some recommendations for the replacements in our
documentation.

Message-Id: <20210811084103.74832-3-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 docs/about/removed-features.rst | 35 +++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/docs/about/removed-features.rst b/docs/about/removed-features.rst
index c18af3c76f..c4b702968e 100644
--- a/docs/about/removed-features.rst
+++ b/docs/about/removed-features.rst
@@ -44,6 +44,41 @@ behaviour that could be changed by this option in qemu-kvm is now the default
 when using the KVM PIT. It still can be requested explicitly using
 ``-global kvm-pit.lost_tick_policy=delay``.
 
+``-drive secs=s``, ``-drive heads=h`` & ``-drive cyls=c`` (removed in 3.0)
+''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
+
+The drive geometry should now be specified via
+``-device ...,drive=dr,cyls=c,heads=h,secs=s`` (together with
+``-drive if=none,id=dr,...``).
+
+``-drive serial=``, ``-drive trans=`` & ``-drive addr=`` (removed in 3.0)
+'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
+
+Use ``-device ...,drive=dr,serial=r,bios-chs-trans=t,addr=a`` instead
+(together with ``-drive if=none,id=dr,...``).
+
+``-net ...,vlan=x`` (removed in 3.0)
+''''''''''''''''''''''''''''''''''''
+
+The term "vlan" was very confusing for most users in this context (it's about
+specifying a hub ID, not about IEEE 802.1Q or something similar), so this
+has been removed. To connect one NIC frontend with a network backend, either
+use ``-nic ...`` (e.g. for on-board NICs) or use ``-netdev ...,id=n`` together
+with ``-device ...,netdev=n`` (for full control over pluggable NICs). To
+connect multiple NICs or network backends via a hub device (which is what
+vlan did), use ``-nic hubport,hubid=x,...`` or
+``-netdev hubport,id=n,hubid=x,...`` (with ``-device ...,netdev=n``) instead.
+
+``-no-kvm-irqchip`` (removed in 3.0)
+''''''''''''''''''''''''''''''''''''
+
+Use ``-machine kernel_irqchip=off`` instead.
+
+``-no-kvm-pit-reinjection`` (removed in 3.0)
+''''''''''''''''''''''''''''''''''''''''''''
+
+Use ``-global kvm-pit.lost_tick_policy=discard`` instead.
+
 ``-net ...,name=...`` (removed in 5.1)
 ''''''''''''''''''''''''''''''''''''''
 
-- 
2.27.0



^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PULL 08/11] docs/about/removed-features: Document removed CLI options from QEMU v3.1
  2021-08-14  6:09 [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4 Thomas Huth
                   ` (6 preceding siblings ...)
  2021-08-14  6:09 ` [PULL 07/11] docs/about/removed-features: Document removed CLI options from QEMU v3.0 Thomas Huth
@ 2021-08-14  6:09 ` Thomas Huth
  2021-08-14  6:09 ` [PULL 09/11] docs/about/removed-features: Document removed HMP commands from QEMU v2.12 Thomas Huth
                   ` (3 subsequent siblings)
  11 siblings, 0 replies; 21+ messages in thread
From: Thomas Huth @ 2021-08-14  6:09 UTC (permalink / raw)
  To: qemu-devel, peter.maydell

These CLI options had been removed/replaced in QEMU v3.1. Still, some
people might want to update from older versions to the recent QEMU version,
so we should give some recommendations for the replacements in our
documentation.

Message-Id: <20210811084103.74832-4-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 docs/about/removed-features.rst | 67 +++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)

diff --git a/docs/about/removed-features.rst b/docs/about/removed-features.rst
index c4b702968e..40d2cc4ffa 100644
--- a/docs/about/removed-features.rst
+++ b/docs/about/removed-features.rst
@@ -79,6 +79,73 @@ Use ``-machine kernel_irqchip=off`` instead.
 
 Use ``-global kvm-pit.lost_tick_policy=discard`` instead.
 
+``-balloon`` (removed in 3.1)
+'''''''''''''''''''''''''''''
+
+The ``-balloon virtio`` option has been replaced by ``-device virtio-balloon``.
+The ``-balloon none`` option was a no-op and has no replacement.
+
+``-bootp`` (removed in 3.1)
+'''''''''''''''''''''''''''
+
+The ``-bootp /some/file`` argument is replaced by either
+``-netdev user,id=x,bootp=/some/file`` (for pluggable NICs, accompanied with
+``-device ...,netdev=x``), or ``-nic user,bootp=/some/file`` (for on-board NICs).
+The new syntax allows different settings to be provided per NIC.
+
+``-redir`` (removed in 3.1)
+'''''''''''''''''''''''''''
+
+The ``-redir [tcp|udp]:hostport:[guestaddr]:guestport`` option is replaced
+by either ``-netdev
+user,id=x,hostfwd=[tcp|udp]:[hostaddr]:hostport-[guestaddr]:guestport``
+(for pluggable NICs, accompanied with ``-device ...,netdev=x``) or by the option
+``-nic user,hostfwd=[tcp|udp]:[hostaddr]:hostport-[guestaddr]:guestport``
+(for on-board NICs). The new syntax allows different settings to be provided
+per NIC.
+
+``-smb`` (removed in 3.1)
+'''''''''''''''''''''''''
+
+The ``-smb /some/dir`` argument is replaced by either
+``-netdev user,id=x,smb=/some/dir`` (for pluggable NICs, accompanied with
+``-device ...,netdev=x``), or ``-nic user,smb=/some/dir`` (for on-board NICs).
+The new syntax allows different settings to be provided per NIC.
+
+``-tftp`` (removed in 3.1)
+''''''''''''''''''''''''''
+
+The ``-tftp /some/dir`` argument is replaced by either
+``-netdev user,id=x,tftp=/some/dir`` (for pluggable NICs, accompanied with
+``-device ...,netdev=x``), or ``-nic user,tftp=/some/dir`` (for embedded NICs).
+The new syntax allows different settings to be provided per NIC.
+
+``-localtime`` (removed in 3.1)
+'''''''''''''''''''''''''''''''
+
+Replaced by ``-rtc base=localtime``.
+
+``-nodefconfig`` (removed in 3.1)
+'''''''''''''''''''''''''''''''''
+
+Use ``-no-user-config`` instead.
+
+``-rtc-td-hack`` (removed in 3.1)
+'''''''''''''''''''''''''''''''''
+
+Use ``-rtc driftfix=slew`` instead.
+
+``-startdate`` (removed in 3.1)
+'''''''''''''''''''''''''''''''
+
+Replaced by ``-rtc base=date``.
+
+``-vnc ...,tls=...``, ``-vnc ...,x509=...`` & ``-vnc ...,x509verify=...``
+'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
+
+The "tls-creds" option should be used instead to point to a "tls-creds-x509"
+object created using "-object".
+
 ``-net ...,name=...`` (removed in 5.1)
 ''''''''''''''''''''''''''''''''''''''
 
-- 
2.27.0



^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PULL 09/11] docs/about/removed-features: Document removed HMP commands from QEMU v2.12
  2021-08-14  6:09 [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4 Thomas Huth
                   ` (7 preceding siblings ...)
  2021-08-14  6:09 ` [PULL 08/11] docs/about/removed-features: Document removed CLI options from QEMU v3.1 Thomas Huth
@ 2021-08-14  6:09 ` Thomas Huth
  2021-08-14  6:09 ` [PULL 10/11] docs/about/removed-features: Document removed devices from older QEMU versions Thomas Huth
                   ` (2 subsequent siblings)
  11 siblings, 0 replies; 21+ messages in thread
From: Thomas Huth @ 2021-08-14  6:09 UTC (permalink / raw)
  To: qemu-devel, peter.maydell

These HMP commands had been removed/replaced in QEMU v2.12. Still, some
people might want to update from older versions to the recent QEMU version,
so we should give some recommendations for the replacements in our
documentation.

Message-Id: <20210811084103.74832-5-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 docs/about/removed-features.rst | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/docs/about/removed-features.rst b/docs/about/removed-features.rst
index 40d2cc4ffa..8bf3ebecab 100644
--- a/docs/about/removed-features.rst
+++ b/docs/about/removed-features.rst
@@ -356,6 +356,17 @@ Specify the properties for the object as top-level arguments instead.
 Human Monitor Protocol (HMP) commands
 -------------------------------------
 
+``usb_add`` and ``usb_remove`` (removed in 2.12)
+''''''''''''''''''''''''''''''''''''''''''''''''
+
+Replaced by ``device_add`` and ``device_del`` (use ``device_add help`` for a
+list of available devices).
+
+``host_net_add`` and ``host_net_remove`` (removed in 2.12)
+''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
+
+Replaced by ``netdev_add`` and ``netdev_del``.
+
 The ``hub_id`` parameter of ``hostfwd_add`` / ``hostfwd_remove`` (removed in 5.0)
 '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
 
-- 
2.27.0



^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PULL 10/11] docs/about/removed-features: Document removed devices from older QEMU versions
  2021-08-14  6:09 [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4 Thomas Huth
                   ` (8 preceding siblings ...)
  2021-08-14  6:09 ` [PULL 09/11] docs/about/removed-features: Document removed HMP commands from QEMU v2.12 Thomas Huth
@ 2021-08-14  6:09 ` Thomas Huth
  2021-08-14  6:09 ` [PULL 11/11] docs/about/removed-features: Document removed machines " Thomas Huth
  2021-08-16  9:01 ` [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4 Peter Maydell
  11 siblings, 0 replies; 21+ messages in thread
From: Thomas Huth @ 2021-08-14  6:09 UTC (permalink / raw)
  To: qemu-devel, peter.maydell

These devices had been removed/replaced in QEMU v2.12 and v4.0.

Message-Id: <20210811084103.74832-6-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 docs/about/removed-features.rst | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/docs/about/removed-features.rst b/docs/about/removed-features.rst
index 8bf3ebecab..0c860be62d 100644
--- a/docs/about/removed-features.rst
+++ b/docs/about/removed-features.rst
@@ -513,6 +513,17 @@ running the old binaries, you can use older versions of QEMU.
 System emulator devices
 -----------------------
 
+``spapr-pci-vfio-host-bridge`` (removed in 2.12)
+'''''''''''''''''''''''''''''''''''''''''''''''''
+
+The ``spapr-pci-vfio-host-bridge`` device type has been replaced by the
+``spapr-pci-host-bridge`` device type.
+
+``ivshmem`` (removed in 4.0)
+''''''''''''''''''''''''''''
+
+Replaced by either the ``ivshmem-plain`` or ``ivshmem-doorbell``.
+
 ``ide-drive`` (removed in 6.0)
 ''''''''''''''''''''''''''''''
 
-- 
2.27.0



^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PULL 11/11] docs/about/removed-features: Document removed machines from older QEMU versions
  2021-08-14  6:09 [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4 Thomas Huth
                   ` (9 preceding siblings ...)
  2021-08-14  6:09 ` [PULL 10/11] docs/about/removed-features: Document removed devices from older QEMU versions Thomas Huth
@ 2021-08-14  6:09 ` Thomas Huth
  2021-08-16  9:01 ` [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4 Peter Maydell
  11 siblings, 0 replies; 21+ messages in thread
From: Thomas Huth @ 2021-08-14  6:09 UTC (permalink / raw)
  To: qemu-devel, peter.maydell

These machines had been removed in the QEMU v2.6 up to 4.0 time frame.

Message-Id: <20210811084103.74832-7-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 docs/about/removed-features.rst | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/docs/about/removed-features.rst b/docs/about/removed-features.rst
index 0c860be62d..cbfa1a8e31 100644
--- a/docs/about/removed-features.rst
+++ b/docs/about/removed-features.rst
@@ -473,6 +473,22 @@ Removed without replacement.
 System emulator machines
 ------------------------
 
+``s390-virtio`` (removed in 2.6)
+''''''''''''''''''''''''''''''''
+
+Use the ``s390-ccw-virtio`` machine instead.
+
+The m68k ``dummy`` machine (removed in 2.9)
+'''''''''''''''''''''''''''''''''''''''''''
+
+Use the ``none`` machine with the ``loader`` device instead.
+
+``xlnx-ep108`` (removed in 3.0)
+'''''''''''''''''''''''''''''''
+
+The EP108 was an early access development board that is no longer used.
+Use the ``xlnx-zcu102`` machine instead.
+
 ``spike_v1.9.1`` and ``spike_v1.10`` (removed in 5.1)
 '''''''''''''''''''''''''''''''''''''''''''''''''''''
 
@@ -491,8 +507,8 @@ mips ``fulong2e`` machine alias (removed in 6.0)
 
 This machine has been renamed ``fuloong2e``.
 
-``pc-1.0``, ``pc-1.1``, ``pc-1.2`` and ``pc-1.3`` (removed in 6.0)
-''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
+``pc-0.10`` up to ``pc-1.3`` (removed in 4.0 up to 6.0)
+'''''''''''''''''''''''''''''''''''''''''''''''''''''''
 
 These machine types were very old and likely could not be used for live
 migration from old QEMU versions anymore. Use a newer machine type instead.
-- 
2.27.0



^ permalink raw reply related	[flat|nested] 21+ messages in thread

* Re: [PULL 02/11] gitlab: skip many more targets in windows cross builds
  2021-08-14  6:09 ` [PULL 02/11] gitlab: skip many more targets in windows " Thomas Huth
@ 2021-08-15 15:47   ` Peter Maydell
  0 siblings, 0 replies; 21+ messages in thread
From: Peter Maydell @ 2021-08-15 15:47 UTC (permalink / raw)
  To: Thomas Huth; +Cc: QEMU Developers

On Sat, 14 Aug 2021 at 07:10, Thomas Huth <thuth@redhat.com> wrote:
>
> From: Daniel P. Berrangé <berrange@redhat.com>
>
> The windows cross builds still take way too long in gitlab CI, so need
> more targets to be skipped. We don't want to hurt coverage of other
> cross builds more though, so we let jobs fine tune with a new env
> variale $CROSS_SKIP_TARGETS.
>
> We take the set of targets that are considered relatively niche or
> very old architectures, and skip approx half of them in win32 builds
> and the other half of them in win64.

I think this pullreq is an improvement on the current situation,
so I'm planning to apply it since we need an rc4 anyway, but
after we reopen for 6.2 I think we might want to reconsider
this one. The Windows builds are pretty good at flagging up
nonportabilities because the host OS is so different, so
I'd rather we covered the whole codebase with both win32 and
win64 rather than doing only half with each.

thanks
-- PMM


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4
  2021-08-14  6:09 [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4 Thomas Huth
                   ` (10 preceding siblings ...)
  2021-08-14  6:09 ` [PULL 11/11] docs/about/removed-features: Document removed machines " Thomas Huth
@ 2021-08-16  9:01 ` Peter Maydell
  2021-08-16 10:22   ` Alexander Bulekov
  11 siblings, 1 reply; 21+ messages in thread
From: Peter Maydell @ 2021-08-16  9:01 UTC (permalink / raw)
  To: Thomas Huth; +Cc: QEMU Developers

On Sat, 14 Aug 2021 at 07:10, Thomas Huth <thuth@redhat.com> wrote:
>
>  Hi Peter,
>
> in case we're going to have an -rc4, here's a pull request that contains
> the fixes for getting the gitlab-CI green again. I also added some doc
> updates since they should be completely riskless. But if we won't have an
> rc4 due to other reasons, this pull request here certainly also does not
> justify another RC, so please ignore this PR in that case.
>
> The following changes since commit 703e8cd6189cf699c8d5c094bc68b5f3afa6ad71:
>
>   Update version for v6.1.0-rc3 release (2021-08-10 19:08:09 +0100)
>
> are available in the Git repository at:
>
>   https://gitlab.com/thuth/qemu.git tags/pull-request-2021-08-11
>
> for you to fetch changes up to 36b508993c4dcc6b3ef4b5c00e293ee9560926ee:
>
>   docs/about/removed-features: Document removed machines from older QEMU versions (2021-08-11 15:39:09 +0200)
>
> CI run can be seen here:
>
>  https://gitlab.com/thuth/qemu/-/pipelines/351602605
>
> ----------------------------------------------------------------
> * Fixes for the gitlab-CI (fix the hanging  build-oss-fuzz pipeline)
> * Add documentation about features that have been removed in older versions
>


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/6.1
for any user-visible changes.

-- PMM


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4
  2021-08-16  9:01 ` [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4 Peter Maydell
@ 2021-08-16 10:22   ` Alexander Bulekov
  2021-08-16 11:11     ` Daniel P. Berrangé
  0 siblings, 1 reply; 21+ messages in thread
From: Alexander Bulekov @ 2021-08-16 10:22 UTC (permalink / raw)
  To: Peter Maydell; +Cc: Thomas Huth, QEMU Developers

On 210816 1001, Peter Maydell wrote:
> On Sat, 14 Aug 2021 at 07:10, Thomas Huth <thuth@redhat.com> wrote:
> >
> >  Hi Peter,
> >
> > in case we're going to have an -rc4, here's a pull request that contains
> > the fixes for getting the gitlab-CI green again. I also added some doc
> > updates since they should be completely riskless. But if we won't have an
> > rc4 due to other reasons, this pull request here certainly also does not
> > justify another RC, so please ignore this PR in that case.
> >
> > The following changes since commit 703e8cd6189cf699c8d5c094bc68b5f3afa6ad71:
> >
> >   Update version for v6.1.0-rc3 release (2021-08-10 19:08:09 +0100)
> >
> > are available in the Git repository at:
> >
> >   https://gitlab.com/thuth/qemu.git tags/pull-request-2021-08-11
> >
> > for you to fetch changes up to 36b508993c4dcc6b3ef4b5c00e293ee9560926ee:
> >
> >   docs/about/removed-features: Document removed machines from older QEMU versions (2021-08-11 15:39:09 +0200)
> >
> > CI run can be seen here:
> >
> >  https://gitlab.com/thuth/qemu/-/pipelines/351602605
> >
> > ----------------------------------------------------------------
> > * Fixes for the gitlab-CI (fix the hanging  build-oss-fuzz pipeline)
> > * Add documentation about features that have been removed in older versions
> >
> 
> 
> Applied, thanks.
> 
> Please update the changelog at https://wiki.qemu.org/ChangeLog/6.1
> for any user-visible changes.

https://gitlab.com/qemu-project/qemu/-/jobs/1505950978
Looks like build-oss-fuzz is still timing out, even without the issue
in the vhost-usr-blk test. At this point the job should essentially just
build + test qemu-system-i386 with some extra time spent on linking
the fuzzer and briefly running through all the fuzzer configs. Maybe the
only way to make this work is to split the job into a build + test
stage?

> 
> -- PMM
> 


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4
  2021-08-16 10:22   ` Alexander Bulekov
@ 2021-08-16 11:11     ` Daniel P. Berrangé
  2021-08-16 11:30       ` Alexander Bulekov
  2021-08-17  7:23       ` Thomas Huth
  0 siblings, 2 replies; 21+ messages in thread
From: Daniel P. Berrangé @ 2021-08-16 11:11 UTC (permalink / raw)
  To: Alexander Bulekov; +Cc: Peter Maydell, Thomas Huth, QEMU Developers

On Mon, Aug 16, 2021 at 06:22:46AM -0400, Alexander Bulekov wrote:
> On 210816 1001, Peter Maydell wrote:
> > On Sat, 14 Aug 2021 at 07:10, Thomas Huth <thuth@redhat.com> wrote:
> > >
> > >  Hi Peter,
> > >
> > > in case we're going to have an -rc4, here's a pull request that contains
> > > the fixes for getting the gitlab-CI green again. I also added some doc
> > > updates since they should be completely riskless. But if we won't have an
> > > rc4 due to other reasons, this pull request here certainly also does not
> > > justify another RC, so please ignore this PR in that case.
> > >
> > > The following changes since commit 703e8cd6189cf699c8d5c094bc68b5f3afa6ad71:
> > >
> > >   Update version for v6.1.0-rc3 release (2021-08-10 19:08:09 +0100)
> > >
> > > are available in the Git repository at:
> > >
> > >   https://gitlab.com/thuth/qemu.git tags/pull-request-2021-08-11
> > >
> > > for you to fetch changes up to 36b508993c4dcc6b3ef4b5c00e293ee9560926ee:
> > >
> > >   docs/about/removed-features: Document removed machines from older QEMU versions (2021-08-11 15:39:09 +0200)
> > >
> > > CI run can be seen here:
> > >
> > >  https://gitlab.com/thuth/qemu/-/pipelines/351602605
> > >
> > > ----------------------------------------------------------------
> > > * Fixes for the gitlab-CI (fix the hanging  build-oss-fuzz pipeline)
> > > * Add documentation about features that have been removed in older versions
> > >
> > 
> > 
> > Applied, thanks.
> > 
> > Please update the changelog at https://wiki.qemu.org/ChangeLog/6.1
> > for any user-visible changes.
> 
> https://gitlab.com/qemu-project/qemu/-/jobs/1505950978
> Looks like build-oss-fuzz is still timing out, even without the issue
> in the vhost-usr-blk test. At this point the job should essentially just
> build + test qemu-system-i386 with some extra time spent on linking
> the fuzzer and briefly running through all the fuzzer configs. Maybe the
> only way to make this work is to split the job into a build + test
> stage?

At this point I think we should just disable the job in gitlab entirely.
We've spent too long debugging this, while leaving CI red for everyone.

Whomever is interested in this can then work to find a way to make it
reliable and request it be re-enabled once confident that it will work.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4
  2021-08-16 11:11     ` Daniel P. Berrangé
@ 2021-08-16 11:30       ` Alexander Bulekov
  2021-08-16 11:40         ` Philippe Mathieu-Daudé
  2021-08-16 11:44         ` Daniel P. Berrangé
  2021-08-17  7:23       ` Thomas Huth
  1 sibling, 2 replies; 21+ messages in thread
From: Alexander Bulekov @ 2021-08-16 11:30 UTC (permalink / raw)
  To: Daniel P. Berrangé; +Cc: Peter Maydell, Thomas Huth, QEMU Developers

On 210816 1211, Daniel P. Berrangé wrote:
> On Mon, Aug 16, 2021 at 06:22:46AM -0400, Alexander Bulekov wrote:
> > On 210816 1001, Peter Maydell wrote:
> > > On Sat, 14 Aug 2021 at 07:10, Thomas Huth <thuth@redhat.com> wrote:
> > > >
> > > >  Hi Peter,
> > > >
> > > > in case we're going to have an -rc4, here's a pull request that contains
> > > > the fixes for getting the gitlab-CI green again. I also added some doc
> > > > updates since they should be completely riskless. But if we won't have an
> > > > rc4 due to other reasons, this pull request here certainly also does not
> > > > justify another RC, so please ignore this PR in that case.
> > > >
> > > > The following changes since commit 703e8cd6189cf699c8d5c094bc68b5f3afa6ad71:
> > > >
> > > >   Update version for v6.1.0-rc3 release (2021-08-10 19:08:09 +0100)
> > > >
> > > > are available in the Git repository at:
> > > >
> > > >   https://gitlab.com/thuth/qemu.git tags/pull-request-2021-08-11
> > > >
> > > > for you to fetch changes up to 36b508993c4dcc6b3ef4b5c00e293ee9560926ee:
> > > >
> > > >   docs/about/removed-features: Document removed machines from older QEMU versions (2021-08-11 15:39:09 +0200)
> > > >
> > > > CI run can be seen here:
> > > >
> > > >  https://gitlab.com/thuth/qemu/-/pipelines/351602605
> > > >
> > > > ----------------------------------------------------------------
> > > > * Fixes for the gitlab-CI (fix the hanging  build-oss-fuzz pipeline)
> > > > * Add documentation about features that have been removed in older versions
> > > >
> > > 
> > > 
> > > Applied, thanks.
> > > 
> > > Please update the changelog at https://wiki.qemu.org/ChangeLog/6.1
> > > for any user-visible changes.
> > 
> > https://gitlab.com/qemu-project/qemu/-/jobs/1505950978
> > Looks like build-oss-fuzz is still timing out, even without the issue
> > in the vhost-usr-blk test. At this point the job should essentially just
> > build + test qemu-system-i386 with some extra time spent on linking
> > the fuzzer and briefly running through all the fuzzer configs. Maybe the
> > only way to make this work is to split the job into a build + test
> > stage?
> 
> At this point I think we should just disable the job in gitlab entirely.
> We've spent too long debugging this, while leaving CI red for everyone.
> 
> Whomever is interested in this can then work to find a way to make it
> reliable and request it be re-enabled once confident that it will work.
> 

On my mirror the job succeeded in 41 minutes... I guess you have to get
lucky with scheduling/ambient load.
https://gitlab.com/a1xndr/qemu/-/jobs/1506197531

If all we want is to check that oss-fuzz builds work while keeping green
CI, the only thing that needs to be done is to remove the "make check
check-qtest-i386 check-unit" from the job.

The problem is that QEMU has a ton of nice tests, and by simply building
with --enable-sanitizers, these tests become even more useful because we
can catch all sorts of memory-corruptions. In fact, some of the
regression tests require --enable-sanitizers to catch issues.  When I
looked at this originally, build-oss-fuzz was the only job that was
running with any sanitizers. Now, it looks like at least
ubuntu-18.04-s390x-clang and ubuntu-20.04-aarch64-clang also build with
ASAN. Is there some other i386 build (gcc or clang) where we could add
--enable-sanitizers?

-Alex


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4
  2021-08-16 11:30       ` Alexander Bulekov
@ 2021-08-16 11:40         ` Philippe Mathieu-Daudé
  2021-08-16 12:00           ` Alexander Bulekov
  2021-08-16 11:44         ` Daniel P. Berrangé
  1 sibling, 1 reply; 21+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-08-16 11:40 UTC (permalink / raw)
  To: Alexander Bulekov, Daniel P. Berrangé
  Cc: Peter Maydell, Thomas Huth, QEMU Developers

On 8/16/21 1:30 PM, Alexander Bulekov wrote:
> On 210816 1211, Daniel P. Berrangé wrote:
>> On Mon, Aug 16, 2021 at 06:22:46AM -0400, Alexander Bulekov wrote:

>>> https://gitlab.com/qemu-project/qemu/-/jobs/1505950978
>>> Looks like build-oss-fuzz is still timing out, even without the issue
>>> in the vhost-usr-blk test. At this point the job should essentially just
>>> build + test qemu-system-i386 with some extra time spent on linking
>>> the fuzzer and briefly running through all the fuzzer configs. Maybe the
>>> only way to make this work is to split the job into a build + test
>>> stage?
>>
>> At this point I think we should just disable the job in gitlab entirely.
>> We've spent too long debugging this, while leaving CI red for everyone.
>>
>> Whomever is interested in this can then work to find a way to make it
>> reliable and request it be re-enabled once confident that it will work.
>>
> 
> On my mirror the job succeeded in 41 minutes... I guess you have to get
> lucky with scheduling/ambient load.
> https://gitlab.com/a1xndr/qemu/-/jobs/1506197531

TBH I stopped looking at this job console out because it fails too often
in my pipelines :(



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4
  2021-08-16 11:30       ` Alexander Bulekov
  2021-08-16 11:40         ` Philippe Mathieu-Daudé
@ 2021-08-16 11:44         ` Daniel P. Berrangé
  1 sibling, 0 replies; 21+ messages in thread
From: Daniel P. Berrangé @ 2021-08-16 11:44 UTC (permalink / raw)
  To: Alexander Bulekov; +Cc: Peter Maydell, Thomas Huth, QEMU Developers

On Mon, Aug 16, 2021 at 07:30:59AM -0400, Alexander Bulekov wrote:
> On 210816 1211, Daniel P. Berrangé wrote:
> > On Mon, Aug 16, 2021 at 06:22:46AM -0400, Alexander Bulekov wrote:
> > > On 210816 1001, Peter Maydell wrote:
> > > > On Sat, 14 Aug 2021 at 07:10, Thomas Huth <thuth@redhat.com> wrote:
> > > > >
> > > > >  Hi Peter,
> > > > >
> > > > > in case we're going to have an -rc4, here's a pull request that contains
> > > > > the fixes for getting the gitlab-CI green again. I also added some doc
> > > > > updates since they should be completely riskless. But if we won't have an
> > > > > rc4 due to other reasons, this pull request here certainly also does not
> > > > > justify another RC, so please ignore this PR in that case.
> > > > >
> > > > > The following changes since commit 703e8cd6189cf699c8d5c094bc68b5f3afa6ad71:
> > > > >
> > > > >   Update version for v6.1.0-rc3 release (2021-08-10 19:08:09 +0100)
> > > > >
> > > > > are available in the Git repository at:
> > > > >
> > > > >   https://gitlab.com/thuth/qemu.git tags/pull-request-2021-08-11
> > > > >
> > > > > for you to fetch changes up to 36b508993c4dcc6b3ef4b5c00e293ee9560926ee:
> > > > >
> > > > >   docs/about/removed-features: Document removed machines from older QEMU versions (2021-08-11 15:39:09 +0200)
> > > > >
> > > > > CI run can be seen here:
> > > > >
> > > > >  https://gitlab.com/thuth/qemu/-/pipelines/351602605
> > > > >
> > > > > ----------------------------------------------------------------
> > > > > * Fixes for the gitlab-CI (fix the hanging  build-oss-fuzz pipeline)
> > > > > * Add documentation about features that have been removed in older versions
> > > > >
> > > > 
> > > > 
> > > > Applied, thanks.
> > > > 
> > > > Please update the changelog at https://wiki.qemu.org/ChangeLog/6.1
> > > > for any user-visible changes.
> > > 
> > > https://gitlab.com/qemu-project/qemu/-/jobs/1505950978
> > > Looks like build-oss-fuzz is still timing out, even without the issue
> > > in the vhost-usr-blk test. At this point the job should essentially just
> > > build + test qemu-system-i386 with some extra time spent on linking
> > > the fuzzer and briefly running through all the fuzzer configs. Maybe the
> > > only way to make this work is to split the job into a build + test
> > > stage?
> > 
> > At this point I think we should just disable the job in gitlab entirely.
> > We've spent too long debugging this, while leaving CI red for everyone.
> > 
> > Whomever is interested in this can then work to find a way to make it
> > reliable and request it be re-enabled once confident that it will work.
> > 
> 
> On my mirror the job succeeded in 41 minutes... I guess you have to get
> lucky with scheduling/ambient load.
> https://gitlab.com/a1xndr/qemu/-/jobs/1506197531

I don't think load would make that much of a difference. It smells like
there is still some non-deterministic bug in there that is causing a
hang of the job.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4
  2021-08-16 11:40         ` Philippe Mathieu-Daudé
@ 2021-08-16 12:00           ` Alexander Bulekov
  0 siblings, 0 replies; 21+ messages in thread
From: Alexander Bulekov @ 2021-08-16 12:00 UTC (permalink / raw)
  To: Philippe Mathieu-Daudé
  Cc: Peter Maydell, Thomas Huth, Daniel P. Berrangé, QEMU Developers

On 210816 1340, Philippe Mathieu-Daudé wrote:
> On 8/16/21 1:30 PM, Alexander Bulekov wrote:
> > On 210816 1211, Daniel P. Berrangé wrote:
> >> On Mon, Aug 16, 2021 at 06:22:46AM -0400, Alexander Bulekov wrote:
> 
> >>> https://gitlab.com/qemu-project/qemu/-/jobs/1505950978
> >>> Looks like build-oss-fuzz is still timing out, even without the issue
> >>> in the vhost-usr-blk test. At this point the job should essentially just
> >>> build + test qemu-system-i386 with some extra time spent on linking
> >>> the fuzzer and briefly running through all the fuzzer configs. Maybe the
> >>> only way to make this work is to split the job into a build + test
> >>> stage?
> >>
> >> At this point I think we should just disable the job in gitlab entirely.
> >> We've spent too long debugging this, while leaving CI red for everyone.
> >>
> >> Whomever is interested in this can then work to find a way to make it
> >> reliable and request it be re-enabled once confident that it will work.
> >>
> > 
> > On my mirror the job succeeded in 41 minutes... I guess you have to get
> > lucky with scheduling/ambient load.
> > https://gitlab.com/a1xndr/qemu/-/jobs/1506197531
> 
> TBH I stopped looking at this job console out because it fails too often
> in my pipelines :(
> 

Of course if the job times out 50% of the time, nobody will want to look
at it. And as a result, issues like the one with the vhost-user-blk
test are unnoticed. My hope was that removing the redundant build would
take care of timeouts. Maybe if we find that this new sporadic
timeout issue is also due to some test-failure, the job will again be
useful for someone.
-Alex


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4
  2021-08-16 11:11     ` Daniel P. Berrangé
  2021-08-16 11:30       ` Alexander Bulekov
@ 2021-08-17  7:23       ` Thomas Huth
  1 sibling, 0 replies; 21+ messages in thread
From: Thomas Huth @ 2021-08-17  7:23 UTC (permalink / raw)
  To: Daniel P. Berrangé, Alexander Bulekov; +Cc: Peter Maydell, QEMU Developers

On 16/08/2021 13.11, Daniel P. Berrangé wrote:
> On Mon, Aug 16, 2021 at 06:22:46AM -0400, Alexander Bulekov wrote:
>> On 210816 1001, Peter Maydell wrote:
>>> On Sat, 14 Aug 2021 at 07:10, Thomas Huth <thuth@redhat.com> wrote:
>>>>
>>>>   Hi Peter,
>>>>
>>>> in case we're going to have an -rc4, here's a pull request that contains
>>>> the fixes for getting the gitlab-CI green again. I also added some doc
>>>> updates since they should be completely riskless. But if we won't have an
>>>> rc4 due to other reasons, this pull request here certainly also does not
>>>> justify another RC, so please ignore this PR in that case.
>>>>
>>>> The following changes since commit 703e8cd6189cf699c8d5c094bc68b5f3afa6ad71:
>>>>
>>>>    Update version for v6.1.0-rc3 release (2021-08-10 19:08:09 +0100)
>>>>
>>>> are available in the Git repository at:
>>>>
>>>>    https://gitlab.com/thuth/qemu.git tags/pull-request-2021-08-11
>>>>
>>>> for you to fetch changes up to 36b508993c4dcc6b3ef4b5c00e293ee9560926ee:
>>>>
>>>>    docs/about/removed-features: Document removed machines from older QEMU versions (2021-08-11 15:39:09 +0200)
>>>>
>>>> CI run can be seen here:
>>>>
>>>>   https://gitlab.com/thuth/qemu/-/pipelines/351602605
>>>>
>>>> ----------------------------------------------------------------
>>>> * Fixes for the gitlab-CI (fix the hanging  build-oss-fuzz pipeline)
>>>> * Add documentation about features that have been removed in older versions
>>>>
>>>
>>>
>>> Applied, thanks.
>>>
>>> Please update the changelog at https://wiki.qemu.org/ChangeLog/6.1
>>> for any user-visible changes.
>>
>> https://gitlab.com/qemu-project/qemu/-/jobs/1505950978
>> Looks like build-oss-fuzz is still timing out, even without the issue
>> in the vhost-usr-blk test.

It worked fine in the staging branch, finished within 40 minutes:

  https://gitlab.com/qemu-project/qemu/-/jobs/1504868929

>> At this point the job should essentially just
>> build + test qemu-system-i386 with some extra time spent on linking
>> the fuzzer and briefly running through all the fuzzer configs. Maybe the
>> only way to make this work is to split the job into a build + test
>> stage?
> 
> At this point I think we should just disable the job in gitlab entirely.
> We've spent too long debugging this, while leaving CI red for everyone.

I don't think so. That seems to be a new, unrelated problem:

Running test qtest-i386/usb-hcd-ehci-test
socket_accept failed: Resource temporarily unavailable
**
ERROR:../tests/qtest/libqtest.c:319:qtest_init_without_qmp_handshake: assertion failed: (s->fd >= 0 && s->qmp_fd >= 0)
ERROR qtest-i386/usb-hcd-ehci-test - Bail out! ERROR:../tests/qtest/libqtest.c:319:qtest_init_without_qmp_handshake: assertion failed: (s->fd >= 0 && s->qmp_fd >= 0)

That "socket_accept failed: Resource temporarily unavailable" sounds like
the build machine was maybe temporarily overloaded?

Question is why the test framework was running into a timeout afterwards
here instead of bailing out immediately?

  Thomas



^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, other threads:[~2021-08-17  7:23 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-14  6:09 [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4 Thomas Huth
2021-08-14  6:09 ` [PULL 01/11] gitlab: exclude sparc-softmmu and riscv32-softmmu from cross builds Thomas Huth
2021-08-14  6:09 ` [PULL 02/11] gitlab: skip many more targets in windows " Thomas Huth
2021-08-15 15:47   ` Peter Maydell
2021-08-14  6:09 ` [PULL 03/11] storage-daemon: Add missing build dependency to the vhost-user-blk-test Thomas Huth
2021-08-14  6:09 ` [PULL 04/11] tests/qtest/vhost-user-blk-test: Check whether qemu-storage-daemon is available Thomas Huth
2021-08-14  6:09 ` [PULL 05/11] fuzz: avoid building twice, when running on gitlab Thomas Huth
2021-08-14  6:09 ` [PULL 06/11] docs/about/removed-features: Document removed CLI options from QEMU v2.12 Thomas Huth
2021-08-14  6:09 ` [PULL 07/11] docs/about/removed-features: Document removed CLI options from QEMU v3.0 Thomas Huth
2021-08-14  6:09 ` [PULL 08/11] docs/about/removed-features: Document removed CLI options from QEMU v3.1 Thomas Huth
2021-08-14  6:09 ` [PULL 09/11] docs/about/removed-features: Document removed HMP commands from QEMU v2.12 Thomas Huth
2021-08-14  6:09 ` [PULL 10/11] docs/about/removed-features: Document removed devices from older QEMU versions Thomas Huth
2021-08-14  6:09 ` [PULL 11/11] docs/about/removed-features: Document removed machines " Thomas Huth
2021-08-16  9:01 ` [PULL 00/11] Optional gitlab-CI and doc fixes for -rc4 Peter Maydell
2021-08-16 10:22   ` Alexander Bulekov
2021-08-16 11:11     ` Daniel P. Berrangé
2021-08-16 11:30       ` Alexander Bulekov
2021-08-16 11:40         ` Philippe Mathieu-Daudé
2021-08-16 12:00           ` Alexander Bulekov
2021-08-16 11:44         ` Daniel P. Berrangé
2021-08-17  7:23       ` Thomas Huth

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.