[Qemu-devel] [PULL 0/4] x86 and memory backends queue, 2016-09-05

[Qemu-devel] [PULL 0/4] x86 and memory backends queue, 2016-09-05

[Eduardo Habkost]

The following changes since commit e87d397e5ef66276ccc49b829527d605ca07d0ad:

  Open 2.8 development tree (2016-09-05 11:38:54 +0100)

are available in the git repository at:

  git://github.com/ehabkost/qemu.git tags/x86-pull-request

for you to fetch changes up to 6546d0dba6c211c1a3eac1252a4f50a0c151a08a:

  vl: Delay initialization of memory backends (2016-09-05 16:03:47 -0300)

----------------------------------------------------------------
x86 and memory backends queue, 2016-09-05

This includes a few features that were submitted just after hard
freeze, and a bug fix for memory backend initialization ordering.

----------------------------------------------------------------

Eduardo Habkost (2):
  vhost-user-test: Use libqos instead of pxe-virtio.rom
  vl: Delay initialization of memory backends

Igor Mammedov (1):
  exec: Ensure the only one cpu_index allocation method is used

Luwei Kang (1):
  target-i386: Add more Intel AVX-512 instructions support

 exec.c                  |  7 +++++++
 target-i386/cpu.c       | 14 +++++++++-----
 target-i386/cpu.h       |  5 +++++
 tests/Makefile.include  |  2 +-
 tests/vhost-user-test.c | 37 ++++++++++++++++++++++++++++++++++---
 vl.c                    | 13 +++++++++++++
 6 files changed, 69 insertions(+), 9 deletions(-)

-- 
2.7.4

[Qemu-devel] [PULL 1/4] exec: Ensure the only one cpu_index allocation method is used

[Eduardo Habkost]

From: Igor Mammedov <imammedo@redhat.com>

Make sure that cpu_index auto allocation isn't used in
combination with manual cpu_index assignment. And
dissallow out of order cpu removal if auto allocation
is in use.

Target that wishes to support out of order unplug should
switch to manual cpu_index assignment. Following patch
could be used as an example:
 (pc: init CPUState->cpu_index with index in possible_cpus[]))

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 exec.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/exec.c b/exec.c
index 8ffde75..80398b0 100644
--- a/exec.c
+++ b/exec.c
@@ -598,11 +598,14 @@ AddressSpace *cpu_get_address_space(CPUState *cpu, int asidx)
 }
 #endif
 
+static bool cpu_index_auto_assigned;
+
 static int cpu_get_free_index(void)
 {
     CPUState *some_cpu;
     int cpu_index = 0;
 
+    cpu_index_auto_assigned = true;
     CPU_FOREACH(some_cpu) {
         cpu_index++;
     }
@@ -620,6 +623,8 @@ void cpu_exec_exit(CPUState *cpu)
         return;
     }
 
+    assert(!(cpu_index_auto_assigned && cpu != QTAILQ_LAST(&cpus, CPUTailQ)));
+
     QTAILQ_REMOVE(&cpus, cpu, node);
     cpu->node.tqe_prev = NULL;
     cpu->cpu_index = UNASSIGNED_CPU_INDEX;
@@ -663,6 +668,8 @@ void cpu_exec_init(CPUState *cpu, Error **errp)
     if (cpu->cpu_index == UNASSIGNED_CPU_INDEX) {
         cpu->cpu_index = cpu_get_free_index();
         assert(cpu->cpu_index != UNASSIGNED_CPU_INDEX);
+    } else {
+        assert(!cpu_index_auto_assigned);
     }
     QTAILQ_INSERT_TAIL(&cpus, cpu, node);
     cpu_list_unlock();
-- 
2.7.4

[Qemu-devel] [PULL 2/4] target-i386: Add more Intel AVX-512 instructions support

[Eduardo Habkost]

From: Luwei Kang <luwei.kang@intel.com>

Add more AVX512 feature bits, include AVX512DQ, AVX512IFMA,
AVX512BW, AVX512VL, AVX512VBMI. Its spec can be found at:
https://software.intel.com/sites/default/files/managed/b4/3a/319433-024.pdf

Signed-off-by: Luwei Kang <luwei.kang@intel.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 target-i386/cpu.c | 14 +++++++++-----
 target-i386/cpu.h |  5 +++++
 2 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 6a1afab..ec674dc 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -298,14 +298,18 @@ static const char *svm_feature_name[] = {
 };
 
 static const char *cpuid_7_0_ebx_feature_name[] = {
-    "fsgsbase", "tsc_adjust", NULL, "bmi1", "hle", "avx2", NULL, "smep",
-    "bmi2", "erms", "invpcid", "rtm", NULL, NULL, "mpx", NULL,
-    "avx512f", NULL, "rdseed", "adx", "smap", NULL, "pcommit", "clflushopt",
-    "clwb", NULL, "avx512pf", "avx512er", "avx512cd", NULL, NULL, NULL,
+    "fsgsbase", "tsc_adjust", NULL, "bmi1",
+    "hle", "avx2", NULL, "smep",
+    "bmi2", "erms", "invpcid", "rtm",
+    NULL, NULL, "mpx", NULL,
+    "avx512f", "avx512dq", "rdseed", "adx",
+    "smap", "avx512ifma", "pcommit", "clflushopt",
+    "clwb", NULL, "avx512pf", "avx512er",
+    "avx512cd", NULL, "avx512bw", "avx512vl",
 };
 
 static const char *cpuid_7_0_ecx_feature_name[] = {
-    NULL, NULL, "umip", "pku",
+    NULL, "avx512vbmi", "umip", "pku",
     "ospke", NULL, NULL, NULL,
     NULL, NULL, NULL, NULL,
     NULL, NULL, NULL, NULL,
diff --git a/target-i386/cpu.h b/target-i386/cpu.h
index 65615c0..cf14bcb 100644
--- a/target-i386/cpu.h
+++ b/target-i386/cpu.h
@@ -606,16 +606,21 @@ typedef uint32_t FeatureWordArray[FEATURE_WORDS];
 #define CPUID_7_0_EBX_RTM      (1U << 11)
 #define CPUID_7_0_EBX_MPX      (1U << 14)
 #define CPUID_7_0_EBX_AVX512F  (1U << 16) /* AVX-512 Foundation */
+#define CPUID_7_0_EBX_AVX512DQ (1U << 17) /* AVX-512 Doubleword & Quadword Instrs */
 #define CPUID_7_0_EBX_RDSEED   (1U << 18)
 #define CPUID_7_0_EBX_ADX      (1U << 19)
 #define CPUID_7_0_EBX_SMAP     (1U << 20)
+#define CPUID_7_0_EBX_AVX512IFMA (1U << 21) /* AVX-512 Integer Fused Multiply Add */
 #define CPUID_7_0_EBX_PCOMMIT  (1U << 22) /* Persistent Commit */
 #define CPUID_7_0_EBX_CLFLUSHOPT (1U << 23) /* Flush a Cache Line Optimized */
 #define CPUID_7_0_EBX_CLWB     (1U << 24) /* Cache Line Write Back */
 #define CPUID_7_0_EBX_AVX512PF (1U << 26) /* AVX-512 Prefetch */
 #define CPUID_7_0_EBX_AVX512ER (1U << 27) /* AVX-512 Exponential and Reciprocal */
 #define CPUID_7_0_EBX_AVX512CD (1U << 28) /* AVX-512 Conflict Detection */
+#define CPUID_7_0_EBX_AVX512BW (1U << 30) /* AVX-512 Byte and Word Instructions */
+#define CPUID_7_0_EBX_AVX512VL (1U << 31) /* AVX-512 Vector Length Extensions */
 
+#define CPUID_7_0_ECX_VBMI     (1U << 1)  /* AVX-512 Vector Byte Manipulation Instrs */
 #define CPUID_7_0_ECX_UMIP     (1U << 2)
 #define CPUID_7_0_ECX_PKU      (1U << 3)
 #define CPUID_7_0_ECX_OSPKE    (1U << 4)
-- 
2.7.4

[Qemu-devel] [PULL 3/4] vhost-user-test: Use libqos instead of pxe-virtio.rom

[Eduardo Habkost]

vhost-user-test relies on iPXE just to initialize the virtio-net
device, and doesn't do any actual packet tx/rx testing.

In addition to that, the test relies on TCG, which is
imcompatible with vhost. The test only worked by accident: a bug
the memory backend initialization made memory regions not have
the DIRTY_MEMORY_CODE bit set in dirty_log_mask.

This changes vhost-user-test to initialize the virtio-net device
using libqos, and not use TCG nor pxe-virtio.rom.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 tests/Makefile.include  |  2 +-
 tests/vhost-user-test.c | 37 ++++++++++++++++++++++++++++++++++---
 2 files changed, 35 insertions(+), 4 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 14be491..03382b5 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -622,7 +622,7 @@ tests/usb-hcd-ehci-test$(EXESUF): tests/usb-hcd-ehci-test.o $(libqos-usb-obj-y)
 tests/usb-hcd-xhci-test$(EXESUF): tests/usb-hcd-xhci-test.o $(libqos-usb-obj-y)
 tests/pc-cpu-test$(EXESUF): tests/pc-cpu-test.o
 tests/postcopy-test$(EXESUF): tests/postcopy-test.o
-tests/vhost-user-test$(EXESUF): tests/vhost-user-test.o qemu-char.o qemu-timer.o $(qtest-obj-y) $(test-io-obj-y)
+tests/vhost-user-test$(EXESUF): tests/vhost-user-test.o qemu-char.o qemu-timer.o $(qtest-obj-y) $(test-io-obj-y) $(libqos-virtio-obj-y)
 tests/qemu-iotests/socket_scm_helper$(EXESUF): tests/qemu-iotests/socket_scm_helper.o
 tests/test-qemu-opts$(EXESUF): tests/test-qemu-opts.o $(test-util-obj-y)
 tests/test-write-threshold$(EXESUF): tests/test-write-threshold.o $(test-block-obj-y)
diff --git a/tests/vhost-user-test.c b/tests/vhost-user-test.c
index 27b10c1..b89a551 100644
--- a/tests/vhost-user-test.c
+++ b/tests/vhost-user-test.c
@@ -16,8 +16,13 @@
 #include "qemu/sockets.h"
 #include "sysemu/char.h"
 #include "sysemu/sysemu.h"
+#include "libqos/libqos.h"
+#include "libqos/pci-pc.h"
+#include "libqos/virtio-pci.h"
 
 #include <linux/vhost.h>
+#include <linux/virtio_ids.h>
+#include <linux/virtio_net.h>
 #include <sys/vfs.h>
 
 /* GLIB version compatibility flags */
@@ -29,14 +34,13 @@
 #define HAVE_MONOTONIC_TIME
 #endif
 
-#define QEMU_CMD_ACCEL  " -machine accel=tcg"
 #define QEMU_CMD_MEM    " -m %d -object memory-backend-file,id=mem,size=%dM,"\
                         "mem-path=%s,share=on -numa node,memdev=mem"
 #define QEMU_CMD_CHR    " -chardev socket,id=%s,path=%s%s"
 #define QEMU_CMD_NETDEV " -netdev vhost-user,id=net0,chardev=%s,vhostforce"
-#define QEMU_CMD_NET    " -device virtio-net-pci,netdev=net0,romfile=./pc-bios/pxe-virtio.rom"
+#define QEMU_CMD_NET    " -device virtio-net-pci,netdev=net0"
 
-#define QEMU_CMD        QEMU_CMD_ACCEL QEMU_CMD_MEM QEMU_CMD_CHR \
+#define QEMU_CMD        QEMU_CMD_MEM QEMU_CMD_CHR \
                         QEMU_CMD_NETDEV QEMU_CMD_NET
 
 #define HUGETLBFS_MAGIC       0x958458f6
@@ -136,6 +140,30 @@ typedef struct TestServer {
 static const char *tmpfs;
 static const char *root;
 
+static void init_virtio_dev(TestServer *s)
+{
+    QPCIBus *bus;
+    QVirtioPCIDevice *dev;
+    uint32_t features;
+
+    bus = qpci_init_pc();
+    g_assert_nonnull(bus);
+
+    dev = qvirtio_pci_device_find(bus, VIRTIO_ID_NET);
+    g_assert_nonnull(dev);
+
+    qvirtio_pci_device_enable(dev);
+    qvirtio_reset(&qvirtio_pci, &dev->vdev);
+    qvirtio_set_acknowledge(&qvirtio_pci, &dev->vdev);
+    qvirtio_set_driver(&qvirtio_pci, &dev->vdev);
+
+    features = qvirtio_get_features(&qvirtio_pci, &dev->vdev);
+    features = features & VIRTIO_NET_F_MAC;
+    qvirtio_set_features(&qvirtio_pci, &dev->vdev, features);
+
+    qvirtio_set_driver_ok(&qvirtio_pci, &dev->vdev);
+}
+
 static void wait_for_fds(TestServer *s)
 {
     gint64 end_time;
@@ -548,6 +576,7 @@ static void test_migrate(void)
     from = qtest_start(cmd);
     g_free(cmd);
 
+    init_virtio_dev(s);
     wait_for_fds(s);
     size = get_log_size(s);
     g_assert_cmpint(size, ==, (2 * 1024 * 1024) / (VHOST_LOG_PAGE * 8));
@@ -662,6 +691,7 @@ static void test_reconnect_subprocess(void)
     qtest_start(cmd);
     g_free(cmd);
 
+    init_virtio_dev(s);
     wait_for_fds(s);
     wait_for_rings_started(s, 2);
 
@@ -728,6 +758,7 @@ int main(int argc, char **argv)
 
     s = qtest_start(qemu_cmd);
     g_free(qemu_cmd);
+    init_virtio_dev(server);
 
     qtest_add_data_func("/vhost-user/read-guest-mem", server, read_guest_mem);
     qtest_add_func("/vhost-user/migrate", test_migrate);
-- 
2.7.4

[Qemu-devel] [PULL 4/4] vl: Delay initialization of memory backends

[Eduardo Habkost]

Initialization of memory backends may take a while when
prealloc=yes is used, depending on their size. Initializing
memory backends before chardevs may delay the creation of monitor
sockets, and trigger timeouts on management software that waits
until the monitor socket is created by QEMU. See, for example,
the bug report at:
https://bugzilla.redhat.com/show_bug.cgi?id=1371211

In addition to that, allocating memory before calling
configure_accelerator() breaks the tcg_enabled() checks at
memory_region_init_*().

This patch fixes those problems by adding "memory-backend-*"
classes to the delayed-initialization list.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 vl.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/vl.c b/vl.c
index b3c80d5..ee557a1 100644
--- a/vl.c
+++ b/vl.c
@@ -2810,6 +2810,19 @@ static bool object_create_initial(const char *type)
         return false;
     }
 
+    /* Memory allocation by backends needs to be done
+     * after configure_accelerator() (due to the tcg_enabled()
+     * checks at memory_region_init_*()).
+     *
+     * Also, allocation of large amounts of memory may delay
+     * chardev initialization for too long, and trigger timeouts
+     * on software that waits for a monitor socket to be created
+     * (e.g. libvirt).
+     */
+    if (g_str_has_prefix(type, "memory-backend-")) {
+        return false;
+    }
+
     return true;
 }
 
-- 
2.7.4

Re: [Qemu-devel] [PULL 0/4] x86 and memory backends queue, 2016-09-05

[Peter Maydell]

On 5 September 2016 at 20:50, Eduardo Habkost <ehabkost@redhat.com> wrote:
> The following changes since commit e87d397e5ef66276ccc49b829527d605ca07d0ad:
>
>   Open 2.8 development tree (2016-09-05 11:38:54 +0100)
>
> are available in the git repository at:
>
>   git://github.com/ehabkost/qemu.git tags/x86-pull-request
>
> for you to fetch changes up to 6546d0dba6c211c1a3eac1252a4f50a0c151a08a:
>
>   vl: Delay initialization of memory backends (2016-09-05 16:03:47 -0300)
>
> ----------------------------------------------------------------
> x86 and memory backends queue, 2016-09-05
>
> This includes a few features that were submitted just after hard
> freeze, and a bug fix for memory backend initialization ordering.
>
> ----------------------------------------------------------------

Applied, thanks.

-- PMM