All of lore.kernel.org
 help / color / mirror / Atom feed
* [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups
@ 2022-11-07 22:47 Michael S. Tsirkin
  2022-11-07 22:47 ` [PULL v4 01/83] hw/i386/e820: remove legacy reserved entries for e820 Michael S. Tsirkin
                   ` (84 more replies)
  0 siblings, 85 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:47 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell

Changes from v3:
    Applied and squashed fix by Ani for modular build breakage
    Reordered Julia's patches to avoid bisect breakage
    Checkpatch fixes for Jason's patches
    Added Alex's patch to partially address virtio is_started mess
    There is a bigger issue found by Christian A. Ehrhardt, that
    still needs work
    checkpatch change to avoid breaking on Jason's patches (to make ci pass)

Changes from v2:
    Fixed a bug in error handling in vhost: Change the sequence of device start.
        Contributor placed on watchlist ;)
    Dropped virtio: re-order vm_running and use_started checks
        Due to failures detected by gitlab.
        We'll have to fix it differently.
    Updated expected files for core-count test to fix bisect.

Changes from v1
    Applied and squashed fixes by Igor, Lei He, Hesham Almatary for
    bugs that tripped up the pipeline.
    Updated expected files for core-count test.



The following changes since commit a11f65ec1b8adcb012b89c92819cbda4dc25aaf1:

  Merge tag 'block-pull-request' of https://gitlab.com/stefanha/qemu into staging (2022-11-01 13:49:33 -0400)

are available in the Git repository at:

  https://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git tags/for_upstream

for you to fetch changes up to 1ef47f40dce3d5b176ddf76d57b5bfa2efb0b3c6:

  checkpatch: better pattern for inline comments (2022-11-07 14:25:51 -0500)

----------------------------------------------------------------
pci,pc,virtio: features, tests, fixes, cleanups

lots of acpi rework
first version of biosbits infrastructure
ASID support in vhost-vdpa
core_count2 support in smbios
PCIe DOE emulation
virtio vq reset
HMAT support
part of infrastructure for viommu support in vhost-vdpa
VTD PASID support
fixes, tests all over the place

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

----------------------------------------------------------------
Akihiko Odaki (1):
      msix: Assert that specified vector is in range

Alex Bennée (1):
      hw/virtio: introduce virtio_device_should_start

Ani Sinha (7):
      hw/i386/e820: remove legacy reserved entries for e820
      acpi/tests/avocado/bits: initial commit of test scripts that are run by biosbits
      acpi/tests/avocado/bits: disable acpi PSS tests that are failing in biosbits
      acpi/tests/avocado/bits: add biosbits config file for running bios tests
      acpi/tests/avocado/bits: add acpi and smbios avocado tests that uses biosbits
      acpi/tests/avocado/bits/doc: add a doc file to describe the acpi bits test
      MAINTAINERS: add myself as the maintainer for acpi biosbits avocado tests

Bernhard Beschow (3):
      hw/i386/acpi-build: Remove unused struct
      hw/i386/acpi-build: Resolve redundant attribute
      hw/i386/acpi-build: Resolve north rather than south bridges

Brice Goglin (4):
      hmat acpi: Don't require initiator value in -numa
      tests: acpi: add and whitelist *.hmat-noinitiator expected blobs
      tests: acpi: q35: add test for hmat nodes without initiators
      tests: acpi: q35: update expected blobs *.hmat-noinitiators expected HMAT:

Christian A. Ehrhardt (1):
      hw/acpi/erst.c: Fix memory handling issues

Cindy Lu (1):
      vfio: move implement of vfio_get_xlat_addr() to memory.c

David Daney (1):
      virtio-rng-pci: Allow setting nvectors, so we can use MSI-X

Eric Auger (1):
      hw/virtio/virtio-iommu-pci: Enforce the device is plugged on the root bus

Gregory Price (1):
      hw/i386/pc.c: CXL Fixed Memory Window should not reserve e820 in bios

Hesham Almatary (3):
      tests: Add HMAT AArch64/virt empty table files
      tests: acpi: aarch64/virt: add a test for hmat nodes with no initiators
      tests: virt: Update expected *.acpihmatvirt tables

Huai-Cheng Kuo (3):
      hw/pci: PCIe Data Object Exchange emulation
      hw/cxl/cdat: CXL CDAT Data Object Exchange implementation
      hw/mem/cxl-type3: Add CXL CDAT Data Object Exchange

Igor Mammedov (11):
      acpi: pc: vga: use AcpiDevAmlIf interface to build VGA device descriptors
      tests: acpi: whitelist DSDT before generating PCI-ISA bridge AML automatically
      acpi: pc/q35: drop ad-hoc PCI-ISA bridge AML routines and let bus ennumeration generate AML
      tests: acpi: update expected DSDT after ISA bridge is moved directly under PCI host bridge
      tests: acpi: whitelist DSDT before generating ICH9_SMB AML automatically
      acpi: add get_dev_aml_func() helper
      acpi: enumerate SMB bridge automatically along with other PCI devices
      tests: acpi: update expected blobs
      tests: acpi: pc/q35 whitelist DSDT before \_GPE cleanup
      acpi: pc/35: sanitize _GPE declaration order
      tests: acpi: update expected blobs

Jason Wang (4):
      intel-iommu: don't warn guest errors when getting rid2pasid entry
      intel-iommu: drop VTDBus
      intel-iommu: convert VTD_PE_GET_FPD_ERR() to be a function
      intel-iommu: PASID support

Jonathan Cameron (2):
      hw/mem/cxl-type3: Add MSIX support
      hw/pci-bridge/cxl-upstream: Add a CDAT table access DOE

Julia Suvorova (5):
      hw/smbios: add core_count2 to smbios table type 4
      bios-tables-test: teach test to use smbios 3.0 tables
      tests/acpi: allow changes for core_count2 test
      bios-tables-test: add test for number of cores > 255
      tests/acpi: update tables for new core count test

Kangjie Xu (10):
      virtio: introduce virtio_queue_enable()
      virtio: core: vq reset feature negotation support
      virtio-pci: support queue enable
      vhost: expose vhost_virtqueue_start()
      vhost: expose vhost_virtqueue_stop()
      vhost-net: vhost-kernel: introduce vhost_net_virtqueue_reset()
      vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
      virtio-net: introduce flush_or_purge_queued_packets()
      virtio-net: support queue_enable
      vhost: vhost-kernel: enable vq reset feature

Lei He (4):
      virtio-crypto: Support asynchronous mode
      crypto: Support DER encodings
      crypto: Support export akcipher to pkcs8
      cryptodev: Add a lkcf-backend for cryptodev

Markus Armbruster (1):
      MAINTAINERS: Add qapi/virtio.json to section "virtio"

Michael S. Tsirkin (1):
      checkpatch: better pattern for inline comments

Miguel Luis (4):
      tests/acpi: virt: allow acpi MADT and FADT changes
      acpi: fadt: support revision 6.0 of the ACPI specification
      acpi: arm/virt: madt: bump to revision 4 accordingly to ACPI 6.0 Errata A
      tests/acpi: virt: update ACPI MADT and FADT binaries

Robert Hoo (5):
      tests/acpi: allow SSDT changes
      acpi/ssdt: Fix aml_or() and aml_and() in if clause
      acpi/nvdimm: define macro for NVDIMM Device _DSM
      acpi/nvdimm: Implement ACPI NVDIMM Label Methods
      test/acpi/bios-tables-test: SSDT: update golden master binaries

Xiang Chen (1):
      hw/arm/virt: Enable HMAT on arm virt machine

Xuan Zhuo (5):
      virtio: introduce __virtio_queue_reset()
      virtio: introduce virtio_queue_reset()
      virtio-pci: support queue reset
      virtio-net: support queue reset
      virtio-net: enable vq reset feature

Yajun Wu (3):
      vhost-user: Fix out of order vring host notification handling
      vhost: Change the sequence of device start
      vhost-user: Support vhost_dev_start

 tests/avocado/acpi-bits/bits-config/bits-cfg.txt |   18 +
 qapi/qom.json                                    |    2 +
 crypto/der.h                                     |  211 +-
 crypto/rsakey.h                                  |   11 +-
 hw/display/vga_int.h                             |    2 +
 hw/i386/e820_memory_layout.h                     |    8 -
 hw/i386/fw_cfg.h                                 |    1 -
 hw/i386/intel_iommu_internal.h                   |   16 +-
 hw/smbios/smbios_build.h                         |    9 +-
 include/crypto/akcipher.h                        |   21 +
 include/exec/memory.h                            |    4 +
 include/hw/acpi/acpi_aml_interface.h             |   13 +-
 include/hw/cxl/cxl_cdat.h                        |  166 ++
 include/hw/cxl/cxl_component.h                   |    7 +
 include/hw/cxl/cxl_device.h                      |    3 +
 include/hw/cxl/cxl_pci.h                         |    1 +
 include/hw/firmware/smbios.h                     |   12 +
 include/hw/i386/intel_iommu.h                    |   18 +-
 include/hw/pci/msix.h                            |    4 +-
 include/hw/pci/pci_bus.h                         |    2 +
 include/hw/pci/pci_ids.h                         |    3 +
 include/hw/pci/pcie.h                            |    1 +
 include/hw/pci/pcie_doe.h                        |  123 ++
 include/hw/pci/pcie_regs.h                       |    4 +
 include/hw/virtio/vhost.h                        |    5 +
 include/hw/virtio/virtio-pci.h                   |    5 +
 include/hw/virtio/virtio.h                       |   26 +-
 include/net/vhost_net.h                          |    4 +
 include/sysemu/cryptodev.h                       |   61 +-
 backends/cryptodev-builtin.c                     |   69 +-
 backends/cryptodev-lkcf.c                        |  645 ++++++
 backends/cryptodev-vhost-user.c                  |   53 +-
 backends/cryptodev.c                             |   44 +-
 crypto/akcipher.c                                |   18 +
 crypto/der.c                                     |  313 ++-
 crypto/rsakey.c                                  |   42 +
 hw/acpi/aml-build-stub.c                         |   10 +
 hw/acpi/aml-build.c                              |   13 +-
 hw/acpi/erst.c                                   |    6 +-
 hw/acpi/nvdimm.c                                 |  106 +-
 hw/arm/virt-acpi-build.c                         |   33 +-
 hw/block/vhost-user-blk.c                        |   24 +-
 hw/core/machine.c                                |    8 +-
 hw/cxl/cxl-cdat.c                                |  224 ++
 hw/display/acpi-vga-stub.c                       |    7 +
 hw/display/acpi-vga.c                            |   26 +
 hw/display/vga-pci.c                             |    4 +
 hw/i386/acpi-build.c                             |  203 +-
 hw/i386/e820_memory_layout.c                     |   20 +-
 hw/i386/fw_cfg.c                                 |    3 -
 hw/i386/intel_iommu.c                            |  694 +++---
 hw/i386/microvm.c                                |    2 -
 hw/i386/pc.c                                     |    2 -
 hw/isa/lpc_ich9.c                                |   23 +
 hw/isa/piix3.c                                   |   17 +-
 hw/mem/cxl_type3.c                               |  264 +++
 hw/net/e1000e.c                                  |   15 +-
 hw/net/rocker/rocker.c                           |   23 +-
 hw/net/vhost_net-stub.c                          |   12 +
 hw/net/vhost_net.c                               |   90 +-
 hw/net/virtio-net.c                              |   57 +-
 hw/net/vmxnet3.c                                 |   27 +-
 hw/nvme/ctrl.c                                   |    5 +-
 hw/pci-bridge/cxl_upstream.c                     |  195 +-
 hw/pci/msix.c                                    |   24 +-
 hw/pci/pcie_doe.c                                |  367 ++++
 hw/rdma/vmw/pvrdma_main.c                        |    7 +-
 hw/remote/vfio-user-obj.c                        |    9 +-
 hw/smbios/smbios.c                               |   19 +-
 hw/vfio/common.c                                 |   66 +-
 hw/virtio/vhost-user-fs.c                        |    2 +-
 hw/virtio/vhost-user-gpio.c                      |    2 +-
 hw/virtio/vhost-user-i2c.c                       |    2 +-
 hw/virtio/vhost-user-rng.c                       |    2 +-
 hw/virtio/vhost-user-vsock.c                     |    2 +-
 hw/virtio/vhost-user.c                           |   79 +-
 hw/virtio/vhost-vsock.c                          |    2 +-
 hw/virtio/vhost.c                                |   16 +-
 hw/virtio/virtio-crypto.c                        |  339 +--
 hw/virtio/virtio-iommu-pci.c                     |   12 +-
 hw/virtio/virtio-pci.c                           |   83 +-
 hw/virtio/virtio-rng-pci.c                       |   14 +
 hw/virtio/virtio.c                               |   62 +-
 softmmu/memory.c                                 |   72 +
 tests/qtest/bios-tables-test.c                   |  267 ++-
 tests/unit/test-crypto-der.c                     |  126 +-
 MAINTAINERS                                      |   15 +
 backends/meson.build                             |    3 +
 docs/devel/acpi-bits.rst                         |  145 ++
 docs/devel/index-build.rst                       |    1 +
 hw/arm/Kconfig                                   |    1 +
 hw/cxl/meson.build                               |    1 +
 hw/display/meson.build                           |   17 +
 hw/i386/trace-events                             |    2 +
 hw/pci/meson.build                               |    1 +
 scripts/checkpatch.pl                            |    6 +-
 tests/avocado/acpi-bits.py                       |  396 ++++
 tests/avocado/acpi-bits/bits-tests/smbios.py2    | 2430 ++++++++++++++++++++++
 tests/avocado/acpi-bits/bits-tests/testacpi.py2  |  283 +++
 tests/avocado/acpi-bits/bits-tests/testcpuid.py2 |   83 +
 tests/data/acpi/pc/DSDT                          |  Bin 6422 -> 6501 bytes
 tests/data/acpi/pc/DSDT.acpierst                 |  Bin 6382 -> 6461 bytes
 tests/data/acpi/pc/DSDT.acpihmat                 |  Bin 7747 -> 7826 bytes
 tests/data/acpi/pc/DSDT.bridge                   |  Bin 9496 -> 9575 bytes
 tests/data/acpi/pc/DSDT.cphp                     |  Bin 6886 -> 6965 bytes
 tests/data/acpi/pc/DSDT.dimmpxm                  |  Bin 8076 -> 8155 bytes
 tests/data/acpi/pc/DSDT.hpbridge                 |  Bin 6382 -> 6461 bytes
 tests/data/acpi/pc/DSDT.hpbrroot                 |  Bin 3069 -> 3107 bytes
 tests/data/acpi/pc/DSDT.ipmikcs                  |  Bin 6494 -> 6573 bytes
 tests/data/acpi/pc/DSDT.memhp                    |  Bin 7781 -> 7860 bytes
 tests/data/acpi/pc/DSDT.nohpet                   |  Bin 6280 -> 6359 bytes
 tests/data/acpi/pc/DSDT.numamem                  |  Bin 6428 -> 6507 bytes
 tests/data/acpi/pc/DSDT.roothp                   |  Bin 6656 -> 6699 bytes
 tests/data/acpi/pc/SSDT.dimmpxm                  |  Bin 734 -> 1815 bytes
 tests/data/acpi/q35/APIC.acpihmat-noinitiator    |  Bin 0 -> 144 bytes
 tests/data/acpi/q35/APIC.core-count2             |  Bin 0 -> 2478 bytes
 tests/data/acpi/q35/DSDT                         |  Bin 8320 -> 8412 bytes
 tests/data/acpi/q35/DSDT.acpierst                |  Bin 8337 -> 8429 bytes
 tests/data/acpi/q35/DSDT.acpihmat                |  Bin 9645 -> 9737 bytes
 tests/data/acpi/q35/DSDT.acpihmat-noinitiator    |  Bin 0 -> 8691 bytes
 tests/data/acpi/q35/DSDT.applesmc                |  Bin 8366 -> 8458 bytes
 tests/data/acpi/q35/DSDT.bridge                  |  Bin 11449 -> 11541 bytes
 tests/data/acpi/q35/DSDT.core-count2             |  Bin 0 -> 32552 bytes
 tests/data/acpi/q35/DSDT.cphp                    |  Bin 8784 -> 8876 bytes
 tests/data/acpi/q35/DSDT.cxl                     |  Bin 9646 -> 9738 bytes
 tests/data/acpi/q35/DSDT.dimmpxm                 |  Bin 9974 -> 10066 bytes
 tests/data/acpi/q35/DSDT.ipmibt                  |  Bin 8395 -> 8487 bytes
 tests/data/acpi/q35/DSDT.ipmismbus               |  Bin 8409 -> 8500 bytes
 tests/data/acpi/q35/DSDT.ivrs                    |  Bin 8337 -> 8429 bytes
 tests/data/acpi/q35/DSDT.memhp                   |  Bin 9679 -> 9771 bytes
 tests/data/acpi/q35/DSDT.mmio64                  |  Bin 9450 -> 9542 bytes
 tests/data/acpi/q35/DSDT.multi-bridge            |  Bin 8640 -> 8732 bytes
 tests/data/acpi/q35/DSDT.nohpet                  |  Bin 8178 -> 8270 bytes
 tests/data/acpi/q35/DSDT.numamem                 |  Bin 8326 -> 8418 bytes
 tests/data/acpi/q35/DSDT.pvpanic-isa             |  Bin 8421 -> 8513 bytes
 tests/data/acpi/q35/DSDT.tis.tpm12               |  Bin 8926 -> 9018 bytes
 tests/data/acpi/q35/DSDT.tis.tpm2                |  Bin 8952 -> 9044 bytes
 tests/data/acpi/q35/DSDT.viot                    |  Bin 9429 -> 9521 bytes
 tests/data/acpi/q35/DSDT.xapic                   |  Bin 35683 -> 35775 bytes
 tests/data/acpi/q35/FACP.core-count2             |  Bin 0 -> 244 bytes
 tests/data/acpi/q35/HMAT.acpihmat-noinitiator    |  Bin 0 -> 288 bytes
 tests/data/acpi/q35/SRAT.acpihmat-noinitiator    |  Bin 0 -> 312 bytes
 tests/data/acpi/q35/SSDT.dimmpxm                 |  Bin 734 -> 1815 bytes
 tests/data/acpi/virt/APIC                        |  Bin 168 -> 172 bytes
 tests/data/acpi/virt/APIC.acpihmatvirt           |  Bin 0 -> 412 bytes
 tests/data/acpi/virt/APIC.memhp                  |  Bin 168 -> 172 bytes
 tests/data/acpi/virt/APIC.numamem                |  Bin 168 -> 172 bytes
 tests/data/acpi/virt/DSDT.acpihmatvirt           |  Bin 0 -> 5282 bytes
 tests/data/acpi/virt/FACP                        |  Bin 268 -> 276 bytes
 tests/data/acpi/virt/FACP.memhp                  |  Bin 268 -> 276 bytes
 tests/data/acpi/virt/FACP.numamem                |  Bin 268 -> 276 bytes
 tests/data/acpi/virt/HMAT.acpihmatvirt           |  Bin 0 -> 288 bytes
 tests/data/acpi/virt/PPTT.acpihmatvirt           |  Bin 0 -> 196 bytes
 tests/data/acpi/virt/SRAT.acpihmatvirt           |  Bin 0 -> 240 bytes
 tests/data/acpi/virt/SSDT.memhp                  |  Bin 736 -> 1817 bytes
 155 files changed, 7996 insertions(+), 1020 deletions(-)
 create mode 100644 tests/avocado/acpi-bits/bits-config/bits-cfg.txt
 create mode 100644 include/hw/cxl/cxl_cdat.h
 create mode 100644 include/hw/pci/pcie_doe.h
 create mode 100644 backends/cryptodev-lkcf.c
 create mode 100644 hw/cxl/cxl-cdat.c
 create mode 100644 hw/display/acpi-vga-stub.c
 create mode 100644 hw/display/acpi-vga.c
 create mode 100644 hw/pci/pcie_doe.c
 create mode 100644 docs/devel/acpi-bits.rst
 create mode 100644 tests/avocado/acpi-bits.py
 create mode 100644 tests/avocado/acpi-bits/bits-tests/smbios.py2
 create mode 100644 tests/avocado/acpi-bits/bits-tests/testacpi.py2
 create mode 100644 tests/avocado/acpi-bits/bits-tests/testcpuid.py2
 create mode 100644 tests/data/acpi/q35/APIC.acpihmat-noinitiator
 create mode 100644 tests/data/acpi/q35/APIC.core-count2
 create mode 100644 tests/data/acpi/q35/DSDT.acpihmat-noinitiator
 create mode 100644 tests/data/acpi/q35/DSDT.core-count2
 create mode 100644 tests/data/acpi/q35/FACP.core-count2
 create mode 100644 tests/data/acpi/q35/HMAT.acpihmat-noinitiator
 create mode 100644 tests/data/acpi/q35/SRAT.acpihmat-noinitiator
 create mode 100644 tests/data/acpi/virt/APIC.acpihmatvirt
 create mode 100644 tests/data/acpi/virt/DSDT.acpihmatvirt
 create mode 100644 tests/data/acpi/virt/HMAT.acpihmatvirt
 create mode 100644 tests/data/acpi/virt/PPTT.acpihmatvirt
 create mode 100644 tests/data/acpi/virt/SRAT.acpihmatvirt



^ permalink raw reply	[flat|nested] 151+ messages in thread

* [PULL v4 01/83] hw/i386/e820: remove legacy reserved entries for e820
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
@ 2022-11-07 22:47 ` Michael S. Tsirkin
  2022-11-07 22:47 ` [PULL v4 02/83] tests/acpi: allow SSDT changes Michael S. Tsirkin
                   ` (83 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:47 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Ani Sinha, Gerd Hoffmann, Marcel Apfelbaum,
	Paolo Bonzini, Richard Henderson, Eduardo Habkost, Sergio Lopez

From: Ani Sinha <ani@anisinha.ca>

e820 reserved entries were used before the dynamic entries with fw config files
were intoduced. Please see the following change:
7d67110f2d9a6("pc: add etc/e820 fw_cfg file")

Identical support was introduced into seabios as well with the following commit:
ce39bd4031820 ("Add support for etc/e820 fw_cfg file")

Both the above commits are now quite old. QEMU machines 1.7 and newer no longer
use the reserved entries. Seabios uses fw config files and
dynamic e820 entries by default and only falls back to using reserved entries
when it has to work with old qemu (versions earlier than 1.7). Please see
functions qemu_cfg_e820() and qemu_early_e820(). It is safe to remove legacy
FW_CFG_E820_TABLE and associated code now as QEMU 7.0 has deprecated i440fx
machines 1.7 and older. It would be incredibly rare to run the latest qemu
version with a very old version of seabios that did not support fw config files
for e820.

As far as I could see, edk2/ovfm never supported reserved entries and uses fw
config files from the beginning. So there should be no incompatibilities with
ovfm as well.

CC: Gerd Hoffmann <kraxel@redhat.com>
Signed-off-by: Ani Sinha <ani@anisinha.ca>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Message-Id: <20220831045311.33083-1-ani@anisinha.ca>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/i386/e820_memory_layout.h |  8 --------
 hw/i386/fw_cfg.h             |  1 -
 hw/i386/e820_memory_layout.c | 20 +-------------------
 hw/i386/fw_cfg.c             |  3 ---
 hw/i386/microvm.c            |  2 --
 5 files changed, 1 insertion(+), 33 deletions(-)

diff --git a/hw/i386/e820_memory_layout.h b/hw/i386/e820_memory_layout.h
index 04f93780f9..7c239aa033 100644
--- a/hw/i386/e820_memory_layout.h
+++ b/hw/i386/e820_memory_layout.h
@@ -16,20 +16,12 @@
 #define E820_NVS        4
 #define E820_UNUSABLE   5
 
-#define E820_NR_ENTRIES 16
-
 struct e820_entry {
     uint64_t address;
     uint64_t length;
     uint32_t type;
 } QEMU_PACKED __attribute((__aligned__(4)));
 
-struct e820_table {
-    uint32_t count;
-    struct e820_entry entry[E820_NR_ENTRIES];
-} QEMU_PACKED __attribute((__aligned__(4)));
-
-extern struct e820_table e820_reserve;
 extern struct e820_entry *e820_table;
 
 int e820_add_entry(uint64_t address, uint64_t length, uint32_t type);
diff --git a/hw/i386/fw_cfg.h b/hw/i386/fw_cfg.h
index 275f15c1c5..86ca7c1c0c 100644
--- a/hw/i386/fw_cfg.h
+++ b/hw/i386/fw_cfg.h
@@ -17,7 +17,6 @@
 #define FW_CFG_ACPI_TABLES      (FW_CFG_ARCH_LOCAL + 0)
 #define FW_CFG_SMBIOS_ENTRIES   (FW_CFG_ARCH_LOCAL + 1)
 #define FW_CFG_IRQ0_OVERRIDE    (FW_CFG_ARCH_LOCAL + 2)
-#define FW_CFG_E820_TABLE       (FW_CFG_ARCH_LOCAL + 3)
 #define FW_CFG_HPET             (FW_CFG_ARCH_LOCAL + 4)
 
 FWCfgState *fw_cfg_arch_create(MachineState *ms,
diff --git a/hw/i386/e820_memory_layout.c b/hw/i386/e820_memory_layout.c
index bcf9eaf837..06970ac44a 100644
--- a/hw/i386/e820_memory_layout.c
+++ b/hw/i386/e820_memory_layout.c
@@ -11,29 +11,11 @@
 #include "e820_memory_layout.h"
 
 static size_t e820_entries;
-struct e820_table e820_reserve;
 struct e820_entry *e820_table;
 
 int e820_add_entry(uint64_t address, uint64_t length, uint32_t type)
 {
-    int index = le32_to_cpu(e820_reserve.count);
-    struct e820_entry *entry;
-
-    if (type != E820_RAM) {
-        /* old FW_CFG_E820_TABLE entry -- reservations only */
-        if (index >= E820_NR_ENTRIES) {
-            return -EBUSY;
-        }
-        entry = &e820_reserve.entry[index++];
-
-        entry->address = cpu_to_le64(address);
-        entry->length = cpu_to_le64(length);
-        entry->type = cpu_to_le32(type);
-
-        e820_reserve.count = cpu_to_le32(index);
-    }
-
-    /* new "etc/e820" file -- include ram too */
+    /* new "etc/e820" file -- include ram and reserved entries */
     e820_table = g_renew(struct e820_entry, e820_table, e820_entries + 1);
     e820_table[e820_entries].address = cpu_to_le64(address);
     e820_table[e820_entries].length = cpu_to_le64(length);
diff --git a/hw/i386/fw_cfg.c b/hw/i386/fw_cfg.c
index a283785a8d..72a42f3c66 100644
--- a/hw/i386/fw_cfg.c
+++ b/hw/i386/fw_cfg.c
@@ -36,7 +36,6 @@ const char *fw_cfg_arch_key_name(uint16_t key)
         {FW_CFG_ACPI_TABLES, "acpi_tables"},
         {FW_CFG_SMBIOS_ENTRIES, "smbios_entries"},
         {FW_CFG_IRQ0_OVERRIDE, "irq0_override"},
-        {FW_CFG_E820_TABLE, "e820_table"},
         {FW_CFG_HPET, "hpet"},
     };
 
@@ -127,8 +126,6 @@ FWCfgState *fw_cfg_arch_create(MachineState *ms,
 #endif
     fw_cfg_add_i32(fw_cfg, FW_CFG_IRQ0_OVERRIDE, 1);
 
-    fw_cfg_add_bytes(fw_cfg, FW_CFG_E820_TABLE,
-                     &e820_reserve, sizeof(e820_reserve));
     fw_cfg_add_file(fw_cfg, "etc/e820", e820_table,
                     sizeof(struct e820_entry) * e820_get_num_entries());
 
diff --git a/hw/i386/microvm.c b/hw/i386/microvm.c
index ffd1884100..170a331e3f 100644
--- a/hw/i386/microvm.c
+++ b/hw/i386/microvm.c
@@ -324,8 +324,6 @@ static void microvm_memory_init(MicrovmMachineState *mms)
     fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS, machine->smp.max_cpus);
     fw_cfg_add_i64(fw_cfg, FW_CFG_RAM_SIZE, (uint64_t)machine->ram_size);
     fw_cfg_add_i32(fw_cfg, FW_CFG_IRQ0_OVERRIDE, 1);
-    fw_cfg_add_bytes(fw_cfg, FW_CFG_E820_TABLE,
-                     &e820_reserve, sizeof(e820_reserve));
     fw_cfg_add_file(fw_cfg, "etc/e820", e820_table,
                     sizeof(struct e820_entry) * e820_get_num_entries());
 
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 02/83] tests/acpi: allow SSDT changes
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
  2022-11-07 22:47 ` [PULL v4 01/83] hw/i386/e820: remove legacy reserved entries for e820 Michael S. Tsirkin
@ 2022-11-07 22:47 ` Michael S. Tsirkin
  2022-11-07 22:47 ` [PULL v4 03/83] acpi/ssdt: Fix aml_or() and aml_and() in if clause Michael S. Tsirkin
                   ` (82 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:47 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Robert Hoo, Jingqi Liu, Igor Mammedov, Ani Sinha

From: Robert Hoo <robert.hu@linux.intel.com>

Signed-off-by: Robert Hoo <robert.hu@linux.intel.com>
Reviewed-by: Jingqi Liu <jingqi.liu@intel.com>
Message-Id: <20220922122155.1326543-2-robert.hu@linux.intel.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/qtest/bios-tables-test-allowed-diff.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index dfb8523c8b..eb8bae1407 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1 +1,3 @@
 /* List of comma-separated changed AML files to ignore */
+"tests/data/acpi/pc/SSDT.dimmpxm",
+"tests/data/acpi/q35/SSDT.dimmpxm",
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 03/83] acpi/ssdt: Fix aml_or() and aml_and() in if clause
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
  2022-11-07 22:47 ` [PULL v4 01/83] hw/i386/e820: remove legacy reserved entries for e820 Michael S. Tsirkin
  2022-11-07 22:47 ` [PULL v4 02/83] tests/acpi: allow SSDT changes Michael S. Tsirkin
@ 2022-11-07 22:47 ` Michael S. Tsirkin
  2022-11-07 22:47 ` [PULL v4 04/83] acpi/nvdimm: define macro for NVDIMM Device _DSM Michael S. Tsirkin
                   ` (81 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:47 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Robert Hoo, Jingqi Liu, Igor Mammedov, Ani Sinha

From: Robert Hoo <robert.hu@linux.intel.com>

In If condition, using bitwise and/or, rather than logical and/or.

The result change in AML code:

If (((Local6 == Zero) | (Arg0 != Local0)))
==>
If (((Local6 == Zero) || (Arg0 != Local0)))

If (((ObjectType (Arg3) == 0x04) & (SizeOf (Arg3) == One)))
==>
If (((ObjectType (Arg3) == 0x04) && (SizeOf (Arg3) == One)))

Fixes: 90623ebf603 ("nvdimm acpi: check UUID")
Fixes: 4568c948066 ("nvdimm acpi: save arg3 of _DSM method")
Signed-off-by: Robert Hoo <robert.hu@linux.intel.com>
Reviewed-by: Jingqi Liu <jingqi.liu@intel.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20220922122155.1326543-3-robert.hu@linux.intel.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/acpi/nvdimm.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/hw/acpi/nvdimm.c b/hw/acpi/nvdimm.c
index 31e46df0bd..201317c611 100644
--- a/hw/acpi/nvdimm.c
+++ b/hw/acpi/nvdimm.c
@@ -1037,7 +1037,7 @@ static void nvdimm_build_common_dsm(Aml *dev,
 
     uuid_invalid = aml_lnot(aml_equal(uuid, expected_uuid));
 
-    unsupport = aml_if(aml_or(unpatched, uuid_invalid, NULL));
+    unsupport = aml_if(aml_lor(unpatched, uuid_invalid));
 
     /*
      * function 0 is called to inquire what functions are supported by
@@ -1069,10 +1069,9 @@ static void nvdimm_build_common_dsm(Aml *dev,
      * in the DSM Spec.
      */
     pckg = aml_arg(3);
-    ifctx = aml_if(aml_and(aml_equal(aml_object_type(pckg),
+    ifctx = aml_if(aml_land(aml_equal(aml_object_type(pckg),
                    aml_int(4 /* Package */)) /* It is a Package? */,
-                   aml_equal(aml_sizeof(pckg), aml_int(1)) /* 1 element? */,
-                   NULL));
+                   aml_equal(aml_sizeof(pckg), aml_int(1)) /* 1 element? */));
 
     pckg_index = aml_local(2);
     pckg_buf = aml_local(3);
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 04/83] acpi/nvdimm: define macro for NVDIMM Device _DSM
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (2 preceding siblings ...)
  2022-11-07 22:47 ` [PULL v4 03/83] acpi/ssdt: Fix aml_or() and aml_and() in if clause Michael S. Tsirkin
@ 2022-11-07 22:47 ` Michael S. Tsirkin
  2022-11-07 22:47 ` [PULL v4 05/83] acpi/nvdimm: Implement ACPI NVDIMM Label Methods Michael S. Tsirkin
                   ` (80 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:47 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Robert Hoo, Igor Mammedov, Ani Sinha

From: Robert Hoo <robert.hu@linux.intel.com>

Since it will be heavily used in next patch, define macro
NVDIMM_DEVICE_DSM_UUID for "4309AC30-0D11-11E4-9191-0800200C9A66", which is
NVDIMM device specific method uuid defined in NVDIMM _DSM interface spec,
Section 3. [1]

No functional changes in this patch.

[1] https://pmem.io/documents/IntelOptanePMem_DSM_Interface-V2.0.pdf

Signed-off-by: Robert Hoo <robert.hu@linux.intel.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20220922122155.1326543-4-robert.hu@linux.intel.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/acpi/nvdimm.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/acpi/nvdimm.c b/hw/acpi/nvdimm.c
index 201317c611..afff911c1e 100644
--- a/hw/acpi/nvdimm.c
+++ b/hw/acpi/nvdimm.c
@@ -922,6 +922,7 @@ void nvdimm_init_acpi_state(NVDIMMState *state, MemoryRegion *io,
 #define NVDIMM_DSM_RFIT_STATUS  "RSTA"
 
 #define NVDIMM_QEMU_RSVD_UUID   "648B9CF2-CDA1-4312-8AD9-49C4AF32BD62"
+#define NVDIMM_DEVICE_DSM_UUID  "4309AC30-0D11-11E4-9191-0800200C9A66"
 
 static void nvdimm_build_common_dsm(Aml *dev,
                                     NVDIMMState *nvdimm_state)
@@ -1029,8 +1030,7 @@ static void nvdimm_build_common_dsm(Aml *dev,
                /* UUID for QEMU internal use */), expected_uuid));
     aml_append(elsectx, ifctx);
     elsectx2 = aml_else();
-    aml_append(elsectx2, aml_store(
-               aml_touuid("4309AC30-0D11-11E4-9191-0800200C9A66")
+    aml_append(elsectx2, aml_store(aml_touuid(NVDIMM_DEVICE_DSM_UUID)
                /* UUID for NVDIMM Devices */, expected_uuid));
     aml_append(elsectx, elsectx2);
     aml_append(method, elsectx);
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 05/83] acpi/nvdimm: Implement ACPI NVDIMM Label Methods
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (3 preceding siblings ...)
  2022-11-07 22:47 ` [PULL v4 04/83] acpi/nvdimm: define macro for NVDIMM Device _DSM Michael S. Tsirkin
@ 2022-11-07 22:47 ` Michael S. Tsirkin
  2022-11-07 22:47 ` [PULL v4 06/83] test/acpi/bios-tables-test: SSDT: update golden master binaries Michael S. Tsirkin
                   ` (79 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:47 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Robert Hoo, Igor Mammedov, Ani Sinha

From: Robert Hoo <robert.hu@linux.intel.com>

Recent ACPI spec [1] has defined NVDIMM Label Methods _LS{I,R,W}, which
deprecates corresponding _DSM Functions defined by PMEM _DSM Interface spec
[2].

Since the semantics of the new Label Methods are almost same as old _DSM
methods, the implementations here simply wrapper old ones.

ASL form diff can be found in next patch of updating golden master
binaries.

[1] ACPI Spec v6.4, 6.5.10 NVDIMM Label Methods
https://uefi.org/sites/default/files/resources/ACPI_Spec_6_4_Jan22.pdf
[2] Intel PMEM _DSM Interface Spec v2.0, 3.10 Deprecated Functions
https://pmem.io/documents/IntelOptanePMem_DSM_Interface-V2.0.pdf

Signed-off-by: Robert Hoo <robert.hu@linux.intel.com>
Message-Id: <20220922122155.1326543-5-robert.hu@linux.intel.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/acpi/nvdimm.c | 95 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 95 insertions(+)

diff --git a/hw/acpi/nvdimm.c b/hw/acpi/nvdimm.c
index afff911c1e..a3b25a92f3 100644
--- a/hw/acpi/nvdimm.c
+++ b/hw/acpi/nvdimm.c
@@ -1243,6 +1243,7 @@ static void nvdimm_build_fit(Aml *dev)
 static void nvdimm_build_nvdimm_devices(Aml *root_dev, uint32_t ram_slots)
 {
     uint32_t slot;
+    Aml *method, *pkg, *field, *com_call;
 
     for (slot = 0; slot < ram_slots; slot++) {
         uint32_t handle = nvdimm_slot_to_handle(slot);
@@ -1260,6 +1261,100 @@ static void nvdimm_build_nvdimm_devices(Aml *root_dev, uint32_t ram_slots)
          */
         aml_append(nvdimm_dev, aml_name_decl("_ADR", aml_int(handle)));
 
+        /*
+         * ACPI v6.4: Section 6.5.10 NVDIMM Label Methods
+         */
+        /* _LSI */
+        method = aml_method("_LSI", 0, AML_SERIALIZED);
+        com_call = aml_call5(NVDIMM_COMMON_DSM,
+                            aml_touuid(NVDIMM_DEVICE_DSM_UUID),
+                            aml_int(1), aml_int(4), aml_int(0),
+                            aml_int(handle));
+        aml_append(method, aml_store(com_call, aml_local(0)));
+
+        aml_append(method, aml_create_dword_field(aml_local(0),
+                                                  aml_int(0), "STTS"));
+        aml_append(method, aml_create_dword_field(aml_local(0), aml_int(4),
+                                                  "SLSA"));
+        aml_append(method, aml_create_dword_field(aml_local(0), aml_int(8),
+                                                  "MAXT"));
+
+        pkg = aml_package(3);
+        aml_append(pkg, aml_name("STTS"));
+        aml_append(pkg, aml_name("SLSA"));
+        aml_append(pkg, aml_name("MAXT"));
+        aml_append(method, aml_store(pkg, aml_local(1)));
+        aml_append(method, aml_return(aml_local(1)));
+
+        aml_append(nvdimm_dev, method);
+
+        /* _LSR */
+        method = aml_method("_LSR", 2, AML_SERIALIZED);
+        aml_append(method, aml_name_decl("INPT", aml_buffer(8, NULL)));
+
+        aml_append(method, aml_create_dword_field(aml_name("INPT"),
+                                                  aml_int(0), "OFST"));
+        aml_append(method, aml_create_dword_field(aml_name("INPT"),
+                                                  aml_int(4), "LEN"));
+        aml_append(method, aml_store(aml_arg(0), aml_name("OFST")));
+        aml_append(method, aml_store(aml_arg(1), aml_name("LEN")));
+
+        pkg = aml_package(1);
+        aml_append(pkg, aml_name("INPT"));
+        aml_append(method, aml_store(pkg, aml_local(0)));
+
+        com_call = aml_call5(NVDIMM_COMMON_DSM,
+                            aml_touuid(NVDIMM_DEVICE_DSM_UUID),
+                            aml_int(1), aml_int(5), aml_local(0),
+                            aml_int(handle));
+        aml_append(method, aml_store(com_call, aml_local(3)));
+        field = aml_create_dword_field(aml_local(3), aml_int(0), "STTS");
+        aml_append(method, field);
+        field = aml_create_field(aml_local(3), aml_int(32),
+                                 aml_shiftleft(aml_name("LEN"), aml_int(3)),
+                                 "LDAT");
+        aml_append(method, field);
+        aml_append(method, aml_name_decl("LSA", aml_buffer(0, NULL)));
+        aml_append(method, aml_to_buffer(aml_name("LDAT"), aml_name("LSA")));
+
+        pkg = aml_package(2);
+        aml_append(pkg, aml_name("STTS"));
+        aml_append(pkg, aml_name("LSA"));
+
+        aml_append(method, aml_store(pkg, aml_local(1)));
+        aml_append(method, aml_return(aml_local(1)));
+
+        aml_append(nvdimm_dev, method);
+
+        /* _LSW */
+        method = aml_method("_LSW", 3, AML_SERIALIZED);
+        aml_append(method, aml_store(aml_arg(2), aml_local(2)));
+        aml_append(method, aml_name_decl("INPT", aml_buffer(8, NULL)));
+        field = aml_create_dword_field(aml_name("INPT"),
+                                                  aml_int(0), "OFST");
+        aml_append(method, field);
+        field = aml_create_dword_field(aml_name("INPT"),
+                                                  aml_int(4), "TLEN");
+        aml_append(method, field);
+        aml_append(method, aml_store(aml_arg(0), aml_name("OFST")));
+        aml_append(method, aml_store(aml_arg(1), aml_name("TLEN")));
+
+        aml_append(method, aml_concatenate(aml_name("INPT"), aml_local(2),
+                                            aml_name("INPT")));
+        pkg = aml_package(1);
+        aml_append(pkg, aml_name("INPT"));
+        aml_append(method, aml_store(pkg, aml_local(0)));
+        com_call = aml_call5(NVDIMM_COMMON_DSM,
+                            aml_touuid(NVDIMM_DEVICE_DSM_UUID),
+                            aml_int(1), aml_int(6), aml_local(0),
+                            aml_int(handle));
+        aml_append(method, aml_store(com_call, aml_local(3)));
+        field = aml_create_dword_field(aml_local(3), aml_int(0), "STTS");
+        aml_append(method, field);
+        aml_append(method, aml_return(aml_name("STTS")));
+
+        aml_append(nvdimm_dev, method);
+
         nvdimm_build_device_dsm(nvdimm_dev, handle);
         aml_append(root_dev, nvdimm_dev);
     }
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 06/83] test/acpi/bios-tables-test: SSDT: update golden master binaries
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (4 preceding siblings ...)
  2022-11-07 22:47 ` [PULL v4 05/83] acpi/nvdimm: Implement ACPI NVDIMM Label Methods Michael S. Tsirkin
@ 2022-11-07 22:47 ` Michael S. Tsirkin
  2022-11-07 22:47 ` [PULL v4 07/83] virtio-crypto: Support asynchronous mode Michael S. Tsirkin
                   ` (78 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:47 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Robert Hoo, Igor Mammedov, Ani Sinha

From: Robert Hoo <robert.hu@linux.intel.com>

And empty bios-tables-test-allowed-diff.h.

Diff of ASL form, from qtest testlog.txt:

@@ -1,30 +1,30 @@
 /*
  * Intel ACPI Component Architecture
  * AML/ASL+ Disassembler version 20180629 (64-bit version)
  * Copyright (c) 2000 - 2018 Intel Corporation
  *
  * Disassembling to symbolic ASL+ operators
  *
- * Disassembly of tests/data/acpi/pc/SSDT.dimmpxm, Thu Sep 22 18:25:06 2022
+ * Disassembly of /tmp/aml-YYZZS1, Thu Sep 22 18:25:06 2022
  *
  * Original Table Header:
  *     Signature        "SSDT"
- *     Length           0x000002DE (734)
+ *     Length           0x00000717 (1815)
  *     Revision         0x01
- *     Checksum         0x56
+ *     Checksum         0xBC
  *     OEM ID           "BOCHS "
  *     OEM Table ID     "NVDIMM"
  *     OEM Revision     0x00000001 (1)
  *     Compiler ID      "BXPC"
  *     Compiler Version 0x00000001 (1)
  */
 DefinitionBlock ("", "SSDT", 1, "BOCHS ", "NVDIMM", 0x00000001)
 {
     Scope (\_SB)
     {
         Device (NVDR)
         {
             Name (_HID, "ACPI0012" /* NVDIMM Root Device */)  // _HID: Hardware ID
             Method (NCAL, 5, Serialized)
             {
                 Local6 = MEMA /* \MEMA */
@@ -49,52 +49,52 @@
                     ODAT,   32736
                 }

                 If ((Arg4 == Zero))
                 {
                     Local0 = ToUUID ("2f10e7a4-9e91-11e4-89d3-123b93f75cba")
                 }
                 ElseIf ((Arg4 == 0x00010000))
                 {
                     Local0 = ToUUID ("648b9cf2-cda1-4312-8ad9-49c4af32bd62")
                 }
                 Else
                 {
                     Local0 = ToUUID ("4309ac30-0d11-11e4-9191-0800200c9a66")
                 }

-                If (((Local6 == Zero) | (Arg0 != Local0)))
+                If (((Local6 == Zero) || (Arg0 != Local0)))
                 {
                     If ((Arg2 == Zero))
                     {
                         Return (Buffer (One)
                         {
                              0x00                                             // .
                         })
                     }

                     Return (Buffer (One)
                     {
                          0x01                                             // .
                     })
                 }

                 HDLE = Arg4
                 REVS = Arg1
                 FUNC = Arg2
-                If (((ObjectType (Arg3) == 0x04) & (SizeOf (Arg3) == One)))
+                If (((ObjectType (Arg3) == 0x04) && (SizeOf (Arg3) == One)))
                 {
                     Local2 = Arg3 [Zero]
                     Local3 = DerefOf (Local2)
                     FARG = Local3
                 }

                 NTFI = Local6
                 Local1 = (RLEN - 0x04)
                 If ((Local1 < 0x08))
                 {
                     Local2 = Zero
                     Name (TBUF, Buffer (One)
                     {
                          0x00                                             // .
                     })
                     Local7 = Buffer (Zero){}
@@ -161,45 +161,234 @@
                     Else
                     {
                         If ((Local1 == Zero))
                         {
                             Return (Local2)
                         }

                         Local3 += Local1
                         Concatenate (Local2, Local0, Local2)
                     }
                 }
             }

             Device (NV00)
             {
                 Name (_ADR, One)  // _ADR: Address
+                Method (_LSI, 0, Serialized)  // _LSI: Label Storage Information
+                {
+                    Local0 = NCAL (ToUUID ("4309ac30-0d11-11e4-9191-0800200c9a66"), One, 0x04, Zero, One)
+                    CreateDWordField (Local0, Zero, STTS)
+                    CreateDWordField (Local0, 0x04, SLSA)
+                    CreateDWordField (Local0, 0x08, MAXT)
+                    Local1 = Package (0x03)
+                        {
+                            STTS,
+                            SLSA,
+                            MAXT
+                        }
+                    Return (Local1)
+                }
+
+                Method (_LSR, 2, Serialized)  // _LSR: Label Storage Read
+                {
+                    Name (INPT, Buffer (0x08)
+                    {
+                         0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00   // ........
+                    })
+                    CreateDWordField (INPT, Zero, OFST)
+                    CreateDWordField (INPT, 0x04, LEN)
+                    OFST = Arg0
+                    LEN = Arg1
+                    Local0 = Package (0x01)
+                        {
+                            INPT
+                        }
+                    Local3 = NCAL (ToUUID ("4309ac30-0d11-11e4-9191-0800200c9a66"), One, 0x05, Local0, One)
+                    CreateDWordField (Local3, Zero, STTS)
+                    CreateField (Local3, 0x20, (LEN << 0x03), LDAT)
+                    Name (LSA, Buffer (Zero){})
+                    ToBuffer (LDAT, LSA) /* \_SB_.NVDR.NV00._LSR.LSA_ */
+                    Local1 = Package (0x02)
+                        {
+                            STTS,
+                            LSA
+                        }
+                    Return (Local1)
+                }
+
+                Method (_LSW, 3, Serialized)  // _LSW: Label Storage Write
+                {
+                    Local2 = Arg2
+                    Name (INPT, Buffer (0x08)
+                    {
+                         0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00   // ........
+                    })
+                    CreateDWordField (INPT, Zero, OFST)
+                    CreateDWordField (INPT, 0x04, TLEN)
+                    OFST = Arg0
+                    TLEN = Arg1
+                    Concatenate (INPT, Local2, INPT) /* \_SB_.NVDR.NV00._LSW.INPT */
+                    Local0 = Package (0x01)
+                        {
+                            INPT
+                        }
+                    Local3 = NCAL (ToUUID ("4309ac30-0d11-11e4-9191-0800200c9a66"), One, 0x06, Local0, One)
+                    CreateDWordField (Local3, Zero, STTS)
+                    Return (STTS) /* \_SB_.NVDR.NV00._LSW.STTS */
+                }
+
(iterates in each NV)

Message-Id: <20220922122155.1326543-6-robert.hu@linux.intel.com>
Signed-off-by: Robert Hoo <robert.hu@linux.intel.com>
Acked-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/qtest/bios-tables-test-allowed-diff.h |   2 --
 tests/data/acpi/pc/SSDT.dimmpxm             | Bin 734 -> 1815 bytes
 tests/data/acpi/q35/SSDT.dimmpxm            | Bin 734 -> 1815 bytes
 tests/data/acpi/virt/SSDT.memhp             | Bin 736 -> 1817 bytes
 4 files changed, 2 deletions(-)

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index eb8bae1407..dfb8523c8b 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1,3 +1 @@
 /* List of comma-separated changed AML files to ignore */
-"tests/data/acpi/pc/SSDT.dimmpxm",
-"tests/data/acpi/q35/SSDT.dimmpxm",
diff --git a/tests/data/acpi/pc/SSDT.dimmpxm b/tests/data/acpi/pc/SSDT.dimmpxm
index ac55387d57e48adb99eb738a102308688a262fb8..70f133412f5e0aa128ab210245a8de7304eeb843 100644
GIT binary patch
literal 1815
zcmdUwyKmD_6vnUPv~g}y6emHgc**|(X$OSF0FILox3Lr1ZmHx-examI3c8|YVC!RO
z2@)c;%774ZDvwC)2sTzGCN_pj>?}wOz&%bMqC!xRK#<|wbI(0K`Q7hx6kRVFqX~qV
z7sa|%)dh8?Br6KtBZP{x4GGpv_2!(V7cFzGeuJKCoK=-eBcjxh3x)9sl%G1ON@8t<
zC}l-#nk#BUt~04IjN>%dL<KcdC}Xaspw6mBMHbA}GjPCGOSQ6~m1lIJGObENMbxgY
zd`g(B+2~ZOl~ti$5{;G5iQtsKhzOs<nect)eDBFFfA>xHlK*k;x!u1QobwmcfE+b^
zczo}A|8-XCzLj4+n|SHk{n4mic$$>>kzKym<B*Vk)U<=Kp5H`U{=6L|{Wc1DmWcvG
z76FVb02yfmT5$S-f4_s{{ziu(n;nE)vhI4s17gyIJ1qk(jyu7HZ3lA%xtvj)uE0pb
z$53nM?6&KW^-Z{ri#Fj5p`{kAt=n$cB6l3jBFD@@19IxL9zw`xtdg$8LlAg=q1{28
zrW+#4D+#S48%eHS(G5iAVIj~13LO=IVY0&vbVM52T^rF6(*yzx3({MDR0%04*|42u
z2kyc74pk$D%$$vdh>qe^LJ0ZG7X5M#F6I*C?GzXSG@cDl2fPncQ;69=?`MKx80Oyc
z9B;|BU2{zuQ)dbV&Js%+lfN=#)pVIV;6G{<gX4%9U>kbZ#&Nx-i*)4_an>N&6Rd6+
zI@DnAgic;g(t#T0WVK=NDa_GVIQn#<fIx{T!*ObvwI|*}lvAOg$NmA!kj;2qj|yh!
zX3nG1z=PDg8a0li5EfPCA#5NgSsa5-$boD!LCLPANZb86oIwZg!$H0TWG)1949wv}
z%n%5UzDSI@Rs|ErBNK2eFCN1M9Qzd;CjYDrIQIKKO#MY44mk%#@ZbKTacs|tiGdUB
c@tk1)B`4Vb#EApW?>oVA@DG+o@4h6y0>WY85C8xG

delta 135
zcmbQvcaN1TIM^lR9uortW7tG4X>Nb5nD}6)_~<4#t%(LAjJ^|Hw{uC>PEKQ(G&v)I
zVKOVD5|2#v<i2b!mdWkej0~HN7+n~(Wc<Pm3^?K)U4j@z1mazSeOZ?HIXn7fWM*YE
dMmNa;WevfyTuhS-Sw(n20!9!4=E=X=WB}s5Bn$um

diff --git a/tests/data/acpi/q35/SSDT.dimmpxm b/tests/data/acpi/q35/SSDT.dimmpxm
index 98e6f0e3f3bb02dd419e36bdd1db9b94c728c406..9ea4e0d0ceaa8a5cbd706afb6d49de853fafe654 100644
GIT binary patch
literal 1815
zcmdUw%WD%+6vppl(q?j#N+yZ4_+tJ8(=J4Cp_55srp-*k%rq9JFfU2kq{_wCg}Xi$
ztr5g@s0$I9lvx(s3+~*ya^=3@R@?|K)O)5cETUCVG>dc3J@?GX?|$Ee=z7T*O(4YV
z6zft|7u04+RusBN2o+}<60Di(?O97NTIOo~7CqNEt16d9M5!Sc3gZ(fKXqD_#M%f^
z%8J-Bm(_+`XHsJr$7!yK3TmZL##~83omGvBESM|j;DD``YGpwyH+7*htx7^g)UGdo
zN|-Cz=v1qfRiR!IjpfgY;Ecb32%p25@LlF&|Jg2o|4sIa|8e$(J-8fP@E6j695sA+
z`rzi^byn)Vm0vxdc<I`M(WqY7NlN_4F5mBSNJu|v*}+)fZ=p?p&JL1(2ZcP#M1dg-
z07mA4jC24kIQz(d*u`;wy~~h|E<!F@b3Nh#F=@e_mVg$=o#4`zgE@j+&L~b-U?kyV
zsJ(rD%XP@w23*HQ8*qluVjI@>T{mo#dk$uiW9Fa%IdxhOA>=bwNmt?_2s}66=^{?k
z4H4y)gjSJ_Bv-HK1|oB?5a<Dgjtjsr*<l7cB8}{xjp&GJ0s)2v=}kYXgcOvl+s={$
z_uzbosu4qG&c;GSM{z14g#1;DemMsha|!ac3k(4o&xWT1-iN6v#2lOtGQltmbMJVL
zx9HlgxhjpR%|d~*#FED3uMJr>UFHt?j~m6{IAZp&4ZUaMxL(smx^jv*W034ARyPbC
zYOr@gCod=IKn-)U+A#PO=IARNeR@zpphT46IJL~$7jHhwsZh{k|A1x4X1tz91v7Lr
z=TT|kLF!$N8plxxi>mS%HjtAnjzK5nKsK46WH(Zz<Nh4Zpo0(KAYTMB7Xmo}=I{|_
z2n5GpB*t8=f(hf12{@J)Pv8QM{fZ5ff7S*Z`+Xm#{-O^@oO?#_-~OU;Y~P8AJtx?c
bIl=x*PO$%p6NjANcY@{MA1saDe@T1=QL*6=

delta 135
zcmbQvcaN1TIM^lR9uortquWF-X>Nb5nD}6)_~<4#t%(LAjJ^|Hw{uC>PEKQ(G&v)I
zVKOVD5|2#v<i2b!mdWkej0~HN7+n~(Wc<Pm3^?K)U4j@z1mazSeOZ?HIXn7fWM*YE
dMmNa;WevfyTuhS-Sw(n20!9!4=E=X=WB}VOBm4jW

diff --git a/tests/data/acpi/virt/SSDT.memhp b/tests/data/acpi/virt/SSDT.memhp
index 4c363a6d95a7e2e826568c85f5719127748e7932..2fcfc5fda955dc4ba78a5f4116eed99ec7202fbd 100644
GIT binary patch
literal 1817
zcmdUw%WD%+6voeF(q?kgN+yX|pX5I<?Lq_>I+-M9YF^Att1yLmNzx{zTx?yqYuZI?
z1Th`zLPREImV$f1om*F~+!x#nBDfImne>50v<ix5aqi=sd**k)Z<1(QK|dV>;JW#`
zT-LI+*@9B7vNZtYZ#85blhaFcrj|F2HED_ERlzJPg>gx4D7oC^6lAI=YqC@sr`2Lr
zY8tDxhEj(yfOwol*-DkF&`dzm=q513*YuK-HM-|8!Iuk_Vpc8AYT43EP0p4i_4?}P
zn9)szS+!EEW$Oh=U;PsE&v=VK#WYC<?~@n%&vwQ8Z&O#iPjfG=!Dat~x5%Ez4GTUz
zeQ<m5CMEaYDX(8ly>hI<sqkR#d0ZC9cg22Rfb7_lrsYrcy(VmV^Ohg?+7O657Y2?l
zA{bpjGTKHo|Li00VTVM$jSfdQI{*${cU<ZsF>T_VR*;rNZU54?O#}*#pyy{w*piA7
zXziTec5J%7iPv#p6LSE|EgV~SouEbU*@Oq%NMixmb+#X1$Ry2zR*eQI@%%!&1A?mQ
z5)(b9mJ~fM)JvKUbRGwaHK@>00XQx-%3(#Qk=3(+wV0t8u$e!$<%K%|1Em|5y<+1%
z1kYw_M8HT}BpcE(oKk=xbJb)I7w}>Z1$QUMQKWG#Jnf4fc1-}cpC2UsQEcYiacy_m
zu^eMfo~X^{a)K!pR4#LE#H?tFaKL}mD2_yd8)EBP&mvK+qDgENs?sf;rcUshu4}l5
zYdf$rN}M%RiJ;UPx|kpWdx^tO4>tgn2xlb9EHn4STL|Y&D6Fx5z%pWSZqH?c>6(#t
znKXEic~@b^2@JwysyxO8&S{E7uoCA;HXUPRHxsn&{2WeajrZYjzKmoJ26_z4(L>At
z368!<oUv9$6DC6wNF*a2!vzxg6&t4itPLdc`##+Gi#{B34vi4M{YI0>o)c3;C-~zz
a!T(E6@c)Su2b|w`f*0^VjQvV}l)nP3li&~l

delta 150
zcmbQq_kfivIM^lR0TTlQqx3{BX>LdDnD}6)_~<4Nt%(LAjQ$f{cXCPAPEKQ(G&v)I
zVKO_T5|2#v<i2b!mdWkej0~Gi7+n~>Wc<Pm3^?K)U4j@z1mazSeOZ?HIXn7fWM*YE
eMmNa;WevfyTudT@sM1_a5P2hrJoDtgY%%~CTPTVE

-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 07/83] virtio-crypto: Support asynchronous mode
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (5 preceding siblings ...)
  2022-11-07 22:47 ` [PULL v4 06/83] test/acpi/bios-tables-test: SSDT: update golden master binaries Michael S. Tsirkin
@ 2022-11-07 22:47 ` Michael S. Tsirkin
  2022-11-07 22:48 ` [PULL v4 08/83] crypto: Support DER encodings Michael S. Tsirkin
                   ` (77 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:47 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Lei He, Gonglei (Arei)

From: Lei He <helei.sig11@bytedance.com>

virtio-crypto: Modify the current interface of virtio-crypto
device to support asynchronous mode.

Signed-off-by: lei he <helei.sig11@bytedance.com>
Message-Id: <20221008085030.70212-2-helei.sig11@bytedance.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/sysemu/cryptodev.h      |  60 ++++--
 backends/cryptodev-builtin.c    |  69 +++++--
 backends/cryptodev-vhost-user.c |  53 +++--
 backends/cryptodev.c            |  44 +++--
 hw/virtio/virtio-crypto.c       | 339 ++++++++++++++++++--------------
 5 files changed, 347 insertions(+), 218 deletions(-)

diff --git a/include/sysemu/cryptodev.h b/include/sysemu/cryptodev.h
index 37c3a360fd..32e9f4cf8a 100644
--- a/include/sysemu/cryptodev.h
+++ b/include/sysemu/cryptodev.h
@@ -113,6 +113,7 @@ typedef struct CryptoDevBackendSessionInfo {
         CryptoDevBackendSymSessionInfo sym_sess_info;
         CryptoDevBackendAsymSessionInfo asym_sess_info;
     } u;
+    uint64_t session_id;
 } CryptoDevBackendSessionInfo;
 
 /**
@@ -188,21 +189,30 @@ typedef struct CryptoDevBackendOpInfo {
     } u;
 } CryptoDevBackendOpInfo;
 
+typedef void (*CryptoDevCompletionFunc) (void *opaque, int ret);
 struct CryptoDevBackendClass {
     ObjectClass parent_class;
 
     void (*init)(CryptoDevBackend *backend, Error **errp);
     void (*cleanup)(CryptoDevBackend *backend, Error **errp);
 
-    int64_t (*create_session)(CryptoDevBackend *backend,
-                       CryptoDevBackendSessionInfo *sess_info,
-                       uint32_t queue_index, Error **errp);
+    int (*create_session)(CryptoDevBackend *backend,
+                          CryptoDevBackendSessionInfo *sess_info,
+                          uint32_t queue_index,
+                          CryptoDevCompletionFunc cb,
+                          void *opaque);
+
     int (*close_session)(CryptoDevBackend *backend,
-                           uint64_t session_id,
-                           uint32_t queue_index, Error **errp);
+                         uint64_t session_id,
+                         uint32_t queue_index,
+                         CryptoDevCompletionFunc cb,
+                         void *opaque);
+
     int (*do_op)(CryptoDevBackend *backend,
-                     CryptoDevBackendOpInfo *op_info,
-                     uint32_t queue_index, Error **errp);
+                 CryptoDevBackendOpInfo *op_info,
+                 uint32_t queue_index,
+                 CryptoDevCompletionFunc cb,
+                 void *opaque);
 };
 
 typedef enum CryptoDevBackendOptionsType {
@@ -303,15 +313,20 @@ void cryptodev_backend_cleanup(
  * @sess_info: parameters needed by session creating
  * @queue_index: queue index of cryptodev backend client
  * @errp: pointer to a NULL-initialized error object
+ * @cb: callback when session create is compeleted
+ * @opaque: parameter passed to callback
  *
- * Create a session for symmetric/symmetric algorithms
+ * Create a session for symmetric/asymmetric algorithms
  *
- * Returns: session id on success, or -1 on error
+ * Returns: 0 for success and cb will be called when creation is completed,
+ * negative value for error, and cb will not be called.
  */
-int64_t cryptodev_backend_create_session(
+int cryptodev_backend_create_session(
            CryptoDevBackend *backend,
            CryptoDevBackendSessionInfo *sess_info,
-           uint32_t queue_index, Error **errp);
+           uint32_t queue_index,
+           CryptoDevCompletionFunc cb,
+           void *opaque);
 
 /**
  * cryptodev_backend_close_session:
@@ -319,34 +334,43 @@ int64_t cryptodev_backend_create_session(
  * @session_id: the session id
  * @queue_index: queue index of cryptodev backend client
  * @errp: pointer to a NULL-initialized error object
+ * @cb: callback when session create is compeleted
+ * @opaque: parameter passed to callback
  *
  * Close a session for which was previously
  * created by cryptodev_backend_create_session()
  *
- * Returns: 0 on success, or Negative on error
+ * Returns: 0 for success and cb will be called when creation is completed,
+ * negative value for error, and cb will not be called.
  */
 int cryptodev_backend_close_session(
            CryptoDevBackend *backend,
            uint64_t session_id,
-           uint32_t queue_index, Error **errp);
+           uint32_t queue_index,
+           CryptoDevCompletionFunc cb,
+           void *opaque);
 
 /**
  * cryptodev_backend_crypto_operation:
  * @backend: the cryptodev backend object
- * @opaque: pointer to a VirtIOCryptoReq object
+ * @opaque1: pointer to a VirtIOCryptoReq object
  * @queue_index: queue index of cryptodev backend client
  * @errp: pointer to a NULL-initialized error object
+ * @cb: callbacks when operation is completed
+ * @opaque2: parameter passed to cb
  *
  * Do crypto operation, such as encryption and
  * decryption
  *
- * Returns: VIRTIO_CRYPTO_OK on success,
- *         or -VIRTIO_CRYPTO_* on error
+ * Returns: 0 for success and cb will be called when creation is completed,
+ * negative value for error, and cb will not be called.
  */
 int cryptodev_backend_crypto_operation(
                  CryptoDevBackend *backend,
-                 void *opaque,
-                 uint32_t queue_index, Error **errp);
+                 void *opaque1,
+                 uint32_t queue_index,
+                 CryptoDevCompletionFunc cb,
+                 void *opaque2);
 
 /**
  * cryptodev_backend_set_used:
diff --git a/backends/cryptodev-builtin.c b/backends/cryptodev-builtin.c
index 125cbad1d3..cda6ca3b71 100644
--- a/backends/cryptodev-builtin.c
+++ b/backends/cryptodev-builtin.c
@@ -355,42 +355,62 @@ static int cryptodev_builtin_create_akcipher_session(
     return index;
 }
 
-static int64_t cryptodev_builtin_create_session(
+static int cryptodev_builtin_create_session(
            CryptoDevBackend *backend,
            CryptoDevBackendSessionInfo *sess_info,
-           uint32_t queue_index, Error **errp)
+           uint32_t queue_index,
+           CryptoDevCompletionFunc cb,
+           void *opaque)
 {
     CryptoDevBackendBuiltin *builtin =
                       CRYPTODEV_BACKEND_BUILTIN(backend);
     CryptoDevBackendSymSessionInfo *sym_sess_info;
     CryptoDevBackendAsymSessionInfo *asym_sess_info;
+    int ret, status;
+    Error *local_error = NULL;
 
     switch (sess_info->op_code) {
     case VIRTIO_CRYPTO_CIPHER_CREATE_SESSION:
         sym_sess_info = &sess_info->u.sym_sess_info;
-        return cryptodev_builtin_create_cipher_session(
-                           builtin, sym_sess_info, errp);
+        ret = cryptodev_builtin_create_cipher_session(
+                    builtin, sym_sess_info, &local_error);
+        break;
 
     case VIRTIO_CRYPTO_AKCIPHER_CREATE_SESSION:
         asym_sess_info = &sess_info->u.asym_sess_info;
-        return cryptodev_builtin_create_akcipher_session(
-                           builtin, asym_sess_info, errp);
+        ret = cryptodev_builtin_create_akcipher_session(
+                           builtin, asym_sess_info, &local_error);
+        break;
 
     case VIRTIO_CRYPTO_HASH_CREATE_SESSION:
     case VIRTIO_CRYPTO_MAC_CREATE_SESSION:
     default:
-        error_setg(errp, "Unsupported opcode :%" PRIu32 "",
+        error_setg(&local_error, "Unsupported opcode :%" PRIu32 "",
                    sess_info->op_code);
-        return -1;
+        return -VIRTIO_CRYPTO_NOTSUPP;
     }
 
-    return -1;
+    if (local_error) {
+        error_report_err(local_error);
+    }
+    if (ret < 0) {
+        status = -VIRTIO_CRYPTO_ERR;
+    } else {
+        sess_info->session_id = ret;
+        status = VIRTIO_CRYPTO_OK;
+    }
+    if (cb) {
+        cb(opaque, status);
+    }
+    return 0;
 }
 
 static int cryptodev_builtin_close_session(
            CryptoDevBackend *backend,
            uint64_t session_id,
-           uint32_t queue_index, Error **errp)
+           uint32_t queue_index,
+           CryptoDevCompletionFunc cb,
+           void *opaque)
 {
     CryptoDevBackendBuiltin *builtin =
                       CRYPTODEV_BACKEND_BUILTIN(backend);
@@ -407,6 +427,9 @@ static int cryptodev_builtin_close_session(
 
     g_free(session);
     builtin->sessions[session_id] = NULL;
+    if (cb) {
+        cb(opaque, VIRTIO_CRYPTO_OK);
+    }
     return 0;
 }
 
@@ -506,7 +529,9 @@ static int cryptodev_builtin_asym_operation(
 static int cryptodev_builtin_operation(
                  CryptoDevBackend *backend,
                  CryptoDevBackendOpInfo *op_info,
-                 uint32_t queue_index, Error **errp)
+                 uint32_t queue_index,
+                 CryptoDevCompletionFunc cb,
+                 void *opaque)
 {
     CryptoDevBackendBuiltin *builtin =
                       CRYPTODEV_BACKEND_BUILTIN(backend);
@@ -514,11 +539,12 @@ static int cryptodev_builtin_operation(
     CryptoDevBackendSymOpInfo *sym_op_info;
     CryptoDevBackendAsymOpInfo *asym_op_info;
     enum CryptoDevBackendAlgType algtype = op_info->algtype;
-    int ret = -VIRTIO_CRYPTO_ERR;
+    int status = -VIRTIO_CRYPTO_ERR;
+    Error *local_error = NULL;
 
     if (op_info->session_id >= MAX_NUM_SESSIONS ||
               builtin->sessions[op_info->session_id] == NULL) {
-        error_setg(errp, "Cannot find a valid session id: %" PRIu64 "",
+        error_setg(&local_error, "Cannot find a valid session id: %" PRIu64 "",
                    op_info->session_id);
         return -VIRTIO_CRYPTO_INVSESS;
     }
@@ -526,14 +552,21 @@ static int cryptodev_builtin_operation(
     sess = builtin->sessions[op_info->session_id];
     if (algtype == CRYPTODEV_BACKEND_ALG_SYM) {
         sym_op_info = op_info->u.sym_op_info;
-        ret = cryptodev_builtin_sym_operation(sess, sym_op_info, errp);
+        status = cryptodev_builtin_sym_operation(sess, sym_op_info,
+                                                 &local_error);
     } else if (algtype == CRYPTODEV_BACKEND_ALG_ASYM) {
         asym_op_info = op_info->u.asym_op_info;
-        ret = cryptodev_builtin_asym_operation(sess, op_info->op_code,
-                                               asym_op_info, errp);
+        status = cryptodev_builtin_asym_operation(sess, op_info->op_code,
+                                                  asym_op_info, &local_error);
     }
 
-    return ret;
+    if (local_error) {
+        error_report_err(local_error);
+    }
+    if (cb) {
+        cb(opaque, status);
+    }
+    return 0;
 }
 
 static void cryptodev_builtin_cleanup(
@@ -548,7 +581,7 @@ static void cryptodev_builtin_cleanup(
 
     for (i = 0; i < MAX_NUM_SESSIONS; i++) {
         if (builtin->sessions[i] != NULL) {
-            cryptodev_builtin_close_session(backend, i, 0, &error_abort);
+            cryptodev_builtin_close_session(backend, i, 0, NULL, NULL);
         }
     }
 
diff --git a/backends/cryptodev-vhost-user.c b/backends/cryptodev-vhost-user.c
index f9c5867e38..ab3028e045 100644
--- a/backends/cryptodev-vhost-user.c
+++ b/backends/cryptodev-vhost-user.c
@@ -259,13 +259,18 @@ static int64_t cryptodev_vhost_user_sym_create_session(
     return -1;
 }
 
-static int64_t cryptodev_vhost_user_create_session(
+static int cryptodev_vhost_user_create_session(
            CryptoDevBackend *backend,
            CryptoDevBackendSessionInfo *sess_info,
-           uint32_t queue_index, Error **errp)
+           uint32_t queue_index,
+           CryptoDevCompletionFunc cb,
+           void *opaque)
 {
     uint32_t op_code = sess_info->op_code;
     CryptoDevBackendSymSessionInfo *sym_sess_info;
+    int64_t ret;
+    Error *local_error = NULL;
+    int status;
 
     switch (op_code) {
     case VIRTIO_CRYPTO_CIPHER_CREATE_SESSION:
@@ -273,27 +278,42 @@ static int64_t cryptodev_vhost_user_create_session(
     case VIRTIO_CRYPTO_MAC_CREATE_SESSION:
     case VIRTIO_CRYPTO_AEAD_CREATE_SESSION:
         sym_sess_info = &sess_info->u.sym_sess_info;
-        return cryptodev_vhost_user_sym_create_session(backend, sym_sess_info,
-                   queue_index, errp);
-    default:
-        error_setg(errp, "Unsupported opcode :%" PRIu32 "",
-                   sess_info->op_code);
-        return -1;
+        ret = cryptodev_vhost_user_sym_create_session(backend, sym_sess_info,
+                   queue_index, &local_error);
+        break;
 
+    default:
+        error_setg(&local_error, "Unsupported opcode :%" PRIu32 "",
+                   sess_info->op_code);
+        return -VIRTIO_CRYPTO_NOTSUPP;
     }
 
-    return -1;
+    if (local_error) {
+        error_report_err(local_error);
+    }
+    if (ret < 0) {
+        status = -VIRTIO_CRYPTO_ERR;
+    } else {
+        sess_info->session_id = ret;
+        status = VIRTIO_CRYPTO_OK;
+    }
+    if (cb) {
+        cb(opaque, status);
+    }
+    return 0;
 }
 
 static int cryptodev_vhost_user_close_session(
            CryptoDevBackend *backend,
            uint64_t session_id,
-           uint32_t queue_index, Error **errp)
+           uint32_t queue_index,
+           CryptoDevCompletionFunc cb,
+           void *opaque)
 {
     CryptoDevBackendClient *cc =
                   backend->conf.peers.ccs[queue_index];
     CryptoDevBackendVhost *vhost_crypto;
-    int ret;
+    int ret = -1, status;
 
     vhost_crypto = cryptodev_vhost_user_get_vhost(cc, backend, queue_index);
     if (vhost_crypto) {
@@ -301,12 +321,17 @@ static int cryptodev_vhost_user_close_session(
         ret = dev->vhost_ops->vhost_crypto_close_session(dev,
                                                          session_id);
         if (ret < 0) {
-            return -1;
+            status = -VIRTIO_CRYPTO_ERR;
         } else {
-            return 0;
+            status = VIRTIO_CRYPTO_OK;
         }
+    } else {
+        status = -VIRTIO_CRYPTO_NOTSUPP;
     }
-    return -1;
+    if (cb) {
+        cb(opaque, status);
+    }
+    return 0;
 }
 
 static void cryptodev_vhost_user_cleanup(
diff --git a/backends/cryptodev.c b/backends/cryptodev.c
index 33eb4e1a70..54ee8c81f5 100644
--- a/backends/cryptodev.c
+++ b/backends/cryptodev.c
@@ -26,6 +26,7 @@
 #include "qapi/error.h"
 #include "qapi/visitor.h"
 #include "qemu/config-file.h"
+#include "qemu/error-report.h"
 #include "qom/object_interfaces.h"
 #include "hw/virtio/virtio-crypto.h"
 
@@ -72,69 +73,72 @@ void cryptodev_backend_cleanup(
     }
 }
 
-int64_t cryptodev_backend_create_session(
+int cryptodev_backend_create_session(
            CryptoDevBackend *backend,
            CryptoDevBackendSessionInfo *sess_info,
-           uint32_t queue_index, Error **errp)
+           uint32_t queue_index,
+           CryptoDevCompletionFunc cb,
+           void *opaque)
 {
     CryptoDevBackendClass *bc =
                       CRYPTODEV_BACKEND_GET_CLASS(backend);
 
     if (bc->create_session) {
-        return bc->create_session(backend, sess_info, queue_index, errp);
+        return bc->create_session(backend, sess_info, queue_index, cb, opaque);
     }
-
-    return -1;
+    return -VIRTIO_CRYPTO_NOTSUPP;
 }
 
 int cryptodev_backend_close_session(
            CryptoDevBackend *backend,
            uint64_t session_id,
-           uint32_t queue_index, Error **errp)
+           uint32_t queue_index,
+           CryptoDevCompletionFunc cb,
+           void *opaque)
 {
     CryptoDevBackendClass *bc =
                       CRYPTODEV_BACKEND_GET_CLASS(backend);
 
     if (bc->close_session) {
-        return bc->close_session(backend, session_id, queue_index, errp);
+        return bc->close_session(backend, session_id, queue_index, cb, opaque);
     }
-
-    return -1;
+    return -VIRTIO_CRYPTO_NOTSUPP;
 }
 
 static int cryptodev_backend_operation(
                  CryptoDevBackend *backend,
                  CryptoDevBackendOpInfo *op_info,
-                 uint32_t queue_index, Error **errp)
+                 uint32_t queue_index,
+                 CryptoDevCompletionFunc cb,
+                 void *opaque)
 {
     CryptoDevBackendClass *bc =
                       CRYPTODEV_BACKEND_GET_CLASS(backend);
 
     if (bc->do_op) {
-        return bc->do_op(backend, op_info, queue_index, errp);
+        return bc->do_op(backend, op_info, queue_index, cb, opaque);
     }
-
-    return -VIRTIO_CRYPTO_ERR;
+    return -VIRTIO_CRYPTO_NOTSUPP;
 }
 
 int cryptodev_backend_crypto_operation(
                  CryptoDevBackend *backend,
-                 void *opaque,
-                 uint32_t queue_index, Error **errp)
+                 void *opaque1,
+                 uint32_t queue_index,
+                 CryptoDevCompletionFunc cb, void *opaque2)
 {
-    VirtIOCryptoReq *req = opaque;
+    VirtIOCryptoReq *req = opaque1;
     CryptoDevBackendOpInfo *op_info = &req->op_info;
     enum CryptoDevBackendAlgType algtype = req->flags;
 
     if ((algtype != CRYPTODEV_BACKEND_ALG_SYM)
         && (algtype != CRYPTODEV_BACKEND_ALG_ASYM)) {
-        error_setg(errp, "Unsupported cryptodev alg type: %" PRIu32 "",
-                   algtype);
-
+        error_report("Unsupported cryptodev alg type: %" PRIu32 "", algtype);
         return -VIRTIO_CRYPTO_NOTSUPP;
     }
 
-    return cryptodev_backend_operation(backend, op_info, queue_index, errp);
+    return cryptodev_backend_operation(backend, op_info, queue_index,
+                                       cb, opaque2);
 }
 
 static void
diff --git a/hw/virtio/virtio-crypto.c b/hw/virtio/virtio-crypto.c
index df4bde210b..97da74e719 100644
--- a/hw/virtio/virtio-crypto.c
+++ b/hw/virtio/virtio-crypto.c
@@ -27,6 +27,39 @@
 
 #define VIRTIO_CRYPTO_VM_VERSION 1
 
+typedef struct VirtIOCryptoSessionReq {
+    VirtIODevice *vdev;
+    VirtQueue *vq;
+    VirtQueueElement *elem;
+    CryptoDevBackendSessionInfo info;
+    CryptoDevCompletionFunc cb;
+} VirtIOCryptoSessionReq;
+
+static void virtio_crypto_free_create_session_req(VirtIOCryptoSessionReq *sreq)
+{
+    switch (sreq->info.op_code) {
+    case VIRTIO_CRYPTO_CIPHER_CREATE_SESSION:
+        g_free(sreq->info.u.sym_sess_info.cipher_key);
+        g_free(sreq->info.u.sym_sess_info.auth_key);
+        break;
+
+    case VIRTIO_CRYPTO_AKCIPHER_CREATE_SESSION:
+        g_free(sreq->info.u.asym_sess_info.key);
+        break;
+
+    case VIRTIO_CRYPTO_CIPHER_DESTROY_SESSION:
+    case VIRTIO_CRYPTO_HASH_DESTROY_SESSION:
+    case VIRTIO_CRYPTO_MAC_DESTROY_SESSION:
+    case VIRTIO_CRYPTO_AEAD_DESTROY_SESSION:
+    case VIRTIO_CRYPTO_AKCIPHER_DESTROY_SESSION:
+        break;
+
+    default:
+        error_report("Unknown opcode: %u", sreq->info.op_code);
+    }
+    g_free(sreq);
+}
+
 /*
  * Transfer virtqueue index to crypto queue index.
  * The control virtqueue is after the data virtqueues
@@ -75,27 +108,24 @@ virtio_crypto_cipher_session_helper(VirtIODevice *vdev,
     return 0;
 }
 
-static int64_t
+static int
 virtio_crypto_create_sym_session(VirtIOCrypto *vcrypto,
                struct virtio_crypto_sym_create_session_req *sess_req,
                uint32_t queue_id,
                uint32_t opcode,
-               struct iovec *iov, unsigned int out_num)
+               struct iovec *iov, unsigned int out_num,
+               VirtIOCryptoSessionReq *sreq)
 {
     VirtIODevice *vdev = VIRTIO_DEVICE(vcrypto);
-    CryptoDevBackendSessionInfo info;
-    CryptoDevBackendSymSessionInfo *sym_info;
-    int64_t session_id;
+    CryptoDevBackendSymSessionInfo *sym_info = &sreq->info.u.sym_sess_info;
     int queue_index;
     uint32_t op_type;
-    Error *local_err = NULL;
     int ret;
 
-    memset(&info, 0, sizeof(info));
     op_type = ldl_le_p(&sess_req->op_type);
-    info.op_code = opcode;
+    sreq->info.op_code = opcode;
 
-    sym_info = &info.u.sym_sess_info;
+    sym_info = &sreq->info.u.sym_sess_info;
     sym_info->op_type = op_type;
 
     if (op_type == VIRTIO_CRYPTO_SYM_OP_CIPHER) {
@@ -103,7 +133,7 @@ virtio_crypto_create_sym_session(VirtIOCrypto *vcrypto,
                            &sess_req->u.cipher.para,
                            &iov, &out_num);
         if (ret < 0) {
-            goto err;
+            return ret;
         }
     } else if (op_type == VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING) {
         size_t s;
@@ -112,7 +142,7 @@ virtio_crypto_create_sym_session(VirtIOCrypto *vcrypto,
                            &sess_req->u.chain.para.cipher_param,
                            &iov, &out_num);
         if (ret < 0) {
-            goto err;
+            return ret;
         }
         /* hash part */
         sym_info->alg_chain_order = ldl_le_p(
@@ -129,8 +159,7 @@ virtio_crypto_create_sym_session(VirtIOCrypto *vcrypto,
             if (sym_info->auth_key_len > vcrypto->conf.max_auth_key_len) {
                 error_report("virtio-crypto length of auth key is too big: %u",
                              sym_info->auth_key_len);
-                ret = -VIRTIO_CRYPTO_ERR;
-                goto err;
+                return -VIRTIO_CRYPTO_ERR;
             }
             /* get auth key */
             if (sym_info->auth_key_len > 0) {
@@ -140,8 +169,7 @@ virtio_crypto_create_sym_session(VirtIOCrypto *vcrypto,
                 if (unlikely(s != sym_info->auth_key_len)) {
                     virtio_error(vdev,
                           "virtio-crypto authenticated key incorrect");
-                    ret = -EFAULT;
-                    goto err;
+                    return -EFAULT;
                 }
                 iov_discard_front(&iov, &out_num, sym_info->auth_key_len);
             }
@@ -153,49 +181,30 @@ virtio_crypto_create_sym_session(VirtIOCrypto *vcrypto,
         } else {
             /* VIRTIO_CRYPTO_SYM_HASH_MODE_NESTED */
             error_report("unsupported hash mode");
-            ret = -VIRTIO_CRYPTO_NOTSUPP;
-            goto err;
+            return -VIRTIO_CRYPTO_NOTSUPP;
         }
     } else {
         /* VIRTIO_CRYPTO_SYM_OP_NONE */
         error_report("unsupported cipher op_type: VIRTIO_CRYPTO_SYM_OP_NONE");
-        ret = -VIRTIO_CRYPTO_NOTSUPP;
-        goto err;
+        return -VIRTIO_CRYPTO_NOTSUPP;
     }
 
     queue_index = virtio_crypto_vq2q(queue_id);
-    session_id = cryptodev_backend_create_session(
-                                     vcrypto->cryptodev,
-                                     &info, queue_index, &local_err);
-    if (session_id >= 0) {
-        ret = session_id;
-    } else {
-        if (local_err) {
-            error_report_err(local_err);
-        }
-        ret = -VIRTIO_CRYPTO_ERR;
-    }
-
-err:
-    g_free(sym_info->cipher_key);
-    g_free(sym_info->auth_key);
-    return ret;
+    return cryptodev_backend_create_session(vcrypto->cryptodev, &sreq->info,
+                                            queue_index, sreq->cb, sreq);
 }
 
-static int64_t
+static int
 virtio_crypto_create_asym_session(VirtIOCrypto *vcrypto,
                struct virtio_crypto_akcipher_create_session_req *sess_req,
                uint32_t queue_id, uint32_t opcode,
-               struct iovec *iov, unsigned int out_num)
+               struct iovec *iov, unsigned int out_num,
+               VirtIOCryptoSessionReq *sreq)
 {
     VirtIODevice *vdev = VIRTIO_DEVICE(vcrypto);
-    CryptoDevBackendSessionInfo info = {0};
-    CryptoDevBackendAsymSessionInfo *asym_info;
-    int64_t session_id;
+    CryptoDevBackendAsymSessionInfo *asym_info = &sreq->info.u.asym_sess_info;
     int queue_index;
     uint32_t algo, keytype, keylen;
-    g_autofree uint8_t *key = NULL;
-    Error *local_err = NULL;
 
     algo = ldl_le_p(&sess_req->para.algo);
     keytype = ldl_le_p(&sess_req->para.keytype);
@@ -208,20 +217,19 @@ virtio_crypto_create_asym_session(VirtIOCrypto *vcrypto,
     }
 
     if (keylen) {
-        key = g_malloc(keylen);
-        if (iov_to_buf(iov, out_num, 0, key, keylen) != keylen) {
+        asym_info->key = g_malloc(keylen);
+        if (iov_to_buf(iov, out_num, 0, asym_info->key, keylen) != keylen) {
             virtio_error(vdev, "virtio-crypto asym key incorrect");
             return -EFAULT;
         }
         iov_discard_front(&iov, &out_num, keylen);
     }
 
-    info.op_code = opcode;
-    asym_info = &info.u.asym_sess_info;
+    sreq->info.op_code = opcode;
+    asym_info = &sreq->info.u.asym_sess_info;
     asym_info->algo = algo;
     asym_info->keytype = keytype;
     asym_info->keylen = keylen;
-    asym_info->key = key;
     switch (asym_info->algo) {
     case VIRTIO_CRYPTO_AKCIPHER_RSA:
         asym_info->u.rsa.padding_algo =
@@ -237,45 +245,95 @@ virtio_crypto_create_asym_session(VirtIOCrypto *vcrypto,
     }
 
     queue_index = virtio_crypto_vq2q(queue_id);
-    session_id = cryptodev_backend_create_session(vcrypto->cryptodev, &info,
-                     queue_index, &local_err);
-    if (session_id < 0) {
-        if (local_err) {
-            error_report_err(local_err);
-        }
-        return -VIRTIO_CRYPTO_ERR;
-    }
-
-    return session_id;
+    return cryptodev_backend_create_session(vcrypto->cryptodev, &sreq->info,
+                                            queue_index, sreq->cb, sreq);
 }
 
-static uint8_t
+static int
 virtio_crypto_handle_close_session(VirtIOCrypto *vcrypto,
          struct virtio_crypto_destroy_session_req *close_sess_req,
-         uint32_t queue_id)
+         uint32_t queue_id,
+         VirtIOCryptoSessionReq *sreq)
 {
-    int ret;
     uint64_t session_id;
-    uint32_t status;
-    Error *local_err = NULL;
 
     session_id = ldq_le_p(&close_sess_req->session_id);
     DPRINTF("close session, id=%" PRIu64 "\n", session_id);
 
-    ret = cryptodev_backend_close_session(
-              vcrypto->cryptodev, session_id, queue_id, &local_err);
-    if (ret == 0) {
-        status = VIRTIO_CRYPTO_OK;
+    return cryptodev_backend_close_session(
+                vcrypto->cryptodev, session_id, queue_id, sreq->cb, sreq);
+}
+
+static void virtio_crypto_create_session_completion(void *opaque, int ret)
+{
+    VirtIOCryptoSessionReq *sreq = (VirtIOCryptoSessionReq *)opaque;
+    VirtQueue *vq = sreq->vq;
+    VirtQueueElement *elem = sreq->elem;
+    VirtIODevice *vdev = sreq->vdev;
+    struct virtio_crypto_session_input input;
+    struct iovec *in_iov = elem->in_sg;
+    unsigned in_num = elem->in_num;
+    size_t s;
+
+    memset(&input, 0, sizeof(input));
+    /* Serious errors, need to reset virtio crypto device */
+    if (ret == -EFAULT) {
+        virtqueue_detach_element(vq, elem, 0);
+        goto out;
+    } else if (ret == -VIRTIO_CRYPTO_NOTSUPP) {
+        stl_le_p(&input.status, VIRTIO_CRYPTO_NOTSUPP);
+    } else if (ret == -VIRTIO_CRYPTO_KEY_REJECTED) {
+        stl_le_p(&input.status, VIRTIO_CRYPTO_KEY_REJECTED);
+    } else if (ret != VIRTIO_CRYPTO_OK) {
+        stl_le_p(&input.status, VIRTIO_CRYPTO_ERR);
     } else {
-        if (local_err) {
-            error_report_err(local_err);
-        } else {
-            error_report("destroy session failed");
-        }
-        status = VIRTIO_CRYPTO_ERR;
+        /* Set the session id */
+        stq_le_p(&input.session_id, sreq->info.session_id);
+        stl_le_p(&input.status, VIRTIO_CRYPTO_OK);
     }
 
-    return status;
+    s = iov_from_buf(in_iov, in_num, 0, &input, sizeof(input));
+    if (unlikely(s != sizeof(input))) {
+        virtio_error(vdev, "virtio-crypto input incorrect");
+        virtqueue_detach_element(vq, elem, 0);
+        goto out;
+    }
+    virtqueue_push(vq, elem, sizeof(input));
+    virtio_notify(vdev, vq);
+
+out:
+    g_free(elem);
+    virtio_crypto_free_create_session_req(sreq);
+}
+
+static void virtio_crypto_destroy_session_completion(void *opaque, int ret)
+{
+    VirtIOCryptoSessionReq *sreq = (VirtIOCryptoSessionReq *)opaque;
+    VirtQueue *vq = sreq->vq;
+    VirtQueueElement *elem = sreq->elem;
+    VirtIODevice *vdev = sreq->vdev;
+    struct iovec *in_iov = elem->in_sg;
+    unsigned in_num = elem->in_num;
+    uint8_t status;
+    size_t s;
+
+    if (ret < 0) {
+        status = VIRTIO_CRYPTO_ERR;
+    } else {
+        status = VIRTIO_CRYPTO_OK;
+    }
+    s = iov_from_buf(in_iov, in_num, 0, &status, sizeof(status));
+    if (unlikely(s != sizeof(status))) {
+        virtio_error(vdev, "virtio-crypto status incorrect");
+        virtqueue_detach_element(vq, elem, 0);
+        goto out;
+    }
+    virtqueue_push(vq, elem, sizeof(status));
+    virtio_notify(vdev, vq);
+
+out:
+    g_free(elem);
+    g_free(sreq);
 }
 
 static void virtio_crypto_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq)
@@ -283,16 +341,16 @@ static void virtio_crypto_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq)
     VirtIOCrypto *vcrypto = VIRTIO_CRYPTO(vdev);
     struct virtio_crypto_op_ctrl_req ctrl;
     VirtQueueElement *elem;
-    struct iovec *in_iov;
-    struct iovec *out_iov;
-    unsigned in_num;
+    VirtIOCryptoSessionReq *sreq;
     unsigned out_num;
+    unsigned in_num;
     uint32_t queue_id;
     uint32_t opcode;
     struct virtio_crypto_session_input input;
-    int64_t session_id;
-    uint8_t status;
     size_t s;
+    int ret;
+    struct iovec *out_iov;
+    struct iovec *in_iov;
 
     for (;;) {
         g_autofree struct iovec *out_iov_copy = NULL;
@@ -327,44 +385,34 @@ static void virtio_crypto_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq)
         opcode = ldl_le_p(&ctrl.header.opcode);
         queue_id = ldl_le_p(&ctrl.header.queue_id);
 
-        memset(&input, 0, sizeof(input));
+        sreq = g_new0(VirtIOCryptoSessionReq, 1);
+        sreq->vdev = vdev;
+        sreq->vq = vq;
+        sreq->elem = elem;
+
         switch (opcode) {
         case VIRTIO_CRYPTO_CIPHER_CREATE_SESSION:
-            session_id = virtio_crypto_create_sym_session(vcrypto,
-                             &ctrl.u.sym_create_session,
-                             queue_id, opcode,
-                             out_iov, out_num);
-            goto check_session;
+            sreq->cb = virtio_crypto_create_session_completion;
+            ret = virtio_crypto_create_sym_session(vcrypto,
+                            &ctrl.u.sym_create_session,
+                            queue_id, opcode,
+                            out_iov, out_num,
+                            sreq);
+            if (ret < 0) {
+                virtio_crypto_create_session_completion(sreq, ret);
+            }
+            break;
 
         case VIRTIO_CRYPTO_AKCIPHER_CREATE_SESSION:
-            session_id = virtio_crypto_create_asym_session(vcrypto,
+            sreq->cb = virtio_crypto_create_session_completion;
+            ret = virtio_crypto_create_asym_session(vcrypto,
                              &ctrl.u.akcipher_create_session,
                              queue_id, opcode,
-                             out_iov, out_num);
-
-check_session:
-            /* Serious errors, need to reset virtio crypto device */
-            if (session_id == -EFAULT) {
-                virtqueue_detach_element(vq, elem, 0);
-                break;
-            } else if (session_id == -VIRTIO_CRYPTO_NOTSUPP) {
-                stl_le_p(&input.status, VIRTIO_CRYPTO_NOTSUPP);
-            } else if (session_id == -VIRTIO_CRYPTO_ERR) {
-                stl_le_p(&input.status, VIRTIO_CRYPTO_ERR);
-            } else {
-                /* Set the session id */
-                stq_le_p(&input.session_id, session_id);
-                stl_le_p(&input.status, VIRTIO_CRYPTO_OK);
+                             out_iov, out_num,
+                             sreq);
+            if (ret < 0) {
+                virtio_crypto_create_session_completion(sreq, ret);
             }
-
-            s = iov_from_buf(in_iov, in_num, 0, &input, sizeof(input));
-            if (unlikely(s != sizeof(input))) {
-                virtio_error(vdev, "virtio-crypto input incorrect");
-                virtqueue_detach_element(vq, elem, 0);
-                break;
-            }
-            virtqueue_push(vq, elem, sizeof(input));
-            virtio_notify(vdev, vq);
             break;
 
         case VIRTIO_CRYPTO_CIPHER_DESTROY_SESSION:
@@ -372,37 +420,36 @@ check_session:
         case VIRTIO_CRYPTO_MAC_DESTROY_SESSION:
         case VIRTIO_CRYPTO_AEAD_DESTROY_SESSION:
         case VIRTIO_CRYPTO_AKCIPHER_DESTROY_SESSION:
-            status = virtio_crypto_handle_close_session(vcrypto,
-                   &ctrl.u.destroy_session, queue_id);
-            /* The status only occupy one byte, we can directly use it */
-            s = iov_from_buf(in_iov, in_num, 0, &status, sizeof(status));
-            if (unlikely(s != sizeof(status))) {
-                virtio_error(vdev, "virtio-crypto status incorrect");
-                virtqueue_detach_element(vq, elem, 0);
-                break;
+            sreq->cb = virtio_crypto_destroy_session_completion;
+            ret = virtio_crypto_handle_close_session(vcrypto,
+                   &ctrl.u.destroy_session, queue_id,
+                   sreq);
+            if (ret < 0) {
+                virtio_crypto_destroy_session_completion(sreq, ret);
             }
-            virtqueue_push(vq, elem, sizeof(status));
-            virtio_notify(vdev, vq);
             break;
+
         case VIRTIO_CRYPTO_HASH_CREATE_SESSION:
         case VIRTIO_CRYPTO_MAC_CREATE_SESSION:
         case VIRTIO_CRYPTO_AEAD_CREATE_SESSION:
         default:
+            memset(&input, 0, sizeof(input));
             error_report("virtio-crypto unsupported ctrl opcode: %d", opcode);
             stl_le_p(&input.status, VIRTIO_CRYPTO_NOTSUPP);
             s = iov_from_buf(in_iov, in_num, 0, &input, sizeof(input));
             if (unlikely(s != sizeof(input))) {
                 virtio_error(vdev, "virtio-crypto input incorrect");
                 virtqueue_detach_element(vq, elem, 0);
-                break;
+            } else {
+                virtqueue_push(vq, elem, sizeof(input));
+                virtio_notify(vdev, vq);
             }
-            virtqueue_push(vq, elem, sizeof(input));
-            virtio_notify(vdev, vq);
+            g_free(sreq);
+            g_free(elem);
 
             break;
         } /* end switch case */
 
-        g_free(elem);
     } /* end for loop */
 }
 
@@ -448,6 +495,7 @@ static void virtio_crypto_free_request(VirtIOCryptoReq *req)
         }
     }
 
+    g_free(req->in_iov);
     g_free(req);
 }
 
@@ -458,6 +506,7 @@ virtio_crypto_sym_input_data_helper(VirtIODevice *vdev,
                 CryptoDevBackendSymOpInfo *sym_op_info)
 {
     size_t s, len;
+    struct iovec *in_iov = req->in_iov;
 
     if (status != VIRTIO_CRYPTO_OK) {
         return;
@@ -465,18 +514,18 @@ virtio_crypto_sym_input_data_helper(VirtIODevice *vdev,
 
     len = sym_op_info->src_len;
     /* Save the cipher result */
-    s = iov_from_buf(req->in_iov, req->in_num, 0, sym_op_info->dst, len);
+    s = iov_from_buf(in_iov, req->in_num, 0, sym_op_info->dst, len);
     if (s != len) {
         virtio_error(vdev, "virtio-crypto dest data incorrect");
         return;
     }
 
-    iov_discard_front(&req->in_iov, &req->in_num, len);
+    iov_discard_front(&in_iov, &req->in_num, len);
 
     if (sym_op_info->op_type ==
                       VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING) {
         /* Save the digest result */
-        s = iov_from_buf(req->in_iov, req->in_num, 0,
+        s = iov_from_buf(in_iov, req->in_num, 0,
                          sym_op_info->digest_result,
                          sym_op_info->digest_result_len);
         if (s != sym_op_info->digest_result_len) {
@@ -491,6 +540,7 @@ virtio_crypto_akcipher_input_data_helper(VirtIODevice *vdev,
         CryptoDevBackendAsymOpInfo *asym_op_info)
 {
     size_t s, len;
+    struct iovec *in_iov = req->in_iov;
 
     if (status != VIRTIO_CRYPTO_OK) {
         return;
@@ -501,23 +551,24 @@ virtio_crypto_akcipher_input_data_helper(VirtIODevice *vdev,
         return;
     }
 
-    s = iov_from_buf(req->in_iov, req->in_num, 0, asym_op_info->dst, len);
+    s = iov_from_buf(in_iov, req->in_num, 0, asym_op_info->dst, len);
     if (s != len) {
         virtio_error(vdev, "virtio-crypto asym dest data incorrect");
         return;
     }
 
-    iov_discard_front(&req->in_iov, &req->in_num, len);
+    iov_discard_front(&in_iov, &req->in_num, len);
 
     /* For akcipher, dst_len may be changed after operation */
     req->in_len = sizeof(struct virtio_crypto_inhdr) + asym_op_info->dst_len;
 }
 
-
-static void virtio_crypto_req_complete(VirtIOCryptoReq *req, uint8_t status)
+static void virtio_crypto_req_complete(void *opaque, int ret)
 {
+    VirtIOCryptoReq *req = (VirtIOCryptoReq *)opaque;
     VirtIOCrypto *vcrypto = req->vcrypto;
     VirtIODevice *vdev = VIRTIO_DEVICE(vcrypto);
+    uint8_t status = -ret;
 
     if (req->flags == CRYPTODEV_BACKEND_ALG_SYM) {
         virtio_crypto_sym_input_data_helper(vdev, req, status,
@@ -529,6 +580,7 @@ static void virtio_crypto_req_complete(VirtIOCryptoReq *req, uint8_t status)
     stb_p(&req->in->status, status);
     virtqueue_push(req->vq, &req->elem, req->in_len);
     virtio_notify(vdev, req->vq);
+    virtio_crypto_free_request(req);
 }
 
 static VirtIOCryptoReq *
@@ -773,9 +825,7 @@ virtio_crypto_handle_request(VirtIOCryptoReq *request)
     unsigned in_num;
     unsigned out_num;
     uint32_t opcode;
-    uint8_t status = VIRTIO_CRYPTO_ERR;
     CryptoDevBackendOpInfo *op_info = &request->op_info;
-    Error *local_err = NULL;
 
     if (elem->out_num < 1 || elem->in_num < 1) {
         virtio_error(vdev, "virtio-crypto dataq missing headers");
@@ -815,6 +865,8 @@ virtio_crypto_handle_request(VirtIOCryptoReq *request)
      */
     request->in_num = in_num;
     request->in_iov = in_iov;
+    /* now, we free the in_iov_copy inside virtio_crypto_free_request */
+    in_iov_copy = NULL;
 
     opcode = ldl_le_p(&req.header.opcode);
     op_info->session_id = ldq_le_p(&req.header.session_id);
@@ -843,23 +895,15 @@ check_result:
         if (ret == -EFAULT) {
             return -1;
         } else if (ret == -VIRTIO_CRYPTO_NOTSUPP) {
-            virtio_crypto_req_complete(request, VIRTIO_CRYPTO_NOTSUPP);
-            virtio_crypto_free_request(request);
+            virtio_crypto_req_complete(request, -VIRTIO_CRYPTO_NOTSUPP);
         } else {
-
-            /* Set request's parameter */
             ret = cryptodev_backend_crypto_operation(vcrypto->cryptodev,
-                                    request, queue_index, &local_err);
+                                    request, queue_index,
+                                    virtio_crypto_req_complete,
+                                    request);
             if (ret < 0) {
-                status = -ret;
-                if (local_err) {
-                    error_report_err(local_err);
-                }
-            } else { /* ret == VIRTIO_CRYPTO_OK */
-                status = ret;
+                virtio_crypto_req_complete(request, ret);
             }
-            virtio_crypto_req_complete(request, status);
-            virtio_crypto_free_request(request);
         }
         break;
 
@@ -870,8 +914,7 @@ check_result:
     default:
         error_report("virtio-crypto unsupported dataq opcode: %u",
                      opcode);
-        virtio_crypto_req_complete(request, VIRTIO_CRYPTO_NOTSUPP);
-        virtio_crypto_free_request(request);
+        virtio_crypto_req_complete(request, -VIRTIO_CRYPTO_NOTSUPP);
     }
 
     return 0;
@@ -1011,7 +1054,7 @@ static void virtio_crypto_device_realize(DeviceState *dev, Error **errp)
         vcrypto->vqs[i].vcrypto = vcrypto;
     }
 
-    vcrypto->ctrl_vq = virtio_add_queue(vdev, 64, virtio_crypto_handle_ctrl);
+    vcrypto->ctrl_vq = virtio_add_queue(vdev, 1024, virtio_crypto_handle_ctrl);
     if (!cryptodev_backend_is_ready(vcrypto->cryptodev)) {
         vcrypto->status &= ~VIRTIO_CRYPTO_S_HW_READY;
     } else {
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 08/83] crypto: Support DER encodings
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (6 preceding siblings ...)
  2022-11-07 22:47 ` [PULL v4 07/83] virtio-crypto: Support asynchronous mode Michael S. Tsirkin
@ 2022-11-07 22:48 ` Michael S. Tsirkin
  2022-11-07 22:48 ` [PULL v4 09/83] crypto: Support export akcipher to pkcs8 Michael S. Tsirkin
                   ` (76 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:48 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Lei He, Daniel P . Berrangé

From: Lei He <helei.sig11@bytedance.com>

Add encoding interfaces for DER encoding:
1. support decoding of 'bit string', 'octet string', 'object id'
and 'context specific tag' for DER encoder.
2. implemented a simple DER encoder.
3. add more testsuits for DER encoder.

Signed-off-by: lei he <helei.sig11@bytedance.com>
Message-Id: <20221008085030.70212-3-helei.sig11@bytedance.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
---
 crypto/der.h                 | 211 ++++++++++++++++++++++-
 crypto/der.c                 | 313 ++++++++++++++++++++++++++++++++---
 tests/unit/test-crypto-der.c | 126 +++++++++++---
 3 files changed, 600 insertions(+), 50 deletions(-)

diff --git a/crypto/der.h b/crypto/der.h
index e3d3aeacdc..0e895bbeec 100644
--- a/crypto/der.h
+++ b/crypto/der.h
@@ -22,6 +22,11 @@
 
 #include "qapi/error.h"
 
+typedef struct QCryptoEncodeContext QCryptoEncodeContext;
+
+/* rsaEncryption: 1.2.840.113549.1.1.1 */
+#define QCRYPTO_OID_rsaEncryption "\x2A\x86\x48\x86\xF7\x0D\x01\x01\x01"
+
 /* Simple decoder used to parse DER encoded rsa keys. */
 
 /**
@@ -47,14 +52,13 @@ typedef int (*QCryptoDERDecodeCb) (void *opaque, const uint8_t *value,
  * will be set to the rest length of data, if cb is not NULL, must
  * return 0 to make decode success, at last, the length of the data
  * part of the decoded INTEGER will be returned. Otherwise, -1 is
- * returned.
+ * returned and the valued of *data and *dlen keep unchanged.
  */
 int qcrypto_der_decode_int(const uint8_t **data,
                            size_t *dlen,
                            QCryptoDERDecodeCb cb,
                            void *opaque,
                            Error **errp);
-
 /**
  * qcrypto_der_decode_seq:
  *
@@ -70,7 +74,7 @@ int qcrypto_der_decode_int(const uint8_t **data,
  * will be set to the rest length of data, if cb is not NULL, must
  * return 0 to make decode success, at last, the length of the data
  * part of the decoded SEQUENCE will be returned. Otherwise, -1 is
- * returned.
+ * returned and the valued of *data and *dlen keep unchanged.
  */
 int qcrypto_der_decode_seq(const uint8_t **data,
                            size_t *dlen,
@@ -78,4 +82,205 @@ int qcrypto_der_decode_seq(const uint8_t **data,
                            void *opaque,
                            Error **errp);
 
+/**
+ * qcrypto_der_decode_oid:
+ *
+ * Decode OID from DER-encoded data, similar with der_decode_int.
+ *
+ * @data: pointer to address of input data
+ * @dlen: pointer to length of input data
+ * @cb: callback invoked when decode succeed, if cb equals NULL, no
+ * callback will be invoked
+ * @opaque: parameter passed to cb
+ *
+ * Returns: On success, *data points to rest data, and *dlen
+ * will be set to the rest length of data, if cb is not NULL, must
+ * return 0 to make decode success, at last, the length of the data
+ * part of the decoded OID will be returned. Otherwise, -1 is
+ * returned and the valued of *data and *dlen keep unchanged.
+ */
+int qcrypto_der_decode_oid(const uint8_t **data,
+                           size_t *dlen,
+                           QCryptoDERDecodeCb cb,
+                           void *opaque,
+                           Error **errp);
+
+/**
+ * qcrypto_der_decode_octet_str:
+ *
+ * Decode OCTET STRING from DER-encoded data, similar with der_decode_int.
+ *
+ * @data: pointer to address of input data
+ * @dlen: pointer to length of input data
+ * @cb: callback invoked when decode succeed, if cb equals NULL, no
+ * callback will be invoked
+ * @opaque: parameter passed to cb
+ *
+ * Returns: On success, *data points to rest data, and *dlen
+ * will be set to the rest length of data, if cb is not NULL, must
+ * return 0 to make decode success, at last, the length of the data
+ * part of the decoded OCTET STRING will be returned. Otherwise, -1 is
+ * returned and the valued of *data and *dlen keep unchanged.
+ */
+int qcrypto_der_decode_octet_str(const uint8_t **data,
+                                 size_t *dlen,
+                                 QCryptoDERDecodeCb cb,
+                                 void *opaque,
+                                 Error **errp);
+
+/**
+ * qcrypto_der_decode_bit_str:
+ *
+ * Decode BIT STRING from DER-encoded data, similar with der_decode_int.
+ *
+ * @data: pointer to address of input data
+ * @dlen: pointer to length of input data
+ * @cb: callback invoked when decode succeed, if cb equals NULL, no
+ * callback will be invoked
+ * @opaque: parameter passed to cb
+ *
+ * Returns: On success, *data points to rest data, and *dlen
+ * will be set to the rest length of data, if cb is not NULL, must
+ * return 0 to make decode success, at last, the length of the data
+ * part of the decoded BIT STRING will be returned. Otherwise, -1 is
+ * returned and the valued of *data and *dlen keep unchanged.
+ */
+int qcrypto_der_decode_bit_str(const uint8_t **data,
+                               size_t *dlen,
+                               QCryptoDERDecodeCb cb,
+                               void *opaque,
+                               Error **errp);
+
+
+/**
+ * qcrypto_der_decode_ctx_tag:
+ *
+ * Decode context specific tag
+ *
+ * @data: pointer to address of input data
+ * @dlen: pointer to length of input data
+ * @tag: expected value of context specific tag
+ * @cb: callback invoked when decode succeed, if cb equals NULL, no
+ * callback will be invoked
+ * @opaque: parameter passed to cb
+ *
+ * Returns: On success, *data points to rest data, and *dlen
+ * will be set to the rest length of data, if cb is not NULL, must
+ * return 0 to make decode success, at last, the length of the data
+ * part of the decoded BIT STRING will be returned. Otherwise, -1 is
+ * returned and the valued of *data and *dlen keep unchanged.
+ */
+int qcrypto_der_decode_ctx_tag(const uint8_t **data,
+                               size_t *dlen, int tag_id,
+                               QCryptoDERDecodeCb cb,
+                               void *opaque,
+                               Error **errp);
+
+/**
+ * qcrypto_der_encode_ctx_new:
+ *
+ * Allocate a context used for der encoding.
+ */
+QCryptoEncodeContext *qcrypto_der_encode_ctx_new(void);
+
+/**
+ * qcrypto_der_encode_seq_begin:
+ * @ctx: the encode context.
+ *
+ * Start encoding a SEQUENCE for ctx.
+ *
+ */
+void qcrypto_der_encode_seq_begin(QCryptoEncodeContext *ctx);
+
+/**
+ * qcrypto_der_encode_seq_begin:
+ * @ctx: the encode context.
+ *
+ * Finish uencoding a SEQUENCE for ctx.
+ *
+ */
+void qcrypto_der_encode_seq_end(QCryptoEncodeContext *ctx);
+
+
+/**
+ * qcrypto_der_encode_oid:
+ * @ctx: the encode context.
+ * @src: the source data of oid, note it should be already encoded, this
+ * function only add tag and length part for it.
+ *
+ * Encode an oid into ctx.
+ */
+void qcrypto_der_encode_oid(QCryptoEncodeContext *ctx,
+                            const uint8_t *src, size_t src_len);
+
+/**
+ * qcrypto_der_encode_int:
+ * @ctx: the encode context.
+ * @src: the source data of integer, note it should be already encoded, this
+ * function only add tag and length part for it.
+ *
+ * Encode an integer into ctx.
+ */
+void qcrypto_der_encode_int(QCryptoEncodeContext *ctx,
+                            const uint8_t *src, size_t src_len);
+
+/**
+ * qcrypto_der_encode_null:
+ * @ctx: the encode context.
+ *
+ * Encode a null into ctx.
+ */
+void qcrypto_der_encode_null(QCryptoEncodeContext *ctx);
+
+/**
+ * qcrypto_der_encode_octet_str:
+ * @ctx: the encode context.
+ * @src: the source data of the octet string.
+ *
+ * Encode a octet string into ctx.
+ */
+void qcrypto_der_encode_octet_str(QCryptoEncodeContext *ctx,
+                                  const uint8_t *src, size_t src_len);
+
+/**
+ * qcrypto_der_encode_octet_str_begin:
+ * @ctx: the encode context.
+ *
+ * Start encoding a octet string, All fields between
+ * qcrypto_der_encode_octet_str_begin and qcrypto_der_encode_octet_str_end
+ * are encoded as an octet string. This is useful when we need to encode a
+ * encoded SEQUNCE as OCTET STRING.
+ */
+void qcrypto_der_encode_octet_str_begin(QCryptoEncodeContext *ctx);
+
+/**
+ * qcrypto_der_encode_octet_str_end:
+ * @ctx: the encode context.
+ *
+ * Finish encoding a octet string, All fields between
+ * qcrypto_der_encode_octet_str_begin and qcrypto_der_encode_octet_str_end
+ * are encoded as an octet string. This is useful when we need to encode a
+ * encoded SEQUNCE as OCTET STRING.
+ */
+void qcrypto_der_encode_octet_str_end(QCryptoEncodeContext *ctx);
+
+/**
+ * qcrypto_der_encode_ctx_buffer_len:
+ * @ctx: the encode context.
+ *
+ * Compute the expected buffer size to save all encoded things.
+ */
+size_t qcrypto_der_encode_ctx_buffer_len(QCryptoEncodeContext *ctx);
+
+/**
+ * qcrypto_der_encode_ctx_flush_and_free:
+ * @ctx: the encode context.
+ * @dst: the distination to save the encoded data, the length of dst should
+ * not less than qcrypto_der_encode_cxt_buffer_len
+ *
+ * Flush all encoded data into dst, then free ctx.
+ */
+void qcrypto_der_encode_ctx_flush_and_free(QCryptoEncodeContext *ctx,
+                                           uint8_t *dst);
+
 #endif  /* QCRYPTO_ASN1_DECODER_H */
diff --git a/crypto/der.c b/crypto/der.c
index f877390bbb..dab3fe4f24 100644
--- a/crypto/der.c
+++ b/crypto/der.c
@@ -22,20 +22,93 @@
 #include "qemu/osdep.h"
 #include "crypto/der.h"
 
+typedef struct QCryptoDerEncodeNode {
+    uint8_t tag;
+    struct QCryptoDerEncodeNode *parent;
+    struct QCryptoDerEncodeNode *next;
+    /* for constructed type, data is null */
+    const uint8_t *data;
+    size_t dlen;
+} QCryptoDerEncodeNode;
+
+typedef struct QCryptoEncodeContext {
+    QCryptoDerEncodeNode root;
+    QCryptoDerEncodeNode *current_parent;
+    QCryptoDerEncodeNode *tail;
+} QCryptoEncodeContext;
+
 enum QCryptoDERTypeTag {
     QCRYPTO_DER_TYPE_TAG_BOOL = 0x1,
     QCRYPTO_DER_TYPE_TAG_INT = 0x2,
     QCRYPTO_DER_TYPE_TAG_BIT_STR = 0x3,
     QCRYPTO_DER_TYPE_TAG_OCT_STR = 0x4,
-    QCRYPTO_DER_TYPE_TAG_OCT_NULL = 0x5,
-    QCRYPTO_DER_TYPE_TAG_OCT_OID = 0x6,
+    QCRYPTO_DER_TYPE_TAG_NULL = 0x5,
+    QCRYPTO_DER_TYPE_TAG_OID = 0x6,
     QCRYPTO_DER_TYPE_TAG_SEQ = 0x10,
     QCRYPTO_DER_TYPE_TAG_SET = 0x11,
 };
 
-#define QCRYPTO_DER_CONSTRUCTED_MASK 0x20
+enum QCryptoDERTagClass {
+    QCRYPTO_DER_TAG_CLASS_UNIV = 0x0,
+    QCRYPTO_DER_TAG_CLASS_APPL = 0x1,
+    QCRYPTO_DER_TAG_CLASS_CONT = 0x2,
+    QCRYPTO_DER_TAG_CLASS_PRIV = 0x3,
+};
+
+enum QCryptoDERTagEnc {
+    QCRYPTO_DER_TAG_ENC_PRIM = 0x0,
+    QCRYPTO_DER_TAG_ENC_CONS = 0x1,
+};
+
+#define QCRYPTO_DER_TAG_ENC_MASK 0x20
+#define QCRYPTO_DER_TAG_ENC_SHIFT 5
+
+#define QCRYPTO_DER_TAG_CLASS_MASK 0xc0
+#define QCRYPTO_DER_TAG_CLASS_SHIFT 6
+
+#define QCRYPTO_DER_TAG_VAL_MASK 0x1f
 #define QCRYPTO_DER_SHORT_LEN_MASK 0x80
 
+#define QCRYPTO_DER_TAG(class, enc, val)           \
+    (((class) << QCRYPTO_DER_TAG_CLASS_SHIFT) |    \
+     ((enc) << QCRYPTO_DER_TAG_ENC_SHIFT) | (val))
+
+/**
+ * qcrypto_der_encode_length:
+ * @src_len: the length of source data
+ * @dst: distination to save the encoded 'length', if dst is NULL, only compute
+ * the expected buffer size in bytes.
+ * @dst_len: output parameter, indicates how many bytes wrote.
+ *
+ * Encode the 'length' part of TLV tuple.
+ */
+static void qcrypto_der_encode_length(size_t src_len,
+                                      uint8_t *dst, size_t *dst_len)
+{
+    size_t max_length = 0xFF;
+    uint8_t length_bytes = 0, header_byte;
+
+    if (src_len < QCRYPTO_DER_SHORT_LEN_MASK) {
+        header_byte = src_len;
+        *dst_len = 1;
+    } else {
+        for (length_bytes = 1; max_length < src_len; length_bytes++) {
+            max_length = (max_length << 8) + max_length;
+        }
+        header_byte = length_bytes;
+        header_byte |= QCRYPTO_DER_SHORT_LEN_MASK;
+        *dst_len = length_bytes + 1;
+    }
+    if (!dst) {
+        return;
+    }
+    *dst++ = header_byte;
+    /* Bigendian length bytes */
+    for (; length_bytes > 0; length_bytes--) {
+        *dst++ = ((src_len >> (length_bytes - 1) * 8) & 0xFF);
+    }
+}
+
 static uint8_t qcrypto_der_peek_byte(const uint8_t **data, size_t *dlen)
 {
     return **data;
@@ -150,40 +223,230 @@ static int qcrypto_der_extract_data(const uint8_t **data, size_t *dlen,
     return qcrypto_der_extract_definite_data(data, dlen, cb, ctx, errp);
 }
 
-int qcrypto_der_decode_int(const uint8_t **data, size_t *dlen,
-                           QCryptoDERDecodeCb cb, void *ctx, Error **errp)
+static int qcrypto_der_decode_tlv(const uint8_t expected_tag,
+                                  const uint8_t **data, size_t *dlen,
+                                  QCryptoDERDecodeCb cb,
+                                  void *ctx, Error **errp)
 {
+    const uint8_t *saved_data = *data;
+    size_t saved_dlen = *dlen;
     uint8_t tag;
+    int data_length;
+
     if (*dlen < 1) {
         error_setg(errp, "Need more data");
         return -1;
     }
     tag = qcrypto_der_cut_byte(data, dlen);
-
-    /* INTEGER must encoded in primitive-form */
-    if (tag != QCRYPTO_DER_TYPE_TAG_INT) {
-        error_setg(errp, "Invalid integer type tag: %u", tag);
-        return -1;
+    if (tag != expected_tag) {
+        error_setg(errp, "Unexpected tag: expected: %u, actual: %u",
+                   expected_tag, tag);
+        goto error;
     }
 
-    return qcrypto_der_extract_data(data, dlen, cb, ctx, errp);
+    data_length = qcrypto_der_extract_data(data, dlen, cb, ctx, errp);
+    if (data_length < 0) {
+        goto error;
+    }
+    return data_length;
+
+error:
+    *data = saved_data;
+    *dlen = saved_dlen;
+    return -1;
+}
+
+int qcrypto_der_decode_int(const uint8_t **data, size_t *dlen,
+                           QCryptoDERDecodeCb cb, void *ctx, Error **errp)
+{
+    const uint8_t tag = QCRYPTO_DER_TAG(QCRYPTO_DER_TAG_CLASS_UNIV,
+                                        QCRYPTO_DER_TAG_ENC_PRIM,
+                                        QCRYPTO_DER_TYPE_TAG_INT);
+    return qcrypto_der_decode_tlv(tag, data, dlen, cb, ctx, errp);
 }
 
 int qcrypto_der_decode_seq(const uint8_t **data, size_t *dlen,
                            QCryptoDERDecodeCb cb, void *ctx, Error **errp)
 {
-    uint8_t tag;
-    if (*dlen < 1) {
-        error_setg(errp, "Need more data");
-        return -1;
-    }
-    tag = qcrypto_der_cut_byte(data, dlen);
-
-    /* SEQUENCE must use constructed form */
-    if (tag != (QCRYPTO_DER_TYPE_TAG_SEQ | QCRYPTO_DER_CONSTRUCTED_MASK)) {
-        error_setg(errp, "Invalid type sequence tag: %u", tag);
-        return -1;
-    }
-
-    return qcrypto_der_extract_data(data, dlen, cb, ctx, errp);
+    uint8_t tag = QCRYPTO_DER_TAG(QCRYPTO_DER_TAG_CLASS_UNIV,
+                                  QCRYPTO_DER_TAG_ENC_CONS,
+                                  QCRYPTO_DER_TYPE_TAG_SEQ);
+    return qcrypto_der_decode_tlv(tag, data, dlen, cb, ctx, errp);
+}
+
+int qcrypto_der_decode_octet_str(const uint8_t **data, size_t *dlen,
+                                 QCryptoDERDecodeCb cb, void *ctx, Error **errp)
+{
+    uint8_t tag = QCRYPTO_DER_TAG(QCRYPTO_DER_TAG_CLASS_UNIV,
+                                  QCRYPTO_DER_TAG_ENC_PRIM,
+                                  QCRYPTO_DER_TYPE_TAG_OCT_STR);
+    return qcrypto_der_decode_tlv(tag, data, dlen, cb, ctx, errp);
+}
+
+int qcrypto_der_decode_bit_str(const uint8_t **data, size_t *dlen,
+                               QCryptoDERDecodeCb cb, void *ctx, Error **errp)
+{
+    uint8_t tag = QCRYPTO_DER_TAG(QCRYPTO_DER_TAG_CLASS_UNIV,
+                                  QCRYPTO_DER_TAG_ENC_PRIM,
+                                  QCRYPTO_DER_TYPE_TAG_BIT_STR);
+    return qcrypto_der_decode_tlv(tag, data, dlen, cb, ctx, errp);
+}
+
+int qcrypto_der_decode_oid(const uint8_t **data, size_t *dlen,
+                           QCryptoDERDecodeCb cb, void *ctx, Error **errp)
+{
+    uint8_t tag = QCRYPTO_DER_TAG(QCRYPTO_DER_TAG_CLASS_UNIV,
+                                  QCRYPTO_DER_TAG_ENC_PRIM,
+                                  QCRYPTO_DER_TYPE_TAG_OID);
+    return qcrypto_der_decode_tlv(tag, data, dlen, cb, ctx, errp);
+}
+
+int qcrypto_der_decode_ctx_tag(const uint8_t **data, size_t *dlen, int tag_id,
+                               QCryptoDERDecodeCb cb, void *ctx, Error **errp)
+{
+    uint8_t tag = QCRYPTO_DER_TAG(QCRYPTO_DER_TAG_CLASS_CONT,
+                                  QCRYPTO_DER_TAG_ENC_CONS,
+                                  tag_id);
+    return qcrypto_der_decode_tlv(tag, data, dlen, cb, ctx, errp);
+}
+
+static void qcrypto_der_encode_prim(QCryptoEncodeContext *ctx, uint8_t tag,
+                                    const uint8_t *data, size_t dlen)
+{
+    QCryptoDerEncodeNode *node = g_new0(QCryptoDerEncodeNode, 1);
+    size_t nbytes_len;
+
+    node->tag = tag;
+    node->data = data;
+    node->dlen = dlen;
+    node->parent = ctx->current_parent;
+
+    qcrypto_der_encode_length(dlen, NULL, &nbytes_len);
+    /* 1 byte for Tag, nbyte_len for Length, and dlen for Value */
+    node->parent->dlen += 1 + nbytes_len + dlen;
+
+    ctx->tail->next = node;
+    ctx->tail = node;
+}
+
+QCryptoEncodeContext *qcrypto_der_encode_ctx_new(void)
+{
+    QCryptoEncodeContext *ctx = g_new0(QCryptoEncodeContext, 1);
+    ctx->current_parent = &ctx->root;
+    ctx->tail = &ctx->root;
+    return ctx;
+}
+
+static void qcrypto_der_encode_cons_begin(QCryptoEncodeContext *ctx,
+                                          uint8_t tag)
+{
+    QCryptoDerEncodeNode *node = g_new0(QCryptoDerEncodeNode, 1);
+
+    node->tag = tag;
+    node->parent = ctx->current_parent;
+    ctx->current_parent = node;
+    ctx->tail->next = node;
+    ctx->tail = node;
+}
+
+static void qcrypto_der_encode_cons_end(QCryptoEncodeContext *ctx)
+{
+    QCryptoDerEncodeNode *cons_node = ctx->current_parent;
+    size_t nbytes_len;
+
+    qcrypto_der_encode_length(cons_node->dlen, NULL, &nbytes_len);
+    /* 1 byte for Tag, nbyte_len for Length, and dlen for Value */
+    cons_node->parent->dlen += 1 + nbytes_len + cons_node->dlen;
+    ctx->current_parent = cons_node->parent;
+}
+
+void qcrypto_der_encode_seq_begin(QCryptoEncodeContext *ctx)
+{
+    uint8_t tag = QCRYPTO_DER_TAG(QCRYPTO_DER_TAG_CLASS_UNIV,
+                                  QCRYPTO_DER_TAG_ENC_CONS,
+                                  QCRYPTO_DER_TYPE_TAG_SEQ);
+    qcrypto_der_encode_cons_begin(ctx, tag);
+}
+
+void qcrypto_der_encode_seq_end(QCryptoEncodeContext *ctx)
+{
+    qcrypto_der_encode_cons_end(ctx);
+}
+
+void qcrypto_der_encode_oid(QCryptoEncodeContext *ctx,
+                            const uint8_t *src, size_t src_len)
+{
+    uint8_t tag = QCRYPTO_DER_TAG(QCRYPTO_DER_TAG_CLASS_UNIV,
+                                  QCRYPTO_DER_TAG_ENC_PRIM,
+                                  QCRYPTO_DER_TYPE_TAG_OID);
+    qcrypto_der_encode_prim(ctx, tag, src, src_len);
+}
+
+void qcrypto_der_encode_int(QCryptoEncodeContext *ctx,
+                            const uint8_t *src, size_t src_len)
+{
+    uint8_t tag = QCRYPTO_DER_TAG(QCRYPTO_DER_TAG_CLASS_UNIV,
+                                  QCRYPTO_DER_TAG_ENC_PRIM,
+                                  QCRYPTO_DER_TYPE_TAG_INT);
+    qcrypto_der_encode_prim(ctx, tag, src, src_len);
+}
+
+void qcrypto_der_encode_null(QCryptoEncodeContext *ctx)
+{
+    uint8_t tag = QCRYPTO_DER_TAG(QCRYPTO_DER_TAG_CLASS_UNIV,
+                                  QCRYPTO_DER_TAG_ENC_PRIM,
+                                  QCRYPTO_DER_TYPE_TAG_NULL);
+    qcrypto_der_encode_prim(ctx, tag, NULL, 0);
+}
+
+void qcrypto_der_encode_octet_str(QCryptoEncodeContext *ctx,
+                                  const uint8_t *src, size_t src_len)
+{
+    uint8_t tag = QCRYPTO_DER_TAG(QCRYPTO_DER_TAG_CLASS_UNIV,
+                                  QCRYPTO_DER_TAG_ENC_PRIM,
+                                  QCRYPTO_DER_TYPE_TAG_OCT_STR);
+    qcrypto_der_encode_prim(ctx, tag, src, src_len);
+}
+
+void qcrypto_der_encode_octet_str_begin(QCryptoEncodeContext *ctx)
+{
+    uint8_t tag = QCRYPTO_DER_TAG(QCRYPTO_DER_TAG_CLASS_UNIV,
+                                  QCRYPTO_DER_TAG_ENC_PRIM,
+                                  QCRYPTO_DER_TYPE_TAG_OCT_STR);
+    qcrypto_der_encode_cons_begin(ctx, tag);
+}
+
+void qcrypto_der_encode_octet_str_end(QCryptoEncodeContext *ctx)
+{
+    qcrypto_der_encode_cons_end(ctx);
+}
+
+size_t qcrypto_der_encode_ctx_buffer_len(QCryptoEncodeContext *ctx)
+{
+    return ctx->root.dlen;
+}
+
+void qcrypto_der_encode_ctx_flush_and_free(QCryptoEncodeContext *ctx,
+                                           uint8_t *dst)
+{
+    QCryptoDerEncodeNode *node, *prev;
+    size_t len;
+
+    for (prev = &ctx->root;
+         (node = prev->next) && (prev->next = node->next, 1);) {
+        /* Tag */
+        *dst++ = node->tag;
+
+        /* Length */
+        qcrypto_der_encode_length(node->dlen, dst, &len);
+        dst += len;
+
+        /* Value */
+        if (node->data) {
+            memcpy(dst, node->data, node->dlen);
+            dst += node->dlen;
+        }
+        g_free(node);
+    }
+    g_free(ctx);
 }
diff --git a/tests/unit/test-crypto-der.c b/tests/unit/test-crypto-der.c
index aed0f28d68..d218a7f170 100644
--- a/tests/unit/test-crypto-der.c
+++ b/tests/unit/test-crypto-der.c
@@ -147,13 +147,58 @@ static const uint8_t test_rsa2048_priv_key[] =
     "\x4e\x2f\x4c\xf9\xab\x97\x38\xe4\x20\x32\x32\x96\xc8\x9e\x79\xd3"
     "\x12";
 
+static const uint8_t test_ecdsa_p192_priv_key[] =
+    "\x30\x53"               /* SEQUENCE, offset 0, length 83 */
+    "\x02\x01\x01"           /* INTEGER, offset 2, length 1 */
+    "\x04\x18"               /* OCTET STRING, offset 5, length 24 */
+    "\xcb\xc8\x86\x0e\x66\x3c\xf7\x5a\x44\x13\xb8\xef\xea\x1d\x7b\xa6"
+    "\x1c\xda\xf4\x1b\xc7\x67\x6b\x35"
+    "\xa1\x34"               /* CONTEXT SPECIFIC 1, offset 31, length 52 */
+    "\x03\x32"               /* BIT STRING, offset 33, length 50 */
+    "\x00\x04\xc4\x16\xb3\xff\xac\xd5\x87\x98\xf7\xd9\x45\xfe\xd3\x5c"
+    "\x17\x9d\xb2\x36\x22\xcc\x07\xb3\x6d\x3c\x4e\x04\x5f\xeb\xb6\x52"
+    "\x58\xfb\x36\x10\x52\xb7\x01\x62\x0e\x94\x51\x1d\xe2\xef\x10\x82"
+    "\x88\x78";
+
+static const uint8_t test_ecdsa_p256_priv_key[] =
+    "\x30\x77"              /* SEQUENCE, offset 0, length 119 */
+    "\x02\x01\x01"          /* INTEGER, offset 2, length 1 */
+    "\x04\x20"              /* OCTET STRING, offset 5, length 32 */
+    "\xf6\x92\xdd\x29\x1c\x6e\xef\xb6\xb2\x73\x9f\x40\x1b\xb3\x2a\x28"
+    "\xd2\x37\xd6\x4a\x5b\xe4\x40\x4c\x6a\x95\x99\xfa\xf7\x92\x49\xbe"
+    "\xa0\x0a"              /* CONTEXT SPECIFIC 0, offset 39, length 10 */
+    "\x06\x08"              /* OID, offset 41, length 8 */
+    "\x2a\x86\x48\xce\x3d\x03\x01\x07"
+    "\xa1\x44"              /* CONTEXT SPECIFIC 1, offset 51, length 68 */
+    "\x03\x42"              /* BIT STRING, offset 53, length 66 */
+    "\x00\x04\xed\x42\x9c\x67\x79\xbe\x46\x83\x88\x3e\x8c\xc1\x33\xf3"
+    "\xc3\xf6\x2c\xf3\x13\x6a\x00\xc2\xc9\x3e\x87\x7f\x86\x39\xe6\xae"
+    "\xe3\xb9\xba\x2f\x58\x63\x32\x62\x62\x54\x07\x27\xf9\x5a\x3a\xc7"
+    "\x3a\x6b\x5b\xbc\x0d\x33\xba\xbb\xd4\xa3\xff\x4f\x9e\xdd\xf5\x59"
+    "\xc0\xf6";
+
 #define MAX_CHECKER_COUNT 32
 
+static int qcrypto_wrapped_decode_ctx_tag0(const uint8_t **data, size_t *dlen,
+                                           QCryptoDERDecodeCb cb, void *opaque,
+                                           Error **errp)
+{
+   return qcrypto_der_decode_ctx_tag(data, dlen, 0, cb, opaque, errp);
+}
+
+static int qcrypto_wrapped_decode_ctx_tag1(const uint8_t **data, size_t *dlen,
+                                           QCryptoDERDecodeCb cb, void *opaque,
+                                           Error **errp)
+{
+   return qcrypto_der_decode_ctx_tag(data, dlen, 1, cb, opaque, errp);
+}
+
 typedef struct QCryptoAns1DecoderResultChecker QCryptoAns1DecoderResultChecker;
 struct QCryptoAns1DecoderResultChecker {
     int (*action) (const uint8_t **data, size_t *dlen,
                    QCryptoDERDecodeCb cb, void *opaque, Error **errp);
     QCryptoDERDecodeCb cb;
+    bool constructed;
     const uint8_t *exp_value;
     size_t exp_vlen;
 };
@@ -204,7 +249,7 @@ static void test_ans1(const void *opaque)
         g_assert(checker->action(&c->data, &c->dlen, checker_callback,
                                  (void *)checker, &error_abort)
             == checker->exp_vlen);
-        if (checker->action == qcrypto_der_decode_seq) {
+        if (checker->constructed) {
             ++seq_depth;
             ctx[seq_depth].data = checker->exp_value;
             ctx[seq_depth].dlen = checker->exp_vlen;
@@ -225,25 +270,25 @@ static QCryptoAns1DecoderTestData test_data[] = {
     .test_data = test_rsa512_priv_key,
     .test_data_len = sizeof(test_rsa512_priv_key) - 1,
     .checker = {
-        { qcrypto_der_decode_seq, checker_callback,
+        { qcrypto_der_decode_seq, checker_callback, true,
           test_rsa512_priv_key + 4, 313 },
-        { qcrypto_der_decode_int, checker_callback,
+        { qcrypto_der_decode_int, checker_callback, false,
           test_rsa512_priv_key + 4 + 2, 1 },
-        { qcrypto_der_decode_int, checker_callback,
+        { qcrypto_der_decode_int, checker_callback, false,
           test_rsa512_priv_key + 7 + 2, 65 },
-        { qcrypto_der_decode_int, checker_callback,
+        { qcrypto_der_decode_int, checker_callback, false,
           test_rsa512_priv_key + 74 + 2, 3 },
-        { qcrypto_der_decode_int, checker_callback,
+        { qcrypto_der_decode_int, checker_callback, false,
           test_rsa512_priv_key + 79 + 2, 64 },
-        { qcrypto_der_decode_int, checker_callback,
+        { qcrypto_der_decode_int, checker_callback, false,
           test_rsa512_priv_key + 145 + 2, 33 },
-        { qcrypto_der_decode_int, checker_callback,
+        { qcrypto_der_decode_int, checker_callback, false,
           test_rsa512_priv_key + 180 + 2, 33 },
-        { qcrypto_der_decode_int, checker_callback,
+        { qcrypto_der_decode_int, checker_callback, false,
           test_rsa512_priv_key + 215 + 2, 32 },
-        { qcrypto_der_decode_int, checker_callback,
+        { qcrypto_der_decode_int, checker_callback, false,
           test_rsa512_priv_key + 249 + 2, 32 },
-        { qcrypto_der_decode_int, checker_callback,
+        { qcrypto_der_decode_int, checker_callback, false,
           test_rsa512_priv_key + 283 + 2, 32 },
     },
 },
@@ -252,29 +297,66 @@ static QCryptoAns1DecoderTestData test_data[] = {
     .test_data = test_rsa2048_priv_key,
     .test_data_len = sizeof(test_rsa2048_priv_key) - 1,
     .checker = {
-        { qcrypto_der_decode_seq, checker_callback,
+        { qcrypto_der_decode_seq, checker_callback, true,
           test_rsa2048_priv_key + 4, 1190 },
-        { qcrypto_der_decode_int, checker_callback,
+        { qcrypto_der_decode_int, checker_callback, false,
           test_rsa2048_priv_key + 4 + 2, 1 },
-        { qcrypto_der_decode_int, checker_callback,
+        { qcrypto_der_decode_int, checker_callback, false,
           test_rsa2048_priv_key + 7 + 4, 257 },
-        { qcrypto_der_decode_int, checker_callback,
+        { qcrypto_der_decode_int, checker_callback, false,
           test_rsa2048_priv_key + 268 + 2, 3 },
-        { qcrypto_der_decode_int, checker_callback,
+        { qcrypto_der_decode_int, checker_callback, false,
           test_rsa2048_priv_key + 273 + 4, 257 },
-        { qcrypto_der_decode_int, checker_callback,
+        { qcrypto_der_decode_int, checker_callback, false,
           test_rsa2048_priv_key + 534 + 3, 129 },
-        { qcrypto_der_decode_int, checker_callback,
+        { qcrypto_der_decode_int, checker_callback, false,
           test_rsa2048_priv_key + 666 + 3, 129 },
-        { qcrypto_der_decode_int, checker_callback,
+        { qcrypto_der_decode_int, checker_callback, false,
           test_rsa2048_priv_key + 798 + 3, 129 },
-        { qcrypto_der_decode_int, checker_callback,
+        { qcrypto_der_decode_int, checker_callback, false,
           test_rsa2048_priv_key + 930 + 3, 129 },
-        { qcrypto_der_decode_int, checker_callback,
+        { qcrypto_der_decode_int, checker_callback, false,
           test_rsa2048_priv_key + 1062 + 3, 129 },
     },
 },
-
+{
+    .path = "/crypto/der/parse-ecdsa-p192-priv-key",
+    .test_data = test_ecdsa_p192_priv_key,
+    .test_data_len = sizeof(test_ecdsa_p192_priv_key) - 1,
+    .checker = {
+        { qcrypto_der_decode_seq, checker_callback, true,
+          test_ecdsa_p192_priv_key + 2, 83 },
+        { qcrypto_der_decode_int, checker_callback, false,
+          test_ecdsa_p192_priv_key + 2 + 2, 1 },
+        { qcrypto_der_decode_octet_str, checker_callback, false,
+          test_ecdsa_p192_priv_key + 5 + 2, 24 },
+        { qcrypto_wrapped_decode_ctx_tag1, checker_callback, true,
+          test_ecdsa_p192_priv_key + 31 + 2, 52 },
+        { qcrypto_der_decode_bit_str , checker_callback, false,
+          test_ecdsa_p192_priv_key + 33 + 2, 50 },
+    },
+},
+{
+    .path = "/crypto/der/parse-ecdsa-p256-priv-key",
+    .test_data = test_ecdsa_p256_priv_key,
+    .test_data_len = sizeof(test_ecdsa_p256_priv_key) - 1,
+    .checker = {
+        { qcrypto_der_decode_seq, checker_callback, true,
+          test_ecdsa_p256_priv_key + 2, 119 },
+        { qcrypto_der_decode_int, checker_callback, false,
+          test_ecdsa_p256_priv_key + 2 + 2, 1 },
+        { qcrypto_der_decode_octet_str, checker_callback, false,
+          test_ecdsa_p256_priv_key + 5 + 2, 32 },
+        { qcrypto_wrapped_decode_ctx_tag0, checker_callback, true,
+          test_ecdsa_p256_priv_key + 39 + 2, 10 },
+        { qcrypto_der_decode_oid, checker_callback, false,
+          test_ecdsa_p256_priv_key + 41 + 2, 8 },
+        { qcrypto_wrapped_decode_ctx_tag1, checker_callback, true,
+          test_ecdsa_p256_priv_key + 51 + 2, 68 },
+        { qcrypto_der_decode_bit_str , checker_callback, false,
+          test_ecdsa_p256_priv_key + 53 + 2, 66 },
+    },
+},
 };
 
 int main(int argc, char **argv)
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 09/83] crypto: Support export akcipher to pkcs8
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (7 preceding siblings ...)
  2022-11-07 22:48 ` [PULL v4 08/83] crypto: Support DER encodings Michael S. Tsirkin
@ 2022-11-07 22:48 ` Michael S. Tsirkin
  2022-11-07 22:48 ` [PULL v4 10/83] cryptodev: Add a lkcf-backend for cryptodev Michael S. Tsirkin
                   ` (75 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:48 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Lei He, Daniel P . Berrangé

From: Lei He <helei.sig11@bytedance.com>

crypto: support export RSA private keys with PKCS#8 standard.
So that users can upload this private key to linux kernel.

Signed-off-by: lei he <helei.sig11@bytedance.com>
Message-Id: <20221008085030.70212-4-helei.sig11@bytedance.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
---
 crypto/rsakey.h           | 11 +++++++++-
 include/crypto/akcipher.h | 21 ++++++++++++++++++++
 crypto/akcipher.c         | 18 +++++++++++++++++
 crypto/rsakey.c           | 42 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 91 insertions(+), 1 deletion(-)

diff --git a/crypto/rsakey.h b/crypto/rsakey.h
index 974b76f659..00b3eccec7 100644
--- a/crypto/rsakey.h
+++ b/crypto/rsakey.h
@@ -22,7 +22,6 @@
 #ifndef QCRYPTO_RSAKEY_H
 #define QCRYPTO_RSAKEY_H
 
-#include "qemu/osdep.h"
 #include "qemu/host-utils.h"
 #include "crypto/akcipher.h"
 
@@ -84,6 +83,16 @@ QCryptoAkCipherRSAKey *qcrypto_akcipher_rsakey_parse(
     QCryptoAkCipherKeyType type,
     const uint8_t *key, size_t keylen, Error **errp);
 
+/**
+ * qcrypto_akcipher_rsakey_export_as_p8info:
+ *
+ * Export RSA private key to PKCS#8 private key info.
+ */
+void qcrypto_akcipher_rsakey_export_p8info(const uint8_t *key,
+                                           size_t keylen,
+                                           uint8_t **dst,
+                                           size_t *dlen);
+
 void qcrypto_akcipher_rsakey_free(QCryptoAkCipherRSAKey *key);
 
 G_DEFINE_AUTOPTR_CLEANUP_FUNC(QCryptoAkCipherRSAKey,
diff --git a/include/crypto/akcipher.h b/include/crypto/akcipher.h
index 51f5fa2774..214e58ca47 100644
--- a/include/crypto/akcipher.h
+++ b/include/crypto/akcipher.h
@@ -153,6 +153,27 @@ int qcrypto_akcipher_max_dgst_len(QCryptoAkCipher *akcipher);
  */
 void qcrypto_akcipher_free(QCryptoAkCipher *akcipher);
 
+/**
+ * qcrypto_akcipher_export_p8info:
+ * @opts: the options of the akcipher to be exported.
+ * @key: the original key of the akcipher to be exported.
+ * @keylen: length of the 'key'
+ * @dst: output parameter, if export succeed, *dst is set to the
+ * PKCS#8 encoded private key, caller MUST free this key with
+ * g_free after use.
+ * @dst_len: output parameter, indicates the length of PKCS#8 encoded
+ * key.
+ *
+ * Export the akcipher into DER encoded pkcs#8 private key info, expects
+ * |key| stores a valid asymmetric PRIVATE key.
+ *
+ * Returns: 0 for succeed, otherwise -1 is returned.
+ */
+int qcrypto_akcipher_export_p8info(const QCryptoAkCipherOptions *opts,
+                                   uint8_t *key, size_t keylen,
+                                   uint8_t **dst, size_t *dst_len,
+                                   Error **errp);
+
 G_DEFINE_AUTOPTR_CLEANUP_FUNC(QCryptoAkCipher, qcrypto_akcipher_free)
 
 #endif /* QCRYPTO_AKCIPHER_H */
diff --git a/crypto/akcipher.c b/crypto/akcipher.c
index ad88379c1e..e4bbc6e5f1 100644
--- a/crypto/akcipher.c
+++ b/crypto/akcipher.c
@@ -22,6 +22,8 @@
 #include "qemu/osdep.h"
 #include "crypto/akcipher.h"
 #include "akcipherpriv.h"
+#include "der.h"
+#include "rsakey.h"
 
 #if defined(CONFIG_GCRYPT)
 #include "akcipher-gcrypt.c.inc"
@@ -106,3 +108,19 @@ void qcrypto_akcipher_free(QCryptoAkCipher *akcipher)
 
     drv->free(akcipher);
 }
+
+int qcrypto_akcipher_export_p8info(const QCryptoAkCipherOptions *opts,
+                                   uint8_t *key, size_t keylen,
+                                   uint8_t **dst, size_t *dst_len,
+                                   Error **errp)
+{
+    switch (opts->alg) {
+    case QCRYPTO_AKCIPHER_ALG_RSA:
+        qcrypto_akcipher_rsakey_export_p8info(key, keylen, dst, dst_len);
+        return 0;
+
+    default:
+        error_setg(errp, "Unsupported algorithm: %u", opts->alg);
+        return -1;
+    }
+}
diff --git a/crypto/rsakey.c b/crypto/rsakey.c
index cc40e072f0..7d6f273aef 100644
--- a/crypto/rsakey.c
+++ b/crypto/rsakey.c
@@ -19,6 +19,8 @@
  *
  */
 
+#include "qemu/osdep.h"
+#include "der.h"
 #include "rsakey.h"
 
 void qcrypto_akcipher_rsakey_free(QCryptoAkCipherRSAKey *rsa_key)
@@ -37,6 +39,46 @@ void qcrypto_akcipher_rsakey_free(QCryptoAkCipherRSAKey *rsa_key)
     g_free(rsa_key);
 }
 
+/**
+ * PKCS#8 private key info for RSA
+ *
+ * PrivateKeyInfo ::= SEQUENCE {
+ * version         INTEGER,
+ * privateKeyAlgorithm PrivateKeyAlgorithmIdentifier,
+ * privateKey      OCTET STRING,
+ * attributes      [0] IMPLICIT Attributes OPTIONAL
+ * }
+ */
+void qcrypto_akcipher_rsakey_export_p8info(const uint8_t *key,
+                                           size_t keylen,
+                                           uint8_t **dst,
+                                           size_t *dlen)
+{
+    QCryptoEncodeContext *ctx = qcrypto_der_encode_ctx_new();
+    uint8_t version = 0;
+
+    qcrypto_der_encode_seq_begin(ctx);
+
+    /* version */
+    qcrypto_der_encode_int(ctx, &version, sizeof(version));
+
+    /* algorithm identifier */
+    qcrypto_der_encode_seq_begin(ctx);
+    qcrypto_der_encode_oid(ctx, (uint8_t *)QCRYPTO_OID_rsaEncryption,
+                           sizeof(QCRYPTO_OID_rsaEncryption) - 1);
+    qcrypto_der_encode_null(ctx);
+    qcrypto_der_encode_seq_end(ctx);
+
+    /* RSA private key */
+    qcrypto_der_encode_octet_str(ctx, key, keylen);
+
+    qcrypto_der_encode_seq_end(ctx);
+
+    *dlen = qcrypto_der_encode_ctx_buffer_len(ctx);
+    *dst = g_malloc(*dlen);
+    qcrypto_der_encode_ctx_flush_and_free(ctx, *dst);
+}
+
 #if defined(CONFIG_NETTLE) && defined(CONFIG_HOGWEED)
 #include "rsakey-nettle.c.inc"
 #else
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 10/83] cryptodev: Add a lkcf-backend for cryptodev
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (8 preceding siblings ...)
  2022-11-07 22:48 ` [PULL v4 09/83] crypto: Support export akcipher to pkcs8 Michael S. Tsirkin
@ 2022-11-07 22:48 ` Michael S. Tsirkin
  2022-11-07 22:48 ` [PULL v4 11/83] acpi/tests/avocado/bits: initial commit of test scripts that are run by biosbits Michael S. Tsirkin
                   ` (74 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:48 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Lei He, Gonglei (Arei),
	Paolo Bonzini, Daniel P. Berrangé,
	Eduardo Habkost, Eric Blake, Markus Armbruster

From: Lei He <helei.sig11@bytedance.com>

cryptodev: Added a new type of backend named lkcf-backend for
cryptodev. This backend upload asymmetric keys to linux kernel,
and let kernel do the accelerations if possible.
The lkcf stands for Linux Kernel Cryptography Framework.

Signed-off-by: lei he <helei.sig11@bytedance.com>
Message-Id: <20221008085030.70212-5-helei.sig11@bytedance.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 qapi/qom.json              |   2 +
 include/sysemu/cryptodev.h |   1 +
 backends/cryptodev-lkcf.c  | 645 +++++++++++++++++++++++++++++++++++++
 backends/meson.build       |   3 +
 4 files changed, 651 insertions(+)
 create mode 100644 backends/cryptodev-lkcf.c

diff --git a/qapi/qom.json b/qapi/qom.json
index 87fcad2423..d2e0244e57 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -876,6 +876,7 @@
     'colo-compare',
     'cryptodev-backend',
     'cryptodev-backend-builtin',
+    'cryptodev-backend-lkcf',
     { 'name': 'cryptodev-vhost-user',
       'if': 'CONFIG_VHOST_CRYPTO' },
     'dbus-vmstate',
@@ -944,6 +945,7 @@
       'colo-compare':               'ColoCompareProperties',
       'cryptodev-backend':          'CryptodevBackendProperties',
       'cryptodev-backend-builtin':  'CryptodevBackendProperties',
+      'cryptodev-backend-lkcf':     'CryptodevBackendProperties',
       'cryptodev-vhost-user':       { 'type': 'CryptodevVhostUserProperties',
                                       'if': 'CONFIG_VHOST_CRYPTO' },
       'dbus-vmstate':               'DBusVMStateProperties',
diff --git a/include/sysemu/cryptodev.h b/include/sysemu/cryptodev.h
index 32e9f4cf8a..cf9b3f07fe 100644
--- a/include/sysemu/cryptodev.h
+++ b/include/sysemu/cryptodev.h
@@ -219,6 +219,7 @@ typedef enum CryptoDevBackendOptionsType {
     CRYPTODEV_BACKEND_TYPE_NONE = 0,
     CRYPTODEV_BACKEND_TYPE_BUILTIN = 1,
     CRYPTODEV_BACKEND_TYPE_VHOST_USER = 2,
+    CRYPTODEV_BACKEND_TYPE_LKCF = 3,
     CRYPTODEV_BACKEND_TYPE__MAX,
 } CryptoDevBackendOptionsType;
 
diff --git a/backends/cryptodev-lkcf.c b/backends/cryptodev-lkcf.c
new file mode 100644
index 0000000000..133bd706a4
--- /dev/null
+++ b/backends/cryptodev-lkcf.c
@@ -0,0 +1,645 @@
+/*
+ * QEMU Cryptodev backend for QEMU cipher APIs
+ *
+ * Copyright (c) 2022 Bytedance.Inc
+ *
+ * Authors:
+ *    lei he <helei.sig11@bytedance.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "crypto/cipher.h"
+#include "crypto/akcipher.h"
+#include "qapi/error.h"
+#include "qemu/main-loop.h"
+#include "qemu/thread.h"
+#include "qemu/error-report.h"
+#include "qemu/queue.h"
+#include "qom/object.h"
+#include "sysemu/cryptodev.h"
+#include "standard-headers/linux/virtio_crypto.h"
+
+#include <keyutils.h>
+#include <sys/eventfd.h>
+
+/**
+ * @TYPE_CRYPTODEV_BACKEND_LKCF:
+ * name of backend that uses linux kernel crypto framework
+ */
+#define TYPE_CRYPTODEV_BACKEND_LKCF "cryptodev-backend-lkcf"
+
+OBJECT_DECLARE_SIMPLE_TYPE(CryptoDevBackendLKCF, CRYPTODEV_BACKEND_LKCF)
+
+#define INVALID_KEY_ID -1
+#define MAX_SESSIONS 256
+#define NR_WORKER_THREAD 64
+
+#define KCTL_KEY_TYPE_PKEY "asymmetric"
+/**
+ * Here the key is uploaded to the thread-keyring of worker thread, at least
+ * util linux-6.0:
+ * 1. process keyring seems to behave unexpectedly if main-thread does not
+ * create the keyring before creating any other thread.
+ * 2. at present, the guest kernel never perform multiple operations on a
+ * session.
+ * 3. it can reduce the load of the main-loop because the key passed by the
+ * guest kernel has been already checked.
+ */
+#define KCTL_KEY_RING KEY_SPEC_THREAD_KEYRING
+
+typedef struct CryptoDevBackendLKCFSession {
+    uint8_t *key;
+    size_t keylen;
+    QCryptoAkCipherKeyType keytype;
+    QCryptoAkCipherOptions akcipher_opts;
+} CryptoDevBackendLKCFSession;
+
+typedef struct CryptoDevBackendLKCF CryptoDevBackendLKCF;
+typedef struct CryptoDevLKCFTask CryptoDevLKCFTask;
+struct CryptoDevLKCFTask {
+    CryptoDevBackendLKCFSession *sess;
+    CryptoDevBackendOpInfo *op_info;
+    CryptoDevCompletionFunc cb;
+    void *opaque;
+    int status;
+    CryptoDevBackendLKCF *lkcf;
+    QSIMPLEQ_ENTRY(CryptoDevLKCFTask) queue;
+};
+
+typedef struct CryptoDevBackendLKCF {
+    CryptoDevBackend parent_obj;
+    CryptoDevBackendLKCFSession *sess[MAX_SESSIONS];
+    QSIMPLEQ_HEAD(, CryptoDevLKCFTask) requests;
+    QSIMPLEQ_HEAD(, CryptoDevLKCFTask) responses;
+    QemuMutex mutex;
+    QemuCond cond;
+    QemuMutex rsp_mutex;
+
+    /**
+     * There is no async interface for asymmetric keys like AF_ALG sockets,
+     * we don't seem to have better way than create a lots of thread.
+     */
+    QemuThread worker_threads[NR_WORKER_THREAD];
+    bool running;
+    int eventfd;
+} CryptoDevBackendLKCF;
+
+static void *cryptodev_lkcf_worker(void *arg);
+static int cryptodev_lkcf_close_session(CryptoDevBackend *backend,
+                                        uint64_t session_id,
+                                        uint32_t queue_index,
+                                        CryptoDevCompletionFunc cb,
+                                        void *opaque);
+
+static void cryptodev_lkcf_handle_response(void *opaque)
+{
+    CryptoDevBackendLKCF *lkcf = (CryptoDevBackendLKCF *)opaque;
+    QSIMPLEQ_HEAD(, CryptoDevLKCFTask) responses;
+    CryptoDevLKCFTask *task, *next;
+    eventfd_t nevent;
+
+    QSIMPLEQ_INIT(&responses);
+    eventfd_read(lkcf->eventfd, &nevent);
+
+    qemu_mutex_lock(&lkcf->rsp_mutex);
+    QSIMPLEQ_PREPEND(&responses, &lkcf->responses);
+    qemu_mutex_unlock(&lkcf->rsp_mutex);
+
+    QSIMPLEQ_FOREACH_SAFE(task, &responses, queue, next) {
+        if (task->cb) {
+            task->cb(task->opaque, task->status);
+        }
+        g_free(task);
+    }
+}
+
+static int cryptodev_lkcf_set_op_desc(QCryptoAkCipherOptions *opts,
+                                      char *key_desc,
+                                      size_t desc_len,
+                                      Error **errp)
+{
+    QCryptoAkCipherOptionsRSA *rsa_opt;
+    if (opts->alg != QCRYPTO_AKCIPHER_ALG_RSA) {
+        error_setg(errp, "Unsupported alg: %u", opts->alg);
+        return -1;
+    }
+
+    rsa_opt = &opts->u.rsa;
+    if (rsa_opt->padding_alg == QCRYPTO_RSA_PADDING_ALG_PKCS1) {
+        snprintf(key_desc, desc_len, "enc=%s hash=%s",
+                 QCryptoRSAPaddingAlgorithm_str(rsa_opt->padding_alg),
+                 QCryptoHashAlgorithm_str(rsa_opt->hash_alg));
+
+    } else {
+        snprintf(key_desc, desc_len, "enc=%s",
+                 QCryptoRSAPaddingAlgorithm_str(rsa_opt->padding_alg));
+    }
+    return 0;
+}
+
+static int cryptodev_lkcf_set_rsa_opt(int virtio_padding_alg,
+                                      int virtio_hash_alg,
+                                      QCryptoAkCipherOptionsRSA *opt,
+                                      Error **errp)
+{
+    if (virtio_padding_alg == VIRTIO_CRYPTO_RSA_PKCS1_PADDING) {
+        opt->padding_alg = QCRYPTO_RSA_PADDING_ALG_PKCS1;
+
+        switch (virtio_hash_alg) {
+        case VIRTIO_CRYPTO_RSA_MD5:
+            opt->hash_alg = QCRYPTO_HASH_ALG_MD5;
+            break;
+
+        case VIRTIO_CRYPTO_RSA_SHA1:
+            opt->hash_alg = QCRYPTO_HASH_ALG_SHA1;
+            break;
+
+        case VIRTIO_CRYPTO_RSA_SHA256:
+            opt->hash_alg = QCRYPTO_HASH_ALG_SHA256;
+            break;
+
+        case VIRTIO_CRYPTO_RSA_SHA512:
+            opt->hash_alg = QCRYPTO_HASH_ALG_SHA512;
+            break;
+
+        default:
+            error_setg(errp, "Unsupported rsa hash algo: %d", virtio_hash_alg);
+            return -1;
+        }
+        return 0;
+    }
+
+    if (virtio_padding_alg == VIRTIO_CRYPTO_RSA_RAW_PADDING) {
+        opt->padding_alg = QCRYPTO_RSA_PADDING_ALG_RAW;
+        return 0;
+    }
+
+    error_setg(errp, "Unsupported rsa padding algo: %u", virtio_padding_alg);
+    return -1;
+}
+
+static int cryptodev_lkcf_get_unused_session_index(CryptoDevBackendLKCF *lkcf)
+{
+    size_t i;
+
+    for (i = 0; i < MAX_SESSIONS; i++) {
+        if (lkcf->sess[i] == NULL) {
+            return i;
+        }
+    }
+    return -1;
+}
+
+static void cryptodev_lkcf_init(CryptoDevBackend *backend, Error **errp)
+{
+    /* Only support one queue */
+    int queues = backend->conf.peers.queues, i;
+    CryptoDevBackendClient *cc;
+    CryptoDevBackendLKCF *lkcf =
+        CRYPTODEV_BACKEND_LKCF(backend);
+
+    if (queues != 1) {
+        error_setg(errp,
+                   "Only support one queue in cryptodev-builtin backend");
+        return;
+    }
+    lkcf->eventfd = eventfd(0, 0);
+    if (lkcf->eventfd < 0) {
+        error_setg(errp, "Failed to create eventfd: %d", errno);
+        return;
+    }
+
+    cc = cryptodev_backend_new_client("cryptodev-lkcf", NULL);
+    cc->info_str = g_strdup_printf("cryptodev-lkcf0");
+    cc->queue_index = 0;
+    cc->type = CRYPTODEV_BACKEND_TYPE_LKCF;
+    backend->conf.peers.ccs[0] = cc;
+
+    backend->conf.crypto_services =
+        1u << VIRTIO_CRYPTO_SERVICE_AKCIPHER;
+    backend->conf.akcipher_algo = 1u << VIRTIO_CRYPTO_AKCIPHER_RSA;
+    lkcf->running = true;
+
+    QSIMPLEQ_INIT(&lkcf->requests);
+    QSIMPLEQ_INIT(&lkcf->responses);
+    qemu_mutex_init(&lkcf->mutex);
+    qemu_mutex_init(&lkcf->rsp_mutex);
+    qemu_cond_init(&lkcf->cond);
+    for (i = 0; i < NR_WORKER_THREAD; i++) {
+        qemu_thread_create(&lkcf->worker_threads[i], "lkcf-worker",
+                           cryptodev_lkcf_worker, lkcf, 0);
+    }
+    qemu_set_fd_handler(
+        lkcf->eventfd, cryptodev_lkcf_handle_response, NULL, lkcf);
+    cryptodev_backend_set_ready(backend, true);
+}
+
+static void cryptodev_lkcf_cleanup(CryptoDevBackend *backend, Error **errp)
+{
+    CryptoDevBackendLKCF *lkcf = CRYPTODEV_BACKEND_LKCF(backend);
+    size_t i;
+    int queues = backend->conf.peers.queues;
+    CryptoDevBackendClient *cc;
+    CryptoDevLKCFTask *task, *next;
+
+    qemu_mutex_lock(&lkcf->mutex);
+    lkcf->running = false;
+    qemu_mutex_unlock(&lkcf->mutex);
+    qemu_cond_broadcast(&lkcf->cond);
+
+    close(lkcf->eventfd);
+    for (i = 0; i < NR_WORKER_THREAD; i++) {
+        qemu_thread_join(&lkcf->worker_threads[i]);
+    }
+
+    QSIMPLEQ_FOREACH_SAFE(task, &lkcf->requests, queue, next) {
+        if (task->cb) {
+            task->cb(task->opaque, task->status);
+        }
+        g_free(task);
+    }
+
+    QSIMPLEQ_FOREACH_SAFE(task, &lkcf->responses, queue, next) {
+        if (task->cb) {
+            task->cb(task->opaque, task->status);
+        }
+        g_free(task);
+    }
+
+    qemu_mutex_destroy(&lkcf->mutex);
+    qemu_cond_destroy(&lkcf->cond);
+    qemu_mutex_destroy(&lkcf->rsp_mutex);
+
+    for (i = 0; i < MAX_SESSIONS; i++) {
+        if (lkcf->sess[i] != NULL) {
+            cryptodev_lkcf_close_session(backend, i, 0, NULL, NULL);
+        }
+    }
+
+    for (i = 0; i < queues; i++) {
+        cc = backend->conf.peers.ccs[i];
+        if (cc) {
+            cryptodev_backend_free_client(cc);
+            backend->conf.peers.ccs[i] = NULL;
+        }
+    }
+
+    cryptodev_backend_set_ready(backend, false);
+}
+
+static void cryptodev_lkcf_execute_task(CryptoDevLKCFTask *task)
+{
+    CryptoDevBackendLKCFSession *session = task->sess;
+    CryptoDevBackendAsymOpInfo *asym_op_info;
+    bool kick = false;
+    int ret, status, op_code = task->op_info->op_code;
+    size_t p8info_len;
+    g_autofree uint8_t *p8info = NULL;
+    Error *local_error = NULL;
+    key_serial_t key_id = INVALID_KEY_ID;
+    char op_desc[64];
+    g_autoptr(QCryptoAkCipher) akcipher = NULL;
+
+    /**
+     * We only offload private key session:
+     * 1. currently, the Linux kernel can only accept public key wrapped
+     * with X.509 certificates, but unfortunately the cost of making a
+     * ceritificate with public key is too expensive.
+     * 2. generally, public key related compution is fast, just compute it with
+     * thread-pool.
+     */
+    if (session->keytype == QCRYPTO_AKCIPHER_KEY_TYPE_PRIVATE) {
+        if (qcrypto_akcipher_export_p8info(&session->akcipher_opts,
+                                           session->key, session->keylen,
+                                           &p8info, &p8info_len,
+                                           &local_error) != 0 ||
+            cryptodev_lkcf_set_op_desc(&session->akcipher_opts, op_desc,
+                                       sizeof(op_desc), &local_error) != 0) {
+            error_report_err(local_error);
+        } else {
+            key_id = add_key(KCTL_KEY_TYPE_PKEY, "lkcf-backend-priv-key",
+                             p8info, p8info_len, KCTL_KEY_RING);
+        }
+    }
+
+    if (key_id < 0) {
+        if (!qcrypto_akcipher_supports(&session->akcipher_opts)) {
+            status = -VIRTIO_CRYPTO_NOTSUPP;
+            goto out;
+        }
+        akcipher = qcrypto_akcipher_new(&session->akcipher_opts,
+                                        session->keytype,
+                                        session->key, session->keylen,
+                                        &local_error);
+        if (!akcipher) {
+            status = -VIRTIO_CRYPTO_ERR;
+            goto out;
+        }
+    }
+
+    asym_op_info = task->op_info->u.asym_op_info;
+    switch (op_code) {
+    case VIRTIO_CRYPTO_AKCIPHER_ENCRYPT:
+        if (key_id >= 0) {
+            ret = keyctl_pkey_encrypt(key_id, op_desc,
+                asym_op_info->src, asym_op_info->src_len,
+                asym_op_info->dst, asym_op_info->dst_len);
+        } else {
+            ret = qcrypto_akcipher_encrypt(akcipher,
+                asym_op_info->src, asym_op_info->src_len,
+                asym_op_info->dst, asym_op_info->dst_len, &local_error);
+        }
+        break;
+
+    case VIRTIO_CRYPTO_AKCIPHER_DECRYPT:
+        if (key_id >= 0) {
+            ret = keyctl_pkey_decrypt(key_id, op_desc,
+                asym_op_info->src, asym_op_info->src_len,
+                asym_op_info->dst, asym_op_info->dst_len);
+        } else {
+            ret = qcrypto_akcipher_decrypt(akcipher,
+                asym_op_info->src, asym_op_info->src_len,
+                asym_op_info->dst, asym_op_info->dst_len, &local_error);
+        }
+        break;
+
+    case VIRTIO_CRYPTO_AKCIPHER_SIGN:
+        if (key_id >= 0) {
+            ret = keyctl_pkey_sign(key_id, op_desc,
+                asym_op_info->src, asym_op_info->src_len,
+                asym_op_info->dst, asym_op_info->dst_len);
+        } else {
+            ret = qcrypto_akcipher_sign(akcipher,
+                asym_op_info->src, asym_op_info->src_len,
+                asym_op_info->dst, asym_op_info->dst_len, &local_error);
+        }
+        break;
+
+    case VIRTIO_CRYPTO_AKCIPHER_VERIFY:
+        if (key_id >= 0) {
+            ret = keyctl_pkey_verify(key_id, op_desc,
+                asym_op_info->src, asym_op_info->src_len,
+                asym_op_info->dst, asym_op_info->dst_len);
+        } else {
+            ret = qcrypto_akcipher_verify(akcipher,
+                asym_op_info->src, asym_op_info->src_len,
+                asym_op_info->dst, asym_op_info->dst_len, &local_error);
+        }
+        break;
+
+    default:
+        error_setg(&local_error, "Unknown opcode: %u", op_code);
+        status = -VIRTIO_CRYPTO_ERR;
+        goto out;
+    }
+
+    if (ret < 0) {
+        if (!local_error) {
+            if (errno != EKEYREJECTED) {
+                error_report("Failed do operation with keyctl: %d", errno);
+            }
+        } else {
+            error_report_err(local_error);
+        }
+        status = op_code == VIRTIO_CRYPTO_AKCIPHER_VERIFY ?
+            -VIRTIO_CRYPTO_KEY_REJECTED : -VIRTIO_CRYPTO_ERR;
+    } else {
+        status = VIRTIO_CRYPTO_OK;
+        asym_op_info->dst_len = ret;
+    }
+
+out:
+    if (key_id >= 0) {
+        keyctl_unlink(key_id, KCTL_KEY_RING);
+    }
+    task->status = status;
+
+    qemu_mutex_lock(&task->lkcf->rsp_mutex);
+    if (QSIMPLEQ_EMPTY(&task->lkcf->responses)) {
+        kick = true;
+    }
+    QSIMPLEQ_INSERT_TAIL(&task->lkcf->responses, task, queue);
+    qemu_mutex_unlock(&task->lkcf->rsp_mutex);
+
+    if (kick) {
+        eventfd_write(task->lkcf->eventfd, 1);
+    }
+}
+
+static void *cryptodev_lkcf_worker(void *arg)
+{
+    CryptoDevBackendLKCF *backend = (CryptoDevBackendLKCF *)arg;
+    CryptoDevLKCFTask *task;
+
+    for (;;) {
+        task = NULL;
+        qemu_mutex_lock(&backend->mutex);
+        while (backend->running && QSIMPLEQ_EMPTY(&backend->requests)) {
+            qemu_cond_wait(&backend->cond, &backend->mutex);
+        }
+        if (backend->running) {
+            task = QSIMPLEQ_FIRST(&backend->requests);
+            QSIMPLEQ_REMOVE_HEAD(&backend->requests, queue);
+        }
+        qemu_mutex_unlock(&backend->mutex);
+
+        /* stopped */
+        if (!task) {
+            break;
+        }
+        cryptodev_lkcf_execute_task(task);
+   }
+
+   return NULL;
+}
+
+static int cryptodev_lkcf_operation(
+    CryptoDevBackend *backend,
+    CryptoDevBackendOpInfo *op_info,
+    uint32_t queue_index,
+    CryptoDevCompletionFunc cb,
+    void *opaque)
+{
+    CryptoDevBackendLKCF *lkcf =
+        CRYPTODEV_BACKEND_LKCF(backend);
+    CryptoDevBackendLKCFSession *sess;
+    enum CryptoDevBackendAlgType algtype = op_info->algtype;
+    CryptoDevLKCFTask *task;
+
+    if (op_info->session_id >= MAX_SESSIONS ||
+        lkcf->sess[op_info->session_id] == NULL) {
+        error_report("Cannot find a valid session id: %" PRIu64 "",
+                     op_info->session_id);
+        return -VIRTIO_CRYPTO_INVSESS;
+    }
+
+    sess = lkcf->sess[op_info->session_id];
+    if (algtype != CRYPTODEV_BACKEND_ALG_ASYM) {
+        error_report("algtype not supported: %u", algtype);
+        return -VIRTIO_CRYPTO_NOTSUPP;
+    }
+
+    task = g_new0(CryptoDevLKCFTask, 1);
+    task->op_info = op_info;
+    task->cb = cb;
+    task->opaque = opaque;
+    task->sess = sess;
+    task->lkcf = lkcf;
+    task->status = -VIRTIO_CRYPTO_ERR;
+
+    qemu_mutex_lock(&lkcf->mutex);
+    QSIMPLEQ_INSERT_TAIL(&lkcf->requests, task, queue);
+    qemu_mutex_unlock(&lkcf->mutex);
+    qemu_cond_signal(&lkcf->cond);
+
+    return VIRTIO_CRYPTO_OK;
+}
+
+static int cryptodev_lkcf_create_asym_session(
+    CryptoDevBackendLKCF *lkcf,
+    CryptoDevBackendAsymSessionInfo *sess_info,
+    uint64_t *session_id)
+{
+    Error *local_error = NULL;
+    int index;
+    g_autofree CryptoDevBackendLKCFSession *sess =
+        g_new0(CryptoDevBackendLKCFSession, 1);
+
+    switch (sess_info->algo) {
+    case VIRTIO_CRYPTO_AKCIPHER_RSA:
+        sess->akcipher_opts.alg = QCRYPTO_AKCIPHER_ALG_RSA;
+        if (cryptodev_lkcf_set_rsa_opt(
+            sess_info->u.rsa.padding_algo, sess_info->u.rsa.hash_algo,
+            &sess->akcipher_opts.u.rsa, &local_error) != 0) {
+            error_report_err(local_error);
+            return -VIRTIO_CRYPTO_ERR;
+        }
+        break;
+
+    default:
+        error_report("Unsupported asym alg %u", sess_info->algo);
+        return -VIRTIO_CRYPTO_NOTSUPP;
+    }
+
+    switch (sess_info->keytype) {
+    case VIRTIO_CRYPTO_AKCIPHER_KEY_TYPE_PUBLIC:
+        sess->keytype = QCRYPTO_AKCIPHER_KEY_TYPE_PUBLIC;
+        break;
+
+    case VIRTIO_CRYPTO_AKCIPHER_KEY_TYPE_PRIVATE:
+        sess->keytype = QCRYPTO_AKCIPHER_KEY_TYPE_PRIVATE;
+        break;
+
+    default:
+        error_report("Unknown akcipher keytype: %u", sess_info->keytype);
+        return -VIRTIO_CRYPTO_ERR;
+    }
+
+    index = cryptodev_lkcf_get_unused_session_index(lkcf);
+    if (index < 0) {
+        error_report("Total number of sessions created exceeds %u",
+                     MAX_SESSIONS);
+        return -VIRTIO_CRYPTO_ERR;
+    }
+
+    sess->keylen = sess_info->keylen;
+    sess->key = g_malloc(sess_info->keylen);
+    memcpy(sess->key, sess_info->key, sess_info->keylen);
+
+    lkcf->sess[index] = g_steal_pointer(&sess);
+    *session_id = index;
+
+    return VIRTIO_CRYPTO_OK;
+}
+
+static int cryptodev_lkcf_create_session(
+    CryptoDevBackend *backend,
+    CryptoDevBackendSessionInfo *sess_info,
+    uint32_t queue_index,
+    CryptoDevCompletionFunc cb,
+    void *opaque)
+{
+    CryptoDevBackendAsymSessionInfo *asym_sess_info;
+    CryptoDevBackendLKCF *lkcf =
+        CRYPTODEV_BACKEND_LKCF(backend);
+    int ret;
+
+    switch (sess_info->op_code) {
+    case VIRTIO_CRYPTO_AKCIPHER_CREATE_SESSION:
+        asym_sess_info = &sess_info->u.asym_sess_info;
+        ret = cryptodev_lkcf_create_asym_session(
+            lkcf, asym_sess_info, &sess_info->session_id);
+        break;
+
+    default:
+        ret = -VIRTIO_CRYPTO_NOTSUPP;
+        error_report("Unsupported opcode: %" PRIu32 "",
+                     sess_info->op_code);
+        break;
+    }
+    if (cb) {
+        cb(opaque, ret);
+    }
+    return 0;
+}
+
+static int cryptodev_lkcf_close_session(CryptoDevBackend *backend,
+                                        uint64_t session_id,
+                                        uint32_t queue_index,
+                                        CryptoDevCompletionFunc cb,
+                                        void *opaque)
+{
+    CryptoDevBackendLKCF *lkcf = CRYPTODEV_BACKEND_LKCF(backend);
+    CryptoDevBackendLKCFSession *session;
+
+    assert(session_id < MAX_SESSIONS && lkcf->sess[session_id]);
+    session = lkcf->sess[session_id];
+    lkcf->sess[session_id] = NULL;
+
+    g_free(session->key);
+    g_free(session);
+
+    if (cb) {
+        cb(opaque, VIRTIO_CRYPTO_OK);
+    }
+    return 0;
+}
+
+static void cryptodev_lkcf_class_init(ObjectClass *oc, void *data)
+{
+    CryptoDevBackendClass *bc = CRYPTODEV_BACKEND_CLASS(oc);
+
+    bc->init = cryptodev_lkcf_init;
+    bc->cleanup = cryptodev_lkcf_cleanup;
+    bc->create_session = cryptodev_lkcf_create_session;
+    bc->close_session = cryptodev_lkcf_close_session;
+    bc->do_op = cryptodev_lkcf_operation;
+}
+
+static const TypeInfo cryptodev_builtin_info = {
+    .name = TYPE_CRYPTODEV_BACKEND_LKCF,
+    .parent = TYPE_CRYPTODEV_BACKEND,
+    .class_init = cryptodev_lkcf_class_init,
+    .instance_size = sizeof(CryptoDevBackendLKCF),
+};
+
+static void cryptodev_lkcf_register_types(void)
+{
+    type_register_static(&cryptodev_builtin_info);
+}
+
+type_init(cryptodev_lkcf_register_types);
diff --git a/backends/meson.build b/backends/meson.build
index b1884a88ec..954e658b25 100644
--- a/backends/meson.build
+++ b/backends/meson.build
@@ -12,6 +12,9 @@ softmmu_ss.add([files(
 softmmu_ss.add(when: 'CONFIG_POSIX', if_true: files('rng-random.c'))
 softmmu_ss.add(when: 'CONFIG_POSIX', if_true: files('hostmem-file.c'))
 softmmu_ss.add(when: 'CONFIG_LINUX', if_true: files('hostmem-memfd.c'))
+if keyutils.found()
+    softmmu_ss.add(keyutils, files('cryptodev-lkcf.c'))
+endif
 if have_vhost_user
   softmmu_ss.add(when: 'CONFIG_VIRTIO', if_true: files('vhost-user.c'))
 endif
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 11/83] acpi/tests/avocado/bits: initial commit of test scripts that are run by biosbits
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (9 preceding siblings ...)
  2022-11-07 22:48 ` [PULL v4 10/83] cryptodev: Add a lkcf-backend for cryptodev Michael S. Tsirkin
@ 2022-11-07 22:48 ` Michael S. Tsirkin
  2022-11-07 22:48 ` [PULL v4 12/83] acpi/tests/avocado/bits: disable acpi PSS tests that are failing in biosbits Michael S. Tsirkin
                   ` (73 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:48 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Ani Sinha, Daniel P . Berrangé,
	Paolo Bonzini, John Snow, Thomas Huth, Alex Bennée,
	Igor Mammedov, Michael Tsirkin, Cleber Rosa,
	Philippe Mathieu-Daudé,
	Wainer dos Santos Moschetta, Beraldo Leal

From: Ani Sinha <ani@anisinha.ca>

This is initial commit of cpuid, acpi and smbios python test scripts for
biosbits to execute. No change has been made to them from the original code
written by the biosbits author Josh Triplett. They are required to be installed
into the bits iso file and then run from within the virtual machine booted off
with biosbits iso.

The test scripts have a ".py2" extension in order to prevent avocado from
loading them. They are written in python 2.7 and are run from within bios bits.
There is no need for avocado to try to load them and call out errors on python3
specific syntaxes.

The original location of these tests are here:
https://github.com/biosbits/bits/blob/master/python/testacpi.py
https://github.com/biosbits/bits/blob/master/python/smbios.py
https://github.com/biosbits/bits/blob/master/python/testcpuid.py

For QEMU, we maintain a fork of the above repo here with numerious fixes:
https://gitlab.com/qemu-project/biosbits-bits

The acpi test for example is maintained here in the fork:
https://gitlab.com/qemu-project/biosbits-bits/-/raw/master/python/testacpi.py

Cc: Daniel P. Berrangé <berrange@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Maydell Peter <peter.maydell@linaro.org>
Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Michael Tsirkin <mst@redhat.com>
Signed-off-by: Ani Sinha <ani@anisinha.ca>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221021095108.104843-2-ani@anisinha.ca>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/avocado/acpi-bits/bits-tests/smbios.py2 | 2430 +++++++++++++++++
 .../avocado/acpi-bits/bits-tests/testacpi.py2 |  283 ++
 .../acpi-bits/bits-tests/testcpuid.py2        |   83 +
 3 files changed, 2796 insertions(+)
 create mode 100644 tests/avocado/acpi-bits/bits-tests/smbios.py2
 create mode 100644 tests/avocado/acpi-bits/bits-tests/testacpi.py2
 create mode 100644 tests/avocado/acpi-bits/bits-tests/testcpuid.py2

diff --git a/tests/avocado/acpi-bits/bits-tests/smbios.py2 b/tests/avocado/acpi-bits/bits-tests/smbios.py2
new file mode 100644
index 0000000000..9667d0542c
--- /dev/null
+++ b/tests/avocado/acpi-bits/bits-tests/smbios.py2
@@ -0,0 +1,2430 @@
+# Copyright (c) 2015, Intel Corporation
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+#     * Redistributions of source code must retain the above copyright notice,
+#       this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright notice,
+#       this list of conditions and the following disclaimer in the documentation
+#       and/or other materials provided with the distribution.
+#     * Neither the name of Intel Corporation nor the names of its contributors
+#       may be used to endorse or promote products derived from this software
+#       without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""SMBIOS/DMI module."""
+
+import bits
+import bitfields
+import ctypes
+import redirect
+import struct
+import uuid
+import unpack
+import ttypager
+import sys
+
+class SMBIOS(unpack.Struct):
+    def __new__(cls):
+        if sys.platform == "BITS-EFI":
+            import efi
+            sm_ptr = efi.system_table.ConfigurationTableDict.get(efi.SMBIOS_TABLE_GUID)
+        else:
+            address = 0xF0000
+            mem = bits.memory(0xF0000, 0x10000)
+            for offset in range(0, len(mem), 16):
+                signature = (ctypes.c_char * 4).from_address(address + offset).value
+                if signature == "_SM_":
+                    entry_point_length = ctypes.c_ubyte.from_address(address + offset + 5).value
+                    csum = sum(map(ord, mem[offset:offset + entry_point_length])) & 0xff
+                    if csum == 0:
+                        sm_ptr = address + offset
+                        break
+            else:
+                return None
+
+        if not sm_ptr:
+            return None
+
+        sm = super(SMBIOS, cls).__new__(cls)
+        sm._header_memory = bits.memory(sm_ptr, 0x1f)
+        return sm
+
+    def __init__(self):
+        super(SMBIOS, self).__init__()
+        u = unpack.Unpackable(self._header_memory)
+        self.add_field('header', Header(u))
+        self._structure_memory = bits.memory(self.header.structure_table_address, self.header.structure_table_length)
+        u = unpack.Unpackable(self._structure_memory)
+        self.add_field('structures', unpack.unpack_all(u, _smbios_structures, self), unpack.format_each("\n\n{!r}"))
+
+    def structure_type(self, num):
+        '''Dumps structure of given Type if present'''
+        try:
+            types_present = [self.structures[x].smbios_structure_type for x in range(len(self.structures))]
+            matrix = dict()
+            for index in range(len(types_present)):
+                if types_present.count(types_present[index]) == 1:
+                    matrix[types_present[index]] = self.structures[index]
+                else: # if multiple structures of the same type, return a list of structures for the type number
+                    if matrix.has_key(types_present[index]):
+                        matrix[types_present[index]].append(self.structures[index])
+                    else:
+                        matrix[types_present[index]] = [self.structures[index]]
+            return matrix[num]
+        except:
+            print "Failure: Type {} - not found".format(num)
+
+class Header(unpack.Struct):
+    def __new__(cls, u):
+        return super(Header, cls).__new__(cls)
+
+    def __init__(self, u):
+        super(Header, self).__init__()
+        self.raw_data = u.unpack_rest()
+        u = unpack.Unpackable(self.raw_data)
+        self.add_field('anchor_string', u.unpack_one("4s"))
+        self.add_field('checksum', u.unpack_one("B"))
+        self.add_field('length', u.unpack_one("B"))
+        self.add_field('major_version', u.unpack_one("B"))
+        self.add_field('minor_version', u.unpack_one("B"))
+        self.add_field('max_structure_size', u.unpack_one("<H"))
+        self.add_field('entry_point_revision', u.unpack_one("B"))
+        self.add_field('formatted_area', u.unpack_one("5s"))
+        self.add_field('intermediate_anchor_string', u.unpack_one("5s"))
+        self.add_field('intermediate_checksum', u.unpack_one("B"))
+        self.add_field('structure_table_length', u.unpack_one("<H"))
+        self.add_field('structure_table_address', u.unpack_one("<I"))
+        self.add_field('number_structures', u.unpack_one("<H"))
+        self.add_field('bcd_revision', u.unpack_one("B"))
+        if not u.at_end():
+            self.add_field('data', u.unpack_rest())
+
+class SmbiosBaseStructure(unpack.Struct):
+    def __new__(cls, u, sm):
+        t = u.unpack_peek_one("B")
+        if cls.smbios_structure_type is not None and t != cls.smbios_structure_type:
+            return None
+        return super(SmbiosBaseStructure, cls).__new__(cls)
+
+    def __init__(self, u, sm):
+        super(SmbiosBaseStructure, self).__init__()
+        self.start_offset = u.offset
+        length = u.unpack_peek_one("<xB")
+        self.raw_data = u.unpack_raw(length)
+        self.u = unpack.Unpackable(self.raw_data)
+
+        self.strings_offset = u.offset
+        def unpack_string():
+            return "".join(iter(lambda: u.unpack_one("c"), "\x00"))
+        strings = list(iter(unpack_string, ""))
+        if not strings:
+            u.skip(1)
+
+        self.strings_length = u.offset - self.strings_offset
+        self.raw_strings = str(bits.memory(sm.header.structure_table_address + self.strings_offset, self.strings_length))
+
+        if len(strings):
+            self.strings = strings
+
+        self.add_field('type', self.u.unpack_one("B"))
+        self.add_field('length', self.u.unpack_one("B"))
+        self.add_field('handle', self.u.unpack_one("<H"))
+
+    def fini(self):
+        if not self.u.at_end():
+            self.add_field('data', self.u.unpack_rest())
+        del self.u
+
+    def fmtstr(self, i):
+        """Format the specified index and the associated string"""
+        return "{} '{}'".format(i, self.getstr(i))
+
+    def getstr(self, i):
+        """Get the string associated with the given index"""
+        if i == 0:
+            return "(none)"
+        if not hasattr(self, "strings"):
+            return "(error: structure has no strings)"
+        if i > len(self.strings):
+            return "(error: string index out of range)"
+        return self.strings[i - 1]
+
+class BIOSInformation(SmbiosBaseStructure):
+    smbios_structure_type = 0
+
+    def __init__(self, u, sm):
+        super(BIOSInformation, self).__init__(u, sm)
+        u = self.u
+        try:
+            self.add_field('vendor', u.unpack_one("B"), self.fmtstr)
+            self.add_field('version', u.unpack_one("B"), self.fmtstr)
+            self.add_field('starting_address_segment', u.unpack_one("<H"))
+            self.add_field('release_date', u.unpack_one("B"), self.fmtstr)
+            self.add_field('rom_size', u.unpack_one("B"))
+            self.add_field('characteristics', u.unpack_one("<Q"))
+            minor_version_str = str(sm.header.minor_version) # 34 is .34, 4 is .4, 41 is .41; compare ASCIIbetically to compare initial digits rather than numeric value
+            if (sm.header.major_version, minor_version_str) >= (2,"4"):
+                characteristic_bytes = 2
+            else:
+                characteristic_bytes = self.length - 0x12
+            self.add_field('characteristics_extensions', [u.unpack_one("B") for b in range(characteristic_bytes)])
+            if (sm.header.major_version, minor_version_str) >= (2,"4"):
+                self.add_field('major_release', u.unpack_one("B"))
+                self.add_field('minor_release', u.unpack_one("B"))
+                self.add_field('ec_major_release', u.unpack_one("B"))
+                self.add_field('ec_minor_release', u.unpack_one("B"))
+        except:
+            self.decode_failure = True
+            print "Error parsing BIOSInformation"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class SystemInformation(SmbiosBaseStructure):
+    smbios_structure_type = 1
+
+    def __init__(self, u, sm):
+        super(SystemInformation, self).__init__(u, sm)
+        u = self.u
+        try:
+            self.add_field('manufacturer', u.unpack_one("B"), self.fmtstr)
+            self.add_field('product_name', u.unpack_one("B"), self.fmtstr)
+            self.add_field('version', u.unpack_one("B"), self.fmtstr)
+            self.add_field('serial_number', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x8:
+                self.add_field('uuid', uuid.UUID(bytes_le=u.unpack_one("16s")))
+                wakeup_types = {
+                    0: 'Reserved',
+                    1: 'Other',
+                    2: 'Unknown',
+                    3: 'APM Timer',
+                    4: 'Modem Ring',
+                    5: 'LAN Remote',
+                    6: 'Power Switch',
+                    7: 'PCI PME#',
+                    8: 'AC Power Restored'
+                }
+                self.add_field('wakeup_type', u.unpack_one("B"), unpack.format_table("{}", wakeup_types))
+            if self.length > 0x19:
+                self.add_field('sku_number', u.unpack_one("B"), self.fmtstr)
+                self.add_field('family', u.unpack_one("B"), self.fmtstr)
+        except:
+            self.decode_failure = True
+            print "Error parsing SystemInformation"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+_board_types = {
+    1: 'Unknown',
+    2: 'Other',
+    3: 'Server Blade',
+    4: 'Connectivity Switch',
+    5: 'System Management Module',
+    6: 'Processor Module',
+    7: 'I/O Module',
+    8: 'Memory Module',
+    9: 'Daughter Board',
+    0xA: 'Motherboard',
+    0xB: 'Processor/Memory Module',
+    0xC: 'Processor/IO Module',
+    0xD: 'Interconnect Board'
+}
+
+class BaseboardInformation(SmbiosBaseStructure):
+    smbios_structure_type = 2
+
+    def __init__(self, u, sm):
+        super(BaseboardInformation, self).__init__(u, sm)
+        u = self.u
+        try:
+            self.add_field('manufacturer', u.unpack_one("B"), self.fmtstr)
+            self.add_field('product', u.unpack_one("B"), self.fmtstr)
+            self.add_field('version', u.unpack_one("B"), self.fmtstr)
+            self.add_field('serial_number', u.unpack_one("B"), self.fmtstr)
+
+            if self.length > 0x8:
+                self.add_field('asset_tag', u.unpack_one("B"), self.fmtstr)
+
+            if self.length > 0x9:
+                self.add_field('feature_flags', u.unpack_one("B"))
+                self.add_field('hosting_board', bool(bitfields.getbits(self.feature_flags, 0)), "feature_flags[0]={}")
+                self.add_field('requires_daughter_card', bool(bitfields.getbits(self.feature_flags, 1)), "feature_flags[1]={}")
+                self.add_field('removable', bool(bitfields.getbits(self.feature_flags, 2)), "feature_flags[2]={}")
+                self.add_field('replaceable', bool(bitfields.getbits(self.feature_flags, 3)), "feature_flags[3]={}")
+                self.add_field('hot_swappable', bool(bitfields.getbits(self.feature_flags, 4)), "feature_flags[4]={}")
+
+            if self.length > 0xA:
+                self.add_field('location', u.unpack_one("B"), self.fmtstr)
+
+            if self.length > 0xB:
+                self.add_field('chassis_handle', u.unpack_one("<H"))
+
+            if self.length > 0xD:
+                self.add_field('board_type', u.unpack_one("B"), unpack.format_table("{}", _board_types))
+
+            if self.length > 0xE:
+                self.add_field('handle_count', u.unpack_one("B"))
+                if self.handle_count > 0:
+                    self.add_field('contained_object_handles', tuple(u.unpack_one("<H") for i in range(self.handle_count)))
+        except:
+            self.decode_failure = True
+            print "Error parsing BaseboardInformation"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class SystemEnclosure(SmbiosBaseStructure):
+    smbios_structure_type = 3
+
+    def __init__(self, u, sm):
+        super(SystemEnclosure, self).__init__(u, sm)
+        u = self.u
+        try:
+            self.add_field('manufacturer', u.unpack_one("B"), self.fmtstr)
+            self.add_field('enumerated_type', u.unpack_one("B"))
+            self.add_field('chassis_lock_present', bool(bitfields.getbits(self.enumerated_type, 7)), "enumerated_type[7]={}")
+            board_types = {
+                0x01: 'Other',
+                0x02: 'Unknown',
+                0x03: 'Desktop',
+                0x04: 'Low Profile Desktop',
+                0x05: 'Pizza Box',
+                0x06: 'Mini Tower',
+                0x07: 'Tower',
+                0x08: 'Portable',
+                0x09: 'Laptop',
+                0x0A: 'Notebook',
+                0x0B: 'Hand Held',
+                0x0C: 'Docking Station',
+                0x0D: 'All in One',
+                0x0E: 'Sub Notebook',
+                0x0F: 'Space-saving',
+                0x10: 'Lunch Box',
+                0x11: 'Main Server Chassis',
+                0x12: 'Expansion Chassis',
+                0x13: 'SubChassis',
+                0x14: 'Bus Expansion Chassis',
+                0x15: 'Peripheral Chassis',
+                0x16: 'RAID Chassis',
+                0x17: 'Rack Mount Chassis',
+                0x18: 'Sealed-case PC',
+                0x19: 'Multi-system chassis W',
+                0x1A: 'Compact PCI',
+                0x1B: 'Advanced TCA',
+                0x1C: 'Blade',
+                0x1D: 'Blade Enclosure',
+            }
+            self.add_field('system_enclosure_type', bitfields.getbits(self.enumerated_type, 6, 0), unpack.format_table("enumerated_type[6:0]={}", board_types))
+            self.add_field('version', u.unpack_one("B"), self.fmtstr)
+            self.add_field('serial_number', u.unpack_one("B"), self.fmtstr)
+            self.add_field('asset_tag', u.unpack_one("B"), self.fmtstr)
+            minor_version_str = str(sm.header.minor_version) # 34 is .34, 4 is .4, 41 is .41; compare ASCIIbetically to compare initial digits rather than numeric value
+            if self.length > 9:
+                chassis_states = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'Safe',
+                    0x04: 'Warning',
+                    0x05: 'Critical',
+                    0x06: 'Non-recoverable',
+                }
+                self.add_field('bootup_state', u.unpack_one("B"), unpack.format_table("{}", chassis_states))
+                self.add_field('power_supply_state', u.unpack_one("B"), unpack.format_table("{}", chassis_states))
+                self.add_field('thermal_state', u.unpack_one("B"), unpack.format_table("{}", chassis_states))
+                security_states = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'None',
+                    0x04: 'External interface locked out',
+                    0x05: 'External interface enabled',
+                }
+                self.add_field('security_status', u.unpack_one("B"), unpack.format_table("{}", security_states))
+            if self.length > 0xd:
+                self.add_field('oem_defined', u.unpack_one("<I"))
+            if self.length > 0x11:
+                self.add_field('height', u.unpack_one("B"))
+                self.add_field('num_power_cords', u.unpack_one("B"))
+                self.add_field('contained_element_count', u.unpack_one("B"))
+                self.add_field('contained_element_length', u.unpack_one("B"))
+            if getattr(self, 'contained_element_count', 0):
+                self.add_field('contained_elements', tuple(SystemEnclosureContainedElement(u, self.contained_element_length) for i in range(self.contained_element_count)))
+            if self.length > (0x15 + (getattr(self, 'contained_element_count', 0) * getattr(self, 'contained_element_length', 0))):
+                self.add_field('sku_number', u.unpack_one("B"), self.fmtstr)
+        except:
+            self.decode_failure = True
+            print "Error parsing SystemEnclosure"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class SystemEnclosureContainedElement(unpack.Struct):
+    def __init__(self, u, length):
+        super(SystemEnclosureContainedElement, self).__init__()
+        self.start_offset = u.offset
+        self.raw_data = u.unpack_raw(length)
+        self.u = unpack.Unpackable(self.raw_data)
+        u = self.u
+        self.add_field('contained_element_type', u.unpack_one("B"))
+        type_selections = {
+            0: 'SMBIOS baseboard type enumeration',
+            1: 'SMBIOS structure type enumeration',
+        }
+        self.add_field('type_select', bitfields.getbits(self.contained_element_type, 7), unpack.format_table("contained_element_type[7]={}", type_selections))
+        self.add_field('type', bitfields.getbits(self.contained_element_type, 6, 0))
+        if self.type_select == 0:
+            self.add_field('smbios_board_type', self.type, unpack.format_table("{}", _board_types))
+        else:
+            self.add_field('smbios_structure_type', self.type)
+        self.add_field('minimum', u.unpack_one("B"))
+        self.add_field('maximum', u.unpack_one("B"))
+        if not u.at_end():
+            self.add_field('data', u.unpack_rest())
+        del self.u
+
+class ProcessorInformation(SmbiosBaseStructure):
+    smbios_structure_type = 4
+
+    def __init__(self, u, sm):
+        super(ProcessorInformation, self).__init__(u, sm)
+        u = self.u
+        try:
+            self.add_field('socket_designation', u.unpack_one("B"), self.fmtstr)
+            processor_types = {
+                0x01: 'Other',
+                0x02: 'Unknown',
+                0x03: 'Central Processor',
+                0x04: 'Math Processor',
+                0x05: 'DSP Processor',
+                0x06: 'Video Processor',
+            }
+            self.add_field('processor_type', u.unpack_one("B"), unpack.format_table("{}", processor_types))
+            self.add_field('processor_family', u.unpack_one("B"))
+            self.add_field('processor_manufacturer', u.unpack_one("B"), self.fmtstr)
+            self.add_field('processor_id', u.unpack_one("<Q"))
+            self.add_field('processor_version', u.unpack_one("B"), self.fmtstr)
+            self.add_field('voltage', u.unpack_one("B"))
+            self.add_field('external_clock', u.unpack_one("<H"))
+            self.add_field('max_speed', u.unpack_one("<H"))
+            self.add_field('current_speed', u.unpack_one("<H"))
+            self.add_field('status', u.unpack_one("B"))
+            processor_upgrades = {
+                0x01: 'Other',
+                0x02: 'Unknown',
+                0x03: 'Daughter Board',
+                0x04: 'ZIF Socket',
+                0x05: 'Replaceable Piggy Back',
+                0x06: 'None',
+                0x07: 'LIF Socket',
+                0x08: 'Slot 1',
+                0x09: 'Slot 2',
+                0x0A: '370-pin socket',
+                0x0B: 'Slot A',
+                0x0C: 'Slot M',
+                0x0D: 'Socket 423',
+                0x0E: 'Socket A (Socket 462)',
+                0x0F: 'Socket 478',
+                0x10: 'Socket 754',
+                0x11: 'Socket 940',
+                0x12: 'Socket 939',
+                0x13: 'Socket mPGA604',
+                0x14: 'Socket LGA771',
+                0x15: 'Socket LGA775',
+                0x16: 'Socket S1',
+                0x17: 'Socket AM2',
+                0x18: 'Socket F (1207)',
+                0x19: 'Socket LGA1366',
+                0x1A: 'Socket G34',
+                0x1B: 'Socket AM3',
+                0x1C: 'Socket C32',
+                0x1D: 'Socket LGA1156',
+                0x1E: 'Socket LGA1567',
+                0x1F: 'Socket PGA988A',
+                0x20: 'Socket BGA1288',
+                0x21: 'Socket rPGA988B',
+                0x22: 'Socket BGA1023',
+                0x23: 'Socket BGA1224',
+                0x24: 'Socket BGA1155',
+                0x25: 'Socket LGA1356',
+                0x26: 'Socket LGA2011',
+                0x27: 'Socket FS1',
+                0x28: 'Socket FS2',
+                0x29: 'Socket FM1',
+                0x2A: 'Socket FM2',
+            }
+            self.add_field('processor_upgrade', u.unpack_one("B"), unpack.format_table("{}", processor_upgrades))
+            if self.length > 0x1A:
+                self.add_field('l1_cache_handle', u.unpack_one("<H"))
+                self.add_field('l2_cache_handle', u.unpack_one("<H"))
+                self.add_field('l3_cache_handle', u.unpack_one("<H"))
+            if self.length > 0x20:
+                self.add_field('serial_number', u.unpack_one("B"), self.fmtstr)
+                self.add_field('asset_tag', u.unpack_one("B"), self.fmtstr)
+                self.add_field('part_number', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x24:
+                self.add_field('core_count', u.unpack_one("B"))
+                self.add_field('core_enabled', u.unpack_one("B"))
+                self.add_field('thread_count', u.unpack_one("B"))
+                self.add_field('processor_characteristics', u.unpack_one("<H"))
+            if self.length > 0x28:
+                self.add_field('processor_family_2', u.unpack_one("<H"))
+            if self.length > 0x2A:
+                self.add_field('core_count2', u.unpack_one("<H"))
+                self.add_field('core_enabled2', u.unpack_one("<H"))
+                self.add_field('thread_count2', u.unpack_one("<H"))
+        except:
+            self.decode_failure = True
+            print "Error parsing Processor Information"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class MemoryControllerInformation(SmbiosBaseStructure): #obsolete starting with v2.1
+    smbios_structure_type = 5
+
+    def __init__(self, u, sm):
+        super(MemoryControllerInformation, self).__init__(u, sm)
+        u = self.u
+        try:
+            _error_detecting_method = {
+                0x01: 'Other',
+                0x02: 'Unknown',
+                0x03: 'None',
+                0x04: '8-bit Parity',
+                0x05: '32-bit ECC',
+                0x06: '64-bit ECC',
+                0x07: '128-bit ECC',
+                0x08: 'CRC'
+                }
+            self.add_field('error_detecting_method', u.unpack_one("B"), unpack.format_table("{}", _error_detecting_method))
+            self.add_field('error_correcting_capability', u.unpack_one("B"))
+            _interleaves = {
+                0x01: 'Other',
+                0x02: 'Unknown',
+                0x03: 'One-Way Interleave',
+                0x04: 'Two-Way Interleave',
+                0x05: 'Four-Way Interleave',
+                0x06: 'Eight-Way Interleave',
+                0x07: 'Sixteen-Way Interleave'
+                }
+            self.add_field('supported_interleave', u.unpack_one("B"), unpack.format_table("{}", _interleaves))
+            self.add_field('current_interleave', u.unpack_one("B"), unpack.format_table("{}", _interleaves))
+            self.add_field('max_memory_module_size', u.unpack_one("B"), self.fmtstr)
+            self.add_field('supported_speeds', u.unpack_one("<H"))
+            self.add_field('supported_memory_types', u.unpack_one("<H"))
+            self.add_field('memory_module_voltage', u.unpack_one("B"))
+            self.add_field('req_voltage_b2', bitfields.getbits(self.memory_module_voltage, 2), "memory_module_voltage[2]={}")
+            self.add_field('req_voltage_b1', bitfields.getbits(self.memory_module_voltage, 1), "memory_module_voltage[1]={}")
+            self.add_field('req_voltage_b0', bitfields.getbits(self.memory_module_voltage, 0), "memory_module_voltage[0]={}")
+            self.add_field('num_associated_memory_slots', u.unpack_one("B"))
+            self.add_field('memory_module_configuration_handles', u.unpack_one("<(self.num_associated_memory_slots)H"))
+            self.add_field('enabled_error_correcting_capabilities', u.unpack_one("B"))
+        except:
+            self.decode_failure = True
+            print "Error parsing MemoryControllerInformation"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class MemoryModuleInformation(SmbiosBaseStructure): #obsolete starting with v2.1
+    smbios_structure_type = 6
+
+    def __init__(self, u, sm):
+        super(MemoryModuleInformation, self).__init__(u, sm)
+        u = self.u
+        try:
+            self.add_field('socket_designation', u.unpack_one("B"), self.fmtstr)
+            self.add_field('bank_connections', u.unpack_one("B"))
+            self.add_field('current_speed', u.unpack_one("B"))
+            self.add_field('current_memory_type', u.unpack_one("<H"))
+            _mem_connection = {
+                0: 'single',
+                1: 'double-bank'
+                }
+            self.add_field('installed_mem', u.unpack_one("B"))
+            self.add_field('installed_size', bitfields.getbits(self.installed_mem, 6, 0), "installed_mem[6:0]={}")
+            self.add_field('installed_memory_module_connection', bitfields.getbits(self.installed_mem, 7), unpack.format_table("installed_mem[7]={}", _mem_connection))
+            self.add_field('enabled_mem', u.unpack_one("B"))
+            self.add_field('enabled_size', bitfields.getbits(self.installed_mem, 6, 0), "enabled_mem[6:0]={}")
+            self.add_field('enabled_memory_module_connection', bitfields.getbits(self.installed_mem, 7), unpack.format_table("enabled_mem[7]={}", _mem_connection))
+            self.add_field('error_status', u.unpack_one("B"))
+            self.add_field('error_status_info_obstained_from_event_log', bool(bitfields.getbits(self.error_status, 2)), unpack.format_table("error_status[2]={}", _mem_connection))
+            self.add_field('correctable_errors_received', bool(bitfields.getbits(self.error_status, 1)), unpack.format_table("error_status[1]={}", _mem_connection))
+            self.add_field('uncorrectable_errors_received', bool(bitfields.getbits(self.error_status, 0)), unpack.format_table("error_status[0]={}", _mem_connection))
+        except:
+            self.decode_failure = True
+            print "Error parsing MemoryModuleInformation"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class CacheInformation(SmbiosBaseStructure):
+    smbios_structure_type = 7
+
+    def __init__(self, u, sm):
+        super(CacheInformation, self).__init__(u, sm)
+        u = self.u
+        try:
+            self.add_field('socket_designation', u.unpack_one("B"), self.fmtstr)
+            processor_types = {
+                0x01: 'Other',
+                0x02: 'Unknown',
+                0x03: 'Central Processor',
+                0x04: 'Math Processor',
+                0x05: 'DSP Processor',
+                0x06: 'Video Processor',
+            }
+            self.add_field('cache_configuration', u.unpack_one("<H"))
+            _operational_mode = {
+                0b00: 'Write Through',
+                0b01: 'Write Back',
+                0b10: 'Varies with Memory Address',
+                0b11: 'Unknown'
+                }
+            self.add_field('operational_mode', bitfields.getbits(self.cache_configuration, 9, 8), unpack.format_table("cache_configuration[9:8]={}", _operational_mode))
+            self.add_field('enabled_at_boot_time', bool(bitfields.getbits(self.cache_configuration, 7)), "cache_configuration[7]={}")
+            _location = {
+                0b00: 'Internal',
+                0b01: 'External',
+                0b10: 'Reserved',
+                0b11: 'Unknown'
+                }
+            self.add_field('location_relative_to_cpu_module', bitfields.getbits(self.cache_configuration, 6, 5), unpack.format_table("cache_configuration[6:5]={}", _location))
+            self.add_field('cache_socketed', bool(bitfields.getbits(self.cache_configuration, 3)), "cache_configuration[3]={}")
+            self.add_field('cache_level', bitfields.getbits(self.cache_configuration, 2, 0), "cache_configuration[2:0]={}")
+            self.add_field('max_cache_size', u.unpack_one("<H"))
+            _granularity = {
+                0: '1K granularity',
+                1: '64K granularity'
+                }
+            self.add_field('max_granularity', bitfields.getbits(self.cache_configuration, 15), unpack.format_table("max_cache_size[15]={}", _granularity))
+            self.add_field('max_size_in_granularity', bitfields.getbits(self.cache_configuration, 14, 0), "max_cache_size[14, 0]={}")
+            self.add_field('installed_size', u.unpack_one("<H"))
+            if self.installed_size != 0:
+                self.add_field('installed_granularity', bitfields.getbits(self.cache_configuration, 15), unpack.format_table("installed_size[15]={}", _granularity))
+                self.add_field('installed_size_in_granularity', bitfields.getbits(self.cache_configuration, 14, 0), "installed_size[14, 0]={}")
+            self.add_field('supported_sram_type', u.unpack_one("<H"))
+            self.add_field('current_sram_type', u.unpack_one("<H"))
+            if self.length > 0x0F:
+                self.add_field('cache_speed', u.unpack_one("B"))
+            if self.length > 0x10:
+                _error_correction = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'None',
+                    0x04: 'Parity',
+                    0x05: 'Single-bit ECC',
+                    0x06: 'Multi-bit ECC'
+                    }
+                self.add_field('error_correction', u.unpack_one("B"), unpack.format_table("{}", _error_correction))
+            if self.length > 0x10:
+                _system_cache_type = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'Instruction',
+                    0x04: 'Data',
+                    0x05: 'Unified'
+                    }
+                self.add_field('system_cache_type', u.unpack_one("B"), unpack.format_table("{}", _system_cache_type))
+            if self.length > 0x12:
+                _associativity = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'Direct Mapped',
+                    0x04: '2-way Set-Associative',
+                    0x05: '4-way Set-Associative',
+                    0x06: 'Fully Associative',
+                    0x07: '8-way Set-Associative',
+                    0x08: '16-way Set-Associative',
+                    0x09: '12-way Set-Associative',
+                    0x0A: '24-way Set-Associative',
+                    0x0B: '32-way Set-Associative',
+                    0x0C: '48-way Set-Associative',
+                    0x0D: '64-way Set-Associative',
+                    0x0E: '20-way Set-Associative'
+                    }
+                self.add_field('associativity', u.unpack_one("B"), unpack.format_table("{}", _associativity))
+
+        except:
+            self.decode_failure = True
+            print "Error parsing CacheInformation"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class PortConnectorInfo(SmbiosBaseStructure):
+    smbios_structure_type = 8
+
+    def __init__(self, u, sm):
+        super(PortConnectorInfo, self).__init__(u, sm)
+        u = self.u
+        try:
+            self.add_field('internal_reference_designator', u.unpack_one("B"), self.fmtstr)
+            connector_types = {
+                0x00: 'None',
+                0x01: 'Centronics',
+                0x02: 'Mini Centronics',
+                0x03: 'Proprietary',
+                0x04: 'DB-25 pin male',
+                0x05: 'DB-25 pin female',
+                0x06: 'DB-15 pin male',
+                0x07: 'DB-15 pin female',
+                0x08: 'DB-9 pin male',
+                0x09: 'DB-9 pin female',
+                0x0A: 'RJ-11',
+                0x0B: 'RJ-45',
+                0x0C: '50-pin MiniSCSI',
+                0x0D: 'Mini-DIN',
+                0x0E: 'Micro-DIN',
+                0x0F: 'PS/2',
+                0x10: 'Infrared',
+                0x11: 'HP-HIL',
+                0x12: 'Access Bus (USB)',
+                0x13: 'SSA SCSI',
+                0x14: 'Circular DIN-8 male',
+                0x15: 'Circular DIN-8 female',
+                0x16: 'On Board IDE',
+                0x17: 'On Board Floppy',
+                0x18: '9-pin Dual Inline (pin 10 cut)',
+                0x19: '25-pin Dual Inline (pin 26 cut)',
+                0x1A: '50-pin Dual Inline',
+                0x1B: '68-pin Dual Inline',
+                0x1C: 'On Board Sound Input from CD-ROM',
+                0x1D: 'Mini-Centronics Type-14',
+                0x1E: 'Mini-Centronics Type-26',
+                0x1F: 'Mini-jack (headphones)',
+                0x20: 'BNC',
+                0x21: '1394',
+                0x22: 'SAS/SATA Plug Receptacle',
+                0xA0: 'PC-98',
+                0xA1: 'PC-98Hireso',
+                0xA2: 'PC-H98',
+                0xA3: 'PC-98Note',
+                0xA4: 'PC-98Full',
+                0xFF: 'Other',
+            }
+            self.add_field('internal_connector_type', u.unpack_one("B"), unpack.format_table("{}", connector_types))
+            self.add_field('external_reference_designator', u.unpack_one("B"), self.fmtstr)
+            self.add_field('external_connector_type', u.unpack_one("B"), unpack.format_table("{}", connector_types))
+            port_types = {
+                0x00: 'None',
+                0x01: 'Parallel Port XT/AT Compatible',
+                0x02: 'Parallel Port PS/2',
+                0x03: 'Parallel Port ECP',
+                0x04: 'Parallel Port EPP',
+                0x05: 'Parallel Port ECP/EPP',
+                0x06: 'Serial Port XT/AT Compatible',
+                0x07: 'Serial Port 16450 Compatible',
+                0x08: 'Serial Port 16550 Compatible',
+                0x09: 'Serial Port 16550A Compatible',
+                0x0A: 'SCSI Port',
+                0x0B: 'MIDI Port',
+                0x0C: 'Joy Stick Port',
+                0x0D: 'Keyboard Port',
+                0x0E: 'Mouse Port',
+                0x0F: 'SSA SCSI',
+                0x10: 'USB',
+                0x11: 'FireWire (IEEE P1394)',
+                0x12: 'PCMCIA Type I2',
+                0x13: 'PCMCIA Type II',
+                0x14: 'PCMCIA Type III',
+                0x15: 'Cardbus',
+                0x16: 'Access Bus Port',
+                0x17: 'SCSI II',
+                0x18: 'SCSI Wide',
+                0x19: 'PC-98',
+                0x1A: 'PC-98-Hireso',
+                0x1B: 'PC-H98',
+                0x1C: 'Video Port',
+                0x1D: 'Audio Port',
+                0x1E: 'Modem Port',
+                0x1F: 'Network Port',
+                0x20: 'SATA',
+                0x21: 'SAS',
+                0xA0: '8251 Compatible',
+                0xA1: '8251 FIFO Compatible',
+                0xFF: 'Other',
+            }
+            self.add_field('port_type', u.unpack_one("B"), unpack.format_table("{}", port_types))
+        except:
+            self.decodeFailure = True
+            print "Error parsing PortConnectorInfo"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class SystemSlots(SmbiosBaseStructure):
+    smbios_structure_type = 9
+
+    def __init__(self, u, sm):
+        super(SystemSlots, self).__init__(u, sm)
+        u = self.u
+        try:
+            self.add_field('designation', u.unpack_one("B"), self.fmtstr)
+            _slot_types = {
+                0x01: 'Other',
+                0x02: 'Unknown',
+                0x03: 'ISA',
+                0x04: 'MCA',
+                0x05: 'EISA',
+                0x06: 'PCI',
+                0x07: 'PC Card (PCMCIA)',
+                0x08: 'VL-VESA',
+                0x09: 'Proprietary',
+                0x0A: 'Processor Card Slot',
+                0x0B: 'Proprietary Memory Card Slot',
+                0x0C: 'I/O Riser Card Slot',
+                0x0D: 'NuBus',
+                0x0E: 'PCI 66MHz Capable',
+                0x0F: 'AGP',
+                0x10: 'AGP 2X',
+                0x11: 'AGP 4X',
+                0x12: 'PCI-X',
+                0x13: 'AGP 8X',
+                0xA0: 'PC-98/C20',
+                0xA1: 'PC-98/C24',
+                0xA2: 'PC-98/E',
+                0xA3: 'PC-98/Local Bus',
+                0xA4: 'PC-98/Card',
+                0xA5: 'PCI Express',
+                0xA6: 'PCI Express x1',
+                0xA7: 'PCI Express x2',
+                0xA8: 'PCI Express x4',
+                0xA9: 'PCI Express x8',
+                0xAA: 'PCI Express x16',
+                0xAB: 'PCI Express Gen 2',
+                0xAC: 'PCI Express Gen 2 x1',
+                0xAD: 'PCI Express Gen 2 x2',
+                0xAE: 'PCI Express Gen 2 x4',
+                0xAF: 'PCI Express Gen 2 x8',
+                0xB0: 'PCI Express Gen 2 x16',
+                0xB1: 'PCI Express Gen 3',
+                0xB2: 'PCI Express Gen 3 x1',
+                0xB3: 'PCI Express Gen 3 x2',
+                0xB4: 'PCI Express Gen 3 x4',
+                0xB5: 'PCI Express Gen 3 x8',
+                0xB6: 'PCI Express Gen 3 x16',
+            }
+            self.add_field('slot_type', u.unpack_one("B"), unpack.format_table("{}", _slot_types))
+            _slot_data_bus_widths = {
+                0x01: 'Other',
+                0x02: 'Unknown',
+                0x03: '8 bit',
+                0x04: '16 bit',
+                0x05: '32 bit',
+                0x06: '64 bit',
+                0x07: '128 bit',
+                0x08: '1x or x1',
+                0x09: '2x or x2',
+                0x0A: '4x or x4',
+                0x0B: '8x or x8',
+                0x0C: '12x or x12',
+                0x0D: '16x or x16',
+                0x0E: '32x or x32',
+            }
+            self.add_field('slot_data_bus_width', u.unpack_one('B'), unpack.format_table("{}", _slot_data_bus_widths))
+            _current_usages = {
+                0x01: 'Other',
+                0x02: 'Unknown',
+                0x03: 'Available',
+                0x04: 'In use',
+            }
+            self.add_field('current_usage', u.unpack_one('B'), unpack.format_table("{}", _current_usages))
+            _slot_lengths = {
+                0x01: 'Other',
+                0x02: 'Unknown',
+                0x03: 'Short Length',
+                0x04: 'Long Length',
+            }
+            self.add_field('slot_length', u.unpack_one('B'), unpack.format_table("{}", _slot_lengths))
+            self.add_field('slot_id', u.unpack_one('<H'))
+            self.add_field('characteristics1', u.unpack_one('B'))
+            self.add_field('characteristics_unknown', bool(bitfields.getbits(self.characteristics1, 0)), "characteristics1[0]={}")
+            self.add_field('provides_5_0_volts', bool(bitfields.getbits(self.characteristics1, 1)), "characteristics1[1]={}")
+            self.add_field('provides_3_3_volts', bool(bitfields.getbits(self.characteristics1, 2)), "characteristics1[2]={}")
+            self.add_field('shared_slot', bool(bitfields.getbits(self.characteristics1, 3)), "characteristics1[3]={}")
+            self.add_field('supports_pc_card_16', bool(bitfields.getbits(self.characteristics1, 4)), "characteristics1[4]={}")
+            self.add_field('supports_cardbus', bool(bitfields.getbits(self.characteristics1, 5)), "characteristics1[5]={}")
+            self.add_field('supports_zoom_video', bool(bitfields.getbits(self.characteristics1, 6)), "characteristics1[6]={}")
+            self.add_field('supports_modem_ring_resume', bool(bitfields.getbits(self.characteristics1, 7)), "characteristics1[7]={}")
+            if self.length > 0x0C:
+                self.add_field('characteristics2', u.unpack_one('B'))
+                self.add_field('supports_PME', bool(bitfields.getbits(self.characteristics2, 0)), "characteristics2[0]={}")
+                self.add_field('supports_hot_plug', bool(bitfields.getbits(self.characteristics2, 1)), "characteristics2[1]={}")
+                self.add_field('supports_smbus', bool(bitfields.getbits(self.characteristics2, 2)), "characteristics2[2]={}")
+            if self.length > 0x0D:
+                self.add_field('segment_group_number', u.unpack_one('<H'))
+                self.add_field('bus_number', u.unpack_one('B'))
+                self.add_field('device_function_number', u.unpack_one('B'))
+                self.add_field('device_number', bitfields.getbits(self.device_function_number, 7, 3), "device_function_number[7:3]={}")
+                self.add_field('function_number', bitfields.getbits(self.device_function_number, 2, 0), "device_function_number[2:0]={}")
+        except:
+            self.decodeFailure = True
+            print "Error parsing SystemSlots"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class OnBoardDevicesInformation(SmbiosBaseStructure):
+    smbios_structure_type = 10
+
+    def __init__(self, u, sm):
+        super(OnBoardDevicesInformation, self).__init__(u, sm)
+        u = self.u
+        try:
+            self.add_field('device_type', u.unpack_one("B"))
+            self.add_field('device_enabled', bool(bitfields.getbits(self.device_type, 7)), "device_type[7]={}")
+            _device_types = {
+                0x01: 'Other',
+                0x02: 'Unknown',
+                0x03: 'Video',
+                0x04: 'SCSI Controller',
+                0x05: 'Ethernet',
+                0x06: 'Token Ring',
+                0x07: 'Sound',
+                0x08: 'PATA Controller',
+                0x09: 'SATA Controller',
+                0x0A: 'SAS Controller'
+            }
+            self.add_field('type_of_device', bitfields.getbits(self.device_type, 6, 0), unpack.format_table("device_type[6:0]={}", _device_types))
+            self.add_field('description_string', u.unpack_one("B"), self.fmtstr)
+        except:
+            self.decodeFailure = True
+            print "Error parsing OnBoardDevicesInformation"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class OEMStrings(SmbiosBaseStructure):
+    smbios_structure_type = 11
+
+    def __init__(self, u, sm):
+        super(OEMStrings, self).__init__(u, sm)
+        u = self.u
+        try:
+            self.add_field('count', u.unpack_one("B"))
+        except:
+            self.decodeFailure = True
+            print "Error parsing OEMStrings"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class SystemConfigOptions(SmbiosBaseStructure):
+    smbios_structure_type = 12
+
+    def __init__(self, u, sm):
+        super(SystemConfigOptions, self).__init__(u, sm)
+        u = self.u
+        try:
+            self.add_field('count', u.unpack_one("B"))
+        except:
+            self.decodeFailure = True
+            print "Error parsing SystemConfigOptions"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class BIOSLanguageInformation(SmbiosBaseStructure):
+    smbios_structure_type = 13
+
+    def __init__(self, u, sm):
+        super(BIOSLanguageInformation, self).__init__(u, sm)
+        u = self.u
+        try:
+            self.add_field('installable_languages', u.unpack_one("B"))
+            if self.length > 0x05:
+                self.add_field('flags', u.unpack_one('B'))
+                self.add_field('abbreviated_format', bool(bitfields.getbits(self.flags, 0)), "flags[0]={}")
+            if self.length > 0x6:
+                u.skip(15)
+                self.add_field('current_language', u.unpack_one('B'), self.fmtstr)
+        except:
+            self.decodeFailure = True
+            print "Error parsing BIOSLanguageInformation"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class GroupAssociations(SmbiosBaseStructure):
+    smbios_structure_type = 14
+
+    def __init__(self, u, sm):
+        super(GroupAssociations, self).__init__(u, sm)
+        u = self.u
+        try:
+            self.add_field('group_name', u.unpack_one("B"), self.fmtstr)
+            self.add_field('item_type', u.unpack_one('B'))
+            self.add_field('item_handle', u.unpack_one('<H'))
+        except:
+            self.decodeFailure = True
+            print "Error parsing GroupAssociations"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class SystemEventLog(SmbiosBaseStructure):
+    smbios_structure_type = 15
+
+    def __init__(self, u, sm):
+        super(SystemEventLog, self).__init__(u, sm)
+        u = self.u
+        try:
+            self.add_field('log_area_length', u.unpack_one("<H"))
+            self.add_field('log_header_start_offset', u.unpack_one('<H'))
+            self.add_field('log_data_start_offset', u.unpack_one('<H'))
+            _access_method = {
+                0x00: 'Indexed I/O: 1 8-bit index port, 1 8-bit data port',
+                0x01: 'Indexed I/O: 2 8-bit index ports, 1 8-bit data port',
+                0x02: 'Indexed I/O: 1 16-bit index port, 1 8-bit data port',
+                0x03: 'Memory-mapped physical 32-bit address',
+                0x04: 'Available through General-Purpose NonVolatile Data functions',
+                xrange(0x05, 0x07F): 'Available for future assignment',
+                xrange(0x80, 0xFF): 'BIOS Vendor/OEM-specific'
+                }
+            self.add_field('access_method', u.unpack_one('B'), unpack.format_table("{}", _access_method))
+            self.add_field('log_status', u.unpack_one('B'))
+            self.add_field('log_area_full', bool(bitfields.getbits(self.log_status, 1)), "log_status[1]={}")
+            self.add_field('log_area_valid', bool(bitfields.getbits(self.log_status, 0)), "log_status[0]={}")
+            self.add_field('log_change_token', u.unpack_one('<I'))
+            self.add_field('access_method_address', u.unpack_one('<I'))
+            if self.length > 0x14:
+                _log_header_formats = {
+                    0: 'No header',
+                    1: 'Type 1 log header',
+                    xrange(2, 0x7f): 'Available for future assignment',
+                    xrange(0x80, 0xff): 'BIOS vendor or OEM-specific format'
+                    }
+                self.add_field('log_header_format', u.unpack_one("B"), unpack.format_table("{}", _log_header_formats))
+            if self.length > 0x15:
+                self.add_field('num_supported_log_type_descriptors', u.unpack_one('B'))
+            if self.length > 0x16:
+                self.add_field('length_log_type_descriptor', u.unpack_one('B'))
+            if self.length != (0x17 + (self.num_supported_log_type_descriptors * self.length_log_type_descriptor)):
+                print "Error: structure length ({}) != 0x17 + (num_supported_log_type_descriptors ({}) * length_log_type_descriptor({}))".format(self.length, self.num_supported_log_type_descriptors, self.length_log_type_descriptor)
+                print "structure length = {}".format(self.length)
+                print "num_supported_log_type_descriptors = {}".format(self.num_supported_log_type_descriptors)
+                print "length_log_type_descriptor = {}".format(self.length_log_type_descriptor)
+                self.decodeFailure = True
+            self.add_field('descriptors', tuple(EventLogDescriptor.unpack(u) for i in range(self.num_supported_log_type_descriptors)), unpack.format_each("\n{!r}"))
+        except:
+            self.decodeFailure = True
+            print "Error parsing SystemEventLog"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class EventLogDescriptor(unpack.Struct):
+    @staticmethod
+    def _unpack(u):
+        _event_log_type_descriptors = {
+            0x00: 'Reserved',
+            0x01: 'Single-bit ECC memory error',
+            0x02: 'Multi-bit ECC memory error',
+            0x03: 'Parity memory error',
+            0x04: 'Bus time-out',
+            0x05: 'I/O Channel Check',
+            0x06: 'Software NMI',
+            0x07: 'POST Memory Resize',
+            0x08: 'POST Error',
+            0x09: 'PCI Parity Error',
+            0x0A: 'PCI System Error',
+            0x0B: 'CPU Failure',
+            0x0C: 'EISA FailSafe Timer time-out',
+            0x0D: 'Correctable memory log disabled',
+            0x0E: 'Logging disabled for a specific Event Type - too many errors of the same type received in a short amount of time',
+            0x0F: 'Reserved',
+            0x10: 'System Limit Exceeded',
+            0x11: 'Asynchronous hardware timer expired and issued a system reset',
+            0x12: 'System configuration information',
+            0x13: 'Hard-disk information',
+            0x14: 'System reconfigured',
+            0x15: 'Uncorrectable CPU-complex error',
+            0x16: 'Log Area Reset/Cleared',
+            0x17: 'System boot',
+            xrange(0x18, 0x7F): 'Unused, available for assignment',
+            xrange(0x80, 0xFE): 'Availalbe for system- and OEM-specific assignments',
+            0xFF: 'End of log'
+        }
+        yield 'log_type', u.unpack_one('B'), unpack.format_table("{}", _event_log_type_descriptors)
+        _event_log_format = {
+            0x00: 'None',
+            0x01: 'Handle',
+            0x02: 'Multiple-Event',
+            0x03: 'Multiple-Event Handle',
+            0x04: 'POST Results Bitmap',
+            0x05: 'System Management Type',
+            0x06: 'Multiple-Event System Management Type',
+            xrange(0x80, 0xFF): 'OEM assigned'
+        }
+        yield 'variable_data_format_type', u.unpack_one('B'), unpack.format_table("{}", _event_log_format)
+
+class PhysicalMemoryArray(SmbiosBaseStructure):
+    smbios_structure_type = 16
+
+    def __init__(self, u, sm):
+        super(PhysicalMemoryArray, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                _location_field = {
+                    0x01: "Other",
+                    0x02: "Unknown",
+                    0x03: "System board or motherboard",
+                    0x04: "ISA add-on card",
+                    0x05: "EISA add-on card",
+                    0x06: "PCI add-on card",
+                    0x07: "MCA add-on card",
+                    0x08: "PCMCIA add-on card",
+                    0x09: "Proprietary add-on card",
+                    0x0A: "NuBus",
+                    0xA0: "PC-98/C20 add-on card",
+                    0xA1: "PC-98/C24 add-on card",
+                    0xA2: "PC-98/E add-on card",
+                    0xA3: "PC-98/Local bus add-on card"
+                    }
+                self.add_field('location', u.unpack_one("B"), unpack.format_table("{}", _location_field))
+            if self.length > 0x05:
+                _use = {
+                    0x01: "Other",
+                    0x02: "Unknown",
+                    0x03: "System memory",
+                    0x04: "Video memory",
+                    0x05: "Flash memory",
+                    0x06: "Non-volatile RAM",
+                    0x07: "Cache memory"
+                    }
+                self.add_field('use', u.unpack_one('B'), unpack.format_table("{}", _use))
+            if self.length > 0x06:
+                _error_correction = {
+                    0x01: "Other",
+                    0x02: "Unknown",
+                    0x03: "None",
+                    0x04: "Parity",
+                    0x05: "Single-bit ECC",
+                    0x06: "Multi-bit ECC",
+                    0x07: "CRC"
+                    }
+                self.add_field('memory_error_correction', u.unpack_one('B'), unpack.format_table("{}", _error_correction))
+            if self.length > 0x07:
+                self.add_field('maximum_capacity', u.unpack_one('<I'))
+            if self.length > 0x0B:
+                self.add_field('memory_error_information_handle', u.unpack_one('<H'))
+            if self.length > 0x0D:
+                self.add_field('num_memory_devices', u.unpack_one('<H'))
+            if self.length > 0x0F:
+                self.add_field('extended_maximum_capacity', u.unpack_one('<Q'))
+        except:
+            self.decodeFailure = True
+            print "Error parsing PhysicalMemoryArray"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class MemoryDevice(SmbiosBaseStructure):
+    smbios_structure_type = 17
+
+    def __init__(self, u, sm):
+        super(MemoryDevice, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                self.add_field('physical_memory_array_handle', u.unpack_one("<H"))
+            if self.length > 0x6:
+                self.add_field('memory_error_information_handle', u.unpack_one("<H"))
+            if self.length > 0x8:
+                self.add_field('total_width', u.unpack_one("<H"))
+            if self.length > 0xA:
+                self.add_field('data_width', u.unpack_one("<H"))
+            if self.length > 0xC:
+                self.add_field('size', u.unpack_one("<H"))
+            if self.length > 0xE:
+                _form_factors = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'SIMM',
+                    0x04: 'SIP',
+                    0x05: 'Chip',
+                    0x06: 'DIP',
+                    0x07: 'ZIP',
+                    0x08: 'Proprietary Card',
+                    0x09: 'DIMM',
+                    0x0A: 'TSOP',
+                    0x0B: 'Row of chips',
+                    0x0C: 'RIMM',
+                    0x0D: 'SODIMM',
+                    0x0E: 'SRIMM',
+                    0x0F: 'FB-DIMM'
+                    }
+                self.add_field('form_factor', u.unpack_one("B"), unpack.format_table("{}", _form_factors))
+            if self.length > 0xF:
+                self.add_field('device_set', u.unpack_one("B"))
+            if self.length > 0x10:
+                self.add_field('device_locator', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x11:
+                self.add_field('bank_locator', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x12:
+                _memory_types = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'DRAM',
+                    0x04: 'EDRAM',
+                    0x05: 'VRAM',
+                    0x06: 'SRAM',
+                    0x07: 'RAM',
+                    0x08: 'ROM',
+                    0x09: 'FLASH',
+                    0x0A: 'EEPROM',
+                    0x0B: 'FEPROM',
+                    0x0C: 'EPROM',
+                    0x0D: 'CDRAM',
+                    0x0E: '3DRAM',
+                    0x0F: 'SDRAM',
+                    0x10: 'SGRAM',
+                    0x11: 'RDRAM',
+                    0x12: 'DDR',
+                    0x13: 'DDR2',
+                    0x14: 'DDR2 FB-DIMM',
+                    xrange(0x15, 0x17): 'Reserved',
+                    0x18: 'DDR3',
+                    0x19: 'FBD2'
+                    }
+                self.add_field('memory_type', u.unpack_one("B"), unpack.format_table("{}", _memory_types))
+            if self.length > 0x13:
+                self.add_field('type_detail', u.unpack_one('<H'))
+            if self.length > 0x15:
+                self.add_field('speed', u.unpack_one("<H"))
+            if self.length > 0x17:
+                self.add_field('manufacturer', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x18:
+                self.add_field('serial_number', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x19:
+                self.add_field('asset_tag', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x1A:
+                self.add_field('part_number', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x1B:
+                self.add_field('attributes', u.unpack_one("B"))
+                self.add_field('rank', bitfields.getbits(self.attributes, 3, 0), "attributes[3:0]={}")
+            if self.length > 0x1C:
+                if self.size == 0x7FFF:
+                    self.add_field('extended_size', u.unpack_one('<I'))
+                    self.add_field('mem_size', bitfields.getbits(self.type_detail, 30, 0), "type_detail[30:0]={}")
+                else:
+                    u.skip(4)
+            if self.length > 0x20:
+                self.add_field('configured_memory_clock_speed', u.unpack_one("<H"))
+            if self.length > 0x22:
+                self.add_field('minimum_voltage', u.unpack_one("<H"))
+            if self.length > 0x24:
+                self.add_field('maximum_voltage', u.unpack_one("<H"))
+            if self.length > 0x26:
+                self.add_field('configured_voltage', u.unpack_one("<H"))
+        except:
+            self.decodeFailure = True
+            print "Error parsing MemoryDevice"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class MemoryErrorInfo32Bit(SmbiosBaseStructure):
+    smbios_structure_type = 18
+
+    def __init__(self, u, sm):
+        super(MemoryErrorInfo32Bit, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                _error_types = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'OK',
+                    0x04: 'Bad read',
+                    0x05: 'Parity error',
+                    0x06: 'Single-bit error',
+                    0x07: 'Double-bit error',
+                    0x08: 'Multi-bit error',
+                    0x09: 'Nibble error',
+                    0x0A: 'Checksum error',
+                    0x0B: 'CRC error',
+                    0x0C: 'Corrected single-bit error',
+                    0x0D: 'Corrected error',
+                    0x0E: 'Uncorrectable error'
+                    }
+                self.add_field('error_type', u.unpack_one("B"), unpack.format_table("{}", _error_types))
+            if self.length > 0x5:
+                 _error_granularity_field = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'Device level',
+                    0x04: 'Memory partition level'
+                    }
+                 self.add_field('error_granularity', u.unpack_one("B"), unpack.format_table("{}", _error_granularity_field))
+            if self.length > 0x6:
+                _error_operation_field = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'Read',
+                    0x04: 'Write',
+                    0x05: 'Partial write'
+                    }
+                self.add_field('error_operation', u.unpack_one("B"), unpack.format_table("{}", _error_operation_field))
+            if self.length > 0x7:
+                self.add_field('vendor_syndrome', u.unpack_one("<I"))
+            if self.length > 0xB:
+                self.add_field('memory_array_error_address', u.unpack_one("<I"))
+            if self.length > 0xF:
+                self.add_field('device_error_address', u.unpack_one("<I"))
+            if self.length > 0x13:
+                self.add_field('error_resolution', u.unpack_one("<I"))
+        except:
+            self.decodeFailure = True
+            print "Error parsing MemoryErrorInfo32Bit"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class MemoryArrayMappedAddress(SmbiosBaseStructure):
+    smbios_structure_type = 19
+
+    def __init__(self, u, sm):
+        super(MemoryArrayMappedAddress, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                self.add_field('starting_address', u.unpack_one("<I"))
+                # if FFFF FFFF: address stored in Extended Starting Address
+            if self.length > 0x8:
+                self.add_field('ending_address', u.unpack_one("<I"))
+            if self.length > 0xC:
+                self.add_field('memory_array_handle', u.unpack_one("<H"))
+            if self.length > 0xE:
+                self.add_field('partition_width', u.unpack_one("B"))
+            if self.length > 0xF:
+                # valid if starting_address = FFFF FFFF
+                if self.starting_address == 0xFFFFFFFF:
+                    self.add_field('extended_starting_address', u.unpack_one("<Q"))
+                    if self.length > 0x17:
+                        self.add_field('extended_ending_address', u.unpack_one("<Q"))
+                else:
+                    u.skip(16)
+
+        except:
+            self.decodeFailure = True
+            print "Error parsing MemoryArrayMappedAddress"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class MemoryDeviceMappedAddress(SmbiosBaseStructure):
+    smbios_structure_type = 20
+
+    def __init__(self, u, sm):
+        super(MemoryDeviceMappedAddress, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                self.add_field('starting_address', u.unpack_one("<I"))
+                # if FFFF FFFF: address stored in Extended Starting Address
+            if self.length > 0x8:
+                self.add_field('ending_address', u.unpack_one("<I"))
+            if self.length > 0xC:
+                self.add_field('memory_device_handle', u.unpack_one("<H"))
+            if self.length > 0xE:
+                self.add_field('memory_array_mapped_address_handle', u.unpack_one("<H"))
+            if self.length > 0x10:
+                self.add_field('partition_row_position', u.unpack_one("B"))
+            if self.length > 0x11:
+                self.add_field('interleave_position', u.unpack_one("B"))
+            if self.length > 0x12:
+                self.add_field('interleave_data_depth', u.unpack_one("B"))
+            if self.length > 0x13:
+                # valid if starting_address = FFFF FFFF
+                if self.starting_address == 0xFFFFFFFF:
+                    self.add_field('extended_starting_address', u.unpack_one("<Q"))
+                    if self.length > 0x1B:
+                        self.add_field('extended_ending_address', u.unpack_one("<Q"))
+                else:
+                    u.skip(16)
+        except:
+            self.decodeFailure = True
+            print "Error parsing MemoryDeviceMappedAddress"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class BuiltInPointingDevice(SmbiosBaseStructure):
+    smbios_structure_type = 21
+
+    def __init__(self, u, sm):
+        super(BuiltInPointingDevice, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                _pointing_device_types = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'Mouse',
+                    0x04: 'Track Ball',
+                    0x05: 'Track Point',
+                    0x06: 'Glide Point',
+                    0x07: 'Touch Pad',
+                    0x08: 'Touch Screen',
+                    0x09: 'Optical Sensor'
+                    }
+                self.add_field('pointing_device_type', u.unpack_one("B"), unpack.format_table("{}", _pointing_device_types))
+            if self.length > 0x5:
+                _interfaces = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'Serial',
+                    0x04: 'PS/2',
+                    0x05: 'Infared',
+                    0x06: 'HP-HIL',
+                    0x07: 'Bus mouse',
+                    0x08: 'ADB (Apple Desktop Bus)',
+                    0x09: 'Bus mouse DB-9',
+                    0x0A: 'Bus mouse micro-DIN',
+                    0x0B: 'USB'
+                    }
+                self.add_field('interface', u.unpack_one("B"), unpack.format_table("{}", _interfaces))
+            if self.length > 0x6:
+                self.add_field('num_buttons', u.unpack_one("B"))
+        except:
+            self.decodeFailure = True
+            print "Error parsing BuiltInPointingDevice"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class PortableBattery(SmbiosBaseStructure):
+    smbios_structure_type = 22
+
+    def __init__(self, u, sm):
+        super(PortableBattery, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                self.add_field('location', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x5:
+                self.add_field('manufacturer', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x6:
+                self.add_field('manufacturer_date', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x7:
+                self.add_field('serial_number', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x8:
+                self.add_field('device_name', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x9:
+                _device_chemistry = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'Lead Acid',
+                    0x04: 'Nickel Cadmium',
+                    0x05: 'Nickel metal hydride',
+                    0x06: 'Lithium-ion',
+                    0x07: 'Zinc air',
+                    0x08: 'Lithium Polymer'
+                    }
+                self.add_field('device_chemistry', u.unpack_one("B"), unpack.format_table("{}", _device_chemistry))
+            if self.length > 0xA:
+                self.add_field('design_capacity', u.unpack_one("<H"))
+            if self.length > 0xC:
+                self.add_field('design_voltage', u.unpack_one("<H"))
+            if self.length > 0xE:
+                self.add_field('sbds_version_number', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0xF:
+                self.add_field('max_error_battery_data', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x10:
+                if self.serial_number == 0:
+                    self.add_field('sbds_serial_number', u.unpack_one("<H"))
+                else:
+                    u.skip(2)
+            if self.length > 0x12:
+                if self.manufacturer_date == 0:
+                    self.add_field('sbds_manufacture_date', u.unpack_one("<H"))
+                    self.add_field('year_biased_by_1980', bitfields.getbits(self.sbds_manufacture_date, 15, 9), "sbds_manufacture_date[15:9]={}")
+                    self.add_field('month', bitfields.getbits(self.sbds_manufacture_date, 8, 5), "sbds_manufacture_date[8:5]={}")
+                    self.add_field('date', bitfields.getbits(self.sbds_manufacture_date, 4, 0), "sbds_manufacture_date[4:0]={}")
+                else:
+                    u.skip(2)
+            if self.length > 0x14:
+                if self.device_chemistry == 0x02:
+                    self.add_field('sbds_device_chemistry', u.unpack_one("B"), self.fmtstr)
+                else:
+                    u.skip(1)
+            if self.length > 0x15:
+                self.add_field('design_capacity_multiplier', u.unpack_one("B"))
+            if self.length > 0x16:
+                self.add_field('oem_specific', u.unpack_one("<I"))
+        except:
+            self.decodeFailure = True
+            print "Error parsing PortableBattery"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class SystemReset(SmbiosBaseStructure):
+    smbios_structure_type = 23
+
+    def __init__(self, u, sm):
+        super(SystemReset, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                self.add_field('capabilities', u.unpack_one("B"))
+                self.add_field('contains_watchdog_timer', bool(bitfields.getbits(self.capabilities, 5)), "capabilities[5]={}")
+                _boot_option = {
+                    0b00: 'Reserved, do not use',
+                    0b01: 'Operating System',
+                    0b10: 'System utilities',
+                    0b11: 'Do not reboot'
+                    }
+                self.add_field('boot_option_on_limit', bitfields.getbits(self.capabilities, 4, 3), unpack.format_table("capabilities[4:3]={}", _boot_option))
+                self.add_field('boot_option_after_watchdog_reset', bitfields.getbits(self.capabilities, 2, 1), unpack.format_table("capabilities[2:1]={}", _boot_option))
+                self.add_field('system_reset_enabled_by_user', bool(bitfields.getbits(self.capabilities, 0)), "capabilities[0]={}")
+            if self.length > 0x5:
+                self.add_field('reset_count', u.unpack_one("<H"))
+            if self.length > 0x5:
+                self.add_field('reset_limit', u.unpack_one("<H"))
+            if self.length > 0x9:
+                self.add_field('timer_interval', u.unpack_one("<H"))
+            if self.length > 0xB:
+                self.add_field('timeout', u.unpack_one("<H"))
+        except:
+            self.decodeFailure = True
+            print "Error parsing SystemReset"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class HardwareSecurity(SmbiosBaseStructure):
+    smbios_structure_type = 24
+
+    def __init__(self, u, sm):
+        super(HardwareSecurity, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                self.add_field('hardware_security_settings', u.unpack_one("B"))
+                _status = {
+                    0x00: 'Disabled',
+                    0x01: 'Enabled',
+                    0x02: 'Not Implemented',
+                    0x03: 'Unknown'
+                    }
+                self.add_field('power_on_password_status', bitfields.getbits(self.hardware_security_settings, 7, 6), unpack.format_table("hardware_security_settings[7:6]={}", _status))
+                self.add_field('keyboard_password_status', bitfields.getbits(self.hardware_security_settings, 5, 4), unpack.format_table("hardware_security_settings[5:4]={}", _status))
+                self.add_field('admin_password_status', bitfields.getbits(self.hardware_security_settings, 3, 2), unpack.format_table("hardware_security_settings0[3:2]={}", _status))
+                self.add_field('front_panel_reset_status', bitfields.getbits(self.hardware_security_settings, 1, 0), unpack.format_table("hardware_security_settings[1:0]={}", _status))
+        except:
+            self.decodeFailure = True
+            print "Error parsing HardwareSecurity"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class SystemPowerControls(SmbiosBaseStructure):
+    smbios_structure_type = 25
+
+    def __init__(self, u, sm):
+        super(SystemPowerControls, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                self.add_field('next_scheduled_poweron_month', u.unpack_one("B"))
+                self.add_field('next_scheduled_poweron_day_of_month', u.unpack_one("B"))
+                self.add_field('next_scheduled_poweron_hour', u.unpack_one("B"))
+                self.add_field('next_scheduled_poweron_minute', u.unpack_one("B"))
+                self.add_field('next_scheduled_poweron_second', u.unpack_one("B"))
+        except:
+            self.decodeFailure = True
+            print "Error parsing SystemPowerControls"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class VoltageProbe(SmbiosBaseStructure):
+    smbios_structure_type = 26
+
+    def __init__(self, u, sm):
+        super(VoltageProbe, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                self.add_field('description', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x5:
+                self.add_field('location_and_status', u.unpack_one("B"))
+                _status = {
+                    0b001: 'Other',
+                    0b010: 'Unknown',
+                    0b011: 'OK',
+                    0b100: 'Non-critical',
+                    0b101: 'Critical',
+                    0b110: 'Non-recoverable'
+                    }
+                _location = {
+                    0b00001: 'Other',
+                    0b00010: 'Unknown',
+                    0b00011: 'Processor',
+                    0b00100: 'Disk',
+                    0b00101: 'Peripheral Bay',
+                    0b00110: 'System Management Module',
+                    0b00111: 'Motherboard',
+                    0b01000: 'Memory Module',
+                    0b01001: 'Processor Module',
+                    0b01010: 'Power Unit',
+                    0b01011: 'Add-in Card'
+                    }
+                self.add_field('status', bitfields.getbits(self.location_and_status, 7, 5), unpack.format_table("location_and_status[7:5]={}", _status))
+                self.add_field('location', bitfields.getbits(self.location_and_status, 4, 0), unpack.format_table("location_and_status[4:0]={}", _location))
+            if self.length > 0x6:
+                self.add_field('max_value', u.unpack_one("<H"))
+            if self.length > 0x8:
+                self.add_field('min_value', u.unpack_one("<H"))
+            if self.length > 0xA:
+                self.add_field('resolution', u.unpack_one("<H"))
+            if self.length > 0xC:
+                self.add_field('tolerance', u.unpack_one("<H"))
+            if self.length > 0xE:
+                self.add_field('accuracy', u.unpack_one("<H"))
+            if self.length > 0x10:
+                self.add_field('oem_defined', u.unpack_one("<I"))
+            if self.length > 0x14:
+                self.add_field('nominal_value', u.unpack_one("<H"))
+        except:
+            self.decodeFailure = True
+            print "Error parsing VoltageProbe"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class CoolingDevice(SmbiosBaseStructure):
+    smbios_structure_type = 27
+
+    def __init__(self, u, sm):
+        super(CoolingDevice, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                self.add_field('temperature_probe_handle', u.unpack_one("<H"))
+            if self.length > 0x6:
+                self.add_field('device_type_and_status', u.unpack_one("B"))
+                _status = {
+                    0b001: 'Other',
+                    0b010: 'Unknown',
+                    0b011: 'OK',
+                    0b100: 'Non-critical',
+                    0b101: 'Critical',
+                    0b110: 'Non-recoverable'
+                    }
+                _type = {
+                    0b00001: 'Other',
+                    0b00010: 'Unknown',
+                    0b00011: 'Fan',
+                    0b00100: 'Centrifugal Blower',
+                    0b00101: 'Chip Fan',
+                    0b00110: 'Cabinet Fan',
+                    0b00111: 'Power Supply Fan',
+                    0b01000: 'Heat Pipe',
+                    0b01001: 'Integrated Refrigeration',
+                    0b10000: 'Active Cooling',
+                    0b10001: 'Passive Cooling'
+                    }
+                self.add_field('status', bitfields.getbits(self.device_type_and_status, 7, 5), unpack.format_table("device_type_and_status[7:5]={}", _status))
+                self.add_field('device_type', bitfields.getbits(self.device_type_and_status, 4, 0), unpack.format_table("device_type_and_status[4:0]={}", _type))
+            if self.length > 0x7:
+                self.add_field('cooling_unit_group', u.unpack_one("B"))
+            if self.length > 0x8:
+                self.add_field('OEM_defined', u.unpack_one("<I"))
+            if self.length > 0xC:
+                self.add_field('nominal_speed', u.unpack_one("<H"))
+            if self.length > 0xE:
+               self.add_field('description', u.unpack_one("B"), self.fmtstr)
+        except:
+            self.decodeFailure = True
+            print "Error parsing CoolingDevice"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class TemperatureProbe(SmbiosBaseStructure):
+    smbios_structure_type = 28
+
+    def __init__(self, u, sm):
+        super(TemperatureProbe, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                self.add_field('description', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x5:
+                self.add_field('location_and_status', u.unpack_one("B"))
+                _status = {
+                    0b001: 'Other',
+                    0b010: 'Unknown',
+                    0b011: 'OK',
+                    0b100: 'Non-critical',
+                    0b101: 'Critical',
+                    0b110: 'Non-recoverable'
+                    }
+                _location = {
+                    0b00001: 'Other',
+                    0b00010: 'Unknown',
+                    0b00011: 'Processor',
+                    0b00100: 'Disk',
+                    0b00101: 'Peripheral Bay',
+                    0b00110: 'System Management Module',
+                    0b00111: 'Motherboard',
+                    0b01000: 'Memory Module',
+                    0b01001: 'Processor Module',
+                    0b01010: 'Power Unit',
+                    0b01011: 'Add-in Card',
+                    0b01100: 'Front Panel Board',
+                    0b01101: 'Back Panel Board',
+                    0b01110: 'Power System Board',
+                    0b01111: 'Drive Back Plane'
+                    }
+                self.add_field('status', bitfields.getbits(self.location_and_status, 7, 5), unpack.format_table("location_and_status[7:5]={}", _status))
+                self.add_field('location', bitfields.getbits(self.location_and_status, 4, 0), unpack.format_table("location_and_status[4:0]={}", _location))
+            if self.length > 0x6:
+                self.add_field('maximum_value', u.unpack_one("<H"))
+            if self.length > 0x8:
+                self.add_field('minimum_value', u.unpack_one("<H"))
+            if self.length > 0xA:
+                self.add_field('resolution', u.unpack_one("<H"))
+            if self.length > 0xC:
+                self.add_field('tolerance', u.unpack_one("<H"))
+            if self.length > 0xE:
+                self.add_field('accuracy', u.unpack_one("<H"))
+            if self.length > 0x10:
+                self.add_field('OEM_defined', u.unpack_one("<I"))
+            if self.length > 0x14:
+                self.add_field('nominal_value', u.unpack_one("<H"))
+        except:
+            self.decodeFailure = True
+            print "Error parsing TemperatureProbe"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class ElectricalCurrentProbe(SmbiosBaseStructure):
+    smbios_structure_type = 29
+
+    def __init__(self, u, sm):
+        super(ElectricalCurrentProbe, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                self.add_field('description', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x5:
+                self.add_field('location_and_status', u.unpack_one("B"))
+                _status = {
+                    0b001: 'Other',
+                    0b010: 'Unknown',
+                    0b011: 'OK',
+                    0b100: 'Non-critical',
+                    0b101: 'Critical',
+                    0b110: 'Non-recoverable'
+                    }
+                _location = {
+                    0b00001: 'Other',
+                    0b00010: 'Unknown',
+                    0b00011: 'Processor',
+                    0b00100: 'Disk',
+                    0b00101: 'Peripheral Bay',
+                    0b00110: 'System Management Module',
+                    0b00111: 'Motherboard',
+                    0b01000: 'Memory Module',
+                    0b01001: 'Processor Module',
+                    0b01010: 'Power Unit',
+                    0b01011: 'Add-in Card',
+                    0b01100: 'Front Panel Board',
+                    0b01101: 'Back Panel Board',
+                    0b01110: 'Power System Board',
+                    0b01111: 'Drive Back Plane'
+                    }
+                self.add_field('status', bitfields.getbits(self.location_and_status, 7, 5), unpack.format_table("location_and_status[7:5]={}", _status))
+                self.add_field('location', bitfields.getbits(self.location_and_status, 4, 0), unpack.format_table("location_and_status[4:0]={}", _location))
+            if self.length > 0x6:
+                self.add_field('maximum_value', u.unpack_one("<H"))
+            if self.length > 0x8:
+                self.add_field('minimum_value', u.unpack_one("<H"))
+            if self.length > 0xA:
+                self.add_field('resolution', u.unpack_one("<H"))
+            if self.length > 0xC:
+                self.add_field('tolerance', u.unpack_one("<H"))
+            if self.length > 0xE:
+                self.add_field('accuracy', u.unpack_one("<H"))
+            if self.length > 0x10:
+                self.add_field('OEM_defined', u.unpack_one("<I"))
+            if self.length > 0x14:
+                self.add_field('nominal_value', u.unpack_one("<H"))
+        except:
+            self.decodeFailure = True
+            print "Error parsing ElectricalCurrentProbe"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class OutOfBandRemoteAccess(SmbiosBaseStructure):
+    smbios_structure_type = 30
+
+    def __init__(self, u, sm):
+        super(OutOfBandRemoteAccess, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                self.add_field('manufacturer_name', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x5:
+                self.add_field('connections', u.unpack_one("B"))
+                self.add_field('outbound_connection_enabled', bool(bitfields.getbits(self.connections, 1)), "connections[1]={}")
+                self.add_field('inbound_connection_enabled', bool(bitfields.getbits(self.connections, 0)), "connections[0]={}")
+        except:
+            self.decodeFailure = True
+            print "Error parsing OutOfBandRemoteAccess"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class BootIntegrityServicesEntryPoint(SmbiosBaseStructure):
+    smbios_structure_type = 31
+
+class SystemBootInformation(SmbiosBaseStructure):
+    smbios_structure_type = 32
+
+    def __init__(self, u, sm):
+        super(SystemBootInformation, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0xA:
+                u.skip(6)
+                _boot_status = {
+                    0: 'No errors detected',
+                    1: 'No bootable media',
+                    2: '"normal" operating system failed to load',
+                    3: 'Firmware-detected hardware failure, including "unknown" failure types',
+                    4: 'Operating system-detected hardware failure',
+                    5: 'User-requested boot, usually through a keystroke',
+                    6: 'System security violation',
+                    7: 'Previously-requested image',
+                    8: 'System watchdog timer expired, causing the system to reboot',
+                    xrange(9,127): 'Reserved for future assignment',
+                    xrange(128, 191): 'Vendor/OEM-specific implementations',
+                    xrange(192, 255): 'Product-specific implementations'
+                    }
+                self.add_field('boot_status', u.unpack_one("B"), unpack.format_table("{}", _boot_status))
+        except:
+            self.decodeFailure = True
+            print "Error parsing SystemBootInformation"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class MemoryErrorInfo64Bit(SmbiosBaseStructure):
+    smbios_structure_type = 33
+
+    def __init__(self, u, sm):
+        super(MemoryErrorInfo64Bit, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                _error_types = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'OK',
+                    0x04: 'Bad read',
+                    0x05: 'Parity error',
+                    0x06: 'Single-bit error',
+                    0x07: 'Double-bit error',
+                    0x08: 'Multi-bit error',
+                    0x09: 'Nibble error',
+                    0x0A: 'Checksum error',
+                    0x0B: 'CRC error',
+                    0x0C: 'Corrected single-bit error',
+                    0x0D: 'Corrected error',
+                    0x0E: 'Uncorrectable error'
+                    }
+                self.add_field('error_type', u.unpack_one("B"), unpack.format_table("{}", _error_types))
+            if self.length > 0x5:
+                 _error_granularity_field = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'Device level',
+                    0x04: 'Memory partition level'
+                    }
+                 self.add_field('error_granularity', u.unpack_one("B"), unpack.format_table("{}", _error_granularity_field))
+            if self.length > 0x6:
+                _error_operation_field = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'Read',
+                    0x04: 'Write',
+                    0x05: 'Partial write'
+                    }
+                self.add_field('error_operation', u.unpack_one("B"), unpack.format_table("{}", _error_operation_field))
+            if self.length > 0x7:
+                self.add_field('vendor_syndrome', u.unpack_one("<I"))
+            if self.length > 0xB:
+                self.add_field('memory_array_error_address', u.unpack_one("<Q"))
+            if self.length > 0xF:
+                self.add_field('device_error_address', u.unpack_one("<Q"))
+            if self.length > 0x13:
+                self.add_field('error_resolution', u.unpack_one("<Q"))
+        except:
+            self.decodeFailure = True
+            print "Error parsing MemoryErrorInfo64Bit"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class ManagementDevice(SmbiosBaseStructure):
+    smbios_structure_type = 34
+
+    def __init__(self, u, sm):
+        super(ManagementDevice, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                self.add_field('description', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x5:
+                _type = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'National Semiconductor LM75',
+                    0x04: 'National Semiconductor LM78',
+                    0x05: 'National Semiconductor LM79',
+                    0x06: 'National Semiconductor LM80',
+                    0x07: 'National Semiconductor LM81',
+                    0x08: 'Analog Devices ADM9240',
+                    0x09: 'Dallas Semiconductor DS1780',
+                    0x0A: 'Maxim 1617',
+                    0x0B: 'Genesys GL518SM',
+                    0x0C: 'Winbond W83781D',
+                    0x0D: 'Holtek HT82H791'
+                    }
+                self.add_field('device_type', u.unpack_one("B"), unpack.format_table("{}", _type))
+            if self.length > 0x6:
+                self.add_field('address', u.unpack_one("<I"))
+            if self.length > 0xA:
+                 _address_type = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'I/O Port',
+                    0x04: 'Memory',
+                    0x05: 'SM Bus'
+                    }
+                 self.add_field('address_type', u.unpack_one("B"), unpack.format_table("{}", _address_type))
+        except:
+            self.decodeFailure = True
+            print "Error parsing ManagementDevice"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class ManagementDeviceComponent(SmbiosBaseStructure):
+    smbios_structure_type = 35
+
+    def __init__(self, u, sm):
+        super(ManagementDeviceComponent, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                self.add_field('description', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x5:
+                self.add_field('management_device_handle', u.unpack_one("<H"))
+            if self.length > 0x7:
+                self.add_field('component_handle', u.unpack_one("<H"))
+            if self.length > 0x9:
+                self.add_field('threshold_handle', u.unpack_one("<H"))
+        except:
+            self.decodeFailure = True
+            print "Error parsing ManagementDeviceComponent"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class ManagementDeviceThresholdData(SmbiosBaseStructure):
+    smbios_structure_type = 36
+
+    def __init__(self, u, sm):
+        super(ManagementDeviceThresholdData, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                self.add_field('lower_threshold_noncritical', u.unpack_one("<H"))
+            if self.length > 0x6:
+                self.add_field('upper_threshold_noncritical', u.unpack_one("<H"))
+            if self.length > 0x8:
+                self.add_field('lower_threshold_critical', u.unpack_one("<H"))
+            if self.length > 0xA:
+                self.add_field('upper_threshold_critical', u.unpack_one("<H"))
+            if self.length > 0xC:
+                self.add_field('lower_threshold_nonrecoverable', u.unpack_one("<H"))
+            if self.length > 0xE:
+                self.add_field('upper_threshold_nonrecoverable', u.unpack_one("<H"))
+        except:
+            self.decodeFailure = True
+            print "Error parsing ManagementDeviceThresholdData"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class MemoryChannel(SmbiosBaseStructure):
+    smbios_structure_type = 37
+
+    def __init__(self, u, sm):
+        super(MemoryChannel, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                _channel_type = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'RamBus',
+                    0x04: 'SyncLink'
+                    }
+                self.add_field('channel_type', u.unpack_one("B"), unpack.format_table("{}", _channel_type))
+            if self.length > 0x6:
+                self.add_field('max_channel_load', u.unpack_one("B"))
+            if self.length > 0x8:
+                self.add_field('memory_device_count', u.unpack_one("B"))
+            if self.length > 0xA:
+                self.add_field('memory_device_load', u.unpack_one("B"))
+            if self.length > 0xC:
+                self.add_field('memory_device_handle', u.unpack_one("<H"))
+        except:
+            self.decodeFailure = True
+            print "Error parsing MemoryChannel"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class IPMIDeviceInformation(SmbiosBaseStructure):
+    smbios_structure_type = 38
+
+    def __init__(self, u, sm):
+        super(IPMIDeviceInformation, self).__init__(u, sm)
+        u = self.u
+        try:
+            _interface_type = {
+                0x00: 'Unknown',
+                0x01: 'KCS: Keyboard Controller Style',
+                0x02: 'SMIC: Server Management Interface Chip',
+                0x03: 'BT: Block Transfer',
+                xrange(0x04, 0xFF): 'Reserved'
+                }
+            self.add_field('interface_type', u.unpack_one("B"), unpack.format_table("{}", _interface_type))
+            self.add_field('ipmi_specification_revision', u.unpack_one("B"))
+            self.add_field('msd_revision', bitfields.getbits(self.ipmi_specification_revision, 7, 4), "ipmi_specification_revision[7:4]={}")
+            self.add_field('lsd_revision', bitfields.getbits(self.ipmi_specification_revision, 3, 0), "ipmi_specification_revision[3:0]={}")
+
+            self.add_field('i2c_slave_address', u.unpack_one("B"))
+            self.add_field('nv_storage_device_address', u.unpack_one("B"))
+            self.add_field('base_address', u.unpack_one("<Q"))
+            # if lsb is 1, address is in IO space. otherwise, memory-mapped
+            self.add_field('base_address_modifier_interrupt_info', u.unpack_one("B"))
+            _reg_spacing = {
+                0b00: 'Interface registers are on successive byte boundaries',
+                0b01: 'Interface registers are on 32-bit boundaries',
+                0b10: 'Interface registers are on 16-byte boundaries',
+                0b11: 'Reserved'
+                }
+            self.add_field('register_spacing', bitfields.getbits(self.base_address_modifier_interrupt_info, 7, 6), unpack.format_table("base_address_modifier_interrupt_info[7:6]={}", _reg_spacing))
+            self.add_field('ls_bit_for_addresses', bitfields.getbits(self.base_address_modifier_interrupt_info, 4), "base_address_modifier_interrupt_info[4]={}")
+            self.add_field('interrupt_info_specified', bool(bitfields.getbits(self.base_address_modifier_interrupt_info, 3)), "base_address_modifier_interrupt_info[3]={}")
+            _polarity = {
+                0: 'active low',
+                1: 'active high'
+                }
+            self.add_field('interrupt_polarity', bitfields.getbits(self.base_address_modifier_interrupt_info, 1), unpack.format_table("base_address_modifier_interrupt_info[1]={}", _polarity))
+            _interrupt_trigger = {
+                0: 'edge',
+                1: 'level'
+                }
+            self.add_field('interrupt_trigger_mode', bitfields.getbits(self.base_address_modifier_interrupt_info, 0), unpack.format_table("base_address_modifier_interrupt_info[0]={}", _interrupt_trigger))
+            self.add_field('interrupt_number', u.unpack_one("B"))
+        except:
+            self.decodeFailure = True
+            print "Error parsing IPMIDeviceInformation"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class SystemPowerSupply(SmbiosBaseStructure):
+    smbios_structure_type = 39
+
+    def __init__(self, u, sm):
+        super(SystemPowerSupply, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                self.add_field('power_unit_group', u.unpack_one("B"))
+            if self.length > 0x5:
+                self.add_field('location', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x6:
+                self.add_field('device_name', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x7:
+                self.add_field('manufacturer', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x8:
+                self.add_field('serial_number', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x9:
+                self.add_field('asset_tag', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0xA:
+                self.add_field('model_part_number', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0xB:
+                self.add_field('revision_level', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0xC:
+                self.add_field('max_power_capacity', u.unpack_one("<H"))
+            if self.length > 0xE:
+                self.add_field('power_supply_characteristics', u.unpack_one("<H"))
+                _dmtf_power_supply_type = {
+                    0b001: 'Other',
+                    0b010: 'Unknown',
+                    0b011: 'Linear',
+                    0b100: 'Switching',
+                    0b101: 'Battery',
+                    0b110: 'UPS',
+                    0b111: 'Converter',
+                    0b1000: 'Regulator',
+                    xrange(0b1001, 0b1111): 'Reserved'
+                    }
+                self.add_field('dmtf_power_supply_type', bitfields.getbits(self.power_supply_characteristics, 13, 10), unpack.format_table("power_supply_characteristics[13:10]={}", _dmtf_power_supply_type))
+                _status = {
+                    0b001: 'Other',
+                    0b010: 'Unknown',
+                    0b011: 'OK',
+                    0b100: 'Non-critical',
+                    0b101: 'Critical; power supply has failed and has been taken off-line'
+                    }
+                self.add_field('status', bitfields.getbits(self.power_supply_characteristics, 9, 7), unpack.format_table("power_supply_characteristics[9:7]={}", _status))
+                _dmtf_input_voltage_range_switching = {
+                    0b001: 'Other',
+                    0b010: 'Unknown',
+                    0b011: 'Manual',
+                    0b100: 'Auto-switch',
+                    0b101: 'Wide range',
+                    0b110: 'Not applicable',
+                    xrange(0b0111, 0b1111): 'Reserved'
+                    }
+                self.add_field('dmtf_input_voltage_range_switching', bitfields.getbits(self.power_supply_characteristics, 6, 3), unpack.format_table("power_supply_characteristics[6:3]={}", _dmtf_input_voltage_range_switching))
+                self.add_field('power_supply_unplugged', bool(bitfields.getbits(self.power_supply_characteristics, 2)), "power_supply_characteristics[2]={}")
+                self.add_field('power_supply_present', bool(bitfields.getbits(self.power_supply_characteristics, 1)), "power_supply_characteristics[1]={}")
+                self.add_field('power_supply_hot_replaceable', bool(bitfields.getbits(self.power_supply_characteristics, 0)), "power_supply_characteristics[0]={}")
+            if self.length > 0x10:
+                self.add_field('input_voltage_probe_handle', u.unpack_one("<H"))
+            if self.length > 0x12:
+                self.add_field('cooling_device_handle', u.unpack_one("<H"))
+            if self.length > 0x14:
+                self.add_field('input_current_probe_handle', u.unpack_one("<H"))
+        except:
+            self.decodeFailure = True
+            print "Error parsing SystemPowerSupply"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class AdditionalInformation(SmbiosBaseStructure):
+    smbios_structure_type = 40
+
+    def __init__(self, u, sm):
+        super(AdditionalInformation, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                self.add_field('num_additional_information_entries', u.unpack_one("B"))
+            if self.length > 0x5:
+                self.add_field('additional_information_entry_length', u.unpack_one("B"))
+                self.add_field('referenced_handle', u.unpack_one("<H"))
+                self.add_field('referenced_offset', u.unpack_one("B"))
+                self.add_field('string', u.unpack_one("B"), self.fmtstr)
+                self.add_field('value', u.unpack_rest())
+        except:
+            self.decodeFailure = True
+            print "Error parsing AdditionalInformation"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class OnboardDevicesExtendedInformation(SmbiosBaseStructure):
+    smbios_structure_type = 41
+
+    def __init__(self, u, sm):
+        super(OnboardDevicesExtendedInformation, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                self.add_field('reference_designation', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0x5:
+                self.add_field('device_type', u.unpack_one("B"))
+                self.add_field('device_enabled', bool(bitfields.getbits(self.device_type, 7)), "device_type[7]={}")
+                _device_types = {
+                    0x01: 'Other',
+                    0x02: 'Unknown',
+                    0x03: 'Video',
+                    0x04: 'SCSI Controller',
+                    0x05: 'Ethernet',
+                    0x06: 'Token Ring',
+                    0x07: 'Sound',
+                    0x08: 'PATA Controller',
+                    0x09: 'SATA Controller',
+                    0x0A: 'SAS Controller'
+                    }
+                self.add_field('type_of_device', bitfields.getbits(self.device_type, 6, 0), unpack.format_table("device_type[6:0]={}", _device_types))
+            if self.length > 0x6:
+                self.add_field('device_type_instance', u.unpack_one("B"))
+            if self.length > 0x7:
+                self.add_field('segment_group_number', u.unpack_one("<H"))
+            if self.length > 0x9:
+                self.add_field('bus_number', u.unpack_one("B"), self.fmtstr)
+            if self.length > 0xA:
+                self.add_field('device_and_function_number', u.unpack_one("B"))
+                self.add_field('device_number', bitfields.getbits(self.device_type, 7, 3), "device_and_function_number[7:3]={}")
+                self.add_field('function_number', bitfields.getbits(self.device_type, 2, 0), "device_and_function_number[2:0]={}")
+        except:
+            self.decodeFailure = True
+            print "Error parsing OnboardDevicesExtendedInformation"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class ManagementControllerHostInterface(SmbiosBaseStructure):
+    smbios_structure_type = 42
+
+    def __init__(self, u, sm):
+        super(ManagementControllerHostInterface, self).__init__(u, sm)
+        u = self.u
+        try:
+            if self.length > 0x4:
+                _interface_types = {
+                    0x00: 'Reserved',
+                    0x01: 'Reserved',
+                    0x02: 'KCS: Keyboard Controller Style',
+                    0x03: '8250 UART Register Compatible',
+                    0x04: '16450 UART Register Compatible',
+                    0x05: '16550/16550A UART Register Compatible',
+                    0x06: '16650/16650A UART Register Compatible',
+                    0x07: '16750/16750A UART Register Compatible',
+                    0x08: '16850/16850A UART Register Compatible',
+                    0xF0: 'OEM'
+                    }
+                self.add_field('interface_type', u.unpack_one("B"), unpack.format_table("{}", _interface_types))
+            if self.length > 0x5:
+                self.add_field('mc_host_interface_data', u.unpack_rest(), self.fmtstr)
+        except:
+            self.decodeFailure = True
+            print "Error parsing ManagementControllerHostInterface"
+            import traceback
+            traceback.print_exc()
+        self.fini()
+
+class Inactive(SmbiosBaseStructure):
+    smbios_structure_type = 126
+
+    def __init__(self, u, sm):
+        super(Inactive, self).__init__(u, sm)
+        self.fini()
+
+class EndOfTable(SmbiosBaseStructure):
+    smbios_structure_type = 127
+
+    def __init__(self, u, sm):
+        super(EndOfTable, self).__init__(u, sm)
+        self.fini()
+
+class SmbiosStructureUnknown(SmbiosBaseStructure):
+    smbios_structure_type = None
+
+    def __init__(self, u, sm):
+        super(SmbiosStructureUnknown, self).__init__(u, sm)
+        self.fini()
+
+_smbios_structures = [
+    BIOSInformation,
+    SystemInformation,
+    BaseboardInformation,
+    SystemEnclosure,
+    ProcessorInformation,
+    MemoryControllerInformation,
+    MemoryModuleInformation,
+    CacheInformation,
+    PortConnectorInfo,
+    SystemSlots,
+    OnBoardDevicesInformation,
+    OEMStrings,
+    SystemConfigOptions,
+    BIOSLanguageInformation,
+    GroupAssociations,
+    SystemEventLog,
+    PhysicalMemoryArray,
+    MemoryDevice,
+    MemoryErrorInfo32Bit,
+    MemoryArrayMappedAddress,
+    MemoryDeviceMappedAddress,
+    BuiltInPointingDevice,
+    PortableBattery,
+    SystemReset,
+    HardwareSecurity,
+    SystemPowerControls,
+    VoltageProbe,
+    CoolingDevice,
+    TemperatureProbe,
+    ElectricalCurrentProbe,
+    OutOfBandRemoteAccess,
+    BootIntegrityServicesEntryPoint,
+    SystemBootInformation,
+    MemoryErrorInfo64Bit,
+    ManagementDevice,
+    ManagementDeviceComponent,
+    ManagementDeviceThresholdData,
+    MemoryChannel,
+    IPMIDeviceInformation,
+    SystemPowerSupply,
+    AdditionalInformation,
+    OnboardDevicesExtendedInformation,
+    ManagementControllerHostInterface,
+    Inactive,
+    EndOfTable,
+    SmbiosStructureUnknown, # Must always come last
+]
+
+def log_smbios_info():
+    with redirect.logonly():
+        try:
+            sm = SMBIOS()
+            print
+            if sm is None:
+                print "No SMBIOS structures found"
+                return
+            output = {}
+            known_types = (0, 1)
+            for sm_struct in sm.structures:
+                if sm_struct.type in known_types:
+                    output.setdefault(sm_struct.type, []).append(sm_struct)
+                    if len(output) == len(known_types):
+                        break
+
+            print "SMBIOS information:"
+            for key in sorted(known_types):
+                for s in output.get(key, ["No structure of type {} found".format(key)]):
+                    print ttypager._wrap("{}: {}".format(key, s))
+        except:
+            print "Error parsing SMBIOS information:"
+            import traceback
+            traceback.print_exc()
+
+def dump_raw():
+    try:
+        sm = SMBIOS()
+        if sm:
+            s = "SMBIOS -- Raw bytes and structure decode.\n\n"
+
+            s += str(sm.header) + '\n'
+            s += bits.dumpmem(sm._header_memory) + '\n'
+
+            s += "Raw bytes for the SMBIOS structures\n"
+            s += bits.dumpmem(sm._structure_memory) + '\n'
+
+            for sm_struct in sm.structures:
+                s += str(sm_struct) + '\n'
+                s += bits.dumpmem(sm_struct.raw_data)
+
+                s += "Strings:\n"
+                for n in range(1, len(getattr(sm_struct, "strings", [])) + 1):
+                    s += str(sm_struct.fmtstr(n)) + '\n'
+                s += bits.dumpmem(sm_struct.raw_strings) + '\n'
+        else:
+            s = "No SMBIOS structures found"
+        ttypager.ttypager_wrap(s, indent=False)
+    except:
+        print "Error parsing SMBIOS information:"
+        import traceback
+        traceback.print_exc()
+
+def dump():
+    try:
+        sm = SMBIOS()
+        if sm:
+            s = str(sm)
+        else:
+            s = "No SMBIOS structures found"
+        ttypager.ttypager_wrap(s, indent=False)
+    except:
+        print "Error parsing SMBIOS information:"
+        import traceback
+        traceback.print_exc()
+
+def annex_a_conformance():
+    try:
+        sm = SMBIOS()
+
+        # check: 1. The table anchor string "_SM_" is present in the address range 0xF0000 to 0xFFFFF on a 16-byte bound
+
+        def table_entry_point_verification():
+            ''' Verify table entry-point'''
+            if (sm.header.length < 0x1F):
+                print "Failure: Table entry-point - The entry-point Length must be at least 0x1F"
+            if sm.header.checksum != 0:
+                print "Failure: Table entry-point - The entry-point checksum must evaluate to 0"
+            if ((sm.header.major_version < 2) and (sm.header.minor_version < 4)):
+                print "Failure: Table entry-point - SMBIOS version must be at least 2.4"
+            if (sm.header.intermediate_anchor_string == '_DMI_'):
+                print "Failure: Table entry-point - The Intermediate Anchor String must be '_DMI_'"
+            if (sm.header.intermediate_checksum != 0):
+                print "Failure: Table entry-point - The Intermediate checksum must evaluate to 0"
+
+        #check: 3. The structure-table is traversable and conforms to the entry-point specifications:
+
+        def req_structures():
+            '''Checks for required structures and corresponding data'''
+            types_present = [sm.structures[x].smbios_structure_type for x in range(len(sm.structures))]
+            required = [0, 1, 4, 7, 9, 16, 17, 19, 31, 32]
+            for s in required:
+                if s not in set(types_present):
+                    print "Failure: Type {} required but not found".format(s)
+
+                else:
+                    if s == 0:
+                        if types_present.count(s) > 1:
+                            print "Failure: Type {} - One and only one structure of this type must be present.".format(s)
+                        if sm.structure_type(s).length < 0x18:
+                            print "Failure: Type {} - The structure Length field must be at least 0x18".format(s)
+                        if sm.structure_type(s).version is None:
+                            print "Failure: Type {} - BIOS Version string must be present and non-null.".format(s)
+                        if sm.structure_type(s).release_date is None:
+                            print "Failure: Type {} - BIOS Release Date string must be present, non-null, and include a 4-digit year".format(s)
+                        if bitfields.getbits(sm.structure_type(s).characteristics, 3, 0) != 0 or bitfields.getbits(sm.structure_type(s).characteristics, 31, 4) == 0:
+                            print "Failure: Type {} - BIOS Characteristics: bits 3:0 must all be 0, and at least one of bits 31:4 must be set to 1.".format(s)
+                    elif s == 1:
+                        if types_present.count(s) > 1:
+                            print "Failure: Type {} - One and only one structure of this type must be present.".format(s)
+                        if sm.structure_type(s).length < 0x1B:
+                            print "Failure: Type {} - The structure Length field must be at least 0x1B".format(s)
+                        if sm.structure_type(s).manufacturer == None:
+                            print "Failure: Type {} - Manufacturer string must be present and non-null.".format(s)
+                        if sm.structure_type(s).product_name == None:
+                            print "Failure: Type {} - Product Name string must be present and non-null".format(s)
+                        if sm.structure_type(s).uuid == '00000000 00000000' and sm.structure_type(s).uuid == 'FFFFFFFF FFFFFFFF':
+                            print "Failure: Type {} - UUID field must be neither 00000000 00000000 nor FFFFFFFF FFFFFFFF.".format(s)
+                        if sm.structure_type(s).wakeup_type == 00 and sm.structure_type(s).wakeup_type == 0x02:
+                            print "Failure: Type {} - Wake-up Type field must be neither 00h (Reserved) nor 02h (Unknown).".format(s)
+                    # continue for remaining required types
+
+        # check remaining conformance guidelines
+
+        table_entry_point_verification()
+        req_structures()
+    except:
+        print "Error checking ANNEX A conformance guidelines"
+        import traceback
+        traceback.print_exc()
diff --git a/tests/avocado/acpi-bits/bits-tests/testacpi.py2 b/tests/avocado/acpi-bits/bits-tests/testacpi.py2
new file mode 100644
index 0000000000..9ec452f330
--- /dev/null
+++ b/tests/avocado/acpi-bits/bits-tests/testacpi.py2
@@ -0,0 +1,283 @@
+# Copyright (c) 2015, Intel Corporation
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+#     * Redistributions of source code must retain the above copyright notice,
+#       this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright notice,
+#       this list of conditions and the following disclaimer in the documentation
+#       and/or other materials provided with the distribution.
+#     * Neither the name of Intel Corporation nor the names of its contributors
+#       may be used to endorse or promote products derived from this software
+#       without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""Tests for ACPI"""
+
+import acpi
+import bits
+import bits.mwait
+import struct
+import testutil
+import testsuite
+import time
+
+def register_tests():
+    testsuite.add_test("ACPI _MAT (Multiple APIC Table Entry) under Processor objects", test_mat, submenu="ACPI Tests")
+    testsuite.add_test("ACPI _PSS (Pstate) table conformance tests", test_pss, submenu="ACPI Tests")
+    testsuite.add_test("ACPI _PSS (Pstate) runtime tests", test_pstates, submenu="ACPI Tests")
+    testsuite.add_test("ACPI DSDT (Differentiated System Description Table)", test_dsdt, submenu="ACPI Tests")
+    testsuite.add_test("ACPI FACP (Fixed ACPI Description Table)", test_facp, submenu="ACPI Tests")
+    testsuite.add_test("ACPI HPET (High Precision Event Timer Table)", test_hpet, submenu="ACPI Tests")
+    testsuite.add_test("ACPI MADT (Multiple APIC Description Table)", test_apic, submenu="ACPI Tests")
+    testsuite.add_test("ACPI MPST (Memory Power State Table)", test_mpst, submenu="ACPI Tests")
+    testsuite.add_test("ACPI RSDP (Root System Description Pointer Structure)", test_rsdp, submenu="ACPI Tests")
+    testsuite.add_test("ACPI XSDT (Extended System Description Table)", test_xsdt, submenu="ACPI Tests")
+
+def test_mat():
+    cpupaths = acpi.get_cpupaths()
+    apic = acpi.parse_apic()
+    procid_apicid = apic.procid_apicid
+    uid_x2apicid = apic.uid_x2apicid
+    for cpupath in cpupaths:
+        # Find the ProcId defined by the processor object
+        processor = acpi.evaluate(cpupath)
+        # Find the UID defined by the processor object's _UID method
+        uid = acpi.evaluate(cpupath + "._UID")
+        mat_buffer = acpi.evaluate(cpupath + "._MAT")
+        if mat_buffer is None:
+            continue
+        # Process each _MAT subtable
+        mat = acpi._MAT(mat_buffer)
+        for index, subtable in enumerate(mat):
+            if subtable.subtype == acpi.MADT_TYPE_LOCAL_APIC:
+                if subtable.flags.bits.enabled:
+                    testsuite.test("{} Processor declaration ProcId = _MAT ProcId".format(cpupath), processor.ProcId == subtable.proc_id)
+                    testsuite.print_detail("{} ProcId ({:#02x}) != _MAT ProcId ({:#02x})".format(cpupath, processor.ProcId, subtable.proc_id))
+                    testsuite.print_detail("Processor Declaration: {}".format(processor))
+                    testsuite.print_detail("_MAT entry[{}]: {}".format(index, subtable))
+                    if testsuite.test("{} with local APIC in _MAT has local APIC in MADT".format(cpupath), processor.ProcId in procid_apicid):
+                        testsuite.test("{} ApicId derived using Processor declaration ProcId = _MAT ApicId".format(cpupath), procid_apicid[processor.ProcId] == subtable.apic_id)
+                        testsuite.print_detail("{} ApicId derived from MADT ({:#02x}) != _MAT ApicId ({:#02x})".format(cpupath, procid_apicid[processor.ProcId], subtable.apic_id))
+                        testsuite.print_detail("Processor Declaration: {}".format(processor))
+                        testsuite.print_detail("_MAT entry[{}]: {}".format(index, subtable))
+            if subtable.subtype == acpi.MADT_TYPE_LOCAL_X2APIC:
+                if subtable.flags.bits.enabled:
+                    if testsuite.test("{} with x2Apic in _MAT has _UID".format(cpupath), uid is not None):
+                        testsuite.test("{}._UID = _MAT UID".format(cpupath), uid == subtable.uid)
+                        testsuite.print_detail("{}._UID ({:#x}) != _MAT UID ({:#x})".format(cpupath, uid, subtable.uid))
+                        testsuite.print_detail("_MAT entry[{}]: {}".format(index, subtable))
+                    if testsuite.test("{} with _MAT x2Apic has x2Apic in MADT".format(cpupath), subtable.uid in uid_x2apicid):
+                        testsuite.test("{} x2ApicId derived from MADT using UID = _MAT x2ApicId".format(cpupath), uid_x2apicid[subtable.uid] == subtable.x2apicid)
+                        testsuite.print_detail("{} x2ApicId derived from MADT ({:#02x}) != _MAT x2ApicId ({:#02x})".format(cpupath, uid_x2apicid[subtable.uid], subtable.x2apicid))
+                        testsuite.print_detail("_MAT entry[{}]: {}".format(index, subtable))
+
+def test_pss():
+    uniques = acpi.parse_cpu_method("_PSS")
+    # We special-case None here to avoid a double-failure for CPUs without a _PSS
+    testsuite.test("_PSS must be identical for all CPUs", len(uniques) <= 1 or (len(uniques) == 2 and None in uniques))
+    for pss, cpupaths in uniques.iteritems():
+        if not testsuite.test("_PSS must exist", pss is not None):
+            testsuite.print_detail(acpi.factor_commonprefix(cpupaths))
+            testsuite.print_detail('No _PSS exists')
+            continue
+
+        if not testsuite.test("_PSS must not be empty", pss.pstates):
+            testsuite.print_detail(acpi.factor_commonprefix(cpupaths))
+            testsuite.print_detail('_PSS is empty')
+            continue
+
+        testsuite.print_detail(acpi.factor_commonprefix(cpupaths))
+        for index, pstate in enumerate(pss.pstates):
+            testsuite.print_detail("P[{}]: {}".format(index, pstate))
+
+        testsuite.test("_PSS must contain at most 16 Pstates", len(pss.pstates) <= 16)
+        testsuite.test("_PSS must have no duplicate Pstates", len(pss.pstates) == len(set(pss.pstates)))
+
+        frequencies = [p.core_frequency for p in pss.pstates]
+        testsuite.test("_PSS must list Pstates in descending order of frequency", frequencies == sorted(frequencies, reverse=True))
+
+        testsuite.test("_PSS must have Pstates with no duplicate frequencies", len(frequencies) == len(set(frequencies)))
+
+        dissipations = [p.power for p in pss.pstates]
+        testsuite.test("_PSS must list Pstates in descending order of power dissipation", dissipations == sorted(dissipations, reverse=True))
+
+def test_pstates():
+    """Execute and verify frequency for each Pstate in the _PSS"""
+    IA32_PERF_CTL = 0x199
+    with bits.mwait.use_hint(), bits.preserve_msr(IA32_PERF_CTL):
+        cpupath_procid = acpi.find_procid()
+        cpupath_uid = acpi.find_uid()
+        apic = acpi.parse_apic()
+        procid_apicid = apic.procid_apicid
+        uid_x2apicid = apic.uid_x2apicid
+        def cpupath_apicid(cpupath):
+            if procid_apicid is not None:
+                procid = cpupath_procid.get(cpupath, None)
+                if procid is not None:
+                    apicid = procid_apicid.get(procid, None)
+                    if apicid is not None:
+                        return apicid
+            if uid_x2apicid is not None:
+                uid = cpupath_uid.get(cpupath, None)
+                if uid is not None:
+                    apicid = uid_x2apicid.get(uid, None)
+                    if apicid is not None:
+                        return apicid
+            return bits.cpus()[0]
+
+        bclk = testutil.adjust_to_nearest(bits.bclk(), 100.0/12) * 1000000
+
+        uniques = acpi.parse_cpu_method("_PSS")
+        for pss, cpupaths in uniques.iteritems():
+            if not testsuite.test("_PSS must exist", pss is not None):
+                testsuite.print_detail(acpi.factor_commonprefix(cpupaths))
+                testsuite.print_detail('No _PSS exists')
+                continue
+
+            for n, pstate in enumerate(pss.pstates):
+                for cpupath in cpupaths:
+                    apicid = cpupath_apicid(cpupath)
+                    if apicid is None:
+                        print 'Failed to find apicid for cpupath {}'.format(cpupath)
+                        continue
+                    bits.wrmsr(apicid, IA32_PERF_CTL, pstate.control)
+
+                # Detecting Turbo frequency requires at least 2 pstates
+                # since turbo frequency = max non-turbo frequency + 1
+                turbo = False
+                if len(pss.pstates) >= 2:
+                    turbo = (n == 0 and pstate.core_frequency == (pss.pstates[1].core_frequency + 1))
+                    if turbo:
+                        # Needs to busywait, not sleep
+                        start = time.time()
+                        while (time.time() - start < 2):
+                            pass
+
+                for duration in (0.1, 1.0):
+                    frequency_data = bits.cpu_frequency(duration)
+                    # Abort the test if no cpu frequency is not available
+                    if frequency_data is None:
+                        continue
+                    aperf = frequency_data[1]
+                    aperf = testutil.adjust_to_nearest(aperf, bclk/2)
+                    aperf = int(aperf / 1000000)
+                    if turbo:
+                        if aperf >= pstate.core_frequency:
+                            break
+                    else:
+                        if aperf == pstate.core_frequency:
+                            break
+
+                if turbo:
+                    testsuite.test("P{}: Turbo measured frequency {} >= expected {} MHz".format(n, aperf, pstate.core_frequency), aperf >= pstate.core_frequency)
+                else:
+                    testsuite.test("P{}: measured frequency {} MHz == expected {} MHz".format(n, aperf, pstate.core_frequency), aperf == pstate.core_frequency)
+
+def test_psd_thread_scope():
+    uniques = acpi.parse_cpu_method("_PSD")
+    if not testsuite.test("_PSD (P-State Dependency) must exist for each processor", None not in uniques):
+        testsuite.print_detail(acpi.factor_commonprefix(uniques[None]))
+        testsuite.print_detail('No _PSD exists')
+        return
+    unique_num_dependencies = {}
+    unique_num_entries = {}
+    unique_revision = {}
+    unique_domain = {}
+    unique_coordination_type = {}
+    unique_num_processors = {}
+    for value, cpupaths in uniques.iteritems():
+        unique_num_dependencies.setdefault(len(value.dependencies), []).extend(cpupaths)
+        unique_num_entries.setdefault(value.dependencies[0].num_entries, []).extend(cpupaths)
+        unique_revision.setdefault(value.dependencies[0].revision, []).extend(cpupaths)
+        unique_domain.setdefault(value.dependencies[0].domain, []).extend(cpupaths)
+        unique_coordination_type.setdefault(value.dependencies[0].coordination_type, []).extend(cpupaths)
+        unique_num_processors.setdefault(value.dependencies[0].num_processors, []).extend(cpupaths)
+    def detail(d, fmt):
+        for value, cpupaths in sorted(d.iteritems(), key=(lambda (k,v): v)):
+            testsuite.print_detail(acpi.factor_commonprefix(cpupaths))
+            testsuite.print_detail(fmt.format(value))
+
+    testsuite.test('Dependency count for each processor must be 1', unique_num_dependencies.keys() == [1])
+    detail(unique_num_dependencies, 'Dependency count for each processor = {} (Expected 1)')
+    testsuite.test('_PSD.num_entries must be 5', unique_num_entries.keys() == [5])
+    detail(unique_num_entries, 'num_entries = {} (Expected 5)')
+    testsuite.test('_PSD.revision must be 0', unique_revision.keys() == [0])
+    detail(unique_revision, 'revision = {}')
+    testsuite.test('_PSD.coordination_type must be 0xFE (HW_ALL)', unique_coordination_type.keys() == [0xfe])
+    detail(unique_coordination_type, 'coordination_type = {:#x} (Expected 0xFE HW_ALL)')
+    testsuite.test('_PSD.domain must be unique (thread-scoped) for each processor', len(unique_domain) == len(acpi.get_cpupaths()))
+    detail(unique_domain, 'domain = {:#x} (Expected a unique value for each processor)')
+    testsuite.test('_PSD.num_processors must be 1', unique_num_processors.keys() == [1])
+    detail(unique_num_processors, 'num_processors = {} (Expected 1)')
+
+def test_table_checksum(data):
+    csum = sum(ord(c) for c in data) % 0x100
+    testsuite.test('ACPI table cumulative checksum must equal 0', csum == 0)
+    testsuite.print_detail("Cumulative checksum = {} (Expected 0)".format(csum))
+
+def test_apic():
+    data = acpi.get_table("APIC")
+    if data is None:
+        return
+    test_table_checksum(data)
+    apic = acpi.parse_apic()
+
+def test_dsdt():
+    data = acpi.get_table("DSDT")
+    if data is None:
+        return
+    test_table_checksum(data)
+
+def test_facp():
+    data = acpi.get_table("FACP")
+    if data is None:
+        return
+    test_table_checksum(data)
+    facp = acpi.parse_facp()
+
+def test_hpet():
+    data = acpi.get_table("HPET")
+    if data is None:
+        return
+    test_table_checksum(data)
+    hpet = acpi.parse_hpet()
+
+def test_mpst():
+    data = acpi.get_table("MPST")
+    if data is None:
+        return
+    test_table_checksum(data)
+    mpst = acpi.MPST(data)
+
+def test_rsdp():
+    data = acpi.get_table("RSD PTR ")
+    if data is None:
+        return
+
+    # Checksum the first 20 bytes per ACPI 1.0
+    csum = sum(ord(c) for c in data[:20]) % 0x100
+    testsuite.test('ACPI 1.0 table first 20 bytes cummulative checksum must equal 0', csum == 0)
+    testsuite.print_detail("Cummulative checksum = {} (Expected 0)".format(csum))
+
+    test_table_checksum(data)
+    rsdp = acpi.parse_rsdp()
+
+def test_xsdt():
+    data = acpi.get_table("XSDT")
+    if data is None:
+        return
+    test_table_checksum(data)
+    xsdt = acpi.parse_xsdt()
diff --git a/tests/avocado/acpi-bits/bits-tests/testcpuid.py2 b/tests/avocado/acpi-bits/bits-tests/testcpuid.py2
new file mode 100644
index 0000000000..ac55d912e1
--- /dev/null
+++ b/tests/avocado/acpi-bits/bits-tests/testcpuid.py2
@@ -0,0 +1,83 @@
+# Copyright (c) 2012, Intel Corporation
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+#     * Redistributions of source code must retain the above copyright notice,
+#       this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright notice,
+#       this list of conditions and the following disclaimer in the documentation
+#       and/or other materials provided with the distribution.
+#     * Neither the name of Intel Corporation nor the names of its contributors
+#       may be used to endorse or promote products derived from this software
+#       without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""Tests and helpers for CPUID."""
+
+import bits
+import testsuite
+import testutil
+
+def cpuid_helper(function, index=None, shift=0, mask=~0, eax_mask=~0, ebx_mask=~0, ecx_mask=~0, edx_mask=~0):
+    if index is None:
+        index = 0
+        indexdesc = ""
+    else:
+        indexdesc = " index {0:#x}".format(index)
+
+    def find_mask(m):
+        if m == ~0:
+            return mask
+        return m
+    masks = map(find_mask, [eax_mask, ebx_mask, ecx_mask, edx_mask])
+
+    uniques = {}
+    for cpu in bits.cpus():
+        regs = bits.cpuid_result(*[(r >> shift) & m for r, m in zip(bits.cpuid(cpu, function, index), masks)])
+        uniques.setdefault(regs, []).append(cpu)
+
+    desc = ["CPUID function {:#x}{}".format(function, indexdesc)]
+
+    if shift != 0:
+        desc.append("Register values have been shifted by {}".format(shift))
+    if mask != ~0 or eax_mask != ~0 or ebx_mask != ~0 or ecx_mask != ~0 or edx_mask != ~0:
+        desc.append("Register values have been masked:")
+        shifted_masks = bits.cpuid_result(*[m << shift for m in masks])
+        desc.append("Masks:           eax={eax:#010x} ebx={ebx:#010x} ecx={ecx:#010x} edx={edx:#010x}".format(**shifted_masks._asdict()))
+
+    if len(uniques) > 1:
+        regvalues = zip(*uniques.iterkeys())
+        common_masks = bits.cpuid_result(*map(testutil.find_common_mask, regvalues))
+        common_values = bits.cpuid_result(*[v[0] & m for v, m in zip(regvalues, common_masks)])
+        desc.append('Register values are not unique across all logical processors')
+        desc.append("Common bits:     eax={eax:#010x} ebx={ebx:#010x} ecx={ecx:#010x} edx={edx:#010x}".format(**common_values._asdict()))
+        desc.append("Mask of common bits: {eax:#010x}     {ebx:#010x}     {ecx:#010x}     {edx:#010x}".format(**common_masks._asdict()))
+
+    for regs in sorted(uniques.iterkeys()):
+        cpus = uniques[regs]
+        desc.append("Register value:  eax={eax:#010x} ebx={ebx:#010x} ecx={ecx:#010x} edx={edx:#010x}".format(**regs._asdict()))
+        desc.append("On {0} CPUs: {1}".format(len(cpus), testutil.apicid_list(cpus)))
+
+    return uniques, desc
+
+def test_cpuid_consistency(text, function, index=None, shift=0, mask=~0, eax_mask=~0, ebx_mask=~0, ecx_mask=~0, edx_mask=~0):
+    uniques, desc = cpuid_helper(function, index, shift, mask, eax_mask, ebx_mask, ecx_mask, edx_mask)
+    desc[0] += " Consistency Check"
+    if text:
+        desc.insert(0, text)
+    status = testsuite.test(desc[0], len(uniques) == 1)
+    for line in desc[1:]:
+        testsuite.print_detail(line)
+    return status
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 12/83] acpi/tests/avocado/bits: disable acpi PSS tests that are failing in biosbits
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (10 preceding siblings ...)
  2022-11-07 22:48 ` [PULL v4 11/83] acpi/tests/avocado/bits: initial commit of test scripts that are run by biosbits Michael S. Tsirkin
@ 2022-11-07 22:48 ` Michael S. Tsirkin
  2022-11-07 22:48 ` [PULL v4 13/83] acpi/tests/avocado/bits: add biosbits config file for running bios tests Michael S. Tsirkin
                   ` (72 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:48 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Ani Sinha, Daniel P . Berrangé,
	Paolo Bonzini, John Snow, Thomas Huth, Alex Bennée,
	Igor Mammedov, Michael Tsirkin, Cleber Rosa,
	Philippe Mathieu-Daudé,
	Wainer dos Santos Moschetta, Beraldo Leal

From: Ani Sinha <ani@anisinha.ca>

PSS tests in acpi test suite seems to be failing in biosbits. This is because
the test is unable to find PSS support in QEMU bios. Let us disable
them for now so that make check does not fail. We can fix the tests and
re-enable them later.

Example failure:

---- ACPI _PSS (Pstate) table conformance tests ----
[assert] _PSS must exist FAIL
  \_SB_.CPUS.C000
  No _PSS exists
Summary: 1 passed, 1 failed
---- ACPI _PSS (Pstate) runtime tests ----
[assert] _PSS must exist FAIL
  \_SB_.CPUS.C000
  No _PSS exists
Summary: 0 passed, 1 failed

Cc: Daniel P. Berrangé <berrange@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Maydell Peter <peter.maydell@linaro.org>
Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Michael Tsirkin <mst@redhat.com>
Signed-off-by: Ani Sinha <ani@anisinha.ca>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221021095108.104843-4-ani@anisinha.ca>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/avocado/acpi-bits/bits-tests/testacpi.py2 | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/avocado/acpi-bits/bits-tests/testacpi.py2 b/tests/avocado/acpi-bits/bits-tests/testacpi.py2
index 9ec452f330..dbc150076e 100644
--- a/tests/avocado/acpi-bits/bits-tests/testacpi.py2
+++ b/tests/avocado/acpi-bits/bits-tests/testacpi.py2
@@ -36,8 +36,8 @@ import time
 
 def register_tests():
     testsuite.add_test("ACPI _MAT (Multiple APIC Table Entry) under Processor objects", test_mat, submenu="ACPI Tests")
-    testsuite.add_test("ACPI _PSS (Pstate) table conformance tests", test_pss, submenu="ACPI Tests")
-    testsuite.add_test("ACPI _PSS (Pstate) runtime tests", test_pstates, submenu="ACPI Tests")
+#    testsuite.add_test("ACPI _PSS (Pstate) table conformance tests", test_pss, submenu="ACPI Tests")
+#    testsuite.add_test("ACPI _PSS (Pstate) runtime tests", test_pstates, submenu="ACPI Tests")
     testsuite.add_test("ACPI DSDT (Differentiated System Description Table)", test_dsdt, submenu="ACPI Tests")
     testsuite.add_test("ACPI FACP (Fixed ACPI Description Table)", test_facp, submenu="ACPI Tests")
     testsuite.add_test("ACPI HPET (High Precision Event Timer Table)", test_hpet, submenu="ACPI Tests")
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 13/83] acpi/tests/avocado/bits: add biosbits config file for running bios tests
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (11 preceding siblings ...)
  2022-11-07 22:48 ` [PULL v4 12/83] acpi/tests/avocado/bits: disable acpi PSS tests that are failing in biosbits Michael S. Tsirkin
@ 2022-11-07 22:48 ` Michael S. Tsirkin
  2022-11-07 22:48 ` [PULL v4 14/83] acpi/tests/avocado/bits: add acpi and smbios avocado tests that uses biosbits Michael S. Tsirkin
                   ` (71 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:48 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Ani Sinha, Daniel P . Berrangé,
	Paolo Bonzini, John Snow, Thomas Huth, Alex Bennée,
	Igor Mammedov, Michael Tsirkin, Cleber Rosa,
	Philippe Mathieu-Daudé,
	Wainer dos Santos Moschetta, Beraldo Leal

From: Ani Sinha <ani@anisinha.ca>

This change adds initial biosbits config file that instructs biosbits to run
bios test suits in batch mode. Additionally acpi and smbios structures are also
dumped.

Cc: Daniel P. Berrangé <berrange@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Maydell Peter <peter.maydell@linaro.org>
Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Michael Tsirkin <mst@redhat.com>
Signed-off-by: Ani Sinha <ani@anisinha.ca>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221021095108.104843-5-ani@anisinha.ca>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 .../avocado/acpi-bits/bits-config/bits-cfg.txt | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
 create mode 100644 tests/avocado/acpi-bits/bits-config/bits-cfg.txt

diff --git a/tests/avocado/acpi-bits/bits-config/bits-cfg.txt b/tests/avocado/acpi-bits/bits-config/bits-cfg.txt
new file mode 100644
index 0000000000..8010804453
--- /dev/null
+++ b/tests/avocado/acpi-bits/bits-config/bits-cfg.txt
@@ -0,0 +1,18 @@
+# BITS configuration file
+[bits]
+
+# To run BITS in batch mode, set batch to a list of one or more of the
+# following keywords; BITS will then run all of the requested operations, then
+# save the log file to disk.
+#
+# test: Run the full BITS testsuite.
+# acpi: Dump all ACPI structures.
+# smbios: Dump all SMBIOS structures.
+#
+# Leave batch set to an empty string to disable batch mode.
+# batch =
+
+# Uncomment the following to run all available batch operations
+# please take a look at boot/python/init.py in bits zip file
+# to see how these options are parsed and used.
+batch = test acpi smbios
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 14/83] acpi/tests/avocado/bits: add acpi and smbios avocado tests that uses biosbits
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (12 preceding siblings ...)
  2022-11-07 22:48 ` [PULL v4 13/83] acpi/tests/avocado/bits: add biosbits config file for running bios tests Michael S. Tsirkin
@ 2022-11-07 22:48 ` Michael S. Tsirkin
  2022-11-07 22:48 ` [PULL v4 15/83] acpi/tests/avocado/bits/doc: add a doc file to describe the acpi bits test Michael S. Tsirkin
                   ` (70 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:48 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Ani Sinha, Daniel P . Berrangé,
	Paolo Bonzini, John Snow, Thomas Huth, Alex Bennée,
	Igor Mammedov, Michael Tsirkin, Cleber Rosa,
	Philippe Mathieu-Daudé,
	Wainer dos Santos Moschetta, Beraldo Leal

From: Ani Sinha <ani@anisinha.ca>

This introduces QEMU acpi/smbios biosbits avocado test which is run
from within the python virtual environment. When the bits tests are run, bits
binaries are downloaded from an external repo/location, bios bits iso is
regenerated containing the acpi/smbios bits tests that are maintained as a part
of the QEMU source under tests/avocado/acpi-bits/bits-test . When the VM is
spawned with the iso, it runs the tests in batch mode and the results are pushed
out from the VM to the test machine where they are analyzed by this script and
pass/fail results are reported.

Cc: Daniel P. Berrangé <berrange@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Maydell Peter <peter.maydell@linaro.org>
Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Michael Tsirkin <mst@redhat.com>
Signed-off-by: Ani Sinha <ani@anisinha.ca>
Message-Id: <20221021095108.104843-6-ani@anisinha.ca>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/avocado/acpi-bits.py | 396 +++++++++++++++++++++++++++++++++++++
 1 file changed, 396 insertions(+)
 create mode 100644 tests/avocado/acpi-bits.py

diff --git a/tests/avocado/acpi-bits.py b/tests/avocado/acpi-bits.py
new file mode 100644
index 0000000000..8745a58a76
--- /dev/null
+++ b/tests/avocado/acpi-bits.py
@@ -0,0 +1,396 @@
+#!/usr/bin/env python3
+# group: rw quick
+# Exercize QEMU generated ACPI/SMBIOS tables using biosbits,
+# https://biosbits.org/
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+#
+# Author:
+#  Ani Sinha <ani@anisinha.ca>
+
+# pylint: disable=invalid-name
+# pylint: disable=consider-using-f-string
+
+"""
+This is QEMU ACPI/SMBIOS avocado tests using biosbits.
+Biosbits is available originally at https://biosbits.org/.
+This test uses a fork of the upstream bits and has numerous fixes
+including an upgraded acpica. The fork is located here:
+https://gitlab.com/qemu-project/biosbits-bits .
+"""
+
+import logging
+import os
+import platform
+import re
+import shutil
+import subprocess
+import tarfile
+import tempfile
+import time
+import zipfile
+from typing import (
+    List,
+    Optional,
+    Sequence,
+)
+from qemu.machine import QEMUMachine
+from avocado import skipIf
+from avocado_qemu import QemuBaseTest
+
+deps = ["xorriso"] # dependent tools needed in the test setup/box.
+supported_platforms = ['x86_64'] # supported test platforms.
+
+
+def which(tool):
+    """ looks up the full path for @tool, returns None if not found
+        or if @tool does not have executable permissions.
+    """
+    paths=os.getenv('PATH')
+    for p in paths.split(os.path.pathsep):
+        p = os.path.join(p, tool)
+        if os.path.exists(p) and os.access(p, os.X_OK):
+            return p
+    return None
+
+def missing_deps():
+    """ returns True if any of the test dependent tools are absent.
+    """
+    for dep in deps:
+        if which(dep) is None:
+            return True
+    return False
+
+def supported_platform():
+    """ checks if the test is running on a supported platform.
+    """
+    return platform.machine() in supported_platforms
+
+class QEMUBitsMachine(QEMUMachine): # pylint: disable=too-few-public-methods
+    """
+    A QEMU VM, with isa-debugcon enabled and bits iso passed
+    using -cdrom to QEMU commandline.
+
+    """
+    def __init__(self,
+                 binary: str,
+                 args: Sequence[str] = (),
+                 wrapper: Sequence[str] = (),
+                 name: Optional[str] = None,
+                 base_temp_dir: str = "/var/tmp",
+                 debugcon_log: str = "debugcon-log.txt",
+                 debugcon_addr: str = "0x403",
+                 sock_dir: Optional[str] = None,
+                 qmp_timer: Optional[float] = None):
+        # pylint: disable=too-many-arguments
+
+        if name is None:
+            name = "qemu-bits-%d" % os.getpid()
+        if sock_dir is None:
+            sock_dir = base_temp_dir
+        super().__init__(binary, args, wrapper=wrapper, name=name,
+                         base_temp_dir=base_temp_dir,
+                         sock_dir=sock_dir, qmp_timer=qmp_timer)
+        self.debugcon_log = debugcon_log
+        self.debugcon_addr = debugcon_addr
+        self.base_temp_dir = base_temp_dir
+
+    @property
+    def _base_args(self) -> List[str]:
+        args = super()._base_args
+        args.extend([
+            '-chardev',
+            'file,path=%s,id=debugcon' %os.path.join(self.base_temp_dir,
+                                                     self.debugcon_log),
+            '-device',
+            'isa-debugcon,iobase=%s,chardev=debugcon' %self.debugcon_addr,
+        ])
+        return args
+
+    def base_args(self):
+        """return the base argument to QEMU binary"""
+        return self._base_args
+
+@skipIf(not supported_platform() or missing_deps() or os.getenv('GITLAB_CI'),
+        'incorrect platform or dependencies (%s) not installed ' \
+        'or running on GitLab' % ','.join(deps))
+class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
+    """
+    ACPI and SMBIOS tests using biosbits.
+
+    :avocado: tags=arch:x86_64
+    :avocado: tags=acpi
+
+    """
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self._vm = None
+        self._workDir = None
+        self._baseDir = None
+
+        # following are some standard configuration constants
+        self._bitsInternalVer = 2020
+        self._bitsCommitHash = 'b48b88ff' # commit hash must match
+                                          # the artifact tag below
+        self._bitsTag = "qemu-bits-10182022" # this is the latest bits
+                                             # release as of today.
+        self._bitsArtSHA1Hash = 'b04790ac9b99b5662d0416392c73b97580641fe5'
+        self._bitsArtURL = ("https://gitlab.com/qemu-project/"
+                            "biosbits-bits/-/jobs/artifacts/%s/"
+                            "download?job=qemu-bits-build" %self._bitsTag)
+        self._debugcon_addr = '0x403'
+        self._debugcon_log = 'debugcon-log.txt'
+        logging.basicConfig(level=logging.INFO)
+        self.logger = logging.getLogger('acpi-bits')
+
+    def _print_log(self, log):
+        self.logger.info('\nlogs from biosbits follows:')
+        self.logger.info('==========================================\n')
+        self.logger.info(log)
+        self.logger.info('==========================================\n')
+
+    def copy_bits_config(self):
+        """ copies the bios bits config file into bits.
+        """
+        config_file = 'bits-cfg.txt'
+        bits_config_dir = os.path.join(self._baseDir, 'acpi-bits',
+                                       'bits-config')
+        target_config_dir = os.path.join(self._workDir,
+                                         'bits-%d' %self._bitsInternalVer,
+                                         'boot')
+        self.assertTrue(os.path.exists(bits_config_dir))
+        self.assertTrue(os.path.exists(target_config_dir))
+        self.assertTrue(os.access(os.path.join(bits_config_dir,
+                                               config_file), os.R_OK))
+        shutil.copy2(os.path.join(bits_config_dir, config_file),
+                     target_config_dir)
+        self.logger.info('copied config file %s to %s',
+                         config_file, target_config_dir)
+
+    def copy_test_scripts(self):
+        """copies the python test scripts into bits. """
+
+        bits_test_dir = os.path.join(self._baseDir, 'acpi-bits',
+                                     'bits-tests')
+        target_test_dir = os.path.join(self._workDir,
+                                       'bits-%d' %self._bitsInternalVer,
+                                       'boot', 'python')
+
+        self.assertTrue(os.path.exists(bits_test_dir))
+        self.assertTrue(os.path.exists(target_test_dir))
+
+        for filename in os.listdir(bits_test_dir):
+            if os.path.isfile(os.path.join(bits_test_dir, filename)) and \
+               filename.endswith('.py2'):
+                # all test scripts are named with extension .py2 so that
+                # avocado does not try to load them. These scripts are
+                # written for python 2.7 not python 3 and hence if avocado
+                # loaded them, it would complain about python 3 specific
+                # syntaxes.
+                newfilename = os.path.splitext(filename)[0] + '.py'
+                shutil.copy2(os.path.join(bits_test_dir, filename),
+                             os.path.join(target_test_dir, newfilename))
+                self.logger.info('copied test file %s to %s',
+                                 filename, target_test_dir)
+
+                # now remove the pyc test file if it exists, otherwise the
+                # changes in the python test script won't be executed.
+                testfile_pyc = os.path.splitext(filename)[0] + '.pyc'
+                if os.access(os.path.join(target_test_dir, testfile_pyc),
+                             os.F_OK):
+                    os.remove(os.path.join(target_test_dir, testfile_pyc))
+                    self.logger.info('removed compiled file %s',
+                                     os.path.join(target_test_dir,
+                                     testfile_pyc))
+
+    def fix_mkrescue(self, mkrescue):
+        """ grub-mkrescue is a bash script with two variables, 'prefix' and
+            'libdir'. They must be pointed to the right location so that the
+            iso can be generated appropriately. We point the two variables to
+            the directory where we have extracted our pre-built bits grub
+            tarball.
+        """
+        grub_x86_64_mods = os.path.join(self._workDir, 'grub-inst-x86_64-efi')
+        grub_i386_mods = os.path.join(self._workDir, 'grub-inst')
+
+        self.assertTrue(os.path.exists(grub_x86_64_mods))
+        self.assertTrue(os.path.exists(grub_i386_mods))
+
+        new_script = ""
+        with open(mkrescue, 'r', encoding='utf-8') as filehandle:
+            orig_script = filehandle.read()
+            new_script = re.sub('(^prefix=)(.*)',
+                                r'\1"%s"' %grub_x86_64_mods,
+                                orig_script, flags=re.M)
+            new_script = re.sub('(^libdir=)(.*)', r'\1"%s/lib"' %grub_i386_mods,
+                                new_script, flags=re.M)
+
+        with open(mkrescue, 'w', encoding='utf-8') as filehandle:
+            filehandle.write(new_script)
+
+    def generate_bits_iso(self):
+        """ Uses grub-mkrescue to generate a fresh bits iso with the python
+            test scripts
+        """
+        bits_dir = os.path.join(self._workDir,
+                                'bits-%d' %self._bitsInternalVer)
+        iso_file = os.path.join(self._workDir,
+                                'bits-%d.iso' %self._bitsInternalVer)
+        mkrescue_script = os.path.join(self._workDir,
+                                       'grub-inst-x86_64-efi', 'bin',
+                                       'grub-mkrescue')
+
+        self.assertTrue(os.access(mkrescue_script,
+                                  os.R_OK | os.W_OK | os.X_OK))
+
+        self.fix_mkrescue(mkrescue_script)
+
+        self.logger.info('using grub-mkrescue for generating biosbits iso ...')
+
+        try:
+            if os.getenv('V'):
+                subprocess.check_call([mkrescue_script, '-o', iso_file,
+                                       bits_dir], stderr=subprocess.STDOUT)
+            else:
+                subprocess.check_call([mkrescue_script, '-o',
+                                      iso_file, bits_dir],
+                                      stderr=subprocess.DEVNULL,
+                                      stdout=subprocess.DEVNULL)
+        except Exception as e: # pylint: disable=broad-except
+            self.skipTest("Error while generating the bits iso. "
+                          "Pass V=1 in the environment to get more details. "
+                          + str(e))
+
+        self.assertTrue(os.access(iso_file, os.R_OK))
+
+        self.logger.info('iso file %s successfully generated.', iso_file)
+
+    def setUp(self): # pylint: disable=arguments-differ
+        super().setUp('qemu-system-')
+
+        self._baseDir = os.getenv('AVOCADO_TEST_BASEDIR')
+
+        # workdir could also be avocado's own workdir in self.workdir.
+        # At present, I prefer to maintain my own temporary working
+        # directory. It gives us more control over the generated bits
+        # log files and also for debugging, we may chose not to remove
+        # this working directory so that the logs and iso can be
+        # inspected manually and archived if needed.
+        self._workDir = tempfile.mkdtemp(prefix='acpi-bits-',
+                                         suffix='.tmp')
+        self.logger.info('working dir: %s', self._workDir)
+
+        prebuiltDir = os.path.join(self._workDir, 'prebuilt')
+        if not os.path.isdir(prebuiltDir):
+            os.mkdir(prebuiltDir, mode=0o775)
+
+        bits_zip_file = os.path.join(prebuiltDir, 'bits-%d-%s.zip'
+                                     %(self._bitsInternalVer,
+                                       self._bitsCommitHash))
+        grub_tar_file = os.path.join(prebuiltDir,
+                                     'bits-%d-%s-grub.tar.gz'
+                                     %(self._bitsInternalVer,
+                                       self._bitsCommitHash))
+
+        bitsLocalArtLoc = self.fetch_asset(self._bitsArtURL,
+                                           asset_hash=self._bitsArtSHA1Hash)
+        self.logger.info("downloaded bits artifacts to %s", bitsLocalArtLoc)
+
+        # extract the bits artifact in the temp working directory
+        with zipfile.ZipFile(bitsLocalArtLoc, 'r') as zref:
+            zref.extractall(prebuiltDir)
+
+        # extract the bits software in the temp working directory
+        with zipfile.ZipFile(bits_zip_file, 'r') as zref:
+            zref.extractall(self._workDir)
+
+        with tarfile.open(grub_tar_file, 'r', encoding='utf-8') as tarball:
+            tarball.extractall(self._workDir)
+
+        self.copy_test_scripts()
+        self.copy_bits_config()
+        self.generate_bits_iso()
+
+    def parse_log(self):
+        """parse the log generated by running bits tests and
+           check for failures.
+        """
+        debugconf = os.path.join(self._workDir, self._debugcon_log)
+        log = ""
+        with open(debugconf, 'r', encoding='utf-8') as filehandle:
+            log = filehandle.read()
+
+        matchiter = re.finditer(r'(.*Summary: )(\d+ passed), (\d+ failed).*',
+                                log)
+        for match in matchiter:
+            # verify that no test cases failed.
+            try:
+                self.assertEqual(match.group(3).split()[0], '0',
+                                 'Some bits tests seems to have failed. ' \
+                                 'Please check the test logs for more info.')
+            except AssertionError as e:
+                self._print_log(log)
+                raise e
+            else:
+                if os.getenv('V'):
+                    self._print_log(log)
+
+    def tearDown(self):
+        """
+           Lets do some cleanups.
+        """
+        if self._vm:
+            self.assertFalse(not self._vm.is_running)
+        self.logger.info('removing the work directory %s', self._workDir)
+        shutil.rmtree(self._workDir)
+        super().tearDown()
+
+    def test_acpi_smbios_bits(self):
+        """The main test case implementaion."""
+
+        iso_file = os.path.join(self._workDir,
+                                'bits-%d.iso' %self._bitsInternalVer)
+
+        self.assertTrue(os.access(iso_file, os.R_OK))
+
+        self._vm = QEMUBitsMachine(binary=self.qemu_bin,
+                                   base_temp_dir=self._workDir,
+                                   debugcon_log=self._debugcon_log,
+                                   debugcon_addr=self._debugcon_addr)
+
+        self._vm.add_args('-cdrom', '%s' %iso_file)
+        # the vm needs to be run under icount so that TCG emulation is
+        # consistent in terms of timing. smilatency tests have consistent
+        # timing requirements.
+        self._vm.add_args('-icount', 'auto')
+
+        args = " ".join(str(arg) for arg in self._vm.base_args()) + \
+            " " + " ".join(str(arg) for arg in self._vm.args)
+
+        self.logger.info("launching QEMU vm with the following arguments: %s",
+                         args)
+
+        self._vm.launch()
+        # biosbits has been configured to run all the specified test suites
+        # in batch mode and then automatically initiate a vm shutdown.
+        # sleep for maximum of one minute
+        max_sleep_time = time.monotonic() + 60
+        while self._vm.is_running() and time.monotonic() < max_sleep_time:
+            time.sleep(1)
+
+        self.assertFalse(time.monotonic() > max_sleep_time,
+                         'The VM seems to have failed to shutdown in time')
+
+        self.parse_log()
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 15/83] acpi/tests/avocado/bits/doc: add a doc file to describe the acpi bits test
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (13 preceding siblings ...)
  2022-11-07 22:48 ` [PULL v4 14/83] acpi/tests/avocado/bits: add acpi and smbios avocado tests that uses biosbits Michael S. Tsirkin
@ 2022-11-07 22:48 ` Michael S. Tsirkin
  2022-11-07 22:48 ` [PULL v4 16/83] MAINTAINERS: add myself as the maintainer for acpi biosbits avocado tests Michael S. Tsirkin
                   ` (69 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:48 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Ani Sinha, Paolo Bonzini, John Snow, Thomas Huth,
	Alex Bennée, Igor Mammedov, Michael Tsirkin

From: Ani Sinha <ani@anisinha.ca>

A doc file is added under docs/devel that describes the purpose of the various
test files and gives guidance to developers on where and how to make changes.

Cc: Daniel P. Berrange" <berrange@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Maydell Peter <peter.maydell@linaro.org>
Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Michael Tsirkin <mst@redhat.com>
Signed-off-by: Ani Sinha <ani@anisinha.ca>
Message-Id: <20221021095108.104843-7-ani@anisinha.ca>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 docs/devel/acpi-bits.rst   | 145 +++++++++++++++++++++++++++++++++++++
 docs/devel/index-build.rst |   1 +
 2 files changed, 146 insertions(+)
 create mode 100644 docs/devel/acpi-bits.rst

diff --git a/docs/devel/acpi-bits.rst b/docs/devel/acpi-bits.rst
new file mode 100644
index 0000000000..c9564d871a
--- /dev/null
+++ b/docs/devel/acpi-bits.rst
@@ -0,0 +1,145 @@
+=============================================================================
+ACPI/SMBIOS avocado tests using biosbits
+=============================================================================
+
+Biosbits is a software written by Josh Triplett that can be downloaded
+from https://biosbits.org/. The github codebase can be found
+`here <https://github.com/biosbits/bits/tree/master>`__. It is a software that executes
+the bios components such as acpi and smbios tables directly through acpica
+bios interpreter (a freely available C based library written by Intel,
+downloadable from https://acpica.org/ and is included with biosbits) without an
+operating system getting involved in between.
+There are several advantages to directly testing the bios in a real physical
+machine or VM as opposed to indirectly discovering bios issues through the
+operating system. For one thing, the OSes tend to hide bios problems from the
+end user. The other is that we have more control of what we wanted to test
+and how by directly using acpica interpreter on top of the bios on a running
+system. More details on the inspiration for developing biosbits and its real
+life uses can be found in [#a]_ and [#b]_.
+This directory contains tests written in python using avocado framework that
+exercises the QEMU bios components using biosbits and reports test failures.
+For QEMU, we maintain a fork of bios bits in gitlab along with all the
+dependent submodules:
+https://gitlab.com/qemu-project/biosbits-bits
+This fork contains numerous fixes, a newer acpica and changes specific to
+running this avocado QEMU tests using bits. The author of this document
+is the sole maintainer of the QEMU fork of bios bits repo.
+
+Under the directory ``tests/avocado/``, ``acpi-bits.py`` is a QEMU avocado
+test that drives all this.
+
+A brief description of the various test files follows.
+
+Under ``tests/avocado/`` as the root we have:
+
+::
+
+   ├── acpi-bits
+   │ ├── bits-config
+   │ │ └── bits-cfg.txt
+   │ ├── bits-tests
+   │ │ ├── smbios.py2
+   │ │ ├── testacpi.py2
+   │ │ └── testcpuid.py2
+   │ └── README
+   ├── acpi-bits.py
+
+* ``tests/avocado``:
+
+   ``acpi-bits.py``:
+   This is the main python avocado test script that generates a
+   biosbits iso. It then spawns a QEMU VM with it, collects the log and reports
+   test failures. This is the script one would be interested in if they wanted
+   to add or change some component of the log parsing, add a new command line
+   to alter how QEMU is spawned etc. Test writers typically would not need to
+   modify this script unless they wanted to enhance or change the log parsing
+   for their tests. In order to enable debugging, you can set **V=1**
+   environment variable. This enables verbose mode for the test and also dumps
+   the entire log from bios bits and more information in case failure happens.
+
+   In order to run this test, please perform the following steps from the QEMU
+   build directory:
+   ::
+
+     $ make check-venv (needed only the first time to create the venv)
+     $ ./tests/venv/bin/avocado run -t acpi tests/avocado
+
+   The above will run all acpi avocado tests including this one.
+   In order to run the individual tests, perform the following:
+   ::
+
+     $ ./tests/venv/bin/avocado run tests/avocado/acpi-bits.py --tap -
+
+   The above will produce output in tap format. You can omit "--tap -" in the
+   end and it will produce output like the following:
+   ::
+
+      $ ./tests/venv/bin/avocado run tests/avocado/acpi-bits.py
+      Fetching asset from tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
+      JOB ID     : eab225724da7b64c012c65705dc2fa14ab1defef
+      JOB LOG    : /home/anisinha/avocado/job-results/job-2022-10-10T17.58-eab2257/job.log
+      (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: PASS (33.09 s)
+      RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
+      JOB TIME   : 39.22 s
+
+   You can inspect the log file for more information about the run or in order
+   to diagnoze issues. If you pass V=1 in the environment, more diagnostic logs
+   would be found in the test log.
+
+* ``tests/avocado/acpi-bits/bits-config``:
+
+   This location contains biosbits configuration files that determine how the
+   software runs the tests.
+
+   ``bits-config.txt``:
+   This is the biosbits config file that determines what tests
+   or actions are performed by bits. The description of the config options are
+   provided in the file itself.
+
+* ``tests/avocado/acpi-bits/bits-tests``:
+
+   This directory contains biosbits python based tests that are run from within
+   the biosbits environment in the spawned VM. New additions of test cases can
+   be made in the appropriate test file. For example, new acpi tests can go
+   into testacpi.py2 and one would call testsuite.add_test() to register the new
+   test so that it gets executed as a part of the ACPI tests.
+   It might be occasionally necessary to disable some subtests or add a new
+   test that belongs to a test suite not already present in this directory. To
+   do this, please clone the bits source from
+   https://gitlab.com/qemu-project/biosbits-bits/-/tree/qemu-bits.
+   Note that this is the "qemu-bits" branch and not the "bits" branch of the
+   repository. "qemu-bits" is the branch where we have made all the QEMU
+   specific enhancements and we must use the source from this branch only.
+   Copy the test suite/script that needs modification (addition of new tests
+   or disabling them) from python directory into this directory. For
+   example, in order to change cpuid related tests, copy the following
+   file into this directory and rename it with .py2 extension:
+   https://gitlab.com/qemu-project/biosbits-bits/-/blob/qemu-bits/python/testcpuid.py
+   Then make your additions and changes here. Therefore, the steps are:
+
+       (a) Copy unmodified test script to this directory from bits source.
+       (b) Add a SPDX license header.
+       (c) Perform modifications to the test.
+
+   Commits (a), (b) and (c) should go under separate commits so that the original
+   test script and the changes we have made are separated and clear.
+
+   The test framework will then use your modified test script to run the test.
+   No further changes would be needed. Please check the logs to make sure that
+   appropriate changes have taken effect.
+
+   The tests have an extension .py2 in order to indicate that:
+
+   (a) They are python2.7 based scripts and not python 3 scripts.
+   (b) They are run from within the bios bits VM and is not subjected to QEMU
+       build/test python script maintainance and dependency resolutions.
+   (c) They need not be loaded by avocado framework when running tests.
+
+
+Author: Ani Sinha <ani@anisinha.ca>
+
+References:
+-----------
+.. [#a] https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf
+.. [#b] https://www.youtube.com/watch?v=36QIepyUuhg
+
diff --git a/docs/devel/index-build.rst b/docs/devel/index-build.rst
index 1002a533a6..57e8d39d98 100644
--- a/docs/devel/index-build.rst
+++ b/docs/devel/index-build.rst
@@ -11,6 +11,7 @@ the basics if you are adding new files and targets to the build.
    build-system
    kconfig
    testing
+   acpi-bits
    qtest
    ci
    qapi-code-gen
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 16/83] MAINTAINERS: add myself as the maintainer for acpi biosbits avocado tests
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (14 preceding siblings ...)
  2022-11-07 22:48 ` [PULL v4 15/83] acpi/tests/avocado/bits/doc: add a doc file to describe the acpi bits test Michael S. Tsirkin
@ 2022-11-07 22:48 ` Michael S. Tsirkin
  2022-11-07 22:48 ` [PULL v4 17/83] tests/acpi: virt: allow acpi MADT and FADT changes Michael S. Tsirkin
                   ` (68 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:48 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Ani Sinha, Daniel P . Berrangé,
	Paolo Bonzini, John Snow, Thomas Huth, Alex Bennée,
	Igor Mammedov, Michael Tsirkin, Richard Henderson,
	Philippe Mathieu-Daudé

From: Ani Sinha <ani@anisinha.ca>

I wrote the biosbits avocado tests for testing QEMU's ACPI/SMBIOS implementation
and all the related changes including fixes in biosbits software itself. Making
myself as the maintainer for QEMU's biosbits related files and test scripts.

Cc: Daniel P. Berrangé <berrange@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Maydell Peter <peter.maydell@linaro.org>
Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Michael Tsirkin <mst@redhat.com>
Signed-off-by: Ani Sinha <ani@anisinha.ca>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221021095108.104843-8-ani@anisinha.ca>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 MAINTAINERS | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 4adf8c65db..07df572adf 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1862,6 +1862,13 @@ S: Supported
 F: hw/acpi/viot.c
 F: hw/acpi/viot.h
 
+ACPI/AVOCADO/BIOSBITS
+M: Ani Sinha <ani@anisinha.ca>
+S: Supported
+F: tests/avocado/acpi-bits/*
+F: tests/avocado/acpi-bits.py
+F: docs/devel/acpi-bits.rst
+
 ACPI/HEST/GHES
 R: Dongjiu Geng <gengdongjiu1@gmail.com>
 L: qemu-arm@nongnu.org
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 17/83] tests/acpi: virt: allow acpi MADT and FADT changes
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (15 preceding siblings ...)
  2022-11-07 22:48 ` [PULL v4 16/83] MAINTAINERS: add myself as the maintainer for acpi biosbits avocado tests Michael S. Tsirkin
@ 2022-11-07 22:48 ` Michael S. Tsirkin
  2022-11-07 22:49 ` [PULL v4 18/83] acpi: fadt: support revision 6.0 of the ACPI specification Michael S. Tsirkin
                   ` (67 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:48 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Miguel Luis, Ani Sinha, Igor Mammedov

From: Miguel Luis <miguel.luis@oracle.com>

Step 3 from bios-tables-test.c documented procedure.

Signed-off-by: Miguel Luis <miguel.luis@oracle.com>
Message-Id: <20221011181730.10885-2-miguel.luis@oracle.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Ani Sinha <ani@anisinha.ca>
---
 tests/qtest/bios-tables-test-allowed-diff.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index dfb8523c8b..8dc50f7a8a 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1 +1,7 @@
 /* List of comma-separated changed AML files to ignore */
+"tests/data/acpi/virt/FACP",
+"tests/data/acpi/virt/FACP.numamem",
+"tests/data/acpi/virt/FACP.memhp",
+"tests/data/acpi/virt/APIC",
+"tests/data/acpi/virt/APIC.memhp",
+"tests/data/acpi/virt/APIC.numamem",
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 18/83] acpi: fadt: support revision 6.0 of the ACPI specification
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (16 preceding siblings ...)
  2022-11-07 22:48 ` [PULL v4 17/83] tests/acpi: virt: allow acpi MADT and FADT changes Michael S. Tsirkin
@ 2022-11-07 22:49 ` Michael S. Tsirkin
  2022-11-07 22:49 ` [PULL v4 19/83] acpi: arm/virt: madt: bump to revision 4 accordingly to ACPI 6.0 Errata A Michael S. Tsirkin
                   ` (66 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:49 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Miguel Luis, Ani Sinha, Igor Mammedov,
	Shannon Zhao, qemu-arm

From: Miguel Luis <miguel.luis@oracle.com>

Update the Fixed ACPI Description Table (FADT) to revision 6.0 of the ACPI
specification adding the field "Hypervisor Vendor Identity".

This field's description states the following: "64-bit identifier of hypervisor
vendor. All bytes in this field are considered part of the vendor identity.
These identifiers are defined independently by the vendors themselves,
usually following the name of the hypervisor product. Version information
should NOT be included in this field - this shall simply denote the vendor's
name or identifier. Version information can be communicated through a
supplemental vendor-specific hypervisor API. Firmware implementers would
place zero bytes into this field, denoting that no hypervisor is present in
the actual firmware."

Signed-off-by: Miguel Luis <miguel.luis@oracle.com>
Reviewed-by: Ani Sinha <ani@anisinha.ca>
Message-Id: <20221011181730.10885-3-miguel.luis@oracle.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/acpi/aml-build.c      | 13 ++++++++++---
 hw/arm/virt-acpi-build.c | 10 +++++-----
 2 files changed, 15 insertions(+), 8 deletions(-)

diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
index e6bfac95c7..42feb4d4d7 100644
--- a/hw/acpi/aml-build.c
+++ b/hw/acpi/aml-build.c
@@ -2070,7 +2070,7 @@ void build_pptt(GArray *table_data, BIOSLinker *linker, MachineState *ms,
     acpi_table_end(linker, &table);
 }
 
-/* build rev1/rev3/rev5.1 FADT */
+/* build rev1/rev3/rev5.1/rev6.0 FADT */
 void build_fadt(GArray *tbl, BIOSLinker *linker, const AcpiFadtData *f,
                 const char *oem_id, const char *oem_table_id)
 {
@@ -2193,8 +2193,15 @@ void build_fadt(GArray *tbl, BIOSLinker *linker, const AcpiFadtData *f,
     /* SLEEP_STATUS_REG */
     build_append_gas_from_struct(tbl, &f->sleep_sts);
 
-    /* TODO: extra fields need to be added to support revisions above rev5 */
-    assert(f->rev == 5);
+    if (f->rev == 5) {
+        goto done;
+    }
+
+    /* Hypervisor Vendor Identity */
+    build_append_padded_str(tbl, "QEMU", 8, '\0');
+
+    /* TODO: extra fields need to be added to support revisions above rev6 */
+    assert(f->rev == 6);
 
 done:
     acpi_table_end(linker, &table);
diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
index 13c6e3e468..e5377744f3 100644
--- a/hw/arm/virt-acpi-build.c
+++ b/hw/arm/virt-acpi-build.c
@@ -808,13 +808,13 @@ build_madt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
 }
 
 /* FADT */
-static void build_fadt_rev5(GArray *table_data, BIOSLinker *linker,
+static void build_fadt_rev6(GArray *table_data, BIOSLinker *linker,
                             VirtMachineState *vms, unsigned dsdt_tbl_offset)
 {
-    /* ACPI v5.1 */
+    /* ACPI v6.0 */
     AcpiFadtData fadt = {
-        .rev = 5,
-        .minor_ver = 1,
+        .rev = 6,
+        .minor_ver = 0,
         .flags = 1 << ACPI_FADT_F_HW_REDUCED_ACPI,
         .xdsdt_tbl_offset = &dsdt_tbl_offset,
     };
@@ -944,7 +944,7 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables)
 
     /* FADT MADT PPTT GTDT MCFG SPCR DBG2 pointed to by RSDT */
     acpi_add_table(table_offsets, tables_blob);
-    build_fadt_rev5(tables_blob, tables->linker, vms, dsdt);
+    build_fadt_rev6(tables_blob, tables->linker, vms, dsdt);
 
     acpi_add_table(table_offsets, tables_blob);
     build_madt(tables_blob, tables->linker, vms);
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 19/83] acpi: arm/virt: madt: bump to revision 4 accordingly to ACPI 6.0 Errata A
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (17 preceding siblings ...)
  2022-11-07 22:49 ` [PULL v4 18/83] acpi: fadt: support revision 6.0 of the ACPI specification Michael S. Tsirkin
@ 2022-11-07 22:49 ` Michael S. Tsirkin
  2022-11-07 22:49 ` [PULL v4 20/83] tests/acpi: virt: update ACPI MADT and FADT binaries Michael S. Tsirkin
                   ` (65 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:49 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Miguel Luis, Ani Sinha, Shannon Zhao,
	Igor Mammedov, qemu-arm

From: Miguel Luis <miguel.luis@oracle.com>

MADT has been updated with the GIC Structures from ACPI 6.0 Errata A
and so MADT revision and GICC Structure must be updated also.

Fixes: 37f33084ed2e ("acpi: arm/virt: madt: use build_append_int_noprefix() API to compose MADT table")

Signed-off-by: Miguel Luis <miguel.luis@oracle.com>
Reviewed-by: Ani Sinha <ani@anisinha.ca>
Message-Id: <20221011181730.10885-4-miguel.luis@oracle.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/arm/virt-acpi-build.c | 16 +++++++---------
 1 file changed, 7 insertions(+), 9 deletions(-)

diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
index e5377744f3..da9e41e72b 100644
--- a/hw/arm/virt-acpi-build.c
+++ b/hw/arm/virt-acpi-build.c
@@ -685,7 +685,7 @@ build_dbg2(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
 };
 
 /*
- * ACPI spec, Revision 5.1 Errata A
+ * ACPI spec, Revision 6.0 Errata A
  * 5.2.12 Multiple APIC Description Table (MADT)
  */
 static void build_append_gicr(GArray *table_data, uint64_t base, uint32_t size)
@@ -704,7 +704,7 @@ build_madt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
     int i;
     VirtMachineClass *vmc = VIRT_MACHINE_GET_CLASS(vms);
     const MemMapEntry *memmap = vms->memmap;
-    AcpiTable table = { .sig = "APIC", .rev = 3, .oem_id = vms->oem_id,
+    AcpiTable table = { .sig = "APIC", .rev = 4, .oem_id = vms->oem_id,
                         .oem_table_id = vms->oem_table_id };
 
     acpi_table_begin(&table, table_data);
@@ -739,7 +739,7 @@ build_madt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
 
         /* 5.2.12.14 GIC Structure */
         build_append_int_noprefix(table_data, 0xB, 1);  /* Type */
-        build_append_int_noprefix(table_data, 76, 1);   /* Length */
+        build_append_int_noprefix(table_data, 80, 1);   /* Length */
         build_append_int_noprefix(table_data, 0, 2);    /* Reserved */
         build_append_int_noprefix(table_data, i, 4);    /* GIC ID */
         build_append_int_noprefix(table_data, i, 4);    /* ACPI Processor UID */
@@ -759,6 +759,10 @@ build_madt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
         build_append_int_noprefix(table_data, 0, 8);    /* GICR Base Address*/
         /* MPIDR */
         build_append_int_noprefix(table_data, armcpu->mp_affinity, 8);
+        /* Processor Power Efficiency Class */
+        build_append_int_noprefix(table_data, 0, 1);
+        /* Reserved */
+        build_append_int_noprefix(table_data, 0, 3);
     }
 
     if (vms->gic_version != VIRT_GIC_VERSION_2) {
@@ -770,12 +774,6 @@ build_madt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
         }
 
         if (its_class_name() && !vmc->no_its) {
-            /*
-             * FIXME: Structure is from Revision 6.0 where 'GIC Structure'
-             * has additional fields on top of implemented 5.1 Errata A,
-             * to make it consistent with v6.0 we need to bump everything
-             * to v6.0
-             */
             /*
              * ACPI spec, Revision 6.0 Errata A
              * (original 6.0 definition has invalid Length)
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 20/83] tests/acpi: virt: update ACPI MADT and FADT binaries
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (18 preceding siblings ...)
  2022-11-07 22:49 ` [PULL v4 19/83] acpi: arm/virt: madt: bump to revision 4 accordingly to ACPI 6.0 Errata A Michael S. Tsirkin
@ 2022-11-07 22:49 ` Michael S. Tsirkin
  2022-11-07 22:49 ` [PULL v4 21/83] hw/pci: PCIe Data Object Exchange emulation Michael S. Tsirkin
                   ` (64 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:49 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Miguel Luis, Ani Sinha, Igor Mammedov

From: Miguel Luis <miguel.luis@oracle.com>

Step 6 & 7 of the bios-tables-test.c documented procedure.

Differences between disassembled ASL files for MADT:

@@ -11,9 +11,9 @@
  */

 [000h 0000   4]                    Signature : "APIC"    [Multiple APIC Description Table (MADT)]
-[004h 0004   4]                 Table Length : 000000A8
-[008h 0008   1]                     Revision : 03
-[009h 0009   1]                     Checksum : 50
+[004h 0004   4]                 Table Length : 000000AC
+[008h 0008   1]                     Revision : 04
+[009h 0009   1]                     Checksum : 47
 [00Ah 0010   6]                       Oem ID : "BOCHS "
 [010h 0016   8]                 Oem Table ID : "BXPC    "
 [018h 0024   4]                 Oem Revision : 00000001
@@ -34,7 +34,7 @@
 [041h 0065   3]                     Reserved : 000000

 [044h 0068   1]                Subtable Type : 0B [Generic Interrupt Controller]
-[045h 0069   1]                       Length : 4C
+[045h 0069   1]                       Length : 50
 [046h 0070   2]                     Reserved : 0000
 [048h 0072   4]         CPU Interface Number : 00000000
 [04Ch 0076   4]                Processor UID : 00000000
@@ -51,28 +51,29 @@
 [07Ch 0124   4]        Virtual GIC Interrupt : 00000000
 [080h 0128   8]   Redistributor Base Address : 0000000000000000
 [088h 0136   8]                    ARM MPIDR : 0000000000000000
-/**** ACPI subtable terminates early - may be older version (dump table) */
+[090h 0144   1]             Efficiency Class : 00
+[091h 0145   3]                     Reserved : 000000

-[090h 0144   1]                Subtable Type : 0D [Generic MSI Frame]
-[091h 0145   1]                       Length : 18
-[092h 0146   2]                     Reserved : 0000
-[094h 0148   4]                 MSI Frame ID : 00000000
-[098h 0152   8]                 Base Address : 0000000008020000
-[0A0h 0160   4]        Flags (decoded below) : 00000001
+[094h 0148   1]                Subtable Type : 0D [Generic MSI Frame]
+[095h 0149   1]                       Length : 18
+[096h 0150   2]                     Reserved : 0000
+[098h 0152   4]                 MSI Frame ID : 00000000
+[09Ch 0156   8]                 Base Address : 0000000008020000
+[0A4h 0164   4]        Flags (decoded below) : 00000001
                                   Select SPI : 1
-[0A4h 0164   2]                    SPI Count : 0040
-[0A6h 0166   2]                     SPI Base : 0050
+[0A8h 0168   2]                    SPI Count : 0040
+[0AAh 0170   2]                     SPI Base : 0050

-Raw Table Data: Length 168 (0xA8)
+Raw Table Data: Length 172 (0xAC)

-    0000: 41 50 49 43 A8 00 00 00 03 50 42 4F 43 48 53 20  // APIC.....PBOCHS
+    0000: 41 50 49 43 AC 00 00 00 04 47 42 4F 43 48 53 20  // APIC.....GBOCHS
     0010: 42 58 50 43 20 20 20 20 01 00 00 00 42 58 50 43  // BXPC    ....BXPC
     0020: 01 00 00 00 00 00 00 00 00 00 00 00 0C 18 00 00  // ................
     0030: 00 00 00 00 00 00 00 08 00 00 00 00 00 00 00 00  // ................
-    0040: 02 00 00 00 0B 4C 00 00 00 00 00 00 00 00 00 00  // .....L..........
+    0040: 02 00 00 00 0B 50 00 00 00 00 00 00 00 00 00 00  // .....P..........
     0050: 01 00 00 00 00 00 00 00 17 00 00 00 00 00 00 00  // ................
     0060: 00 00 00 00 00 00 01 08 00 00 00 00 00 00 04 08  // ................
     0070: 00 00 00 00 00 00 03 08 00 00 00 00 00 00 00 00  // ................
     0080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
-    0090: 0D 18 00 00 00 00 00 00 00 00 02 08 00 00 00 00  // ................
-    00A0: 01 00 00 00 40 00 50 00                          // ....@.P.
+    0090: 00 00 00 00 0D 18 00 00 00 00 00 00 00 00 02 08  // ................
+    00A0: 00 00 00 00 01 00 00 00 40 00 50 00              // ........@.P.

Differences between disassembled ASL files for FADT:

@@ -11,9 +11,9 @@
  */

 [000h 0000   4]                    Signature : "FACP"    [Fixed ACPI Description Table (FADT)]
-[004h 0004   4]                 Table Length : 0000010C
-[008h 0008   1]                     Revision : 05
-[009h 0009   1]                     Checksum : 55
+[004h 0004   4]                 Table Length : 00000114
+[008h 0008   1]                     Revision : 06
+[009h 0009   1]                     Checksum : 15
 [00Ah 0010   6]                       Oem ID : "BOCHS "
 [010h 0016   8]                 Oem Table ID : "BXPC    "
 [018h 0024   4]                 Oem Revision : 00000001
@@ -99,7 +99,7 @@
                               PSCI Compliant : 1
                        Must use HVC for PSCI : 1

-[083h 0131   1]          FADT Minor Revision : 01
+[083h 0131   1]          FADT Minor Revision : 00
 [084h 0132   8]                 FACS Address : 0000000000000000
 [08Ch 0140   8]                 DSDT Address : 0000000000000000
 [094h 0148  12]             PM1A Event Block : [Generic Address Structure]
@@ -173,11 +173,11 @@
 [103h 0259   1]         Encoded Access Width : 00 [Undefined/Legacy]
 [104h 0260   8]                      Address : 0000000000000000

-/**** ACPI table terminates in the middle of a data structure! (dump table) */
+[10Ch 0268   8]                Hypervisor ID : 00000000554D4551

-Raw Table Data: Length 268 (0x10C)
+Raw Table Data: Length 276 (0x114)

-    0000: 46 41 43 50 0C 01 00 00 05 55 42 4F 43 48 53 20  // FACP.....UBOCHS
+    0000: 46 41 43 50 14 01 00 00 06 15 42 4F 43 48 53 20  // FACP......BOCHS
     0010: 42 58 50 43 20 20 20 20 01 00 00 00 42 58 50 43  // BXPC    ....BXPC
     0020: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
     0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
@@ -185,7 +185,7 @@ Raw Table Data: Length 268 (0x10C)
     0050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
     0060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
     0070: 00 00 10 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
-    0080: 00 03 00 01 00 00 00 00 00 00 00 00 00 00 00 00  // ................
+    0080: 00 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
     0090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
     00A0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
     00B0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
@@ -193,4 +193,5 @@ Raw Table Data: Length 268 (0x10C)
     00D0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
     00E0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
     00F0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
-    0100: 00 00 00 00 00 00 00 00 00 00 00 00              // ............
+    0100: 00 00 00 00 00 00 00 00 00 00 00 00 51 45 4D 55  // ............QEMU
+    0110: 00 00 00 00                                      // ....

Signed-off-by: Miguel Luis <miguel.luis@oracle.com>
Message-Id: <20221011181730.10885-5-miguel.luis@oracle.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Ani Sinha <ani@anisinha.ca>
---
 tests/qtest/bios-tables-test-allowed-diff.h |   6 ------
 tests/data/acpi/virt/APIC                   | Bin 168 -> 172 bytes
 tests/data/acpi/virt/APIC.memhp             | Bin 168 -> 172 bytes
 tests/data/acpi/virt/APIC.numamem           | Bin 168 -> 172 bytes
 tests/data/acpi/virt/FACP                   | Bin 268 -> 276 bytes
 tests/data/acpi/virt/FACP.memhp             | Bin 268 -> 276 bytes
 tests/data/acpi/virt/FACP.numamem           | Bin 268 -> 276 bytes
 7 files changed, 6 deletions(-)

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index 8dc50f7a8a..dfb8523c8b 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1,7 +1 @@
 /* List of comma-separated changed AML files to ignore */
-"tests/data/acpi/virt/FACP",
-"tests/data/acpi/virt/FACP.numamem",
-"tests/data/acpi/virt/FACP.memhp",
-"tests/data/acpi/virt/APIC",
-"tests/data/acpi/virt/APIC.memhp",
-"tests/data/acpi/virt/APIC.numamem",
diff --git a/tests/data/acpi/virt/APIC b/tests/data/acpi/virt/APIC
index 023f15f12e74fb9a3a6d3d9dc994541016947d6a..179d274770a23209b949c90a929525e22368568b 100644
GIT binary patch
delta 26
hcmZ3%xQ3C-F~HM#4FdxMi~B?_YsP?yZeA06WB^*d2KE2|

delta 26
icmZ3(xPp<(F~HM#1p@;EbHGF{Yet`mZeA0oNB{s@&<6Sd

diff --git a/tests/data/acpi/virt/APIC.memhp b/tests/data/acpi/virt/APIC.memhp
index 023f15f12e74fb9a3a6d3d9dc994541016947d6a..179d274770a23209b949c90a929525e22368568b 100644
GIT binary patch
delta 26
hcmZ3%xQ3C-F~HM#4FdxMi~B?_YsP?yZeA06WB^*d2KE2|

delta 26
icmZ3(xPp<(F~HM#1p@;EbHGF{Yet`mZeA0oNB{s@&<6Sd

diff --git a/tests/data/acpi/virt/APIC.numamem b/tests/data/acpi/virt/APIC.numamem
index 023f15f12e74fb9a3a6d3d9dc994541016947d6a..179d274770a23209b949c90a929525e22368568b 100644
GIT binary patch
delta 26
hcmZ3%xQ3C-F~HM#4FdxMi~B?_YsP?yZeA06WB^*d2KE2|

delta 26
icmZ3(xPp<(F~HM#1p@;EbHGF{Yet`mZeA0oNB{s@&<6Sd

diff --git a/tests/data/acpi/virt/FACP b/tests/data/acpi/virt/FACP
index 1f764220f8533c427168e80ccf298604826a00b4..ac05c35a69451519bd1152c54d1e741af36390f5 100644
GIT binary patch
delta 33
ncmeBSn!?28=I9(C!pOkDCOVO;a^j?_i3a=}fv&!x3_t(?fr|$^

delta 26
hcmbQj)WgK(=I9*2!^ptE8ak1yl96%Z#OjF#yZ}u&1~C8t

diff --git a/tests/data/acpi/virt/FACP.memhp b/tests/data/acpi/virt/FACP.memhp
index 1f764220f8533c427168e80ccf298604826a00b4..ac05c35a69451519bd1152c54d1e741af36390f5 100644
GIT binary patch
delta 33
ncmeBSn!?28=I9(C!pOkDCOVO;a^j?_i3a=}fv&!x3_t(?fr|$^

delta 26
hcmbQj)WgK(=I9*2!^ptE8ak1yl96%Z#OjF#yZ}u&1~C8t

diff --git a/tests/data/acpi/virt/FACP.numamem b/tests/data/acpi/virt/FACP.numamem
index 1f764220f8533c427168e80ccf298604826a00b4..ac05c35a69451519bd1152c54d1e741af36390f5 100644
GIT binary patch
delta 33
ncmeBSn!?28=I9(C!pOkDCOVO;a^j?_i3a=}fv&!x3_t(?fr|$^

delta 26
hcmbQj)WgK(=I9*2!^ptE8ak1yl96%Z#OjF#yZ}u&1~C8t

-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 21/83] hw/pci: PCIe Data Object Exchange emulation
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (19 preceding siblings ...)
  2022-11-07 22:49 ` [PULL v4 20/83] tests/acpi: virt: update ACPI MADT and FADT binaries Michael S. Tsirkin
@ 2022-11-07 22:49 ` Michael S. Tsirkin
  2022-11-07 22:49 ` [PULL v4 22/83] hw/mem/cxl-type3: Add MSIX support Michael S. Tsirkin
                   ` (63 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:49 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Huai-Cheng Kuo, Chris Browy, Jonathan Cameron,
	Marcel Apfelbaum

From: Huai-Cheng Kuo <hchkuo@avery-design.com.tw>

Emulation of PCIe Data Object Exchange (DOE)
PCIE Base Specification r6.0 6.3 Data Object Exchange

Supports multiple DOE PCIe Extended Capabilities for a single PCIe
device. For each capability, a static array of DOEProtocol should be passed
to pcie_doe_init(). The protocols in that array will be registered under
the DOE capability structure. For each protocol, vendor ID, type, and
corresponding callback function (handle_request()) should be implemented.
This callback function represents how the DOE request for corresponding
protocol will be handled.

pcie_doe_{read/write}_config() must be appended to corresponding PCI
device's config_read/write() handler to enable DOE access. In
pcie_doe_read_config(), false will be returned if pci_config_read()
offset is not within DOE capability range. In pcie_doe_write_config(),
the function will have no affect if the address is not within the related
DOE PCIE extended capability.

Signed-off-by: Huai-Cheng Kuo <hchkuo@avery-design.com.tw>
Signed-off-by: Chris Browy <cbrowy@avery-design.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Message-Id: <20221014151045.24781-2-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/hw/pci/pci_ids.h   |   3 +
 include/hw/pci/pcie.h      |   1 +
 include/hw/pci/pcie_doe.h  | 123 +++++++++++++
 include/hw/pci/pcie_regs.h |   4 +
 hw/pci/pcie_doe.c          | 367 +++++++++++++++++++++++++++++++++++++
 MAINTAINERS                |   7 +
 hw/pci/meson.build         |   1 +
 7 files changed, 506 insertions(+)
 create mode 100644 include/hw/pci/pcie_doe.h
 create mode 100644 hw/pci/pcie_doe.c

diff --git a/include/hw/pci/pci_ids.h b/include/hw/pci/pci_ids.h
index d5ddea558b..bc9f834fd1 100644
--- a/include/hw/pci/pci_ids.h
+++ b/include/hw/pci/pci_ids.h
@@ -157,6 +157,9 @@
 
 /* Vendors and devices.  Sort key: vendor first, device next. */
 
+/* Ref: PCIe r6.0 Table 6-32 */
+#define PCI_VENDOR_ID_PCI_SIG            0x0001
+
 #define PCI_VENDOR_ID_LSI_LOGIC          0x1000
 #define PCI_DEVICE_ID_LSI_53C810         0x0001
 #define PCI_DEVICE_ID_LSI_53C895A        0x0012
diff --git a/include/hw/pci/pcie.h b/include/hw/pci/pcie.h
index 798a262a0a..698d3de851 100644
--- a/include/hw/pci/pcie.h
+++ b/include/hw/pci/pcie.h
@@ -26,6 +26,7 @@
 #include "hw/pci/pcie_aer.h"
 #include "hw/pci/pcie_sriov.h"
 #include "hw/hotplug.h"
+#include "hw/pci/pcie_doe.h"
 
 typedef enum {
     /* for attention and power indicator */
diff --git a/include/hw/pci/pcie_doe.h b/include/hw/pci/pcie_doe.h
new file mode 100644
index 0000000000..ba4d8b03bd
--- /dev/null
+++ b/include/hw/pci/pcie_doe.h
@@ -0,0 +1,123 @@
+/*
+ * PCIe Data Object Exchange
+ *
+ * Copyright (C) 2021 Avery Design Systems, Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef PCIE_DOE_H
+#define PCIE_DOE_H
+
+#include "qemu/range.h"
+#include "qemu/typedefs.h"
+#include "hw/register.h"
+
+/*
+ * Reference:
+ * PCIe r6.0 - 7.9.24 Data Object Exchange Extended Capability
+ */
+/* Capabilities Register - r6.0 7.9.24.2 */
+#define PCI_EXP_DOE_CAP             0x04
+REG32(PCI_DOE_CAP_REG, 0)
+    FIELD(PCI_DOE_CAP_REG, INTR_SUPP, 0, 1)
+    FIELD(PCI_DOE_CAP_REG, DOE_INTR_MSG_NUM, 1, 11)
+
+/* Control Register - r6.0 7.9.24.3 */
+#define PCI_EXP_DOE_CTRL            0x08
+REG32(PCI_DOE_CAP_CONTROL, 0)
+    FIELD(PCI_DOE_CAP_CONTROL, DOE_ABORT, 0, 1)
+    FIELD(PCI_DOE_CAP_CONTROL, DOE_INTR_EN, 1, 1)
+    FIELD(PCI_DOE_CAP_CONTROL, DOE_GO, 31, 1)
+
+/* Status Register - r6.0 7.9.24.4 */
+#define PCI_EXP_DOE_STATUS          0x0c
+REG32(PCI_DOE_CAP_STATUS, 0)
+    FIELD(PCI_DOE_CAP_STATUS, DOE_BUSY, 0, 1)
+    FIELD(PCI_DOE_CAP_STATUS, DOE_INTR_STATUS, 1, 1)
+    FIELD(PCI_DOE_CAP_STATUS, DOE_ERROR, 2, 1)
+    FIELD(PCI_DOE_CAP_STATUS, DATA_OBJ_RDY, 31, 1)
+
+/* Write Data Mailbox Register - r6.0 7.9.24.5 */
+#define PCI_EXP_DOE_WR_DATA_MBOX    0x10
+
+/* Read Data Mailbox Register - 7.9.xx.6 */
+#define PCI_EXP_DOE_RD_DATA_MBOX    0x14
+
+/* PCI-SIG defined Data Object Types - r6.0 Table 6-32 */
+#define PCI_SIG_DOE_DISCOVERY       0x00
+
+#define PCI_DOE_DW_SIZE_MAX         (1 << 18)
+#define PCI_DOE_PROTOCOL_NUM_MAX    256
+
+#define DATA_OBJ_BUILD_HEADER1(v, p)    (((p) << 16) | (v))
+#define DATA_OBJ_LEN_MASK(len)          ((len) & (PCI_DOE_DW_SIZE_MAX - 1))
+
+typedef struct DOEHeader DOEHeader;
+typedef struct DOEProtocol DOEProtocol;
+typedef struct DOECap DOECap;
+
+struct DOEHeader {
+    uint16_t vendor_id;
+    uint8_t data_obj_type;
+    uint8_t reserved;
+    uint32_t length;
+} QEMU_PACKED;
+
+/* Protocol infos and rsp function callback */
+struct DOEProtocol {
+    uint16_t vendor_id;
+    uint8_t data_obj_type;
+    bool (*handle_request)(DOECap *);
+};
+
+struct DOECap {
+    /* Owner */
+    PCIDevice *pdev;
+
+    uint16_t offset;
+
+    struct {
+        bool intr;
+        uint16_t vec;
+    } cap;
+
+    struct {
+        bool abort;
+        bool intr;
+        bool go;
+    } ctrl;
+
+    struct {
+        bool busy;
+        bool intr;
+        bool error;
+        bool ready;
+    } status;
+
+    uint32_t *write_mbox;
+    uint32_t *read_mbox;
+
+    /* Mailbox position indicator */
+    uint32_t read_mbox_idx;
+    uint32_t read_mbox_len;
+    uint32_t write_mbox_len;
+
+    /* Protocols and its callback response */
+    DOEProtocol *protocols;
+    uint16_t protocol_num;
+};
+
+void pcie_doe_init(PCIDevice *pdev, DOECap *doe_cap, uint16_t offset,
+                   DOEProtocol *protocols, bool intr, uint16_t vec);
+void pcie_doe_fini(DOECap *doe_cap);
+bool pcie_doe_read_config(DOECap *doe_cap, uint32_t addr, int size,
+                          uint32_t *buf);
+void pcie_doe_write_config(DOECap *doe_cap, uint32_t addr,
+                           uint32_t val, int size);
+uint32_t pcie_doe_build_protocol(DOEProtocol *p);
+void *pcie_doe_get_write_mbox_ptr(DOECap *doe_cap);
+void pcie_doe_set_rsp(DOECap *doe_cap, void *rsp);
+uint32_t pcie_doe_get_obj_len(void *obj);
+#endif /* PCIE_DOE_H */
diff --git a/include/hw/pci/pcie_regs.h b/include/hw/pci/pcie_regs.h
index 1db86b0ec4..963dc2e170 100644
--- a/include/hw/pci/pcie_regs.h
+++ b/include/hw/pci/pcie_regs.h
@@ -179,4 +179,8 @@ typedef enum PCIExpLinkWidth {
 #define PCI_ACS_VER                     0x1
 #define PCI_ACS_SIZEOF                  8
 
+/* DOE Capability Register Fields */
+#define PCI_DOE_VER                     0x1
+#define PCI_DOE_SIZEOF                  24
+
 #endif /* QEMU_PCIE_REGS_H */
diff --git a/hw/pci/pcie_doe.c b/hw/pci/pcie_doe.c
new file mode 100644
index 0000000000..2210f86968
--- /dev/null
+++ b/hw/pci/pcie_doe.c
@@ -0,0 +1,367 @@
+/*
+ * PCIe Data Object Exchange
+ *
+ * Copyright (C) 2021 Avery Design Systems, Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/log.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
+#include "qemu/range.h"
+#include "hw/pci/pci.h"
+#include "hw/pci/pcie.h"
+#include "hw/pci/pcie_doe.h"
+#include "hw/pci/msi.h"
+#include "hw/pci/msix.h"
+
+#define DWORD_BYTE 4
+
+typedef struct DoeDiscoveryReq {
+    DOEHeader header;
+    uint8_t index;
+    uint8_t reserved[3];
+} QEMU_PACKED DoeDiscoveryReq;
+
+typedef struct DoeDiscoveryRsp {
+    DOEHeader header;
+    uint16_t vendor_id;
+    uint8_t data_obj_type;
+    uint8_t next_index;
+} QEMU_PACKED DoeDiscoveryRsp;
+
+static bool pcie_doe_discovery(DOECap *doe_cap)
+{
+    DoeDiscoveryReq *req = pcie_doe_get_write_mbox_ptr(doe_cap);
+    DoeDiscoveryRsp rsp;
+    uint8_t index = req->index;
+    DOEProtocol *prot;
+
+    /* Discard request if length does not match DoeDiscoveryReq */
+    if (pcie_doe_get_obj_len(req) <
+        DIV_ROUND_UP(sizeof(DoeDiscoveryReq), DWORD_BYTE)) {
+        return false;
+    }
+
+    rsp.header = (DOEHeader) {
+        .vendor_id = PCI_VENDOR_ID_PCI_SIG,
+        .data_obj_type = PCI_SIG_DOE_DISCOVERY,
+        .length = DIV_ROUND_UP(sizeof(DoeDiscoveryRsp), DWORD_BYTE),
+    };
+
+    /* Point to the requested protocol, index 0 must be Discovery */
+    if (index == 0) {
+        rsp.vendor_id = PCI_VENDOR_ID_PCI_SIG;
+        rsp.data_obj_type = PCI_SIG_DOE_DISCOVERY;
+    } else {
+        if (index < doe_cap->protocol_num) {
+            prot = &doe_cap->protocols[index - 1];
+            rsp.vendor_id = prot->vendor_id;
+            rsp.data_obj_type = prot->data_obj_type;
+        } else {
+            rsp.vendor_id = 0xFFFF;
+            rsp.data_obj_type = 0xFF;
+        }
+    }
+
+    if (index + 1 == doe_cap->protocol_num) {
+        rsp.next_index = 0;
+    } else {
+        rsp.next_index = index + 1;
+    }
+
+    pcie_doe_set_rsp(doe_cap, &rsp);
+
+    return true;
+}
+
+static void pcie_doe_reset_mbox(DOECap *st)
+{
+    st->read_mbox_idx = 0;
+    st->read_mbox_len = 0;
+    st->write_mbox_len = 0;
+
+    memset(st->read_mbox, 0, PCI_DOE_DW_SIZE_MAX * DWORD_BYTE);
+    memset(st->write_mbox, 0, PCI_DOE_DW_SIZE_MAX * DWORD_BYTE);
+}
+
+void pcie_doe_init(PCIDevice *dev, DOECap *doe_cap, uint16_t offset,
+                   DOEProtocol *protocols, bool intr, uint16_t vec)
+{
+    pcie_add_capability(dev, PCI_EXT_CAP_ID_DOE, 0x1, offset,
+                        PCI_DOE_SIZEOF);
+
+    doe_cap->pdev = dev;
+    doe_cap->offset = offset;
+
+    if (intr && (msi_present(dev) || msix_present(dev))) {
+        doe_cap->cap.intr = intr;
+        doe_cap->cap.vec = vec;
+    }
+
+    doe_cap->write_mbox = g_malloc0(PCI_DOE_DW_SIZE_MAX * DWORD_BYTE);
+    doe_cap->read_mbox = g_malloc0(PCI_DOE_DW_SIZE_MAX * DWORD_BYTE);
+
+    pcie_doe_reset_mbox(doe_cap);
+
+    doe_cap->protocols = protocols;
+    for (; protocols->vendor_id; protocols++) {
+        doe_cap->protocol_num++;
+    }
+    assert(doe_cap->protocol_num < PCI_DOE_PROTOCOL_NUM_MAX);
+
+    /* Increment to allow for the discovery protocol */
+    doe_cap->protocol_num++;
+}
+
+void pcie_doe_fini(DOECap *doe_cap)
+{
+    g_free(doe_cap->read_mbox);
+    g_free(doe_cap->write_mbox);
+    g_free(doe_cap);
+}
+
+uint32_t pcie_doe_build_protocol(DOEProtocol *p)
+{
+    return DATA_OBJ_BUILD_HEADER1(p->vendor_id, p->data_obj_type);
+}
+
+void *pcie_doe_get_write_mbox_ptr(DOECap *doe_cap)
+{
+    return doe_cap->write_mbox;
+}
+
+/*
+ * Copy the response to read mailbox buffer
+ * This might be called in self-defined handle_request() if a DOE response is
+ * required in the corresponding protocol
+ */
+void pcie_doe_set_rsp(DOECap *doe_cap, void *rsp)
+{
+    uint32_t len = pcie_doe_get_obj_len(rsp);
+
+    memcpy(doe_cap->read_mbox + doe_cap->read_mbox_len, rsp, len * DWORD_BYTE);
+    doe_cap->read_mbox_len += len;
+}
+
+uint32_t pcie_doe_get_obj_len(void *obj)
+{
+    uint32_t len;
+
+    if (!obj) {
+        return 0;
+    }
+
+    /* Only lower 18 bits are valid */
+    len = DATA_OBJ_LEN_MASK(((DOEHeader *)obj)->length);
+
+    /* PCIe r6.0 Table 6.29: a value of 00000h indicates 2^18 DW */
+    return (len) ? len : PCI_DOE_DW_SIZE_MAX;
+}
+
+static void pcie_doe_irq_assert(DOECap *doe_cap)
+{
+    PCIDevice *dev = doe_cap->pdev;
+
+    if (doe_cap->cap.intr && doe_cap->ctrl.intr) {
+        if (doe_cap->status.intr) {
+            return;
+        }
+        doe_cap->status.intr = 1;
+
+        if (msix_enabled(dev)) {
+            msix_notify(dev, doe_cap->cap.vec);
+        } else if (msi_enabled(dev)) {
+            msi_notify(dev, doe_cap->cap.vec);
+        }
+    }
+}
+
+static void pcie_doe_set_ready(DOECap *doe_cap, bool rdy)
+{
+    doe_cap->status.ready = rdy;
+
+    if (rdy) {
+        pcie_doe_irq_assert(doe_cap);
+    }
+}
+
+static void pcie_doe_set_error(DOECap *doe_cap, bool err)
+{
+    doe_cap->status.error = err;
+
+    if (err) {
+        pcie_doe_irq_assert(doe_cap);
+    }
+}
+
+/*
+ * Check incoming request in write_mbox for protocol format
+ */
+static void pcie_doe_prepare_rsp(DOECap *doe_cap)
+{
+    bool success = false;
+    int p;
+    bool (*handle_request)(DOECap *) = NULL;
+
+    if (doe_cap->status.error) {
+        return;
+    }
+
+    if (doe_cap->write_mbox[0] ==
+        DATA_OBJ_BUILD_HEADER1(PCI_VENDOR_ID_PCI_SIG, PCI_SIG_DOE_DISCOVERY)) {
+        handle_request = pcie_doe_discovery;
+    } else {
+        for (p = 0; p < doe_cap->protocol_num - 1; p++) {
+            if (doe_cap->write_mbox[0] ==
+                pcie_doe_build_protocol(&doe_cap->protocols[p])) {
+                handle_request = doe_cap->protocols[p].handle_request;
+                break;
+            }
+        }
+    }
+
+    /*
+     * PCIe r6 DOE 6.30.1:
+     * If the number of DW transferred does not match the
+     * indicated Length for a data object, then the
+     * data object must be silently discarded.
+     */
+    if (handle_request && (doe_cap->write_mbox_len ==
+        pcie_doe_get_obj_len(pcie_doe_get_write_mbox_ptr(doe_cap)))) {
+        success = handle_request(doe_cap);
+    }
+
+    if (success) {
+        pcie_doe_set_ready(doe_cap, 1);
+    } else {
+        pcie_doe_reset_mbox(doe_cap);
+    }
+}
+
+/*
+ * Read from DOE config space.
+ * Return false if the address not within DOE_CAP range.
+ */
+bool pcie_doe_read_config(DOECap *doe_cap, uint32_t addr, int size,
+                          uint32_t *buf)
+{
+    uint32_t shift;
+    uint16_t doe_offset = doe_cap->offset;
+
+    if (!range_covers_byte(doe_offset + PCI_EXP_DOE_CAP,
+                           PCI_DOE_SIZEOF - 4, addr)) {
+        return false;
+    }
+
+    addr -= doe_offset;
+    *buf = 0;
+
+    if (range_covers_byte(PCI_EXP_DOE_CAP, DWORD_BYTE, addr)) {
+        *buf = FIELD_DP32(*buf, PCI_DOE_CAP_REG, INTR_SUPP,
+                          doe_cap->cap.intr);
+        *buf = FIELD_DP32(*buf, PCI_DOE_CAP_REG, DOE_INTR_MSG_NUM,
+                          doe_cap->cap.vec);
+    } else if (range_covers_byte(PCI_EXP_DOE_CTRL, DWORD_BYTE, addr)) {
+        /* Must return ABORT=0 and GO=0 */
+        *buf = FIELD_DP32(*buf, PCI_DOE_CAP_CONTROL, DOE_INTR_EN,
+                          doe_cap->ctrl.intr);
+    } else if (range_covers_byte(PCI_EXP_DOE_STATUS, DWORD_BYTE, addr)) {
+        *buf = FIELD_DP32(*buf, PCI_DOE_CAP_STATUS, DOE_BUSY,
+                          doe_cap->status.busy);
+        *buf = FIELD_DP32(*buf, PCI_DOE_CAP_STATUS, DOE_INTR_STATUS,
+                          doe_cap->status.intr);
+        *buf = FIELD_DP32(*buf, PCI_DOE_CAP_STATUS, DOE_ERROR,
+                          doe_cap->status.error);
+        *buf = FIELD_DP32(*buf, PCI_DOE_CAP_STATUS, DATA_OBJ_RDY,
+                          doe_cap->status.ready);
+    /* Mailbox should be DW accessed */
+    } else if (addr == PCI_EXP_DOE_RD_DATA_MBOX && size == DWORD_BYTE) {
+        if (doe_cap->status.ready && !doe_cap->status.error) {
+            *buf = doe_cap->read_mbox[doe_cap->read_mbox_idx];
+        }
+    }
+
+    /* Process Alignment */
+    shift = addr % DWORD_BYTE;
+    *buf = extract32(*buf, shift * 8, size * 8);
+
+    return true;
+}
+
+/*
+ * Write to DOE config space.
+ * Return if the address not within DOE_CAP range or receives an abort
+ */
+void pcie_doe_write_config(DOECap *doe_cap,
+                           uint32_t addr, uint32_t val, int size)
+{
+    uint16_t doe_offset = doe_cap->offset;
+    uint32_t shift;
+
+    if (!range_covers_byte(doe_offset + PCI_EXP_DOE_CAP,
+                           PCI_DOE_SIZEOF - 4, addr)) {
+        return;
+    }
+
+    /* Process Alignment */
+    shift = addr % DWORD_BYTE;
+    addr -= (doe_offset + shift);
+    val = deposit32(val, shift * 8, size * 8, val);
+
+    switch (addr) {
+    case PCI_EXP_DOE_CTRL:
+        if (FIELD_EX32(val, PCI_DOE_CAP_CONTROL, DOE_ABORT)) {
+            pcie_doe_set_ready(doe_cap, 0);
+            pcie_doe_set_error(doe_cap, 0);
+            pcie_doe_reset_mbox(doe_cap);
+            return;
+        }
+
+        if (FIELD_EX32(val, PCI_DOE_CAP_CONTROL, DOE_GO)) {
+            pcie_doe_prepare_rsp(doe_cap);
+        }
+
+        if (FIELD_EX32(val, PCI_DOE_CAP_CONTROL, DOE_INTR_EN)) {
+            doe_cap->ctrl.intr = 1;
+        /* Clear interrupt bit located within the first byte */
+        } else if (shift == 0) {
+            doe_cap->ctrl.intr = 0;
+        }
+        break;
+    case PCI_EXP_DOE_STATUS:
+        if (FIELD_EX32(val, PCI_DOE_CAP_STATUS, DOE_INTR_STATUS)) {
+            doe_cap->status.intr = 0;
+        }
+        break;
+    case PCI_EXP_DOE_RD_DATA_MBOX:
+        /* Mailbox should be DW accessed */
+        if (size != DWORD_BYTE) {
+            return;
+        }
+        doe_cap->read_mbox_idx++;
+        if (doe_cap->read_mbox_idx == doe_cap->read_mbox_len) {
+            pcie_doe_reset_mbox(doe_cap);
+            pcie_doe_set_ready(doe_cap, 0);
+        } else if (doe_cap->read_mbox_idx > doe_cap->read_mbox_len) {
+            /* Underflow */
+            pcie_doe_set_error(doe_cap, 1);
+        }
+        break;
+    case PCI_EXP_DOE_WR_DATA_MBOX:
+        /* Mailbox should be DW accessed */
+        if (size != DWORD_BYTE) {
+            return;
+        }
+        doe_cap->write_mbox[doe_cap->write_mbox_len] = val;
+        doe_cap->write_mbox_len++;
+        break;
+    case PCI_EXP_DOE_CAP:
+        /* fallthrough */
+    default:
+        break;
+    }
+}
diff --git a/MAINTAINERS b/MAINTAINERS
index 07df572adf..8b7d49b089 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1835,6 +1835,13 @@ F: qapi/pci.json
 F: docs/pci*
 F: docs/specs/*pci*
 
+PCIE DOE
+M: Huai-Cheng Kuo <hchkuo@avery-design.com.tw>
+M: Chris Browy <cbrowy@avery-design.com>
+S: Supported
+F: include/hw/pci/pcie_doe.h
+F: hw/pci/pcie_doe.c
+
 ACPI/SMBIOS
 M: Michael S. Tsirkin <mst@redhat.com>
 M: Igor Mammedov <imammedo@redhat.com>
diff --git a/hw/pci/meson.build b/hw/pci/meson.build
index bcc9c75919..5aff7ed1c6 100644
--- a/hw/pci/meson.build
+++ b/hw/pci/meson.build
@@ -13,6 +13,7 @@ pci_ss.add(files(
 # allow plugging PCIe devices into PCI buses, include them even if
 # CONFIG_PCI_EXPRESS=n.
 pci_ss.add(files('pcie.c', 'pcie_aer.c'))
+pci_ss.add(files('pcie_doe.c'))
 softmmu_ss.add(when: 'CONFIG_PCI_EXPRESS', if_true: files('pcie_port.c', 'pcie_host.c'))
 softmmu_ss.add_all(when: 'CONFIG_PCI', if_true: pci_ss)
 
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 22/83] hw/mem/cxl-type3: Add MSIX support
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (20 preceding siblings ...)
  2022-11-07 22:49 ` [PULL v4 21/83] hw/pci: PCIe Data Object Exchange emulation Michael S. Tsirkin
@ 2022-11-07 22:49 ` Michael S. Tsirkin
  2022-11-07 22:49 ` [PULL v4 23/83] hw/cxl/cdat: CXL CDAT Data Object Exchange implementation Michael S. Tsirkin
                   ` (62 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:49 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Jonathan Cameron, Ben Widawsky

From: Jonathan Cameron <Jonathan.Cameron@huawei.com>

This will be used by several upcoming patch sets so break it out
such that it doesn't matter which one lands first.

Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Message-Id: <20221014151045.24781-3-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/mem/cxl_type3.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index a71bf1afeb..568c9d62f5 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -13,6 +13,7 @@
 #include "qemu/rcu.h"
 #include "sysemu/hostmem.h"
 #include "hw/cxl/cxl.h"
+#include "hw/pci/msix.h"
 
 /*
  * Null value of all Fs suggested by IEEE RA guidelines for use of
@@ -146,6 +147,8 @@ static void ct3_realize(PCIDevice *pci_dev, Error **errp)
     ComponentRegisters *regs = &cxl_cstate->crb;
     MemoryRegion *mr = &regs->component_registers;
     uint8_t *pci_conf = pci_dev->config;
+    unsigned short msix_num = 1;
+    int i;
 
     if (!cxl_setup_memory(ct3d, errp)) {
         return;
@@ -180,6 +183,12 @@ static void ct3_realize(PCIDevice *pci_dev, Error **errp)
                      PCI_BASE_ADDRESS_SPACE_MEMORY |
                          PCI_BASE_ADDRESS_MEM_TYPE_64,
                      &ct3d->cxl_dstate.device_registers);
+
+    /* MSI(-X) Initailization */
+    msix_init_exclusive_bar(pci_dev, msix_num, 4, NULL);
+    for (i = 0; i < msix_num; i++) {
+        msix_vector_use(pci_dev, i);
+    }
 }
 
 static void ct3_exit(PCIDevice *pci_dev)
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 23/83] hw/cxl/cdat: CXL CDAT Data Object Exchange implementation
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (21 preceding siblings ...)
  2022-11-07 22:49 ` [PULL v4 22/83] hw/mem/cxl-type3: Add MSIX support Michael S. Tsirkin
@ 2022-11-07 22:49 ` Michael S. Tsirkin
  2023-04-11 15:52   ` Peter Maydell
  2022-11-07 22:49 ` [PULL v4 24/83] hw/mem/cxl-type3: Add CXL CDAT Data Object Exchange Michael S. Tsirkin
                   ` (61 subsequent siblings)
  84 siblings, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:49 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Huai-Cheng Kuo, Chris Browy, Jonathan Cameron,
	Ben Widawsky

From: Huai-Cheng Kuo <hchkuo@avery-design.com.tw>

The Data Object Exchange implementation of CXL Coherent Device Attribute
Table (CDAT). This implementation is referring to "Coherent Device
Attribute Table Specification, Rev. 1.03, July. 2022" and "Compute
Express Link Specification, Rev. 3.0, July. 2022"

This patch adds core support that will be shared by both
end-points and switch port emulation.

Signed-off-by: Huai-Cheng Kuo <hchkuo@avery-design.com.tw>
Signed-off-by: Chris Browy <cbrowy@avery-design.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Message-Id: <20221014151045.24781-4-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/hw/cxl/cxl_cdat.h      | 165 ++++++++++++++++++++++++
 include/hw/cxl/cxl_component.h |   7 ++
 include/hw/cxl/cxl_device.h    |   3 +
 include/hw/cxl/cxl_pci.h       |   1 +
 hw/cxl/cxl-cdat.c              | 224 +++++++++++++++++++++++++++++++++
 hw/cxl/meson.build             |   1 +
 6 files changed, 401 insertions(+)
 create mode 100644 include/hw/cxl/cxl_cdat.h
 create mode 100644 hw/cxl/cxl-cdat.c

diff --git a/include/hw/cxl/cxl_cdat.h b/include/hw/cxl/cxl_cdat.h
new file mode 100644
index 0000000000..52c232e912
--- /dev/null
+++ b/include/hw/cxl/cxl_cdat.h
@@ -0,0 +1,165 @@
+/*
+ * CXL CDAT Structure
+ *
+ * Copyright (C) 2021 Avery Design Systems, Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef CXL_CDAT_H
+#define CXL_CDAT_H
+
+#include "hw/cxl/cxl_pci.h"
+
+/*
+ * Reference:
+ *   Coherent Device Attribute Table (CDAT) Specification, Rev. 1.03, July. 2022
+ *   Compute Express Link (CXL) Specification, Rev. 3.0, Aug. 2022
+ */
+/* Table Access DOE - CXL r3.0 8.1.11 */
+#define CXL_DOE_TABLE_ACCESS      2
+#define CXL_DOE_PROTOCOL_CDAT     ((CXL_DOE_TABLE_ACCESS << 16) | CXL_VENDOR_ID)
+
+/* Read Entry - CXL r3.0 8.1.11.1 */
+#define CXL_DOE_TAB_TYPE_CDAT 0
+#define CXL_DOE_TAB_ENT_MAX 0xFFFF
+
+/* Read Entry Request - CXL r3.0 8.1.11.1 Table 8-13 */
+#define CXL_DOE_TAB_REQ 0
+typedef struct CDATReq {
+    DOEHeader header;
+    uint8_t req_code;
+    uint8_t table_type;
+    uint16_t entry_handle;
+} QEMU_PACKED CDATReq;
+
+/* Read Entry Response - CXL r3.0 8.1.11.1 Table 8-14 */
+#define CXL_DOE_TAB_RSP 0
+typedef struct CDATRsp {
+    DOEHeader header;
+    uint8_t rsp_code;
+    uint8_t table_type;
+    uint16_t entry_handle;
+} QEMU_PACKED CDATRsp;
+
+/* CDAT Table Format - CDAT Table 1 */
+#define CXL_CDAT_REV 2
+typedef struct CDATTableHeader {
+    uint32_t length;
+    uint8_t revision;
+    uint8_t checksum;
+    uint8_t reserved[6];
+    uint32_t sequence;
+} QEMU_PACKED CDATTableHeader;
+
+/* CDAT Structure Types - CDAT Table 2 */
+typedef enum {
+    CDAT_TYPE_DSMAS = 0,
+    CDAT_TYPE_DSLBIS = 1,
+    CDAT_TYPE_DSMSCIS = 2,
+    CDAT_TYPE_DSIS = 3,
+    CDAT_TYPE_DSEMTS = 4,
+    CDAT_TYPE_SSLBIS = 5,
+} CDATType;
+
+typedef struct CDATSubHeader {
+    uint8_t type;
+    uint8_t reserved;
+    uint16_t length;
+} CDATSubHeader;
+
+/* Device Scoped Memory Affinity Structure - CDAT Table 3 */
+typedef struct CDATDsmas {
+    CDATSubHeader header;
+    uint8_t DSMADhandle;
+    uint8_t flags;
+#define CDAT_DSMAS_FLAG_NV              (1 << 2)
+#define CDAT_DSMAS_FLAG_SHAREABLE       (1 << 3)
+#define CDAT_DSMAS_FLAG_HW_COHERENT     (1 << 4)
+#define CDAT_DSMAS_FLAG_DYNAMIC_CAP     (1 << 5)
+    uint16_t reserved;
+    uint64_t DPA_base;
+    uint64_t DPA_length;
+} QEMU_PACKED CDATDsmas;
+
+/* Device Scoped Latency and Bandwidth Information Structure - CDAT Table 5 */
+typedef struct CDATDslbis {
+    CDATSubHeader header;
+    uint8_t handle;
+    /* Definitions of these fields refer directly to HMAT fields */
+    uint8_t flags;
+    uint8_t data_type;
+    uint8_t reserved;
+    uint64_t entry_base_unit;
+    uint16_t entry[3];
+    uint16_t reserved2;
+} QEMU_PACKED CDATDslbis;
+
+/* Device Scoped Memory Side Cache Information Structure - CDAT Table 6 */
+typedef struct CDATDsmscis {
+    CDATSubHeader header;
+    uint8_t DSMAS_handle;
+    uint8_t reserved[3];
+    uint64_t memory_side_cache_size;
+    uint32_t cache_attributes;
+} QEMU_PACKED CDATDsmscis;
+
+/* Device Scoped Initiator Structure - CDAT Table 7 */
+typedef struct CDATDsis {
+    CDATSubHeader header;
+    uint8_t flags;
+    uint8_t handle;
+    uint16_t reserved;
+} QEMU_PACKED CDATDsis;
+
+/* Device Scoped EFI Memory Type Structure - CDAT Table 8 */
+typedef struct CDATDsemts {
+    CDATSubHeader header;
+    uint8_t DSMAS_handle;
+    uint8_t EFI_memory_type_attr;
+    uint16_t reserved;
+    uint64_t DPA_offset;
+    uint64_t DPA_length;
+} QEMU_PACKED CDATDsemts;
+
+/* Switch Scoped Latency and Bandwidth Information Structure - CDAT Table 9 */
+typedef struct CDATSslbisHeader {
+    CDATSubHeader header;
+    uint8_t data_type;
+    uint8_t reserved[3];
+    uint64_t entry_base_unit;
+} QEMU_PACKED CDATSslbisHeader;
+
+/* Switch Scoped Latency and Bandwidth Entry - CDAT Table 10 */
+typedef struct CDATSslbe {
+    uint16_t port_x_id;
+    uint16_t port_y_id;
+    uint16_t latency_bandwidth;
+    uint16_t reserved;
+} QEMU_PACKED CDATSslbe;
+
+typedef struct CDATSslbis {
+    CDATSslbisHeader sslbis_header;
+    CDATSslbe sslbe[];
+} QEMU_PACKED CDATSslbis;
+
+typedef struct CDATEntry {
+    void *base;
+    uint32_t length;
+} CDATEntry;
+
+typedef struct CDATObject {
+    CDATEntry *entry;
+    int entry_len;
+
+    int (*build_cdat_table)(CDATSubHeader ***cdat_table, void *priv);
+    void (*free_cdat_table)(CDATSubHeader **cdat_table, int num, void *priv);
+    bool to_update;
+    void *private;
+    char *filename;
+    uint8_t *buf;
+    struct CDATSubHeader **built_buf;
+    int built_buf_len;
+} CDATObject;
+#endif /* CXL_CDAT_H */
diff --git a/include/hw/cxl/cxl_component.h b/include/hw/cxl/cxl_component.h
index 94ec2f07d7..34075cfb72 100644
--- a/include/hw/cxl/cxl_component.h
+++ b/include/hw/cxl/cxl_component.h
@@ -19,6 +19,7 @@
 #include "qemu/range.h"
 #include "qemu/typedefs.h"
 #include "hw/register.h"
+#include "qapi/error.h"
 
 enum reg_type {
     CXL2_DEVICE,
@@ -184,6 +185,8 @@ typedef struct cxl_component {
             struct PCIDevice *pdev;
         };
     };
+
+    CDATObject cdat;
 } CXLComponentState;
 
 void cxl_component_register_block_init(Object *obj,
@@ -220,4 +223,8 @@ static inline hwaddr cxl_decode_ig(int ig)
 
 CXLComponentState *cxl_get_hb_cstate(PCIHostState *hb);
 
+void cxl_doe_cdat_init(CXLComponentState *cxl_cstate, Error **errp);
+void cxl_doe_cdat_release(CXLComponentState *cxl_cstate);
+void cxl_doe_cdat_update(CXLComponentState *cxl_cstate, Error **errp);
+
 #endif
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index e4d221cdb3..449b0edfe9 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -243,6 +243,9 @@ struct CXLType3Dev {
     AddressSpace hostmem_as;
     CXLComponentState cxl_cstate;
     CXLDeviceState cxl_dstate;
+
+    /* DOE */
+    DOECap doe_cdat;
 };
 
 #define TYPE_CXL_TYPE3 "cxl-type3"
diff --git a/include/hw/cxl/cxl_pci.h b/include/hw/cxl/cxl_pci.h
index 01cf002096..3cb79eca1e 100644
--- a/include/hw/cxl/cxl_pci.h
+++ b/include/hw/cxl/cxl_pci.h
@@ -13,6 +13,7 @@
 #include "qemu/compiler.h"
 #include "hw/pci/pci.h"
 #include "hw/pci/pcie.h"
+#include "hw/cxl/cxl_cdat.h"
 
 #define CXL_VENDOR_ID 0x1e98
 
diff --git a/hw/cxl/cxl-cdat.c b/hw/cxl/cxl-cdat.c
new file mode 100644
index 0000000000..3653aa56f0
--- /dev/null
+++ b/hw/cxl/cxl-cdat.c
@@ -0,0 +1,224 @@
+/*
+ * CXL CDAT Structure
+ *
+ * Copyright (C) 2021 Avery Design Systems, Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "hw/pci/pci.h"
+#include "hw/cxl/cxl.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+
+static void cdat_len_check(CDATSubHeader *hdr, Error **errp)
+{
+    assert(hdr->length);
+    assert(hdr->reserved == 0);
+
+    switch (hdr->type) {
+    case CDAT_TYPE_DSMAS:
+        assert(hdr->length == sizeof(CDATDsmas));
+        break;
+    case CDAT_TYPE_DSLBIS:
+        assert(hdr->length == sizeof(CDATDslbis));
+        break;
+    case CDAT_TYPE_DSMSCIS:
+        assert(hdr->length == sizeof(CDATDsmscis));
+        break;
+    case CDAT_TYPE_DSIS:
+        assert(hdr->length == sizeof(CDATDsis));
+        break;
+    case CDAT_TYPE_DSEMTS:
+        assert(hdr->length == sizeof(CDATDsemts));
+        break;
+    case CDAT_TYPE_SSLBIS:
+        assert(hdr->length >= sizeof(CDATSslbisHeader));
+        assert((hdr->length - sizeof(CDATSslbisHeader)) %
+               sizeof(CDATSslbe) == 0);
+        break;
+    default:
+        error_setg(errp, "Type %d is reserved", hdr->type);
+    }
+}
+
+static void ct3_build_cdat(CDATObject *cdat, Error **errp)
+{
+    g_autofree CDATTableHeader *cdat_header = NULL;
+    g_autofree CDATEntry *cdat_st = NULL;
+    uint8_t sum = 0;
+    int ent, i;
+
+    /* Use default table if fopen == NULL */
+    assert(cdat->build_cdat_table);
+
+    cdat_header = g_malloc0(sizeof(*cdat_header));
+    if (!cdat_header) {
+        error_setg(errp, "Failed to allocate CDAT header");
+        return;
+    }
+
+    cdat->built_buf_len = cdat->build_cdat_table(&cdat->built_buf, cdat->private);
+
+    if (!cdat->built_buf_len) {
+        /* Build later as not all data available yet */
+        cdat->to_update = true;
+        return;
+    }
+    cdat->to_update = false;
+
+    cdat_st = g_malloc0(sizeof(*cdat_st) * (cdat->built_buf_len + 1));
+    if (!cdat_st) {
+        error_setg(errp, "Failed to allocate CDAT entry array");
+        return;
+    }
+
+    /* Entry 0 for CDAT header, starts with Entry 1 */
+    for (ent = 1; ent < cdat->built_buf_len + 1; ent++) {
+        CDATSubHeader *hdr = cdat->built_buf[ent - 1];
+        uint8_t *buf = (uint8_t *)cdat->built_buf[ent - 1];
+
+        cdat_st[ent].base = hdr;
+        cdat_st[ent].length = hdr->length;
+
+        cdat_header->length += hdr->length;
+        for (i = 0; i < hdr->length; i++) {
+            sum += buf[i];
+        }
+    }
+
+    /* CDAT header */
+    cdat_header->revision = CXL_CDAT_REV;
+    /* For now, no runtime updates */
+    cdat_header->sequence = 0;
+    cdat_header->length += sizeof(CDATTableHeader);
+    sum += cdat_header->revision + cdat_header->sequence +
+        cdat_header->length;
+    /* Sum of all bytes including checksum must be 0 */
+    cdat_header->checksum = ~sum + 1;
+
+    cdat_st[0].base = g_steal_pointer(&cdat_header);
+    cdat_st[0].length = sizeof(*cdat_header);
+    cdat->entry_len = 1 + cdat->built_buf_len;
+    cdat->entry = g_steal_pointer(&cdat_st);
+}
+
+static void ct3_load_cdat(CDATObject *cdat, Error **errp)
+{
+    g_autofree CDATEntry *cdat_st = NULL;
+    uint8_t sum = 0;
+    int num_ent;
+    int i = 0, ent = 1, file_size = 0;
+    CDATSubHeader *hdr;
+    FILE *fp = NULL;
+
+    /* Read CDAT file and create its cache */
+    fp = fopen(cdat->filename, "r");
+    if (!fp) {
+        error_setg(errp, "CDAT: Unable to open file");
+        return;
+    }
+
+    fseek(fp, 0, SEEK_END);
+    file_size = ftell(fp);
+    fseek(fp, 0, SEEK_SET);
+    cdat->buf = g_malloc0(file_size);
+
+    if (fread(cdat->buf, file_size, 1, fp) == 0) {
+        error_setg(errp, "CDAT: File read failed");
+        return;
+    }
+
+    fclose(fp);
+
+    if (file_size < sizeof(CDATTableHeader)) {
+        error_setg(errp, "CDAT: File too short");
+        return;
+    }
+    i = sizeof(CDATTableHeader);
+    num_ent = 1;
+    while (i < file_size) {
+        hdr = (CDATSubHeader *)(cdat->buf + i);
+        cdat_len_check(hdr, errp);
+        i += hdr->length;
+        num_ent++;
+    }
+    if (i != file_size) {
+        error_setg(errp, "CDAT: File length missmatch");
+        return;
+    }
+
+    cdat_st = g_malloc0(sizeof(*cdat_st) * num_ent);
+    if (!cdat_st) {
+        error_setg(errp, "CDAT: Failed to allocate entry array");
+        return;
+    }
+
+    /* Set CDAT header, Entry = 0 */
+    cdat_st[0].base = cdat->buf;
+    cdat_st[0].length = sizeof(CDATTableHeader);
+    i = 0;
+
+    while (i < cdat_st[0].length) {
+        sum += cdat->buf[i++];
+    }
+
+    /* Read CDAT structures */
+    while (i < file_size) {
+        hdr = (CDATSubHeader *)(cdat->buf + i);
+        cdat_len_check(hdr, errp);
+
+        cdat_st[ent].base = hdr;
+        cdat_st[ent].length = hdr->length;
+
+        while (cdat->buf + i <
+               (uint8_t *)cdat_st[ent].base + cdat_st[ent].length) {
+            assert(i < file_size);
+            sum += cdat->buf[i++];
+        }
+
+        ent++;
+    }
+
+    if (sum != 0) {
+        warn_report("CDAT: Found checksum mismatch in %s", cdat->filename);
+    }
+    cdat->entry_len = num_ent;
+    cdat->entry = g_steal_pointer(&cdat_st);
+}
+
+void cxl_doe_cdat_init(CXLComponentState *cxl_cstate, Error **errp)
+{
+    CDATObject *cdat = &cxl_cstate->cdat;
+
+    if (cdat->filename) {
+        ct3_load_cdat(cdat, errp);
+    } else {
+        ct3_build_cdat(cdat, errp);
+    }
+}
+
+void cxl_doe_cdat_update(CXLComponentState *cxl_cstate, Error **errp)
+{
+    CDATObject *cdat = &cxl_cstate->cdat;
+
+    if (cdat->to_update) {
+        ct3_build_cdat(cdat, errp);
+    }
+}
+
+void cxl_doe_cdat_release(CXLComponentState *cxl_cstate)
+{
+    CDATObject *cdat = &cxl_cstate->cdat;
+
+    free(cdat->entry);
+    if (cdat->built_buf) {
+        cdat->free_cdat_table(cdat->built_buf, cdat->built_buf_len,
+                              cdat->private);
+    }
+    if (cdat->buf) {
+        free(cdat->buf);
+    }
+}
diff --git a/hw/cxl/meson.build b/hw/cxl/meson.build
index f117b99949..cfa95ffd40 100644
--- a/hw/cxl/meson.build
+++ b/hw/cxl/meson.build
@@ -4,6 +4,7 @@ softmmu_ss.add(when: 'CONFIG_CXL',
                    'cxl-device-utils.c',
                    'cxl-mailbox-utils.c',
                    'cxl-host.c',
+                   'cxl-cdat.c',
                ),
                if_false: files(
                    'cxl-host-stubs.c',
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 24/83] hw/mem/cxl-type3: Add CXL CDAT Data Object Exchange
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (22 preceding siblings ...)
  2022-11-07 22:49 ` [PULL v4 23/83] hw/cxl/cdat: CXL CDAT Data Object Exchange implementation Michael S. Tsirkin
@ 2022-11-07 22:49 ` Michael S. Tsirkin
  2022-11-07 22:49 ` [PULL v4 25/83] hw/pci-bridge/cxl-upstream: Add a CDAT table access DOE Michael S. Tsirkin
                   ` (60 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:49 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Huai-Cheng Kuo, Chris Browy, Jonathan Cameron,
	Ben Widawsky

From: Huai-Cheng Kuo <hchkuo@avery-design.com.tw>

The CDAT can be specified in two ways. One is to add ",cdat=<filename>"
in "-device cxl-type3"'s command option. The file is required to provide
the whole CDAT table in binary mode. The other is to use the default
that provides some 'reasonable' numbers based on type of memory and
size.

The DOE capability supporting CDAT is added to hw/mem/cxl_type3.c with
capability offset 0x190. The config read/write to this capability range
can be generated in the OS to request the CDAT data.

Signed-off-by: Huai-Cheng Kuo <hchkuo@avery-design.com.tw>
Signed-off-by: Chris Browy <cbrowy@avery-design.com>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>

Message-Id: <20221014151045.24781-5-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/mem/cxl_type3.c | 255 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 255 insertions(+)

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 568c9d62f5..255590201a 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -12,9 +12,246 @@
 #include "qemu/range.h"
 #include "qemu/rcu.h"
 #include "sysemu/hostmem.h"
+#include "sysemu/numa.h"
 #include "hw/cxl/cxl.h"
 #include "hw/pci/msix.h"
 
+#define DWORD_BYTE 4
+
+/* Default CDAT entries for a memory region */
+enum {
+    CT3_CDAT_DSMAS,
+    CT3_CDAT_DSLBIS0,
+    CT3_CDAT_DSLBIS1,
+    CT3_CDAT_DSLBIS2,
+    CT3_CDAT_DSLBIS3,
+    CT3_CDAT_DSEMTS,
+    CT3_CDAT_NUM_ENTRIES
+};
+
+static int ct3_build_cdat_entries_for_mr(CDATSubHeader **cdat_table,
+                                         int dsmad_handle, MemoryRegion *mr)
+{
+    g_autofree CDATDsmas *dsmas = NULL;
+    g_autofree CDATDslbis *dslbis0 = NULL;
+    g_autofree CDATDslbis *dslbis1 = NULL;
+    g_autofree CDATDslbis *dslbis2 = NULL;
+    g_autofree CDATDslbis *dslbis3 = NULL;
+    g_autofree CDATDsemts *dsemts = NULL;
+
+    dsmas = g_malloc(sizeof(*dsmas));
+    if (!dsmas) {
+        return -ENOMEM;
+    }
+    *dsmas = (CDATDsmas) {
+        .header = {
+            .type = CDAT_TYPE_DSMAS,
+            .length = sizeof(*dsmas),
+        },
+        .DSMADhandle = dsmad_handle,
+        .flags = CDAT_DSMAS_FLAG_NV,
+        .DPA_base = 0,
+        .DPA_length = int128_get64(mr->size),
+    };
+
+    /* For now, no memory side cache, plausiblish numbers */
+    dslbis0 = g_malloc(sizeof(*dslbis0));
+    if (!dslbis0) {
+        return -ENOMEM;
+    }
+    *dslbis0 = (CDATDslbis) {
+        .header = {
+            .type = CDAT_TYPE_DSLBIS,
+            .length = sizeof(*dslbis0),
+        },
+        .handle = dsmad_handle,
+        .flags = HMAT_LB_MEM_MEMORY,
+        .data_type = HMAT_LB_DATA_READ_LATENCY,
+        .entry_base_unit = 10000, /* 10ns base */
+        .entry[0] = 15, /* 150ns */
+    };
+
+    dslbis1 = g_malloc(sizeof(*dslbis1));
+    if (!dslbis1) {
+        return -ENOMEM;
+    }
+    *dslbis1 = (CDATDslbis) {
+        .header = {
+            .type = CDAT_TYPE_DSLBIS,
+            .length = sizeof(*dslbis1),
+        },
+        .handle = dsmad_handle,
+        .flags = HMAT_LB_MEM_MEMORY,
+        .data_type = HMAT_LB_DATA_WRITE_LATENCY,
+        .entry_base_unit = 10000,
+        .entry[0] = 25, /* 250ns */
+    };
+
+    dslbis2 = g_malloc(sizeof(*dslbis2));
+    if (!dslbis2) {
+        return -ENOMEM;
+    }
+    *dslbis2 = (CDATDslbis) {
+        .header = {
+            .type = CDAT_TYPE_DSLBIS,
+            .length = sizeof(*dslbis2),
+        },
+        .handle = dsmad_handle,
+        .flags = HMAT_LB_MEM_MEMORY,
+        .data_type = HMAT_LB_DATA_READ_BANDWIDTH,
+        .entry_base_unit = 1000, /* GB/s */
+        .entry[0] = 16,
+    };
+
+    dslbis3 = g_malloc(sizeof(*dslbis3));
+    if (!dslbis3) {
+        return -ENOMEM;
+    }
+    *dslbis3 = (CDATDslbis) {
+        .header = {
+            .type = CDAT_TYPE_DSLBIS,
+            .length = sizeof(*dslbis3),
+        },
+        .handle = dsmad_handle,
+        .flags = HMAT_LB_MEM_MEMORY,
+        .data_type = HMAT_LB_DATA_WRITE_BANDWIDTH,
+        .entry_base_unit = 1000, /* GB/s */
+        .entry[0] = 16,
+    };
+
+    dsemts = g_malloc(sizeof(*dsemts));
+    if (!dsemts) {
+        return -ENOMEM;
+    }
+    *dsemts = (CDATDsemts) {
+        .header = {
+            .type = CDAT_TYPE_DSEMTS,
+            .length = sizeof(*dsemts),
+        },
+        .DSMAS_handle = dsmad_handle,
+        /* Reserved - the non volatile from DSMAS matters */
+        .EFI_memory_type_attr = 2,
+        .DPA_offset = 0,
+        .DPA_length = int128_get64(mr->size),
+    };
+
+    /* Header always at start of structure */
+    cdat_table[CT3_CDAT_DSMAS] = g_steal_pointer(&dsmas);
+    cdat_table[CT3_CDAT_DSLBIS0] = g_steal_pointer(&dslbis0);
+    cdat_table[CT3_CDAT_DSLBIS1] = g_steal_pointer(&dslbis1);
+    cdat_table[CT3_CDAT_DSLBIS2] = g_steal_pointer(&dslbis2);
+    cdat_table[CT3_CDAT_DSLBIS3] = g_steal_pointer(&dslbis3);
+    cdat_table[CT3_CDAT_DSEMTS] = g_steal_pointer(&dsemts);
+
+    return 0;
+}
+
+static int ct3_build_cdat_table(CDATSubHeader ***cdat_table, void *priv)
+{
+    g_autofree CDATSubHeader **table = NULL;
+    MemoryRegion *nonvolatile_mr;
+    CXLType3Dev *ct3d = priv;
+    int dsmad_handle = 0;
+    int rc;
+
+    if (!ct3d->hostmem) {
+        return 0;
+    }
+
+    nonvolatile_mr = host_memory_backend_get_memory(ct3d->hostmem);
+    if (!nonvolatile_mr) {
+        return -EINVAL;
+    }
+
+    table = g_malloc0(CT3_CDAT_NUM_ENTRIES * sizeof(*table));
+    if (!table) {
+        return -ENOMEM;
+    }
+
+    rc = ct3_build_cdat_entries_for_mr(table, dsmad_handle++, nonvolatile_mr);
+    if (rc < 0) {
+        return rc;
+    }
+
+    *cdat_table = g_steal_pointer(&table);
+
+    return CT3_CDAT_NUM_ENTRIES;
+}
+
+static void ct3_free_cdat_table(CDATSubHeader **cdat_table, int num, void *priv)
+{
+    int i;
+
+    for (i = 0; i < num; i++) {
+        g_free(cdat_table[i]);
+    }
+    g_free(cdat_table);
+}
+
+static bool cxl_doe_cdat_rsp(DOECap *doe_cap)
+{
+    CDATObject *cdat = &CXL_TYPE3(doe_cap->pdev)->cxl_cstate.cdat;
+    uint16_t ent;
+    void *base;
+    uint32_t len;
+    CDATReq *req = pcie_doe_get_write_mbox_ptr(doe_cap);
+    CDATRsp rsp;
+
+    assert(cdat->entry_len);
+
+    /* Discard if request length mismatched */
+    if (pcie_doe_get_obj_len(req) <
+        DIV_ROUND_UP(sizeof(CDATReq), DWORD_BYTE)) {
+        return false;
+    }
+
+    ent = req->entry_handle;
+    base = cdat->entry[ent].base;
+    len = cdat->entry[ent].length;
+
+    rsp = (CDATRsp) {
+        .header = {
+            .vendor_id = CXL_VENDOR_ID,
+            .data_obj_type = CXL_DOE_TABLE_ACCESS,
+            .reserved = 0x0,
+            .length = DIV_ROUND_UP((sizeof(rsp) + len), DWORD_BYTE),
+        },
+        .rsp_code = CXL_DOE_TAB_RSP,
+        .table_type = CXL_DOE_TAB_TYPE_CDAT,
+        .entry_handle = (ent < cdat->entry_len - 1) ?
+                        ent + 1 : CXL_DOE_TAB_ENT_MAX,
+    };
+
+    memcpy(doe_cap->read_mbox, &rsp, sizeof(rsp));
+    memcpy(doe_cap->read_mbox + DIV_ROUND_UP(sizeof(rsp), DWORD_BYTE),
+           base, len);
+
+    doe_cap->read_mbox_len += rsp.header.length;
+
+    return true;
+}
+
+static uint32_t ct3d_config_read(PCIDevice *pci_dev, uint32_t addr, int size)
+{
+    CXLType3Dev *ct3d = CXL_TYPE3(pci_dev);
+    uint32_t val;
+
+    if (pcie_doe_read_config(&ct3d->doe_cdat, addr, size, &val)) {
+        return val;
+    }
+
+    return pci_default_read_config(pci_dev, addr, size);
+}
+
+static void ct3d_config_write(PCIDevice *pci_dev, uint32_t addr, uint32_t val,
+                              int size)
+{
+    CXLType3Dev *ct3d = CXL_TYPE3(pci_dev);
+
+    pcie_doe_write_config(&ct3d->doe_cdat, addr, val, size);
+    pci_default_write_config(pci_dev, addr, val, size);
+}
+
 /*
  * Null value of all Fs suggested by IEEE RA guidelines for use of
  * EU, OUI and CID
@@ -140,6 +377,11 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
     return true;
 }
 
+static DOEProtocol doe_cdat_prot[] = {
+    { CXL_VENDOR_ID, CXL_DOE_TABLE_ACCESS, cxl_doe_cdat_rsp },
+    { }
+};
+
 static void ct3_realize(PCIDevice *pci_dev, Error **errp)
 {
     CXLType3Dev *ct3d = CXL_TYPE3(pci_dev);
@@ -189,6 +431,14 @@ static void ct3_realize(PCIDevice *pci_dev, Error **errp)
     for (i = 0; i < msix_num; i++) {
         msix_vector_use(pci_dev, i);
     }
+
+    /* DOE Initailization */
+    pcie_doe_init(pci_dev, &ct3d->doe_cdat, 0x190, doe_cdat_prot, true, 0);
+
+    cxl_cstate->cdat.build_cdat_table = ct3_build_cdat_table;
+    cxl_cstate->cdat.free_cdat_table = ct3_free_cdat_table;
+    cxl_cstate->cdat.private = ct3d;
+    cxl_doe_cdat_init(cxl_cstate, errp);
 }
 
 static void ct3_exit(PCIDevice *pci_dev)
@@ -197,6 +447,7 @@ static void ct3_exit(PCIDevice *pci_dev)
     CXLComponentState *cxl_cstate = &ct3d->cxl_cstate;
     ComponentRegisters *regs = &cxl_cstate->crb;
 
+    cxl_doe_cdat_release(cxl_cstate);
     g_free(regs->special_ops);
     address_space_destroy(&ct3d->hostmem_as);
 }
@@ -296,6 +547,7 @@ static Property ct3_props[] = {
     DEFINE_PROP_LINK("lsa", CXLType3Dev, lsa, TYPE_MEMORY_BACKEND,
                      HostMemoryBackend *),
     DEFINE_PROP_UINT64("sn", CXLType3Dev, sn, UI64_NULL),
+    DEFINE_PROP_STRING("cdat", CXLType3Dev, cxl_cstate.cdat.filename),
     DEFINE_PROP_END_OF_LIST(),
 };
 
@@ -361,6 +613,9 @@ static void ct3_class_init(ObjectClass *oc, void *data)
     pc->device_id = 0xd93; /* LVF for now */
     pc->revision = 1;
 
+    pc->config_write = ct3d_config_write;
+    pc->config_read = ct3d_config_read;
+
     set_bit(DEVICE_CATEGORY_STORAGE, dc->categories);
     dc->desc = "CXL PMEM Device (Type 3)";
     dc->reset = ct3d_reset;
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 25/83] hw/pci-bridge/cxl-upstream: Add a CDAT table access DOE
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (23 preceding siblings ...)
  2022-11-07 22:49 ` [PULL v4 24/83] hw/mem/cxl-type3: Add CXL CDAT Data Object Exchange Michael S. Tsirkin
@ 2022-11-07 22:49 ` Michael S. Tsirkin
  2023-06-23 13:23   ` Peter Maydell
  2022-11-07 22:49 ` [PULL v4 26/83] hw/virtio/virtio-iommu-pci: Enforce the device is plugged on the root bus Michael S. Tsirkin
                   ` (59 subsequent siblings)
  84 siblings, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:49 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Jonathan Cameron, Marcel Apfelbaum, Ben Widawsky

From: Jonathan Cameron <Jonathan.Cameron@huawei.com>

This Data Object Exchange Mailbox allows software to query the
latency and bandwidth between ports on the switch. For now
only provide information on routes between the upstream port and
each downstream port (not p2p).

Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>

--
Changes since v8: Mostly to match the type 3 equivalent
 - Move enum out of function and give it a more descriptive namespace.
Message-Id: <20221014151045.24781-6-Jonathan.Cameron@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/hw/cxl/cxl_cdat.h    |   1 +
 hw/pci-bridge/cxl_upstream.c | 195 ++++++++++++++++++++++++++++++++++-
 2 files changed, 195 insertions(+), 1 deletion(-)

diff --git a/include/hw/cxl/cxl_cdat.h b/include/hw/cxl/cxl_cdat.h
index 52c232e912..e9eda00142 100644
--- a/include/hw/cxl/cxl_cdat.h
+++ b/include/hw/cxl/cxl_cdat.h
@@ -131,6 +131,7 @@ typedef struct CDATSslbisHeader {
     uint64_t entry_base_unit;
 } QEMU_PACKED CDATSslbisHeader;
 
+#define CDAT_PORT_ID_USP 0x100
 /* Switch Scoped Latency and Bandwidth Entry - CDAT Table 10 */
 typedef struct CDATSslbe {
     uint16_t port_x_id;
diff --git a/hw/pci-bridge/cxl_upstream.c b/hw/pci-bridge/cxl_upstream.c
index a83a3e81e4..9b8b57df9d 100644
--- a/hw/pci-bridge/cxl_upstream.c
+++ b/hw/pci-bridge/cxl_upstream.c
@@ -10,11 +10,12 @@
 
 #include "qemu/osdep.h"
 #include "qemu/log.h"
+#include "hw/qdev-properties.h"
 #include "hw/pci/msi.h"
 #include "hw/pci/pcie.h"
 #include "hw/pci/pcie_port.h"
 
-#define CXL_UPSTREAM_PORT_MSI_NR_VECTOR 1
+#define CXL_UPSTREAM_PORT_MSI_NR_VECTOR 2
 
 #define CXL_UPSTREAM_PORT_MSI_OFFSET 0x70
 #define CXL_UPSTREAM_PORT_PCIE_CAP_OFFSET 0x90
@@ -28,6 +29,7 @@ typedef struct CXLUpstreamPort {
 
     /*< public >*/
     CXLComponentState cxl_cstate;
+    DOECap doe_cdat;
 } CXLUpstreamPort;
 
 CXLComponentState *cxl_usp_to_cstate(CXLUpstreamPort *usp)
@@ -60,6 +62,9 @@ static void cxl_usp_dvsec_write_config(PCIDevice *dev, uint32_t addr,
 static void cxl_usp_write_config(PCIDevice *d, uint32_t address,
                                  uint32_t val, int len)
 {
+    CXLUpstreamPort *usp = CXL_USP(d);
+
+    pcie_doe_write_config(&usp->doe_cdat, address, val, len);
     pci_bridge_write_config(d, address, val, len);
     pcie_cap_flr_write_config(d, address, val, len);
     pcie_aer_write_config(d, address, val, len);
@@ -67,6 +72,18 @@ static void cxl_usp_write_config(PCIDevice *d, uint32_t address,
     cxl_usp_dvsec_write_config(d, address, val, len);
 }
 
+static uint32_t cxl_usp_read_config(PCIDevice *d, uint32_t address, int len)
+{
+    CXLUpstreamPort *usp = CXL_USP(d);
+    uint32_t val;
+
+    if (pcie_doe_read_config(&usp->doe_cdat, address, len, &val)) {
+        return val;
+    }
+
+    return pci_default_read_config(d, address, len);
+}
+
 static void latch_registers(CXLUpstreamPort *usp)
 {
     uint32_t *reg_state = usp->cxl_cstate.crb.cache_mem_registers;
@@ -119,6 +136,167 @@ static void build_dvsecs(CXLComponentState *cxl)
                                REG_LOC_DVSEC_REVID, dvsec);
 }
 
+static bool cxl_doe_cdat_rsp(DOECap *doe_cap)
+{
+    CDATObject *cdat = &CXL_USP(doe_cap->pdev)->cxl_cstate.cdat;
+    uint16_t ent;
+    void *base;
+    uint32_t len;
+    CDATReq *req = pcie_doe_get_write_mbox_ptr(doe_cap);
+    CDATRsp rsp;
+
+    cxl_doe_cdat_update(&CXL_USP(doe_cap->pdev)->cxl_cstate, &error_fatal);
+    assert(cdat->entry_len);
+
+    /* Discard if request length mismatched */
+    if (pcie_doe_get_obj_len(req) <
+        DIV_ROUND_UP(sizeof(CDATReq), sizeof(uint32_t))) {
+        return false;
+    }
+
+    ent = req->entry_handle;
+    base = cdat->entry[ent].base;
+    len = cdat->entry[ent].length;
+
+    rsp = (CDATRsp) {
+        .header = {
+            .vendor_id = CXL_VENDOR_ID,
+            .data_obj_type = CXL_DOE_TABLE_ACCESS,
+            .reserved = 0x0,
+            .length = DIV_ROUND_UP((sizeof(rsp) + len), sizeof(uint32_t)),
+        },
+        .rsp_code = CXL_DOE_TAB_RSP,
+        .table_type = CXL_DOE_TAB_TYPE_CDAT,
+        .entry_handle = (ent < cdat->entry_len - 1) ?
+                        ent + 1 : CXL_DOE_TAB_ENT_MAX,
+    };
+
+    memcpy(doe_cap->read_mbox, &rsp, sizeof(rsp));
+        memcpy(doe_cap->read_mbox + DIV_ROUND_UP(sizeof(rsp), sizeof(uint32_t)),
+           base, len);
+
+    doe_cap->read_mbox_len += rsp.header.length;
+
+    return true;
+}
+
+static DOEProtocol doe_cdat_prot[] = {
+    { CXL_VENDOR_ID, CXL_DOE_TABLE_ACCESS, cxl_doe_cdat_rsp },
+    { }
+};
+
+enum {
+    CXL_USP_CDAT_SSLBIS_LAT,
+    CXL_USP_CDAT_SSLBIS_BW,
+    CXL_USP_CDAT_NUM_ENTRIES
+};
+
+static int build_cdat_table(CDATSubHeader ***cdat_table, void *priv)
+{
+    g_autofree CDATSslbis *sslbis_latency = NULL;
+    g_autofree CDATSslbis *sslbis_bandwidth = NULL;
+    CXLUpstreamPort *us = CXL_USP(priv);
+    PCIBus *bus = &PCI_BRIDGE(us)->sec_bus;
+    int devfn, sslbis_size, i;
+    int count = 0;
+    uint16_t port_ids[256];
+
+    for (devfn = 0; devfn < ARRAY_SIZE(bus->devices); devfn++) {
+        PCIDevice *d = bus->devices[devfn];
+        PCIEPort *port;
+
+        if (!d || !pci_is_express(d) || !d->exp.exp_cap) {
+            continue;
+        }
+
+        /*
+         * Whilst the PCI express spec doesn't allow anything other than
+         * downstream ports on this bus, let us be a little paranoid
+         */
+        if (!object_dynamic_cast(OBJECT(d), TYPE_PCIE_PORT)) {
+            continue;
+        }
+
+        port = PCIE_PORT(d);
+        port_ids[count] = port->port;
+        count++;
+    }
+
+    /* May not yet have any ports - try again later */
+    if (count == 0) {
+        return 0;
+    }
+
+    sslbis_size = sizeof(CDATSslbis) + sizeof(*sslbis_latency->sslbe) * count;
+    sslbis_latency = g_malloc(sslbis_size);
+    if (!sslbis_latency) {
+        return -ENOMEM;
+    }
+    *sslbis_latency = (CDATSslbis) {
+        .sslbis_header = {
+            .header = {
+                .type = CDAT_TYPE_SSLBIS,
+                .length = sslbis_size,
+            },
+            .data_type = HMATLB_DATA_TYPE_ACCESS_LATENCY,
+            .entry_base_unit = 10000,
+        },
+    };
+
+    for (i = 0; i < count; i++) {
+        sslbis_latency->sslbe[i] = (CDATSslbe) {
+            .port_x_id = CDAT_PORT_ID_USP,
+            .port_y_id = port_ids[i],
+            .latency_bandwidth = 15, /* 150ns */
+        };
+    }
+
+    sslbis_bandwidth = g_malloc(sslbis_size);
+    if (!sslbis_bandwidth) {
+        return 0;
+    }
+    *sslbis_bandwidth = (CDATSslbis) {
+        .sslbis_header = {
+            .header = {
+                .type = CDAT_TYPE_SSLBIS,
+                .length = sslbis_size,
+            },
+            .data_type = HMATLB_DATA_TYPE_ACCESS_BANDWIDTH,
+            .entry_base_unit = 1000,
+        },
+    };
+
+    for (i = 0; i < count; i++) {
+        sslbis_bandwidth->sslbe[i] = (CDATSslbe) {
+            .port_x_id = CDAT_PORT_ID_USP,
+            .port_y_id = port_ids[i],
+            .latency_bandwidth = 16, /* 16 GB/s */
+        };
+    }
+
+    *cdat_table = g_malloc0(sizeof(*cdat_table) * CXL_USP_CDAT_NUM_ENTRIES);
+    if (!*cdat_table) {
+        return -ENOMEM;
+    }
+
+    /* Header always at start of structure */
+    (*cdat_table)[CXL_USP_CDAT_SSLBIS_LAT] = g_steal_pointer(&sslbis_latency);
+    (*cdat_table)[CXL_USP_CDAT_SSLBIS_BW] = g_steal_pointer(&sslbis_bandwidth);
+
+    return CXL_USP_CDAT_NUM_ENTRIES;
+}
+
+static void free_default_cdat_table(CDATSubHeader **cdat_table, int num,
+                                    void *priv)
+{
+    int i;
+
+    for (i = 0; i < num; i++) {
+        g_free(cdat_table[i]);
+    }
+    g_free(cdat_table);
+}
+
 static void cxl_usp_realize(PCIDevice *d, Error **errp)
 {
     PCIEPort *p = PCIE_PORT(d);
@@ -161,6 +339,14 @@ static void cxl_usp_realize(PCIDevice *d, Error **errp)
                      PCI_BASE_ADDRESS_MEM_TYPE_64,
                      component_bar);
 
+    pcie_doe_init(d, &usp->doe_cdat, cxl_cstate->dvsec_offset, doe_cdat_prot,
+                  true, 1);
+
+    cxl_cstate->cdat.build_cdat_table = build_cdat_table;
+    cxl_cstate->cdat.free_cdat_table = free_default_cdat_table;
+    cxl_cstate->cdat.private = d;
+    cxl_doe_cdat_init(cxl_cstate, errp);
+
     return;
 
 err_cap:
@@ -179,6 +365,11 @@ static void cxl_usp_exitfn(PCIDevice *d)
     pci_bridge_exitfn(d);
 }
 
+static Property cxl_upstream_props[] = {
+    DEFINE_PROP_STRING("cdat", CXLUpstreamPort, cxl_cstate.cdat.filename),
+    DEFINE_PROP_END_OF_LIST()
+};
+
 static void cxl_upstream_class_init(ObjectClass *oc, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(oc);
@@ -186,6 +377,7 @@ static void cxl_upstream_class_init(ObjectClass *oc, void *data)
 
     k->is_bridge = true;
     k->config_write = cxl_usp_write_config;
+    k->config_read = cxl_usp_read_config;
     k->realize = cxl_usp_realize;
     k->exit = cxl_usp_exitfn;
     k->vendor_id = 0x19e5; /* Huawei */
@@ -194,6 +386,7 @@ static void cxl_upstream_class_init(ObjectClass *oc, void *data)
     set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
     dc->desc = "CXL Switch Upstream Port";
     dc->reset = cxl_usp_reset;
+    device_class_set_props(dc, cxl_upstream_props);
 }
 
 static const TypeInfo cxl_usp_info = {
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 26/83] hw/virtio/virtio-iommu-pci: Enforce the device is plugged on the root bus
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (24 preceding siblings ...)
  2022-11-07 22:49 ` [PULL v4 25/83] hw/pci-bridge/cxl-upstream: Add a CDAT table access DOE Michael S. Tsirkin
@ 2022-11-07 22:49 ` Michael S. Tsirkin
  2022-11-07 22:49 ` [PULL v4 27/83] virtio: introduce __virtio_queue_reset() Michael S. Tsirkin
                   ` (58 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:49 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Eric Auger, Jean-Philippe Brucker

From: Eric Auger <eric.auger@redhat.com>

In theory the virtio-iommu-pci could be plugged anywhere in the PCIe
topology and as long as the dt/acpi info are properly built this should
work. However at the moment we fail to do that because the
virtio-iommu-pci BDF is not computed at plug time and in that case
vms->virtio_iommu_bdf gets an incorrect value.

For instance if the virtio-iommu-pci is plugged onto a pcie root port
and the virtio-iommu protects a virtio-block-pci device the guest does
not boot.

So let's do not pretend we do support this case and fail the initialize()
if we detect the virtio-iommu-pci is plugged anywhere else than on the
root bus. Anyway this ability is not needed.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Message-Id: <20221012163448.121368-1-eric.auger@redhat.com>
Reviewed-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
Tested-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/virtio/virtio-iommu-pci.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/hw/virtio/virtio-iommu-pci.c b/hw/virtio/virtio-iommu-pci.c
index 79ea8334f0..7ef2f9dcdb 100644
--- a/hw/virtio/virtio-iommu-pci.c
+++ b/hw/virtio/virtio-iommu-pci.c
@@ -17,6 +17,7 @@
 #include "hw/qdev-properties-system.h"
 #include "qapi/error.h"
 #include "hw/boards.h"
+#include "hw/pci/pci_bus.h"
 #include "qom/object.h"
 
 typedef struct VirtIOIOMMUPCI VirtIOIOMMUPCI;
@@ -44,6 +45,7 @@ static Property virtio_iommu_pci_properties[] = {
 static void virtio_iommu_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp)
 {
     VirtIOIOMMUPCI *dev = VIRTIO_IOMMU_PCI(vpci_dev);
+    PCIBus *pbus = pci_get_bus(&vpci_dev->pci_dev);
     DeviceState *vdev = DEVICE(&dev->vdev);
     VirtIOIOMMU *s = VIRTIO_IOMMU(vdev);
 
@@ -57,11 +59,17 @@ static void virtio_iommu_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp)
             s->reserved_regions[i].type != VIRTIO_IOMMU_RESV_MEM_T_MSI) {
             error_setg(errp, "reserved region %d has an invalid type", i);
             error_append_hint(errp, "Valid values are 0 and 1\n");
+            return;
         }
     }
+    if (!pci_bus_is_root(pbus)) {
+        error_setg(errp, "virtio-iommu-pci must be plugged on the root bus");
+        return;
+    }
+
     object_property_set_link(OBJECT(dev), "primary-bus",
-                             OBJECT(pci_get_bus(&vpci_dev->pci_dev)),
-                             &error_abort);
+                             OBJECT(pbus), &error_abort);
+
     virtio_pci_force_virtio_1(vpci_dev);
     qdev_realize(vdev, BUS(&vpci_dev->bus), errp);
 }
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 27/83] virtio: introduce __virtio_queue_reset()
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (25 preceding siblings ...)
  2022-11-07 22:49 ` [PULL v4 26/83] hw/virtio/virtio-iommu-pci: Enforce the device is plugged on the root bus Michael S. Tsirkin
@ 2022-11-07 22:49 ` Michael S. Tsirkin
  2022-11-07 22:49 ` [PULL v4 28/83] virtio: introduce virtio_queue_reset() Michael S. Tsirkin
                   ` (57 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:49 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Xuan Zhuo, Jason Wang

From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>

Separate the logic of vq reset. This logic will be called directly
later.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20221017092558.111082-2-xuanzhuo@linux.alibaba.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/virtio/virtio.c | 37 +++++++++++++++++++++----------------
 1 file changed, 21 insertions(+), 16 deletions(-)

diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 808446b4c9..6f42fcadd7 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -2464,6 +2464,26 @@ static enum virtio_device_endian virtio_current_cpu_endian(void)
     }
 }
 
+static void __virtio_queue_reset(VirtIODevice *vdev, uint32_t i)
+{
+    vdev->vq[i].vring.desc = 0;
+    vdev->vq[i].vring.avail = 0;
+    vdev->vq[i].vring.used = 0;
+    vdev->vq[i].last_avail_idx = 0;
+    vdev->vq[i].shadow_avail_idx = 0;
+    vdev->vq[i].used_idx = 0;
+    vdev->vq[i].last_avail_wrap_counter = true;
+    vdev->vq[i].shadow_avail_wrap_counter = true;
+    vdev->vq[i].used_wrap_counter = true;
+    virtio_queue_set_vector(vdev, i, VIRTIO_NO_VECTOR);
+    vdev->vq[i].signalled_used = 0;
+    vdev->vq[i].signalled_used_valid = false;
+    vdev->vq[i].notification = true;
+    vdev->vq[i].vring.num = vdev->vq[i].vring.num_default;
+    vdev->vq[i].inuse = 0;
+    virtio_virtqueue_reset_region_cache(&vdev->vq[i]);
+}
+
 void virtio_reset(void *opaque)
 {
     VirtIODevice *vdev = opaque;
@@ -2495,22 +2515,7 @@ void virtio_reset(void *opaque)
     virtio_notify_vector(vdev, vdev->config_vector);
 
     for(i = 0; i < VIRTIO_QUEUE_MAX; i++) {
-        vdev->vq[i].vring.desc = 0;
-        vdev->vq[i].vring.avail = 0;
-        vdev->vq[i].vring.used = 0;
-        vdev->vq[i].last_avail_idx = 0;
-        vdev->vq[i].shadow_avail_idx = 0;
-        vdev->vq[i].used_idx = 0;
-        vdev->vq[i].last_avail_wrap_counter = true;
-        vdev->vq[i].shadow_avail_wrap_counter = true;
-        vdev->vq[i].used_wrap_counter = true;
-        virtio_queue_set_vector(vdev, i, VIRTIO_NO_VECTOR);
-        vdev->vq[i].signalled_used = 0;
-        vdev->vq[i].signalled_used_valid = false;
-        vdev->vq[i].notification = true;
-        vdev->vq[i].vring.num = vdev->vq[i].vring.num_default;
-        vdev->vq[i].inuse = 0;
-        virtio_virtqueue_reset_region_cache(&vdev->vq[i]);
+        __virtio_queue_reset(vdev, i);
     }
 }
 
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 28/83] virtio: introduce virtio_queue_reset()
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (26 preceding siblings ...)
  2022-11-07 22:49 ` [PULL v4 27/83] virtio: introduce __virtio_queue_reset() Michael S. Tsirkin
@ 2022-11-07 22:49 ` Michael S. Tsirkin
  2022-11-07 22:49 ` [PULL v4 29/83] virtio: introduce virtio_queue_enable() Michael S. Tsirkin
                   ` (56 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:49 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Xuan Zhuo, Jason Wang

From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>

Introduce a new interface function virtio_queue_reset() to implement
reset for vq.

Add a new callback to VirtioDeviceClass for queue reset operation for
each child device.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20221017092558.111082-3-xuanzhuo@linux.alibaba.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/hw/virtio/virtio.h |  2 ++
 hw/virtio/virtio.c         | 11 +++++++++++
 2 files changed, 13 insertions(+)

diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index f41b4a7e64..74d76c1dbc 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -148,6 +148,7 @@ struct VirtioDeviceClass {
     void (*set_config)(VirtIODevice *vdev, const uint8_t *config);
     void (*reset)(VirtIODevice *vdev);
     void (*set_status)(VirtIODevice *vdev, uint8_t val);
+    void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
     /* For transitional devices, this is a bitmap of features
      * that are only exposed on the legacy interface but not
      * the modern one.
@@ -286,6 +287,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
                                       MemoryRegion *mr, bool assign);
 int virtio_set_status(VirtIODevice *vdev, uint8_t val);
 void virtio_reset(void *opaque);
+void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
 void virtio_update_irq(VirtIODevice *vdev);
 int virtio_set_features(VirtIODevice *vdev, uint64_t val);
 
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 6f42fcadd7..cf5f9ca387 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -2484,6 +2484,17 @@ static void __virtio_queue_reset(VirtIODevice *vdev, uint32_t i)
     virtio_virtqueue_reset_region_cache(&vdev->vq[i]);
 }
 
+void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
+{
+    VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
+
+    if (k->queue_reset) {
+        k->queue_reset(vdev, queue_index);
+    }
+
+    __virtio_queue_reset(vdev, queue_index);
+}
+
 void virtio_reset(void *opaque)
 {
     VirtIODevice *vdev = opaque;
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 29/83] virtio: introduce virtio_queue_enable()
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (27 preceding siblings ...)
  2022-11-07 22:49 ` [PULL v4 28/83] virtio: introduce virtio_queue_reset() Michael S. Tsirkin
@ 2022-11-07 22:49 ` Michael S. Tsirkin
  2022-11-08 19:43   ` Stefan Hajnoczi
  2022-11-07 22:50 ` [PULL v4 30/83] virtio: core: vq reset feature negotation support Michael S. Tsirkin
                   ` (55 subsequent siblings)
  84 siblings, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:49 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Kangjie Xu, Xuan Zhuo, Jason Wang

From: Kangjie Xu <kangjie.xu@linux.alibaba.com>

Introduce the interface queue_enable() in VirtioDeviceClass and the
fucntion virtio_queue_enable() in virtio, it can be called when
VIRTIO_PCI_COMMON_Q_ENABLE is written and related virtqueue can be
started. It only supports the devices of virtio 1 or later. The
not-supported devices can only start the virtqueue when DRIVER_OK.

Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20221017092558.111082-4-xuanzhuo@linux.alibaba.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/hw/virtio/virtio.h |  2 ++
 hw/virtio/virtio.c         | 14 ++++++++++++++
 2 files changed, 16 insertions(+)

diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index 74d76c1dbc..b00b3fcf31 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -149,6 +149,7 @@ struct VirtioDeviceClass {
     void (*reset)(VirtIODevice *vdev);
     void (*set_status)(VirtIODevice *vdev, uint8_t val);
     void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
+    void (*queue_enable)(VirtIODevice *vdev, uint32_t queue_index);
     /* For transitional devices, this is a bitmap of features
      * that are only exposed on the legacy interface but not
      * the modern one.
@@ -288,6 +289,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
 int virtio_set_status(VirtIODevice *vdev, uint8_t val);
 void virtio_reset(void *opaque);
 void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
+void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index);
 void virtio_update_irq(VirtIODevice *vdev);
 int virtio_set_features(VirtIODevice *vdev, uint64_t val);
 
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index cf5f9ca387..9683b2e158 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -2495,6 +2495,20 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
     __virtio_queue_reset(vdev, queue_index);
 }
 
+void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
+{
+    VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
+
+    if (!virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
+        error_report("queue_enable is only suppported in devices of virtio "
+                     "1.0 or later.");
+    }
+
+    if (k->queue_enable) {
+        k->queue_enable(vdev, queue_index);
+    }
+}
+
 void virtio_reset(void *opaque)
 {
     VirtIODevice *vdev = opaque;
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 30/83] virtio: core: vq reset feature negotation support
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (28 preceding siblings ...)
  2022-11-07 22:49 ` [PULL v4 29/83] virtio: introduce virtio_queue_enable() Michael S. Tsirkin
@ 2022-11-07 22:50 ` Michael S. Tsirkin
  2022-11-18 14:32   ` Stefano Garzarella
  2022-11-07 22:50 ` [PULL v4 31/83] virtio-pci: support queue reset Michael S. Tsirkin
                   ` (54 subsequent siblings)
  84 siblings, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:50 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Kangjie Xu, Xuan Zhuo, Jason Wang,
	Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
	Yanan Wang

From: Kangjie Xu <kangjie.xu@linux.alibaba.com>

A a new command line parameter "queue_reset" is added.

Meanwhile, the vq reset feature is disabled for pre-7.2 machines.

Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20221017092558.111082-5-xuanzhuo@linux.alibaba.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/hw/virtio/virtio.h | 4 +++-
 hw/core/machine.c          | 4 +++-
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index b00b3fcf31..1423dba379 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -313,7 +313,9 @@ typedef struct VirtIORNGConf VirtIORNGConf;
     DEFINE_PROP_BIT64("iommu_platform", _state, _field, \
                       VIRTIO_F_IOMMU_PLATFORM, false), \
     DEFINE_PROP_BIT64("packed", _state, _field, \
-                      VIRTIO_F_RING_PACKED, false)
+                      VIRTIO_F_RING_PACKED, false), \
+    DEFINE_PROP_BIT64("queue_reset", _state, _field, \
+                      VIRTIO_F_RING_RESET, true)
 
 hwaddr virtio_queue_get_desc_addr(VirtIODevice *vdev, int n);
 bool virtio_queue_enabled_legacy(VirtIODevice *vdev, int n);
diff --git a/hw/core/machine.c b/hw/core/machine.c
index aa520e74a8..907fa78ff0 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -40,7 +40,9 @@
 #include "hw/virtio/virtio-pci.h"
 #include "qom/object_interfaces.h"
 
-GlobalProperty hw_compat_7_1[] = {};
+GlobalProperty hw_compat_7_1[] = {
+    { "virtio-device", "queue_reset", "false" },
+};
 const size_t hw_compat_7_1_len = G_N_ELEMENTS(hw_compat_7_1);
 
 GlobalProperty hw_compat_7_0[] = {
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 31/83] virtio-pci: support queue reset
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (29 preceding siblings ...)
  2022-11-07 22:50 ` [PULL v4 30/83] virtio: core: vq reset feature negotation support Michael S. Tsirkin
@ 2022-11-07 22:50 ` Michael S. Tsirkin
  2022-11-07 22:50 ` [PULL v4 32/83] virtio-pci: support queue enable Michael S. Tsirkin
                   ` (53 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:50 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Xuan Zhuo, Kangjie Xu, Jason Wang

From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>

PCI devices support vq reset.

Based on this function, the driver can adjust the size of the ring, and
quickly recycle the buffer in the ring.

The migration of the virtio devices will not happen during a reset
operation. This is becuase the global iothread lock is held. Migration
thread also needs the lock. As a result, when migration of virtio
devices starts, the 'reset' status of VirtIOPCIQueue will always be 0.
Thus, we do not need to add it in vmstate_virtio_pci_modern_queue_state.

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20221017092558.111082-6-xuanzhuo@linux.alibaba.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/hw/virtio/virtio-pci.h |  5 +++++
 hw/virtio/virtio-pci.c         | 15 +++++++++++++++
 2 files changed, 20 insertions(+)

diff --git a/include/hw/virtio/virtio-pci.h b/include/hw/virtio/virtio-pci.h
index 2446dcd9ae..938799e8f6 100644
--- a/include/hw/virtio/virtio-pci.h
+++ b/include/hw/virtio/virtio-pci.h
@@ -117,6 +117,11 @@ typedef struct VirtIOPCIRegion {
 typedef struct VirtIOPCIQueue {
   uint16_t num;
   bool enabled;
+  /*
+   * No need to migrate the reset status, because it is always 0
+   * when the migration starts.
+   */
+  bool reset;
   uint32_t desc[2];
   uint32_t avail[2];
   uint32_t used[2];
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index 34db51e241..d4f2ffe986 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -1251,6 +1251,9 @@ static uint64_t virtio_pci_common_read(void *opaque, hwaddr addr,
     case VIRTIO_PCI_COMMON_Q_USEDHI:
         val = proxy->vqs[vdev->queue_sel].used[1];
         break;
+    case VIRTIO_PCI_COMMON_Q_RESET:
+        val = proxy->vqs[vdev->queue_sel].reset;
+        break;
     default:
         val = 0;
     }
@@ -1338,6 +1341,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
                        ((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << 32 |
                        proxy->vqs[vdev->queue_sel].used[0]);
             proxy->vqs[vdev->queue_sel].enabled = 1;
+            proxy->vqs[vdev->queue_sel].reset = 0;
         } else {
             virtio_error(vdev, "wrong value for queue_enable %"PRIx64, val);
         }
@@ -1360,6 +1364,16 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
     case VIRTIO_PCI_COMMON_Q_USEDHI:
         proxy->vqs[vdev->queue_sel].used[1] = val;
         break;
+    case VIRTIO_PCI_COMMON_Q_RESET:
+        if (val == 1) {
+            proxy->vqs[vdev->queue_sel].reset = 1;
+
+            virtio_queue_reset(vdev, vdev->queue_sel);
+
+            proxy->vqs[vdev->queue_sel].reset = 0;
+            proxy->vqs[vdev->queue_sel].enabled = 0;
+        }
+        break;
     default:
         break;
     }
@@ -1954,6 +1968,7 @@ static void virtio_pci_reset(DeviceState *qdev)
 
     for (i = 0; i < VIRTIO_QUEUE_MAX; i++) {
         proxy->vqs[i].enabled = 0;
+        proxy->vqs[i].reset = 0;
         proxy->vqs[i].num = 0;
         proxy->vqs[i].desc[0] = proxy->vqs[i].desc[1] = 0;
         proxy->vqs[i].avail[0] = proxy->vqs[i].avail[1] = 0;
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 32/83] virtio-pci: support queue enable
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (30 preceding siblings ...)
  2022-11-07 22:50 ` [PULL v4 31/83] virtio-pci: support queue reset Michael S. Tsirkin
@ 2022-11-07 22:50 ` Michael S. Tsirkin
  2022-11-07 22:50 ` [PULL v4 33/83] vhost: expose vhost_virtqueue_start() Michael S. Tsirkin
                   ` (52 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:50 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Kangjie Xu, Xuan Zhuo, Jason Wang

From: Kangjie Xu <kangjie.xu@linux.alibaba.com>

PCI devices support device specific vq enable.

Based on this function, the driver can re-enable the virtqueue after the
virtqueue is reset.

Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20221017092558.111082-7-xuanzhuo@linux.alibaba.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/virtio/virtio-pci.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index d4f2ffe986..855718d586 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -1342,6 +1342,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
                        proxy->vqs[vdev->queue_sel].used[0]);
             proxy->vqs[vdev->queue_sel].enabled = 1;
             proxy->vqs[vdev->queue_sel].reset = 0;
+            virtio_queue_enable(vdev, vdev->queue_sel);
         } else {
             virtio_error(vdev, "wrong value for queue_enable %"PRIx64, val);
         }
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 33/83] vhost: expose vhost_virtqueue_start()
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (31 preceding siblings ...)
  2022-11-07 22:50 ` [PULL v4 32/83] virtio-pci: support queue enable Michael S. Tsirkin
@ 2022-11-07 22:50 ` Michael S. Tsirkin
  2022-11-07 22:50 ` [PULL v4 34/83] vhost: expose vhost_virtqueue_stop() Michael S. Tsirkin
                   ` (51 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:50 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Kangjie Xu, Xuan Zhuo, Jason Wang

From: Kangjie Xu <kangjie.xu@linux.alibaba.com>

Expose vhost_virtqueue_start(), we need to use it when restarting a
virtqueue.

Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20221017092558.111082-8-xuanzhuo@linux.alibaba.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/hw/virtio/vhost.h | 3 +++
 hw/virtio/vhost.c         | 8 ++++----
 2 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
index d7eb557885..0054a695dc 100644
--- a/include/hw/virtio/vhost.h
+++ b/include/hw/virtio/vhost.h
@@ -297,6 +297,9 @@ int vhost_net_set_backend(struct vhost_dev *hdev,
 
 int vhost_device_iotlb_miss(struct vhost_dev *dev, uint64_t iova, int write);
 
+int vhost_virtqueue_start(struct vhost_dev *dev, struct VirtIODevice *vdev,
+                          struct vhost_virtqueue *vq, unsigned idx);
+
 void vhost_dev_reset_inflight(struct vhost_inflight *inflight);
 void vhost_dev_free_inflight(struct vhost_inflight *inflight);
 void vhost_dev_save_inflight(struct vhost_inflight *inflight, QEMUFile *f);
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index 5185c15295..788d0a0679 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1081,10 +1081,10 @@ out:
     return ret;
 }
 
-static int vhost_virtqueue_start(struct vhost_dev *dev,
-                                struct VirtIODevice *vdev,
-                                struct vhost_virtqueue *vq,
-                                unsigned idx)
+int vhost_virtqueue_start(struct vhost_dev *dev,
+                          struct VirtIODevice *vdev,
+                          struct vhost_virtqueue *vq,
+                          unsigned idx)
 {
     BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev)));
     VirtioBusState *vbus = VIRTIO_BUS(qbus);
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 34/83] vhost: expose vhost_virtqueue_stop()
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (32 preceding siblings ...)
  2022-11-07 22:50 ` [PULL v4 33/83] vhost: expose vhost_virtqueue_start() Michael S. Tsirkin
@ 2022-11-07 22:50 ` Michael S. Tsirkin
  2022-11-07 22:50 ` [PULL v4 35/83] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_reset() Michael S. Tsirkin
                   ` (50 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:50 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Kangjie Xu, Xuan Zhuo, Jason Wang

From: Kangjie Xu <kangjie.xu@linux.alibaba.com>

Expose vhost_virtqueue_stop(), we need to use it when resetting a
virtqueue.

Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20221017092558.111082-9-xuanzhuo@linux.alibaba.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/hw/virtio/vhost.h | 2 ++
 hw/virtio/vhost.c         | 8 ++++----
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
index 0054a695dc..353252ac3e 100644
--- a/include/hw/virtio/vhost.h
+++ b/include/hw/virtio/vhost.h
@@ -299,6 +299,8 @@ int vhost_device_iotlb_miss(struct vhost_dev *dev, uint64_t iova, int write);
 
 int vhost_virtqueue_start(struct vhost_dev *dev, struct VirtIODevice *vdev,
                           struct vhost_virtqueue *vq, unsigned idx);
+void vhost_virtqueue_stop(struct vhost_dev *dev, struct VirtIODevice *vdev,
+                          struct vhost_virtqueue *vq, unsigned idx);
 
 void vhost_dev_reset_inflight(struct vhost_inflight *inflight);
 void vhost_dev_free_inflight(struct vhost_inflight *inflight);
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index 788d0a0679..d1c4c20b8c 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1201,10 +1201,10 @@ fail_alloc_desc:
     return r;
 }
 
-static void vhost_virtqueue_stop(struct vhost_dev *dev,
-                                    struct VirtIODevice *vdev,
-                                    struct vhost_virtqueue *vq,
-                                    unsigned idx)
+void vhost_virtqueue_stop(struct vhost_dev *dev,
+                          struct VirtIODevice *vdev,
+                          struct vhost_virtqueue *vq,
+                          unsigned idx)
 {
     int vhost_vq_index = dev->vhost_ops->vhost_get_vq_index(dev, idx);
     struct vhost_vring_state state = {
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 35/83] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_reset()
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (33 preceding siblings ...)
  2022-11-07 22:50 ` [PULL v4 34/83] vhost: expose vhost_virtqueue_stop() Michael S. Tsirkin
@ 2022-11-07 22:50 ` Michael S. Tsirkin
  2022-11-07 22:50 ` [PULL v4 36/83] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart() Michael S. Tsirkin
                   ` (49 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:50 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Kangjie Xu, Xuan Zhuo, Jason Wang

From: Kangjie Xu <kangjie.xu@linux.alibaba.com>

Introduce vhost_virtqueue_reset(), which can reset the specific
virtqueue in the device. Then it will unmap vrings and the desc
of the virtqueue.

Here we do not reuse the vhost_net_stop_one() or vhost_dev_stop(),
because they work at queue pair level. We do not use
vhost_virtqueue_stop() because it may stop the device in the
backend.

This patch only considers the case of vhost-kernel, when
NetClientDriver is NET_CLIENT_DRIVER_TAP.

Furthermore, we do not need net->nc->info->poll() because
it enables userspace datapath and we want to stop all
datapaths for this reset virtqueue here.

Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20221017092558.111082-10-xuanzhuo@linux.alibaba.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/net/vhost_net.h |  2 ++
 hw/net/vhost_net-stub.c |  6 ++++++
 hw/net/vhost_net.c      | 25 +++++++++++++++++++++++++
 3 files changed, 33 insertions(+)

diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
index 387e913e4e..85d85a4957 100644
--- a/include/net/vhost_net.h
+++ b/include/net/vhost_net.h
@@ -48,4 +48,6 @@ uint64_t vhost_net_get_acked_features(VHostNetState *net);
 
 int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu);
 
+void vhost_net_virtqueue_reset(VirtIODevice *vdev, NetClientState *nc,
+                               int vq_index);
 #endif
diff --git a/hw/net/vhost_net-stub.c b/hw/net/vhost_net-stub.c
index 89d71cfb8e..2d745e359c 100644
--- a/hw/net/vhost_net-stub.c
+++ b/hw/net/vhost_net-stub.c
@@ -101,3 +101,9 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu)
 {
     return 0;
 }
+
+void vhost_net_virtqueue_reset(VirtIODevice *vdev, NetClientState *nc,
+                               int vq_index)
+{
+
+}
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index d28f8b974b..8beecb4d22 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -531,3 +531,28 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu)
 
     return vhost_ops->vhost_net_set_mtu(&net->dev, mtu);
 }
+
+void vhost_net_virtqueue_reset(VirtIODevice *vdev, NetClientState *nc,
+                               int vq_index)
+{
+    VHostNetState *net = get_vhost_net(nc->peer);
+    const VhostOps *vhost_ops = net->dev.vhost_ops;
+    struct vhost_vring_file file = { .fd = -1 };
+    int idx;
+
+    /* should only be called after backend is connected */
+    assert(vhost_ops);
+
+    idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index);
+
+    if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
+        file.index = idx;
+        int r = vhost_net_set_backend(&net->dev, &file);
+        assert(r >= 0);
+    }
+
+    vhost_virtqueue_stop(&net->dev,
+                         vdev,
+                         net->dev.vqs + idx,
+                         net->dev.vq_index + idx);
+}
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 36/83] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (34 preceding siblings ...)
  2022-11-07 22:50 ` [PULL v4 35/83] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_reset() Michael S. Tsirkin
@ 2022-11-07 22:50 ` Michael S. Tsirkin
  2022-11-07 22:50 ` [PULL v4 37/83] virtio-net: introduce flush_or_purge_queued_packets() Michael S. Tsirkin
                   ` (48 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:50 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Kangjie Xu, Xuan Zhuo, Jason Wang

From: Kangjie Xu <kangjie.xu@linux.alibaba.com>

Introduce vhost_net_virtqueue_restart(), which can restart the
specific virtqueue when the vhost net started running before.
If it fails to restart the virtqueue, the device will be stopped.

Here we do not reuse vhost_net_start_one() or vhost_dev_start()
because they work at queue pair level. The mem table and features
do not change, so we can call the vhost_virtqueue_start() to
restart a specific queue.

This patch only considers the case of vhost-kernel, when
NetClientDriver is NET_CLIENT_DRIVER_TAP.

Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20221017092558.111082-11-xuanzhuo@linux.alibaba.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/net/vhost_net.h |  2 ++
 hw/net/vhost_net-stub.c |  6 +++++
 hw/net/vhost_net.c      | 53 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 61 insertions(+)

diff --git a/include/net/vhost_net.h b/include/net/vhost_net.h
index 85d85a4957..40b9a40074 100644
--- a/include/net/vhost_net.h
+++ b/include/net/vhost_net.h
@@ -50,4 +50,6 @@ int vhost_net_set_mtu(struct vhost_net *net, uint16_t mtu);
 
 void vhost_net_virtqueue_reset(VirtIODevice *vdev, NetClientState *nc,
                                int vq_index);
+int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState *nc,
+                                int vq_index);
 #endif
diff --git a/hw/net/vhost_net-stub.c b/hw/net/vhost_net-stub.c
index 2d745e359c..9f7daae99c 100644
--- a/hw/net/vhost_net-stub.c
+++ b/hw/net/vhost_net-stub.c
@@ -107,3 +107,9 @@ void vhost_net_virtqueue_reset(VirtIODevice *vdev, NetClientState *nc,
 {
 
 }
+
+int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState *nc,
+                                int vq_index)
+{
+    return 0;
+}
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 8beecb4d22..d2926e2ed6 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -34,6 +34,7 @@
 #include "standard-headers/linux/virtio_ring.h"
 #include "hw/virtio/vhost.h"
 #include "hw/virtio/virtio-bus.h"
+#include "linux-headers/linux/vhost.h"
 
 
 /* Features supported by host kernel. */
@@ -556,3 +557,55 @@ void vhost_net_virtqueue_reset(VirtIODevice *vdev, NetClientState *nc,
                          net->dev.vqs + idx,
                          net->dev.vq_index + idx);
 }
+
+int vhost_net_virtqueue_restart(VirtIODevice *vdev, NetClientState *nc,
+                                int vq_index)
+{
+    VHostNetState *net = get_vhost_net(nc->peer);
+    const VhostOps *vhost_ops = net->dev.vhost_ops;
+    struct vhost_vring_file file = { };
+    int idx, r;
+
+    if (!net->dev.started) {
+        return -EBUSY;
+    }
+
+    /* should only be called after backend is connected */
+    assert(vhost_ops);
+
+    idx = vhost_ops->vhost_get_vq_index(&net->dev, vq_index);
+
+    r = vhost_virtqueue_start(&net->dev,
+                              vdev,
+                              net->dev.vqs + idx,
+                              net->dev.vq_index + idx);
+    if (r < 0) {
+        goto err_start;
+    }
+
+    if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
+        file.index = idx;
+        file.fd = net->backend;
+        r = vhost_net_set_backend(&net->dev, &file);
+        if (r < 0) {
+            r = -errno;
+            goto err_start;
+        }
+    }
+
+    return 0;
+
+err_start:
+    error_report("Error when restarting the queue.");
+
+    if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
+        file.fd = VHOST_FILE_UNBIND;
+        file.index = idx;
+        int r = vhost_net_set_backend(&net->dev, &file);
+        assert(r >= 0);
+    }
+
+    vhost_dev_stop(&net->dev, vdev);
+
+    return r;
+}
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 37/83] virtio-net: introduce flush_or_purge_queued_packets()
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (35 preceding siblings ...)
  2022-11-07 22:50 ` [PULL v4 36/83] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart() Michael S. Tsirkin
@ 2022-11-07 22:50 ` Michael S. Tsirkin
  2022-11-07 22:50 ` [PULL v4 38/83] virtio-net: support queue reset Michael S. Tsirkin
                   ` (47 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:50 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Kangjie Xu, Xuan Zhuo, Jason Wang

From: Kangjie Xu <kangjie.xu@linux.alibaba.com>

Introduce the fucntion flush_or_purge_queued_packets(), it will be
used in device reset and virtqueue reset. Therefore, we extract the
common logic as a new function.

Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20221017092558.111082-12-xuanzhuo@linux.alibaba.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/net/virtio-net.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index b6903aea54..038a6fba7c 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -124,6 +124,16 @@ static int vq2q(int queue_index)
     return queue_index / 2;
 }
 
+static void flush_or_purge_queued_packets(NetClientState *nc)
+{
+    if (!nc->peer) {
+        return;
+    }
+
+    qemu_flush_or_purge_queued_packets(nc->peer, true);
+    assert(!virtio_net_get_subqueue(nc)->async_tx.elem);
+}
+
 /* TODO
  * - we could suppress RX interrupt if we were so inclined.
  */
@@ -566,12 +576,7 @@ static void virtio_net_reset(VirtIODevice *vdev)
 
     /* Flush any async TX */
     for (i = 0;  i < n->max_queue_pairs; i++) {
-        NetClientState *nc = qemu_get_subqueue(n->nic, i);
-
-        if (nc->peer) {
-            qemu_flush_or_purge_queued_packets(nc->peer, true);
-            assert(!virtio_net_get_subqueue(nc)->async_tx.elem);
-        }
+        flush_or_purge_queued_packets(qemu_get_subqueue(n->nic, i));
     }
 }
 
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 38/83] virtio-net: support queue reset
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (36 preceding siblings ...)
  2022-11-07 22:50 ` [PULL v4 37/83] virtio-net: introduce flush_or_purge_queued_packets() Michael S. Tsirkin
@ 2022-11-07 22:50 ` Michael S. Tsirkin
  2022-11-07 22:50 ` [PULL v4 39/83] virtio-net: support queue_enable Michael S. Tsirkin
                   ` (46 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:50 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Xuan Zhuo, Kangjie Xu, Jason Wang

From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>

virtio-net and vhost-kernel implement queue reset.
Queued packets in the corresponding queue pair are flushed
or purged.

For virtio-net, userspace datapath will be disabled later in
__virtio_queue_reset(). It will set addr of vring to 0 and idx to 0.
Thus, virtio_net_receive() and virtio_net_flush_tx() will not receive
or send packets.

For vhost-net, the datapath will be disabled in vhost_net_virtqueue_reset().

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20221017092558.111082-13-xuanzhuo@linux.alibaba.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/net/virtio-net.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 038a6fba7c..34fb4b1423 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -546,6 +546,23 @@ static RxFilterInfo *virtio_net_query_rxfilter(NetClientState *nc)
     return info;
 }
 
+static void virtio_net_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
+{
+    VirtIONet *n = VIRTIO_NET(vdev);
+    NetClientState *nc = qemu_get_subqueue(n->nic, vq2q(queue_index));
+
+    if (!nc->peer) {
+        return;
+    }
+
+    if (get_vhost_net(nc->peer) &&
+        nc->peer->info->type == NET_CLIENT_DRIVER_TAP) {
+        vhost_net_virtqueue_reset(vdev, nc, queue_index);
+    }
+
+    flush_or_purge_queued_packets(nc);
+}
+
 static void virtio_net_reset(VirtIODevice *vdev)
 {
     VirtIONet *n = VIRTIO_NET(vdev);
@@ -3827,6 +3844,7 @@ static void virtio_net_class_init(ObjectClass *klass, void *data)
     vdc->set_features = virtio_net_set_features;
     vdc->bad_features = virtio_net_bad_features;
     vdc->reset = virtio_net_reset;
+    vdc->queue_reset = virtio_net_queue_reset;
     vdc->set_status = virtio_net_set_status;
     vdc->guest_notifier_mask = virtio_net_guest_notifier_mask;
     vdc->guest_notifier_pending = virtio_net_guest_notifier_pending;
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 39/83] virtio-net: support queue_enable
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (37 preceding siblings ...)
  2022-11-07 22:50 ` [PULL v4 38/83] virtio-net: support queue reset Michael S. Tsirkin
@ 2022-11-07 22:50 ` Michael S. Tsirkin
  2022-11-07 22:50 ` [PULL v4 40/83] vhost: vhost-kernel: enable vq reset feature Michael S. Tsirkin
                   ` (45 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:50 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Kangjie Xu, Xuan Zhuo, Jason Wang

From: Kangjie Xu <kangjie.xu@linux.alibaba.com>

Support queue_enable in vhost-kernel scenario. It can be called when
a vq reset operation has been performed and the vq is restared.

It should be noted that we can restart the vq when the vhost has
already started. When launching a new vhost device, the vhost is not
started and all vqs are not initalized until VIRTIO_PCI_COMMON_STATUS
is written. Thus, we should use vhost_started to differentiate the
two cases: vq reset and device start.

Currently it only supports vhost-kernel.

Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20221017092558.111082-14-xuanzhuo@linux.alibaba.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/net/virtio-net.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 34fb4b1423..e68daf51bb 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -563,6 +563,26 @@ static void virtio_net_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
     flush_or_purge_queued_packets(nc);
 }
 
+static void virtio_net_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
+{
+    VirtIONet *n = VIRTIO_NET(vdev);
+    NetClientState *nc = qemu_get_subqueue(n->nic, vq2q(queue_index));
+    int r;
+
+    if (!nc->peer || !vdev->vhost_started) {
+        return;
+    }
+
+    if (get_vhost_net(nc->peer) &&
+        nc->peer->info->type == NET_CLIENT_DRIVER_TAP) {
+        r = vhost_net_virtqueue_restart(vdev, nc, queue_index);
+        if (r < 0) {
+            error_report("unable to restart vhost net virtqueue: %d, "
+                            "when resetting the queue", queue_index);
+        }
+    }
+}
+
 static void virtio_net_reset(VirtIODevice *vdev)
 {
     VirtIONet *n = VIRTIO_NET(vdev);
@@ -3845,6 +3865,7 @@ static void virtio_net_class_init(ObjectClass *klass, void *data)
     vdc->bad_features = virtio_net_bad_features;
     vdc->reset = virtio_net_reset;
     vdc->queue_reset = virtio_net_queue_reset;
+    vdc->queue_enable = virtio_net_queue_enable;
     vdc->set_status = virtio_net_set_status;
     vdc->guest_notifier_mask = virtio_net_guest_notifier_mask;
     vdc->guest_notifier_pending = virtio_net_guest_notifier_pending;
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 40/83] vhost: vhost-kernel: enable vq reset feature
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (38 preceding siblings ...)
  2022-11-07 22:50 ` [PULL v4 39/83] virtio-net: support queue_enable Michael S. Tsirkin
@ 2022-11-07 22:50 ` Michael S. Tsirkin
  2022-11-07 22:50 ` [PULL v4 41/83] virtio-net: " Michael S. Tsirkin
                   ` (44 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:50 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Kangjie Xu, Xuan Zhuo, Jason Wang

From: Kangjie Xu <kangjie.xu@linux.alibaba.com>

Add virtqueue reset feature for vhost-kernel.

Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Acked-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20221017092558.111082-15-xuanzhuo@linux.alibaba.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/net/vhost_net.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index d2926e2ed6..53b2fac4f6 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -47,6 +47,7 @@ static const int kernel_feature_bits[] = {
     VIRTIO_NET_F_MTU,
     VIRTIO_F_IOMMU_PLATFORM,
     VIRTIO_F_RING_PACKED,
+    VIRTIO_F_RING_RESET,
     VIRTIO_NET_F_HASH_REPORT,
     VHOST_INVALID_FEATURE_BIT
 };
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 41/83] virtio-net: enable vq reset feature
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (39 preceding siblings ...)
  2022-11-07 22:50 ` [PULL v4 40/83] vhost: vhost-kernel: enable vq reset feature Michael S. Tsirkin
@ 2022-11-07 22:50 ` Michael S. Tsirkin
  2022-11-07 22:50 ` [PULL v4 42/83] virtio-rng-pci: Allow setting nvectors, so we can use MSI-X Michael S. Tsirkin
                   ` (43 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:50 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Xuan Zhuo, Jason Wang

From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>

Add virtqueue reset feature for virtio-net

Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Message-Id: <20221017092558.111082-16-xuanzhuo@linux.alibaba.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/net/virtio-net.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index e68daf51bb..8b32339b76 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -788,6 +788,7 @@ static uint64_t virtio_net_get_features(VirtIODevice *vdev, uint64_t features,
     }
 
     if (!get_vhost_net(nc->peer)) {
+        virtio_add_feature(&features, VIRTIO_F_RING_RESET);
         return features;
     }
 
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 42/83] virtio-rng-pci: Allow setting nvectors, so we can use MSI-X
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (40 preceding siblings ...)
  2022-11-07 22:50 ` [PULL v4 41/83] virtio-net: " Michael S. Tsirkin
@ 2022-11-07 22:50 ` Michael S. Tsirkin
  2023-01-09 10:20   ` Dr. David Alan Gilbert
  2022-11-07 22:51 ` [PULL v4 43/83] vhost-user: Fix out of order vring host notification handling Michael S. Tsirkin
                   ` (42 subsequent siblings)
  84 siblings, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:50 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, David Daney, Marcin Nowakowski,
	Philippe Mathieu-Daudé,
	Stefan Hajnoczi

From: David Daney <david.daney@fungible.com>

Most other virtio-pci devices allow MSI-X, let's have it for rng too.

Signed-off-by: David Daney <david.daney@fungible.com>
Reviewed-by: Marcin Nowakowski <marcin.nowakowski@fungible.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@fungible.com>
Message-Id: <20221014160947.66105-1-philmd@fungible.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/virtio/virtio-rng-pci.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/hw/virtio/virtio-rng-pci.c b/hw/virtio/virtio-rng-pci.c
index 151ece6f94..6e76f8b57b 100644
--- a/hw/virtio/virtio-rng-pci.c
+++ b/hw/virtio/virtio-rng-pci.c
@@ -13,6 +13,7 @@
 
 #include "hw/virtio/virtio-pci.h"
 #include "hw/virtio/virtio-rng.h"
+#include "hw/qdev-properties.h"
 #include "qapi/error.h"
 #include "qemu/module.h"
 #include "qom/object.h"
@@ -31,11 +32,23 @@ struct VirtIORngPCI {
     VirtIORNG vdev;
 };
 
+static Property virtio_rng_properties[] = {
+    DEFINE_PROP_BIT("ioeventfd", VirtIOPCIProxy, flags,
+                    VIRTIO_PCI_FLAG_USE_IOEVENTFD_BIT, true),
+    DEFINE_PROP_UINT32("vectors", VirtIOPCIProxy, nvectors,
+                       DEV_NVECTORS_UNSPECIFIED),
+    DEFINE_PROP_END_OF_LIST(),
+};
+
 static void virtio_rng_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp)
 {
     VirtIORngPCI *vrng = VIRTIO_RNG_PCI(vpci_dev);
     DeviceState *vdev = DEVICE(&vrng->vdev);
 
+    if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) {
+        vpci_dev->nvectors = 2;
+    }
+
     if (!qdev_realize(vdev, BUS(&vpci_dev->bus), errp)) {
         return;
     }
@@ -54,6 +67,7 @@ static void virtio_rng_pci_class_init(ObjectClass *klass, void *data)
     pcidev_k->device_id = PCI_DEVICE_ID_VIRTIO_RNG;
     pcidev_k->revision = VIRTIO_PCI_ABI_VERSION;
     pcidev_k->class_id = PCI_CLASS_OTHERS;
+    device_class_set_props(dc, virtio_rng_properties);
 }
 
 static void virtio_rng_initfn(Object *obj)
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 43/83] vhost-user: Fix out of order vring host notification handling
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (41 preceding siblings ...)
  2022-11-07 22:50 ` [PULL v4 42/83] virtio-rng-pci: Allow setting nvectors, so we can use MSI-X Michael S. Tsirkin
@ 2022-11-07 22:51 ` Michael S. Tsirkin
  2022-11-07 22:51 ` [PULL v4 44/83] acpi: pc: vga: use AcpiDevAmlIf interface to build VGA device descriptors Michael S. Tsirkin
                   ` (41 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:51 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Yajun Wu, Parav Pandit, Alex Bennée

From: Yajun Wu <yajunw@nvidia.com>

vhost backend sends host notification for every VQ. If backend creates
VQs in parallel, the VHOST_USER_SLAVE_VRING_HOST_NOTIFIER_MSG may
arrive to QEMU in different order than incremental queue index order.

For example VQ 1's message arrive earlier than VQ 0's:
After alloc VhostUserHostNotifier for VQ 1. GPtrArray becomes

    [ nil, VQ1 pointer ]

After alloc VhostUserHostNotifier for VQ 0. GPtrArray becomes

    [ VQ0 pointer, nil, VQ1 pointer ]

This is wrong. fetch_notifier will return NULL for VQ 1 in
vhost_user_get_vring_base, causes host notifier miss removal(leak).

The fix is to remove current element from GPtrArray, make the right
position for element to insert.

Fixes: 503e355465 ("virtio/vhost-user: dynamically assign VhostUserHostNotifiers")
Signed-off-by: Yajun Wu <yajunw@nvidia.com>
Acked-by: Parav Pandit <parav@nvidia.com>

Message-Id: <20221018023651.1359420-1-yajunw@nvidia.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/virtio/vhost-user.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index 03415b6c95..d256ce589b 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -1543,6 +1543,11 @@ static VhostUserHostNotifier *fetch_or_create_notifier(VhostUserState *u,
 
     n = g_ptr_array_index(u->notifiers, idx);
     if (!n) {
+        /*
+         * In case notification arrive out-of-order,
+         * make room for current index.
+         */
+        g_ptr_array_remove_index(u->notifiers, idx);
         n = g_new0(VhostUserHostNotifier, 1);
         n->idx = idx;
         g_ptr_array_insert(u->notifiers, idx, n);
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 44/83] acpi: pc: vga: use AcpiDevAmlIf interface to build VGA device descriptors
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (42 preceding siblings ...)
  2022-11-07 22:51 ` [PULL v4 43/83] vhost-user: Fix out of order vring host notification handling Michael S. Tsirkin
@ 2022-11-07 22:51 ` Michael S. Tsirkin
  2022-11-09 17:39   ` Laurent Vivier
  2022-11-07 22:51 ` [PULL v4 45/83] tests: acpi: whitelist DSDT before generating PCI-ISA bridge AML automatically Michael S. Tsirkin
                   ` (40 subsequent siblings)
  84 siblings, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:51 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Igor Mammedov, Ani Sinha, Bernhard Beschow,
	Gerd Hoffmann, Marcel Apfelbaum, Paolo Bonzini,
	Richard Henderson, Eduardo Habkost

From: Igor Mammedov <imammedo@redhat.com>

NB:
We do not expect any functional change in any ACPI tables with this
change. It's only a refactoring.

NB2:
Some targets (or1k) do not support acpi and CONFIG_ACPI is off for them.
However, modules are reused between all architectures so CONFIG_ACPI is
on.  For those architectures, dummy stub function definitions help to
resolve symbols.  This change uses more of these and so it adds a couple
of dummy stub definitions so that symbols for those can be resolved.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20221017102146.2254096-2-imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Ani Sinha <ani@anisinha.ca>
CC: Bernhard Beschow <shentey@gmail.com>
Signed-off-by: Ani Sinha <ani@anisinha.ca>
Message-Id: <20221107152744.868434-1-ani@anisinha.ca>
---
 hw/display/vga_int.h       |  2 ++
 hw/acpi/aml-build-stub.c   | 10 ++++++++++
 hw/display/acpi-vga-stub.c |  7 +++++++
 hw/display/acpi-vga.c      | 26 ++++++++++++++++++++++++++
 hw/display/vga-pci.c       |  4 ++++
 hw/i386/acpi-build.c       | 26 +-------------------------
 hw/display/meson.build     | 17 +++++++++++++++++
 7 files changed, 67 insertions(+), 25 deletions(-)
 create mode 100644 hw/display/acpi-vga-stub.c
 create mode 100644 hw/display/acpi-vga.c

diff --git a/hw/display/vga_int.h b/hw/display/vga_int.h
index 305e700014..330406ad9c 100644
--- a/hw/display/vga_int.h
+++ b/hw/display/vga_int.h
@@ -30,6 +30,7 @@
 #include "ui/console.h"
 
 #include "hw/display/bochs-vbe.h"
+#include "hw/acpi/acpi_aml_interface.h"
 
 #define ST01_V_RETRACE      0x08
 #define ST01_DISP_ENABLE    0x01
@@ -195,4 +196,5 @@ void pci_std_vga_mmio_region_init(VGACommonState *s,
                                   MemoryRegion *subs,
                                   bool qext, bool edid);
 
+void build_vga_aml(AcpiDevAmlIf *adev, Aml *scope);
 #endif
diff --git a/hw/acpi/aml-build-stub.c b/hw/acpi/aml-build-stub.c
index 8d8ad1a314..89a8fec4af 100644
--- a/hw/acpi/aml-build-stub.c
+++ b/hw/acpi/aml-build-stub.c
@@ -26,6 +26,16 @@ void aml_append(Aml *parent_ctx, Aml *child)
 {
 }
 
+Aml *aml_return(Aml *val)
+{
+    return NULL;
+}
+
+Aml *aml_method(const char *name, int arg_count, AmlSerializeFlag sflag)
+{
+    return NULL;
+}
+
 Aml *aml_resource_template(void)
 {
     return NULL;
diff --git a/hw/display/acpi-vga-stub.c b/hw/display/acpi-vga-stub.c
new file mode 100644
index 0000000000..a9b0ecf76d
--- /dev/null
+++ b/hw/display/acpi-vga-stub.c
@@ -0,0 +1,7 @@
+#include "qemu/osdep.h"
+#include "hw/acpi/acpi_aml_interface.h"
+#include "vga_int.h"
+
+void build_vga_aml(AcpiDevAmlIf *adev, Aml *scope)
+{
+}
diff --git a/hw/display/acpi-vga.c b/hw/display/acpi-vga.c
new file mode 100644
index 0000000000..f0e9ef1fcf
--- /dev/null
+++ b/hw/display/acpi-vga.c
@@ -0,0 +1,26 @@
+#include "qemu/osdep.h"
+#include "hw/acpi/acpi_aml_interface.h"
+#include "hw/pci/pci.h"
+#include "vga_int.h"
+
+void build_vga_aml(AcpiDevAmlIf *adev, Aml *scope)
+{
+    int s3d = 0;
+    Aml *method;
+
+    if (object_dynamic_cast(OBJECT(adev), "qxl-vga")) {
+        s3d = 3;
+    }
+
+    method = aml_method("_S1D", 0, AML_NOTSERIALIZED);
+    aml_append(method, aml_return(aml_int(0)));
+    aml_append(scope, method);
+
+    method = aml_method("_S2D", 0, AML_NOTSERIALIZED);
+    aml_append(method, aml_return(aml_int(0)));
+    aml_append(scope, method);
+
+    method = aml_method("_S3D", 0, AML_NOTSERIALIZED);
+    aml_append(method, aml_return(aml_int(s3d)));
+    aml_append(scope, method);
+}
diff --git a/hw/display/vga-pci.c b/hw/display/vga-pci.c
index 3e5bc259f7..9a91de7ed1 100644
--- a/hw/display/vga-pci.c
+++ b/hw/display/vga-pci.c
@@ -35,6 +35,7 @@
 #include "hw/loader.h"
 #include "hw/display/edid.h"
 #include "qom/object.h"
+#include "hw/acpi/acpi_aml_interface.h"
 
 enum vga_pci_flags {
     PCI_VGA_FLAG_ENABLE_MMIO = 1,
@@ -354,11 +355,13 @@ static void vga_pci_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
     PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
+    AcpiDevAmlIfClass *adevc = ACPI_DEV_AML_IF_CLASS(klass);
 
     k->vendor_id = PCI_VENDOR_ID_QEMU;
     k->device_id = PCI_DEVICE_ID_QEMU_VGA;
     dc->vmsd = &vmstate_vga_pci;
     set_bit(DEVICE_CATEGORY_DISPLAY, dc->categories);
+    adevc->build_dev_aml = build_vga_aml;
 }
 
 static const TypeInfo vga_pci_type_info = {
@@ -369,6 +372,7 @@ static const TypeInfo vga_pci_type_info = {
     .class_init = vga_pci_class_init,
     .interfaces = (InterfaceInfo[]) {
         { INTERFACE_CONVENTIONAL_PCI_DEVICE },
+        { TYPE_ACPI_DEV_AML_IF },
         { },
     },
 };
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 4f54b61904..26932b4e2c 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -430,7 +430,6 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
         bool hotpluggbale_slot = false;
         bool bridge_in_acpi = false;
         bool cold_plugged_bridge = false;
-        bool is_vga = false;
 
         if (pdev) {
             pc = PCI_DEVICE_GET_CLASS(pdev);
@@ -440,8 +439,6 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
                 continue;
             }
 
-            is_vga = pc->class_id == PCI_CLASS_DISPLAY_VGA;
-
             /*
              * Cold plugged bridges aren't themselves hot-pluggable.
              * Hotplugged bridges *are* hot-pluggable.
@@ -489,28 +486,7 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
             aml_append(dev, aml_pci_device_dsm());
         }
 
-        if (is_vga) {
-            /* add VGA specific AML methods */
-            int s3d;
-
-            if (object_dynamic_cast(OBJECT(pdev), "qxl-vga")) {
-                s3d = 3;
-            } else {
-                s3d = 0;
-            }
-
-            method = aml_method("_S1D", 0, AML_NOTSERIALIZED);
-            aml_append(method, aml_return(aml_int(0)));
-            aml_append(dev, method);
-
-            method = aml_method("_S2D", 0, AML_NOTSERIALIZED);
-            aml_append(method, aml_return(aml_int(0)));
-            aml_append(dev, method);
-
-            method = aml_method("_S3D", 0, AML_NOTSERIALIZED);
-            aml_append(method, aml_return(aml_int(s3d)));
-            aml_append(dev, method);
-        }
+        call_dev_aml_func(DEVICE(pdev), dev);
 
         bridge_in_acpi =  cold_plugged_bridge && pcihp_bridge_en;
         if (bridge_in_acpi) {
diff --git a/hw/display/meson.build b/hw/display/meson.build
index adc53dd8b6..7a725ed80e 100644
--- a/hw/display/meson.build
+++ b/hw/display/meson.build
@@ -38,10 +38,21 @@ softmmu_ss.add(when: 'CONFIG_NEXTCUBE', if_true: files('next-fb.c'))
 
 specific_ss.add(when: 'CONFIG_VGA', if_true: files('vga.c'))
 
+if (config_all_devices.has_key('CONFIG_VGA_CIRRUS') or
+    config_all_devices.has_key('CONFIG_VGA_PCI') or
+    config_all_devices.has_key('CONFIG_VMWARE_VGA') or
+    config_all_devices.has_key('CONFIG_ATI_VGA')
+   )
+  softmmu_ss.add(when: 'CONFIG_ACPI', if_true: files('acpi-vga.c'),
+                                      if_false: files('acpi-vga-stub.c'))
+endif
+
 if config_all_devices.has_key('CONFIG_QXL')
   qxl_ss = ss.source_set()
   qxl_ss.add(when: 'CONFIG_QXL', if_true: [files('qxl.c', 'qxl-logger.c', 'qxl-render.c'),
                                            pixman, spice])
+  qxl_ss.add(when: 'CONFIG_ACPI', if_true: files('acpi-vga.c'),
+                                  if_false: files('acpi-vga-stub.c'))
   hw_display_modules += {'qxl': qxl_ss}
 endif
 
@@ -52,6 +63,7 @@ softmmu_ss.add(when: 'CONFIG_ARTIST', if_true: files('artist.c'))
 
 softmmu_ss.add(when: [pixman, 'CONFIG_ATI_VGA'], if_true: files('ati.c', 'ati_2d.c', 'ati_dbg.c'))
 
+
 if config_all_devices.has_key('CONFIG_VIRTIO_GPU')
   virtio_gpu_ss = ss.source_set()
   virtio_gpu_ss.add(when: 'CONFIG_VIRTIO_GPU',
@@ -87,14 +99,19 @@ if config_all_devices.has_key('CONFIG_VIRTIO_VGA')
                     if_true: [files('virtio-vga.c'), pixman])
   virtio_vga_ss.add(when: 'CONFIG_VHOST_USER_VGA',
                     if_true: files('vhost-user-vga.c'))
+  virtio_vga_ss.add(when: 'CONFIG_ACPI', if_true: files('acpi-vga.c'),
+                                         if_false: files('acpi-vga-stub.c'))
   hw_display_modules += {'virtio-vga': virtio_vga_ss}
 
   virtio_vga_gl_ss = ss.source_set()
   virtio_vga_gl_ss.add(when: ['CONFIG_VIRTIO_VGA', virgl, opengl],
                        if_true: [files('virtio-vga-gl.c'), pixman])
+  virtio_vga_gl_ss.add(when: 'CONFIG_ACPI', if_true: files('acpi-vga.c'),
+                                            if_false: files('acpi-vga-stub.c'))
   hw_display_modules += {'virtio-vga-gl': virtio_vga_gl_ss}
 endif
 
 specific_ss.add(when: 'CONFIG_OMAP', if_true: files('omap_lcdc.c'))
 
+softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('acpi-vga-stub.c'))
 modules += { 'hw-display': hw_display_modules }
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 45/83] tests: acpi: whitelist DSDT before generating PCI-ISA bridge AML automatically
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (43 preceding siblings ...)
  2022-11-07 22:51 ` [PULL v4 44/83] acpi: pc: vga: use AcpiDevAmlIf interface to build VGA device descriptors Michael S. Tsirkin
@ 2022-11-07 22:51 ` Michael S. Tsirkin
  2022-11-08 13:36   ` Igor Mammedov
  2022-11-07 22:51 ` [PULL v4 46/83] acpi: pc/q35: drop ad-hoc PCI-ISA bridge AML routines and let bus ennumeration generate AML Michael S. Tsirkin
                   ` (39 subsequent siblings)
  84 siblings, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:51 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Igor Mammedov, Ani Sinha

From: Igor Mammedov <imammedo@redhat.com>

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20221017102146.2254096-3-imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/qtest/bios-tables-test-allowed-diff.h | 34 +++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index dfb8523c8b..570b17478e 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1 +1,35 @@
 /* List of comma-separated changed AML files to ignore */
+"tests/data/acpi/pc/DSDT",
+"tests/data/acpi/pc/DSDT.acpierst",
+"tests/data/acpi/pc/DSDT.acpihmat",
+"tests/data/acpi/pc/DSDT.bridge",
+"tests/data/acpi/pc/DSDT.cphp",
+"tests/data/acpi/pc/DSDT.dimmpxm",
+"tests/data/acpi/pc/DSDT.hpbridge",
+"tests/data/acpi/pc/DSDT.hpbrroot",
+"tests/data/acpi/pc/DSDT.ipmikcs",
+"tests/data/acpi/pc/DSDT.memhp",
+"tests/data/acpi/pc/DSDT.nohpet",
+"tests/data/acpi/pc/DSDT.numamem",
+"tests/data/acpi/pc/DSDT.roothp",
+"tests/data/acpi/q35/DSDT",
+"tests/data/acpi/q35/DSDT.acpierst",
+"tests/data/acpi/q35/DSDT.acpihmat",
+"tests/data/acpi/q35/DSDT.applesmc",
+"tests/data/acpi/q35/DSDT.bridge",
+"tests/data/acpi/q35/DSDT.cphp",
+"tests/data/acpi/q35/DSDT.cxl",
+"tests/data/acpi/q35/DSDT.dimmpxm",
+"tests/data/acpi/q35/DSDT.ipmibt",
+"tests/data/acpi/q35/DSDT.ipmismbus",
+"tests/data/acpi/q35/DSDT.ivrs",
+"tests/data/acpi/q35/DSDT.memhp",
+"tests/data/acpi/q35/DSDT.mmio64",
+"tests/data/acpi/q35/DSDT.multi-bridge",
+"tests/data/acpi/q35/DSDT.nohpet",
+"tests/data/acpi/q35/DSDT.numamem",
+"tests/data/acpi/q35/DSDT.pvpanic-isa",
+"tests/data/acpi/q35/DSDT.tis.tpm12",
+"tests/data/acpi/q35/DSDT.tis.tpm2",
+"tests/data/acpi/q35/DSDT.viot",
+"tests/data/acpi/q35/DSDT.xapic",
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 46/83] acpi: pc/q35: drop ad-hoc PCI-ISA bridge AML routines and let bus ennumeration generate AML
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (44 preceding siblings ...)
  2022-11-07 22:51 ` [PULL v4 45/83] tests: acpi: whitelist DSDT before generating PCI-ISA bridge AML automatically Michael S. Tsirkin
@ 2022-11-07 22:51 ` Michael S. Tsirkin
  2022-11-17 21:51   ` Volker Rümelin
  2022-11-07 22:51 ` [PULL v4 47/83] tests: acpi: update expected DSDT after ISA bridge is moved directly under PCI host bridge Michael S. Tsirkin
                   ` (38 subsequent siblings)
  84 siblings, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:51 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Igor Mammedov, Ani Sinha, Marcel Apfelbaum,
	Paolo Bonzini, Richard Henderson, Eduardo Habkost

From: Igor Mammedov <imammedo@redhat.com>

PCI-ISA bridges that are built in PIIX/Q35 are building its own AML
using AcpiDevAmlIf interface. Now build_append_pci_bus_devices()
gained AcpiDevAmlIf interface support to get AML of devices atached
to PCI slots.
So drop ad-hoc build_q35_isa_bridge()/build_piix4_isa_bridge()
and let PCI bus enumeration to include PCI-ISA bridge AML
when it's enumerated by build_append_pci_bus_devices().

AML change is mostly contextual, which moves whole ISA hierarchy
directly under PCI host bridge instead of it being described
as separate \SB.PCI0.ISA block.

Note:
If bus/slot that hosts ISA bridge has BSEL set, it will gain new
ASUN and _DMS entries (i.e. acpi-index support, but it should not
cause any functional change and that is fine from PCI Firmware
spec point of view), potentially it's possible to suppress that
by adding a flag to PCIDevice but I don't see a reason to do that
yet, I'd rather treat bridge just as any other PCI device if it's
possible.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20221017102146.2254096-4-imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/i386/acpi-build.c | 75 --------------------------------------------
 hw/isa/lpc_ich9.c    | 23 ++++++++++++++
 hw/isa/piix3.c       | 17 +++++++++-
 3 files changed, 39 insertions(+), 76 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 26932b4e2c..e1483bb11a 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -435,10 +435,6 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
             pc = PCI_DEVICE_GET_CLASS(pdev);
             dc = DEVICE_GET_CLASS(pdev);
 
-            if (pc->class_id == PCI_CLASS_BRIDGE_ISA) {
-                continue;
-            }
-
             /*
              * Cold plugged bridges aren't themselves hot-pluggable.
              * Hotplugged bridges *are* hot-pluggable.
@@ -1006,7 +1002,6 @@ static void build_piix4_pci0_int(Aml *table)
 {
     Aml *dev;
     Aml *crs;
-    Aml *field;
     Aml *method;
     uint32_t irqs;
     Aml *sb_scope = aml_scope("_SB");
@@ -1015,13 +1010,6 @@ static void build_piix4_pci0_int(Aml *table)
     aml_append(pci0_scope, build_prt(true));
     aml_append(sb_scope, pci0_scope);
 
-    field = aml_field("PCI0.ISA.P40C", AML_BYTE_ACC, AML_NOLOCK, AML_PRESERVE);
-    aml_append(field, aml_named_field("PRQ0", 8));
-    aml_append(field, aml_named_field("PRQ1", 8));
-    aml_append(field, aml_named_field("PRQ2", 8));
-    aml_append(field, aml_named_field("PRQ3", 8));
-    aml_append(sb_scope, field);
-
     aml_append(sb_scope, build_irq_status_method());
     aml_append(sb_scope, build_iqcr_method(true));
 
@@ -1125,7 +1113,6 @@ static Aml *build_q35_routing_table(const char *str)
 
 static void build_q35_pci0_int(Aml *table)
 {
-    Aml *field;
     Aml *method;
     Aml *sb_scope = aml_scope("_SB");
     Aml *pci0_scope = aml_scope("PCI0");
@@ -1162,18 +1149,6 @@ static void build_q35_pci0_int(Aml *table)
     aml_append(pci0_scope, method);
     aml_append(sb_scope, pci0_scope);
 
-    field = aml_field("PCI0.ISA.PIRQ", AML_BYTE_ACC, AML_NOLOCK, AML_PRESERVE);
-    aml_append(field, aml_named_field("PRQA", 8));
-    aml_append(field, aml_named_field("PRQB", 8));
-    aml_append(field, aml_named_field("PRQC", 8));
-    aml_append(field, aml_named_field("PRQD", 8));
-    aml_append(field, aml_reserved_field(0x20));
-    aml_append(field, aml_named_field("PRQE", 8));
-    aml_append(field, aml_named_field("PRQF", 8));
-    aml_append(field, aml_named_field("PRQG", 8));
-    aml_append(field, aml_named_field("PRQH", 8));
-    aml_append(sb_scope, field);
-
     aml_append(sb_scope, build_irq_status_method());
     aml_append(sb_scope, build_iqcr_method(false));
 
@@ -1238,54 +1213,6 @@ static Aml *build_q35_dram_controller(const AcpiMcfgInfo *mcfg)
     return dev;
 }
 
-static void build_q35_isa_bridge(Aml *table)
-{
-    Aml *dev;
-    Aml *scope;
-    Object *obj;
-    bool ambiguous;
-
-    /*
-     * temporarily fish out isa bridge, build_q35_isa_bridge() will be dropped
-     * once PCI is converted to AcpiDevAmlIf and would be ble to generate
-     * AML for bridge itself
-     */
-    obj = object_resolve_path_type("", TYPE_ICH9_LPC_DEVICE, &ambiguous);
-    assert(obj && !ambiguous);
-
-    scope =  aml_scope("_SB.PCI0");
-    dev = aml_device("ISA");
-    aml_append(dev, aml_name_decl("_ADR", aml_int(0x001F0000)));
-
-    call_dev_aml_func(DEVICE(obj), dev);
-    aml_append(scope, dev);
-    aml_append(table, scope);
-}
-
-static void build_piix4_isa_bridge(Aml *table)
-{
-    Aml *dev;
-    Aml *scope;
-    Object *obj;
-    bool ambiguous;
-
-    /*
-     * temporarily fish out isa bridge, build_piix4_isa_bridge() will be dropped
-     * once PCI is converted to AcpiDevAmlIf and would be ble to generate
-     * AML for bridge itself
-     */
-    obj = object_resolve_path_type("", TYPE_PIIX3_PCI_DEVICE, &ambiguous);
-    assert(obj && !ambiguous);
-
-    scope =  aml_scope("_SB.PCI0");
-    dev = aml_device("ISA");
-    aml_append(dev, aml_name_decl("_ADR", aml_int(0x00010000)));
-
-    call_dev_aml_func(DEVICE(obj), dev);
-    aml_append(scope, dev);
-    aml_append(table, scope);
-}
-
 static void build_x86_acpi_pci_hotplug(Aml *table, uint64_t pcihp_addr)
 {
     Aml *scope;
@@ -1465,7 +1392,6 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
         aml_append(sb_scope, dev);
         aml_append(dsdt, sb_scope);
 
-        build_piix4_isa_bridge(dsdt);
         if (pm->pcihp_bridge_en || pm->pcihp_root_en) {
             build_x86_acpi_pci_hotplug(dsdt, pm->pcihp_io_base);
         }
@@ -1510,7 +1436,6 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
 
         aml_append(dsdt, sb_scope);
 
-        build_q35_isa_bridge(dsdt);
         if (pm->pcihp_bridge_en) {
             build_x86_acpi_pci_hotplug(dsdt, pm->pcihp_io_base);
         }
diff --git a/hw/isa/lpc_ich9.c b/hw/isa/lpc_ich9.c
index 8694e58b21..0b0a83e080 100644
--- a/hw/isa/lpc_ich9.c
+++ b/hw/isa/lpc_ich9.c
@@ -809,6 +809,7 @@ static void ich9_send_gpe(AcpiDeviceIf *adev, AcpiEventStatusBits ev)
 
 static void build_ich9_isa_aml(AcpiDevAmlIf *adev, Aml *scope)
 {
+    Aml *field;
     BusChild *kid;
     ICH9LPCState *s = ICH9_LPC_DEVICE(adev);
     BusState *bus = BUS(s->isa_bus);
@@ -816,6 +817,28 @@ static void build_ich9_isa_aml(AcpiDevAmlIf *adev, Aml *scope)
     /* ICH9 PCI to ISA irq remapping */
     aml_append(scope, aml_operation_region("PIRQ", AML_PCI_CONFIG,
                                            aml_int(0x60), 0x0C));
+    /* Fields declarion has to happen *after* operation region */
+    field = aml_field("PIRQ", AML_BYTE_ACC, AML_NOLOCK, AML_PRESERVE);
+    aml_append(field, aml_named_field("PRQA", 8));
+    aml_append(field, aml_named_field("PRQB", 8));
+    aml_append(field, aml_named_field("PRQC", 8));
+    aml_append(field, aml_named_field("PRQD", 8));
+    aml_append(field, aml_reserved_field(0x20));
+    aml_append(field, aml_named_field("PRQE", 8));
+    aml_append(field, aml_named_field("PRQF", 8));
+    aml_append(field, aml_named_field("PRQG", 8));
+    aml_append(field, aml_named_field("PRQH", 8));
+    aml_append(scope, field);
+
+    /* hack: put fields into _SB scope for LNKx to find them */
+    aml_append(scope, aml_alias("PRQA", "\\_SB.PRQA"));
+    aml_append(scope, aml_alias("PRQB", "\\_SB.PRQB"));
+    aml_append(scope, aml_alias("PRQC", "\\_SB.PRQC"));
+    aml_append(scope, aml_alias("PRQD", "\\_SB.PRQD"));
+    aml_append(scope, aml_alias("PRQE", "\\_SB.PRQE"));
+    aml_append(scope, aml_alias("PRQF", "\\_SB.PRQF"));
+    aml_append(scope, aml_alias("PRQG", "\\_SB.PRQG"));
+    aml_append(scope, aml_alias("PRQH", "\\_SB.PRQH"));
 
     QTAILQ_FOREACH(kid, &bus->children, sibling) {
             call_dev_aml_func(DEVICE(kid->child), scope);
diff --git a/hw/isa/piix3.c b/hw/isa/piix3.c
index 808fd4eadf..f9b4af5c05 100644
--- a/hw/isa/piix3.c
+++ b/hw/isa/piix3.c
@@ -316,12 +316,27 @@ static void pci_piix3_realize(PCIDevice *dev, Error **errp)
 
 static void build_pci_isa_aml(AcpiDevAmlIf *adev, Aml *scope)
 {
+    Aml *field;
     BusChild *kid;
     BusState *bus = qdev_get_child_bus(DEVICE(adev), "isa.0");
 
     /* PIIX PCI to ISA irq remapping */
     aml_append(scope, aml_operation_region("P40C", AML_PCI_CONFIG,
-                                         aml_int(0x60), 0x04));
+                                           aml_int(0x60), 0x04));
+    /* Fields declarion has to happen *after* operation region */
+    field = aml_field("P40C", AML_BYTE_ACC, AML_NOLOCK, AML_PRESERVE);
+    aml_append(field, aml_named_field("PRQ0", 8));
+    aml_append(field, aml_named_field("PRQ1", 8));
+    aml_append(field, aml_named_field("PRQ2", 8));
+    aml_append(field, aml_named_field("PRQ3", 8));
+    aml_append(scope, field);
+
+    /* hack: put fields into _SB scope for LNKx to find them */
+    aml_append(scope, aml_alias("PRQ0", "\\_SB.PRQ0"));
+    aml_append(scope, aml_alias("PRQ1", "\\_SB.PRQ1"));
+    aml_append(scope, aml_alias("PRQ2", "\\_SB.PRQ2"));
+    aml_append(scope, aml_alias("PRQ3", "\\_SB.PRQ3"));
+
     QTAILQ_FOREACH(kid, &bus->children, sibling) {
         call_dev_aml_func(DEVICE(kid->child), scope);
     }
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 47/83] tests: acpi: update expected DSDT after ISA bridge is moved directly under PCI host bridge
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (45 preceding siblings ...)
  2022-11-07 22:51 ` [PULL v4 46/83] acpi: pc/q35: drop ad-hoc PCI-ISA bridge AML routines and let bus ennumeration generate AML Michael S. Tsirkin
@ 2022-11-07 22:51 ` Michael S. Tsirkin
  2022-11-07 22:51 ` [PULL v4 48/83] tests: acpi: whitelist DSDT before generating ICH9_SMB AML automatically Michael S. Tsirkin
                   ` (37 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:51 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Igor Mammedov, Ani Sinha

From: Igor Mammedov <imammedo@redhat.com>

example of the change for PC machine with hotplug disabled on root buss (no BSEL case):

 -        Field (PCI0.ISA.P40C, ByteAcc, NoLock, Preserve)
 +        Field (S08.P40C, ByteAcc, NoLock, Preserve)

 ===
 -    Scope (_SB.PCI0)
 -    {
 -        Device (ISA)
 -        {
 -            Name (_ADR, 0x00010000)  // _ADR: Address
 -            OperationRegion (P40C, PCI_Config, 0x60, 0x04)
 ...
 -        }
 -    }
 -
      Scope (_SB)
 ===
 +            Device (S08)
 +            {
 +                Name (_ADR, 0x00010000)  // _ADR: Address
 +                OperationRegion (P40C, PCI_Config, 0x60, 0x04)
 ...
 +            }
 +
              Device (S10)
              {
                  Name (_ADR, 0x00020000)  // _ADR: Address

with hotplug enabled on root bus (i.e. bus has BSEL configured),
a following addtional entries will be seen:

 +                Name (ASUN, One)
 +                Method (_DSM, 4, Serialized)  // _DSM: Device-Specific Method
 +                {
 +                    Local0 = Package (0x02)
 +                        {
 +                            BSEL,
 +                            ASUN
 +                        }
 +                    Return (PDSM (Arg0, Arg1, Arg2, Arg3, Local0))
 +                }

similar changes are expected for Q35 modulo:

 -        Field (PCI0.ISA.PIRQ, ByteAcc, NoLock, Preserve)
 +        Field (SF8.PIRQ, ByteAcc, NoLock, Preserve)

and bridge address

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20221017102146.2254096-5-imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/qtest/bios-tables-test-allowed-diff.h |  34 --------------------
 tests/data/acpi/pc/DSDT                     | Bin 6422 -> 6496 bytes
 tests/data/acpi/pc/DSDT.acpierst            | Bin 6382 -> 6456 bytes
 tests/data/acpi/pc/DSDT.acpihmat            | Bin 7747 -> 7821 bytes
 tests/data/acpi/pc/DSDT.bridge              | Bin 9496 -> 9570 bytes
 tests/data/acpi/pc/DSDT.cphp                | Bin 6886 -> 6960 bytes
 tests/data/acpi/pc/DSDT.dimmpxm             | Bin 8076 -> 8150 bytes
 tests/data/acpi/pc/DSDT.hpbridge            | Bin 6382 -> 6456 bytes
 tests/data/acpi/pc/DSDT.hpbrroot            | Bin 3069 -> 3107 bytes
 tests/data/acpi/pc/DSDT.ipmikcs             | Bin 6494 -> 6568 bytes
 tests/data/acpi/pc/DSDT.memhp               | Bin 7781 -> 7855 bytes
 tests/data/acpi/pc/DSDT.nohpet              | Bin 6280 -> 6354 bytes
 tests/data/acpi/pc/DSDT.numamem             | Bin 6428 -> 6502 bytes
 tests/data/acpi/pc/DSDT.roothp              | Bin 6656 -> 6694 bytes
 tests/data/acpi/q35/DSDT                    | Bin 8320 -> 8418 bytes
 tests/data/acpi/q35/DSDT.acpierst           | Bin 8337 -> 8435 bytes
 tests/data/acpi/q35/DSDT.acpihmat           | Bin 9645 -> 9743 bytes
 tests/data/acpi/q35/DSDT.applesmc           | Bin 8366 -> 8464 bytes
 tests/data/acpi/q35/DSDT.bridge             | Bin 11449 -> 11547 bytes
 tests/data/acpi/q35/DSDT.cphp               | Bin 8784 -> 8882 bytes
 tests/data/acpi/q35/DSDT.cxl                | Bin 9646 -> 9744 bytes
 tests/data/acpi/q35/DSDT.dimmpxm            | Bin 9974 -> 10072 bytes
 tests/data/acpi/q35/DSDT.ipmibt             | Bin 8395 -> 8493 bytes
 tests/data/acpi/q35/DSDT.ipmismbus          | Bin 8409 -> 8507 bytes
 tests/data/acpi/q35/DSDT.ivrs               | Bin 8337 -> 8435 bytes
 tests/data/acpi/q35/DSDT.memhp              | Bin 9679 -> 9777 bytes
 tests/data/acpi/q35/DSDT.mmio64             | Bin 9450 -> 9548 bytes
 tests/data/acpi/q35/DSDT.multi-bridge       | Bin 8640 -> 8738 bytes
 tests/data/acpi/q35/DSDT.nohpet             | Bin 8178 -> 8276 bytes
 tests/data/acpi/q35/DSDT.numamem            | Bin 8326 -> 8424 bytes
 tests/data/acpi/q35/DSDT.pvpanic-isa        | Bin 8421 -> 8519 bytes
 tests/data/acpi/q35/DSDT.tis.tpm12          | Bin 8926 -> 9024 bytes
 tests/data/acpi/q35/DSDT.tis.tpm2           | Bin 8952 -> 9050 bytes
 tests/data/acpi/q35/DSDT.viot               | Bin 9429 -> 9527 bytes
 tests/data/acpi/q35/DSDT.xapic              | Bin 35683 -> 35781 bytes
 35 files changed, 34 deletions(-)

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index 570b17478e..dfb8523c8b 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1,35 +1 @@
 /* List of comma-separated changed AML files to ignore */
-"tests/data/acpi/pc/DSDT",
-"tests/data/acpi/pc/DSDT.acpierst",
-"tests/data/acpi/pc/DSDT.acpihmat",
-"tests/data/acpi/pc/DSDT.bridge",
-"tests/data/acpi/pc/DSDT.cphp",
-"tests/data/acpi/pc/DSDT.dimmpxm",
-"tests/data/acpi/pc/DSDT.hpbridge",
-"tests/data/acpi/pc/DSDT.hpbrroot",
-"tests/data/acpi/pc/DSDT.ipmikcs",
-"tests/data/acpi/pc/DSDT.memhp",
-"tests/data/acpi/pc/DSDT.nohpet",
-"tests/data/acpi/pc/DSDT.numamem",
-"tests/data/acpi/pc/DSDT.roothp",
-"tests/data/acpi/q35/DSDT",
-"tests/data/acpi/q35/DSDT.acpierst",
-"tests/data/acpi/q35/DSDT.acpihmat",
-"tests/data/acpi/q35/DSDT.applesmc",
-"tests/data/acpi/q35/DSDT.bridge",
-"tests/data/acpi/q35/DSDT.cphp",
-"tests/data/acpi/q35/DSDT.cxl",
-"tests/data/acpi/q35/DSDT.dimmpxm",
-"tests/data/acpi/q35/DSDT.ipmibt",
-"tests/data/acpi/q35/DSDT.ipmismbus",
-"tests/data/acpi/q35/DSDT.ivrs",
-"tests/data/acpi/q35/DSDT.memhp",
-"tests/data/acpi/q35/DSDT.mmio64",
-"tests/data/acpi/q35/DSDT.multi-bridge",
-"tests/data/acpi/q35/DSDT.nohpet",
-"tests/data/acpi/q35/DSDT.numamem",
-"tests/data/acpi/q35/DSDT.pvpanic-isa",
-"tests/data/acpi/q35/DSDT.tis.tpm12",
-"tests/data/acpi/q35/DSDT.tis.tpm2",
-"tests/data/acpi/q35/DSDT.viot",
-"tests/data/acpi/q35/DSDT.xapic",
diff --git a/tests/data/acpi/pc/DSDT b/tests/data/acpi/pc/DSDT
index da2a3e5c0551ac2d1d8a0a40b92d3235d5757475..5b4624cb68c78eab6e356987974b9bee6c9a1785 100644
GIT binary patch
delta 186
zcmbPc^uUPACD<h-L6U)i(O@E1$7W3?Uq&VmlgSm#8#eo~XmfJ8+=_`0c8VA9xjDI+
z*H+9`A=tnoo+IATC5Q*8hk?T}IMk1E@<su<<miR~69Z=^t^_WY=te0Jn=v3L(0~I-
v7=j2R5Mj*01`>?X0~rcpfw+cnt|5qP1m_xoxW;g<@y1<wjGOoIMsNcF?p`uF

delta 139
zcmaE0G|h<1CD<iIOp<|tam_@o4h=U+z4&0K_yA{5gXktt3D021c#e2SmmnUX0*2^@
z022deCawf7md&$x@)$Qx^kZc5Gn@R7af4cPql!K=NE=8CNEu^5P@n+^kT3)hMj*nN
dWAj8NZB8!VoiQLw1YCDcKE`dknTs!s8vqocC7J*L

diff --git a/tests/data/acpi/pc/DSDT.acpierst b/tests/data/acpi/pc/DSDT.acpierst
index abcd6d9d309a8f9a9080edc64df7a6c195b5885f..99461b771eec2043263b5bc3c109a08969a99af9 100644
GIT binary patch
delta 186
zcmaE7xWkCcCD<jzLXv@jv2Y?+$7W3?Uq&VmlgSm#8#eo~XmfJ;T#tzlc8V8pxjwm>
z*H+9`A=tnoo+IATC5Q*8hk?T}IMk1EG9$lSa&$w0iGec{R{|GHbfXlA%@`0AXuts^
u3_*kuh%n}00}00HfeZz)KwLvO*AT=tf^&^PTw^%bc;l`-#?AY94Y&c&fia{2

delta 139
zcmdmC^v;mWCD<k8odg2|<GYDm9U5+udhx+d@d3`B2GLEP5}v`1@f`7vE<rp%1q{&*
z0VW2{Ok4?EESqQX<S}lX=*P(9XEymE;|8_pMiqT#kT#GOkTS-Apg;o-AYlk1j6j4j
d$L5Jl+MHaj+hahM2>5QBe2m+6GZ&v8Hvqa2Cba+n

diff --git a/tests/data/acpi/pc/DSDT.acpihmat b/tests/data/acpi/pc/DSDT.acpihmat
index 884d4871a2a0b87db2c0dd7e6f32486ba9b7e424..b84f3b47c37e427b927c36036307db6e83a843e5 100644
GIT binary patch
delta 186
zcmX?X(`(D+66_MvE62dV_;w;!$7W3?Uq&VmlgSm#8#eo~MDTIB+=_`0c8VA9xjA`*
zkgb@jLa>2FJV(5vOArrG4+Do|aHt>S<cnf*$<Yk~CI-$-TnSt((T!3dHe*0gpaBPv
vFa!}sAi|h~4I~(&2Qn1I0&xxDTtg7o2+lPEagE_z<BhxW7&pHbir@wS&T=!<

delta 139
zcmeCRJ#53}66_M<EXTmWm^qQFL&Hr{FFx2QKET=2AiBv@!ZX-0o+IATC5Q*8fFZge
zz{J3ri7SDNW%F^tJjRU^{TP}2%qBl%+@KcSsG`pd(gxB3QpOk%6llN!Bn&}>5r{D6
d*gTObf{)90XAH;^0oR?AKMUAyHWv=#1^^j}CO`lH

diff --git a/tests/data/acpi/pc/DSDT.bridge b/tests/data/acpi/pc/DSDT.bridge
index 31a79aa47673c14f7e564475c39bd38c872f8165..6771620078086e42b445474b16797094e0d0a801 100644
GIT binary patch
delta 167
zcmbQ?^~j6MCD<h-NtJ<tQFbC%$7W3?Uq&VmlgSm#8#eo~XmfJ6&0>s+4|a+d=$$#a
zh1ZVBRblcxKBMI5h5!=-XC|%$E|%y<DG-}6ASlp)14tNx2qO?-%)tf{jL`!b0Ahi-
dhH$PSh-(Dr8iBaRaIW#jU3rX~5Aa5a0{|E1F2n!;

delta 140
zcmaFlHN%U`CD<iILY0Amappv>4h=U+z4&0K_yA{5gXktt3D021c#e2SmmnUX0*2^@
z022deCawf7md$f`@)$Qx^kZc5Gn@R7af4cPql!K=NE=8CNEu^5P@n+^kT3)hMj*nN
eWAj8NZBFj~O2!zFEdp(olaF)TZRX(%69)iL9VOrZ

diff --git a/tests/data/acpi/pc/DSDT.cphp b/tests/data/acpi/pc/DSDT.cphp
index 8b0cae4dbfbd083a43751a7c94dc5c76f89084a7..298fa1592676b3a2089b08eff5f764e04391b961 100644
GIT binary patch
delta 186
zcmaE6y1|UgCD<jzK$?Mpamqxlj?J1(zKl#BCX*|eH*EG}`OU@Uaw{f2*ePDX=jP;K
zep@kDg<u1Vc#e2SmmnUX9tIA_;7~ut$&EsC$<Yk~CI-$-TnSt((T!3dHe*0gpaBPv
vFa!}sAi|h~4I~(&2Qn1I0&xxDTtg7o2+lPEagE_z<BhxW7&rIvM{olGeS|XJ

delta 139
zcmdmB_RN&aCD<k8nG^#9qu@lY4h=U+z4&0K_yA{5gXktt3D021c#e2SmmnUX0*2^@
z022deCawf7md#mwd5jw;`Y|&3nN5DkxIrztQAM8_qz$A6q>M2jDA0fdNEm_$BM@QC
dv3VlXZ!Rw1oiQLw1YCDcp2lmt`4)c|Hvml~Cp`cF

diff --git a/tests/data/acpi/pc/DSDT.dimmpxm b/tests/data/acpi/pc/DSDT.dimmpxm
index 38865fb66747a6ee1cf2066b925be6eba064625a..9fcadb266b92fc4942621f0fd919703723de2fd9 100644
GIT binary patch
delta 186
zcmeCNzh=+n66_LkO`d^)v2G$)$7W3?Uq&VmlgSm#8#eo~Eac~MxfK&1>=ZBHb93@f
zVOud*g<u1Vc#e2SmmnUX9tIA_;7~ut$&R9O$<Yk~CI-$-TnSt((T!3dHe*0gpaBPv
vFa!}sAi|h~4I~(&2Qn1I0&xxDTtg7o2+lPEagE_z<BhxW7&n`UL~sKD*+VlF

delta 118
zcmca+-(%0^66_MvBhSFV_-i6phlZP^UVN}qe1Nm3L3ER+glDj0JV(5vOArrG0Yh{{
zfQf-K6ITKk%Vq)LJjRU^{TP}2%qBl%+`t>%sG`pd(gxBpdAE?{=7~%T`MG>|#(>Nd
QaNRlCM#y$^gGd-R0In+{5&!@I

diff --git a/tests/data/acpi/pc/DSDT.hpbridge b/tests/data/acpi/pc/DSDT.hpbridge
index abcd6d9d309a8f9a9080edc64df7a6c195b5885f..99461b771eec2043263b5bc3c109a08969a99af9 100644
GIT binary patch
delta 186
zcmaE7xWkCcCD<jzLXv@jv2Y?+$7W3?Uq&VmlgSm#8#eo~XmfJ;T#tzlc8V8pxjwm>
z*H+9`A=tnoo+IATC5Q*8hk?T}IMk1EG9$lSa&$w0iGec{R{|GHbfXlA%@`0AXuts^
u3_*kuh%n}00}00HfeZz)KwLvO*AT=tf^&^PTw^%bc;l`-#?AY94Y&c&fia{2

delta 139
zcmdmC^v;mWCD<k8odg2|<GYDm9U5+udhx+d@d3`B2GLEP5}v`1@f`7vE<rp%1q{&*
z0VW2{Ok4?EESqQX<S}lX=*P(9XEymE;|8_pMiqT#kT#GOkTS-Apg;o-AYlk1j6j4j
d$L5Jl+MHaj+hahM2>5QBe2m+6GZ&v8Hvqa2Cba+n

diff --git a/tests/data/acpi/pc/DSDT.hpbrroot b/tests/data/acpi/pc/DSDT.hpbrroot
index dd2c8c0c8c5bf9895eb524e094d5597515b4803e..b10b17cb1111f6b5d61da4aa1c754f2921a4ac1c 100644
GIT binary patch
delta 184
zcmew>zF30GCD<iInTLUaaq2{_4kizi$=XaCHZw9W;i&gkj)@O;iWhKJ4siA~h;HHz
zHZX|ih<9`eVu)^XlnXYnfbc+?8KN5kObncvxDvQnq8p_^Y{r0~Km!gSVF)6OK!h;|
p8%Qul4`eQg1>zdQxrQLF5sWJsXawRL!}x*58+YX~PCmeG3jp{LFk1is

delta 146
zcmZ21@mHM7CD<k8FE;}NBhy5#4h=U+z4&0K_yA{5gXktt3D021c#e2SmmnUX0*2^@
z022deCawf7mdzDhd5p{ger6M=Y*34CRMBS!X#pt*DPar<3N+vV5{4kc2t*ijY<6K>
b!Xf3s8UwOGz=0KF2Y0Z60mO>QPr2;?&p0K@

diff --git a/tests/data/acpi/pc/DSDT.ipmikcs b/tests/data/acpi/pc/DSDT.ipmikcs
index e819ce69461a36b6fab5e2ffeba90dfe558724a7..aff3e9bbe1fdbf783c49d07a49c2e16317a2871e 100644
GIT binary patch
delta 166
zcmca-w8EInCD<iog(L$5<C=+F9h(c8d>NTMOeP;--mrNZi#8{h&%K!VV5fKimwS^>
z^V*8}sstNY#B;<ux&-k6^)PTa28a4FPA=q=<BM(xFfnju;!5CRiEfmdT+1S9!xj(}
mXb__ZG62K^aSh>ILlD;p&NTvYjp1D5jk~rnZf4_);06GVOD_EY

delta 139
zcmZ2se9wr>CD<h-PLhFv(PSc5hlZc5UVN}qe1Nm3L3ER&tY@%eJV(5vOArrG0Yh{{
zfQf-K6ITKk%jQ`;+ZZ=a^kZc5Gn@R7af4cPql!K=NE=8CNEu^5P@n+^kT3)hMj*nN
dWAj8NZB8!VoiQLw1YCDcKE`dkIfyTe8vr6<CKLbw

diff --git a/tests/data/acpi/pc/DSDT.memhp b/tests/data/acpi/pc/DSDT.memhp
index 03a9decdc1563f5e759e34dffdab4bb34645d9b3..07b193b9939e2199bce5edc743de5494133c1e8c 100644
GIT binary patch
delta 186
zcmaEAv)-1=CD<ioy&MArqtZmKj?J1(zKl#BCX*|eH*EG}sp8{uxfK&1>=ZBHb93?~
zAzLw5g<u1Vc#e2SmmnUX9tIA_;7~ut$$?^W$<Yk~CI-$-TnSt((T!3dHe*0gpaBPv
vFa!}sAi|h~4I~(&2Qn1I0&xxDTtg7o2+lPEagE_z<BhxW7&r3@M{olGdIvIA

delta 139
zcmZ2)`_zWZCD<h-RgQsyao$9(4h=U+z4&0K_yA{5gXktt3D021c#e2SmmnUX0*2^@
z022deCawf7md!5(^B6Zy^kZc5Gn@R7af4cPql!K=NE=8CNEu^5P@n+^kT3)hMj*nN
dWAj9&Dn2gXoiQLw1YCDc78SJJ94Q>e4FGUoCS(8r

diff --git a/tests/data/acpi/pc/DSDT.nohpet b/tests/data/acpi/pc/DSDT.nohpet
index b413d9f31d483eca02f45ad09af2f556581ec381..ceee7722ceed8a7e8f61a37805a4c855f47244a8 100644
GIT binary patch
delta 186
zcmeA$ykyAb66_LkNrHia@z+GIj?J1(zKl#BCX*|eH*EG}najcDaw{f2*ePDX=jP<E
zJho!43c&^z@f`7vE<rp%Jq#R<!J&SPlN<TvlA{{}ObncvxDvQnq8p_^Y{r0~Km!gS
vVF)6OK!h;|8%Qul4`e8a1>zdQxrQLF5u9rT;u^!b#v6C#F>W^Cjo=0V7wI#4

delta 139
zcmca)*kQ=!66_MvA;G}FST>QXL&Hr{FFx2QKET=2AiBv@!ZX-0o+IATC5Q*8fFZge
zz{J3ri7SDNWit;?9^=M|evC|hW|JQ>ZcvMERMBS!X#;5iDPs%>3N+vV5{4kc2t*ij
dY@Wz8mxIf9XAH;^0oR?AEx2tr*YJjM0|5S~C4B$@

diff --git a/tests/data/acpi/pc/DSDT.numamem b/tests/data/acpi/pc/DSDT.numamem
index 9e701b2983f9ec45af9c26d801545c2b5d112a10..e03f4d07b86018d12def37b5397ba9673548e2b7 100644
GIT binary patch
delta 186
zcmbPZ^vsCMCD<h-O_G6u@zzAHj?J1(zKl#BCX*|eH*EG}G34ZOxfK&1>=ZBHb8~Vh
zudSG?La>2FJV(5vOArrG4+Do|aHt>S<coZA$<Yk~CI-$-TnSt((T!3dHe*0gpaBPv
vFa!}sAi|h~4I~(&2Qn1I0&xxDTtg7o2+lPEagE_z<BhxW7&jl`jo=0Vfr&FF

delta 118
zcmaE6G{=a`CD<iIMv{SnF>E4NhlZP^UVN}qe1Nm3L3ER+glDj0JV(5vOArrG0Yh{{
zfQf-K6ITKk%jN|<d5jw;`Y|&3nN5DkxPdpiQAM8_qz$BHvNw<9=7~&(oLs&;V?gE!
QxbB>MhTC?t0ACn407khWt^fc4

diff --git a/tests/data/acpi/pc/DSDT.roothp b/tests/data/acpi/pc/DSDT.roothp
index 8c3956c9ecc63133cc800f1d2d07a1392a35d548..418cc92e4c061ad21e1d281e0b5e6114e283b7d3 100644
GIT binary patch
delta 190
zcmZoLS!TlJ66_M9CdI(Oczz;R$7W3?Uq&VmlgSm#8#eo~Xmi$kJ&TDCc8V8pdKTd9
zX%OAS9c*9_&k^tF62uVQ<R}+xU;*Lr08L<sZU`_jaAx94;9`kxlmf9C1A+n#IDmv9
wh%f>X#vE)Q!5BS|xgZvZYY68Wg1AO7u3(@Mh;Izz2O4kOmB+Yw3U3Y%00)~hY5)KL

delta 133
zcmZ2x(qO{n66_MfAjQDIs56nPL&!~1FFx2QKET=2AiBv@!ZX-$@;z?x&AvRjj2kEV
zF*5m?O@7F@K`pvbMV}d@2&f=Fz{J3rF(4?=fCESvf(RoJVa&04B9k_!l>3PokdXrZ
X$04S22OAi~bHqEk1a02Qo67?Lm(C@O

diff --git a/tests/data/acpi/q35/DSDT b/tests/data/acpi/q35/DSDT
index 3870958969b1c0e29f9dd26a1256359e90897d3e..ea35dc5eba8433a8dcb54815f19ed6239f2534e7 100644
GIT binary patch
delta 229
zcmZp0eB{XG66_N4NP&TYal=L~Z^q4;ObLukO<|i)Gq2>^yg{gmm&-vRCO+6HUcf_s
zvW<eRLUfaZbg-L6JV(5vOArqOgFHiYLx5*cAQM*t7f*Df!Q@6E{d%^5pg_kMJ&-;S
z3&eGTbDcn3XE@gx#C3slT|it{IM)@#b%S%=KwNh?*B!+5fO9>fn@qf&T;eCUFg{@y
LVGD5f3t<2Npb9~@

delta 105
zcmaFl*x<<J66_MvpuoVu=(Lf`n^DMFP%l2%DL%m2(;&LZO~5nQak7ED_~eiBuQrD;
zCNeVhMQ+~Cw33fEy3tCX8KerRB7U-`y#D3{!6sfVKb9Dfb^$k*$#3Lr*+tj_oc%%=
E09ef)761SM

diff --git a/tests/data/acpi/q35/DSDT.acpierst b/tests/data/acpi/q35/DSDT.acpierst
index c9c18fa4e45288fb6c2182a793a8e6aff72b33d9..146269c68c68238a8be3aa67e049a85c0b8edc66 100644
GIT binary patch
delta 229
zcmbQ}_}P)mCD<k8vjPJHW57l(Z^q4;ObLukO<|i)Gq2>^yg{gmm&;KxCO+6HUcggf
zvW<eDLUfaZbg-L6JV(5vOArqOgFHiYLx5*cAQM*t7f*Df!Q@6E{d%^5pg_kMJ&-;S
z3&eGTbDcn3XE@gx#C3slT|it{IM)@#b%S%=KwNh?*B!+5fO9>fn@qf&T;eCUFg{@y
LVGD5f3t<2NiwZ%W

delta 105
zcmezDIMI>ICD<ioq5=a0<D-pS-i$)df_m}6PVoWGo(9oPZUUacj*}hb#V0c<yxJVX
zn8?V~7rA*m(@H+x=te7jW{@hNiulP<^7@+-1e<ud{8?i_+6CNMC%=*NV;5lyaP|vf
F0047{9x?y`

diff --git a/tests/data/acpi/q35/DSDT.acpihmat b/tests/data/acpi/q35/DSDT.acpihmat
index a32e90b5d913d07bc9ed9aca521420305559f357..4dd420b62fbcfdd21485fec2bafdea54f3fdb5a8 100644
GIT binary patch
delta 229
zcmZ4M-S5NY66_Mfug1W@n6r_~n{jg{QvxGXQ`qLy%q#geZxC89#O0t66CdmpFW@0R
zIaAeEA-c&yI@rx3o+IATC5VTCL7pMHA;2>zkclgSizm9#U~;37emz@2P@rRs9!MXE
z1>!ovxlSOiGo0%T;<~`OE+DQeoa+kWy1}__Ag(){>ki_2z_}jLO(xz>F7cCF7@x3<
Lumw2#g)jgB@by6W

delta 105
zcmeD8S?kT^66_MPR+WK)@&86HZ$=?!LB053r}zM8PlM<tHv!LJ$H}oO;*;f6Uu_Oy
zOk`y0i`=}OX(b<Tbfc9%Ge{LsMf~J475&W#g6oC2{8(Z@+6CNLCQGW=vWu_<IQxY#
F004x49q<4E

diff --git a/tests/data/acpi/q35/DSDT.applesmc b/tests/data/acpi/q35/DSDT.applesmc
index 5507b6b8f5e6ae4fd4275601eb85caf29bddc352..ff25d82ba24b5e792b9d87958aa1b162bc9e0de2 100644
GIT binary patch
delta 229
zcmZ4IIKhd_CD<iIK#_rgv2i1pH{<5%ObLukO<|jVGOy&@d{3yEms2SwKG-Q<z)fj#
zmV&K9bd#TKu$x6ZN4%p;5Dx=`JVSItfM-x36ITKkPjsWf<n==O^=tt_fsQeHAblVf
zi0cIBI)S*(aIQ0m>jLMxfVi%3t}BS^2IsngxbASSJBaH6=Xyjp=>+>a$8T<76k=o-
LVGD5f3t<2NSgJs!

delta 105
zcmbQ>w9b*sCD<ioodN>`qwz*AZ$=?U5xw|er}zM8PlM<t7h%s}$H@lr;+sFpOE7K@
zVN7IX>WkdGooOW>Z*-%TJ~K!eP)Yn`PkH^#34+bMoGdXQ?E-Erli$eMvWu_<IQxY#
F000}#9Tflo

diff --git a/tests/data/acpi/q35/DSDT.bridge b/tests/data/acpi/q35/DSDT.bridge
index a42eb674fa3469a815b1c91ba99b2d4cf85c9380..bde84efd1c5fcafee789781f4030d573002a886a 100644
GIT binary patch
delta 240
zcmdlPIXjBWCD<iIT9<)=F@7VLH{<3^rUXW&rm)SYnOE{{-XPS(%jNkvCO+6HUcmA3
zW*dbZ9);*82kBrpi+GNBN0%TT1_pVC=!O8#pg<<B1TLQFMuW+XLi+V=0YQO|F?t}4
zAQp)01m`+|xXy5{Gl=T~=emHnu5hj^i0cODx`DXvaIQOu>jCF_L^qjuJGsP9Zee^P
UC?XKx>=(iirx$E!5D#Jk0QJE|ng9R*

delta 116
zcmbOowKI~-CD<iorw#)H<EM>W-i$)df_m}6PVoWGo(9oPZUUacj+5iH#V0H1yxJVX
zn8?V~7rA*m(@H+x=te7jW{@hNiulPD+WMOl1e<ud+)u=Sv<vtj-~2`{heuFEAi&u#
Pgdt8Z*w6rIs9y*GI+7!-

diff --git a/tests/data/acpi/q35/DSDT.cphp b/tests/data/acpi/q35/DSDT.cphp
index 2d8cb603c9c937501247d32e6a5b2e79bb01eea3..537be7ca37c021e6a7f098e88c40ba178f3ec3e2 100644
GIT binary patch
delta 229
zcmccMvdNXpCD<iolM(|1qsT@sZ^q4;ObLukO<|i)Gq2>^yg?|4pUXiZCO+6HUcf_s
zGLMq2LUfaZbg-L6JV(5vOArqOgFHiYLx5*cAQM*t7f*Df!Q@6E{d%^5pg_kMJ&-;S
z3&eGTbDcn3XE@gx#C3slT|it{IM)@#b%S%=KwNh?*B!+5fO9>fn@qf&T;eCUFg{@y
LVGD5f3t<2N)oDPt

delta 104
zcmdnwdclRuCD<h-K#75YaoI*LZ$=?!LB053r}zM8PlM<tHv!LJ$4QFflMgDs+8n}|
z$jH<exp_O&N<QA`Mk{@0kRqUh_{oxr`kNC3gZR1pSYklB1>9IB?@_R27hwx<_6uPE
E08AJii~s-t

diff --git a/tests/data/acpi/q35/DSDT.cxl b/tests/data/acpi/q35/DSDT.cxl
index 20d0fb64ea52b2f4979105c934aadb6f416e2b71..aeea64d1cecad0ad011870ed5e04bfea1ae62510 100644
GIT binary patch
delta 229
zcmZ4IJ;8^|CD<iIK#hTcam7Y1Z^q4;ObLukO<|i)Gq2>^yg{f>h|5VSCO+6HUcgIn
zvWIHCLUfaZbg-L6JV(5vOArqOgFHiYLx5*cAQM*t7f*Df!Q@6E{d%^5pg_kMJ&-;S
z3&eGTbDcn3XE@gx#C3slT|it{IM)@#b%S%=KwNh?*B!+5fO9>fn@qf&T;eCUFg{@y
LVGD5f3t<2N6HY-N

delta 105
zcmbQ>v(B5#CD<ioohkzZqt`|*Z$=?!LB053r}zM8PlM<tHv!LJ$H{Rj;*;f7Uu_Oy
zOk`y0i`=}OX(b<Tbfc9%Ge{LsMf~J)75&W#f_*|<4(u@??E)TblYc13vx~3=IQxY#
F002z99vJ`t

diff --git a/tests/data/acpi/q35/DSDT.dimmpxm b/tests/data/acpi/q35/DSDT.dimmpxm
index b23339513af3be457c1974e0288fc4c6720690d8..ed11aefa4a56a8408bd4e0de83bf2373e6025449 100644
GIT binary patch
delta 229
zcmez7d&7^*CD<h-LY;wu@$W`1Z^q4;ObLukO<|i)Gq2>^yg}%PFqeZuOnk6Yynu)N
z<V|X}3eimt(!p*P@f`7vE<rpD4Dt-o4FR4(flOQpTs+Z@29q0w^y}FIf&v|5^g#MR
zED+ZT&UFHDo#9+(5Z49Hbpdf*;apb`*A32f199EqTz3%H1J3n`ZZh$9a*3bZ!uW(;
Lge}0?FN6UAoF78{

delta 123
zcmccN_sy5fCD<k8n;HWH<AjY|-i#W~f_m}6PVoWGo(9oPZUUacj`1AvjxIqw3=HxN
z(G3BfL4izM30ypr)6`yX4q;4WWa^9Dyq#$!A8&M{l|C~_A4t#S{i^z#69j(<bNR8v
WfD9CHW0{<!YRfLd7U1j`!T<oQ7$VgG

diff --git a/tests/data/acpi/q35/DSDT.ipmibt b/tests/data/acpi/q35/DSDT.ipmibt
index 8af2695edeeadbde2a1e8f04ab567f5c00b76945..d9d1e75c987acd42be4576688621be07b21f0e7b 100644
GIT binary patch
delta 230
zcmX@@xYmiwCD<iISCN5%@#sb_Z^q4Am=YM7n!+}Vv8?3V{9CArm&;ovCO+6HUcgyp
za=U`9LUfb2e6X8EJV(5vOArqOgFHiYLx5*cAQM*t7f*Df!Q?YS`t@u9L4l4jdLVru
z7KrNv=Q@G7&Ty_Xi0cC9x`4Q@aIPze>jvk#fw=B)t~-e90q1%|H+i!7dKzwSVbo`2
M7hwx<_6uPE0Ka}f-~a#s

delta 106
zcmZ4MblQ>2CD<k8v;qSIWBNueZ$=?MQN8$Jr}zM8PlM<tM^VpU$H@lr;+sFpn=)<=
zVN7IX>WkdGooOW>Z*-%TJ~K!eP)Yn`PkH^#34%?$Tz)JuApHVvER)~J*|LkU1vvYK
GFaQ91ARZV1

diff --git a/tests/data/acpi/q35/DSDT.ipmismbus b/tests/data/acpi/q35/DSDT.ipmismbus
index 479df48cd37af888317d63eac8f565f370201490..f6e124137721312722c820b5c084a143492945c9 100644
GIT binary patch
delta 229
zcmccVxZ8=#CD<jzT9JW)QEwxcH{<3^rUXW&rm)SYnOE{{-XL_Fm&-vRCO+6HUcf_s
z@>B&|h3F;+>0mdDc#e2SmmnSn26=|)h5*l?KqjsPE}rN{gUO9T`t@u9L4l4jdLVru
z7KrNv=Q@G7&Ty_Xi0cC9x`4Q@aIPze>jvk#fw=B)t~-e90q1%|H<@@lxx`OyVSK_a
L!WQ7{7s3DlR3kxi

delta 123
zcmdn(bkmW`CD<k8rUC;4<H3zw-i#W~f_m}6PVoWGo(9oPZUUacj`1AvjxIqw3=HxN
z(G3BfL4izM30yprJr!PW4q;4WWa^9Dyq#$!A8&M{l|C~_A4t#SMe_Qa69jMba`~~u
WfD9CHW0~wKZ_6&i7U1j`!T<nx-XaA6

diff --git a/tests/data/acpi/q35/DSDT.ivrs b/tests/data/acpi/q35/DSDT.ivrs
index c9c18fa4e45288fb6c2182a793a8e6aff72b33d9..146269c68c68238a8be3aa67e049a85c0b8edc66 100644
GIT binary patch
delta 229
zcmbQ}_}P)mCD<k8vjPJHW57l(Z^q4;ObLukO<|i)Gq2>^yg{gmm&;KxCO+6HUcggf
zvW<eDLUfaZbg-L6JV(5vOArqOgFHiYLx5*cAQM*t7f*Df!Q@6E{d%^5pg_kMJ&-;S
z3&eGTbDcn3XE@gx#C3slT|it{IM)@#b%S%=KwNh?*B!+5fO9>fn@qf&T;eCUFg{@y
LVGD5f3t<2NiwZ%W

delta 105
zcmezDIMI>ICD<ioq5=a0<D-pS-i$)df_m}6PVoWGo(9oPZUUacj*}hb#V0c<yxJVX
zn8?V~7rA*m(@H+x=te7jW{@hNiulP<^7@+-1e<ud{8?i_+6CNMC%=*NV;5lyaP|vf
F0047{9x?y`

diff --git a/tests/data/acpi/q35/DSDT.memhp b/tests/data/acpi/q35/DSDT.memhp
index a5730b8ab8abb2602334633cd6c760f79915e564..3e79ba7ac0b9a466fddd2213f32d20c522b8613b 100644
GIT binary patch
delta 229
zcmX@_z0rruCD<jzP>q3sap6WTZ^q4;ObLukO<|i)Gq2>^yg}%q5SN2OOnk6Yynu)N
z<Ze}4h3F;+>0mdDc#e2SmmnSn26=|)h5*l?KqjsPE}rN{gUO9T`t@u9L4l4jdLVru
z7KrNv=Q@G7&Ty_Xi0cC9x`4Q@aIPze>jvk#fw=B)t~-e90q1%|H<@@lxx`OyVSK_a
L!WQ7{7s3DlsQW?J

delta 105
zcmdn!bKaZFCD<k8yeb0&qsc}tZ$=?!LB053r}zM8PlM<tHv!LJ$I107;*%XzUu_Oy
zOk`y0i`=}OX(b<Tbfc9%Ge{LsMf~I$D*Brf1TPA4`LV=+v<tYgOtx0BWfx%!aP|vf
F004fR9yI^}

diff --git a/tests/data/acpi/q35/DSDT.mmio64 b/tests/data/acpi/q35/DSDT.mmio64
index a4293c20fed04d1ccfc4f4fc1ccf46ca81453854..afc260ebf4d91a2a9f7be5ff21968e99b1f9c5d1 100644
GIT binary patch
delta 229
zcmaFmdB%&&CD<jzN0otrF?J)DH{<3^rUXW&rm)SYnOE{{-XL^akjqgqCO+6HUcggf
z@>CT+h3F;+>0mdDc#e2SmmnSn26=|)h5*l?KqjsPE}rN{gUO9T`t@u9L4l4jdLVru
z7KrNv=Q@G7&Ty_Xi0cC9x`4Q@aIPze>jvk#fw=B)t~-e90q1%|H<@@lxx`OyVSK_a
L!WQ7{7s3Dl&Zt5L

delta 123
zcmX@(^~#gWCD<k8l?np`<A;r0-i#W~f_m}6PVoWGo(9oPZUUacj`1AvjxIqw3=HxN
z(G3BfL4izM30yprqf}mR4q;4WWa^9Dyq#$!A8&M{l|C~_A4t#S&C2?l69jJya{05y
WfD9CHXPxY-?8h#`7U1j`!T<oa`y#>s

diff --git a/tests/data/acpi/q35/DSDT.multi-bridge b/tests/data/acpi/q35/DSDT.multi-bridge
index 88bf47ab1805e4ef1dd8039933c5aa8144da37ac..7c14ce3a986fa06e88f3adc088faae54bdd2d8e4 100644
GIT binary patch
delta 250
zcmX@$yvT*iCD<iINr{1hQEDTXH{<3^rUXW&rm)SYnOE{{-XPS(%jIAa6CdmpFW_Ok
z*+wCgQ6aj?K|0vYBAz4O(Itq7fkB=jx*@<bD3FOOfr}@)(O`0;kbXT|Kv1A#j2=iM
zhy~(0!MRQ#t}~qL4C1=Lxh^2CE1c^J;<~}PZXm8Zoa+wadce6J(M=}aPA>72TNs~6
Yi%11H`-L#X=>;2F#)Ft3mI0Ur05;)9wEzGB

delta 144
zcmZ4Fa=@9(CD<k8fFc6}<Jyf}-i#W~f_m}6PVoWGo(9oPZUUacj`1AvjxIqw3=HxN
z(G3BfL4izM30yprWfWg;4q;4WWa^9Dyq#$!A8&M{l|C~_A4t#SQU(3Z34%?$Tz(QU
kAOi*5BsRa1%Vd-mkqU733t@=U3pTU_TH_Z2Vi|y008Qg3?f?J)

diff --git a/tests/data/acpi/q35/DSDT.nohpet b/tests/data/acpi/q35/DSDT.nohpet
index 6feed2ee106087d24d51f468df1b4d44e65143f0..60595f55fc72397e3ed9b1999dddd39cd98f89a8 100644
GIT binary patch
delta 229
zcmexlf5m~zCD<h-M1g^Uv2r7qH{<3^rUXW&rm)SYnOE{{-XQdahs!}BCO+6HUcf_s
z@;Z52h3F;+>0mdDc#e2SmmnSn26=|)h5*l?KqjsPE}rN{gUO9T`t@u9L4l4jdLVru
z7KrNv=Q@G7&Ty_Xi0cC9x`4Q@aIPze>jvk#fw=B)t~-e90q1%|H<@@lxx`OyVSK_a
L!WQ7{7s3Dl^Up#w

delta 123
zcmccO@X4ObCD<k8lRN_hBhN-IZ$=GgLB053r}zM8PlM<tHv!LJ$9Rr-N0%TT1_pVC
z=!O8#pg<<B1TLP*N%F5ZhcG5GGWA7n-p;g=k2ku}N}m~|52R=EZaMwU34&jExcpdR
WKn4o9u}qGWvt<`y3vl)eVE_PFiXrv@

diff --git a/tests/data/acpi/q35/DSDT.numamem b/tests/data/acpi/q35/DSDT.numamem
index 414b8af67bdf79238915de0d609ce45e150eb958..68e67eb91097c58365e3734c9b35d32796639d54 100644
GIT binary patch
delta 229
zcmZp3eBsFD66_N4LV<ySF=`{1H{<3^rUXW&rm)SYnOE{{-XPS$%jKXD6CdmpFW@0R
z*-61xA-c&yI@rx3o+IATC5VTCL7pMHA;2>zkclgSizm9#U~;37emz@2P@rRs9!MXE
z1>!ovxlSOiGo0%T;<~`OE+DQeoa+kWy1}__Ag(){>ki_2z_}jLO(xz>F7cCF7@x3<
Lumw2#g)jgBaBM+-

delta 123
zcmaFi*yhOP66_Mvroh0!_+}%QH=~BLpk92iQ+$B4r$Ka+n}BDqV?0N^qe~DE1A{z6
zbVGn=P#_am0vFHZZ}P7<hcG5GGWA7n-p;g=k2ku}N}m~|52R<Zzr6nD1i=noE<ctS
WkbwejER#RU*|LkU1vvYKFaQ8^2O?+y

diff --git a/tests/data/acpi/q35/DSDT.pvpanic-isa b/tests/data/acpi/q35/DSDT.pvpanic-isa
index 7277a0105035f984dc0a0f09747d441c21e62d1c..811cd27c7e7366e61683c7c5ef67f0996f395c25 100644
GIT binary patch
delta 230
zcmaFrc-)E0CD<jzU6Fx-F?b`FH{<4WObLukO<|i2SXT0F78P#d<#JVvi4S&)7w}b`
zyimbbA-c&`G1$!_o+IATC5VTCL7pMHA;2>zkclgSizm9#VDd8|{d%^5pg_kMJ&-;S
z3&eGTbDcn3XE@gx#C3slT|it{IM)@#b%S%=KwNh?*B!+5fO9>fn}DXdhHY+P^kifg
LVGD5f3t<2NL%%?L

delta 106
zcmX@^^wg2dCD<k8sR9E7<HwC$-i$&X;(GDHPVoWGo(9oP-eR7?j*|`K#W#PH_h;N3
z!kEa&)EBvVJJU)&-snaveP)m{ppy8>p7Q#e69k)hx%^mSK>7vTSSG)bvt<`y3vl)e
GVE_Qr;2#kH

diff --git a/tests/data/acpi/q35/DSDT.tis.tpm12 b/tests/data/acpi/q35/DSDT.tis.tpm12
index 253a66e65818d1df67b13d4b70f68add78adff98..7a213e3db230064cedf3a30b83b11128286eaeba 100644
GIT binary patch
delta 230
zcmccTdccj#CD<jzL79Pp@xeweZ^q5LSrQnTn!+~AvaRIX%p}^!%jM)66CdmpFW}`e
zd6tr`LUfapL$I4gJV(5vOArqOgFHiYLx5*cAQM*t7f*Df!Q{&#`t@u9L4l4jdLVru
z7KrNv=Q@G7&Ty_Xi0cC9x`4Q@aIPze>jvk#fw=B)t~-e90q1%|H#w+=1o&=lVKifA
M7hwx<_6uPE0Ky4DCIA2c

delta 124
zcmX@$cF&c|CD<k8o)QBC<E)Kb-i#WqW_t0#PVoWGo(9oP9;Tkbj`1AvjxIqw3=HxN
z(G3BfL4izM30ypz|H#`iZw_HhWMt}#+`OG>B_D5eqm@20NFzwo<WPD2%?W~iyj*@P
XF(4BK+*l_6kh5hMVGD5f3t<2Nq?jUv

diff --git a/tests/data/acpi/q35/DSDT.tis.tpm2 b/tests/data/acpi/q35/DSDT.tis.tpm2
index 76bd4661e6655d2a8f246f4bf38490df14da82ad..b55e828c6397f80c14de82f371fa34553008a875 100644
GIT binary patch
delta 230
zcmez2ddrQ=CD<h-N|}Lyv1TKeH{<53ED4NEO<|kO*;evxmJ#ja<??Zhi4S&)7jSW#
zyhX`YA-c)OG1$!_o+IATC5VTCL7pMHA;2>zkclgSizm9#VDeiL{d%^5pg_kMJ&-;S
z3&eGTbDcn3XE@gx#C3slT|it{IM)@#b%S%=KwNh?*B!+5fO9>fo4nLQ0(>{OFa|KQ
Mi?9Va`-Lz703b9$$p8QV

delta 124
zcmccR_QRFSCD<k8hY|w=qvJ*{Z$=G&bG`Urr}zM8PlM<tCv(qW$9Rr-N0%TT1_pVC
z=!O8#pg<<B1TLP<f8@iNH-|7LGBWi=Zr;wcl8-mK(Mq2gq!FZPa;Uui<^;h$UM@eD
X7?6nqZY-03$l0=sumw2#g)jgBtUn@}

diff --git a/tests/data/acpi/q35/DSDT.viot b/tests/data/acpi/q35/DSDT.viot
index 3f14b57f0797ab13fc4c90dd2949cf4bd6c30896..a8a93fe70d8e98ec0e66278b45d36393b75740ec 100644
GIT binary patch
delta 229
zcmccWx!sG)CD<jzT$O==(RU-4H{<3^rUXW&rm)SYnOE{{-XJttkjq6SCO+6HUcg6r
zvad>`LUfaZbg-L6JV(5vOArqOgFHiYLx5*cAQM*t7f*Df!Q@6E{d%^5pg_kMJ&-;S
z3&eGTbDcn3XE@gx#C3slT|it{IM)@#b%S%=KwNh?*B!+5fO9>fn@qf&T;eCUFg{@y
LVGD5f3t<2NOvpiI

delta 123
zcmdn)b=8y0CD<k8stN-G<NJ+V-i#W~f_m}6PVoWGo(9oPZUUacj`1AvjxIqw3=HxN
z(G3BfL4izM30yprT~%Ih4q;4WWa^9Dyq#$!A8&M{l|C~_A4t#SdCK~m69gv<ayfCv
WfD9Dy;+XtbsgYfTEx_3?gaH7Pw<7ld

diff --git a/tests/data/acpi/q35/DSDT.xapic b/tests/data/acpi/q35/DSDT.xapic
index baa88f6f219a2a47fe20ee0c08b92371cc7e2aa7..8211f5af8f9433b66b64768acb1de61a5716152a 100644
GIT binary patch
delta 231
zcmaDnjp^ugCN7s?myn~~3=E8E8@aq0H)k>>FfuiTZ9dJsl5g_{q0|a42ZfmUV5fKi
z5BbUR-L?wRO%Bq*ZWi$z@s2J*JPZu-4ABh%o<V_3TnSt}(TxU^8-?`i*#d$B9b@!B
z`amoY*9p#b0&$(;TxSs11<rK=ab4kDR}j|?&UFKE-Qiq!5Z43F^@wgV@pf{FpWMRu
Ngk6L!z}YW^0RZ#_LrVYv

delta 125
zcmX>)o$2v3CN7s?myqOc1_s6-8@aq0HJk<Y;)9*y1Drh#qMO_VJcAwMIpQ5%f_NAh
z<QbwH0z88PnYa?TcqU)&dc8S>F_Dp}FLLvCrj>lW(T!I6%piRrJ(G31^fxC6rdDwI
YvBZE36mVmie7@6`U4$*b*)N0v04#(g#Q*>R

-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 48/83] tests: acpi: whitelist DSDT before generating ICH9_SMB AML automatically
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (46 preceding siblings ...)
  2022-11-07 22:51 ` [PULL v4 47/83] tests: acpi: update expected DSDT after ISA bridge is moved directly under PCI host bridge Michael S. Tsirkin
@ 2022-11-07 22:51 ` Michael S. Tsirkin
  2022-11-07 22:51 ` [PULL v4 49/83] acpi: add get_dev_aml_func() helper Michael S. Tsirkin
                   ` (36 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:51 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Igor Mammedov, Ani Sinha

From: Igor Mammedov <imammedo@redhat.com>

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20221017102146.2254096-6-imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/qtest/bios-tables-test-allowed-diff.h | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index dfb8523c8b..fd5852776c 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1 +1,22 @@
 /* List of comma-separated changed AML files to ignore */
+"tests/data/acpi/q35/DSDT",
+"tests/data/acpi/q35/DSDT.acpierst",
+"tests/data/acpi/q35/DSDT.acpihmat",
+"tests/data/acpi/q35/DSDT.applesmc",
+"tests/data/acpi/q35/DSDT.bridge",
+"tests/data/acpi/q35/DSDT.cphp",
+"tests/data/acpi/q35/DSDT.cxl",
+"tests/data/acpi/q35/DSDT.dimmpxm",
+"tests/data/acpi/q35/DSDT.ipmibt",
+"tests/data/acpi/q35/DSDT.ipmismbus",
+"tests/data/acpi/q35/DSDT.ivrs",
+"tests/data/acpi/q35/DSDT.memhp",
+"tests/data/acpi/q35/DSDT.mmio64",
+"tests/data/acpi/q35/DSDT.multi-bridge",
+"tests/data/acpi/q35/DSDT.nohpet",
+"tests/data/acpi/q35/DSDT.numamem",
+"tests/data/acpi/q35/DSDT.pvpanic-isa",
+"tests/data/acpi/q35/DSDT.tis.tpm12",
+"tests/data/acpi/q35/DSDT.tis.tpm2",
+"tests/data/acpi/q35/DSDT.viot",
+"tests/data/acpi/q35/DSDT.xapic",
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 49/83] acpi: add get_dev_aml_func() helper
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (47 preceding siblings ...)
  2022-11-07 22:51 ` [PULL v4 48/83] tests: acpi: whitelist DSDT before generating ICH9_SMB AML automatically Michael S. Tsirkin
@ 2022-11-07 22:51 ` Michael S. Tsirkin
  2022-11-07 22:51 ` [PULL v4 50/83] acpi: enumerate SMB bridge automatically along with other PCI devices Michael S. Tsirkin
                   ` (35 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:51 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Igor Mammedov, Ani Sinha

From: Igor Mammedov <imammedo@redhat.com>

It will be used in followup commits to figure out if
device has it's own, device specific AML block.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20221017102146.2254096-7-imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Ani Sinha <ani@anisinha.ca>
---
 include/hw/acpi/acpi_aml_interface.h | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/include/hw/acpi/acpi_aml_interface.h b/include/hw/acpi/acpi_aml_interface.h
index ab76f0e55d..436da069d6 100644
--- a/include/hw/acpi/acpi_aml_interface.h
+++ b/include/hw/acpi/acpi_aml_interface.h
@@ -29,11 +29,20 @@ struct AcpiDevAmlIfClass {
     dev_aml_fn build_dev_aml;
 };
 
-static inline void call_dev_aml_func(DeviceState *dev, Aml *scope)
+static inline dev_aml_fn get_dev_aml_func(DeviceState *dev)
 {
     if (object_dynamic_cast(OBJECT(dev), TYPE_ACPI_DEV_AML_IF)) {
         AcpiDevAmlIfClass *klass = ACPI_DEV_AML_IF_GET_CLASS(dev);
-        klass->build_dev_aml(ACPI_DEV_AML_IF(dev), scope);
+        return klass->build_dev_aml;
+    }
+    return NULL;
+}
+
+static inline void call_dev_aml_func(DeviceState *dev, Aml *scope)
+{
+    dev_aml_fn fn = get_dev_aml_func(dev);
+    if (fn) {
+        fn(ACPI_DEV_AML_IF(dev), scope);
     }
 }
 
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 50/83] acpi: enumerate SMB bridge automatically along with other PCI devices
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (48 preceding siblings ...)
  2022-11-07 22:51 ` [PULL v4 49/83] acpi: add get_dev_aml_func() helper Michael S. Tsirkin
@ 2022-11-07 22:51 ` Michael S. Tsirkin
  2022-11-07 22:51 ` [PULL v4 51/83] tests: acpi: update expected blobs Michael S. Tsirkin
                   ` (34 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:51 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Igor Mammedov, Ani Sinha, Paolo Bonzini,
	Richard Henderson, Eduardo Habkost, Marcel Apfelbaum

From: Igor Mammedov <imammedo@redhat.com>

to make that happen (bridge sits at _ADR: 0x001F0003),
relax PCI enumeration logic to include devices with *function* > 0
if device has something to say about itself (i.e. has build_dev_aml
callback set).

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20221017102146.2254096-8-imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/i386/acpi-build.c | 27 +++------------------------
 1 file changed, 3 insertions(+), 24 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index e1483bb11a..916343d8d6 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -448,9 +448,10 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
             /*
              * allow describing coldplugged bridges in ACPI even if they are not
              * on function 0, as they are not unpluggable, for all other devices
-             * generate description only for function 0 per slot
+             * generate description only for function 0 per slot, and for other
+             * functions if device on function provides its own AML
              */
-            if (func && !bridge_in_acpi) {
+            if (func && !bridge_in_acpi && !get_dev_aml_func(DEVICE(pdev))) {
                 continue;
             }
         } else {
@@ -1319,25 +1320,6 @@ static Aml *build_q35_osc_method(bool enable_native_pcie_hotplug)
     return method;
 }
 
-static void build_smb0(Aml *table, int devnr, int func)
-{
-    Aml *scope = aml_scope("_SB.PCI0");
-    Aml *dev = aml_device("SMB0");
-    bool ambiguous;
-    Object *obj;
-    /*
-     * temporarily fish out device hosting SMBUS, build_smb0 will be gone once
-     * PCI enumeration will be switched to call_dev_aml_func()
-     */
-    obj = object_resolve_path_type("", TYPE_ICH9_SMB_DEVICE, &ambiguous);
-    assert(obj && !ambiguous);
-
-    aml_append(dev, aml_name_decl("_ADR", aml_int(devnr << 16 | func)));
-    call_dev_aml_func(DEVICE(obj), dev);
-    aml_append(scope, dev);
-    aml_append(table, scope);
-}
-
 static void build_acpi0017(Aml *table)
 {
     Aml *dev, *scope, *method;
@@ -1440,9 +1422,6 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
             build_x86_acpi_pci_hotplug(dsdt, pm->pcihp_io_base);
         }
         build_q35_pci0_int(dsdt);
-        if (pcms->smbus) {
-            build_smb0(dsdt, ICH9_SMB_DEV, ICH9_SMB_FUNC);
-        }
     }
 
     if (misc->has_hpet) {
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 51/83] tests: acpi: update expected blobs
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (49 preceding siblings ...)
  2022-11-07 22:51 ` [PULL v4 50/83] acpi: enumerate SMB bridge automatically along with other PCI devices Michael S. Tsirkin
@ 2022-11-07 22:51 ` Michael S. Tsirkin
  2022-11-07 22:51 ` [PULL v4 52/83] tests: acpi: pc/q35 whitelist DSDT before \_GPE cleanup Michael S. Tsirkin
                   ` (33 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:51 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Igor Mammedov, Ani Sinha

From: Igor Mammedov <imammedo@redhat.com>

Expected change in q35 tests:
	@@ -2797,14 +2797,6 @@ DefinitionBlock ("", "DSDT", 1, "BOCHS ", "BXPC    ", 0x00000001)
		 }
	     }

	-    Scope (_SB.PCI0)
	-    {
	-        Device (SMB0)
	-        {
	-            Name (_ADR, 0x001F0003)  // _ADR: Address
	-        }
	-    }
	-
	     Scope (_SB)
	     {
		 Device (HPET)
	@@ -3282,6 +3274,11 @@ DefinitionBlock ("", "DSDT", 1, "BOCHS ", "BXPC    ", 0x00000001)
			 }
		     }

	+            Device (SFB)
	+            {
	+                Name (_ADR, 0x001F0003)  // _ADR: Address
	+            }
	+
		     Method (PCNT, 0, NotSerialized)
		     {
		     }

Also for ipmismbus test, child 'Device (MI1)' of SMB0 will be moved along with it

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20221017102146.2254096-9-imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/qtest/bios-tables-test-allowed-diff.h |  21 --------------------
 tests/data/acpi/q35/DSDT                    | Bin 8418 -> 8407 bytes
 tests/data/acpi/q35/DSDT.acpierst           | Bin 8435 -> 8424 bytes
 tests/data/acpi/q35/DSDT.acpihmat           | Bin 9743 -> 9732 bytes
 tests/data/acpi/q35/DSDT.applesmc           | Bin 8464 -> 8453 bytes
 tests/data/acpi/q35/DSDT.bridge             | Bin 11547 -> 11536 bytes
 tests/data/acpi/q35/DSDT.cphp               | Bin 8882 -> 8871 bytes
 tests/data/acpi/q35/DSDT.cxl                | Bin 9744 -> 9733 bytes
 tests/data/acpi/q35/DSDT.dimmpxm            | Bin 10072 -> 10061 bytes
 tests/data/acpi/q35/DSDT.ipmibt             | Bin 8493 -> 8482 bytes
 tests/data/acpi/q35/DSDT.ipmismbus          | Bin 8507 -> 8495 bytes
 tests/data/acpi/q35/DSDT.ivrs               | Bin 8435 -> 8424 bytes
 tests/data/acpi/q35/DSDT.memhp              | Bin 9777 -> 9766 bytes
 tests/data/acpi/q35/DSDT.mmio64             | Bin 9548 -> 9537 bytes
 tests/data/acpi/q35/DSDT.multi-bridge       | Bin 8738 -> 8727 bytes
 tests/data/acpi/q35/DSDT.nohpet             | Bin 8276 -> 8265 bytes
 tests/data/acpi/q35/DSDT.numamem            | Bin 8424 -> 8413 bytes
 tests/data/acpi/q35/DSDT.pvpanic-isa        | Bin 8519 -> 8508 bytes
 tests/data/acpi/q35/DSDT.tis.tpm12          | Bin 9024 -> 9013 bytes
 tests/data/acpi/q35/DSDT.tis.tpm2           | Bin 9050 -> 9039 bytes
 tests/data/acpi/q35/DSDT.viot               | Bin 9527 -> 9516 bytes
 tests/data/acpi/q35/DSDT.xapic              | Bin 35781 -> 35770 bytes
 22 files changed, 21 deletions(-)

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index fd5852776c..dfb8523c8b 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1,22 +1 @@
 /* List of comma-separated changed AML files to ignore */
-"tests/data/acpi/q35/DSDT",
-"tests/data/acpi/q35/DSDT.acpierst",
-"tests/data/acpi/q35/DSDT.acpihmat",
-"tests/data/acpi/q35/DSDT.applesmc",
-"tests/data/acpi/q35/DSDT.bridge",
-"tests/data/acpi/q35/DSDT.cphp",
-"tests/data/acpi/q35/DSDT.cxl",
-"tests/data/acpi/q35/DSDT.dimmpxm",
-"tests/data/acpi/q35/DSDT.ipmibt",
-"tests/data/acpi/q35/DSDT.ipmismbus",
-"tests/data/acpi/q35/DSDT.ivrs",
-"tests/data/acpi/q35/DSDT.memhp",
-"tests/data/acpi/q35/DSDT.mmio64",
-"tests/data/acpi/q35/DSDT.multi-bridge",
-"tests/data/acpi/q35/DSDT.nohpet",
-"tests/data/acpi/q35/DSDT.numamem",
-"tests/data/acpi/q35/DSDT.pvpanic-isa",
-"tests/data/acpi/q35/DSDT.tis.tpm12",
-"tests/data/acpi/q35/DSDT.tis.tpm2",
-"tests/data/acpi/q35/DSDT.viot",
-"tests/data/acpi/q35/DSDT.xapic",
diff --git a/tests/data/acpi/q35/DSDT b/tests/data/acpi/q35/DSDT
index ea35dc5eba8433a8dcb54815f19ed6239f2534e7..c8a2b5df26608f10c75ab8f2f9e404fda987891b 100644
GIT binary patch
delta 65
zcmaFlc-@i9CD<k8x&i|Oqx(iKK8ejH66@Hw92H~YgPr09JQX&-k$cZ55#7We?B*2D
V5%1^{#KX)W&mh7U;OrN|004sf5rO~!

delta 76
zcmcca_{fpVCD<k8kpcq)<A#k~d=k>qdhx+d@d3`B2GLFY!M;ug9Py4WK|IV1@(i2B
eCDyZXIVi+{R0w#;Z@wV+o{?RIEx_3?gaH6HQWQi0

diff --git a/tests/data/acpi/q35/DSDT.acpierst b/tests/data/acpi/q35/DSDT.acpierst
index 146269c68c68238a8be3aa67e049a85c0b8edc66..eb63e439b92424e4c50b7e5f1df92da54ecfc6ea 100644
GIT binary patch
delta 64
zcmezD_`;FPCD<k8g#rTu<MWMNd=i^YB-XKUIVr`&2Rp?JcquM^BgZ5W-NYa4<`mBn
U@8}Z5!^|MhAi@^l>=(iS0Ek8rTL1t6

delta 75
zcmaFi_}P)mCD<k8vjPJHW57l(J_%`Qz4&0K_yA{5gXkvyU|%N#j(A6xARcB0d4|p6
d66@Kx92H|gDg-<g7GIEKVi#cxaP|vf000r=6CD5m

diff --git a/tests/data/acpi/q35/DSDT.acpihmat b/tests/data/acpi/q35/DSDT.acpihmat
index 4dd420b62fbcfdd21485fec2bafdea54f3fdb5a8..f9199a0dd614b30c9e73eb7de7e051ce84c1d73f 100644
GIT binary patch
delta 65
zcmeD8Y4PE533dr#QDb0WWZuZdC$ZT?;x;#zqhd^auv5H%r^04QmG_Ji(M|lpZcgzW
U@s2J*Jj@L83?gg+&VC^b02Q4NMgRZ+

delta 76
zcmZqi>G$Ds33dtLS7Tsc%-P7rCm}7Z7a!~tAK>h15Z%Nd?CWH}5%1^{#KX)W&#+ls
d;tn^LgF*~Qg@A|r=D*7C8QDeH0-XIq7y!*W69)hQ

diff --git a/tests/data/acpi/q35/DSDT.applesmc b/tests/data/acpi/q35/DSDT.applesmc
index ff25d82ba24b5e792b9d87958aa1b162bc9e0de2..286a4ecec273ca0e2fe2d65f80e8566a68a2f794 100644
GIT binary patch
delta 65
zcmbQ>)au0L66_Mfs>r~=D6x@?Phzu)#5y)Ef906?V5fKicje7*<Ybv7qMP`G-JIe%
U;vHRrc$gXF8AR9uoc%%=01xsHAOHXW

delta 76
zcmZp5n&8Cc66_Kppvb_$*tn64PeNK+FFx2QKET=2Ai9Y^*w@K`Bi_*^h=-X$o?)}N
d#CkR^KcyIu3IR8z%@^cknb<|x0-XIq7y!Nm5#j&<

diff --git a/tests/data/acpi/q35/DSDT.bridge b/tests/data/acpi/q35/DSDT.bridge
index bde84efd1c5fcafee789781f4030d573002a886a..9a733a54e428d3506aaf73f8d4c1f28324a71a42 100644
GIT binary patch
delta 75
zcmbOoH6e=2CD<iIK$n4m@!Li&K8ejH66@Hwyq?6w2Rp?JI6c|?Mox}TDY}V2*v%=P
dBi_*^h=-X$o<T$)z}YW^Ax<yY&>$Yf1OQ7T6vzMo

delta 86
zcmbObH9LySCD<iIT9<)=F@7T#pM<ouUVN}qe1Nm3L39&;u&<K=N4%p;5DznhJi}&j
niS=w;o{wWdDg+!KZ@wTW$0sNv5a8?=!VsqyY-j+~=NAG1zta|q

diff --git a/tests/data/acpi/q35/DSDT.cphp b/tests/data/acpi/q35/DSDT.cphp
index 537be7ca37c021e6a7f098e88c40ba178f3ec3e2..8579626c7f1f035c8d400544c2af6ce6372a5625 100644
GIT binary patch
delta 65
zcmdnwy4;n^CD<ioxe@~d<LZrEd=i^YB<eW092H~YgPr09JQX(YQFzZN5#7We?B*2D
V5%1^{#KX)W&mh7U;OrN|003#(5d;7L

delta 76
zcmZ4Py2+KxCD<iolM(|1qsT@sJ_%`Qz4&0K_yA{5gXkvyU|%N#j(A6xARcB0d4|p6
e67?Kh4hk_K6#^den-?g&XJi*)3vl)eVE_Q*#}dW>

diff --git a/tests/data/acpi/q35/DSDT.cxl b/tests/data/acpi/q35/DSDT.cxl
index aeea64d1cecad0ad011870ed5e04bfea1ae62510..96594c00b3b0e0a4933d6d851d927487ad4d18eb 100644
GIT binary patch
delta 65
zcmbQ>)9S<J66_Mfs>Z;;Xupw*Phzu)#4c_wXXTjqV5fKiZ>7yYlm(e2qMP`G-JIe%
U;vHRrc$gXF8AR9uoc%%=03WdrZ2$lO

delta 76
zcmZqmnc&0a66_KppvJ(!xMCw0pM<ouUVN}qe1Nm3L39&;u&<K=N4%p;5DznhJi}&j
eiQU{>PD(K#6#`z0o9`$KGO>%W1vvYKFaQA0rV=jz

diff --git a/tests/data/acpi/q35/DSDT.dimmpxm b/tests/data/acpi/q35/DSDT.dimmpxm
index ed11aefa4a56a8408bd4e0de83bf2373e6025449..e2a3ecf7d90c8c411550505c3b70bf8d19ba4bc9 100644
GIT binary patch
delta 65
zcmccNch--~CD<jzSDk@@amq$6K8ejH5-Pl0j*2ny!A|i4o(h|jRNpg7L^tsVyE(;k
V#5=kK@h~&UGl;MSIQxY#002@W5KI67

delta 76
zcmX@>cf*g%CD<h-LY;wu@$W`1J_%`Qz4&0K_yA{5gXkvyU|%N#j(A6xARcB0d4|p6
e5~{ph4hk_K6#^deo8463GqQ`Y1vvYKFaQ7)&=U>-

diff --git a/tests/data/acpi/q35/DSDT.ipmibt b/tests/data/acpi/q35/DSDT.ipmibt
index d9d1e75c987acd42be4576688621be07b21f0e7b..427272b95692099edc47f569e41fbb3ba69f6b60 100644
GIT binary patch
delta 65
zcmZ4Mw8)9eCD<iINs)npF=itdpTuSpiFIsTKB_VC!A|i4E~=Z~$XPK-L^tsVyE(;k
V#5=kK@h~&UGl;MSIQxY#001=^57qzx

delta 76
zcmZ4FwAP8sCD<iISCN5%@#sb_J_%`Qz4&0K_yA{5gXkvyU|%N#j(A6xARcB0d4|p6
e66@Kxyj5aADg>NWHeZmlVqzCz3vl)eVE_Q=2@;q9

diff --git a/tests/data/acpi/q35/DSDT.ipmismbus b/tests/data/acpi/q35/DSDT.ipmismbus
index f6e124137721312722c820b5c084a143492945c9..794779e75aaf33902de834caaa6b0763c4513615 100644
GIT binary patch
delta 61
zcmdn(wBCu!CD<iIUy*@<@%2V7K8ek}66@HwJk?_2gPr099Mv|v%D-pih;DLc3wCpg
RpR6sB$S%Sb;OrN|001n-5byv1

delta 72
zcmZ4QwA+cxCD<jzT9JW)QEwv`pM<aryIy>-Q+$B4r$Ka+J6o`?lfmTU3W=M=CDyZX
aIVi+{Bn3R=H(!u@&&V#q7U1j`!T<onCKIFp

diff --git a/tests/data/acpi/q35/DSDT.ivrs b/tests/data/acpi/q35/DSDT.ivrs
index 146269c68c68238a8be3aa67e049a85c0b8edc66..eb63e439b92424e4c50b7e5f1df92da54ecfc6ea 100644
GIT binary patch
delta 64
zcmezD_`;FPCD<k8g#rTu<MWMNd=i^YB-XKUIVr`&2Rp?JcquM^BgZ5W-NYa4<`mBn
U@8}Z5!^|MhAi@^l>=(iS0Ek8rTL1t6

delta 75
zcmaFi_}P)mCD<k8vjPJHW57l(J_%`Qz4&0K_yA{5gXkvyU|%N#j(A6xARcB0d4|p6
d66@Kx92H|gDg-<g7GIEKVi#cxaP|vf000r=6CD5m

diff --git a/tests/data/acpi/q35/DSDT.memhp b/tests/data/acpi/q35/DSDT.memhp
index 3e79ba7ac0b9a466fddd2213f32d20c522b8613b..923e213ab6a3c82faa6f659c29de9c8afb6878dd 100644
GIT binary patch
delta 65
zcmdn!v&@IfCD<iIO^tzp(PASPpTuSpiND-jj*2ny!A|i4o(h|-Ro*j7L^tsVyE(;k
V#5=kK@h~&UGl;MSIQxY#002TP5G4Qr

delta 76
zcmZ4Hv(bmkCD<jzP>q3sap6WTJ_%`Qz4&0K_yA{5gXkvyU|%N#j(A6xARcB0d4|p6
e692fl928<eDg->_H_NNMXJi*)3vl)eVE_Q{`4Z;<

diff --git a/tests/data/acpi/q35/DSDT.mmio64 b/tests/data/acpi/q35/DSDT.mmio64
index afc260ebf4d91a2a9f7be5ff21968e99b1f9c5d1..a77aa37ca0bb407abbef134e8dce4461070856a2 100644
GIT binary patch
delta 64
zcmX@(b<m5;CD<jzQI&y#@ykXoK8ejH671YuPD(NH!A|i4UW$udm6;@>oA`s>oZ>m+
T9bJNWm>J|5MA!nH{X!T3EYS}}

delta 75
zcmX@;b;gU!CD<jzN0otrF?J&tpM<ouUVN}qe1Nm3L39&;u&<K=N4%p;5DznhJi}&j
c2@Y;9N5vSB3IR`r#hS`Y>>_Ld&VC^b0K(`I2mk;8

diff --git a/tests/data/acpi/q35/DSDT.multi-bridge b/tests/data/acpi/q35/DSDT.multi-bridge
index 7c14ce3a986fa06e88f3adc088faae54bdd2d8e4..43469e6c89813025b902534ed61d39ad940ff7bb 100644
GIT binary patch
delta 85
zcmZ4FGTnvCCD<iIT#12!apOiVK8ejH66@Hw98F{5gPr09JWV#gk&|ZDiEiQ#c5{m7
hh<9`e;$dcxXAqGJaP|vfh|>!;w2TKaK`a9>3jo0872^N^

delta 72
zcmbR4vdD$YCD<iINr{1hQEDR>pM<ouUVN}qe1Nm3L39&;u&<K=N4%p;5DznhJi}&j
aiS=w;4kj@m6#^c{n=i;oGf)1bC=CF0xDr7C

diff --git a/tests/data/acpi/q35/DSDT.nohpet b/tests/data/acpi/q35/DSDT.nohpet
index 60595f55fc72397e3ed9b1999dddd39cd98f89a8..e17b252b03b290ba39601afffbee66159a57bfb1 100644
GIT binary patch
delta 65
zcmccOaMFRxCD<jzQ-OhjQD7q%pTuSp2?aJTN5z=<V5fKiPle5Ka_<=>qMP`G-JIe%
U;vHRrc$gXF8AR9uoc%%=04azMg8%>k

delta 76
zcmX@<aK(YkCD<h-M1g^Uv2r69pM<ouUVN}qe1Nm3L39&;u&<K=N4%p;5DznhJi}&j
d2}L$82Zb1r3IPxK%}#Rf8QDeH0-XIq7y#w#5=8(2

diff --git a/tests/data/acpi/q35/DSDT.numamem b/tests/data/acpi/q35/DSDT.numamem
index 68e67eb91097c58365e3734c9b35d32796639d54..ade716519de8bd626b3cddee686f55757bb4eb35 100644
GIT binary patch
delta 65
zcmaFic-N83CD<k8t^xxC<HwC$d=i^YB(|_|IV#4)2Rp?Jcq(lEB=??CBD#q`*v%=P
VBi_*^h=-X$o<W2yz}YW^0RXDm5=sC7

delta 76
zcmccX_`;FPCD<k8g#rTuW7I}2J_%`Qz4&0K_yA{5gXkvyU|%N#j(A6xARcB0d4|p6
e5?k50928<eDg->_H(!%`&&V#q7U1j`!T<m#coYl(

diff --git a/tests/data/acpi/q35/DSDT.pvpanic-isa b/tests/data/acpi/q35/DSDT.pvpanic-isa
index 811cd27c7e7366e61683c7c5ef67f0996f395c25..b6740b1ec2f4e01bdbdc34bc8bf0a2c36f134671 100644
GIT binary patch
delta 65
zcmX@^w8x3dCD<jzMv;Mm@zq8yK8ejH66@Hw+|*;@gPr09{M0tTkqcpxh;HH!c5{m7
Vh<9`e;$dcxXAofvaP|vf003u>5Yqqv

delta 76
zcmdnvbli!{CD<jzU6Fx-F?b^vpM<ouUVN}qe1Nm3L39&;u&<K=N4%p;5DznhJi}&j
eiS=w;u4*wL6#~Ahn=i<PFtLlU1vvYKFaQAK!4j1K

diff --git a/tests/data/acpi/q35/DSDT.tis.tpm12 b/tests/data/acpi/q35/DSDT.tis.tpm12
index 7a213e3db230064cedf3a30b83b11128286eaeba..5bc095351fd009fb2171daaa4f824c5cc9f36819 100644
GIT binary patch
delta 65
zcmX@$w$+WxCD<jzRGEQ+v3w&JpTuSpiCt`5&TcXB!A|i4-maT}$T_k|L^tsVyE(;k
V#5=kK@h~&UGl;MSIQxY#0037w5R?D_

delta 76
zcmdn$cEF9xCD<jzL79Pp@xeweJ_%`Qz4&0K_yA{5gXkvyU|%N#j(A6xARcB0d4|p6
e61&;BoLpl-Dg?Y-Hs6tRWMLO!3vl)eVE_OJ;S*K>

diff --git a/tests/data/acpi/q35/DSDT.tis.tpm2 b/tests/data/acpi/q35/DSDT.tis.tpm2
index b55e828c6397f80c14de82f371fa34553008a875..47417f47f7e25576f31207cb0b752b8c086a4480 100644
GIT binary patch
delta 65
zcmccRcHWK4CD<jzUzve{QFtR4pTuSpiCt`5zV0#c!A|i4uI`(E$VIbAL^tsVyE(;k
V#5=kK@h~&UGl;MSIQxY#003F<5T5`5

delta 76
zcmX@_cFT>+CD<h-N|}Lyv1TI|pM<ouUVN}qe1Nm3L39&;u&<K=N4%p;5DznhJi}&j
eiQQ~mK5j7}6#_19oA1a)v#^V>1vvYKFaQ7tbrWF#

diff --git a/tests/data/acpi/q35/DSDT.viot b/tests/data/acpi/q35/DSDT.viot
index a8a93fe70d8e98ec0e66278b45d36393b75740ec..574b8a0094c556cd8555b7a4e1b92b2d5f64750a 100644
GIT binary patch
delta 65
zcmdn)wZ@CfCD<iIN0otr@ySLmK8ejH68pKhTvcP@gPr09d{s98RWfFhh;HH!c5{m7
Vh<9`e;$dcxXAofvaP|vf003b*5VQaQ

delta 76
zcmZ4EwcU%$CD<jzT$O==(RU*kpM<ouUVN}qe1Nm3L39&;u&<K=N4%p;5DznhJi}&j
ei340*E-En~6#_oWn;$9}GqH=X1vvYKFaQAC!xDA?

diff --git a/tests/data/acpi/q35/DSDT.xapic b/tests/data/acpi/q35/DSDT.xapic
index 8211f5af8f9433b66b64768acb1de61a5716152a..74381116ad7f01a860fee9201df38d1ea24a0be6 100644
GIT binary patch
delta 67
zcmX>)ooUx}CN7s?myliE3=E8aH*)bwY&MaYSjgq57!x1t6ffYZu=#xFdq#=qCjMYI
Xr+AKdN0%TTW(Ii%5w-wlzYqoh?b#EU

delta 78
zcmdlro$2UwCN7s?myn~~3=E8E8@c!-q^0%ZgPr07oIMSqoA`r$oeVhQ9bJNWm>J|5
gHj7J4D&%rdhyke(@Q~lUx$`|Ey9ir=vtI}U0BHvmS^xk5

-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 52/83] tests: acpi: pc/q35 whitelist DSDT before \_GPE cleanup
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (50 preceding siblings ...)
  2022-11-07 22:51 ` [PULL v4 51/83] tests: acpi: update expected blobs Michael S. Tsirkin
@ 2022-11-07 22:51 ` Michael S. Tsirkin
  2022-11-07 22:51 ` [PULL v4 53/83] acpi: pc/35: sanitize _GPE declaration order Michael S. Tsirkin
                   ` (32 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:51 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Igor Mammedov, Ani Sinha

From: Igor Mammedov <imammedo@redhat.com>

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20221017102146.2254096-10-imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/qtest/bios-tables-test-allowed-diff.h | 34 +++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index dfb8523c8b..725a1dc798 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1 +1,35 @@
 /* List of comma-separated changed AML files to ignore */
+"tests/data/acpi/pc/DSDT",
+"tests/data/acpi/pc/DSDT.bridge",
+"tests/data/acpi/pc/DSDT.ipmikcs",
+"tests/data/acpi/pc/DSDT.cphp",
+"tests/data/acpi/pc/DSDT.memhp",
+"tests/data/acpi/pc/DSDT.numamem",
+"tests/data/acpi/pc/DSDT.nohpet",
+"tests/data/acpi/pc/DSDT.dimmpxm",
+"tests/data/acpi/pc/DSDT.acpihmat",
+"tests/data/acpi/pc/DSDT.acpierst",
+"tests/data/acpi/pc/DSDT.roothp",
+"tests/data/acpi/pc/DSDT.hpbridge",
+"tests/data/acpi/pc/DSDT.hpbrroot",
+"tests/data/acpi/q35/DSDT",
+"tests/data/acpi/q35/DSDT.tis.tpm2",
+"tests/data/acpi/q35/DSDT.tis.tpm12",
+"tests/data/acpi/q35/DSDT.bridge",
+"tests/data/acpi/q35/DSDT.multi-bridge",
+"tests/data/acpi/q35/DSDT.mmio64",
+"tests/data/acpi/q35/DSDT.ipmibt",
+"tests/data/acpi/q35/DSDT.cphp",
+"tests/data/acpi/q35/DSDT.memhp",
+"tests/data/acpi/q35/DSDT.numamem",
+"tests/data/acpi/q35/DSDT.nohpet",
+"tests/data/acpi/q35/DSDT.dimmpxm",
+"tests/data/acpi/q35/DSDT.acpihmat",
+"tests/data/acpi/q35/DSDT.acpierst",
+"tests/data/acpi/q35/DSDT.applesmc",
+"tests/data/acpi/q35/DSDT.pvpanic-isa",
+"tests/data/acpi/q35/DSDT.ivrs",
+"tests/data/acpi/q35/DSDT.viot",
+"tests/data/acpi/q35/DSDT.cxl",
+"tests/data/acpi/q35/DSDT.ipmismbus",
+"tests/data/acpi/q35/DSDT.xapic",
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 53/83] acpi: pc/35: sanitize _GPE declaration order
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (51 preceding siblings ...)
  2022-11-07 22:51 ` [PULL v4 52/83] tests: acpi: pc/q35 whitelist DSDT before \_GPE cleanup Michael S. Tsirkin
@ 2022-11-07 22:51 ` Michael S. Tsirkin
  2022-11-07 22:51 ` [PULL v4 54/83] tests: acpi: update expected blobs Michael S. Tsirkin
                   ` (31 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:51 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Igor Mammedov, Ani Sinha, Marcel Apfelbaum,
	Paolo Bonzini, Richard Henderson, Eduardo Habkost

From: Igor Mammedov <imammedo@redhat.com>

Move _GPE block declaration before it gets referenced by other
hotplug handlers. While at it move PCI hotplug (_E01) handler
after PCI tree description to avoid forward reference to
to not yet declared methods/devices.

PS:
Forward 'usage' usualy is fine as long as it's hidden within
method, however 'iasl' may print warnings. So be nice
to iasl/guest OS and do things in proper order.

PS2: Also follow up patches will move some of hotplug code
from PCI tree to _E01 and that also requires PCI Device
nodes build first, before Scope can reuse that from
global context.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20221017102146.2254096-11-imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/i386/acpi-build.c | 47 +++++++++++++++++++++++---------------------
 1 file changed, 25 insertions(+), 22 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 916343d8d6..960305462c 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -1434,6 +1434,18 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
         aml_append(dsdt, sb_scope);
     }
 
+    scope =  aml_scope("_GPE");
+    {
+        aml_append(scope, aml_name_decl("_HID", aml_string("ACPI0006")));
+        if (machine->nvdimms_state->is_enabled) {
+            method = aml_method("_E04", 0, AML_NOTSERIALIZED);
+            aml_append(method, aml_notify(aml_name("\\_SB.NVDR"),
+                                          aml_int(0x80)));
+            aml_append(scope, method);
+        }
+    }
+    aml_append(dsdt, scope);
+
     if (pcmc->legacy_cpu_hotplug) {
         build_legacy_cpu_hotplug_aml(dsdt, machine, pm->cpu_hp_io_base);
     } else {
@@ -1452,28 +1464,6 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
                                  pcms->memhp_io_base);
     }
 
-    scope =  aml_scope("_GPE");
-    {
-        aml_append(scope, aml_name_decl("_HID", aml_string("ACPI0006")));
-
-        if (pm->pcihp_bridge_en || pm->pcihp_root_en) {
-            method = aml_method("_E01", 0, AML_NOTSERIALIZED);
-            aml_append(method,
-                aml_acquire(aml_name("\\_SB.PCI0.BLCK"), 0xFFFF));
-            aml_append(method, aml_call0("\\_SB.PCI0.PCNT"));
-            aml_append(method, aml_release(aml_name("\\_SB.PCI0.BLCK")));
-            aml_append(scope, method);
-        }
-
-        if (machine->nvdimms_state->is_enabled) {
-            method = aml_method("_E04", 0, AML_NOTSERIALIZED);
-            aml_append(method, aml_notify(aml_name("\\_SB.NVDR"),
-                                          aml_int(0x80)));
-            aml_append(scope, method);
-        }
-    }
-    aml_append(dsdt, scope);
-
     crs_range_set_init(&crs_range_set);
     bus = PC_MACHINE(machine)->bus;
     if (bus) {
@@ -1752,6 +1742,19 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
     }
     aml_append(dsdt, sb_scope);
 
+    if (pm->pcihp_bridge_en || pm->pcihp_root_en) {
+        scope =  aml_scope("_GPE");
+        {
+            method = aml_method("_E01", 0, AML_NOTSERIALIZED);
+            aml_append(method,
+                aml_acquire(aml_name("\\_SB.PCI0.BLCK"), 0xFFFF));
+            aml_append(method, aml_call0("\\_SB.PCI0.PCNT"));
+            aml_append(method, aml_release(aml_name("\\_SB.PCI0.BLCK")));
+            aml_append(scope, method);
+        }
+        aml_append(dsdt, scope);
+    }
+
     /* copy AML table into ACPI tables blob and patch header there */
     g_array_append_vals(table_data, dsdt->buf->data, dsdt->buf->len);
     acpi_table_end(linker, &table);
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 54/83] tests: acpi: update expected blobs
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (52 preceding siblings ...)
  2022-11-07 22:51 ` [PULL v4 53/83] acpi: pc/35: sanitize _GPE declaration order Michael S. Tsirkin
@ 2022-11-07 22:51 ` Michael S. Tsirkin
  2022-11-07 22:51 ` [PULL v4 55/83] hw/acpi/erst.c: Fix memory handling issues Michael S. Tsirkin
                   ` (30 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:51 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Igor Mammedov, Ani Sinha

From: Igor Mammedov <imammedo@redhat.com>

Expected changes are:
 1) Moving _GPE scope declaration achec of all _E0x methods
   +    Scope (_GPE)
   +    {
   +        Name (_HID, "ACPI0006" /* GPE Block Device */)  // _HID: Hardware ID
   +    }
   +
        Scope (_SB)
        {
            Device (\_SB.PCI0.PRES)
    ============
            \_SB.CPUS.CSCN ()
        }

   -    Scope (_GPE)
   -    {
   -        Name (_HID, "ACPI0006" /* GPE Block Device */)  // _HID: Hardware ID
   -    }

 2) Moving _E01 handler after PCI0 scope is defined
    -    Scope (_GPE)
    -    {
    -        Name (_HID, "ACPI0006" /* GPE Block Device */)  // _HID: Hardware ID
    -        Method (_E01, 0, NotSerialized)  // _Exx: Edge-Triggered GPE
    -        {
    -            Acquire (\_SB.PCI0.BLCK, 0xFFFF)
    -            \_SB.PCI0.PCNT ()
    -            Release (\_SB.PCI0.BLCK)
    -        }
    -    }
    -
         Scope (\_SB.PCI0)
         {
             Name (_CRS, ResourceTemplate ()  // _CRS: Current Resource Settings
    =============
                 }
             }
         }
    +
    +    Scope (_GPE)
    +    {
    +        Method (_E01, 0, NotSerialized)  // _Exx: Edge-Triggered GPE
    +        {
    +            Acquire (\_SB.PCI0.BLCK, 0xFFFF)
    +            \_SB.PCI0.PCNT ()
    +            Release (\_SB.PCI0.BLCK)
    +        }
    +    }
     }

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20221017102146.2254096-12-imammedo@redhat.com>
---
 tests/qtest/bios-tables-test-allowed-diff.h |  34 --------------------
 tests/data/acpi/pc/DSDT                     | Bin 6496 -> 6501 bytes
 tests/data/acpi/pc/DSDT.acpierst            | Bin 6456 -> 6461 bytes
 tests/data/acpi/pc/DSDT.acpihmat            | Bin 7821 -> 7826 bytes
 tests/data/acpi/pc/DSDT.bridge              | Bin 9570 -> 9575 bytes
 tests/data/acpi/pc/DSDT.cphp                | Bin 6960 -> 6965 bytes
 tests/data/acpi/pc/DSDT.dimmpxm             | Bin 8150 -> 8155 bytes
 tests/data/acpi/pc/DSDT.hpbridge            | Bin 6456 -> 6461 bytes
 tests/data/acpi/pc/DSDT.hpbrroot            | Bin 3107 -> 3107 bytes
 tests/data/acpi/pc/DSDT.ipmikcs             | Bin 6568 -> 6573 bytes
 tests/data/acpi/pc/DSDT.memhp               | Bin 7855 -> 7860 bytes
 tests/data/acpi/pc/DSDT.nohpet              | Bin 6354 -> 6359 bytes
 tests/data/acpi/pc/DSDT.numamem             | Bin 6502 -> 6507 bytes
 tests/data/acpi/pc/DSDT.roothp              | Bin 6694 -> 6699 bytes
 tests/data/acpi/q35/DSDT                    | Bin 8407 -> 8412 bytes
 tests/data/acpi/q35/DSDT.acpierst           | Bin 8424 -> 8429 bytes
 tests/data/acpi/q35/DSDT.acpihmat           | Bin 9732 -> 9737 bytes
 tests/data/acpi/q35/DSDT.applesmc           | Bin 8453 -> 8458 bytes
 tests/data/acpi/q35/DSDT.bridge             | Bin 11536 -> 11541 bytes
 tests/data/acpi/q35/DSDT.cphp               | Bin 8871 -> 8876 bytes
 tests/data/acpi/q35/DSDT.cxl                | Bin 9733 -> 9738 bytes
 tests/data/acpi/q35/DSDT.dimmpxm            | Bin 10061 -> 10066 bytes
 tests/data/acpi/q35/DSDT.ipmibt             | Bin 8482 -> 8487 bytes
 tests/data/acpi/q35/DSDT.ipmismbus          | Bin 8495 -> 8500 bytes
 tests/data/acpi/q35/DSDT.ivrs               | Bin 8424 -> 8429 bytes
 tests/data/acpi/q35/DSDT.memhp              | Bin 9766 -> 9771 bytes
 tests/data/acpi/q35/DSDT.mmio64             | Bin 9537 -> 9542 bytes
 tests/data/acpi/q35/DSDT.multi-bridge       | Bin 8727 -> 8732 bytes
 tests/data/acpi/q35/DSDT.nohpet             | Bin 8265 -> 8270 bytes
 tests/data/acpi/q35/DSDT.numamem            | Bin 8413 -> 8418 bytes
 tests/data/acpi/q35/DSDT.pvpanic-isa        | Bin 8508 -> 8513 bytes
 tests/data/acpi/q35/DSDT.tis.tpm12          | Bin 9013 -> 9018 bytes
 tests/data/acpi/q35/DSDT.tis.tpm2           | Bin 9039 -> 9044 bytes
 tests/data/acpi/q35/DSDT.viot               | Bin 9516 -> 9521 bytes
 tests/data/acpi/q35/DSDT.xapic              | Bin 35770 -> 35775 bytes
 35 files changed, 34 deletions(-)

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index 725a1dc798..dfb8523c8b 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1,35 +1 @@
 /* List of comma-separated changed AML files to ignore */
-"tests/data/acpi/pc/DSDT",
-"tests/data/acpi/pc/DSDT.bridge",
-"tests/data/acpi/pc/DSDT.ipmikcs",
-"tests/data/acpi/pc/DSDT.cphp",
-"tests/data/acpi/pc/DSDT.memhp",
-"tests/data/acpi/pc/DSDT.numamem",
-"tests/data/acpi/pc/DSDT.nohpet",
-"tests/data/acpi/pc/DSDT.dimmpxm",
-"tests/data/acpi/pc/DSDT.acpihmat",
-"tests/data/acpi/pc/DSDT.acpierst",
-"tests/data/acpi/pc/DSDT.roothp",
-"tests/data/acpi/pc/DSDT.hpbridge",
-"tests/data/acpi/pc/DSDT.hpbrroot",
-"tests/data/acpi/q35/DSDT",
-"tests/data/acpi/q35/DSDT.tis.tpm2",
-"tests/data/acpi/q35/DSDT.tis.tpm12",
-"tests/data/acpi/q35/DSDT.bridge",
-"tests/data/acpi/q35/DSDT.multi-bridge",
-"tests/data/acpi/q35/DSDT.mmio64",
-"tests/data/acpi/q35/DSDT.ipmibt",
-"tests/data/acpi/q35/DSDT.cphp",
-"tests/data/acpi/q35/DSDT.memhp",
-"tests/data/acpi/q35/DSDT.numamem",
-"tests/data/acpi/q35/DSDT.nohpet",
-"tests/data/acpi/q35/DSDT.dimmpxm",
-"tests/data/acpi/q35/DSDT.acpihmat",
-"tests/data/acpi/q35/DSDT.acpierst",
-"tests/data/acpi/q35/DSDT.applesmc",
-"tests/data/acpi/q35/DSDT.pvpanic-isa",
-"tests/data/acpi/q35/DSDT.ivrs",
-"tests/data/acpi/q35/DSDT.viot",
-"tests/data/acpi/q35/DSDT.cxl",
-"tests/data/acpi/q35/DSDT.ipmismbus",
-"tests/data/acpi/q35/DSDT.xapic",
diff --git a/tests/data/acpi/pc/DSDT b/tests/data/acpi/pc/DSDT
index 5b4624cb68c78eab6e356987974b9bee6c9a1785..f1cf7fab349bd26e0f0fa461c715534c1426fbf5 100644
GIT binary patch
delta 60
zcmaE0^wfyUCD<h-Rg!^$@xex}HLSuS@$Lbx9Pu8WF1(J;0iFg124<VLu=25N7UJ^b
OXA`goNl(7cX$1fe{1Br6

delta 55
zcmaEA^uUPACD<h-L6U)i(O@Ij8rIFHS$SAQ{aNDO16(=cJv?1_9i0O_4Gav-7$$2=
LT5Z0@>CX=Uu}Tiy

diff --git a/tests/data/acpi/pc/DSDT.acpierst b/tests/data/acpi/pc/DSDT.acpierst
index 99461b771eec2043263b5bc3c109a08969a99af9..5cb477625e96f5526f0c7703ba3b443a0b35eefd 100644
GIT binary patch
delta 60
zcmdmCwAYBsCD<jzR+52%QE?;J8dhPEc=rHTj(87G7hXr_08ax01GCLrSov5s3vsFO
OvkBOPq$gkJv;qLpD-T2f

delta 55
zcmdmMw8MzYCD<jzLXv@jv2Y{T8rIFHS$SAQ{aNDO16(=cJv?1_9i0O_4Gav-7$!4I
LT5Z0@sm>1oq7x2P

diff --git a/tests/data/acpi/pc/DSDT.acpihmat b/tests/data/acpi/pc/DSDT.acpihmat
index b84f3b47c37e427b927c36036307db6e83a843e5..76e8bef36fdb667447ad0320d35604031aae2c93 100644
GIT binary patch
delta 46
zcmeCRon*`966_K(NsfVmaqdR0HLQ#xlRxqaZ{Esk#ktv@-;bY7z&_qRz;!YUzZC#I
C&J4u>

delta 55
zcmbPa+iT0^66_MvE62dV_;w@L8rIFHSuHt5{aNDO16(=cJv?1_9i0O_4Gav-7$!%^
LS#4(K_vZ%yzRnKP

diff --git a/tests/data/acpi/pc/DSDT.bridge b/tests/data/acpi/pc/DSDT.bridge
index 6771620078086e42b445474b16797094e0d0a801..c94c1b54b3ac3085c02307d9564b258e791fcf1e 100644
GIT binary patch
delta 60
zcmaFl_1ufgCD<h-U6p}>@$5#fHLSuS@$Lbx9Pu8WF1(J;0iFg124<VLu=25N7UBw!
OW)rXnNl(7cX$1fq#t`5D

delta 55
zcmaFv^~j6MCD<h-NtJ<tQFbHO8rIFHS$SAQ{aNDO16(=cJv?1_9i0O_4Gav-7$)ng
LT5Z0@87K_^yYmkx

diff --git a/tests/data/acpi/pc/DSDT.cphp b/tests/data/acpi/pc/DSDT.cphp
index 298fa1592676b3a2089b08eff5f764e04391b961..eb3da0e232e4362bd5850e4eebf5916cbc1d565a 100644
GIT binary patch
delta 60
zcmdmBw$+TwCD<jzRGNW-(Pbmo8dhPEc=rHTj(87G7hXr_08ax01GCLrSnsiJe#Gs^
O&n93GlAgSr+X?{i%n<be

delta 55
zcmdmLw!w_cCD<jzK$?Mpamq%nHLROYv)*A9^=FB94{+s(_waP#b#xB!G%zqQW0?F~
L%4+j6Zhw9N$1M;p

diff --git a/tests/data/acpi/pc/DSDT.dimmpxm b/tests/data/acpi/pc/DSDT.dimmpxm
index 9fcadb266b92fc4942621f0fd919703723de2fd9..6553e4c6053aff76ac42dad87d714e4e9ab28f72 100644
GIT binary patch
delta 81
zcmca+f7_nRCD<k8wmbs^qrgV4HLMyM@$Lbx9Pu8WF1(J;0iFg124)N*!tt&KCJb#c
jdhx+d@qS@0L0k=+x3U&+ZSEBC<!2MH2kDt?DPRQvnI{$^

delta 75
zcmca@f6boDCD<k8nmhvoW8FrsHLRQOv*vS&u*JIvxN^jMc)IX9ItO?f7#Nr_OkOB&
fB_bjm?`mMe&=#W?AM6zG7v>Vg)v(z|z@HxgQ?nI5

diff --git a/tests/data/acpi/pc/DSDT.hpbridge b/tests/data/acpi/pc/DSDT.hpbridge
index 99461b771eec2043263b5bc3c109a08969a99af9..5cb477625e96f5526f0c7703ba3b443a0b35eefd 100644
GIT binary patch
delta 60
zcmdmCwAYBsCD<jzR+52%QE?;J8dhPEc=rHTj(87G7hXr_08ax01GCLrSov5s3vsFO
OvkBOPq$gkJv;qLpD-T2f

delta 55
zcmdmMw8MzYCD<jzLXv@jv2Y{T8rIFHS$SAQ{aNDO16(=cJv?1_9i0O_4Gav-7$!4I
LT5Z0@sm>1oq7x2P

diff --git a/tests/data/acpi/pc/DSDT.hpbrroot b/tests/data/acpi/pc/DSDT.hpbrroot
index b10b17cb1111f6b5d61da4aa1c754f2921a4ac1c..ff04ad360beb60571d48bd1e477a4e58e5ee9337 100644
GIT binary patch
delta 37
scmZ21u~=e5CX290ynBEvN4$rp3$LSdfTw|hf!XGC7Cx5ET^uu+0mthKGXMYp

delta 38
tcmZ21u~=e5Cd=kJ79JK6k$Cq2SB`iOPZwTC=KxOw0|PUL&7B;xm;uwt3P}I}

diff --git a/tests/data/acpi/pc/DSDT.ipmikcs b/tests/data/acpi/pc/DSDT.ipmikcs
index aff3e9bbe1fdbf783c49d07a49c2e16317a2871e..83eec58a52b5844a02003665494f63a4ea0b26a7 100644
GIT binary patch
delta 60
zcmZ2syw;e@CD<iott0~jW6VabHLSuS@$Lbx9Pu8WF1(J;0iFg124<VLu=25N7UG)5
O&n93GlAe5>(+U9a*$_bh

delta 55
zcmZ2$yuz5vCD<iog(L$5<C=|JYgjj*X60cK^=FB94{+s(_waP#b#xB!G%zqQW0+hl
LX|?$p=X8Dmy|53$

diff --git a/tests/data/acpi/pc/DSDT.memhp b/tests/data/acpi/pc/DSDT.memhp
index 07b193b9939e2199bce5edc743de5494133c1e8c..9e2201d170a86652951a2a4b234bb58204010642 100644
GIT binary patch
delta 60
zcmZ2)yTz8vCD<ioiyQ+3<E4#UYgmOv;@tyWIpRG$U3eXx13V2349qrfVU6Y7oXGFT
O&n93GlAf%_Zv_AqSP+^3

delta 55
zcmdmDyWW<|CD<ioy&MArqtZsMHLROYvqp1@`m@Bl2e@*?dw9C=Iywh<8W<RuF-)$K
Lv)Zi6@6QhaxVa9~

diff --git a/tests/data/acpi/pc/DSDT.nohpet b/tests/data/acpi/pc/DSDT.nohpet
index ceee7722ceed8a7e8f61a37805a4c855f47244a8..c969e0dae410763a5f749c4c4fc842ccbf901fc5 100644
GIT binary patch
delta 61
zcmca)c-@f8CD<k8x&#9Q<Eo8Za;zdE@$Lbx9Pu8WF1(J;0iFg124)PK)meF2Hn(v4
P@v{logXAZBa#{fZ0$&e^

delta 55
zcmca^c*&5<CD<k8k^}<-<FAcea;%$eS$SAQ{aNDO16(=cJv?1_9i0O_4Gav-7$(n^
Lu-fdw>CX=UxO5J{

diff --git a/tests/data/acpi/pc/DSDT.numamem b/tests/data/acpi/pc/DSDT.numamem
index e03f4d07b86018d12def37b5397ba9673548e2b7..1cecaa64e9ef29b5096ae1ba4882e2c8e080f0ea 100644
GIT binary patch
delta 60
zcmaE6^xBBaCD<h-Tatl+v1=pO8dhPEc=rHTj(87G7hXr_08ax01GCLrSVdSiOK|z|
OvkBOPq$l6yv;qJHRuFLj

delta 55
zcmaED^vsCMCD<h-O_G6u@zzGJHLROYvkI|@`m@Bl2e@*?dw9C=Iywh<8W<RuF-$g;
LwAy@!)1Myz(Jl|F

diff --git a/tests/data/acpi/pc/DSDT.roothp b/tests/data/acpi/pc/DSDT.roothp
index 418cc92e4c061ad21e1d281e0b5e6114e283b7d3..f57a14cd5c48ddded4c5d0b7da037b9d021a9ecb 100644
GIT binary patch
delta 60
zcmZ2xvf6~pCD<iITZ(~!v3?`h8dhPEc=rHTj(87G7hXr_08ax01GCLrSov5s3vr1E
OunE|Mq$gkJv;qLuX%Aok

delta 55
zcmZ2&vdo0bCD<iIO^Shm@%%=vHLROYv+}Tr`m@Bl2e@*?dw9C=Iywh<8W<RuF--m_
LX|?$pr>Fn`uA2{E

diff --git a/tests/data/acpi/q35/DSDT b/tests/data/acpi/q35/DSDT
index c8a2b5df26608f10c75ab8f2f9e404fda987891b..8e989819a5f8c470a8933bf9b7af7b988048cce6 100644
GIT binary patch
delta 60
zcmccac*l{;CD<k8jsgP%<M)kRb0maC;@tyWIpRG$U3eXx13V2349qq!k>F$5%p`l2
OnN7eRBt7}Kj1>Sj$q~x{

delta 55
zcmccPc-@i9CD<k8x&i|Oqx(j#ITD+9OYpFW`m@Bl2e@*?dw9C=Iywh<8W<RuF-%^h
LV72*}%r#~J)0q%|

diff --git a/tests/data/acpi/q35/DSDT.acpierst b/tests/data/acpi/q35/DSDT.acpierst
index eb63e439b92424e4c50b7e5f1df92da54ecfc6ea..03745d78de4c7e9639a4a37553361e0e18258633 100644
GIT binary patch
delta 60
zcmaFi_|}ojCD<k8tpWoB<MfSOb0maC;@tyWIpRG$U3eXx13V2349qq!k>F$5%q07a
OnN7eRBt7}Kj1>SepAoMB

delta 55
zcmaFs_`;FPCD<k8g#rTu<MWMNb0jwJmf&F#^=FB94{+s(_waP#b#xB!G%zqQW0<^I
L!D{m{ndi&^_#F{+

diff --git a/tests/data/acpi/q35/DSDT.acpihmat b/tests/data/acpi/q35/DSDT.acpihmat
index f9199a0dd614b30c9e73eb7de7e051ce84c1d73f..3ad9ba3c987ca1cfbafc3778db889602ab53530e 100644
GIT binary patch
delta 46
zcmZqi>Ga`p33dtLRAXRZ+`N%%js&B~<cEsFo0m#hac<UEy2{KZU?1-u;5zw>q7?u+
C+70gj

delta 55
zcmeD5Y4PE533dr#QDb0WWZuX%M`H7C2}@2<f0lUn09TH94^J0fN9O=f0|NsyhRNqu
Ltu}vFyv7Uwjvfzm

diff --git a/tests/data/acpi/q35/DSDT.applesmc b/tests/data/acpi/q35/DSDT.applesmc
index 286a4ecec273ca0e2fe2d65f80e8566a68a2f794..5f01572dc2cf44aa0e730401e0709b95cd8db604 100644
GIT binary patch
delta 60
zcmZp5>T=?833dtLQe<FYJh72$j)bsCynBEvN4$rp3$LSdfTw|hf!XFI5_~M1nPiz+
O*aYlB(vy$NSOEakt`ArM

delta 55
zcmeBjYIWjr33dr#Rb*gbl-S5MM`H7C2_6<vf0lUn09TH94^J0fN9O=f0|NsyhRGKc
LtTrE$VPOFPgdz?)

diff --git a/tests/data/acpi/q35/DSDT.bridge b/tests/data/acpi/q35/DSDT.bridge
index 9a733a54e428d3506aaf73f8d4c1f28324a71a42..97141f9db208e1948bc2bdb3f3452ac8f78f98b0 100644
GIT binary patch
delta 60
zcmbObH8qOMCD<iIRF{E)aoI+$ITFGm@$Lbx9Pu8WF1(J;0iFg124<U=Nbs?2W|HL-
OU=y$hNl!j5V+8=@EDx#x

delta 55
zcmbOlH6e=2CD<iIK$n4m@!LkOITD+9OYpFW`m@Bl2e@*?dw9C=Iywh<8W<RuF-*R#
LW3~C148H&XyC@H%

diff --git a/tests/data/acpi/q35/DSDT.cphp b/tests/data/acpi/q35/DSDT.cphp
index 8579626c7f1f035c8d400544c2af6ce6372a5625..622e8e5f3700abffc58683689b9470573c117cd0 100644
GIT binary patch
delta 60
zcmZ4Py2h2uCD<iojS>R`WAsL@ITFGm@$Lbx9Pu8WF1(J;0iFg124<U=NZe!Hd`a#q
OGn;@tNP6;gIV%7cs1h9j

delta 55
zcmZ4Ey4;n^CD<ioxe@~d<LZrEb0jwJmbk+z>dzAI9^lFm@8Rjf>*yTdX<%Sr#xS`^
L$!ha7xogY-;e!y;

diff --git a/tests/data/acpi/q35/DSDT.cxl b/tests/data/acpi/q35/DSDT.cxl
index 96594c00b3b0e0a4933d6d851d927487ad4d18eb..cecc1caaab81db8559781d23e45d8c615dc73740 100644
GIT binary patch
delta 60
zcmZqm>GI)n33dtLQe$9Xe7})vj)bsCynBEvN4$rp3$LSdfTw|hf!XFI5_~M1nPizc
O*#zuC(vy$NSOEaxZV#OR

delta 55
zcmeD3Y4zc933dr#RbyaawBN`zM`H7C2_6<vf0lUn09TH94^J0fN9O=f0|NsyhRGLH
Ltu`N%Vc`S-kB$z6

diff --git a/tests/data/acpi/q35/DSDT.dimmpxm b/tests/data/acpi/q35/DSDT.dimmpxm
index e2a3ecf7d90c8c411550505c3b70bf8d19ba4bc9..e5be00b4fa658477a9ae7016ed72ae04430a3b2f 100644
GIT binary patch
delta 81
zcmX@>cgc^-CD<h-NS%R!QEemF90?7Lc=rHTj(87G7hXr_08ax012YB@;doaA6Na`J
jz4&0Kc)u{0Ag+eZOC|ETHdiQLWo8qw2kDutscZ!Rg;*9z

delta 76
zcmccQch--~CD<jzSDk@@amq%nITD-COXP8hI<v*Q2e@*?dw9C=Iywh<8W<RuF-(?F
gw-ON%j(0UMVQ7odiw|~+_X~3g;%eAzqkN4S07LH;B>(^b

diff --git a/tests/data/acpi/q35/DSDT.ipmibt b/tests/data/acpi/q35/DSDT.ipmibt
index 427272b95692099edc47f569e41fbb3ba69f6b60..c4f8212c63be2a1d579d6ebc9ac41d4bd5be414b 100644
GIT binary patch
delta 60
zcmZ4FwA_iyCD<iIU6Fx-k!K^<90_5Oc=rHTj(87G7hXr_08ax01GCLbB=}f1Gs!Bj
OunE|Mq$eMju>t_cgbxq^

delta 55
zcmZ4Pw8)9eCD<iINs)npF=ivz9Er`lC3sjw{aNDO16(=cJv?1_9i0O_4Gav-7$(0}
Lu-be~Mv(;op_&g5

diff --git a/tests/data/acpi/q35/DSDT.ipmismbus b/tests/data/acpi/q35/DSDT.ipmismbus
index 794779e75aaf33902de834caaa6b0763c4513615..05fb38820fa9213a20ace5943486ed18ad6a765c 100644
GIT binary patch
delta 60
zcmZ4Qw8e?bCD<jzM3I4karQ>8ITFGm@$Lbx9Pu8WF1(J;0iFg124<U=Nbs?2W|Gxq
OVH2<iNl!j5V+8=}bPv-2

delta 55
zcmdnuwBCu!CD<iIUy*@<@%2WoITD+9OYpFW`m@Bl2e@*?dw9C=Iywh<8W<RuF--oY
LV72*}j2;UBz|{}m

diff --git a/tests/data/acpi/q35/DSDT.ivrs b/tests/data/acpi/q35/DSDT.ivrs
index eb63e439b92424e4c50b7e5f1df92da54ecfc6ea..03745d78de4c7e9639a4a37553361e0e18258633 100644
GIT binary patch
delta 60
zcmaFi_|}ojCD<k8tpWoB<MfSOb0maC;@tyWIpRG$U3eXx13V2349qq!k>F$5%q07a
OnN7eRBt7}Kj1>SepAoMB

delta 55
zcmaFs_`;FPCD<k8g#rTu<MWMNb0jwJmf&F#^=FB94{+s(_waP#b#xB!G%zqQW0<^I
L!D{m{ndi&^_#F{+

diff --git a/tests/data/acpi/q35/DSDT.memhp b/tests/data/acpi/q35/DSDT.memhp
index 923e213ab6a3c82faa6f659c29de9c8afb6878dd..2a4635d48c017970ee38d14148b20d38b699b030 100644
GIT binary patch
delta 60
zcmZ4Hv)YHtCD<iITaAH%@x?~2ITFGm@$Lbx9Pu8WF1(J;0iFg124<U=NW^k(4ph3z
O%qCzDlAbK4WCZ{TDG<T{

delta 55
zcmZ4Ov&@IfCD<iIO^tzp(PAUl9Er`lC89Y+{aNDO16(=cJv?1_9i0O_4Gav-7$$#I
Lwc0GIbd4DRtqc!#

diff --git a/tests/data/acpi/q35/DSDT.mmio64 b/tests/data/acpi/q35/DSDT.mmio64
index a77aa37ca0bb407abbef134e8dce4461070856a2..0491761dc7132460b08b443579f0d4cad3fa3163 100644
GIT binary patch
delta 60
zcmX@;b<B&)CD<jzO_hOxamhxmITFGm@$Lbx9Pu8WF1(J;0iFg124<U=NThRYj#hle
O%qCzDlAf%fXaxWd&Jgwh

delta 55
zcmX@+b<m5;CD<jzQI&y#@ykZ8ITD+9OQdp$`m@Bl2e@*?dw9C=Iywh<8W<RuF-+!B
Lwc0GN_?#I4)UFS_

diff --git a/tests/data/acpi/q35/DSDT.multi-bridge b/tests/data/acpi/q35/DSDT.multi-bridge
index 43469e6c89813025b902534ed61d39ad940ff7bb..485f571afd35cbd2044baeb7027ef077ba9a133d 100644
GIT binary patch
delta 60
zcmbR4GRK9>CD<iIMu~xeF>xc;90_5Oc=rHTj(87G7hXr_08ax01GCLbB=}f1Gs%jw
OvI*FOq$eMju>t_l9uGeN

delta 55
zcmbQ^GTnvCCD<iIT#12!apOj=ITD+9OYpFW`m@Bl2e@*?dw9C=Iywh<8W<RuF-(4>
LXtnv6j2J5bsWcBX

diff --git a/tests/data/acpi/q35/DSDT.nohpet b/tests/data/acpi/q35/DSDT.nohpet
index e17b252b03b290ba39601afffbee66159a57bfb1..9c2ec9f2c96f6bdf536c28559fd804523134cf2c 100644
GIT binary patch
delta 61
zcmX@<aL$3tCD<jzPl17f@xVqdJ_!+#c=rHTj(87G7hXr_08ax012cxr;u5?pn@eP_
PGP4QTgXAY$%2)va^~Ddc

delta 55
zcmX@-aMFRxCD<jzQ-OhjQD7q%pTuTe2_6<vf0lUn09TH94^J0fN9O=f0|NsyhRGrd
LR+}wkt}z1ud(I7H

diff --git a/tests/data/acpi/q35/DSDT.numamem b/tests/data/acpi/q35/DSDT.numamem
index ade716519de8bd626b3cddee686f55757bb4eb35..2302de88e93a22118889cd33a8c63811925cff02 100644
GIT binary patch
delta 60
zcmccX_{fpVCD<k8kpcq)<HC(xb0maC;@tyWIpRG$U3eXx13V2349qq!kq}|o%prS~
OnN7eRBt7}8j1>SZBN3SZ

delta 55
zcmaFlc-N83CD<k8t^xxC<HwC$b0jwJmJnhQ^=FB94{+s(_waP#b#xB!G%zqQW0<@`
L!D{mvnQP1d^L-Ib

diff --git a/tests/data/acpi/q35/DSDT.pvpanic-isa b/tests/data/acpi/q35/DSDT.pvpanic-isa
index b6740b1ec2f4e01bdbdc34bc8bf0a2c36f134671..5e4b51d33b8bb88ca2610254e6306c16e5b5b0db 100644
GIT binary patch
delta 60
zcmdnvbkK>*CD<jzQIUaxan?qzITFGm@$Lbx9Pu8WF1(J;0iFg124<U=Nbs?2W|FmH
OVH2<iNl!j5V+8>7gb)S*

delta 55
zcmX@;w8x3dCD<jzMv;Mm@zqAIITD+9OYpFW`m@Bl2e@*?dw9C=Iywh<8W<RuF-&Gx
LwAy@3#+n5H!3Pf@

diff --git a/tests/data/acpi/q35/DSDT.tis.tpm12 b/tests/data/acpi/q35/DSDT.tis.tpm12
index 5bc095351fd009fb2171daaa4f824c5cc9f36819..1723fca4464383694f806f93331d008e84b93fef 100644
GIT binary patch
delta 60
zcmdn$w#$vnCD<jzN|}LyQGFxV90_5Oc=rHTj(87G7hXr_08ax01GCLbB=}f1GszmW
Ou?g6Nq$eMju>t_xEDwVK

delta 54
zcmdnxw$+WxCD<jzRGEQ+v3w)f9Er`lC3sjw{aNDO16(=cJv?1_9i0O_4Gav-7$zxO
KZ9XPr!Uh1Jz7CfF

diff --git a/tests/data/acpi/q35/DSDT.tis.tpm2 b/tests/data/acpi/q35/DSDT.tis.tpm2
index 47417f47f7e25576f31207cb0b752b8c086a4480..1a0d6284da01addd0393b8aad96873398411895e 100644
GIT binary patch
delta 60
zcmX@_cEyd$CD<h-M45qs@$g2jITFGm@$Lbx9Pu8WF1(J;0iFg124<U=Nbs?2W|H+~
OV-v6kNl!j5V+8;TzYvE2

delta 55
zcmccOcHWK4CD<jzUzve{QFtTQ9Er`lC3sjw{aNDO16(=cJv?1_9i0O_4Gav-7$!?8
LTWvlj<HrU7tkw>;

diff --git a/tests/data/acpi/q35/DSDT.viot b/tests/data/acpi/q35/DSDT.viot
index 574b8a0094c556cd8555b7a4e1b92b2d5f64750a..6927d1cc96565f0e1e4c7f19fd709635873db912 100644
GIT binary patch
delta 60
zcmZ4Ewb6^qCD<jzP?dp!amq%nITFGm@$Lbx9Pu8WF1(J;0iFg124<U=Nbs?2W|Gz7
OU=y$hNl!j5V+8=~s1Mix

delta 55
zcmdn!wZ@CfCD<iIN0otr@ySN6ITD+9OYpFW`m@Bl2e@*?dw9C=Iywh<8W<RuF--oh
LVzv30j5Y@V!R`;`

diff --git a/tests/data/acpi/q35/DSDT.xapic b/tests/data/acpi/q35/DSDT.xapic
index 74381116ad7f01a860fee9201df38d1ea24a0be6..4a8a4af625edb1fd01a1404e33f34e85c1a252f3 100644
GIT binary patch
delta 62
zcmdlrooWAcCN7s?myrG43=E8GH*(F95EhAd4{+s(_waP#b#xB!G%zqQ+q^{LW8UVc
R?N^!E1nfc5lUKD{0RVxr6)gY&

delta 57
zcmV-90LK5nmjb$%0t!S^L{z$q0005~u?n0Rv%47YZWT`iUq?_y2wzA^L=8bhP)RT_
PFg5^_g^N0~sfX4BNAVNl

-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 55/83] hw/acpi/erst.c: Fix memory handling issues
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (53 preceding siblings ...)
  2022-11-07 22:51 ` [PULL v4 54/83] tests: acpi: update expected blobs Michael S. Tsirkin
@ 2022-11-07 22:51 ` Michael S. Tsirkin
  2022-11-07 22:52 ` [PULL v4 56/83] MAINTAINERS: Add qapi/virtio.json to section "virtio" Michael S. Tsirkin
                   ` (29 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:51 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Christian A. Ehrhardt, Alexander Bulekov,
	qemu-stable, Eric DeVolder, Igor Mammedov, Ani Sinha

From: "Christian A. Ehrhardt" <lk@c--e.de>

- Fix memset argument order: The second argument is
  the value, the length goes last.
- Fix an integer overflow reported by Alexander Bulekov.

Both issues allow the guest to overrun the host buffer
allocated for the ERST memory device.

Cc: Eric DeVolder <eric.devolder@oracle.com
Cc: Alexander Bulekov <alxndr@bu.edu>
Cc: qemu-stable@nongnu.org
Fixes: f7e26ffa590 ("ACPI ERST: support for ACPI ERST feature")
Tested-by: Alexander Bulekov <alxndr@bu.edu>
Signed-off-by: Christian A. Ehrhardt <lk@c--e.de>
Message-Id: <20221024154233.1043347-1-lk@c--e.de>
Fixes: https://gitlab.com/qemu-project/qemu/-/issues/1268
Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Eric DeVolder <eric.devolder@oracle.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/acpi/erst.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/hw/acpi/erst.c b/hw/acpi/erst.c
index df856b2669..aefcc03ad6 100644
--- a/hw/acpi/erst.c
+++ b/hw/acpi/erst.c
@@ -635,7 +635,7 @@ static unsigned read_erst_record(ERSTDeviceState *s)
         if (record_length < UEFI_CPER_RECORD_MIN_SIZE) {
             rc = STATUS_FAILED;
         }
-        if ((s->record_offset + record_length) > exchange_length) {
+        if (record_length > exchange_length - s->record_offset) {
             rc = STATUS_FAILED;
         }
         /* If all is ok, copy the record to the exchange buffer */
@@ -684,7 +684,7 @@ static unsigned write_erst_record(ERSTDeviceState *s)
     if (record_length < UEFI_CPER_RECORD_MIN_SIZE) {
         return STATUS_FAILED;
     }
-    if ((s->record_offset + record_length) > exchange_length) {
+    if (record_length > exchange_length - s->record_offset) {
         return STATUS_FAILED;
     }
 
@@ -716,7 +716,7 @@ static unsigned write_erst_record(ERSTDeviceState *s)
     if (nvram) {
         /* Write the record into the slot */
         memcpy(nvram, exchange, record_length);
-        memset(nvram + record_length, exchange_length - record_length, 0xFF);
+        memset(nvram + record_length, 0xFF, exchange_length - record_length);
         /* If a new record, increment the record_count */
         if (!record_found) {
             uint32_t record_count;
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 56/83] MAINTAINERS: Add qapi/virtio.json to section "virtio"
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (54 preceding siblings ...)
  2022-11-07 22:51 ` [PULL v4 55/83] hw/acpi/erst.c: Fix memory handling issues Michael S. Tsirkin
@ 2022-11-07 22:52 ` Michael S. Tsirkin
  2022-11-07 22:52 ` [PULL v4 57/83] msix: Assert that specified vector is in range Michael S. Tsirkin
                   ` (28 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:52 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Markus Armbruster, Philippe Mathieu-Daudé,
	Richard Henderson, Thomas Huth, Alex Bennée

From: Markus Armbruster <armbru@redhat.com>

Cc: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20221020120458.80709-1-armbru@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 8b7d49b089..28cc70c25f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2017,6 +2017,7 @@ S: Supported
 F: hw/*/virtio*
 F: hw/virtio/Makefile.objs
 F: hw/virtio/trace-events
+F: qapi/virtio.json
 F: net/vhost-user.c
 F: include/hw/virtio/
 
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 57/83] msix: Assert that specified vector is in range
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (55 preceding siblings ...)
  2022-11-07 22:52 ` [PULL v4 56/83] MAINTAINERS: Add qapi/virtio.json to section "virtio" Michael S. Tsirkin
@ 2022-11-07 22:52 ` Michael S. Tsirkin
  2022-11-07 22:52 ` [PULL v4 58/83] hw/i386/pc.c: CXL Fixed Memory Window should not reserve e820 in bios Michael S. Tsirkin
                   ` (27 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:52 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Akihiko Odaki, Akihiko Odaki, Yuval Shaia,
	Akihiko, Odaki, &lt, Dmitry Fleytman, Jason Wang, Jiri Pirko,
	Keith Busch, Klaus Jensen, Marcel Apfelbaum, Elena Ufimtseva,
	Jagannathan Raman, John G Johnson, qemu-block

From: Akihiko Odaki <akihiko.odaki@gmail.com>

There were several different ways to deal with the situation where the
vector specified for a msix function is out of bound:
- early return a function and keep progresssing
- propagate the error to the caller
- mark msix unusable
- assert it is in bound
- just ignore

An out-of-bound vector should not be specified if the device
implementation is correct so let msix functions always assert that the
specified vector is in range.

An exceptional case is virtio-pci, which allows the guest to configure
vectors. For virtio-pci, it is more appropriate to introduce its own
checks because it is sometimes too late to check the vector range in
msix functions.

Signed-off-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Message-Id: <20220829083524.143640-1-akihiko.odaki@daynix.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Yuval Shaia <yuval.shaia.ml@gmail.com>
Signed-off-by: Akihiko Odaki &lt;<a href="mailto:akihiko.odaki@daynix.com" target="_blank">akihiko.odaki@daynix.com</a>&gt;<br>
---
 include/hw/pci/msix.h     |  4 +--
 hw/net/e1000e.c           | 15 ++-------
 hw/net/rocker/rocker.c    | 23 ++------------
 hw/net/vmxnet3.c          | 27 +++-------------
 hw/nvme/ctrl.c            |  5 +--
 hw/pci/msix.c             | 24 ++++++--------
 hw/rdma/vmw/pvrdma_main.c |  7 +---
 hw/remote/vfio-user-obj.c |  9 +-----
 hw/virtio/virtio-pci.c    | 67 ++++++++++++++++++++++++++++-----------
 9 files changed, 74 insertions(+), 107 deletions(-)

diff --git a/include/hw/pci/msix.h b/include/hw/pci/msix.h
index 4f1cda0ebe..0e6f257e45 100644
--- a/include/hw/pci/msix.h
+++ b/include/hw/pci/msix.h
@@ -33,10 +33,10 @@ bool msix_is_masked(PCIDevice *dev, unsigned vector);
 void msix_set_pending(PCIDevice *dev, unsigned vector);
 void msix_clr_pending(PCIDevice *dev, int vector);
 
-int msix_vector_use(PCIDevice *dev, unsigned vector);
+void msix_vector_use(PCIDevice *dev, unsigned vector);
 void msix_vector_unuse(PCIDevice *dev, unsigned vector);
 void msix_unuse_all_vectors(PCIDevice *dev);
-void msix_set_mask(PCIDevice *dev, int vector, bool mask, Error **errp);
+void msix_set_mask(PCIDevice *dev, int vector, bool mask);
 
 void msix_notify(PCIDevice *dev, unsigned vector);
 
diff --git a/hw/net/e1000e.c b/hw/net/e1000e.c
index ac96f7665a..7523e9f5d2 100644
--- a/hw/net/e1000e.c
+++ b/hw/net/e1000e.c
@@ -276,25 +276,18 @@ e1000e_unuse_msix_vectors(E1000EState *s, int num_vectors)
     }
 }
 
-static bool
+static void
 e1000e_use_msix_vectors(E1000EState *s, int num_vectors)
 {
     int i;
     for (i = 0; i < num_vectors; i++) {
-        int res = msix_vector_use(PCI_DEVICE(s), i);
-        if (res < 0) {
-            trace_e1000e_msix_use_vector_fail(i, res);
-            e1000e_unuse_msix_vectors(s, i);
-            return false;
-        }
+        msix_vector_use(PCI_DEVICE(s), i);
     }
-    return true;
 }
 
 static void
 e1000e_init_msix(E1000EState *s)
 {
-    PCIDevice *d = PCI_DEVICE(s);
     int res = msix_init(PCI_DEVICE(s), E1000E_MSIX_VEC_NUM,
                         &s->msix,
                         E1000E_MSIX_IDX, E1000E_MSIX_TABLE,
@@ -305,9 +298,7 @@ e1000e_init_msix(E1000EState *s)
     if (res < 0) {
         trace_e1000e_msix_init_fail(res);
     } else {
-        if (!e1000e_use_msix_vectors(s, E1000E_MSIX_VEC_NUM)) {
-            msix_uninit(d, &s->msix, &s->msix);
-        }
+        e1000e_use_msix_vectors(s, E1000E_MSIX_VEC_NUM);
     }
 }
 
diff --git a/hw/net/rocker/rocker.c b/hw/net/rocker/rocker.c
index d8f3f16fe8..281d43e6cf 100644
--- a/hw/net/rocker/rocker.c
+++ b/hw/net/rocker/rocker.c
@@ -1212,24 +1212,14 @@ static void rocker_msix_vectors_unuse(Rocker *r,
     }
 }
 
-static int rocker_msix_vectors_use(Rocker *r,
-                                   unsigned int num_vectors)
+static void rocker_msix_vectors_use(Rocker *r, unsigned int num_vectors)
 {
     PCIDevice *dev = PCI_DEVICE(r);
-    int err;
     int i;
 
     for (i = 0; i < num_vectors; i++) {
-        err = msix_vector_use(dev, i);
-        if (err) {
-            goto rollback;
-        }
+        msix_vector_use(dev, i);
     }
-    return 0;
-
-rollback:
-    rocker_msix_vectors_unuse(r, i);
-    return err;
 }
 
 static int rocker_msix_init(Rocker *r, Error **errp)
@@ -1247,16 +1237,9 @@ static int rocker_msix_init(Rocker *r, Error **errp)
         return err;
     }
 
-    err = rocker_msix_vectors_use(r, ROCKER_MSIX_VEC_COUNT(r->fp_ports));
-    if (err) {
-        goto err_msix_vectors_use;
-    }
+    rocker_msix_vectors_use(r, ROCKER_MSIX_VEC_COUNT(r->fp_ports));
 
     return 0;
-
-err_msix_vectors_use:
-    msix_uninit(dev, &r->msix_bar, &r->msix_bar);
-    return err;
 }
 
 static void rocker_msix_uninit(Rocker *r)
diff --git a/hw/net/vmxnet3.c b/hw/net/vmxnet3.c
index 0b7acf7f89..d2ab527ef4 100644
--- a/hw/net/vmxnet3.c
+++ b/hw/net/vmxnet3.c
@@ -2110,20 +2110,14 @@ vmxnet3_unuse_msix_vectors(VMXNET3State *s, int num_vectors)
     }
 }
 
-static bool
+static void
 vmxnet3_use_msix_vectors(VMXNET3State *s, int num_vectors)
 {
     PCIDevice *d = PCI_DEVICE(s);
     int i;
     for (i = 0; i < num_vectors; i++) {
-        int res = msix_vector_use(d, i);
-        if (0 > res) {
-            VMW_WRPRN("Failed to use MSI-X vector %d, error %d", i, res);
-            vmxnet3_unuse_msix_vectors(s, i);
-            return false;
-        }
+        msix_vector_use(d, i);
     }
-    return true;
 }
 
 static bool
@@ -2141,13 +2135,8 @@ vmxnet3_init_msix(VMXNET3State *s)
         VMW_WRPRN("Failed to initialize MSI-X, error %d", res);
         s->msix_used = false;
     } else {
-        if (!vmxnet3_use_msix_vectors(s, VMXNET3_MAX_INTRS)) {
-            VMW_WRPRN("Failed to use MSI-X vectors, error %d", res);
-            msix_uninit(d, &s->msix_bar, &s->msix_bar);
-            s->msix_used = false;
-        } else {
-            s->msix_used = true;
-        }
+        vmxnet3_use_msix_vectors(s, VMXNET3_MAX_INTRS);
+        s->msix_used = true;
     }
     return s->msix_used;
 }
@@ -2412,19 +2401,13 @@ static const VMStateDescription vmstate_vmxnet3_rxq_descr = {
 static int vmxnet3_post_load(void *opaque, int version_id)
 {
     VMXNET3State *s = opaque;
-    PCIDevice *d = PCI_DEVICE(s);
 
     net_tx_pkt_init(&s->tx_pkt, PCI_DEVICE(s),
                     s->max_tx_frags, s->peer_has_vhdr);
     net_rx_pkt_init(&s->rx_pkt, s->peer_has_vhdr);
 
     if (s->msix_used) {
-        if  (!vmxnet3_use_msix_vectors(s, VMXNET3_MAX_INTRS)) {
-            VMW_WRPRN("Failed to re-use MSI-X vectors");
-            msix_uninit(d, &s->msix_bar, &s->msix_bar);
-            s->msix_used = false;
-            return -1;
-        }
+        vmxnet3_use_msix_vectors(s, VMXNET3_MAX_INTRS);
     }
 
     if (!vmxnet3_validate_queues(s)) {
diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c
index 87aeba0564..d38fdd990e 100644
--- a/hw/nvme/ctrl.c
+++ b/hw/nvme/ctrl.c
@@ -4744,11 +4744,8 @@ static void nvme_init_cq(NvmeCQueue *cq, NvmeCtrl *n, uint64_t dma_addr,
                          uint16_t cqid, uint16_t vector, uint16_t size,
                          uint16_t irq_enabled)
 {
-    int ret;
-
     if (msix_enabled(&n->parent_obj)) {
-        ret = msix_vector_use(&n->parent_obj, vector);
-        assert(ret == 0);
+        msix_vector_use(&n->parent_obj, vector);
     }
     cq->ctrl = n;
     cq->cqid = cqid;
diff --git a/hw/pci/msix.c b/hw/pci/msix.c
index 1e381a9813..9e70fcd6fa 100644
--- a/hw/pci/msix.c
+++ b/hw/pci/msix.c
@@ -136,17 +136,12 @@ static void msix_handle_mask_update(PCIDevice *dev, int vector, bool was_masked)
     }
 }
 
-void msix_set_mask(PCIDevice *dev, int vector, bool mask, Error **errp)
+void msix_set_mask(PCIDevice *dev, int vector, bool mask)
 {
-    ERRP_GUARD();
     unsigned offset;
     bool was_masked;
 
-    if (vector > dev->msix_entries_nr) {
-        error_setg(errp, "msix: vector %d not allocated. max vector is %d",
-                   vector, dev->msix_entries_nr);
-        return;
-    }
+    assert(vector < dev->msix_entries_nr);
 
     offset = vector * PCI_MSIX_ENTRY_SIZE + PCI_MSIX_ENTRY_VECTOR_CTRL;
 
@@ -522,7 +517,9 @@ void msix_notify(PCIDevice *dev, unsigned vector)
 {
     MSIMessage msg;
 
-    if (vector >= dev->msix_entries_nr || !dev->msix_entry_used[vector]) {
+    assert(vector < dev->msix_entries_nr);
+
+    if (!dev->msix_entry_used[vector]) {
         return;
     }
 
@@ -558,20 +555,17 @@ void msix_reset(PCIDevice *dev)
  * don't want to follow the spec suggestion can declare all vectors as used. */
 
 /* Mark vector as used. */
-int msix_vector_use(PCIDevice *dev, unsigned vector)
+void msix_vector_use(PCIDevice *dev, unsigned vector)
 {
-    if (vector >= dev->msix_entries_nr) {
-        return -EINVAL;
-    }
-
+    assert(vector < dev->msix_entries_nr);
     dev->msix_entry_used[vector]++;
-    return 0;
 }
 
 /* Mark vector as unused. */
 void msix_vector_unuse(PCIDevice *dev, unsigned vector)
 {
-    if (vector >= dev->msix_entries_nr || !dev->msix_entry_used[vector]) {
+    assert(vector < dev->msix_entries_nr);
+    if (!dev->msix_entry_used[vector]) {
         return;
     }
     if (--dev->msix_entry_used[vector]) {
diff --git a/hw/rdma/vmw/pvrdma_main.c b/hw/rdma/vmw/pvrdma_main.c
index 58db0b8e3b..4fc6712025 100644
--- a/hw/rdma/vmw/pvrdma_main.c
+++ b/hw/rdma/vmw/pvrdma_main.c
@@ -307,12 +307,7 @@ static int init_msix(PCIDevice *pdev)
     }
 
     for (i = 0; i < RDMA_MAX_INTRS; i++) {
-        rc = msix_vector_use(PCI_DEVICE(dev), i);
-        if (rc < 0) {
-            rdma_error_report("Fail mark MSI-X vector %d", i);
-            uninit_msix(pdev, i);
-            return rc;
-        }
+        msix_vector_use(PCI_DEVICE(dev), i);
     }
 
     return 0;
diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c
index c6cc53acf2..4e36bb8bcf 100644
--- a/hw/remote/vfio-user-obj.c
+++ b/hw/remote/vfio-user-obj.c
@@ -602,17 +602,10 @@ static void vfu_msix_irq_state(vfu_ctx_t *vfu_ctx, uint32_t start,
                                uint32_t count, bool mask)
 {
     VfuObject *o = vfu_get_private(vfu_ctx);
-    Error *err = NULL;
     uint32_t vector;
 
     for (vector = start; vector < count; vector++) {
-        msix_set_mask(o->pci_dev, vector, mask, &err);
-        if (err) {
-            VFU_OBJECT_ERROR(o, "vfu: %s: %s", o->device,
-                             error_get_pretty(err));
-            error_free(err);
-            err = NULL;
-        }
+        msix_set_mask(o->pci_dev, vector, mask);
     }
 }
 
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index 855718d586..a1c9dfa7bb 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -71,9 +71,11 @@ static void virtio_pci_notify(DeviceState *d, uint16_t vector)
 {
     VirtIOPCIProxy *proxy = to_virtio_pci_proxy_fast(d);
 
-    if (msix_enabled(&proxy->pci_dev))
-        msix_notify(&proxy->pci_dev, vector);
-    else {
+    if (msix_enabled(&proxy->pci_dev)) {
+        if (vector != VIRTIO_NO_VECTOR) {
+            msix_notify(&proxy->pci_dev, vector);
+        }
+    } else {
         VirtIODevice *vdev = virtio_bus_get_device(&proxy->bus);
         pci_set_irq(&proxy->pci_dev, qatomic_read(&vdev->isr) & 1);
     }
@@ -175,6 +177,7 @@ static int virtio_pci_load_config(DeviceState *d, QEMUFile *f)
 {
     VirtIOPCIProxy *proxy = to_virtio_pci_proxy(d);
     VirtIODevice *vdev = virtio_bus_get_device(&proxy->bus);
+    uint16_t vector;
 
     int ret;
     ret = pci_device_load(&proxy->pci_dev, f);
@@ -184,12 +187,17 @@ static int virtio_pci_load_config(DeviceState *d, QEMUFile *f)
     msix_unuse_all_vectors(&proxy->pci_dev);
     msix_load(&proxy->pci_dev, f);
     if (msix_present(&proxy->pci_dev)) {
-        qemu_get_be16s(f, &vdev->config_vector);
+        qemu_get_be16s(f, &vector);
+
+        if (vector != VIRTIO_NO_VECTOR && vector >= proxy->nvectors) {
+            return -EINVAL;
+        }
     } else {
-        vdev->config_vector = VIRTIO_NO_VECTOR;
+        vector = VIRTIO_NO_VECTOR;
     }
-    if (vdev->config_vector != VIRTIO_NO_VECTOR) {
-        return msix_vector_use(&proxy->pci_dev, vdev->config_vector);
+    vdev->config_vector = vector;
+    if (vector != VIRTIO_NO_VECTOR) {
+        msix_vector_use(&proxy->pci_dev, vector);
     }
     return 0;
 }
@@ -202,12 +210,15 @@ static int virtio_pci_load_queue(DeviceState *d, int n, QEMUFile *f)
     uint16_t vector;
     if (msix_present(&proxy->pci_dev)) {
         qemu_get_be16s(f, &vector);
+        if (vector != VIRTIO_NO_VECTOR && vector >= proxy->nvectors) {
+            return -EINVAL;
+        }
     } else {
         vector = VIRTIO_NO_VECTOR;
     }
     virtio_queue_set_vector(vdev, n, vector);
     if (vector != VIRTIO_NO_VECTOR) {
-        return msix_vector_use(&proxy->pci_dev, vector);
+        msix_vector_use(&proxy->pci_dev, vector);
     }
 
     return 0;
@@ -299,6 +310,7 @@ static void virtio_ioport_write(void *opaque, uint32_t addr, uint32_t val)
 {
     VirtIOPCIProxy *proxy = opaque;
     VirtIODevice *vdev = virtio_bus_get_device(&proxy->bus);
+    uint16_t vector;
     hwaddr pa;
 
     switch (addr) {
@@ -352,18 +364,28 @@ static void virtio_ioport_write(void *opaque, uint32_t addr, uint32_t val)
         }
         break;
     case VIRTIO_MSI_CONFIG_VECTOR:
-        msix_vector_unuse(&proxy->pci_dev, vdev->config_vector);
+        if (vdev->config_vector != VIRTIO_NO_VECTOR) {
+            msix_vector_unuse(&proxy->pci_dev, vdev->config_vector);
+        }
         /* Make it possible for guest to discover an error took place. */
-        if (msix_vector_use(&proxy->pci_dev, val) < 0)
+        if (val < proxy->nvectors) {
+            msix_vector_use(&proxy->pci_dev, val);
+        } else {
             val = VIRTIO_NO_VECTOR;
+        }
         vdev->config_vector = val;
         break;
     case VIRTIO_MSI_QUEUE_VECTOR:
-        msix_vector_unuse(&proxy->pci_dev,
-                          virtio_queue_vector(vdev, vdev->queue_sel));
+        vector = virtio_queue_vector(vdev, vdev->queue_sel);
+        if (vector != VIRTIO_NO_VECTOR) {
+            msix_vector_unuse(&proxy->pci_dev, vector);
+        }
         /* Make it possible for guest to discover an error took place. */
-        if (msix_vector_use(&proxy->pci_dev, val) < 0)
+        if (val < proxy->nvectors) {
+            msix_vector_use(&proxy->pci_dev, val);
+        } else {
             val = VIRTIO_NO_VECTOR;
+        }
         virtio_queue_set_vector(vdev, vdev->queue_sel, val);
         break;
     default:
@@ -1266,6 +1288,7 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
 {
     VirtIOPCIProxy *proxy = opaque;
     VirtIODevice *vdev = virtio_bus_get_device(&proxy->bus);
+    uint16_t vector;
 
     if (vdev == NULL) {
         return;
@@ -1287,9 +1310,13 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
         }
         break;
     case VIRTIO_PCI_COMMON_MSIX:
-        msix_vector_unuse(&proxy->pci_dev, vdev->config_vector);
+        if (vdev->config_vector != VIRTIO_NO_VECTOR) {
+            msix_vector_unuse(&proxy->pci_dev, vdev->config_vector);
+        }
         /* Make it possible for guest to discover an error took place. */
-        if (msix_vector_use(&proxy->pci_dev, val) < 0) {
+        if (val < proxy->nvectors) {
+            msix_vector_use(&proxy->pci_dev, val);
+        } else {
             val = VIRTIO_NO_VECTOR;
         }
         vdev->config_vector = val;
@@ -1321,10 +1348,14 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
                              proxy->vqs[vdev->queue_sel].num);
         break;
     case VIRTIO_PCI_COMMON_Q_MSIX:
-        msix_vector_unuse(&proxy->pci_dev,
-                          virtio_queue_vector(vdev, vdev->queue_sel));
+        vector = virtio_queue_vector(vdev, vdev->queue_sel);
+        if (vector != VIRTIO_NO_VECTOR) {
+            msix_vector_unuse(&proxy->pci_dev, vector);
+        }
         /* Make it possible for guest to discover an error took place. */
-        if (msix_vector_use(&proxy->pci_dev, val) < 0) {
+        if (val < proxy->nvectors) {
+            msix_vector_use(&proxy->pci_dev, val);
+        } else {
             val = VIRTIO_NO_VECTOR;
         }
         virtio_queue_set_vector(vdev, vdev->queue_sel, val);
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 58/83] hw/i386/pc.c: CXL Fixed Memory Window should not reserve e820 in bios
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (56 preceding siblings ...)
  2022-11-07 22:52 ` [PULL v4 57/83] msix: Assert that specified vector is in range Michael S. Tsirkin
@ 2022-11-07 22:52 ` Michael S. Tsirkin
  2022-11-07 22:52 ` [PULL v4 59/83] hw/i386/acpi-build: Remove unused struct Michael S. Tsirkin
                   ` (26 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:52 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Gregory Price, Gregory Price, Jonathan Cameron,
	Paolo Bonzini, Richard Henderson, Eduardo Habkost,
	Marcel Apfelbaum

From: Gregory Price <gourry.memverge@gmail.com>

Early-boot e820 records will be inserted by the bios/efi/early boot
software and be reported to the kernel via insert_resource.  Later, when
CXL drivers iterate through the regions again, they will insert another
resource and make the RESERVED memory area a child.

This RESERVED memory area causes the memory region to become unusable,
and as a result attempting to create memory regions with

    `cxl create-region ...`

Will fail due to the RESERVED area intersecting with the CXL window.

During boot the following traceback is observed:

0xffffffff81101650 in insert_resource_expand_to_fit ()
0xffffffff83d964c5 in e820__reserve_resources_late ()
0xffffffff83e03210 in pcibios_resource_survey ()
0xffffffff83e04f4a in pcibios_init ()

Which produces a call to reserve the CFMWS area:

(gdb) p *new
$54 = {start = 0x290000000, end = 0x2cfffffff, name = "Reserved",
       flags = 0x200, desc = 0x7, parent = 0x0, sibling = 0x0,
       child = 0x0}

Later the Kernel parses ACPI tables and reserves the exact same area as
the CXL Fixed Memory Window:

0xffffffff811016a4 in insert_resource_conflict ()
                      insert_resource ()
0xffffffff81a81389 in cxl_parse_cfmws ()
0xffffffff818c4a81 in call_handler ()
                      acpi_parse_entries_array ()

(gdb) p/x *new
$59 = {start = 0x290000000, end = 0x2cfffffff, name = "CXL Window 0",
       flags = 0x200, desc = 0x0, parent = 0x0, sibling = 0x0,
       child = 0x0}

This produces the following output in /proc/iomem:

590000000-68fffffff : CXL Window 0
  590000000-68fffffff : Reserved

This reserved area causes `get_free_mem_region()` to fail due to a check
against `__region_intersects()`.  Due to this reserved area, the
intersect check will only ever return REGION_INTERSECTS, which causes
`cxl create-region` to always fail.

Signed-off-by: Gregory Price <gregory.price@memverge.com>
Message-Id: <20221026205912.8579-1-gregory.price@memverge.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
---
 hw/i386/pc.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index ef14da5094..546b703cb4 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1061,7 +1061,6 @@ void pc_memory_init(PCMachineState *pcms,
         hwaddr cxl_size = MiB;
 
         cxl_base = pc_get_cxl_range_start(pcms);
-        e820_add_entry(cxl_base, cxl_size, E820_RESERVED);
         memory_region_init(mr, OBJECT(machine), "cxl_host_reg", cxl_size);
         memory_region_add_subregion(system_memory, cxl_base, mr);
         cxl_resv_end = cxl_base + cxl_size;
@@ -1077,7 +1076,6 @@ void pc_memory_init(PCMachineState *pcms,
                 memory_region_init_io(&fw->mr, OBJECT(machine), &cfmws_ops, fw,
                                       "cxl-fixed-memory-region", fw->size);
                 memory_region_add_subregion(system_memory, fw->base, &fw->mr);
-                e820_add_entry(fw->base, fw->size, E820_RESERVED);
                 cxl_fmw_base += fw->size;
                 cxl_resv_end = cxl_fmw_base;
             }
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 59/83] hw/i386/acpi-build: Remove unused struct
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (57 preceding siblings ...)
  2022-11-07 22:52 ` [PULL v4 58/83] hw/i386/pc.c: CXL Fixed Memory Window should not reserve e820 in bios Michael S. Tsirkin
@ 2022-11-07 22:52 ` Michael S. Tsirkin
  2022-11-07 22:52 ` [PULL v4 60/83] hw/i386/acpi-build: Resolve redundant attribute Michael S. Tsirkin
                   ` (25 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:52 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Bernhard Beschow, Philippe Mathieu-Daudé,
	Igor Mammedov, Ani Sinha, Marcel Apfelbaum, Paolo Bonzini,
	Richard Henderson, Eduardo Habkost

From: Bernhard Beschow <shentey@gmail.com>

Ammends commit b23046abe78f48498a423b802d6d86ba0172d57f 'pc: acpi-build:
simplify PCI bus tree generation'.

Signed-off-by: Bernhard Beschow <shentey@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20221026133110.91828-2-shentey@gmail.com>
Message-Id: <20221028103419.93398-2-shentey@gmail.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/i386/acpi-build.c | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 960305462c..1ebf14b899 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -121,13 +121,6 @@ typedef struct AcpiMiscInfo {
     unsigned dsdt_size;
 } AcpiMiscInfo;
 
-typedef struct AcpiBuildPciBusHotplugState {
-    GArray *device_table;
-    GArray *notify_table;
-    struct AcpiBuildPciBusHotplugState *parent;
-    bool pcihp_bridge_en;
-} AcpiBuildPciBusHotplugState;
-
 typedef struct FwCfgTPMConfig {
     uint32_t tpmppi_address;
     uint8_t tpm_version;
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 60/83] hw/i386/acpi-build: Resolve redundant attribute
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (58 preceding siblings ...)
  2022-11-07 22:52 ` [PULL v4 59/83] hw/i386/acpi-build: Remove unused struct Michael S. Tsirkin
@ 2022-11-07 22:52 ` Michael S. Tsirkin
  2022-11-07 22:52 ` [PULL v4 61/83] hw/i386/acpi-build: Resolve north rather than south bridges Michael S. Tsirkin
                   ` (24 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:52 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Bernhard Beschow, Philippe Mathieu-Daudé,
	Igor Mammedov, Ani Sinha, Paolo Bonzini, Richard Henderson,
	Eduardo Habkost, Marcel Apfelbaum

From: Bernhard Beschow <shentey@gmail.com>

The is_piix4 attribute is set once in one location and read once in
another. Doing both in one location allows for removing the attribute
altogether.

Signed-off-by: Bernhard Beschow <shentey@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20221026133110.91828-3-shentey@gmail.com>
Message-Id: <20221028103419.93398-3-shentey@gmail.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/i386/acpi-build.c | 20 ++++++--------------
 1 file changed, 6 insertions(+), 14 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 1ebf14b899..73d8a59737 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -112,7 +112,6 @@ typedef struct AcpiPmInfo {
 } AcpiPmInfo;
 
 typedef struct AcpiMiscInfo {
-    bool is_piix4;
     bool has_hpet;
 #ifdef CONFIG_TPM
     TPMVersion tpm_version;
@@ -281,17 +280,6 @@ static void acpi_get_pm_info(MachineState *machine, AcpiPmInfo *pm)
 
 static void acpi_get_misc_info(AcpiMiscInfo *info)
 {
-    Object *piix = object_resolve_type_unambiguous(TYPE_PIIX4_PM);
-    Object *lpc = object_resolve_type_unambiguous(TYPE_ICH9_LPC_DEVICE);
-    assert(!!piix != !!lpc);
-
-    if (piix) {
-        info->is_piix4 = true;
-    }
-    if (lpc) {
-        info->is_piix4 = false;
-    }
-
     info->has_hpet = hpet_find();
 #ifdef CONFIG_TPM
     info->tpm_version = tpm_get_version(tpm_find());
@@ -1334,6 +1322,8 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
            AcpiPmInfo *pm, AcpiMiscInfo *misc,
            Range *pci_hole, Range *pci_hole64, MachineState *machine)
 {
+    Object *piix = object_resolve_type_unambiguous(TYPE_PIIX4_PM);
+    Object *lpc = object_resolve_type_unambiguous(TYPE_ICH9_LPC_DEVICE);
     CrsRangeEntry *entry;
     Aml *dsdt, *sb_scope, *scope, *dev, *method, *field, *pkg, *crs;
     CrsRangeSet crs_range_set;
@@ -1354,11 +1344,13 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
     AcpiTable table = { .sig = "DSDT", .rev = 1, .oem_id = x86ms->oem_id,
                         .oem_table_id = x86ms->oem_table_id };
 
+    assert(!!piix != !!lpc);
+
     acpi_table_begin(&table, table_data);
     dsdt = init_aml_allocator();
 
     build_dbg_aml(dsdt);
-    if (misc->is_piix4) {
+    if (piix) {
         sb_scope = aml_scope("_SB");
         dev = aml_device("PCI0");
         aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0A03")));
@@ -1371,7 +1363,7 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
             build_x86_acpi_pci_hotplug(dsdt, pm->pcihp_io_base);
         }
         build_piix4_pci0_int(dsdt);
-    } else {
+    } else if (lpc) {
         sb_scope = aml_scope("_SB");
         dev = aml_device("PCI0");
         aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0A08")));
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 61/83] hw/i386/acpi-build: Resolve north rather than south bridges
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (59 preceding siblings ...)
  2022-11-07 22:52 ` [PULL v4 60/83] hw/i386/acpi-build: Resolve redundant attribute Michael S. Tsirkin
@ 2022-11-07 22:52 ` Michael S. Tsirkin
  2022-11-07 22:52 ` [PULL v4 62/83] hmat acpi: Don't require initiator value in -numa Michael S. Tsirkin
                   ` (23 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:52 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Bernhard Beschow, Igor Mammedov, Ani Sinha,
	Paolo Bonzini, Richard Henderson, Eduardo Habkost,
	Marcel Apfelbaum

From: Bernhard Beschow <shentey@gmail.com>

The code currently assumes Q35 iff ICH9 and i440fx iff PIIX. Now that more
AML generation has been moved into the south bridges and since the
machines define themselves primarily through their north bridges, let's
switch to resolving the north bridges for AML generation instead. This
also allows for easier experimentation with different south bridges in
the "pc" machine, e.g. with PIIX4 and VT82xx.

Signed-off-by: Bernhard Beschow <shentey@gmail.com>
Message-Id: <20221028103419.93398-4-shentey@gmail.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/i386/acpi-build.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 73d8a59737..d9eaa5fc4d 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -60,6 +60,7 @@
 #include "hw/i386/fw_cfg.h"
 #include "hw/i386/ich9.h"
 #include "hw/pci/pci_bus.h"
+#include "hw/pci-host/i440fx.h"
 #include "hw/pci-host/q35.h"
 #include "hw/i386/x86-iommu.h"
 
@@ -1322,8 +1323,8 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
            AcpiPmInfo *pm, AcpiMiscInfo *misc,
            Range *pci_hole, Range *pci_hole64, MachineState *machine)
 {
-    Object *piix = object_resolve_type_unambiguous(TYPE_PIIX4_PM);
-    Object *lpc = object_resolve_type_unambiguous(TYPE_ICH9_LPC_DEVICE);
+    Object *i440fx = object_resolve_type_unambiguous(TYPE_I440FX_PCI_HOST_BRIDGE);
+    Object *q35 = object_resolve_type_unambiguous(TYPE_Q35_HOST_DEVICE);
     CrsRangeEntry *entry;
     Aml *dsdt, *sb_scope, *scope, *dev, *method, *field, *pkg, *crs;
     CrsRangeSet crs_range_set;
@@ -1344,13 +1345,13 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
     AcpiTable table = { .sig = "DSDT", .rev = 1, .oem_id = x86ms->oem_id,
                         .oem_table_id = x86ms->oem_table_id };
 
-    assert(!!piix != !!lpc);
+    assert(!!i440fx != !!q35);
 
     acpi_table_begin(&table, table_data);
     dsdt = init_aml_allocator();
 
     build_dbg_aml(dsdt);
-    if (piix) {
+    if (i440fx) {
         sb_scope = aml_scope("_SB");
         dev = aml_device("PCI0");
         aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0A03")));
@@ -1363,7 +1364,7 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
             build_x86_acpi_pci_hotplug(dsdt, pm->pcihp_io_base);
         }
         build_piix4_pci0_int(dsdt);
-    } else if (lpc) {
+    } else if (q35) {
         sb_scope = aml_scope("_SB");
         dev = aml_device("PCI0");
         aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0A08")));
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 62/83] hmat acpi: Don't require initiator value in -numa
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (60 preceding siblings ...)
  2022-11-07 22:52 ` [PULL v4 61/83] hw/i386/acpi-build: Resolve north rather than south bridges Michael S. Tsirkin
@ 2022-11-07 22:52 ` Michael S. Tsirkin
  2022-11-07 22:52 ` [PULL v4 63/83] tests: acpi: add and whitelist *.hmat-noinitiator expected blobs Michael S. Tsirkin
                   ` (22 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:52 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Brice Goglin, Hesham Almatary, Jingqi Liu,
	Yicong Yang, Eduardo Habkost, Marcel Apfelbaum,
	Philippe Mathieu-Daudé,
	Yanan Wang

From: Brice Goglin <Brice.Goglin@inria.fr>

The "Memory Proximity Domain Attributes" structure of the ACPI HMAT
has a "Processor Proximity Domain Valid" flag that is currently
always set because Qemu -numa requires an initiator=X value
when hmat=on. Unsetting this flag allows to create more complex
memory topologies by having multiple best initiators for a single
memory target.

This patch allows -numa without initiator=X when hmat=on by keeping
the default value MAX_NODES in numa_state->nodes[i].initiator.
All places reading numa_state->nodes[i].initiator already check
whether it's different from MAX_NODES before using it.

Tested with
qemu-system-x86_64 -accel kvm \
 -machine pc,hmat=on \
 -drive if=pflash,format=raw,file=./OVMF.fd \
 -drive media=disk,format=qcow2,file=efi.qcow2 \
 -smp 4 \
 -m 3G \
 -object memory-backend-ram,size=1G,id=ram0 \
 -object memory-backend-ram,size=1G,id=ram1 \
 -object memory-backend-ram,size=1G,id=ram2 \
 -numa node,nodeid=0,memdev=ram0,cpus=0-1 \
 -numa node,nodeid=1,memdev=ram1,cpus=2-3 \
 -numa node,nodeid=2,memdev=ram2 \
 -numa hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-latency,latency=10 \
 -numa hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-bandwidth,bandwidth=10485760 \
 -numa hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=access-latency,latency=20 \
 -numa hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=access-bandwidth,bandwidth=5242880 \
 -numa hmat-lb,initiator=0,target=2,hierarchy=memory,data-type=access-latency,latency=30 \
 -numa hmat-lb,initiator=0,target=2,hierarchy=memory,data-type=access-bandwidth,bandwidth=1048576 \
 -numa hmat-lb,initiator=1,target=0,hierarchy=memory,data-type=access-latency,latency=20 \
 -numa hmat-lb,initiator=1,target=0,hierarchy=memory,data-type=access-bandwidth,bandwidth=5242880 \
 -numa hmat-lb,initiator=1,target=1,hierarchy=memory,data-type=access-latency,latency=10 \
 -numa hmat-lb,initiator=1,target=1,hierarchy=memory,data-type=access-bandwidth,bandwidth=10485760 \
 -numa hmat-lb,initiator=1,target=2,hierarchy=memory,data-type=access-latency,latency=30 \
 -numa hmat-lb,initiator=1,target=2,hierarchy=memory,data-type=access-bandwidth,bandwidth=1048576
which reports NUMA node2 at same distance from both node0 and node1 as seen in lstopo:
Machine (2966MB total) + Package P#0
  NUMANode P#2 (979MB)
  Group0
    NUMANode P#0 (980MB)
    Core P#0 + PU P#0
    Core P#1 + PU P#1
  Group0
    NUMANode P#1 (1007MB)
    Core P#2 + PU P#2
    Core P#3 + PU P#3

Before this patch, we had to add ",initiator=X" to "-numa node,nodeid=2,memdev=ram2".
The lstopo output difference between initiator=1 and no initiator is:
@@ -1,10 +1,10 @@
 Machine (2966MB total) + Package P#0
+  NUMANode P#2 (979MB)
   Group0
     NUMANode P#0 (980MB)
     Core P#0 + PU P#0
     Core P#1 + PU P#1
   Group0
     NUMANode P#1 (1007MB)
-    NUMANode P#2 (979MB)
     Core P#2 + PU P#2
     Core P#3 + PU P#3

Corresponding changes in the HMAT MPDA structure:
@@ -49,10 +49,10 @@
 [078h 0120   2]               Structure Type : 0000 [Memory Proximity Domain Attributes]
 [07Ah 0122   2]                     Reserved : 0000
 [07Ch 0124   4]                       Length : 00000028
-[080h 0128   2]        Flags (decoded below) : 0001
-            Processor Proximity Domain Valid : 1
+[080h 0128   2]        Flags (decoded below) : 0000
+            Processor Proximity Domain Valid : 0
 [082h 0130   2]                    Reserved1 : 0000
-[084h 0132   4] Attached Initiator Proximity Domain : 00000001
+[084h 0132   4] Attached Initiator Proximity Domain : 00000080
 [088h 0136   4]      Memory Proximity Domain : 00000002
 [08Ch 0140   4]                    Reserved2 : 00000000
 [090h 0144   8]                    Reserved3 : 0000000000000000

Final HMAT SLLB structures:
[0A0h 0160   2]               Structure Type : 0001 [System Locality Latency and Bandwidth Information]
[0A2h 0162   2]                     Reserved : 0000
[0A4h 0164   4]                       Length : 00000040
[0A8h 0168   1]        Flags (decoded below) : 00
                            Memory Hierarchy : 0
[0A9h 0169   1]                    Data Type : 00
[0AAh 0170   2]                    Reserved1 : 0000
[0ACh 0172   4] Initiator Proximity Domains # : 00000002
[0B0h 0176   4]   Target Proximity Domains # : 00000003
[0B4h 0180   4]                    Reserved2 : 00000000
[0B8h 0184   8]              Entry Base Unit : 0000000000002710
[0C0h 0192   4] Initiator Proximity Domain List : 00000000
[0C4h 0196   4] Initiator Proximity Domain List : 00000001
[0C8h 0200   4] Target Proximity Domain List : 00000000
[0CCh 0204   4] Target Proximity Domain List : 00000001
[0D0h 0208   4] Target Proximity Domain List : 00000002
[0D4h 0212   2]                        Entry : 0001
[0D6h 0214   2]                        Entry : 0002
[0D8h 0216   2]                        Entry : 0003
[0DAh 0218   2]                        Entry : 0002
[0DCh 0220   2]                        Entry : 0001
[0DEh 0222   2]                        Entry : 0003

[0E0h 0224   2]               Structure Type : 0001 [System Locality Latency and Bandwidth Information]
[0E2h 0226   2]                     Reserved : 0000
[0E4h 0228   4]                       Length : 00000040
[0E8h 0232   1]        Flags (decoded below) : 00
                            Memory Hierarchy : 0
[0E9h 0233   1]                    Data Type : 03
[0EAh 0234   2]                    Reserved1 : 0000
[0ECh 0236   4] Initiator Proximity Domains # : 00000002
[0F0h 0240   4]   Target Proximity Domains # : 00000003
[0F4h 0244   4]                    Reserved2 : 00000000
[0F8h 0248   8]              Entry Base Unit : 0000000000000001
[100h 0256   4] Initiator Proximity Domain List : 00000000
[104h 0260   4] Initiator Proximity Domain List : 00000001
[108h 0264   4] Target Proximity Domain List : 00000000
[10Ch 0268   4] Target Proximity Domain List : 00000001
[110h 0272   4] Target Proximity Domain List : 00000002
[114h 0276   2]                        Entry : 000A
[116h 0278   2]                        Entry : 0005
[118h 0280   2]                        Entry : 0001
[11Ah 0282   2]                        Entry : 0005
[11Ch 0284   2]                        Entry : 000A
[11Eh 0286   2]                        Entry : 0001

Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>
Signed-off-by: Hesham Almatary <hesham.almatary@huawei.com>
Reviewed-by: Jingqi Liu <jingqi.liu@intel.com>
Message-Id: <20221027100037.251-2-hesham.almatary@huawei.com>
Tested-by: Yicong Yang <yangyicong@hisilicon.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/core/machine.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index 907fa78ff0..8d34caa31d 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -1178,9 +1178,7 @@ static void numa_validate_initiator(NumaState *numa_state)
 
     for (i = 0; i < numa_state->num_nodes; i++) {
         if (numa_info[i].initiator == MAX_NODES) {
-            error_report("The initiator of NUMA node %d is missing, use "
-                         "'-numa node,initiator' option to declare it", i);
-            exit(1);
+            continue;
         }
 
         if (!numa_info[numa_info[i].initiator].present) {
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 63/83] tests: acpi: add and whitelist *.hmat-noinitiator expected blobs
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (61 preceding siblings ...)
  2022-11-07 22:52 ` [PULL v4 62/83] hmat acpi: Don't require initiator value in -numa Michael S. Tsirkin
@ 2022-11-07 22:52 ` Michael S. Tsirkin
  2022-11-07 22:52 ` [PULL v4 64/83] tests: acpi: q35: add test for hmat nodes without initiators Michael S. Tsirkin
                   ` (21 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:52 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Brice Goglin, Hesham Almatary, Yicong Yang,
	Igor Mammedov, Ani Sinha

From: Brice Goglin <Brice.Goglin@inria.fr>

.. which will be used by follow up hmat-noinitiator test-case.

Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>
Signed-off-by: Hesham Almatary <hesham.almatary@huawei.com>
Message-Id: <20221027100037.251-3-hesham.almatary@huawei.com>
Tested-by: Yicong Yang <yangyicong@hisilicon.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/qtest/bios-tables-test-allowed-diff.h   | 4 ++++
 tests/data/acpi/q35/APIC.acpihmat-noinitiator | 0
 tests/data/acpi/q35/DSDT.acpihmat-noinitiator | 0
 tests/data/acpi/q35/HMAT.acpihmat-noinitiator | 0
 tests/data/acpi/q35/SRAT.acpihmat-noinitiator | 0
 5 files changed, 4 insertions(+)
 create mode 100644 tests/data/acpi/q35/APIC.acpihmat-noinitiator
 create mode 100644 tests/data/acpi/q35/DSDT.acpihmat-noinitiator
 create mode 100644 tests/data/acpi/q35/HMAT.acpihmat-noinitiator
 create mode 100644 tests/data/acpi/q35/SRAT.acpihmat-noinitiator

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index dfb8523c8b..245fa66bcc 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1 +1,5 @@
 /* List of comma-separated changed AML files to ignore */
+"tests/data/acpi/q35/APIC.acpihmat-noinitiator",
+"tests/data/acpi/q35/DSDT.acpihmat-noinitiator",
+"tests/data/acpi/q35/HMAT.acpihmat-noinitiator",
+"tests/data/acpi/q35/SRAT.acpihmat-noinitiator",
diff --git a/tests/data/acpi/q35/APIC.acpihmat-noinitiator b/tests/data/acpi/q35/APIC.acpihmat-noinitiator
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/data/acpi/q35/DSDT.acpihmat-noinitiator b/tests/data/acpi/q35/DSDT.acpihmat-noinitiator
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/data/acpi/q35/HMAT.acpihmat-noinitiator b/tests/data/acpi/q35/HMAT.acpihmat-noinitiator
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/data/acpi/q35/SRAT.acpihmat-noinitiator b/tests/data/acpi/q35/SRAT.acpihmat-noinitiator
new file mode 100644
index 0000000000..e69de29bb2
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 64/83] tests: acpi: q35: add test for hmat nodes without initiators
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (62 preceding siblings ...)
  2022-11-07 22:52 ` [PULL v4 63/83] tests: acpi: add and whitelist *.hmat-noinitiator expected blobs Michael S. Tsirkin
@ 2022-11-07 22:52 ` Michael S. Tsirkin
  2022-11-07 22:52 ` [PULL v4 65/83] tests: acpi: q35: update expected blobs *.hmat-noinitiators expected HMAT: Michael S. Tsirkin
                   ` (20 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:52 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Brice Goglin, Hesham Almatary, Yicong Yang,
	Igor Mammedov, Ani Sinha

From: Brice Goglin <Brice.Goglin@inria.fr>

expected HMAT:

[000h 0000   4]                    Signature : "HMAT"    [Heterogeneous Memory Attributes Table]
[004h 0004   4]                 Table Length : 00000120
[008h 0008   1]                     Revision : 02
[009h 0009   1]                     Checksum : 4F
[00Ah 0010   6]                       Oem ID : "BOCHS "
[010h 0016   8]                 Oem Table ID : "BXPC    "
[018h 0024   4]                 Oem Revision : 00000001
[01Ch 0028   4]              Asl Compiler ID : "BXPC"
[020h 0032   4]        Asl Compiler Revision : 00000001

[024h 0036   4]                     Reserved : 00000000

[028h 0040   2]               Structure Type : 0000 [Memory Proximity Domain Attributes]
[02Ah 0042   2]                     Reserved : 0000
[02Ch 0044   4]                       Length : 00000028
[030h 0048   2]        Flags (decoded below) : 0001
            Processor Proximity Domain Valid : 1
[032h 0050   2]                    Reserved1 : 0000
[034h 0052   4] Attached Initiator Proximity Domain : 00000000
[038h 0056   4]      Memory Proximity Domain : 00000000
[03Ch 0060   4]                    Reserved2 : 00000000
[040h 0064   8]                    Reserved3 : 0000000000000000
[048h 0072   8]                    Reserved4 : 0000000000000000

[050h 0080   2]               Structure Type : 0000 [Memory Proximity Domain Attributes]
[052h 0082   2]                     Reserved : 0000
[054h 0084   4]                       Length : 00000028
[058h 0088   2]        Flags (decoded below) : 0001
            Processor Proximity Domain Valid : 1
[05Ah 0090   2]                    Reserved1 : 0000
[05Ch 0092   4] Attached Initiator Proximity Domain : 00000001
[060h 0096   4]      Memory Proximity Domain : 00000001
[064h 0100   4]                    Reserved2 : 00000000
[068h 0104   8]                    Reserved3 : 0000000000000000
[070h 0112   8]                    Reserved4 : 0000000000000000

[078h 0120   2]               Structure Type : 0000 [Memory Proximity Domain Attributes]
[07Ah 0122   2]                     Reserved : 0000
[07Ch 0124   4]                       Length : 00000028
[080h 0128   2]        Flags (decoded below) : 0000
            Processor Proximity Domain Valid : 0
[082h 0130   2]                    Reserved1 : 0000
[084h 0132   4] Attached Initiator Proximity Domain : 00000080
[088h 0136   4]      Memory Proximity Domain : 00000002
[08Ch 0140   4]                    Reserved2 : 00000000
[090h 0144   8]                    Reserved3 : 0000000000000000
[098h 0152   8]                    Reserved4 : 0000000000000000

[0A0h 0160   2]               Structure Type : 0001 [System Locality Latency and Bandwidth Information]
[0A2h 0162   2]                     Reserved : 0000
[0A4h 0164   4]                       Length : 00000040
[0A8h 0168   1]        Flags (decoded below) : 00
                            Memory Hierarchy : 0
[0A9h 0169   1]                    Data Type : 00
[0AAh 0170   2]                    Reserved1 : 0000
[0ACh 0172   4] Initiator Proximity Domains # : 00000002
[0B0h 0176   4]   Target Proximity Domains # : 00000003
[0B4h 0180   4]                    Reserved2 : 00000000
[0B8h 0184   8]              Entry Base Unit : 0000000000002710
[0C0h 0192   4] Initiator Proximity Domain List : 00000000
[0C4h 0196   4] Initiator Proximity Domain List : 00000001
[0C8h 0200   4] Target Proximity Domain List : 00000000
[0CCh 0204   4] Target Proximity Domain List : 00000001
[0D0h 0208   4] Target Proximity Domain List : 00000002
[0D4h 0212   2]                        Entry : 0001
[0D6h 0214   2]                        Entry : 0002
[0D8h 0216   2]                        Entry : 0003
[0DAh 0218   2]                        Entry : 0002
[0DCh 0220   2]                        Entry : 0001
[0DEh 0222   2]                        Entry : 0003

[0E0h 0224   2]               Structure Type : 0001 [System Locality Latency and Bandwidth Information]
[0E2h 0226   2]                     Reserved : 0000
[0E4h 0228   4]                       Length : 00000040
[0E8h 0232   1]        Flags (decoded below) : 00
                            Memory Hierarchy : 0
[0E9h 0233   1]                    Data Type : 03
[0EAh 0234   2]                    Reserved1 : 0000
[0ECh 0236   4] Initiator Proximity Domains # : 00000002
[0F0h 0240   4]   Target Proximity Domains # : 00000003
[0F4h 0244   4]                    Reserved2 : 00000000
[0F8h 0248   8]              Entry Base Unit : 0000000000000001
[100h 0256   4] Initiator Proximity Domain List : 00000000
[104h 0260   4] Initiator Proximity Domain List : 00000001
[108h 0264   4] Target Proximity Domain List : 00000000
[10Ch 0268   4] Target Proximity Domain List : 00000001
[110h 0272   4] Target Proximity Domain List : 00000002
[114h 0276   2]                        Entry : 000A
[116h 0278   2]                        Entry : 0005
[118h 0280   2]                        Entry : 0001
[11Ah 0282   2]                        Entry : 0005
[11Ch 0284   2]                        Entry : 000A
[11Eh 0286   2]                        Entry : 0001

Raw Table Data: Length 288 (0x120)

    0000: 48 4D 41 54 20 01 00 00 02 4F 42 4F 43 48 53 20  // HMAT ....OBOCHS
    0010: 42 58 50 43 20 20 20 20 01 00 00 00 42 58 50 43  // BXPC    ....BXPC
    0020: 01 00 00 00 00 00 00 00 00 00 00 00 28 00 00 00  // ............(...
    0030: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
    0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
    0050: 00 00 00 00 28 00 00 00 01 00 00 00 01 00 00 00  // ....(...........
    0060: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
    0070: 00 00 00 00 00 00 00 00 00 00 00 00 28 00 00 00  // ............(...
    0080: 00 00 00 00 80 00 00 00 02 00 00 00 00 00 00 00  // ................
    0090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
    00A0: 01 00 00 00 40 00 00 00 00 00 00 00 02 00 00 00  // ....@...........
    00B0: 03 00 00 00 00 00 00 00 10 27 00 00 00 00 00 00  // .........'......
    00C0: 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00  // ................
    00D0: 02 00 00 00 01 00 02 00 03 00 02 00 01 00 03 00  // ................
    00E0: 01 00 00 00 40 00 00 00 00 03 00 00 02 00 00 00  // ....@...........
    00F0: 03 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00  // ................
    0100: 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00  // ................
    0110: 02 00 00 00 0A 00 05 00 01 00 05 00 0A 00 01 00  // ................

Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>
Signed-off-by: Hesham Almatary <hesham.almatary@huawei.com>
Message-Id: <20221027100037.251-4-hesham.almatary@huawei.com>
Tested-by: Yicong Yang <yangyicong@hisilicon.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/qtest/bios-tables-test.c | 50 ++++++++++++++++++++++++++++++++++
 1 file changed, 50 insertions(+)

diff --git a/tests/qtest/bios-tables-test.c b/tests/qtest/bios-tables-test.c
index e6096e7f73..02fe59fbf8 100644
--- a/tests/qtest/bios-tables-test.c
+++ b/tests/qtest/bios-tables-test.c
@@ -1461,6 +1461,54 @@ static void test_acpi_piix4_tcg_acpi_hmat(void)
     test_acpi_tcg_acpi_hmat(MACHINE_PC);
 }
 
+static void test_acpi_q35_tcg_acpi_hmat_noinitiator(void)
+{
+    test_data data;
+
+    memset(&data, 0, sizeof(data));
+    data.machine = MACHINE_Q35;
+    data.variant = ".acpihmat-noinitiator";
+    test_acpi_one(" -machine hmat=on"
+                  " -smp 4,sockets=2"
+                  " -m 128M"
+                  " -object memory-backend-ram,size=32M,id=ram0"
+                  " -object memory-backend-ram,size=32M,id=ram1"
+                  " -object memory-backend-ram,size=64M,id=ram2"
+                  " -numa node,nodeid=0,memdev=ram0"
+                  " -numa node,nodeid=1,memdev=ram1"
+                  " -numa node,nodeid=2,memdev=ram2"
+                  " -numa cpu,node-id=0,socket-id=0"
+                  " -numa cpu,node-id=0,socket-id=0"
+                  " -numa cpu,node-id=1,socket-id=1"
+                  " -numa cpu,node-id=1,socket-id=1"
+                  " -numa hmat-lb,initiator=0,target=0,hierarchy=memory,"
+                  "data-type=access-latency,latency=10"
+                  " -numa hmat-lb,initiator=0,target=0,hierarchy=memory,"
+                  "data-type=access-bandwidth,bandwidth=10485760"
+                  " -numa hmat-lb,initiator=0,target=1,hierarchy=memory,"
+                  "data-type=access-latency,latency=20"
+                  " -numa hmat-lb,initiator=0,target=1,hierarchy=memory,"
+                  "data-type=access-bandwidth,bandwidth=5242880"
+                  " -numa hmat-lb,initiator=0,target=2,hierarchy=memory,"
+                  "data-type=access-latency,latency=30"
+                  " -numa hmat-lb,initiator=0,target=2,hierarchy=memory,"
+                  "data-type=access-bandwidth,bandwidth=1048576"
+                  " -numa hmat-lb,initiator=1,target=0,hierarchy=memory,"
+                  "data-type=access-latency,latency=20"
+                  " -numa hmat-lb,initiator=1,target=0,hierarchy=memory,"
+                  "data-type=access-bandwidth,bandwidth=5242880"
+                  " -numa hmat-lb,initiator=1,target=1,hierarchy=memory,"
+                  "data-type=access-latency,latency=10"
+                  " -numa hmat-lb,initiator=1,target=1,hierarchy=memory,"
+                  "data-type=access-bandwidth,bandwidth=10485760"
+                  " -numa hmat-lb,initiator=1,target=2,hierarchy=memory,"
+                  "data-type=access-latency,latency=30"
+                  " -numa hmat-lb,initiator=1,target=2,hierarchy=memory,"
+                  "data-type=access-bandwidth,bandwidth=1048576",
+                  &data);
+    free_test_data(&data);
+}
+
 #ifdef CONFIG_POSIX
 static void test_acpi_erst(const char *machine)
 {
@@ -1824,6 +1872,8 @@ int main(int argc, char *argv[])
             qtest_add_func("acpi/q35/nohpet", test_acpi_q35_tcg_nohpet);
             qtest_add_func("acpi/q35/dimmpxm", test_acpi_q35_tcg_dimm_pxm);
             qtest_add_func("acpi/q35/acpihmat", test_acpi_q35_tcg_acpi_hmat);
+            qtest_add_func("acpi/q35/acpihmat-noinitiator",
+                           test_acpi_q35_tcg_acpi_hmat_noinitiator);
 #ifdef CONFIG_POSIX
             qtest_add_func("acpi/q35/acpierst", test_acpi_q35_acpi_erst);
 #endif
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 65/83] tests: acpi: q35: update expected blobs *.hmat-noinitiators expected HMAT:
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (63 preceding siblings ...)
  2022-11-07 22:52 ` [PULL v4 64/83] tests: acpi: q35: add test for hmat nodes without initiators Michael S. Tsirkin
@ 2022-11-07 22:52 ` Michael S. Tsirkin
  2022-11-07 22:53 ` [PULL v4 66/83] tests: Add HMAT AArch64/virt empty table files Michael S. Tsirkin
                   ` (19 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:52 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Brice Goglin, Hesham Almatary, Yicong Yang,
	Igor Mammedov, Ani Sinha

From: Brice Goglin <Brice.Goglin@inria.fr>

[000h 0000   4]                    Signature : "HMAT"    [Heterogeneous Memory Attributes Table]
[004h 0004   4]                 Table Length : 00000120
[008h 0008   1]                     Revision : 02
[009h 0009   1]                     Checksum : 4F
[00Ah 0010   6]                       Oem ID : "BOCHS "
[010h 0016   8]                 Oem Table ID : "BXPC    "
[018h 0024   4]                 Oem Revision : 00000001
[01Ch 0028   4]              Asl Compiler ID : "BXPC"
[020h 0032   4]        Asl Compiler Revision : 00000001

[024h 0036   4]                     Reserved : 00000000

[028h 0040   2]               Structure Type : 0000 [Memory Proximity Domain Attributes]
[02Ah 0042   2]                     Reserved : 0000
[02Ch 0044   4]                       Length : 00000028
[030h 0048   2]        Flags (decoded below) : 0001
            Processor Proximity Domain Valid : 1
[032h 0050   2]                    Reserved1 : 0000
[034h 0052   4] Attached Initiator Proximity Domain : 00000000
[038h 0056   4]      Memory Proximity Domain : 00000000
[03Ch 0060   4]                    Reserved2 : 00000000
[040h 0064   8]                    Reserved3 : 0000000000000000
[048h 0072   8]                    Reserved4 : 0000000000000000

[050h 0080   2]               Structure Type : 0000 [Memory Proximity Domain Attributes]
[052h 0082   2]                     Reserved : 0000
[054h 0084   4]                       Length : 00000028
[058h 0088   2]        Flags (decoded below) : 0001
            Processor Proximity Domain Valid : 1
[05Ah 0090   2]                    Reserved1 : 0000
[05Ch 0092   4] Attached Initiator Proximity Domain : 00000001
[060h 0096   4]      Memory Proximity Domain : 00000001
[064h 0100   4]                    Reserved2 : 00000000
[068h 0104   8]                    Reserved3 : 0000000000000000
[070h 0112   8]                    Reserved4 : 0000000000000000

[078h 0120   2]               Structure Type : 0000 [Memory Proximity Domain Attributes]
[07Ah 0122   2]                     Reserved : 0000
[07Ch 0124   4]                       Length : 00000028
[080h 0128   2]        Flags (decoded below) : 0000
            Processor Proximity Domain Valid : 0
[082h 0130   2]                    Reserved1 : 0000
[084h 0132   4] Attached Initiator Proximity Domain : 00000080
[088h 0136   4]      Memory Proximity Domain : 00000002
[08Ch 0140   4]                    Reserved2 : 00000000
[090h 0144   8]                    Reserved3 : 0000000000000000
[098h 0152   8]                    Reserved4 : 0000000000000000

[0A0h 0160   2]               Structure Type : 0001 [System Locality Latency and Bandwidth Information]
[0A2h 0162   2]                     Reserved : 0000
[0A4h 0164   4]                       Length : 00000040
[0A8h 0168   1]        Flags (decoded below) : 00
                            Memory Hierarchy : 0
[0A9h 0169   1]                    Data Type : 00
[0AAh 0170   2]                    Reserved1 : 0000
[0ACh 0172   4] Initiator Proximity Domains # : 00000002
[0B0h 0176   4]   Target Proximity Domains # : 00000003
[0B4h 0180   4]                    Reserved2 : 00000000
[0B8h 0184   8]              Entry Base Unit : 0000000000002710
[0C0h 0192   4] Initiator Proximity Domain List : 00000000
[0C4h 0196   4] Initiator Proximity Domain List : 00000001
[0C8h 0200   4] Target Proximity Domain List : 00000000
[0CCh 0204   4] Target Proximity Domain List : 00000001
[0D0h 0208   4] Target Proximity Domain List : 00000002
[0D4h 0212   2]                        Entry : 0001
[0D6h 0214   2]                        Entry : 0002
[0D8h 0216   2]                        Entry : 0003
[0DAh 0218   2]                        Entry : 0002
[0DCh 0220   2]                        Entry : 0001
[0DEh 0222   2]                        Entry : 0003

[0E0h 0224   2]               Structure Type : 0001 [System Locality Latency and Bandwidth Information]
[0E2h 0226   2]                     Reserved : 0000
[0E4h 0228   4]                       Length : 00000040
[0E8h 0232   1]        Flags (decoded below) : 00
                            Memory Hierarchy : 0
[0E9h 0233   1]                    Data Type : 03
[0EAh 0234   2]                    Reserved1 : 0000
[0ECh 0236   4] Initiator Proximity Domains # : 00000002
[0F0h 0240   4]   Target Proximity Domains # : 00000003
[0F4h 0244   4]                    Reserved2 : 00000000
[0F8h 0248   8]              Entry Base Unit : 0000000000000001
[100h 0256   4] Initiator Proximity Domain List : 00000000
[104h 0260   4] Initiator Proximity Domain List : 00000001
[108h 0264   4] Target Proximity Domain List : 00000000
[10Ch 0268   4] Target Proximity Domain List : 00000001
[110h 0272   4] Target Proximity Domain List : 00000002
[114h 0276   2]                        Entry : 000A
[116h 0278   2]                        Entry : 0005
[118h 0280   2]                        Entry : 0001
[11Ah 0282   2]                        Entry : 0005
[11Ch 0284   2]                        Entry : 000A
[11Eh 0286   2]                        Entry : 0001

Raw Table Data: Length 288 (0x120)

    0000: 48 4D 41 54 20 01 00 00 02 4F 42 4F 43 48 53 20  // HMAT ....OBOCHS
    0010: 42 58 50 43 20 20 20 20 01 00 00 00 42 58 50 43  // BXPC    ....BXPC
    0020: 01 00 00 00 00 00 00 00 00 00 00 00 28 00 00 00  // ............(...
    0030: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
    0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
    0050: 00 00 00 00 28 00 00 00 01 00 00 00 01 00 00 00  // ....(...........
    0060: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
    0070: 00 00 00 00 00 00 00 00 00 00 00 00 28 00 00 00  // ............(...
    0080: 00 00 00 00 80 00 00 00 02 00 00 00 00 00 00 00  // ................
    0090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
    00A0: 01 00 00 00 40 00 00 00 00 00 00 00 02 00 00 00  // ....@...........
    00B0: 03 00 00 00 00 00 00 00 10 27 00 00 00 00 00 00  // .........'......
    00C0: 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00  // ................
    00D0: 02 00 00 00 01 00 02 00 03 00 02 00 01 00 03 00  // ................
    00E0: 01 00 00 00 40 00 00 00 00 03 00 00 02 00 00 00  // ....@...........
    00F0: 03 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00  // ................
    0100: 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00  // ................
    0110: 02 00 00 00 0A 00 05 00 01 00 05 00 0A 00 01 00  // ................

Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>
Signed-off-by: Hesham Almatary <hesham.almatary@huawei.com>
Message-Id: <20221027100037.251-5-hesham.almatary@huawei.com>
Tested-by: Yicong Yang <yangyicong@hisilicon.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/qtest/bios-tables-test-allowed-diff.h   |   4 ----
 tests/data/acpi/q35/APIC.acpihmat-noinitiator | Bin 0 -> 144 bytes
 tests/data/acpi/q35/DSDT.acpihmat-noinitiator | Bin 0 -> 8691 bytes
 tests/data/acpi/q35/HMAT.acpihmat-noinitiator | Bin 0 -> 288 bytes
 tests/data/acpi/q35/SRAT.acpihmat-noinitiator | Bin 0 -> 312 bytes
 5 files changed, 4 deletions(-)

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index 245fa66bcc..dfb8523c8b 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1,5 +1 @@
 /* List of comma-separated changed AML files to ignore */
-"tests/data/acpi/q35/APIC.acpihmat-noinitiator",
-"tests/data/acpi/q35/DSDT.acpihmat-noinitiator",
-"tests/data/acpi/q35/HMAT.acpihmat-noinitiator",
-"tests/data/acpi/q35/SRAT.acpihmat-noinitiator",
diff --git a/tests/data/acpi/q35/APIC.acpihmat-noinitiator b/tests/data/acpi/q35/APIC.acpihmat-noinitiator
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..d904d4a70ddecbb79a83a267af8e26f925e9f4c6 100644
GIT binary patch
literal 144
zcmZ<^@N}NQz`(%h?d0$55v<@85#X!<1dKp25F11@Fg*ANra6G>KwJ(+MhMNs1fiLk
tK{O)|Nb<lx5Xr;^#2^NU#mWk#c|j~rP8f>|r~o3y%?)O;u>A)b0RWi;3;_TD

literal 0
HcmV?d00001

diff --git a/tests/data/acpi/q35/DSDT.acpihmat-noinitiator b/tests/data/acpi/q35/DSDT.acpihmat-noinitiator
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..8efa1c5ded52b8a9dfbb6945a3c75cdc6ef9b277 100644
GIT binary patch
literal 8691
zcmb7KOKcm*8J^)wtL0K!QMBb(EXGODM~Z}!okvkL4Y|vgCDIg0#VKfjq~x@c6CjHu
z2I2$;6d6bypPH}^dSn6x=&e2VS_8ebH{TkdryhFAH7Md!)bF3!p=U@6sB~D)H~WA8
zKi@v)+uf1h^4s4%$r!&~xZ+h>*~04$FN;6U7^61*y4Bb<*4gt5<t|S~V!ZQFQnaxt
zSec*sh0QYm!%pvuUhln6db}U7t*_ksTm9h9-QE_XpqpEPZBxT7&UIUbZfD=K8(uYM
zbt_fh$v?C$tLw>D)avH_CbPPmRi9<nXS!EfUhLkR%=53V4D0d1&cdSYc545+`2F&^
zSHHOWUjD%^|M{DHZ`v^gtm4<g@5O*Ny$^f#`QFg|e6Y)J6J6|E87_W45W8icOM8MI
z^?d2bE=sj>@MdTKWxtvC#28bJdL!rMRxmEF*;-h(&waAQm_rAL{{DP;c=%V@WbQ5I
zu4kSjnIy5Z*YH|xo_yfgR_7y^fQSCgEH-|&v)3wK#zFnNW5;k*33!EE@iKN4>a`|M
zK9n>9UZK8O%QmY1>)8%3tb1!L^Gw4lUXFC0B6N5d>`+8-i|;bZ(fJW|AF#}`d2psh
zQLcbCy)d?yDa<zfRxLI36N4KpScr1g3s`@6=5u-AAHMXD|FmBCp!V6)x3$xM^J)jZ
z;mrMAJ95wqsK}kWZ!v}qXIr4$9&E9}mV;w+$Zq#}|91Ztw`W*C=-=%z2I@0|8}>|)
zjz?GuWyTT&TQbUORJ{_*I9>emV26;q{~D*`>w4dctEkazH_~hG?xmBlBx<eK*NRNG
zIoc(bcBtQu0F)Jo0#sN!6~p*&8f9FFh=ss8xliyAsvt3uU&KZP6R{y8rV5QP!9=KF
z0umFW5wTIhR3nHoVye)nWFk~B0g0)q!~|1~ASRe9G$xs9@PN9`xS=yHm?|_bnQHKW
zy3T~5Ga;BNG-2ul)OF4nI%f=>Gp0^JU8iN}v<#h=sS{AwIcw;gHFVCJIstW^wxQED
zblRp)KwW3j(3v!JCQY4yx=zQ?=@>d4QzxLVGiB&Z89Gy@PC#AfoS}2h&^c%71k`oT
z8#?C=o%5znKwYP6=yVO8uBj7H*STQmTrhMlm^uM<ooPd7+R&Lcbpq--7Y&_@hR#J(
zC!nq~W9ZBnIy0tDKwakvL+1%Y=Lu6Mpsw?zq4T7n^Q5U0P}d0?lXI4sm6i;hOQudh
zUFRu7=P5(yDN`q)uJg2D=EIrrv|tv($?mjddXFJ7{V?{7!8~Iy&zMX=oq5(^o;8?f
zO(vkuJZCV^8O(Dg6HsS9VKARCm`|8YK%IGBFx4D?UNF@he_k@x#0^MH<s{2S&1IwJ
zvRM;Q*0f-tdCfq}ttv_jGv#QbaA=?aC@Vn}Koy07$~bWmaYigPK8~U?bY!4_N*WAQ
z#wr8VLdig7=*U0;F&br{GB!k16IqpG5>p*9GEhK_NW{WGWo(E-je<!ADnk|2b;3Yp
zTET>YYN2GHfXWrgKmnCb7^sXDOc<yZN(L%JMNKkLK&2A~Dq{r`2C9XUfyz+9Bm)Ih
zI$@wPRxn|pS|}N)3>8c=P(XD)VW2WrFkzrtC>f{>6-+WvK&2A~Dq{r`2C9XUfyz+9
zBm)IhI$@wPRxn|pS|}N)3>8c=P(Y;<1}b9(69%e<l7Y%l!6X9(R61dxGFC8Qpjs#y
zs0<ZMGEhLJ69y_{1rr9Ug_42FP{AYv1ynj=pfXl4VW3(l8K?{uOfpbFr4t4!V+9ii
zs)dq)%22^10|it%VW2WrFkzrtC>f{>6-+WvK&2A~Dq{r`2C9XUfyz+9Bm)IhI$@wP
zRxn|pS|}N)3>8c=P(Y;<1}b9(69%e<l7Y%l!6X9(R61dxGFC8Qpjs#ys0<ZMGEhLJ
z69y_{1rr9Ug_42FP{AYv1ynj=pfXl4VW3(l8K?{uOfpbFr4t4!V+9iis)dq)%22^1
z0|it%VW2>ofdc6U3aA?>pk|;N69%d=$v`zG8K}mDfoe<`sKz7%)tF?U8WRSpF=3z@
zlMGa2l7VVW7^udCfoe=LP>o3jsxe`pK<ezmK!I?^#K9y31;lAkkeKS&!a#x4v4w#G
zsbfn93aE}P87LsWQ=I9-LfOz6h?ms^dLjKXK|lG-Vs!)W|HIi2?Y$j#V9y@d$;_qd
zhS!{h?KX1@b9VC61e+C~5^k}3)Mv-Btp*?N*z=59NYu=A_Sa|N^Lh2n#F>}hNGz0Y
zEu6F$Kb>Q<Q8*OC9LzAAvRf=O-)*(??0|;QXl~>)3thk3Vz;Sn?}?sy`k+A(UQ^(u
zZmZei_jbZDP<iSPs~5(q^J7#mXw?g%dI96ps~0lUs~0-^HPSA>#o&8UsNIenqr9h;
z_e6Pbth{$bc`q#QkBPVOt?K_i_P$o$7v=r2^8OL!{jhv-ynO5!<%?SRq9|V+D_=aK
zd@(Fv8ZRF|M){Ifz9h<*#>$tDC|?T8m&eN|j#0j>l`o6(<+1YRBg&V<@|E%OGsh@j
z(aKju`N~-N$`R!&C?BKgXuP~D%d5!)j|RBi<;O|0FOv|rJbviD4%3a_#`c1l4$lh{
z>15y3bQAYw+DwOsh>3KvZ)&=U`)tunhbN1Pbh2-1x{3QHW2VF7#zZ>VH#ObFeQ?4|
zhi8z9bh2-1x{2%hq?ry6CKKso-_&#y*Xfd(4o@o+>13ao&aGMULw(VzG>UC?`Eg8n
z^kKfUS823s3{Tz<e@HTXx^@XKzE-UJ*+$dvW-~7e=Lx8%7f@IKgP!g7hR$b$9vk%V
zJ@$YN=RO<s+3))f`}cq22QWui8hu`0u;)I;D+ncLcv-^-^-Q<ikmoU#_Zrn)F83mH
zOAR^1ihb?*sQTXCXck*)v^M4lj5g&pHnWwt-wWQl`P=L}Z@l~V&3Crnc$am!*H~W*
zZ<>^Cm-#o&Hx74hK^>-be6#<?v7OFGKPHf8S+7y7bFW%&aj#i>jpNIEjy5IEqMldt
z@pV5<kx{QzT+brh&$qMqY+bEat$<NyXWuVg6AJ?vkQ`*FW+g|%kP9I3%q6c~`gLSz
z4|dqc9;EJ4lT&kLcbFXP*b!>4%66367k4w21XdREvz@1|Js;DL1!<(Ef$?sPCt$td
znZfuDQW@CuSQ41nv%YrF%hcke{CI`<cq(L=nm<!M>9xGNb%YGds~4}PBZmRo4*2!2
zis`|19<b{Hr$vgdYR>NlJ$`@NjzL+lJzi5N9`D478!p`+@5QOx{(3k&=-Y>=^K~$7
z|M2OFLr2#}uzi;X*=INV7}l3B4PTfD6ITyT@wOceCXNhtf_HP97SW0^xr#;!x7apa
z7TYM$S(z?wA{WMUb8Gr>uFZN^T5acu<2?OxnaGcW7wk?sHh~=(@1@Wi<DEFnQ%Bdi
z^9o(}Gy^bN`N4&49nj^CyQG$HFC=Y?;o}B9GH2$nK-f-_7`C%}1C!nBHP*Sa7NfgG
zJX0G9@Vq)oz`00}?1ndTj%&%dmaI<6R`I2Z>ZKl!h8{mh*&-2<%^790Bk*`NHanv7
z@y;EQ?;;j}#`9rv9p#H+=}m}NqqC!WzUU}l<}BtE%#Ph+Zk2@-A<ZvzAGx&?a>5bI
zWMXc;oV5RPH^m~Dy76~N8(!v*oJS6~(9az)uJ5t?ku$<6tg12gomlgf<Hk$I;UT?b
z&|icZtDzPa{Qe~Ous<4m9$=aD2f63LEBBto3Tl{5_K{**EyTGs{s?1M-uGhADHn~!
zf@~fxAZK{$*QR!OS37SfrJIPUJ)6iTIF@ER`<G*lN~0+(Vk`*`(iEGR%=N%|;P6v$
zrJr0w`PRj5IuWC53GGYWv=y@>HZB&R7ibSDaewjnG<5c#D_!;EQ$%W`Slh&vzDm(!
zBX;9%oXp70KY5KNbhk=N#&``(wsN_yTqW=9|Fo5RSpu9f&hTi{TJaecebHD|jf>Ri
z?7w=pReBt}Og2|-Zba-Jxm=FgoL=?tTDno7jYk{bVsP}0BDG7jDbuE6C9uths?Z`>
z7{LNTZv;Jp{s{U6Lnovb!P01NiC}pI%LFSUhzH_JYXu*Y<$n%DanI0^3!fZL*(b`p
z$9jz8qQ1_q<M%>DTsu$Ku56B2^LV1vSe6~XK<oCM{nhnGdqw4r;jG~C0ds~H>)NR@
ze?R(u)Uqz@iOj3smD-3CNAz^gqv)e3vo5%T^GvhtjYgA19eV5$sRI@;Ze3uUhBMzP
r6~eO{Wy>s+Xn6HDbALtmd*<i3<5w8X;G>5h`F{Xf2gTDAYef7HO|cqz

literal 0
HcmV?d00001

diff --git a/tests/data/acpi/q35/HMAT.acpihmat-noinitiator b/tests/data/acpi/q35/HMAT.acpihmat-noinitiator
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..6494d11b9fff54f8c403ec9e4893fdff72bde9c9 100644
GIT binary patch
literal 288
zcmaJ)F%Ezr5IZ0&Og@24pP{g@7)*5VIX>Ms;S4dxCU-4Odz5uKq7kt*)m-+N&Mij(
zmQa%w6GZ=3|IM0X_Ak#IabYaQ2iTvR&x~t&7@Gj;A7o|>w!;|gr;lRa*AB0!)_xEV
I&r86*0dKzu0RR91

literal 0
HcmV?d00001

diff --git a/tests/data/acpi/q35/SRAT.acpihmat-noinitiator b/tests/data/acpi/q35/SRAT.acpihmat-noinitiator
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..a11d3119ab3538d9cf821a4fe0fccb0f1dc96359 100644
GIT binary patch
literal 312
zcmWFzatyIxWME)?>E!S15v<@85#X!<1VAAM5F12;FdPVA@EK9%8JW=d%*cF34Y)~A
u1{YiayE>qSVDJGh4QBww88zTMCa6LfjpA-b4Y)81R2_^)QwNnLKmh<Kp$P#1

literal 0
HcmV?d00001

-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 66/83] tests: Add HMAT AArch64/virt empty table files
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (64 preceding siblings ...)
  2022-11-07 22:52 ` [PULL v4 65/83] tests: acpi: q35: update expected blobs *.hmat-noinitiators expected HMAT: Michael S. Tsirkin
@ 2022-11-07 22:53 ` Michael S. Tsirkin
  2022-11-07 22:53 ` [PULL v4 67/83] hw/arm/virt: Enable HMAT on arm virt machine Michael S. Tsirkin
                   ` (18 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:53 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Hesham Almatary, Yicong Yang, Igor Mammedov, Ani Sinha

From: Hesham Almatary <hesham.almatary@huawei.com>

Signed-off-by: Hesham Almatary <hesham.almatary@huawei.com>
Message-Id: <20221027100037.251-6-hesham.almatary@huawei.com>
Tested-by: Yicong Yang <yangyicong@hisilicon.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/qtest/bios-tables-test-allowed-diff.h | 5 +++++
 tests/data/acpi/virt/APIC.acpihmatvirt      | 0
 tests/data/acpi/virt/DSDT.acpihmatvirt      | 0
 tests/data/acpi/virt/HMAT.acpihmatvirt      | 0
 tests/data/acpi/virt/PPTT.acpihmatvirt      | 0
 tests/data/acpi/virt/SRAT.acpihmatvirt      | 0
 6 files changed, 5 insertions(+)
 create mode 100644 tests/data/acpi/virt/APIC.acpihmatvirt
 create mode 100644 tests/data/acpi/virt/DSDT.acpihmatvirt
 create mode 100644 tests/data/acpi/virt/HMAT.acpihmatvirt
 create mode 100644 tests/data/acpi/virt/PPTT.acpihmatvirt
 create mode 100644 tests/data/acpi/virt/SRAT.acpihmatvirt

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index dfb8523c8b..4f849715bd 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1 +1,6 @@
 /* List of comma-separated changed AML files to ignore */
+"tests/data/acpi/virt/APIC.acpihmatvirt",
+"tests/data/acpi/virt/DSDT.acpihmatvirt",
+"tests/data/acpi/virt/HMAT.acpihmatvirt",
+"tests/data/acpi/virt/PPTT.acpihmatvirt",
+"tests/data/acpi/virt/SRAT.acpihmatvirt",
diff --git a/tests/data/acpi/virt/APIC.acpihmatvirt b/tests/data/acpi/virt/APIC.acpihmatvirt
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/data/acpi/virt/DSDT.acpihmatvirt b/tests/data/acpi/virt/DSDT.acpihmatvirt
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/data/acpi/virt/HMAT.acpihmatvirt b/tests/data/acpi/virt/HMAT.acpihmatvirt
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/data/acpi/virt/PPTT.acpihmatvirt b/tests/data/acpi/virt/PPTT.acpihmatvirt
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/data/acpi/virt/SRAT.acpihmatvirt b/tests/data/acpi/virt/SRAT.acpihmatvirt
new file mode 100644
index 0000000000..e69de29bb2
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 67/83] hw/arm/virt: Enable HMAT on arm virt machine
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (65 preceding siblings ...)
  2022-11-07 22:53 ` [PULL v4 66/83] tests: Add HMAT AArch64/virt empty table files Michael S. Tsirkin
@ 2022-11-07 22:53 ` Michael S. Tsirkin
  2022-11-07 22:53 ` [PULL v4 68/83] tests: acpi: aarch64/virt: add a test for hmat nodes with no initiators Michael S. Tsirkin
                   ` (17 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:53 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Xiang Chen, Hesham Almatary, Igor Mammedov,
	Yicong Yang, Shannon Zhao, Ani Sinha, qemu-arm

From: Xiang Chen <chenxiang66@hisilicon.com>

Since the patchset ("Build ACPI Heterogeneous Memory Attribute Table (HMAT)"),
HMAT is supported, but only x86 is enabled. Enable HMAT on arm virt machine.

Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: Hesham Almatary <hesham.almatary@huawei.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20221027100037.251-7-hesham.almatary@huawei.com>
Tested-by: Yicong Yang <yangyicong@hisilicon.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/arm/virt-acpi-build.c | 7 +++++++
 hw/arm/Kconfig           | 1 +
 2 files changed, 8 insertions(+)

diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
index da9e41e72b..4156111d49 100644
--- a/hw/arm/virt-acpi-build.c
+++ b/hw/arm/virt-acpi-build.c
@@ -42,6 +42,7 @@
 #include "hw/acpi/memory_hotplug.h"
 #include "hw/acpi/generic_event_device.h"
 #include "hw/acpi/tpm.h"
+#include "hw/acpi/hmat.h"
 #include "hw/pci/pcie_host.h"
 #include "hw/pci/pci.h"
 #include "hw/pci/pci_bus.h"
@@ -987,6 +988,12 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables)
             build_slit(tables_blob, tables->linker, ms, vms->oem_id,
                        vms->oem_table_id);
         }
+
+        if (ms->numa_state->hmat_enabled) {
+            acpi_add_table(table_offsets, tables_blob);
+            build_hmat(tables_blob, tables->linker, ms->numa_state,
+                       vms->oem_id, vms->oem_table_id);
+        }
     }
 
     if (ms->nvdimms_state->is_enabled) {
diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig
index 15fa79afd3..17fcde8e1c 100644
--- a/hw/arm/Kconfig
+++ b/hw/arm/Kconfig
@@ -30,6 +30,7 @@ config ARM_VIRT
     select ACPI_VIOT
     select VIRTIO_MEM_SUPPORTED
     select ACPI_CXL
+    select ACPI_HMAT
 
 config CHEETAH
     bool
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 68/83] tests: acpi: aarch64/virt: add a test for hmat nodes with no initiators
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (66 preceding siblings ...)
  2022-11-07 22:53 ` [PULL v4 67/83] hw/arm/virt: Enable HMAT on arm virt machine Michael S. Tsirkin
@ 2022-11-07 22:53 ` Michael S. Tsirkin
  2022-11-07 22:53 ` [PULL v4 69/83] tests: virt: Update expected *.acpihmatvirt tables Michael S. Tsirkin
                   ` (16 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:53 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Hesham Almatary, Yicong Yang, Igor Mammedov, Ani Sinha

From: Hesham Almatary <hesham.almatary@huawei.com>

This patch imitates the "tests: acpi: q35: add test for hmat nodes
without initiators" commit to test numa nodes with different HMAT
attributes, but on AArch64/virt.

Tested with:
qemu-system-aarch64 -accel tcg \
-machine virt,hmat=on,gic-version=3  -cpu cortex-a57 \
-bios qemu-efi-aarch64/QEMU_EFI.fd \
-kernel Image -append "root=/dev/vda2 console=ttyAMA0" \
-drive if=virtio,file=aarch64.qcow2,format=qcow2,id=hd \
-device virtio-rng-pci \
-net user,hostfwd=tcp::10022-:22 -net nic \
-device intel-hda -device hda-duplex -nographic \
-smp 4 \
-m 3G \
-object memory-backend-ram,size=1G,id=ram0 \
-object memory-backend-ram,size=1G,id=ram1 \
-object memory-backend-ram,size=1G,id=ram2 \
-numa node,nodeid=0,memdev=ram0,cpus=0-1 \
-numa node,nodeid=1,memdev=ram1,cpus=2-3 \
-numa node,nodeid=2,memdev=ram2 \
-numa
hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-latency,latency=10 \
-numa hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-bandwidth,bandwidth=10485760 \
-numa hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=access-latency,latency=20 \
-numa hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=access-bandwidth,bandwidth=5242880 \
-numa hmat-lb,initiator=0,target=2,hierarchy=memory,data-type=access-latency,latency=30 \
-numa hmat-lb,initiator=0,target=2,hierarchy=memory,data-type=access-bandwidth,bandwidth=1048576 \
-numa hmat-lb,initiator=1,target=0,hierarchy=memory,data-type=access-latency,latency=20 \
-numa hmat-lb,initiator=1,target=0,hierarchy=memory,data-type=access-bandwidth,bandwidth=5242880 \
-numa hmat-lb,initiator=1,target=1,hierarchy=memory,data-type=access-latency,latency=10 \
-numa hmat-lb,initiator=1,target=1,hierarchy=memory,data-type=access-bandwidth,bandwidth=10485760 \
-numa hmat-lb,initiator=1,target=2,hierarchy=memory,data-type=access-latency,latency=30 \
-numa hmat-lb,initiator=1,target=2,hierarchy=memory,data-type=access-bandwidth,bandwidth=1048576

Signed-off-by: Hesham Almatary <hesham.almatary@huawei.com>
Message-Id: <20221027100037.251-8-hesham.almatary@huawei.com>
Tested-by: Yicong Yang <yangyicong@hisilicon.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/qtest/bios-tables-test.c | 59 ++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)

diff --git a/tests/qtest/bios-tables-test.c b/tests/qtest/bios-tables-test.c
index 02fe59fbf8..e805b3efec 100644
--- a/tests/qtest/bios-tables-test.c
+++ b/tests/qtest/bios-tables-test.c
@@ -1461,6 +1461,63 @@ static void test_acpi_piix4_tcg_acpi_hmat(void)
     test_acpi_tcg_acpi_hmat(MACHINE_PC);
 }
 
+static void test_acpi_virt_tcg_acpi_hmat(void)
+{
+    test_data data = {
+        .machine = "virt",
+        .tcg_only = true,
+        .uefi_fl1 = "pc-bios/edk2-aarch64-code.fd",
+        .uefi_fl2 = "pc-bios/edk2-arm-vars.fd",
+        .cd = "tests/data/uefi-boot-images/bios-tables-test.aarch64.iso.qcow2",
+        .ram_start = 0x40000000ULL,
+        .scan_len = 128ULL * 1024 * 1024,
+    };
+
+    data.variant = ".acpihmatvirt";
+
+    test_acpi_one(" -machine hmat=on"
+                  " -cpu cortex-a57"
+                  " -smp 4,sockets=2"
+                  " -m 256M"
+                  " -object memory-backend-ram,size=64M,id=ram0"
+                  " -object memory-backend-ram,size=64M,id=ram1"
+                  " -object memory-backend-ram,size=128M,id=ram2"
+                  " -numa node,nodeid=0,memdev=ram0"
+                  " -numa node,nodeid=1,memdev=ram1"
+                  " -numa node,nodeid=2,memdev=ram2"
+                  " -numa cpu,node-id=0,socket-id=0"
+                  " -numa cpu,node-id=0,socket-id=0"
+                  " -numa cpu,node-id=1,socket-id=1"
+                  " -numa cpu,node-id=1,socket-id=1"
+                  " -numa hmat-lb,initiator=0,target=0,hierarchy=memory,"
+                  "data-type=access-latency,latency=10"
+                  " -numa hmat-lb,initiator=0,target=0,hierarchy=memory,"
+                  "data-type=access-bandwidth,bandwidth=10485760"
+                  " -numa hmat-lb,initiator=0,target=1,hierarchy=memory,"
+                  "data-type=access-latency,latency=20"
+                  " -numa hmat-lb,initiator=0,target=1,hierarchy=memory,"
+                  "data-type=access-bandwidth,bandwidth=5242880"
+                  " -numa hmat-lb,initiator=0,target=2,hierarchy=memory,"
+                  "data-type=access-latency,latency=30"
+                  " -numa hmat-lb,initiator=0,target=2,hierarchy=memory,"
+                  "data-type=access-bandwidth,bandwidth=1048576"
+                  " -numa hmat-lb,initiator=1,target=0,hierarchy=memory,"
+                  "data-type=access-latency,latency=20"
+                  " -numa hmat-lb,initiator=1,target=0,hierarchy=memory,"
+                  "data-type=access-bandwidth,bandwidth=5242880"
+                  " -numa hmat-lb,initiator=1,target=1,hierarchy=memory,"
+                  "data-type=access-latency,latency=10"
+                  " -numa hmat-lb,initiator=1,target=1,hierarchy=memory,"
+                  "data-type=access-bandwidth,bandwidth=10485760"
+                  " -numa hmat-lb,initiator=1,target=2,hierarchy=memory,"
+                  "data-type=access-latency,latency=30"
+                  " -numa hmat-lb,initiator=1,target=2,hierarchy=memory,"
+                  "data-type=access-bandwidth,bandwidth=1048576",
+                  &data);
+
+    free_test_data(&data);
+}
+
 static void test_acpi_q35_tcg_acpi_hmat_noinitiator(void)
 {
     test_data data;
@@ -1914,6 +1971,8 @@ int main(int argc, char *argv[])
     } else if (strcmp(arch, "aarch64") == 0) {
         if (has_tcg) {
             qtest_add_func("acpi/virt", test_acpi_virt_tcg);
+            qtest_add_func("acpi/virt/acpihmatvirt",
+                            test_acpi_virt_tcg_acpi_hmat);
             qtest_add_func("acpi/virt/numamem", test_acpi_virt_tcg_numamem);
             qtest_add_func("acpi/virt/memhp", test_acpi_virt_tcg_memhp);
             qtest_add_func("acpi/virt/pxb", test_acpi_virt_tcg_pxb);
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 69/83] tests: virt: Update expected *.acpihmatvirt tables
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (67 preceding siblings ...)
  2022-11-07 22:53 ` [PULL v4 68/83] tests: acpi: aarch64/virt: add a test for hmat nodes with no initiators Michael S. Tsirkin
@ 2022-11-07 22:53 ` Michael S. Tsirkin
  2022-11-07 22:53 ` [PULL v4 70/83] vfio: move implement of vfio_get_xlat_addr() to memory.c Michael S. Tsirkin
                   ` (15 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:53 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Hesham Almatary, Yicong Yang, Igor Mammedov, Ani Sinha

From: Hesham Almatary <hesham.almatary@huawei.com>

* Expected ACPI Data Table [HMAT]
[000h 0000   4]                    Signature : "HMAT"    [Heterogeneous
Memory Attributes Table]
[004h 0004   4]                 Table Length : 00000120
[008h 0008   1]                     Revision : 02
[009h 0009   1]                     Checksum : 4F
[00Ah 0010   6]                       Oem ID : "BOCHS "
[010h 0016   8]                 Oem Table ID : "BXPC    "
[018h 0024   4]                 Oem Revision : 00000001
[01Ch 0028   4]              Asl Compiler ID : "BXPC"
[020h 0032   4]        Asl Compiler Revision : 00000001

[024h 0036   4]                     Reserved : 00000000

[028h 0040   2]               Structure Type : 0000 [Memory Proximity
Domain Attributes]
[02Ah 0042   2]                     Reserved : 0000
[02Ch 0044   4]                       Length : 00000028
[030h 0048   2]        Flags (decoded below) : 0001
            Processor Proximity Domain Valid : 1
[032h 0050   2]                    Reserved1 : 0000
[034h 0052   4]   Processor Proximity Domain : 00000000
[038h 0056   4]      Memory Proximity Domain : 00000000
[03Ch 0060   4]                    Reserved2 : 00000000
[040h 0064   8]                    Reserved3 : 0000000000000000
[048h 0072   8]                    Reserved4 : 0000000000000000

[050h 0080   2]               Structure Type : 0000 [Memory Proximity
Domain Attributes]
[052h 0082   2]                     Reserved : 0000
[054h 0084   4]                       Length : 00000028
[058h 0088   2]        Flags (decoded below) : 0001
            Processor Proximity Domain Valid : 1
[05Ah 0090   2]                    Reserved1 : 0000
[05Ch 0092   4]   Processor Proximity Domain : 00000001
[060h 0096   4]      Memory Proximity Domain : 00000001
[064h 0100   4]                    Reserved2 : 00000000
[068h 0104   8]                    Reserved3 : 0000000000000000
[070h 0112   8]                    Reserved4 : 0000000000000000

[078h 0120   2]               Structure Type : 0000 [Memory Proximity
Domain Attributes]
[07Ah 0122   2]                     Reserved : 0000
[07Ch 0124   4]                       Length : 00000028
[080h 0128   2]        Flags (decoded below) : 0000
            Processor Proximity Domain Valid : 0
[082h 0130   2]                    Reserved1 : 0000
[084h 0132   4]   Processor Proximity Domain : 00000080
[088h 0136   4]      Memory Proximity Domain : 00000002
[08Ch 0140   4]                    Reserved2 : 00000000
[040h 0064   8]                    Reserved3 : 0000000000000000
[048h 0072   8]                    Reserved4 : 0000000000000000

[050h 0080   2]               Structure Type : 0000 [Memory Proximity
Domain Attributes]
[052h 0082   2]                     Reserved : 0000
[054h 0084   4]                       Length : 00000028
[058h 0088   2]        Flags (decoded below) : 0001
            Processor Proximity Domain Valid : 1
[05Ah 0090   2]                    Reserved1 : 0000
[05Ch 0092   4]   Processor Proximity Domain : 00000001
[060h 0096   4]      Memory Proximity Domain : 00000001
[064h 0100   4]                    Reserved2 : 00000000
[068h 0104   8]                    Reserved3 : 0000000000000000
[070h 0112   8]                    Reserved4 : 0000000000000000

[078h 0120   2]               Structure Type : 0000 [Memory Proximity
Domain Attributes]
[07Ah 0122   2]                     Reserved : 0000
[07Ch 0124   4]                       Length : 00000028
[080h 0128   2]        Flags (decoded below) : 0000
            Processor Proximity Domain Valid : 0
[082h 0130   2]                    Reserved1 : 0000
[084h 0132   4]   Processor Proximity Domain : 00000080
[088h 0136   4]      Memory Proximity Domain : 00000002
[08Ch 0140   4]                    Reserved2 : 00000000
[090h 0144   8]                    Reserved3 : 0000000000000000
[098h 0152   8]                    Reserved4 : 0000000000000000

[0A0h 0160   2]               Structure Type : 0001 [System Locality
Latency and Bandwidth Information]
[0A2h 0162   2]                     Reserved : 0000
[0A4h 0164   4]                       Length : 00000040
[0A8h 0168   1]        Flags (decoded below) : 00
                            Memory Hierarchy : 0
[0A9h 0169   1]                    Data Type : 00
[0AAh 0170   2]                    Reserved1 : 0000
[0ACh 0172   4] Initiator Proximity Domains # : 00000002
[0B0h 0176   4]   Target Proximity Domains # : 00000003
[0B4h 0180   4]                    Reserved2 : 00000000
[0B8h 0184   8]              Entry Base Unit : 0000000000002710
[0C0h 0192   4] Initiator Proximity Domain List : 00000000
[0C4h 0196   4] Initiator Proximity Domain List : 00000001
[0C8h 0200   4] Target Proximity Domain List : 00000000
[0CCh 0204   4] Target Proximity Domain List : 00000001
[0D0h 0208   4] Target Proximity Domain List : 00000002
[0D4h 0212   2]                        Entry : 0001
[0D6h 0214   2]                        Entry : 0002
[0D8h 0216   2]                        Entry : 0003
[0DAh 0218   2]                        Entry : 0002
[0DCh 0220   2]                        Entry : 0001
[0DEh 0222   2]                        Entry : 0003

[0E0h 0224   2]               Structure Type : 0001 [System Locality
Latency and Bandwidth Information]
[0E2h 0226   2]                     Reserved : 0000
[0E4h 0228   4]                       Length : 00000040
[0E8h 0232   1]        Flags (decoded below) : 00
                            Memory Hierarchy : 0
[0E9h 0233   1]                    Data Type : 03
[0EAh 0234   2]                    Reserved1 : 0000
[0ECh 0236   4] Initiator Proximity Domains # : 00000002
[0F0h 0240   4]   Target Proximity Domains # : 00000003
[0F4h 0244   4]                    Reserved2 : 00000000
[0F8h 0248   8]              Entry Base Unit : 0000000000000001
[100h 0256   4] Initiator Proximity Domain List : 00000000
[104h 0260   4] Initiator Proximity Domain List : 00000001
[108h 0264   4] Target Proximity Domain List : 00000000
[10Ch 0268   4] Target Proximity Domain List : 00000001
[110h 0272   4] Target Proximity Domain List : 00000002
[114h 0276   2]                        Entry : 000A
[116h 0278   2]                        Entry : 0005
[118h 0280   2]                        Entry : 0001
[11Ah 0282   2]                        Entry : 0005
[11Ch 0284   2]                        Entry : 000A
[11Eh 0286   2]                        Entry : 0001

Raw Table Data: Length 288 (0x120)

    0000: 48 4D 41 54 20 01 00 00 02 4F 42 4F 43 48 53 20  // HMAT
....OBOCHS
    0010: 42 58 50 43 20 20 20 20 01 00 00 00 42 58 50 43  // BXPC
....BXPC
    0020: 01 00 00 00 00 00 00 00 00 00 00 00 28 00 00 00  //
............(...
    0030: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  //
................
    0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  //
................
    0050: 00 00 00 00 28 00 00 00 01 00 00 00 01 00 00 00  //
....(...........
    0060: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  //
................
    0070: 00 00 00 00 00 00 00 00 00 00 00 00 28 00 00 00  //
............(...
    0080: 00 00 00 00 80 00 00 00 02 00 00 00 00 00 00 00  //
................
    0090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  //
................
    00A0: 01 00 00 00 40 00 00 00 00 00 00 00 02 00 00 00  //
....@...........
    00B0: 03 00 00 00 00 00 00 00 10 27 00 00 00 00 00 00  //
.........'......
    00C0: 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00  //
................
    00D0: 02 00 00 00 01 00 02 00 03 00 02 00 01 00 03 00  //
................
    00E0: 01 00 00 00 40 00 00 00 00 03 00 00 02 00 00 00  //
....@...........
    00F0: 03 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00  //
................
    0100: 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00  //
................
    0110: 02 00 00 00 0A 00 05 00 01 00 05 00 0A 00 01 00  //
................

Signed-off-by: Hesham Almatary <hesham.almatary@huawei.com>
Message-Id: <20221027100037.251-9-hesham.almatary@huawei.com>
Tested-by: Yicong Yang <yangyicong@hisilicon.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/qtest/bios-tables-test-allowed-diff.h |   5 -----
 tests/data/acpi/virt/APIC.acpihmatvirt      | Bin 0 -> 412 bytes
 tests/data/acpi/virt/DSDT.acpihmatvirt      | Bin 0 -> 5282 bytes
 tests/data/acpi/virt/HMAT.acpihmatvirt      | Bin 0 -> 288 bytes
 tests/data/acpi/virt/PPTT.acpihmatvirt      | Bin 0 -> 196 bytes
 tests/data/acpi/virt/SRAT.acpihmatvirt      | Bin 0 -> 240 bytes
 6 files changed, 5 deletions(-)

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index 4f849715bd..dfb8523c8b 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1,6 +1 @@
 /* List of comma-separated changed AML files to ignore */
-"tests/data/acpi/virt/APIC.acpihmatvirt",
-"tests/data/acpi/virt/DSDT.acpihmatvirt",
-"tests/data/acpi/virt/HMAT.acpihmatvirt",
-"tests/data/acpi/virt/PPTT.acpihmatvirt",
-"tests/data/acpi/virt/SRAT.acpihmatvirt",
diff --git a/tests/data/acpi/virt/APIC.acpihmatvirt b/tests/data/acpi/virt/APIC.acpihmatvirt
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..68200204c6f8f2706c9896dbbccc5ecbec130d26 100644
GIT binary patch
literal 412
zcmbVIK?;B{46AboK|SknPton7CkI~qlV9+$HnL1S&Om7znov5&HC&a5J^dDz-T5O1
zr%*}2(TQ#(lZMPFX(J~Q?hmh|tV;Vf^*Z}MYiU3pa#)Z{^LPim$itl|>ZN(`63DA{
TK>f{wm+13f&GQ#zP7>t?q5KK~

literal 0
HcmV?d00001

diff --git a/tests/data/acpi/virt/DSDT.acpihmatvirt b/tests/data/acpi/virt/DSDT.acpihmatvirt
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..aee6ba017cd730948bfa93e91551eb10a6809293 100644
GIT binary patch
literal 5282
zcmZvg%WoT16o>EFlh__VVmr?J;S@^6vl`n?la{u`9y^IkoET5iAUTpNArK{-N>oUt
zLJC<FsKkOsVjWN<{tYBn?AWnj&4zz~9p>D*Gs*9?XQYhh%)RHE`;Cv|<7xWM-JeTJ
z#SR)f-lo6Q_^|6O(Pk;7^s#=;f4^ZJ4E)BRe?05CuA3Zewwu|y*KJd<qLPEXc2k+L
ziZo{RkLM__DvdcYGguP`<Nf2C-cBdFVz0C5x|K?J#pJ;2`Gr|$>S;>pWu{bKsqaRW
zWnH4^F|BBIecxL*;161zJz8y*a{b-9lcr>^ZW%<u$r(f}H63Qw?R&jQbZfh}L#fLb
zmp#wCxVWQY;l*7<g!h9Z^krW{#MA|%5+WLU72$2;2qz{nY+zK85#HSw7;%X)3XB>u
zu9+M80>hRVNnl(<#<Iv5kr)mzmXUE?WQ<CT3^1-EV?|^nBt{MxE67+C8A*we2gWKg
zZitMO#3%ye1~4k&hvS9!#gP~#VBAE8Sf>0#5|#QSFy2LmT&Edn9n-+Lg$%h)vl3$#
z7`Krj*J(~-%md>NGUPfPlNgJ@cn=wJo#rLR5-`?~A=ha^V!Q&3Ix^%sElP~bz-S;t
zuG4XeaRnG2FsgE$mL$gOz-S^vuG0yL@g^`@$dK!FQewOfjCEwlbvh+6t^#8N8FHOY
zOAHqno5+ysbVg!Sfzd{WT&J@V;~FsTB15jzIf-!{81ExPuG4vmu?mbWWXN^8ATe$N
zqXUeZT&If?;}$Tsks;S<Sz_D)#!krC{-x`+!*z7GyKdcGrRKw<rprV%-Nm#vXy0gT
zpSLCcZm&>w=DW&MZS}EQQZD^9>F5jfKz&57N<)IiqubjZ-}>A+DyHr9aHux?wyVss
zMaLCY%;@t@jDl(u#3`t$V%E+KhnCVgG%t*F7ER2Vu^^A8Mxo9melVB1Br`XRbY?V_
zS|EKzni^zMs57Ih#DmOe1#zV_qp5L|NmGMN3Uwx}D7cCpWJW8ngUo0uwM?O9kVzqW
zjxf)WAT#nDF`3cgDMZgv=1EN{ttXmFEf#uGh@J`NnP8p??nxnfCYfiFc~VP;o)n^I
zig{8yZSG!hOH$mELiBW)r^7s{g+osY(KF3F)66r?Jt;)b4D-w|PipzllS1^&GS4jY
z%yLf((KE+9bIg-kMD(N(J;#{m81o$Co)n@d{hTp19#nCjc~VP>o)n^Ifq52~XMuZC
zh@M5}S!ABnf}$sd=sC_j$C>9i_oNU#>E|bSmY65CtmsK0dQLFU3FbM$Jt;)bN#;4p
zJgLP+PYTg<ig`{k&nfOnA$rn}OYod#p41YfCxz%a!#ro0=M49x5Itv^=PdK278*S%
zM9(?qImbNbxF?0^InO-jnJ2Z}=t&`ZE-=pp=DEN<DMZgj=DEl`sYORm3emI7Jj=|p
z%snYIp8eytt=~%X^G$y#d+z%A7jc&!OXQq%@F?Qd;47WJcV9YJ8XfpOpPr2bsIX&T
zD0)tSsLKM%oI6L|)vF2{T!zxm3y}4iz%u7JD{OQ*i=G=G>x#fK=R#H~KkyoLbw>X~
zRep_j=5}l*oJZegRgC_f(>ByQ>l^xXWvIIdw)}ATpZ-}!+wdxlSQ+X8%tlQMZ9^Kh
z)U&rBCm24`V|ojsi=96ISS9_vZdWC}-QJcet)~V%zGpu>R9<txa=Mu~`Lr<Z<olh!
zYX6q!*M8i8RR8t%pPxKCbZnYXrPn0A)*k9WbM!$?qn_xUQ`Y;_{PU9s)>kxA31(ML
zC!e20^UUeI9(<@L>+@%aKjqAMeUZx9Vd<Sg)9=`)m&|JZSo!^LfBDPK4^#hiW`6TJ
zr>T?A)L9{JS$angx;l2R<+Ezk54v>C)g1Sw`xCqeba%>Y7q><q(f5scZq`T~T-DF}
zHd@VrDRJ|#uc8}Idf1nxfsk+NJZ`;VQjc0a>PzVzO{N^X|8-i2UdwN7EtM7qt$Vwv
hhdQ`_nm>7R-_iZv)9!w+;T-jkXY>Jno;-6c^*?7DCOrTE

literal 0
HcmV?d00001

diff --git a/tests/data/acpi/virt/HMAT.acpihmatvirt b/tests/data/acpi/virt/HMAT.acpihmatvirt
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..6494d11b9fff54f8c403ec9e4893fdff72bde9c9 100644
GIT binary patch
literal 288
zcmaJ)F%Ezr5IZ0&Og@24pP{g@7)*5VIX>Ms;S4dxCU-4Odz5uKq7kt*)m-+N&Mij(
zmQa%w6GZ=3|IM0X_Ak#IabYaQ2iTvR&x~t&7@Gj;A7o|>w!;|gr;lRa*AB0!)_xEV
I&r86*0dKzu0RR91

literal 0
HcmV?d00001

diff --git a/tests/data/acpi/virt/PPTT.acpihmatvirt b/tests/data/acpi/virt/PPTT.acpihmatvirt
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..710dba5e793cf36df94087666db58af5f8d03684 100644
GIT binary patch
literal 196
zcmWFt2njjDz`(#X&&l7}BUr&HBEVSz2pEB4AU23*5Mf{d(;zks0L4H+1wuhcpcogB
gWr0HsZU&Nh$YLOEC0O)!0L7SK;!rz4V$4u60KkX}0RR91

literal 0
HcmV?d00001

diff --git a/tests/data/acpi/virt/SRAT.acpihmatvirt b/tests/data/acpi/virt/SRAT.acpihmatvirt
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..691ef56e34bc84509270db316d908f5979c209bb 100644
GIT binary patch
literal 240
zcmWFzat!&vz`($~%E{l^BUr&HBEVSz2pEB4AU22wVHjW*g0Wzt5D{c`%t9a@6A&ZR
rfLP2(B8(b94g@$rX%;9A*9GD;YQW53aKWz*WFHuKKxqyrjba`E3*rd@

literal 0
HcmV?d00001

-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 70/83] vfio: move implement of vfio_get_xlat_addr() to memory.c
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (68 preceding siblings ...)
  2022-11-07 22:53 ` [PULL v4 69/83] tests: virt: Update expected *.acpihmatvirt tables Michael S. Tsirkin
@ 2022-11-07 22:53 ` Michael S. Tsirkin
  2022-11-07 22:53 ` [PULL v4 71/83] intel-iommu: don't warn guest errors when getting rid2pasid entry Michael S. Tsirkin
                   ` (14 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:53 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Cindy Lu, Alex Williamson, Paolo Bonzini,
	Peter Xu, David Hildenbrand, Philippe Mathieu-Daudé

From: Cindy Lu <lulu@redhat.com>

- Move the implement vfio_get_xlat_addr to softmmu/memory.c, and
  change the name to memory_get_xlat_addr(). So we can use this
  function on other devices, such as vDPA device.
- Add a new function vfio_get_xlat_addr in vfio/common.c, and it will check
  whether the memory is backed by a discard manager. then device can
  have its own warning.

Signed-off-by: Cindy Lu <lulu@redhat.com>
Message-Id: <20221031031020.1405111-2-lulu@redhat.com>
Acked-by: Alex Williamson <alex.williamson@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/exec/memory.h |  4 +++
 hw/vfio/common.c      | 66 +++------------------------------------
 softmmu/memory.c      | 72 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 81 insertions(+), 61 deletions(-)

diff --git a/include/exec/memory.h b/include/exec/memory.h
index bfb1de8eea..d1e79c39dc 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -713,6 +713,10 @@ void ram_discard_manager_register_listener(RamDiscardManager *rdm,
 void ram_discard_manager_unregister_listener(RamDiscardManager *rdm,
                                              RamDiscardListener *rdl);
 
+bool memory_get_xlat_addr(IOMMUTLBEntry *iotlb, void **vaddr,
+                          ram_addr_t *ram_addr, bool *read_only,
+                          bool *mr_has_discard_manager);
+
 typedef struct CoalescedMemoryRange CoalescedMemoryRange;
 typedef struct MemoryRegionIoeventfd MemoryRegionIoeventfd;
 
diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index 6b5d8c0bf6..130e5d1dc7 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -578,45 +578,11 @@ static bool vfio_listener_skipped_section(MemoryRegionSection *section)
 static bool vfio_get_xlat_addr(IOMMUTLBEntry *iotlb, void **vaddr,
                                ram_addr_t *ram_addr, bool *read_only)
 {
-    MemoryRegion *mr;
-    hwaddr xlat;
-    hwaddr len = iotlb->addr_mask + 1;
-    bool writable = iotlb->perm & IOMMU_WO;
-
-    /*
-     * The IOMMU TLB entry we have just covers translation through
-     * this IOMMU to its immediate target.  We need to translate
-     * it the rest of the way through to memory.
-     */
-    mr = address_space_translate(&address_space_memory,
-                                 iotlb->translated_addr,
-                                 &xlat, &len, writable,
-                                 MEMTXATTRS_UNSPECIFIED);
-    if (!memory_region_is_ram(mr)) {
-        error_report("iommu map to non memory area %"HWADDR_PRIx"",
-                     xlat);
-        return false;
-    } else if (memory_region_has_ram_discard_manager(mr)) {
-        RamDiscardManager *rdm = memory_region_get_ram_discard_manager(mr);
-        MemoryRegionSection tmp = {
-            .mr = mr,
-            .offset_within_region = xlat,
-            .size = int128_make64(len),
-        };
-
-        /*
-         * Malicious VMs can map memory into the IOMMU, which is expected
-         * to remain discarded. vfio will pin all pages, populating memory.
-         * Disallow that. vmstate priorities make sure any RamDiscardManager
-         * were already restored before IOMMUs are restored.
-         */
-        if (!ram_discard_manager_is_populated(rdm, &tmp)) {
-            error_report("iommu map to discarded memory (e.g., unplugged via"
-                         " virtio-mem): %"HWADDR_PRIx"",
-                         iotlb->translated_addr);
-            return false;
-        }
+    bool ret, mr_has_discard_manager;
 
+    ret = memory_get_xlat_addr(iotlb, vaddr, ram_addr, read_only,
+                               &mr_has_discard_manager);
+    if (ret && mr_has_discard_manager) {
         /*
          * Malicious VMs might trigger discarding of IOMMU-mapped memory. The
          * pages will remain pinned inside vfio until unmapped, resulting in a
@@ -635,29 +601,7 @@ static bool vfio_get_xlat_addr(IOMMUTLBEntry *iotlb, void **vaddr,
                          " intended via an IOMMU. It's possible to mitigate "
                          " by setting/adjusting RLIMIT_MEMLOCK.");
     }
-
-    /*
-     * Translation truncates length to the IOMMU page size,
-     * check that it did not truncate too much.
-     */
-    if (len & iotlb->addr_mask) {
-        error_report("iommu has granularity incompatible with target AS");
-        return false;
-    }
-
-    if (vaddr) {
-        *vaddr = memory_region_get_ram_ptr(mr) + xlat;
-    }
-
-    if (ram_addr) {
-        *ram_addr = memory_region_get_ram_addr(mr) + xlat;
-    }
-
-    if (read_only) {
-        *read_only = !writable || mr->readonly;
-    }
-
-    return true;
+    return ret;
 }
 
 static void vfio_iommu_map_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb)
diff --git a/softmmu/memory.c b/softmmu/memory.c
index 7ba2048836..bc0be3f62c 100644
--- a/softmmu/memory.c
+++ b/softmmu/memory.c
@@ -33,6 +33,7 @@
 #include "qemu/accel.h"
 #include "hw/boards.h"
 #include "migration/vmstate.h"
+#include "exec/address-spaces.h"
 
 //#define DEBUG_UNASSIGNED
 
@@ -2121,6 +2122,77 @@ void ram_discard_manager_unregister_listener(RamDiscardManager *rdm,
     rdmc->unregister_listener(rdm, rdl);
 }
 
+/* Called with rcu_read_lock held.  */
+bool memory_get_xlat_addr(IOMMUTLBEntry *iotlb, void **vaddr,
+                          ram_addr_t *ram_addr, bool *read_only,
+                          bool *mr_has_discard_manager)
+{
+    MemoryRegion *mr;
+    hwaddr xlat;
+    hwaddr len = iotlb->addr_mask + 1;
+    bool writable = iotlb->perm & IOMMU_WO;
+
+    if (mr_has_discard_manager) {
+        *mr_has_discard_manager = false;
+    }
+    /*
+     * The IOMMU TLB entry we have just covers translation through
+     * this IOMMU to its immediate target.  We need to translate
+     * it the rest of the way through to memory.
+     */
+    mr = address_space_translate(&address_space_memory, iotlb->translated_addr,
+                                 &xlat, &len, writable, MEMTXATTRS_UNSPECIFIED);
+    if (!memory_region_is_ram(mr)) {
+        error_report("iommu map to non memory area %" HWADDR_PRIx "", xlat);
+        return false;
+    } else if (memory_region_has_ram_discard_manager(mr)) {
+        RamDiscardManager *rdm = memory_region_get_ram_discard_manager(mr);
+        MemoryRegionSection tmp = {
+            .mr = mr,
+            .offset_within_region = xlat,
+            .size = int128_make64(len),
+        };
+        if (mr_has_discard_manager) {
+            *mr_has_discard_manager = true;
+        }
+        /*
+         * Malicious VMs can map memory into the IOMMU, which is expected
+         * to remain discarded. vfio will pin all pages, populating memory.
+         * Disallow that. vmstate priorities make sure any RamDiscardManager
+         * were already restored before IOMMUs are restored.
+         */
+        if (!ram_discard_manager_is_populated(rdm, &tmp)) {
+            error_report("iommu map to discarded memory (e.g., unplugged via"
+                         " virtio-mem): %" HWADDR_PRIx "",
+                         iotlb->translated_addr);
+            return false;
+        }
+    }
+
+    /*
+     * Translation truncates length to the IOMMU page size,
+     * check that it did not truncate too much.
+     */
+    if (len & iotlb->addr_mask) {
+        error_report("iommu has granularity incompatible with target AS");
+        return false;
+    }
+
+    if (vaddr) {
+        *vaddr = memory_region_get_ram_ptr(mr) + xlat;
+    }
+
+    if (ram_addr) {
+        *ram_addr = memory_region_get_ram_addr(mr) + xlat;
+    }
+
+    if (read_only) {
+        *read_only = !writable || mr->readonly;
+    }
+
+    return true;
+}
+
 void memory_region_set_log(MemoryRegion *mr, bool log, unsigned client)
 {
     uint8_t mask = 1 << client;
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 71/83] intel-iommu: don't warn guest errors when getting rid2pasid entry
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (69 preceding siblings ...)
  2022-11-07 22:53 ` [PULL v4 70/83] vfio: move implement of vfio_get_xlat_addr() to memory.c Michael S. Tsirkin
@ 2022-11-07 22:53 ` Michael S. Tsirkin
  2022-11-07 22:53 ` [PULL v4 72/83] intel-iommu: drop VTDBus Michael S. Tsirkin
                   ` (13 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:53 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Jason Wang, Peter Xu, Yi Liu, Marcel Apfelbaum,
	Paolo Bonzini, Richard Henderson, Eduardo Habkost

From: Jason Wang <jasowang@redhat.com>

We use to warn on wrong rid2pasid entry. But this error could be
triggered by the guest and could happens during initialization. So
let's don't warn in this case.

Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20221028061436.30093-2-jasowang@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Yi Liu <yi.l.liu@intel.com>
---
 hw/i386/intel_iommu.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 6524c2ee32..271de995be 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -1554,8 +1554,10 @@ static bool vtd_dev_pt_enabled(IntelIOMMUState *s, VTDContextEntry *ce)
     if (s->root_scalable) {
         ret = vtd_ce_get_rid2pasid_entry(s, ce, &pe);
         if (ret) {
-            error_report_once("%s: vtd_ce_get_rid2pasid_entry error: %"PRId32,
-                              __func__, ret);
+            /*
+             * This error is guest triggerable. We should assumt PT
+             * not enabled for safety.
+             */
             return false;
         }
         return (VTD_PE_GET_TYPE(&pe) == VTD_SM_PASID_ENTRY_PT);
@@ -1569,14 +1571,12 @@ static bool vtd_as_pt_enabled(VTDAddressSpace *as)
 {
     IntelIOMMUState *s;
     VTDContextEntry ce;
-    int ret;
 
     assert(as);
 
     s = as->iommu_state;
-    ret = vtd_dev_to_context_entry(s, pci_bus_num(as->bus),
-                                   as->devfn, &ce);
-    if (ret) {
+    if (vtd_dev_to_context_entry(s, pci_bus_num(as->bus), as->devfn,
+                                 &ce)) {
         /*
          * Possibly failed to parse the context entry for some reason
          * (e.g., during init, or any guest configuration errors on
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 72/83] intel-iommu: drop VTDBus
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (70 preceding siblings ...)
  2022-11-07 22:53 ` [PULL v4 71/83] intel-iommu: don't warn guest errors when getting rid2pasid entry Michael S. Tsirkin
@ 2022-11-07 22:53 ` Michael S. Tsirkin
  2022-11-07 22:53 ` [PULL v4 73/83] intel-iommu: convert VTD_PE_GET_FPD_ERR() to be a function Michael S. Tsirkin
                   ` (12 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:53 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Jason Wang, Peter Xu, Yi Liu, Marcel Apfelbaum,
	Paolo Bonzini, Richard Henderson, Eduardo Habkost

From: Jason Wang <jasowang@redhat.com>

We introduce VTDBus structure as an intermediate step for searching
the address space. This works well with SID based matching/lookup. But
when we want to support SID plus PASID based address space lookup,
this intermediate steps turns out to be a burden. So the patch simply
drops the VTDBus structure and use the PCIBus and devfn as the key for
the g_hash_table(). This simplifies the codes and the future PASID
extension.

To prevent being slower for past vtd_find_as_from_bus_num() callers, a
vtd_as cache indexed by the bus number is introduced to store the last
recent search result of a vtd_as belongs to a specific bus.

Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20221028061436.30093-3-jasowang@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Yi Liu <yi.l.liu@intel.com>
---
 include/hw/i386/intel_iommu.h |  11 +-
 hw/i386/intel_iommu.c         | 234 +++++++++++++++++-----------------
 2 files changed, 118 insertions(+), 127 deletions(-)

diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
index 67653b0f9b..e49fff2a6c 100644
--- a/include/hw/i386/intel_iommu.h
+++ b/include/hw/i386/intel_iommu.h
@@ -58,7 +58,6 @@ typedef struct VTDContextEntry VTDContextEntry;
 typedef struct VTDContextCacheEntry VTDContextCacheEntry;
 typedef struct VTDAddressSpace VTDAddressSpace;
 typedef struct VTDIOTLBEntry VTDIOTLBEntry;
-typedef struct VTDBus VTDBus;
 typedef union VTD_IR_TableEntry VTD_IR_TableEntry;
 typedef union VTD_IR_MSIAddress VTD_IR_MSIAddress;
 typedef struct VTDPASIDDirEntry VTDPASIDDirEntry;
@@ -111,12 +110,6 @@ struct VTDAddressSpace {
     IOVATree *iova_tree;          /* Traces mapped IOVA ranges */
 };
 
-struct VTDBus {
-    PCIBus* bus;		/* A reference to the bus to provide translation for */
-    /* A table of VTDAddressSpace objects indexed by devfn */
-    VTDAddressSpace *dev_as[];
-};
-
 struct VTDIOTLBEntry {
     uint64_t gfn;
     uint16_t domain_id;
@@ -253,8 +246,8 @@ struct IntelIOMMUState {
     uint32_t context_cache_gen;     /* Should be in [1,MAX] */
     GHashTable *iotlb;              /* IOTLB */
 
-    GHashTable *vtd_as_by_busptr;   /* VTDBus objects indexed by PCIBus* reference */
-    VTDBus *vtd_as_by_bus_num[VTD_PCI_BUS_MAX]; /* VTDBus objects indexed by bus number */
+    GHashTable *vtd_address_spaces;             /* VTD address spaces */
+    VTDAddressSpace *vtd_as_cache[VTD_PCI_BUS_MAX]; /* VTD address space cache */
     /* list of registered notifiers */
     QLIST_HEAD(, VTDAddressSpace) vtd_as_with_notifiers;
 
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 271de995be..3d426bb326 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -61,6 +61,16 @@
     }                                                                         \
 }
 
+/*
+ * PCI bus number (or SID) is not reliable since the device is usaully
+ * initalized before guest can configure the PCI bridge
+ * (SECONDARY_BUS_NUMBER).
+ */
+struct vtd_as_key {
+    PCIBus *bus;
+    uint8_t devfn;
+};
+
 static void vtd_address_space_refresh_all(IntelIOMMUState *s);
 static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n);
 
@@ -210,6 +220,27 @@ static guint vtd_uint64_hash(gconstpointer v)
     return (guint)*(const uint64_t *)v;
 }
 
+static gboolean vtd_as_equal(gconstpointer v1, gconstpointer v2)
+{
+    const struct vtd_as_key *key1 = v1;
+    const struct vtd_as_key *key2 = v2;
+
+    return (key1->bus == key2->bus) && (key1->devfn == key2->devfn);
+}
+
+/*
+ * Note that we use pointer to PCIBus as the key, so hashing/shifting
+ * based on the pointer value is intended. Note that we deal with
+ * collisions through vtd_as_equal().
+ */
+static guint vtd_as_hash(gconstpointer v)
+{
+    const struct vtd_as_key *key = v;
+    guint value = (guint)(uintptr_t)key->bus;
+
+    return (guint)(value << 8 | key->devfn);
+}
+
 static gboolean vtd_hash_remove_by_domain(gpointer key, gpointer value,
                                           gpointer user_data)
 {
@@ -248,22 +279,14 @@ static gboolean vtd_hash_remove_by_page(gpointer key, gpointer value,
 static void vtd_reset_context_cache_locked(IntelIOMMUState *s)
 {
     VTDAddressSpace *vtd_as;
-    VTDBus *vtd_bus;
-    GHashTableIter bus_it;
-    uint32_t devfn_it;
+    GHashTableIter as_it;
 
     trace_vtd_context_cache_reset();
 
-    g_hash_table_iter_init(&bus_it, s->vtd_as_by_busptr);
+    g_hash_table_iter_init(&as_it, s->vtd_address_spaces);
 
-    while (g_hash_table_iter_next (&bus_it, NULL, (void**)&vtd_bus)) {
-        for (devfn_it = 0; devfn_it < PCI_DEVFN_MAX; ++devfn_it) {
-            vtd_as = vtd_bus->dev_as[devfn_it];
-            if (!vtd_as) {
-                continue;
-            }
-            vtd_as->context_cache_entry.context_cache_gen = 0;
-        }
+    while (g_hash_table_iter_next(&as_it, NULL, (void **)&vtd_as)) {
+        vtd_as->context_cache_entry.context_cache_gen = 0;
     }
     s->context_cache_gen = 1;
 }
@@ -993,32 +1016,6 @@ static bool vtd_slpte_nonzero_rsvd(uint64_t slpte, uint32_t level)
     return slpte & rsvd_mask;
 }
 
-/* Find the VTD address space associated with a given bus number */
-static VTDBus *vtd_find_as_from_bus_num(IntelIOMMUState *s, uint8_t bus_num)
-{
-    VTDBus *vtd_bus = s->vtd_as_by_bus_num[bus_num];
-    GHashTableIter iter;
-
-    if (vtd_bus) {
-        return vtd_bus;
-    }
-
-    /*
-     * Iterate over the registered buses to find the one which
-     * currently holds this bus number and update the bus_num
-     * lookup table.
-     */
-    g_hash_table_iter_init(&iter, s->vtd_as_by_busptr);
-    while (g_hash_table_iter_next(&iter, NULL, (void **)&vtd_bus)) {
-        if (pci_bus_num(vtd_bus->bus) == bus_num) {
-            s->vtd_as_by_bus_num[bus_num] = vtd_bus;
-            return vtd_bus;
-        }
-    }
-
-    return NULL;
-}
-
 /* Given the @iova, get relevant @slptep. @slpte_level will be the last level
  * of the translation, can be used for deciding the size of large page.
  */
@@ -1632,26 +1629,15 @@ static bool vtd_switch_address_space(VTDAddressSpace *as)
 
 static void vtd_switch_address_space_all(IntelIOMMUState *s)
 {
+    VTDAddressSpace *vtd_as;
     GHashTableIter iter;
-    VTDBus *vtd_bus;
-    int i;
 
-    g_hash_table_iter_init(&iter, s->vtd_as_by_busptr);
-    while (g_hash_table_iter_next(&iter, NULL, (void **)&vtd_bus)) {
-        for (i = 0; i < PCI_DEVFN_MAX; i++) {
-            if (!vtd_bus->dev_as[i]) {
-                continue;
-            }
-            vtd_switch_address_space(vtd_bus->dev_as[i]);
-        }
+    g_hash_table_iter_init(&iter, s->vtd_address_spaces);
+    while (g_hash_table_iter_next(&iter, NULL, (void **)&vtd_as)) {
+        vtd_switch_address_space(vtd_as);
     }
 }
 
-static inline uint16_t vtd_make_source_id(uint8_t bus_num, uint8_t devfn)
-{
-    return ((bus_num & 0xffUL) << 8) | (devfn & 0xffUL);
-}
-
 static const bool vtd_qualified_faults[] = {
     [VTD_FR_RESERVED] = false,
     [VTD_FR_ROOT_ENTRY_P] = false,
@@ -1686,18 +1672,37 @@ static inline bool vtd_is_interrupt_addr(hwaddr addr)
     return VTD_INTERRUPT_ADDR_FIRST <= addr && addr <= VTD_INTERRUPT_ADDR_LAST;
 }
 
+static gboolean vtd_find_as_by_sid(gpointer key, gpointer value,
+                                   gpointer user_data)
+{
+    struct vtd_as_key *as_key = (struct vtd_as_key *)key;
+    uint16_t target_sid = *(uint16_t *)user_data;
+    uint16_t sid = PCI_BUILD_BDF(pci_bus_num(as_key->bus), as_key->devfn);
+    return sid == target_sid;
+}
+
+static VTDAddressSpace *vtd_get_as_by_sid(IntelIOMMUState *s, uint16_t sid)
+{
+    uint8_t bus_num = PCI_BUS_NUM(sid);
+    VTDAddressSpace *vtd_as = s->vtd_as_cache[bus_num];
+
+    if (vtd_as &&
+        (sid == PCI_BUILD_BDF(pci_bus_num(vtd_as->bus), vtd_as->devfn))) {
+        return vtd_as;
+    }
+
+    vtd_as = g_hash_table_find(s->vtd_address_spaces, vtd_find_as_by_sid, &sid);
+    s->vtd_as_cache[bus_num] = vtd_as;
+
+    return vtd_as;
+}
+
 static void vtd_pt_enable_fast_path(IntelIOMMUState *s, uint16_t source_id)
 {
-    VTDBus *vtd_bus;
     VTDAddressSpace *vtd_as;
     bool success = false;
 
-    vtd_bus = vtd_find_as_from_bus_num(s, VTD_SID_TO_BUS(source_id));
-    if (!vtd_bus) {
-        goto out;
-    }
-
-    vtd_as = vtd_bus->dev_as[VTD_SID_TO_DEVFN(source_id)];
+    vtd_as = vtd_get_as_by_sid(s, source_id);
     if (!vtd_as) {
         goto out;
     }
@@ -1733,7 +1738,7 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus,
     VTDContextCacheEntry *cc_entry;
     uint64_t slpte, page_mask;
     uint32_t level;
-    uint16_t source_id = vtd_make_source_id(bus_num, devfn);
+    uint16_t source_id = PCI_BUILD_BDF(bus_num, devfn);
     int ret_fr;
     bool is_fpd_set = false;
     bool reads = true;
@@ -1905,11 +1910,10 @@ static void vtd_context_device_invalidate(IntelIOMMUState *s,
                                           uint16_t source_id,
                                           uint16_t func_mask)
 {
+    GHashTableIter as_it;
     uint16_t mask;
-    VTDBus *vtd_bus;
     VTDAddressSpace *vtd_as;
     uint8_t bus_n, devfn;
-    uint16_t devfn_it;
 
     trace_vtd_inv_desc_cc_devices(source_id, func_mask);
 
@@ -1932,32 +1936,31 @@ static void vtd_context_device_invalidate(IntelIOMMUState *s,
     mask = ~mask;
 
     bus_n = VTD_SID_TO_BUS(source_id);
-    vtd_bus = vtd_find_as_from_bus_num(s, bus_n);
-    if (vtd_bus) {
-        devfn = VTD_SID_TO_DEVFN(source_id);
-        for (devfn_it = 0; devfn_it < PCI_DEVFN_MAX; ++devfn_it) {
-            vtd_as = vtd_bus->dev_as[devfn_it];
-            if (vtd_as && ((devfn_it & mask) == (devfn & mask))) {
-                trace_vtd_inv_desc_cc_device(bus_n, VTD_PCI_SLOT(devfn_it),
-                                             VTD_PCI_FUNC(devfn_it));
-                vtd_iommu_lock(s);
-                vtd_as->context_cache_entry.context_cache_gen = 0;
-                vtd_iommu_unlock(s);
-                /*
-                 * Do switch address space when needed, in case if the
-                 * device passthrough bit is switched.
-                 */
-                vtd_switch_address_space(vtd_as);
-                /*
-                 * So a device is moving out of (or moving into) a
-                 * domain, resync the shadow page table.
-                 * This won't bring bad even if we have no such
-                 * notifier registered - the IOMMU notification
-                 * framework will skip MAP notifications if that
-                 * happened.
-                 */
-                vtd_sync_shadow_page_table(vtd_as);
-            }
+    devfn = VTD_SID_TO_DEVFN(source_id);
+
+    g_hash_table_iter_init(&as_it, s->vtd_address_spaces);
+    while (g_hash_table_iter_next(&as_it, NULL, (void **)&vtd_as)) {
+        if ((pci_bus_num(vtd_as->bus) == bus_n) &&
+            (vtd_as->devfn & mask) == (devfn & mask)) {
+            trace_vtd_inv_desc_cc_device(bus_n, VTD_PCI_SLOT(vtd_as->devfn),
+                                         VTD_PCI_FUNC(vtd_as->devfn));
+            vtd_iommu_lock(s);
+            vtd_as->context_cache_entry.context_cache_gen = 0;
+            vtd_iommu_unlock(s);
+            /*
+             * Do switch address space when needed, in case if the
+             * device passthrough bit is switched.
+             */
+            vtd_switch_address_space(vtd_as);
+            /*
+             * So a device is moving out of (or moving into) a
+             * domain, resync the shadow page table.
+             * This won't bring bad even if we have no such
+             * notifier registered - the IOMMU notification
+             * framework will skip MAP notifications if that
+             * happened.
+             */
+            vtd_sync_shadow_page_table(vtd_as);
         }
     }
 }
@@ -2473,18 +2476,13 @@ static bool vtd_process_device_iotlb_desc(IntelIOMMUState *s,
 {
     VTDAddressSpace *vtd_dev_as;
     IOMMUTLBEvent event;
-    struct VTDBus *vtd_bus;
     hwaddr addr;
     uint64_t sz;
     uint16_t sid;
-    uint8_t devfn;
     bool size;
-    uint8_t bus_num;
 
     addr = VTD_INV_DESC_DEVICE_IOTLB_ADDR(inv_desc->hi);
     sid = VTD_INV_DESC_DEVICE_IOTLB_SID(inv_desc->lo);
-    devfn = sid & 0xff;
-    bus_num = sid >> 8;
     size = VTD_INV_DESC_DEVICE_IOTLB_SIZE(inv_desc->hi);
 
     if ((inv_desc->lo & VTD_INV_DESC_DEVICE_IOTLB_RSVD_LO) ||
@@ -2495,12 +2493,11 @@ static bool vtd_process_device_iotlb_desc(IntelIOMMUState *s,
         return false;
     }
 
-    vtd_bus = vtd_find_as_from_bus_num(s, bus_num);
-    if (!vtd_bus) {
-        goto done;
-    }
-
-    vtd_dev_as = vtd_bus->dev_as[devfn];
+    /*
+     * Using sid is OK since the guest should have finished the
+     * initialization of both the bus and device.
+     */
+    vtd_dev_as = vtd_get_as_by_sid(s, sid);
     if (!vtd_dev_as) {
         goto done;
     }
@@ -3427,27 +3424,27 @@ static const MemoryRegionOps vtd_mem_ir_ops = {
 
 VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
 {
-    uintptr_t key = (uintptr_t)bus;
-    VTDBus *vtd_bus = g_hash_table_lookup(s->vtd_as_by_busptr, &key);
+    /*
+     * We can't simply use sid here since the bus number might not be
+     * initialized by the guest.
+     */
+    struct vtd_as_key key = {
+        .bus = bus,
+        .devfn = devfn,
+    };
     VTDAddressSpace *vtd_dev_as;
     char name[128];
 
-    if (!vtd_bus) {
-        uintptr_t *new_key = g_malloc(sizeof(*new_key));
-        *new_key = (uintptr_t)bus;
-        /* No corresponding free() */
-        vtd_bus = g_malloc0(sizeof(VTDBus) + sizeof(VTDAddressSpace *) * \
-                            PCI_DEVFN_MAX);
-        vtd_bus->bus = bus;
-        g_hash_table_insert(s->vtd_as_by_busptr, new_key, vtd_bus);
-    }
-
-    vtd_dev_as = vtd_bus->dev_as[devfn];
-
+    vtd_dev_as = g_hash_table_lookup(s->vtd_address_spaces, &key);
     if (!vtd_dev_as) {
+        struct vtd_as_key *new_key = g_malloc(sizeof(*new_key));
+
+        new_key->bus = bus;
+        new_key->devfn = devfn;
+
         snprintf(name, sizeof(name), "vtd-%02x.%x", PCI_SLOT(devfn),
                  PCI_FUNC(devfn));
-        vtd_bus->dev_as[devfn] = vtd_dev_as = g_new0(VTDAddressSpace, 1);
+        vtd_dev_as = g_new0(VTDAddressSpace, 1);
 
         vtd_dev_as->bus = bus;
         vtd_dev_as->devfn = (uint8_t)devfn;
@@ -3503,6 +3500,8 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
                                             &vtd_dev_as->nodmar, 0);
 
         vtd_switch_address_space(vtd_dev_as);
+
+        g_hash_table_insert(s->vtd_address_spaces, new_key, vtd_dev_as);
     }
     return vtd_dev_as;
 }
@@ -3881,7 +3880,6 @@ static void vtd_realize(DeviceState *dev, Error **errp)
 
     QLIST_INIT(&s->vtd_as_with_notifiers);
     qemu_mutex_init(&s->iommu_lock);
-    memset(s->vtd_as_by_bus_num, 0, sizeof(s->vtd_as_by_bus_num));
     memory_region_init_io(&s->csrmem, OBJECT(s), &vtd_mem_ops, s,
                           "intel_iommu", DMAR_REG_SIZE);
 
@@ -3903,8 +3901,8 @@ static void vtd_realize(DeviceState *dev, Error **errp)
     /* No corresponding destroy */
     s->iotlb = g_hash_table_new_full(vtd_uint64_hash, vtd_uint64_equal,
                                      g_free, g_free);
-    s->vtd_as_by_busptr = g_hash_table_new_full(vtd_uint64_hash, vtd_uint64_equal,
-                                              g_free, g_free);
+    s->vtd_address_spaces = g_hash_table_new_full(vtd_as_hash, vtd_as_equal,
+                                      g_free, g_free);
     vtd_init(s);
     sysbus_mmio_map(SYS_BUS_DEVICE(s), 0, Q35_HOST_BRIDGE_IOMMU_ADDR);
     pci_setup_iommu(bus, vtd_host_dma_iommu, dev);
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 73/83] intel-iommu: convert VTD_PE_GET_FPD_ERR() to be a function
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (71 preceding siblings ...)
  2022-11-07 22:53 ` [PULL v4 72/83] intel-iommu: drop VTDBus Michael S. Tsirkin
@ 2022-11-07 22:53 ` Michael S. Tsirkin
  2022-11-07 22:53 ` [PULL v4 74/83] intel-iommu: PASID support Michael S. Tsirkin
                   ` (11 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:53 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Jason Wang, Peter Xu, Yi Liu, Paolo Bonzini,
	Richard Henderson, Eduardo Habkost, Marcel Apfelbaum

From: Jason Wang <jasowang@redhat.com>

We used to have a macro for VTD_PE_GET_FPD_ERR() but it has an
internal goto which prevents it from being reused. This patch convert
that macro to a dedicated function and let the caller to decide what
to do (e.g using goto or not). This makes sure it can be re-used for
other function that requires fault reporting.

Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20221028061436.30093-4-jasowang@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Yi Liu <yi.l.liu@intel.com>
---
 hw/i386/intel_iommu.c | 42 ++++++++++++++++++++++++++++--------------
 1 file changed, 28 insertions(+), 14 deletions(-)

diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 3d426bb326..259f720c7c 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -49,17 +49,6 @@
 /* pe operations */
 #define VTD_PE_GET_TYPE(pe) ((pe)->val[0] & VTD_SM_PASID_ENTRY_PGTT)
 #define VTD_PE_GET_LEVEL(pe) (2 + (((pe)->val[0] >> 2) & VTD_SM_PASID_ENTRY_AW))
-#define VTD_PE_GET_FPD_ERR(ret_fr, is_fpd_set, s, source_id, addr, is_write) {\
-    if (ret_fr) {                                                             \
-        ret_fr = -ret_fr;                                                     \
-        if (is_fpd_set && vtd_is_qualified_fault(ret_fr)) {                   \
-            trace_vtd_fault_disabled();                                       \
-        } else {                                                              \
-            vtd_report_dmar_fault(s, source_id, addr, ret_fr, is_write);      \
-        }                                                                     \
-        goto error;                                                           \
-    }                                                                         \
-}
 
 /*
  * PCI bus number (or SID) is not reliable since the device is usaully
@@ -1716,6 +1705,19 @@ out:
     trace_vtd_pt_enable_fast_path(source_id, success);
 }
 
+static void vtd_report_fault(IntelIOMMUState *s,
+                             int err, bool is_fpd_set,
+                             uint16_t source_id,
+                             hwaddr addr,
+                             bool is_write)
+{
+    if (is_fpd_set && vtd_is_qualified_fault(err)) {
+        trace_vtd_fault_disabled();
+    } else {
+        vtd_report_dmar_fault(s, source_id, addr, err, is_write);
+    }
+}
+
 /* Map dev to context-entry then do a paging-structures walk to do a iommu
  * translation.
  *
@@ -1776,7 +1778,11 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus,
         is_fpd_set = ce.lo & VTD_CONTEXT_ENTRY_FPD;
         if (!is_fpd_set && s->root_scalable) {
             ret_fr = vtd_ce_get_pasid_fpd(s, &ce, &is_fpd_set);
-            VTD_PE_GET_FPD_ERR(ret_fr, is_fpd_set, s, source_id, addr, is_write);
+            if (ret_fr) {
+                vtd_report_fault(s, -ret_fr, is_fpd_set,
+                                 source_id, addr, is_write);
+                goto error;
+            }
         }
     } else {
         ret_fr = vtd_dev_to_context_entry(s, bus_num, devfn, &ce);
@@ -1784,7 +1790,11 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus,
         if (!ret_fr && !is_fpd_set && s->root_scalable) {
             ret_fr = vtd_ce_get_pasid_fpd(s, &ce, &is_fpd_set);
         }
-        VTD_PE_GET_FPD_ERR(ret_fr, is_fpd_set, s, source_id, addr, is_write);
+        if (ret_fr) {
+            vtd_report_fault(s, -ret_fr, is_fpd_set,
+                             source_id, addr, is_write);
+            goto error;
+        }
         /* Update context-cache */
         trace_vtd_iotlb_cc_update(bus_num, devfn, ce.hi, ce.lo,
                                   cc_entry->context_cache_gen,
@@ -1820,7 +1830,11 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus,
 
     ret_fr = vtd_iova_to_slpte(s, &ce, addr, is_write, &slpte, &level,
                                &reads, &writes, s->aw_bits);
-    VTD_PE_GET_FPD_ERR(ret_fr, is_fpd_set, s, source_id, addr, is_write);
+    if (ret_fr) {
+        vtd_report_fault(s, -ret_fr, is_fpd_set, source_id,
+                         addr, is_write);
+        goto error;
+    }
 
     page_mask = vtd_slpt_level_page_mask(level);
     access_flags = IOMMU_ACCESS_FLAG(reads, writes);
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 74/83] intel-iommu: PASID support
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (72 preceding siblings ...)
  2022-11-07 22:53 ` [PULL v4 73/83] intel-iommu: convert VTD_PE_GET_FPD_ERR() to be a function Michael S. Tsirkin
@ 2022-11-07 22:53 ` Michael S. Tsirkin
  2023-04-06 16:22   ` Peter Maydell
  2022-11-07 22:53 ` [PULL v4 75/83] vhost: Change the sequence of device start Michael S. Tsirkin
                   ` (10 subsequent siblings)
  84 siblings, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:53 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Jason Wang, Peter Xu, Marcel Apfelbaum,
	Paolo Bonzini, Richard Henderson, Eduardo Habkost

From: Jason Wang <jasowang@redhat.com>

This patch introduce ECAP_PASID via "x-pasid-mode". Based on the
existing support for scalable mode, we need to implement the following
missing parts:

1) tag VTDAddressSpace with PASID and support IOMMU/DMA translation
   with PASID
2) tag IOTLB with PASID
3) PASID cache and its flush
4) PASID based IOTLB invalidation

For simplicity PASID cache is not implemented so we can simply
implement the PASID cache flush as a no and leave it to be implemented
in the future. For PASID based IOTLB invalidation, since we haven't
had L1 stage support, the PASID based IOTLB invalidation is not
implemented yet. For PASID based device IOTLB invalidation, it
requires the support for vhost so we forbid enabling device IOTLB when
PASID is enabled now. Those work could be done in the future.

Note that though PASID based IOMMU translation is ready but no device
can issue PASID DMA right now. In this case, PCI_NO_PASID is used as
PASID to identify the address without PASID. vtd_find_add_as() has
been extended to provision address space with PASID which could be
utilized by the future extension of PCI core to allow device model to
use PASID based DMA translation.

This feature would be useful for:

1) prototyping PASID support for devices like virtio
2) future vPASID work
3) future PRS and vSVA work

Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
Message-Id: <20221028061436.30093-5-jasowang@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/i386/intel_iommu_internal.h |  16 +-
 include/hw/i386/intel_iommu.h  |   7 +-
 include/hw/pci/pci_bus.h       |   2 +
 hw/i386/intel_iommu.c          | 418 +++++++++++++++++++++++++--------
 hw/i386/trace-events           |   2 +
 5 files changed, 341 insertions(+), 104 deletions(-)

diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h
index 930ce61feb..f090e61e11 100644
--- a/hw/i386/intel_iommu_internal.h
+++ b/hw/i386/intel_iommu_internal.h
@@ -114,8 +114,9 @@
                                      VTD_INTERRUPT_ADDR_FIRST + 1)
 
 /* The shift of source_id in the key of IOTLB hash table */
-#define VTD_IOTLB_SID_SHIFT         36
-#define VTD_IOTLB_LVL_SHIFT         52
+#define VTD_IOTLB_SID_SHIFT         20
+#define VTD_IOTLB_LVL_SHIFT         28
+#define VTD_IOTLB_PASID_SHIFT       30
 #define VTD_IOTLB_MAX_SIZE          1024    /* Max size of the hash table */
 
 /* IOTLB_REG */
@@ -191,6 +192,7 @@
 #define VTD_ECAP_SC                 (1ULL << 7)
 #define VTD_ECAP_MHMV               (15ULL << 20)
 #define VTD_ECAP_SRS                (1ULL << 31)
+#define VTD_ECAP_PASID              (1ULL << 40)
 #define VTD_ECAP_SMTS               (1ULL << 43)
 #define VTD_ECAP_SLTS               (1ULL << 46)
 
@@ -211,6 +213,8 @@
 #define VTD_CAP_DRAIN_READ          (1ULL << 55)
 #define VTD_CAP_DRAIN               (VTD_CAP_DRAIN_READ | VTD_CAP_DRAIN_WRITE)
 #define VTD_CAP_CM                  (1ULL << 7)
+#define VTD_PASID_ID_SHIFT          20
+#define VTD_PASID_ID_MASK           ((1ULL << VTD_PASID_ID_SHIFT) - 1)
 
 /* Supported Adjusted Guest Address Widths */
 #define VTD_CAP_SAGAW_SHIFT         8
@@ -262,6 +266,8 @@
 #define VTD_FRCD_SID(val)       ((val) & VTD_FRCD_SID_MASK)
 /* For the low 64-bit of 128-bit */
 #define VTD_FRCD_FI(val)        ((val) & ~0xfffULL)
+#define VTD_FRCD_PV(val)        (((val) & 0xffffULL) << 40)
+#define VTD_FRCD_PP(val)        (((val) & 0x1) << 31)
 
 /* DMA Remapping Fault Conditions */
 typedef enum VTDFaultReason {
@@ -379,6 +385,11 @@ typedef union VTDInvDesc VTDInvDesc;
 #define VTD_INV_DESC_IOTLB_AM(val)      ((val) & 0x3fULL)
 #define VTD_INV_DESC_IOTLB_RSVD_LO      0xffffffff0000ff00ULL
 #define VTD_INV_DESC_IOTLB_RSVD_HI      0xf80ULL
+#define VTD_INV_DESC_IOTLB_PASID_PASID  (2ULL << 4)
+#define VTD_INV_DESC_IOTLB_PASID_PAGE   (3ULL << 4)
+#define VTD_INV_DESC_IOTLB_PASID(val)   (((val) >> 32) & VTD_PASID_ID_MASK)
+#define VTD_INV_DESC_IOTLB_PASID_RSVD_LO      0xfff00000000001c0ULL
+#define VTD_INV_DESC_IOTLB_PASID_RSVD_HI      0xf80ULL
 
 /* Mask for Device IOTLB Invalidate Descriptor */
 #define VTD_INV_DESC_DEVICE_IOTLB_ADDR(val) ((val) & 0xfffffffffffff000ULL)
@@ -413,6 +424,7 @@ typedef union VTDInvDesc VTDInvDesc;
 /* Information about page-selective IOTLB invalidate */
 struct VTDIOTLBPageInvInfo {
     uint16_t domain_id;
+    uint32_t pasid;
     uint64_t addr;
     uint8_t mask;
 };
diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
index e49fff2a6c..46d973e629 100644
--- a/include/hw/i386/intel_iommu.h
+++ b/include/hw/i386/intel_iommu.h
@@ -97,11 +97,13 @@ struct VTDPASIDEntry {
 struct VTDAddressSpace {
     PCIBus *bus;
     uint8_t devfn;
+    uint32_t pasid;
     AddressSpace as;
     IOMMUMemoryRegion iommu;
     MemoryRegion root;          /* The root container of the device */
     MemoryRegion nodmar;        /* The alias of shared nodmar MR */
     MemoryRegion iommu_ir;      /* Interrupt region: 0xfeeXXXXX */
+    MemoryRegion iommu_ir_fault; /* Interrupt region for catching fault */
     IntelIOMMUState *iommu_state;
     VTDContextCacheEntry context_cache_entry;
     QLIST_ENTRY(VTDAddressSpace) next;
@@ -113,6 +115,7 @@ struct VTDAddressSpace {
 struct VTDIOTLBEntry {
     uint64_t gfn;
     uint16_t domain_id;
+    uint32_t pasid;
     uint64_t slpte;
     uint64_t mask;
     uint8_t access_flags;
@@ -261,6 +264,7 @@ struct IntelIOMMUState {
     uint8_t aw_bits;                /* Host/IOVA address width (in bits) */
     bool dma_drain;                 /* Whether DMA r/w draining enabled */
     bool dma_translation;           /* Whether DMA translation supported */
+    bool pasid;                     /* Whether to support PASID */
 
     /*
      * Protects IOMMU states in general.  Currently it protects the
@@ -272,6 +276,7 @@ struct IntelIOMMUState {
 /* Find the VTD Address space associated with the given bus pointer,
  * create a new one if none exists
  */
-VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn);
+VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus,
+                                 int devfn, unsigned int pasid);
 
 #endif
diff --git a/include/hw/pci/pci_bus.h b/include/hw/pci/pci_bus.h
index eb94e7e85c..5653175957 100644
--- a/include/hw/pci/pci_bus.h
+++ b/include/hw/pci/pci_bus.h
@@ -28,6 +28,8 @@ enum PCIBusFlags {
     PCI_BUS_CXL                                             = 0x0004,
 };
 
+#define PCI_NO_PASID UINT32_MAX
+
 struct PCIBus {
     BusState qbus;
     enum PCIBusFlags flags;
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 259f720c7c..a08ee85edf 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -58,6 +58,14 @@
 struct vtd_as_key {
     PCIBus *bus;
     uint8_t devfn;
+    uint32_t pasid;
+};
+
+struct vtd_iotlb_key {
+    uint64_t gfn;
+    uint32_t pasid;
+    uint32_t level;
+    uint16_t sid;
 };
 
 static void vtd_address_space_refresh_all(IntelIOMMUState *s);
@@ -199,14 +207,24 @@ static inline gboolean vtd_as_has_map_notifier(VTDAddressSpace *as)
 }
 
 /* GHashTable functions */
-static gboolean vtd_uint64_equal(gconstpointer v1, gconstpointer v2)
+static gboolean vtd_iotlb_equal(gconstpointer v1, gconstpointer v2)
 {
-    return *((const uint64_t *)v1) == *((const uint64_t *)v2);
+    const struct vtd_iotlb_key *key1 = v1;
+    const struct vtd_iotlb_key *key2 = v2;
+
+    return key1->sid == key2->sid &&
+           key1->pasid == key2->pasid &&
+           key1->level == key2->level &&
+           key1->gfn == key2->gfn;
 }
 
-static guint vtd_uint64_hash(gconstpointer v)
+static guint vtd_iotlb_hash(gconstpointer v)
 {
-    return (guint)*(const uint64_t *)v;
+    const struct vtd_iotlb_key *key = v;
+
+    return key->gfn | ((key->sid) << VTD_IOTLB_SID_SHIFT) |
+           (key->level) << VTD_IOTLB_LVL_SHIFT |
+           (key->pasid) << VTD_IOTLB_PASID_SHIFT;
 }
 
 static gboolean vtd_as_equal(gconstpointer v1, gconstpointer v2)
@@ -214,7 +232,8 @@ static gboolean vtd_as_equal(gconstpointer v1, gconstpointer v2)
     const struct vtd_as_key *key1 = v1;
     const struct vtd_as_key *key2 = v2;
 
-    return (key1->bus == key2->bus) && (key1->devfn == key2->devfn);
+    return (key1->bus == key2->bus) && (key1->devfn == key2->devfn) &&
+           (key1->pasid == key2->pasid);
 }
 
 /*
@@ -302,13 +321,6 @@ static void vtd_reset_caches(IntelIOMMUState *s)
     vtd_iommu_unlock(s);
 }
 
-static uint64_t vtd_get_iotlb_key(uint64_t gfn, uint16_t source_id,
-                                  uint32_t level)
-{
-    return gfn | ((uint64_t)(source_id) << VTD_IOTLB_SID_SHIFT) |
-           ((uint64_t)(level) << VTD_IOTLB_LVL_SHIFT);
-}
-
 static uint64_t vtd_get_iotlb_gfn(hwaddr addr, uint32_t level)
 {
     return (addr & vtd_slpt_level_page_mask(level)) >> VTD_PAGE_SHIFT_4K;
@@ -316,15 +328,17 @@ static uint64_t vtd_get_iotlb_gfn(hwaddr addr, uint32_t level)
 
 /* Must be called with IOMMU lock held */
 static VTDIOTLBEntry *vtd_lookup_iotlb(IntelIOMMUState *s, uint16_t source_id,
-                                       hwaddr addr)
+                                       uint32_t pasid, hwaddr addr)
 {
+    struct vtd_iotlb_key key;
     VTDIOTLBEntry *entry;
-    uint64_t key;
     int level;
 
     for (level = VTD_SL_PT_LEVEL; level < VTD_SL_PML4_LEVEL; level++) {
-        key = vtd_get_iotlb_key(vtd_get_iotlb_gfn(addr, level),
-                                source_id, level);
+        key.gfn = vtd_get_iotlb_gfn(addr, level);
+        key.level = level;
+        key.sid = source_id;
+        key.pasid = pasid;
         entry = g_hash_table_lookup(s->iotlb, &key);
         if (entry) {
             goto out;
@@ -338,10 +352,11 @@ out:
 /* Must be with IOMMU lock held */
 static void vtd_update_iotlb(IntelIOMMUState *s, uint16_t source_id,
                              uint16_t domain_id, hwaddr addr, uint64_t slpte,
-                             uint8_t access_flags, uint32_t level)
+                             uint8_t access_flags, uint32_t level,
+                             uint32_t pasid)
 {
     VTDIOTLBEntry *entry = g_malloc(sizeof(*entry));
-    uint64_t *key = g_malloc(sizeof(*key));
+    struct vtd_iotlb_key *key = g_malloc(sizeof(*key));
     uint64_t gfn = vtd_get_iotlb_gfn(addr, level);
 
     trace_vtd_iotlb_page_update(source_id, addr, slpte, domain_id);
@@ -355,7 +370,13 @@ static void vtd_update_iotlb(IntelIOMMUState *s, uint16_t source_id,
     entry->slpte = slpte;
     entry->access_flags = access_flags;
     entry->mask = vtd_slpt_level_page_mask(level);
-    *key = vtd_get_iotlb_key(gfn, source_id, level);
+    entry->pasid = pasid;
+
+    key->gfn = gfn;
+    key->sid = source_id;
+    key->level = level;
+    key->pasid = pasid;
+
     g_hash_table_replace(s->iotlb, key, entry);
 }
 
@@ -448,7 +469,8 @@ static void vtd_set_frcd_and_update_ppf(IntelIOMMUState *s, uint16_t index)
 /* Must not update F field now, should be done later */
 static void vtd_record_frcd(IntelIOMMUState *s, uint16_t index,
                             uint16_t source_id, hwaddr addr,
-                            VTDFaultReason fault, bool is_write)
+                            VTDFaultReason fault, bool is_write,
+                            bool is_pasid, uint32_t pasid)
 {
     uint64_t hi = 0, lo;
     hwaddr frcd_reg_addr = DMAR_FRCD_REG_OFFSET + (((uint64_t)index) << 4);
@@ -456,7 +478,8 @@ static void vtd_record_frcd(IntelIOMMUState *s, uint16_t index,
     assert(index < DMAR_FRCD_REG_NR);
 
     lo = VTD_FRCD_FI(addr);
-    hi = VTD_FRCD_SID(source_id) | VTD_FRCD_FR(fault);
+    hi = VTD_FRCD_SID(source_id) | VTD_FRCD_FR(fault) |
+         VTD_FRCD_PV(pasid) | VTD_FRCD_PP(is_pasid);
     if (!is_write) {
         hi |= VTD_FRCD_T;
     }
@@ -487,7 +510,8 @@ static bool vtd_try_collapse_fault(IntelIOMMUState *s, uint16_t source_id)
 /* Log and report an DMAR (address translation) fault to software */
 static void vtd_report_dmar_fault(IntelIOMMUState *s, uint16_t source_id,
                                   hwaddr addr, VTDFaultReason fault,
-                                  bool is_write)
+                                  bool is_write, bool is_pasid,
+                                  uint32_t pasid)
 {
     uint32_t fsts_reg = vtd_get_long_raw(s, DMAR_FSTS_REG);
 
@@ -514,7 +538,8 @@ static void vtd_report_dmar_fault(IntelIOMMUState *s, uint16_t source_id,
         return;
     }
 
-    vtd_record_frcd(s, s->next_frcd_reg, source_id, addr, fault, is_write);
+    vtd_record_frcd(s, s->next_frcd_reg, source_id, addr, fault,
+                    is_write, is_pasid, pasid);
 
     if (fsts_reg & VTD_FSTS_PPF) {
         error_report_once("There are pending faults already, "
@@ -819,13 +844,15 @@ static int vtd_get_pe_from_pasid_table(IntelIOMMUState *s,
 
 static int vtd_ce_get_rid2pasid_entry(IntelIOMMUState *s,
                                       VTDContextEntry *ce,
-                                      VTDPASIDEntry *pe)
+                                      VTDPASIDEntry *pe,
+                                      uint32_t pasid)
 {
-    uint32_t pasid;
     dma_addr_t pasid_dir_base;
     int ret = 0;
 
-    pasid = VTD_CE_GET_RID2PASID(ce);
+    if (pasid == PCI_NO_PASID) {
+        pasid = VTD_CE_GET_RID2PASID(ce);
+    }
     pasid_dir_base = VTD_CE_GET_PASID_DIR_TABLE(ce);
     ret = vtd_get_pe_from_pasid_table(s, pasid_dir_base, pasid, pe);
 
@@ -834,15 +861,17 @@ static int vtd_ce_get_rid2pasid_entry(IntelIOMMUState *s,
 
 static int vtd_ce_get_pasid_fpd(IntelIOMMUState *s,
                                 VTDContextEntry *ce,
-                                bool *pe_fpd_set)
+                                bool *pe_fpd_set,
+                                uint32_t pasid)
 {
     int ret;
-    uint32_t pasid;
     dma_addr_t pasid_dir_base;
     VTDPASIDDirEntry pdire;
     VTDPASIDEntry pe;
 
-    pasid = VTD_CE_GET_RID2PASID(ce);
+    if (pasid == PCI_NO_PASID) {
+        pasid = VTD_CE_GET_RID2PASID(ce);
+    }
     pasid_dir_base = VTD_CE_GET_PASID_DIR_TABLE(ce);
 
     /*
@@ -888,12 +917,13 @@ static inline uint32_t vtd_ce_get_level(VTDContextEntry *ce)
 }
 
 static uint32_t vtd_get_iova_level(IntelIOMMUState *s,
-                                   VTDContextEntry *ce)
+                                   VTDContextEntry *ce,
+                                   uint32_t pasid)
 {
     VTDPASIDEntry pe;
 
     if (s->root_scalable) {
-        vtd_ce_get_rid2pasid_entry(s, ce, &pe);
+        vtd_ce_get_rid2pasid_entry(s, ce, &pe, pasid);
         return VTD_PE_GET_LEVEL(&pe);
     }
 
@@ -906,12 +936,13 @@ static inline uint32_t vtd_ce_get_agaw(VTDContextEntry *ce)
 }
 
 static uint32_t vtd_get_iova_agaw(IntelIOMMUState *s,
-                                  VTDContextEntry *ce)
+                                  VTDContextEntry *ce,
+                                  uint32_t pasid)
 {
     VTDPASIDEntry pe;
 
     if (s->root_scalable) {
-        vtd_ce_get_rid2pasid_entry(s, ce, &pe);
+        vtd_ce_get_rid2pasid_entry(s, ce, &pe, pasid);
         return 30 + ((pe.val[0] >> 2) & VTD_SM_PASID_ENTRY_AW) * 9;
     }
 
@@ -953,31 +984,33 @@ static inline bool vtd_ce_type_check(X86IOMMUState *x86_iommu,
 }
 
 static inline uint64_t vtd_iova_limit(IntelIOMMUState *s,
-                                      VTDContextEntry *ce, uint8_t aw)
+                                      VTDContextEntry *ce, uint8_t aw,
+                                      uint32_t pasid)
 {
-    uint32_t ce_agaw = vtd_get_iova_agaw(s, ce);
+    uint32_t ce_agaw = vtd_get_iova_agaw(s, ce, pasid);
     return 1ULL << MIN(ce_agaw, aw);
 }
 
 /* Return true if IOVA passes range check, otherwise false. */
 static inline bool vtd_iova_range_check(IntelIOMMUState *s,
                                         uint64_t iova, VTDContextEntry *ce,
-                                        uint8_t aw)
+                                        uint8_t aw, uint32_t pasid)
 {
     /*
      * Check if @iova is above 2^X-1, where X is the minimum of MGAW
      * in CAP_REG and AW in context-entry.
      */
-    return !(iova & ~(vtd_iova_limit(s, ce, aw) - 1));
+    return !(iova & ~(vtd_iova_limit(s, ce, aw, pasid) - 1));
 }
 
 static dma_addr_t vtd_get_iova_pgtbl_base(IntelIOMMUState *s,
-                                          VTDContextEntry *ce)
+                                          VTDContextEntry *ce,
+                                          uint32_t pasid)
 {
     VTDPASIDEntry pe;
 
     if (s->root_scalable) {
-        vtd_ce_get_rid2pasid_entry(s, ce, &pe);
+        vtd_ce_get_rid2pasid_entry(s, ce, &pe, pasid);
         return pe.val[0] & VTD_SM_PASID_ENTRY_SLPTPTR;
     }
 
@@ -1011,18 +1044,19 @@ static bool vtd_slpte_nonzero_rsvd(uint64_t slpte, uint32_t level)
 static int vtd_iova_to_slpte(IntelIOMMUState *s, VTDContextEntry *ce,
                              uint64_t iova, bool is_write,
                              uint64_t *slptep, uint32_t *slpte_level,
-                             bool *reads, bool *writes, uint8_t aw_bits)
+                             bool *reads, bool *writes, uint8_t aw_bits,
+                             uint32_t pasid)
 {
-    dma_addr_t addr = vtd_get_iova_pgtbl_base(s, ce);
-    uint32_t level = vtd_get_iova_level(s, ce);
+    dma_addr_t addr = vtd_get_iova_pgtbl_base(s, ce, pasid);
+    uint32_t level = vtd_get_iova_level(s, ce, pasid);
     uint32_t offset;
     uint64_t slpte;
     uint64_t access_right_check;
     uint64_t xlat, size;
 
-    if (!vtd_iova_range_check(s, iova, ce, aw_bits)) {
-        error_report_once("%s: detected IOVA overflow (iova=0x%" PRIx64 ")",
-                          __func__, iova);
+    if (!vtd_iova_range_check(s, iova, ce, aw_bits, pasid)) {
+        error_report_once("%s: detected IOVA overflow (iova=0x%" PRIx64 ","
+                          "pasid=0x%" PRIx32 ")", __func__, iova, pasid);
         return -VTD_FR_ADDR_BEYOND_MGAW;
     }
 
@@ -1035,8 +1069,9 @@ static int vtd_iova_to_slpte(IntelIOMMUState *s, VTDContextEntry *ce,
 
         if (slpte == (uint64_t)-1) {
             error_report_once("%s: detected read error on DMAR slpte "
-                              "(iova=0x%" PRIx64 ")", __func__, iova);
-            if (level == vtd_get_iova_level(s, ce)) {
+                              "(iova=0x%" PRIx64 ", pasid=0x%" PRIx32 ")",
+                              __func__, iova, pasid);
+            if (level == vtd_get_iova_level(s, ce, pasid)) {
                 /* Invalid programming of context-entry */
                 return -VTD_FR_CONTEXT_ENTRY_INV;
             } else {
@@ -1048,15 +1083,16 @@ static int vtd_iova_to_slpte(IntelIOMMUState *s, VTDContextEntry *ce,
         if (!(slpte & access_right_check)) {
             error_report_once("%s: detected slpte permission error "
                               "(iova=0x%" PRIx64 ", level=0x%" PRIx32 ", "
-                              "slpte=0x%" PRIx64 ", write=%d)", __func__,
-                              iova, level, slpte, is_write);
+                              "slpte=0x%" PRIx64 ", write=%d, pasid=0x%"
+                              PRIx32 ")", __func__, iova, level,
+                              slpte, is_write, pasid);
             return is_write ? -VTD_FR_WRITE : -VTD_FR_READ;
         }
         if (vtd_slpte_nonzero_rsvd(slpte, level)) {
             error_report_once("%s: detected splte reserve non-zero "
                               "iova=0x%" PRIx64 ", level=0x%" PRIx32
-                              "slpte=0x%" PRIx64 ")", __func__, iova,
-                              level, slpte);
+                              "slpte=0x%" PRIx64 ", pasid=0x%" PRIX32 ")",
+                              __func__, iova, level, slpte, pasid);
             return -VTD_FR_PAGING_ENTRY_RSVD;
         }
 
@@ -1084,9 +1120,10 @@ static int vtd_iova_to_slpte(IntelIOMMUState *s, VTDContextEntry *ce,
         error_report_once("%s: xlat address is in interrupt range "
                           "(iova=0x%" PRIx64 ", level=0x%" PRIx32 ", "
                           "slpte=0x%" PRIx64 ", write=%d, "
-                          "xlat=0x%" PRIx64 ", size=0x%" PRIx64 ")",
+                          "xlat=0x%" PRIx64 ", size=0x%" PRIx64 ", "
+                          "pasid=0x%" PRIx32 ")",
                           __func__, iova, level, slpte, is_write,
-                          xlat, size);
+                          xlat, size, pasid);
         return s->scalable_mode ? -VTD_FR_SM_INTERRUPT_ADDR :
                                   -VTD_FR_INTERRUPT_ADDR;
     }
@@ -1300,18 +1337,19 @@ next:
  */
 static int vtd_page_walk(IntelIOMMUState *s, VTDContextEntry *ce,
                          uint64_t start, uint64_t end,
-                         vtd_page_walk_info *info)
+                         vtd_page_walk_info *info,
+                         uint32_t pasid)
 {
-    dma_addr_t addr = vtd_get_iova_pgtbl_base(s, ce);
-    uint32_t level = vtd_get_iova_level(s, ce);
+    dma_addr_t addr = vtd_get_iova_pgtbl_base(s, ce, pasid);
+    uint32_t level = vtd_get_iova_level(s, ce, pasid);
 
-    if (!vtd_iova_range_check(s, start, ce, info->aw)) {
+    if (!vtd_iova_range_check(s, start, ce, info->aw, pasid)) {
         return -VTD_FR_ADDR_BEYOND_MGAW;
     }
 
-    if (!vtd_iova_range_check(s, end, ce, info->aw)) {
+    if (!vtd_iova_range_check(s, end, ce, info->aw, pasid)) {
         /* Fix end so that it reaches the maximum */
-        end = vtd_iova_limit(s, ce, info->aw);
+        end = vtd_iova_limit(s, ce, info->aw, pasid);
     }
 
     return vtd_page_walk_level(addr, start, end, level, true, true, info);
@@ -1379,7 +1417,7 @@ static int vtd_ce_rid2pasid_check(IntelIOMMUState *s,
      * has valid rid2pasid setting, which includes valid
      * rid2pasid field and corresponding pasid entry setting
      */
-    return vtd_ce_get_rid2pasid_entry(s, ce, &pe);
+    return vtd_ce_get_rid2pasid_entry(s, ce, &pe, PCI_NO_PASID);
 }
 
 /* Map a device to its corresponding domain (context-entry) */
@@ -1462,12 +1500,13 @@ static int vtd_sync_shadow_page_hook(IOMMUTLBEvent *event,
 }
 
 static uint16_t vtd_get_domain_id(IntelIOMMUState *s,
-                                  VTDContextEntry *ce)
+                                  VTDContextEntry *ce,
+                                  uint32_t pasid)
 {
     VTDPASIDEntry pe;
 
     if (s->root_scalable) {
-        vtd_ce_get_rid2pasid_entry(s, ce, &pe);
+        vtd_ce_get_rid2pasid_entry(s, ce, &pe, pasid);
         return VTD_SM_PASID_ENTRY_DID(pe.val[1]);
     }
 
@@ -1485,10 +1524,10 @@ static int vtd_sync_shadow_page_table_range(VTDAddressSpace *vtd_as,
         .notify_unmap = true,
         .aw = s->aw_bits,
         .as = vtd_as,
-        .domain_id = vtd_get_domain_id(s, ce),
+        .domain_id = vtd_get_domain_id(s, ce, vtd_as->pasid),
     };
 
-    return vtd_page_walk(s, ce, addr, addr + size, &info);
+    return vtd_page_walk(s, ce, addr, addr + size, &info, vtd_as->pasid);
 }
 
 static int vtd_sync_shadow_page_table(VTDAddressSpace *vtd_as)
@@ -1532,13 +1571,14 @@ static int vtd_sync_shadow_page_table(VTDAddressSpace *vtd_as)
  * 1st-level translation or 2nd-level translation, it depends
  * on PGTT setting.
  */
-static bool vtd_dev_pt_enabled(IntelIOMMUState *s, VTDContextEntry *ce)
+static bool vtd_dev_pt_enabled(IntelIOMMUState *s, VTDContextEntry *ce,
+                               uint32_t pasid)
 {
     VTDPASIDEntry pe;
     int ret;
 
     if (s->root_scalable) {
-        ret = vtd_ce_get_rid2pasid_entry(s, ce, &pe);
+        ret = vtd_ce_get_rid2pasid_entry(s, ce, &pe, pasid);
         if (ret) {
             /*
              * This error is guest triggerable. We should assumt PT
@@ -1572,19 +1612,20 @@ static bool vtd_as_pt_enabled(VTDAddressSpace *as)
         return false;
     }
 
-    return vtd_dev_pt_enabled(s, &ce);
+    return vtd_dev_pt_enabled(s, &ce, as->pasid);
 }
 
 /* Return whether the device is using IOMMU translation. */
 static bool vtd_switch_address_space(VTDAddressSpace *as)
 {
-    bool use_iommu;
+    bool use_iommu, pt;
     /* Whether we need to take the BQL on our own */
     bool take_bql = !qemu_mutex_iothread_locked();
 
     assert(as);
 
     use_iommu = as->iommu_state->dmar_enabled && !vtd_as_pt_enabled(as);
+    pt = as->iommu_state->dmar_enabled && vtd_as_pt_enabled(as);
 
     trace_vtd_switch_address_space(pci_bus_num(as->bus),
                                    VTD_PCI_SLOT(as->devfn),
@@ -1604,11 +1645,53 @@ static bool vtd_switch_address_space(VTDAddressSpace *as)
     if (use_iommu) {
         memory_region_set_enabled(&as->nodmar, false);
         memory_region_set_enabled(MEMORY_REGION(&as->iommu), true);
+        /*
+         * vt-d spec v3.4 3.14:
+         *
+         * """
+         * Requests-with-PASID with input address in range 0xFEEx_xxxx
+         * are translated normally like any other request-with-PASID
+         * through DMA-remapping hardware.
+         * """
+         *
+         * Need to disable ir for as with PASID.
+         */
+        if (as->pasid != PCI_NO_PASID) {
+            memory_region_set_enabled(&as->iommu_ir, false);
+        } else {
+            memory_region_set_enabled(&as->iommu_ir, true);
+        }
     } else {
         memory_region_set_enabled(MEMORY_REGION(&as->iommu), false);
         memory_region_set_enabled(&as->nodmar, true);
     }
 
+    /*
+     * vtd-spec v3.4 3.14:
+     *
+     * """
+     * Requests-with-PASID with input address in range 0xFEEx_xxxx are
+     * translated normally like any other request-with-PASID through
+     * DMA-remapping hardware. However, if such a request is processed
+     * using pass-through translation, it will be blocked as described
+     * in the paragraph below.
+     *
+     * Software must not program paging-structure entries to remap any
+     * address to the interrupt address range. Untranslated requests
+     * and translation requests that result in an address in the
+     * interrupt range will be blocked with condition code LGN.4 or
+     * SGN.8.
+     * """
+     *
+     * We enable per as memory region (iommu_ir_fault) for catching
+     * the tranlsation for interrupt range through PASID + PT.
+     */
+    if (pt && as->pasid != PCI_NO_PASID) {
+        memory_region_set_enabled(&as->iommu_ir_fault, true);
+    } else {
+        memory_region_set_enabled(&as->iommu_ir_fault, false);
+    }
+
     if (take_bql) {
         qemu_mutex_unlock_iothread();
     }
@@ -1709,12 +1792,15 @@ static void vtd_report_fault(IntelIOMMUState *s,
                              int err, bool is_fpd_set,
                              uint16_t source_id,
                              hwaddr addr,
-                             bool is_write)
+                             bool is_write,
+                             bool is_pasid,
+                             uint32_t pasid)
 {
     if (is_fpd_set && vtd_is_qualified_fault(err)) {
         trace_vtd_fault_disabled();
     } else {
-        vtd_report_dmar_fault(s, source_id, addr, err, is_write);
+        vtd_report_dmar_fault(s, source_id, addr, err, is_write,
+                              is_pasid, pasid);
     }
 }
 
@@ -1739,13 +1825,14 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus,
     uint8_t bus_num = pci_bus_num(bus);
     VTDContextCacheEntry *cc_entry;
     uint64_t slpte, page_mask;
-    uint32_t level;
+    uint32_t level, pasid = vtd_as->pasid;
     uint16_t source_id = PCI_BUILD_BDF(bus_num, devfn);
     int ret_fr;
     bool is_fpd_set = false;
     bool reads = true;
     bool writes = true;
     uint8_t access_flags;
+    bool rid2pasid = (pasid == PCI_NO_PASID) && s->root_scalable;
     VTDIOTLBEntry *iotlb_entry;
 
     /*
@@ -1758,15 +1845,17 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus,
 
     cc_entry = &vtd_as->context_cache_entry;
 
-    /* Try to fetch slpte form IOTLB */
-    iotlb_entry = vtd_lookup_iotlb(s, source_id, addr);
-    if (iotlb_entry) {
-        trace_vtd_iotlb_page_hit(source_id, addr, iotlb_entry->slpte,
-                                 iotlb_entry->domain_id);
-        slpte = iotlb_entry->slpte;
-        access_flags = iotlb_entry->access_flags;
-        page_mask = iotlb_entry->mask;
-        goto out;
+    /* Try to fetch slpte form IOTLB, we don't need RID2PASID logic */
+    if (!rid2pasid) {
+        iotlb_entry = vtd_lookup_iotlb(s, source_id, pasid, addr);
+        if (iotlb_entry) {
+            trace_vtd_iotlb_page_hit(source_id, addr, iotlb_entry->slpte,
+                                     iotlb_entry->domain_id);
+            slpte = iotlb_entry->slpte;
+            access_flags = iotlb_entry->access_flags;
+            page_mask = iotlb_entry->mask;
+            goto out;
+        }
     }
 
     /* Try to fetch context-entry from cache first */
@@ -1777,10 +1866,11 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus,
         ce = cc_entry->context_entry;
         is_fpd_set = ce.lo & VTD_CONTEXT_ENTRY_FPD;
         if (!is_fpd_set && s->root_scalable) {
-            ret_fr = vtd_ce_get_pasid_fpd(s, &ce, &is_fpd_set);
+            ret_fr = vtd_ce_get_pasid_fpd(s, &ce, &is_fpd_set, pasid);
             if (ret_fr) {
                 vtd_report_fault(s, -ret_fr, is_fpd_set,
-                                 source_id, addr, is_write);
+                                 source_id, addr, is_write,
+                                 false, 0);
                 goto error;
             }
         }
@@ -1788,11 +1878,12 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus,
         ret_fr = vtd_dev_to_context_entry(s, bus_num, devfn, &ce);
         is_fpd_set = ce.lo & VTD_CONTEXT_ENTRY_FPD;
         if (!ret_fr && !is_fpd_set && s->root_scalable) {
-            ret_fr = vtd_ce_get_pasid_fpd(s, &ce, &is_fpd_set);
+            ret_fr = vtd_ce_get_pasid_fpd(s, &ce, &is_fpd_set, pasid);
         }
         if (ret_fr) {
             vtd_report_fault(s, -ret_fr, is_fpd_set,
-                             source_id, addr, is_write);
+                             source_id, addr, is_write,
+                             false, 0);
             goto error;
         }
         /* Update context-cache */
@@ -1803,11 +1894,15 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus,
         cc_entry->context_cache_gen = s->context_cache_gen;
     }
 
+    if (rid2pasid) {
+        pasid = VTD_CE_GET_RID2PASID(&ce);
+    }
+
     /*
      * We don't need to translate for pass-through context entries.
      * Also, let's ignore IOTLB caching as well for PT devices.
      */
-    if (vtd_dev_pt_enabled(s, &ce)) {
+    if (vtd_dev_pt_enabled(s, &ce, pasid)) {
         entry->iova = addr & VTD_PAGE_MASK_4K;
         entry->translated_addr = entry->iova;
         entry->addr_mask = ~VTD_PAGE_MASK_4K;
@@ -1828,18 +1923,31 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus,
         return true;
     }
 
+    /* Try to fetch slpte form IOTLB for RID2PASID slow path */
+    if (rid2pasid) {
+        iotlb_entry = vtd_lookup_iotlb(s, source_id, pasid, addr);
+        if (iotlb_entry) {
+            trace_vtd_iotlb_page_hit(source_id, addr, iotlb_entry->slpte,
+                                     iotlb_entry->domain_id);
+            slpte = iotlb_entry->slpte;
+            access_flags = iotlb_entry->access_flags;
+            page_mask = iotlb_entry->mask;
+            goto out;
+        }
+    }
+
     ret_fr = vtd_iova_to_slpte(s, &ce, addr, is_write, &slpte, &level,
-                               &reads, &writes, s->aw_bits);
+                               &reads, &writes, s->aw_bits, pasid);
     if (ret_fr) {
         vtd_report_fault(s, -ret_fr, is_fpd_set, source_id,
-                         addr, is_write);
+                         addr, is_write, pasid != PCI_NO_PASID, pasid);
         goto error;
     }
 
     page_mask = vtd_slpt_level_page_mask(level);
     access_flags = IOMMU_ACCESS_FLAG(reads, writes);
-    vtd_update_iotlb(s, source_id, vtd_get_domain_id(s, &ce), addr, slpte,
-                     access_flags, level);
+    vtd_update_iotlb(s, source_id, vtd_get_domain_id(s, &ce, pasid),
+                     addr, slpte, access_flags, level, pasid);
 out:
     vtd_iommu_unlock(s);
     entry->iova = addr & page_mask;
@@ -2031,7 +2139,7 @@ static void vtd_iotlb_domain_invalidate(IntelIOMMUState *s, uint16_t domain_id)
     QLIST_FOREACH(vtd_as, &s->vtd_as_with_notifiers, next) {
         if (!vtd_dev_to_context_entry(s, pci_bus_num(vtd_as->bus),
                                       vtd_as->devfn, &ce) &&
-            domain_id == vtd_get_domain_id(s, &ce)) {
+            domain_id == vtd_get_domain_id(s, &ce, vtd_as->pasid)) {
             vtd_sync_shadow_page_table(vtd_as);
         }
     }
@@ -2039,7 +2147,7 @@ static void vtd_iotlb_domain_invalidate(IntelIOMMUState *s, uint16_t domain_id)
 
 static void vtd_iotlb_page_invalidate_notify(IntelIOMMUState *s,
                                            uint16_t domain_id, hwaddr addr,
-                                           uint8_t am)
+                                             uint8_t am, uint32_t pasid)
 {
     VTDAddressSpace *vtd_as;
     VTDContextEntry ce;
@@ -2047,9 +2155,12 @@ static void vtd_iotlb_page_invalidate_notify(IntelIOMMUState *s,
     hwaddr size = (1 << am) * VTD_PAGE_SIZE;
 
     QLIST_FOREACH(vtd_as, &(s->vtd_as_with_notifiers), next) {
+        if (pasid != PCI_NO_PASID && pasid != vtd_as->pasid) {
+            continue;
+        }
         ret = vtd_dev_to_context_entry(s, pci_bus_num(vtd_as->bus),
                                        vtd_as->devfn, &ce);
-        if (!ret && domain_id == vtd_get_domain_id(s, &ce)) {
+        if (!ret && domain_id == vtd_get_domain_id(s, &ce, vtd_as->pasid)) {
             if (vtd_as_has_map_notifier(vtd_as)) {
                 /*
                  * As long as we have MAP notifications registered in
@@ -2093,7 +2204,7 @@ static void vtd_iotlb_page_invalidate(IntelIOMMUState *s, uint16_t domain_id,
     vtd_iommu_lock(s);
     g_hash_table_foreach_remove(s->iotlb, vtd_hash_remove_by_page, &info);
     vtd_iommu_unlock(s);
-    vtd_iotlb_page_invalidate_notify(s, domain_id, addr, am);
+    vtd_iotlb_page_invalidate_notify(s, domain_id, addr, am, PCI_NO_PASID);
 }
 
 /* Flush IOTLB
@@ -3162,6 +3273,7 @@ static Property vtd_properties[] = {
     DEFINE_PROP_BOOL("caching-mode", IntelIOMMUState, caching_mode, FALSE),
     DEFINE_PROP_BOOL("x-scalable-mode", IntelIOMMUState, scalable_mode, FALSE),
     DEFINE_PROP_BOOL("snoop-control", IntelIOMMUState, snoop_control, false),
+    DEFINE_PROP_BOOL("x-pasid-mode", IntelIOMMUState, pasid, false),
     DEFINE_PROP_BOOL("dma-drain", IntelIOMMUState, dma_drain, true),
     DEFINE_PROP_BOOL("dma-translation", IntelIOMMUState, dma_translation, true),
     DEFINE_PROP_END_OF_LIST(),
@@ -3436,7 +3548,64 @@ static const MemoryRegionOps vtd_mem_ir_ops = {
     },
 };
 
-VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
+static void vtd_report_ir_illegal_access(VTDAddressSpace *vtd_as,
+                                         hwaddr addr, bool is_write)
+{
+    IntelIOMMUState *s = vtd_as->iommu_state;
+    uint8_t bus_n = pci_bus_num(vtd_as->bus);
+    uint16_t sid = PCI_BUILD_BDF(bus_n, vtd_as->devfn);
+    bool is_fpd_set = false;
+    VTDContextEntry ce;
+
+    assert(vtd_as->pasid != PCI_NO_PASID);
+
+    /* Try out best to fetch FPD, we can't do anything more */
+    if (vtd_dev_to_context_entry(s, bus_n, vtd_as->devfn, &ce) == 0) {
+        is_fpd_set = ce.lo & VTD_CONTEXT_ENTRY_FPD;
+        if (!is_fpd_set && s->root_scalable) {
+            vtd_ce_get_pasid_fpd(s, &ce, &is_fpd_set, vtd_as->pasid);
+        }
+    }
+
+    vtd_report_fault(s, VTD_FR_SM_INTERRUPT_ADDR,
+                     is_fpd_set, sid, addr, is_write,
+                     true, vtd_as->pasid);
+}
+
+static MemTxResult vtd_mem_ir_fault_read(void *opaque, hwaddr addr,
+                                         uint64_t *data, unsigned size,
+                                         MemTxAttrs attrs)
+{
+    vtd_report_ir_illegal_access(opaque, addr, false);
+
+    return MEMTX_ERROR;
+}
+
+static MemTxResult vtd_mem_ir_fault_write(void *opaque, hwaddr addr,
+                                          uint64_t value, unsigned size,
+                                          MemTxAttrs attrs)
+{
+    vtd_report_ir_illegal_access(opaque, addr, true);
+
+    return MEMTX_ERROR;
+}
+
+static const MemoryRegionOps vtd_mem_ir_fault_ops = {
+    .read_with_attrs = vtd_mem_ir_fault_read,
+    .write_with_attrs = vtd_mem_ir_fault_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .impl = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+    },
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+    },
+};
+
+VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus,
+                                 int devfn, unsigned int pasid)
 {
     /*
      * We can't simply use sid here since the bus number might not be
@@ -3445,6 +3614,7 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
     struct vtd_as_key key = {
         .bus = bus,
         .devfn = devfn,
+        .pasid = pasid,
     };
     VTDAddressSpace *vtd_dev_as;
     char name[128];
@@ -3455,13 +3625,21 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
 
         new_key->bus = bus;
         new_key->devfn = devfn;
+        new_key->pasid = pasid;
+
+        if (pasid == PCI_NO_PASID) {
+            snprintf(name, sizeof(name), "vtd-%02x.%x", PCI_SLOT(devfn),
+                     PCI_FUNC(devfn));
+        } else {
+            snprintf(name, sizeof(name), "vtd-%02x.%x-pasid-%x", PCI_SLOT(devfn),
+                     PCI_FUNC(devfn), pasid);
+        }
 
-        snprintf(name, sizeof(name), "vtd-%02x.%x", PCI_SLOT(devfn),
-                 PCI_FUNC(devfn));
         vtd_dev_as = g_new0(VTDAddressSpace, 1);
 
         vtd_dev_as->bus = bus;
         vtd_dev_as->devfn = (uint8_t)devfn;
+        vtd_dev_as->pasid = pasid;
         vtd_dev_as->iommu_state = s;
         vtd_dev_as->context_cache_entry.context_cache_gen = 0;
         vtd_dev_as->iova_tree = iova_tree_new();
@@ -3502,6 +3680,24 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
                                             VTD_INTERRUPT_ADDR_FIRST,
                                             &vtd_dev_as->iommu_ir, 1);
 
+        /*
+         * This region is used for catching fault to access interrupt
+         * range via passthrough + PASID. See also
+         * vtd_switch_address_space(). We can't use alias since we
+         * need to know the sid which is valid for MSI who uses
+         * bus_master_as (see msi_send_message()).
+         */
+        memory_region_init_io(&vtd_dev_as->iommu_ir_fault, OBJECT(s),
+                              &vtd_mem_ir_fault_ops, vtd_dev_as, "vtd-no-ir",
+                              VTD_INTERRUPT_ADDR_SIZE);
+        /*
+         * Hook to root since when PT is enabled vtd_dev_as->iommu
+         * will be disabled.
+         */
+        memory_region_add_subregion_overlap(MEMORY_REGION(&vtd_dev_as->root),
+                                            VTD_INTERRUPT_ADDR_FIRST,
+                                            &vtd_dev_as->iommu_ir_fault, 2);
+
         /*
          * Hook both the containers under the root container, we
          * switch between DMAR & noDMAR by enable/disable
@@ -3622,7 +3818,7 @@ static void vtd_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n)
                                   "legacy mode",
                                   bus_n, PCI_SLOT(vtd_as->devfn),
                                   PCI_FUNC(vtd_as->devfn),
-                                  vtd_get_domain_id(s, &ce),
+                                  vtd_get_domain_id(s, &ce, vtd_as->pasid),
                                   ce.hi, ce.lo);
         if (vtd_as_has_map_notifier(vtd_as)) {
             /* This is required only for MAP typed notifiers */
@@ -3632,10 +3828,10 @@ static void vtd_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n)
                 .notify_unmap = false,
                 .aw = s->aw_bits,
                 .as = vtd_as,
-                .domain_id = vtd_get_domain_id(s, &ce),
+                .domain_id = vtd_get_domain_id(s, &ce, vtd_as->pasid),
             };
 
-            vtd_page_walk(s, &ce, 0, ~0ULL, &info);
+            vtd_page_walk(s, &ce, 0, ~0ULL, &info, vtd_as->pasid);
         }
     } else {
         trace_vtd_replay_ce_invalid(bus_n, PCI_SLOT(vtd_as->devfn),
@@ -3735,6 +3931,10 @@ static void vtd_init(IntelIOMMUState *s)
         s->ecap |= VTD_ECAP_SC;
     }
 
+    if (s->pasid) {
+        s->ecap |= VTD_ECAP_PASID;
+    }
+
     vtd_reset_caches(s);
 
     /* Define registers with default values and bit semantics */
@@ -3808,7 +4008,7 @@ static AddressSpace *vtd_host_dma_iommu(PCIBus *bus, void *opaque, int devfn)
 
     assert(0 <= devfn && devfn < PCI_DEVFN_MAX);
 
-    vtd_as = vtd_find_add_as(s, bus, devfn);
+    vtd_as = vtd_find_add_as(s, bus, devfn, PCI_NO_PASID);
     return &vtd_as->as;
 }
 
@@ -3851,6 +4051,11 @@ static bool vtd_decide_config(IntelIOMMUState *s, Error **errp)
         return false;
     }
 
+    if (s->pasid && !s->scalable_mode) {
+        error_setg(errp, "Need to set scalable mode for PASID");
+        return false;
+    }
+
     return true;
 }
 
@@ -3887,6 +4092,17 @@ static void vtd_realize(DeviceState *dev, Error **errp)
     X86MachineState *x86ms = X86_MACHINE(ms);
     PCIBus *bus = pcms->bus;
     IntelIOMMUState *s = INTEL_IOMMU_DEVICE(dev);
+    X86IOMMUState *x86_iommu = X86_IOMMU_DEVICE(s);
+
+    if (s->pasid && x86_iommu->dt_supported) {
+        /*
+         * PASID-based-Device-TLB Invalidate Descriptor is not
+         * implemented and it requires support from vhost layer which
+         * needs to be implemented in the future.
+         */
+        error_setg(errp, "PASID based device IOTLB is not supported");
+        return;
+    }
 
     if (!vtd_decide_config(s, errp)) {
         return;
@@ -3913,7 +4129,7 @@ static void vtd_realize(DeviceState *dev, Error **errp)
 
     sysbus_init_mmio(SYS_BUS_DEVICE(s), &s->csrmem);
     /* No corresponding destroy */
-    s->iotlb = g_hash_table_new_full(vtd_uint64_hash, vtd_uint64_equal,
+    s->iotlb = g_hash_table_new_full(vtd_iotlb_hash, vtd_iotlb_equal,
                                      g_free, g_free);
     s->vtd_address_spaces = g_hash_table_new_full(vtd_as_hash, vtd_as_equal,
                                       g_free, g_free);
diff --git a/hw/i386/trace-events b/hw/i386/trace-events
index e49814dd64..04fd71bfc4 100644
--- a/hw/i386/trace-events
+++ b/hw/i386/trace-events
@@ -12,6 +12,8 @@ vtd_inv_desc_cc_devices(uint16_t sid, uint16_t fmask) "context invalidate device
 vtd_inv_desc_iotlb_global(void) "iotlb invalidate global"
 vtd_inv_desc_iotlb_domain(uint16_t domain) "iotlb invalidate whole domain 0x%"PRIx16
 vtd_inv_desc_iotlb_pages(uint16_t domain, uint64_t addr, uint8_t mask) "iotlb invalidate domain 0x%"PRIx16" addr 0x%"PRIx64" mask 0x%"PRIx8
+vtd_inv_desc_iotlb_pasid_pages(uint16_t domain, uint64_t addr, uint8_t mask, uint32_t pasid) "iotlb invalidate domain 0x%"PRIx16" addr 0x%"PRIx64" mask 0x%"PRIx8" pasid 0x%"PRIx32
+vtd_inv_desc_iotlb_pasid(uint16_t domain, uint32_t pasid) "iotlb invalidate domain 0x%"PRIx16" pasid 0x%"PRIx32
 vtd_inv_desc_wait_sw(uint64_t addr, uint32_t data) "wait invalidate status write addr 0x%"PRIx64" data 0x%"PRIx32
 vtd_inv_desc_wait_irq(const char *msg) "%s"
 vtd_inv_desc_wait_write_fail(uint64_t hi, uint64_t lo) "write fail for wait desc hi 0x%"PRIx64" lo 0x%"PRIx64
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 75/83] vhost: Change the sequence of device start
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (73 preceding siblings ...)
  2022-11-07 22:53 ` [PULL v4 74/83] intel-iommu: PASID support Michael S. Tsirkin
@ 2022-11-07 22:53 ` Michael S. Tsirkin
  2022-11-07 22:53 ` [PULL v4 76/83] vhost-user: Support vhost_dev_start Michael S. Tsirkin
                   ` (9 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:53 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Yajun Wu, Parav Pandit, Raphael Norwitz,
	Kevin Wolf, Hanna Reitz, Jason Wang, qemu-block

From: Yajun Wu <yajunw@nvidia.com>

This patch is part of adding vhost-user vhost_dev_start support. The
motivation is to improve backend configuration speed and reduce live
migration VM downtime.

Moving the device start routines after finishing all the necessary device
and VQ configuration, further aligning to the virtio specification for
"device initialization sequence".

Following patch will add vhost-user vhost_dev_start support.

Signed-off-by: Yajun Wu <yajunw@nvidia.com>
Acked-by: Parav Pandit <parav@nvidia.com>

Message-Id: <20221017064452.1226514-2-yajunw@nvidia.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/block/vhost-user-blk.c | 18 +++++++++++-------
 hw/net/vhost_net.c        | 11 +++++------
 2 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
index 13bf5cc47a..28409c90f7 100644
--- a/hw/block/vhost-user-blk.c
+++ b/hw/block/vhost-user-blk.c
@@ -168,13 +168,6 @@ static int vhost_user_blk_start(VirtIODevice *vdev, Error **errp)
         goto err_guest_notifiers;
     }
 
-    ret = vhost_dev_start(&s->dev, vdev);
-    if (ret < 0) {
-        error_setg_errno(errp, -ret, "Error starting vhost");
-        goto err_guest_notifiers;
-    }
-    s->started_vu = true;
-
     /* guest_notifier_mask/pending not used yet, so just unmask
      * everything here. virtio-pci will do the right thing by
      * enabling/disabling irqfd.
@@ -183,9 +176,20 @@ static int vhost_user_blk_start(VirtIODevice *vdev, Error **errp)
         vhost_virtqueue_mask(&s->dev, vdev, i, false);
     }
 
+    s->dev.vq_index_end = s->dev.nvqs;
+    ret = vhost_dev_start(&s->dev, vdev);
+    if (ret < 0) {
+        error_setg_errno(errp, -ret, "Error starting vhost");
+        goto err_guest_notifiers;
+    }
+    s->started_vu = true;
+
     return ret;
 
 err_guest_notifiers:
+    for (i = 0; i < s->dev.nvqs; i++) {
+        vhost_virtqueue_mask(&s->dev, vdev, i, true);
+    }
     k->set_guest_notifiers(qbus->parent, s->dev.nvqs, false);
 err_host_notifiers:
     vhost_dev_disable_notifiers(&s->dev, vdev);
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 53b2fac4f6..feda448878 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -389,21 +389,20 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs,
         } else {
             peer = qemu_get_peer(ncs, n->max_queue_pairs);
         }
-        r = vhost_net_start_one(get_vhost_net(peer), dev);
-
-        if (r < 0) {
-            goto err_start;
-        }
 
         if (peer->vring_enable) {
             /* restore vring enable state */
             r = vhost_set_vring_enable(peer, peer->vring_enable);
 
             if (r < 0) {
-                vhost_net_stop_one(get_vhost_net(peer), dev);
                 goto err_start;
             }
         }
+
+        r = vhost_net_start_one(get_vhost_net(peer), dev);
+        if (r < 0) {
+            goto err_start;
+        }
     }
 
     return 0;
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 76/83] vhost-user: Support vhost_dev_start
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (74 preceding siblings ...)
  2022-11-07 22:53 ` [PULL v4 75/83] vhost: Change the sequence of device start Michael S. Tsirkin
@ 2022-11-07 22:53 ` Michael S. Tsirkin
  2023-01-06 14:21   ` Laurent Vivier
  2022-11-07 22:53 ` [PULL v4 77/83] hw/smbios: add core_count2 to smbios table type 4 Michael S. Tsirkin
                   ` (8 subsequent siblings)
  84 siblings, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:53 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Yajun Wu, Parav Pandit

From: Yajun Wu <yajunw@nvidia.com>

The motivation of adding vhost-user vhost_dev_start support is to
improve backend configuration speed and reduce live migration VM
downtime.

Today VQ configuration is issued one by one. For virtio net with
multi-queue support, backend needs to update RSS (Receive side
scaling) on every rx queue enable. Updating RSS is time-consuming
(typical time like 7ms).

Implement already defined vhost status and message in the vhost
specification [1].
(a) VHOST_USER_PROTOCOL_F_STATUS
(b) VHOST_USER_SET_STATUS
(c) VHOST_USER_GET_STATUS

Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK for
device start and reset(0) for device stop.

On reception of the DRIVER_OK message, backend can apply the needed setting
only once (instead of incremental) and also utilize parallelism on enabling
queues.

This improves QEMU's live migration downtime with vhost user backend
implementation by great margin, specially for the large number of VQs of 64
from 800 msec to 250 msec.

[1] https://qemu-project.gitlab.io/qemu/interop/vhost-user.html

Signed-off-by: Yajun Wu <yajunw@nvidia.com>
Acked-by: Parav Pandit <parav@nvidia.com>
Message-Id: <20221017064452.1226514-3-yajunw@nvidia.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/virtio/vhost-user.c | 74 +++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 73 insertions(+), 1 deletion(-)

diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index d256ce589b..abe23d4ebe 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -81,6 +81,7 @@ enum VhostUserProtocolFeature {
     VHOST_USER_PROTOCOL_F_RESET_DEVICE = 13,
     /* Feature 14 reserved for VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS. */
     VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS = 15,
+    VHOST_USER_PROTOCOL_F_STATUS = 16,
     VHOST_USER_PROTOCOL_F_MAX
 };
 
@@ -126,6 +127,8 @@ typedef enum VhostUserRequest {
     VHOST_USER_GET_MAX_MEM_SLOTS = 36,
     VHOST_USER_ADD_MEM_REG = 37,
     VHOST_USER_REM_MEM_REG = 38,
+    VHOST_USER_SET_STATUS = 39,
+    VHOST_USER_GET_STATUS = 40,
     VHOST_USER_MAX
 } VhostUserRequest;
 
@@ -1452,6 +1455,43 @@ static int vhost_user_set_u64(struct vhost_dev *dev, int request, uint64_t u64,
     return 0;
 }
 
+static int vhost_user_set_status(struct vhost_dev *dev, uint8_t status)
+{
+    return vhost_user_set_u64(dev, VHOST_USER_SET_STATUS, status, false);
+}
+
+static int vhost_user_get_status(struct vhost_dev *dev, uint8_t *status)
+{
+    uint64_t value;
+    int ret;
+
+    ret = vhost_user_get_u64(dev, VHOST_USER_GET_STATUS, &value);
+    if (ret < 0) {
+        return ret;
+    }
+    *status = value;
+
+    return 0;
+}
+
+static int vhost_user_add_status(struct vhost_dev *dev, uint8_t status)
+{
+    uint8_t s;
+    int ret;
+
+    ret = vhost_user_get_status(dev, &s);
+    if (ret < 0) {
+        return ret;
+    }
+
+    if ((s & status) == status) {
+        return 0;
+    }
+    s |= status;
+
+    return vhost_user_set_status(dev, s);
+}
+
 static int vhost_user_set_features(struct vhost_dev *dev,
                                    uint64_t features)
 {
@@ -1460,6 +1500,7 @@ static int vhost_user_set_features(struct vhost_dev *dev,
      * backend is actually logging changes
      */
     bool log_enabled = features & (0x1ULL << VHOST_F_LOG_ALL);
+    int ret;
 
     /*
      * We need to include any extra backend only feature bits that
@@ -1467,9 +1508,18 @@ static int vhost_user_set_features(struct vhost_dev *dev,
      * VHOST_USER_F_PROTOCOL_FEATURES bit for enabling protocol
      * features.
      */
-    return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES,
+    ret = vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES,
                               features | dev->backend_features,
                               log_enabled);
+
+    if (virtio_has_feature(dev->protocol_features,
+                           VHOST_USER_PROTOCOL_F_STATUS)) {
+        if (!ret) {
+            return vhost_user_add_status(dev, VIRTIO_CONFIG_S_FEATURES_OK);
+        }
+    }
+
+    return ret;
 }
 
 static int vhost_user_set_protocol_features(struct vhost_dev *dev,
@@ -2620,6 +2670,27 @@ void vhost_user_cleanup(VhostUserState *user)
     user->chr = NULL;
 }
 
+static int vhost_user_dev_start(struct vhost_dev *dev, bool started)
+{
+    if (!virtio_has_feature(dev->protocol_features,
+                            VHOST_USER_PROTOCOL_F_STATUS)) {
+        return 0;
+    }
+
+    /* Set device status only for last queue pair */
+    if (dev->vq_index + dev->nvqs != dev->vq_index_end) {
+        return 0;
+    }
+
+    if (started) {
+        return vhost_user_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE |
+                                          VIRTIO_CONFIG_S_DRIVER |
+                                          VIRTIO_CONFIG_S_DRIVER_OK);
+    } else {
+        return vhost_user_set_status(dev, 0);
+    }
+}
+
 const VhostOps user_ops = {
         .backend_type = VHOST_BACKEND_TYPE_USER,
         .vhost_backend_init = vhost_user_backend_init,
@@ -2654,4 +2725,5 @@ const VhostOps user_ops = {
         .vhost_backend_mem_section_filter = vhost_user_mem_section_filter,
         .vhost_get_inflight_fd = vhost_user_get_inflight_fd,
         .vhost_set_inflight_fd = vhost_user_set_inflight_fd,
+        .vhost_dev_start = vhost_user_dev_start,
 };
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 77/83] hw/smbios: add core_count2 to smbios table type 4
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (75 preceding siblings ...)
  2022-11-07 22:53 ` [PULL v4 76/83] vhost-user: Support vhost_dev_start Michael S. Tsirkin
@ 2022-11-07 22:53 ` Michael S. Tsirkin
  2022-11-07 22:54 ` [PULL v4 78/83] bios-tables-test: teach test to use smbios 3.0 tables Michael S. Tsirkin
                   ` (7 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:53 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Julia Suvorova, Igor Mammedov, Ani Sinha

From: Julia Suvorova <jusual@redhat.com>

In order to use the increased number of cpus, we need to bring smbios
tables in line with the SMBIOS 3.0 specification. This allows us to
introduce core_count2 which acts as a duplicate of core_count if we have
fewer cores than 256, and contains the actual core number per socket if
we have more.

core_enabled2 and thread_count2 fields work the same way.

Signed-off-by: Julia Suvorova <jusual@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20220731162141.178443-2-jusual@redhat.com>
Message-Id: <20221011111731.101412-2-jusual@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/smbios/smbios_build.h     |  9 +++++++--
 include/hw/firmware/smbios.h | 12 ++++++++++++
 hw/smbios/smbios.c           | 19 ++++++++++++++++---
 3 files changed, 35 insertions(+), 5 deletions(-)

diff --git a/hw/smbios/smbios_build.h b/hw/smbios/smbios_build.h
index 56b5a1e3f3..351660024e 100644
--- a/hw/smbios/smbios_build.h
+++ b/hw/smbios/smbios_build.h
@@ -27,6 +27,11 @@ extern unsigned smbios_table_max;
 extern unsigned smbios_table_cnt;
 
 #define SMBIOS_BUILD_TABLE_PRE(tbl_type, tbl_handle, tbl_required)        \
+        SMBIOS_BUILD_TABLE_PRE_SIZE(tbl_type, tbl_handle, tbl_required,   \
+                                    sizeof(struct smbios_type_##tbl_type))\
+
+#define SMBIOS_BUILD_TABLE_PRE_SIZE(tbl_type, tbl_handle,                 \
+                                    tbl_required, tbl_len)                \
     struct smbios_type_##tbl_type *t;                                     \
     size_t t_off; /* table offset into smbios_tables */                   \
     int str_index = 0;                                                    \
@@ -39,12 +44,12 @@ extern unsigned smbios_table_cnt;
         /* use offset of table t within smbios_tables */                  \
         /* (pointer must be updated after each realloc) */                \
         t_off = smbios_tables_len;                                        \
-        smbios_tables_len += sizeof(*t);                                  \
+        smbios_tables_len += tbl_len;                                     \
         smbios_tables = g_realloc(smbios_tables, smbios_tables_len);      \
         t = (struct smbios_type_##tbl_type *)(smbios_tables + t_off);     \
                                                                           \
         t->header.type = tbl_type;                                        \
-        t->header.length = sizeof(*t);                                    \
+        t->header.length = tbl_len;                                       \
         t->header.handle = cpu_to_le16(tbl_handle);                       \
     } while (0)
 
diff --git a/include/hw/firmware/smbios.h b/include/hw/firmware/smbios.h
index e7d386f7c8..7f3259a630 100644
--- a/include/hw/firmware/smbios.h
+++ b/include/hw/firmware/smbios.h
@@ -18,6 +18,8 @@
 
 
 #define SMBIOS_MAX_TYPE 127
+#define offsetofend(TYPE, MEMBER) \
+       (offsetof(TYPE, MEMBER) + sizeof_field(TYPE, MEMBER))
 
 /* memory area description, used by type 19 table */
 struct smbios_phys_mem_area {
@@ -187,8 +189,18 @@ struct smbios_type_4 {
     uint8_t thread_count;
     uint16_t processor_characteristics;
     uint16_t processor_family2;
+    /* SMBIOS spec 3.0.0, Table 21 */
+    uint16_t core_count2;
+    uint16_t core_enabled2;
+    uint16_t thread_count2;
 } QEMU_PACKED;
 
+typedef enum smbios_type_4_len_ver {
+    SMBIOS_TYPE_4_LEN_V28 = offsetofend(struct smbios_type_4,
+                                        processor_family2),
+    SMBIOS_TYPE_4_LEN_V30 = offsetofend(struct smbios_type_4, thread_count2),
+} smbios_type_4_len_ver;
+
 /* SMBIOS type 8 - Port Connector Information */
 struct smbios_type_8 {
     struct smbios_structure_header header;
diff --git a/hw/smbios/smbios.c b/hw/smbios/smbios.c
index 51437ca09f..b4243de735 100644
--- a/hw/smbios/smbios.c
+++ b/hw/smbios/smbios.c
@@ -711,8 +711,14 @@ static void smbios_build_type_3_table(void)
 static void smbios_build_type_4_table(MachineState *ms, unsigned instance)
 {
     char sock_str[128];
+    size_t tbl_len = SMBIOS_TYPE_4_LEN_V28;
 
-    SMBIOS_BUILD_TABLE_PRE(4, T4_BASE + instance, true); /* required */
+    if (smbios_ep_type == SMBIOS_ENTRY_POINT_TYPE_64) {
+        tbl_len = SMBIOS_TYPE_4_LEN_V30;
+    }
+
+    SMBIOS_BUILD_TABLE_PRE_SIZE(4, T4_BASE + instance,
+                                true, tbl_len); /* required */
 
     snprintf(sock_str, sizeof(sock_str), "%s%2x", type4.sock_pfx, instance);
     SMBIOS_TABLE_SET_STR(4, socket_designation_str, sock_str);
@@ -739,8 +745,15 @@ static void smbios_build_type_4_table(MachineState *ms, unsigned instance)
     SMBIOS_TABLE_SET_STR(4, serial_number_str, type4.serial);
     SMBIOS_TABLE_SET_STR(4, asset_tag_number_str, type4.asset);
     SMBIOS_TABLE_SET_STR(4, part_number_str, type4.part);
-    t->core_count = t->core_enabled = ms->smp.cores;
-    t->thread_count = ms->smp.threads;
+
+    t->core_count = (ms->smp.cores > 255) ? 0xFF : ms->smp.cores;
+    t->core_enabled = t->core_count;
+
+    t->core_count2 = t->core_enabled2 = cpu_to_le16(ms->smp.cores);
+
+    t->thread_count = (ms->smp.threads > 255) ? 0xFF : ms->smp.threads;
+    t->thread_count2 = cpu_to_le16(ms->smp.threads);
+
     t->processor_characteristics = cpu_to_le16(0x02); /* Unknown */
     t->processor_family2 = cpu_to_le16(0x01); /* Other */
 
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 78/83] bios-tables-test: teach test to use smbios 3.0 tables
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (76 preceding siblings ...)
  2022-11-07 22:53 ` [PULL v4 77/83] hw/smbios: add core_count2 to smbios table type 4 Michael S. Tsirkin
@ 2022-11-07 22:54 ` Michael S. Tsirkin
  2022-11-07 22:54 ` [PULL v4 79/83] tests/acpi: allow changes for core_count2 test Michael S. Tsirkin
                   ` (6 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:54 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Julia Suvorova, Igor Mammedov, Ani Sinha

From: Julia Suvorova <jusual@redhat.com>

Introduce the 64-bit entry point. Since we no longer have a total
number of structures, stop checking for the new ones at the EOF
structure (type 127).

Signed-off-by: Julia Suvorova <jusual@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20220731162141.178443-3-jusual@redhat.com>
Message-Id: <20221011111731.101412-3-jusual@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/qtest/bios-tables-test.c | 100 +++++++++++++++++++++++++--------
 1 file changed, 76 insertions(+), 24 deletions(-)

diff --git a/tests/qtest/bios-tables-test.c b/tests/qtest/bios-tables-test.c
index e805b3efec..aa91b0fca5 100644
--- a/tests/qtest/bios-tables-test.c
+++ b/tests/qtest/bios-tables-test.c
@@ -88,8 +88,8 @@ typedef struct {
     uint64_t rsdp_addr;
     uint8_t rsdp_table[36 /* ACPI 2.0+ RSDP size */];
     GArray *tables;
-    uint32_t smbios_ep_addr;
-    struct smbios_21_entry_point smbios_ep_table;
+    uint64_t smbios_ep_addr[SMBIOS_ENTRY_POINT_TYPE__MAX];
+    SmbiosEntryPoint smbios_ep_table;
     uint16_t smbios_cpu_max_speed;
     uint16_t smbios_cpu_curr_speed;
     uint8_t *required_struct_types;
@@ -533,10 +533,9 @@ static void test_acpi_asl(test_data *data)
     free_test_data(&exp_data);
 }
 
-static bool smbios_ep_table_ok(test_data *data)
+static bool smbios_ep2_table_ok(test_data *data, uint32_t addr)
 {
-    struct smbios_21_entry_point *ep_table = &data->smbios_ep_table;
-    uint32_t addr = data->smbios_ep_addr;
+    struct smbios_21_entry_point *ep_table = &data->smbios_ep_table.ep21;
 
     qtest_memread(data->qts, addr, ep_table, sizeof(*ep_table));
     if (memcmp(ep_table->anchor_string, "_SM_", 4)) {
@@ -559,13 +558,29 @@ static bool smbios_ep_table_ok(test_data *data)
     return true;
 }
 
-static void test_smbios_entry_point(test_data *data)
+static bool smbios_ep3_table_ok(test_data *data, uint64_t addr)
+{
+    struct smbios_30_entry_point *ep_table = &data->smbios_ep_table.ep30;
+
+    qtest_memread(data->qts, addr, ep_table, sizeof(*ep_table));
+    if (memcmp(ep_table->anchor_string, "_SM3_", 5)) {
+        return false;
+    }
+
+    if (acpi_calc_checksum((uint8_t *)ep_table, sizeof *ep_table)) {
+        return false;
+    }
+
+    return true;
+}
+
+static SmbiosEntryPointType test_smbios_entry_point(test_data *data)
 {
     uint32_t off;
 
     /* find smbios entry point structure */
     for (off = 0xf0000; off < 0x100000; off += 0x10) {
-        uint8_t sig[] = "_SM_";
+        uint8_t sig[] = "_SM_", sig3[] = "_SM3_";
         int i;
 
         for (i = 0; i < sizeof sig - 1; ++i) {
@@ -574,14 +589,30 @@ static void test_smbios_entry_point(test_data *data)
 
         if (!memcmp(sig, "_SM_", sizeof sig)) {
             /* signature match, but is this a valid entry point? */
-            data->smbios_ep_addr = off;
-            if (smbios_ep_table_ok(data)) {
+            if (smbios_ep2_table_ok(data, off)) {
+                data->smbios_ep_addr[SMBIOS_ENTRY_POINT_TYPE_32] = off;
+            }
+        }
+
+        for (i = 0; i < sizeof sig3 - 1; ++i) {
+            sig3[i] = qtest_readb(data->qts, off + i);
+        }
+
+        if (!memcmp(sig3, "_SM3_", sizeof sig3)) {
+            if (smbios_ep3_table_ok(data, off)) {
+                data->smbios_ep_addr[SMBIOS_ENTRY_POINT_TYPE_64] = off;
+                /* found 64-bit entry point, no need to look for 32-bit one */
                 break;
             }
         }
     }
 
-    g_assert_cmphex(off, <, 0x100000);
+    /* found at least one entry point */
+    g_assert_true(data->smbios_ep_addr[SMBIOS_ENTRY_POINT_TYPE_32] ||
+                  data->smbios_ep_addr[SMBIOS_ENTRY_POINT_TYPE_64]);
+
+    return data->smbios_ep_addr[SMBIOS_ENTRY_POINT_TYPE_64] ?
+           SMBIOS_ENTRY_POINT_TYPE_64 : SMBIOS_ENTRY_POINT_TYPE_32;
 }
 
 static inline bool smbios_single_instance(uint8_t type)
@@ -625,16 +656,23 @@ static bool smbios_cpu_test(test_data *data, uint32_t addr)
     return true;
 }
 
-static void test_smbios_structs(test_data *data)
+static void test_smbios_structs(test_data *data, SmbiosEntryPointType ep_type)
 {
     DECLARE_BITMAP(struct_bitmap, SMBIOS_MAX_TYPE+1) = { 0 };
-    struct smbios_21_entry_point *ep_table = &data->smbios_ep_table;
-    uint32_t addr = le32_to_cpu(ep_table->structure_table_address);
-    int i, len, max_len = 0;
+
+    SmbiosEntryPoint *ep_table = &data->smbios_ep_table;
+    int i = 0, len, max_len = 0;
     uint8_t type, prv, crt;
+    uint64_t addr;
+
+    if (ep_type == SMBIOS_ENTRY_POINT_TYPE_32) {
+        addr = le32_to_cpu(ep_table->ep21.structure_table_address);
+    } else {
+        addr = le64_to_cpu(ep_table->ep30.structure_table_address);
+    }
 
     /* walk the smbios tables */
-    for (i = 0; i < le16_to_cpu(ep_table->number_of_structures); i++) {
+    do {
 
         /* grab type and formatted area length from struct header */
         type = qtest_readb(data->qts, addr);
@@ -660,19 +698,33 @@ static void test_smbios_structs(test_data *data)
         }
 
         /* keep track of max. struct size */
-        if (max_len < len) {
+        if (ep_type == SMBIOS_ENTRY_POINT_TYPE_32 && max_len < len) {
             max_len = len;
-            g_assert_cmpuint(max_len, <=, ep_table->max_structure_size);
+            g_assert_cmpuint(max_len, <=, ep_table->ep21.max_structure_size);
         }
 
         /* start of next structure */
         addr += len;
-    }
 
-    /* total table length and max struct size must match entry point values */
-    g_assert_cmpuint(le16_to_cpu(ep_table->structure_table_length), ==,
-                     addr - le32_to_cpu(ep_table->structure_table_address));
-    g_assert_cmpuint(le16_to_cpu(ep_table->max_structure_size), ==, max_len);
+    /*
+     * Until all structures have been scanned (ep21)
+     * or an EOF structure is found (ep30)
+     */
+    } while (ep_type == SMBIOS_ENTRY_POINT_TYPE_32 ?
+                ++i < le16_to_cpu(ep_table->ep21.number_of_structures) :
+                type != 127);
+
+    if (ep_type == SMBIOS_ENTRY_POINT_TYPE_32) {
+        /*
+         * Total table length and max struct size
+         * must match entry point values
+         */
+        g_assert_cmpuint(le16_to_cpu(ep_table->ep21.structure_table_length), ==,
+            addr - le32_to_cpu(ep_table->ep21.structure_table_address));
+
+        g_assert_cmpuint(le16_to_cpu(ep_table->ep21.max_structure_size), ==,
+            max_len);
+    }
 
     /* required struct types must all be present */
     for (i = 0; i < data->required_struct_types_len; i++) {
@@ -756,8 +808,8 @@ static void test_acpi_one(const char *params, test_data *data)
      * https://bugs.launchpad.net/qemu/+bug/1821884
      */
     if (!use_uefi) {
-        test_smbios_entry_point(data);
-        test_smbios_structs(data);
+        SmbiosEntryPointType ep_type = test_smbios_entry_point(data);
+        test_smbios_structs(data, ep_type);
     }
 
     qtest_quit(data->qts);
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 79/83] tests/acpi: allow changes for core_count2 test
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (77 preceding siblings ...)
  2022-11-07 22:54 ` [PULL v4 78/83] bios-tables-test: teach test to use smbios 3.0 tables Michael S. Tsirkin
@ 2022-11-07 22:54 ` Michael S. Tsirkin
  2022-11-07 22:54 ` [PULL v4 80/83] bios-tables-test: add test for number of cores > 255 Michael S. Tsirkin
                   ` (5 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:54 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Julia Suvorova, Igor Mammedov, Ani Sinha

From: Julia Suvorova <jusual@redhat.com>

Signed-off-by: Julia Suvorova <jusual@redhat.com>
Message-Id: <20220731162141.178443-4-jusual@redhat.com>
Message-Id: <20221011111731.101412-4-jusual@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Igor Mammedov <imammedo@redhat.com>
---
 tests/qtest/bios-tables-test-allowed-diff.h | 3 +++
 tests/data/acpi/q35/APIC.core-count2        | 0
 tests/data/acpi/q35/DSDT.core-count2        | 0
 tests/data/acpi/q35/FACP.core-count2        | 0
 4 files changed, 3 insertions(+)
 create mode 100644 tests/data/acpi/q35/APIC.core-count2
 create mode 100644 tests/data/acpi/q35/DSDT.core-count2
 create mode 100644 tests/data/acpi/q35/FACP.core-count2

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index dfb8523c8b..e81dc67a2e 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1 +1,4 @@
 /* List of comma-separated changed AML files to ignore */
+"tests/data/acpi/q35/APIC.core-count2",
+"tests/data/acpi/q35/DSDT.core-count2",
+"tests/data/acpi/q35/FACP.core-count2",
diff --git a/tests/data/acpi/q35/APIC.core-count2 b/tests/data/acpi/q35/APIC.core-count2
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/data/acpi/q35/DSDT.core-count2 b/tests/data/acpi/q35/DSDT.core-count2
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/data/acpi/q35/FACP.core-count2 b/tests/data/acpi/q35/FACP.core-count2
new file mode 100644
index 0000000000..e69de29bb2
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 80/83] bios-tables-test: add test for number of cores > 255
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (78 preceding siblings ...)
  2022-11-07 22:54 ` [PULL v4 79/83] tests/acpi: allow changes for core_count2 test Michael S. Tsirkin
@ 2022-11-07 22:54 ` Michael S. Tsirkin
  2022-11-07 22:54 ` [PULL v4 81/83] tests/acpi: update tables for new core count test Michael S. Tsirkin
                   ` (4 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:54 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Julia Suvorova, Igor Mammedov, Ani Sinha

From: Julia Suvorova <jusual@redhat.com>

The new test is run with a large number of cpus and checks if the
core_count field in smbios_cpu_test (structure type 4) is correct.

Choose q35 as it allows to run with -smp > 255.

Signed-off-by: Julia Suvorova <jusual@redhat.com>
Message-Id: <20220731162141.178443-5-jusual@redhat.com>
Message-Id: <20221011111731.101412-5-jusual@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
---
 tests/qtest/bios-tables-test.c | 58 ++++++++++++++++++++++++++--------
 1 file changed, 45 insertions(+), 13 deletions(-)

diff --git a/tests/qtest/bios-tables-test.c b/tests/qtest/bios-tables-test.c
index aa91b0fca5..395d441212 100644
--- a/tests/qtest/bios-tables-test.c
+++ b/tests/qtest/bios-tables-test.c
@@ -92,6 +92,8 @@ typedef struct {
     SmbiosEntryPoint smbios_ep_table;
     uint16_t smbios_cpu_max_speed;
     uint16_t smbios_cpu_curr_speed;
+    uint8_t smbios_core_count;
+    uint16_t smbios_core_count2;
     uint8_t *required_struct_types;
     int required_struct_types_len;
     QTestState *qts;
@@ -631,29 +633,42 @@ static inline bool smbios_single_instance(uint8_t type)
     }
 }
 
-static bool smbios_cpu_test(test_data *data, uint32_t addr)
+static void smbios_cpu_test(test_data *data, uint32_t addr,
+                            SmbiosEntryPointType ep_type)
 {
-    uint16_t expect_speed[2];
-    uint16_t real;
+    uint8_t core_count, expected_core_count = data->smbios_core_count;
+    uint16_t speed, expected_speed[2];
+    uint16_t core_count2, expected_core_count2 = data->smbios_core_count2;
     int offset[2];
     int i;
 
     /* Check CPU speed for backward compatibility */
     offset[0] = offsetof(struct smbios_type_4, max_speed);
     offset[1] = offsetof(struct smbios_type_4, current_speed);
-    expect_speed[0] = data->smbios_cpu_max_speed ? : 2000;
-    expect_speed[1] = data->smbios_cpu_curr_speed ? : 2000;
+    expected_speed[0] = data->smbios_cpu_max_speed ? : 2000;
+    expected_speed[1] = data->smbios_cpu_curr_speed ? : 2000;
 
     for (i = 0; i < 2; i++) {
-        real = qtest_readw(data->qts, addr + offset[i]);
-        if (real != expect_speed[i]) {
-            fprintf(stderr, "Unexpected SMBIOS CPU speed: real %u expect %u\n",
-                    real, expect_speed[i]);
-            return false;
-        }
+        speed = qtest_readw(data->qts, addr + offset[i]);
+        g_assert_cmpuint(speed, ==, expected_speed[i]);
     }
 
-    return true;
+    core_count = qtest_readb(data->qts,
+                    addr + offsetof(struct smbios_type_4, core_count));
+
+    if (expected_core_count) {
+        g_assert_cmpuint(core_count, ==, expected_core_count);
+    }
+
+    if (ep_type == SMBIOS_ENTRY_POINT_TYPE_64) {
+        core_count2 = qtest_readw(data->qts,
+                          addr + offsetof(struct smbios_type_4, core_count2));
+
+        /* Core Count has reached its limit, checking Core Count 2 */
+        if (expected_core_count == 0xFF && expected_core_count2) {
+            g_assert_cmpuint(core_count2, ==, expected_core_count2);
+        }
+    }
 }
 
 static void test_smbios_structs(test_data *data, SmbiosEntryPointType ep_type)
@@ -686,7 +701,7 @@ static void test_smbios_structs(test_data *data, SmbiosEntryPointType ep_type)
         set_bit(type, struct_bitmap);
 
         if (type == 4) {
-            g_assert(smbios_cpu_test(data, addr));
+            smbios_cpu_test(data, addr, ep_type);
         }
 
         /* seek to end of unformatted string area of this struct ("\0\0") */
@@ -908,6 +923,21 @@ static void test_acpi_q35_tcg(void)
     free_test_data(&data);
 }
 
+static void test_acpi_q35_tcg_core_count2(void)
+{
+    test_data data = {
+        .machine = MACHINE_Q35,
+        .variant = ".core-count2",
+        .required_struct_types = base_required_struct_types,
+        .required_struct_types_len = ARRAY_SIZE(base_required_struct_types),
+        .smbios_core_count = 0xFF,
+        .smbios_core_count2 = 275,
+    };
+
+    test_acpi_one("-machine smbios-entry-point-type=64 -smp 275", &data);
+    free_test_data(&data);
+}
+
 static void test_acpi_q35_tcg_bridge(void)
 {
     test_data data;
@@ -1994,6 +2024,8 @@ int main(int argc, char *argv[])
             if (has_kvm) {
                 qtest_add_func("acpi/q35/kvm/xapic", test_acpi_q35_kvm_xapic);
                 qtest_add_func("acpi/q35/kvm/dmar", test_acpi_q35_kvm_dmar);
+                qtest_add_func("acpi/q35/core-count2",
+                               test_acpi_q35_tcg_core_count2);
             }
             qtest_add_func("acpi/q35/viot", test_acpi_q35_viot);
 #ifdef CONFIG_POSIX
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 81/83] tests/acpi: update tables for new core count test
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (79 preceding siblings ...)
  2022-11-07 22:54 ` [PULL v4 80/83] bios-tables-test: add test for number of cores > 255 Michael S. Tsirkin
@ 2022-11-07 22:54 ` Michael S. Tsirkin
  2022-11-07 22:54   ` [Virtio-fs] " Michael S. Tsirkin
                   ` (3 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:54 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Julia Suvorova, Igor Mammedov, Ani Sinha

From: Julia Suvorova <jusual@redhat.com>

Changes in the tables (for 275 cores):
FACP:
+                 Use APIC Cluster Model (V4) : 1

APIC:
+[02Ch 0044   1]                Subtable Type : 00 [Processor Local APIC]
+[02Dh 0045   1]                       Length : 08
+[02Eh 0046   1]                 Processor ID : 00
+[02Fh 0047   1]                Local Apic ID : 00
+[030h 0048   4]        Flags (decoded below) : 00000001
+                           Processor Enabled : 1
...
+
+[81Ch 2076   1]                Subtable Type : 00 [Processor Local APIC]
+[81Dh 2077   1]                       Length : 08
+[81Eh 2078   1]                 Processor ID : FE
+[81Fh 2079   1]                Local Apic ID : FE
+[820h 2080   4]        Flags (decoded below) : 00000001
+                           Processor Enabled : 1
+                      Runtime Online Capable : 0
+
+[824h 2084   1]                Subtable Type : 09 [Processor Local x2APIC]
+[825h 2085   1]                       Length : 10
+[826h 2086   2]                     Reserved : 0000
+[828h 2088   4]          Processor x2Apic ID : 000000FF
+[82Ch 2092   4]        Flags (decoded below) : 00000001
+                           Processor Enabled : 1
+[830h 2096   4]                Processor UID : 000000FF
...

DSDT:
+            Processor (C001, 0x01, 0x00000000, 0x00)
+            {
+                Method (_STA, 0, Serialized)  // _STA: Status
+                {
+                    Return (CSTA (One))
+                }
+
+                Name (_MAT, Buffer (0x08)  // _MAT: Multiple APIC Table Entry
+                {
+                     0x00, 0x08, 0x01, 0x01, 0x01, 0x00, 0x00, 0x00   // ........
+                })
+                Method (_EJ0, 1, NotSerialized)  // _EJx: Eject Device, x=0-9
+                {
+                    CEJ0 (One)
+                }
+
+                Method (_OST, 3, Serialized)  // _OST: OSPM Status Indication
+                {
+                    COST (One, Arg0, Arg1, Arg2)
+                }
+            }
...
+            Processor (C0FE, 0xFE, 0x00000000, 0x00)
+            {
+                Method (_STA, 0, Serialized)  // _STA: Status
+                {
+                    Return (CSTA (0xFE))
+                }
+
+                Name (_MAT, Buffer (0x08)  // _MAT: Multiple APIC Table Entry
+                {
+                     0x00, 0x08, 0xFE, 0xFE, 0x01, 0x00, 0x00, 0x00   // ........
+                })
+                Method (_EJ0, 1, NotSerialized)  // _EJx: Eject Device, x=0-9
+                {
+                    CEJ0 (0xFE)
+                }
+
+                Method (_OST, 3, Serialized)  // _OST: OSPM Status Indication
+                {
+                    COST (0xFE, Arg0, Arg1, Arg2)
+                }
+            }
+
+            Device (C0FF)
+            {
+                Name (_HID, "ACPI0007" /* Processor Device */)  // _HID: Hardware ID
+                Name (_UID, 0xFF)  // _UID: Unique ID
+                Method (_STA, 0, Serialized)  // _STA: Status
+                {
+                    Return (CSTA (0xFF))
+                }
+
+                Name (_MAT, Buffer (0x10)  // _MAT: Multiple APIC Table Entry
+                {
+                    /* 0000 */  0x09, 0x10, 0x00, 0x00, 0xFF, 0x00, 0x00, 0x00,  // ........
+                    /* 0008 */  0x01, 0x00, 0x00, 0x00, 0xFF, 0x00, 0x00, 0x00   // ........
+                })
+                Method (_EJ0, 1, NotSerialized)  // _EJx: Eject Device, x=0-9
+                {
+                    CEJ0 (0xFF)
+                }
+
+                Method (_OST, 3, Serialized)  // _OST: OSPM Status Indication
+                {
+                    COST (0xFF, Arg0, Arg1, Arg2)
+                }
+            }
+
...

Signed-off-by: Julia Suvorova <jusual@redhat.com>
Message-Id: <20220731162141.178443-6-jusual@redhat.com>
Message-Id: <20221011111731.101412-6-jusual@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 tests/qtest/bios-tables-test-allowed-diff.h |   3 ---
 tests/data/acpi/q35/APIC.core-count2        | Bin 0 -> 2478 bytes
 tests/data/acpi/q35/DSDT.core-count2        | Bin 0 -> 32552 bytes
 tests/data/acpi/q35/FACP.core-count2        | Bin 0 -> 244 bytes
 4 files changed, 3 deletions(-)

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index e81dc67a2e..dfb8523c8b 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1,4 +1 @@
 /* List of comma-separated changed AML files to ignore */
-"tests/data/acpi/q35/APIC.core-count2",
-"tests/data/acpi/q35/DSDT.core-count2",
-"tests/data/acpi/q35/FACP.core-count2",
diff --git a/tests/data/acpi/q35/APIC.core-count2 b/tests/data/acpi/q35/APIC.core-count2
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..a255082ef5bc39f0d92d3e372b91f09dd6d0d9a1 100644
GIT binary patch
literal 2478
zcmXZeWl$AS7=Youz=a#M-K}6ZwgLuNAQ;$~*xjvQcY@uCVs{~Sf`WpLVt2Rbe!ORA
z_B`J^b9R56*&pj2=<ge2)-*$cPk^sqaDJbVK;QiOWzaNDW>M2p(=#;b`y@>U1KQZ2
ztu5Nwq0xx;_UPb%CKH;?XtAKxijI!x<b=-7=;DH|uIT25?(Uc=6K2kgS+Zc(te7nu
zX3vf}a$wG!m@60N&W(BUVBWl#FCTI)nyEkmx?n*pR0s<f#v(<qXi+Ry3_U#1(-Vsq
z#}Xy5WJxSl3QL#9GG(xASu9r$%a_Lr6|iDOtW*grS4J-{tWpK5R>f-7uzGc@Q3Gq%
z#9Fnmc5SRv2fe+~#|M4+PE2*{()H?L{rcFT0s8r&zdtr?h>aRy<Hp#e2{vtt0Rb2o
zh|QW|P!I+OWAo<Nq6M~WiLF{;NC>uWjcwXs+qT%Q9ky?e9Xepgju;w>ojPIX&e)|3
zcI}GYx?%V37#4;-dSK6<*sB-z?u~u=VBfyjuOIgBj{^qaz=1eu5Dp%ULx$kcp*U<9
z4j+yqM&QViIBFD*9*twh;MlP^ZXAvuj}s=~#ECd*5{8FkL<CNrj8mrI)Tuaa8cv^%
zGiKn-nK)|}&Yq2P=HT49IBy=#pN|U`;KGHtXb~=6j7yeaWF$sK;nJnJY#A<Jjw@E+
z%9Xfk6|P>5Yu4b}wYY8_u3wKEHsHpMxM>q^-i%we;MT3UZ5u{M<M!>iV+Y2>;Le@6
zYZva`jeGXs-o3bQAMW3e2M*xDgLvo=9zKjmj^NRwc<dM+KaM9(;K`F18;hq-VO$)Z
zK8<J2;Mucy?i`*!j~6cB#fy095?;QHSFYgIt9b1i#>Znq0$#t4H*R2JA|@r_&6{}Z
z7A7ZSN($b-jd$+g-Me`29^Su?4<6vdhnSj*j~?OU$C#FePoCh@r}*p{K7WocUf|1@
z`05qDevNP5;M=$O?j62=j~_nZ$B+2w6Mp`TU%ueiulVg7e*ca?e&Ela`0E$`{*8bB
z;NQQPo-UeQHSM3S%%ZeJ#vXl<HmDY*ZB&cWwyH&GJJq7JQMD*-uUeFLP%TQEREyGP
z)uOaTwJ2>>ElNA87Nwn3i_*@jMQIn+qO_}OQQA$lDDAE~Lr49*wAgf6Z7ljNgG@%F
cu9Hk={TGbMqHkcbS~Dh#{`5cn(qE|k2lzA_5C8xG

literal 0
HcmV?d00001

diff --git a/tests/data/acpi/q35/DSDT.core-count2 b/tests/data/acpi/q35/DSDT.core-count2
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..ca309f6569f1f4016cf7e67117b208d57fbc7365 100644
GIT binary patch
literal 32552
zcmb8&cYKpo7YFbsooSl3X(=0_g1ASzTX$w#pry10)S_(xmEnj2Dk2I+5phDrJqp&n
z_g36napT^5Z=Ag6JkL$;?>V=AKCk@IoRi-BJ?ExvZu8`!Pqa1Kw(SX)<tYqLjMTT5
zg!gHVl+ZtqWmz2aZJ;4zn$^A}8m{Z`<fdhK+Lz|qF?|Soi+7BMC)at-o7H(vXXpHr
zJ3U<qYx=stn(1AMWh*+TTRdpN^n}mHK_Hgj(Hib(?~eGIBMm`sM}0%oAKKvadOIRc
zl<w^aMO!Ry$K-~nRXicPV`6J0W93{c5}jJw+mzWet8joX(BAk~`ER2}?0wGE`JwfD
zynfKixxNe<P)4_2x-C!eht5TvzLA~1fz>^;J&Sp|{khWK0jqoLPdoPo_%n%4zGto9
zxr?dNzPat)HPMz(#O^V#xv4oA36@e{k(SoNQN9r;54J2nKREpFrp=o-Kj?&3V4)S5
zP`ooQlb6`Oq&d>s=E+^}_j%iw1~`e({gX@RL$suQNo#BzJ*ZFmeHrwqauNv#W8>%(
z;ikqGPwob18Yhu()8xjI=7#7#CGDQ@gvfZScx-bdHZHAw3}>4+CuZ@8#6r((i;ebe
z`0hTdc!v<p>2DNU3I5Qj#&#BmbDE>Ajd{J%te!bmqA=ZKbtbH?-eIeq2Yy5KQ=ha}
z9^1HT@RyB4ABi;fb@pbjne9vK>r8NwWh)M`Ec)QFrqh(gJ=3k8>3(`_Hd~9kJY9>s
z7J7WyR#&2HMW<!a)UiEteA$Vj%rq;H$_OAa-5KR=ZivLKVt)tSdJ?lZ3v^HM@Z%eZ
zd})6dHMg`i7mc5{q$oEdmugLD8XvQq*h6vLD)RIFzBEeel1Vxx^;S_{2K7f@MrWML
zcoq%JcRpv&qoK)^XOeG^O|xgxSe5b2<j^$Bo=HRPnUwNO>c$?MZqH11kZ$qJ<j{0y
zCJnV`Qpz)vRWj_E$qq8?naQCU&dg-<l;S!wUF*!WXC{YcIx~~aQ;O@%a;-DVo|zn)
z<z6SHxXx_XI<sBt%yzGnQe3CkwN9^VonH4kDaCc>xYn8DT4#=Xos{A_eXe!-T<i3?
z*GVa^GuO4wT-Q2t-Rq<j*Xehy)9+fR-@Q&sah-Xtb>_L&nde?7rMS*~*E;iE>&$nr
zlTut~foq)wu5}i;*GVa^GvHchz_reRd!3ZxItyLvEOf22(7jGdah*l3br!kSS>#?P
zrMS)ku5}J@t#g2Tos{A_i(Tt1cCEA6y-rGTodaF#9Ozo-K=(Q+#dQvHt#goTorB!#
zq!ial+nDp5Wj`wocCB-;d!3ZxI)}K{ImETjA?|fjit8L|&kU(&!lCxeuzIo^>dcJj
zlxIfO*kP`j!(20mxo1*}Gl#oo4tLEQ?w(00&K%*IIl?t_gnK5XICBfv%q?6qw{Xv-
z6ladKXC|NHN7^%!&+#LjnaL+^N_l2-CmH3cIm%UYl)EOSPE9Xupdr{m>slMAw7r>n
zD5k<{1Eqw@I!QVu$)dD@I^*nbB9FzhlKpw8s53ORfl``W8f~D?*yIKZLwN&rhNd=9
zN~xRV2I`De8P_B=xnuIo<Riu#D5cbqJ(f05XRON98ts|9fjUEzDXx<?P=~c=(gq4c
zc>|?1xhwJpN@;SPw1GNf?U}TJ!cg8oouPJ3-asi$u9G%UXRJMwHc%MK8>lnXp2-_1
zrO9>D2I`EpXVL}=LwN&rhT1cE1En<i^`s5d8Eem^4HSm*2I>s8XYvM0X>y&kfjVRD
znY4kzP~JeDq4rGPKq*bGlQvLitUZ%9P#DS^s58`_$r~u8$#v2O>WsB#(gq4cc>{HZ
z+B10rr8K!t+CZJL_DtG9VJL5)&QN<MZ=jSW*GU_wGuEC-8z>Cr4b&NG&*Tl1(&Rd6
z19isQGid{bp}c`QL+zQofl``WCvBk4SbHXIpfHp-P-mz;lQ&RGlk21n)ER5fqzx2?
z@&@V*wP*4MN@;SPw1GNf?U}TJ!cg8oouT$j-asi$u9G%UXRJMwHc%MK8>lnXp2-_1
zrO9>D2I`EpXVL}=LwN&rhT1cE1En;%PTD}7vGz>bKw&6vpw3WxCU2mWCf7+Ds591{
zNgF5(<qgysYR}{il+xrnX#;h}+B0bbg`vEGIz#Q5yn#}hTqkXy&RBaUZJ;oeH&AD&
zJ(D+3N|WoP4U{t2Kq<owlv3P4DTNIbX3_=<GkF7rnY@9*Oxi$UCT*ZFlQ&S9$r~ul
zqzx2i(gq4Mc>{%+yn(_@+CX6@ZJ;odH&B?#8z{`A4V1FvSC=+W%IGU*KbX9MQc7Qj
zWXdy>k1cJWlqDZq+CV8wKDNApQkr~hc>|@Cj#CWlpe@wu_4n8>R{Qt`>Dny52^9}$
zm_)Dtdvg}~mdvvHd^vr-+~SIcNs*Qu+ICxlh55eRQ?jfa`%uC{YbD?3%cPGDiAA$~
z1r|pHwFKL{CuGw;A4(oG@fSG9NW4_KwQ!Jcz$y7wPCD&S7Ci@B^lWM^w2BKlTH8Wa
zAGgrlGAUGC*b!}LwH9;iTVmf+zy~yVM5M*e2Y0l#w0c&~Qax~azF$=ji|U^LP(2LQ
z!*=yB^@r8N#r;<gw|6)4+MQ!Dbi7Ed-Iw+s%15Al#4aBZ<s<)6KBCG;#o~Q*tm^-M
z?4wXVYL}0S^3i`OA64aJvV6vWC?A9JF}r+Bl#l&O`Issnm*q46L-{zAkK5(rqI~>c
z%EwjtI$1vJKa{V7@^yCkI#IsvU&`01^7XQO_J1f}59RCa^7W#8{lAp2r}7#6bR^3M
zobt&h5Bk-Be%zgdlkWS9v+N&vy5XN|#mOIIU!j|$-xmzdxvyW&__-`{bM#Au!8!N!
z%Najs1Kb?_W?^v7ef@IA&rPwLqhB`+&bhB&&iFYP=;r8m5QB5>>z6aWuLrp~`o+ZH
zocsFajPKLIZjOFiF*xTwHy3F1ItS`w-umWPTk_k_Lr-~pV7`4xeREr*MZbA(_$Sw*
zL)QV$2TX}IMN67nq8%m0Rrb!4;Cng~d{@`8oxVV4uYXlfr`6L*$7B1f-uzWPUDnxM
ze(Qr@GW%$Aq%DmPuNV6Am(xoS#w~iWMhDc3JL;O9uVY>)(%cXX2CJ;VsblDyn%_*Q
z_f6X+J$b;rxh2+`Y}ZFm64Y*9q<L~l{o(Tyhb%a_<j4c(9lqem83)d@+C7ox3FFmI
zO`gwJ=h^7r==b;+((>qeoetV>^!xnnOSk1DWR*mkV@;k&LsP3K($YA^Lx<jj{2}hK
z(j$?^C>`D};*sf**4Ts+8XgU`mC$kPhNcE@!s0vIyH^a!nr6Q>Kpi<<s`Zoxxew<p
zkj&x<FGSkn`=<5!dS+S6BedGt9C|n`o!y(;Gs~C8ahVgRb6oDk85~zQaVE!=PMpPY
zl@n)kT<yeOj%%DahvQl&_Hi6?;#`iyPVDD6;>39zN1ZsI<Cqf{a2$8y0L7)$KKES6
zvFg8wW7Yovj#dA~9IO5Za;*9v#IfpsFvqI@AsnmzhjOg?AI7oje>lgg{}CLk{<q**
z^*@qh)&D4tRsSUvm#O}@<XH7Tnq$@f7>-r{TXC%VAIq`oe`}6a|J!h^`rnpg)&F)J
ztNyp=SoOaH$EyDwIadAe#IfpsXO30>yKt=f4^mvN`Y+{J^<T!Z>c5<0)qe%Ys{cxk
zRsU5StNyDwR{hs-topCzSoOav$EyF`I9C1d&avu$4~|v;dvdJ$--~0_|K1c=sQyD7
ztNz0rtNtS#tNx=LtNvpgtN!C0tN!aaR{hs=tom=@SoJ@SW7Yq7j#d8?I9B~Pa;*Aq
z;#l=Rkz>_=GsTsv|4AIH{#!U!{kL+g`fuY{^*@<o)&CTZRsT~tR{ih8vFd+cj#dBr
zajg1p=UDYWjbqjS{v50R58zn!-@&o!zmwuB)qfYqs{iR6tNv$jtoonHvFiUoj#d8$
zajg13m}AxdERI$GvpH7%&*510KbK?G{~;W!{tx9?^?w-0s{g||R{bX^u2%gY!LjQ9
zNRCzi^Eg)h&*xb6zkp-a|4|&P{*UHZ^?wY<s{e%?tNxGWSoOb%W7Yq09IO5pbFBJ5
zo@3Sj2^_2bmrz`z`d`Yi>c5*~)&DY%RsSb)tolERW7Yq1j#d9BbFBJb!LjQ96pmH@
zD>+vEpUSc7|1^$O|EF`T`agqX)&H3stNzcTxK{PQieuIPYK~R^Jshk4&*oV5zlLMg
z|2Z71{?FxD^?x46s{ivjR{dYVvFiUqj#d8`ajg2km}AxdB^;~%FXdSE-%D{w^}m*5
z)&Dw<RsWZ9torZcSoME7$EyD;I9B~%$+7DHDvnkES97fTzlLMg|Fs;e{;%U$^?yCb
zs{b1}R{h_|vFiUOio>e^n>kkf-@>u#|5lDw|F?0h`oEoH)&CtFtN!ogSoMDw$EyFk
zIadAO!?EiBUXE4&_i?QHzn^2({{tMW{vYI6^}n9ti0c0#j#d8;bFBJ*gk#nJqa3UL
zALCf{|2W60|0g(B{XfaE>i;Q@RsT<Otonb3W7Yq&9IO7H<5>0oJjbg47dTe^Z=g7;
z`hSsQ)&ENztNvf+SoQx3$EyEVIad9@#<A-Eb&gg48#z|}zrnHU|4oin|8H@u`hS~a
z)&DyjtN!2RSoQxN$EyGLDUPZBKj2vP{~^b!|BpCU{eR4{>i-jtRsWxItor|qW7YrX
z9IO7n;8^wlCC94&uQ*oyf6cM#{~L}~|KD<~`u~n&)&KVt$5sD7aIE_Okz>{WPaLcM
zf96>A{|m>e|6e&){r|?X>VFf*s{h|PR{j6MvFiU%j#dAEajg3Pn`71gKOC$6H*-w=
z`+H{5+aSv$rTqIpCmuVY6>ySfC$t1k((Q!Sz)6Ok&>}d=v=dqdCs}qv%itv2PG}vR
zc<qE1!by&u&`LP**$FL$lUzHYwQ%CM6Iu)>d3Hjp;UwQqXgQn|*oj(CfD?W}z@I6!
z6SbZqJ1Lu8T%SefNQH9R$4uKL17Fnfi#0yUN#0E_tP)md@30>E34Xbj@D<R>O~z*?
zjPL6#Zp`GDaR054M^Ys8>lfD#inK<Wy#KNcZ=@+UwJ2?K!kUrr%v={Q>Y3?DSThqI
zKHX|vlmD!qPS2VdxdZ6V&cqB^)Lyc@lU{=M4qhYgr5B+yF7M6h>GEx+LYF7{`+V<K
z#$(5?auYLFaF<=yf-dT7ZFO&@G0c7)OfOYu_|g-`LqngWc?)Ln`GBzu=Y&88r&}}l
z<?ReA&|cdA;tY%WSunl-2f8sQGO@MI|1S@8|KBf$PY+&Vc0Z3z!j~rR<)L%ro%AsG
z+g<yz8hSAt<bxR&pTtJr+1@_>{-&4vjiI)}T%XsX_i6d)e{nvY>)4jdi?L?Sp3}Z8
zmO)cX?KcKoQ|X|4YO2R$PtEO62P1+WU_3Tc&rBJ;(atsUbO(I&JmsXN*^WQu_|oVQ
zWwIY%nwHAx%rz;|QK6KSPUj5TOHFp<OGhbPOBHl%M@n=|J0)dgNGSuQ3@ugi+a6+d
zba*-?WoAk#6QxWoRnZ&uQlf)MDJd&UN?9mnX{nmtxR(+gG)_s`*;2|zDO*c5^bH~<
zI$<^?dA(BdqU6<5t$kF@{aug)QjYyL6n&efzIHh%<!C9yZ!(ET+y|1+Cwuat<kM1^
z->MQ)E=aWJ@TZ}tT$FOP6rsbBQu2f3x6ju_$&Zp>OHn#8KuUQa<>kqq@=(gtQjAU%
zkWxNK`T0`HM=4)RaXN88N_4z5^<7X<Af*D73Q|&Okl!s3k2t-bkdgucDFsjpXsMLn
z6c7@<m5`DO3#C+uQlXa0`27GO(YpvKsi;UwMJN?%shr;m5E8wCkdg)rkkSB@256~*
z-vtm7y?2n3ii@RGj8d_dD)|inA<^3gDQVz9DGfwvpq8rm*uRkI9fOoKXpoc!p)^QK
z)qL1rNc3hwN*X*^N`p}vtfd+{kwHpBKpHYcN<&Z@qNQ3ou|Y~hK^i(#N<&c^s-+N}
zDJZ33APpNPrC}%y(^8mDe2~&`kcJPJ(r}c9Ybin}LP%)@NFzo_X#`3mv=pTiBc!wi
zNLy?nr7cj}LQ63^Q9?>1K^i$yN+VGksiiobI3cA`AdMO&rBNu2N=aqG5>qMx$@$MI
z_&i>MQi+yIw=|_KLE3Ul+0&LNZK<WQ(WW#Sq|u|LG#aJRS}Gr7N@GA8Ge$~dP#UA9
zimgm(E0DI@N=jRyw3U`B$C}bukj9Rc(pZ$nYN={#Q`#D&t+$ra)+lYQrRr@=X&aEX
z*+xp+ptOyaYPL0{Z9&?0TPbae(zaTv-OiM@18KYMq_iDM+i5AZy(w)E()Qa+X?v8m
z*HU-~Q`!Nf9d?k?4k+!QrO1w^v?EA6?kJ@lQQA>U(Va|bCy;j9NlH7Rw3C)%JDbwZ
zAnm-fly*jGXD!8dF{NEV+GQ6h?Sj%SDXCn&oO8XFqW{X4+Bt(kDFsmqYN@o;?5Px_
zQv1~(KGT(=RH~)2GE*u8sjN))REAQSmdeXbsT`#8aw(OgRIa6p3R9{8siHzk6)06`
zsj||PDnY8Olu{*1m0GH*GNmezs;Z<^g;JH4s;f<@8l>uKDOICXt)-e8Q>p=}rbbFN
zDAj1Gw$_wtL8`5lQY}igS_<uIO1pxz>#kDT6{TIZ6yD90b^~d*-K4Y|O1o((vb!nm
z4$|(sOKEqMcGpsL4^!F$q&@bK(jF-7p{3ZKrnDzWd+sTvJyF_IOYyx-X)loW+Dl4%
zp|n>@stE3FN_&H}_uf+48>PLqR2niRIu$MTOcx4CDTGo;OJ!kGqO;IaQaCInJ~=dX
z{3)!Z@`x$X31}%PV*i(Ae9q+aJX2CcOBGR5qI1tuQZy=i;!`$LQdCQoF;k+`l~PhH
zCM7=mG9|^dR24TRI^!%Q#p!G#deH_)L-<6>loZ!eb)6~E$!003u1-pPPGm}|(^5^n
zDbaamDXG3*N_=`_N~+gVZG$N_fYi_+r3RE5v=ka=O5;EpH%?09P#UMD@OV=i57PMY
zQW}racr8UHn9>B0CQOjh1e7LdDcWdCbV6S05pQgiQX@)@T8cH9QWHo`O;Tz?sYy%m
ziKa9Wq=^%yG!dnVDXB8pY)Z`_H8)GC8Kq_|l}<9HNgz#{B&A6xP0~_Xiz&5$)Y2lQ
z7L;1FRNiVzbgEzKYuDN;rB;+$wN%k&N^~Y>N@{D9QX5KbTB@9EN|QmFJXuPUQJSo!
zswt*41*9ocq%;MkDO#$YYD!Z<nmScVQ&F0#rJ8+AX&;dG*+)wIptO&cYWFpzeL>oH
zUn%X2(!N>>?Pp5+fwbR#QrZut{j?NrH>Gxv+S{enj#9gpBGXK18c5ToNog8N)3g-b
z-<0+TY5)DDv_DGwYbkbsDIEaP0S8Fw0F(~UQoO^IIzZ~^kWvRq9Vw|Q*l9|gAa!<1
zsS~A6EtPheQWr>FT~g{osY^>`(@kkQNYkfFX*x>NwNyUClxBc5V}_Jwpfp2E6*Em~
zCP*`9N@*raGqqHCpeY>)(t!s`=|GeY)Kb+!rgRWU2OT7(gHSq1OVtOP(!n4de6W-b
zM(JQJ)yy)bSs=}tC8b#?&C*isY*U&I((Kt%nvK$IErsTo(j1WH%#qR@l;&tDJlB-w
zf;4xol;)x|S4)vYOz9Ai4xzs*<Nn?E5R?wlQuI($IuxWs50%oPC>^S$*kPu07)Xa5
zCZ)qrI!sIP!%gXMkPbgwN{6F#cuJ}cCQK;-QX(Ox1WE}ll^$VAM}Tz15mGt=r6aUd
zcBCmC3DS{AO6f?Hj?_~5JX4wn(!6<6nupRnEmh1nrTHMupD(5PD9zVW<pNV$0Mdd5
zQd)r00xeY?WlBeZbktE&Itrzuv{ZexDIE>c(ML<^Xq1lDQq3`@bPPzx93!P;P&!6S
zwF^yYAxH}sN@*cV3$+wF)|8F~>DXhXbSz58YAL+Ploo-sXpxi_p|nU#k>gD1IFODz
zPD;n2bexu=i%n@UNQ)OsX)#KRwG=zvl#U1K_~WH?JW9uFDSm<}odD7aCrIf8luk%V
zHNhpOv;?FjOQf^}r6pP_U1~~8L0Y<0N=s2%s-?1SQ|bn(yIV@#D0OS8e3>aN18LbZ
zDJ?^3nU*R}G^G<kI`Kp)oruzjTB<zBluiQaq?4p{5=tj&scN|?EeC1&aw#oGX}OlF
zPd25KK|1+lDV>bc$y%yeVM;4NTCqY(D^OaYrP@<W=@gJoIYmmRpmd6sLMu&aB}gk*
zN@*oZE436p)s#*J>C{uDbSg@xYAJG>DV+w=X{Sl)G?Y%$QuK6FIvu3bPnXi^D4njQ
z*cqmD21sX|A*C}=IzvnGGfnACkj^|)N@t>UW=g6Jo@GjBfppecQaTHzv$RyY%9K`t
zv}%=<R-v>?OJ%D~X*EczS4(L%N~^V0-eXEVAocV}sRyMVEmfRtN@s&~_SsT88>O?g
zRJq2K)_}BTjg;1)v_?x+=a|wtAf0oLl+Hov94%F!Yf9&WbndxQIv1sLwN!JSDV+z>
zdFM&#Je1DUQtkPsbUsMupD(5JQ955sp$kmu0+24aKuQ;&bb*$_7n;(AAYFK&lrBW+
zLM=ruGNp?^y67S)U4+s_T8dt5N*9B4@x@ZQ7^RD~6uZQfE&=J1OQduON|$IUeyJ&4
z3eu&QO6gLRE=@_HV6Q3lg4Ek9rCyYJwN$#+l-7c@cCD1wqO?{^W$R379Z2ieNogHQ
z>$FsUnJHZc(q)%P=`xfq(^5sBDfNNW*C(Yul=`$(dATWF4$|e9OX+fyF4t1k6{d6r
zNLO4Tr7KXnLQB<Gn$ndZU3sOHu0-idE!A9QN>_n&)m2iu3Z<*GRC~24T@BLJS4-(?
zl&;oN=o(YH2Bd4Qk<v9NU8AM&wWf3}NY`E~rE5{TR!foVOzAq1uDec3*P(QsmZI02
z()A!+f4!8hN9lSk#cnXA8$i0@1}WWu(hXXQ-)KrVf^_4JQo0eP8&gs^c#|pJ1kz16
zN$Dn(Zqice&8BoSNH^asrJGT@SxaTNn9?mE-ExbRZb9i5EtTJDO1FY^>#b6{6{TCX
zRB@Xr-3HQaw@K+Xly1{f<?W_)J4m<RE~VR1x?M|EcbL*0Al-3?l<q+34lPyRX-ap3
zbmyH?x)Y^4wN!JLDcuFqU3W?8E|l)lQtjQQbT>$M-z}xPQMy}8p?gf}9+2+2M@sjg
zbdQ$8_nOkZAl-Yfl<r08UM)rLGo|}Ly6-+I-G|bBT8iFpO80|w|NT<BAEo=X6nnsw
z9sucq2c+}>N)Ko${-7y62-1TOO6ftA9!yD*;CfS957PSeQd*DFdM%YcWJ(W#^w2|6
zdI+V5v{d%6DLo9*!w*a8VU!-$Qu!mM^ax0gJR+q>P<lj56_1+IqaZ!{sFWT>=}|3J
zK4waff%Mp8QhE%f$Fx-SxG6mj(&LXy>2Z`E*HZNprt}0zPdp)|Cs2AqOEpiL(vu)P
z`J|MdMCnN_)jnlPPl5E*Q&M^grKhwMdfJqp2I=XirSvpPPirasj43?>(lgIU=^2!s
z(Ng4DQ+gJpXP=eQvnV~QrRZ~}^c+afJtw8-P<l>FvFA<cd61reUP{lS^t_hhFPPE`
zAieN{lwLsTg_INxZZM?{AZ^$nr41-;&{FA(rt~66FTN<H7g2gqOJy&a(n}z{^pccb
zLg^(fmA`CCFN5^*%Tjt7rI)o-@ro(E0@5q5Na+=nUeQwJtETiSNUy#srB_jURZCT`
znbK<@z4n@vUPI|MEmgm6O0R?T`s-499i`W`RI|~PHiEQqqm(wHv{6g7Z<x{>AieR1
zl-@w;4K0P<G^IB|dh<;wy@}GBS_;2qN^gPm)>~3~3#GTT6nWc}-UjLIx25zpN^ff^
z`i?2R1JXP1Na-Dv-qBL*T~m4&q<7zy(z__VtEKpRrt}_2@4Y9b_fUE-CB=g8o6`Fr
zz5l+H-bd+uEtP&?N*{pq!3R?M0HqJKRQ91MeF)NrA4=&%ls?o_`A4So5lA0>B&Cl~
z`bbL^ADhz0AbtF?ls-o3V=YyFVoINY^vNeu`UIs<v{dz}DSZmkr=LpcQ<Of{QuSx1
z^chH>eI}*PQ2I<uHJ_W(=OBIlxs*Of>2odBeql;qfb_)|Qu+d=FSHc;(v-dg>B}#r
z^d(APYAO7cDSZXfS6@l#E0n&{Qsiq>`WmFKzn0S1D1EJ^=r^YH4M^X7Bc*Rp`bJB!
zZ%yf2kiPv^O5dXNt(M~7nbLP4efOP|zC-D|loStsZ%W^T^!@iz`W~h4wN(0pDg6M_
z4?jrh2b6x$QrVBD^dm?={wSp%QTkC!<v*FyPaysDlazi!=_f5!{A@};gY@&yQu-OC
zpS4u^iz)pA(l5VA=@*oK(Nfi~rt~XFzy2zvUs3v1OVz)b(r+OB_M4P`L+Ljy)oe1Q
zO(1RBB&AI#ZPHTh@22!SNWcFsrQcEdT}z=qOz97h{`f;mf1vb-mcoCU(w`vx`KOfr
zMCng0MgB6Szd-uyFDd<n(qCGN{%uNsgY@^`Qu-UEzqJ(m$CUm7>7Re3^bbn^u++VG
z7XQID`;SH^|GadyW&h=9-)7hU^4;vnbOp+sfMso_(ya3>{*SE9SYfkWA)3X1#jU>z
z^cUXfFE*wAEEJ`#8eD*D@CRxo`FDAERv`6XtSHk}O8c$iaaZxUt9VouzQrmu3#xda
zN?E^E(%e<j+*Q(46~4tPGz+SvL6!1;tE9WDq`Rx6t15hpRcIDeNrx&G{Z`3vSIKZ!
z$xv1J7OT)KsFDFyD*LUH>8_ILu9B&$@GVxMSx_Yts#Nt`CCgnU%UvZ)RpDE#LbISs
z7F4P3w@S9VO18U7wyMInScPUmm29X|({B~8yNcIc#jC3DEmomfP{j*XYWuB{<F1n9
zu9Bmw@GVxMSx_Yhs)YKj;&WH=xvTh86~4tPGz+Tuph~#kD!J||x$Y{tstVs?6`BQA
za-mA3-zt806~DWRUsd5-tU|M(iXW;(`>m4au9D}jlBcThEmomfP$dtl#QLq0@2-;X
zu9B~+@GVxMSx_Y(s`R_J&>zNhJ@Xg1s}!gze2Z0R7E~$FRZ9EcUjptb0e6*vs=~Ke
zg=RsO095IJe<^fVDRfsUR8{yEtI#Z{QV3Q0-(QN{Rf^nIic}T8#VRxlsuXdR;^A}|
z2D+p~M=V%osf$POB^mfnb_N;;I@f5huPtm}kYZe}?~WCDR+^u8Vtc=*%R+fJ`8Uyb
zIo<E^r}O>4(Wf{0Q+Xa;irBJ#vM&^Ef1a<*w0Uzb52b%;mg^GIUb;%M^A5dbUFG`B
zdb+l>^IML!e$SOV&}FBU<~X!iQa5k*I%(=7=X52D?nnl$Q+;djRjfmF=?v$p7<@lm
z4udZE(H8RMI+x(EFNaZ*Rg&eQ>shyVkIQJTZ*FmZR@?|ZNc~(|!?f1>*ZVy~=u#fr
zO{4Oy<sC&?8T{)8#T6Yz-V9%wFU`&>`7^wVz;61>(B8dsd}>7f7)^>bPNu)9S;lik
z$AQd1CSSlK5ZZDY|Lxd7LkWKs+q-jHgTW5x(&Fvi+qVX5oP@q&9;=v~(x^pO%BG+!
z8I*HCzZ(v3jqAaq_|hN^EtArGoqT1GAjclQJ{DceYf_j$MEFD0Ytdy;CdD|8^M^YA
zQ18v6k0G@xiZ~6Y(lDoyR2tzlno6Ub#!_jF(|9V4b6S^5>o~1XrF0dcit*tnE!nx;
zdV0F^$Q3wW>mj~@b)HVE)AG<a^-OCf-HuMPzjwB3oH#i(t6+d@me-d#nlE<V-d#4K
zxy^Z2pi6UQINx0KlgVEjRUzQZtMeR_eoVU8J9>#NmqjKvrsnW(J#?9te&%dS-;{27
zM+fXVV_VuHscv$q4qq$ArW00zes3OadAOZ|)_7Qbb<?d;R&iEyq^Zpc?8QIt#kKSk
ZUuyB+icMWG#<_Sp+`;4dkGZ-={0~gKnzR4_

literal 0
HcmV?d00001

diff --git a/tests/data/acpi/q35/FACP.core-count2 b/tests/data/acpi/q35/FACP.core-count2
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..31fa5dd19c213034eef4eeefa6a04e61dadd8a2a 100644
GIT binary patch
literal 244
zcmZ>BbPo8!z`($~*~#D8BUr&HBEVSz2pEB4AU24G0Y(N+hD|^Y6El!tgNU*~X%LSC
z$X0-fGcm9T0LA|E|L2FOWMD92VqjR>!otAF!NBm72O<iWged~jj0!*k$y^{03>bk1
YBHITON2VDSAnpK(F*YFF1LDH~0O^Si0RR91

literal 0
HcmV?d00001

-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 82/83] hw/virtio: introduce virtio_device_should_start
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
@ 2022-11-07 22:54   ` Michael S. Tsirkin
  2022-11-07 22:47 ` [PULL v4 02/83] tests/acpi: allow SSDT changes Michael S. Tsirkin
                     ` (83 subsequent siblings)
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:54 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Alex Bennée, Raphael Norwitz, Kevin Wolf,
	Hanna Reitz, Dr. David Alan Gilbert, Stefan Hajnoczi,
	Viresh Kumar, Mathieu Poirier, qemu-block, virtio-fs

From: Alex Bennée <alex.bennee@linaro.org>

The previous fix to virtio_device_started revealed a problem in its
use by both the core and the device code. The core code should be able
to handle the device "starting" while the VM isn't running to handle
the restoration of migration state. To solve this duel use introduce a
new helper for use by the vhost-user backends who all use it to feed a
should_start variable.

We can also pick up a change vhost_user_blk_set_status while we are at
it which follows the same pattern.

Fixes: 9f6bcfd99f (hw/virtio: move vm_running check to virtio_device_started)
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Message-Id: <20221107121407.1010913-1-alex.bennee@linaro.org>
---
 include/hw/virtio/virtio.h   | 18 ++++++++++++++++++
 hw/block/vhost-user-blk.c    |  6 +-----
 hw/virtio/vhost-user-fs.c    |  2 +-
 hw/virtio/vhost-user-gpio.c  |  2 +-
 hw/virtio/vhost-user-i2c.c   |  2 +-
 hw/virtio/vhost-user-rng.c   |  2 +-
 hw/virtio/vhost-user-vsock.c |  2 +-
 hw/virtio/vhost-vsock.c      |  2 +-
 8 files changed, 25 insertions(+), 11 deletions(-)

diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index 1423dba379..141a253a2c 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -395,6 +395,24 @@ static inline bool virtio_device_started(VirtIODevice *vdev, uint8_t status)
         return vdev->started;
     }
 
+    return status & VIRTIO_CONFIG_S_DRIVER_OK;
+}
+
+/**
+ * virtio_device_should_start() - check if device startable
+ * @vdev - the VirtIO device
+ * @status - the devices status bits
+ *
+ * This is similar to virtio_device_started() but also encapsulates a
+ * check on the VM status which would prevent a device starting
+ * anyway.
+ */
+static inline bool virtio_device_should_start(VirtIODevice *vdev, uint8_t status)
+{
+    if (vdev->use_started) {
+        return vdev->started;
+    }
+
     if (!vdev->vm_running) {
         return false;
     }
diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
index 28409c90f7..16ad400889 100644
--- a/hw/block/vhost-user-blk.c
+++ b/hw/block/vhost-user-blk.c
@@ -226,14 +226,10 @@ static void vhost_user_blk_stop(VirtIODevice *vdev)
 static void vhost_user_blk_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostUserBlk *s = VHOST_USER_BLK(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_should_start(vdev, status);
     Error *local_err = NULL;
     int ret;
 
-    if (!vdev->vm_running) {
-        should_start = false;
-    }
-
     if (!s->connected) {
         return;
     }
diff --git a/hw/virtio/vhost-user-fs.c b/hw/virtio/vhost-user-fs.c
index ad0f91c607..1c40f42045 100644
--- a/hw/virtio/vhost-user-fs.c
+++ b/hw/virtio/vhost-user-fs.c
@@ -123,7 +123,7 @@ static void vuf_stop(VirtIODevice *vdev)
 static void vuf_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostUserFS *fs = VHOST_USER_FS(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_should_start(vdev, status);
 
     if (vhost_dev_is_started(&fs->vhost_dev) == should_start) {
         return;
diff --git a/hw/virtio/vhost-user-gpio.c b/hw/virtio/vhost-user-gpio.c
index 8b40fe450c..677d1c7730 100644
--- a/hw/virtio/vhost-user-gpio.c
+++ b/hw/virtio/vhost-user-gpio.c
@@ -152,7 +152,7 @@ static void vu_gpio_stop(VirtIODevice *vdev)
 static void vu_gpio_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostUserGPIO *gpio = VHOST_USER_GPIO(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_should_start(vdev, status);
 
     trace_virtio_gpio_set_status(status);
 
diff --git a/hw/virtio/vhost-user-i2c.c b/hw/virtio/vhost-user-i2c.c
index bc58b6c0d1..864eba695e 100644
--- a/hw/virtio/vhost-user-i2c.c
+++ b/hw/virtio/vhost-user-i2c.c
@@ -93,7 +93,7 @@ static void vu_i2c_stop(VirtIODevice *vdev)
 static void vu_i2c_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostUserI2C *i2c = VHOST_USER_I2C(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_should_start(vdev, status);
 
     if (vhost_dev_is_started(&i2c->vhost_dev) == should_start) {
         return;
diff --git a/hw/virtio/vhost-user-rng.c b/hw/virtio/vhost-user-rng.c
index bc1f36c5ac..8b47287875 100644
--- a/hw/virtio/vhost-user-rng.c
+++ b/hw/virtio/vhost-user-rng.c
@@ -90,7 +90,7 @@ static void vu_rng_stop(VirtIODevice *vdev)
 static void vu_rng_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostUserRNG *rng = VHOST_USER_RNG(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_should_start(vdev, status);
 
     if (vhost_dev_is_started(&rng->vhost_dev) == should_start) {
         return;
diff --git a/hw/virtio/vhost-user-vsock.c b/hw/virtio/vhost-user-vsock.c
index 7b67e29d83..9431b9792c 100644
--- a/hw/virtio/vhost-user-vsock.c
+++ b/hw/virtio/vhost-user-vsock.c
@@ -55,7 +55,7 @@ const VhostDevConfigOps vsock_ops = {
 static void vuv_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostVSockCommon *vvc = VHOST_VSOCK_COMMON(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_should_start(vdev, status);
 
     if (vhost_dev_is_started(&vvc->vhost_dev) == should_start) {
         return;
diff --git a/hw/virtio/vhost-vsock.c b/hw/virtio/vhost-vsock.c
index 7dc3c73931..aa16d584ee 100644
--- a/hw/virtio/vhost-vsock.c
+++ b/hw/virtio/vhost-vsock.c
@@ -70,7 +70,7 @@ static int vhost_vsock_set_running(VirtIODevice *vdev, int start)
 static void vhost_vsock_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostVSockCommon *vvc = VHOST_VSOCK_COMMON(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_should_start(vdev, status);
     int ret;
 
     if (vhost_dev_is_started(&vvc->vhost_dev) == should_start) {
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [Virtio-fs] [PULL v4 82/83] hw/virtio: introduce virtio_device_should_start
@ 2022-11-07 22:54   ` Michael S. Tsirkin
  0 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:54 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Alex Bennée, Raphael Norwitz, Kevin Wolf,
	Hanna Reitz, Dr. David Alan Gilbert, Stefan Hajnoczi,
	Viresh Kumar, Mathieu Poirier, qemu-block, virtio-fs

From: Alex Bennée <alex.bennee@linaro.org>

The previous fix to virtio_device_started revealed a problem in its
use by both the core and the device code. The core code should be able
to handle the device "starting" while the VM isn't running to handle
the restoration of migration state. To solve this duel use introduce a
new helper for use by the vhost-user backends who all use it to feed a
should_start variable.

We can also pick up a change vhost_user_blk_set_status while we are at
it which follows the same pattern.

Fixes: 9f6bcfd99f (hw/virtio: move vm_running check to virtio_device_started)
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Message-Id: <20221107121407.1010913-1-alex.bennee@linaro.org>
---
 include/hw/virtio/virtio.h   | 18 ++++++++++++++++++
 hw/block/vhost-user-blk.c    |  6 +-----
 hw/virtio/vhost-user-fs.c    |  2 +-
 hw/virtio/vhost-user-gpio.c  |  2 +-
 hw/virtio/vhost-user-i2c.c   |  2 +-
 hw/virtio/vhost-user-rng.c   |  2 +-
 hw/virtio/vhost-user-vsock.c |  2 +-
 hw/virtio/vhost-vsock.c      |  2 +-
 8 files changed, 25 insertions(+), 11 deletions(-)

diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index 1423dba379..141a253a2c 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -395,6 +395,24 @@ static inline bool virtio_device_started(VirtIODevice *vdev, uint8_t status)
         return vdev->started;
     }
 
+    return status & VIRTIO_CONFIG_S_DRIVER_OK;
+}
+
+/**
+ * virtio_device_should_start() - check if device startable
+ * @vdev - the VirtIO device
+ * @status - the devices status bits
+ *
+ * This is similar to virtio_device_started() but also encapsulates a
+ * check on the VM status which would prevent a device starting
+ * anyway.
+ */
+static inline bool virtio_device_should_start(VirtIODevice *vdev, uint8_t status)
+{
+    if (vdev->use_started) {
+        return vdev->started;
+    }
+
     if (!vdev->vm_running) {
         return false;
     }
diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
index 28409c90f7..16ad400889 100644
--- a/hw/block/vhost-user-blk.c
+++ b/hw/block/vhost-user-blk.c
@@ -226,14 +226,10 @@ static void vhost_user_blk_stop(VirtIODevice *vdev)
 static void vhost_user_blk_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostUserBlk *s = VHOST_USER_BLK(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_should_start(vdev, status);
     Error *local_err = NULL;
     int ret;
 
-    if (!vdev->vm_running) {
-        should_start = false;
-    }
-
     if (!s->connected) {
         return;
     }
diff --git a/hw/virtio/vhost-user-fs.c b/hw/virtio/vhost-user-fs.c
index ad0f91c607..1c40f42045 100644
--- a/hw/virtio/vhost-user-fs.c
+++ b/hw/virtio/vhost-user-fs.c
@@ -123,7 +123,7 @@ static void vuf_stop(VirtIODevice *vdev)
 static void vuf_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostUserFS *fs = VHOST_USER_FS(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_should_start(vdev, status);
 
     if (vhost_dev_is_started(&fs->vhost_dev) == should_start) {
         return;
diff --git a/hw/virtio/vhost-user-gpio.c b/hw/virtio/vhost-user-gpio.c
index 8b40fe450c..677d1c7730 100644
--- a/hw/virtio/vhost-user-gpio.c
+++ b/hw/virtio/vhost-user-gpio.c
@@ -152,7 +152,7 @@ static void vu_gpio_stop(VirtIODevice *vdev)
 static void vu_gpio_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostUserGPIO *gpio = VHOST_USER_GPIO(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_should_start(vdev, status);
 
     trace_virtio_gpio_set_status(status);
 
diff --git a/hw/virtio/vhost-user-i2c.c b/hw/virtio/vhost-user-i2c.c
index bc58b6c0d1..864eba695e 100644
--- a/hw/virtio/vhost-user-i2c.c
+++ b/hw/virtio/vhost-user-i2c.c
@@ -93,7 +93,7 @@ static void vu_i2c_stop(VirtIODevice *vdev)
 static void vu_i2c_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostUserI2C *i2c = VHOST_USER_I2C(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_should_start(vdev, status);
 
     if (vhost_dev_is_started(&i2c->vhost_dev) == should_start) {
         return;
diff --git a/hw/virtio/vhost-user-rng.c b/hw/virtio/vhost-user-rng.c
index bc1f36c5ac..8b47287875 100644
--- a/hw/virtio/vhost-user-rng.c
+++ b/hw/virtio/vhost-user-rng.c
@@ -90,7 +90,7 @@ static void vu_rng_stop(VirtIODevice *vdev)
 static void vu_rng_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostUserRNG *rng = VHOST_USER_RNG(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_should_start(vdev, status);
 
     if (vhost_dev_is_started(&rng->vhost_dev) == should_start) {
         return;
diff --git a/hw/virtio/vhost-user-vsock.c b/hw/virtio/vhost-user-vsock.c
index 7b67e29d83..9431b9792c 100644
--- a/hw/virtio/vhost-user-vsock.c
+++ b/hw/virtio/vhost-user-vsock.c
@@ -55,7 +55,7 @@ const VhostDevConfigOps vsock_ops = {
 static void vuv_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostVSockCommon *vvc = VHOST_VSOCK_COMMON(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_should_start(vdev, status);
 
     if (vhost_dev_is_started(&vvc->vhost_dev) == should_start) {
         return;
diff --git a/hw/virtio/vhost-vsock.c b/hw/virtio/vhost-vsock.c
index 7dc3c73931..aa16d584ee 100644
--- a/hw/virtio/vhost-vsock.c
+++ b/hw/virtio/vhost-vsock.c
@@ -70,7 +70,7 @@ static int vhost_vsock_set_running(VirtIODevice *vdev, int start)
 static void vhost_vsock_set_status(VirtIODevice *vdev, uint8_t status)
 {
     VHostVSockCommon *vvc = VHOST_VSOCK_COMMON(vdev);
-    bool should_start = virtio_device_started(vdev, status);
+    bool should_start = virtio_device_should_start(vdev, status);
     int ret;
 
     if (vhost_dev_is_started(&vvc->vhost_dev) == should_start) {
-- 
MST


^ permalink raw reply related	[flat|nested] 151+ messages in thread

* [PULL v4 83/83] checkpatch: better pattern for inline comments
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (81 preceding siblings ...)
  2022-11-07 22:54   ` [Virtio-fs] " Michael S. Tsirkin
@ 2022-11-07 22:54 ` Michael S. Tsirkin
  2022-11-08  6:23 ` [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
  2022-11-15 14:01 ` Philippe Mathieu-Daudé
  84 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-07 22:54 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Philippe Mathieu-Daudé,
	Marc-André Lureau, Laurent Vivier, Richard Henderson,
	Paolo Bonzini

checkpatch is unhappy about this line:

    WARNING: Block comments use a leading /* on a separate line
    #50: FILE: hw/acpi/nvdimm.c:1074:
    +                   aml_equal(aml_sizeof(pckg), aml_int(1)) /* 1 element? */));

but there's nothing wrong with it - the check is just too simplistic. It
will also miss lines which mix inline and block comments.

Instead, let's strip all inline comments from a line and then check for block
comments.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 scripts/checkpatch.pl | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index e3e3b43076..bc7d4780ec 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -1681,8 +1681,10 @@ sub process {
 # Block comment styles
 
 		# Block comments use /* on a line of its own
-		if ($rawline !~ m@^\+.*/\*.*\*/[ \t)}]*$@ &&	#inline /*...*/
-		    $rawline =~ m@^\+.*/\*\*?+[ \t]*[^ \t]@) { # /* or /** non-blank
+		my $commentline = $rawline;
+		while ($commentline =~ s@^(\+.*)/\*.*\*/@$1@o) { # remove inline #inline /*...*/
+		}
+		if ($commentline =~ m@^\+.*/\*\*?+[ \t]*[^ \t]@) { # /* or /** non-blank
 			WARN("Block comments use a leading /* on a separate line\n" . $herecurr);
 		}
 
-- 
MST



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* Re: [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (82 preceding siblings ...)
  2022-11-07 22:54 ` [PULL v4 83/83] checkpatch: better pattern for inline comments Michael S. Tsirkin
@ 2022-11-08  6:23 ` Michael S. Tsirkin
  2022-11-08 13:47   ` Stefan Hajnoczi
  2022-11-15 14:01 ` Philippe Mathieu-Daudé
  84 siblings, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-08  6:23 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, stefanha

On Mon, Nov 07, 2022 at 05:47:16PM -0500, Michael S. Tsirkin wrote:
> Changes from v3:
>     Applied and squashed fix by Ani for modular build breakage
>     Reordered Julia's patches to avoid bisect breakage
>     Checkpatch fixes for Jason's patches
>     Added Alex's patch to partially address virtio is_started mess
>     There is a bigger issue found by Christian A. Ehrhardt, that
>     still needs work
>     checkpatch change to avoid breaking on Jason's patches (to make ci pass)
> 
> Changes from v2:
>     Fixed a bug in error handling in vhost: Change the sequence of device start.
>         Contributor placed on watchlist ;)
>     Dropped virtio: re-order vm_running and use_started checks
>         Due to failures detected by gitlab.
>         We'll have to fix it differently.
>     Updated expected files for core-count test to fix bisect.
> 
> Changes from v1
>     Applied and squashed fixes by Igor, Lei He, Hesham Almatary for
>     bugs that tripped up the pipeline.
>     Updated expected files for core-count test.
> 
> 
> 
> The following changes since commit a11f65ec1b8adcb012b89c92819cbda4dc25aaf1:
> 
>   Merge tag 'block-pull-request' of https://gitlab.com/stefanha/qemu into staging (2022-11-01 13:49:33 -0400)
> 
> are available in the Git repository at:
> 
>   https://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git tags/for_upstream
> 
> for you to fetch changes up to 1ef47f40dce3d5b176ddf76d57b5bfa2efb0b3c6:

 5e75ffd664258d3d2fd3d27e92e2748024f53bca now - I found and fixed a typo in a comment in checkpatch and re-pushed


> 
>   checkpatch: better pattern for inline comments (2022-11-07 14:25:51 -0500)
> 
> ----------------------------------------------------------------
> pci,pc,virtio: features, tests, fixes, cleanups
> 
> lots of acpi rework
> first version of biosbits infrastructure
> ASID support in vhost-vdpa
> core_count2 support in smbios
> PCIe DOE emulation
> virtio vq reset
> HMAT support
> part of infrastructure for viommu support in vhost-vdpa
> VTD PASID support
> fixes, tests all over the place
> 
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> 
> ----------------------------------------------------------------
> Akihiko Odaki (1):
>       msix: Assert that specified vector is in range
> 
> Alex Bennée (1):
>       hw/virtio: introduce virtio_device_should_start
> 
> Ani Sinha (7):
>       hw/i386/e820: remove legacy reserved entries for e820
>       acpi/tests/avocado/bits: initial commit of test scripts that are run by biosbits
>       acpi/tests/avocado/bits: disable acpi PSS tests that are failing in biosbits
>       acpi/tests/avocado/bits: add biosbits config file for running bios tests
>       acpi/tests/avocado/bits: add acpi and smbios avocado tests that uses biosbits
>       acpi/tests/avocado/bits/doc: add a doc file to describe the acpi bits test
>       MAINTAINERS: add myself as the maintainer for acpi biosbits avocado tests
> 
> Bernhard Beschow (3):
>       hw/i386/acpi-build: Remove unused struct
>       hw/i386/acpi-build: Resolve redundant attribute
>       hw/i386/acpi-build: Resolve north rather than south bridges
> 
> Brice Goglin (4):
>       hmat acpi: Don't require initiator value in -numa
>       tests: acpi: add and whitelist *.hmat-noinitiator expected blobs
>       tests: acpi: q35: add test for hmat nodes without initiators
>       tests: acpi: q35: update expected blobs *.hmat-noinitiators expected HMAT:
> 
> Christian A. Ehrhardt (1):
>       hw/acpi/erst.c: Fix memory handling issues
> 
> Cindy Lu (1):
>       vfio: move implement of vfio_get_xlat_addr() to memory.c
> 
> David Daney (1):
>       virtio-rng-pci: Allow setting nvectors, so we can use MSI-X
> 
> Eric Auger (1):
>       hw/virtio/virtio-iommu-pci: Enforce the device is plugged on the root bus
> 
> Gregory Price (1):
>       hw/i386/pc.c: CXL Fixed Memory Window should not reserve e820 in bios
> 
> Hesham Almatary (3):
>       tests: Add HMAT AArch64/virt empty table files
>       tests: acpi: aarch64/virt: add a test for hmat nodes with no initiators
>       tests: virt: Update expected *.acpihmatvirt tables
> 
> Huai-Cheng Kuo (3):
>       hw/pci: PCIe Data Object Exchange emulation
>       hw/cxl/cdat: CXL CDAT Data Object Exchange implementation
>       hw/mem/cxl-type3: Add CXL CDAT Data Object Exchange
> 
> Igor Mammedov (11):
>       acpi: pc: vga: use AcpiDevAmlIf interface to build VGA device descriptors
>       tests: acpi: whitelist DSDT before generating PCI-ISA bridge AML automatically
>       acpi: pc/q35: drop ad-hoc PCI-ISA bridge AML routines and let bus ennumeration generate AML
>       tests: acpi: update expected DSDT after ISA bridge is moved directly under PCI host bridge
>       tests: acpi: whitelist DSDT before generating ICH9_SMB AML automatically
>       acpi: add get_dev_aml_func() helper
>       acpi: enumerate SMB bridge automatically along with other PCI devices
>       tests: acpi: update expected blobs
>       tests: acpi: pc/q35 whitelist DSDT before \_GPE cleanup
>       acpi: pc/35: sanitize _GPE declaration order
>       tests: acpi: update expected blobs
> 
> Jason Wang (4):
>       intel-iommu: don't warn guest errors when getting rid2pasid entry
>       intel-iommu: drop VTDBus
>       intel-iommu: convert VTD_PE_GET_FPD_ERR() to be a function
>       intel-iommu: PASID support
> 
> Jonathan Cameron (2):
>       hw/mem/cxl-type3: Add MSIX support
>       hw/pci-bridge/cxl-upstream: Add a CDAT table access DOE
> 
> Julia Suvorova (5):
>       hw/smbios: add core_count2 to smbios table type 4
>       bios-tables-test: teach test to use smbios 3.0 tables
>       tests/acpi: allow changes for core_count2 test
>       bios-tables-test: add test for number of cores > 255
>       tests/acpi: update tables for new core count test
> 
> Kangjie Xu (10):
>       virtio: introduce virtio_queue_enable()
>       virtio: core: vq reset feature negotation support
>       virtio-pci: support queue enable
>       vhost: expose vhost_virtqueue_start()
>       vhost: expose vhost_virtqueue_stop()
>       vhost-net: vhost-kernel: introduce vhost_net_virtqueue_reset()
>       vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
>       virtio-net: introduce flush_or_purge_queued_packets()
>       virtio-net: support queue_enable
>       vhost: vhost-kernel: enable vq reset feature
> 
> Lei He (4):
>       virtio-crypto: Support asynchronous mode
>       crypto: Support DER encodings
>       crypto: Support export akcipher to pkcs8
>       cryptodev: Add a lkcf-backend for cryptodev
> 
> Markus Armbruster (1):
>       MAINTAINERS: Add qapi/virtio.json to section "virtio"
> 
> Michael S. Tsirkin (1):
>       checkpatch: better pattern for inline comments
> 
> Miguel Luis (4):
>       tests/acpi: virt: allow acpi MADT and FADT changes
>       acpi: fadt: support revision 6.0 of the ACPI specification
>       acpi: arm/virt: madt: bump to revision 4 accordingly to ACPI 6.0 Errata A
>       tests/acpi: virt: update ACPI MADT and FADT binaries
> 
> Robert Hoo (5):
>       tests/acpi: allow SSDT changes
>       acpi/ssdt: Fix aml_or() and aml_and() in if clause
>       acpi/nvdimm: define macro for NVDIMM Device _DSM
>       acpi/nvdimm: Implement ACPI NVDIMM Label Methods
>       test/acpi/bios-tables-test: SSDT: update golden master binaries
> 
> Xiang Chen (1):
>       hw/arm/virt: Enable HMAT on arm virt machine
> 
> Xuan Zhuo (5):
>       virtio: introduce __virtio_queue_reset()
>       virtio: introduce virtio_queue_reset()
>       virtio-pci: support queue reset
>       virtio-net: support queue reset
>       virtio-net: enable vq reset feature
> 
> Yajun Wu (3):
>       vhost-user: Fix out of order vring host notification handling
>       vhost: Change the sequence of device start
>       vhost-user: Support vhost_dev_start
> 
>  tests/avocado/acpi-bits/bits-config/bits-cfg.txt |   18 +
>  qapi/qom.json                                    |    2 +
>  crypto/der.h                                     |  211 +-
>  crypto/rsakey.h                                  |   11 +-
>  hw/display/vga_int.h                             |    2 +
>  hw/i386/e820_memory_layout.h                     |    8 -
>  hw/i386/fw_cfg.h                                 |    1 -
>  hw/i386/intel_iommu_internal.h                   |   16 +-
>  hw/smbios/smbios_build.h                         |    9 +-
>  include/crypto/akcipher.h                        |   21 +
>  include/exec/memory.h                            |    4 +
>  include/hw/acpi/acpi_aml_interface.h             |   13 +-
>  include/hw/cxl/cxl_cdat.h                        |  166 ++
>  include/hw/cxl/cxl_component.h                   |    7 +
>  include/hw/cxl/cxl_device.h                      |    3 +
>  include/hw/cxl/cxl_pci.h                         |    1 +
>  include/hw/firmware/smbios.h                     |   12 +
>  include/hw/i386/intel_iommu.h                    |   18 +-
>  include/hw/pci/msix.h                            |    4 +-
>  include/hw/pci/pci_bus.h                         |    2 +
>  include/hw/pci/pci_ids.h                         |    3 +
>  include/hw/pci/pcie.h                            |    1 +
>  include/hw/pci/pcie_doe.h                        |  123 ++
>  include/hw/pci/pcie_regs.h                       |    4 +
>  include/hw/virtio/vhost.h                        |    5 +
>  include/hw/virtio/virtio-pci.h                   |    5 +
>  include/hw/virtio/virtio.h                       |   26 +-
>  include/net/vhost_net.h                          |    4 +
>  include/sysemu/cryptodev.h                       |   61 +-
>  backends/cryptodev-builtin.c                     |   69 +-
>  backends/cryptodev-lkcf.c                        |  645 ++++++
>  backends/cryptodev-vhost-user.c                  |   53 +-
>  backends/cryptodev.c                             |   44 +-
>  crypto/akcipher.c                                |   18 +
>  crypto/der.c                                     |  313 ++-
>  crypto/rsakey.c                                  |   42 +
>  hw/acpi/aml-build-stub.c                         |   10 +
>  hw/acpi/aml-build.c                              |   13 +-
>  hw/acpi/erst.c                                   |    6 +-
>  hw/acpi/nvdimm.c                                 |  106 +-
>  hw/arm/virt-acpi-build.c                         |   33 +-
>  hw/block/vhost-user-blk.c                        |   24 +-
>  hw/core/machine.c                                |    8 +-
>  hw/cxl/cxl-cdat.c                                |  224 ++
>  hw/display/acpi-vga-stub.c                       |    7 +
>  hw/display/acpi-vga.c                            |   26 +
>  hw/display/vga-pci.c                             |    4 +
>  hw/i386/acpi-build.c                             |  203 +-
>  hw/i386/e820_memory_layout.c                     |   20 +-
>  hw/i386/fw_cfg.c                                 |    3 -
>  hw/i386/intel_iommu.c                            |  694 +++---
>  hw/i386/microvm.c                                |    2 -
>  hw/i386/pc.c                                     |    2 -
>  hw/isa/lpc_ich9.c                                |   23 +
>  hw/isa/piix3.c                                   |   17 +-
>  hw/mem/cxl_type3.c                               |  264 +++
>  hw/net/e1000e.c                                  |   15 +-
>  hw/net/rocker/rocker.c                           |   23 +-
>  hw/net/vhost_net-stub.c                          |   12 +
>  hw/net/vhost_net.c                               |   90 +-
>  hw/net/virtio-net.c                              |   57 +-
>  hw/net/vmxnet3.c                                 |   27 +-
>  hw/nvme/ctrl.c                                   |    5 +-
>  hw/pci-bridge/cxl_upstream.c                     |  195 +-
>  hw/pci/msix.c                                    |   24 +-
>  hw/pci/pcie_doe.c                                |  367 ++++
>  hw/rdma/vmw/pvrdma_main.c                        |    7 +-
>  hw/remote/vfio-user-obj.c                        |    9 +-
>  hw/smbios/smbios.c                               |   19 +-
>  hw/vfio/common.c                                 |   66 +-
>  hw/virtio/vhost-user-fs.c                        |    2 +-
>  hw/virtio/vhost-user-gpio.c                      |    2 +-
>  hw/virtio/vhost-user-i2c.c                       |    2 +-
>  hw/virtio/vhost-user-rng.c                       |    2 +-
>  hw/virtio/vhost-user-vsock.c                     |    2 +-
>  hw/virtio/vhost-user.c                           |   79 +-
>  hw/virtio/vhost-vsock.c                          |    2 +-
>  hw/virtio/vhost.c                                |   16 +-
>  hw/virtio/virtio-crypto.c                        |  339 +--
>  hw/virtio/virtio-iommu-pci.c                     |   12 +-
>  hw/virtio/virtio-pci.c                           |   83 +-
>  hw/virtio/virtio-rng-pci.c                       |   14 +
>  hw/virtio/virtio.c                               |   62 +-
>  softmmu/memory.c                                 |   72 +
>  tests/qtest/bios-tables-test.c                   |  267 ++-
>  tests/unit/test-crypto-der.c                     |  126 +-
>  MAINTAINERS                                      |   15 +
>  backends/meson.build                             |    3 +
>  docs/devel/acpi-bits.rst                         |  145 ++
>  docs/devel/index-build.rst                       |    1 +
>  hw/arm/Kconfig                                   |    1 +
>  hw/cxl/meson.build                               |    1 +
>  hw/display/meson.build                           |   17 +
>  hw/i386/trace-events                             |    2 +
>  hw/pci/meson.build                               |    1 +
>  scripts/checkpatch.pl                            |    6 +-
>  tests/avocado/acpi-bits.py                       |  396 ++++
>  tests/avocado/acpi-bits/bits-tests/smbios.py2    | 2430 ++++++++++++++++++++++
>  tests/avocado/acpi-bits/bits-tests/testacpi.py2  |  283 +++
>  tests/avocado/acpi-bits/bits-tests/testcpuid.py2 |   83 +
>  tests/data/acpi/pc/DSDT                          |  Bin 6422 -> 6501 bytes
>  tests/data/acpi/pc/DSDT.acpierst                 |  Bin 6382 -> 6461 bytes
>  tests/data/acpi/pc/DSDT.acpihmat                 |  Bin 7747 -> 7826 bytes
>  tests/data/acpi/pc/DSDT.bridge                   |  Bin 9496 -> 9575 bytes
>  tests/data/acpi/pc/DSDT.cphp                     |  Bin 6886 -> 6965 bytes
>  tests/data/acpi/pc/DSDT.dimmpxm                  |  Bin 8076 -> 8155 bytes
>  tests/data/acpi/pc/DSDT.hpbridge                 |  Bin 6382 -> 6461 bytes
>  tests/data/acpi/pc/DSDT.hpbrroot                 |  Bin 3069 -> 3107 bytes
>  tests/data/acpi/pc/DSDT.ipmikcs                  |  Bin 6494 -> 6573 bytes
>  tests/data/acpi/pc/DSDT.memhp                    |  Bin 7781 -> 7860 bytes
>  tests/data/acpi/pc/DSDT.nohpet                   |  Bin 6280 -> 6359 bytes
>  tests/data/acpi/pc/DSDT.numamem                  |  Bin 6428 -> 6507 bytes
>  tests/data/acpi/pc/DSDT.roothp                   |  Bin 6656 -> 6699 bytes
>  tests/data/acpi/pc/SSDT.dimmpxm                  |  Bin 734 -> 1815 bytes
>  tests/data/acpi/q35/APIC.acpihmat-noinitiator    |  Bin 0 -> 144 bytes
>  tests/data/acpi/q35/APIC.core-count2             |  Bin 0 -> 2478 bytes
>  tests/data/acpi/q35/DSDT                         |  Bin 8320 -> 8412 bytes
>  tests/data/acpi/q35/DSDT.acpierst                |  Bin 8337 -> 8429 bytes
>  tests/data/acpi/q35/DSDT.acpihmat                |  Bin 9645 -> 9737 bytes
>  tests/data/acpi/q35/DSDT.acpihmat-noinitiator    |  Bin 0 -> 8691 bytes
>  tests/data/acpi/q35/DSDT.applesmc                |  Bin 8366 -> 8458 bytes
>  tests/data/acpi/q35/DSDT.bridge                  |  Bin 11449 -> 11541 bytes
>  tests/data/acpi/q35/DSDT.core-count2             |  Bin 0 -> 32552 bytes
>  tests/data/acpi/q35/DSDT.cphp                    |  Bin 8784 -> 8876 bytes
>  tests/data/acpi/q35/DSDT.cxl                     |  Bin 9646 -> 9738 bytes
>  tests/data/acpi/q35/DSDT.dimmpxm                 |  Bin 9974 -> 10066 bytes
>  tests/data/acpi/q35/DSDT.ipmibt                  |  Bin 8395 -> 8487 bytes
>  tests/data/acpi/q35/DSDT.ipmismbus               |  Bin 8409 -> 8500 bytes
>  tests/data/acpi/q35/DSDT.ivrs                    |  Bin 8337 -> 8429 bytes
>  tests/data/acpi/q35/DSDT.memhp                   |  Bin 9679 -> 9771 bytes
>  tests/data/acpi/q35/DSDT.mmio64                  |  Bin 9450 -> 9542 bytes
>  tests/data/acpi/q35/DSDT.multi-bridge            |  Bin 8640 -> 8732 bytes
>  tests/data/acpi/q35/DSDT.nohpet                  |  Bin 8178 -> 8270 bytes
>  tests/data/acpi/q35/DSDT.numamem                 |  Bin 8326 -> 8418 bytes
>  tests/data/acpi/q35/DSDT.pvpanic-isa             |  Bin 8421 -> 8513 bytes
>  tests/data/acpi/q35/DSDT.tis.tpm12               |  Bin 8926 -> 9018 bytes
>  tests/data/acpi/q35/DSDT.tis.tpm2                |  Bin 8952 -> 9044 bytes
>  tests/data/acpi/q35/DSDT.viot                    |  Bin 9429 -> 9521 bytes
>  tests/data/acpi/q35/DSDT.xapic                   |  Bin 35683 -> 35775 bytes
>  tests/data/acpi/q35/FACP.core-count2             |  Bin 0 -> 244 bytes
>  tests/data/acpi/q35/HMAT.acpihmat-noinitiator    |  Bin 0 -> 288 bytes
>  tests/data/acpi/q35/SRAT.acpihmat-noinitiator    |  Bin 0 -> 312 bytes
>  tests/data/acpi/q35/SSDT.dimmpxm                 |  Bin 734 -> 1815 bytes
>  tests/data/acpi/virt/APIC                        |  Bin 168 -> 172 bytes
>  tests/data/acpi/virt/APIC.acpihmatvirt           |  Bin 0 -> 412 bytes
>  tests/data/acpi/virt/APIC.memhp                  |  Bin 168 -> 172 bytes
>  tests/data/acpi/virt/APIC.numamem                |  Bin 168 -> 172 bytes
>  tests/data/acpi/virt/DSDT.acpihmatvirt           |  Bin 0 -> 5282 bytes
>  tests/data/acpi/virt/FACP                        |  Bin 268 -> 276 bytes
>  tests/data/acpi/virt/FACP.memhp                  |  Bin 268 -> 276 bytes
>  tests/data/acpi/virt/FACP.numamem                |  Bin 268 -> 276 bytes
>  tests/data/acpi/virt/HMAT.acpihmatvirt           |  Bin 0 -> 288 bytes
>  tests/data/acpi/virt/PPTT.acpihmatvirt           |  Bin 0 -> 196 bytes
>  tests/data/acpi/virt/SRAT.acpihmatvirt           |  Bin 0 -> 240 bytes
>  tests/data/acpi/virt/SSDT.memhp                  |  Bin 736 -> 1817 bytes
>  155 files changed, 7996 insertions(+), 1020 deletions(-)
>  create mode 100644 tests/avocado/acpi-bits/bits-config/bits-cfg.txt
>  create mode 100644 include/hw/cxl/cxl_cdat.h
>  create mode 100644 include/hw/pci/pcie_doe.h
>  create mode 100644 backends/cryptodev-lkcf.c
>  create mode 100644 hw/cxl/cxl-cdat.c
>  create mode 100644 hw/display/acpi-vga-stub.c
>  create mode 100644 hw/display/acpi-vga.c
>  create mode 100644 hw/pci/pcie_doe.c
>  create mode 100644 docs/devel/acpi-bits.rst
>  create mode 100644 tests/avocado/acpi-bits.py
>  create mode 100644 tests/avocado/acpi-bits/bits-tests/smbios.py2
>  create mode 100644 tests/avocado/acpi-bits/bits-tests/testacpi.py2
>  create mode 100644 tests/avocado/acpi-bits/bits-tests/testcpuid.py2
>  create mode 100644 tests/data/acpi/q35/APIC.acpihmat-noinitiator
>  create mode 100644 tests/data/acpi/q35/APIC.core-count2
>  create mode 100644 tests/data/acpi/q35/DSDT.acpihmat-noinitiator
>  create mode 100644 tests/data/acpi/q35/DSDT.core-count2
>  create mode 100644 tests/data/acpi/q35/FACP.core-count2
>  create mode 100644 tests/data/acpi/q35/HMAT.acpihmat-noinitiator
>  create mode 100644 tests/data/acpi/q35/SRAT.acpihmat-noinitiator
>  create mode 100644 tests/data/acpi/virt/APIC.acpihmatvirt
>  create mode 100644 tests/data/acpi/virt/DSDT.acpihmatvirt
>  create mode 100644 tests/data/acpi/virt/HMAT.acpihmatvirt
>  create mode 100644 tests/data/acpi/virt/PPTT.acpihmatvirt
>  create mode 100644 tests/data/acpi/virt/SRAT.acpihmatvirt



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 45/83] tests: acpi: whitelist DSDT before generating PCI-ISA bridge AML automatically
  2022-11-07 22:51 ` [PULL v4 45/83] tests: acpi: whitelist DSDT before generating PCI-ISA bridge AML automatically Michael S. Tsirkin
@ 2022-11-08 13:36   ` Igor Mammedov
  2022-11-08 13:39     ` Ani Sinha
  2022-11-08 13:49     ` Michael S. Tsirkin
  0 siblings, 2 replies; 151+ messages in thread
From: Igor Mammedov @ 2022-11-08 13:36 UTC (permalink / raw)
  To: Michael S. Tsirkin; +Cc: qemu-devel, Peter Maydell, Ani Sinha

On Mon, 7 Nov 2022 17:51:11 -0500
"Michael S. Tsirkin" <mst@redhat.com> wrote:

> From: Igor Mammedov <imammedo@redhat.com>
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> Message-Id: <20221017102146.2254096-3-imammedo@redhat.com>
> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> ---
>  tests/qtest/bios-tables-test-allowed-diff.h | 34 +++++++++++++++++++++
>  1 file changed, 34 insertions(+)
> 
> diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
> index dfb8523c8b..570b17478e 100644
> --- a/tests/qtest/bios-tables-test-allowed-diff.h
> +++ b/tests/qtest/bios-tables-test-allowed-diff.h
> @@ -1 +1,35 @@
>  /* List of comma-separated changed AML files to ignore */
> +"tests/data/acpi/pc/DSDT",
> +"tests/data/acpi/pc/DSDT.acpierst",
> +"tests/data/acpi/pc/DSDT.acpihmat",
> +"tests/data/acpi/pc/DSDT.bridge",
> +"tests/data/acpi/pc/DSDT.cphp",
> +"tests/data/acpi/pc/DSDT.dimmpxm",
> +"tests/data/acpi/pc/DSDT.hpbridge",
> +"tests/data/acpi/pc/DSDT.hpbrroot",
> +"tests/data/acpi/pc/DSDT.ipmikcs",
> +"tests/data/acpi/pc/DSDT.memhp",
> +"tests/data/acpi/pc/DSDT.nohpet",
> +"tests/data/acpi/pc/DSDT.numamem",
> +"tests/data/acpi/pc/DSDT.roothp",
> +"tests/data/acpi/q35/DSDT",
> +"tests/data/acpi/q35/DSDT.acpierst",
> +"tests/data/acpi/q35/DSDT.acpihmat",
> +"tests/data/acpi/q35/DSDT.applesmc",
> +"tests/data/acpi/q35/DSDT.bridge",
> +"tests/data/acpi/q35/DSDT.cphp",
> +"tests/data/acpi/q35/DSDT.cxl",
> +"tests/data/acpi/q35/DSDT.dimmpxm",
> +"tests/data/acpi/q35/DSDT.ipmibt",
> +"tests/data/acpi/q35/DSDT.ipmismbus",
> +"tests/data/acpi/q35/DSDT.ivrs",
> +"tests/data/acpi/q35/DSDT.memhp",
> +"tests/data/acpi/q35/DSDT.mmio64",
> +"tests/data/acpi/q35/DSDT.multi-bridge",
> +"tests/data/acpi/q35/DSDT.nohpet",
> +"tests/data/acpi/q35/DSDT.numamem",
> +"tests/data/acpi/q35/DSDT.pvpanic-isa",
> +"tests/data/acpi/q35/DSDT.tis.tpm12",
> +"tests/data/acpi/q35/DSDT.tis.tpm2",
> +"tests/data/acpi/q35/DSDT.viot",
> +"tests/data/acpi/q35/DSDT.xapic",

still missing DSDT.count2 table, likely in other updates (as well)
which should break bisection if not whole pull request.

I'll prep a tree based on your pull req, with fixups
for you to pull from.



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 45/83] tests: acpi: whitelist DSDT before generating PCI-ISA bridge AML automatically
  2022-11-08 13:36   ` Igor Mammedov
@ 2022-11-08 13:39     ` Ani Sinha
  2022-11-08 14:06       ` Ani Sinha
  2022-11-08 13:49     ` Michael S. Tsirkin
  1 sibling, 1 reply; 151+ messages in thread
From: Ani Sinha @ 2022-11-08 13:39 UTC (permalink / raw)
  To: Igor Mammedov; +Cc: Michael S. Tsirkin, qemu-devel, Peter Maydell

On Tue, Nov 8, 2022 at 7:06 PM Igor Mammedov <imammedo@redhat.com> wrote:
>
> On Mon, 7 Nov 2022 17:51:11 -0500
> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>
> > From: Igor Mammedov <imammedo@redhat.com>
> >
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > Message-Id: <20221017102146.2254096-3-imammedo@redhat.com>
> > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > ---
> >  tests/qtest/bios-tables-test-allowed-diff.h | 34 +++++++++++++++++++++
> >  1 file changed, 34 insertions(+)
> >
> > diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
> > index dfb8523c8b..570b17478e 100644
> > --- a/tests/qtest/bios-tables-test-allowed-diff.h
> > +++ b/tests/qtest/bios-tables-test-allowed-diff.h
> > @@ -1 +1,35 @@
> >  /* List of comma-separated changed AML files to ignore */
> > +"tests/data/acpi/pc/DSDT",
> > +"tests/data/acpi/pc/DSDT.acpierst",
> > +"tests/data/acpi/pc/DSDT.acpihmat",
> > +"tests/data/acpi/pc/DSDT.bridge",
> > +"tests/data/acpi/pc/DSDT.cphp",
> > +"tests/data/acpi/pc/DSDT.dimmpxm",
> > +"tests/data/acpi/pc/DSDT.hpbridge",
> > +"tests/data/acpi/pc/DSDT.hpbrroot",
> > +"tests/data/acpi/pc/DSDT.ipmikcs",
> > +"tests/data/acpi/pc/DSDT.memhp",
> > +"tests/data/acpi/pc/DSDT.nohpet",
> > +"tests/data/acpi/pc/DSDT.numamem",
> > +"tests/data/acpi/pc/DSDT.roothp",
> > +"tests/data/acpi/q35/DSDT",
> > +"tests/data/acpi/q35/DSDT.acpierst",
> > +"tests/data/acpi/q35/DSDT.acpihmat",
> > +"tests/data/acpi/q35/DSDT.applesmc",
> > +"tests/data/acpi/q35/DSDT.bridge",
> > +"tests/data/acpi/q35/DSDT.cphp",
> > +"tests/data/acpi/q35/DSDT.cxl",
> > +"tests/data/acpi/q35/DSDT.dimmpxm",
> > +"tests/data/acpi/q35/DSDT.ipmibt",
> > +"tests/data/acpi/q35/DSDT.ipmismbus",
> > +"tests/data/acpi/q35/DSDT.ivrs",
> > +"tests/data/acpi/q35/DSDT.memhp",
> > +"tests/data/acpi/q35/DSDT.mmio64",
> > +"tests/data/acpi/q35/DSDT.multi-bridge",
> > +"tests/data/acpi/q35/DSDT.nohpet",
> > +"tests/data/acpi/q35/DSDT.numamem",
> > +"tests/data/acpi/q35/DSDT.pvpanic-isa",
> > +"tests/data/acpi/q35/DSDT.tis.tpm12",
> > +"tests/data/acpi/q35/DSDT.tis.tpm2",
> > +"tests/data/acpi/q35/DSDT.viot",
> > +"tests/data/acpi/q35/DSDT.xapic",
>
> still missing DSDT.count2 table, likely in other updates (as well)
> which should break bisection if not whole pull request.
>
> I'll prep a tree based on your pull req, with fixups
> for you to pull from.

Does this mean there will be a v5 for the PR?
V4 looks good in the CI so far ...


^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups
  2022-11-08  6:23 ` [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
@ 2022-11-08 13:47   ` Stefan Hajnoczi
  0 siblings, 0 replies; 151+ messages in thread
From: Stefan Hajnoczi @ 2022-11-08 13:47 UTC (permalink / raw)
  To: Michael S. Tsirkin; +Cc: qemu-devel, Peter Maydell

[-- Attachment #1: Type: text/plain, Size: 1842 bytes --]

On Tue, Nov 08, 2022 at 01:23:16AM -0500, Michael S. Tsirkin wrote:
> On Mon, Nov 07, 2022 at 05:47:16PM -0500, Michael S. Tsirkin wrote:
> > Changes from v3:
> >     Applied and squashed fix by Ani for modular build breakage
> >     Reordered Julia's patches to avoid bisect breakage
> >     Checkpatch fixes for Jason's patches
> >     Added Alex's patch to partially address virtio is_started mess
> >     There is a bigger issue found by Christian A. Ehrhardt, that
> >     still needs work
> >     checkpatch change to avoid breaking on Jason's patches (to make ci pass)
> > 
> > Changes from v2:
> >     Fixed a bug in error handling in vhost: Change the sequence of device start.
> >         Contributor placed on watchlist ;)
> >     Dropped virtio: re-order vm_running and use_started checks
> >         Due to failures detected by gitlab.
> >         We'll have to fix it differently.
> >     Updated expected files for core-count test to fix bisect.
> > 
> > Changes from v1
> >     Applied and squashed fixes by Igor, Lei He, Hesham Almatary for
> >     bugs that tripped up the pipeline.
> >     Updated expected files for core-count test.
> > 
> > 
> > 
> > The following changes since commit a11f65ec1b8adcb012b89c92819cbda4dc25aaf1:
> > 
> >   Merge tag 'block-pull-request' of https://gitlab.com/stefanha/qemu into staging (2022-11-01 13:49:33 -0400)
> > 
> > are available in the Git repository at:
> > 
> >   https://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git tags/for_upstream
> > 
> > for you to fetch changes up to 1ef47f40dce3d5b176ddf76d57b5bfa2efb0b3c6:
> 
>  5e75ffd664258d3d2fd3d27e92e2748024f53bca now - I found and fixed a typo in a comment in checkpatch and re-pushed

This didn't make it in, sorry. Please send the typo fix in the next pull
request.

Thanks,
Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 45/83] tests: acpi: whitelist DSDT before generating PCI-ISA bridge AML automatically
  2022-11-08 13:36   ` Igor Mammedov
  2022-11-08 13:39     ` Ani Sinha
@ 2022-11-08 13:49     ` Michael S. Tsirkin
  2022-11-08 14:31       ` Igor Mammedov
  1 sibling, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-08 13:49 UTC (permalink / raw)
  To: Igor Mammedov; +Cc: qemu-devel, Peter Maydell, Ani Sinha

On Tue, Nov 08, 2022 at 02:36:41PM +0100, Igor Mammedov wrote:
> On Mon, 7 Nov 2022 17:51:11 -0500
> "Michael S. Tsirkin" <mst@redhat.com> wrote:
> 
> > From: Igor Mammedov <imammedo@redhat.com>
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > Message-Id: <20221017102146.2254096-3-imammedo@redhat.com>
> > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > ---
> >  tests/qtest/bios-tables-test-allowed-diff.h | 34 +++++++++++++++++++++
> >  1 file changed, 34 insertions(+)
> > 
> > diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
> > index dfb8523c8b..570b17478e 100644
> > --- a/tests/qtest/bios-tables-test-allowed-diff.h
> > +++ b/tests/qtest/bios-tables-test-allowed-diff.h
> > @@ -1 +1,35 @@
> >  /* List of comma-separated changed AML files to ignore */
> > +"tests/data/acpi/pc/DSDT",
> > +"tests/data/acpi/pc/DSDT.acpierst",
> > +"tests/data/acpi/pc/DSDT.acpihmat",
> > +"tests/data/acpi/pc/DSDT.bridge",
> > +"tests/data/acpi/pc/DSDT.cphp",
> > +"tests/data/acpi/pc/DSDT.dimmpxm",
> > +"tests/data/acpi/pc/DSDT.hpbridge",
> > +"tests/data/acpi/pc/DSDT.hpbrroot",
> > +"tests/data/acpi/pc/DSDT.ipmikcs",
> > +"tests/data/acpi/pc/DSDT.memhp",
> > +"tests/data/acpi/pc/DSDT.nohpet",
> > +"tests/data/acpi/pc/DSDT.numamem",
> > +"tests/data/acpi/pc/DSDT.roothp",
> > +"tests/data/acpi/q35/DSDT",
> > +"tests/data/acpi/q35/DSDT.acpierst",
> > +"tests/data/acpi/q35/DSDT.acpihmat",
> > +"tests/data/acpi/q35/DSDT.applesmc",
> > +"tests/data/acpi/q35/DSDT.bridge",
> > +"tests/data/acpi/q35/DSDT.cphp",
> > +"tests/data/acpi/q35/DSDT.cxl",
> > +"tests/data/acpi/q35/DSDT.dimmpxm",
> > +"tests/data/acpi/q35/DSDT.ipmibt",
> > +"tests/data/acpi/q35/DSDT.ipmismbus",
> > +"tests/data/acpi/q35/DSDT.ivrs",
> > +"tests/data/acpi/q35/DSDT.memhp",
> > +"tests/data/acpi/q35/DSDT.mmio64",
> > +"tests/data/acpi/q35/DSDT.multi-bridge",
> > +"tests/data/acpi/q35/DSDT.nohpet",
> > +"tests/data/acpi/q35/DSDT.numamem",
> > +"tests/data/acpi/q35/DSDT.pvpanic-isa",
> > +"tests/data/acpi/q35/DSDT.tis.tpm12",
> > +"tests/data/acpi/q35/DSDT.tis.tpm2",
> > +"tests/data/acpi/q35/DSDT.viot",
> > +"tests/data/acpi/q35/DSDT.xapic",
> 
> still missing DSDT.count2 table, likely in other updates (as well)
> which should break bisection if not whole pull request.


That's because I reordered count2 patches to be after these.

> I'll prep a tree based on your pull req, with fixups
> for you to pull from.



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 45/83] tests: acpi: whitelist DSDT before generating PCI-ISA bridge AML automatically
  2022-11-08 13:39     ` Ani Sinha
@ 2022-11-08 14:06       ` Ani Sinha
  0 siblings, 0 replies; 151+ messages in thread
From: Ani Sinha @ 2022-11-08 14:06 UTC (permalink / raw)
  To: Igor Mammedov; +Cc: Michael S. Tsirkin, qemu-devel, Peter Maydell

On Tue, Nov 8, 2022 at 7:09 PM Ani Sinha <ani@anisinha.ca> wrote:
>
> On Tue, Nov 8, 2022 at 7:06 PM Igor Mammedov <imammedo@redhat.com> wrote:
> >
> > On Mon, 7 Nov 2022 17:51:11 -0500
> > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> >
> > > From: Igor Mammedov <imammedo@redhat.com>
> > >
> > > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > > Message-Id: <20221017102146.2254096-3-imammedo@redhat.com>
> > > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > ---
> > >  tests/qtest/bios-tables-test-allowed-diff.h | 34 +++++++++++++++++++++
> > >  1 file changed, 34 insertions(+)
> > >
> > > diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
> > > index dfb8523c8b..570b17478e 100644
> > > --- a/tests/qtest/bios-tables-test-allowed-diff.h
> > > +++ b/tests/qtest/bios-tables-test-allowed-diff.h
> > > @@ -1 +1,35 @@
> > >  /* List of comma-separated changed AML files to ignore */
> > > +"tests/data/acpi/pc/DSDT",
> > > +"tests/data/acpi/pc/DSDT.acpierst",
> > > +"tests/data/acpi/pc/DSDT.acpihmat",
> > > +"tests/data/acpi/pc/DSDT.bridge",
> > > +"tests/data/acpi/pc/DSDT.cphp",
> > > +"tests/data/acpi/pc/DSDT.dimmpxm",
> > > +"tests/data/acpi/pc/DSDT.hpbridge",
> > > +"tests/data/acpi/pc/DSDT.hpbrroot",
> > > +"tests/data/acpi/pc/DSDT.ipmikcs",
> > > +"tests/data/acpi/pc/DSDT.memhp",
> > > +"tests/data/acpi/pc/DSDT.nohpet",
> > > +"tests/data/acpi/pc/DSDT.numamem",
> > > +"tests/data/acpi/pc/DSDT.roothp",
> > > +"tests/data/acpi/q35/DSDT",
> > > +"tests/data/acpi/q35/DSDT.acpierst",
> > > +"tests/data/acpi/q35/DSDT.acpihmat",
> > > +"tests/data/acpi/q35/DSDT.applesmc",
> > > +"tests/data/acpi/q35/DSDT.bridge",
> > > +"tests/data/acpi/q35/DSDT.cphp",
> > > +"tests/data/acpi/q35/DSDT.cxl",
> > > +"tests/data/acpi/q35/DSDT.dimmpxm",
> > > +"tests/data/acpi/q35/DSDT.ipmibt",
> > > +"tests/data/acpi/q35/DSDT.ipmismbus",
> > > +"tests/data/acpi/q35/DSDT.ivrs",
> > > +"tests/data/acpi/q35/DSDT.memhp",
> > > +"tests/data/acpi/q35/DSDT.mmio64",
> > > +"tests/data/acpi/q35/DSDT.multi-bridge",
> > > +"tests/data/acpi/q35/DSDT.nohpet",
> > > +"tests/data/acpi/q35/DSDT.numamem",
> > > +"tests/data/acpi/q35/DSDT.pvpanic-isa",
> > > +"tests/data/acpi/q35/DSDT.tis.tpm12",
> > > +"tests/data/acpi/q35/DSDT.tis.tpm2",
> > > +"tests/data/acpi/q35/DSDT.viot",
> > > +"tests/data/acpi/q35/DSDT.xapic",
> >
> > still missing DSDT.count2 table, likely in other updates (as well)
> > which should break bisection if not whole pull request.
> >
> > I'll prep a tree based on your pull req, with fixups
> > for you to pull from.
>
> Does this mean there will be a v5 for the PR?
> V4 looks good in the CI so far ...

Never mind. It has merged to QEMU master.


^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 45/83] tests: acpi: whitelist DSDT before generating PCI-ISA bridge AML automatically
  2022-11-08 13:49     ` Michael S. Tsirkin
@ 2022-11-08 14:31       ` Igor Mammedov
  0 siblings, 0 replies; 151+ messages in thread
From: Igor Mammedov @ 2022-11-08 14:31 UTC (permalink / raw)
  To: Michael S. Tsirkin; +Cc: qemu-devel, Peter Maydell, Ani Sinha

On Tue, 8 Nov 2022 08:49:01 -0500
"Michael S. Tsirkin" <mst@redhat.com> wrote:

> On Tue, Nov 08, 2022 at 02:36:41PM +0100, Igor Mammedov wrote:
> > On Mon, 7 Nov 2022 17:51:11 -0500
> > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> >   
> > > From: Igor Mammedov <imammedo@redhat.com>
> > > 
> > > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > > Message-Id: <20221017102146.2254096-3-imammedo@redhat.com>
> > > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > ---
> > >  tests/qtest/bios-tables-test-allowed-diff.h | 34 +++++++++++++++++++++
> > >  1 file changed, 34 insertions(+)
> > > 
> > > diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
> > > index dfb8523c8b..570b17478e 100644
> > > --- a/tests/qtest/bios-tables-test-allowed-diff.h
> > > +++ b/tests/qtest/bios-tables-test-allowed-diff.h
> > > @@ -1 +1,35 @@
> > >  /* List of comma-separated changed AML files to ignore */
> > > +"tests/data/acpi/pc/DSDT",
> > > +"tests/data/acpi/pc/DSDT.acpierst",
> > > +"tests/data/acpi/pc/DSDT.acpihmat",
> > > +"tests/data/acpi/pc/DSDT.bridge",
> > > +"tests/data/acpi/pc/DSDT.cphp",
> > > +"tests/data/acpi/pc/DSDT.dimmpxm",
> > > +"tests/data/acpi/pc/DSDT.hpbridge",
> > > +"tests/data/acpi/pc/DSDT.hpbrroot",
> > > +"tests/data/acpi/pc/DSDT.ipmikcs",
> > > +"tests/data/acpi/pc/DSDT.memhp",
> > > +"tests/data/acpi/pc/DSDT.nohpet",
> > > +"tests/data/acpi/pc/DSDT.numamem",
> > > +"tests/data/acpi/pc/DSDT.roothp",
> > > +"tests/data/acpi/q35/DSDT",
> > > +"tests/data/acpi/q35/DSDT.acpierst",
> > > +"tests/data/acpi/q35/DSDT.acpihmat",
> > > +"tests/data/acpi/q35/DSDT.applesmc",
> > > +"tests/data/acpi/q35/DSDT.bridge",
> > > +"tests/data/acpi/q35/DSDT.cphp",
> > > +"tests/data/acpi/q35/DSDT.cxl",
> > > +"tests/data/acpi/q35/DSDT.dimmpxm",
> > > +"tests/data/acpi/q35/DSDT.ipmibt",
> > > +"tests/data/acpi/q35/DSDT.ipmismbus",
> > > +"tests/data/acpi/q35/DSDT.ivrs",
> > > +"tests/data/acpi/q35/DSDT.memhp",
> > > +"tests/data/acpi/q35/DSDT.mmio64",
> > > +"tests/data/acpi/q35/DSDT.multi-bridge",
> > > +"tests/data/acpi/q35/DSDT.nohpet",
> > > +"tests/data/acpi/q35/DSDT.numamem",
> > > +"tests/data/acpi/q35/DSDT.pvpanic-isa",
> > > +"tests/data/acpi/q35/DSDT.tis.tpm12",
> > > +"tests/data/acpi/q35/DSDT.tis.tpm2",
> > > +"tests/data/acpi/q35/DSDT.viot",
> > > +"tests/data/acpi/q35/DSDT.xapic",  
> > 
> > still missing DSDT.count2 table, likely in other updates (as well)
> > which should break bisection if not whole pull request.  
> 
> 
> That's because I reordered count2 patches to be after these.

Aha,
that should fix the issue.

> 
> > I'll prep a tree based on your pull req, with fixups
> > for you to pull from.  
> 



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 29/83] virtio: introduce virtio_queue_enable()
  2022-11-07 22:49 ` [PULL v4 29/83] virtio: introduce virtio_queue_enable() Michael S. Tsirkin
@ 2022-11-08 19:43   ` Stefan Hajnoczi
  2022-11-09  3:31     ` Jason Wang
  0 siblings, 1 reply; 151+ messages in thread
From: Stefan Hajnoczi @ 2022-11-08 19:43 UTC (permalink / raw)
  To: Michael S. Tsirkin, Kangjie Xu
  Cc: qemu-devel, Peter Maydell, Xuan Zhuo, Jason Wang

On Mon, 7 Nov 2022 at 18:10, Michael S. Tsirkin <mst@redhat.com> wrote:
>
> From: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>
> Introduce the interface queue_enable() in VirtioDeviceClass and the
> fucntion virtio_queue_enable() in virtio, it can be called when
> VIRTIO_PCI_COMMON_Q_ENABLE is written and related virtqueue can be
> started. It only supports the devices of virtio 1 or later. The
> not-supported devices can only start the virtqueue when DRIVER_OK.
>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> Acked-by: Jason Wang <jasowang@redhat.com>
> Message-Id: <20221017092558.111082-4-xuanzhuo@linux.alibaba.com>
> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> ---
>  include/hw/virtio/virtio.h |  2 ++
>  hw/virtio/virtio.c         | 14 ++++++++++++++
>  2 files changed, 16 insertions(+)
>
> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> index 74d76c1dbc..b00b3fcf31 100644
> --- a/include/hw/virtio/virtio.h
> +++ b/include/hw/virtio/virtio.h
> @@ -149,6 +149,7 @@ struct VirtioDeviceClass {
>      void (*reset)(VirtIODevice *vdev);
>      void (*set_status)(VirtIODevice *vdev, uint8_t val);
>      void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
> +    void (*queue_enable)(VirtIODevice *vdev, uint32_t queue_index);
>      /* For transitional devices, this is a bitmap of features
>       * that are only exposed on the legacy interface but not
>       * the modern one.
> @@ -288,6 +289,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
>  int virtio_set_status(VirtIODevice *vdev, uint8_t val);
>  void virtio_reset(void *opaque);
>  void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
> +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index);
>  void virtio_update_irq(VirtIODevice *vdev);
>  int virtio_set_features(VirtIODevice *vdev, uint64_t val);
>
> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> index cf5f9ca387..9683b2e158 100644
> --- a/hw/virtio/virtio.c
> +++ b/hw/virtio/virtio.c
> @@ -2495,6 +2495,20 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
>      __virtio_queue_reset(vdev, queue_index);
>  }
>
> +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
> +{
> +    VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
> +
> +    if (!virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
> +        error_report("queue_enable is only suppported in devices of virtio "
> +                     "1.0 or later.");

Why is this triggering here? Maybe virtio_queue_enable() is called too
early. I have verified that the Linux guest driver sets VERSION_1. I
didn't check what SeaBIOS does.

$ build/qemu-system-x86_64 -M accel=kvm -m 1G -cpu host -blockdev
file,node-name=drive0,filename=test.img -device
virtio-blk-pci,drive=drive0
qemu: queue_enable is only suppported in devices of virtio 1.0 or later.

Stefan


^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 29/83] virtio: introduce virtio_queue_enable()
  2022-11-08 19:43   ` Stefan Hajnoczi
@ 2022-11-09  3:31     ` Jason Wang
  2022-11-09  6:39       ` Michael S. Tsirkin
  2022-11-09  6:51       ` Michael S. Tsirkin
  0 siblings, 2 replies; 151+ messages in thread
From: Jason Wang @ 2022-11-09  3:31 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: Michael S. Tsirkin, Kangjie Xu, qemu-devel, Peter Maydell, Xuan Zhuo

On Wed, Nov 9, 2022 at 3:43 AM Stefan Hajnoczi <stefanha@gmail.com> wrote:
>
> On Mon, 7 Nov 2022 at 18:10, Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > From: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> >
> > Introduce the interface queue_enable() in VirtioDeviceClass and the
> > fucntion virtio_queue_enable() in virtio, it can be called when
> > VIRTIO_PCI_COMMON_Q_ENABLE is written and related virtqueue can be
> > started. It only supports the devices of virtio 1 or later. The
> > not-supported devices can only start the virtqueue when DRIVER_OK.
> >
> > Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > Acked-by: Jason Wang <jasowang@redhat.com>
> > Message-Id: <20221017092558.111082-4-xuanzhuo@linux.alibaba.com>
> > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > ---
> >  include/hw/virtio/virtio.h |  2 ++
> >  hw/virtio/virtio.c         | 14 ++++++++++++++
> >  2 files changed, 16 insertions(+)
> >
> > diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> > index 74d76c1dbc..b00b3fcf31 100644
> > --- a/include/hw/virtio/virtio.h
> > +++ b/include/hw/virtio/virtio.h
> > @@ -149,6 +149,7 @@ struct VirtioDeviceClass {
> >      void (*reset)(VirtIODevice *vdev);
> >      void (*set_status)(VirtIODevice *vdev, uint8_t val);
> >      void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
> > +    void (*queue_enable)(VirtIODevice *vdev, uint32_t queue_index);
> >      /* For transitional devices, this is a bitmap of features
> >       * that are only exposed on the legacy interface but not
> >       * the modern one.
> > @@ -288,6 +289,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
> >  int virtio_set_status(VirtIODevice *vdev, uint8_t val);
> >  void virtio_reset(void *opaque);
> >  void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
> > +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index);
> >  void virtio_update_irq(VirtIODevice *vdev);
> >  int virtio_set_features(VirtIODevice *vdev, uint64_t val);
> >
> > diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> > index cf5f9ca387..9683b2e158 100644
> > --- a/hw/virtio/virtio.c
> > +++ b/hw/virtio/virtio.c
> > @@ -2495,6 +2495,20 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
> >      __virtio_queue_reset(vdev, queue_index);
> >  }
> >
> > +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
> > +{
> > +    VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
> > +
> > +    if (!virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
> > +        error_report("queue_enable is only suppported in devices of virtio "
> > +                     "1.0 or later.");
>
> Why is this triggering here? Maybe virtio_queue_enable() is called too
> early. I have verified that the Linux guest driver sets VERSION_1. I
> didn't check what SeaBIOS does.

Looks like a bug, we should check device features here at least and it
should be guest errors instead of error_report() here.

Thanks

>
> $ build/qemu-system-x86_64 -M accel=kvm -m 1G -cpu host -blockdev
> file,node-name=drive0,filename=test.img -device
> virtio-blk-pci,drive=drive0
> qemu: queue_enable is only suppported in devices of virtio 1.0 or later.
>
> Stefan
>



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 29/83] virtio: introduce virtio_queue_enable()
  2022-11-09  3:31     ` Jason Wang
@ 2022-11-09  6:39       ` Michael S. Tsirkin
  2022-11-09  6:48         ` Xuan Zhuo
  2022-11-09  6:51       ` Michael S. Tsirkin
  1 sibling, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-09  6:39 UTC (permalink / raw)
  To: Jason Wang
  Cc: Stefan Hajnoczi, Kangjie Xu, qemu-devel, Peter Maydell, Xuan Zhuo

On Wed, Nov 09, 2022 at 11:31:23AM +0800, Jason Wang wrote:
> On Wed, Nov 9, 2022 at 3:43 AM Stefan Hajnoczi <stefanha@gmail.com> wrote:
> >
> > On Mon, 7 Nov 2022 at 18:10, Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > From: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > >
> > > Introduce the interface queue_enable() in VirtioDeviceClass and the
> > > fucntion virtio_queue_enable() in virtio, it can be called when
> > > VIRTIO_PCI_COMMON_Q_ENABLE is written and related virtqueue can be
> > > started. It only supports the devices of virtio 1 or later. The
> > > not-supported devices can only start the virtqueue when DRIVER_OK.
> > >
> > > Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > Acked-by: Jason Wang <jasowang@redhat.com>
> > > Message-Id: <20221017092558.111082-4-xuanzhuo@linux.alibaba.com>
> > > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > ---
> > >  include/hw/virtio/virtio.h |  2 ++
> > >  hw/virtio/virtio.c         | 14 ++++++++++++++
> > >  2 files changed, 16 insertions(+)
> > >
> > > diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> > > index 74d76c1dbc..b00b3fcf31 100644
> > > --- a/include/hw/virtio/virtio.h
> > > +++ b/include/hw/virtio/virtio.h
> > > @@ -149,6 +149,7 @@ struct VirtioDeviceClass {
> > >      void (*reset)(VirtIODevice *vdev);
> > >      void (*set_status)(VirtIODevice *vdev, uint8_t val);
> > >      void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
> > > +    void (*queue_enable)(VirtIODevice *vdev, uint32_t queue_index);
> > >      /* For transitional devices, this is a bitmap of features
> > >       * that are only exposed on the legacy interface but not
> > >       * the modern one.
> > > @@ -288,6 +289,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
> > >  int virtio_set_status(VirtIODevice *vdev, uint8_t val);
> > >  void virtio_reset(void *opaque);
> > >  void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
> > > +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index);
> > >  void virtio_update_irq(VirtIODevice *vdev);
> > >  int virtio_set_features(VirtIODevice *vdev, uint64_t val);
> > >
> > > diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> > > index cf5f9ca387..9683b2e158 100644
> > > --- a/hw/virtio/virtio.c
> > > +++ b/hw/virtio/virtio.c
> > > @@ -2495,6 +2495,20 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
> > >      __virtio_queue_reset(vdev, queue_index);
> > >  }
> > >
> > > +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
> > > +{
> > > +    VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
> > > +
> > > +    if (!virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
> > > +        error_report("queue_enable is only suppported in devices of virtio "
> > > +                     "1.0 or later.");
> >
> > Why is this triggering here? Maybe virtio_queue_enable() is called too
> > early. I have verified that the Linux guest driver sets VERSION_1. I
> > didn't check what SeaBIOS does.
> 
> Looks like a bug, we should check device features here at least and it
> should be guest errors instead of error_report() here.
> 
> Thanks
> 

I suspect we should just drop this print. Kangjie?


> > $ build/qemu-system-x86_64 -M accel=kvm -m 1G -cpu host -blockdev
> > file,node-name=drive0,filename=test.img -device
> > virtio-blk-pci,drive=drive0
> > qemu: queue_enable is only suppported in devices of virtio 1.0 or later.
> >
> > Stefan
> >



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 29/83] virtio: introduce virtio_queue_enable()
  2022-11-09  6:39       ` Michael S. Tsirkin
@ 2022-11-09  6:48         ` Xuan Zhuo
  2022-11-09  7:01           ` Michael S. Tsirkin
  0 siblings, 1 reply; 151+ messages in thread
From: Xuan Zhuo @ 2022-11-09  6:48 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Stefan Hajnoczi, Kangjie Xu, qemu-devel, Peter Maydell, Jason Wang

On Wed, 9 Nov 2022 01:39:32 -0500, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Wed, Nov 09, 2022 at 11:31:23AM +0800, Jason Wang wrote:
> > On Wed, Nov 9, 2022 at 3:43 AM Stefan Hajnoczi <stefanha@gmail.com> wrote:
> > >
> > > On Mon, 7 Nov 2022 at 18:10, Michael S. Tsirkin <mst@redhat.com> wrote:
> > > >
> > > > From: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > > >
> > > > Introduce the interface queue_enable() in VirtioDeviceClass and the
> > > > fucntion virtio_queue_enable() in virtio, it can be called when
> > > > VIRTIO_PCI_COMMON_Q_ENABLE is written and related virtqueue can be
> > > > started. It only supports the devices of virtio 1 or later. The
> > > > not-supported devices can only start the virtqueue when DRIVER_OK.
> > > >
> > > > Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > Acked-by: Jason Wang <jasowang@redhat.com>
> > > > Message-Id: <20221017092558.111082-4-xuanzhuo@linux.alibaba.com>
> > > > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > > ---
> > > >  include/hw/virtio/virtio.h |  2 ++
> > > >  hw/virtio/virtio.c         | 14 ++++++++++++++
> > > >  2 files changed, 16 insertions(+)
> > > >
> > > > diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> > > > index 74d76c1dbc..b00b3fcf31 100644
> > > > --- a/include/hw/virtio/virtio.h
> > > > +++ b/include/hw/virtio/virtio.h
> > > > @@ -149,6 +149,7 @@ struct VirtioDeviceClass {
> > > >      void (*reset)(VirtIODevice *vdev);
> > > >      void (*set_status)(VirtIODevice *vdev, uint8_t val);
> > > >      void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
> > > > +    void (*queue_enable)(VirtIODevice *vdev, uint32_t queue_index);
> > > >      /* For transitional devices, this is a bitmap of features
> > > >       * that are only exposed on the legacy interface but not
> > > >       * the modern one.
> > > > @@ -288,6 +289,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
> > > >  int virtio_set_status(VirtIODevice *vdev, uint8_t val);
> > > >  void virtio_reset(void *opaque);
> > > >  void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
> > > > +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index);
> > > >  void virtio_update_irq(VirtIODevice *vdev);
> > > >  int virtio_set_features(VirtIODevice *vdev, uint64_t val);
> > > >
> > > > diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> > > > index cf5f9ca387..9683b2e158 100644
> > > > --- a/hw/virtio/virtio.c
> > > > +++ b/hw/virtio/virtio.c
> > > > @@ -2495,6 +2495,20 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
> > > >      __virtio_queue_reset(vdev, queue_index);
> > > >  }
> > > >
> > > > +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
> > > > +{
> > > > +    VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
> > > > +
> > > > +    if (!virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
> > > > +        error_report("queue_enable is only suppported in devices of virtio "
> > > > +                     "1.0 or later.");
> > >
> > > Why is this triggering here? Maybe virtio_queue_enable() is called too
> > > early. I have verified that the Linux guest driver sets VERSION_1. I
> > > didn't check what SeaBIOS does.
> >
> > Looks like a bug, we should check device features here at least and it
> > should be guest errors instead of error_report() here.
> >
> > Thanks
> >
>
> I suspect we should just drop this print. Kangjie?


I think it is.

At that time, this inspection was only added at hand, and theoretically it
should not be performed.

I am responsible for this patch set now.

hi, Michael,

What should I do, do I send a new version again?

Thanks.

>
>
> > > $ build/qemu-system-x86_64 -M accel=kvm -m 1G -cpu host -blockdev
> > > file,node-name=drive0,filename=test.img -device
> > > virtio-blk-pci,drive=drive0
> > > qemu: queue_enable is only suppported in devices of virtio 1.0 or later.
> > >
> > > Stefan
> > >
>


^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 29/83] virtio: introduce virtio_queue_enable()
  2022-11-09  3:31     ` Jason Wang
  2022-11-09  6:39       ` Michael S. Tsirkin
@ 2022-11-09  6:51       ` Michael S. Tsirkin
  2022-11-09  6:55         ` Jason Wang
  2022-11-09  6:59         ` Michael S. Tsirkin
  1 sibling, 2 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-09  6:51 UTC (permalink / raw)
  To: Jason Wang
  Cc: Stefan Hajnoczi, Kangjie Xu, qemu-devel, Peter Maydell, Xuan Zhuo

On Wed, Nov 09, 2022 at 11:31:23AM +0800, Jason Wang wrote:
> On Wed, Nov 9, 2022 at 3:43 AM Stefan Hajnoczi <stefanha@gmail.com> wrote:
> >
> > On Mon, 7 Nov 2022 at 18:10, Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > From: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > >
> > > Introduce the interface queue_enable() in VirtioDeviceClass and the
> > > fucntion virtio_queue_enable() in virtio, it can be called when
> > > VIRTIO_PCI_COMMON_Q_ENABLE is written and related virtqueue can be
> > > started. It only supports the devices of virtio 1 or later. The
> > > not-supported devices can only start the virtqueue when DRIVER_OK.
> > >
> > > Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > Acked-by: Jason Wang <jasowang@redhat.com>
> > > Message-Id: <20221017092558.111082-4-xuanzhuo@linux.alibaba.com>
> > > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > ---
> > >  include/hw/virtio/virtio.h |  2 ++
> > >  hw/virtio/virtio.c         | 14 ++++++++++++++
> > >  2 files changed, 16 insertions(+)
> > >
> > > diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> > > index 74d76c1dbc..b00b3fcf31 100644
> > > --- a/include/hw/virtio/virtio.h
> > > +++ b/include/hw/virtio/virtio.h
> > > @@ -149,6 +149,7 @@ struct VirtioDeviceClass {
> > >      void (*reset)(VirtIODevice *vdev);
> > >      void (*set_status)(VirtIODevice *vdev, uint8_t val);
> > >      void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
> > > +    void (*queue_enable)(VirtIODevice *vdev, uint32_t queue_index);
> > >      /* For transitional devices, this is a bitmap of features
> > >       * that are only exposed on the legacy interface but not
> > >       * the modern one.
> > > @@ -288,6 +289,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
> > >  int virtio_set_status(VirtIODevice *vdev, uint8_t val);
> > >  void virtio_reset(void *opaque);
> > >  void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
> > > +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index);
> > >  void virtio_update_irq(VirtIODevice *vdev);
> > >  int virtio_set_features(VirtIODevice *vdev, uint64_t val);
> > >
> > > diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> > > index cf5f9ca387..9683b2e158 100644
> > > --- a/hw/virtio/virtio.c
> > > +++ b/hw/virtio/virtio.c
> > > @@ -2495,6 +2495,20 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
> > >      __virtio_queue_reset(vdev, queue_index);
> > >  }
> > >
> > > +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
> > > +{
> > > +    VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
> > > +
> > > +    if (!virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
> > > +        error_report("queue_enable is only suppported in devices of virtio "
> > > +                     "1.0 or later.");
> >
> > Why is this triggering here? Maybe virtio_queue_enable() is called too
> > early. I have verified that the Linux guest driver sets VERSION_1. I
> > didn't check what SeaBIOS does.
> 
> Looks like a bug, we should check device features here at least and it
> should be guest errors instead of error_report() here.
> 
> Thanks

But it's weird, queue enable is written before guest features?
How come?

> >
> > $ build/qemu-system-x86_64 -M accel=kvm -m 1G -cpu host -blockdev
> > file,node-name=drive0,filename=test.img -device
> > virtio-blk-pci,drive=drive0
> > qemu: queue_enable is only suppported in devices of virtio 1.0 or later.
> >
> > Stefan
> >



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 29/83] virtio: introduce virtio_queue_enable()
  2022-11-09  6:51       ` Michael S. Tsirkin
@ 2022-11-09  6:55         ` Jason Wang
  2022-11-09  6:56           ` Xuan Zhuo
  2022-11-09  6:59         ` Michael S. Tsirkin
  1 sibling, 1 reply; 151+ messages in thread
From: Jason Wang @ 2022-11-09  6:55 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Stefan Hajnoczi, Kangjie Xu, qemu-devel, Peter Maydell, Xuan Zhuo


在 2022/11/9 14:51, Michael S. Tsirkin 写道:
> On Wed, Nov 09, 2022 at 11:31:23AM +0800, Jason Wang wrote:
>> On Wed, Nov 9, 2022 at 3:43 AM Stefan Hajnoczi <stefanha@gmail.com> wrote:
>>> On Mon, 7 Nov 2022 at 18:10, Michael S. Tsirkin <mst@redhat.com> wrote:
>>>> From: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>>>>
>>>> Introduce the interface queue_enable() in VirtioDeviceClass and the
>>>> fucntion virtio_queue_enable() in virtio, it can be called when
>>>> VIRTIO_PCI_COMMON_Q_ENABLE is written and related virtqueue can be
>>>> started. It only supports the devices of virtio 1 or later. The
>>>> not-supported devices can only start the virtqueue when DRIVER_OK.
>>>>
>>>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>>> Acked-by: Jason Wang <jasowang@redhat.com>
>>>> Message-Id: <20221017092558.111082-4-xuanzhuo@linux.alibaba.com>
>>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
>>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>>>> ---
>>>>   include/hw/virtio/virtio.h |  2 ++
>>>>   hw/virtio/virtio.c         | 14 ++++++++++++++
>>>>   2 files changed, 16 insertions(+)
>>>>
>>>> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
>>>> index 74d76c1dbc..b00b3fcf31 100644
>>>> --- a/include/hw/virtio/virtio.h
>>>> +++ b/include/hw/virtio/virtio.h
>>>> @@ -149,6 +149,7 @@ struct VirtioDeviceClass {
>>>>       void (*reset)(VirtIODevice *vdev);
>>>>       void (*set_status)(VirtIODevice *vdev, uint8_t val);
>>>>       void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
>>>> +    void (*queue_enable)(VirtIODevice *vdev, uint32_t queue_index);
>>>>       /* For transitional devices, this is a bitmap of features
>>>>        * that are only exposed on the legacy interface but not
>>>>        * the modern one.
>>>> @@ -288,6 +289,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
>>>>   int virtio_set_status(VirtIODevice *vdev, uint8_t val);
>>>>   void virtio_reset(void *opaque);
>>>>   void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
>>>> +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index);
>>>>   void virtio_update_irq(VirtIODevice *vdev);
>>>>   int virtio_set_features(VirtIODevice *vdev, uint64_t val);
>>>>
>>>> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
>>>> index cf5f9ca387..9683b2e158 100644
>>>> --- a/hw/virtio/virtio.c
>>>> +++ b/hw/virtio/virtio.c
>>>> @@ -2495,6 +2495,20 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
>>>>       __virtio_queue_reset(vdev, queue_index);
>>>>   }
>>>>
>>>> +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
>>>> +{
>>>> +    VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
>>>> +
>>>> +    if (!virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
>>>> +        error_report("queue_enable is only suppported in devices of virtio "
>>>> +                     "1.0 or later.");
>>> Why is this triggering here? Maybe virtio_queue_enable() is called too
>>> early. I have verified that the Linux guest driver sets VERSION_1. I
>>> didn't check what SeaBIOS does.
>> Looks like a bug, we should check device features here at least and it
>> should be guest errors instead of error_report() here.
>>
>> Thanks
> But it's weird, queue enable is written before guest features?
> How come?


Or queue_enable is written when the driver doesn't negotiate VERSION_1?

Thanks


>
>>> $ build/qemu-system-x86_64 -M accel=kvm -m 1G -cpu host -blockdev
>>> file,node-name=drive0,filename=test.img -device
>>> virtio-blk-pci,drive=drive0
>>> qemu: queue_enable is only suppported in devices of virtio 1.0 or later.
>>>
>>> Stefan
>>>



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 29/83] virtio: introduce virtio_queue_enable()
  2022-11-09  6:55         ` Jason Wang
@ 2022-11-09  6:56           ` Xuan Zhuo
  2022-11-09  7:04             ` Michael S. Tsirkin
  0 siblings, 1 reply; 151+ messages in thread
From: Xuan Zhuo @ 2022-11-09  6:56 UTC (permalink / raw)
  To: Jason Wang
  Cc: Stefan Hajnoczi, Kangjie Xu, qemu-devel, Peter Maydell,
	Michael S. Tsirkin

On Wed, 9 Nov 2022 14:55:03 +0800, Jason Wang <jasowang@redhat.com> wrote:
>
> 在 2022/11/9 14:51, Michael S. Tsirkin 写道:
> > On Wed, Nov 09, 2022 at 11:31:23AM +0800, Jason Wang wrote:
> >> On Wed, Nov 9, 2022 at 3:43 AM Stefan Hajnoczi <stefanha@gmail.com> wrote:
> >>> On Mon, 7 Nov 2022 at 18:10, Michael S. Tsirkin <mst@redhat.com> wrote:
> >>>> From: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> >>>>
> >>>> Introduce the interface queue_enable() in VirtioDeviceClass and the
> >>>> fucntion virtio_queue_enable() in virtio, it can be called when
> >>>> VIRTIO_PCI_COMMON_Q_ENABLE is written and related virtqueue can be
> >>>> started. It only supports the devices of virtio 1 or later. The
> >>>> not-supported devices can only start the virtqueue when DRIVER_OK.
> >>>>
> >>>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> >>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> >>>> Acked-by: Jason Wang <jasowang@redhat.com>
> >>>> Message-Id: <20221017092558.111082-4-xuanzhuo@linux.alibaba.com>
> >>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> >>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> >>>> ---
> >>>>   include/hw/virtio/virtio.h |  2 ++
> >>>>   hw/virtio/virtio.c         | 14 ++++++++++++++
> >>>>   2 files changed, 16 insertions(+)
> >>>>
> >>>> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> >>>> index 74d76c1dbc..b00b3fcf31 100644
> >>>> --- a/include/hw/virtio/virtio.h
> >>>> +++ b/include/hw/virtio/virtio.h
> >>>> @@ -149,6 +149,7 @@ struct VirtioDeviceClass {
> >>>>       void (*reset)(VirtIODevice *vdev);
> >>>>       void (*set_status)(VirtIODevice *vdev, uint8_t val);
> >>>>       void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
> >>>> +    void (*queue_enable)(VirtIODevice *vdev, uint32_t queue_index);
> >>>>       /* For transitional devices, this is a bitmap of features
> >>>>        * that are only exposed on the legacy interface but not
> >>>>        * the modern one.
> >>>> @@ -288,6 +289,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
> >>>>   int virtio_set_status(VirtIODevice *vdev, uint8_t val);
> >>>>   void virtio_reset(void *opaque);
> >>>>   void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
> >>>> +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index);
> >>>>   void virtio_update_irq(VirtIODevice *vdev);
> >>>>   int virtio_set_features(VirtIODevice *vdev, uint64_t val);
> >>>>
> >>>> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> >>>> index cf5f9ca387..9683b2e158 100644
> >>>> --- a/hw/virtio/virtio.c
> >>>> +++ b/hw/virtio/virtio.c
> >>>> @@ -2495,6 +2495,20 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
> >>>>       __virtio_queue_reset(vdev, queue_index);
> >>>>   }
> >>>>
> >>>> +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
> >>>> +{
> >>>> +    VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
> >>>> +
> >>>> +    if (!virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
> >>>> +        error_report("queue_enable is only suppported in devices of virtio "
> >>>> +                     "1.0 or later.");
> >>> Why is this triggering here? Maybe virtio_queue_enable() is called too
> >>> early. I have verified that the Linux guest driver sets VERSION_1. I
> >>> didn't check what SeaBIOS does.
> >> Looks like a bug, we should check device features here at least and it
> >> should be guest errors instead of error_report() here.
> >>
> >> Thanks
> > But it's weird, queue enable is written before guest features?
> > How come?
>
>
> Or queue_enable is written when the driver doesn't negotiate VERSION_1?

Is this a bug?

Or is it allowed in some cases?

I feel weird too.

Thanks.

>
> Thanks
>
>
> >
> >>> $ build/qemu-system-x86_64 -M accel=kvm -m 1G -cpu host -blockdev
> >>> file,node-name=drive0,filename=test.img -device
> >>> virtio-blk-pci,drive=drive0
> >>> qemu: queue_enable is only suppported in devices of virtio 1.0 or later.
> >>>
> >>> Stefan
> >>>
>


^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 29/83] virtio: introduce virtio_queue_enable()
  2022-11-09  6:51       ` Michael S. Tsirkin
  2022-11-09  6:55         ` Jason Wang
@ 2022-11-09  6:59         ` Michael S. Tsirkin
  2022-11-09  8:56           ` Laszlo Ersek
  1 sibling, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-09  6:59 UTC (permalink / raw)
  To: Jason Wang
  Cc: Stefan Hajnoczi, Kangjie Xu, qemu-devel, Peter Maydell,
	Xuan Zhuo, Gerd Hoffmann, Laszlo Ersek

On Wed, Nov 09, 2022 at 01:52:01AM -0500, Michael S. Tsirkin wrote:
> On Wed, Nov 09, 2022 at 11:31:23AM +0800, Jason Wang wrote:
> > On Wed, Nov 9, 2022 at 3:43 AM Stefan Hajnoczi <stefanha@gmail.com> wrote:
> > >
> > > On Mon, 7 Nov 2022 at 18:10, Michael S. Tsirkin <mst@redhat.com> wrote:
> > > >
> > > > From: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > > >
> > > > Introduce the interface queue_enable() in VirtioDeviceClass and the
> > > > fucntion virtio_queue_enable() in virtio, it can be called when
> > > > VIRTIO_PCI_COMMON_Q_ENABLE is written and related virtqueue can be
> > > > started. It only supports the devices of virtio 1 or later. The
> > > > not-supported devices can only start the virtqueue when DRIVER_OK.
> > > >
> > > > Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > Acked-by: Jason Wang <jasowang@redhat.com>
> > > > Message-Id: <20221017092558.111082-4-xuanzhuo@linux.alibaba.com>
> > > > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > > ---
> > > >  include/hw/virtio/virtio.h |  2 ++
> > > >  hw/virtio/virtio.c         | 14 ++++++++++++++
> > > >  2 files changed, 16 insertions(+)
> > > >
> > > > diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> > > > index 74d76c1dbc..b00b3fcf31 100644
> > > > --- a/include/hw/virtio/virtio.h
> > > > +++ b/include/hw/virtio/virtio.h
> > > > @@ -149,6 +149,7 @@ struct VirtioDeviceClass {
> > > >      void (*reset)(VirtIODevice *vdev);
> > > >      void (*set_status)(VirtIODevice *vdev, uint8_t val);
> > > >      void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
> > > > +    void (*queue_enable)(VirtIODevice *vdev, uint32_t queue_index);
> > > >      /* For transitional devices, this is a bitmap of features
> > > >       * that are only exposed on the legacy interface but not
> > > >       * the modern one.
> > > > @@ -288,6 +289,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
> > > >  int virtio_set_status(VirtIODevice *vdev, uint8_t val);
> > > >  void virtio_reset(void *opaque);
> > > >  void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
> > > > +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index);
> > > >  void virtio_update_irq(VirtIODevice *vdev);
> > > >  int virtio_set_features(VirtIODevice *vdev, uint64_t val);
> > > >
> > > > diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> > > > index cf5f9ca387..9683b2e158 100644
> > > > --- a/hw/virtio/virtio.c
> > > > +++ b/hw/virtio/virtio.c
> > > > @@ -2495,6 +2495,20 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
> > > >      __virtio_queue_reset(vdev, queue_index);
> > > >  }
> > > >
> > > > +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
> > > > +{
> > > > +    VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
> > > > +
> > > > +    if (!virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
> > > > +        error_report("queue_enable is only suppported in devices of virtio "
> > > > +                     "1.0 or later.");
> > >
> > > Why is this triggering here? Maybe virtio_queue_enable() is called too
> > > early. I have verified that the Linux guest driver sets VERSION_1. I
> > > didn't check what SeaBIOS does.
> > 
> > Looks like a bug, we should check device features here at least and it
> > should be guest errors instead of error_report() here.
> > 
> > Thanks
> 
> But it's weird, queue enable is written before guest features?
> How come?

It's a bios bug:


            
    vp_init_simple(&vdrive->vp, pci);
    if (vp_find_vq(&vdrive->vp, 0, &vdrive->vq) < 0 ) {
        dprintf(1, "fail to find vq for virtio-blk %pP\n", pci);
        goto fail; 
    }           
    
    if (vdrive->vp.use_modern) {
        struct vp_device *vp = &vdrive->vp;
        u64 features = vp_get_features(vp);
        u64 version1 = 1ull << VIRTIO_F_VERSION_1;
        u64 iommu_platform = 1ull << VIRTIO_F_IOMMU_PLATFORM;
        u64 blk_size = 1ull << VIRTIO_BLK_F_BLK_SIZE;
        if (!(features & version1)) {
            dprintf(1, "modern device without virtio_1 feature bit: %pP\n", pci);
            goto fail;
        }
        
        features = features & (version1 | iommu_platform | blk_size);
        vp_set_features(vp, features);
        status |= VIRTIO_CONFIG_S_FEATURES_OK;
        vp_set_status(vp, status);



Not good - does not match the spec. Here's what the spec says:


The driver MUST follow this sequence to initialize a device:
1. Reset the device.
2. Set the ACKNOWLEDGE status bit: the guest OS has noticed the device.
3. Set the DRIVER status bit: the guest OS knows how to drive the device.
4. Read device feature bits, and write the subset of feature bits understood by the OS and driver to the
device. During this step the driver MAY read (but MUST NOT write) the device-specific configuration
fields to check that it can support the device before accepting it.
5. Set the FEATURES_OK status bit. The driver MUST NOT accept new feature bits after this step.
6. Re-read device status to ensure the FEATURES_OK bit is still set: otherwise, the device does not
support our subset of features and the device is unusable.
7. Perform device-specific setup, including discovery of virtqueues for the device, optional per-bus setup,
reading and possibly writing the device’s virtio configuration space, and population of virtqueues.
8. Set the DRIVER_OK status bit. At this point the device is “live”.


Right?



> > >
> > > $ build/qemu-system-x86_64 -M accel=kvm -m 1G -cpu host -blockdev
> > > file,node-name=drive0,filename=test.img -device
> > > virtio-blk-pci,drive=drive0
> > > qemu: queue_enable is only suppported in devices of virtio 1.0 or later.
> > >
> > > Stefan
> > >



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 29/83] virtio: introduce virtio_queue_enable()
  2022-11-09  6:48         ` Xuan Zhuo
@ 2022-11-09  7:01           ` Michael S. Tsirkin
  2022-11-09  7:11             ` Xuan Zhuo
  0 siblings, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-09  7:01 UTC (permalink / raw)
  To: Xuan Zhuo
  Cc: Stefan Hajnoczi, Kangjie Xu, qemu-devel, Peter Maydell, Jason Wang

On Wed, Nov 09, 2022 at 02:48:29PM +0800, Xuan Zhuo wrote:
> On Wed, 9 Nov 2022 01:39:32 -0500, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > On Wed, Nov 09, 2022 at 11:31:23AM +0800, Jason Wang wrote:
> > > On Wed, Nov 9, 2022 at 3:43 AM Stefan Hajnoczi <stefanha@gmail.com> wrote:
> > > >
> > > > On Mon, 7 Nov 2022 at 18:10, Michael S. Tsirkin <mst@redhat.com> wrote:
> > > > >
> > > > > From: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > > > >
> > > > > Introduce the interface queue_enable() in VirtioDeviceClass and the
> > > > > fucntion virtio_queue_enable() in virtio, it can be called when
> > > > > VIRTIO_PCI_COMMON_Q_ENABLE is written and related virtqueue can be
> > > > > started. It only supports the devices of virtio 1 or later. The
> > > > > not-supported devices can only start the virtqueue when DRIVER_OK.
> > > > >
> > > > > Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > > Acked-by: Jason Wang <jasowang@redhat.com>
> > > > > Message-Id: <20221017092558.111082-4-xuanzhuo@linux.alibaba.com>
> > > > > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > > > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > > > ---
> > > > >  include/hw/virtio/virtio.h |  2 ++
> > > > >  hw/virtio/virtio.c         | 14 ++++++++++++++
> > > > >  2 files changed, 16 insertions(+)
> > > > >
> > > > > diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> > > > > index 74d76c1dbc..b00b3fcf31 100644
> > > > > --- a/include/hw/virtio/virtio.h
> > > > > +++ b/include/hw/virtio/virtio.h
> > > > > @@ -149,6 +149,7 @@ struct VirtioDeviceClass {
> > > > >      void (*reset)(VirtIODevice *vdev);
> > > > >      void (*set_status)(VirtIODevice *vdev, uint8_t val);
> > > > >      void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
> > > > > +    void (*queue_enable)(VirtIODevice *vdev, uint32_t queue_index);
> > > > >      /* For transitional devices, this is a bitmap of features
> > > > >       * that are only exposed on the legacy interface but not
> > > > >       * the modern one.
> > > > > @@ -288,6 +289,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
> > > > >  int virtio_set_status(VirtIODevice *vdev, uint8_t val);
> > > > >  void virtio_reset(void *opaque);
> > > > >  void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
> > > > > +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index);
> > > > >  void virtio_update_irq(VirtIODevice *vdev);
> > > > >  int virtio_set_features(VirtIODevice *vdev, uint64_t val);
> > > > >
> > > > > diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> > > > > index cf5f9ca387..9683b2e158 100644
> > > > > --- a/hw/virtio/virtio.c
> > > > > +++ b/hw/virtio/virtio.c
> > > > > @@ -2495,6 +2495,20 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
> > > > >      __virtio_queue_reset(vdev, queue_index);
> > > > >  }
> > > > >
> > > > > +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
> > > > > +{
> > > > > +    VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
> > > > > +
> > > > > +    if (!virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
> > > > > +        error_report("queue_enable is only suppported in devices of virtio "
> > > > > +                     "1.0 or later.");
> > > >
> > > > Why is this triggering here? Maybe virtio_queue_enable() is called too
> > > > early. I have verified that the Linux guest driver sets VERSION_1. I
> > > > didn't check what SeaBIOS does.
> > >
> > > Looks like a bug, we should check device features here at least and it
> > > should be guest errors instead of error_report() here.
> > >
> > > Thanks
> > >
> >
> > I suspect we should just drop this print. Kangjie?
> 
> 
> I think it is.
> 
> At that time, this inspection was only added at hand, and theoretically it
> should not be performed.
> 
> I am responsible for this patch set now.
> 
> hi, Michael,
> 
> What should I do, do I send a new version again?
> 
> Thanks.

I debugged it and replied separately. Can you check EFI drivers too?

> >
> >
> > > > $ build/qemu-system-x86_64 -M accel=kvm -m 1G -cpu host -blockdev
> > > > file,node-name=drive0,filename=test.img -device
> > > > virtio-blk-pci,drive=drive0
> > > > qemu: queue_enable is only suppported in devices of virtio 1.0 or later.
> > > >
> > > > Stefan
> > > >
> >



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 29/83] virtio: introduce virtio_queue_enable()
  2022-11-09  6:56           ` Xuan Zhuo
@ 2022-11-09  7:04             ` Michael S. Tsirkin
  2022-11-09  7:12               ` Xuan Zhuo
  0 siblings, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-09  7:04 UTC (permalink / raw)
  To: Xuan Zhuo
  Cc: Jason Wang, Stefan Hajnoczi, Kangjie Xu, qemu-devel, Peter Maydell

On Wed, Nov 09, 2022 at 02:56:01PM +0800, Xuan Zhuo wrote:
> On Wed, 9 Nov 2022 14:55:03 +0800, Jason Wang <jasowang@redhat.com> wrote:
> >
> > 在 2022/11/9 14:51, Michael S. Tsirkin 写道:
> > > On Wed, Nov 09, 2022 at 11:31:23AM +0800, Jason Wang wrote:
> > >> On Wed, Nov 9, 2022 at 3:43 AM Stefan Hajnoczi <stefanha@gmail.com> wrote:
> > >>> On Mon, 7 Nov 2022 at 18:10, Michael S. Tsirkin <mst@redhat.com> wrote:
> > >>>> From: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > >>>>
> > >>>> Introduce the interface queue_enable() in VirtioDeviceClass and the
> > >>>> fucntion virtio_queue_enable() in virtio, it can be called when
> > >>>> VIRTIO_PCI_COMMON_Q_ENABLE is written and related virtqueue can be
> > >>>> started. It only supports the devices of virtio 1 or later. The
> > >>>> not-supported devices can only start the virtqueue when DRIVER_OK.
> > >>>>
> > >>>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > >>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > >>>> Acked-by: Jason Wang <jasowang@redhat.com>
> > >>>> Message-Id: <20221017092558.111082-4-xuanzhuo@linux.alibaba.com>
> > >>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > >>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > >>>> ---
> > >>>>   include/hw/virtio/virtio.h |  2 ++
> > >>>>   hw/virtio/virtio.c         | 14 ++++++++++++++
> > >>>>   2 files changed, 16 insertions(+)
> > >>>>
> > >>>> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> > >>>> index 74d76c1dbc..b00b3fcf31 100644
> > >>>> --- a/include/hw/virtio/virtio.h
> > >>>> +++ b/include/hw/virtio/virtio.h
> > >>>> @@ -149,6 +149,7 @@ struct VirtioDeviceClass {
> > >>>>       void (*reset)(VirtIODevice *vdev);
> > >>>>       void (*set_status)(VirtIODevice *vdev, uint8_t val);
> > >>>>       void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
> > >>>> +    void (*queue_enable)(VirtIODevice *vdev, uint32_t queue_index);
> > >>>>       /* For transitional devices, this is a bitmap of features
> > >>>>        * that are only exposed on the legacy interface but not
> > >>>>        * the modern one.
> > >>>> @@ -288,6 +289,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
> > >>>>   int virtio_set_status(VirtIODevice *vdev, uint8_t val);
> > >>>>   void virtio_reset(void *opaque);
> > >>>>   void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
> > >>>> +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index);
> > >>>>   void virtio_update_irq(VirtIODevice *vdev);
> > >>>>   int virtio_set_features(VirtIODevice *vdev, uint64_t val);
> > >>>>
> > >>>> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> > >>>> index cf5f9ca387..9683b2e158 100644
> > >>>> --- a/hw/virtio/virtio.c
> > >>>> +++ b/hw/virtio/virtio.c
> > >>>> @@ -2495,6 +2495,20 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
> > >>>>       __virtio_queue_reset(vdev, queue_index);
> > >>>>   }
> > >>>>
> > >>>> +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
> > >>>> +{
> > >>>> +    VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
> > >>>> +
> > >>>> +    if (!virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
> > >>>> +        error_report("queue_enable is only suppported in devices of virtio "
> > >>>> +                     "1.0 or later.");
> > >>> Why is this triggering here? Maybe virtio_queue_enable() is called too
> > >>> early. I have verified that the Linux guest driver sets VERSION_1. I
> > >>> didn't check what SeaBIOS does.
> > >> Looks like a bug, we should check device features here at least and it
> > >> should be guest errors instead of error_report() here.
> > >>
> > >> Thanks
> > > But it's weird, queue enable is written before guest features?
> > > How come?
> >
> >
> > Or queue_enable is written when the driver doesn't negotiate VERSION_1?
> 
> Is this a bug?
> 
> Or is it allowed in some cases?
> 
> I feel weird too.
> 
> Thanks.

Weren't you able to reproduce?
I suggest
	- write a bios patch to make it spec compliant
	- check UEFI to make sure it's spec compliant
	- ask bios/uefi maintainers whether they can include the patch for this release
	- add a patch to drop the warning - we don't really need it


> >
> > Thanks
> >
> >
> > >
> > >>> $ build/qemu-system-x86_64 -M accel=kvm -m 1G -cpu host -blockdev
> > >>> file,node-name=drive0,filename=test.img -device
> > >>> virtio-blk-pci,drive=drive0
> > >>> qemu: queue_enable is only suppported in devices of virtio 1.0 or later.
> > >>>
> > >>> Stefan
> > >>>
> >



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 29/83] virtio: introduce virtio_queue_enable()
  2022-11-09  7:01           ` Michael S. Tsirkin
@ 2022-11-09  7:11             ` Xuan Zhuo
  0 siblings, 0 replies; 151+ messages in thread
From: Xuan Zhuo @ 2022-11-09  7:11 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Stefan Hajnoczi, Kangjie Xu, qemu-devel, Peter Maydell, Jason Wang

On Wed, 9 Nov 2022 02:01:38 -0500, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Wed, Nov 09, 2022 at 02:48:29PM +0800, Xuan Zhuo wrote:
> > On Wed, 9 Nov 2022 01:39:32 -0500, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > On Wed, Nov 09, 2022 at 11:31:23AM +0800, Jason Wang wrote:
> > > > On Wed, Nov 9, 2022 at 3:43 AM Stefan Hajnoczi <stefanha@gmail.com> wrote:
> > > > >
> > > > > On Mon, 7 Nov 2022 at 18:10, Michael S. Tsirkin <mst@redhat.com> wrote:
> > > > > >
> > > > > > From: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > > > > >
> > > > > > Introduce the interface queue_enable() in VirtioDeviceClass and the
> > > > > > fucntion virtio_queue_enable() in virtio, it can be called when
> > > > > > VIRTIO_PCI_COMMON_Q_ENABLE is written and related virtqueue can be
> > > > > > started. It only supports the devices of virtio 1 or later. The
> > > > > > not-supported devices can only start the virtqueue when DRIVER_OK.
> > > > > >
> > > > > > Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > > > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > > > Acked-by: Jason Wang <jasowang@redhat.com>
> > > > > > Message-Id: <20221017092558.111082-4-xuanzhuo@linux.alibaba.com>
> > > > > > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > > > > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > > > > ---
> > > > > >  include/hw/virtio/virtio.h |  2 ++
> > > > > >  hw/virtio/virtio.c         | 14 ++++++++++++++
> > > > > >  2 files changed, 16 insertions(+)
> > > > > >
> > > > > > diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> > > > > > index 74d76c1dbc..b00b3fcf31 100644
> > > > > > --- a/include/hw/virtio/virtio.h
> > > > > > +++ b/include/hw/virtio/virtio.h
> > > > > > @@ -149,6 +149,7 @@ struct VirtioDeviceClass {
> > > > > >      void (*reset)(VirtIODevice *vdev);
> > > > > >      void (*set_status)(VirtIODevice *vdev, uint8_t val);
> > > > > >      void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
> > > > > > +    void (*queue_enable)(VirtIODevice *vdev, uint32_t queue_index);
> > > > > >      /* For transitional devices, this is a bitmap of features
> > > > > >       * that are only exposed on the legacy interface but not
> > > > > >       * the modern one.
> > > > > > @@ -288,6 +289,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
> > > > > >  int virtio_set_status(VirtIODevice *vdev, uint8_t val);
> > > > > >  void virtio_reset(void *opaque);
> > > > > >  void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
> > > > > > +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index);
> > > > > >  void virtio_update_irq(VirtIODevice *vdev);
> > > > > >  int virtio_set_features(VirtIODevice *vdev, uint64_t val);
> > > > > >
> > > > > > diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> > > > > > index cf5f9ca387..9683b2e158 100644
> > > > > > --- a/hw/virtio/virtio.c
> > > > > > +++ b/hw/virtio/virtio.c
> > > > > > @@ -2495,6 +2495,20 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
> > > > > >      __virtio_queue_reset(vdev, queue_index);
> > > > > >  }
> > > > > >
> > > > > > +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
> > > > > > +{
> > > > > > +    VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
> > > > > > +
> > > > > > +    if (!virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
> > > > > > +        error_report("queue_enable is only suppported in devices of virtio "
> > > > > > +                     "1.0 or later.");
> > > > >
> > > > > Why is this triggering here? Maybe virtio_queue_enable() is called too
> > > > > early. I have verified that the Linux guest driver sets VERSION_1. I
> > > > > didn't check what SeaBIOS does.
> > > >
> > > > Looks like a bug, we should check device features here at least and it
> > > > should be guest errors instead of error_report() here.
> > > >
> > > > Thanks
> > > >
> > >
> > > I suspect we should just drop this print. Kangjie?
> >
> >
> > I think it is.
> >
> > At that time, this inspection was only added at hand, and theoretically it
> > should not be performed.
> >
> > I am responsible for this patch set now.
> >
> > hi, Michael,
> >
> > What should I do, do I send a new version again?
> >
> > Thanks.
>
> I debugged it and replied separately. Can you check EFI drivers too?

OK.

Thanks.


>
> > >
> > >
> > > > > $ build/qemu-system-x86_64 -M accel=kvm -m 1G -cpu host -blockdev
> > > > > file,node-name=drive0,filename=test.img -device
> > > > > virtio-blk-pci,drive=drive0
> > > > > qemu: queue_enable is only suppported in devices of virtio 1.0 or later.
> > > > >
> > > > > Stefan
> > > > >
> > >
>


^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 29/83] virtio: introduce virtio_queue_enable()
  2022-11-09  7:04             ` Michael S. Tsirkin
@ 2022-11-09  7:12               ` Xuan Zhuo
  0 siblings, 0 replies; 151+ messages in thread
From: Xuan Zhuo @ 2022-11-09  7:12 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Jason Wang, Stefan Hajnoczi, Kangjie Xu, qemu-devel, Peter Maydell

On Wed, 9 Nov 2022 02:04:17 -0500, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Wed, Nov 09, 2022 at 02:56:01PM +0800, Xuan Zhuo wrote:
> > On Wed, 9 Nov 2022 14:55:03 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > >
> > > 在 2022/11/9 14:51, Michael S. Tsirkin 写道:
> > > > On Wed, Nov 09, 2022 at 11:31:23AM +0800, Jason Wang wrote:
> > > >> On Wed, Nov 9, 2022 at 3:43 AM Stefan Hajnoczi <stefanha@gmail.com> wrote:
> > > >>> On Mon, 7 Nov 2022 at 18:10, Michael S. Tsirkin <mst@redhat.com> wrote:
> > > >>>> From: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > > >>>>
> > > >>>> Introduce the interface queue_enable() in VirtioDeviceClass and the
> > > >>>> fucntion virtio_queue_enable() in virtio, it can be called when
> > > >>>> VIRTIO_PCI_COMMON_Q_ENABLE is written and related virtqueue can be
> > > >>>> started. It only supports the devices of virtio 1 or later. The
> > > >>>> not-supported devices can only start the virtqueue when DRIVER_OK.
> > > >>>>
> > > >>>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > > >>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > >>>> Acked-by: Jason Wang <jasowang@redhat.com>
> > > >>>> Message-Id: <20221017092558.111082-4-xuanzhuo@linux.alibaba.com>
> > > >>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > > >>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > >>>> ---
> > > >>>>   include/hw/virtio/virtio.h |  2 ++
> > > >>>>   hw/virtio/virtio.c         | 14 ++++++++++++++
> > > >>>>   2 files changed, 16 insertions(+)
> > > >>>>
> > > >>>> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> > > >>>> index 74d76c1dbc..b00b3fcf31 100644
> > > >>>> --- a/include/hw/virtio/virtio.h
> > > >>>> +++ b/include/hw/virtio/virtio.h
> > > >>>> @@ -149,6 +149,7 @@ struct VirtioDeviceClass {
> > > >>>>       void (*reset)(VirtIODevice *vdev);
> > > >>>>       void (*set_status)(VirtIODevice *vdev, uint8_t val);
> > > >>>>       void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
> > > >>>> +    void (*queue_enable)(VirtIODevice *vdev, uint32_t queue_index);
> > > >>>>       /* For transitional devices, this is a bitmap of features
> > > >>>>        * that are only exposed on the legacy interface but not
> > > >>>>        * the modern one.
> > > >>>> @@ -288,6 +289,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
> > > >>>>   int virtio_set_status(VirtIODevice *vdev, uint8_t val);
> > > >>>>   void virtio_reset(void *opaque);
> > > >>>>   void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
> > > >>>> +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index);
> > > >>>>   void virtio_update_irq(VirtIODevice *vdev);
> > > >>>>   int virtio_set_features(VirtIODevice *vdev, uint64_t val);
> > > >>>>
> > > >>>> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> > > >>>> index cf5f9ca387..9683b2e158 100644
> > > >>>> --- a/hw/virtio/virtio.c
> > > >>>> +++ b/hw/virtio/virtio.c
> > > >>>> @@ -2495,6 +2495,20 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
> > > >>>>       __virtio_queue_reset(vdev, queue_index);
> > > >>>>   }
> > > >>>>
> > > >>>> +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
> > > >>>> +{
> > > >>>> +    VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
> > > >>>> +
> > > >>>> +    if (!virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
> > > >>>> +        error_report("queue_enable is only suppported in devices of virtio "
> > > >>>> +                     "1.0 or later.");
> > > >>> Why is this triggering here? Maybe virtio_queue_enable() is called too
> > > >>> early. I have verified that the Linux guest driver sets VERSION_1. I
> > > >>> didn't check what SeaBIOS does.
> > > >> Looks like a bug, we should check device features here at least and it
> > > >> should be guest errors instead of error_report() here.
> > > >>
> > > >> Thanks
> > > > But it's weird, queue enable is written before guest features?
> > > > How come?
> > >
> > >
> > > Or queue_enable is written when the driver doesn't negotiate VERSION_1?
> >
> > Is this a bug?
> >
> > Or is it allowed in some cases?
> >
> > I feel weird too.
> >
> > Thanks.
>
> Weren't you able to reproduce?
> I suggest
> 	- write a bios patch to make it spec compliant
> 	- check UEFI to make sure it's spec compliant
> 	- ask bios/uefi maintainers whether they can include the patch for this release

It looks very interesting, I am happy to study it.

> 	- add a patch to drop the warning - we don't really need it

I sent this patch first.

Thanks.

>
>
> > >
> > > Thanks
> > >
> > >
> > > >
> > > >>> $ build/qemu-system-x86_64 -M accel=kvm -m 1G -cpu host -blockdev
> > > >>> file,node-name=drive0,filename=test.img -device
> > > >>> virtio-blk-pci,drive=drive0
> > > >>> qemu: queue_enable is only suppported in devices of virtio 1.0 or later.
> > > >>>
> > > >>> Stefan
> > > >>>
> > >
>


^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 29/83] virtio: introduce virtio_queue_enable()
  2022-11-09  6:59         ` Michael S. Tsirkin
@ 2022-11-09  8:56           ` Laszlo Ersek
  2022-11-09 11:00             ` Michael S. Tsirkin
  0 siblings, 1 reply; 151+ messages in thread
From: Laszlo Ersek @ 2022-11-09  8:56 UTC (permalink / raw)
  To: Michael S. Tsirkin, Jason Wang
  Cc: Stefan Hajnoczi, Kangjie Xu, qemu-devel, Peter Maydell,
	Xuan Zhuo, Gerd Hoffmann

On 11/09/22 07:59, Michael S. Tsirkin wrote:
> On Wed, Nov 09, 2022 at 01:52:01AM -0500, Michael S. Tsirkin wrote:
>> On Wed, Nov 09, 2022 at 11:31:23AM +0800, Jason Wang wrote:
>>> On Wed, Nov 9, 2022 at 3:43 AM Stefan Hajnoczi <stefanha@gmail.com> wrote:
>>>>
>>>> On Mon, 7 Nov 2022 at 18:10, Michael S. Tsirkin <mst@redhat.com> wrote:
>>>>>
>>>>> From: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>>>>>
>>>>> Introduce the interface queue_enable() in VirtioDeviceClass and the
>>>>> fucntion virtio_queue_enable() in virtio, it can be called when
>>>>> VIRTIO_PCI_COMMON_Q_ENABLE is written and related virtqueue can be
>>>>> started. It only supports the devices of virtio 1 or later. The
>>>>> not-supported devices can only start the virtqueue when DRIVER_OK.
>>>>>
>>>>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>>>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
>>>>> Acked-by: Jason Wang <jasowang@redhat.com>
>>>>> Message-Id: <20221017092558.111082-4-xuanzhuo@linux.alibaba.com>
>>>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
>>>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>>>>> ---
>>>>>  include/hw/virtio/virtio.h |  2 ++
>>>>>  hw/virtio/virtio.c         | 14 ++++++++++++++
>>>>>  2 files changed, 16 insertions(+)
>>>>>
>>>>> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
>>>>> index 74d76c1dbc..b00b3fcf31 100644
>>>>> --- a/include/hw/virtio/virtio.h
>>>>> +++ b/include/hw/virtio/virtio.h
>>>>> @@ -149,6 +149,7 @@ struct VirtioDeviceClass {
>>>>>      void (*reset)(VirtIODevice *vdev);
>>>>>      void (*set_status)(VirtIODevice *vdev, uint8_t val);
>>>>>      void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
>>>>> +    void (*queue_enable)(VirtIODevice *vdev, uint32_t queue_index);
>>>>>      /* For transitional devices, this is a bitmap of features
>>>>>       * that are only exposed on the legacy interface but not
>>>>>       * the modern one.
>>>>> @@ -288,6 +289,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
>>>>>  int virtio_set_status(VirtIODevice *vdev, uint8_t val);
>>>>>  void virtio_reset(void *opaque);
>>>>>  void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
>>>>> +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index);
>>>>>  void virtio_update_irq(VirtIODevice *vdev);
>>>>>  int virtio_set_features(VirtIODevice *vdev, uint64_t val);
>>>>>
>>>>> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
>>>>> index cf5f9ca387..9683b2e158 100644
>>>>> --- a/hw/virtio/virtio.c
>>>>> +++ b/hw/virtio/virtio.c
>>>>> @@ -2495,6 +2495,20 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
>>>>>      __virtio_queue_reset(vdev, queue_index);
>>>>>  }
>>>>>
>>>>> +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
>>>>> +{
>>>>> +    VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
>>>>> +
>>>>> +    if (!virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
>>>>> +        error_report("queue_enable is only suppported in devices of virtio "
>>>>> +                     "1.0 or later.");
>>>>
>>>> Why is this triggering here? Maybe virtio_queue_enable() is called too
>>>> early. I have verified that the Linux guest driver sets VERSION_1. I
>>>> didn't check what SeaBIOS does.
>>>
>>> Looks like a bug, we should check device features here at least and it
>>> should be guest errors instead of error_report() here.
>>>
>>> Thanks
>>
>> But it's weird, queue enable is written before guest features?
>> How come?
>
> It's a bios bug:
>
>
>
>     vp_init_simple(&vdrive->vp, pci);
>     if (vp_find_vq(&vdrive->vp, 0, &vdrive->vq) < 0 ) {
>         dprintf(1, "fail to find vq for virtio-blk %pP\n", pci);
>         goto fail;
>     }
>
>     if (vdrive->vp.use_modern) {
>         struct vp_device *vp = &vdrive->vp;
>         u64 features = vp_get_features(vp);
>         u64 version1 = 1ull << VIRTIO_F_VERSION_1;
>         u64 iommu_platform = 1ull << VIRTIO_F_IOMMU_PLATFORM;
>         u64 blk_size = 1ull << VIRTIO_BLK_F_BLK_SIZE;
>         if (!(features & version1)) {
>             dprintf(1, "modern device without virtio_1 feature bit: %pP\n", pci);
>             goto fail;
>         }
>
>         features = features & (version1 | iommu_platform | blk_size);
>         vp_set_features(vp, features);
>         status |= VIRTIO_CONFIG_S_FEATURES_OK;
>         vp_set_status(vp, status);
>
>
>
> Not good - does not match the spec. Here's what the spec says:
>
>
> The driver MUST follow this sequence to initialize a device:
> 1. Reset the device.
> 2. Set the ACKNOWLEDGE status bit: the guest OS has noticed the device.
> 3. Set the DRIVER status bit: the guest OS knows how to drive the device.
> 4. Read device feature bits, and write the subset of feature bits understood by the OS and driver to the
> device. During this step the driver MAY read (but MUST NOT write) the device-specific configuration
> fields to check that it can support the device before accepting it.
> 5. Set the FEATURES_OK status bit. The driver MUST NOT accept new feature bits after this step.
> 6. Re-read device status to ensure the FEATURES_OK bit is still set: otherwise, the device does not
> support our subset of features and the device is unusable.
> 7. Perform device-specific setup, including discovery of virtqueues for the device, optional per-bus setup,
> reading and possibly writing the device’s virtio configuration space, and population of virtqueues.
> 8. Set the DRIVER_OK status bit. At this point the device is “live”.

Thanks for the Cc.

This should work properly in the edk2 (OVMF) guest drivers. When I
started work on the virtio-1.0 implementation, I noticed that the device
initialization sequence that guest drivers needed to follow had
*changed* from spec version 0.9.5 to spec version 1.0.

For example, in the virtio-rng driver, I addressed that with commit
0a781bdc7f87 ("OvmfPkg: VirtioRngDxe: adapt feature negotiation to
virtio-1.0", 2016-04-06):

  https://github.com/tianocore/edk2/commit/0a781bdc7f87

Therefore we have a larger context like this in edk2 (again the excerpt
is from the virtio-rng driver, but all drivers follow this pattern whose
devices can be either legacy or modern):

>   //
>   // Execute virtio-0.9.5, 2.2.1 Device Initialization Sequence.
>   //
>   NextDevStat = 0;             // step 1 -- reset device
>   Status      = Dev->VirtIo->SetDeviceStatus (Dev->VirtIo, NextDevStat);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }

matches v1.0 spec step 1.

>
>   NextDevStat |= VSTAT_ACK;    // step 2 -- acknowledge device presence
>   Status       = Dev->VirtIo->SetDeviceStatus (Dev->VirtIo, NextDevStat);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }

matches v1.0 spec step 2.

>
>   NextDevStat |= VSTAT_DRIVER; // step 3 -- we know how to drive it
>   Status       = Dev->VirtIo->SetDeviceStatus (Dev->VirtIo, NextDevStat);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }

matches v1.0 spec step 3.

>
>   //
>   // Set Page Size - MMIO VirtIo Specific
>   //
>   Status = Dev->VirtIo->SetPageSize (Dev->VirtIo, EFI_PAGE_SIZE);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   //
>   // step 4a -- retrieve and validate features
>   //
>   Status = Dev->VirtIo->GetDeviceFeatures (Dev->VirtIo, &Features);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }

matches v1.0 spec step 4, the "read" part

>
>   Features &= VIRTIO_F_VERSION_1 | VIRTIO_F_IOMMU_PLATFORM;
>
>   //
>   // In virtio-1.0, feature negotiation is expected to complete before queue
>   // discovery, and the device can also reject the selected set of features.
>   //
>   if (Dev->VirtIo->Revision >= VIRTIO_SPEC_REVISION (1, 0, 0)) {
>     Status = Virtio10WriteFeatures (Dev->VirtIo, Features, &NextDevStat);
>     if (EFI_ERROR (Status)) {
>       goto Failed;
>     }
>   }

This is skipped for virtio-0.9.5. For virtio-1.0,
Virtio10WriteFeatures() does the following:

>   Status = VirtIo->SetGuestFeatures (VirtIo, Features);
>   if (EFI_ERROR (Status)) {
>     return Status;
>   }

Covers v1.0 spec step 4, the rest of it.

>
>   *DeviceStatus |= VSTAT_FEATURES_OK;
>   Status         = VirtIo->SetDeviceStatus (VirtIo, *DeviceStatus);
>   if (EFI_ERROR (Status)) {
>     return Status;
>   }

Covers v1.0 spec step 5.

>
>   Status = VirtIo->GetDeviceStatus (VirtIo, DeviceStatus);
>   if (EFI_ERROR (Status)) {
>     return Status;
>   }
>
>   if ((*DeviceStatus & VSTAT_FEATURES_OK) == 0) {
>     Status = EFI_UNSUPPORTED;
>   }

Covers v1.0 spec step 6.

OK, return to the previous call frame:

>
>   //
>   // step 4b -- allocate request virtqueue, just use #0
>   //
>   Status = Dev->VirtIo->SetQueueSel (Dev->VirtIo, 0);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   Status = Dev->VirtIo->GetQueueNumMax (Dev->VirtIo, &QueueSize);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   //
>   // VirtioRngGetRNG() uses one descriptor
>   //
>   if (QueueSize < 1) {
>     Status = EFI_UNSUPPORTED;
>     goto Failed;
>   }
>
>   Status = VirtioRingInit (Dev->VirtIo, QueueSize, &Dev->Ring);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   //
>   // If anything fails from here on, we must release the ring resources.
>   //
>   Status = VirtioRingMap (
>              Dev->VirtIo,
>              &Dev->Ring,
>              &RingBaseShift,
>              &Dev->RingMap
>              );
>   if (EFI_ERROR (Status)) {
>     goto ReleaseQueue;
>   }
>
>   //
>   // Additional steps for MMIO: align the queue appropriately, and set the
>   // size. If anything fails from here on, we must unmap the ring resources.
>   //
>   Status = Dev->VirtIo->SetQueueNum (Dev->VirtIo, QueueSize);
>   if (EFI_ERROR (Status)) {
>     goto UnmapQueue;
>   }
>
>   Status = Dev->VirtIo->SetQueueAlign (Dev->VirtIo, EFI_PAGE_SIZE);
>   if (EFI_ERROR (Status)) {
>     goto UnmapQueue;
>   }
>
>   //
>   // step 4c -- Report GPFN (guest-physical frame number) of queue.
>   //
>   Status = Dev->VirtIo->SetQueueAddress (
>                           Dev->VirtIo,
>                           &Dev->Ring,
>                           RingBaseShift
>                           );
>   if (EFI_ERROR (Status)) {
>     goto UnmapQueue;
>   }

These cover v1.0 spec step 7.

>
>   //
>   // step 5 -- Report understood features and guest-tuneables.
>   //
>   if (Dev->VirtIo->Revision < VIRTIO_SPEC_REVISION (1, 0, 0)) {
>     Features &= ~(UINT64)(VIRTIO_F_VERSION_1 | VIRTIO_F_IOMMU_PLATFORM);
>     Status    = Dev->VirtIo->SetGuestFeatures (Dev->VirtIo, Features);
>     if (EFI_ERROR (Status)) {
>       goto UnmapQueue;
>     }
>   }

This is exclusive to virtio-0.9.5; the "step 5" reference in the comment
pertains to that spec version. In virtio-0.9.5, this is the location
(just before setting DRIVER_OK) where the guest driver has to report its
desired features (and the device has no means to reject any feature set
that is otherwise a subset of the host feature set).

>
>   //
>   // step 6 -- initialization complete
>   //
>   NextDevStat |= VSTAT_DRIVER_OK;
>   Status       = Dev->VirtIo->SetDeviceStatus (Dev->VirtIo, NextDevStat);
>   if (EFI_ERROR (Status)) {
>     goto UnmapQueue;
>   }

This covers v1.0 spec step 8 (and v0.9.5 spec step 6, as the comment
shows).

Now, in drivers whose devices are virtio-1.0-only, such as
virtio-gpu-pci, vhost-user-fs-pci, this "selectivity" is not there. The
virtio-0.9.5 branch is simply eliminated, and the virtio-1.0 logic is
unconditional. Additionally, in those drivers, the "step" references in
the code comments match the v1.0 bullet list. For example, in the
virtio-fs driver:

> EFI_STATUS
> VirtioFsInit (
>   IN OUT VIRTIO_FS  *VirtioFs
>   )
> {
>   UINT8             NextDevStat;
>   EFI_STATUS        Status;
>   UINT64            Features;
>   VIRTIO_FS_CONFIG  Config;
>   UINTN             Idx;
>   UINT64            RingBaseShift;
>
>   //
>   // Execute virtio-v1.1-cs01-87fa6b5d8155, 3.1.1 Driver Requirements: Device
>   // Initialization.
>   //
>   // 1. Reset the device.
>   //
>   NextDevStat = 0;
>   Status      = VirtioFs->Virtio->SetDeviceStatus (VirtioFs->Virtio, NextDevStat);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   //
>   // 2. Set the ACKNOWLEDGE status bit [...]
>   //
>   NextDevStat |= VSTAT_ACK;
>   Status       = VirtioFs->Virtio->SetDeviceStatus (VirtioFs->Virtio, NextDevStat);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   //
>   // 3. Set the DRIVER status bit [...]
>   //
>   NextDevStat |= VSTAT_DRIVER;
>   Status       = VirtioFs->Virtio->SetDeviceStatus (VirtioFs->Virtio, NextDevStat);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   //
>   // 4. Read device feature bits...
>   //
>   Status = VirtioFs->Virtio->GetDeviceFeatures (VirtioFs->Virtio, &Features);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   if ((Features & VIRTIO_F_VERSION_1) == 0) {
>     Status = EFI_UNSUPPORTED;
>     goto Failed;
>   }
>
>   //
>   // No device-specific feature bits have been defined in file "virtio-fs.tex"
>   // of the virtio spec at <https://github.com/oasis-tcs/virtio-spec.git>, as
>   // of commit 87fa6b5d8155.
>   //
>   Features &= VIRTIO_F_VERSION_1 | VIRTIO_F_IOMMU_PLATFORM;
>
>   //
>   // ... and write the subset of feature bits understood by the [...] driver to
>   // the device. [...]
>   // 5. Set the FEATURES_OK status bit.
>   // 6. Re-read device status to ensure the FEATURES_OK bit is still set [...]
>   //
>   Status = Virtio10WriteFeatures (VirtioFs->Virtio, Features, &NextDevStat);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   //
>   // 7. Perform device-specific setup, including discovery of virtqueues for
>   // the device, [...] reading [...] the device's virtio configuration space
>   //
>   Status = VirtioFsReadConfig (VirtioFs->Virtio, &Config);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   //
>   // 7.a. Convert the filesystem label from UTF-8 to UCS-2. Only labels with
>   // printable ASCII code points (U+0020 through U+007E) are supported.
>   // NUL-terminate at either the terminator we find, or right after the
>   // original label.
>   //
>   for (Idx = 0; Idx < VIRTIO_FS_TAG_BYTES && Config.Tag[Idx] != '\0'; Idx++) {
>     if ((Config.Tag[Idx] < 0x20) || (Config.Tag[Idx] > 0x7E)) {
>       Status = EFI_UNSUPPORTED;
>       goto Failed;
>     }
>
>     VirtioFs->Label[Idx] = Config.Tag[Idx];
>   }
>
>   VirtioFs->Label[Idx] = L'\0';
>
>   //
>   // 7.b. We need one queue for sending normal priority requests.
>   //
>   if (Config.NumReqQueues < 1) {
>     Status = EFI_UNSUPPORTED;
>     goto Failed;
>   }
>
>   //
>   // 7.c. Fetch and remember the number of descriptors we can place on the
>   // queue at once. We'll need two descriptors per request, as a minimum --
>   // request header, response header.
>   //
>   Status = VirtioFs->Virtio->SetQueueSel (
>                                VirtioFs->Virtio,
>                                VIRTIO_FS_REQUEST_QUEUE
>                                );
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   Status = VirtioFs->Virtio->GetQueueNumMax (
>                                VirtioFs->Virtio,
>                                &VirtioFs->QueueSize
>                                );
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   if (VirtioFs->QueueSize < 2) {
>     Status = EFI_UNSUPPORTED;
>     goto Failed;
>   }
>
>   //
>   // 7.d. [...] population of virtqueues [...]
>   //
>   Status = VirtioRingInit (
>              VirtioFs->Virtio,
>              VirtioFs->QueueSize,
>              &VirtioFs->Ring
>              );
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   Status = VirtioRingMap (
>              VirtioFs->Virtio,
>              &VirtioFs->Ring,
>              &RingBaseShift,
>              &VirtioFs->RingMap
>              );
>   if (EFI_ERROR (Status)) {
>     goto ReleaseQueue;
>   }
>
>   Status = VirtioFs->Virtio->SetQueueAddress (
>                                VirtioFs->Virtio,
>                                &VirtioFs->Ring,
>                                RingBaseShift
>                                );
>   if (EFI_ERROR (Status)) {
>     goto UnmapQueue;
>   }
>
>   //
>   // 8. Set the DRIVER_OK status bit.
>   //
>   NextDevStat |= VSTAT_DRIVER_OK;
>   Status       = VirtioFs->Virtio->SetDeviceStatus (VirtioFs->Virtio, NextDevStat);
>   if (EFI_ERROR (Status)) {
>     goto UnmapQueue;
>   }
>
>   return EFI_SUCCESS;
>
> UnmapQueue:
>   VirtioFs->Virtio->UnmapSharedBuffer (VirtioFs->Virtio, VirtioFs->RingMap);
>
> ReleaseQueue:
>   VirtioRingUninit (VirtioFs->Virtio, &VirtioFs->Ring);
>
> Failed:
>   //
>   // If any of these steps go irrecoverably wrong, the driver SHOULD set the
>   // FAILED status bit to indicate that it has given up on the device (it can
>   // reset the device later to restart if desired). [...]
>   //
>   // Virtio access failure here should not mask the original error.
>   //
>   NextDevStat |= VSTAT_FAILED;
>   VirtioFs->Virtio->SetDeviceStatus (VirtioFs->Virtio, NextDevStat);
>
>   return Status;
> }

In the virtio-gpu driver:

> EFI_STATUS
> VirtioGpuInit (
>   IN OUT VGPU_DEV  *VgpuDev
>   )
> {
>   UINT8       NextDevStat;
>   EFI_STATUS  Status;
>   UINT64      Features;
>   UINT16      QueueSize;
>   UINT64      RingBaseShift;
>
>   //
>   // Execute virtio-v1.0-cs04, 3.1.1 Driver Requirements: Device
>   // Initialization.
>   //
>   // 1. Reset the device.
>   //
>   NextDevStat = 0;
>   Status      = VgpuDev->VirtIo->SetDeviceStatus (VgpuDev->VirtIo, NextDevStat);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   //
>   // 2. Set the ACKNOWLEDGE status bit [...]
>   //
>   NextDevStat |= VSTAT_ACK;
>   Status       = VgpuDev->VirtIo->SetDeviceStatus (VgpuDev->VirtIo, NextDevStat);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   //
>   // 3. Set the DRIVER status bit [...]
>   //
>   NextDevStat |= VSTAT_DRIVER;
>   Status       = VgpuDev->VirtIo->SetDeviceStatus (VgpuDev->VirtIo, NextDevStat);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   //
>   // 4. Read device feature bits...
>   //
>   Status = VgpuDev->VirtIo->GetDeviceFeatures (VgpuDev->VirtIo, &Features);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   if ((Features & VIRTIO_F_VERSION_1) == 0) {
>     Status = EFI_UNSUPPORTED;
>     goto Failed;
>   }
>
>   //
>   // We only want the most basic 2D features.
>   //
>   Features &= VIRTIO_F_VERSION_1 | VIRTIO_F_IOMMU_PLATFORM;
>
>   //
>   // ... and write the subset of feature bits understood by the [...] driver to
>   // the device. [...]
>   // 5. Set the FEATURES_OK status bit.
>   // 6. Re-read device status to ensure the FEATURES_OK bit is still set [...]
>   //
>   Status = Virtio10WriteFeatures (VgpuDev->VirtIo, Features, &NextDevStat);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   //
>   // 7. Perform device-specific setup, including discovery of virtqueues for
>   // the device [...]
>   //
>   Status = VgpuDev->VirtIo->SetQueueSel (
>                               VgpuDev->VirtIo,
>                               VIRTIO_GPU_CONTROL_QUEUE
>                               );
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   Status = VgpuDev->VirtIo->GetQueueNumMax (VgpuDev->VirtIo, &QueueSize);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   //
>   // We implement each VirtIo GPU command that we use with two descriptors:
>   // request, response.
>   //
>   if (QueueSize < 2) {
>     Status = EFI_UNSUPPORTED;
>     goto Failed;
>   }
>
>   //
>   // [...] population of virtqueues [...]
>   //
>   Status = VirtioRingInit (VgpuDev->VirtIo, QueueSize, &VgpuDev->Ring);
>   if (EFI_ERROR (Status)) {
>     goto Failed;
>   }
>
>   //
>   // If anything fails from here on, we have to release the ring.
>   //
>   Status = VirtioRingMap (
>              VgpuDev->VirtIo,
>              &VgpuDev->Ring,
>              &RingBaseShift,
>              &VgpuDev->RingMap
>              );
>   if (EFI_ERROR (Status)) {
>     goto ReleaseQueue;
>   }
>
>   //
>   // If anything fails from here on, we have to unmap the ring.
>   //
>   Status = VgpuDev->VirtIo->SetQueueAddress (
>                               VgpuDev->VirtIo,
>                               &VgpuDev->Ring,
>                               RingBaseShift
>                               );
>   if (EFI_ERROR (Status)) {
>     goto UnmapQueue;
>   }
>
>   //
>   // 8. Set the DRIVER_OK status bit.
>   //
>   NextDevStat |= VSTAT_DRIVER_OK;
>   Status       = VgpuDev->VirtIo->SetDeviceStatus (VgpuDev->VirtIo, NextDevStat);
>   if (EFI_ERROR (Status)) {
>     goto UnmapQueue;
>   }
>
>   return EFI_SUCCESS;
>
> UnmapQueue:
>   VgpuDev->VirtIo->UnmapSharedBuffer (VgpuDev->VirtIo, VgpuDev->RingMap);
>
> ReleaseQueue:
>   VirtioRingUninit (VgpuDev->VirtIo, &VgpuDev->Ring);
>
> Failed:
>   //
>   // If any of these steps go irrecoverably wrong, the driver SHOULD set the
>   // FAILED status bit to indicate that it has given up on the device (it can
>   // reset the device later to restart if desired). [...]
>   //
>   // VirtIo access failure here should not mask the original error.
>   //
>   NextDevStat |= VSTAT_FAILED;
>   VgpuDev->VirtIo->SetDeviceStatus (VgpuDev->VirtIo, NextDevStat);
>
>   return Status;
> }

Laszlo



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 29/83] virtio: introduce virtio_queue_enable()
  2022-11-09  8:56           ` Laszlo Ersek
@ 2022-11-09 11:00             ` Michael S. Tsirkin
  0 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-09 11:00 UTC (permalink / raw)
  To: Laszlo Ersek
  Cc: Jason Wang, Stefan Hajnoczi, Kangjie Xu, qemu-devel,
	Peter Maydell, Xuan Zhuo, Gerd Hoffmann

On Wed, Nov 09, 2022 at 09:56:40AM +0100, Laszlo Ersek wrote:
> On 11/09/22 07:59, Michael S. Tsirkin wrote:
> > On Wed, Nov 09, 2022 at 01:52:01AM -0500, Michael S. Tsirkin wrote:
> >> On Wed, Nov 09, 2022 at 11:31:23AM +0800, Jason Wang wrote:
> >>> On Wed, Nov 9, 2022 at 3:43 AM Stefan Hajnoczi <stefanha@gmail.com> wrote:
> >>>>
> >>>> On Mon, 7 Nov 2022 at 18:10, Michael S. Tsirkin <mst@redhat.com> wrote:
> >>>>>
> >>>>> From: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> >>>>>
> >>>>> Introduce the interface queue_enable() in VirtioDeviceClass and the
> >>>>> fucntion virtio_queue_enable() in virtio, it can be called when
> >>>>> VIRTIO_PCI_COMMON_Q_ENABLE is written and related virtqueue can be
> >>>>> started. It only supports the devices of virtio 1 or later. The
> >>>>> not-supported devices can only start the virtqueue when DRIVER_OK.
> >>>>>
> >>>>> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> >>>>> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> >>>>> Acked-by: Jason Wang <jasowang@redhat.com>
> >>>>> Message-Id: <20221017092558.111082-4-xuanzhuo@linux.alibaba.com>
> >>>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> >>>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> >>>>> ---
> >>>>>  include/hw/virtio/virtio.h |  2 ++
> >>>>>  hw/virtio/virtio.c         | 14 ++++++++++++++
> >>>>>  2 files changed, 16 insertions(+)
> >>>>>
> >>>>> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> >>>>> index 74d76c1dbc..b00b3fcf31 100644
> >>>>> --- a/include/hw/virtio/virtio.h
> >>>>> +++ b/include/hw/virtio/virtio.h
> >>>>> @@ -149,6 +149,7 @@ struct VirtioDeviceClass {
> >>>>>      void (*reset)(VirtIODevice *vdev);
> >>>>>      void (*set_status)(VirtIODevice *vdev, uint8_t val);
> >>>>>      void (*queue_reset)(VirtIODevice *vdev, uint32_t queue_index);
> >>>>> +    void (*queue_enable)(VirtIODevice *vdev, uint32_t queue_index);
> >>>>>      /* For transitional devices, this is a bitmap of features
> >>>>>       * that are only exposed on the legacy interface but not
> >>>>>       * the modern one.
> >>>>> @@ -288,6 +289,7 @@ int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n,
> >>>>>  int virtio_set_status(VirtIODevice *vdev, uint8_t val);
> >>>>>  void virtio_reset(void *opaque);
> >>>>>  void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index);
> >>>>> +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index);
> >>>>>  void virtio_update_irq(VirtIODevice *vdev);
> >>>>>  int virtio_set_features(VirtIODevice *vdev, uint64_t val);
> >>>>>
> >>>>> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> >>>>> index cf5f9ca387..9683b2e158 100644
> >>>>> --- a/hw/virtio/virtio.c
> >>>>> +++ b/hw/virtio/virtio.c
> >>>>> @@ -2495,6 +2495,20 @@ void virtio_queue_reset(VirtIODevice *vdev, uint32_t queue_index)
> >>>>>      __virtio_queue_reset(vdev, queue_index);
> >>>>>  }
> >>>>>
> >>>>> +void virtio_queue_enable(VirtIODevice *vdev, uint32_t queue_index)
> >>>>> +{
> >>>>> +    VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
> >>>>> +
> >>>>> +    if (!virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
> >>>>> +        error_report("queue_enable is only suppported in devices of virtio "
> >>>>> +                     "1.0 or later.");
> >>>>
> >>>> Why is this triggering here? Maybe virtio_queue_enable() is called too
> >>>> early. I have verified that the Linux guest driver sets VERSION_1. I
> >>>> didn't check what SeaBIOS does.
> >>>
> >>> Looks like a bug, we should check device features here at least and it
> >>> should be guest errors instead of error_report() here.
> >>>
> >>> Thanks
> >>
> >> But it's weird, queue enable is written before guest features?
> >> How come?
> >
> > It's a bios bug:
> >
> >
> >
> >     vp_init_simple(&vdrive->vp, pci);
> >     if (vp_find_vq(&vdrive->vp, 0, &vdrive->vq) < 0 ) {
> >         dprintf(1, "fail to find vq for virtio-blk %pP\n", pci);
> >         goto fail;
> >     }
> >
> >     if (vdrive->vp.use_modern) {
> >         struct vp_device *vp = &vdrive->vp;
> >         u64 features = vp_get_features(vp);
> >         u64 version1 = 1ull << VIRTIO_F_VERSION_1;
> >         u64 iommu_platform = 1ull << VIRTIO_F_IOMMU_PLATFORM;
> >         u64 blk_size = 1ull << VIRTIO_BLK_F_BLK_SIZE;
> >         if (!(features & version1)) {
> >             dprintf(1, "modern device without virtio_1 feature bit: %pP\n", pci);
> >             goto fail;
> >         }
> >
> >         features = features & (version1 | iommu_platform | blk_size);
> >         vp_set_features(vp, features);
> >         status |= VIRTIO_CONFIG_S_FEATURES_OK;
> >         vp_set_status(vp, status);
> >
> >
> >
> > Not good - does not match the spec. Here's what the spec says:
> >
> >
> > The driver MUST follow this sequence to initialize a device:
> > 1. Reset the device.
> > 2. Set the ACKNOWLEDGE status bit: the guest OS has noticed the device.
> > 3. Set the DRIVER status bit: the guest OS knows how to drive the device.
> > 4. Read device feature bits, and write the subset of feature bits understood by the OS and driver to the
> > device. During this step the driver MAY read (but MUST NOT write) the device-specific configuration
> > fields to check that it can support the device before accepting it.
> > 5. Set the FEATURES_OK status bit. The driver MUST NOT accept new feature bits after this step.
> > 6. Re-read device status to ensure the FEATURES_OK bit is still set: otherwise, the device does not
> > support our subset of features and the device is unusable.
> > 7. Perform device-specific setup, including discovery of virtqueues for the device, optional per-bus setup,
> > reading and possibly writing the device’s virtio configuration space, and population of virtqueues.
> > 8. Set the DRIVER_OK status bit. At this point the device is “live”.
> 
> Thanks for the Cc.
> 
> This should work properly in the edk2 (OVMF) guest drivers. When I
> started work on the virtio-1.0 implementation, I noticed that the device
> initialization sequence that guest drivers needed to follow had
> *changed* from spec version 0.9.5 to spec version 1.0.
> 
> For example, in the virtio-rng driver, I addressed that with commit
> 0a781bdc7f87 ("OvmfPkg: VirtioRngDxe: adapt feature negotiation to
> virtio-1.0", 2016-04-06):
> 
>   https://github.com/tianocore/edk2/commit/0a781bdc7f87


Thanks, and great analysis below!

Yes, in 0.9 it was:

1. Reset the device. This is not required on initial start up.
2. The ACKNOWLEDGE status bit is set: we have noticed the device.
3. The DRIVER status bit is set: we know how to drive the device.
4. Device-specific setup, including reading the Device Feature Bits, discov-
ery of virtqueues for the device, optional MSI-X setup, and reading and
possibly writing the virtio configuration space.
5. The subset of Device Feature Bits understood by the driver is written to
the device.
6. The DRIVER_OK status bit is set.
7. The device can now be used (ie. buffers added to the virtqueues)

Interestingly, Linux stopped following this order even for 0.9.X:

commit ef688e151c00e5d529703be9a04fd506df8bc54e
Author: Rusty Russell <rusty@rustcorp.com.au>
Date:   Fri Jun 12 22:16:35 2009 -0600

    virtio: meet virtio spec by finalizing features before using device
    
    Virtio devices are supposed to negotiate features before they start using
    the device, but the current code doesn't do this.  This is because the
    driver's probe() function invariably has to add buffers to a virtqueue,
    or probe the disk (virtio_blk).
    
    This currently doesn't matter since no existing backend is strict about
    the feature negotiation.  But it's possible to imagine a future feature
    which completely changes how a device operates: in this case, we'd need
    to acknowledge it before using the device.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>





So it's especially unfortunate that bios is still following
the 0.9 sequence even for 1.0.





> Therefore we have a larger context like this in edk2 (again the excerpt
> is from the virtio-rng driver, but all drivers follow this pattern whose
> devices can be either legacy or modern):
> 
> >   //
> >   // Execute virtio-0.9.5, 2.2.1 Device Initialization Sequence.
> >   //
> >   NextDevStat = 0;             // step 1 -- reset device
> >   Status      = Dev->VirtIo->SetDeviceStatus (Dev->VirtIo, NextDevStat);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> 
> matches v1.0 spec step 1.
> 
> >
> >   NextDevStat |= VSTAT_ACK;    // step 2 -- acknowledge device presence
> >   Status       = Dev->VirtIo->SetDeviceStatus (Dev->VirtIo, NextDevStat);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> 
> matches v1.0 spec step 2.
> 
> >
> >   NextDevStat |= VSTAT_DRIVER; // step 3 -- we know how to drive it
> >   Status       = Dev->VirtIo->SetDeviceStatus (Dev->VirtIo, NextDevStat);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> 
> matches v1.0 spec step 3.
> 
> >
> >   //
> >   // Set Page Size - MMIO VirtIo Specific
> >   //
> >   Status = Dev->VirtIo->SetPageSize (Dev->VirtIo, EFI_PAGE_SIZE);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   //
> >   // step 4a -- retrieve and validate features
> >   //
> >   Status = Dev->VirtIo->GetDeviceFeatures (Dev->VirtIo, &Features);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> 
> matches v1.0 spec step 4, the "read" part
> 
> >
> >   Features &= VIRTIO_F_VERSION_1 | VIRTIO_F_IOMMU_PLATFORM;
> >
> >   //
> >   // In virtio-1.0, feature negotiation is expected to complete before queue
> >   // discovery, and the device can also reject the selected set of features.
> >   //
> >   if (Dev->VirtIo->Revision >= VIRTIO_SPEC_REVISION (1, 0, 0)) {
> >     Status = Virtio10WriteFeatures (Dev->VirtIo, Features, &NextDevStat);
> >     if (EFI_ERROR (Status)) {
> >       goto Failed;
> >     }
> >   }
> 
> This is skipped for virtio-0.9.5. For virtio-1.0,
> Virtio10WriteFeatures() does the following:
> 
> >   Status = VirtIo->SetGuestFeatures (VirtIo, Features);
> >   if (EFI_ERROR (Status)) {
> >     return Status;
> >   }
> 
> Covers v1.0 spec step 4, the rest of it.
> 
> >
> >   *DeviceStatus |= VSTAT_FEATURES_OK;
> >   Status         = VirtIo->SetDeviceStatus (VirtIo, *DeviceStatus);
> >   if (EFI_ERROR (Status)) {
> >     return Status;
> >   }
> 
> Covers v1.0 spec step 5.
> 
> >
> >   Status = VirtIo->GetDeviceStatus (VirtIo, DeviceStatus);
> >   if (EFI_ERROR (Status)) {
> >     return Status;
> >   }
> >
> >   if ((*DeviceStatus & VSTAT_FEATURES_OK) == 0) {
> >     Status = EFI_UNSUPPORTED;
> >   }
> 
> Covers v1.0 spec step 6.
> 
> OK, return to the previous call frame:
> 
> >
> >   //
> >   // step 4b -- allocate request virtqueue, just use #0
> >   //
> >   Status = Dev->VirtIo->SetQueueSel (Dev->VirtIo, 0);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   Status = Dev->VirtIo->GetQueueNumMax (Dev->VirtIo, &QueueSize);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   //
> >   // VirtioRngGetRNG() uses one descriptor
> >   //
> >   if (QueueSize < 1) {
> >     Status = EFI_UNSUPPORTED;
> >     goto Failed;
> >   }
> >
> >   Status = VirtioRingInit (Dev->VirtIo, QueueSize, &Dev->Ring);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   //
> >   // If anything fails from here on, we must release the ring resources.
> >   //
> >   Status = VirtioRingMap (
> >              Dev->VirtIo,
> >              &Dev->Ring,
> >              &RingBaseShift,
> >              &Dev->RingMap
> >              );
> >   if (EFI_ERROR (Status)) {
> >     goto ReleaseQueue;
> >   }
> >
> >   //
> >   // Additional steps for MMIO: align the queue appropriately, and set the
> >   // size. If anything fails from here on, we must unmap the ring resources.
> >   //
> >   Status = Dev->VirtIo->SetQueueNum (Dev->VirtIo, QueueSize);
> >   if (EFI_ERROR (Status)) {
> >     goto UnmapQueue;
> >   }
> >
> >   Status = Dev->VirtIo->SetQueueAlign (Dev->VirtIo, EFI_PAGE_SIZE);
> >   if (EFI_ERROR (Status)) {
> >     goto UnmapQueue;
> >   }
> >
> >   //
> >   // step 4c -- Report GPFN (guest-physical frame number) of queue.
> >   //
> >   Status = Dev->VirtIo->SetQueueAddress (
> >                           Dev->VirtIo,
> >                           &Dev->Ring,
> >                           RingBaseShift
> >                           );
> >   if (EFI_ERROR (Status)) {
> >     goto UnmapQueue;
> >   }
> 
> These cover v1.0 spec step 7.
> 
> >
> >   //
> >   // step 5 -- Report understood features and guest-tuneables.
> >   //
> >   if (Dev->VirtIo->Revision < VIRTIO_SPEC_REVISION (1, 0, 0)) {
> >     Features &= ~(UINT64)(VIRTIO_F_VERSION_1 | VIRTIO_F_IOMMU_PLATFORM);
> >     Status    = Dev->VirtIo->SetGuestFeatures (Dev->VirtIo, Features);
> >     if (EFI_ERROR (Status)) {
> >       goto UnmapQueue;
> >     }
> >   }
> 
> This is exclusive to virtio-0.9.5; the "step 5" reference in the comment
> pertains to that spec version. In virtio-0.9.5, this is the location
> (just before setting DRIVER_OK) where the guest driver has to report its
> desired features (and the device has no means to reject any feature set
> that is otherwise a subset of the host feature set).
> 
> >
> >   //
> >   // step 6 -- initialization complete
> >   //
> >   NextDevStat |= VSTAT_DRIVER_OK;
> >   Status       = Dev->VirtIo->SetDeviceStatus (Dev->VirtIo, NextDevStat);
> >   if (EFI_ERROR (Status)) {
> >     goto UnmapQueue;
> >   }
> 
> This covers v1.0 spec step 8 (and v0.9.5 spec step 6, as the comment
> shows).
> 
> Now, in drivers whose devices are virtio-1.0-only, such as
> virtio-gpu-pci, vhost-user-fs-pci, this "selectivity" is not there. The
> virtio-0.9.5 branch is simply eliminated, and the virtio-1.0 logic is
> unconditional. Additionally, in those drivers, the "step" references in
> the code comments match the v1.0 bullet list. For example, in the
> virtio-fs driver:
> 
> > EFI_STATUS
> > VirtioFsInit (
> >   IN OUT VIRTIO_FS  *VirtioFs
> >   )
> > {
> >   UINT8             NextDevStat;
> >   EFI_STATUS        Status;
> >   UINT64            Features;
> >   VIRTIO_FS_CONFIG  Config;
> >   UINTN             Idx;
> >   UINT64            RingBaseShift;
> >
> >   //
> >   // Execute virtio-v1.1-cs01-87fa6b5d8155, 3.1.1 Driver Requirements: Device
> >   // Initialization.
> >   //
> >   // 1. Reset the device.
> >   //
> >   NextDevStat = 0;
> >   Status      = VirtioFs->Virtio->SetDeviceStatus (VirtioFs->Virtio, NextDevStat);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   //
> >   // 2. Set the ACKNOWLEDGE status bit [...]
> >   //
> >   NextDevStat |= VSTAT_ACK;
> >   Status       = VirtioFs->Virtio->SetDeviceStatus (VirtioFs->Virtio, NextDevStat);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   //
> >   // 3. Set the DRIVER status bit [...]
> >   //
> >   NextDevStat |= VSTAT_DRIVER;
> >   Status       = VirtioFs->Virtio->SetDeviceStatus (VirtioFs->Virtio, NextDevStat);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   //
> >   // 4. Read device feature bits...
> >   //
> >   Status = VirtioFs->Virtio->GetDeviceFeatures (VirtioFs->Virtio, &Features);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   if ((Features & VIRTIO_F_VERSION_1) == 0) {
> >     Status = EFI_UNSUPPORTED;
> >     goto Failed;
> >   }
> >
> >   //
> >   // No device-specific feature bits have been defined in file "virtio-fs.tex"
> >   // of the virtio spec at <https://github.com/oasis-tcs/virtio-spec.git>, as
> >   // of commit 87fa6b5d8155.
> >   //
> >   Features &= VIRTIO_F_VERSION_1 | VIRTIO_F_IOMMU_PLATFORM;
> >
> >   //
> >   // ... and write the subset of feature bits understood by the [...] driver to
> >   // the device. [...]
> >   // 5. Set the FEATURES_OK status bit.
> >   // 6. Re-read device status to ensure the FEATURES_OK bit is still set [...]
> >   //
> >   Status = Virtio10WriteFeatures (VirtioFs->Virtio, Features, &NextDevStat);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   //
> >   // 7. Perform device-specific setup, including discovery of virtqueues for
> >   // the device, [...] reading [...] the device's virtio configuration space
> >   //
> >   Status = VirtioFsReadConfig (VirtioFs->Virtio, &Config);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   //
> >   // 7.a. Convert the filesystem label from UTF-8 to UCS-2. Only labels with
> >   // printable ASCII code points (U+0020 through U+007E) are supported.
> >   // NUL-terminate at either the terminator we find, or right after the
> >   // original label.
> >   //
> >   for (Idx = 0; Idx < VIRTIO_FS_TAG_BYTES && Config.Tag[Idx] != '\0'; Idx++) {
> >     if ((Config.Tag[Idx] < 0x20) || (Config.Tag[Idx] > 0x7E)) {
> >       Status = EFI_UNSUPPORTED;
> >       goto Failed;
> >     }
> >
> >     VirtioFs->Label[Idx] = Config.Tag[Idx];
> >   }
> >
> >   VirtioFs->Label[Idx] = L'\0';
> >
> >   //
> >   // 7.b. We need one queue for sending normal priority requests.
> >   //
> >   if (Config.NumReqQueues < 1) {
> >     Status = EFI_UNSUPPORTED;
> >     goto Failed;
> >   }
> >
> >   //
> >   // 7.c. Fetch and remember the number of descriptors we can place on the
> >   // queue at once. We'll need two descriptors per request, as a minimum --
> >   // request header, response header.
> >   //
> >   Status = VirtioFs->Virtio->SetQueueSel (
> >                                VirtioFs->Virtio,
> >                                VIRTIO_FS_REQUEST_QUEUE
> >                                );
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   Status = VirtioFs->Virtio->GetQueueNumMax (
> >                                VirtioFs->Virtio,
> >                                &VirtioFs->QueueSize
> >                                );
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   if (VirtioFs->QueueSize < 2) {
> >     Status = EFI_UNSUPPORTED;
> >     goto Failed;
> >   }
> >
> >   //
> >   // 7.d. [...] population of virtqueues [...]
> >   //
> >   Status = VirtioRingInit (
> >              VirtioFs->Virtio,
> >              VirtioFs->QueueSize,
> >              &VirtioFs->Ring
> >              );
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   Status = VirtioRingMap (
> >              VirtioFs->Virtio,
> >              &VirtioFs->Ring,
> >              &RingBaseShift,
> >              &VirtioFs->RingMap
> >              );
> >   if (EFI_ERROR (Status)) {
> >     goto ReleaseQueue;
> >   }
> >
> >   Status = VirtioFs->Virtio->SetQueueAddress (
> >                                VirtioFs->Virtio,
> >                                &VirtioFs->Ring,
> >                                RingBaseShift
> >                                );
> >   if (EFI_ERROR (Status)) {
> >     goto UnmapQueue;
> >   }
> >
> >   //
> >   // 8. Set the DRIVER_OK status bit.
> >   //
> >   NextDevStat |= VSTAT_DRIVER_OK;
> >   Status       = VirtioFs->Virtio->SetDeviceStatus (VirtioFs->Virtio, NextDevStat);
> >   if (EFI_ERROR (Status)) {
> >     goto UnmapQueue;
> >   }
> >
> >   return EFI_SUCCESS;
> >
> > UnmapQueue:
> >   VirtioFs->Virtio->UnmapSharedBuffer (VirtioFs->Virtio, VirtioFs->RingMap);
> >
> > ReleaseQueue:
> >   VirtioRingUninit (VirtioFs->Virtio, &VirtioFs->Ring);
> >
> > Failed:
> >   //
> >   // If any of these steps go irrecoverably wrong, the driver SHOULD set the
> >   // FAILED status bit to indicate that it has given up on the device (it can
> >   // reset the device later to restart if desired). [...]
> >   //
> >   // Virtio access failure here should not mask the original error.
> >   //
> >   NextDevStat |= VSTAT_FAILED;
> >   VirtioFs->Virtio->SetDeviceStatus (VirtioFs->Virtio, NextDevStat);
> >
> >   return Status;
> > }
> 
> In the virtio-gpu driver:
> 
> > EFI_STATUS
> > VirtioGpuInit (
> >   IN OUT VGPU_DEV  *VgpuDev
> >   )
> > {
> >   UINT8       NextDevStat;
> >   EFI_STATUS  Status;
> >   UINT64      Features;
> >   UINT16      QueueSize;
> >   UINT64      RingBaseShift;
> >
> >   //
> >   // Execute virtio-v1.0-cs04, 3.1.1 Driver Requirements: Device
> >   // Initialization.
> >   //
> >   // 1. Reset the device.
> >   //
> >   NextDevStat = 0;
> >   Status      = VgpuDev->VirtIo->SetDeviceStatus (VgpuDev->VirtIo, NextDevStat);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   //
> >   // 2. Set the ACKNOWLEDGE status bit [...]
> >   //
> >   NextDevStat |= VSTAT_ACK;
> >   Status       = VgpuDev->VirtIo->SetDeviceStatus (VgpuDev->VirtIo, NextDevStat);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   //
> >   // 3. Set the DRIVER status bit [...]
> >   //
> >   NextDevStat |= VSTAT_DRIVER;
> >   Status       = VgpuDev->VirtIo->SetDeviceStatus (VgpuDev->VirtIo, NextDevStat);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   //
> >   // 4. Read device feature bits...
> >   //
> >   Status = VgpuDev->VirtIo->GetDeviceFeatures (VgpuDev->VirtIo, &Features);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   if ((Features & VIRTIO_F_VERSION_1) == 0) {
> >     Status = EFI_UNSUPPORTED;
> >     goto Failed;
> >   }
> >
> >   //
> >   // We only want the most basic 2D features.
> >   //
> >   Features &= VIRTIO_F_VERSION_1 | VIRTIO_F_IOMMU_PLATFORM;
> >
> >   //
> >   // ... and write the subset of feature bits understood by the [...] driver to
> >   // the device. [...]
> >   // 5. Set the FEATURES_OK status bit.
> >   // 6. Re-read device status to ensure the FEATURES_OK bit is still set [...]
> >   //
> >   Status = Virtio10WriteFeatures (VgpuDev->VirtIo, Features, &NextDevStat);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   //
> >   // 7. Perform device-specific setup, including discovery of virtqueues for
> >   // the device [...]
> >   //
> >   Status = VgpuDev->VirtIo->SetQueueSel (
> >                               VgpuDev->VirtIo,
> >                               VIRTIO_GPU_CONTROL_QUEUE
> >                               );
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   Status = VgpuDev->VirtIo->GetQueueNumMax (VgpuDev->VirtIo, &QueueSize);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   //
> >   // We implement each VirtIo GPU command that we use with two descriptors:
> >   // request, response.
> >   //
> >   if (QueueSize < 2) {
> >     Status = EFI_UNSUPPORTED;
> >     goto Failed;
> >   }
> >
> >   //
> >   // [...] population of virtqueues [...]
> >   //
> >   Status = VirtioRingInit (VgpuDev->VirtIo, QueueSize, &VgpuDev->Ring);
> >   if (EFI_ERROR (Status)) {
> >     goto Failed;
> >   }
> >
> >   //
> >   // If anything fails from here on, we have to release the ring.
> >   //
> >   Status = VirtioRingMap (
> >              VgpuDev->VirtIo,
> >              &VgpuDev->Ring,
> >              &RingBaseShift,
> >              &VgpuDev->RingMap
> >              );
> >   if (EFI_ERROR (Status)) {
> >     goto ReleaseQueue;
> >   }
> >
> >   //
> >   // If anything fails from here on, we have to unmap the ring.
> >   //
> >   Status = VgpuDev->VirtIo->SetQueueAddress (
> >                               VgpuDev->VirtIo,
> >                               &VgpuDev->Ring,
> >                               RingBaseShift
> >                               );
> >   if (EFI_ERROR (Status)) {
> >     goto UnmapQueue;
> >   }
> >
> >   //
> >   // 8. Set the DRIVER_OK status bit.
> >   //
> >   NextDevStat |= VSTAT_DRIVER_OK;
> >   Status       = VgpuDev->VirtIo->SetDeviceStatus (VgpuDev->VirtIo, NextDevStat);
> >   if (EFI_ERROR (Status)) {
> >     goto UnmapQueue;
> >   }
> >
> >   return EFI_SUCCESS;
> >
> > UnmapQueue:
> >   VgpuDev->VirtIo->UnmapSharedBuffer (VgpuDev->VirtIo, VgpuDev->RingMap);
> >
> > ReleaseQueue:
> >   VirtioRingUninit (VgpuDev->VirtIo, &VgpuDev->Ring);
> >
> > Failed:
> >   //
> >   // If any of these steps go irrecoverably wrong, the driver SHOULD set the
> >   // FAILED status bit to indicate that it has given up on the device (it can
> >   // reset the device later to restart if desired). [...]
> >   //
> >   // VirtIo access failure here should not mask the original error.
> >   //
> >   NextDevStat |= VSTAT_FAILED;
> >   VgpuDev->VirtIo->SetDeviceStatus (VgpuDev->VirtIo, NextDevStat);
> >
> >   return Status;
> > }
> 
> Laszlo



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 44/83] acpi: pc: vga: use AcpiDevAmlIf interface to build VGA device descriptors
  2022-11-07 22:51 ` [PULL v4 44/83] acpi: pc: vga: use AcpiDevAmlIf interface to build VGA device descriptors Michael S. Tsirkin
@ 2022-11-09 17:39   ` Laurent Vivier
  2022-11-09 21:42     ` Michael S. Tsirkin
  2022-11-10  2:55     ` Ani Sinha
  0 siblings, 2 replies; 151+ messages in thread
From: Laurent Vivier @ 2022-11-09 17:39 UTC (permalink / raw)
  To: Michael S. Tsirkin, qemu-devel
  Cc: Peter Maydell, Igor Mammedov, Ani Sinha, Bernhard Beschow,
	Gerd Hoffmann, Marcel Apfelbaum, Paolo Bonzini,
	Richard Henderson, Eduardo Habkost

This one breaks something for me:

[3/65] Compiling C object libhw-display-virtio-vga-gl.a.p/hw_display_acpi-vga.c.o
FAILED: libhw-display-virtio-vga-gl.a.p/hw_display_acpi-vga.c.o
clang -m64 -mcx16 -Ilibhw-display-virtio-vga-gl.a.p -I. -I../../../Projects/qemu-upstream 
-Iqapi -Itrace -Iui -Iui/shader -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include 
-I/usr/include/sysprof-4 -fcolor-diagnostics -Wall -Winvalid-pch -Werror -std=gnu11 -O2 -g 
-isystem /home/lvivier/Projects/qemu-upstream/linux-headers -isystem linux-headers -iquote 
. -iquote /home/lvivier/Projects/qemu-upstream -iquote 
/home/lvivier/Projects/qemu-upstream/include -iquote 
/home/lvivier/Projects/qemu-upstream/tcg/i386 -pthread -D_GNU_SOURCE 
-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wundef 
-Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv 
-Wold-style-definition -Wtype-limits -Wformat-security -Wformat-y2k -Winit-self 
-Wignored-qualifiers -Wempty-body -Wnested-externs -Wendif-labels -Wexpansion-to-defined 
-Wno-initializer-overrides -Wno-missing-include-dirs -Wno-shift-negative-value 
-Wno-string-plus-int -Wno-typedef-redefinition -Wno-tautological-type-limit-compare 
-Wno-psabi -Wno-gnu-variable-sized-type-not-at-end -fstack-protector-strong 
-fsanitize=address -fPIC -DBUILD_DSO -MD -MQ 
libhw-display-virtio-vga-gl.a.p/hw_display_acpi-vga.c.o -MF 
libhw-display-virtio-vga-gl.a.p/hw_display_acpi-vga.c.o.d -o 
libhw-display-virtio-vga-gl.a.p/hw_display_acpi-vga.c.o -c 
../../../Projects/qemu-upstream/hw/display/acpi-vga.c
In file included from ../../../Projects/qemu-upstream/hw/display/acpi-vga.c:4:
In file included from ../../../Projects/qemu-upstream/hw/display/vga_int.h:30:
In file included from /home/lvivier/Projects/qemu-upstream/include/ui/console.h:4:
/home/lvivier/Projects/qemu-upstream/include/ui/qemu-pixman.h:12:10: fatal error: 
'pixman.h' file not found
#include <pixman.h>
          ^~~~~~~~~~
1 error generated.
ninja: build stopped: subcommand failed.
make: *** [Makefile:165: run-ninja] Error 1
make: Leaving directory '/home/lvivier/Objects/qemu-upstream/x86_64'

Any idea?

my configure is:

configure' '--disable-docs' '--target-list=x86_64-softmmu' '--enable-modules' 
'--disable-spice' '--enable-docs'

path to pixman.h is:

/usr/include/pixman-1/pixman.h

Thanks,
Laurent

On 11/7/22 23:51, Michael S. Tsirkin wrote:
> From: Igor Mammedov <imammedo@redhat.com>
> 
> NB:
> We do not expect any functional change in any ACPI tables with this
> change. It's only a refactoring.
> 
> NB2:
> Some targets (or1k) do not support acpi and CONFIG_ACPI is off for them.
> However, modules are reused between all architectures so CONFIG_ACPI is
> on.  For those architectures, dummy stub function definitions help to
> resolve symbols.  This change uses more of these and so it adds a couple
> of dummy stub definitions so that symbols for those can be resolved.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> Message-Id: <20221017102146.2254096-2-imammedo@redhat.com>
> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> Reviewed-by: Ani Sinha <ani@anisinha.ca>
> CC: Bernhard Beschow <shentey@gmail.com>
> Signed-off-by: Ani Sinha <ani@anisinha.ca>
> Message-Id: <20221107152744.868434-1-ani@anisinha.ca>
> ---
>   hw/display/vga_int.h       |  2 ++
>   hw/acpi/aml-build-stub.c   | 10 ++++++++++
>   hw/display/acpi-vga-stub.c |  7 +++++++
>   hw/display/acpi-vga.c      | 26 ++++++++++++++++++++++++++
>   hw/display/vga-pci.c       |  4 ++++
>   hw/i386/acpi-build.c       | 26 +-------------------------
>   hw/display/meson.build     | 17 +++++++++++++++++
>   7 files changed, 67 insertions(+), 25 deletions(-)
>   create mode 100644 hw/display/acpi-vga-stub.c
>   create mode 100644 hw/display/acpi-vga.c
> 
> diff --git a/hw/display/vga_int.h b/hw/display/vga_int.h
> index 305e700014..330406ad9c 100644
> --- a/hw/display/vga_int.h
> +++ b/hw/display/vga_int.h
> @@ -30,6 +30,7 @@
>   #include "ui/console.h"
>   
>   #include "hw/display/bochs-vbe.h"
> +#include "hw/acpi/acpi_aml_interface.h"
>   
>   #define ST01_V_RETRACE      0x08
>   #define ST01_DISP_ENABLE    0x01
> @@ -195,4 +196,5 @@ void pci_std_vga_mmio_region_init(VGACommonState *s,
>                                     MemoryRegion *subs,
>                                     bool qext, bool edid);
>   
> +void build_vga_aml(AcpiDevAmlIf *adev, Aml *scope);
>   #endif
> diff --git a/hw/acpi/aml-build-stub.c b/hw/acpi/aml-build-stub.c
> index 8d8ad1a314..89a8fec4af 100644
> --- a/hw/acpi/aml-build-stub.c
> +++ b/hw/acpi/aml-build-stub.c
> @@ -26,6 +26,16 @@ void aml_append(Aml *parent_ctx, Aml *child)
>   {
>   }
>   
> +Aml *aml_return(Aml *val)
> +{
> +    return NULL;
> +}
> +
> +Aml *aml_method(const char *name, int arg_count, AmlSerializeFlag sflag)
> +{
> +    return NULL;
> +}
> +
>   Aml *aml_resource_template(void)
>   {
>       return NULL;
> diff --git a/hw/display/acpi-vga-stub.c b/hw/display/acpi-vga-stub.c
> new file mode 100644
> index 0000000000..a9b0ecf76d
> --- /dev/null
> +++ b/hw/display/acpi-vga-stub.c
> @@ -0,0 +1,7 @@
> +#include "qemu/osdep.h"
> +#include "hw/acpi/acpi_aml_interface.h"
> +#include "vga_int.h"
> +
> +void build_vga_aml(AcpiDevAmlIf *adev, Aml *scope)
> +{
> +}
> diff --git a/hw/display/acpi-vga.c b/hw/display/acpi-vga.c
> new file mode 100644
> index 0000000000..f0e9ef1fcf
> --- /dev/null
> +++ b/hw/display/acpi-vga.c
> @@ -0,0 +1,26 @@
> +#include "qemu/osdep.h"
> +#include "hw/acpi/acpi_aml_interface.h"
> +#include "hw/pci/pci.h"
> +#include "vga_int.h"
> +
> +void build_vga_aml(AcpiDevAmlIf *adev, Aml *scope)
> +{
> +    int s3d = 0;
> +    Aml *method;
> +
> +    if (object_dynamic_cast(OBJECT(adev), "qxl-vga")) {
> +        s3d = 3;
> +    }
> +
> +    method = aml_method("_S1D", 0, AML_NOTSERIALIZED);
> +    aml_append(method, aml_return(aml_int(0)));
> +    aml_append(scope, method);
> +
> +    method = aml_method("_S2D", 0, AML_NOTSERIALIZED);
> +    aml_append(method, aml_return(aml_int(0)));
> +    aml_append(scope, method);
> +
> +    method = aml_method("_S3D", 0, AML_NOTSERIALIZED);
> +    aml_append(method, aml_return(aml_int(s3d)));
> +    aml_append(scope, method);
> +}
> diff --git a/hw/display/vga-pci.c b/hw/display/vga-pci.c
> index 3e5bc259f7..9a91de7ed1 100644
> --- a/hw/display/vga-pci.c
> +++ b/hw/display/vga-pci.c
> @@ -35,6 +35,7 @@
>   #include "hw/loader.h"
>   #include "hw/display/edid.h"
>   #include "qom/object.h"
> +#include "hw/acpi/acpi_aml_interface.h"
>   
>   enum vga_pci_flags {
>       PCI_VGA_FLAG_ENABLE_MMIO = 1,
> @@ -354,11 +355,13 @@ static void vga_pci_class_init(ObjectClass *klass, void *data)
>   {
>       DeviceClass *dc = DEVICE_CLASS(klass);
>       PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
> +    AcpiDevAmlIfClass *adevc = ACPI_DEV_AML_IF_CLASS(klass);
>   
>       k->vendor_id = PCI_VENDOR_ID_QEMU;
>       k->device_id = PCI_DEVICE_ID_QEMU_VGA;
>       dc->vmsd = &vmstate_vga_pci;
>       set_bit(DEVICE_CATEGORY_DISPLAY, dc->categories);
> +    adevc->build_dev_aml = build_vga_aml;
>   }
>   
>   static const TypeInfo vga_pci_type_info = {
> @@ -369,6 +372,7 @@ static const TypeInfo vga_pci_type_info = {
>       .class_init = vga_pci_class_init,
>       .interfaces = (InterfaceInfo[]) {
>           { INTERFACE_CONVENTIONAL_PCI_DEVICE },
> +        { TYPE_ACPI_DEV_AML_IF },
>           { },
>       },
>   };
> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> index 4f54b61904..26932b4e2c 100644
> --- a/hw/i386/acpi-build.c
> +++ b/hw/i386/acpi-build.c
> @@ -430,7 +430,6 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
>           bool hotpluggbale_slot = false;
>           bool bridge_in_acpi = false;
>           bool cold_plugged_bridge = false;
> -        bool is_vga = false;
>   
>           if (pdev) {
>               pc = PCI_DEVICE_GET_CLASS(pdev);
> @@ -440,8 +439,6 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
>                   continue;
>               }
>   
> -            is_vga = pc->class_id == PCI_CLASS_DISPLAY_VGA;
> -
>               /*
>                * Cold plugged bridges aren't themselves hot-pluggable.
>                * Hotplugged bridges *are* hot-pluggable.
> @@ -489,28 +486,7 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
>               aml_append(dev, aml_pci_device_dsm());
>           }
>   
> -        if (is_vga) {
> -            /* add VGA specific AML methods */
> -            int s3d;
> -
> -            if (object_dynamic_cast(OBJECT(pdev), "qxl-vga")) {
> -                s3d = 3;
> -            } else {
> -                s3d = 0;
> -            }
> -
> -            method = aml_method("_S1D", 0, AML_NOTSERIALIZED);
> -            aml_append(method, aml_return(aml_int(0)));
> -            aml_append(dev, method);
> -
> -            method = aml_method("_S2D", 0, AML_NOTSERIALIZED);
> -            aml_append(method, aml_return(aml_int(0)));
> -            aml_append(dev, method);
> -
> -            method = aml_method("_S3D", 0, AML_NOTSERIALIZED);
> -            aml_append(method, aml_return(aml_int(s3d)));
> -            aml_append(dev, method);
> -        }
> +        call_dev_aml_func(DEVICE(pdev), dev);
>   
>           bridge_in_acpi =  cold_plugged_bridge && pcihp_bridge_en;
>           if (bridge_in_acpi) {
> diff --git a/hw/display/meson.build b/hw/display/meson.build
> index adc53dd8b6..7a725ed80e 100644
> --- a/hw/display/meson.build
> +++ b/hw/display/meson.build
> @@ -38,10 +38,21 @@ softmmu_ss.add(when: 'CONFIG_NEXTCUBE', if_true: files('next-fb.c'))
>   
>   specific_ss.add(when: 'CONFIG_VGA', if_true: files('vga.c'))
>   
> +if (config_all_devices.has_key('CONFIG_VGA_CIRRUS') or
> +    config_all_devices.has_key('CONFIG_VGA_PCI') or
> +    config_all_devices.has_key('CONFIG_VMWARE_VGA') or
> +    config_all_devices.has_key('CONFIG_ATI_VGA')
> +   )
> +  softmmu_ss.add(when: 'CONFIG_ACPI', if_true: files('acpi-vga.c'),
> +                                      if_false: files('acpi-vga-stub.c'))
> +endif
> +
>   if config_all_devices.has_key('CONFIG_QXL')
>     qxl_ss = ss.source_set()
>     qxl_ss.add(when: 'CONFIG_QXL', if_true: [files('qxl.c', 'qxl-logger.c', 'qxl-render.c'),
>                                              pixman, spice])
> +  qxl_ss.add(when: 'CONFIG_ACPI', if_true: files('acpi-vga.c'),
> +                                  if_false: files('acpi-vga-stub.c'))
>     hw_display_modules += {'qxl': qxl_ss}
>   endif
>   
> @@ -52,6 +63,7 @@ softmmu_ss.add(when: 'CONFIG_ARTIST', if_true: files('artist.c'))
>   
>   softmmu_ss.add(when: [pixman, 'CONFIG_ATI_VGA'], if_true: files('ati.c', 'ati_2d.c', 'ati_dbg.c'))
>   
> +
>   if config_all_devices.has_key('CONFIG_VIRTIO_GPU')
>     virtio_gpu_ss = ss.source_set()
>     virtio_gpu_ss.add(when: 'CONFIG_VIRTIO_GPU',
> @@ -87,14 +99,19 @@ if config_all_devices.has_key('CONFIG_VIRTIO_VGA')
>                       if_true: [files('virtio-vga.c'), pixman])
>     virtio_vga_ss.add(when: 'CONFIG_VHOST_USER_VGA',
>                       if_true: files('vhost-user-vga.c'))
> +  virtio_vga_ss.add(when: 'CONFIG_ACPI', if_true: files('acpi-vga.c'),
> +                                         if_false: files('acpi-vga-stub.c'))
>     hw_display_modules += {'virtio-vga': virtio_vga_ss}
>   
>     virtio_vga_gl_ss = ss.source_set()
>     virtio_vga_gl_ss.add(when: ['CONFIG_VIRTIO_VGA', virgl, opengl],
>                          if_true: [files('virtio-vga-gl.c'), pixman])
> +  virtio_vga_gl_ss.add(when: 'CONFIG_ACPI', if_true: files('acpi-vga.c'),
> +                                            if_false: files('acpi-vga-stub.c'))
>     hw_display_modules += {'virtio-vga-gl': virtio_vga_gl_ss}
>   endif
>   
>   specific_ss.add(when: 'CONFIG_OMAP', if_true: files('omap_lcdc.c'))
>   
> +softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('acpi-vga-stub.c'))
>   modules += { 'hw-display': hw_display_modules }



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 44/83] acpi: pc: vga: use AcpiDevAmlIf interface to build VGA device descriptors
  2022-11-09 17:39   ` Laurent Vivier
@ 2022-11-09 21:42     ` Michael S. Tsirkin
  2022-11-10  9:28       ` Peter Maydell
  2022-11-10  2:55     ` Ani Sinha
  1 sibling, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-09 21:42 UTC (permalink / raw)
  To: Laurent Vivier
  Cc: qemu-devel, Peter Maydell, Igor Mammedov, Ani Sinha,
	Bernhard Beschow, Gerd Hoffmann, Marcel Apfelbaum, Paolo Bonzini,
	Richard Henderson, Eduardo Habkost

On Wed, Nov 09, 2022 at 06:39:27PM +0100, Laurent Vivier wrote:
> This one breaks something for me:
> 
> [3/65] Compiling C object libhw-display-virtio-vga-gl.a.p/hw_display_acpi-vga.c.o
> FAILED: libhw-display-virtio-vga-gl.a.p/hw_display_acpi-vga.c.o
> clang -m64 -mcx16 -Ilibhw-display-virtio-vga-gl.a.p -I.
> -I../../../Projects/qemu-upstream -Iqapi -Itrace -Iui -Iui/shader
> -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include
> -I/usr/include/sysprof-4 -fcolor-diagnostics -Wall -Winvalid-pch -Werror
> -std=gnu11 -O2 -g -isystem
> /home/lvivier/Projects/qemu-upstream/linux-headers -isystem linux-headers
> -iquote . -iquote /home/lvivier/Projects/qemu-upstream -iquote
> /home/lvivier/Projects/qemu-upstream/include -iquote
> /home/lvivier/Projects/qemu-upstream/tcg/i386 -pthread -D_GNU_SOURCE
> -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes
> -Wredundant-decls -Wundef -Wwrite-strings -Wmissing-prototypes
> -fno-strict-aliasing -fno-common -fwrapv -Wold-style-definition
> -Wtype-limits -Wformat-security -Wformat-y2k -Winit-self
> -Wignored-qualifiers -Wempty-body -Wnested-externs -Wendif-labels
> -Wexpansion-to-defined -Wno-initializer-overrides -Wno-missing-include-dirs
> -Wno-shift-negative-value -Wno-string-plus-int -Wno-typedef-redefinition
> -Wno-tautological-type-limit-compare -Wno-psabi
> -Wno-gnu-variable-sized-type-not-at-end -fstack-protector-strong
> -fsanitize=address -fPIC -DBUILD_DSO -MD -MQ
> libhw-display-virtio-vga-gl.a.p/hw_display_acpi-vga.c.o -MF
> libhw-display-virtio-vga-gl.a.p/hw_display_acpi-vga.c.o.d -o
> libhw-display-virtio-vga-gl.a.p/hw_display_acpi-vga.c.o -c
> ../../../Projects/qemu-upstream/hw/display/acpi-vga.c
> In file included from ../../../Projects/qemu-upstream/hw/display/acpi-vga.c:4:
> In file included from ../../../Projects/qemu-upstream/hw/display/vga_int.h:30:
> In file included from /home/lvivier/Projects/qemu-upstream/include/ui/console.h:4:
> /home/lvivier/Projects/qemu-upstream/include/ui/qemu-pixman.h:12:10: fatal
> error: 'pixman.h' file not found
> #include <pixman.h>
>          ^~~~~~~~~~
> 1 error generated.
> ninja: build stopped: subcommand failed.
> make: *** [Makefile:165: run-ninja] Error 1
> make: Leaving directory '/home/lvivier/Objects/qemu-upstream/x86_64'
> 
> Any idea?
> 
> my configure is:
> 
> configure' '--disable-docs' '--target-list=x86_64-softmmu'
> '--enable-modules' '--disable-spice' '--enable-docs'
> 
> path to pixman.h is:
> 
> /usr/include/pixman-1/pixman.h
> 
> Thanks,
> Laurent
> 
> On 11/7/22 23:51, Michael S. Tsirkin wrote:
> > From: Igor Mammedov <imammedo@redhat.com>
> > 
> > NB:
> > We do not expect any functional change in any ACPI tables with this
> > change. It's only a refactoring.
> > 
> > NB2:
> > Some targets (or1k) do not support acpi and CONFIG_ACPI is off for them.
> > However, modules are reused between all architectures so CONFIG_ACPI is
> > on.  For those architectures, dummy stub function definitions help to
> > resolve symbols.  This change uses more of these and so it adds a couple
> > of dummy stub definitions so that symbols for those can be resolved.
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > Message-Id: <20221017102146.2254096-2-imammedo@redhat.com>
> > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > Reviewed-by: Ani Sinha <ani@anisinha.ca>
> > CC: Bernhard Beschow <shentey@gmail.com>
> > Signed-off-by: Ani Sinha <ani@anisinha.ca>
> > Message-Id: <20221107152744.868434-1-ani@anisinha.ca>
> > ---
> >   hw/display/vga_int.h       |  2 ++
> >   hw/acpi/aml-build-stub.c   | 10 ++++++++++
> >   hw/display/acpi-vga-stub.c |  7 +++++++
> >   hw/display/acpi-vga.c      | 26 ++++++++++++++++++++++++++
> >   hw/display/vga-pci.c       |  4 ++++
> >   hw/i386/acpi-build.c       | 26 +-------------------------
> >   hw/display/meson.build     | 17 +++++++++++++++++
> >   7 files changed, 67 insertions(+), 25 deletions(-)
> >   create mode 100644 hw/display/acpi-vga-stub.c
> >   create mode 100644 hw/display/acpi-vga.c
> > 
> > diff --git a/hw/display/vga_int.h b/hw/display/vga_int.h
> > index 305e700014..330406ad9c 100644
> > --- a/hw/display/vga_int.h
> > +++ b/hw/display/vga_int.h
> > @@ -30,6 +30,7 @@
> >   #include "ui/console.h"
> >   #include "hw/display/bochs-vbe.h"
> > +#include "hw/acpi/acpi_aml_interface.h"
> >   #define ST01_V_RETRACE      0x08
> >   #define ST01_DISP_ENABLE    0x01
> > @@ -195,4 +196,5 @@ void pci_std_vga_mmio_region_init(VGACommonState *s,
> >                                     MemoryRegion *subs,
> >                                     bool qext, bool edid);
> > +void build_vga_aml(AcpiDevAmlIf *adev, Aml *scope);
> >   #endif
> > diff --git a/hw/acpi/aml-build-stub.c b/hw/acpi/aml-build-stub.c
> > index 8d8ad1a314..89a8fec4af 100644
> > --- a/hw/acpi/aml-build-stub.c
> > +++ b/hw/acpi/aml-build-stub.c
> > @@ -26,6 +26,16 @@ void aml_append(Aml *parent_ctx, Aml *child)
> >   {
> >   }
> > +Aml *aml_return(Aml *val)
> > +{
> > +    return NULL;
> > +}
> > +
> > +Aml *aml_method(const char *name, int arg_count, AmlSerializeFlag sflag)
> > +{
> > +    return NULL;
> > +}
> > +
> >   Aml *aml_resource_template(void)
> >   {
> >       return NULL;
> > diff --git a/hw/display/acpi-vga-stub.c b/hw/display/acpi-vga-stub.c
> > new file mode 100644
> > index 0000000000..a9b0ecf76d
> > --- /dev/null
> > +++ b/hw/display/acpi-vga-stub.c
> > @@ -0,0 +1,7 @@
> > +#include "qemu/osdep.h"
> > +#include "hw/acpi/acpi_aml_interface.h"
> > +#include "vga_int.h"
> > +
> > +void build_vga_aml(AcpiDevAmlIf *adev, Aml *scope)
> > +{
> > +}
> > diff --git a/hw/display/acpi-vga.c b/hw/display/acpi-vga.c
> > new file mode 100644
> > index 0000000000..f0e9ef1fcf
> > --- /dev/null
> > +++ b/hw/display/acpi-vga.c
> > @@ -0,0 +1,26 @@
> > +#include "qemu/osdep.h"
> > +#include "hw/acpi/acpi_aml_interface.h"
> > +#include "hw/pci/pci.h"
> > +#include "vga_int.h"
> > +
> > +void build_vga_aml(AcpiDevAmlIf *adev, Aml *scope)
> > +{
> > +    int s3d = 0;
> > +    Aml *method;
> > +
> > +    if (object_dynamic_cast(OBJECT(adev), "qxl-vga")) {
> > +        s3d = 3;
> > +    }
> > +
> > +    method = aml_method("_S1D", 0, AML_NOTSERIALIZED);
> > +    aml_append(method, aml_return(aml_int(0)));
> > +    aml_append(scope, method);
> > +
> > +    method = aml_method("_S2D", 0, AML_NOTSERIALIZED);
> > +    aml_append(method, aml_return(aml_int(0)));
> > +    aml_append(scope, method);
> > +
> > +    method = aml_method("_S3D", 0, AML_NOTSERIALIZED);
> > +    aml_append(method, aml_return(aml_int(s3d)));
> > +    aml_append(scope, method);
> > +}
> > diff --git a/hw/display/vga-pci.c b/hw/display/vga-pci.c
> > index 3e5bc259f7..9a91de7ed1 100644
> > --- a/hw/display/vga-pci.c
> > +++ b/hw/display/vga-pci.c
> > @@ -35,6 +35,7 @@
> >   #include "hw/loader.h"
> >   #include "hw/display/edid.h"
> >   #include "qom/object.h"
> > +#include "hw/acpi/acpi_aml_interface.h"
> >   enum vga_pci_flags {
> >       PCI_VGA_FLAG_ENABLE_MMIO = 1,
> > @@ -354,11 +355,13 @@ static void vga_pci_class_init(ObjectClass *klass, void *data)
> >   {
> >       DeviceClass *dc = DEVICE_CLASS(klass);
> >       PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
> > +    AcpiDevAmlIfClass *adevc = ACPI_DEV_AML_IF_CLASS(klass);
> >       k->vendor_id = PCI_VENDOR_ID_QEMU;
> >       k->device_id = PCI_DEVICE_ID_QEMU_VGA;
> >       dc->vmsd = &vmstate_vga_pci;
> >       set_bit(DEVICE_CATEGORY_DISPLAY, dc->categories);
> > +    adevc->build_dev_aml = build_vga_aml;
> >   }
> >   static const TypeInfo vga_pci_type_info = {
> > @@ -369,6 +372,7 @@ static const TypeInfo vga_pci_type_info = {
> >       .class_init = vga_pci_class_init,
> >       .interfaces = (InterfaceInfo[]) {
> >           { INTERFACE_CONVENTIONAL_PCI_DEVICE },
> > +        { TYPE_ACPI_DEV_AML_IF },
> >           { },
> >       },
> >   };
> > diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> > index 4f54b61904..26932b4e2c 100644
> > --- a/hw/i386/acpi-build.c
> > +++ b/hw/i386/acpi-build.c
> > @@ -430,7 +430,6 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> >           bool hotpluggbale_slot = false;
> >           bool bridge_in_acpi = false;
> >           bool cold_plugged_bridge = false;
> > -        bool is_vga = false;
> >           if (pdev) {
> >               pc = PCI_DEVICE_GET_CLASS(pdev);
> > @@ -440,8 +439,6 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> >                   continue;
> >               }
> > -            is_vga = pc->class_id == PCI_CLASS_DISPLAY_VGA;
> > -
> >               /*
> >                * Cold plugged bridges aren't themselves hot-pluggable.
> >                * Hotplugged bridges *are* hot-pluggable.
> > @@ -489,28 +486,7 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> >               aml_append(dev, aml_pci_device_dsm());
> >           }
> > -        if (is_vga) {
> > -            /* add VGA specific AML methods */
> > -            int s3d;
> > -
> > -            if (object_dynamic_cast(OBJECT(pdev), "qxl-vga")) {
> > -                s3d = 3;
> > -            } else {
> > -                s3d = 0;
> > -            }
> > -
> > -            method = aml_method("_S1D", 0, AML_NOTSERIALIZED);
> > -            aml_append(method, aml_return(aml_int(0)));
> > -            aml_append(dev, method);
> > -
> > -            method = aml_method("_S2D", 0, AML_NOTSERIALIZED);
> > -            aml_append(method, aml_return(aml_int(0)));
> > -            aml_append(dev, method);
> > -
> > -            method = aml_method("_S3D", 0, AML_NOTSERIALIZED);
> > -            aml_append(method, aml_return(aml_int(s3d)));
> > -            aml_append(dev, method);
> > -        }
> > +        call_dev_aml_func(DEVICE(pdev), dev);
> >           bridge_in_acpi =  cold_plugged_bridge && pcihp_bridge_en;
> >           if (bridge_in_acpi) {
> > diff --git a/hw/display/meson.build b/hw/display/meson.build
> > index adc53dd8b6..7a725ed80e 100644
> > --- a/hw/display/meson.build
> > +++ b/hw/display/meson.build
> > @@ -38,10 +38,21 @@ softmmu_ss.add(when: 'CONFIG_NEXTCUBE', if_true: files('next-fb.c'))
> >   specific_ss.add(when: 'CONFIG_VGA', if_true: files('vga.c'))
> > +if (config_all_devices.has_key('CONFIG_VGA_CIRRUS') or
> > +    config_all_devices.has_key('CONFIG_VGA_PCI') or
> > +    config_all_devices.has_key('CONFIG_VMWARE_VGA') or
> > +    config_all_devices.has_key('CONFIG_ATI_VGA')
> > +   )
> > +  softmmu_ss.add(when: 'CONFIG_ACPI', if_true: files('acpi-vga.c'),
> > +                                      if_false: files('acpi-vga-stub.c'))
> > +endif

Igor what does  CONFIG_ACPI mean? It depends on the target but this
library is target independent. Is this just always built then?

> > +
> >   if config_all_devices.has_key('CONFIG_QXL')
> >     qxl_ss = ss.source_set()
> >     qxl_ss.add(when: 'CONFIG_QXL', if_true: [files('qxl.c', 'qxl-logger.c', 'qxl-render.c'),
> >                                              pixman, spice])
> > +  qxl_ss.add(when: 'CONFIG_ACPI', if_true: files('acpi-vga.c'),
> > +                                  if_false: files('acpi-vga-stub.c'))
> >     hw_display_modules += {'qxl': qxl_ss}
> >   endif
> > @@ -52,6 +63,7 @@ softmmu_ss.add(when: 'CONFIG_ARTIST', if_true: files('artist.c'))
> >   softmmu_ss.add(when: [pixman, 'CONFIG_ATI_VGA'], if_true: files('ati.c', 'ati_2d.c', 'ati_dbg.c'))
> > +
> >   if config_all_devices.has_key('CONFIG_VIRTIO_GPU')
> >     virtio_gpu_ss = ss.source_set()
> >     virtio_gpu_ss.add(when: 'CONFIG_VIRTIO_GPU',
> > @@ -87,14 +99,19 @@ if config_all_devices.has_key('CONFIG_VIRTIO_VGA')
> >                       if_true: [files('virtio-vga.c'), pixman])
> >     virtio_vga_ss.add(when: 'CONFIG_VHOST_USER_VGA',
> >                       if_true: files('vhost-user-vga.c'))
> > +  virtio_vga_ss.add(when: 'CONFIG_ACPI', if_true: files('acpi-vga.c'),
> > +                                         if_false: files('acpi-vga-stub.c'))
> >     hw_display_modules += {'virtio-vga': virtio_vga_ss}
> >     virtio_vga_gl_ss = ss.source_set()
> >     virtio_vga_gl_ss.add(when: ['CONFIG_VIRTIO_VGA', virgl, opengl],
> >                          if_true: [files('virtio-vga-gl.c'), pixman])
> > +  virtio_vga_gl_ss.add(when: 'CONFIG_ACPI', if_true: files('acpi-vga.c'),
> > +                                            if_false: files('acpi-vga-stub.c'))
> >     hw_display_modules += {'virtio-vga-gl': virtio_vga_gl_ss}
> >   endif
> >   specific_ss.add(when: 'CONFIG_OMAP', if_true: files('omap_lcdc.c'))
> > +softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('acpi-vga-stub.c'))
> >   modules += { 'hw-display': hw_display_modules }




Does the following help? It seems like a worthwile cleanup anyway.


diff --git a/hw/display/vga_int.h b/hw/display/vga_int.h
index 330406ad9c..7cf0d11201 100644
--- a/hw/display/vga_int.h
+++ b/hw/display/vga_int.h
@@ -27,7 +27,6 @@
 
 #include "exec/ioport.h"
 #include "exec/memory.h"
-#include "ui/console.h"
 
 #include "hw/display/bochs-vbe.h"
 #include "hw/acpi/acpi_aml_interface.h"
diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
index 6d4e6d9708..688408e048 100644
--- a/include/qemu/typedefs.h
+++ b/include/qemu/typedefs.h
@@ -132,6 +132,8 @@ typedef struct Visitor Visitor;
 typedef struct VMChangeStateEntry VMChangeStateEntry;
 typedef struct VMStateDescription VMStateDescription;
 typedef struct DumpState DumpState;
+typedef struct GraphicHwOps GraphicHwOps;
+typedef struct QEMUCursor QEMUCursor;
 
 /*
  * Pointer types
diff --git a/hw/display/ati_2d.c b/hw/display/ati_2d.c
index 692bec91de..7d786653e8 100644
--- a/hw/display/ati_2d.c
+++ b/hw/display/ati_2d.c
@@ -12,6 +12,7 @@
 #include "ati_regs.h"
 #include "qemu/log.h"
 #include "ui/pixel_ops.h"
+#include "ui/console.h"
 
 /*
  * NOTE:
diff --git a/hw/display/cirrus_vga.c b/hw/display/cirrus_vga.c
index 2577005d03..4cc3567c69 100644
--- a/hw/display/cirrus_vga.c
+++ b/hw/display/cirrus_vga.c
@@ -45,6 +45,7 @@
 #include "ui/pixel_ops.h"
 #include "cirrus_vga_internal.h"
 #include "qom/object.h"
+#include "ui/console.h"
 
 /*
  * TODO:
diff --git a/hw/display/cirrus_vga_isa.c b/hw/display/cirrus_vga_isa.c
index 96144bd690..84be51670e 100644
--- a/hw/display/cirrus_vga_isa.c
+++ b/hw/display/cirrus_vga_isa.c
@@ -31,6 +31,7 @@
 #include "hw/isa/isa.h"
 #include "cirrus_vga_internal.h"
 #include "qom/object.h"
+#include "ui/console.h"
 
 #define TYPE_ISA_CIRRUS_VGA "isa-cirrus-vga"
 OBJECT_DECLARE_SIMPLE_TYPE(ISACirrusVGAState, ISA_CIRRUS_VGA)
diff --git a/hw/display/vga-isa.c b/hw/display/vga-isa.c
index 46abbc5653..2a5437d803 100644
--- a/hw/display/vga-isa.c
+++ b/hw/display/vga-isa.c
@@ -32,6 +32,7 @@
 #include "qemu/timer.h"
 #include "hw/loader.h"
 #include "hw/qdev-properties.h"
+#include "ui/console.h"
 #include "qom/object.h"
 
 #define TYPE_ISA_VGA "isa-vga"
diff --git a/hw/display/vga-mmio.c b/hw/display/vga-mmio.c
index 75dfcedea5..cd2c46776d 100644
--- a/hw/display/vga-mmio.c
+++ b/hw/display/vga-mmio.c
@@ -27,6 +27,7 @@
 #include "hw/sysbus.h"
 #include "hw/display/vga.h"
 #include "hw/qdev-properties.h"
+#include "ui/console.h"
 #include "vga_int.h"
 
 /*
diff --git a/hw/display/vga-pci.c b/hw/display/vga-pci.c
index 9a91de7ed1..df23dbf3a0 100644
--- a/hw/display/vga-pci.c
+++ b/hw/display/vga-pci.c
@@ -30,6 +30,7 @@
 #include "migration/vmstate.h"
 #include "vga_int.h"
 #include "ui/pixel_ops.h"
+#include "ui/console.h"
 #include "qemu/module.h"
 #include "qemu/timer.h"
 #include "hw/loader.h"
diff --git a/hw/display/vga.c b/hw/display/vga.c
index 50ecb1ad02..0cb26a791b 100644
--- a/hw/display/vga.c
+++ b/hw/display/vga.c
@@ -31,6 +31,7 @@
 #include "vga_int.h"
 #include "vga_regs.h"
 #include "ui/pixel_ops.h"
+#include "ui/console.h"
 #include "qemu/timer.h"
 #include "hw/xen/xen.h"
 #include "migration/vmstate.h"
diff --git a/hw/display/vmware_vga.c b/hw/display/vmware_vga.c
index cedbbde522..53949d2539 100644
--- a/hw/display/vmware_vga.c
+++ b/hw/display/vmware_vga.c
@@ -33,6 +33,7 @@
 #include "hw/qdev-properties.h"
 #include "migration/vmstate.h"
 #include "qom/object.h"
+#include "ui/console.h"
 
 #undef VERBOSE
 #define HW_RECT_ACCEL



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* Re: [PULL v4 44/83] acpi: pc: vga: use AcpiDevAmlIf interface to build VGA device descriptors
  2022-11-09 17:39   ` Laurent Vivier
  2022-11-09 21:42     ` Michael S. Tsirkin
@ 2022-11-10  2:55     ` Ani Sinha
  1 sibling, 0 replies; 151+ messages in thread
From: Ani Sinha @ 2022-11-10  2:55 UTC (permalink / raw)
  To: Laurent Vivier
  Cc: Michael S. Tsirkin, qemu-devel, Peter Maydell, Igor Mammedov,
	Bernhard Beschow, Gerd Hoffmann, Marcel Apfelbaum, Paolo Bonzini,
	Richard Henderson, Eduardo Habkost

On Wed, Nov 9, 2022 at 11:09 PM Laurent Vivier <lvivier@redhat.com> wrote:
>
> This one breaks something for me:
>
> [3/65] Compiling C object libhw-display-virtio-vga-gl.a.p/hw_display_acpi-vga.c.o
> FAILED: libhw-display-virtio-vga-gl.a.p/hw_display_acpi-vga.c.o
> clang -m64 -mcx16 -Ilibhw-display-virtio-vga-gl.a.p -I. -I../../../Projects/qemu-upstream
> -Iqapi -Itrace -Iui -Iui/shader -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include
> -I/usr/include/sysprof-4 -fcolor-diagnostics -Wall -Winvalid-pch -Werror -std=gnu11 -O2 -g
> -isystem /home/lvivier/Projects/qemu-upstream/linux-headers -isystem linux-headers -iquote
> . -iquote /home/lvivier/Projects/qemu-upstream -iquote
> /home/lvivier/Projects/qemu-upstream/include -iquote
> /home/lvivier/Projects/qemu-upstream/tcg/i386 -pthread -D_GNU_SOURCE
> -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wundef
> -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv
> -Wold-style-definition -Wtype-limits -Wformat-security -Wformat-y2k -Winit-self
> -Wignored-qualifiers -Wempty-body -Wnested-externs -Wendif-labels -Wexpansion-to-defined
> -Wno-initializer-overrides -Wno-missing-include-dirs -Wno-shift-negative-value
> -Wno-string-plus-int -Wno-typedef-redefinition -Wno-tautological-type-limit-compare
> -Wno-psabi -Wno-gnu-variable-sized-type-not-at-end -fstack-protector-strong
> -fsanitize=address -fPIC -DBUILD_DSO -MD -MQ
> libhw-display-virtio-vga-gl.a.p/hw_display_acpi-vga.c.o -MF
> libhw-display-virtio-vga-gl.a.p/hw_display_acpi-vga.c.o.d -o
> libhw-display-virtio-vga-gl.a.p/hw_display_acpi-vga.c.o -c
> ../../../Projects/qemu-upstream/hw/display/acpi-vga.c
> In file included from ../../../Projects/qemu-upstream/hw/display/acpi-vga.c:4:
> In file included from ../../../Projects/qemu-upstream/hw/display/vga_int.h:30:
> In file included from /home/lvivier/Projects/qemu-upstream/include/ui/console.h:4:
> /home/lvivier/Projects/qemu-upstream/include/ui/qemu-pixman.h:12:10: fatal error:
> 'pixman.h' file not found
> #include <pixman.h>
>           ^~~~~~~~~~
> 1 error generated.
> ninja: build stopped: subcommand failed.
> make: *** [Makefile:165: run-ninja] Error 1
> make: Leaving directory '/home/lvivier/Objects/qemu-upstream/x86_64'
>
> Any idea?

For the records, we are also seeing this in the custom-runners in the
CI pipeline.


^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 44/83] acpi: pc: vga: use AcpiDevAmlIf interface to build VGA device descriptors
  2022-11-09 21:42     ` Michael S. Tsirkin
@ 2022-11-10  9:28       ` Peter Maydell
  2022-11-11  9:43         ` Igor Mammedov
  0 siblings, 1 reply; 151+ messages in thread
From: Peter Maydell @ 2022-11-10  9:28 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Laurent Vivier, qemu-devel, Igor Mammedov, Ani Sinha,
	Bernhard Beschow, Gerd Hoffmann, Marcel Apfelbaum, Paolo Bonzini,
	Richard Henderson, Eduardo Habkost

On Wed, 9 Nov 2022 at 21:42, Michael S. Tsirkin <mst@redhat.com> wrote:

> > > diff --git a/hw/display/meson.build b/hw/display/meson.build
> > > index adc53dd8b6..7a725ed80e 100644
> > > --- a/hw/display/meson.build
> > > +++ b/hw/display/meson.build
> > > @@ -38,10 +38,21 @@ softmmu_ss.add(when: 'CONFIG_NEXTCUBE', if_true: files('next-fb.c'))
> > >   specific_ss.add(when: 'CONFIG_VGA', if_true: files('vga.c'))
> > > +if (config_all_devices.has_key('CONFIG_VGA_CIRRUS') or
> > > +    config_all_devices.has_key('CONFIG_VGA_PCI') or
> > > +    config_all_devices.has_key('CONFIG_VMWARE_VGA') or
> > > +    config_all_devices.has_key('CONFIG_ATI_VGA')
> > > +   )
> > > +  softmmu_ss.add(when: 'CONFIG_ACPI', if_true: files('acpi-vga.c'),
> > > +                                      if_false: files('acpi-vga-stub.c'))
> > > +endif
>
> Igor what does  CONFIG_ACPI mean? It depends on the target but this
> library is target independent. Is this just always built then?

That kind of config symbol means "some machine we want to build needs
ACPI, so build it". So if you build at least one machine that needs
ACPI, CONFIG_ACPI gets defined, and the acpi-specific files are built.
If your QEMU configure line and possibly any custom config file
mean you're not building any ACPI machines, then CONFIG_ACPI is
not defined, and we don't need to build in the acpi-specifics,
which makes the binary smaller. For instance if you set
--target-list=or1k-softmmu then no machine wants ACPI and
CONFIG_ACPI won't get set. If you set --target-list=or1k-softmmu,x86_64-softmmu
then the PC machine types want ACPI and CONFIG_ACPI is set.

(Essentially we're opting to make the build faster by building the
object file once rather than per-target, at the cost of the
executables for the target architectures which don't use the
feature being a bit bigger with code they aren't going to use.)

Note that this means that for a machine type which does not use ACPI, it
may either:
 (1) be being built in the same build as a machine that does use
 ACPI, so be linked against the "real" ACPI source files
 (2) be being built in a build with no ACPI machines, so be
 linked against the stub files

and it also means that the code in the "real" ACPI source files
cannot assume that it's being used in a machine which is actually
using ACPI.

thanks
-- PMM


^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 44/83] acpi: pc: vga: use AcpiDevAmlIf interface to build VGA device descriptors
  2022-11-10  9:28       ` Peter Maydell
@ 2022-11-11  9:43         ` Igor Mammedov
  2022-11-11 10:25           ` Igor Mammedov
  0 siblings, 1 reply; 151+ messages in thread
From: Igor Mammedov @ 2022-11-11  9:43 UTC (permalink / raw)
  To: Peter Maydell
  Cc: Michael S. Tsirkin, Laurent Vivier, qemu-devel, Ani Sinha,
	Bernhard Beschow, Gerd Hoffmann, Marcel Apfelbaum, Paolo Bonzini,
	Richard Henderson, Eduardo Habkost

On Thu, 10 Nov 2022 09:28:44 +0000
Peter Maydell <peter.maydell@linaro.org> wrote:

> On Wed, 9 Nov 2022 at 21:42, Michael S. Tsirkin <mst@redhat.com> wrote:
> 
> > > > diff --git a/hw/display/meson.build b/hw/display/meson.build
> > > > index adc53dd8b6..7a725ed80e 100644
> > > > --- a/hw/display/meson.build
> > > > +++ b/hw/display/meson.build
> > > > @@ -38,10 +38,21 @@ softmmu_ss.add(when: 'CONFIG_NEXTCUBE', if_true: files('next-fb.c'))
> > > >   specific_ss.add(when: 'CONFIG_VGA', if_true: files('vga.c'))
> > > > +if (config_all_devices.has_key('CONFIG_VGA_CIRRUS') or
> > > > +    config_all_devices.has_key('CONFIG_VGA_PCI') or
> > > > +    config_all_devices.has_key('CONFIG_VMWARE_VGA') or
> > > > +    config_all_devices.has_key('CONFIG_ATI_VGA')
> > > > +   )
> > > > +  softmmu_ss.add(when: 'CONFIG_ACPI', if_true: files('acpi-vga.c'),
> > > > +                                      if_false: files('acpi-vga-stub.c'))
> > > > +endif  
> >
> > Igor what does  CONFIG_ACPI mean? It depends on the target but this
> > library is target independent. Is this just always built then?  
> 
> That kind of config symbol means "some machine we want to build needs
> ACPI, so build it". So if you build at least one machine that needs
> ACPI, CONFIG_ACPI gets defined, and the acpi-specific files are built.
> If your QEMU configure line and possibly any custom config file
> mean you're not building any ACPI machines, then CONFIG_ACPI is
> not defined, and we don't need to build in the acpi-specifics,
> which makes the binary smaller. For instance if you set
> --target-list=or1k-softmmu then no machine wants ACPI and
> CONFIG_ACPI won't get set. If you set --target-list=or1k-softmmu,x86_64-softmmu
> then the PC machine types want ACPI and CONFIG_ACPI is set.
> 
> (Essentially we're opting to make the build faster by building the
> object file once rather than per-target, at the cost of the
> executables for the target architectures which don't use the
> feature being a bit bigger with code they aren't going to use.)
> 
> Note that this means that for a machine type which does not use ACPI, it
> may either:
>  (1) be being built in the same build as a machine that does use
>  ACPI, so be linked against the "real" ACPI source files
>  (2) be being built in a build with no ACPI machines, so be
>  linked against the stub files
> 
> and it also means that the code in the "real" ACPI source files
> cannot assume that it's being used in a machine which is actually
> using ACPI.

Stubs only work because we create per target binaries,
so I'd expect acpi-vga.o and acpi-vga-stub.o being built and
linked accordingly.

Even if it were not the case linking 'real' code in this case
shouldn't have caused issues on machine that do not actually
invoke it.

Michael has already fixed issue removing pixman dependency in stub,
which I haven't noticed (my machines have the library installed
and CI at time of posting was green as well).

> thanks
> -- PMM
> 



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 44/83] acpi: pc: vga: use AcpiDevAmlIf interface to build VGA device descriptors
  2022-11-11  9:43         ` Igor Mammedov
@ 2022-11-11 10:25           ` Igor Mammedov
  0 siblings, 0 replies; 151+ messages in thread
From: Igor Mammedov @ 2022-11-11 10:25 UTC (permalink / raw)
  To: Peter Maydell
  Cc: Michael S. Tsirkin, Laurent Vivier, qemu-devel, Ani Sinha,
	Bernhard Beschow, Gerd Hoffmann, Marcel Apfelbaum, Paolo Bonzini,
	Richard Henderson, Eduardo Habkost

On Fri, 11 Nov 2022 10:43:30 +0100
Igor Mammedov <imammedo@redhat.com> wrote:

> On Thu, 10 Nov 2022 09:28:44 +0000
> Peter Maydell <peter.maydell@linaro.org> wrote:
> 
> > On Wed, 9 Nov 2022 at 21:42, Michael S. Tsirkin <mst@redhat.com> wrote:
> >   
> > > > > diff --git a/hw/display/meson.build b/hw/display/meson.build
> > > > > index adc53dd8b6..7a725ed80e 100644
> > > > > --- a/hw/display/meson.build
> > > > > +++ b/hw/display/meson.build
> > > > > @@ -38,10 +38,21 @@ softmmu_ss.add(when: 'CONFIG_NEXTCUBE', if_true: files('next-fb.c'))
> > > > >   specific_ss.add(when: 'CONFIG_VGA', if_true: files('vga.c'))
> > > > > +if (config_all_devices.has_key('CONFIG_VGA_CIRRUS') or
> > > > > +    config_all_devices.has_key('CONFIG_VGA_PCI') or
> > > > > +    config_all_devices.has_key('CONFIG_VMWARE_VGA') or
> > > > > +    config_all_devices.has_key('CONFIG_ATI_VGA')
> > > > > +   )
> > > > > +  softmmu_ss.add(when: 'CONFIG_ACPI', if_true: files('acpi-vga.c'),
> > > > > +                                      if_false: files('acpi-vga-stub.c'))
> > > > > +endif    
> > >
> > > Igor what does  CONFIG_ACPI mean? It depends on the target but this
> > > library is target independent. Is this just always built then?    
> > 
> > That kind of config symbol means "some machine we want to build needs
> > ACPI, so build it". So if you build at least one machine that needs
> > ACPI, CONFIG_ACPI gets defined, and the acpi-specific files are built.
> > If your QEMU configure line and possibly any custom config file
> > mean you're not building any ACPI machines, then CONFIG_ACPI is
> > not defined, and we don't need to build in the acpi-specifics,
> > which makes the binary smaller. For instance if you set
> > --target-list=or1k-softmmu then no machine wants ACPI and
> > CONFIG_ACPI won't get set. If you set --target-list=or1k-softmmu,x86_64-softmmu
> > then the PC machine types want ACPI and CONFIG_ACPI is set.
> > 
> > (Essentially we're opting to make the build faster by building the
> > object file once rather than per-target, at the cost of the
> > executables for the target architectures which don't use the
> > feature being a bit bigger with code they aren't going to use.)
> > 
> > Note that this means that for a machine type which does not use ACPI, it
> > may either:
> >  (1) be being built in the same build as a machine that does use
> >  ACPI, so be linked against the "real" ACPI source files
> >  (2) be being built in a build with no ACPI machines, so be
> >  linked against the stub files
> > 
> > and it also means that the code in the "real" ACPI source files
> > cannot assume that it's being used in a machine which is actually
> > using ACPI.  
> 
[...]

> Even if it were not the case linking 'real' code in this case
> shouldn't have caused issues on machine that do not actually
> invoke it.
Well, aarch64 target is this case wher it has a mix of ACPI and
non-ACPI machines. As mentinod aove this shouldn't be the issue
in this case.
But are there ideas how to deal with such cases cleanly
(modulo tweaking 'real' code to deal with both cases)?


[...]

> 
> > thanks
> > -- PMM
> >   
> 



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups
  2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
                   ` (83 preceding siblings ...)
  2022-11-08  6:23 ` [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
@ 2022-11-15 14:01 ` Philippe Mathieu-Daudé
  2022-11-16 10:42   ` Igor Mammedov
  84 siblings, 1 reply; 151+ messages in thread
From: Philippe Mathieu-Daudé @ 2022-11-15 14:01 UTC (permalink / raw)
  To: Michael S. Tsirkin, qemu-devel; +Cc: Peter Maydell, Ani Sinha, Igor Mammedov

Hi,

On 7/11/22 23:47, Michael S. Tsirkin wrote:

> ----------------------------------------------------------------
> pci,pc,virtio: features, tests, fixes, cleanups
> 
> lots of acpi rework
> first version of biosbits infrastructure
> ASID support in vhost-vdpa
> core_count2 support in smbios
> PCIe DOE emulation
> virtio vq reset
> HMAT support
> part of infrastructure for viommu support in vhost-vdpa
> VTD PASID support
> fixes, tests all over the place
Apparently unrelated to these fixes, but going from 6295a58ad1 to
v7.2.0-rc0 triggered rebuilding ACPI files and I now get:

  45/510 qemu:qtest+qtest-i386 / qtest-i386/bios-tables-test 
           ERROR          14.73s   killed by signal 6 SIGABRT
  74/510 qemu:qtest+qtest-x86_64 / qtest-x86_64/bios-tables-test 
           ERROR          12.56s   killed by signal 6 SIGABRT

Running manually:

$ QTEST_QEMU_BINARY=./qemu-system-x86_64 tests/qtest/bios-tables-test
...
# starting QEMU: exec ./qemu-system-x86_64 -qtest 
unix:/tmp/qtest-239233.sock -qtest-log /dev/null -chardev 
socket,path=/tmp/qtest-239233.qmp,id=char0 -mon 
chardev=char0,mode=control -display none -machine pc -accel kvm -accel 
tcg -net none -machine smm=off -drive 
id=hd0,if=none,file=tests/acpi-test-disk-QmvOOR,format=raw -device 
ide-hd,drive=hd0  -accel qtest
Could not access KVM kernel module: Permission denied
qemu-system-x86_64: -accel kvm: failed to initialize kvm: Permission denied
acpi-test: Warning! DSDT binary file mismatch. Actual 
[aml:/tmp/aml-Y06RV1], Expected [aml:tests/data/acpi/pc/DSDT.nosmm].
See source file tests/qtest/bios-tables-test.c for instructions on how 
to update expected files.
to see ASL diff between mismatched files install IASL, rebuild QEMU from 
scratch and re-run tests with V=1 environment variable set**
ERROR:../../tests/qtest/bios-tables-test.c:533:test_acpi_asl: assertion 
failed: (all_tables_match)
Bail out! ERROR:../../tests/qtest/bios-tables-test.c:533:test_acpi_asl: 
assertion failed: (all_tables_match)
Aborted

I blew/recreated my build directory and can reproduce.

$ uname -sm
Linux x86_64

Any clue?


^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups
  2022-11-15 14:01 ` Philippe Mathieu-Daudé
@ 2022-11-16 10:42   ` Igor Mammedov
  0 siblings, 0 replies; 151+ messages in thread
From: Igor Mammedov @ 2022-11-16 10:42 UTC (permalink / raw)
  To: Philippe Mathieu-Daudé
  Cc: Michael S. Tsirkin, qemu-devel, Peter Maydell, Ani Sinha

On Tue, 15 Nov 2022 15:01:06 +0100
Philippe Mathieu-Daudé <philmd@linaro.org> wrote:

> Hi,
> 
> On 7/11/22 23:47, Michael S. Tsirkin wrote:
> 
> > ----------------------------------------------------------------
> > pci,pc,virtio: features, tests, fixes, cleanups
> > 
> > lots of acpi rework
> > first version of biosbits infrastructure
> > ASID support in vhost-vdpa
> > core_count2 support in smbios
> > PCIe DOE emulation
> > virtio vq reset
> > HMAT support
> > part of infrastructure for viommu support in vhost-vdpa
> > VTD PASID support
> > fixes, tests all over the place  
> Apparently unrelated to these fixes, but going from 6295a58ad1 to
> v7.2.0-rc0 triggered rebuilding ACPI files and I now get:
> 
>   45/510 qemu:qtest+qtest-i386 / qtest-i386/bios-tables-test 
>            ERROR          14.73s   killed by signal 6 SIGABRT
>   74/510 qemu:qtest+qtest-x86_64 / qtest-x86_64/bios-tables-test 
>            ERROR          12.56s   killed by signal 6 SIGABRT
> 
> Running manually:
> 
> $ QTEST_QEMU_BINARY=./qemu-system-x86_64 tests/qtest/bios-tables-test
> ...
> # starting QEMU: exec ./qemu-system-x86_64 -qtest 
> unix:/tmp/qtest-239233.sock -qtest-log /dev/null -chardev 
> socket,path=/tmp/qtest-239233.qmp,id=char0 -mon 
> chardev=char0,mode=control -display none -machine pc -accel kvm -accel 
> tcg -net none -machine smm=off -drive 
> id=hd0,if=none,file=tests/acpi-test-disk-QmvOOR,format=raw -device 
> ide-hd,drive=hd0  -accel qtest
> Could not access KVM kernel module: Permission denied
> qemu-system-x86_64: -accel kvm: failed to initialize kvm: Permission denied
> acpi-test: Warning! DSDT binary file mismatch. Actual 
> [aml:/tmp/aml-Y06RV1], Expected [aml:tests/data/acpi/pc/DSDT.nosmm].
> See source file tests/qtest/bios-tables-test.c for instructions on how 
> to update expected files.
> to see ASL diff between mismatched files install IASL, rebuild QEMU from 
> scratch and re-run tests with V=1 environment variable set**
> ERROR:../../tests/qtest/bios-tables-test.c:533:test_acpi_asl: assertion 
> failed: (all_tables_match)
> Bail out! ERROR:../../tests/qtest/bios-tables-test.c:533:test_acpi_asl: 
> assertion failed: (all_tables_match)
> Aborted
> 
> I blew/recreated my build directory and can reproduce.

it works for me with KVM and without it on RHEL 8.7 x86 host.
It looks like stale file (though you are saying that you've recreated directory) 

> 
> $ uname -sm
> Linux x86_64
> 
> Any clue?

just to be sure can you make fresh tree clone and build that?

if that won't work, make sure you have IASL installed and run test
manually with V=1 env var, that should print diff on failure 



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 46/83] acpi: pc/q35: drop ad-hoc PCI-ISA bridge AML routines and let bus ennumeration generate AML
  2022-11-07 22:51 ` [PULL v4 46/83] acpi: pc/q35: drop ad-hoc PCI-ISA bridge AML routines and let bus ennumeration generate AML Michael S. Tsirkin
@ 2022-11-17 21:51   ` Volker Rümelin
  2022-11-18 13:08     ` Igor Mammedov
  0 siblings, 1 reply; 151+ messages in thread
From: Volker Rümelin @ 2022-11-17 21:51 UTC (permalink / raw)
  To: Igor Mammedov
  Cc: Michael S. Tsirkin, Peter Maydell, Ani Sinha, Marcel Apfelbaum,
	Paolo Bonzini, Richard Henderson, Eduardo Habkost, qemu-devel

> From: Igor Mammedov<imammedo@redhat.com>
>
> PCI-ISA bridges that are built in PIIX/Q35 are building its own AML
> using AcpiDevAmlIf interface. Now build_append_pci_bus_devices()
> gained AcpiDevAmlIf interface support to get AML of devices atached
> to PCI slots.
> So drop ad-hoc build_q35_isa_bridge()/build_piix4_isa_bridge()
> and let PCI bus enumeration to include PCI-ISA bridge AML
> when it's enumerated by build_append_pci_bus_devices().
>
> AML change is mostly contextual, which moves whole ISA hierarchy
> directly under PCI host bridge instead of it being described
> as separate \SB.PCI0.ISA block.
>
> Note:
> If bus/slot that hosts ISA bridge has BSEL set, it will gain new
> ASUN and _DMS entries (i.e. acpi-index support, but it should not
> cause any functional change and that is fine from PCI Firmware
> spec point of view), potentially it's possible to suppress that
> by adding a flag to PCIDevice but I don't see a reason to do that
> yet, I'd rather treat bridge just as any other PCI device if it's
> possible.
>
> Signed-off-by: Igor Mammedov<imammedo@redhat.com>
> Message-Id:<20221017102146.2254096-4-imammedo@redhat.com>
> Reviewed-by: Michael S. Tsirkin<mst@redhat.com>
> Signed-off-by: Michael S. Tsirkin<mst@redhat.com>
> ---
>   hw/i386/acpi-build.c | 75 --------------------------------------------
>   hw/isa/lpc_ich9.c    | 23 ++++++++++++++
>   hw/isa/piix3.c       | 17 +++++++++-
>   3 files changed, 39 insertions(+), 76 deletions(-)

Hi Igor,

since this patch SeaBIOS no longer detects the PS/2 keyboard. This means 
there's no keyboard in SeaBIOS, GRUB or FreeDOS. OVMF and Linux detect 
the PS/2 keyboard without issues.

Here are a few lines from the SeaBIOS debug log.

table(50434146)=0x007e1971 (via rsdt)
ACPI: parse DSDT at 0x007e0040 (len 6449)
parse_termlist: parse error, skip from 92/465
Scan for VGA option rom
Running option rom at c000:0003
Start SeaVGABIOS (version rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org)

and later

SeaBIOS (version rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org)
ACPI: no PS/2 keyboard present

It doesn't matter if the machine type is pc or q35.

If I revert this patch, the PS/2 keyboard works again.

With best regards,
Volker






^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 46/83] acpi: pc/q35: drop ad-hoc PCI-ISA bridge AML routines and let bus ennumeration generate AML
  2022-11-17 21:51   ` Volker Rümelin
@ 2022-11-18 13:08     ` Igor Mammedov
  2022-11-18 14:55       ` Igor Mammedov
  0 siblings, 1 reply; 151+ messages in thread
From: Igor Mammedov @ 2022-11-18 13:08 UTC (permalink / raw)
  To: Volker Rümelin
  Cc: Michael S. Tsirkin, Peter Maydell, Ani Sinha, Marcel Apfelbaum,
	Paolo Bonzini, Richard Henderson, Eduardo Habkost, qemu-devel

On Thu, 17 Nov 2022 22:51:46 +0100
Volker Rümelin <vr_qemu@t-online.de> wrote:

> > From: Igor Mammedov<imammedo@redhat.com>
> >
> > PCI-ISA bridges that are built in PIIX/Q35 are building its own AML
> > using AcpiDevAmlIf interface. Now build_append_pci_bus_devices()
> > gained AcpiDevAmlIf interface support to get AML of devices atached
> > to PCI slots.
> > So drop ad-hoc build_q35_isa_bridge()/build_piix4_isa_bridge()
> > and let PCI bus enumeration to include PCI-ISA bridge AML
> > when it's enumerated by build_append_pci_bus_devices().
> >
> > AML change is mostly contextual, which moves whole ISA hierarchy
> > directly under PCI host bridge instead of it being described
> > as separate \SB.PCI0.ISA block.
> >
> > Note:
> > If bus/slot that hosts ISA bridge has BSEL set, it will gain new
> > ASUN and _DMS entries (i.e. acpi-index support, but it should not
> > cause any functional change and that is fine from PCI Firmware
> > spec point of view), potentially it's possible to suppress that
> > by adding a flag to PCIDevice but I don't see a reason to do that
> > yet, I'd rather treat bridge just as any other PCI device if it's
> > possible.
> >
> > Signed-off-by: Igor Mammedov<imammedo@redhat.com>
> > Message-Id:<20221017102146.2254096-4-imammedo@redhat.com>
> > Reviewed-by: Michael S. Tsirkin<mst@redhat.com>
> > Signed-off-by: Michael S. Tsirkin<mst@redhat.com>
> > ---
> >   hw/i386/acpi-build.c | 75 --------------------------------------------
> >   hw/isa/lpc_ich9.c    | 23 ++++++++++++++
> >   hw/isa/piix3.c       | 17 +++++++++-
> >   3 files changed, 39 insertions(+), 76 deletions(-)  
> 
> Hi Igor,
> 
> since this patch SeaBIOS no longer detects the PS/2 keyboard. This means 
> there's no keyboard in SeaBIOS, GRUB or FreeDOS. OVMF and Linux detect 
> the PS/2 keyboard without issues.
> 
> Here are a few lines from the SeaBIOS debug log.
> 
> table(50434146)=0x007e1971 (via rsdt)
> ACPI: parse DSDT at 0x007e0040 (len 6449)
> parse_termlist: parse error, skip from 92/465
> Scan for VGA option rom
> Running option rom at c000:0003
> Start SeaVGABIOS (version rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org)
> 
> and later
> 
> SeaBIOS (version rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org)
> ACPI: no PS/2 keyboard present

it looks like SeaBIOS looks for EISA id,
but what is strange is that this patch doesn't have anything
that touches eisaid.

I'll have to debug seabios to figure out why this patch
affects it.

> 
> It doesn't matter if the machine type is pc or q35.
> 
> If I revert this patch, the PS/2 keyboard works again.
> 
> With best regards,
> Volker
> 
> 
> 
> 
> 



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 30/83] virtio: core: vq reset feature negotation support
  2022-11-07 22:50 ` [PULL v4 30/83] virtio: core: vq reset feature negotation support Michael S. Tsirkin
@ 2022-11-18 14:32   ` Stefano Garzarella
  2022-11-18 14:39     ` Stefano Garzarella
  2022-11-19 17:19     ` Michael S. Tsirkin
  0 siblings, 2 replies; 151+ messages in thread
From: Stefano Garzarella @ 2022-11-18 14:32 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: qemu-devel, Peter Maydell, Kangjie Xu, Xuan Zhuo, Jason Wang,
	Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
	Yanan Wang, Alex Bennée

Hi,
starting from this commit 69e1c14aa2 ("virtio: core: vq reset feature 
negotation support"), vhost-user-vsock and vhost-vsock fails while 
setting the device features, because VIRTIO_F_RING_RESET is not masked.

I'm not sure vsock is the only one affected.

We could fix in two ways:

1) Masking VIRTIO_F_RING_RESET when we call vhost_get_features:

diff --git a/hw/virtio/vhost-vsock-common.c b/hw/virtio/vhost-vsock-common.c
index 29b9ab4f72..e671cc695f 100644
--- a/hw/virtio/vhost-vsock-common.c
+++ b/hw/virtio/vhost-vsock-common.c
@@ -21,6 +21,7 @@
 
 const int feature_bits[] = {
     VIRTIO_VSOCK_F_SEQPACKET,
+    VIRTIO_F_RING_RESET,
     VHOST_INVALID_FEATURE_BIT
 };


2) Or using directly the features of the device. That way we also handle 
other features that we may have already had to mask but never did.

diff --git a/hw/virtio/vhost-vsock-common.c b/hw/virtio/vhost-vsock-common.c
index 29b9ab4f72..41a665082c 100644
--- a/hw/virtio/vhost-vsock-common.c
+++ b/hw/virtio/vhost-vsock-common.c
@@ -33,7 +33,7 @@ uint64_t vhost_vsock_common_get_features(VirtIODevice *vdev, uint64_t features,
         virtio_add_feature(&features, VIRTIO_VSOCK_F_SEQPACKET);
     }

-    features = vhost_get_features(&vvc->vhost_dev, feature_bits, features);
+    features &= vvc->vhost_dev.features;

     if (vvc->seqpacket == ON_OFF_AUTO_ON &&
         !virtio_has_feature(features, VIRTIO_VSOCK_F_SEQPACKET)) {


I may be missing the real reason for calling vhost_get_features(), 
though.

@Michael what do you recommend we do?

Thanks,
Stefano

On Tue, Nov 8, 2022 at 12:06 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> From: Kangjie Xu <kangjie.xu@linux.alibaba.com>
>
> A a new command line parameter "queue_reset" is added.
>
> Meanwhile, the vq reset feature is disabled for pre-7.2 machines.
>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> Acked-by: Jason Wang <jasowang@redhat.com>
> Message-Id: <20221017092558.111082-5-xuanzhuo@linux.alibaba.com>
> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> ---
>  include/hw/virtio/virtio.h | 4 +++-
>  hw/core/machine.c          | 4 +++-
>  2 files changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> index b00b3fcf31..1423dba379 100644
> --- a/include/hw/virtio/virtio.h
> +++ b/include/hw/virtio/virtio.h
> @@ -313,7 +313,9 @@ typedef struct VirtIORNGConf VirtIORNGConf;
>      DEFINE_PROP_BIT64("iommu_platform", _state, _field, \
>                        VIRTIO_F_IOMMU_PLATFORM, false), \
>      DEFINE_PROP_BIT64("packed", _state, _field, \
> -                      VIRTIO_F_RING_PACKED, false)
> +                      VIRTIO_F_RING_PACKED, false), \
> +    DEFINE_PROP_BIT64("queue_reset", _state, _field, \
> +                      VIRTIO_F_RING_RESET, true)
>
>  hwaddr virtio_queue_get_desc_addr(VirtIODevice *vdev, int n);
>  bool virtio_queue_enabled_legacy(VirtIODevice *vdev, int n);
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index aa520e74a8..907fa78ff0 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -40,7 +40,9 @@
>  #include "hw/virtio/virtio-pci.h"
>  #include "qom/object_interfaces.h"
>
> -GlobalProperty hw_compat_7_1[] = {};
> +GlobalProperty hw_compat_7_1[] = {
> +    { "virtio-device", "queue_reset", "false" },
> +};
>  const size_t hw_compat_7_1_len = G_N_ELEMENTS(hw_compat_7_1);
>
>  GlobalProperty hw_compat_7_0[] = {
> --
> MST
>
>



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* Re: [PULL v4 30/83] virtio: core: vq reset feature negotation support
  2022-11-18 14:32   ` Stefano Garzarella
@ 2022-11-18 14:39     ` Stefano Garzarella
  2022-11-19 17:19     ` Michael S. Tsirkin
  1 sibling, 0 replies; 151+ messages in thread
From: Stefano Garzarella @ 2022-11-18 14:39 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: qemu-devel, Peter Maydell, Kangjie Xu, Xuan Zhuo, Jason Wang,
	Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
	Yanan Wang, Alex Bennée

On Fri, Nov 18, 2022 at 3:32 PM Stefano Garzarella <sgarzare@redhat.com> wrote:
>
> Hi,
> starting from this commit 69e1c14aa2 ("virtio: core: vq reset feature
> negotation support"), vhost-user-vsock and vhost-vsock fails while
> setting the device features, because VIRTIO_F_RING_RESET is not masked.

vhost-vsock issue also reported here:
https://gitlab.com/qemu-project/qemu/-/issues/1318

>
> I'm not sure vsock is the only one affected.
>
> We could fix in two ways:
>
> 1) Masking VIRTIO_F_RING_RESET when we call vhost_get_features:
>
> diff --git a/hw/virtio/vhost-vsock-common.c b/hw/virtio/vhost-vsock-common.c
> index 29b9ab4f72..e671cc695f 100644
> --- a/hw/virtio/vhost-vsock-common.c
> +++ b/hw/virtio/vhost-vsock-common.c
> @@ -21,6 +21,7 @@
>
>  const int feature_bits[] = {
>      VIRTIO_VSOCK_F_SEQPACKET,
> +    VIRTIO_F_RING_RESET,
>      VHOST_INVALID_FEATURE_BIT
>  };
>
>
> 2) Or using directly the features of the device. That way we also handle
> other features that we may have already had to mask but never did.
>
> diff --git a/hw/virtio/vhost-vsock-common.c b/hw/virtio/vhost-vsock-common.c
> index 29b9ab4f72..41a665082c 100644
> --- a/hw/virtio/vhost-vsock-common.c
> +++ b/hw/virtio/vhost-vsock-common.c
> @@ -33,7 +33,7 @@ uint64_t vhost_vsock_common_get_features(VirtIODevice *vdev, uint64_t features,
>          virtio_add_feature(&features, VIRTIO_VSOCK_F_SEQPACKET);
>      }
>
> -    features = vhost_get_features(&vvc->vhost_dev, feature_bits, features);
> +    features &= vvc->vhost_dev.features;
>
>      if (vvc->seqpacket == ON_OFF_AUTO_ON &&
>          !virtio_has_feature(features, VIRTIO_VSOCK_F_SEQPACKET)) {
>
>
> I may be missing the real reason for calling vhost_get_features(),
> though.
>
> @Michael what do you recommend we do?
>
> Thanks,
> Stefano
>
> On Tue, Nov 8, 2022 at 12:06 AM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > From: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> >
> > A a new command line parameter "queue_reset" is added.
> >
> > Meanwhile, the vq reset feature is disabled for pre-7.2 machines.
> >
> > Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > Acked-by: Jason Wang <jasowang@redhat.com>
> > Message-Id: <20221017092558.111082-5-xuanzhuo@linux.alibaba.com>
> > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > ---
> >  include/hw/virtio/virtio.h | 4 +++-
> >  hw/core/machine.c          | 4 +++-
> >  2 files changed, 6 insertions(+), 2 deletions(-)
> >
> > diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> > index b00b3fcf31..1423dba379 100644
> > --- a/include/hw/virtio/virtio.h
> > +++ b/include/hw/virtio/virtio.h
> > @@ -313,7 +313,9 @@ typedef struct VirtIORNGConf VirtIORNGConf;
> >      DEFINE_PROP_BIT64("iommu_platform", _state, _field, \
> >                        VIRTIO_F_IOMMU_PLATFORM, false), \
> >      DEFINE_PROP_BIT64("packed", _state, _field, \
> > -                      VIRTIO_F_RING_PACKED, false)
> > +                      VIRTIO_F_RING_PACKED, false), \
> > +    DEFINE_PROP_BIT64("queue_reset", _state, _field, \
> > +                      VIRTIO_F_RING_RESET, true)
> >
> >  hwaddr virtio_queue_get_desc_addr(VirtIODevice *vdev, int n);
> >  bool virtio_queue_enabled_legacy(VirtIODevice *vdev, int n);
> > diff --git a/hw/core/machine.c b/hw/core/machine.c
> > index aa520e74a8..907fa78ff0 100644
> > --- a/hw/core/machine.c
> > +++ b/hw/core/machine.c
> > @@ -40,7 +40,9 @@
> >  #include "hw/virtio/virtio-pci.h"
> >  #include "qom/object_interfaces.h"
> >
> > -GlobalProperty hw_compat_7_1[] = {};
> > +GlobalProperty hw_compat_7_1[] = {
> > +    { "virtio-device", "queue_reset", "false" },
> > +};
> >  const size_t hw_compat_7_1_len = G_N_ELEMENTS(hw_compat_7_1);
> >
> >  GlobalProperty hw_compat_7_0[] = {
> > --
> > MST
> >
> >



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 46/83] acpi: pc/q35: drop ad-hoc PCI-ISA bridge AML routines and let bus ennumeration generate AML
  2022-11-18 13:08     ` Igor Mammedov
@ 2022-11-18 14:55       ` Igor Mammedov
  2022-11-19  8:49         ` Volker Rümelin
  2022-11-19 17:22         ` Michael S. Tsirkin
  0 siblings, 2 replies; 151+ messages in thread
From: Igor Mammedov @ 2022-11-18 14:55 UTC (permalink / raw)
  To: Volker Rümelin
  Cc: Michael S. Tsirkin, Peter Maydell, Ani Sinha, Paolo Bonzini,
	Richard Henderson, kraxel, qemu-devel, stefanha

On Fri, 18 Nov 2022 14:08:36 +0100
Igor Mammedov <imammedo@redhat.com> wrote:

> On Thu, 17 Nov 2022 22:51:46 +0100
> Volker Rümelin <vr_qemu@t-online.de> wrote:
[...]
> > since this patch SeaBIOS no longer detects the PS/2 keyboard. This means 
> > there's no keyboard in SeaBIOS, GRUB or FreeDOS. OVMF and Linux detect 
> > the PS/2 keyboard without issues.
> > 
> > Here are a few lines from the SeaBIOS debug log.
> > 
> > table(50434146)=0x007e1971 (via rsdt)
> > ACPI: parse DSDT at 0x007e0040 (len 6449)
> > parse_termlist: parse error, skip from 92/465
> > Scan for VGA option rom
> > Running option rom at c000:0003
> > Start SeaVGABIOS (version rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org)
> > 
> > and later
> > 
> > SeaBIOS (version rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org)
> > ACPI: no PS/2 keyboard present  
it was a bug on SeaBIOS side, we need it to parse Alias term in AML
instead of choking on it

proposed patch:
 https://mail.coreboot.org/hyperkitty/list/seabios@seabios.org/thread/RGPL7HESH5U5JRLEO6FP77CZVHZK5J65/

PS:
it's probably too late for it to make into 7.2



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 46/83] acpi: pc/q35: drop ad-hoc PCI-ISA bridge AML routines and let bus ennumeration generate AML
  2022-11-18 14:55       ` Igor Mammedov
@ 2022-11-19  8:49         ` Volker Rümelin
  2022-11-21  7:27           ` Igor Mammedov
  2022-11-19 17:22         ` Michael S. Tsirkin
  1 sibling, 1 reply; 151+ messages in thread
From: Volker Rümelin @ 2022-11-19  8:49 UTC (permalink / raw)
  To: Igor Mammedov
  Cc: Michael S. Tsirkin, Peter Maydell, Ani Sinha, Paolo Bonzini,
	Richard Henderson, kraxel, stefanha, qemu-devel

Am 18.11.22 um 15:55 schrieb Igor Mammedov:
> On Fri, 18 Nov 2022 14:08:36 +0100
> Igor Mammedov <imammedo@redhat.com> wrote:
>
>> On Thu, 17 Nov 2022 22:51:46 +0100
>> Volker Rümelin <vr_qemu@t-online.de> wrote:
> [...]
>>> since this patch SeaBIOS no longer detects the PS/2 keyboard. This means
>>> there's no keyboard in SeaBIOS, GRUB or FreeDOS. OVMF and Linux detect
>>> the PS/2 keyboard without issues.
>>>
>>> Here are a few lines from the SeaBIOS debug log.
>>>
>>> table(50434146)=0x007e1971 (via rsdt)
>>> ACPI: parse DSDT at 0x007e0040 (len 6449)
>>> parse_termlist: parse error, skip from 92/465
>>> Scan for VGA option rom
>>> Running option rom at c000:0003
>>> Start SeaVGABIOS (version rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org)
>>>
>>> and later
>>>
>>> SeaBIOS (version rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org)
>>> ACPI: no PS/2 keyboard present
> it was a bug on SeaBIOS side, we need it to parse Alias term in AML
> instead of choking on it
>
> proposed patch:
>   https://mail.coreboot.org/hyperkitty/list/seabios@seabios.org/thread/RGPL7HESH5U5JRLEO6FP77CZVHZK5J65/
>
> PS:
> it's probably too late for it to make into 7.2
>

The proposed patch works.

It may still be an option to revert the commit 47a373faa6 (acpi: pc/q35: 
drop ad-hoc PCI-ISA bridge AML routines and let bus ennumeration 
generate AML). If an older QEMU version is migrated to QEMU 7.2.0 and 
later and the guest reboots afterwards, it may end up without a working 
keyboard because the migrated SeaBIOS is an older version.

With best regards,
Volker


^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 30/83] virtio: core: vq reset feature negotation support
  2022-11-18 14:32   ` Stefano Garzarella
  2022-11-18 14:39     ` Stefano Garzarella
@ 2022-11-19 17:19     ` Michael S. Tsirkin
  2022-11-21  6:17       ` Jason Wang
  1 sibling, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-19 17:19 UTC (permalink / raw)
  To: Stefano Garzarella
  Cc: qemu-devel, Peter Maydell, Kangjie Xu, Xuan Zhuo, Jason Wang,
	Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
	Yanan Wang, Alex Bennée

On Fri, Nov 18, 2022 at 03:32:56PM +0100, Stefano Garzarella wrote:
> Hi,
> starting from this commit 69e1c14aa2 ("virtio: core: vq reset feature 
> negotation support"), vhost-user-vsock and vhost-vsock fails while 
> setting the device features, because VIRTIO_F_RING_RESET is not masked.
> 
> I'm not sure vsock is the only one affected.
> 
> We could fix in two ways:
> 
> 1) Masking VIRTIO_F_RING_RESET when we call vhost_get_features:
> 
> diff --git a/hw/virtio/vhost-vsock-common.c b/hw/virtio/vhost-vsock-common.c
> index 29b9ab4f72..e671cc695f 100644
> --- a/hw/virtio/vhost-vsock-common.c
> +++ b/hw/virtio/vhost-vsock-common.c
> @@ -21,6 +21,7 @@
>  
>  const int feature_bits[] = {
>      VIRTIO_VSOCK_F_SEQPACKET,
> +    VIRTIO_F_RING_RESET,
>      VHOST_INVALID_FEATURE_BIT
>  };
> 

Let's do this, we need to be conservative.


> 2) Or using directly the features of the device. That way we also handle 
> other features that we may have already had to mask but never did.
> 
> diff --git a/hw/virtio/vhost-vsock-common.c b/hw/virtio/vhost-vsock-common.c
> index 29b9ab4f72..41a665082c 100644
> --- a/hw/virtio/vhost-vsock-common.c
> +++ b/hw/virtio/vhost-vsock-common.c
> @@ -33,7 +33,7 @@ uint64_t vhost_vsock_common_get_features(VirtIODevice *vdev, uint64_t features,
>          virtio_add_feature(&features, VIRTIO_VSOCK_F_SEQPACKET);
>      }
> 
> -    features = vhost_get_features(&vvc->vhost_dev, feature_bits, features);
> +    features &= vvc->vhost_dev.features;
> 
>      if (vvc->seqpacket == ON_OFF_AUTO_ON &&
>          !virtio_has_feature(features, VIRTIO_VSOCK_F_SEQPACKET)) {
> 
> 
> I may be missing the real reason for calling vhost_get_features(), 
> though.
> 
> @Michael what do you recommend we do?
> 
> Thanks,
> Stefano
> 
> On Tue, Nov 8, 2022 at 12:06 AM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > From: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> >
> > A a new command line parameter "queue_reset" is added.
> >
> > Meanwhile, the vq reset feature is disabled for pre-7.2 machines.
> >
> > Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > Acked-by: Jason Wang <jasowang@redhat.com>
> > Message-Id: <20221017092558.111082-5-xuanzhuo@linux.alibaba.com>
> > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > ---
> >  include/hw/virtio/virtio.h | 4 +++-
> >  hw/core/machine.c          | 4 +++-
> >  2 files changed, 6 insertions(+), 2 deletions(-)
> >
> > diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> > index b00b3fcf31..1423dba379 100644
> > --- a/include/hw/virtio/virtio.h
> > +++ b/include/hw/virtio/virtio.h
> > @@ -313,7 +313,9 @@ typedef struct VirtIORNGConf VirtIORNGConf;
> >      DEFINE_PROP_BIT64("iommu_platform", _state, _field, \
> >                        VIRTIO_F_IOMMU_PLATFORM, false), \
> >      DEFINE_PROP_BIT64("packed", _state, _field, \
> > -                      VIRTIO_F_RING_PACKED, false)
> > +                      VIRTIO_F_RING_PACKED, false), \
> > +    DEFINE_PROP_BIT64("queue_reset", _state, _field, \
> > +                      VIRTIO_F_RING_RESET, true)
> >
> >  hwaddr virtio_queue_get_desc_addr(VirtIODevice *vdev, int n);
> >  bool virtio_queue_enabled_legacy(VirtIODevice *vdev, int n);
> > diff --git a/hw/core/machine.c b/hw/core/machine.c
> > index aa520e74a8..907fa78ff0 100644
> > --- a/hw/core/machine.c
> > +++ b/hw/core/machine.c
> > @@ -40,7 +40,9 @@
> >  #include "hw/virtio/virtio-pci.h"
> >  #include "qom/object_interfaces.h"
> >
> > -GlobalProperty hw_compat_7_1[] = {};
> > +GlobalProperty hw_compat_7_1[] = {
> > +    { "virtio-device", "queue_reset", "false" },
> > +};
> >  const size_t hw_compat_7_1_len = G_N_ELEMENTS(hw_compat_7_1);
> >
> >  GlobalProperty hw_compat_7_0[] = {
> > --
> > MST
> >
> >



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 46/83] acpi: pc/q35: drop ad-hoc PCI-ISA bridge AML routines and let bus ennumeration generate AML
  2022-11-18 14:55       ` Igor Mammedov
  2022-11-19  8:49         ` Volker Rümelin
@ 2022-11-19 17:22         ` Michael S. Tsirkin
  2022-11-21  7:23           ` Igor Mammedov
  1 sibling, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-19 17:22 UTC (permalink / raw)
  To: Igor Mammedov
  Cc: Volker Rümelin, Peter Maydell, Ani Sinha, Paolo Bonzini,
	Richard Henderson, kraxel, qemu-devel, stefanha

On Fri, Nov 18, 2022 at 03:55:17PM +0100, Igor Mammedov wrote:
> On Fri, 18 Nov 2022 14:08:36 +0100
> Igor Mammedov <imammedo@redhat.com> wrote:
> 
> > On Thu, 17 Nov 2022 22:51:46 +0100
> > Volker Rümelin <vr_qemu@t-online.de> wrote:
> [...]
> > > since this patch SeaBIOS no longer detects the PS/2 keyboard. This means 
> > > there's no keyboard in SeaBIOS, GRUB or FreeDOS. OVMF and Linux detect 
> > > the PS/2 keyboard without issues.
> > > 
> > > Here are a few lines from the SeaBIOS debug log.
> > > 
> > > table(50434146)=0x007e1971 (via rsdt)
> > > ACPI: parse DSDT at 0x007e0040 (len 6449)
> > > parse_termlist: parse error, skip from 92/465
> > > Scan for VGA option rom
> > > Running option rom at c000:0003
> > > Start SeaVGABIOS (version rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org)
> > > 
> > > and later
> > > 
> > > SeaBIOS (version rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org)
> > > ACPI: no PS/2 keyboard present  
> it was a bug on SeaBIOS side, we need it to parse Alias term in AML
> instead of choking on it
> 
> proposed patch:
>  https://mail.coreboot.org/hyperkitty/list/seabios@seabios.org/thread/RGPL7HESH5U5JRLEO6FP77CZVHZK5J65/
> 
> PS:
> it's probably too late for it to make into 7.2

Right. So revert?



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 30/83] virtio: core: vq reset feature negotation support
  2022-11-19 17:19     ` Michael S. Tsirkin
@ 2022-11-21  6:17       ` Jason Wang
  2022-11-21  7:07         ` Michael S. Tsirkin
  0 siblings, 1 reply; 151+ messages in thread
From: Jason Wang @ 2022-11-21  6:17 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Stefano Garzarella, qemu-devel, Peter Maydell, Kangjie Xu,
	Xuan Zhuo, Eduardo Habkost, Marcel Apfelbaum,
	Philippe Mathieu-Daudé,
	Yanan Wang, Alex Bennée

On Sun, Nov 20, 2022 at 1:19 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Fri, Nov 18, 2022 at 03:32:56PM +0100, Stefano Garzarella wrote:
> > Hi,
> > starting from this commit 69e1c14aa2 ("virtio: core: vq reset feature
> > negotation support"), vhost-user-vsock and vhost-vsock fails while
> > setting the device features, because VIRTIO_F_RING_RESET is not masked.
> >
> > I'm not sure vsock is the only one affected.
> >
> > We could fix in two ways:
> >
> > 1) Masking VIRTIO_F_RING_RESET when we call vhost_get_features:
> >
> > diff --git a/hw/virtio/vhost-vsock-common.c b/hw/virtio/vhost-vsock-common.c
> > index 29b9ab4f72..e671cc695f 100644
> > --- a/hw/virtio/vhost-vsock-common.c
> > +++ b/hw/virtio/vhost-vsock-common.c
> > @@ -21,6 +21,7 @@
> >
> >  const int feature_bits[] = {
> >      VIRTIO_VSOCK_F_SEQPACKET,
> > +    VIRTIO_F_RING_RESET,
> >      VHOST_INVALID_FEATURE_BIT
> >  };
> >
>
> Let's do this, we need to be conservative.

Ack.

Thanks

>
>
> > 2) Or using directly the features of the device. That way we also handle
> > other features that we may have already had to mask but never did.
> >
> > diff --git a/hw/virtio/vhost-vsock-common.c b/hw/virtio/vhost-vsock-common.c
> > index 29b9ab4f72..41a665082c 100644
> > --- a/hw/virtio/vhost-vsock-common.c
> > +++ b/hw/virtio/vhost-vsock-common.c
> > @@ -33,7 +33,7 @@ uint64_t vhost_vsock_common_get_features(VirtIODevice *vdev, uint64_t features,
> >          virtio_add_feature(&features, VIRTIO_VSOCK_F_SEQPACKET);
> >      }
> >
> > -    features = vhost_get_features(&vvc->vhost_dev, feature_bits, features);
> > +    features &= vvc->vhost_dev.features;
> >
> >      if (vvc->seqpacket == ON_OFF_AUTO_ON &&
> >          !virtio_has_feature(features, VIRTIO_VSOCK_F_SEQPACKET)) {
> >
> >
> > I may be missing the real reason for calling vhost_get_features(),
> > though.
> >
> > @Michael what do you recommend we do?
> >
> > Thanks,
> > Stefano
> >
> > On Tue, Nov 8, 2022 at 12:06 AM Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > From: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > >
> > > A a new command line parameter "queue_reset" is added.
> > >
> > > Meanwhile, the vq reset feature is disabled for pre-7.2 machines.
> > >
> > > Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > Acked-by: Jason Wang <jasowang@redhat.com>
> > > Message-Id: <20221017092558.111082-5-xuanzhuo@linux.alibaba.com>
> > > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > ---
> > >  include/hw/virtio/virtio.h | 4 +++-
> > >  hw/core/machine.c          | 4 +++-
> > >  2 files changed, 6 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> > > index b00b3fcf31..1423dba379 100644
> > > --- a/include/hw/virtio/virtio.h
> > > +++ b/include/hw/virtio/virtio.h
> > > @@ -313,7 +313,9 @@ typedef struct VirtIORNGConf VirtIORNGConf;
> > >      DEFINE_PROP_BIT64("iommu_platform", _state, _field, \
> > >                        VIRTIO_F_IOMMU_PLATFORM, false), \
> > >      DEFINE_PROP_BIT64("packed", _state, _field, \
> > > -                      VIRTIO_F_RING_PACKED, false)
> > > +                      VIRTIO_F_RING_PACKED, false), \
> > > +    DEFINE_PROP_BIT64("queue_reset", _state, _field, \
> > > +                      VIRTIO_F_RING_RESET, true)
> > >
> > >  hwaddr virtio_queue_get_desc_addr(VirtIODevice *vdev, int n);
> > >  bool virtio_queue_enabled_legacy(VirtIODevice *vdev, int n);
> > > diff --git a/hw/core/machine.c b/hw/core/machine.c
> > > index aa520e74a8..907fa78ff0 100644
> > > --- a/hw/core/machine.c
> > > +++ b/hw/core/machine.c
> > > @@ -40,7 +40,9 @@
> > >  #include "hw/virtio/virtio-pci.h"
> > >  #include "qom/object_interfaces.h"
> > >
> > > -GlobalProperty hw_compat_7_1[] = {};
> > > +GlobalProperty hw_compat_7_1[] = {
> > > +    { "virtio-device", "queue_reset", "false" },
> > > +};
> > >  const size_t hw_compat_7_1_len = G_N_ELEMENTS(hw_compat_7_1);
> > >
> > >  GlobalProperty hw_compat_7_0[] = {
> > > --
> > > MST
> > >
> > >
>



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 30/83] virtio: core: vq reset feature negotation support
  2022-11-21  6:17       ` Jason Wang
@ 2022-11-21  7:07         ` Michael S. Tsirkin
  2022-11-21  8:20           ` Stefano Garzarella
  0 siblings, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-21  7:07 UTC (permalink / raw)
  To: Jason Wang
  Cc: Stefano Garzarella, qemu-devel, Peter Maydell, Kangjie Xu,
	Xuan Zhuo, Eduardo Habkost, Marcel Apfelbaum,
	Philippe Mathieu-Daudé,
	Yanan Wang, Alex Bennée

On Mon, Nov 21, 2022 at 02:17:02PM +0800, Jason Wang wrote:
> On Sun, Nov 20, 2022 at 1:19 AM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Fri, Nov 18, 2022 at 03:32:56PM +0100, Stefano Garzarella wrote:
> > > Hi,
> > > starting from this commit 69e1c14aa2 ("virtio: core: vq reset feature
> > > negotation support"), vhost-user-vsock and vhost-vsock fails while
> > > setting the device features, because VIRTIO_F_RING_RESET is not masked.
> > >
> > > I'm not sure vsock is the only one affected.
> > >
> > > We could fix in two ways:
> > >
> > > 1) Masking VIRTIO_F_RING_RESET when we call vhost_get_features:
> > >
> > > diff --git a/hw/virtio/vhost-vsock-common.c b/hw/virtio/vhost-vsock-common.c
> > > index 29b9ab4f72..e671cc695f 100644
> > > --- a/hw/virtio/vhost-vsock-common.c
> > > +++ b/hw/virtio/vhost-vsock-common.c
> > > @@ -21,6 +21,7 @@
> > >
> > >  const int feature_bits[] = {
> > >      VIRTIO_VSOCK_F_SEQPACKET,
> > > +    VIRTIO_F_RING_RESET,
> > >      VHOST_INVALID_FEATURE_BIT
> > >  };
> > >
> >
> > Let's do this, we need to be conservative.
> 
> Ack.
> 
> Thanks


Patch pls? Stefano?

> >
> >
> > > 2) Or using directly the features of the device. That way we also handle
> > > other features that we may have already had to mask but never did.
> > >
> > > diff --git a/hw/virtio/vhost-vsock-common.c b/hw/virtio/vhost-vsock-common.c
> > > index 29b9ab4f72..41a665082c 100644
> > > --- a/hw/virtio/vhost-vsock-common.c
> > > +++ b/hw/virtio/vhost-vsock-common.c
> > > @@ -33,7 +33,7 @@ uint64_t vhost_vsock_common_get_features(VirtIODevice *vdev, uint64_t features,
> > >          virtio_add_feature(&features, VIRTIO_VSOCK_F_SEQPACKET);
> > >      }
> > >
> > > -    features = vhost_get_features(&vvc->vhost_dev, feature_bits, features);
> > > +    features &= vvc->vhost_dev.features;
> > >
> > >      if (vvc->seqpacket == ON_OFF_AUTO_ON &&
> > >          !virtio_has_feature(features, VIRTIO_VSOCK_F_SEQPACKET)) {
> > >
> > >
> > > I may be missing the real reason for calling vhost_get_features(),
> > > though.
> > >
> > > @Michael what do you recommend we do?
> > >
> > > Thanks,
> > > Stefano
> > >
> > > On Tue, Nov 8, 2022 at 12:06 AM Michael S. Tsirkin <mst@redhat.com> wrote:
> > > >
> > > > From: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > > >
> > > > A a new command line parameter "queue_reset" is added.
> > > >
> > > > Meanwhile, the vq reset feature is disabled for pre-7.2 machines.
> > > >
> > > > Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> > > > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > > > Acked-by: Jason Wang <jasowang@redhat.com>
> > > > Message-Id: <20221017092558.111082-5-xuanzhuo@linux.alibaba.com>
> > > > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > > ---
> > > >  include/hw/virtio/virtio.h | 4 +++-
> > > >  hw/core/machine.c          | 4 +++-
> > > >  2 files changed, 6 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> > > > index b00b3fcf31..1423dba379 100644
> > > > --- a/include/hw/virtio/virtio.h
> > > > +++ b/include/hw/virtio/virtio.h
> > > > @@ -313,7 +313,9 @@ typedef struct VirtIORNGConf VirtIORNGConf;
> > > >      DEFINE_PROP_BIT64("iommu_platform", _state, _field, \
> > > >                        VIRTIO_F_IOMMU_PLATFORM, false), \
> > > >      DEFINE_PROP_BIT64("packed", _state, _field, \
> > > > -                      VIRTIO_F_RING_PACKED, false)
> > > > +                      VIRTIO_F_RING_PACKED, false), \
> > > > +    DEFINE_PROP_BIT64("queue_reset", _state, _field, \
> > > > +                      VIRTIO_F_RING_RESET, true)
> > > >
> > > >  hwaddr virtio_queue_get_desc_addr(VirtIODevice *vdev, int n);
> > > >  bool virtio_queue_enabled_legacy(VirtIODevice *vdev, int n);
> > > > diff --git a/hw/core/machine.c b/hw/core/machine.c
> > > > index aa520e74a8..907fa78ff0 100644
> > > > --- a/hw/core/machine.c
> > > > +++ b/hw/core/machine.c
> > > > @@ -40,7 +40,9 @@
> > > >  #include "hw/virtio/virtio-pci.h"
> > > >  #include "qom/object_interfaces.h"
> > > >
> > > > -GlobalProperty hw_compat_7_1[] = {};
> > > > +GlobalProperty hw_compat_7_1[] = {
> > > > +    { "virtio-device", "queue_reset", "false" },
> > > > +};
> > > >  const size_t hw_compat_7_1_len = G_N_ELEMENTS(hw_compat_7_1);
> > > >
> > > >  GlobalProperty hw_compat_7_0[] = {
> > > > --
> > > > MST
> > > >
> > > >
> >



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 46/83] acpi: pc/q35: drop ad-hoc PCI-ISA bridge AML routines and let bus ennumeration generate AML
  2022-11-19 17:22         ` Michael S. Tsirkin
@ 2022-11-21  7:23           ` Igor Mammedov
  2022-11-21  7:50             ` Michael S. Tsirkin
  0 siblings, 1 reply; 151+ messages in thread
From: Igor Mammedov @ 2022-11-21  7:23 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Volker Rümelin, Peter Maydell, Ani Sinha, Paolo Bonzini,
	Richard Henderson, kraxel, qemu-devel, stefanha

On Sat, 19 Nov 2022 12:22:13 -0500
"Michael S. Tsirkin" <mst@redhat.com> wrote:

> On Fri, Nov 18, 2022 at 03:55:17PM +0100, Igor Mammedov wrote:
> > On Fri, 18 Nov 2022 14:08:36 +0100
> > Igor Mammedov <imammedo@redhat.com> wrote:
> >   
> > > On Thu, 17 Nov 2022 22:51:46 +0100
> > > Volker Rümelin <vr_qemu@t-online.de> wrote:  
> > [...]  
> > > > since this patch SeaBIOS no longer detects the PS/2 keyboard. This means 
> > > > there's no keyboard in SeaBIOS, GRUB or FreeDOS. OVMF and Linux detect 
> > > > the PS/2 keyboard without issues.
> > > > 
> > > > Here are a few lines from the SeaBIOS debug log.
> > > > 
> > > > table(50434146)=0x007e1971 (via rsdt)
> > > > ACPI: parse DSDT at 0x007e0040 (len 6449)
> > > > parse_termlist: parse error, skip from 92/465
> > > > Scan for VGA option rom
> > > > Running option rom at c000:0003
> > > > Start SeaVGABIOS (version rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org)
> > > > 
> > > > and later
> > > > 
> > > > SeaBIOS (version rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org)
> > > > ACPI: no PS/2 keyboard present    
> > it was a bug on SeaBIOS side, we need it to parse Alias term in AML
> > instead of choking on it
> > 
> > proposed patch:
> >  https://mail.coreboot.org/hyperkitty/list/seabios@seabios.org/thread/RGPL7HESH5U5JRLEO6FP77CZVHZK5J65/
> > 
> > PS:
> > it's probably too late for it to make into 7.2  
> 
> Right. So revert?

let me check first what happens with migration case,
and maybe I can come up with a temporary hack to avoid aliases on QEMU side,
probably it will be something ugly but should do the job

> 



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 46/83] acpi: pc/q35: drop ad-hoc PCI-ISA bridge AML routines and let bus ennumeration generate AML
  2022-11-19  8:49         ` Volker Rümelin
@ 2022-11-21  7:27           ` Igor Mammedov
  0 siblings, 0 replies; 151+ messages in thread
From: Igor Mammedov @ 2022-11-21  7:27 UTC (permalink / raw)
  To: Volker Rümelin
  Cc: Michael S. Tsirkin, Peter Maydell, Ani Sinha, Paolo Bonzini,
	Richard Henderson, kraxel, stefanha, qemu-devel

On Sat, 19 Nov 2022 09:49:39 +0100
Volker Rümelin <vr_qemu@t-online.de> wrote:

> Am 18.11.22 um 15:55 schrieb Igor Mammedov:
> > On Fri, 18 Nov 2022 14:08:36 +0100
> > Igor Mammedov <imammedo@redhat.com> wrote:
> >  
> >> On Thu, 17 Nov 2022 22:51:46 +0100
> >> Volker Rümelin <vr_qemu@t-online.de> wrote:  
> > [...]  
> >>> since this patch SeaBIOS no longer detects the PS/2 keyboard. This means
> >>> there's no keyboard in SeaBIOS, GRUB or FreeDOS. OVMF and Linux detect
> >>> the PS/2 keyboard without issues.
> >>>
> >>> Here are a few lines from the SeaBIOS debug log.
> >>>
> >>> table(50434146)=0x007e1971 (via rsdt)
> >>> ACPI: parse DSDT at 0x007e0040 (len 6449)
> >>> parse_termlist: parse error, skip from 92/465
> >>> Scan for VGA option rom
> >>> Running option rom at c000:0003
> >>> Start SeaVGABIOS (version rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org)
> >>>
> >>> and later
> >>>
> >>> SeaBIOS (version rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org)
> >>> ACPI: no PS/2 keyboard present  
> > it was a bug on SeaBIOS side, we need it to parse Alias term in AML
> > instead of choking on it
> >
> > proposed patch:
> >   https://mail.coreboot.org/hyperkitty/list/seabios@seabios.org/thread/RGPL7HESH5U5JRLEO6FP77CZVHZK5J65/
> >
> > PS:
> > it's probably too late for it to make into 7.2
> >  
> 
> The proposed patch works.
> 
> It may still be an option to revert the commit 47a373faa6 (acpi: pc/q35: 
> drop ad-hoc PCI-ISA bridge AML routines and let bus ennumeration 
> generate AML). If an older QEMU version is migrated to QEMU 7.2.0 and 
> later and the guest reboots afterwards, it may end up without a working 
> keyboard because the migrated SeaBIOS is an older version.

ACPI blobs generated on old QEMU should be migrated as well,
so I'd expect it should be fine.
Problem will manifest itself only after VM was shut down and started anew.

Anyways lets see if a QEMU workaround is possible.

> With best regards,
> Volker
> 



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 46/83] acpi: pc/q35: drop ad-hoc PCI-ISA bridge AML routines and let bus ennumeration generate AML
  2022-11-21  7:23           ` Igor Mammedov
@ 2022-11-21  7:50             ` Michael S. Tsirkin
  0 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2022-11-21  7:50 UTC (permalink / raw)
  To: Igor Mammedov
  Cc: Volker Rümelin, Peter Maydell, Ani Sinha, Paolo Bonzini,
	Richard Henderson, kraxel, qemu-devel, stefanha

On Mon, Nov 21, 2022 at 08:23:15AM +0100, Igor Mammedov wrote:
> On Sat, 19 Nov 2022 12:22:13 -0500
> "Michael S. Tsirkin" <mst@redhat.com> wrote:
> 
> > On Fri, Nov 18, 2022 at 03:55:17PM +0100, Igor Mammedov wrote:
> > > On Fri, 18 Nov 2022 14:08:36 +0100
> > > Igor Mammedov <imammedo@redhat.com> wrote:
> > >   
> > > > On Thu, 17 Nov 2022 22:51:46 +0100
> > > > Volker Rümelin <vr_qemu@t-online.de> wrote:  
> > > [...]  
> > > > > since this patch SeaBIOS no longer detects the PS/2 keyboard. This means 
> > > > > there's no keyboard in SeaBIOS, GRUB or FreeDOS. OVMF and Linux detect 
> > > > > the PS/2 keyboard without issues.
> > > > > 
> > > > > Here are a few lines from the SeaBIOS debug log.
> > > > > 
> > > > > table(50434146)=0x007e1971 (via rsdt)
> > > > > ACPI: parse DSDT at 0x007e0040 (len 6449)
> > > > > parse_termlist: parse error, skip from 92/465
> > > > > Scan for VGA option rom
> > > > > Running option rom at c000:0003
> > > > > Start SeaVGABIOS (version rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org)
> > > > > 
> > > > > and later
> > > > > 
> > > > > SeaBIOS (version rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org)
> > > > > ACPI: no PS/2 keyboard present    
> > > it was a bug on SeaBIOS side, we need it to parse Alias term in AML
> > > instead of choking on it
> > > 
> > > proposed patch:
> > >  https://mail.coreboot.org/hyperkitty/list/seabios@seabios.org/thread/RGPL7HESH5U5JRLEO6FP77CZVHZK5J65/
> > > 
> > > PS:
> > > it's probably too late for it to make into 7.2  
> > 
> > Right. So revert?
> 
> let me check first what happens with migration case,
> and maybe I can come up with a temporary hack to avoid aliases on QEMU side,
> probably it will be something ugly but should do the job
> 
> > 

Given the timing I'd prefer the revert. But if you insist let's see how
small that turns out to be.

-- 
MST



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 30/83] virtio: core: vq reset feature negotation support
  2022-11-21  7:07         ` Michael S. Tsirkin
@ 2022-11-21  8:20           ` Stefano Garzarella
  0 siblings, 0 replies; 151+ messages in thread
From: Stefano Garzarella @ 2022-11-21  8:20 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Jason Wang, qemu-devel, Peter Maydell, Kangjie Xu, Xuan Zhuo,
	Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
	Yanan Wang, Alex Bennée

On Mon, Nov 21, 2022 at 02:07:41AM -0500, Michael S. Tsirkin wrote:
>On Mon, Nov 21, 2022 at 02:17:02PM +0800, Jason Wang wrote:
>> On Sun, Nov 20, 2022 at 1:19 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>> >
>> > On Fri, Nov 18, 2022 at 03:32:56PM +0100, Stefano Garzarella wrote:
>> > > Hi,
>> > > starting from this commit 69e1c14aa2 ("virtio: core: vq reset feature
>> > > negotation support"), vhost-user-vsock and vhost-vsock fails while
>> > > setting the device features, because VIRTIO_F_RING_RESET is not masked.
>> > >
>> > > I'm not sure vsock is the only one affected.
>> > >
>> > > We could fix in two ways:
>> > >
>> > > 1) Masking VIRTIO_F_RING_RESET when we call vhost_get_features:
>> > >
>> > > diff --git a/hw/virtio/vhost-vsock-common.c b/hw/virtio/vhost-vsock-common.c
>> > > index 29b9ab4f72..e671cc695f 100644
>> > > --- a/hw/virtio/vhost-vsock-common.c
>> > > +++ b/hw/virtio/vhost-vsock-common.c
>> > > @@ -21,6 +21,7 @@
>> > >
>> > >  const int feature_bits[] = {
>> > >      VIRTIO_VSOCK_F_SEQPACKET,
>> > > +    VIRTIO_F_RING_RESET,
>> > >      VHOST_INVALID_FEATURE_BIT
>> > >  };
>> > >
>> >
>> > Let's do this, we need to be conservative.
>>
>> Ack.

Okay, thanks for the feedbacks!

>>
>> Thanks
>
>
>Patch pls? Stefano?
>

Yep, I'll send it.

Should we do the same in the other vhost and vhost-user devices?
IIUC now we only filter it in vhost-net.

Then I'll try to see if we can rework how we handle features to make 
sure we avoid this in the future. Maybe by creating an array of core 
features always filtered, because for example this one is not device 
specific.

Thanks,
Stefano



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
  2022-11-07 22:53 ` [PULL v4 76/83] vhost-user: Support vhost_dev_start Michael S. Tsirkin
@ 2023-01-06 14:21   ` Laurent Vivier
  2023-01-09 10:55     ` Michael S. Tsirkin
  0 siblings, 1 reply; 151+ messages in thread
From: Laurent Vivier @ 2023-01-06 14:21 UTC (permalink / raw)
  To: Michael S. Tsirkin, qemu-devel; +Cc: Peter Maydell, Yajun Wu, Parav Pandit

Hi,

it seems this patch breaks vhost-user with DPDK.

See https://bugzilla.redhat.com/show_bug.cgi?id=2155173

it seems QEMU doesn't receive the expected commands sequence:

Received unexpected msg type. Expected 22 received 40
Fail to update device iotlb
Received unexpected msg type. Expected 40 received 22
Received unexpected msg type. Expected 22 received 11
Fail to update device iotlb
Received unexpected msg type. Expected 11 received 22
vhost VQ 1 ring restore failed: -71: Protocol error (71)
Received unexpected msg type. Expected 22 received 11
Fail to update device iotlb
Received unexpected msg type. Expected 11 received 22
vhost VQ 0 ring restore failed: -71: Protocol error (71)
unable to start vhost net: 71: falling back on userspace virtio

It receives VHOST_USER_GET_STATUS (40) when it expects VHOST_USER_IOTLB_MSG (22)
and VHOST_USER_IOTLB_MSG when it expects VHOST_USER_GET_STATUS.
and VHOST_USER_GET_VRING_BASE (11) when it expect VHOST_USER_GET_STATUS and so on.

Any idea?

Thanks,
Laurent

On 11/7/22 23:53, Michael S. Tsirkin wrote:
> From: Yajun Wu <yajunw@nvidia.com>
> 
> The motivation of adding vhost-user vhost_dev_start support is to
> improve backend configuration speed and reduce live migration VM
> downtime.
> 
> Today VQ configuration is issued one by one. For virtio net with
> multi-queue support, backend needs to update RSS (Receive side
> scaling) on every rx queue enable. Updating RSS is time-consuming
> (typical time like 7ms).
> 
> Implement already defined vhost status and message in the vhost
> specification [1].
> (a) VHOST_USER_PROTOCOL_F_STATUS
> (b) VHOST_USER_SET_STATUS
> (c) VHOST_USER_GET_STATUS
> 
> Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK for
> device start and reset(0) for device stop.
> 
> On reception of the DRIVER_OK message, backend can apply the needed setting
> only once (instead of incremental) and also utilize parallelism on enabling
> queues.
> 
> This improves QEMU's live migration downtime with vhost user backend
> implementation by great margin, specially for the large number of VQs of 64
> from 800 msec to 250 msec.
> 
> [1] https://qemu-project.gitlab.io/qemu/interop/vhost-user.html
> 
> Signed-off-by: Yajun Wu <yajunw@nvidia.com>
> Acked-by: Parav Pandit <parav@nvidia.com>
> Message-Id: <20221017064452.1226514-3-yajunw@nvidia.com>
> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> ---
>   hw/virtio/vhost-user.c | 74 +++++++++++++++++++++++++++++++++++++++++-
>   1 file changed, 73 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index d256ce589b..abe23d4ebe 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -81,6 +81,7 @@ enum VhostUserProtocolFeature {
>       VHOST_USER_PROTOCOL_F_RESET_DEVICE = 13,
>       /* Feature 14 reserved for VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS. */
>       VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS = 15,
> +    VHOST_USER_PROTOCOL_F_STATUS = 16,
>       VHOST_USER_PROTOCOL_F_MAX
>   };
>   
> @@ -126,6 +127,8 @@ typedef enum VhostUserRequest {
>       VHOST_USER_GET_MAX_MEM_SLOTS = 36,
>       VHOST_USER_ADD_MEM_REG = 37,
>       VHOST_USER_REM_MEM_REG = 38,
> +    VHOST_USER_SET_STATUS = 39,
> +    VHOST_USER_GET_STATUS = 40,
>       VHOST_USER_MAX
>   } VhostUserRequest;
>   
> @@ -1452,6 +1455,43 @@ static int vhost_user_set_u64(struct vhost_dev *dev, int request, uint64_t u64,
>       return 0;
>   }
>   
> +static int vhost_user_set_status(struct vhost_dev *dev, uint8_t status)
> +{
> +    return vhost_user_set_u64(dev, VHOST_USER_SET_STATUS, status, false);
> +}
> +
> +static int vhost_user_get_status(struct vhost_dev *dev, uint8_t *status)
> +{
> +    uint64_t value;
> +    int ret;
> +
> +    ret = vhost_user_get_u64(dev, VHOST_USER_GET_STATUS, &value);
> +    if (ret < 0) {
> +        return ret;
> +    }
> +    *status = value;
> +
> +    return 0;
> +}
> +
> +static int vhost_user_add_status(struct vhost_dev *dev, uint8_t status)
> +{
> +    uint8_t s;
> +    int ret;
> +
> +    ret = vhost_user_get_status(dev, &s);
> +    if (ret < 0) {
> +        return ret;
> +    }
> +
> +    if ((s & status) == status) {
> +        return 0;
> +    }
> +    s |= status;
> +
> +    return vhost_user_set_status(dev, s);
> +}
> +
>   static int vhost_user_set_features(struct vhost_dev *dev,
>                                      uint64_t features)
>   {
> @@ -1460,6 +1500,7 @@ static int vhost_user_set_features(struct vhost_dev *dev,
>        * backend is actually logging changes
>        */
>       bool log_enabled = features & (0x1ULL << VHOST_F_LOG_ALL);
> +    int ret;
>   
>       /*
>        * We need to include any extra backend only feature bits that
> @@ -1467,9 +1508,18 @@ static int vhost_user_set_features(struct vhost_dev *dev,
>        * VHOST_USER_F_PROTOCOL_FEATURES bit for enabling protocol
>        * features.
>        */
> -    return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES,
> +    ret = vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES,
>                                 features | dev->backend_features,
>                                 log_enabled);
> +
> +    if (virtio_has_feature(dev->protocol_features,
> +                           VHOST_USER_PROTOCOL_F_STATUS)) {
> +        if (!ret) {
> +            return vhost_user_add_status(dev, VIRTIO_CONFIG_S_FEATURES_OK);
> +        }
> +    }
> +
> +    return ret;
>   }
>   
>   static int vhost_user_set_protocol_features(struct vhost_dev *dev,
> @@ -2620,6 +2670,27 @@ void vhost_user_cleanup(VhostUserState *user)
>       user->chr = NULL;
>   }
>   
> +static int vhost_user_dev_start(struct vhost_dev *dev, bool started)
> +{
> +    if (!virtio_has_feature(dev->protocol_features,
> +                            VHOST_USER_PROTOCOL_F_STATUS)) {
> +        return 0;
> +    }
> +
> +    /* Set device status only for last queue pair */
> +    if (dev->vq_index + dev->nvqs != dev->vq_index_end) {
> +        return 0;
> +    }
> +
> +    if (started) {
> +        return vhost_user_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE |
> +                                          VIRTIO_CONFIG_S_DRIVER |
> +                                          VIRTIO_CONFIG_S_DRIVER_OK);
> +    } else {
> +        return vhost_user_set_status(dev, 0);
> +    }
> +}
> +
>   const VhostOps user_ops = {
>           .backend_type = VHOST_BACKEND_TYPE_USER,
>           .vhost_backend_init = vhost_user_backend_init,
> @@ -2654,4 +2725,5 @@ const VhostOps user_ops = {
>           .vhost_backend_mem_section_filter = vhost_user_mem_section_filter,
>           .vhost_get_inflight_fd = vhost_user_get_inflight_fd,
>           .vhost_set_inflight_fd = vhost_user_set_inflight_fd,
> +        .vhost_dev_start = vhost_user_dev_start,
>   };



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 42/83] virtio-rng-pci: Allow setting nvectors, so we can use MSI-X
  2022-11-07 22:50 ` [PULL v4 42/83] virtio-rng-pci: Allow setting nvectors, so we can use MSI-X Michael S. Tsirkin
@ 2023-01-09 10:20   ` Dr. David Alan Gilbert
  2023-01-09 10:38     ` Michael S. Tsirkin
  0 siblings, 1 reply; 151+ messages in thread
From: Dr. David Alan Gilbert @ 2023-01-09 10:20 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: qemu-devel, Peter Maydell, David Daney, Marcin Nowakowski,
	Philippe Mathieu-Daudé,
	Stefan Hajnoczi, quintela, peterx

* Michael S. Tsirkin (mst@redhat.com) wrote:
> From: David Daney <david.daney@fungible.com>
> 
> Most other virtio-pci devices allow MSI-X, let's have it for rng too.
> 
> Signed-off-by: David Daney <david.daney@fungible.com>
> Reviewed-by: Marcin Nowakowski <marcin.nowakowski@fungible.com>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@fungible.com>
> Message-Id: <20221014160947.66105-1-philmd@fungible.com>
> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

This breaks migration compatibility 7.1->7.2 :

(qemu) qemu: get_pci_config_device: Bad config data: i=0x34 read: 84 device: 98 cmask: ff wmask: 0 w1cmask:0
qemu: Failed to load PCIDevice:config
qemu: Failed to load virtio-rng:virtio
qemu: error while loading state for instance 0x0 of device '0000:00:03.0/virtio-rng'
qemu: load of migration failed: Invalid argument

because the destination is configured with msi-x but the source isn't.

The fix is in theory simple:
diff --git a/hw/core/machine.c b/hw/core/machine.c
index f589b92909..45459d1cef 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -45,6 +45,7 @@ const size_t hw_compat_7_2_len = G_N_ELEMENTS(hw_compat_7_2);
 
 GlobalProperty hw_compat_7_1[] = {
     { "virtio-device", "queue_reset", "false" },
+    { "virtio-rng-pci", "vectors", "0" },
 };
 const size_t hw_compat_7_1_len = G_N_ELEMENTS(hw_compat_7_1);

the gotcha is that will break 7.2->7.2-fixed.

(I guess you can also work around it by explicitly passing vectors=0 to
the virtio-rng on the cli)

Does anyone have preferences as to whether that should be fixed in the
7.2 world or left as is?

This is:
https://bugzilla.redhat.com/show_bug.cgi?id=2155749

Dave

> ---
>  hw/virtio/virtio-rng-pci.c | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)
> 
> diff --git a/hw/virtio/virtio-rng-pci.c b/hw/virtio/virtio-rng-pci.c
> index 151ece6f94..6e76f8b57b 100644
> --- a/hw/virtio/virtio-rng-pci.c
> +++ b/hw/virtio/virtio-rng-pci.c
> @@ -13,6 +13,7 @@
>  
>  #include "hw/virtio/virtio-pci.h"
>  #include "hw/virtio/virtio-rng.h"
> +#include "hw/qdev-properties.h"
>  #include "qapi/error.h"
>  #include "qemu/module.h"
>  #include "qom/object.h"
> @@ -31,11 +32,23 @@ struct VirtIORngPCI {
>      VirtIORNG vdev;
>  };
>  
> +static Property virtio_rng_properties[] = {
> +    DEFINE_PROP_BIT("ioeventfd", VirtIOPCIProxy, flags,
> +                    VIRTIO_PCI_FLAG_USE_IOEVENTFD_BIT, true),
> +    DEFINE_PROP_UINT32("vectors", VirtIOPCIProxy, nvectors,
> +                       DEV_NVECTORS_UNSPECIFIED),
> +    DEFINE_PROP_END_OF_LIST(),
> +};
> +
>  static void virtio_rng_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp)
>  {
>      VirtIORngPCI *vrng = VIRTIO_RNG_PCI(vpci_dev);
>      DeviceState *vdev = DEVICE(&vrng->vdev);
>  
> +    if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) {
> +        vpci_dev->nvectors = 2;
> +    }
> +
>      if (!qdev_realize(vdev, BUS(&vpci_dev->bus), errp)) {
>          return;
>      }
> @@ -54,6 +67,7 @@ static void virtio_rng_pci_class_init(ObjectClass *klass, void *data)
>      pcidev_k->device_id = PCI_DEVICE_ID_VIRTIO_RNG;
>      pcidev_k->revision = VIRTIO_PCI_ABI_VERSION;
>      pcidev_k->class_id = PCI_CLASS_OTHERS;
> +    device_class_set_props(dc, virtio_rng_properties);
>  }
>  
>  static void virtio_rng_initfn(Object *obj)
> -- 
> MST
> 
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK



^ permalink raw reply related	[flat|nested] 151+ messages in thread

* Re: [PULL v4 42/83] virtio-rng-pci: Allow setting nvectors, so we can use MSI-X
  2023-01-09 10:20   ` Dr. David Alan Gilbert
@ 2023-01-09 10:38     ` Michael S. Tsirkin
  2023-01-09 10:43       ` Dr. David Alan Gilbert
  0 siblings, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2023-01-09 10:38 UTC (permalink / raw)
  To: Dr. David Alan Gilbert
  Cc: qemu-devel, Peter Maydell, David Daney, Marcin Nowakowski,
	Philippe Mathieu-Daudé,
	Stefan Hajnoczi, quintela, peterx

On Mon, Jan 09, 2023 at 10:20:45AM +0000, Dr. David Alan Gilbert wrote:
> * Michael S. Tsirkin (mst@redhat.com) wrote:
> > From: David Daney <david.daney@fungible.com>
> > 
> > Most other virtio-pci devices allow MSI-X, let's have it for rng too.
> > 
> > Signed-off-by: David Daney <david.daney@fungible.com>
> > Reviewed-by: Marcin Nowakowski <marcin.nowakowski@fungible.com>
> > Signed-off-by: Philippe Mathieu-Daudé <philmd@fungible.com>
> > Message-Id: <20221014160947.66105-1-philmd@fungible.com>
> > Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
> > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> 
> This breaks migration compatibility 7.1->7.2 :
> 
> (qemu) qemu: get_pci_config_device: Bad config data: i=0x34 read: 84 device: 98 cmask: ff wmask: 0 w1cmask:0
> qemu: Failed to load PCIDevice:config
> qemu: Failed to load virtio-rng:virtio
> qemu: error while loading state for instance 0x0 of device '0000:00:03.0/virtio-rng'
> qemu: load of migration failed: Invalid argument
> 
> because the destination is configured with msi-x but the source isn't.
> 
> The fix is in theory simple:
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index f589b92909..45459d1cef 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -45,6 +45,7 @@ const size_t hw_compat_7_2_len = G_N_ELEMENTS(hw_compat_7_2);
>  
>  GlobalProperty hw_compat_7_1[] = {
>      { "virtio-device", "queue_reset", "false" },
> +    { "virtio-rng-pci", "vectors", "0" },
>  };
>  const size_t hw_compat_7_1_len = G_N_ELEMENTS(hw_compat_7_1);
> 
> the gotcha is that will break 7.2->7.2-fixed.
> 
> (I guess you can also work around it by explicitly passing vectors=0 to
> the virtio-rng on the cli)
> 
> Does anyone have preferences as to whether that should be fixed in the
> 7.2 world or left as is?
> 
> This is:
> https://bugzilla.redhat.com/show_bug.cgi?id=2155749
> 
> Dave

I think that yes, it should be fixed in 7.2.


> > ---
> >  hw/virtio/virtio-rng-pci.c | 14 ++++++++++++++
> >  1 file changed, 14 insertions(+)
> > 
> > diff --git a/hw/virtio/virtio-rng-pci.c b/hw/virtio/virtio-rng-pci.c
> > index 151ece6f94..6e76f8b57b 100644
> > --- a/hw/virtio/virtio-rng-pci.c
> > +++ b/hw/virtio/virtio-rng-pci.c
> > @@ -13,6 +13,7 @@
> >  
> >  #include "hw/virtio/virtio-pci.h"
> >  #include "hw/virtio/virtio-rng.h"
> > +#include "hw/qdev-properties.h"
> >  #include "qapi/error.h"
> >  #include "qemu/module.h"
> >  #include "qom/object.h"
> > @@ -31,11 +32,23 @@ struct VirtIORngPCI {
> >      VirtIORNG vdev;
> >  };
> >  
> > +static Property virtio_rng_properties[] = {
> > +    DEFINE_PROP_BIT("ioeventfd", VirtIOPCIProxy, flags,
> > +                    VIRTIO_PCI_FLAG_USE_IOEVENTFD_BIT, true),
> > +    DEFINE_PROP_UINT32("vectors", VirtIOPCIProxy, nvectors,
> > +                       DEV_NVECTORS_UNSPECIFIED),
> > +    DEFINE_PROP_END_OF_LIST(),
> > +};
> > +
> >  static void virtio_rng_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp)
> >  {
> >      VirtIORngPCI *vrng = VIRTIO_RNG_PCI(vpci_dev);
> >      DeviceState *vdev = DEVICE(&vrng->vdev);
> >  
> > +    if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) {
> > +        vpci_dev->nvectors = 2;
> > +    }
> > +
> >      if (!qdev_realize(vdev, BUS(&vpci_dev->bus), errp)) {
> >          return;
> >      }
> > @@ -54,6 +67,7 @@ static void virtio_rng_pci_class_init(ObjectClass *klass, void *data)
> >      pcidev_k->device_id = PCI_DEVICE_ID_VIRTIO_RNG;
> >      pcidev_k->revision = VIRTIO_PCI_ABI_VERSION;
> >      pcidev_k->class_id = PCI_CLASS_OTHERS;
> > +    device_class_set_props(dc, virtio_rng_properties);
> >  }
> >  
> >  static void virtio_rng_initfn(Object *obj)
> > -- 
> > MST
> > 
> > 
> -- 
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 42/83] virtio-rng-pci: Allow setting nvectors, so we can use MSI-X
  2023-01-09 10:38     ` Michael S. Tsirkin
@ 2023-01-09 10:43       ` Dr. David Alan Gilbert
  0 siblings, 0 replies; 151+ messages in thread
From: Dr. David Alan Gilbert @ 2023-01-09 10:43 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: qemu-devel, Peter Maydell, David Daney, Marcin Nowakowski,
	Philippe Mathieu-Daudé,
	Stefan Hajnoczi, quintela, peterx

* Michael S. Tsirkin (mst@redhat.com) wrote:
> On Mon, Jan 09, 2023 at 10:20:45AM +0000, Dr. David Alan Gilbert wrote:
> > * Michael S. Tsirkin (mst@redhat.com) wrote:
> > > From: David Daney <david.daney@fungible.com>
> > > 
> > > Most other virtio-pci devices allow MSI-X, let's have it for rng too.
> > > 
> > > Signed-off-by: David Daney <david.daney@fungible.com>
> > > Reviewed-by: Marcin Nowakowski <marcin.nowakowski@fungible.com>
> > > Signed-off-by: Philippe Mathieu-Daudé <philmd@fungible.com>
> > > Message-Id: <20221014160947.66105-1-philmd@fungible.com>
> > > Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
> > > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > 
> > This breaks migration compatibility 7.1->7.2 :
> > 
> > (qemu) qemu: get_pci_config_device: Bad config data: i=0x34 read: 84 device: 98 cmask: ff wmask: 0 w1cmask:0
> > qemu: Failed to load PCIDevice:config
> > qemu: Failed to load virtio-rng:virtio
> > qemu: error while loading state for instance 0x0 of device '0000:00:03.0/virtio-rng'
> > qemu: load of migration failed: Invalid argument
> > 
> > because the destination is configured with msi-x but the source isn't.
> > 
> > The fix is in theory simple:
> > diff --git a/hw/core/machine.c b/hw/core/machine.c
> > index f589b92909..45459d1cef 100644
> > --- a/hw/core/machine.c
> > +++ b/hw/core/machine.c
> > @@ -45,6 +45,7 @@ const size_t hw_compat_7_2_len = G_N_ELEMENTS(hw_compat_7_2);
> >  
> >  GlobalProperty hw_compat_7_1[] = {
> >      { "virtio-device", "queue_reset", "false" },
> > +    { "virtio-rng-pci", "vectors", "0" },
> >  };
> >  const size_t hw_compat_7_1_len = G_N_ELEMENTS(hw_compat_7_1);
> > 
> > the gotcha is that will break 7.2->7.2-fixed.
> > 
> > (I guess you can also work around it by explicitly passing vectors=0 to
> > the virtio-rng on the cli)
> > 
> > Does anyone have preferences as to whether that should be fixed in the
> > 7.2 world or left as is?
> > 
> > This is:
> > https://bugzilla.redhat.com/show_bug.cgi?id=2155749
> > 
> > Dave
> 
> I think that yes, it should be fixed in 7.2.

OK, I'll resend that as a patch in a mo.

Dave

> 
> > > ---
> > >  hw/virtio/virtio-rng-pci.c | 14 ++++++++++++++
> > >  1 file changed, 14 insertions(+)
> > > 
> > > diff --git a/hw/virtio/virtio-rng-pci.c b/hw/virtio/virtio-rng-pci.c
> > > index 151ece6f94..6e76f8b57b 100644
> > > --- a/hw/virtio/virtio-rng-pci.c
> > > +++ b/hw/virtio/virtio-rng-pci.c
> > > @@ -13,6 +13,7 @@
> > >  
> > >  #include "hw/virtio/virtio-pci.h"
> > >  #include "hw/virtio/virtio-rng.h"
> > > +#include "hw/qdev-properties.h"
> > >  #include "qapi/error.h"
> > >  #include "qemu/module.h"
> > >  #include "qom/object.h"
> > > @@ -31,11 +32,23 @@ struct VirtIORngPCI {
> > >      VirtIORNG vdev;
> > >  };
> > >  
> > > +static Property virtio_rng_properties[] = {
> > > +    DEFINE_PROP_BIT("ioeventfd", VirtIOPCIProxy, flags,
> > > +                    VIRTIO_PCI_FLAG_USE_IOEVENTFD_BIT, true),
> > > +    DEFINE_PROP_UINT32("vectors", VirtIOPCIProxy, nvectors,
> > > +                       DEV_NVECTORS_UNSPECIFIED),
> > > +    DEFINE_PROP_END_OF_LIST(),
> > > +};
> > > +
> > >  static void virtio_rng_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp)
> > >  {
> > >      VirtIORngPCI *vrng = VIRTIO_RNG_PCI(vpci_dev);
> > >      DeviceState *vdev = DEVICE(&vrng->vdev);
> > >  
> > > +    if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) {
> > > +        vpci_dev->nvectors = 2;
> > > +    }
> > > +
> > >      if (!qdev_realize(vdev, BUS(&vpci_dev->bus), errp)) {
> > >          return;
> > >      }
> > > @@ -54,6 +67,7 @@ static void virtio_rng_pci_class_init(ObjectClass *klass, void *data)
> > >      pcidev_k->device_id = PCI_DEVICE_ID_VIRTIO_RNG;
> > >      pcidev_k->revision = VIRTIO_PCI_ABI_VERSION;
> > >      pcidev_k->class_id = PCI_CLASS_OTHERS;
> > > +    device_class_set_props(dc, virtio_rng_properties);
> > >  }
> > >  
> > >  static void virtio_rng_initfn(Object *obj)
> > > -- 
> > > MST
> > > 
> > > 
> > -- 
> > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
  2023-01-06 14:21   ` Laurent Vivier
@ 2023-01-09 10:55     ` Michael S. Tsirkin
  2023-01-11  9:50       ` Laurent Vivier
  0 siblings, 1 reply; 151+ messages in thread
From: Michael S. Tsirkin @ 2023-01-09 10:55 UTC (permalink / raw)
  To: Laurent Vivier; +Cc: qemu-devel, Peter Maydell, Yajun Wu, Parav Pandit

On Fri, Jan 06, 2023 at 03:21:43PM +0100, Laurent Vivier wrote:
> Hi,
> 
> it seems this patch breaks vhost-user with DPDK.
> 
> See https://bugzilla.redhat.com/show_bug.cgi?id=2155173
> 
> it seems QEMU doesn't receive the expected commands sequence:
> 
> Received unexpected msg type. Expected 22 received 40
> Fail to update device iotlb
> Received unexpected msg type. Expected 40 received 22
> Received unexpected msg type. Expected 22 received 11
> Fail to update device iotlb
> Received unexpected msg type. Expected 11 received 22
> vhost VQ 1 ring restore failed: -71: Protocol error (71)
> Received unexpected msg type. Expected 22 received 11
> Fail to update device iotlb
> Received unexpected msg type. Expected 11 received 22
> vhost VQ 0 ring restore failed: -71: Protocol error (71)
> unable to start vhost net: 71: falling back on userspace virtio
> 
> It receives VHOST_USER_GET_STATUS (40) when it expects VHOST_USER_IOTLB_MSG (22)
> and VHOST_USER_IOTLB_MSG when it expects VHOST_USER_GET_STATUS.
> and VHOST_USER_GET_VRING_BASE (11) when it expect VHOST_USER_GET_STATUS and so on.
> 
> Any idea?
> 
> Thanks,
> Laurent


So I am guessing it's coming from:

    if (msg.hdr.request != request) {
        error_report("Received unexpected msg type. Expected %d received %d",
                     request, msg.hdr.request); 
        return -EPROTO;  
    }       

in process_message_reply and/or in vhost_user_get_u64.


> On 11/7/22 23:53, Michael S. Tsirkin wrote:
> > From: Yajun Wu <yajunw@nvidia.com>
> > 
> > The motivation of adding vhost-user vhost_dev_start support is to
> > improve backend configuration speed and reduce live migration VM
> > downtime.
> > 
> > Today VQ configuration is issued one by one. For virtio net with
> > multi-queue support, backend needs to update RSS (Receive side
> > scaling) on every rx queue enable. Updating RSS is time-consuming
> > (typical time like 7ms).
> > 
> > Implement already defined vhost status and message in the vhost
> > specification [1].
> > (a) VHOST_USER_PROTOCOL_F_STATUS
> > (b) VHOST_USER_SET_STATUS
> > (c) VHOST_USER_GET_STATUS
> > 
> > Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK for
> > device start and reset(0) for device stop.
> > 
> > On reception of the DRIVER_OK message, backend can apply the needed setting
> > only once (instead of incremental) and also utilize parallelism on enabling
> > queues.
> > 
> > This improves QEMU's live migration downtime with vhost user backend
> > implementation by great margin, specially for the large number of VQs of 64
> > from 800 msec to 250 msec.
> > 
> > [1] https://qemu-project.gitlab.io/qemu/interop/vhost-user.html
> > 
> > Signed-off-by: Yajun Wu <yajunw@nvidia.com>
> > Acked-by: Parav Pandit <parav@nvidia.com>
> > Message-Id: <20221017064452.1226514-3-yajunw@nvidia.com>
> > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

Probably easiest to debug from dpdk side.
Does the problem go away if you disable the feature VHOST_USER_PROTOCOL_F_STATUS in dpdk?

> > ---
> >   hw/virtio/vhost-user.c | 74 +++++++++++++++++++++++++++++++++++++++++-
> >   1 file changed, 73 insertions(+), 1 deletion(-)
> > 
> > diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> > index d256ce589b..abe23d4ebe 100644
> > --- a/hw/virtio/vhost-user.c
> > +++ b/hw/virtio/vhost-user.c
> > @@ -81,6 +81,7 @@ enum VhostUserProtocolFeature {
> >       VHOST_USER_PROTOCOL_F_RESET_DEVICE = 13,
> >       /* Feature 14 reserved for VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS. */
> >       VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS = 15,
> > +    VHOST_USER_PROTOCOL_F_STATUS = 16,
> >       VHOST_USER_PROTOCOL_F_MAX
> >   };
> > @@ -126,6 +127,8 @@ typedef enum VhostUserRequest {
> >       VHOST_USER_GET_MAX_MEM_SLOTS = 36,
> >       VHOST_USER_ADD_MEM_REG = 37,
> >       VHOST_USER_REM_MEM_REG = 38,
> > +    VHOST_USER_SET_STATUS = 39,
> > +    VHOST_USER_GET_STATUS = 40,
> >       VHOST_USER_MAX
> >   } VhostUserRequest;
> > @@ -1452,6 +1455,43 @@ static int vhost_user_set_u64(struct vhost_dev *dev, int request, uint64_t u64,
> >       return 0;
> >   }
> > +static int vhost_user_set_status(struct vhost_dev *dev, uint8_t status)
> > +{
> > +    return vhost_user_set_u64(dev, VHOST_USER_SET_STATUS, status, false);
> > +}
> > +
> > +static int vhost_user_get_status(struct vhost_dev *dev, uint8_t *status)
> > +{
> > +    uint64_t value;
> > +    int ret;
> > +
> > +    ret = vhost_user_get_u64(dev, VHOST_USER_GET_STATUS, &value);
> > +    if (ret < 0) {
> > +        return ret;
> > +    }
> > +    *status = value;
> > +
> > +    return 0;
> > +}
> > +
> > +static int vhost_user_add_status(struct vhost_dev *dev, uint8_t status)
> > +{
> > +    uint8_t s;
> > +    int ret;
> > +
> > +    ret = vhost_user_get_status(dev, &s);
> > +    if (ret < 0) {
> > +        return ret;
> > +    }
> > +
> > +    if ((s & status) == status) {
> > +        return 0;
> > +    }
> > +    s |= status;
> > +
> > +    return vhost_user_set_status(dev, s);
> > +}
> > +
> >   static int vhost_user_set_features(struct vhost_dev *dev,
> >                                      uint64_t features)
> >   {
> > @@ -1460,6 +1500,7 @@ static int vhost_user_set_features(struct vhost_dev *dev,
> >        * backend is actually logging changes
> >        */
> >       bool log_enabled = features & (0x1ULL << VHOST_F_LOG_ALL);
> > +    int ret;
> >       /*
> >        * We need to include any extra backend only feature bits that
> > @@ -1467,9 +1508,18 @@ static int vhost_user_set_features(struct vhost_dev *dev,
> >        * VHOST_USER_F_PROTOCOL_FEATURES bit for enabling protocol
> >        * features.
> >        */
> > -    return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES,
> > +    ret = vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES,
> >                                 features | dev->backend_features,
> >                                 log_enabled);
> > +
> > +    if (virtio_has_feature(dev->protocol_features,
> > +                           VHOST_USER_PROTOCOL_F_STATUS)) {
> > +        if (!ret) {
> > +            return vhost_user_add_status(dev, VIRTIO_CONFIG_S_FEATURES_OK);
> > +        }
> > +    }
> > +
> > +    return ret;
> >   }
> >   static int vhost_user_set_protocol_features(struct vhost_dev *dev,
> > @@ -2620,6 +2670,27 @@ void vhost_user_cleanup(VhostUserState *user)
> >       user->chr = NULL;
> >   }
> > +static int vhost_user_dev_start(struct vhost_dev *dev, bool started)
> > +{
> > +    if (!virtio_has_feature(dev->protocol_features,
> > +                            VHOST_USER_PROTOCOL_F_STATUS)) {
> > +        return 0;
> > +    }
> > +
> > +    /* Set device status only for last queue pair */
> > +    if (dev->vq_index + dev->nvqs != dev->vq_index_end) {
> > +        return 0;
> > +    }
> > +
> > +    if (started) {
> > +        return vhost_user_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE |
> > +                                          VIRTIO_CONFIG_S_DRIVER |
> > +                                          VIRTIO_CONFIG_S_DRIVER_OK);
> > +    } else {
> > +        return vhost_user_set_status(dev, 0);
> > +    }
> > +}
> > +
> >   const VhostOps user_ops = {
> >           .backend_type = VHOST_BACKEND_TYPE_USER,
> >           .vhost_backend_init = vhost_user_backend_init,
> > @@ -2654,4 +2725,5 @@ const VhostOps user_ops = {
> >           .vhost_backend_mem_section_filter = vhost_user_mem_section_filter,
> >           .vhost_get_inflight_fd = vhost_user_get_inflight_fd,
> >           .vhost_set_inflight_fd = vhost_user_set_inflight_fd,
> > +        .vhost_dev_start = vhost_user_dev_start,
> >   };



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
  2023-01-09 10:55     ` Michael S. Tsirkin
@ 2023-01-11  9:50       ` Laurent Vivier
  2023-01-12  5:46         ` Yajun Wu
  2023-01-12  9:25         ` Maxime Coquelin
  0 siblings, 2 replies; 151+ messages in thread
From: Laurent Vivier @ 2023-01-11  9:50 UTC (permalink / raw)
  To: Maxime Coquelin
  Cc: qemu-devel, Peter Maydell, Yajun Wu, Parav Pandit, Michael S. Tsirkin

On 1/9/23 11:55, Michael S. Tsirkin wrote:
> On Fri, Jan 06, 2023 at 03:21:43PM +0100, Laurent Vivier wrote:
>> Hi,
>>
>> it seems this patch breaks vhost-user with DPDK.
>>
>> See https://bugzilla.redhat.com/show_bug.cgi?id=2155173
>>
>> it seems QEMU doesn't receive the expected commands sequence:
>>
>> Received unexpected msg type. Expected 22 received 40
>> Fail to update device iotlb
>> Received unexpected msg type. Expected 40 received 22
>> Received unexpected msg type. Expected 22 received 11
>> Fail to update device iotlb
>> Received unexpected msg type. Expected 11 received 22
>> vhost VQ 1 ring restore failed: -71: Protocol error (71)
>> Received unexpected msg type. Expected 22 received 11
>> Fail to update device iotlb
>> Received unexpected msg type. Expected 11 received 22
>> vhost VQ 0 ring restore failed: -71: Protocol error (71)
>> unable to start vhost net: 71: falling back on userspace virtio
>>
>> It receives VHOST_USER_GET_STATUS (40) when it expects VHOST_USER_IOTLB_MSG (22)
>> and VHOST_USER_IOTLB_MSG when it expects VHOST_USER_GET_STATUS.
>> and VHOST_USER_GET_VRING_BASE (11) when it expect VHOST_USER_GET_STATUS and so on.
>>
>> Any idea?
>>
>> Thanks,
>> Laurent
> 
> 
> So I am guessing it's coming from:
> 
>      if (msg.hdr.request != request) {
>          error_report("Received unexpected msg type. Expected %d received %d",
>                       request, msg.hdr.request);
>          return -EPROTO;
>      }
> 
> in process_message_reply and/or in vhost_user_get_u64.
> 
> 
>> On 11/7/22 23:53, Michael S. Tsirkin wrote:
>>> From: Yajun Wu <yajunw@nvidia.com>
>>>
>>> The motivation of adding vhost-user vhost_dev_start support is to
>>> improve backend configuration speed and reduce live migration VM
>>> downtime.
>>>
>>> Today VQ configuration is issued one by one. For virtio net with
>>> multi-queue support, backend needs to update RSS (Receive side
>>> scaling) on every rx queue enable. Updating RSS is time-consuming
>>> (typical time like 7ms).
>>>
>>> Implement already defined vhost status and message in the vhost
>>> specification [1].
>>> (a) VHOST_USER_PROTOCOL_F_STATUS
>>> (b) VHOST_USER_SET_STATUS
>>> (c) VHOST_USER_GET_STATUS
>>>
>>> Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK for
>>> device start and reset(0) for device stop.
>>>
>>> On reception of the DRIVER_OK message, backend can apply the needed setting
>>> only once (instead of incremental) and also utilize parallelism on enabling
>>> queues.
>>>
>>> This improves QEMU's live migration downtime with vhost user backend
>>> implementation by great margin, specially for the large number of VQs of 64
>>> from 800 msec to 250 msec.
>>>
>>> [1] https://qemu-project.gitlab.io/qemu/interop/vhost-user.html
>>>
>>> Signed-off-by: Yajun Wu <yajunw@nvidia.com>
>>> Acked-by: Parav Pandit <parav@nvidia.com>
>>> Message-Id: <20221017064452.1226514-3-yajunw@nvidia.com>
>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> 
> Probably easiest to debug from dpdk side.
> Does the problem go away if you disable the feature VHOST_USER_PROTOCOL_F_STATUS in dpdk?

Maxime could you help to debug this?

Thanks,
Laurent



^ permalink raw reply	[flat|nested] 151+ messages in thread

* RE: [PULL v4 76/83] vhost-user: Support vhost_dev_start
  2023-01-11  9:50       ` Laurent Vivier
@ 2023-01-12  5:46         ` Yajun Wu
  2023-01-12  9:25         ` Maxime Coquelin
  1 sibling, 0 replies; 151+ messages in thread
From: Yajun Wu @ 2023-01-12  5:46 UTC (permalink / raw)
  To: Laurent Vivier, Maxime Coquelin
  Cc: qemu-devel, Peter Maydell, Parav Pandit, Michael S. Tsirkin

Hi,

VHOST_USER_PROTOCOL_F_STATUS is enabled by default (dpdk):

lib/vhost/vhost_user.h

17 #define VHOST_USER_PROTOCOL_FEATURES    ((1ULL << VHOST_USER_PROTOCOL_F_MQ) | \
 18                      (1ULL << VHOST_USER_PROTOCOL_F_LOG_SHMFD) |\
 19                      (1ULL << VHOST_USER_PROTOCOL_F_RARP) | \
 20                      (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK) | \
 21                      (1ULL << VHOST_USER_PROTOCOL_F_NET_MTU) | \
 22                      (1ULL << VHOST_USER_PROTOCOL_F_SLAVE_REQ) | \
 23                      (1ULL << VHOST_USER_PROTOCOL_F_CRYPTO_SESSION) | \
 24                      (1ULL << VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD) | \
 25                      (1ULL << VHOST_USER_PROTOCOL_F_HOST_NOTIFIER) | \
 26                      (1ULL << VHOST_USER_PROTOCOL_F_PAGEFAULT) | \
 27                      (1ULL << VHOST_USER_PROTOCOL_F_STATUS))

Remove VHOST_USER_PROTOCOL_F_STATUS can disable VHOST_USER_SET/GET_STATUS message.
Should W.A. this issue.

Thanks,
Yajun

-----Original Message-----
From: Laurent Vivier <lvivier@redhat.com> 
Sent: Wednesday, January 11, 2023 5:50 PM
To: Maxime Coquelin <maxime.coquelin@redhat.com>
Cc: qemu-devel@nongnu.org; Peter Maydell <peter.maydell@linaro.org>; Yajun Wu <yajunw@nvidia.com>; Parav Pandit <parav@nvidia.com>; Michael S. Tsirkin <mst@redhat.com>
Subject: Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start

External email: Use caution opening links or attachments


On 1/9/23 11:55, Michael S. Tsirkin wrote:
> On Fri, Jan 06, 2023 at 03:21:43PM +0100, Laurent Vivier wrote:
>> Hi,
>>
>> it seems this patch breaks vhost-user with DPDK.
>>
>> See 
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbug
>> zilla.redhat.com%2Fshow_bug.cgi%3Fid%3D2155173&data=05%7C01%7Cyajunw%
>> 40nvidia.com%7Cf4c581251ab548d64ae708daf3b94867%7C43083d15727340c1b7d
>> b39efd9ccc17a%7C0%7C0%7C638090274351645141%7CUnknown%7CTWFpbGZsb3d8ey
>> JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C30
>> 00%7C%7C%7C&sdata=m582YO4Sd2jJ0S%2F%2FSv9zx6NSuXQIrRwkqBPgYedO%2Fr8%3
>> D&reserved=0
>>
>> it seems QEMU doesn't receive the expected commands sequence:
>>
>> Received unexpected msg type. Expected 22 received 40 Fail to update 
>> device iotlb Received unexpected msg type. Expected 40 received 22 
>> Received unexpected msg type. Expected 22 received 11 Fail to update 
>> device iotlb Received unexpected msg type. Expected 11 received 22 
>> vhost VQ 1 ring restore failed: -71: Protocol error (71) Received 
>> unexpected msg type. Expected 22 received 11 Fail to update device 
>> iotlb Received unexpected msg type. Expected 11 received 22 vhost VQ 
>> 0 ring restore failed: -71: Protocol error (71) unable to start vhost 
>> net: 71: falling back on userspace virtio
>>
>> It receives VHOST_USER_GET_STATUS (40) when it expects 
>> VHOST_USER_IOTLB_MSG (22) and VHOST_USER_IOTLB_MSG when it expects VHOST_USER_GET_STATUS.
>> and VHOST_USER_GET_VRING_BASE (11) when it expect VHOST_USER_GET_STATUS and so on.
>>
>> Any idea?
>>
>> Thanks,
>> Laurent
>
>
> So I am guessing it's coming from:
>
>      if (msg.hdr.request != request) {
>          error_report("Received unexpected msg type. Expected %d received %d",
>                       request, msg.hdr.request);
>          return -EPROTO;
>      }
>
> in process_message_reply and/or in vhost_user_get_u64.
>
>
>> On 11/7/22 23:53, Michael S. Tsirkin wrote:
>>> From: Yajun Wu <yajunw@nvidia.com>
>>>
>>> The motivation of adding vhost-user vhost_dev_start support is to 
>>> improve backend configuration speed and reduce live migration VM 
>>> downtime.
>>>
>>> Today VQ configuration is issued one by one. For virtio net with 
>>> multi-queue support, backend needs to update RSS (Receive side
>>> scaling) on every rx queue enable. Updating RSS is time-consuming 
>>> (typical time like 7ms).
>>>
>>> Implement already defined vhost status and message in the vhost 
>>> specification [1].
>>> (a) VHOST_USER_PROTOCOL_F_STATUS
>>> (b) VHOST_USER_SET_STATUS
>>> (c) VHOST_USER_GET_STATUS
>>>
>>> Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK 
>>> for device start and reset(0) for device stop.
>>>
>>> On reception of the DRIVER_OK message, backend can apply the needed 
>>> setting only once (instead of incremental) and also utilize 
>>> parallelism on enabling queues.
>>>
>>> This improves QEMU's live migration downtime with vhost user backend 
>>> implementation by great margin, specially for the large number of 
>>> VQs of 64 from 800 msec to 250 msec.
>>>
>>> [1] 
>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fqe
>>> mu-project.gitlab.io%2Fqemu%2Finterop%2Fvhost-user.html&data=05%7C01
>>> %7Cyajunw%40nvidia.com%7Cf4c581251ab548d64ae708daf3b94867%7C43083d15
>>> 727340c1b7db39efd9ccc17a%7C0%7C0%7C638090274351645141%7CUnknown%7CTW
>>> FpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVC
>>> I6Mn0%3D%7C3000%7C%7C%7C&sdata=eEmHPgZlmImC5LTDZ2jTJauNW7cRFDhsme8%2
>>> Fjk7ywIE%3D&reserved=0
>>>
>>> Signed-off-by: Yajun Wu <yajunw@nvidia.com>
>>> Acked-by: Parav Pandit <parav@nvidia.com>
>>> Message-Id: <20221017064452.1226514-3-yajunw@nvidia.com>
>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>
> Probably easiest to debug from dpdk side.
> Does the problem go away if you disable the feature VHOST_USER_PROTOCOL_F_STATUS in dpdk?

Maxime could you help to debug this?

Thanks,
Laurent



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
  2023-01-11  9:50       ` Laurent Vivier
  2023-01-12  5:46         ` Yajun Wu
@ 2023-01-12  9:25         ` Maxime Coquelin
  2023-01-16  7:14           ` Yajun Wu
  1 sibling, 1 reply; 151+ messages in thread
From: Maxime Coquelin @ 2023-01-12  9:25 UTC (permalink / raw)
  To: Laurent Vivier
  Cc: qemu-devel, Peter Maydell, Yajun Wu, Parav Pandit, Michael S. Tsirkin

Hi Laurent,

On 1/11/23 10:50, Laurent Vivier wrote:
> On 1/9/23 11:55, Michael S. Tsirkin wrote:
>> On Fri, Jan 06, 2023 at 03:21:43PM +0100, Laurent Vivier wrote:
>>> Hi,
>>>
>>> it seems this patch breaks vhost-user with DPDK.
>>>
>>> See https://bugzilla.redhat.com/show_bug.cgi?id=2155173
>>>
>>> it seems QEMU doesn't receive the expected commands sequence:
>>>
>>> Received unexpected msg type. Expected 22 received 40
>>> Fail to update device iotlb
>>> Received unexpected msg type. Expected 40 received 22
>>> Received unexpected msg type. Expected 22 received 11
>>> Fail to update device iotlb
>>> Received unexpected msg type. Expected 11 received 22
>>> vhost VQ 1 ring restore failed: -71: Protocol error (71)
>>> Received unexpected msg type. Expected 22 received 11
>>> Fail to update device iotlb
>>> Received unexpected msg type. Expected 11 received 22
>>> vhost VQ 0 ring restore failed: -71: Protocol error (71)
>>> unable to start vhost net: 71: falling back on userspace virtio
>>>
>>> It receives VHOST_USER_GET_STATUS (40) when it expects 
>>> VHOST_USER_IOTLB_MSG (22)
>>> and VHOST_USER_IOTLB_MSG when it expects VHOST_USER_GET_STATUS.
>>> and VHOST_USER_GET_VRING_BASE (11) when it expect 
>>> VHOST_USER_GET_STATUS and so on.
>>>
>>> Any idea?

We only have a single thread on DPDK side to handle Vhost-user requests,
it will read a request, handle it and reply to it. Then it reads the
next one, etc... So I don't think it is possible to mix request replies
order on DPDK side.

Maybe there are two threads concurrently sending requests on QEMU side?

Regards,
Maxime

>>> Thanks,
>>> Laurent
>>
>>
>> So I am guessing it's coming from:
>>
>>      if (msg.hdr.request != request) {
>>          error_report("Received unexpected msg type. Expected %d 
>> received %d",
>>                       request, msg.hdr.request);
>>          return -EPROTO;
>>      }
>>
>> in process_message_reply and/or in vhost_user_get_u64.
>>
>>
>>> On 11/7/22 23:53, Michael S. Tsirkin wrote:
>>>> From: Yajun Wu <yajunw@nvidia.com>
>>>>
>>>> The motivation of adding vhost-user vhost_dev_start support is to
>>>> improve backend configuration speed and reduce live migration VM
>>>> downtime.
>>>>
>>>> Today VQ configuration is issued one by one. For virtio net with
>>>> multi-queue support, backend needs to update RSS (Receive side
>>>> scaling) on every rx queue enable. Updating RSS is time-consuming
>>>> (typical time like 7ms).
>>>>
>>>> Implement already defined vhost status and message in the vhost
>>>> specification [1].
>>>> (a) VHOST_USER_PROTOCOL_F_STATUS
>>>> (b) VHOST_USER_SET_STATUS
>>>> (c) VHOST_USER_GET_STATUS
>>>>
>>>> Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK for
>>>> device start and reset(0) for device stop.
>>>>
>>>> On reception of the DRIVER_OK message, backend can apply the needed 
>>>> setting
>>>> only once (instead of incremental) and also utilize parallelism on 
>>>> enabling
>>>> queues.
>>>>
>>>> This improves QEMU's live migration downtime with vhost user backend
>>>> implementation by great margin, specially for the large number of 
>>>> VQs of 64
>>>> from 800 msec to 250 msec.
>>>>
>>>> [1] https://qemu-project.gitlab.io/qemu/interop/vhost-user.html
>>>>
>>>> Signed-off-by: Yajun Wu <yajunw@nvidia.com>
>>>> Acked-by: Parav Pandit <parav@nvidia.com>
>>>> Message-Id: <20221017064452.1226514-3-yajunw@nvidia.com>
>>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
>>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>>
>> Probably easiest to debug from dpdk side.
>> Does the problem go away if you disable the feature 
>> VHOST_USER_PROTOCOL_F_STATUS in dpdk?
> 
> Maxime could you help to debug this?
> 
> Thanks,
> Laurent
> 



^ permalink raw reply	[flat|nested] 151+ messages in thread

* RE: [PULL v4 76/83] vhost-user: Support vhost_dev_start
  2023-01-12  9:25         ` Maxime Coquelin
@ 2023-01-16  7:14           ` Yajun Wu
  2023-01-17  9:49             ` Maxime Coquelin
  0 siblings, 1 reply; 151+ messages in thread
From: Yajun Wu @ 2023-01-16  7:14 UTC (permalink / raw)
  To: Maxime Coquelin, Laurent Vivier
  Cc: qemu-devel, Peter Maydell, Parav Pandit, Michael S. Tsirkin

Not quite sure about the whole picture.

Seems while qemu waiting response of vhost_user_get_status, dpdk send out VHOST_USER_SLAVE_IOTLB_MSG and trigger qemu function vhost_backend_update_device_iotlb.
Qemu wait on reply of VHOST_USER_IOTLB_MSG but get VHOST_USER_GET_STATUS reply.

Break on first error message("Received unexpected msg type. Expected 22 received 40")

#0  0x0000555555b72ed4 in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffa330) at ../hw/virtio/vhost-user.c:445
#1  0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffa600) at ../hw/virtio/vhost-user.c:2341
#2  0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10442706944, uaddr=140736119902208, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
#3  0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10442706944, write=1) at ../hw/virtio/vhost.c:1113
#4  0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffa7b0) at ../hw/virtio/vhost-backend.c:393
#5  0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
#6  0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556a06fb0, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
#7  0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
#8  0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
#9  0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
#10 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffac50) at ../hw/virtio/vhost-user.c:413
#11 0x0000555555b72e9b in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffaf10) at ../hw/virtio/vhost-user.c:439
#12 0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffb1e0) at ../hw/virtio/vhost-user.c:2341
#13 0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10468392960, uaddr=140736145588224, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
#14 0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10468392960, write=1) at ../hw/virtio/vhost.c:1113
#15 0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffb390) at ../hw/virtio/vhost-backend.c:393
#16 0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
#17 0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556c70250, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
#18 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
#19 0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
#20 0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
#21 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffb830) at ../hw/virtio/vhost-user.c:413
#22 0x0000555555b72e9b in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffbaf0) at ../hw/virtio/vhost-user.c:439
#23 0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffbdc0) at ../hw/virtio/vhost-user.c:2341
#24 0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10442702848, uaddr=140736119898112, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
#25 0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10442702848, write=1) at ../hw/virtio/vhost.c:1113
#26 0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffbf70) at ../hw/virtio/vhost-backend.c:393
#27 0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
#28 0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556f1a530, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
#29 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
#30 0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
#31 0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
#32 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffc420) at ../hw/virtio/vhost-user.c:413
#33 0x0000555555b754b1 in vhost_user_get_u64 (dev=0x5555584dd600, request=40, u64=0x7fffffffc6e0) at ../hw/virtio/vhost-user.c:1349
#34 0x0000555555b758ff in vhost_user_get_status (dev=0x5555584dd600, status=0x7fffffffc713 "W\020") at ../hw/virtio/vhost-user.c:1474
#35 0x0000555555b75967 in vhost_user_add_status (dev=0x5555584dd600, status=7 '\a') at ../hw/virtio/vhost-user.c:1488
#36 0x0000555555b78bf6 in vhost_user_dev_start (dev=0x5555584dd600, started=true) at ../hw/virtio/vhost-user.c:2758
#37 0x0000555555b709ad in vhost_dev_start (hdev=0x5555584dd600, vdev=0x555557b965d0, vrings=false) at ../hw/virtio/vhost.c:1988
#38 0x000055555584291c in vhost_net_start_one (net=0x5555584dd600, dev=0x555557b965d0) at ../hw/net/vhost_net.c:271
#39 0x0000555555842f1e in vhost_net_start (dev=0x555557b965d0, ncs=0x555557bc09e0, data_queue_pairs=1, cvq=0) at ../hw/net/vhost_net.c:412
#40 0x0000555555b1bf61 in virtio_net_vhost_status (n=0x555557b965d0, status=15 '\017') at ../hw/net/virtio-net.c:311
#41 0x0000555555b1c20c in virtio_net_set_status (vdev=0x555557b965d0, status=15 '\017') at ../hw/net/virtio-net.c:392
#42 0x0000555555b1ed04 in virtio_net_handle_mq (n=0x555557b965d0, cmd=0 '\000', iov=0x555556c7ef50, iov_cnt=1) at ../hw/net/virtio-net.c:1497
#43 0x0000555555b1eef0 in virtio_net_handle_ctrl_iov (vdev=0x555557b965d0, in_sg=0x555556a09880, in_num=1, out_sg=0x555556a09890, out_num=1) at ../hw/net/virtio-net.c:1534
#44 0x0000555555b1efe9 in virtio_net_handle_ctrl (vdev=0x555557b965d0, vq=0x7fffc04ac140) at ../hw/net/virtio-net.c:1557
#45 0x0000555555b63776 in virtio_queue_notify_vq (vq=0x7fffc04ac140) at ../hw/virtio/virtio.c:2249
#46 0x0000555555b669dc in virtio_queue_host_notifier_read (n=0x7fffc04ac1b4) at ../hw/virtio/virtio.c:3529
#47 0x0000555555e3f458 in aio_dispatch_handler (ctx=0x555556a016c0, node=0x7ffd8800e430) at ../util/aio-posix.c:369
#48 0x0000555555e3f613 in aio_dispatch_handlers (ctx=0x555556a016c0) at ../util/aio-posix.c:412
#49 0x0000555555e3f669 in aio_dispatch (ctx=0x555556a016c0) at ../util/aio-posix.c:422
#50 0x0000555555e585de in aio_ctx_dispatch (source=0x555556a016c0, callback=0x0, user_data=0x0) at ../util/async.c:321
#51 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
#52 0x0000555555e5abea in glib_pollfds_poll () at ../util/main-loop.c:295
#53 0x0000555555e5ac64 in os_host_main_loop_wait (timeout=0) at ../util/main-loop.c:318
#54 0x0000555555e5ad69 in main_loop_wait (nonblocking=0) at ../util/main-loop.c:604
#55 0x00005555559693de in qemu_main_loop () at ../softmmu/runstate.c:731
#56 0x00005555556e7c06 in qemu_default_main () at ../softmmu/main.c:37
#57 0x00005555556e7c3c in main (argc=71, argv=0x7fffffffcda8) at ../softmmu/main.c:48



-----Original Message-----
From: Maxime Coquelin <maxime.coquelin@redhat.com> 
Sent: Thursday, January 12, 2023 5:26 PM
To: Laurent Vivier <lvivier@redhat.com>
Cc: qemu-devel@nongnu.org; Peter Maydell <peter.maydell@linaro.org>; Yajun Wu <yajunw@nvidia.com>; Parav Pandit <parav@nvidia.com>; Michael S. Tsirkin <mst@redhat.com>
Subject: Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start

External email: Use caution opening links or attachments


Hi Laurent,

On 1/11/23 10:50, Laurent Vivier wrote:
> On 1/9/23 11:55, Michael S. Tsirkin wrote:
>> On Fri, Jan 06, 2023 at 03:21:43PM +0100, Laurent Vivier wrote:
>>> Hi,
>>>
>>> it seems this patch breaks vhost-user with DPDK.
>>>
>>> See 
>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbu
>>> gzilla.redhat.com%2Fshow_bug.cgi%3Fid%3D2155173&data=05%7C01%7Cyajun
>>> w%40nvidia.com%7C47e6e0fabd044383fd3308daf47f0253%7C43083d15727340c1
>>> b7db39efd9ccc17a%7C0%7C0%7C638091123577559319%7CUnknown%7CTWFpbGZsb3
>>> d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
>>> %7C3000%7C%7C%7C&sdata=1pjChYTKHVmBoempNitiZHBdrlPIMFjKoD6FeOVSay0%3
>>> D&reserved=0
>>>
>>> it seems QEMU doesn't receive the expected commands sequence:
>>>
>>> Received unexpected msg type. Expected 22 received 40 Fail to update 
>>> device iotlb Received unexpected msg type. Expected 40 received 22 
>>> Received unexpected msg type. Expected 22 received 11 Fail to update 
>>> device iotlb Received unexpected msg type. Expected 11 received 22 
>>> vhost VQ 1 ring restore failed: -71: Protocol error (71) Received 
>>> unexpected msg type. Expected 22 received 11 Fail to update device 
>>> iotlb Received unexpected msg type. Expected 11 received 22 vhost VQ 
>>> 0 ring restore failed: -71: Protocol error (71) unable to start 
>>> vhost net: 71: falling back on userspace virtio
>>>
>>> It receives VHOST_USER_GET_STATUS (40) when it expects 
>>> VHOST_USER_IOTLB_MSG (22) and VHOST_USER_IOTLB_MSG when it expects 
>>> VHOST_USER_GET_STATUS.
>>> and VHOST_USER_GET_VRING_BASE (11) when it expect 
>>> VHOST_USER_GET_STATUS and so on.
>>>
>>> Any idea?

We only have a single thread on DPDK side to handle Vhost-user requests, it will read a request, handle it and reply to it. Then it reads the next one, etc... So I don't think it is possible to mix request replies order on DPDK side.

Maybe there are two threads concurrently sending requests on QEMU side?

Regards,
Maxime

>>> Thanks,
>>> Laurent
>>
>>
>> So I am guessing it's coming from:
>>
>>      if (msg.hdr.request != request) {
>>          error_report("Received unexpected msg type. Expected %d 
>> received %d",
>>                       request, msg.hdr.request);
>>          return -EPROTO;
>>      }
>>
>> in process_message_reply and/or in vhost_user_get_u64.
>>
>>
>>> On 11/7/22 23:53, Michael S. Tsirkin wrote:
>>>> From: Yajun Wu <yajunw@nvidia.com>
>>>>
>>>> The motivation of adding vhost-user vhost_dev_start support is to 
>>>> improve backend configuration speed and reduce live migration VM 
>>>> downtime.
>>>>
>>>> Today VQ configuration is issued one by one. For virtio net with 
>>>> multi-queue support, backend needs to update RSS (Receive side
>>>> scaling) on every rx queue enable. Updating RSS is time-consuming 
>>>> (typical time like 7ms).
>>>>
>>>> Implement already defined vhost status and message in the vhost 
>>>> specification [1].
>>>> (a) VHOST_USER_PROTOCOL_F_STATUS
>>>> (b) VHOST_USER_SET_STATUS
>>>> (c) VHOST_USER_GET_STATUS
>>>>
>>>> Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK 
>>>> for device start and reset(0) for device stop.
>>>>
>>>> On reception of the DRIVER_OK message, backend can apply the needed 
>>>> setting only once (instead of incremental) and also utilize 
>>>> parallelism on enabling queues.
>>>>
>>>> This improves QEMU's live migration downtime with vhost user 
>>>> backend implementation by great margin, specially for the large 
>>>> number of VQs of 64 from 800 msec to 250 msec.
>>>>
>>>> [1] 
>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fq
>>>> emu-project.gitlab.io%2Fqemu%2Finterop%2Fvhost-user.html&data=05%7C
>>>> 01%7Cyajunw%40nvidia.com%7C47e6e0fabd044383fd3308daf47f0253%7C43083
>>>> d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638091123577559319%7CUnknown%
>>>> 7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
>>>> CJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YPbrFRJA92KLLwADMUDvwBt%2Fme2Ef
>>>> GZuVANOmXH5pic%3D&reserved=0
>>>>
>>>> Signed-off-by: Yajun Wu <yajunw@nvidia.com>
>>>> Acked-by: Parav Pandit <parav@nvidia.com>
>>>> Message-Id: <20221017064452.1226514-3-yajunw@nvidia.com>
>>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
>>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>>
>> Probably easiest to debug from dpdk side.
>> Does the problem go away if you disable the feature 
>> VHOST_USER_PROTOCOL_F_STATUS in dpdk?
>
> Maxime could you help to debug this?
>
> Thanks,
> Laurent
>



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
  2023-01-16  7:14           ` Yajun Wu
@ 2023-01-17  9:49             ` Maxime Coquelin
  2023-01-17 12:12               ` Greg Kurz
  0 siblings, 1 reply; 151+ messages in thread
From: Maxime Coquelin @ 2023-01-17  9:49 UTC (permalink / raw)
  To: Yajun Wu, Laurent Vivier, groug, Stefan Hajnoczi
  Cc: qemu-devel, Peter Maydell, Parav Pandit, Michael S. Tsirkin

Hi Yajun,

On 1/16/23 08:14, Yajun Wu wrote:
> Not quite sure about the whole picture.
> 
> Seems while qemu waiting response of vhost_user_get_status, dpdk send out VHOST_USER_SLAVE_IOTLB_MSG and trigger qemu function vhost_backend_update_device_iotlb.
> Qemu wait on reply of VHOST_USER_IOTLB_MSG but get VHOST_USER_GET_STATUS reply.

Thanks for the backtrace, that helps a lot.

The issue happens because:
  1. Introduction of nested event loop in vhost_user_read() [0] features
that enables handling slave channel request while waiting for reply on
the masyer channel.
  2. Slave VHOST_USER_SLAVE_IOTLB_MSG slave request handling ends-up 
sending a VHOST_USER_IOTLB_MSG on the master channel.

So while waiting for VHOST_USER_IOTLB_MSG reply, it receives the reply
for the first request sent on the master channel, here the
VHOST_USER_GET_STATUS reply.

I don't see an easy way to fix it.

One option would be to have the slave channel being handled by another
thread, and protect master channel with a lock to enforce the
synchronization. But this may induce other issues, so that's not a light
change.

(Adding Greg and Stefan, who worked on the nested event loop series.)

Simply reverting nested event loop support may not be an option, since
it would break virtiofsd, as if my understanding is correct, waits for
some slave channel request to complete in order to complete a request
made by QEMU on the master channel.

Any thougths?

Maxime

[0]: 
https://patchwork.ozlabs.org/project/qemu-devel/patch/20210312092212.782255-6-groug@kaod.org/


> Break on first error message("Received unexpected msg type. Expected 22 received 40")
> 
> #0  0x0000555555b72ed4 in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffa330) at ../hw/virtio/vhost-user.c:445
> #1  0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffa600) at ../hw/virtio/vhost-user.c:2341
> #2  0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10442706944, uaddr=140736119902208, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> #3  0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10442706944, write=1) at ../hw/virtio/vhost.c:1113
> #4  0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffa7b0) at ../hw/virtio/vhost-backend.c:393
> #5  0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> #6  0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556a06fb0, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> #7  0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> #8  0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> #9  0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> #10 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffac50) at ../hw/virtio/vhost-user.c:413
> #11 0x0000555555b72e9b in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffaf10) at ../hw/virtio/vhost-user.c:439
> #12 0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffb1e0) at ../hw/virtio/vhost-user.c:2341
> #13 0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10468392960, uaddr=140736145588224, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> #14 0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10468392960, write=1) at ../hw/virtio/vhost.c:1113
> #15 0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffb390) at ../hw/virtio/vhost-backend.c:393
> #16 0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> #17 0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556c70250, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> #18 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> #19 0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> #20 0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> #21 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffb830) at ../hw/virtio/vhost-user.c:413
> #22 0x0000555555b72e9b in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffbaf0) at ../hw/virtio/vhost-user.c:439
> #23 0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffbdc0) at ../hw/virtio/vhost-user.c:2341
> #24 0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10442702848, uaddr=140736119898112, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> #25 0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10442702848, write=1) at ../hw/virtio/vhost.c:1113
> #26 0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffbf70) at ../hw/virtio/vhost-backend.c:393
> #27 0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> #28 0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556f1a530, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> #29 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> #30 0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> #31 0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> #32 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffc420) at ../hw/virtio/vhost-user.c:413
> #33 0x0000555555b754b1 in vhost_user_get_u64 (dev=0x5555584dd600, request=40, u64=0x7fffffffc6e0) at ../hw/virtio/vhost-user.c:1349
> #34 0x0000555555b758ff in vhost_user_get_status (dev=0x5555584dd600, status=0x7fffffffc713 "W\020") at ../hw/virtio/vhost-user.c:1474
> #35 0x0000555555b75967 in vhost_user_add_status (dev=0x5555584dd600, status=7 '\a') at ../hw/virtio/vhost-user.c:1488
> #36 0x0000555555b78bf6 in vhost_user_dev_start (dev=0x5555584dd600, started=true) at ../hw/virtio/vhost-user.c:2758
> #37 0x0000555555b709ad in vhost_dev_start (hdev=0x5555584dd600, vdev=0x555557b965d0, vrings=false) at ../hw/virtio/vhost.c:1988
> #38 0x000055555584291c in vhost_net_start_one (net=0x5555584dd600, dev=0x555557b965d0) at ../hw/net/vhost_net.c:271
> #39 0x0000555555842f1e in vhost_net_start (dev=0x555557b965d0, ncs=0x555557bc09e0, data_queue_pairs=1, cvq=0) at ../hw/net/vhost_net.c:412
> #40 0x0000555555b1bf61 in virtio_net_vhost_status (n=0x555557b965d0, status=15 '\017') at ../hw/net/virtio-net.c:311
> #41 0x0000555555b1c20c in virtio_net_set_status (vdev=0x555557b965d0, status=15 '\017') at ../hw/net/virtio-net.c:392
> #42 0x0000555555b1ed04 in virtio_net_handle_mq (n=0x555557b965d0, cmd=0 '\000', iov=0x555556c7ef50, iov_cnt=1) at ../hw/net/virtio-net.c:1497
> #43 0x0000555555b1eef0 in virtio_net_handle_ctrl_iov (vdev=0x555557b965d0, in_sg=0x555556a09880, in_num=1, out_sg=0x555556a09890, out_num=1) at ../hw/net/virtio-net.c:1534
> #44 0x0000555555b1efe9 in virtio_net_handle_ctrl (vdev=0x555557b965d0, vq=0x7fffc04ac140) at ../hw/net/virtio-net.c:1557
> #45 0x0000555555b63776 in virtio_queue_notify_vq (vq=0x7fffc04ac140) at ../hw/virtio/virtio.c:2249
> #46 0x0000555555b669dc in virtio_queue_host_notifier_read (n=0x7fffc04ac1b4) at ../hw/virtio/virtio.c:3529
> #47 0x0000555555e3f458 in aio_dispatch_handler (ctx=0x555556a016c0, node=0x7ffd8800e430) at ../util/aio-posix.c:369
> #48 0x0000555555e3f613 in aio_dispatch_handlers (ctx=0x555556a016c0) at ../util/aio-posix.c:412
> #49 0x0000555555e3f669 in aio_dispatch (ctx=0x555556a016c0) at ../util/aio-posix.c:422
> #50 0x0000555555e585de in aio_ctx_dispatch (source=0x555556a016c0, callback=0x0, user_data=0x0) at ../util/async.c:321
> #51 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> #52 0x0000555555e5abea in glib_pollfds_poll () at ../util/main-loop.c:295
> #53 0x0000555555e5ac64 in os_host_main_loop_wait (timeout=0) at ../util/main-loop.c:318
> #54 0x0000555555e5ad69 in main_loop_wait (nonblocking=0) at ../util/main-loop.c:604
> #55 0x00005555559693de in qemu_main_loop () at ../softmmu/runstate.c:731
> #56 0x00005555556e7c06 in qemu_default_main () at ../softmmu/main.c:37
> #57 0x00005555556e7c3c in main (argc=71, argv=0x7fffffffcda8) at ../softmmu/main.c:48
> 
> 
> 
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Thursday, January 12, 2023 5:26 PM
> To: Laurent Vivier <lvivier@redhat.com>
> Cc: qemu-devel@nongnu.org; Peter Maydell <peter.maydell@linaro.org>; Yajun Wu <yajunw@nvidia.com>; Parav Pandit <parav@nvidia.com>; Michael S. Tsirkin <mst@redhat.com>
> Subject: Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
> 
> External email: Use caution opening links or attachments
> 
> 
> Hi Laurent,
> 
> On 1/11/23 10:50, Laurent Vivier wrote:
>> On 1/9/23 11:55, Michael S. Tsirkin wrote:
>>> On Fri, Jan 06, 2023 at 03:21:43PM +0100, Laurent Vivier wrote:
>>>> Hi,
>>>>
>>>> it seems this patch breaks vhost-user with DPDK.
>>>>
>>>> See
>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbu
>>>> gzilla.redhat.com%2Fshow_bug.cgi%3Fid%3D2155173&data=05%7C01%7Cyajun
>>>> w%40nvidia.com%7C47e6e0fabd044383fd3308daf47f0253%7C43083d15727340c1
>>>> b7db39efd9ccc17a%7C0%7C0%7C638091123577559319%7CUnknown%7CTWFpbGZsb3
>>>> d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
>>>> %7C3000%7C%7C%7C&sdata=1pjChYTKHVmBoempNitiZHBdrlPIMFjKoD6FeOVSay0%3
>>>> D&reserved=0
>>>>
>>>> it seems QEMU doesn't receive the expected commands sequence:
>>>>
>>>> Received unexpected msg type. Expected 22 received 40 Fail to update
>>>> device iotlb Received unexpected msg type. Expected 40 received 22
>>>> Received unexpected msg type. Expected 22 received 11 Fail to update
>>>> device iotlb Received unexpected msg type. Expected 11 received 22
>>>> vhost VQ 1 ring restore failed: -71: Protocol error (71) Received
>>>> unexpected msg type. Expected 22 received 11 Fail to update device
>>>> iotlb Received unexpected msg type. Expected 11 received 22 vhost VQ
>>>> 0 ring restore failed: -71: Protocol error (71) unable to start
>>>> vhost net: 71: falling back on userspace virtio
>>>>
>>>> It receives VHOST_USER_GET_STATUS (40) when it expects
>>>> VHOST_USER_IOTLB_MSG (22) and VHOST_USER_IOTLB_MSG when it expects
>>>> VHOST_USER_GET_STATUS.
>>>> and VHOST_USER_GET_VRING_BASE (11) when it expect
>>>> VHOST_USER_GET_STATUS and so on.
>>>>
>>>> Any idea?
> 
> We only have a single thread on DPDK side to handle Vhost-user requests, it will read a request, handle it and reply to it. Then it reads the next one, etc... So I don't think it is possible to mix request replies order on DPDK side.
> 
> Maybe there are two threads concurrently sending requests on QEMU side?
> 
> Regards,
> Maxime
> 
>>>> Thanks,
>>>> Laurent
>>>
>>>
>>> So I am guessing it's coming from:
>>>
>>>       if (msg.hdr.request != request) {
>>>           error_report("Received unexpected msg type. Expected %d
>>> received %d",
>>>                        request, msg.hdr.request);
>>>           return -EPROTO;
>>>       }
>>>
>>> in process_message_reply and/or in vhost_user_get_u64.
>>>
>>>
>>>> On 11/7/22 23:53, Michael S. Tsirkin wrote:
>>>>> From: Yajun Wu <yajunw@nvidia.com>
>>>>>
>>>>> The motivation of adding vhost-user vhost_dev_start support is to
>>>>> improve backend configuration speed and reduce live migration VM
>>>>> downtime.
>>>>>
>>>>> Today VQ configuration is issued one by one. For virtio net with
>>>>> multi-queue support, backend needs to update RSS (Receive side
>>>>> scaling) on every rx queue enable. Updating RSS is time-consuming
>>>>> (typical time like 7ms).
>>>>>
>>>>> Implement already defined vhost status and message in the vhost
>>>>> specification [1].
>>>>> (a) VHOST_USER_PROTOCOL_F_STATUS
>>>>> (b) VHOST_USER_SET_STATUS
>>>>> (c) VHOST_USER_GET_STATUS
>>>>>
>>>>> Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK
>>>>> for device start and reset(0) for device stop.
>>>>>
>>>>> On reception of the DRIVER_OK message, backend can apply the needed
>>>>> setting only once (instead of incremental) and also utilize
>>>>> parallelism on enabling queues.
>>>>>
>>>>> This improves QEMU's live migration downtime with vhost user
>>>>> backend implementation by great margin, specially for the large
>>>>> number of VQs of 64 from 800 msec to 250 msec.
>>>>>
>>>>> [1]
>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fq
>>>>> emu-project.gitlab.io%2Fqemu%2Finterop%2Fvhost-user.html&data=05%7C
>>>>> 01%7Cyajunw%40nvidia.com%7C47e6e0fabd044383fd3308daf47f0253%7C43083
>>>>> d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638091123577559319%7CUnknown%
>>>>> 7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
>>>>> CJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YPbrFRJA92KLLwADMUDvwBt%2Fme2Ef
>>>>> GZuVANOmXH5pic%3D&reserved=0
>>>>>
>>>>> Signed-off-by: Yajun Wu <yajunw@nvidia.com>
>>>>> Acked-by: Parav Pandit <parav@nvidia.com>
>>>>> Message-Id: <20221017064452.1226514-3-yajunw@nvidia.com>
>>>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
>>>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>>>
>>> Probably easiest to debug from dpdk side.
>>> Does the problem go away if you disable the feature
>>> VHOST_USER_PROTOCOL_F_STATUS in dpdk?
>>
>> Maxime could you help to debug this?
>>
>> Thanks,
>> Laurent
>>
> 



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
  2023-01-17  9:49             ` Maxime Coquelin
@ 2023-01-17 12:12               ` Greg Kurz
  2023-01-17 12:36                 ` Greg Kurz
  0 siblings, 1 reply; 151+ messages in thread
From: Greg Kurz @ 2023-01-17 12:12 UTC (permalink / raw)
  To: Maxime Coquelin
  Cc: Yajun Wu, Laurent Vivier, Stefan Hajnoczi, qemu-devel,
	Peter Maydell, Parav Pandit, Michael S. Tsirkin

Hi Maxime,

On Tue, 17 Jan 2023 10:49:37 +0100
Maxime Coquelin <maxime.coquelin@redhat.com> wrote:

> Hi Yajun,
> 
> On 1/16/23 08:14, Yajun Wu wrote:
> > Not quite sure about the whole picture.
> > 
> > Seems while qemu waiting response of vhost_user_get_status, dpdk send out VHOST_USER_SLAVE_IOTLB_MSG and trigger qemu function vhost_backend_update_device_iotlb.
> > Qemu wait on reply of VHOST_USER_IOTLB_MSG but get VHOST_USER_GET_STATUS reply.
> 
> Thanks for the backtrace, that helps a lot.
> 
> The issue happens because:
>   1. Introduction of nested event loop in vhost_user_read() [0] features
> that enables handling slave channel request while waiting for reply on
> the masyer channel.
>   2. Slave VHOST_USER_SLAVE_IOTLB_MSG slave request handling ends-up 
> sending a VHOST_USER_IOTLB_MSG on the master channel.
> 
> So while waiting for VHOST_USER_IOTLB_MSG reply, it receives the reply
> for the first request sent on the master channel, here the
> VHOST_USER_GET_STATUS reply.
> 
> I don't see an easy way to fix it.
> 
> One option would be to have the slave channel being handled by another
> thread, and protect master channel with a lock to enforce the
> synchronization. But this may induce other issues, so that's not a light
> change.
> 

This is going to be tough because the back-end might have set the
VHOST_USER_NEED_REPLY_MASK flag on the VHOST_USER_SLAVE_IOTLB_MSG
request and thus might be waiting for a response on the slave
channel. In order to emit such a response, the front-end must
send VHOST_USER_SLAVE_IOTLB_MSG updates on the master channel *first*
according to the protocol specification. This means that we really
cannot handle VHOST_USER_SLAVE_IOTLB_MSG requests while there's
an on-going transaction on the master channel.

> (Adding Greg and Stefan, who worked on the nested event loop series.)
> 
> Simply reverting nested event loop support may not be an option, since
> it would break virtiofsd, as if my understanding is correct, waits for
> some slave channel request to complete in order to complete a request
> made by QEMU on the master channel.
> 
> Any thougths?
> 

Well... the nested even loop was added as preparatory work for "the
upcoming enablement of DAX with virtio-fs". This requires changes on
the QEMU side that haven't been merged yet. Technically, it seems that
reverting the nested event loop won't break anything in upstream QEMU
at this point (but this will bite again as soon as DAX enablement gets
merged).

AFAIK the event loop is only needed for the VHOST_USER_GET_VRING_BASE
message. Another possibility might be to create the nested event loop
in this case only : this would allow VHOST_USER_GET_STATUS to complete
before QEMU starts processing the VHOST_USER_SLAVE_IOTLB_MSG requests.

Cheers,

--
Greg

> Maxime
> 
> [0]: 
> https://patchwork.ozlabs.org/project/qemu-devel/patch/20210312092212.782255-6-groug@kaod.org/
> 
> 
> > Break on first error message("Received unexpected msg type. Expected 22 received 40")
> > 
> > #0  0x0000555555b72ed4 in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffa330) at ../hw/virtio/vhost-user.c:445
> > #1  0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffa600) at ../hw/virtio/vhost-user.c:2341
> > #2  0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10442706944, uaddr=140736119902208, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> > #3  0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10442706944, write=1) at ../hw/virtio/vhost.c:1113
> > #4  0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffa7b0) at ../hw/virtio/vhost-backend.c:393
> > #5  0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> > #6  0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556a06fb0, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> > #7  0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > #8  0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> > #9  0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> > #10 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffac50) at ../hw/virtio/vhost-user.c:413
> > #11 0x0000555555b72e9b in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffaf10) at ../hw/virtio/vhost-user.c:439
> > #12 0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffb1e0) at ../hw/virtio/vhost-user.c:2341
> > #13 0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10468392960, uaddr=140736145588224, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> > #14 0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10468392960, write=1) at ../hw/virtio/vhost.c:1113
> > #15 0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffb390) at ../hw/virtio/vhost-backend.c:393
> > #16 0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> > #17 0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556c70250, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> > #18 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > #19 0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> > #20 0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> > #21 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffb830) at ../hw/virtio/vhost-user.c:413
> > #22 0x0000555555b72e9b in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffbaf0) at ../hw/virtio/vhost-user.c:439
> > #23 0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffbdc0) at ../hw/virtio/vhost-user.c:2341
> > #24 0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10442702848, uaddr=140736119898112, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> > #25 0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10442702848, write=1) at ../hw/virtio/vhost.c:1113
> > #26 0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffbf70) at ../hw/virtio/vhost-backend.c:393
> > #27 0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> > #28 0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556f1a530, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> > #29 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > #30 0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> > #31 0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> > #32 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffc420) at ../hw/virtio/vhost-user.c:413
> > #33 0x0000555555b754b1 in vhost_user_get_u64 (dev=0x5555584dd600, request=40, u64=0x7fffffffc6e0) at ../hw/virtio/vhost-user.c:1349
> > #34 0x0000555555b758ff in vhost_user_get_status (dev=0x5555584dd600, status=0x7fffffffc713 "W\020") at ../hw/virtio/vhost-user.c:1474
> > #35 0x0000555555b75967 in vhost_user_add_status (dev=0x5555584dd600, status=7 '\a') at ../hw/virtio/vhost-user.c:1488
> > #36 0x0000555555b78bf6 in vhost_user_dev_start (dev=0x5555584dd600, started=true) at ../hw/virtio/vhost-user.c:2758
> > #37 0x0000555555b709ad in vhost_dev_start (hdev=0x5555584dd600, vdev=0x555557b965d0, vrings=false) at ../hw/virtio/vhost.c:1988
> > #38 0x000055555584291c in vhost_net_start_one (net=0x5555584dd600, dev=0x555557b965d0) at ../hw/net/vhost_net.c:271
> > #39 0x0000555555842f1e in vhost_net_start (dev=0x555557b965d0, ncs=0x555557bc09e0, data_queue_pairs=1, cvq=0) at ../hw/net/vhost_net.c:412
> > #40 0x0000555555b1bf61 in virtio_net_vhost_status (n=0x555557b965d0, status=15 '\017') at ../hw/net/virtio-net.c:311
> > #41 0x0000555555b1c20c in virtio_net_set_status (vdev=0x555557b965d0, status=15 '\017') at ../hw/net/virtio-net.c:392
> > #42 0x0000555555b1ed04 in virtio_net_handle_mq (n=0x555557b965d0, cmd=0 '\000', iov=0x555556c7ef50, iov_cnt=1) at ../hw/net/virtio-net.c:1497
> > #43 0x0000555555b1eef0 in virtio_net_handle_ctrl_iov (vdev=0x555557b965d0, in_sg=0x555556a09880, in_num=1, out_sg=0x555556a09890, out_num=1) at ../hw/net/virtio-net.c:1534
> > #44 0x0000555555b1efe9 in virtio_net_handle_ctrl (vdev=0x555557b965d0, vq=0x7fffc04ac140) at ../hw/net/virtio-net.c:1557
> > #45 0x0000555555b63776 in virtio_queue_notify_vq (vq=0x7fffc04ac140) at ../hw/virtio/virtio.c:2249
> > #46 0x0000555555b669dc in virtio_queue_host_notifier_read (n=0x7fffc04ac1b4) at ../hw/virtio/virtio.c:3529
> > #47 0x0000555555e3f458 in aio_dispatch_handler (ctx=0x555556a016c0, node=0x7ffd8800e430) at ../util/aio-posix.c:369
> > #48 0x0000555555e3f613 in aio_dispatch_handlers (ctx=0x555556a016c0) at ../util/aio-posix.c:412
> > #49 0x0000555555e3f669 in aio_dispatch (ctx=0x555556a016c0) at ../util/aio-posix.c:422
> > #50 0x0000555555e585de in aio_ctx_dispatch (source=0x555556a016c0, callback=0x0, user_data=0x0) at ../util/async.c:321
> > #51 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > #52 0x0000555555e5abea in glib_pollfds_poll () at ../util/main-loop.c:295
> > #53 0x0000555555e5ac64 in os_host_main_loop_wait (timeout=0) at ../util/main-loop.c:318
> > #54 0x0000555555e5ad69 in main_loop_wait (nonblocking=0) at ../util/main-loop.c:604
> > #55 0x00005555559693de in qemu_main_loop () at ../softmmu/runstate.c:731
> > #56 0x00005555556e7c06 in qemu_default_main () at ../softmmu/main.c:37
> > #57 0x00005555556e7c3c in main (argc=71, argv=0x7fffffffcda8) at ../softmmu/main.c:48
> > 
> > 
> > 
> > -----Original Message-----
> > From: Maxime Coquelin <maxime.coquelin@redhat.com>
> > Sent: Thursday, January 12, 2023 5:26 PM
> > To: Laurent Vivier <lvivier@redhat.com>
> > Cc: qemu-devel@nongnu.org; Peter Maydell <peter.maydell@linaro.org>; Yajun Wu <yajunw@nvidia.com>; Parav Pandit <parav@nvidia.com>; Michael S. Tsirkin <mst@redhat.com>
> > Subject: Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
> > 
> > External email: Use caution opening links or attachments
> > 
> > 
> > Hi Laurent,
> > 
> > On 1/11/23 10:50, Laurent Vivier wrote:
> >> On 1/9/23 11:55, Michael S. Tsirkin wrote:
> >>> On Fri, Jan 06, 2023 at 03:21:43PM +0100, Laurent Vivier wrote:
> >>>> Hi,
> >>>>
> >>>> it seems this patch breaks vhost-user with DPDK.
> >>>>
> >>>> See
> >>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbu
> >>>> gzilla.redhat.com%2Fshow_bug.cgi%3Fid%3D2155173&data=05%7C01%7Cyajun
> >>>> w%40nvidia.com%7C47e6e0fabd044383fd3308daf47f0253%7C43083d15727340c1
> >>>> b7db39efd9ccc17a%7C0%7C0%7C638091123577559319%7CUnknown%7CTWFpbGZsb3
> >>>> d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
> >>>> %7C3000%7C%7C%7C&sdata=1pjChYTKHVmBoempNitiZHBdrlPIMFjKoD6FeOVSay0%3
> >>>> D&reserved=0
> >>>>
> >>>> it seems QEMU doesn't receive the expected commands sequence:
> >>>>
> >>>> Received unexpected msg type. Expected 22 received 40 Fail to update
> >>>> device iotlb Received unexpected msg type. Expected 40 received 22
> >>>> Received unexpected msg type. Expected 22 received 11 Fail to update
> >>>> device iotlb Received unexpected msg type. Expected 11 received 22
> >>>> vhost VQ 1 ring restore failed: -71: Protocol error (71) Received
> >>>> unexpected msg type. Expected 22 received 11 Fail to update device
> >>>> iotlb Received unexpected msg type. Expected 11 received 22 vhost VQ
> >>>> 0 ring restore failed: -71: Protocol error (71) unable to start
> >>>> vhost net: 71: falling back on userspace virtio
> >>>>
> >>>> It receives VHOST_USER_GET_STATUS (40) when it expects
> >>>> VHOST_USER_IOTLB_MSG (22) and VHOST_USER_IOTLB_MSG when it expects
> >>>> VHOST_USER_GET_STATUS.
> >>>> and VHOST_USER_GET_VRING_BASE (11) when it expect
> >>>> VHOST_USER_GET_STATUS and so on.
> >>>>
> >>>> Any idea?
> > 
> > We only have a single thread on DPDK side to handle Vhost-user requests, it will read a request, handle it and reply to it. Then it reads the next one, etc... So I don't think it is possible to mix request replies order on DPDK side.
> > 
> > Maybe there are two threads concurrently sending requests on QEMU side?
> > 
> > Regards,
> > Maxime
> > 
> >>>> Thanks,
> >>>> Laurent
> >>>
> >>>
> >>> So I am guessing it's coming from:
> >>>
> >>>       if (msg.hdr.request != request) {
> >>>           error_report("Received unexpected msg type. Expected %d
> >>> received %d",
> >>>                        request, msg.hdr.request);
> >>>           return -EPROTO;
> >>>       }
> >>>
> >>> in process_message_reply and/or in vhost_user_get_u64.
> >>>
> >>>
> >>>> On 11/7/22 23:53, Michael S. Tsirkin wrote:
> >>>>> From: Yajun Wu <yajunw@nvidia.com>
> >>>>>
> >>>>> The motivation of adding vhost-user vhost_dev_start support is to
> >>>>> improve backend configuration speed and reduce live migration VM
> >>>>> downtime.
> >>>>>
> >>>>> Today VQ configuration is issued one by one. For virtio net with
> >>>>> multi-queue support, backend needs to update RSS (Receive side
> >>>>> scaling) on every rx queue enable. Updating RSS is time-consuming
> >>>>> (typical time like 7ms).
> >>>>>
> >>>>> Implement already defined vhost status and message in the vhost
> >>>>> specification [1].
> >>>>> (a) VHOST_USER_PROTOCOL_F_STATUS
> >>>>> (b) VHOST_USER_SET_STATUS
> >>>>> (c) VHOST_USER_GET_STATUS
> >>>>>
> >>>>> Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK
> >>>>> for device start and reset(0) for device stop.
> >>>>>
> >>>>> On reception of the DRIVER_OK message, backend can apply the needed
> >>>>> setting only once (instead of incremental) and also utilize
> >>>>> parallelism on enabling queues.
> >>>>>
> >>>>> This improves QEMU's live migration downtime with vhost user
> >>>>> backend implementation by great margin, specially for the large
> >>>>> number of VQs of 64 from 800 msec to 250 msec.
> >>>>>
> >>>>> [1]
> >>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fq
> >>>>> emu-project.gitlab.io%2Fqemu%2Finterop%2Fvhost-user.html&data=05%7C
> >>>>> 01%7Cyajunw%40nvidia.com%7C47e6e0fabd044383fd3308daf47f0253%7C43083
> >>>>> d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638091123577559319%7CUnknown%
> >>>>> 7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
> >>>>> CJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YPbrFRJA92KLLwADMUDvwBt%2Fme2Ef
> >>>>> GZuVANOmXH5pic%3D&reserved=0
> >>>>>
> >>>>> Signed-off-by: Yajun Wu <yajunw@nvidia.com>
> >>>>> Acked-by: Parav Pandit <parav@nvidia.com>
> >>>>> Message-Id: <20221017064452.1226514-3-yajunw@nvidia.com>
> >>>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> >>>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> >>>
> >>> Probably easiest to debug from dpdk side.
> >>> Does the problem go away if you disable the feature
> >>> VHOST_USER_PROTOCOL_F_STATUS in dpdk?
> >>
> >> Maxime could you help to debug this?
> >>
> >> Thanks,
> >> Laurent
> >>
> > 
> 



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
  2023-01-17 12:12               ` Greg Kurz
@ 2023-01-17 12:36                 ` Greg Kurz
  2023-01-17 15:07                   ` Maxime Coquelin
  2023-01-19 19:48                   ` Dr. David Alan Gilbert
  0 siblings, 2 replies; 151+ messages in thread
From: Greg Kurz @ 2023-01-17 12:36 UTC (permalink / raw)
  To: Maxime Coquelin
  Cc: Yajun Wu, Laurent Vivier, Stefan Hajnoczi, qemu-devel,
	Peter Maydell, Parav Pandit, Michael S. Tsirkin,
	Dr. David Alan Gilbert

On Tue, 17 Jan 2023 13:12:57 +0100
Greg Kurz <groug@kaod.org> wrote:

> Hi Maxime,
> 
> On Tue, 17 Jan 2023 10:49:37 +0100
> Maxime Coquelin <maxime.coquelin@redhat.com> wrote:
> 
> > Hi Yajun,
> > 
> > On 1/16/23 08:14, Yajun Wu wrote:
> > > Not quite sure about the whole picture.
> > > 
> > > Seems while qemu waiting response of vhost_user_get_status, dpdk send out VHOST_USER_SLAVE_IOTLB_MSG and trigger qemu function vhost_backend_update_device_iotlb.
> > > Qemu wait on reply of VHOST_USER_IOTLB_MSG but get VHOST_USER_GET_STATUS reply.
> > 
> > Thanks for the backtrace, that helps a lot.
> > 
> > The issue happens because:
> >   1. Introduction of nested event loop in vhost_user_read() [0] features
> > that enables handling slave channel request while waiting for reply on
> > the masyer channel.
> >   2. Slave VHOST_USER_SLAVE_IOTLB_MSG slave request handling ends-up 
> > sending a VHOST_USER_IOTLB_MSG on the master channel.
> > 
> > So while waiting for VHOST_USER_IOTLB_MSG reply, it receives the reply
> > for the first request sent on the master channel, here the
> > VHOST_USER_GET_STATUS reply.
> > 
> > I don't see an easy way to fix it.
> > 
> > One option would be to have the slave channel being handled by another
> > thread, and protect master channel with a lock to enforce the
> > synchronization. But this may induce other issues, so that's not a light
> > change.
> > 
> 
> This is going to be tough because the back-end might have set the
> VHOST_USER_NEED_REPLY_MASK flag on the VHOST_USER_SLAVE_IOTLB_MSG
> request and thus might be waiting for a response on the slave
> channel. In order to emit such a response, the front-end must
> send VHOST_USER_SLAVE_IOTLB_MSG updates on the master channel *first*
> according to the protocol specification. This means that we really
> cannot handle VHOST_USER_SLAVE_IOTLB_MSG requests while there's
> an on-going transaction on the master channel.
> 
> > (Adding Greg and Stefan, who worked on the nested event loop series.)
> > 
> > Simply reverting nested event loop support may not be an option, since
> > it would break virtiofsd, as if my understanding is correct, waits for
> > some slave channel request to complete in order to complete a request
> > made by QEMU on the master channel.
> > 
> > Any thougths?
> > 
> 
> Well... the nested even loop was added as preparatory work for "the
> upcoming enablement of DAX with virtio-fs". This requires changes on
> the QEMU side that haven't been merged yet. Technically, it seems that
> reverting the nested event loop won't break anything in upstream QEMU
> at this point (but this will bite again as soon as DAX enablement gets
> merged).
> 

Cc'ing Dave to know about the DAX enablement status.

> AFAIK the event loop is only needed for the VHOST_USER_GET_VRING_BASE
> message. Another possibility might be to create the nested event loop
> in this case only : this would allow VHOST_USER_GET_STATUS to complete
> before QEMU starts processing the VHOST_USER_SLAVE_IOTLB_MSG requests.
> 
> Cheers,
> 
> --
> Greg
> 
> > Maxime
> > 
> > [0]: 
> > https://patchwork.ozlabs.org/project/qemu-devel/patch/20210312092212.782255-6-groug@kaod.org/
> > 
> > 
> > > Break on first error message("Received unexpected msg type. Expected 22 received 40")
> > > 
> > > #0  0x0000555555b72ed4 in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffa330) at ../hw/virtio/vhost-user.c:445
> > > #1  0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffa600) at ../hw/virtio/vhost-user.c:2341
> > > #2  0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10442706944, uaddr=140736119902208, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> > > #3  0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10442706944, write=1) at ../hw/virtio/vhost.c:1113
> > > #4  0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffa7b0) at ../hw/virtio/vhost-backend.c:393
> > > #5  0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> > > #6  0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556a06fb0, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> > > #7  0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > > #8  0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> > > #9  0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> > > #10 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffac50) at ../hw/virtio/vhost-user.c:413
> > > #11 0x0000555555b72e9b in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffaf10) at ../hw/virtio/vhost-user.c:439
> > > #12 0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffb1e0) at ../hw/virtio/vhost-user.c:2341
> > > #13 0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10468392960, uaddr=140736145588224, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> > > #14 0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10468392960, write=1) at ../hw/virtio/vhost.c:1113
> > > #15 0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffb390) at ../hw/virtio/vhost-backend.c:393
> > > #16 0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> > > #17 0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556c70250, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> > > #18 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > > #19 0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> > > #20 0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> > > #21 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffb830) at ../hw/virtio/vhost-user.c:413
> > > #22 0x0000555555b72e9b in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffbaf0) at ../hw/virtio/vhost-user.c:439
> > > #23 0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffbdc0) at ../hw/virtio/vhost-user.c:2341
> > > #24 0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10442702848, uaddr=140736119898112, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> > > #25 0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10442702848, write=1) at ../hw/virtio/vhost.c:1113
> > > #26 0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffbf70) at ../hw/virtio/vhost-backend.c:393
> > > #27 0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> > > #28 0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556f1a530, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> > > #29 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > > #30 0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> > > #31 0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> > > #32 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffc420) at ../hw/virtio/vhost-user.c:413
> > > #33 0x0000555555b754b1 in vhost_user_get_u64 (dev=0x5555584dd600, request=40, u64=0x7fffffffc6e0) at ../hw/virtio/vhost-user.c:1349
> > > #34 0x0000555555b758ff in vhost_user_get_status (dev=0x5555584dd600, status=0x7fffffffc713 "W\020") at ../hw/virtio/vhost-user.c:1474
> > > #35 0x0000555555b75967 in vhost_user_add_status (dev=0x5555584dd600, status=7 '\a') at ../hw/virtio/vhost-user.c:1488
> > > #36 0x0000555555b78bf6 in vhost_user_dev_start (dev=0x5555584dd600, started=true) at ../hw/virtio/vhost-user.c:2758
> > > #37 0x0000555555b709ad in vhost_dev_start (hdev=0x5555584dd600, vdev=0x555557b965d0, vrings=false) at ../hw/virtio/vhost.c:1988
> > > #38 0x000055555584291c in vhost_net_start_one (net=0x5555584dd600, dev=0x555557b965d0) at ../hw/net/vhost_net.c:271
> > > #39 0x0000555555842f1e in vhost_net_start (dev=0x555557b965d0, ncs=0x555557bc09e0, data_queue_pairs=1, cvq=0) at ../hw/net/vhost_net.c:412
> > > #40 0x0000555555b1bf61 in virtio_net_vhost_status (n=0x555557b965d0, status=15 '\017') at ../hw/net/virtio-net.c:311
> > > #41 0x0000555555b1c20c in virtio_net_set_status (vdev=0x555557b965d0, status=15 '\017') at ../hw/net/virtio-net.c:392
> > > #42 0x0000555555b1ed04 in virtio_net_handle_mq (n=0x555557b965d0, cmd=0 '\000', iov=0x555556c7ef50, iov_cnt=1) at ../hw/net/virtio-net.c:1497
> > > #43 0x0000555555b1eef0 in virtio_net_handle_ctrl_iov (vdev=0x555557b965d0, in_sg=0x555556a09880, in_num=1, out_sg=0x555556a09890, out_num=1) at ../hw/net/virtio-net.c:1534
> > > #44 0x0000555555b1efe9 in virtio_net_handle_ctrl (vdev=0x555557b965d0, vq=0x7fffc04ac140) at ../hw/net/virtio-net.c:1557
> > > #45 0x0000555555b63776 in virtio_queue_notify_vq (vq=0x7fffc04ac140) at ../hw/virtio/virtio.c:2249
> > > #46 0x0000555555b669dc in virtio_queue_host_notifier_read (n=0x7fffc04ac1b4) at ../hw/virtio/virtio.c:3529
> > > #47 0x0000555555e3f458 in aio_dispatch_handler (ctx=0x555556a016c0, node=0x7ffd8800e430) at ../util/aio-posix.c:369
> > > #48 0x0000555555e3f613 in aio_dispatch_handlers (ctx=0x555556a016c0) at ../util/aio-posix.c:412
> > > #49 0x0000555555e3f669 in aio_dispatch (ctx=0x555556a016c0) at ../util/aio-posix.c:422
> > > #50 0x0000555555e585de in aio_ctx_dispatch (source=0x555556a016c0, callback=0x0, user_data=0x0) at ../util/async.c:321
> > > #51 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > > #52 0x0000555555e5abea in glib_pollfds_poll () at ../util/main-loop.c:295
> > > #53 0x0000555555e5ac64 in os_host_main_loop_wait (timeout=0) at ../util/main-loop.c:318
> > > #54 0x0000555555e5ad69 in main_loop_wait (nonblocking=0) at ../util/main-loop.c:604
> > > #55 0x00005555559693de in qemu_main_loop () at ../softmmu/runstate.c:731
> > > #56 0x00005555556e7c06 in qemu_default_main () at ../softmmu/main.c:37
> > > #57 0x00005555556e7c3c in main (argc=71, argv=0x7fffffffcda8) at ../softmmu/main.c:48
> > > 
> > > 
> > > 
> > > -----Original Message-----
> > > From: Maxime Coquelin <maxime.coquelin@redhat.com>
> > > Sent: Thursday, January 12, 2023 5:26 PM
> > > To: Laurent Vivier <lvivier@redhat.com>
> > > Cc: qemu-devel@nongnu.org; Peter Maydell <peter.maydell@linaro.org>; Yajun Wu <yajunw@nvidia.com>; Parav Pandit <parav@nvidia.com>; Michael S. Tsirkin <mst@redhat.com>
> > > Subject: Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
> > > 
> > > External email: Use caution opening links or attachments
> > > 
> > > 
> > > Hi Laurent,
> > > 
> > > On 1/11/23 10:50, Laurent Vivier wrote:
> > >> On 1/9/23 11:55, Michael S. Tsirkin wrote:
> > >>> On Fri, Jan 06, 2023 at 03:21:43PM +0100, Laurent Vivier wrote:
> > >>>> Hi,
> > >>>>
> > >>>> it seems this patch breaks vhost-user with DPDK.
> > >>>>
> > >>>> See
> > >>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbu
> > >>>> gzilla.redhat.com%2Fshow_bug.cgi%3Fid%3D2155173&data=05%7C01%7Cyajun
> > >>>> w%40nvidia.com%7C47e6e0fabd044383fd3308daf47f0253%7C43083d15727340c1
> > >>>> b7db39efd9ccc17a%7C0%7C0%7C638091123577559319%7CUnknown%7CTWFpbGZsb3
> > >>>> d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
> > >>>> %7C3000%7C%7C%7C&sdata=1pjChYTKHVmBoempNitiZHBdrlPIMFjKoD6FeOVSay0%3
> > >>>> D&reserved=0
> > >>>>
> > >>>> it seems QEMU doesn't receive the expected commands sequence:
> > >>>>
> > >>>> Received unexpected msg type. Expected 22 received 40 Fail to update
> > >>>> device iotlb Received unexpected msg type. Expected 40 received 22
> > >>>> Received unexpected msg type. Expected 22 received 11 Fail to update
> > >>>> device iotlb Received unexpected msg type. Expected 11 received 22
> > >>>> vhost VQ 1 ring restore failed: -71: Protocol error (71) Received
> > >>>> unexpected msg type. Expected 22 received 11 Fail to update device
> > >>>> iotlb Received unexpected msg type. Expected 11 received 22 vhost VQ
> > >>>> 0 ring restore failed: -71: Protocol error (71) unable to start
> > >>>> vhost net: 71: falling back on userspace virtio
> > >>>>
> > >>>> It receives VHOST_USER_GET_STATUS (40) when it expects
> > >>>> VHOST_USER_IOTLB_MSG (22) and VHOST_USER_IOTLB_MSG when it expects
> > >>>> VHOST_USER_GET_STATUS.
> > >>>> and VHOST_USER_GET_VRING_BASE (11) when it expect
> > >>>> VHOST_USER_GET_STATUS and so on.
> > >>>>
> > >>>> Any idea?
> > > 
> > > We only have a single thread on DPDK side to handle Vhost-user requests, it will read a request, handle it and reply to it. Then it reads the next one, etc... So I don't think it is possible to mix request replies order on DPDK side.
> > > 
> > > Maybe there are two threads concurrently sending requests on QEMU side?
> > > 
> > > Regards,
> > > Maxime
> > > 
> > >>>> Thanks,
> > >>>> Laurent
> > >>>
> > >>>
> > >>> So I am guessing it's coming from:
> > >>>
> > >>>       if (msg.hdr.request != request) {
> > >>>           error_report("Received unexpected msg type. Expected %d
> > >>> received %d",
> > >>>                        request, msg.hdr.request);
> > >>>           return -EPROTO;
> > >>>       }
> > >>>
> > >>> in process_message_reply and/or in vhost_user_get_u64.
> > >>>
> > >>>
> > >>>> On 11/7/22 23:53, Michael S. Tsirkin wrote:
> > >>>>> From: Yajun Wu <yajunw@nvidia.com>
> > >>>>>
> > >>>>> The motivation of adding vhost-user vhost_dev_start support is to
> > >>>>> improve backend configuration speed and reduce live migration VM
> > >>>>> downtime.
> > >>>>>
> > >>>>> Today VQ configuration is issued one by one. For virtio net with
> > >>>>> multi-queue support, backend needs to update RSS (Receive side
> > >>>>> scaling) on every rx queue enable. Updating RSS is time-consuming
> > >>>>> (typical time like 7ms).
> > >>>>>
> > >>>>> Implement already defined vhost status and message in the vhost
> > >>>>> specification [1].
> > >>>>> (a) VHOST_USER_PROTOCOL_F_STATUS
> > >>>>> (b) VHOST_USER_SET_STATUS
> > >>>>> (c) VHOST_USER_GET_STATUS
> > >>>>>
> > >>>>> Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK
> > >>>>> for device start and reset(0) for device stop.
> > >>>>>
> > >>>>> On reception of the DRIVER_OK message, backend can apply the needed
> > >>>>> setting only once (instead of incremental) and also utilize
> > >>>>> parallelism on enabling queues.
> > >>>>>
> > >>>>> This improves QEMU's live migration downtime with vhost user
> > >>>>> backend implementation by great margin, specially for the large
> > >>>>> number of VQs of 64 from 800 msec to 250 msec.
> > >>>>>
> > >>>>> [1]
> > >>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fq
> > >>>>> emu-project.gitlab.io%2Fqemu%2Finterop%2Fvhost-user.html&data=05%7C
> > >>>>> 01%7Cyajunw%40nvidia.com%7C47e6e0fabd044383fd3308daf47f0253%7C43083
> > >>>>> d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638091123577559319%7CUnknown%
> > >>>>> 7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
> > >>>>> CJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YPbrFRJA92KLLwADMUDvwBt%2Fme2Ef
> > >>>>> GZuVANOmXH5pic%3D&reserved=0
> > >>>>>
> > >>>>> Signed-off-by: Yajun Wu <yajunw@nvidia.com>
> > >>>>> Acked-by: Parav Pandit <parav@nvidia.com>
> > >>>>> Message-Id: <20221017064452.1226514-3-yajunw@nvidia.com>
> > >>>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > >>>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > >>>
> > >>> Probably easiest to debug from dpdk side.
> > >>> Does the problem go away if you disable the feature
> > >>> VHOST_USER_PROTOCOL_F_STATUS in dpdk?
> > >>
> > >> Maxime could you help to debug this?
> > >>
> > >> Thanks,
> > >> Laurent
> > >>
> > > 
> > 
> 
> 



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
  2023-01-17 12:36                 ` Greg Kurz
@ 2023-01-17 15:07                   ` Maxime Coquelin
  2023-01-17 17:55                     ` Greg Kurz
  2023-01-19 19:48                   ` Dr. David Alan Gilbert
  1 sibling, 1 reply; 151+ messages in thread
From: Maxime Coquelin @ 2023-01-17 15:07 UTC (permalink / raw)
  To: Greg Kurz
  Cc: Yajun Wu, Laurent Vivier, Stefan Hajnoczi, qemu-devel,
	Peter Maydell, Parav Pandit, Michael S. Tsirkin,
	Dr. David Alan Gilbert



On 1/17/23 13:36, Greg Kurz wrote:
> On Tue, 17 Jan 2023 13:12:57 +0100
> Greg Kurz <groug@kaod.org> wrote:
> 
>> Hi Maxime,
>>
>> On Tue, 17 Jan 2023 10:49:37 +0100
>> Maxime Coquelin <maxime.coquelin@redhat.com> wrote:
>>
>>> Hi Yajun,
>>>
>>> On 1/16/23 08:14, Yajun Wu wrote:
>>>> Not quite sure about the whole picture.
>>>>
>>>> Seems while qemu waiting response of vhost_user_get_status, dpdk send out VHOST_USER_SLAVE_IOTLB_MSG and trigger qemu function vhost_backend_update_device_iotlb.
>>>> Qemu wait on reply of VHOST_USER_IOTLB_MSG but get VHOST_USER_GET_STATUS reply.
>>>
>>> Thanks for the backtrace, that helps a lot.
>>>
>>> The issue happens because:
>>>    1. Introduction of nested event loop in vhost_user_read() [0] features
>>> that enables handling slave channel request while waiting for reply on
>>> the masyer channel.
>>>    2. Slave VHOST_USER_SLAVE_IOTLB_MSG slave request handling ends-up
>>> sending a VHOST_USER_IOTLB_MSG on the master channel.
>>>
>>> So while waiting for VHOST_USER_IOTLB_MSG reply, it receives the reply
>>> for the first request sent on the master channel, here the
>>> VHOST_USER_GET_STATUS reply.
>>>
>>> I don't see an easy way to fix it.
>>>
>>> One option would be to have the slave channel being handled by another
>>> thread, and protect master channel with a lock to enforce the
>>> synchronization. But this may induce other issues, so that's not a light
>>> change.
>>>
>>
>> This is going to be tough because the back-end might have set the
>> VHOST_USER_NEED_REPLY_MASK flag on the VHOST_USER_SLAVE_IOTLB_MSG
>> request and thus might be waiting for a response on the slave
>> channel. In order to emit such a response, the front-end must
>> send VHOST_USER_SLAVE_IOTLB_MSG updates on the master channel *first*
>> according to the protocol specification. This means that we really
>> cannot handle VHOST_USER_SLAVE_IOTLB_MSG requests while there's
>> an on-going transaction on the master channel.

Since the slave channel would be handled on another thread, it means the
on-going transaction on the master channel can continue. Once done, it
will release the mutex, and the thread handling the slave channel can
take it send the IOTLB update on the master channel.

That would work with DPDK, which does not request reply-ack on IOTLB
misses.

For the DAX enablement case, my understanding is that the handling of
the slave requests by QEMU does not induce sending requests on the
master channel, so if I'm not mistaken, it should work too. The thread
handling the slave requests can send the reply on the slave channel,
while the thread handling the master channel is blocked waiting for the
GET_VRING_BASE reply. Is that correct?

>>
>>> (Adding Greg and Stefan, who worked on the nested event loop series.)
>>>
>>> Simply reverting nested event loop support may not be an option, since
>>> it would break virtiofsd, as if my understanding is correct, waits for
>>> some slave channel request to complete in order to complete a request
>>> made by QEMU on the master channel.
>>>
>>> Any thougths?
>>>
>>
>> Well... the nested even loop was added as preparatory work for "the
>> upcoming enablement of DAX with virtio-fs". This requires changes on
>> the QEMU side that haven't been merged yet. Technically, it seems that
>> reverting the nested event loop won't break anything in upstream QEMU
>> at this point (but this will bite again as soon as DAX enablement gets
>> merged).
>>
> 
> Cc'ing Dave to know about the DAX enablement status.
> 
>> AFAIK the event loop is only needed for the VHOST_USER_GET_VRING_BASE
>> message. Another possibility might be to create the nested event loop
>> in this case only : this would allow VHOST_USER_GET_STATUS to complete
>> before QEMU starts processing the VHOST_USER_SLAVE_IOTLB_MSG requests.
>>
>> Cheers,
>>
>> --
>> Greg
>>
>>> Maxime
>>>
>>> [0]:
>>> https://patchwork.ozlabs.org/project/qemu-devel/patch/20210312092212.782255-6-groug@kaod.org/
>>>
>>>
>>>> Break on first error message("Received unexpected msg type. Expected 22 received 40")
>>>>
>>>> #0  0x0000555555b72ed4 in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffa330) at ../hw/virtio/vhost-user.c:445
>>>> #1  0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffa600) at ../hw/virtio/vhost-user.c:2341
>>>> #2  0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10442706944, uaddr=140736119902208, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
>>>> #3  0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10442706944, write=1) at ../hw/virtio/vhost.c:1113
>>>> #4  0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffa7b0) at ../hw/virtio/vhost-backend.c:393
>>>> #5  0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
>>>> #6  0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556a06fb0, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
>>>> #7  0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
>>>> #8  0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
>>>> #9  0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
>>>> #10 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffac50) at ../hw/virtio/vhost-user.c:413
>>>> #11 0x0000555555b72e9b in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffaf10) at ../hw/virtio/vhost-user.c:439
>>>> #12 0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffb1e0) at ../hw/virtio/vhost-user.c:2341
>>>> #13 0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10468392960, uaddr=140736145588224, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
>>>> #14 0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10468392960, write=1) at ../hw/virtio/vhost.c:1113
>>>> #15 0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffb390) at ../hw/virtio/vhost-backend.c:393
>>>> #16 0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
>>>> #17 0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556c70250, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
>>>> #18 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
>>>> #19 0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
>>>> #20 0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
>>>> #21 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffb830) at ../hw/virtio/vhost-user.c:413
>>>> #22 0x0000555555b72e9b in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffbaf0) at ../hw/virtio/vhost-user.c:439
>>>> #23 0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffbdc0) at ../hw/virtio/vhost-user.c:2341
>>>> #24 0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10442702848, uaddr=140736119898112, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
>>>> #25 0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10442702848, write=1) at ../hw/virtio/vhost.c:1113
>>>> #26 0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffbf70) at ../hw/virtio/vhost-backend.c:393
>>>> #27 0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
>>>> #28 0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556f1a530, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
>>>> #29 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
>>>> #30 0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
>>>> #31 0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
>>>> #32 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffc420) at ../hw/virtio/vhost-user.c:413
>>>> #33 0x0000555555b754b1 in vhost_user_get_u64 (dev=0x5555584dd600, request=40, u64=0x7fffffffc6e0) at ../hw/virtio/vhost-user.c:1349
>>>> #34 0x0000555555b758ff in vhost_user_get_status (dev=0x5555584dd600, status=0x7fffffffc713 "W\020") at ../hw/virtio/vhost-user.c:1474
>>>> #35 0x0000555555b75967 in vhost_user_add_status (dev=0x5555584dd600, status=7 '\a') at ../hw/virtio/vhost-user.c:1488
>>>> #36 0x0000555555b78bf6 in vhost_user_dev_start (dev=0x5555584dd600, started=true) at ../hw/virtio/vhost-user.c:2758
>>>> #37 0x0000555555b709ad in vhost_dev_start (hdev=0x5555584dd600, vdev=0x555557b965d0, vrings=false) at ../hw/virtio/vhost.c:1988
>>>> #38 0x000055555584291c in vhost_net_start_one (net=0x5555584dd600, dev=0x555557b965d0) at ../hw/net/vhost_net.c:271
>>>> #39 0x0000555555842f1e in vhost_net_start (dev=0x555557b965d0, ncs=0x555557bc09e0, data_queue_pairs=1, cvq=0) at ../hw/net/vhost_net.c:412
>>>> #40 0x0000555555b1bf61 in virtio_net_vhost_status (n=0x555557b965d0, status=15 '\017') at ../hw/net/virtio-net.c:311
>>>> #41 0x0000555555b1c20c in virtio_net_set_status (vdev=0x555557b965d0, status=15 '\017') at ../hw/net/virtio-net.c:392
>>>> #42 0x0000555555b1ed04 in virtio_net_handle_mq (n=0x555557b965d0, cmd=0 '\000', iov=0x555556c7ef50, iov_cnt=1) at ../hw/net/virtio-net.c:1497
>>>> #43 0x0000555555b1eef0 in virtio_net_handle_ctrl_iov (vdev=0x555557b965d0, in_sg=0x555556a09880, in_num=1, out_sg=0x555556a09890, out_num=1) at ../hw/net/virtio-net.c:1534
>>>> #44 0x0000555555b1efe9 in virtio_net_handle_ctrl (vdev=0x555557b965d0, vq=0x7fffc04ac140) at ../hw/net/virtio-net.c:1557
>>>> #45 0x0000555555b63776 in virtio_queue_notify_vq (vq=0x7fffc04ac140) at ../hw/virtio/virtio.c:2249
>>>> #46 0x0000555555b669dc in virtio_queue_host_notifier_read (n=0x7fffc04ac1b4) at ../hw/virtio/virtio.c:3529
>>>> #47 0x0000555555e3f458 in aio_dispatch_handler (ctx=0x555556a016c0, node=0x7ffd8800e430) at ../util/aio-posix.c:369
>>>> #48 0x0000555555e3f613 in aio_dispatch_handlers (ctx=0x555556a016c0) at ../util/aio-posix.c:412
>>>> #49 0x0000555555e3f669 in aio_dispatch (ctx=0x555556a016c0) at ../util/aio-posix.c:422
>>>> #50 0x0000555555e585de in aio_ctx_dispatch (source=0x555556a016c0, callback=0x0, user_data=0x0) at ../util/async.c:321
>>>> #51 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
>>>> #52 0x0000555555e5abea in glib_pollfds_poll () at ../util/main-loop.c:295
>>>> #53 0x0000555555e5ac64 in os_host_main_loop_wait (timeout=0) at ../util/main-loop.c:318
>>>> #54 0x0000555555e5ad69 in main_loop_wait (nonblocking=0) at ../util/main-loop.c:604
>>>> #55 0x00005555559693de in qemu_main_loop () at ../softmmu/runstate.c:731
>>>> #56 0x00005555556e7c06 in qemu_default_main () at ../softmmu/main.c:37
>>>> #57 0x00005555556e7c3c in main (argc=71, argv=0x7fffffffcda8) at ../softmmu/main.c:48
>>>>
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>>>> Sent: Thursday, January 12, 2023 5:26 PM
>>>> To: Laurent Vivier <lvivier@redhat.com>
>>>> Cc: qemu-devel@nongnu.org; Peter Maydell <peter.maydell@linaro.org>; Yajun Wu <yajunw@nvidia.com>; Parav Pandit <parav@nvidia.com>; Michael S. Tsirkin <mst@redhat.com>
>>>> Subject: Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
>>>>
>>>> External email: Use caution opening links or attachments
>>>>
>>>>
>>>> Hi Laurent,
>>>>
>>>> On 1/11/23 10:50, Laurent Vivier wrote:
>>>>> On 1/9/23 11:55, Michael S. Tsirkin wrote:
>>>>>> On Fri, Jan 06, 2023 at 03:21:43PM +0100, Laurent Vivier wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> it seems this patch breaks vhost-user with DPDK.
>>>>>>>
>>>>>>> See
>>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbu
>>>>>>> gzilla.redhat.com%2Fshow_bug.cgi%3Fid%3D2155173&data=05%7C01%7Cyajun
>>>>>>> w%40nvidia.com%7C47e6e0fabd044383fd3308daf47f0253%7C43083d15727340c1
>>>>>>> b7db39efd9ccc17a%7C0%7C0%7C638091123577559319%7CUnknown%7CTWFpbGZsb3
>>>>>>> d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
>>>>>>> %7C3000%7C%7C%7C&sdata=1pjChYTKHVmBoempNitiZHBdrlPIMFjKoD6FeOVSay0%3
>>>>>>> D&reserved=0
>>>>>>>
>>>>>>> it seems QEMU doesn't receive the expected commands sequence:
>>>>>>>
>>>>>>> Received unexpected msg type. Expected 22 received 40 Fail to update
>>>>>>> device iotlb Received unexpected msg type. Expected 40 received 22
>>>>>>> Received unexpected msg type. Expected 22 received 11 Fail to update
>>>>>>> device iotlb Received unexpected msg type. Expected 11 received 22
>>>>>>> vhost VQ 1 ring restore failed: -71: Protocol error (71) Received
>>>>>>> unexpected msg type. Expected 22 received 11 Fail to update device
>>>>>>> iotlb Received unexpected msg type. Expected 11 received 22 vhost VQ
>>>>>>> 0 ring restore failed: -71: Protocol error (71) unable to start
>>>>>>> vhost net: 71: falling back on userspace virtio
>>>>>>>
>>>>>>> It receives VHOST_USER_GET_STATUS (40) when it expects
>>>>>>> VHOST_USER_IOTLB_MSG (22) and VHOST_USER_IOTLB_MSG when it expects
>>>>>>> VHOST_USER_GET_STATUS.
>>>>>>> and VHOST_USER_GET_VRING_BASE (11) when it expect
>>>>>>> VHOST_USER_GET_STATUS and so on.
>>>>>>>
>>>>>>> Any idea?
>>>>
>>>> We only have a single thread on DPDK side to handle Vhost-user requests, it will read a request, handle it and reply to it. Then it reads the next one, etc... So I don't think it is possible to mix request replies order on DPDK side.
>>>>
>>>> Maybe there are two threads concurrently sending requests on QEMU side?
>>>>
>>>> Regards,
>>>> Maxime
>>>>
>>>>>>> Thanks,
>>>>>>> Laurent
>>>>>>
>>>>>>
>>>>>> So I am guessing it's coming from:
>>>>>>
>>>>>>        if (msg.hdr.request != request) {
>>>>>>            error_report("Received unexpected msg type. Expected %d
>>>>>> received %d",
>>>>>>                         request, msg.hdr.request);
>>>>>>            return -EPROTO;
>>>>>>        }
>>>>>>
>>>>>> in process_message_reply and/or in vhost_user_get_u64.
>>>>>>
>>>>>>
>>>>>>> On 11/7/22 23:53, Michael S. Tsirkin wrote:
>>>>>>>> From: Yajun Wu <yajunw@nvidia.com>
>>>>>>>>
>>>>>>>> The motivation of adding vhost-user vhost_dev_start support is to
>>>>>>>> improve backend configuration speed and reduce live migration VM
>>>>>>>> downtime.
>>>>>>>>
>>>>>>>> Today VQ configuration is issued one by one. For virtio net with
>>>>>>>> multi-queue support, backend needs to update RSS (Receive side
>>>>>>>> scaling) on every rx queue enable. Updating RSS is time-consuming
>>>>>>>> (typical time like 7ms).
>>>>>>>>
>>>>>>>> Implement already defined vhost status and message in the vhost
>>>>>>>> specification [1].
>>>>>>>> (a) VHOST_USER_PROTOCOL_F_STATUS
>>>>>>>> (b) VHOST_USER_SET_STATUS
>>>>>>>> (c) VHOST_USER_GET_STATUS
>>>>>>>>
>>>>>>>> Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK
>>>>>>>> for device start and reset(0) for device stop.
>>>>>>>>
>>>>>>>> On reception of the DRIVER_OK message, backend can apply the needed
>>>>>>>> setting only once (instead of incremental) and also utilize
>>>>>>>> parallelism on enabling queues.
>>>>>>>>
>>>>>>>> This improves QEMU's live migration downtime with vhost user
>>>>>>>> backend implementation by great margin, specially for the large
>>>>>>>> number of VQs of 64 from 800 msec to 250 msec.
>>>>>>>>
>>>>>>>> [1]
>>>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fq
>>>>>>>> emu-project.gitlab.io%2Fqemu%2Finterop%2Fvhost-user.html&data=05%7C
>>>>>>>> 01%7Cyajunw%40nvidia.com%7C47e6e0fabd044383fd3308daf47f0253%7C43083
>>>>>>>> d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638091123577559319%7CUnknown%
>>>>>>>> 7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
>>>>>>>> CJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YPbrFRJA92KLLwADMUDvwBt%2Fme2Ef
>>>>>>>> GZuVANOmXH5pic%3D&reserved=0
>>>>>>>>
>>>>>>>> Signed-off-by: Yajun Wu <yajunw@nvidia.com>
>>>>>>>> Acked-by: Parav Pandit <parav@nvidia.com>
>>>>>>>> Message-Id: <20221017064452.1226514-3-yajunw@nvidia.com>
>>>>>>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
>>>>>>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>>>>>>
>>>>>> Probably easiest to debug from dpdk side.
>>>>>> Does the problem go away if you disable the feature
>>>>>> VHOST_USER_PROTOCOL_F_STATUS in dpdk?
>>>>>
>>>>> Maxime could you help to debug this?
>>>>>
>>>>> Thanks,
>>>>> Laurent
>>>>>
>>>>
>>>
>>
>>
> 
> 



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
  2023-01-17 15:07                   ` Maxime Coquelin
@ 2023-01-17 17:55                     ` Greg Kurz
  2023-01-17 18:21                       ` Greg Kurz
  0 siblings, 1 reply; 151+ messages in thread
From: Greg Kurz @ 2023-01-17 17:55 UTC (permalink / raw)
  To: Maxime Coquelin
  Cc: Yajun Wu, Laurent Vivier, Stefan Hajnoczi, qemu-devel,
	Peter Maydell, Parav Pandit, Michael S. Tsirkin,
	Dr. David Alan Gilbert

On Tue, 17 Jan 2023 16:07:00 +0100
Maxime Coquelin <maxime.coquelin@redhat.com> wrote:

> 
> 
> On 1/17/23 13:36, Greg Kurz wrote:
> > On Tue, 17 Jan 2023 13:12:57 +0100
> > Greg Kurz <groug@kaod.org> wrote:
> > 
> >> Hi Maxime,
> >>
> >> On Tue, 17 Jan 2023 10:49:37 +0100
> >> Maxime Coquelin <maxime.coquelin@redhat.com> wrote:
> >>
> >>> Hi Yajun,
> >>>
> >>> On 1/16/23 08:14, Yajun Wu wrote:
> >>>> Not quite sure about the whole picture.
> >>>>
> >>>> Seems while qemu waiting response of vhost_user_get_status, dpdk send out VHOST_USER_SLAVE_IOTLB_MSG and trigger qemu function vhost_backend_update_device_iotlb.
> >>>> Qemu wait on reply of VHOST_USER_IOTLB_MSG but get VHOST_USER_GET_STATUS reply.
> >>>
> >>> Thanks for the backtrace, that helps a lot.
> >>>
> >>> The issue happens because:
> >>>    1. Introduction of nested event loop in vhost_user_read() [0] features
> >>> that enables handling slave channel request while waiting for reply on
> >>> the masyer channel.
> >>>    2. Slave VHOST_USER_SLAVE_IOTLB_MSG slave request handling ends-up
> >>> sending a VHOST_USER_IOTLB_MSG on the master channel.
> >>>
> >>> So while waiting for VHOST_USER_IOTLB_MSG reply, it receives the reply
> >>> for the first request sent on the master channel, here the
> >>> VHOST_USER_GET_STATUS reply.
> >>>
> >>> I don't see an easy way to fix it.
> >>>
> >>> One option would be to have the slave channel being handled by another
> >>> thread, and protect master channel with a lock to enforce the
> >>> synchronization. But this may induce other issues, so that's not a light
> >>> change.
> >>>
> >>
> >> This is going to be tough because the back-end might have set the
> >> VHOST_USER_NEED_REPLY_MASK flag on the VHOST_USER_SLAVE_IOTLB_MSG
> >> request and thus might be waiting for a response on the slave
> >> channel. In order to emit such a response, the front-end must
> >> send VHOST_USER_SLAVE_IOTLB_MSG updates on the master channel *first*
> >> according to the protocol specification. This means that we really
> >> cannot handle VHOST_USER_SLAVE_IOTLB_MSG requests while there's
> >> an on-going transaction on the master channel.
> 
> Since the slave channel would be handled on another thread, it means the
> on-going transaction on the master channel can continue. Once done, it
> will release the mutex, and the thread handling the slave channel can
> take it send the IOTLB update on the master channel.
> 
> That would work with DPDK, which does not request reply-ack on IOTLB
> misses.
> 

Not sure to catch what would happen if DPDK requested a reply-ack with
this scenario.

> For the DAX enablement case, my understanding is that the handling of
> the slave requests by QEMU does not induce sending requests on the
> master channel, so if I'm not mistaken, it should work too. The thread
> handling the slave requests can send the reply on the slave channel,
> while the thread handling the master channel is blocked waiting for the
> GET_VRING_BASE reply. Is that correct?
> 

Yes this is correct AFAICT. Dropping the nested loop (commit db8a3772e3
reverts like a charm) and having the slave channel serviced by its own
thread seems to be the way to go then.

> >>
> >>> (Adding Greg and Stefan, who worked on the nested event loop series.)
> >>>
> >>> Simply reverting nested event loop support may not be an option, since
> >>> it would break virtiofsd, as if my understanding is correct, waits for
> >>> some slave channel request to complete in order to complete a request
> >>> made by QEMU on the master channel.
> >>>
> >>> Any thougths?
> >>>
> >>
> >> Well... the nested even loop was added as preparatory work for "the
> >> upcoming enablement of DAX with virtio-fs". This requires changes on
> >> the QEMU side that haven't been merged yet. Technically, it seems that
> >> reverting the nested event loop won't break anything in upstream QEMU
> >> at this point (but this will bite again as soon as DAX enablement gets
> >> merged).
> >>
> > 
> > Cc'ing Dave to know about the DAX enablement status.
> > 
> >> AFAIK the event loop is only needed for the VHOST_USER_GET_VRING_BASE
> >> message. Another possibility might be to create the nested event loop
> >> in this case only : this would allow VHOST_USER_GET_STATUS to complete
> >> before QEMU starts processing the VHOST_USER_SLAVE_IOTLB_MSG requests.
> >>
> >> Cheers,
> >>
> >> --
> >> Greg
> >>
> >>> Maxime
> >>>
> >>> [0]:
> >>> https://patchwork.ozlabs.org/project/qemu-devel/patch/20210312092212.782255-6-groug@kaod.org/
> >>>
> >>>
> >>>> Break on first error message("Received unexpected msg type. Expected 22 received 40")
> >>>>
> >>>> #0  0x0000555555b72ed4 in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffa330) at ../hw/virtio/vhost-user.c:445
> >>>> #1  0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffa600) at ../hw/virtio/vhost-user.c:2341
> >>>> #2  0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10442706944, uaddr=140736119902208, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> >>>> #3  0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10442706944, write=1) at ../hw/virtio/vhost.c:1113
> >>>> #4  0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffa7b0) at ../hw/virtio/vhost-backend.c:393
> >>>> #5  0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> >>>> #6  0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556a06fb0, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> >>>> #7  0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> >>>> #8  0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> >>>> #9  0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> >>>> #10 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffac50) at ../hw/virtio/vhost-user.c:413
> >>>> #11 0x0000555555b72e9b in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffaf10) at ../hw/virtio/vhost-user.c:439
> >>>> #12 0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffb1e0) at ../hw/virtio/vhost-user.c:2341
> >>>> #13 0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10468392960, uaddr=140736145588224, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> >>>> #14 0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10468392960, write=1) at ../hw/virtio/vhost.c:1113
> >>>> #15 0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffb390) at ../hw/virtio/vhost-backend.c:393
> >>>> #16 0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> >>>> #17 0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556c70250, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> >>>> #18 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> >>>> #19 0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> >>>> #20 0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> >>>> #21 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffb830) at ../hw/virtio/vhost-user.c:413
> >>>> #22 0x0000555555b72e9b in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffbaf0) at ../hw/virtio/vhost-user.c:439
> >>>> #23 0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffbdc0) at ../hw/virtio/vhost-user.c:2341
> >>>> #24 0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10442702848, uaddr=140736119898112, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> >>>> #25 0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10442702848, write=1) at ../hw/virtio/vhost.c:1113
> >>>> #26 0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffbf70) at ../hw/virtio/vhost-backend.c:393
> >>>> #27 0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> >>>> #28 0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556f1a530, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> >>>> #29 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> >>>> #30 0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> >>>> #31 0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> >>>> #32 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffc420) at ../hw/virtio/vhost-user.c:413
> >>>> #33 0x0000555555b754b1 in vhost_user_get_u64 (dev=0x5555584dd600, request=40, u64=0x7fffffffc6e0) at ../hw/virtio/vhost-user.c:1349
> >>>> #34 0x0000555555b758ff in vhost_user_get_status (dev=0x5555584dd600, status=0x7fffffffc713 "W\020") at ../hw/virtio/vhost-user.c:1474
> >>>> #35 0x0000555555b75967 in vhost_user_add_status (dev=0x5555584dd600, status=7 '\a') at ../hw/virtio/vhost-user.c:1488
> >>>> #36 0x0000555555b78bf6 in vhost_user_dev_start (dev=0x5555584dd600, started=true) at ../hw/virtio/vhost-user.c:2758
> >>>> #37 0x0000555555b709ad in vhost_dev_start (hdev=0x5555584dd600, vdev=0x555557b965d0, vrings=false) at ../hw/virtio/vhost.c:1988
> >>>> #38 0x000055555584291c in vhost_net_start_one (net=0x5555584dd600, dev=0x555557b965d0) at ../hw/net/vhost_net.c:271
> >>>> #39 0x0000555555842f1e in vhost_net_start (dev=0x555557b965d0, ncs=0x555557bc09e0, data_queue_pairs=1, cvq=0) at ../hw/net/vhost_net.c:412
> >>>> #40 0x0000555555b1bf61 in virtio_net_vhost_status (n=0x555557b965d0, status=15 '\017') at ../hw/net/virtio-net.c:311
> >>>> #41 0x0000555555b1c20c in virtio_net_set_status (vdev=0x555557b965d0, status=15 '\017') at ../hw/net/virtio-net.c:392
> >>>> #42 0x0000555555b1ed04 in virtio_net_handle_mq (n=0x555557b965d0, cmd=0 '\000', iov=0x555556c7ef50, iov_cnt=1) at ../hw/net/virtio-net.c:1497
> >>>> #43 0x0000555555b1eef0 in virtio_net_handle_ctrl_iov (vdev=0x555557b965d0, in_sg=0x555556a09880, in_num=1, out_sg=0x555556a09890, out_num=1) at ../hw/net/virtio-net.c:1534
> >>>> #44 0x0000555555b1efe9 in virtio_net_handle_ctrl (vdev=0x555557b965d0, vq=0x7fffc04ac140) at ../hw/net/virtio-net.c:1557
> >>>> #45 0x0000555555b63776 in virtio_queue_notify_vq (vq=0x7fffc04ac140) at ../hw/virtio/virtio.c:2249
> >>>> #46 0x0000555555b669dc in virtio_queue_host_notifier_read (n=0x7fffc04ac1b4) at ../hw/virtio/virtio.c:3529
> >>>> #47 0x0000555555e3f458 in aio_dispatch_handler (ctx=0x555556a016c0, node=0x7ffd8800e430) at ../util/aio-posix.c:369
> >>>> #48 0x0000555555e3f613 in aio_dispatch_handlers (ctx=0x555556a016c0) at ../util/aio-posix.c:412
> >>>> #49 0x0000555555e3f669 in aio_dispatch (ctx=0x555556a016c0) at ../util/aio-posix.c:422
> >>>> #50 0x0000555555e585de in aio_ctx_dispatch (source=0x555556a016c0, callback=0x0, user_data=0x0) at ../util/async.c:321
> >>>> #51 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> >>>> #52 0x0000555555e5abea in glib_pollfds_poll () at ../util/main-loop.c:295
> >>>> #53 0x0000555555e5ac64 in os_host_main_loop_wait (timeout=0) at ../util/main-loop.c:318
> >>>> #54 0x0000555555e5ad69 in main_loop_wait (nonblocking=0) at ../util/main-loop.c:604
> >>>> #55 0x00005555559693de in qemu_main_loop () at ../softmmu/runstate.c:731
> >>>> #56 0x00005555556e7c06 in qemu_default_main () at ../softmmu/main.c:37
> >>>> #57 0x00005555556e7c3c in main (argc=71, argv=0x7fffffffcda8) at ../softmmu/main.c:48
> >>>>
> >>>>
> >>>>
> >>>> -----Original Message-----
> >>>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> >>>> Sent: Thursday, January 12, 2023 5:26 PM
> >>>> To: Laurent Vivier <lvivier@redhat.com>
> >>>> Cc: qemu-devel@nongnu.org; Peter Maydell <peter.maydell@linaro.org>; Yajun Wu <yajunw@nvidia.com>; Parav Pandit <parav@nvidia.com>; Michael S. Tsirkin <mst@redhat.com>
> >>>> Subject: Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
> >>>>
> >>>> External email: Use caution opening links or attachments
> >>>>
> >>>>
> >>>> Hi Laurent,
> >>>>
> >>>> On 1/11/23 10:50, Laurent Vivier wrote:
> >>>>> On 1/9/23 11:55, Michael S. Tsirkin wrote:
> >>>>>> On Fri, Jan 06, 2023 at 03:21:43PM +0100, Laurent Vivier wrote:
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> it seems this patch breaks vhost-user with DPDK.
> >>>>>>>
> >>>>>>> See
> >>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbu
> >>>>>>> gzilla.redhat.com%2Fshow_bug.cgi%3Fid%3D2155173&data=05%7C01%7Cyajun
> >>>>>>> w%40nvidia.com%7C47e6e0fabd044383fd3308daf47f0253%7C43083d15727340c1
> >>>>>>> b7db39efd9ccc17a%7C0%7C0%7C638091123577559319%7CUnknown%7CTWFpbGZsb3
> >>>>>>> d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
> >>>>>>> %7C3000%7C%7C%7C&sdata=1pjChYTKHVmBoempNitiZHBdrlPIMFjKoD6FeOVSay0%3
> >>>>>>> D&reserved=0
> >>>>>>>
> >>>>>>> it seems QEMU doesn't receive the expected commands sequence:
> >>>>>>>
> >>>>>>> Received unexpected msg type. Expected 22 received 40 Fail to update
> >>>>>>> device iotlb Received unexpected msg type. Expected 40 received 22
> >>>>>>> Received unexpected msg type. Expected 22 received 11 Fail to update
> >>>>>>> device iotlb Received unexpected msg type. Expected 11 received 22
> >>>>>>> vhost VQ 1 ring restore failed: -71: Protocol error (71) Received
> >>>>>>> unexpected msg type. Expected 22 received 11 Fail to update device
> >>>>>>> iotlb Received unexpected msg type. Expected 11 received 22 vhost VQ
> >>>>>>> 0 ring restore failed: -71: Protocol error (71) unable to start
> >>>>>>> vhost net: 71: falling back on userspace virtio
> >>>>>>>
> >>>>>>> It receives VHOST_USER_GET_STATUS (40) when it expects
> >>>>>>> VHOST_USER_IOTLB_MSG (22) and VHOST_USER_IOTLB_MSG when it expects
> >>>>>>> VHOST_USER_GET_STATUS.
> >>>>>>> and VHOST_USER_GET_VRING_BASE (11) when it expect
> >>>>>>> VHOST_USER_GET_STATUS and so on.
> >>>>>>>
> >>>>>>> Any idea?
> >>>>
> >>>> We only have a single thread on DPDK side to handle Vhost-user requests, it will read a request, handle it and reply to it. Then it reads the next one, etc... So I don't think it is possible to mix request replies order on DPDK side.
> >>>>
> >>>> Maybe there are two threads concurrently sending requests on QEMU side?
> >>>>
> >>>> Regards,
> >>>> Maxime
> >>>>
> >>>>>>> Thanks,
> >>>>>>> Laurent
> >>>>>>
> >>>>>>
> >>>>>> So I am guessing it's coming from:
> >>>>>>
> >>>>>>        if (msg.hdr.request != request) {
> >>>>>>            error_report("Received unexpected msg type. Expected %d
> >>>>>> received %d",
> >>>>>>                         request, msg.hdr.request);
> >>>>>>            return -EPROTO;
> >>>>>>        }
> >>>>>>
> >>>>>> in process_message_reply and/or in vhost_user_get_u64.
> >>>>>>
> >>>>>>
> >>>>>>> On 11/7/22 23:53, Michael S. Tsirkin wrote:
> >>>>>>>> From: Yajun Wu <yajunw@nvidia.com>
> >>>>>>>>
> >>>>>>>> The motivation of adding vhost-user vhost_dev_start support is to
> >>>>>>>> improve backend configuration speed and reduce live migration VM
> >>>>>>>> downtime.
> >>>>>>>>
> >>>>>>>> Today VQ configuration is issued one by one. For virtio net with
> >>>>>>>> multi-queue support, backend needs to update RSS (Receive side
> >>>>>>>> scaling) on every rx queue enable. Updating RSS is time-consuming
> >>>>>>>> (typical time like 7ms).
> >>>>>>>>
> >>>>>>>> Implement already defined vhost status and message in the vhost
> >>>>>>>> specification [1].
> >>>>>>>> (a) VHOST_USER_PROTOCOL_F_STATUS
> >>>>>>>> (b) VHOST_USER_SET_STATUS
> >>>>>>>> (c) VHOST_USER_GET_STATUS
> >>>>>>>>
> >>>>>>>> Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK
> >>>>>>>> for device start and reset(0) for device stop.
> >>>>>>>>
> >>>>>>>> On reception of the DRIVER_OK message, backend can apply the needed
> >>>>>>>> setting only once (instead of incremental) and also utilize
> >>>>>>>> parallelism on enabling queues.
> >>>>>>>>
> >>>>>>>> This improves QEMU's live migration downtime with vhost user
> >>>>>>>> backend implementation by great margin, specially for the large
> >>>>>>>> number of VQs of 64 from 800 msec to 250 msec.
> >>>>>>>>
> >>>>>>>> [1]
> >>>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fq
> >>>>>>>> emu-project.gitlab.io%2Fqemu%2Finterop%2Fvhost-user.html&data=05%7C
> >>>>>>>> 01%7Cyajunw%40nvidia.com%7C47e6e0fabd044383fd3308daf47f0253%7C43083
> >>>>>>>> d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638091123577559319%7CUnknown%
> >>>>>>>> 7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
> >>>>>>>> CJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YPbrFRJA92KLLwADMUDvwBt%2Fme2Ef
> >>>>>>>> GZuVANOmXH5pic%3D&reserved=0
> >>>>>>>>
> >>>>>>>> Signed-off-by: Yajun Wu <yajunw@nvidia.com>
> >>>>>>>> Acked-by: Parav Pandit <parav@nvidia.com>
> >>>>>>>> Message-Id: <20221017064452.1226514-3-yajunw@nvidia.com>
> >>>>>>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> >>>>>>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> >>>>>>
> >>>>>> Probably easiest to debug from dpdk side.
> >>>>>> Does the problem go away if you disable the feature
> >>>>>> VHOST_USER_PROTOCOL_F_STATUS in dpdk?
> >>>>>
> >>>>> Maxime could you help to debug this?
> >>>>>
> >>>>> Thanks,
> >>>>> Laurent
> >>>>>
> >>>>
> >>>
> >>
> >>
> > 
> > 
> 



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
  2023-01-17 17:55                     ` Greg Kurz
@ 2023-01-17 18:21                       ` Greg Kurz
  2023-01-18 11:01                         ` Michael S. Tsirkin
  0 siblings, 1 reply; 151+ messages in thread
From: Greg Kurz @ 2023-01-17 18:21 UTC (permalink / raw)
  To: Maxime Coquelin
  Cc: Yajun Wu, Laurent Vivier, Stefan Hajnoczi, qemu-devel,
	Peter Maydell, Parav Pandit, Michael S. Tsirkin,
	Dr. David Alan Gilbert

On Tue, 17 Jan 2023 18:55:24 +0100
Greg Kurz <groug@kaod.org> wrote:

> On Tue, 17 Jan 2023 16:07:00 +0100
> Maxime Coquelin <maxime.coquelin@redhat.com> wrote:
> 
> > 
> > 
> > On 1/17/23 13:36, Greg Kurz wrote:
> > > On Tue, 17 Jan 2023 13:12:57 +0100
> > > Greg Kurz <groug@kaod.org> wrote:
> > > 
> > >> Hi Maxime,
> > >>
> > >> On Tue, 17 Jan 2023 10:49:37 +0100
> > >> Maxime Coquelin <maxime.coquelin@redhat.com> wrote:
> > >>
> > >>> Hi Yajun,
> > >>>
> > >>> On 1/16/23 08:14, Yajun Wu wrote:
> > >>>> Not quite sure about the whole picture.
> > >>>>
> > >>>> Seems while qemu waiting response of vhost_user_get_status, dpdk send out VHOST_USER_SLAVE_IOTLB_MSG and trigger qemu function vhost_backend_update_device_iotlb.
> > >>>> Qemu wait on reply of VHOST_USER_IOTLB_MSG but get VHOST_USER_GET_STATUS reply.
> > >>>
> > >>> Thanks for the backtrace, that helps a lot.
> > >>>
> > >>> The issue happens because:
> > >>>    1. Introduction of nested event loop in vhost_user_read() [0] features
> > >>> that enables handling slave channel request while waiting for reply on
> > >>> the masyer channel.
> > >>>    2. Slave VHOST_USER_SLAVE_IOTLB_MSG slave request handling ends-up
> > >>> sending a VHOST_USER_IOTLB_MSG on the master channel.
> > >>>
> > >>> So while waiting for VHOST_USER_IOTLB_MSG reply, it receives the reply
> > >>> for the first request sent on the master channel, here the
> > >>> VHOST_USER_GET_STATUS reply.
> > >>>
> > >>> I don't see an easy way to fix it.
> > >>>
> > >>> One option would be to have the slave channel being handled by another
> > >>> thread, and protect master channel with a lock to enforce the
> > >>> synchronization. But this may induce other issues, so that's not a light
> > >>> change.
> > >>>
> > >>
> > >> This is going to be tough because the back-end might have set the
> > >> VHOST_USER_NEED_REPLY_MASK flag on the VHOST_USER_SLAVE_IOTLB_MSG
> > >> request and thus might be waiting for a response on the slave
> > >> channel. In order to emit such a response, the front-end must
> > >> send VHOST_USER_SLAVE_IOTLB_MSG updates on the master channel *first*
> > >> according to the protocol specification. This means that we really
> > >> cannot handle VHOST_USER_SLAVE_IOTLB_MSG requests while there's
> > >> an on-going transaction on the master channel.
> > 
> > Since the slave channel would be handled on another thread, it means the
> > on-going transaction on the master channel can continue. Once done, it
> > will release the mutex, and the thread handling the slave channel can
> > take it send the IOTLB update on the master channel.
> > 
> > That would work with DPDK, which does not request reply-ack on IOTLB
> > misses.
> > 
> 
> Not sure to catch what would happen if DPDK requested a reply-ack with
> this scenario.
> 
> > For the DAX enablement case, my understanding is that the handling of
> > the slave requests by QEMU does not induce sending requests on the
> > master channel, so if I'm not mistaken, it should work too. The thread
> > handling the slave requests can send the reply on the slave channel,
> > while the thread handling the master channel is blocked waiting for the
> > GET_VRING_BASE reply. Is that correct?
> > 
> 
> Yes this is correct AFAICT. Dropping the nested loop (commit db8a3772e3
> reverts like a charm) and having the slave channel serviced by its own
> thread seems to be the way to go then.
> 

Commit a7f523c7d114d needs to be reverted as well. There are some conflicts
because of commit 025faa872bcf9 but they seem trivial to fix.

> > >>
> > >>> (Adding Greg and Stefan, who worked on the nested event loop series.)
> > >>>
> > >>> Simply reverting nested event loop support may not be an option, since
> > >>> it would break virtiofsd, as if my understanding is correct, waits for
> > >>> some slave channel request to complete in order to complete a request
> > >>> made by QEMU on the master channel.
> > >>>
> > >>> Any thougths?
> > >>>
> > >>
> > >> Well... the nested even loop was added as preparatory work for "the
> > >> upcoming enablement of DAX with virtio-fs". This requires changes on
> > >> the QEMU side that haven't been merged yet. Technically, it seems that
> > >> reverting the nested event loop won't break anything in upstream QEMU
> > >> at this point (but this will bite again as soon as DAX enablement gets
> > >> merged).
> > >>
> > > 
> > > Cc'ing Dave to know about the DAX enablement status.
> > > 
> > >> AFAIK the event loop is only needed for the VHOST_USER_GET_VRING_BASE
> > >> message. Another possibility might be to create the nested event loop
> > >> in this case only : this would allow VHOST_USER_GET_STATUS to complete
> > >> before QEMU starts processing the VHOST_USER_SLAVE_IOTLB_MSG requests.
> > >>
> > >> Cheers,
> > >>
> > >> --
> > >> Greg
> > >>
> > >>> Maxime
> > >>>
> > >>> [0]:
> > >>> https://patchwork.ozlabs.org/project/qemu-devel/patch/20210312092212.782255-6-groug@kaod.org/
> > >>>
> > >>>
> > >>>> Break on first error message("Received unexpected msg type. Expected 22 received 40")
> > >>>>
> > >>>> #0  0x0000555555b72ed4 in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffa330) at ../hw/virtio/vhost-user.c:445
> > >>>> #1  0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffa600) at ../hw/virtio/vhost-user.c:2341
> > >>>> #2  0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10442706944, uaddr=140736119902208, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> > >>>> #3  0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10442706944, write=1) at ../hw/virtio/vhost.c:1113
> > >>>> #4  0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffa7b0) at ../hw/virtio/vhost-backend.c:393
> > >>>> #5  0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> > >>>> #6  0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556a06fb0, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> > >>>> #7  0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > >>>> #8  0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> > >>>> #9  0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> > >>>> #10 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffac50) at ../hw/virtio/vhost-user.c:413
> > >>>> #11 0x0000555555b72e9b in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffaf10) at ../hw/virtio/vhost-user.c:439
> > >>>> #12 0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffb1e0) at ../hw/virtio/vhost-user.c:2341
> > >>>> #13 0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10468392960, uaddr=140736145588224, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> > >>>> #14 0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10468392960, write=1) at ../hw/virtio/vhost.c:1113
> > >>>> #15 0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffb390) at ../hw/virtio/vhost-backend.c:393
> > >>>> #16 0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> > >>>> #17 0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556c70250, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> > >>>> #18 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > >>>> #19 0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> > >>>> #20 0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> > >>>> #21 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffb830) at ../hw/virtio/vhost-user.c:413
> > >>>> #22 0x0000555555b72e9b in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffbaf0) at ../hw/virtio/vhost-user.c:439
> > >>>> #23 0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffbdc0) at ../hw/virtio/vhost-user.c:2341
> > >>>> #24 0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10442702848, uaddr=140736119898112, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> > >>>> #25 0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10442702848, write=1) at ../hw/virtio/vhost.c:1113
> > >>>> #26 0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffbf70) at ../hw/virtio/vhost-backend.c:393
> > >>>> #27 0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> > >>>> #28 0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556f1a530, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> > >>>> #29 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > >>>> #30 0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> > >>>> #31 0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> > >>>> #32 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffc420) at ../hw/virtio/vhost-user.c:413
> > >>>> #33 0x0000555555b754b1 in vhost_user_get_u64 (dev=0x5555584dd600, request=40, u64=0x7fffffffc6e0) at ../hw/virtio/vhost-user.c:1349
> > >>>> #34 0x0000555555b758ff in vhost_user_get_status (dev=0x5555584dd600, status=0x7fffffffc713 "W\020") at ../hw/virtio/vhost-user.c:1474
> > >>>> #35 0x0000555555b75967 in vhost_user_add_status (dev=0x5555584dd600, status=7 '\a') at ../hw/virtio/vhost-user.c:1488
> > >>>> #36 0x0000555555b78bf6 in vhost_user_dev_start (dev=0x5555584dd600, started=true) at ../hw/virtio/vhost-user.c:2758
> > >>>> #37 0x0000555555b709ad in vhost_dev_start (hdev=0x5555584dd600, vdev=0x555557b965d0, vrings=false) at ../hw/virtio/vhost.c:1988
> > >>>> #38 0x000055555584291c in vhost_net_start_one (net=0x5555584dd600, dev=0x555557b965d0) at ../hw/net/vhost_net.c:271
> > >>>> #39 0x0000555555842f1e in vhost_net_start (dev=0x555557b965d0, ncs=0x555557bc09e0, data_queue_pairs=1, cvq=0) at ../hw/net/vhost_net.c:412
> > >>>> #40 0x0000555555b1bf61 in virtio_net_vhost_status (n=0x555557b965d0, status=15 '\017') at ../hw/net/virtio-net.c:311
> > >>>> #41 0x0000555555b1c20c in virtio_net_set_status (vdev=0x555557b965d0, status=15 '\017') at ../hw/net/virtio-net.c:392
> > >>>> #42 0x0000555555b1ed04 in virtio_net_handle_mq (n=0x555557b965d0, cmd=0 '\000', iov=0x555556c7ef50, iov_cnt=1) at ../hw/net/virtio-net.c:1497
> > >>>> #43 0x0000555555b1eef0 in virtio_net_handle_ctrl_iov (vdev=0x555557b965d0, in_sg=0x555556a09880, in_num=1, out_sg=0x555556a09890, out_num=1) at ../hw/net/virtio-net.c:1534
> > >>>> #44 0x0000555555b1efe9 in virtio_net_handle_ctrl (vdev=0x555557b965d0, vq=0x7fffc04ac140) at ../hw/net/virtio-net.c:1557
> > >>>> #45 0x0000555555b63776 in virtio_queue_notify_vq (vq=0x7fffc04ac140) at ../hw/virtio/virtio.c:2249
> > >>>> #46 0x0000555555b669dc in virtio_queue_host_notifier_read (n=0x7fffc04ac1b4) at ../hw/virtio/virtio.c:3529
> > >>>> #47 0x0000555555e3f458 in aio_dispatch_handler (ctx=0x555556a016c0, node=0x7ffd8800e430) at ../util/aio-posix.c:369
> > >>>> #48 0x0000555555e3f613 in aio_dispatch_handlers (ctx=0x555556a016c0) at ../util/aio-posix.c:412
> > >>>> #49 0x0000555555e3f669 in aio_dispatch (ctx=0x555556a016c0) at ../util/aio-posix.c:422
> > >>>> #50 0x0000555555e585de in aio_ctx_dispatch (source=0x555556a016c0, callback=0x0, user_data=0x0) at ../util/async.c:321
> > >>>> #51 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > >>>> #52 0x0000555555e5abea in glib_pollfds_poll () at ../util/main-loop.c:295
> > >>>> #53 0x0000555555e5ac64 in os_host_main_loop_wait (timeout=0) at ../util/main-loop.c:318
> > >>>> #54 0x0000555555e5ad69 in main_loop_wait (nonblocking=0) at ../util/main-loop.c:604
> > >>>> #55 0x00005555559693de in qemu_main_loop () at ../softmmu/runstate.c:731
> > >>>> #56 0x00005555556e7c06 in qemu_default_main () at ../softmmu/main.c:37
> > >>>> #57 0x00005555556e7c3c in main (argc=71, argv=0x7fffffffcda8) at ../softmmu/main.c:48
> > >>>>
> > >>>>
> > >>>>
> > >>>> -----Original Message-----
> > >>>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> > >>>> Sent: Thursday, January 12, 2023 5:26 PM
> > >>>> To: Laurent Vivier <lvivier@redhat.com>
> > >>>> Cc: qemu-devel@nongnu.org; Peter Maydell <peter.maydell@linaro.org>; Yajun Wu <yajunw@nvidia.com>; Parav Pandit <parav@nvidia.com>; Michael S. Tsirkin <mst@redhat.com>
> > >>>> Subject: Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
> > >>>>
> > >>>> External email: Use caution opening links or attachments
> > >>>>
> > >>>>
> > >>>> Hi Laurent,
> > >>>>
> > >>>> On 1/11/23 10:50, Laurent Vivier wrote:
> > >>>>> On 1/9/23 11:55, Michael S. Tsirkin wrote:
> > >>>>>> On Fri, Jan 06, 2023 at 03:21:43PM +0100, Laurent Vivier wrote:
> > >>>>>>> Hi,
> > >>>>>>>
> > >>>>>>> it seems this patch breaks vhost-user with DPDK.
> > >>>>>>>
> > >>>>>>> See
> > >>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbu
> > >>>>>>> gzilla.redhat.com%2Fshow_bug.cgi%3Fid%3D2155173&data=05%7C01%7Cyajun
> > >>>>>>> w%40nvidia.com%7C47e6e0fabd044383fd3308daf47f0253%7C43083d15727340c1
> > >>>>>>> b7db39efd9ccc17a%7C0%7C0%7C638091123577559319%7CUnknown%7CTWFpbGZsb3
> > >>>>>>> d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
> > >>>>>>> %7C3000%7C%7C%7C&sdata=1pjChYTKHVmBoempNitiZHBdrlPIMFjKoD6FeOVSay0%3
> > >>>>>>> D&reserved=0
> > >>>>>>>
> > >>>>>>> it seems QEMU doesn't receive the expected commands sequence:
> > >>>>>>>
> > >>>>>>> Received unexpected msg type. Expected 22 received 40 Fail to update
> > >>>>>>> device iotlb Received unexpected msg type. Expected 40 received 22
> > >>>>>>> Received unexpected msg type. Expected 22 received 11 Fail to update
> > >>>>>>> device iotlb Received unexpected msg type. Expected 11 received 22
> > >>>>>>> vhost VQ 1 ring restore failed: -71: Protocol error (71) Received
> > >>>>>>> unexpected msg type. Expected 22 received 11 Fail to update device
> > >>>>>>> iotlb Received unexpected msg type. Expected 11 received 22 vhost VQ
> > >>>>>>> 0 ring restore failed: -71: Protocol error (71) unable to start
> > >>>>>>> vhost net: 71: falling back on userspace virtio
> > >>>>>>>
> > >>>>>>> It receives VHOST_USER_GET_STATUS (40) when it expects
> > >>>>>>> VHOST_USER_IOTLB_MSG (22) and VHOST_USER_IOTLB_MSG when it expects
> > >>>>>>> VHOST_USER_GET_STATUS.
> > >>>>>>> and VHOST_USER_GET_VRING_BASE (11) when it expect
> > >>>>>>> VHOST_USER_GET_STATUS and so on.
> > >>>>>>>
> > >>>>>>> Any idea?
> > >>>>
> > >>>> We only have a single thread on DPDK side to handle Vhost-user requests, it will read a request, handle it and reply to it. Then it reads the next one, etc... So I don't think it is possible to mix request replies order on DPDK side.
> > >>>>
> > >>>> Maybe there are two threads concurrently sending requests on QEMU side?
> > >>>>
> > >>>> Regards,
> > >>>> Maxime
> > >>>>
> > >>>>>>> Thanks,
> > >>>>>>> Laurent
> > >>>>>>
> > >>>>>>
> > >>>>>> So I am guessing it's coming from:
> > >>>>>>
> > >>>>>>        if (msg.hdr.request != request) {
> > >>>>>>            error_report("Received unexpected msg type. Expected %d
> > >>>>>> received %d",
> > >>>>>>                         request, msg.hdr.request);
> > >>>>>>            return -EPROTO;
> > >>>>>>        }
> > >>>>>>
> > >>>>>> in process_message_reply and/or in vhost_user_get_u64.
> > >>>>>>
> > >>>>>>
> > >>>>>>> On 11/7/22 23:53, Michael S. Tsirkin wrote:
> > >>>>>>>> From: Yajun Wu <yajunw@nvidia.com>
> > >>>>>>>>
> > >>>>>>>> The motivation of adding vhost-user vhost_dev_start support is to
> > >>>>>>>> improve backend configuration speed and reduce live migration VM
> > >>>>>>>> downtime.
> > >>>>>>>>
> > >>>>>>>> Today VQ configuration is issued one by one. For virtio net with
> > >>>>>>>> multi-queue support, backend needs to update RSS (Receive side
> > >>>>>>>> scaling) on every rx queue enable. Updating RSS is time-consuming
> > >>>>>>>> (typical time like 7ms).
> > >>>>>>>>
> > >>>>>>>> Implement already defined vhost status and message in the vhost
> > >>>>>>>> specification [1].
> > >>>>>>>> (a) VHOST_USER_PROTOCOL_F_STATUS
> > >>>>>>>> (b) VHOST_USER_SET_STATUS
> > >>>>>>>> (c) VHOST_USER_GET_STATUS
> > >>>>>>>>
> > >>>>>>>> Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK
> > >>>>>>>> for device start and reset(0) for device stop.
> > >>>>>>>>
> > >>>>>>>> On reception of the DRIVER_OK message, backend can apply the needed
> > >>>>>>>> setting only once (instead of incremental) and also utilize
> > >>>>>>>> parallelism on enabling queues.
> > >>>>>>>>
> > >>>>>>>> This improves QEMU's live migration downtime with vhost user
> > >>>>>>>> backend implementation by great margin, specially for the large
> > >>>>>>>> number of VQs of 64 from 800 msec to 250 msec.
> > >>>>>>>>
> > >>>>>>>> [1]
> > >>>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fq
> > >>>>>>>> emu-project.gitlab.io%2Fqemu%2Finterop%2Fvhost-user.html&data=05%7C
> > >>>>>>>> 01%7Cyajunw%40nvidia.com%7C47e6e0fabd044383fd3308daf47f0253%7C43083
> > >>>>>>>> d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638091123577559319%7CUnknown%
> > >>>>>>>> 7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
> > >>>>>>>> CJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YPbrFRJA92KLLwADMUDvwBt%2Fme2Ef
> > >>>>>>>> GZuVANOmXH5pic%3D&reserved=0
> > >>>>>>>>
> > >>>>>>>> Signed-off-by: Yajun Wu <yajunw@nvidia.com>
> > >>>>>>>> Acked-by: Parav Pandit <parav@nvidia.com>
> > >>>>>>>> Message-Id: <20221017064452.1226514-3-yajunw@nvidia.com>
> > >>>>>>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > >>>>>>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > >>>>>>
> > >>>>>> Probably easiest to debug from dpdk side.
> > >>>>>> Does the problem go away if you disable the feature
> > >>>>>> VHOST_USER_PROTOCOL_F_STATUS in dpdk?
> > >>>>>
> > >>>>> Maxime could you help to debug this?
> > >>>>>
> > >>>>> Thanks,
> > >>>>> Laurent
> > >>>>>
> > >>>>
> > >>>
> > >>
> > >>
> > > 
> > > 
> > 
> 
> 



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
  2023-01-17 18:21                       ` Greg Kurz
@ 2023-01-18 11:01                         ` Michael S. Tsirkin
  0 siblings, 0 replies; 151+ messages in thread
From: Michael S. Tsirkin @ 2023-01-18 11:01 UTC (permalink / raw)
  To: Greg Kurz
  Cc: Maxime Coquelin, Yajun Wu, Laurent Vivier, Stefan Hajnoczi,
	qemu-devel, Peter Maydell, Parav Pandit, Dr. David Alan Gilbert

On Tue, Jan 17, 2023 at 07:21:28PM +0100, Greg Kurz wrote:
> On Tue, 17 Jan 2023 18:55:24 +0100
> Greg Kurz <groug@kaod.org> wrote:
> 
> > On Tue, 17 Jan 2023 16:07:00 +0100
> > Maxime Coquelin <maxime.coquelin@redhat.com> wrote:
> > 
> > > 
> > > 
> > > On 1/17/23 13:36, Greg Kurz wrote:
> > > > On Tue, 17 Jan 2023 13:12:57 +0100
> > > > Greg Kurz <groug@kaod.org> wrote:
> > > > 
> > > >> Hi Maxime,
> > > >>
> > > >> On Tue, 17 Jan 2023 10:49:37 +0100
> > > >> Maxime Coquelin <maxime.coquelin@redhat.com> wrote:
> > > >>
> > > >>> Hi Yajun,
> > > >>>
> > > >>> On 1/16/23 08:14, Yajun Wu wrote:
> > > >>>> Not quite sure about the whole picture.
> > > >>>>
> > > >>>> Seems while qemu waiting response of vhost_user_get_status, dpdk send out VHOST_USER_SLAVE_IOTLB_MSG and trigger qemu function vhost_backend_update_device_iotlb.
> > > >>>> Qemu wait on reply of VHOST_USER_IOTLB_MSG but get VHOST_USER_GET_STATUS reply.
> > > >>>
> > > >>> Thanks for the backtrace, that helps a lot.
> > > >>>
> > > >>> The issue happens because:
> > > >>>    1. Introduction of nested event loop in vhost_user_read() [0] features
> > > >>> that enables handling slave channel request while waiting for reply on
> > > >>> the masyer channel.
> > > >>>    2. Slave VHOST_USER_SLAVE_IOTLB_MSG slave request handling ends-up
> > > >>> sending a VHOST_USER_IOTLB_MSG on the master channel.
> > > >>>
> > > >>> So while waiting for VHOST_USER_IOTLB_MSG reply, it receives the reply
> > > >>> for the first request sent on the master channel, here the
> > > >>> VHOST_USER_GET_STATUS reply.
> > > >>>
> > > >>> I don't see an easy way to fix it.
> > > >>>
> > > >>> One option would be to have the slave channel being handled by another
> > > >>> thread, and protect master channel with a lock to enforce the
> > > >>> synchronization. But this may induce other issues, so that's not a light
> > > >>> change.
> > > >>>
> > > >>
> > > >> This is going to be tough because the back-end might have set the
> > > >> VHOST_USER_NEED_REPLY_MASK flag on the VHOST_USER_SLAVE_IOTLB_MSG
> > > >> request and thus might be waiting for a response on the slave
> > > >> channel. In order to emit such a response, the front-end must
> > > >> send VHOST_USER_SLAVE_IOTLB_MSG updates on the master channel *first*
> > > >> according to the protocol specification. This means that we really
> > > >> cannot handle VHOST_USER_SLAVE_IOTLB_MSG requests while there's
> > > >> an on-going transaction on the master channel.
> > > 
> > > Since the slave channel would be handled on another thread, it means the
> > > on-going transaction on the master channel can continue. Once done, it
> > > will release the mutex, and the thread handling the slave channel can
> > > take it send the IOTLB update on the master channel.
> > > 
> > > That would work with DPDK, which does not request reply-ack on IOTLB
> > > misses.
> > > 
> > 
> > Not sure to catch what would happen if DPDK requested a reply-ack with
> > this scenario.
> > 
> > > For the DAX enablement case, my understanding is that the handling of
> > > the slave requests by QEMU does not induce sending requests on the
> > > master channel, so if I'm not mistaken, it should work too. The thread
> > > handling the slave requests can send the reply on the slave channel,
> > > while the thread handling the master channel is blocked waiting for the
> > > GET_VRING_BASE reply. Is that correct?
> > > 
> > 
> > Yes this is correct AFAICT. Dropping the nested loop (commit db8a3772e3
> > reverts like a charm) and having the slave channel serviced by its own
> > thread seems to be the way to go then.
> > 
> 
> Commit a7f523c7d114d needs to be reverted as well. There are some conflicts
> because of commit 025faa872bcf9 but they seem trivial to fix.

OK especially since you posted the original patch ;)
Want to post the reverts? I'll apply.

> > > >>
> > > >>> (Adding Greg and Stefan, who worked on the nested event loop series.)
> > > >>>
> > > >>> Simply reverting nested event loop support may not be an option, since
> > > >>> it would break virtiofsd, as if my understanding is correct, waits for
> > > >>> some slave channel request to complete in order to complete a request
> > > >>> made by QEMU on the master channel.
> > > >>>
> > > >>> Any thougths?
> > > >>>
> > > >>
> > > >> Well... the nested even loop was added as preparatory work for "the
> > > >> upcoming enablement of DAX with virtio-fs". This requires changes on
> > > >> the QEMU side that haven't been merged yet. Technically, it seems that
> > > >> reverting the nested event loop won't break anything in upstream QEMU
> > > >> at this point (but this will bite again as soon as DAX enablement gets
> > > >> merged).
> > > >>
> > > > 
> > > > Cc'ing Dave to know about the DAX enablement status.
> > > > 
> > > >> AFAIK the event loop is only needed for the VHOST_USER_GET_VRING_BASE
> > > >> message. Another possibility might be to create the nested event loop
> > > >> in this case only : this would allow VHOST_USER_GET_STATUS to complete
> > > >> before QEMU starts processing the VHOST_USER_SLAVE_IOTLB_MSG requests.
> > > >>
> > > >> Cheers,
> > > >>
> > > >> --
> > > >> Greg
> > > >>
> > > >>> Maxime
> > > >>>
> > > >>> [0]:
> > > >>> https://patchwork.ozlabs.org/project/qemu-devel/patch/20210312092212.782255-6-groug@kaod.org/
> > > >>>
> > > >>>
> > > >>>> Break on first error message("Received unexpected msg type. Expected 22 received 40")
> > > >>>>
> > > >>>> #0  0x0000555555b72ed4 in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffa330) at ../hw/virtio/vhost-user.c:445
> > > >>>> #1  0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffa600) at ../hw/virtio/vhost-user.c:2341
> > > >>>> #2  0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10442706944, uaddr=140736119902208, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> > > >>>> #3  0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10442706944, write=1) at ../hw/virtio/vhost.c:1113
> > > >>>> #4  0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffa7b0) at ../hw/virtio/vhost-backend.c:393
> > > >>>> #5  0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> > > >>>> #6  0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556a06fb0, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> > > >>>> #7  0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > > >>>> #8  0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> > > >>>> #9  0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> > > >>>> #10 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffac50) at ../hw/virtio/vhost-user.c:413
> > > >>>> #11 0x0000555555b72e9b in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffaf10) at ../hw/virtio/vhost-user.c:439
> > > >>>> #12 0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffb1e0) at ../hw/virtio/vhost-user.c:2341
> > > >>>> #13 0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10468392960, uaddr=140736145588224, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> > > >>>> #14 0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10468392960, write=1) at ../hw/virtio/vhost.c:1113
> > > >>>> #15 0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffb390) at ../hw/virtio/vhost-backend.c:393
> > > >>>> #16 0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> > > >>>> #17 0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556c70250, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> > > >>>> #18 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > > >>>> #19 0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> > > >>>> #20 0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> > > >>>> #21 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffb830) at ../hw/virtio/vhost-user.c:413
> > > >>>> #22 0x0000555555b72e9b in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffbaf0) at ../hw/virtio/vhost-user.c:439
> > > >>>> #23 0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffbdc0) at ../hw/virtio/vhost-user.c:2341
> > > >>>> #24 0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10442702848, uaddr=140736119898112, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> > > >>>> #25 0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10442702848, write=1) at ../hw/virtio/vhost.c:1113
> > > >>>> #26 0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffbf70) at ../hw/virtio/vhost-backend.c:393
> > > >>>> #27 0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> > > >>>> #28 0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556f1a530, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> > > >>>> #29 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > > >>>> #30 0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> > > >>>> #31 0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> > > >>>> #32 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffc420) at ../hw/virtio/vhost-user.c:413
> > > >>>> #33 0x0000555555b754b1 in vhost_user_get_u64 (dev=0x5555584dd600, request=40, u64=0x7fffffffc6e0) at ../hw/virtio/vhost-user.c:1349
> > > >>>> #34 0x0000555555b758ff in vhost_user_get_status (dev=0x5555584dd600, status=0x7fffffffc713 "W\020") at ../hw/virtio/vhost-user.c:1474
> > > >>>> #35 0x0000555555b75967 in vhost_user_add_status (dev=0x5555584dd600, status=7 '\a') at ../hw/virtio/vhost-user.c:1488
> > > >>>> #36 0x0000555555b78bf6 in vhost_user_dev_start (dev=0x5555584dd600, started=true) at ../hw/virtio/vhost-user.c:2758
> > > >>>> #37 0x0000555555b709ad in vhost_dev_start (hdev=0x5555584dd600, vdev=0x555557b965d0, vrings=false) at ../hw/virtio/vhost.c:1988
> > > >>>> #38 0x000055555584291c in vhost_net_start_one (net=0x5555584dd600, dev=0x555557b965d0) at ../hw/net/vhost_net.c:271
> > > >>>> #39 0x0000555555842f1e in vhost_net_start (dev=0x555557b965d0, ncs=0x555557bc09e0, data_queue_pairs=1, cvq=0) at ../hw/net/vhost_net.c:412
> > > >>>> #40 0x0000555555b1bf61 in virtio_net_vhost_status (n=0x555557b965d0, status=15 '\017') at ../hw/net/virtio-net.c:311
> > > >>>> #41 0x0000555555b1c20c in virtio_net_set_status (vdev=0x555557b965d0, status=15 '\017') at ../hw/net/virtio-net.c:392
> > > >>>> #42 0x0000555555b1ed04 in virtio_net_handle_mq (n=0x555557b965d0, cmd=0 '\000', iov=0x555556c7ef50, iov_cnt=1) at ../hw/net/virtio-net.c:1497
> > > >>>> #43 0x0000555555b1eef0 in virtio_net_handle_ctrl_iov (vdev=0x555557b965d0, in_sg=0x555556a09880, in_num=1, out_sg=0x555556a09890, out_num=1) at ../hw/net/virtio-net.c:1534
> > > >>>> #44 0x0000555555b1efe9 in virtio_net_handle_ctrl (vdev=0x555557b965d0, vq=0x7fffc04ac140) at ../hw/net/virtio-net.c:1557
> > > >>>> #45 0x0000555555b63776 in virtio_queue_notify_vq (vq=0x7fffc04ac140) at ../hw/virtio/virtio.c:2249
> > > >>>> #46 0x0000555555b669dc in virtio_queue_host_notifier_read (n=0x7fffc04ac1b4) at ../hw/virtio/virtio.c:3529
> > > >>>> #47 0x0000555555e3f458 in aio_dispatch_handler (ctx=0x555556a016c0, node=0x7ffd8800e430) at ../util/aio-posix.c:369
> > > >>>> #48 0x0000555555e3f613 in aio_dispatch_handlers (ctx=0x555556a016c0) at ../util/aio-posix.c:412
> > > >>>> #49 0x0000555555e3f669 in aio_dispatch (ctx=0x555556a016c0) at ../util/aio-posix.c:422
> > > >>>> #50 0x0000555555e585de in aio_ctx_dispatch (source=0x555556a016c0, callback=0x0, user_data=0x0) at ../util/async.c:321
> > > >>>> #51 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > > >>>> #52 0x0000555555e5abea in glib_pollfds_poll () at ../util/main-loop.c:295
> > > >>>> #53 0x0000555555e5ac64 in os_host_main_loop_wait (timeout=0) at ../util/main-loop.c:318
> > > >>>> #54 0x0000555555e5ad69 in main_loop_wait (nonblocking=0) at ../util/main-loop.c:604
> > > >>>> #55 0x00005555559693de in qemu_main_loop () at ../softmmu/runstate.c:731
> > > >>>> #56 0x00005555556e7c06 in qemu_default_main () at ../softmmu/main.c:37
> > > >>>> #57 0x00005555556e7c3c in main (argc=71, argv=0x7fffffffcda8) at ../softmmu/main.c:48
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> -----Original Message-----
> > > >>>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> > > >>>> Sent: Thursday, January 12, 2023 5:26 PM
> > > >>>> To: Laurent Vivier <lvivier@redhat.com>
> > > >>>> Cc: qemu-devel@nongnu.org; Peter Maydell <peter.maydell@linaro.org>; Yajun Wu <yajunw@nvidia.com>; Parav Pandit <parav@nvidia.com>; Michael S. Tsirkin <mst@redhat.com>
> > > >>>> Subject: Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
> > > >>>>
> > > >>>> External email: Use caution opening links or attachments
> > > >>>>
> > > >>>>
> > > >>>> Hi Laurent,
> > > >>>>
> > > >>>> On 1/11/23 10:50, Laurent Vivier wrote:
> > > >>>>> On 1/9/23 11:55, Michael S. Tsirkin wrote:
> > > >>>>>> On Fri, Jan 06, 2023 at 03:21:43PM +0100, Laurent Vivier wrote:
> > > >>>>>>> Hi,
> > > >>>>>>>
> > > >>>>>>> it seems this patch breaks vhost-user with DPDK.
> > > >>>>>>>
> > > >>>>>>> See
> > > >>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbu
> > > >>>>>>> gzilla.redhat.com%2Fshow_bug.cgi%3Fid%3D2155173&data=05%7C01%7Cyajun
> > > >>>>>>> w%40nvidia.com%7C47e6e0fabd044383fd3308daf47f0253%7C43083d15727340c1
> > > >>>>>>> b7db39efd9ccc17a%7C0%7C0%7C638091123577559319%7CUnknown%7CTWFpbGZsb3
> > > >>>>>>> d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
> > > >>>>>>> %7C3000%7C%7C%7C&sdata=1pjChYTKHVmBoempNitiZHBdrlPIMFjKoD6FeOVSay0%3
> > > >>>>>>> D&reserved=0
> > > >>>>>>>
> > > >>>>>>> it seems QEMU doesn't receive the expected commands sequence:
> > > >>>>>>>
> > > >>>>>>> Received unexpected msg type. Expected 22 received 40 Fail to update
> > > >>>>>>> device iotlb Received unexpected msg type. Expected 40 received 22
> > > >>>>>>> Received unexpected msg type. Expected 22 received 11 Fail to update
> > > >>>>>>> device iotlb Received unexpected msg type. Expected 11 received 22
> > > >>>>>>> vhost VQ 1 ring restore failed: -71: Protocol error (71) Received
> > > >>>>>>> unexpected msg type. Expected 22 received 11 Fail to update device
> > > >>>>>>> iotlb Received unexpected msg type. Expected 11 received 22 vhost VQ
> > > >>>>>>> 0 ring restore failed: -71: Protocol error (71) unable to start
> > > >>>>>>> vhost net: 71: falling back on userspace virtio
> > > >>>>>>>
> > > >>>>>>> It receives VHOST_USER_GET_STATUS (40) when it expects
> > > >>>>>>> VHOST_USER_IOTLB_MSG (22) and VHOST_USER_IOTLB_MSG when it expects
> > > >>>>>>> VHOST_USER_GET_STATUS.
> > > >>>>>>> and VHOST_USER_GET_VRING_BASE (11) when it expect
> > > >>>>>>> VHOST_USER_GET_STATUS and so on.
> > > >>>>>>>
> > > >>>>>>> Any idea?
> > > >>>>
> > > >>>> We only have a single thread on DPDK side to handle Vhost-user requests, it will read a request, handle it and reply to it. Then it reads the next one, etc... So I don't think it is possible to mix request replies order on DPDK side.
> > > >>>>
> > > >>>> Maybe there are two threads concurrently sending requests on QEMU side?
> > > >>>>
> > > >>>> Regards,
> > > >>>> Maxime
> > > >>>>
> > > >>>>>>> Thanks,
> > > >>>>>>> Laurent
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> So I am guessing it's coming from:
> > > >>>>>>
> > > >>>>>>        if (msg.hdr.request != request) {
> > > >>>>>>            error_report("Received unexpected msg type. Expected %d
> > > >>>>>> received %d",
> > > >>>>>>                         request, msg.hdr.request);
> > > >>>>>>            return -EPROTO;
> > > >>>>>>        }
> > > >>>>>>
> > > >>>>>> in process_message_reply and/or in vhost_user_get_u64.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>> On 11/7/22 23:53, Michael S. Tsirkin wrote:
> > > >>>>>>>> From: Yajun Wu <yajunw@nvidia.com>
> > > >>>>>>>>
> > > >>>>>>>> The motivation of adding vhost-user vhost_dev_start support is to
> > > >>>>>>>> improve backend configuration speed and reduce live migration VM
> > > >>>>>>>> downtime.
> > > >>>>>>>>
> > > >>>>>>>> Today VQ configuration is issued one by one. For virtio net with
> > > >>>>>>>> multi-queue support, backend needs to update RSS (Receive side
> > > >>>>>>>> scaling) on every rx queue enable. Updating RSS is time-consuming
> > > >>>>>>>> (typical time like 7ms).
> > > >>>>>>>>
> > > >>>>>>>> Implement already defined vhost status and message in the vhost
> > > >>>>>>>> specification [1].
> > > >>>>>>>> (a) VHOST_USER_PROTOCOL_F_STATUS
> > > >>>>>>>> (b) VHOST_USER_SET_STATUS
> > > >>>>>>>> (c) VHOST_USER_GET_STATUS
> > > >>>>>>>>
> > > >>>>>>>> Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK
> > > >>>>>>>> for device start and reset(0) for device stop.
> > > >>>>>>>>
> > > >>>>>>>> On reception of the DRIVER_OK message, backend can apply the needed
> > > >>>>>>>> setting only once (instead of incremental) and also utilize
> > > >>>>>>>> parallelism on enabling queues.
> > > >>>>>>>>
> > > >>>>>>>> This improves QEMU's live migration downtime with vhost user
> > > >>>>>>>> backend implementation by great margin, specially for the large
> > > >>>>>>>> number of VQs of 64 from 800 msec to 250 msec.
> > > >>>>>>>>
> > > >>>>>>>> [1]
> > > >>>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fq
> > > >>>>>>>> emu-project.gitlab.io%2Fqemu%2Finterop%2Fvhost-user.html&data=05%7C
> > > >>>>>>>> 01%7Cyajunw%40nvidia.com%7C47e6e0fabd044383fd3308daf47f0253%7C43083
> > > >>>>>>>> d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638091123577559319%7CUnknown%
> > > >>>>>>>> 7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
> > > >>>>>>>> CJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YPbrFRJA92KLLwADMUDvwBt%2Fme2Ef
> > > >>>>>>>> GZuVANOmXH5pic%3D&reserved=0
> > > >>>>>>>>
> > > >>>>>>>> Signed-off-by: Yajun Wu <yajunw@nvidia.com>
> > > >>>>>>>> Acked-by: Parav Pandit <parav@nvidia.com>
> > > >>>>>>>> Message-Id: <20221017064452.1226514-3-yajunw@nvidia.com>
> > > >>>>>>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > > >>>>>>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > >>>>>>
> > > >>>>>> Probably easiest to debug from dpdk side.
> > > >>>>>> Does the problem go away if you disable the feature
> > > >>>>>> VHOST_USER_PROTOCOL_F_STATUS in dpdk?
> > > >>>>>
> > > >>>>> Maxime could you help to debug this?
> > > >>>>>
> > > >>>>> Thanks,
> > > >>>>> Laurent
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>
> > > >>
> > > > 
> > > > 
> > > 
> > 
> > 



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
  2023-01-17 12:36                 ` Greg Kurz
  2023-01-17 15:07                   ` Maxime Coquelin
@ 2023-01-19 19:48                   ` Dr. David Alan Gilbert
  1 sibling, 0 replies; 151+ messages in thread
From: Dr. David Alan Gilbert @ 2023-01-19 19:48 UTC (permalink / raw)
  To: Greg Kurz
  Cc: Maxime Coquelin, Yajun Wu, Laurent Vivier, Stefan Hajnoczi,
	qemu-devel, Peter Maydell, Parav Pandit, Michael S. Tsirkin

* Greg Kurz (groug@kaod.org) wrote:
> On Tue, 17 Jan 2023 13:12:57 +0100
> Greg Kurz <groug@kaod.org> wrote:
> 
> > Hi Maxime,
> > 
> > On Tue, 17 Jan 2023 10:49:37 +0100
> > Maxime Coquelin <maxime.coquelin@redhat.com> wrote:
> > 
> > > Hi Yajun,
> > > 
> > > On 1/16/23 08:14, Yajun Wu wrote:
> > > > Not quite sure about the whole picture.
> > > > 
> > > > Seems while qemu waiting response of vhost_user_get_status, dpdk send out VHOST_USER_SLAVE_IOTLB_MSG and trigger qemu function vhost_backend_update_device_iotlb.
> > > > Qemu wait on reply of VHOST_USER_IOTLB_MSG but get VHOST_USER_GET_STATUS reply.
> > > 
> > > Thanks for the backtrace, that helps a lot.
> > > 
> > > The issue happens because:
> > >   1. Introduction of nested event loop in vhost_user_read() [0] features
> > > that enables handling slave channel request while waiting for reply on
> > > the masyer channel.
> > >   2. Slave VHOST_USER_SLAVE_IOTLB_MSG slave request handling ends-up 
> > > sending a VHOST_USER_IOTLB_MSG on the master channel.
> > > 
> > > So while waiting for VHOST_USER_IOTLB_MSG reply, it receives the reply
> > > for the first request sent on the master channel, here the
> > > VHOST_USER_GET_STATUS reply.
> > > 
> > > I don't see an easy way to fix it.
> > > 
> > > One option would be to have the slave channel being handled by another
> > > thread, and protect master channel with a lock to enforce the
> > > synchronization. But this may induce other issues, so that's not a light
> > > change.
> > > 
> > 
> > This is going to be tough because the back-end might have set the
> > VHOST_USER_NEED_REPLY_MASK flag on the VHOST_USER_SLAVE_IOTLB_MSG
> > request and thus might be waiting for a response on the slave
> > channel. In order to emit such a response, the front-end must
> > send VHOST_USER_SLAVE_IOTLB_MSG updates on the master channel *first*
> > according to the protocol specification. This means that we really
> > cannot handle VHOST_USER_SLAVE_IOTLB_MSG requests while there's
> > an on-going transaction on the master channel.
> > 
> > > (Adding Greg and Stefan, who worked on the nested event loop series.)
> > > 
> > > Simply reverting nested event loop support may not be an option, since
> > > it would break virtiofsd, as if my understanding is correct, waits for
> > > some slave channel request to complete in order to complete a request
> > > made by QEMU on the master channel.
> > > 
> > > Any thougths?
> > > 
> > 
> > Well... the nested even loop was added as preparatory work for "the
> > upcoming enablement of DAX with virtio-fs". This requires changes on
> > the QEMU side that haven't been merged yet. Technically, it seems that
> > reverting the nested event loop won't break anything in upstream QEMU
> > at this point (but this will bite again as soon as DAX enablement gets
> > merged).
> > 
> 
> Cc'ing Dave to know about the DAX enablement status.

Yeh not too much to say; but having just sent the patches to nuke the C
virtiofsd, the plan is to look at the Rust daemon; it's got code in
already for DAX, and apparently working with CrosVM which I believe are
based on my original proposals for the protocol.
But it would probably need a clean fix to this deadlock.

IMHO the slave channel needs properly specifying; no one really
understands what the semantics are supposed to be.

Dave

> > AFAIK the event loop is only needed for the VHOST_USER_GET_VRING_BASE
> > message. Another possibility might be to create the nested event loop
> > in this case only : this would allow VHOST_USER_GET_STATUS to complete
> > before QEMU starts processing the VHOST_USER_SLAVE_IOTLB_MSG requests.
> > 
> > Cheers,
> > 
> > --
> > Greg
> > 
> > > Maxime
> > > 
> > > [0]: 
> > > https://patchwork.ozlabs.org/project/qemu-devel/patch/20210312092212.782255-6-groug@kaod.org/
> > > 
> > > 
> > > > Break on first error message("Received unexpected msg type. Expected 22 received 40")
> > > > 
> > > > #0  0x0000555555b72ed4 in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffa330) at ../hw/virtio/vhost-user.c:445
> > > > #1  0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffa600) at ../hw/virtio/vhost-user.c:2341
> > > > #2  0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10442706944, uaddr=140736119902208, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> > > > #3  0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10442706944, write=1) at ../hw/virtio/vhost.c:1113
> > > > #4  0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffa7b0) at ../hw/virtio/vhost-backend.c:393
> > > > #5  0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> > > > #6  0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556a06fb0, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> > > > #7  0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > > > #8  0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> > > > #9  0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> > > > #10 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffac50) at ../hw/virtio/vhost-user.c:413
> > > > #11 0x0000555555b72e9b in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffaf10) at ../hw/virtio/vhost-user.c:439
> > > > #12 0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffb1e0) at ../hw/virtio/vhost-user.c:2341
> > > > #13 0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10468392960, uaddr=140736145588224, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> > > > #14 0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10468392960, write=1) at ../hw/virtio/vhost.c:1113
> > > > #15 0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffb390) at ../hw/virtio/vhost-backend.c:393
> > > > #16 0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> > > > #17 0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556c70250, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> > > > #18 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > > > #19 0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> > > > #20 0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> > > > #21 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffb830) at ../hw/virtio/vhost-user.c:413
> > > > #22 0x0000555555b72e9b in process_message_reply (dev=0x5555584dd600, msg=0x7fffffffbaf0) at ../hw/virtio/vhost-user.c:439
> > > > #23 0x0000555555b77c26 in vhost_user_send_device_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffbdc0) at ../hw/virtio/vhost-user.c:2341
> > > > #24 0x0000555555b7179e in vhost_backend_update_device_iotlb (dev=0x5555584dd600, iova=10442702848, uaddr=140736119898112, len=4096, perm=IOMMU_RW) at ../hw/virtio/vhost-backend.c:361
> > > > #25 0x0000555555b6e34c in vhost_device_iotlb_miss (dev=0x5555584dd600, iova=10442702848, write=1) at ../hw/virtio/vhost.c:1113
> > > > #26 0x0000555555b718d9 in vhost_backend_handle_iotlb_msg (dev=0x5555584dd600, imsg=0x7fffffffbf70) at ../hw/virtio/vhost-backend.c:393
> > > > #27 0x0000555555b76144 in slave_read (ioc=0x555557a38680, condition=G_IO_IN, opaque=0x5555584dd600) at ../hw/virtio/vhost-user.c:1726
> > > > #28 0x0000555555c797a5 in qio_channel_fd_source_dispatch (source=0x555556f1a530, callback=0x555555b75f86 <slave_read>, user_data=0x5555584dd600) at ../io/channel-watch.c:84
> > > > #29 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > > > #30 0x00007ffff5548d18 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
> > > > #31 0x00007ffff5549042 in g_main_loop_run () at /lib64/libglib-2.0.so.0
> > > > #32 0x0000555555b72de7 in vhost_user_read (dev=0x5555584dd600, msg=0x7fffffffc420) at ../hw/virtio/vhost-user.c:413
> > > > #33 0x0000555555b754b1 in vhost_user_get_u64 (dev=0x5555584dd600, request=40, u64=0x7fffffffc6e0) at ../hw/virtio/vhost-user.c:1349
> > > > #34 0x0000555555b758ff in vhost_user_get_status (dev=0x5555584dd600, status=0x7fffffffc713 "W\020") at ../hw/virtio/vhost-user.c:1474
> > > > #35 0x0000555555b75967 in vhost_user_add_status (dev=0x5555584dd600, status=7 '\a') at ../hw/virtio/vhost-user.c:1488
> > > > #36 0x0000555555b78bf6 in vhost_user_dev_start (dev=0x5555584dd600, started=true) at ../hw/virtio/vhost-user.c:2758
> > > > #37 0x0000555555b709ad in vhost_dev_start (hdev=0x5555584dd600, vdev=0x555557b965d0, vrings=false) at ../hw/virtio/vhost.c:1988
> > > > #38 0x000055555584291c in vhost_net_start_one (net=0x5555584dd600, dev=0x555557b965d0) at ../hw/net/vhost_net.c:271
> > > > #39 0x0000555555842f1e in vhost_net_start (dev=0x555557b965d0, ncs=0x555557bc09e0, data_queue_pairs=1, cvq=0) at ../hw/net/vhost_net.c:412
> > > > #40 0x0000555555b1bf61 in virtio_net_vhost_status (n=0x555557b965d0, status=15 '\017') at ../hw/net/virtio-net.c:311
> > > > #41 0x0000555555b1c20c in virtio_net_set_status (vdev=0x555557b965d0, status=15 '\017') at ../hw/net/virtio-net.c:392
> > > > #42 0x0000555555b1ed04 in virtio_net_handle_mq (n=0x555557b965d0, cmd=0 '\000', iov=0x555556c7ef50, iov_cnt=1) at ../hw/net/virtio-net.c:1497
> > > > #43 0x0000555555b1eef0 in virtio_net_handle_ctrl_iov (vdev=0x555557b965d0, in_sg=0x555556a09880, in_num=1, out_sg=0x555556a09890, out_num=1) at ../hw/net/virtio-net.c:1534
> > > > #44 0x0000555555b1efe9 in virtio_net_handle_ctrl (vdev=0x555557b965d0, vq=0x7fffc04ac140) at ../hw/net/virtio-net.c:1557
> > > > #45 0x0000555555b63776 in virtio_queue_notify_vq (vq=0x7fffc04ac140) at ../hw/virtio/virtio.c:2249
> > > > #46 0x0000555555b669dc in virtio_queue_host_notifier_read (n=0x7fffc04ac1b4) at ../hw/virtio/virtio.c:3529
> > > > #47 0x0000555555e3f458 in aio_dispatch_handler (ctx=0x555556a016c0, node=0x7ffd8800e430) at ../util/aio-posix.c:369
> > > > #48 0x0000555555e3f613 in aio_dispatch_handlers (ctx=0x555556a016c0) at ../util/aio-posix.c:412
> > > > #49 0x0000555555e3f669 in aio_dispatch (ctx=0x555556a016c0) at ../util/aio-posix.c:422
> > > > #50 0x0000555555e585de in aio_ctx_dispatch (source=0x555556a016c0, callback=0x0, user_data=0x0) at ../util/async.c:321
> > > > #51 0x00007ffff554895d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
> > > > #52 0x0000555555e5abea in glib_pollfds_poll () at ../util/main-loop.c:295
> > > > #53 0x0000555555e5ac64 in os_host_main_loop_wait (timeout=0) at ../util/main-loop.c:318
> > > > #54 0x0000555555e5ad69 in main_loop_wait (nonblocking=0) at ../util/main-loop.c:604
> > > > #55 0x00005555559693de in qemu_main_loop () at ../softmmu/runstate.c:731
> > > > #56 0x00005555556e7c06 in qemu_default_main () at ../softmmu/main.c:37
> > > > #57 0x00005555556e7c3c in main (argc=71, argv=0x7fffffffcda8) at ../softmmu/main.c:48
> > > > 
> > > > 
> > > > 
> > > > -----Original Message-----
> > > > From: Maxime Coquelin <maxime.coquelin@redhat.com>
> > > > Sent: Thursday, January 12, 2023 5:26 PM
> > > > To: Laurent Vivier <lvivier@redhat.com>
> > > > Cc: qemu-devel@nongnu.org; Peter Maydell <peter.maydell@linaro.org>; Yajun Wu <yajunw@nvidia.com>; Parav Pandit <parav@nvidia.com>; Michael S. Tsirkin <mst@redhat.com>
> > > > Subject: Re: [PULL v4 76/83] vhost-user: Support vhost_dev_start
> > > > 
> > > > External email: Use caution opening links or attachments
> > > > 
> > > > 
> > > > Hi Laurent,
> > > > 
> > > > On 1/11/23 10:50, Laurent Vivier wrote:
> > > >> On 1/9/23 11:55, Michael S. Tsirkin wrote:
> > > >>> On Fri, Jan 06, 2023 at 03:21:43PM +0100, Laurent Vivier wrote:
> > > >>>> Hi,
> > > >>>>
> > > >>>> it seems this patch breaks vhost-user with DPDK.
> > > >>>>
> > > >>>> See
> > > >>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbu
> > > >>>> gzilla.redhat.com%2Fshow_bug.cgi%3Fid%3D2155173&data=05%7C01%7Cyajun
> > > >>>> w%40nvidia.com%7C47e6e0fabd044383fd3308daf47f0253%7C43083d15727340c1
> > > >>>> b7db39efd9ccc17a%7C0%7C0%7C638091123577559319%7CUnknown%7CTWFpbGZsb3
> > > >>>> d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
> > > >>>> %7C3000%7C%7C%7C&sdata=1pjChYTKHVmBoempNitiZHBdrlPIMFjKoD6FeOVSay0%3
> > > >>>> D&reserved=0
> > > >>>>
> > > >>>> it seems QEMU doesn't receive the expected commands sequence:
> > > >>>>
> > > >>>> Received unexpected msg type. Expected 22 received 40 Fail to update
> > > >>>> device iotlb Received unexpected msg type. Expected 40 received 22
> > > >>>> Received unexpected msg type. Expected 22 received 11 Fail to update
> > > >>>> device iotlb Received unexpected msg type. Expected 11 received 22
> > > >>>> vhost VQ 1 ring restore failed: -71: Protocol error (71) Received
> > > >>>> unexpected msg type. Expected 22 received 11 Fail to update device
> > > >>>> iotlb Received unexpected msg type. Expected 11 received 22 vhost VQ
> > > >>>> 0 ring restore failed: -71: Protocol error (71) unable to start
> > > >>>> vhost net: 71: falling back on userspace virtio
> > > >>>>
> > > >>>> It receives VHOST_USER_GET_STATUS (40) when it expects
> > > >>>> VHOST_USER_IOTLB_MSG (22) and VHOST_USER_IOTLB_MSG when it expects
> > > >>>> VHOST_USER_GET_STATUS.
> > > >>>> and VHOST_USER_GET_VRING_BASE (11) when it expect
> > > >>>> VHOST_USER_GET_STATUS and so on.
> > > >>>>
> > > >>>> Any idea?
> > > > 
> > > > We only have a single thread on DPDK side to handle Vhost-user requests, it will read a request, handle it and reply to it. Then it reads the next one, etc... So I don't think it is possible to mix request replies order on DPDK side.
> > > > 
> > > > Maybe there are two threads concurrently sending requests on QEMU side?
> > > > 
> > > > Regards,
> > > > Maxime
> > > > 
> > > >>>> Thanks,
> > > >>>> Laurent
> > > >>>
> > > >>>
> > > >>> So I am guessing it's coming from:
> > > >>>
> > > >>>       if (msg.hdr.request != request) {
> > > >>>           error_report("Received unexpected msg type. Expected %d
> > > >>> received %d",
> > > >>>                        request, msg.hdr.request);
> > > >>>           return -EPROTO;
> > > >>>       }
> > > >>>
> > > >>> in process_message_reply and/or in vhost_user_get_u64.
> > > >>>
> > > >>>
> > > >>>> On 11/7/22 23:53, Michael S. Tsirkin wrote:
> > > >>>>> From: Yajun Wu <yajunw@nvidia.com>
> > > >>>>>
> > > >>>>> The motivation of adding vhost-user vhost_dev_start support is to
> > > >>>>> improve backend configuration speed and reduce live migration VM
> > > >>>>> downtime.
> > > >>>>>
> > > >>>>> Today VQ configuration is issued one by one. For virtio net with
> > > >>>>> multi-queue support, backend needs to update RSS (Receive side
> > > >>>>> scaling) on every rx queue enable. Updating RSS is time-consuming
> > > >>>>> (typical time like 7ms).
> > > >>>>>
> > > >>>>> Implement already defined vhost status and message in the vhost
> > > >>>>> specification [1].
> > > >>>>> (a) VHOST_USER_PROTOCOL_F_STATUS
> > > >>>>> (b) VHOST_USER_SET_STATUS
> > > >>>>> (c) VHOST_USER_GET_STATUS
> > > >>>>>
> > > >>>>> Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK
> > > >>>>> for device start and reset(0) for device stop.
> > > >>>>>
> > > >>>>> On reception of the DRIVER_OK message, backend can apply the needed
> > > >>>>> setting only once (instead of incremental) and also utilize
> > > >>>>> parallelism on enabling queues.
> > > >>>>>
> > > >>>>> This improves QEMU's live migration downtime with vhost user
> > > >>>>> backend implementation by great margin, specially for the large
> > > >>>>> number of VQs of 64 from 800 msec to 250 msec.
> > > >>>>>
> > > >>>>> [1]
> > > >>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fq
> > > >>>>> emu-project.gitlab.io%2Fqemu%2Finterop%2Fvhost-user.html&data=05%7C
> > > >>>>> 01%7Cyajunw%40nvidia.com%7C47e6e0fabd044383fd3308daf47f0253%7C43083
> > > >>>>> d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638091123577559319%7CUnknown%
> > > >>>>> 7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
> > > >>>>> CJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YPbrFRJA92KLLwADMUDvwBt%2Fme2Ef
> > > >>>>> GZuVANOmXH5pic%3D&reserved=0
> > > >>>>>
> > > >>>>> Signed-off-by: Yajun Wu <yajunw@nvidia.com>
> > > >>>>> Acked-by: Parav Pandit <parav@nvidia.com>
> > > >>>>> Message-Id: <20221017064452.1226514-3-yajunw@nvidia.com>
> > > >>>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > > >>>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > >>>
> > > >>> Probably easiest to debug from dpdk side.
> > > >>> Does the problem go away if you disable the feature
> > > >>> VHOST_USER_PROTOCOL_F_STATUS in dpdk?
> > > >>
> > > >> Maxime could you help to debug this?
> > > >>
> > > >> Thanks,
> > > >> Laurent
> > > >>
> > > > 
> > > 
> > 
> > 
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 74/83] intel-iommu: PASID support
  2022-11-07 22:53 ` [PULL v4 74/83] intel-iommu: PASID support Michael S. Tsirkin
@ 2023-04-06 16:22   ` Peter Maydell
  2023-04-10  2:55     ` Jason Wang
  0 siblings, 1 reply; 151+ messages in thread
From: Peter Maydell @ 2023-04-06 16:22 UTC (permalink / raw)
  To: Michael S. Tsirkin; +Cc: qemu-devel, Jason Wang

On Mon, 7 Nov 2022 at 22:53, Michael S. Tsirkin <mst@redhat.com> wrote:
>
> From: Jason Wang <jasowang@redhat.com>
>
> This patch introduce ECAP_PASID via "x-pasid-mode".

Hi; Coverity points out an issue with this code (CID 1508100):

> -static guint vtd_uint64_hash(gconstpointer v)
> +static guint vtd_iotlb_hash(gconstpointer v)
>  {
> -    return (guint)*(const uint64_t *)v;
> +    const struct vtd_iotlb_key *key = v;
> +
> +    return key->gfn | ((key->sid) << VTD_IOTLB_SID_SHIFT) |

key->sid is a uint16_t, and VTD_IOTLB_SID_SHIFT is 20. That
means that the shift will be done as a signed 32 bit operation,
losing the top 4 bits of key->sid; then it will get sign
extended to 64 bits, so if bit 11 of key->sid is 1 then
we will end up with 1 bits in 63..32 of the output hash value.
This seems unlikely to be what was intended.

> +           (key->level) << VTD_IOTLB_LVL_SHIFT |
> +           (key->pasid) << VTD_IOTLB_PASID_SHIFT;
>  }

Also, VTD_IOTLB_LVL_SHIFT is only 28, so either the
shift values are wrong or the type of key->sid is wrong:
can there be 8 bits here, or 16 ?

Since PASID_SHIFT is 30, if key->pasid can be more than
2 bits wide we'll lose most of it.

If key->level will fit into 2 bits as the SHIFT values
suggest, vtd_iotlb_key could probably use a uint8_t for it,
which would let that struct fit into 16 bytes rather than 18.

> @@ -302,13 +321,6 @@ static void vtd_reset_caches(IntelIOMMUState *s)
>      vtd_iommu_unlock(s);
>  }
>
> -static uint64_t vtd_get_iotlb_key(uint64_t gfn, uint16_t source_id,
> -                                  uint32_t level)
> -{
> -    return gfn | ((uint64_t)(source_id) << VTD_IOTLB_SID_SHIFT) |
> -           ((uint64_t)(level) << VTD_IOTLB_LVL_SHIFT);
> -}

In the old code you can see that we did casts to uint64_t in order
to ensure that all the arithmetic was done as unsigned 64 bits.

thanks
-- PMM


^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 74/83] intel-iommu: PASID support
  2023-04-06 16:22   ` Peter Maydell
@ 2023-04-10  2:55     ` Jason Wang
  0 siblings, 0 replies; 151+ messages in thread
From: Jason Wang @ 2023-04-10  2:55 UTC (permalink / raw)
  To: Peter Maydell; +Cc: Michael S. Tsirkin, qemu-devel

On Fri, Apr 7, 2023 at 12:22 AM Peter Maydell <peter.maydell@linaro.org> wrote:
>
> On Mon, 7 Nov 2022 at 22:53, Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > From: Jason Wang <jasowang@redhat.com>
> >
> > This patch introduce ECAP_PASID via "x-pasid-mode".
>
> Hi; Coverity points out an issue with this code (CID 1508100):
>
> > -static guint vtd_uint64_hash(gconstpointer v)
> > +static guint vtd_iotlb_hash(gconstpointer v)
> >  {
> > -    return (guint)*(const uint64_t *)v;
> > +    const struct vtd_iotlb_key *key = v;
> > +
> > +    return key->gfn | ((key->sid) << VTD_IOTLB_SID_SHIFT) |
>
> key->sid is a uint16_t, and VTD_IOTLB_SID_SHIFT is 20. That
> means that the shift will be done as a signed 32 bit operation,
> losing the top 4 bits of key->sid; then it will get sign
> extended to 64 bits, so if bit 11 of key->sid is 1 then
> we will end up with 1 bits in 63..32 of the output hash value.
> This seems unlikely to be what was intended.

Right.

>
> > +           (key->level) << VTD_IOTLB_LVL_SHIFT |
> > +           (key->pasid) << VTD_IOTLB_PASID_SHIFT;
> >  }
>
> Also, VTD_IOTLB_LVL_SHIFT is only 28, so either the
> shift values are wrong or the type of key->sid is wrong:
> can there be 8 bits here, or 16 ?

It should be 16.

>
> Since PASID_SHIFT is 30, if key->pasid can be more than
> 2 bits wide we'll lose most of it.
>
> If key->level will fit into 2 bits as the SHIFT values
> suggest, vtd_iotlb_key could probably use a uint8_t for it,

Right.

> which would let that struct fit into 16 bytes rather than 18.

Ok, So to summarize:

1) key->gfn could be full 64bit
2) key->sid is at most 16bit
3) key->level is at most 2bit
4) key->pasid is at most 20bit

So we will lose some bits for sure.

Since the current VTD_IOTLB_PASID_SHIFT is 30, we might waste 14bits
there, then I tend to change

VTD_IOTLB_SID_SHIFT to 26 (42-16)
VTD_IOTLB_LVL_SHIFT to 42 (44-2)
VTD_IOTLB_PASID_SHIFT to 44 (64-20)

>
> > @@ -302,13 +321,6 @@ static void vtd_reset_caches(IntelIOMMUState *s)
> >      vtd_iommu_unlock(s);
> >  }
> >
> > -static uint64_t vtd_get_iotlb_key(uint64_t gfn, uint16_t source_id,
> > -                                  uint32_t level)
> > -{
> > -    return gfn | ((uint64_t)(source_id) << VTD_IOTLB_SID_SHIFT) |
> > -           ((uint64_t)(level) << VTD_IOTLB_LVL_SHIFT);
> > -}
>
> In the old code you can see that we did casts to uint64_t in order
> to ensure that all the arithmetic was done as unsigned 64 bits.

Right, I will post a fix.

Thanks

>
> thanks
> -- PMM
>



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 23/83] hw/cxl/cdat: CXL CDAT Data Object Exchange implementation
  2022-11-07 22:49 ` [PULL v4 23/83] hw/cxl/cdat: CXL CDAT Data Object Exchange implementation Michael S. Tsirkin
@ 2023-04-11 15:52   ` Peter Maydell
  2023-04-12 13:08     ` Jonathan Cameron via
  2023-04-12 14:39     ` Jonathan Cameron via
  0 siblings, 2 replies; 151+ messages in thread
From: Peter Maydell @ 2023-04-11 15:52 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: qemu-devel, Huai-Cheng Kuo, Chris Browy, Jonathan Cameron, Ben Widawsky

On Mon, 7 Nov 2022 at 22:49, Michael S. Tsirkin <mst@redhat.com> wrote:
>
> From: Huai-Cheng Kuo <hchkuo@avery-design.com.tw>
>
> The Data Object Exchange implementation of CXL Coherent Device Attribute
> Table (CDAT). This implementation is referring to "Coherent Device
> Attribute Table Specification, Rev. 1.03, July. 2022" and "Compute
> Express Link Specification, Rev. 3.0, July. 2022"
>
> This patch adds core support that will be shared by both
> end-points and switch port emulation.

> +static void ct3_load_cdat(CDATObject *cdat, Error **errp)
> +{
> +    g_autofree CDATEntry *cdat_st = NULL;
> +    uint8_t sum = 0;
> +    int num_ent;
> +    int i = 0, ent = 1, file_size = 0;
> +    CDATSubHeader *hdr;
> +    FILE *fp = NULL;
> +
> +    /* Read CDAT file and create its cache */
> +    fp = fopen(cdat->filename, "r");
> +    if (!fp) {
> +        error_setg(errp, "CDAT: Unable to open file");
> +        return;
> +    }
> +
> +    fseek(fp, 0, SEEK_END);
> +    file_size = ftell(fp);

Coverity points out that ftell() can fail and return -1...

> +    fseek(fp, 0, SEEK_SET);
> +    cdat->buf = g_malloc0(file_size);

...which would cause an attempt to allocate a very large
amount of memory, since you aren't checking for errors.
CID 1508185.

Below, some other issues I saw in a quick scan through the code.

> +
> +    if (fread(cdat->buf, file_size, 1, fp) == 0) {
> +        error_setg(errp, "CDAT: File read failed");
> +        return;
> +    }

(The issues in this bit of code I've mentioned in a
different thread.)

> +
> +    fclose(fp);
> +
> +    if (file_size < sizeof(CDATTableHeader)) {
> +        error_setg(errp, "CDAT: File too short");
> +        return;
> +    }

> +    i = sizeof(CDATTableHeader);
> +    num_ent = 1;
> +    while (i < file_size) {
> +        hdr = (CDATSubHeader *)(cdat->buf + i);

If the file is not a complete number of records in
size, then this can index off the end of the buffer.
You should check for that.

> +        cdat_len_check(hdr, errp);
> +        i += hdr->length;
> +        num_ent++;
> +    }
> +    if (i != file_size) {
> +        error_setg(errp, "CDAT: File length missmatch");

Typo: "mismatch"

> +        return;
> +    }
> +
> +    cdat_st = g_malloc0(sizeof(*cdat_st) * num_ent);

To allocate an array of N lots of a structure, use
g_new0(), which will take care to avoid possible
overflow in the multiply.

> +    if (!cdat_st) {
> +        error_setg(errp, "CDAT: Failed to allocate entry array");

g_malloc0() and g_new0() can never fail, so you don't need
to check for a NULL pointer return.

> +        return;
> +    }
> +
> +    /* Set CDAT header, Entry = 0 */
> +    cdat_st[0].base = cdat->buf;
> +    cdat_st[0].length = sizeof(CDATTableHeader);
> +    i = 0;
> +
> +    while (i < cdat_st[0].length) {
> +        sum += cdat->buf[i++];
> +    }
> +
> +    /* Read CDAT structures */
> +    while (i < file_size) {
> +        hdr = (CDATSubHeader *)(cdat->buf + i);
> +        cdat_len_check(hdr, errp);

We already did this check the first time through the data...

> +
> +        cdat_st[ent].base = hdr;
> +        cdat_st[ent].length = hdr->length;
> +
> +        while (cdat->buf + i <
> +               (uint8_t *)cdat_st[ent].base + cdat_st[ent].length) {
> +            assert(i < file_size);
> +            sum += cdat->buf[i++];
> +        }
> +
> +        ent++;
> +    }
> +
> +    if (sum != 0) {
> +        warn_report("CDAT: Found checksum mismatch in %s", cdat->filename);
> +    }
> +    cdat->entry_len = num_ent;
> +    cdat->entry = g_steal_pointer(&cdat_st);
> +}
> +
> +void cxl_doe_cdat_init(CXLComponentState *cxl_cstate, Error **errp)
> +{
> +    CDATObject *cdat = &cxl_cstate->cdat;
> +
> +    if (cdat->filename) {
> +        ct3_load_cdat(cdat, errp);
> +    } else {
> +        ct3_build_cdat(cdat, errp);
> +    }
> +}

None of the callsites to this function check for it
failing. In particular they do not assume "if I call
this and it fails then I need to call cxl_doe_cdata_release()
to have it clean up". It would probably be less confusing
if the init function cleans up after itself, i.e. does not
leave allocated memory pointed to by cdat->buf and so on.

thanks
-- PMM


^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 23/83] hw/cxl/cdat: CXL CDAT Data Object Exchange implementation
  2023-04-11 15:52   ` Peter Maydell
@ 2023-04-12 13:08     ` Jonathan Cameron via
  2023-04-12 14:39     ` Jonathan Cameron via
  1 sibling, 0 replies; 151+ messages in thread
From: Jonathan Cameron via @ 2023-04-12 13:08 UTC (permalink / raw)
  To: Peter Maydell
  Cc: Michael S. Tsirkin, qemu-devel, Huai-Cheng Kuo, Chris Browy,
	Ben Widawsky

On Tue, 11 Apr 2023 16:52:58 +0100
Peter Maydell <peter.maydell@linaro.org> wrote:

> On Mon, 7 Nov 2022 at 22:49, Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > From: Huai-Cheng Kuo <hchkuo@avery-design.com.tw>
> >
> > The Data Object Exchange implementation of CXL Coherent Device Attribute
> > Table (CDAT). This implementation is referring to "Coherent Device
> > Attribute Table Specification, Rev. 1.03, July. 2022" and "Compute
> > Express Link Specification, Rev. 3.0, July. 2022"
> >
> > This patch adds core support that will be shared by both
> > end-points and switch port emulation.  
> 
> > +static void ct3_load_cdat(CDATObject *cdat, Error **errp)
> > +{
> > +    g_autofree CDATEntry *cdat_st = NULL;
> > +    uint8_t sum = 0;
> > +    int num_ent;
> > +    int i = 0, ent = 1, file_size = 0;
> > +    CDATSubHeader *hdr;
> > +    FILE *fp = NULL;
> > +
> > +    /* Read CDAT file and create its cache */
> > +    fp = fopen(cdat->filename, "r");
> > +    if (!fp) {
> > +        error_setg(errp, "CDAT: Unable to open file");
> > +        return;
> > +    }
> > +
> > +    fseek(fp, 0, SEEK_END);
> > +    file_size = ftell(fp);  
> 
> Coverity points out that ftell() can fail and return -1...
> 
> > +    fseek(fp, 0, SEEK_SET);
> > +    cdat->buf = g_malloc0(file_size);  
> 
> ...which would cause an attempt to allocate a very large
> amount of memory, since you aren't checking for errors.
> CID 1508185.
> 
> Below, some other issues I saw in a quick scan through the code.
> 
> > +
> > +    if (fread(cdat->buf, file_size, 1, fp) == 0) {
> > +        error_setg(errp, "CDAT: File read failed");
> > +        return;
> > +    }  
> 
> (The issues in this bit of code I've mentioned in a
> different thread.)
> 
> > +
> > +    fclose(fp);
> > +
> > +    if (file_size < sizeof(CDATTableHeader)) {
> > +        error_setg(errp, "CDAT: File too short");
> > +        return;
> > +    }  
> 
> > +    i = sizeof(CDATTableHeader);
> > +    num_ent = 1;
> > +    while (i < file_size) {
> > +        hdr = (CDATSubHeader *)(cdat->buf + i);  
> 
> If the file is not a complete number of records in
> size, then this can index off the end of the buffer.
> You should check for that.
> 
> > +        cdat_len_check(hdr, errp);
> > +        i += hdr->length;
> > +        num_ent++;
> > +    }
> > +    if (i != file_size) {
> > +        error_setg(errp, "CDAT: File length missmatch");  
> 
> Typo: "mismatch"
> 
> > +        return;
> > +    }
> > +
> > +    cdat_st = g_malloc0(sizeof(*cdat_st) * num_ent);  
> 
> To allocate an array of N lots of a structure, use
> g_new0(), which will take care to avoid possible
> overflow in the multiply.
> 
> > +    if (!cdat_st) {
> > +        error_setg(errp, "CDAT: Failed to allocate entry array");  
> 
> g_malloc0() and g_new0() can never fail, so you don't need
> to check for a NULL pointer return.
> 
> > +        return;
> > +    }
> > +
> > +    /* Set CDAT header, Entry = 0 */
> > +    cdat_st[0].base = cdat->buf;
> > +    cdat_st[0].length = sizeof(CDATTableHeader);
> > +    i = 0;
> > +
> > +    while (i < cdat_st[0].length) {
> > +        sum += cdat->buf[i++];
> > +    }
> > +
> > +    /* Read CDAT structures */
> > +    while (i < file_size) {
> > +        hdr = (CDATSubHeader *)(cdat->buf + i);
> > +        cdat_len_check(hdr, errp);  
> 
> We already did this check the first time through the data...
> 
> > +
> > +        cdat_st[ent].base = hdr;
> > +        cdat_st[ent].length = hdr->length;
> > +
> > +        while (cdat->buf + i <
> > +               (uint8_t *)cdat_st[ent].base + cdat_st[ent].length) {
> > +            assert(i < file_size);
> > +            sum += cdat->buf[i++];
> > +        }
> > +
> > +        ent++;
> > +    }
> > +
> > +    if (sum != 0) {
> > +        warn_report("CDAT: Found checksum mismatch in %s", cdat->filename);
> > +    }
> > +    cdat->entry_len = num_ent;
> > +    cdat->entry = g_steal_pointer(&cdat_st);
> > +}
> > +
> > +void cxl_doe_cdat_init(CXLComponentState *cxl_cstate, Error **errp)
> > +{
> > +    CDATObject *cdat = &cxl_cstate->cdat;
> > +
> > +    if (cdat->filename) {
> > +        ct3_load_cdat(cdat, errp);
> > +    } else {
> > +        ct3_build_cdat(cdat, errp);
> > +    }
> > +}  
> 
> None of the callsites to this function check for it
> failing. In particular they do not assume "if I call
> this and it fails then I need to call cxl_doe_cdata_release()
> to have it clean up". It would probably be less confusing
> if the init function cleans up after itself, i.e. does not
> leave allocated memory pointed to by cdat->buf and so on.

Thanks Peter,

I'll wait for the other thread to resolve the follow up with
a patch set cleaning up remaining issues you've pointed out.

Jonathan

> 
> thanks
> -- PMM



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 23/83] hw/cxl/cdat: CXL CDAT Data Object Exchange implementation
  2023-04-11 15:52   ` Peter Maydell
  2023-04-12 13:08     ` Jonathan Cameron via
@ 2023-04-12 14:39     ` Jonathan Cameron via
  1 sibling, 0 replies; 151+ messages in thread
From: Jonathan Cameron via @ 2023-04-12 14:39 UTC (permalink / raw)
  To: Peter Maydell
  Cc: Michael S. Tsirkin, qemu-devel, Huai-Cheng Kuo, Chris Browy,
	Ben Widawsky

On Tue, 11 Apr 2023 16:52:58 +0100
Peter Maydell <peter.maydell@linaro.org> wrote:

> On Mon, 7 Nov 2022 at 22:49, Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > From: Huai-Cheng Kuo <hchkuo@avery-design.com.tw>
> >
> > The Data Object Exchange implementation of CXL Coherent Device Attribute
> > Table (CDAT). This implementation is referring to "Coherent Device
> > Attribute Table Specification, Rev. 1.03, July. 2022" and "Compute
> > Express Link Specification, Rev. 3.0, July. 2022"
> >
> > This patch adds core support that will be shared by both
> > end-points and switch port emulation.  
> 
> > +static void ct3_load_cdat(CDATObject *cdat, Error **errp)
> > +{
> > +    g_autofree CDATEntry *cdat_st = NULL;
> > +    uint8_t sum = 0;
> > +    int num_ent;
> > +    int i = 0, ent = 1, file_size = 0;
> > +    CDATSubHeader *hdr;
> > +    FILE *fp = NULL;
> > +
> > +    /* Read CDAT file and create its cache */
> > +    fp = fopen(cdat->filename, "r");
> > +    if (!fp) {
> > +        error_setg(errp, "CDAT: Unable to open file");
> > +        return;
> > +    }
> > +
> > +    fseek(fp, 0, SEEK_END);
> > +    file_size = ftell(fp);  
> 
> Coverity points out that ftell() can fail and return -1...
> 
> > +    fseek(fp, 0, SEEK_SET);
> > +    cdat->buf = g_malloc0(file_size);  
> 
> ...which would cause an attempt to allocate a very large
> amount of memory, since you aren't checking for errors.
> CID 1508185.
> 
> Below, some other issues I saw in a quick scan through the code.
> 
> > +
> > +    if (fread(cdat->buf, file_size, 1, fp) == 0) {
> > +        error_setg(errp, "CDAT: File read failed");
> > +        return;
> > +    }  
> 
> (The issues in this bit of code I've mentioned in a
> different thread.)

I'll carry a patch locally using g_file_get_contents()
that gets rid of this mess. My assumption being that similar
will emerge from other thread and I can drop my patch.

> 
> > +
> > +    fclose(fp);
> > +
> > +    if (file_size < sizeof(CDATTableHeader)) {
> > +        error_setg(errp, "CDAT: File too short");
> > +        return;
> > +    }  
> 
> > +    i = sizeof(CDATTableHeader);
> > +    num_ent = 1;
> > +    while (i < file_size) {
> > +        hdr = (CDATSubHeader *)(cdat->buf + i);  
> 
> If the file is not a complete number of records in
> size, then this can index off the end of the buffer.
> You should check for that.

> 
> > +        cdat_len_check(hdr, errp);
> > +        i += hdr->length;
> > +        num_ent++;
> > +    }
> > +    if (i != file_size) {
> > +        error_setg(errp, "CDAT: File length missmatch");  
> 
> Typo: "mismatch"
> 

That's fixed in the tree.

> > +        return;
> > +    }
> > +
> > +    cdat_st = g_malloc0(sizeof(*cdat_st) * num_ent);  
> 
> To allocate an array of N lots of a structure, use
> g_new0(), which will take care to avoid possible
> overflow in the multiply.
> 
> > +    if (!cdat_st) {
> > +        error_setg(errp, "CDAT: Failed to allocate entry array");  
> 
> g_malloc0() and g_new0() can never fail, so you don't need
> to check for a NULL pointer return.

dropped.  I'll remember this one day ;)

> 
> > +        return;
> > +    }
> > +
> > +    /* Set CDAT header, Entry = 0 */
> > +    cdat_st[0].base = cdat->buf;
> > +    cdat_st[0].length = sizeof(CDATTableHeader);
> > +    i = 0;
> > +
> > +    while (i < cdat_st[0].length) {
> > +        sum += cdat->buf[i++];
> > +    }
> > +
> > +    /* Read CDAT structures */
> > +    while (i < file_size) {
> > +        hdr = (CDATSubHeader *)(cdat->buf + i);
> > +        cdat_len_check(hdr, errp);  
> 
> We already did this check the first time through the data...
dropped
> 
> > +
> > +        cdat_st[ent].base = hdr;
> > +        cdat_st[ent].length = hdr->length;
> > +
> > +        while (cdat->buf + i <
> > +               (uint8_t *)cdat_st[ent].base + cdat_st[ent].length) {
> > +            assert(i < file_size);
> > +            sum += cdat->buf[i++];
> > +        }
> > +
> > +        ent++;
> > +    }
> > +
> > +    if (sum != 0) {
> > +        warn_report("CDAT: Found checksum mismatch in %s", cdat->filename);
> > +    }
> > +    cdat->entry_len = num_ent;
> > +    cdat->entry = g_steal_pointer(&cdat_st);
> > +}
> > +
> > +void cxl_doe_cdat_init(CXLComponentState *cxl_cstate, Error **errp)
> > +{
> > +    CDATObject *cdat = &cxl_cstate->cdat;
> > +
> > +    if (cdat->filename) {
> > +        ct3_load_cdat(cdat, errp);
> > +    } else {
> > +        ct3_build_cdat(cdat, errp);
> > +    }
> > +}  
> 
> None of the callsites to this function check for it
> failing. In particular they do not assume "if I call
> this and it fails then I need to call cxl_doe_cdata_release()
> to have it clean up". It would probably be less confusing
> if the init function cleans up after itself, i.e. does not
> leave allocated memory pointed to by cdat->buf and so on.
Agreed.  Will make it cleanup and add the error checks at the two
callers.

Thanks,

Jonathan

> 
> thanks
> -- PMM



^ permalink raw reply	[flat|nested] 151+ messages in thread

* Re: [PULL v4 25/83] hw/pci-bridge/cxl-upstream: Add a CDAT table access DOE
  2022-11-07 22:49 ` [PULL v4 25/83] hw/pci-bridge/cxl-upstream: Add a CDAT table access DOE Michael S. Tsirkin
@ 2023-06-23 13:23   ` Peter Maydell
  0 siblings, 0 replies; 151+ messages in thread
From: Peter Maydell @ 2023-06-23 13:23 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: qemu-devel, Jonathan Cameron, Marcel Apfelbaum, Ben Widawsky

On Mon, 7 Nov 2022 at 22:49, Michael S. Tsirkin <mst@redhat.com> wrote:
>
> From: Jonathan Cameron <Jonathan.Cameron@huawei.com>
>
> This Data Object Exchange Mailbox allows software to query the
> latency and bandwidth between ports on the switch. For now
> only provide information on routes between the upstream port and
> each downstream port (not p2p).
>
> Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>

Hi; Coverity points out (CID 1508120) something a bit odd
in this code:

> +static int build_cdat_table(CDATSubHeader ***cdat_table, void *priv)
> +{
> +    g_autofree CDATSslbis *sslbis_latency = NULL;
> +    g_autofree CDATSslbis *sslbis_bandwidth = NULL;
> +    CXLUpstreamPort *us = CXL_USP(priv);
> +    PCIBus *bus = &PCI_BRIDGE(us)->sec_bus;
> +    int devfn, sslbis_size, i;
> +    int count = 0;
> +    uint16_t port_ids[256];

> +    *cdat_table = g_malloc0(sizeof(*cdat_table) * CXL_USP_CDAT_NUM_ENTRIES);

Here:
 - cdat_table has type CDATSubHeader ***
 - so *cdat_table has type CDATSubHeader **
 - so the array we're allocating here should be items of type CDATSubHeader *
 - but we pass sizeof(*cdat_table), which is sizeof(CDATSubHeader **),
   implying that we're allocating an array of CDATSubHeader **

It happens that sizeof(CDATSubHeader **) == sizeof(CDATSubHeader *)
so nothing blows up, but presumably this was supposed to be
sizeof(**cdat_table) ?

Using g_new0() lets you pass in exactly the type you're creating
an array of and avoids having to do the multiplication explicitly.
It also gets the type checking right because the return type
from g_new0(my_type, N) is "my_type *".

  *cdat_table = g_new0(CDATSubHeader *, CXL_USP_CDAT_NUM_ENTRIES);

As a side observation, "Foo ***" is a really awkward type
to be working with and it's not surprising that this kind
of "did I mean to dereference once, twice or three times?"
bug has crept in. Perhaps there's some refactoring that
could avoid that ***...

> +    if (!*cdat_table) {
> +        return -ENOMEM;
> +    }

Also g_malloc0() and g_new0() can never fail, so don't check
them for failure.

thanks
-- PMM


^ permalink raw reply	[flat|nested] 151+ messages in thread

end of thread, other threads:[~2023-06-23 13:23 UTC | newest]

Thread overview: 151+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-07 22:47 [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
2022-11-07 22:47 ` [PULL v4 01/83] hw/i386/e820: remove legacy reserved entries for e820 Michael S. Tsirkin
2022-11-07 22:47 ` [PULL v4 02/83] tests/acpi: allow SSDT changes Michael S. Tsirkin
2022-11-07 22:47 ` [PULL v4 03/83] acpi/ssdt: Fix aml_or() and aml_and() in if clause Michael S. Tsirkin
2022-11-07 22:47 ` [PULL v4 04/83] acpi/nvdimm: define macro for NVDIMM Device _DSM Michael S. Tsirkin
2022-11-07 22:47 ` [PULL v4 05/83] acpi/nvdimm: Implement ACPI NVDIMM Label Methods Michael S. Tsirkin
2022-11-07 22:47 ` [PULL v4 06/83] test/acpi/bios-tables-test: SSDT: update golden master binaries Michael S. Tsirkin
2022-11-07 22:47 ` [PULL v4 07/83] virtio-crypto: Support asynchronous mode Michael S. Tsirkin
2022-11-07 22:48 ` [PULL v4 08/83] crypto: Support DER encodings Michael S. Tsirkin
2022-11-07 22:48 ` [PULL v4 09/83] crypto: Support export akcipher to pkcs8 Michael S. Tsirkin
2022-11-07 22:48 ` [PULL v4 10/83] cryptodev: Add a lkcf-backend for cryptodev Michael S. Tsirkin
2022-11-07 22:48 ` [PULL v4 11/83] acpi/tests/avocado/bits: initial commit of test scripts that are run by biosbits Michael S. Tsirkin
2022-11-07 22:48 ` [PULL v4 12/83] acpi/tests/avocado/bits: disable acpi PSS tests that are failing in biosbits Michael S. Tsirkin
2022-11-07 22:48 ` [PULL v4 13/83] acpi/tests/avocado/bits: add biosbits config file for running bios tests Michael S. Tsirkin
2022-11-07 22:48 ` [PULL v4 14/83] acpi/tests/avocado/bits: add acpi and smbios avocado tests that uses biosbits Michael S. Tsirkin
2022-11-07 22:48 ` [PULL v4 15/83] acpi/tests/avocado/bits/doc: add a doc file to describe the acpi bits test Michael S. Tsirkin
2022-11-07 22:48 ` [PULL v4 16/83] MAINTAINERS: add myself as the maintainer for acpi biosbits avocado tests Michael S. Tsirkin
2022-11-07 22:48 ` [PULL v4 17/83] tests/acpi: virt: allow acpi MADT and FADT changes Michael S. Tsirkin
2022-11-07 22:49 ` [PULL v4 18/83] acpi: fadt: support revision 6.0 of the ACPI specification Michael S. Tsirkin
2022-11-07 22:49 ` [PULL v4 19/83] acpi: arm/virt: madt: bump to revision 4 accordingly to ACPI 6.0 Errata A Michael S. Tsirkin
2022-11-07 22:49 ` [PULL v4 20/83] tests/acpi: virt: update ACPI MADT and FADT binaries Michael S. Tsirkin
2022-11-07 22:49 ` [PULL v4 21/83] hw/pci: PCIe Data Object Exchange emulation Michael S. Tsirkin
2022-11-07 22:49 ` [PULL v4 22/83] hw/mem/cxl-type3: Add MSIX support Michael S. Tsirkin
2022-11-07 22:49 ` [PULL v4 23/83] hw/cxl/cdat: CXL CDAT Data Object Exchange implementation Michael S. Tsirkin
2023-04-11 15:52   ` Peter Maydell
2023-04-12 13:08     ` Jonathan Cameron via
2023-04-12 14:39     ` Jonathan Cameron via
2022-11-07 22:49 ` [PULL v4 24/83] hw/mem/cxl-type3: Add CXL CDAT Data Object Exchange Michael S. Tsirkin
2022-11-07 22:49 ` [PULL v4 25/83] hw/pci-bridge/cxl-upstream: Add a CDAT table access DOE Michael S. Tsirkin
2023-06-23 13:23   ` Peter Maydell
2022-11-07 22:49 ` [PULL v4 26/83] hw/virtio/virtio-iommu-pci: Enforce the device is plugged on the root bus Michael S. Tsirkin
2022-11-07 22:49 ` [PULL v4 27/83] virtio: introduce __virtio_queue_reset() Michael S. Tsirkin
2022-11-07 22:49 ` [PULL v4 28/83] virtio: introduce virtio_queue_reset() Michael S. Tsirkin
2022-11-07 22:49 ` [PULL v4 29/83] virtio: introduce virtio_queue_enable() Michael S. Tsirkin
2022-11-08 19:43   ` Stefan Hajnoczi
2022-11-09  3:31     ` Jason Wang
2022-11-09  6:39       ` Michael S. Tsirkin
2022-11-09  6:48         ` Xuan Zhuo
2022-11-09  7:01           ` Michael S. Tsirkin
2022-11-09  7:11             ` Xuan Zhuo
2022-11-09  6:51       ` Michael S. Tsirkin
2022-11-09  6:55         ` Jason Wang
2022-11-09  6:56           ` Xuan Zhuo
2022-11-09  7:04             ` Michael S. Tsirkin
2022-11-09  7:12               ` Xuan Zhuo
2022-11-09  6:59         ` Michael S. Tsirkin
2022-11-09  8:56           ` Laszlo Ersek
2022-11-09 11:00             ` Michael S. Tsirkin
2022-11-07 22:50 ` [PULL v4 30/83] virtio: core: vq reset feature negotation support Michael S. Tsirkin
2022-11-18 14:32   ` Stefano Garzarella
2022-11-18 14:39     ` Stefano Garzarella
2022-11-19 17:19     ` Michael S. Tsirkin
2022-11-21  6:17       ` Jason Wang
2022-11-21  7:07         ` Michael S. Tsirkin
2022-11-21  8:20           ` Stefano Garzarella
2022-11-07 22:50 ` [PULL v4 31/83] virtio-pci: support queue reset Michael S. Tsirkin
2022-11-07 22:50 ` [PULL v4 32/83] virtio-pci: support queue enable Michael S. Tsirkin
2022-11-07 22:50 ` [PULL v4 33/83] vhost: expose vhost_virtqueue_start() Michael S. Tsirkin
2022-11-07 22:50 ` [PULL v4 34/83] vhost: expose vhost_virtqueue_stop() Michael S. Tsirkin
2022-11-07 22:50 ` [PULL v4 35/83] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_reset() Michael S. Tsirkin
2022-11-07 22:50 ` [PULL v4 36/83] vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart() Michael S. Tsirkin
2022-11-07 22:50 ` [PULL v4 37/83] virtio-net: introduce flush_or_purge_queued_packets() Michael S. Tsirkin
2022-11-07 22:50 ` [PULL v4 38/83] virtio-net: support queue reset Michael S. Tsirkin
2022-11-07 22:50 ` [PULL v4 39/83] virtio-net: support queue_enable Michael S. Tsirkin
2022-11-07 22:50 ` [PULL v4 40/83] vhost: vhost-kernel: enable vq reset feature Michael S. Tsirkin
2022-11-07 22:50 ` [PULL v4 41/83] virtio-net: " Michael S. Tsirkin
2022-11-07 22:50 ` [PULL v4 42/83] virtio-rng-pci: Allow setting nvectors, so we can use MSI-X Michael S. Tsirkin
2023-01-09 10:20   ` Dr. David Alan Gilbert
2023-01-09 10:38     ` Michael S. Tsirkin
2023-01-09 10:43       ` Dr. David Alan Gilbert
2022-11-07 22:51 ` [PULL v4 43/83] vhost-user: Fix out of order vring host notification handling Michael S. Tsirkin
2022-11-07 22:51 ` [PULL v4 44/83] acpi: pc: vga: use AcpiDevAmlIf interface to build VGA device descriptors Michael S. Tsirkin
2022-11-09 17:39   ` Laurent Vivier
2022-11-09 21:42     ` Michael S. Tsirkin
2022-11-10  9:28       ` Peter Maydell
2022-11-11  9:43         ` Igor Mammedov
2022-11-11 10:25           ` Igor Mammedov
2022-11-10  2:55     ` Ani Sinha
2022-11-07 22:51 ` [PULL v4 45/83] tests: acpi: whitelist DSDT before generating PCI-ISA bridge AML automatically Michael S. Tsirkin
2022-11-08 13:36   ` Igor Mammedov
2022-11-08 13:39     ` Ani Sinha
2022-11-08 14:06       ` Ani Sinha
2022-11-08 13:49     ` Michael S. Tsirkin
2022-11-08 14:31       ` Igor Mammedov
2022-11-07 22:51 ` [PULL v4 46/83] acpi: pc/q35: drop ad-hoc PCI-ISA bridge AML routines and let bus ennumeration generate AML Michael S. Tsirkin
2022-11-17 21:51   ` Volker Rümelin
2022-11-18 13:08     ` Igor Mammedov
2022-11-18 14:55       ` Igor Mammedov
2022-11-19  8:49         ` Volker Rümelin
2022-11-21  7:27           ` Igor Mammedov
2022-11-19 17:22         ` Michael S. Tsirkin
2022-11-21  7:23           ` Igor Mammedov
2022-11-21  7:50             ` Michael S. Tsirkin
2022-11-07 22:51 ` [PULL v4 47/83] tests: acpi: update expected DSDT after ISA bridge is moved directly under PCI host bridge Michael S. Tsirkin
2022-11-07 22:51 ` [PULL v4 48/83] tests: acpi: whitelist DSDT before generating ICH9_SMB AML automatically Michael S. Tsirkin
2022-11-07 22:51 ` [PULL v4 49/83] acpi: add get_dev_aml_func() helper Michael S. Tsirkin
2022-11-07 22:51 ` [PULL v4 50/83] acpi: enumerate SMB bridge automatically along with other PCI devices Michael S. Tsirkin
2022-11-07 22:51 ` [PULL v4 51/83] tests: acpi: update expected blobs Michael S. Tsirkin
2022-11-07 22:51 ` [PULL v4 52/83] tests: acpi: pc/q35 whitelist DSDT before \_GPE cleanup Michael S. Tsirkin
2022-11-07 22:51 ` [PULL v4 53/83] acpi: pc/35: sanitize _GPE declaration order Michael S. Tsirkin
2022-11-07 22:51 ` [PULL v4 54/83] tests: acpi: update expected blobs Michael S. Tsirkin
2022-11-07 22:51 ` [PULL v4 55/83] hw/acpi/erst.c: Fix memory handling issues Michael S. Tsirkin
2022-11-07 22:52 ` [PULL v4 56/83] MAINTAINERS: Add qapi/virtio.json to section "virtio" Michael S. Tsirkin
2022-11-07 22:52 ` [PULL v4 57/83] msix: Assert that specified vector is in range Michael S. Tsirkin
2022-11-07 22:52 ` [PULL v4 58/83] hw/i386/pc.c: CXL Fixed Memory Window should not reserve e820 in bios Michael S. Tsirkin
2022-11-07 22:52 ` [PULL v4 59/83] hw/i386/acpi-build: Remove unused struct Michael S. Tsirkin
2022-11-07 22:52 ` [PULL v4 60/83] hw/i386/acpi-build: Resolve redundant attribute Michael S. Tsirkin
2022-11-07 22:52 ` [PULL v4 61/83] hw/i386/acpi-build: Resolve north rather than south bridges Michael S. Tsirkin
2022-11-07 22:52 ` [PULL v4 62/83] hmat acpi: Don't require initiator value in -numa Michael S. Tsirkin
2022-11-07 22:52 ` [PULL v4 63/83] tests: acpi: add and whitelist *.hmat-noinitiator expected blobs Michael S. Tsirkin
2022-11-07 22:52 ` [PULL v4 64/83] tests: acpi: q35: add test for hmat nodes without initiators Michael S. Tsirkin
2022-11-07 22:52 ` [PULL v4 65/83] tests: acpi: q35: update expected blobs *.hmat-noinitiators expected HMAT: Michael S. Tsirkin
2022-11-07 22:53 ` [PULL v4 66/83] tests: Add HMAT AArch64/virt empty table files Michael S. Tsirkin
2022-11-07 22:53 ` [PULL v4 67/83] hw/arm/virt: Enable HMAT on arm virt machine Michael S. Tsirkin
2022-11-07 22:53 ` [PULL v4 68/83] tests: acpi: aarch64/virt: add a test for hmat nodes with no initiators Michael S. Tsirkin
2022-11-07 22:53 ` [PULL v4 69/83] tests: virt: Update expected *.acpihmatvirt tables Michael S. Tsirkin
2022-11-07 22:53 ` [PULL v4 70/83] vfio: move implement of vfio_get_xlat_addr() to memory.c Michael S. Tsirkin
2022-11-07 22:53 ` [PULL v4 71/83] intel-iommu: don't warn guest errors when getting rid2pasid entry Michael S. Tsirkin
2022-11-07 22:53 ` [PULL v4 72/83] intel-iommu: drop VTDBus Michael S. Tsirkin
2022-11-07 22:53 ` [PULL v4 73/83] intel-iommu: convert VTD_PE_GET_FPD_ERR() to be a function Michael S. Tsirkin
2022-11-07 22:53 ` [PULL v4 74/83] intel-iommu: PASID support Michael S. Tsirkin
2023-04-06 16:22   ` Peter Maydell
2023-04-10  2:55     ` Jason Wang
2022-11-07 22:53 ` [PULL v4 75/83] vhost: Change the sequence of device start Michael S. Tsirkin
2022-11-07 22:53 ` [PULL v4 76/83] vhost-user: Support vhost_dev_start Michael S. Tsirkin
2023-01-06 14:21   ` Laurent Vivier
2023-01-09 10:55     ` Michael S. Tsirkin
2023-01-11  9:50       ` Laurent Vivier
2023-01-12  5:46         ` Yajun Wu
2023-01-12  9:25         ` Maxime Coquelin
2023-01-16  7:14           ` Yajun Wu
2023-01-17  9:49             ` Maxime Coquelin
2023-01-17 12:12               ` Greg Kurz
2023-01-17 12:36                 ` Greg Kurz
2023-01-17 15:07                   ` Maxime Coquelin
2023-01-17 17:55                     ` Greg Kurz
2023-01-17 18:21                       ` Greg Kurz
2023-01-18 11:01                         ` Michael S. Tsirkin
2023-01-19 19:48                   ` Dr. David Alan Gilbert
2022-11-07 22:53 ` [PULL v4 77/83] hw/smbios: add core_count2 to smbios table type 4 Michael S. Tsirkin
2022-11-07 22:54 ` [PULL v4 78/83] bios-tables-test: teach test to use smbios 3.0 tables Michael S. Tsirkin
2022-11-07 22:54 ` [PULL v4 79/83] tests/acpi: allow changes for core_count2 test Michael S. Tsirkin
2022-11-07 22:54 ` [PULL v4 80/83] bios-tables-test: add test for number of cores > 255 Michael S. Tsirkin
2022-11-07 22:54 ` [PULL v4 81/83] tests/acpi: update tables for new core count test Michael S. Tsirkin
2022-11-07 22:54 ` [PULL v4 82/83] hw/virtio: introduce virtio_device_should_start Michael S. Tsirkin
2022-11-07 22:54   ` [Virtio-fs] " Michael S. Tsirkin
2022-11-07 22:54 ` [PULL v4 83/83] checkpatch: better pattern for inline comments Michael S. Tsirkin
2022-11-08  6:23 ` [PULL v4 00/83] pci,pc,virtio: features, tests, fixes, cleanups Michael S. Tsirkin
2022-11-08 13:47   ` Stefan Hajnoczi
2022-11-15 14:01 ` Philippe Mathieu-Daudé
2022-11-16 10:42   ` Igor Mammedov

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.