* [Qemu-devel] [PATCH] docs: updates on the POWER9 XIVE interrupt controller documentation
@ 2019-06-12 16:04 Cédric Le Goater
2019-06-12 23:59 ` David Gibson
0 siblings, 1 reply; 2+ messages in thread
From: Cédric Le Goater @ 2019-06-12 16:04 UTC (permalink / raw)
To: David Gibson
Cc: Alexey Kardashevskiy, Cédric Le Goater, qemu-ppc, Greg Kurz,
qemu-devel
This includes various small updates and a better description of the
chosen interrupt mode resulting from the combination of the 'ic-mode'
machine option, the 'kernel_irqchip' option, guest support and KVM
support.
Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
David, please check the tables below. Thanks
docs/specs/ppc-spapr-xive.rst | 112 ++++++++++++++++++++++++++++++++--
docs/specs/ppc-xive.rst | 9 +--
2 files changed, 111 insertions(+), 10 deletions(-)
diff --git a/docs/specs/ppc-spapr-xive.rst b/docs/specs/ppc-spapr-xive.rst
index 539ce7ca4e90..7a64c9d04951 100644
--- a/docs/specs/ppc-spapr-xive.rst
+++ b/docs/specs/ppc-spapr-xive.rst
@@ -34,19 +34,118 @@ CAS Negotiation
---------------
QEMU advertises the supported interrupt modes in the device tree
-property "ibm,arch-vec-5-platform-support" in byte 23 and the OS
-Selection for XIVE is indicated in the "ibm,architecture-vec-5"
+property ``ibm,arch-vec-5-platform-support`` in byte 23 and the OS
+Selection for XIVE is indicated in the ``ibm,architecture-vec-5``
property byte 23.
The interrupt modes supported by the machine depend on the CPU type
(POWER9 is required for XIVE) but also on the machine property
``ic-mode`` which can be set on the command line. It can take the
-following values: ``xics``, ``xive``, ``dual`` and currently ``xics``
-is the default but it may change in the future.
+following values: ``xics``, ``xive``, and ``dual`` which is the
+default mode. ``dual`` means that both modes XICS **and** XIVE are
+supported and if the guest OS supports XIVE, this mode will be
+selected.
The choosen interrupt mode is activated after a reconfiguration done
in a machine reset.
+KVM negotiation
+---------------
+
+When the guest starts under KVM, the capabilities of the host kernel
+and QEMU are also negotiated. Depending on the version of the host
+kernel, KVM will advertise the XIVE capability to QEMU or not.
+
+Nevertheless, the available interrupt modes in the machine should not
+depend on the XIVE KVM capability of the host. On older kernels
+without XIVE KVM support, QEMU will use the emulated XIVE device as a
+fallback and on newer kernels (>=5.2), the KVM XIVE device.
+
+As a final refinement, the user can also switch the use of the KVM
+device with the machine option ``kernel_irqchip``.
+
+
+XIVE support in KVM
+~~~~~~~~~~~~~~~~~~~
+
+For guest OSes supporting XIVE, the resulting interrupt modes on host
+kernels with XIVE KVM support are the following:
+
+============== ============= ============= ================
+ic-mode kernel_irqchip
+-------------- ----------------------------------------------
+/ allowed off on
+ (default)
+============== ============= ============= ================
+dual (default) XIVE KVM XIVE emul. XIVE KVM
+xive XIVE KVM XIVE emul. XIVE KVM
+xics XICS KVM XICS emul. XICS KVM
+============== ============= ============= ================
+
+For legacy guest OSes without XIVE support, the resulting interrupt
+modes are the following:
+
+============== ============= ============= ================
+ic-mode kernel_irqchip
+-------------- ----------------------------------------------
+/ allowed off on
+ (default)
+============== ============= ============= ================
+dual (default) XICS KVM XICS emul. XICS KVM
+xive QEMU error(3) QEMU error(3) QEMU error(3)
+xics XICS KVM XICS emul. XICS KVM
+============== ============= ============= ================
+
+(3) QEMU fails at CAS with ``Guest requested unavailable interrupt
+ mode (XICS), either don't set the ic-mode machine property or try
+ ic-mode=xics or ic-mode=dual``
+
+
+No XIVE support in KVM
+~~~~~~~~~~~~~~~~~~~~~~
+
+For guest OSes supporting XIVE, the resulting interrupt modes on host
+kernels without XIVE KVM support are the following:
+
+============== ============= ============= ================
+ic-mode kernel_irqchip
+-------------- ----------------------------------------------
+/ allowed off on
+ (default)
+============== ============= ============= ================
+dual (default) XIVE emul.(1) XIVE emul. QEMU error (2)
+xive XIVE emul.(1) XIVE emul. QEMU error (2)
+xics XICS KVM XICS emul. XICS KVM
+============== ============= ============= ================
+
+
+(1) QEMU warns with ``warning: kernel_irqchip requested but unavailable:
+ IRQ_XIVE capability must be present for KVM``
+(2) QEMU fails with ``kernel_irqchip requested but unavailable:
+ IRQ_XIVE capability must be present for KVM``
+
+
+For legacy guest OSes without XIVE support, the resulting interrupt
+modes are the following:
+
+============== ============= ============= ================
+ic-mode kernel_irqchip
+-------------- ----------------------------------------------
+/ allowed off on
+ (default)
+============== ============= ============= ================
+dual (default) QEMU error(4) XICS emul. QEMU error(4)
+xive QEMU error(3) QEMU error(3) QEMU error(3)
+xics XICS KVM XICS emul. XICS KVM
+============== ============= ============= ================
+
+(3) QEMU fails at CAS with ``Guest requested unavailable interrupt
+ mode (XICS), either don't set the ic-mode machine property or try
+ ic-mode=xics or ic-mode=dual``
+(4) QEMU/KVM incompatibility due to device destruction in reset. This
+ needs to be addressed more cleanly with an error.
+
+
XIVE Device tree properties
---------------------------
@@ -92,10 +191,11 @@ for both interrupt mode. The different ranges are defined as follow :
- ``0x0000 .. 0x0FFF`` 4K CPU IPIs (only used under XIVE)
- ``0x1000 .. 0x1000`` 1 EPOW
- ``0x1001 .. 0x1001`` 1 HOTPLUG
+- ``0x1002 .. 0x10FF`` unused
- ``0x1100 .. 0x11FF`` 256 VIO devices
-- ``0x1200 .. 0x127F`` 32 PHBs devices
+- ``0x1200 .. 0x127F`` 32x4 LSIs for PHB devices
- ``0x1280 .. 0x12FF`` unused
-- ``0x1300 .. 0x1FFF`` PHB MSIs
+- ``0x1300 .. 0x1FFF`` PHB MSIs (dynamically allocated)
Monitoring XIVE
---------------
diff --git a/docs/specs/ppc-xive.rst b/docs/specs/ppc-xive.rst
index b997dc062910..148d57eb6ab2 100644
--- a/docs/specs/ppc-xive.rst
+++ b/docs/specs/ppc-xive.rst
@@ -20,10 +20,11 @@ The XIVE IC is composed of three sub-engines, each taking care of a
processing layer of external interrupts:
- Interrupt Virtualization Source Engine (IVSE), or Source Controller
- (SC). These are found in PCI PHBs, in the PSI host bridge
- controller, but also inside the main controller for the core IPIs
- and other sub-chips (NX, CAP, NPU) of the chip/processor. They are
- configured to feed the IVRE with events.
+ (SC). These are found in PCI PHBs, in the Processor Service
+ Interface (PSI) host bridge Controller, but also inside the main
+ controller for the core IPIs and other sub-chips (NX, CAP, NPU) of
+ the chip/processor. They are configured to feed the IVRE with
+ events.
- Interrupt Virtualization Routing Engine (IVRE) or Virtualization
Controller (VC). It handles event coalescing and perform interrupt
routing by matching an event source number with an Event
--
2.21.0
^ permalink raw reply related [flat|nested] 2+ messages in thread
* Re: [Qemu-devel] [PATCH] docs: updates on the POWER9 XIVE interrupt controller documentation
2019-06-12 16:04 [Qemu-devel] [PATCH] docs: updates on the POWER9 XIVE interrupt controller documentation Cédric Le Goater
@ 2019-06-12 23:59 ` David Gibson
0 siblings, 0 replies; 2+ messages in thread
From: David Gibson @ 2019-06-12 23:59 UTC (permalink / raw)
To: Cédric Le Goater
Cc: Alexey Kardashevskiy, qemu-ppc, Greg Kurz, qemu-devel
[-- Attachment #1: Type: text/plain, Size: 8331 bytes --]
On Wed, Jun 12, 2019 at 06:04:25PM +0200, Cédric Le Goater wrote:
> This includes various small updates and a better description of the
> chosen interrupt mode resulting from the combination of the 'ic-mode'
> machine option, the 'kernel_irqchip' option, guest support and KVM
> support.
>
> Signed-off-by: Cédric Le Goater <clg@kaod.org>
> ---
>
> David, please check the tables below. Thanks
LGTM, applied.
>
> docs/specs/ppc-spapr-xive.rst | 112 ++++++++++++++++++++++++++++++++--
> docs/specs/ppc-xive.rst | 9 +--
> 2 files changed, 111 insertions(+), 10 deletions(-)
>
> diff --git a/docs/specs/ppc-spapr-xive.rst b/docs/specs/ppc-spapr-xive.rst
> index 539ce7ca4e90..7a64c9d04951 100644
> --- a/docs/specs/ppc-spapr-xive.rst
> +++ b/docs/specs/ppc-spapr-xive.rst
> @@ -34,19 +34,118 @@ CAS Negotiation
> ---------------
>
> QEMU advertises the supported interrupt modes in the device tree
> -property "ibm,arch-vec-5-platform-support" in byte 23 and the OS
> -Selection for XIVE is indicated in the "ibm,architecture-vec-5"
> +property ``ibm,arch-vec-5-platform-support`` in byte 23 and the OS
> +Selection for XIVE is indicated in the ``ibm,architecture-vec-5``
> property byte 23.
>
> The interrupt modes supported by the machine depend on the CPU type
> (POWER9 is required for XIVE) but also on the machine property
> ``ic-mode`` which can be set on the command line. It can take the
> -following values: ``xics``, ``xive``, ``dual`` and currently ``xics``
> -is the default but it may change in the future.
> +following values: ``xics``, ``xive``, and ``dual`` which is the
> +default mode. ``dual`` means that both modes XICS **and** XIVE are
> +supported and if the guest OS supports XIVE, this mode will be
> +selected.
>
> The choosen interrupt mode is activated after a reconfiguration done
> in a machine reset.
>
> +KVM negotiation
> +---------------
> +
> +When the guest starts under KVM, the capabilities of the host kernel
> +and QEMU are also negotiated. Depending on the version of the host
> +kernel, KVM will advertise the XIVE capability to QEMU or not.
> +
> +Nevertheless, the available interrupt modes in the machine should not
> +depend on the XIVE KVM capability of the host. On older kernels
> +without XIVE KVM support, QEMU will use the emulated XIVE device as a
> +fallback and on newer kernels (>=5.2), the KVM XIVE device.
> +
> +As a final refinement, the user can also switch the use of the KVM
> +device with the machine option ``kernel_irqchip``.
> +
> +
> +XIVE support in KVM
> +~~~~~~~~~~~~~~~~~~~
> +
> +For guest OSes supporting XIVE, the resulting interrupt modes on host
> +kernels with XIVE KVM support are the following:
> +
> +============== ============= ============= ================
> +ic-mode kernel_irqchip
> +-------------- ----------------------------------------------
> +/ allowed off on
> + (default)
> +============== ============= ============= ================
> +dual (default) XIVE KVM XIVE emul. XIVE KVM
> +xive XIVE KVM XIVE emul. XIVE KVM
> +xics XICS KVM XICS emul. XICS KVM
> +============== ============= ============= ================
> +
> +For legacy guest OSes without XIVE support, the resulting interrupt
> +modes are the following:
> +
> +============== ============= ============= ================
> +ic-mode kernel_irqchip
> +-------------- ----------------------------------------------
> +/ allowed off on
> + (default)
> +============== ============= ============= ================
> +dual (default) XICS KVM XICS emul. XICS KVM
> +xive QEMU error(3) QEMU error(3) QEMU error(3)
> +xics XICS KVM XICS emul. XICS KVM
> +============== ============= ============= ================
> +
> +(3) QEMU fails at CAS with ``Guest requested unavailable interrupt
> + mode (XICS), either don't set the ic-mode machine property or try
> + ic-mode=xics or ic-mode=dual``
> +
> +
> +No XIVE support in KVM
> +~~~~~~~~~~~~~~~~~~~~~~
> +
> +For guest OSes supporting XIVE, the resulting interrupt modes on host
> +kernels without XIVE KVM support are the following:
> +
> +============== ============= ============= ================
> +ic-mode kernel_irqchip
> +-------------- ----------------------------------------------
> +/ allowed off on
> + (default)
> +============== ============= ============= ================
> +dual (default) XIVE emul.(1) XIVE emul. QEMU error (2)
> +xive XIVE emul.(1) XIVE emul. QEMU error (2)
> +xics XICS KVM XICS emul. XICS KVM
> +============== ============= ============= ================
> +
> +
> +(1) QEMU warns with ``warning: kernel_irqchip requested but unavailable:
> + IRQ_XIVE capability must be present for KVM``
> +(2) QEMU fails with ``kernel_irqchip requested but unavailable:
> + IRQ_XIVE capability must be present for KVM``
> +
> +
> +For legacy guest OSes without XIVE support, the resulting interrupt
> +modes are the following:
> +
> +============== ============= ============= ================
> +ic-mode kernel_irqchip
> +-------------- ----------------------------------------------
> +/ allowed off on
> + (default)
> +============== ============= ============= ================
> +dual (default) QEMU error(4) XICS emul. QEMU error(4)
> +xive QEMU error(3) QEMU error(3) QEMU error(3)
> +xics XICS KVM XICS emul. XICS KVM
> +============== ============= ============= ================
> +
> +(3) QEMU fails at CAS with ``Guest requested unavailable interrupt
> + mode (XICS), either don't set the ic-mode machine property or try
> + ic-mode=xics or ic-mode=dual``
> +(4) QEMU/KVM incompatibility due to device destruction in reset. This
> + needs to be addressed more cleanly with an error.
> +
> +
> XIVE Device tree properties
> ---------------------------
>
> @@ -92,10 +191,11 @@ for both interrupt mode. The different ranges are defined as follow :
> - ``0x0000 .. 0x0FFF`` 4K CPU IPIs (only used under XIVE)
> - ``0x1000 .. 0x1000`` 1 EPOW
> - ``0x1001 .. 0x1001`` 1 HOTPLUG
> +- ``0x1002 .. 0x10FF`` unused
> - ``0x1100 .. 0x11FF`` 256 VIO devices
> -- ``0x1200 .. 0x127F`` 32 PHBs devices
> +- ``0x1200 .. 0x127F`` 32x4 LSIs for PHB devices
> - ``0x1280 .. 0x12FF`` unused
> -- ``0x1300 .. 0x1FFF`` PHB MSIs
> +- ``0x1300 .. 0x1FFF`` PHB MSIs (dynamically allocated)
>
> Monitoring XIVE
> ---------------
> diff --git a/docs/specs/ppc-xive.rst b/docs/specs/ppc-xive.rst
> index b997dc062910..148d57eb6ab2 100644
> --- a/docs/specs/ppc-xive.rst
> +++ b/docs/specs/ppc-xive.rst
> @@ -20,10 +20,11 @@ The XIVE IC is composed of three sub-engines, each taking care of a
> processing layer of external interrupts:
>
> - Interrupt Virtualization Source Engine (IVSE), or Source Controller
> - (SC). These are found in PCI PHBs, in the PSI host bridge
> - controller, but also inside the main controller for the core IPIs
> - and other sub-chips (NX, CAP, NPU) of the chip/processor. They are
> - configured to feed the IVRE with events.
> + (SC). These are found in PCI PHBs, in the Processor Service
> + Interface (PSI) host bridge Controller, but also inside the main
> + controller for the core IPIs and other sub-chips (NX, CAP, NPU) of
> + the chip/processor. They are configured to feed the IVRE with
> + events.
> - Interrupt Virtualization Routing Engine (IVRE) or Virtualization
> Controller (VC). It handles event coalescing and perform interrupt
> routing by matching an event source number with an Event
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2019-06-13 3:46 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-06-12 16:04 [Qemu-devel] [PATCH] docs: updates on the POWER9 XIVE interrupt controller documentation Cédric Le Goater
2019-06-12 23:59 ` David Gibson
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.