All of lore.kernel.org
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCH v2 0/4] target-arm: Add some omitted dma cache coherency flags
@ 2017-02-09 12:53 Alexander Graf
  2017-02-09 12:53 ` [Qemu-devel] [PATCH v2 1/4] target-arm: Declare virtio-mmio as dma-coherent in dt Alexander Graf
                   ` (6 more replies)
  0 siblings, 7 replies; 13+ messages in thread
From: Alexander Graf @ 2017-02-09 12:53 UTC (permalink / raw)
  To: qemu-devel; +Cc: qemu-arm, mst, Laszlo Ersek, Shannon Zhao, Ard Biesheuvel

ARM is amazing when it comes to cache coherency and VMs. While any sane
architecture allows the host to override the guest's caching attributes,
that's very hard to do on ARM.

That means that the guest may directly access guest memory bypassing the
cache while QEMU happily writes to / reads from cache. The end result is
very nasty, because both sides see very different views of the world.
 
That means that we need to be very cautious to tell guests that devices
that QEMU emulates are going to use data in the cache rather than directly
on memory.

We added this to PCI a while back for DT (5d636e21 "hw/arm/virt: mark the PCIe
host controller as DMA coherent in the DT") and ACPI (bc64b96 "hw/arm/virt-
acpi-build: _CCA attribute is compulsory") but never updated virtio-mmio or
fw-cfg in DT or ACPI tables.

This patch set adds the respective cache coherency flags for them in both DT and
ACPI.

Fortunately, no guests except for Linux 4.9.7 and 4.9.8 are broken because of
this. Upstream realized quickly enough that every user of virtio-mmio out there
describes its cache coherency incorrectly and reverted the patch that would
require said dma coherency flag. But we should be safe for the future and "do
the right thing".

Alexander Graf (4):
  target-arm: Declare virtio-mmio as dma-coherent in dt
  hw/arm/virt: Declare virtio-mmio as dma cache coherent in ACPI
  hw/arm/virt: Declare fwcfg as dma cache coherent in ACPI
  hw/arm/virt: Declare fwcfg as dma cache coherent in dt

 hw/arm/vexpress.c        | 1 +
 hw/arm/virt-acpi-build.c | 2 ++
 hw/arm/virt.c            | 2 ++
 3 files changed, 5 insertions(+)

-- 
2.10.0

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [Qemu-devel] [PATCH v2 1/4] target-arm: Declare virtio-mmio as dma-coherent in dt
  2017-02-09 12:53 [Qemu-devel] [PATCH v2 0/4] target-arm: Add some omitted dma cache coherency flags Alexander Graf
@ 2017-02-09 12:53 ` Alexander Graf
  2017-02-09 12:53 ` [Qemu-devel] [PATCH v2 2/4] hw/arm/virt: Declare virtio-mmio as dma cache coherent in ACPI Alexander Graf
                   ` (5 subsequent siblings)
  6 siblings, 0 replies; 13+ messages in thread
From: Alexander Graf @ 2017-02-09 12:53 UTC (permalink / raw)
  To: qemu-devel; +Cc: qemu-arm, mst, Laszlo Ersek, Shannon Zhao, Ard Biesheuvel

QEMU emulated hardware is always dma coherent with its guest. We do
annotate that correctly on the PCI host controller, but left out
virtio-mmio.

Recent kernels have started to interpret that flag rather than take
dma coherency as granted with virtio-mmio. While that is considered
a kernel bug, as it breaks previously working systems, it showed that
our dt description is incomplete.

This patch adds the respective marker that allows guest OSs to evaluate
that our virtio-mmio devices are indeed cache coherent.

Signed-off-by: Alexander Graf <agraf@suse.de>
---
 hw/arm/vexpress.c | 1 +
 hw/arm/virt.c     | 1 +
 2 files changed, 2 insertions(+)

diff --git a/hw/arm/vexpress.c b/hw/arm/vexpress.c
index 58760f4..e057568 100644
--- a/hw/arm/vexpress.c
+++ b/hw/arm/vexpress.c
@@ -452,6 +452,7 @@ static int add_virtio_mmio_node(void *fdt, uint32_t acells, uint32_t scells,
                                        acells, addr, scells, size);
     qemu_fdt_setprop_cells(fdt, nodename, "interrupt-parent", intc);
     qemu_fdt_setprop_cells(fdt, nodename, "interrupts", 0, irq, 1);
+    qemu_fdt_setprop(fdt, nodename, "dma-coherent", NULL, 0);
     g_free(nodename);
     if (rc) {
         return -1;
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 1f216cf..14881fa 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -797,6 +797,7 @@ static void create_virtio_devices(const VirtMachineState *vms, qemu_irq *pic)
         qemu_fdt_setprop_cells(vms->fdt, nodename, "interrupts",
                                GIC_FDT_IRQ_TYPE_SPI, irq,
                                GIC_FDT_IRQ_FLAGS_EDGE_LO_HI);
+        qemu_fdt_setprop(vms->fdt, nodename, "dma-coherent", NULL, 0);
         g_free(nodename);
     }
 }
-- 
2.10.0

^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [Qemu-devel] [PATCH v2 2/4] hw/arm/virt: Declare virtio-mmio as dma cache coherent in ACPI
  2017-02-09 12:53 [Qemu-devel] [PATCH v2 0/4] target-arm: Add some omitted dma cache coherency flags Alexander Graf
  2017-02-09 12:53 ` [Qemu-devel] [PATCH v2 1/4] target-arm: Declare virtio-mmio as dma-coherent in dt Alexander Graf
@ 2017-02-09 12:53 ` Alexander Graf
  2017-02-10  2:49   ` Shannon Zhao
  2017-02-09 12:53 ` [Qemu-devel] [PATCH v2 3/4] hw/arm/virt: Declare fwcfg " Alexander Graf
                   ` (4 subsequent siblings)
  6 siblings, 1 reply; 13+ messages in thread
From: Alexander Graf @ 2017-02-09 12:53 UTC (permalink / raw)
  To: qemu-devel; +Cc: qemu-arm, mst, Laszlo Ersek, Shannon Zhao, Ard Biesheuvel

Virtio-mmio devices can directly access guest memory and do so in cache
coherent fashion. Tell the guest about that fact when it's using ACPI.

Signed-off-by: Alexander Graf <agraf@suse.de>
---
 hw/arm/virt-acpi-build.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
index 07a10ac..8955a9d 100644
--- a/hw/arm/virt-acpi-build.c
+++ b/hw/arm/virt-acpi-build.c
@@ -135,6 +135,7 @@ static void acpi_dsdt_add_virtio(Aml *scope,
         Aml *dev = aml_device("VR%02u", i);
         aml_append(dev, aml_name_decl("_HID", aml_string("LNRO0005")));
         aml_append(dev, aml_name_decl("_UID", aml_int(i)));
+        aml_append(dev, aml_name_decl("_CCA", aml_int(1)));
 
         Aml *crs = aml_resource_template();
         aml_append(crs, aml_memory32_fixed(base, size, AML_READ_WRITE));
-- 
2.10.0

^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [Qemu-devel] [PATCH v2 3/4] hw/arm/virt: Declare fwcfg as dma cache coherent in ACPI
  2017-02-09 12:53 [Qemu-devel] [PATCH v2 0/4] target-arm: Add some omitted dma cache coherency flags Alexander Graf
  2017-02-09 12:53 ` [Qemu-devel] [PATCH v2 1/4] target-arm: Declare virtio-mmio as dma-coherent in dt Alexander Graf
  2017-02-09 12:53 ` [Qemu-devel] [PATCH v2 2/4] hw/arm/virt: Declare virtio-mmio as dma cache coherent in ACPI Alexander Graf
@ 2017-02-09 12:53 ` Alexander Graf
  2017-02-10  2:49   ` Shannon Zhao
  2017-02-09 12:53 ` [Qemu-devel] [PATCH v2 4/4] hw/arm/virt: Declare fwcfg as dma cache coherent in dt Alexander Graf
                   ` (3 subsequent siblings)
  6 siblings, 1 reply; 13+ messages in thread
From: Alexander Graf @ 2017-02-09 12:53 UTC (permalink / raw)
  To: qemu-devel; +Cc: qemu-arm, mst, Laszlo Ersek, Shannon Zhao, Ard Biesheuvel

Fw-cfg recently learned how to directly access guest memory and does so in
cache coherent fashion. Tell the guest about that fact when it's using ACPI.

Signed-off-by: Alexander Graf <agraf@suse.de>
---
 hw/arm/virt-acpi-build.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
index 8955a9d..0835e59 100644
--- a/hw/arm/virt-acpi-build.c
+++ b/hw/arm/virt-acpi-build.c
@@ -90,6 +90,7 @@ static void acpi_dsdt_add_fw_cfg(Aml *scope, const MemMapEntry *fw_cfg_memmap)
     aml_append(dev, aml_name_decl("_HID", aml_string("QEMU0002")));
     /* device present, functioning, decoding, not shown in UI */
     aml_append(dev, aml_name_decl("_STA", aml_int(0xB)));
+    aml_append(dev, aml_name_decl("_CCA", aml_int(1)));
 
     Aml *crs = aml_resource_template();
     aml_append(crs, aml_memory32_fixed(fw_cfg_memmap->base,
-- 
2.10.0

^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [Qemu-devel] [PATCH v2 4/4] hw/arm/virt: Declare fwcfg as dma cache coherent in dt
  2017-02-09 12:53 [Qemu-devel] [PATCH v2 0/4] target-arm: Add some omitted dma cache coherency flags Alexander Graf
                   ` (2 preceding siblings ...)
  2017-02-09 12:53 ` [Qemu-devel] [PATCH v2 3/4] hw/arm/virt: Declare fwcfg " Alexander Graf
@ 2017-02-09 12:53 ` Alexander Graf
  2017-02-09 13:15 ` [Qemu-devel] [PATCH v2 0/4] target-arm: Add some omitted dma cache coherency flags Laszlo Ersek
                   ` (2 subsequent siblings)
  6 siblings, 0 replies; 13+ messages in thread
From: Alexander Graf @ 2017-02-09 12:53 UTC (permalink / raw)
  To: qemu-devel; +Cc: qemu-arm, mst, Laszlo Ersek, Shannon Zhao, Ard Biesheuvel

Fw-cfg recently learned how to directly access guest memory and does so in
cache coherent fashion. Tell the guest about that fact when it's using DT.

Signed-off-by: Alexander Graf <agraf@suse.de>
---
 hw/arm/virt.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 14881fa..da9622f 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -929,6 +929,7 @@ static FWCfgState *create_fw_cfg(const VirtMachineState *vms, AddressSpace *as)
                             "compatible", "qemu,fw-cfg-mmio");
     qemu_fdt_setprop_sized_cells(vms->fdt, nodename, "reg",
                                  2, base, 2, size);
+    qemu_fdt_setprop(vms->fdt, nodename, "dma-coherent", NULL, 0);
     g_free(nodename);
     return fw_cfg;
 }
-- 
2.10.0

^ permalink raw reply related	[flat|nested] 13+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/4] target-arm: Add some omitted dma cache coherency flags
  2017-02-09 12:53 [Qemu-devel] [PATCH v2 0/4] target-arm: Add some omitted dma cache coherency flags Alexander Graf
                   ` (3 preceding siblings ...)
  2017-02-09 12:53 ` [Qemu-devel] [PATCH v2 4/4] hw/arm/virt: Declare fwcfg as dma cache coherent in dt Alexander Graf
@ 2017-02-09 13:15 ` Laszlo Ersek
  2017-02-09 18:13   ` Michael S. Tsirkin
  2017-02-09 17:51 ` Ard Biesheuvel
  2017-02-10 15:02 ` [Qemu-devel] [Qemu-arm] " Peter Maydell
  6 siblings, 1 reply; 13+ messages in thread
From: Laszlo Ersek @ 2017-02-09 13:15 UTC (permalink / raw)
  To: Alexander Graf, qemu-devel; +Cc: qemu-arm, mst, Shannon Zhao, Ard Biesheuvel

On 02/09/17 13:53, Alexander Graf wrote:
> ARM is amazing when it comes to cache coherency and VMs. While any sane
> architecture allows the host to override the guest's caching attributes,
> that's very hard to do on ARM.
> 
> That means that the guest may directly access guest memory bypassing the
> cache while QEMU happily writes to / reads from cache. The end result is
> very nasty, because both sides see very different views of the world.
>  
> That means that we need to be very cautious to tell guests that devices
> that QEMU emulates are going to use data in the cache rather than directly
> on memory.
> 
> We added this to PCI a while back for DT (5d636e21 "hw/arm/virt: mark the PCIe
> host controller as DMA coherent in the DT") and ACPI (bc64b96 "hw/arm/virt-
> acpi-build: _CCA attribute is compulsory") but never updated virtio-mmio or
> fw-cfg in DT or ACPI tables.
> 
> This patch set adds the respective cache coherency flags for them in both DT and
> ACPI.
> 
> Fortunately, no guests except for Linux 4.9.7 and 4.9.8 are broken because of
> this. Upstream realized quickly enough that every user of virtio-mmio out there
> describes its cache coherency incorrectly and reverted the patch that would
> require said dma coherency flag. But we should be safe for the future and "do
> the right thing".
> 
> Alexander Graf (4):
>   target-arm: Declare virtio-mmio as dma-coherent in dt
>   hw/arm/virt: Declare virtio-mmio as dma cache coherent in ACPI
>   hw/arm/virt: Declare fwcfg as dma cache coherent in ACPI
>   hw/arm/virt: Declare fwcfg as dma cache coherent in dt
> 
>  hw/arm/vexpress.c        | 1 +
>  hw/arm/virt-acpi-build.c | 2 ++
>  hw/arm/virt.c            | 2 ++
>  3 files changed, 5 insertions(+)
> 

Famous last words:
series
Reviewed-by: Laszlo Ersek <lersek@redhat.com>

Should we replicate patch #3 to QEMU0002 / FWCF in
"hw/i386/acpi-build.c" too? Or is it that we couldn't care less about
_CCA on x86? :) (Can't really muster the energy right now to look it up
in the ACPI spec, sorry!)

Thanks
Laszlo

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/4] target-arm: Add some omitted dma cache coherency flags
  2017-02-09 12:53 [Qemu-devel] [PATCH v2 0/4] target-arm: Add some omitted dma cache coherency flags Alexander Graf
                   ` (4 preceding siblings ...)
  2017-02-09 13:15 ` [Qemu-devel] [PATCH v2 0/4] target-arm: Add some omitted dma cache coherency flags Laszlo Ersek
@ 2017-02-09 17:51 ` Ard Biesheuvel
  2017-02-10 15:02 ` [Qemu-devel] [Qemu-arm] " Peter Maydell
  6 siblings, 0 replies; 13+ messages in thread
From: Ard Biesheuvel @ 2017-02-09 17:51 UTC (permalink / raw)
  To: Alexander Graf
  Cc: QEMU Developers, qemu-arm, Michael S. Tsirkin, Laszlo Ersek,
	Shannon Zhao

On 9 February 2017 at 12:53, Alexander Graf <agraf@suse.de> wrote:
> ARM is amazing when it comes to cache coherency and VMs. While any sane
> architecture allows the host to override the guest's caching attributes,
> that's very hard to do on ARM.
>
> That means that the guest may directly access guest memory bypassing the
> cache while QEMU happily writes to / reads from cache. The end result is
> very nasty, because both sides see very different views of the world.
>
> That means that we need to be very cautious to tell guests that devices
> that QEMU emulates are going to use data in the cache rather than directly
> on memory.
>
> We added this to PCI a while back for DT (5d636e21 "hw/arm/virt: mark the PCIe
> host controller as DMA coherent in the DT") and ACPI (bc64b96 "hw/arm/virt-
> acpi-build: _CCA attribute is compulsory") but never updated virtio-mmio or
> fw-cfg in DT or ACPI tables.
>
> This patch set adds the respective cache coherency flags for them in both DT and
> ACPI.
>
> Fortunately, no guests except for Linux 4.9.7 and 4.9.8 are broken because of
> this. Upstream realized quickly enough that every user of virtio-mmio out there
> describes its cache coherency incorrectly and reverted the patch that would
> require said dma coherency flag. But we should be safe for the future and "do
> the right thing".
>
> Alexander Graf (4):
>   target-arm: Declare virtio-mmio as dma-coherent in dt
>   hw/arm/virt: Declare virtio-mmio as dma cache coherent in ACPI
>   hw/arm/virt: Declare fwcfg as dma cache coherent in ACPI
>   hw/arm/virt: Declare fwcfg as dma cache coherent in dt
>

This looks correct to me.

Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/4] target-arm: Add some omitted dma cache coherency flags
  2017-02-09 13:15 ` [Qemu-devel] [PATCH v2 0/4] target-arm: Add some omitted dma cache coherency flags Laszlo Ersek
@ 2017-02-09 18:13   ` Michael S. Tsirkin
  2017-02-09 18:27     ` Alexander Graf
  0 siblings, 1 reply; 13+ messages in thread
From: Michael S. Tsirkin @ 2017-02-09 18:13 UTC (permalink / raw)
  To: Laszlo Ersek
  Cc: Alexander Graf, qemu-devel, qemu-arm, Shannon Zhao, Ard Biesheuvel

On Thu, Feb 09, 2017 at 02:15:36PM +0100, Laszlo Ersek wrote:
> On 02/09/17 13:53, Alexander Graf wrote:
> > ARM is amazing when it comes to cache coherency and VMs. While any sane
> > architecture allows the host to override the guest's caching attributes,
> > that's very hard to do on ARM.
> > 
> > That means that the guest may directly access guest memory bypassing the
> > cache while QEMU happily writes to / reads from cache. The end result is
> > very nasty, because both sides see very different views of the world.
> >  
> > That means that we need to be very cautious to tell guests that devices
> > that QEMU emulates are going to use data in the cache rather than directly
> > on memory.
> > 
> > We added this to PCI a while back for DT (5d636e21 "hw/arm/virt: mark the PCIe
> > host controller as DMA coherent in the DT") and ACPI (bc64b96 "hw/arm/virt-
> > acpi-build: _CCA attribute is compulsory") but never updated virtio-mmio or
> > fw-cfg in DT or ACPI tables.
> > 
> > This patch set adds the respective cache coherency flags for them in both DT and
> > ACPI.
> > 
> > Fortunately, no guests except for Linux 4.9.7 and 4.9.8 are broken because of
> > this. Upstream realized quickly enough that every user of virtio-mmio out there
> > describes its cache coherency incorrectly and reverted the patch that would
> > require said dma coherency flag. But we should be safe for the future and "do
> > the right thing".
> > 
> > Alexander Graf (4):
> >   target-arm: Declare virtio-mmio as dma-coherent in dt
> >   hw/arm/virt: Declare virtio-mmio as dma cache coherent in ACPI
> >   hw/arm/virt: Declare fwcfg as dma cache coherent in ACPI
> >   hw/arm/virt: Declare fwcfg as dma cache coherent in dt
> > 
> >  hw/arm/vexpress.c        | 1 +
> >  hw/arm/virt-acpi-build.c | 2 ++
> >  hw/arm/virt.c            | 2 ++
> >  3 files changed, 5 insertions(+)
> > 
> 
> Famous last words:
> series
> Reviewed-by: Laszlo Ersek <lersek@redhat.com>
> 
> Should we replicate patch #3 to QEMU0002 / FWCF in
> "hw/i386/acpi-build.c" too? Or is it that we couldn't care less about
> _CCA on x86? :) (Can't really muster the energy right now to look it up
> in the ACPI spec, sorry!)
> 
> Thanks
> Laszlo

ACPI spec says:
On platforms for which existing default cache-coherency behavior of the OS is not adequate, _CCA
enables the OS to adapt to the differences

So I think we don't need it on x86.

-- 
MST

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/4] target-arm: Add some omitted dma cache coherency flags
  2017-02-09 18:13   ` Michael S. Tsirkin
@ 2017-02-09 18:27     ` Alexander Graf
  2017-02-09 20:28       ` Laszlo Ersek
  0 siblings, 1 reply; 13+ messages in thread
From: Alexander Graf @ 2017-02-09 18:27 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Laszlo Ersek, qemu-devel, qemu-arm, Shannon Zhao, Ard Biesheuvel



> Am 09.02.2017 um 19:13 schrieb Michael S. Tsirkin <mst@redhat.com>:
> 
>> On Thu, Feb 09, 2017 at 02:15:36PM +0100, Laszlo Ersek wrote:
>>> On 02/09/17 13:53, Alexander Graf wrote:
>>> ARM is amazing when it comes to cache coherency and VMs. While any sane
>>> architecture allows the host to override the guest's caching attributes,
>>> that's very hard to do on ARM.
>>> 
>>> That means that the guest may directly access guest memory bypassing the
>>> cache while QEMU happily writes to / reads from cache. The end result is
>>> very nasty, because both sides see very different views of the world.
>>> 
>>> That means that we need to be very cautious to tell guests that devices
>>> that QEMU emulates are going to use data in the cache rather than directly
>>> on memory.
>>> 
>>> We added this to PCI a while back for DT (5d636e21 "hw/arm/virt: mark the PCIe
>>> host controller as DMA coherent in the DT") and ACPI (bc64b96 "hw/arm/virt-
>>> acpi-build: _CCA attribute is compulsory") but never updated virtio-mmio or
>>> fw-cfg in DT or ACPI tables.
>>> 
>>> This patch set adds the respective cache coherency flags for them in both DT and
>>> ACPI.
>>> 
>>> Fortunately, no guests except for Linux 4.9.7 and 4.9.8 are broken because of
>>> this. Upstream realized quickly enough that every user of virtio-mmio out there
>>> describes its cache coherency incorrectly and reverted the patch that would
>>> require said dma coherency flag. But we should be safe for the future and "do
>>> the right thing".
>>> 
>>> Alexander Graf (4):
>>>  target-arm: Declare virtio-mmio as dma-coherent in dt
>>>  hw/arm/virt: Declare virtio-mmio as dma cache coherent in ACPI
>>>  hw/arm/virt: Declare fwcfg as dma cache coherent in ACPI
>>>  hw/arm/virt: Declare fwcfg as dma cache coherent in dt
>>> 
>>> hw/arm/vexpress.c        | 1 +
>>> hw/arm/virt-acpi-build.c | 2 ++
>>> hw/arm/virt.c            | 2 ++
>>> 3 files changed, 5 insertions(+)
>>> 
>> 
>> Famous last words:
>> series
>> Reviewed-by: Laszlo Ersek <lersek@redhat.com>
>> 
>> Should we replicate patch #3 to QEMU0002 / FWCF in
>> "hw/i386/acpi-build.c" too? Or is it that we couldn't care less about
>> _CCA on x86? :) (Can't really muster the energy right now to look it up
>> in the ACPI spec, sorry!)
>> 
>> Thanks
>> Laszlo
> 
> ACPI spec says:
> On platforms for which existing default cache-coherency behavior of the OS is not adequate, _CCA
> enables the OS to adapt to the differences
> 
> So I think we don't need it on x86.

According to acpi 6.1, x86 explicitly defaults to dma coherent if _CCA is omitted. It's only illegal for ARM.

Alex

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/4] target-arm: Add some omitted dma cache coherency flags
  2017-02-09 18:27     ` Alexander Graf
@ 2017-02-09 20:28       ` Laszlo Ersek
  0 siblings, 0 replies; 13+ messages in thread
From: Laszlo Ersek @ 2017-02-09 20:28 UTC (permalink / raw)
  To: Alexander Graf, Michael S. Tsirkin
  Cc: qemu-devel, qemu-arm, Shannon Zhao, Ard Biesheuvel

On 02/09/17 19:27, Alexander Graf wrote:
> 
> 
>> Am 09.02.2017 um 19:13 schrieb Michael S. Tsirkin <mst@redhat.com>:
>>
>>> On Thu, Feb 09, 2017 at 02:15:36PM +0100, Laszlo Ersek wrote:
>>>> On 02/09/17 13:53, Alexander Graf wrote:
>>>> ARM is amazing when it comes to cache coherency and VMs. While any sane
>>>> architecture allows the host to override the guest's caching attributes,
>>>> that's very hard to do on ARM.
>>>>
>>>> That means that the guest may directly access guest memory bypassing the
>>>> cache while QEMU happily writes to / reads from cache. The end result is
>>>> very nasty, because both sides see very different views of the world.
>>>>
>>>> That means that we need to be very cautious to tell guests that devices
>>>> that QEMU emulates are going to use data in the cache rather than directly
>>>> on memory.
>>>>
>>>> We added this to PCI a while back for DT (5d636e21 "hw/arm/virt: mark the PCIe
>>>> host controller as DMA coherent in the DT") and ACPI (bc64b96 "hw/arm/virt-
>>>> acpi-build: _CCA attribute is compulsory") but never updated virtio-mmio or
>>>> fw-cfg in DT or ACPI tables.
>>>>
>>>> This patch set adds the respective cache coherency flags for them in both DT and
>>>> ACPI.
>>>>
>>>> Fortunately, no guests except for Linux 4.9.7 and 4.9.8 are broken because of
>>>> this. Upstream realized quickly enough that every user of virtio-mmio out there
>>>> describes its cache coherency incorrectly and reverted the patch that would
>>>> require said dma coherency flag. But we should be safe for the future and "do
>>>> the right thing".
>>>>
>>>> Alexander Graf (4):
>>>>  target-arm: Declare virtio-mmio as dma-coherent in dt
>>>>  hw/arm/virt: Declare virtio-mmio as dma cache coherent in ACPI
>>>>  hw/arm/virt: Declare fwcfg as dma cache coherent in ACPI
>>>>  hw/arm/virt: Declare fwcfg as dma cache coherent in dt
>>>>
>>>> hw/arm/vexpress.c        | 1 +
>>>> hw/arm/virt-acpi-build.c | 2 ++
>>>> hw/arm/virt.c            | 2 ++
>>>> 3 files changed, 5 insertions(+)
>>>>
>>>
>>> Famous last words:
>>> series
>>> Reviewed-by: Laszlo Ersek <lersek@redhat.com>
>>>
>>> Should we replicate patch #3 to QEMU0002 / FWCF in
>>> "hw/i386/acpi-build.c" too? Or is it that we couldn't care less about
>>> _CCA on x86? :) (Can't really muster the energy right now to look it up
>>> in the ACPI spec, sorry!)
>>>
>>> Thanks
>>> Laszlo
>>
>> ACPI spec says:
>> On platforms for which existing default cache-coherency behavior of the OS is not adequate, _CCA
>> enables the OS to adapt to the differences
>>
>> So I think we don't need it on x86.
> 
> According to acpi 6.1, x86 explicitly defaults to dma coherent if _CCA is omitted. It's only illegal for ARM.

Incredible; a finding that, for a change, does not create more work.

Thanks.
Laszlo

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [Qemu-devel] [PATCH v2 2/4] hw/arm/virt: Declare virtio-mmio as dma cache coherent in ACPI
  2017-02-09 12:53 ` [Qemu-devel] [PATCH v2 2/4] hw/arm/virt: Declare virtio-mmio as dma cache coherent in ACPI Alexander Graf
@ 2017-02-10  2:49   ` Shannon Zhao
  0 siblings, 0 replies; 13+ messages in thread
From: Shannon Zhao @ 2017-02-10  2:49 UTC (permalink / raw)
  To: Alexander Graf, qemu-devel
  Cc: Ard Biesheuvel, qemu-arm, Laszlo Ersek, Shannon Zhao, mst



On 2017/2/9 20:53, Alexander Graf wrote:
> Virtio-mmio devices can directly access guest memory and do so in cache
> coherent fashion. Tell the guest about that fact when it's using ACPI.
> 
> Signed-off-by: Alexander Graf <agraf@suse.de>
> ---
>  hw/arm/virt-acpi-build.c | 1 +
>  1 file changed, 1 insertion(+)
> 
Reviewed-by: Shannon Zhao <shannon.zhao@linaro.org>

> diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
> index 07a10ac..8955a9d 100644
> --- a/hw/arm/virt-acpi-build.c
> +++ b/hw/arm/virt-acpi-build.c
> @@ -135,6 +135,7 @@ static void acpi_dsdt_add_virtio(Aml *scope,
>          Aml *dev = aml_device("VR%02u", i);
>          aml_append(dev, aml_name_decl("_HID", aml_string("LNRO0005")));
>          aml_append(dev, aml_name_decl("_UID", aml_int(i)));
> +        aml_append(dev, aml_name_decl("_CCA", aml_int(1)));
>  
>          Aml *crs = aml_resource_template();
>          aml_append(crs, aml_memory32_fixed(base, size, AML_READ_WRITE));
> 

-- 
Shannon

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [Qemu-devel] [PATCH v2 3/4] hw/arm/virt: Declare fwcfg as dma cache coherent in ACPI
  2017-02-09 12:53 ` [Qemu-devel] [PATCH v2 3/4] hw/arm/virt: Declare fwcfg " Alexander Graf
@ 2017-02-10  2:49   ` Shannon Zhao
  0 siblings, 0 replies; 13+ messages in thread
From: Shannon Zhao @ 2017-02-10  2:49 UTC (permalink / raw)
  To: Alexander Graf, qemu-devel
  Cc: Ard Biesheuvel, qemu-arm, Laszlo Ersek, Shannon Zhao, mst



On 2017/2/9 20:53, Alexander Graf wrote:
> Fw-cfg recently learned how to directly access guest memory and does so in
> cache coherent fashion. Tell the guest about that fact when it's using ACPI.
> 
> Signed-off-by: Alexander Graf <agraf@suse.de>
> ---
>  hw/arm/virt-acpi-build.c | 1 +
>  1 file changed, 1 insertion(+)
> 
Reviewed-by: Shannon Zhao <shannon.zhao@linaro.org>

> diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
> index 8955a9d..0835e59 100644
> --- a/hw/arm/virt-acpi-build.c
> +++ b/hw/arm/virt-acpi-build.c
> @@ -90,6 +90,7 @@ static void acpi_dsdt_add_fw_cfg(Aml *scope, const MemMapEntry *fw_cfg_memmap)
>      aml_append(dev, aml_name_decl("_HID", aml_string("QEMU0002")));
>      /* device present, functioning, decoding, not shown in UI */
>      aml_append(dev, aml_name_decl("_STA", aml_int(0xB)));
> +    aml_append(dev, aml_name_decl("_CCA", aml_int(1)));
>  
>      Aml *crs = aml_resource_template();
>      aml_append(crs, aml_memory32_fixed(fw_cfg_memmap->base,
> 

-- 
Shannon

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [Qemu-devel] [Qemu-arm] [PATCH v2 0/4] target-arm: Add some omitted dma cache coherency flags
  2017-02-09 12:53 [Qemu-devel] [PATCH v2 0/4] target-arm: Add some omitted dma cache coherency flags Alexander Graf
                   ` (5 preceding siblings ...)
  2017-02-09 17:51 ` Ard Biesheuvel
@ 2017-02-10 15:02 ` Peter Maydell
  6 siblings, 0 replies; 13+ messages in thread
From: Peter Maydell @ 2017-02-10 15:02 UTC (permalink / raw)
  To: Alexander Graf
  Cc: QEMU Developers, Ard Biesheuvel, qemu-arm, Laszlo Ersek,
	Shannon Zhao, Michael S. Tsirkin

On 9 February 2017 at 12:53, Alexander Graf <agraf@suse.de> wrote:
> ARM is amazing when it comes to cache coherency and VMs. While any sane
> architecture allows the host to override the guest's caching attributes,
> that's very hard to do on ARM.
>
> That means that the guest may directly access guest memory bypassing the
> cache while QEMU happily writes to / reads from cache. The end result is
> very nasty, because both sides see very different views of the world.
>
> That means that we need to be very cautious to tell guests that devices
> that QEMU emulates are going to use data in the cache rather than directly
> on memory.
>
> We added this to PCI a while back for DT (5d636e21 "hw/arm/virt: mark the PCIe
> host controller as DMA coherent in the DT") and ACPI (bc64b96 "hw/arm/virt-
> acpi-build: _CCA attribute is compulsory") but never updated virtio-mmio or
> fw-cfg in DT or ACPI tables.
>
> This patch set adds the respective cache coherency flags for them in both DT and
> ACPI.
>
> Fortunately, no guests except for Linux 4.9.7 and 4.9.8 are broken because of
> this. Upstream realized quickly enough that every user of virtio-mmio out there
> describes its cache coherency incorrectly and reverted the patch that would
> require said dma coherency flag. But we should be safe for the future and "do
> the right thing".
>
> Alexander Graf (4):
>   target-arm: Declare virtio-mmio as dma-coherent in dt
>   hw/arm/virt: Declare virtio-mmio as dma cache coherent in ACPI
>   hw/arm/virt: Declare fwcfg as dma cache coherent in ACPI
>   hw/arm/virt: Declare fwcfg as dma cache coherent in dt
>
>  hw/arm/vexpress.c        | 1 +
>  hw/arm/virt-acpi-build.c | 2 ++
>  hw/arm/virt.c            | 2 ++
>  3 files changed, 5 insertions(+)

The patches in this series have more lines of Reviewed-by: tags than
they do actual code changes :-)

Thanks, applied to target-arm.next.

-- PMM

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2017-02-10 15:03 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-02-09 12:53 [Qemu-devel] [PATCH v2 0/4] target-arm: Add some omitted dma cache coherency flags Alexander Graf
2017-02-09 12:53 ` [Qemu-devel] [PATCH v2 1/4] target-arm: Declare virtio-mmio as dma-coherent in dt Alexander Graf
2017-02-09 12:53 ` [Qemu-devel] [PATCH v2 2/4] hw/arm/virt: Declare virtio-mmio as dma cache coherent in ACPI Alexander Graf
2017-02-10  2:49   ` Shannon Zhao
2017-02-09 12:53 ` [Qemu-devel] [PATCH v2 3/4] hw/arm/virt: Declare fwcfg " Alexander Graf
2017-02-10  2:49   ` Shannon Zhao
2017-02-09 12:53 ` [Qemu-devel] [PATCH v2 4/4] hw/arm/virt: Declare fwcfg as dma cache coherent in dt Alexander Graf
2017-02-09 13:15 ` [Qemu-devel] [PATCH v2 0/4] target-arm: Add some omitted dma cache coherency flags Laszlo Ersek
2017-02-09 18:13   ` Michael S. Tsirkin
2017-02-09 18:27     ` Alexander Graf
2017-02-09 20:28       ` Laszlo Ersek
2017-02-09 17:51 ` Ard Biesheuvel
2017-02-10 15:02 ` [Qemu-devel] [Qemu-arm] " Peter Maydell

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.