Re: [PATCH kernel 1/2] powerpc/pseries/dma: Allow swiotlb

[Alexey Kardashevskiy <aik@ozlabs.ru>]

On 11/05/2019 08:36, Thiago Jung Bauermann wrote:
> 
> Alexey Kardashevskiy <aik@ozlabs.ru> writes:
> 
>> The commit 8617a5c5bc00 ("powerpc/dma: handle iommu bypass in
>> dma_iommu_ops") merged direct DMA ops into the IOMMU DMA ops allowing
>> SWIOTLB as well but only for mapping; the unmapping and bouncing parts
>> were left unmodified.
>>
>> This adds missing direct unmapping calls to .unmap_page() and .unmap_sg().
>>
>> This adds missing sync callbacks and directs them to the direct DMA hooks.
>>
>> Fixes: 8617a5c5bc00 (powerpc/dma: handle iommu bypass in dma_iommu_ops)
>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> 
> Nice! Thanks for working on this. I have the patch at the end of this
> email to get virtio-scsi-pci and virtio-blk-pci working in a secure
> guest.

I saw the set_pci_dma_ops(NULL) patch but could not figure out how pass
NULL there sets the DMA ops to direct.

> 
> I applied your patch and reverted my patch and unfortunately the guest
> hangs right after mounting the disk:

Have you applied it on upstream kernel? I cannot see how it affects
current guests as it is...


> 
> [    0.185659] virtio-pci 0000:00:04.0: enabling device (0100 -> 0102)
> [    0.187082] virtio-pci 0000:00:04.0: ibm,query-pe-dma-windows(2026) 2000 8000000 20000000 returned 0
> [    0.187497] virtio-pci 0000:00:04.0: ibm,create-pe-dma-window(2027) 2000 8000000 20000000 10 20 returned 0 (liobn = 0x80000001 startin
> g addr = 8000000 0)
> [    0.226654] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
> [    0.227094] Non-volatile memory driver v1.3
> [    0.228950] brd: module loaded
> [    0.230666] loop: module loaded
> [    0.230773] ipr: IBM Power RAID SCSI Device Driver version: 2.6.4 (March 14, 2017)
> [    0.233323] scsi host0: Virtio SCSI HBA
> [    0.235439] scsi 0:0:0:0: Direct-Access     QEMU     QEMU HARDDISK    2.5+ PQ: 0 ANSI: 5
> [    0.369009] random: fast init done
> [    0.370819] sd 0:0:0:0: Attached scsi generic sg0 type 0
> [    0.371320] sd 0:0:0:0: Power-on or device reset occurred
> 
> <snip>
> 
> [    0.380378] sd 0:0:0:0: [sda] 31457280 512-byte logical blocks: (16.1 GB/15.0 GiB)
> [    0.381102] sd 0:0:0:0: [sda] Write Protect is off
> [    0.381195] sd 0:0:0:0: [sda] Mode Sense: 63 00 00 08
> [    0.382436] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
> [    0.383630] sd 0:0:0:0: [sda] Optimal transfer size 0 bytes < PAGE_SIZE (65536 bytes)
> [    0.391562]  sda: sda1 sda2
> [    0.398101] sd 0:0:0:0: [sda] Attached SCSI disk
> [    0.398205] md: Waiting for all devices to be available before autodetect
> [    0.398318] md: If you don't use raid, use raid=noautodetect
> [    0.398515] md: Autodetecting RAID arrays.
> [    0.398585] md: autorun ...
> [    0.398631] md: ... autorun DONE.
> [    0.403552] EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
> [    0.403700] VFS: Mounted root (ext4 filesystem) readonly on device 8:2.
> [    0.405258] devtmpfs: mounted
> [    0.406427] Freeing unused kernel memory: 4224K
> [    0.406519] This architecture does not have kernel memory protection.
> [    0.406633] Run /sbin/init as init process
> 
> Sorry, I don't have any information on where the guest is stuck. I tried
> <sysrq>+l, <sysrq>+t and <sysrq>+w but nothing out of the ordinary
> showed up. Will try something else later.
> 
> --
> Thiago Jung Bauermann
> IBM Linux Technology Center
> 
> 
> 
> From 70d2fba809119ae2d35c9ca4269405bb5c28413a Mon Sep 17 00:00:00 2001
> From: Thiago Jung Bauermann <bauerman@linux.ibm.com>
> Date: Thu, 24 Jan 2019 22:40:16 -0200
> Subject: [PATCH 1/1] powerpc/pseries/iommu: Don't use dma_iommu_ops on secure
>  guests
> 
> Secure guest memory is inacessible to devices so regular DMA isn't
> possible.
> 
> In that case set devices' dma_map_ops to NULL so that the generic
> DMA code path will use SWIOTLB and DMA to bounce buffers.
> 
> Signed-off-by: Thiago Jung Bauermann <bauerman@linux.ibm.com>
> ---
>  arch/powerpc/platforms/pseries/iommu.c | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
> index 36eb1ddbac69..1636306007eb 100644
> --- a/arch/powerpc/platforms/pseries/iommu.c
> +++ b/arch/powerpc/platforms/pseries/iommu.c
> @@ -50,6 +50,7 @@
>  #include <asm/udbg.h>
>  #include <asm/mmzone.h>
>  #include <asm/plpar_wrappers.h>
> +#include <asm/svm.h>
> 
>  #include "pseries.h"
> 
> @@ -1335,7 +1336,10 @@ void iommu_init_early_pSeries(void)
>  	of_reconfig_notifier_register(&iommu_reconfig_nb);
>  	register_memory_notifier(&iommu_mem_nb);
> 
> -	set_pci_dma_ops(&dma_iommu_ops);
> +	if (is_secure_guest())
> +		set_pci_dma_ops(NULL);
> +	else
> +		set_pci_dma_ops(&dma_iommu_ops);
>  }
> 
>  static int __init disable_multitce(char *str)
> 

-- 
Alexey