Re: [PATCH v5 1/2] powerpc/pseries/iommu: Share the per-cpu TCE page with the hypervisor.

[Alexey Kardashevskiy <aik@ozlabs.ru>]

On 10/12/2019 16:12, Ram Pai wrote:
> On Tue, Dec 10, 2019 at 02:07:36PM +1100, Alexey Kardashevskiy wrote:
>>
>>
>> On 07/12/2019 12:12, Ram Pai wrote:
>>> H_PUT_TCE_INDIRECT hcall uses a page filled with TCE entries, as one of
>>> its parameters.  On secure VMs, hypervisor cannot access the contents of
>>> this page since it gets encrypted.  Hence share the page with the
>>> hypervisor, and unshare when done.
>>
>>
>> I thought the idea was to use H_PUT_TCE and avoid sharing any extra
>> pages. There is small problem that when DDW is enabled,
>> FW_FEATURE_MULTITCE is ignored (easy to fix); I also noticed complains
>> about the performance on slack but this is caused by initial cleanup of
>> the default TCE window (which we do not use anyway) and to battle this
>> we can simply reduce its size by adding
> 
> something that takes hardly any time with H_PUT_TCE_INDIRECT,  takes
> 13secs per device for H_PUT_TCE approach, during boot. This is with a
> 30GB guest. With larger guest, the time will further detoriate.


No it will not, I checked. The time is the same for 2GB and 32GB guests-
the delay is caused by clearing the small DMA window which is small by
the space mapped (1GB) but quite huge in TCEs as it uses 4K pages; and
for DDW window + emulated devices the IOMMU page size will be 2M/16M/1G
(depends on the system) so the number of TCEs is much smaller.


> 
>>
>> -global
>> spapr-pci-host-bridge.dma_win_size=0x4000000
> 
> This option, speeds it up tremendously.  But than should this option be
> enabled in qemu by default?  only for secure VMs? for both VMs?


As discussed in slack, by default we do not need to clear the entire TCE
table and we only have to map swiotlb buffer using the small window. It
is a guest kernel change only. Thanks,



-- 
Alexey