Re: [PATCH v2] swiotlb: Adjust SWIOTBL bounce buffer size for SEV guests.

From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
To: Ashish Kalra <ashish.kalra@amd.com>
Cc: Konrad Rzeszutek Wilk <konrad@darnok.org>,
	hch@lst.de, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
	hpa@zytor.com, x86@kernel.org, luto@kernel.org,
	peterz@infradead.org, dave.hansen@linux-intel.com,
	iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org,
	brijesh.singh@amd.com, Thomas.Lendacky@amd.com
Subject: Re: [PATCH v2] swiotlb: Adjust SWIOTBL bounce buffer size for SEV guests.
Date: Tue, 3 Mar 2020 12:03:53 -0500	[thread overview]
Message-ID: <20200303170353.GC31627@char.us.oracle.com> (raw)
In-Reply-To: <20200204193500.GA15564@ashkalra_ubuntu_server>

On Tue, Feb 04, 2020 at 07:35:00PM +0000, Ashish Kalra wrote:
> Hello Konrad,
> 
> Looking fwd. to your feedback regarding support of other memory
> encryption architectures such as Power, S390, etc.
> 
> Thanks,
> Ashish
> 
> On Fri, Jan 24, 2020 at 11:00:08PM +0000, Ashish Kalra wrote:
> > On Tue, Jan 21, 2020 at 03:54:03PM -0500, Konrad Rzeszutek Wilk wrote:
> > > > 
> > > > Additional memory calculations based on # of PCI devices and
> > > > their memory ranges will make it more complicated with so
> > > > many other permutations and combinations to explore, it is
> > > > essential to keep this patch as simple as possible by 
> > > > adjusting the bounce buffer size simply by determining it
> > > > from the amount of provisioned guest memory.
> > >> 
> > >> Please rework the patch to:
> > >> 
> > >>  - Use a log solution instead of the multiplication.
> > >>    Feel free to cap it at a sensible value.
> > 
> > Ok.
> > 
> > >> 
> > >>  - Also the code depends on SWIOTLB calling in to the
> > >>    adjust_swiotlb_default_size which looks wrong.
> > >> 
> > >>    You should not adjust io_tlb_nslabs from swiotlb_size_or_default.
> > 
> > >>    That function's purpose is to report a value.
> > >> 
> > >>  - Make io_tlb_nslabs be visible outside of the SWIOTLB code.
> > >> 
> > >>  - Can you utilize the IOMMU_INIT APIs and have your own detect which would
> > >>    modify the io_tlb_nslabs (and set swiotbl=1?).
> > 
> > This seems to be a nice option, but then IOMMU_INIT APIs are
> > x86-specific and this swiotlb buffer size adjustment is also needed
> > for other memory encryption architectures like Power, S390, etc.

Oh dear. That I hadn't considered.
> > 
> > >> 
> > >>    Actually you seem to be piggybacking on pci_swiotlb_detect_4gb - so
> > >>    perhaps add in this code ? Albeit it really should be in it's own
> > >>    file, not in arch/x86/kernel/pci-swiotlb.c
> > 
> > Actually, we piggyback on pci_swiotlb_detect_override which sets
> > swiotlb=1 as x86_64_start_kernel() and invocation of sme_early_init()
> > forces swiotlb on, but again this is all x86 architecture specific.

Then it looks like the best bet is to do it from within swiotlb_init?
We really can't do it from swiotlb_size_or_default - that function
should just return a value and nothing else.

> > 
> > Thanks,
> > Ashish