Re: [PATCH v7] swiotlb: Adjust SWIOTBL bounce buffer size for SEV guests.

From: Ashish Kalra <ashish.kalra@amd.com>
To: Borislav Petkov <bp@alien8.de>
Cc: Thomas.Lendacky@amd.com, Jon.Grimm@amd.com,
	brijesh.singh@amd.com, dave.hansen@linux-intel.com,
	konrad.wilk@oracle.com, peterz@infradead.org, x86@kernel.org,
	linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org,
	mingo@redhat.com, luto@kernel.org, hpa@zytor.com,
	rientjes@google.com, tglx@linutronix.de, hch@lst.de
Subject: Re: [PATCH v7] swiotlb: Adjust SWIOTBL bounce buffer size for SEV guests.
Date: Mon, 7 Dec 2020 22:06:24 +0000	[thread overview]
Message-ID: <20201207220624.GB6855@ashkalra_ubuntu_server> (raw)
In-Reply-To: <20201207121007.GD20489@zn.tnic>

Hello Boris, 

On Mon, Dec 07, 2020 at 01:10:07PM +0100, Borislav Petkov wrote:
> On Thu, Dec 03, 2020 at 03:25:59AM +0000, Ashish Kalra wrote:
> > diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
> > index 1bcfbcd2bfd7..46549bd3d840 100644
> > --- a/arch/x86/mm/mem_encrypt.c
> > +++ b/arch/x86/mm/mem_encrypt.c
> > @@ -485,7 +485,38 @@ static void print_mem_encrypt_feature_info(void)
> >  	pr_cont("\n");
> >  }
> 
> Any text about why 6% was chosen? A rule of thumb or so? Measurements?
>

This is related to the earlier static adjustment of the SWIOTLB buffers
as per guest memory size and Konrad's feedback on the same, as copied
below : 

>>That is eating 128MB for 1GB, aka 12% of the guest memory allocated statically for this.
>> 
>> And for guests that are 2GB, that is 12% until it gets to 3GB when 
>> it is 8% and then 6% at 4GB.
>> 
>> I would prefer this to be based on your memory count, that is 6% of 
>> total memory. 

Thanks,
Ashish

> > +#define SEV_ADJUST_SWIOTLB_SIZE_PERCENT	6
> > +
> >  /* Architecture __weak replacement functions */
> > +unsigned long __init arch_swiotlb_adjust(unsigned long iotlb_default_size)
> > +{
> > +	unsigned long size = iotlb_default_size;
> > +
> > +	/*
> > +	 * For SEV, all DMA has to occur via shared/unencrypted pages.
> > +	 * SEV uses SWOTLB to make this happen without changing device
> > +	 * drivers. However, depending on the workload being run, the
> > +	 * default 64MB of SWIOTLB may not be enough and`SWIOTLB may
> > +	 * run out of buffers for DMA, resulting in I/O errors and/or
> > +	 * performance degradation especially with high I/O workloads.
> > +	 * Adjust the default size of SWIOTLB for SEV guests using
> > +	 * a percentage of guest memory for SWIOTLB buffers.
> > +	 * Also as the SWIOTLB bounce buffer memory is allocated
> > +	 * from low memory, ensure that the adjusted size is within
> > +	 * the limits of low available memory.
> > +	 *
> > +	 */
> > +	if (sev_active()) {
> > +		phys_addr_t total_mem = memblock_phys_mem_size();
> 
> Please integrate scripts/checkpatch.pl into your patch creation
> workflow. Some of the warnings/errors *actually* make sense:
> 
> WARNING: Missing a blank line after declarations
> #95: FILE: arch/x86/mm/mem_encrypt.c:511:
> +               phys_addr_t total_mem = memblock_phys_mem_size();
> +               size = total_mem * SEV_ADJUST_SWIOTLB_SIZE_PERCENT / 100;
> 
> But no need to resend now - just a hint for the future.
> 
> Konrad, ack?
> 
> On a 2G guest here, it says:
> 
> [    0.018373] SWIOTLB bounce buffer size adjusted to 122MB for SEV
> 
> so it makes sense to me.
> 
> Thx.
> 
> -- 
> Regards/Gruss,
>     Boris.
> 
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpeople.kernel.org%2Ftglx%2Fnotes-about-netiquette&amp;data=04%7C01%7CAshish.Kalra%40amd.com%7C5c7096203e774538383308d89aa95b4a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637429399488203154%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=G8JOTIQ5eVyOU50iBAI3%2FotUlGV1EBuOZectZNWlUyE%3D&amp;reserved=0
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu