Re: [Regression] Re: [PATCH 18/18] iommu/vt-d: Remove IOVA handling code from the non-dma_ops path

[Alex Williamson <alex.williamson@redhat.com>]

On Thu, 18 Jun 2020 09:52:56 +0800
Lu Baolu <baolu.lu@linux.intel.com> wrote:

> Hi Alex,
> 
> Thanks for the report.
> 
> On 6/18/20 4:06 AM, Alex Williamson wrote:
> > On Sat, 16 May 2020 14:21:01 +0800
> > Lu Baolu<baolu.lu@linux.intel.com>  wrote:
> >   
> >> From: Tom Murphy<murphyt7@tcd.ie>
> >>
> >> There's no need for the non-dma_ops path to keep track of IOVAs. The
> >> whole point of the non-dma_ops path is that it allows the IOVAs to be
> >> handled separately. The IOVA handling code removed in this patch is
> >> pointless.
> >>
> >> Signed-off-by: Tom Murphy<murphyt7@tcd.ie>
> >> Signed-off-by: Lu Baolu<baolu.lu@linux.intel.com>
> >> ---
> >>   drivers/iommu/intel-iommu.c | 95 +++++++++++++------------------------
> >>   1 file changed, 32 insertions(+), 63 deletions(-)  
> > This commit results in a massive increase in memory use from the VT-d
> > code.  I have a 16GB system where I reserve 7168 2MB hugespages for VM
> > usage (14GB), leaving the host with 2GB.  I can no longer even boot the
> > host in this configuration.  Bisecting to this commit, I find that
> > reverting this change shows the following memory usage difference
> > immediately after boot (no hugepages, nosmp, single user,
> > intel_iommu=on iommu=pt):
> > 
> > @e70b081c6f37:
> >               total        used        free      shared  buff/cache   available
> > Mem:       16090860     2219372    13673044        1040      198444    13602664
> > Swap:       2097148           0     2097148
> > 
> > reverting e70b081c6f37:
> >                total        used        free      shared  buff/cache   available
> > Mem:       16090852      101648    15789156        1040      200048    15719572
> > Swap:       2097148           0     2097148
> > 
> > More than 2GB of additional memory used!  There's also a notable stall
> > during bootup for this allocation:
> > 
> > [    9.703360] DMAR: No ATSR found
> > [    9.709768] DMAR: dmar0: Using Queued invalidation
> > [    9.719370] DMAR: dmar1: Using Queued invalidation
> > 
> > ### 4+ seconds! ###
> > 
> > [   14.076387] pci 0000:00:00.0: Adding to iommu group 0
> > [   14.086515] pci 0000:00:01.0: Adding to iommu group 1
> > [   14.096635] pci 0000:00:02.0: Adding to iommu group 2  
> 
> Can you please try below changes?
> 
> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> index 887f184b900d..7eb29167e8f9 100644
> --- a/drivers/iommu/intel/iommu.c
> +++ b/drivers/iommu/intel/iommu.c
> @@ -2705,7 +2705,9 @@ static int __init si_domain_init(int hw)
>                                      end >> agaw_to_width(si_domain->agaw)))
>                                  continue;
> 
> -                       ret = iommu_domain_identity_map(si_domain, 
> start, end);
> +                       ret = iommu_domain_identity_map(si_domain,
> +                                       mm_to_dma_pfn(start >> PAGE_SHIFT),
> +                                       mm_to_dma_pfn(end >> PAGE_SHIFT));
>                          if (ret)
>                                  return ret;
>                  }

Yes, this resolves it.  Used memory after boot returns to similar
numbers as without this commit.  Thanks,

Alex

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu