iommu/amd: bug report: page table memory leak

[Daniel Jordan <daniel.m.jordan@oracle.com>]

[-- Attachment #1: Type: text/plain, Size: 5024 bytes --]

Hi,

I've hit a memory leak while testing qemu v6.2.0-rc4 on an AMD EPYC 7J13
(Milan) system.  Starting an almost 1T guest, the leak is over 1.5G per
qemu invocation.  I haven't checked whether the leak is proportional to
guest size.  It happens with a vfio device, and only when the guest's
memory is preallocated using qemu prealloc (this latter part is kinda
strange).  It happens when the guest memory uses THP but not hugetlb.

Bisection:

# bad: [df0cc57e057f18e44dac8e6c18aba47ab53202f9] Linux 5.16                
# good: [f40ddce88593482919761f74910f42f4b84c004b] Linux 5.11
git bisect start 'df0cc57e057f1' 'f40ddce885934' '--' 'drivers/vfio' 'drivers/iommu' 'include/linux/amd-iommu.h' 'include/linux/dma-iommu.h' 'include/linux/intel-iommu.h' 'include/linux/iommu-helper.h' 'include/linux/of_iommu.h' 'include/
linux/omap-iommu.h' 'include/linux/platform_data/iommu-omap.h' 'include/linux/iommu.h' 'include/trace/events/intel_iommu.h' 'include/trace/events/iommu.h' 'include/uapi/linux/iommu.h' 'include/uapi/linux/virtio_iommu.h' 'arch/x86/events/a
md/iommu.h' 'arch/x86/events/amd/iommu.c' 'arch/x86/include/asm/iommu.h' 'arch/x86/include/asm/iommu_table.h' 'arch/x86/kernel/pci-iommu_table.c'
# bad: [cee57d4fe74e82e784f6566bad3e3bb1ca51a211] iommu/vt-d: Remove unnecessary braces
git bisect bad cee57d4fe74e82e784f6566bad3e3bb1ca51a211                           
# bad: [9fb5fad562fa0a41c84691714d99c23f54168a9e] iommu: remove DOMAIN_ATTR_PAGING
git bisect bad 9fb5fad562fa0a41c84691714d99c23f54168a9e                           
# bad: [45e606f2726926b04094e1c9bf809bca4884c57f] Merge branches 'arm/renesas', 'arm/smmu', 'x86/amd', 'x86/vt-d' and 'core' into next
git bisect bad 45e606f2726926b04094e1c9bf809bca4884c57f
# good: [7060377ce06f9cd3ed6274c0f2310463feb5baec] Merge branch 'for-joerg/mtk' into for-joerg/arm-smmu/updates
git bisect good 7060377ce06f9cd3ed6274c0f2310463feb5baec
# bad: [6778ff5b21bd8e78c8bd547fd66437cf2657fd9b] iommu/amd: Fix performance counter initialization
git bisect bad 6778ff5b21bd8e78c8bd547fd66437cf2657fd9b
# good: [f9b4df790aa4372bfa11b7d212e537b763295429] iommu/amd: Declare functions as extern
git bisect good f9b4df790aa4372bfa11b7d212e537b763295429
# bad: [33aef9786046d9a5744cd1e8d5d0ce800d611fdc] iommu/amd: Rename variables to be consistent with struct io_pgtable_ops
git bisect bad 33aef9786046d9a5744cd1e8d5d0ce800d611fdc       
# bad: [e42ba0633064ef23eb1c8c21edf96bac1541bd4b] iommu/amd: Restructure code for freeing page table
git bisect bad e42ba0633064ef23eb1c8c21edf96bac1541bd4b       
# good: [18954252a1d0b12e1b77087b55c37fb43b09e12a] iommu/amd: Move IO page table related functions
git bisect good 18954252a1d0b12e1b77087b55c37fb43b09e12a      
# first bad commit: [e42ba0633064ef23eb1c8c21edf96bac1541bd4b] iommu/amd: Restructure code for freeing page table

commit e42ba0633064ef23eb1c8c21edf96bac1541bd4b                         
Author: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>                            
Date:   Tue Dec 15 01:36:59 2020 -0600                                  
                                                                                         
    iommu/amd: Restructure code for freeing page table                    
                                                                                         
    By consolidate logic into v1_free_pgtable helper function,            
    which is called from IO page table framework.                                        
                                                                            
    Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
    Link: https://lore.kernel.org/r/20201215073705.123786-8-suravee.suthikulpanit@amd.com
    Signed-off-by: Joerg Roedel <jroedel@suse.de>                         
                                                                            
 drivers/iommu/amd/amd_iommu.h  |  1 -                                    
 drivers/iommu/amd/io_pgtable.c | 41 ++++++++++++++++++++++++-----------------
 drivers/iommu/amd/iommu.c      | 21 ++++-----------------                
 3 files changed, 28 insertions(+), 35 deletions(-)                         

Qemu command line:

numactl -m 1 -N 1 "$QEMU" \
    -name vmol74 \
    -machine q35,accel=kvm,usb=off,dump-guest-core=off,memory-backend=pc.ram \
    -cpu host,host-phys-bits=true \
    -smp cpus=32 \
    -no-user-config \
    -nodefaults \
    -rtc base=utc,driftfix=slew \
    -global kvm-pit.lost_tick_policy=delay \
    -no-hpet \
    -no-shutdown \
    -boot strict=on \
    -drive file=${vm_image},format=raw,if=none,id=drive-ide0-0-0 \
    -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=2 \
    -device vfio-pci,host=${pci_addr},id=net2,bus=pcie.0 \
    -msg timestamp=on \
    -nographic \
    -object memory-backend-ram,id=pc.ram,size=980g,prealloc=on,prealloc-threads=16 -m 980g \
    -daemonize

Kernel config attached, and I can provide other details too if I left
anything out.

thanks,
Daniel

[-- Attachment #2: amd-leak-config.gz --]
[-- Type: application/gzip, Size: 54009 bytes --]

[-- Attachment #3: Type: text/plain, Size: 156 bytes --]

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu