From: Pasha Tatashin <pasha.tatashin@soleen.com> To: akpm@linux-foundation.org, alex.williamson@redhat.com, alim.akhtar@samsung.com, alyssa@rosenzweig.io, asahi@lists.linux.dev, baolu.lu@linux.intel.com, bhelgaas@google.com, cgroups@vger.kernel.org, corbet@lwn.net, david@redhat.com, dwmw2@infradead.org, hannes@cmpxchg.org, heiko@sntech.de, iommu@lists.linux.dev, jasowang@redhat.com, jernej.skrabec@gmail.com, jgg@ziepe.ca, jonathanh@nvidia.com, joro@8bytes.org, kevin.tian@intel.com, krzysztof.kozlowski@linaro.org, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, lizefan.x@bytedance.com, marcan@marcan.st, mhiramat@kernel.org, mst@redhat.com, m.szyprowski@samsung.com, netdev@vger.kernel.org, pasha.tatashin@soleen.com, paulmck@kernel.org, rdunlap@infradead.org, robin.murphy@arm.com, samuel@sholland.org, suravee.suthikulpanit@amd.com, sven@svenpeter.dev, thierry.reding@gmail.com, tj@kernel.org, tomas.mudrunka@gmail.com, vdumpa@nvidia.com, virtualization@lists.linux.dev, wens@csie.org, will@kernel.org, yu-cheng.yu@intel.com Subject: [PATCH 00/16] IOMMU memory observability Date: Tue, 28 Nov 2023 20:49:22 +0000 [thread overview] Message-ID: <20231128204938.1453583-1-pasha.tatashin@soleen.com> (raw) From: Pasha Tatashin <tatashin@google.com> IOMMU subsystem may contain state that is in gigabytes. Majority of that state is iommu page tables. Yet, there is currently, no way to observe how much memory is actually used by the iommu subsystem. This patch series solves this problem by adding both observability to all pages that are allocated by IOMMU, and also accountability, so admins can limit the amount if via cgroups. The system-wide observability is using /proc/meminfo: SecPageTables: 438176 kB Contains IOMMU and KVM memory. Per-node observability: /sys/devices/system/node/nodeN/meminfo Node N SecPageTables: 422204 kB Contains IOMMU and KVM memory memory in the given NUMA node. Per-node IOMMU only observability: /sys/devices/system/node/nodeN/vmstat nr_iommu_pages 105555 Contains number of pages IOMMU allocated in the given node. Accountability: using sec_pagetables cgroup-v2 memory.stat entry. With the change, iova_stress[1] stops as limit is reached: # ./iova_stress iova space: 0T free memory: 497G iova space: 1T free memory: 495G iova space: 2T free memory: 493G iova space: 3T free memory: 491G stops as limit is reached. This series encorporates suggestions that came from the discussion at LPC [2]. [1] https://github.com/soleen/iova_stress [2] https://lpc.events/event/17/contributions/1466 Pasha Tatashin (16): iommu/vt-d: add wrapper functions for page allocations iommu/amd: use page allocation function provided by iommu-pages.h iommu/io-pgtable-arm: use page allocation function provided by iommu-pages.h iommu/io-pgtable-dart: use page allocation function provided by iommu-pages.h iommu/io-pgtable-arm-v7s: use page allocation function provided by iommu-pages.h iommu/dma: use page allocation function provided by iommu-pages.h iommu/exynos: use page allocation function provided by iommu-pages.h iommu/fsl: use page allocation function provided by iommu-pages.h iommu/iommufd: use page allocation function provided by iommu-pages.h iommu/rockchip: use page allocation function provided by iommu-pages.h iommu/sun50i: use page allocation function provided by iommu-pages.h iommu/tegra-smmu: use page allocation function provided by iommu-pages.h iommu: observability of the IOMMU allocations iommu: account IOMMU allocated memory vhost-vdpa: account iommu allocations vfio: account iommu allocations Documentation/admin-guide/cgroup-v2.rst | 2 +- Documentation/filesystems/proc.rst | 4 +- drivers/iommu/amd/amd_iommu.h | 8 - drivers/iommu/amd/init.c | 91 +++++----- drivers/iommu/amd/io_pgtable.c | 13 +- drivers/iommu/amd/io_pgtable_v2.c | 20 +- drivers/iommu/amd/iommu.c | 13 +- drivers/iommu/dma-iommu.c | 8 +- drivers/iommu/exynos-iommu.c | 14 +- drivers/iommu/fsl_pamu.c | 5 +- drivers/iommu/intel/dmar.c | 10 +- drivers/iommu/intel/iommu.c | 47 ++--- drivers/iommu/intel/iommu.h | 2 - drivers/iommu/intel/irq_remapping.c | 10 +- drivers/iommu/intel/pasid.c | 12 +- drivers/iommu/intel/svm.c | 7 +- drivers/iommu/io-pgtable-arm-v7s.c | 9 +- drivers/iommu/io-pgtable-arm.c | 7 +- drivers/iommu/io-pgtable-dart.c | 37 ++-- drivers/iommu/iommu-pages.h | 231 ++++++++++++++++++++++++ drivers/iommu/iommufd/iova_bitmap.c | 6 +- drivers/iommu/rockchip-iommu.c | 14 +- drivers/iommu/sun50i-iommu.c | 7 +- drivers/iommu/tegra-smmu.c | 18 +- drivers/vfio/vfio_iommu_type1.c | 8 +- drivers/vhost/vdpa.c | 3 +- include/linux/mmzone.h | 5 +- mm/vmstat.c | 3 + 28 files changed, 415 insertions(+), 199 deletions(-) create mode 100644 drivers/iommu/iommu-pages.h -- 2.43.0.rc2.451.g8631bc7472-goog
WARNING: multiple messages have this Message-ID (diff)
From: Pasha Tatashin <pasha.tatashin@soleen.com> To: akpm@linux-foundation.org, alex.williamson@redhat.com, alim.akhtar@samsung.com, alyssa@rosenzweig.io, asahi@lists.linux.dev, baolu.lu@linux.intel.com, bhelgaas@google.com, cgroups@vger.kernel.org, corbet@lwn.net, david@redhat.com, dwmw2@infradead.org, hannes@cmpxchg.org, heiko@sntech.de, iommu@lists.linux.dev, jasowang@redhat.com, jernej.skrabec@gmail.com, jgg@ziepe.ca, jonathanh@nvidia.com, joro@8bytes.org, kevin.tian@intel.com, krzysztof.kozlowski@linaro.org, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, lizefan.x@bytedance.com, marcan@marcan.st, mhiramat@kernel.org, mst@redhat.com, m.szyprowski@samsung.com, netdev@vger.kernel.org, pasha.tatashin@soleen.com, paulmck@kernel.org, rdunlap@infradead.org, robin.murphy@arm.com, samuel@sholland.org, suravee.suthikulpanit@amd.com, sven@svenpeter.dev, thierry.reding@gmail.com, tj@kernel.org, tomas.mudrunka@gmail.com, vdumpa@nvidia.com, virtualization@lists.linux.dev, wens@csie.org, will@kernel.org, yu-cheng.yu@intel.com Subject: [PATCH 00/16] IOMMU memory observability Date: Tue, 28 Nov 2023 20:49:22 +0000 [thread overview] Message-ID: <20231128204938.1453583-1-pasha.tatashin@soleen.com> (raw) From: Pasha Tatashin <tatashin@google.com> IOMMU subsystem may contain state that is in gigabytes. Majority of that state is iommu page tables. Yet, there is currently, no way to observe how much memory is actually used by the iommu subsystem. This patch series solves this problem by adding both observability to all pages that are allocated by IOMMU, and also accountability, so admins can limit the amount if via cgroups. The system-wide observability is using /proc/meminfo: SecPageTables: 438176 kB Contains IOMMU and KVM memory. Per-node observability: /sys/devices/system/node/nodeN/meminfo Node N SecPageTables: 422204 kB Contains IOMMU and KVM memory memory in the given NUMA node. Per-node IOMMU only observability: /sys/devices/system/node/nodeN/vmstat nr_iommu_pages 105555 Contains number of pages IOMMU allocated in the given node. Accountability: using sec_pagetables cgroup-v2 memory.stat entry. With the change, iova_stress[1] stops as limit is reached: # ./iova_stress iova space: 0T free memory: 497G iova space: 1T free memory: 495G iova space: 2T free memory: 493G iova space: 3T free memory: 491G stops as limit is reached. This series encorporates suggestions that came from the discussion at LPC [2]. [1] https://github.com/soleen/iova_stress [2] https://lpc.events/event/17/contributions/1466 Pasha Tatashin (16): iommu/vt-d: add wrapper functions for page allocations iommu/amd: use page allocation function provided by iommu-pages.h iommu/io-pgtable-arm: use page allocation function provided by iommu-pages.h iommu/io-pgtable-dart: use page allocation function provided by iommu-pages.h iommu/io-pgtable-arm-v7s: use page allocation function provided by iommu-pages.h iommu/dma: use page allocation function provided by iommu-pages.h iommu/exynos: use page allocation function provided by iommu-pages.h iommu/fsl: use page allocation function provided by iommu-pages.h iommu/iommufd: use page allocation function provided by iommu-pages.h iommu/rockchip: use page allocation function provided by iommu-pages.h iommu/sun50i: use page allocation function provided by iommu-pages.h iommu/tegra-smmu: use page allocation function provided by iommu-pages.h iommu: observability of the IOMMU allocations iommu: account IOMMU allocated memory vhost-vdpa: account iommu allocations vfio: account iommu allocations Documentation/admin-guide/cgroup-v2.rst | 2 +- Documentation/filesystems/proc.rst | 4 +- drivers/iommu/amd/amd_iommu.h | 8 - drivers/iommu/amd/init.c | 91 +++++----- drivers/iommu/amd/io_pgtable.c | 13 +- drivers/iommu/amd/io_pgtable_v2.c | 20 +- drivers/iommu/amd/iommu.c | 13 +- drivers/iommu/dma-iommu.c | 8 +- drivers/iommu/exynos-iommu.c | 14 +- drivers/iommu/fsl_pamu.c | 5 +- drivers/iommu/intel/dmar.c | 10 +- drivers/iommu/intel/iommu.c | 47 ++--- drivers/iommu/intel/iommu.h | 2 - drivers/iommu/intel/irq_remapping.c | 10 +- drivers/iommu/intel/pasid.c | 12 +- drivers/iommu/intel/svm.c | 7 +- drivers/iommu/io-pgtable-arm-v7s.c | 9 +- drivers/iommu/io-pgtable-arm.c | 7 +- drivers/iommu/io-pgtable-dart.c | 37 ++-- drivers/iommu/iommu-pages.h | 231 ++++++++++++++++++++++++ drivers/iommu/iommufd/iova_bitmap.c | 6 +- drivers/iommu/rockchip-iommu.c | 14 +- drivers/iommu/sun50i-iommu.c | 7 +- drivers/iommu/tegra-smmu.c | 18 +- drivers/vfio/vfio_iommu_type1.c | 8 +- drivers/vhost/vdpa.c | 3 +- include/linux/mmzone.h | 5 +- mm/vmstat.c | 3 + 28 files changed, 415 insertions(+), 199 deletions(-) create mode 100644 drivers/iommu/iommu-pages.h -- 2.43.0.rc2.451.g8631bc7472-goog _______________________________________________ Linux-rockchip mailing list Linux-rockchip@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-rockchip
next reply other threads:[~2023-11-28 20:49 UTC|newest] Thread overview: 103+ messages / expand[flat|nested] mbox.gz Atom feed top 2023-11-28 20:49 Pasha Tatashin [this message] 2023-11-28 20:49 ` [PATCH 00/16] IOMMU memory observability Pasha Tatashin 2023-11-28 20:49 ` [PATCH 01/16] iommu/vt-d: add wrapper functions for page allocations Pasha Tatashin 2023-11-28 20:49 ` Pasha Tatashin 2023-11-28 20:49 ` [PATCH 02/16] iommu/amd: use page allocation function provided by iommu-pages.h Pasha Tatashin 2023-11-28 20:49 ` Pasha Tatashin 2023-11-28 20:49 ` [PATCH 03/16] iommu/io-pgtable-arm: " Pasha Tatashin 2023-11-28 20:49 ` Pasha Tatashin 2023-11-28 20:49 ` [PATCH 04/16] iommu/io-pgtable-dart: " Pasha Tatashin 2023-11-28 20:49 ` Pasha Tatashin 2023-11-29 7:49 ` Janne Grunau 2023-11-29 7:49 ` Janne Grunau 2023-11-29 21:49 ` Pasha Tatashin 2023-11-29 21:49 ` Pasha Tatashin 2023-11-28 20:49 ` [PATCH 05/16] iommu/io-pgtable-arm-v7s: " Pasha Tatashin 2023-11-28 20:49 ` Pasha Tatashin 2023-11-28 22:46 ` Robin Murphy 2023-11-28 22:46 ` Robin Murphy 2023-11-28 22:55 ` Pasha Tatashin 2023-11-28 22:55 ` Pasha Tatashin 2023-11-28 23:07 ` Robin Murphy 2023-11-28 23:07 ` Robin Murphy 2023-11-28 23:32 ` Pasha Tatashin 2023-11-28 23:32 ` Pasha Tatashin 2023-11-28 20:49 ` [PATCH 06/16] iommu/dma: " Pasha Tatashin 2023-11-28 20:49 ` Pasha Tatashin 2023-11-28 22:33 ` Robin Murphy 2023-11-28 22:33 ` Robin Murphy 2023-11-28 22:50 ` Pasha Tatashin 2023-11-28 22:50 ` Pasha Tatashin 2023-11-28 22:59 ` Robin Murphy 2023-11-28 22:59 ` Robin Murphy 2023-11-28 23:06 ` Pasha Tatashin 2023-11-28 23:06 ` Pasha Tatashin 2023-11-28 23:08 ` Pasha Tatashin 2023-11-28 23:08 ` Pasha Tatashin 2023-11-28 20:49 ` [PATCH 07/16] iommu/exynos: " Pasha Tatashin 2023-11-28 20:49 ` Pasha Tatashin 2023-11-28 20:49 ` [PATCH 08/16] iommu/fsl: " Pasha Tatashin 2023-11-28 20:49 ` Pasha Tatashin 2023-11-28 22:53 ` Robin Murphy 2023-11-28 22:53 ` Robin Murphy 2023-11-28 23:00 ` Pasha Tatashin 2023-11-28 23:00 ` Pasha Tatashin 2023-11-28 23:50 ` Jason Gunthorpe 2023-11-28 23:50 ` Jason Gunthorpe 2023-11-29 16:48 ` Robin Murphy 2023-11-29 16:48 ` Robin Murphy 2023-11-29 19:45 ` Pasha Tatashin 2023-11-29 19:45 ` Pasha Tatashin 2023-11-29 20:03 ` Jason Gunthorpe 2023-11-29 20:03 ` Jason Gunthorpe 2023-11-29 20:44 ` Pasha Tatashin 2023-11-29 20:44 ` Pasha Tatashin 2023-11-28 20:49 ` [PATCH 09/16] iommu/iommufd: " Pasha Tatashin 2023-11-28 20:49 ` Pasha Tatashin 2023-11-28 23:52 ` Jason Gunthorpe 2023-11-28 23:52 ` Jason Gunthorpe 2023-11-29 21:59 ` Pasha Tatashin 2023-11-29 21:59 ` Pasha Tatashin 2023-11-30 0:02 ` Jason Gunthorpe 2023-11-30 0:02 ` Jason Gunthorpe 2023-11-28 20:49 ` [PATCH 10/16] iommu/rockchip: " Pasha Tatashin 2023-11-28 20:49 ` Pasha Tatashin 2023-11-28 20:49 ` [PATCH 11/16] iommu/sun50i: " Pasha Tatashin 2023-11-28 20:49 ` Pasha Tatashin 2023-11-28 20:49 ` [PATCH 12/16] iommu/tegra-smmu: " Pasha Tatashin 2023-11-28 20:49 ` Pasha Tatashin 2023-11-28 20:49 ` [PATCH 13/16] iommu: observability of the IOMMU allocations Pasha Tatashin 2023-11-28 20:49 ` Pasha Tatashin 2023-11-30 14:03 ` kernel test robot 2023-11-30 14:03 ` kernel test robot 2023-11-30 14:03 ` kernel test robot 2023-11-28 20:49 ` [PATCH 14/16] iommu: account IOMMU allocated memory Pasha Tatashin 2023-11-28 20:49 ` Pasha Tatashin 2023-11-28 20:49 ` [PATCH 15/16] vhost-vdpa: account iommu allocations Pasha Tatashin 2023-11-28 20:49 ` Pasha Tatashin 2023-12-25 16:09 ` Michael S. Tsirkin 2023-12-25 16:09 ` Michael S. Tsirkin 2023-12-26 18:23 ` Pasha Tatashin 2023-12-26 18:23 ` Pasha Tatashin 2023-11-28 20:49 ` [PATCH 16/16] vfio: " Pasha Tatashin 2023-11-28 20:49 ` Pasha Tatashin 2023-11-28 23:53 ` Jason Gunthorpe 2023-11-28 23:53 ` Jason Gunthorpe 2023-11-29 21:36 ` Pasha Tatashin 2023-11-29 21:36 ` Pasha Tatashin 2023-11-28 21:33 ` [PATCH 00/16] IOMMU memory observability Yosry Ahmed 2023-11-28 21:33 ` Yosry Ahmed 2023-11-28 22:31 ` Pasha Tatashin 2023-11-28 22:31 ` Pasha Tatashin 2023-11-28 23:03 ` Yosry Ahmed 2023-11-28 23:03 ` Yosry Ahmed 2023-11-28 23:52 ` Jason Gunthorpe 2023-11-28 23:52 ` Jason Gunthorpe 2023-11-29 0:25 ` Yosry Ahmed 2023-11-29 0:25 ` Yosry Ahmed 2023-11-29 0:28 ` Jason Gunthorpe 2023-11-29 0:28 ` Jason Gunthorpe 2023-11-29 0:30 ` Yosry Ahmed 2023-11-29 0:30 ` Yosry Ahmed 2023-11-29 0:54 ` Jason Gunthorpe 2023-11-29 0:54 ` Jason Gunthorpe
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20231128204938.1453583-1-pasha.tatashin@soleen.com \ --to=pasha.tatashin@soleen.com \ --cc=akpm@linux-foundation.org \ --cc=alex.williamson@redhat.com \ --cc=alim.akhtar@samsung.com \ --cc=alyssa@rosenzweig.io \ --cc=asahi@lists.linux.dev \ --cc=baolu.lu@linux.intel.com \ --cc=bhelgaas@google.com \ --cc=cgroups@vger.kernel.org \ --cc=corbet@lwn.net \ --cc=david@redhat.com \ --cc=dwmw2@infradead.org \ --cc=hannes@cmpxchg.org \ --cc=heiko@sntech.de \ --cc=iommu@lists.linux.dev \ --cc=jasowang@redhat.com \ --cc=jernej.skrabec@gmail.com \ --cc=jgg@ziepe.ca \ --cc=jonathanh@nvidia.com \ --cc=joro@8bytes.org \ --cc=kevin.tian@intel.com \ --cc=krzysztof.kozlowski@linaro.org \ --cc=kvm@vger.kernel.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-doc@vger.kernel.org \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-rockchip@lists.infradead.org \ --cc=linux-samsung-soc@vger.kernel.org \ --cc=linux-sunxi@lists.linux.dev \ --cc=linux-tegra@vger.kernel.org \ --cc=lizefan.x@bytedance.com \ --cc=m.szyprowski@samsung.com \ --cc=marcan@marcan.st \ --cc=mhiramat@kernel.org \ --cc=mst@redhat.com \ --cc=netdev@vger.kernel.org \ --cc=paulmck@kernel.org \ --cc=rdunlap@infradead.org \ --cc=robin.murphy@arm.com \ --cc=samuel@sholland.org \ --cc=suravee.suthikulpanit@amd.com \ --cc=sven@svenpeter.dev \ --cc=thierry.reding@gmail.com \ --cc=tj@kernel.org \ --cc=tomas.mudrunka@gmail.com \ --cc=vdumpa@nvidia.com \ --cc=virtualization@lists.linux.dev \ --cc=wens@csie.org \ --cc=will@kernel.org \ --cc=yu-cheng.yu@intel.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.