From: John Garry <john.garry@huawei.com> To: <joro@8bytes.org>, <will@kernel.org>, <robin.murphy@arm.com>, <baolu.lu@linux.intel.com> Cc: <iommu@lists.linux-foundation.org>, <linuxarm@huawei.com>, <thierry.reding@gmail.com>, <airlied@linux.ie>, <daniel@ffwll.ch>, <jonathanh@nvidia.com>, <sakari.ailus@linux.intel.com>, <bingbu.cao@intel.com>, <tian.shu.qiu@intel.com>, <mchehab@kernel.org>, <gregkh@linuxfoundation.org>, <digetx@gmail.com>, <mst@redhat.com>, <jasowang@redhat.com>, <linux-kernel@vger.kernel.org>, <chenxiang66@hisilicon.com>, John Garry <john.garry@huawei.com> Subject: [PATCH v4 0/6] iommu: Allow IOVA rcache range be configured Date: Wed, 14 Jul 2021 18:36:37 +0800 [thread overview] Message-ID: <1626259003-201303-1-git-send-email-john.garry@huawei.com> (raw) For streaming DMA mappings involving an IOMMU and whose IOVA len regularly exceeds the IOVA rcache upper limit (meaning that they are not cached), performance can be reduced. This may be much more pronounced from commit 4e89dce72521 ("iommu/iova: Retry from last rb tree node if iova search fails"), as discussed at [0]. IOVAs which cannot be cached are highly involved in the IOVA ageing issue, as discussed at [1]. This series allows the IOVA rcache range be configured, so that we may cache all IOVAs per domain, thus improving performance. A new IOMMU group sysfs file is added - max_opt_dma_size - which is used indirectly to configure the IOVA rcache range: /sys/kernel/iommu_groups/X/max_opt_dma_size This file is updated same as how the IOMMU group default domain type is updated, i.e. must unbind the only device in the group first. The inspiration here comes from block layer request queue sysfs "optimal_io_size" file, in /sys/block/sdX/queue/optimal_io_size Some figures for storage scenario (when increasing IOVA rcache range to cover all DMA mapping sizes from the LLD): v5.13-rc1 baseline: 1200K IOPS With series: 1800K IOPS All above are for IOMMU strict mode. Non-strict mode gives ~1800K IOPS in all scenarios. [0] https://lore.kernel.org/linux-iommu/20210129092120.1482-1-thunder.leizhen@huawei.com/ [1] https://lore.kernel.org/linux-iommu/1607538189-237944-1-git-send-email-john.garry@huawei.com/ Note that I cc'ed maintainers/reviewers only for the changes associated with patch #5 since it just touches their code in only a minor way. John Garry (6): iommu: Refactor iommu_group_store_type() iova: Allow rcache range upper limit to be flexible iommu: Allow iommu_change_dev_def_domain() realloc default domain for same type iommu: Allow max opt DMA len be set for a group via sysfs iova: Add iova_len argument to init_iova_domain() dma-iommu: Pass iova len for IOVA domain init .../ABI/testing/sysfs-kernel-iommu_groups | 16 ++ drivers/gpu/drm/tegra/drm.c | 2 +- drivers/gpu/host1x/dev.c | 2 +- drivers/iommu/dma-iommu.c | 15 +- drivers/iommu/iommu.c | 172 ++++++++++++------ drivers/iommu/iova.c | 39 +++- drivers/staging/media/ipu3/ipu3-dmamap.c | 2 +- drivers/staging/media/tegra-vde/iommu.c | 2 +- drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 +- include/linux/iommu.h | 6 + include/linux/iova.h | 9 +- 11 files changed, 194 insertions(+), 73 deletions(-) -- 2.26.2
WARNING: multiple messages have this Message-ID (diff)
From: John Garry <john.garry@huawei.com> To: <joro@8bytes.org>, <will@kernel.org>, <robin.murphy@arm.com>, <baolu.lu@linux.intel.com> Cc: linux-kernel@vger.kernel.org, sakari.ailus@linux.intel.com, mst@redhat.com, airlied@linux.ie, gregkh@linuxfoundation.org, jasowang@redhat.com, linuxarm@huawei.com, jonathanh@nvidia.com, iommu@lists.linux-foundation.org, thierry.reding@gmail.com, daniel@ffwll.ch, bingbu.cao@intel.com, digetx@gmail.com, mchehab@kernel.org, tian.shu.qiu@intel.com Subject: [PATCH v4 0/6] iommu: Allow IOVA rcache range be configured Date: Wed, 14 Jul 2021 18:36:37 +0800 [thread overview] Message-ID: <1626259003-201303-1-git-send-email-john.garry@huawei.com> (raw) For streaming DMA mappings involving an IOMMU and whose IOVA len regularly exceeds the IOVA rcache upper limit (meaning that they are not cached), performance can be reduced. This may be much more pronounced from commit 4e89dce72521 ("iommu/iova: Retry from last rb tree node if iova search fails"), as discussed at [0]. IOVAs which cannot be cached are highly involved in the IOVA ageing issue, as discussed at [1]. This series allows the IOVA rcache range be configured, so that we may cache all IOVAs per domain, thus improving performance. A new IOMMU group sysfs file is added - max_opt_dma_size - which is used indirectly to configure the IOVA rcache range: /sys/kernel/iommu_groups/X/max_opt_dma_size This file is updated same as how the IOMMU group default domain type is updated, i.e. must unbind the only device in the group first. The inspiration here comes from block layer request queue sysfs "optimal_io_size" file, in /sys/block/sdX/queue/optimal_io_size Some figures for storage scenario (when increasing IOVA rcache range to cover all DMA mapping sizes from the LLD): v5.13-rc1 baseline: 1200K IOPS With series: 1800K IOPS All above are for IOMMU strict mode. Non-strict mode gives ~1800K IOPS in all scenarios. [0] https://lore.kernel.org/linux-iommu/20210129092120.1482-1-thunder.leizhen@huawei.com/ [1] https://lore.kernel.org/linux-iommu/1607538189-237944-1-git-send-email-john.garry@huawei.com/ Note that I cc'ed maintainers/reviewers only for the changes associated with patch #5 since it just touches their code in only a minor way. John Garry (6): iommu: Refactor iommu_group_store_type() iova: Allow rcache range upper limit to be flexible iommu: Allow iommu_change_dev_def_domain() realloc default domain for same type iommu: Allow max opt DMA len be set for a group via sysfs iova: Add iova_len argument to init_iova_domain() dma-iommu: Pass iova len for IOVA domain init .../ABI/testing/sysfs-kernel-iommu_groups | 16 ++ drivers/gpu/drm/tegra/drm.c | 2 +- drivers/gpu/host1x/dev.c | 2 +- drivers/iommu/dma-iommu.c | 15 +- drivers/iommu/iommu.c | 172 ++++++++++++------ drivers/iommu/iova.c | 39 +++- drivers/staging/media/ipu3/ipu3-dmamap.c | 2 +- drivers/staging/media/tegra-vde/iommu.c | 2 +- drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 +- include/linux/iommu.h | 6 + include/linux/iova.h | 9 +- 11 files changed, 194 insertions(+), 73 deletions(-) -- 2.26.2 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
next reply other threads:[~2021-07-14 10:41 UTC|newest] Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-07-14 10:36 John Garry [this message] 2021-07-14 10:36 ` [PATCH v4 0/6] iommu: Allow IOVA rcache range be configured John Garry 2021-07-14 10:36 ` [PATCH v4 1/6] iommu: Refactor iommu_group_store_type() John Garry 2021-07-14 10:36 ` John Garry 2021-08-02 14:46 ` Will Deacon 2021-08-02 14:46 ` Will Deacon 2021-07-14 10:36 ` [PATCH v4 2/6] iova: Allow rcache range upper limit to be flexible John Garry 2021-07-14 10:36 ` John Garry 2021-08-02 15:01 ` Will Deacon 2021-08-02 15:01 ` Will Deacon 2021-08-02 15:23 ` John Garry 2021-08-02 15:23 ` John Garry 2021-08-02 16:09 ` Robin Murphy 2021-08-02 16:09 ` Robin Murphy 2021-07-14 10:36 ` [PATCH v4 3/6] iommu: Allow iommu_change_dev_def_domain() realloc default domain for same type John Garry 2021-07-14 10:36 ` John Garry 2021-07-14 10:36 ` [PATCH v4 4/6] iommu: Allow max opt DMA len be set for a group via sysfs John Garry 2021-07-14 10:36 ` John Garry 2021-07-14 10:36 ` [PATCH v4 5/6] iova: Add iova_len argument to init_iova_domain() John Garry 2021-07-14 10:36 ` John Garry 2021-08-02 15:06 ` Will Deacon 2021-08-02 15:06 ` Will Deacon 2021-08-02 16:06 ` John Garry 2021-08-02 16:06 ` John Garry 2021-08-02 16:16 ` Robin Murphy 2021-08-02 16:16 ` Robin Murphy 2021-08-02 16:40 ` John Garry 2021-08-02 16:40 ` John Garry 2021-08-02 17:18 ` John Garry 2021-08-02 17:18 ` John Garry 2021-09-21 8:48 ` John Garry 2021-09-21 8:48 ` John Garry 2021-07-14 10:36 ` [PATCH v4 6/6] dma-iommu: Pass iova len for IOVA domain init John Garry 2021-07-14 10:36 ` John Garry 2021-07-15 1:36 kernel test robot 2021-07-19 7:58 ` Dan Carpenter 2021-07-19 7:58 ` Dan Carpenter 2021-07-19 9:12 ` John Garry 2021-07-19 9:12 ` John Garry 2021-07-19 9:32 ` Robin Murphy 2021-07-19 9:32 ` Robin Murphy 2021-07-19 10:45 ` John Garry 2021-07-19 10:45 ` John Garry
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1626259003-201303-1-git-send-email-john.garry@huawei.com \ --to=john.garry@huawei.com \ --cc=airlied@linux.ie \ --cc=baolu.lu@linux.intel.com \ --cc=bingbu.cao@intel.com \ --cc=chenxiang66@hisilicon.com \ --cc=daniel@ffwll.ch \ --cc=digetx@gmail.com \ --cc=gregkh@linuxfoundation.org \ --cc=iommu@lists.linux-foundation.org \ --cc=jasowang@redhat.com \ --cc=jonathanh@nvidia.com \ --cc=joro@8bytes.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linuxarm@huawei.com \ --cc=mchehab@kernel.org \ --cc=mst@redhat.com \ --cc=robin.murphy@arm.com \ --cc=sakari.ailus@linux.intel.com \ --cc=thierry.reding@gmail.com \ --cc=tian.shu.qiu@intel.com \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.