From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756968AbcEETiX (ORCPT ); Thu, 5 May 2016 15:38:23 -0400 Received: from mail-bl2on0094.outbound.protection.outlook.com ([65.55.169.94]:33632 "EHLO na01-bl2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754771AbcEETiW (ORCPT ); Thu, 5 May 2016 15:38:22 -0400 X-Greylist: delayed 957 seconds by postgrey-1.27 at vger.kernel.org; Thu, 05 May 2016 15:38:21 EDT From: "Chalamarla, Tirumalesh" To: Eric Auger , "eric.auger@st.com" , "robin.murphy@arm.com" , "alex.williamson@redhat.com" , "will.deacon@arm.com" , "joro@8bytes.org" , "tglx@linutronix.de" , "jason@lakedaemon.net" , "marc.zyngier@arm.com" , "christoffer.dall@linaro.org" , "linux-arm-kernel@lists.infradead.org" CC: "julien.grall@arm.com" , "patches@linaro.org" , "Jean-Philippe.Brucker@arm.com" , "p.fedin@samsung.com" , "linux-kernel@vger.kernel.org" , "Bharat.Bhushan@freescale.com" , "iommu@lists.linux-foundation.org" , "pranav.sawargaonkar@gmail.com" , "yehuday@marvell.com" Subject: Re: [PATCH v9 4/7] vfio: allow reserved msi iova registration Thread-Topic: [PATCH v9 4/7] vfio: allow reserved msi iova registration Thread-Index: AQHRpfxF3ucH3hF0UUmGzdbhkvsFl5+qRZCA Date: Thu, 5 May 2016 19:22:20 +0000 Message-ID: References: <1462362858-2925-1-git-send-email-eric.auger@linaro.org> <1462362858-2925-5-git-send-email-eric.auger@linaro.org> In-Reply-To: <1462362858-2925-5-git-send-email-eric.auger@linaro.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/f.15.1.160411 authentication-results: linaro.org; dkim=none (message not signed) header.d=none;linaro.org; dmarc=none action=none header.from=caviumnetworks.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [2602:306:32e8:da10:c1af:ec9e:fccb:81e3] x-ms-office365-filtering-correlation-id: 0554cc3d-9392-4050-5879-08d3751a9483 x-microsoft-exchange-diagnostics: 1;BLUPR0701MB1780;5:QB1ysKRb5ZHeOiVLU9JyiDOHeyrF9L6tNm7v0Z/OwA3Q6P7VbNQbjopmLhS1nc7EUdvXEJcsyYxDj7IRYzZfyfmN4+/axz1yv5aH+bKP7T5Pyd5hP8U+e2bo2bKFQLPNQnbsdQXOrU7xb63pSdrLzg==;24:b+yKtL6APwchfuchBBY8ymushkVRWy9IrJtO6cB1OTBOkJQHSbFqB7CNVF7ofKo11xzSE6yTPcS8mz+ixHWmNplOFxBo6CJ02Rd5eDpzG8A=;7:GX6IuLmuR8pd5zr2GtJQT0rDJenqVqLaarVpXIbMDWxcE6M41hW+6JivUH9cMJwhAWGLVOaz89FYdVNzd2aeSlXnsBSshb1ls7hkGbKP9zmeKJEDRkTn+MWKD4cE07iFb3ztc0vARQqwMsjrathXWRMtuRNlQ9xwdzuGRpzXlMUbSgKuuEu4Ln5eYozHK46D x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BLUPR0701MB1780; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(9101521098)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046);SRVR:BLUPR0701MB1780;BCL:0;PCL:0;RULEID:;SRVR:BLUPR0701MB1780; x-forefront-prvs: 0933E9FD8D x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(6009001)(377454003)(24454002)(82746002)(83506001)(2906002)(4326007)(1220700001)(6116002)(102836003)(586003)(19580405001)(19580395003)(87936001)(5002640100001)(5008740100001)(81166005)(36756003)(3280700002)(3660700001)(2900100001)(106116001)(99286002)(189998001)(10400500002)(11100500001)(8936002)(1720100001)(15975445007)(77096005)(4001350100001)(5004730100002)(2501003)(92566002)(86362001)(2201001)(33656002)(83716003)(2950100001)(50986999)(122556002)(5001770100001)(54356999)(76176999)(7059030)(921003)(3826002)(1121003)(104396002);DIR:OUT;SFP:1101;SCL:1;SRVR:BLUPR0701MB1780;H:BLUPR0701MB1780.namprd07.prod.outlook.com;FPR:;SPF:None;MLV:sfv;LANG:en; spamdiagnosticoutput: 1:23 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-ID: <09F7CC7C3F3C084EA4D6749DBE1486D1@namprd07.prod.outlook.com> MIME-Version: 1.0 X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-originalarrivaltime: 05 May 2016 19:22:20.9982 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR0701MB1780 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by mail.home.local id u45JcSbX012560 On 5/4/16, 4:54 AM, "linux-arm-kernel on behalf of Eric Auger" wrote: >The user is allowed to register a reserved MSI IOVA range by using the >DMA MAP API and setting the new flag: VFIO_DMA_MAP_FLAG_MSI_RESERVED_IOVA. >This region is stored in the vfio_dma rb tree. At that point the iova >range is not mapped to any target address yet. The host kernel will use >those iova when needed, typically when MSIs are allocated. > >Signed-off-by: Eric Auger >Signed-off-by: Bharat Bhushan > >--- >v7 -> v8: >- use iommu_msi_set_aperture function. There is no notion of > unregistration anymore since the reserved msi slot remains > until the container gets closed. > >v6 -> v7: >- use iommu_free_reserved_iova_domain >- convey prot attributes downto dma-reserved-iommu iova domain creation >- reserved bindings teardown now performed on iommu domain destruction >- rename VFIO_DMA_MAP_FLAG_MSI_RESERVED_IOVA into > VFIO_DMA_MAP_FLAG_RESERVED_MSI_IOVA >- change title >- pass the protection attribute to dma-reserved-iommu API > >v3 -> v4: >- use iommu_alloc/free_reserved_iova_domain exported by dma-reserved-iommu >- protect vfio_register_reserved_iova_range implementation with > CONFIG_IOMMU_DMA_RESERVED >- handle unregistration by user-space and on vfio_iommu_type1 release > >v1 -> v2: >- set returned value according to alloc_reserved_iova_domain result >- free the iova domains in case any error occurs > >RFC v1 -> v1: >- takes into account Alex comments, based on > [RFC PATCH 1/6] vfio: Add interface for add/del reserved iova region: >- use the existing dma map/unmap ioctl interface with a flag to register > a reserved IOVA range. A single reserved iova region is allowed. >--- > drivers/vfio/vfio_iommu_type1.c | 78 ++++++++++++++++++++++++++++++++++++++++- > include/uapi/linux/vfio.h | 10 +++++- > 2 files changed, 86 insertions(+), 2 deletions(-) > >diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c >index 94a9916..4d3a6f1 100644 >--- a/drivers/vfio/vfio_iommu_type1.c >+++ b/drivers/vfio/vfio_iommu_type1.c >@@ -36,6 +36,7 @@ > #include > #include > #include >+#include > > #define DRIVER_VERSION "0.2" > #define DRIVER_AUTHOR "Alex Williamson " >@@ -445,6 +446,20 @@ static void vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma) > vfio_lock_acct(-unlocked); > } > >+static int vfio_set_msi_aperture(struct vfio_iommu *iommu, >+ dma_addr_t iova, size_t size) >+{ >+ struct vfio_domain *d; >+ int ret = 0; >+ >+ list_for_each_entry(d, &iommu->domain_list, next) { >+ ret = iommu_msi_set_aperture(d->domain, iova, iova + size - 1); >+ if (ret) >+ break; >+ } >+ return ret; >+} >+ > static void vfio_remove_dma(struct vfio_iommu *iommu, struct vfio_dma *dma) > { > vfio_unmap_unpin(iommu, dma); >@@ -693,6 +708,63 @@ static int vfio_dma_do_map(struct vfio_iommu *iommu, > return ret; > } > >+static int vfio_register_msi_range(struct vfio_iommu *iommu, >+ struct vfio_iommu_type1_dma_map *map) >+{ >+ dma_addr_t iova = map->iova; >+ size_t size = map->size; >+ int ret = 0; >+ struct vfio_dma *dma; >+ unsigned long order; >+ uint64_t mask; >+ >+ /* Verify that none of our __u64 fields overflow */ >+ if (map->size != size || map->iova != iova) >+ return -EINVAL; >+ >+ order = __ffs(vfio_pgsize_bitmap(iommu)); >+ mask = ((uint64_t)1 << order) - 1; >+ >+ WARN_ON(mask & PAGE_MASK); >+ >+ if (!size || (size | iova) & mask) >+ return -EINVAL; >+ >+ /* Don't allow IOVA address wrap */ >+ if (iova + size - 1 < iova) >+ return -EINVAL; >+ >+ mutex_lock(&iommu->lock); >+ >+ if (vfio_find_dma(iommu, iova, size, VFIO_IOVA_ANY)) { >+ ret = -EEXIST; >+ goto unlock; >+ } >+ >+ dma = kzalloc(sizeof(*dma), GFP_KERNEL); >+ if (!dma) { >+ ret = -ENOMEM; >+ goto unlock; >+ } >+ >+ dma->iova = iova; >+ dma->size = size; >+ dma->type = VFIO_IOVA_RESERVED; >+ >+ ret = vfio_set_msi_aperture(iommu, iova, size); >+ if (ret) >+ goto free_unlock; >+ >+ vfio_link_dma(iommu, dma); >+ goto unlock; >+ >+free_unlock: >+ kfree(dma); >+unlock: >+ mutex_unlock(&iommu->lock); >+ return ret; >+} >+ > static int vfio_bus_type(struct device *dev, void *data) > { > struct bus_type **bus = data; >@@ -1062,7 +1134,8 @@ static long vfio_iommu_type1_ioctl(void *iommu_data, > } else if (cmd == VFIO_IOMMU_MAP_DMA) { > struct vfio_iommu_type1_dma_map map; > uint32_t mask = VFIO_DMA_MAP_FLAG_READ | >- VFIO_DMA_MAP_FLAG_WRITE; >+ VFIO_DMA_MAP_FLAG_WRITE | >+ VFIO_DMA_MAP_FLAG_RESERVED_MSI_IOVA; > > minsz = offsetofend(struct vfio_iommu_type1_dma_map, size); > >@@ -1072,6 +1145,9 @@ static long vfio_iommu_type1_ioctl(void *iommu_data, > if (map.argsz < minsz || map.flags & ~mask) > return -EINVAL; > >+ if (map.flags & VFIO_DMA_MAP_FLAG_RESERVED_MSI_IOVA) >+ return vfio_register_msi_range(iommu, &map); >+ How are we getting the PA and any validation needed? > return vfio_dma_do_map(iommu, &map); > > } else if (cmd == VFIO_IOMMU_UNMAP_DMA) { >diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h >index 255a211..4a9dbc2 100644 >--- a/include/uapi/linux/vfio.h >+++ b/include/uapi/linux/vfio.h >@@ -498,12 +498,19 @@ struct vfio_iommu_type1_info { > * > * Map process virtual addresses to IO virtual addresses using the > * provided struct vfio_dma_map. Caller sets argsz. READ &/ WRITE required. >+ * >+ * In case RESERVED_MSI_IOVA flag is set, the API only aims at registering an >+ * IOVA region that will be used on some platforms to map the host MSI frames. >+ * In that specific case, vaddr is ignored. Once registered, an MSI reserved >+ * IOVA region stays until the container is closed. > */ > struct vfio_iommu_type1_dma_map { > __u32 argsz; > __u32 flags; > #define VFIO_DMA_MAP_FLAG_READ (1 << 0) /* readable from device */ > #define VFIO_DMA_MAP_FLAG_WRITE (1 << 1) /* writable from device */ >+/* reserved iova for MSI vectors*/ >+#define VFIO_DMA_MAP_FLAG_RESERVED_MSI_IOVA (1 << 2) > __u64 vaddr; /* Process virtual address */ > __u64 iova; /* IO virtual address */ > __u64 size; /* Size of mapping (bytes) */ >@@ -519,7 +526,8 @@ struct vfio_iommu_type1_dma_map { > * Caller sets argsz. The actual unmapped size is returned in the size > * field. No guarantee is made to the user that arbitrary unmaps of iova > * or size different from those used in the original mapping call will >- * succeed. >+ * succeed. Once registered, an MSI region cannot be unmapped and stays >+ * until the container is closed. > */ > struct vfio_iommu_type1_dma_unmap { > __u32 argsz; >-- >1.9.1 > > >_______________________________________________ >linux-arm-kernel mailing list >linux-arm-kernel@lists.infradead.org >http://lists.infradead.org/mailman/listinfo/linux-arm-kernel