From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1D38BC433EF for ; Thu, 19 May 2022 07:24:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id DC4BE405E0; Thu, 19 May 2022 07:24:53 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dqZN3GgJvrmS; Thu, 19 May 2022 07:24:53 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id C13194057C; Thu, 19 May 2022 07:24:52 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id AACBEC0032; Thu, 19 May 2022 07:24:52 +0000 (UTC) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id AA3CEC002D for ; Thu, 19 May 2022 07:24:51 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 989824057C for ; Thu, 19 May 2022 07:24:51 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QekJ9b9zMDC7 for ; Thu, 19 May 2022 07:24:50 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by smtp2.osuosl.org (Postfix) with ESMTPS id DA39A4091A for ; Thu, 19 May 2022 07:24:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652945090; x=1684481090; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=YeJzuFoZ/UdMGyllJXeJZ4l5ck5D6EfYJagEliigKTg=; b=TFXEFeZ7CP/KYqvF7TTapsG0gbxd/R7v9gWoCQDcw3EbM/uEgFcj0Kjd Q1lCjWBXOapcfdSLkEByuneZ4ab9ADrU/1ZTYFL9ctkgnIMcmqXKmBlww mV6yVml2zrDva/+WuIqeaNlTR+pBpCuFuMAwdNXQxjr2FSm9z9WV7Sxiu xPn7d8maTn8SZT8DNgTiRZzxAE0SJrbfhL8MymXyNVBAHYY/e45g5BGb2 A/uOoajTM+PgTLUa6ehq18UYcFfLzNkK1gdkTYf563l2sy7hPcfW047oZ alRfAtMLG22EUd4dTQDEBnW0v8KP4rtsj27+41InmxSEUD9vdnKJocu3w g==; X-IronPort-AV: E=McAfee;i="6400,9594,10351"; a="335113265" X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="335113265" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 May 2022 00:24:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="714853152" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 19 May 2022 00:24:46 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Subject: [PATCH v7 08/10] iommu: Prepare IOMMU domain for IOPF Date: Thu, 19 May 2022 15:20:45 +0800 Message-Id: <20220519072047.2996983-9-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220519072047.2996983-1-baolu.lu@linux.intel.com> References: <20220519072047.2996983-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Cc: Jean-Philippe Brucker , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, Jacob jun Pan X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" This adds some mechanisms around the iommu_domain so that the I/O page fault handling framework could route a page fault to the domain and call the fault handler from it. Add pointers to the page fault handler and its private data in struct iommu_domain. The fault handler will be called with the private data as a parameter once a page fault is routed to the domain. Any kernel component which owns an iommu domain could install handler and its private parameter so that the page fault could be further routed and handled. This also prepares the SVA implementation to be the first consumer of the per-domain page fault handling model. Suggested-by: Jean-Philippe Brucker Signed-off-by: Lu Baolu --- include/linux/iommu.h | 3 ++ drivers/iommu/io-pgfault.c | 7 ++++ drivers/iommu/iommu-sva-lib.c | 65 +++++++++++++++++++++++++++++++++++ 3 files changed, 75 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index e4ce2fe0e144..45f274b2640d 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -100,6 +100,9 @@ struct iommu_domain { void *handler_token; struct iommu_domain_geometry geometry; struct iommu_dma_cookie *iova_cookie; + enum iommu_page_response_code (*iopf_handler)(struct iommu_fault *fault, + void *data); + void *fault_data; }; static inline bool iommu_is_dma_domain(struct iommu_domain *domain) diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c index 1df8c1dcae77..aee9e033012f 100644 --- a/drivers/iommu/io-pgfault.c +++ b/drivers/iommu/io-pgfault.c @@ -181,6 +181,13 @@ static void iopf_handle_group(struct work_struct *work) * request completes, outstanding faults will have been dealt with by the time * the PASID is freed. * + * Any valid page fault will be eventually routed to an iommu domain and the + * page fault handler installed there will get called. The users of this + * handling framework should guarantee that the iommu domain could only be + * freed after the device has stopped generating page faults (or the iommu + * hardware has been set to block the page faults) and the pending page faults + * have been flushed. + * * Return: 0 on success and <0 on error. */ int iommu_queue_iopf(struct iommu_fault *fault, void *cookie) diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index 568e0f64edac..317ab8e8c149 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -72,6 +72,69 @@ struct mm_struct *iommu_sva_find(ioasid_t pasid) } EXPORT_SYMBOL_GPL(iommu_sva_find); +/* + * I/O page fault handler for SVA + * + * Copied from io-pgfault.c with mmget_not_zero() added before + * mmap_read_lock(). + */ +static enum iommu_page_response_code +iommu_sva_handle_iopf(struct iommu_fault *fault, void *data) +{ + vm_fault_t ret; + struct mm_struct *mm; + struct vm_area_struct *vma; + unsigned int access_flags = 0; + struct iommu_domain *domain = data; + unsigned int fault_flags = FAULT_FLAG_REMOTE; + struct iommu_fault_page_request *prm = &fault->prm; + enum iommu_page_response_code status = IOMMU_PAGE_RESP_INVALID; + + if (!(prm->flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID)) + return status; + + mm = domain_to_mm(domain); + if (IS_ERR_OR_NULL(mm) || !mmget_not_zero(mm)) + return status; + + mmap_read_lock(mm); + + vma = find_extend_vma(mm, prm->addr); + if (!vma) + /* Unmapped area */ + goto out_put_mm; + + if (prm->perm & IOMMU_FAULT_PERM_READ) + access_flags |= VM_READ; + + if (prm->perm & IOMMU_FAULT_PERM_WRITE) { + access_flags |= VM_WRITE; + fault_flags |= FAULT_FLAG_WRITE; + } + + if (prm->perm & IOMMU_FAULT_PERM_EXEC) { + access_flags |= VM_EXEC; + fault_flags |= FAULT_FLAG_INSTRUCTION; + } + + if (!(prm->perm & IOMMU_FAULT_PERM_PRIV)) + fault_flags |= FAULT_FLAG_USER; + + if (access_flags & ~vma->vm_flags) + /* Access fault */ + goto out_put_mm; + + ret = handle_mm_fault(vma, prm->addr, fault_flags, NULL); + status = ret & VM_FAULT_ERROR ? IOMMU_PAGE_RESP_INVALID : + IOMMU_PAGE_RESP_SUCCESS; + +out_put_mm: + mmap_read_unlock(mm); + mmput(mm); + + return status; +} + /* * IOMMU SVA driver-oriented interfaces */ @@ -94,6 +157,8 @@ iommu_sva_alloc_domain(struct bus_type *bus, struct mm_struct *mm) domain = &sva_domain->domain; domain->type = IOMMU_DOMAIN_SVA; domain->ops = bus->iommu_ops->sva_domain_ops; + domain->iopf_handler = iommu_sva_handle_iopf; + domain->fault_data = domain; return domain; } -- 2.25.1 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4DC1DC4332F for ; Thu, 19 May 2022 07:25:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234786AbiESHZQ (ORCPT ); Thu, 19 May 2022 03:25:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234779AbiESHYx (ORCPT ); Thu, 19 May 2022 03:24:53 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A4BBA8020E for ; Thu, 19 May 2022 00:24:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652945090; x=1684481090; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=YeJzuFoZ/UdMGyllJXeJZ4l5ck5D6EfYJagEliigKTg=; b=TFXEFeZ7CP/KYqvF7TTapsG0gbxd/R7v9gWoCQDcw3EbM/uEgFcj0Kjd Q1lCjWBXOapcfdSLkEByuneZ4ab9ADrU/1ZTYFL9ctkgnIMcmqXKmBlww mV6yVml2zrDva/+WuIqeaNlTR+pBpCuFuMAwdNXQxjr2FSm9z9WV7Sxiu xPn7d8maTn8SZT8DNgTiRZzxAE0SJrbfhL8MymXyNVBAHYY/e45g5BGb2 A/uOoajTM+PgTLUa6ehq18UYcFfLzNkK1gdkTYf563l2sy7hPcfW047oZ alRfAtMLG22EUd4dTQDEBnW0v8KP4rtsj27+41InmxSEUD9vdnKJocu3w g==; X-IronPort-AV: E=McAfee;i="6400,9594,10351"; a="272195986" X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="272195986" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 May 2022 00:24:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,236,1647327600"; d="scan'208";a="714853152" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 19 May 2022 00:24:46 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v7 08/10] iommu: Prepare IOMMU domain for IOPF Date: Thu, 19 May 2022 15:20:45 +0800 Message-Id: <20220519072047.2996983-9-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220519072047.2996983-1-baolu.lu@linux.intel.com> References: <20220519072047.2996983-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This adds some mechanisms around the iommu_domain so that the I/O page fault handling framework could route a page fault to the domain and call the fault handler from it. Add pointers to the page fault handler and its private data in struct iommu_domain. The fault handler will be called with the private data as a parameter once a page fault is routed to the domain. Any kernel component which owns an iommu domain could install handler and its private parameter so that the page fault could be further routed and handled. This also prepares the SVA implementation to be the first consumer of the per-domain page fault handling model. Suggested-by: Jean-Philippe Brucker Signed-off-by: Lu Baolu --- include/linux/iommu.h | 3 ++ drivers/iommu/io-pgfault.c | 7 ++++ drivers/iommu/iommu-sva-lib.c | 65 +++++++++++++++++++++++++++++++++++ 3 files changed, 75 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index e4ce2fe0e144..45f274b2640d 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -100,6 +100,9 @@ struct iommu_domain { void *handler_token; struct iommu_domain_geometry geometry; struct iommu_dma_cookie *iova_cookie; + enum iommu_page_response_code (*iopf_handler)(struct iommu_fault *fault, + void *data); + void *fault_data; }; static inline bool iommu_is_dma_domain(struct iommu_domain *domain) diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c index 1df8c1dcae77..aee9e033012f 100644 --- a/drivers/iommu/io-pgfault.c +++ b/drivers/iommu/io-pgfault.c @@ -181,6 +181,13 @@ static void iopf_handle_group(struct work_struct *work) * request completes, outstanding faults will have been dealt with by the time * the PASID is freed. * + * Any valid page fault will be eventually routed to an iommu domain and the + * page fault handler installed there will get called. The users of this + * handling framework should guarantee that the iommu domain could only be + * freed after the device has stopped generating page faults (or the iommu + * hardware has been set to block the page faults) and the pending page faults + * have been flushed. + * * Return: 0 on success and <0 on error. */ int iommu_queue_iopf(struct iommu_fault *fault, void *cookie) diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index 568e0f64edac..317ab8e8c149 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -72,6 +72,69 @@ struct mm_struct *iommu_sva_find(ioasid_t pasid) } EXPORT_SYMBOL_GPL(iommu_sva_find); +/* + * I/O page fault handler for SVA + * + * Copied from io-pgfault.c with mmget_not_zero() added before + * mmap_read_lock(). + */ +static enum iommu_page_response_code +iommu_sva_handle_iopf(struct iommu_fault *fault, void *data) +{ + vm_fault_t ret; + struct mm_struct *mm; + struct vm_area_struct *vma; + unsigned int access_flags = 0; + struct iommu_domain *domain = data; + unsigned int fault_flags = FAULT_FLAG_REMOTE; + struct iommu_fault_page_request *prm = &fault->prm; + enum iommu_page_response_code status = IOMMU_PAGE_RESP_INVALID; + + if (!(prm->flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID)) + return status; + + mm = domain_to_mm(domain); + if (IS_ERR_OR_NULL(mm) || !mmget_not_zero(mm)) + return status; + + mmap_read_lock(mm); + + vma = find_extend_vma(mm, prm->addr); + if (!vma) + /* Unmapped area */ + goto out_put_mm; + + if (prm->perm & IOMMU_FAULT_PERM_READ) + access_flags |= VM_READ; + + if (prm->perm & IOMMU_FAULT_PERM_WRITE) { + access_flags |= VM_WRITE; + fault_flags |= FAULT_FLAG_WRITE; + } + + if (prm->perm & IOMMU_FAULT_PERM_EXEC) { + access_flags |= VM_EXEC; + fault_flags |= FAULT_FLAG_INSTRUCTION; + } + + if (!(prm->perm & IOMMU_FAULT_PERM_PRIV)) + fault_flags |= FAULT_FLAG_USER; + + if (access_flags & ~vma->vm_flags) + /* Access fault */ + goto out_put_mm; + + ret = handle_mm_fault(vma, prm->addr, fault_flags, NULL); + status = ret & VM_FAULT_ERROR ? IOMMU_PAGE_RESP_INVALID : + IOMMU_PAGE_RESP_SUCCESS; + +out_put_mm: + mmap_read_unlock(mm); + mmput(mm); + + return status; +} + /* * IOMMU SVA driver-oriented interfaces */ @@ -94,6 +157,8 @@ iommu_sva_alloc_domain(struct bus_type *bus, struct mm_struct *mm) domain = &sva_domain->domain; domain->type = IOMMU_DOMAIN_SVA; domain->ops = bus->iommu_ops->sva_domain_ops; + domain->iopf_handler = iommu_sva_handle_iopf; + domain->fault_data = domain; return domain; } -- 2.25.1