From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39DDCC433F5 for ; Mon, 2 May 2022 01:52:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1356921AbiEBB4O (ORCPT ); Sun, 1 May 2022 21:56:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54974 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1379593AbiEBB4H (ORCPT ); Sun, 1 May 2022 21:56:07 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B786237DA for ; Sun, 1 May 2022 18:52:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1651456344; x=1682992344; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=M/ivYWYb1uHUMYAnPkhVKPcpt1S+M8X85T3T6KK3ZXI=; b=fs8mUcIB7FU/sZgAKVNCCIsI8NZ7OZ4goVsfNJDRDB6dWcUD/ua+bRAO RoYCoIqQjpsFnDeD5cKrIwQVEBX2zNrxAR/1B5830lReGz8Vjxwq7ATf8 pFJw+E2LfVbQttchks35vc30BTTVUekrpytbMuKfTwaN87Aze0c6PTYlj uuYmJI35uzPa2coU4DjvQiMPY8sbKVCxUYvLKIu69w9DK7bkmn7HWi+Y/ 9mAWDz0+sw+b9tHMVIPhBAsKGbeOvoAWzKcABoh2xRSVDhnTn5ed2zq9h JMChI1ECbFh8GAEUcxGp21H5+XcoZopQ9Qxd7QINkVTe0kqJM3smJ0+es g==; X-IronPort-AV: E=McAfee;i="6400,9594,10334"; a="249062084" X-IronPort-AV: E=Sophos;i="5.91,190,1647327600"; d="scan'208";a="249062084" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 May 2022 18:52:24 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,190,1647327600"; d="scan'208";a="707406520" Received: from allen-box.sh.intel.com ([10.239.159.48]) by fmsmga001.fm.intel.com with ESMTP; 01 May 2022 18:52:20 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v5 10/12] iommu: Prepare IOMMU domain for IOPF Date: Mon, 2 May 2022 09:48:40 +0800 Message-Id: <20220502014842.991097-11-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220502014842.991097-1-baolu.lu@linux.intel.com> References: <20220502014842.991097-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This adds some mechanisms around the iommu_domain so that the I/O page fault handling framework could route a page fault to the domain and call the fault handler from it. Add pointers to the page fault handler and its private data in struct iommu_domain. The fault handler will be called with the private data as a parameter once a page fault is routed to the domain. Any kernel component which owns an iommu domain could install handler and its private parameter so that the page fault could be further routed and handled. A new helper iommu_get_domain_for_dev_pasid() which retrieves attached domain for a {device, PASID} is added. It will be used by the page fault handling framework which knows {device, PASID} reported from the iommu driver. We have a guarantee that the SVA domain doesn't go away during IOPF handling, because unbind() waits for pending faults with iopf_queue_flush_dev() before freeing the domain. Hence, there's no need to synchronize life cycle of the iommu domains between the unbind() and the interrupt threads. This also prepares the SVA implementation to be the first consumer of the per-domain page fault handling model. Signed-off-by: Lu Baolu --- include/linux/iommu.h | 12 +++++++ drivers/iommu/iommu-sva-lib.c | 65 +++++++++++++++++++++++++++++++++++ drivers/iommu/iommu.c | 21 +++++++++++ 3 files changed, 98 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 19718939d9df..1164524814cb 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -102,6 +102,9 @@ struct iommu_domain { struct iommu_domain_geometry geometry; struct iommu_dma_cookie *iova_cookie; struct iommu_sva_ioas *sva_ioas; + enum iommu_page_response_code (*iopf_handler)(struct iommu_fault *fault, + void *data); + void *fault_data; }; static inline bool iommu_is_dma_domain(struct iommu_domain *domain) @@ -686,6 +689,9 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t pasid); void iommu_detach_device_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t pasid); +struct iommu_domain * +iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid); + #else /* CONFIG_IOMMU_API */ struct iommu_ops {}; @@ -1055,6 +1061,12 @@ static inline void iommu_detach_device_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t pasid) { } + +static inline struct iommu_domain * +iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid) +{ + return NULL; +} #endif /* CONFIG_IOMMU_API */ #ifdef CONFIG_IOMMU_SVA diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index 992388106da0..05a7d2f0e46f 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -135,6 +135,69 @@ static void iommu_sva_ioas_put(struct iommu_sva_ioas *ioas) } } +/* + * I/O page fault handler for SVA + * + * Copied from io-pgfault.c with mmget_not_zero() added before + * mmap_read_lock(). + */ +static enum iommu_page_response_code +iommu_sva_handle_iopf(struct iommu_fault *fault, void *data) +{ + vm_fault_t ret; + struct mm_struct *mm; + struct vm_area_struct *vma; + unsigned int access_flags = 0; + struct iommu_domain *domain = data; + unsigned int fault_flags = FAULT_FLAG_REMOTE; + struct iommu_fault_page_request *prm = &fault->prm; + enum iommu_page_response_code status = IOMMU_PAGE_RESP_INVALID; + + if (!(prm->flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID)) + return status; + + mm = iommu_sva_domain_mm(domain); + if (IS_ERR_OR_NULL(mm) || !mmget_not_zero(mm)) + return status; + + mmap_read_lock(mm); + + vma = find_extend_vma(mm, prm->addr); + if (!vma) + /* Unmapped area */ + goto out_put_mm; + + if (prm->perm & IOMMU_FAULT_PERM_READ) + access_flags |= VM_READ; + + if (prm->perm & IOMMU_FAULT_PERM_WRITE) { + access_flags |= VM_WRITE; + fault_flags |= FAULT_FLAG_WRITE; + } + + if (prm->perm & IOMMU_FAULT_PERM_EXEC) { + access_flags |= VM_EXEC; + fault_flags |= FAULT_FLAG_INSTRUCTION; + } + + if (!(prm->perm & IOMMU_FAULT_PERM_PRIV)) + fault_flags |= FAULT_FLAG_USER; + + if (access_flags & ~vma->vm_flags) + /* Access fault */ + goto out_put_mm; + + ret = handle_mm_fault(vma, prm->addr, fault_flags, NULL); + status = ret & VM_FAULT_ERROR ? IOMMU_PAGE_RESP_INVALID : + IOMMU_PAGE_RESP_SUCCESS; + +out_put_mm: + mmap_read_unlock(mm); + mmput(mm); + + return status; +} + /* * IOMMU SVA driver-oriented interfaces */ @@ -154,6 +217,8 @@ iommu_sva_alloc_domain(struct device *dev, struct iommu_sva_ioas *ioas) /* The caller must hold a reference to ioas. */ domain->sva_ioas = ioas; domain->type = IOMMU_DOMAIN_SVA; + domain->iopf_handler = iommu_sva_handle_iopf; + domain->fault_data = domain; return domain; } diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 7cae631c1baa..33449523afbe 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -3174,3 +3174,24 @@ void iommu_detach_device_pasid(struct iommu_domain *domain, iommu_group_put(group); } + +struct iommu_domain *iommu_get_domain_for_dev_pasid(struct device *dev, + ioasid_t pasid) +{ + struct iommu_domain *domain; + struct iommu_group *group; + + if (!pasid_valid(pasid)) + return NULL; + + group = iommu_group_get(dev); + if (!group) + return NULL; + + mutex_lock(&group->mutex); + domain = xa_load(&group->pasid_array, pasid); + mutex_unlock(&group->mutex); + iommu_group_put(group); + + return domain; +} -- 2.25.1