From: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
To: linux-arm-kernel@lists.infradead.org, linux-pci@vger.kernel.org,
linux-acpi@vger.kernel.org, devicetree@vger.kernel.org,
iommu@lists.linux-foundation.org, kvm@vger.kernel.org,
linux-mm@kvack.org
Cc: joro@8bytes.org, will.deacon@arm.com, robin.murphy@arm.com,
alex.williamson@redhat.com, tn@semihalf.com, liubo95@huawei.com,
thunder.leizhen@huawei.com, xieyisheng1@huawei.com,
xuzaibo@huawei.com, ilias.apalodimas@linaro.org,
jonathan.cameron@huawei.com, liudongdong3@huawei.com,
shunyong.yang@hxt-semitech.com, nwatters@codeaurora.org,
okaya@codeaurora.org, jcrouse@codeaurora.org, rfranz@cavium.com,
dwmw2@infradead.org, jacob.jun.pan@linux.intel.com,
yi.l.liu@intel.com, ashok.raj@intel.com, kevin.tian@intel.com,
baolu.lu@linux.intel.com, robdclark@gmail.com,
christian.koenig@amd.com, bharatku@xilinx.com,
rgummal@xilinx.com
Subject: [PATCH v2 08/40] iommu/iopf: Handle mm faults
Date: Fri, 11 May 2018 20:06:09 +0100 [thread overview]
Message-ID: <20180511190641.23008-9-jean-philippe.brucker@arm.com> (raw)
In-Reply-To: <20180511190641.23008-1-jean-philippe.brucker@arm.com>
When a recoverable page fault is handled by the fault workqueue, find the
associated mm and call handle_mm_fault.
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
---
v1->v2: let IOMMU drivers deal with Stop PASID
---
drivers/iommu/io-pgfault.c | 86 +++++++++++++++++++++++++++++++++++++-
1 file changed, 84 insertions(+), 2 deletions(-)
diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c
index 321c00dd3a3d..dd2639e5c03b 100644
--- a/drivers/iommu/io-pgfault.c
+++ b/drivers/iommu/io-pgfault.c
@@ -7,6 +7,7 @@
#include <linux/iommu.h>
#include <linux/list.h>
+#include <linux/sched/mm.h>
#include <linux/slab.h>
#include <linux/workqueue.h>
@@ -65,8 +66,65 @@ static int iopf_complete(struct device *dev, struct iommu_fault_event *evt,
static enum page_response_code
iopf_handle_single(struct iopf_context *fault)
{
- /* TODO */
- return -ENODEV;
+ int ret;
+ struct mm_struct *mm;
+ struct vm_area_struct *vma;
+ unsigned int access_flags = 0;
+ unsigned int fault_flags = FAULT_FLAG_REMOTE;
+ struct iommu_fault_event *evt = &fault->evt;
+ enum page_response_code status = IOMMU_PAGE_RESP_INVALID;
+
+ if (!evt->pasid_valid)
+ return status;
+
+ mm = iommu_sva_find(evt->pasid);
+ if (!mm)
+ return status;
+
+ down_read(&mm->mmap_sem);
+
+ vma = find_extend_vma(mm, evt->addr);
+ if (!vma)
+ /* Unmapped area */
+ goto out_put_mm;
+
+ if (evt->prot & IOMMU_FAULT_READ)
+ access_flags |= VM_READ;
+
+ if (evt->prot & IOMMU_FAULT_WRITE) {
+ access_flags |= VM_WRITE;
+ fault_flags |= FAULT_FLAG_WRITE;
+ }
+
+ if (evt->prot & IOMMU_FAULT_EXEC) {
+ access_flags |= VM_EXEC;
+ fault_flags |= FAULT_FLAG_INSTRUCTION;
+ }
+
+ if (!(evt->prot & IOMMU_FAULT_PRIV))
+ fault_flags |= FAULT_FLAG_USER;
+
+ if (access_flags & ~vma->vm_flags)
+ /* Access fault */
+ goto out_put_mm;
+
+ ret = handle_mm_fault(vma, evt->addr, fault_flags);
+ status = ret & VM_FAULT_ERROR ? IOMMU_PAGE_RESP_INVALID :
+ IOMMU_PAGE_RESP_SUCCESS;
+
+out_put_mm:
+ up_read(&mm->mmap_sem);
+
+ /*
+ * If the process exits while we're handling the fault on its mm, we
+ * can't do mmput(). exit_mmap() would release the MMU notifier, calling
+ * iommu_notifier_release(), which has to flush the fault queue that
+ * we're executing on... So mmput_async() moves the release of the mm to
+ * another thread, if we're the last user.
+ */
+ mmput_async(mm);
+
+ return status;
}
static void iopf_handle_group(struct work_struct *work)
@@ -100,6 +158,30 @@ static void iopf_handle_group(struct work_struct *work)
* @cookie: struct device, passed to iommu_register_device_fault_handler.
*
* Add a fault to the device workqueue, to be handled by mm.
+ *
+ * This module doesn't handle PCI PASID Stop Marker; IOMMU drivers must discard
+ * them before reporting faults. A PASID Stop Marker (LRW = 0b100) doesn't
+ * expect a response. It may be generated when disabling a PASID (issuing a
+ * PASID stop request) by some PCI devices.
+ *
+ * The PASID stop request is triggered by the mm_exit() callback. When the
+ * callback returns from the device driver, no page request is generated for
+ * this PASID anymore and outstanding ones have been pushed to the IOMMU (as per
+ * PCIe 4.0r1.0 - 6.20.1 and 10.4.1.2 - Managing PASID TLP Prefix Usage). Some
+ * PCI devices will wait for all outstanding page requests to come back with a
+ * response before completing the PASID stop request. Others do not wait for
+ * page responses, and instead issue this Stop Marker that tells us when the
+ * PASID can be reallocated.
+ *
+ * It is safe to discard the Stop Marker because it is an optimization.
+ * a. Page requests, which are posted requests, have been flushed to the IOMMU
+ * when mm_exit() returns,
+ * b. We flush all fault queues after mm_exit() returns and before freeing the
+ * PASID.
+ *
+ * So even though the Stop Marker might be issued by the device *after* the stop
+ * request completes, outstanding faults will have been dealt with by the time
+ * we free the PASID.
*/
int iommu_queue_iopf(struct iommu_fault_event *evt, void *cookie)
{
--
2.17.0
next prev parent reply other threads:[~2018-05-11 19:08 UTC|newest]
Thread overview: 125+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-05-11 19:06 [PATCH v2 00/40] Shared Virtual Addressing for the IOMMU Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 01/40] iommu: Introduce Shared Virtual Addressing API Jean-Philippe Brucker
2018-05-16 20:41 ` Jacob Pan
2018-05-17 10:02 ` Jean-Philippe Brucker
2018-05-17 17:00 ` Jacob Pan
2018-09-05 11:29 ` Auger Eric
2018-09-06 11:09 ` Jean-Philippe Brucker
2018-09-06 11:12 ` Christian König
2018-09-06 12:45 ` Jean-Philippe Brucker
2018-09-07 8:55 ` Christian König
2018-09-07 15:45 ` Jean-Philippe Brucker
2018-09-07 18:02 ` Christian König
2018-09-07 21:25 ` Jacob Pan
2018-09-08 7:29 ` Christian König
2018-09-12 12:40 ` Jean-Philippe Brucker
2018-09-12 12:56 ` Christian König
2018-09-13 7:15 ` Tian, Kevin
2018-09-13 7:26 ` Tian, Kevin
2018-05-11 19:06 ` [PATCH v2 02/40] iommu/sva: Bind process address spaces to devices Jean-Philippe Brucker
2018-05-17 13:10 ` Jonathan Cameron
2018-05-21 14:43 ` Jean-Philippe Brucker
2018-09-05 11:29 ` Auger Eric
2018-09-06 11:09 ` Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 03/40] iommu/sva: Manage process address spaces Jean-Philippe Brucker
2018-05-16 23:31 ` Jacob Pan
2018-05-17 10:02 ` Jean-Philippe Brucker
2018-05-22 16:43 ` Jacob Pan
2018-05-24 11:44 ` Jean-Philippe Brucker
2018-05-24 11:50 ` Ilias Apalodimas
2018-05-24 15:04 ` Jean-Philippe Brucker
2018-05-25 6:33 ` Ilias Apalodimas
2018-05-25 8:39 ` Jonathan Cameron
2018-05-26 2:24 ` Kenneth Lee
[not found] ` <20180525093959.000040a7-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2018-05-26 2:24 ` Kenneth Lee
2018-05-26 2:24 ` Kenneth Lee
[not found] ` <20180526022445.GA6069@kllp05>
2018-06-11 16:10 ` Kenneth Lee
2018-06-11 16:10 ` Kenneth Lee
2018-06-11 16:10 ` Kenneth Lee
2018-06-11 16:32 ` Kenneth Lee
2018-05-17 14:25 ` Jonathan Cameron
2018-05-21 14:44 ` Jean-Philippe Brucker
2018-09-05 12:14 ` Auger Eric
2018-09-05 18:18 ` Jacob Pan
2018-09-06 17:40 ` Jean-Philippe Brucker
2018-09-06 11:10 ` Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 04/40] iommu/sva: Add a mm_exit callback for device drivers Jean-Philippe Brucker
2018-09-05 13:23 ` Auger Eric
2018-09-06 11:10 ` Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 05/40] iommu/sva: Track mm changes with an MMU notifier Jean-Philippe Brucker
2018-05-17 14:25 ` Jonathan Cameron
2018-05-21 14:44 ` Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 06/40] iommu/sva: Search mm by PASID Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 07/40] iommu: Add a page fault handler Jean-Philippe Brucker
2018-05-17 15:25 ` Jonathan Cameron
2018-05-21 14:48 ` Jean-Philippe Brucker
2018-05-18 18:04 ` Jacob Pan
2018-05-21 14:49 ` Jean-Philippe Brucker
2018-05-22 23:35 ` Jacob Pan
2018-05-24 11:44 ` Jean-Philippe Brucker
2018-05-26 0:35 ` Jacob Pan
2018-05-29 10:00 ` Jean-Philippe Brucker
2018-05-11 19:06 ` Jean-Philippe Brucker [this message]
2018-05-11 19:06 ` [PATCH v2 09/40] iommu/sva: Register " Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 10/40] mm: export symbol mm_access Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 11/40] mm: export symbol find_get_task_by_vpid Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 12/40] mm: export symbol mmput_async Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 13/40] vfio: Add support for Shared Virtual Addressing Jean-Philippe Brucker
2018-05-17 15:58 ` Jonathan Cameron
2018-05-21 14:51 ` Jean-Philippe Brucker
2018-05-23 9:38 ` Xu Zaibo
2018-05-24 11:44 ` Jean-Philippe Brucker
2018-05-24 12:35 ` Xu Zaibo
2018-05-24 15:04 ` Jean-Philippe Brucker
2018-05-25 2:39 ` Xu Zaibo
2018-05-25 9:47 ` Jean-Philippe Brucker
2018-05-26 3:53 ` Xu Zaibo
2018-05-29 11:55 ` Jean-Philippe Brucker
2018-05-29 12:24 ` Xu Zaibo
2018-08-27 8:06 ` Xu Zaibo
2018-08-31 13:34 ` Jean-Philippe Brucker
2018-09-01 2:23 ` Xu Zaibo
2018-09-03 10:34 ` Jean-Philippe Brucker
2018-09-04 2:12 ` Xu Zaibo
2018-09-04 10:57 ` Jean-Philippe Brucker
2018-09-05 3:15 ` Xu Zaibo
2018-09-05 11:02 ` Jean-Philippe Brucker
2018-09-06 7:26 ` Xu Zaibo
2018-05-11 19:06 ` [PATCH v2 14/40] dt-bindings: document stall and PASID properties for IOMMU masters Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 15/40] iommu/of: Add stall and pasid properties to iommu_fwspec Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 16/40] arm64: mm: Pin down ASIDs for sharing mm with devices Jean-Philippe Brucker
2018-05-15 14:16 ` Catalin Marinas
2018-05-17 10:01 ` Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 17/40] iommu/arm-smmu-v3: Link domains and devices Jean-Philippe Brucker
2018-05-17 16:07 ` Jonathan Cameron
2018-05-21 14:49 ` Jean-Philippe Brucker
2018-09-10 15:16 ` Auger Eric
2018-05-11 19:06 ` [PATCH v2 18/40] iommu/io-pgtable-arm: Factor out ARM LPAE register defines Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 19/40] iommu: Add generic PASID table library Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 20/40] iommu/arm-smmu-v3: Move context descriptor code Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 21/40] iommu/arm-smmu-v3: Add support for Substream IDs Jean-Philippe Brucker
2018-05-31 11:01 ` Bharat Kumar Gogada
2018-06-01 10:46 ` Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 22/40] iommu/arm-smmu-v3: Add second level of context descriptor table Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 23/40] iommu/arm-smmu-v3: Share process page tables Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 24/40] iommu/arm-smmu-v3: Seize private ASID Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 25/40] iommu/arm-smmu-v3: Add support for VHE Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 26/40] iommu/arm-smmu-v3: Enable broadcast TLB maintenance Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 27/40] iommu/arm-smmu-v3: Add SVA feature checking Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 28/40] iommu/arm-smmu-v3: Implement mm operations Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 29/40] iommu/arm-smmu-v3: Add support for Hardware Translation Table Update Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 30/40] iommu/arm-smmu-v3: Register I/O Page Fault queue Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 31/40] iommu/arm-smmu-v3: Improve add_device error handling Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 32/40] iommu/arm-smmu-v3: Maintain a SID->device structure Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 33/40] iommu/arm-smmu-v3: Add stall support for platform devices Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 34/40] ACPI/IORT: Check ATS capability in root complex nodes Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 35/40] iommu/arm-smmu-v3: Add support for PCI ATS Jean-Philippe Brucker
2018-05-19 17:25 ` Sinan Kaya
2018-05-21 14:52 ` Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 36/40] iommu/arm-smmu-v3: Hook up ATC invalidation to mm ops Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 37/40] iommu/arm-smmu-v3: Disable tagged pointers Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 38/40] PCI: Make "PRG Response PASID Required" handling common Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 39/40] iommu/arm-smmu-v3: Add support for PRI Jean-Philippe Brucker
2018-05-25 14:08 ` Bharat Kumar Gogada
2018-05-29 10:27 ` Jean-Philippe Brucker
2018-05-11 19:06 ` [PATCH v2 40/40] iommu/arm-smmu-v3: Add support for PCI PASID Jean-Philippe Brucker
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180511190641.23008-9-jean-philippe.brucker@arm.com \
--to=jean-philippe.brucker@arm.com \
--cc=alex.williamson@redhat.com \
--cc=ashok.raj@intel.com \
--cc=baolu.lu@linux.intel.com \
--cc=bharatku@xilinx.com \
--cc=christian.koenig@amd.com \
--cc=devicetree@vger.kernel.org \
--cc=dwmw2@infradead.org \
--cc=ilias.apalodimas@linaro.org \
--cc=iommu@lists.linux-foundation.org \
--cc=jacob.jun.pan@linux.intel.com \
--cc=jcrouse@codeaurora.org \
--cc=jonathan.cameron@huawei.com \
--cc=joro@8bytes.org \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-acpi@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-mm@kvack.org \
--cc=linux-pci@vger.kernel.org \
--cc=liubo95@huawei.com \
--cc=liudongdong3@huawei.com \
--cc=nwatters@codeaurora.org \
--cc=okaya@codeaurora.org \
--cc=rfranz@cavium.com \
--cc=rgummal@xilinx.com \
--cc=robdclark@gmail.com \
--cc=robin.murphy@arm.com \
--cc=shunyong.yang@hxt-semitech.com \
--cc=thunder.leizhen@huawei.com \
--cc=tn@semihalf.com \
--cc=will.deacon@arm.com \
--cc=xieyisheng1@huawei.com \
--cc=xuzaibo@huawei.com \
--cc=yi.l.liu@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).