From: Fenghua Yu <fenghua.yu@intel.com> To: "Thomas Gleixner" <tglx@linutronix.de>, "Dave Hansen" <dave.hansen@linux.intel.com>, "Ingo Molnar" <mingo@redhat.com>, "Borislav Petkov" <bp@alien8.de>, "Peter Zijlstra" <peterz@infradead.org>, "Andy Lutomirski" <luto@kernel.org>, "Tony Luck" <tony.luck@intel.com>, "Lu Baolu" <baolu.lu@linux.intel.com>, "Joerg Roedel" <joro@8bytes.org>, Josh Poimboeuf <jpoimboe@redhat.com>, "Jacob Pan" <jacob.jun.pan@linux.intel.com>, "Ashok Raj" <ashok.raj@intel.com>, "Ravi V Shankar" <ravi.v.shankar@intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com>, iommu@lists.linux-foundation.org, x86 <x86@kernel.org>, linux-kernel <linux-kernel@vger.kernel.org> Subject: [PATCH v4 11/11] docs: x86: Change documentation for SVA (Shared Virtual Addressing) Date: Mon, 7 Feb 2022 15:02:54 -0800 [thread overview] Message-ID: <20220207230254.3342514-12-fenghua.yu@intel.com> (raw) In-Reply-To: <20220207230254.3342514-1-fenghua.yu@intel.com> Since allocating, freeing and fixing up PASID are changed, the documentation is updated to reflect the changes. Originally-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Fenghua Yu <fenghua.yu@intel.com> Reviewed-by: Tony Luck <tony.luck@intel.com> --- v3: - Remove PASID refcount description (Thomas). v2: - Update life cycle info (Tony and Thomas). - Update initial PASID value to INVALID_IOASID on fork(). Documentation/x86/sva.rst | 53 ++++++++++++++++++++++++++++++--------- 1 file changed, 41 insertions(+), 12 deletions(-) diff --git a/Documentation/x86/sva.rst b/Documentation/x86/sva.rst index 076efd51ef1f..1a22020735a3 100644 --- a/Documentation/x86/sva.rst +++ b/Documentation/x86/sva.rst @@ -104,18 +104,47 @@ The MSR must be configured on each logical CPU before any application thread can interact with a device. Threads that belong to the same process share the same page tables, thus the same MSR value. -PASID is cleared when a process is created. The PASID allocation and MSR -programming may occur long after a process and its threads have been created. -One thread must call iommu_sva_bind_device() to allocate the PASID for the -process. If a thread uses ENQCMD without the MSR first being populated, a #GP -will be raised. The kernel will update the PASID MSR with the PASID for all -threads in the process. A single process PASID can be used simultaneously -with multiple devices since they all share the same address space. - -One thread can call iommu_sva_unbind_device() to free the allocated PASID. -The kernel will clear the PASID MSR for all threads belonging to the process. - -New threads inherit the MSR value from the parent. +PASID Life Cycle Management +========================== + +PASID is initialized as INVALID_IOASID (-1) when a process is created. + +Only processes that access SVA-capable devices need to have a PASID +allocated. This allocation happens when a process opens/binds an SVA-capable +device but finds no PASID for this process. Subsequent binds of the same, or +other devices will share the same PASID. + +Although the PASID is allocated to the process by opening a device, +it is not active in any of the threads of that process. It's loaded to the +IA32_PASID MSR lazily when a thread tries to submit a work descriptor +to a device using the ENQCMD. + +That first access will trigger a #GP fault because the IA32_PASID MSR +has not been initialized with the PASID value assigned to the process +when the device was opened. The Linux #GP handler notes that a PASID has +been allocated for the process, and so initializes the IA32_PASID MSR +and returns so that the ENQCMD instruction is re-executed. + +On fork(2) or exec(2) the PASID is removed from the process as it no +longer has the same address space that it had when the device was opened. + +On clone(2) the new task shares the same address space, so will be +able to use the PASID allocated to the process. The IA32_PASID is not +preemptively initialized as the PASID value might not be allocated yet or +the kernel does not know whether this thread is going to access the device +and the cleared IA32_PASID MSR reduces context switch overhead by xstate +init optimization. Since #GP faults have to be handled on any threads that +were created before the PASID was assigned to the mm of the process, newly +created threads might as well be treated in a consistent way. + +Due to complexity of freeing the PASID and clearing all IA32_PASID MSRs in +all threads in unbind, free the PASID lazily only on mm exit. + +If a process does a close(2) of the device file descriptor and munmap(2) +of the device MMIO portal, then the driver will unbind the device. The +PASID is still marked VALID in the PASID_MSR for any threads in the +process that accessed the device. But this is harmless as without the +MMIO portal they cannot submit new work to the device. Relationships ============= -- 2.35.1 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
WARNING: multiple messages have this Message-ID (diff)
From: Fenghua Yu <fenghua.yu@intel.com> To: "Thomas Gleixner" <tglx@linutronix.de>, "Dave Hansen" <dave.hansen@linux.intel.com>, "Ingo Molnar" <mingo@redhat.com>, "Borislav Petkov" <bp@alien8.de>, "Peter Zijlstra" <peterz@infradead.org>, "Andy Lutomirski" <luto@kernel.org>, "Tony Luck" <tony.luck@intel.com>, "Lu Baolu" <baolu.lu@linux.intel.com>, "Joerg Roedel" <joro@8bytes.org>, Josh Poimboeuf <jpoimboe@redhat.com>, "Jacob Pan" <jacob.jun.pan@linux.intel.com>, "Ashok Raj" <ashok.raj@intel.com>, "Ravi V Shankar" <ravi.v.shankar@intel.com> Cc: iommu@lists.linux-foundation.org, "x86" <x86@kernel.org>, "linux-kernel" <linux-kernel@vger.kernel.org>, Fenghua Yu <fenghua.yu@intel.com> Subject: [PATCH v4 11/11] docs: x86: Change documentation for SVA (Shared Virtual Addressing) Date: Mon, 7 Feb 2022 15:02:54 -0800 [thread overview] Message-ID: <20220207230254.3342514-12-fenghua.yu@intel.com> (raw) In-Reply-To: <20220207230254.3342514-1-fenghua.yu@intel.com> Since allocating, freeing and fixing up PASID are changed, the documentation is updated to reflect the changes. Originally-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Fenghua Yu <fenghua.yu@intel.com> Reviewed-by: Tony Luck <tony.luck@intel.com> --- v3: - Remove PASID refcount description (Thomas). v2: - Update life cycle info (Tony and Thomas). - Update initial PASID value to INVALID_IOASID on fork(). Documentation/x86/sva.rst | 53 ++++++++++++++++++++++++++++++--------- 1 file changed, 41 insertions(+), 12 deletions(-) diff --git a/Documentation/x86/sva.rst b/Documentation/x86/sva.rst index 076efd51ef1f..1a22020735a3 100644 --- a/Documentation/x86/sva.rst +++ b/Documentation/x86/sva.rst @@ -104,18 +104,47 @@ The MSR must be configured on each logical CPU before any application thread can interact with a device. Threads that belong to the same process share the same page tables, thus the same MSR value. -PASID is cleared when a process is created. The PASID allocation and MSR -programming may occur long after a process and its threads have been created. -One thread must call iommu_sva_bind_device() to allocate the PASID for the -process. If a thread uses ENQCMD without the MSR first being populated, a #GP -will be raised. The kernel will update the PASID MSR with the PASID for all -threads in the process. A single process PASID can be used simultaneously -with multiple devices since they all share the same address space. - -One thread can call iommu_sva_unbind_device() to free the allocated PASID. -The kernel will clear the PASID MSR for all threads belonging to the process. - -New threads inherit the MSR value from the parent. +PASID Life Cycle Management +========================== + +PASID is initialized as INVALID_IOASID (-1) when a process is created. + +Only processes that access SVA-capable devices need to have a PASID +allocated. This allocation happens when a process opens/binds an SVA-capable +device but finds no PASID for this process. Subsequent binds of the same, or +other devices will share the same PASID. + +Although the PASID is allocated to the process by opening a device, +it is not active in any of the threads of that process. It's loaded to the +IA32_PASID MSR lazily when a thread tries to submit a work descriptor +to a device using the ENQCMD. + +That first access will trigger a #GP fault because the IA32_PASID MSR +has not been initialized with the PASID value assigned to the process +when the device was opened. The Linux #GP handler notes that a PASID has +been allocated for the process, and so initializes the IA32_PASID MSR +and returns so that the ENQCMD instruction is re-executed. + +On fork(2) or exec(2) the PASID is removed from the process as it no +longer has the same address space that it had when the device was opened. + +On clone(2) the new task shares the same address space, so will be +able to use the PASID allocated to the process. The IA32_PASID is not +preemptively initialized as the PASID value might not be allocated yet or +the kernel does not know whether this thread is going to access the device +and the cleared IA32_PASID MSR reduces context switch overhead by xstate +init optimization. Since #GP faults have to be handled on any threads that +were created before the PASID was assigned to the mm of the process, newly +created threads might as well be treated in a consistent way. + +Due to complexity of freeing the PASID and clearing all IA32_PASID MSRs in +all threads in unbind, free the PASID lazily only on mm exit. + +If a process does a close(2) of the device file descriptor and munmap(2) +of the device MMIO portal, then the driver will unbind the device. The +PASID is still marked VALID in the PASID_MSR for any threads in the +process that accessed the device. But this is harmless as without the +MMIO portal they cannot submit new work to the device. Relationships ============= -- 2.35.1
next prev parent reply other threads:[~2022-02-07 23:03 UTC|newest] Thread overview: 214+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-02-07 23:02 [PATCH v4 00/11] Re-enable ENQCMD and PASID MSR Fenghua Yu 2022-02-07 23:02 ` Fenghua Yu 2022-02-07 23:02 ` [PATCH v4 01/11] iommu/sva: Rename CONFIG_IOMMU_SVA_LIB to CONFIG_IOMMU_SVA Fenghua Yu 2022-02-07 23:02 ` Fenghua Yu 2022-02-08 2:39 ` Lu Baolu 2022-02-08 2:39 ` Lu Baolu 2022-02-15 10:54 ` [tip: x86/pasid] " tip-bot2 for Fenghua Yu 2022-02-07 23:02 ` [PATCH v4 02/11] mm: Change CONFIG option for mm->pasid field Fenghua Yu 2022-02-07 23:02 ` Fenghua Yu 2022-02-08 2:40 ` Lu Baolu 2022-02-08 2:40 ` Lu Baolu 2022-02-15 10:54 ` [tip: x86/pasid] " tip-bot2 for Fenghua Yu 2022-02-07 23:02 ` [PATCH v4 03/11] iommu/ioasid: Introduce a helper to check for valid PASIDs Fenghua Yu 2022-02-07 23:02 ` Fenghua Yu 2022-02-08 2:40 ` Lu Baolu 2022-02-08 2:40 ` Lu Baolu 2022-02-15 10:54 ` [tip: x86/pasid] " tip-bot2 for Fenghua Yu 2022-02-07 23:02 ` [PATCH v4 04/11] kernel/fork: Initialize mm's PASID Fenghua Yu 2022-02-07 23:02 ` Fenghua Yu 2022-02-14 17:23 ` Thomas Gleixner 2022-02-14 17:23 ` Thomas Gleixner 2022-02-15 10:54 ` [tip: x86/pasid] " tip-bot2 for Fenghua Yu 2022-02-07 23:02 ` [PATCH v4 05/11] iommu/sva: Assign a PASID to mm on PASID allocation and free it on mm exit Fenghua Yu 2022-02-07 23:02 ` Fenghua Yu 2022-02-08 2:41 ` Lu Baolu 2022-02-08 2:41 ` Lu Baolu 2022-02-08 15:01 ` Fenghua Yu 2022-02-08 15:01 ` Fenghua Yu 2022-02-10 3:16 ` Jacob Pan 2022-02-10 3:16 ` Jacob Pan 2022-02-10 16:27 ` Fenghua Yu 2022-02-10 16:27 ` Fenghua Yu 2022-02-10 17:24 ` Luck, Tony 2022-02-10 17:24 ` Luck, Tony 2022-02-10 18:31 ` Fenghua Yu 2022-02-10 18:31 ` Fenghua Yu 2022-02-10 23:52 ` Fenghua Yu 2022-02-10 23:52 ` Fenghua Yu 2022-02-10 18:49 ` Jacob Pan 2022-02-10 18:49 ` Jacob Pan 2022-02-10 23:15 ` Fenghua Yu 2022-02-10 23:15 ` Fenghua Yu 2022-02-11 22:00 ` Dave Hansen 2022-02-11 22:00 ` Dave Hansen 2022-02-14 17:24 ` Thomas Gleixner 2022-02-14 17:24 ` Thomas Gleixner 2022-02-15 9:55 ` Joerg Roedel 2022-02-15 9:55 ` Joerg Roedel 2022-04-11 14:00 ` Zhangfei Gao 2022-04-11 14:10 ` Dave Hansen 2022-04-11 14:10 ` Dave Hansen 2022-04-11 14:20 ` zhangfei.gao 2022-04-11 14:20 ` zhangfei.gao 2022-04-11 14:36 ` Dave Hansen 2022-04-11 14:36 ` Dave Hansen 2022-04-11 14:44 ` zhangfei.gao 2022-04-11 14:44 ` zhangfei.gao 2022-04-11 14:52 ` Dave Hansen 2022-04-11 14:52 ` Dave Hansen 2022-04-11 15:13 ` zhangfei.gao 2022-04-11 15:13 ` zhangfei.gao 2022-04-12 7:04 ` zhangfei.gao 2022-04-12 7:04 ` zhangfei.gao 2022-04-12 13:41 ` Fenghua Yu 2022-04-12 13:41 ` Fenghua Yu 2022-04-12 14:39 ` Dave Hansen 2022-04-12 14:39 ` Dave Hansen 2022-04-15 9:59 ` Fenghua Yu 2022-04-15 9:59 ` Fenghua Yu 2022-04-12 15:35 ` zhangfei.gao 2022-04-12 15:35 ` zhangfei.gao 2022-04-14 10:08 ` zhangfei.gao 2022-04-14 10:08 ` zhangfei.gao 2022-04-15 9:51 ` Fenghua Yu 2022-04-15 9:51 ` Fenghua Yu 2022-04-15 10:14 ` zhangfei.gao 2022-04-15 10:14 ` zhangfei.gao 2022-04-15 10:14 ` zhangfei.gao 2022-04-15 10:50 ` Fenghua Yu 2022-04-15 10:50 ` Fenghua Yu 2022-04-15 11:52 ` zhangfei.gao 2022-04-15 11:52 ` zhangfei.gao 2022-04-15 12:37 ` Fenghua Yu 2022-04-15 12:37 ` Fenghua Yu 2022-04-16 1:30 ` zhangfei.gao 2022-04-16 1:30 ` zhangfei.gao 2022-04-15 19:07 ` Fenghua Yu 2022-04-15 19:07 ` Fenghua Yu 2022-04-15 21:00 ` Jacob Pan 2022-04-15 21:00 ` Jacob Pan 2022-04-16 1:43 ` zhangfei.gao 2022-04-16 1:43 ` zhangfei.gao 2022-04-18 18:14 ` Jacob Pan 2022-04-18 18:14 ` Jacob Pan 2022-04-19 1:02 ` zhangfei.gao 2022-04-19 1:02 ` zhangfei.gao 2022-04-18 6:34 ` Tian, Kevin 2022-04-18 6:34 ` Tian, Kevin 2022-04-18 18:11 ` Jacob Pan 2022-04-18 18:11 ` Jacob Pan 2022-04-20 16:45 ` Jean-Philippe Brucker 2022-04-20 16:45 ` Jean-Philippe Brucker 2022-04-21 6:47 ` zhangfei.gao 2022-04-21 6:47 ` zhangfei.gao 2022-04-22 9:03 ` zhangfei.gao 2022-04-22 9:03 ` zhangfei.gao 2022-04-22 10:11 ` Jean-Philippe Brucker 2022-04-22 10:11 ` Jean-Philippe Brucker 2022-04-22 13:15 ` zhangfei.gao 2022-04-22 13:15 ` zhangfei.gao 2022-04-22 15:50 ` Jean-Philippe Brucker 2022-04-22 15:50 ` Jean-Philippe Brucker 2022-04-23 11:13 ` zhangfei.gao 2022-04-23 11:13 ` zhangfei.gao 2022-04-24 2:58 ` Zhangfei Gao 2022-04-24 2:58 ` Zhangfei Gao 2022-04-24 9:52 ` Zhangfei Gao 2022-04-24 9:52 ` Zhangfei Gao 2022-04-25 13:53 ` Jean-Philippe Brucker 2022-04-25 13:53 ` Jean-Philippe Brucker 2022-04-25 14:18 ` Dave Hansen 2022-04-25 14:18 ` Dave Hansen 2022-04-25 14:26 ` Jean-Philippe Brucker 2022-04-25 14:26 ` Jean-Philippe Brucker 2022-04-25 15:34 ` Jacob Pan 2022-04-25 15:34 ` Jacob Pan 2022-04-25 16:13 ` Jean-Philippe Brucker 2022-04-25 16:13 ` Jean-Philippe Brucker 2022-04-25 22:32 ` Jacob Pan 2022-04-25 22:32 ` Jacob Pan 2022-04-26 4:20 ` Fenghua Yu 2022-04-26 4:20 ` Fenghua Yu 2022-04-26 5:04 ` Zhangfei Gao 2022-04-26 5:04 ` Zhangfei Gao 2022-04-28 0:54 ` Fenghua Yu 2022-04-28 0:54 ` Fenghua Yu 2022-04-28 8:43 ` Jean-Philippe Brucker 2022-04-28 8:43 ` Jean-Philippe Brucker 2022-04-28 15:09 ` Dave Hansen 2022-04-28 15:09 ` Dave Hansen 2022-04-28 15:28 ` Fenghua Yu 2022-04-28 15:28 ` Fenghua Yu 2022-04-28 15:42 ` Dave Hansen 2022-04-28 15:42 ` Dave Hansen 2022-04-28 16:01 ` Jean-Philippe Brucker 2022-04-28 16:01 ` Jean-Philippe Brucker 2022-04-28 16:35 ` Dave Hansen 2022-04-28 16:35 ` Dave Hansen 2022-04-26 4:28 ` Zhangfei Gao 2022-04-26 4:28 ` Zhangfei Gao 2022-04-26 4:36 ` Fenghua Yu 2022-04-26 4:36 ` Fenghua Yu 2022-04-26 5:19 ` Zhangfei Gao 2022-04-26 5:19 ` Zhangfei Gao 2022-04-25 15:55 ` Dave Hansen 2022-04-25 15:55 ` Dave Hansen 2022-04-25 16:40 ` Jean-Philippe Brucker 2022-04-25 16:40 ` Jean-Philippe Brucker 2022-04-26 15:27 ` Dave Hansen 2022-04-26 15:27 ` Dave Hansen 2022-04-26 16:48 ` Jean-Philippe Brucker 2022-04-26 16:48 ` Jean-Philippe Brucker 2022-04-26 23:31 ` Dave Hansen 2022-04-26 23:31 ` Dave Hansen 2022-04-28 8:39 ` Jean-Philippe Brucker 2022-04-28 8:39 ` Jean-Philippe Brucker 2022-04-29 7:53 ` Baolu Lu 2022-04-29 7:53 ` Baolu Lu 2022-04-29 13:51 ` Fenghua Yu 2022-04-29 13:51 ` Fenghua Yu 2022-04-29 14:34 ` Jean-Philippe Brucker 2022-04-29 14:34 ` Jean-Philippe Brucker 2022-04-29 22:19 ` Fenghua Yu 2022-04-29 22:19 ` Fenghua Yu 2022-04-30 7:33 ` Baolu Lu 2022-04-30 7:33 ` Baolu Lu 2022-05-03 7:49 ` Jean-Philippe Brucker 2022-05-03 7:49 ` Jean-Philippe Brucker 2022-05-06 5:36 ` Baolu Lu 2022-05-06 5:36 ` Baolu Lu 2022-04-12 14:36 ` Dave Hansen 2022-04-12 14:36 ` Dave Hansen 2022-04-12 15:10 ` Jean-Philippe Brucker 2022-04-12 15:10 ` Jean-Philippe Brucker 2022-04-12 15:35 ` Dave Hansen 2022-04-12 15:35 ` Dave Hansen 2022-04-13 11:14 ` Lu Baolu 2022-04-13 11:14 ` Lu Baolu 2022-04-25 2:57 ` zhangfei.gao 2022-04-25 2:57 ` zhangfei.gao 2022-02-15 10:54 ` [tip: x86/pasid] " tip-bot2 for Fenghua Yu 2022-02-07 23:02 ` [PATCH v4 06/11] x86/fpu: Clear PASID when copying fpstate Fenghua Yu 2022-02-07 23:02 ` Fenghua Yu 2022-02-15 10:54 ` [tip: x86/pasid] " tip-bot2 for Fenghua Yu 2022-02-07 23:02 ` [PATCH v4 07/11] sched: Define and initialize a flag to identify valid PASID in the task Fenghua Yu 2022-02-07 23:02 ` Fenghua Yu 2022-02-15 10:54 ` [tip: x86/pasid] " tip-bot2 for Peter Zijlstra 2022-02-07 23:02 ` [PATCH v4 08/11] x86/traps: Demand-populate PASID MSR via #GP Fenghua Yu 2022-02-07 23:02 ` Fenghua Yu 2022-02-15 10:54 ` [tip: x86/pasid] " tip-bot2 for Fenghua Yu 2022-02-07 23:02 ` [PATCH v4 09/11] x86/cpufeatures: Re-enable ENQCMD Fenghua Yu 2022-02-07 23:02 ` Fenghua Yu 2022-02-15 10:54 ` [tip: x86/pasid] " tip-bot2 for Fenghua Yu 2022-02-07 23:02 ` [PATCH v4 10/11] tools/objtool: Check for use of the ENQCMD instruction in the kernel Fenghua Yu 2022-02-07 23:02 ` Fenghua Yu 2022-02-15 10:54 ` [tip: x86/pasid] " tip-bot2 for Fenghua Yu 2022-03-09 7:55 ` [tip: x86/core] " tip-bot2 for Fenghua Yu 2022-03-15 10:44 ` tip-bot2 for Fenghua Yu 2022-02-07 23:02 ` Fenghua Yu [this message] 2022-02-07 23:02 ` [PATCH v4 11/11] docs: x86: Change documentation for SVA (Shared Virtual Addressing) Fenghua Yu 2022-02-14 17:25 ` Thomas Gleixner 2022-02-14 17:25 ` Thomas Gleixner 2022-02-15 10:54 ` [tip: x86/pasid] Documentation/x86: Update " tip-bot2 for Fenghua Yu 2022-02-11 20:04 ` [PATCH v4 00/11] Re-enable ENQCMD and PASID MSR Fenghua Yu 2022-02-11 20:04 ` Fenghua Yu
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220207230254.3342514-12-fenghua.yu@intel.com \ --to=fenghua.yu@intel.com \ --cc=ashok.raj@intel.com \ --cc=baolu.lu@linux.intel.com \ --cc=bp@alien8.de \ --cc=dave.hansen@linux.intel.com \ --cc=iommu@lists.linux-foundation.org \ --cc=jacob.jun.pan@linux.intel.com \ --cc=joro@8bytes.org \ --cc=jpoimboe@redhat.com \ --cc=linux-kernel@vger.kernel.org \ --cc=luto@kernel.org \ --cc=mingo@redhat.com \ --cc=peterz@infradead.org \ --cc=ravi.v.shankar@intel.com \ --cc=tglx@linutronix.de \ --cc=tony.luck@intel.com \ --cc=x86@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.