From: "Jason A. Donenfeld" <Jason@zx2c4.com>
To: linux-kernel@vger.kernel.org, patches@lists.linux.dev,
tglx@linutronix.de
Cc: "Jason A. Donenfeld" <Jason@zx2c4.com>,
linux-crypto@vger.kernel.org, linux-api@vger.kernel.org,
x86@kernel.org, Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Adhemerval Zanella Netto <adhemerval.zanella@linaro.org>,
Carlos O'Donell <carlos@redhat.com>,
Florian Weimer <fweimer@redhat.com>,
Arnd Bergmann <arnd@arndb.de>, Jann Horn <jannh@google.com>,
Christian Brauner <brauner@kernel.org>,
linux-mm@kvack.org
Subject: [PATCH v14 3/7] x86: mm: Skip faulting instruction for VM_DROPPABLE faults
Date: Sun, 1 Jan 2023 17:29:06 +0100 [thread overview]
Message-ID: <20230101162910.710293-4-Jason@zx2c4.com> (raw)
In-Reply-To: <20230101162910.710293-1-Jason@zx2c4.com>
The prior commit introduced VM_DROPPABLE, but in a limited form where
the faulting instruction was retried instead of skipped. Finish that up
with the platform-specific aspect of skipping the actual instruction.
This works by copying userspace's %rip to a stack buffer of size
MAX_INSN_SIZE, decoding it, and then adding the length of the decoded
instruction to userspace's %rip. In the event any of these fail, just
fallback to not advancing %rip and trying again.
Cc: linux-mm@kvack.org
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
---
arch/x86/mm/fault.c | 19 +++++++++++++++++++
include/linux/mm_types.h | 5 ++++-
mm/memory.c | 4 +++-
3 files changed, 26 insertions(+), 2 deletions(-)
diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index 7b0d4ab894c8..76ca99ab6eb7 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -33,6 +33,8 @@
#include <asm/kvm_para.h> /* kvm_handle_async_pf */
#include <asm/vdso.h> /* fixup_vdso_exception() */
#include <asm/irq_stack.h>
+#include <asm/insn.h> /* struct insn */
+#include <asm/insn-eval.h> /* insn_fetch_from_user(), ... */
#define CREATE_TRACE_POINTS
#include <asm/trace/exceptions.h>
@@ -1454,6 +1456,23 @@ void do_user_addr_fault(struct pt_regs *regs,
}
mmap_read_unlock(mm);
+
+ if (fault & VM_FAULT_SKIP_INSN) {
+ u8 buf[MAX_INSN_SIZE];
+ struct insn insn;
+ int nr_copied;
+
+ nr_copied = insn_fetch_from_user(regs, buf);
+ if (nr_copied <= 0)
+ return;
+
+ if (!insn_decode_from_regs(&insn, regs, buf, nr_copied))
+ return;
+
+ regs->ip += insn.length;
+ return;
+ }
+
if (likely(!(fault & VM_FAULT_ERROR)))
return;
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 3b8475007734..e76ab9ad555c 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -945,6 +945,7 @@ typedef __bitwise unsigned int vm_fault_t;
* fsync() to complete (for synchronous page faults
* in DAX)
* @VM_FAULT_COMPLETED: ->fault completed, meanwhile mmap lock released
+ * @VM_FAULT_SKIP_INSN: ->handle the fault by skipping faulting instruction
* @VM_FAULT_HINDEX_MASK: mask HINDEX value
*
*/
@@ -962,6 +963,7 @@ enum vm_fault_reason {
VM_FAULT_DONE_COW = (__force vm_fault_t)0x001000,
VM_FAULT_NEEDDSYNC = (__force vm_fault_t)0x002000,
VM_FAULT_COMPLETED = (__force vm_fault_t)0x004000,
+ VM_FAULT_SKIP_INSN = (__force vm_fault_t)0x008000,
VM_FAULT_HINDEX_MASK = (__force vm_fault_t)0x0f0000,
};
@@ -985,7 +987,8 @@ enum vm_fault_reason {
{ VM_FAULT_RETRY, "RETRY" }, \
{ VM_FAULT_FALLBACK, "FALLBACK" }, \
{ VM_FAULT_DONE_COW, "DONE_COW" }, \
- { VM_FAULT_NEEDDSYNC, "NEEDDSYNC" }
+ { VM_FAULT_NEEDDSYNC, "NEEDDSYNC" }, \
+ { VM_FAULT_SKIP_INSN, "SKIP_INSN" }
struct vm_special_mapping {
const char *name; /* The name, e.g. "[vdso]". */
diff --git a/mm/memory.c b/mm/memory.c
index 1ade407ccbf9..62ba9b7b713e 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -5221,8 +5221,10 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address,
lru_gen_exit_fault();
/* If the mapping is droppable, then errors due to OOM aren't fatal. */
- if (vma->vm_flags & VM_DROPPABLE)
+ if ((ret & VM_FAULT_OOM) && (vma->vm_flags & VM_DROPPABLE)) {
ret &= ~VM_FAULT_OOM;
+ ret |= VM_FAULT_SKIP_INSN;
+ }
if (flags & FAULT_FLAG_USER) {
mem_cgroup_exit_user_fault();
--
2.39.0
next prev parent reply other threads:[~2023-01-01 16:29 UTC|newest]
Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-01 16:29 [PATCH v14 0/7] implement getrandom() in vDSO Jason A. Donenfeld
2023-01-01 16:29 ` [PATCH v14 1/7] x86: lib: Separate instruction decoder MMIO type from MMIO trace Jason A. Donenfeld
2023-01-03 10:32 ` Ingo Molnar
2023-01-03 14:51 ` Jason A. Donenfeld
2023-01-03 17:00 ` Ingo Molnar
2023-01-03 17:29 ` Borislav Petkov
2023-01-03 17:30 ` Jason A. Donenfeld
2023-01-03 17:47 ` Ingo Molnar
2023-01-03 17:48 ` Jason A. Donenfeld
2023-01-04 20:25 ` Ingo Molnar
2023-01-04 20:29 ` Jason A. Donenfeld
2023-01-03 11:00 ` [tip: x86/asm] x86/insn: Avoid namespace clash by separating instruction decoder MMIO type from MMIO trace type tip-bot2 for Jason A. Donenfeld
2023-01-03 17:53 ` [tip: x86/urgent] " tip-bot2 for Jason A. Donenfeld
2023-01-01 16:29 ` [PATCH v14 2/7] mm: add VM_DROPPABLE for designating always lazily freeable mappings Jason A. Donenfeld
2023-01-03 10:50 ` Ingo Molnar
2023-01-03 15:01 ` Jason A. Donenfeld
2023-01-03 18:15 ` Ingo Molnar
2023-01-03 18:51 ` Jason A. Donenfeld
2023-01-03 18:36 ` Andy Lutomirski
2023-01-03 19:05 ` Jason A. Donenfeld
2023-01-03 20:52 ` Andy Lutomirski
2023-01-03 19:19 ` Linus Torvalds
2023-01-03 19:35 ` Jason A. Donenfeld
2023-01-03 19:54 ` Linus Torvalds
2023-01-03 20:03 ` Jason A. Donenfeld
2023-01-03 20:15 ` Linus Torvalds
2023-01-03 20:25 ` Linus Torvalds
2023-01-03 20:44 ` Jason A. Donenfeld
2023-01-05 21:57 ` Yann Droneaud
2023-01-05 22:57 ` Jason A. Donenfeld
2023-01-06 1:02 ` Linus Torvalds
2023-01-06 2:08 ` Linus Torvalds
2023-01-06 2:42 ` Jason A. Donenfeld
2023-01-06 20:53 ` Andy Lutomirski
2023-01-06 21:10 ` Linus Torvalds
2023-01-10 11:01 ` Dr. Greg
2023-01-06 21:36 ` Jason A. Donenfeld
2023-01-06 21:42 ` Matthew Wilcox
2023-01-06 22:06 ` Linus Torvalds
2023-01-06 2:14 ` Jason A. Donenfeld
2023-01-09 10:34 ` Florian Weimer
2023-01-09 14:28 ` Linus Torvalds
2023-01-11 7:27 ` Eric Biggers
2023-01-11 12:07 ` Linus Torvalds
2023-01-01 16:29 ` Jason A. Donenfeld [this message]
2023-01-01 16:29 ` [PATCH v14 4/7] random: add vgetrandom_alloc() syscall Jason A. Donenfeld
2023-01-01 16:29 ` [PATCH v14 5/7] arch: allocate vgetrandom_alloc() syscall number Jason A. Donenfeld
2023-01-01 16:29 ` [PATCH v14 6/7] random: introduce generic vDSO getrandom() implementation Jason A. Donenfeld
2023-01-01 16:29 ` [PATCH v14 7/7] x86: vdso: Wire up getrandom() vDSO implementation Jason A. Donenfeld
2023-01-12 17:27 ` Christophe Leroy
2023-01-12 17:49 ` Jason A. Donenfeld
2023-01-11 22:23 ` [PATCH v14 0/7] implement getrandom() in vDSO Mathieu Desnoyers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230101162910.710293-4-Jason@zx2c4.com \
--to=jason@zx2c4.com \
--cc=adhemerval.zanella@linaro.org \
--cc=arnd@arndb.de \
--cc=brauner@kernel.org \
--cc=carlos@redhat.com \
--cc=fweimer@redhat.com \
--cc=gregkh@linuxfoundation.org \
--cc=jannh@google.com \
--cc=linux-api@vger.kernel.org \
--cc=linux-crypto@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=patches@lists.linux.dev \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).