* [PATCH v16 1/7] x86/cet/ibt: Update Kconfig for user-mode Indirect Branch Tracking
2020-12-09 22:27 [PATCH v16 0/7] Control-flow Enforcement: Indirect Branch Tracking Yu-cheng Yu
@ 2020-12-09 22:27 ` Yu-cheng Yu
2020-12-09 22:27 ` [PATCH v16 2/7] x86/cet/ibt: User-mode Indirect Branch Tracking support Yu-cheng Yu
` (5 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Yu-cheng Yu @ 2020-12-09 22:27 UTC (permalink / raw)
To: x86, H. Peter Anvin, Thomas Gleixner, Ingo Molnar, linux-kernel,
linux-doc, linux-mm, linux-arch, linux-api, Arnd Bergmann,
Andy Lutomirski, Balbir Singh, Borislav Petkov, Cyrill Gorcunov,
Dave Hansen, Eugene Syromiatnikov, Florian Weimer, H.J. Lu,
Jann Horn, Jonathan Corbet, Kees Cook, Mike Kravetz, Nadav Amit,
Oleg Nesterov, Pavel Machek, Peter Zijlstra, Randy Dunlap,
Ravi V. Shankar, Vedvyas Shanbhogue, Dave Martin, Weijiang Yang,
Pengfei Xu
Cc: Yu-cheng Yu
Indirect branch tracking is a hardware security feature that verifies near
indirect call/jump instructions arrive at intended targets, which are
labeled by the compiler with ENDBR opcodes. If such instructions reach
unlabeled locations, the processor raises control-protection faults.
Check the compiler is up-to-date at config time.
Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
---
arch/x86/Kconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 264de177a721..d08c0eb563da 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1942,6 +1942,7 @@ config X86_CET_USER
def_bool n
depends on CPU_SUP_INTEL && X86_64
depends on AS_WRUSS
+ depends on $(cc-option,-fcf-protection)
select ARCH_USES_HIGH_VMA_FLAGS
select ARCH_HAS_SHADOW_STACK
select ARCH_MAYBE_MKWRITE
--
2.21.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v16 2/7] x86/cet/ibt: User-mode Indirect Branch Tracking support
2020-12-09 22:27 [PATCH v16 0/7] Control-flow Enforcement: Indirect Branch Tracking Yu-cheng Yu
2020-12-09 22:27 ` [PATCH v16 1/7] x86/cet/ibt: Update Kconfig for user-mode " Yu-cheng Yu
@ 2020-12-09 22:27 ` Yu-cheng Yu
2020-12-09 22:27 ` [PATCH v16 3/7] x86/cet/ibt: Handle signals for Indirect Branch Tracking Yu-cheng Yu
` (4 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Yu-cheng Yu @ 2020-12-09 22:27 UTC (permalink / raw)
To: x86, H. Peter Anvin, Thomas Gleixner, Ingo Molnar, linux-kernel,
linux-doc, linux-mm, linux-arch, linux-api, Arnd Bergmann,
Andy Lutomirski, Balbir Singh, Borislav Petkov, Cyrill Gorcunov,
Dave Hansen, Eugene Syromiatnikov, Florian Weimer, H.J. Lu,
Jann Horn, Jonathan Corbet, Kees Cook, Mike Kravetz, Nadav Amit,
Oleg Nesterov, Pavel Machek, Peter Zijlstra, Randy Dunlap,
Ravi V. Shankar, Vedvyas Shanbhogue, Dave Martin, Weijiang Yang,
Pengfei Xu
Cc: Yu-cheng Yu
Introduce user-mode Indirect Branch Tracking (IBT) support. Add routines
for the setup/disable of IBT.
Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
---
arch/x86/include/asm/cet.h | 3 +++
arch/x86/kernel/cet.c | 33 +++++++++++++++++++++++++++++++++
2 files changed, 36 insertions(+)
diff --git a/arch/x86/include/asm/cet.h b/arch/x86/include/asm/cet.h
index 5e44605ae9c5..7ee1e712463a 100644
--- a/arch/x86/include/asm/cet.h
+++ b/arch/x86/include/asm/cet.h
@@ -15,6 +15,7 @@ struct cet_status {
unsigned long shstk_base;
unsigned long shstk_size;
unsigned int locked:1;
+ unsigned int ibt_enabled:1;
};
#ifdef CONFIG_X86_CET_USER
@@ -26,6 +27,8 @@ void cet_free_shstk(struct task_struct *p);
int cet_verify_rstor_token(bool ia32, unsigned long ssp, unsigned long *new_ssp);
void cet_restore_signal(struct sc_ext *sc);
int cet_setup_signal(bool ia32, unsigned long rstor, struct sc_ext *sc);
+int cet_setup_ibt(void);
+void cet_disable_ibt(void);
#else
static inline int prctl_cet(int option, u64 arg2) { return -EINVAL; }
static inline int cet_setup_thread_shstk(struct task_struct *p,
diff --git a/arch/x86/kernel/cet.c b/arch/x86/kernel/cet.c
index 038419f06fc9..e8871bccce65 100644
--- a/arch/x86/kernel/cet.c
+++ b/arch/x86/kernel/cet.c
@@ -13,6 +13,8 @@
#include <linux/uaccess.h>
#include <linux/sched/signal.h>
#include <linux/compat.h>
+#include <linux/vmalloc.h>
+#include <linux/bitops.h>
#include <asm/msr.h>
#include <asm/user.h>
#include <asm/fpu/internal.h>
@@ -341,3 +343,34 @@ int cet_setup_signal(bool ia32, unsigned long rstor_addr, struct sc_ext *sc_ext)
return 0;
}
+
+int cet_setup_ibt(void)
+{
+ u64 msr_val;
+
+ if (!static_cpu_has(X86_FEATURE_IBT))
+ return -EOPNOTSUPP;
+
+ start_update_msrs();
+ rdmsrl(MSR_IA32_U_CET, msr_val);
+ msr_val |= (CET_ENDBR_EN | CET_NO_TRACK_EN);
+ wrmsrl(MSR_IA32_U_CET, msr_val);
+ end_update_msrs();
+ current->thread.cet.ibt_enabled = 1;
+ return 0;
+}
+
+void cet_disable_ibt(void)
+{
+ u64 msr_val;
+
+ if (!static_cpu_has(X86_FEATURE_IBT))
+ return;
+
+ start_update_msrs();
+ rdmsrl(MSR_IA32_U_CET, msr_val);
+ msr_val &= ~CET_ENDBR_EN;
+ wrmsrl(MSR_IA32_U_CET, msr_val);
+ end_update_msrs();
+ current->thread.cet.ibt_enabled = 0;
+}
--
2.21.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v16 3/7] x86/cet/ibt: Handle signals for Indirect Branch Tracking
2020-12-09 22:27 [PATCH v16 0/7] Control-flow Enforcement: Indirect Branch Tracking Yu-cheng Yu
2020-12-09 22:27 ` [PATCH v16 1/7] x86/cet/ibt: Update Kconfig for user-mode " Yu-cheng Yu
2020-12-09 22:27 ` [PATCH v16 2/7] x86/cet/ibt: User-mode Indirect Branch Tracking support Yu-cheng Yu
@ 2020-12-09 22:27 ` Yu-cheng Yu
2020-12-09 22:27 ` [PATCH v16 4/7] x86/cet/ibt: Update ELF header parsing " Yu-cheng Yu
` (3 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Yu-cheng Yu @ 2020-12-09 22:27 UTC (permalink / raw)
To: x86, H. Peter Anvin, Thomas Gleixner, Ingo Molnar, linux-kernel,
linux-doc, linux-mm, linux-arch, linux-api, Arnd Bergmann,
Andy Lutomirski, Balbir Singh, Borislav Petkov, Cyrill Gorcunov,
Dave Hansen, Eugene Syromiatnikov, Florian Weimer, H.J. Lu,
Jann Horn, Jonathan Corbet, Kees Cook, Mike Kravetz, Nadav Amit,
Oleg Nesterov, Pavel Machek, Peter Zijlstra, Randy Dunlap,
Ravi V. Shankar, Vedvyas Shanbhogue, Dave Martin, Weijiang Yang,
Pengfei Xu
Cc: Yu-cheng Yu
When an indirect CALL/JMP instruction is executed and before it reaches
the target, it is in 'WAIT_ENDBR' status, which can be read from
MSR_IA32_U_CET. The status is part of a task's status before a signal is
raised and preserved in the signal frame. It is restored for sigreturn.
IBT state machine is described in Intel SDM Vol. 1, Sec. 18.3.
Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
---
arch/x86/kernel/cet.c | 27 +++++++++++++++++++++++++--
arch/x86/kernel/fpu/signal.c | 8 +++++---
2 files changed, 30 insertions(+), 5 deletions(-)
diff --git a/arch/x86/kernel/cet.c b/arch/x86/kernel/cet.c
index e8871bccce65..d23f68074ab6 100644
--- a/arch/x86/kernel/cet.c
+++ b/arch/x86/kernel/cet.c
@@ -295,6 +295,13 @@ void cet_restore_signal(struct sc_ext *sc_ext)
msr_val |= CET_SHSTK_EN;
}
+ if (cet->ibt_enabled) {
+ msr_val |= (CET_ENDBR_EN | CET_NO_TRACK_EN);
+
+ if (sc_ext->wait_endbr)
+ msr_val |= CET_WAIT_ENDBR;
+ }
+
if (test_thread_flag(TIF_NEED_FPU_LOAD))
cet_user_state->user_cet = msr_val;
else
@@ -335,9 +342,25 @@ int cet_setup_signal(bool ia32, unsigned long rstor_addr, struct sc_ext *sc_ext)
sc_ext->ssp = new_ssp;
}
- if (ssp) {
+ if (ssp || cet->ibt_enabled) {
+
start_update_msrs();
- wrmsrl(MSR_IA32_PL3_SSP, ssp);
+
+ if (ssp)
+ wrmsrl(MSR_IA32_PL3_SSP, ssp);
+
+ if (cet->ibt_enabled) {
+ u64 r;
+
+ rdmsrl(MSR_IA32_U_CET, r);
+
+ if (r & CET_WAIT_ENDBR) {
+ sc_ext->wait_endbr = 1;
+ r &= ~CET_WAIT_ENDBR;
+ wrmsrl(MSR_IA32_U_CET, r);
+ }
+ }
+
end_update_msrs();
}
diff --git a/arch/x86/kernel/fpu/signal.c b/arch/x86/kernel/fpu/signal.c
index d5d02b34f516..50f28b37e093 100644
--- a/arch/x86/kernel/fpu/signal.c
+++ b/arch/x86/kernel/fpu/signal.c
@@ -57,7 +57,8 @@ int save_cet_to_sigframe(int ia32, void __user *fp, unsigned long restorer)
{
int err = 0;
- if (!current->thread.cet.shstk_size)
+ if (!current->thread.cet.shstk_size &&
+ !current->thread.cet.ibt_enabled)
return 0;
if (fp) {
@@ -89,7 +90,8 @@ static int get_cet_from_sigframe(int ia32, void __user *fp, struct sc_ext *ext)
memset(ext, 0, sizeof(*ext));
- if (!current->thread.cet.shstk_size)
+ if (!current->thread.cet.shstk_size &&
+ !current->thread.cet.ibt_enabled)
return 0;
if (fp) {
@@ -577,7 +579,7 @@ static unsigned long fpu__alloc_sigcontext_ext(unsigned long sp)
* sigcontext_ext is at: fpu + fpu_user_xstate_size +
* FP_XSTATE_MAGIC2_SIZE, then aligned to 8.
*/
- if (cet->shstk_size)
+ if (cet->shstk_size || cet->ibt_enabled)
sp -= (sizeof(struct sc_ext) + 8);
return sp;
--
2.21.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v16 4/7] x86/cet/ibt: Update ELF header parsing for Indirect Branch Tracking
2020-12-09 22:27 [PATCH v16 0/7] Control-flow Enforcement: Indirect Branch Tracking Yu-cheng Yu
` (2 preceding siblings ...)
2020-12-09 22:27 ` [PATCH v16 3/7] x86/cet/ibt: Handle signals for Indirect Branch Tracking Yu-cheng Yu
@ 2020-12-09 22:27 ` Yu-cheng Yu
2020-12-09 22:27 ` [PATCH v16 5/7] x86/cet/ibt: Update arch_prctl functions " Yu-cheng Yu
` (2 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Yu-cheng Yu @ 2020-12-09 22:27 UTC (permalink / raw)
To: x86, H. Peter Anvin, Thomas Gleixner, Ingo Molnar, linux-kernel,
linux-doc, linux-mm, linux-arch, linux-api, Arnd Bergmann,
Andy Lutomirski, Balbir Singh, Borislav Petkov, Cyrill Gorcunov,
Dave Hansen, Eugene Syromiatnikov, Florian Weimer, H.J. Lu,
Jann Horn, Jonathan Corbet, Kees Cook, Mike Kravetz, Nadav Amit,
Oleg Nesterov, Pavel Machek, Peter Zijlstra, Randy Dunlap,
Ravi V. Shankar, Vedvyas Shanbhogue, Dave Martin, Weijiang Yang,
Pengfei Xu
Cc: Yu-cheng Yu
An ELF file's .note.gnu.property indicates features the file supports.
The property is parsed at loading time and passed to arch_setup_elf_
property(). Update it for Indirect Branch Tracking.
Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
---
arch/x86/kernel/process_64.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c
index 2586745b2392..bf3d38394edf 100644
--- a/arch/x86/kernel/process_64.c
+++ b/arch/x86/kernel/process_64.c
@@ -866,6 +866,14 @@ int arch_setup_elf_property(struct arch_elf_state *state)
r = cet_setup_shstk();
}
+ if (r < 0)
+ return r;
+
+ if (static_cpu_has(X86_FEATURE_IBT)) {
+ if (state->gnu_property & GNU_PROPERTY_X86_FEATURE_1_IBT)
+ r = cet_setup_ibt();
+ }
+
return r;
}
#endif
--
2.21.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v16 5/7] x86/cet/ibt: Update arch_prctl functions for Indirect Branch Tracking
2020-12-09 22:27 [PATCH v16 0/7] Control-flow Enforcement: Indirect Branch Tracking Yu-cheng Yu
` (3 preceding siblings ...)
2020-12-09 22:27 ` [PATCH v16 4/7] x86/cet/ibt: Update ELF header parsing " Yu-cheng Yu
@ 2020-12-09 22:27 ` Yu-cheng Yu
2020-12-09 22:27 ` [PATCH v16 6/7] x86/vdso/32: Add ENDBR32 to __kernel_vsyscall entry point Yu-cheng Yu
2020-12-09 22:27 ` [PATCH v16 7/7] x86/vdso: Insert endbr32/endbr64 to vDSO Yu-cheng Yu
6 siblings, 0 replies; 8+ messages in thread
From: Yu-cheng Yu @ 2020-12-09 22:27 UTC (permalink / raw)
To: x86, H. Peter Anvin, Thomas Gleixner, Ingo Molnar, linux-kernel,
linux-doc, linux-mm, linux-arch, linux-api, Arnd Bergmann,
Andy Lutomirski, Balbir Singh, Borislav Petkov, Cyrill Gorcunov,
Dave Hansen, Eugene Syromiatnikov, Florian Weimer, H.J. Lu,
Jann Horn, Jonathan Corbet, Kees Cook, Mike Kravetz, Nadav Amit,
Oleg Nesterov, Pavel Machek, Peter Zijlstra, Randy Dunlap,
Ravi V. Shankar, Vedvyas Shanbhogue, Dave Martin, Weijiang Yang,
Pengfei Xu
Cc: Yu-cheng Yu
From: "H.J. Lu" <hjl.tools@gmail.com>
Update ARCH_X86_CET_STATUS and ARCH_X86_CET_DISABLE for Indirect Branch
Tracking.
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
---
arch/x86/kernel/cet_prctl.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/arch/x86/kernel/cet_prctl.c b/arch/x86/kernel/cet_prctl.c
index 4197d985b5ff..92120d8bdd9f 100644
--- a/arch/x86/kernel/cet_prctl.c
+++ b/arch/x86/kernel/cet_prctl.c
@@ -22,6 +22,9 @@ static int copy_status_to_user(struct cet_status *cet, u64 arg2)
buf[2] = (u64)cet->shstk_size;
}
+ if (cet->ibt_enabled)
+ buf[0] |= GNU_PROPERTY_X86_FEATURE_1_IBT;
+
return copy_to_user((u64 __user *)arg2, buf, sizeof(buf));
}
@@ -56,6 +59,8 @@ int prctl_cet(int option, u64 arg2)
return -EINVAL;
if (features & GNU_PROPERTY_X86_FEATURE_1_SHSTK)
cet_disable_shstk();
+ if (features & GNU_PROPERTY_X86_FEATURE_1_IBT)
+ cet_disable_ibt();
return 0;
case ARCH_X86_CET_LOCK:
--
2.21.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v16 6/7] x86/vdso/32: Add ENDBR32 to __kernel_vsyscall entry point
2020-12-09 22:27 [PATCH v16 0/7] Control-flow Enforcement: Indirect Branch Tracking Yu-cheng Yu
` (4 preceding siblings ...)
2020-12-09 22:27 ` [PATCH v16 5/7] x86/cet/ibt: Update arch_prctl functions " Yu-cheng Yu
@ 2020-12-09 22:27 ` Yu-cheng Yu
2020-12-09 22:27 ` [PATCH v16 7/7] x86/vdso: Insert endbr32/endbr64 to vDSO Yu-cheng Yu
6 siblings, 0 replies; 8+ messages in thread
From: Yu-cheng Yu @ 2020-12-09 22:27 UTC (permalink / raw)
To: x86, H. Peter Anvin, Thomas Gleixner, Ingo Molnar, linux-kernel,
linux-doc, linux-mm, linux-arch, linux-api, Arnd Bergmann,
Andy Lutomirski, Balbir Singh, Borislav Petkov, Cyrill Gorcunov,
Dave Hansen, Eugene Syromiatnikov, Florian Weimer, H.J. Lu,
Jann Horn, Jonathan Corbet, Kees Cook, Mike Kravetz, Nadav Amit,
Oleg Nesterov, Pavel Machek, Peter Zijlstra, Randy Dunlap,
Ravi V. Shankar, Vedvyas Shanbhogue, Dave Martin, Weijiang Yang,
Pengfei Xu
Cc: Yu-cheng Yu
From: "H.J. Lu" <hjl.tools@gmail.com>
Add ENDBR32 to __kernel_vsyscall entry point.
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
Acked-by: Andy Lutomirski <luto@kernel.org>
---
arch/x86/entry/vdso/vdso32/system_call.S | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/x86/entry/vdso/vdso32/system_call.S b/arch/x86/entry/vdso/vdso32/system_call.S
index de1fff7188aa..d28d20d8d4ce 100644
--- a/arch/x86/entry/vdso/vdso32/system_call.S
+++ b/arch/x86/entry/vdso/vdso32/system_call.S
@@ -14,6 +14,9 @@
ALIGN
__kernel_vsyscall:
CFI_STARTPROC
+#ifdef CONFIG_X86_CET_USER
+ endbr32
+#endif
/*
* Reshuffle regs so that all of any of the entry instructions
* will preserve enough state.
--
2.21.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v16 7/7] x86/vdso: Insert endbr32/endbr64 to vDSO
2020-12-09 22:27 [PATCH v16 0/7] Control-flow Enforcement: Indirect Branch Tracking Yu-cheng Yu
` (5 preceding siblings ...)
2020-12-09 22:27 ` [PATCH v16 6/7] x86/vdso/32: Add ENDBR32 to __kernel_vsyscall entry point Yu-cheng Yu
@ 2020-12-09 22:27 ` Yu-cheng Yu
6 siblings, 0 replies; 8+ messages in thread
From: Yu-cheng Yu @ 2020-12-09 22:27 UTC (permalink / raw)
To: x86, H. Peter Anvin, Thomas Gleixner, Ingo Molnar, linux-kernel,
linux-doc, linux-mm, linux-arch, linux-api, Arnd Bergmann,
Andy Lutomirski, Balbir Singh, Borislav Petkov, Cyrill Gorcunov,
Dave Hansen, Eugene Syromiatnikov, Florian Weimer, H.J. Lu,
Jann Horn, Jonathan Corbet, Kees Cook, Mike Kravetz, Nadav Amit,
Oleg Nesterov, Pavel Machek, Peter Zijlstra, Randy Dunlap,
Ravi V. Shankar, Vedvyas Shanbhogue, Dave Martin, Weijiang Yang,
Pengfei Xu
Cc: Yu-cheng Yu
From: "H.J. Lu" <hjl.tools@gmail.com>
When Indirect Branch Tracking (IBT) is enabled, vDSO functions may be
called indirectly, and must have ENDBR32 or ENDBR64 as the first
instruction. The compiler must support -fcf-protection=branch so that it
can be used to compile vDSO.
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
Acked-by: Andy Lutomirski <luto@kernel.org>
---
arch/x86/entry/vdso/Makefile | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile
index 21243747965d..b007f6a30209 100644
--- a/arch/x86/entry/vdso/Makefile
+++ b/arch/x86/entry/vdso/Makefile
@@ -92,6 +92,10 @@ endif
$(vobjs): KBUILD_CFLAGS := $(filter-out $(GCC_PLUGINS_CFLAGS) $(RETPOLINE_CFLAGS),$(KBUILD_CFLAGS)) $(CFL)
+ifdef CONFIG_X86_CET_USER
+$(vobjs) $(vobjs32): KBUILD_CFLAGS += -fcf-protection=branch
+endif
+
#
# vDSO code runs in userspace and -pg doesn't help with profiling anyway.
#
--
2.21.0
^ permalink raw reply related [flat|nested] 8+ messages in thread