From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933792AbeBLKSF (ORCPT ); Mon, 12 Feb 2018 05:18:05 -0500 Received: from [198.137.202.136] ([198.137.202.136]:58785 "EHLO terminus.zytor.com" rhost-flags-FAIL-FAIL-OK-OK) by vger.kernel.org with ESMTP id S933355AbeBLKSD (ORCPT ); Mon, 12 Feb 2018 05:18:03 -0500 Date: Mon, 12 Feb 2018 02:17:27 -0800 From: tip-bot for Dominik Brodowski Message-ID: Cc: hpa@zytor.com, torvalds@linux-foundation.org, mingo@kernel.org, luto@kernel.org, linux@dominikbrodowski.net, bp@alien8.de, linux-kernel@vger.kernel.org, peterz@infradead.org, brgerst@gmail.com, jpoimboe@redhat.com, dvlasenk@redhat.com, tglx@linutronix.de Reply-To: hpa@zytor.com, torvalds@linux-foundation.org, luto@kernel.org, mingo@kernel.org, bp@alien8.de, linux-kernel@vger.kernel.org, linux@dominikbrodowski.net, brgerst@gmail.com, peterz@infradead.org, tglx@linutronix.de, jpoimboe@redhat.com, dvlasenk@redhat.com In-Reply-To: <20180211104949.12992-5-linux@dominikbrodowski.net> References: <20180211104949.12992-5-linux@dominikbrodowski.net> To: linux-tip-commits@vger.kernel.org Subject: [tip:x86/pti] x86/entry/64: Introduce the PUSH_AND_CLEAN_REGS macro Git-Commit-ID: 7b7b09f110f06f3c006e9b3f4590f7d9ba91345b X-Mailer: tip-git-log-daemon Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Commit-ID: 7b7b09f110f06f3c006e9b3f4590f7d9ba91345b Gitweb: https://git.kernel.org/tip/7b7b09f110f06f3c006e9b3f4590f7d9ba91345b Author: Dominik Brodowski AuthorDate: Sun, 11 Feb 2018 11:49:45 +0100 Committer: Ingo Molnar CommitDate: Mon, 12 Feb 2018 08:06:36 +0100 x86/entry/64: Introduce the PUSH_AND_CLEAN_REGS macro Those instances where ALLOC_PT_GPREGS_ON_STACK is called just before SAVE_AND_CLEAR_REGS can trivially be replaced by PUSH_AND_CLEAN_REGS. This macro uses PUSH instead of MOV and should therefore be faster, at least on newer CPUs. Suggested-by: Linus Torvalds Signed-off-by: Dominik Brodowski Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Brian Gerst Cc: Denys Vlasenko Cc: H. Peter Anvin Cc: Josh Poimboeuf Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: dan.j.williams@intel.com Link: http://lkml.kernel.org/r/20180211104949.12992-5-linux@dominikbrodowski.net Signed-off-by: Ingo Molnar --- arch/x86/entry/calling.h | 36 ++++++++++++++++++++++++++++++++++++ arch/x86/entry/entry_64.S | 6 ++---- 2 files changed, 38 insertions(+), 4 deletions(-) diff --git a/arch/x86/entry/calling.h b/arch/x86/entry/calling.h index a05cbb8..57b1b87 100644 --- a/arch/x86/entry/calling.h +++ b/arch/x86/entry/calling.h @@ -137,6 +137,42 @@ For 32-bit we have the following conventions - kernel is built with UNWIND_HINT_REGS offset=\offset .endm + .macro PUSH_AND_CLEAR_REGS + /* + * Push registers and sanitize registers of values that a + * speculation attack might otherwise want to exploit. The + * lower registers are likely clobbered well before they + * could be put to use in a speculative execution gadget. + * Interleave XOR with PUSH for better uop scheduling: + */ + pushq %rdi /* pt_regs->di */ + pushq %rsi /* pt_regs->si */ + pushq %rdx /* pt_regs->dx */ + pushq %rcx /* pt_regs->cx */ + pushq %rax /* pt_regs->ax */ + pushq %r8 /* pt_regs->r8 */ + xorq %r8, %r8 /* nospec r8 */ + pushq %r9 /* pt_regs->r9 */ + xorq %r9, %r9 /* nospec r9 */ + pushq %r10 /* pt_regs->r10 */ + xorq %r10, %r10 /* nospec r10 */ + pushq %r11 /* pt_regs->r11 */ + xorq %r11, %r11 /* nospec r11*/ + pushq %rbx /* pt_regs->rbx */ + xorl %ebx, %ebx /* nospec rbx*/ + pushq %rbp /* pt_regs->rbp */ + xorl %ebp, %ebp /* nospec rbp*/ + pushq %r12 /* pt_regs->r12 */ + xorq %r12, %r12 /* nospec r12*/ + pushq %r13 /* pt_regs->r13 */ + xorq %r13, %r13 /* nospec r13*/ + pushq %r14 /* pt_regs->r14 */ + xorq %r14, %r14 /* nospec r14*/ + pushq %r15 /* pt_regs->r15 */ + xorq %r15, %r15 /* nospec r15*/ + UNWIND_HINT_REGS + .endm + .macro POP_REGS pop_rdi=1 skip_r11rcx=0 popq %r15 popq %r14 diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index 07692b4..cf4a9ae 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -564,8 +564,7 @@ END(irq_entries_start) call switch_to_thread_stack 1: - ALLOC_PT_GPREGS_ON_STACK - SAVE_AND_CLEAR_REGS + PUSH_AND_CLEAR_REGS ENCODE_FRAME_POINTER testb $3, CS(%rsp) @@ -1112,8 +1111,7 @@ ENTRY(xen_failsafe_callback) addq $0x30, %rsp UNWIND_HINT_IRET_REGS pushq $-1 /* orig_ax = -1 => not a system call */ - ALLOC_PT_GPREGS_ON_STACK - SAVE_AND_CLEAR_REGS + PUSH_AND_CLEAR_REGS ENCODE_FRAME_POINTER jmp error_exit END(xen_failsafe_callback)