From mboxrd@z Thu Jan 1 00:00:00 1970 From: Thomas Garnier via Virtualization Subject: [PATCH v2 00/27] x86: PIE support and option to extend KASLR randomization Date: Tue, 13 Mar 2018 13:59:18 -0700 Message-ID: <20180313205945.245105-1-thgarnie@google.com> Reply-To: Thomas Garnier Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: linux-arch@vger.kernel.org, kvm@vger.kernel.org, linux-pm@vger.kernel.org, x86@kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-sparse@vger.kernel.org, linux-crypto@vger.kernel.org, kernel-hardening@lists.openwall.com, xen-devel@lists.xenproject.org To: Herbert Xu , "David S . Miller" , Thomas Gleixner , Ingo Molnar , "H . Peter Anvin" , Peter Zijlstra , Josh Poimboeuf , Greg Kroah-Hartman , Kate Stewart , Thomas Garnier , Arnd Bergmann , Philippe Ombredanne , Arnaldo Carvalho de Melo , Andrey Ryabinin , Matthias Kaehlcke , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Andy Lutomirski , Dominik Brodowski , Borislav Petkov , Borislav Petkov , "Raf Return-path: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org List-Id: linux-crypto.vger.kernel.org Changes: - patch v2: - Adapt patch to work post KPTI and compiler changes - Redo all performance testing with latest configs and compilers - Simplify mov macro on PIE (MOVABS now) - Reduce GOT footprint - patch v1: - Simplify ftrace implementation. - Use gcc mstack-protector-guard-reg=%gs with PIE when possible. - rfc v3: - Use --emit-relocs instead of -pie to reduce dynamic relocation space on mapped memory. It also simplifies the relocation process. - Move the start the module section next to the kernel. Remove the need for -mcmodel=large on modules. Extends module space from 1 to 2G maximum. - Support for XEN PVH as 32-bit relocations can be ignored with --emit-relocs. - Support for GOT relocations previously done automatically with -pie. - Remove need for dynamic PLT in modules. - Support dymamic GOT for modules. - rfc v2: - Add support for global stack cookie while compiler default to fs without mcmodel=kernel - Change patch 7 to correctly jump out of the identity mapping on kexec load preserve. These patches make the changes necessary to build the kernel as Position Independent Executable (PIE) on x86_64. A PIE kernel can be relocated below the top 2G of the virtual address space. It allows to optionally extend the KASLR randomization range from 1G to 3G. Thanks a lot to Ard Biesheuvel & Kees Cook on their feedback on compiler changes, PIE support and KASLR in general. Thanks to Roland McGrath on his feedback for using -pie versus --emit-relocs and details on compiler code generation. The patches: - 1-3, 5-13, 18-19: Change in assembly code to be PIE compliant. - 4: Add a new _ASM_GET_PTR macro to fetch a symbol address generically. - 14: Adapt percpu design to work correctly when PIE is enabled. - 15: Provide an option to default visibility to hidden except for key symbols. It removes errors between compilation units. - 16: Add PROVIDE_HIDDEN replacement on the linker script for weak symbols to reduce GOT footprint. - 17: Adapt relocation tool to handle PIE binary correctly. - 20: Add support for global cookie. - 21: Support ftrace with PIE (used on Ubuntu config). - 22: Add option to move the module section just after the kernel. - 23: Adapt module loading to support PIE with dynamic GOT. - 24: Make the GOT read-only. - 25: Add the CONFIG_X86_PIE option (off by default). - 26: Adapt relocation tool to generate a 64-bit relocation table. - 27: Add the CONFIG_RANDOMIZE_BASE_LARGE option to increase relocation range from 1G to 3G (off by default). Performance/Size impact: Size of vmlinux (Default configuration): File size: - PIE disabled: +0.18% - PIE enabled: -1.977% (less relocations) .text section: - PIE disabled: same - PIE enabled: same Size of vmlinux (Ubuntu configuration): File size: - PIE disabled: +0.21% - PIE enabled: +10% .text section: - PIE disabled: same - PIE enabled: +0.001% The size increase is mainly due to not having access to the 32-bit signed relocation that can be used with mcmodel=kernel. A small part is due to reduced optimization for PIE code. This bug [1] was opened with gcc to provide a better code generation for kernel PIE. Hackbench (50% and 1600% on thread/process for pipe/sockets): - PIE disabled: no significant change (avg -/+ 0.5% on latest test). - PIE enabled: between -1% to +1% in average (default and Ubuntu config). Kernbench (average of 10 Half and Optimal runs): Elapsed Time: - PIE disabled: no significant change (avg -0.5%) - PIE enabled: average -0.5% to +0.5% System Time: - PIE disabled: no significant change (avg -0.1%) - PIE enabled: average -0.4% to +0.4%. [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=82303 diffstat: Documentation/x86/x86_64/mm.txt | 3 arch/x86/Kconfig | 45 ++++++ arch/x86/Makefile | 58 ++++++++ arch/x86/boot/boot.h | 2 arch/x86/boot/compressed/Makefile | 5 arch/x86/boot/compressed/misc.c | 10 + arch/x86/crypto/aes-x86_64-asm_64.S | 45 ++++-- arch/x86/crypto/aesni-intel_asm.S | 8 - arch/x86/crypto/aesni-intel_avx-x86_64.S | 6 arch/x86/crypto/camellia-aesni-avx-asm_64.S | 42 +++--- arch/x86/crypto/camellia-aesni-avx2-asm_64.S | 44 +++--- arch/x86/crypto/camellia-x86_64-asm_64.S | 8 - arch/x86/crypto/cast5-avx-x86_64-asm_64.S | 50 ++++--- arch/x86/crypto/cast6-avx-x86_64-asm_64.S | 44 +++--- arch/x86/crypto/des3_ede-asm_64.S | 96 +++++++++----- arch/x86/crypto/ghash-clmulni-intel_asm.S | 4 arch/x86/crypto/glue_helper-asm-avx.S | 4 arch/x86/crypto/glue_helper-asm-avx2.S | 6 arch/x86/crypto/sha256-avx2-asm.S | 23 ++- arch/x86/entry/calling.h | 2 arch/x86/entry/entry_32.S | 3 arch/x86/entry/entry_64.S | 23 ++- arch/x86/include/asm/asm.h | 1 arch/x86/include/asm/bug.h | 2 arch/x86/include/asm/ftrace.h | 6 arch/x86/include/asm/jump_label.h | 8 - arch/x86/include/asm/kvm_host.h | 6 arch/x86/include/asm/module.h | 11 + arch/x86/include/asm/page_64_types.h | 9 + arch/x86/include/asm/paravirt_types.h | 12 + arch/x86/include/asm/percpu.h | 25 ++- arch/x86/include/asm/pgtable_64_types.h | 6 arch/x86/include/asm/pm-trace.h | 2 arch/x86/include/asm/processor.h | 12 + arch/x86/include/asm/sections.h | 8 + arch/x86/include/asm/setup.h | 2 arch/x86/include/asm/stackprotector.h | 19 ++ arch/x86/kernel/Makefile | 6 arch/x86/kernel/acpi/wakeup_64.S | 31 ++-- arch/x86/kernel/asm-offsets.c | 3 arch/x86/kernel/asm-offsets_32.c | 3 arch/x86/kernel/asm-offsets_64.c | 3 arch/x86/kernel/cpu/common.c | 7 - arch/x86/kernel/cpu/microcode/core.c | 4 arch/x86/kernel/ftrace.c | 42 +++++- arch/x86/kernel/head64.c | 23 ++- arch/x86/kernel/head_32.S | 3 arch/x86/kernel/head_64.S | 41 +++++- arch/x86/kernel/kvm.c | 6 arch/x86/kernel/module.c | 181 ++++++++++++++++++++++++++- arch/x86/kernel/module.lds | 3 arch/x86/kernel/process.c | 5 arch/x86/kernel/relocate_kernel_64.S | 16 +- arch/x86/kernel/setup_percpu.c | 2 arch/x86/kernel/vmlinux.lds.S | 13 + arch/x86/kvm/svm.c | 4 arch/x86/lib/cmpxchg16b_emu.S | 8 - arch/x86/mm/dump_pagetables.c | 3 arch/x86/power/hibernate_asm_64.S | 4 arch/x86/tools/relocs.c | 169 +++++++++++++++++++++++-- arch/x86/tools/relocs.h | 4 arch/x86/tools/relocs_common.c | 15 +- arch/x86/xen/xen-asm.S | 12 - arch/x86/xen/xen-head.S | 11 - arch/x86/xen/xen-pvh.S | 13 + drivers/base/firmware_class.c | 4 include/asm-generic/sections.h | 6 include/asm-generic/vmlinux.lds.h | 12 + include/linux/compiler.h | 7 + init/Kconfig | 16 ++ kernel/kallsyms.c | 16 +- kernel/trace/trace.h | 4 lib/dynamic_debug.c | 4 scripts/link-vmlinux.sh | 14 ++ 74 files changed, 1063 insertions(+), 315 deletions(-) From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: ARC-Seal: i=1; a=rsa-sha256; t=1520974811; cv=none; d=google.com; s=arc-20160816; b=y/1b9/KAtXFiwY83p5Gfiy6/wqtpCU8PQ9PqzhDaIVDTKSXIV1Ec2XLXKUbvGvlPWT 5vE4F/BBZ0gBfllHgA2iiItDo6GJ+rsL1lXVZr4GzM4pFN274oIZkOi80dt/oCsYYqgv +gxXuoK2mHlUW8EmOCXZaf7eV4zDrsWPa0n1rVNk4F0O918bt5DpnJVqyd/vGCAuK3ee KHNI4ux11pnG6WKqT8MOI6u+jQ0jnuLDGqSgTxDJoqQtBmIB2HIe91U0QcFN9voYd/o6 lFbnOqYgNjB6SObIeGBeXGrFkvXiIfGlXQt27u1BBXdF0QE8jIdwEs4G8/mr1/zhagHU NwXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=+Z0I+Rbg3nmUTm+emVXn/AAop6lpLB5a9aV/ZXPykuY=; b=IdmJASoPjqrtIU9zEdkhrVDSYtpQM88PDRAn3Ce/OJ+76xDeIS6m06Z8r69RsEfjn6 LopPjoLvKxEj1xUPQ52+H8r7OLFprQpxrR0EKPOTcRiemb4J6KB33m+5za9YwJ+uvl2+ jaBII2bvwGergCzoWr1Y/hhzDbr8DOK3UlG4uNMQumD1Nc3ppukvfT5hcF5giwJD6A6U 3QM7BjtaklqsiIUeX6XkrHUHaNPTCgFGPdHyhp8+5+Veb1yOvgN6RJXLXhrqusjWoJ47 LJ7q22OerFPXvyMDCRF5NfoTl3Zd3jrJtxlPQGbr+EXe1dlxmiZclwlCyTP1Ggq0Ak6Q tR7Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=iEx+Oq1V; spf=pass (google.com: domain of thgarnie@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=thgarnie@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=iEx+Oq1V; spf=pass (google.com: domain of thgarnie@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=thgarnie@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com X-Google-Smtp-Source: AG47ELt7p5bTSiCVSOk7KCrhQILTuRRXMohYvVE1pQDkage7G5NXFlBnoUs6s7i/k4OFV6TiSEJLAw== From: Thomas Garnier To: Herbert Xu , "David S . Miller" , Thomas Gleixner , Ingo Molnar , "H . Peter Anvin" , Peter Zijlstra , Josh Poimboeuf , Greg Kroah-Hartman , Kate Stewart , Thomas Garnier , Arnd Bergmann , Philippe Ombredanne , Arnaldo Carvalho de Melo , Andrey Ryabinin , Matthias Kaehlcke , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Andy Lutomirski , Dominik Brodowski , Borislav Petkov , Borislav Petkov , "Rafael J . Wysocki" , Len Brown , Pavel Machek , Juergen Gross , Alok Kataria , Steven Rostedt , Tejun Heo , Christoph Lameter , Dennis Zhou , Boris Ostrovsky , David Woodhouse , Alexey Dobriyan , "Paul E . McKenney" , Andrew Morton , Nicolas Pitre , Randy Dunlap , "Luis R . Rodriguez" , Christopher Li , Jason Baron , Ashish Kalra , Kyle McMartin , Dou Liyang , Lukas Wunner , Petr Mladek , Sergey Senozhatsky , Masahiro Yamada , Ingo Molnar , Nicholas Piggin , Cao jin , "H . J . Lu" , Paolo Bonzini , =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= , Joerg Roedel , Dave Hansen , Rik van Riel , Jia Zhang , Jiri Slaby , Kyle Huey , Jonathan Corbet , Matthew Wilcox , Michal Hocko , Rob Landley , Baoquan He , Daniel Micay , =?UTF-8?q?Jan=20H=20=2E=20Sch=C3=B6nherr?= Cc: x86@kernel.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, virtualization@lists.linux-foundation.org, xen-devel@lists.xenproject.org, linux-arch@vger.kernel.org, linux-sparse@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, kernel-hardening@lists.openwall.com Subject: [PATCH v2 00/27] x86: PIE support and option to extend KASLR randomization Date: Tue, 13 Mar 2018 13:59:18 -0700 Message-Id: <20180313205945.245105-1-thgarnie@google.com> X-Mailer: git-send-email 2.16.2.660.g709887971b-goog X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: =?utf-8?q?1594857684125982930?= X-GMAIL-MSGID: =?utf-8?q?1594857684125982930?= X-Mailing-List: linux-kernel@vger.kernel.org List-ID: Changes: - patch v2: - Adapt patch to work post KPTI and compiler changes - Redo all performance testing with latest configs and compilers - Simplify mov macro on PIE (MOVABS now) - Reduce GOT footprint - patch v1: - Simplify ftrace implementation. - Use gcc mstack-protector-guard-reg=%gs with PIE when possible. - rfc v3: - Use --emit-relocs instead of -pie to reduce dynamic relocation space on mapped memory. It also simplifies the relocation process. - Move the start the module section next to the kernel. Remove the need for -mcmodel=large on modules. Extends module space from 1 to 2G maximum. - Support for XEN PVH as 32-bit relocations can be ignored with --emit-relocs. - Support for GOT relocations previously done automatically with -pie. - Remove need for dynamic PLT in modules. - Support dymamic GOT for modules. - rfc v2: - Add support for global stack cookie while compiler default to fs without mcmodel=kernel - Change patch 7 to correctly jump out of the identity mapping on kexec load preserve. These patches make the changes necessary to build the kernel as Position Independent Executable (PIE) on x86_64. A PIE kernel can be relocated below the top 2G of the virtual address space. It allows to optionally extend the KASLR randomization range from 1G to 3G. Thanks a lot to Ard Biesheuvel & Kees Cook on their feedback on compiler changes, PIE support and KASLR in general. Thanks to Roland McGrath on his feedback for using -pie versus --emit-relocs and details on compiler code generation. The patches: - 1-3, 5-13, 18-19: Change in assembly code to be PIE compliant. - 4: Add a new _ASM_GET_PTR macro to fetch a symbol address generically. - 14: Adapt percpu design to work correctly when PIE is enabled. - 15: Provide an option to default visibility to hidden except for key symbols. It removes errors between compilation units. - 16: Add PROVIDE_HIDDEN replacement on the linker script for weak symbols to reduce GOT footprint. - 17: Adapt relocation tool to handle PIE binary correctly. - 20: Add support for global cookie. - 21: Support ftrace with PIE (used on Ubuntu config). - 22: Add option to move the module section just after the kernel. - 23: Adapt module loading to support PIE with dynamic GOT. - 24: Make the GOT read-only. - 25: Add the CONFIG_X86_PIE option (off by default). - 26: Adapt relocation tool to generate a 64-bit relocation table. - 27: Add the CONFIG_RANDOMIZE_BASE_LARGE option to increase relocation range from 1G to 3G (off by default). Performance/Size impact: Size of vmlinux (Default configuration): File size: - PIE disabled: +0.18% - PIE enabled: -1.977% (less relocations) .text section: - PIE disabled: same - PIE enabled: same Size of vmlinux (Ubuntu configuration): File size: - PIE disabled: +0.21% - PIE enabled: +10% .text section: - PIE disabled: same - PIE enabled: +0.001% The size increase is mainly due to not having access to the 32-bit signed relocation that can be used with mcmodel=kernel. A small part is due to reduced optimization for PIE code. This bug [1] was opened with gcc to provide a better code generation for kernel PIE. Hackbench (50% and 1600% on thread/process for pipe/sockets): - PIE disabled: no significant change (avg -/+ 0.5% on latest test). - PIE enabled: between -1% to +1% in average (default and Ubuntu config). Kernbench (average of 10 Half and Optimal runs): Elapsed Time: - PIE disabled: no significant change (avg -0.5%) - PIE enabled: average -0.5% to +0.5% System Time: - PIE disabled: no significant change (avg -0.1%) - PIE enabled: average -0.4% to +0.4%. [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=82303 diffstat: Documentation/x86/x86_64/mm.txt | 3 arch/x86/Kconfig | 45 ++++++ arch/x86/Makefile | 58 ++++++++ arch/x86/boot/boot.h | 2 arch/x86/boot/compressed/Makefile | 5 arch/x86/boot/compressed/misc.c | 10 + arch/x86/crypto/aes-x86_64-asm_64.S | 45 ++++-- arch/x86/crypto/aesni-intel_asm.S | 8 - arch/x86/crypto/aesni-intel_avx-x86_64.S | 6 arch/x86/crypto/camellia-aesni-avx-asm_64.S | 42 +++--- arch/x86/crypto/camellia-aesni-avx2-asm_64.S | 44 +++--- arch/x86/crypto/camellia-x86_64-asm_64.S | 8 - arch/x86/crypto/cast5-avx-x86_64-asm_64.S | 50 ++++--- arch/x86/crypto/cast6-avx-x86_64-asm_64.S | 44 +++--- arch/x86/crypto/des3_ede-asm_64.S | 96 +++++++++----- arch/x86/crypto/ghash-clmulni-intel_asm.S | 4 arch/x86/crypto/glue_helper-asm-avx.S | 4 arch/x86/crypto/glue_helper-asm-avx2.S | 6 arch/x86/crypto/sha256-avx2-asm.S | 23 ++- arch/x86/entry/calling.h | 2 arch/x86/entry/entry_32.S | 3 arch/x86/entry/entry_64.S | 23 ++- arch/x86/include/asm/asm.h | 1 arch/x86/include/asm/bug.h | 2 arch/x86/include/asm/ftrace.h | 6 arch/x86/include/asm/jump_label.h | 8 - arch/x86/include/asm/kvm_host.h | 6 arch/x86/include/asm/module.h | 11 + arch/x86/include/asm/page_64_types.h | 9 + arch/x86/include/asm/paravirt_types.h | 12 + arch/x86/include/asm/percpu.h | 25 ++- arch/x86/include/asm/pgtable_64_types.h | 6 arch/x86/include/asm/pm-trace.h | 2 arch/x86/include/asm/processor.h | 12 + arch/x86/include/asm/sections.h | 8 + arch/x86/include/asm/setup.h | 2 arch/x86/include/asm/stackprotector.h | 19 ++ arch/x86/kernel/Makefile | 6 arch/x86/kernel/acpi/wakeup_64.S | 31 ++-- arch/x86/kernel/asm-offsets.c | 3 arch/x86/kernel/asm-offsets_32.c | 3 arch/x86/kernel/asm-offsets_64.c | 3 arch/x86/kernel/cpu/common.c | 7 - arch/x86/kernel/cpu/microcode/core.c | 4 arch/x86/kernel/ftrace.c | 42 +++++- arch/x86/kernel/head64.c | 23 ++- arch/x86/kernel/head_32.S | 3 arch/x86/kernel/head_64.S | 41 +++++- arch/x86/kernel/kvm.c | 6 arch/x86/kernel/module.c | 181 ++++++++++++++++++++++++++- arch/x86/kernel/module.lds | 3 arch/x86/kernel/process.c | 5 arch/x86/kernel/relocate_kernel_64.S | 16 +- arch/x86/kernel/setup_percpu.c | 2 arch/x86/kernel/vmlinux.lds.S | 13 + arch/x86/kvm/svm.c | 4 arch/x86/lib/cmpxchg16b_emu.S | 8 - arch/x86/mm/dump_pagetables.c | 3 arch/x86/power/hibernate_asm_64.S | 4 arch/x86/tools/relocs.c | 169 +++++++++++++++++++++++-- arch/x86/tools/relocs.h | 4 arch/x86/tools/relocs_common.c | 15 +- arch/x86/xen/xen-asm.S | 12 - arch/x86/xen/xen-head.S | 11 - arch/x86/xen/xen-pvh.S | 13 + drivers/base/firmware_class.c | 4 include/asm-generic/sections.h | 6 include/asm-generic/vmlinux.lds.h | 12 + include/linux/compiler.h | 7 + init/Kconfig | 16 ++ kernel/kallsyms.c | 16 +- kernel/trace/trace.h | 4 lib/dynamic_debug.c | 4 scripts/link-vmlinux.sh | 14 ++ 74 files changed, 1063 insertions(+), 315 deletions(-)