From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 698DFC433EF for ; Wed, 20 Oct 2021 11:01:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4F35F61372 for ; Wed, 20 Oct 2021 11:01:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230293AbhJTLD2 (ORCPT ); Wed, 20 Oct 2021 07:03:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59594 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230234AbhJTLDQ (ORCPT ); Wed, 20 Oct 2021 07:03:16 -0400 Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2A623C061746 for ; Wed, 20 Oct 2021 04:01:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Subject:Cc:To:From:Date:Message-ID: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To:References; bh=+vjq1HBXXgooiZit+xtiGrYctvDitJr9cOTnQB+e3fQ=; b=UcaNUzPVWzg/PncgnPbCTPfsIn WOU0+pBwIvOk9CDq98tY18IOoRFClmsU5DSmijvdAh3NAWfM5Vh80PRBBwpVXpGhLowAJJ2qVK065 K1Flzxe8FpxYrF+eNf6Zqrp35ElsSG5tmTS3ljrT+ISFixJfZMODfsb/3gxfqqSzhVdw1RSv8VpiK H1JfV8FS9vrG4JBvzNvAiJY7ffhk34+sixE6uDW/uDbuwyFZs15TSJ+Btxixp2XLghMMUxY5N3Pxc QozKy7wt0dPpVI3w1fBvafOA9NEaTIHVkz304J5sSOsdhKN+ijwcEMwPJwaH03T1mbUWbVqcf9uiC QDvi67nA==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1md9L0-00Aw10-Rr; Wed, 20 Oct 2021 11:00:51 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 02DA6300056; Wed, 20 Oct 2021 13:00:49 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id D36FC201BB3C6; Wed, 20 Oct 2021 13:00:49 +0200 (CEST) Message-ID: <20211020104442.021802560@infradead.org> User-Agent: quilt/0.66 Date: Wed, 20 Oct 2021 12:44:42 +0200 From: Peter Zijlstra To: x86@kernel.org, jpoimboe@redhat.com, andrew.cooper3@citrix.com Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, alexei.starovoitov@gmail.com, ndesaulniers@google.com Subject: [PATCH v2 00/14] x86: Rewrite the retpoline rewrite logic Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi, These patches rewrite the way retpolines are rewritten. Currently objtool emits alternative entries for most retpoline calls. However trying to extend that led to trouble (ELF files are horrid). Therefore completely overhaul this and have objtool emit a .retpoline_sites section that lists all compiler generated retpoline thunk calls. Then the kernel can do with them as it pleases. Notably it will: - rewrite them to indirect instructions for !RETPOLINE - rewrite them to lfence; indirect; for RETPOLINE_AMD, where size allows (boo clang!) Specifically, the !RETPOLINE case can now also deal with the clang-special conditional-indirect-tail-call: Jcc __x86_indirect_thunk_\reg. Finally, also update the x86 BPF jit to catch up to recent times and do these same things. All this should help improve performance by removing an indirection. Patches can also be found here: git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git objtool/core Changes since v1: - objtool: avoid self-modifying-code in .altinstr_replacement - objtool: shrink struct instruction - objtool: more cleanups - x86: more #ifdef - x86: GEN-for-each-reg cleanups/comments - x86: retpoline thunk array - x86,bpf: more complete retpoline replacement - build fixes --- arch/um/kernel/um_arch.c | 4 + arch/x86/include/asm/GEN-for-each-reg.h | 13 ++- arch/x86/include/asm/alternative.h | 1 + arch/x86/include/asm/asm-prototypes.h | 6 +- arch/x86/include/asm/nospec-branch.h | 59 ---------- arch/x86/kernel/alternative.c | 192 ++++++++++++++++++++++++++++++- arch/x86/kernel/cpu/bugs.c | 7 -- arch/x86/kernel/module.c | 9 +- arch/x86/kernel/vmlinux.lds.S | 14 +++ arch/x86/lib/retpoline.S | 50 ++------ arch/x86/net/bpf_jit_comp.c | 71 +++++++----- arch/x86/net/bpf_jit_comp32.c | 22 +++- tools/objtool/arch/x86/decode.c | 120 -------------------- tools/objtool/check.c | 195 ++++++++++++++++++++++---------- tools/objtool/elf.c | 84 -------------- tools/objtool/include/objtool/check.h | 1 - tools/objtool/include/objtool/elf.h | 2 +- tools/objtool/special.c | 8 -- 18 files changed, 431 insertions(+), 427 deletions(-)