From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org, stable@vger.kernel.org
Cc: Josh Poimboeuf <jpoimboe@redhat.com>,
Randy Dunlap <rdunlap@infradead.org>,
Thomas Gleixner <tglx@linutronix.de>,
Alexei Starovoitov <ast@kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Sasha Levin <sashal@kernel.org>,
netdev@vger.kernel.org, bpf@vger.kernel.org
Subject: [PATCH AUTOSEL 5.2 82/85] bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()
Date: Fri, 26 Jul 2019 09:39:32 -0400 [thread overview]
Message-ID: <20190726133936.11177-82-sashal@kernel.org> (raw)
In-Reply-To: <20190726133936.11177-1-sashal@kernel.org>
From: Josh Poimboeuf <jpoimboe@redhat.com>
[ Upstream commit 3193c0836f203a91bef96d88c64cccf0be090d9c ]
On x86-64, with CONFIG_RETPOLINE=n, GCC's "global common subexpression
elimination" optimization results in ___bpf_prog_run()'s jumptable code
changing from this:
select_insn:
jmp *jumptable(, %rax, 8)
...
ALU64_ADD_X:
...
jmp *jumptable(, %rax, 8)
ALU_ADD_X:
...
jmp *jumptable(, %rax, 8)
to this:
select_insn:
mov jumptable, %r12
jmp *(%r12, %rax, 8)
...
ALU64_ADD_X:
...
jmp *(%r12, %rax, 8)
ALU_ADD_X:
...
jmp *(%r12, %rax, 8)
The jumptable address is placed in a register once, at the beginning of
the function. The function execution can then go through multiple
indirect jumps which rely on that same register value. This has a few
issues:
1) Objtool isn't smart enough to be able to track such a register value
across multiple recursive indirect jumps through the jump table.
2) With CONFIG_RETPOLINE enabled, this optimization actually results in
a small slowdown. I measured a ~4.7% slowdown in the test_bpf
"tcpdump port 22" selftest.
This slowdown is actually predicted by the GCC manual:
Note: When compiling a program using computed gotos, a GCC
extension, you may get better run-time performance if you
disable the global common subexpression elimination pass by
adding -fno-gcse to the command line.
So just disable the optimization for this function.
Fixes: e55a73251da3 ("bpf: Fix ORC unwinding in non-JIT BPF code")
Reported-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/30c3ca29ba037afcbd860a8672eef0021addf9fe.1563413318.git.jpoimboe@redhat.com
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
include/linux/compiler-gcc.h | 2 ++
include/linux/compiler_types.h | 4 ++++
kernel/bpf/core.c | 2 +-
3 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h
index e8579412ad21..d7ee4c6bad48 100644
--- a/include/linux/compiler-gcc.h
+++ b/include/linux/compiler-gcc.h
@@ -170,3 +170,5 @@
#else
#define __diag_GCC_8(s)
#endif
+
+#define __no_fgcse __attribute__((optimize("-fno-gcse")))
diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h
index 19e58b9138a0..0454d82f8bd8 100644
--- a/include/linux/compiler_types.h
+++ b/include/linux/compiler_types.h
@@ -187,6 +187,10 @@ struct ftrace_likely_data {
#define asm_volatile_goto(x...) asm goto(x)
#endif
+#ifndef __no_fgcse
+# define __no_fgcse
+#endif
+
/* Are two types/vars the same type (ignoring qualifiers)? */
#define __same_type(a, b) __builtin_types_compatible_p(typeof(a), typeof(b))
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 080e2bb644cc..ebfd189916dc 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1295,7 +1295,7 @@ bool bpf_opcode_in_insntable(u8 code)
*
* Decode and execute eBPF instructions.
*/
-static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn, u64 *stack)
+static u64 __no_fgcse ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn, u64 *stack)
{
#define BPF_INSN_2_LBL(x, y) [BPF_##x | BPF_##y] = &&x##_##y
#define BPF_INSN_3_LBL(x, y, z) [BPF_##x | BPF_##y | BPF_##z] = &&x##_##y##_##z
--
2.20.1
next prev parent reply other threads:[~2019-07-26 13:42 UTC|newest]
Thread overview: 86+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-26 13:38 [PATCH AUTOSEL 5.2 01/85] ARM: riscpc: fix DMA Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 02/85] ARM: dts: rockchip: Make rk3288-veyron-minnie run at hs200 Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 03/85] ARM: dts: rockchip: Make rk3288-veyron-mickey's emmc work again Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 04/85] clk: meson: mpll: properly handle spread spectrum Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 05/85] ARM: dts: rockchip: Mark that the rk3288 timer might stop in suspend Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 06/85] ftrace: Enable trampoline when rec count returns back to one Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 07/85] arm64: dts: qcom: qcs404-evb: fix l3 min voltage Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 08/85] soc: qcom: rpmpd: fixup rpmpd set performance state Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 09/85] arm64: dts: marvell: mcbin: enlarge PCI memory window Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 10/85] soc: imx: soc-imx8: Correct return value of error handle Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 11/85] dmaengine: tegra-apb: Error out if DMA_PREP_INTERRUPT flag is unset Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 12/85] arm64: dts: rockchip: fix isp iommu clocks and power domain Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 13/85] kernel/module.c: Only return -EEXIST for modules that have finished loading Sasha Levin
2019-07-26 13:46 ` Prarit Bhargava
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 14/85] PCI: OF: Initialize dev->fwnode appropriately Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 15/85] firmware/psci: psci_checker: Park kthreads before stopping them Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 16/85] soc: imx8: Fix potential kernel dump in error path Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 17/85] arm64: qcom: qcs404: Add reset-cells to GCC node Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 18/85] swiotlb: fix phys_addr_t overflow warning Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 19/85] MIPS: lantiq: Fix bitfield masking Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 20/85] dmaengine: rcar-dmac: Reject zero-length slave DMA requests Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 21/85] ARM: exynos: Only build MCPM support if used Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 22/85] clk: tegra210: fix PLLU and PLLU_OUT1 Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 23/85] fs/adfs: super: fix use-after-free bug Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 24/85] clk: sprd: Add check for return value of sprd_clk_regmap_init() Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 25/85] arm64: dts: rockchip: Fix USB3 Type-C on rk3399-sapphire Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 26/85] btrfs: tree-checker: Check if the file extent end overflows Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 27/85] btrfs: fix minimum number of chunk errors for DUP Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 28/85] btrfs: Flush before reflinking any extent to prevent NOCOW write falling back to COW without data reservation Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 29/85] remoteproc: copy parent dma_pfn_offset for vdev Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 30/85] btrfs: qgroup: Don't hold qgroup_ioctl_lock in btrfs_qgroup_inherit() Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 31/85] cifs: Fix a race condition with cifs_echo_request Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 32/85] ceph: fix listxattr vxattr buffer length calculation Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 33/85] ceph: fix improper use of smp_mb__before_atomic() Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 34/85] ceph: fix dir_lease_is_valid() Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 35/85] ceph: return -ERANGE if virtual xattr value didn't fit in buffer Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 36/85] virtio-mmio: add error check for platform_get_irq Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 37/85] ACPI: blacklist: fix clang warning for unused DMI table Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 38/85] scsi: zfcp: fix GCC compiler warning emitted with -Wmaybe-uninitialized Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 39/85] selftests/bpf: do not ignore clang failures Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 40/85] drm/amd/display: Expose audio inst from DC to DM Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 41/85] cifs: fix crash in cifs_dfs_do_automount Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 42/85] perf version: Fix segfault due to missing OPT_END() Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 43/85] x86: kvm: avoid constant-conversion warning Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 44/85] ACPI: fix false-positive -Wuninitialized warning Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 45/85] ISDN: hfcsusb: checking idx of ep configuration Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 46/85] KVM: nVMX: Ignore segment base for VMX memory operand when segment not FS or GS Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 47/85] bpf: fix BTF verifier size resolution logic Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 48/85] libbpf: fix another GCC8 warning for strncpy Sasha Levin
2019-07-26 13:38 ` [PATCH AUTOSEL 5.2 49/85] be2net: Signal that the device cannot transmit during reconfiguration Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 50/85] mm/z3fold: don't try to use buddy slots after free Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 51/85] mm/slab_common.c: work around clang bug #42570 Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 52/85] mm/memcontrol.c: keep local VM counters in sync with the hierarchical ones Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 53/85] mm/z3fold.c: reinitialize zhdr structs after migration Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 54/85] x86/apic: Silence -Wtype-limits compiler warnings Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 55/85] x86: math-emu: Hide clang warnings for 16-bit overflow Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 56/85] mm/cma.c: fail if fixed declaration can't be honored Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 57/85] lib/test_overflow.c: avoid tainting the kernel and fix wrap size Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 58/85] lib/test_string.c: avoid masking memset16/32/64 failures Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 59/85] mm/ioremap: check virtual address alignment while creating huge mappings Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 60/85] coda: add error handling for fget Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 61/85] coda: fix build using bare-metal toolchain Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 62/85] uapi linux/coda_psdev.h: move upc_req definition from uapi to kernel side headers Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 63/85] drivers/rapidio/devices/rio_mport_cdev.c: NUL terminate some strings Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 64/85] drivers/pps/pps.c: clear offset flags in PPS_SETPARAMS ioctl Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 65/85] ipc/mqueue.c: only perform resource calculation if user valid Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 66/85] nds32: fix asm/syscall.h Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 67/85] device-dax: fix memory and resource leak if hotplug fails Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 68/85] mm/hotplug: make remove_memory() interface usable Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 69/85] floppy: fix div-by-zero in setup_format_params Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 70/85] floppy: fix out-of-bounds read in copy_buffer Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 71/85] mlxsw: spectrum_dcb: Configure DSCP map as the last rule is removed Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 72/85] stacktrace: Force USER_DS for stack_trace_save_user() Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 73/85] crypto: ccp - Fix SEV_VERSION_GREATER_OR_EQUAL Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 74/85] xen: let alloc_xenballooned_pages() fail if not enough memory free Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 75/85] xen/pv: Fix a boot up hang revealed by int3 self test Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 76/85] x86/uaccess: Remove ELF function annotation from copy_user_handle_tail() Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 77/85] objtool: Add mcsafe_handle_tail() to the uaccess safe list Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 78/85] x86/kvm: Don't call kvm_spurious_fault() from .fixup Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 79/85] x86/paravirt: Fix callee-saved function ELF sizes Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 80/85] bnxt_en: Fix VNIC accounting when enabling aRFS on 57500 chips Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 81/85] x86, boot: Remove multiple copy of static function sanitize_boot_params() Sasha Levin
2019-07-26 13:39 ` Sasha Levin [this message]
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 83/85] drm/nouveau: fix memory leak in nouveau_conn_reset() Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 84/85] dma-direct: correct the physical addr in dma_direct_sync_sg_for_cpu/device Sasha Levin
2019-07-26 13:39 ` [PATCH AUTOSEL 5.2 85/85] drm/nouveau/dmem: missing mutex_lock in error path Sasha Levin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190726133936.11177-82-sashal@kernel.org \
--to=sashal@kernel.org \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=jpoimboe@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=rdunlap@infradead.org \
--cc=stable@vger.kernel.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).