From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org, stable@vger.kernel.org
Cc: Yonghong Song <yhs@fb.com>, Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Sasha Levin <sashal@kernel.org>,
netdev@vger.kernel.org, bpf@vger.kernel.org
Subject: [PATCH AUTOSEL 5.4 04/19] bpf, x86: Use kvmalloc_array instead kmalloc_array in bpf_jit_comp
Date: Mon, 29 Mar 2021 18:22:47 -0400 [thread overview]
Message-ID: <20210329222303.2383319-4-sashal@kernel.org> (raw)
In-Reply-To: <20210329222303.2383319-1-sashal@kernel.org>
From: Yonghong Song <yhs@fb.com>
[ Upstream commit de920fc64cbaa031f947e9be964bda05fd090380 ]
x86 bpf_jit_comp.c used kmalloc_array to store jited addresses
for each bpf insn. With a large bpf program, we have see the
following allocation failures in our production server:
page allocation failure: order:5, mode:0x40cc0(GFP_KERNEL|__GFP_COMP),
nodemask=(null),cpuset=/,mems_allowed=0"
Call Trace:
dump_stack+0x50/0x70
warn_alloc.cold.120+0x72/0xd2
? __alloc_pages_direct_compact+0x157/0x160
__alloc_pages_slowpath+0xcdb/0xd00
? get_page_from_freelist+0xe44/0x1600
? vunmap_page_range+0x1ba/0x340
__alloc_pages_nodemask+0x2c9/0x320
kmalloc_order+0x18/0x80
kmalloc_order_trace+0x1d/0xa0
bpf_int_jit_compile+0x1e2/0x484
? kmalloc_order_trace+0x1d/0xa0
bpf_prog_select_runtime+0xc3/0x150
bpf_prog_load+0x480/0x720
? __mod_memcg_lruvec_state+0x21/0x100
__do_sys_bpf+0xc31/0x2040
? close_pdeo+0x86/0xe0
do_syscall_64+0x42/0x110
entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x7f2f300f7fa9
Code: Bad RIP value.
Dumped assembly:
ffffffff810b6d70 <bpf_int_jit_compile>:
; {
ffffffff810b6d70: e8 eb a5 b4 00 callq 0xffffffff81c01360 <__fentry__>
ffffffff810b6d75: 41 57 pushq %r15
...
ffffffff810b6f39: e9 72 fe ff ff jmp 0xffffffff810b6db0 <bpf_int_jit_compile+0x40>
; addrs = kmalloc_array(prog->len + 1, sizeof(*addrs), GFP_KERNEL);
ffffffff810b6f3e: 8b 45 0c movl 12(%rbp), %eax
; return __kmalloc(bytes, flags);
ffffffff810b6f41: be c0 0c 00 00 movl $3264, %esi
; addrs = kmalloc_array(prog->len + 1, sizeof(*addrs), GFP_KERNEL);
ffffffff810b6f46: 8d 78 01 leal 1(%rax), %edi
; if (unlikely(check_mul_overflow(n, size, &bytes)))
ffffffff810b6f49: 48 c1 e7 02 shlq $2, %rdi
; return __kmalloc(bytes, flags);
ffffffff810b6f4d: e8 8e 0c 1d 00 callq 0xffffffff81287be0 <__kmalloc>
; if (!addrs) {
ffffffff810b6f52: 48 85 c0 testq %rax, %rax
Change kmalloc_array() to kvmalloc_array() to avoid potential
allocation error for big bpf programs.
Signed-off-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20210309015647.3657852-1-yhs@fb.com
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
arch/x86/net/bpf_jit_comp.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 18936533666e..44c7d7aef8c1 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -1118,7 +1118,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
extra_pass = true;
goto skip_init_addrs;
}
- addrs = kmalloc_array(prog->len + 1, sizeof(*addrs), GFP_KERNEL);
+ addrs = kvmalloc_array(prog->len + 1, sizeof(*addrs), GFP_KERNEL);
if (!addrs) {
prog = orig_prog;
goto out_addrs;
@@ -1195,7 +1195,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
if (image)
bpf_prog_fill_jited_linfo(prog, addrs + 1);
out_addrs:
- kfree(addrs);
+ kvfree(addrs);
kfree(jit_data);
prog->aux->jit_data = NULL;
}
--
2.30.1
next prev parent reply other threads:[~2021-03-29 22:28 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-29 22:22 [PATCH AUTOSEL 5.4 01/19] ARM: dts: am33xx: add aliases for mmc interfaces Sasha Levin
2021-03-29 22:22 ` [PATCH AUTOSEL 5.4 02/19] bus: ti-sysc: Fix warning on unbind if reset is not deasserted Sasha Levin
2021-03-29 22:22 ` [PATCH AUTOSEL 5.4 03/19] platform/x86: intel-hid: Support Lenovo ThinkPad X1 Tablet Gen 2 Sasha Levin
2021-03-29 22:22 ` Sasha Levin [this message]
2021-03-29 22:22 ` [PATCH AUTOSEL 5.4 05/19] net/mlx5e: Enforce minimum value check for ICOSQ size Sasha Levin
2021-03-29 22:22 ` [PATCH AUTOSEL 5.4 06/19] net: pxa168_eth: Fix a potential data race in pxa168_eth_remove Sasha Levin
2021-03-29 22:22 ` [PATCH AUTOSEL 5.4 07/19] mISDN: fix crash in fritzpci Sasha Levin
2021-03-29 22:22 ` [PATCH AUTOSEL 5.4 08/19] mac80211: fix double free in ibss_leave Sasha Levin
2021-03-29 22:22 ` [PATCH AUTOSEL 5.4 09/19] mac80211: choose first enabled channel for monitor Sasha Levin
2021-03-29 22:22 ` [PATCH AUTOSEL 5.4 10/19] drm/msm/adreno: a5xx_power: Don't apply A540 lm_setup to other GPUs Sasha Levin
2021-03-29 22:22 ` [PATCH AUTOSEL 5.4 11/19] drm/msm: Ratelimit invalid-fence message Sasha Levin
2021-03-29 22:22 ` [PATCH AUTOSEL 5.4 12/19] netfilter: conntrack: Fix gre tunneling over ipv6 Sasha Levin
2021-03-29 22:22 ` [PATCH AUTOSEL 5.4 13/19] platform/x86: thinkpad_acpi: Allow the FnLock LED to change state Sasha Levin
2021-03-29 22:22 ` [PATCH AUTOSEL 5.4 14/19] x86/build: Turn off -fcf-protection for realmode targets Sasha Levin
2021-03-29 22:22 ` [PATCH AUTOSEL 5.4 15/19] scsi: target: pscsi: Clean up after failure in pscsi_map_sg() Sasha Levin
2021-03-29 22:22 ` [PATCH AUTOSEL 5.4 16/19] ia64: mca: allocate early mca with GFP_ATOMIC Sasha Levin
2021-03-29 22:23 ` [PATCH AUTOSEL 5.4 17/19] ia64: fix format strings for err_inject Sasha Levin
2021-03-29 22:23 ` [PATCH AUTOSEL 5.4 18/19] cifs: revalidate mapping when we open files for SMB1 POSIX Sasha Levin
2021-03-29 22:23 ` [PATCH AUTOSEL 5.4 19/19] cifs: Silently ignore unknown oplock break handle Sasha Levin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210329222303.2383319-4-sashal@kernel.org \
--to=sashal@kernel.org \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=stable@vger.kernel.org \
--cc=yhs@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).