From: Song Liu <song@kernel.org>
To: <linux-mm@kvack.org>, <bpf@vger.kernel.org>,
<netdev@vger.kernel.org>, <x86@kernel.org>
Cc: <ast@kernel.org>, <daniel@iogearbox.net>, <andrii@kernel.org>,
<kernel-team@fb.com>, <akpm@linux-foundation.org>,
<pmenzel@molgen.mpg.de>, <rick.p.edgecombe@intel.com>,
Song Liu <song@kernel.org>
Subject: [PATCH bpf 4/4] bpf: use __vmalloc_node_range() with VM_TRY_HUGE_VMAP for bpf_prog_pack
Date: Wed, 30 Mar 2022 15:56:42 -0700 [thread overview]
Message-ID: <20220330225642.1163897-5-song@kernel.org> (raw)
In-Reply-To: <20220330225642.1163897-1-song@kernel.org>
We cannot yet savely enable HAVE_ARCH_HUGE_VMAP for all vmalloc in X86_64.
Let bpf_prog_pack to call __vmalloc_node_range() with VM_TRY_HUGE_VMAP
directly.
Signed-off-by: Song Liu <song@kernel.org>
---
kernel/bpf/core.c | 21 ++++++++++++++++++---
1 file changed, 18 insertions(+), 3 deletions(-)
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 13e9dbeeedf3..257c6457f256 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -851,13 +851,28 @@ static LIST_HEAD(pack_list);
#define BPF_HPAGE_MASK PAGE_MASK
#endif
+static void *bpf_prog_pack_vmalloc(unsigned long size)
+{
+#if defined(MODULES_VADDR)
+ unsigned long start = MODULES_VADDR;
+ unsigned long end = MODULES_END;
+#else
+ unsigned long start = VMALLOC_START;
+ unsigned long end = VMALLOC_END;
+#endif
+
+ return __vmalloc_node_range(size, PAGE_SIZE, start, end, GFP_KERNEL, PAGE_KERNEL,
+ VM_DEFER_KMEMLEAK | VM_TRY_HUGE_VMAP,
+ NUMA_NO_NODE, __builtin_return_address(0));
+}
+
static size_t select_bpf_prog_pack_size(void)
{
size_t size;
void *ptr;
size = BPF_HPAGE_SIZE * num_online_nodes();
- ptr = module_alloc(size);
+ ptr = bpf_prog_pack_vmalloc(size);
/* Test whether we can get huge pages. If not just use PAGE_SIZE
* packs.
@@ -881,7 +896,7 @@ static struct bpf_prog_pack *alloc_new_pack(void)
GFP_KERNEL);
if (!pack)
return NULL;
- pack->ptr = module_alloc(bpf_prog_pack_size);
+ pack->ptr = bpf_prog_pack_vmalloc(bpf_prog_pack_size);
if (!pack->ptr) {
kfree(pack);
return NULL;
@@ -970,7 +985,7 @@ static void bpf_prog_pack_free(struct bpf_binary_header *hdr)
if (bitmap_find_next_zero_area(pack->bitmap, bpf_prog_chunk_count(), 0,
bpf_prog_chunk_count(), 0) == 0) {
list_del(&pack->list);
- module_memfree(pack->ptr);
+ vfree(pack->ptr);
kfree(pack);
}
out:
--
2.30.2
next prev parent reply other threads:[~2022-03-30 23:08 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-30 22:56 [PATCH bpf 0/4] introduce HAVE_ARCH_HUGE_VMALLOC_FLAG for bpf_prog_pack Song Liu
2022-03-30 22:56 ` [PATCH bpf 1/4] x86: disable HAVE_ARCH_HUGE_VMALLOC Song Liu
2022-03-30 23:47 ` Thomas Gleixner
2022-03-30 22:56 ` [PATCH bpf 2/4] vmalloc: introduce HAVE_ARCH_HUGE_VMALLOC_FLAG Song Liu
2022-03-30 23:40 ` Edgecombe, Rick P
2022-03-31 0:26 ` Song Liu
2022-03-30 22:56 ` [PATCH bpf 3/4] x86: select HAVE_ARCH_HUGE_VMALLOC_FLAG for X86_64 Song Liu
2022-03-30 23:54 ` Thomas Gleixner
2022-03-31 0:30 ` Song Liu
2022-03-30 22:56 ` Song Liu [this message]
2022-03-31 0:00 ` [PATCH bpf 4/4] bpf: use __vmalloc_node_range() with VM_TRY_HUGE_VMAP for bpf_prog_pack Thomas Gleixner
2022-03-31 0:31 ` Song Liu
2022-03-31 0:04 ` [PATCH bpf 0/4] introduce HAVE_ARCH_HUGE_VMALLOC_FLAG " Edgecombe, Rick P
2022-03-31 0:46 ` Song Liu
2022-03-31 16:19 ` Edgecombe, Rick P
2022-03-31 5:37 ` Christoph Hellwig
2022-03-31 23:59 ` Song Liu
2022-04-01 22:22 ` Song Liu
2022-04-05 7:07 ` Christoph Hellwig
2022-04-05 23:54 ` Song Liu
2022-04-07 19:57 ` Song Liu
2022-04-08 10:08 ` Claudio Imbrenda
2022-04-08 21:22 ` Song Liu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220330225642.1163897-5-song@kernel.org \
--to=song@kernel.org \
--cc=akpm@linux-foundation.org \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=kernel-team@fb.com \
--cc=linux-mm@kvack.org \
--cc=netdev@vger.kernel.org \
--cc=pmenzel@molgen.mpg.de \
--cc=rick.p.edgecombe@intel.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).