From: Roman Gushchin <guro@fb.com>
To: <bpf@vger.kernel.org>
Cc: <ast@kernel.org>, <daniel@iogearbox.net>,
<netdev@vger.kernel.org>, <andrii@kernel.org>,
<akpm@linux-foundation.org>, <linux-mm@kvack.org>,
<linux-kernel@vger.kernel.org>, <kernel-team@fb.com>
Subject: [PATCH bpf-next v9 09/34] bpf: refine memcg-based memory accounting for cpumap maps
Date: Tue, 1 Dec 2020 13:58:35 -0800 [thread overview]
Message-ID: <20201201215900.3569844-10-guro@fb.com> (raw)
In-Reply-To: <20201201215900.3569844-1-guro@fb.com>
Include metadata and percpu data into the memcg-based memory
accounting.
Signed-off-by: Roman Gushchin <guro@fb.com>
---
kernel/bpf/cpumap.c | 21 ++++++++++++---------
1 file changed, 12 insertions(+), 9 deletions(-)
diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c
index c61a23b564aa..90b949666605 100644
--- a/kernel/bpf/cpumap.c
+++ b/kernel/bpf/cpumap.c
@@ -97,7 +97,7 @@ static struct bpf_map *cpu_map_alloc(union bpf_attr *attr)
attr->map_flags & ~BPF_F_NUMA_NODE)
return ERR_PTR(-EINVAL);
- cmap = kzalloc(sizeof(*cmap), GFP_USER);
+ cmap = kzalloc(sizeof(*cmap), GFP_USER | __GFP_ACCOUNT);
if (!cmap)
return ERR_PTR(-ENOMEM);
@@ -412,7 +412,8 @@ static int __cpu_map_load_bpf_program(struct bpf_cpu_map_entry *rcpu, int fd)
}
static struct bpf_cpu_map_entry *
-__cpu_map_entry_alloc(struct bpf_cpumap_val *value, u32 cpu, int map_id)
+__cpu_map_entry_alloc(struct bpf_map *map, struct bpf_cpumap_val *value,
+ u32 cpu)
{
int numa, err, i, fd = value->bpf_prog.fd;
gfp_t gfp = GFP_KERNEL | __GFP_NOWARN;
@@ -422,13 +423,13 @@ __cpu_map_entry_alloc(struct bpf_cpumap_val *value, u32 cpu, int map_id)
/* Have map->numa_node, but choose node of redirect target CPU */
numa = cpu_to_node(cpu);
- rcpu = kzalloc_node(sizeof(*rcpu), gfp, numa);
+ rcpu = bpf_map_kmalloc_node(map, sizeof(*rcpu), gfp | __GFP_ZERO, numa);
if (!rcpu)
return NULL;
/* Alloc percpu bulkq */
- rcpu->bulkq = __alloc_percpu_gfp(sizeof(*rcpu->bulkq),
- sizeof(void *), gfp);
+ rcpu->bulkq = bpf_map_alloc_percpu(map, sizeof(*rcpu->bulkq),
+ sizeof(void *), gfp);
if (!rcpu->bulkq)
goto free_rcu;
@@ -438,7 +439,8 @@ __cpu_map_entry_alloc(struct bpf_cpumap_val *value, u32 cpu, int map_id)
}
/* Alloc queue */
- rcpu->queue = kzalloc_node(sizeof(*rcpu->queue), gfp, numa);
+ rcpu->queue = bpf_map_kmalloc_node(map, sizeof(*rcpu->queue), gfp,
+ numa);
if (!rcpu->queue)
goto free_bulkq;
@@ -447,7 +449,7 @@ __cpu_map_entry_alloc(struct bpf_cpumap_val *value, u32 cpu, int map_id)
goto free_queue;
rcpu->cpu = cpu;
- rcpu->map_id = map_id;
+ rcpu->map_id = map->id;
rcpu->value.qsize = value->qsize;
if (fd > 0 && __cpu_map_load_bpf_program(rcpu, fd))
@@ -455,7 +457,8 @@ __cpu_map_entry_alloc(struct bpf_cpumap_val *value, u32 cpu, int map_id)
/* Setup kthread */
rcpu->kthread = kthread_create_on_node(cpu_map_kthread_run, rcpu, numa,
- "cpumap/%d/map:%d", cpu, map_id);
+ "cpumap/%d/map:%d", cpu,
+ map->id);
if (IS_ERR(rcpu->kthread))
goto free_prog;
@@ -571,7 +574,7 @@ static int cpu_map_update_elem(struct bpf_map *map, void *key, void *value,
rcpu = NULL; /* Same as deleting */
} else {
/* Updating qsize cause re-allocation of bpf_cpu_map_entry */
- rcpu = __cpu_map_entry_alloc(&cpumap_value, key_cpu, map->id);
+ rcpu = __cpu_map_entry_alloc(map, &cpumap_value, key_cpu);
if (!rcpu)
return -ENOMEM;
rcpu->cmap = cmap;
--
2.26.2
next prev parent reply other threads:[~2020-12-01 22:00 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-01 21:58 [PATCH bpf-next v9 00/34] bpf: switch to memcg-based memory accounting Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 01/34] mm: memcontrol: use helpers to read page's memcg data Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 02/34] mm: memcontrol/slab: use helpers to access slab page's memcg_data Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 03/34] mm: introduce page memcg flags Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 04/34] mm: convert page kmemcg type to a page memcg flag Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 05/34] bpf: memcg-based memory accounting for bpf progs Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 06/34] bpf: prepare for memcg-based memory accounting for bpf maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 07/34] bpf: " Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 08/34] bpf: refine memcg-based memory accounting for arraymap maps Roman Gushchin
2020-12-01 21:58 ` Roman Gushchin [this message]
2020-12-01 21:58 ` [PATCH bpf-next v9 10/34] bpf: memcg-based memory accounting for cgroup storage maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 11/34] bpf: refine memcg-based memory accounting for devmap maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 12/34] bpf: refine memcg-based memory accounting for hashtab maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 13/34] bpf: memcg-based memory accounting for lpm_trie maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 14/34] bpf: memcg-based memory accounting for bpf ringbuffer Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 15/34] bpf: memcg-based memory accounting for bpf local storage maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 16/34] bpf: refine memcg-based memory accounting for sockmap and sockhash maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 17/34] bpf: refine memcg-based memory accounting for xskmap maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 18/34] bpf: eliminate rlimit-based memory accounting for arraymap maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 19/34] bpf: eliminate rlimit-based memory accounting for bpf_struct_ops maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 20/34] bpf: eliminate rlimit-based memory accounting for cpumap maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 21/34] bpf: eliminate rlimit-based memory accounting for cgroup storage maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 22/34] bpf: eliminate rlimit-based memory accounting for devmap maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 23/34] bpf: eliminate rlimit-based memory accounting for hashtab maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 24/34] bpf: eliminate rlimit-based memory accounting for lpm_trie maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 25/34] bpf: eliminate rlimit-based memory accounting for queue_stack_maps maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 26/34] bpf: eliminate rlimit-based memory accounting for reuseport_array maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 27/34] bpf: eliminate rlimit-based memory accounting for bpf ringbuffer Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 28/34] bpf: eliminate rlimit-based memory accounting for sockmap and sockhash maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 29/34] bpf: eliminate rlimit-based memory accounting for stackmap maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 30/34] bpf: eliminate rlimit-based memory accounting for xskmap maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 31/34] bpf: eliminate rlimit-based memory accounting for bpf local storage maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 32/34] bpf: eliminate rlimit-based memory accounting infra for bpf maps Roman Gushchin
2020-12-01 21:58 ` [PATCH bpf-next v9 33/34] bpf: eliminate rlimit-based memory accounting for bpf progs Roman Gushchin
2020-12-01 21:59 ` [PATCH bpf-next v9 34/34] bpf: samples: do not touch RLIMIT_MEMLOCK Roman Gushchin
2020-12-03 2:50 ` [PATCH bpf-next v9 00/34] bpf: switch to memcg-based memory accounting patchwork-bot+netdevbpf
2020-12-03 2:54 ` Alexei Starovoitov
2020-12-03 3:26 ` Roman Gushchin
[not found] ` <6abdd146-c584-9c66-261d-d7d39ff3f499@iogearbox.net>
2020-12-08 2:53 ` Andrii Nakryiko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201201215900.3569844-10-guro@fb.com \
--to=guro@fb.com \
--cc=akpm@linux-foundation.org \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=kernel-team@fb.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).