From: Anton Protopopov <aspsk@isovalent.com>
To: Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
John Fastabend <john.fastabend@gmail.com>,
Andrii Nakryiko <andrii@kernel.org>,
Martin KaFai Lau <martin.lau@linux.dev>,
Song Liu <song@kernel.org>, Yonghong Song <yhs@fb.com>,
KP Singh <kpsingh@kernel.org>,
Stanislav Fomichev <sdf@google.com>, Hao Luo <haoluo@google.com>,
Jiri Olsa <jolsa@kernel.org>,
bpf@vger.kernel.org
Cc: Anton Protopopov <aspsk@isovalent.com>
Subject: [v3 PATCH bpf-next 0/6] bpf: add percpu stats for bpf_map
Date: Fri, 30 Jun 2023 08:25:10 +0000 [thread overview]
Message-ID: <20230630082516.16286-1-aspsk@isovalent.com> (raw)
This series adds a mechanism for maps to populate per-cpu counters on
insertions/deletions. The sum of these counters can be accessed by a new kfunc
from map iterator and tracing programs.
The following patches are present in the series:
* Patch 1 adds a generic per-cpu counter to struct bpf_map
* Patch 2 adds a new kfunc to access the sum of per-cpu counters
* Patch 3 utilizes this mechanism for hash-based maps
* Patch 4 extends the preloaded map iterator to dump the sum
* Patch 5 adds a self-test for the change
* Patch 6 patches map_ptr selftest to check that elem_count was initialized
The reason for adding this functionality in our case (Cilium) is to get signals
about how full some heavy-used maps are and what the actual dynamic profile of
map capacity is. In the case of LRU maps this is impossible to get this
information anyhow else. The original presentation can be found here [1].
[1] https://lpc.events/event/16/contributions/1368/
v2 -> v3:
- split commits to better represent update logic
- remove filter from kfunc to allow all tracing programs
- extend selftests
v1 -> v2:
- make the counters generic part of struct bpf_map
- don't use map_info and /proc/self/fdinfo in favor of a kfunc
Anton Protopopov (6):
bpf: add percpu stats for bpf_map elements insertions/deletions
bpf: add a new kfunc to return current bpf_map elements count
bpf: populate the per-cpu insertions/deletions counters for hashmaps
bpf: make preloaded map iterators to display map elements count
selftests/bpf: test map percpu stats
selftests/bpf: check that ->elem_count is non-zero for the hash map
include/linux/bpf.h | 30 +
kernel/bpf/hashtab.c | 23 +-
kernel/bpf/map_iter.c | 39 +-
kernel/bpf/preload/iterators/iterators.bpf.c | 9 +-
.../iterators/iterators.lskel-little-endian.h | 526 +++++++++---------
.../bpf/map_tests/map_percpu_stats.c | 336 +++++++++++
.../selftests/bpf/progs/map_percpu_stats.c | 24 +
.../selftests/bpf/progs/map_ptr_kern.c | 3 +
8 files changed, 726 insertions(+), 264 deletions(-)
create mode 100644 tools/testing/selftests/bpf/map_tests/map_percpu_stats.c
create mode 100644 tools/testing/selftests/bpf/progs/map_percpu_stats.c
--
2.34.1
next reply other threads:[~2023-06-30 8:24 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-06-30 8:25 Anton Protopopov [this message]
2023-06-30 8:25 ` [v3 PATCH bpf-next 1/6] bpf: add percpu stats for bpf_map elements insertions/deletions Anton Protopopov
2023-06-30 8:25 ` [v3 PATCH bpf-next 2/6] bpf: add a new kfunc to return current bpf_map elements count Anton Protopopov
2023-06-30 8:25 ` [v3 PATCH bpf-next 3/6] bpf: populate the per-cpu insertions/deletions counters for hashmaps Anton Protopopov
2023-07-04 13:56 ` Hou Tao
2023-07-04 14:34 ` Anton Protopopov
2023-07-06 2:01 ` Hou Tao
2023-07-06 12:25 ` Anton Protopopov
2023-07-06 12:30 ` Hou Tao
2023-06-30 8:25 ` [v3 PATCH bpf-next 4/6] bpf: make preloaded map iterators to display map elements count Anton Protopopov
2023-06-30 8:25 ` [v3 PATCH bpf-next 5/6] selftests/bpf: test map percpu stats Anton Protopopov
2023-07-04 14:41 ` Hou Tao
2023-07-04 15:02 ` Anton Protopopov
2023-07-04 15:23 ` Anton Protopopov
2023-07-04 15:49 ` Anton Protopopov
2023-07-05 0:46 ` Hou Tao
2023-07-05 15:41 ` Anton Protopopov
2023-07-05 3:03 ` Hou Tao
2023-07-05 15:34 ` Anton Protopopov
2023-06-30 8:25 ` [v3 PATCH bpf-next 6/6] selftests/bpf: check that ->elem_count is non-zero for the hash map Anton Protopopov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230630082516.16286-1-aspsk@isovalent.com \
--to=aspsk@isovalent.com \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=haoluo@google.com \
--cc=john.fastabend@gmail.com \
--cc=jolsa@kernel.org \
--cc=kpsingh@kernel.org \
--cc=martin.lau@linux.dev \
--cc=sdf@google.com \
--cc=song@kernel.org \
--cc=yhs@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).