Re: [RFC bpf-next 0/3] tools: bpftool: add subcommand to count map entries

From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
To: Quentin Monnet <quentin.monnet@netronome.com>
Cc: Alexei Starovoitov <ast@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	bpf@vger.kernel.org, netdev@vger.kernel.org,
	oss-drivers@netronome.com
Subject: Re: [RFC bpf-next 0/3] tools: bpftool: add subcommand to count map entries
Date: Tue, 13 Aug 2019 18:51:51 -0700	[thread overview]
Message-ID: <20190814015149.b4pmubo3s4ou5yek@ast-mbp> (raw)
In-Reply-To: <20190813130921.10704-1-quentin.monnet@netronome.com>

On Tue, Aug 13, 2019 at 02:09:18PM +0100, Quentin Monnet wrote:
> This series adds a "bpftool map count" subcommand to count the number of
> entries present in a BPF map. This results from a customer request for a
> tool to count the number of entries in BPF maps used in production (for
> example, to know how many free entries are left in a given map).
> 
> The first two commits actually contain some clean-up in preparation for the
> new subcommand.
> 
> The third commit adds the new subcommand. Because what data should count as
> an entry is not entirely clear for all map types, we actually dump several
> counters, and leave it to the users to interpret the values.
> 
> Sending as a RFC because I'm looking for feedback on the approach. Is
> printing several values the good thing to do? Also, note that some map
> types such as queue/stack maps do not support any type of counting, this
> would need to be implemented in the kernel I believe.
> 
> More generally, we have a use case where (hash) maps are under pressure
> (many additions/deletions from the BPF program), and counting the entries
> by iterating other the different keys is not at all reliable. Would that
> make sense to add a new bpf() subcommand to count the entries on the kernel
> side instead of cycling over the entries in bpftool? If so, we would need
> to agree on what makes an entry for each kind of map.

I don't mind new bpftool sub-command, but against adding kernel interface.
Can you elaborate what is the actual use case?
The same can be achieved by 'bpftool map dump|grep key|wc -l', no?

> Note that we are also facing similar issues for purging map from their
> entries (deleting all entries at once). We can iterate on the keys and
> delete elements one by one, but this is very inefficient when entries are
> being added/removed in parallel from the BPF program, and having another
> dedicated command accessible from the bpf() system call might help here as
> well.

I think that fits into the batch processing of map commands discussion.