* [PATCH] bpf/stackmap: Implement bpf_get_next_key
@ 2018-01-25 4:37 Joel Fernandes
2018-01-25 4:42 ` Alexei Starovoitov
0 siblings, 1 reply; 3+ messages in thread
From: Joel Fernandes @ 2018-01-25 4:37 UTC (permalink / raw)
To: linux-kernel
Cc: iovisor-dev, kernel-team, Joel Fernandes, Alexei Starovoitov,
Daniel Borkmann, Brendan Gregg, Brenden Blanco
Currently stackmaps can't be iterated over. The keys are obtained
through other maps and look ups have to be performed. In new usecases,
its useful to be able to iterate over the stackmap independently.
Implement bpf_get_next_key to make this possible.
More details of use case:
Currently iterating over stack maps is done like so, for example
in BCC tools offcputime script:
1. Obtain keys from the 'counts' hash map.
2. Look up stackmap with each of the keys obtained from step 1.
This makes the iteration of stackmap dependent on the counts map.
In a new tool I'm working on called BPFd [1], it gives a huge speed
up when I dump entire maps before transmitting them to BCC tools
on a different machine [2]. This patch makes it possible to dump
stackmaps independent of other maps.
Tested on x86 and arm64 machines.
[1] https://lwn.net/Articles/744522/
[2] https://github.com/joelagnel/bpfd/issues/8
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Brendan Gregg <bgregg@netflix.com>
Cc: Brenden Blanco <bblanco@gmail.com>
Signed-off-by: Joel Fernandes <joelaf@google.com>
---
kernel/bpf/stackmap.c | 18 +++++++++++++++++-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c
index a15bc636cc98..b0bf7d009f76 100644
--- a/kernel/bpf/stackmap.c
+++ b/kernel/bpf/stackmap.c
@@ -228,7 +228,23 @@ int bpf_stackmap_copy(struct bpf_map *map, void *key, void *value)
static int stack_map_get_next_key(struct bpf_map *map, void *key, void *next_key)
{
- return -EINVAL;
+ struct bpf_stack_map *smap = container_of(map, struct bpf_stack_map, map);
+ u32 id = 0;
+
+ if (key) {
+ id = *(u32 *)key;
+ if (unlikely(id >= smap->n_buckets))
+ return -ENOENT;
+ id++;
+ }
+
+ for ( ; id < smap->n_buckets; id++) {
+ if (READ_ONCE(smap->buckets[id])) {
+ *(u32 *)next_key = id;
+ return 0;
+ }
+ }
+ return -ENOENT;
}
static int stack_map_update_elem(struct bpf_map *map, void *key, void *value,
--
2.16.0.rc1.238.g530d649a79-goog
^ permalink raw reply related [flat|nested] 3+ messages in thread* Re: [PATCH] bpf/stackmap: Implement bpf_get_next_key
2018-01-25 4:37 [PATCH] bpf/stackmap: Implement bpf_get_next_key Joel Fernandes
@ 2018-01-25 4:42 ` Alexei Starovoitov
2018-01-25 4:53 ` Joel Fernandes
0 siblings, 1 reply; 3+ messages in thread
From: Alexei Starovoitov @ 2018-01-25 4:42 UTC (permalink / raw)
To: Joel Fernandes; +Cc: linux-kernel, kernel-team, Daniel Borkmann, Brendan Gregg
On Wed, Jan 24, 2018 at 08:37:52PM -0800, Joel Fernandes wrote:
> Currently stackmaps can't be iterated over. The keys are obtained
> through other maps and look ups have to be performed. In new usecases,
> its useful to be able to iterate over the stackmap independently.
> Implement bpf_get_next_key to make this possible.
>
> More details of use case:
> Currently iterating over stack maps is done like so, for example
> in BCC tools offcputime script:
> 1. Obtain keys from the 'counts' hash map.
> 2. Look up stackmap with each of the keys obtained from step 1.
>
> This makes the iteration of stackmap dependent on the counts map.
> In a new tool I'm working on called BPFd [1], it gives a huge speed
> up when I dump entire maps before transmitting them to BCC tools
> on a different machine [2]. This patch makes it possible to dump
> stackmaps independent of other maps.
>
> Tested on x86 and arm64 machines.
>
> [1] https://lwn.net/Articles/744522/
> [2] https://github.com/joelagnel/bpfd/issues/8
>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Daniel Borkmann <daniel@iogearbox.net>
> Cc: Brendan Gregg <bgregg@netflix.com>
> Cc: Brenden Blanco <bblanco@gmail.com>
> Signed-off-by: Joel Fernandes <joelaf@google.com>
> ---
> kernel/bpf/stackmap.c | 18 +++++++++++++++++-
> 1 file changed, 17 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c
> index a15bc636cc98..b0bf7d009f76 100644
> --- a/kernel/bpf/stackmap.c
> +++ b/kernel/bpf/stackmap.c
> @@ -228,7 +228,23 @@ int bpf_stackmap_copy(struct bpf_map *map, void *key, void *value)
>
> static int stack_map_get_next_key(struct bpf_map *map, void *key, void *next_key)
> {
> - return -EINVAL;
> + struct bpf_stack_map *smap = container_of(map, struct bpf_stack_map, map);
> + u32 id = 0;
did you check net-next tree before sending the patch?
Also please see Documentation/bpf/bpf_devel_QA.txt
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH] bpf/stackmap: Implement bpf_get_next_key
2018-01-25 4:42 ` Alexei Starovoitov
@ 2018-01-25 4:53 ` Joel Fernandes
0 siblings, 0 replies; 3+ messages in thread
From: Joel Fernandes @ 2018-01-25 4:53 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: LKML, Cc: Android Kernel, Daniel Borkmann, Brendan Gregg
On Wed, Jan 24, 2018 at 8:42 PM, Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
> On Wed, Jan 24, 2018 at 08:37:52PM -0800, Joel Fernandes wrote:
>> Currently stackmaps can't be iterated over. The keys are obtained
>> through other maps and look ups have to be performed. In new usecases,
>> its useful to be able to iterate over the stackmap independently.
>> Implement bpf_get_next_key to make this possible.
>>
>> More details of use case:
>> Currently iterating over stack maps is done like so, for example
>> in BCC tools offcputime script:
>> 1. Obtain keys from the 'counts' hash map.
>> 2. Look up stackmap with each of the keys obtained from step 1.
>>
>> This makes the iteration of stackmap dependent on the counts map.
>> In a new tool I'm working on called BPFd [1], it gives a huge speed
>> up when I dump entire maps before transmitting them to BCC tools
>> on a different machine [2]. This patch makes it possible to dump
>> stackmaps independent of other maps.
>>
>> Tested on x86 and arm64 machines.
>>
>> [1] https://lwn.net/Articles/744522/
>> [2] https://github.com/joelagnel/bpfd/issues/8
>>
>> Cc: Alexei Starovoitov <ast@kernel.org>
>> Cc: Daniel Borkmann <daniel@iogearbox.net>
>> Cc: Brendan Gregg <bgregg@netflix.com>
>> Cc: Brenden Blanco <bblanco@gmail.com>
>> Signed-off-by: Joel Fernandes <joelaf@google.com>
>> ---
>> kernel/bpf/stackmap.c | 18 +++++++++++++++++-
>> 1 file changed, 17 insertions(+), 1 deletion(-)
>>
>> diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c
>> index a15bc636cc98..b0bf7d009f76 100644
>> --- a/kernel/bpf/stackmap.c
>> +++ b/kernel/bpf/stackmap.c
>> @@ -228,7 +228,23 @@ int bpf_stackmap_copy(struct bpf_map *map, void *key, void *value)
>>
>> static int stack_map_get_next_key(struct bpf_map *map, void *key, void *next_key)
>> {
>> - return -EINVAL;
>> + struct bpf_stack_map *smap = container_of(map, struct bpf_stack_map, map);
>> + u32 id = 0;
>
> did you check net-next tree before sending the patch?
Oh. I was a bit late to send it! Glad its done though.
> Also please see Documentation/bpf/bpf_devel_QA.txt
Sure, if you don't mind could you elaborate what I screwed up from the
QA with regard to this patch? Sorry if I missed something.
thanks,
- Joel
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2018-01-25 4:53 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-01-25 4:37 [PATCH] bpf/stackmap: Implement bpf_get_next_key Joel Fernandes
2018-01-25 4:42 ` Alexei Starovoitov
2018-01-25 4:53 ` Joel Fernandes
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).