* [BUG] bpf:
@ 2019-10-30 15:43 Jiri Olsa
2019-10-30 15:48 ` [BUG] bpf: oops in kfree_skb test Jiri Olsa
2019-10-30 16:44 ` [BUG] bpf: Alexei Starovoitov
0 siblings, 2 replies; 4+ messages in thread
From: Jiri Olsa @ 2019-10-30 15:43 UTC (permalink / raw)
To: Alexei Starovoitov; +Cc: davem, daniel, x86, netdev, bpf, kernel-team
hi,
I'm getting oops when running the kfree_skb test:
dell-r440-01 login: [ 758.049877] BUG: kernel NULL pointer dereference, address: 0000000000000000^M
[ 758.056834] #PF: supervisor read access in kernel mode^M
[ 758.061975] #PF: error_code(0x0000) - not-present page^M
[ 758.067112] PGD 8000000befba8067 P4D 8000000befba8067 PUD bffe11067 PMD 0 ^M
[ 758.073987] Oops: 0000 [#1] SMP PTI^M
[ 758.077478] CPU: 16 PID: 6854 Comm: test_progs Not tainted 5.4.0-rc3+ #96^M
[ 758.084263] Hardware name: Dell Inc. PowerEdge R440/08CYF7, BIOS 1.7.0 12/14/2018^M
[ 758.091745] RIP: 0010:0xffffffffc03b672c^M
[ 758.095669] Code: 4c 8b 6a 00 4c 89 6d c0 8b 77 00 89 75 cc 31 ff 89 75 fc 48 8b 71 00 48 01 fe bf 78 00 00 00 48 89 da 48 01 fa bf 08 00 00 00 <4c> 8b 76 00 4c 89 f6 48 01 fe 4c 8b 7e 00 48 89 ef 48 83 c7 f9 be^M
[ 758.114414] RSP: 0018:ffffaa3287583d20 EFLAGS: 00010286^M
[ 758.119640] RAX: ffffffffc03b66ac RBX: ffff9cef028c3900 RCX: ffff9cef0a652018^M
[ 758.126775] RDX: ffff9cef028c3978 RSI: 0000000000000000 RDI: 0000000000000008^M
[ 758.133906] RBP: ffffaa3287583d90 R08: 00000000000000b0 R09: 0000000000000000^M
[ 758.141040] R10: 98ff036c00000000 R11: 0000000000000040 R12: ffffffffba8b5c37^M
[ 758.148170] R13: ffff9cfb05daf440 R14: 0000000000000000 R15: 000000000000004a^M
[ 758.155303] FS: 00007f08a18d3740(0000) GS:ffff9cef10c00000(0000) knlGS:0000000000000000^M
[ 758.163392] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033^M
[ 758.169136] CR2: 0000000000000000 CR3: 0000000c08e50001 CR4: 00000000007606e0^M
[ 758.176268] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000^M
[ 758.183401] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400^M
[ 758.190534] PKRU: 55555554^M
[ 758.193248] Call Trace:^M
[ 758.195704] ? bpf_test_run+0x13d/0x230^M
[ 758.199539] ? _cond_resched+0x15/0x30^M
[ 758.203304] bpf_trace_run2+0x37/0x90^M
[ 758.206967] ? bpf_prog_test_run_skb+0x337/0x450^M
[ 758.211589] kfree_skb+0x73/0xa0^M
[ 758.214820] bpf_prog_test_run_skb+0x337/0x450^M
[ 758.219293] __do_sys_bpf+0x82e/0x1730^M
[ 758.223043] ? ep_show_fdinfo+0x80/0x80^M
[ 758.226885] do_syscall_64+0x5b/0x180^M
[ 758.230550] entry_SYSCALL_64_after_hwframe+0x44/0xa9^M
[ 758.235620] RIP: 0033:0x7f08a19e91fd^M
[ 758.239198] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 5b 8c 0c 00 f7 d8 64 89 01 48^M
this seems to be the culprit:
; ptr = dev->ifalias->rcuhead.next;
80: mov 0x0(%rsi),%r14
I used the patch below to bypass the crash, but I guess
verifier should not let this through
also the net_device struct in the test seems outdated
thanks,
jirka
---
tools/testing/selftests/bpf/progs/kfree_skb.c | 15 +++++++--------
1 file changed, 7 insertions(+), 8 deletions(-)
diff --git a/tools/testing/selftests/bpf/progs/kfree_skb.c b/tools/testing/selftests/bpf/progs/kfree_skb.c
index 89af8a921ee4..64d8c0237186 100644
--- a/tools/testing/selftests/bpf/progs/kfree_skb.c
+++ b/tools/testing/selftests/bpf/progs/kfree_skb.c
@@ -3,6 +3,7 @@
#include <linux/bpf.h>
#include "bpf_helpers.h"
#include "bpf_endian.h"
+#include "bpf_core_read.h"
char _license[] SEC("license") = "GPL";
struct {
@@ -70,14 +71,12 @@ int trace_kfree_skb(struct trace_kfree_skb *ctx)
unsigned short pkt_data;
char pkt_type;
- __builtin_preserve_access_index(({
- users = skb->users.refs.counter;
- data = skb->data;
- dev = skb->dev;
- ifindex = dev->ifindex;
- ptr = dev->ifalias->rcuhead.next;
- func = ptr->func;
- }));
+ users = BPF_CORE_READ(skb, users.refs.counter);
+ data = BPF_CORE_READ(skb, data);
+ dev = BPF_CORE_READ(skb, dev);
+ ifindex = BPF_CORE_READ(dev, ifindex);
+ ptr = BPF_CORE_READ(dev, ifalias, rcuhead.next);
+ func = BPF_CORE_READ(ptr, func);
bpf_probe_read(&pkt_type, sizeof(pkt_type), _(&skb->__pkt_type_offset));
pkt_type &= 7;
--
2.21.0
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [BUG] bpf: oops in kfree_skb test
2019-10-30 15:43 [BUG] bpf: Jiri Olsa
@ 2019-10-30 15:48 ` Jiri Olsa
2019-10-30 16:44 ` [BUG] bpf: Alexei Starovoitov
1 sibling, 0 replies; 4+ messages in thread
From: Jiri Olsa @ 2019-10-30 15:48 UTC (permalink / raw)
To: Alexei Starovoitov; +Cc: davem, daniel, x86, netdev, bpf, kernel-team
ugh, sry for missing subject
On Wed, Oct 30, 2019 at 04:43:23PM +0100, Jiri Olsa wrote:
>
> hi,
> I'm getting oops when running the kfree_skb test:
>
> dell-r440-01 login: [ 758.049877] BUG: kernel NULL pointer dereference, address: 0000000000000000^M
> [ 758.056834] #PF: supervisor read access in kernel mode^M
> [ 758.061975] #PF: error_code(0x0000) - not-present page^M
> [ 758.067112] PGD 8000000befba8067 P4D 8000000befba8067 PUD bffe11067 PMD 0 ^M
> [ 758.073987] Oops: 0000 [#1] SMP PTI^M
> [ 758.077478] CPU: 16 PID: 6854 Comm: test_progs Not tainted 5.4.0-rc3+ #96^M
> [ 758.084263] Hardware name: Dell Inc. PowerEdge R440/08CYF7, BIOS 1.7.0 12/14/2018^M
> [ 758.091745] RIP: 0010:0xffffffffc03b672c^M
> [ 758.095669] Code: 4c 8b 6a 00 4c 89 6d c0 8b 77 00 89 75 cc 31 ff 89 75 fc 48 8b 71 00 48 01 fe bf 78 00 00 00 48 89 da 48 01 fa bf 08 00 00 00 <4c> 8b 76 00 4c 89 f6 48 01 fe 4c 8b 7e 00 48 89 ef 48 83 c7 f9 be^M
> [ 758.114414] RSP: 0018:ffffaa3287583d20 EFLAGS: 00010286^M
> [ 758.119640] RAX: ffffffffc03b66ac RBX: ffff9cef028c3900 RCX: ffff9cef0a652018^M
> [ 758.126775] RDX: ffff9cef028c3978 RSI: 0000000000000000 RDI: 0000000000000008^M
> [ 758.133906] RBP: ffffaa3287583d90 R08: 00000000000000b0 R09: 0000000000000000^M
> [ 758.141040] R10: 98ff036c00000000 R11: 0000000000000040 R12: ffffffffba8b5c37^M
> [ 758.148170] R13: ffff9cfb05daf440 R14: 0000000000000000 R15: 000000000000004a^M
> [ 758.155303] FS: 00007f08a18d3740(0000) GS:ffff9cef10c00000(0000) knlGS:0000000000000000^M
> [ 758.163392] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033^M
> [ 758.169136] CR2: 0000000000000000 CR3: 0000000c08e50001 CR4: 00000000007606e0^M
> [ 758.176268] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000^M
> [ 758.183401] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400^M
> [ 758.190534] PKRU: 55555554^M
> [ 758.193248] Call Trace:^M
> [ 758.195704] ? bpf_test_run+0x13d/0x230^M
> [ 758.199539] ? _cond_resched+0x15/0x30^M
> [ 758.203304] bpf_trace_run2+0x37/0x90^M
> [ 758.206967] ? bpf_prog_test_run_skb+0x337/0x450^M
> [ 758.211589] kfree_skb+0x73/0xa0^M
> [ 758.214820] bpf_prog_test_run_skb+0x337/0x450^M
> [ 758.219293] __do_sys_bpf+0x82e/0x1730^M
> [ 758.223043] ? ep_show_fdinfo+0x80/0x80^M
> [ 758.226885] do_syscall_64+0x5b/0x180^M
> [ 758.230550] entry_SYSCALL_64_after_hwframe+0x44/0xa9^M
> [ 758.235620] RIP: 0033:0x7f08a19e91fd^M
> [ 758.239198] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 5b 8c 0c 00 f7 d8 64 89 01 48^M
>
>
> this seems to be the culprit:
>
> ; ptr = dev->ifalias->rcuhead.next;
> 80: mov 0x0(%rsi),%r14
>
> I used the patch below to bypass the crash, but I guess
> verifier should not let this through
>
> also the net_device struct in the test seems outdated
>
> thanks,
> jirka
>
>
> ---
> tools/testing/selftests/bpf/progs/kfree_skb.c | 15 +++++++--------
> 1 file changed, 7 insertions(+), 8 deletions(-)
>
> diff --git a/tools/testing/selftests/bpf/progs/kfree_skb.c b/tools/testing/selftests/bpf/progs/kfree_skb.c
> index 89af8a921ee4..64d8c0237186 100644
> --- a/tools/testing/selftests/bpf/progs/kfree_skb.c
> +++ b/tools/testing/selftests/bpf/progs/kfree_skb.c
> @@ -3,6 +3,7 @@
> #include <linux/bpf.h>
> #include "bpf_helpers.h"
> #include "bpf_endian.h"
> +#include "bpf_core_read.h"
>
> char _license[] SEC("license") = "GPL";
> struct {
> @@ -70,14 +71,12 @@ int trace_kfree_skb(struct trace_kfree_skb *ctx)
> unsigned short pkt_data;
> char pkt_type;
>
> - __builtin_preserve_access_index(({
> - users = skb->users.refs.counter;
> - data = skb->data;
> - dev = skb->dev;
> - ifindex = dev->ifindex;
> - ptr = dev->ifalias->rcuhead.next;
> - func = ptr->func;
> - }));
> + users = BPF_CORE_READ(skb, users.refs.counter);
> + data = BPF_CORE_READ(skb, data);
> + dev = BPF_CORE_READ(skb, dev);
> + ifindex = BPF_CORE_READ(dev, ifindex);
> + ptr = BPF_CORE_READ(dev, ifalias, rcuhead.next);
> + func = BPF_CORE_READ(ptr, func);
>
> bpf_probe_read(&pkt_type, sizeof(pkt_type), _(&skb->__pkt_type_offset));
> pkt_type &= 7;
> --
> 2.21.0
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [BUG] bpf:
2019-10-30 15:43 [BUG] bpf: Jiri Olsa
2019-10-30 15:48 ` [BUG] bpf: oops in kfree_skb test Jiri Olsa
@ 2019-10-30 16:44 ` Alexei Starovoitov
[not found] ` <20191030175735.GL20826@krava>
1 sibling, 1 reply; 4+ messages in thread
From: Alexei Starovoitov @ 2019-10-30 16:44 UTC (permalink / raw)
To: Jiri Olsa
Cc: Alexei Starovoitov, David S. Miller, Daniel Borkmann, X86 ML,
Network Development, bpf, Kernel Team
On Wed, Oct 30, 2019 at 8:43 AM Jiri Olsa <jolsa@redhat.com> wrote:
>
>
> hi,
> I'm getting oops when running the kfree_skb test:
>
> dell-r440-01 login: [ 758.049877] BUG: kernel NULL pointer dereference, address: 0000000000000000^M
> [ 758.056834] #PF: supervisor read access in kernel mode^M
> [ 758.061975] #PF: error_code(0x0000) - not-present page^M
> [ 758.067112] PGD 8000000befba8067 P4D 8000000befba8067 PUD bffe11067 PMD 0 ^M
> [ 758.073987] Oops: 0000 [#1] SMP PTI^M
> [ 758.077478] CPU: 16 PID: 6854 Comm: test_progs Not tainted 5.4.0-rc3+ #96^M
> [ 758.084263] Hardware name: Dell Inc. PowerEdge R440/08CYF7, BIOS 1.7.0 12/14/2018^M
> [ 758.091745] RIP: 0010:0xffffffffc03b672c^M
> [ 758.095669] Code: 4c 8b 6a 00 4c 89 6d c0 8b 77 00 89 75 cc 31 ff 89 75 fc 48 8b 71 00 48 01 fe bf 78 00 00 00 48 89 da 48 01 fa bf 08 00 00 00 <4c> 8b 76 00 4c 89 f6 48 01 fe 4c 8b 7e 00 48 89 ef 48 83 c7 f9 be^M
> [ 758.114414] RSP: 0018:ffffaa3287583d20 EFLAGS: 00010286^M
> [ 758.119640] RAX: ffffffffc03b66ac RBX: ffff9cef028c3900 RCX: ffff9cef0a652018^M
> [ 758.126775] RDX: ffff9cef028c3978 RSI: 0000000000000000 RDI: 0000000000000008^M
> [ 758.133906] RBP: ffffaa3287583d90 R08: 00000000000000b0 R09: 0000000000000000^M
> [ 758.141040] R10: 98ff036c00000000 R11: 0000000000000040 R12: ffffffffba8b5c37^M
> [ 758.148170] R13: ffff9cfb05daf440 R14: 0000000000000000 R15: 000000000000004a^M
> [ 758.155303] FS: 00007f08a18d3740(0000) GS:ffff9cef10c00000(0000) knlGS:0000000000000000^M
> [ 758.163392] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033^M
> [ 758.169136] CR2: 0000000000000000 CR3: 0000000c08e50001 CR4: 00000000007606e0^M
> [ 758.176268] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000^M
> [ 758.183401] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400^M
> [ 758.190534] PKRU: 55555554^M
> [ 758.193248] Call Trace:^M
> [ 758.195704] ? bpf_test_run+0x13d/0x230^M
> [ 758.199539] ? _cond_resched+0x15/0x30^M
> [ 758.203304] bpf_trace_run2+0x37/0x90^M
> [ 758.206967] ? bpf_prog_test_run_skb+0x337/0x450^M
> [ 758.211589] kfree_skb+0x73/0xa0^M
> [ 758.214820] bpf_prog_test_run_skb+0x337/0x450^M
> [ 758.219293] __do_sys_bpf+0x82e/0x1730^M
> [ 758.223043] ? ep_show_fdinfo+0x80/0x80^M
> [ 758.226885] do_syscall_64+0x5b/0x180^M
> [ 758.230550] entry_SYSCALL_64_after_hwframe+0x44/0xa9^M
> [ 758.235620] RIP: 0033:0x7f08a19e91fd^M
> [ 758.239198] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 5b 8c 0c 00 f7 d8 64 89 01 48^M
>
>
> this seems to be the culprit:
>
> ; ptr = dev->ifalias->rcuhead.next;
> 80: mov 0x0(%rsi),%r14
with rsi being zero. yes. that's the point of the test.
> I used the patch below to bypass the crash, but I guess
> verifier should not let this through
Could you please send me your .config and the way you
run test_progs ?
Is it with or without jit?
I thought I've tested all combinations. Something slipped through.
> also the net_device struct in the test seems outdated
It's not outdated. It's specifically done this way on purpose.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [BUG] bpf: oops in kfree_skb test
[not found] ` <20191030175735.GL20826@krava>
@ 2019-10-30 18:42 ` Alexei Starovoitov
0 siblings, 0 replies; 4+ messages in thread
From: Alexei Starovoitov @ 2019-10-30 18:42 UTC (permalink / raw)
To: Jiri Olsa
Cc: Alexei Starovoitov, David S. Miller, Daniel Borkmann, X86 ML,
Network Development, bpf, Kernel Team
On Wed, Oct 30, 2019 at 10:57 AM Jiri Olsa <jolsa@redhat.com> wrote:
>
> it's jited and I'm just running test_progs
> the kerne is latest bpf-next, config is attached
Reproduced it.
I missed the case where jit is on, but bpf_jit_kallsyms is off.
Please use sysctl net.core.bpf_jit_kallsyms=1 as a workaround.
I'll send a fix shortly.
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2019-10-30 18:42 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-30 15:43 [BUG] bpf: Jiri Olsa
2019-10-30 15:48 ` [BUG] bpf: oops in kfree_skb test Jiri Olsa
2019-10-30 16:44 ` [BUG] bpf: Alexei Starovoitov
[not found] ` <20191030175735.GL20826@krava>
2019-10-30 18:42 ` [BUG] bpf: oops in kfree_skb test Alexei Starovoitov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).