All of lore.kernel.org
 help / color / mirror / Atom feed
* Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-06-22 11:19 ` Peter Robinson
  0 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-06-22 11:19 UTC (permalink / raw)
  To: netdev, linux-arm-kernel; +Cc: labbott

Hi All,

I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
(doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
others, both LPAE/normal kernels.

I'm a bit out of my depth in this part of the kernel but I'm wondering
if it's known, I couldn't find anything that looked obvious on a few
mailing lists.

Peter

[    9.955543] Modules linked in:
[    9.955562] CPU: 1 PID: 213 Comm: systemd-udevd Tainted: G      D
        4.18.0-0.rc1.git0.1.fc29.armv7hl #1
[    9.955566] Hardware name: BCM2835
[    9.955584] PC is at sk_filter_trim_cap+0x15c/0x1b8
[    9.955590] LR is at   (null)
[    9.955597] pc : [<c09d4d58>]    lr : [<00000000>]    psr: 60000013
[    9.955602] sp : c2cf9d58  ip : 00000000  fp : 00000000
[    9.955608] r10: ef2c3c00  r9 : c13093c0  r8 : 00000000
[    9.955615] r7 : 00000000  r6 : 00000001  r5 : f0f6a000  r4 : 00000000
[    9.955621] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
[    9.955629] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[    9.955640] Control: 10c5387d  Table: 02e6406a  DAC: 00000051
[    9.963334] Unable to handle kernel NULL pointer dereference at
virtual address 0000000c
[    9.964631] Process systemd-udevd (pid: 213, stack limit = 0x(ptrval))
[    9.964640] Stack: (0xc2cf9d58 to 0xc2cfa000)
[    9.964649] 9d40:
    00000000 c2c90540
[    9.964663] 9d60: 006000c0 00000000 00000000 c09a233c c2c90b40
c2c90b40 c2c90540 00000000
[    9.964678] 9d80: 00000000 00000000 c13093c0 c09fa2bc 006000c0
00000001 ee7f1800 00000000
[    9.964691] 9da0: 00000002 00000000 00000001 ef2c3c64 c2cf9f70
00000002 c2c90540 00000000
[    9.964706] 9dc0: c2cf9f68 00000083 ee7f1800 00000008 00000000
c09fa3b8 006000c0 00000000
[    9.964724] 9de0: 00000000 00000002 00000002 c09fc704 006000c0
00000000 ee7c7c00 00000000
[    9.976159] pgd = (ptrval)
[    9.979536] 9e00: 000000d5 00000000 00000000 00000000 c126a314
c2cf9f68 eec77880 c2cf9e50
[    9.979550] 9e20: 00000040 00000000 eec77880 00000000 00000000
c099a624 c2cf9f68 00000000
[    9.979565] 9e40: c2cf9e50 c099ae48 00000100 00000000 00000080
c04ab918 ee78e8c0 7fff0000
[    9.979580] 9e60: c2cf9e90 c2cf9eec ffff0000 000000a0 bed817e4
00000028 01a040a8 0000005b
[    9.979594] 9e80: 00000000 00000000 00000000 01a0ef00 00000128
40000028 b6cd9548 00000000
[    9.979607] 9ea0: 0000000d 00000000 bed817b8 00000000 00000010
00000000 00000002 00000000
[    9.985866] [0000000c] *pgd=00000000
[    9.988810] 9ec0: 00000000 00000000 01a0ef00 00000000 c2cf9fb0
00000128 bed817b8 00000000
[    9.988825] 9ee0: 00000000 c0407f18 00000000 00000000 c120bbec
b6e2ba00 c2cf9fb0 10c5387d
[    9.988841] 9f00: 01a0efb8 bed81720 bed81728 c03165fc 00005010
00001000 3e600000 c04ced24
[    9.988855] 9f20: ee4b5010 00000ff0 ee4b5000 00000000 ee4b6000
eec77880 bed817b8 00000000
[    9.988875] 9f40: 00000128 c0301204 c2cf8000 00000128 00000000
c099bc5c 00000000 00000000
[   10.000948] 9f60: 00000000 fffffff7 c2cf9eb0 0000000c 00000001
00000000 00000000 c2cf9e80
[   10.000961] 9f80: 00000000 c030ac08 00000000 00000000 00000040
00000000 00000000 01a0ef00
[   10.000976] 9fa0: bed817b8 c03011d4 00000000 01a0ef00 0000000d
bed817b8 00000000 00000000
[   10.000995] 9fc0: 00000000 01a0ef00 bed817b8 00000128 0000005b
01a0af00 01a0f620 00000000
[   10.228876] 9fe0: b6f9fad4 bed81780 b6de4780 b6cd9548 60000010
0000000d 00000000 00000000
[   10.237081] [<c09d4d58>] (sk_filter_trim_cap) from [<c09fa2bc>]
(netlink_broadcast_filtered+0x304/0x3dc)
[   10.246575] [<c09fa2bc>] (netlink_broadcast_filtered) from
[<c09fa3b8>] (netlink_broadcast+0x24/0x2c)
[   10.255806] [<c09fa3b8>] (netlink_broadcast) from [<c09fc704>]
(netlink_sendmsg+0x30c/0x340)
[   10.264258] [<c09fc704>] (netlink_sendmsg) from [<c099a624>]
(sock_sendmsg+0x3c/0x4c)
[   10.272100] [<c099a624>] (sock_sendmsg) from [<c099ae48>]
(___sys_sendmsg+0x1d8/0x218)
[   10.280030] [<c099ae48>] (___sys_sendmsg) from [<c099bc5c>]
(__sys_sendmsg+0x48/0x6c)
[   10.287872] [<c099bc5c>] (__sys_sendmsg) from [<c03011d4>]
(__sys_trace_return+0x0/0x10)
[   10.295962] Exception stack(0xc2cf9fa8 to 0xc2cf9ff0)
[   10.301018] 9fa0:                   00000000 01a0ef00 0000000d
bed817b8 00000000 00000000
[   10.309202] 9fc0: 00000000 01a0ef00 bed817b8 00000128 0000005b
01a0af00 01a0f620 00000000
[   10.317381] 9fe0: b6f9fad4 bed81780 b6de4780 b6cd9548
[   10.322442] Code: 1afffff7 e59c0000 e5830000 e3520000 (e584800c)
[   10.328557] Internal error: Oops: 805 [#8] SMP ARM
[   10.328768] ---[ end trace 2cb865e83300a747 ]---
[   10.333357] Modules linked in:
[   10.333374] CPU: 2 PID: 212 Comm: systemd-udevd Tainted: G      D
        4.18.0-0.rc1.git0.1.fc29.armv7hl #1
[   10.333378] Hardware name: BCM2835
[   10.333396] PC is at sk_filter_trim_cap+0x15c/0x1b8
[   10.333409] LR is at   (null)
[   10.341840] Unable to handle kernel NULL pointer dereference at
virtual address 0000000c
[   10.351172] pc : [<c09d4d58>]    lr : [<00000000>]    psr: 60000013
[   10.351179] sp : c2e5dd58  ip : 00000000  fp : 00000000
[   10.351185] r10: ef2c3c00  r9 : c13093c0  r8 : 00000000
[   10.351192] r7 : 00000000  r6 : 00000001  r5 : f0f6a000  r4 : 00000000
[   10.351198] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
[   10.351207] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[   10.351215] Control: 10c5387d  Table: 02e6006a  DAC: 00000051
[   10.351231] Process systemd-udevd (pid: 212, stack limit = 0x(ptrval))
[   10.354654] pgd = (ptrval)
[   10.359496] Stack: (0xc2e5dd58 to 0xc2e5e000)
[   10.359505] dd40:
    00000000 ef3c0540
[   10.359520] dd60: 006000c0 00000000 00000000 c09a233c ef3c0b40
ef3c0b40 ef3c0540 00000000
[   10.359534] dd80: 00000000 00000000 c13093c0 c09fa2bc 006000c0
00000001 ee7f2000 00000000
[   10.359548] dda0: 00000002 00000000 00000001 ef2c3c64 c2e5df70
00000002 ef3c0540 00000000
[   10.359563] ddc0: c2e5df68 00000084 ee7f2000 00000008 00000000
c09fa3b8 006000c0 00000000
[   10.359585] dde0: 00000000 00000002 00000002 c09fc704 006000c0
00000000 c2c68d00 00000000
[   10.362574] [0000000c] *pgd=00000000
[   10.382706] de00: 000000d4 00000000 00000000 00000000 c126a314
c2e5df68 eec76c40 c2e5de50
[   10.382721] de20: 00000040 00000000 eec76c40 00000000 00000000
c099a624 c2e5df68 00000000
[   10.382735] de40: c2e5de50 c099ae48 00000100 00000000 00000080
c04ab918 ee78e8c0 7fff0000
[   10.382750] de60: c2e5de90 c2e5deec ffff0000 000000a0 bed817e4
00000028 01a040a8 0000005c
[   10.382764] de80: 00000000 00000000 00000000 01a0e0f0 00000128
40000028 b6cd9548 00000000
[   10.382780] dea0: 0000000d 00000000 bed817b8 00000000 00000010
00000000 00000002 00000000
[   10.397129] dec0: 00000000 00000000 01a0e0f0 00000000 c2e5dfb0
00000128 bed817b8 00000000
[   10.397144] dee0: 00000000 c0407f18 00000000 00000000 c120bbec
b6e2ba00 c2e5dfb0 10c5387d
[   10.397159] df00: 01a0e1a8 bed81720 bed81728 c03165fc 00006010
00001000 3e600000 c04ced24
[   10.397174] df20: ef216010 00000ff0 ef216000 00000000 ef217000
eec76c40 bed817b8 00000000
[   10.397189] df40: 00000128 c0301204 c2e5c000 00000128 00000000
c099bc5c 00000000 00000000
[   10.589571] df60: 00000000 fffffff7 c2e5deb0 0000000c 00000001
00000000 00000000 c2e5de80
[   10.589596] df80: 00000000 c030ac08 00000000 00000000 00000040
00000000 00000000 01a0e0f0
[   10.605946] dfa0: bed817b8 c03011d4 00000000 01a0e0f0 0000000d
bed817b8 00000000 00000000
[   10.614131] dfc0: 00000000 01a0e0f0 bed817b8 00000128 0000005c
01a0af00 01a0e920 00000000
[   10.622316] dfe0: b6f9fad4 bed81780 b6de4780 b6cd9548 60000010
0000000d 00000000 00000000
[   10.630594] [<c09d4d58>] (sk_filter_trim_cap) from [<c09fa2bc>]
(netlink_broadcast_filtered+0x304/0x3dc)
[   10.640088] [<c09fa2bc>] (netlink_broadcast_filtered) from
[<c09fa3b8>] (netlink_broadcast+0x24/0x2c)
[   10.650447] [<c09fa3b8>] (netlink_broadcast) from [<c09fc704>]
(netlink_sendmsg+0x30c/0x340)
[   10.658899] [<c09fc704>] (netlink_sendmsg) from [<c099a624>]
(sock_sendmsg+0x3c/0x4c)
[   10.666742] [<c099a624>] (sock_sendmsg) from [<c099ae48>]
(___sys_sendmsg+0x1d8/0x218)
[   10.674673] [<c099ae48>] (___sys_sendmsg) from [<c099bc5c>]
(__sys_sendmsg+0x48/0x6c)
[   10.682515] [<c099bc5c>] (__sys_sendmsg) from [<c03011d4>]
(__sys_trace_return+0x0/0x10)
[   10.690604] Exception stack(0xc2e5dfa8 to 0xc2e5dff0)
[   10.695660] dfa0:                   00000000 01a0e0f0 0000000d
bed817b8 00000000 00000000
[   10.703845] dfc0: 00000000 01a0e0f0 bed817b8 00000128 0000005c
01a0af00 01a0e920 00000000
[   10.712025] dfe0: b6f9fad4 bed81780 b6de4780 b6cd9548
[   10.717086] Code: 1afffff7 e59c0000 e5830000 e3520000 (e584800c)
[   10.723199] Internal error: Oops: 805 [#9] SMP ARM
[   10.723343] ---[ end trace 2cb865e83300a748 ]---

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-06-22 11:19 ` Peter Robinson
  0 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-06-22 11:19 UTC (permalink / raw)
  To: linux-arm-kernel

Hi All,

I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
(doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
others, both LPAE/normal kernels.

I'm a bit out of my depth in this part of the kernel but I'm wondering
if it's known, I couldn't find anything that looked obvious on a few
mailing lists.

Peter

[    9.955543] Modules linked in:
[    9.955562] CPU: 1 PID: 213 Comm: systemd-udevd Tainted: G      D
        4.18.0-0.rc1.git0.1.fc29.armv7hl #1
[    9.955566] Hardware name: BCM2835
[    9.955584] PC is at sk_filter_trim_cap+0x15c/0x1b8
[    9.955590] LR is at   (null)
[    9.955597] pc : [<c09d4d58>]    lr : [<00000000>]    psr: 60000013
[    9.955602] sp : c2cf9d58  ip : 00000000  fp : 00000000
[    9.955608] r10: ef2c3c00  r9 : c13093c0  r8 : 00000000
[    9.955615] r7 : 00000000  r6 : 00000001  r5 : f0f6a000  r4 : 00000000
[    9.955621] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
[    9.955629] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[    9.955640] Control: 10c5387d  Table: 02e6406a  DAC: 00000051
[    9.963334] Unable to handle kernel NULL pointer dereference at
virtual address 0000000c
[    9.964631] Process systemd-udevd (pid: 213, stack limit = 0x(ptrval))
[    9.964640] Stack: (0xc2cf9d58 to 0xc2cfa000)
[    9.964649] 9d40:
    00000000 c2c90540
[    9.964663] 9d60: 006000c0 00000000 00000000 c09a233c c2c90b40
c2c90b40 c2c90540 00000000
[    9.964678] 9d80: 00000000 00000000 c13093c0 c09fa2bc 006000c0
00000001 ee7f1800 00000000
[    9.964691] 9da0: 00000002 00000000 00000001 ef2c3c64 c2cf9f70
00000002 c2c90540 00000000
[    9.964706] 9dc0: c2cf9f68 00000083 ee7f1800 00000008 00000000
c09fa3b8 006000c0 00000000
[    9.964724] 9de0: 00000000 00000002 00000002 c09fc704 006000c0
00000000 ee7c7c00 00000000
[    9.976159] pgd = (ptrval)
[    9.979536] 9e00: 000000d5 00000000 00000000 00000000 c126a314
c2cf9f68 eec77880 c2cf9e50
[    9.979550] 9e20: 00000040 00000000 eec77880 00000000 00000000
c099a624 c2cf9f68 00000000
[    9.979565] 9e40: c2cf9e50 c099ae48 00000100 00000000 00000080
c04ab918 ee78e8c0 7fff0000
[    9.979580] 9e60: c2cf9e90 c2cf9eec ffff0000 000000a0 bed817e4
00000028 01a040a8 0000005b
[    9.979594] 9e80: 00000000 00000000 00000000 01a0ef00 00000128
40000028 b6cd9548 00000000
[    9.979607] 9ea0: 0000000d 00000000 bed817b8 00000000 00000010
00000000 00000002 00000000
[    9.985866] [0000000c] *pgd=00000000
[    9.988810] 9ec0: 00000000 00000000 01a0ef00 00000000 c2cf9fb0
00000128 bed817b8 00000000
[    9.988825] 9ee0: 00000000 c0407f18 00000000 00000000 c120bbec
b6e2ba00 c2cf9fb0 10c5387d
[    9.988841] 9f00: 01a0efb8 bed81720 bed81728 c03165fc 00005010
00001000 3e600000 c04ced24
[    9.988855] 9f20: ee4b5010 00000ff0 ee4b5000 00000000 ee4b6000
eec77880 bed817b8 00000000
[    9.988875] 9f40: 00000128 c0301204 c2cf8000 00000128 00000000
c099bc5c 00000000 00000000
[   10.000948] 9f60: 00000000 fffffff7 c2cf9eb0 0000000c 00000001
00000000 00000000 c2cf9e80
[   10.000961] 9f80: 00000000 c030ac08 00000000 00000000 00000040
00000000 00000000 01a0ef00
[   10.000976] 9fa0: bed817b8 c03011d4 00000000 01a0ef00 0000000d
bed817b8 00000000 00000000
[   10.000995] 9fc0: 00000000 01a0ef00 bed817b8 00000128 0000005b
01a0af00 01a0f620 00000000
[   10.228876] 9fe0: b6f9fad4 bed81780 b6de4780 b6cd9548 60000010
0000000d 00000000 00000000
[   10.237081] [<c09d4d58>] (sk_filter_trim_cap) from [<c09fa2bc>]
(netlink_broadcast_filtered+0x304/0x3dc)
[   10.246575] [<c09fa2bc>] (netlink_broadcast_filtered) from
[<c09fa3b8>] (netlink_broadcast+0x24/0x2c)
[   10.255806] [<c09fa3b8>] (netlink_broadcast) from [<c09fc704>]
(netlink_sendmsg+0x30c/0x340)
[   10.264258] [<c09fc704>] (netlink_sendmsg) from [<c099a624>]
(sock_sendmsg+0x3c/0x4c)
[   10.272100] [<c099a624>] (sock_sendmsg) from [<c099ae48>]
(___sys_sendmsg+0x1d8/0x218)
[   10.280030] [<c099ae48>] (___sys_sendmsg) from [<c099bc5c>]
(__sys_sendmsg+0x48/0x6c)
[   10.287872] [<c099bc5c>] (__sys_sendmsg) from [<c03011d4>]
(__sys_trace_return+0x0/0x10)
[   10.295962] Exception stack(0xc2cf9fa8 to 0xc2cf9ff0)
[   10.301018] 9fa0:                   00000000 01a0ef00 0000000d
bed817b8 00000000 00000000
[   10.309202] 9fc0: 00000000 01a0ef00 bed817b8 00000128 0000005b
01a0af00 01a0f620 00000000
[   10.317381] 9fe0: b6f9fad4 bed81780 b6de4780 b6cd9548
[   10.322442] Code: 1afffff7 e59c0000 e5830000 e3520000 (e584800c)
[   10.328557] Internal error: Oops: 805 [#8] SMP ARM
[   10.328768] ---[ end trace 2cb865e83300a747 ]---
[   10.333357] Modules linked in:
[   10.333374] CPU: 2 PID: 212 Comm: systemd-udevd Tainted: G      D
        4.18.0-0.rc1.git0.1.fc29.armv7hl #1
[   10.333378] Hardware name: BCM2835
[   10.333396] PC is at sk_filter_trim_cap+0x15c/0x1b8
[   10.333409] LR is at   (null)
[   10.341840] Unable to handle kernel NULL pointer dereference at
virtual address 0000000c
[   10.351172] pc : [<c09d4d58>]    lr : [<00000000>]    psr: 60000013
[   10.351179] sp : c2e5dd58  ip : 00000000  fp : 00000000
[   10.351185] r10: ef2c3c00  r9 : c13093c0  r8 : 00000000
[   10.351192] r7 : 00000000  r6 : 00000001  r5 : f0f6a000  r4 : 00000000
[   10.351198] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
[   10.351207] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[   10.351215] Control: 10c5387d  Table: 02e6006a  DAC: 00000051
[   10.351231] Process systemd-udevd (pid: 212, stack limit = 0x(ptrval))
[   10.354654] pgd = (ptrval)
[   10.359496] Stack: (0xc2e5dd58 to 0xc2e5e000)
[   10.359505] dd40:
    00000000 ef3c0540
[   10.359520] dd60: 006000c0 00000000 00000000 c09a233c ef3c0b40
ef3c0b40 ef3c0540 00000000
[   10.359534] dd80: 00000000 00000000 c13093c0 c09fa2bc 006000c0
00000001 ee7f2000 00000000
[   10.359548] dda0: 00000002 00000000 00000001 ef2c3c64 c2e5df70
00000002 ef3c0540 00000000
[   10.359563] ddc0: c2e5df68 00000084 ee7f2000 00000008 00000000
c09fa3b8 006000c0 00000000
[   10.359585] dde0: 00000000 00000002 00000002 c09fc704 006000c0
00000000 c2c68d00 00000000
[   10.362574] [0000000c] *pgd=00000000
[   10.382706] de00: 000000d4 00000000 00000000 00000000 c126a314
c2e5df68 eec76c40 c2e5de50
[   10.382721] de20: 00000040 00000000 eec76c40 00000000 00000000
c099a624 c2e5df68 00000000
[   10.382735] de40: c2e5de50 c099ae48 00000100 00000000 00000080
c04ab918 ee78e8c0 7fff0000
[   10.382750] de60: c2e5de90 c2e5deec ffff0000 000000a0 bed817e4
00000028 01a040a8 0000005c
[   10.382764] de80: 00000000 00000000 00000000 01a0e0f0 00000128
40000028 b6cd9548 00000000
[   10.382780] dea0: 0000000d 00000000 bed817b8 00000000 00000010
00000000 00000002 00000000
[   10.397129] dec0: 00000000 00000000 01a0e0f0 00000000 c2e5dfb0
00000128 bed817b8 00000000
[   10.397144] dee0: 00000000 c0407f18 00000000 00000000 c120bbec
b6e2ba00 c2e5dfb0 10c5387d
[   10.397159] df00: 01a0e1a8 bed81720 bed81728 c03165fc 00006010
00001000 3e600000 c04ced24
[   10.397174] df20: ef216010 00000ff0 ef216000 00000000 ef217000
eec76c40 bed817b8 00000000
[   10.397189] df40: 00000128 c0301204 c2e5c000 00000128 00000000
c099bc5c 00000000 00000000
[   10.589571] df60: 00000000 fffffff7 c2e5deb0 0000000c 00000001
00000000 00000000 c2e5de80
[   10.589596] df80: 00000000 c030ac08 00000000 00000000 00000040
00000000 00000000 01a0e0f0
[   10.605946] dfa0: bed817b8 c03011d4 00000000 01a0e0f0 0000000d
bed817b8 00000000 00000000
[   10.614131] dfc0: 00000000 01a0e0f0 bed817b8 00000128 0000005c
01a0af00 01a0e920 00000000
[   10.622316] dfe0: b6f9fad4 bed81780 b6de4780 b6cd9548 60000010
0000000d 00000000 00000000
[   10.630594] [<c09d4d58>] (sk_filter_trim_cap) from [<c09fa2bc>]
(netlink_broadcast_filtered+0x304/0x3dc)
[   10.640088] [<c09fa2bc>] (netlink_broadcast_filtered) from
[<c09fa3b8>] (netlink_broadcast+0x24/0x2c)
[   10.650447] [<c09fa3b8>] (netlink_broadcast) from [<c09fc704>]
(netlink_sendmsg+0x30c/0x340)
[   10.658899] [<c09fc704>] (netlink_sendmsg) from [<c099a624>]
(sock_sendmsg+0x3c/0x4c)
[   10.666742] [<c099a624>] (sock_sendmsg) from [<c099ae48>]
(___sys_sendmsg+0x1d8/0x218)
[   10.674673] [<c099ae48>] (___sys_sendmsg) from [<c099bc5c>]
(__sys_sendmsg+0x48/0x6c)
[   10.682515] [<c099bc5c>] (__sys_sendmsg) from [<c03011d4>]
(__sys_trace_return+0x0/0x10)
[   10.690604] Exception stack(0xc2e5dfa8 to 0xc2e5dff0)
[   10.695660] dfa0:                   00000000 01a0e0f0 0000000d
bed817b8 00000000 00000000
[   10.703845] dfc0: 00000000 01a0e0f0 bed817b8 00000128 0000005c
01a0af00 01a0e920 00000000
[   10.712025] dfe0: b6f9fad4 bed81780 b6de4780 b6cd9548
[   10.717086] Code: 1afffff7 e59c0000 e5830000 e3520000 (e584800c)
[   10.723199] Internal error: Oops: 805 [#9] SMP ARM
[   10.723343] ---[ end trace 2cb865e83300a748 ]---

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-06-22 11:19 ` Peter Robinson
@ 2018-06-22 12:55   ` Eric Dumazet
  -1 siblings, 0 replies; 50+ messages in thread
From: Eric Dumazet @ 2018-06-22 12:55 UTC (permalink / raw)
  To: Peter Robinson, netdev, linux-arm-kernel; +Cc: labbott



On 06/22/2018 04:19 AM, Peter Robinson wrote:
> Hi All,
> 
> I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
> a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
> (doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
> others, both LPAE/normal kernels.
> 
> I'm a bit out of my depth in this part of the kernel but I'm wondering
> if it's known, I couldn't find anything that looked obvious on a few
> mailing lists.
> 
> Peter

Hi Peter

Could you provide symbolic information ?

Thanks !

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-06-22 12:55   ` Eric Dumazet
  0 siblings, 0 replies; 50+ messages in thread
From: Eric Dumazet @ 2018-06-22 12:55 UTC (permalink / raw)
  To: linux-arm-kernel



On 06/22/2018 04:19 AM, Peter Robinson wrote:
> Hi All,
> 
> I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
> a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
> (doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
> others, both LPAE/normal kernels.
> 
> I'm a bit out of my depth in this part of the kernel but I'm wondering
> if it's known, I couldn't find anything that looked obvious on a few
> mailing lists.
> 
> Peter

Hi Peter

Could you provide symbolic information ?

Thanks !

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-06-22 12:55   ` Eric Dumazet
@ 2018-06-24  9:24     ` Peter Robinson
  -1 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-06-24  9:24 UTC (permalink / raw)
  To: Eric Dumazet; +Cc: netdev, linux-arm-kernel, labbott

>> I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
>> a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
>> (doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
>> others, both LPAE/normal kernels.
>>
>> I'm a bit out of my depth in this part of the kernel but I'm wondering
>> if it's known, I couldn't find anything that looked obvious on a few
>> mailing lists.
>>
>> Peter
>
> Hi Peter
>
> Could you provide symbolic information ?

I passed in through scripts/decode_stacktrace.sh is that what you were after:

[    8.673880] Internal error: Oops: a06 [#10] SMP ARM
[    8.673949] ---[ end trace 049df4786ea3140a ]---
[    8.678754] Modules linked in:
[    8.678766] CPU: 1 PID: 206 Comm: systemd-udevd Tainted: G      D
        4.18.0-0.rc1.git0.1.fc29.armv7hl+lpae #1
[    8.678769] Hardware name: Allwinner sun8i Family
[    8.678781] PC is at sk_filter_trim_cap ()
[    8.678790] LR is at   (null)
[    8.709463] pc : lr : psr: 60000013 ()
[    8.715722] sp : c996bd60  ip : 00000000  fp : 00000000
[    8.720939] r10: ee79dc00  r9 : c12c9f80  r8 : 00000000
[    8.726157] r7 : 00000000  r6 : 00000001  r5 : f1648000  r4 : 00000000
[    8.732674] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
[    8.739193] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[    8.746318] Control: 30c5387d  Table: 6e7bc880  DAC: ffe75ece
[    8.752055] Process systemd-udevd (pid: 206, stack limit = 0x(ptrval))
[    8.758574] Stack: (0xc996bd60 to 0xc996c000)
[    8.762929] bd60: 00000000 ee7ad0c0 006000c0 00000000 00000000
c0a64ab8 ee7ad240 ee7ad240
[    8.771098] bd80: ee7ad0c0 00000000 00000000 00000000 c12c9f80
c0abbb8c ef001a00 00000001
[    8.779267] bda0: ee722400 00000000 00000002 00000000 00000001
ee79dc64 c996bf70 00000002
[    8.787435] bdc0: ee7ad0c0 00000000 c996bf68 0000008b ee722400
00000008 00000000 c0abbc88
[    8.795604] bde0: 006000c0 00000000 00000000 00000002 00000002
c0abdfb0 006000c0 00000000
[    8.803772] be00: c98ce580 00000000 000000ce 00000000 00000000
00000000 c124ebf4 c996bf68
[    8.811941] be20: eead4c40 c996be58 00000040 00000000 eead4c40
00000000 00000000 c0a5d198
[    8.820110] be40: c996bf68 00000000 c996be58 c0a5d958 00000000
00000000 ee78c2c0 7fff0000
[    8.828278] be60: c996be90 c996beec ffff0000 000000a0 00000000
c05103ac bef897e4 00000028
[    8.836447] be80: 004ee0a8 00000063 00000000 004f3820 00000128
40000028 b6c9a548 00000000
[    8.844615] bea0: 0000000d 00000000 bef897b8 00000000 00000000
00000000 00000010 00000000
[    8.852784] bec0: 00000002 00000000 004f3820 00000000 c996bfb0
00000128 bef897b8 00000000
[    8.860953] bee0: 00000000 c0510450 00000000 00000000 c120eaa4
b6deca00 c996bfb0 30c5387d
[    8.869122] bf00: 004f38d8 bef89720 bef89728 c0434e94 00000000
c05e0290 ee4e6010 00000ff0
[    8.877291] bf20: ee4e6010 00000ff0 ee4e6000 00000000 00000000
c0506354 eead4c40 bef897b8
[    8.885460] bf40: 00000000 00000128 c0401324 c996a000 00000128
c0a5e6d4 00000000 00000000
[    8.893628] bf60: 00000000 fffffff7 c996beb8 0000000c 00000001
00000000 00000000 c996be88
[    8.901796] bf80: 00000000 c0429ac0 00000000 00000000 00000040
00000000 00000000 004f3820
[    8.909965] bfa0: bef897b8 c04012e8 00000000 004f3820 0000000d
bef897b8 00000000 00000000
[    8.918134] bfc0: 00000000 004f3820 bef897b8 00000128 00000063
004eae70 004f4078 00000000
[    8.926302] bfe0: b6f60ad4 bef89780 b6da5780 b6c9a548 60000010
0000000d 00000000 00000000
[    8.934488] (sk_filter_trim_cap) from netlink_broadcast_filtered ()
[    8.943963] (netlink_broadcast_filtered) from netlink_broadcast ()
[    8.953174] (netlink_broadcast) from netlink_sendmsg ()
[    8.961608] (netlink_sendmsg) from sock_sendmsg ()
[    8.969432] (sock_sendmsg) from ___sys_sendmsg ()
[    8.977343] (___sys_sendmsg) from __sys_sendmsg ()
[    8.985170] (__sys_sendmsg) from __sys_trace_return ()
[    8.993247] Exception stack(0xc996bfa8 to 0xc996bff0)
[    8.998294] bfa0:                   00000000 004f3820 0000000d
bef897b8 00000000 00000000
[    9.006463] bfc0: 00000000 004f3820 bef897b8 00000128 00000063
004eae70 004f4078 00000000
[    9.014629] bfe0: b6f60ad4 bef89780 b6da5780 b6c9a548
[ 9.019680] Code: 1afffff7 e59c0000 e5830000 e3520000 (e584800c)
All code
========
   0:   1afffff7        .word   0x1afffff7
   4:   e59c0000        .word   0xe59c0000
   8:   e5830000        .word   0xe5830000
   c:   e3520000        .word   0xe3520000
  10:*  e584800c        .word   0xe584800c              <-- trapping instruction

Code starting with the faulting instruction
===========================================
   0:   e584800c        .word   0xe584800c
[    9.025823] ---[ end trace 049df4786ea3140b ]---

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-06-24  9:24     ` Peter Robinson
  0 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-06-24  9:24 UTC (permalink / raw)
  To: linux-arm-kernel

>> I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
>> a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
>> (doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
>> others, both LPAE/normal kernels.
>>
>> I'm a bit out of my depth in this part of the kernel but I'm wondering
>> if it's known, I couldn't find anything that looked obvious on a few
>> mailing lists.
>>
>> Peter
>
> Hi Peter
>
> Could you provide symbolic information ?

I passed in through scripts/decode_stacktrace.sh is that what you were after:

[    8.673880] Internal error: Oops: a06 [#10] SMP ARM
[    8.673949] ---[ end trace 049df4786ea3140a ]---
[    8.678754] Modules linked in:
[    8.678766] CPU: 1 PID: 206 Comm: systemd-udevd Tainted: G      D
        4.18.0-0.rc1.git0.1.fc29.armv7hl+lpae #1
[    8.678769] Hardware name: Allwinner sun8i Family
[    8.678781] PC is at sk_filter_trim_cap ()
[    8.678790] LR is at   (null)
[    8.709463] pc : lr : psr: 60000013 ()
[    8.715722] sp : c996bd60  ip : 00000000  fp : 00000000
[    8.720939] r10: ee79dc00  r9 : c12c9f80  r8 : 00000000
[    8.726157] r7 : 00000000  r6 : 00000001  r5 : f1648000  r4 : 00000000
[    8.732674] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
[    8.739193] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[    8.746318] Control: 30c5387d  Table: 6e7bc880  DAC: ffe75ece
[    8.752055] Process systemd-udevd (pid: 206, stack limit = 0x(ptrval))
[    8.758574] Stack: (0xc996bd60 to 0xc996c000)
[    8.762929] bd60: 00000000 ee7ad0c0 006000c0 00000000 00000000
c0a64ab8 ee7ad240 ee7ad240
[    8.771098] bd80: ee7ad0c0 00000000 00000000 00000000 c12c9f80
c0abbb8c ef001a00 00000001
[    8.779267] bda0: ee722400 00000000 00000002 00000000 00000001
ee79dc64 c996bf70 00000002
[    8.787435] bdc0: ee7ad0c0 00000000 c996bf68 0000008b ee722400
00000008 00000000 c0abbc88
[    8.795604] bde0: 006000c0 00000000 00000000 00000002 00000002
c0abdfb0 006000c0 00000000
[    8.803772] be00: c98ce580 00000000 000000ce 00000000 00000000
00000000 c124ebf4 c996bf68
[    8.811941] be20: eead4c40 c996be58 00000040 00000000 eead4c40
00000000 00000000 c0a5d198
[    8.820110] be40: c996bf68 00000000 c996be58 c0a5d958 00000000
00000000 ee78c2c0 7fff0000
[    8.828278] be60: c996be90 c996beec ffff0000 000000a0 00000000
c05103ac bef897e4 00000028
[    8.836447] be80: 004ee0a8 00000063 00000000 004f3820 00000128
40000028 b6c9a548 00000000
[    8.844615] bea0: 0000000d 00000000 bef897b8 00000000 00000000
00000000 00000010 00000000
[    8.852784] bec0: 00000002 00000000 004f3820 00000000 c996bfb0
00000128 bef897b8 00000000
[    8.860953] bee0: 00000000 c0510450 00000000 00000000 c120eaa4
b6deca00 c996bfb0 30c5387d
[    8.869122] bf00: 004f38d8 bef89720 bef89728 c0434e94 00000000
c05e0290 ee4e6010 00000ff0
[    8.877291] bf20: ee4e6010 00000ff0 ee4e6000 00000000 00000000
c0506354 eead4c40 bef897b8
[    8.885460] bf40: 00000000 00000128 c0401324 c996a000 00000128
c0a5e6d4 00000000 00000000
[    8.893628] bf60: 00000000 fffffff7 c996beb8 0000000c 00000001
00000000 00000000 c996be88
[    8.901796] bf80: 00000000 c0429ac0 00000000 00000000 00000040
00000000 00000000 004f3820
[    8.909965] bfa0: bef897b8 c04012e8 00000000 004f3820 0000000d
bef897b8 00000000 00000000
[    8.918134] bfc0: 00000000 004f3820 bef897b8 00000128 00000063
004eae70 004f4078 00000000
[    8.926302] bfe0: b6f60ad4 bef89780 b6da5780 b6c9a548 60000010
0000000d 00000000 00000000
[    8.934488] (sk_filter_trim_cap) from netlink_broadcast_filtered ()
[    8.943963] (netlink_broadcast_filtered) from netlink_broadcast ()
[    8.953174] (netlink_broadcast) from netlink_sendmsg ()
[    8.961608] (netlink_sendmsg) from sock_sendmsg ()
[    8.969432] (sock_sendmsg) from ___sys_sendmsg ()
[    8.977343] (___sys_sendmsg) from __sys_sendmsg ()
[    8.985170] (__sys_sendmsg) from __sys_trace_return ()
[    8.993247] Exception stack(0xc996bfa8 to 0xc996bff0)
[    8.998294] bfa0:                   00000000 004f3820 0000000d
bef897b8 00000000 00000000
[    9.006463] bfc0: 00000000 004f3820 bef897b8 00000128 00000063
004eae70 004f4078 00000000
[    9.014629] bfe0: b6f60ad4 bef89780 b6da5780 b6c9a548
[ 9.019680] Code: 1afffff7 e59c0000 e5830000 e3520000 (e584800c)
All code
========
   0:   1afffff7        .word   0x1afffff7
   4:   e59c0000        .word   0xe59c0000
   8:   e5830000        .word   0xe5830000
   c:   e3520000        .word   0xe3520000
  10:*  e584800c        .word   0xe584800c              <-- trapping instruction

Code starting with the faulting instruction
===========================================
   0:   e584800c        .word   0xe584800c
[    9.025823] ---[ end trace 049df4786ea3140b ]---

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-06-24  9:24     ` Peter Robinson
@ 2018-06-25  8:48       ` Daniel Borkmann
  -1 siblings, 0 replies; 50+ messages in thread
From: Daniel Borkmann @ 2018-06-25  8:48 UTC (permalink / raw)
  To: Peter Robinson, Eric Dumazet; +Cc: netdev, linux-arm-kernel, labbott

On 06/24/2018 11:24 AM, Peter Robinson wrote:
>>> I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
>>> a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
>>> (doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
>>> others, both LPAE/normal kernels.
>>>
>>> I'm a bit out of my depth in this part of the kernel but I'm wondering
>>> if it's known, I couldn't find anything that looked obvious on a few
>>> mailing lists.
>>>
>>> Peter
>>
>> Hi Peter
>>
>> Could you provide symbolic information ?
> 
> I passed in through scripts/decode_stacktrace.sh is that what you were after:
> 
> [    8.673880] Internal error: Oops: a06 [#10] SMP ARM
> [    8.673949] ---[ end trace 049df4786ea3140a ]---
> [    8.678754] Modules linked in:
> [    8.678766] CPU: 1 PID: 206 Comm: systemd-udevd Tainted: G      D
>         4.18.0-0.rc1.git0.1.fc29.armv7hl+lpae #1
> [    8.678769] Hardware name: Allwinner sun8i Family
> [    8.678781] PC is at sk_filter_trim_cap ()
> [    8.678790] LR is at   (null)
> [    8.709463] pc : lr : psr: 60000013 ()
> [    8.715722] sp : c996bd60  ip : 00000000  fp : 00000000
> [    8.720939] r10: ee79dc00  r9 : c12c9f80  r8 : 00000000
> [    8.726157] r7 : 00000000  r6 : 00000001  r5 : f1648000  r4 : 00000000
> [    8.732674] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
> [    8.739193] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
> [    8.746318] Control: 30c5387d  Table: 6e7bc880  DAC: ffe75ece
> [    8.752055] Process systemd-udevd (pid: 206, stack limit = 0x(ptrval))
> [    8.758574] Stack: (0xc996bd60 to 0xc996c000)
[...]

Should be fixed by (PR to Linus with fix is pending):

https://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git/commit/?id=9262478220eac908ae6e168c3df2c453c87e2da3

Sorry for that,
Daniel

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-06-25  8:48       ` Daniel Borkmann
  0 siblings, 0 replies; 50+ messages in thread
From: Daniel Borkmann @ 2018-06-25  8:48 UTC (permalink / raw)
  To: linux-arm-kernel

On 06/24/2018 11:24 AM, Peter Robinson wrote:
>>> I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
>>> a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
>>> (doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
>>> others, both LPAE/normal kernels.
>>>
>>> I'm a bit out of my depth in this part of the kernel but I'm wondering
>>> if it's known, I couldn't find anything that looked obvious on a few
>>> mailing lists.
>>>
>>> Peter
>>
>> Hi Peter
>>
>> Could you provide symbolic information ?
> 
> I passed in through scripts/decode_stacktrace.sh is that what you were after:
> 
> [    8.673880] Internal error: Oops: a06 [#10] SMP ARM
> [    8.673949] ---[ end trace 049df4786ea3140a ]---
> [    8.678754] Modules linked in:
> [    8.678766] CPU: 1 PID: 206 Comm: systemd-udevd Tainted: G      D
>         4.18.0-0.rc1.git0.1.fc29.armv7hl+lpae #1
> [    8.678769] Hardware name: Allwinner sun8i Family
> [    8.678781] PC is at sk_filter_trim_cap ()
> [    8.678790] LR is at   (null)
> [    8.709463] pc : lr : psr: 60000013 ()
> [    8.715722] sp : c996bd60  ip : 00000000  fp : 00000000
> [    8.720939] r10: ee79dc00  r9 : c12c9f80  r8 : 00000000
> [    8.726157] r7 : 00000000  r6 : 00000001  r5 : f1648000  r4 : 00000000
> [    8.732674] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
> [    8.739193] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
> [    8.746318] Control: 30c5387d  Table: 6e7bc880  DAC: ffe75ece
> [    8.752055] Process systemd-udevd (pid: 206, stack limit = 0x(ptrval))
> [    8.758574] Stack: (0xc996bd60 to 0xc996c000)
[...]

Should be fixed by (PR to Linus with fix is pending):

https://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git/commit/?id=9262478220eac908ae6e168c3df2c453c87e2da3

Sorry for that,
Daniel

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-06-25  8:48       ` Daniel Borkmann
@ 2018-06-25 12:03         ` Peter Robinson
  -1 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-06-25 12:03 UTC (permalink / raw)
  To: Daniel Borkmann; +Cc: Eric Dumazet, netdev, linux-arm-kernel, labbott

On Mon, Jun 25, 2018 at 9:48 AM, Daniel Borkmann <daniel@iogearbox.net> wrote:
> On 06/24/2018 11:24 AM, Peter Robinson wrote:
>>>> I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
>>>> a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
>>>> (doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
>>>> others, both LPAE/normal kernels.
>>>>
>>>> I'm a bit out of my depth in this part of the kernel but I'm wondering
>>>> if it's known, I couldn't find anything that looked obvious on a few
>>>> mailing lists.
>>>>
>>>> Peter
>>>
>>> Hi Peter
>>>
>>> Could you provide symbolic information ?
>>
>> I passed in through scripts/decode_stacktrace.sh is that what you were after:
>>
>> [    8.673880] Internal error: Oops: a06 [#10] SMP ARM
>> [    8.673949] ---[ end trace 049df4786ea3140a ]---
>> [    8.678754] Modules linked in:
>> [    8.678766] CPU: 1 PID: 206 Comm: systemd-udevd Tainted: G      D
>>         4.18.0-0.rc1.git0.1.fc29.armv7hl+lpae #1
>> [    8.678769] Hardware name: Allwinner sun8i Family
>> [    8.678781] PC is at sk_filter_trim_cap ()
>> [    8.678790] LR is at   (null)
>> [    8.709463] pc : lr : psr: 60000013 ()
>> [    8.715722] sp : c996bd60  ip : 00000000  fp : 00000000
>> [    8.720939] r10: ee79dc00  r9 : c12c9f80  r8 : 00000000
>> [    8.726157] r7 : 00000000  r6 : 00000001  r5 : f1648000  r4 : 00000000
>> [    8.732674] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
>> [    8.739193] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
>> [    8.746318] Control: 30c5387d  Table: 6e7bc880  DAC: ffe75ece
>> [    8.752055] Process systemd-udevd (pid: 206, stack limit = 0x(ptrval))
>> [    8.758574] Stack: (0xc996bd60 to 0xc996c000)
> [...]
>
> Should be fixed by (PR to Linus with fix is pending):
>
> https://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git/commit/?id=9262478220eac908ae6e168c3df2c453c87e2da3

Unfortunately it's not, building against the git checkout of the first
failed Fedora kernel (see rc2 issue below) it has the same effect :-(

I thought it might have been [1] because it touches bits of that code
but if I trying with rc2 and that reverted I got no output at all,
checking the vanilla Fedora build from friday (so almost rc2) it
doesn't boot at all either so I've got a second thing to investigate.

Peter

[1] https://lkml.org/lkml/2018/4/29/30

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-06-25 12:03         ` Peter Robinson
  0 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-06-25 12:03 UTC (permalink / raw)
  To: linux-arm-kernel

On Mon, Jun 25, 2018 at 9:48 AM, Daniel Borkmann <daniel@iogearbox.net> wrote:
> On 06/24/2018 11:24 AM, Peter Robinson wrote:
>>>> I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
>>>> a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
>>>> (doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
>>>> others, both LPAE/normal kernels.
>>>>
>>>> I'm a bit out of my depth in this part of the kernel but I'm wondering
>>>> if it's known, I couldn't find anything that looked obvious on a few
>>>> mailing lists.
>>>>
>>>> Peter
>>>
>>> Hi Peter
>>>
>>> Could you provide symbolic information ?
>>
>> I passed in through scripts/decode_stacktrace.sh is that what you were after:
>>
>> [    8.673880] Internal error: Oops: a06 [#10] SMP ARM
>> [    8.673949] ---[ end trace 049df4786ea3140a ]---
>> [    8.678754] Modules linked in:
>> [    8.678766] CPU: 1 PID: 206 Comm: systemd-udevd Tainted: G      D
>>         4.18.0-0.rc1.git0.1.fc29.armv7hl+lpae #1
>> [    8.678769] Hardware name: Allwinner sun8i Family
>> [    8.678781] PC is at sk_filter_trim_cap ()
>> [    8.678790] LR is at   (null)
>> [    8.709463] pc : lr : psr: 60000013 ()
>> [    8.715722] sp : c996bd60  ip : 00000000  fp : 00000000
>> [    8.720939] r10: ee79dc00  r9 : c12c9f80  r8 : 00000000
>> [    8.726157] r7 : 00000000  r6 : 00000001  r5 : f1648000  r4 : 00000000
>> [    8.732674] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
>> [    8.739193] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
>> [    8.746318] Control: 30c5387d  Table: 6e7bc880  DAC: ffe75ece
>> [    8.752055] Process systemd-udevd (pid: 206, stack limit = 0x(ptrval))
>> [    8.758574] Stack: (0xc996bd60 to 0xc996c000)
> [...]
>
> Should be fixed by (PR to Linus with fix is pending):
>
> https://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git/commit/?id=9262478220eac908ae6e168c3df2c453c87e2da3

Unfortunately it's not, building against the git checkout of the first
failed Fedora kernel (see rc2 issue below) it has the same effect :-(

I thought it might have been [1] because it touches bits of that code
but if I trying with rc2 and that reverted I got no output at all,
checking the vanilla Fedora build from friday (so almost rc2) it
doesn't boot at all either so I've got a second thing to investigate.

Peter

[1] https://lkml.org/lkml/2018/4/29/30

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
       [not found]       ` <CALeDE9PBZWJBp8KB0mB4zoNXqscmzxWzz+LnuqRA-z4t1e9T8g@mail.gmail.com>
@ 2018-06-25 16:41           ` Peter Robinson
  0 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-06-25 16:41 UTC (permalink / raw)
  To: Daniel Borkmann; +Cc: Eric Dumazet, netdev, linux-arm-kernel, labbott

On Mon, Jun 25, 2018 at 2:39 PM, Peter Robinson <pbrobinson@gmail.com> wrote:
> Hi Daniel,
>
>> On 06/24/2018 11:24 AM, Peter Robinson wrote:
>>>>> I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
>>>>> a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
>>>>> (doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
>>>>> others, both LPAE/normal kernels.
>>
>> So this is arm32 right?
>
> Correct.
>
>>>>> I'm a bit out of my depth in this part of the kernel but I'm wondering
>>>>> if it's known, I couldn't find anything that looked obvious on a few
>>>>> mailing lists.
>>>>>
>>>>> Peter
>>>>
>>>> Hi Peter
>>>>
>>>> Could you provide symbolic information ?
>>>
>>> I passed in through scripts/decode_stacktrace.sh is that what you were after:
>>>
>>> [    8.673880] Internal error: Oops: a06 [#10] SMP ARM
>>> [    8.673949] ---[ end trace 049df4786ea3140a ]---
>>> [    8.678754] Modules linked in:
>>> [    8.678766] CPU: 1 PID: 206 Comm: systemd-udevd Tainted: G      D
>>>         4.18.0-0.rc1.git0.1.fc29.armv7hl+lpae #1
>>> [    8.678769] Hardware name: Allwinner sun8i Family
>>> [    8.678781] PC is at sk_filter_trim_cap ()
>>> [    8.678790] LR is at   (null)
>>> [    8.709463] pc : lr : psr: 60000013 ()
>>> [    8.715722] sp : c996bd60  ip : 00000000  fp : 00000000
>>> [    8.720939] r10: ee79dc00  r9 : c12c9f80  r8 : 00000000
>>> [    8.726157] r7 : 00000000  r6 : 00000001  r5 : f1648000  r4 : 00000000
>>> [    8.732674] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
>>> [    8.739193] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
>>> [    8.746318] Control: 30c5387d  Table: 6e7bc880  DAC: ffe75ece
>>> [    8.752055] Process systemd-udevd (pid: 206, stack limit = 0x(ptrval))
>>> [    8.758574] Stack: (0xc996bd60 to 0xc996c000)
>>
>> Do you have BPF JIT enabled or disabled? Does it happen with disabled?
>
> Enabled, I can test with it disabled, BPF configs bits are:
> CONFIG_BPF_EVENTS=y
> # CONFIG_BPFILTER is not set
> CONFIG_BPF_JIT_ALWAYS_ON=y
> CONFIG_BPF_JIT=y
> CONFIG_BPF_STREAM_PARSER=y
> CONFIG_BPF_SYSCALL=y
> CONFIG_BPF=y
> CONFIG_CGROUP_BPF=y
> CONFIG_HAVE_EBPF_JIT=y
> CONFIG_IPV6_SEG6_BPF=y
> CONFIG_LWTUNNEL_BPF=y
> # CONFIG_NBPFAXI_DMA is not set
> CONFIG_NET_ACT_BPF=m
> CONFIG_NET_CLS_BPF=m
> CONFIG_NETFILTER_XT_MATCH_BPF=m
> # CONFIG_TEST_BPF is not set
>
>> I can see one bug, but your stack trace seems unrelated.
>>
>> Anyway, could you try with this?
>
> Build in process.
>
>> diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
>> index 6e8b716..f6a62ae 100644
>> --- a/arch/arm/net/bpf_jit_32.c
>> +++ b/arch/arm/net/bpf_jit_32.c
>> @@ -1844,7 +1844,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
>>                 /* there are 2 passes here */
>>                 bpf_jit_dump(prog->len, image_size, 2, ctx.target);
>>
>> -       set_memory_ro((unsigned long)header, header->pages);
>> +       bpf_jit_binary_lock_ro(header);
>>         prog->bpf_func = (void *)ctx.target;
>>         prog->jited = 1;
>>         prog->jited_len = image_size;

So with that and the other fix there was no improvement, with those
and the BPF JIT disabled it works, I'm not sure if the two patches
have any effect with the JIT disabled though.

Will look at the other patches shortly, there's been some other issue
introduced between rc1 and rc2 which I have to work out before I can
test those though.

Peter

^ permalink raw reply	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-06-25 16:41           ` Peter Robinson
  0 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-06-25 16:41 UTC (permalink / raw)
  To: linux-arm-kernel

On Mon, Jun 25, 2018 at 2:39 PM, Peter Robinson <pbrobinson@gmail.com> wrote:
> Hi Daniel,
>
>> On 06/24/2018 11:24 AM, Peter Robinson wrote:
>>>>> I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
>>>>> a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
>>>>> (doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
>>>>> others, both LPAE/normal kernels.
>>
>> So this is arm32 right?
>
> Correct.
>
>>>>> I'm a bit out of my depth in this part of the kernel but I'm wondering
>>>>> if it's known, I couldn't find anything that looked obvious on a few
>>>>> mailing lists.
>>>>>
>>>>> Peter
>>>>
>>>> Hi Peter
>>>>
>>>> Could you provide symbolic information ?
>>>
>>> I passed in through scripts/decode_stacktrace.sh is that what you were after:
>>>
>>> [    8.673880] Internal error: Oops: a06 [#10] SMP ARM
>>> [    8.673949] ---[ end trace 049df4786ea3140a ]---
>>> [    8.678754] Modules linked in:
>>> [    8.678766] CPU: 1 PID: 206 Comm: systemd-udevd Tainted: G      D
>>>         4.18.0-0.rc1.git0.1.fc29.armv7hl+lpae #1
>>> [    8.678769] Hardware name: Allwinner sun8i Family
>>> [    8.678781] PC is at sk_filter_trim_cap ()
>>> [    8.678790] LR is at   (null)
>>> [    8.709463] pc : lr : psr: 60000013 ()
>>> [    8.715722] sp : c996bd60  ip : 00000000  fp : 00000000
>>> [    8.720939] r10: ee79dc00  r9 : c12c9f80  r8 : 00000000
>>> [    8.726157] r7 : 00000000  r6 : 00000001  r5 : f1648000  r4 : 00000000
>>> [    8.732674] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
>>> [    8.739193] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
>>> [    8.746318] Control: 30c5387d  Table: 6e7bc880  DAC: ffe75ece
>>> [    8.752055] Process systemd-udevd (pid: 206, stack limit = 0x(ptrval))
>>> [    8.758574] Stack: (0xc996bd60 to 0xc996c000)
>>
>> Do you have BPF JIT enabled or disabled? Does it happen with disabled?
>
> Enabled, I can test with it disabled, BPF configs bits are:
> CONFIG_BPF_EVENTS=y
> # CONFIG_BPFILTER is not set
> CONFIG_BPF_JIT_ALWAYS_ON=y
> CONFIG_BPF_JIT=y
> CONFIG_BPF_STREAM_PARSER=y
> CONFIG_BPF_SYSCALL=y
> CONFIG_BPF=y
> CONFIG_CGROUP_BPF=y
> CONFIG_HAVE_EBPF_JIT=y
> CONFIG_IPV6_SEG6_BPF=y
> CONFIG_LWTUNNEL_BPF=y
> # CONFIG_NBPFAXI_DMA is not set
> CONFIG_NET_ACT_BPF=m
> CONFIG_NET_CLS_BPF=m
> CONFIG_NETFILTER_XT_MATCH_BPF=m
> # CONFIG_TEST_BPF is not set
>
>> I can see one bug, but your stack trace seems unrelated.
>>
>> Anyway, could you try with this?
>
> Build in process.
>
>> diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
>> index 6e8b716..f6a62ae 100644
>> --- a/arch/arm/net/bpf_jit_32.c
>> +++ b/arch/arm/net/bpf_jit_32.c
>> @@ -1844,7 +1844,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
>>                 /* there are 2 passes here */
>>                 bpf_jit_dump(prog->len, image_size, 2, ctx.target);
>>
>> -       set_memory_ro((unsigned long)header, header->pages);
>> +       bpf_jit_binary_lock_ro(header);
>>         prog->bpf_func = (void *)ctx.target;
>>         prog->jited = 1;
>>         prog->jited_len = image_size;

So with that and the other fix there was no improvement, with those
and the BPF JIT disabled it works, I'm not sure if the two patches
have any effect with the JIT disabled though.

Will look at the other patches shortly, there's been some other issue
introduced between rc1 and rc2 which I have to work out before I can
test those though.

Peter

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-06-25 16:41           ` Peter Robinson
@ 2018-06-26 12:23             ` Peter Robinson
  -1 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-06-26 12:23 UTC (permalink / raw)
  To: Daniel Borkmann; +Cc: Eric Dumazet, netdev, linux-arm-kernel, labbott

Hi Daniel,

>>> On 06/24/2018 11:24 AM, Peter Robinson wrote:
>>>>>> I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
>>>>>> a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
>>>>>> (doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
>>>>>> others, both LPAE/normal kernels.
>>>
>>> So this is arm32 right?
>>
>> Correct.
>>
>>>>>> I'm a bit out of my depth in this part of the kernel but I'm wondering
>>>>>> if it's known, I couldn't find anything that looked obvious on a few
>>>>>> mailing lists.
>>>>>>
>>>>>> Peter
>>>>>
>>>>> Hi Peter
>>>>>
>>>>> Could you provide symbolic information ?
>>>>
>>>> I passed in through scripts/decode_stacktrace.sh is that what you were after:
>>>>
>>>> [    8.673880] Internal error: Oops: a06 [#10] SMP ARM
>>>> [    8.673949] ---[ end trace 049df4786ea3140a ]---
>>>> [    8.678754] Modules linked in:
>>>> [    8.678766] CPU: 1 PID: 206 Comm: systemd-udevd Tainted: G      D
>>>>         4.18.0-0.rc1.git0.1.fc29.armv7hl+lpae #1
>>>> [    8.678769] Hardware name: Allwinner sun8i Family
>>>> [    8.678781] PC is at sk_filter_trim_cap ()
>>>> [    8.678790] LR is at   (null)
>>>> [    8.709463] pc : lr : psr: 60000013 ()
>>>> [    8.715722] sp : c996bd60  ip : 00000000  fp : 00000000
>>>> [    8.720939] r10: ee79dc00  r9 : c12c9f80  r8 : 00000000
>>>> [    8.726157] r7 : 00000000  r6 : 00000001  r5 : f1648000  r4 : 00000000
>>>> [    8.732674] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
>>>> [    8.739193] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
>>>> [    8.746318] Control: 30c5387d  Table: 6e7bc880  DAC: ffe75ece
>>>> [    8.752055] Process systemd-udevd (pid: 206, stack limit = 0x(ptrval))
>>>> [    8.758574] Stack: (0xc996bd60 to 0xc996c000)
>>>
>>> Do you have BPF JIT enabled or disabled? Does it happen with disabled?
>>
>> Enabled, I can test with it disabled, BPF configs bits are:
>> CONFIG_BPF_EVENTS=y
>> # CONFIG_BPFILTER is not set
>> CONFIG_BPF_JIT_ALWAYS_ON=y
>> CONFIG_BPF_JIT=y
>> CONFIG_BPF_STREAM_PARSER=y
>> CONFIG_BPF_SYSCALL=y
>> CONFIG_BPF=y
>> CONFIG_CGROUP_BPF=y
>> CONFIG_HAVE_EBPF_JIT=y
>> CONFIG_IPV6_SEG6_BPF=y
>> CONFIG_LWTUNNEL_BPF=y
>> # CONFIG_NBPFAXI_DMA is not set
>> CONFIG_NET_ACT_BPF=m
>> CONFIG_NET_CLS_BPF=m
>> CONFIG_NETFILTER_XT_MATCH_BPF=m
>> # CONFIG_TEST_BPF is not set
>>
>>> I can see one bug, but your stack trace seems unrelated.
>>>
>>> Anyway, could you try with this?
>>
>> Build in process.
>>
>>> diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
>>> index 6e8b716..f6a62ae 100644
>>> --- a/arch/arm/net/bpf_jit_32.c
>>> +++ b/arch/arm/net/bpf_jit_32.c
>>> @@ -1844,7 +1844,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
>>>                 /* there are 2 passes here */
>>>                 bpf_jit_dump(prog->len, image_size, 2, ctx.target);
>>>
>>> -       set_memory_ro((unsigned long)header, header->pages);
>>> +       bpf_jit_binary_lock_ro(header);
>>>         prog->bpf_func = (void *)ctx.target;
>>>         prog->jited = 1;
>>>         prog->jited_len = image_size;
>
> So with that and the other fix there was no improvement, with those
> and the BPF JIT disabled it works, I'm not sure if the two patches
> have any effect with the JIT disabled though.
>
> Will look at the other patches shortly, there's been some other issue
> introduced between rc1 and rc2 which I have to work out before I can
> test those though.

Quick update, with linus's head as of yesterday, basically rc2 plus
davem's network fixes it works if the JIT is disabled IE:
# CONFIG_BPF_JIT_ALWAYS_ON is not set
# CONFIG_BPF_JIT is not set

If I enable it the boot breaks even worse than the errors above in
that I get no console output at all, even with earlycon, so we've gone
backwards since rc1 somehow.

I'll try the above two reverted unless you have any other suggestions.

Peter

^ permalink raw reply	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-06-26 12:23             ` Peter Robinson
  0 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-06-26 12:23 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Daniel,

>>> On 06/24/2018 11:24 AM, Peter Robinson wrote:
>>>>>> I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
>>>>>> a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
>>>>>> (doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
>>>>>> others, both LPAE/normal kernels.
>>>
>>> So this is arm32 right?
>>
>> Correct.
>>
>>>>>> I'm a bit out of my depth in this part of the kernel but I'm wondering
>>>>>> if it's known, I couldn't find anything that looked obvious on a few
>>>>>> mailing lists.
>>>>>>
>>>>>> Peter
>>>>>
>>>>> Hi Peter
>>>>>
>>>>> Could you provide symbolic information ?
>>>>
>>>> I passed in through scripts/decode_stacktrace.sh is that what you were after:
>>>>
>>>> [    8.673880] Internal error: Oops: a06 [#10] SMP ARM
>>>> [    8.673949] ---[ end trace 049df4786ea3140a ]---
>>>> [    8.678754] Modules linked in:
>>>> [    8.678766] CPU: 1 PID: 206 Comm: systemd-udevd Tainted: G      D
>>>>         4.18.0-0.rc1.git0.1.fc29.armv7hl+lpae #1
>>>> [    8.678769] Hardware name: Allwinner sun8i Family
>>>> [    8.678781] PC is at sk_filter_trim_cap ()
>>>> [    8.678790] LR is at   (null)
>>>> [    8.709463] pc : lr : psr: 60000013 ()
>>>> [    8.715722] sp : c996bd60  ip : 00000000  fp : 00000000
>>>> [    8.720939] r10: ee79dc00  r9 : c12c9f80  r8 : 00000000
>>>> [    8.726157] r7 : 00000000  r6 : 00000001  r5 : f1648000  r4 : 00000000
>>>> [    8.732674] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
>>>> [    8.739193] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
>>>> [    8.746318] Control: 30c5387d  Table: 6e7bc880  DAC: ffe75ece
>>>> [    8.752055] Process systemd-udevd (pid: 206, stack limit = 0x(ptrval))
>>>> [    8.758574] Stack: (0xc996bd60 to 0xc996c000)
>>>
>>> Do you have BPF JIT enabled or disabled? Does it happen with disabled?
>>
>> Enabled, I can test with it disabled, BPF configs bits are:
>> CONFIG_BPF_EVENTS=y
>> # CONFIG_BPFILTER is not set
>> CONFIG_BPF_JIT_ALWAYS_ON=y
>> CONFIG_BPF_JIT=y
>> CONFIG_BPF_STREAM_PARSER=y
>> CONFIG_BPF_SYSCALL=y
>> CONFIG_BPF=y
>> CONFIG_CGROUP_BPF=y
>> CONFIG_HAVE_EBPF_JIT=y
>> CONFIG_IPV6_SEG6_BPF=y
>> CONFIG_LWTUNNEL_BPF=y
>> # CONFIG_NBPFAXI_DMA is not set
>> CONFIG_NET_ACT_BPF=m
>> CONFIG_NET_CLS_BPF=m
>> CONFIG_NETFILTER_XT_MATCH_BPF=m
>> # CONFIG_TEST_BPF is not set
>>
>>> I can see one bug, but your stack trace seems unrelated.
>>>
>>> Anyway, could you try with this?
>>
>> Build in process.
>>
>>> diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
>>> index 6e8b716..f6a62ae 100644
>>> --- a/arch/arm/net/bpf_jit_32.c
>>> +++ b/arch/arm/net/bpf_jit_32.c
>>> @@ -1844,7 +1844,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
>>>                 /* there are 2 passes here */
>>>                 bpf_jit_dump(prog->len, image_size, 2, ctx.target);
>>>
>>> -       set_memory_ro((unsigned long)header, header->pages);
>>> +       bpf_jit_binary_lock_ro(header);
>>>         prog->bpf_func = (void *)ctx.target;
>>>         prog->jited = 1;
>>>         prog->jited_len = image_size;
>
> So with that and the other fix there was no improvement, with those
> and the BPF JIT disabled it works, I'm not sure if the two patches
> have any effect with the JIT disabled though.
>
> Will look at the other patches shortly, there's been some other issue
> introduced between rc1 and rc2 which I have to work out before I can
> test those though.

Quick update, with linus's head as of yesterday, basically rc2 plus
davem's network fixes it works if the JIT is disabled IE:
# CONFIG_BPF_JIT_ALWAYS_ON is not set
# CONFIG_BPF_JIT is not set

If I enable it the boot breaks even worse than the errors above in
that I get no console output at all, even with earlycon, so we've gone
backwards since rc1 somehow.

I'll try the above two reverted unless you have any other suggestions.

Peter

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-06-26 12:23             ` Peter Robinson
@ 2018-06-26 12:52               ` Daniel Borkmann
  -1 siblings, 0 replies; 50+ messages in thread
From: Daniel Borkmann @ 2018-06-26 12:52 UTC (permalink / raw)
  To: Peter Robinson; +Cc: Eric Dumazet, netdev, linux-arm-kernel, labbott

On 06/26/2018 02:23 PM, Peter Robinson wrote:
>>>> On 06/24/2018 11:24 AM, Peter Robinson wrote:
>>>>>>> I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
>>>>>>> a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
>>>>>>> (doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
>>>>>>> others, both LPAE/normal kernels.
>>>>
>>>> So this is arm32 right?
>>>
>>> Correct.
>>>
>>>>>>> I'm a bit out of my depth in this part of the kernel but I'm wondering
>>>>>>> if it's known, I couldn't find anything that looked obvious on a few
>>>>>>> mailing lists.
>>>>>>>
>>>>>>> Peter
>>>>>>
>>>>>> Hi Peter
>>>>>>
>>>>>> Could you provide symbolic information ?
>>>>>
>>>>> I passed in through scripts/decode_stacktrace.sh is that what you were after:
>>>>>
>>>>> [    8.673880] Internal error: Oops: a06 [#10] SMP ARM
>>>>> [    8.673949] ---[ end trace 049df4786ea3140a ]---
>>>>> [    8.678754] Modules linked in:
>>>>> [    8.678766] CPU: 1 PID: 206 Comm: systemd-udevd Tainted: G      D
>>>>>         4.18.0-0.rc1.git0.1.fc29.armv7hl+lpae #1
>>>>> [    8.678769] Hardware name: Allwinner sun8i Family
>>>>> [    8.678781] PC is at sk_filter_trim_cap ()
>>>>> [    8.678790] LR is at   (null)
>>>>> [    8.709463] pc : lr : psr: 60000013 ()
>>>>> [    8.715722] sp : c996bd60  ip : 00000000  fp : 00000000
>>>>> [    8.720939] r10: ee79dc00  r9 : c12c9f80  r8 : 00000000
>>>>> [    8.726157] r7 : 00000000  r6 : 00000001  r5 : f1648000  r4 : 00000000
>>>>> [    8.732674] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
>>>>> [    8.739193] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
>>>>> [    8.746318] Control: 30c5387d  Table: 6e7bc880  DAC: ffe75ece
>>>>> [    8.752055] Process systemd-udevd (pid: 206, stack limit = 0x(ptrval))
>>>>> [    8.758574] Stack: (0xc996bd60 to 0xc996c000)
>>>>
>>>> Do you have BPF JIT enabled or disabled? Does it happen with disabled?
>>>
>>> Enabled, I can test with it disabled, BPF configs bits are:
>>> CONFIG_BPF_EVENTS=y
>>> # CONFIG_BPFILTER is not set
>>> CONFIG_BPF_JIT_ALWAYS_ON=y
>>> CONFIG_BPF_JIT=y
>>> CONFIG_BPF_STREAM_PARSER=y
>>> CONFIG_BPF_SYSCALL=y
>>> CONFIG_BPF=y
>>> CONFIG_CGROUP_BPF=y
>>> CONFIG_HAVE_EBPF_JIT=y
>>> CONFIG_IPV6_SEG6_BPF=y
>>> CONFIG_LWTUNNEL_BPF=y
>>> # CONFIG_NBPFAXI_DMA is not set
>>> CONFIG_NET_ACT_BPF=m
>>> CONFIG_NET_CLS_BPF=m
>>> CONFIG_NETFILTER_XT_MATCH_BPF=m
>>> # CONFIG_TEST_BPF is not set
>>>
>>>> I can see one bug, but your stack trace seems unrelated.
>>>>
>>>> Anyway, could you try with this?
>>>
>>> Build in process.
>>>
>>>> diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
>>>> index 6e8b716..f6a62ae 100644
>>>> --- a/arch/arm/net/bpf_jit_32.c
>>>> +++ b/arch/arm/net/bpf_jit_32.c
>>>> @@ -1844,7 +1844,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
>>>>                 /* there are 2 passes here */
>>>>                 bpf_jit_dump(prog->len, image_size, 2, ctx.target);
>>>>
>>>> -       set_memory_ro((unsigned long)header, header->pages);
>>>> +       bpf_jit_binary_lock_ro(header);
>>>>         prog->bpf_func = (void *)ctx.target;
>>>>         prog->jited = 1;
>>>>         prog->jited_len = image_size;
>>
>> So with that and the other fix there was no improvement, with those
>> and the BPF JIT disabled it works, I'm not sure if the two patches
>> have any effect with the JIT disabled though.
>>
>> Will look at the other patches shortly, there's been some other issue
>> introduced between rc1 and rc2 which I have to work out before I can
>> test those though.
> 
> Quick update, with linus's head as of yesterday, basically rc2 plus
> davem's network fixes it works if the JIT is disabled IE:
> # CONFIG_BPF_JIT_ALWAYS_ON is not set
> # CONFIG_BPF_JIT is not set
> 
> If I enable it the boot breaks even worse than the errors above in
> that I get no console output at all, even with earlycon, so we've gone
> backwards since rc1 somehow.
> 
> I'll try the above two reverted unless you have any other suggestions.

Ok, thanks, lets do that!

I'm still working on fixes meanwhile, should have something by end of day.

Thanks,
Daniel

^ permalink raw reply	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-06-26 12:52               ` Daniel Borkmann
  0 siblings, 0 replies; 50+ messages in thread
From: Daniel Borkmann @ 2018-06-26 12:52 UTC (permalink / raw)
  To: linux-arm-kernel

On 06/26/2018 02:23 PM, Peter Robinson wrote:
>>>> On 06/24/2018 11:24 AM, Peter Robinson wrote:
>>>>>>> I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
>>>>>>> a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
>>>>>>> (doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
>>>>>>> others, both LPAE/normal kernels.
>>>>
>>>> So this is arm32 right?
>>>
>>> Correct.
>>>
>>>>>>> I'm a bit out of my depth in this part of the kernel but I'm wondering
>>>>>>> if it's known, I couldn't find anything that looked obvious on a few
>>>>>>> mailing lists.
>>>>>>>
>>>>>>> Peter
>>>>>>
>>>>>> Hi Peter
>>>>>>
>>>>>> Could you provide symbolic information ?
>>>>>
>>>>> I passed in through scripts/decode_stacktrace.sh is that what you were after:
>>>>>
>>>>> [    8.673880] Internal error: Oops: a06 [#10] SMP ARM
>>>>> [    8.673949] ---[ end trace 049df4786ea3140a ]---
>>>>> [    8.678754] Modules linked in:
>>>>> [    8.678766] CPU: 1 PID: 206 Comm: systemd-udevd Tainted: G      D
>>>>>         4.18.0-0.rc1.git0.1.fc29.armv7hl+lpae #1
>>>>> [    8.678769] Hardware name: Allwinner sun8i Family
>>>>> [    8.678781] PC is at sk_filter_trim_cap ()
>>>>> [    8.678790] LR is at   (null)
>>>>> [    8.709463] pc : lr : psr: 60000013 ()
>>>>> [    8.715722] sp : c996bd60  ip : 00000000  fp : 00000000
>>>>> [    8.720939] r10: ee79dc00  r9 : c12c9f80  r8 : 00000000
>>>>> [    8.726157] r7 : 00000000  r6 : 00000001  r5 : f1648000  r4 : 00000000
>>>>> [    8.732674] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
>>>>> [    8.739193] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
>>>>> [    8.746318] Control: 30c5387d  Table: 6e7bc880  DAC: ffe75ece
>>>>> [    8.752055] Process systemd-udevd (pid: 206, stack limit = 0x(ptrval))
>>>>> [    8.758574] Stack: (0xc996bd60 to 0xc996c000)
>>>>
>>>> Do you have BPF JIT enabled or disabled? Does it happen with disabled?
>>>
>>> Enabled, I can test with it disabled, BPF configs bits are:
>>> CONFIG_BPF_EVENTS=y
>>> # CONFIG_BPFILTER is not set
>>> CONFIG_BPF_JIT_ALWAYS_ON=y
>>> CONFIG_BPF_JIT=y
>>> CONFIG_BPF_STREAM_PARSER=y
>>> CONFIG_BPF_SYSCALL=y
>>> CONFIG_BPF=y
>>> CONFIG_CGROUP_BPF=y
>>> CONFIG_HAVE_EBPF_JIT=y
>>> CONFIG_IPV6_SEG6_BPF=y
>>> CONFIG_LWTUNNEL_BPF=y
>>> # CONFIG_NBPFAXI_DMA is not set
>>> CONFIG_NET_ACT_BPF=m
>>> CONFIG_NET_CLS_BPF=m
>>> CONFIG_NETFILTER_XT_MATCH_BPF=m
>>> # CONFIG_TEST_BPF is not set
>>>
>>>> I can see one bug, but your stack trace seems unrelated.
>>>>
>>>> Anyway, could you try with this?
>>>
>>> Build in process.
>>>
>>>> diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
>>>> index 6e8b716..f6a62ae 100644
>>>> --- a/arch/arm/net/bpf_jit_32.c
>>>> +++ b/arch/arm/net/bpf_jit_32.c
>>>> @@ -1844,7 +1844,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
>>>>                 /* there are 2 passes here */
>>>>                 bpf_jit_dump(prog->len, image_size, 2, ctx.target);
>>>>
>>>> -       set_memory_ro((unsigned long)header, header->pages);
>>>> +       bpf_jit_binary_lock_ro(header);
>>>>         prog->bpf_func = (void *)ctx.target;
>>>>         prog->jited = 1;
>>>>         prog->jited_len = image_size;
>>
>> So with that and the other fix there was no improvement, with those
>> and the BPF JIT disabled it works, I'm not sure if the two patches
>> have any effect with the JIT disabled though.
>>
>> Will look at the other patches shortly, there's been some other issue
>> introduced between rc1 and rc2 which I have to work out before I can
>> test those though.
> 
> Quick update, with linus's head as of yesterday, basically rc2 plus
> davem's network fixes it works if the JIT is disabled IE:
> # CONFIG_BPF_JIT_ALWAYS_ON is not set
> # CONFIG_BPF_JIT is not set
> 
> If I enable it the boot breaks even worse than the errors above in
> that I get no console output at all, even with earlycon, so we've gone
> backwards since rc1 somehow.
> 
> I'll try the above two reverted unless you have any other suggestions.

Ok, thanks, lets do that!

I'm still working on fixes meanwhile, should have something by end of day.

Thanks,
Daniel

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-06-26 12:52               ` Daniel Borkmann
@ 2018-07-04  7:33                 ` Peter Robinson
  -1 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-07-04  7:33 UTC (permalink / raw)
  To: Daniel Borkmann; +Cc: Eric Dumazet, netdev, linux-arm-kernel, labbott

On Tue, Jun 26, 2018 at 1:52 PM, Daniel Borkmann <daniel@iogearbox.net> wrote:
> On 06/26/2018 02:23 PM, Peter Robinson wrote:
>>>>> On 06/24/2018 11:24 AM, Peter Robinson wrote:
>>>>>>>> I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
>>>>>>>> a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
>>>>>>>> (doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
>>>>>>>> others, both LPAE/normal kernels.
>>>>>
>>>>> So this is arm32 right?
>>>>
>>>> Correct.
>>>>
>>>>>>>> I'm a bit out of my depth in this part of the kernel but I'm wondering
>>>>>>>> if it's known, I couldn't find anything that looked obvious on a few
>>>>>>>> mailing lists.
>>>>>>>>
>>>>>>>> Peter
>>>>>>>
>>>>>>> Hi Peter
>>>>>>>
>>>>>>> Could you provide symbolic information ?
>>>>>>
>>>>>> I passed in through scripts/decode_stacktrace.sh is that what you were after:
>>>>>>
>>>>>> [    8.673880] Internal error: Oops: a06 [#10] SMP ARM
>>>>>> [    8.673949] ---[ end trace 049df4786ea3140a ]---
>>>>>> [    8.678754] Modules linked in:
>>>>>> [    8.678766] CPU: 1 PID: 206 Comm: systemd-udevd Tainted: G      D
>>>>>>         4.18.0-0.rc1.git0.1.fc29.armv7hl+lpae #1
>>>>>> [    8.678769] Hardware name: Allwinner sun8i Family
>>>>>> [    8.678781] PC is at sk_filter_trim_cap ()
>>>>>> [    8.678790] LR is at   (null)
>>>>>> [    8.709463] pc : lr : psr: 60000013 ()
>>>>>> [    8.715722] sp : c996bd60  ip : 00000000  fp : 00000000
>>>>>> [    8.720939] r10: ee79dc00  r9 : c12c9f80  r8 : 00000000
>>>>>> [    8.726157] r7 : 00000000  r6 : 00000001  r5 : f1648000  r4 : 00000000
>>>>>> [    8.732674] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
>>>>>> [    8.739193] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
>>>>>> [    8.746318] Control: 30c5387d  Table: 6e7bc880  DAC: ffe75ece
>>>>>> [    8.752055] Process systemd-udevd (pid: 206, stack limit = 0x(ptrval))
>>>>>> [    8.758574] Stack: (0xc996bd60 to 0xc996c000)
>>>>>
>>>>> Do you have BPF JIT enabled or disabled? Does it happen with disabled?
>>>>
>>>> Enabled, I can test with it disabled, BPF configs bits are:
>>>> CONFIG_BPF_EVENTS=y
>>>> # CONFIG_BPFILTER is not set
>>>> CONFIG_BPF_JIT_ALWAYS_ON=y
>>>> CONFIG_BPF_JIT=y
>>>> CONFIG_BPF_STREAM_PARSER=y
>>>> CONFIG_BPF_SYSCALL=y
>>>> CONFIG_BPF=y
>>>> CONFIG_CGROUP_BPF=y
>>>> CONFIG_HAVE_EBPF_JIT=y
>>>> CONFIG_IPV6_SEG6_BPF=y
>>>> CONFIG_LWTUNNEL_BPF=y
>>>> # CONFIG_NBPFAXI_DMA is not set
>>>> CONFIG_NET_ACT_BPF=m
>>>> CONFIG_NET_CLS_BPF=m
>>>> CONFIG_NETFILTER_XT_MATCH_BPF=m
>>>> # CONFIG_TEST_BPF is not set
>>>>
>>>>> I can see one bug, but your stack trace seems unrelated.
>>>>>
>>>>> Anyway, could you try with this?
>>>>
>>>> Build in process.
>>>>
>>>>> diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
>>>>> index 6e8b716..f6a62ae 100644
>>>>> --- a/arch/arm/net/bpf_jit_32.c
>>>>> +++ b/arch/arm/net/bpf_jit_32.c
>>>>> @@ -1844,7 +1844,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
>>>>>                 /* there are 2 passes here */
>>>>>                 bpf_jit_dump(prog->len, image_size, 2, ctx.target);
>>>>>
>>>>> -       set_memory_ro((unsigned long)header, header->pages);
>>>>> +       bpf_jit_binary_lock_ro(header);
>>>>>         prog->bpf_func = (void *)ctx.target;
>>>>>         prog->jited = 1;
>>>>>         prog->jited_len = image_size;
>>>
>>> So with that and the other fix there was no improvement, with those
>>> and the BPF JIT disabled it works, I'm not sure if the two patches
>>> have any effect with the JIT disabled though.
>>>
>>> Will look at the other patches shortly, there's been some other issue
>>> introduced between rc1 and rc2 which I have to work out before I can
>>> test those though.
>>
>> Quick update, with linus's head as of yesterday, basically rc2 plus
>> davem's network fixes it works if the JIT is disabled IE:
>> # CONFIG_BPF_JIT_ALWAYS_ON is not set
>> # CONFIG_BPF_JIT is not set
>>
>> If I enable it the boot breaks even worse than the errors above in
>> that I get no console output at all, even with earlycon, so we've gone
>> backwards since rc1 somehow.
>>
>> I'll try the above two reverted unless you have any other suggestions.
>
> Ok, thanks, lets do that!
>
> I'm still working on fixes meanwhile, should have something by end of day.

Sorry for the delay on this from my end. I noticed there was some bpf
bits land in the last net fixes pull request landed Monday so I built
a kernel with the JIT reenabled. It seems it's improved in that the
completely dead no output boot has gone but the original problem that
arrived in the merge window still persists:

[   17.564142] note: systemd-udevd[194] exited with preempt_count 1
[   17.592739] Unable to handle kernel NULL pointer dereference at
virtual address 0000000c
[   17.601002] pgd = (ptrval)
[   17.603819] [0000000c] *pgd=00000000
[   17.607487] Internal error: Oops: 805 [#10] SMP ARM
[   17.612396] Modules linked in:
[   17.615484] CPU: 0 PID: 195 Comm: systemd-udevd Tainted: G      D
        4.18.0-0.rc3.git1.1.bpf1.fc29.armv7hl #1
[   17.626056] Hardware name: Generic AM33XX (Flattened Device Tree)
[   17.632198] PC is at sk_filter_trim_cap+0x218/0x2fc
[   17.637102] LR is at   (null)
[   17.640086] pc : [<c0ab03b4>]    lr : [<00000000>]    psr: 60000013
[   17.646384] sp : cfe1dd48  ip : 00000000  fp : 00000000
[   17.651635] r10: d837e000  r9 : d833be00  r8 : 00000000
[   17.656887] r7 : 00000001  r6 : e003d000  r5 : 00000000  r4 : 00000000
[   17.663447] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
[   17.670009] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[   17.677180] Control: 10c5387d  Table: 8fe20019  DAC: 00000051
[   17.682956] Process systemd-udevd (pid: 195, stack limit = 0x(ptrval))
[   17.689518] Stack: (0xcfe1dd48 to 0xcfe1e000)
[   17.693901] dd40:                   00000000 00000000 c0ab0234
c1308f38 da610180 da610180
[   17.702123] dd60: 006000c0 00000000 00000000 c0a74524 da610600
da610600 da610180 00000000
[   17.710345] dd80: 00000000 00000000 c1424000 c0addd48 cfc72400
00000001 cfc72400 00000000
[   17.718567] dda0: 00000002 00000000 00000001 d837e064 cfe1de78
00000002 da610180 00000000
[   17.726790] ddc0: cfe1df68 00000085 cfc72400 00000008 00000000
c0adde48 006000c0 00000000
[   17.735012] dde0: 00000000 00000002 00000002 c0ae0c5c 006000c0
00000000 cfd5b580 00000000
[   17.743234] de00: 000000c3 00000000 00000000 00000000 c1379d6c
cfe1df68 cf979900 cfe1de50
[   17.751456] de20: 00000040 00000000 cf979900 00000000 00000000
c0a6ac80 cfe1df68 00000000
[   17.759678] de40: cfe1de50 c0a6b4c4 00000003 00000000 d83f2940
7fff0000 cfe1de88 cfe1dee4
[   17.767899] de60: ffff0000 000000a0 00000000 c044b6c0 beec47e4
00000028 012c6750 0000005d
[   17.776121] de80: 00000000 012d71c8 00000128 40000028 b6c35548
00000000 0000000d 00000000
[   17.784343] dea0: beec47b8 00000000 00000000 00000000 00000010
00000000 00000002 00000002
[   17.792565] dec0: 60000093 c13085ec 00000000 c03bf5f4 00000001
00000080 00000000 c0438d18
[   17.800787] dee0: 00000000 00000000 c1bcd804 00000001 c1bcd7c0
c1bcd804 cfe1df40 c0438d18
[   17.809009] df00: 00000000 60000013 00000001 c03f6064 00000001
00000000 c0438d18 00000000
[   17.817231] df20: cfc73800 cfe1df40 00000001 00000000 beec47b8
cf979900 beec47b8 00000000
[   17.825453] df40: 00000128 c03011c4 cfe1c000 00000128 00000000
c0a6c314 00000000 00000000
[   17.833674] df60: 00000000 fffffff7 cfe1deb0 0000000c 00000001
00000000 00000000 cfe1de80
[   17.841894] df80: 00000000 00000128 00000000 00000000 00000040
00000000 00000000 012d71c8
[   17.850117] dfa0: beec47b8 c03011a0 00000000 012d71c8 0000000d
beec47b8 00000000 00000000
[   17.858339] dfc0: 00000000 012d71c8 beec47b8 00000128 0000005d
012bb998 012d78e8 00000000
[   17.866561] dfe0: b6efbad4 beec4780 b6d40780 b6c35548 60000010
0000000d 00000000 00000000
[   17.874805] [<c0ab03b4>] (sk_filter_trim_cap) from [<c0addd48>]
(netlink_broadcast_filtered+0x2e0/0x3bc)
[   17.884341] [<c0addd48>] (netlink_broadcast_filtered) from
[<c0adde48>] (netlink_broadcast+0x24/0x2c)
[   17.893615] [<c0adde48>] (netlink_broadcast) from [<c0ae0c5c>]
(netlink_sendmsg+0x338/0x370)
[   17.902107] [<c0ae0c5c>] (netlink_sendmsg) from [<c0a6ac80>]
(sock_sendmsg+0x3c/0x4c)
[   17.909986] [<c0a6ac80>] (sock_sendmsg) from [<c0a6b4c4>]
(___sys_sendmsg+0x1e4/0x228)
[   17.917949] [<c0a6b4c4>] (___sys_sendmsg) from [<c0a6c314>]
(__sys_sendmsg+0x48/0x6c)
[   17.925828] [<c0a6c314>] (__sys_sendmsg) from [<c03011a0>]
(__sys_trace_return+0x0/0x10)
[   17.933957] Exception stack(0xcfe1dfa8 to 0xcfe1dff0)
[   17.939037] dfa0:                   00000000 012d71c8 0000000d
beec47b8 00000000 00000000
[   17.947259] dfc0: 00000000 012d71c8 beec47b8 00000128 0000005d
012bb998 012d78e8 00000000
[   17.955478] dfe0: b6efbad4 beec4780 b6d40780 b6c35548
[   17.960563] Code: 1afffff7 e59c0000 e5830000 e3520000 (e584800c)
[   17.966827] ---[ end trace 27a2820a2162a4fd ]---

^ permalink raw reply	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-07-04  7:33                 ` Peter Robinson
  0 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-07-04  7:33 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Jun 26, 2018 at 1:52 PM, Daniel Borkmann <daniel@iogearbox.net> wrote:
> On 06/26/2018 02:23 PM, Peter Robinson wrote:
>>>>> On 06/24/2018 11:24 AM, Peter Robinson wrote:
>>>>>>>> I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
>>>>>>>> a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
>>>>>>>> (doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
>>>>>>>> others, both LPAE/normal kernels.
>>>>>
>>>>> So this is arm32 right?
>>>>
>>>> Correct.
>>>>
>>>>>>>> I'm a bit out of my depth in this part of the kernel but I'm wondering
>>>>>>>> if it's known, I couldn't find anything that looked obvious on a few
>>>>>>>> mailing lists.
>>>>>>>>
>>>>>>>> Peter
>>>>>>>
>>>>>>> Hi Peter
>>>>>>>
>>>>>>> Could you provide symbolic information ?
>>>>>>
>>>>>> I passed in through scripts/decode_stacktrace.sh is that what you were after:
>>>>>>
>>>>>> [    8.673880] Internal error: Oops: a06 [#10] SMP ARM
>>>>>> [    8.673949] ---[ end trace 049df4786ea3140a ]---
>>>>>> [    8.678754] Modules linked in:
>>>>>> [    8.678766] CPU: 1 PID: 206 Comm: systemd-udevd Tainted: G      D
>>>>>>         4.18.0-0.rc1.git0.1.fc29.armv7hl+lpae #1
>>>>>> [    8.678769] Hardware name: Allwinner sun8i Family
>>>>>> [    8.678781] PC is at sk_filter_trim_cap ()
>>>>>> [    8.678790] LR is at   (null)
>>>>>> [    8.709463] pc : lr : psr: 60000013 ()
>>>>>> [    8.715722] sp : c996bd60  ip : 00000000  fp : 00000000
>>>>>> [    8.720939] r10: ee79dc00  r9 : c12c9f80  r8 : 00000000
>>>>>> [    8.726157] r7 : 00000000  r6 : 00000001  r5 : f1648000  r4 : 00000000
>>>>>> [    8.732674] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
>>>>>> [    8.739193] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
>>>>>> [    8.746318] Control: 30c5387d  Table: 6e7bc880  DAC: ffe75ece
>>>>>> [    8.752055] Process systemd-udevd (pid: 206, stack limit = 0x(ptrval))
>>>>>> [    8.758574] Stack: (0xc996bd60 to 0xc996c000)
>>>>>
>>>>> Do you have BPF JIT enabled or disabled? Does it happen with disabled?
>>>>
>>>> Enabled, I can test with it disabled, BPF configs bits are:
>>>> CONFIG_BPF_EVENTS=y
>>>> # CONFIG_BPFILTER is not set
>>>> CONFIG_BPF_JIT_ALWAYS_ON=y
>>>> CONFIG_BPF_JIT=y
>>>> CONFIG_BPF_STREAM_PARSER=y
>>>> CONFIG_BPF_SYSCALL=y
>>>> CONFIG_BPF=y
>>>> CONFIG_CGROUP_BPF=y
>>>> CONFIG_HAVE_EBPF_JIT=y
>>>> CONFIG_IPV6_SEG6_BPF=y
>>>> CONFIG_LWTUNNEL_BPF=y
>>>> # CONFIG_NBPFAXI_DMA is not set
>>>> CONFIG_NET_ACT_BPF=m
>>>> CONFIG_NET_CLS_BPF=m
>>>> CONFIG_NETFILTER_XT_MATCH_BPF=m
>>>> # CONFIG_TEST_BPF is not set
>>>>
>>>>> I can see one bug, but your stack trace seems unrelated.
>>>>>
>>>>> Anyway, could you try with this?
>>>>
>>>> Build in process.
>>>>
>>>>> diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
>>>>> index 6e8b716..f6a62ae 100644
>>>>> --- a/arch/arm/net/bpf_jit_32.c
>>>>> +++ b/arch/arm/net/bpf_jit_32.c
>>>>> @@ -1844,7 +1844,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
>>>>>                 /* there are 2 passes here */
>>>>>                 bpf_jit_dump(prog->len, image_size, 2, ctx.target);
>>>>>
>>>>> -       set_memory_ro((unsigned long)header, header->pages);
>>>>> +       bpf_jit_binary_lock_ro(header);
>>>>>         prog->bpf_func = (void *)ctx.target;
>>>>>         prog->jited = 1;
>>>>>         prog->jited_len = image_size;
>>>
>>> So with that and the other fix there was no improvement, with those
>>> and the BPF JIT disabled it works, I'm not sure if the two patches
>>> have any effect with the JIT disabled though.
>>>
>>> Will look at the other patches shortly, there's been some other issue
>>> introduced between rc1 and rc2 which I have to work out before I can
>>> test those though.
>>
>> Quick update, with linus's head as of yesterday, basically rc2 plus
>> davem's network fixes it works if the JIT is disabled IE:
>> # CONFIG_BPF_JIT_ALWAYS_ON is not set
>> # CONFIG_BPF_JIT is not set
>>
>> If I enable it the boot breaks even worse than the errors above in
>> that I get no console output at all, even with earlycon, so we've gone
>> backwards since rc1 somehow.
>>
>> I'll try the above two reverted unless you have any other suggestions.
>
> Ok, thanks, lets do that!
>
> I'm still working on fixes meanwhile, should have something by end of day.

Sorry for the delay on this from my end. I noticed there was some bpf
bits land in the last net fixes pull request landed Monday so I built
a kernel with the JIT reenabled. It seems it's improved in that the
completely dead no output boot has gone but the original problem that
arrived in the merge window still persists:

[   17.564142] note: systemd-udevd[194] exited with preempt_count 1
[   17.592739] Unable to handle kernel NULL pointer dereference at
virtual address 0000000c
[   17.601002] pgd = (ptrval)
[   17.603819] [0000000c] *pgd=00000000
[   17.607487] Internal error: Oops: 805 [#10] SMP ARM
[   17.612396] Modules linked in:
[   17.615484] CPU: 0 PID: 195 Comm: systemd-udevd Tainted: G      D
        4.18.0-0.rc3.git1.1.bpf1.fc29.armv7hl #1
[   17.626056] Hardware name: Generic AM33XX (Flattened Device Tree)
[   17.632198] PC is at sk_filter_trim_cap+0x218/0x2fc
[   17.637102] LR is at   (null)
[   17.640086] pc : [<c0ab03b4>]    lr : [<00000000>]    psr: 60000013
[   17.646384] sp : cfe1dd48  ip : 00000000  fp : 00000000
[   17.651635] r10: d837e000  r9 : d833be00  r8 : 00000000
[   17.656887] r7 : 00000001  r6 : e003d000  r5 : 00000000  r4 : 00000000
[   17.663447] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
[   17.670009] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[   17.677180] Control: 10c5387d  Table: 8fe20019  DAC: 00000051
[   17.682956] Process systemd-udevd (pid: 195, stack limit = 0x(ptrval))
[   17.689518] Stack: (0xcfe1dd48 to 0xcfe1e000)
[   17.693901] dd40:                   00000000 00000000 c0ab0234
c1308f38 da610180 da610180
[   17.702123] dd60: 006000c0 00000000 00000000 c0a74524 da610600
da610600 da610180 00000000
[   17.710345] dd80: 00000000 00000000 c1424000 c0addd48 cfc72400
00000001 cfc72400 00000000
[   17.718567] dda0: 00000002 00000000 00000001 d837e064 cfe1de78
00000002 da610180 00000000
[   17.726790] ddc0: cfe1df68 00000085 cfc72400 00000008 00000000
c0adde48 006000c0 00000000
[   17.735012] dde0: 00000000 00000002 00000002 c0ae0c5c 006000c0
00000000 cfd5b580 00000000
[   17.743234] de00: 000000c3 00000000 00000000 00000000 c1379d6c
cfe1df68 cf979900 cfe1de50
[   17.751456] de20: 00000040 00000000 cf979900 00000000 00000000
c0a6ac80 cfe1df68 00000000
[   17.759678] de40: cfe1de50 c0a6b4c4 00000003 00000000 d83f2940
7fff0000 cfe1de88 cfe1dee4
[   17.767899] de60: ffff0000 000000a0 00000000 c044b6c0 beec47e4
00000028 012c6750 0000005d
[   17.776121] de80: 00000000 012d71c8 00000128 40000028 b6c35548
00000000 0000000d 00000000
[   17.784343] dea0: beec47b8 00000000 00000000 00000000 00000010
00000000 00000002 00000002
[   17.792565] dec0: 60000093 c13085ec 00000000 c03bf5f4 00000001
00000080 00000000 c0438d18
[   17.800787] dee0: 00000000 00000000 c1bcd804 00000001 c1bcd7c0
c1bcd804 cfe1df40 c0438d18
[   17.809009] df00: 00000000 60000013 00000001 c03f6064 00000001
00000000 c0438d18 00000000
[   17.817231] df20: cfc73800 cfe1df40 00000001 00000000 beec47b8
cf979900 beec47b8 00000000
[   17.825453] df40: 00000128 c03011c4 cfe1c000 00000128 00000000
c0a6c314 00000000 00000000
[   17.833674] df60: 00000000 fffffff7 cfe1deb0 0000000c 00000001
00000000 00000000 cfe1de80
[   17.841894] df80: 00000000 00000128 00000000 00000000 00000040
00000000 00000000 012d71c8
[   17.850117] dfa0: beec47b8 c03011a0 00000000 012d71c8 0000000d
beec47b8 00000000 00000000
[   17.858339] dfc0: 00000000 012d71c8 beec47b8 00000128 0000005d
012bb998 012d78e8 00000000
[   17.866561] dfe0: b6efbad4 beec4780 b6d40780 b6c35548 60000010
0000000d 00000000 00000000
[   17.874805] [<c0ab03b4>] (sk_filter_trim_cap) from [<c0addd48>]
(netlink_broadcast_filtered+0x2e0/0x3bc)
[   17.884341] [<c0addd48>] (netlink_broadcast_filtered) from
[<c0adde48>] (netlink_broadcast+0x24/0x2c)
[   17.893615] [<c0adde48>] (netlink_broadcast) from [<c0ae0c5c>]
(netlink_sendmsg+0x338/0x370)
[   17.902107] [<c0ae0c5c>] (netlink_sendmsg) from [<c0a6ac80>]
(sock_sendmsg+0x3c/0x4c)
[   17.909986] [<c0a6ac80>] (sock_sendmsg) from [<c0a6b4c4>]
(___sys_sendmsg+0x1e4/0x228)
[   17.917949] [<c0a6b4c4>] (___sys_sendmsg) from [<c0a6c314>]
(__sys_sendmsg+0x48/0x6c)
[   17.925828] [<c0a6c314>] (__sys_sendmsg) from [<c03011a0>]
(__sys_trace_return+0x0/0x10)
[   17.933957] Exception stack(0xcfe1dfa8 to 0xcfe1dff0)
[   17.939037] dfa0:                   00000000 012d71c8 0000000d
beec47b8 00000000 00000000
[   17.947259] dfc0: 00000000 012d71c8 beec47b8 00000128 0000005d
012bb998 012d78e8 00000000
[   17.955478] dfe0: b6efbad4 beec4780 b6d40780 b6c35548
[   17.960563] Code: 1afffff7 e59c0000 e5830000 e3520000 (e584800c)
[   17.966827] ---[ end trace 27a2820a2162a4fd ]---

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-07-04  7:33                 ` Peter Robinson
@ 2018-07-04 23:10                   ` Daniel Borkmann
  -1 siblings, 0 replies; 50+ messages in thread
From: Daniel Borkmann @ 2018-07-04 23:10 UTC (permalink / raw)
  To: Peter Robinson; +Cc: Eric Dumazet, netdev, linux-arm-kernel, labbott

On 07/04/2018 09:33 AM, Peter Robinson wrote:
> On Tue, Jun 26, 2018 at 1:52 PM, Daniel Borkmann <daniel@iogearbox.net> wrote:
>> On 06/26/2018 02:23 PM, Peter Robinson wrote:
>>>>>> On 06/24/2018 11:24 AM, Peter Robinson wrote:
>>>>>>>>> I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
>>>>>>>>> a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
>>>>>>>>> (doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
>>>>>>>>> others, both LPAE/normal kernels.
>>>>>>
>>>>>> So this is arm32 right?
>>>>>
>>>>> Correct.
>>>>>
>>>>>>>>> I'm a bit out of my depth in this part of the kernel but I'm wondering
>>>>>>>>> if it's known, I couldn't find anything that looked obvious on a few
>>>>>>>>> mailing lists.
>>>>>>>>>
>>>>>>>>> Peter
>>>>>>>>
>>>>>>>> Hi Peter
>>>>>>>>
>>>>>>>> Could you provide symbolic information ?
>>>>>>>
>>>>>>> I passed in through scripts/decode_stacktrace.sh is that what you were after:
>>>>>>>
>>>>>>> [    8.673880] Internal error: Oops: a06 [#10] SMP ARM
>>>>>>> [    8.673949] ---[ end trace 049df4786ea3140a ]---
>>>>>>> [    8.678754] Modules linked in:
>>>>>>> [    8.678766] CPU: 1 PID: 206 Comm: systemd-udevd Tainted: G      D
>>>>>>>         4.18.0-0.rc1.git0.1.fc29.armv7hl+lpae #1
>>>>>>> [    8.678769] Hardware name: Allwinner sun8i Family
>>>>>>> [    8.678781] PC is at sk_filter_trim_cap ()
>>>>>>> [    8.678790] LR is at   (null)
>>>>>>> [    8.709463] pc : lr : psr: 60000013 ()
>>>>>>> [    8.715722] sp : c996bd60  ip : 00000000  fp : 00000000
>>>>>>> [    8.720939] r10: ee79dc00  r9 : c12c9f80  r8 : 00000000
>>>>>>> [    8.726157] r7 : 00000000  r6 : 00000001  r5 : f1648000  r4 : 00000000
>>>>>>> [    8.732674] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
>>>>>>> [    8.739193] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
>>>>>>> [    8.746318] Control: 30c5387d  Table: 6e7bc880  DAC: ffe75ece
>>>>>>> [    8.752055] Process systemd-udevd (pid: 206, stack limit = 0x(ptrval))
>>>>>>> [    8.758574] Stack: (0xc996bd60 to 0xc996c000)
>>>>>>
>>>>>> Do you have BPF JIT enabled or disabled? Does it happen with disabled?
>>>>>
>>>>> Enabled, I can test with it disabled, BPF configs bits are:
>>>>> CONFIG_BPF_EVENTS=y
>>>>> # CONFIG_BPFILTER is not set
>>>>> CONFIG_BPF_JIT_ALWAYS_ON=y
>>>>> CONFIG_BPF_JIT=y
>>>>> CONFIG_BPF_STREAM_PARSER=y
>>>>> CONFIG_BPF_SYSCALL=y
>>>>> CONFIG_BPF=y
>>>>> CONFIG_CGROUP_BPF=y
>>>>> CONFIG_HAVE_EBPF_JIT=y
>>>>> CONFIG_IPV6_SEG6_BPF=y
>>>>> CONFIG_LWTUNNEL_BPF=y
>>>>> # CONFIG_NBPFAXI_DMA is not set
>>>>> CONFIG_NET_ACT_BPF=m
>>>>> CONFIG_NET_CLS_BPF=m
>>>>> CONFIG_NETFILTER_XT_MATCH_BPF=m
>>>>> # CONFIG_TEST_BPF is not set
>>>>>
>>>>>> I can see one bug, but your stack trace seems unrelated.
>>>>>>
>>>>>> Anyway, could you try with this?
>>>>>
>>>>> Build in process.
>>>>>
>>>>>> diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
>>>>>> index 6e8b716..f6a62ae 100644
>>>>>> --- a/arch/arm/net/bpf_jit_32.c
>>>>>> +++ b/arch/arm/net/bpf_jit_32.c
>>>>>> @@ -1844,7 +1844,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
>>>>>>                 /* there are 2 passes here */
>>>>>>                 bpf_jit_dump(prog->len, image_size, 2, ctx.target);
>>>>>>
>>>>>> -       set_memory_ro((unsigned long)header, header->pages);
>>>>>> +       bpf_jit_binary_lock_ro(header);
>>>>>>         prog->bpf_func = (void *)ctx.target;
>>>>>>         prog->jited = 1;
>>>>>>         prog->jited_len = image_size;
>>>>
>>>> So with that and the other fix there was no improvement, with those
>>>> and the BPF JIT disabled it works, I'm not sure if the two patches
>>>> have any effect with the JIT disabled though.
>>>>
>>>> Will look at the other patches shortly, there's been some other issue
>>>> introduced between rc1 and rc2 which I have to work out before I can
>>>> test those though.
>>>
>>> Quick update, with linus's head as of yesterday, basically rc2 plus
>>> davem's network fixes it works if the JIT is disabled IE:
>>> # CONFIG_BPF_JIT_ALWAYS_ON is not set
>>> # CONFIG_BPF_JIT is not set
>>>
>>> If I enable it the boot breaks even worse than the errors above in
>>> that I get no console output at all, even with earlycon, so we've gone
>>> backwards since rc1 somehow.
>>>
>>> I'll try the above two reverted unless you have any other suggestions.
>>
>> Ok, thanks, lets do that!
>>
>> I'm still working on fixes meanwhile, should have something by end of day.
> 
> Sorry for the delay on this from my end. I noticed there was some bpf
> bits land in the last net fixes pull request landed Monday so I built
> a kernel with the JIT reenabled. It seems it's improved in that the
> completely dead no output boot has gone but the original problem that
> arrived in the merge window still persists:

Okay, thanks a lot! And on top of that tree could you try with the below
applied to check whether it fixes the issue?

diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index f6a62ae..45e6b49 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -234,11 +234,11 @@ static void jit_fill_hole(void *area, unsigned int size)
 #define SCRATCH_SIZE 80

 /* total stack size used in JITed code */
-#define _STACK_SIZE	(ctx->prog->aux->stack_depth + SCRATCH_SIZE)
+#define _STACK_SIZE	(ctx->prog->aux->stack_depth + SCRATCH_SIZE + 4)
 #define STACK_SIZE	ALIGN(_STACK_SIZE, STACK_ALIGNMENT)

 /* Get the offset of eBPF REGISTERs stored on scratch space. */
-#define STACK_VAR(off) (STACK_SIZE - off)
+#define STACK_VAR(off) (STACK_SIZE - 4 - off)

 #if __LINUX_ARM_ARCH__ < 7

^ permalink raw reply related	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-07-04 23:10                   ` Daniel Borkmann
  0 siblings, 0 replies; 50+ messages in thread
From: Daniel Borkmann @ 2018-07-04 23:10 UTC (permalink / raw)
  To: linux-arm-kernel

On 07/04/2018 09:33 AM, Peter Robinson wrote:
> On Tue, Jun 26, 2018 at 1:52 PM, Daniel Borkmann <daniel@iogearbox.net> wrote:
>> On 06/26/2018 02:23 PM, Peter Robinson wrote:
>>>>>> On 06/24/2018 11:24 AM, Peter Robinson wrote:
>>>>>>>>> I'm seeing this netlink/sk_filter_trim_cap crash on ARMv7 across quite
>>>>>>>>> a few ARMv7 platforms on Fedora with 4.18rc1. I've tested RPi2/RPi3
>>>>>>>>> (doesn't happen on aarch64), AllWinner H3, BeagleBone and a few
>>>>>>>>> others, both LPAE/normal kernels.
>>>>>>
>>>>>> So this is arm32 right?
>>>>>
>>>>> Correct.
>>>>>
>>>>>>>>> I'm a bit out of my depth in this part of the kernel but I'm wondering
>>>>>>>>> if it's known, I couldn't find anything that looked obvious on a few
>>>>>>>>> mailing lists.
>>>>>>>>>
>>>>>>>>> Peter
>>>>>>>>
>>>>>>>> Hi Peter
>>>>>>>>
>>>>>>>> Could you provide symbolic information ?
>>>>>>>
>>>>>>> I passed in through scripts/decode_stacktrace.sh is that what you were after:
>>>>>>>
>>>>>>> [    8.673880] Internal error: Oops: a06 [#10] SMP ARM
>>>>>>> [    8.673949] ---[ end trace 049df4786ea3140a ]---
>>>>>>> [    8.678754] Modules linked in:
>>>>>>> [    8.678766] CPU: 1 PID: 206 Comm: systemd-udevd Tainted: G      D
>>>>>>>         4.18.0-0.rc1.git0.1.fc29.armv7hl+lpae #1
>>>>>>> [    8.678769] Hardware name: Allwinner sun8i Family
>>>>>>> [    8.678781] PC is at sk_filter_trim_cap ()
>>>>>>> [    8.678790] LR is at   (null)
>>>>>>> [    8.709463] pc : lr : psr: 60000013 ()
>>>>>>> [    8.715722] sp : c996bd60  ip : 00000000  fp : 00000000
>>>>>>> [    8.720939] r10: ee79dc00  r9 : c12c9f80  r8 : 00000000
>>>>>>> [    8.726157] r7 : 00000000  r6 : 00000001  r5 : f1648000  r4 : 00000000
>>>>>>> [    8.732674] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
>>>>>>> [    8.739193] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
>>>>>>> [    8.746318] Control: 30c5387d  Table: 6e7bc880  DAC: ffe75ece
>>>>>>> [    8.752055] Process systemd-udevd (pid: 206, stack limit = 0x(ptrval))
>>>>>>> [    8.758574] Stack: (0xc996bd60 to 0xc996c000)
>>>>>>
>>>>>> Do you have BPF JIT enabled or disabled? Does it happen with disabled?
>>>>>
>>>>> Enabled, I can test with it disabled, BPF configs bits are:
>>>>> CONFIG_BPF_EVENTS=y
>>>>> # CONFIG_BPFILTER is not set
>>>>> CONFIG_BPF_JIT_ALWAYS_ON=y
>>>>> CONFIG_BPF_JIT=y
>>>>> CONFIG_BPF_STREAM_PARSER=y
>>>>> CONFIG_BPF_SYSCALL=y
>>>>> CONFIG_BPF=y
>>>>> CONFIG_CGROUP_BPF=y
>>>>> CONFIG_HAVE_EBPF_JIT=y
>>>>> CONFIG_IPV6_SEG6_BPF=y
>>>>> CONFIG_LWTUNNEL_BPF=y
>>>>> # CONFIG_NBPFAXI_DMA is not set
>>>>> CONFIG_NET_ACT_BPF=m
>>>>> CONFIG_NET_CLS_BPF=m
>>>>> CONFIG_NETFILTER_XT_MATCH_BPF=m
>>>>> # CONFIG_TEST_BPF is not set
>>>>>
>>>>>> I can see one bug, but your stack trace seems unrelated.
>>>>>>
>>>>>> Anyway, could you try with this?
>>>>>
>>>>> Build in process.
>>>>>
>>>>>> diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
>>>>>> index 6e8b716..f6a62ae 100644
>>>>>> --- a/arch/arm/net/bpf_jit_32.c
>>>>>> +++ b/arch/arm/net/bpf_jit_32.c
>>>>>> @@ -1844,7 +1844,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
>>>>>>                 /* there are 2 passes here */
>>>>>>                 bpf_jit_dump(prog->len, image_size, 2, ctx.target);
>>>>>>
>>>>>> -       set_memory_ro((unsigned long)header, header->pages);
>>>>>> +       bpf_jit_binary_lock_ro(header);
>>>>>>         prog->bpf_func = (void *)ctx.target;
>>>>>>         prog->jited = 1;
>>>>>>         prog->jited_len = image_size;
>>>>
>>>> So with that and the other fix there was no improvement, with those
>>>> and the BPF JIT disabled it works, I'm not sure if the two patches
>>>> have any effect with the JIT disabled though.
>>>>
>>>> Will look at the other patches shortly, there's been some other issue
>>>> introduced between rc1 and rc2 which I have to work out before I can
>>>> test those though.
>>>
>>> Quick update, with linus's head as of yesterday, basically rc2 plus
>>> davem's network fixes it works if the JIT is disabled IE:
>>> # CONFIG_BPF_JIT_ALWAYS_ON is not set
>>> # CONFIG_BPF_JIT is not set
>>>
>>> If I enable it the boot breaks even worse than the errors above in
>>> that I get no console output at all, even with earlycon, so we've gone
>>> backwards since rc1 somehow.
>>>
>>> I'll try the above two reverted unless you have any other suggestions.
>>
>> Ok, thanks, lets do that!
>>
>> I'm still working on fixes meanwhile, should have something by end of day.
> 
> Sorry for the delay on this from my end. I noticed there was some bpf
> bits land in the last net fixes pull request landed Monday so I built
> a kernel with the JIT reenabled. It seems it's improved in that the
> completely dead no output boot has gone but the original problem that
> arrived in the merge window still persists:

Okay, thanks a lot! And on top of that tree could you try with the below
applied to check whether it fixes the issue?

diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index f6a62ae..45e6b49 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -234,11 +234,11 @@ static void jit_fill_hole(void *area, unsigned int size)
 #define SCRATCH_SIZE 80

 /* total stack size used in JITed code */
-#define _STACK_SIZE	(ctx->prog->aux->stack_depth + SCRATCH_SIZE)
+#define _STACK_SIZE	(ctx->prog->aux->stack_depth + SCRATCH_SIZE + 4)
 #define STACK_SIZE	ALIGN(_STACK_SIZE, STACK_ALIGNMENT)

 /* Get the offset of eBPF REGISTERs stored on scratch space. */
-#define STACK_VAR(off) (STACK_SIZE - off)
+#define STACK_VAR(off) (STACK_SIZE - 4 - off)

 #if __LINUX_ARM_ARCH__ < 7

^ permalink raw reply related	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-07-04  7:33                 ` Peter Robinson
@ 2018-07-04 23:41                   ` Russell King - ARM Linux
  -1 siblings, 0 replies; 50+ messages in thread
From: Russell King - ARM Linux @ 2018-07-04 23:41 UTC (permalink / raw)
  To: Peter Robinson
  Cc: Daniel Borkmann, netdev, labbott, linux-arm-kernel, Eric Dumazet

Subject says offlist, but this isn't...

On Wed, Jul 04, 2018 at 08:33:20AM +0100, Peter Robinson wrote:
> Sorry for the delay on this from my end. I noticed there was some bpf
> bits land in the last net fixes pull request landed Monday so I built
> a kernel with the JIT reenabled. It seems it's improved in that the
> completely dead no output boot has gone but the original problem that
> arrived in the merge window still persists:
> 
> [   17.564142] note: systemd-udevd[194] exited with preempt_count 1
> [   17.592739] Unable to handle kernel NULL pointer dereference at
> virtual address 0000000c
> [   17.601002] pgd = (ptrval)
> [   17.603819] [0000000c] *pgd=00000000
> [   17.607487] Internal error: Oops: 805 [#10] SMP ARM
> [   17.612396] Modules linked in:
> [   17.615484] CPU: 0 PID: 195 Comm: systemd-udevd Tainted: G      D
>         4.18.0-0.rc3.git1.1.bpf1.fc29.armv7hl #1
> [   17.626056] Hardware name: Generic AM33XX (Flattened Device Tree)
> [   17.632198] PC is at sk_filter_trim_cap+0x218/0x2fc
> [   17.637102] LR is at   (null)
> [   17.640086] pc : [<c0ab03b4>]    lr : [<00000000>]    psr: 60000013
> [   17.646384] sp : cfe1dd48  ip : 00000000  fp : 00000000
> [   17.651635] r10: d837e000  r9 : d833be00  r8 : 00000000
> [   17.656887] r7 : 00000001  r6 : e003d000  r5 : 00000000  r4 : 00000000
> [   17.663447] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
> [   17.670009] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
> [   17.677180] Control: 10c5387d  Table: 8fe20019  DAC: 00000051
> [   17.682956] Process systemd-udevd (pid: 195, stack limit = 0x(ptrval))
> [   17.689518] Stack: (0xcfe1dd48 to 0xcfe1e000)

Can you provide a full disassembly of sk_filter_trim_cap from vmlinux
(iow, annotated with its linked address) for the above dump please -
alternatively a new dump with matching disassembly.  Thanks.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 13.8Mbps down 630kbps up
According to speedtest.net: 13Mbps down 490kbps up

^ permalink raw reply	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-07-04 23:41                   ` Russell King - ARM Linux
  0 siblings, 0 replies; 50+ messages in thread
From: Russell King - ARM Linux @ 2018-07-04 23:41 UTC (permalink / raw)
  To: linux-arm-kernel

Subject says offlist, but this isn't...

On Wed, Jul 04, 2018 at 08:33:20AM +0100, Peter Robinson wrote:
> Sorry for the delay on this from my end. I noticed there was some bpf
> bits land in the last net fixes pull request landed Monday so I built
> a kernel with the JIT reenabled. It seems it's improved in that the
> completely dead no output boot has gone but the original problem that
> arrived in the merge window still persists:
> 
> [   17.564142] note: systemd-udevd[194] exited with preempt_count 1
> [   17.592739] Unable to handle kernel NULL pointer dereference at
> virtual address 0000000c
> [   17.601002] pgd = (ptrval)
> [   17.603819] [0000000c] *pgd=00000000
> [   17.607487] Internal error: Oops: 805 [#10] SMP ARM
> [   17.612396] Modules linked in:
> [   17.615484] CPU: 0 PID: 195 Comm: systemd-udevd Tainted: G      D
>         4.18.0-0.rc3.git1.1.bpf1.fc29.armv7hl #1
> [   17.626056] Hardware name: Generic AM33XX (Flattened Device Tree)
> [   17.632198] PC is at sk_filter_trim_cap+0x218/0x2fc
> [   17.637102] LR is at   (null)
> [   17.640086] pc : [<c0ab03b4>]    lr : [<00000000>]    psr: 60000013
> [   17.646384] sp : cfe1dd48  ip : 00000000  fp : 00000000
> [   17.651635] r10: d837e000  r9 : d833be00  r8 : 00000000
> [   17.656887] r7 : 00000001  r6 : e003d000  r5 : 00000000  r4 : 00000000
> [   17.663447] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
> [   17.670009] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
> [   17.677180] Control: 10c5387d  Table: 8fe20019  DAC: 00000051
> [   17.682956] Process systemd-udevd (pid: 195, stack limit = 0x(ptrval))
> [   17.689518] Stack: (0xcfe1dd48 to 0xcfe1e000)

Can you provide a full disassembly of sk_filter_trim_cap from vmlinux
(iow, annotated with its linked address) for the above dump please -
alternatively a new dump with matching disassembly.  Thanks.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 13.8Mbps down 630kbps up
According to speedtest.net: 13Mbps down 490kbps up

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-07-04 23:41                   ` Russell King - ARM Linux
@ 2018-07-05  7:31                     ` Russell King - ARM Linux
  -1 siblings, 0 replies; 50+ messages in thread
From: Russell King - ARM Linux @ 2018-07-05  7:31 UTC (permalink / raw)
  To: Peter Robinson
  Cc: netdev, labbott, linux-arm-kernel, Daniel Borkmann, Eric Dumazet

On Thu, Jul 05, 2018 at 12:41:54AM +0100, Russell King - ARM Linux wrote:
> Subject says offlist, but this isn't...
> 
> On Wed, Jul 04, 2018 at 08:33:20AM +0100, Peter Robinson wrote:
> > Sorry for the delay on this from my end. I noticed there was some bpf
> > bits land in the last net fixes pull request landed Monday so I built
> > a kernel with the JIT reenabled. It seems it's improved in that the
> > completely dead no output boot has gone but the original problem that
> > arrived in the merge window still persists:
> > 
> > [   17.564142] note: systemd-udevd[194] exited with preempt_count 1
> > [   17.592739] Unable to handle kernel NULL pointer dereference at
> > virtual address 0000000c
> > [   17.601002] pgd = (ptrval)
> > [   17.603819] [0000000c] *pgd=00000000
> > [   17.607487] Internal error: Oops: 805 [#10] SMP ARM
> > [   17.612396] Modules linked in:
> > [   17.615484] CPU: 0 PID: 195 Comm: systemd-udevd Tainted: G      D
> >         4.18.0-0.rc3.git1.1.bpf1.fc29.armv7hl #1
> > [   17.626056] Hardware name: Generic AM33XX (Flattened Device Tree)
> > [   17.632198] PC is at sk_filter_trim_cap+0x218/0x2fc
> > [   17.637102] LR is at   (null)
> > [   17.640086] pc : [<c0ab03b4>]    lr : [<00000000>]    psr: 60000013
> > [   17.646384] sp : cfe1dd48  ip : 00000000  fp : 00000000
> > [   17.651635] r10: d837e000  r9 : d833be00  r8 : 00000000
> > [   17.656887] r7 : 00000001  r6 : e003d000  r5 : 00000000  r4 : 00000000
> > [   17.663447] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
> > [   17.670009] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
> > [   17.677180] Control: 10c5387d  Table: 8fe20019  DAC: 00000051
> > [   17.682956] Process systemd-udevd (pid: 195, stack limit = 0x(ptrval))
> > [   17.689518] Stack: (0xcfe1dd48 to 0xcfe1e000)
> 
> Can you provide a full disassembly of sk_filter_trim_cap from vmlinux
> (iow, annotated with its linked address) for the above dump please -
> alternatively a new dump with matching disassembly.  Thanks.

Also probably a good idea to have bpf_jit_enable set to 2 to get a
dump of the bpf program being run, which I think for your problem,
you'll have to hack the kernel source to do that.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 13.8Mbps down 630kbps up
According to speedtest.net: 13Mbps down 490kbps up

^ permalink raw reply	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-07-05  7:31                     ` Russell King - ARM Linux
  0 siblings, 0 replies; 50+ messages in thread
From: Russell King - ARM Linux @ 2018-07-05  7:31 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Jul 05, 2018 at 12:41:54AM +0100, Russell King - ARM Linux wrote:
> Subject says offlist, but this isn't...
> 
> On Wed, Jul 04, 2018 at 08:33:20AM +0100, Peter Robinson wrote:
> > Sorry for the delay on this from my end. I noticed there was some bpf
> > bits land in the last net fixes pull request landed Monday so I built
> > a kernel with the JIT reenabled. It seems it's improved in that the
> > completely dead no output boot has gone but the original problem that
> > arrived in the merge window still persists:
> > 
> > [   17.564142] note: systemd-udevd[194] exited with preempt_count 1
> > [   17.592739] Unable to handle kernel NULL pointer dereference at
> > virtual address 0000000c
> > [   17.601002] pgd = (ptrval)
> > [   17.603819] [0000000c] *pgd=00000000
> > [   17.607487] Internal error: Oops: 805 [#10] SMP ARM
> > [   17.612396] Modules linked in:
> > [   17.615484] CPU: 0 PID: 195 Comm: systemd-udevd Tainted: G      D
> >         4.18.0-0.rc3.git1.1.bpf1.fc29.armv7hl #1
> > [   17.626056] Hardware name: Generic AM33XX (Flattened Device Tree)
> > [   17.632198] PC is at sk_filter_trim_cap+0x218/0x2fc
> > [   17.637102] LR is at   (null)
> > [   17.640086] pc : [<c0ab03b4>]    lr : [<00000000>]    psr: 60000013
> > [   17.646384] sp : cfe1dd48  ip : 00000000  fp : 00000000
> > [   17.651635] r10: d837e000  r9 : d833be00  r8 : 00000000
> > [   17.656887] r7 : 00000001  r6 : e003d000  r5 : 00000000  r4 : 00000000
> > [   17.663447] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
> > [   17.670009] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
> > [   17.677180] Control: 10c5387d  Table: 8fe20019  DAC: 00000051
> > [   17.682956] Process systemd-udevd (pid: 195, stack limit = 0x(ptrval))
> > [   17.689518] Stack: (0xcfe1dd48 to 0xcfe1e000)
> 
> Can you provide a full disassembly of sk_filter_trim_cap from vmlinux
> (iow, annotated with its linked address) for the above dump please -
> alternatively a new dump with matching disassembly.  Thanks.

Also probably a good idea to have bpf_jit_enable set to 2 to get a
dump of the bpf program being run, which I think for your problem,
you'll have to hack the kernel source to do that.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 13.8Mbps down 630kbps up
According to speedtest.net: 13Mbps down 490kbps up

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-07-05  7:31                     ` Russell King - ARM Linux
@ 2018-07-05  7:46                       ` Daniel Borkmann
  -1 siblings, 0 replies; 50+ messages in thread
From: Daniel Borkmann @ 2018-07-05  7:46 UTC (permalink / raw)
  To: Russell King - ARM Linux, Peter Robinson
  Cc: netdev, labbott, linux-arm-kernel, Eric Dumazet

On 07/05/2018 09:31 AM, Russell King - ARM Linux wrote:
> On Thu, Jul 05, 2018 at 12:41:54AM +0100, Russell King - ARM Linux wrote:
>> Subject says offlist, but this isn't...
>>
>> On Wed, Jul 04, 2018 at 08:33:20AM +0100, Peter Robinson wrote:
>>> Sorry for the delay on this from my end. I noticed there was some bpf
>>> bits land in the last net fixes pull request landed Monday so I built
>>> a kernel with the JIT reenabled. It seems it's improved in that the
>>> completely dead no output boot has gone but the original problem that
>>> arrived in the merge window still persists:
>>>
>>> [   17.564142] note: systemd-udevd[194] exited with preempt_count 1
>>> [   17.592739] Unable to handle kernel NULL pointer dereference at
>>> virtual address 0000000c
>>> [   17.601002] pgd = (ptrval)
>>> [   17.603819] [0000000c] *pgd=00000000
>>> [   17.607487] Internal error: Oops: 805 [#10] SMP ARM
>>> [   17.612396] Modules linked in:
>>> [   17.615484] CPU: 0 PID: 195 Comm: systemd-udevd Tainted: G      D
>>>         4.18.0-0.rc3.git1.1.bpf1.fc29.armv7hl #1
>>> [   17.626056] Hardware name: Generic AM33XX (Flattened Device Tree)
>>> [   17.632198] PC is at sk_filter_trim_cap+0x218/0x2fc
>>> [   17.637102] LR is at   (null)
>>> [   17.640086] pc : [<c0ab03b4>]    lr : [<00000000>]    psr: 60000013
>>> [   17.646384] sp : cfe1dd48  ip : 00000000  fp : 00000000
>>> [   17.651635] r10: d837e000  r9 : d833be00  r8 : 00000000
>>> [   17.656887] r7 : 00000001  r6 : e003d000  r5 : 00000000  r4 : 00000000
>>> [   17.663447] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
>>> [   17.670009] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
>>> [   17.677180] Control: 10c5387d  Table: 8fe20019  DAC: 00000051
>>> [   17.682956] Process systemd-udevd (pid: 195, stack limit = 0x(ptrval))
>>> [   17.689518] Stack: (0xcfe1dd48 to 0xcfe1e000)
>>
>> Can you provide a full disassembly of sk_filter_trim_cap from vmlinux
>> (iow, annotated with its linked address) for the above dump please -
>> alternatively a new dump with matching disassembly.  Thanks.
> 
> Also probably a good idea to have bpf_jit_enable set to 2 to get a
> dump of the bpf program being run, which I think for your problem,
> you'll have to hack the kernel source to do that.

Agree, that would be good as well. You could use something like the below
to bail out to interpreter after JIT did the dump.

Dump will then land in kernel log which you could paste here.

diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index f6a62ae..d6a7dfd 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -1844,6 +1844,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
 		/* there are 2 passes here */
 		bpf_jit_dump(prog->len, image_size, 2, ctx.target);

+	/* Defer to interpreter after dump. */
+	if (1) {
+		bpf_jit_binary_free(header);
+		prog = orig_prog;
+		goto out_imms;
+	}
+
 	bpf_jit_binary_lock_ro(header);
 	prog->bpf_func = (void *)ctx.target;
 	prog->jited = 1;

^ permalink raw reply related	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-07-05  7:46                       ` Daniel Borkmann
  0 siblings, 0 replies; 50+ messages in thread
From: Daniel Borkmann @ 2018-07-05  7:46 UTC (permalink / raw)
  To: linux-arm-kernel

On 07/05/2018 09:31 AM, Russell King - ARM Linux wrote:
> On Thu, Jul 05, 2018 at 12:41:54AM +0100, Russell King - ARM Linux wrote:
>> Subject says offlist, but this isn't...
>>
>> On Wed, Jul 04, 2018 at 08:33:20AM +0100, Peter Robinson wrote:
>>> Sorry for the delay on this from my end. I noticed there was some bpf
>>> bits land in the last net fixes pull request landed Monday so I built
>>> a kernel with the JIT reenabled. It seems it's improved in that the
>>> completely dead no output boot has gone but the original problem that
>>> arrived in the merge window still persists:
>>>
>>> [   17.564142] note: systemd-udevd[194] exited with preempt_count 1
>>> [   17.592739] Unable to handle kernel NULL pointer dereference at
>>> virtual address 0000000c
>>> [   17.601002] pgd = (ptrval)
>>> [   17.603819] [0000000c] *pgd=00000000
>>> [   17.607487] Internal error: Oops: 805 [#10] SMP ARM
>>> [   17.612396] Modules linked in:
>>> [   17.615484] CPU: 0 PID: 195 Comm: systemd-udevd Tainted: G      D
>>>         4.18.0-0.rc3.git1.1.bpf1.fc29.armv7hl #1
>>> [   17.626056] Hardware name: Generic AM33XX (Flattened Device Tree)
>>> [   17.632198] PC is at sk_filter_trim_cap+0x218/0x2fc
>>> [   17.637102] LR is at   (null)
>>> [   17.640086] pc : [<c0ab03b4>]    lr : [<00000000>]    psr: 60000013
>>> [   17.646384] sp : cfe1dd48  ip : 00000000  fp : 00000000
>>> [   17.651635] r10: d837e000  r9 : d833be00  r8 : 00000000
>>> [   17.656887] r7 : 00000001  r6 : e003d000  r5 : 00000000  r4 : 00000000
>>> [   17.663447] r3 : 00000007  r2 : 00000000  r1 : 00000000  r0 : 00000000
>>> [   17.670009] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
>>> [   17.677180] Control: 10c5387d  Table: 8fe20019  DAC: 00000051
>>> [   17.682956] Process systemd-udevd (pid: 195, stack limit = 0x(ptrval))
>>> [   17.689518] Stack: (0xcfe1dd48 to 0xcfe1e000)
>>
>> Can you provide a full disassembly of sk_filter_trim_cap from vmlinux
>> (iow, annotated with its linked address) for the above dump please -
>> alternatively a new dump with matching disassembly.  Thanks.
> 
> Also probably a good idea to have bpf_jit_enable set to 2 to get a
> dump of the bpf program being run, which I think for your problem,
> you'll have to hack the kernel source to do that.

Agree, that would be good as well. You could use something like the below
to bail out to interpreter after JIT did the dump.

Dump will then land in kernel log which you could paste here.

diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index f6a62ae..d6a7dfd 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -1844,6 +1844,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
 		/* there are 2 passes here */
 		bpf_jit_dump(prog->len, image_size, 2, ctx.target);

+	/* Defer to interpreter after dump. */
+	if (1) {
+		bpf_jit_binary_free(header);
+		prog = orig_prog;
+		goto out_imms;
+	}
+
 	bpf_jit_binary_lock_ro(header);
 	prog->bpf_func = (void *)ctx.target;
 	prog->jited = 1;

^ permalink raw reply related	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-06-25 16:41           ` Peter Robinson
@ 2018-08-16 20:35             ` Marc Haber
  -1 siblings, 0 replies; 50+ messages in thread
From: Marc Haber @ 2018-08-16 20:35 UTC (permalink / raw)
  To: Peter Robinson
  Cc: linux-arm-kernel, netdev, labbott, Eric Dumazet, Daniel Borkmann

On Mon, Jun 25, 2018 at 05:41:27PM +0100, Peter Robinson wrote:
> So with that and the other fix there was no improvement, with those
> and the BPF JIT disabled it works, I'm not sure if the two patches
> have any effect with the JIT disabled though.

I can confirm the crash with the released 4.18.1 on Banana Pi, and I can
also confirm that disabling BPF JIT makes the Banana Pi work again.,

Greetings
Marc

[    0.004930] /cpus/cpu@0 missing clock-frequency property
[    0.004965] /cpus/cpu@1 missing clock-frequency property
[    4.959858] zswap: default zpool zbud not available
[    4.964820] zswap: pool creation failed
  WARNING: Failed to connect to lvmetad. Falling back to device scanning.
  WARNING: Failed to connect to lvmetad. Falling back to device scanning.
[   10.721077] Unable to handle kernel NULL pointer dereference at virtual address 0000000c
[   10.722949] Unable to handle kernel NULL pointer dereference at virtual address 0000000c
[   10.729288] pgd = (ptrval)
[   10.729299] [0000000c] *pgd=6dc65003, *pmd=00000000
[   10.737464] pgd = (ptrval)
[   10.740176] Internal error: Oops: a06 [#1] SMP ARM
[   10.745056] [0000000c] *pgd=6e72a003
[   10.747742] Modules linked in: ip_tables x_tables autofs4 btrfs
[   10.752561] , *pmd=00000000
[   10.756113]  libcrc32c crc32c_generic xor zstd_decompress zstd_compress xxhash
[   10.764833]  zlib_deflate raid6_pq dm_mod dax axp20x_regulator realtek ahci_sunxi dwmac_sunxi stmmac_platform libahci_platform stmmac i2c_mv64xxx libahci libata scsi_mod ohci_platform ohci_hcd ehci_platform ehci_hcd phy_sun4i_usb sunxi_mmc
[   10.793306] CPU: 1 PID: 238 Comm: systemd-udevd Not tainted 4.18.1-zgbpi-armmp-lpae #3
[   10.801212] Hardware name: Allwinner sun7i (A20) Family
[   10.806448] PC is at sk_filter_trim_cap+0xa0/0x1d4
[   10.811238] LR is at   (null)
[   10.814205] pc : [<c06de388>]    lr : [<00000000>]    psr: 600f0013
[   10.820466] sp : edc7dcf8  ip : 00000000  fp : edc7dd34
[   10.825686] r10: 00000000  r9 : 00000000  r8 : 00000000
[   10.830907] r7 : 00000001  r6 : f0e96000  r5 : c0e04cc8  r4 : 00000000
[   10.837428] r3 : 00000007  r2 : fb5e2d70  r1 : 00000000  r0 : 00000000
[   10.843952] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[   10.851081] Control: 30c5387d  Table: 6e6c7580  DAC: 2c983336
[   10.856822] Process systemd-udevd (pid: 238, stack limit = 0x(ptrval))
[   10.863344] Stack: (0xedc7dcf8 to 0xedc7e000)
[   10.867700] dce0:                                                       edc7dd1c edc7dd08
[   10.875873] dd00: c06a41dc c06a4048 ee7d39c0 fb5e2d70 ee479800 ee6c2400 edc33840 c0e6aac0
[   10.884046] dd20: 00000000 00000001 edc7dd8c edc7dd38 c0705884 c06de2f4 edc7de24 00000001
[   10.892219] dd40: c0ec649c ee479864 00000000 00000000 ee7d39c0 00000000 00000000 00000002
[   10.900391] dd60: 00000000 edc7df44 c0e04cc8 ee7d39c0 ee6c2400 00000000 0000008c 00000002
[   10.908565] dd80: edc7ddf4 edc7dd90 c0705ee0 c0705610 006000c0 00000000 00000000 fb5e2d70
[   10.916737] dda0: 00000008 00000000 00000000 ef357c80 00000000 000000ee 00000000 00000000
[   10.924910] ddc0: 00000000 fb5e2d70 0000008c edc7df44 eef08700 00000040 00000000 eef08700
[   10.933083] dde0: 00000000 edc7dedc edc7de0c edc7ddf8 c069b948 c0705b78 edc7df44 c0e04cc8
[   10.941256] de00: edc7df2c edc7de10 c069c2f8 c069b910 c0e04cc8 edc7dec0 00000000 be8dcfac
[   10.949428] de20: 00000028 0186a660 00000064 bf387954 edc7df48 be8dcf80 00000000 00000000
[   10.957602] de40: be8dcf80 b6f19ce8 00000128 40000028 b6e01346 00000000 0000000e 00000010
[   10.965774] de60: 00000000 00000002 00000000 00000000 00000000 00000000 be8dcf80 00000000
[   10.973948] de80: b6f19ce8 00000000 00000000 fb5e2d70 edc7deb4 ffffe000 00000000 c0e04cc8
[   10.982120] dea0: 00000128 c0201204 00000000 00000080 edc7df6c edc7dec0 c02f5e2c c02f5c18
[   10.990293] dec0: 00000000 fb5e2d70 edc7def4 a0010013 c9f1e000 c03f986c edc7df50 00000000
[   10.998466] dee0: 0000000e 00004000 edc7df3c fb5e2d70 c0409c98 c0409d34 edc7df14 fb5e2d70
[   11.006639] df00: c0409d34 c0e04cc8 be8dcf80 00000000 eef08700 c0201204 edc7c000 00000128
[   11.014812] df20: edc7df94 edc7df30 c069d818 c069c0a0 00000000 00000000 c0e04cc8 00000000
[   11.022984] df40: fffffff7 edc7de5c 0000000c 00000001 00000000 00000000 edc7de2c 00000000
[   11.031156] df60: edc7df7c 00000000 00000000 00000040 00000000 fb5e2d70 be8dcf80 b6f19ce8
[   11.039329] df80: 01878670 00000128 edc7dfa4 edc7df98 c069d870 c069d7c4 00000000 edc7dfa8
[   11.047502] dfa0: c02011cc c069d860 be8dcf80 b6f19ce8 0000000e be8dcf80 00000000 00000000
[   11.055675] dfc0: be8dcf80 b6f19ce8 01878670 00000128 00000000 00000064 01878e80 00000000
[   11.063848] dfe0: 00000128 be8dcf50 b6e003e3 b6e01346 200f0030 0000000e 00000000 00000000
[   11.072038] [<c06de388>] (sk_filter_trim_cap) from [<c0705884>] (netlink_broadcast_filtered+0x280/0x460)
[   11.081517] [<c0705884>] (netlink_broadcast_filtered) from [<c0705ee0>] (netlink_sendmsg+0x374/0x3b0)
[   11.090734] [<c0705ee0>] (netlink_sendmsg) from [<c069b948>] (sock_sendmsg+0x44/0x54)
[   11.098567] [<c069b948>] (sock_sendmsg) from [<c069c2f8>] (___sys_sendmsg+0x264/0x278)
[   11.106485] [<c069c2f8>] (___sys_sendmsg) from [<c069d818>] (__sys_sendmsg+0x60/0x9c)
[   11.114315] [<c069d818>] (__sys_sendmsg) from [<c069d870>] (sys_sendmsg+0x1c/0x20)
[   11.121886] [<c069d870>] (sys_sendmsg) from [<c02011cc>] (__sys_trace_return+0x0/0x10)
[   11.129793] Exception stack(0xedc7dfa8 to 0xedc7dff0)
[   11.134845] dfa0:                   be8dcf80 b6f19ce8 0000000e be8dcf80 00000000 00000000
[   11.143019] dfc0: be8dcf80 b6f19ce8 01878670 00000128 00000000 00000064 01878e80 00000000
[   11.151188] dfe0: 00000128 be8dcf50 b6e003e3 b6e01346
[   11.156243] Code: e3130010 e1a0c000 1a000030 e35c0000 (e584900c) 
[   11.162340] Internal error: Oops: a06 [#2] SMP ARM
[   11.162559] ---[ end trace 1b60255ae59ac006 ]---
[   11.167129] Modules linked in: ip_tables x_tables autofs4 btrfs libcrc32c crc32c_generic xor zstd_decompress zstd_compress xxhash zlib_deflate raid6_pq dm_mod dax axp20x_regulator realtek ahci_sunxi dwmac_sunxi stmmac_platform libahci_platform stmmac i2c_mv64xxx libahci libata scsi_mod ohci_platform ohci_hcd ehci_platform ehci_hcd phy_sun4i_usb sunxi_mmc
[   11.185005] Unable to handle kernel NULL pointer dereference at virtual address 0000000c
[   11.203186] CPU: 0 PID: 237 Comm: systemd-udevd Tainted: G      D           4.18.1-zgbpi-armmp-lpae #3
[   11.203191] Hardware name: Allwinner sun7i (A20) Family
[   11.203216] PC is at sk_filter_trim_cap+0xa0/0x1d4
[   11.203223] LR is at   (null)
[   11.203229] pc : [<c06de388>]    lr : [<00000000>]    psr: 600f0013
[   11.203234] sp : edc41cf8  ip : 00000000  fp : edc41d34
[   11.203239] r10: 00000000  r9 : 00000000  r8 : 00000000
[   11.203245] r7 : 00000001  r6 : f0e96000  r5 : c0e04cc8  r4 : 00000000
[   11.203250] r3 : 00000007  r2 : fb5e2d70  r1 : 00000000  r0 : 00000000
[   11.203258] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[   11.203264] Control: 30c5387d  Table: 6e6c84c0  DAC: fffffffd
[   11.203270] Process systemd-udevd (pid: 237, stack limit = 0x(ptrval))
[   11.203276] Stack: (0xedc41cf8 to 0xedc42000)
[   11.203288] 1ce0:                                                       edc41d1c edc41d08
[   11.211398] pgd = (ptrval)
[   11.220660] 1d00: c06a41dc c06a4048 c9c16cc0 fb5e2d70 ee479800 ee6c6400 c9c16240 c0e6aac0
[   11.220670] 1d20: 00000000 00000001 edc41d8c edc41d38 c0705884 c06de2f4 edc41e24 00000001
[   11.220680] 1d40: c0ec649c ee479864 00000000 00000000 c9c16cc0 00000000 00000000 00000002
[   11.220693] 1d60: 00000000 edc41f44 c0e04cc8 c9c16cc0 ee6c6400 00000000 00000085 00000002
[   11.226034] [0000000c] *pgd=6dc79003
[   11.230697] 1d80: edc41df4 edc41d90 c0705ee0 c0705610 006000c0 00000000 00000000 fb5e2d70
[   11.230707] 1da0: 00000008 00000000 00000000 ef357300 00000000 000000ed 00000000 00000000
[   11.230717] 1dc0: 00000000 fb5e2d70 00000085 edc41f44 ee0591c0 00000040 00000000 ee0591c0
[   11.230730] 1de0: 00000000 edc41edc edc41e0c edc41df8 c069b948 c0705b78 edc41f44 c0e04cc8
[   11.233692] , *pmd=00000000
[   11.239953] 1e00: edc41f2c edc41e10 c069c2f8 c069b910 c0e04cc8 edc41ec0 00000000 be8dcfac
[   11.239963] 1e20: 00000028 0186a660 0000005d bf387954 edc41f48 be8dcf80 00000000 00000000
[   11.239973] 1e40: be8dcf80 b6f19ce8 00000128 40000028 b6e01346 00000000 0000000d 00000010
[   11.239982] 1e60: 00000000 00000002 00000000 00000000 00000000 00000000 be8dcf80 00000000
[   11.239992] 1e80: b6f19ce8 00000000 00000000 fb5e2d70 edc41eb4 ffffe000 00000000 c0e04cc8
[   11.240002] 1ea0: 00000128 c0201204 00000000 00000080 edc41f6c edc41ec0 c02f5e2c c02f5c18
[   11.250433] 1ec0: 00000000 fb5e2d70 edc41ef4 a0010013 c9def000 c03f986c edc41f50 00000000
[   11.250443] 1ee0: 0000000d 00004000 edc41f3c fb5e2d70 c0409c98 c0409d34 edc41f14 fb5e2d70
[   11.250454] 1f00: c0409d34 c0e04cc8 be8dcf80 00000000 ee0591c0 c0201204 edc40000 00000128
[   11.250463] 1f20: edc41f94 edc41f30 c069d818 c069c0a0 00000000 00000000 c0e04cc8 00000000
[   11.451342] 1f40: fffffff7 edc41e5c 0000000c 00000001 00000000 00000000 edc41e2c 00000000
[   11.459515] 1f60: edc41f7c 00000000 00000000 00000040 00000000 fb5e2d70 be8dcf80 b6f19ce8
[   11.467688] 1f80: 0186d740 00000128 edc41fa4 edc41f98 c069d870 c069d7c4 00000000 edc41fa8
[   11.475861] 1fa0: c02011cc c069d860 be8dcf80 b6f19ce8 0000000d be8dcf80 00000000 00000000
[   11.484034] 1fc0: be8dcf80 b6f19ce8 0186d740 00000128 00000000 0000005d 018776c0 00000000
[   11.492207] 1fe0: 00000128 be8dcf50 b6e003e3 b6e01346 200f0030 0000000d 00000000 00000000
[   11.500397] [<c06de388>] (sk_filter_trim_cap) from [<c0705884>] (netlink_broadcast_filtered+0x280/0x460)
[   11.509876] [<c0705884>] (netlink_broadcast_filtered) from [<c0705ee0>] (netlink_sendmsg+0x374/0x3b0)
[   11.519093] [<c0705ee0>] (netlink_sendmsg) from [<c069b948>] (sock_sendmsg+0x44/0x54)
[   11.526925] [<c069b948>] (sock_sendmsg) from [<c069c2f8>] (___sys_sendmsg+0x264/0x278)
[   11.534842] [<c069c2f8>] (___sys_sendmsg) from [<c069d818>] (__sys_sendmsg+0x60/0x9c)
[   11.542673] [<c069d818>] (__sys_sendmsg) from [<c069d870>] (sys_sendmsg+0x1c/0x20)
[   11.550244] [<c069d870>] (sys_sendmsg) from [<c02011cc>] (__sys_trace_return+0x0/0x10)
[   11.558151] Exception stack(0xedc41fa8 to 0xedc41ff0)
[   11.563202] 1fa0:                   be8dcf80 b6f19ce8 0000000d be8dcf80 00000000 00000000
[   11.571375] 1fc0: be8dcf80 b6f19ce8 0186d740 00000128 00000000 0000005d 018776c0 00000000
[   11.579544] 1fe0: 00000128 be8dcf50 b6e003e3 b6e01346
[   11.584600] Code: e3130010 e1a0c000 1a000030 e35c0000 (e584900c) 
[   11.590702] Internal error: Oops: a06 [#3] SMP ARM
[   11.590859] ---[ end trace 1b60255ae59ac007 ]---
[   11.595493] Modules linked in: ip_tables x_tables autofs4 btrfs libcrc32c crc32c_generic xor zstd_decompress zstd_compress xxhash zlib_deflate raid6_pq dm_mod dax axp20x_regulator realtek ahci_sunxi dwmac_sunxi stmmac_platform libahci_platform stmmac i2c_mv64xxx libahci libata scsi_mod ohci_platform ohci_hcd ehci_platform ehci_hcd phy_sun4i_usb sunxi_mmc
[   11.602116] Unable to handle kernel NULL pointer dereference at virtual address 0000000c
[   11.631550] CPU: 1 PID: 240 Comm: systemd-udevd Tainted: G      D           4.18.1-zgbpi-armmp-lpae #3
[   11.631555] Hardware name: Allwinner sun7i (A20) Family
[   11.631576] PC is at sk_filter_trim_cap+0xa0/0x1d4
[   11.631582] LR is at   (null)
[   11.631593] pc : [<c06de388>]    lr : [<00000000>]    psr: 600f0013
[   11.639693] pgd = (ptrval)
[   11.648959] sp : edc81cf8  ip : 00000000  fp : edc81d34
[   11.648964] r10: 00000000  r9 : 00000000  r8 : 00000000
[   11.648970] r7 : 00000001  r6 : f0e96000  r5 : c0e04cc8  r4 : 00000000
[   11.648976] r3 : 00000007  r2 : fb5e2d70  r1 : 00000000  r0 : 00000000
[   11.648983] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[   11.648990] Control: 30c5387d  Table: 6e71a180  DAC: 2c983336
[   11.654224] [0000000c] *pgd=6dc6e003
[   11.658989] Process systemd-udevd (pid: 240, stack limit = 0x(ptrval))
[   11.658995] Stack: (0xedc81cf8 to 0xedc82000)
[   11.659002] 1ce0:                                                       edc81d1c edc81d08
[   11.659013] 1d00: c06a41dc c06a4048 ee7d36c0 fb5e2d70 ee479800 edc77800 ee7d3d80 c0e6aac0
[   11.661987] , *pmd=00000000
[   11.668231] 1d20: 00000000 00000001 edc81d8c edc81d38 c0705884 c06de2f4 edc81e24 00000001
[   11.668241] 1d40: c0ec649c ee479864 00000000 00000000 ee7d36c0 00000000 00000000 00000002
[   11.668251] 1d60: 00000000 edc81f44 c0e04cc8 ee7d36c0 edc77800 00000000 0000008a 00000002
[   11.676168] 1d80: edc81df4 edc81d90 c0705ee0 c0705610 006000c0 00000000 00000000 fb5e2d70
[   11.676178] 1da0: 00000008 00000000 00000000 ef0ea980 00000000 000000f0 00000000 00000000
[   11.676188] 1dc0: 00000000 fb5e2d70 0000008a edc81f44 ee059a80 00000040 00000000 ee059a80
[   11.789803] 1de0: 00000000 edc81edc edc81e0c edc81df8 c069b948 c0705b78 edc81f44 c0e04cc8
[   11.797977] 1e00: edc81f2c edc81e10 c069c2f8 c069b910 c0e04cc8 edc81ec0 00000000 be8dcfac
[   11.806149] 1e20: 00000028 0186ade8 00000062 bf387954 edc81f48 be8dcf80 00000000 00000000
[   11.814322] 1e40: be8dcf80 b6f19ce8 00000128 40000028 b6e01346 00000000 0000000e 00000010
[   11.822494] 1e60: 00000000 00000002 00000000 00000000 00000000 00000000 be8dcf80 00000000
[   11.830667] 1e80: b6f19ce8 00000000 00000000 fb5e2d70 edc81eb4 ffffe000 00000000 c0e04cc8
[   11.838840] 1ea0: 00000128 c0201204 00000000 00000080 edc81f6c edc81ec0 c02f5e2c c02f5c18
[   11.847013] 1ec0: 00000000 fb5e2d70 edc81ef4 a00b0013 ef3c3000 c03f986c edc81f50 00000000
[   11.855186] 1ee0: 0000000e 00004000 edc81f3c fb5e2d70 c0409c98 c0409d34 edc81f14 fb5e2d70
[   11.863359] 1f00: c0409d34 c0e04cc8 be8dcf80 00000000 ee059a80 c0201204 edc80000 00000128
[   11.871532] 1f20: edc81f94 edc81f30 c069d818 c069c0a0 00000000 00000000 c0e04cc8 00000000
[   11.879705] 1f40: fffffff7 edc81e5c 0000000c 00000001 00000000 00000000 edc81e2c 00000000
[   11.887877] 1f60: edc81f7c 00000000 00000000 00000040 00000000 fb5e2d70 be8dcf80 b6f19ce8
[   11.896051] 1f80: 0186aea0 00000128 edc81fa4 edc81f98 c069d870 c069d7c4 00000000 edc81fa8
[   11.904223] 1fa0: c02011cc c069d860 be8dcf80 b6f19ce8 0000000e be8dcf80 00000000 00000000
[   11.912397] 1fc0: be8dcf80 b6f19ce8 0186aea0 00000128 00000000 00000062 0186b6e8 00000000
[   11.920569] 1fe0: 00000128 be8dcf50 b6e003e3 b6e01346 200f0030 0000000e 00000000 00000000
[   11.928757] [<c06de388>] (sk_filter_trim_cap) from [<c0705884>] (netlink_broadcast_filtered+0x280/0x460)
[   11.938235] [<c0705884>] (netlink_broadcast_filtered) from [<c0705ee0>] (netlink_sendmsg+0x374/0x3b0)
[   11.947452] [<c0705ee0>] (netlink_sendmsg) from [<c069b948>] (sock_sendmsg+0x44/0x54)
[   11.955284] [<c069b948>] (sock_sendmsg) from [<c069c2f8>] (___sys_sendmsg+0x264/0x278)
[   11.963201] [<c069c2f8>] (___sys_sendmsg) from [<c069d818>] (__sys_sendmsg+0x60/0x9c)
[   11.971031] [<c069d818>] (__sys_sendmsg) from [<c069d870>] (sys_sendmsg+0x1c/0x20)
[   11.978602] [<c069d870>] (sys_sendmsg) from [<c02011cc>] (__sys_trace_return+0x0/0x10)
[   11.986509] Exception stack(0xedc81fa8 to 0xedc81ff0)
[   11.991560] 1fa0:                   be8dcf80 b6f19ce8 0000000e be8dcf80 00000000 00000000
[   11.999732] 1fc0: be8dcf80 b6f19ce8 0186aea0 00000128 00000000 00000062 0186b6e8 00000000
[   12.007902] 1fe0: 00000128 be8dcf50 b6e003e3 b6e01346
[   12.012957] Code: e3130010 e1a0c000 1a000030 e35c0000 (e584900c) 
[   12.019056] Internal error: Oops: a06 [#4] SMP ARM
[   12.019171] ---[ end trace 1b60255ae59ac008 ]---


-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

^ permalink raw reply	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-08-16 20:35             ` Marc Haber
  0 siblings, 0 replies; 50+ messages in thread
From: Marc Haber @ 2018-08-16 20:35 UTC (permalink / raw)
  To: linux-arm-kernel

On Mon, Jun 25, 2018 at 05:41:27PM +0100, Peter Robinson wrote:
> So with that and the other fix there was no improvement, with those
> and the BPF JIT disabled it works, I'm not sure if the two patches
> have any effect with the JIT disabled though.

I can confirm the crash with the released 4.18.1 on Banana Pi, and I can
also confirm that disabling BPF JIT makes the Banana Pi work again.,

Greetings
Marc

[    0.004930] /cpus/cpu at 0 missing clock-frequency property
[    0.004965] /cpus/cpu at 1 missing clock-frequency property
[    4.959858] zswap: default zpool zbud not available
[    4.964820] zswap: pool creation failed
  WARNING: Failed to connect to lvmetad. Falling back to device scanning.
  WARNING: Failed to connect to lvmetad. Falling back to device scanning.
[   10.721077] Unable to handle kernel NULL pointer dereference at virtual address 0000000c
[   10.722949] Unable to handle kernel NULL pointer dereference at virtual address 0000000c
[   10.729288] pgd = (ptrval)
[   10.729299] [0000000c] *pgd=6dc65003, *pmd=00000000
[   10.737464] pgd = (ptrval)
[   10.740176] Internal error: Oops: a06 [#1] SMP ARM
[   10.745056] [0000000c] *pgd=6e72a003
[   10.747742] Modules linked in: ip_tables x_tables autofs4 btrfs
[   10.752561] , *pmd=00000000
[   10.756113]  libcrc32c crc32c_generic xor zstd_decompress zstd_compress xxhash
[   10.764833]  zlib_deflate raid6_pq dm_mod dax axp20x_regulator realtek ahci_sunxi dwmac_sunxi stmmac_platform libahci_platform stmmac i2c_mv64xxx libahci libata scsi_mod ohci_platform ohci_hcd ehci_platform ehci_hcd phy_sun4i_usb sunxi_mmc
[   10.793306] CPU: 1 PID: 238 Comm: systemd-udevd Not tainted 4.18.1-zgbpi-armmp-lpae #3
[   10.801212] Hardware name: Allwinner sun7i (A20) Family
[   10.806448] PC is at sk_filter_trim_cap+0xa0/0x1d4
[   10.811238] LR is at   (null)
[   10.814205] pc : [<c06de388>]    lr : [<00000000>]    psr: 600f0013
[   10.820466] sp : edc7dcf8  ip : 00000000  fp : edc7dd34
[   10.825686] r10: 00000000  r9 : 00000000  r8 : 00000000
[   10.830907] r7 : 00000001  r6 : f0e96000  r5 : c0e04cc8  r4 : 00000000
[   10.837428] r3 : 00000007  r2 : fb5e2d70  r1 : 00000000  r0 : 00000000
[   10.843952] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[   10.851081] Control: 30c5387d  Table: 6e6c7580  DAC: 2c983336
[   10.856822] Process systemd-udevd (pid: 238, stack limit = 0x(ptrval))
[   10.863344] Stack: (0xedc7dcf8 to 0xedc7e000)
[   10.867700] dce0:                                                       edc7dd1c edc7dd08
[   10.875873] dd00: c06a41dc c06a4048 ee7d39c0 fb5e2d70 ee479800 ee6c2400 edc33840 c0e6aac0
[   10.884046] dd20: 00000000 00000001 edc7dd8c edc7dd38 c0705884 c06de2f4 edc7de24 00000001
[   10.892219] dd40: c0ec649c ee479864 00000000 00000000 ee7d39c0 00000000 00000000 00000002
[   10.900391] dd60: 00000000 edc7df44 c0e04cc8 ee7d39c0 ee6c2400 00000000 0000008c 00000002
[   10.908565] dd80: edc7ddf4 edc7dd90 c0705ee0 c0705610 006000c0 00000000 00000000 fb5e2d70
[   10.916737] dda0: 00000008 00000000 00000000 ef357c80 00000000 000000ee 00000000 00000000
[   10.924910] ddc0: 00000000 fb5e2d70 0000008c edc7df44 eef08700 00000040 00000000 eef08700
[   10.933083] dde0: 00000000 edc7dedc edc7de0c edc7ddf8 c069b948 c0705b78 edc7df44 c0e04cc8
[   10.941256] de00: edc7df2c edc7de10 c069c2f8 c069b910 c0e04cc8 edc7dec0 00000000 be8dcfac
[   10.949428] de20: 00000028 0186a660 00000064 bf387954 edc7df48 be8dcf80 00000000 00000000
[   10.957602] de40: be8dcf80 b6f19ce8 00000128 40000028 b6e01346 00000000 0000000e 00000010
[   10.965774] de60: 00000000 00000002 00000000 00000000 00000000 00000000 be8dcf80 00000000
[   10.973948] de80: b6f19ce8 00000000 00000000 fb5e2d70 edc7deb4 ffffe000 00000000 c0e04cc8
[   10.982120] dea0: 00000128 c0201204 00000000 00000080 edc7df6c edc7dec0 c02f5e2c c02f5c18
[   10.990293] dec0: 00000000 fb5e2d70 edc7def4 a0010013 c9f1e000 c03f986c edc7df50 00000000
[   10.998466] dee0: 0000000e 00004000 edc7df3c fb5e2d70 c0409c98 c0409d34 edc7df14 fb5e2d70
[   11.006639] df00: c0409d34 c0e04cc8 be8dcf80 00000000 eef08700 c0201204 edc7c000 00000128
[   11.014812] df20: edc7df94 edc7df30 c069d818 c069c0a0 00000000 00000000 c0e04cc8 00000000
[   11.022984] df40: fffffff7 edc7de5c 0000000c 00000001 00000000 00000000 edc7de2c 00000000
[   11.031156] df60: edc7df7c 00000000 00000000 00000040 00000000 fb5e2d70 be8dcf80 b6f19ce8
[   11.039329] df80: 01878670 00000128 edc7dfa4 edc7df98 c069d870 c069d7c4 00000000 edc7dfa8
[   11.047502] dfa0: c02011cc c069d860 be8dcf80 b6f19ce8 0000000e be8dcf80 00000000 00000000
[   11.055675] dfc0: be8dcf80 b6f19ce8 01878670 00000128 00000000 00000064 01878e80 00000000
[   11.063848] dfe0: 00000128 be8dcf50 b6e003e3 b6e01346 200f0030 0000000e 00000000 00000000
[   11.072038] [<c06de388>] (sk_filter_trim_cap) from [<c0705884>] (netlink_broadcast_filtered+0x280/0x460)
[   11.081517] [<c0705884>] (netlink_broadcast_filtered) from [<c0705ee0>] (netlink_sendmsg+0x374/0x3b0)
[   11.090734] [<c0705ee0>] (netlink_sendmsg) from [<c069b948>] (sock_sendmsg+0x44/0x54)
[   11.098567] [<c069b948>] (sock_sendmsg) from [<c069c2f8>] (___sys_sendmsg+0x264/0x278)
[   11.106485] [<c069c2f8>] (___sys_sendmsg) from [<c069d818>] (__sys_sendmsg+0x60/0x9c)
[   11.114315] [<c069d818>] (__sys_sendmsg) from [<c069d870>] (sys_sendmsg+0x1c/0x20)
[   11.121886] [<c069d870>] (sys_sendmsg) from [<c02011cc>] (__sys_trace_return+0x0/0x10)
[   11.129793] Exception stack(0xedc7dfa8 to 0xedc7dff0)
[   11.134845] dfa0:                   be8dcf80 b6f19ce8 0000000e be8dcf80 00000000 00000000
[   11.143019] dfc0: be8dcf80 b6f19ce8 01878670 00000128 00000000 00000064 01878e80 00000000
[   11.151188] dfe0: 00000128 be8dcf50 b6e003e3 b6e01346
[   11.156243] Code: e3130010 e1a0c000 1a000030 e35c0000 (e584900c) 
[   11.162340] Internal error: Oops: a06 [#2] SMP ARM
[   11.162559] ---[ end trace 1b60255ae59ac006 ]---
[   11.167129] Modules linked in: ip_tables x_tables autofs4 btrfs libcrc32c crc32c_generic xor zstd_decompress zstd_compress xxhash zlib_deflate raid6_pq dm_mod dax axp20x_regulator realtek ahci_sunxi dwmac_sunxi stmmac_platform libahci_platform stmmac i2c_mv64xxx libahci libata scsi_mod ohci_platform ohci_hcd ehci_platform ehci_hcd phy_sun4i_usb sunxi_mmc
[   11.185005] Unable to handle kernel NULL pointer dereference at virtual address 0000000c
[   11.203186] CPU: 0 PID: 237 Comm: systemd-udevd Tainted: G      D           4.18.1-zgbpi-armmp-lpae #3
[   11.203191] Hardware name: Allwinner sun7i (A20) Family
[   11.203216] PC is at sk_filter_trim_cap+0xa0/0x1d4
[   11.203223] LR is at   (null)
[   11.203229] pc : [<c06de388>]    lr : [<00000000>]    psr: 600f0013
[   11.203234] sp : edc41cf8  ip : 00000000  fp : edc41d34
[   11.203239] r10: 00000000  r9 : 00000000  r8 : 00000000
[   11.203245] r7 : 00000001  r6 : f0e96000  r5 : c0e04cc8  r4 : 00000000
[   11.203250] r3 : 00000007  r2 : fb5e2d70  r1 : 00000000  r0 : 00000000
[   11.203258] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[   11.203264] Control: 30c5387d  Table: 6e6c84c0  DAC: fffffffd
[   11.203270] Process systemd-udevd (pid: 237, stack limit = 0x(ptrval))
[   11.203276] Stack: (0xedc41cf8 to 0xedc42000)
[   11.203288] 1ce0:                                                       edc41d1c edc41d08
[   11.211398] pgd = (ptrval)
[   11.220660] 1d00: c06a41dc c06a4048 c9c16cc0 fb5e2d70 ee479800 ee6c6400 c9c16240 c0e6aac0
[   11.220670] 1d20: 00000000 00000001 edc41d8c edc41d38 c0705884 c06de2f4 edc41e24 00000001
[   11.220680] 1d40: c0ec649c ee479864 00000000 00000000 c9c16cc0 00000000 00000000 00000002
[   11.220693] 1d60: 00000000 edc41f44 c0e04cc8 c9c16cc0 ee6c6400 00000000 00000085 00000002
[   11.226034] [0000000c] *pgd=6dc79003
[   11.230697] 1d80: edc41df4 edc41d90 c0705ee0 c0705610 006000c0 00000000 00000000 fb5e2d70
[   11.230707] 1da0: 00000008 00000000 00000000 ef357300 00000000 000000ed 00000000 00000000
[   11.230717] 1dc0: 00000000 fb5e2d70 00000085 edc41f44 ee0591c0 00000040 00000000 ee0591c0
[   11.230730] 1de0: 00000000 edc41edc edc41e0c edc41df8 c069b948 c0705b78 edc41f44 c0e04cc8
[   11.233692] , *pmd=00000000
[   11.239953] 1e00: edc41f2c edc41e10 c069c2f8 c069b910 c0e04cc8 edc41ec0 00000000 be8dcfac
[   11.239963] 1e20: 00000028 0186a660 0000005d bf387954 edc41f48 be8dcf80 00000000 00000000
[   11.239973] 1e40: be8dcf80 b6f19ce8 00000128 40000028 b6e01346 00000000 0000000d 00000010
[   11.239982] 1e60: 00000000 00000002 00000000 00000000 00000000 00000000 be8dcf80 00000000
[   11.239992] 1e80: b6f19ce8 00000000 00000000 fb5e2d70 edc41eb4 ffffe000 00000000 c0e04cc8
[   11.240002] 1ea0: 00000128 c0201204 00000000 00000080 edc41f6c edc41ec0 c02f5e2c c02f5c18
[   11.250433] 1ec0: 00000000 fb5e2d70 edc41ef4 a0010013 c9def000 c03f986c edc41f50 00000000
[   11.250443] 1ee0: 0000000d 00004000 edc41f3c fb5e2d70 c0409c98 c0409d34 edc41f14 fb5e2d70
[   11.250454] 1f00: c0409d34 c0e04cc8 be8dcf80 00000000 ee0591c0 c0201204 edc40000 00000128
[   11.250463] 1f20: edc41f94 edc41f30 c069d818 c069c0a0 00000000 00000000 c0e04cc8 00000000
[   11.451342] 1f40: fffffff7 edc41e5c 0000000c 00000001 00000000 00000000 edc41e2c 00000000
[   11.459515] 1f60: edc41f7c 00000000 00000000 00000040 00000000 fb5e2d70 be8dcf80 b6f19ce8
[   11.467688] 1f80: 0186d740 00000128 edc41fa4 edc41f98 c069d870 c069d7c4 00000000 edc41fa8
[   11.475861] 1fa0: c02011cc c069d860 be8dcf80 b6f19ce8 0000000d be8dcf80 00000000 00000000
[   11.484034] 1fc0: be8dcf80 b6f19ce8 0186d740 00000128 00000000 0000005d 018776c0 00000000
[   11.492207] 1fe0: 00000128 be8dcf50 b6e003e3 b6e01346 200f0030 0000000d 00000000 00000000
[   11.500397] [<c06de388>] (sk_filter_trim_cap) from [<c0705884>] (netlink_broadcast_filtered+0x280/0x460)
[   11.509876] [<c0705884>] (netlink_broadcast_filtered) from [<c0705ee0>] (netlink_sendmsg+0x374/0x3b0)
[   11.519093] [<c0705ee0>] (netlink_sendmsg) from [<c069b948>] (sock_sendmsg+0x44/0x54)
[   11.526925] [<c069b948>] (sock_sendmsg) from [<c069c2f8>] (___sys_sendmsg+0x264/0x278)
[   11.534842] [<c069c2f8>] (___sys_sendmsg) from [<c069d818>] (__sys_sendmsg+0x60/0x9c)
[   11.542673] [<c069d818>] (__sys_sendmsg) from [<c069d870>] (sys_sendmsg+0x1c/0x20)
[   11.550244] [<c069d870>] (sys_sendmsg) from [<c02011cc>] (__sys_trace_return+0x0/0x10)
[   11.558151] Exception stack(0xedc41fa8 to 0xedc41ff0)
[   11.563202] 1fa0:                   be8dcf80 b6f19ce8 0000000d be8dcf80 00000000 00000000
[   11.571375] 1fc0: be8dcf80 b6f19ce8 0186d740 00000128 00000000 0000005d 018776c0 00000000
[   11.579544] 1fe0: 00000128 be8dcf50 b6e003e3 b6e01346
[   11.584600] Code: e3130010 e1a0c000 1a000030 e35c0000 (e584900c) 
[   11.590702] Internal error: Oops: a06 [#3] SMP ARM
[   11.590859] ---[ end trace 1b60255ae59ac007 ]---
[   11.595493] Modules linked in: ip_tables x_tables autofs4 btrfs libcrc32c crc32c_generic xor zstd_decompress zstd_compress xxhash zlib_deflate raid6_pq dm_mod dax axp20x_regulator realtek ahci_sunxi dwmac_sunxi stmmac_platform libahci_platform stmmac i2c_mv64xxx libahci libata scsi_mod ohci_platform ohci_hcd ehci_platform ehci_hcd phy_sun4i_usb sunxi_mmc
[   11.602116] Unable to handle kernel NULL pointer dereference at virtual address 0000000c
[   11.631550] CPU: 1 PID: 240 Comm: systemd-udevd Tainted: G      D           4.18.1-zgbpi-armmp-lpae #3
[   11.631555] Hardware name: Allwinner sun7i (A20) Family
[   11.631576] PC is at sk_filter_trim_cap+0xa0/0x1d4
[   11.631582] LR is at   (null)
[   11.631593] pc : [<c06de388>]    lr : [<00000000>]    psr: 600f0013
[   11.639693] pgd = (ptrval)
[   11.648959] sp : edc81cf8  ip : 00000000  fp : edc81d34
[   11.648964] r10: 00000000  r9 : 00000000  r8 : 00000000
[   11.648970] r7 : 00000001  r6 : f0e96000  r5 : c0e04cc8  r4 : 00000000
[   11.648976] r3 : 00000007  r2 : fb5e2d70  r1 : 00000000  r0 : 00000000
[   11.648983] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[   11.648990] Control: 30c5387d  Table: 6e71a180  DAC: 2c983336
[   11.654224] [0000000c] *pgd=6dc6e003
[   11.658989] Process systemd-udevd (pid: 240, stack limit = 0x(ptrval))
[   11.658995] Stack: (0xedc81cf8 to 0xedc82000)
[   11.659002] 1ce0:                                                       edc81d1c edc81d08
[   11.659013] 1d00: c06a41dc c06a4048 ee7d36c0 fb5e2d70 ee479800 edc77800 ee7d3d80 c0e6aac0
[   11.661987] , *pmd=00000000
[   11.668231] 1d20: 00000000 00000001 edc81d8c edc81d38 c0705884 c06de2f4 edc81e24 00000001
[   11.668241] 1d40: c0ec649c ee479864 00000000 00000000 ee7d36c0 00000000 00000000 00000002
[   11.668251] 1d60: 00000000 edc81f44 c0e04cc8 ee7d36c0 edc77800 00000000 0000008a 00000002
[   11.676168] 1d80: edc81df4 edc81d90 c0705ee0 c0705610 006000c0 00000000 00000000 fb5e2d70
[   11.676178] 1da0: 00000008 00000000 00000000 ef0ea980 00000000 000000f0 00000000 00000000
[   11.676188] 1dc0: 00000000 fb5e2d70 0000008a edc81f44 ee059a80 00000040 00000000 ee059a80
[   11.789803] 1de0: 00000000 edc81edc edc81e0c edc81df8 c069b948 c0705b78 edc81f44 c0e04cc8
[   11.797977] 1e00: edc81f2c edc81e10 c069c2f8 c069b910 c0e04cc8 edc81ec0 00000000 be8dcfac
[   11.806149] 1e20: 00000028 0186ade8 00000062 bf387954 edc81f48 be8dcf80 00000000 00000000
[   11.814322] 1e40: be8dcf80 b6f19ce8 00000128 40000028 b6e01346 00000000 0000000e 00000010
[   11.822494] 1e60: 00000000 00000002 00000000 00000000 00000000 00000000 be8dcf80 00000000
[   11.830667] 1e80: b6f19ce8 00000000 00000000 fb5e2d70 edc81eb4 ffffe000 00000000 c0e04cc8
[   11.838840] 1ea0: 00000128 c0201204 00000000 00000080 edc81f6c edc81ec0 c02f5e2c c02f5c18
[   11.847013] 1ec0: 00000000 fb5e2d70 edc81ef4 a00b0013 ef3c3000 c03f986c edc81f50 00000000
[   11.855186] 1ee0: 0000000e 00004000 edc81f3c fb5e2d70 c0409c98 c0409d34 edc81f14 fb5e2d70
[   11.863359] 1f00: c0409d34 c0e04cc8 be8dcf80 00000000 ee059a80 c0201204 edc80000 00000128
[   11.871532] 1f20: edc81f94 edc81f30 c069d818 c069c0a0 00000000 00000000 c0e04cc8 00000000
[   11.879705] 1f40: fffffff7 edc81e5c 0000000c 00000001 00000000 00000000 edc81e2c 00000000
[   11.887877] 1f60: edc81f7c 00000000 00000000 00000040 00000000 fb5e2d70 be8dcf80 b6f19ce8
[   11.896051] 1f80: 0186aea0 00000128 edc81fa4 edc81f98 c069d870 c069d7c4 00000000 edc81fa8
[   11.904223] 1fa0: c02011cc c069d860 be8dcf80 b6f19ce8 0000000e be8dcf80 00000000 00000000
[   11.912397] 1fc0: be8dcf80 b6f19ce8 0186aea0 00000128 00000000 00000062 0186b6e8 00000000
[   11.920569] 1fe0: 00000128 be8dcf50 b6e003e3 b6e01346 200f0030 0000000e 00000000 00000000
[   11.928757] [<c06de388>] (sk_filter_trim_cap) from [<c0705884>] (netlink_broadcast_filtered+0x280/0x460)
[   11.938235] [<c0705884>] (netlink_broadcast_filtered) from [<c0705ee0>] (netlink_sendmsg+0x374/0x3b0)
[   11.947452] [<c0705ee0>] (netlink_sendmsg) from [<c069b948>] (sock_sendmsg+0x44/0x54)
[   11.955284] [<c069b948>] (sock_sendmsg) from [<c069c2f8>] (___sys_sendmsg+0x264/0x278)
[   11.963201] [<c069c2f8>] (___sys_sendmsg) from [<c069d818>] (__sys_sendmsg+0x60/0x9c)
[   11.971031] [<c069d818>] (__sys_sendmsg) from [<c069d870>] (sys_sendmsg+0x1c/0x20)
[   11.978602] [<c069d870>] (sys_sendmsg) from [<c02011cc>] (__sys_trace_return+0x0/0x10)
[   11.986509] Exception stack(0xedc81fa8 to 0xedc81ff0)
[   11.991560] 1fa0:                   be8dcf80 b6f19ce8 0000000e be8dcf80 00000000 00000000
[   11.999732] 1fc0: be8dcf80 b6f19ce8 0186aea0 00000128 00000000 00000062 0186b6e8 00000000
[   12.007902] 1fe0: 00000128 be8dcf50 b6e003e3 b6e01346
[   12.012957] Code: e3130010 e1a0c000 1a000030 e35c0000 (e584900c) 
[   12.019056] Internal error: Oops: a06 [#4] SMP ARM
[   12.019171] ---[ end trace 1b60255ae59ac008 ]---


-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-08-16 20:35             ` Marc Haber
@ 2018-08-16 22:58               ` Russell King - ARM Linux
  -1 siblings, 0 replies; 50+ messages in thread
From: Russell King - ARM Linux @ 2018-08-16 22:58 UTC (permalink / raw)
  To: Marc Haber
  Cc: Peter Robinson, linux-arm-kernel, netdev, labbott, Eric Dumazet,
	Daniel Borkmann

On Thu, Aug 16, 2018 at 10:35:16PM +0200, Marc Haber wrote:
> On Mon, Jun 25, 2018 at 05:41:27PM +0100, Peter Robinson wrote:
> > So with that and the other fix there was no improvement, with those
> > and the BPF JIT disabled it works, I'm not sure if the two patches
> > have any effect with the JIT disabled though.
> 
> I can confirm the crash with the released 4.18.1 on Banana Pi, and I can
> also confirm that disabling BPF JIT makes the Banana Pi work again.,

Hi,

I'm afraid that the information in the crash dumps is insufficient
to be able to work very much out about these crashes.

We need a recipe (kernel configuration and what userspace is doing)
so that it's possible to recreate the crash, or we need responses
to requests for information - I requested the disassembly of
sk_filter_trim_cap and the BPF code dump via setting a sysctl back
in early July.  Without this, as I say, I don't see how this problem
can be progressed.

If the problem is at boot, one way to set the sysctl would be to
hack the kernel and explicitly initialise the sysctl to '2', or
boot with init=/bin/sh, then manually mount /proc, set the sysctl,
and then "exec /sbin/init" from that shell.  (Remember there's no
job control in that shell, so ^z, ^c, etc do not work.)

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 13.8Mbps down 630kbps up
According to speedtest.net: 13Mbps down 490kbps up

^ permalink raw reply	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-08-16 22:58               ` Russell King - ARM Linux
  0 siblings, 0 replies; 50+ messages in thread
From: Russell King - ARM Linux @ 2018-08-16 22:58 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Aug 16, 2018 at 10:35:16PM +0200, Marc Haber wrote:
> On Mon, Jun 25, 2018 at 05:41:27PM +0100, Peter Robinson wrote:
> > So with that and the other fix there was no improvement, with those
> > and the BPF JIT disabled it works, I'm not sure if the two patches
> > have any effect with the JIT disabled though.
> 
> I can confirm the crash with the released 4.18.1 on Banana Pi, and I can
> also confirm that disabling BPF JIT makes the Banana Pi work again.,

Hi,

I'm afraid that the information in the crash dumps is insufficient
to be able to work very much out about these crashes.

We need a recipe (kernel configuration and what userspace is doing)
so that it's possible to recreate the crash, or we need responses
to requests for information - I requested the disassembly of
sk_filter_trim_cap and the BPF code dump via setting a sysctl back
in early July.  Without this, as I say, I don't see how this problem
can be progressed.

If the problem is at boot, one way to set the sysctl would be to
hack the kernel and explicitly initialise the sysctl to '2', or
boot with init=/bin/sh, then manually mount /proc, set the sysctl,
and then "exec /sbin/init" from that shell.  (Remember there's no
job control in that shell, so ^z, ^c, etc do not work.)

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 13.8Mbps down 630kbps up
According to speedtest.net: 13Mbps down 490kbps up

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-08-16 22:58               ` Russell King - ARM Linux
@ 2018-08-17 12:25                 ` Peter Robinson
  -1 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-08-17 12:25 UTC (permalink / raw)
  To: Russell King - ARM Linux
  Cc: Marc Haber, linux-arm-kernel, netdev, labbott, Eric Dumazet,
	Daniel Borkmann

On Thu, Aug 16, 2018 at 11:58 PM, Russell King - ARM Linux
<linux@armlinux.org.uk> wrote:
> On Thu, Aug 16, 2018 at 10:35:16PM +0200, Marc Haber wrote:
>> On Mon, Jun 25, 2018 at 05:41:27PM +0100, Peter Robinson wrote:
>> > So with that and the other fix there was no improvement, with those
>> > and the BPF JIT disabled it works, I'm not sure if the two patches
>> > have any effect with the JIT disabled though.
>>
>> I can confirm the crash with the released 4.18.1 on Banana Pi, and I can
>> also confirm that disabling BPF JIT makes the Banana Pi work again.,
>
> Hi,
>
> I'm afraid that the information in the crash dumps is insufficient
> to be able to work very much out about these crashes.
>
> We need a recipe (kernel configuration and what userspace is doing)
> so that it's possible to recreate the crash, or we need responses
> to requests for information - I requested the disassembly of
> sk_filter_trim_cap and the BPF code dump via setting a sysctl back
> in early July.  Without this, as I say, I don't see how this problem
> can be progressed.

I can provide a kernel config [1] but I've not had enough time to sit
down and get the rest of the stuff and debug it due to a combination
of travel and other priorities.

> If the problem is at boot, one way to set the sysctl would be to
> hack the kernel and explicitly initialise the sysctl to '2', or
> boot with init=/bin/sh, then manually mount /proc, set the sysctl,
> and then "exec /sbin/init" from that shell.  (Remember there's no
> job control in that shell, so ^z, ^c, etc do not work.)

It starts to happen in the early kernel boot long before we get to any
userspace across a number of ARMv7 devices (RPi2/3, BeagleBone and
AllWinner H3 based devices at least).

[1] https://pbrobinson.fedorapeople.org/kernel-armv7hl.config

^ permalink raw reply	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-08-17 12:25                 ` Peter Robinson
  0 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-08-17 12:25 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Aug 16, 2018 at 11:58 PM, Russell King - ARM Linux
<linux@armlinux.org.uk> wrote:
> On Thu, Aug 16, 2018 at 10:35:16PM +0200, Marc Haber wrote:
>> On Mon, Jun 25, 2018 at 05:41:27PM +0100, Peter Robinson wrote:
>> > So with that and the other fix there was no improvement, with those
>> > and the BPF JIT disabled it works, I'm not sure if the two patches
>> > have any effect with the JIT disabled though.
>>
>> I can confirm the crash with the released 4.18.1 on Banana Pi, and I can
>> also confirm that disabling BPF JIT makes the Banana Pi work again.,
>
> Hi,
>
> I'm afraid that the information in the crash dumps is insufficient
> to be able to work very much out about these crashes.
>
> We need a recipe (kernel configuration and what userspace is doing)
> so that it's possible to recreate the crash, or we need responses
> to requests for information - I requested the disassembly of
> sk_filter_trim_cap and the BPF code dump via setting a sysctl back
> in early July.  Without this, as I say, I don't see how this problem
> can be progressed.

I can provide a kernel config [1] but I've not had enough time to sit
down and get the rest of the stuff and debug it due to a combination
of travel and other priorities.

> If the problem is at boot, one way to set the sysctl would be to
> hack the kernel and explicitly initialise the sysctl to '2', or
> boot with init=/bin/sh, then manually mount /proc, set the sysctl,
> and then "exec /sbin/init" from that shell.  (Remember there's no
> job control in that shell, so ^z, ^c, etc do not work.)

It starts to happen in the early kernel boot long before we get to any
userspace across a number of ARMv7 devices (RPi2/3, BeagleBone and
AllWinner H3 based devices at least).

[1] https://pbrobinson.fedorapeople.org/kernel-armv7hl.config

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-08-17 12:25                 ` Peter Robinson
@ 2018-08-17 12:40                   ` Daniel Borkmann
  -1 siblings, 0 replies; 50+ messages in thread
From: Daniel Borkmann @ 2018-08-17 12:40 UTC (permalink / raw)
  To: Peter Robinson, Russell King - ARM Linux
  Cc: Marc Haber, linux-arm-kernel, netdev, labbott, Eric Dumazet

On 08/17/2018 02:25 PM, Peter Robinson wrote:
> On Thu, Aug 16, 2018 at 11:58 PM, Russell King - ARM Linux
> <linux@armlinux.org.uk> wrote:
>> On Thu, Aug 16, 2018 at 10:35:16PM +0200, Marc Haber wrote:
>>> On Mon, Jun 25, 2018 at 05:41:27PM +0100, Peter Robinson wrote:
>>>> So with that and the other fix there was no improvement, with those
>>>> and the BPF JIT disabled it works, I'm not sure if the two patches
>>>> have any effect with the JIT disabled though.
>>>
>>> I can confirm the crash with the released 4.18.1 on Banana Pi, and I can
>>> also confirm that disabling BPF JIT makes the Banana Pi work again.,
>>
>> I'm afraid that the information in the crash dumps is insufficient
>> to be able to work very much out about these crashes.
>>
>> We need a recipe (kernel configuration and what userspace is doing)
>> so that it's possible to recreate the crash, or we need responses
>> to requests for information - I requested the disassembly of
>> sk_filter_trim_cap and the BPF code dump via setting a sysctl back
>> in early July.  Without this, as I say, I don't see how this problem
>> can be progressed.
> 
> I can provide a kernel config [1] but I've not had enough time to sit
> down and get the rest of the stuff and debug it due to a combination
> of travel and other priorities.

Did you get a chance to try latest kernel from Linus' tree [1] from last
few days to see whether the issue is still persistent? There have been
a number of improvements, bit strange why e.g. Russell didn't run into
it while others have, hmm. Perhaps due to EABI vs non EABI.

[1] git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

>> If the problem is at boot, one way to set the sysctl would be to
>> hack the kernel and explicitly initialise the sysctl to '2', or
>> boot with init=/bin/sh, then manually mount /proc, set the sysctl,
>> and then "exec /sbin/init" from that shell.  (Remember there's no
>> job control in that shell, so ^z, ^c, etc do not work.)
> 
> It starts to happen in the early kernel boot long before we get to any
> userspace across a number of ARMv7 devices (RPi2/3, BeagleBone and
> AllWinner H3 based devices at least).
> 
> [1] https://pbrobinson.fedorapeople.org/kernel-armv7hl.config

I'd have one potential bug suspicion, for the 4.18 one you were trying,
could you run with the below patch to see whether it would help?

Thanks,
Daniel

diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index f6a62ae..c864f6b 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -238,7 +238,7 @@ static void jit_fill_hole(void *area, unsigned int size)
 #define STACK_SIZE	ALIGN(_STACK_SIZE, STACK_ALIGNMENT)

 /* Get the offset of eBPF REGISTERs stored on scratch space. */
-#define STACK_VAR(off) (STACK_SIZE - off)
+#define STACK_VAR(off) (STACK_SIZE - off - 4)

 #if __LINUX_ARM_ARCH__ < 7

^ permalink raw reply related	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-08-17 12:40                   ` Daniel Borkmann
  0 siblings, 0 replies; 50+ messages in thread
From: Daniel Borkmann @ 2018-08-17 12:40 UTC (permalink / raw)
  To: linux-arm-kernel

On 08/17/2018 02:25 PM, Peter Robinson wrote:
> On Thu, Aug 16, 2018 at 11:58 PM, Russell King - ARM Linux
> <linux@armlinux.org.uk> wrote:
>> On Thu, Aug 16, 2018 at 10:35:16PM +0200, Marc Haber wrote:
>>> On Mon, Jun 25, 2018 at 05:41:27PM +0100, Peter Robinson wrote:
>>>> So with that and the other fix there was no improvement, with those
>>>> and the BPF JIT disabled it works, I'm not sure if the two patches
>>>> have any effect with the JIT disabled though.
>>>
>>> I can confirm the crash with the released 4.18.1 on Banana Pi, and I can
>>> also confirm that disabling BPF JIT makes the Banana Pi work again.,
>>
>> I'm afraid that the information in the crash dumps is insufficient
>> to be able to work very much out about these crashes.
>>
>> We need a recipe (kernel configuration and what userspace is doing)
>> so that it's possible to recreate the crash, or we need responses
>> to requests for information - I requested the disassembly of
>> sk_filter_trim_cap and the BPF code dump via setting a sysctl back
>> in early July.  Without this, as I say, I don't see how this problem
>> can be progressed.
> 
> I can provide a kernel config [1] but I've not had enough time to sit
> down and get the rest of the stuff and debug it due to a combination
> of travel and other priorities.

Did you get a chance to try latest kernel from Linus' tree [1] from last
few days to see whether the issue is still persistent? There have been
a number of improvements, bit strange why e.g. Russell didn't run into
it while others have, hmm. Perhaps due to EABI vs non EABI.

[1] git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

>> If the problem is at boot, one way to set the sysctl would be to
>> hack the kernel and explicitly initialise the sysctl to '2', or
>> boot with init=/bin/sh, then manually mount /proc, set the sysctl,
>> and then "exec /sbin/init" from that shell.  (Remember there's no
>> job control in that shell, so ^z, ^c, etc do not work.)
> 
> It starts to happen in the early kernel boot long before we get to any
> userspace across a number of ARMv7 devices (RPi2/3, BeagleBone and
> AllWinner H3 based devices at least).
> 
> [1] https://pbrobinson.fedorapeople.org/kernel-armv7hl.config

I'd have one potential bug suspicion, for the 4.18 one you were trying,
could you run with the below patch to see whether it would help?

Thanks,
Daniel

diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index f6a62ae..c864f6b 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -238,7 +238,7 @@ static void jit_fill_hole(void *area, unsigned int size)
 #define STACK_SIZE	ALIGN(_STACK_SIZE, STACK_ALIGNMENT)

 /* Get the offset of eBPF REGISTERs stored on scratch space. */
-#define STACK_VAR(off) (STACK_SIZE - off)
+#define STACK_VAR(off) (STACK_SIZE - off - 4)

 #if __LINUX_ARM_ARCH__ < 7

^ permalink raw reply related	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-08-17 12:40                   ` Daniel Borkmann
@ 2018-08-17 14:32                     ` Peter Robinson
  -1 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-08-17 14:32 UTC (permalink / raw)
  To: Daniel Borkmann
  Cc: Russell King - ARM Linux, Marc Haber, linux-arm-kernel, netdev,
	labbott, Eric Dumazet

On Fri, Aug 17, 2018 at 1:40 PM, Daniel Borkmann <daniel@iogearbox.net> wrote:
> On 08/17/2018 02:25 PM, Peter Robinson wrote:
>> On Thu, Aug 16, 2018 at 11:58 PM, Russell King - ARM Linux
>> <linux@armlinux.org.uk> wrote:
>>> On Thu, Aug 16, 2018 at 10:35:16PM +0200, Marc Haber wrote:
>>>> On Mon, Jun 25, 2018 at 05:41:27PM +0100, Peter Robinson wrote:
>>>>> So with that and the other fix there was no improvement, with those
>>>>> and the BPF JIT disabled it works, I'm not sure if the two patches
>>>>> have any effect with the JIT disabled though.
>>>>
>>>> I can confirm the crash with the released 4.18.1 on Banana Pi, and I can
>>>> also confirm that disabling BPF JIT makes the Banana Pi work again.,
>>>
>>> I'm afraid that the information in the crash dumps is insufficient
>>> to be able to work very much out about these crashes.
>>>
>>> We need a recipe (kernel configuration and what userspace is doing)
>>> so that it's possible to recreate the crash, or we need responses
>>> to requests for information - I requested the disassembly of
>>> sk_filter_trim_cap and the BPF code dump via setting a sysctl back
>>> in early July.  Without this, as I say, I don't see how this problem
>>> can be progressed.
>>
>> I can provide a kernel config [1] but I've not had enough time to sit
>> down and get the rest of the stuff and debug it due to a combination
>> of travel and other priorities.
>
> Did you get a chance to try latest kernel from Linus' tree [1] from last
> few days to see whether the issue is still persistent? There have been
> a number of improvements, bit strange why e.g. Russell didn't run into
> it while others have, hmm. Perhaps due to EABI vs non EABI.

I haven't had a chance to try anything from the 4.19 merge window as
yet, I'm traveling this week so it was on the list for next week to
try.

> [1] git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
>
>>> If the problem is at boot, one way to set the sysctl would be to
>>> hack the kernel and explicitly initialise the sysctl to '2', or
>>> boot with init=/bin/sh, then manually mount /proc, set the sysctl,
>>> and then "exec /sbin/init" from that shell.  (Remember there's no
>>> job control in that shell, so ^z, ^c, etc do not work.)
>>
>> It starts to happen in the early kernel boot long before we get to any
>> userspace across a number of ARMv7 devices (RPi2/3, BeagleBone and
>> AllWinner H3 based devices at least).
>>
>> [1] https://pbrobinson.fedorapeople.org/kernel-armv7hl.config
>
> I'd have one potential bug suspicion, for the 4.18 one you were trying,
> could you run with the below patch to see whether it would help?

I will try and get someone to test that today, thanks

> Thanks,
> Daniel
>
> diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
> index f6a62ae..c864f6b 100644
> --- a/arch/arm/net/bpf_jit_32.c
> +++ b/arch/arm/net/bpf_jit_32.c
> @@ -238,7 +238,7 @@ static void jit_fill_hole(void *area, unsigned int size)
>  #define STACK_SIZE     ALIGN(_STACK_SIZE, STACK_ALIGNMENT)
>
>  /* Get the offset of eBPF REGISTERs stored on scratch space. */
> -#define STACK_VAR(off) (STACK_SIZE - off)
> +#define STACK_VAR(off) (STACK_SIZE - off - 4)
>
>  #if __LINUX_ARM_ARCH__ < 7
>

^ permalink raw reply	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-08-17 14:32                     ` Peter Robinson
  0 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-08-17 14:32 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Aug 17, 2018 at 1:40 PM, Daniel Borkmann <daniel@iogearbox.net> wrote:
> On 08/17/2018 02:25 PM, Peter Robinson wrote:
>> On Thu, Aug 16, 2018 at 11:58 PM, Russell King - ARM Linux
>> <linux@armlinux.org.uk> wrote:
>>> On Thu, Aug 16, 2018 at 10:35:16PM +0200, Marc Haber wrote:
>>>> On Mon, Jun 25, 2018 at 05:41:27PM +0100, Peter Robinson wrote:
>>>>> So with that and the other fix there was no improvement, with those
>>>>> and the BPF JIT disabled it works, I'm not sure if the two patches
>>>>> have any effect with the JIT disabled though.
>>>>
>>>> I can confirm the crash with the released 4.18.1 on Banana Pi, and I can
>>>> also confirm that disabling BPF JIT makes the Banana Pi work again.,
>>>
>>> I'm afraid that the information in the crash dumps is insufficient
>>> to be able to work very much out about these crashes.
>>>
>>> We need a recipe (kernel configuration and what userspace is doing)
>>> so that it's possible to recreate the crash, or we need responses
>>> to requests for information - I requested the disassembly of
>>> sk_filter_trim_cap and the BPF code dump via setting a sysctl back
>>> in early July.  Without this, as I say, I don't see how this problem
>>> can be progressed.
>>
>> I can provide a kernel config [1] but I've not had enough time to sit
>> down and get the rest of the stuff and debug it due to a combination
>> of travel and other priorities.
>
> Did you get a chance to try latest kernel from Linus' tree [1] from last
> few days to see whether the issue is still persistent? There have been
> a number of improvements, bit strange why e.g. Russell didn't run into
> it while others have, hmm. Perhaps due to EABI vs non EABI.

I haven't had a chance to try anything from the 4.19 merge window as
yet, I'm traveling this week so it was on the list for next week to
try.

> [1] git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
>
>>> If the problem is at boot, one way to set the sysctl would be to
>>> hack the kernel and explicitly initialise the sysctl to '2', or
>>> boot with init=/bin/sh, then manually mount /proc, set the sysctl,
>>> and then "exec /sbin/init" from that shell.  (Remember there's no
>>> job control in that shell, so ^z, ^c, etc do not work.)
>>
>> It starts to happen in the early kernel boot long before we get to any
>> userspace across a number of ARMv7 devices (RPi2/3, BeagleBone and
>> AllWinner H3 based devices at least).
>>
>> [1] https://pbrobinson.fedorapeople.org/kernel-armv7hl.config
>
> I'd have one potential bug suspicion, for the 4.18 one you were trying,
> could you run with the below patch to see whether it would help?

I will try and get someone to test that today, thanks

> Thanks,
> Daniel
>
> diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
> index f6a62ae..c864f6b 100644
> --- a/arch/arm/net/bpf_jit_32.c
> +++ b/arch/arm/net/bpf_jit_32.c
> @@ -238,7 +238,7 @@ static void jit_fill_hole(void *area, unsigned int size)
>  #define STACK_SIZE     ALIGN(_STACK_SIZE, STACK_ALIGNMENT)
>
>  /* Get the offset of eBPF REGISTERs stored on scratch space. */
> -#define STACK_VAR(off) (STACK_SIZE - off)
> +#define STACK_VAR(off) (STACK_SIZE - off - 4)
>
>  #if __LINUX_ARM_ARCH__ < 7
>

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-08-17 12:40                   ` Daniel Borkmann
@ 2018-08-17 16:17                     ` Russell King - ARM Linux
  -1 siblings, 0 replies; 50+ messages in thread
From: Russell King - ARM Linux @ 2018-08-17 16:17 UTC (permalink / raw)
  To: Daniel Borkmann
  Cc: Peter Robinson, Marc Haber, linux-arm-kernel, netdev, labbott,
	Eric Dumazet

On Fri, Aug 17, 2018 at 02:40:19PM +0200, Daniel Borkmann wrote:
> I'd have one potential bug suspicion, for the 4.18 one you were trying,
> could you run with the below patch to see whether it would help?

I think this is almost certainly the problem - looking at the history,
it seems that the "-4" was assumed to be part of the scratch stuff in
commit 38ca93060163 ("bpf, arm32: save 4 bytes of unneeded stack space")
but it isn't - it's because "off" of zero refers to the top word in the
stack (iow at STACK_SIZE-4).

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 13.8Mbps down 630kbps up
According to speedtest.net: 13Mbps down 490kbps up

^ permalink raw reply	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-08-17 16:17                     ` Russell King - ARM Linux
  0 siblings, 0 replies; 50+ messages in thread
From: Russell King - ARM Linux @ 2018-08-17 16:17 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Aug 17, 2018 at 02:40:19PM +0200, Daniel Borkmann wrote:
> I'd have one potential bug suspicion, for the 4.18 one you were trying,
> could you run with the below patch to see whether it would help?

I think this is almost certainly the problem - looking at the history,
it seems that the "-4" was assumed to be part of the scratch stuff in
commit 38ca93060163 ("bpf, arm32: save 4 bytes of unneeded stack space")
but it isn't - it's because "off" of zero refers to the top word in the
stack (iow at STACK_SIZE-4).

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 13.8Mbps down 630kbps up
According to speedtest.net: 13Mbps down 490kbps up

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-08-17 16:17                     ` Russell King - ARM Linux
@ 2018-08-17 18:30                       ` Daniel Borkmann
  -1 siblings, 0 replies; 50+ messages in thread
From: Daniel Borkmann @ 2018-08-17 18:30 UTC (permalink / raw)
  To: Russell King - ARM Linux
  Cc: Peter Robinson, Marc Haber, linux-arm-kernel, netdev, labbott,
	Eric Dumazet

On 08/17/2018 06:17 PM, Russell King - ARM Linux wrote:
> On Fri, Aug 17, 2018 at 02:40:19PM +0200, Daniel Borkmann wrote:
>> I'd have one potential bug suspicion, for the 4.18 one you were trying,
>> could you run with the below patch to see whether it would help?
> 
> I think this is almost certainly the problem - looking at the history,
> it seems that the "-4" was assumed to be part of the scratch stuff in
> commit 38ca93060163 ("bpf, arm32: save 4 bytes of unneeded stack space")
> but it isn't - it's because "off" of zero refers to the top word in the
> stack (iow at STACK_SIZE-4).

Yeah agree, my thinking as well (albeit bit late, sigh, sorry about that).
Waiting for Peter to get back with results for definite confirmation. Your
rework in 1c35ba122d4a ("ARM: net: bpf: use negative numbers for stacked
registers") and 96cced4e774a ("ARM: net: bpf: access eBPF scratch space using
ARM FP register") fixes this in mainline, so unless I'm missing something this
would only need a stand-alone fix for 4.18/stable which I can cook up and
submit then.

Thanks,
Daniel

^ permalink raw reply	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-08-17 18:30                       ` Daniel Borkmann
  0 siblings, 0 replies; 50+ messages in thread
From: Daniel Borkmann @ 2018-08-17 18:30 UTC (permalink / raw)
  To: linux-arm-kernel

On 08/17/2018 06:17 PM, Russell King - ARM Linux wrote:
> On Fri, Aug 17, 2018 at 02:40:19PM +0200, Daniel Borkmann wrote:
>> I'd have one potential bug suspicion, for the 4.18 one you were trying,
>> could you run with the below patch to see whether it would help?
> 
> I think this is almost certainly the problem - looking at the history,
> it seems that the "-4" was assumed to be part of the scratch stuff in
> commit 38ca93060163 ("bpf, arm32: save 4 bytes of unneeded stack space")
> but it isn't - it's because "off" of zero refers to the top word in the
> stack (iow at STACK_SIZE-4).

Yeah agree, my thinking as well (albeit bit late, sigh, sorry about that).
Waiting for Peter to get back with results for definite confirmation. Your
rework in 1c35ba122d4a ("ARM: net: bpf: use negative numbers for stacked
registers") and 96cced4e774a ("ARM: net: bpf: access eBPF scratch space using
ARM FP register") fixes this in mainline, so unless I'm missing something this
would only need a stand-alone fix for 4.18/stable which I can cook up and
submit then.

Thanks,
Daniel

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-08-17 18:30                       ` Daniel Borkmann
@ 2018-08-17 18:51                         ` Stefan Wahren
  -1 siblings, 0 replies; 50+ messages in thread
From: Stefan Wahren @ 2018-08-17 18:51 UTC (permalink / raw)
  To: Daniel Borkmann
  Cc: Eric Dumazet, netdev, Marc Haber, Russell King - ARM Linux,
	Peter Robinson, labbott, linux-arm-kernel

Hi Daniel,

> Daniel Borkmann <daniel@iogearbox.net> hat am 17. August 2018 um 20:30 geschrieben:
> 
> 
> On 08/17/2018 06:17 PM, Russell King - ARM Linux wrote:
> > On Fri, Aug 17, 2018 at 02:40:19PM +0200, Daniel Borkmann wrote:
> >> I'd have one potential bug suspicion, for the 4.18 one you were trying,
> >> could you run with the below patch to see whether it would help?
> > 
> > I think this is almost certainly the problem - looking at the history,
> > it seems that the "-4" was assumed to be part of the scratch stuff in
> > commit 38ca93060163 ("bpf, arm32: save 4 bytes of unneeded stack space")
> > but it isn't - it's because "off" of zero refers to the top word in the
> > stack (iow at STACK_SIZE-4).
> 
> Yeah agree, my thinking as well (albeit bit late, sigh, sorry about that).
> Waiting for Peter to get back with results for definite confirmation. Your
> rework in 1c35ba122d4a ("ARM: net: bpf: use negative numbers for stacked
> registers") and 96cced4e774a ("ARM: net: bpf: access eBPF scratch space using
> ARM FP register") fixes this in mainline, so unless I'm missing something this
> would only need a stand-alone fix for 4.18/stable which I can cook up and
> submit then.

i was able to reproduce this issue on RPi 3 with Linux 4.18.1 + multi_v7_defconfig and the following  config changes:

 --- a/arch/arm/configs/multi_v7_defconfig
+++ b/arch/arm/configs/multi_v7_defconfig
@@ -2,7 +2,10 @@ CONFIG_SYSVIPC=y
 CONFIG_NO_HZ=y
 CONFIG_HIGH_RES_TIMERS=y
 CONFIG_CGROUPS=y
+CONFIG_CGROUP_BPF=y
 CONFIG_BLK_DEV_INITRD=y
+CONFIG_BPF_SYSCALL=y
+CONFIG_BPF_JIT_ALWAYS_ON=y
 CONFIG_EMBEDDED=y
 CONFIG_PERF_EVENTS=y
 CONFIG_MODULES=y
@@ -153,6 +156,8 @@ CONFIG_IPV6_MIP6=m
 CONFIG_IPV6_TUNNEL=m
 CONFIG_IPV6_MULTIPLE_TABLES=y
 CONFIG_NET_DSA=m
+CONFIG_BPF_JIT=y
+CONFIG_BPF_STREAM_PARSER=y
 CONFIG_CAN=y
 CONFIG_CAN_AT91=m
 CONFIG_CAN_FLEXCAN=m

After applying the "-4" patch the oopses doesn't appear during boot anymore.

Stefan

> 
> Thanks,
> Daniel
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-08-17 18:51                         ` Stefan Wahren
  0 siblings, 0 replies; 50+ messages in thread
From: Stefan Wahren @ 2018-08-17 18:51 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Daniel,

> Daniel Borkmann <daniel@iogearbox.net> hat am 17. August 2018 um 20:30 geschrieben:
> 
> 
> On 08/17/2018 06:17 PM, Russell King - ARM Linux wrote:
> > On Fri, Aug 17, 2018 at 02:40:19PM +0200, Daniel Borkmann wrote:
> >> I'd have one potential bug suspicion, for the 4.18 one you were trying,
> >> could you run with the below patch to see whether it would help?
> > 
> > I think this is almost certainly the problem - looking at the history,
> > it seems that the "-4" was assumed to be part of the scratch stuff in
> > commit 38ca93060163 ("bpf, arm32: save 4 bytes of unneeded stack space")
> > but it isn't - it's because "off" of zero refers to the top word in the
> > stack (iow at STACK_SIZE-4).
> 
> Yeah agree, my thinking as well (albeit bit late, sigh, sorry about that).
> Waiting for Peter to get back with results for definite confirmation. Your
> rework in 1c35ba122d4a ("ARM: net: bpf: use negative numbers for stacked
> registers") and 96cced4e774a ("ARM: net: bpf: access eBPF scratch space using
> ARM FP register") fixes this in mainline, so unless I'm missing something this
> would only need a stand-alone fix for 4.18/stable which I can cook up and
> submit then.

i was able to reproduce this issue on RPi 3 with Linux 4.18.1 + multi_v7_defconfig and the following  config changes:

 --- a/arch/arm/configs/multi_v7_defconfig
+++ b/arch/arm/configs/multi_v7_defconfig
@@ -2,7 +2,10 @@ CONFIG_SYSVIPC=y
 CONFIG_NO_HZ=y
 CONFIG_HIGH_RES_TIMERS=y
 CONFIG_CGROUPS=y
+CONFIG_CGROUP_BPF=y
 CONFIG_BLK_DEV_INITRD=y
+CONFIG_BPF_SYSCALL=y
+CONFIG_BPF_JIT_ALWAYS_ON=y
 CONFIG_EMBEDDED=y
 CONFIG_PERF_EVENTS=y
 CONFIG_MODULES=y
@@ -153,6 +156,8 @@ CONFIG_IPV6_MIP6=m
 CONFIG_IPV6_TUNNEL=m
 CONFIG_IPV6_MULTIPLE_TABLES=y
 CONFIG_NET_DSA=m
+CONFIG_BPF_JIT=y
+CONFIG_BPF_STREAM_PARSER=y
 CONFIG_CAN=y
 CONFIG_CAN_AT91=m
 CONFIG_CAN_FLEXCAN=m

After applying the "-4" patch the oopses doesn't appear during boot anymore.

Stefan

> 
> Thanks,
> Daniel
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-08-17 16:17                     ` Russell King - ARM Linux
@ 2018-08-17 21:12                       ` Peter Robinson
  -1 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-08-17 21:12 UTC (permalink / raw)
  To: Russell King - ARM Linux
  Cc: Daniel Borkmann, Marc Haber, linux-arm-kernel, netdev, labbott,
	Eric Dumazet

On Fri, Aug 17, 2018 at 5:17 PM, Russell King - ARM Linux
<linux@armlinux.org.uk> wrote:
> On Fri, Aug 17, 2018 at 02:40:19PM +0200, Daniel Borkmann wrote:
>> I'd have one potential bug suspicion, for the 4.18 one you were trying,
>> could you run with the below patch to see whether it would help?
>
> I think this is almost certainly the problem - looking at the history,
> it seems that the "-4" was assumed to be part of the scratch stuff in
> commit 38ca93060163 ("bpf, arm32: save 4 bytes of unneeded stack space")
> but it isn't - it's because "off" of zero refers to the top word in the
> stack (iow at STACK_SIZE-4).

I can confirm that patch fixes the problem I was seeing.

Peter

^ permalink raw reply	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-08-17 21:12                       ` Peter Robinson
  0 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-08-17 21:12 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Aug 17, 2018 at 5:17 PM, Russell King - ARM Linux
<linux@armlinux.org.uk> wrote:
> On Fri, Aug 17, 2018 at 02:40:19PM +0200, Daniel Borkmann wrote:
>> I'd have one potential bug suspicion, for the 4.18 one you were trying,
>> could you run with the below patch to see whether it would help?
>
> I think this is almost certainly the problem - looking at the history,
> it seems that the "-4" was assumed to be part of the scratch stuff in
> commit 38ca93060163 ("bpf, arm32: save 4 bytes of unneeded stack space")
> but it isn't - it's because "off" of zero refers to the top word in the
> stack (iow at STACK_SIZE-4).

I can confirm that patch fixes the problem I was seeing.

Peter

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-08-17 18:30                       ` Daniel Borkmann
@ 2018-08-17 21:13                         ` Peter Robinson
  -1 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-08-17 21:13 UTC (permalink / raw)
  To: Daniel Borkmann
  Cc: Russell King - ARM Linux, Marc Haber, linux-arm-kernel, netdev,
	labbott, Eric Dumazet

On Fri, Aug 17, 2018 at 7:30 PM, Daniel Borkmann <daniel@iogearbox.net> wrote:
> On 08/17/2018 06:17 PM, Russell King - ARM Linux wrote:
>> On Fri, Aug 17, 2018 at 02:40:19PM +0200, Daniel Borkmann wrote:
>>> I'd have one potential bug suspicion, for the 4.18 one you were trying,
>>> could you run with the below patch to see whether it would help?
>>
>> I think this is almost certainly the problem - looking at the history,
>> it seems that the "-4" was assumed to be part of the scratch stuff in
>> commit 38ca93060163 ("bpf, arm32: save 4 bytes of unneeded stack space")
>> but it isn't - it's because "off" of zero refers to the top word in the
>> stack (iow at STACK_SIZE-4).
>
> Yeah agree, my thinking as well (albeit bit late, sigh, sorry about that).
> Waiting for Peter to get back with results for definite confirmation. Your
> rework in 1c35ba122d4a ("ARM: net: bpf: use negative numbers for stacked
> registers") and 96cced4e774a ("ARM: net: bpf: access eBPF scratch space using
> ARM FP register") fixes this in mainline, so unless I'm missing something this
> would only need a stand-alone fix for 4.18/stable which I can cook up and
> submit then.

I can confirm that fixes the problems I was seeing on Fedora 29.

Feel free to add a tested by from me:

Tested-by: Peter Robinson <pbrobinson@gmail.com>

^ permalink raw reply	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-08-17 21:13                         ` Peter Robinson
  0 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-08-17 21:13 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Aug 17, 2018 at 7:30 PM, Daniel Borkmann <daniel@iogearbox.net> wrote:
> On 08/17/2018 06:17 PM, Russell King - ARM Linux wrote:
>> On Fri, Aug 17, 2018 at 02:40:19PM +0200, Daniel Borkmann wrote:
>>> I'd have one potential bug suspicion, for the 4.18 one you were trying,
>>> could you run with the below patch to see whether it would help?
>>
>> I think this is almost certainly the problem - looking at the history,
>> it seems that the "-4" was assumed to be part of the scratch stuff in
>> commit 38ca93060163 ("bpf, arm32: save 4 bytes of unneeded stack space")
>> but it isn't - it's because "off" of zero refers to the top word in the
>> stack (iow at STACK_SIZE-4).
>
> Yeah agree, my thinking as well (albeit bit late, sigh, sorry about that).
> Waiting for Peter to get back with results for definite confirmation. Your
> rework in 1c35ba122d4a ("ARM: net: bpf: use negative numbers for stacked
> registers") and 96cced4e774a ("ARM: net: bpf: access eBPF scratch space using
> ARM FP register") fixes this in mainline, so unless I'm missing something this
> would only need a stand-alone fix for 4.18/stable which I can cook up and
> submit then.

I can confirm that fixes the problems I was seeing on Fedora 29.

Feel free to add a tested by from me:

Tested-by: Peter Robinson <pbrobinson@gmail.com>

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-08-17 18:51                         ` Stefan Wahren
@ 2018-08-17 21:15                           ` Peter Robinson
  -1 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-08-17 21:15 UTC (permalink / raw)
  To: Stefan Wahren
  Cc: Daniel Borkmann, Marc Haber, labbott, Eric Dumazet, netdev,
	Russell King - ARM Linux, linux-arm-kernel

Hi Stefan,

>> On 08/17/2018 06:17 PM, Russell King - ARM Linux wrote:
>> > On Fri, Aug 17, 2018 at 02:40:19PM +0200, Daniel Borkmann wrote:
>> >> I'd have one potential bug suspicion, for the 4.18 one you were trying,
>> >> could you run with the below patch to see whether it would help?
>> >
>> > I think this is almost certainly the problem - looking at the history,
>> > it seems that the "-4" was assumed to be part of the scratch stuff in
>> > commit 38ca93060163 ("bpf, arm32: save 4 bytes of unneeded stack space")
>> > but it isn't - it's because "off" of zero refers to the top word in the
>> > stack (iow at STACK_SIZE-4).
>>
>> Yeah agree, my thinking as well (albeit bit late, sigh, sorry about that).
>> Waiting for Peter to get back with results for definite confirmation. Your
>> rework in 1c35ba122d4a ("ARM: net: bpf: use negative numbers for stacked
>> registers") and 96cced4e774a ("ARM: net: bpf: access eBPF scratch space using
>> ARM FP register") fixes this in mainline, so unless I'm missing something this
>> would only need a stand-alone fix for 4.18/stable which I can cook up and
>> submit then.
>
> i was able to reproduce this issue on RPi 3 with Linux 4.18.1 + multi_v7_defconfig and the following  config changes:
>
>  --- a/arch/arm/configs/multi_v7_defconfig
> +++ b/arch/arm/configs/multi_v7_defconfig
> @@ -2,7 +2,10 @@ CONFIG_SYSVIPC=y
>  CONFIG_NO_HZ=y
>  CONFIG_HIGH_RES_TIMERS=y
>  CONFIG_CGROUPS=y
> +CONFIG_CGROUP_BPF=y
>  CONFIG_BLK_DEV_INITRD=y
> +CONFIG_BPF_SYSCALL=y
> +CONFIG_BPF_JIT_ALWAYS_ON=y
>  CONFIG_EMBEDDED=y
>  CONFIG_PERF_EVENTS=y
>  CONFIG_MODULES=y
> @@ -153,6 +156,8 @@ CONFIG_IPV6_MIP6=m
>  CONFIG_IPV6_TUNNEL=m
>  CONFIG_IPV6_MULTIPLE_TABLES=y
>  CONFIG_NET_DSA=m
> +CONFIG_BPF_JIT=y
> +CONFIG_BPF_STREAM_PARSER=y
>  CONFIG_CAN=y
>  CONFIG_CAN_AT91=m
>  CONFIG_CAN_FLEXCAN=m
>
> After applying the "-4" patch the oopses doesn't appear during boot anymore.

Would be fab to get that into the kernel so this is widely tested
moving forward.

Peter

^ permalink raw reply	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-08-17 21:15                           ` Peter Robinson
  0 siblings, 0 replies; 50+ messages in thread
From: Peter Robinson @ 2018-08-17 21:15 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Stefan,

>> On 08/17/2018 06:17 PM, Russell King - ARM Linux wrote:
>> > On Fri, Aug 17, 2018 at 02:40:19PM +0200, Daniel Borkmann wrote:
>> >> I'd have one potential bug suspicion, for the 4.18 one you were trying,
>> >> could you run with the below patch to see whether it would help?
>> >
>> > I think this is almost certainly the problem - looking at the history,
>> > it seems that the "-4" was assumed to be part of the scratch stuff in
>> > commit 38ca93060163 ("bpf, arm32: save 4 bytes of unneeded stack space")
>> > but it isn't - it's because "off" of zero refers to the top word in the
>> > stack (iow at STACK_SIZE-4).
>>
>> Yeah agree, my thinking as well (albeit bit late, sigh, sorry about that).
>> Waiting for Peter to get back with results for definite confirmation. Your
>> rework in 1c35ba122d4a ("ARM: net: bpf: use negative numbers for stacked
>> registers") and 96cced4e774a ("ARM: net: bpf: access eBPF scratch space using
>> ARM FP register") fixes this in mainline, so unless I'm missing something this
>> would only need a stand-alone fix for 4.18/stable which I can cook up and
>> submit then.
>
> i was able to reproduce this issue on RPi 3 with Linux 4.18.1 + multi_v7_defconfig and the following  config changes:
>
>  --- a/arch/arm/configs/multi_v7_defconfig
> +++ b/arch/arm/configs/multi_v7_defconfig
> @@ -2,7 +2,10 @@ CONFIG_SYSVIPC=y
>  CONFIG_NO_HZ=y
>  CONFIG_HIGH_RES_TIMERS=y
>  CONFIG_CGROUPS=y
> +CONFIG_CGROUP_BPF=y
>  CONFIG_BLK_DEV_INITRD=y
> +CONFIG_BPF_SYSCALL=y
> +CONFIG_BPF_JIT_ALWAYS_ON=y
>  CONFIG_EMBEDDED=y
>  CONFIG_PERF_EVENTS=y
>  CONFIG_MODULES=y
> @@ -153,6 +156,8 @@ CONFIG_IPV6_MIP6=m
>  CONFIG_IPV6_TUNNEL=m
>  CONFIG_IPV6_MULTIPLE_TABLES=y
>  CONFIG_NET_DSA=m
> +CONFIG_BPF_JIT=y
> +CONFIG_BPF_STREAM_PARSER=y
>  CONFIG_CAN=y
>  CONFIG_CAN_AT91=m
>  CONFIG_CAN_FLEXCAN=m
>
> After applying the "-4" patch the oopses doesn't appear during boot anymore.

Would be fab to get that into the kernel so this is widely tested
moving forward.

Peter

^ permalink raw reply	[flat|nested] 50+ messages in thread
* Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
  2018-08-17 21:13                         ` Peter Robinson
@ 2018-08-17 22:06                           ` Daniel Borkmann
  -1 siblings, 0 replies; 50+ messages in thread
From: Daniel Borkmann @ 2018-08-17 22:06 UTC (permalink / raw)
  To: Peter Robinson
  Cc: Russell King - ARM Linux, Marc Haber, linux-arm-kernel, netdev,
	labbott, Eric Dumazet

On 08/17/2018 11:13 PM, Peter Robinson wrote:
> On Fri, Aug 17, 2018 at 7:30 PM, Daniel Borkmann <daniel@iogearbox.net> wrote:
>> On 08/17/2018 06:17 PM, Russell King - ARM Linux wrote:
>>> On Fri, Aug 17, 2018 at 02:40:19PM +0200, Daniel Borkmann wrote:
>>>> I'd have one potential bug suspicion, for the 4.18 one you were trying,
>>>> could you run with the below patch to see whether it would help?
>>>
>>> I think this is almost certainly the problem - looking at the history,
>>> it seems that the "-4" was assumed to be part of the scratch stuff in
>>> commit 38ca93060163 ("bpf, arm32: save 4 bytes of unneeded stack space")
>>> but it isn't - it's because "off" of zero refers to the top word in the
>>> stack (iow at STACK_SIZE-4).
>>
>> Yeah agree, my thinking as well (albeit bit late, sigh, sorry about that).
>> Waiting for Peter to get back with results for definite confirmation. Your
>> rework in 1c35ba122d4a ("ARM: net: bpf: use negative numbers for stacked
>> registers") and 96cced4e774a ("ARM: net: bpf: access eBPF scratch space using
>> ARM FP register") fixes this in mainline, so unless I'm missing something this
>> would only need a stand-alone fix for 4.18/stable which I can cook up and
>> submit then.
> 
> I can confirm that fixes the problems I was seeing on Fedora 29.
> 
> Feel free to add a tested by from me:
> 
> Tested-by: Peter Robinson <pbrobinson@gmail.com>

Great, thanks everyone! Will get it out asap.

^ permalink raw reply	[flat|nested] 50+ messages in thread
* [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
@ 2018-08-17 22:06                           ` Daniel Borkmann
  0 siblings, 0 replies; 50+ messages in thread
From: Daniel Borkmann @ 2018-08-17 22:06 UTC (permalink / raw)
  To: linux-arm-kernel

On 08/17/2018 11:13 PM, Peter Robinson wrote:
> On Fri, Aug 17, 2018 at 7:30 PM, Daniel Borkmann <daniel@iogearbox.net> wrote:
>> On 08/17/2018 06:17 PM, Russell King - ARM Linux wrote:
>>> On Fri, Aug 17, 2018 at 02:40:19PM +0200, Daniel Borkmann wrote:
>>>> I'd have one potential bug suspicion, for the 4.18 one you were trying,
>>>> could you run with the below patch to see whether it would help?
>>>
>>> I think this is almost certainly the problem - looking at the history,
>>> it seems that the "-4" was assumed to be part of the scratch stuff in
>>> commit 38ca93060163 ("bpf, arm32: save 4 bytes of unneeded stack space")
>>> but it isn't - it's because "off" of zero refers to the top word in the
>>> stack (iow at STACK_SIZE-4).
>>
>> Yeah agree, my thinking as well (albeit bit late, sigh, sorry about that).
>> Waiting for Peter to get back with results for definite confirmation. Your
>> rework in 1c35ba122d4a ("ARM: net: bpf: use negative numbers for stacked
>> registers") and 96cced4e774a ("ARM: net: bpf: access eBPF scratch space using
>> ARM FP register") fixes this in mainline, so unless I'm missing something this
>> would only need a stand-alone fix for 4.18/stable which I can cook up and
>> submit then.
> 
> I can confirm that fixes the problems I was seeing on Fedora 29.
> 
> Feel free to add a tested by from me:
> 
> Tested-by: Peter Robinson <pbrobinson@gmail.com>

Great, thanks everyone! Will get it out asap.

^ permalink raw reply	[flat|nested] 50+ messages in thread
end of thread, other threads:[~2018-08-18  1:11 UTC | newest]

Thread overview: 50+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-06-22 11:19 Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1 Peter Robinson
2018-06-22 11:19 ` Peter Robinson
2018-06-22 12:55 ` Eric Dumazet
2018-06-22 12:55   ` Eric Dumazet
2018-06-24  9:24   ` Peter Robinson
2018-06-24  9:24     ` Peter Robinson
2018-06-25  8:48     ` Daniel Borkmann
2018-06-25  8:48       ` Daniel Borkmann
2018-06-25 12:03       ` Peter Robinson
2018-06-25 12:03         ` Peter Robinson
     [not found]     ` <ad98d60c-bd60-b495-c4bd-507fc29c8bcd@iogearbox.net>
     [not found]       ` <CALeDE9PBZWJBp8KB0mB4zoNXqscmzxWzz+LnuqRA-z4t1e9T8g@mail.gmail.com>
2018-06-25 16:41         ` [offlist] " Peter Robinson
2018-06-25 16:41           ` Peter Robinson
2018-06-26 12:23           ` Peter Robinson
2018-06-26 12:23             ` Peter Robinson
2018-06-26 12:52             ` Daniel Borkmann
2018-06-26 12:52               ` Daniel Borkmann
2018-07-04  7:33               ` Peter Robinson
2018-07-04  7:33                 ` Peter Robinson
2018-07-04 23:10                 ` Daniel Borkmann
2018-07-04 23:10                   ` Daniel Borkmann
2018-07-04 23:41                 ` Russell King - ARM Linux
2018-07-04 23:41                   ` Russell King - ARM Linux
2018-07-05  7:31                   ` Russell King - ARM Linux
2018-07-05  7:31                     ` Russell King - ARM Linux
2018-07-05  7:46                     ` Daniel Borkmann
2018-07-05  7:46                       ` Daniel Borkmann
2018-08-16 20:35           ` Marc Haber
2018-08-16 20:35             ` Marc Haber
2018-08-16 22:58             ` Russell King - ARM Linux
2018-08-16 22:58               ` Russell King - ARM Linux
2018-08-17 12:25               ` Peter Robinson
2018-08-17 12:25                 ` Peter Robinson
2018-08-17 12:40                 ` Daniel Borkmann
2018-08-17 12:40                   ` Daniel Borkmann
2018-08-17 14:32                   ` Peter Robinson
2018-08-17 14:32                     ` Peter Robinson
2018-08-17 16:17                   ` Russell King - ARM Linux
2018-08-17 16:17                     ` Russell King - ARM Linux
2018-08-17 18:30                     ` Daniel Borkmann
2018-08-17 18:30                       ` Daniel Borkmann
2018-08-17 18:51                       ` Stefan Wahren
2018-08-17 18:51                         ` Stefan Wahren
2018-08-17 21:15                         ` Peter Robinson
2018-08-17 21:15                           ` Peter Robinson
2018-08-17 21:13                       ` Peter Robinson
2018-08-17 21:13                         ` Peter Robinson
2018-08-17 22:06                         ` Daniel Borkmann
2018-08-17 22:06                           ` Daniel Borkmann
2018-08-17 21:12                     ` Peter Robinson
2018-08-17 21:12                       ` Peter Robinson

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.