All of lore.kernel.org
 help / color / mirror / Atom feed
From: Daniel Borkmann <daniel@iogearbox.net>
To: Peter Robinson <pbrobinson@gmail.com>,
	Russell King - ARM Linux <linux@armlinux.org.uk>
Cc: Marc Haber <mh+netdev@zugschlus.de>,
	linux-arm-kernel@lists.infradead.org, netdev@vger.kernel.org,
	labbott@redhat.com, Eric Dumazet <eric.dumazet@gmail.com>
Subject: Re: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
Date: Fri, 17 Aug 2018 14:40:19 +0200	[thread overview]
Message-ID: <1c2218cb-63bf-1528-6156-8ce93f46169c@iogearbox.net> (raw)
In-Reply-To: <CALeDE9Pm7nsqDFL0m0ZsNdnEti6YAbEPhtfbNMPe=UZCUyyHMA@mail.gmail.com>

On 08/17/2018 02:25 PM, Peter Robinson wrote:
> On Thu, Aug 16, 2018 at 11:58 PM, Russell King - ARM Linux
> <linux@armlinux.org.uk> wrote:
>> On Thu, Aug 16, 2018 at 10:35:16PM +0200, Marc Haber wrote:
>>> On Mon, Jun 25, 2018 at 05:41:27PM +0100, Peter Robinson wrote:
>>>> So with that and the other fix there was no improvement, with those
>>>> and the BPF JIT disabled it works, I'm not sure if the two patches
>>>> have any effect with the JIT disabled though.
>>>
>>> I can confirm the crash with the released 4.18.1 on Banana Pi, and I can
>>> also confirm that disabling BPF JIT makes the Banana Pi work again.,
>>
>> I'm afraid that the information in the crash dumps is insufficient
>> to be able to work very much out about these crashes.
>>
>> We need a recipe (kernel configuration and what userspace is doing)
>> so that it's possible to recreate the crash, or we need responses
>> to requests for information - I requested the disassembly of
>> sk_filter_trim_cap and the BPF code dump via setting a sysctl back
>> in early July.  Without this, as I say, I don't see how this problem
>> can be progressed.
> 
> I can provide a kernel config [1] but I've not had enough time to sit
> down and get the rest of the stuff and debug it due to a combination
> of travel and other priorities.

Did you get a chance to try latest kernel from Linus' tree [1] from last
few days to see whether the issue is still persistent? There have been
a number of improvements, bit strange why e.g. Russell didn't run into
it while others have, hmm. Perhaps due to EABI vs non EABI.

[1] git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

>> If the problem is at boot, one way to set the sysctl would be to
>> hack the kernel and explicitly initialise the sysctl to '2', or
>> boot with init=/bin/sh, then manually mount /proc, set the sysctl,
>> and then "exec /sbin/init" from that shell.  (Remember there's no
>> job control in that shell, so ^z, ^c, etc do not work.)
> 
> It starts to happen in the early kernel boot long before we get to any
> userspace across a number of ARMv7 devices (RPi2/3, BeagleBone and
> AllWinner H3 based devices at least).
> 
> [1] https://pbrobinson.fedorapeople.org/kernel-armv7hl.config

I'd have one potential bug suspicion, for the 4.18 one you were trying,
could you run with the below patch to see whether it would help?

Thanks,
Daniel

diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index f6a62ae..c864f6b 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -238,7 +238,7 @@ static void jit_fill_hole(void *area, unsigned int size)
 #define STACK_SIZE	ALIGN(_STACK_SIZE, STACK_ALIGNMENT)

 /* Get the offset of eBPF REGISTERs stored on scratch space. */
-#define STACK_VAR(off) (STACK_SIZE - off)
+#define STACK_VAR(off) (STACK_SIZE - off - 4)

 #if __LINUX_ARM_ARCH__ < 7

WARNING: multiple messages have this Message-ID (diff)
From: daniel@iogearbox.net (Daniel Borkmann)
To: linux-arm-kernel@lists.infradead.org
Subject: [offlist] Re: Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1
Date: Fri, 17 Aug 2018 14:40:19 +0200	[thread overview]
Message-ID: <1c2218cb-63bf-1528-6156-8ce93f46169c@iogearbox.net> (raw)
In-Reply-To: <CALeDE9Pm7nsqDFL0m0ZsNdnEti6YAbEPhtfbNMPe=UZCUyyHMA@mail.gmail.com>

On 08/17/2018 02:25 PM, Peter Robinson wrote:
> On Thu, Aug 16, 2018 at 11:58 PM, Russell King - ARM Linux
> <linux@armlinux.org.uk> wrote:
>> On Thu, Aug 16, 2018 at 10:35:16PM +0200, Marc Haber wrote:
>>> On Mon, Jun 25, 2018 at 05:41:27PM +0100, Peter Robinson wrote:
>>>> So with that and the other fix there was no improvement, with those
>>>> and the BPF JIT disabled it works, I'm not sure if the two patches
>>>> have any effect with the JIT disabled though.
>>>
>>> I can confirm the crash with the released 4.18.1 on Banana Pi, and I can
>>> also confirm that disabling BPF JIT makes the Banana Pi work again.,
>>
>> I'm afraid that the information in the crash dumps is insufficient
>> to be able to work very much out about these crashes.
>>
>> We need a recipe (kernel configuration and what userspace is doing)
>> so that it's possible to recreate the crash, or we need responses
>> to requests for information - I requested the disassembly of
>> sk_filter_trim_cap and the BPF code dump via setting a sysctl back
>> in early July.  Without this, as I say, I don't see how this problem
>> can be progressed.
> 
> I can provide a kernel config [1] but I've not had enough time to sit
> down and get the rest of the stuff and debug it due to a combination
> of travel and other priorities.

Did you get a chance to try latest kernel from Linus' tree [1] from last
few days to see whether the issue is still persistent? There have been
a number of improvements, bit strange why e.g. Russell didn't run into
it while others have, hmm. Perhaps due to EABI vs non EABI.

[1] git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

>> If the problem is at boot, one way to set the sysctl would be to
>> hack the kernel and explicitly initialise the sysctl to '2', or
>> boot with init=/bin/sh, then manually mount /proc, set the sysctl,
>> and then "exec /sbin/init" from that shell.  (Remember there's no
>> job control in that shell, so ^z, ^c, etc do not work.)
> 
> It starts to happen in the early kernel boot long before we get to any
> userspace across a number of ARMv7 devices (RPi2/3, BeagleBone and
> AllWinner H3 based devices at least).
> 
> [1] https://pbrobinson.fedorapeople.org/kernel-armv7hl.config

I'd have one potential bug suspicion, for the 4.18 one you were trying,
could you run with the below patch to see whether it would help?

Thanks,
Daniel

diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
index f6a62ae..c864f6b 100644
--- a/arch/arm/net/bpf_jit_32.c
+++ b/arch/arm/net/bpf_jit_32.c
@@ -238,7 +238,7 @@ static void jit_fill_hole(void *area, unsigned int size)
 #define STACK_SIZE	ALIGN(_STACK_SIZE, STACK_ALIGNMENT)

 /* Get the offset of eBPF REGISTERs stored on scratch space. */
-#define STACK_VAR(off) (STACK_SIZE - off)
+#define STACK_VAR(off) (STACK_SIZE - off - 4)

 #if __LINUX_ARM_ARCH__ < 7

  reply	other threads:[~2018-08-17 15:43 UTC|newest]

Thread overview: 50+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-06-22 11:19 Crash in netlink/sk_filter_trim_cap on ARMv7 on 4.18rc1 Peter Robinson
2018-06-22 11:19 ` Peter Robinson
2018-06-22 12:55 ` Eric Dumazet
2018-06-22 12:55   ` Eric Dumazet
2018-06-24  9:24   ` Peter Robinson
2018-06-24  9:24     ` Peter Robinson
2018-06-25  8:48     ` Daniel Borkmann
2018-06-25  8:48       ` Daniel Borkmann
2018-06-25 12:03       ` Peter Robinson
2018-06-25 12:03         ` Peter Robinson
     [not found]     ` <ad98d60c-bd60-b495-c4bd-507fc29c8bcd@iogearbox.net>
     [not found]       ` <CALeDE9PBZWJBp8KB0mB4zoNXqscmzxWzz+LnuqRA-z4t1e9T8g@mail.gmail.com>
2018-06-25 16:41         ` [offlist] " Peter Robinson
2018-06-25 16:41           ` Peter Robinson
2018-06-26 12:23           ` Peter Robinson
2018-06-26 12:23             ` Peter Robinson
2018-06-26 12:52             ` Daniel Borkmann
2018-06-26 12:52               ` Daniel Borkmann
2018-07-04  7:33               ` Peter Robinson
2018-07-04  7:33                 ` Peter Robinson
2018-07-04 23:10                 ` Daniel Borkmann
2018-07-04 23:10                   ` Daniel Borkmann
2018-07-04 23:41                 ` Russell King - ARM Linux
2018-07-04 23:41                   ` Russell King - ARM Linux
2018-07-05  7:31                   ` Russell King - ARM Linux
2018-07-05  7:31                     ` Russell King - ARM Linux
2018-07-05  7:46                     ` Daniel Borkmann
2018-07-05  7:46                       ` Daniel Borkmann
2018-08-16 20:35           ` Marc Haber
2018-08-16 20:35             ` Marc Haber
2018-08-16 22:58             ` Russell King - ARM Linux
2018-08-16 22:58               ` Russell King - ARM Linux
2018-08-17 12:25               ` Peter Robinson
2018-08-17 12:25                 ` Peter Robinson
2018-08-17 12:40                 ` Daniel Borkmann [this message]
2018-08-17 12:40                   ` Daniel Borkmann
2018-08-17 14:32                   ` Peter Robinson
2018-08-17 14:32                     ` Peter Robinson
2018-08-17 16:17                   ` Russell King - ARM Linux
2018-08-17 16:17                     ` Russell King - ARM Linux
2018-08-17 18:30                     ` Daniel Borkmann
2018-08-17 18:30                       ` Daniel Borkmann
2018-08-17 18:51                       ` Stefan Wahren
2018-08-17 18:51                         ` Stefan Wahren
2018-08-17 21:15                         ` Peter Robinson
2018-08-17 21:15                           ` Peter Robinson
2018-08-17 21:13                       ` Peter Robinson
2018-08-17 21:13                         ` Peter Robinson
2018-08-17 22:06                         ` Daniel Borkmann
2018-08-17 22:06                           ` Daniel Borkmann
2018-08-17 21:12                     ` Peter Robinson
2018-08-17 21:12                       ` Peter Robinson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1c2218cb-63bf-1528-6156-8ce93f46169c@iogearbox.net \
    --to=daniel@iogearbox.net \
    --cc=eric.dumazet@gmail.com \
    --cc=labbott@redhat.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux@armlinux.org.uk \
    --cc=mh+netdev@zugschlus.de \
    --cc=netdev@vger.kernel.org \
    --cc=pbrobinson@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.