Re: latency histogram with BPF

From: Daniel Wagner <daniel.wagner@bmw-carit.de>
To: Daniel Wagner <daniel.wagner@bmw-carit.de>,
	Alexei Starovoitov <ast@plumgrid.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Tom Zanussi <tom.zanussi@linux.intel.com>
Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	Wang Nan <wangnan0@huawei.com>
Subject: Re: latency histogram with BPF
Date: Fri, 12 Jun 2015 16:33:17 +0200	[thread overview]
Message-ID: <557AEDAD.2060507@bmw-carit.de> (raw)
In-Reply-To: <557A783C.5080803@bmw-carit.de>

On 06/12/2015 08:12 AM, Daniel Wagner wrote:
> On 06/12/2015 12:08 AM, Alexei Starovoitov wrote:
>> On 6/11/15 12:25 AM, Daniel Wagner wrote:
>> If you have any suggestions on where to look, I'm all ears.
>> My stack traces look like:
>> Running with 10*40 (== 400) tasks.
>> [   12.032571] kernel BUG at ../mm/slub.c:3413!

I hit this as well.

After looking and playing around for while I think I found the source of
the problem: The path from the BPF program into the hash table code is
triggering the crash.

Attaching kprobes to trace_preempt_[on|off] works fine. Empty BPF
programs connected to the probes is no problem as well. So I changed the
BPF program to use only arrays instead of hash tables. No crash anymore.

Thanks Steven for the tip :)

I suspect the hash table code will call trace_preempt_[off|on]
eventually and that is not going to fly.

The program is still in a rough state. I'll send an cleanup version next
week. (My brain is melting down, too hot here...)

cheers,
daniel