All of lore.kernel.org
 help / color / mirror / Atom feed
From: Tony Lindgren <tony@atomide.com>
To: "Rafael J. Wysocki" <rafael@kernel.org>
Cc: Guenter Roeck <linux@roeck-us.net>,
	Viresh Kumar <viresh.kumar@linaro.org>,
	"linux-pm@vger.kernel.org" <linux-pm@vger.kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	"Rafael J. Wysocki" <rafael.j.wysocki@intel.com>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	linux-next@vger.kernel.org,
	"linux-arm-kernel@lists.infradead.org" 
	<linux-arm-kernel@lists.infradead.org>
Subject: Re: Crashes in arm qemu emulations due to 'cpufreq: governor: Replace timers with utilization ...'
Date: Mon, 15 Feb 2016 11:01:17 -0800	[thread overview]
Message-ID: <20160215190116.GX19432@atomide.com> (raw)
In-Reply-To: <CAJZ5v0gLankSuziQq25qTCyNqeOX43yD9jnJu_XXwbdyajfmKg@mail.gmail.com>

* Rafael J. Wysocki <rafael@kernel.org> [160215 10:44]:
> On Mon, Feb 15, 2016 at 6:05 PM, Guenter Roeck <linux@roeck-us.net> wrote:
> > Rafael,
> 
> Hi,
> 
> Thanks for the report!
> 
> > I see crashes in various arm qemu tests due to 'cpufreq: governor: Replace
> > timers with utilization update callbacks' with next-20160215. An example
> > crash log and bisect results are attached below.
> >
> > Please let me know if there is anything I can do to help tracking down
> > the problem.
> 
> It looks like we've uncovered some nastiness in the arch ARM code (see below).
> 
> [cut]
> 
> > [    1.340000] Unable to handle kernel NULL pointer dereference at virtual address 00000000
> > [    1.340000] pgd = c0204000
> > [    1.340000] [00000000] *pgd=00000000
> > [    1.340000] Internal error: Oops: 80000005 [#1] SMP ARM
> > [    1.340000] Modules linked in:
> > [    1.340000] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.5.0-rc4-next-20160215 #1
> > [    1.340000] Hardware name: Generic OMAP3-GP (Flattened Device Tree)
> > [    1.340000] task: cb060000 ti: cb05a000 task.ti: cb05a000
> > [    1.340000] PC is at 0x0
> > [    1.340000] LR is at arch_send_call_function_single_ipi+0x34/0x38
> 
> Since this is ARM, arch_send_call_function_single_ipi() looks like this:
> 
> void arch_send_call_function_single_ipi(int cpu)
> {
>          smp_cross_call(cpumask_of(cpu), IPI_CALL_FUNC_SINGLE);
> }
> 
> so I'm not sure how the NULL pointer deref is possible even.
> 
> The only thing coming to mind would be that cpumask_of(cpu) triggers
> this, but I'm not sure how exactly that can happen.
> 
> I need help from somebody who knows how this low-level stuff works on ARM.

That's not even an SMP machine? I suspect a bunch of out of the
65 boot failures here are related to this:

https://kernelci.org/boot/all/job/next/kernel/next-20160215/

The SMP ones seem to fail with some regulator issues?

Regards,

Tony

WARNING: multiple messages have this Message-ID (diff)
From: tony@atomide.com (Tony Lindgren)
To: linux-arm-kernel@lists.infradead.org
Subject: Crashes in arm qemu emulations due to 'cpufreq: governor: Replace timers with utilization ...'
Date: Mon, 15 Feb 2016 11:01:17 -0800	[thread overview]
Message-ID: <20160215190116.GX19432@atomide.com> (raw)
In-Reply-To: <CAJZ5v0gLankSuziQq25qTCyNqeOX43yD9jnJu_XXwbdyajfmKg@mail.gmail.com>

* Rafael J. Wysocki <rafael@kernel.org> [160215 10:44]:
> On Mon, Feb 15, 2016 at 6:05 PM, Guenter Roeck <linux@roeck-us.net> wrote:
> > Rafael,
> 
> Hi,
> 
> Thanks for the report!
> 
> > I see crashes in various arm qemu tests due to 'cpufreq: governor: Replace
> > timers with utilization update callbacks' with next-20160215. An example
> > crash log and bisect results are attached below.
> >
> > Please let me know if there is anything I can do to help tracking down
> > the problem.
> 
> It looks like we've uncovered some nastiness in the arch ARM code (see below).
> 
> [cut]
> 
> > [    1.340000] Unable to handle kernel NULL pointer dereference at virtual address 00000000
> > [    1.340000] pgd = c0204000
> > [    1.340000] [00000000] *pgd=00000000
> > [    1.340000] Internal error: Oops: 80000005 [#1] SMP ARM
> > [    1.340000] Modules linked in:
> > [    1.340000] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.5.0-rc4-next-20160215 #1
> > [    1.340000] Hardware name: Generic OMAP3-GP (Flattened Device Tree)
> > [    1.340000] task: cb060000 ti: cb05a000 task.ti: cb05a000
> > [    1.340000] PC is at 0x0
> > [    1.340000] LR is at arch_send_call_function_single_ipi+0x34/0x38
> 
> Since this is ARM, arch_send_call_function_single_ipi() looks like this:
> 
> void arch_send_call_function_single_ipi(int cpu)
> {
>          smp_cross_call(cpumask_of(cpu), IPI_CALL_FUNC_SINGLE);
> }
> 
> so I'm not sure how the NULL pointer deref is possible even.
> 
> The only thing coming to mind would be that cpumask_of(cpu) triggers
> this, but I'm not sure how exactly that can happen.
> 
> I need help from somebody who knows how this low-level stuff works on ARM.

That's not even an SMP machine? I suspect a bunch of out of the
65 boot failures here are related to this:

https://kernelci.org/boot/all/job/next/kernel/next-20160215/

The SMP ones seem to fail with some regulator issues?

Regards,

Tony

  parent reply	other threads:[~2016-02-15 19:01 UTC|newest]

Thread overview: 81+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-02-15 17:05 Crashes in arm qemu emulations due to 'cpufreq: governor: Replace timers with utilization ...' Guenter Roeck
2016-02-15 17:05 ` Guenter Roeck
2016-02-15 18:41 ` Rafael J. Wysocki
2016-02-15 18:41   ` Rafael J. Wysocki
2016-02-15 18:41   ` Rafael J. Wysocki
2016-02-15 18:49   ` Rafael J. Wysocki
2016-02-15 18:49     ` Rafael J. Wysocki
2016-02-15 18:49     ` Rafael J. Wysocki
2016-02-15 18:49   ` Marc Zyngier
2016-02-15 18:49     ` Marc Zyngier
2016-02-15 18:49     ` Marc Zyngier
2016-02-15 18:54     ` Rafael J. Wysocki
2016-02-15 18:54       ` Rafael J. Wysocki
2016-02-15 18:54       ` Rafael J. Wysocki
2016-02-15 19:03       ` Marc Zyngier
2016-02-15 19:03         ` Marc Zyngier
2016-02-15 19:03         ` Marc Zyngier
2016-02-15 19:12         ` Rafael J. Wysocki
2016-02-15 19:12           ` Rafael J. Wysocki
2016-02-15 19:12           ` Rafael J. Wysocki
2016-02-15 19:28           ` Rafael J. Wysocki
2016-02-15 19:28             ` Rafael J. Wysocki
2016-02-15 19:28             ` Rafael J. Wysocki
2016-02-15 19:42             ` Tony Lindgren
2016-02-15 19:42               ` Tony Lindgren
2016-02-15 19:42               ` Tony Lindgren
2016-02-15 19:46               ` Guenter Roeck
2016-02-15 19:46                 ` Guenter Roeck
2016-02-15 19:46                 ` Guenter Roeck
2016-02-15 19:57                 ` Tony Lindgren
2016-02-15 19:57                   ` Tony Lindgren
2016-02-15 19:57                   ` Tony Lindgren
2016-02-15 19:23         ` Russell King - ARM Linux
2016-02-15 19:23           ` Russell King - ARM Linux
2016-02-15 19:23           ` Russell King - ARM Linux
2016-02-15 20:41           ` Rafael J. Wysocki
2016-02-15 20:41             ` Rafael J. Wysocki
2016-02-15 20:41             ` Rafael J. Wysocki
2016-02-15 19:07       ` Russell King - ARM Linux
2016-02-15 19:07         ` Russell King - ARM Linux
2016-02-15 19:07         ` Russell King - ARM Linux
2016-02-15 19:01   ` Tony Lindgren [this message]
2016-02-15 19:01     ` Tony Lindgren
2016-02-15 19:01     ` Tony Lindgren
2016-02-15 19:40     ` Guenter Roeck
2016-02-15 19:40       ` Guenter Roeck
2016-02-15 19:40       ` Guenter Roeck
2016-02-15 19:58       ` Tony Lindgren
2016-02-15 19:58         ` Tony Lindgren
2016-02-15 19:58         ` Tony Lindgren
2016-02-15 20:09         ` Guenter Roeck
2016-02-15 20:09           ` Guenter Roeck
2016-02-15 20:09           ` Guenter Roeck
2016-02-15 20:38           ` Rafael J. Wysocki
2016-02-15 20:38             ` Rafael J. Wysocki
2016-02-15 20:38             ` Rafael J. Wysocki
2016-02-15 20:37         ` Rafael J. Wysocki
2016-02-15 20:37           ` Rafael J. Wysocki
2016-02-15 20:37           ` Rafael J. Wysocki
2016-02-15 21:36           ` Tony Lindgren
2016-02-15 21:36             ` Tony Lindgren
2016-02-15 21:36             ` Tony Lindgren
2016-02-16  1:38             ` Guenter Roeck
2016-02-16  1:38               ` Guenter Roeck
2016-02-16  1:38               ` Guenter Roeck
2016-02-15 19:02   ` Russell King - ARM Linux
2016-02-15 19:02     ` Russell King - ARM Linux
2016-02-15 19:02     ` Russell King - ARM Linux
2016-02-16  1:13   ` Viresh Kumar
2016-02-16  1:13     ` Viresh Kumar
2016-02-16  1:13     ` Viresh Kumar
2016-02-16  1:27     ` Rafael J. Wysocki
2016-02-16  1:27       ` Rafael J. Wysocki
2016-02-16  1:27       ` Rafael J. Wysocki
2016-02-16  1:36       ` Viresh Kumar
2016-02-16  1:36         ` Viresh Kumar
2016-02-16  1:36         ` Viresh Kumar
2016-02-15 22:29 ` Peter Maydell
2016-02-15 22:29   ` Peter Maydell
2016-02-15 23:19   ` Guenter Roeck
2016-02-15 23:19     ` Guenter Roeck

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20160215190116.GX19432@atomide.com \
    --to=tony@atomide.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-next@vger.kernel.org \
    --cc=linux-pm@vger.kernel.org \
    --cc=linux@roeck-us.net \
    --cc=peterz@infradead.org \
    --cc=rafael.j.wysocki@intel.com \
    --cc=rafael@kernel.org \
    --cc=viresh.kumar@linaro.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.