All of lore.kernel.org
 help / color / mirror / Atom feed
From: John Stultz <john.stultz@linaro.org>
To: "Stephen Boyd" <sboyd@codeaurora.org>,
	"Linus Torvalds" <torvalds@linux-foundation.org>,
	"Krzysztof Hałasa" <khalasa@piap.pl>
Cc: "Uwe Kleine-König" <u.kleine-koenig@pengutronix.de>,
	"Willy Tarreau" <w@1wt.eu>, lkml <linux-kernel@vger.kernel.org>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>,
	"Ingo Molnar" <mingo@kernel.org>
Subject: Re: v3.13-rc6+ regression (ARM board)
Date: Thu, 02 Jan 2014 12:52:56 -0800	[thread overview]
Message-ID: <52C5D1A8.8070706@linaro.org> (raw)
In-Reply-To: <52C5CF38.1010704@codeaurora.org>

On 01/02/2014 12:42 PM, Stephen Boyd wrote:
> On 01/02/14 12:30, John Stultz wrote:
>> On 01/02/2014 12:03 PM, John Stultz wrote:
>>> On 01/02/2014 11:38 AM, Linus Torvalds wrote:
>>>> On Thu, Jan 2, 2014 at 4:07 AM, Krzysztof Hałasa <khalasa@piap.pl> wrote:
>>>>> This means these two commits don't like each other:
>>>>>
>>>>>     seqcount: Add lockdep functionality to seqcount/seqlock structures
>>>>>     sched_clock: Use seqcount instead of rolling our own
>>>> Does something like this fix it for you?
>>>>
>>>>   --- a/kernel/time/sched_clock.c
>>>>   +++ b/kernel/time/sched_clock.c
>>>>   @@ -36,6 +36,7 @@ core_param(irqtime, irqtime, int, 0400);
>>>>
>>>>    static struct clock_data cd = {
>>>>           .mult   = NSEC_PER_SEC / HZ,
>>>>   +       .seq = SEQCNT_ZERO(cd.seq),
>>>>    };
>>>>
>>>>    static u64 __read_mostly sched_clock_mask;
>>>>
>>>> (The above is not even compile-tested, because x86 doesn't use
>>>> GENERIC_SCHED_CLOCK. So I did the patch blindly, but I think you get
>>>> the idea..)
>>> Sheesh. Just finishing up holiday email backlog and Linus already has a
>>> fix. :)
>>>
>>> This looks like it should fix the issue, and does build for me.
>>>
>>> Assuming it works for Krzysztof,
>> So something else may be at play. Even with Linus' patch I reproduced a
>> similar hang here.
>>
>> Still chasing it down, but it looks like a seqlock deadlock where we're
>> calling read while holding the lock.
>>
> Do you have tracing enabled? When I moved this code over to use
> seqcounts it relied on the fact that the compiler wouldn't be generating
> any function calls to the tracing code. Before seqcounts got lockdep
> support it all collapsed down into sched_clock() due to the use of
> inline on the seqlock API.

Hrm. I have tracing compiled in, I'll see if disabling it avoids the issue.

If this is the problem, I'm guessing we may need to change it to use
read_seqcount_begin_no_lockdep() then.

But I still don't have a clear sense of exactly whats happening yet.

thanks
-john


WARNING: multiple messages have this Message-ID (diff)
From: john.stultz@linaro.org (John Stultz)
To: linux-arm-kernel@lists.infradead.org
Subject: v3.13-rc6+ regression (ARM board)
Date: Thu, 02 Jan 2014 12:52:56 -0800	[thread overview]
Message-ID: <52C5D1A8.8070706@linaro.org> (raw)
In-Reply-To: <52C5CF38.1010704@codeaurora.org>

On 01/02/2014 12:42 PM, Stephen Boyd wrote:
> On 01/02/14 12:30, John Stultz wrote:
>> On 01/02/2014 12:03 PM, John Stultz wrote:
>>> On 01/02/2014 11:38 AM, Linus Torvalds wrote:
>>>> On Thu, Jan 2, 2014 at 4:07 AM, Krzysztof Ha?asa <khalasa@piap.pl> wrote:
>>>>> This means these two commits don't like each other:
>>>>>
>>>>>     seqcount: Add lockdep functionality to seqcount/seqlock structures
>>>>>     sched_clock: Use seqcount instead of rolling our own
>>>> Does something like this fix it for you?
>>>>
>>>>   --- a/kernel/time/sched_clock.c
>>>>   +++ b/kernel/time/sched_clock.c
>>>>   @@ -36,6 +36,7 @@ core_param(irqtime, irqtime, int, 0400);
>>>>
>>>>    static struct clock_data cd = {
>>>>           .mult   = NSEC_PER_SEC / HZ,
>>>>   +       .seq = SEQCNT_ZERO(cd.seq),
>>>>    };
>>>>
>>>>    static u64 __read_mostly sched_clock_mask;
>>>>
>>>> (The above is not even compile-tested, because x86 doesn't use
>>>> GENERIC_SCHED_CLOCK. So I did the patch blindly, but I think you get
>>>> the idea..)
>>> Sheesh. Just finishing up holiday email backlog and Linus already has a
>>> fix. :)
>>>
>>> This looks like it should fix the issue, and does build for me.
>>>
>>> Assuming it works for Krzysztof,
>> So something else may be at play. Even with Linus' patch I reproduced a
>> similar hang here.
>>
>> Still chasing it down, but it looks like a seqlock deadlock where we're
>> calling read while holding the lock.
>>
> Do you have tracing enabled? When I moved this code over to use
> seqcounts it relied on the fact that the compiler wouldn't be generating
> any function calls to the tracing code. Before seqcounts got lockdep
> support it all collapsed down into sched_clock() due to the use of
> inline on the seqlock API.

Hrm. I have tracing compiled in, I'll see if disabling it avoids the issue.

If this is the problem, I'm guessing we may need to change it to use
read_seqcount_begin_no_lockdep() then.

But I still don't have a clear sense of exactly whats happening yet.

thanks
-john

  reply	other threads:[~2014-01-02 20:53 UTC|newest]

Thread overview: 52+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-12-31 10:37 v3.13-rc6+ regression (ARM board) Krzysztof Hałasa
2013-12-31 10:37 ` Krzysztof Hałasa
2013-12-31 10:45 ` Willy Tarreau
2013-12-31 10:45   ` Willy Tarreau
2014-01-02 10:02   ` Krzysztof Hałasa
2014-01-02 10:02     ` Krzysztof Hałasa
2014-01-02 10:14     ` Uwe Kleine-König
2014-01-02 10:14       ` Uwe Kleine-König
2014-01-02 12:07       ` Krzysztof Hałasa
2014-01-02 12:07         ` Krzysztof Hałasa
2014-01-02 19:38         ` Linus Torvalds
2014-01-02 19:38           ` Linus Torvalds
2014-01-02 20:03           ` John Stultz
2014-01-02 20:03             ` John Stultz
2014-01-02 20:30             ` John Stultz
2014-01-02 20:30               ` John Stultz
2014-01-02 20:42               ` Stephen Boyd
2014-01-02 20:42                 ` Stephen Boyd
2014-01-02 20:52                 ` John Stultz [this message]
2014-01-02 20:52                   ` John Stultz
2014-01-02 20:43               ` Linus Torvalds
2014-01-02 20:43                 ` Linus Torvalds
2014-01-02 21:34                 ` John Stultz
2014-01-02 21:34                   ` John Stultz
2014-01-02 21:54                   ` [PATCH] sched_clock: Disable seqlock lockdep usage in sched_clock John Stultz
2014-01-02 21:54                     ` John Stultz
2014-01-02 22:15                     ` Linus Torvalds
2014-01-02 22:15                       ` Linus Torvalds
2014-01-02 22:21                       ` John Stultz
2014-01-02 22:21                         ` John Stultz
2014-01-02 23:11                         ` [PATCH 1/2] seqlock: Use raw_ prefix instead of _no_lockdep John Stultz
2014-01-02 23:11                           ` John Stultz
2014-01-02 23:11                           ` [PATCH 2/2] sched_clock: Disable seqlock lockdep usage in sched_clock John Stultz
2014-01-02 23:11                             ` John Stultz
2014-01-03  0:46                             ` Stephen Boyd
2014-01-03  0:46                               ` Stephen Boyd
2014-01-03  6:05                             ` Krzysztof Hałasa
2014-01-03  6:05                               ` Krzysztof Hałasa
2014-01-12 18:42                             ` [tip:core/urgent] sched_clock: Disable seqlock lockdep usage in sched_clock() tip-bot for John Stultz
2014-01-14 19:18                               ` John Stultz
2014-01-15  6:38                                 ` Ingo Molnar
2014-01-03  0:46                           ` [PATCH 1/2] seqlock: Use raw_ prefix instead of _no_lockdep Stephen Boyd
2014-01-03  0:46                             ` Stephen Boyd
2014-01-03  0:50                           ` Linus Torvalds
2014-01-03  0:50                             ` Linus Torvalds
2014-01-04  0:28                             ` John Stultz
2014-01-04  0:28                               ` John Stultz
2014-01-06 10:10                               ` Peter Zijlstra
2014-01-06 10:10                                 ` Peter Zijlstra
2014-01-12 18:42                           ` [tip:core/urgent] " tip-bot for John Stultz
2014-01-03  6:01           ` v3.13-rc6+ regression (ARM board) Krzysztof Hałasa
2014-01-03  6:01             ` Krzysztof Hałasa

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=52C5D1A8.8070706@linaro.org \
    --to=john.stultz@linaro.org \
    --cc=khalasa@piap.pl \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@kernel.org \
    --cc=sboyd@codeaurora.org \
    --cc=torvalds@linux-foundation.org \
    --cc=u.kleine-koenig@pengutronix.de \
    --cc=w@1wt.eu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.