linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Dexuan Cui <decui@microsoft.com>
To: "paulmck@kernel.org" <paulmck@kernel.org>
Cc: "boqun.feng@gmail.com" <boqun.feng@gmail.com>,
	Ingo Molnar <mingo@redhat.com>,
	"rcu@vger.kernel.org" <rcu@vger.kernel.org>,
	vkuznets <vkuznets@redhat.com>,
	Michael Kelley <mikelley@microsoft.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: kdump always hangs in rcu_barrier() -> wait_for_completion()
Date: Thu, 26 Nov 2020 22:59:19 +0000	[thread overview]
Message-ID: <MW2PR2101MB18011DA2FCF66D03BF8BB0CCBFF91@MW2PR2101MB1801.namprd21.prod.outlook.com> (raw)
In-Reply-To: <20201126214226.GS1437@paulmck-ThinkPad-P72>

> From: Paul E. McKenney <paulmck@kernel.org>
> Sent: Thursday, November 26, 2020 1:42 PM
> 
> > > Another possibility is that rcu_state.gp_kthread is non-NULL, but that
> > > something else is preventing RCU grace periods from completing, but in
> >
> > It looks like somehow the scheduling is not working here: in rcu_barrier()
> > , if I replace the wait_for_completion() with
> > wait_for_completion_timeout(&rcu_state.barrier_completion, 30*HZ), the
> > issue persists.
> 
> Have you tried using sysreq-t to see what the various tasks are doing?

Will try it.

BTW, this is a "Generation 2" VM on Hyper-V, meaning sysrq only starts to
work after the Hyper-V para-virtualized keyboard driver loads... So, at this
early point, sysrq is not working. :-( I'll have to hack the code and use a 
virtual NMI interrupt to force the sysrq handler to be called.
 
> Having interrupts disabled on all CPUs would have the effect of disabling
> the RCU CPU stall warnings.
> 							Thanx, Paul

I'm sure the interrupts are not disabled. Here the VM only has 1 virtual CPU,
and when the hang issue happens the virtual serial console is still responding
when I press Enter (it prints a new line) or Ctrl+C (it prints ^C).

Here the VM does not use the "legacy timers" (PIT, Local APIC timer, etc.) at all.
Instead, the VM uses the Hyper-V para-virtualized timers. It looks the Hyper-V
timer never fires in the kdump kernel when the hang issue happens. I'm 
looking into this... I suspect this hang issue may only be specific to Hyper-V.

Thanks,
-- Dexuan

  reply	other threads:[~2020-11-26 22:59 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-25  4:56 kdump always hangs in rcu_barrier() -> wait_for_completion() Dexuan Cui
2020-11-26 15:46 ` Paul E. McKenney
2020-11-26 21:25   ` Dexuan Cui
2020-11-26 21:42     ` Paul E. McKenney
2020-11-26 22:59       ` Dexuan Cui [this message]
2020-11-26 23:54         ` Paul E. McKenney
2021-01-28  7:28           ` Dexuan Cui
2021-01-28 14:58             ` Paul E. McKenney

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=MW2PR2101MB18011DA2FCF66D03BF8BB0CCBFF91@MW2PR2101MB1801.namprd21.prod.outlook.com \
    --to=decui@microsoft.com \
    --cc=boqun.feng@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mikelley@microsoft.com \
    --cc=mingo@redhat.com \
    --cc=paulmck@kernel.org \
    --cc=rcu@vger.kernel.org \
    --cc=vkuznets@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).