From: John Garry <john.garry@huawei.com>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Question on threaded handlers for managed interrupts
Date: Thu, 22 Apr 2021 17:10:49 +0100 [thread overview]
Message-ID: <b8c4be8c-1d67-c16c-570e-d3c883c77ea2@huawei.com> (raw)
Hi Thomas,
I am finding that I can pretty easily trigger a system hang for certain
scenarios with my storage controller.
So I'm getting something like this when running moderately heavy data
throughput:
Starting 6 processes
[70.656622] sched: RT throttling activatedB/s][r=356k,w=0 IOPS][eta
01h:14m:43s]
[ 207.632161] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:ta
01h:12m:26s]
[ 207.638261] rcu: 0-...!: (1 GPs behind)
idle=312/1/0x4000000000000000 softirq=508/512 fqs=0
[ 207.646777] rcu: 1-...!: (1 GPs behind) idle=694/0/0x0
It ends pretty badly - see [0].
The multi-queue storage controller (see [1] for memory refresh, but note
that I can also trigger on PCI device host controller as well) is using
managed interrupts and threaded handlers. Since the threaded handler
uses SCHED_FIFO, aren't we always vulnerable to this situation with the
managed interrupt and threaded handler combo? Would the advice be to
just use irq polling here?
I unsuccessfully tried to trigger the same on NVMe PCI - however I have
only 1x card, so hardly overloading the system.
Thanks,
John
[0]
https://lore.kernel.org/rcu/412926e8-d3e1-3071-8cb9-098a7f49b64c@huawei.com/T/#mbd60463c543e04f87090d89301e1a5f10de958dd
[1]
https://lore.kernel.org/linux-scsi/1606905417-183214-1-git-send-email-john.garry@huawei.com/#t
next reply other threads:[~2021-04-22 16:13 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-22 16:10 John Garry [this message]
2021-04-23 10:50 ` Question on threaded handlers for managed interrupts Thomas Gleixner
2021-04-23 12:02 ` John Garry
2021-04-23 13:01 ` Thomas Gleixner
2021-04-23 15:00 ` John Garry
2021-05-21 12:46 ` John Garry
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b8c4be8c-1d67-c16c-570e-d3c883c77ea2@huawei.com \
--to=john.garry@huawei.com \
--cc=linux-kernel@vger.kernel.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.