From: Baolu Lu <baolu.lu@linux.intel.com>
To: Lennert Buytenhek <buytenh@wantstofly.org>,
Bart Van Assche <bvanassche@acm.org>
Cc: baolu.lu@linux.intel.com, Sasha Levin <sashal@kernel.org>,
David Woodhouse <dwmw2@infradead.org>,
Joerg Roedel <joro@8bytes.org>,
iommu@lists.linux.dev, Will Deacon <will@kernel.org>,
Robin Murphy <robin.murphy@arm.com>,
Kevin Tian <kevin.tian@intel.com>,
Ashok Raj <ashok.raj@intel.com>,
Christoph Hellwig <hch@infradead.org>,
Jason Gunthorpe <jgg@nvidia.com>, Liu Yi L <yi.l.liu@intel.com>,
Jacob jun Pan <jacob.jun.pan@intel.com>,
linux-kernel@vger.kernel.org,
Scarlett Gourley <scarlett@arista.com>,
James Sewart <jamessewart@arista.com>,
Jack O'Sullivan <jack@arista.com>
Subject: Re: lockdep splat due to klist iteration from atomic context in Intel IOMMU driver
Date: Wed, 17 Aug 2022 12:04:10 +0800 [thread overview]
Message-ID: <94921c7a-778c-911d-4d72-d7815d95c46b@linux.intel.com> (raw)
In-Reply-To: <YvpQ5M7//AlrpJGP@wantstofly.org>
On 2022/8/15 21:57, Lennert Buytenhek wrote:
> On Mon, Aug 15, 2022 at 06:32:24AM -0700, Bart Van Assche wrote:
>
>>> On a build of 7ebfc85e2cd7 ("Merge tag 'net-6.0-rc1' of
>>> git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net"), with
>>> CONFIG_INTEL_IOMMU_DEBUGFS enabled, I am seeing the lockdep splat
>>> below when an I/O page fault occurs on a machine with an Intel
>>> IOMMU in it.
>>>
>>> The issue seems to be the klist iterator functions using
>>> spin_*lock_irq*() but the klist insertion functions using
>>> spin_*lock(), combined with the Intel DMAR IOMMU driver iterating
>>> over klists from atomic (hardirq) context as of commit 8ac0b64b9735
>>> ("iommu/vt-d: Use pci_get_domain_bus_and_slot() in pgtable_walk()")
>>> when CONFIG_INTEL_IOMMU_DEBUGFS is enabled, where
>>> pci_get_domain_bus_and_slot() calls into bus_find_device() which
>>> iterates over klists.
>>>
>>> I found this commit from 2018:
>>>
>>> commit 624fa7790f80575a4ec28fbdb2034097dc18d051
>>> Author: Bart Van Assche<bvanassche@acm.org>
>>> Date: Fri Jun 22 14:54:49 2018 -0700
>>>
>>> scsi: klist: Make it safe to use klists in atomic context
>>>
>>> This commit switched lib/klist.c:klist_{prev,next} from
>>> spin_{,un}lock() to spin_{lock_irqsave,unlock_irqrestore}(), but left
>>> the spin_{,un}lock() calls in add_{head,tail}() untouched.
>>>
>>> The simplest fix for this would be to switch lib/klist.c:add_{head,tail}()
>>> over to use the IRQ-safe spinlock variants as well?
>> Another possibility would be to evaluate whether it is safe to revert commit
>> 624fa7790f80 ("scsi: klist: Make it safe to use klists in atomic context").
>> That commit is no longer needed by the SRP transport driver since the legacy
>> block layer has been removed from the kernel.
> And then to fix the 6.0-rc1 iommu/vt-d lockdep splat with
> CONFIG_INTEL_IOMMU_DEBUGFS enabled, we could convert the Intel DMAR
> IRQ handler to a threaded IRQ handler. We (Arista) carry the patch
> below in our kernel tree, and the last two hunks of the patch do
> exactly that, for the same reason (having to call
> pci_get_domain_bus_and_slot() from the IRQ handler) but this is
> probably too big of a change for 6.0-rc.
>
>
>
> commit 90a8e7da0facf198692a641fcfe6f89c478608e0
> Author: Lennert Buytenhek<buytenh@wantstofly.org>
> Date: Wed Jul 13 15:34:30 2022 +0300
>
> iommu/vt-d: Use report_iommu_fault()
>
> This patch makes iommu/vt-d call report_iommu_fault() when an I/O
> page fault occurs, which has two effects:
>
> 1) It allows device drivers to register a callback to be notified
> of I/O page faults, via the iommu_set_fault_handler() API.
>
> 2) It triggers the io_page_fault tracepoint in report_iommu_fault()
> when an I/O page fault occurs.
>
> The latter point is the main aim of this patch, as it allows
> rasdaemon-like daemons to be notified of I/O page faults, and to
> possibly initiate corrective action in response.
The IOMMU subsystem already has a framework to handle I/O page faults:
commit fc36479db74e9 "iommu: Add a page fault handler"
And below series,
https://lore.kernel.org/linux-iommu/20220817012024.3251276-1-baolu.lu@linux.intel.com/
is trying to make it more generic. It seems to be more suitable for your
case.
The report_iommu_fault() probably will be replaced by
iommu_register_device_fault_handler() eventually. So I don't encourage
its usage in the VT-d driver.
Best regards,
baolu
next prev parent reply other threads:[~2022-08-17 4:04 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-08-15 12:05 lockdep splat due to klist iteration from atomic context in Intel IOMMU driver Lennert Buytenhek
2022-08-15 13:32 ` Bart Van Assche
2022-08-15 13:57 ` Lennert Buytenhek
2022-08-17 4:04 ` Baolu Lu [this message]
2022-08-17 6:09 ` Lennert Buytenhek
2022-08-17 7:20 ` Baolu Lu
2022-08-17 4:45 ` Baolu Lu
2022-08-17 5:49 ` Lennert Buytenhek
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=94921c7a-778c-911d-4d72-d7815d95c46b@linux.intel.com \
--to=baolu.lu@linux.intel.com \
--cc=ashok.raj@intel.com \
--cc=buytenh@wantstofly.org \
--cc=bvanassche@acm.org \
--cc=dwmw2@infradead.org \
--cc=hch@infradead.org \
--cc=iommu@lists.linux.dev \
--cc=jack@arista.com \
--cc=jacob.jun.pan@intel.com \
--cc=jamessewart@arista.com \
--cc=jgg@nvidia.com \
--cc=joro@8bytes.org \
--cc=kevin.tian@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=robin.murphy@arm.com \
--cc=sashal@kernel.org \
--cc=scarlett@arista.com \
--cc=will@kernel.org \
--cc=yi.l.liu@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).