From: Sinan Kaya <okaya@codeaurora.org>
To: Lorenzo Pieralisi <Lorenzo.Pieralisi@arm.com>,
Will Deacon <will.deacon@arm.com>,
Linux PCI <linux-pci@vger.kernel.org>,
linux-arm Mailing List <linux-arm-kernel@lists.infradead.org>,
Nate Watterson <nwatters@codeaurora.org>,
"shankerd@codeaurora.org" <shankerd@codeaurora.org>,
Vikram Sethi <vikrams@codeaurora.org>,
"Goel, Sameer" <sgoel@codeaurora.org>,
kexec@lists.infradead.org
Subject: RFC on Kdump and PCIe on ARM64
Date: Thu, 1 Mar 2018 12:44:26 -0500 [thread overview]
Message-ID: <c5c2d8d2-0384-442a-7bc6-8929457b8447@codeaurora.org> (raw)
Hi,
We are seeing IOMMU faults when booting the kdump kernel on ARM64.
[ 7.220162] arm-smmu-v3 arm-smmu-v3.0.auto: event 0x02 received:
[ 7.226123] arm-smmu-v3 arm-smmu-v3.0.auto: 0x0000010000000002
[ 7.232023] arm-smmu-v3 arm-smmu-v3.0.auto: 0x0000000000000000
[ 7.237925] arm-smmu-v3 arm-smmu-v3.0.auto: 0x0000000000000000
[ 7.243827] arm-smmu-v3 arm-smmu-v3.0.auto: 0x0000000000000000
This is Nate's interpretation of the fault:
"The PCI device is sending transactions just after the SMMU was reset/reinitialized
which is problematic because the device has not yet been added to the SMMU
and thus should not be doing *any* DMA. DMA from the PCI devices should be
quiesced prior to starting the crashdump kernel or you risk overwriting portions
of memory you meant to preserve. In this case the SMMU was actually doing you a favor
by blocking these errant DMA operations!!"
I think this makes sense especially for the IOMMU enabled case on the host where an
IOVA can overlap with the region of memory kdump reserved for itself.
Apparently, there has been similar concerns in the past.
https://www.fujitsu.com/jp/documents/products/software/os/linux/catalog/LinuxConJapan2013-Indoh.pdf
and was not addressed globally due to IOMMU+PCI driver ordering issues and bugs in
HW due to hot reset.
https://lkml.org/lkml/2012/8/3/160
Hot reset as mentioned is destructive and may not be the best implementation choice.
However, most of the modern endpoints support PCIE function level reset.
One other solution is for SMMUv3 driver to reserve the kdump used IOVA addresses.
Another solution is for the SMMUv3 driver to disable PCIe devices behind the SMMU
if it see SMMU is already enabled.
Appreciate the feedback,
Sinan
--
Sinan Kaya
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
next reply other threads:[~2018-03-01 17:44 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-03-01 17:44 Sinan Kaya [this message]
2018-03-01 19:05 ` RFC on Kdump and PCIe on ARM64 Bjorn Helgaas
2018-03-01 19:19 ` Sinan Kaya
2018-03-02 0:03 ` Bjorn Helgaas
2018-03-02 10:30 ` Will Deacon
2018-03-02 14:12 ` Sinan Kaya
2018-03-02 18:02 ` Will Deacon
2018-03-02 18:37 ` Sinan Kaya
2018-03-02 14:20 ` Sinan Kaya
2018-03-02 1:44 ` Baoquan He
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c5c2d8d2-0384-442a-7bc6-8929457b8447@codeaurora.org \
--to=okaya@codeaurora.org \
--cc=Lorenzo.Pieralisi@arm.com \
--cc=kexec@lists.infradead.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-pci@vger.kernel.org \
--cc=nwatters@codeaurora.org \
--cc=sgoel@codeaurora.org \
--cc=shankerd@codeaurora.org \
--cc=vikrams@codeaurora.org \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).