linux-arm-kernel.lists.infradead.org archive mirror
 help / color / mirror / Atom feed
From: Johan Hovold <johan+linaro@kernel.org>
To: Bjorn Andersson <andersson@kernel.org>
Cc: Andy Gross <agross@kernel.org>,
	Konrad Dybcio <konrad.dybcio@somainline.org>,
	Rob Herring <robh+dt@kernel.org>,
	Krzysztof Kozlowski <krzysztof.kozlowski+dt@linaro.org>,
	Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>,
	Will Deacon <will@kernel.org>,
	Robin Murphy <robin.murphy@arm.com>,
	Christoph Hellwig <hch@lst.de>, Ard Biesheuvel <ardb@kernel.org>,
	Catalin Marinas <catalin.marinas@arm.com>,
	linux-arm-msm@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org, devicetree@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	Johan Hovold <johan+linaro@kernel.org>
Subject: [PATCH] arm64: dts: qcom: sc8280xp: fix PCIe DMA coherency
Date: Thu, 24 Nov 2022 15:25:01 +0100	[thread overview]
Message-ID: <20221124142501.29314-1-johan+linaro@kernel.org> (raw)

The devices on the SC8280XP PCIe buses are cache coherent and must be
marked as such to avoid data corruption.

A coherent device can, for example, end up snooping stale data from the
caches instead of using data written by the CPU through the
non-cacheable mapping which is used for consistent DMA buffers for
non-coherent devices.

Note that this is much more likely to happen since commit c44094eee32f
("arm64: dma: Drop cache invalidation from arch_dma_prep_coherent()")
that was added in 6.1 and which removed the cache invalidation when
setting up the non-cacheable mapping.

Marking the PCIe devices as coherent specifically fixes the intermittent
NVMe probe failures observed on the Thinkpad X13s, which was due to
corruption of the submission and completion queues. This was typically
observed as corruption of the admin submission queue (with well-formed
completion):

	could not locate request for tag 0x0
	nvme nvme0: invalid id 0 completed on queue 0

or corruption of the admin or I/O completion queues (malformed
completion):

	could not locate request for tag 0x45f
	nvme nvme0: invalid id 25695 completed on queue 25965

presumably as these queues are small enough to not be allocated using
CMA which in turn make them more likely to be cached (e.g. due to
accesses to nearby pages through the cacheable linear map). Increasing
the buffer sizes to two pages to force CMA allocation also appears to
make the problem go away.

Fixes: 813e83157001 ("arm64: dts: qcom: sc8280xp/sa8540p: add PCIe2-4 nodes")
Signed-off-by: Johan Hovold <johan+linaro@kernel.org>
---
 arch/arm64/boot/dts/qcom/sc8280xp.dtsi | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/arch/arm64/boot/dts/qcom/sc8280xp.dtsi b/arch/arm64/boot/dts/qcom/sc8280xp.dtsi
index 27f5c2f82338..7748cd29276d 100644
--- a/arch/arm64/boot/dts/qcom/sc8280xp.dtsi
+++ b/arch/arm64/boot/dts/qcom/sc8280xp.dtsi
@@ -854,6 +854,8 @@ pcie4: pcie@1c00000 {
 				 <0x02000000 0x0 0x30300000 0x0 0x30300000 0x0 0x1d00000>;
 			bus-range = <0x00 0xff>;
 
+			dma-coherent;
+
 			linux,pci-domain = <6>;
 			num-lanes = <1>;
 
@@ -951,6 +953,8 @@ pcie3b: pcie@1c08000 {
 				 <0x02000000 0x0 0x32300000 0x0 0x32300000 0x0 0x1d00000>;
 			bus-range = <0x00 0xff>;
 
+			dma-coherent;
+
 			linux,pci-domain = <5>;
 			num-lanes = <2>;
 
@@ -1046,6 +1050,8 @@ pcie3a: pcie@1c10000 {
 				 <0x02000000 0x0 0x34300000 0x0 0x34300000 0x0 0x1d00000>;
 			bus-range = <0x00 0xff>;
 
+			dma-coherent;
+
 			linux,pci-domain = <4>;
 			num-lanes = <4>;
 
@@ -1144,6 +1150,8 @@ pcie2b: pcie@1c18000 {
 				 <0x02000000 0x0 0x38300000 0x0 0x38300000 0x0 0x1d00000>;
 			bus-range = <0x00 0xff>;
 
+			dma-coherent;
+
 			linux,pci-domain = <3>;
 			num-lanes = <2>;
 
@@ -1239,6 +1247,8 @@ pcie2a: pcie@1c20000 {
 				 <0x02000000 0x0 0x3c300000 0x0 0x3c300000 0x0 0x1d00000>;
 			bus-range = <0x00 0xff>;
 
+			dma-coherent;
+
 			linux,pci-domain = <2>;
 			num-lanes = <4>;
 
-- 
2.37.4


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

             reply	other threads:[~2022-11-24 14:31 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-11-24 14:25 Johan Hovold [this message]
2022-11-24 14:32 ` [PATCH] arm64: dts: qcom: sc8280xp: fix PCIe DMA coherency Konrad Dybcio
2022-11-25 14:26 ` Manivannan Sadhasivam
2022-11-25 14:43   ` Johan Hovold
2022-11-25 14:53     ` Manivannan Sadhasivam
2022-11-25 15:49       ` Johan Hovold
2022-12-02 20:58 ` Bjorn Andersson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20221124142501.29314-1-johan+linaro@kernel.org \
    --to=johan+linaro@kernel.org \
    --cc=agross@kernel.org \
    --cc=andersson@kernel.org \
    --cc=ardb@kernel.org \
    --cc=catalin.marinas@arm.com \
    --cc=devicetree@vger.kernel.org \
    --cc=hch@lst.de \
    --cc=konrad.dybcio@somainline.org \
    --cc=krzysztof.kozlowski+dt@linaro.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-arm-msm@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=manivannan.sadhasivam@linaro.org \
    --cc=robh+dt@kernel.org \
    --cc=robin.murphy@arm.com \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).