archive mirror
 help / color / mirror / Atom feed
From: Ard Biesheuvel <>
Cc:, Ard Biesheuvel <>,,,,
Subject: [PATCH v3] nvme: retain split access workaround for capability reads
Date: Mon,  7 Oct 2019 13:42:53 +0200	[thread overview]
Message-ID: <> (raw)

Commit 7fd8930f26be4

  "nvme: add a common helper to read Identify Controller data"

has re-introduced an issue that we have attempted to work around in the
past, in commit a310acd7a7ea ("NVMe: use split lo_hi_{read,write}q").

The problem is that some PCIe NVMe controllers do not implement 64-bit
outbound accesses correctly, which is why the commit above switched
to using lo_hi_[read|write]q for all 64-bit BAR accesses occuring in
the code.

In the mean time, the NVMe subsystem has been refactored, and now calls
into the PCIe support layer for NVMe via a .reg_read64() method, which
fails to use lo_hi_readq(), and thus reintroduces the problem that the
workaround above aimed to address.

Given that, at the moment, .reg_read64() is only used to read the
capability register [which is known to tolerate split reads], let's
switch .reg_read64() to lo_hi_readq() as well.

This fixes a boot issue on some ARM boxes with NVMe behind a Synopsys
DesignWare PCIe host controller.

Fixes: 7fd8930f26be4 ("nvme: add a common helper to read Identify Controller data")
Signed-off-by: Ard Biesheuvel <>
v3: add comment explaining why we need the split read
v2: drop WARN_ON(), add reference to commit that [re-]introduced the issue

 drivers/nvme/host/pci.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index bb88681f4dc3..f908acc5d2ef 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2672,7 +2672,8 @@ static int nvme_pci_reg_write32(struct nvme_ctrl *ctrl, u32 off, u32 val)
 static int nvme_pci_reg_read64(struct nvme_ctrl *ctrl, u32 off, u64 *val)
-	*val = readq(to_nvme_dev(ctrl)->bar + off);
+	/* use a split read to work around buggy interconnects */
+	*val = lo_hi_readq(to_nvme_dev(ctrl)->bar + off);
 	return 0;

Linux-nvme mailing list

             reply	other threads:[~2019-10-07 11:43 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-07 11:42 Ard Biesheuvel [this message]
2019-10-07 12:07 ` [PATCH v3] nvme: retain split access workaround for capability reads Christoph Hellwig
2019-10-07 12:24   ` Ard Biesheuvel
2019-10-07 12:27     ` Christoph Hellwig
2019-10-07 12:32       ` Ard Biesheuvel
2019-10-07 12:47         ` Christoph Hellwig
2019-10-07 12:48         ` Keith Busch
2019-10-07 13:20           ` Ard Biesheuvel
2019-10-07 13:32             ` Keith Busch
2019-10-07 13:33               ` Ard Biesheuvel

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).