linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Sven Peter <sven@svenpeter.dev>
To: Keith Busch <kbusch@kernel.org>, Jens Axboe <axboe@fb.com>,
	Christoph Hellwig <hch@lst.de>, Sagi Grimberg <sagi@grimberg.me>
Cc: Sven Peter <sven@svenpeter.dev>, Hector Martin <marcan@marcan.st>,
	Alyssa Rosenzweig <alyssa@rosenzweig.io>,
	Rob Herring <robh+dt@kernel.org>, Arnd Bergmann <arnd@arndb.de>,
	Marc Zyngier <maz@kernel.org>,
	devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
	linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org,
	Jens Axboe <axboe@kernel.dk>
Subject: [PATCH 7/9] nvme-apple: Serialize command issue
Date: Mon, 21 Mar 2022 17:50:47 +0100	[thread overview]
Message-ID: <20220321165049.35985-8-sven@svenpeter.dev> (raw)
In-Reply-To: <20220321165049.35985-1-sven@svenpeter.dev>

From: Jens Axboe <axboe@kernel.dk>

This controller shouldn't need serialization of command issue since
the SQ is replaced by a simple array and commands are issued by writing
the array index to a MMIO register.
Without serialization however sometimes commands are still executed
correctly and appear in the CQ but never trigger an interrupt.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
[sven: added our best guess why this needs to be done]
Signed-off-by: Sven Peter <sven@svenpeter.dev>
---
 drivers/nvme/host/apple.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/nvme/host/apple.c b/drivers/nvme/host/apple.c
index 587d6c7014a0..a4193429564e 100644
--- a/drivers/nvme/host/apple.c
+++ b/drivers/nvme/host/apple.c
@@ -292,6 +292,7 @@ static void apple_nvmmu_inval(struct apple_nvme_queue *q, unsigned int tag)
 static void apple_nvme_submit_cmd(struct apple_nvme_queue *q,
 				  struct nvme_command *cmd)
 {
+	struct apple_nvme *anv = queue_to_apple_nvme(q);
 	u32 tag = nvme_tag_from_cid(cmd->common.command_id);
 	struct apple_nvmmu_tcb *tcb = &q->tcbs[tag];
 
@@ -308,7 +309,18 @@ static void apple_nvme_submit_cmd(struct apple_nvme_queue *q,
 		tcb->dma_flags |= APPLE_ANS_TCB_DMA_FROM_DEVICE;
 
 	memcpy(&q->sqes[tag], cmd, sizeof(*cmd));
+
+	/*
+	 * This lock here doesn't make much sense at a first glace but
+	 * removing it will result in occasional missed completetion
+	 * interrupts even though the commands still appear on the CQ.
+	 * It's unclear why this happens but our best guess is that
+	 * there is a bug in the firmware triggered when a write to the
+	 * CQ and the SQ happen simultaneously.
+	 */
+	spin_lock_irq(&anv->lock);
 	writel(tag, q->sq_db);
+	spin_unlock_irq(&anv->lock);
 }
 
 /*
-- 
2.25.1


  parent reply	other threads:[~2022-03-21 16:52 UTC|newest]

Thread overview: 55+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-03-21 16:50 [PATCH 0/9] Apple M1 (Pro/Max) NVMe driver Sven Peter
2022-03-21 16:50 ` [PATCH 1/9] dt-bindings: soc: apple: Add Apple SART Sven Peter
2022-03-31 21:23   ` Rob Herring
2022-04-02 12:58     ` Sven Peter
2022-03-21 16:50 ` [PATCH 2/9] dt-bindings: soc: apple: Add ANS NVMe Sven Peter
2022-03-23 11:14   ` Krzysztof Kozlowski
2022-04-02 13:05     ` Sven Peter
2022-04-02 16:06       ` Krzysztof Kozlowski
2022-03-21 16:50 ` [PATCH 3/9] soc: apple: Always include Makefile Sven Peter
2022-03-21 16:50 ` [PATCH 4/9] soc: apple: Add SART driver Sven Peter
2022-03-21 17:07   ` Arnd Bergmann
2022-04-02 12:38     ` Sven Peter
2022-04-02 19:07       ` Arnd Bergmann
2022-04-04 14:58         ` Rob Herring
2022-04-04 15:01           ` Hector Martin
2022-04-05 15:37           ` Sven Peter
2022-03-21 17:17   ` Alyssa Rosenzweig
2022-04-02 12:40     ` Sven Peter
2022-03-21 16:50 ` [PATCH 5/9] soc: apple: Add RTKit IPC library Sven Peter
2022-03-22 13:13   ` Arnd Bergmann
2022-03-22 17:41     ` Robin Murphy
2022-04-02 12:56     ` Sven Peter
2022-04-02 18:30       ` Arnd Bergmann
2022-04-03 10:45         ` Sven Peter
2022-03-22 17:23   ` Alyssa Rosenzweig
2022-04-02 12:50     ` Sven Peter
2022-03-23 11:19   ` Krzysztof Kozlowski
2022-04-02 13:51     ` Sven Peter
2022-04-02 16:08       ` Krzysztof Kozlowski
2022-04-04 15:02       ` Rob Herring
2022-04-04 15:47         ` Hector Martin
2022-03-21 16:50 ` [PATCH 6/9] nvme-apple: Add initial Apple SoC NVMe driver Sven Peter
2022-03-21 17:01   ` Keith Busch
2022-04-02 13:10     ` Sven Peter
2022-03-22 13:38   ` Arnd Bergmann
2022-04-02 13:34     ` Sven Peter
2022-04-02 13:58       ` Janne Grunau
2022-04-02 14:02         ` Sven Peter
2022-04-02 21:26       ` Arnd Bergmann
2022-03-24  6:16   ` Christoph Hellwig
2022-04-02 12:47     ` Sven Peter
2022-04-04 15:57     ` Hector Martin
2022-04-04 15:59       ` Christoph Hellwig
2022-04-04 16:03         ` Sven Peter
2022-04-04 16:05           ` hch
2022-04-04 16:05         ` Hector Martin
2022-04-04 18:29         ` Jens Axboe
2022-04-05  6:24           ` Christoph Hellwig
2022-03-21 16:50 ` Sven Peter [this message]
2022-03-24  6:16   ` [PATCH 7/9] nvme-apple: Serialize command issue Christoph Hellwig
2022-04-02 12:42     ` Sven Peter
2022-03-21 16:50 ` [PATCH 8/9] nvme-apple: Add support for multiple power domains Sven Peter
2022-03-21 16:50 ` [PATCH 9/9] nvme-apple: Add support for suspend/resume Sven Peter
2022-03-24  6:17   ` Christoph Hellwig
2022-03-22 17:26 ` [PATCH 0/9] Apple M1 (Pro/Max) NVMe driver Alyssa Rosenzweig

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220321165049.35985-8-sven@svenpeter.dev \
    --to=sven@svenpeter.dev \
    --cc=alyssa@rosenzweig.io \
    --cc=arnd@arndb.de \
    --cc=axboe@fb.com \
    --cc=axboe@kernel.dk \
    --cc=devicetree@vger.kernel.org \
    --cc=hch@lst.de \
    --cc=kbusch@kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nvme@lists.infradead.org \
    --cc=marcan@marcan.st \
    --cc=maz@kernel.org \
    --cc=robh+dt@kernel.org \
    --cc=sagi@grimberg.me \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).