From: Sven Peter <sven@svenpeter.dev> To: Keith Busch <kbusch@kernel.org>, Jens Axboe <axboe@fb.com>, Christoph Hellwig <hch@lst.de>, Sagi Grimberg <sagi@grimberg.me> Cc: Sven Peter <sven@svenpeter.dev>, Hector Martin <marcan@marcan.st>, Alyssa Rosenzweig <alyssa@rosenzweig.io>, Rob Herring <robh+dt@kernel.org>, Arnd Bergmann <arnd@arndb.de>, Marc Zyngier <maz@kernel.org>, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, Jens Axboe <axboe@kernel.dk> Subject: [PATCH 7/9] nvme-apple: Serialize command issue Date: Mon, 21 Mar 2022 17:50:47 +0100 [thread overview] Message-ID: <20220321165049.35985-8-sven@svenpeter.dev> (raw) In-Reply-To: <20220321165049.35985-1-sven@svenpeter.dev> From: Jens Axboe <axboe@kernel.dk> This controller shouldn't need serialization of command issue since the SQ is replaced by a simple array and commands are issued by writing the array index to a MMIO register. Without serialization however sometimes commands are still executed correctly and appear in the CQ but never trigger an interrupt. Signed-off-by: Jens Axboe <axboe@kernel.dk> [sven: added our best guess why this needs to be done] Signed-off-by: Sven Peter <sven@svenpeter.dev> --- drivers/nvme/host/apple.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/drivers/nvme/host/apple.c b/drivers/nvme/host/apple.c index 587d6c7014a0..a4193429564e 100644 --- a/drivers/nvme/host/apple.c +++ b/drivers/nvme/host/apple.c @@ -292,6 +292,7 @@ static void apple_nvmmu_inval(struct apple_nvme_queue *q, unsigned int tag) static void apple_nvme_submit_cmd(struct apple_nvme_queue *q, struct nvme_command *cmd) { + struct apple_nvme *anv = queue_to_apple_nvme(q); u32 tag = nvme_tag_from_cid(cmd->common.command_id); struct apple_nvmmu_tcb *tcb = &q->tcbs[tag]; @@ -308,7 +309,18 @@ static void apple_nvme_submit_cmd(struct apple_nvme_queue *q, tcb->dma_flags |= APPLE_ANS_TCB_DMA_FROM_DEVICE; memcpy(&q->sqes[tag], cmd, sizeof(*cmd)); + + /* + * This lock here doesn't make much sense at a first glace but + * removing it will result in occasional missed completetion + * interrupts even though the commands still appear on the CQ. + * It's unclear why this happens but our best guess is that + * there is a bug in the firmware triggered when a write to the + * CQ and the SQ happen simultaneously. + */ + spin_lock_irq(&anv->lock); writel(tag, q->sq_db); + spin_unlock_irq(&anv->lock); } /* -- 2.25.1
WARNING: multiple messages have this Message-ID (diff)
From: Sven Peter <sven@svenpeter.dev> To: Keith Busch <kbusch@kernel.org>, Jens Axboe <axboe@fb.com>, Christoph Hellwig <hch@lst.de>, Sagi Grimberg <sagi@grimberg.me> Cc: Sven Peter <sven@svenpeter.dev>, Hector Martin <marcan@marcan.st>, Alyssa Rosenzweig <alyssa@rosenzweig.io>, Rob Herring <robh+dt@kernel.org>, Arnd Bergmann <arnd@arndb.de>, Marc Zyngier <maz@kernel.org>, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, Jens Axboe <axboe@kernel.dk> Subject: [PATCH 7/9] nvme-apple: Serialize command issue Date: Mon, 21 Mar 2022 17:50:47 +0100 [thread overview] Message-ID: <20220321165049.35985-8-sven@svenpeter.dev> (raw) In-Reply-To: <20220321165049.35985-1-sven@svenpeter.dev> From: Jens Axboe <axboe@kernel.dk> This controller shouldn't need serialization of command issue since the SQ is replaced by a simple array and commands are issued by writing the array index to a MMIO register. Without serialization however sometimes commands are still executed correctly and appear in the CQ but never trigger an interrupt. Signed-off-by: Jens Axboe <axboe@kernel.dk> [sven: added our best guess why this needs to be done] Signed-off-by: Sven Peter <sven@svenpeter.dev> --- drivers/nvme/host/apple.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/drivers/nvme/host/apple.c b/drivers/nvme/host/apple.c index 587d6c7014a0..a4193429564e 100644 --- a/drivers/nvme/host/apple.c +++ b/drivers/nvme/host/apple.c @@ -292,6 +292,7 @@ static void apple_nvmmu_inval(struct apple_nvme_queue *q, unsigned int tag) static void apple_nvme_submit_cmd(struct apple_nvme_queue *q, struct nvme_command *cmd) { + struct apple_nvme *anv = queue_to_apple_nvme(q); u32 tag = nvme_tag_from_cid(cmd->common.command_id); struct apple_nvmmu_tcb *tcb = &q->tcbs[tag]; @@ -308,7 +309,18 @@ static void apple_nvme_submit_cmd(struct apple_nvme_queue *q, tcb->dma_flags |= APPLE_ANS_TCB_DMA_FROM_DEVICE; memcpy(&q->sqes[tag], cmd, sizeof(*cmd)); + + /* + * This lock here doesn't make much sense at a first glace but + * removing it will result in occasional missed completetion + * interrupts even though the commands still appear on the CQ. + * It's unclear why this happens but our best guess is that + * there is a bug in the firmware triggered when a write to the + * CQ and the SQ happen simultaneously. + */ + spin_lock_irq(&anv->lock); writel(tag, q->sq_db); + spin_unlock_irq(&anv->lock); } /* -- 2.25.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2022-03-21 16:52 UTC|newest] Thread overview: 114+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-03-21 16:50 [PATCH 0/9] Apple M1 (Pro/Max) NVMe driver Sven Peter 2022-03-21 16:50 ` Sven Peter 2022-03-21 16:50 ` Sven Peter 2022-03-21 16:50 ` [PATCH 1/9] dt-bindings: soc: apple: Add Apple SART Sven Peter 2022-03-21 16:50 ` Sven Peter 2022-03-31 21:23 ` Rob Herring 2022-03-31 21:23 ` Rob Herring 2022-04-02 12:58 ` Sven Peter 2022-04-02 12:58 ` Sven Peter 2022-03-21 16:50 ` [PATCH 2/9] dt-bindings: soc: apple: Add ANS NVMe Sven Peter 2022-03-21 16:50 ` Sven Peter 2022-03-23 11:14 ` Krzysztof Kozlowski 2022-03-23 11:14 ` Krzysztof Kozlowski 2022-04-02 13:05 ` Sven Peter 2022-04-02 13:05 ` Sven Peter 2022-04-02 16:06 ` Krzysztof Kozlowski 2022-04-02 16:06 ` Krzysztof Kozlowski 2022-03-21 16:50 ` [PATCH 3/9] soc: apple: Always include Makefile Sven Peter 2022-03-21 16:50 ` Sven Peter 2022-03-21 16:50 ` Sven Peter 2022-03-21 16:50 ` [PATCH 4/9] soc: apple: Add SART driver Sven Peter 2022-03-21 16:50 ` Sven Peter 2022-03-21 16:50 ` Sven Peter 2022-03-21 17:07 ` Arnd Bergmann 2022-03-21 17:07 ` Arnd Bergmann 2022-04-02 12:38 ` Sven Peter 2022-04-02 12:38 ` Sven Peter 2022-04-02 19:07 ` Arnd Bergmann 2022-04-02 19:07 ` Arnd Bergmann 2022-04-04 14:58 ` Rob Herring 2022-04-04 14:58 ` Rob Herring 2022-04-04 15:01 ` Hector Martin 2022-04-04 15:01 ` Hector Martin 2022-04-05 15:37 ` Sven Peter 2022-04-05 15:37 ` Sven Peter 2022-03-21 17:17 ` Alyssa Rosenzweig 2022-03-21 17:17 ` Alyssa Rosenzweig 2022-04-02 12:40 ` Sven Peter 2022-04-02 12:40 ` Sven Peter 2022-03-21 16:50 ` [PATCH 5/9] soc: apple: Add RTKit IPC library Sven Peter 2022-03-21 16:50 ` Sven Peter 2022-03-21 16:50 ` Sven Peter 2022-03-22 13:13 ` Arnd Bergmann 2022-03-22 13:13 ` Arnd Bergmann 2022-03-22 17:41 ` Robin Murphy 2022-03-22 17:41 ` Robin Murphy 2022-04-02 12:56 ` Sven Peter 2022-04-02 12:56 ` Sven Peter 2022-04-02 18:30 ` Arnd Bergmann 2022-04-02 18:30 ` Arnd Bergmann 2022-04-03 10:45 ` Sven Peter 2022-04-03 10:45 ` Sven Peter 2022-03-22 17:23 ` Alyssa Rosenzweig 2022-03-22 17:23 ` Alyssa Rosenzweig 2022-04-02 12:50 ` Sven Peter 2022-04-02 12:50 ` Sven Peter 2022-03-23 11:19 ` Krzysztof Kozlowski 2022-03-23 11:19 ` Krzysztof Kozlowski 2022-04-02 13:51 ` Sven Peter 2022-04-02 13:51 ` Sven Peter 2022-04-02 16:08 ` Krzysztof Kozlowski 2022-04-02 16:08 ` Krzysztof Kozlowski 2022-04-04 15:02 ` Rob Herring 2022-04-04 15:02 ` Rob Herring 2022-04-04 15:47 ` Hector Martin 2022-04-04 15:47 ` Hector Martin 2022-03-21 16:50 ` [PATCH 6/9] nvme-apple: Add initial Apple SoC NVMe driver Sven Peter 2022-03-21 16:50 ` Sven Peter 2022-03-21 17:01 ` Keith Busch 2022-03-21 17:01 ` Keith Busch 2022-04-02 13:10 ` Sven Peter 2022-04-02 13:10 ` Sven Peter 2022-03-22 13:38 ` Arnd Bergmann 2022-03-22 13:38 ` Arnd Bergmann 2022-04-02 13:34 ` Sven Peter 2022-04-02 13:34 ` Sven Peter 2022-04-02 13:58 ` Janne Grunau 2022-04-02 13:58 ` Janne Grunau 2022-04-02 14:02 ` Sven Peter 2022-04-02 14:02 ` Sven Peter 2022-04-02 21:26 ` Arnd Bergmann 2022-04-02 21:26 ` Arnd Bergmann 2022-03-24 6:16 ` Christoph Hellwig 2022-03-24 6:16 ` Christoph Hellwig 2022-04-02 12:47 ` Sven Peter 2022-04-02 12:47 ` Sven Peter 2022-04-04 15:57 ` Hector Martin 2022-04-04 15:57 ` Hector Martin 2022-04-04 15:59 ` Christoph Hellwig 2022-04-04 15:59 ` Christoph Hellwig 2022-04-04 16:03 ` Sven Peter 2022-04-04 16:03 ` Sven Peter 2022-04-04 16:05 ` hch 2022-04-04 16:05 ` hch 2022-04-04 16:05 ` Hector Martin 2022-04-04 16:05 ` Hector Martin 2022-04-04 18:29 ` Jens Axboe 2022-04-04 18:29 ` Jens Axboe 2022-04-05 6:24 ` Christoph Hellwig 2022-04-05 6:24 ` Christoph Hellwig 2022-03-21 16:50 ` Sven Peter [this message] 2022-03-21 16:50 ` [PATCH 7/9] nvme-apple: Serialize command issue Sven Peter 2022-03-24 6:16 ` Christoph Hellwig 2022-03-24 6:16 ` Christoph Hellwig 2022-04-02 12:42 ` Sven Peter 2022-04-02 12:42 ` Sven Peter 2022-03-21 16:50 ` [PATCH 8/9] nvme-apple: Add support for multiple power domains Sven Peter 2022-03-21 16:50 ` Sven Peter 2022-03-21 16:50 ` [PATCH 9/9] nvme-apple: Add support for suspend/resume Sven Peter 2022-03-21 16:50 ` Sven Peter 2022-03-24 6:17 ` Christoph Hellwig 2022-03-24 6:17 ` Christoph Hellwig 2022-03-22 17:26 ` [PATCH 0/9] Apple M1 (Pro/Max) NVMe driver Alyssa Rosenzweig 2022-03-22 17:26 ` Alyssa Rosenzweig
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220321165049.35985-8-sven@svenpeter.dev \ --to=sven@svenpeter.dev \ --cc=alyssa@rosenzweig.io \ --cc=arnd@arndb.de \ --cc=axboe@fb.com \ --cc=axboe@kernel.dk \ --cc=devicetree@vger.kernel.org \ --cc=hch@lst.de \ --cc=kbusch@kernel.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-nvme@lists.infradead.org \ --cc=marcan@marcan.st \ --cc=maz@kernel.org \ --cc=robh+dt@kernel.org \ --cc=sagi@grimberg.me \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.