From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C7E5EC433EF for ; Fri, 14 Jan 2022 11:06:36 +0000 (UTC) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id E958883510; Fri, 14 Jan 2022 12:05:45 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=none (p=none dis=none) header.from=openbsd.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; secure) header.d=kpnmail.nl header.i=@kpnmail.nl header.b="hcK+JdfG"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 5B55A8340A; Fri, 14 Jan 2022 12:05:24 +0100 (CET) Received: from ewsoutbound.kpnmail.nl (ewsoutbound.kpnmail.nl [195.121.94.183]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 85D7E83481 for ; Fri, 14 Jan 2022 12:05:15 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=none (p=none dis=none) header.from=openbsd.org Authentication-Results: phobos.denx.de; spf=fail smtp.mailfrom=kettenis@openbsd.org X-KPN-MessageId: 238a6202-7529-11ec-a02e-005056992ed3 Received: from smtp.kpnmail.nl (unknown [10.31.155.8]) by ewsoutbound.so.kpn.org (Halon) with ESMTPS id 238a6202-7529-11ec-a02e-005056992ed3; Fri, 14 Jan 2022 12:00:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kpnmail.nl; s=kpnmail01; h=mime-version:message-id:date:subject:to:from; bh=YK8I87YsMxfomiwqbqydOu4kCzzZ+clbIFzdwaA3wiE=; b=hcK+JdfGaiksWhqAw7lHfvqRuF73vHBMnITGwHAOZCjhFO7yWLc3xCBhn7lnE2gfUqi9FRWu4vysT Bd9WYbPbHf/XWtwEmip7GY5LfbA3LJt3+r/99ZxiaKiGgmQJDQewvJUCIBB9abrULkvY84KhFdwzy6 ohq+Mmkgaui0gpgo= X-KPN-VerifiedSender: No X-CMASSUN: 33|uikb721PAgNne2gslm+tFnonSepfmJ1H7PjMESnU3oWU9yyFbTy7VRE0hdSed8D HbBNXb6mnP/eAaQx4oDTblQ== X-Originating-IP: 83.163.83.176 Received: from copland.sibelius.xs4all.nl (sibelius.xs4all.nl [83.163.83.176]) by smtp.xs4all.nl (Halon) with ESMTPSA id eb8c41a9-7529-11ec-a83c-00505699d6e5; Fri, 14 Jan 2022 12:05:44 +0100 (CET) From: Mark Kettenis To: u-boot@lists.denx.de Cc: sjg@chromium.org, jh80.chung@samsung.com, trini@konsulko.com, sven@svenpeter.dev, marcan@marcan.st, bmeng.cn@gmail.com, Mark Kettenis Subject: [PATCH 4/8] nvme: Introduce driver ops Date: Fri, 14 Jan 2022 12:04:34 +0100 Message-Id: <20220114110438.58452-5-kettenis@openbsd.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220114110438.58452-1-kettenis@openbsd.org> References: <20220114110438.58452-1-kettenis@openbsd.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.2 at phobos.denx.de X-Virus-Status: Clean The NVMe storage controller integrated on Apple SoCs deviates from the NVMe standard in two aspects. It uses a "linear" submission queue and it integrates an NVMMU that needs to be programmed for each NVMe command. Introduce driver ops such that we can set up the linear submission queue and program the NVMMU in the driver for this strange beast. Signed-off-by: Mark Kettenis --- drivers/nvme/nvme.c | 45 ++++++++++++++++++--------------------------- drivers/nvme/nvme.h | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 51 insertions(+), 27 deletions(-) diff --git a/drivers/nvme/nvme.c b/drivers/nvme/nvme.c index be518ec20b..e2d0f9c668 100644 --- a/drivers/nvme/nvme.c +++ b/drivers/nvme/nvme.c @@ -27,33 +27,6 @@ #define IO_TIMEOUT 30 #define MAX_PRP_POOL 512 -enum nvme_queue_id { - NVME_ADMIN_Q, - NVME_IO_Q, - NVME_Q_NUM, -}; - -/* - * An NVM Express queue. Each device has at least two (one for admin - * commands and one for I/O commands). - */ -struct nvme_queue { - struct nvme_dev *dev; - struct nvme_command *sq_cmds; - struct nvme_completion *cqes; - wait_queue_head_t sq_full; - u32 __iomem *q_db; - u16 q_depth; - s16 cq_vector; - u16 sq_head; - u16 sq_tail; - u16 cq_head; - u16 qid; - u8 cq_phase; - u8 cqe_seen; - unsigned long cmdid_data[]; -}; - static int nvme_wait_ready(struct nvme_dev *dev, bool enabled) { u32 bit = enabled ? NVME_CSTS_RDY : 0; @@ -167,12 +140,19 @@ static u16 nvme_read_completion_status(struct nvme_queue *nvmeq, u16 index) */ static void nvme_submit_cmd(struct nvme_queue *nvmeq, struct nvme_command *cmd) { + struct nvme_ops *ops; u16 tail = nvmeq->sq_tail; memcpy(&nvmeq->sq_cmds[tail], cmd, sizeof(*cmd)); flush_dcache_range((ulong)&nvmeq->sq_cmds[tail], (ulong)&nvmeq->sq_cmds[tail] + sizeof(*cmd)); + ops = (struct nvme_ops *)nvmeq->dev->udev->driver->ops; + if (ops && ops->submit_cmd) { + ops->submit_cmd(nvmeq, cmd); + return; + } + if (++tail == nvmeq->q_depth) tail = 0; writel(tail, nvmeq->q_db); @@ -183,6 +163,7 @@ static int nvme_submit_sync_cmd(struct nvme_queue *nvmeq, struct nvme_command *cmd, u32 *result, unsigned timeout) { + struct nvme_ops *ops; u16 head = nvmeq->cq_head; u16 phase = nvmeq->cq_phase; u16 status; @@ -203,6 +184,10 @@ static int nvme_submit_sync_cmd(struct nvme_queue *nvmeq, return -ETIMEDOUT; } + ops = (struct nvme_ops *)nvmeq->dev->udev->driver->ops; + if (ops && ops->complete_cmd) + ops->complete_cmd(nvmeq, cmd); + status >>= 1; if (status) { printf("ERROR: status = %x, phase = %d, head = %d\n", @@ -243,6 +228,7 @@ static int nvme_submit_admin_cmd(struct nvme_dev *dev, struct nvme_command *cmd, static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev, int qid, int depth) { + struct nvme_ops *ops; struct nvme_queue *nvmeq = malloc(sizeof(*nvmeq)); if (!nvmeq) return NULL; @@ -268,6 +254,10 @@ static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev, dev->queue_count++; dev->queues[qid] = nvmeq; + ops = (struct nvme_ops *)dev->udev->driver->ops; + if (ops && ops->alloc_queue) + ops->alloc_queue(nvmeq); + return nvmeq; free_queue: @@ -821,6 +811,7 @@ int nvme_init(struct udevice *udev) struct nvme_id_ns *id; int ret; + ndev->udev = udev; INIT_LIST_HEAD(&ndev->namespaces); if (readl(&ndev->bar->csts) == -1) { ret = -ENODEV; diff --git a/drivers/nvme/nvme.h b/drivers/nvme/nvme.h index 8e9ae3c7f6..57803b43fd 100644 --- a/drivers/nvme/nvme.h +++ b/drivers/nvme/nvme.h @@ -596,6 +596,7 @@ enum { /* Represents an NVM Express device. Each nvme_dev is a PCI function. */ struct nvme_dev { + struct udevice *udev; struct list_head node; struct nvme_queue **queues; u32 __iomem *dbs; @@ -622,6 +623,32 @@ struct nvme_dev { u32 nn; }; +enum nvme_queue_id { + NVME_ADMIN_Q, + NVME_IO_Q, + NVME_Q_NUM, +}; + +/* + * An NVM Express queue. Each device has at least two (one for admin + * commands and one for I/O commands). + */ +struct nvme_queue { + struct nvme_dev *dev; + struct nvme_command *sq_cmds; + struct nvme_completion *cqes; + u32 __iomem *q_db; + u16 q_depth; + s16 cq_vector; + u16 sq_head; + u16 sq_tail; + u16 cq_head; + u16 qid; + u8 cq_phase; + u8 cqe_seen; + unsigned long cmdid_data[]; +}; + /* * An NVM Express namespace is equivalent to a SCSI LUN. * Each namespace is operated as an independent "device". @@ -636,6 +663,12 @@ struct nvme_ns { u8 flbas; }; +struct nvme_ops { + int (*alloc_queue)(struct nvme_queue *); + void (*submit_cmd)(struct nvme_queue *, struct nvme_command *); + void (*complete_cmd)(struct nvme_queue *, struct nvme_command *); +}; + int nvme_init(struct udevice *udev); #endif /* __DRIVER_NVME_H__ */ -- 2.34.1