[U-Boot] [PATCH 00/14] nvme: Various bug fixes and updates

[U-Boot] [PATCH 00/14] nvme: Various bug fixes and updates

[Bin Meng]

This series fixes various bugs and clean up on current NVMe driver.
It also enables NVMe driver on sandbox for build testing.

This series is available at u-boot-x86/nvme-working for testing.


Bin Meng (14):
  nvme: Remove useless defines
  nvme: Fix getting PCI vendor id of the NVMe block device
  nvme: Fix ndev->queues allocation
  nvme: Fix endianness assignment to prp2 in nvme_identify()
  nvme: Cache controller's capabilities
  nvme: Respect timeout when en/disabling the controller
  nvme: Use macros to access NVMe queues
  nvme: Consolidate block read and write routines
  nvme: Apply cache operations on the DMA buffers
  nvme: Use blk_create_devicef() API
  nvme: Get rid of the global variable nvme_info
  nvme: Adjust the 'nvme' command to use blk_common_cmd()
  sandbox: Add a dummy invalidate_dcache_range() function
  sandbox: Enable NVMe driver for build testing

 arch/sandbox/cpu/cpu.c             |   4 +
 cmd/nvme.c                         | 195 +++++--------------------------------
 configs/sandbox_defconfig          |   1 +
 configs/sandbox_flattree_defconfig |   1 +
 configs/sandbox_spl_defconfig      |   1 +
 doc/README.nvme                    |   4 +-
 drivers/nvme/nvme-uclass.c         |  35 ++-----
 drivers/nvme/nvme.c                | 179 +++++++++++++++-------------------
 drivers/nvme/nvme.h                |  71 +-------------
 drivers/nvme/nvme_show.c           |   7 +-
 include/config_fallbacks.h         |   1 +
 11 files changed, 131 insertions(+), 368 deletions(-)

-- 
2.9.2

[U-Boot] [PATCH 01/14] nvme: Remove useless defines

[Bin Meng]

These are leftover when the driver was ported from Linux and are not
used by the U-Boot driver.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
---

 drivers/nvme/nvme.h | 60 -----------------------------------------------------
 1 file changed, 60 deletions(-)

diff --git a/drivers/nvme/nvme.h b/drivers/nvme/nvme.h
index b7fdd0b..267884e 100644
--- a/drivers/nvme/nvme.h
+++ b/drivers/nvme/nvme.h
@@ -528,42 +528,6 @@ struct nvme_completion {
 	__le16	status;		/* did the command fail, and if so, why? */
 };
 
-struct nvme_user_io {
-	__u8	opcode;
-	__u8	flags;
-	__u16	control;
-	__u16	nblocks;
-	__u16	rsvd;
-	__u64	metadata;
-	__u64	addr;
-	__u64	slba;
-	__u32	dsmgmt;
-	__u32	reftag;
-	__u16	apptag;
-	__u16	appmask;
-};
-
-struct nvme_passthru_cmd {
-	__u8	opcode;
-	__u8	flags;
-	__u16	rsvd1;
-	__u32	nsid;
-	__u32	cdw2;
-	__u32	cdw3;
-	__u64	metadata;
-	__u64	addr;
-	__u32	metadata_len;
-	__u32	data_len;
-	__u32	cdw10;
-	__u32	cdw11;
-	__u32	cdw12;
-	__u32	cdw13;
-	__u32	cdw14;
-	__u32	cdw15;
-	__u32	timeout_ms;
-	__u32	result;
-};
-
 /*
  * Registers should always be accessed with double word or quad word
  * accesses. Registers with 64-bit address pointers should be written
@@ -644,11 +608,8 @@ struct nvme_dev {
 	struct list_head node;
 	struct nvme_queue **queues;
 	u32 __iomem *dbs;
-	unsigned int cardnum;
 	struct udevice *pdev;
-	pci_dev_t pci_dev;
 	int instance;
-	uint8_t *hw_addr;
 	unsigned queue_count;
 	unsigned online_queues;
 	unsigned max_qid;
@@ -657,16 +618,12 @@ struct nvme_dev {
 	u32 ctrl_config;
 	struct nvme_bar __iomem *bar;
 	struct list_head namespaces;
-	const char *name;
 	char serial[20];
 	char model[40];
 	char firmware_rev[8];
 	u32 max_transfer_shift;
 	u32 stripe_size;
 	u32 page_size;
-	u16 oncs;
-	u16 abort_limit;
-	u8 event_limit;
 	u8 vwc;
 	u64 *prp_pool;
 	u32 prp_entry_num;
@@ -681,21 +638,6 @@ struct nvme_info {
 };
 
 /*
- * The nvme_iod describes the data in an I/O, including the list of PRP
- * entries.  You can't see it in this data structure because C doesn't let
- * me express that.  Use nvme_alloc_iod to ensure there's enough space
- * allocated to store the PRP list.
- */
-struct nvme_iod {
-	unsigned long private;	/* For the use of the submitter of the I/O */
-	int npages;		/* In the PRP list. 0 means small pool in use */
-	int offset;		/* Of PRP list */
-	int nents;		/* Used in scatterlist */
-	int length;		/* Of data, in bytes */
-	dma_addr_t first_dma;
-};
-
-/*
  * An NVM Express namespace is equivalent to a SCSI LUN.
  * Each namespace is operated as an independent "device".
  */
@@ -705,9 +647,7 @@ struct nvme_ns {
 	unsigned ns_id;
 	int devnum;
 	int lba_shift;
-	u16 ms;
 	u8 flbas;
-	u8 pi_type;
 	u64 mode_select_num_blocks;
 	u32 mode_select_block_len;
 };
-- 
2.9.2

[U-Boot] [PATCH 02/14] nvme: Fix getting PCI vendor id of the NVMe block device

[Bin Meng]

The codes currently try to read PCI vendor id of the NVMe block
device by dm_pci_read_config16() with its parameter set as its
root complex controller (ndev->pdev) instead of itself. This is
seriously wrong. We can read the vendor id by passing the correct
udevice parameter to the dm_pci_read_config16() API, however there
is a shortcut by reading the cached vendor id from the PCI device's
struct pci_child_platdata.

While we are here fixing this bug, apparently the quirk stuff handle
codes in nvme_get_info_from_identify() never takes effect since its
logic has never been true at all. Remove these codes completely.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
---

 drivers/nvme/nvme.c | 24 +++---------------------
 drivers/nvme/nvme.h |  1 -
 2 files changed, 3 insertions(+), 22 deletions(-)

diff --git a/drivers/nvme/nvme.c b/drivers/nvme/nvme.c
index 151fe92..2ac0870 100644
--- a/drivers/nvme/nvme.c
+++ b/drivers/nvme/nvme.c
@@ -547,7 +547,6 @@ static int nvme_setup_io_queues(struct nvme_dev *dev)
 
 static int nvme_get_info_from_identify(struct nvme_dev *dev)
 {
-	u16 vendor, device;
 	struct nvme_id_ctrl buf, *ctrl = &buf;
 	int ret;
 	int shift = NVME_CAP_MPSMIN(nvme_readq(&dev->bar->cap)) + 12;
@@ -585,22 +584,6 @@ static int nvme_get_info_from_identify(struct nvme_dev *dev)
 		dev->max_transfer_shift = 20;
 	}
 
-	/* Apply quirk stuff */
-	dm_pci_read_config16(dev->pdev, PCI_VENDOR_ID, &vendor);
-	dm_pci_read_config16(dev->pdev, PCI_DEVICE_ID, &device);
-	if ((vendor == PCI_VENDOR_ID_INTEL) &&
-	    (device == 0x0953) && ctrl->vs[3]) {
-		unsigned int max_transfer_shift;
-		dev->stripe_size = (ctrl->vs[3] + shift);
-		max_transfer_shift = (ctrl->vs[3] + 18);
-		if (dev->max_transfer_shift) {
-			dev->max_transfer_shift = min(max_transfer_shift,
-						      dev->max_transfer_shift);
-		} else {
-			dev->max_transfer_shift = max_transfer_shift;
-		}
-	}
-
 	return 0;
 }
 
@@ -629,8 +612,8 @@ static int nvme_blk_probe(struct udevice *udev)
 	struct blk_desc *desc = dev_get_uclass_platdata(udev);
 	struct nvme_ns *ns = dev_get_priv(udev);
 	u8 flbas;
-	u16 vendor;
 	struct nvme_id_ns buf, *id = &buf;
+	struct pci_child_platdata *pplat;
 
 	memset(ns, 0, sizeof(*ns));
 	ns->dev = ndev;
@@ -649,8 +632,8 @@ static int nvme_blk_probe(struct udevice *udev)
 	desc->log2blksz = ns->lba_shift;
 	desc->blksz = 1 << ns->lba_shift;
 	desc->bdev = udev;
-	dm_pci_read_config16(ndev->pdev, PCI_VENDOR_ID, &vendor);
-	sprintf(desc->vendor, "0x%.4x", vendor);
+	pplat = dev_get_parent_platdata(udev->parent);
+	sprintf(desc->vendor, "0x%.4x", pplat->vendor);
 	memcpy(desc->product, ndev->serial, sizeof(ndev->serial));
 	memcpy(desc->revision, ndev->firmware_rev, sizeof(ndev->firmware_rev));
 	part_init(desc);
@@ -791,7 +774,6 @@ static int nvme_probe(struct udevice *udev)
 	struct nvme_dev *ndev = dev_get_priv(udev);
 	u64 cap;
 
-	ndev->pdev = pci_get_controller(udev);
 	ndev->instance = trailing_strtol(udev->name);
 
 	INIT_LIST_HEAD(&ndev->namespaces);
diff --git a/drivers/nvme/nvme.h b/drivers/nvme/nvme.h
index 267884e..cd411be 100644
--- a/drivers/nvme/nvme.h
+++ b/drivers/nvme/nvme.h
@@ -608,7 +608,6 @@ struct nvme_dev {
 	struct list_head node;
 	struct nvme_queue **queues;
 	u32 __iomem *dbs;
-	struct udevice *pdev;
 	int instance;
 	unsigned queue_count;
 	unsigned online_queues;
-- 
2.9.2

[U-Boot] [PATCH 03/14] nvme: Fix ndev->queues allocation

[Bin Meng]

ndev->queues is a pointer to pointer, but the allocation wrongly
requests sizeof(struct nvme_queue). Fix it.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
---

 drivers/nvme/nvme.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/nvme.c b/drivers/nvme/nvme.c
index 2ac0870..5d39cab 100644
--- a/drivers/nvme/nvme.c
+++ b/drivers/nvme/nvme.c
@@ -785,13 +785,13 @@ static int nvme_probe(struct udevice *udev)
 		goto free_nvme;
 	}
 
-	ndev->queues = malloc(2 * sizeof(struct nvme_queue));
+	ndev->queues = malloc(2 * sizeof(struct nvme_queue *));
 	if (!ndev->queues) {
 		ret = -ENOMEM;
 		printf("Error: %s: Out of memory!\n", udev->name);
 		goto free_nvme;
 	}
-	memset(ndev->queues, 0, sizeof(2 * sizeof(struct nvme_queue)));
+	memset(ndev->queues, 0, sizeof(2 * sizeof(struct nvme_queue *)));
 
 	ndev->prp_pool = malloc(MAX_PRP_POOL);
 	if (!ndev->prp_pool) {
-- 
2.9.2

[U-Boot] [PATCH 04/14] nvme: Fix endianness assignment to prp2 in nvme_identify()

[Bin Meng]

So far this is not causing any issue due to NVMe and x86 are using
the same endianness, but for correctness, it should be fixed.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
---

 drivers/nvme/nvme.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/nvme.c b/drivers/nvme/nvme.c
index 5d39cab..2ae947c 100644
--- a/drivers/nvme/nvme.c
+++ b/drivers/nvme/nvme.c
@@ -431,7 +431,7 @@ int nvme_identify(struct nvme_dev *dev, unsigned nsid,
 		c.identify.prp2 = 0;
 	} else {
 		dma_addr += (page_size - offset);
-		c.identify.prp2 = dma_addr;
+		c.identify.prp2 = cpu_to_le64(dma_addr);
 	}
 
 	c.identify.cns = cpu_to_le32(cns);
-- 
2.9.2

[U-Boot] [PATCH 05/14] nvme: Cache controller's capabilities

[Bin Meng]

Capabilities register is RO and accessed at various places in the
driver. Let's cache it in the controller driver's priv struct.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
---

 drivers/nvme/nvme.c | 11 +++++------
 drivers/nvme/nvme.h |  1 +
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/nvme/nvme.c b/drivers/nvme/nvme.c
index 2ae947c..d92273e 100644
--- a/drivers/nvme/nvme.c
+++ b/drivers/nvme/nvme.c
@@ -318,7 +318,7 @@ static int nvme_configure_admin_queue(struct nvme_dev *dev)
 {
 	int result;
 	u32 aqa;
-	u64 cap = nvme_readq(&dev->bar->cap);
+	u64 cap = dev->cap;
 	struct nvme_queue *nvmeq;
 	/* most architectures use 4KB as the page size */
 	unsigned page_shift = 12;
@@ -549,7 +549,7 @@ static int nvme_get_info_from_identify(struct nvme_dev *dev)
 {
 	struct nvme_id_ctrl buf, *ctrl = &buf;
 	int ret;
-	int shift = NVME_CAP_MPSMIN(nvme_readq(&dev->bar->cap)) + 12;
+	int shift = NVME_CAP_MPSMIN(dev->cap) + 12;
 
 	ret = nvme_identify(dev, 0, 1, (dma_addr_t)ctrl);
 	if (ret)
@@ -772,7 +772,6 @@ static int nvme_probe(struct udevice *udev)
 {
 	int ret;
 	struct nvme_dev *ndev = dev_get_priv(udev);
-	u64 cap;
 
 	ndev->instance = trailing_strtol(udev->name);
 
@@ -801,9 +800,9 @@ static int nvme_probe(struct udevice *udev)
 	}
 	ndev->prp_entry_num = MAX_PRP_POOL >> 3;
 
-	cap = nvme_readq(&ndev->bar->cap);
-	ndev->q_depth = min_t(int, NVME_CAP_MQES(cap) + 1, NVME_Q_DEPTH);
-	ndev->db_stride = 1 << NVME_CAP_STRIDE(cap);
+	ndev->cap = nvme_readq(&ndev->bar->cap);
+	ndev->q_depth = min_t(int, NVME_CAP_MQES(ndev->cap) + 1, NVME_Q_DEPTH);
+	ndev->db_stride = 1 << NVME_CAP_STRIDE(ndev->cap);
 	ndev->dbs = ((void __iomem *)ndev->bar) + 4096;
 
 	ret = nvme_configure_admin_queue(ndev);
diff --git a/drivers/nvme/nvme.h b/drivers/nvme/nvme.h
index cd411be..f0fa639 100644
--- a/drivers/nvme/nvme.h
+++ b/drivers/nvme/nvme.h
@@ -621,6 +621,7 @@ struct nvme_dev {
 	char model[40];
 	char firmware_rev[8];
 	u32 max_transfer_shift;
+	u64 cap;
 	u32 stripe_size;
 	u32 page_size;
 	u8 vwc;
-- 
2.9.2

[U-Boot] [PATCH 06/14] nvme: Respect timeout when en/disabling the controller

[Bin Meng]

So far the driver unconditionally delays 10ms when en/disabling the
controller and still return 0 if 10ms times out. In fact, spec defines
a timeout value in the CAP register that is the worst case time that
host software shall wait for the controller to become ready.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
---

 drivers/nvme/nvme.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/drivers/nvme/nvme.c b/drivers/nvme/nvme.c
index d92273e..8867977 100644
--- a/drivers/nvme/nvme.c
+++ b/drivers/nvme/nvme.c
@@ -47,11 +47,19 @@ struct nvme_queue {
 static int nvme_wait_ready(struct nvme_dev *dev, bool enabled)
 {
 	u32 bit = enabled ? NVME_CSTS_RDY : 0;
+	int timeout;
+	ulong start;
 
-	while ((readl(&dev->bar->csts) & NVME_CSTS_RDY) != bit)
-		udelay(10000);
+	/* Timeout field in the CAP register is in 500 millisecond units */
+	timeout = NVME_CAP_TIMEOUT(dev->cap) * 500;
 
-	return 0;
+	start = get_timer(0);
+	while (get_timer(start) < timeout) {
+		if ((readl(&dev->bar->csts) & NVME_CSTS_RDY) == bit)
+			return 0;
+	}
+
+	return -ETIME;
 }
 
 static int nvme_setup_prps(struct nvme_dev *dev, u64 *prp2,
-- 
2.9.2

[U-Boot] [PATCH 07/14] nvme: Use macros to access NVMe queues

[Bin Meng]

NVMe driver only uses two queues. The first one is allocated to do
admin stuff, while the second one is for IO stuff. So far the driver
uses magic number (0/1) to access them. Change to use macros.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
---

 drivers/nvme/nvme.c | 22 +++++++++++++++-------
 1 file changed, 15 insertions(+), 7 deletions(-)

diff --git a/drivers/nvme/nvme.c b/drivers/nvme/nvme.c
index 8867977..868ff45 100644
--- a/drivers/nvme/nvme.c
+++ b/drivers/nvme/nvme.c
@@ -23,6 +23,12 @@ struct nvme_info *nvme_info;
 #define IO_TIMEOUT		30
 #define MAX_PRP_POOL		512
 
+enum nvme_queue_id {
+	NVME_ADMIN_Q,
+	NVME_IO_Q,
+	NVME_Q_NUM,
+};
+
 /*
  * An NVM Express queue. Each device has at least two (one for admin
  * commands and one for I/O commands).
@@ -209,7 +215,8 @@ static int nvme_submit_sync_cmd(struct nvme_queue *nvmeq,
 static int nvme_submit_admin_cmd(struct nvme_dev *dev, struct nvme_command *cmd,
 				 u32 *result)
 {
-	return nvme_submit_sync_cmd(dev->queues[0], cmd, result, ADMIN_TIMEOUT);
+	return nvme_submit_sync_cmd(dev->queues[NVME_ADMIN_Q], cmd,
+				    result, ADMIN_TIMEOUT);
 }
 
 static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev,
@@ -349,7 +356,7 @@ static int nvme_configure_admin_queue(struct nvme_dev *dev)
 	if (result < 0)
 		return result;
 
-	nvmeq = dev->queues[0];
+	nvmeq = dev->queues[NVME_ADMIN_Q];
 	if (!nvmeq) {
 		nvmeq = nvme_alloc_queue(dev, 0, NVME_AQ_DEPTH);
 		if (!nvmeq)
@@ -377,7 +384,7 @@ static int nvme_configure_admin_queue(struct nvme_dev *dev)
 
 	nvmeq->cq_vector = 0;
 
-	nvme_init_queue(dev->queues[0], 0);
+	nvme_init_queue(dev->queues[NVME_ADMIN_Q], 0);
 
 	return result;
 
@@ -691,7 +698,7 @@ static ulong nvme_blk_read(struct udevice *udev, lbaint_t blknr,
 		c.rw.length = cpu_to_le16(lbas - 1);
 		c.rw.prp1 = cpu_to_le64((ulong)buffer);
 		c.rw.prp2 = cpu_to_le64(prp2);
-		status = nvme_submit_sync_cmd(dev->queues[1],
+		status = nvme_submit_sync_cmd(dev->queues[NVME_IO_Q],
 				&c, NULL, IO_TIMEOUT);
 		if (status)
 			break;
@@ -744,7 +751,7 @@ static ulong nvme_blk_write(struct udevice *udev, lbaint_t blknr,
 		c.rw.length = cpu_to_le16(lbas - 1);
 		c.rw.prp1 = cpu_to_le64((ulong)buffer);
 		c.rw.prp2 = cpu_to_le64(prp2);
-		status = nvme_submit_sync_cmd(dev->queues[1],
+		status = nvme_submit_sync_cmd(dev->queues[NVME_IO_Q],
 				&c, NULL, IO_TIMEOUT);
 		if (status)
 			break;
@@ -792,13 +799,14 @@ static int nvme_probe(struct udevice *udev)
 		goto free_nvme;
 	}
 
-	ndev->queues = malloc(2 * sizeof(struct nvme_queue *));
+	ndev->queues = malloc(NVME_Q_NUM * sizeof(struct nvme_queue *));
 	if (!ndev->queues) {
 		ret = -ENOMEM;
 		printf("Error: %s: Out of memory!\n", udev->name);
 		goto free_nvme;
 	}
-	memset(ndev->queues, 0, sizeof(2 * sizeof(struct nvme_queue *)));
+	memset(ndev->queues, 0,
+	       sizeof(NVME_Q_NUM * sizeof(struct nvme_queue *)));
 
 	ndev->prp_pool = malloc(MAX_PRP_POOL);
 	if (!ndev->prp_pool) {
-- 
2.9.2

[U-Boot] [PATCH 08/14] nvme: Consolidate block read and write routines

[Bin Meng]

The NVMe block read and write routines are almost the same except
the command opcode. Let's consolidate them to avoid duplication.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
---

 drivers/nvme/nvme.c | 65 ++++++++++-------------------------------------------
 1 file changed, 12 insertions(+), 53 deletions(-)

diff --git a/drivers/nvme/nvme.c b/drivers/nvme/nvme.c
index 868ff45..c545ce7 100644
--- a/drivers/nvme/nvme.c
+++ b/drivers/nvme/nvme.c
@@ -656,8 +656,8 @@ static int nvme_blk_probe(struct udevice *udev)
 	return 0;
 }
 
-static ulong nvme_blk_read(struct udevice *udev, lbaint_t blknr,
-			   lbaint_t blkcnt, void *buffer)
+static ulong nvme_blk_rw(struct udevice *udev, lbaint_t blknr,
+			 lbaint_t blkcnt, void *buffer, bool read)
 {
 	struct nvme_ns *ns = dev_get_priv(udev);
 	struct nvme_dev *dev = ns->dev;
@@ -672,7 +672,7 @@ static ulong nvme_blk_read(struct udevice *udev, lbaint_t blknr,
 	u16 lbas = 1 << (dev->max_transfer_shift - ns->lba_shift);
 	u64 total_lbas = blkcnt;
 
-	c.rw.opcode = nvme_cmd_read;
+	c.rw.opcode = read ? nvme_cmd_read : nvme_cmd_write;
 	c.rw.flags = 0;
 	c.rw.nsid = cpu_to_le32(ns->ns_id);
 	c.rw.control = 0;
@@ -690,8 +690,8 @@ static ulong nvme_blk_read(struct udevice *udev, lbaint_t blknr,
 			total_lbas -= lbas;
 		}
 
-		if (nvme_setup_prps
-		   (dev, &prp2, lbas << ns->lba_shift, (ulong)buffer))
+		if (nvme_setup_prps(dev, &prp2,
+				    lbas << ns->lba_shift, (ulong)buffer))
 			return -EIO;
 		c.rw.slba = cpu_to_le64(slba);
 		slba += lbas;
@@ -709,57 +709,16 @@ static ulong nvme_blk_read(struct udevice *udev, lbaint_t blknr,
 	return (total_len - temp_len) >> desc->log2blksz;
 }
 
+static ulong nvme_blk_read(struct udevice *udev, lbaint_t blknr,
+			   lbaint_t blkcnt, void *buffer)
+{
+	return nvme_blk_rw(udev, blknr, blkcnt, buffer, true);
+}
+
 static ulong nvme_blk_write(struct udevice *udev, lbaint_t blknr,
 			    lbaint_t blkcnt, const void *buffer)
 {
-	struct nvme_ns *ns = dev_get_priv(udev);
-	struct nvme_dev *dev = ns->dev;
-	struct nvme_command c;
-	struct blk_desc *desc = dev_get_uclass_platdata(udev);
-	int status;
-	u64 prp2;
-	u64 total_len = blkcnt << desc->log2blksz;
-	u64 temp_len = total_len;
-
-	u64 slba = blknr;
-	u16 lbas = 1 << (dev->max_transfer_shift - ns->lba_shift);
-	u64 total_lbas = blkcnt;
-
-	c.rw.opcode = nvme_cmd_write;
-	c.rw.flags = 0;
-	c.rw.nsid = cpu_to_le32(ns->ns_id);
-	c.rw.control = 0;
-	c.rw.dsmgmt = 0;
-	c.rw.reftag = 0;
-	c.rw.apptag = 0;
-	c.rw.appmask = 0;
-	c.rw.metadata = 0;
-
-	while (total_lbas) {
-		if (total_lbas < lbas) {
-			lbas = (u16)total_lbas;
-			total_lbas = 0;
-		} else {
-			total_lbas -= lbas;
-		}
-
-		if (nvme_setup_prps
-		   (dev, &prp2, lbas << ns->lba_shift, (ulong)buffer))
-			return -EIO;
-		c.rw.slba = cpu_to_le64(slba);
-		slba += lbas;
-		c.rw.length = cpu_to_le16(lbas - 1);
-		c.rw.prp1 = cpu_to_le64((ulong)buffer);
-		c.rw.prp2 = cpu_to_le64(prp2);
-		status = nvme_submit_sync_cmd(dev->queues[NVME_IO_Q],
-				&c, NULL, IO_TIMEOUT);
-		if (status)
-			break;
-		temp_len -= lbas << ns->lba_shift;
-		buffer += lbas << ns->lba_shift;
-	}
-
-	return (total_len - temp_len) >> desc->log2blksz;
+	return nvme_blk_rw(udev, blknr, blkcnt, (void *)buffer, false);
 }
 
 static const struct blk_ops nvme_blk_ops = {
-- 
2.9.2

[U-Boot] [PATCH 09/14] nvme: Apply cache operations on the DMA buffers

[Bin Meng]

So far cache operations are only applied on the submission queue and
completion queue, but they are missing in other places like identify
and block read/write routines.

In order to correctly operate on the caches, the DMA buffer passed
to identify routine must be allocated properly on the stack with the
existing macro ALLOC_CACHE_ALIGN_BUFFER().

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
---

 drivers/nvme/nvme.c      | 32 +++++++++++++++++++++++++++++---
 drivers/nvme/nvme_show.c |  7 +++++--
 2 files changed, 34 insertions(+), 5 deletions(-)

diff --git a/drivers/nvme/nvme.c b/drivers/nvme/nvme.c
index c545ce7..67f7d75 100644
--- a/drivers/nvme/nvme.c
+++ b/drivers/nvme/nvme.c
@@ -435,6 +435,7 @@ int nvme_identify(struct nvme_dev *dev, unsigned nsid,
 	u32 page_size = dev->page_size;
 	int offset = dma_addr & (page_size - 1);
 	int length = sizeof(struct nvme_id_ctrl);
+	int ret;
 
 	memset(&c, 0, sizeof(c));
 	c.identify.opcode = nvme_admin_identify;
@@ -451,7 +452,12 @@ int nvme_identify(struct nvme_dev *dev, unsigned nsid,
 
 	c.identify.cns = cpu_to_le32(cns);
 
-	return nvme_submit_admin_cmd(dev, &c, NULL);
+	ret = nvme_submit_admin_cmd(dev, &c, NULL);
+	if (!ret)
+		invalidate_dcache_range(dma_addr,
+					dma_addr + sizeof(struct nvme_id_ctrl));
+
+	return ret;
 }
 
 int nvme_get_features(struct nvme_dev *dev, unsigned fid, unsigned nsid,
@@ -465,6 +471,11 @@ int nvme_get_features(struct nvme_dev *dev, unsigned fid, unsigned nsid,
 	c.features.prp1 = cpu_to_le64(dma_addr);
 	c.features.fid = cpu_to_le32(fid);
 
+	/*
+	 * TODO: add cache invalidate operation when the size of
+	 * the DMA buffer is known
+	 */
+
 	return nvme_submit_admin_cmd(dev, &c, result);
 }
 
@@ -479,6 +490,11 @@ int nvme_set_features(struct nvme_dev *dev, unsigned fid, unsigned dword11,
 	c.features.fid = cpu_to_le32(fid);
 	c.features.dword11 = cpu_to_le32(dword11);
 
+	/*
+	 * TODO: add cache flush operation when the size of
+	 * the DMA buffer is known
+	 */
+
 	return nvme_submit_admin_cmd(dev, &c, result);
 }
 
@@ -562,7 +578,8 @@ static int nvme_setup_io_queues(struct nvme_dev *dev)
 
 static int nvme_get_info_from_identify(struct nvme_dev *dev)
 {
-	struct nvme_id_ctrl buf, *ctrl = &buf;
+	ALLOC_CACHE_ALIGN_BUFFER(char, buf, sizeof(struct nvme_id_ctrl));
+	struct nvme_id_ctrl *ctrl = (struct nvme_id_ctrl *)buf;
 	int ret;
 	int shift = NVME_CAP_MPSMIN(dev->cap) + 12;
 
@@ -627,7 +644,8 @@ static int nvme_blk_probe(struct udevice *udev)
 	struct blk_desc *desc = dev_get_uclass_platdata(udev);
 	struct nvme_ns *ns = dev_get_priv(udev);
 	u8 flbas;
-	struct nvme_id_ns buf, *id = &buf;
+	ALLOC_CACHE_ALIGN_BUFFER(char, buf, sizeof(struct nvme_id_ns));
+	struct nvme_id_ns *id = (struct nvme_id_ns *)buf;
 	struct pci_child_platdata *pplat;
 
 	memset(ns, 0, sizeof(*ns));
@@ -672,6 +690,10 @@ static ulong nvme_blk_rw(struct udevice *udev, lbaint_t blknr,
 	u16 lbas = 1 << (dev->max_transfer_shift - ns->lba_shift);
 	u64 total_lbas = blkcnt;
 
+	if (!read)
+		flush_dcache_range((unsigned long)buffer,
+				   (unsigned long)buffer + total_len);
+
 	c.rw.opcode = read ? nvme_cmd_read : nvme_cmd_write;
 	c.rw.flags = 0;
 	c.rw.nsid = cpu_to_le32(ns->ns_id);
@@ -706,6 +728,10 @@ static ulong nvme_blk_rw(struct udevice *udev, lbaint_t blknr,
 		buffer += lbas << ns->lba_shift;
 	}
 
+	if (read)
+		invalidate_dcache_range((unsigned long)buffer,
+					(unsigned long)buffer + total_len);
+
 	return (total_len - temp_len) >> desc->log2blksz;
 }
 
diff --git a/drivers/nvme/nvme_show.c b/drivers/nvme/nvme_show.c
index 5577e5d..5235138 100644
--- a/drivers/nvme/nvme_show.c
+++ b/drivers/nvme/nvme_show.c
@@ -8,6 +8,7 @@
 #include <common.h>
 #include <dm.h>
 #include <errno.h>
+#include <memalign.h>
 #include <nvme.h>
 #include "nvme.h"
 
@@ -106,8 +107,10 @@ int nvme_print_info(struct udevice *udev)
 {
 	struct nvme_ns *ns = dev_get_priv(udev);
 	struct nvme_dev *dev = ns->dev;
-	struct nvme_id_ns buf_ns, *id = &buf_ns;
-	struct nvme_id_ctrl buf_ctrl, *ctrl = &buf_ctrl;
+	ALLOC_CACHE_ALIGN_BUFFER(char, buf_ns, sizeof(struct nvme_id_ns));
+	struct nvme_id_ns *id = (struct nvme_id_ns *)buf_ns;
+	ALLOC_CACHE_ALIGN_BUFFER(char, buf_ctrl, sizeof(struct nvme_id_ctrl));
+	struct nvme_id_ctrl *ctrl = (struct nvme_id_ctrl *)buf_ctrl;
 
 	if (nvme_identify(dev, 0, 1, (dma_addr_t)ctrl))
 		return -EIO;
-- 
2.9.2

[U-Boot] [PATCH 10/14] nvme: Use blk_create_devicef() API

[Bin Meng]

The codes in nvme_uclass_post_probe() can be replaced to call the
blk_create_devicef() API directly.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
---

 drivers/nvme/nvme-uclass.c | 12 +++---------
 1 file changed, 3 insertions(+), 9 deletions(-)

diff --git a/drivers/nvme/nvme-uclass.c b/drivers/nvme/nvme-uclass.c
index 0895bc9..334628c 100644
--- a/drivers/nvme/nvme-uclass.c
+++ b/drivers/nvme/nvme-uclass.c
@@ -26,28 +26,22 @@ static int nvme_info_init(struct uclass *uc)
 static int nvme_uclass_post_probe(struct udevice *udev)
 {
 	char name[20];
-	char *str;
 	struct udevice *ns_udev;
 	int i, ret;
 	struct nvme_dev *ndev = dev_get_priv(udev);
 
 	/* Create a blk device for each namespace */
 	for (i = 0; i < ndev->nn; i++) {
-		sprintf(name, "nvme-blk#%d", nvme_info->ns_num);
-		str = strdup(name);
-		if (!str)
-			return -ENOMEM;
+		sprintf(name, "blk#%d", nvme_info->ns_num);
 
 		/* The real blksz and size will be set by nvme_blk_probe() */
-		ret = blk_create_device(udev, "nvme-blk", str, IF_TYPE_NVME,
-					nvme_info->ns_num++, 512, 0, &ns_udev);
+		ret = blk_create_devicef(udev, "nvme-blk", name, IF_TYPE_NVME,
+					 nvme_info->ns_num++, 512, 0, &ns_udev);
 		if (ret) {
-			free(str);
 			nvme_info->ns_num--;
 
 			return ret;
 		}
-		device_set_name_alloced(ns_udev);
 	}
 
 	return 0;
-- 
2.9.2

[U-Boot] [PATCH 11/14] nvme: Get rid of the global variable nvme_info

[Bin Meng]

At present the NVMe uclass driver uses a global variable nvme_info
to store global information like namespace id, and NVMe controller
driver's priv struct has a blk_dev_start that is used to calculate
the namespace id based on the global information from nvme_info.

This is not a good design in the DM world and can be replaced with
the following changes:

- Encode the namespace id in the NVMe block device name during
  the NVMe uclass post probe
- Extract the namespace id from the device name during the NVMe
  block device probe
- Let BLK uclass calculate the devnum for us by passing -1 to
  blk_create_devicef() as the devnum

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
---

 drivers/nvme/nvme-uclass.c | 27 +++++++--------------------
 drivers/nvme/nvme.c        | 11 +++++------
 drivers/nvme/nvme.h        |  9 ---------
 3 files changed, 12 insertions(+), 35 deletions(-)

diff --git a/drivers/nvme/nvme-uclass.c b/drivers/nvme/nvme-uclass.c
index 334628c..56a6171 100644
--- a/drivers/nvme/nvme-uclass.c
+++ b/drivers/nvme/nvme-uclass.c
@@ -11,18 +11,6 @@
 #include <dm/device.h>
 #include "nvme.h"
 
-static int nvme_info_init(struct uclass *uc)
-{
-	struct nvme_info *info = (struct nvme_info *)uc->priv;
-
-	info->ns_num = 0;
-	info->ndev_num = 0;
-	INIT_LIST_HEAD(&info->dev_list);
-	nvme_info = info;
-
-	return 0;
-}
-
 static int nvme_uclass_post_probe(struct udevice *udev)
 {
 	char name[20];
@@ -32,16 +20,17 @@ static int nvme_uclass_post_probe(struct udevice *udev)
 
 	/* Create a blk device for each namespace */
 	for (i = 0; i < ndev->nn; i++) {
-		sprintf(name, "blk#%d", nvme_info->ns_num);
+		/*
+		 * Encode the namespace id to the device name so that
+		 * we can extract it when doing the probe.
+		 */
+		sprintf(name, "blk#%d", i);
 
 		/* The real blksz and size will be set by nvme_blk_probe() */
 		ret = blk_create_devicef(udev, "nvme-blk", name, IF_TYPE_NVME,
-					 nvme_info->ns_num++, 512, 0, &ns_udev);
-		if (ret) {
-			nvme_info->ns_num--;
-
+					 -1, 512, 0, &ns_udev);
+		if (ret)
 			return ret;
-		}
 	}
 
 	return 0;
@@ -50,7 +39,5 @@ static int nvme_uclass_post_probe(struct udevice *udev)
 UCLASS_DRIVER(nvme) = {
 	.name	= "nvme",
 	.id	= UCLASS_NVME,
-	.init	= nvme_info_init,
 	.post_probe = nvme_uclass_post_probe,
-	.priv_auto_alloc_size = sizeof(struct nvme_info),
 };
diff --git a/drivers/nvme/nvme.c b/drivers/nvme/nvme.c
index 67f7d75..ec32d0d 100644
--- a/drivers/nvme/nvme.c
+++ b/drivers/nvme/nvme.c
@@ -13,8 +13,6 @@
 #include <dm/device-internal.h>
 #include "nvme.h"
 
-struct nvme_info *nvme_info;
-
 #define NVME_Q_DEPTH		2
 #define NVME_AQ_DEPTH		2
 #define NVME_SQ_SIZE(depth)	(depth * sizeof(struct nvme_command))
@@ -650,7 +648,8 @@ static int nvme_blk_probe(struct udevice *udev)
 
 	memset(ns, 0, sizeof(*ns));
 	ns->dev = ndev;
-	ns->ns_id = desc->devnum - ndev->blk_dev_start + 1;
+	/* extract the namespace id from the block device name */
+	ns->ns_id = trailing_strtol(udev->name) + 1;
 	if (nvme_identify(ndev, ns->ns_id, 0, (dma_addr_t)id))
 		return -EIO;
 
@@ -762,8 +761,10 @@ U_BOOT_DRIVER(nvme_blk) = {
 
 static int nvme_bind(struct udevice *udev)
 {
+	static int ndev_num;
 	char name[20];
-	sprintf(name, "nvme#%d", nvme_info->ndev_num++);
+
+	sprintf(name, "nvme#%d", ndev_num++);
 
 	return device_set_name(udev, name);
 }
@@ -815,8 +816,6 @@ static int nvme_probe(struct udevice *udev)
 		goto free_queue;
 
 	nvme_get_info_from_identify(ndev);
-	ndev->blk_dev_start = nvme_info->ns_num;
-	list_add(&ndev->node, &nvme_info->dev_list);
 
 	return 0;
 
diff --git a/drivers/nvme/nvme.h b/drivers/nvme/nvme.h
index f0fa639..67bf6e1 100644
--- a/drivers/nvme/nvme.h
+++ b/drivers/nvme/nvme.h
@@ -628,13 +628,6 @@ struct nvme_dev {
 	u64 *prp_pool;
 	u32 prp_entry_num;
 	u32 nn;
-	u32 blk_dev_start;
-};
-
-struct nvme_info {
-	int ns_num;	/*the number of nvme namespaces*/
-	int ndev_num;	/*the number of nvme devices*/
-	struct list_head dev_list;
 };
 
 /*
@@ -652,6 +645,4 @@ struct nvme_ns {
 	u32 mode_select_block_len;
 };
 
-extern struct nvme_info *nvme_info;
-
 #endif /* __DRIVER_NVME_H__ */
-- 
2.9.2

[U-Boot] [PATCH 12/14] nvme: Adjust the 'nvme' command to use blk_common_cmd()

[Bin Meng]

Instead of having separate code in the 'nvme' command, adjust it to use
the common function.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
---

 cmd/nvme.c                 | 195 +++++++--------------------------------------
 doc/README.nvme            |   4 +-
 include/config_fallbacks.h |   1 +
 3 files changed, 30 insertions(+), 170 deletions(-)

diff --git a/cmd/nvme.c b/cmd/nvme.c
index e1ef95f..63a8e5a 100644
--- a/cmd/nvme.c
+++ b/cmd/nvme.c
@@ -9,189 +9,48 @@
 #include <command.h>
 #include <dm.h>
 #include <nvme.h>
-#include <part.h>
-#include <linux/math64.h>
 
-static int nvme_curr_device;
+static int nvme_curr_dev;
 
-static int do_nvme_scan(cmd_tbl_t *cmdtp, int flag,
-		int argc, char * const argv[])
+static int do_nvme(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])
 {
 	int ret;
 
-	ret = nvme_scan_namespace();
-	if (ret)
-		return CMD_RET_FAILURE;
+	if (argc == 2) {
+		if (strncmp(argv[1], "scan", 4) == 0) {
+			ret = nvme_scan_namespace();
+			if (ret)
+				return CMD_RET_FAILURE;
 
-	return CMD_RET_SUCCESS;
-}
-
-static int do_nvme_list(cmd_tbl_t *cmdtp, int flag,
-		int argc, char * const argv[])
-{
-	blk_list_devices(IF_TYPE_NVME);
-
-	return CMD_RET_SUCCESS;
-}
-
-static int do_nvme_info(cmd_tbl_t *cmdtp, int flag,
-		int argc, char * const argv[])
-{
-	int devnum;
-	struct udevice *udev;
-	int ret;
-
-	if (argc > 1)
-		devnum = (int)simple_strtoul(argv[1], NULL, 10);
-	else
-		devnum = nvme_curr_device;
-
-	ret = blk_get_device(IF_TYPE_NVME, devnum, &udev);
-	if (ret < 0)
-		return CMD_RET_FAILURE;
-
-	nvme_print_info(udev);
-
-	return CMD_RET_SUCCESS;
-}
-
-static int do_nvme_device(cmd_tbl_t *cmdtp, int flag,
-		int argc, char * const argv[])
-{
-	if (argc > 1) {
-		int devnum = (int)simple_strtoul(argv[1], NULL, 10);
-
-		if (!blk_show_device(IF_TYPE_NVME, devnum)) {
-			nvme_curr_device = devnum;
-			printf("... is now current device\n");
-		} else {
-			return CMD_RET_FAILURE;
+			return ret;
 		}
-	} else {
-		blk_show_device(IF_TYPE_NVME, nvme_curr_device);
-	}
+		if (strncmp(argv[1], "deta", 4) == 0) {
+			struct udevice *udev;
 
-	return CMD_RET_SUCCESS;
-}
+			ret = blk_get_device(IF_TYPE_NVME, nvme_curr_dev,
+					     &udev);
+			if (ret < 0)
+				return CMD_RET_FAILURE;
 
-static int do_nvme_part(cmd_tbl_t *cmdtp, int flag,
-		int argc, char * const argv[])
-{
-	if (argc > 1) {
-		int devnum = (int)simple_strtoul(argv[2], NULL, 10);
+			nvme_print_info(udev);
 
-		if (blk_print_part_devnum(IF_TYPE_NVME, devnum)) {
-			printf("\nNVMe device %d not available\n", devnum);
-			return CMD_RET_FAILURE;
+			return ret;
 		}
-	} else {
-		blk_print_part_devnum(IF_TYPE_NVME, nvme_curr_device);
-	}
-
-	return CMD_RET_SUCCESS;
-}
-
-static int do_nvme_read(cmd_tbl_t *cmdtp, int flag, int argc,
-		char * const argv[])
-{
-	unsigned long time;
-	if (argc != 4)
-		return CMD_RET_USAGE;
-
-	ulong addr = simple_strtoul(argv[1], NULL, 16);
-	ulong cnt = simple_strtoul(argv[3], NULL, 16);
-	ulong n;
-	lbaint_t blk = simple_strtoul(argv[2], NULL, 16);
-
-	printf("\nNVMe read: device %d block # " LBAFU " count %ld ... ",
-	       nvme_curr_device, blk, cnt);
-
-	time = get_timer(0);
-	n = blk_read_devnum(IF_TYPE_NVME, nvme_curr_device, blk,
-			    cnt, (ulong *)addr);
-	time = get_timer(time);
-
-	printf("read: %s\n", (n == cnt) ? "OK" : "ERROR");
-	printf("%lu bytes read in %lu ms", cnt * 512, time);
-	if (time > 0) {
-		puts(" (");
-		print_size(div_u64(cnt * 512, time) * 1000, "/s");
-		puts(")");
 	}
-	puts("\n");
-
-	return (n == cnt) ? CMD_RET_SUCCESS : CMD_RET_FAILURE;
-}
-
-static int do_nvme_write(cmd_tbl_t *cmdtp, int flag, int argc,
-		char * const argv[])
-{
-	unsigned long time;
-	if (argc != 4)
-		return CMD_RET_USAGE;
-
-	ulong addr = simple_strtoul(argv[1], NULL, 16);
-	ulong cnt = simple_strtoul(argv[3], NULL, 16);
-	ulong n;
-	lbaint_t blk = simple_strtoul(argv[2], NULL, 16);
-
-	printf("\nNVMe write: device %d block # " LBAFU " count %ld ... ",
-	       nvme_curr_device, blk, cnt);
-
-	time = get_timer(0);
-	n = blk_write_devnum(IF_TYPE_NVME, nvme_curr_device, blk,
-			    cnt, (ulong *)addr);
-	time = get_timer(time);
-
-	printf("write: %s\n", (n == cnt) ? "OK" : "ERROR");
-	printf("%lu bytes write in %lu ms", cnt * 512, time);
-	if (time > 0) {
-		puts(" (");
-		print_size(div_u64(cnt * 512, time) * 1000, "/s");
-		puts(")");
-	}
-	puts("\n");
-
-	return (n == cnt) ? CMD_RET_SUCCESS : CMD_RET_FAILURE;
-}
-
-static cmd_tbl_t cmd_nvme[] = {
-	U_BOOT_CMD_MKENT(scan, 1, 1, do_nvme_scan, "", ""),
-	U_BOOT_CMD_MKENT(list, 1, 1, do_nvme_list, "", ""),
-	U_BOOT_CMD_MKENT(info, 2, 1, do_nvme_info, "", ""),
-	U_BOOT_CMD_MKENT(device, 2, 1, do_nvme_device, "", ""),
-	U_BOOT_CMD_MKENT(part, 2, 1, do_nvme_part, "", ""),
-	U_BOOT_CMD_MKENT(write, 4, 0, do_nvme_write, "", ""),
-	U_BOOT_CMD_MKENT(read, 4, 0, do_nvme_read, "", "")
-};
-
-static int do_nvmecops(cmd_tbl_t *cmdtp, int flag, int argc,
-		char * const argv[])
-{
-	cmd_tbl_t *cp;
-
-	cp = find_cmd_tbl(argv[1], cmd_nvme, ARRAY_SIZE(cmd_nvme));
-
-	argc--;
-	argv++;
-
-	if (cp == NULL || argc > cp->maxargs)
-		return CMD_RET_USAGE;
-
-	if (flag == CMD_FLAG_REPEAT && !cp->repeatable)
-		return CMD_RET_SUCCESS;
 
-	return cp->cmd(cmdtp, flag, argc, argv);
+	return blk_common_cmd(argc, argv, IF_TYPE_NVME, &nvme_curr_dev);
 }
 
 U_BOOT_CMD(
-	nvme, 8, 1, do_nvmecops,
+	nvme, 8, 1, do_nvme,
 	"NVM Express sub-system",
-	"\nnvme scan - scan NVMe blk devices\n"
-	"nvme list - show all available NVMe blk devices\n"
-	"nvme info [dev]- show current or a specific NVMe blk device\n"
-	"nvme device [dev] - show or set current device\n"
-	"nvme part [dev] - print partition table\n"
-	"nvme read  addr blk# cnt\n"
-	"nvme write addr blk# cnt"
+	"scan - scan NVMe devices\n"
+	"nvme detail - show details of current NVMe device\n"
+	"nvme info - show all available NVMe devices\n"
+	"nvme device [dev] - show or set current NVMe device\n"
+	"nvme part [dev] - print partition table of one or all NVMe devices\n"
+	"nvme read addr blk# cnt - read `cnt' blocks starting at block\n"
+	"     `blk#' to memory address `addr'\n"
+	"nvme write addr blk# cnt - write `cnt' blocks starting at block\n"
+	"     `blk#' from memory address `addr'"
 );
diff --git a/doc/README.nvme b/doc/README.nvme
index 3afa912..d9bda23 100644
--- a/doc/README.nvme
+++ b/doc/README.nvme
@@ -50,14 +50,14 @@ identified.
 
 To list all of the NVMe hard disks, try:
 
-  => nvme list
+  => nvme info
   Device 0: Vendor: 0x8086 Rev: 8DV10131 Prod: CVFT535600LS400BGN
 	    Type: Hard Disk
 	    Capacity: 381554.0 MB = 372.6 GB (781422768 x 512)
 
 and print out detailed information for controller and namespaces via:
 
-  => nvme info
+  => nvme detail
 
 Raw block read/write to can be done via the 'nvme read/write' commands:
 
diff --git a/include/config_fallbacks.h b/include/config_fallbacks.h
index 961a83d..d1bde5f 100644
--- a/include/config_fallbacks.h
+++ b/include/config_fallbacks.h
@@ -50,6 +50,7 @@
 	defined(CONFIG_CMD_PART) || \
 	defined(CONFIG_CMD_GPT) || \
 	defined(CONFIG_MMC) || \
+	defined(CONFIG_NVME) || \
 	defined(CONFIG_SYSTEMACE) || \
 	defined(CONFIG_SANDBOX)
 #define HAVE_BLOCK_DEVICE
-- 
2.9.2

[U-Boot] [PATCH 13/14] sandbox: Add a dummy invalidate_dcache_range() function

[Bin Meng]

This adds invalidate_dcache_range() so that some drivers can build
without error on sandbox.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
---

 arch/sandbox/cpu/cpu.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/sandbox/cpu/cpu.c b/arch/sandbox/cpu/cpu.c
index eefed2e..0199104 100644
--- a/arch/sandbox/cpu/cpu.c
+++ b/arch/sandbox/cpu/cpu.c
@@ -100,6 +100,10 @@ void flush_dcache_range(unsigned long start, unsigned long stop)
 {
 }
 
+void invalidate_dcache_range(unsigned long start, unsigned long stop)
+{
+}
+
 int sandbox_read_fdt_from_file(void)
 {
 	struct sandbox_state *state = state_get_current();
-- 
2.9.2

[U-Boot] [PATCH 14/14] sandbox: Enable NVMe driver for build testing

[Bin Meng]

This enables NVMe driver on sandbox for build testing.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

---

 configs/sandbox_defconfig          | 1 +
 configs/sandbox_flattree_defconfig | 1 +
 configs/sandbox_spl_defconfig      | 1 +
 3 files changed, 3 insertions(+)

diff --git a/configs/sandbox_defconfig b/configs/sandbox_defconfig
index 070262f..750a621 100644
--- a/configs/sandbox_defconfig
+++ b/configs/sandbox_defconfig
@@ -119,6 +119,7 @@ CONFIG_SPI_FLASH_STMICRO=y
 CONFIG_SPI_FLASH_SST=y
 CONFIG_SPI_FLASH_WINBOND=y
 CONFIG_DM_ETH=y
+CONFIG_NVME=y
 CONFIG_PCI=y
 CONFIG_DM_PCI=y
 CONFIG_DM_PCI_COMPAT=y
diff --git a/configs/sandbox_flattree_defconfig b/configs/sandbox_flattree_defconfig
index 1eecf1e..15a90e6 100644
--- a/configs/sandbox_flattree_defconfig
+++ b/configs/sandbox_flattree_defconfig
@@ -107,6 +107,7 @@ CONFIG_SPI_FLASH_STMICRO=y
 CONFIG_SPI_FLASH_SST=y
 CONFIG_SPI_FLASH_WINBOND=y
 CONFIG_DM_ETH=y
+CONFIG_NVME=y
 CONFIG_PCI=y
 CONFIG_DM_PCI=y
 CONFIG_DM_PCI_COMPAT=y
diff --git a/configs/sandbox_spl_defconfig b/configs/sandbox_spl_defconfig
index d1ccdfe..c08257c 100644
--- a/configs/sandbox_spl_defconfig
+++ b/configs/sandbox_spl_defconfig
@@ -127,6 +127,7 @@ CONFIG_SPI_FLASH_STMICRO=y
 CONFIG_SPI_FLASH_SST=y
 CONFIG_SPI_FLASH_WINBOND=y
 CONFIG_DM_ETH=y
+CONFIG_NVME=y
 CONFIG_PCI=y
 CONFIG_DM_PCI=y
 CONFIG_DM_PCI_COMPAT=y
-- 
2.9.2

[U-Boot] [PATCH 00/14] nvme: Various bug fixes and updates

[Tom Rini]

On Tue, Aug 22, 2017 at 08:15:05AM -0700, Bin Meng wrote:

> This series fixes various bugs and clean up on current NVMe driver.
> It also enables NVMe driver on sandbox for build testing.
> 
> This series is available at u-boot-x86/nvme-working for testing.
> 
> 
> Bin Meng (14):
>   nvme: Remove useless defines
>   nvme: Fix getting PCI vendor id of the NVMe block device
>   nvme: Fix ndev->queues allocation
>   nvme: Fix endianness assignment to prp2 in nvme_identify()
>   nvme: Cache controller's capabilities
>   nvme: Respect timeout when en/disabling the controller
>   nvme: Use macros to access NVMe queues
>   nvme: Consolidate block read and write routines
>   nvme: Apply cache operations on the DMA buffers
>   nvme: Use blk_create_devicef() API
>   nvme: Get rid of the global variable nvme_info
>   nvme: Adjust the 'nvme' command to use blk_common_cmd()
>   sandbox: Add a dummy invalidate_dcache_range() function
>   sandbox: Enable NVMe driver for build testing

I'll grab this probably Monday, barring further comments requiring
changes.  Thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170824/db1e0777/attachment.sig>

[U-Boot] [U-Boot,01/14] nvme: Remove useless defines

[Tom Rini]

On Tue, Aug 22, 2017 at 08:15:06AM -0700, Bin Meng wrote:

> These are leftover when the driver was ported from Linux and are not
> used by the U-Boot driver.
> 
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170828/ac48292c/attachment.sig>

[U-Boot] [U-Boot, 02/14] nvme: Fix getting PCI vendor id of the NVMe block device

[Tom Rini]

On Tue, Aug 22, 2017 at 08:15:07AM -0700, Bin Meng wrote:

> The codes currently try to read PCI vendor id of the NVMe block
> device by dm_pci_read_config16() with its parameter set as its
> root complex controller (ndev->pdev) instead of itself. This is
> seriously wrong. We can read the vendor id by passing the correct
> udevice parameter to the dm_pci_read_config16() API, however there
> is a shortcut by reading the cached vendor id from the PCI device's
> struct pci_child_platdata.
> 
> While we are here fixing this bug, apparently the quirk stuff handle
> codes in nvme_get_info_from_identify() never takes effect since its
> logic has never been true at all. Remove these codes completely.
> 
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170828/5f87168d/attachment.sig>

[U-Boot] [U-Boot,03/14] nvme: Fix ndev->queues allocation

[Tom Rini]

On Tue, Aug 22, 2017 at 08:15:08AM -0700, Bin Meng wrote:

> ndev->queues is a pointer to pointer, but the allocation wrongly
> requests sizeof(struct nvme_queue). Fix it.
> 
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170828/cf965ea7/attachment.sig>

[U-Boot] [U-Boot, 04/14] nvme: Fix endianness assignment to prp2 in nvme_identify()

[Tom Rini]

On Tue, Aug 22, 2017 at 08:15:09AM -0700, Bin Meng wrote:

> So far this is not causing any issue due to NVMe and x86 are using
> the same endianness, but for correctness, it should be fixed.
> 
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170828/44d9183c/attachment.sig>

[U-Boot] [U-Boot,05/14] nvme: Cache controller's capabilities

[Tom Rini]

On Tue, Aug 22, 2017 at 08:15:10AM -0700, Bin Meng wrote:

> Capabilities register is RO and accessed at various places in the
> driver. Let's cache it in the controller driver's priv struct.
> 
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170828/b30ec194/attachment.sig>

[U-Boot] [U-Boot, 06/14] nvme: Respect timeout when en/disabling the controller

[Tom Rini]

On Tue, Aug 22, 2017 at 08:15:11AM -0700, Bin Meng wrote:

> So far the driver unconditionally delays 10ms when en/disabling the
> controller and still return 0 if 10ms times out. In fact, spec defines
> a timeout value in the CAP register that is the worst case time that
> host software shall wait for the controller to become ready.
> 
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170828/f0ece969/attachment.sig>

[U-Boot] [U-Boot,07/14] nvme: Use macros to access NVMe queues

[Tom Rini]

On Tue, Aug 22, 2017 at 08:15:12AM -0700, Bin Meng wrote:

> NVMe driver only uses two queues. The first one is allocated to do
> admin stuff, while the second one is for IO stuff. So far the driver
> uses magic number (0/1) to access them. Change to use macros.
> 
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170828/1effdb2b/attachment.sig>

[U-Boot] [U-Boot, 08/14] nvme: Consolidate block read and write routines

[Tom Rini]

On Tue, Aug 22, 2017 at 08:15:13AM -0700, Bin Meng wrote:

> The NVMe block read and write routines are almost the same except
> the command opcode. Let's consolidate them to avoid duplication.
> 
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170828/935c1a11/attachment.sig>

[U-Boot] [U-Boot, 09/14] nvme: Apply cache operations on the DMA buffers

[Tom Rini]

On Tue, Aug 22, 2017 at 08:15:14AM -0700, Bin Meng wrote:

> So far cache operations are only applied on the submission queue and
> completion queue, but they are missing in other places like identify
> and block read/write routines.
> 
> In order to correctly operate on the caches, the DMA buffer passed
> to identify routine must be allocated properly on the stack with the
> existing macro ALLOC_CACHE_ALIGN_BUFFER().
> 
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170828/242b1944/attachment.sig>

[U-Boot] [U-Boot,10/14] nvme: Use blk_create_devicef() API

[Tom Rini]

On Tue, Aug 22, 2017 at 08:15:15AM -0700, Bin Meng wrote:

> The codes in nvme_uclass_post_probe() can be replaced to call the
> blk_create_devicef() API directly.
> 
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170828/134f17ad/attachment.sig>

[U-Boot] [U-Boot, 11/14] nvme: Get rid of the global variable nvme_info

[Tom Rini]

On Tue, Aug 22, 2017 at 08:15:16AM -0700, Bin Meng wrote:

> At present the NVMe uclass driver uses a global variable nvme_info
> to store global information like namespace id, and NVMe controller
> driver's priv struct has a blk_dev_start that is used to calculate
> the namespace id based on the global information from nvme_info.
> 
> This is not a good design in the DM world and can be replaced with
> the following changes:
> 

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170828/84a0da56/attachment.sig>

[U-Boot] [U-Boot, 12/14] nvme: Adjust the 'nvme' command to use blk_common_cmd()

[Tom Rini]

On Tue, Aug 22, 2017 at 08:15:17AM -0700, Bin Meng wrote:

> Instead of having separate code in the 'nvme' command, adjust it to use
> the common function.
> 
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170828/f0e569a1/attachment.sig>

[U-Boot] [U-Boot, 13/14] sandbox: Add a dummy invalidate_dcache_range() function

[Tom Rini]

On Tue, Aug 22, 2017 at 08:15:18AM -0700, Bin Meng wrote:

> This adds invalidate_dcache_range() so that some drivers can build
> without error on sandbox.
> 
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170828/426734bd/attachment.sig>

[U-Boot] [U-Boot, 14/14] sandbox: Enable NVMe driver for build testing

[Tom Rini]

On Tue, Aug 22, 2017 at 08:15:19AM -0700, Bin Meng wrote:

> This enables NVMe driver on sandbox for build testing.
> 
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170828/fe0f825d/attachment.sig>