Linux-NVME Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH 00/15 V6] nvme-rdma/nvmet-rdma: Add metadata/T10-PI support
@ 2020-04-28 13:11 Max Gurtovoy
  2020-04-28 13:11 ` [PATCH 01/15] nvme: introduce namespace features flag Max Gurtovoy
                   ` (14 more replies)
  0 siblings, 15 replies; 43+ messages in thread
From: Max Gurtovoy @ 2020-04-28 13:11 UTC (permalink / raw)
  To: maxg, linux-nvme, kbusch, hch, sagi, martin.petersen, jsmart2021, axboe
  Cc: vladimirk, shlomin, israelr, idanb, oren, nitzanc

Hello Sagi, Christoph, Keith, Martin, James and Co

This patchset adds metadata (T10-PI) support for NVMeoF/RDMA host side
and target side, using signature verbs API. This set starts with a few
preparation commits to the NVMe host core layer. It continues with
NVMeoF/RDMA host implementation and preparations to NVMe target core
layer. The patchset ends with NVMeoF/RDMA target implementation.

In V6 I mainly did some updates for host side configuration and removed
the NVMe cli patch. For the initial mode, we'll use T10-PI/metadata for
capable devices and act according to target controller and namespaces
configuration. I also used James's suggestion of making nvme_ns_has_pi
accessible and implemented Christoph's suggestions for ease the code in
NVMe host core. Code looks more simple and readable now and hopefully we
can continue with the review and the merge it soon.

Configuration:
Host:
 - same as before

Target:
 - echo 1 > /config/nvmet/subsystems/${NAME}/attr_pi_enable
 - echo 1 > /config/nvmet/ports/${PORT_NUM}/param_pi_enable

The code was tested using Mellanox's ConnectX-4/ConnectX-5 HCAs.
This series applies on top of nvme_5.8 branch cleanly.

Changes from v5:
 - rebased over nvme_5.8
 - configure RDMA controllers for T10-PI by default (if capable)
 - removed RW api patch for exporting single MR length (will add it in the future if needed)
 - Implemented Christoph's suggestions for NVMe host core
 - fail enabling target port in case of PI capability mismatch (from Christoph)
 
Changes from v4:
 - removed get_mdts patches (merged)
 - added enum nvme_ns_features instead of defines (patch 1/17)
 - rename pi/prot to md (patches 2/17 + 6/17 + 8/17 + 9/17 + 10/17)
 - another rebase

Changes from v3:
 - Added Reviewed-by signatures
 - New RDMA/rw patch (Patch 17/19)
 - Add mdts setting op for controllers (Patches 14/19, 18/19)
 - Rename NVME_NS_DIX_SUPPORTED to NVME_NS_MD_HOST_SUPPORTED and
   NVME_NS_DIF_SUPPORTED to NVME_NS_MD_CTRL_SUPPORTED (Patch 01/19)
 - Split "nvme: Introduce namespace features flag" patch (patch 02/19)
 - Rename nvmet_rdma_set_diff_domain to nvmet_rdma_set_sig_domain
   and nvme_rdma_set_diff_domain to nvme_rdma_set_sig_domain
   (Patches 08/19, 19/19)
 - Remove ns parameter from nvme_rdma_set_sig_domain/nvmet_rdma_set_sig_domain
   functions (patch 08/19, 19/19)
 - Rebase over nvme-5.7 branch

Changes from v2:
 - Convert the virtual start sector (which passed to bip_set_seed function)
   to be in integrity interval units (Patch 14/15)
 - Clarify some commit messages

Changes from v1:
 - Added Reviewed-by signatures
 - Added namespace features flag (Patch 01/15)
 - Remove nvme_ns_has_pi function (Patch 01/15)
 - Added has_pi field to struct nvme_request (Patch 01/15)
 - Subject change for patch 02/15
 - Fix comment for PCI metadata (Patch 03/15)
 - Rebase over "nvme: Avoid preallocating big SGL for data" patchset
 - Introduce NVME_INLINE_PROT_SG_CNT flag (Patch 05/15)
 - Introduce nvme_rdma_sgl structure (Patch 06/15)
 - Remove first_sgl pointer from struct nvme_rdma_request (Patch 06/15)
 - Split nvme-rdma patches (Patches 06/15, 07/15)
 - Rename is_protected to use_pi (Patch 07/15)
 - Refactor nvme_rdma_get_max_fr_pages function (Patch 07/15)
 - Added ifdef CONFIG_BLK_DEV_INTEGRITY (Patches 07/15, 09/15, 13/15,
   14/15, 15/15)
 - Added port configfs pi_enable (Patch 14/15)

Israel Rukshin (9):
  nvme: introduce NVME_INLINE_MD_SG_CNT
  nvme-rdma: introduce nvme_rdma_sgl structure
  nvmet: add metadata characteristics for a namespace
  nvmet: rename nvmet_rw_len to nvmet_rw_data_len
  nvmet: rename nvmet_check_data_len to nvmet_check_transfer_len
  nvme: add Metadata Capabilities enumerations
  nvmet: add metadata/T10-PI support
  nvmet: add metadata support for block devices
  nvmet-rdma: add metadata/T10-PI support

James Smart (1):
  nvme: make nvme_ns_has_pi accessible to transports

Max Gurtovoy (5):
  nvme: introduce namespace features flag
  nvme: introduce NVME_NS_METADATA_SUPPORTED flag
  nvme: enforce extended LBA format for fabrics metadata
  nvme: introduce max_integrity_segments ctrl attribute
  nvme-rdma: add metadata/T10-PI support

 drivers/nvme/host/core.c          |  83 ++++++---
 drivers/nvme/host/nvme.h          |  18 +-
 drivers/nvme/host/pci.c           |   6 +
 drivers/nvme/host/rdma.c          | 358 ++++++++++++++++++++++++++++++++------
 drivers/nvme/target/admin-cmd.c   |  40 +++--
 drivers/nvme/target/configfs.c    |  61 +++++++
 drivers/nvme/target/core.c        | 115 +++++++++---
 drivers/nvme/target/discovery.c   |   8 +-
 drivers/nvme/target/fabrics-cmd.c |  16 +-
 drivers/nvme/target/io-cmd-bdev.c | 116 +++++++++++-
 drivers/nvme/target/io-cmd-file.c |   6 +-
 drivers/nvme/target/nvmet.h       |  27 ++-
 drivers/nvme/target/rdma.c        | 234 +++++++++++++++++++++++--
 include/linux/nvme.h              |   6 +
 14 files changed, 947 insertions(+), 147 deletions(-)

-- 
1.8.3.1


_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply	[flat|nested] 43+ messages in thread
* [PATCH 00/15 V3] nvme-rdma/nvmet-rdma: Add metadata/T10-PI support
@ 2020-01-06 13:37 Max Gurtovoy
  2020-01-06 13:37 ` [PATCH 15/15] nvmet-rdma: " Max Gurtovoy
  0 siblings, 1 reply; 43+ messages in thread
From: Max Gurtovoy @ 2020-01-06 13:37 UTC (permalink / raw)
  To: linux-nvme, kbusch, hch, sagi, martin.petersen
  Cc: axboe, vladimirk, shlomin, israelr, idanb, oren, maxg

Hello Sagi, Christoph, Keith, Martin and Co

This patchset adds metadata (T10-PI) support for NVMeoF/RDMA host side and
target side, using signature verbs API. This set starts with a few preparation
commits to the NVMe host core layer. It continues with NVMeoF/RDMA host
implementation + few preparation commits to the NVMe target core layer.
The patchset ends with NVMeoF/RDMA target implementation.

Configuration:
Host:
 - nvme connect --pi_enable --transport=rdma --traddr=10.0.1.1 --nqn=test-nvme

Target:
 - echo 1 > /config/nvmet/subsystems/${NAME}/attr_pi_enable
 - echo 1 > /config/nvmet/ports/${PORT_NUM}/param_pi_enable

The code was tested using Mellanox's ConnectX-4/ConnectX-5 HCAs.
This series applies on top of nvme_5.5 branch cleanly.

Changes from v2:
 - Convert the virtual start sector (which passed to bip_set_seed function)
   to be in integrity interval units (Patch 14/15)
 - Clarify some commit messages

Changes from v1:
 - Added Reviewed-by signatures
 - Added namespace features flag (Patch 01/15)
 - Remove nvme_ns_has_pi function (Patch 01/15)
 - Added has_pi field to struct nvme_request (Patch 01/15)
 - Subject change for patch 02/15
 - Fix comment for PCI metadata (Patch 03/15)
 - Rebase over "nvme: Avoid preallocating big SGL for data" patchset
 - Introduce NVME_INLINE_PROT_SG_CNT flag (Patch 05/15)
 - Introduce nvme_rdma_sgl structure (Patch 06/15)
 - Remove first_sgl pointer from struct nvme_rdma_request (Patch 06/15)
 - Split nvme-rdma patches (Patches 06/15, 07/15)
 - Rename is_protected to use_pi (Patch 07/15)
 - Refactor nvme_rdma_get_max_fr_pages function (Patch 07/15)
 - Added ifdef CONFIG_BLK_DEV_INTEGRITY (Patches 07/15, 09/15, 13/15,
   14/15, 15/15)
 - Added port configfs pi_enable (Patch 14/15)

Israel Rukshin (12):
  nvme: Introduce namespace features flag
  nvme-fabrics: Allow user enabling metadata/T10-PI support
  nvme: Introduce NVME_INLINE_PROT_SG_CNT
  nvme-rdma: Introduce nvme_rdma_sgl structure
  nvmet: Prepare metadata request
  nvmet: Add metadata characteristics for a namespace
  nvmet: Rename nvmet_rw_len to nvmet_rw_data_len
  nvmet: Rename nvmet_check_data_len to nvmet_check_transfer_len
  nvme: Add Metadata Capabilities enumerations
  nvmet: Add metadata/T10-PI support
  nvmet: Add metadata support for block devices
  nvmet-rdma: Add metadata/T10-PI support

Max Gurtovoy (3):
  nvme: Enforce extended LBA format for fabrics metadata
  nvme: Introduce max_integrity_segments ctrl attribute
  nvme-rdma: Add metadata/T10-PI support

 drivers/nvme/host/core.c          |  75 +++++---
 drivers/nvme/host/fabrics.c       |  11 ++
 drivers/nvme/host/fabrics.h       |   3 +
 drivers/nvme/host/nvme.h          |   9 +-
 drivers/nvme/host/pci.c           |   7 +
 drivers/nvme/host/rdma.c          | 368 +++++++++++++++++++++++++++++++++-----
 drivers/nvme/target/admin-cmd.c   |  37 ++--
 drivers/nvme/target/configfs.c    |  61 +++++++
 drivers/nvme/target/core.c        |  54 ++++--
 drivers/nvme/target/discovery.c   |   8 +-
 drivers/nvme/target/fabrics-cmd.c |  19 +-
 drivers/nvme/target/io-cmd-bdev.c | 115 +++++++++++-
 drivers/nvme/target/io-cmd-file.c |   8 +-
 drivers/nvme/target/nvmet.h       |  40 ++++-
 drivers/nvme/target/rdma.c        | 235 ++++++++++++++++++++++--
 include/linux/nvme.h              |   2 +
 16 files changed, 925 insertions(+), 127 deletions(-)

-- 
1.8.3.1


_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply	[flat|nested] 43+ messages in thread
* [PATCH 00/15] nvme-rdma/nvmet-rdma: Add metadata/T10-PI support
@ 2019-11-05 16:20 Max Gurtovoy
  2019-11-05 16:20 ` [PATCH 15/15] nvmet-rdma: " Max Gurtovoy
  0 siblings, 1 reply; 43+ messages in thread
From: Max Gurtovoy @ 2019-11-05 16:20 UTC (permalink / raw)
  To: linux-nvme, kbusch, hch, sagi
  Cc: vladimirk, idanb, israelr, shlomin, oren, maxg

Hello Sagi, Christoph, Keith, Jason and Co

This patchset adds metadata (T10-PI) support for NVMeoF/RDMA host side and
target side, using signature verbs API. This set starts with a few preparation
commits to the NVMe host core layer. It continues with NVMeoF/RDMA host
implementation + new bip flag patch + few preparation commits to the NVMe
target core layer. The patchset ends with NVMeoF/RDMA target implementation.

Configuration:
Host:
 - nvme connect --pi_enable --transport=rdma --traddr=10.0.1.1 --nqn=test-nvme

Target:
 - modprobe nvmet-rdma pi_enable=Y
 - echo 1 > /config/nvmet/subsystems/${NAME}/attr_pi_enable

The code was tested using Mellanox's ConnectX-4/ConnectX-5 HCA.
This series applies on top of nvme_5.5 branch cleanly.

Israel Rukshin (11):
  nvme-fabrics: allow user enabling metadata/T10-PI support
  block: Introduce BIP_NOMAP_INTEGRITY bip_flag
  nvmet: Prepare metadata request
  nvmet: Add metadata characteristics for a namespace
  nvmet: Rename nvmet_rw_len to nvmet_rw_data_len
  nvmet: Rename nvmet_check_data_len to nvmet_check_transfer_len
  nvmet: Introduce nvmet_rw_prot_len and nvmet_ns_has_pi
  nvme: Add Metadata Capabilities enumerations
  nvmet: Add metadata/T10-PI support
  nvmet: Add metadata support for block devices
  nvmet-rdma: Add metadata/T10-PI support

Max Gurtovoy (4):
  nvme: Fail __nvme_revalidate_disk in case of a spec violation
  nvme: Introduce max_integrity_segments ctrl attribute
  nvme: Inline nvme_ns_has_pi function
  nvme-rdma: Add metadata/T10-PI support

 block/t10-pi.c                    |   7 +
 drivers/nvme/host/core.c          |  55 +++---
 drivers/nvme/host/fabrics.c       |   5 +
 drivers/nvme/host/fabrics.h       |   3 +
 drivers/nvme/host/nvme.h          |   7 +
 drivers/nvme/host/pci.c           |   7 +
 drivers/nvme/host/rdma.c          | 346 ++++++++++++++++++++++++++++++++------
 drivers/nvme/target/admin-cmd.c   |  37 ++--
 drivers/nvme/target/configfs.c    |  24 +++
 drivers/nvme/target/core.c        |  54 ++++--
 drivers/nvme/target/discovery.c   |   8 +-
 drivers/nvme/target/fabrics-cmd.c |  19 ++-
 drivers/nvme/target/io-cmd-bdev.c | 105 +++++++++++-
 drivers/nvme/target/io-cmd-file.c |   8 +-
 drivers/nvme/target/nvmet.h       |  27 ++-
 drivers/nvme/target/rdma.c        | 221 ++++++++++++++++++++++--
 include/linux/bio.h               |   1 +
 include/linux/nvme.h              |   2 +
 18 files changed, 811 insertions(+), 125 deletions(-)

-- 
1.8.3.1


_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply	[flat|nested] 43+ messages in thread

end of thread, back to index

Thread overview: 43+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-04-28 13:11 [PATCH 00/15 V6] nvme-rdma/nvmet-rdma: Add metadata/T10-PI support Max Gurtovoy
2020-04-28 13:11 ` [PATCH 01/15] nvme: introduce namespace features flag Max Gurtovoy
2020-05-01 13:20   ` Christoph Hellwig
2020-05-01 14:24   ` Christoph Hellwig
2020-05-01 14:33     ` Max Gurtovoy
2020-04-28 13:11 ` [PATCH 02/15] nvme: introduce NVME_NS_METADATA_SUPPORTED flag Max Gurtovoy
2020-05-01 13:20   ` Christoph Hellwig
2020-04-28 13:11 ` [PATCH 03/15] nvme: make nvme_ns_has_pi accessible to transports Max Gurtovoy
2020-05-01 13:20   ` Christoph Hellwig
2020-04-28 13:11 ` [PATCH 04/15] nvme: enforce extended LBA format for fabrics metadata Max Gurtovoy
2020-05-01 13:21   ` Christoph Hellwig
2020-05-01 13:41   ` Christoph Hellwig
2020-04-28 13:11 ` [PATCH 05/15] nvme: introduce max_integrity_segments ctrl attribute Max Gurtovoy
2020-04-28 13:11 ` [PATCH 06/15] nvme: introduce NVME_INLINE_MD_SG_CNT Max Gurtovoy
2020-04-28 13:11 ` [PATCH 07/15] nvme-rdma: introduce nvme_rdma_sgl structure Max Gurtovoy
2020-04-28 13:11 ` [PATCH 08/15] nvme-rdma: add metadata/T10-PI support Max Gurtovoy
2020-05-01 14:26   ` Christoph Hellwig
2020-05-01 15:00     ` Max Gurtovoy
2020-04-28 13:11 ` [PATCH 09/15] nvmet: add metadata characteristics for a namespace Max Gurtovoy
2020-05-01 15:50   ` Christoph Hellwig
2020-04-28 13:11 ` [PATCH 10/15] nvmet: rename nvmet_rw_len to nvmet_rw_data_len Max Gurtovoy
2020-04-28 13:11 ` [PATCH 11/15] nvmet: rename nvmet_check_data_len to nvmet_check_transfer_len Max Gurtovoy
2020-04-28 13:11 ` [PATCH 12/15] nvme: add Metadata Capabilities enumerations Max Gurtovoy
2020-05-01 15:53   ` Christoph Hellwig
2020-05-03 12:43     ` Max Gurtovoy
2020-04-28 13:11 ` [PATCH 13/15] nvmet: add metadata/T10-PI support Max Gurtovoy
2020-05-01 15:58   ` Christoph Hellwig
2020-05-01 16:19   ` Christoph Hellwig
2020-04-28 13:11 ` [PATCH 14/15] nvmet: add metadata support for block devices Max Gurtovoy
2020-05-01 16:25   ` Christoph Hellwig
2020-04-28 13:11 ` [PATCH 15/15] nvmet-rdma: add metadata/T10-PI support Max Gurtovoy
2020-05-01 16:48   ` Christoph Hellwig
2020-05-03 16:29     ` Max Gurtovoy
2020-05-04 14:08       ` Christoph Hellwig
2020-05-04 14:19         ` Max Gurtovoy
  -- strict thread matches above, loose matches on Subject: below --
2020-01-06 13:37 [PATCH 00/15 V3] nvme-rdma/nvmet-rdma: Add " Max Gurtovoy
2020-01-06 13:37 ` [PATCH 15/15] nvmet-rdma: " Max Gurtovoy
2020-01-09  3:29   ` Martin K. Petersen
2019-11-05 16:20 [PATCH 00/15] nvme-rdma/nvmet-rdma: " Max Gurtovoy
2019-11-05 16:20 ` [PATCH 15/15] nvmet-rdma: " Max Gurtovoy
2019-11-05 18:02   ` Christoph Hellwig
2019-11-07 13:43     ` Max Gurtovoy
2019-11-12 18:34   ` Sagi Grimberg
2019-11-13 13:56     ` Max Gurtovoy
2019-11-14 23:45       ` Sagi Grimberg

Linux-NVME Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-nvme/0 linux-nvme/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-nvme linux-nvme/ https://lore.kernel.org/linux-nvme \
		linux-nvme@lists.infradead.org
	public-inbox-index linux-nvme

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.infradead.lists.linux-nvme


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git