Re: [PATCH 02/15] nvme: Enforce extended LBA format for fabrics metadata

From: James Smart <james.smart@broadcom.com>
To: Max Gurtovoy <maxg@mellanox.com>,
	linux-nvme@lists.infradead.org, kbusch@kernel.org, hch@lst.de,
	sagi@grimberg.me, martin.petersen@oracle.com
Cc: axboe@kernel.dk, vladimirk@mellanox.com, idanb@mellanox.com,
	israelr@mellanox.com, shlomin@mellanox.com, oren@mellanox.com
Subject: Re: [PATCH 02/15] nvme: Enforce extended LBA format for fabrics metadata
Date: Thu, 16 Jan 2020 15:53:28 -0800	[thread overview]
Message-ID: <57aa8b5c-b8c9-7617-bc11-b6797587fdc8@broadcom.com> (raw)
In-Reply-To: <20200106133736.123038-4-maxg@mellanox.com>

On 1/6/2020 5:37 AM, Max Gurtovoy wrote:
> An extended LBA is a larger LBA that is created when metadata associated
> with the LBA is transferred contiguously with the LBA data (AKA
> interleaved). The metadata may be either transferred as part of the LBA
> (creating an extended LBA) or it may be transferred as a separate
> contiguous buffer of data. According to the NVMeoF spec, a fabrics ctrl
> supports only an Extended LBA format. Fail revalidation in case we have a
> spec violation. Also initialize the integrity profile for the block device
> for fabrics ctrl.
>
> Signed-off-by: Max Gurtovoy <maxg@mellanox.com>
> Signed-off-by: Israel Rukshin <israelr@mellanox.com>
> ---
>   drivers/nvme/host/core.c | 25 +++++++++++++++++++++----
>   1 file changed, 21 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index d98eb48..089cdc3c 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -1818,7 +1818,7 @@ static void nvme_update_disk_info(struct gendisk *disk,
>   	blk_mq_unfreeze_queue(disk->queue);
>   }
>   
> -static void __nvme_revalidate_disk(struct gendisk *disk, struct nvme_id_ns *id)
> +static int __nvme_revalidate_disk(struct gendisk *disk, struct nvme_id_ns *id)
>   {
>   	struct nvme_ns *ns = disk->private_data;
>   
> @@ -1846,11 +1846,21 @@ static void __nvme_revalidate_disk(struct gendisk *disk, struct nvme_id_ns *id)
>   			ns->features |= NVME_NS_EXT_LBAS;
>   
>   		/*
> +		 * For Fabrics, only metadata as part of extended data LBA is
> +		 * supported. Fail in case of a spec violation.
> +		 */
> +		if (ns->ctrl->ops->flags & NVME_F_FABRICS) {
> +			if (WARN_ON_ONCE(!(ns->features & NVME_NS_EXT_LBAS)))
> +				return -EINVAL;
> +		}
> +
> +		/*
>   		 * For PCI, Extended logical block will be generated by the
>   		 * controller.
>   		 */
>   		if (ns->ctrl->ops->flags & NVME_F_METADATA_SUPPORTED) {
> -			if (!(ns->features & NVME_NS_EXT_LBAS))
> +			if (ns->ctrl->ops->flags & NVME_F_FABRICS ||
> +			    !(ns->features & NVME_NS_EXT_LBAS))
>   				ns->features |= NVME_NS_DIX_SUPPORTED;

This last change seems odd - why is DIX set if NVME_F_FABRICS ?

Per patch description above, Fabrics spec requires metadata as an 
extended LBA, thus it doesn't support DIX.

Which is touches on a lot of odd things with the nvme spec as it's 
certainly possible for, within the os host implementation, to have the 
host transmitting engine to convert an OS separate DIF buf to an 
extended lba transmission on the wire and as presented to the 
controller.  Transports can certainly help make this happen - and add 
egress checking as the data leaves the host.    Which means - I'm not 
sure this hard DIX definition being implemented this way is the way to go.

-- james

_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme