RE: [PATCH V2 0/2] nvme: Support for fused NVME_IOCTL_SUBMIT_IO

From: Clay Mayers <Clay.Mayers@kioxia.com>
To: Keith Busch <kbusch@kernel.org>
Cc: Jens Axboe <axboe@fb.com>, Sagi Grimberg <sagi@grimberg.me>,
	Chaitanya Kulkarni <Chaitanya.Kulkarni@wdc.com>,
	"linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>,
	Christoph Hellwig <hch@lst.de>
Subject: RE: [PATCH V2 0/2] nvme: Support for fused NVME_IOCTL_SUBMIT_IO
Date: Tue, 9 Feb 2021 15:38:53 +0000	[thread overview]
Message-ID: <16731baa6d8f4729a10edeea57267c41@kioxia.com> (raw)
In-Reply-To: <20210209031252.GA97526@C02WT3WMHTD6>

> From: Keith Busch <kbusch@kernel.org>
> Sent: Monday, February 8, 2021 7:13 PM
> To: Clay Mayers <Clay.Mayers@kioxia.com>
> Cc: linux-nvme@lists.infradead.org; Jens Axboe <axboe@fb.com>; Chaitanya
> Kulkarni <Chaitanya.Kulkarni@wdc.com>; Christoph Hellwig <hch@lst.de>;
> Sagi Grimberg <sagi@grimberg.me>
> Subject: Re: [PATCH V2 0/2] nvme: Support for fused
> NVME_IOCTL_SUBMIT_IO
> 
> On Tue, Feb 09, 2021 at 12:53:17AM +0000, Clay Mayers wrote:
> > Is there any other feedback on V2?
> >
> > My main concern I have about my implementation is how fused requests
> > are tunneled through the mq request layer.  The 1st request is marked
> > as started but it won't be in the device until the 2nd command is
> > queued.  As Keith pointed out, a device reset can split the two so
> > care must be taken to correctly handle this case.  Despite this, I
> > thought this was a better approach than modifying mq requests to be
> > fused.  Especially given Christoph's concern of cost vs value.  This is the
> lightest touch I could come up with.
> >
> > Further consideration of this patch may need a more compelling use case.
> > I've worked on a proprietary storage systems that relied on fused
> > NVMeOF support so it seems compelling to me.  There's a comment in
> > target/core.c that there is "no support for fused commands yet"
> > implying it's been considered.  Is pci only support for fused too soon
> > or too little?  What would make it more compelling?
> 
> The complications it introduces to the IO path and error handling for an
> archaic feature has me on the "Nak" side. NVMeOF was introduced well after
> the spec define Reservations, and the kernel has supported that capability
> for many years. I'm not aware of any other use case for fused commands, so
> it appears to be dead weight in the spec.

Thanks for sharing your perspective.  I can see the point that reservations are
good enough for many.  For VmWare, moving from reservations to cmp/write
on SCSI led to an increase in performance (couldn't find an actual # but heard
30% anecdotally). That's a specialized use case (storage for VMs) on a much
slower medium that may not translate to more general storage systems or
much faster storage.

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme