From: Bart Van Assche <bvanassche@acm.org>
To: Kanchan Joshi <joshiiitr@gmail.com>
Cc: Mike Snitzer <snitzer@redhat.com>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
djwong@kernel.org, linux-nvme@lists.infradead.org,
dm-devel@redhat.com, Christoph Hellwig <hch@lst.de>,
agk@redhat.com, linux-scsi@vger.kernel.org,
nitheshshetty@gmail.com, Matthew Wilcox <willy@infradead.org>,
Nitesh Shetty <nj.shetty@samsung.com>,
kch@kernel.org, SelvaKumar S <selvakuma.s1@samsung.com>,
Selva Jove <selvajove@gmail.com>,
linux-block@vger.kernel.org, mpatocka@redhat.com,
Keith Busch <kbusch@kernel.org>, Jens Axboe <axboe@kernel.dk>,
Damien Le Moal <damien.lemoal@wdc.com>,
"Martin K. Petersen" <martin.petersen@oracle.com>,
KANCHAN JOSHI <joshi.k@samsung.com>,
linux-api@vger.kernel.org,
Johannes Thumshirn <johannes.thumshirn@wdc.com>,
linux-fsdevel@vger.kernel.org,
Javier Gonzalez <javier.gonz@samsung.com>,
Pavel Begunkov <asml.silence@gmail.com>
Subject: Re: [dm-devel] [PATCH 3/7] block: copy offload support infrastructure
Date: Fri, 20 Aug 2021 14:18:29 -0700 [thread overview]
Message-ID: <fb9931ae-de27-820a-1333-f24e020913ff@acm.org> (raw)
In-Reply-To: <CA+1E3rK2ULVajQRkNTZJdwKoqBeGvkfoVYNF=WyK6Net85YkhA@mail.gmail.com>
On 8/20/21 3:39 AM, Kanchan Joshi wrote:
> Bart, Mikulas
>
> On Tue, Aug 17, 2021 at 10:44 PM Bart Van Assche <bvanassche@acm.org> wrote:
>>
>> On 8/17/21 3:14 AM, SelvaKumar S wrote:
>>> Introduce REQ_OP_COPY, a no-merge copy offload operation. Create
>>> bio with control information as payload and submit to the device.
>>> Larger copy operation may be divided if necessary by looking at device
>>> limits. REQ_OP_COPY(19) is a write op and takes zone_write_lock when
>>> submitted to zoned device.
>>> Native copy offload is not supported for stacked devices.
>>
>> Using a single operation for copy-offloading instead of separate
>> operations for reading and writing is fundamentally incompatible with
>> the device mapper. I think we need a copy-offloading implementation that
>> is compatible with the device mapper.
>>
>
> While each read/write command is for a single contiguous range of
> device, with simple-copy we get to operate on multiple discontiguous
> ranges, with a single command.
> That seemed like a good opportunity to reduce control-plane traffic
> (compared to read/write operations) as well.
>
> With a separate read-and-write bio approach, each source-range will
> spawn at least one read, one write and eventually one SCC command. And
> it only gets worse as there could be many such discontiguous ranges (for
> GC use-case at least) coming from user-space in a single payload.
> Overall sequence will be
> - Receive a payload from user-space
> - Disassemble into many read-write pair bios at block-layer
> - Assemble those (somehow) in NVMe to reduce simple-copy commands
> - Send commands to device
>
> We thought payload could be a good way to reduce the
> disassembly/assembly work and traffic between block-layer to nvme.
> How do you see this tradeoff? What seems necessary for device-mapper
> usecase, appears to be a cost when device-mapper isn't used.
> Especially for SCC (since copy is within single ns), device-mappers
> may not be too compelling anyway.
>
> Must device-mapper support be a requirement for the initial support atop SCC?
> Or do you think it will still be a progress if we finalize the
> user-space interface to cover all that is foreseeable.And for
> device-mapper compatible transport between block-layer and NVMe - we
> do it in the later stage when NVMe too comes up with better copy
> capabilities?
Hi Kanchan,
These days there might be more systems that run the device mapper on top
of the NVMe driver or a SCSI driver than systems that do use the device
mapper. It is common practice these days to use dm-crypt on personal
workstations and laptops. LVM (dm-linear) is popular because it is more
flexible than a traditional partition table. Android phones use
dm-verity on top of hardware encryption. In other words, not supporting
the device mapper means that a very large number of use cases is
excluded. So I think supporting the device mapper from the start is
important, even if that means combining individual bios at the bottom of
the storage stack into simple copy commands.
Thanks,
Bart.
--
dm-devel mailing list
dm-devel@redhat.com
https://listman.redhat.com/mailman/listinfo/dm-devel
next prev parent reply other threads:[~2021-08-20 21:19 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CGME20210817101741epcas5p174ca0a539587da6a67b9f58cd13f2bad@epcas5p1.samsung.com>
2021-08-17 10:14 ` [dm-devel] [PATCH 0/7] add simple copy support SelvaKumar S
[not found] ` <CGME20210817101747epcas5p1242e63ec29b127b03b6f9f5f1b57f86e@epcas5p1.samsung.com>
2021-08-17 10:14 ` [dm-devel] [PATCH 1/7] block: make bio_map_kern() non static SelvaKumar S
[not found] ` <CGME20210817101753epcas5p4f4257f8edda27e184ecbb273b700ccbc@epcas5p4.samsung.com>
2021-08-17 10:14 ` [dm-devel] [PATCH 2/7] block: Introduce queue limits for copy-offload support SelvaKumar S
2021-08-17 13:08 ` Greg KH
2021-08-17 14:42 ` Nitesh Shetty
[not found] ` <CGME20210817101758epcas5p1ec353b3838d64654e69488229256d9eb@epcas5p1.samsung.com>
2021-08-17 10:14 ` [dm-devel] [PATCH 3/7] block: copy offload support infrastructure SelvaKumar S
2021-08-17 17:14 ` Bart Van Assche
2021-08-17 20:41 ` Mikulas Patocka
2021-08-17 21:53 ` Douglas Gilbert
2021-08-17 22:06 ` Bart Van Assche
2021-08-20 10:39 ` Kanchan Joshi
2021-08-20 21:18 ` Bart Van Assche [this message]
2021-08-26 7:46 ` Nitesh Shetty
2021-08-17 20:35 ` kernel test robot
2021-08-18 18:35 ` Martin K. Petersen
2021-08-20 11:11 ` Kanchan Joshi
[not found] ` <CGME20210817101803epcas5p10cda1d52f8a8f1172e34b1f9cf8eef3b@epcas5p1.samsung.com>
2021-08-17 10:14 ` [dm-devel] [PATCH 4/7] block: Introduce a new ioctl for simple copy SelvaKumar S
2021-08-17 13:09 ` Greg KH
2021-08-17 13:10 ` Greg KH
2021-08-17 14:48 ` Nitesh Shetty
2021-08-17 23:36 ` Darrick J. Wong
2021-08-18 15:37 ` Nitesh Shetty
2021-08-18 16:17 ` Darrick J. Wong
[not found] ` <CGME20210817101809epcas5p39eed3531ed82f5f08127eb3dba1fc50f@epcas5p3.samsung.com>
2021-08-17 10:14 ` [dm-devel] [PATCH 5/7] block: add emulation " SelvaKumar S
2021-08-17 22:10 ` kernel test robot
[not found] ` <CGME20210817101814epcas5p41db3d7269f5139efcaf2ca685cd04a16@epcas5p4.samsung.com>
2021-08-17 10:14 ` [dm-devel] [PATCH 6/7] nvme: add simple copy support SelvaKumar S
[not found] ` <CGME20210817101822epcas5p470644cf681d5e8db5367dc7998305c65@epcas5p4.samsung.com>
2021-08-17 10:14 ` [dm-devel] [PATCH 7/7] dm kcopyd: add simple copy offload support SelvaKumar S
2021-08-17 20:29 ` Mikulas Patocka
2021-08-17 23:37 ` [dm-devel] [PATCH 0/7] add simple copy support Darrick J. Wong
2021-08-18 15:40 ` Nitesh Shetty
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=fb9931ae-de27-820a-1333-f24e020913ff@acm.org \
--to=bvanassche@acm.org \
--cc=agk@redhat.com \
--cc=asml.silence@gmail.com \
--cc=axboe@kernel.dk \
--cc=damien.lemoal@wdc.com \
--cc=djwong@kernel.org \
--cc=dm-devel@redhat.com \
--cc=gregkh@linuxfoundation.org \
--cc=hch@lst.de \
--cc=javier.gonz@samsung.com \
--cc=johannes.thumshirn@wdc.com \
--cc=joshi.k@samsung.com \
--cc=joshiiitr@gmail.com \
--cc=kbusch@kernel.org \
--cc=kch@kernel.org \
--cc=linux-api@vger.kernel.org \
--cc=linux-block@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=linux-scsi@vger.kernel.org \
--cc=martin.petersen@oracle.com \
--cc=mpatocka@redhat.com \
--cc=nitheshshetty@gmail.com \
--cc=nj.shetty@samsung.com \
--cc=selvajove@gmail.com \
--cc=selvakuma.s1@samsung.com \
--cc=snitzer@redhat.com \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).