Re: [dm-devel] [RFC PATCH 0/2] add simple copy support

Re: [dm-devel] [RFC PATCH 0/2] add simple copy support

[Damien Le Moal]

+ Mike and DM list

On 2020/12/01 16:12, SelvaKumar S wrote:
> This patchset tries to add support for TP4065a ("Simple Copy Command"),
> v2020.05.04 ("Ratified")
> 
> The Specification can be found in following link.
> https://nvmexpress.org/wp-content/uploads/NVM-Express-1.4-Ratified-TPs-1.zip
> 
> This is an RFC. Looking forward for any feedbacks or other alternate
> designs for plumbing simple copy to IO stack.
> 
> Simple copy command is a copy offloading operation and is  used to copy
> multiple contiguous ranges (source_ranges) of LBA's to a single destination
> LBA within the device reducing traffic between host and device.
> 
> This implementation accepts destination, no of sources and arrays of
> source ranges from application and attach it as payload to the bio and
> submits to the device.
> 
> Following limits are added to queue limits and are exposed in sysfs
> to userspace
> 	- *max_copy_sectors* limits the sum of all source_range length
> 	- *max_copy_nr_ranges* limits the number of source ranges
> 	- *max_copy_range_sectors* limit the maximum number of sectors
> 		that can constitute a single source range.

This is interesting. I think there are several possible use in the kernel in
various components: FS (btrfs rebalance, f2fs GC, liklely others) and DM at the
very least.

However, your patches add support only for NVMe devices that have native support
for simple copy, leaving all other block devices out. That seriously limits the
use cases and also does not make this solution attractive since any use of it
would need to be conditional on the underlying drive capabilities. That means
more code for the file systems or device mapper developers and maintainers, not
less.

To avoid this, I would suggest that this code be extended to add emulation for
drives that do not implement simple copy natively. This would allow this
interface to work on any block device, including SAS & SATA HDDs and RAID arrays.

The emulation part of this copy service could I think be based on dm-kcopyd. See
include/linux/dm-kcopyd.h for the interface. The current dm-kcopyd interface
takes one source and multiple destination, the reverse of simple copy. But it
would be fairly straightforward to also allow multiple sources and one
destination. Simple copy native support would accelerate this case, everything
else using the regular BIO read+write interface. Moving dm-kcopyd from DM
infrastructure into the block layer as a set a generic block device sector copy
service would allow its use in more places. And SCSI XCOPY could also be
integrated in there as a different drive native support command.

> 
> 
> SelvaKumar S (2):
>   block: add simple copy support
>   nvme: add simple copy support
> 
>  block/blk-core.c          | 104 +++++++++++++++++++++++++++++++---
>  block/blk-lib.c           | 116 ++++++++++++++++++++++++++++++++++++++
>  block/blk-merge.c         |   2 +
>  block/blk-settings.c      |  11 ++++
>  block/blk-sysfs.c         |  23 ++++++++
>  block/blk-zoned.c         |   1 +
>  block/bounce.c            |   1 +
>  block/ioctl.c             |  43 ++++++++++++++
>  drivers/nvme/host/core.c  |  91 ++++++++++++++++++++++++++++++
>  drivers/nvme/host/nvme.h  |   4 ++
>  include/linux/bio.h       |   1 +
>  include/linux/blk_types.h |   7 +++
>  include/linux/blkdev.h    |  15 +++++
>  include/linux/nvme.h      |  45 +++++++++++++--
>  include/uapi/linux/fs.h   |  21 +++++++
>  15 files changed, 473 insertions(+), 12 deletions(-)
> 


-- 
Damien Le Moal
Western Digital Research



--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

Re: [dm-devel] [RFC PATCH 0/2] add simple copy support

[javier.gonz@samsung.com]

On 01.12.2020 11:22, Damien Le Moal wrote:
>+ Mike and DM list
>
>On 2020/12/01 16:12, SelvaKumar S wrote:
>> This patchset tries to add support for TP4065a ("Simple Copy Command"),
>> v2020.05.04 ("Ratified")
>>
>> The Specification can be found in following link.
>> https://nvmexpress.org/wp-content/uploads/NVM-Express-1.4-Ratified-TPs-1.zip
>>
>> This is an RFC. Looking forward for any feedbacks or other alternate
>> designs for plumbing simple copy to IO stack.
>>
>> Simple copy command is a copy offloading operation and is  used to copy
>> multiple contiguous ranges (source_ranges) of LBA's to a single destination
>> LBA within the device reducing traffic between host and device.
>>
>> This implementation accepts destination, no of sources and arrays of
>> source ranges from application and attach it as payload to the bio and
>> submits to the device.
>>
>> Following limits are added to queue limits and are exposed in sysfs
>> to userspace
>> 	- *max_copy_sectors* limits the sum of all source_range length
>> 	- *max_copy_nr_ranges* limits the number of source ranges
>> 	- *max_copy_range_sectors* limit the maximum number of sectors
>> 		that can constitute a single source range.
>
>This is interesting. I think there are several possible use in the kernel in
>various components: FS (btrfs rebalance, f2fs GC, liklely others) and DM at the
>very least.

Totally agree. We have more patches for simple copy, among others work
on F2FS that leverages for GC. We wanted to start with a simple patchset
enabling IOCTL to be an easy to review start. The rest of the patches
will come.


>However, your patches add support only for NVMe devices that have native support
>for simple copy, leaving all other block devices out. That seriously limits the
>use cases and also does not make this solution attractive since any use of it
>would need to be conditional on the underlying drive capabilities. That means
>more code for the file systems or device mapper developers and maintainers, not
>less.

Makes sense.

>
>To avoid this, I would suggest that this code be extended to add emulation for
>drives that do not implement simple copy natively. This would allow this
>interface to work on any block device, including SAS & SATA HDDs and RAID arrays.
>
>The emulation part of this copy service could I think be based on dm-kcopyd. See
>include/linux/dm-kcopyd.h for the interface. The current dm-kcopyd interface
>takes one source and multiple destination, the reverse of simple copy. But it
>would be fairly straightforward to also allow multiple sources and one
>destination. Simple copy native support would accelerate this case, everything
>else using the regular BIO read+write interface. Moving dm-kcopyd from DM
>infrastructure into the block layer as a set a generic block device sector copy
>service would allow its use in more places. And SCSI XCOPY could also be
>integrated in there as a different drive native support command.

Let us look into this. It makes sense.

It seems fair to start support bottom up from NVMe and then extend to
block layer and F2FS (potentially others). At this stage, we were
assuming that all the work that people like Martin, Bart and other have
been doing through the years on XCOPY could be integrated. Emulation can
be a part of this.

How can we move forward to do this in stages?

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel