RE: [RFC PATCH 1/3] block: add copy offload support

["Knight, Frederick" <Frederick.Knight@netapp.com>]

Just FYI about copy domains.

SCSI already has copy domains defined (it is optional in the standard).  Third-party Copy Descriptor - Copy Group Identifier. Here's the SCSI text:

A Copy Group is a set of logical units that have a high probability of using high performance methods (e.g.,
copy on write snapshot) for third-party copy operations involving logical units that are in the same Copy
Group. Each logical unit may be a member of zero or one Copy Group. A logical unit indicates membership in
a Copy Group using the Copy Group Identifier third-party copy descriptor (see 7.7.18.13).

If a third-party copy operation involves logical units that are in different Copy Groups, then that third-party copy
operation has a high probability of using low performance methods (e.g., copy manager read operations from
the source CSCD or copy manager write operations to the destination CSCD).

NVMe today can only copy between LBAs on the SAME Namespace.  There is a new project just getting started to allow copy across namespaces in the same NVM subsystem (included as part of that project is to define how the copy domains will work).  So for NVMe, copy domains are still a work in progress.

Just FYI.

	Fred

-----Original Message-----
From: Keith Busch <kbusch@kernel.org> 
Sent: Wednesday, February 2, 2022 11:22 AM
To: Mikulas Patocka <mpatocka@redhat.com>
Cc: Javier González <javier@javigon.com>; Chaitanya Kulkarni <chaitanyak@nvidia.com>; linux-block@vger.kernel.org; linux-scsi@vger.kernel.org; dm-devel@redhat.com; linux-nvme@lists.infradead.org; linux-fsdevel <linux-fsdevel@vger.kernel.org>; Jens Axboe <axboe@kernel.dk>; msnitzer@redhat.com >> msnitzer@redhat.com <msnitzer@redhat.com>; Bart Van Assche <bvanassche@acm.org>; martin.petersen@oracle.com >> Martin K. Petersen <martin.petersen@oracle.com>; roland@purestorage.com; Hannes Reinecke <hare@suse.de>; Christoph Hellwig <hch@lst.de>; Knight, Frederick <Frederick.Knight@netapp.com>; zach.brown@ni.com; osandov@fb.com; lsf-pc@lists.linux-foundation.org; djwong@kernel.org; josef@toxicpanda.com; clm@fb.com; dsterba@suse.com; tytso@mit.edu; jack@suse.com; Kanchan Joshi <joshi.k@samsung.com>
Subject: Re: [RFC PATCH 1/3] block: add copy offload support

NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.




On Tue, Feb 01, 2022 at 01:32:29PM -0500, Mikulas Patocka wrote:
> +int blkdev_issue_copy(struct block_device *bdev1, sector_t sector1,
> +                   struct block_device *bdev2, sector_t sector2,
> +                   sector_t nr_sects, sector_t *copied, gfp_t 
> +gfp_mask) {
> +     struct page *token;
> +     sector_t m;
> +     int r = 0;
> +     struct completion comp;
> +
> +     *copied = 0;
> +
> +     m = min(bdev_max_copy_sectors(bdev1), bdev_max_copy_sectors(bdev2));
> +     if (!m)
> +             return -EOPNOTSUPP;
> +     m = min(m, (sector_t)round_down(UINT_MAX, PAGE_SIZE) >> 9);
> +
> +     if (unlikely(bdev_read_only(bdev2)))
> +             return -EPERM;
> +
> +     token = alloc_page(gfp_mask);
> +     if (unlikely(!token))
> +             return -ENOMEM;
> +
> +     while (nr_sects) {
> +             struct bio *read_bio, *write_bio;
> +             sector_t this_step = min(nr_sects, m);
> +
> +             read_bio = bio_alloc(gfp_mask, 1);
> +             if (unlikely(!read_bio)) {
> +                     r = -ENOMEM;
> +                     break;
> +             }
> +             bio_set_op_attrs(read_bio, REQ_OP_COPY_READ_TOKEN, REQ_NOMERGE);
> +             bio_set_dev(read_bio, bdev1);
> +             __bio_add_page(read_bio, token, PAGE_SIZE, 0);

You have this "token" payload as driver specific data, but there's no check that bdev1 and bdev2 subscribe to the same driver specific format.

I thought we discussed defining something like a "copy domain" that establishes which block devices can offload copy operations to/from each other, and that should be checked before proceeding with the copy operation.