From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93EC6C433FE for ; Tue, 1 Feb 2022 18:32:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237968AbiBAScQ (ORCPT ); Tue, 1 Feb 2022 13:32:16 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:23902 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231693AbiBAScP (ORCPT ); Tue, 1 Feb 2022 13:32:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643740335; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=TZGPu0c4d8zM+GPEdhVnS2gfnUOmYVJ4nIlDhy1/n6A=; b=i9qm8iN/taRdnfPDsgjB54DeUy5lJnjIAlg5rCJ1WwXEJeCkMeprLHc4Mfh469SoMLj3vs 3yFBVbvFQyTIs55g0EVJkgnVxNIoTFJxtEpGlTP13lu11HMnelwtXFLwd0M+VDzv3VUntM cbmDpkS+zPL82PMT0GCJReDDrlg4a24= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-451-_BYmkupWO1qxJJWnnqHHaw-1; Tue, 01 Feb 2022 13:32:12 -0500 X-MC-Unique: _BYmkupWO1qxJJWnnqHHaw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7A8E82F7DF; Tue, 1 Feb 2022 18:32:07 +0000 (UTC) Received: from file01.intranet.prod.int.rdu2.redhat.com (file01.intranet.prod.int.rdu2.redhat.com [10.11.5.7]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A93EA798D8; Tue, 1 Feb 2022 18:31:52 +0000 (UTC) Received: from file01.intranet.prod.int.rdu2.redhat.com (localhost [127.0.0.1]) by file01.intranet.prod.int.rdu2.redhat.com (8.14.4/8.14.4) with ESMTP id 211IVqOM019366; Tue, 1 Feb 2022 13:31:52 -0500 Received: from localhost (mpatocka@localhost) by file01.intranet.prod.int.rdu2.redhat.com (8.14.4/8.14.4/Submit) with ESMTP id 211IVpQx019362; Tue, 1 Feb 2022 13:31:51 -0500 X-Authentication-Warning: file01.intranet.prod.int.rdu2.redhat.com: mpatocka owned process doing -bs Date: Tue, 1 Feb 2022 13:31:51 -0500 (EST) From: Mikulas Patocka X-X-Sender: mpatocka@file01.intranet.prod.int.rdu2.redhat.com To: =?ISO-8859-15?Q?Javier_Gonz=E1lez?= cc: Chaitanya Kulkarni , "linux-block@vger.kernel.org" , "linux-scsi@vger.kernel.org" , "dm-devel@redhat.com" , "linux-nvme@lists.infradead.org" , linux-fsdevel , Jens Axboe , "msnitzer@redhat.com >> msnitzer@redhat.com" , Bart Van Assche , "martin.petersen@oracle.com >> Martin K. Petersen" , "roland@purestorage.com" , Hannes Reinecke , "kbus @imap.gmail.com>> Keith Busch" , Christoph Hellwig , "Frederick.Knight@netapp.com" , "zach.brown@ni.com" , "osandov@fb.com" , "lsf-pc@lists.linux-foundation.org" , "djwong@kernel.org" , "josef@toxicpanda.com" , "clm@fb.com" , "dsterba@suse.com" , "tytso@mit.edu" , "jack@suse.com" , Kanchan Joshi Subject: [RFC PATCH 0/3] NVMe copy offload patches In-Reply-To: <20220201102122.4okwj2gipjbvuyux@mpHalley-2> Message-ID: References: <20220201102122.4okwj2gipjbvuyux@mpHalley-2> User-Agent: Alpine 2.02 (LRH 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Hi Here I'm submitting the first version of NVMe copy offload patches as a request for comment. They use the token-based approach as we discussed on the phone call. The first patch adds generic copy offload support to the block layer - it adds two new bio types (REQ_OP_COPY_READ_TOKEN and REQ_OP_COPY_WRITE_TOKEN) and a new ioctl BLKCOPY and a kernel function blkdev_issue_copy. The second patch adds copy offload support to the NVMe subsystem. The third patch implements a "nvme-debug" driver - it is similar to "scsi-debug", it simulates a nvme host controller, it keeps data in memory and it supports copy offload according to NVMe Command Set Specification 1.0a. (there are no hardware or software implementations supporting copy offload so far, so I implemented it in nvme-debug) TODO: * implement copy offload in device mapper linear target * implement copy offload in software NVMe target driver * make it possible to complete REQ_OP_COPY_WRITE_TOKEN bios asynchronously * should we use copy_file_range instead of a new ioctl? How to test this: * apply the three patches * select CONFIG_NVME_DEBUG * compile the kernel * modprobe nvme-debug; nvme connect -t debug -a 123 -n 456 * issue the BLKCOPY ioctl on /dev/nvme0n1, for example, you can use this program: http://people.redhat.com/~mpatocka/patches/kernel/xcopy/example/blkcopy.c Mikulas From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 064C5C433F5 for ; Tue, 1 Feb 2022 18:32:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643740348; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=9H1cYBlORdIbMUzoeCUhJr2W+QRiHeEp52wS5fYU0Tg=; b=cXYeKb5B8DHoMI4+q+p5xx5X5YSOL1jdr+whzlFWDoW7ikugBYWQjHpOaOvd5CVX8APBhx jOl1mM0/HXhsaeh4SkZyXio/+EkvNbuc16v4LAMuxKmVHgmAENrnYWub93d0FXrAaqUmlL 1LrbvUDbrIOZAm57UZpTS835Pm1yDw4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-635-enevXC5kMT6ELCX58CxuWg-1; Tue, 01 Feb 2022 13:32:25 -0500 X-MC-Unique: enevXC5kMT6ELCX58CxuWg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1F94B8144E7; Tue, 1 Feb 2022 18:32:20 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id F2994D1921; Tue, 1 Feb 2022 18:32:15 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 885424BB7C; Tue, 1 Feb 2022 18:32:12 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 211IW7b3003722 for ; Tue, 1 Feb 2022 13:32:07 -0500 Received: by smtp.corp.redhat.com (Postfix) id 791CA79A0F; Tue, 1 Feb 2022 18:32:07 +0000 (UTC) Received: from file01.intranet.prod.int.rdu2.redhat.com (file01.intranet.prod.int.rdu2.redhat.com [10.11.5.7]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A93EA798D8; Tue, 1 Feb 2022 18:31:52 +0000 (UTC) Received: from file01.intranet.prod.int.rdu2.redhat.com (localhost [127.0.0.1]) by file01.intranet.prod.int.rdu2.redhat.com (8.14.4/8.14.4) with ESMTP id 211IVqOM019366; Tue, 1 Feb 2022 13:31:52 -0500 Received: from localhost (mpatocka@localhost) by file01.intranet.prod.int.rdu2.redhat.com (8.14.4/8.14.4/Submit) with ESMTP id 211IVpQx019362; Tue, 1 Feb 2022 13:31:51 -0500 X-Authentication-Warning: file01.intranet.prod.int.rdu2.redhat.com: mpatocka owned process doing -bs Date: Tue, 1 Feb 2022 13:31:51 -0500 (EST) From: Mikulas Patocka X-X-Sender: mpatocka@file01.intranet.prod.int.rdu2.redhat.com To: =?ISO-8859-15?Q?Javier_Gonz=E1lez?= In-Reply-To: <20220201102122.4okwj2gipjbvuyux@mpHalley-2> Message-ID: References: <20220201102122.4okwj2gipjbvuyux@mpHalley-2> User-Agent: Alpine 2.02 (LRH 1266 2009-07-14) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: dm-devel@redhat.com Cc: "djwong@kernel.org" , "linux-nvme@lists.infradead.org" , "clm@fb.com" , "dm-devel@redhat.com" , "osandov@fb.com" , "msnitzer@redhat.com >> msnitzer@redhat.com" , Bart Van Assche , "linux-scsi@vger.kernel.org" , Christoph Hellwig , "roland@purestorage.com" , "zach.brown@ni.com" , Chaitanya Kulkarni , "josef@toxicpanda.com" , "linux-block@vger.kernel.org" , "dsterba@suse.com" , "kbus @imap.gmail.com>> Keith Busch" , "Frederick.Knight@netapp.com" , Jens Axboe , "tytso@mit.edu" , Kanchan Joshi , "martin.petersen@oracle.com >> Martin K. Petersen" , "jack@suse.com" , linux-fsdevel , "lsf-pc@lists.linux-foundation.org" Subject: [dm-devel] [RFC PATCH 0/3] NVMe copy offload patches X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dm-devel-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Hi Here I'm submitting the first version of NVMe copy offload patches as a request for comment. They use the token-based approach as we discussed on the phone call. The first patch adds generic copy offload support to the block layer - it adds two new bio types (REQ_OP_COPY_READ_TOKEN and REQ_OP_COPY_WRITE_TOKEN) and a new ioctl BLKCOPY and a kernel function blkdev_issue_copy. The second patch adds copy offload support to the NVMe subsystem. The third patch implements a "nvme-debug" driver - it is similar to "scsi-debug", it simulates a nvme host controller, it keeps data in memory and it supports copy offload according to NVMe Command Set Specification 1.0a. (there are no hardware or software implementations supporting copy offload so far, so I implemented it in nvme-debug) TODO: * implement copy offload in device mapper linear target * implement copy offload in software NVMe target driver * make it possible to complete REQ_OP_COPY_WRITE_TOKEN bios asynchronously * should we use copy_file_range instead of a new ioctl? How to test this: * apply the three patches * select CONFIG_NVME_DEBUG * compile the kernel * modprobe nvme-debug; nvme connect -t debug -a 123 -n 456 * issue the BLKCOPY ioctl on /dev/nvme0n1, for example, you can use this program: http://people.redhat.com/~mpatocka/patches/kernel/xcopy/example/blkcopy.c Mikulas -- dm-devel mailing list dm-devel@redhat.com https://listman.redhat.com/mailman/listinfo/dm-devel