From: Sergei Shtepa <sergei.shtepa@veeam.com> To: Christoph Hellwig <hch@infradead.org>, Mike Snitzer <snitzer@redhat.com>, Alasdair Kergon <agk@redhat.com>, Hannes Reinecke <hare@suse.de>, Jens Axboe <axboe@kernel.dk>, <dm-devel@redhat.com>, <linux-block@vger.kernel.org>, <linux-kernel@vger.kernel.org>, <linux-api@vger.kernel.org> Cc: <sergei.shtepa@veeam.com>, <pavel.tide@veeam.com> Subject: [PATCH v7 0/3] block device interposer Date: Fri, 12 Mar 2021 18:44:52 +0300 [thread overview] Message-ID: <1615563895-28565-1-git-send-email-sergei.shtepa@veeam.com> (raw) Hi all. I'm joyful to suggest the block device interposer (bdev_interposer) v7. bdev_interposer allows to redirect bio requests to other block devices. In this series of patches I suggest a different implementation of the bio interception mechanism. Now the interposer is a different block device. Instead of an additional hook, the function fops->submit_bio() of the interposer device is used. This implementation greatly simplifies the application of this bdev_interposer in device-mapper. But there is one limitation - the size of the interposer device must be greater than or equal to the size of the original device. The first patch adds the function blk_mq_is_queue_frozen(). It allows to check a queue state. The second patch is dedicated to bdev_interposer itself, which provides the ability to redirect bio to the interposer device. The third one adds the DM_INTERPOSED_FLAG flag. When this flag is applied with the ioctl DM_TABLE_LOAD_CMD, the underlying devices are opened without the FMODE_EXCL flag and connected via bdev_interposer. Changes in this patchset v7: * the request interception mechanism. Now the interposer is a block device that receives requests instead of the original device; * code design fixes. History: v6 - https://patchwork.kernel.org/project/linux-block/cover/1614774618-22410-1-git-send-email-sergei.shtepa@veeam.com/ * designed for 5.12; * thanks to the new design of the bio structure in v5.12, it is possible to perform interception not for the entire disk, but for each block device; * instead of the new ioctl DM_DEV_REMAP_CMD and the 'noexcl' option, the DM_INTERPOSED_FLAG flag for the ioctl DM_TABLE_LOAD_CMD is applied. v5 - https://patchwork.kernel.org/project/linux-block/cover/1612881028-7878-1-git-send-email-sergei.shtepa@veeam.com/ * rebase for v5.11-rc7; * patch set organization; * fix defects in documentation; * add some comments; * change mutex names for better code readability; * remove calling bd_unlink_disk_holder() for targets with non-exclusive flag; * change type for struct dm_remap_param from uint8_t to __u8. v4 - https://patchwork.kernel.org/project/linux-block/cover/1612367638-3794-1-git-send-email-sergei.shtepa@veeam.com/ Mostly changes were made, due to Damien's comments: * on the design of the code; * by the patch set organization; * bug with passing a wrong parameter to dm_get_device(); * description of the 'noexcl' parameter in the linear.rst. Also added remap_and_filter.rst. v3 - https://patchwork.kernel.org/project/linux-block/cover/1611853955-32167-1-git-send-email-sergei.shtepa@veeam.com/ In this version, I already suggested blk_interposer to apply to dm-linear. Problems were solved: * Interception of bio requests from a specific device on the disk, not from the entire disk. To do this, we added the dm_interposed_dev structure and an interval tree to store these structures. * Implemented ioctl DM_DEV_REMAP_CMD. A patch with changes in the lvm2 project was sent to the team lvm-devel@redhat.com. * Added the 'noexcl' option for dm-linear, which allows you to open the underlying block-device without FMODE_EXCL mode. v2 - https://patchwork.kernel.org/project/linux-block/cover/1607518911-30692-1-git-send-email-sergei.shtepa@veeam.com/ I tried to suggest blk_interposer without using it in device mapper, but with the addition of a sample of its use. It was then that I learned about the maintainers' attitudes towards the samples directory :). v1 - https://lwn.net/ml/linux-block/20201119164924.74401-1-hare@suse.de/ This Hannes's patch can be considered as a starting point, since this is where the interception mechanism and the term blk_interposer itself appeared. It became clear that blk_interposer can be useful for device mapper. before v1 - https://patchwork.kernel.org/project/linux-block/cover/1603271049-20681-1-git-send-email-sergei.shtepa@veeam.com/ I tried to offer a rather cumbersome blk-filter and a monster-like blk-snap module for creating snapshots. Thank you to everyone who was able to take the time to review the previous versions. I hope that this time I achieved the required quality. Thanks, Sergei. Sergei Shtepa (3): block: add blk_mq_is_queue_frozen() block: add bdev_interposer dm: add DM_INTERPOSED_FLAG block/bio.c | 2 ++ block/blk-core.c | 57 ++++++++++++++++++++++++++++++++ block/blk-mq.c | 13 ++++++++ block/genhd.c | 54 +++++++++++++++++++++++++++++++ drivers/md/dm-core.h | 3 ++ drivers/md/dm-ioctl.c | 13 ++++++++ drivers/md/dm-table.c | 61 +++++++++++++++++++++++++++++------ drivers/md/dm.c | 38 +++++++++++++++------- include/linux/blk-mq.h | 1 + include/linux/blk_types.h | 3 ++ include/linux/blkdev.h | 9 ++++++ include/linux/device-mapper.h | 1 + include/uapi/linux/dm-ioctl.h | 6 ++++ 13 files changed, 240 insertions(+), 21 deletions(-) -- 2.20.1
WARNING: multiple messages have this Message-ID (diff)
From: Sergei Shtepa <sergei.shtepa@veeam.com> To: Christoph Hellwig <hch@infradead.org>, Mike Snitzer <snitzer@redhat.com>, Alasdair Kergon <agk@redhat.com>, Hannes Reinecke <hare@suse.de>, Jens Axboe <axboe@kernel.dk>, <dm-devel@redhat.com>, <linux-block@vger.kernel.org>, <linux-kernel@vger.kernel.org>, <linux-api@vger.kernel.org> Cc: pavel.tide@veeam.com, sergei.shtepa@veeam.com Subject: [dm-devel] [PATCH v7 0/3] block device interposer Date: Fri, 12 Mar 2021 18:44:52 +0300 [thread overview] Message-ID: <1615563895-28565-1-git-send-email-sergei.shtepa@veeam.com> (raw) Hi all. I'm joyful to suggest the block device interposer (bdev_interposer) v7. bdev_interposer allows to redirect bio requests to other block devices. In this series of patches I suggest a different implementation of the bio interception mechanism. Now the interposer is a different block device. Instead of an additional hook, the function fops->submit_bio() of the interposer device is used. This implementation greatly simplifies the application of this bdev_interposer in device-mapper. But there is one limitation - the size of the interposer device must be greater than or equal to the size of the original device. The first patch adds the function blk_mq_is_queue_frozen(). It allows to check a queue state. The second patch is dedicated to bdev_interposer itself, which provides the ability to redirect bio to the interposer device. The third one adds the DM_INTERPOSED_FLAG flag. When this flag is applied with the ioctl DM_TABLE_LOAD_CMD, the underlying devices are opened without the FMODE_EXCL flag and connected via bdev_interposer. Changes in this patchset v7: * the request interception mechanism. Now the interposer is a block device that receives requests instead of the original device; * code design fixes. History: v6 - https://patchwork.kernel.org/project/linux-block/cover/1614774618-22410-1-git-send-email-sergei.shtepa@veeam.com/ * designed for 5.12; * thanks to the new design of the bio structure in v5.12, it is possible to perform interception not for the entire disk, but for each block device; * instead of the new ioctl DM_DEV_REMAP_CMD and the 'noexcl' option, the DM_INTERPOSED_FLAG flag for the ioctl DM_TABLE_LOAD_CMD is applied. v5 - https://patchwork.kernel.org/project/linux-block/cover/1612881028-7878-1-git-send-email-sergei.shtepa@veeam.com/ * rebase for v5.11-rc7; * patch set organization; * fix defects in documentation; * add some comments; * change mutex names for better code readability; * remove calling bd_unlink_disk_holder() for targets with non-exclusive flag; * change type for struct dm_remap_param from uint8_t to __u8. v4 - https://patchwork.kernel.org/project/linux-block/cover/1612367638-3794-1-git-send-email-sergei.shtepa@veeam.com/ Mostly changes were made, due to Damien's comments: * on the design of the code; * by the patch set organization; * bug with passing a wrong parameter to dm_get_device(); * description of the 'noexcl' parameter in the linear.rst. Also added remap_and_filter.rst. v3 - https://patchwork.kernel.org/project/linux-block/cover/1611853955-32167-1-git-send-email-sergei.shtepa@veeam.com/ In this version, I already suggested blk_interposer to apply to dm-linear. Problems were solved: * Interception of bio requests from a specific device on the disk, not from the entire disk. To do this, we added the dm_interposed_dev structure and an interval tree to store these structures. * Implemented ioctl DM_DEV_REMAP_CMD. A patch with changes in the lvm2 project was sent to the team lvm-devel@redhat.com. * Added the 'noexcl' option for dm-linear, which allows you to open the underlying block-device without FMODE_EXCL mode. v2 - https://patchwork.kernel.org/project/linux-block/cover/1607518911-30692-1-git-send-email-sergei.shtepa@veeam.com/ I tried to suggest blk_interposer without using it in device mapper, but with the addition of a sample of its use. It was then that I learned about the maintainers' attitudes towards the samples directory :). v1 - https://lwn.net/ml/linux-block/20201119164924.74401-1-hare@suse.de/ This Hannes's patch can be considered as a starting point, since this is where the interception mechanism and the term blk_interposer itself appeared. It became clear that blk_interposer can be useful for device mapper. before v1 - https://patchwork.kernel.org/project/linux-block/cover/1603271049-20681-1-git-send-email-sergei.shtepa@veeam.com/ I tried to offer a rather cumbersome blk-filter and a monster-like blk-snap module for creating snapshots. Thank you to everyone who was able to take the time to review the previous versions. I hope that this time I achieved the required quality. Thanks, Sergei. Sergei Shtepa (3): block: add blk_mq_is_queue_frozen() block: add bdev_interposer dm: add DM_INTERPOSED_FLAG block/bio.c | 2 ++ block/blk-core.c | 57 ++++++++++++++++++++++++++++++++ block/blk-mq.c | 13 ++++++++ block/genhd.c | 54 +++++++++++++++++++++++++++++++ drivers/md/dm-core.h | 3 ++ drivers/md/dm-ioctl.c | 13 ++++++++ drivers/md/dm-table.c | 61 +++++++++++++++++++++++++++++------ drivers/md/dm.c | 38 +++++++++++++++------- include/linux/blk-mq.h | 1 + include/linux/blk_types.h | 3 ++ include/linux/blkdev.h | 9 ++++++ include/linux/device-mapper.h | 1 + include/uapi/linux/dm-ioctl.h | 6 ++++ 13 files changed, 240 insertions(+), 21 deletions(-) -- 2.20.1 -- dm-devel mailing list dm-devel@redhat.com https://listman.redhat.com/mailman/listinfo/dm-devel
next reply other threads:[~2021-03-12 15:46 UTC|newest] Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-03-12 15:44 Sergei Shtepa [this message] 2021-03-12 15:44 ` [dm-devel] [PATCH v7 0/3] block device interposer Sergei Shtepa 2021-03-12 15:44 ` [PATCH v7 1/3] block: add blk_mq_is_queue_frozen() Sergei Shtepa 2021-03-12 15:44 ` [dm-devel] " Sergei Shtepa 2021-03-12 19:06 ` Mike Snitzer 2021-03-12 19:06 ` [dm-devel] " Mike Snitzer 2021-03-14 9:14 ` Christoph Hellwig 2021-03-14 9:14 ` [dm-devel] " Christoph Hellwig 2021-03-15 12:06 ` Sergei Shtepa 2021-03-15 12:06 ` [dm-devel] " Sergei Shtepa 2021-03-12 15:44 ` [PATCH v7 2/3] block: add bdev_interposer Sergei Shtepa 2021-03-12 15:44 ` [dm-devel] " Sergei Shtepa 2021-03-14 9:28 ` Christoph Hellwig 2021-03-14 9:28 ` [dm-devel] " Christoph Hellwig 2021-03-15 13:06 ` Sergei Shtepa 2021-03-15 13:06 ` [dm-devel] " Sergei Shtepa 2021-03-16 8:09 ` Ming Lei 2021-03-16 8:09 ` [dm-devel] " Ming Lei 2021-03-16 16:35 ` Sergei Shtepa 2021-03-16 16:35 ` [dm-devel] " Sergei Shtepa 2021-03-17 3:03 ` Ming Lei 2021-03-17 3:03 ` [dm-devel] " Ming Lei 2021-03-17 12:22 ` Sergei Shtepa 2021-03-17 12:22 ` [dm-devel] " Sergei Shtepa 2021-03-17 15:04 ` Mike Snitzer 2021-03-17 15:04 ` [dm-devel] " Mike Snitzer 2021-03-17 18:14 ` Sergei Shtepa 2021-03-17 18:14 ` [dm-devel] " Sergei Shtepa 2021-03-17 19:13 ` Mike Snitzer 2021-03-17 19:13 ` [dm-devel] " Mike Snitzer 2021-03-18 14:56 ` Sergei Shtepa 2021-03-18 14:56 ` [dm-devel] " Sergei Shtepa 2021-03-17 14:58 ` Mike Snitzer 2021-03-17 14:58 ` [dm-devel] " Mike Snitzer 2021-03-12 15:44 ` [PATCH v7 3/3] dm: add DM_INTERPOSED_FLAG Sergei Shtepa 2021-03-12 15:44 ` [dm-devel] " Sergei Shtepa 2021-03-12 19:00 ` Mike Snitzer 2021-03-12 19:00 ` [dm-devel] " Mike Snitzer 2021-03-15 12:29 ` Sergei Shtepa 2021-03-15 12:29 ` [dm-devel] " Sergei Shtepa 2021-03-14 9:30 ` Christoph Hellwig 2021-03-14 9:30 ` [dm-devel] " Christoph Hellwig 2021-03-15 13:25 ` Sergei Shtepa 2021-03-15 13:25 ` [dm-devel] " Sergei Shtepa 2021-03-16 15:23 ` Christoph Hellwig 2021-03-16 15:23 ` [dm-devel] " Christoph Hellwig 2021-03-16 15:25 ` Christoph Hellwig 2021-03-16 15:25 ` [dm-devel] " Christoph Hellwig 2021-03-16 16:20 ` Sergei Shtepa 2021-03-16 16:20 ` [dm-devel] " Sergei Shtepa
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1615563895-28565-1-git-send-email-sergei.shtepa@veeam.com \ --to=sergei.shtepa@veeam.com \ --cc=agk@redhat.com \ --cc=axboe@kernel.dk \ --cc=dm-devel@redhat.com \ --cc=hare@suse.de \ --cc=hch@infradead.org \ --cc=linux-api@vger.kernel.org \ --cc=linux-block@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=pavel.tide@veeam.com \ --cc=snitzer@redhat.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.