From: Damien Le Moal <Damien.LeMoal@wdc.com>
To: Sergei Shtepa <sergei.shtepa@veeam.com>, Hannes Reinecke <hare@suse.de>
Cc: "axboe@kernel.dk" <axboe@kernel.dk>,
"viro@zeniv.linux.org.uk" <viro@zeniv.linux.org.uk>,
"hch@infradead.org" <hch@infradead.org>,
"darrick.wong@oracle.com" <darrick.wong@oracle.com>,
"linux-xfs@vger.kernel.org" <linux-xfs@vger.kernel.org>,
"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
"rjw@rjwysocki.net" <rjw@rjwysocki.net>,
"len.brown@intel.com" <len.brown@intel.com>,
"pavel@ucw.cz" <pavel@ucw.cz>,
"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
Johannes Thumshirn <Johannes.Thumshirn@wdc.com>,
"ming.lei@redhat.com" <ming.lei@redhat.com>,
"jack@suse.cz" <jack@suse.cz>, "tj@kernel.org" <tj@kernel.org>,
"gustavo@embeddedor.com" <gustavo@embeddedor.com>,
"bvanassche@acm.org" <bvanassche@acm.org>,
"osandov@fb.com" <osandov@fb.com>,
"koct9i@gmail.com" <koct9i@gmail.com>,
"steve@sk2.org" <steve@sk2.org>,
"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"linux-pm@vger.kernel.org" <linux-pm@vger.kernel.org>,
"linux-mm@kvack.org" <linux-mm@kvack.org>
Subject: Re: [PATCH 0/2] block layer filter and block device snapshot module
Date: Thu, 22 Oct 2020 10:28:28 +0000 [thread overview]
Message-ID: <BL0PR04MB6514AC1B1FF313E6A14D122CE71D0@BL0PR04MB6514.namprd04.prod.outlook.com> (raw)
In-Reply-To: 20201022094402.GA21466@veeam.com
On 2020/10/22 18:43, Sergei Shtepa wrote:
> The 10/22/2020 08:58, Hannes Reinecke wrote:
>> On 10/21/20 4:10 PM, Sergei Shtepa wrote:
>>> The 10/21/2020 16:31, Hannes Reinecke wrote:
>>>> I do understand where you are coming from, but then we already have a
>>>> dm-snap which does exactly what you want to achieve.
>>>> Of course, that would require a reconfiguration of the storage stack on
>>>> the machine, which is not always possible (or desired).
>>>
>>> Yes, reconfiguring the storage stack on a machine is almost impossible.
>>>
>>>>
>>>> What I _could_ imagine would be a 'dm-intercept' thingie, which
>>>> redirects the current submit_bio() function for any block device, and
>>>> re-routes that to a linear device-mapper device pointing back to the
>>>> original block device.
>>>>
>>>> That way you could attach it to basically any block device, _and_ can
>>>> use the existing device-mapper functionality to do fancy stuff once the
>>>> submit_io() callback has been re-routed.
>>>>
>>>> And it also would help in other scenarios, too; with such a
>>>> functionality we could seamlessly clone devices without having to move
>>>> the whole setup to device-mapper first.
>>>
>>> Hm...
>>> Did I understand correctly that the filter itself can be left approximately
>>> as it is, but the blk-snap module can be replaced with 'dm-intercept',
>>> which would use the re-route mechanism from the dm?
>>> I think I may be able to implement it, if you describe your idea in more
>>> detail.
>>>
>>>
>> Actually, once we have an dm-intercept, why do you need the block-layer
>> filter at all?
>> From you initial description the block-layer filter was implemented
>> such that blk-snap could work; but if we have dm-intercept (and with it
>> the ability to use device-mapper functionality even for normal block
>> devices) there wouldn't be any need for the block-layer filter, no?
>
> Maybe, but the problem is that I can't imagine how to implement
> dm-intercept yet.
> How to use dm to implement interception without changing the stack
> of block devices. We'll have to make a hook somewhere, isn`t it?
Once your dm-intercept target driver is inserted with "dmsetup" or any user land
tool you implement using libdevicemapper, the "hooks" will naturally be in place
since the dm infrastructure already does that: all submitted BIOs will be passed
to dm-intercept through the "map" operation defined in the target_type
descriptor. It is then that driver job to execute the BIOs as it sees fit.
Look at simple device mappers like dm-linear or dm-flakey for hints of how
things work (driver/md/dm-linear.c). More complex dm drivers like dm-crypt,
dm-writecache or dm-thin can give you hints about more features of device mapper.
Functions such as __map_bio() in drivers/md/dm.c are the core of DM and show
what happens to BIOs depending on the the return value of the map operation.
dm_submit_bio() and __split_and_process_bio() is the entry points for BIO
processing in DM.
>
>>
>> Cheers,
>>
>> Hannes
>> --
>> Dr. Hannes Reinecke Kernel Storage Architect
>> hare@suse.de +49 911 74053 688
>> SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
>> HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
>
--
Damien Le Moal
Western Digital Research
next prev parent reply other threads:[~2020-10-22 10:28 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-21 9:04 [PATCH 0/2] block layer filter and block device snapshot module Sergei Shtepa
2020-10-21 9:04 ` [PATCH 1/2] Block layer filter - second version Sergei Shtepa
2020-10-21 9:14 ` Johannes Thumshirn
2020-10-21 10:01 ` Sergei Shtepa
2020-10-21 9:21 ` Damien Le Moal
2020-10-21 10:27 ` Sergei Shtepa
2020-10-21 11:44 ` Matthew Wilcox
2020-10-21 12:55 ` Sergei Shtepa
2020-10-21 13:07 ` Matthew Wilcox
2020-10-21 14:35 ` Sergei Shtepa
2020-10-21 15:09 ` Randy Dunlap
2020-10-24 14:53 ` Greg KH
2020-10-21 9:04 ` [PATCH 2/2] blk-snap - snapshots and change-tracking for block devices Sergei Shtepa
2020-10-21 9:08 ` Pavel Machek
2020-10-21 9:37 ` Sergei Shtepa
2020-10-21 9:23 ` Damien Le Moal
2020-10-21 11:15 ` Sergei Shtepa
2020-10-21 10:48 ` kernel test robot
2020-10-21 15:11 ` Randy Dunlap
2020-10-21 13:31 ` [PATCH 0/2] block layer filter and block device snapshot module Hannes Reinecke
2020-10-21 14:10 ` Sergei Shtepa
2020-10-22 5:58 ` Hannes Reinecke
2020-10-22 9:44 ` Sergei Shtepa
2020-10-22 10:28 ` Damien Le Moal [this message]
2020-10-22 13:52 ` Sergei Shtepa
2020-10-22 15:14 ` Darrick J. Wong
2020-10-22 17:54 ` Mike Snitzer
2020-10-23 9:13 ` hch
2020-10-23 10:31 ` Hannes Reinecke
2020-10-23 11:04 ` Sergei Shtepa
2020-10-23 11:12 ` [dm-devel] " hch
2020-10-22 18:35 ` Mike Snitzer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=BL0PR04MB6514AC1B1FF313E6A14D122CE71D0@BL0PR04MB6514.namprd04.prod.outlook.com \
--to=damien.lemoal@wdc.com \
--cc=Johannes.Thumshirn@wdc.com \
--cc=akpm@linux-foundation.org \
--cc=axboe@kernel.dk \
--cc=bvanassche@acm.org \
--cc=darrick.wong@oracle.com \
--cc=gustavo@embeddedor.com \
--cc=hare@suse.de \
--cc=hch@infradead.org \
--cc=jack@suse.cz \
--cc=koct9i@gmail.com \
--cc=len.brown@intel.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-pm@vger.kernel.org \
--cc=linux-xfs@vger.kernel.org \
--cc=ming.lei@redhat.com \
--cc=osandov@fb.com \
--cc=pavel@ucw.cz \
--cc=rjw@rjwysocki.net \
--cc=sergei.shtepa@veeam.com \
--cc=steve@sk2.org \
--cc=tj@kernel.org \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).