From: Sergei Shtepa <sergei.shtepa@veeam.com>
To: Yu Kuai <yukuai1@huaweicloud.com>, <axboe@kernel.dk>,
<hch@infradead.org>, <corbet@lwn.net>, <snitzer@kernel.org>
Cc: <viro@zeniv.linux.org.uk>, <brauner@kernel.org>,
<dchinner@redhat.com>, <willy@infradead.org>,
<dlemoal@kernel.org>, <linux@weissschuh.net>, <jack@suse.cz>,
<ming.lei@redhat.com>, <linux-block@vger.kernel.org>,
<linux-doc@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
<linux-fsdevel@vger.kernel.org>,
Donald Buczek <buczek@molgen.mpg.de>,
"yukuai (C)" <yukuai3@huawei.com>
Subject: Re: [PATCH v5 02/11] block: Block Device Filtering Mechanism
Date: Wed, 19 Jul 2023 10:36:32 +0200 [thread overview]
Message-ID: <c33df221-968c-9f31-e545-27dd4e90729f@veeam.com> (raw)
In-Reply-To: <8257903a-1905-49c5-bed4-d15ca06c6d3b@huaweicloud.com>
On 7/19/23 09:28, Yu Kuai wrote:
> Subject:
> Re: [PATCH v5 02/11] block: Block Device Filtering Mechanism
> From:
> Yu Kuai <yukuai1@huaweicloud.com>
> Date:
> 7/19/23, 09:28
>
> To:
> Sergei Shtepa <sergei.shtepa@veeam.com>, Yu Kuai <yukuai1@huaweicloud.com>, axboe@kernel.dk, hch@infradead.org, corbet@lwn.net, snitzer@kernel.org
> CC:
> viro@zeniv.linux.org.uk, brauner@kernel.org, dchinner@redhat.com, willy@infradead.org, dlemoal@kernel.org, linux@weissschuh.net, jack@suse.cz, ming.lei@redhat.com, linux-block@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Donald Buczek <buczek@molgen.mpg.de>, "yukuai (C)" <yukuai3@huawei.com>
>
>
> Hi,
>
> 在 2023/07/19 0:33, Sergei Shtepa 写道:
>>
>>
>> On 7/18/23 14:32, Yu Kuai wrote:
>>> Subject:
>>> Re: [PATCH v5 02/11] block: Block Device Filtering Mechanism
>>> From:
>>> Yu Kuai <yukuai1@huaweicloud.com>
>>> Date:
>>> 7/18/23, 14:32
>>>
>>> To:
>>> Sergei Shtepa <sergei.shtepa@veeam.com>, Yu Kuai <yukuai1@huaweicloud.com>, axboe@kernel.dk, hch@infradead.org, corbet@lwn.net, snitzer@kernel.org
>>> CC:
>>> viro@zeniv.linux.org.uk, brauner@kernel.org, dchinner@redhat.com, willy@infradead.org, dlemoal@kernel.org, linux@weissschuh.net, jack@suse.cz, ming.lei@redhat.com, linux-block@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Donald Buczek <buczek@molgen.mpg.de>, "yukuai (C)" <yukuai3@huawei.com>
>>>
>>>
>>> Hi,
>>>
>>> 在 2023/07/18 19:25, Sergei Shtepa 写道:
>>>> Hi.
>>>>
>>>> On 7/18/23 03:37, Yu Kuai wrote:
>>>>> Subject:
>>>>> Re: [PATCH v5 02/11] block: Block Device Filtering Mechanism
>>>>> From:
>>>>> Yu Kuai <yukuai1@huaweicloud.com>
>>>>> Date:
>>>>> 7/18/23, 03:37
>>>>>
>>>>> To:
>>>>> Sergei Shtepa <sergei.shtepa@veeam.com>, Yu Kuai <yukuai1@huaweicloud.com>, axboe@kernel.dk, hch@infradead.org, corbet@lwn.net, snitzer@kernel.org
>>>>> CC:
>>>>> viro@zeniv.linux.org.uk, brauner@kernel.org, dchinner@redhat.com, willy@infradead.org, dlemoal@kernel.org, linux@weissschuh.net, jack@suse.cz, ming.lei@redhat.com, linux-block@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Donald Buczek <buczek@molgen.mpg.de>, "yukuai (C)" <yukuai3@huawei.com>
>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> 在 2023/07/17 22:39, Sergei Shtepa 写道:
>>>>>>
>>>>>>
>>>>>> On 7/11/23 04:02, Yu Kuai wrote:
>>>>>>> bdev_disk_changed() is not handled, where delete_partition() and
>>>>>>> add_partition() will be called, this means blkfilter for partiton will
>>>>>>> be removed after partition rescan. Am I missing something?
>>>>>>
>>>>>> Yes, when the bdev_disk_changed() is called, all disk block devices
>>>>>> are deleted and new ones are re-created. Therefore, the information
>>>>>> about the attached filters will be lost. This is equivalent to
>>>>>> removing the disk and adding it back.
>>>>>>
>>>>>> For the blksnap module, partition rescan will mean the loss of the
>>>>>> change trackers data. If a snapshot was created, then such
>>>>>> a partition rescan will cause the snapshot to be corrupted.
>>>>>>
>>>>>
>>>>> I haven't review blksnap code yet, but this sounds like a problem.
>>>>
>>>> I can't imagine a case where this could be a problem.
>>>> Partition rescan is possible only if the file system has not been
>>>> mounted on any of the disk partitions. Ioctl BLKRRPART will return
>>>> -EBUSY. Therefore, during normal operation of the system, rescan is
>>>> not performed.
>>>> And if the file systems have not been mounted, it is possible that
>>>> the disk partition structure has changed or the disk in the media
>>>> device has changed. In this case, it is better to detach the
>>>> filter, otherwise it may lead to incorrect operation of the module.
>>>>
>>>> We can add prechange/postchange callback functions so that the
>>>> filter can track rescan process. But at the moment, this is not
>>>> necessary for the blksnap module.
>>>
>>> So you mean that blkfilter is only used for the case that partition
>>> is mounted? (Or you mean that partition is opened)
>>>
>>> Then, I think you mean that filter should only be used for the partition
>>> that is opended? Otherwise, filter can be gone at any time since
>>> partition rescan can be gone.
>>>
>>> //user
>>> 1. attach filter
>>> // other context rescan partition
>>> 2. mount fs
>>> // user will found filter is gone.
>>
>> Mmm... The fact is that at the moment the user of the filter is the
>> blksnap module. There are no other filter users yet. The blksnap module
>> solves the problem of creating snapshots, primarily for backup purposes.
>> Therefore, the main use case is to attach a filter for an already running
>> system, where all partitions are marked up, file systems are mounted.
>>
>> If the server is being serviced, during which the disk is being
>> re-partitioned, then disabling the filter is normal. In this case, the
>> change tracker will be reset, and at the next backup, the filter will be
>> attached again.
>
> Thanks for the explanation, I was thinking that blkshap can replace
> dm-snapshot.
Thanks!
At the moment I am creating blksnap with the Veeam product needs in mind.
I would be glad if blksnap would be useful in other products as well.
If you have any thoughts/questions/suggestions/comments, then write to me
directly. I'll be happy to discuss everything.
To work on the patch, I use the branch here
Link: https://github.com/SergeiShtepa/linux/tree/blksnap-master
The user-space libs, tools and tests, compatible with the upstream is here
Link: https://github.com/veeam/blksnap/tree/stable-v2.0
Perhaps it will be useful to you.
>
> Thanks,
> Kuai
>
>>
>> But if I were still solving the problem of saving the filter when rescanning,
>> then it is necessary to take into account the UUID and name of the partition
>> (struct partition_meta_info). It is unacceptable that due to a change in the
>> structure of partitions, the filter is attached to another partition by mistake.
>> The changed() callback would also be good to add so that the filter receives
>> a notification that the block device has been updated.
>>
>> But I'm not sure that this should be done, since if some code is not used in
>> the kernel, then it should not be in the kernel.
>>
>>>
>>> Thanks,
>>> Kuai
>>>
>>>>
>>>> Therefore, I will refrain from making changes for now.
>>>>
>>>>>
>>>>> possible solutions I have in mind:
>>>>>
>>>>> 1. Store blkfilter for each partition from bdev_disk_changed() before
>>>>> delete_partition(), and add blkfilter back after add_partition().
>>>>>
>>>>> 2. Store blkfilter from gendisk as a xarray, and protect it by
>>>>> 'open_mutex' like 'part_tbl', block_device can keep the pointer to
>>>>> reference blkfilter so that performance from fast path is ok, and the
>>>>> lifetime of blkfiter can be managed separately.
>>>>>
>>>>>> There was an idea to do filtering at the disk level,
>>>>>> but I abandoned it.
>>>>>> .
>>>>>>
>>>>> I think it's better to do filtering at the partition level as well.
>>>>>
>>>>> Thanks,
>>>>> Kuai
>>>>>
>>>> .
>>>>
>>>
>> .
>>
>
next prev parent reply other threads:[~2023-07-19 8:37 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-06-12 13:52 [PATCH v5 00/11] blksnap - block devices snapshots module Sergei Shtepa
2023-06-12 13:52 ` [PATCH v5 01/11] documentation: Block Device Filtering Mechanism Sergei Shtepa
2023-06-12 13:52 ` [PATCH v5 02/11] block: " Sergei Shtepa
2023-07-11 2:02 ` Yu Kuai
2023-07-12 10:04 ` Yu Kuai
2023-07-12 12:34 ` Yu Kuai
2023-07-17 17:39 ` Sergei Shtepa
2023-07-18 1:21 ` Yu Kuai
2023-07-17 16:22 ` Sergei Shtepa
2023-07-17 14:39 ` Sergei Shtepa
2023-07-18 1:37 ` Yu Kuai
2023-07-18 11:25 ` Sergei Shtepa
2023-07-18 12:32 ` Yu Kuai
2023-07-18 16:33 ` Sergei Shtepa
2023-07-19 7:28 ` Yu Kuai
2023-07-19 8:36 ` Sergei Shtepa [this message]
2023-07-20 6:14 ` Christoph Hellwig
2023-06-12 13:52 ` [PATCH v5 03/11] documentation: Block Devices Snapshots Module Sergei Shtepa
2023-06-12 13:52 ` [PATCH v5 04/11] blksnap: header file of the module interface Sergei Shtepa
2023-06-13 22:25 ` Dave Chinner
2023-06-14 6:26 ` Christoph Hellwig
2023-06-14 9:26 ` Sergei Shtepa
2023-06-14 14:07 ` Christoph Hellwig
2023-06-14 16:43 ` Sergei Shtepa
2023-06-15 0:08 ` Dave Chinner
2023-07-17 18:57 ` Thomas Weißschuh
2023-07-18 9:53 ` Sergei Shtepa
2023-07-20 6:16 ` Christoph Hellwig
2023-06-12 13:52 ` [PATCH v5 05/11] blksnap: module management interface functions Sergei Shtepa
2023-06-12 13:52 ` [PATCH v5 06/11] blksnap: handling and tracking I/O units Sergei Shtepa
2023-06-12 13:52 ` [PATCH v5 07/11] blksnap: minimum data storage unit of the original block device Sergei Shtepa
2023-06-12 13:52 ` [PATCH v5 08/11] blksnap: difference storage Sergei Shtepa
2023-06-12 13:52 ` [PATCH v5 09/11] blksnap: event queue from the " Sergei Shtepa
2023-06-12 13:52 ` [PATCH v5 10/11] blksnap: snapshot and snapshot image block device Sergei Shtepa
2023-06-12 13:52 ` [PATCH v5 11/11] blksnap: Kconfig and Makefile Sergei Shtepa
2023-06-12 14:32 ` [PATCH v5 00/11] blksnap - block devices snapshots module Christoph Hellwig
2023-06-12 16:19 ` Eric Biggers
2023-06-13 5:50 ` Christoph Hellwig
2023-06-13 10:12 ` Sergei Shtepa
2023-06-14 17:22 ` Eric Biggers
-- strict thread matches above, loose matches on Subject: below --
2023-06-12 13:21 Sergei Shtepa
2023-06-12 13:21 ` [PATCH v5 02/11] block: Block Device Filtering Mechanism Sergei Shtepa
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c33df221-968c-9f31-e545-27dd4e90729f@veeam.com \
--to=sergei.shtepa@veeam.com \
--cc=axboe@kernel.dk \
--cc=brauner@kernel.org \
--cc=buczek@molgen.mpg.de \
--cc=corbet@lwn.net \
--cc=dchinner@redhat.com \
--cc=dlemoal@kernel.org \
--cc=hch@infradead.org \
--cc=jack@suse.cz \
--cc=linux-block@vger.kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@weissschuh.net \
--cc=ming.lei@redhat.com \
--cc=snitzer@kernel.org \
--cc=viro@zeniv.linux.org.uk \
--cc=willy@infradead.org \
--cc=yukuai1@huaweicloud.com \
--cc=yukuai3@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).