From: Sergei Shtepa <sergei.shtepa@veeam.com>
To: <axboe@kernel.dk>, <mchehab+huawei@kernel.org>,
<davem@davemloft.net>, <robh@kernel.org>, <koct9i@gmail.com>,
<damien.lemoal@wdc.com>, <jack@suse.cz>, <ming.lei@redhat.com>,
<steve@sk2.org>, <linux-kernel@vger.kernel.org>,
<linux-kbuild@vger.kernel.org>, <linux-block@vger.kernel.org>
Cc: Sergei Shtepa <sergei.shtepa@veeam.com>
Subject: [PATCH 0/1] Block snapshot module and block layer filter API
Date: Fri, 2 Oct 2020 15:56:01 +0300 [thread overview]
Message-ID: <1601643362-7370-1-git-send-email-sergei.shtepa@veeam.com> (raw)
Hello everyone! Requesting for your comments and suggestions.
We propose a new kernel module - blk-snap.
This module implements snapshot and changed block tracking functionality.
It is intended to create backup copies of any block devices without usage
of device-mapper.
Snapshots are temporary and are destroyed after the backup process has
finished. Changed block tracking allows for incremental and differential
backup copies.
blk-snap uses block layer filter API.
Block layer filter API provides a callback to intercept bio-requests.
If a block device disappears for whatever reason, send a synchronous
request to remove the device from filtering.
Previously, we have already offered a patch with the implementation of the
block layer filter api (https://patchwork.kernel.org/patch/11741447/).
Link to that discussion:
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg2290127.html.
Now we also offer the in-tree module.
blk-snap kernel module is a product of a deep refactoring of the
out-of-tree kernel veeamsnap (https://github.com/veeam/veeamsnap/) module:
* all conditional compilation branches that served for the purpose of
compatibility with older kernels have been removed;
* linux kernel code style has been applied;
* blk-snap mostly takes advantage of the existing kernel code instead of
reinventing the wheel;
* all redundant code (such as persistent cbt and snapstore collector) has
been removed.
We would appreciate your feedback!
Several important things are still have to be done:
* refactoring the module interface for interaction with a user-space code,
it is already clear that the implementation of some calls can be improved;
* haven't yet tested the build on architectures other than x86_64;
* the user-space library is not ready yet and tool to control the module
is to be created;
* autotests need to be created;
* regression testing has been conducted in a cursory manner, so there is
a chance for mistakes.
Sergei Shtepa (1):
blk-snap - Block snapshot module This module implements snapshot and
changed block tracking functionality. It is intended to create
backup copies of any block devices without usage of device-mapper.
MAINTAINERS | 14 +
block/Kconfig | 11 +
block/Makefile | 1 +
block/blk-core.c | 11 +-
block/blk-filter-internal.h | 34 +
block/blk-filter.c | 288 ++++++
block/genhd.c | 24 +
drivers/block/Kconfig | 6 +-
drivers/block/Makefile | 3 +-
drivers/block/blk-snap/Kconfig | 24 +
drivers/block/blk-snap/Makefile | 29 +
drivers/block/blk-snap/big_buffer.c | 193 ++++
drivers/block/blk-snap/big_buffer.h | 24 +
drivers/block/blk-snap/blk-snap-ctl.h | 190 ++++
drivers/block/blk-snap/blk_deferred.c | 566 +++++++++++
drivers/block/blk-snap/blk_deferred.h | 67 ++
drivers/block/blk-snap/blk_descr_file.c | 82 ++
drivers/block/blk-snap/blk_descr_file.h | 26 +
drivers/block/blk-snap/blk_descr_mem.c | 66 ++
drivers/block/blk-snap/blk_descr_mem.h | 14 +
drivers/block/blk-snap/blk_descr_multidev.c | 86 ++
drivers/block/blk-snap/blk_descr_multidev.h | 25 +
drivers/block/blk-snap/blk_descr_pool.c | 190 ++++
drivers/block/blk-snap/blk_descr_pool.h | 38 +
drivers/block/blk-snap/blk_redirect.c | 507 ++++++++++
drivers/block/blk-snap/blk_redirect.h | 73 ++
drivers/block/blk-snap/blk_util.c | 33 +
drivers/block/blk-snap/blk_util.h | 10 +
drivers/block/blk-snap/cbt_map.c | 210 +++++
drivers/block/blk-snap/cbt_map.h | 62 ++
drivers/block/blk-snap/common.h | 29 +
drivers/block/blk-snap/ctrl_fops.c | 693 ++++++++++++++
drivers/block/blk-snap/ctrl_fops.h | 19 +
drivers/block/blk-snap/ctrl_pipe.c | 562 +++++++++++
drivers/block/blk-snap/ctrl_pipe.h | 34 +
drivers/block/blk-snap/ctrl_sysfs.c | 73 ++
drivers/block/blk-snap/ctrl_sysfs.h | 5 +
drivers/block/blk-snap/defer_io.c | 393 ++++++++
drivers/block/blk-snap/defer_io.h | 39 +
drivers/block/blk-snap/filter.c | 72 ++
drivers/block/blk-snap/filter.h | 7 +
drivers/block/blk-snap/main.c | 83 ++
drivers/block/blk-snap/params.c | 57 ++
drivers/block/blk-snap/params.h | 29 +
drivers/block/blk-snap/rangevector.c | 85 ++
drivers/block/blk-snap/rangevector.h | 31 +
drivers/block/blk-snap/snapimage.c | 982 ++++++++++++++++++++
drivers/block/blk-snap/snapimage.h | 16 +
drivers/block/blk-snap/snapshot.c | 228 +++++
drivers/block/blk-snap/snapshot.h | 17 +
drivers/block/blk-snap/snapstore.c | 929 ++++++++++++++++++
drivers/block/blk-snap/snapstore.h | 68 ++
drivers/block/blk-snap/snapstore_device.c | 532 +++++++++++
drivers/block/blk-snap/snapstore_device.h | 63 ++
drivers/block/blk-snap/snapstore_file.c | 52 ++
drivers/block/blk-snap/snapstore_file.h | 15 +
drivers/block/blk-snap/snapstore_mem.c | 89 ++
drivers/block/blk-snap/snapstore_mem.h | 20 +
drivers/block/blk-snap/snapstore_multidev.c | 118 +++
drivers/block/blk-snap/snapstore_multidev.h | 22 +
drivers/block/blk-snap/tracker.c | 521 +++++++++++
drivers/block/blk-snap/tracker.h | 50 +
drivers/block/blk-snap/tracking.c | 260 ++++++
drivers/block/blk-snap/tracking.h | 12 +
drivers/block/blk-snap/version.h | 7 +
include/linux/blk-filter.h | 41 +
include/linux/genhd.h | 3 +-
67 files changed, 9157 insertions(+), 6 deletions(-)
create mode 100644 block/blk-filter-internal.h
create mode 100644 block/blk-filter.c
create mode 100644 drivers/block/blk-snap/Kconfig
create mode 100644 drivers/block/blk-snap/Makefile
create mode 100644 drivers/block/blk-snap/big_buffer.c
create mode 100644 drivers/block/blk-snap/big_buffer.h
create mode 100644 drivers/block/blk-snap/blk-snap-ctl.h
create mode 100644 drivers/block/blk-snap/blk_deferred.c
create mode 100644 drivers/block/blk-snap/blk_deferred.h
create mode 100644 drivers/block/blk-snap/blk_descr_file.c
create mode 100644 drivers/block/blk-snap/blk_descr_file.h
create mode 100644 drivers/block/blk-snap/blk_descr_mem.c
create mode 100644 drivers/block/blk-snap/blk_descr_mem.h
create mode 100644 drivers/block/blk-snap/blk_descr_multidev.c
create mode 100644 drivers/block/blk-snap/blk_descr_multidev.h
create mode 100644 drivers/block/blk-snap/blk_descr_pool.c
create mode 100644 drivers/block/blk-snap/blk_descr_pool.h
create mode 100644 drivers/block/blk-snap/blk_redirect.c
create mode 100644 drivers/block/blk-snap/blk_redirect.h
create mode 100644 drivers/block/blk-snap/blk_util.c
create mode 100644 drivers/block/blk-snap/blk_util.h
create mode 100644 drivers/block/blk-snap/cbt_map.c
create mode 100644 drivers/block/blk-snap/cbt_map.h
create mode 100644 drivers/block/blk-snap/common.h
create mode 100644 drivers/block/blk-snap/ctrl_fops.c
create mode 100644 drivers/block/blk-snap/ctrl_fops.h
create mode 100644 drivers/block/blk-snap/ctrl_pipe.c
create mode 100644 drivers/block/blk-snap/ctrl_pipe.h
create mode 100644 drivers/block/blk-snap/ctrl_sysfs.c
create mode 100644 drivers/block/blk-snap/ctrl_sysfs.h
create mode 100644 drivers/block/blk-snap/defer_io.c
create mode 100644 drivers/block/blk-snap/defer_io.h
create mode 100644 drivers/block/blk-snap/filter.c
create mode 100644 drivers/block/blk-snap/filter.h
create mode 100644 drivers/block/blk-snap/main.c
create mode 100644 drivers/block/blk-snap/params.c
create mode 100644 drivers/block/blk-snap/params.h
create mode 100644 drivers/block/blk-snap/rangevector.c
create mode 100644 drivers/block/blk-snap/rangevector.h
create mode 100644 drivers/block/blk-snap/snapimage.c
create mode 100644 drivers/block/blk-snap/snapimage.h
create mode 100644 drivers/block/blk-snap/snapshot.c
create mode 100644 drivers/block/blk-snap/snapshot.h
create mode 100644 drivers/block/blk-snap/snapstore.c
create mode 100644 drivers/block/blk-snap/snapstore.h
create mode 100644 drivers/block/blk-snap/snapstore_device.c
create mode 100644 drivers/block/blk-snap/snapstore_device.h
create mode 100644 drivers/block/blk-snap/snapstore_file.c
create mode 100644 drivers/block/blk-snap/snapstore_file.h
create mode 100644 drivers/block/blk-snap/snapstore_mem.c
create mode 100644 drivers/block/blk-snap/snapstore_mem.h
create mode 100644 drivers/block/blk-snap/snapstore_multidev.c
create mode 100644 drivers/block/blk-snap/snapstore_multidev.h
create mode 100644 drivers/block/blk-snap/tracker.c
create mode 100644 drivers/block/blk-snap/tracker.h
create mode 100644 drivers/block/blk-snap/tracking.c
create mode 100644 drivers/block/blk-snap/tracking.h
create mode 100644 drivers/block/blk-snap/version.h
create mode 100644 include/linux/blk-filter.h
--
2.28.0
next reply other threads:[~2020-10-02 13:01 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-02 12:56 Sergei Shtepa [this message]
2020-10-02 12:56 ` [PATCH 1/1] blk-snap - Block snapshot module This module implements snapshot and changed block tracking functionality. It is intended to create backup copies of any block devices without usage of device-mapper Sergei Shtepa
2020-10-02 15:49 ` kernel test robot
2020-10-02 16:23 ` kernel test robot
[not found] ` <CALYGNiORw=DrKxoaoQPJP8TNM-S_W0Zdtpvra_eMs+Ri5f2P-g@mail.gmail.com>
2020-10-06 8:43 ` Sergei Shtepa
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1601643362-7370-1-git-send-email-sergei.shtepa@veeam.com \
--to=sergei.shtepa@veeam.com \
--cc=axboe@kernel.dk \
--cc=damien.lemoal@wdc.com \
--cc=davem@davemloft.net \
--cc=jack@suse.cz \
--cc=koct9i@gmail.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-kbuild@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mchehab+huawei@kernel.org \
--cc=ming.lei@redhat.com \
--cc=robh@kernel.org \
--cc=steve@sk2.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).