* [git pull] device mapper changes for 5.8
@ 2020-06-05 14:51 Mike Snitzer
2020-06-05 18:17 ` Mikulas Patocka
2020-06-05 19:16 ` [git pull v2] " Mike Snitzer
0 siblings, 2 replies; 6+ messages in thread
From: Mike Snitzer @ 2020-06-05 14:51 UTC (permalink / raw)
To: Linus Torvalds
Cc: dm-devel, linux-block, Alasdair G Kergon, Dmitry Baryshkov,
Eric Biggers, Gabriel Krisman Bertazi, Gustavo A. R. Silva,
Hannes Reinecke, Heinz Mauelshagen, Khazhismel Kumykov,
Martin Wilck, Mikulas Patocka, Nathan Chancellor, YueHaibing,
Zhiqiang Liu
Hi Linus,
For the following changes there is one dm-zoned-metadata.c conflict that
linux-next has been carrying for a while. See commit d77e96f277 ("Merge
remote-tracking branch 'device-mapper/for-next'") from next-20200605.
It resolves conflict from linux-block's commit 9398554fb3 ("block:
remove the error_sector argument to blkdev_issue_flush").
The following changes since commit 2ef96a5bb12be62ef75b5828c0aab838ebb29cb8:
Linux 5.7-rc5 (2020-05-10 15:16:58 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git tags/for-5.8/dm-changes
for you to fetch changes up to d776dc57aea4f0dfae2173c52538becfca54ab9f:
dm crypt: avoid truncating the logical block size (2020-06-04 20:57:59 -0400)
Please pull, thanks!
Mike
----------------------------------------------------------------
- Largest change for this cycle is the DM zoned target's metadata
version 2 feature that adds support for pairing regular block
devices with a zoned device to ease performance impact associated
with finite random zones of zoned device. Changes came in 3
batches: first prepared for and then added the ability to pair a
single regular block device, second was a batch of fixes to improve
zoned's reclaim heuristic, third removed the limitation of only
adding a single additional regular block device to allow many
devices. Testing has shown linear scaling as more devices are
added.
- Add new emulated block size (ebs) target that emulates a smaller
logical_block_size than a block device supports. Primary use-case
is to emulate "512e" devices that have 512 byte logical_block_size
and 4KB physical_block_size. This is useful to some legacy
applications otherwise wouldn't be ablee to be used on 4K devices
because they depend on issuing IO in 512 byte granularity.
- Add discard interfaces to DM bufio. First consumer of the interface
is the dm-ebs target that makes heavy use of dm-bufio.
- Add DM crypt support for using encryopted keys, fix crypt to sleep
and retry if crypto layer returns -ENOMEM, and fix crypt's block
queue_limits stacking to not truncate logic_block_size.
- Fix DM integrity to also sleep and retry in response to -ENOMEM from
the crypto layer. Also add status line info to Documentation.
- Switch DMDEBUG from a compile time config option to instead use
dynamic debug via pr_debug.
- Fix DM multipath target's hueristic for how it manages
"queue_if_no_path" state internally. DM multipath now avoids
disabling "queue_if_no_path" unless it is actually needed (e.g. in
response to configure timeout or explicit "fail_if_no_path"
message). This fixes reports of spurious -EIO being reported back
to userspace application during fault tolerance testing with an NVMe
backend. Added various dynamic DMDEBUG messages to assist with
debugging queue_if_no_path in the future.
- Add a new DM multipath "Historical Service Time" Path Selector.
- Fix DM multipath's dm_blk_ioctl() to switch paths on IO error.
- Improve DM writecache target performance by using explicit
cache flushing for target's single-threaded usecase and a small
cleanup to remove unnecessary test in persistent_memory_claim.
- Other small cleanups in DM core, dm-persistent-data, and DM integrity.
----------------------------------------------------------------
Dmitry Baryshkov (1):
dm crypt: support using encrypted keys
Eric Biggers (1):
dm crypt: avoid truncating the logical block size
Gabriel Krisman Bertazi (1):
dm mpath: pass IO start time to path selector
Gustavo A. R. Silva (1):
dm: replace zero-length array with flexible-array
Hannes Reinecke (38):
dm zoned: add 'status' callback
dm zoned: add 'message' callback
dm zoned: store zone id within the zone structure and kill dmz_id()
dm zoned: use array for superblock zones
dm zoned: store device in struct dmz_sb
dm zoned: move fields from struct dmz_dev to dmz_metadata
dm zoned: introduce dmz_metadata_label() to format device name
dm zoned: Introduce dmz_dev_is_dying() and dmz_check_dev()
dm zoned: remove 'dev' argument from reclaim
dm zoned: replace 'target' pointer in the bio context
dm zoned: use dmz_zone_to_dev() when handling metadata I/O
dm zoned: add metadata logging functions
dm zoned: Reduce logging output on startup
dm zoned: ignore metadata zone in dmz_alloc_zone()
dm zoned: metadata version 2
dm: use dynamic debug instead of compile-time config option
dm zoned: remove spurious newlines from debugging messages
dm zoned: return NULL if dmz_get_zone_for_reclaim() fails to find a zone
dm zoned: separate random and cache zones
dm zoned: reclaim random zones when idle
dm zoned: start reclaim with sequential zones
dm zoned: terminate reclaim on congestion
dm zoned: remove leftover hunk for switching to sequential zones
dm zoned: add debugging message for reading superblocks
dm zoned: avoid unnecessary device recalulation for secondary superblock
dm zoned: improve logging messages for reclaim
dm zoned: add a 'reserved' zone flag
dm zoned: convert to xarray
dm zoned: allocate temporary superblock for tertiary devices
dm zoned: add device pointer to struct dm_zone
dm zoned: add metadata pointer to struct dmz_dev
dm zoned: per-device reclaim
dm zoned: move random and sequential zones into struct dmz_dev
dm zoned: support arbitrary number of devices
dm zoned: allocate zone by device index
dm zoned: select reclaim zone based on device index
dm zoned: prefer full zones for reclaim
dm zoned: check superblock location
Heinz Mauelshagen (2):
dm: add emulated block size target
dm ebs: pass discards down to underlying device
Khazhismel Kumykov (1):
dm mpath: add Historical Service Time Path Selector
Martin Wilck (1):
dm mpath: switch paths in dm_blk_ioctl() code path
Mike Snitzer (5):
dm: use DMDEBUG macros now that they use pr_debug variants
dm mpath: simplify __must_push_back
dm mpath: restrict queue_if_no_path state machine
dm mpath: enhance queue_if_no_path debugging
dm mpath: add DM device name to Failing/Reinstating path log messages
Mikulas Patocka (10):
dm bufio: implement discard
dm writecache: remove superfluous test in persistent_memory_claim
dm writecache: improve performance on DDR persistent memory (Optane)
dm bufio: delete unused and inefficient dm_bufio_discard_buffers
dm crypt: sleep and retry on allocation errors
dm integrity: sleep and retry on allocation errors
dm integrity: add status line documentation
dm bufio: clean up rbtree block ordering
dm bufio: introduce forget_buffer_locked
dm ebs: use dm_bufio_forget_buffers
Nathan Chancellor (1):
dm zoned: Avoid 64-bit division error in dmz_fixup_devices
YueHaibing (1):
dm integrity: remove set but not used variables
Zhiqiang Liu (1):
dm persistent data: switch exit_ro_spine to return void
Documentation/admin-guide/device-mapper/dm-ebs.rst | 51 +
.../admin-guide/device-mapper/dm-integrity.rst | 8 +
.../admin-guide/device-mapper/dm-zoned.rst | 62 +-
drivers/md/Kconfig | 20 +
drivers/md/Makefile | 3 +
drivers/md/dm-bufio.c | 109 +-
drivers/md/dm-crypt.c | 91 +-
drivers/md/dm-ebs-target.c | 471 +++++++++
drivers/md/dm-historical-service-time.c | 561 +++++++++++
drivers/md/dm-integrity.c | 11 +-
drivers/md/dm-log-writes.c | 2 +-
drivers/md/dm-mpath.c | 123 ++-
drivers/md/dm-path-selector.h | 2 +-
drivers/md/dm-queue-length.c | 2 +-
drivers/md/dm-raid.c | 2 +-
drivers/md/dm-raid1.c | 2 +-
drivers/md/dm-service-time.c | 2 +-
drivers/md/dm-stats.c | 2 +-
drivers/md/dm-stripe.c | 2 +-
drivers/md/dm-switch.c | 2 +-
drivers/md/dm-writecache.c | 42 +-
drivers/md/dm-zoned-metadata.c | 1046 +++++++++++++++-----
drivers/md/dm-zoned-reclaim.c | 210 ++--
drivers/md/dm-zoned-target.c | 463 ++++++---
drivers/md/dm-zoned.h | 113 ++-
drivers/md/dm.c | 11 +-
drivers/md/persistent-data/dm-btree-internal.h | 4 +-
drivers/md/persistent-data/dm-btree-spine.c | 6 +-
include/linux/device-mapper.h | 9 +-
include/linux/dm-bufio.h | 12 +
30 files changed, 2794 insertions(+), 650 deletions(-)
create mode 100644 Documentation/admin-guide/device-mapper/dm-ebs.rst
create mode 100644 drivers/md/dm-ebs-target.c
create mode 100644 drivers/md/dm-historical-service-time.c
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [git pull] device mapper changes for 5.8
2020-06-05 14:51 [git pull] device mapper changes for 5.8 Mike Snitzer
@ 2020-06-05 18:17 ` Mikulas Patocka
2020-06-05 19:16 ` [git pull v2] " Mike Snitzer
1 sibling, 0 replies; 6+ messages in thread
From: Mikulas Patocka @ 2020-06-05 18:17 UTC (permalink / raw)
To: Mike Snitzer; +Cc: dm-devel
Hi
I'd like to ask to remove these two patches:
> dm crypt: sleep and retry on allocation errors
> dm integrity: sleep and retry on allocation errors
Retrying the request in dm-crypt is not safe because crypt_convert_block
already advanced to the next block (even in the case of allocation failure
and an error).
We must retry in crypt_convert_block_skcipher, now I am trying to find out
how to do it properly.
Mikulas
^ permalink raw reply [flat|nested] 6+ messages in thread
* [git pull v2] device mapper changes for 5.8
2020-06-05 14:51 [git pull] device mapper changes for 5.8 Mike Snitzer
2020-06-05 18:17 ` Mikulas Patocka
@ 2020-06-05 19:16 ` Mike Snitzer
2020-06-05 23:15 ` pr-tracker-bot
2020-06-09 13:59 ` [git pull v3] " Mike Snitzer
1 sibling, 2 replies; 6+ messages in thread
From: Mike Snitzer @ 2020-06-05 19:16 UTC (permalink / raw)
To: Linus Torvalds
Cc: dm-devel, linux-block, Alasdair G Kergon, Dmitry Baryshkov,
Eric Biggers, Gabriel Krisman Bertazi, Hannes Reinecke,
Heinz Mauelshagen, Khazhismel Kumykov, Martin Wilck,
Mikulas Patocka, Nathan Chancellor, YueHaibing, Zhiqiang Liu
Hi Linus,
I had some miscommunication with Mikulas on his -ENOMEM sleep and retry
changes for dm-crypt and dm-integrity, he no longer thinks them
appropriate: https://www.redhat.com/archives/dm-devel/2020-June/msg00061.html
So I dropped them via rebase and updated the tag (to avoid any
potential for linux-stable churn due to them having cc'd
linux-stable). Hope that was the correct way forward...
For the following changes there is one dm-zoned-metadata.c conflict that
linux-next has been carrying for a while. See commit d77e96f277 ("Merge
remote-tracking branch 'device-mapper/for-next'") from next-20200605.
It resolves conflict from linux-block's commit 9398554fb3 ("block:
remove the_error_sector argument to blkdev_issue_flush").
The following changes since commit 2ef96a5bb12be62ef75b5828c0aab838ebb29cb8:
Linux 5.7-rc5 (2020-05-10 15:16:58 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git tags/for-5.8/dm-changes
for you to fetch changes up to 64611a15ca9da91ff532982429c44686f4593b5f:
dm crypt: avoid truncating the logical block size (2020-06-05 14:59:59 -0400)
Please pull, thanks!
Mike
----------------------------------------------------------------
- Largest change for this cycle is the DM zoned target's metadata
version 2 feature that adds support for pairing regular block
devices with a zoned device to ease performance impact associated
with finite random zones of zoned device. Changes came in 3
batches: first prepared for and then added the ability to pair a
single regular block device, second was a batch of fixes to improve
zoned's reclaim heuristic, third removed the limitation of only
adding a single additional regular block device to allow many
devices. Testing has shown linear scaling as more devices are
added.
- Add new emulated block size (ebs) target that emulates a smaller
logical_block_size than a block device supports. Primary use-case
is to emulate "512e" devices that have 512 byte logical_block_size
and 4KB physical_block_size. This is useful to some legacy
applications otherwise wouldn't be ablee to be used on 4K devices
because they depend on issuing IO in 512 byte granularity.
- Add discard interfaces to DM bufio. First consumer of the interface
is the dm-ebs target that makes heavy use of dm-bufio.
- Fix DM crypt's block queue_limits stacking to not truncate
logic_block_size.
- Add Documentation for DM integrity's status line.
- Switch DMDEBUG from a compile time config option to instead use
dynamic debug via pr_debug.
- Fix DM multipath target's hueristic for how it manages
"queue_if_no_path" state internally. DM multipath now avoids
disabling "queue_if_no_path" unless it is actually needed (e.g. in
response to configure timeout or explicit "fail_if_no_path"
message). This fixes reports of spurious -EIO being reported back
to userspace application during fault tolerance testing with an NVMe
backend. Added various dynamic DMDEBUG messages to assist with
debugging queue_if_no_path in the future.
- Add a new DM multipath "Historical Service Time" Path Selector.
- Fix DM multipath's dm_blk_ioctl() to switch paths on IO error.
- Improve DM writecache target performance by using explicit
cache flushing for target's single-threaded usecase and a small
cleanup to remove unnecessary test in persistent_memory_claim.
- Other small cleanups in DM core, dm-persistent-data, and DM integrity.
----------------------------------------------------------------
Dmitry Baryshkov (1):
dm crypt: support using encrypted keys
Eric Biggers (1):
dm crypt: avoid truncating the logical block size
Gabriel Krisman Bertazi (1):
dm mpath: pass IO start time to path selector
Gustavo A. R. Silva (1):
dm: replace zero-length array with flexible-array
Hannes Reinecke (38):
dm zoned: add 'status' callback
dm zoned: add 'message' callback
dm zoned: store zone id within the zone structure and kill dmz_id()
dm zoned: use array for superblock zones
dm zoned: store device in struct dmz_sb
dm zoned: move fields from struct dmz_dev to dmz_metadata
dm zoned: introduce dmz_metadata_label() to format device name
dm zoned: Introduce dmz_dev_is_dying() and dmz_check_dev()
dm zoned: remove 'dev' argument from reclaim
dm zoned: replace 'target' pointer in the bio context
dm zoned: use dmz_zone_to_dev() when handling metadata I/O
dm zoned: add metadata logging functions
dm zoned: Reduce logging output on startup
dm zoned: ignore metadata zone in dmz_alloc_zone()
dm zoned: metadata version 2
dm: use dynamic debug instead of compile-time config option
dm zoned: remove spurious newlines from debugging messages
dm zoned: return NULL if dmz_get_zone_for_reclaim() fails to find a zone
dm zoned: separate random and cache zones
dm zoned: reclaim random zones when idle
dm zoned: start reclaim with sequential zones
dm zoned: terminate reclaim on congestion
dm zoned: remove leftover hunk for switching to sequential zones
dm zoned: add debugging message for reading superblocks
dm zoned: avoid unnecessary device recalulation for secondary superblock
dm zoned: improve logging messages for reclaim
dm zoned: add a 'reserved' zone flag
dm zoned: convert to xarray
dm zoned: allocate temporary superblock for tertiary devices
dm zoned: add device pointer to struct dm_zone
dm zoned: add metadata pointer to struct dmz_dev
dm zoned: per-device reclaim
dm zoned: move random and sequential zones into struct dmz_dev
dm zoned: support arbitrary number of devices
dm zoned: allocate zone by device index
dm zoned: select reclaim zone based on device index
dm zoned: prefer full zones for reclaim
dm zoned: check superblock location
Heinz Mauelshagen (2):
dm: add emulated block size target
dm ebs: pass discards down to underlying device
Khazhismel Kumykov (1):
dm mpath: add Historical Service Time Path Selector
Martin Wilck (1):
dm mpath: switch paths in dm_blk_ioctl() code path
Mike Snitzer (5):
dm: use DMDEBUG macros now that they use pr_debug variants
dm mpath: simplify __must_push_back
dm mpath: restrict queue_if_no_path state machine
dm mpath: enhance queue_if_no_path debugging
dm mpath: add DM device name to Failing/Reinstating path log messages
Mikulas Patocka (8):
dm bufio: implement discard
dm writecache: remove superfluous test in persistent_memory_claim
dm writecache: improve performance on DDR persistent memory (Optane)
dm bufio: delete unused and inefficient dm_bufio_discard_buffers
dm integrity: add status line documentation
dm bufio: clean up rbtree block ordering
dm bufio: introduce forget_buffer_locked
dm ebs: use dm_bufio_forget_buffers
Nathan Chancellor (1):
dm zoned: Avoid 64-bit division error in dmz_fixup_devices
YueHaibing (1):
dm integrity: remove set but not used variables
Zhiqiang Liu (1):
dm persistent data: switch exit_ro_spine to return void
Documentation/admin-guide/device-mapper/dm-ebs.rst | 51 +
.../admin-guide/device-mapper/dm-integrity.rst | 8 +
.../admin-guide/device-mapper/dm-zoned.rst | 62 +-
drivers/md/Kconfig | 20 +
drivers/md/Makefile | 3 +
drivers/md/dm-bufio.c | 109 +-
drivers/md/dm-crypt.c | 80 +-
drivers/md/dm-ebs-target.c | 471 +++++++++
drivers/md/dm-historical-service-time.c | 561 +++++++++++
drivers/md/dm-integrity.c | 6 +-
drivers/md/dm-log-writes.c | 2 +-
drivers/md/dm-mpath.c | 123 ++-
drivers/md/dm-path-selector.h | 2 +-
drivers/md/dm-queue-length.c | 2 +-
drivers/md/dm-raid.c | 2 +-
drivers/md/dm-raid1.c | 2 +-
drivers/md/dm-service-time.c | 2 +-
drivers/md/dm-stats.c | 2 +-
drivers/md/dm-stripe.c | 2 +-
drivers/md/dm-switch.c | 2 +-
drivers/md/dm-writecache.c | 42 +-
drivers/md/dm-zoned-metadata.c | 1046 +++++++++++++++-----
drivers/md/dm-zoned-reclaim.c | 210 ++--
drivers/md/dm-zoned-target.c | 463 ++++++---
drivers/md/dm-zoned.h | 113 ++-
drivers/md/dm.c | 11 +-
drivers/md/persistent-data/dm-btree-internal.h | 4 +-
drivers/md/persistent-data/dm-btree-spine.c | 6 +-
include/linux/device-mapper.h | 9 +-
include/linux/dm-bufio.h | 12 +
30 files changed, 2779 insertions(+), 649 deletions(-)
create mode 100644 Documentation/admin-guide/device-mapper/dm-ebs.rst
create mode 100644 drivers/md/dm-ebs-target.c
create mode 100644 drivers/md/dm-historical-service-time.c
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [git pull v2] device mapper changes for 5.8
2020-06-05 19:16 ` [git pull v2] " Mike Snitzer
@ 2020-06-05 23:15 ` pr-tracker-bot
2020-06-09 13:59 ` [git pull v3] " Mike Snitzer
1 sibling, 0 replies; 6+ messages in thread
From: pr-tracker-bot @ 2020-06-05 23:15 UTC (permalink / raw)
To: Mike Snitzer
Cc: Linus Torvalds, dm-devel, linux-block, Alasdair G Kergon,
Dmitry Baryshkov, Eric Biggers, Gabriel Krisman Bertazi,
Hannes Reinecke, Heinz Mauelshagen, Khazhismel Kumykov,
Martin Wilck, Mikulas Patocka, Nathan Chancellor, YueHaibing,
Zhiqiang Liu
The pull request you sent on Fri, 5 Jun 2020 15:16:13 -0400:
> git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git tags/for-5.8/dm-changes
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/b25c6644bfd3affd7d0127ce95c5c96c155a7515
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.wiki.kernel.org/userdoc/prtracker
^ permalink raw reply [flat|nested] 6+ messages in thread
* [git pull v3] device mapper changes for 5.8
2020-06-05 19:16 ` [git pull v2] " Mike Snitzer
2020-06-05 23:15 ` pr-tracker-bot
@ 2020-06-09 13:59 ` Mike Snitzer
2020-06-10 12:44 ` Mike Snitzer
1 sibling, 1 reply; 6+ messages in thread
From: Mike Snitzer @ 2020-06-09 13:59 UTC (permalink / raw)
To: Linus Torvalds
Cc: dm-devel, linux-block, Alasdair G Kergon, Dmitry Baryshkov,
Eric Biggers, Gabriel Krisman Bertazi, Gustavo A. R. Silva,
Hannes Reinecke, Heinz Mauelshagen, Khazhismel Kumykov,
Martin Wilck, Mikulas Patocka, Nathan Chancellor, YueHaibing,
Zhiqiang Liu
[resending as v3 due to my From: email addr in v2 being inexplicably wrong]
Hi Linus,
I had some miscommunication with Mikulas on his -ENOMEM sleep and retry
changes for dm-crypt and dm-integrity, he no longer thinks them
appropriate: https://www.redhat.com/archives/dm-devel/2020-June/msg00061.html
So I dropped them via rebase and updated the tag (to avoid any
potential for linux-stable churn due to Mikulas' commits having
cc'd linux-stable). Hope that was the correct way forward...
For the following changes there is one dm-zoned-metadata.c conflict that
linux-next has been carrying for a while. See commit d77e96f277 ("Merge
remote-tracking branch 'device-mapper/for-next'") from next-20200605.
It resolves conflict from linux-block's commit 9398554fb3 ("block:
remove the_error_sector argument to blkdev_issue_flush").
The following changes since commit 2ef96a5bb12be62ef75b5828c0aab838ebb29cb8:
Linux 5.7-rc5 (2020-05-10 15:16:58 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git tags/for-5.8/dm-changes
for you to fetch changes up to 64611a15ca9da91ff532982429c44686f4593b5f:
dm crypt: avoid truncating the logical block size (2020-06-05 14:59:59 -0400)
Please pull, thanks!
Mike
----------------------------------------------------------------
- Largest change for this cycle is the DM zoned target's metadata
version 2 feature that adds support for pairing regular block
devices with a zoned device to ease performance impact associated
with finite random zones of zoned device. Changes came in 3
batches: first prepared for and then added the ability to pair a
single regular block device, second was a batch of fixes to improve
zoned's reclaim heuristic, third removed the limitation of only
adding a single additional regular block device to allow many
devices. Testing has shown linear scaling as more devices are
added.
- Add new emulated block size (ebs) target that emulates a smaller
logical_block_size than a block device supports. Primary use-case
is to emulate "512e" devices that have 512 byte logical_block_size
and 4KB physical_block_size. This is useful to some legacy
applications otherwise wouldn't be ablee to be used on 4K devices
because they depend on issuing IO in 512 byte granularity.
- Add discard interfaces to DM bufio. First consumer of the interface
is the dm-ebs target that makes heavy use of dm-bufio.
- Fix DM crypt's block queue_limits stacking to not truncate
logic_block_size.
- Add Documentation for DM integrity's status line.
- Switch DMDEBUG from a compile time config option to instead use
dynamic debug via pr_debug.
- Fix DM multipath target's hueristic for how it manages
"queue_if_no_path" state internally. DM multipath now avoids
disabling "queue_if_no_path" unless it is actually needed (e.g. in
response to configure timeout or explicit "fail_if_no_path"
message). This fixes reports of spurious -EIO being reported back
to userspace application during fault tolerance testing with an NVMe
backend. Added various dynamic DMDEBUG messages to assist with
debugging queue_if_no_path in the future.
- Add a new DM multipath "Historical Service Time" Path Selector.
- Fix DM multipath's dm_blk_ioctl() to switch paths on IO error.
- Improve DM writecache target performance by using explicit
cache flushing for target's single-threaded usecase and a small
cleanup to remove unnecessary test in persistent_memory_claim.
- Other small cleanups in DM core, dm-persistent-data, and DM integrity.
----------------------------------------------------------------
Dmitry Baryshkov (1):
dm crypt: support using encrypted keys
Eric Biggers (1):
dm crypt: avoid truncating the logical block size
Gabriel Krisman Bertazi (1):
dm mpath: pass IO start time to path selector
Gustavo A. R. Silva (1):
dm: replace zero-length array with flexible-array
Hannes Reinecke (38):
dm zoned: add 'status' callback
dm zoned: add 'message' callback
dm zoned: store zone id within the zone structure and kill dmz_id()
dm zoned: use array for superblock zones
dm zoned: store device in struct dmz_sb
dm zoned: move fields from struct dmz_dev to dmz_metadata
dm zoned: introduce dmz_metadata_label() to format device name
dm zoned: Introduce dmz_dev_is_dying() and dmz_check_dev()
dm zoned: remove 'dev' argument from reclaim
dm zoned: replace 'target' pointer in the bio context
dm zoned: use dmz_zone_to_dev() when handling metadata I/O
dm zoned: add metadata logging functions
dm zoned: Reduce logging output on startup
dm zoned: ignore metadata zone in dmz_alloc_zone()
dm zoned: metadata version 2
dm: use dynamic debug instead of compile-time config option
dm zoned: remove spurious newlines from debugging messages
dm zoned: return NULL if dmz_get_zone_for_reclaim() fails to find a zone
dm zoned: separate random and cache zones
dm zoned: reclaim random zones when idle
dm zoned: start reclaim with sequential zones
dm zoned: terminate reclaim on congestion
dm zoned: remove leftover hunk for switching to sequential zones
dm zoned: add debugging message for reading superblocks
dm zoned: avoid unnecessary device recalulation for secondary superblock
dm zoned: improve logging messages for reclaim
dm zoned: add a 'reserved' zone flag
dm zoned: convert to xarray
dm zoned: allocate temporary superblock for tertiary devices
dm zoned: add device pointer to struct dm_zone
dm zoned: add metadata pointer to struct dmz_dev
dm zoned: per-device reclaim
dm zoned: move random and sequential zones into struct dmz_dev
dm zoned: support arbitrary number of devices
dm zoned: allocate zone by device index
dm zoned: select reclaim zone based on device index
dm zoned: prefer full zones for reclaim
dm zoned: check superblock location
Heinz Mauelshagen (2):
dm: add emulated block size target
dm ebs: pass discards down to underlying device
Khazhismel Kumykov (1):
dm mpath: add Historical Service Time Path Selector
Martin Wilck (1):
dm mpath: switch paths in dm_blk_ioctl() code path
Mike Snitzer (5):
dm: use DMDEBUG macros now that they use pr_debug variants
dm mpath: simplify __must_push_back
dm mpath: restrict queue_if_no_path state machine
dm mpath: enhance queue_if_no_path debugging
dm mpath: add DM device name to Failing/Reinstating path log messages
Mikulas Patocka (8):
dm bufio: implement discard
dm writecache: remove superfluous test in persistent_memory_claim
dm writecache: improve performance on DDR persistent memory (Optane)
dm bufio: delete unused and inefficient dm_bufio_discard_buffers
dm integrity: add status line documentation
dm bufio: clean up rbtree block ordering
dm bufio: introduce forget_buffer_locked
dm ebs: use dm_bufio_forget_buffers
Nathan Chancellor (1):
dm zoned: Avoid 64-bit division error in dmz_fixup_devices
YueHaibing (1):
dm integrity: remove set but not used variables
Zhiqiang Liu (1):
dm persistent data: switch exit_ro_spine to return void
Documentation/admin-guide/device-mapper/dm-ebs.rst | 51 +
.../admin-guide/device-mapper/dm-integrity.rst | 8 +
.../admin-guide/device-mapper/dm-zoned.rst | 62 +-
drivers/md/Kconfig | 20 +
drivers/md/Makefile | 3 +
drivers/md/dm-bufio.c | 109 +-
drivers/md/dm-crypt.c | 80 +-
drivers/md/dm-ebs-target.c | 471 +++++++++
drivers/md/dm-historical-service-time.c | 561 +++++++++++
drivers/md/dm-integrity.c | 6 +-
drivers/md/dm-log-writes.c | 2 +-
drivers/md/dm-mpath.c | 123 ++-
drivers/md/dm-path-selector.h | 2 +-
drivers/md/dm-queue-length.c | 2 +-
drivers/md/dm-raid.c | 2 +-
drivers/md/dm-raid1.c | 2 +-
drivers/md/dm-service-time.c | 2 +-
drivers/md/dm-stats.c | 2 +-
drivers/md/dm-stripe.c | 2 +-
drivers/md/dm-switch.c | 2 +-
drivers/md/dm-writecache.c | 42 +-
drivers/md/dm-zoned-metadata.c | 1046 +++++++++++++++-----
drivers/md/dm-zoned-reclaim.c | 210 ++--
drivers/md/dm-zoned-target.c | 463 ++++++---
drivers/md/dm-zoned.h | 113 ++-
drivers/md/dm.c | 11 +-
drivers/md/persistent-data/dm-btree-internal.h | 4 +-
drivers/md/persistent-data/dm-btree-spine.c | 6 +-
include/linux/device-mapper.h | 9 +-
include/linux/dm-bufio.h | 12 +
30 files changed, 2779 insertions(+), 649 deletions(-)
create mode 100644 Documentation/admin-guide/device-mapper/dm-ebs.rst
create mode 100644 drivers/md/dm-ebs-target.c
create mode 100644 drivers/md/dm-historical-service-time.c
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [git pull v3] device mapper changes for 5.8
2020-06-09 13:59 ` [git pull v3] " Mike Snitzer
@ 2020-06-10 12:44 ` Mike Snitzer
0 siblings, 0 replies; 6+ messages in thread
From: Mike Snitzer @ 2020-06-10 12:44 UTC (permalink / raw)
To: Linus Torvalds
Cc: dm-devel, linux-block, Alasdair G Kergon, Dmitry Baryshkov,
Eric Biggers, Gabriel Krisman Bertazi, Gustavo A. R. Silva,
Hannes Reinecke, Heinz Mauelshagen, Khazhismel Kumykov,
Martin Wilck, Mikulas Patocka, Nathan Chancellor, YueHaibing,
Zhiqiang Liu
On Tue, Jun 09 2020 at 9:59am -0400,
Mike Snitzer <snitzer@redhat.com> wrote:
> [resending as v3 due to my From: email addr in v2 being inexplicably wrong]
The pr-tracker-bot responded to that email address and I missed it:
https://www.redhat.com/archives/dm-devel/2020-June/msg00067.html
I should've just looked at your tree. Thanks for your quick pull.
Sorry for the noise.
Mike
>
> Hi Linus,
>
> I had some miscommunication with Mikulas on his -ENOMEM sleep and retry
> changes for dm-crypt and dm-integrity, he no longer thinks them
> appropriate: https://www.redhat.com/archives/dm-devel/2020-June/msg00061.html
> So I dropped them via rebase and updated the tag (to avoid any
> potential for linux-stable churn due to Mikulas' commits having
> cc'd linux-stable). Hope that was the correct way forward...
>
> For the following changes there is one dm-zoned-metadata.c conflict that
> linux-next has been carrying for a while. See commit d77e96f277 ("Merge
> remote-tracking branch 'device-mapper/for-next'") from next-20200605.
> It resolves conflict from linux-block's commit 9398554fb3 ("block:
> remove the_error_sector argument to blkdev_issue_flush").
>
> The following changes since commit 2ef96a5bb12be62ef75b5828c0aab838ebb29cb8:
>
> Linux 5.7-rc5 (2020-05-10 15:16:58 -0700)
>
> are available in the Git repository at:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git tags/for-5.8/dm-changes
>
> for you to fetch changes up to 64611a15ca9da91ff532982429c44686f4593b5f:
>
> dm crypt: avoid truncating the logical block size (2020-06-05 14:59:59 -0400)
>
> Please pull, thanks!
> Mike
>
> ----------------------------------------------------------------
> - Largest change for this cycle is the DM zoned target's metadata
> version 2 feature that adds support for pairing regular block
> devices with a zoned device to ease performance impact associated
> with finite random zones of zoned device. Changes came in 3
> batches: first prepared for and then added the ability to pair a
> single regular block device, second was a batch of fixes to improve
> zoned's reclaim heuristic, third removed the limitation of only
> adding a single additional regular block device to allow many
> devices. Testing has shown linear scaling as more devices are
> added.
>
> - Add new emulated block size (ebs) target that emulates a smaller
> logical_block_size than a block device supports. Primary use-case
> is to emulate "512e" devices that have 512 byte logical_block_size
> and 4KB physical_block_size. This is useful to some legacy
> applications otherwise wouldn't be ablee to be used on 4K devices
> because they depend on issuing IO in 512 byte granularity.
>
> - Add discard interfaces to DM bufio. First consumer of the interface
> is the dm-ebs target that makes heavy use of dm-bufio.
>
> - Fix DM crypt's block queue_limits stacking to not truncate
> logic_block_size.
>
> - Add Documentation for DM integrity's status line.
>
> - Switch DMDEBUG from a compile time config option to instead use
> dynamic debug via pr_debug.
>
> - Fix DM multipath target's hueristic for how it manages
> "queue_if_no_path" state internally. DM multipath now avoids
> disabling "queue_if_no_path" unless it is actually needed (e.g. in
> response to configure timeout or explicit "fail_if_no_path"
> message). This fixes reports of spurious -EIO being reported back
> to userspace application during fault tolerance testing with an NVMe
> backend. Added various dynamic DMDEBUG messages to assist with
> debugging queue_if_no_path in the future.
>
> - Add a new DM multipath "Historical Service Time" Path Selector.
>
> - Fix DM multipath's dm_blk_ioctl() to switch paths on IO error.
>
> - Improve DM writecache target performance by using explicit
> cache flushing for target's single-threaded usecase and a small
> cleanup to remove unnecessary test in persistent_memory_claim.
>
> - Other small cleanups in DM core, dm-persistent-data, and DM integrity.
>
> ----------------------------------------------------------------
> Dmitry Baryshkov (1):
> dm crypt: support using encrypted keys
>
> Eric Biggers (1):
> dm crypt: avoid truncating the logical block size
>
> Gabriel Krisman Bertazi (1):
> dm mpath: pass IO start time to path selector
>
> Gustavo A. R. Silva (1):
> dm: replace zero-length array with flexible-array
>
> Hannes Reinecke (38):
> dm zoned: add 'status' callback
> dm zoned: add 'message' callback
> dm zoned: store zone id within the zone structure and kill dmz_id()
> dm zoned: use array for superblock zones
> dm zoned: store device in struct dmz_sb
> dm zoned: move fields from struct dmz_dev to dmz_metadata
> dm zoned: introduce dmz_metadata_label() to format device name
> dm zoned: Introduce dmz_dev_is_dying() and dmz_check_dev()
> dm zoned: remove 'dev' argument from reclaim
> dm zoned: replace 'target' pointer in the bio context
> dm zoned: use dmz_zone_to_dev() when handling metadata I/O
> dm zoned: add metadata logging functions
> dm zoned: Reduce logging output on startup
> dm zoned: ignore metadata zone in dmz_alloc_zone()
> dm zoned: metadata version 2
> dm: use dynamic debug instead of compile-time config option
> dm zoned: remove spurious newlines from debugging messages
> dm zoned: return NULL if dmz_get_zone_for_reclaim() fails to find a zone
> dm zoned: separate random and cache zones
> dm zoned: reclaim random zones when idle
> dm zoned: start reclaim with sequential zones
> dm zoned: terminate reclaim on congestion
> dm zoned: remove leftover hunk for switching to sequential zones
> dm zoned: add debugging message for reading superblocks
> dm zoned: avoid unnecessary device recalulation for secondary superblock
> dm zoned: improve logging messages for reclaim
> dm zoned: add a 'reserved' zone flag
> dm zoned: convert to xarray
> dm zoned: allocate temporary superblock for tertiary devices
> dm zoned: add device pointer to struct dm_zone
> dm zoned: add metadata pointer to struct dmz_dev
> dm zoned: per-device reclaim
> dm zoned: move random and sequential zones into struct dmz_dev
> dm zoned: support arbitrary number of devices
> dm zoned: allocate zone by device index
> dm zoned: select reclaim zone based on device index
> dm zoned: prefer full zones for reclaim
> dm zoned: check superblock location
>
> Heinz Mauelshagen (2):
> dm: add emulated block size target
> dm ebs: pass discards down to underlying device
>
> Khazhismel Kumykov (1):
> dm mpath: add Historical Service Time Path Selector
>
> Martin Wilck (1):
> dm mpath: switch paths in dm_blk_ioctl() code path
>
> Mike Snitzer (5):
> dm: use DMDEBUG macros now that they use pr_debug variants
> dm mpath: simplify __must_push_back
> dm mpath: restrict queue_if_no_path state machine
> dm mpath: enhance queue_if_no_path debugging
> dm mpath: add DM device name to Failing/Reinstating path log messages
>
> Mikulas Patocka (8):
> dm bufio: implement discard
> dm writecache: remove superfluous test in persistent_memory_claim
> dm writecache: improve performance on DDR persistent memory (Optane)
> dm bufio: delete unused and inefficient dm_bufio_discard_buffers
> dm integrity: add status line documentation
> dm bufio: clean up rbtree block ordering
> dm bufio: introduce forget_buffer_locked
> dm ebs: use dm_bufio_forget_buffers
>
> Nathan Chancellor (1):
> dm zoned: Avoid 64-bit division error in dmz_fixup_devices
>
> YueHaibing (1):
> dm integrity: remove set but not used variables
>
> Zhiqiang Liu (1):
> dm persistent data: switch exit_ro_spine to return void
>
> Documentation/admin-guide/device-mapper/dm-ebs.rst | 51 +
> .../admin-guide/device-mapper/dm-integrity.rst | 8 +
> .../admin-guide/device-mapper/dm-zoned.rst | 62 +-
> drivers/md/Kconfig | 20 +
> drivers/md/Makefile | 3 +
> drivers/md/dm-bufio.c | 109 +-
> drivers/md/dm-crypt.c | 80 +-
> drivers/md/dm-ebs-target.c | 471 +++++++++
> drivers/md/dm-historical-service-time.c | 561 +++++++++++
> drivers/md/dm-integrity.c | 6 +-
> drivers/md/dm-log-writes.c | 2 +-
> drivers/md/dm-mpath.c | 123 ++-
> drivers/md/dm-path-selector.h | 2 +-
> drivers/md/dm-queue-length.c | 2 +-
> drivers/md/dm-raid.c | 2 +-
> drivers/md/dm-raid1.c | 2 +-
> drivers/md/dm-service-time.c | 2 +-
> drivers/md/dm-stats.c | 2 +-
> drivers/md/dm-stripe.c | 2 +-
> drivers/md/dm-switch.c | 2 +-
> drivers/md/dm-writecache.c | 42 +-
> drivers/md/dm-zoned-metadata.c | 1046 +++++++++++++++-----
> drivers/md/dm-zoned-reclaim.c | 210 ++--
> drivers/md/dm-zoned-target.c | 463 ++++++---
> drivers/md/dm-zoned.h | 113 ++-
> drivers/md/dm.c | 11 +-
> drivers/md/persistent-data/dm-btree-internal.h | 4 +-
> drivers/md/persistent-data/dm-btree-spine.c | 6 +-
> include/linux/device-mapper.h | 9 +-
> include/linux/dm-bufio.h | 12 +
> 30 files changed, 2779 insertions(+), 649 deletions(-)
> create mode 100644 Documentation/admin-guide/device-mapper/dm-ebs.rst
> create mode 100644 drivers/md/dm-ebs-target.c
> create mode 100644 drivers/md/dm-historical-service-time.c
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-06-10 12:44 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-05 14:51 [git pull] device mapper changes for 5.8 Mike Snitzer
2020-06-05 18:17 ` Mikulas Patocka
2020-06-05 19:16 ` [git pull v2] " Mike Snitzer
2020-06-05 23:15 ` pr-tracker-bot
2020-06-09 13:59 ` [git pull v3] " Mike Snitzer
2020-06-10 12:44 ` Mike Snitzer
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.