All of lore.kernel.org
 help / color / mirror / Atom feed
* consolidate btrfs checksumming, repair and bio splitting v2
@ 2022-11-20 12:47 Christoph Hellwig
  2022-11-20 12:47 ` [PATCH 01/19] block: export bio_split_rw Christoph Hellwig
                   ` (19 more replies)
  0 siblings, 20 replies; 38+ messages in thread
From: Christoph Hellwig @ 2022-11-20 12:47 UTC (permalink / raw)
  To: Chris Mason, Josef Bacik, David Sterba
  Cc: Damien Le Moal, Naohiro Aota, Johannes Thumshirn, Qu Wenruo,
	Jens Axboe, Darrick J. Wong, linux-block, linux-btrfs,
	linux-fsdevel

Hi all,

this series moves a large amount of duplicate code below btrfs_submit_bio
into what I call the 'storage' layer.  Instead of duplicating code to
checksum, check checksums and repair and split bios in all the caller
of btrfs_submit_bio (buffered I/O, direct I/O, compressed I/O, encoded
I/O), the work is done one in a central place, often more optiomal and
without slight changes in behavior.  Once that is done the upper layers
also don't need to split the bios for extent boundaries, as the storage
layer can do that itself, including splitting the bios for the zone
append limits for zoned I/O.

The split work is inspired by an earlier series from Qu, from which it
also reuses a few patches.

The rebasing against the latest misc-next was a bit painful due to the
various large cleanups, but very little logic changed, so I've kept the
review tags for now, but I'd appreciated another careful round of eyes.

A git tree is also available:

    git://git.infradead.org/users/hch/misc.git btrfs-bio-split

Gitweb:

    http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/btrfs-bio-split

Changes since v1:
 - rebased to latest btrfs/misc-next
 - added a new patch to remove the fs_info argument to btrfs_submit_bio
 - clean up the repair_one_sector calling convention a bit
 - don't move file_offset for ONE_ORDERED bios in btrfs_split_bio
 - temporarily use btrfs_zoned_get_device in alloc_compressed_bio
 - minor refactoring of some of the checksum helpers
 - split up a few patches
 - drop a few of the blk_status_t to int conversion for now
 - various whitespace fixes

Diffstat:
 block/blk-merge.c         |    3 
 fs/btrfs/bio.c            |  547 ++++++++++++++++++++++++++++++++++++----
 fs/btrfs/bio.h            |   67 +---
 fs/btrfs/btrfs_inode.h    |   22 -
 fs/btrfs/compression.c    |  273 +++-----------------
 fs/btrfs/compression.h    |    3 
 fs/btrfs/disk-io.c        |  199 +-------------
 fs/btrfs/disk-io.h        |   11 
 fs/btrfs/extent-io-tree.h |    1 
 fs/btrfs/extent_io.c      |  554 +++--------------------------------------
 fs/btrfs/extent_io.h      |   31 --
 fs/btrfs/file-item.c      |   74 +----
 fs/btrfs/file-item.h      |    8 
 fs/btrfs/fs.h             |    5 
 fs/btrfs/inode.c          |  621 ++++++----------------------------------------
 fs/btrfs/volumes.c        |  115 ++------
 fs/btrfs/volumes.h        |   18 -
 fs/btrfs/zoned.c          |   85 ++----
 fs/btrfs/zoned.h          |   16 -
 fs/iomap/direct-io.c      |   10 
 include/linux/bio.h       |    4 
 include/linux/iomap.h     |    1 
 22 files changed, 817 insertions(+), 1851 deletions(-)

^ permalink raw reply	[flat|nested] 38+ messages in thread
* consolidate btrfs checksumming, repair and bio splitting v3
@ 2023-01-12  9:05 Christoph Hellwig
  2023-01-12  9:05 ` [PATCH 10/19] btrfs: remove stripe boundary calculation for buffered I/O Christoph Hellwig
  0 siblings, 1 reply; 38+ messages in thread
From: Christoph Hellwig @ 2023-01-12  9:05 UTC (permalink / raw)
  To: Chris Mason, Josef Bacik, David Sterba
  Cc: Damien Le Moal, Naohiro Aota, Johannes Thumshirn, Qu Wenruo,
	Jens Axboe, Darrick J. Wong, linux-block, linux-btrfs,
	linux-fsdevel

Hi all,

this series moves a large amount of duplicate code below btrfs_submit_bio
into what I call the 'storage' layer.  Instead of duplicating code to
checksum, check checksums and repair and split bios in all the caller
of btrfs_submit_bio (buffered I/O, direct I/O, compressed I/O, encoded
I/O), the work is done one in a central place, often more optiomal and
without slight changes in behavior.  Once that is done the upper layers
also don't need to split the bios for extent boundaries, as the storage
layer can do that itself, including splitting the bios for the zone
append limits for zoned I/O.

The split work is inspired by an earlier series from Qu, from which it
also reuses a few patches.

Note that Dave commented on patch 2 to be split due to "doing multiple
things at once".  I've got not comment on how that should be done,
except for the discussion with Josef on the previous iteration, where
opting into the new functionality by the callers through a new flag
would be possible, but actually making the whole series more complex
and harder to bisect.  I can still do that even if I think it's not
very helpful, but I can't really think of any other split.

A git tree is also available:

    git://git.infradead.org/users/hch/misc.git btrfs-bio-split

Gitweb:

    http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/btrfs-bio-split

Changes since v2:
 - minor rebase to latest misc-next
 - pick up various reviews for all patches

Changes since v1:
 - rebased to latest btrfs/misc-next
 - added a new patch to remove the fs_info argument to btrfs_submit_bio
 - clean up the repair_one_sector calling convention a bit
 - don't move file_offset for ONE_ORDERED bios in btrfs_split_bio
 - temporarily use btrfs_zoned_get_device in alloc_compressed_bio
 - minor refactoring of some of the checksum helpers
 - split up a few patches
 - drop a few of the blk_status_t to int conversion for now
 - various whitespace fixes

Diffstat:
 block/blk-merge.c         |    3 
 fs/btrfs/bio.c            |  547 ++++++++++++++++++++++++++++++++++++----
 fs/btrfs/bio.h            |   67 +---
 fs/btrfs/btrfs_inode.h    |   22 -
 fs/btrfs/compression.c    |  273 +++-----------------
 fs/btrfs/compression.h    |    3 
 fs/btrfs/disk-io.c        |  203 +--------------
 fs/btrfs/disk-io.h        |   11 
 fs/btrfs/extent-io-tree.h |    1 
 fs/btrfs/extent_io.c      |  556 ++++-------------------------------------
 fs/btrfs/extent_io.h      |   31 --
 fs/btrfs/file-item.c      |   74 +----
 fs/btrfs/file-item.h      |    8 
 fs/btrfs/fs.h             |    5 
 fs/btrfs/inode.c          |  621 ++++++----------------------------------------
 fs/btrfs/volumes.c        |  115 ++------
 fs/btrfs/volumes.h        |   18 -
 fs/btrfs/zoned.c          |   85 ++----
 fs/btrfs/zoned.h          |   16 -
 fs/iomap/direct-io.c      |   10 
 include/linux/bio.h       |    4 
 include/linux/iomap.h     |    3 
 22 files changed, 819 insertions(+), 1857 deletions(-)

^ permalink raw reply	[flat|nested] 38+ messages in thread

end of thread, other threads:[~2023-01-12  9:12 UTC | newest]

Thread overview: 38+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-20 12:47 consolidate btrfs checksumming, repair and bio splitting v2 Christoph Hellwig
2022-11-20 12:47 ` [PATCH 01/19] block: export bio_split_rw Christoph Hellwig
2022-11-23 22:44   ` Chaitanya Kulkarni
2022-11-20 12:47 ` [PATCH 02/19] btrfs: handle checksum validation and repair at the storage layer Christoph Hellwig
2022-12-06 11:04   ` Johannes Thumshirn
2022-12-12 22:13   ` David Sterba
2022-12-13  5:53     ` Christoph Hellwig
2022-11-20 12:47 ` [PATCH 03/19] btrfs: remove the submit_bio_start helpers Christoph Hellwig
2022-12-06 11:13   ` Johannes Thumshirn
2022-11-20 12:47 ` [PATCH 04/19] btrfs: simplify the btrfs_csum_one_bio calling convention Christoph Hellwig
2022-12-06 11:16   ` Johannes Thumshirn
2022-11-20 12:47 ` [PATCH 05/19] btrfs: handle checksum generation in the storage layer Christoph Hellwig
2022-12-06 11:21   ` Johannes Thumshirn
2022-11-20 12:47 ` [PATCH 06/19] btrfs: handle recording of zoned writes " Christoph Hellwig
2022-11-20 12:47 ` [PATCH 07/19] btrfs: support cloned bios in btree_csum_one_bio Christoph Hellwig
2022-12-06 11:23   ` Johannes Thumshirn
2022-11-20 12:47 ` [PATCH 08/19] btrfs: allow btrfs_submit_bio to split bios Christoph Hellwig
2022-11-20 12:47 ` [PATCH 09/19] btrfs: pass the iomap bio to btrfs_submit_bio Christoph Hellwig
2022-12-06 11:27   ` Johannes Thumshirn
2022-11-20 12:47 ` [PATCH 10/19] btrfs: remove stripe boundary calculation for buffered I/O Christoph Hellwig
2022-12-06 11:29   ` Johannes Thumshirn
2022-11-20 12:47 ` [PATCH 11/19] btrfs: remove stripe boundary calculation for compressed I/O Christoph Hellwig
2022-11-20 12:47 ` [PATCH 12/19] btrfs: remove stripe boundary calculation for encoded I/O Christoph Hellwig
2022-11-20 12:47 ` [PATCH 13/19] btrfs: remove struct btrfs_io_geometry Christoph Hellwig
2022-12-06 11:41   ` Johannes Thumshirn
2022-11-20 12:47 ` [PATCH 14/19] btrfs: remove submit_encoded_read_bio Christoph Hellwig
2022-11-20 12:47 ` [PATCH 15/19] btrfs: remove the fs_info argument to btrfs_submit_bio Christoph Hellwig
2022-12-06 11:42   ` Johannes Thumshirn
2022-11-20 12:47 ` [PATCH 16/19] btrfs: remove now spurious bio submission helpers Christoph Hellwig
2022-11-20 12:47 ` [PATCH 17/19] btrfs: calculate file system wide queue limit for zoned mode Christoph Hellwig
2022-12-06 11:46   ` Johannes Thumshirn
2022-11-20 12:47 ` [PATCH 18/19] btrfs: split zone append bios in btrfs_submit_bio Christoph Hellwig
2022-12-06 11:51   ` Johannes Thumshirn
2022-11-20 12:47 ` [PATCH 19/19] iomap: remove IOMAP_F_ZONE_APPEND Christoph Hellwig
2022-11-21 17:07   ` Darrick J. Wong
2022-11-23 22:44   ` Chaitanya Kulkarni
2022-12-12 18:46 ` consolidate btrfs checksumming, repair and bio splitting v2 Josef Bacik
2023-01-12  9:05 consolidate btrfs checksumming, repair and bio splitting v3 Christoph Hellwig
2023-01-12  9:05 ` [PATCH 10/19] btrfs: remove stripe boundary calculation for buffered I/O Christoph Hellwig

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.