Linux-BTRFS Archive on
 help / color / Atom feed
From: David Sterba <>
Cc: David Sterba <>,,
Subject: [GIT PULL] Btrfs updates for 5.9
Date: Fri, 31 Jul 2020 18:42:03 +0200
Message-ID: <> (raw)


we don't have any big feature updates this time, there are lots of small
enhacements or fixes. A hilight perhaps are the parallel fsync
performance improvements, numbers below.

Regarding the dio/iomap that was reverted last time, the required API
changes are likely to land in the upcoming cycle, the btrfs part will be
updated afterwards.

Please pull, thanks.


User visible changes:

- new mount option rescue= to group all recovery-related mount options
  so we don't have many specific options, currently introducing only
  aliases for existing options, future extensions are in development to
  allow read-only mount with partially damaged structures
  - usebackuproot is an alias for rescue=usebackuproot
  - nologreplay is an alias for rescue=nologreplay

- start deprecation of mount option inode_cache, removal scheduled to

- removed deprecated mount options alloc_start and subvolrootid

- device stats corruption counter gets incremented when a checksum
  mismatch is found

- qgroup information exported in sysfs /sys/fs/btrfs/<UUID>/qgroups/<id>

- add link /sys/fs/btrfs/<UUID>/bdi pointing to the associated backing
  dev info

- FS_INFO ioctl enhancements:
  - add flags to request/describe newly added items
  - new item: numeric checksum type and checksum size
  - new item: generation
  - new item: metadata_uuid

- seed device: with one new read-write device added, print the new
  device information in /proc/mounts

- balance: detect cancellation by Ctrl-C in existing cancellation points

Performance improvements:

- optimized versions of various helpers on little-endian architectures,
  where we don't have to do LE/BE conversion from on-disk format

- tree-log/fsync optimizations leading to lower max latency reported by
  dbench, reduced by about 12%

- all chunk tree leaves are prefetched at mount time, can improve mount
  time on large (terabyte-sized) filesystems

- speed up parallel fsync of files with reflinked/deduped extents, with
  jobs 16 to 1024 the throughput gets improved roughly by 50% on average
  and runtime decreased roughly by 30% on average, notable outlier is
  128 jobs with +121.2% on throughput and -54.6% runtime

- another speed up of parallel fsync, reduce number of checksum tree
  lookups and contention, the improvements start to show up with 2
  tasks with +20% throughput and -16% runtime up to 64 with +200%
  throughput and -66% runtime


- umount-time qgroup leak checker

- qgroups
  - add a way to unreserve partial range after failure, avoiding some
    EDQUOT errors
  - improved flushing logic when EDQUOT is hit

- possible EINTR interruption caused by failed reservations after
  transaction start is better handled and documented

- transaction abort errors are unified to EROFS in case it's not the
  original reason of abort or we don't have other way to determine the


- make truncate succeed on a NOCOW file even if data space is exhausted

- fix cancelling balance on filesystem with exhausted metadata space

- anon block device:
  - preallocate anon bdev when subvolume is created to report failure
  - shorten time the anon bdev id is allocated
  - don't allocate anon bdev for internal roots

- minor memory leak in ref-verify

- refuse invalid combinations of compression and NOCOW file flags

- lockdep fixes, updating the device locks

- remove obsolete fallback logic for block group profile adjustments
  when switching from 1 to more devices, causing allocation of unwanted
  block groups


- cleanups, refactoring, simplifications
  - conversions from struct inode to struct btrfs_inode in internal
  - removal of unused struct members

The following changes since commit 92ed301919932f777713b9172e525674157e983d:

  Linux 5.8-rc7 (2020-07-26 14:14:06 -0700)

are available in the Git repository at:

  git:// for-5.9-tag

for you to fetch changes up to 5e548b32018d96c377fda4bdac2bf511a448ca67:

  btrfs: do not set the full sync flag on the inode during page release (2020-07-27 12:55:48 +0200)

Anand Jain (3):
      btrfs: let btrfs_return_cluster_to_free_space() return void
      btrfs: use helper btrfs_get_block_group
      btrfs: don't traverse into the seed devices in show_devname

David Sterba (18):
      btrfs: scrub: remove kmap/kunmap of pages
      btrfs: scrub: unify naming of page address variables
      btrfs: scrub: simplify superblock checksum calculation
      btrfs: scrub: remove temporary csum array in scrub_checksum_super
      btrfs: scrub: clean up temporary page variables in scrub_checksum_super
      btrfs: scrub: simplify data block checksum calculation
      btrfs: scrub: clean up temporary page variables in scrub_checksum_data
      btrfs: scrub: simplify tree block checksum calculation
      btrfs: scrub: clean up temporary page variables in scrub_checksum_tree_block
      btrfs: add little-endian optimized key helpers
      btrfs: don't use UAPI types for fiemap callback
      btrfs: remove unused btrfs_root::defrag_trans_start
      btrfs: start deprecation of mount option inode_cache
      btrfs: allow use of global block reserve for balance item deletion
      btrfs: remove deprecated mount option alloc_start
      btrfs: remove deprecated mount option subvolrootid
      btrfs: prefetch chunk tree leaves at mount
      btrfs: add missing check for nocow and compression inode flags

Denis Efremov (1):
      btrfs: tests: remove if duplicate in __check_free_space_extents()

Filipe Manana (13):
      btrfs: remove no longer necessary chunk mutex locking cases
      btrfs: remove the start argument from btrfs_free_reserved_data_space_noquota()
      btrfs: use btrfs_alloc_data_chunk_ondemand() when allocating space for relocation
      btrfs: remove no longer used log_list member of struct btrfs_ordered_extent
      btrfs: remove no longer used trans_list member of struct btrfs_ordered_extent
      btrfs: only commit the delayed inode when doing a full fsync
      btrfs: only commit delayed items at fsync if we are logging a directory
      btrfs: stop incremening log_batch for the log root tree when syncing log
      btrfs: remove no longer needed use of log_writers for the log root tree
      btrfs: reduce contention on log trees when logging checksums
      btrfs: fix race between page release and a fast fsync
      btrfs: release old extent maps during page release
      btrfs: do not set the full sync flag on the inode during page release

Johannes Thumshirn (7):
      btrfs: get mapping tree directly from fsinfo in find_first_block_group
      btrfs: factor out reading of bg from find_frist_block_group
      btrfs: use free_root_extent_buffer to free root
      btrfs: pass checksum type via BTRFS_IOC_FS_INFO ioctl
      btrfs: add filesystem generation to FS_INFO ioctl
      btrfs: add metadata_uuid to FS_INFO ioctl
      btrfs: open-code remount flag setting in btrfs_remount

Josef Bacik (10):
      btrfs: convert block group refcount to refcount_t
      btrfs: don't WARN if we abort a transaction with EROFS
      btrfs: document special case error codes for fs errors
      btrfs: return EROFS for BTRFS_FS_STATE_ERROR cases
      btrfs: sysfs: use NOFS for device creation
      btrfs: open device without device_list_mutex
      btrfs: move the chunk_mutex in btrfs_read_chunk_tree
      btrfs: fix lockdep splat from btrfs_dump_space_info
      btrfs: don't adjust bg flags and use default allocation profiles
      btrfs: if we're restriping, use the target restripe profile

Liao Pingfang (1):
      btrfs: check-integrity: remove unnecessary failure messages during memory allocation

Marcos Paulo de Souza (1):
      btrfs: add multi-statement protection to btrfs_set/clear_and_info macros

Nikolay Borisov (74):
      btrfs: don't balance btree inode pages from buffered write path
      btrfs: read stripe len directly in btrfs_rmap_block
      btrfs: simplify checks when adding excluded ranges
      btrfs: make __btrfs_add_ordered_extent take struct btrfs_inode
      btrfs: make get_extent_allocation_hint take btrfs_inode
      btrfs: make btrfs_lookup_ordered_extent take btrfs_inode
      btrfs: make btrfs_reloc_clone_csums take btrfs_inode
      btrfs: make create_io_em take btrfs_inode
      btrfs: make extent_clear_unlock_delalloc take btrfs_inode
      btrfs: make btrfs_csum_one_bio takae btrfs_inode
      btrfs: make __btrfs_drop_extents take btrfs_inode
      btrfs: remove hole check in prealloc_file_extent_cluster
      btrfs: perform data management operations outside of inode lock
      btrfs: use for loop in prealloc_file_extent_cluster
      btrfs: tracepoints: fix btrfs_trigger_flush symbolic string for flags
      btrfs: tracepoints: fix extent type symbolic name print
      btrfs: tracepoints: move FLUSH_ACTIONS define
      btrfs: tracepoints: fix qgroup reservation type printing
      btrfs: tracepoints: switch extent_io_tree_owner to using EM macro
      btrfs: tracepoints: convert flush states to using EM macros
      btrfs: make qgroup_free_reserved_data take btrfs_inode
      btrfs: make __btrfs_qgroup_release_data take btrfs_inode
      btrfs: make btrfs_qgroup_free_data take btrfs_inode
      btrfs: make cow_file_range_inline take btrfs_inode
      btrfs: make btrfs_add_ordered_extent take btrfs_inode
      btrfs: make cow_file_range take btrfs_inode
      btrfs: make btrfs_add_ordered_extent_compress take btrfs_inode
      btrfs: make btrfs_submit_compressed_write take btrfs_inode
      btrfs: make submit_compressed_extents take btrfs_inode
      btrfs: make btrfs_qgroup_release_data take btrfs_inode
      btrfs: make insert_reserved_file_extent take btrfs_inode
      btrfs: make fallback_to_cow take btrfs_inode
      btrfs: make run_delalloc_nocow take btrfs_inode
      btrfs: make cow_file_range_async take btrfs_inode
      btrfs: make btrfs_dec_test_first_ordered_pending take btrfs_inode
      btrfs: make __endio_write_update_ordered take btrfs_inode
      btrfs: make btrfs_cleanup_ordered_extents take btrfs_inode
      btrfs: make inode_can_compress take btrfs_inode
      btrfs: make inode_need_compress take btrfs_inode
      btrfs: make need_force_cow take btrfs_inode
      btrfs: make btrfs_run_delalloc_range take btrfs_inode
      btrfs: make btrfs_add_ordered_extent_dio take btrfs_inode
      btrfs: make btrfs_create_dio_extent take btrfs_inode
      btrfs: make btrfs_new_extent_direct take btrfs_inode
      btrfs: make __extent_writepage_io take btrfs_inode
      btrfs: make writepage_delalloc take btrfs_inode
      btrfs: make btrfs_set_extent_delalloc take btrfs_inode
      btrfs: make btrfs_dirty_pages take btrfs_inode
      btrfs: make btrfs_qgroup_reserve_data take btrfs_inode
      btrfs: make btrfs_free_reserved_data_space_noquota take btrfs_fs_info
      btrfs: make btrfs_free_reserved_data_space take btrfs_inode
      btrfs: make btrfs_delalloc_release_space take btrfs_inode
      btrfs: make btrfs_check_data_free_space take btrfs_inode
      btrfs: make btrfs_delalloc_reserve_space take btrfs_inode
      btrfs: remove BTRFS_I calls in btrfs_writepage_fixup_worker
      btrfs: make prealloc_file_extent_cluster take btrfs_inode
      btrfs: make btrfs_set_inode_last_trans take btrfs_inode
      btrfs: make btrfs_qgroup_check_reserved_leak take btrfs_inode
      btrfs: make get_state_failrec return failrec directly
      btrfs: streamline btrfs_get_io_failure_record logic
      btrfs: record btrfs_device directly in btrfs_io_bio
      btrfs: don't check for btrfs_device::bdev in btrfs_end_bio
      btrfs: increment device corruption error in case of checksum error
      btrfs: remove needless ASSERT check of orig_bio in end_compressed_bio_read
      btrfs: increment corrupt device counter during compressed read
      btrfs: sysfs: add bdi link to the fsid directory
      btrfs: always initialize btrfs_bio::tgtdev_map/raid_map pointers
      btrfs: raid56: remove redundant device check in rbio_add_io_page
      btrfs: raid56: assign bio in while() when using bio_list_pop
      btrfs: raid56: use in_range where applicable
      btrfs: raid56: don't opencode swap() in __raid_recover_end_io
      btrfs: remove fail label in check_compressed_csum
      btrfs: raid56: remove out label in __raid56_parity_recover
      btrfs: remove done label in writepage_delalloc

Qu Wenruo (22):
      btrfs: introduce "rescue=" mount option
      btrfs: inode: refactor the parameters of insert_reserved_file_extent()
      btrfs: inode: move qgroup reserved space release to the callers of insert_reserved_file_extent()
      btrfs: file: reserve qgroup space after the hole punch range is locked
      btrfs: change timing for qgroup reserved space for ordered extents to fix reserved space leak
      btrfs: qgroup: catch reserved space leaks at unmount time
      btrfs: allow btrfs_truncate_block() to fallback to nocow for data space reservation
      btrfs: add comments for btrfs_check_can_nocow() and can_nocow_extent()
      btrfs: refactor btrfs_check_can_nocow() into two variants
      btrfs: use __u16 for the return value of btrfs_qgroup_level()
      btrfs: qgroup: export qgroups in sysfs
      btrfs: don't allocate anonymous block device for user invisible roots
      btrfs: free anon block device right after subvolume deletion
      btrfs: preallocate anon block device at first phase of snapshot creation
      btrfs: qgroup: allow to unreserve range without releasing other ranges
      btrfs: qgroup: try to flush qgroup space when we get -EDQUOT
      btrfs: qgroup: remove ASYNC_COMMIT mechanism in favor of reserve retry-after-EDQUOT
      btrfs: qgroup: free per-trans reserved space when a subvolume gets dropped
      btrfs: relocation: allow signal to cancel balance
      btrfs: avoid possible signal interruption of btrfs_drop_snapshot() on relocation tree
      btrfs: relocation: review the call sites which can be interrupted by signal
      btrfs: add comments for btrfs_reserve_flush_enum

Tom Rix (1):
      btrfs: ref-verify: fix memory leak in add_block_entry

 fs/btrfs/block-group.c                 | 211 +++++--------
 fs/btrfs/block-group.h                 |   3 +-
 fs/btrfs/btrfs_inode.h                 |  11 +
 fs/btrfs/check-integrity.c             |  27 +-
 fs/btrfs/compression.c                 |  30 +-
 fs/btrfs/compression.h                 |   4 +-
 fs/btrfs/ctree.c                       |  17 ++
 fs/btrfs/ctree.h                       | 127 ++++++--
 fs/btrfs/delalloc-space.c              |  36 +--
 fs/btrfs/delalloc-space.h              |  10 +-
 fs/btrfs/disk-io.c                     |  92 +++++-
 fs/btrfs/disk-io.h                     |   2 +
 fs/btrfs/extent-io-tree.h              |   5 +-
 fs/btrfs/extent-tree.c                 |  17 +-
 fs/btrfs/extent_io.c                   | 241 ++++++++-------
 fs/btrfs/extent_io.h                   |   4 +-
 fs/btrfs/file-item.c                   |   4 +-
 fs/btrfs/file.c                        | 143 +++++----
 fs/btrfs/free-space-cache.c            |  23 +-
 fs/btrfs/free-space-cache.h            |   2 +-
 fs/btrfs/inode-map.c                   |   3 +-
 fs/btrfs/inode.c                       | 530 ++++++++++++++++++---------------
 fs/btrfs/ioctl.c                       |  86 +++++-
 fs/btrfs/ordered-data.c                |  63 ++--
 fs/btrfs/ordered-data.h                |  19 +-
 fs/btrfs/qgroup.c                      | 359 ++++++++++++++++------
 fs/btrfs/qgroup.h                      |  24 +-
 fs/btrfs/raid56.c                      |  65 ++--
 fs/btrfs/ref-verify.c                  |   2 +
 fs/btrfs/reflink.c                     |  26 +-
 fs/btrfs/relocation.c                  |  71 ++---
 fs/btrfs/scrub.c                       | 153 +++-------
 fs/btrfs/space-info.c                  |   2 +-
 fs/btrfs/super.c                       | 144 ++++++---
 fs/btrfs/sysfs.c                       | 163 +++++++++-
 fs/btrfs/sysfs.h                       |   7 +
 fs/btrfs/tests/free-space-tree-tests.c |   2 -
 fs/btrfs/tests/inode-tests.c           |  14 +-
 fs/btrfs/transaction.c                 |   8 +-
 fs/btrfs/transaction.h                 |  28 +-
 fs/btrfs/tree-defrag.c                 |   5 +-
 fs/btrfs/tree-log.c                    |  50 ++--
 fs/btrfs/volumes.c                     | 133 ++++++---
 fs/btrfs/volumes.h                     |   2 +-
 include/trace/events/btrfs.h           | 137 +++++----
 include/uapi/linux/btrfs.h             |  21 +-
 include/uapi/linux/btrfs_tree.h        |   4 +-
 47 files changed, 1909 insertions(+), 1221 deletions(-)

             reply index

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-31 16:42 David Sterba [this message]
2020-08-03 17:55 ` pr-tracker-bot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-BTRFS Archive on

Archives are clonable:
	git clone --mirror linux-btrfs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-btrfs linux-btrfs/ \
	public-inbox-index linux-btrfs

Example config snippet for mirrors

Newsgroup available over NNTP:

AGPL code for this site: git clone