[GIT PULL] Btrfs updates for 5.8

[David Sterba <dsterba@suse.com>]

Hi,

please pull the following updates to btrfs. Thanks.

Highlights:

- speedup dead root detection during orphan cleanup, eg. when there are
  many deleted subvolumes waiting to be cleaned, the trees are now looked
  up in radix tree instead of a O(N^2) search

- snapshot creation with inherited qgroup will mark the qgroup
  inconsistent, requires a rescan

- send will emit file capabilities after chown, this produces a stream
  that does not need postprocessing to set the capabilities again

- direct io ported to iomap infrastructure, cleaned up and simplified
  code, notably removing last use of struct buffer_head in btrfs code

Core changes:

- factor out backreference iteration, to be used by ordinary
  backreferences and relocation code

- improved global block reserve utilization
  * better logic to serialize requests
  * increased maximum available for unlink
  * improved handling on large pages (64K)

- direct io cleanups and fixes
  * simplify layering, where cloned bios were unnecessarily created for
    some cases
  * error handling fixes (submit, endio)
  * remove repair worker thread, used to avoid deadlocks during
    repair

- refactored block group reading code, preparatory work for new type of
  block group storage that should improve mount time on large filesystems

Cleanups:

- cleaned up (and slightly sped up) set/get helpers for metadata data
  structure members

- root bit REF_COWS got renamed to SHAREABLE to reflect the that the
  blocks of the tree get shared either among subvolumes or with the
  relocation trees

Fixes:

- when subvolume deletion fails due to ENOSPC, the filesystem is not
  turned read-only

- device scan deals with devices from other filesystems that changed
  ownership due to overwrite (mkfs)

- fix a race between scrub and block group removal/allocation

- fix long standing bug of a runaway balance operation, printing the same
  line to the syslog, caused by a stale status bit on a reloc tree that
  prevented progress

- fix corrupt log due to concurrent fsync of inodes with shared extents

- fix space underflow for NODATACOW and buffered writes when it for some
  reason needs to fallback to COW mode

----------------------------------------------------------------
The following changes since commit 9cb1fd0efd195590b828b9b865421ad345a4a145:

  Linux 5.7-rc7 (2020-05-24 15:32:54 -0700)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-5.8-tag

for you to fetch changes up to 2166e5edce9ac1edf3b113d6091ef72fcac2d6c4:

  btrfs: fix space_info bytes_may_use underflow during space cache writeout (2020-05-28 14:01:53 +0200)

----------------------------------------------------------------
Anand Jain (5):
      btrfs: drop useless goto in open_fs_devices
      btrfs: include non-missing as a qualifier for the latest_bdev
      btrfs: free alien device after device add
      btrfs: drop stale reference to volume_mutex
      btrfs: unexport btrfs_compress_set_level()

Christoph Hellwig (1):
      btrfs: split btrfs_direct_IO to read and write part

David Sterba (26):
      btrfs: don't force read-only after error in drop snapshot
      btrfs: sort error decoder entries
      btrfs: add more codes to decoder table
      btrfs: remove more obsolete v0 extent ref declarations
      btrfs: use the token::eb for all set/get helpers
      btrfs: drop eb parameter from set/get token helpers
      btrfs: don't use set/get token for single assignment in overwrite_item
      btrfs: don't use set/get token in leaf_space_used
      btrfs: preset set/get token with first page and drop condition
      btrfs: add separate bounds checker for set/get helpers
      btrfs: speed up btrfs_get_##bits helpers
      btrfs: speed up btrfs_get_token_##bits helpers
      btrfs: speed up btrfs_set_##bits helpers
      btrfs: speed up btrfs_set_token_##bits helpers
      btrfs: speed up and simplify generic_bin_search
      btrfs: remove unused map_private_extent_buffer
      btrfs: constify extent_buffer in the API functions
      btrfs: drop unnecessary offset_in_page in extent buffer helpers
      btrfs: optimize split page read in btrfs_get_##bits
      btrfs: optimize split page read in btrfs_get_token_##bits
      btrfs: optimize split page write in btrfs_set_##bits
      btrfs: optimize split page write in btrfs_set_token_##bits
      btrfs: update documentation of set/get helpers
      btrfs: simplify root lookup by id
      btrfs: open code read_fs_root
      btrfs: simplify iget helpers

Eric Biggers (1):
      btrfs: use crypto_shash_digest() instead of open coding

Filipe Manana (16):
      btrfs: remove pointless assertion on reclaim_size counter
      btrfs: simplify error handling of clean_pinned_extents()
      btrfs: remove useless check for copy_items() return value
      btrfs: fix a race between scrub and block group removal/allocation
      btrfs: rename member 'trimming' of block group to a more generic name
      btrfs: move the block group freeze/unfreeze helpers into block-group.c
      btrfs: scrub, only lookup for csums if we are dealing with a data extent
      btrfs: fix corrupt log due to concurrent fsync of inodes with shared extents
      btrfs: make checksum item extension more efficient
      btrfs: do not ignore error from btrfs_next_leaf() when inserting checksums
      btrfs: remove useless 'fail_unlock' label from btrfs_csum_file_blocks()
      btrfs: include error on messages about failure to write space/inode caches
      btrfs: turn space cache writeout failure messages into debug messages
      btrfs: fix wrong file range cleanup after an error filling dealloc range
      btrfs: fix space_info bytes_may_use underflow after nocow buffered write
      btrfs: fix space_info bytes_may_use underflow during space cache writeout

Goldwyn Rodrigues (6):
      fs: export generic_file_buffered_read()
      iomap: add a filesystem hook for direct I/O bio submission
      iomap: remove lockdep_assert_held()
      btrfs: switch to iomap_dio_rw() for dio
      fs: remove dio_end_io()
      btrfs: remove BTRFS_INODE_READDIO_NEED_LOCK

Josef Bacik (6):
      btrfs: improve global reserve stealing logic
      btrfs: allow to use up to 90% of the global block rsv for unlink
      btrfs: account for trans_block_rsv in may_commit_transaction
      btrfs: only check priority tickets for priority flushing
      btrfs: run btrfs_try_granting_tickets if a priority ticket fails
      btrfs: force chunk allocation if our global rsv is larger than metadata

Jules Irenge (2):
      btrfs: add missing annotation for btrfs_lock_cluster()
      btrfs: add missing annotation for btrfs_tree_lock()

Marcos Paulo de Souza (1):
      btrfs: send: emit file capabilities after chown

Nikolay Borisov (4):
      btrfs: use list_for_each_entry_safe in free_reloc_roots
      btrfs: make btrfs_read_disk_super return struct btrfs_disk_super
      btrfs: open code key_search
      btrfs: remove redundant local variable in read_block_for_search

Omar Sandoval (15):
      block: add bio_for_each_bvec_all()
      btrfs: fix error handling when submitting direct I/O bio
      btrfs: fix double __endio_write_update_ordered in direct I/O
      btrfs: look at full bi_io_vec for repair decision
      btrfs: don't do repair validation for checksum errors
      btrfs: clarify btrfs_lookup_bio_sums documentation
      btrfs: rename __readpage_endio_check to check_data_csum
      btrfs: make btrfs_check_repairable() static
      btrfs: remove unused btrfs_dio_private::private
      btrfs: convert btrfs_dio_private->pending_bios to refcount_t
      btrfs: put direct I/O checksums in btrfs_dio_private instead of bio
      btrfs: get rid of one layer of bios in direct I/O
      btrfs: simplify direct I/O read repair
      btrfs: get rid of endio_repair_workers
      btrfs: unify buffered and direct I/O read repair

Qu Wenruo (44):
      btrfs: backref: introduce the skeleton of btrfs_backref_iter
      btrfs: backref: implement btrfs_backref_iter_next()
      btrfs: reloc: use btrfs_backref_iter infrastructure
      btrfs: reloc: rename mark_block_processed and __mark_block_processed
      btrfs: reloc: add backref_cache::pending_edge and backref_cache::useless_node
      btrfs: reloc: add backref_cache::fs_info member
      btrfs: reloc: make reloc root search-specific for relocation backref cache
      btrfs: reloc: refactor direct tree backref processing into its own function
      btrfs: reloc: refactor indirect tree backref processing into its own function
      btrfs: reloc: use wrapper to replace open-coded edge linking
      btrfs: reloc: pass essential members for alloc_backref_node()
      btrfs: reloc: remove the open-coded goto loop for breadth-first search
      btrfs: reloc: refactor finishing part of upper linkage into finish_upper_links()
      btrfs: reloc: refactor useless nodes handling into its own function
      btrfs: reloc: add btrfs_ prefix for backref_node/edge/cache
      btrfs: backref: move btrfs_backref_(node|edge|cache) structures to backref.h
      btrfs: rename tree_entry to rb_simple_node and export it
      btrfs: backref: rename and move backref_cache_init()
      btrfs: backref: rename and move alloc_backref_node()
      btrfs: backref: rename and move alloc_backref_edge()
      btrfs: backref: rename and move link_backref_edge()
      btrfs: backref: rename and move free_backref_(node|edge)
      btrfs: backref: rename and move drop_backref_node()
      btrfs: backref: rename and move remove_backref_node()
      btrfs: backref: rename and move backref_cache_cleanup()
      btrfs: backref: rename and move backref_tree_panic()
      btrfs: backref: rename and move should_ignore_root()
      btrfs: reloc: open code read_fs_root() for handle_indirect_tree_backref()
      btrfs: backref: rename and move handle_one_tree_block()
      btrfs: backref: rename and move finish_upper_links()
      btrfs: reloc: move error handling of build_backref_tree() to backref.c
      btrfs: backref: distinguish reloc and non-reloc use of indirect resolution
      btrfs: remove the redundant parameter level in btrfs_bin_search()
      btrfs: qgroup: mark qgroup inconsistent if we're inherting snapshot to a new qgroup
      btrfs: block-group: don't set the wrong READA flag for btrfs_read_block_groups()
      btrfs: block-group: refactor how we read one block group item
      btrfs: block-group: refactor how we delete one block group item
      btrfs: block-group: refactor how we insert a block group item
      btrfs: block-group: rename write_one_cache_group()
      btrfs: rename BTRFS_ROOT_REF_COWS to BTRFS_ROOT_SHAREABLE
      btrfs: inode: cleanup the log-tree exceptions in btrfs_truncate_inode_items()
      btrfs: don't set SHAREABLE flag for data reloc tree
      btrfs: reloc: fix reloc root leak and NULL pointer dereference
      btrfs: reloc: clear DEAD_RELOC_TREE bit for orphan roots to prevent runaway balance

Robbie Ko (2):
      btrfs: speedup dead root detection during orphan cleanup
      btrfs: reduce lock contention when creating snapshot

Tiezhu Yang (1):
      btrfs: remove duplicated include in block-group.c

YueHaibing (2):
      btrfs: remove unused function heads_to_leaves
      btrfs: remove unused function btrfs_dev_extent_chunk_tree_uuid

Zheng Wei (1):
      btrfs: tree-checker: remove duplicate definition of 'inode_item_err'

 .clang-format                   |    1 +
 Documentation/block/biovecs.rst |    2 +
 fs/btrfs/Kconfig                |    1 +
 fs/btrfs/backref.c              |  837 ++++++++++++++++++++++++-
 fs/btrfs/backref.h              |  297 +++++++++
 fs/btrfs/block-group.c          |  233 ++++---
 fs/btrfs/block-group.h          |   14 +-
 fs/btrfs/block-rsv.c            |    5 +-
 fs/btrfs/btrfs_inode.h          |   44 +-
 fs/btrfs/compression.c          |   36 +-
 fs/btrfs/compression.h          |    2 -
 fs/btrfs/ctree.c                |  180 +++---
 fs/btrfs/ctree.h                |  121 ++--
 fs/btrfs/disk-io.c              |   93 +--
 fs/btrfs/disk-io.h              |    4 +-
 fs/btrfs/export.c               |   17 +-
 fs/btrfs/extent-io-tree.h       |    1 +
 fs/btrfs/extent-tree.c          |   23 +-
 fs/btrfs/extent_io.c            |  288 ++++-----
 fs/btrfs/extent_io.h            |   67 +-
 fs/btrfs/file-item.c            |   62 +-
 fs/btrfs/file.c                 |  111 +++-
 fs/btrfs/free-space-cache.c     |   81 +--
 fs/btrfs/inode.c                | 1319 ++++++++++++++-------------------------
 fs/btrfs/ioctl.c                |  102 ++-
 fs/btrfs/locking.c              |    1 +
 fs/btrfs/misc.h                 |   54 ++
 fs/btrfs/props.c                |    9 +-
 fs/btrfs/qgroup.c               |   14 +
 fs/btrfs/relocation.c           | 1319 +++++++++------------------------------
 fs/btrfs/root-tree.c            |   12 +-
 fs/btrfs/scrub.c                |   59 +-
 fs/btrfs/send.c                 |   89 ++-
 fs/btrfs/space-info.c           |   81 ++-
 fs/btrfs/space-info.h           |    1 +
 fs/btrfs/struct-funcs.c         |  223 +++----
 fs/btrfs/super.c                |   38 +-
 fs/btrfs/transaction.c          |   78 +--
 fs/btrfs/transaction.h          |    3 +-
 fs/btrfs/tree-checker.c         |    4 -
 fs/btrfs/tree-defrag.c          |    2 +-
 fs/btrfs/tree-log.c             |  192 +++---
 fs/btrfs/uuid-tree.c            |    6 +-
 fs/btrfs/volumes.c              |   80 +--
 fs/direct-io.c                  |   19 -
 fs/iomap/direct-io.c            |   17 +-
 include/linux/bio.h             |    8 +
 include/linux/fs.h              |    4 +-
 include/linux/iomap.h           |    2 +
 include/trace/events/btrfs.h    |    1 +
 include/uapi/linux/btrfs_tree.h |    9 -
 mm/filemap.c                    |    3 +-
 52 files changed, 3224 insertions(+), 3045 deletions(-)