linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: David Sterba <dsterba@suse.com>
To: torvalds@linux-foundation.org
Cc: David Sterba <dsterba@suse.com>,
	linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [GIT PULL] Btrfs updates for 5.14
Date: Mon, 28 Jun 2021 20:16:26 +0200	[thread overview]
Message-ID: <cover.1624891843.git.dsterba@suse.com> (raw)

Hi,

here's a normal mix of improvements, core changes and features that user
have been missing or complaining about.

Please pull, thanks.

User visible changes:

- new sysfs exports
  - add sysfs knob to limit scrub IO bandwidth per device
  - device stats are also available in
    /sys/fs/btrfs/FSID/devinfo/DEVID/error_stats

- support cancellable resize and device delete ioctls

- change how the empty value is interpreted when setting a property,
  so far we have only 'btrfs.compression' and we need to distinguish a
  reset to defaults and setting "do not compress", in general the empty
  value will always mean 'reset to defaults' for any other property, for
  compression it's either 'no' or 'none' to forbid compression

Performance improvements:

- no need for full sync when truncation does not touch extents, reported
  run time change is -12%

- avoid unnecessary logging of xattrs during fast fsyncs (+17%
  throughput, -17% runtime on xattr stress workload)

Core:

- preemptive flushing improvements and fixes
  - adjust clamping logic on multi-threaded workloads to avoid flushing
    too soon
  - take into account global block reserve, may help on almost full
    filesystems
  - continue flushing when there are enough pending delalloc and ordered
    bytes

- simplify logic around conditional transaction commit, a workaround
  used in the past for throttling that's been superseded by ticket
  reservations that manage the throttling in a better way

- subpage blocksize preparation:
  - submit read time repair only for each corrupted sector
  - scrub repair now works with sectors and not pages
  - free space cache (v1) works with sectors and not pages
  - more fine grained bio tracking for extents
  - subpage support in page callbacks, extent callbacks, end io
    callbacks

- simplify transaction abort logic and always abort and don't check
  various potentially unreliable stats tracked by the transaction

- exclusive operations can do more checks when started and allow eg.
  cancellation of the same running operation

- ensure relocation never runs while we have send operations running,
  e.g. when zoned background auto reclaim starts

Fixes:

- zoned: more sanity checks of write pointer

- improve error handling in delayed inodes

- send:
  - fix invalid path for unlink operations after parent orphanization
  - fix crash when memory allocations trigger reclaim

- skip compression of we have only one page (can't make things better)

- empty value of a property newly means reset to default

Other:

- lots of cleanups, comment updates, yearly typo fixing
- disable build on platforms having page size 256K

----------------------------------------------------------------
The following changes since commit 13311e74253fe64329390df80bed3f07314ddd61:

  Linux 5.13-rc7 (2021-06-20 15:03:15 -0700)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-5.14-tag

for you to fetch changes up to 629e33a16809ae0274e1f5fc3d22b92b9bd0fdf1:

  btrfs: remove unused btrfs_fs_info::total_pinned (2021-06-22 19:58:26 +0200)

----------------------------------------------------------------
Anand Jain (4):
      btrfs: reduce the variable size to fit nr_pages
      btrfs: optimize variables size in btrfs_submit_compressed_read
      btrfs: optimize variables size in btrfs_submit_compressed_write
      btrfs: fix comment about max_out in btrfs_compress_pages

Baokun Li (1):
      btrfs: send: use list_move_tail instead of list_del/list_add_tail

Christophe Leroy (1):
      btrfs: disable build on platforms having page size 256K

David Sterba (24):
      btrfs: scrub: per-device bandwidth control
      btrfs: sysfs: fix format string for some discard stats
      btrfs: clear defrag status of a root if starting transaction fails
      btrfs: clear log tree recovering status if starting transaction fails
      btrfs: scrub: factor out common scrub_stripe constraints
      btrfs: document byte swap optimization of root_item::flags accessors
      btrfs: reduce compressed_bio members' types
      btrfs: remove extra sb::s_id from message in btrfs_validate_metadata_buffer
      btrfs: simplify eb checksum verification in btrfs_validate_metadata_buffer
      btrfs: clean up header members offsets in write helpers
      btrfs: protect exclusive_operation by super_lock
      btrfs: add cancellable chunk relocation support
      btrfs: introduce try-lock semantics for exclusive op start
      btrfs: add wrapper for conditional start of exclusive operation
      btrfs: add cancellation to resize
      btrfs: add device delete cancel
      btrfs: sink wait_for_unblock parameter to async commit
      btrfs: inline wait_current_trans_commit_start in its caller
      btrfs: fix typos in comments
      btrfs: sysfs: export dev stats in devinfo directory
      btrfs: compression: don't try to compress if we don't have enough pages
      btrfs: props: change how empty value is interpreted
      btrfs: switch mount option bits to enums and use wider type
      btrfs: shorten integrity checker extent data mount option

Filipe Manana (6):
      btrfs: fix misleading and incomplete comment of btrfs_truncate()
      btrfs: don't set the full sync flag when truncation does not touch extents
      btrfs: avoid unnecessary logging of xattrs during fast fsyncs
      btrfs: send: fix invalid path for unlink operations after parent orphanization
      btrfs: ensure relocation never runs while we have send operations running
      btrfs: send: fix crash when memory allocations trigger reclaim

Goldwyn Rodrigues (1):
      btrfs: correct try_lock_extent() usage in read_extent_buffer_subpage()

Johannes Thumshirn (3):
      btrfs: zoned: bail out if we can't read a reliable write pointer
      btrfs: rename check_async_write and let it return bool
      btrfs: zoned: factor out zoned device lookup

Josef Bacik (15):
      btrfs: check worker before need_preemptive_reclaim
      btrfs: only clamp the first time we have to start flushing
      btrfs: take into account global rsv in need_preemptive_reclaim
      btrfs: use the global rsv size in the preemptive thresh calculation
      btrfs: don't include the global rsv size in the preemptive used amount
      btrfs: only ignore delalloc if delalloc is much smaller than ordered
      btrfs: handle preemptive delalloc flushing slightly differently
      btrfs: make btrfs_release_delayed_iref handle the !iref case
      btrfs: fix error handling in __btrfs_update_delayed_inode
      btrfs: abort transaction if we fail to update the delayed inode
      btrfs: always abort the transaction if we abort a trans handle
      btrfs: rip out may_commit_transaction
      btrfs: remove FLUSH_DELAYED_REFS from data ENOSPC flushing
      btrfs: rip the first_ticket_bytes logic from fail_all_tickets
      btrfs: rip out btrfs_space_info::total_bytes_pinned

Naohiro Aota (2):
      btrfs: zoned: print message when zone sanity check type fails
      btrfs: fix unbalanced unlock in qgroup_account_snapshot()

Nathan Chancellor (1):
      btrfs: remove total_data_size variable in btrfs_batch_insert_items()

Nikolay Borisov (3):
      btrfs: use list_last_entry in add_falloc_range
      btrfs: eliminate insert label in add_falloc_range
      btrfs: remove unused btrfs_fs_info::total_pinned

Qu Wenruo (38):
      btrfs: make btrfs_verify_data_csum() to return a bitmap
      btrfs: submit read time repair only for each corrupted sector
      btrfs: remove io_failure_record::in_validation
      btrfs: scrub: fix subpage repair error caused by hard coded PAGE_SIZE
      btrfs: make free space cache size consistent across different PAGE_SIZE
      btrfs: remove the unused parameter @len for btrfs_bio_fits_in_stripe()
      btrfs: allow btrfs_bio_fits_in_stripe() to accept bio without any page
      btrfs: refactor submit_extent_page() to make bio and its flag tracing easier
      btrfs: make subpage metadata write path call its own endio functions
      btrfs: pass btrfs_inode to btrfs_writepage_endio_finish_ordered()
      btrfs: make Private2 lifespan more consistent
      btrfs: refactor how we finish ordered extent io for endio functions
      btrfs: update comments in btrfs_invalidatepage()
      btrfs: introduce btrfs_lookup_first_ordered_range()
      btrfs: refactor btrfs_invalidatepage() for subpage support
      btrfs: rename PagePrivate2 to PageOrdered inside btrfs
      btrfs: fix hang when run_delalloc_range() failed
      btrfs: pass bytenr directly to __process_pages_contig()
      btrfs: refactor page status update into process_one_page()
      btrfs: provide btrfs_page_clamp_*() helpers
      btrfs: only require sector size alignment for end_bio_extent_writepage()
      btrfs: make btrfs_dirty_pages() to be subpage compatible
      btrfs: make __process_pages_contig() to handle subpage dirty/error/writeback status
      btrfs: make end_bio_extent_writepage() to be subpage compatible
      btrfs: make process_one_page() to handle subpage locking
      btrfs: introduce helpers for subpage ordered status
      btrfs: make page Ordered bit to be subpage compatible
      btrfs: update locked page dirty/writeback/error bits in __process_pages_contig
      btrfs: prevent extent_clear_unlock_delalloc() to unlock page not locked by __process_pages_contig()
      btrfs: make btrfs_set_range_writeback() subpage compatible
      btrfs: make __extent_writepage_io() only submit dirty range for subpage
      btrfs: make btrfs_truncate_block() to be subpage compatible
      btrfs: make btrfs_page_mkwrite() to be subpage compatible
      btrfs: reflink: make copy_inline_to_page() to be subpage compatible
      btrfs: fix the filemap_range_has_page() call in btrfs_punch_hole_lock_range()
      btrfs: don't clear page extent mapped if we're not invalidating the full page
      btrfs: subpage: fix a rare race between metadata endio and eb freeing
      btrfs: remove a stale comment for btrfs_decompress_bio()

Su Yue (1):
      btrfs: remove stale comment for argument seed of btrfs_find_device

Tian Tao (1):
      btrfs: return EAGAIN if defrag is canceled

 fs/btrfs/Kconfig                  |   2 +
 fs/btrfs/backref.c                |   2 +-
 fs/btrfs/block-group.c            |  31 +-
 fs/btrfs/compression.c            |  57 +--
 fs/btrfs/compression.h            |  26 +-
 fs/btrfs/ctree.c                  |   5 +-
 fs/btrfs/ctree.h                  | 120 +++--
 fs/btrfs/delalloc-space.c         |   2 +-
 fs/btrfs/delayed-inode.c          |  41 +-
 fs/btrfs/delayed-ref.c            |  26 -
 fs/btrfs/dev-replace.c            |   2 +-
 fs/btrfs/discard.c                |   2 +-
 fs/btrfs/disk-io.c                |  55 +--
 fs/btrfs/extent-tree.c            |  18 +-
 fs/btrfs/extent_io.c              | 973 ++++++++++++++++++++++----------------
 fs/btrfs/extent_io.h              |  29 +-
 fs/btrfs/file-item.c              |   2 +-
 fs/btrfs/file.c                   |  44 +-
 fs/btrfs/free-space-cache.c       |   2 +-
 fs/btrfs/inode.c                  | 517 +++++++++++---------
 fs/btrfs/ioctl.c                  | 184 +++++--
 fs/btrfs/locking.c                |   4 +-
 fs/btrfs/ordered-data.c           | 253 +++++++---
 fs/btrfs/ordered-data.h           |  10 +-
 fs/btrfs/props.c                  |  16 +-
 fs/btrfs/qgroup.c                 |  10 +-
 fs/btrfs/reflink.c                |  14 +-
 fs/btrfs/relocation.c             |  75 ++-
 fs/btrfs/scrub.c                  | 159 +++++--
 fs/btrfs/send.c                   |  47 +-
 fs/btrfs/space-info.c             | 233 ++-------
 fs/btrfs/space-info.h             |  30 --
 fs/btrfs/subpage.c                | 155 +++++-
 fs/btrfs/subpage.h                |  33 +-
 fs/btrfs/super.c                  |  16 +-
 fs/btrfs/sysfs.c                  |  74 ++-
 fs/btrfs/tests/extent-map-tests.c |   2 +-
 fs/btrfs/transaction.c            |  61 +--
 fs/btrfs/transaction.h            |   6 +-
 fs/btrfs/tree-log.c               |  22 +-
 fs/btrfs/volumes.c                |  24 +-
 fs/btrfs/volumes.h                |   5 +-
 fs/btrfs/zoned.c                  |  43 +-
 fs/btrfs/zoned.h                  |   9 +
 include/trace/events/btrfs.h      |  23 +-
 include/uapi/linux/btrfs.h        |   4 +-
 include/uapi/linux/btrfs_tree.h   |   4 +-
 47 files changed, 2041 insertions(+), 1431 deletions(-)

             reply	other threads:[~2021-06-28 18:19 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-28 18:16 David Sterba [this message]
2021-06-28 23:36 ` pr-tracker-bot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1624891843.git.dsterba@suse.com \
    --to=dsterba@suse.com \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=torvalds@linux-foundation.org \
    --subject='Re: [GIT PULL] Btrfs updates for 5.14' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
on how to clone and mirror all data and code used for this inbox