All of lore.kernel.org
 help / color / mirror / Atom feed
* [RFC v1.1] xfs-linux: rtalloc-speedups-6.7 updated to b67199695696
@ 2023-10-17 15:37 Darrick J. Wong
  2023-10-17 15:43 ` Darrick J. Wong
                   ` (5 more replies)
  0 siblings, 6 replies; 89+ messages in thread
From: Darrick J. Wong @ 2023-10-17 15:37 UTC (permalink / raw)
  To: dchinner, hch, linux-xfs, osandov, osandov

Hi folks,

It turns out that yesterday's branch broke bisection, so I've push -f'd
a work branch for all of our realtime cleanups and optimizations in the
rtalloc-speedups-6.7 branch of the xfs-linux repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git

I started by rebasing last week's rt cleanups patchsets against 6.6-rc6
TOT instead of djwong-dev, then I added Dave's xfs_rtalloc_args cleanup,
and then added Omar's rt allocator speedups.  This branch has now
survived overnight testing.

The new head of the rtalloc-speedups-6.7 branch is commit:

b67199695696 xfs: don't look for end of extent further than necessary in xfs_rtallocate_extent_near()

36 new commits:

Darrick J. Wong (28):
[6953ad8d0bae] xfs: make xchk_iget safer in the presence of corrupt inode btrees
[1948e87e4723] xfs: bump max fsgeom struct version
[c0f654f9e230] xfs: hoist freeing of rt data fork extent mappings
[03cc2dc5552c] xfs: prevent rt growfs when quota is enabled
[b9a1d7039b5b] xfs: rt stubs should return negative errnos when rt disabled
[2f21b6b1abff] xfs: fix units conversion error in xfs_bmap_del_extent_delay
[003e9f00c1bb] xfs: make sure maxlen is still congruent with prod when rounding down
[869a7bb10764] xfs: move the xfs_rtbitmap.c declarations to xfs_rtbitmap.h
[6e8471604ee5] xfs: convert xfs_extlen_t to xfs_rtxlen_t in the rt allocator
[1e98fe26bd18] xfs: convert rt bitmap/summary block numbers to xfs_fileoff_t
[fab6068dbd4c] xfs: convert rt bitmap extent lengths to xfs_rtbxlen_t
[b5b1ec2011f7] xfs: rename xfs_verify_rtext to xfs_verify_rtbext
[efafd16f2dcb] xfs: convert rt extent numbers to xfs_rtxnum_t
[e902ec670453] xfs: create a helper to convert rtextents to rtblocks
[d0069a4dc6bb] xfs: create a helper to compute leftovers of realtime extents
[c317ec75fe0d] xfs: create a helper to convert extlen to rtextlen
[316da55c7ecf] xfs: create helpers to convert rt block numbers to rt extent numbers
[0e90edd13659] xfs: convert do_div calls to xfs_rtb_to_rtx helper calls
[2d4cf0892cfe] xfs: create rt extent rounding helpers for realtime extent blocks
[f538cf95a5b8] xfs: use shifting and masking when converting rt extents, if possible
[e780da4b8067] xfs: convert the rtbitmap block and bit macros to static inline functions
[710a06e09cfe] xfs: remove XFS_BLOCKWSIZE and XFS_BLOCKWMASK macros
[a705854970f8] xfs: convert open-coded xfs_rtword_t pointer accesses to helper
[4cecf034d685] xfs: convert rt summary macros to helpers
[4d9e06b25a33] xfs: create helpers for rtbitmap block/wordcount computations
[625af2f8cf01] xfs: use accessor functions for bitmap words
[c7078ff43a80] xfs: create helpers for rtsummary block/wordcount computations
[ac9c57723b70] xfs: use accessor functions for summary info words

Dave Chinner (1):
[98ab1a255b81] xfs: consolidate realtime allocation arguments

Omar Sandoval (6):
[947d029f1677] xfs: cache last bitmap block in realtime allocator
[36cb8887dc72] xfs: invert the realtime summary cache
[c6bebaf43313] xfs: return maximum free size from xfs_rtany_summary()
[ff31bbc39fb6] xfs: limit maxlen based on available space in xfs_rtallocate_extent_near()
[90127b4188e3] xfs: don't try redundant allocations in xfs_rtallocate_extent_near()
[b67199695696] xfs: don't look for end of extent further than necessary in xfs_rtallocate_extent_near()

Shiyang Ruan (1):
[1937b0813e81] mm, pmem, xfs: Introduce MF_MEM_PRE_REMOVE for unbind

Code Diffstat:

drivers/dax/super.c            |   3 +-
fs/xfs/libxfs/xfs_bmap.c       |  45 +--
fs/xfs/libxfs/xfs_format.h     |  34 +-
fs/xfs/libxfs/xfs_rtbitmap.c   | 733 +++++++++++++++++++++++++----------------
fs/xfs/libxfs/xfs_rtbitmap.h   | 326 ++++++++++++++++++
fs/xfs/libxfs/xfs_sb.c         |   2 +
fs/xfs/libxfs/xfs_sb.h         |   2 +-
fs/xfs/libxfs/xfs_trans_resv.c |  10 +-
fs/xfs/libxfs/xfs_types.c      |   4 +-
fs/xfs/libxfs/xfs_types.h      |  10 +-
fs/xfs/scrub/bmap.c            |   2 +-
fs/xfs/scrub/common.c          |   6 +-
fs/xfs/scrub/common.h          |  19 ++
fs/xfs/scrub/fscounters.c      |   2 +-
fs/xfs/scrub/inode.c           |   7 +-
fs/xfs/scrub/rtbitmap.c        |  28 +-
fs/xfs/scrub/rtsummary.c       |  57 ++--
fs/xfs/scrub/trace.c           |   1 +
fs/xfs/scrub/trace.h           |   9 +-
fs/xfs/xfs_bmap_util.c         |  50 ++-
fs/xfs/xfs_fsmap.c             |  15 +-
fs/xfs/xfs_inode_item.c        |   3 +-
fs/xfs/xfs_ioctl.c             |   5 +-
fs/xfs/xfs_linux.h             |  12 +
fs/xfs/xfs_mount.h             |   8 +-
fs/xfs/xfs_notify_failure.c    | 108 +++++-
fs/xfs/xfs_ondisk.h            |   4 +
fs/xfs/xfs_rtalloc.c           | 602 +++++++++++++++++----------------
fs/xfs/xfs_rtalloc.h           |  94 +-----
fs/xfs/xfs_super.c             |   3 +-
fs/xfs/xfs_trans.c             |   7 +-
include/linux/mm.h             |   1 +
mm/memory-failure.c            |  21 +-
33 files changed, 1391 insertions(+), 842 deletions(-)
create mode 100644 fs/xfs/libxfs/xfs_rtbitmap.h


^ permalink raw reply	[flat|nested] 89+ messages in thread
* [PATCHSET RFC v1.0 0/8] xfs: refactor rtbitmap/summary macros
@ 2023-10-11 18:01 Darrick J. Wong
  2023-10-11 18:08 ` [PATCH 8/8] xfs: use accessor functions for summary info words Darrick J. Wong
  0 siblings, 1 reply; 89+ messages in thread
From: Darrick J. Wong @ 2023-10-11 18:01 UTC (permalink / raw)
  To: djwong; +Cc: linux-xfs, osandov, hch

Hi all,

In preparation for adding block headers and enforcing endian order in
rtbitmap and rtsummary blocks, replace open-coded geometry computations
and fugly macros with proper helper functions that can be typechecked.
Soon we'll be needing to add more complex logic to the helpers.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

This has been running on the djcloud for months with no problems.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=refactor-rtbitmap-macros

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=refactor-rtbitmap-macros
---
 fs/xfs/libxfs/xfs_format.h      |   32 ++---
 fs/xfs/libxfs/xfs_rtbitmap.c    |  268 ++++++++++++++++++++++++++++-----------
 fs/xfs/libxfs/xfs_rtbitmap.h    |  133 +++++++++++++++++++
 fs/xfs/libxfs/xfs_trans_resv.c  |    9 +
 fs/xfs/libxfs/xfs_types.h       |    2 
 fs/xfs/scrub/rtsummary.c        |   50 ++++---
 fs/xfs/scrub/rtsummary.h        |    6 -
 fs/xfs/scrub/rtsummary_repair.c |    7 +
 fs/xfs/scrub/trace.c            |    1 
 fs/xfs/scrub/trace.h            |    4 -
 fs/xfs/xfs_ondisk.h             |    4 +
 fs/xfs/xfs_rtalloc.c            |   39 +++---
 12 files changed, 408 insertions(+), 147 deletions(-)


^ permalink raw reply	[flat|nested] 89+ messages in thread
* [PATCHSET v1.0 0/8] xfs: refactor rtbitmap/summary macros
@ 2022-12-30 22:17 Darrick J. Wong
  2022-12-30 22:17 ` [PATCH 8/8] xfs: use accessor functions for summary info words Darrick J. Wong
  0 siblings, 1 reply; 89+ messages in thread
From: Darrick J. Wong @ 2022-12-30 22:17 UTC (permalink / raw)
  To: djwong; +Cc: linux-xfs

Hi all,

In preparation for adding block headers and enforcing endian order in
rtbitmap and rtsummary blocks, replace open-coded geometry computations
and fugly macros with proper helper functions that can be typechecked.
Soon we'll be needing to add more complex logic to the helpers.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=refactor-rtbitmap-macros

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=refactor-rtbitmap-macros
---
 fs/xfs/libxfs/xfs_format.h      |   32 ++---
 fs/xfs/libxfs/xfs_rtbitmap.c    |  268 ++++++++++++++++++++++++++++-----------
 fs/xfs/libxfs/xfs_rtbitmap.h    |  133 +++++++++++++++++++
 fs/xfs/libxfs/xfs_trans_resv.c  |    9 +
 fs/xfs/libxfs/xfs_types.h       |    2 
 fs/xfs/scrub/rtsummary.c        |   52 ++++----
 fs/xfs/scrub/rtsummary.h        |    6 -
 fs/xfs/scrub/rtsummary_repair.c |    7 +
 fs/xfs/scrub/trace.c            |    1 
 fs/xfs/scrub/trace.h            |    4 -
 fs/xfs/xfs_ondisk.h             |    4 +
 fs/xfs/xfs_rtalloc.c            |   39 +++---
 12 files changed, 409 insertions(+), 148 deletions(-)


^ permalink raw reply	[flat|nested] 89+ messages in thread

end of thread, other threads:[~2023-10-19 16:14 UTC | newest]

Thread overview: 89+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-10-17 15:37 [RFC v1.1] xfs-linux: rtalloc-speedups-6.7 updated to b67199695696 Darrick J. Wong
2023-10-17 15:43 ` Darrick J. Wong
2023-10-17 15:45 ` [PATCHSET RFC v1.1 0/4] xfs: minor bugfixes for rt stuff Darrick J. Wong
2023-10-17 15:47   ` [PATCH 1/4] xfs: bump max fsgeom struct version Darrick J. Wong
2023-10-17 15:47   ` [PATCH 2/4] xfs: hoist freeing of rt data fork extent mappings Darrick J. Wong
2023-10-17 16:00     ` Christoph Hellwig
2023-10-17 15:47   ` [PATCH 3/4] xfs: prevent rt growfs when quota is enabled Darrick J. Wong
2023-10-17 15:47   ` [PATCH 4/4] xfs: rt stubs should return negative errnos when rt disabled Darrick J. Wong
2023-10-17 15:46 ` [PATCHSET RFC v1.1 0/8] xfs: clean up realtime type usage Darrick J. Wong
2023-10-17 15:48   ` [PATCH 1/8] xfs: fix units conversion error in xfs_bmap_del_extent_delay Darrick J. Wong
2023-10-17 16:01     ` Christoph Hellwig
2023-10-17 15:48   ` [PATCH 2/8] xfs: make sure maxlen is still congruent with prod when rounding down Darrick J. Wong
2023-10-17 15:48   ` [PATCH 3/8] xfs: move the xfs_rtbitmap.c declarations to xfs_rtbitmap.h Darrick J. Wong
2023-10-17 15:48   ` [PATCH 4/8] xfs: convert xfs_extlen_t to xfs_rtxlen_t in the rt allocator Darrick J. Wong
2023-10-17 15:49   ` [PATCH 5/8] xfs: convert rt bitmap/summary block numbers to xfs_fileoff_t Darrick J. Wong
2023-10-17 15:49   ` [PATCH 6/8] xfs: convert rt bitmap extent lengths to xfs_rtbxlen_t Darrick J. Wong
2023-10-17 15:49   ` [PATCH 7/8] xfs: rename xfs_verify_rtext to xfs_verify_rtbext Darrick J. Wong
2023-10-17 15:49   ` [PATCH 8/8] xfs: convert rt extent numbers to xfs_rtxnum_t Darrick J. Wong
2023-10-17 15:46 ` [PATCHSET RFC v1.1 0/7] xfs: refactor rt extent unit conversions Darrick J. Wong
2023-10-17 15:50   ` [PATCH 1/7] xfs: create a helper to convert rtextents to rtblocks Darrick J. Wong
2023-10-17 15:50   ` [PATCH 2/7] xfs: create a helper to compute leftovers of realtime extents Darrick J. Wong
2023-10-17 15:50   ` [PATCH 3/7] xfs: create a helper to convert extlen to rtextlen Darrick J. Wong
2023-10-17 15:50   ` [PATCH 4/7] xfs: create helpers to convert rt block numbers to rt extent numbers Darrick J. Wong
2023-10-17 16:04     ` Christoph Hellwig
2023-10-17 15:51   ` [PATCH 5/7] xfs: convert do_div calls to xfs_rtb_to_rtx helper calls Darrick J. Wong
2023-10-17 16:05     ` Christoph Hellwig
2023-10-17 15:51   ` [PATCH 6/7] xfs: create rt extent rounding helpers for realtime extent blocks Darrick J. Wong
2023-10-17 15:51   ` [PATCH 7/7] xfs: use shifting and masking when converting rt extents, if possible Darrick J. Wong
2023-10-17 15:46 ` [PATCHSET RFC v1.1 0/8] xfs: refactor rtbitmap/summary macros Darrick J. Wong
2023-10-17 15:52   ` [PATCH 1/8] xfs: convert the rtbitmap block and bit macros to static inline functions Darrick J. Wong
2023-10-17 15:52   ` [PATCH 2/8] xfs: remove XFS_BLOCKWSIZE and XFS_BLOCKWMASK macros Darrick J. Wong
2023-10-17 15:52   ` [PATCH 3/8] xfs: convert open-coded xfs_rtword_t pointer accesses to helper Darrick J. Wong
2023-10-17 16:31     ` Christoph Hellwig
2023-10-17 15:52   ` [PATCH 4/8] xfs: convert rt summary macros to helpers Darrick J. Wong
2023-10-17 15:53   ` [PATCH 5/8] xfs: create helpers for rtbitmap block/wordcount computations Darrick J. Wong
2023-10-17 16:33     ` Christoph Hellwig
2023-10-17 15:53   ` [PATCH 6/8] xfs: use accessor functions for bitmap words Darrick J. Wong
2023-10-17 18:53     ` Christoph Hellwig
2023-10-18  2:01       ` Darrick J. Wong
2023-10-18  4:50         ` Christoph Hellwig
2023-10-17 15:53   ` [PATCH 7/8] xfs: create helpers for rtsummary block/wordcount computations Darrick J. Wong
2023-10-17 15:53   ` [PATCH 8/8] xfs: use accessor functions for summary info words Darrick J. Wong
2023-10-18  2:10   ` [PATCHSET RFC v1.2 0/4] xfs: refactor rtbitmap/summary accessors Darrick J. Wong
2023-10-18  2:10     ` [PATCH 1/4] xfs: create a helper to handle logging parts of rt bitmap blocks Darrick J. Wong
2023-10-18  4:52       ` Christoph Hellwig
2023-10-18  2:10     ` [PATCH 2/4] xfs: use accessor functions for bitmap words Darrick J. Wong
2023-10-18  2:19       ` Darrick J. Wong
2023-10-18  4:54       ` Christoph Hellwig
2023-10-18  4:54         ` Christoph Hellwig
2023-10-18 16:27         ` Darrick J. Wong
2023-10-18 16:28           ` Christoph Hellwig
2023-10-18  2:10     ` [PATCH 3/4] xfs: create helpers for rtsummary block/wordcount computations Darrick J. Wong
2023-10-18  2:10     ` [PATCH 4/4] xfs: use accessor functions for summary info words Darrick J. Wong
2023-10-18  5:19       ` Christoph Hellwig
2023-10-18  5:31         ` Darrick J. Wong
2023-10-18  5:35           ` Christoph Hellwig
2023-10-18  6:16           ` Darrick J. Wong
2023-10-17 15:46 ` [PATCHSET RFC 2.1 0/7] xfs: CPU usage optimizations for realtime allocator Darrick J. Wong
2023-10-17 15:54   ` [PATCH 1/7] xfs: consolidate realtime allocation arguments Darrick J. Wong
2023-10-18  6:16     ` Christoph Hellwig
2023-10-18 17:04       ` Darrick J. Wong
2023-10-17 15:54   ` [PATCH 2/7] xfs: cache last bitmap block in realtime allocator Darrick J. Wong
2023-10-18  6:19     ` Christoph Hellwig
2023-10-18 16:33       ` Darrick J. Wong
2023-10-19  0:00         ` Darrick J. Wong
2023-10-18 18:28     ` Darrick J. Wong
2023-10-17 15:54   ` [PATCH 3/7] xfs: invert the realtime summary cache Darrick J. Wong
2023-10-17 15:54   ` [PATCH 4/7] xfs: return maximum free size from xfs_rtany_summary() Darrick J. Wong
2023-10-17 15:55   ` [PATCH 5/7] xfs: limit maxlen based on available space in xfs_rtallocate_extent_near() Darrick J. Wong
2023-10-17 15:55   ` [PATCH 6/7] xfs: don't try redundant allocations " Darrick J. Wong
2023-10-17 15:55   ` [PATCH 7/7] xfs: don't look for end of extent further than necessary " Darrick J. Wong
2023-10-19  0:00   ` [PATCHSET v2.2 0/9] xfs: CPU usage optimizations for realtime allocator Darrick J. Wong
2023-10-19  0:00     ` [PATCH 1/9] xfs: consolidate realtime allocation arguments Darrick J. Wong
2023-10-19  5:08       ` Christoph Hellwig
2023-10-19  0:01     ` [PATCH 2/9] xfs: cache last bitmap block in realtime allocator Darrick J. Wong
2023-10-19  0:01     ` [PATCH 3/9] xfs: simplify xfs_rtbuf_get calling conventions Darrick J. Wong
2023-10-19  5:11       ` Christoph Hellwig
2023-10-19  0:01     ` [PATCH 4/9] xfs: simplify rt bitmap/summary block accessor functions Darrick J. Wong
2023-10-19  5:13       ` Christoph Hellwig
2023-10-19 16:13         ` Darrick J. Wong
2023-10-19  0:01     ` [PATCH 5/9] xfs: invert the realtime summary cache Darrick J. Wong
2023-10-19  0:01     ` [PATCH 6/9] xfs: return maximum free size from xfs_rtany_summary() Darrick J. Wong
2023-10-19  0:01     ` [PATCH 7/9] xfs: limit maxlen based on available space in xfs_rtallocate_extent_near() Darrick J. Wong
2023-10-19  0:01     ` [PATCH 8/9] xfs: don't try redundant allocations " Darrick J. Wong
2023-10-19  0:01     ` [PATCH 9/9] xfs: don't look for end of extent further than necessary " Darrick J. Wong
  -- strict thread matches above, loose matches on Subject: below --
2023-10-11 18:01 [PATCHSET RFC v1.0 0/8] xfs: refactor rtbitmap/summary macros Darrick J. Wong
2023-10-11 18:08 ` [PATCH 8/8] xfs: use accessor functions for summary info words Darrick J. Wong
2023-10-12  6:26   ` Christoph Hellwig
2023-10-12 22:21     ` Darrick J. Wong
2022-12-30 22:17 [PATCHSET v1.0 0/8] xfs: refactor rtbitmap/summary macros Darrick J. Wong
2022-12-30 22:17 ` [PATCH 8/8] xfs: use accessor functions for summary info words Darrick J. Wong

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.