[GIT PULL] xfs: initial agnumber -> perag conversions for shrink

[Dave Chinner <david@fromorbit.com>]

Hi Darrick,

Can you please pull the perag conversion changes inot the next
for-next compose from the tag below? I pulled most of the original
patchset description into the tag itself, so I won't repeat any of
it here.....

Cheers,

Dave.

The following changes since commit d07f6ca923ea0927a1024dfccafc5b53b61cfecc:

  Linux 5.13-rc2 (2021-05-16 15:27:44 -0700)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/dgc/linux-xfs.git tags/xfs-perag-conv-tag

for you to fetch changes up to 509201163fca3d4d906bd50a5320115d42818748:

  xfs: remove xfs_perag_t (2021-06-02 10:48:51 +1000)

----------------------------------------------------------------
xfs: initial agnumber -> perag conversions for shrink

If we want to use active references to the perag to be able to gate
shrink removing AGs and hence perags safely, we've got a fair bit of
work to do actually use perags in all the places we need to.

There's a lot of code that iterates ag numbers and then
looks up perags from that, often multiple times for the same perag
in the one operation. If we want to use reference counted perags for
access control, then we need to convert all these uses to perag
iterators, not agno iterators.

[Patches 1-4]

The first step of this is consolidating all the perag management -
init, free, get, put, etc into a common location. THis is spread all
over the place right now, so move it all into libxfs/xfs_ag.[ch].
This does expose kernel only bits of the perag to libxfs and hence
userspace, so the structures and code is rearranged to minimise the
number of ifdefs that need to be added to the userspace codebase.
The perag iterator in xfs_icache.c is promoted to a first class API
and expanded to the needs of the code as required.

[Patches 5-10]

These are the first basic perag iterator conversions and changes to
pass the perag down the stack from those iterators where
appropriate. A lot of this is obvious, simple changes, though in
some places we stop passing the perag down the stack because the
code enters into an as yet unconverted subsystem that still uses raw
AGs.

[Patches 11-16]

These replace the agno passed in the btree cursor for per-ag btree
operations with a perag that is passed to the cursor init function.
The cursor takes it's own reference to the perag, and the reference
is dropped when the cursor is deleted. Hence we get reference
coverage for the entire time the cursor is active, even if the code
that initialised the cursor drops it's reference before the cursor
or any of it's children (duplicates) have been deleted.

The first patch adds the perag infrastructure for the cursor, the
next four patches convert a btree cursor at a time, and the last
removes the agno from the cursor once it is unused.

[Patches 17-21]

These patches are a demonstration of the simplifications and
cleanups that come from plumbing the perag through interfaces that
select and then operate on a specific AG. In this case the inode
allocation algorithm does up to three walks across all AGs before it
either allocates an inode or fails. Two of these walks are purely
just to select the AG, and even then it doesn't guarantee inode
allocation success so there's a third walk if the selected AG
allocation fails.

These patches collapse the selection and allocation into a single
loop, simplifies the error handling because xfs_dir_ialloc() always
returns ENOSPC if no AG was selected for inode allocation or we fail
to allocate an inode in any AG, gets rid of xfs_dir_ialloc()
wrapper, converts inode allocation to run entirely from a single
perag instance, and then factors xfs_dialloc() into a much, much
simpler loop which is easy to understand.

Hence we end up with the same inode allocation logic, but it only
needs two complete iterations at worst, makes AG selection and
allocation atomic w.r.t. shrink and chops out out over 100 lines of
code from this hot code path.

[Patch 22]

Converts the unlink path to pass perags through it.

There's more conversion work to be done, but this patchset gets
through a large chunk of it in one hit. Most of the iterators are
converted, so once this is solidified we can move on to converting
these to active references for being able to free perags while the
fs is still active.

----------------------------------------------------------------
Dave Chinner (23):
      xfs: move xfs_perag_get/put to xfs_ag.[ch]
      xfs: prepare for moving perag definitions and support to libxfs
      xfs: move perag structure and setup to libxfs/xfs_ag.[ch]
      xfs: make for_each_perag... a first class citizen
      xfs: convert raw ag walks to use for_each_perag
      xfs: convert xfs_iwalk to use perag references
      xfs: convert secondary superblock walk to use perags
      xfs: pass perags through to the busy extent code
      xfs: push perags through the ag reservation callouts
      xfs: pass perags around in fsmap data dev functions
      xfs: add a perag to the btree cursor
      xfs: convert rmap btree cursor to using a perag
      xfs: convert refcount btree cursor to use perags
      xfs: convert allocbt cursors to use perags
      xfs: use perag for ialloc btree cursors
      xfs: remove agno from btree cursor
      xfs: simplify xfs_dialloc_select_ag() return values
      xfs: collapse AG selection for inode allocation
      xfs: get rid of xfs_dir_ialloc()
      xfs: inode allocation can use a single perag instance
      xfs: clean up and simplify xfs_dialloc()
      xfs: use perag through unlink processing
      xfs: remove xfs_perag_t

 fs/xfs/libxfs/xfs_ag.c             | 272 +++++++++++++++++++++++++++++++++++++++++++++++-
 fs/xfs/libxfs/xfs_ag.h             | 136 ++++++++++++++++++++++++
 fs/xfs/libxfs/xfs_ag_resv.c        |  11 +-
 fs/xfs/libxfs/xfs_ag_resv.h        |  15 +++
 fs/xfs/libxfs/xfs_alloc.c          | 110 ++++++++++----------
 fs/xfs/libxfs/xfs_alloc.h          |   2 +-
 fs/xfs/libxfs/xfs_alloc_btree.c    |  31 +++---
 fs/xfs/libxfs/xfs_alloc_btree.h    |   9 +-
 fs/xfs/libxfs/xfs_attr_leaf.c      |   1 +
 fs/xfs/libxfs/xfs_bmap.c           |   1 +
 fs/xfs/libxfs/xfs_btree.c          |  15 +--
 fs/xfs/libxfs/xfs_btree.h          |  10 +-
 fs/xfs/libxfs/xfs_ialloc.c         | 608 ++++++++++++++++++++++++++++++++++++++++++++++++++----------------------------------------------------------
 fs/xfs/libxfs/xfs_ialloc.h         |  40 ++------
 fs/xfs/libxfs/xfs_ialloc_btree.c   |  46 +++++----
 fs/xfs/libxfs/xfs_ialloc_btree.h   |  13 +--
 fs/xfs/libxfs/xfs_refcount.c       | 122 +++++++++++-----------
 fs/xfs/libxfs/xfs_refcount.h       |   9 +-
 fs/xfs/libxfs/xfs_refcount_btree.c |  39 +++----
 fs/xfs/libxfs/xfs_refcount_btree.h |   7 +-
 fs/xfs/libxfs/xfs_rmap.c           | 147 +++++++++++++-------------
 fs/xfs/libxfs/xfs_rmap.h           |   6 +-
 fs/xfs/libxfs/xfs_rmap_btree.c     |  46 ++++-----
 fs/xfs/libxfs/xfs_rmap_btree.h     |   6 +-
 fs/xfs/libxfs/xfs_sb.c             | 146 ++------------------------
 fs/xfs/libxfs/xfs_sb.h             |   9 --
 fs/xfs/libxfs/xfs_types.c          |   4 +-
 fs/xfs/scrub/agheader.c            |   1 +
 fs/xfs/scrub/agheader_repair.c     |  33 +++---
 fs/xfs/scrub/alloc.c               |   3 +-
 fs/xfs/scrub/bmap.c                |  21 ++--
 fs/xfs/scrub/common.c              |  15 ++-
 fs/xfs/scrub/fscounters.c          |  42 +++-----
 fs/xfs/scrub/health.c              |   2 +-
 fs/xfs/scrub/ialloc.c              |   9 +-
 fs/xfs/scrub/refcount.c            |   3 +-
 fs/xfs/scrub/repair.c              |  14 +--
 fs/xfs/scrub/rmap.c                |   3 +-
 fs/xfs/scrub/trace.c               |   3 +-
 fs/xfs/xfs_buf.c                   |   2 +-
 fs/xfs/xfs_discard.c               |   6 +-
 fs/xfs/xfs_extent_busy.c           |  33 ++----
 fs/xfs/xfs_extent_busy.h           |   7 +-
 fs/xfs/xfs_filestream.c            |   2 +-
 fs/xfs/xfs_fsmap.c                 |  80 +++++++++------
 fs/xfs/xfs_fsops.c                 |   8 +-
 fs/xfs/xfs_health.c                |   6 +-
 fs/xfs/xfs_icache.c                |  17 +--
 fs/xfs/xfs_inode.c                 | 202 ++++++++++++++++--------------------
 fs/xfs/xfs_inode.h                 |   9 +-
 fs/xfs/xfs_iwalk.c                 |  84 +++++++++------
 fs/xfs/xfs_log_recover.c           |  56 +++++-----
 fs/xfs/xfs_mount.c                 | 126 +----------------------
 fs/xfs/xfs_mount.h                 | 110 +-------------------
 fs/xfs/xfs_qm.c                    |  10 +-
 fs/xfs/xfs_reflink.c               |  13 ++-
 fs/xfs/xfs_super.c                 |   1 +
 fs/xfs/xfs_symlink.c               |   9 +-
 fs/xfs/xfs_trace.c                 |   2 +
 fs/xfs/xfs_trace.h                 |   4 +-
 60 files changed, 1408 insertions(+), 1389 deletions(-)

-- 
Dave Chinner
david@fromorbit.com