[RFC DELUGE v9r2d1] xfs: Parent Pointers

* [RFC DELUGE v9r2d1] xfs: Parent Pointers
@ 2023-02-16 20:06 Darrick J. Wong
  2023-02-16 20:26 ` [PATCHSET v9r2d1 00/28] " Darrick J. Wong
                   ` (25 more replies)
  0 siblings, 26 replies; 228+ messages in thread
From: Darrick J. Wong @ 2023-02-16 20:06 UTC (permalink / raw)
  To: allison.henderson; +Cc: linux-xfs

Hi everyone,

This deluge contains all of the additions to the parent pointers
patchset that I've been working on for the past month.  The kernel and
xfsprogs patchsets are based on Allison's v9r2 tag from last week;
the fstests patches are merely a part of my development tree.  To recap
Allison's cover letter:

"The goal of this patch set is to add a parent pointer attribute to each
inode.  The attribute name containing the parent inode, generation, and
directory offset, while the  attribute value contains the file name.
This feature will enable future optimizations for online scrub, shrink,
nfs handles, verity, or any other feature that could make use of quickly
deriving an inodes path from the mount point."

The kernel branches start with a number of buf fixes that I need to get
fstests to pass.  I also restructured the kernel implementation of
GETPARENTS to cut the memory usage considerably.

For userspace, I cleaned up the xfsprogs patches so that libxfs-diff
shows no discrepancies with the kernel and cleaned up the parent pointer
usage code that I prototyped in 2017 so that it's less buggy and moldy.
I also rewired xfs_scrub to use GETPARENTS to report file paths of
corrupt files instead of inode numbers, since that part had bitrotted
badly.

With that out of the way, I implemented a prototype of online repairs
for directories and parent pointers.  This is only a proof of concept,
because I had already backported many many patches from part 1 of online
repair, and didn't feel like porting the parts needed to commit new
structures atomically and reap the old dir/xattr blocks.  IOWs, the
prototype scans the filesystem to build a parallel directory or xattr
structure, and then reports on any discrepancies between the two
versions.  Obviously this won't fix a corrupt directory tree, but it
enables us to test the repair code on a consistent filesystem to
demonstrate that it works.

Next, I implemented fully functional parent pointer checking and repair
for xfs_repair.  This was less hard than I guessed it would be because
the current design of phase 6 includes a walk of all directories.  From
the dirent data, we can build a per-AG index of all the parent pointers
for all the inodes in that AG, then walk all the inodes in that AG to
compare the lists.  As you might guess, this eats a fair amount of
memory, even with a rudimentary dirent name deduplication table to cut
down on memory usage.

After that, I moved on to solving the major problem that I've been
having with the directory repair code, and that is the problem of
reconstructing dirents at the offsets specified by the parent pointers.
The details of the problem and how I dealt with it are captured in the
cover letter for those patches.  Suffice to say, we now encode the
dirent name in the parent pointer attrname (or a collision resistant
hash if it doesn't fit), which makes it possible to commit new
directories atomically.

The last part of this patchset reorganizes the XFS_IOC_GETPARENTS ioctl
to encode variable length parent pointer records in the caller's buffer.
The denser encodings mean that we can extract the parent list with fewer
kernel calls.

--D

^ permalink raw reply	[flat|nested] 228+ messages in thread