linux-xfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [RFC DELUGE v10r1d2] xfs: Parent Pointers
@ 2023-03-16 18:54 Darrick J. Wong
  2023-03-16 19:17 ` [PATCHSET v10r1d2 0/7] xfs: bug fixes for parent pointers Darrick J. Wong
                   ` (10 more replies)
  0 siblings, 11 replies; 95+ messages in thread
From: Darrick J. Wong @ 2023-03-16 18:54 UTC (permalink / raw)
  To: Allison Henderson; +Cc: xfs, fstests

Hi everyone,

This deluge contains all of the additions to the parent pointers
patchset that I've been working since last month's deluge.  The kernel
and xfsprogs patchsets are based on Allison's v10 tag from last week;
the fstests patches are merely a part of my development tree.  To recap
Allison's cover letter:

"The goal of this patch set is to add a parent pointer attribute to each
inode.  The attribute name containing the parent inode, generation, and
directory offset, while the  attribute value contains the file name.
This feature will enable future optimizations for online scrub, shrink,
nfs handles, verity, or any other feature that could make use of quickly
deriving an inodes path from the mount point."

v10r1d2 rebases everything against 6.3-rc2.  I still want to remove the
diroffset from the ondisk parent pointer, but for v10 I've replaced the
sha512 hashing code with modifications to the xattr code to support
lookups based on name *and* value.  With that working, we can encode
parent pointers like this:

	(parent_ino, parent_gen, name[])

xattr lookups still work correctly, and repair doesn't have to deal with
keeping the diroffsets in sync if the directory gets rebuilt.  With this
change applied, I'm ready to weave my new changes into Allison's v10 and
call parent pointers done. :)

The online directory and parent pointer code are exactly the same as the
v9r2d1 release, so I'm eliding that and everything that was in Allison's
recent v10 patchset.  IOWs, this deluge includes only the bug fixes I've
made to parent pointers, the updates I've made to the ondisk format, and
the necessary changes to fstests to get everything to pass.

If you want to pull the whole thing, use these links:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs-drop-unnecessary
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs-drop-unnecessary
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs-name-in-attr-key

Allison: Could you please resync libxfs in the following patches under
https://github.com/allisonhenderson/xfsprogs/commits/xfsprogs_new_pptrs_v10
please?

xfsprogs: add parent pointer support to attribute code
xfsprogs: extend transaction reservations for parent attributes
xfsprogs: parent pointer attribute creation
xfsprogs: remove parent pointers in unlink
xfsprogs: Add parent pointers to rename
xfsprogs: move/add parent pointer validators to xfs_parent

There are discrepancies between the two, which makes ./tools/libxfs-diff
unhappy.  Or, if you want me to merge my ondisk format changes into my
branches, I'll put out v11 with everything taken care of.

--D

^ permalink raw reply	[flat|nested] 95+ messages in thread
* [PATCHSET 00/14] fstests: adjust tests for xfs parent pointers
@ 2023-02-16 20:32 Darrick J. Wong
  2023-02-16 21:15 ` [PATCH 08/14] xfs: add parent pointer test Darrick J. Wong
  0 siblings, 1 reply; 95+ messages in thread
From: Darrick J. Wong @ 2023-02-16 20:32 UTC (permalink / raw)
  To: djwong, zlang
  Cc: Catherine Hoang, Allison Henderson, linux-xfs, fstests, guan

Hi all,

Adjust fstests as necessary to test the new xfs parent pointers feature.
At some point this section needs to grow some specific functionality
tests for repair and dumping.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=pptrs

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=pptrs

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=pptrs
---
 common/parent                        |  209 +++++++
 common/populate                      |   38 +
 common/rc                            |    7 
 common/xfs                           |   12 
 doc/group-names.txt                  |    1 
 src/popdir.pl                        |   11 
 tests/generic/050                    |    9 
 tests/generic/050.cfg                |    1 
 tests/generic/050.out.xfsquotaparent |   23 +
 tests/xfs/018                        |    7 
 tests/xfs/021                        |   15 -
 tests/xfs/021.cfg                    |    1 
 tests/xfs/021.out.default            |    0 
 tests/xfs/021.out.parent             |   64 ++
 tests/xfs/122.out                    |    4 
 tests/xfs/191                        |    7 
 tests/xfs/206                        |    3 
 tests/xfs/288                        |    7 
 tests/xfs/306                        |    9 
 tests/xfs/851                        |  116 ++++
 tests/xfs/851.out                    |   69 ++
 tests/xfs/852                        |   69 ++
 tests/xfs/852.out                    | 1002 ++++++++++++++++++++++++++++++++++
 tests/xfs/853                        |   85 +++
 tests/xfs/853.out                    |   14 
 25 files changed, 1772 insertions(+), 11 deletions(-)
 create mode 100644 common/parent
 create mode 100644 tests/generic/050.out.xfsquotaparent
 create mode 100644 tests/xfs/021.cfg
 rename tests/xfs/{021.out => 021.out.default} (100%)
 create mode 100644 tests/xfs/021.out.parent
 create mode 100755 tests/xfs/851
 create mode 100644 tests/xfs/851.out
 create mode 100755 tests/xfs/852
 create mode 100644 tests/xfs/852.out
 create mode 100755 tests/xfs/853
 create mode 100644 tests/xfs/853.out


^ permalink raw reply	[flat|nested] 95+ messages in thread

end of thread, other threads:[~2023-03-30  1:57 UTC | newest]

Thread overview: 95+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-03-16 18:54 [RFC DELUGE v10r1d2] xfs: Parent Pointers Darrick J. Wong
2023-03-16 19:17 ` [PATCHSET v10r1d2 0/7] xfs: bug fixes for parent pointers Darrick J. Wong
2023-03-16 19:19   ` [PATCH 1/7] xfs: validate parent pointer xattrs in getparent Darrick J. Wong
2023-03-16 19:20   ` [PATCH 2/7] xfs: rename xfs_pptr_info to xfs_getparents Darrick J. Wong
2023-03-16 19:20   ` [PATCH 3/7] xfs: rename xfs_parent_ptr Darrick J. Wong
2023-03-16 19:20   ` [PATCH 4/7] xfs: fix GETPARENTS ioctl Darrick J. Wong
2023-03-16 19:20   ` [PATCH 5/7] xfs: add tracing to the " Darrick J. Wong
2023-03-16 19:21   ` [PATCH 6/7] xfs: shorten parent pointer function names Darrick J. Wong
2023-03-16 19:21   ` [PATCH 7/7] xfs: rearrange bits of the parent pointer apis for fsck Darrick J. Wong
2023-03-16 19:17 ` [PATCHSET v10r1d2 00/17] xfs: encode parent pointer name in xattr key Darrick J. Wong
2023-03-16 19:21   ` [PATCH 01/17] xfs: document the ri_total validation in xlog_recover_attri_commit_pass2 Darrick J. Wong
2023-03-16 19:22   ` [PATCH 02/17] xfs: make xfs_attr_set require XFS_DA_OP_REMOVE Darrick J. Wong
2023-03-16 19:22   ` [PATCH 03/17] xfs: allow xattr matching on value for local/sf attrs Darrick J. Wong
2023-03-16 19:22   ` [PATCH 04/17] xfs: preserve VLOOKUP in xfs_attr_set Darrick J. Wong
2023-03-16 19:22   ` [PATCH 05/17] xfs: restructure xfs_attr_complete_op a bit Darrick J. Wong
2023-03-16 19:23   ` [PATCH 06/17] xfs: use helpers to extract xattr op from opflags Darrick J. Wong
2023-03-16 19:23   ` [PATCH 07/17] xfs: validate recovered name buffers when recovering xattr items Darrick J. Wong
2023-03-16 19:23   ` [PATCH 08/17] xfs: always set args->value in xfs_attri_item_recover Darrick J. Wong
2023-03-16 19:23   ` [PATCH 09/17] xfs: flip nvreplace detection in xfs_attr_complete_op Darrick J. Wong
2023-03-16 19:24   ` [PATCH 10/17] xfs: log VLOOKUP xattr removal operations Darrick J. Wong
2023-03-16 19:24   ` [PATCH 11/17] xfs: log VLOOKUP xattr setting operations Darrick J. Wong
2023-03-16 19:24   ` [PATCH 12/17] xfs: overlay alfi_nname_len atop alfi_name_len for NVREPLACE Darrick J. Wong
2023-03-16 19:24   ` [PATCH 13/17] xfs: refactor value length in xlog_recover_attri_commit_pass2 Darrick J. Wong
2023-03-16 19:25   ` [PATCH 14/17] xfs: rename nname to newname Darrick J. Wong
2023-03-16 19:25   ` [PATCH 15/17] xfs: log VLOOKUP xattr nvreplace operations Darrick J. Wong
2023-03-16 19:25   ` [PATCH 16/17] xfs: log new xattr values for NVREPLACEXXX operations Darrick J. Wong
2023-03-16 19:25   ` [PATCH 17/17] xfs: replace parent pointer diroffset with full dirent name Darrick J. Wong
2023-03-24 17:10   ` [PATCHSET v10r1d2 00/17] xfs: encode parent pointer name in xattr key Allison Henderson
2023-03-25  7:59     ` Amir Goldstein
2023-03-25 17:01       ` Darrick J. Wong
2023-03-26  3:21         ` Amir Goldstein
2023-03-28  1:29           ` Darrick J. Wong
2023-03-28  7:21             ` Amir Goldstein
2023-03-28 22:29             ` Dave Chinner
2023-03-28 23:54               ` Darrick J. Wong
2023-03-29  0:19                 ` Dave Chinner
2023-03-29  0:46                   ` Darrick J. Wong
2023-03-30  1:56                     ` Darrick J. Wong
2023-03-25 17:03     ` Darrick J. Wong
2023-03-16 19:17 ` [PATCHSET v10r1d2 0/9] xfsprogs: tool fixes for parent pointers Darrick J. Wong
2023-03-16 19:26   ` [PATCH 1/9] libxfs: initialize the slab cache for parent defer items Darrick J. Wong
2023-03-16 19:26   ` [PATCH 2/9] mkfs: fix libxfs api misuse Darrick J. Wong
2023-03-16 19:26   ` [PATCH 3/9] libxfs: create new files with attr forks if necessary Darrick J. Wong
2023-03-16 19:26   ` [PATCH 4/9] mkfs: fix subdir parent pointer creation Darrick J. Wong
2023-03-16 19:27   ` [PATCH 5/9] xfs_db: report parent pointer keys Darrick J. Wong
2023-03-16 19:27   ` [PATCH 6/9] xfs_db: obfuscate dirent and pptr names consistently Darrick J. Wong
2023-03-16 19:27   ` [PATCH 7/9] xfs_io: print path in path_print Darrick J. Wong
2023-03-16 19:27   ` [PATCH 8/9] xfs_io: parent command is not experts-only Darrick J. Wong
2023-03-16 19:28   ` [PATCH 9/9] xfs_repair: fix incorrect dabtree hashval comparison Darrick J. Wong
2023-03-16 19:18 ` [PATCHSET v10r1d2 0/2] xfsprogs: actually use getparent ioctl Darrick J. Wong
2023-03-16 19:28   ` [PATCH 1/2] xfs_scrub: revert unnecessary code from "implement the upper half of parent pointers" Darrick J. Wong
2023-03-16 19:28   ` [PATCH 2/2] xfs_scrub: use parent pointers when possible to report file operations Darrick J. Wong
2023-03-16 19:18 ` [PATCHSET v10r1d2 0/7] libfrog: fix parent pointer library code Darrick J. Wong
2023-03-16 19:29   ` [PATCH 1/7] xfs_io: move parent pointer filtering and formatting flags out of libhandle Darrick J. Wong
2023-03-16 19:29   ` [PATCH 2/7] libfrog: remove all the parent pointer code from libhandle Darrick J. Wong
2023-03-16 19:29   ` [PATCH 3/7] libfrog: fix indenting errors in xfss_pptr_alloc Darrick J. Wong
2023-03-16 19:29   ` [PATCH 4/7] libfrog: return positive errno in pptrs.c Darrick J. Wong
2023-03-16 19:30   ` [PATCH 5/7] libfrog: only walk one parent pointer at a time in handle_walk_parent_path_ptr Darrick J. Wong
2023-03-16 19:30   ` [PATCH 6/7] libfrog: trim trailing slashes when printing pptr paths Darrick J. Wong
2023-03-16 19:30   ` [PATCH 7/7] libfrog: fix a buffer overrun in path_list_to_string Darrick J. Wong
2023-03-16 19:18 ` [PATCHSET v10r1d2 0/5] xfsprogs: bug fixes for parent pointers Darrick J. Wong
2023-03-16 19:30   ` [PATCH 1/5] xfs: rename xfs_pptr_info to xfs_getparents Darrick J. Wong
2023-03-16 19:31   ` [PATCH 2/5] xfs: rename xfs_parent_ptr Darrick J. Wong
2023-03-16 19:31   ` [PATCH 3/5] xfs: fix GETPARENTS ioctl Darrick J. Wong
2023-03-16 19:31   ` [PATCH 4/5] xfs: shorten parent pointer function names Darrick J. Wong
2023-03-16 19:31   ` [PATCH 5/5] xfs: rearrange bits of the parent pointer apis for fsck Darrick J. Wong
2023-03-16 19:18 ` [PATCHSET v10r1d2 0/4] xfs_logprint: clean up attri/pptr dumping Darrick J. Wong
2023-03-16 19:32   ` [PATCH 1/4] xfs: revert "xfsprogs: Print pptrs in ATTRI items" Darrick J. Wong
2023-03-16 19:32   ` [PATCH 2/4] xfs_logprint: print missing attri header fields Darrick J. Wong
2023-03-16 19:32   ` [PATCH 3/4] xfs: make logprint note attr names and newnames consistently Darrick J. Wong
2023-03-16 19:32   ` [PATCH 4/4] xfs_logprint: decode parent pointers fully Darrick J. Wong
2023-03-16 19:19 ` [PATCHSET v10r1d2 00/14] fstests: adjust tests for xfs parent pointers Darrick J. Wong
2023-03-16 19:33   ` [PATCH 01/14] xfs/122: update for " Darrick J. Wong
2023-03-16 19:33   ` [PATCH 02/14] populate: create hardlinks " Darrick J. Wong
2023-03-16 19:33   ` [PATCH 03/14] xfs/021: adapt golden output files " Darrick J. Wong
2023-03-16 19:33   ` [PATCH 04/14] generic/050: adapt " Darrick J. Wong
2023-03-16 19:34   ` [PATCH 05/14] xfs/018: disable parent pointers for this test Darrick J. Wong
2023-03-16 19:34   ` [PATCH 06/14] xfs/306: fix formatting failures with parent pointers Darrick J. Wong
2023-03-16 19:34   ` [PATCH 07/14] common: add helpers for parent pointer tests Darrick J. Wong
2023-03-16 19:35   ` [PATCH 08/14] xfs: add parent pointer test Darrick J. Wong
2023-03-16 19:35   ` [PATCH 09/14] xfs: add multi link " Darrick J. Wong
2023-03-16 19:35   ` [PATCH 10/14] xfs: add parent pointer inject test Darrick J. Wong
2023-03-16 19:35   ` [PATCH 11/14] common/parent: add license and copyright Darrick J. Wong
2023-03-16 19:36   ` [PATCH 12/14] common/parent: don't _fail on missing parent pointer components Darrick J. Wong
2023-03-16 19:36   ` [PATCH 13/14] common/parent: check xfs_io parent command paths Darrick J. Wong
2023-03-16 19:36   ` [PATCH 14/14] xfs/851: test xfs_io parent -p too Darrick J. Wong
2023-03-16 19:19 ` [PATCHSET v10r1d2 0/1] xfs: bug fixes for parent pointers Darrick J. Wong
2023-03-16 19:36   ` [PATCH 1/1] xfs/122: fix parent pointer ioctl structure sizes Darrick J. Wong
2023-03-16 19:19 ` [PATCHSET v10r1d2 0/1] fstests: encode parent pointer name in xattr key Darrick J. Wong
2023-03-16 19:37   ` [PATCH 1/1] xfs/{021,122}: adjust parent pointer encoding format Darrick J. Wong
2023-03-17 19:06 ` [RFC DELUGE v10r1d2] xfs: Parent Pointers Allison Henderson
2023-03-17 23:45   ` Darrick J. Wong
2023-03-21 21:14     ` Allison Henderson
2023-03-25 17:02       ` Darrick J. Wong
  -- strict thread matches above, loose matches on Subject: below --
2023-02-16 20:32 [PATCHSET 00/14] fstests: adjust tests for xfs parent pointers Darrick J. Wong
2023-02-16 21:15 ` [PATCH 08/14] xfs: add parent pointer test Darrick J. Wong

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).