* [PATCH v22 0/6] xfs: online repair of inode data
@ 2020-01-01 1:03 Darrick J. Wong
2020-01-01 1:03 ` [PATCH 1/6] xfs: create a blob array data structure Darrick J. Wong
` (5 more replies)
0 siblings, 6 replies; 16+ messages in thread
From: Darrick J. Wong @ 2020-01-01 1:03 UTC (permalink / raw)
To: darrick.wong; +Cc: linux-xfs
Hi all,
For the third part of the twenty-second revision of the online repair
patchset, we implement repair of extended attribute data.
Patch 1 implements a new data structure for storing arbitrary key/value
pairs, which we're going to need to reconstruct extended attribute
forks.
Patches 2-4 clean up the block unmapping code so that we will be able
to perform a mass reset of an inode's fork. This is a key component for
salvaging extended attributes, freeing all the attr fork blocks, and
reconstructing the extended attribute data.
Patch 5 implements extended attribute salvage operations. There is no
redundant or secondary xattr metadata, so the best we can do is trawl
through the attr leaves looking for intact entities.
Patch 6 augments scrub to rebuild extended attributes when any of the
attr blocks are fragmented.
If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy!
Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-inode-data
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=repair-inode-data
^ permalink raw reply [flat|nested] 16+ messages in thread* [PATCH 1/6] xfs: create a blob array data structure 2020-01-01 1:03 [PATCH v22 0/6] xfs: online repair of inode data Darrick J. Wong @ 2020-01-01 1:03 ` Darrick J. Wong 2020-01-01 1:03 ` [PATCH 2/6] xfs: convert xfs_itruncate_extents_flags to use __xfs_bunmapi Darrick J. Wong ` (4 subsequent siblings) 5 siblings, 0 replies; 16+ messages in thread From: Darrick J. Wong @ 2020-01-01 1:03 UTC (permalink / raw) To: darrick.wong; +Cc: linux-xfs From: Darrick J. Wong <darrick.wong@oracle.com> Create a simple 'blob array' data structure for storage of arbitrarily sized metadata objects that will be used to reconstruct metadata. For the intended usage (temporarily storing extended attribute names and values) we only have to support storing objects and retrieving them. Use the xfile abstraction to store the attribute information in memory that can be swapped out. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> --- fs/xfs/Makefile | 1 fs/xfs/scrub/blob.c | 145 +++++++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/blob.h | 24 ++++++++ 3 files changed, 170 insertions(+) create mode 100644 fs/xfs/scrub/blob.c create mode 100644 fs/xfs/scrub/blob.h diff --git a/fs/xfs/Makefile b/fs/xfs/Makefile index f746ba679948..1a9a5e1840d2 100644 --- a/fs/xfs/Makefile +++ b/fs/xfs/Makefile @@ -162,6 +162,7 @@ xfs-y += $(addprefix scrub/, \ alloc_repair.o \ array.o \ bitmap.o \ + blob.o \ bmap_repair.o \ ialloc_repair.o \ inode_repair.o \ diff --git a/fs/xfs/scrub/blob.c b/fs/xfs/scrub/blob.c new file mode 100644 index 000000000000..94912fcb1fd1 --- /dev/null +++ b/fs/xfs/scrub/blob.c @@ -0,0 +1,145 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (C) 2019 Oracle. All Rights Reserved. + * Author: Darrick J. Wong <darrick.wong@oracle.com> + */ +#include "xfs.h" +#include "xfs_fs.h" +#include "xfs_shared.h" +#include "scrub/array.h" +#include "scrub/blob.h" +#include "scrub/xfile.h" + +/* + * XFS Blob Storage + * ================ + * Stores and retrieves blobs using a memfd object. Objects are appended to + * the file and the offset is returned as a magic cookie for retrieval. + */ + +#define XB_KEY_MAGIC 0xABAADDAD +struct xb_key { + uint32_t magic; + uint32_t size; + loff_t offset; + /* blob comes after here */ +} __packed; + +/* Initialize a blob storage object. */ +struct xblob * +xblob_init(void) +{ + struct xblob *blob; + struct file *filp; + int error; + + filp = xfile_create("blob storage"); + if (!filp) + return ERR_PTR(-ENOMEM); + if (IS_ERR(filp)) + return ERR_CAST(filp); + + error = -ENOMEM; + blob = kmem_alloc(sizeof(struct xblob), KM_NOFS | KM_MAYFAIL); + if (!blob) + goto out_filp; + + blob->filp = filp; + blob->last_offset = PAGE_SIZE; + return blob; +out_filp: + fput(filp); + return ERR_PTR(error); +} + +/* Destroy a blob storage object. */ +void +xblob_destroy( + struct xblob *blob) +{ + xfile_destroy(blob->filp); + kmem_free(blob); +} + +/* Retrieve a blob. */ +int +xblob_get( + struct xblob *blob, + xblob_cookie cookie, + void *ptr, + uint32_t size) +{ + struct xb_key key; + loff_t pos = cookie; + int error; + + error = xfile_io(blob->filp, XFILE_IO_READ, &pos, &key, sizeof(key)); + if (error) + return error; + + if (key.magic != XB_KEY_MAGIC || key.offset != cookie) { + ASSERT(0); + return -ENODATA; + } + if (size < key.size) { + ASSERT(0); + return -EFBIG; + } + + return xfile_io(blob->filp, XFILE_IO_READ, &pos, ptr, key.size); +} + +/* Store a blob. */ +int +xblob_put( + struct xblob *blob, + xblob_cookie *cookie, + void *ptr, + uint32_t size) +{ + struct xb_key key = { + .offset = blob->last_offset, + .magic = XB_KEY_MAGIC, + .size = size, + }; + loff_t pos = blob->last_offset; + int error; + + error = xfile_io(blob->filp, XFILE_IO_WRITE, &pos, &key, sizeof(key)); + if (error) + goto out_err; + + error = xfile_io(blob->filp, XFILE_IO_WRITE, &pos, ptr, size); + if (error) + goto out_err; + + *cookie = blob->last_offset; + blob->last_offset = pos; + return 0; +out_err: + xfile_discard(blob->filp, blob->last_offset, pos - 1); + return -ENOMEM; +} + +/* Free a blob. */ +int +xblob_free( + struct xblob *blob, + xblob_cookie cookie) +{ + struct xb_key key; + loff_t pos = cookie; + int error; + + error = xfile_io(blob->filp, XFILE_IO_READ, &pos, &key, sizeof(key)); + if (error) + return error; + + if (key.magic != XB_KEY_MAGIC || key.offset != cookie) { + ASSERT(0); + return -ENODATA; + } + + xfile_discard(blob->filp, cookie, cookie + sizeof(key) + key.size - 1); + return 0; +} diff --git a/fs/xfs/scrub/blob.h b/fs/xfs/scrub/blob.h new file mode 100644 index 000000000000..c6f6c6a2e084 --- /dev/null +++ b/fs/xfs/scrub/blob.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2019 Oracle. All Rights Reserved. + * Author: Darrick J. Wong <darrick.wong@oracle.com> + */ +#ifndef __XFS_SCRUB_BLOB_H__ +#define __XFS_SCRUB_BLOB_H__ + +struct xblob { + struct file *filp; + loff_t last_offset; +}; + +typedef loff_t xblob_cookie; + +struct xblob *xblob_init(void); +void xblob_destroy(struct xblob *blob); +int xblob_get(struct xblob *blob, xblob_cookie cookie, void *ptr, + uint32_t size); +int xblob_put(struct xblob *blob, xblob_cookie *cookie, void *ptr, + uint32_t size); +int xblob_free(struct xblob *blob, xblob_cookie cookie); + +#endif /* __XFS_SCRUB_BLOB_H__ */ ^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH 2/6] xfs: convert xfs_itruncate_extents_flags to use __xfs_bunmapi 2020-01-01 1:03 [PATCH v22 0/6] xfs: online repair of inode data Darrick J. Wong 2020-01-01 1:03 ` [PATCH 1/6] xfs: create a blob array data structure Darrick J. Wong @ 2020-01-01 1:03 ` Darrick J. Wong 2020-01-01 1:04 ` [PATCH 3/6] xfs: remove unnecessary inode-transaction roll Darrick J. Wong ` (3 subsequent siblings) 5 siblings, 0 replies; 16+ messages in thread From: Darrick J. Wong @ 2020-01-01 1:03 UTC (permalink / raw) To: darrick.wong; +Cc: linux-xfs From: Darrick J. Wong <darrick.wong@oracle.com> There's no reason why we can't consume unmap_len, just use the raw version. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> --- fs/xfs/xfs_inode.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index 401da197f012..12eeeea4df16 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -1521,7 +1521,6 @@ xfs_itruncate_extents_flags( xfs_fileoff_t last_block; xfs_filblks_t unmap_len; int error = 0; - int done = 0; ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); ASSERT(!atomic_read(&VFS_I(ip)->i_count) || @@ -1552,10 +1551,10 @@ xfs_itruncate_extents_flags( ASSERT(first_unmap_block < last_block); unmap_len = last_block - first_unmap_block + 1; - while (!done) { + while (unmap_len > 0) { ASSERT(tp->t_firstblock == NULLFSBLOCK); - error = xfs_bunmapi(tp, ip, first_unmap_block, unmap_len, flags, - XFS_ITRUNC_MAX_EXTENTS, &done); + error = __xfs_bunmapi(tp, ip, first_unmap_block, &unmap_len, + flags, XFS_ITRUNC_MAX_EXTENTS); if (error) goto out; ^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH 3/6] xfs: remove unnecessary inode-transaction roll 2020-01-01 1:03 [PATCH v22 0/6] xfs: online repair of inode data Darrick J. Wong 2020-01-01 1:03 ` [PATCH 1/6] xfs: create a blob array data structure Darrick J. Wong 2020-01-01 1:03 ` [PATCH 2/6] xfs: convert xfs_itruncate_extents_flags to use __xfs_bunmapi Darrick J. Wong @ 2020-01-01 1:04 ` Darrick J. Wong 2020-01-01 1:04 ` [PATCH 4/6] xfs: create a new inode fork block unmap helper Darrick J. Wong ` (2 subsequent siblings) 5 siblings, 0 replies; 16+ messages in thread From: Darrick J. Wong @ 2020-01-01 1:04 UTC (permalink / raw) To: darrick.wong; +Cc: linux-xfs From: Darrick J. Wong <darrick.wong@oracle.com> Remove the transaction roll at the end of the loop in xfs_itruncate_extents_flags. xfs_defer_finish takes care of rolling the transaction as needed and reattaching the inode, which means we already start each loop with a clean transaction. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> --- fs/xfs/xfs_inode.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index 12eeeea4df16..c6986517074e 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -1565,10 +1565,6 @@ xfs_itruncate_extents_flags( error = xfs_defer_finish(&tp); if (error) goto out; - - error = xfs_trans_roll_inode(&tp, ip); - if (error) - goto out; } if (whichfork == XFS_DATA_FORK) { ^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH 4/6] xfs: create a new inode fork block unmap helper 2020-01-01 1:03 [PATCH v22 0/6] xfs: online repair of inode data Darrick J. Wong ` (2 preceding siblings ...) 2020-01-01 1:04 ` [PATCH 3/6] xfs: remove unnecessary inode-transaction roll Darrick J. Wong @ 2020-01-01 1:04 ` Darrick J. Wong 2020-01-01 1:04 ` [PATCH 5/6] xfs: repair extended attributes Darrick J. Wong 2020-01-01 1:04 ` [PATCH 6/6] xfs: scrub should set preen if attr leaf has holes Darrick J. Wong 5 siblings, 0 replies; 16+ messages in thread From: Darrick J. Wong @ 2020-01-01 1:04 UTC (permalink / raw) To: darrick.wong; +Cc: linux-xfs From: Darrick J. Wong <darrick.wong@oracle.com> Create a new helper to unmap blocks from an inode's fork. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> --- fs/xfs/libxfs/xfs_bmap.c | 43 +++++++++++++++++++++++++++++++++++++++++++ fs/xfs/libxfs/xfs_bmap.h | 3 +++ fs/xfs/xfs_inode.c | 29 ++++------------------------- 3 files changed, 50 insertions(+), 25 deletions(-) diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c index 84ddd4654975..f15cc7cc3b62 100644 --- a/fs/xfs/libxfs/xfs_bmap.c +++ b/fs/xfs/libxfs/xfs_bmap.c @@ -6327,3 +6327,46 @@ xfs_bmap_validate_extent( return xfs_bmap_validate_extent_raw(ip->i_mount, XFS_IS_REALTIME_INODE(ip), whichfork, irec); } + +/* + * Used in xfs_itruncate_extents(). This is the maximum number of extents + * freed from a file in a single transaction. + */ +#define XFS_ITRUNC_MAX_EXTENTS 2 + +/* + * Unmap every extent in part of an inode's fork. We don't do any higher level + * invalidation work at all. + */ +int +xfs_bunmapi_range( + struct xfs_trans **tpp, + struct xfs_inode *ip, + int whichfork, + xfs_fileoff_t startoff, + xfs_filblks_t unmap_len, + int bunmapi_flags) +{ + int error = 0; + + ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + + bunmapi_flags |= xfs_bmapi_aflag(whichfork); + while (unmap_len > 0) { + ASSERT((*tpp)->t_firstblock == NULLFSBLOCK); + error = __xfs_bunmapi(*tpp, ip, startoff, &unmap_len, + bunmapi_flags, XFS_ITRUNC_MAX_EXTENTS); + if (error) + goto out; + + /* + * Duplicate the transaction that has the permanent + * reservation and commit the old transaction. + */ + error = xfs_defer_finish(tpp); + if (error) + goto out; + } +out: + return error; +} diff --git a/fs/xfs/libxfs/xfs_bmap.h b/fs/xfs/libxfs/xfs_bmap.h index 07c9726d40c2..ec29d5012a49 100644 --- a/fs/xfs/libxfs/xfs_bmap.h +++ b/fs/xfs/libxfs/xfs_bmap.h @@ -286,5 +286,8 @@ xfs_failaddr_t xfs_bmap_validate_extent(struct xfs_inode *ip, int whichfork, int xfs_bmapi_remap(struct xfs_trans *tp, struct xfs_inode *ip, xfs_fileoff_t bno, xfs_filblks_t len, xfs_fsblock_t startblock, int flags); +int xfs_bunmapi_range(struct xfs_trans **tpp, struct xfs_inode *ip, + int whichfork, xfs_fileoff_t startoff, xfs_filblks_t unmap_len, + int bunmapi_flags); #endif /* __XFS_BMAP_H__ */ diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index c6986517074e..8a424fac2d27 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -38,12 +38,6 @@ kmem_zone_t *xfs_inode_zone; -/* - * Used in xfs_itruncate_extents(). This is the maximum number of extents - * freed from a file in a single transaction. - */ -#define XFS_ITRUNC_MAX_EXTENTS 2 - STATIC int xfs_iflush_int(struct xfs_inode *, struct xfs_buf *); STATIC int xfs_iunlink(struct xfs_trans *, struct xfs_inode *); STATIC int xfs_iunlink_remove(struct xfs_trans *, struct xfs_inode *); @@ -1519,7 +1513,6 @@ xfs_itruncate_extents_flags( struct xfs_trans *tp = *tpp; xfs_fileoff_t first_unmap_block; xfs_fileoff_t last_block; - xfs_filblks_t unmap_len; int error = 0; ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); @@ -1533,8 +1526,6 @@ xfs_itruncate_extents_flags( trace_xfs_itruncate_extents_start(ip, new_size); - flags |= xfs_bmapi_aflag(whichfork); - /* * Since it is possible for space to become allocated beyond * the end of the file (in a crash where the space is allocated @@ -1550,22 +1541,10 @@ xfs_itruncate_extents_flags( return 0; ASSERT(first_unmap_block < last_block); - unmap_len = last_block - first_unmap_block + 1; - while (unmap_len > 0) { - ASSERT(tp->t_firstblock == NULLFSBLOCK); - error = __xfs_bunmapi(tp, ip, first_unmap_block, &unmap_len, - flags, XFS_ITRUNC_MAX_EXTENTS); - if (error) - goto out; - - /* - * Duplicate the transaction that has the permanent - * reservation and commit the old transaction. - */ - error = xfs_defer_finish(&tp); - if (error) - goto out; - } + error = xfs_bunmapi_range(&tp, ip, whichfork, first_unmap_block, + last_block - first_unmap_block + 1, flags); + if (error) + goto out; if (whichfork == XFS_DATA_FORK) { /* Remove all pending CoW reservations. */ ^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH 5/6] xfs: repair extended attributes 2020-01-01 1:03 [PATCH v22 0/6] xfs: online repair of inode data Darrick J. Wong ` (3 preceding siblings ...) 2020-01-01 1:04 ` [PATCH 4/6] xfs: create a new inode fork block unmap helper Darrick J. Wong @ 2020-01-01 1:04 ` Darrick J. Wong 2020-01-01 1:04 ` [PATCH 6/6] xfs: scrub should set preen if attr leaf has holes Darrick J. Wong 5 siblings, 0 replies; 16+ messages in thread From: Darrick J. Wong @ 2020-01-01 1:04 UTC (permalink / raw) To: darrick.wong; +Cc: linux-xfs From: Darrick J. Wong <darrick.wong@oracle.com> If the extended attributes look bad, try to sift through the rubble to find whatever keys/values we can, zap the attr tree, and re-add the values. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> --- fs/xfs/Makefile | 1 fs/xfs/scrub/attr.c | 10 - fs/xfs/scrub/attr.h | 10 + fs/xfs/scrub/attr_repair.c | 727 ++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.h | 2 fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/scrub.h | 3 7 files changed, 752 insertions(+), 3 deletions(-) create mode 100644 fs/xfs/scrub/attr_repair.c diff --git a/fs/xfs/Makefile b/fs/xfs/Makefile index 1a9a5e1840d2..257d8ffef721 100644 --- a/fs/xfs/Makefile +++ b/fs/xfs/Makefile @@ -161,6 +161,7 @@ xfs-y += $(addprefix scrub/, \ agheader_repair.o \ alloc_repair.o \ array.o \ + attr_repair.o \ bitmap.o \ blob.o \ bmap_repair.o \ diff --git a/fs/xfs/scrub/attr.c b/fs/xfs/scrub/attr.c index d9f0dd444b80..5ecb194d8ec8 100644 --- a/fs/xfs/scrub/attr.c +++ b/fs/xfs/scrub/attr.c @@ -38,9 +38,15 @@ xchk_setup_xattr_buf( * We need enough space to read an xattr value from the file or enough * space to hold three copies of the xattr free space bitmap. We don't * need the buffer space for both purposes at the same time. + * + * If we're doing a repair, we need enough space to hold the largest + * xattr value and the largest xattr name. */ sz = 3 * sizeof(long) * BITS_TO_LONGS(sc->mp->m_attr_geo->blksize); - sz = max_t(size_t, sz, value_size); + if (sc->sm->sm_flags & XFS_SCRUB_IFLAG_REPAIR) + sz = max_t(size_t, sz, value_size + XATTR_NAME_MAX + 1); + else + sz = max_t(size_t, sz, value_size); /* * If there's already a buffer, figure out if we need to reallocate it @@ -182,7 +188,7 @@ xchk_xattr_listent( * Within a char, the lowest bit of the char represents the byte with * the smallest address */ -STATIC bool +bool xchk_xattr_set_map( struct xfs_scrub *sc, unsigned long *map, diff --git a/fs/xfs/scrub/attr.h b/fs/xfs/scrub/attr.h index 13a1d2e8424d..b2d758953300 100644 --- a/fs/xfs/scrub/attr.h +++ b/fs/xfs/scrub/attr.h @@ -37,6 +37,16 @@ xchk_xattr_valuebuf( return ab->buf; } +/* A place to store attribute names. */ +static inline unsigned char * +xchk_xattr_namebuf( + struct xfs_scrub *sc) +{ + struct xchk_xattr_buf *ab = sc->buf; + + return (unsigned char *)ab->buf + ab->sz - XATTR_NAME_MAX - 1; +} + /* A bitmap of space usage computed by walking an attr leaf block. */ static inline unsigned long * xchk_xattr_usedmap( diff --git a/fs/xfs/scrub/attr_repair.c b/fs/xfs/scrub/attr_repair.c new file mode 100644 index 000000000000..66d2fe00a439 --- /dev/null +++ b/fs/xfs/scrub/attr_repair.c @@ -0,0 +1,727 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (C) 2019 Oracle. All Rights Reserved. + * Author: Darrick J. Wong <darrick.wong@oracle.com> + */ +#include "xfs.h" +#include "xfs_fs.h" +#include "xfs_shared.h" +#include "xfs_format.h" +#include "xfs_trans_resv.h" +#include "xfs_mount.h" +#include "xfs_defer.h" +#include "xfs_btree.h" +#include "xfs_bit.h" +#include "xfs_log_format.h" +#include "xfs_trans.h" +#include "xfs_sb.h" +#include "xfs_inode.h" +#include "xfs_da_format.h" +#include "xfs_da_btree.h" +#include "xfs_dir2.h" +#include "xfs_attr.h" +#include "xfs_attr_leaf.h" +#include "xfs_attr_sf.h" +#include "xfs_attr_remote.h" +#include "xfs_bmap.h" +#include "scrub/xfs_scrub.h" +#include "scrub/scrub.h" +#include "scrub/common.h" +#include "scrub/trace.h" +#include "scrub/repair.h" +#include "scrub/array.h" +#include "scrub/blob.h" +#include "scrub/attr.h" + +/* + * Extended Attribute Repair + * ========================= + * + * We repair extended attributes by reading the attribute fork blocks looking + * for keys and values, then truncate the entire attr fork and reinsert all + * the attributes. Unfortunately, there's no secondary copy of most extended + * attribute data, which means that if we blow up midway through there's + * little we can do. + */ + +struct xrep_xattr_key { + xblob_cookie value_cookie; + xblob_cookie name_cookie; + uint hash; + int flags; + uint32_t valuelen; + uint16_t namelen; +} __packed; + +struct xrep_xattr { + struct xfs_scrub *sc; + struct xfbma *xattr_records; + struct xblob *xattr_blobs; + + /* Size of the largest attribute value we're trying to salvage. */ + size_t max_valuelen; +}; + +/* + * Decide if we want to salvage this attribute. We don't bother with + * incomplete or oversized keys or values. + */ +STATIC int +xrep_xattr_want_salvage( + int flags, + const void *name, + int namelen, + int valuelen) +{ + if (flags & XFS_ATTR_INCOMPLETE) + return false; + if (namelen > XATTR_NAME_MAX || namelen <= 0) + return false; + if (valuelen > XATTR_SIZE_MAX || valuelen < 0) + return false; + if (!xfs_attr_namecheck(name, namelen)) + return false; + return true; +} + +/* Allocate an in-core record to hold xattrs while we rebuild the xattr data. */ +STATIC int +xrep_xattr_salvage_key( + struct xrep_xattr *rx, + int flags, + unsigned char *name, + int namelen, + unsigned char *value, + int valuelen) +{ + struct xrep_xattr_key key = { + .valuelen = valuelen, + .flags = flags & (XFS_ATTR_ROOT | XFS_ATTR_SECURE), + .namelen = namelen, + }; + int error = 0; + + if (xchk_should_terminate(rx->sc, &error)) + return error; + + error = xblob_put(rx->xattr_blobs, &key.name_cookie, name, namelen); + if (error) + return error; + error = xblob_put(rx->xattr_blobs, &key.value_cookie, value, valuelen); + if (error) + return error; + + key.hash = xfs_da_hashname(name, namelen); + + error = xfbma_append(rx->xattr_records, &key); + if (error) + return error; + + rx->max_valuelen = max_t(size_t, rx->max_valuelen, valuelen); + return 0; +} + +/* + * Record a shortform extended attribute key & value for later reinsertion + * into the inode. + */ +STATIC int +xrep_xattr_salvage_sf_attr( + struct xrep_xattr *rx, + struct xfs_attr_sf_entry *sfe) +{ + unsigned char *value = &sfe->nameval[sfe->namelen]; + + if (!xrep_xattr_want_salvage(sfe->flags, sfe->nameval, sfe->namelen, + sfe->valuelen)) + return 0; + + return xrep_xattr_salvage_key(rx, sfe->flags, sfe->nameval, + sfe->namelen, value, sfe->valuelen); +} + +/* + * Record a local format extended attribute key & value for later reinsertion + * into the inode. + */ +STATIC int +xrep_xattr_salvage_local_attr( + struct xrep_xattr *rx, + struct xfs_attr_leaf_entry *ent, + unsigned int nameidx, + const char *buf_end, + struct xfs_attr_leaf_name_local *lentry) +{ + unsigned char *value; + unsigned long *usedmap = xchk_xattr_usedmap(rx->sc); + unsigned int valuelen; + unsigned int namesize; + + /* + * Decode the leaf local entry format. If something seems wrong, we + * junk the attribute. + */ + valuelen = be16_to_cpu(lentry->valuelen); + namesize = xfs_attr_leaf_entsize_local(lentry->namelen, valuelen); + if ((char *)lentry + namesize > buf_end) + return 0; + if (!xrep_xattr_want_salvage(ent->flags, lentry->nameval, + lentry->namelen, valuelen)) + return 0; + if (!xchk_xattr_set_map(rx->sc, usedmap, nameidx, namesize)) + return 0; + + /* Try to save this attribute. */ + value = &lentry->nameval[lentry->namelen]; + return xrep_xattr_salvage_key(rx, ent->flags, lentry->nameval, + lentry->namelen, value, valuelen); +} + +/* + * Record a remote format extended attribute key & value for later reinsertion + * into the inode. + */ +STATIC int +xrep_xattr_salvage_remote_attr( + struct xrep_xattr *rx, + struct xfs_attr_leaf_entry *ent, + unsigned int nameidx, + const char *buf_end, + struct xfs_attr_leaf_name_remote *rentry, + unsigned int ent_idx, + struct xfs_buf *leaf_bp) +{ + struct xfs_da_args args = { + .trans = rx->sc->tp, + .dp = rx->sc->ip, + .index = ent_idx, + .geo = rx->sc->mp->m_attr_geo, + }; + unsigned long *usedmap = xchk_xattr_usedmap(rx->sc); + unsigned char *value; + unsigned int valuelen; + unsigned int namesize; + int error; + + /* + * Decode the leaf remote entry format. If something seems wrong, we + * junk the attribute. Note that we should never find a zero-length + * remote attribute value. + */ + valuelen = be32_to_cpu(rentry->valuelen); + namesize = xfs_attr_leaf_entsize_remote(rentry->namelen); + if ((char *)rentry + namesize > buf_end) + return 0; + if (valuelen == 0 || + !xrep_xattr_want_salvage(ent->flags, rentry->name, rentry->namelen, + valuelen)) + return 0; + if (!xchk_xattr_set_map(rx->sc, usedmap, nameidx, namesize)) + return 0; + + /* + * Find somewhere to save this value. We can't use the xchk_xattr_buf + * here because we're still using the memory for the attr block bitmap. + */ + value = kmem_alloc_large(valuelen, KM_MAYFAIL); + if (!value) + return -ENOMEM; + + /* Look up the remote value and stash it for reconstruction. */ + args.valuelen = valuelen; + args.namelen = rentry->namelen; + args.name = rentry->name; + args.value = value; + error = xfs_attr3_leaf_getvalue(leaf_bp, &args); + if (error || args.rmtblkno == 0) + goto err_free; + + error = xfs_attr_rmtval_get(&args); + if (error) + goto err_free; + + /* Try to save this attribute. */ + error = xrep_xattr_salvage_key(rx, ent->flags, rentry->name, + rentry->namelen, value, valuelen); +err_free: + /* remote value was garbage, junk it */ + if (error == -EFSBADCRC || error == -EFSCORRUPTED) + error = 0; + kmem_free(value); + return error; +} + +/* Extract every xattr key that we can from this attr fork block. */ +STATIC int +xrep_xattr_recover_leaf( + struct xrep_xattr *rx, + struct xfs_buf *bp) +{ + struct xfs_attr3_icleaf_hdr leafhdr; + struct xfs_scrub *sc = rx->sc; + struct xfs_mount *mp = sc->mp; + struct xfs_attr_leafblock *leaf; + unsigned long *usedmap = xchk_xattr_usedmap(sc); + struct xfs_attr_leaf_name_local *lentry; + struct xfs_attr_leaf_name_remote *rentry; + struct xfs_attr_leaf_entry *ent; + struct xfs_attr_leaf_entry *entries; + char *buf_end; + size_t off; + unsigned int nameidx; + unsigned int hdrsize; + int i; + int error = 0; + + bitmap_zero(usedmap, mp->m_attr_geo->blksize); + + /* Check the leaf header */ + leaf = bp->b_addr; + xfs_attr3_leaf_hdr_from_disk(mp->m_attr_geo, &leafhdr, leaf); + hdrsize = xfs_attr3_leaf_hdr_size(leaf); + xchk_xattr_set_map(sc, usedmap, 0, hdrsize); + entries = xfs_attr3_leaf_entryp(leaf); + + buf_end = (char *)bp->b_addr + mp->m_attr_geo->blksize; + for (i = 0, ent = entries; i < leafhdr.count; ent++, i++) { + if (xchk_should_terminate(sc, &error)) + break; + + /* Skip key if it conflicts with something else? */ + off = (char *)ent - (char *)leaf; + if (!xchk_xattr_set_map(sc, usedmap, off, + sizeof(xfs_attr_leaf_entry_t))) + continue; + + /* Check the name information. */ + nameidx = be16_to_cpu(ent->nameidx); + if (nameidx < leafhdr.firstused || + nameidx >= mp->m_attr_geo->blksize) + continue; + + if (ent->flags & XFS_ATTR_LOCAL) { + lentry = xfs_attr3_leaf_name_local(leaf, i); + error = xrep_xattr_salvage_local_attr(rx, ent, nameidx, + buf_end, lentry); + } else { + rentry = xfs_attr3_leaf_name_remote(leaf, i); + error = xrep_xattr_salvage_remote_attr(rx, ent, nameidx, + buf_end, rentry, i, bp); + } + if (error) + break; + } + + return error; +} + +/* Try to recover shortform attrs. */ +STATIC int +xrep_xattr_recover_sf( + struct xrep_xattr *rx) +{ + struct xfs_attr_shortform *sf; + struct xfs_attr_sf_entry *sfe; + struct xfs_attr_sf_entry *next; + struct xfs_ifork *ifp; + unsigned char *end; + int i; + int error; + + ifp = XFS_IFORK_PTR(rx->sc->ip, XFS_ATTR_FORK); + sf = (struct xfs_attr_shortform *)rx->sc->ip->i_afp->if_u1.if_data; + end = (unsigned char *)ifp->if_u1.if_data + ifp->if_bytes; + + for (i = 0, sfe = &sf->list[0]; i < sf->hdr.count; i++) { + if (xchk_should_terminate(rx->sc, &error)) + break; + + next = XFS_ATTR_SF_NEXTENTRY(sfe); + if ((unsigned char *)next > end) + break; + + /* Ok, let's save this key/value. */ + error = xrep_xattr_salvage_sf_attr(rx, sfe); + if (error) + return error; + + sfe = next; + } + + return 0; +} + +/* Extract as many attribute keys and values as we can. */ +STATIC int +xrep_xattr_recover( + struct xrep_xattr *rx) +{ + struct xfs_iext_cursor icur; + struct xfs_bmbt_irec got; + struct xfs_scrub *sc = rx->sc; + struct xfs_ifork *ifp; + struct xfs_da_blkinfo *info; + struct xfs_buf *bp; + struct xfs_da_geometry *geo = sc->mp->m_attr_geo; + xfs_dablk_t dabno; + int error = 0; + + if (sc->ip->i_d.di_aformat == XFS_DINODE_FMT_LOCAL) + return xrep_xattr_recover_sf(rx); + + /* + * Set the xchk_attr_buf to be as large as we're going to need it to be + * to compute space usage bitmaps for each attr block we try to + * salvage. We don't salvage attrs whose name and value areas are + * crosslinked with anything else. + */ + error = xchk_setup_xattr_buf(sc, 0, KM_MAYFAIL); + if (error == -ENOMEM) + return -EDEADLOCK; + if (error) + return error; + + /* Iterate each attr block in the attr fork. */ + ifp = XFS_IFORK_PTR(sc->ip, XFS_ATTR_FORK); + for_each_xfs_iext(ifp, &icur, &got) { + xfs_trim_extent(&got, 0, (xfs_dablk_t)-1U); + if (got.br_blockcount == 0) + continue; + for (dabno = round_up(got.br_startoff, geo->fsbcount); + dabno < got.br_startoff + got.br_blockcount; + dabno += geo->fsbcount) { + if (xchk_should_terminate(sc, &error)) + return error; + + /* + * Try to read buffer. We invalidate them in the next + * step so we don't bother to set a buffer type or + * ops. + */ + error = xfs_da_read_buf(sc->tp, sc->ip, dabno, -1, &bp, + XFS_ATTR_FORK, NULL); + if (error || !bp) + continue; + + /* Screen out non-leaves & other garbage. */ + info = bp->b_addr; + if (info->magic != cpu_to_be16(XFS_ATTR3_LEAF_MAGIC) || + xfs_attr3_leaf_buf_ops.verify_struct(bp) != NULL) + continue; + + error = xrep_xattr_recover_leaf(rx, bp); + if (error) + return error; + } + } + + return error; +} + +/* Reset a shortform attr fork. */ +static void +xrep_xattr_reset_attr_local( + struct xfs_scrub *sc, + uint64_t nr_attrs) +{ + struct xfs_attr_sf_hdr *hdr; + struct xfs_ifork *ifp; + + /* + * If the data fork isn't in btree format (or there are no attrs) then + * all we need to do is zap the attr fork. + */ + if (nr_attrs == 0 || sc->ip->i_d.di_format != XFS_DINODE_FMT_BTREE) { + xfs_attr_fork_remove(sc->ip, sc->tp); + return; + } + + /* + * If the data fork is in btree format we can't change di_forkoff + * because we could run afoul of the rule that forks aren't supposed to + * be in btree format if there's enough space in the fork that we could + * have extents format. Instead, reinitialize the shortform fork to + * have zero attributes. + */ + ifp = XFS_IFORK_PTR(sc->ip, XFS_ATTR_FORK); + xfs_idata_realloc(sc->ip, (int)sizeof(*hdr) - ifp->if_bytes, + XFS_ATTR_FORK); + hdr = (struct xfs_attr_sf_hdr *)ifp->if_u1.if_data; + hdr->count = 0; + hdr->totsize = cpu_to_be16(sizeof(*hdr)); + xfs_trans_log_inode(sc->tp, sc->ip, XFS_ILOG_CORE | XFS_ILOG_ADATA); +} + +/* Free all the attribute fork blocks and delete the fork. */ +STATIC int +xrep_xattr_reset_fork( + struct xfs_scrub *sc, + uint64_t nr_attrs) +{ + struct xfs_iext_cursor icur; + struct xfs_bmbt_irec got; + struct xfs_ifork *ifp; + struct xfs_buf *bp; + struct xfs_da_geometry *geo = sc->mp->m_attr_geo; + xfs_dablk_t dabno; + int error; + + xfs_trans_ijoin(sc->tp, sc->ip, 0); + + if (sc->ip->i_d.di_aformat == XFS_DINODE_FMT_LOCAL) { + xrep_xattr_reset_attr_local(sc, nr_attrs); + return 0; + } + + /* Invalidate each attr block in the attr fork. */ + ifp = XFS_IFORK_PTR(sc->ip, XFS_ATTR_FORK); + for_each_xfs_iext(ifp, &icur, &got) { + xfs_trim_extent(&got, 0, (xfs_dablk_t)-1U); + if (got.br_blockcount == 0) + continue; + for (dabno = round_up(got.br_startoff, geo->fsbcount); + dabno < got.br_startoff + got.br_blockcount; + dabno += geo->fsbcount) { + error = xfs_da_get_buf(sc->tp, sc->ip, dabno, &bp, + XFS_ATTR_FORK); + if (error || !bp) + continue; + xfs_trans_binval(sc->tp, bp); + error = xfs_trans_roll_inode(&sc->tp, sc->ip); + if (error) + return error; + } + } + + /* Now free all the blocks. */ + error = xfs_bunmapi_range(&sc->tp, sc->ip, XFS_ATTR_FORK, 0, -1ULL, + XFS_BMAPI_NODISCARD); + if (error) + return error; + + /* Log the inode core to keep it moving forward in the log. */ + xfs_trans_log_inode(sc->tp, sc->ip, XFS_ILOG_CORE); + + /* Reset the attribute fork - this also destroys the in-core fork */ + xfs_attr_fork_remove(sc->ip, sc->tp); + return 0; +} + +/* + * Compare two xattr keys. ATTR_SECURE keys come before ATTR_ROOT and + * ATTR_ROOT keys come before user attrs. Otherwise sort in hash order. + */ +static int +xrep_xattr_key_cmp( + const void *a, + const void *b) +{ + const struct xrep_xattr_key *ap = a; + const struct xrep_xattr_key *bp = b; + + if (ap->flags > bp->flags) + return 1; + else if (ap->flags < bp->flags) + return -1; + + if (ap->hash > bp->hash) + return 1; + else if (ap->hash < bp->hash) + return -1; + return 0; +} + +/* + * Find all the extended attributes for this inode by scraping them out of the + * attribute key blocks by hand. The caller must clean up the lists if + * anything goes wrong. + */ +STATIC int +xrep_xattr_find_attributes( + struct xfs_scrub *sc, + struct xfbma *xattr_records, + struct xblob *xattr_blobs) +{ + struct xrep_xattr rx = { + .sc = sc, + .xattr_records = xattr_records, + .xattr_blobs = xattr_blobs, + }; + struct xfs_ifork *ifp; + int error; + + error = xrep_ino_dqattach(sc); + if (error) + return error; + + /* Extent map should be loaded. */ + ifp = XFS_IFORK_PTR(sc->ip, XFS_ATTR_FORK); + if (XFS_IFORK_FORMAT(sc->ip, XFS_ATTR_FORK) != XFS_DINODE_FMT_LOCAL && + !(ifp->if_flags & XFS_IFEXTENTS)) { + error = xfs_iread_extents(sc->tp, sc->ip, XFS_ATTR_FORK); + if (error) + return error; + } + + /* Read every attr key and value and record them in memory. */ + error = xrep_xattr_recover(&rx); + if (error) + return error; + + /* + * Reset the xchk_attr_buf to be as large as we're going to need it to + * be to store each attribute name and value as we re-add them to the + * file. We must preallocate the memory here because once we start + * to modify the filesystem we cannot afford an ENOMEM. + */ + error = xchk_setup_xattr_buf(sc, rx.max_valuelen, KM_MAYFAIL); + if (error == -ENOMEM) + return -EDEADLOCK; + if (error) + return error; + + return 0; +} + +struct xrep_add_attr { + struct xfs_scrub *sc; + struct xfbma *xattr_records; + struct xblob *xattr_blobs; +}; + +/* Insert one xattr key/value. */ +STATIC int +xrep_xattr_insert_rec( + const void *item, + void *priv) +{ + const struct xrep_xattr_key *key = item; + struct xrep_add_attr *x = priv; + unsigned char *name = xchk_xattr_namebuf(x->sc); + unsigned char *value = xchk_xattr_valuebuf(x->sc); + int error; + + /* + * The attribute name is stored near the end of the in-core buffer, + * though we reserve one more byte to ensure null termination. + */ + name[XATTR_NAME_MAX] = 0; + + error = xblob_get(x->xattr_blobs, key->name_cookie, name, key->namelen); + if (error) + return error; + + error = xblob_free(x->xattr_blobs, key->name_cookie); + if (error) + return error; + + error = xblob_get(x->xattr_blobs, key->value_cookie, value, + key->valuelen); + if (error) + return error; + + error = xblob_free(x->xattr_blobs, key->value_cookie); + if (error) + return error; + + name[key->namelen] = 0; + value[key->valuelen] = 0; + + return xfs_attr_set(x->sc->ip, name, value, key->valuelen, + XFS_ATTR_NSP_ONDISK_TO_ARGS(key->flags)); +} + +/* + * Insert all the attributes that we collected. + * + * Commit the repair transaction and drop the ilock because the attribute + * setting code needs to be able to allocate special transactions and take the + * ilock on its own. Some day we'll have deferred attribute setting, at which + * point we'll be able to use that to replace the attributes atomically and + * safely. + */ +STATIC int +xrep_xattr_rebuild_tree( + struct xfs_scrub *sc, + struct xfbma *xattr_records, + struct xblob *xattr_blobs) +{ + struct xrep_add_attr x = { + .sc = sc, + .xattr_records = xattr_records, + .xattr_blobs = xattr_blobs, + }; + int error; + + error = xfs_trans_commit(sc->tp); + sc->tp = NULL; + if (error) + return error; + + xfs_iunlock(sc->ip, XFS_ILOCK_EXCL); + sc->ilock_flags &= ~XFS_ILOCK_EXCL; + + /* + * Sort the attribute keys by hash to minimize dabtree splits when we + * rebuild the extended attribute information. + */ + error = xfbma_sort(xattr_records, xrep_xattr_key_cmp); + if (error) + return error; + + /* Re-add every attr to the file. */ + return xfbma_iter_del(xattr_records, xrep_xattr_insert_rec, &x); +} + +/* + * Repair the extended attribute metadata. + * + * XXX: Remote attribute value buffers encompass the entire (up to 64k) buffer. + * The buffer cache in XFS can't handle aliased multiblock buffers, so this + * might misbehave if the attr fork is crosslinked with other filesystem + * metadata. + */ +int +xrep_xattr( + struct xfs_scrub *sc) +{ + struct xfbma *xattr_records; + struct xblob *xattr_blobs; + int error; + + if (!xfs_inode_hasattr(sc->ip)) + return -ENOENT; + + /* Set up some storage */ + xattr_records = xfbma_init(sizeof(struct xrep_xattr_key)); + if (IS_ERR(xattr_records)) + return PTR_ERR(xattr_records); + xattr_blobs = xblob_init(); + if (IS_ERR(xattr_blobs)) { + error = PTR_ERR(xattr_blobs); + goto out_arr; + } + + /* Collect extended attributes by parsing raw blocks. */ + error = xrep_xattr_find_attributes(sc, xattr_records, xattr_blobs); + if (error) + goto out; + + /* + * Invalidate and truncate all attribute fork extents. This is the + * point at which we are no longer able to bail out gracefully. + * We commit the transaction here because xfs_attr_set allocates its + * own transactions. + */ + error = xrep_xattr_reset_fork(sc, xfbma_length(xattr_records)); + if (error) + goto out; + + /* Now rebuild the attribute information. */ + error = xrep_xattr_rebuild_tree(sc, xattr_records, xattr_blobs); +out: + xblob_destroy(xattr_blobs); +out_arr: + xfbma_destroy(xattr_records); + return error; +} diff --git a/fs/xfs/scrub/repair.h b/fs/xfs/scrub/repair.h index 4ff872b7b8cd..4282e249be14 100644 --- a/fs/xfs/scrub/repair.h +++ b/fs/xfs/scrub/repair.h @@ -75,6 +75,7 @@ int xrep_inode(struct xfs_scrub *sc); int xrep_bmap_data(struct xfs_scrub *sc); int xrep_bmap_attr(struct xfs_scrub *sc); int xrep_symlink(struct xfs_scrub *sc); +int xrep_xattr(struct xfs_scrub *sc); struct xrep_newbt_resv { /* Link to list of extents that we've reserved. */ @@ -177,6 +178,7 @@ xrep_reset_perag_resv( #define xrep_bmap_data xrep_notsupported #define xrep_bmap_attr xrep_notsupported #define xrep_symlink xrep_notsupported +#define xrep_xattr xrep_notsupported #endif /* CONFIG_XFS_ONLINE_REPAIR */ diff --git a/fs/xfs/scrub/scrub.c b/fs/xfs/scrub/scrub.c index ff06bc5031ae..86493e6cc712 100644 --- a/fs/xfs/scrub/scrub.c +++ b/fs/xfs/scrub/scrub.c @@ -291,7 +291,7 @@ static const struct xchk_meta_ops meta_scrub_ops[] = { .type = ST_INODE, .setup = xchk_setup_xattr, .scrub = xchk_xattr, - .repair = xrep_notsupported, + .repair = xrep_xattr, }, [XFS_SCRUB_TYPE_SYMLINK] = { /* symbolic link */ .type = ST_INODE, diff --git a/fs/xfs/scrub/scrub.h b/fs/xfs/scrub/scrub.h index 16ed1d3e1404..99c4a3021284 100644 --- a/fs/xfs/scrub/scrub.h +++ b/fs/xfs/scrub/scrub.h @@ -170,4 +170,7 @@ struct xchk_fscounters { unsigned long long icount_max; }; +bool xchk_xattr_set_map(struct xfs_scrub *sc, unsigned long *map, + unsigned int start, unsigned int len); + #endif /* __XFS_SCRUB_SCRUB_H__ */ ^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH 6/6] xfs: scrub should set preen if attr leaf has holes 2020-01-01 1:03 [PATCH v22 0/6] xfs: online repair of inode data Darrick J. Wong ` (4 preceding siblings ...) 2020-01-01 1:04 ` [PATCH 5/6] xfs: repair extended attributes Darrick J. Wong @ 2020-01-01 1:04 ` Darrick J. Wong 5 siblings, 0 replies; 16+ messages in thread From: Darrick J. Wong @ 2020-01-01 1:04 UTC (permalink / raw) To: darrick.wong; +Cc: linux-xfs, Dave Chinner From: Darrick J. Wong <darrick.wong@oracle.com> If an attr block indicates that it could use compaction, set the preen flag to have the attr fork rebuilt, since the attr fork rebuilder can take care of that for us. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Dave Chinner <dchinner@redhat.com> --- fs/xfs/scrub/attr.c | 2 ++ fs/xfs/scrub/dabtree.c | 16 ++++++++++++++++ fs/xfs/scrub/dabtree.h | 1 + fs/xfs/scrub/trace.h | 1 + 4 files changed, 20 insertions(+) diff --git a/fs/xfs/scrub/attr.c b/fs/xfs/scrub/attr.c index 5ecb194d8ec8..3fbde6b8f852 100644 --- a/fs/xfs/scrub/attr.c +++ b/fs/xfs/scrub/attr.c @@ -364,6 +364,8 @@ xchk_xattr_block( xchk_da_set_corrupt(ds, level); if (!xchk_xattr_set_map(ds->sc, usedmap, 0, hdrsize)) xchk_da_set_corrupt(ds, level); + if (leafhdr.holes) + xchk_da_set_preen(ds, level); if (ds->sc->sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT) goto out; diff --git a/fs/xfs/scrub/dabtree.c b/fs/xfs/scrub/dabtree.c index 97a15b6f2865..e9c087b8987a 100644 --- a/fs/xfs/scrub/dabtree.c +++ b/fs/xfs/scrub/dabtree.c @@ -77,6 +77,22 @@ xchk_da_set_corrupt( __return_address); } +/* Flag a da btree node in need of optimization. */ +void +xchk_da_set_preen( + struct xchk_da_btree *ds, + int level) +{ + struct xfs_scrub *sc = ds->sc; + + sc->sm->sm_flags |= XFS_SCRUB_OFLAG_PREEN; + trace_xchk_fblock_preen(sc, ds->dargs.whichfork, + xfs_dir2_da_to_db(ds->dargs.geo, + ds->state->path.blk[level].blkno), + __return_address); +} + +/* Find an entry at a certain level in a da btree. */ static struct xfs_da_node_entry * xchk_da_btree_node_entry( struct xchk_da_btree *ds, diff --git a/fs/xfs/scrub/dabtree.h b/fs/xfs/scrub/dabtree.h index 1f3515c6d5a8..8066fa00dc1b 100644 --- a/fs/xfs/scrub/dabtree.h +++ b/fs/xfs/scrub/dabtree.h @@ -35,6 +35,7 @@ bool xchk_da_process_error(struct xchk_da_btree *ds, int level, int *error); /* Check for da btree corruption. */ void xchk_da_set_corrupt(struct xchk_da_btree *ds, int level); +void xchk_da_set_preen(struct xchk_da_btree *ds, int level); int xchk_da_btree_hash(struct xchk_da_btree *ds, int level, __be32 *hashp); int xchk_da_btree(struct xfs_scrub *sc, int whichfork, diff --git a/fs/xfs/scrub/trace.h b/fs/xfs/scrub/trace.h index 592f9512115c..7b208e36b8e9 100644 --- a/fs/xfs/scrub/trace.h +++ b/fs/xfs/scrub/trace.h @@ -298,6 +298,7 @@ DEFINE_EVENT(xchk_fblock_error_class, name, \ DEFINE_SCRUB_FBLOCK_ERROR_EVENT(xchk_fblock_error); DEFINE_SCRUB_FBLOCK_ERROR_EVENT(xchk_fblock_warning); +DEFINE_SCRUB_FBLOCK_ERROR_EVENT(xchk_fblock_preen); TRACE_EVENT(xchk_incomplete, TP_PROTO(struct xfs_scrub *sc, void *ret_ip), ^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCHSET v29.0 20/28] xfs: online repair of extended attributes @ 2023-12-31 19:30 Darrick J. Wong 2023-12-31 20:35 ` [PATCH 1/6] xfs: create a blob array data structure Darrick J. Wong 0 siblings, 1 reply; 16+ messages in thread From: Darrick J. Wong @ 2023-12-31 19:30 UTC (permalink / raw) To: djwong; +Cc: Dave Chinner, linux-xfs Hi all, This series employs atomic extent swapping to enable safe reconstruction of extended attribute data attached to a file. Because xattrs do not have any redundant information to draw off of, we can at best salvage as much data as we can and build a new structure. Rebuilding an extended attribute structure consists of these three steps: First, we walk the existing attributes to salvage as many of them as we can, by adding them as new attributes attached to the repair tempfile. We need to add a new xfile-based data structure to hold blobs of arbitrary length to stage the xattr names and values. Second, we write the salvaged attributes to a temporary file, and use atomic extent swaps to exchange the entire attribute fork between the two files. Finally, we reap the old xattr blocks (which are now in the temporary file) as carefully as we can. If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below. This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome. --D kernel git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-xattrs xfsprogs git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=repair-xattrs fstests git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=repair-xattrs --- fs/xfs/Makefile | 3 fs/xfs/libxfs/xfs_attr.c | 2 fs/xfs/libxfs/xfs_attr.h | 2 fs/xfs/libxfs/xfs_da_format.h | 5 fs/xfs/libxfs/xfs_swapext.c | 2 fs/xfs/libxfs/xfs_swapext.h | 1 fs/xfs/scrub/attr.c | 158 +++-- fs/xfs/scrub/attr.h | 7 fs/xfs/scrub/attr_repair.c | 1203 +++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/attr_repair.h | 11 fs/xfs/scrub/dab_bitmap.h | 37 + fs/xfs/scrub/dabtree.c | 16 + fs/xfs/scrub/dabtree.h | 3 fs/xfs/scrub/listxattr.c | 310 +++++++++++ fs/xfs/scrub/listxattr.h | 17 + fs/xfs/scrub/repair.c | 46 ++ fs/xfs/scrub/repair.h | 6 fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/tempfile.c | 203 +++++++ fs/xfs/scrub/tempfile.h | 3 fs/xfs/scrub/tempswap.h | 2 fs/xfs/scrub/trace.h | 84 +++ fs/xfs/scrub/xfarray.c | 17 + fs/xfs/scrub/xfarray.h | 2 fs/xfs/scrub/xfblob.c | 168 ++++++ fs/xfs/scrub/xfblob.h | 26 + fs/xfs/scrub/xfile.h | 12 fs/xfs/xfs_buf.c | 3 fs/xfs/xfs_trace.h | 2 29 files changed, 2270 insertions(+), 83 deletions(-) create mode 100644 fs/xfs/scrub/attr_repair.c create mode 100644 fs/xfs/scrub/attr_repair.h create mode 100644 fs/xfs/scrub/dab_bitmap.h create mode 100644 fs/xfs/scrub/listxattr.c create mode 100644 fs/xfs/scrub/listxattr.h create mode 100644 fs/xfs/scrub/xfblob.c create mode 100644 fs/xfs/scrub/xfblob.h ^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH 1/6] xfs: create a blob array data structure 2023-12-31 19:30 [PATCHSET v29.0 20/28] xfs: online repair of extended attributes Darrick J. Wong @ 2023-12-31 20:35 ` Darrick J. Wong 2024-01-05 5:53 ` Christoph Hellwig 0 siblings, 1 reply; 16+ messages in thread From: Darrick J. Wong @ 2023-12-31 20:35 UTC (permalink / raw) To: djwong; +Cc: linux-xfs From: Darrick J. Wong <djwong@kernel.org> Create a simple 'blob array' data structure for storage of arbitrarily sized metadata objects that will be used to reconstruct metadata. For the intended usage (temporarily storing extended attribute names and values) we only have to support storing objects and retrieving them. Use the xfile abstraction to store the attribute information in memory that can be swapped out. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- fs/xfs/Makefile | 1 fs/xfs/scrub/xfblob.c | 151 +++++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/xfblob.h | 24 ++++++++ 3 files changed, 176 insertions(+) create mode 100644 fs/xfs/scrub/xfblob.c create mode 100644 fs/xfs/scrub/xfblob.h diff --git a/fs/xfs/Makefile b/fs/xfs/Makefile index 62e38f70c304b..72df9890edcf6 100644 --- a/fs/xfs/Makefile +++ b/fs/xfs/Makefile @@ -208,6 +208,7 @@ xfs-y += $(addprefix scrub/, \ repair.o \ rmap_repair.o \ tempfile.o \ + xfblob.o \ xfbtree.o \ ) diff --git a/fs/xfs/scrub/xfblob.c b/fs/xfs/scrub/xfblob.c new file mode 100644 index 0000000000000..216f9cb2965a7 --- /dev/null +++ b/fs/xfs/scrub/xfblob.c @@ -0,0 +1,151 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (c) 2021-2024 Oracle. All Rights Reserved. + * Author: Darrick J. Wong <djwong@kernel.org> + */ +#include "xfs.h" +#include "xfs_fs.h" +#include "xfs_shared.h" +#include "xfs_format.h" +#include "scrub/scrub.h" +#include "scrub/xfile.h" +#include "scrub/xfarray.h" +#include "scrub/xfblob.h" + +/* + * XFS Blob Storage + * ================ + * Stores and retrieves blobs using an xfile. Objects are appended to the file + * and the offset is returned as a magic cookie for retrieval. + */ + +#define XB_KEY_MAGIC 0xABAADDAD +struct xb_key { + uint32_t xb_magic; /* XB_KEY_MAGIC */ + uint32_t xb_size; /* size of the blob, in bytes */ + loff_t xb_offset; /* byte offset of this key */ + /* blob comes after here */ +} __packed; + +/* Initialize a blob storage object. */ +int +xfblob_create( + const char *description, + struct xfblob **blobp) +{ + struct xfblob *blob; + struct xfile *xfile; + int error; + + error = xfile_create(description, 0, &xfile); + if (error) + return error; + + blob = kmalloc(sizeof(struct xfblob), XCHK_GFP_FLAGS); + if (!blob) { + error = -ENOMEM; + goto out_xfile; + } + + blob->xfile = xfile; + blob->last_offset = PAGE_SIZE; + + *blobp = blob; + return 0; + +out_xfile: + xfile_destroy(xfile); + return error; +} + +/* Destroy a blob storage object. */ +void +xfblob_destroy( + struct xfblob *blob) +{ + xfile_destroy(blob->xfile); + kfree(blob); +} + +/* Retrieve a blob. */ +int +xfblob_load( + struct xfblob *blob, + xfblob_cookie cookie, + void *ptr, + uint32_t size) +{ + struct xb_key key; + int error; + + error = xfile_obj_load(blob->xfile, &key, sizeof(key), cookie); + if (error) + return error; + + if (key.xb_magic != XB_KEY_MAGIC || key.xb_offset != cookie) { + ASSERT(0); + return -ENODATA; + } + if (size < key.xb_size) { + ASSERT(0); + return -EFBIG; + } + + return xfile_obj_load(blob->xfile, ptr, key.xb_size, + cookie + sizeof(key)); +} + +/* Store a blob. */ +int +xfblob_store( + struct xfblob *blob, + xfblob_cookie *cookie, + const void *ptr, + uint32_t size) +{ + struct xb_key key = { + .xb_offset = blob->last_offset, + .xb_magic = XB_KEY_MAGIC, + .xb_size = size, + }; + loff_t pos = blob->last_offset; + int error; + + error = xfile_obj_store(blob->xfile, &key, sizeof(key), pos); + if (error) + return error; + + pos += sizeof(key); + error = xfile_obj_store(blob->xfile, ptr, size, pos); + if (error) + goto out_err; + + *cookie = blob->last_offset; + blob->last_offset += sizeof(key) + size; + return 0; +out_err: + xfile_discard(blob->xfile, blob->last_offset, sizeof(key)); + return error; +} + +/* Free a blob. */ +int +xfblob_free( + struct xfblob *blob, + xfblob_cookie cookie) +{ + struct xb_key key; + int error; + + error = xfile_obj_load(blob->xfile, &key, sizeof(key), cookie); + if (error) + return error; + + if (key.xb_magic != XB_KEY_MAGIC || key.xb_offset != cookie) { + ASSERT(0); + return -ENODATA; + } + + xfile_discard(blob->xfile, cookie, sizeof(key) + key.xb_size); + return 0; +} diff --git a/fs/xfs/scrub/xfblob.h b/fs/xfs/scrub/xfblob.h new file mode 100644 index 0000000000000..bd98647407f1d --- /dev/null +++ b/fs/xfs/scrub/xfblob.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (c) 2021-2024 Oracle. All Rights Reserved. + * Author: Darrick J. Wong <djwong@kernel.org> + */ +#ifndef __XFS_SCRUB_XFBLOB_H__ +#define __XFS_SCRUB_XFBLOB_H__ + +struct xfblob { + struct xfile *xfile; + loff_t last_offset; +}; + +typedef loff_t xfblob_cookie; + +int xfblob_create(const char *descr, struct xfblob **blobp); +void xfblob_destroy(struct xfblob *blob); +int xfblob_load(struct xfblob *blob, xfblob_cookie cookie, void *ptr, + uint32_t size); +int xfblob_store(struct xfblob *blob, xfblob_cookie *cookie, const void *ptr, + uint32_t size); +int xfblob_free(struct xfblob *blob, xfblob_cookie cookie); + +#endif /* __XFS_SCRUB_XFBLOB_H__ */ ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [PATCH 1/6] xfs: create a blob array data structure 2023-12-31 20:35 ` [PATCH 1/6] xfs: create a blob array data structure Darrick J. Wong @ 2024-01-05 5:53 ` Christoph Hellwig 2024-01-06 1:33 ` Darrick J. Wong 0 siblings, 1 reply; 16+ messages in thread From: Christoph Hellwig @ 2024-01-05 5:53 UTC (permalink / raw) To: Darrick J. Wong; +Cc: linux-xfs On Sun, Dec 31, 2023 at 12:35:11PM -0800, Darrick J. Wong wrote: > From: Darrick J. Wong <djwong@kernel.org> > > Create a simple 'blob array' data structure for storage of arbitrarily > sized metadata objects that will be used to reconstruct metadata. For > the intended usage (temporarily storing extended attribute names and > values) we only have to support storing objects and retrieving them. > Use the xfile abstraction to store the attribute information in memory > that can be swapped out. Can't this simply be supported by xfiles directly? Just add a xfile_append that writes at i_size and retuns the offset and we're done? ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH 1/6] xfs: create a blob array data structure 2024-01-05 5:53 ` Christoph Hellwig @ 2024-01-06 1:33 ` Darrick J. Wong 2024-01-06 6:42 ` Christoph Hellwig 0 siblings, 1 reply; 16+ messages in thread From: Darrick J. Wong @ 2024-01-06 1:33 UTC (permalink / raw) To: Christoph Hellwig; +Cc: linux-xfs On Thu, Jan 04, 2024 at 09:53:33PM -0800, Christoph Hellwig wrote: > On Sun, Dec 31, 2023 at 12:35:11PM -0800, Darrick J. Wong wrote: > > From: Darrick J. Wong <djwong@kernel.org> > > > > Create a simple 'blob array' data structure for storage of arbitrarily > > sized metadata objects that will be used to reconstruct metadata. For > > the intended usage (temporarily storing extended attribute names and > > values) we only have to support storing objects and retrieving them. > > Use the xfile abstraction to store the attribute information in memory > > that can be swapped out. > > Can't this simply be supported by xfiles directly? Just add a > xfile_append that writes at i_size and retuns the offset and we're done? Yeah, xfile could just do an "append and tell me where you wrote it". That said, i_size_read is less direct than reading a u64 out of a struct. Another speedbump with doing that is that eventually xfs_repair ports the xfblob to userspace to support parent pointers. For that, a statx call is much more expensive, so I decided that both implementations should just have their own private u64 write pointer. (Unless you want to sponsor a pwrite variant that actually does "append and tell me where"? ;)) --D ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH 1/6] xfs: create a blob array data structure 2024-01-06 1:33 ` Darrick J. Wong @ 2024-01-06 6:42 ` Christoph Hellwig 2024-01-06 18:55 ` Darrick J. Wong 2024-01-08 17:12 ` Darrick J. Wong 0 siblings, 2 replies; 16+ messages in thread From: Christoph Hellwig @ 2024-01-06 6:42 UTC (permalink / raw) To: Darrick J. Wong; +Cc: Christoph Hellwig, linux-xfs On Fri, Jan 05, 2024 at 05:33:16PM -0800, Darrick J. Wong wrote: > (Unless you want to sponsor a pwrite variant that actually does "append > and tell me where"? ;)) Damien and I have adding that on our TODO list (through io_uring) to better support zonefs and programming models like this one on regular files. But I somehow doubt you'd want xfs_repair to depend on it.. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH 1/6] xfs: create a blob array data structure 2024-01-06 6:42 ` Christoph Hellwig @ 2024-01-06 18:55 ` Darrick J. Wong 2024-01-08 17:12 ` Darrick J. Wong 1 sibling, 0 replies; 16+ messages in thread From: Darrick J. Wong @ 2024-01-06 18:55 UTC (permalink / raw) To: Christoph Hellwig; +Cc: linux-xfs On Fri, Jan 05, 2024 at 10:42:01PM -0800, Christoph Hellwig wrote: > On Fri, Jan 05, 2024 at 05:33:16PM -0800, Darrick J. Wong wrote: > > (Unless you want to sponsor a pwrite variant that actually does "append > > and tell me where"? ;)) > > Damien and I have adding that on our TODO list (through io_uring) to > better support zonefs and programming models like this one on regular > files. > > But I somehow doubt you'd want xfs_repair to depend on it.. Well in theory some day Dave might come back with his libaio patches for xfsprogs. After this much time it's a fair question if we'd be better off aiming for io_uring. <shrug> https://lore.kernel.org/linux-xfs/20201015072155.1631135-1-david@fromorbit.com/ --D ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH 1/6] xfs: create a blob array data structure 2024-01-06 6:42 ` Christoph Hellwig 2024-01-06 18:55 ` Darrick J. Wong @ 2024-01-08 17:12 ` Darrick J. Wong 1 sibling, 0 replies; 16+ messages in thread From: Darrick J. Wong @ 2024-01-08 17:12 UTC (permalink / raw) To: Christoph Hellwig; +Cc: linux-xfs On Fri, Jan 05, 2024 at 10:42:01PM -0800, Christoph Hellwig wrote: > On Fri, Jan 05, 2024 at 05:33:16PM -0800, Darrick J. Wong wrote: > > (Unless you want to sponsor a pwrite variant that actually does "append > > and tell me where"? ;)) > > Damien and I have adding that on our TODO list (through io_uring) to > better support zonefs and programming models like this one on regular > files. > > But I somehow doubt you'd want xfs_repair to depend on it.. Nope, not for a few years anyway. :) --D ^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCHSET v29.4 07/13] xfs: online repair of extended attributes @ 2024-02-27 2:18 Darrick J. Wong 2024-02-27 2:28 ` [PATCH 1/6] xfs: create a blob array data structure Darrick J. Wong 0 siblings, 1 reply; 16+ messages in thread From: Darrick J. Wong @ 2024-02-27 2:18 UTC (permalink / raw) To: djwong; +Cc: Dave Chinner, linux-xfs, hch Hi all, This series employs atomic extent swapping to enable safe reconstruction of extended attribute data attached to a file. Because xattrs do not have any redundant information to draw off of, we can at best salvage as much data as we can and build a new structure. Rebuilding an extended attribute structure consists of these three steps: First, we walk the existing attributes to salvage as many of them as we can, by adding them as new attributes attached to the repair tempfile. We need to add a new xfile-based data structure to hold blobs of arbitrary length to stage the xattr names and values. Second, we write the salvaged attributes to a temporary file, and use atomic extent swaps to exchange the entire attribute fork between the two files. Finally, we reap the old xattr blocks (which are now in the temporary file) as carefully as we can. If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below. This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome. --D kernel git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=repair-xattrs xfsprogs git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=repair-xattrs fstests git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=repair-xattrs --- Commits in this patchset: * xfs: create a blob array data structure * xfs: use atomic extent swapping to fix user file fork data * xfs: repair extended attributes * xfs: scrub should set preen if attr leaf has holes * xfs: flag empty xattr leaf blocks for optimization * xfs: create an xattr iteration function for scrub --- fs/xfs/Makefile | 3 fs/xfs/libxfs/xfs_attr.c | 2 fs/xfs/libxfs/xfs_attr.h | 2 fs/xfs/libxfs/xfs_da_format.h | 5 fs/xfs/libxfs/xfs_exchmaps.c | 2 fs/xfs/libxfs/xfs_exchmaps.h | 1 fs/xfs/scrub/attr.c | 158 +++-- fs/xfs/scrub/attr.h | 7 fs/xfs/scrub/attr_repair.c | 1205 +++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/attr_repair.h | 11 fs/xfs/scrub/dab_bitmap.h | 37 + fs/xfs/scrub/dabtree.c | 16 + fs/xfs/scrub/dabtree.h | 3 fs/xfs/scrub/listxattr.c | 310 +++++++++++ fs/xfs/scrub/listxattr.h | 17 + fs/xfs/scrub/repair.c | 46 ++ fs/xfs/scrub/repair.h | 6 fs/xfs/scrub/scrub.c | 2 fs/xfs/scrub/tempexch.h | 2 fs/xfs/scrub/tempfile.c | 204 +++++++ fs/xfs/scrub/tempfile.h | 3 fs/xfs/scrub/trace.h | 85 +++ fs/xfs/scrub/xfarray.c | 17 + fs/xfs/scrub/xfarray.h | 2 fs/xfs/scrub/xfblob.c | 168 ++++++ fs/xfs/scrub/xfblob.h | 26 + fs/xfs/scrub/xfile.c | 12 fs/xfs/scrub/xfile.h | 6 fs/xfs/xfs_buf.c | 3 fs/xfs/xfs_trace.h | 2 30 files changed, 2280 insertions(+), 83 deletions(-) create mode 100644 fs/xfs/scrub/attr_repair.c create mode 100644 fs/xfs/scrub/attr_repair.h create mode 100644 fs/xfs/scrub/dab_bitmap.h create mode 100644 fs/xfs/scrub/listxattr.c create mode 100644 fs/xfs/scrub/listxattr.h create mode 100644 fs/xfs/scrub/xfblob.c create mode 100644 fs/xfs/scrub/xfblob.h ^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH 1/6] xfs: create a blob array data structure 2024-02-27 2:18 [PATCHSET v29.4 07/13] xfs: online repair of extended attributes Darrick J. Wong @ 2024-02-27 2:28 ` Darrick J. Wong 2024-02-28 16:00 ` Christoph Hellwig 0 siblings, 1 reply; 16+ messages in thread From: Darrick J. Wong @ 2024-02-27 2:28 UTC (permalink / raw) To: djwong; +Cc: linux-xfs, hch From: Darrick J. Wong <djwong@kernel.org> Create a simple 'blob array' data structure for storage of arbitrarily sized metadata objects that will be used to reconstruct metadata. For the intended usage (temporarily storing extended attribute names and values) we only have to support storing objects and retrieving them. Use the xfile abstraction to store the attribute information in memory that can be swapped out. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- fs/xfs/Makefile | 1 fs/xfs/scrub/trace.h | 1 fs/xfs/scrub/xfblob.c | 151 +++++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/scrub/xfblob.h | 24 ++++++++ fs/xfs/scrub/xfile.c | 12 ++++ fs/xfs/scrub/xfile.h | 1 6 files changed, 190 insertions(+) create mode 100644 fs/xfs/scrub/xfblob.c create mode 100644 fs/xfs/scrub/xfblob.h diff --git a/fs/xfs/Makefile b/fs/xfs/Makefile index 66415ac8f5717..91af9b7f418e9 100644 --- a/fs/xfs/Makefile +++ b/fs/xfs/Makefile @@ -209,6 +209,7 @@ xfs-y += $(addprefix scrub/, \ repair.o \ rmap_repair.o \ tempfile.o \ + xfblob.o \ ) xfs-$(CONFIG_XFS_RT) += $(addprefix scrub/, \ diff --git a/fs/xfs/scrub/trace.h b/fs/xfs/scrub/trace.h index 5edbabacc31a8..b5fe49a6da534 100644 --- a/fs/xfs/scrub/trace.h +++ b/fs/xfs/scrub/trace.h @@ -948,6 +948,7 @@ DEFINE_XFILE_EVENT(xfile_store); DEFINE_XFILE_EVENT(xfile_seek_data); DEFINE_XFILE_EVENT(xfile_get_folio); DEFINE_XFILE_EVENT(xfile_put_folio); +DEFINE_XFILE_EVENT(xfile_discard); TRACE_EVENT(xfarray_create, TP_PROTO(struct xfarray *xfa, unsigned long long required_capacity), diff --git a/fs/xfs/scrub/xfblob.c b/fs/xfs/scrub/xfblob.c new file mode 100644 index 0000000000000..cec668debce5a --- /dev/null +++ b/fs/xfs/scrub/xfblob.c @@ -0,0 +1,151 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (c) 2021-2024 Oracle. All Rights Reserved. + * Author: Darrick J. Wong <djwong@kernel.org> + */ +#include "xfs.h" +#include "xfs_fs.h" +#include "xfs_shared.h" +#include "xfs_format.h" +#include "scrub/scrub.h" +#include "scrub/xfile.h" +#include "scrub/xfarray.h" +#include "scrub/xfblob.h" + +/* + * XFS Blob Storage + * ================ + * Stores and retrieves blobs using an xfile. Objects are appended to the file + * and the offset is returned as a magic cookie for retrieval. + */ + +#define XB_KEY_MAGIC 0xABAADDAD +struct xb_key { + uint32_t xb_magic; /* XB_KEY_MAGIC */ + uint32_t xb_size; /* size of the blob, in bytes */ + loff_t xb_offset; /* byte offset of this key */ + /* blob comes after here */ +} __packed; + +/* Initialize a blob storage object. */ +int +xfblob_create( + const char *description, + struct xfblob **blobp) +{ + struct xfblob *blob; + struct xfile *xfile; + int error; + + error = xfile_create(description, 0, &xfile); + if (error) + return error; + + blob = kmalloc(sizeof(struct xfblob), XCHK_GFP_FLAGS); + if (!blob) { + error = -ENOMEM; + goto out_xfile; + } + + blob->xfile = xfile; + blob->last_offset = PAGE_SIZE; + + *blobp = blob; + return 0; + +out_xfile: + xfile_destroy(xfile); + return error; +} + +/* Destroy a blob storage object. */ +void +xfblob_destroy( + struct xfblob *blob) +{ + xfile_destroy(blob->xfile); + kfree(blob); +} + +/* Retrieve a blob. */ +int +xfblob_load( + struct xfblob *blob, + xfblob_cookie cookie, + void *ptr, + uint32_t size) +{ + struct xb_key key; + int error; + + error = xfile_load(blob->xfile, &key, sizeof(key), cookie); + if (error) + return error; + + if (key.xb_magic != XB_KEY_MAGIC || key.xb_offset != cookie) { + ASSERT(0); + return -ENODATA; + } + if (size < key.xb_size) { + ASSERT(0); + return -EFBIG; + } + + return xfile_load(blob->xfile, ptr, key.xb_size, + cookie + sizeof(key)); +} + +/* Store a blob. */ +int +xfblob_store( + struct xfblob *blob, + xfblob_cookie *cookie, + const void *ptr, + uint32_t size) +{ + struct xb_key key = { + .xb_offset = blob->last_offset, + .xb_magic = XB_KEY_MAGIC, + .xb_size = size, + }; + loff_t pos = blob->last_offset; + int error; + + error = xfile_store(blob->xfile, &key, sizeof(key), pos); + if (error) + return error; + + pos += sizeof(key); + error = xfile_store(blob->xfile, ptr, size, pos); + if (error) + goto out_err; + + *cookie = blob->last_offset; + blob->last_offset += sizeof(key) + size; + return 0; +out_err: + xfile_discard(blob->xfile, blob->last_offset, sizeof(key)); + return error; +} + +/* Free a blob. */ +int +xfblob_free( + struct xfblob *blob, + xfblob_cookie cookie) +{ + struct xb_key key; + int error; + + error = xfile_load(blob->xfile, &key, sizeof(key), cookie); + if (error) + return error; + + if (key.xb_magic != XB_KEY_MAGIC || key.xb_offset != cookie) { + ASSERT(0); + return -ENODATA; + } + + xfile_discard(blob->xfile, cookie, sizeof(key) + key.xb_size); + return 0; +} diff --git a/fs/xfs/scrub/xfblob.h b/fs/xfs/scrub/xfblob.h new file mode 100644 index 0000000000000..bd98647407f1d --- /dev/null +++ b/fs/xfs/scrub/xfblob.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (c) 2021-2024 Oracle. All Rights Reserved. + * Author: Darrick J. Wong <djwong@kernel.org> + */ +#ifndef __XFS_SCRUB_XFBLOB_H__ +#define __XFS_SCRUB_XFBLOB_H__ + +struct xfblob { + struct xfile *xfile; + loff_t last_offset; +}; + +typedef loff_t xfblob_cookie; + +int xfblob_create(const char *descr, struct xfblob **blobp); +void xfblob_destroy(struct xfblob *blob); +int xfblob_load(struct xfblob *blob, xfblob_cookie cookie, void *ptr, + uint32_t size); +int xfblob_store(struct xfblob *blob, xfblob_cookie *cookie, const void *ptr, + uint32_t size); +int xfblob_free(struct xfblob *blob, xfblob_cookie cookie); + +#endif /* __XFS_SCRUB_XFBLOB_H__ */ diff --git a/fs/xfs/scrub/xfile.c b/fs/xfs/scrub/xfile.c index 8cdd863db5850..459e513ade464 100644 --- a/fs/xfs/scrub/xfile.c +++ b/fs/xfs/scrub/xfile.c @@ -310,3 +310,15 @@ xfile_put_folio( folio_unlock(folio); folio_put(folio); } + +/* Discard pages backing a range of the xfile. */ +void +xfile_discard( + struct xfile *xf, + loff_t pos, + u64 count) +{ + trace_xfile_discard(xf, pos, count); + + shmem_truncate_range(file_inode(xf->file), pos, pos + count - 1); +} diff --git a/fs/xfs/scrub/xfile.h b/fs/xfs/scrub/xfile.h index 76d78dba7e347..8dfbae1fe33a5 100644 --- a/fs/xfs/scrub/xfile.h +++ b/fs/xfs/scrub/xfile.h @@ -17,6 +17,7 @@ int xfile_load(struct xfile *xf, void *buf, size_t count, loff_t pos); int xfile_store(struct xfile *xf, const void *buf, size_t count, loff_t pos); +void xfile_discard(struct xfile *xf, loff_t pos, u64 count); loff_t xfile_seek_data(struct xfile *xf, loff_t pos); #define XFILE_MAX_FOLIO_SIZE (PAGE_SIZE << MAX_PAGECACHE_ORDER) ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [PATCH 1/6] xfs: create a blob array data structure 2024-02-27 2:28 ` [PATCH 1/6] xfs: create a blob array data structure Darrick J. Wong @ 2024-02-28 16:00 ` Christoph Hellwig 2024-02-28 17:48 ` Darrick J. Wong 0 siblings, 1 reply; 16+ messages in thread From: Christoph Hellwig @ 2024-02-28 16:00 UTC (permalink / raw) To: Darrick J. Wong; +Cc: linux-xfs, hch > +/* Discard pages backing a range of the xfile. */ > +void > +xfile_discard( > + struct xfile *xf, > + loff_t pos, > + u64 count) > +{ > + trace_xfile_discard(xf, pos, count); > + > + shmem_truncate_range(file_inode(xf->file), pos, pos + count - 1); > +} Can you split this xfile infrastructure addition into a separate patch instead of hiding it here? Otherwise looks good: Reviewed-by: Christoph Hellwig <hch@lst.de> ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH 1/6] xfs: create a blob array data structure 2024-02-28 16:00 ` Christoph Hellwig @ 2024-02-28 17:48 ` Darrick J. Wong 0 siblings, 0 replies; 16+ messages in thread From: Darrick J. Wong @ 2024-02-28 17:48 UTC (permalink / raw) To: Christoph Hellwig; +Cc: linux-xfs, hch On Wed, Feb 28, 2024 at 08:00:50AM -0800, Christoph Hellwig wrote: > > +/* Discard pages backing a range of the xfile. */ > > +void > > +xfile_discard( > > + struct xfile *xf, > > + loff_t pos, > > + u64 count) > > +{ > > + trace_xfile_discard(xf, pos, count); > > + > > + shmem_truncate_range(file_inode(xf->file), pos, pos + count - 1); > > +} > > Can you split this xfile infrastructure addition into a separate patch > instead of hiding it here? Done. > Otherwise looks good: > > Reviewed-by: Christoph Hellwig <hch@lst.de> Thanks! --D ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2024-02-28 17:48 UTC | newest] Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-01-01 1:03 [PATCH v22 0/6] xfs: online repair of inode data Darrick J. Wong 2020-01-01 1:03 ` [PATCH 1/6] xfs: create a blob array data structure Darrick J. Wong 2020-01-01 1:03 ` [PATCH 2/6] xfs: convert xfs_itruncate_extents_flags to use __xfs_bunmapi Darrick J. Wong 2020-01-01 1:04 ` [PATCH 3/6] xfs: remove unnecessary inode-transaction roll Darrick J. Wong 2020-01-01 1:04 ` [PATCH 4/6] xfs: create a new inode fork block unmap helper Darrick J. Wong 2020-01-01 1:04 ` [PATCH 5/6] xfs: repair extended attributes Darrick J. Wong 2020-01-01 1:04 ` [PATCH 6/6] xfs: scrub should set preen if attr leaf has holes Darrick J. Wong 2023-12-31 19:30 [PATCHSET v29.0 20/28] xfs: online repair of extended attributes Darrick J. Wong 2023-12-31 20:35 ` [PATCH 1/6] xfs: create a blob array data structure Darrick J. Wong 2024-01-05 5:53 ` Christoph Hellwig 2024-01-06 1:33 ` Darrick J. Wong 2024-01-06 6:42 ` Christoph Hellwig 2024-01-06 18:55 ` Darrick J. Wong 2024-01-08 17:12 ` Darrick J. Wong 2024-02-27 2:18 [PATCHSET v29.4 07/13] xfs: online repair of extended attributes Darrick J. Wong 2024-02-27 2:28 ` [PATCH 1/6] xfs: create a blob array data structure Darrick J. Wong 2024-02-28 16:00 ` Christoph Hellwig 2024-02-28 17:48 ` Darrick J. Wong
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).