All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Darrick J. Wong" <darrick.wong@oracle.com>
To: darrick.wong@oracle.com
Cc: linux-xfs@vger.kernel.org
Subject: [PATCH v18 00/17] xfs: online repair support
Date: Mon, 31 Dec 2018 18:09:58 -0800	[thread overview]
Message-ID: <154630859850.15625.17640302184521917854.stgit@magnolia> (raw)

Hi all,

This is the first part of the eighteenth revision of a patchset that
adds to XFS kernel support for online metadata scrubbing and repair.
There aren't any on-disk format changes.

New for v18 is a rebase against 4.20 and the creation of explicit data
structures for storing intermediate reconstructed records.  Otherwise,
there hasn't been much activity here, as we've been too busy fixing
clone/copy/deduperange ioctls.

Patch 1 moves all the inode geometry information into a separate
structure so that we can later add more fields to make the online fsck
code less messy w.r.t. inode geometry.

Patch 2 creates a new data structure that provides an abstraction of a
big memory array by using linked lists.  This is where we store records
for btree reconstruction.  This first implementation is memory
inefficient and consumes a /lot/ of kernel memory, but lays the
groundwork for the last patch in the set to convert the implementation
to use a (memfd) swap file, which enables us to use pageable memory
without pounding the slab cache.

Patches 3-9 implement reconstruction of the free space btrees, the inode
btrees, the inodes, the inode forks, the inode block maps, and symbolic
links.

Patch 10 implements a new data structure for storing arbitrary key/value
pairs, which we're going to need to reconstruct extended attribute
forks.

Patches 11-13 clean up the block unmapping code so that we will be able
to perform a mass reset of an inode's fork.  This is a key component for
salvaging extended attributes, freeing all the attr fork blocks, and
reconstructing the extended attribute data.

Patch 14 implements extended attribute salvage.

Patch 15 augments scrub to rebuild extended attributes when any of the
attr blocks are fragmented.

Patch 16 implements reconstruction of quota blocks.

Patch 17 converts both in-memory array implementations from the clunky
linked list implementation to something resembling C arrays.  The array
data are backed by a (memfd) file, which means that idle data can be
paged out to disk instead of pinning kernel memory.

If you're going to start using this mess, you probably ought to just
pull from my git trees.  The kernel patches[1] should apply against
4.20.  xfsprogs[2] and xfstests[3] can be found in their usual
places.  The git trees contain all four series' worth of changes.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

[1] https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=djwong-devel
[2] https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=djwong-devel
[3] https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=djwong-devel

             reply	other threads:[~2019-01-01  2:10 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-01-01  2:09 Darrick J. Wong [this message]
2019-01-01  2:10 ` [PATCH 01/17] xfs: separate inode geometry Darrick J. Wong
2019-01-01  2:10 ` [PATCH 02/17] xfs: create a big array data structure Darrick J. Wong
2019-01-01  2:10 ` [PATCH 03/17] xfs: repair free space btrees Darrick J. Wong
2019-01-01  2:10 ` [PATCH 04/17] xfs: repair inode btrees Darrick J. Wong
2019-01-01  2:10 ` [PATCH 05/17] xfs: repair refcount btrees Darrick J. Wong
2019-01-01  2:10 ` [PATCH 06/17] xfs: repair inode records Darrick J. Wong
2019-01-01  2:10 ` [PATCH 07/17] xfs: zap broken inode forks Darrick J. Wong
2019-01-01  2:10 ` [PATCH 08/17] xfs: repair inode block maps Darrick J. Wong
2019-01-01  2:10 ` [PATCH 09/17] xfs: repair damaged symlinks Darrick J. Wong
2019-01-01  2:11 ` [PATCH 10/17] xfs: create a blob array data structure Darrick J. Wong
2019-01-01  2:11 ` [PATCH 11/17] xfs: convert xfs_itruncate_extents_flags to use __xfs_bunmapi Darrick J. Wong
2019-01-01  2:11 ` [PATCH 12/17] xfs: remove unnecessary inode-transaction roll Darrick J. Wong
2019-01-01  2:11 ` [PATCH 13/17] xfs: create a new inode fork block unmap helper Darrick J. Wong
2019-01-01  2:11 ` [PATCH 14/17] xfs: repair extended attributes Darrick J. Wong
2019-01-01  2:11 ` [PATCH 15/17] xfs: scrub should set preen if attr leaf has holes Darrick J. Wong
2019-01-01  2:11 ` [PATCH 16/17] xfs: repair quotas Darrick J. Wong
2019-01-01  2:11 ` [PATCH 17/17] xfs: convert big array and blob array to use memfd backend Darrick J. Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=154630859850.15625.17640302184521917854.stgit@magnolia \
    --to=darrick.wong@oracle.com \
    --cc=linux-xfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.