From: Dave Chinner <david@fromorbit.com>
To: linux-xfs@vger.kernel.org
Subject: [PATCH 0/8 v5] xfs: intent whiteouts
Date: Wed, 27 Apr 2022 12:22:51 +1000 [thread overview]
Message-ID: <20220427022259.695399-1-david@fromorbit.com> (raw)
Hi folks,
This is a patchset inspired by the performance regressions that were
seen from logging 64k xattrs with Allison's delayed attribute
patchset and trying to work out how to minimise the impact of
logging xattrs. Most of that is explained in the "xfs: intent item
whiteouts" patch, so I won't repeat it here.
The whiteouts massively reduce the journal write overhead of logging
xattrs - with this patchset I've reduced 2.5GB/s of log traffic (16
way file create w/64k xattr workload) down to approximately 220MB of
log traffic, and performance has increased from 9k creates/s to 36k
creates/s. The workload still writes to disk at 2.5GB/s, but that's
what writing 35k x 64k xattrs to disk does.
This is still short of the non-logged attribute mechanism, which
runs at 40-45k creates a second and 3.5-4GB/s to disk, but it brings
logged attrs to within roughly 5-15% of non-logged attrs across the
full range of attribute sizes.
So, while this patchset was clearly insired and has major positive
impact on Allison's delayed attribute work, it also applies
generically to all other intent/intent done pairs that already
exist. Hence I've created this patchset as a stand-alone patchset
that isn't dependent on the delayed attributes being committed, nor
does the delayed attribute patchset need this to funciton properly.
IOWs, they can be merged in parallel and then the attribute log item
implementation be updated to support whiteouts after the fact.
This patchset is separate to the attr code, though, because
intent whiteouts are not specific to the attr code. They are a
generic mechanism that can be applied to all the intent/intent done
item pairs we already have. This patch set modifies all those
intents to use whiteouts, and so there is benefits from the patch
set for all operations that use these intents.
With respect to the delayed attribute patchset, it can be merged
with/without whiteout support and still work correctly with/without
this patchset in place. Once both intent whiteouts and delayed attrs
are merged, we can add whiteout support to delayed attributes with
only a few lines of extra code.
-Dave.
Changelog:
Version 5:
- rebased on 5.18-rc2 + linux-xfs/for-next
- converted transaction flags to unsigned to match tp->t_flags definition
Version 4:
- not published
- rebased on 5.17 + for-next + log shutdown fixes + xlog-write-rework
- fixed memory leak of CUI shadow buffers from log recovery when clearing
leftover reflink entries.
Version 3:
- https://lore.kernel.org/linux-xfs/20220314220631.3093283-1-david@fromorbit.com/
- rebased on 5.17-rc4 + xlog-write-rework
- no longer dependent on xfs-cil-scalability, so there's some porting changes
that was needed to remove all the per-cpu CIL dependencies.
Version 2:
- not published
- rebased on 5.15-rc2 + xfs-cil-scalability
- dropped the kvmalloc changes for CIL shadow buffers as that's a
separate perf problem and not something related to intent
whiteouts.
- dropped all the delayed attribute modifications so that the
patchset is not dependent on Allison's dev tree.
- Thanks to Allison for an initial quick review pass - I haven't
included those RVB tags because every patch in the series has
changed since the original RFC posting.
RFC:
- https://lore.kernel.org/linux-xfs/20210909212133.GE2361455@dread.disaster.area/
next reply other threads:[~2022-04-27 2:23 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-27 2:22 Dave Chinner [this message]
2022-04-27 2:22 ` [PATCH 1/8] xfs: hide log iovec alignment constraints Dave Chinner
2022-04-27 3:14 ` Darrick J. Wong
2022-04-27 4:50 ` Dave Chinner
2022-04-27 16:45 ` Darrick J. Wong
2022-04-28 13:00 ` Christoph Hellwig
2022-04-27 2:22 ` [PATCH 2/8] xfs: don't commit the first deferred transaction without intents Dave Chinner
2022-04-27 3:03 ` Darrick J. Wong
2022-04-27 4:52 ` Dave Chinner
2022-04-28 13:02 ` Christoph Hellwig
2022-04-30 17:02 ` Alli
2022-04-27 2:22 ` [PATCH 3/8] xfs: add log item flags to indicate intents Dave Chinner
2022-04-27 3:04 ` Darrick J. Wong
2022-04-28 13:04 ` Christoph Hellwig
2022-04-27 2:22 ` [PATCH 4/8] xfs: tag transactions that contain intent done items Dave Chinner
2022-04-27 3:06 ` Darrick J. Wong
2022-04-28 13:05 ` Christoph Hellwig
2022-04-27 2:22 ` [PATCH 5/8] xfs: factor and move some code in xfs_log_cil.c Dave Chinner
2022-04-27 3:15 ` Darrick J. Wong
2022-04-27 4:56 ` Dave Chinner
2022-04-28 13:06 ` Christoph Hellwig
2022-04-29 1:56 ` Alli
2022-04-27 2:22 ` [PATCH 6/8] xfs: add log item method to return related intents Dave Chinner
2022-04-27 3:18 ` Darrick J. Wong
2022-04-28 13:10 ` Christoph Hellwig
2022-04-27 2:22 ` [PATCH 7/8] xfs: whiteouts release intents that are not in the AIL Dave Chinner
2022-04-27 3:19 ` Darrick J. Wong
2022-04-28 13:15 ` Christoph Hellwig
2022-04-27 2:22 ` [PATCH 8/8] xfs: intent item whiteouts Dave Chinner
2022-04-27 3:32 ` Darrick J. Wong
2022-04-27 5:47 ` Dave Chinner
2022-04-27 17:31 ` Darrick J. Wong
2022-04-27 22:05 ` Dave Chinner
2022-04-28 13:22 ` Christoph Hellwig
2022-04-28 21:38 ` Dave Chinner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220427022259.695399-1-david@fromorbit.com \
--to=david@fromorbit.com \
--cc=linux-xfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.