From: David Howells <dhowells@redhat.com>
To: Dave Chinner <david@fromorbit.com>
Cc: dhowells@redhat.com, Theodore Ts'o <tytso@mit.edu>,
Andreas Dilger <adilger.kernel@dilger.ca>,
"Darrick J. Wong" <djwong@kernel.org>, Chris Mason <clm@fb.com>,
linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org,
linux-btrfs@vger.kernel.org, linux-cachefs@redhat.com,
linux-fsdevel@vger.kernel.org
Subject: Re: How capacious and well-indexed are ext4, xfs and btrfs directories?
Date: Tue, 18 May 2021 08:24:19 +0100 [thread overview]
Message-ID: <272939.1621322659@warthog.procyon.org.uk> (raw)
In-Reply-To: <20210517232237.GE2893@dread.disaster.area>
Dave Chinner <david@fromorbit.com> wrote:
> > What I'd like to do is remove the fanout directories, so that for each logical
> > "volume"[*] I have a single directory with all the files in it. But that
> > means sticking massive amounts of entries into a single directory and hoping
> > it (a) isn't too slow and (b) doesn't hit the capacity limit.
>
> Note that if you use a single directory, you are effectively single
> threading modifications to your file index. You still need to use
> fanout directories if you want concurrency during modification for
> the cachefiles index, but that's a different design criteria
> compared to directory capacity and modification/lookup scalability.
I knew there was something I was overlooking. This might be a more important
criterion. I should try benchmarking this, see what difference it makes
eliminating the extra lookup step (which is probably cheap) versus the
concurrency.
David
next prev parent reply other threads:[~2021-05-18 7:24 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-05-17 15:06 How capacious and well-indexed are ext4, xfs and btrfs directories? David Howells
2021-05-17 23:22 ` Dave Chinner
2021-05-17 23:40 ` Chris Mason
2021-05-19 8:00 ` Avi Kivity
2021-05-19 12:57 ` Dave Chinner
2021-05-19 14:13 ` Avi Kivity
2021-05-18 7:24 ` David Howells [this message]
2021-05-21 5:13 ` Andreas Dilger
2021-05-23 5:51 ` Josh Triplett
2021-05-25 4:21 ` Darrick J. Wong
2021-05-25 5:00 ` Christoph Hellwig
2021-05-25 21:13 ` Andreas Dilger
2021-05-25 21:26 ` Matthew Wilcox
2021-05-25 22:13 ` Darrick J. Wong
2021-05-25 22:48 ` Andreas Dilger
2021-05-26 0:24 ` Chris Mason
2021-06-22 0:50 ` Josh Triplett
2021-05-25 22:31 ` David Howells
2021-05-25 22:58 ` Andreas Dilger
2021-05-26 0:00 ` David Howells
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=272939.1621322659@warthog.procyon.org.uk \
--to=dhowells@redhat.com \
--cc=adilger.kernel@dilger.ca \
--cc=clm@fb.com \
--cc=david@fromorbit.com \
--cc=djwong@kernel.org \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-cachefs@redhat.com \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-xfs@vger.kernel.org \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).