linux-ext4.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jan Kara <jack@suse.cz>
To: linux-ext4@vger.kernel.org
Cc: Jan Kara <jack@suse.cz>
Subject: [PATCH 0/3 v2] ext4: Speedup orphan file handling
Date: Fri, 22 May 2015 13:21:53 +0200	[thread overview]
Message-ID: <1432293717-24010-1-git-send-email-jack@suse.cz> (raw)

  Hello,

This is the second version of my patches to speed up orphan inode handling
in ext4.

Orphan inode handling in ext4 is a bottleneck for workloads which heavily
excercise truncate / unlink of small files as they contend on global
s_orphan_mutex (when you have fast enough storage). This patch set implements
new way of handling orphan inodes - instead of using a linked list, we store
inode numbers of orphaned inodes in a file which is possible to implement in a
more scalable manner than linked list manipulations. See description of patch
2/3 for more details.

The patch set achieves significant gains both for a micro benchmark stressing
orphan inode handling (truncating file byte-by-byte, several threads in
parallel) and for reaim new_fserver workload. As a highlight, microbenchmark
runtime for 128 threads is reduced from original 160 s down to 71 s, which
is also the time it takes the benchmark to run when orphan inode handling
is completely disabled. For full numbers you can check commit logs of
patches 2/3 and 3/3. You can also check my presentation from Vault at
http://events.linuxfoundation.org/sites/events/files/slides/ext4-scaling.pdf
for graphs from tests.

I'm happy for any review, thoughts, ideas about the patches.

The kernel part of the feature is complete, I have also implemented full
support in e2fsprogs. That still needs some debugging (especially the e2fsck
part) but support in mke2fs or tune2fs is fine. I'll post these as a separate
patch so that people can try this out.

For now I'm using inode 9 for orphan file. I know that is reserved as
EXT2_EXCLUDE_INO but at least for the sake of testing that should be fine.

								Honza

Changes since v1:
* orphan blocks have now magic numbers
* split out orphan handling to a separate source file
* some smaller updates according to review

             reply	other threads:[~2015-05-22 11:22 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-05-22 11:21 Jan Kara [this message]
2015-05-22 11:21 ` [PATCH 1/4] ext4: Support for checksumming from journal triggers Jan Kara
2015-05-22 11:21 ` [PATCH 2/4] ext4: Move orphan inode handling into a separate file Jan Kara
2015-05-22 11:21 ` [PATCH 3/4] ext4: Speedup ext4 orphan inode handling Jan Kara
2015-05-22 11:21 ` [PATCH 4/4] ext4: Improve scalability of ext4 orphan file handling Jan Kara

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1432293717-24010-1-git-send-email-jack@suse.cz \
    --to=jack@suse.cz \
    --cc=linux-ext4@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).