From: Dave Chinner <david@fromorbit.com>
To: Chris Mason <clm@fb.com>,
Linus Torvalds <torvalds@linux-foundation.org>,
Josef Bacik <jbacik@fb.com>, LKML <linux-kernel@vger.kernel.org>,
linux-fsdevel <linux-fsdevel@vger.kernel.org>,
Neil Brown <neilb@suse.de>, Jan Kara <jack@suse.cz>,
Christoph Hellwig <hch@lst.de>
Subject: Re: [PATCH] fs-writeback: drop wb->list_lock during blk_finish_plug()
Date: Mon, 14 Sep 2015 09:12:58 +1000 [thread overview]
Message-ID: <20150913231258.GS26895@dastard> (raw)
In-Reply-To: <20150912230027.GE4150@ret.masoncoding.com>
On Sat, Sep 12, 2015 at 07:00:27PM -0400, Chris Mason wrote:
> On Fri, Sep 11, 2015 at 04:36:39PM -0700, Linus Torvalds wrote:
> > On Fri, Sep 11, 2015 at 4:16 PM, Chris Mason <clm@fb.com> wrote:
> > >
> > > For 4.3 timeframes, what runs do you want to see numbers for:
> > >
> > > 1) revert
> > > 2) my hack
> > > 3) plug over multiple sbs (on different devices)
> > > 4) ?
> >
> > Just 2 or 3.
> >
> > I don't think the plain revert is all that interesting, and I think
> > the "anything else" is far too late for this merge window.
>
> I did the plain revert as well, just to have a baseline. This box is a
> little different from Dave's. Bare metal two socket box (E5-2660 v2 @
> 2.20Ghz) with 144GB of ram. I have two pcie flash devices, one nvme and
> one fusionio, and I put a one FS on each device (two mounts total).
>
> The test created 1.6M files, 4K each. I used Dave's fs_mark command
> line, spread out over 16 directories from each mounted filesystem. In
> btrfs they are spread over subvolumes to cut down lock contention.
>
> I need to change around the dirty ratios more to smooth out the IO, and
> I had trouble with both XFS and btrfs getting runs that were not CPU
> bound. I included the time to run sync at the end of the run because
> the results were not very consistent without it.
>
> The XFS runs generally had one CPU pegged at 100%, and I think this is
> throwing off the results. On Monday I'll redo them with two (four?)
> filesystems per flash device, hopefully that'll break things up.
>
> The btrfs runs generally had all the CPUs pegged at 100%. I switched to
> mount -o nodatasum and squeezed out an extra 50K files/sec at much lower
> CPU utilization.
>
> wall time fs_mark files/sec bytes written/sec
>
> XFS:
> baseline v4.2: 5m6s 118,578 272MB/s
> Dave's patch: 4m46s 151,421 294MB/s
> my hack: 5m5s 150,714 275MB/s
> Linus plug: 5m15s 147,735 266MB/s
>
>
> Btrfs (nodatasum):
> baseline v4.2: 4m39s 242,643 313MB/s
> Dave's patch: 3m46s 252,452 389MB/s
> my hack: 3m48s 257,924 379MB/s
> Linus plug: 3m58s 247,528 369MB/s
Really need to run these numbers on slower disks where block layer
merging makes a difference to performance. The high level plugging
improves performance on spinning disks by a huge amount on XFS
because the merging reduces the number of IOs issued to disk by 2
orders of magnitude. Plugging makes comparitively little difference
on devices that can sustain extremely high IOPS and hence sink the
tens to hundreds of thousand individual 4k IOs that this workload
generates through writeback.
i.e. while throughput increases, that's not the numbers that matters
here - it's the change in write IO behaviour that needs to be
categorised and measured by the different patches...
(I'm on holidays, so I'm not going to get to this any time soon)
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
next prev parent reply other threads:[~2015-09-13 23:13 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-09-11 19:37 [PATCH] fs-writeback: drop wb->list_lock during blk_finish_plug() Chris Mason
2015-09-11 20:02 ` Linus Torvalds
2015-09-11 20:37 ` Linus Torvalds
2015-09-11 20:40 ` Josef Bacik
2015-09-11 21:04 ` Linus Torvalds
2015-09-11 22:06 ` Linus Torvalds
2015-09-11 23:16 ` Chris Mason
2015-09-11 23:36 ` Linus Torvalds
2015-09-12 0:52 ` Linus Torvalds
2015-09-12 2:15 ` Chris Mason
2015-09-12 2:27 ` Linus Torvalds
2015-09-12 23:00 ` Chris Mason
2015-09-12 23:29 ` Linus Torvalds
2015-09-12 23:46 ` Chris Mason
2015-09-13 13:12 ` Chris Mason
2015-09-13 22:56 ` Dave Chinner
2015-09-13 23:12 ` Dave Chinner [this message]
2015-09-14 20:06 ` Linus Torvalds
2015-09-16 15:16 ` Chris Mason
2015-09-16 19:58 ` Jan Kara
2015-09-16 20:00 ` Chris Mason
2015-09-16 22:07 ` Dave Chinner
2015-09-17 0:37 ` Dave Chinner
2015-09-17 1:12 ` Linus Torvalds
2015-09-17 2:14 ` Dave Chinner
2015-09-17 19:39 ` Linus Torvalds
2015-09-17 22:42 ` Chris Mason
2015-09-17 23:08 ` Linus Torvalds
2015-09-17 23:56 ` Chris Mason
2015-09-18 0:37 ` Dave Chinner
2015-09-18 1:50 ` Linus Torvalds
2015-09-18 5:40 ` Dave Chinner
2015-09-18 6:04 ` Linus Torvalds
2015-09-18 6:06 ` Linus Torvalds
2015-09-18 14:21 ` Jens Axboe
2015-09-18 13:16 ` Chris Mason
2015-09-18 14:23 ` Jens Axboe
2015-09-18 15:32 ` Linus Torvalds
2015-09-18 15:59 ` Peter Zijlstra
2015-09-18 16:02 ` Peter Zijlstra
2015-09-18 16:12 ` Linus Torvalds
2015-09-28 14:47 ` Peter Zijlstra
2015-09-28 16:08 ` Linus Torvalds
2015-09-29 7:55 ` Ingo Molnar
2015-09-18 22:17 ` Dave Chinner
2015-09-21 9:24 ` Jan Kara
2015-09-21 9:24 ` Jan Kara
2015-09-21 20:21 ` Andrew Morton
2015-09-21 20:21 ` Andrew Morton
2015-09-17 23:03 ` Dave Chinner
2015-09-17 23:13 ` Linus Torvalds
2015-09-17 3:48 ` Chris Mason
2015-09-17 4:30 ` Dave Chinner
2015-09-17 12:13 ` Chris Mason
2015-09-11 23:06 ` Chris Mason
2015-09-11 23:13 ` Linus Torvalds
-- strict thread matches above, loose matches on Subject: below --
2015-09-09 15:23 Chris Mason
2015-09-11 18:49 ` Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150913231258.GS26895@dastard \
--to=david@fromorbit.com \
--cc=clm@fb.com \
--cc=hch@lst.de \
--cc=jack@suse.cz \
--cc=jbacik@fb.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=neilb@suse.de \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.