From: Dave Chinner <david@fromorbit.com>
To: Amir Goldstein <amir73il@gmail.com>
Cc: Matthew Wilcox <willy@infradead.org>,
"Darrick J. Wong" <darrick.wong@oracle.com>,
linux-xfs <linux-xfs@vger.kernel.org>,
Christoph Hellwig <hch@lst.de>,
linux-fsdevel <linux-fsdevel@vger.kernel.org>
Subject: Re: [QUESTION] Long read latencies on mixed rw buffered IO
Date: Tue, 26 Mar 2019 10:48:38 +1100 [thread overview]
Message-ID: <20190325234838.GC23020@dastard> (raw)
In-Reply-To: <CAOQ4uxiTRUe2EUYcuN5xi3SCw6C-=DM+yA1rsRKh_fi0YPEf6Q@mail.gmail.com>
On Mon, Mar 25, 2019 at 09:57:46PM +0200, Amir Goldstein wrote:
> On Mon, Mar 25, 2019 at 9:40 PM Matthew Wilcox <willy@infradead.org> wrote:
> >
> > On Mon, Mar 25, 2019 at 09:18:51PM +0200, Amir Goldstein wrote:
> > > On Mon, Mar 25, 2019 at 8:22 PM Matthew Wilcox <willy@infradead.org> wrote:
> > > > On Mon, Mar 25, 2019 at 07:30:39PM +0200, Amir Goldstein wrote:
> > > > > On Mon, Mar 25, 2019 at 6:41 PM Matthew Wilcox <willy@infradead.org> wrote:
> > > > > > I think it is a bug that we only wake readers at the front of the queue;
> > > > > > I think we would get better performance if we wake all readers. ie here:
> > >
> > > So I have no access to the test machine of former tests right now,
> > > but when running the same filebench randomrw workload
> > > (8 writers, 8 readers) on VM with 2 CPUs and SSD drive, results
> > > are not looking good for this patch:
> > >
> > > --- v5.1-rc1 / xfs ---
> > > rand-write1 852404ops 14202ops/s 110.9mb/s 0.6ms/op
> > > [0.01ms - 553.45ms]
> > > rand-read1 26117ops 435ops/s 3.4mb/s 18.4ms/op
> > > [0.04ms - 632.29ms]
> > > 61.088: IO Summary: 878521 ops 14636.774 ops/s 435/14202 rd/wr
> > > 114.3mb/s 1.1ms/op
> > >
>
> --- v5.1-rc1 / xfs + patch v2 below ---
> rand-write1 852487ops 14175ops/s 110.7mb/s 0.6ms/op
> [0.01ms - 755.24ms]
> rand-read1 23194ops 386ops/s 3.0mb/s 20.7ms/op
> [0.03ms - 755.25ms]
> 61.187: IO Summary: 875681 ops 14560.980 ops/s 386/14175 rd/wr
> 113.8mb/s 1.1ms/op
>
> Not as bad as v1. Only a little bit worse than master...
> The whole deal with the read/write balance and on SSD, I imagine
> the balance really changes. That's why I was skeptical about
> one-size-fits all read/write balance.
You're not testing your SSD. You're testing writes into cache vs
reads from disk. There is a massive latency difference in the two
operations, so unless you use O_DSYNC for the writes, you are going
to see this cache-vs-uncached performance unbalance. i.e. unless the
rwsem is truly fair, there is always going to be more writer
access to the lock because they spend less time holding it and so
can put much more pressure on it.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
next prev parent reply other threads:[~2019-03-25 23:48 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CAOQ4uxi0pGczXBX7GRAFs88Uw0n1ERJZno3JSeZR71S1dXg+2w@mail.gmail.com>
[not found] ` <20190325001044.GA23020@dastard>
2019-03-25 7:49 ` [QUESTION] Long read latencies on mixed rw buffered IO Amir Goldstein
2019-03-25 15:47 ` Darrick J. Wong
2019-03-25 16:41 ` Matthew Wilcox
2019-03-25 17:30 ` Amir Goldstein
2019-03-25 18:22 ` Matthew Wilcox
2019-03-25 19:18 ` Amir Goldstein
2019-03-25 19:40 ` Matthew Wilcox
2019-03-25 19:57 ` Amir Goldstein
2019-03-25 23:48 ` Dave Chinner [this message]
2019-03-26 3:44 ` Amir Goldstein
2019-03-27 1:29 ` Dave Chinner
2019-03-25 17:56 ` Amir Goldstein
2019-03-25 18:02 ` Christoph Hellwig
2019-03-25 18:44 ` Amir Goldstein
2019-03-25 23:43 ` Dave Chinner
2019-03-26 4:36 ` Amir Goldstein
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190325234838.GC23020@dastard \
--to=david@fromorbit.com \
--cc=amir73il@gmail.com \
--cc=darrick.wong@oracle.com \
--cc=hch@lst.de \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-xfs@vger.kernel.org \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).