From: Dave Chinner <david@fromorbit.com>
To: Christoph Hellwig <hch@infradead.org>
Cc: "Darrick J. Wong" <djwong@kernel.org>, linux-xfs@vger.kernel.org
Subject: Re: [PATCH 2/8] xfs: separate CIL commit record IO
Date: Wed, 3 Mar 2021 11:11:57 +1100 [thread overview]
Message-ID: <20210303001157.GK4662@dread.disaster.area> (raw)
In-Reply-To: <20210301090901.GA3698088@infradead.org>
On Mon, Mar 01, 2021 at 09:09:01AM +0000, Christoph Hellwig wrote:
> On Fri, Feb 26, 2021 at 07:47:55AM +1100, Dave Chinner wrote:
> > Sorry, I/O wait might matter for what?
>
> Think you have a SAS hard drive, WCE=0, typical queue depth of a few
> dozend commands.
Yup, so typical IO latency of 2-3ms, assuming 10krpm, assuming we
aren't doing metadata writeback which will blow this out.
I've tested on slower iSCSI devices than this (7-8ms typical av.
seek time), and it didn't show up any performance anomalies.
> Before that we'd submit a bunch of iclogs, which are generally
> sequential except of course for the log wrap around case. The drive
> can now easily take all the iclogs and write them in one rotation.
Even if we take the best case for your example, this still means we
block on every 8 iclogs waiting 2-3ms for the spindle to rotate and
complete the IOs. Hence for a checkpoint of 32MB with 256kB iclogs,
we're blocking for 2-3ms at least 16 times before we get to the
commit iclog. With default iclog size of 32kB, we'll block a couple
of hundred times waiting on iclog IO...
IOWs, we're already talking about a best case checkpoint commit
latency of 30-50ms here.
[ And this isn't even considering media bandwidth there - 32MB on a
drive that can do maybe 200MB/s in the middle of the spindle where
the log is. That's another 150ms of data transfer time to physical
media. So if the drive is actually writing to physical media because
WCE=0, then we're taking *at least* 200ms per 32MB checkpoint. ]
> Now if we wait for the previous iclogs before submitting the
> commit_iclog we need at least one more additional full roundtrip.
So we add an average of 2-3ms to what is already taking, in the best
case, 30-50ms.
And these are mostly async commits this overhead is added to, so
there's rarely anything waiting on it and hence the extra small
latency is almost always lost in the noise. Even if the extra delay
is larger, there is rarely anything waiting on it so it's still
noise...
I just don't see anything relevant that stands out from the noise on
my systems.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
next prev parent reply other threads:[~2021-03-03 3:19 UTC|newest]
Thread overview: 59+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-23 3:34 [PATCH v2] xfs: various log stuff Dave Chinner
2021-02-23 3:34 ` [PATCH 1/8] xfs: log stripe roundoff is a property of the log Dave Chinner
2021-02-23 10:29 ` Chandan Babu R
2021-02-24 20:14 ` Darrick J. Wong
2021-02-25 8:32 ` Christoph Hellwig
2021-03-01 15:13 ` Brian Foster
2021-02-23 3:34 ` [PATCH 2/8] xfs: separate CIL commit record IO Dave Chinner
2021-02-23 12:12 ` Chandan Babu R
2021-02-24 20:34 ` Darrick J. Wong
2021-02-24 21:44 ` Dave Chinner
2021-02-24 23:06 ` Darrick J. Wong
2021-02-25 8:34 ` Christoph Hellwig
2021-02-25 20:47 ` Dave Chinner
2021-03-01 9:09 ` Christoph Hellwig
2021-03-03 0:11 ` Dave Chinner [this message]
2021-02-26 2:48 ` Darrick J. Wong
2021-02-28 16:36 ` Brian Foster
2021-02-28 23:46 ` Dave Chinner
2021-03-01 15:33 ` Brian Foster
2021-03-01 15:19 ` Brian Foster
2021-03-03 0:41 ` Dave Chinner
2021-03-03 15:22 ` Brian Foster
2021-03-04 22:57 ` Dave Chinner
2021-03-05 0:44 ` Dave Chinner
2021-02-23 3:34 ` [PATCH 3/8] xfs: move and rename xfs_blkdev_issue_flush Dave Chinner
2021-02-23 12:57 ` Chandan Babu R
2021-02-24 20:45 ` Darrick J. Wong
2021-02-24 22:01 ` Dave Chinner
2021-02-25 8:36 ` Christoph Hellwig
2021-02-23 3:34 ` [PATCH 4/8] xfs: async blkdev cache flush Dave Chinner
2021-02-23 5:29 ` Chaitanya Kulkarni
2021-02-23 14:02 ` Chandan Babu R
2021-02-24 20:51 ` Darrick J. Wong
2021-02-23 3:34 ` [PATCH 5/8] xfs: CIL checkpoint flushes caches unconditionally Dave Chinner
2021-02-24 7:16 ` Chandan Babu R
2021-02-24 20:57 ` Darrick J. Wong
2021-02-25 8:42 ` Christoph Hellwig
2021-02-25 21:07 ` Dave Chinner
2021-02-23 3:34 ` [PATCH 6/8] xfs: remove need_start_rec parameter from xlog_write() Dave Chinner
2021-02-24 7:17 ` Chandan Babu R
2021-02-24 20:59 ` Darrick J. Wong
2021-02-25 8:49 ` Christoph Hellwig
2021-02-25 20:55 ` Dave Chinner
2021-02-23 3:34 ` [PATCH 7/8] xfs: journal IO cache flush reductions Dave Chinner
2021-02-23 8:05 ` [PATCH 7/8 v2] " Dave Chinner
2021-02-24 12:27 ` Chandan Babu R
2021-02-24 20:32 ` Dave Chinner
2021-02-24 21:13 ` Darrick J. Wong
2021-02-24 22:03 ` Dave Chinner
2021-02-25 4:09 ` Chandan Babu R
2021-02-25 7:13 ` Chandan Babu R
2021-03-01 5:44 ` Dave Chinner
2021-03-01 5:56 ` Dave Chinner
2021-02-25 8:58 ` Christoph Hellwig
2021-02-25 21:06 ` Dave Chinner
2021-03-01 19:29 ` Brian Foster
2021-02-23 3:34 ` [PATCH 8/8] xfs: Fix CIL throttle hang when CIL space used going backwards Dave Chinner
2021-02-24 21:18 ` Darrick J. Wong
2021-02-24 22:05 ` Dave Chinner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210303001157.GK4662@dread.disaster.area \
--to=david@fromorbit.com \
--cc=djwong@kernel.org \
--cc=hch@infradead.org \
--cc=linux-xfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.