From: Daniel Forrest <dforrest@wisc.edu>
To: Trond Myklebust <trondmy@hammerspace.com>
Cc: "alkisg@gmail.com" <alkisg@gmail.com>,
"linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>
Subject: Re: rsize,wsize=1M causes severe lags in 10/100 Mbps
Date: Thu, 19 Sep 2019 16:19:12 -0500 [thread overview]
Message-ID: <20190919211912.GA21865@cosmos.ssec.wisc.edu> (raw)
In-Reply-To: <d7ea48b4cd665eced45783bf94d6b1ff1f211960.camel@hammerspace.com>
On Thu, Sep 19, 2019 at 08:40:41PM +0000, Trond Myklebust wrote:
> On Thu, 2019-09-19 at 23:20 +0300, Alkis Georgopoulos wrote:
> > On 9/19/19 11:05 PM, Trond Myklebust wrote:
> > > There are plenty of operations that can take longer than 700 ms to
> > > complete. Synchronous writes to disk are one, but COMMIT (i.e. the
> > > NFS
> > > equivalent of fsync()) can often take much longer even though it
> > > has no
> > > payload.
> > >
> > > So the problem is not the size of the WRITE payload. The real
> > > problem
> > > is the timeout.
> > >
> > > The bottom line is that if you want to keep timeo=7 as a mount
> > > option
> > > for TCP, then you are on your own.
> > >
> >
> > The problem isn't timeo at all.
> > If I understand it correctly, when I try to launch firefox over
> > nfsroot,
> > NFS will wait until it fills 1M before "replying" to the application.
> > Thus the applications will launch a lot slower, as they get "disk
> > feedback" in larger chunks and not "snappy".
> >
> > In numbers:
> > timeo=600,rsize=1M => firefox opens in 30 secs
> > timeo=600,rsize=32k => firefox opens in 20 secs
> >
>
> That's a different problem, and is most likely due to readahead causing
> your client to read more data than it needs to. It is also true that
> the maximum readahead size is proportional to the rsize and that maybe
> it shouldn't be.
> However the VM layer is supposed to ensure that the kernel doesn't try
> to read ahead more than necessary. It is bounded by the maximum we set
> in the NFS layer, but it isn't supposed to hit that maximum unless the
> readahead heuristics show that the application may need it.
>
> --
> Trond Myklebust
> Linux NFS client maintainer, Hammerspace
> trond.myklebust@hammerspace.com
What may be happening here is something I have noticed with glibc.
- statfs reports the rsize/wsize as the block size of the filesystem.
- glibc uses the block size as the default buffer size for fread/fwrite.
If an application is using fread/fwrite on an NFS mounted file with an
rsize/wsize of 1M it will try to fill a 1MB buffer.
I have often changed mounts to use rsize/wsize=64K to alleviate this.
--
Dan Forrest
Space Science and Engineering Center, University of Wisconsin, Madison
dforrest@wisc.edu
next prev parent reply other threads:[~2019-09-19 21:25 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-19 7:29 rsize,wsize=1M causes severe lags in 10/100 Mbps, what sets those defaults? Alkis Georgopoulos
2019-09-19 15:08 ` Trond Myklebust
2019-09-19 15:58 ` rsize,wsize=1M causes severe lags in 10/100 Mbps Alkis Georgopoulos
2019-09-19 16:11 ` Trond Myklebust
2019-09-19 19:21 ` Alkis Georgopoulos
2019-09-19 19:51 ` Trond Myklebust
2019-09-19 19:57 ` Alkis Georgopoulos
2019-09-19 20:05 ` Trond Myklebust
2019-09-19 20:20 ` Alkis Georgopoulos
2019-09-19 20:40 ` Trond Myklebust
2019-09-19 21:19 ` Daniel Forrest [this message]
2019-09-19 21:42 ` Trond Myklebust
2019-09-19 22:16 ` Daniel Forrest
2019-09-20 9:25 ` Alkis Georgopoulos
2019-09-20 9:48 ` Alkis Georgopoulos
2019-09-20 10:04 ` Alkis Georgopoulos
2019-09-21 7:52 ` Alkis Georgopoulos
2019-09-21 7:59 ` Alkis Georgopoulos
2019-09-21 11:02 ` Alkis Georgopoulos
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190919211912.GA21865@cosmos.ssec.wisc.edu \
--to=dforrest@wisc.edu \
--cc=alkisg@gmail.com \
--cc=linux-nfs@vger.kernel.org \
--cc=trondmy@hammerspace.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).