From: Alan Post <adp@prgmr.com>
To: linux-nfs <linux-nfs@vger.kernel.org>
Subject: Re: User process NFS write hang in wait_on_commit with kworker
Date: Fri, 28 Jun 2019 12:33:24 -0600 [thread overview]
Message-ID: <20190628183324.GJ4158@turtle.email> (raw)
In-Reply-To: <20190621204723.GU4158@turtle.email>
On Fri, Jun 21, 2019 at 02:47:23PM -0600, Alan Post wrote:
> > Verifying this is the problem could be done by setting up some rolling
> > network captures.. but sometimes it can be hard to not have the capture
> > fill up with continuing traffic from other processes.
> >
>
> I did go ahead and set up a rolling capture between this NFS
> server and one rack of clients--I hope I can catch the event as
> it happens. Time will tell.
>
I've run this rolling capture and did catch four candidate events.
I haven't confirmed any of them are real--I don't really know
what it is I'm looking for, so I've been approaching the problem
by incrementally/recursively throwing stuff out and manually
working through what's left.
As far as I understand it, for a particular xid, there should be a
call and a reply. The approach I took then was to pull out these
fields from my capture and ignore RPC calls where both are present
in my capture. It seems this is simplistic, as the number of RPC
calls I have without an attendant reply isn't lining up with my
incident window.
In one example, I have a series of READ calls which cease
generating RPC reply messages as the offset for the file continues
to increases. After a couple/few dozen messages, the RPC replies
continue as they were. Is there a normal or routine explanation
for this?
RFC 5531 and the NetworkTracing page on wiki.linux-nfs.org have
been quite helpful bringing me up to speed. If any of you have
advice or guidance or can clarify my understanding of how the
call/reply RPC mechanism works I appreciate it.
-A
--
Alan Post | Xen VPS hosting for the technically adept
PO Box 61688 | Sunnyvale, CA 94088-1681 | https://prgmr.com/
email: adp@prgmr.com
next prev parent reply other threads:[~2019-06-28 18:31 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-18 0:06 User process NFS write hang in wait_on_commit with kworker Alan Post
2019-06-18 15:29 ` Benjamin Coddington
2019-06-19 0:07 ` Alan Post
2019-06-19 12:38 ` Benjamin Coddington
2019-06-21 20:47 ` Alan Post
2019-06-28 18:33 ` Alan Post [this message]
2019-07-02 9:55 ` Benjamin Coddington
2019-07-03 21:32 ` Alan Post
2019-07-05 23:53 ` Tom Talpey
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190628183324.GJ4158@turtle.email \
--to=adp@prgmr.com \
--cc=linux-nfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).