Re: [PATCH v3 40/44] SUNRPC: Simplify TCP receive code by switching to using iterators

[Trond Myklebust <trondmy@hammerspace.com>]

On Thu, 2018-11-29 at 19:28 +0000, Cristian Marussi wrote:
> Hi Trond, Catalin
> 
> On 09/11/2018 11:19, Catalin Marinas wrote:
> > Hi Trond,
> > 
> > On Mon, Sep 17, 2018 at 09:03:31AM -0400, Trond Myklebust wrote:
> > > Most of this code should also be reusable with other socket
> > > types.
> > > 
> > > Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
> > > ---
> > >  include/linux/sunrpc/xprtsock.h |  19 +-
> > >  include/trace/events/sunrpc.h   |  15 +-
> > >  net/sunrpc/xprtsock.c           | 694 +++++++++++++++-----------
> > > ------
> > >  3 files changed, 335 insertions(+), 393 deletions(-)
> > 
> > With latest mainline (24ccea7e102d, it includes Al Viro's iov_iter
> > fixup) I started hitting some severe slowdown and systemd timeouts
> > with
> > nfsroot on arm64 machines (physical or guests under KVM).
> > Interestingly,
> > it only happens when the client kernel is configured with 64K
> > pages, the
> > 4K pages configuration runs fine. It also runs fine if I add
> > rsize=65536
> > to the nfsroot= argument.
> > 
> > Bisecting led me to commit 277e4ab7d530 ("SUNRPC: Simplify TCP
> > receive
> > code by switching to using iterators"). Prior to this commit, it
> > works
> > fine.
> > 
> > Some more info:
> > 
> > - defconfig with CONFIG_ARM64_64K_PAGES enabled
> > 
> > - kernel cmdline arg: nfsroot=<some-server>:/srv/nfs/debian-
> > arm64,tcp,v4
> > 
> > - if it matters, the server is also an arm64 machine running 4.19
> > with
> >   4K pages configuration
> > 

Question to you both: when this happens, does /proc/*/stack show any of
the processes hanging in the socket or sunrpc code? If so, can you
please send me examples of those stack traces (i.e. the contents of
/proc/<pid>/stack for the processes that are hanging)

I'd be particularly interested  if the processes in question are
related to the rpciod workqueue.

Thanks
   Trond

-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com