Linux-NFS Archive on lore.kernel.org
 help / color / Atom feed
From: Kevin Vasko <kvasko@gmail.com>
To: Bruce Fields <bfields@fieldses.org>
Cc: Chuck Lever <chuck.lever@oracle.com>,
	Trond Myklebust <trond.myklebust@hammerspace.com>,
	Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: Re: NFSv4 client locks up on larger writes with Kerberos enabled
Date: Thu, 26 Sep 2019 11:25:42 -0500
Message-ID: <3260B7EA-2B89-4EA2-BCFE-833C63EF0441@gmail.com> (raw)

Bruce,

Yes. It’s a Dell EMC Unity 300 storage appliance. 

From what I have gathered, under the hood it is running SLES12SP1

3.12.74-60.64.66.1.NEOKERNEL_SLES12SP1 #2 SMP Fri May 17 06;41:36 EDT 2019 x86_64

Unfortunately I only have limited access to the appliance through a “service” account. I do not have root privileges and I can only run certain commands under the service account. 

I’ve got a ticket open with them and it’s been escalated to their “engineering” department but they haven’t provided much insight as of yet.

If you give me some idea on what might help I’ll try to get more info if I can.


-Kevin

> On Sep 26, 2019, at 11:06 AM, Bruce Fields <bfields@fieldses.org> wrote:
> 
> On Thu, Sep 26, 2019 at 08:55:17AM -0700, Chuck Lever wrote:
>>>> On Sep 25, 2019, at 1:07 PM, Bruce Fields <bfields@fieldses.org> wrote:
>>> 
>>> On Wed, Sep 25, 2019 at 11:49:14AM -0700, Chuck Lever wrote:
>>>> Sounds like the NFS server is dropping the connection. With
>>>> GSS enabled, that's usually a sign that the GSS window has
>>>> overflowed.
>>> 
>>> Would that show up in the rpc statistics on the client somehow?
>> 
>> More likely on the server. The client just sees a disconnect
>> without any explanation attached.
> 
> So watching a count of disconnects might at least tell us something?
> 
>> gss_verify_header is where the checking is done on the server.
>> Disappointingly, I see some dprintk's in there, but no static
>> trace events.
> 
> Kevin, was this a Linux server?
> 
> --b.
> 
>>> In that case--I seem to remember there's a way to configure the size of
>>> the client's slot table, maybe lowering that (decreasing the number of
>>> rpc's allowed to be outstanding at a time) would work around the
>>> problem.
>> 
>>> Should the client be doing something different to avoid or recover from
>>> overflows of the gss window?
>> 
>> The client attempts to meter the request stream so that it stays
>> within the bounds of the GSS sequence number window. The stream
>> of requests is typically unordered coming out of the transmit
>> queue.
>> 
>> There is some new code (since maybe v5.0?) that handles the
>> metering: gss_xmit_need_reencode().

             reply index

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-09-26 16:25 Kevin Vasko [this message]
  -- strict thread matches above, loose matches on Subject: below --
2019-09-18 23:36 Kevin Vasko
     [not found] ` <E172CA50-EC89-4072-9C1D-1B825DC3FE8B@lysator.liu.se>
2019-09-19 14:19   ` Kevin Vasko
     [not found]     ` <72E82F62-C743-4783-B81E-98E6A8E35738@lysator.liu.se>
2019-09-19 14:58       ` Kevin Vasko
2019-09-25 16:48 ` bfields
2019-09-25 17:06   ` Chuck Lever
2019-09-25 18:44     ` Kevin Vasko
2019-09-25 18:49       ` Chuck Lever
2019-09-25 19:10         ` Kevin Vasko
2019-09-25 20:07         ` Bruce Fields
2019-09-26 15:55           ` Chuck Lever
2019-09-26 16:05             ` Bruce Fields
2019-09-26 19:55             ` Bruce Fields
2019-09-30 14:51               ` Kevin Vasko
2019-09-30 16:19                 ` Bruce Fields
2019-09-26  7:30       ` Daniel Kobras

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3260B7EA-2B89-4EA2-BCFE-833C63EF0441@gmail.com \
    --to=kvasko@gmail.com \
    --cc=bfields@fieldses.org \
    --cc=chuck.lever@oracle.com \
    --cc=linux-nfs@vger.kernel.org \
    --cc=trond.myklebust@hammerspace.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-NFS Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-nfs/0 linux-nfs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-nfs linux-nfs/ https://lore.kernel.org/linux-nfs \
		linux-nfs@vger.kernel.org
	public-inbox-index linux-nfs

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-nfs


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git