From: Murphy Zhou <jencce.kernel@gmail.com>
To: Olga Kornievskaia <aglo@umich.edu>
Cc: Murphy Zhou <jencce.kernel@gmail.com>,
linux-nfs <linux-nfs@vger.kernel.org>,
Trond Myklebust <trondmy@gmail.com>
Subject: Re: [PATCH] NFSv4: fix stateid refreshing when CLOSE racing with OPEN
Date: Fri, 11 Oct 2019 17:42:53 +0800 [thread overview]
Message-ID: <20191011094253.yjwncvqv4cnq7wpn@xzhoux.usersys.redhat.com> (raw)
In-Reply-To: <CAN-5tyGKNFsxQHne4hFhON1VRZzCkUjwFMX3ZuzfLASgEN0pMw@mail.gmail.com>
On Thu, Oct 10, 2019 at 01:32:50PM -0400, Olga Kornievskaia wrote:
> On Thu, Oct 10, 2019 at 3:42 AM Murphy Zhou <jencce.kernel@gmail.com> wrote:
> >
> > Since commit:
> > [0e0cb35] NFSv4: Handle NFS4ERR_OLD_STATEID in CLOSE/OPEN_DOWNGRADE
> >
> > xfstests generic/168 on v4.2 starts to fail because reflink call gets:
> > +XFS_IOC_CLONE_RANGE: Resource temporarily unavailable
>
> I don't believe this failure has to do with getting ERR_OLD_STATEID on
> the CLOSE. What you see on the network trace is expected as the client
> in parallel sends OPEN/CLOSE thus server will fail the CLOSE with the
> ERR_OLD_STATEID since it already updated its stateid for the OPEN.
>
> > In tshark output, NFS4ERR_OLD_STATEID stands out when comparing with
> > good ones:
> >
> > 5210 NFS 406 V4 Reply (Call In 5209) OPEN StateID: 0xadb5
> > 5211 NFS 314 V4 Call GETATTR FH: 0x8d44a6b1
> > 5212 NFS 250 V4 Reply (Call In 5211) GETATTR
> > 5213 NFS 314 V4 Call GETATTR FH: 0x8d44a6b1
> > 5214 NFS 250 V4 Reply (Call In 5213) GETATTR
> > 5216 NFS 422 V4 Call WRITE StateID: 0xa818 Offset: 851968 Len: 65536
> > 5218 NFS 266 V4 Reply (Call In 5216) WRITE
> > 5219 NFS 382 V4 Call OPEN DH: 0x8d44a6b1/
> > 5220 NFS 338 V4 Call CLOSE StateID: 0xadb5
> > 5222 NFS 406 V4 Reply (Call In 5219) OPEN StateID: 0xa342
> > 5223 NFS 250 V4 Reply (Call In 5220) CLOSE Status: NFS4ERR_OLD_STATEID
> > 5225 NFS 338 V4 Call CLOSE StateID: 0xa342
> > 5226 NFS 314 V4 Call GETATTR FH: 0x8d44a6b1
> > 5227 NFS 266 V4 Reply (Call In 5225) CLOSE
> > 5228 NFS 250 V4 Reply (Call In 5226) GETATTR
>
> "resource temporarily unavailable" is more likely to do with ulimit limits.
>
> I also saw the same error. After I increased the ulimit for the stack
> size, the problem went away. There might still be a problem somewhere
> in the kernel.
Do you mean ulimit -s ? I set it to 'unlimited' and still can reproduce
this.
Thanks,
M
>
> Trond, is it possible that we have too many CLOSE recovery on the
> stack that's eating up stack space?
>
> > It's easy to reproduce. By printing some logs, found that we are making
> > CLOSE seqid larger then OPEN seqid when racing.
> >
> > Fix this by not bumping seqid when it's equal to OPEN seqid. Also
> > put the whole changing process into seqlock read protection in case
> > really bad luck, and this is the same locking behavior with the
> > old deleted function.
> >
> > Fixes: 0e0cb35b417f ("NFSv4: Handle NFS4ERR_OLD_STATEID in CLOSE/OPEN_DOWNGRADE")
> > Signed-off-by: Murphy Zhou <jencce.kernel@gmail.com>
> > ---
> > fs/nfs/nfs4proc.c | 13 ++++++++-----
> > 1 file changed, 8 insertions(+), 5 deletions(-)
> >
> > diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
> > index 11eafcf..6db5a09 100644
> > --- a/fs/nfs/nfs4proc.c
> > +++ b/fs/nfs/nfs4proc.c
> > @@ -3334,12 +3334,13 @@ static void nfs4_sync_open_stateid(nfs4_stateid *dst,
> > break;
> > }
> > seqid_open = state->open_stateid.seqid;
> > - if (read_seqretry(&state->seqlock, seq))
> > - continue;
> >
> > dst_seqid = be32_to_cpu(dst->seqid);
> > if ((s32)(dst_seqid - be32_to_cpu(seqid_open)) < 0)
> > dst->seqid = seqid_open;
> > +
> > + if (read_seqretry(&state->seqlock, seq))
> > + continue;
> > break;
> > }
> > }
> > @@ -3367,14 +3368,16 @@ static bool nfs4_refresh_open_old_stateid(nfs4_stateid *dst,
> > break;
> > }
> > seqid_open = state->open_stateid.seqid;
> > - if (read_seqretry(&state->seqlock, seq))
> > - continue;
> >
> > dst_seqid = be32_to_cpu(dst->seqid);
> > - if ((s32)(dst_seqid - be32_to_cpu(seqid_open)) >= 0)
> > + if ((s32)(dst_seqid - be32_to_cpu(seqid_open)) > 0)
> > dst->seqid = cpu_to_be32(dst_seqid + 1);
> > else
> > dst->seqid = seqid_open;
> > +
> > + if (read_seqretry(&state->seqlock, seq))
> > + continue;
> > +
> > ret = true;
> > break;
> > }
> > --
> > 1.8.3.1
> >
next prev parent reply other threads:[~2019-10-11 9:43 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-10 7:40 [PATCH] NFSv4: fix stateid refreshing when CLOSE racing with OPEN Murphy Zhou
2019-10-10 14:46 ` Trond Myklebust
2019-10-11 8:49 ` Murphy Zhou
2019-10-11 14:14 ` Trond Myklebust
2020-09-03 17:54 ` Benjamin Coddington
2020-09-04 3:04 ` Murphy Zhou
2020-09-04 10:55 ` Benjamin Coddington
2020-09-04 14:14 ` Chuck Lever
2020-09-08 12:43 ` Benjamin Coddington
2020-09-04 16:13 ` Olga Kornievskaia
2019-10-10 17:32 ` Olga Kornievskaia
2019-10-11 9:42 ` Murphy Zhou [this message]
2019-10-11 14:18 ` Trond Myklebust
2019-10-11 18:50 ` Olga Kornievskaia
2019-10-19 0:34 ` Olga Kornievskaia
2019-10-21 17:15 ` Olga Kornievskaia
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191011094253.yjwncvqv4cnq7wpn@xzhoux.usersys.redhat.com \
--to=jencce.kernel@gmail.com \
--cc=aglo@umich.edu \
--cc=linux-nfs@vger.kernel.org \
--cc=trondmy@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).