[PATCH] NFS: Don't let readdirplus revalidate an inode that was marked as stale

[PATCH] NFS: Don't let readdirplus revalidate an inode that was marked as stale

[Trond Myklebust]

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
---
 fs/nfs/dir.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c
index aaf7bd0cbae2..a924d66b5608 100644
--- a/fs/nfs/dir.c
+++ b/fs/nfs/dir.c
@@ -424,12 +424,17 @@ static int xdr_decode(nfs_readdir_descriptor_t *desc,
 static
 int nfs_same_file(struct dentry *dentry, struct nfs_entry *entry)
 {
+	struct inode *inode;
 	struct nfs_inode *nfsi;
 
 	if (d_really_is_negative(dentry))
 		return 0;
 
-	nfsi = NFS_I(d_inode(dentry));
+	inode = d_inode(dentry);
+	if (is_bad_inode(inode) || NFS_STALE(inode))
+		return 0;
+
+	nfsi = NFS_I(inode);
 	if (entry->fattr->fileid == nfsi->fileid)
 		return 1;
 	if (nfs_compare_fh(entry->fh, &nfsi->fh) == 0)
-- 
2.5.5

grace period

[Marc Eshel]

Hi Bruce,
I see that setting the number of nfsd threads to 0 (echo 0 > 
/proc/fs/nfsd/threads) is not releasing the locks and putting the server 
in grace mode. What is the best way to go into grace period, in new 
version of the kernel, without restarting the nfs server?
Thanks, Marc.

Re: grace period

[Bruce Fields]

On Thu, Jun 30, 2016 at 02:46:19PM -0700, Marc Eshel wrote:
> I see that setting the number of nfsd threads to 0 (echo 0 > 
> /proc/fs/nfsd/threads) is not releasing the locks and putting the server 
> in grace mode.

Writing 0 to /proc/fs/nfsd/threads shuts down knfsd.  So it should
certainly drop locks.  If that's not happening, there's a bug, but we'd
need to know more details (version numbers, etc.) to help.

That alone has never been enough to start a grace period--you'd have to
start knfsd again to do that.

> What is the best way to go into grace period, in new version of the
> kernel, without restarting the nfs server?

Restarting the nfs server is the only way.  That's true on older kernels
true, as far as I know.  (OK, you can apparently make lockd do something
like this with a signal, I don't know if that's used much, and I doubt
it works outside an NFSv3-only environment.)

So if you want locks dropped and a new grace period, then you should run
"systemctl restart nfs-server", or your distro's equivalent.

But you're probably doing something more complicated than that.  I'm not
sure I understand the question....

--b.

Re: grace period

[Marc Eshel]

It used to be that sending KILL signal to lockd would free locks and start 
Grace period, and when setting nfsd threads to zero, nfsd_last_thread() 
calls nfsd_shutdown that called lockd_down that I believe was causing both 
freeing of locks and starting grace period or maybe it was setting it back 
to a value > 0 that started the grace period.
Any way starting with the kernels that are in RHEL7.1 and up echo 0 > 
/proc/fs/nfsd/threads doesn't do it anymore, I assume going to common 
grace period for NLM and NFSv4 changed things.
The question is how to do IP fail-over, so when a node fails and the IP is 
moving to another node, we need to go into grace period on all the nodes 
in the cluster so the locks of the failed node are not given to anyone 
other than the client that is reclaiming his locks. Restarting NFS server 
is to distractive. For NFSv3 KILL signal to lockd still works but for 
NFSv4 have no way to do it for v4.
Marc. 



From:   Bruce Fields <bfields@fieldses.org>
To:     Marc Eshel/Almaden/IBM@IBMUS
Cc:     linux-nfs@vger.kernel.org
Date:   07/01/2016 09:09 AM
Subject:        Re: grace period



On Thu, Jun 30, 2016 at 02:46:19PM -0700, Marc Eshel wrote:
> I see that setting the number of nfsd threads to 0 (echo 0 > 
> /proc/fs/nfsd/threads) is not releasing the locks and putting the server 

> in grace mode.

Writing 0 to /proc/fs/nfsd/threads shuts down knfsd.  So it should
certainly drop locks.  If that's not happening, there's a bug, but we'd
need to know more details (version numbers, etc.) to help.

That alone has never been enough to start a grace period--you'd have to
start knfsd again to do that.

> What is the best way to go into grace period, in new version of the
> kernel, without restarting the nfs server?

Restarting the nfs server is the only way.  That's true on older kernels
true, as far as I know.  (OK, you can apparently make lockd do something
like this with a signal, I don't know if that's used much, and I doubt
it works outside an NFSv3-only environment.)

So if you want locks dropped and a new grace period, then you should run
"systemctl restart nfs-server", or your distro's equivalent.

But you're probably doing something more complicated than that.  I'm not
sure I understand the question....

--b.

Re: grace period

[Bruce Fields]

On Fri, Jul 01, 2016 at 10:31:55AM -0700, Marc Eshel wrote:
> It used to be that sending KILL signal to lockd would free locks and start 
> Grace period, and when setting nfsd threads to zero, nfsd_last_thread() 
> calls nfsd_shutdown that called lockd_down that I believe was causing both 
> freeing of locks and starting grace period or maybe it was setting it back 
> to a value > 0 that started the grace period.

OK, apologies, I didn't know (or forgot) that.

> Any way starting with the kernels that are in RHEL7.1 and up echo 0 > 
> /proc/fs/nfsd/threads doesn't do it anymore, I assume going to common 
> grace period for NLM and NFSv4 changed things.
> The question is how to do IP fail-over, so when a node fails and the IP is 
> moving to another node, we need to go into grace period on all the nodes 
> in the cluster so the locks of the failed node are not given to anyone 
> other than the client that is reclaiming his locks. Restarting NFS server 
> is to distractive.

What's the difference?  Just that clients don't have to reestablish tcp
connections?

--b.

> For NFSv3 KILL signal to lockd still works but for 
> NFSv4 have no way to do it for v4.
> Marc. 
> 
> 
> 
> From:   Bruce Fields <bfields@fieldses.org>
> To:     Marc Eshel/Almaden/IBM@IBMUS
> Cc:     linux-nfs@vger.kernel.org
> Date:   07/01/2016 09:09 AM
> Subject:        Re: grace period
> 
> 
> 
> On Thu, Jun 30, 2016 at 02:46:19PM -0700, Marc Eshel wrote:
> > I see that setting the number of nfsd threads to 0 (echo 0 > 
> > /proc/fs/nfsd/threads) is not releasing the locks and putting the server 
> 
> > in grace mode.
> 
> Writing 0 to /proc/fs/nfsd/threads shuts down knfsd.  So it should
> certainly drop locks.  If that's not happening, there's a bug, but we'd
> need to know more details (version numbers, etc.) to help.
> 
> That alone has never been enough to start a grace period--you'd have to
> start knfsd again to do that.
> 
> > What is the best way to go into grace period, in new version of the
> > kernel, without restarting the nfs server?
> 
> Restarting the nfs server is the only way.  That's true on older kernels
> true, as far as I know.  (OK, you can apparently make lockd do something
> like this with a signal, I don't know if that's used much, and I doubt
> it works outside an NFSv3-only environment.)
> 
> So if you want locks dropped and a new grace period, then you should run
> "systemctl restart nfs-server", or your distro's equivalent.
> 
> But you're probably doing something more complicated than that.  I'm not
> sure I understand the question....
> 
> --b.
> 
> 
> 
> 

Re: grace period

[Marc Eshel]

linux-nfs-owner@vger.kernel.org wrote on 07/01/2016 01:07:42 PM:

> From: Bruce Fields <bfields@fieldses.org>
> To: Marc Eshel/Almaden/IBM@IBMUS
> Cc: linux-nfs@vger.kernel.org
> Date: 07/01/2016 01:07 PM
> Subject: Re: grace period
> Sent by: linux-nfs-owner@vger.kernel.org
> 
> On Fri, Jul 01, 2016 at 10:31:55AM -0700, Marc Eshel wrote:
> > It used to be that sending KILL signal to lockd would free locks and 
start 
> > Grace period, and when setting nfsd threads to zero, 
nfsd_last_thread() 
> > calls nfsd_shutdown that called lockd_down that I believe was causing 
both 
> > freeing of locks and starting grace period or maybe it was setting it 
back 
> > to a value > 0 that started the grace period.
> 
> OK, apologies, I didn't know (or forgot) that.
> 
> > Any way starting with the kernels that are in RHEL7.1 and up echo 0 > 
> > /proc/fs/nfsd/threads doesn't do it anymore, I assume going to common 
> > grace period for NLM and NFSv4 changed things.
> > The question is how to do IP fail-over, so when a node fails and the 
IP is 
> > moving to another node, we need to go into grace period on all the 
nodes 
> > in the cluster so the locks of the failed node are not given to anyone 

> > other than the client that is reclaiming his locks. Restarting NFS 
server 
> > is to distractive.
> 
> What's the difference?  Just that clients don't have to reestablish tcp
> connections?

I am not sure what else systemctl will do but I need to control the order 
of the restart so the client will not see any errors.
I don't think that echo 0 > /proc/fs/nfsd/threads is freeing the lock, at 
least not the v3 locks, I will try again with v4.
The question is what is the most basic operation that can be done to start 
grace, will echo 8 > /proc/fs/nfsd/threads following echo 0 do it?
or is there any other primitive that will do it?
Marc.

> 
> --b.
> 
> > For NFSv3 KILL signal to lockd still works but for 
> > NFSv4 have no way to do it for v4.
> > Marc. 
> > 
> > 
> > 
> > From:   Bruce Fields <bfields@fieldses.org>
> > To:     Marc Eshel/Almaden/IBM@IBMUS
> > Cc:     linux-nfs@vger.kernel.org
> > Date:   07/01/2016 09:09 AM
> > Subject:        Re: grace period
> > 
> > 
> > 
> > On Thu, Jun 30, 2016 at 02:46:19PM -0700, Marc Eshel wrote:
> > > I see that setting the number of nfsd threads to 0 (echo 0 > 
> > > /proc/fs/nfsd/threads) is not releasing the locks and putting the 
server 
> > 
> > > in grace mode.
> > 
> > Writing 0 to /proc/fs/nfsd/threads shuts down knfsd.  So it should
> > certainly drop locks.  If that's not happening, there's a bug, but 
we'd
> > need to know more details (version numbers, etc.) to help.
> > 
> > That alone has never been enough to start a grace period--you'd have 
to
> > start knfsd again to do that.
> > 
> > > What is the best way to go into grace period, in new version of the
> > > kernel, without restarting the nfs server?
> > 
> > Restarting the nfs server is the only way.  That's true on older 
kernels
> > true, as far as I know.  (OK, you can apparently make lockd do 
something
> > like this with a signal, I don't know if that's used much, and I doubt
> > it works outside an NFSv3-only environment.)
> > 
> > So if you want locks dropped and a new grace period, then you should 
run
> > "systemctl restart nfs-server", or your distro's equivalent.
> > 
> > But you're probably doing something more complicated than that.  I'm 
not
> > sure I understand the question....
> > 
> > --b.
> > 
> > 
> > 
> > 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: grace period

[Marc Eshel]

This is my v3 test that show the lock still there after echo 0 > 
/proc/fs/nfsd/threads

[root@sonascl21 ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.2 (Maipo)

[root@sonascl21 ~]# uname -a
Linux sonascl21.sonasad.almaden.ibm.com 3.10.0-327.el7.x86_64 #1 SMP Thu 
Oct 29 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux

[root@sonascl21 ~]# cat /proc/locks | grep 999
3: POSIX  ADVISORY  WRITE 2349 00:2a:489486 0 999

[root@sonascl21 ~]# echo 0 > /proc/fs/nfsd/threads
[root@sonascl21 ~]# cat /proc/fs/nfsd/threads
0

[root@sonascl21 ~]# cat /proc/locks | grep 999
3: POSIX  ADVISORY  WRITE 2349 00:2a:489486 0 999




From:   Bruce Fields <bfields@fieldses.org>
To:     Marc Eshel/Almaden/IBM@IBMUS
Cc:     linux-nfs@vger.kernel.org
Date:   07/01/2016 01:07 PM
Subject:        Re: grace period



On Fri, Jul 01, 2016 at 10:31:55AM -0700, Marc Eshel wrote:
> It used to be that sending KILL signal to lockd would free locks and 
start 
> Grace period, and when setting nfsd threads to zero, nfsd_last_thread() 
> calls nfsd_shutdown that called lockd_down that I believe was causing 
both 
> freeing of locks and starting grace period or maybe it was setting it 
back 
> to a value > 0 that started the grace period.

OK, apologies, I didn't know (or forgot) that.

> Any way starting with the kernels that are in RHEL7.1 and up echo 0 > 
> /proc/fs/nfsd/threads doesn't do it anymore, I assume going to common 
> grace period for NLM and NFSv4 changed things.
> The question is how to do IP fail-over, so when a node fails and the IP 
is 
> moving to another node, we need to go into grace period on all the nodes 

> in the cluster so the locks of the failed node are not given to anyone 
> other than the client that is reclaiming his locks. Restarting NFS 
server 
> is to distractive.

What's the difference?  Just that clients don't have to reestablish tcp
connections?

--b.

> For NFSv3 KILL signal to lockd still works but for 
> NFSv4 have no way to do it for v4.
> Marc. 
> 
> 
> 
> From:   Bruce Fields <bfields@fieldses.org>
> To:     Marc Eshel/Almaden/IBM@IBMUS
> Cc:     linux-nfs@vger.kernel.org
> Date:   07/01/2016 09:09 AM
> Subject:        Re: grace period
> 
> 
> 
> On Thu, Jun 30, 2016 at 02:46:19PM -0700, Marc Eshel wrote:
> > I see that setting the number of nfsd threads to 0 (echo 0 > 
> > /proc/fs/nfsd/threads) is not releasing the locks and putting the 
server 
> 
> > in grace mode.
> 
> Writing 0 to /proc/fs/nfsd/threads shuts down knfsd.  So it should
> certainly drop locks.  If that's not happening, there's a bug, but we'd
> need to know more details (version numbers, etc.) to help.
> 
> That alone has never been enough to start a grace period--you'd have to
> start knfsd again to do that.
> 
> > What is the best way to go into grace period, in new version of the
> > kernel, without restarting the nfs server?
> 
> Restarting the nfs server is the only way.  That's true on older kernels
> true, as far as I know.  (OK, you can apparently make lockd do something
> like this with a signal, I don't know if that's used much, and I doubt
> it works outside an NFSv3-only environment.)
> 
> So if you want locks dropped and a new grace period, then you should run
> "systemctl restart nfs-server", or your distro's equivalent.
> 
> But you're probably doing something more complicated than that.  I'm not
> sure I understand the question....
> 
> --b.
> 
> 
> 
> 

Re: grace period

[Bruce Fields]

On Fri, Jul 01, 2016 at 01:24:48PM -0700, Marc Eshel wrote:
> linux-nfs-owner@vger.kernel.org wrote on 07/01/2016 01:07:42 PM:
> 
> > From: Bruce Fields <bfields@fieldses.org>
> > To: Marc Eshel/Almaden/IBM@IBMUS
> > Cc: linux-nfs@vger.kernel.org
> > Date: 07/01/2016 01:07 PM
> > Subject: Re: grace period
> > Sent by: linux-nfs-owner@vger.kernel.org
> > 
> > On Fri, Jul 01, 2016 at 10:31:55AM -0700, Marc Eshel wrote:
> > > It used to be that sending KILL signal to lockd would free locks and 
> start 
> > > Grace period, and when setting nfsd threads to zero, 
> nfsd_last_thread() 
> > > calls nfsd_shutdown that called lockd_down that I believe was causing 
> both 
> > > freeing of locks and starting grace period or maybe it was setting it 
> back 
> > > to a value > 0 that started the grace period.
> > 
> > OK, apologies, I didn't know (or forgot) that.
> > 
> > > Any way starting with the kernels that are in RHEL7.1 and up echo 0 > 
> > > /proc/fs/nfsd/threads doesn't do it anymore, I assume going to common 
> > > grace period for NLM and NFSv4 changed things.
> > > The question is how to do IP fail-over, so when a node fails and the 
> IP is 
> > > moving to another node, we need to go into grace period on all the 
> nodes 
> > > in the cluster so the locks of the failed node are not given to anyone 
> 
> > > other than the client that is reclaiming his locks. Restarting NFS 
> server 
> > > is to distractive.
> > 
> > What's the difference?  Just that clients don't have to reestablish tcp
> > connections?
> 
> I am not sure what else systemctl will do but I need to control the order 
> of the restart so the client will not see any errors.
> I don't think that echo 0 > /proc/fs/nfsd/threads is freeing the lock, at 
> least not the v3 locks, I will try again with v4.
> The question is what is the most basic operation that can be done to start 
> grace, will echo 8 > /proc/fs/nfsd/threads following echo 0 do it?
> or is there any other primitive that will do it?

That should do it, though really so should just "systemctl restart
nfs-server"--if that causes errors then there's a bug somewhere.

--b.

> Marc.
> 
> > 
> > --b.
> > 
> > > For NFSv3 KILL signal to lockd still works but for 
> > > NFSv4 have no way to do it for v4.
> > > Marc. 
> > > 
> > > 
> > > 
> > > From:   Bruce Fields <bfields@fieldses.org>
> > > To:     Marc Eshel/Almaden/IBM@IBMUS
> > > Cc:     linux-nfs@vger.kernel.org
> > > Date:   07/01/2016 09:09 AM
> > > Subject:        Re: grace period
> > > 
> > > 
> > > 
> > > On Thu, Jun 30, 2016 at 02:46:19PM -0700, Marc Eshel wrote:
> > > > I see that setting the number of nfsd threads to 0 (echo 0 > 
> > > > /proc/fs/nfsd/threads) is not releasing the locks and putting the 
> server 
> > > 
> > > > in grace mode.
> > > 
> > > Writing 0 to /proc/fs/nfsd/threads shuts down knfsd.  So it should
> > > certainly drop locks.  If that's not happening, there's a bug, but 
> we'd
> > > need to know more details (version numbers, etc.) to help.
> > > 
> > > That alone has never been enough to start a grace period--you'd have 
> to
> > > start knfsd again to do that.
> > > 
> > > > What is the best way to go into grace period, in new version of the
> > > > kernel, without restarting the nfs server?
> > > 
> > > Restarting the nfs server is the only way.  That's true on older 
> kernels
> > > true, as far as I know.  (OK, you can apparently make lockd do 
> something
> > > like this with a signal, I don't know if that's used much, and I doubt
> > > it works outside an NFSv3-only environment.)
> > > 
> > > So if you want locks dropped and a new grace period, then you should 
> run
> > > "systemctl restart nfs-server", or your distro's equivalent.
> > > 
> > > But you're probably doing something more complicated than that.  I'm 
> not
> > > sure I understand the question....
> > > 
> > > --b.
> > > 
> > > 
> > > 
> > > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> 

Re: grace period

[Marc Eshel]

echo 0 > /proc/fs/nfsd/threads does delete the locks for v4 but not for v3
Marc.



From:   Marc Eshel/Almaden/IBM
To:     Bruce Fields <bfields@fieldses.org>
Cc:     linux-nfs@vger.kernel.org, Tomer Perry/Israel/IBM@IBMIL
Date:   07/01/2016 01:46 PM
Subject:        Re: grace period


This is my v3 test that show the lock still there after echo 0 > 
/proc/fs/nfsd/threads

[root@sonascl21 ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.2 (Maipo)

[root@sonascl21 ~]# uname -a
Linux sonascl21.sonasad.almaden.ibm.com 3.10.0-327.el7.x86_64 #1 SMP Thu 
Oct 29 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux

[root@sonascl21 ~]# cat /proc/locks | grep 999
3: POSIX  ADVISORY  WRITE 2349 00:2a:489486 0 999

[root@sonascl21 ~]# echo 0 > /proc/fs/nfsd/threads
[root@sonascl21 ~]# cat /proc/fs/nfsd/threads
0

[root@sonascl21 ~]# cat /proc/locks | grep 999
3: POSIX  ADVISORY  WRITE 2349 00:2a:489486 0 999





From:   Bruce Fields <bfields@fieldses.org>
To:     Marc Eshel/Almaden/IBM@IBMUS
Cc:     linux-nfs@vger.kernel.org
Date:   07/01/2016 01:07 PM
Subject:        Re: grace period



On Fri, Jul 01, 2016 at 10:31:55AM -0700, Marc Eshel wrote:
> It used to be that sending KILL signal to lockd would free locks and 
start 
> Grace period, and when setting nfsd threads to zero, nfsd_last_thread() 
> calls nfsd_shutdown that called lockd_down that I believe was causing 
both 
> freeing of locks and starting grace period or maybe it was setting it 
back 
> to a value > 0 that started the grace period.

OK, apologies, I didn't know (or forgot) that.

> Any way starting with the kernels that are in RHEL7.1 and up echo 0 > 
> /proc/fs/nfsd/threads doesn't do it anymore, I assume going to common 
> grace period for NLM and NFSv4 changed things.
> The question is how to do IP fail-over, so when a node fails and the IP 
is 
> moving to another node, we need to go into grace period on all the nodes 

> in the cluster so the locks of the failed node are not given to anyone 
> other than the client that is reclaiming his locks. Restarting NFS 
server 
> is to distractive.

What's the difference?  Just that clients don't have to reestablish tcp
connections?

--b.

> For NFSv3 KILL signal to lockd still works but for 
> NFSv4 have no way to do it for v4.
> Marc. 
> 
> 
> 
> From:   Bruce Fields <bfields@fieldses.org>
> To:     Marc Eshel/Almaden/IBM@IBMUS
> Cc:     linux-nfs@vger.kernel.org
> Date:   07/01/2016 09:09 AM
> Subject:        Re: grace period
> 
> 
> 
> On Thu, Jun 30, 2016 at 02:46:19PM -0700, Marc Eshel wrote:
> > I see that setting the number of nfsd threads to 0 (echo 0 > 
> > /proc/fs/nfsd/threads) is not releasing the locks and putting the 
server 
> 
> > in grace mode.
> 
> Writing 0 to /proc/fs/nfsd/threads shuts down knfsd.  So it should
> certainly drop locks.  If that's not happening, there's a bug, but we'd
> need to know more details (version numbers, etc.) to help.
> 
> That alone has never been enough to start a grace period--you'd have to
> start knfsd again to do that.
> 
> > What is the best way to go into grace period, in new version of the
> > kernel, without restarting the nfs server?
> 
> Restarting the nfs server is the only way.  That's true on older kernels
> true, as far as I know.  (OK, you can apparently make lockd do something
> like this with a signal, I don't know if that's used much, and I doubt
> it works outside an NFSv3-only environment.)
> 
> So if you want locks dropped and a new grace period, then you should run
> "systemctl restart nfs-server", or your distro's equivalent.
> 
> But you're probably doing something more complicated than that.  I'm not
> sure I understand the question....
> 
> --b.
> 
> 
> 
> 

Re: grace period

[Bruce Fields]

On Fri, Jul 01, 2016 at 01:46:42PM -0700, Marc Eshel wrote:
> This is my v3 test that show the lock still there after echo 0 > 
> /proc/fs/nfsd/threads
> 
> [root@sonascl21 ~]# cat /etc/redhat-release 
> Red Hat Enterprise Linux Server release 7.2 (Maipo)
> 
> [root@sonascl21 ~]# uname -a
> Linux sonascl21.sonasad.almaden.ibm.com 3.10.0-327.el7.x86_64 #1 SMP Thu 
> Oct 29 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux
> 
> [root@sonascl21 ~]# cat /proc/locks | grep 999
> 3: POSIX  ADVISORY  WRITE 2349 00:2a:489486 0 999
> 
> [root@sonascl21 ~]# echo 0 > /proc/fs/nfsd/threads
> [root@sonascl21 ~]# cat /proc/fs/nfsd/threads
> 0
> 
> [root@sonascl21 ~]# cat /proc/locks | grep 999
> 3: POSIX  ADVISORY  WRITE 2349 00:2a:489486 0 999

Huh, that's not what I see.  Are you positive that's the lock on the
backend filesystem and not the client-side lock (in case you're doing a
loopback mount?)

--b.

> 
> 
> 
> 
> From:   Bruce Fields <bfields@fieldses.org>
> To:     Marc Eshel/Almaden/IBM@IBMUS
> Cc:     linux-nfs@vger.kernel.org
> Date:   07/01/2016 01:07 PM
> Subject:        Re: grace period
> 
> 
> 
> On Fri, Jul 01, 2016 at 10:31:55AM -0700, Marc Eshel wrote:
> > It used to be that sending KILL signal to lockd would free locks and 
> start 
> > Grace period, and when setting nfsd threads to zero, nfsd_last_thread() 
> > calls nfsd_shutdown that called lockd_down that I believe was causing 
> both 
> > freeing of locks and starting grace period or maybe it was setting it 
> back 
> > to a value > 0 that started the grace period.
> 
> OK, apologies, I didn't know (or forgot) that.
> 
> > Any way starting with the kernels that are in RHEL7.1 and up echo 0 > 
> > /proc/fs/nfsd/threads doesn't do it anymore, I assume going to common 
> > grace period for NLM and NFSv4 changed things.
> > The question is how to do IP fail-over, so when a node fails and the IP 
> is 
> > moving to another node, we need to go into grace period on all the nodes 
> 
> > in the cluster so the locks of the failed node are not given to anyone 
> > other than the client that is reclaiming his locks. Restarting NFS 
> server 
> > is to distractive.
> 
> What's the difference?  Just that clients don't have to reestablish tcp
> connections?
> 
> --b.
> 
> > For NFSv3 KILL signal to lockd still works but for 
> > NFSv4 have no way to do it for v4.
> > Marc. 
> > 
> > 
> > 
> > From:   Bruce Fields <bfields@fieldses.org>
> > To:     Marc Eshel/Almaden/IBM@IBMUS
> > Cc:     linux-nfs@vger.kernel.org
> > Date:   07/01/2016 09:09 AM
> > Subject:        Re: grace period
> > 
> > 
> > 
> > On Thu, Jun 30, 2016 at 02:46:19PM -0700, Marc Eshel wrote:
> > > I see that setting the number of nfsd threads to 0 (echo 0 > 
> > > /proc/fs/nfsd/threads) is not releasing the locks and putting the 
> server 
> > 
> > > in grace mode.
> > 
> > Writing 0 to /proc/fs/nfsd/threads shuts down knfsd.  So it should
> > certainly drop locks.  If that's not happening, there's a bug, but we'd
> > need to know more details (version numbers, etc.) to help.
> > 
> > That alone has never been enough to start a grace period--you'd have to
> > start knfsd again to do that.
> > 
> > > What is the best way to go into grace period, in new version of the
> > > kernel, without restarting the nfs server?
> > 
> > Restarting the nfs server is the only way.  That's true on older kernels
> > true, as far as I know.  (OK, you can apparently make lockd do something
> > like this with a signal, I don't know if that's used much, and I doubt
> > it works outside an NFSv3-only environment.)
> > 
> > So if you want locks dropped and a new grace period, then you should run
> > "systemctl restart nfs-server", or your distro's equivalent.
> > 
> > But you're probably doing something more complicated than that.  I'm not
> > sure I understand the question....
> > 
> > --b.
> > 
> > 
> > 
> > 
> 
> 
> 
> 

Re: grace period

[Marc Eshel]

Yes, the locks are requested from another node, what fs are you using, I 
don't think it should make any difference, but I can try it with the same 
fs. 
Make sure you are using v3, it does work for v4.
Marc.



From:   Bruce Fields <bfields@fieldses.org>
To:     Marc Eshel/Almaden/IBM@IBMUS
Cc:     linux-nfs@vger.kernel.org, Tomer Perry <TOMP@il.ibm.com>
Date:   07/01/2016 02:01 PM
Subject:        Re: grace period



On Fri, Jul 01, 2016 at 01:46:42PM -0700, Marc Eshel wrote:
> This is my v3 test that show the lock still there after echo 0 > 
> /proc/fs/nfsd/threads
> 
> [root@sonascl21 ~]# cat /etc/redhat-release 
> Red Hat Enterprise Linux Server release 7.2 (Maipo)
> 
> [root@sonascl21 ~]# uname -a
> Linux sonascl21.sonasad.almaden.ibm.com 3.10.0-327.el7.x86_64 #1 SMP Thu 

> Oct 29 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux
> 
> [root@sonascl21 ~]# cat /proc/locks | grep 999
> 3: POSIX  ADVISORY  WRITE 2349 00:2a:489486 0 999
> 
> [root@sonascl21 ~]# echo 0 > /proc/fs/nfsd/threads
> [root@sonascl21 ~]# cat /proc/fs/nfsd/threads
> 0
> 
> [root@sonascl21 ~]# cat /proc/locks | grep 999
> 3: POSIX  ADVISORY  WRITE 2349 00:2a:489486 0 999

Huh, that's not what I see.  Are you positive that's the lock on the
backend filesystem and not the client-side lock (in case you're doing a
loopback mount?)

--b.

> 
> 
> 
> 
> From:   Bruce Fields <bfields@fieldses.org>
> To:     Marc Eshel/Almaden/IBM@IBMUS
> Cc:     linux-nfs@vger.kernel.org
> Date:   07/01/2016 01:07 PM
> Subject:        Re: grace period
> 
> 
> 
> On Fri, Jul 01, 2016 at 10:31:55AM -0700, Marc Eshel wrote:
> > It used to be that sending KILL signal to lockd would free locks and 
> start 
> > Grace period, and when setting nfsd threads to zero, 
nfsd_last_thread() 
> > calls nfsd_shutdown that called lockd_down that I believe was causing 
> both 
> > freeing of locks and starting grace period or maybe it was setting it 
> back 
> > to a value > 0 that started the grace period.
> 
> OK, apologies, I didn't know (or forgot) that.
> 
> > Any way starting with the kernels that are in RHEL7.1 and up echo 0 > 
> > /proc/fs/nfsd/threads doesn't do it anymore, I assume going to common 
> > grace period for NLM and NFSv4 changed things.
> > The question is how to do IP fail-over, so when a node fails and the 
IP 
> is 
> > moving to another node, we need to go into grace period on all the 
nodes 
> 
> > in the cluster so the locks of the failed node are not given to anyone 

> > other than the client that is reclaiming his locks. Restarting NFS 
> server 
> > is to distractive.
> 
> What's the difference?  Just that clients don't have to reestablish tcp
> connections?
> 
> --b.
> 
> > For NFSv3 KILL signal to lockd still works but for 
> > NFSv4 have no way to do it for v4.
> > Marc. 
> > 
> > 
> > 
> > From:   Bruce Fields <bfields@fieldses.org>
> > To:     Marc Eshel/Almaden/IBM@IBMUS
> > Cc:     linux-nfs@vger.kernel.org
> > Date:   07/01/2016 09:09 AM
> > Subject:        Re: grace period
> > 
> > 
> > 
> > On Thu, Jun 30, 2016 at 02:46:19PM -0700, Marc Eshel wrote:
> > > I see that setting the number of nfsd threads to 0 (echo 0 > 
> > > /proc/fs/nfsd/threads) is not releasing the locks and putting the 
> server 
> > 
> > > in grace mode.
> > 
> > Writing 0 to /proc/fs/nfsd/threads shuts down knfsd.  So it should
> > certainly drop locks.  If that's not happening, there's a bug, but 
we'd
> > need to know more details (version numbers, etc.) to help.
> > 
> > That alone has never been enough to start a grace period--you'd have 
to
> > start knfsd again to do that.
> > 
> > > What is the best way to go into grace period, in new version of the
> > > kernel, without restarting the nfs server?
> > 
> > Restarting the nfs server is the only way.  That's true on older 
kernels
> > true, as far as I know.  (OK, you can apparently make lockd do 
something
> > like this with a signal, I don't know if that's used much, and I doubt
> > it works outside an NFSv3-only environment.)
> > 
> > So if you want locks dropped and a new grace period, then you should 
run
> > "systemctl restart nfs-server", or your distro's equivalent.
> > 
> > But you're probably doing something more complicated than that.  I'm 
not
> > sure I understand the question....
> > 
> > --b.
> > 
> > 
> > 
> > 
> 
> 
> 
> 

Re: grace period

[Bruce Fields]

On Fri, Jul 01, 2016 at 03:42:43PM -0700, Marc Eshel wrote:
> Yes, the locks are requested from another node, what fs are you using, I 
> don't think it should make any difference, but I can try it with the same 
> fs. 
> Make sure you are using v3, it does work for v4.

I tested v3 on upstream.--b.

> Marc.
> 
> 
> 
> From:   Bruce Fields <bfields@fieldses.org>
> To:     Marc Eshel/Almaden/IBM@IBMUS
> Cc:     linux-nfs@vger.kernel.org, Tomer Perry <TOMP@il.ibm.com>
> Date:   07/01/2016 02:01 PM
> Subject:        Re: grace period
> 
> 
> 
> On Fri, Jul 01, 2016 at 01:46:42PM -0700, Marc Eshel wrote:
> > This is my v3 test that show the lock still there after echo 0 > 
> > /proc/fs/nfsd/threads
> > 
> > [root@sonascl21 ~]# cat /etc/redhat-release 
> > Red Hat Enterprise Linux Server release 7.2 (Maipo)
> > 
> > [root@sonascl21 ~]# uname -a
> > Linux sonascl21.sonasad.almaden.ibm.com 3.10.0-327.el7.x86_64 #1 SMP Thu 
> 
> > Oct 29 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux
> > 
> > [root@sonascl21 ~]# cat /proc/locks | grep 999
> > 3: POSIX  ADVISORY  WRITE 2349 00:2a:489486 0 999
> > 
> > [root@sonascl21 ~]# echo 0 > /proc/fs/nfsd/threads
> > [root@sonascl21 ~]# cat /proc/fs/nfsd/threads
> > 0
> > 
> > [root@sonascl21 ~]# cat /proc/locks | grep 999
> > 3: POSIX  ADVISORY  WRITE 2349 00:2a:489486 0 999
> 
> Huh, that's not what I see.  Are you positive that's the lock on the
> backend filesystem and not the client-side lock (in case you're doing a
> loopback mount?)
> 
> --b.
> 
> > 
> > 
> > 
> > 
> > From:   Bruce Fields <bfields@fieldses.org>
> > To:     Marc Eshel/Almaden/IBM@IBMUS
> > Cc:     linux-nfs@vger.kernel.org
> > Date:   07/01/2016 01:07 PM
> > Subject:        Re: grace period
> > 
> > 
> > 
> > On Fri, Jul 01, 2016 at 10:31:55AM -0700, Marc Eshel wrote:
> > > It used to be that sending KILL signal to lockd would free locks and 
> > start 
> > > Grace period, and when setting nfsd threads to zero, 
> nfsd_last_thread() 
> > > calls nfsd_shutdown that called lockd_down that I believe was causing 
> > both 
> > > freeing of locks and starting grace period or maybe it was setting it 
> > back 
> > > to a value > 0 that started the grace period.
> > 
> > OK, apologies, I didn't know (or forgot) that.
> > 
> > > Any way starting with the kernels that are in RHEL7.1 and up echo 0 > 
> > > /proc/fs/nfsd/threads doesn't do it anymore, I assume going to common 
> > > grace period for NLM and NFSv4 changed things.
> > > The question is how to do IP fail-over, so when a node fails and the 
> IP 
> > is 
> > > moving to another node, we need to go into grace period on all the 
> nodes 
> > 
> > > in the cluster so the locks of the failed node are not given to anyone 
> 
> > > other than the client that is reclaiming his locks. Restarting NFS 
> > server 
> > > is to distractive.
> > 
> > What's the difference?  Just that clients don't have to reestablish tcp
> > connections?
> > 
> > --b.
> > 
> > > For NFSv3 KILL signal to lockd still works but for 
> > > NFSv4 have no way to do it for v4.
> > > Marc. 
> > > 
> > > 
> > > 
> > > From:   Bruce Fields <bfields@fieldses.org>
> > > To:     Marc Eshel/Almaden/IBM@IBMUS
> > > Cc:     linux-nfs@vger.kernel.org
> > > Date:   07/01/2016 09:09 AM
> > > Subject:        Re: grace period
> > > 
> > > 
> > > 
> > > On Thu, Jun 30, 2016 at 02:46:19PM -0700, Marc Eshel wrote:
> > > > I see that setting the number of nfsd threads to 0 (echo 0 > 
> > > > /proc/fs/nfsd/threads) is not releasing the locks and putting the 
> > server 
> > > 
> > > > in grace mode.
> > > 
> > > Writing 0 to /proc/fs/nfsd/threads shuts down knfsd.  So it should
> > > certainly drop locks.  If that's not happening, there's a bug, but 
> we'd
> > > need to know more details (version numbers, etc.) to help.
> > > 
> > > That alone has never been enough to start a grace period--you'd have 
> to
> > > start knfsd again to do that.
> > > 
> > > > What is the best way to go into grace period, in new version of the
> > > > kernel, without restarting the nfs server?
> > > 
> > > Restarting the nfs server is the only way.  That's true on older 
> kernels
> > > true, as far as I know.  (OK, you can apparently make lockd do 
> something
> > > like this with a signal, I don't know if that's used much, and I doubt
> > > it works outside an NFSv3-only environment.)
> > > 
> > > So if you want locks dropped and a new grace period, then you should 
> run
> > > "systemctl restart nfs-server", or your distro's equivalent.
> > > 
> > > But you're probably doing something more complicated than that.  I'm 
> not
> > > sure I understand the question....
> > > 
> > > --b.
> > > 
> > > 
> > > 
> > > 
> > 
> > 
> > 
> > 
> 
> 
> 
> 

Re: grace period

[Marc Eshel]

I tried again NFSv3 locks with xfs export. "echo 0 > 
/proc/fs/nfsd/threads" releases locks on rhel7.0 but not on rhel7.2
What else can I show you to find the problem?
Marc.
 
works:
[root@boar11 ~]# uname -a
Linux boar11 3.10.0-123.el7.x86_64 #1 SMP Mon May 5 11:16:57 EDT 2014 
x86_64 x86_64 x86_64 GNU/Linux
[root@boar11 ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.0 (Maipo)

not working:
[root@sonascl21 ~]# uname -a
Linux sonascl21.sonasad.almaden.ibm.com 3.10.0-327.el7.x86_64 #1 SMP Thu 
Oct 29 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux
[root@sonascl21 ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.2 (Maipo)
[root@sonascl21 ~]# cat /proc/fs/nfsd/threads 
0
[root@sonascl21 ~]# cat /proc/locks
1: POSIX  ADVISORY  WRITE 2346 fd:00:1612092569 0 9999



From:   Bruce Fields <bfields@fieldses.org>
To:     Marc Eshel/Almaden/IBM@IBMUS
Cc:     linux-nfs@vger.kernel.org, Tomer Perry <TOMP@il.ibm.com>
Date:   07/01/2016 05:58 PM
Subject:        Re: grace period



On Fri, Jul 01, 2016 at 03:42:43PM -0700, Marc Eshel wrote:
> Yes, the locks are requested from another node, what fs are you using, I 

> don't think it should make any difference, but I can try it with the 
same 
> fs. 
> Make sure you are using v3, it does work for v4.

I tested v3 on upstream.--b.

> Marc.
> 
> 
> 
> From:   Bruce Fields <bfields@fieldses.org>
> To:     Marc Eshel/Almaden/IBM@IBMUS
> Cc:     linux-nfs@vger.kernel.org, Tomer Perry <TOMP@il.ibm.com>
> Date:   07/01/2016 02:01 PM
> Subject:        Re: grace period
> 
> 
> 
> On Fri, Jul 01, 2016 at 01:46:42PM -0700, Marc Eshel wrote:
> > This is my v3 test that show the lock still there after echo 0 > 
> > /proc/fs/nfsd/threads
> > 
> > [root@sonascl21 ~]# cat /etc/redhat-release 
> > Red Hat Enterprise Linux Server release 7.2 (Maipo)
> > 
> > [root@sonascl21 ~]# uname -a
> > Linux sonascl21.sonasad.almaden.ibm.com 3.10.0-327.el7.x86_64 #1 SMP 
Thu 
> 
> > Oct 29 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux
> > 
> > [root@sonascl21 ~]# cat /proc/locks | grep 999
> > 3: POSIX  ADVISORY  WRITE 2349 00:2a:489486 0 999
> > 
> > [root@sonascl21 ~]# echo 0 > /proc/fs/nfsd/threads
> > [root@sonascl21 ~]# cat /proc/fs/nfsd/threads
> > 0
> > 
> > [root@sonascl21 ~]# cat /proc/locks | grep 999
> > 3: POSIX  ADVISORY  WRITE 2349 00:2a:489486 0 999
> 
> Huh, that's not what I see.  Are you positive that's the lock on the
> backend filesystem and not the client-side lock (in case you're doing a
> loopback mount?)
> 
> --b.
> 
> > 
> > 
> > 
> > 
> > From:   Bruce Fields <bfields@fieldses.org>
> > To:     Marc Eshel/Almaden/IBM@IBMUS
> > Cc:     linux-nfs@vger.kernel.org
> > Date:   07/01/2016 01:07 PM
> > Subject:        Re: grace period
> > 
> > 
> > 
> > On Fri, Jul 01, 2016 at 10:31:55AM -0700, Marc Eshel wrote:
> > > It used to be that sending KILL signal to lockd would free locks and 

> > start 
> > > Grace period, and when setting nfsd threads to zero, 
> nfsd_last_thread() 
> > > calls nfsd_shutdown that called lockd_down that I believe was 
causing 
> > both 
> > > freeing of locks and starting grace period or maybe it was setting 
it 
> > back 
> > > to a value > 0 that started the grace period.
> > 
> > OK, apologies, I didn't know (or forgot) that.
> > 
> > > Any way starting with the kernels that are in RHEL7.1 and up echo 0 
> 
> > > /proc/fs/nfsd/threads doesn't do it anymore, I assume going to 
common 
> > > grace period for NLM and NFSv4 changed things.
> > > The question is how to do IP fail-over, so when a node fails and the 

> IP 
> > is 
> > > moving to another node, we need to go into grace period on all the 
> nodes 
> > 
> > > in the cluster so the locks of the failed node are not given to 
anyone 
> 
> > > other than the client that is reclaiming his locks. Restarting NFS 
> > server 
> > > is to distractive.
> > 
> > What's the difference?  Just that clients don't have to reestablish 
tcp
> > connections?
> > 
> > --b.
> > 
> > > For NFSv3 KILL signal to lockd still works but for 
> > > NFSv4 have no way to do it for v4.
> > > Marc. 
> > > 
> > > 
> > > 
> > > From:   Bruce Fields <bfields@fieldses.org>
> > > To:     Marc Eshel/Almaden/IBM@IBMUS
> > > Cc:     linux-nfs@vger.kernel.org
> > > Date:   07/01/2016 09:09 AM
> > > Subject:        Re: grace period
> > > 
> > > 
> > > 
> > > On Thu, Jun 30, 2016 at 02:46:19PM -0700, Marc Eshel wrote:
> > > > I see that setting the number of nfsd threads to 0 (echo 0 > 
> > > > /proc/fs/nfsd/threads) is not releasing the locks and putting the 
> > server 
> > > 
> > > > in grace mode.
> > > 
> > > Writing 0 to /proc/fs/nfsd/threads shuts down knfsd.  So it should
> > > certainly drop locks.  If that's not happening, there's a bug, but 
> we'd
> > > need to know more details (version numbers, etc.) to help.
> > > 
> > > That alone has never been enough to start a grace period--you'd have 

> to
> > > start knfsd again to do that.
> > > 
> > > > What is the best way to go into grace period, in new version of 
the
> > > > kernel, without restarting the nfs server?
> > > 
> > > Restarting the nfs server is the only way.  That's true on older 
> kernels
> > > true, as far as I know.  (OK, you can apparently make lockd do 
> something
> > > like this with a signal, I don't know if that's used much, and I 
doubt
> > > it works outside an NFSv3-only environment.)
> > > 
> > > So if you want locks dropped and a new grace period, then you should 

> run
> > > "systemctl restart nfs-server", or your distro's equivalent.
> > > 
> > > But you're probably doing something more complicated than that.  I'm 

> not
> > > sure I understand the question....
> > > 
> > > --b.
> > > 
> > > 
> > > 
> > > 
> > 
> > 
> > 
> > 
> 
> 
> 
> 

HA NFS

[Marc Eshel]

Hi Steve,
I did not pay attention for a while and now I see that since RHEL7.0 there 
a major changes in NFSv4 recovery for a signal machine and for cluster 
file system. Is there any write up on the changes like the use of 
/var/lib/nfs/nfsdcltrack/main.sqlite, I see it being used in 7.0 but not 
in 7.2. Any information would be appreciated.
Thanks, Marc. 

Re: HA NFS

[Steve Dickson]

On 07/04/2016 07:53 PM, Marc Eshel wrote:
> Hi Steve,
> I did not pay attention for a while and now I see that since RHEL7.0 there 
> a major changes in NFSv4 recovery for a signal machine and for cluster 
> file system. Is there any write up on the changes like the use of 
> /var/lib/nfs/nfsdcltrack/main.sqlite, I see it being used in 7.0 but not 
> in 7.2. Any information would be appreciated.

That file is still being used... but there were some changes.

In RHEL 7.2 this was added for bz 1234598
commit c41a3d0a17baa61a07d48d8536e99908d765de9b
Author: Jeff Layton <jlayton@primarydata.com>
Date:   Fri Sep 19 11:07:31 2014 -0400

    nfsdcltrack: fetch NFSDCLTRACK_GRACE_START out of environment


In RHEL 7.3 there will be this for bz 1285097
commit d479ad3adb0671c48d6fbf3e36bd52a31159c413
Author: Jeff Layton <jlayton@primarydata.com>
Date:   Fri Sep 19 11:03:45 2014 -0400

    nfsdcltrack: update schema to v2

I hope this helps... 

steved.

Re: grace period

[Bruce Fields]

On Sat, Jul 02, 2016 at 10:30:11PM -0700, Marc Eshel wrote:
> I tried again NFSv3 locks with xfs export. "echo 0 > 
> /proc/fs/nfsd/threads" releases locks on rhel7.0 but not on rhel7.2
> What else can I show you to find the problem?

Sorry, I can't reproduce, though I've only tried a slightly later kernel
than that.  Could you submit a RHEL bug?

--b.

> Marc.
>  
> works:
> [root@boar11 ~]# uname -a
> Linux boar11 3.10.0-123.el7.x86_64 #1 SMP Mon May 5 11:16:57 EDT 2014 
> x86_64 x86_64 x86_64 GNU/Linux
> [root@boar11 ~]# cat /etc/redhat-release 
> Red Hat Enterprise Linux Server release 7.0 (Maipo)
> 
> not working:
> [root@sonascl21 ~]# uname -a
> Linux sonascl21.sonasad.almaden.ibm.com 3.10.0-327.el7.x86_64 #1 SMP Thu 
> Oct 29 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux
> [root@sonascl21 ~]# cat /etc/redhat-release 
> Red Hat Enterprise Linux Server release 7.2 (Maipo)
> [root@sonascl21 ~]# cat /proc/fs/nfsd/threads 
> 0
> [root@sonascl21 ~]# cat /proc/locks
> 1: POSIX  ADVISORY  WRITE 2346 fd:00:1612092569 0 9999
> 
> 
> 
> From:   Bruce Fields <bfields@fieldses.org>
> To:     Marc Eshel/Almaden/IBM@IBMUS
> Cc:     linux-nfs@vger.kernel.org, Tomer Perry <TOMP@il.ibm.com>
> Date:   07/01/2016 05:58 PM
> Subject:        Re: grace period
> 
> 
> 
> On Fri, Jul 01, 2016 at 03:42:43PM -0700, Marc Eshel wrote:
> > Yes, the locks are requested from another node, what fs are you using, I 
> 
> > don't think it should make any difference, but I can try it with the 
> same 
> > fs. 
> > Make sure you are using v3, it does work for v4.
> 
> I tested v3 on upstream.--b.
> 
> > Marc.
> > 
> > 
> > 
> > From:   Bruce Fields <bfields@fieldses.org>
> > To:     Marc Eshel/Almaden/IBM@IBMUS
> > Cc:     linux-nfs@vger.kernel.org, Tomer Perry <TOMP@il.ibm.com>
> > Date:   07/01/2016 02:01 PM
> > Subject:        Re: grace period
> > 
> > 
> > 
> > On Fri, Jul 01, 2016 at 01:46:42PM -0700, Marc Eshel wrote:
> > > This is my v3 test that show the lock still there after echo 0 > 
> > > /proc/fs/nfsd/threads
> > > 
> > > [root@sonascl21 ~]# cat /etc/redhat-release 
> > > Red Hat Enterprise Linux Server release 7.2 (Maipo)
> > > 
> > > [root@sonascl21 ~]# uname -a
> > > Linux sonascl21.sonasad.almaden.ibm.com 3.10.0-327.el7.x86_64 #1 SMP 
> Thu 
> > 
> > > Oct 29 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux
> > > 
> > > [root@sonascl21 ~]# cat /proc/locks | grep 999
> > > 3: POSIX  ADVISORY  WRITE 2349 00:2a:489486 0 999
> > > 
> > > [root@sonascl21 ~]# echo 0 > /proc/fs/nfsd/threads
> > > [root@sonascl21 ~]# cat /proc/fs/nfsd/threads
> > > 0
> > > 
> > > [root@sonascl21 ~]# cat /proc/locks | grep 999
> > > 3: POSIX  ADVISORY  WRITE 2349 00:2a:489486 0 999
> > 
> > Huh, that's not what I see.  Are you positive that's the lock on the
> > backend filesystem and not the client-side lock (in case you're doing a
> > loopback mount?)
> > 
> > --b.
> > 
> > > 
> > > 
> > > 
> > > 
> > > From:   Bruce Fields <bfields@fieldses.org>
> > > To:     Marc Eshel/Almaden/IBM@IBMUS
> > > Cc:     linux-nfs@vger.kernel.org
> > > Date:   07/01/2016 01:07 PM
> > > Subject:        Re: grace period
> > > 
> > > 
> > > 
> > > On Fri, Jul 01, 2016 at 10:31:55AM -0700, Marc Eshel wrote:
> > > > It used to be that sending KILL signal to lockd would free locks and 
> 
> > > start 
> > > > Grace period, and when setting nfsd threads to zero, 
> > nfsd_last_thread() 
> > > > calls nfsd_shutdown that called lockd_down that I believe was 
> causing 
> > > both 
> > > > freeing of locks and starting grace period or maybe it was setting 
> it 
> > > back 
> > > > to a value > 0 that started the grace period.
> > > 
> > > OK, apologies, I didn't know (or forgot) that.
> > > 
> > > > Any way starting with the kernels that are in RHEL7.1 and up echo 0 
> > 
> > > > /proc/fs/nfsd/threads doesn't do it anymore, I assume going to 
> common 
> > > > grace period for NLM and NFSv4 changed things.
> > > > The question is how to do IP fail-over, so when a node fails and the 
> 
> > IP 
> > > is 
> > > > moving to another node, we need to go into grace period on all the 
> > nodes 
> > > 
> > > > in the cluster so the locks of the failed node are not given to 
> anyone 
> > 
> > > > other than the client that is reclaiming his locks. Restarting NFS 
> > > server 
> > > > is to distractive.
> > > 
> > > What's the difference?  Just that clients don't have to reestablish 
> tcp
> > > connections?
> > > 
> > > --b.
> > > 
> > > > For NFSv3 KILL signal to lockd still works but for 
> > > > NFSv4 have no way to do it for v4.
> > > > Marc. 
> > > > 
> > > > 
> > > > 
> > > > From:   Bruce Fields <bfields@fieldses.org>
> > > > To:     Marc Eshel/Almaden/IBM@IBMUS
> > > > Cc:     linux-nfs@vger.kernel.org
> > > > Date:   07/01/2016 09:09 AM
> > > > Subject:        Re: grace period
> > > > 
> > > > 
> > > > 
> > > > On Thu, Jun 30, 2016 at 02:46:19PM -0700, Marc Eshel wrote:
> > > > > I see that setting the number of nfsd threads to 0 (echo 0 > 
> > > > > /proc/fs/nfsd/threads) is not releasing the locks and putting the 
> > > server 
> > > > 
> > > > > in grace mode.
> > > > 
> > > > Writing 0 to /proc/fs/nfsd/threads shuts down knfsd.  So it should
> > > > certainly drop locks.  If that's not happening, there's a bug, but 
> > we'd
> > > > need to know more details (version numbers, etc.) to help.
> > > > 
> > > > That alone has never been enough to start a grace period--you'd have 
> 
> > to
> > > > start knfsd again to do that.
> > > > 
> > > > > What is the best way to go into grace period, in new version of 
> the
> > > > > kernel, without restarting the nfs server?
> > > > 
> > > > Restarting the nfs server is the only way.  That's true on older 
> > kernels
> > > > true, as far as I know.  (OK, you can apparently make lockd do 
> > something
> > > > like this with a signal, I don't know if that's used much, and I 
> doubt
> > > > it works outside an NFSv3-only environment.)
> > > > 
> > > > So if you want locks dropped and a new grace period, then you should 
> 
> > run
> > > > "systemctl restart nfs-server", or your distro's equivalent.
> > > > 
> > > > But you're probably doing something more complicated than that.  I'm 
> 
> > not
> > > > sure I understand the question....
> > > > 
> > > > --b.
> > > > 
> > > > 
> > > > 
> > > > 
> > > 
> > > 
> > > 
> > > 
> > 
> > 
> > 
> > 
> 
> 
> 
> 

Re: HA NFS

[Marc Eshel]

Thanks for the pointer Steve, I am not sure how much of the changes 
considered RedHat and how much is Linux kernel changes so if another 
mailing list is more appropriate please let me know. I now see on RHEL7.0 
that /var/lib/nfs/nfsdcltrack/main.sqlite is updated as I open a file from 
an NFS client, but going to RHEL7.2 that file is created but not updated 
with a new client open. Did something change in this area between 7.0 and 
7.2 ?
Marc.



From:   Steve Dickson <SteveD@redhat.com>
To:     Marc Eshel/Almaden/IBM@IBMUS
Cc:     linux-nfs@vger.kernel.org, Tomer Perry <TOMP@il.ibm.com>
Date:   07/05/2016 08:08 AM
Subject:        Re: HA NFS





On 07/04/2016 07:53 PM, Marc Eshel wrote:
> Hi Steve,
> I did not pay attention for a while and now I see that since RHEL7.0 
there 
> a major changes in NFSv4 recovery for a signal machine and for cluster 
> file system. Is there any write up on the changes like the use of 
> /var/lib/nfs/nfsdcltrack/main.sqlite, I see it being used in 7.0 but not 

> in 7.2. Any information would be appreciated.

That file is still being used... but there were some changes.

In RHEL 7.2 this was added for bz 1234598
commit c41a3d0a17baa61a07d48d8536e99908d765de9b
Author: Jeff Layton <jlayton@primarydata.com>
Date:   Fri Sep 19 11:07:31 2014 -0400

    nfsdcltrack: fetch NFSDCLTRACK_GRACE_START out of environment


In RHEL 7.3 there will be this for bz 1285097
commit d479ad3adb0671c48d6fbf3e36bd52a31159c413
Author: Jeff Layton <jlayton@primarydata.com>
Date:   Fri Sep 19 11:03:45 2014 -0400

    nfsdcltrack: update schema to v2

I hope this helps... 

steved.

Re: grace period

[Marc Eshel]

Can you please point me to the kernel that you are using so I can check if 
it is an obvious problem before I open an RHEL bug?
Thanks, Marc. 



From:   Bruce Fields <bfields@fieldses.org>
To:     Marc Eshel/Almaden/IBM@IBMUS
Cc:     linux-nfs@vger.kernel.org, Tomer Perry <TOMP@il.ibm.com>
Date:   07/05/2016 01:52 PM
Subject:        Re: grace period
Sent by:        linux-nfs-owner@vger.kernel.org



On Sat, Jul 02, 2016 at 10:30:11PM -0700, Marc Eshel wrote:
> I tried again NFSv3 locks with xfs export. "echo 0 > 
> /proc/fs/nfsd/threads" releases locks on rhel7.0 but not on rhel7.2
> What else can I show you to find the problem?

Sorry, I can't reproduce, though I've only tried a slightly later kernel
than that.  Could you submit a RHEL bug?

--b.

> Marc.
> 
> works:
> [root@boar11 ~]# uname -a
> Linux boar11 3.10.0-123.el7.x86_64 #1 SMP Mon May 5 11:16:57 EDT 2014 
> x86_64 x86_64 x86_64 GNU/Linux
> [root@boar11 ~]# cat /etc/redhat-release 
> Red Hat Enterprise Linux Server release 7.0 (Maipo)
> 
> not working:
> [root@sonascl21 ~]# uname -a
> Linux sonascl21.sonasad.almaden.ibm.com 3.10.0-327.el7.x86_64 #1 SMP Thu 

> Oct 29 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux
> [root@sonascl21 ~]# cat /etc/redhat-release 
> Red Hat Enterprise Linux Server release 7.2 (Maipo)
> [root@sonascl21 ~]# cat /proc/fs/nfsd/threads 
> 0
> [root@sonascl21 ~]# cat /proc/locks
> 1: POSIX  ADVISORY  WRITE 2346 fd:00:1612092569 0 9999
> 
> 
> 
> From:   Bruce Fields <bfields@fieldses.org>
> To:     Marc Eshel/Almaden/IBM@IBMUS
> Cc:     linux-nfs@vger.kernel.org, Tomer Perry <TOMP@il.ibm.com>
> Date:   07/01/2016 05:58 PM
> Subject:        Re: grace period
> 
> 
> 
> On Fri, Jul 01, 2016 at 03:42:43PM -0700, Marc Eshel wrote:
> > Yes, the locks are requested from another node, what fs are you using, 
I 
> 
> > don't think it should make any difference, but I can try it with the 
> same 
> > fs. 
> > Make sure you are using v3, it does work for v4.
> 
> I tested v3 on upstream.--b.
> 
> > Marc.
> > 
> > 
> > 
> > From:   Bruce Fields <bfields@fieldses.org>
> > To:     Marc Eshel/Almaden/IBM@IBMUS
> > Cc:     linux-nfs@vger.kernel.org, Tomer Perry <TOMP@il.ibm.com>
> > Date:   07/01/2016 02:01 PM
> > Subject:        Re: grace period
> > 
> > 
> > 
> > On Fri, Jul 01, 2016 at 01:46:42PM -0700, Marc Eshel wrote:
> > > This is my v3 test that show the lock still there after echo 0 > 
> > > /proc/fs/nfsd/threads
> > > 
> > > [root@sonascl21 ~]# cat /etc/redhat-release 
> > > Red Hat Enterprise Linux Server release 7.2 (Maipo)
> > > 
> > > [root@sonascl21 ~]# uname -a
> > > Linux sonascl21.sonasad.almaden.ibm.com 3.10.0-327.el7.x86_64 #1 SMP 

> Thu 
> > 
> > > Oct 29 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux
> > > 
> > > [root@sonascl21 ~]# cat /proc/locks | grep 999
> > > 3: POSIX  ADVISORY  WRITE 2349 00:2a:489486 0 999
> > > 
> > > [root@sonascl21 ~]# echo 0 > /proc/fs/nfsd/threads
> > > [root@sonascl21 ~]# cat /proc/fs/nfsd/threads
> > > 0
> > > 
> > > [root@sonascl21 ~]# cat /proc/locks | grep 999
> > > 3: POSIX  ADVISORY  WRITE 2349 00:2a:489486 0 999
> > 
> > Huh, that's not what I see.  Are you positive that's the lock on the
> > backend filesystem and not the client-side lock (in case you're doing 
a
> > loopback mount?)
> > 
> > --b.
> > 
> > > 
> > > 
> > > 
> > > 
> > > From:   Bruce Fields <bfields@fieldses.org>
> > > To:     Marc Eshel/Almaden/IBM@IBMUS
> > > Cc:     linux-nfs@vger.kernel.org
> > > Date:   07/01/2016 01:07 PM
> > > Subject:        Re: grace period
> > > 
> > > 
> > > 
> > > On Fri, Jul 01, 2016 at 10:31:55AM -0700, Marc Eshel wrote:
> > > > It used to be that sending KILL signal to lockd would free locks 
and 
> 
> > > start 
> > > > Grace period, and when setting nfsd threads to zero, 
> > nfsd_last_thread() 
> > > > calls nfsd_shutdown that called lockd_down that I believe was 
> causing 
> > > both 
> > > > freeing of locks and starting grace period or maybe it was setting 

> it 
> > > back 
> > > > to a value > 0 that started the grace period.
> > > 
> > > OK, apologies, I didn't know (or forgot) that.
> > > 
> > > > Any way starting with the kernels that are in RHEL7.1 and up echo 
0 
> > 
> > > > /proc/fs/nfsd/threads doesn't do it anymore, I assume going to 
> common 
> > > > grace period for NLM and NFSv4 changed things.
> > > > The question is how to do IP fail-over, so when a node fails and 
the 
> 
> > IP 
> > > is 
> > > > moving to another node, we need to go into grace period on all the 

> > nodes 
> > > 
> > > > in the cluster so the locks of the failed node are not given to 
> anyone 
> > 
> > > > other than the client that is reclaiming his locks. Restarting NFS 

> > > server 
> > > > is to distractive.
> > > 
> > > What's the difference?  Just that clients don't have to reestablish 
> tcp
> > > connections?
> > > 
> > > --b.
> > > 
> > > > For NFSv3 KILL signal to lockd still works but for 
> > > > NFSv4 have no way to do it for v4.
> > > > Marc. 
> > > > 
> > > > 
> > > > 
> > > > From:   Bruce Fields <bfields@fieldses.org>
> > > > To:     Marc Eshel/Almaden/IBM@IBMUS
> > > > Cc:     linux-nfs@vger.kernel.org
> > > > Date:   07/01/2016 09:09 AM
> > > > Subject:        Re: grace period
> > > > 
> > > > 
> > > > 
> > > > On Thu, Jun 30, 2016 at 02:46:19PM -0700, Marc Eshel wrote:
> > > > > I see that setting the number of nfsd threads to 0 (echo 0 > 
> > > > > /proc/fs/nfsd/threads) is not releasing the locks and putting 
the 
> > > server 
> > > > 
> > > > > in grace mode.
> > > > 
> > > > Writing 0 to /proc/fs/nfsd/threads shuts down knfsd.  So it should
> > > > certainly drop locks.  If that's not happening, there's a bug, but 

> > we'd
> > > > need to know more details (version numbers, etc.) to help.
> > > > 
> > > > That alone has never been enough to start a grace period--you'd 
have 
> 
> > to
> > > > start knfsd again to do that.
> > > > 
> > > > > What is the best way to go into grace period, in new version of 
> the
> > > > > kernel, without restarting the nfs server?
> > > > 
> > > > Restarting the nfs server is the only way.  That's true on older 
> > kernels
> > > > true, as far as I know.  (OK, you can apparently make lockd do 
> > something
> > > > like this with a signal, I don't know if that's used much, and I 
> doubt
> > > > it works outside an NFSv3-only environment.)
> > > > 
> > > > So if you want locks dropped and a new grace period, then you 
should 
> 
> > run
> > > > "systemctl restart nfs-server", or your distro's equivalent.
> > > > 
> > > > But you're probably doing something more complicated than that. 
I'm 
> 
> > not
> > > > sure I understand the question....
> > > > 
> > > > --b.
> > > > 
> > > > 
> > > > 
> > > > 
> > > 
> > > 
> > > 
> > > 
> > 
> > 
> > 
> > 
> 
> 
> 
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: grace period

[Bruce Fields]

On Tue, Jul 05, 2016 at 04:05:56PM -0700, Marc Eshel wrote:
> Can you please point me to the kernel that you are using so I can check if 
> it is an obvious problem before I open an RHEL bug?

I've tried it on the latest upstream and on rhel 3.10.0-327.13.1.el7.

--b.

> Thanks, Marc. 
> 
> 
> 
> From:   Bruce Fields <bfields@fieldses.org>
> To:     Marc Eshel/Almaden/IBM@IBMUS
> Cc:     linux-nfs@vger.kernel.org, Tomer Perry <TOMP@il.ibm.com>
> Date:   07/05/2016 01:52 PM
> Subject:        Re: grace period
> Sent by:        linux-nfs-owner@vger.kernel.org
> 
> 
> 
> On Sat, Jul 02, 2016 at 10:30:11PM -0700, Marc Eshel wrote:
> > I tried again NFSv3 locks with xfs export. "echo 0 > 
> > /proc/fs/nfsd/threads" releases locks on rhel7.0 but not on rhel7.2
> > What else can I show you to find the problem?
> 
> Sorry, I can't reproduce, though I've only tried a slightly later kernel
> than that.  Could you submit a RHEL bug?
> 
> --b.
> 
> > Marc.
> > 
> > works:
> > [root@boar11 ~]# uname -a
> > Linux boar11 3.10.0-123.el7.x86_64 #1 SMP Mon May 5 11:16:57 EDT 2014 
> > x86_64 x86_64 x86_64 GNU/Linux
> > [root@boar11 ~]# cat /etc/redhat-release 
> > Red Hat Enterprise Linux Server release 7.0 (Maipo)
> > 
> > not working:
> > [root@sonascl21 ~]# uname -a
> > Linux sonascl21.sonasad.almaden.ibm.com 3.10.0-327.el7.x86_64 #1 SMP Thu 
> 
> > Oct 29 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux
> > [root@sonascl21 ~]# cat /etc/redhat-release 
> > Red Hat Enterprise Linux Server release 7.2 (Maipo)
> > [root@sonascl21 ~]# cat /proc/fs/nfsd/threads 
> > 0
> > [root@sonascl21 ~]# cat /proc/locks
> > 1: POSIX  ADVISORY  WRITE 2346 fd:00:1612092569 0 9999
> > 
> > 
> > 
> > From:   Bruce Fields <bfields@fieldses.org>
> > To:     Marc Eshel/Almaden/IBM@IBMUS
> > Cc:     linux-nfs@vger.kernel.org, Tomer Perry <TOMP@il.ibm.com>
> > Date:   07/01/2016 05:58 PM
> > Subject:        Re: grace period
> > 
> > 
> > 
> > On Fri, Jul 01, 2016 at 03:42:43PM -0700, Marc Eshel wrote:
> > > Yes, the locks are requested from another node, what fs are you using, 
> I 
> > 
> > > don't think it should make any difference, but I can try it with the 
> > same 
> > > fs. 
> > > Make sure you are using v3, it does work for v4.
> > 
> > I tested v3 on upstream.--b.
> > 
> > > Marc.
> > > 
> > > 
> > > 
> > > From:   Bruce Fields <bfields@fieldses.org>
> > > To:     Marc Eshel/Almaden/IBM@IBMUS
> > > Cc:     linux-nfs@vger.kernel.org, Tomer Perry <TOMP@il.ibm.com>
> > > Date:   07/01/2016 02:01 PM
> > > Subject:        Re: grace period
> > > 
> > > 
> > > 
> > > On Fri, Jul 01, 2016 at 01:46:42PM -0700, Marc Eshel wrote:
> > > > This is my v3 test that show the lock still there after echo 0 > 
> > > > /proc/fs/nfsd/threads
> > > > 
> > > > [root@sonascl21 ~]# cat /etc/redhat-release 
> > > > Red Hat Enterprise Linux Server release 7.2 (Maipo)
> > > > 
> > > > [root@sonascl21 ~]# uname -a
> > > > Linux sonascl21.sonasad.almaden.ibm.com 3.10.0-327.el7.x86_64 #1 SMP 
> 
> > Thu 
> > > 
> > > > Oct 29 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux
> > > > 
> > > > [root@sonascl21 ~]# cat /proc/locks | grep 999
> > > > 3: POSIX  ADVISORY  WRITE 2349 00:2a:489486 0 999
> > > > 
> > > > [root@sonascl21 ~]# echo 0 > /proc/fs/nfsd/threads
> > > > [root@sonascl21 ~]# cat /proc/fs/nfsd/threads
> > > > 0
> > > > 
> > > > [root@sonascl21 ~]# cat /proc/locks | grep 999
> > > > 3: POSIX  ADVISORY  WRITE 2349 00:2a:489486 0 999
> > > 
> > > Huh, that's not what I see.  Are you positive that's the lock on the
> > > backend filesystem and not the client-side lock (in case you're doing 
> a
> > > loopback mount?)
> > > 
> > > --b.
> > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > From:   Bruce Fields <bfields@fieldses.org>
> > > > To:     Marc Eshel/Almaden/IBM@IBMUS
> > > > Cc:     linux-nfs@vger.kernel.org
> > > > Date:   07/01/2016 01:07 PM
> > > > Subject:        Re: grace period
> > > > 
> > > > 
> > > > 
> > > > On Fri, Jul 01, 2016 at 10:31:55AM -0700, Marc Eshel wrote:
> > > > > It used to be that sending KILL signal to lockd would free locks 
> and 
> > 
> > > > start 
> > > > > Grace period, and when setting nfsd threads to zero, 
> > > nfsd_last_thread() 
> > > > > calls nfsd_shutdown that called lockd_down that I believe was 
> > causing 
> > > > both 
> > > > > freeing of locks and starting grace period or maybe it was setting 
> 
> > it 
> > > > back 
> > > > > to a value > 0 that started the grace period.
> > > > 
> > > > OK, apologies, I didn't know (or forgot) that.
> > > > 
> > > > > Any way starting with the kernels that are in RHEL7.1 and up echo 
> 0 
> > > 
> > > > > /proc/fs/nfsd/threads doesn't do it anymore, I assume going to 
> > common 
> > > > > grace period for NLM and NFSv4 changed things.
> > > > > The question is how to do IP fail-over, so when a node fails and 
> the 
> > 
> > > IP 
> > > > is 
> > > > > moving to another node, we need to go into grace period on all the 
> 
> > > nodes 
> > > > 
> > > > > in the cluster so the locks of the failed node are not given to 
> > anyone 
> > > 
> > > > > other than the client that is reclaiming his locks. Restarting NFS 
> 
> > > > server 
> > > > > is to distractive.
> > > > 
> > > > What's the difference?  Just that clients don't have to reestablish 
> > tcp
> > > > connections?
> > > > 
> > > > --b.
> > > > 
> > > > > For NFSv3 KILL signal to lockd still works but for 
> > > > > NFSv4 have no way to do it for v4.
> > > > > Marc. 
> > > > > 
> > > > > 
> > > > > 
> > > > > From:   Bruce Fields <bfields@fieldses.org>
> > > > > To:     Marc Eshel/Almaden/IBM@IBMUS
> > > > > Cc:     linux-nfs@vger.kernel.org
> > > > > Date:   07/01/2016 09:09 AM
> > > > > Subject:        Re: grace period
> > > > > 
> > > > > 
> > > > > 
> > > > > On Thu, Jun 30, 2016 at 02:46:19PM -0700, Marc Eshel wrote:
> > > > > > I see that setting the number of nfsd threads to 0 (echo 0 > 
> > > > > > /proc/fs/nfsd/threads) is not releasing the locks and putting 
> the 
> > > > server 
> > > > > 
> > > > > > in grace mode.
> > > > > 
> > > > > Writing 0 to /proc/fs/nfsd/threads shuts down knfsd.  So it should
> > > > > certainly drop locks.  If that's not happening, there's a bug, but 
> 
> > > we'd
> > > > > need to know more details (version numbers, etc.) to help.
> > > > > 
> > > > > That alone has never been enough to start a grace period--you'd 
> have 
> > 
> > > to
> > > > > start knfsd again to do that.
> > > > > 
> > > > > > What is the best way to go into grace period, in new version of 
> > the
> > > > > > kernel, without restarting the nfs server?
> > > > > 
> > > > > Restarting the nfs server is the only way.  That's true on older 
> > > kernels
> > > > > true, as far as I know.  (OK, you can apparently make lockd do 
> > > something
> > > > > like this with a signal, I don't know if that's used much, and I 
> > doubt
> > > > > it works outside an NFSv3-only environment.)
> > > > > 
> > > > > So if you want locks dropped and a new grace period, then you 
> should 
> > 
> > > run
> > > > > "systemctl restart nfs-server", or your distro's equivalent.
> > > > > 
> > > > > But you're probably doing something more complicated than that. 
> I'm 
> > 
> > > not
> > > > > sure I understand the question....
> > > > > 
> > > > > --b.
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > 
> > > 
> > > 
> > > 
> > 
> > 
> > 
> > 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
> 
>