linux NFS client lock file cannot get a expected response

linux NFS client lock file cannot get a expected response

[Gefei Li]

Hi NFSDevs,

I am recently testing linux nfs lock with NFS share from a WinServer
2016. I tried to write a file which has already been locked with fcntl
exclusively, but the response of `write` syscall is neither
`Permission denied`, nor successfully written with file content
changed. Here is several experiments I did:

The first shell runs c program, calling `fcntl` to lock file
exclusively “fcntl(fd, F_SETLKW, &fl)”

2.1 The second shell tried to open with flag O_RDWR, and write a
buffer to the same file, write returned the correct bytes written, but
the file content remained unchanged.
2.2 The second shell tried to open with flag O_RDWR|O_SYNC, write the
same buffer to the same file, write operation returned EAGAIN
2.3 The same operation on ext4 file system gives me a expected
behavior the same as advisory lock expressed, successfully written
with file content changed.

I reviewed NFS 4.1 protocol(RFC 5661 page 185), the nfs server can
determine whether byte range lock can be either mandatory or advisory,
but I think 2.1 and 2.2 gives me some unexpected behavior as these
two.. What’s your idea about this? Can you give me some tips to work
this out?

Looking forward for your reply. Thanks in advance!

BTW, my linux kernel version is 4.15.0, linux release: ubuntu 16.04
from Azure marketplace, my nfs-common version is
"nfs-common/xenial-updates,now 1:1.2.8-9ubuntu12.1 amd64" from ubuntu
apt repo.

Thanks,
Gefei

Re: linux NFS client lock file cannot get a expected response

[J. Bruce Fields]

On Mon, Feb 18, 2019 at 05:11:47PM +0800, Gefei Li wrote:
> I am recently testing linux nfs lock with NFS share from a WinServer
> 2016. I tried to write a file which has already been locked with fcntl
> exclusively, but the response of `write` syscall is neither
> `Permission denied`, nor successfully written with file content
> changed. Here is several experiments I did:
> 
> The first shell runs c program, calling `fcntl` to lock file
> exclusively “fcntl(fd, F_SETLKW, &fl)”
> 
> 2.1 The second shell tried to open with flag O_RDWR, and write a
> buffer to the same file, write returned the correct bytes written, but
> the file content remained unchanged.

You're checking the file content on the server somehow?

The client's caching the write--it returns success to the application
and then sends the actual write to the server later.  Anything else will
hurt performance of a lot of applications.

> 2.2 The second shell tried to open with flag O_RDWR|O_SYNC, write the
> same buffer to the same file, write operation returned EAGAIN
> 2.3 The same operation on ext4 file system gives me a expected
> behavior the same as advisory lock expressed, successfully written
> with file content changed.
> 
> I reviewed NFS 4.1 protocol(RFC 5661 page 185), the nfs server can
> determine whether byte range lock can be either mandatory or advisory,
> but I think 2.1 and 2.2 gives me some unexpected behavior as these
> two.. What’s your idea about this? Can you give me some tips to work
> this out?

There's also section 10.3.3, but I never understood how that was really
supposed to work.  It recommends the client turn off caching when
mandatory locking is in effect--but the only way it gives the client
that mandatory locking is in effect is by seeing an ERR_LOCKED reply to
a READ or WRITE, and by that point it's too late.

Anyway, as far as I can tell the results you report are what's expected.

--b.

> Looking forward for your reply. Thanks in advance!
> 
> BTW, my linux kernel version is 4.15.0, linux release: ubuntu 16.04
> from Azure marketplace, my nfs-common version is
> "nfs-common/xenial-updates,now 1:1.2.8-9ubuntu12.1 amd64" from ubuntu
> apt repo.
> 
> Thanks,
> Gefei

Re: linux NFS client lock file cannot get a expected response

[Gefei Li]

Thanks a lot for your reply! I tried to capture and analyse the
network packets with wireshark and find that a NFS4ERR_LOCKED(10012)
replied in my experiment 2.2(open with O_SYNC and write) Write
response. Does this mean that my NFS server uses mandatory locking?

But something unexpected is that an EAGAIN(Resource temporarily
unavailable) returned in user space, I think an EACCES/EIO is better
for locked situations, what's your idea about this?

>> You're checking the file content on the server somehow?
I close my lock program in my first shell, and check from both
server-side(use notepad to view the file) and client-side(cat/vim to
view the file), the file content remained unchanged.

Best Regards,
Gefei

On Tue, Feb 19, 2019 at 4:22 AM J. Bruce Fields <bfields@fieldses.org> wrote:
>
> On Mon, Feb 18, 2019 at 05:11:47PM +0800, Gefei Li wrote:
> > I am recently testing linux nfs lock with NFS share from a WinServer
> > 2016. I tried to write a file which has already been locked with fcntl
> > exclusively, but the response of `write` syscall is neither
> > `Permission denied`, nor successfully written with file content
> > changed. Here is several experiments I did:
> >
> > The first shell runs c program, calling `fcntl` to lock file
> > exclusively “fcntl(fd, F_SETLKW, &fl)”
> >
> > 2.1 The second shell tried to open with flag O_RDWR, and write a
> > buffer to the same file, write returned the correct bytes written, but
> > the file content remained unchanged.
>
> You're checking the file content on the server somehow?
>
> The client's caching the write--it returns success to the application
> and then sends the actual write to the server later.  Anything else will
> hurt performance of a lot of applications.
>
> > 2.2 The second shell tried to open with flag O_RDWR|O_SYNC, write the
> > same buffer to the same file, write operation returned EAGAIN
> > 2.3 The same operation on ext4 file system gives me a expected
> > behavior the same as advisory lock expressed, successfully written
> > with file content changed.
> >
> > I reviewed NFS 4.1 protocol(RFC 5661 page 185), the nfs server can
> > determine whether byte range lock can be either mandatory or advisory,
> > but I think 2.1 and 2.2 gives me some unexpected behavior as these
> > two.. What’s your idea about this? Can you give me some tips to work
> > this out?
>
> There's also section 10.3.3, but I never understood how that was really
> supposed to work.  It recommends the client turn off caching when
> mandatory locking is in effect--but the only way it gives the client
> that mandatory locking is in effect is by seeing an ERR_LOCKED reply to
> a READ or WRITE, and by that point it's too late.
>
> Anyway, as far as I can tell the results you report are what's expected.
>
> --b.
>
> > Looking forward for your reply. Thanks in advance!
> >
> > BTW, my linux kernel version is 4.15.0, linux release: ubuntu 16.04
> > from Azure marketplace, my nfs-common version is
> > "nfs-common/xenial-updates,now 1:1.2.8-9ubuntu12.1 amd64" from ubuntu
> > apt repo.
> >
> > Thanks,
> > Gefei

Re: linux NFS client lock file cannot get a expected response

[J. Bruce Fields]

On Tue, Feb 19, 2019 at 10:09:25AM +0800, Gefei Li wrote:
> Thanks a lot for your reply! I tried to capture and analyse the
> network packets with wireshark and find that a NFS4ERR_LOCKED(10012)
> replied in my experiment 2.2(open with O_SYNC and write) Write
> response. Does this mean that my NFS server uses mandatory locking?

Yes.  I think that's expected for Windows.

> But something unexpected is that an EAGAIN(Resource temporarily
> unavailable) returned in user space, I think an EACCES/EIO is better
> for locked situations, what's your idea about this?

I don't know.

EAGAIN is what fcntl returns when there's a conflicting lock.  It
suggests that retrying might work, while EIO and EACCES suggest that the
failure is permanent.

In the case of linux mandatory locks (see the "mandatory locking"
section in fcntl(2)), read and write block unless you open with
O_NONBLOCK, in which case they return EAGAIN.  Blocking would be more
complicated for the NFS client as all it could do would be to retry the
write repeatedly.

> >> You're checking the file content on the server somehow?
> I close my lock program in my first shell, and check from both
> server-side(use notepad to view the file) and client-side(cat/vim to
> view the file), the file content remained unchanged.

Got it.

--b.