Strange NFS file glob behavior

[DANIEL K FORREST <dforrest@wisc.edu>]

I am seeing a strange glob behavior on NFS that I can't explain.

The server has 16 files, foo/bar{01..16}.

There are other files in foo/, and there are many other processes on
the client accessing files in the directory, but the mount is readonly
so the only create/delete activity is on the server, and it's all
rsync, so create file and rename file, but no file deletions.


When the 16th file is created (random order) processing is triggered
by a message from a different host that is running the rsyncs.

On the client, I run this command:

$ stat -c'%z %n' foo/bar{01..16}

And I see all 16 files.

However, if I immediately follow that command with:

$ stat -c'%z %n' foo/bar*

On rare occasions I see fewer than 16 files.

The missing files are the ones most recently created, they can be seen
by stat when explicitly enumerated, but the shell glob does not see
all of the files.  This test is for verifying a problem with a program
that is also sometimes not seeing files using readdir/glob.


How can all 16 files be seen by stat, but not by readdir/glob?


OS is CentOS 7.9.2009, 3.10.0-1127.19.1.el7.x86_64
NFS mount is version 3, readonly, nordirplus, lookupcache=pos


-- 
Daniel K. Forrest		Space Science and
dforrest@wisc.edu		Engineering Center
(608) 890 - 0558		University of Wisconsin, Madison