NFS stale filehandles

NFS stale filehandles

[Brian Chrisman]

There seems to be a bug in the ceph kernel client NFS implementation.
Triage:  using the same cluster/servers and client
1) export ceph filesystem via NFS, mount on client, copy over files
and build --> stale filehandle/corruption issues
2) export ext filesystem via NFS (same server node as in #1), mount on
client, copy over files and build --> no problem
3) mount ceph locally on build client, copy over files and build --> no problem

From this, it seems most likely that the problem lies in the kernel
client NFS export code.

Kernel 2.6.37rc8
Mounting via NFSv4 (noticed other issues with directories disappearing
when previously testing with NFSv3 that I can retest after figuring
this out)
Other than kernel, systems are a rhel6 beta distro
I wanted to check whether/what testing has been done of the export
code thus far, and see whether this is a known issue.

Re: NFS stale filehandles

[Brian Chrisman]

Ahh from the module source... I'll add this to the wiki page:
http://ceph.newdream.net/wiki/Re-exporting_NFS

/*
 * NFS export support
 *
 * NFS re-export of a ceph mount is, at present, only semireliable.
 * The basic issue is that the Ceph architectures doesn't lend itself
 * well to generating filehandles that will remain valid forever.
 *
 * So, we do our best.  If you're lucky, your inode will be in the
 * client's cache.  If it's not, and you have a connectable fh, then
 * the MDS server may be able to find it for you.  Otherwise, you get
 * ESTALE.
 *
 * There are ways to this more reliable, but in the non-connectable fh
 * case, we won't every work perfectly, and in the connectable case,
 * some changes are needed on the MDS side to work better.
 */




On Wed, Feb 2, 2011 at 10:52 AM, Brian Chrisman <brchrisman@gmail.com> wrote:
> There seems to be a bug in the ceph kernel client NFS implementation.
> Triage:  using the same cluster/servers and client
> 1) export ceph filesystem via NFS, mount on client, copy over files
> and build --> stale filehandle/corruption issues
> 2) export ext filesystem via NFS (same server node as in #1), mount on
> client, copy over files and build --> no problem
> 3) mount ceph locally on build client, copy over files and build --> no problem
>
> From this, it seems most likely that the problem lies in the kernel
> client NFS export code.
>
> Kernel 2.6.37rc8
> Mounting via NFSv4 (noticed other issues with directories disappearing
> when previously testing with NFSv3 that I can retest after figuring
> this out)
> Other than kernel, systems are a rhel6 beta distro
> I wanted to check whether/what testing has been done of the export
> code thus far, and see whether this is a known issue.
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html