How do I set up pNFS?

["Chan, Ewen (E.)" <echan17@ford.com>]

To Whom It May Concern:

Here is what I am looking to do:

I have a small cluster consisting of four dual socket Xeon nodes and each node has 128 GB of RAM.

What I want to do is create a 64 GB tmpfs mount point on all four nodes and then export it using pNFS (NFS either version 4.1 or 4.2) so that all four nodes would "see" a 256 GB tmpfs mount point (4*64 GB) that is then shared acrossed all four nodes.

(I also have an Mellanox ConnectX-4 dual port 100 Gbps 4x EDR Infiniband adapter, so bandwidth shouldn't be an issue for me).

In my /etc/exports, I've already added proto=rdma, port=20049 to the mount point export options.

In doing some of my research on here: http://wiki.linux-nfs.org/wiki/index.php/Main_Page, it gives the example that the server appears to be just a single server and that pNFS is just for multiple clients to access the same NFS export mount point rather than having multiple servers contribute its own portion of the space to the shared, scalable NFS mount point.

If I want to do something like this, how would I create it such that the exported mount point and the client mount point would know that it is a distributed, shared pNFS pool/mount point?

i.e. would all four nodes be both pNFS (NFS v4.1 or v4.2) server AND client, such that each node will have a line in /etc/exports and it would also have the corresponding line to mount it in /etc/fstab?

Will having all four nodes add towards the pNFS mount point or will pNFS get confused because there are four NFS servers trying to be the NFS server simultaneously?

I hope that my question makes sense.

Your help in this matter is greatly appreciated.

Thank you.

Ewen Chan
CX482/483 FE Engineer
Ford Motor Company