Re: pnfs setup on Linux 4.15

["Benjamin Coddington" <bcodding@redhat.com>]

Hi jrk,

The upstream linux knfsd server currently only supports a very simple 
flexfiles layout where the MDS and the DS are the same server, so 
there's no way (yet) to configure knfsd to give out flexfiles layouts 
that point to other DS servers.

See commit 9b9960a0ca4773e21c4b153ed355583946346b25 in the linux git 
repo for the work that implements this simple server.

Ben

On 6 Mar 2019, at 5:39, Kanika wrote:

> Hi,
> I am trying to deploy pnfs using the mainstream-ed NFSv4 server/client
> available with 4.15 kernels. I have been able to setup the MDS and
> client using the flexfile layout. I can't find any documentation on
> how to inform MDS about the data servers.
>
> The setup I have so far:
> Metadata server and data server are running Ubuntu 18.04
> Client is Centos 7.5
>
> MDS (intended), ip: 192.168.122.92
> root@ubuntu1804:~/#cat /etc/exports
> /sudosrv *(rw,sync,fsid=0,no_subtree_check,no_root_squash,pnfs) <-- 
> PNFS set
> /sudosrv/share1 *(rw,sync,fsid=1,no_subtree_check,no_root_squash)
>
> Intended DS, ip: 192.168.122.83
> root@ubuntu18_04_2:~#cat /etc/exports
> /srv/homes *(rw,sync,fsid=4,no_root_squash,no_subtree_check)
>
> Client, ip:192.168.122.5 (Centos 7.5)
> Mount the pseudo root filesystem exported by MDS
>
> #mount -t nfs -o v4.2 -o rw 192.168.122.92:/ /mnt/
> #cat /proc/self/mountstats
>
> device nfsd mounted on /proc/fs/nfsd with fstype nfsd
> device 192.168.122.92:/ mounted on /mnt with fstype nfs4 statvers=1.1
> opts: 
> rw,vers=4.2,rsize=524288,wsize=524288,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.122.5,local_lock=none
> age: 127
> impl_id: name='',domain='',date='0,0'
> caps: caps=0x1fbffdf,wtmult=512,dtsize=32768,bsize=0,namlen=255
> nfsv4: 
> bm0=0xfdffbfff,bm1=0x40f9be3e,bm2=0x20803,acl=0x3,sessions,pnfs=LAYOUT_FLEX_FILES
>  <--- flex file layout
> sec: flavor=1,pseudoflavor=1
>
> I tried 2 ways to link MDS and DS
> 1. Use the "refer" while exporting the DS share from MDS. This works
> as referrals are expected to work but not like "pnfs".
> 2. Mount the DS on MDS in a subdir of /sudosrv/share1. All IO
> operations go only to the MDS as if it were a local share. No exchange
> between the client and the DS.
>
> How can the DS be known to an MDS for a local filesystem like ext4?
> Any help will be appreciated.
>
> Thanks,
> jrk