pNFS/Flexfiles testing at Oct. Bakeathon

pNFS/Flexfiles testing at Oct. Bakeathon

[Rick Macklem]

Hi,

Tom has a few IETF drafts describing changes that I believe are
aimed at improving pNFS Flex Files layout.

I was wondering if implementations of any of these will be
available at the Oct. Bakeathon?

I have not implemented any of them as yet, but I might have
time to do an experimental implementation of some of them
for the Oct. Bakeathon, if there will be implementations to test
against.

Thanks for any info, rick

Re: [nfsv4] pNFS/Flexfiles testing at Oct. Bakeathon

[Rick Macklem]

On Mon, Aug 28, 2023 at 7:03 PM Tom Haynes <loghyr@gmail.com> wrote:
>
> Hi Rick,
>
> I'll have server implementations of all of them.
>
> Wait, maybe not the WCC.
I was thinking that WCC was the interesting one. The FreeBSD pNFS server uses
v4.2 DSs, so it would be client side only for FreeBSD. (It's too bad
that NFSv4 DSs
cannot do this, but I cannot think how they could unless a new
Write-with-WCC operation
was added to NFSv4.2.)

Will you know soon if you'd have a server for the Bakeathon?

>
> My clients will also have the implementations, normally they only talk to my server, but I can bring a standalone client for testing.
The FreeBSD pNFS server never issues delegations, so delstid wouldn't
really apply.
(The non-pNFS server does do delegations. Is delstid generic enough
that a non-pNFS
server should do it?)

The only other one I recall is the LayoutReturn with an error after the MDS
reboot. Not something we'd want to test at the Bakeathon, I'd guess?

Thanks for the info, rick

>
> Tom
>
> On Mon, Aug 28, 2023, 18:40 Rick Macklem <rick.macklem@gmail.com> wrote:
>>
>> Hi,
>>
>> Tom has a few IETF drafts describing changes that I believe are
>> aimed at improving pNFS Flex Files layout.
>>
>> I was wondering if implementations of any of these will be
>> available at the Oct. Bakeathon?
>>
>> I have not implemented any of them as yet, but I might have
>> time to do an experimental implementation of some of them
>> for the Oct. Bakeathon, if there will be implementations to test
>> against.
>>
>> Thanks for any info, rick
>>
>> _______________________________________________
>> nfsv4 mailing list
>> nfsv4@ietf.org
>> https://www.ietf.org/mailman/listinfo/nfsv4

Re: [nfsv4] pNFS/Flexfiles testing at Oct. Bakeathon

[Chris Inacio]

[-- Attachment #1: Type: text/plain, Size: 802 bytes --]

> 
>> 
>> My clients will also have the implementations, normally they only talk to my server, but I can bring a standalone client for testing.
> The FreeBSD pNFS server never issues delegations, so delstid wouldn't
> really apply.
> (The non-pNFS server does do delegations. Is delstid generic enough
> that a non-pNFS
> server should do it?)
> 

[ci] For the noobs in the audience, if you don’ t mind, why doesn’t FreeBSD ever offer delegations?  (I’m getting just smart enough to be a bit dangerous.  Give me a few more months and I’ll really be able to ruin something…)


> The only other one I recall is the LayoutReturn with an error after the MDS
> reboot. Not something we'd want to test at the Bakeathon, I'd guess?
> 
> Thanks for the info, rick
> 
>> 
>> Tom


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 4057 bytes --]

Re: [nfsv4] pNFS/Flexfiles testing at Oct. Bakeathon

[Rick Macklem]

On Tue, Aug 29, 2023 at 6:24 PM Chris Inacio <inacio@cert.org> wrote:
>
> >
> >>
> >> My clients will also have the implementations, normally they only talk to my server, but I can bring a standalone client for testing.
> > The FreeBSD pNFS server never issues delegations, so delstid wouldn't
> > really apply.
> > (The non-pNFS server does do delegations. Is delstid generic enough
> > that a non-pNFS
> > server should do it?)
> >
>
> [ci] For the noobs in the audience, if you don’ t mind, why doesn’t FreeBSD ever offer delegations?  (I’m getting just smart enough to be a bit dangerous.  Give me a few more months and I’ll really be able to ruin something…)
Good question, although I am not sure I have a good answer...
First off, implementing delegations isn't easy and for a long time
the FreeBSD server probably had bugs. (Delegations are more difficult
for NFSv4.0, since the server cannot easily determine if the client RPC
is from the client that holds the delegation.)
In my opinion, delegations can do two things:
1 - Allow a file to be Open'd locally in the client. For the FreeBSD client
     this has never seemed very useful, since apps. do not normally
      open/close/open/close... the same file.
2 - Allow for improved caching. I actually have a very bit rotted patch
     that did extensive file caching on non-volatile storage (the whole
     file for small files) when it held a delegation for the file.
     This code was never completed, so the effects of a delegation
     on caching in the FreeBSD client is minimal.
Without #2, I do not think delegations are worth the complexity/effort
to do them.  As such, the FreeBSD server does not have them enabled
by default (they can be anabled via a tunable). Historically this was
because of bugs/interoperability problems. More recently, mostly just
do not seem useful.

For the specific case of the pNFS server (which almost no one uses),
a problem was detected at a Bakeathon last year. The client was one
that I do not normally get to test against. The problem was related to
file removal when a delegation was issued to the client (I think it was
4.0, but cannot remember for sure?).
--> I cannot recall the details, but the only obvious fix where there was
     time to test during the Bakeathon was disabling delegations.
     --> As such, I just made the tunable to enable delegations not
          usable for the pNFS server case.
(My vague recollection was that the problem was vaguely similar to
 the Linux knfsd problem for a 4.0 client with a write delegation, in that
 it was difficult to both recall the delegation and remove the data on the DS
 correctly. But, I'll admit I cannot recall exactly what it was.)

rick
ps: It does offer delegations for the non-pNFS server if the tunable is used
      to turn them on.

>
>
> > The only other one I recall is the LayoutReturn with an error after the MDS
> > reboot. Not something we'd want to test at the Bakeathon, I'd guess?
> >
> > Thanks for the info, rick
> >
> >>
> >> Tom
>