layoutcommit space reservation for the objlayout driver

layoutcommit space reservation for the objlayout driver

[Christoph Hellwig]

Currently the NFS client does not reserved any space for the layoutupdate
field when encoding a layoutcommit.  For the file layout driver which doesn't
have a layoutupdate that's obviously fine, and for the block layoutdriver
I just fixed the issue in my series by allowing to split in a set of
preallocated pages.  The objlayout driver still needs this fixed, either
by convincing Trond to increase the reservatation for the few objlayout
fields, or by splicing in the pages like the block layout driver does.

Can someone with an objlayout setup take care of this?

Re: layoutcommit space reservation for the objlayout driver

[Boaz Harrosh]

On 09/14/2014 07:46 PM, Christoph Hellwig wrote:
> Currently the NFS client does not reserved any space for the layoutupdate
> field when encoding a layoutcommit.  For the file layout driver which doesn't
> have a layoutupdate that's obviously fine, and for the block layoutdriver
> I just fixed the issue in my series by allowing to split in a set of
> preallocated pages.  The objlayout driver still needs this fixed, either
> by convincing Trond to increase the reservatation for the few objlayout
> fields, or by splicing in the pages like the block layout driver does.
> 
> Can someone with an objlayout setup take care of this?
> 

I do not understand. This used to work just fine. Please point me to
the patch that broke it?
(And since when are we allowed to brake old code?)

It will take me a bit of time to get up to speed. Do you want to attempt
a patch and I'll test it?

Thanks
Boaz

Re: layoutcommit space reservation for the objlayout driver

[Christoph Hellwig]

On Mon, Sep 15, 2014 at 12:05:55PM +0300, Boaz Harrosh wrote:
> I do not understand. This used to work just fine. Please point me to
> the patch that broke it?
> (And since when are we allowed to brake old code?)

I bet it still works the same way it does as commited, which doesn't
make it right.  The way the NFS XDR buffer allocations work seems to be
that you can actually slightly overun it without things falling apart,
but that doesn't make it right.  E.g. it took a fairly larger overrun
in the block layout driver to finally find the issues, and most of the
time it was just silent truncations, real corruption happened less
often.

But to make sure things work properly in the long run you'd really want
to have the proper allocation.

Re: layoutcommit space reservation for the objlayout driver

[Boaz Harrosh]

On 09/15/2014 08:12 PM, Christoph Hellwig wrote:
> On Mon, Sep 15, 2014 at 12:05:55PM +0300, Boaz Harrosh wrote:
>> I do not understand. This used to work just fine. Please point me to
>> the patch that broke it?
>> (And since when are we allowed to brake old code?)
> 
> I bet it still works the same way it does as commited, which doesn't
> make it right.  The way the NFS XDR buffer allocations work seems to be
> that you can actually slightly overun it without things falling apart,
> but that doesn't make it right.  E.g. it took a fairly larger overrun
> in the block layout driver to finally find the issues, and most of the
> time it was just silent truncations, real corruption happened less
> often.
> 
> But to make sure things work properly in the long run you'd really want
> to have the proper allocation.
> 

Thanks Christoph for explaining. OK make sense what you said. Actually I
know why it works. You know that extra page we have at nfs-xdr for if
a struct spans two pages than the xdr code will memcpy it it a spare
buffer contiguous and return that. Ha, no, that is for decoding.

But I bet for encoding there is spare buffers as well.

I will try to push for a two 64bit growth at the core reservation for
pnf-obj. No point in adding a full page for extra 16 bytes. I bet it
will not change anything because of the reserved space.

I'm not familiar with this code, I'll try to find where
core reservation is allocated for layout update?

Thanks
Boaz

Re: layoutcommit space reservation for the objlayout driver

[Christoph Hellwig]

On Mon, Sep 15, 2014 at 02:22:33PM -0400, Daniel Gryniewicz wrote:
> This is probably the case, then.  The object layout layoutupdate is only 16 bytes long and is fixed size, so it wouldn't take much extra space for it to never hit.

Yes, I suspect at the moment we just overallocate the XDR buffer due
to slab or nfs implementation details, which are bound to chance as some
point without notice.

Re: layoutcommit space reservation for the objlayout driver

[Daniel Gryniewicz]

----- "Christoph Hellwig" <hch@lst.de> wrote:

> On Mon, Sep 15, 2014 at 12:05:55PM +0300, Boaz Harrosh wrote:
> > I do not understand. This used to work just fine. Please point me
> to
> > the patch that broke it?
> > (And since when are we allowed to brake old code?)
> 
> I bet it still works the same way it does as commited, which doesn't
> make it right.  The way the NFS XDR buffer allocations work seems to
> be
> that you can actually slightly overun it without things falling
> apart,
> but that doesn't make it right.  E.g. it took a fairly larger overrun
> in the block layout driver to finally find the issues, and most of
> the
> time it was just silent truncations, real corruption happened less
> often.
> 
> But to make sure things work properly in the long run you'd really
> want
> to have the proper allocation.

This is probably the case, then.  The object layout layoutupdate is only 16 bytes long and is fixed size, so it wouldn't take much extra space for it to never hit.

Daniel