cephfs inode size handling and inode_drop field in struct MetaRequest

cephfs inode size handling and inode_drop field in struct MetaRequest

[Jeff Layton]

I'm working on a F_SETLEASE implementation for kcephfs, and am hitting a
deadlock of sorts, due to a truncate triggering a cap revoke at an
inopportune time.

The issue is that truncates to a smaller size are always done via 
synchronous call to the MDS, whereas a truncate larger does not if Fx
caps are held. That synchronous call causes the MDS to issue the client
a cap revoke for caps that the lease holds references on (Frw, in
particular). 

The client code has been this way since the inception and I haven't been
able to locate any rationale for it. Some questions about this:

1) Why doesn't the client ever buffer a truncate to smaller size? It
seems like that is something that could be done without a synchronous
MDS call if we hold Fx caps.

2) The client setattr implementations set inode_drop values in the
MetaRequest, but as far as I can tell, those values end up being ignored
by the MDS. What purpose does inode_drop actually serve? Is this field
vestigial?

Thanks,
-- 
Jeff Layton <jlayton@redhat.com>

Re: cephfs inode size handling and inode_drop field in struct MetaRequest

[Jeff Layton]

On Thu, 2020-10-29 at 11:19 -0400, Jeff Layton wrote:
> I'm working on a F_SETLEASE implementation for kcephfs, and am hitting a
> deadlock of sorts, due to a truncate triggering a cap revoke at an
> inopportune time.
> 
> The issue is that truncates to a smaller size are always done via 
> synchronous call to the MDS, whereas a truncate larger does not if Fx
> caps are held. That synchronous call causes the MDS to issue the client
> a cap revoke for caps that the lease holds references on (Frw, in
> particular). 
> 
> The client code has been this way since the inception and I haven't been
> able to locate any rationale for it. Some questions about this:
> 
> 1) Why doesn't the client ever buffer a truncate to smaller size? It
> seems like that is something that could be done without a synchronous
> MDS call if we hold Fx caps.
> 
> 2) The client setattr implementations set inode_drop values in the
> MetaRequest, but as far as I can tell, those values end up being ignored
> by the MDS. What purpose does inode_drop actually serve? Is this field
> vestigial?


I think I answered the second question myself. It _is_ potentially used
to encoded a cap release into the call. That's not happening here
because of the extra references held by the lease. I think I see a
couple of potential fixes for that problem.

I think the first question is still valid though.
-- 
Jeff Layton <jlayton@redhat.com>

Re: cephfs inode size handling and inode_drop field in struct MetaRequest

[Yan, Zheng]

On Fri, Oct 30, 2020 at 12:53 AM Jeff Layton <jlayton@redhat.com> wrote:
>
> On Thu, 2020-10-29 at 11:19 -0400, Jeff Layton wrote:
> > I'm working on a F_SETLEASE implementation for kcephfs, and am hitting a
> > deadlock of sorts, due to a truncate triggering a cap revoke at an
> > inopportune time.
> >
> > The issue is that truncates to a smaller size are always done via
> > synchronous call to the MDS, whereas a truncate larger does not if Fx
> > caps are held. That synchronous call causes the MDS to issue the client
> > a cap revoke for caps that the lease holds references on (Frw, in
> > particular).
> >
> > The client code has been this way since the inception and I haven't been
> > able to locate any rationale for it. Some questions about this:
> >
> > 1) Why doesn't the client ever buffer a truncate to smaller size? It
> > seems like that is something that could be done without a synchronous
> > MDS call if we hold Fx caps.
> >

because we need to increate truncate_seq. truncate_seq makes OSD drop/adjust
write operation that were sent before the truncate.

> > 2) The client setattr implementations set inode_drop values in the
> > MetaRequest, but as far as I can tell, those values end up being ignored
> > by the MDS. What purpose does inode_drop actually serve? Is this field
> > vestigial?
>
>
> I think I answered the second question myself. It _is_ potentially used
> to encoded a cap release into the call. That's not happening here
> because of the extra references held by the lease. I think I see a
> couple of potential fixes for that problem.
>
> I think the first question is still valid though.
> --
> Jeff Layton <jlayton@redhat.com>
>