cephfs snapc writeback order

cephfs snapc writeback order

[Jeff Layton]

I'm taking a stab at converting ceph to use the netfs write helpers. One
thing I'm seeing though is that the kclient takes great care to write
back data to the OSDs in order of snap context age, such that an older
snapc will get written back before a newer one. With the netfs helpers,
that may not happen quite as naturally. We may end up with it trying to
flush data for a newer snapc ahead of the older one.

My question is: is that necessarily a problem? We'd be sending along the
correct info from the capsnap for each snapc, which seems like it should
be sufficient to ensure that the writes get applied correctly. If we
were to send these out of order, what would break and how?
-- 
Jeff Layton <jlayton@redhat.com>

Re: cephfs snapc writeback order

[Jeff Layton]

On Tue, 2022-06-07 at 14:04 -0500, Sage Weil wrote:
> Hi Jeff!
> 
> On Tue, Jun 7, 2022 at 1:21 PM Jeff Layton <jlayton@redhat.com> wrote:
> > I'm taking a stab at converting ceph to use the netfs write helpers.
> > One
> > thing I'm seeing though is that the kclient takes great care to
> > write
> > back data to the OSDs in order of snap context age, such that an
> > older
> > snapc will get written back before a newer one. With the netfs
> > helpers,
> > that may not happen quite as naturally. We may end up with it trying
> > to
> > flush data for a newer snapc ahead of the older one.
> > 
> > My question is: is that necessarily a problem? We'd be sending along
> > the
> > correct info from the capsnap for each snapc, which seems like it
> > should
> > be sufficient to ensure that the writes get applied correctly. If we
> > were to send these out of order, what would break and how?
> > 
> 
> 
> Writing back snapshotted data out of order would corrupt the snapshot
> content.  The OSD can only create clone objects (snapshotted data)
> from the live/head/most recent version of the object it has, which
> means that a newer write followed by an older one will apply the old
> data to the newer object.  Exactly how that shakes out depends on
> where the writes are relative to object boundaries (the cloning on the
> OSD is by object) and how the writes were ordered relative to the
> snapshots, but in any case the end result will not be correct.
> 

Got it, thanks. Also thanks to Greg for his response.

> :( Hopefully that doesn't make the netfs transition unworkable!
> 


No, it doesn't make the transition unworkable, but we'll need to
implement something in the netfs layer to ensure that things get written
back in the correct order. David and I are still discussing what to do,
but it should be doable.

-- 
Jeff Layton <jlayton@redhat.com>