extent generation_id

extent generation_id

[Ganesh Sangle]

hi,
i am new to the email list - and i have a question. Please let me know
if this is not the right forum for this question.

While iterating the extents for a subvolume, btrfs_file_extent_item
returns a generation_id - which is the "transaction id that created
this extent".
Is it safe to assume that every pwrite syscall will endup a generating
new generation id for an extent, regardless if it is over-writing an
existing extent (offset) or writing to a new never written (offset) ?
In other words, can we assume that if we have generation id associated
with all extents of a snapshot (S1) of a volume, then we delete this
snapshot, and then do some i/o(write/discard) to the volume and create
a new snapshot (S2) from this volume, if we iterate over the extents
of this new snapshot (S2) we will see a different generation id
(compared to the one we got from the snapshot (S1)) if the i/o
(write/discard) happened at an offset in that extent ?

Thanks for the help !

Re: extent generation_id

[Qu Wenruo]

[-- Attachment #1.1: Type: text/plain, Size: 2135 bytes --]



On 2020/3/25 上午7:01, Ganesh Sangle wrote:
> hi,
> i am new to the email list - and i have a question. Please let me know
> if this is not the right forum for this question.
> 
> While iterating the extents for a subvolume, btrfs_file_extent_item
> returns a generation_id - which is the "transaction id that created
> this extent".
> Is it safe to assume that every pwrite syscall will endup a generating
> new generation id for an extent, regardless if it is over-writing an
> existing extent (offset) or writing to a new never written (offset) ?

For data cow (the default behavior), all data writes, no matter buffered
(including pwrite) or direct IO, will lead to new extents, thus a new
generation.

Furthermore, even if we overwrite the whole existing extent, we won't
write directly into the existing extent, we will keep the existing
extent until current transaction is committed.
This is proper COW behavior, to ensure even we had a power loss, we will
either see the old data (and metadata) or new data (and metadata)

For no data cow case, overwriting existing extent won't update the
extent item, thus not update the generation (which skips some metadata
IO, and improves performance).

But no data cow case is limited in btrfs, even a file has NODATACOW bit
set, if there is a snapshot referring to that file, we will still do COW.

> In other words, can we assume that if we have generation id associated
> with all extents of a snapshot (S1) of a volume, then we delete this
> snapshot, and then do some i/o(write/discard) to the volume and create
> a new snapshot (S2) from this volume, if we iterate over the extents
> of this new snapshot (S2) we will see a different generation id
> (compared to the one we got from the snapshot (S1)) if the i/o
> (write/discard) happened at an offset in that extent ?

Since the write happens after S1 is deleted, there is no guarantee that
CoW will still happen for NODATACOW files.
Thus we can overwrite existing extents, thus it's possible that no new
generation id generated.

Thanks,
Qu

> 
> Thanks for the help !
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: extent generation_id

[Ganesh Sangle]

Thank you for the confirmation and explanation.

On Tue, Mar 24, 2020 at 5:12 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>
>
>
> On 2020/3/25 上午7:01, Ganesh Sangle wrote:
> > hi,
> > i am new to the email list - and i have a question. Please let me know
> > if this is not the right forum for this question.
> >
> > While iterating the extents for a subvolume, btrfs_file_extent_item
> > returns a generation_id - which is the "transaction id that created
> > this extent".
> > Is it safe to assume that every pwrite syscall will endup a generating
> > new generation id for an extent, regardless if it is over-writing an
> > existing extent (offset) or writing to a new never written (offset) ?
>
> For data cow (the default behavior), all data writes, no matter buffered
> (including pwrite) or direct IO, will lead to new extents, thus a new
> generation.
>
> Furthermore, even if we overwrite the whole existing extent, we won't
> write directly into the existing extent, we will keep the existing
> extent until current transaction is committed.
> This is proper COW behavior, to ensure even we had a power loss, we will
> either see the old data (and metadata) or new data (and metadata)
>
> For no data cow case, overwriting existing extent won't update the
> extent item, thus not update the generation (which skips some metadata
> IO, and improves performance).
>
> But no data cow case is limited in btrfs, even a file has NODATACOW bit
> set, if there is a snapshot referring to that file, we will still do COW.
>
> > In other words, can we assume that if we have generation id associated
> > with all extents of a snapshot (S1) of a volume, then we delete this
> > snapshot, and then do some i/o(write/discard) to the volume and create
> > a new snapshot (S2) from this volume, if we iterate over the extents
> > of this new snapshot (S2) we will see a different generation id
> > (compared to the one we got from the snapshot (S1)) if the i/o
> > (write/discard) happened at an offset in that extent ?
>
> Since the write happens after S1 is deleted, there is no guarantee that
> CoW will still happen for NODATACOW files.
> Thus we can overwrite existing extents, thus it's possible that no new
> generation id generated.
>
> Thanks,
> Qu
>
> >
> > Thanks for the help !
> >
>