when does btrfs create sparse extents?

when does btrfs create sparse extents?

[Marek Behun]

Hello,

there was a bug fixed recently in U-Boot's btrfs driver - the driver
failed to read files with sparse extents. This causes that sometimes
device failes to boot Linux, since the kernel fails to load from
storage.

So when does kernel's btrfs driver write sparse extents? Is it always
when it finds a PAGE_SIZEd and aligned all-zeros block? Or is it more
complicated?

Thanks.

Marek

Re: when does btrfs create sparse extents?

[Chris Murphy]

On Wed, Apr 22, 2020 at 12:52 PM Marek Behun <marek.behun@nic.cz> wrote:
>
> Hello,
>
> there was a bug fixed recently in U-Boot's btrfs driver - the driver
> failed to read files with sparse extents. This causes that sometimes
> device failes to boot Linux, since the kernel fails to load from
> storage.
>
> So when does kernel's btrfs driver write sparse extents? Is it always
> when it finds a PAGE_SIZEd and aligned all-zeros block? Or is it more
> complicated?

I'm not a btrfs developer, so with respect to kernel code behavior I
can't answer directly.

But I wonder if other sources of this sparseness has been considered?
Maybe the build system is creating or preserving sparseness? e.g. `tar
--hole-detection` or `--sparse` is used.

Another possibility is Btrfs supports two kinds of holes in the
on-disk format for sparse files. Maybe uboot only supported the
original (current default) type, and the bug really fixed the newer
'no-holes' feature version?


-- 
Chris Murphy

Re: when does btrfs create sparse extents?

[Marek Behun]

On Wed, 22 Apr 2020 14:26:31 -0600
Chris Murphy <lists@colorremedies.com> wrote:

> But I wonder if other sources of this sparseness has been considered?
> Maybe the build system is creating or preserving sparseness? e.g. `tar
> --hole-detection` or `--sparse` is used.

I don't actually know how the kernel is copied into the /boot directory
in this case.

> Another possibility is Btrfs supports two kinds of holes in the
> on-disk format for sparse files. Maybe uboot only supported the
> original (current default) type, and the bug really fixed the newer
> 'no-holes' feature version?

No, U-Boot did not support holes at all.

Re: when does btrfs create sparse extents?

[Chris Murphy]

e.g. from a 10m file created with truncate on two Btrfs file systems

original holes format (default)

    item 6 key (257 EXTENT_DATA 0) itemoff 15768 itemsize 53
        generation 7412 type 1 (regular)
        extent data disk byte 0 nr 0
        extent data offset 0 nr 10485760 ram 10485760
        extent compression 0 (none)

On a file system with no-holes feature set, this item simply doesn't
exist. I think basically it works by inference. Both kinds of files
have size in the INODE_ITEM, e.g.

    item 4 key (257 INODE_ITEM 0) itemoff 32245 itemsize 160
        generation 889509 transid 889509 size 10485760 nbytes 0

Sparse extents are explicitly stated in the original format with disk
byte 0 in an EXTENT_DATA item; whereas in the newer format, sparse
extents exist whenever EXTENT_DATA items don't completely describe the
file's size.

--
Chris Murphy

Re: when does btrfs create sparse extents?

[Marek Behun]

On Wed, 22 Apr 2020 14:44:46 -0600
Chris Murphy <lists@colorremedies.com> wrote:

> e.g. from a 10m file created with truncate on two Btrfs file systems
> 
> original holes format (default)
> 
>     item 6 key (257 EXTENT_DATA 0) itemoff 15768 itemsize 53
>         generation 7412 type 1 (regular)
>         extent data disk byte 0 nr 0
>         extent data offset 0 nr 10485760 ram 10485760
>         extent compression 0 (none)
> 
> On a file system with no-holes feature set, this item simply doesn't
> exist. I think basically it works by inference. Both kinds of files
> have size in the INODE_ITEM, e.g.
> 
>     item 4 key (257 INODE_ITEM 0) itemoff 32245 itemsize 160
>         generation 889509 transid 889509 size 10485760 nbytes 0
> 
> Sparse extents are explicitly stated in the original format with disk
> byte 0 in an EXTENT_DATA item; whereas in the newer format, sparse
> extents exist whenever EXTENT_DATA items don't completely describe the
> file's size.

Ok this means that U-Boot currently gained support for the original
sparse extents.

I fear that current u-boot does not handle the new no-holes feature.

Re: when does btrfs create sparse extents?

[Chris Murphy]

On Wed, Apr 22, 2020 at 2:58 PM Marek Behun <marek.behun@nic.cz> wrote:
>
> On Wed, 22 Apr 2020 14:44:46 -0600
> Chris Murphy <lists@colorremedies.com> wrote:
>
> > e.g. from a 10m file created with truncate on two Btrfs file systems
> >
> > original holes format (default)
> >
> >     item 6 key (257 EXTENT_DATA 0) itemoff 15768 itemsize 53
> >         generation 7412 type 1 (regular)
> >         extent data disk byte 0 nr 0
> >         extent data offset 0 nr 10485760 ram 10485760
> >         extent compression 0 (none)
> >
> > On a file system with no-holes feature set, this item simply doesn't
> > exist. I think basically it works by inference. Both kinds of files
> > have size in the INODE_ITEM, e.g.
> >
> >     item 4 key (257 INODE_ITEM 0) itemoff 32245 itemsize 160
> >         generation 889509 transid 889509 size 10485760 nbytes 0
> >
> > Sparse extents are explicitly stated in the original format with disk
> > byte 0 in an EXTENT_DATA item; whereas in the newer format, sparse
> > extents exist whenever EXTENT_DATA items don't completely describe the
> > file's size.
>
> Ok this means that U-Boot currently gained support for the original
> sparse extents.
>
> I fear that current u-boot does not handle the new no-holes feature.

I'd advise it should support it, I think it's expected to become the
default. But I don't know the time frame for that.

Also, when the no-holes feature flag is not set, you can be certain
only the original hole type is used. But when no-holes feature is set,
it's possible for both types of holes to exist, because no-holes can
be set after mkfs time by btrfstune.

-- 
Chris Murphy

Re: when does btrfs create sparse extents?

[Andrei Borzenkov]

22.04.2020 21:52, Marek Behun пишет:
> Hello,
> 
> there was a bug fixed recently in U-Boot's btrfs driver - the driver
> failed to read files with sparse extents. This causes that sometimes
> device failes to boot Linux, since the kernel fails to load from
> storage.
> 

Do you mean that kernel image (vmlinuz?) had holes? Or there were some
other files that caused U-Boot to fail?

> So when does kernel's btrfs driver write sparse extents? Is it always
> when it finds a PAGE_SIZEd and aligned all-zeros block? Or is it more
> complicated?
> 
> Thanks.
> 
> Marek
> 

Re: when does btrfs create sparse extents?

[Marek Behun]

On Thu, 23 Apr 2020 08:57:59 +0300
Andrei Borzenkov <arvidjaar@gmail.com> wrote:

> 22.04.2020 21:52, Marek Behun пишет:
> > Hello,
> > 
> > there was a bug fixed recently in U-Boot's btrfs driver - the driver
> > failed to read files with sparse extents. This causes that sometimes
> > device failes to boot Linux, since the kernel fails to load from
> > storage.
> >   
> 
> Do you mean that kernel image (vmlinuz?) had holes? Or there were some
> other files that caused U-Boot to fail?

Kernel image (non-compressed Image for arm64) had ~200 PAGE_SIZEd
all-zero blocks.

Re: when does btrfs create sparse extents?

[Filipe Manana]

On Wed, Apr 22, 2020 at 10:00 PM Marek Behun <marek.behun@nic.cz> wrote:
>
> On Wed, 22 Apr 2020 14:44:46 -0600
> Chris Murphy <lists@colorremedies.com> wrote:
>
> > e.g. from a 10m file created with truncate on two Btrfs file systems
> >
> > original holes format (default)
> >
> >     item 6 key (257 EXTENT_DATA 0) itemoff 15768 itemsize 53
> >         generation 7412 type 1 (regular)
> >         extent data disk byte 0 nr 0
> >         extent data offset 0 nr 10485760 ram 10485760
> >         extent compression 0 (none)
> >
> > On a file system with no-holes feature set, this item simply doesn't
> > exist. I think basically it works by inference. Both kinds of files
> > have size in the INODE_ITEM, e.g.
> >
> >     item 4 key (257 INODE_ITEM 0) itemoff 32245 itemsize 160
> >         generation 889509 transid 889509 size 10485760 nbytes 0
> >
> > Sparse extents are explicitly stated in the original format with disk
> > byte 0 in an EXTENT_DATA item; whereas in the newer format, sparse
> > extents exist whenever EXTENT_DATA items don't completely describe the
> > file's size.
>
> Ok this means that U-Boot currently gained support for the original
> sparse extents.

To clear any confusion, what you mean by sparse extents is actually holes.
The concept of sparse files exists (files with holes, regions of a
file for which there is no allocated extent), but not sparse extents.

>
> I fear that current u-boot does not handle the new no-holes feature.

The no-holes feature has been around since 2013, not exactly new, but
it's not the default yet when creating a new filesystem.

As it has been mentioned earlier by Chris, it just removes the need
for explicitly having metadata representing holes.
When not using the no-holes feature, there is an explicit file extent
item pointing to a disk location of 0 (disk_bytenr field has a value
of 0) for each file hole.
When using no-holes, there's no such file extent item - btrfs knows
about the hole by checking that there is a gap between two consecutive
file extent items (both having a disk_bytenr > 0).



-- 
Filipe David Manana,

“Whether you think you can, or you think you can't — you're right.”

Re: when does btrfs create sparse extents?

[Marek Behun]

On Thu, 23 Apr 2020 11:49:16 +0100
Filipe Manana <fdmanana@gmail.com> wrote:

> On Wed, Apr 22, 2020 at 10:00 PM Marek Behun <marek.behun@nic.cz> wrote:
> >
> > On Wed, 22 Apr 2020 14:44:46 -0600
> > Chris Murphy <lists@colorremedies.com> wrote:
> >  
> > > e.g. from a 10m file created with truncate on two Btrfs file systems
> > >
> > > original holes format (default)
> > >
> > >     item 6 key (257 EXTENT_DATA 0) itemoff 15768 itemsize 53
> > >         generation 7412 type 1 (regular)
> > >         extent data disk byte 0 nr 0
> > >         extent data offset 0 nr 10485760 ram 10485760
> > >         extent compression 0 (none)
> > >
> > > On a file system with no-holes feature set, this item simply doesn't
> > > exist. I think basically it works by inference. Both kinds of files
> > > have size in the INODE_ITEM, e.g.
> > >
> > >     item 4 key (257 INODE_ITEM 0) itemoff 32245 itemsize 160
> > >         generation 889509 transid 889509 size 10485760 nbytes 0
> > >
> > > Sparse extents are explicitly stated in the original format with disk
> > > byte 0 in an EXTENT_DATA item; whereas in the newer format, sparse
> > > extents exist whenever EXTENT_DATA items don't completely describe the
> > > file's size.  
> >
> > Ok this means that U-Boot currently gained support for the original
> > sparse extents.  
> 
> To clear any confusion, what you mean by sparse extents is actually holes.
> The concept of sparse files exists (files with holes, regions of a
> file for which there is no allocated extent), but not sparse extents.
> 
> >
> > I fear that current u-boot does not handle the new no-holes feature.  
> 
> The no-holes feature has been around since 2013, not exactly new, but
> it's not the default yet when creating a new filesystem.
> 
> As it has been mentioned earlier by Chris, it just removes the need
> for explicitly having metadata representing holes.
> When not using the no-holes feature, there is an explicit file extent
> item pointing to a disk location of 0 (disk_bytenr field has a value
> of 0) for each file hole.
> When using no-holes, there's no such file extent item - btrfs knows
> about the hole by checking that there is a gap between two consecutive
> file extent items (both having a disk_bytenr > 0).

This I already understand. My main question though is: does kernel or
btrfs do checking (at least sometimes) when writing a block of data onto
disk if this block is all zero, and if yes, then this block is written
as a hole (either by writing hole item or not writing anything)?

Or does this happen ONLY when requested by userspace?

Because for the love of god I cannot find why our kernel is being
written this way onto disk - the installer doesn't explicitly request
for PUNCH_HOLES nor anything, as far as I looked.

Marek

Marek

Re: when does btrfs create sparse extents?

[Filipe Manana]

On Thu, Apr 23, 2020 at 12:42 PM Marek Behun <marek.behun@nic.cz> wrote:
>
> On Thu, 23 Apr 2020 11:49:16 +0100
> Filipe Manana <fdmanana@gmail.com> wrote:
>
> > On Wed, Apr 22, 2020 at 10:00 PM Marek Behun <marek.behun@nic.cz> wrote:
> > >
> > > On Wed, 22 Apr 2020 14:44:46 -0600
> > > Chris Murphy <lists@colorremedies.com> wrote:
> > >
> > > > e.g. from a 10m file created with truncate on two Btrfs file systems
> > > >
> > > > original holes format (default)
> > > >
> > > >     item 6 key (257 EXTENT_DATA 0) itemoff 15768 itemsize 53
> > > >         generation 7412 type 1 (regular)
> > > >         extent data disk byte 0 nr 0
> > > >         extent data offset 0 nr 10485760 ram 10485760
> > > >         extent compression 0 (none)
> > > >
> > > > On a file system with no-holes feature set, this item simply doesn't
> > > > exist. I think basically it works by inference. Both kinds of files
> > > > have size in the INODE_ITEM, e.g.
> > > >
> > > >     item 4 key (257 INODE_ITEM 0) itemoff 32245 itemsize 160
> > > >         generation 889509 transid 889509 size 10485760 nbytes 0
> > > >
> > > > Sparse extents are explicitly stated in the original format with disk
> > > > byte 0 in an EXTENT_DATA item; whereas in the newer format, sparse
> > > > extents exist whenever EXTENT_DATA items don't completely describe the
> > > > file's size.
> > >
> > > Ok this means that U-Boot currently gained support for the original
> > > sparse extents.
> >
> > To clear any confusion, what you mean by sparse extents is actually holes.
> > The concept of sparse files exists (files with holes, regions of a
> > file for which there is no allocated extent), but not sparse extents.
> >
> > >
> > > I fear that current u-boot does not handle the new no-holes feature.
> >
> > The no-holes feature has been around since 2013, not exactly new, but
> > it's not the default yet when creating a new filesystem.
> >
> > As it has been mentioned earlier by Chris, it just removes the need
> > for explicitly having metadata representing holes.
> > When not using the no-holes feature, there is an explicit file extent
> > item pointing to a disk location of 0 (disk_bytenr field has a value
> > of 0) for each file hole.
> > When using no-holes, there's no such file extent item - btrfs knows
> > about the hole by checking that there is a gap between two consecutive
> > file extent items (both having a disk_bytenr > 0).
>
> This I already understand. My main question though is: does kernel or
> btrfs do checking (at least sometimes) when writing a block of data onto
> disk if this block is all zero, and if yes, then this block is written
> as a hole (either by writing hole item or not writing anything)?
>
> Or does this happen ONLY when requested by userspace?

There's nothing in btrfs that converts a sequence of zeroes
automatically to a hole.

It always has to be done by user space, either by writes that leave
holes intentionally (e.g. create file, write 64K to offset 0, write 4K
to offset 128, leaves a hole from range 64K to 128K) or by hole
punching through fallocate().

>
> Because for the love of god I cannot find why our kernel is being
> written this way onto disk - the installer doesn't explicitly request
> for PUNCH_HOLES nor anything, as far as I looked.
>
> Marek
>
> Marek



-- 
Filipe David Manana,

“Whether you think you can, or you think you can't — you're right.”

Re: when does btrfs create sparse extents?

[Marek Behun]

On Thu, 23 Apr 2020 12:51:32 +0100
Filipe Manana <fdmanana@gmail.com> wrote:

> There's nothing in btrfs that converts a sequence of zeroes
> automatically to a hole.
> 
> It always has to be done by user space, either by writes that leave
> holes intentionally (e.g. create file, write 64K to offset 0, write 4K
> to offset 128, leaves a hole from range 64K to 128K) or by hole
> punching through fallocate().

Thanks for this information. If you ever come to Prague, let me know,
we can have a beer :D

Re: when does btrfs create sparse extents?

[Filipe Manana]

On Thu, Apr 23, 2020 at 1:06 PM Marek Behun <marek.behun@nic.cz> wrote:
>
> On Thu, 23 Apr 2020 12:51:32 +0100
> Filipe Manana <fdmanana@gmail.com> wrote:
>
> > There's nothing in btrfs that converts a sequence of zeroes
> > automatically to a hole.
> >
> > It always has to be done by user space, either by writes that leave
> > holes intentionally (e.g. create file, write 64K to offset 0, write 4K
> > to offset 128, leaves a hole from range 64K to 128K) or by hole
> > punching through fallocate().
>
> Thanks for this information. If you ever come to Prague, let me know,
> we can have a beer :D

Noted! (Through it will be a long while until travel is allowed and safe)

Another case I forgot is a hole created by truncation - truncate a
file to a larger size, then you get a hole for the range that goes
from the old file size to the new file size.



-- 
Filipe David Manana,

“Whether you think you can, or you think you can't — you're right.”

Re: when does btrfs create sparse extents?

[Chris Murphy]

On Thu, Apr 23, 2020 at 6:39 AM Filipe Manana <fdmanana@gmail.com> wrote:
>
> On Thu, Apr 23, 2020 at 1:06 PM Marek Behun <marek.behun@nic.cz> wrote:
> >
> > On Thu, 23 Apr 2020 12:51:32 +0100
> > Filipe Manana <fdmanana@gmail.com> wrote:
> >
> > > There's nothing in btrfs that converts a sequence of zeroes
> > > automatically to a hole.
> > >
> > > It always has to be done by user space, either by writes that leave
> > > holes intentionally (e.g. create file, write 64K to offset 0, write 4K
> > > to offset 128, leaves a hole from range 64K to 128K) or by hole
> > > punching through fallocate().
> >
> > Thanks for this information. If you ever come to Prague, let me know,
> > we can have a beer :D
>
> Noted! (Through it will be a long while until travel is allowed and safe)
>
> Another case I forgot is a hole created by truncation - truncate a
> file to a larger size, then you get a hole for the range that goes
> from the old file size to the new file size.


I recently discovered the benefit of using both truncate and partially
fallocating a VM backing file.

I start with truncate so I can set a typical device size for the
guest's view, which I don't have the local space if I were to
fallocate. e.g. 100G local, but do truncate -s 1T. Then follow that
with fallocate -l 12G so that I don't end up with a massively
fragmented backing file.

To be clear, this backing file is set with chattr +C and never
snapshot or reflinked. Btrfs initially doesn't seem to perform any
different with sparse or fallocated backing files. But with aging, the
sparse file can get really fragmented depending on the guest file
system. Tens of thousands of extents are possible, and that does
impact performance it seems.

-- 
Chris Murphy