Punch hole on full fs

Punch hole on full fs

[anand.jain]

Test script [1] tries to punch hole on a full FS and it works fine as 
long as the hole size and the offset is aligned with the sectorsize and 
the extent, so that it could just drop the relevant extent to create the 
hole.

The reason why this test fails for non aligned hole size and offset is 
because, as it rightfully tries to truncate the non aligned extent at 
the front and back of the hole, it tries to create new extent which ends 
up with ENOSPC.

Any idea, how do we solve this?

xfs is fine.

[1]
cat ./punch-hole-on-full-fs
------------------------
cleanup()
{
	umount /dev/sdb > /dev/null 2>&1
	btrfs_reload
}

full_fs_setup()
{
	btrfs-dyndbg disable
	mkfs.$fs -b 200M -fq $mkfs_options /dev/sdb || exit
	mount $mount_options /dev/sdb /mnt/scratch || exit
	dd status=none if=/dev/zero of=/mnt/scratch/filler bs=512 > /dev/null 2>&1
}

test()
{
	cleanup
	full_fs_setup

	btrfs-dyndbg enable
	echo "---- fallocate -p -o 0 -l $punch_hole_size /mnt/scratch/filler ----"
	fallocate -p -o 0 -l $punch_hole_size /mnt/scratch/filler
}

fs=btrfs; mkfs_options=""; mount_options="";
[[ $1 ]] || { echo "usage: $0 <hole-len>"; exit; }
punch_hole_size=$1 && test
------------------------

./punch-hole-on-full-fs 512
-- fallocate -p -o 0 -l 512 /mnt/scratch/filler --
fallocate: /mnt/scratch/filler: fallocate failed: No space left on device

./punch-hole-on-full-fs 8000
-- fallocate -p -o 0 -l 8000 /mnt/scratch/filler --
fallocate: /mnt/scratch/filler: fallocate failed: No space left on device


Thanks, Anand

Re: Punch hole on full fs

[Qu Wenruo]

[-- Attachment #1.1: Type: text/plain, Size: 2188 bytes --]



On 2018/9/20 下午1:26, anand.jain@oracle.com wrote:
> 
> 
> Test script [1] tries to punch hole on a full FS and it works fine as
> long as the hole size and the offset is aligned with the sectorsize and
> the extent, so that it could just drop the relevant extent to create the
> hole.
> 
> The reason why this test fails for non aligned hole size and offset is
> because, as it rightfully tries to truncate the non aligned extent at
> the front and back of the hole, it tries to create new extent which ends
> up with ENOSPC.

This is what btrfs should do, I don't see any problem though.

> 
> Any idea, how do we solve this?

I don't really think this is a problem though.

As long as we need to do data COW, such ENOSPC would happen.

> 
> xfs is fine.

Of course xfs is fine, XFS defaults to do NODATACOW, in contrast to
btrfs' default DATACOW.

Have you tried to do the same when the file is reflinked?

Thanks,
Qu

> 
> [1]
> cat ./punch-hole-on-full-fs
> ------------------------
> cleanup()
> {
>     umount /dev/sdb > /dev/null 2>&1
>     btrfs_reload
> }
> 
> full_fs_setup()
> {
>     btrfs-dyndbg disable
>     mkfs.$fs -b 200M -fq $mkfs_options /dev/sdb || exit
>     mount $mount_options /dev/sdb /mnt/scratch || exit
>     dd status=none if=/dev/zero of=/mnt/scratch/filler bs=512 >
> /dev/null 2>&1
> }
> 
> test()
> {
>     cleanup
>     full_fs_setup
> 
>     btrfs-dyndbg enable
>     echo "---- fallocate -p -o 0 -l $punch_hole_size /mnt/scratch/filler
> ----"
>     fallocate -p -o 0 -l $punch_hole_size /mnt/scratch/filler
> }
> 
> fs=btrfs; mkfs_options=""; mount_options="";
> [[ $1 ]] || { echo "usage: $0 <hole-len>"; exit; }
> punch_hole_size=$1 && test
> ------------------------
> 
> ./punch-hole-on-full-fs 512
> -- fallocate -p -o 0 -l 512 /mnt/scratch/filler --
> fallocate: /mnt/scratch/filler: fallocate failed: No space left on device
> 
> ./punch-hole-on-full-fs 8000
> -- fallocate -p -o 0 -l 8000 /mnt/scratch/filler --
> fallocate: /mnt/scratch/filler: fallocate failed: No space left on device
> 
> 
> Thanks, Anand


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Punch hole on full fs

[Anand Jain]

On 09/20/2018 04:45 PM, Qu Wenruo wrote:
> 
> 
> On 2018/9/20 下午1:26, anand.jain@oracle.com wrote:
>>
>>
>> Test script [1] tries to punch hole on a full FS and it works fine as
>> long as the hole size and the offset is aligned with the sectorsize and
>> the extent, so that it could just drop the relevant extent to create the
>> hole.
>>
>> The reason why this test fails for non aligned hole size and offset is
>> because, as it rightfully tries to truncate the non aligned extent at
>> the front and back of the hole, it tries to create new extent which ends
>> up with ENOSPC.
> 
> This is what btrfs should do, I don't see any problem though.
> 
>>
>> Any idea, how do we solve this?
> 
> I don't really think this is a problem though.
> 
> As long as we need to do data COW, such ENOSPC would happen.

It happens even with '-o max_inline=0 -o nodatacow' so certainly there 
is some problem, but looking at btrfs_punch_hole() it apparently looks 
like there is no way to fix this as sub-sectorsize hole-size depends on 
the truncate which needs to alloc space.

>>
>> xfs is fine.
> 
> Of course xfs is fine, XFS defaults to do NODATACOW, in contrast to
> btrfs' default DATACOW.
> 
> Have you tried to do the same when the file is reflinked?

  No but I presume it will rightfully fail.

Thanks, -Anand

> Thanks,
> Qu
> 
>>
>> [1]
>> cat ./punch-hole-on-full-fs
>> ------------------------
>> cleanup()
>> {
>>      umount /dev/sdb > /dev/null 2>&1
>>      btrfs_reload
>> }
>>
>> full_fs_setup()
>> {
>>      btrfs-dyndbg disable
>>      mkfs.$fs -b 200M -fq $mkfs_options /dev/sdb || exit
>>      mount $mount_options /dev/sdb /mnt/scratch || exit
>>      dd status=none if=/dev/zero of=/mnt/scratch/filler bs=512 >
>> /dev/null 2>&1
>> }
>>
>> test()
>> {
>>      cleanup
>>      full_fs_setup
>>
>>      btrfs-dyndbg enable
>>      echo "---- fallocate -p -o 0 -l $punch_hole_size /mnt/scratch/filler
>> ----"
>>      fallocate -p -o 0 -l $punch_hole_size /mnt/scratch/filler
>> }
>>
>> fs=btrfs; mkfs_options=""; mount_options="";
>> [[ $1 ]] || { echo "usage: $0 <hole-len>"; exit; }
>> punch_hole_size=$1 && test
>> ------------------------
>>
>> ./punch-hole-on-full-fs 512
>> -- fallocate -p -o 0 -l 512 /mnt/scratch/filler --
>> fallocate: /mnt/scratch/filler: fallocate failed: No space left on device
>>
>> ./punch-hole-on-full-fs 8000
>> -- fallocate -p -o 0 -l 8000 /mnt/scratch/filler --
>> fallocate: /mnt/scratch/filler: fallocate failed: No space left on device
>>
>>
>> Thanks, Anand
> 

Re: Punch hole on full fs

[Qu Wenruo]

[-- Attachment #1.1: Type: text/plain, Size: 4051 bytes --]



On 2018/9/20 下午5:04, Anand Jain wrote:
> 
> 
> On 09/20/2018 04:45 PM, Qu Wenruo wrote:
>>
>>
>> On 2018/9/20 下午1:26, anand.jain@oracle.com wrote:
>>>
>>>
>>> Test script [1] tries to punch hole on a full FS and it works fine as
>>> long as the hole size and the offset is aligned with the sectorsize and
>>> the extent, so that it could just drop the relevant extent to create the
>>> hole.
>>>
>>> The reason why this test fails for non aligned hole size and offset is
>>> because, as it rightfully tries to truncate the non aligned extent at
>>> the front and back of the hole, it tries to create new extent which ends
>>> up with ENOSPC.
>>
>> This is what btrfs should do, I don't see any problem though.
>>
>>>
>>> Any idea, how do we solve this?
>>
>> I don't really think this is a problem though.
>>
>> As long as we need to do data COW, such ENOSPC would happen.
> 
> It happens even with '-o max_inline=0 -o nodatacow' so certainly there
> is some problem, but looking at btrfs_punch_hole() it apparently looks
> like there is no way to fix this as sub-sectorsize hole-size depends on
> the truncate which needs to alloc space.

Well, at least in my test, that's not the case.

I did a pretty simpler test, and for NODATACOW inode, the sub-sectorsize
part didn't get COWed, just as what we expect:

# mkfs.btrfs -f $dev
# mount $dev -o nodatacow $dev $mnt
# xfs_io -f -c "pwrite 0 16K" -c sync $mnt/file
# btrfs ins dump-tree -t 5 $dev
# xfs_io -c "fpunch 2K 6K" $mnt/file
# btrfs ins dump-tree -t 5 $dev

The last punch hole did created a 4K hole as expect, but the 0~4K extent
doesn't get COWed at all.

	item 6 key (257 EXTENT_DATA 0) itemoff 15813 itemsize 53
		generation 6 type 1 (regular)
		extent data disk byte 13631488 nr 16384 <<<
		extent data offset 0 nr 4096 ram 16384
		extent compression 0 (none)
	item 7 key (257 EXTENT_DATA 4096) itemoff 15760 itemsize 53
		generation 8 type 1 (regular)
		extent data disk byte 0 nr 0
		extent data offset 0 nr 4096 ram 4096
		extent compression 0 (none)
	item 8 key (257 EXTENT_DATA 8192) itemoff 15707 itemsize 53
		generation 6 type 1 (regular)
		extent data disk byte 13631488 nr 16384 <<<
		extent data offset 8192 nr 8192 ram 16384
		extent compression 0 (none)

So the failure even with nodatacow mount option looks a little
suspicious now.

Thanks,
Qu

> 
>>>
>>> xfs is fine.
>>
>> Of course xfs is fine, XFS defaults to do NODATACOW, in contrast to
>> btrfs' default DATACOW.
>>
>> Have you tried to do the same when the file is reflinked?
> 
>  No but I presume it will rightfully fail.
> 
> Thanks, -Anand
> 
>> Thanks,
>> Qu
>>
>>>
>>> [1]
>>> cat ./punch-hole-on-full-fs
>>> ------------------------
>>> cleanup()
>>> {
>>>      umount /dev/sdb > /dev/null 2>&1
>>>      btrfs_reload
>>> }
>>>
>>> full_fs_setup()
>>> {
>>>      btrfs-dyndbg disable
>>>      mkfs.$fs -b 200M -fq $mkfs_options /dev/sdb || exit
>>>      mount $mount_options /dev/sdb /mnt/scratch || exit
>>>      dd status=none if=/dev/zero of=/mnt/scratch/filler bs=512 >
>>> /dev/null 2>&1
>>> }
>>>
>>> test()
>>> {
>>>      cleanup
>>>      full_fs_setup
>>>
>>>      btrfs-dyndbg enable
>>>      echo "---- fallocate -p -o 0 -l $punch_hole_size
>>> /mnt/scratch/filler
>>> ----"
>>>      fallocate -p -o 0 -l $punch_hole_size /mnt/scratch/filler
>>> }
>>>
>>> fs=btrfs; mkfs_options=""; mount_options="";
>>> [[ $1 ]] || { echo "usage: $0 <hole-len>"; exit; }
>>> punch_hole_size=$1 && test
>>> ------------------------
>>>
>>> ./punch-hole-on-full-fs 512
>>> -- fallocate -p -o 0 -l 512 /mnt/scratch/filler --
>>> fallocate: /mnt/scratch/filler: fallocate failed: No space left on
>>> device
>>>
>>> ./punch-hole-on-full-fs 8000
>>> -- fallocate -p -o 0 -l 8000 /mnt/scratch/filler --
>>> fallocate: /mnt/scratch/filler: fallocate failed: No space left on
>>> device
>>>
>>>
>>> Thanks, Anand
>>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Punch hole on full fs

[Qu Wenruo]

[-- Attachment #1.1: Type: text/plain, Size: 3345 bytes --]



On 2018/9/20 下午5:04, Anand Jain wrote:
> 
> 
> On 09/20/2018 04:45 PM, Qu Wenruo wrote:
>>
>>
>> On 2018/9/20 下午1:26, anand.jain@oracle.com wrote:
>>>
>>>
>>> Test script [1] tries to punch hole on a full FS and it works fine as
>>> long as the hole size and the offset is aligned with the sectorsize and
>>> the extent, so that it could just drop the relevant extent to create the
>>> hole.
>>>
>>> The reason why this test fails for non aligned hole size and offset is
>>> because, as it rightfully tries to truncate the non aligned extent at
>>> the front and back of the hole, it tries to create new extent which ends
>>> up with ENOSPC.
>>
>> This is what btrfs should do, I don't see any problem though.
>>
>>>
>>> Any idea, how do we solve this?
>>
>> I don't really think this is a problem though.
>>
>> As long as we need to do data COW, such ENOSPC would happen.
> 
> It happens even with '-o max_inline=0 -o nodatacow' so certainly there
> is some problem, but looking at btrfs_punch_hole() it apparently looks
> like there is no way to fix this as sub-sectorsize hole-size depends on
> the truncate which needs to alloc space.

Forgot to mention, in your test script, it only fills data space, so we
should still have enough free metadata space to fulfill fs/extent/root
tree operations.

Although the problem is, btrfs_truncate_block() will call
btrfs_delalloc_reserve_space() even the inode has NODATACOW flag.
(buffered write part does the extra check, although not completely
perfect check).

So in short, you could still fix it false ENOSPC problem by enhance
btrfs_truncate_block() to follow NODATACOW check.

Thanks,
Qu

> 
>>>
>>> xfs is fine.
>>
>> Of course xfs is fine, XFS defaults to do NODATACOW, in contrast to
>> btrfs' default DATACOW.
>>
>> Have you tried to do the same when the file is reflinked?
> 
>  No but I presume it will rightfully fail.
> 
> Thanks, -Anand
> 
>> Thanks,
>> Qu
>>
>>>
>>> [1]
>>> cat ./punch-hole-on-full-fs
>>> ------------------------
>>> cleanup()
>>> {
>>>      umount /dev/sdb > /dev/null 2>&1
>>>      btrfs_reload
>>> }
>>>
>>> full_fs_setup()
>>> {
>>>      btrfs-dyndbg disable
>>>      mkfs.$fs -b 200M -fq $mkfs_options /dev/sdb || exit
>>>      mount $mount_options /dev/sdb /mnt/scratch || exit
>>>      dd status=none if=/dev/zero of=/mnt/scratch/filler bs=512 >
>>> /dev/null 2>&1
>>> }
>>>
>>> test()
>>> {
>>>      cleanup
>>>      full_fs_setup
>>>
>>>      btrfs-dyndbg enable
>>>      echo "---- fallocate -p -o 0 -l $punch_hole_size
>>> /mnt/scratch/filler
>>> ----"
>>>      fallocate -p -o 0 -l $punch_hole_size /mnt/scratch/filler
>>> }
>>>
>>> fs=btrfs; mkfs_options=""; mount_options="";
>>> [[ $1 ]] || { echo "usage: $0 <hole-len>"; exit; }
>>> punch_hole_size=$1 && test
>>> ------------------------
>>>
>>> ./punch-hole-on-full-fs 512
>>> -- fallocate -p -o 0 -l 512 /mnt/scratch/filler --
>>> fallocate: /mnt/scratch/filler: fallocate failed: No space left on
>>> device
>>>
>>> ./punch-hole-on-full-fs 8000
>>> -- fallocate -p -o 0 -l 8000 /mnt/scratch/filler --
>>> fallocate: /mnt/scratch/filler: fallocate failed: No space left on
>>> device
>>>
>>>
>>> Thanks, Anand
>>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]