[PATCH v2 0/9] Fix _scratch_mkfs_sized() for btrfs

[PATCH v2 0/9] Fix _scratch_mkfs_sized() for btrfs

[Anand Jain]

v1->v2: Fix the min size for _scratch_mkfs_sized() to 256MB for btrfs and
         add comment about its ref.
	Fix _scratch_mkfs_sized() options to use at least 256MB in each
	 test case.
        Update change long.

Currently _scratch_mkfs_sized() checks if the requested size is below 1G
for btrfs and forces the --mixed mkfs option. The 1G size cut off is too
larger where the actual size is ~256mb. The patch 1/9 fixes the
_scratch_mkfs_sized() to check for 256mb instead.

Also fix the test cases requesting to create FS below 256mb. And open
code the fssize computation where possible (that is the variable used
is only used once).

Further, there are test cases generic/250,252,256,416 which uses
_scratch_mkfs_sized with < 200mb but its fix isn't straight forward
and they are not yet ready for the review.

The affected testcases (which calls _scratch_mkfs_sized) [1] have
been tested on btrfs. And the test cases [2] which are being patches
here are tested on xfs ext4 and btrfs for no regression.

[1]
generic/015 generic/027 generic/077 generic/081 generic/083 generic/085 generic/096 generic/102 generic/171 generic/172 generic/173 generic/174 generic/204 generic/224 generic/226 generic/250 generic/252 generic/256 generic/269 generic/270 generic/273 generic/274 generic/275 generic/300 generic/312 generic/320 generic/333 generic/334 generic/361 generic/371 generic/387 generic/399 generic/416 generic/427 generic/449 generic/459 generic/466 generic/488 generic/505 btrfs/004 btrfs/007 btrfs/132 btrfs/170

[2]
generic/015 generic/077 generic/083 generic/387 generic/449

Anand Jain (9):
  fstests: btrfs: _scratch_mkfs_sized fix min size without mixed option
  generic/015 fix to test the default non-mixed mode
  geneirc/077 fix min size for btrfs
  generic/083 fix btrfs fs size for default mode
  generic/102 open code dev_size _scratch_mkfs_sized()
  generic/204 open code SIZE for _scratch_mkfs_sized()
  generic/312 open code fs_size _scratch_mkfs_sized()
  generic/449 fix fs size for _scratch_mkfs_sized for btrfs
  generic/387 fix _scratch_mkfs_sized option for btrfs

 common/rc         | 4 +++-
 tests/generic/015 | 8 +++-----
 tests/generic/077 | 3 +--
 tests/generic/083 | 2 +-
 tests/generic/102 | 3 +--
 tests/generic/204 | 3 +--
 tests/generic/312 | 7 +++----
 tests/generic/387 | 3 ++-
 tests/generic/449 | 4 ++--
 9 files changed, 17 insertions(+), 20 deletions(-)

-- 
1.8.3.1

[PATCH v2 1/9] fstests: btrfs: _scratch_mkfs_sized fix min size without mixed option

[Anand Jain]

As of now _scratch_mkfs_sized() checks if the requested size is below 1G
and forces the --mixed option for the mkfs.btrfs. Well the correct size
considering all possible group profiles at which we need to force the
mixed option is roughly 256Mbytes. So fix that.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
 common/rc | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/common/rc b/common/rc
index d5bb1feee2c3..90dc3002bc3d 100644
--- a/common/rc
+++ b/common/rc
@@ -969,7 +969,10 @@ _scratch_mkfs_sized()
 	;;
     btrfs)
 	local mixed_opt=
-	(( fssize <= 1024 * 1024 * 1024 )) && mixed_opt='--mixed'
+	# minimum size that's needed without the mixed option.
+	# Ref: btrfs-prog: btrfs_min_dev_size()
+	# Non mixed mode is also the default option.
+	(( fssize < $((256 * 1024 *1024)) )) && mixed_opt='--mixed'
 	$MKFS_BTRFS_PROG $MKFS_OPTIONS $mixed_opt -b $fssize $SCRATCH_DEV
 	;;
     jfs)
-- 
1.8.3.1

[PATCH v2 2/9] generic/015 fix to test the default non-mixed mode

[Anand Jain]

commit 97575acd7495 (generic/015: Change the test filesystem size to
101mb), created 101mb FS instead of 100mb FS to make sure we create
a FS which is non mixed mode.

btrfs-progs commit 18e2663db3e1 (btrfs-progs: Add minimum device size
check) added a more accurate minimum required space to create the btrfs
FS in non mixed mode depending on the group profile, and considering
any group profiles we would need at least 256MB (with upward round off).

So this patch changes the FS size to be created by _scratch_sized_mkfs()
to 256MB so that we create the FS in non mixed mode for any group
profile.

Mixed blockgroup can be tested using the MKFS_OPTIONS explicitly.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
 tests/generic/015 | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/tests/generic/015 b/tests/generic/015
index 0f4d29800f4f..e6c8d7c37c07 100755
--- a/tests/generic/015
+++ b/tests/generic/015
@@ -37,11 +37,9 @@ _supported_os Linux
 _require_scratch
 _require_no_large_scratch_dev
 
-# With filesystems less than 100mb btrfs is created in mixed mode
-# which can lead to slight accounting errors of 1mb. Having the
-# fs be at least 101 mb ensures those errors are within the error
-# tolerance of 1%
-_scratch_mkfs_sized `expr 101 \* 1024 \* 1024` >/dev/null 2>&1 \
+# btrfs needs at least 256MB (with upward round off) to create a non-mixed mode
+# fs. Ref: btrfs-progs: btrfs_min_dev_size()
+_scratch_mkfs_sized $((256 * 1024 * 1024)) >> $seqres.full 2>&1 \
     || _fail "mkfs failed"
 _scratch_mount
 out=$SCRATCH_MNT/fillup.$$
-- 
1.8.3.1

[PATCH v2 3/9] geneirc/077 fix min size for btrfs

[Anand Jain]

If btrfs need to be tested at its default blockgroup which is non-mixed,
then it needs at least 256mb.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
 tests/generic/077 | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/tests/generic/077 b/tests/generic/077
index ef6af18c83e3..ec236992513f 100755
--- a/tests/generic/077
+++ b/tests/generic/077
@@ -49,8 +49,7 @@ rm -f $seqres.full
 _scratch_unmount >/dev/null 2>&1
 echo "*** MKFS ***"                         >>$seqres.full
 echo ""                                     >>$seqres.full
-SIZE=`expr 50 \* 1024 \* 1024`
-_scratch_mkfs_sized $SIZE                   >>$seqres.full 2>&1 \
+_scratch_mkfs_sized $((256 * 1024 *1024))   >>$seqres.full 2>&1 \
 	|| _fail "mkfs failed"
 _scratch_mount
 mkdir $SCRATCH_MNT/subdir
-- 
1.8.3.1

[PATCH v2 4/9] generic/083 fix btrfs fs size for default mode

[Anand Jain]

btrfs needs at least 256mb to create a FS with the defaults options.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
 tests/generic/083 | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/generic/083 b/tests/generic/083
index 63d96ba2ab88..eabff02a36eb 100755
--- a/tests/generic/083
+++ b/tests/generic/083
@@ -71,7 +71,7 @@ workout()
 
 echo "*** test out-of-space handling for random write operations"
 
-filesize=`expr 100 \* 1024 \* 1024`
+filesize=`expr 256 \* 1024 \* 1024`
 agcount=6
 numprocs=15
 numops=1500
-- 
1.8.3.1

[PATCH v2 5/9] generic/102 open code dev_size _scratch_mkfs_sized()

[Anand Jain]

Open code helps to grep and find out parameter sent to the
_scratch_mkfs_sized here.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
 tests/generic/102 | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/tests/generic/102 b/tests/generic/102
index faf940ac5070..aad496a5bc69 100755
--- a/tests/generic/102
+++ b/tests/generic/102
@@ -31,8 +31,7 @@ _require_scratch
 
 rm -f $seqres.full
 
-dev_size=$((512 * 1024 * 1024))     # 512MB filesystem
-_scratch_mkfs_sized $dev_size >>$seqres.full 2>&1
+_scratch_mkfs_sized $((512 * 1024 * 1024)) >>$seqres.full 2>&1
 _scratch_mount
 
 for ((i = 0; i < 10; i++)); do
-- 
1.8.3.1

[PATCH v2 6/9] generic/204 open code SIZE for _scratch_mkfs_sized()

[Anand Jain]

By open code we could grep to find out FS size being use by this test
case.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
 tests/generic/204 | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/tests/generic/204 b/tests/generic/204
index 349f5eff351f..576c4eabd359 100755
--- a/tests/generic/204
+++ b/tests/generic/204
@@ -42,8 +42,7 @@ _scratch_mkfs 2> /dev/null | _filter_mkfs 2> $tmp.mkfs > /dev/null
 # time solves this problem.
 [ $FSTYP = "xfs" ] && MKFS_OPTIONS="$MKFS_OPTIONS -l size=16m -i maxpct=50"
 
-SIZE=`expr 115 \* 1024 \* 1024`
-_scratch_mkfs_sized $SIZE $dbsize 2> /dev/null > $tmp.mkfs.raw
+_scratch_mkfs_sized $((115 * 1024 * 1024)) $dbsize 2> /dev/null > $tmp.mkfs.raw
 cat $tmp.mkfs.raw | _filter_mkfs 2> $tmp.mkfs > /dev/null
 _scratch_mount
 
-- 
1.8.3.1

[PATCH v2 7/9] generic/312 open code fs_size _scratch_mkfs_sized()

[Anand Jain]

fs_size isn't used anywhere else further more now grep tells what size
is being used.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
 tests/generic/312 | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/tests/generic/312 b/tests/generic/312
index 756f5c3e4c5a..465009c8d184 100755
--- a/tests/generic/312
+++ b/tests/generic/312
@@ -34,11 +34,10 @@ _supported_os Linux
 _require_xfs_io_command "falloc"
 _require_scratch
 
-# 5G in byte
-fssize=$((2**30 * 5))
-
 rm -f $seqres.full
-_scratch_mkfs_sized $fssize >>$seqres.full 2>&1
+
+# 5G FS
+_scratch_mkfs_sized $((2**30 * 5)) >>$seqres.full 2>&1
 _scratch_mount >>$seqres.full 2>&1
 
 echo "Silence is golden"
-- 
1.8.3.1

[PATCH v2 8/9] generic/449 fix fs size for _scratch_mkfs_sized for btrfs

[Anand Jain]

btrfs needs 256m to create btrfs with default options like non mixed
block groups.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
 tests/generic/449 | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/generic/449 b/tests/generic/449
index 88f918654b54..21b920bf6a50 100755
--- a/tests/generic/449
+++ b/tests/generic/449
@@ -41,7 +41,7 @@ _require_test
 _require_acls
 _require_attrs
 
-_scratch_mkfs_sized $((50 * 1024 * 1024)) >> $seqres.full 2>&1
+_scratch_mkfs_sized $((256 * 1024 * 1024)) >> $seqres.full 2>&1
 _scratch_mount || _fail "mount failed"
 
 TFILE=$SCRATCH_MNT/testfile.$seq
@@ -52,7 +52,7 @@ chmod u+rwx $TFILE
 chmod go-rwx $TFILE
 
 # Try to run out of space so setfacl will fail
-$XFS_IO_PROG -c "pwrite 0 50m" $TFILE >>$seqres.full 2>&1
+$XFS_IO_PROG -c "pwrite 0 256m" $TFILE >>$seqres.full 2>&1
 i=1
 
 # Setting acls on an xfs filesystem will succeed even after running out of
-- 
1.8.3.1

[PATCH v2 9/9] generic/387 fix _scratch_mkfs_sized option for btrfs

[Anand Jain]

btrfs needs 256mb to create a fs with default block group which is non
mixed, so pass 256mb to _scratch_mkfs_sized().

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
 tests/generic/387 | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/generic/387 b/tests/generic/387
index 68a48012faa5..54a782bba3bc 100755
--- a/tests/generic/387
+++ b/tests/generic/387
@@ -35,7 +35,8 @@ _supported_fs generic
 _supported_os Linux
 _require_scratch_reflink
 
-_scratch_mkfs_sized $((32 * 1024 * 1024)) >> $seqres.full 2>&1
+#btrfs needs 256mb to create default blockgroup fs
+_scratch_mkfs_sized $((256 * 1024 * 1024)) >> $seqres.full 2>&1
 _scratch_mount
 
 testfile=$SCRATCH_MNT/testfile
-- 
1.8.3.1

Re: [PATCH v2 0/9] Fix _scratch_mkfs_sized() for btrfs

[Qu Wenruo]

[-- Attachment #1.1: Type: text/plain, Size: 3033 bytes --]



On 2018/9/25 下午12:24, Anand Jain wrote:
> v1->v2: Fix the min size for _scratch_mkfs_sized() to 256MB for btrfs and
>          add comment about its ref.
> 	Fix _scratch_mkfs_sized() options to use at least 256MB in each
> 	 test case.
>         Update change long.
> 
> Currently _scratch_mkfs_sized() checks if the requested size is below 1G
> for btrfs and forces the --mixed mkfs option. The 1G size cut off is too
> larger where the actual size is ~256mb. The patch 1/9 fixes the
> _scratch_mkfs_sized() to check for 256mb instead.

From the aspect of btrfs, the change to 256m totally makes sense and
avoids complex profile probing.

The patchset looks good to me.

Reviewed-by: Qu Wenruo <wqu@suse.com>

However I'm wondering if other fses is OK to increase the minimal fs size.
AFAIK btrfs has the largest minimal dev size requirement among
xfs/ext4/btrfs, so it should be OK, but still needs extra confirm from
xfs/ext4 people.

Thanks,
Qu

> 
> Also fix the test cases requesting to create FS below 256mb. And open
> code the fssize computation where possible (that is the variable used
> is only used once).
> 
> Further, there are test cases generic/250,252,256,416 which uses
> _scratch_mkfs_sized with < 200mb but its fix isn't straight forward
> and they are not yet ready for the review.
> 
> The affected testcases (which calls _scratch_mkfs_sized) [1] have
> been tested on btrfs. And the test cases [2] which are being patches
> here are tested on xfs ext4 and btrfs for no regression.
> 
> [1]
> generic/015 generic/027 generic/077 generic/081 generic/083 generic/085 generic/096 generic/102 generic/171 generic/172 generic/173 generic/174 generic/204 generic/224 generic/226 generic/250 generic/252 generic/256 generic/269 generic/270 generic/273 generic/274 generic/275 generic/300 generic/312 generic/320 generic/333 generic/334 generic/361 generic/371 generic/387 generic/399 generic/416 generic/427 generic/449 generic/459 generic/466 generic/488 generic/505 btrfs/004 btrfs/007 btrfs/132 btrfs/170
> 
> [2]
> generic/015 generic/077 generic/083 generic/387 generic/449
> 
> Anand Jain (9):
>   fstests: btrfs: _scratch_mkfs_sized fix min size without mixed option
>   generic/015 fix to test the default non-mixed mode
>   geneirc/077 fix min size for btrfs
>   generic/083 fix btrfs fs size for default mode
>   generic/102 open code dev_size _scratch_mkfs_sized()
>   generic/204 open code SIZE for _scratch_mkfs_sized()
>   generic/312 open code fs_size _scratch_mkfs_sized()
>   generic/449 fix fs size for _scratch_mkfs_sized for btrfs
>   generic/387 fix _scratch_mkfs_sized option for btrfs
> 
>  common/rc         | 4 +++-
>  tests/generic/015 | 8 +++-----
>  tests/generic/077 | 3 +--
>  tests/generic/083 | 2 +-
>  tests/generic/102 | 3 +--
>  tests/generic/204 | 3 +--
>  tests/generic/312 | 7 +++----
>  tests/generic/387 | 3 ++-
>  tests/generic/449 | 4 ++--
>  9 files changed, 17 insertions(+), 20 deletions(-)
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 1/9] fstests: btrfs: _scratch_mkfs_sized fix min size without mixed option

[Nikolay Borisov]

On 25.09.2018 07:24, Anand Jain wrote:
> As of now _scratch_mkfs_sized() checks if the requested size is below 1G
> and forces the --mixed option for the mkfs.btrfs. Well the correct size
> considering all possible group profiles at which we need to force the
> mixed option is roughly 256Mbytes. So fix that.
> 
> Signed-off-by: Anand Jain <anand.jain@oracle.com>

Have you considered the implications of this w.r.t commit
d4da414a9a9d ("common/rc: raise btrfs mixed mode threshold to 1GB")

Initially this threshold was 100mb then Omar changed it to 1g. Does this
change affect generic/427?

> ---
>  common/rc | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/common/rc b/common/rc
> index d5bb1feee2c3..90dc3002bc3d 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -969,7 +969,10 @@ _scratch_mkfs_sized()
>  	;;
>      btrfs)
>  	local mixed_opt=
> -	(( fssize <= 1024 * 1024 * 1024 )) && mixed_opt='--mixed'
> +	# minimum size that's needed without the mixed option.
> +	# Ref: btrfs-prog: btrfs_min_dev_size()
> +	# Non mixed mode is also the default option.
> +	(( fssize < $((256 * 1024 *1024)) )) && mixed_opt='--mixed'
>  	$MKFS_BTRFS_PROG $MKFS_OPTIONS $mixed_opt -b $fssize $SCRATCH_DEV
>  	;;
>      jfs)
> 

Re: [PATCH v2 5/9] generic/102 open code dev_size _scratch_mkfs_sized()

[Nikolay Borisov]

On 25.09.2018 07:24, Anand Jain wrote:
> Open code helps to grep and find out parameter sent to the
> _scratch_mkfs_sized here.
> 
> Signed-off-by: Anand Jain <anand.jain@oracle.com>

IMO this is noise, you can just as simply do
"grep _scratch_mkfs_sized" and then open the file to inspect the actual
argument. But it's up to the xfstest maintainers
> ---
>  tests/generic/102 | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/tests/generic/102 b/tests/generic/102
> index faf940ac5070..aad496a5bc69 100755
> --- a/tests/generic/102
> +++ b/tests/generic/102
> @@ -31,8 +31,7 @@ _require_scratch
>  
>  rm -f $seqres.full
>  
> -dev_size=$((512 * 1024 * 1024))     # 512MB filesystem
> -_scratch_mkfs_sized $dev_size >>$seqres.full 2>&1
> +_scratch_mkfs_sized $((512 * 1024 * 1024)) >>$seqres.full 2>&1
>  _scratch_mount
>  
>  for ((i = 0; i < 10; i++)); do
> 

Re: [PATCH v2 2/9] generic/015 fix to test the default non-mixed mode

[Nikolay Borisov]

On 25.09.2018 07:24, Anand Jain wrote:
> commit 97575acd7495 (generic/015: Change the test filesystem size to
> 101mb), created 101mb FS instead of 100mb FS to make sure we create
> a FS which is non mixed mode.
> 
> btrfs-progs commit 18e2663db3e1 (btrfs-progs: Add minimum device size
> check) added a more accurate minimum required space to create the btrfs
> FS in non mixed mode depending on the group profile, and considering
> any group profiles we would need at least 256MB (with upward round off).
> 
> So this patch changes the FS size to be created by _scratch_sized_mkfs()
> to 256MB so that we create the FS in non mixed mode for any group
> profile.
> 
> Mixed blockgroup can be tested using the MKFS_OPTIONS explicitly.
> 
> Signed-off-by: Anand Jain <anand.jain@oracle.com>
> ---
>  tests/generic/015 | 8 +++-----
>  1 file changed, 3 insertions(+), 5 deletions(-)
> 
> diff --git a/tests/generic/015 b/tests/generic/015
> index 0f4d29800f4f..e6c8d7c37c07 100755
> --- a/tests/generic/015
> +++ b/tests/generic/015
> @@ -37,11 +37,9 @@ _supported_os Linux
>  _require_scratch
>  _require_no_large_scratch_dev
>  
> -# With filesystems less than 100mb btrfs is created in mixed mode
> -# which can lead to slight accounting errors of 1mb. Having the
> -# fs be at least 101 mb ensures those errors are within the error
> -# tolerance of 1%
> -_scratch_mkfs_sized `expr 101 \* 1024 \* 1024` >/dev/null 2>&1 \
> +# btrfs needs at least 256MB (with upward round off) to create a non-mixed mode
> +# fs. Ref: btrfs-progs: btrfs_min_dev_size()
> +_scratch_mkfs_sized $((256 * 1024 * 1024)) >> $seqres.full 2>&1 \

Yeah, this test seems rather flaky with mixed block groups I had fixed
it and then Omar changed the mixed bg threshold to 1g and it started
failing again internally. I'm happy with this change provided the first
patch lands as well.

>      || _fail "mkfs failed"
>  _scratch_mount
>  out=$SCRATCH_MNT/fillup.$$
> 

Re: [PATCH v2 1/9] fstests: btrfs: _scratch_mkfs_sized fix min size without mixed option

[Anand Jain]

On 09/25/2018 06:51 PM, Nikolay Borisov wrote:
> 
> 
> On 25.09.2018 07:24, Anand Jain wrote:
>> As of now _scratch_mkfs_sized() checks if the requested size is below 1G
>> and forces the --mixed option for the mkfs.btrfs. Well the correct size
>> considering all possible group profiles at which we need to force the
>> mixed option is roughly 256Mbytes. So fix that.
>>
>> Signed-off-by: Anand Jain <anand.jain@oracle.com>
> 
> Have you considered the implications of this w.r.t commit
> d4da414a9a9d ("common/rc: raise btrfs mixed mode threshold to 1GB")
 >
> Initially this threshold was 100mb then Omar changed it to 1g. Does this
> change affect generic/427?

d4da414a9a9d does not explain what was the problem that Omar wanted to 
address, mainly what was the failure about.

And no it does not affect. I have verified generic/427 with kernel 4.1 
and 4.19-rc5 with  btrfs-progs 4.1, 4.9 and latest from kdave they all 
run fine. Good to integrate.

Thanks, Anand

> 
>> ---
>>   common/rc | 5 ++++-
>>   1 file changed, 4 insertions(+), 1 deletion(-)
>>
>> diff --git a/common/rc b/common/rc
>> index d5bb1feee2c3..90dc3002bc3d 100644
>> --- a/common/rc
>> +++ b/common/rc
>> @@ -969,7 +969,10 @@ _scratch_mkfs_sized()
>>   	;;
>>       btrfs)
>>   	local mixed_opt=
>> -	(( fssize <= 1024 * 1024 * 1024 )) && mixed_opt='--mixed'
>> +	# minimum size that's needed without the mixed option.
>> +	# Ref: btrfs-prog: btrfs_min_dev_size()
>> +	# Non mixed mode is also the default option.
>> +	(( fssize < $((256 * 1024 *1024)) )) && mixed_opt='--mixed'
>>   	$MKFS_BTRFS_PROG $MKFS_OPTIONS $mixed_opt -b $fssize $SCRATCH_DEV
>>   	;;
>>       jfs)
>>

Re: [PATCH v2 5/9] generic/102 open code dev_size _scratch_mkfs_sized()

[Anand Jain]

On 09/25/2018 06:54 PM, Nikolay Borisov wrote:
> 
> 
> On 25.09.2018 07:24, Anand Jain wrote:
>> Open code helps to grep and find out parameter sent to the
>> _scratch_mkfs_sized here.
>>
>> Signed-off-by: Anand Jain <anand.jain@oracle.com>
> 
> IMO this is noise, you can just as simply do
> "grep _scratch_mkfs_sized" and then open the file to inspect the actual
> argument. But it's up to the xfstest maintainers

  I am ok. Its just a nice cleanup.

Thanks, Anand

>> ---
>>   tests/generic/102 | 3 +--
>>   1 file changed, 1 insertion(+), 2 deletions(-)
>>
>> diff --git a/tests/generic/102 b/tests/generic/102
>> index faf940ac5070..aad496a5bc69 100755
>> --- a/tests/generic/102
>> +++ b/tests/generic/102
>> @@ -31,8 +31,7 @@ _require_scratch
>>   
>>   rm -f $seqres.full
>>   
>> -dev_size=$((512 * 1024 * 1024))     # 512MB filesystem
>> -_scratch_mkfs_sized $dev_size >>$seqres.full 2>&1
>> +_scratch_mkfs_sized $((512 * 1024 * 1024)) >>$seqres.full 2>&1
>>   _scratch_mount
>>   
>>   for ((i = 0; i < 10; i++)); do
>>

Re: [PATCH v2 1/9] fstests: btrfs: _scratch_mkfs_sized fix min size without mixed option

[Nikolay Borisov]

On 26.09.2018 07:07, Anand Jain wrote:
> 
> 
> On 09/25/2018 06:51 PM, Nikolay Borisov wrote:
>>
>>
>> On 25.09.2018 07:24, Anand Jain wrote:
>>> As of now _scratch_mkfs_sized() checks if the requested size is below 1G
>>> and forces the --mixed option for the mkfs.btrfs. Well the correct size
>>> considering all possible group profiles at which we need to force the
>>> mixed option is roughly 256Mbytes. So fix that.
>>>
>>> Signed-off-by: Anand Jain <anand.jain@oracle.com>
>>
>> Have you considered the implications of this w.r.t commit
>> d4da414a9a9d ("common/rc: raise btrfs mixed mode threshold to 1GB")
>>
>> Initially this threshold was 100mb then Omar changed it to 1g. Does this
>> change affect generic/427?
> 
> d4da414a9a9d does not explain what was the problem that Omar wanted to
> address, mainly what was the failure about.

I just retested on upstream 4.19.0-rc3 with Omar's patch reverted (so
anything above 100m for fs size is created with non-mixed block groups)
and the test succeeded. So indeed your change seems to not make a
difference for this test.

> 
> And no it does not affect. I have verified generic/427 with kernel 4.1
> and 4.19-rc5 with  btrfs-progs 4.1, 4.9 and latest from kdave they all
> run fine. Good to integrate.
> 
> Thanks, Anand
> 
>>
>>> ---
>>>   common/rc | 5 ++++-
>>>   1 file changed, 4 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/common/rc b/common/rc
>>> index d5bb1feee2c3..90dc3002bc3d 100644
>>> --- a/common/rc
>>> +++ b/common/rc
>>> @@ -969,7 +969,10 @@ _scratch_mkfs_sized()
>>>       ;;
>>>       btrfs)
>>>       local mixed_opt=
>>> -    (( fssize <= 1024 * 1024 * 1024 )) && mixed_opt='--mixed'
>>> +    # minimum size that's needed without the mixed option.
>>> +    # Ref: btrfs-prog: btrfs_min_dev_size()
>>> +    # Non mixed mode is also the default option.
>>> +    (( fssize < $((256 * 1024 *1024)) )) && mixed_opt='--mixed'
>>>       $MKFS_BTRFS_PROG $MKFS_OPTIONS $mixed_opt -b $fssize $SCRATCH_DEV
>>>       ;;
>>>       jfs)
>>>
> 

Re: [PATCH v2 1/9] fstests: btrfs: _scratch_mkfs_sized fix min size without mixed option

[Anand Jain]

On 09/26/2018 02:34 PM, Nikolay Borisov wrote:
> 
> 
> On 26.09.2018 07:07, Anand Jain wrote:
>>
>>
>> On 09/25/2018 06:51 PM, Nikolay Borisov wrote:
>>>
>>>
>>> On 25.09.2018 07:24, Anand Jain wrote:
>>>> As of now _scratch_mkfs_sized() checks if the requested size is below 1G
>>>> and forces the --mixed option for the mkfs.btrfs. Well the correct size
>>>> considering all possible group profiles at which we need to force the
>>>> mixed option is roughly 256Mbytes. So fix that.
>>>>
>>>> Signed-off-by: Anand Jain <anand.jain@oracle.com>
>>>
>>> Have you considered the implications of this w.r.t commit
>>> d4da414a9a9d ("common/rc: raise btrfs mixed mode threshold to 1GB")
>>>
>>> Initially this threshold was 100mb then Omar changed it to 1g. Does this
>>> change affect generic/427?
>>
>> d4da414a9a9d does not explain what was the problem that Omar wanted to
>> address, mainly what was the failure about.
> 
> I just retested on upstream 4.19.0-rc3 with Omar's patch reverted (so
> anything above 100m for fs size is created with non-mixed block groups)
> and the test succeeded. So indeed your change seems to not make a
> difference for this test.

  Thanks for testing.
Anand

>>
>> And no it does not affect. I have verified generic/427 with kernel 4.1
>> and 4.19-rc5 with  btrfs-progs 4.1, 4.9 and latest from kdave they all
>> run fine. Good to integrate.
>>
>> Thanks, Anand
>>
>>>
>>>> ---
>>>>    common/rc | 5 ++++-
>>>>    1 file changed, 4 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/common/rc b/common/rc
>>>> index d5bb1feee2c3..90dc3002bc3d 100644
>>>> --- a/common/rc
>>>> +++ b/common/rc
>>>> @@ -969,7 +969,10 @@ _scratch_mkfs_sized()
>>>>        ;;
>>>>        btrfs)
>>>>        local mixed_opt=
>>>> -    (( fssize <= 1024 * 1024 * 1024 )) && mixed_opt='--mixed'
>>>> +    # minimum size that's needed without the mixed option.
>>>> +    # Ref: btrfs-prog: btrfs_min_dev_size()
>>>> +    # Non mixed mode is also the default option.
>>>> +    (( fssize < $((256 * 1024 *1024)) )) && mixed_opt='--mixed'
>>>>        $MKFS_BTRFS_PROG $MKFS_OPTIONS $mixed_opt -b $fssize $SCRATCH_DEV
>>>>        ;;
>>>>        jfs)
>>>>
>>

Re: [PATCH v2 1/9] fstests: btrfs: _scratch_mkfs_sized fix min size without mixed option

[Omar Sandoval]

On Wed, Sep 26, 2018 at 09:34:27AM +0300, Nikolay Borisov wrote:
> 
> 
> On 26.09.2018 07:07, Anand Jain wrote:
> > 
> > 
> > On 09/25/2018 06:51 PM, Nikolay Borisov wrote:
> >>
> >>
> >> On 25.09.2018 07:24, Anand Jain wrote:
> >>> As of now _scratch_mkfs_sized() checks if the requested size is below 1G
> >>> and forces the --mixed option for the mkfs.btrfs. Well the correct size
> >>> considering all possible group profiles at which we need to force the
> >>> mixed option is roughly 256Mbytes. So fix that.
> >>>
> >>> Signed-off-by: Anand Jain <anand.jain@oracle.com>
> >>
> >> Have you considered the implications of this w.r.t commit
> >> d4da414a9a9d ("common/rc: raise btrfs mixed mode threshold to 1GB")
> >>
> >> Initially this threshold was 100mb then Omar changed it to 1g. Does this
> >> change affect generic/427?
> > 
> > d4da414a9a9d does not explain what was the problem that Omar wanted to
> > address, mainly what was the failure about.
> 
> I just retested on upstream 4.19.0-rc3 with Omar's patch reverted (so
> anything above 100m for fs size is created with non-mixed block groups)
> and the test succeeded. So indeed your change seems to not make a
> difference for this test.
> 
> > 
> > And no it does not affect. I have verified generic/427 with kernel 4.1
> > and 4.19-rc5 with  btrfs-progs 4.1, 4.9 and latest from kdave they all
> > run fine. Good to integrate.

I had to double check, but it only happens with -m dup. If I apply the
following patch:

diff --git a/common/rc b/common/rc
index d5bb1fe..989b846 100644
--- a/common/rc
+++ b/common/rc
@@ -969,7 +969,7 @@ _scratch_mkfs_sized()
 	;;
     btrfs)
 	local mixed_opt=
-	(( fssize <= 1024 * 1024 * 1024 )) && mixed_opt='--mixed'
+	(( fssize <= 100 * 1024 * 1024 )) && mixed_opt='--mixed'
 	$MKFS_BTRFS_PROG $MKFS_OPTIONS $mixed_opt -b $fssize $SCRATCH_DEV
 	;;
     jfs)
diff --git a/tests/generic/427 b/tests/generic/427
index e8ebffe..206cf08 100755
--- a/tests/generic/427
+++ b/tests/generic/427
@@ -65,6 +65,7 @@ fi
 # start a background aio writer, which does several extending loops
 # internally and check data integrality
 $AIO_TEST -s $fsize -b 65536 $SCRATCH_MNT/tst-aio-dio-eof-race.$seq
+btrfs fi usage $SCRATCH_MNT
 status=$?
 
 kill $open_close_pid

And run with MKFS_OPTIONS="-m dup", then we don't have enough data space
for the test:

--- /root/linux/xfstests/tests/generic/427.out	2017-11-28 16:05:46.811435644 -0800
+++ /root/linux/xfstests/results/generic/427.out.bad	2018-09-27 13:01:00.540510385 -0700
@@ -1,2 +1,24 @@
 QA output created by 427
-Success, all done.
+pwrite: No space left on device
+Overall:
+    Device size:		 256.00MiB
+    Device allocated:		 255.00MiB
+    Device unallocated:		   1.00MiB
+    Device missing:		     0.00B
+    Used:			 179.03MiB
+    Free (estimated):		     0.00B	(min: 0.00B)
+    Data ratio:			      1.00
+    Metadata ratio:		      2.00
+    Global reserve:		  16.00MiB	(used: 0.00B)
+
+Data,single: Size:175.00MiB, Used:175.00MiB
+   /dev/nvme0n1p2	 175.00MiB
+
+Metadata,DUP: Size:32.00MiB, Used:2.00MiB
+   /dev/nvme0n1p2	  64.00MiB
+
+System,DUP: Size:8.00MiB, Used:16.00KiB
+   /dev/nvme0n1p2	  16.00MiB
+
+Unallocated:
+   /dev/nvme0n1p2	   1.00MiB

Re: [PATCH v2 1/9] fstests: btrfs: _scratch_mkfs_sized fix min size without mixed option

[Anand Jain]

On 09/28/2018 04:07 AM, Omar Sandoval wrote:
> On Wed, Sep 26, 2018 at 09:34:27AM +0300, Nikolay Borisov wrote:
>>
>>
>> On 26.09.2018 07:07, Anand Jain wrote:
>>>
>>>
>>> On 09/25/2018 06:51 PM, Nikolay Borisov wrote:
>>>>
>>>>
>>>> On 25.09.2018 07:24, Anand Jain wrote:
>>>>> As of now _scratch_mkfs_sized() checks if the requested size is below 1G
>>>>> and forces the --mixed option for the mkfs.btrfs. Well the correct size
>>>>> considering all possible group profiles at which we need to force the
>>>>> mixed option is roughly 256Mbytes. So fix that.
>>>>>
>>>>> Signed-off-by: Anand Jain <anand.jain@oracle.com>
>>>>
>>>> Have you considered the implications of this w.r.t commit
>>>> d4da414a9a9d ("common/rc: raise btrfs mixed mode threshold to 1GB")
>>>>
>>>> Initially this threshold was 100mb then Omar changed it to 1g. Does this
>>>> change affect generic/427?
>>>
>>> d4da414a9a9d does not explain what was the problem that Omar wanted to
>>> address, mainly what was the failure about.
>>
>> I just retested on upstream 4.19.0-rc3 with Omar's patch reverted (so
>> anything above 100m for fs size is created with non-mixed block groups)
>> and the test succeeded. So indeed your change seems to not make a
>> difference for this test.
>>
>>>
>>> And no it does not affect. I have verified generic/427 with kernel 4.1
>>> and 4.19-rc5 with  btrfs-progs 4.1, 4.9 and latest from kdave they all
>>> run fine. Good to integrate.
> 
> I had to double check, but it only happens with -m dup. If I apply the
> following patch:
> 
> diff --git a/common/rc b/common/rc
> index d5bb1fe..989b846 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -969,7 +969,7 @@ _scratch_mkfs_sized()
>   	;;
>       btrfs)
>   	local mixed_opt=
> -	(( fssize <= 1024 * 1024 * 1024 )) && mixed_opt='--mixed'
> +	(( fssize <= 100 * 1024 * 1024 )) && mixed_opt='--mixed'
>   	$MKFS_BTRFS_PROG $MKFS_OPTIONS $mixed_opt -b $fssize $SCRATCH_DEV
>   	;;
>       jfs)
> diff --git a/tests/generic/427 b/tests/generic/427
> index e8ebffe..206cf08 100755
> --- a/tests/generic/427
> +++ b/tests/generic/427
> @@ -65,6 +65,7 @@ fi
>   # start a background aio writer, which does several extending loops
>   # internally and check data integrality
>   $AIO_TEST -s $fsize -b 65536 $SCRATCH_MNT/tst-aio-dio-eof-race.$seq
> +btrfs fi usage $SCRATCH_MNT
>   status=$?
>   
>   kill $open_close_pid
> 
> And run with MKFS_OPTIONS="-m dup", then we don't have enough data space
> for the test:
> 
> --- /root/linux/xfstests/tests/generic/427.out	2017-11-28 16:05:46.811435644 -0800
> +++ /root/linux/xfstests/results/generic/427.out.bad	2018-09-27 13:01:00.540510385 -0700
> @@ -1,2 +1,24 @@
>   QA output created by 427
> -Success, all done.
> +pwrite: No space left on device

Thanks Omar. Unfortunately I can't reproduce with the diff as above + 
MKFS_OPTIONS="-m dup".

In any case the objective of this patch is to ensure 
_scratch_mkfs_sized() provides default group profile with the minimum 
disk size that's actually be required. And related to that there isn't 
any issue in this patch.

Thanks, Anand

> +Overall:
> +    Device size:		 256.00MiB
> +    Device allocated:		 255.00MiB
> +    Device unallocated:		   1.00MiB
> +    Device missing:		     0.00B
> +    Used:			 179.03MiB
> +    Free (estimated):		     0.00B	(min: 0.00B)
> +    Data ratio:			      1.00
> +    Metadata ratio:		      2.00
> +    Global reserve:		  16.00MiB	(used: 0.00B)
> +
> +Data,single: Size:175.00MiB, Used:175.00MiB
> +   /dev/nvme0n1p2	 175.00MiB
> +
> +Metadata,DUP: Size:32.00MiB, Used:2.00MiB
> +   /dev/nvme0n1p2	  64.00MiB
> +
> +System,DUP: Size:8.00MiB, Used:16.00KiB
> +   /dev/nvme0n1p2	  16.00MiB
> +
> +Unallocated:
> +   /dev/nvme0n1p2	   1.00MiB
> 

Re: [PATCH v2 3/9] geneirc/077 fix min size for btrfs

[Eryu Guan]

On Tue, Sep 25, 2018 at 12:24:16PM +0800, Anand Jain wrote:
> If btrfs need to be tested at its default blockgroup which is non-mixed,
> then it needs at least 256mb.
> 
> Signed-off-by: Anand Jain <anand.jain@oracle.com>

(Sorry for the late review..)

> ---
>  tests/generic/077 | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/tests/generic/077 b/tests/generic/077
> index ef6af18c83e3..ec236992513f 100755
> --- a/tests/generic/077
> +++ b/tests/generic/077
> @@ -49,8 +49,7 @@ rm -f $seqres.full
>  _scratch_unmount >/dev/null 2>&1
>  echo "*** MKFS ***"                         >>$seqres.full
>  echo ""                                     >>$seqres.full
> -SIZE=`expr 50 \* 1024 \* 1024`
> -_scratch_mkfs_sized $SIZE                   >>$seqres.full 2>&1 \
> +_scratch_mkfs_sized $((256 * 1024 *1024))   >>$seqres.full 2>&1 \
>  	|| _fail "mkfs failed"

Hmm, this test copies "/lib/modules/" to fill the original 50M
filesystem (which seems a bad way to me)

"
# Something w/ enough data to fill 50M of fs...
filler=/lib/modules/
...
echo "*** populate filesystem, pass #1" | tee -a $seqres.full
cp -rf $filler $SCRATCH_MNT/subdir >$seqres.full 2>&1
...
"

It works most of the time as "/lib/modules" is usually larger than 50M,
but it may not fullfil the fs with 256M size.

I think we should fix the way to fill the fs too.

Thanks,
Eryu

>  _scratch_mount
>  mkdir $SCRATCH_MNT/subdir
> -- 
> 1.8.3.1
> 

Re: [PATCH v2 5/9] generic/102 open code dev_size _scratch_mkfs_sized()

[Eryu Guan]

On Wed, Sep 26, 2018 at 12:08:56PM +0800, Anand Jain wrote:
> 
> 
> On 09/25/2018 06:54 PM, Nikolay Borisov wrote:
> > 
> > 
> > On 25.09.2018 07:24, Anand Jain wrote:
> > > Open code helps to grep and find out parameter sent to the
> > > _scratch_mkfs_sized here.
> > > 
> > > Signed-off-by: Anand Jain <anand.jain@oracle.com>
> > 
> > IMO this is noise, you can just as simply do
> > "grep _scratch_mkfs_sized" and then open the file to inspect the actual
> > argument. But it's up to the xfstest maintainers
> 
>  I am ok. Its just a nice cleanup.
> 
> Thanks, Anand

I prefer dropping patch 5/6/7, as I don't think they're that necessary.

BTW, other patches from this series but patch 3 ("geneirc/077 fix min
size for btrfs") look fine to me, I'm taking them in this week's update.

Thanks,
Eryu

> 
> > > ---
> > >   tests/generic/102 | 3 +--
> > >   1 file changed, 1 insertion(+), 2 deletions(-)
> > > 
> > > diff --git a/tests/generic/102 b/tests/generic/102
> > > index faf940ac5070..aad496a5bc69 100755
> > > --- a/tests/generic/102
> > > +++ b/tests/generic/102
> > > @@ -31,8 +31,7 @@ _require_scratch
> > >   rm -f $seqres.full
> > > -dev_size=$((512 * 1024 * 1024))     # 512MB filesystem
> > > -_scratch_mkfs_sized $dev_size >>$seqres.full 2>&1
> > > +_scratch_mkfs_sized $((512 * 1024 * 1024)) >>$seqres.full 2>&1
> > >   _scratch_mount
> > >   for ((i = 0; i < 10; i++)); do
> > > 

Re: [PATCH v2 3/9] geneirc/077 fix min size for btrfs

[Anand Jain]

On 10/06/2018 07:25 PM, Eryu Guan wrote:
> On Tue, Sep 25, 2018 at 12:24:16PM +0800, Anand Jain wrote:
>> If btrfs need to be tested at its default blockgroup which is non-mixed,
>> then it needs at least 256mb.
>>
>> Signed-off-by: Anand Jain <anand.jain@oracle.com>
> 
> (Sorry for the late review..)

  Its fine. Sorry for the delay at my end too.

>> ---
>>   tests/generic/077 | 3 +--
>>   1 file changed, 1 insertion(+), 2 deletions(-)
>>
>> diff --git a/tests/generic/077 b/tests/generic/077
>> index ef6af18c83e3..ec236992513f 100755
>> --- a/tests/generic/077
>> +++ b/tests/generic/077
>> @@ -49,8 +49,7 @@ rm -f $seqres.full
>>   _scratch_unmount >/dev/null 2>&1
>>   echo "*** MKFS ***"                         >>$seqres.full
>>   echo ""                                     >>$seqres.full
>> -SIZE=`expr 50 \* 1024 \* 1024`
>> -_scratch_mkfs_sized $SIZE                   >>$seqres.full 2>&1 \
>> +_scratch_mkfs_sized $((256 * 1024 *1024))   >>$seqres.full 2>&1 \
>>   	|| _fail "mkfs failed"
> 
> Hmm, this test copies "/lib/modules/" to fill the original 50M
> filesystem (which seems a bad way to me)

Yep. I didn't like either. I started using the _populate_fs() but then 
was doubtful if it still meets the objective of this test case as it 
used the dd instead of cp. Moreover it kind of getting diverged from 
what's the object of this patch. If needed we could fix the cp approach 
in a separate patch so that we shall have better tracking.

> "
> # Something w/ enough data to fill 50M of fs...
> filler=/lib/modules/
> ...
> echo "*** populate filesystem, pass #1" | tee -a $seqres.full
> cp -rf $filler $SCRATCH_MNT/subdir >$seqres.full 2>&1
> ...
> "
> 
> It works most of the time as "/lib/modules" is usually larger than 50M,
> but it may not fullfil the fs with 256M size.
> 
> I think we should fix the way to fill the fs too.

  Right. There are two iterations of cp. So a check if the assigned 
filler is big enough to fill the 256M will suffice, as we already 
_notrun the test if empty filler.

Pls find v2 for review.

Thanks, Anand

> Thanks,
> Eryu
> 
>>   _scratch_mount
>>   mkdir $SCRATCH_MNT/subdir
>> -- 
>> 1.8.3.1
>>

Re: [PATCH v2 3/9] geneirc/077 fix min size for btrfs

[Anand Jain]

On 10/11/2018 11:14 AM, Anand Jain wrote:
> 
> 
> On 10/06/2018 07:25 PM, Eryu Guan wrote:
>> On Tue, Sep 25, 2018 at 12:24:16PM +0800, Anand Jain wrote:
>>> If btrfs need to be tested at its default blockgroup which is non-mixed,
>>> then it needs at least 256mb.
>>>
>>> Signed-off-by: Anand Jain <anand.jain@oracle.com>
>>
>> (Sorry for the late review..)
> 
>   Its fine. Sorry for the delay at my end too.
> 
>>> ---
>>>   tests/generic/077 | 3 +--
>>>   1 file changed, 1 insertion(+), 2 deletions(-)
>>>
>>> diff --git a/tests/generic/077 b/tests/generic/077
>>> index ef6af18c83e3..ec236992513f 100755
>>> --- a/tests/generic/077
>>> +++ b/tests/generic/077
>>> @@ -49,8 +49,7 @@ rm -f $seqres.full
>>>   _scratch_unmount >/dev/null 2>&1
>>>   echo "*** MKFS ***"                         >>$seqres.full
>>>   echo ""                                     >>$seqres.full
>>> -SIZE=`expr 50 \* 1024 \* 1024`
>>> -_scratch_mkfs_sized $SIZE                   >>$seqres.full 2>&1 \
>>> +_scratch_mkfs_sized $((256 * 1024 *1024))   >>$seqres.full 2>&1 \
>>>       || _fail "mkfs failed"
>>
>> Hmm, this test copies "/lib/modules/" to fill the original 50M
>> filesystem (which seems a bad way to me)
> 
> Yep. I didn't like either. I started using the _populate_fs() but then 
> was doubtful if it still meets the objective of this test case as it 
> used the dd instead of cp. Moreover it kind of getting diverged from 
> what's the object of this patch. If needed we could fix the cp approach 
> in a separate patch so that we shall have better tracking.
> 
>> "
>> # Something w/ enough data to fill 50M of fs...
>> filler=/lib/modules/
>> ...
>> echo "*** populate filesystem, pass #1" | tee -a $seqres.full
>> cp -rf $filler $SCRATCH_MNT/subdir >$seqres.full 2>&1
>> ...
>> "
>>
>> It works most of the time as "/lib/modules" is usually larger than 50M,
>> but it may not fullfil the fs with 256M size.
>>
>> I think we should fix the way to fill the fs too.
> 
>   Right. There are two iterations of cp. So a check if the assigned 
> filler is big enough to fill the 256M will suffice, as we already 
> _notrun the test if empty filler.
> 
> Pls find v2 for review.

  Sorry for the typo it should be v3.

Thanks, Anand

> Thanks, Anand
> 
>> Thanks,
>> Eryu
>>
>>>   _scratch_mount
>>>   mkdir $SCRATCH_MNT/subdir
>>> -- 
>>> 1.8.3.1
>>>