question of xfs/148 and xfs/149

question of xfs/148 and xfs/149

[Xu, Yang]

HI All

When I investigated xfs/030 failure on upstream kernel after mering xfstests commit d0e484ac699f
("check: wipe scratch devices between tests"), I found two similar cases(xfs/148,xfs/149).

xfs/148 is a clone of test 030 using xfs_prepair64 instead of xfs_repair.
xfs/149 is a clone of test 031 using xfs_prepair instead of xfs_repair

But I don't find these two commands and know nothing about them.  If these commands have been
obsoleted long time ago, I think we can remove the two cases. Or may I miss something?

Thanks
Yang Xu

Re: question of xfs/148 and xfs/149

[Darrick J. Wong]

[add linux-xfs to cc]

On Tue, Sep 17, 2019 at 09:00:57AM +0000, Xu, Yang wrote:
> HI All
> 
> When I investigated xfs/030 failure on upstream kernel after mering
> xfstests commit d0e484ac699f ("check: wipe scratch devices between
> tests"), I found two similar cases(xfs/148,xfs/149).
> 
> xfs/148 is a clone of test 030 using xfs_prepair64 instead of xfs_repair.
> xfs/149 is a clone of test 031 using xfs_prepair instead of xfs_repair
> 
> But I don't find these two commands and know nothing about them.  If
> these commands have been obsoleted long time ago, I think we can
> remove the two cases. Or may I miss something?

<shrug> I think your analysis is correct, but let's see what the xfs
list thinks.

--D

> 
> Thanks
> Yang Xu
> 
> 

Re: question of xfs/148 and xfs/149

[Zorro Lang]

On Tue, Sep 17, 2019 at 09:39:33AM -0700, Darrick J. Wong wrote:
> [add linux-xfs to cc]
> 
> On Tue, Sep 17, 2019 at 09:00:57AM +0000, Xu, Yang wrote:
> > HI All
> > 
> > When I investigated xfs/030 failure on upstream kernel after mering
> > xfstests commit d0e484ac699f ("check: wipe scratch devices between
> > tests"), I found two similar cases(xfs/148,xfs/149).

xfs/030 is weird, I've found it long time ago.

If I do a 'whole disk mkfs' (_scratch_mkfs_xfs), before this sized mkfs:

  _scratch_mkfs_xfs $DSIZE >/dev/null 2>&1

Everything looks clear, and test pass. I can't send a patch to do this,
because I don't know the reason.

I'm not familiar with xfs_repair so much, so I don't know what happens
underlying. I suppose the the part after the $DSIZE affect the xfs_repair,
but I don't know why the wipefs can cause that, wipefs only erase 4 bytes
at the beginning.

Darrick, do you know more about that?

Thanks,
Zorro

> > 
> > xfs/148 is a clone of test 030 using xfs_prepair64 instead of xfs_repair.
> > xfs/149 is a clone of test 031 using xfs_prepair instead of xfs_repair

I'm not worried about it too much, due to it always 'not run' and never
fails:)

xfs/148 [not run] parallel repair binary xfs_prepair64 is not installed
xfs/149 [not run] parallel repair binary xfs_prepair is not installed
Ran: xfs/148 xfs/149
Not run: xfs/148 xfs/149
Passed all 2 tests

> > 
> > But I don't find these two commands and know nothing about them.  If
> > these commands have been obsoleted long time ago, I think we can
> > remove the two cases. Or may I miss something?
> 
> <shrug> I think your analysis is correct, but let's see what the xfs
> list thinks.
> 
> --D
> 
> > 
> > Thanks
> > Yang Xu
> > 
> > 

Re: question of xfs/148 and xfs/149

[Yang Xu]

on 2019/09/18 10:59, Zorro Lang wrote:
> xfs/030 is weird, I've found it long time ago.
> 
> If I do a 'whole disk mkfs' (_scratch_mkfs_xfs), before this sized mkfs:
> 
>    _scratch_mkfs_xfs $DSIZE >/dev/null 2>&1
> 
> Everything looks clear, and test pass. I can't send a patch to do this,
> because I don't know the reason.
Yes. I also found running _scratch_mkfs_xfs in xfs/030 can slove this 
problem yesterday. Or, we can adjust _try_wipe_scratch_devs order in 
check(But I dont't have enough reason to explain why adjust it). as below:
--- a/check
+++ b/check
@@ -753,7 +753,6 @@ for section in $HOST_OPTIONS_SECTIONS; do
                         # _check_dmesg depends on this log in dmesg
                         touch ${RESULT_DIR}/check_dmesg
                 fi
-               _try_wipe_scratch_devs > /dev/null 2>&1
                 if [ "$DUMP_OUTPUT" = true ]; then
                         _run_seq 2>&1 | tee $tmp.out
                         # Because $? would get tee's return code
@@ -799,7 +798,7 @@ for section in $HOST_OPTIONS_SECTIONS; do
                 # Scan for memory leaks after every test so that 
associating
                 # a leak to a particular test will be as accurate as 
possible.
                 _check_kmemleak || err=true
-
+               _try_wipe_scratch_devs > /dev/null 2>&1
                 # test ends after all checks are done.
                 $timestamp && _timestamp
                 stop=`_wallclock`

> 
> I'm not familiar with xfs_repair so much, so I don't know what happens
> underlying. I suppose the the part after the $DSIZE affect the xfs_repair,
> but I don't know why the wipefs can cause that, wipefs only erase 4 bytes
> at the beginning.
> 
  I am finding the reasion. It seems wipefs wipes important information 
and $DSIZE option(using single agcount or dsize, it also fails ) can not 
format disk completely. If we use other options, it can pass.
> Darrick, do you know more about that?
> 
> Thanks,
> Zorro
> 
>>> xfs/148 is a clone of test 030 using xfs_prepair64 instead of xfs_repair.
>>> xfs/149 is a clone of test 031 using xfs_prepair instead of xfs_repair
> I'm not worried about it too much, due to it always 'not run' and never
> failsYes. But I perfer to remove them because IMO they are useless.
> 

> xfs/148 [not run] parallel repair binary xfs_prepair64 is not installed
> xfs/149 [not run] parallel repair binary xfs_prepair is not installed
> Ran: xfs/148 xfs/149
> Not run: xfs/148 xfs/149
> Passed all 2 tests
> 

Re: question of xfs/148 and xfs/149

[Darrick J. Wong]

On Wed, Sep 18, 2019 at 11:24:47AM +0800, Yang Xu wrote:
> 
> 
> on 2019/09/18 10:59, Zorro Lang wrote:
> > xfs/030 is weird, I've found it long time ago.
> > 
> > If I do a 'whole disk mkfs' (_scratch_mkfs_xfs), before this sized mkfs:
> > 
> >    _scratch_mkfs_xfs $DSIZE >/dev/null 2>&1
> > 
> > Everything looks clear, and test pass. I can't send a patch to do this,
> > because I don't know the reason.
> Yes. I also found running _scratch_mkfs_xfs in xfs/030 can slove this
> problem yesterday. Or, we can adjust _try_wipe_scratch_devs order in
> check(But I dont't have enough reason to explain why adjust it). as below:

(Yeah, I don't see any obvious reason why that would change outcomes...)

> --- a/check
> +++ b/check
> @@ -753,7 +753,6 @@ for section in $HOST_OPTIONS_SECTIONS; do
>                         # _check_dmesg depends on this log in dmesg
>                         touch ${RESULT_DIR}/check_dmesg
>                 fi
> -               _try_wipe_scratch_devs > /dev/null 2>&1
>                 if [ "$DUMP_OUTPUT" = true ]; then
>                         _run_seq 2>&1 | tee $tmp.out
>                         # Because $? would get tee's return code
> @@ -799,7 +798,7 @@ for section in $HOST_OPTIONS_SECTIONS; do
>                 # Scan for memory leaks after every test so that associating
>                 # a leak to a particular test will be as accurate as
> possible.
>                 _check_kmemleak || err=true
> -
> +               _try_wipe_scratch_devs > /dev/null 2>&1
>                 # test ends after all checks are done.
>                 $timestamp && _timestamp
>                 stop=`_wallclock`
> 
> > 
> > I'm not familiar with xfs_repair so much, so I don't know what happens
> > underlying. I suppose the the part after the $DSIZE affect the xfs_repair,
> > but I don't know why the wipefs can cause that, wipefs only erase 4 bytes
> > at the beginning.
> > 
>  I am finding the reasion. It seems wipefs wipes important information and
> $DSIZE option(using single agcount or dsize, it also fails ) can not format
> disk completely. If we use other options, it can pass.

How does mkfs fail, specifically?

Also, what's your storage configuration?  And lsblk -D output?

--D

> > Darrick, do you know more about that?
> > 
> > Thanks,
> > Zorro
> > 
> > > > xfs/148 is a clone of test 030 using xfs_prepair64 instead of xfs_repair.
> > > > xfs/149 is a clone of test 031 using xfs_prepair instead of xfs_repair
> > I'm not worried about it too much, due to it always 'not run' and never
> > failsYes. But I perfer to remove them because IMO they are useless.
> > 
> 
> > xfs/148 [not run] parallel repair binary xfs_prepair64 is not installed
> > xfs/149 [not run] parallel repair binary xfs_prepair is not installed
> > Ran: xfs/148 xfs/149
> > Not run: xfs/148 xfs/149
> > Passed all 2 tests
> > 
> 
> 

Re: question of xfs/148 and xfs/149

[Darrick J. Wong]

On Wed, Sep 18, 2019 at 09:37:11AM -0700, Darrick J. Wong wrote:
> On Wed, Sep 18, 2019 at 11:24:47AM +0800, Yang Xu wrote:
> > 
> > 
> > on 2019/09/18 10:59, Zorro Lang wrote:
> > > xfs/030 is weird, I've found it long time ago.
> > > 
> > > If I do a 'whole disk mkfs' (_scratch_mkfs_xfs), before this sized mkfs:
> > > 
> > >    _scratch_mkfs_xfs $DSIZE >/dev/null 2>&1
> > > 
> > > Everything looks clear, and test pass. I can't send a patch to do this,
> > > because I don't know the reason.
> > Yes. I also found running _scratch_mkfs_xfs in xfs/030 can slove this
> > problem yesterday. Or, we can adjust _try_wipe_scratch_devs order in
> > check(But I dont't have enough reason to explain why adjust it). as below:
> 
> (Yeah, I don't see any obvious reason why that would change outcomes...)
> 
> > --- a/check
> > +++ b/check
> > @@ -753,7 +753,6 @@ for section in $HOST_OPTIONS_SECTIONS; do
> >                         # _check_dmesg depends on this log in dmesg
> >                         touch ${RESULT_DIR}/check_dmesg
> >                 fi
> > -               _try_wipe_scratch_devs > /dev/null 2>&1
> >                 if [ "$DUMP_OUTPUT" = true ]; then
> >                         _run_seq 2>&1 | tee $tmp.out
> >                         # Because $? would get tee's return code
> > @@ -799,7 +798,7 @@ for section in $HOST_OPTIONS_SECTIONS; do
> >                 # Scan for memory leaks after every test so that associating
> >                 # a leak to a particular test will be as accurate as
> > possible.
> >                 _check_kmemleak || err=true
> > -
> > +               _try_wipe_scratch_devs > /dev/null 2>&1
> >                 # test ends after all checks are done.
> >                 $timestamp && _timestamp
> >                 stop=`_wallclock`
> > 
> > > 
> > > I'm not familiar with xfs_repair so much, so I don't know what happens
> > > underlying. I suppose the the part after the $DSIZE affect the xfs_repair,
> > > but I don't know why the wipefs can cause that, wipefs only erase 4 bytes
> > > at the beginning.
> > > 
> >  I am finding the reasion. It seems wipefs wipes important information and
> > $DSIZE option(using single agcount or dsize, it also fails ) can not format
> > disk completely. If we use other options, it can pass.
> 
> How does mkfs fail, specifically?
> 
> Also, what's your storage configuration?  And lsblk -D output?

I'm still interested in the answer to these questions, but I've done a
little more research and noticed that yes, xfs/030 fails if the device
doesn't support zeroing discard.

First, if mkfs.xfs detects an old primary superblock, it will write
zeroes to all superblocks before formatting the new filesystem.
Obviously this won't be done if the device doesn't have a primary
superblock.

(1) So let's say that a previous test formatted a 4GB scratch disk with
all defaults, and let's say that we have 4 AGs.  The disk will look like
this:

  SB0 [1G space] SB1 [1G space] SB2 [1G space] SB3 [1G space]

(2) Now we _try_wipe_scratch_devs, which wipes out the primary label:

  000 [1G space] SB1 [1G space] SB2 [1G space] SB3 [1G space]

(3) Now xfs/030 runs its special mkfs command (6AGs, 100MB disk).  If the
disk supports zeroing discard, it will discard the whole device:

  <4GB of zeroes>

(4) Then it will lay down its own filesystem:

  SB0 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 4G>

(5) Next, xfs/030 zaps the primary superblock:

  000 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 4G>

(6) Next, xfs/030 runs xfs_repair.  It fails to find the primary sb, so it
tries to find secondary superblocks.  Its first strategy is to compute
the fs geometry assuming all default options.  In this case, that means
4 AGs, spaced 1G apart.  They're all zero, so it falls back to a linear
scan of the disk.  It finds SB1, uses that to rewrite the primary super,
and continues with the repair (which is mostly uneventful).  The test
passes; this is why it works on my computer.

---------

Now let's see what happened before _try_wipe_scratch_devs.  In step (3)
mkfs would find the old superblocks and wipe the superblocks, before
laying down the new superblocks:

  SB0 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 1G> \
      000 [1G space] 000 [1G space] 000 [1G space]

Step (5) zaps the primary, yielding:

  000 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 1G> \
      000 [1G space] 000 [1G space] 000 [1G space]

Step (6) fails to find a primary superblock so it tries to read backup
superblocks at 1G, 2G, and 3G, but they're all zero so it falls back to
the linear scan and picks up SB1 and proceeds with a mostly uneventful
repair.  The test passes.

---------

However, with _try_wipe_scratch_devs and a device that doesn't support
discard (or MKFS_OPTIONS includes -K), we have a problem.  mkfs.xfs
doesn't discard the device nor does it find a primary superblock, so it
simply formats the new filesystem.  We end up with:

  SB0 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 1G> \
      SB'1 [1G space] SB'2 [1G space] SB'3 [1G space]

Where SB[0-5] are from the filesystem that xfs/030 formatted but
SB'[1-3] are from the filesystem that was on the scratch disk before
xfs/030 even started.  Uhoh.

Step (5) zaps the primary, yielding:

  000 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 1G> \
      SB'1 [1G space] SB'2 [1G space] SB'3 [1G space]

Step (6) fails to find a primary superblock so it tries to read backup
superblocks at 1G.  It finds SB'1 and uses that to reconstruct the /old/
filesystem, with what looks like massive filesystem damage.  This
results in test failure.  Oops.

----------

The reason for adding _try_wipe_scratch_devs was to detect broken tests
that started using the filesystem on the scratch device (if any) before
(or without!) formatting the scratch device.  That broken behavior could
result in spurious test failures when xfstests was run in random order
mode either due to mounting an unformatted device or mounting a corrupt
fs that some other test left behind.

I guess a fix for XFS would be have _try_wipe_scratch_devs try to read
the primary superblock to compute the AG geometry and then erase all
superblocks that could be on the disk; and then compute the default
geometry and wipe out all those superblocks too.

Does any of that square with what you've been seeing?

--D

> --D
> 
> > > Darrick, do you know more about that?
> > > 
> > > Thanks,
> > > Zorro
> > > 
> > > > > xfs/148 is a clone of test 030 using xfs_prepair64 instead of xfs_repair.
> > > > > xfs/149 is a clone of test 031 using xfs_prepair instead of xfs_repair
> > > I'm not worried about it too much, due to it always 'not run' and never
> > > failsYes. But I perfer to remove them because IMO they are useless.
> > > 
> > 
> > > xfs/148 [not run] parallel repair binary xfs_prepair64 is not installed
> > > xfs/149 [not run] parallel repair binary xfs_prepair is not installed
> > > Ran: xfs/148 xfs/149
> > > Not run: xfs/148 xfs/149
> > > Passed all 2 tests
> > > 
> > 
> > 

Re: question of xfs/148 and xfs/149

[Zorro Lang]

On Wed, Sep 18, 2019 at 04:10:50PM -0700, Darrick J. Wong wrote:
> On Wed, Sep 18, 2019 at 09:37:11AM -0700, Darrick J. Wong wrote:
> > On Wed, Sep 18, 2019 at 11:24:47AM +0800, Yang Xu wrote:
> > > 
> > > 
> > > on 2019/09/18 10:59, Zorro Lang wrote:
> > > > xfs/030 is weird, I've found it long time ago.
> > > > 
> > > > If I do a 'whole disk mkfs' (_scratch_mkfs_xfs), before this sized mkfs:
> > > > 
> > > >    _scratch_mkfs_xfs $DSIZE >/dev/null 2>&1
> > > > 
> > > > Everything looks clear, and test pass. I can't send a patch to do this,
> > > > because I don't know the reason.
> > > Yes. I also found running _scratch_mkfs_xfs in xfs/030 can slove this
> > > problem yesterday. Or, we can adjust _try_wipe_scratch_devs order in
> > > check(But I dont't have enough reason to explain why adjust it). as below:
> > 
> > (Yeah, I don't see any obvious reason why that would change outcomes...)
> > 
> > > --- a/check
> > > +++ b/check
> > > @@ -753,7 +753,6 @@ for section in $HOST_OPTIONS_SECTIONS; do
> > >                         # _check_dmesg depends on this log in dmesg
> > >                         touch ${RESULT_DIR}/check_dmesg
> > >                 fi
> > > -               _try_wipe_scratch_devs > /dev/null 2>&1
> > >                 if [ "$DUMP_OUTPUT" = true ]; then
> > >                         _run_seq 2>&1 | tee $tmp.out
> > >                         # Because $? would get tee's return code
> > > @@ -799,7 +798,7 @@ for section in $HOST_OPTIONS_SECTIONS; do
> > >                 # Scan for memory leaks after every test so that associating
> > >                 # a leak to a particular test will be as accurate as
> > > possible.
> > >                 _check_kmemleak || err=true
> > > -
> > > +               _try_wipe_scratch_devs > /dev/null 2>&1
> > >                 # test ends after all checks are done.
> > >                 $timestamp && _timestamp
> > >                 stop=`_wallclock`
> > > 
> > > > 
> > > > I'm not familiar with xfs_repair so much, so I don't know what happens
> > > > underlying. I suppose the the part after the $DSIZE affect the xfs_repair,
> > > > but I don't know why the wipefs can cause that, wipefs only erase 4 bytes
> > > > at the beginning.
> > > > 
> > >  I am finding the reasion. It seems wipefs wipes important information and
> > > $DSIZE option(using single agcount or dsize, it also fails ) can not format
> > > disk completely. If we use other options, it can pass.
> > 
> > How does mkfs fail, specifically?
> > 
> > Also, what's your storage configuration?  And lsblk -D output?
> 
> I'm still interested in the answer to these questions, but I've done a
> little more research and noticed that yes, xfs/030 fails if the device
> doesn't support zeroing discard.
> 
> First, if mkfs.xfs detects an old primary superblock, it will write
> zeroes to all superblocks before formatting the new filesystem.
> Obviously this won't be done if the device doesn't have a primary
> superblock.
> 
> (1) So let's say that a previous test formatted a 4GB scratch disk with
> all defaults, and let's say that we have 4 AGs.  The disk will look like
> this:
> 
>   SB0 [1G space] SB1 [1G space] SB2 [1G space] SB3 [1G space]
> 
> (2) Now we _try_wipe_scratch_devs, which wipes out the primary label:
> 
>   000 [1G space] SB1 [1G space] SB2 [1G space] SB3 [1G space]
> 
> (3) Now xfs/030 runs its special mkfs command (6AGs, 100MB disk).  If the
> disk supports zeroing discard, it will discard the whole device:
> 
>   <4GB of zeroes>
> 
> (4) Then it will lay down its own filesystem:
> 
>   SB0 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 4G>
> 
> (5) Next, xfs/030 zaps the primary superblock:
> 
>   000 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 4G>
> 
> (6) Next, xfs/030 runs xfs_repair.  It fails to find the primary sb, so it
> tries to find secondary superblocks.  Its first strategy is to compute
> the fs geometry assuming all default options.  In this case, that means
> 4 AGs, spaced 1G apart.  They're all zero, so it falls back to a linear
> scan of the disk.  It finds SB1, uses that to rewrite the primary super,
> and continues with the repair (which is mostly uneventful).  The test
> passes; this is why it works on my computer.
> 
> ---------
> 
> Now let's see what happened before _try_wipe_scratch_devs.  In step (3)
> mkfs would find the old superblocks and wipe the superblocks, before
> laying down the new superblocks:
> 
>   SB0 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 1G> \
>       000 [1G space] 000 [1G space] 000 [1G space]
> 
> Step (5) zaps the primary, yielding:
> 
>   000 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 1G> \
>       000 [1G space] 000 [1G space] 000 [1G space]
> 
> Step (6) fails to find a primary superblock so it tries to read backup
> superblocks at 1G, 2G, and 3G, but they're all zero so it falls back to
> the linear scan and picks up SB1 and proceeds with a mostly uneventful
> repair.  The test passes.
> 
> ---------
> 
> However, with _try_wipe_scratch_devs and a device that doesn't support
> discard (or MKFS_OPTIONS includes -K), we have a problem.  mkfs.xfs
> doesn't discard the device nor does it find a primary superblock, so it
> simply formats the new filesystem.  We end up with:
> 
>   SB0 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 1G> \
>       SB'1 [1G space] SB'2 [1G space] SB'3 [1G space]
> 
> Where SB[0-5] are from the filesystem that xfs/030 formatted but
> SB'[1-3] are from the filesystem that was on the scratch disk before
> xfs/030 even started.  Uhoh.
> 
> Step (5) zaps the primary, yielding:
> 
>   000 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 1G> \
>       SB'1 [1G space] SB'2 [1G space] SB'3 [1G space]
> 
> Step (6) fails to find a primary superblock so it tries to read backup
> superblocks at 1G.  It finds SB'1 and uses that to reconstruct the /old/
> filesystem, with what looks like massive filesystem damage.  This
> results in test failure.  Oops.
> 
> ----------
> 
> The reason for adding _try_wipe_scratch_devs was to detect broken tests
> that started using the filesystem on the scratch device (if any) before
> (or without!) formatting the scratch device.  That broken behavior could
> result in spurious test failures when xfstests was run in random order
> mode either due to mounting an unformatted device or mounting a corrupt
> fs that some other test left behind.
> 
> I guess a fix for XFS would be have _try_wipe_scratch_devs try to read
> the primary superblock to compute the AG geometry and then erase all
> superblocks that could be on the disk; and then compute the default
> geometry and wipe out all those superblocks too.
> 
> Does any of that square with what you've been seeing?

Thanks Darrick, so what I supposed might be true?
"
  > > > > I'm not familiar with xfs_repair so much, so I don't know what happens
  > > > > underlying. I suppose the the part after the $DSIZE affect the xfs_repair,
"

The sized mkfs.xfs (without discard) leave old on-disk structure behind $DSIZE
space, it cause xfs_repair try to use odd things to do the checking.

When I tried to erase the 1st block of each AGs[1], the test passed[2].
Is that what you talked as above?

Thanks,
Zorro

[1]
diff --git a/common/rc b/common/rc
index e0b087c1..19b7ab02 100644
--- a/common/rc
+++ b/common/rc
@@ -4048,6 +4048,10 @@ _try_wipe_scratch_devs()
        for dev in $SCRATCH_DEV_POOL $SCRATCH_DEV $SCRATCH_LOGDEV $SCRATCH_RTDEV; do
                test -b $dev && $WIPEFS_PROG -a $dev
        done
+
+       if [ "$FSTYP" = "xfs" ];then
+               _try_wipe_scratch_xfs
+       fi
 }
 
 # Only run this on xfs if xfs_scrub is available and has the unicode checker
diff --git a/common/xfs b/common/xfs
index 1bce3c18..53f33d12 100644
--- a/common/xfs
+++ b/common/xfs
@@ -884,3 +884,24 @@ _xfs_mount_agcount()
 {
        $XFS_INFO_PROG "$1" | grep agcount= | sed -e 's/^.*agcount=\([0-9]*\),.*$/\1/g'
 }
+
+_try_wipe_scratch_xfs()
+{
+       local tmp=`mktemp -u`
+
+       _scratch_mkfs_xfs -N 2>/dev/null | perl -ne '
+               if (/^meta-data=.*\s+agcount=(\d+), agsize=(\d+) blks/) {
+                       print STDOUT "agcount=$1\nagsize=$2\n";
+               }
+               if (/^data\s+=\s+bsize=(\d+)\s/) {
+                       print STDOUT "dbsize=$1\n";
+               }' > $tmp.mkfs
+       . $tmp.mkfs
+       if [ -n "$agcount" -a -n "$agsize" -a -n "$dbsize" ];then
+               for((i=0; i<agcount; i++)); do
+                       $XFS_IO_PROG -c "pwrite $((i * dbsize * agsize)) $dbsize" \
+                               $SCRATCH_DEV >/dev/null;
+               done
+       fi
+       rm -f $tmp.mkfs
+}

[2]
# ./check xfs/030
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 xxx-xxxx-xx xxx-xxxx-xx-xxx
MKFS_OPTIONS  -- -f -bsize=4096 /dev/mapper/scratchdev
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/mapper/scratchdev /mnt/scratch

xfs/030 24s ...  25s
Ran: xfs/030
Passed all 1 tests

> 
> --D
> 
> > --D
> > 
> > > > Darrick, do you know more about that?
> > > > 
> > > > Thanks,
> > > > Zorro
> > > > 
> > > > > > xfs/148 is a clone of test 030 using xfs_prepair64 instead of xfs_repair.
> > > > > > xfs/149 is a clone of test 031 using xfs_prepair instead of xfs_repair
> > > > I'm not worried about it too much, due to it always 'not run' and never
> > > > failsYes. But I perfer to remove them because IMO they are useless.
> > > > 
> > > 
> > > > xfs/148 [not run] parallel repair binary xfs_prepair64 is not installed
> > > > xfs/149 [not run] parallel repair binary xfs_prepair is not installed
> > > > Ran: xfs/148 xfs/149
> > > > Not run: xfs/148 xfs/149
> > > > Passed all 2 tests
> > > > 
> > > 
> > > 

Re: question of xfs/148 and xfs/149

[Darrick J. Wong]

On Thu, Sep 19, 2019 at 01:20:33PM +0800, Zorro Lang wrote:
> On Wed, Sep 18, 2019 at 04:10:50PM -0700, Darrick J. Wong wrote:
> > On Wed, Sep 18, 2019 at 09:37:11AM -0700, Darrick J. Wong wrote:
> > > On Wed, Sep 18, 2019 at 11:24:47AM +0800, Yang Xu wrote:
> > > > 
> > > > 
> > > > on 2019/09/18 10:59, Zorro Lang wrote:
> > > > > xfs/030 is weird, I've found it long time ago.
> > > > > 
> > > > > If I do a 'whole disk mkfs' (_scratch_mkfs_xfs), before this sized mkfs:
> > > > > 
> > > > >    _scratch_mkfs_xfs $DSIZE >/dev/null 2>&1
> > > > > 
> > > > > Everything looks clear, and test pass. I can't send a patch to do this,
> > > > > because I don't know the reason.
> > > > Yes. I also found running _scratch_mkfs_xfs in xfs/030 can slove this
> > > > problem yesterday. Or, we can adjust _try_wipe_scratch_devs order in
> > > > check(But I dont't have enough reason to explain why adjust it). as below:
> > > 
> > > (Yeah, I don't see any obvious reason why that would change outcomes...)
> > > 
> > > > --- a/check
> > > > +++ b/check
> > > > @@ -753,7 +753,6 @@ for section in $HOST_OPTIONS_SECTIONS; do
> > > >                         # _check_dmesg depends on this log in dmesg
> > > >                         touch ${RESULT_DIR}/check_dmesg
> > > >                 fi
> > > > -               _try_wipe_scratch_devs > /dev/null 2>&1
> > > >                 if [ "$DUMP_OUTPUT" = true ]; then
> > > >                         _run_seq 2>&1 | tee $tmp.out
> > > >                         # Because $? would get tee's return code
> > > > @@ -799,7 +798,7 @@ for section in $HOST_OPTIONS_SECTIONS; do
> > > >                 # Scan for memory leaks after every test so that associating
> > > >                 # a leak to a particular test will be as accurate as
> > > > possible.
> > > >                 _check_kmemleak || err=true
> > > > -
> > > > +               _try_wipe_scratch_devs > /dev/null 2>&1
> > > >                 # test ends after all checks are done.
> > > >                 $timestamp && _timestamp
> > > >                 stop=`_wallclock`
> > > > 
> > > > > 
> > > > > I'm not familiar with xfs_repair so much, so I don't know what happens
> > > > > underlying. I suppose the the part after the $DSIZE affect the xfs_repair,
> > > > > but I don't know why the wipefs can cause that, wipefs only erase 4 bytes
> > > > > at the beginning.
> > > > > 
> > > >  I am finding the reasion. It seems wipefs wipes important information and
> > > > $DSIZE option(using single agcount or dsize, it also fails ) can not format
> > > > disk completely. If we use other options, it can pass.
> > > 
> > > How does mkfs fail, specifically?
> > > 
> > > Also, what's your storage configuration?  And lsblk -D output?
> > 
> > I'm still interested in the answer to these questions, but I've done a
> > little more research and noticed that yes, xfs/030 fails if the device
> > doesn't support zeroing discard.
> > 
> > First, if mkfs.xfs detects an old primary superblock, it will write
> > zeroes to all superblocks before formatting the new filesystem.
> > Obviously this won't be done if the device doesn't have a primary
> > superblock.
> > 
> > (1) So let's say that a previous test formatted a 4GB scratch disk with
> > all defaults, and let's say that we have 4 AGs.  The disk will look like
> > this:
> > 
> >   SB0 [1G space] SB1 [1G space] SB2 [1G space] SB3 [1G space]
> > 
> > (2) Now we _try_wipe_scratch_devs, which wipes out the primary label:
> > 
> >   000 [1G space] SB1 [1G space] SB2 [1G space] SB3 [1G space]
> > 
> > (3) Now xfs/030 runs its special mkfs command (6AGs, 100MB disk).  If the
> > disk supports zeroing discard, it will discard the whole device:
> > 
> >   <4GB of zeroes>
> > 
> > (4) Then it will lay down its own filesystem:
> > 
> >   SB0 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 4G>
> > 
> > (5) Next, xfs/030 zaps the primary superblock:
> > 
> >   000 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 4G>
> > 
> > (6) Next, xfs/030 runs xfs_repair.  It fails to find the primary sb, so it
> > tries to find secondary superblocks.  Its first strategy is to compute
> > the fs geometry assuming all default options.  In this case, that means
> > 4 AGs, spaced 1G apart.  They're all zero, so it falls back to a linear
> > scan of the disk.  It finds SB1, uses that to rewrite the primary super,
> > and continues with the repair (which is mostly uneventful).  The test
> > passes; this is why it works on my computer.
> > 
> > ---------
> > 
> > Now let's see what happened before _try_wipe_scratch_devs.  In step (3)
> > mkfs would find the old superblocks and wipe the superblocks, before
> > laying down the new superblocks:
> > 
> >   SB0 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 1G> \
> >       000 [1G space] 000 [1G space] 000 [1G space]
> > 
> > Step (5) zaps the primary, yielding:
> > 
> >   000 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 1G> \
> >       000 [1G space] 000 [1G space] 000 [1G space]
> > 
> > Step (6) fails to find a primary superblock so it tries to read backup
> > superblocks at 1G, 2G, and 3G, but they're all zero so it falls back to
> > the linear scan and picks up SB1 and proceeds with a mostly uneventful
> > repair.  The test passes.
> > 
> > ---------
> > 
> > However, with _try_wipe_scratch_devs and a device that doesn't support
> > discard (or MKFS_OPTIONS includes -K), we have a problem.  mkfs.xfs
> > doesn't discard the device nor does it find a primary superblock, so it
> > simply formats the new filesystem.  We end up with:
> > 
> >   SB0 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 1G> \
> >       SB'1 [1G space] SB'2 [1G space] SB'3 [1G space]
> > 
> > Where SB[0-5] are from the filesystem that xfs/030 formatted but
> > SB'[1-3] are from the filesystem that was on the scratch disk before
> > xfs/030 even started.  Uhoh.
> > 
> > Step (5) zaps the primary, yielding:
> > 
> >   000 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 1G> \
> >       SB'1 [1G space] SB'2 [1G space] SB'3 [1G space]
> > 
> > Step (6) fails to find a primary superblock so it tries to read backup
> > superblocks at 1G.  It finds SB'1 and uses that to reconstruct the /old/
> > filesystem, with what looks like massive filesystem damage.  This
> > results in test failure.  Oops.
> > 
> > ----------
> > 
> > The reason for adding _try_wipe_scratch_devs was to detect broken tests
> > that started using the filesystem on the scratch device (if any) before
> > (or without!) formatting the scratch device.  That broken behavior could
> > result in spurious test failures when xfstests was run in random order
> > mode either due to mounting an unformatted device or mounting a corrupt
> > fs that some other test left behind.
> > 
> > I guess a fix for XFS would be have _try_wipe_scratch_devs try to read
> > the primary superblock to compute the AG geometry and then erase all
> > superblocks that could be on the disk; and then compute the default
> > geometry and wipe out all those superblocks too.
> > 
> > Does any of that square with what you've been seeing?
> 
> Thanks Darrick, so what I supposed might be true?
> "
>   > > > > I'm not familiar with xfs_repair so much, so I don't know what happens
>   > > > > underlying. I suppose the the part after the $DSIZE affect the xfs_repair,
> "
> 
> The sized mkfs.xfs (without discard) leave old on-disk structure behind $DSIZE
> space, it cause xfs_repair try to use odd things to do the checking.
> 
> When I tried to erase the 1st block of each AGs[1], the test passed[2].
> Is that what you talked as above?

Yep.  The version I wrote also uses xfs_db to see if there's a primary
sb at block zero that can point us to other backup superblocks to zap,
though if you want to send your version go ahead because I'll be busy
for a while dealing with this iomap mess. :/

--D

> Thanks,
> Zorro
> 
> [1]
> diff --git a/common/rc b/common/rc
> index e0b087c1..19b7ab02 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -4048,6 +4048,10 @@ _try_wipe_scratch_devs()
>         for dev in $SCRATCH_DEV_POOL $SCRATCH_DEV $SCRATCH_LOGDEV $SCRATCH_RTDEV; do
>                 test -b $dev && $WIPEFS_PROG -a $dev
>         done
> +
> +       if [ "$FSTYP" = "xfs" ];then
> +               _try_wipe_scratch_xfs
> +       fi
>  }
>  
>  # Only run this on xfs if xfs_scrub is available and has the unicode checker
> diff --git a/common/xfs b/common/xfs
> index 1bce3c18..53f33d12 100644
> --- a/common/xfs
> +++ b/common/xfs
> @@ -884,3 +884,24 @@ _xfs_mount_agcount()
>  {
>         $XFS_INFO_PROG "$1" | grep agcount= | sed -e 's/^.*agcount=\([0-9]*\),.*$/\1/g'
>  }
> +
> +_try_wipe_scratch_xfs()
> +{
> +       local tmp=`mktemp -u`
> +
> +       _scratch_mkfs_xfs -N 2>/dev/null | perl -ne '
> +               if (/^meta-data=.*\s+agcount=(\d+), agsize=(\d+) blks/) {
> +                       print STDOUT "agcount=$1\nagsize=$2\n";
> +               }
> +               if (/^data\s+=\s+bsize=(\d+)\s/) {
> +                       print STDOUT "dbsize=$1\n";
> +               }' > $tmp.mkfs
> +       . $tmp.mkfs
> +       if [ -n "$agcount" -a -n "$agsize" -a -n "$dbsize" ];then
> +               for((i=0; i<agcount; i++)); do
> +                       $XFS_IO_PROG -c "pwrite $((i * dbsize * agsize)) $dbsize" \
> +                               $SCRATCH_DEV >/dev/null;
> +               done
> +       fi
> +       rm -f $tmp.mkfs
> +}
> 
> [2]
> # ./check xfs/030
> FSTYP         -- xfs (non-debug)
> PLATFORM      -- Linux/x86_64 xxx-xxxx-xx xxx-xxxx-xx-xxx
> MKFS_OPTIONS  -- -f -bsize=4096 /dev/mapper/scratchdev
> MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/mapper/scratchdev /mnt/scratch
> 
> xfs/030 24s ...  25s
> Ran: xfs/030
> Passed all 1 tests
> 
> > 
> > --D
> > 
> > > --D
> > > 
> > > > > Darrick, do you know more about that?
> > > > > 
> > > > > Thanks,
> > > > > Zorro
> > > > > 
> > > > > > > xfs/148 is a clone of test 030 using xfs_prepair64 instead of xfs_repair.
> > > > > > > xfs/149 is a clone of test 031 using xfs_prepair instead of xfs_repair
> > > > > I'm not worried about it too much, due to it always 'not run' and never
> > > > > failsYes. But I perfer to remove them because IMO they are useless.
> > > > > 
> > > > 
> > > > > xfs/148 [not run] parallel repair binary xfs_prepair64 is not installed
> > > > > xfs/149 [not run] parallel repair binary xfs_prepair is not installed
> > > > > Ran: xfs/148 xfs/149
> > > > > Not run: xfs/148 xfs/149
> > > > > Passed all 2 tests
> > > > > 
> > > > 
> > > > 

Re: question of xfs/148 and xfs/149

[Zorro Lang]

On Wed, Sep 18, 2019 at 11:33:12PM -0700, Darrick J. Wong wrote:
> On Thu, Sep 19, 2019 at 01:20:33PM +0800, Zorro Lang wrote:
> > On Wed, Sep 18, 2019 at 04:10:50PM -0700, Darrick J. Wong wrote:
> > > On Wed, Sep 18, 2019 at 09:37:11AM -0700, Darrick J. Wong wrote:
> > > > On Wed, Sep 18, 2019 at 11:24:47AM +0800, Yang Xu wrote:
> > > > > 
> > > > > 
> > > > > on 2019/09/18 10:59, Zorro Lang wrote:
> > > > > > xfs/030 is weird, I've found it long time ago.
> > > > > > 
> > > > > > If I do a 'whole disk mkfs' (_scratch_mkfs_xfs), before this sized mkfs:
> > > > > > 
> > > > > >    _scratch_mkfs_xfs $DSIZE >/dev/null 2>&1
> > > > > > 
> > > > > > Everything looks clear, and test pass. I can't send a patch to do this,
> > > > > > because I don't know the reason.
> > > > > Yes. I also found running _scratch_mkfs_xfs in xfs/030 can slove this
> > > > > problem yesterday. Or, we can adjust _try_wipe_scratch_devs order in
> > > > > check(But I dont't have enough reason to explain why adjust it). as below:
> > > > 
> > > > (Yeah, I don't see any obvious reason why that would change outcomes...)
> > > > 
> > > > > --- a/check
> > > > > +++ b/check
> > > > > @@ -753,7 +753,6 @@ for section in $HOST_OPTIONS_SECTIONS; do
> > > > >                         # _check_dmesg depends on this log in dmesg
> > > > >                         touch ${RESULT_DIR}/check_dmesg
> > > > >                 fi
> > > > > -               _try_wipe_scratch_devs > /dev/null 2>&1
> > > > >                 if [ "$DUMP_OUTPUT" = true ]; then
> > > > >                         _run_seq 2>&1 | tee $tmp.out
> > > > >                         # Because $? would get tee's return code
> > > > > @@ -799,7 +798,7 @@ for section in $HOST_OPTIONS_SECTIONS; do
> > > > >                 # Scan for memory leaks after every test so that associating
> > > > >                 # a leak to a particular test will be as accurate as
> > > > > possible.
> > > > >                 _check_kmemleak || err=true
> > > > > -
> > > > > +               _try_wipe_scratch_devs > /dev/null 2>&1
> > > > >                 # test ends after all checks are done.
> > > > >                 $timestamp && _timestamp
> > > > >                 stop=`_wallclock`
> > > > > 
> > > > > > 
> > > > > > I'm not familiar with xfs_repair so much, so I don't know what happens
> > > > > > underlying. I suppose the the part after the $DSIZE affect the xfs_repair,
> > > > > > but I don't know why the wipefs can cause that, wipefs only erase 4 bytes
> > > > > > at the beginning.
> > > > > > 
> > > > >  I am finding the reasion. It seems wipefs wipes important information and
> > > > > $DSIZE option(using single agcount or dsize, it also fails ) can not format
> > > > > disk completely. If we use other options, it can pass.
> > > > 
> > > > How does mkfs fail, specifically?
> > > > 
> > > > Also, what's your storage configuration?  And lsblk -D output?
> > > 
> > > I'm still interested in the answer to these questions, but I've done a
> > > little more research and noticed that yes, xfs/030 fails if the device
> > > doesn't support zeroing discard.
> > > 
> > > First, if mkfs.xfs detects an old primary superblock, it will write
> > > zeroes to all superblocks before formatting the new filesystem.
> > > Obviously this won't be done if the device doesn't have a primary
> > > superblock.
> > > 
> > > (1) So let's say that a previous test formatted a 4GB scratch disk with
> > > all defaults, and let's say that we have 4 AGs.  The disk will look like
> > > this:
> > > 
> > >   SB0 [1G space] SB1 [1G space] SB2 [1G space] SB3 [1G space]
> > > 
> > > (2) Now we _try_wipe_scratch_devs, which wipes out the primary label:
> > > 
> > >   000 [1G space] SB1 [1G space] SB2 [1G space] SB3 [1G space]
> > > 
> > > (3) Now xfs/030 runs its special mkfs command (6AGs, 100MB disk).  If the
> > > disk supports zeroing discard, it will discard the whole device:
> > > 
> > >   <4GB of zeroes>
> > > 
> > > (4) Then it will lay down its own filesystem:
> > > 
> > >   SB0 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 4G>
> > > 
> > > (5) Next, xfs/030 zaps the primary superblock:
> > > 
> > >   000 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 4G>
> > > 
> > > (6) Next, xfs/030 runs xfs_repair.  It fails to find the primary sb, so it
> > > tries to find secondary superblocks.  Its first strategy is to compute
> > > the fs geometry assuming all default options.  In this case, that means
> > > 4 AGs, spaced 1G apart.  They're all zero, so it falls back to a linear
> > > scan of the disk.  It finds SB1, uses that to rewrite the primary super,
> > > and continues with the repair (which is mostly uneventful).  The test
> > > passes; this is why it works on my computer.
> > > 
> > > ---------
> > > 
> > > Now let's see what happened before _try_wipe_scratch_devs.  In step (3)
> > > mkfs would find the old superblocks and wipe the superblocks, before
> > > laying down the new superblocks:
> > > 
> > >   SB0 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 1G> \
> > >       000 [1G space] 000 [1G space] 000 [1G space]
> > > 
> > > Step (5) zaps the primary, yielding:
> > > 
> > >   000 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 1G> \
> > >       000 [1G space] 000 [1G space] 000 [1G space]
> > > 
> > > Step (6) fails to find a primary superblock so it tries to read backup
> > > superblocks at 1G, 2G, and 3G, but they're all zero so it falls back to
> > > the linear scan and picks up SB1 and proceeds with a mostly uneventful
> > > repair.  The test passes.
> > > 
> > > ---------
> > > 
> > > However, with _try_wipe_scratch_devs and a device that doesn't support
> > > discard (or MKFS_OPTIONS includes -K), we have a problem.  mkfs.xfs
> > > doesn't discard the device nor does it find a primary superblock, so it
> > > simply formats the new filesystem.  We end up with:
> > > 
> > >   SB0 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 1G> \
> > >       SB'1 [1G space] SB'2 [1G space] SB'3 [1G space]
> > > 
> > > Where SB[0-5] are from the filesystem that xfs/030 formatted but
> > > SB'[1-3] are from the filesystem that was on the scratch disk before
> > > xfs/030 even started.  Uhoh.
> > > 
> > > Step (5) zaps the primary, yielding:
> > > 
> > >   000 [16M zeroes] SB1 [16M zeroes] <4 more AGs> <zeroes from 100M to 1G> \
> > >       SB'1 [1G space] SB'2 [1G space] SB'3 [1G space]
> > > 
> > > Step (6) fails to find a primary superblock so it tries to read backup
> > > superblocks at 1G.  It finds SB'1 and uses that to reconstruct the /old/
> > > filesystem, with what looks like massive filesystem damage.  This
> > > results in test failure.  Oops.
> > > 
> > > ----------
> > > 
> > > The reason for adding _try_wipe_scratch_devs was to detect broken tests
> > > that started using the filesystem on the scratch device (if any) before
> > > (or without!) formatting the scratch device.  That broken behavior could
> > > result in spurious test failures when xfstests was run in random order
> > > mode either due to mounting an unformatted device or mounting a corrupt
> > > fs that some other test left behind.
> > > 
> > > I guess a fix for XFS would be have _try_wipe_scratch_devs try to read
> > > the primary superblock to compute the AG geometry and then erase all
> > > superblocks that could be on the disk; and then compute the default
> > > geometry and wipe out all those superblocks too.
> > > 
> > > Does any of that square with what you've been seeing?
> > 
> > Thanks Darrick, so what I supposed might be true?
> > "
> >   > > > > I'm not familiar with xfs_repair so much, so I don't know what happens
> >   > > > > underlying. I suppose the the part after the $DSIZE affect the xfs_repair,
> > "
> > 
> > The sized mkfs.xfs (without discard) leave old on-disk structure behind $DSIZE
> > space, it cause xfs_repair try to use odd things to do the checking.
> > 
> > When I tried to erase the 1st block of each AGs[1], the test passed[2].
> > Is that what you talked as above?
> 
> Yep.  The version I wrote also uses xfs_db to see if there's a primary
> sb at block zero that can point us to other backup superblocks to zap,
> though if you want to send your version go ahead because I'll be busy
> for a while dealing with this iomap mess. :/

Yeah, I saw Dave talked about that with you, so I never thought you
was working on this small issue ;)

As this test failure brings much trouble to us(than you), I'd like to fix
it if you have more other works to do.

Thanks,
Zorro

> 
> --D
> 
> > Thanks,
> > Zorro
> > 
> > [1]
> > diff --git a/common/rc b/common/rc
> > index e0b087c1..19b7ab02 100644
> > --- a/common/rc
> > +++ b/common/rc
> > @@ -4048,6 +4048,10 @@ _try_wipe_scratch_devs()
> >         for dev in $SCRATCH_DEV_POOL $SCRATCH_DEV $SCRATCH_LOGDEV $SCRATCH_RTDEV; do
> >                 test -b $dev && $WIPEFS_PROG -a $dev
> >         done
> > +
> > +       if [ "$FSTYP" = "xfs" ];then
> > +               _try_wipe_scratch_xfs
> > +       fi
> >  }
> >  
> >  # Only run this on xfs if xfs_scrub is available and has the unicode checker
> > diff --git a/common/xfs b/common/xfs
> > index 1bce3c18..53f33d12 100644
> > --- a/common/xfs
> > +++ b/common/xfs
> > @@ -884,3 +884,24 @@ _xfs_mount_agcount()
> >  {
> >         $XFS_INFO_PROG "$1" | grep agcount= | sed -e 's/^.*agcount=\([0-9]*\),.*$/\1/g'
> >  }
> > +
> > +_try_wipe_scratch_xfs()
> > +{
> > +       local tmp=`mktemp -u`
> > +
> > +       _scratch_mkfs_xfs -N 2>/dev/null | perl -ne '
> > +               if (/^meta-data=.*\s+agcount=(\d+), agsize=(\d+) blks/) {
> > +                       print STDOUT "agcount=$1\nagsize=$2\n";
> > +               }
> > +               if (/^data\s+=\s+bsize=(\d+)\s/) {
> > +                       print STDOUT "dbsize=$1\n";
> > +               }' > $tmp.mkfs
> > +       . $tmp.mkfs
> > +       if [ -n "$agcount" -a -n "$agsize" -a -n "$dbsize" ];then
> > +               for((i=0; i<agcount; i++)); do
> > +                       $XFS_IO_PROG -c "pwrite $((i * dbsize * agsize)) $dbsize" \
> > +                               $SCRATCH_DEV >/dev/null;
> > +               done
> > +       fi
> > +       rm -f $tmp.mkfs
> > +}
> > 
> > [2]
> > # ./check xfs/030
> > FSTYP         -- xfs (non-debug)
> > PLATFORM      -- Linux/x86_64 xxx-xxxx-xx xxx-xxxx-xx-xxx
> > MKFS_OPTIONS  -- -f -bsize=4096 /dev/mapper/scratchdev
> > MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/mapper/scratchdev /mnt/scratch
> > 
> > xfs/030 24s ...  25s
> > Ran: xfs/030
> > Passed all 1 tests
> > 
> > > 
> > > --D
> > > 
> > > > --D
> > > > 
> > > > > > Darrick, do you know more about that?
> > > > > > 
> > > > > > Thanks,
> > > > > > Zorro
> > > > > > 
> > > > > > > > xfs/148 is a clone of test 030 using xfs_prepair64 instead of xfs_repair.
> > > > > > > > xfs/149 is a clone of test 031 using xfs_prepair instead of xfs_repair
> > > > > > I'm not worried about it too much, due to it always 'not run' and never
> > > > > > failsYes. But I perfer to remove them because IMO they are useless.
> > > > > > 
> > > > > 
> > > > > > xfs/148 [not run] parallel repair binary xfs_prepair64 is not installed
> > > > > > xfs/149 [not run] parallel repair binary xfs_prepair is not installed
> > > > > > Ran: xfs/148 xfs/149
> > > > > > Not run: xfs/148 xfs/149
> > > > > > Passed all 2 tests
> > > > > > 
> > > > > 
> > > > > 

Re: question of xfs/148 and xfs/149

[Yang Xu]

on 2019/09/19 0:37, Darrick J. Wong wrote:
>>   I am finding the reasion. It seems wipefs wipes important information and
>> $DSIZE option(using single agcount or dsize, it also fails ) can not format
>> disk completely. If we use other options, it can pass.
> How does mkfs fail, specifically?
> 
> Also, what's your storage configuration?  And lsblk -D output?

I only guess it from result. Even though, mkfs.xfs $DSIZE 
successfully($? is 0), but UUID mismatch in 030.full, so it may
format the first superblock failed.  This is just my guess.

 From your detailed explanation, I understand why it fails.

Thanks
Yang Xu