[PATCH] common: get fs type again using device canonical name in _fs_type

[PATCH] common: get fs type again using device canonical name in _fs_type

[Eryu Guan]

When testing with lvm, a previous btrfsck run could change df output
from something like

/dev/mapper/rhel_hp--dl388eg8--01-testlv1 btrfs 15728640 900 13602172 1% /mnt/btrfs

to

/dev/dm-3 btrfs 15728640 900 13602172 1% /mnt/btrfs

So "_fs_type $device" couldn't find the mounted btrfs in
_check_btrfs_filesystem and the fs couldn't get unmounted, then a latter
btrfsck reports the btrfs is currently mounted, which could fail the test.

Signed-off-by: Eryu Guan <eguan@redhat.com>
---
 common/rc | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/common/rc b/common/rc
index 2c83340..a4fb21d 100644
--- a/common/rc
+++ b/common/rc
@@ -802,7 +802,13 @@ _fs_type()
     # Fix the filesystem type up here so that the callers don't
     # have to bother with this quirk.
     #
-    _df_device $1 | $AWK_PROG '{ print $2 }' | sed -e 's/nfs4/nfs/'
+    local fstype=`_df_device $1 | $AWK_PROG '{ print $2 }' | sed -e 's/nfs4/nfs/'`
+    # in case $device is a symlink(lvm) and df gives out the canonical name
+    if [ "$fstype" == "" ]; then
+        _df_device `_real_dev $1` | $AWK_PROG '{ print $2 }' | sed -e 's/nfs4/nfs/'`
+    else
+        echo $fstype
+    fi
 }
 
 # return the FS mount options of a mounted device
-- 
1.9.3

[PATCH v2] common: get fs type again using device canonical name in _fs_type

[Eryu Guan]

When testing with lvm, a previous btrfsck run could change df output
from something like

/dev/mapper/rhel_hp--dl388eg8--01-testlv1 btrfs 15728640 900 13602172 1% /mnt/btrfs

to

/dev/dm-3 btrfs 15728640 900 13602172 1% /mnt/btrfs

So "_fs_type $device" couldn't find the mounted btrfs in
_check_btrfs_filesystem and the fs couldn't get unmounted, then a latter
btrfsck reports the btrfs is currently mounted, which could fail the test.

Signed-off-by: Eryu Guan <eguan@redhat.com>
---
v2:
 - remove a trailing `, which introduces syntax error

 common/rc | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/common/rc b/common/rc
index 2c83340..7ea1614 100644
--- a/common/rc
+++ b/common/rc
@@ -802,7 +802,13 @@ _fs_type()
     # Fix the filesystem type up here so that the callers don't
     # have to bother with this quirk.
     #
-    _df_device $1 | $AWK_PROG '{ print $2 }' | sed -e 's/nfs4/nfs/'
+    local fstype=`_df_device $1 | $AWK_PROG '{ print $2 }' | sed -e 's/nfs4/nfs/'`
+    # in case $device is a symlink(lvm) and df gives out the canonical name
+    if [ "$fstype" == "" ]; then
+        _df_device `_real_dev $1` | $AWK_PROG '{ print $2 }' | sed -e 's/nfs4/nfs/'
+    else
+        echo $fstype
+    fi
 }
 
 # return the FS mount options of a mounted device
-- 
1.9.3

Re: [PATCH v2] common: get fs type again using device canonical name in _fs_type

[Dave Chinner]

On Thu, Jul 31, 2014 at 06:52:37PM +0800, Eryu Guan wrote:
> When testing with lvm, a previous btrfsck run could change df output
> from something like
> 
> /dev/mapper/rhel_hp--dl388eg8--01-testlv1 btrfs 15728640 900 13602172 1% /mnt/btrfs
> 
> to
> 
> /dev/dm-3 btrfs 15728640 900 13602172 1% /mnt/btrfs

I don't follow you. Why would running btrfsck change the name of the
device? If the filesystem is umounted and mounted again, then the
device could change, but btrfsck should not be not doing the
unmount/mount, and so unless the TEST_DEV/SCRATCH_DEV is changing
the output of df should be identical...

So before we change the _fs_type() code, can you explain exactly
how, when and why the device name is changing to me?

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH v2] common: get fs type again using device canonical name in _fs_type

[Eryu Guan]

On Fri, Aug 01, 2014 at 10:21:59AM +1000, Dave Chinner wrote:
> On Thu, Jul 31, 2014 at 06:52:37PM +0800, Eryu Guan wrote:
> > When testing with lvm, a previous btrfsck run could change df output
> > from something like
> > 
> > /dev/mapper/rhel_hp--dl388eg8--01-testlv1 btrfs 15728640 900 13602172 1% /mnt/btrfs
> > 
> > to
> > 
> > /dev/dm-3 btrfs 15728640 900 13602172 1% /mnt/btrfs
> 
> I don't follow you. Why would running btrfsck change the name of the
> device? If the filesystem is umounted and mounted again, then the
> device could change, but btrfsck should not be not doing the
> unmount/mount, and so unless the TEST_DEV/SCRATCH_DEV is changing
> the output of df should be identical...
> 
> So before we change the _fs_type() code, can you explain exactly
> how, when and why the device name is changing to me?

Assume that we have two btrfs filesystems, kernel is 3.16.0-rc4+

[root@hp-dl388eg8-01 btrfs-progs]# btrfs fi show
Label: none  uuid: 1aba7da5-ce2b-4af0-a716-db732abc60b2
        Total devices 1 FS bytes used 384.00KiB
        devid    1 size 15.00GiB used 2.04GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv1

Label: none  uuid: 26ff4f12-f6d9-4cbc-aae2-57febeefde37
        Total devices 2 FS bytes used 112.00KiB
        devid    1 size 15.00GiB used 2.03GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv2
        devid    2 size 15.00GiB used 2.01GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv3

Btrfs v3.14.2

And testlv1 was mounted at /mnt/btrfs

[root@hp-dl388eg8-01 btrfs-progs]# df -TP /mnt/btrfs
Filesystem                                Type  1024-blocks  Used Available Capacity Mounted on
/dev/mapper/rhel_hp--dl388eg8--01-testlv1 btrfs    15728640   512  13602560       1% /mnt/btrfs

Now run btrfsck on testlv2, btrfsck will scan all btrfs devices and
somehow change the device name.

[root@hp-dl388eg8-01 btrfs-progs]# btrfsck /dev/mapper/rhel_hp--dl388eg8--01-testlv2 >/dev/null 2>&1

# device name changed in df output and btrfs fi show output
[root@hp-dl388eg8-01 btrfs-progs]# df -TP /mnt/btrfs
Filesystem     Type  1024-blocks  Used Available Capacity Mounted on
/dev/dm-3      btrfs    15728640   512  13602560       1% /mnt/btrfs
[root@hp-dl388eg8-01 btrfs-progs]# btrfs fi show
Label: none  uuid: 1aba7da5-ce2b-4af0-a716-db732abc60b2
        Total devices 1 FS bytes used 384.00KiB
        devid    1 size 15.00GiB used 2.04GiB path /dev/dm-3

Label: none  uuid: 26ff4f12-f6d9-4cbc-aae2-57febeefde37
        Total devices 2 FS bytes used 112.00KiB
        devid    1 size 15.00GiB used 2.03GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv2
        devid    2 size 15.00GiB used 2.01GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv3

Btrfs v3.14.2

This only happens when btrfsck a btrfs with multiple devices, so this
only affects xfstests run on btrfs with SCRATCH_DEV_POOL set to lvm
lvs.

Maybe this is a bug of btrfs-progs and we should fix it there?

If we decide to take this patch, I'll put more details in commit log
in v3.

Thanks,
Eryu
> 
> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@fromorbit.com

Re: [PATCH v2] common: get fs type again using device canonical name in _fs_type

[Dave Chinner]

On Fri, Aug 01, 2014 at 12:02:41PM +0800, Eryu Guan wrote:
> On Fri, Aug 01, 2014 at 10:21:59AM +1000, Dave Chinner wrote:
> > On Thu, Jul 31, 2014 at 06:52:37PM +0800, Eryu Guan wrote:
> > > When testing with lvm, a previous btrfsck run could change df output
> > > from something like
> > > 
> > > /dev/mapper/rhel_hp--dl388eg8--01-testlv1 btrfs 15728640 900 13602172 1% /mnt/btrfs
> > > 
> > > to
> > > 
> > > /dev/dm-3 btrfs 15728640 900 13602172 1% /mnt/btrfs
> > 
> > I don't follow you. Why would running btrfsck change the name of the
> > device? If the filesystem is umounted and mounted again, then the
> > device could change, but btrfsck should not be not doing the
> > unmount/mount, and so unless the TEST_DEV/SCRATCH_DEV is changing
> > the output of df should be identical...
> > 
> > So before we change the _fs_type() code, can you explain exactly
> > how, when and why the device name is changing to me?
> 
> Assume that we have two btrfs filesystems, kernel is 3.16.0-rc4+
> 
> [root@hp-dl388eg8-01 btrfs-progs]# btrfs fi show
> Label: none  uuid: 1aba7da5-ce2b-4af0-a716-db732abc60b2
>         Total devices 1 FS bytes used 384.00KiB
>         devid    1 size 15.00GiB used 2.04GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv1
> 
> Label: none  uuid: 26ff4f12-f6d9-4cbc-aae2-57febeefde37
>         Total devices 2 FS bytes used 112.00KiB
>         devid    1 size 15.00GiB used 2.03GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv2
>         devid    2 size 15.00GiB used 2.01GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv3
> 
> Btrfs v3.14.2
> 
> And testlv1 was mounted at /mnt/btrfs
> 
> [root@hp-dl388eg8-01 btrfs-progs]# df -TP /mnt/btrfs
> Filesystem                                Type  1024-blocks  Used Available Capacity Mounted on
> /dev/mapper/rhel_hp--dl388eg8--01-testlv1 btrfs    15728640   512  13602560       1% /mnt/btrfs
> 
> Now run btrfsck on testlv2, btrfsck will scan all btrfs devices and
> somehow change the device name.
> 
> [root@hp-dl388eg8-01 btrfs-progs]# btrfsck /dev/mapper/rhel_hp--dl388eg8--01-testlv2 >/dev/null 2>&1
> 
> # device name changed in df output and btrfs fi show output
> [root@hp-dl388eg8-01 btrfs-progs]# df -TP /mnt/btrfs
> Filesystem     Type  1024-blocks  Used Available Capacity Mounted on
> /dev/dm-3      btrfs    15728640   512  13602560       1% /mnt/btrfs
> [root@hp-dl388eg8-01 btrfs-progs]# btrfs fi show
> Label: none  uuid: 1aba7da5-ce2b-4af0-a716-db732abc60b2
>         Total devices 1 FS bytes used 384.00KiB
>         devid    1 size 15.00GiB used 2.04GiB path /dev/dm-3
> 
> Label: none  uuid: 26ff4f12-f6d9-4cbc-aae2-57febeefde37
>         Total devices 2 FS bytes used 112.00KiB
>         devid    1 size 15.00GiB used 2.03GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv2
>         devid    2 size 15.00GiB used 2.01GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv3
> 
> Btrfs v3.14.2
> 
> This only happens when btrfsck a btrfs with multiple devices, so this
> only affects xfstests run on btrfs with SCRATCH_DEV_POOL set to lvm
> lvs.
> 
> Maybe this is a bug of btrfs-progs and we should fix it there?

Yes, that smells of a btrfs-progs bug. If your /etc/mtab a link to
/proc/mounts? If not, does the contents change when you run btrfsck,
and does the problem go away when you replace /etc/mtab with a link
to /proc/mounts?

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH v2] common: get fs type again using device canonical name in _fs_type

[Eryu Guan]

On Fri, Aug 01, 2014 at 02:49:10PM +1000, Dave Chinner wrote:
> On Fri, Aug 01, 2014 at 12:02:41PM +0800, Eryu Guan wrote:
> > On Fri, Aug 01, 2014 at 10:21:59AM +1000, Dave Chinner wrote:
> > > On Thu, Jul 31, 2014 at 06:52:37PM +0800, Eryu Guan wrote:
> > > > When testing with lvm, a previous btrfsck run could change df output
> > > > from something like
> > > > 
> > > > /dev/mapper/rhel_hp--dl388eg8--01-testlv1 btrfs 15728640 900 13602172 1% /mnt/btrfs
> > > > 
> > > > to
> > > > 
> > > > /dev/dm-3 btrfs 15728640 900 13602172 1% /mnt/btrfs
> > > 
> > > I don't follow you. Why would running btrfsck change the name of the
> > > device? If the filesystem is umounted and mounted again, then the
> > > device could change, but btrfsck should not be not doing the
> > > unmount/mount, and so unless the TEST_DEV/SCRATCH_DEV is changing
> > > the output of df should be identical...
> > > 
> > > So before we change the _fs_type() code, can you explain exactly
> > > how, when and why the device name is changing to me?
> > 
> > Assume that we have two btrfs filesystems, kernel is 3.16.0-rc4+
> > 
> > [root@hp-dl388eg8-01 btrfs-progs]# btrfs fi show
> > Label: none  uuid: 1aba7da5-ce2b-4af0-a716-db732abc60b2
> >         Total devices 1 FS bytes used 384.00KiB
> >         devid    1 size 15.00GiB used 2.04GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv1
> > 
> > Label: none  uuid: 26ff4f12-f6d9-4cbc-aae2-57febeefde37
> >         Total devices 2 FS bytes used 112.00KiB
> >         devid    1 size 15.00GiB used 2.03GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv2
> >         devid    2 size 15.00GiB used 2.01GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv3
> > 
> > Btrfs v3.14.2
> > 
> > And testlv1 was mounted at /mnt/btrfs
> > 
> > [root@hp-dl388eg8-01 btrfs-progs]# df -TP /mnt/btrfs
> > Filesystem                                Type  1024-blocks  Used Available Capacity Mounted on
> > /dev/mapper/rhel_hp--dl388eg8--01-testlv1 btrfs    15728640   512  13602560       1% /mnt/btrfs
> > 
> > Now run btrfsck on testlv2, btrfsck will scan all btrfs devices and
> > somehow change the device name.
> > 
> > [root@hp-dl388eg8-01 btrfs-progs]# btrfsck /dev/mapper/rhel_hp--dl388eg8--01-testlv2 >/dev/null 2>&1
> > 
> > # device name changed in df output and btrfs fi show output
> > [root@hp-dl388eg8-01 btrfs-progs]# df -TP /mnt/btrfs
> > Filesystem     Type  1024-blocks  Used Available Capacity Mounted on
> > /dev/dm-3      btrfs    15728640   512  13602560       1% /mnt/btrfs
> > [root@hp-dl388eg8-01 btrfs-progs]# btrfs fi show
> > Label: none  uuid: 1aba7da5-ce2b-4af0-a716-db732abc60b2
> >         Total devices 1 FS bytes used 384.00KiB
> >         devid    1 size 15.00GiB used 2.04GiB path /dev/dm-3
> > 
> > Label: none  uuid: 26ff4f12-f6d9-4cbc-aae2-57febeefde37
> >         Total devices 2 FS bytes used 112.00KiB
> >         devid    1 size 15.00GiB used 2.03GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv2
> >         devid    2 size 15.00GiB used 2.01GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv3
> > 
> > Btrfs v3.14.2
> > 
> > This only happens when btrfsck a btrfs with multiple devices, so this
> > only affects xfstests run on btrfs with SCRATCH_DEV_POOL set to lvm
> > lvs.
> > 
> > Maybe this is a bug of btrfs-progs and we should fix it there?
> 
> Yes, that smells of a btrfs-progs bug. If your /etc/mtab a link to
> /proc/mounts? If not, does the contents change when you run btrfsck,
> and does the problem go away when you replace /etc/mtab with a link
> to /proc/mounts?

/etc/mtab is a symlink to /proc/self/mounts, so does /proc/mounts

[root@hp-dl388eg8-01 btrfs-progs]# ls -l /etc/mtab
lrwxrwxrwx. 1 root root 17 Sep 22  2013 /etc/mtab -> /proc/self/mounts
[root@hp-dl388eg8-01 btrfs-progs]# ls -l /proc/mounts
lrwxrwxrwx. 1 root root 11 Aug  1 00:59 /proc/mounts -> self/mounts

And the device name also changed in /proc/mounts

[root@hp-dl388eg8-01 btrfs-progs]# grep btrfs /proc/mounts
/dev/dm-3 /mnt/btrfs btrfs rw,seclabel,relatime,space_cache 0 0

Thanks,
Eryu
> 
> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@fromorbit.com
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] common: get fs type again using device canonical name in _fs_type

[Dave Chinner]

On Fri, Aug 01, 2014 at 01:02:58PM +0800, Eryu Guan wrote:
> On Fri, Aug 01, 2014 at 02:49:10PM +1000, Dave Chinner wrote:
> > On Fri, Aug 01, 2014 at 12:02:41PM +0800, Eryu Guan wrote:
> > > On Fri, Aug 01, 2014 at 10:21:59AM +1000, Dave Chinner wrote:
> > > > On Thu, Jul 31, 2014 at 06:52:37PM +0800, Eryu Guan wrote:
> > > > > When testing with lvm, a previous btrfsck run could change df output
> > > > > from something like
> > > > > 
> > > > > /dev/mapper/rhel_hp--dl388eg8--01-testlv1 btrfs 15728640 900 13602172 1% /mnt/btrfs
> > > > > 
> > > > > to
> > > > > 
> > > > > /dev/dm-3 btrfs 15728640 900 13602172 1% /mnt/btrfs
> > > > 
> > > > I don't follow you. Why would running btrfsck change the name of the
> > > > device? If the filesystem is umounted and mounted again, then the
> > > > device could change, but btrfsck should not be not doing the
> > > > unmount/mount, and so unless the TEST_DEV/SCRATCH_DEV is changing
> > > > the output of df should be identical...
> > > > 
> > > > So before we change the _fs_type() code, can you explain exactly
> > > > how, when and why the device name is changing to me?
> > > 
> > > Assume that we have two btrfs filesystems, kernel is 3.16.0-rc4+
> > > 
> > > [root@hp-dl388eg8-01 btrfs-progs]# btrfs fi show
> > > Label: none  uuid: 1aba7da5-ce2b-4af0-a716-db732abc60b2
> > >         Total devices 1 FS bytes used 384.00KiB
> > >         devid    1 size 15.00GiB used 2.04GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv1
> > > 
> > > Label: none  uuid: 26ff4f12-f6d9-4cbc-aae2-57febeefde37
> > >         Total devices 2 FS bytes used 112.00KiB
> > >         devid    1 size 15.00GiB used 2.03GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv2
> > >         devid    2 size 15.00GiB used 2.01GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv3
> > > 
> > > Btrfs v3.14.2
> > > 
> > > And testlv1 was mounted at /mnt/btrfs
> > > 
> > > [root@hp-dl388eg8-01 btrfs-progs]# df -TP /mnt/btrfs
> > > Filesystem                                Type  1024-blocks  Used Available Capacity Mounted on
> > > /dev/mapper/rhel_hp--dl388eg8--01-testlv1 btrfs    15728640   512  13602560       1% /mnt/btrfs
> > > 
> > > Now run btrfsck on testlv2, btrfsck will scan all btrfs devices and
> > > somehow change the device name.
> > > 
> > > [root@hp-dl388eg8-01 btrfs-progs]# btrfsck /dev/mapper/rhel_hp--dl388eg8--01-testlv2 >/dev/null 2>&1
> > > 
> > > # device name changed in df output and btrfs fi show output
> > > [root@hp-dl388eg8-01 btrfs-progs]# df -TP /mnt/btrfs
> > > Filesystem     Type  1024-blocks  Used Available Capacity Mounted on
> > > /dev/dm-3      btrfs    15728640   512  13602560       1% /mnt/btrfs
> > > [root@hp-dl388eg8-01 btrfs-progs]# btrfs fi show
> > > Label: none  uuid: 1aba7da5-ce2b-4af0-a716-db732abc60b2
> > >         Total devices 1 FS bytes used 384.00KiB
> > >         devid    1 size 15.00GiB used 2.04GiB path /dev/dm-3
> > > 
> > > Label: none  uuid: 26ff4f12-f6d9-4cbc-aae2-57febeefde37
> > >         Total devices 2 FS bytes used 112.00KiB
> > >         devid    1 size 15.00GiB used 2.03GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv2
> > >         devid    2 size 15.00GiB used 2.01GiB path /dev/mapper/rhel_hp--dl388eg8--01-testlv3
> > > 
> > > Btrfs v3.14.2
> > > 
> > > This only happens when btrfsck a btrfs with multiple devices, so this
> > > only affects xfstests run on btrfs with SCRATCH_DEV_POOL set to lvm
> > > lvs.
> > > 
> > > Maybe this is a bug of btrfs-progs and we should fix it there?
> > 
> > Yes, that smells of a btrfs-progs bug. If your /etc/mtab a link to
> > /proc/mounts? If not, does the contents change when you run btrfsck,
> > and does the problem go away when you replace /etc/mtab with a link
> > to /proc/mounts?
> 
> /etc/mtab is a symlink to /proc/self/mounts, so does /proc/mounts
> 
> [root@hp-dl388eg8-01 btrfs-progs]# ls -l /etc/mtab
> lrwxrwxrwx. 1 root root 17 Sep 22  2013 /etc/mtab -> /proc/self/mounts
> [root@hp-dl388eg8-01 btrfs-progs]# ls -l /proc/mounts
> lrwxrwxrwx. 1 root root 11 Aug  1 00:59 /proc/mounts -> self/mounts
> 
> And the device name also changed in /proc/mounts
> 
> [root@hp-dl388eg8-01 btrfs-progs]# grep btrfs /proc/mounts
> /dev/dm-3 /mnt/btrfs btrfs rw,seclabel,relatime,space_cache 0 0

Well, that's exactly the last thing *I* expected. The kernel just
doesn't change device names on mounted filesystems like that.

Oh, the device name comes from btrfs_show_devname().

So this definitely seems to me to be a btrfs bug - btrfsck is
causing the btrfs kernel code to change the name of devices
associated with unrelated, mounted filesystems to that which it is
operating on. That's just wrong. IOWs, btrfs needs fixing, not
xfstests, because that can bite during any test that runs btrfsck in
the middle of it....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com