[PATCH] fstests: introduce btrfs-map-logical

[PATCH] fstests: introduce btrfs-map-logical

[Liu Bo]

A typical use case of 'btrfs-map-logical' is to translate btrfs logical
address to physical address on each disk.

Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
---
 common/config | 1 +
 1 file changed, 1 insertion(+)

diff --git a/common/config b/common/config
index 59041a3..b7c06bf 100644
--- a/common/config
+++ b/common/config
@@ -249,6 +249,7 @@ case "$HOSTOS" in
         export BTRFS_UTIL_PROG="`set_prog_path btrfs`"
         export BTRFS_SHOW_SUPER_PROG="`set_prog_path btrfs-show-super`"
 	export BTRFS_CONVERT_PROG="`set_prog_path btrfs-convert`"
+	export BTRFS_MAP_LOGICAL_PROG="`set_prog_path btrfs-map-logical`"
         export XFS_FSR_PROG="`set_prog_path xfs_fsr`"
         export MKFS_NFS_PROG="false"
         export MKFS_CIFS_PROG="false"
-- 
2.5.0

Re: [PATCH] fstests: introduce btrfs-map-logical

[Qu Wenruo]

At 04/12/2017 09:27 AM, Liu Bo wrote:
> A typical use case of 'btrfs-map-logical' is to translate btrfs logical
> address to physical address on each disk.

Could we avoid usage of btrfs-map-logical here?

I understand that we need to do corruption so that we can test if the 
repair works, but I'm not sure if the output format will change, or if 
the program will get replace by "btrfs inspect-internal" group.

Thanks,
Qu

> 
> Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
> ---
>   common/config | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/common/config b/common/config
> index 59041a3..b7c06bf 100644
> --- a/common/config
> +++ b/common/config
> @@ -249,6 +249,7 @@ case "$HOSTOS" in
>           export BTRFS_UTIL_PROG="`set_prog_path btrfs`"
>           export BTRFS_SHOW_SUPER_PROG="`set_prog_path btrfs-show-super`"
>   	export BTRFS_CONVERT_PROG="`set_prog_path btrfs-convert`"
> +	export BTRFS_MAP_LOGICAL_PROG="`set_prog_path btrfs-map-logical`"
>           export XFS_FSR_PROG="`set_prog_path xfs_fsr`"
>           export MKFS_NFS_PROG="false"
>           export MKFS_CIFS_PROG="false"
> 

Re: [PATCH] fstests: introduce btrfs-map-logical

[David Sterba]

On Wed, Apr 12, 2017 at 09:35:00AM +0800, Qu Wenruo wrote:
> 
> 
> At 04/12/2017 09:27 AM, Liu Bo wrote:
> > A typical use case of 'btrfs-map-logical' is to translate btrfs logical
> > address to physical address on each disk.
> 
> Could we avoid usage of btrfs-map-logical here?

Agreed.

> I understand that we need to do corruption so that we can test if the 
> repair works, but I'm not sure if the output format will change, or if 
> the program will get replace by "btrfs inspect-internal" group.

In the long-term it will be repleaced, but there's no ETA.

Re: [PATCH] fstests: introduce btrfs-map-logical

[David Sterba]

On Wed, Apr 12, 2017 at 02:32:02PM +0200, David Sterba wrote:
> On Wed, Apr 12, 2017 at 09:35:00AM +0800, Qu Wenruo wrote:
> > 
> > 
> > At 04/12/2017 09:27 AM, Liu Bo wrote:
> > > A typical use case of 'btrfs-map-logical' is to translate btrfs logical
> > > address to physical address on each disk.
> > 
> > Could we avoid usage of btrfs-map-logical here?
> 
> Agreed.
> 
> > I understand that we need to do corruption so that we can test if the 
> > repair works, but I'm not sure if the output format will change, or if 
> > the program will get replace by "btrfs inspect-internal" group.
> 
> In the long-term it will be repleaced, but there's no ETA.

Possibly, if fstests maintainer agrees, we can add btrfs-map-logical to
fstests. It's small and uses headers from libbtrfs, so this would become
a new dependency but I believe is still bearable.

I'm not sure if we should export all debuging functionality in 'btrfs'
as this is typically something that a user will never want, not even in
the emergency environments. There's an overlap in the information to be
exported but I'd be more inclined to satisfy user needs than testsuite
needs. So an independent tool would give us more freedom on both sides.

Re: [PATCH] fstests: introduce btrfs-map-logical

[Qu Wenruo]

At 04/12/2017 08:52 PM, David Sterba wrote:
> On Wed, Apr 12, 2017 at 02:32:02PM +0200, David Sterba wrote:
>> On Wed, Apr 12, 2017 at 09:35:00AM +0800, Qu Wenruo wrote:
>>>
>>>
>>> At 04/12/2017 09:27 AM, Liu Bo wrote:
>>>> A typical use case of 'btrfs-map-logical' is to translate btrfs logical
>>>> address to physical address on each disk.
>>>
>>> Could we avoid usage of btrfs-map-logical here?
>>
>> Agreed.
>>
>>> I understand that we need to do corruption so that we can test if the
>>> repair works, but I'm not sure if the output format will change, or if
>>> the program will get replace by "btrfs inspect-internal" group.
>>
>> In the long-term it will be repleaced, but there's no ETA.
> 
> Possibly, if fstests maintainer agrees, we can add btrfs-map-logical to
> fstests. It's small and uses headers from libbtrfs, so this would become
> a new dependency but I believe is still bearable.
> 
> I'm not sure if we should export all debuging functionality in 'btrfs'
> as this is typically something that a user will never want, not even in
> the emergency environments. There's an overlap in the information to be
> exported but I'd be more inclined to satisfy user needs than testsuite
> needs. So an independent tool would give us more freedom on both sides.
> 
I'm working on the new btrfs-corrupt-block equivalent, considering the 
demand to corrupt on-disk data for recovery test, I could provide tool 
with fundamental corruption support.

Which could corrupt on-disk data, either specified by (root, inode, 
offset, length) or just (logical address, length).
And support to corrupt given mirror or even P/Q for RAID56.
(With btrfs_map_block_v2 from offline scrub)

I'm not sure if I should just replace btrfs-corrupt-block or add a new 
individual prog or add a btrfs subcommand group which is disabled by 
default?

Thanks,
Qu

Re: [PATCH] fstests: introduce btrfs-map-logical

[Qu Wenruo]

At 04/12/2017 08:52 PM, David Sterba wrote:
> On Wed, Apr 12, 2017 at 02:32:02PM +0200, David Sterba wrote:
>> On Wed, Apr 12, 2017 at 09:35:00AM +0800, Qu Wenruo wrote:
>>>
>>>
>>> At 04/12/2017 09:27 AM, Liu Bo wrote:
>>>> A typical use case of 'btrfs-map-logical' is to translate btrfs logical
>>>> address to physical address on each disk.
>>>
>>> Could we avoid usage of btrfs-map-logical here?
>>
>> Agreed.
>>
>>> I understand that we need to do corruption so that we can test if the
>>> repair works, but I'm not sure if the output format will change, or if
>>> the program will get replace by "btrfs inspect-internal" group.
>>
>> In the long-term it will be repleaced, but there's no ETA.
> 
> Possibly, if fstests maintainer agrees, we can add btrfs-map-logical to
> fstests. It's small and uses headers from libbtrfs, so this would become
> a new dependency but I believe is still bearable.
> 
> I'm not sure if we should export all debuging functionality in 'btrfs'
> as this is typically something that a user will never want, not even in
> the emergency environments. There's an overlap in the information to be
> exported but I'd be more inclined to satisfy user needs than testsuite
> needs. So an independent tool would give us more freedom on both sides.
> 
I'm working on the new btrfs-corrupt-block equivalent, considering the 
demand to corrupt on-disk data for recovery test, I could provide tool 
with fundamental corruption support.

Which could corrupt on-disk data, either specified by (root, inode, 
offset, length) or just (logical address, length).
And support to corrupt given mirror or even P/Q for RAID56.
(With btrfs_map_block_v2 from offline scrub)

I'm not sure if I should just replace btrfs-corrupt-block or add a new 
individual prog or add a btrfs subcommand group which is disabled by 
default?

Thanks,
Qu

Re: [PATCH] fstests: introduce btrfs-map-logical

[Eryu Guan]

On Wed, Apr 12, 2017 at 02:52:23PM +0200, David Sterba wrote:
> > > I understand that we need to do corruption so that we can test if the 
> > > repair works, but I'm not sure if the output format will change, or if 
> > > the program will get replace by "btrfs inspect-internal" group.
> > 
> > In the long-term it will be repleaced, but there's no ETA.
> 
> Possibly, if fstests maintainer agrees, we can add btrfs-map-logical to
> fstests. It's small and uses headers from libbtrfs, so this would become
> a new dependency but I believe is still bearable.

IMHO, I think the ability to poke btrfs internal really should be
provided by btrfs-progs package and maintained by btrfs community.
fstests provides some fs-independent c helpers to assist testing, but
not necessarily needs to "understand" filesystem internals.

For historical reason, building fstests requires xfsprogs development
headers, we'd better not introduce new fs-specific dependencies.

Thanks,
Eryu