[PATCH 0/3] btrfs-progs: ins: Update for logical-resolve command

[PATCH 0/3] btrfs-progs: ins: Update for logical-resolve command

[Misono Tomohiro]

Patch 1 and 2 fix some issues in logical-resolve.
Patch 3 is a main part and adds v2 support of BTRFS_IOC_LOGICA_INO_V2

Misono Tomohiro (3):
  btrfs-progs: ins: logical-resolve: Set correct error value
  btrfs-progs: ins: logical-resolve: Print message when path cannot be
    resolved
  btrfs-progs: ins: logical-resolve: Add v2 ioctl support

 Documentation/btrfs-inspect-internal.asciidoc | 15 ++++--
 cmds-inspect.c                                | 67 +++++++++++++++++++++++++--
 ioctl.h                                       | 11 ++++-
 libbtrfsutil/btrfs.h                          | 10 +++-
 4 files changed, 93 insertions(+), 10 deletions(-)

-- 
2.14.4

[PATCH 1/3] btrfs-progs: ins: logical-resolve: Set correct error value

[Misono Tomohiro]

Reset ret value to zero after snprintf(), which returns the number
of written chars. Otherwise non-zero value returns after command
success with -P option. Also set return value from __ino_to_path_fd() to
reflect the final status for default behavior.

Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
---
 cmds-inspect.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/cmds-inspect.c b/cmds-inspect.c
index 2fc50c1a..ac77a5ee 100644
--- a/cmds-inspect.c
+++ b/cmds-inspect.c
@@ -211,6 +211,7 @@ static int cmd_inspect_logical_resolve(int argc, char **argv)
 		error("path buffer too small: %d bytes", bytes_left);
 		goto out;
 	}
+	ret = 0;
 
 	for (i = 0; i < inodes->elem_cnt; i += 3) {
 		u64 inum = inodes->val[i];
@@ -245,7 +246,8 @@ static int cmd_inspect_logical_resolve(int argc, char **argv)
 					goto out;
 				}
 			}
-			__ino_to_path_fd(inum, path_fd, verbose, full_path);
+			ret = __ino_to_path_fd(inum, path_fd, verbose,
+						full_path);
 			if (path_fd != fd)
 				close_file_or_dir(path_fd, dirs);
 		} else {
-- 
2.14.4

[PATCH 2/3] btrfs-progs: ins: logical-resolve: Print message when path cannot be resolved

[Misono Tomohiro]

Since BTRFS_IOC_INO_PATHS requires fd of subvolume, logical-resolve
cannot find the path when mount point is not FS_TREE
(because the subvolume path cannot be opened).

In that case, print message to try -P option instead.

Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
---
 cmds-inspect.c | 34 +++++++++++++++++++++++++++++++++-
 1 file changed, 33 insertions(+), 1 deletion(-)

diff --git a/cmds-inspect.c b/cmds-inspect.c
index ac77a5ee..21aa2903 100644
--- a/cmds-inspect.c
+++ b/cmds-inspect.c
@@ -137,16 +137,18 @@ static const char * const cmd_inspect_logical_resolve_usage[] = {
 static int cmd_inspect_logical_resolve(int argc, char **argv)
 {
 	int ret;
-	int fd;
+	int fd = -1;
 	int i;
 	int verbose = 0;
 	int getpath = 1;
 	int bytes_left;
+	u64 rootid;
 	struct btrfs_ioctl_logical_ino_args loi;
 	struct btrfs_data_container *inodes;
 	u64 size = 4096;
 	char full_path[PATH_MAX];
 	char *path_ptr;
+	char *mount_path = NULL;
 	DIR *dirstream = NULL;
 
 	optind = 0;
@@ -178,6 +180,35 @@ static int cmd_inspect_logical_resolve(int argc, char **argv)
 	if (!inodes)
 		return 1;
 
+	/* Check if mount root is FS_ROOT */
+	if (getpath) {
+		ret = find_mount_root(argv[optind + 1], &mount_path);
+		if (ret) {
+			error("cannot find mount root: %m");
+			goto out;
+		}
+
+		fd = btrfs_open_file_or_dir(mount_path, &dirstream, 1);
+		if (fd < 0) {
+			ret = 1;
+			goto out;
+		}
+
+		ret = lookup_path_rootid(fd, &rootid);
+		if (ret) {
+			error("failed to lookup root id: %m");
+			goto out;
+		}
+
+		if (rootid != BTRFS_FS_TREE_OBJECTID) {
+			ret = 1;
+printf("cannot resolve path when subvolume is mounted directly. try -P option\n");
+			goto out;
+		}
+
+		close_file_or_dir(fd, dirstream);
+	}
+
 	memset(inodes, 0, sizeof(*inodes));
 	loi.logical = arg_strtou64(argv[optind]);
 	loi.size = size;
@@ -259,6 +290,7 @@ static int cmd_inspect_logical_resolve(int argc, char **argv)
 out:
 	close_file_or_dir(fd, dirstream);
 	free(inodes);
+	free(mount_path);
 	return !!ret;
 }
 
-- 
2.14.4

[PATCH v2 3/3] btrfs-progs: ins: logical-resolve: Add v2 ioctl support

[Misono Tomohiro]

Add --ignore-offset option to logical-resolve command
to show how BTRFS_IOC_LOGICAL_INO_V2 ioctl works
(return every ref to the extent of given logical address).

Documentation is also updated to make meaning more clear.
See below example for more detailed explanation.

[Example]
$ mkfs.btrfs -f $DEV
$ mount $DEV /mnt

$ dd if=/dev/urandom of=/mnt/file bs=4k count=100
$ sync
// split above extent
$ dd if=/dev/urandom of=/mnt/file bs=4k seek=10 count=1 conv=notrunc
$ sync

$ filefrag -v /mnt/file
Filesystem type is: 9123683e
File size of /mnt/file is 409600 (100 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..       9:       3392..      3401:     10:
   1:       10..      10:       3328..      3328:      1:       3402:
   2:       11..      99:       3403..      3491:     89:       3329: last,eof
/mnt/file: 3 extents found

// Actually extent 0 and 2 point to the same extent (with different offset):
$ btrfs inspect-internal dump-tree $DEV | \
    grep -A 2 "$((3392*4096) EXTENT_ITEM"
  item 2 key (13893632 EXTENT_ITEM 409600) itemoff 16153 itemsize 53
          refs 2 gen 7 flags DATA
          extent data backref root FS_TREE objectid 257 offset 0 count 2

$ btrfs inspect-internal dump-tree $DEV | \
    grep -A 4 "257 EXTENT_ITEM"
        item 7 key (257 EXTENT_DATA 0) itemoff 15733 itemsize 53
                generation 7 type 1 (regular)
                extent data disk byte 13893632 nr 409600
                extent data offset 0 nr 40960 ram 409600
                extent compression 0 (none)
        item 8 key (257 EXTENT_DATA 40960) itemoff 15680 itemsize 53
                generation 8 type 1 (regular)
                extent data disk byte 13631488 nr 4096
                extent data offset 0 nr 4096 ram 4096
                extent compression 0 (none)
        item 9 key (257 EXTENT_DATA 45056) itemoff 15627 itemsize 53
                generation 7 type 1 (regular)
                extent data disk byte 13893632 nr 409600
                extent data offset 45056 nr 364544 ram 409600
                extent compression 0 (none)

// v1 ioctl only returns the refs pointing given address block
$ btrfs inspect-internal logical-resolve -P $((3392*4096)) /mnt
inode 257 offset 0 root 5
$ btrfs inspect-internal logical-resolve -P $((3403*4096)) /mnt
inode 257 offset 45056 root 5

// v2 ioctl returns all refs pointing at least one block of given extent
$ btrfs inspect-internal logical-resolve -P --ignore-offset \
  $((3392*4096)) /mnt
inode 257 offset 0 root 5
inode 257 offset 45056 root 5

Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
---
 v1 -> v2
   - add explnation
   - add build assert
   - use long option istead of -i

 Documentation/btrfs-inspect-internal.asciidoc | 15 +++++++++++---
 cmds-inspect.c                                | 29 ++++++++++++++++++++++++---
 ioctl.h                                       | 11 +++++++++-
 libbtrfsutil/btrfs.h                          | 10 ++++++++-
 4 files changed, 57 insertions(+), 8 deletions(-)

diff --git a/Documentation/btrfs-inspect-internal.asciidoc b/Documentation/btrfs-inspect-internal.asciidoc
index e2db6466..5f80323e 100644
--- a/Documentation/btrfs-inspect-internal.asciidoc
+++ b/Documentation/btrfs-inspect-internal.asciidoc
@@ -112,19 +112,28 @@ at 'path', ie. all hardlinks
 -v::::
 verbose mode, print count of returned paths and ioctl() return value
 
-*logical-resolve* [-Pv] [-s <bufsize>] <logical> <path>::
+*logical-resolve* [options] <logical> <path>::
 (needs root privileges)
 +
-resolve paths to all files at given 'logical' address in the linear filesystem space
+get all inode information whose extents containing the given 'logical'
+address blcok and then resolve each filesystem path
 +
 `Options`
 +
 -P::::
-skip the path resolving and print the inodes instead
+skip the path resolving and print the inode number, owner's subvolume id
+and offset in the extent
 -v::::
 verbose mode, print count of returned paths and all ioctl() return values
 -s <bufsize>::::
 set internal buffer for storing the file names to 'bufsize', default is 4096, maximum 64k
+--ignore-offset::::
+return all inode information which points to the extent
+containing the given logical address block regardless of offset.
+i.e. with this option, all files which includes at least one block of
+the extent will be shown, even if given logical address block is not
+included
+This requires version 2 ioctl support (BTRFS_IOC_LOGICAL_INO_V2, since 4.15)
 
 *min-dev-size* [options] <path>::
 (needs root privileges)
diff --git a/cmds-inspect.c b/cmds-inspect.c
index 21aa2903..c261e773 100644
--- a/cmds-inspect.c
+++ b/cmds-inspect.c
@@ -124,13 +124,20 @@ static int cmd_inspect_inode_resolve(int argc, char **argv)
 }
 
 static const char * const cmd_inspect_logical_resolve_usage[] = {
-	"btrfs inspect-internal logical-resolve [-Pv] [-s bufsize] <logical> <path>",
+	"btrfs inspect-internal logical-resolve [options] <logical> <path>",
 	"Get file system paths for the given logical address",
 	"-P          skip the path resolving and print the inodes instead",
 	"-v          verbose mode",
 	"-s bufsize  set inode container's size. This is used to increase inode",
 	"            container's size in case it is not enough to read all the ",
 	"            resolved results. The max value one can set is 64k",
+	"--ignore-offset",
+	"            return all inode information which points to the extent",
+	"            containing the given logical address block regardless of",
+	"            offset. i.e. with this option, all files which includes",
+	"            at least one block of the extent will be shown, even if",
+	"            given logical address block is not included",
+	"            (requires version 2 ioctl support)",
 	NULL
 };
 
@@ -143,7 +150,9 @@ static int cmd_inspect_logical_resolve(int argc, char **argv)
 	int getpath = 1;
 	int bytes_left;
 	u64 rootid;
+	int ignore_offset = 0;
 	struct btrfs_ioctl_logical_ino_args loi;
+	unsigned long ioctl_num = BTRFS_IOC_LOGICAL_INO;
 	struct btrfs_data_container *inodes;
 	u64 size = 4096;
 	char full_path[PATH_MAX];
@@ -153,7 +162,13 @@ static int cmd_inspect_logical_resolve(int argc, char **argv)
 
 	optind = 0;
 	while (1) {
-		int c = getopt(argc, argv, "Pvs:");
+		int c;
+		static const struct option long_options[] = {
+			{ "ignore-offset", no_argument, NULL, 'i' },
+			{NULL, 0, NULL, 0}
+		};
+
+		c = getopt_long(argc, argv, "Pvs:", long_options, NULL);
 		if (c < 0)
 			break;
 
@@ -167,6 +182,9 @@ static int cmd_inspect_logical_resolve(int argc, char **argv)
 		case 's':
 			size = arg_strtou64(optarg);
 			break;
+		case 'i':
+			ignore_offset = 1;
+			break;
 		default:
 			usage(cmd_inspect_logical_resolve_usage);
 		}
@@ -214,13 +232,18 @@ printf("cannot resolve path when subvolume is mounted directly. try -P option\n"
 	loi.size = size;
 	loi.inodes = ptr_to_u64(inodes);
 
+	if (ignore_offset) {
+		loi.flags = BTRFS_LOGICAL_INO_ARGS_IGNORE_OFFSET;
+		ioctl_num = BTRFS_IOC_LOGICAL_INO_V2;
+	}
+
 	fd = btrfs_open_dir(argv[optind + 1], &dirstream, 1);
 	if (fd < 0) {
 		ret = 12;
 		goto out;
 	}
 
-	ret = ioctl(fd, BTRFS_IOC_LOGICAL_INO, &loi);
+	ret = ioctl(fd, ioctl_num, &loi);
 	if (ret < 0) {
 		error("logical ino ioctl: %m");
 		goto out;
diff --git a/ioctl.h b/ioctl.h
index 709e996f..2d6f5d1a 100644
--- a/ioctl.h
+++ b/ioctl.h
@@ -491,10 +491,17 @@ BUILD_ASSERT(sizeof(struct btrfs_ioctl_ino_path_args) == 56);
 struct btrfs_ioctl_logical_ino_args {
 	__u64				logical;	/* in */
 	__u64				size;		/* in */
-	__u64				reserved[4];
+	__u64				reserved[3];	/* must be 0 for now */
+	__u64				flags;		/* in, v2 only */
 	/* struct btrfs_data_container	*inodes;	out   */
 	__u64				inodes;
 };
+BUILD_ASSERT(offsetof(struct btrfs_ioctl_logical_ino_args, flags) == 40);
+/*
+ * Return every ref to the extent, not just those containing logical block.
+ * Requires logical == extent bytenr.
+ */
+#define BTRFS_LOGICAL_INO_ARGS_IGNORE_OFFSET	(1ULL << 0)
 
 enum btrfs_dev_stat_values {
 	/* disk I/O failure stats */
@@ -828,6 +835,8 @@ static inline char *btrfs_err_str(enum btrfs_err_code err_code)
                                   struct btrfs_ioctl_feature_flags[3])
 #define BTRFS_IOC_RM_DEV_V2	_IOW(BTRFS_IOCTL_MAGIC, 58, \
 				   struct btrfs_ioctl_vol_args_v2)
+#define BTRFS_IOC_LOGICAL_INO_V2 _IOWR(BTRFS_IOCTL_MAGIC, 59, \
+					struct btrfs_ioctl_logical_ino_args)
 #ifdef __cplusplus
 }
 #endif
diff --git a/libbtrfsutil/btrfs.h b/libbtrfsutil/btrfs.h
index c293f6bf..b4debba7 100644
--- a/libbtrfsutil/btrfs.h
+++ b/libbtrfsutil/btrfs.h
@@ -609,10 +609,16 @@ struct btrfs_ioctl_ino_path_args {
 struct btrfs_ioctl_logical_ino_args {
 	__u64				logical;	/* in */
 	__u64				size;		/* in */
-	__u64				reserved[4];
+	__u64				reserved[3];	/* must be 0 for now */
+	__u64				flags;		/* in, v2 only */
 	/* struct btrfs_data_container	*inodes;	out   */
 	__u64				inodes;
 };
+/*
+ * Return every ref to the extent, not just those containing logical block.
+ * Requires logical == extent bytenr.
+ */
+#define BTRFS_LOGICAL_INO_ARGS_IGNORE_OFFSET	(1ULL << 0)
 
 enum btrfs_dev_stat_values {
 	/* disk I/O failure stats */
@@ -836,5 +842,7 @@ enum btrfs_err_code {
 				   struct btrfs_ioctl_feature_flags[3])
 #define BTRFS_IOC_RM_DEV_V2 _IOW(BTRFS_IOCTL_MAGIC, 58, \
 				   struct btrfs_ioctl_vol_args_v2)
+#define BTRFS_IOC_LOGICAL_INO_V2 _IOWR(BTRFS_IOCTL_MAGIC, 59, \
+					struct btrfs_ioctl_logical_ino_args)
 
 #endif /* _LINUX_BTRFS_H */
-- 
2.14.4

Re: [PATCH 2/3] btrfs-progs: ins: logical-resolve: Print message when path cannot be resolved

[David Sterba]

On Wed, Jul 25, 2018 at 05:20:17PM +0900, Misono Tomohiro wrote:
> Since BTRFS_IOC_INO_PATHS requires fd of subvolume,

Does it? AFAICS btrfs_ioctl_ino_to_path gets root that's the containing
subvolume of the path given by the user.

> logical-resolve
> cannot find the path when mount point is not FS_TREE
> (because the subvolume path cannot be opened).

Sorry, I don't understand what's the problem here. What you write sounds
like there's a reproducer. If yes, please post it.

Re: [PATCH 1/3] btrfs-progs: ins: logical-resolve: Set correct error value

[David Sterba]

On Wed, Jul 25, 2018 at 05:20:16PM +0900, Misono Tomohiro wrote:
> Reset ret value to zero after snprintf(), which returns the number
> of written chars. Otherwise non-zero value returns after command
> success with -P option. Also set return value from __ino_to_path_fd() to
> reflect the final status for default behavior.
> 
> Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>

Applied, thanks.

Re: [PATCH 2/3] btrfs-progs: ins: logical-resolve: Print message when path cannot be resolved

[Misono Tomohiro]

On 2018/08/04 0:06, David Sterba wrote:
> On Wed, Jul 25, 2018 at 05:20:17PM +0900, Misono Tomohiro wrote:
>> Since BTRFS_IOC_INO_PATHS requires fd of subvolume,
> 
> Does it? AFAICS btrfs_ioctl_ino_to_path gets root that's the containing
> subvolume of the path given by the user.

It is reverse; it returns all paths of given inode number in given subvolume fd.

> 
>> logical-resolve
>> cannot find the path when mount point is not FS_TREE
>> (because the subvolume path cannot be opened).
> 
> Sorry, I don't understand what's the problem here. What you write sounds
> like there's a reproducer. If yes, please post it.

Sure. If mountpoint is FS_TREE, everything is ok:

$ mkfs.btrfs -fq $DEV
$ mount $DEV /mnt

// create snapshot and hardlink
$ btrfs sub create /mnt/sub
$ dd if=/dev/urandom of=/mnt/sub/file bs=1k count=1000
$ btrfs sub snap /mnt/sub /mnt/snap
$ ln /mnt/sub/file /mnt/sub/file2
$ tree --inodes /mnt
/mnt
|-- [    256]  snap
|   `-- [    257]  file
`-- [    256]  sub
    |-- [    257]  file
    `-- [    257]  file2

inode-resolve returns all paths of given inode number in given subvolume:
(it calls IOC_INO_PATHS)
$ btrfs ins inode-resolve 257 /mnt/sub
/mnt/sub/file
/mnt/sub/file2

$ btrfs ins inode-resolve 257 /mnt/snap
/mnt/snap/file

logical-resolve returns all paths containing given logical address in fs:
(it calls IOC_INO_PATHS after IOC_LOGICAL_INO)
$ filefrag -v /mnt/sub/file
Filesystem type is: 9123683e
File size of /mnt/sub/file is 1024000 (250 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..     249:       3392..      3641:    250:             last,shared,eof
/mnt/sub/file: 1 extent found
$ btrfs ins logical-resolve $((3392*4096)) /mnt
/mnt/snap/file
/mnt/sub/file
/mnt/sub/file2


However, when mountpoint is not FS_TREE, above logical-resolve fails:

$ umount /mnt
$ mount -o subvol=sub $DEV /mnt
$ btrfs ins logical-resolve $((3392*4096)) /mnt
ERROR: cannot access '/mnt/snap': No such file or directory

The reasons are
(1) btrfs_list_path_for_root() returns paths from FS_TREE
(2) subvolume path cannot be opened and therefore cannot call IOC_INO_PATHS

(1) may be fixed for mounted subvolume, but if a subvolume is not accessible at all,
there is no way to call IOC_INO_PATHS.
To solve this, we need to pass arbitrary treeid just like INO_LOOKUP ioctl.

Thanks,
Misono