[PATCH 0/6] btrfs-progs: add support for LOGICAL_INO_V2 features in logical-resolve

[PATCH 0/6] btrfs-progs: add support for LOGICAL_INO_V2 features in logical-resolve

[Zygo Blaxell]

This patch set adds support for LOGICAL_INO_V2 features:

        - bigger buffer size (16M instead of 64K, default also increased to 64K)

        - IGNORE_OFFSETS flag to look up references by extent instead of block

If the V2 options are used, it calls the V2 ioctl; otherwise, it calls
the V1 ioctl for old kernel compatibility.

[PATCH 1/6] btrfs-progs: ioctl-test: add LOGICAL_INO_V2

[Zygo Blaxell]

Same as LOGICAL_INO, except a different magic number.

Signed-off-by: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
---
 ioctl-test.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/ioctl-test.c b/ioctl-test.c
index 65d584be..a79a13b0 100644
--- a/ioctl-test.c
+++ b/ioctl-test.c
@@ -82,7 +82,8 @@
 	ONE(BTRFS_IOC_GET_FEATURES)		\
 	ONE(BTRFS_IOC_SET_FEATURES)		\
 	ONE(BTRFS_IOC_GET_SUPPORTED_FEATURES)	\
-	ONE(BTRFS_IOC_RM_DEV_V2)
+	ONE(BTRFS_IOC_RM_DEV_V2)		\
+	ONE(BTRFS_IOC_LOGICAL_INO_V2)
 
 #define LIST					\
 	LIST_BASE				\
@@ -160,6 +161,7 @@ static struct ioctl_number expected_list[] = {
 	{ BTRFS_IOC_SET_FEATURES,                   0x0040309439 },
 	{ BTRFS_IOC_GET_SUPPORTED_FEATURES,         0x0080489439 },
 	{ BTRFS_IOC_RM_DEV_V2,                      0x005000943a },
+	{ BTRFS_IOC_LOGICAL_INO_V2,                 0x00c038943b },
 };
 
 static struct btrfs_ioctl_vol_args used_vol_args __attribute__((used));
-- 
2.20.1

[PATCH 2/6] btrfs-progs: libbtrfsutil: add LOGICAL_INO_V2

[Zygo Blaxell]

Update the args structure, add the flags constant and the ioctl magic
number.

Signed-off-by: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
---
 libbtrfsutil/btrfs.h | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/libbtrfsutil/btrfs.h b/libbtrfsutil/btrfs.h
index 944d5013..daa769fd 100644
--- a/libbtrfsutil/btrfs.h
+++ b/libbtrfsutil/btrfs.h
@@ -624,10 +624,14 @@ struct btrfs_ioctl_ino_path_args {
 struct btrfs_ioctl_logical_ino_args {
 	__u64				logical;	/* in */
 	__u64				size;		/* in */
-	__u64				reserved[4];
+	__u64				reserved[3];
+	__u64				flags;		/* in */
 	/* struct btrfs_data_container	*inodes;	out   */
 	__u64				inodes;
 };
+/* Return every ref to the extent, not just those containing logical block.
+ * Requires logical == extent bytenr. */
+#define BTRFS_LOGICAL_INO_ARGS_IGNORE_OFFSET    (1ULL << 0)
 
 enum btrfs_dev_stat_values {
 	/* disk I/O failure stats */
@@ -927,6 +931,8 @@ enum btrfs_err_code {
 				   struct btrfs_ioctl_feature_flags[3])
 #define BTRFS_IOC_RM_DEV_V2 _IOW(BTRFS_IOCTL_MAGIC, 58, \
 				   struct btrfs_ioctl_vol_args_v2)
+#define BTRFS_IOC_LOGICAL_INO_V2 _IOWR(BTRFS_IOCTL_MAGIC, 59, \
+                                     struct btrfs_ioctl_logical_ino_args)
 #define BTRFS_IOC_GET_SUBVOL_INFO _IOR(BTRFS_IOCTL_MAGIC, 60, \
 				struct btrfs_ioctl_get_subvol_info_args)
 #define BTRFS_IOC_GET_SUBVOL_ROOTREF _IOWR(BTRFS_IOCTL_MAGIC, 61, \
-- 
2.20.1

[PATCH 3/6] btrfs-progs: add LOGICAL_INO_V2 to ioctl.h

[Zygo Blaxell]

Update the args structure, add the flags constant and the ioctl magic
number.

Signed-off-by: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
---
 ioctl.h | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/ioctl.h b/ioctl.h
index 66ee599f..1aa80b7b 100644
--- a/ioctl.h
+++ b/ioctl.h
@@ -507,10 +507,14 @@ BUILD_ASSERT(sizeof(struct btrfs_ioctl_ino_path_args) == 56);
 struct btrfs_ioctl_logical_ino_args {
 	__u64				logical;	/* in */
 	__u64				size;		/* in */
-	__u64				reserved[4];
+	__u64				reserved[3];
+	__u64				flags;		/* in */
 	/* struct btrfs_data_container	*inodes;	out   */
 	__u64				inodes;
 };
+/* Return every ref to the extent, not just those containing logical block.
+ * Requires logical == extent bytenr. */
+#define BTRFS_LOGICAL_INO_ARGS_IGNORE_OFFSET    (1ULL << 0)
 
 enum btrfs_dev_stat_values {
 	/* disk I/O failure stats */
@@ -923,6 +927,8 @@ static inline char *btrfs_err_str(enum btrfs_err_code err_code)
                                   struct btrfs_ioctl_feature_flags[3])
 #define BTRFS_IOC_RM_DEV_V2	_IOW(BTRFS_IOCTL_MAGIC, 58, \
 				   struct btrfs_ioctl_vol_args_v2)
+#define BTRFS_IOC_LOGICAL_INO_V2 _IOWR(BTRFS_IOCTL_MAGIC, 59, \
+                                     struct btrfs_ioctl_logical_ino_args)
 #define BTRFS_IOC_GET_SUBVOL_INFO _IOR(BTRFS_IOCTL_MAGIC, 60, \
 				struct btrfs_ioctl_get_subvol_info_args)
 #define BTRFS_IOC_GET_SUBVOL_ROOTREF _IOWR(BTRFS_IOCTL_MAGIC, 61, \
-- 
2.20.1

[PATCH 4/6] btrfs-progs: inspect: add support for LOGICAL_INO_V2 ioctl

[Zygo Blaxell]

Increase the maximum buffer size to SZ_16M.

Add an option (-o) to set the ..._IGNORE_OFFSET flag.

If the buffer size is greater than 64K or the IGNORE_OFFSET option
is used, call ioctl V2; otherwise, use ioctl V1 to be compatible with
older kernels.

Signed-off-by: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
---
 cmds/inspect.c | 22 +++++++++++++++++-----
 1 file changed, 17 insertions(+), 5 deletions(-)

diff --git a/cmds/inspect.c b/cmds/inspect.c
index 758b6e60..81eb8125 100644
--- a/cmds/inspect.c
+++ b/cmds/inspect.c
@@ -126,14 +126,17 @@ static int cmd_inspect_inode_resolve(const struct cmd_struct *cmd,
 static DEFINE_SIMPLE_COMMAND(inspect_inode_resolve, "inode-resolve");
 
 static const char * const cmd_inspect_logical_resolve_usage[] = {
-	"btrfs inspect-internal logical-resolve [-Pv] [-s bufsize] <logical> <path>",
+	"btrfs inspect-internal logical-resolve [-Pvo] [-s bufsize] <logical> <path>",
 	"Get file system paths for the given logical address",
 	"",
 	"-P          skip the path resolving and print the inodes instead",
 	"-v          verbose mode",
+	"-o          ignore offsets when matching references (requires v2 ioctl",
+	"            support in the kernel)",
 	"-s bufsize  set inode container's size. This is used to increase inode",
 	"            container's size in case it is not enough to read all the ",
-	"            resolved results. The max value one can set is 64k",
+	"            resolved results. The max value one can set is 64k with the",
+	"            v1 ioctl. Sizes over 64k will use the v2 ioctl.",
 	NULL
 };
 
@@ -152,10 +155,12 @@ static int cmd_inspect_logical_resolve(const struct cmd_struct *cmd,
 	char full_path[PATH_MAX];
 	char *path_ptr;
 	DIR *dirstream = NULL;
+	u64 flags = 0;
+	unsigned long request = BTRFS_IOC_LOGICAL_INO;
 
 	optind = 0;
 	while (1) {
-		int c = getopt(argc, argv, "Pvs:");
+		int c = getopt(argc, argv, "Pvos:");
 		if (c < 0)
 			break;
 
@@ -166,6 +171,9 @@ static int cmd_inspect_logical_resolve(const struct cmd_struct *cmd,
 		case 'v':
 			verbose = 1;
 			break;
+		case 'o':
+			flags |= BTRFS_LOGICAL_INO_ARGS_IGNORE_OFFSET;
+			break;
 		case 's':
 			size = arg_strtou64(optarg);
 			break;
@@ -177,14 +185,18 @@ static int cmd_inspect_logical_resolve(const struct cmd_struct *cmd,
 	if (check_argc_exact(argc - optind, 2))
 		return 1;
 
-	size = min(size, (u64)SZ_64K);
+	size = min(size, (u64)SZ_16M);
 	inodes = malloc(size);
 	if (!inodes)
 		return 1;
 
+	if (size > SZ_64K || flags != 0)
+		request = BTRFS_IOC_LOGICAL_INO_V2;
+
 	memset(inodes, 0, sizeof(*inodes));
 	loi.logical = arg_strtou64(argv[optind]);
 	loi.size = size;
+	loi.flags = flags;
 	loi.inodes = ptr_to_u64(inodes);
 
 	fd = btrfs_open_dir(argv[optind + 1], &dirstream, 1);
@@ -193,7 +205,7 @@ static int cmd_inspect_logical_resolve(const struct cmd_struct *cmd,
 		goto out;
 	}
 
-	ret = ioctl(fd, BTRFS_IOC_LOGICAL_INO, &loi);
+	ret = ioctl(fd, request, &loi);
 	if (ret < 0) {
 		error("logical ino ioctl: %m");
 		goto out;
-- 
2.20.1

[PATCH 5/6] btrfs-progs: inspect: increase logical-resolve default buffer size to 64K

[Zygo Blaxell]

Filesystems with nontrivial snapshots or dedupe will easily overflow
a 4K buffer.  Bump the size up to the largest size supported by the
V1 ioctl.

Signed-off-by: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
---
 cmds/inspect.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/cmds/inspect.c b/cmds/inspect.c
index 81eb8125..c3c41905 100644
--- a/cmds/inspect.c
+++ b/cmds/inspect.c
@@ -151,7 +151,7 @@ static int cmd_inspect_logical_resolve(const struct cmd_struct *cmd,
 	int bytes_left;
 	struct btrfs_ioctl_logical_ino_args loi;
 	struct btrfs_data_container *inodes;
-	u64 size = 4096;
+	u64 size = SZ_64K;
 	char full_path[PATH_MAX];
 	char *path_ptr;
 	DIR *dirstream = NULL;
-- 
2.20.1

[PATCH 6/6] btrfs-progs: inspect-internal: document new logical-resolve options and kernel requirements

[Zygo Blaxell]

Document the new options requiring the V2 ioctl and the increased
default buffer size.

Signed-off-by: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
---
 Documentation/btrfs-inspect-internal.asciidoc | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/Documentation/btrfs-inspect-internal.asciidoc b/Documentation/btrfs-inspect-internal.asciidoc
index 2abf044a..39e41e74 100644
--- a/Documentation/btrfs-inspect-internal.asciidoc
+++ b/Documentation/btrfs-inspect-internal.asciidoc
@@ -125,7 +125,7 @@ at 'path', ie. all hardlinks
 -v::::
 verbose mode, print count of returned paths and ioctl() return value
 
-*logical-resolve* [-Pv] [-s <bufsize>] <logical> <path>::
+*logical-resolve* [-Pvo] [-s <bufsize>] <logical> <path>::
 (needs root privileges)
 +
 resolve paths to all files at given 'logical' address in the linear filesystem space
@@ -136,8 +136,10 @@ resolve paths to all files at given 'logical' address in the linear filesystem s
 skip the path resolving and print the inodes instead
 -v::::
 verbose mode, print count of returned paths and all ioctl() return values
+-o::::
+ignore offsets, find all references to an extent instead of a single block.  Requires kernel support for the V2 ioctl.
 -s <bufsize>::::
-set internal buffer for storing the file names to 'bufsize', default is 4096, maximum 64k
+set internal buffer for storing the file names to 'bufsize', default is 64k, maximum 16m.  Buffer sizes over 64K require kernel support for the V2 ioctl.
 
 *min-dev-size* [options] <path>::
 (needs root privileges)
-- 
2.20.1

Re: [PATCH 0/6] btrfs-progs: add support for LOGICAL_INO_V2 features in logical-resolve

[Anand Jain]

On 27/11/19 11:55 AM, Zygo Blaxell wrote:
> This patch set adds support for LOGICAL_INO_V2 features:
> 
>          - bigger buffer size (16M instead of 64K, default also increased to 64K)
> 
>          - IGNORE_OFFSETS flag to look up references by extent instead of block
> 
> If the V2 options are used, it calls the V2 ioctl; otherwise, it calls
> the V1 ioctl for old kernel compatibility.
> 
> 

For the whole series.

Reviewed-by: Anand Jain <anand.jain@oracle.com>

(Nit: This should be v4 as this patch was submitted before:

https://lore.kernel.org/linux-btrfs/20170922175847.6071-1-ce3g8jdj@umail.furryterror.org/T/#t
)

Re: [PATCH 0/6] btrfs-progs: add support for LOGICAL_INO_V2 features in logical-resolve

[Zygo Blaxell]

[-- Attachment #1: Type: text/plain, Size: 932 bytes --]

On Mon, Dec 02, 2019 at 07:02:26PM +0800, Anand Jain wrote:
> On 27/11/19 11:55 AM, Zygo Blaxell wrote:
> > This patch set adds support for LOGICAL_INO_V2 features:
> > 
> >          - bigger buffer size (16M instead of 64K, default also increased to 64K)
> > 
> >          - IGNORE_OFFSETS flag to look up references by extent instead of block
> > 
> > If the V2 options are used, it calls the V2 ioctl; otherwise, it calls
> > the V1 ioctl for old kernel compatibility.
> > 
> > 
> 
> For the whole series.
> 
> Reviewed-by: Anand Jain <anand.jain@oracle.com>
> 
> (Nit: This should be v4 as this patch was submitted before:
> 
> https://lore.kernel.org/linux-btrfs/20170922175847.6071-1-ce3g8jdj@umail.furryterror.org/T/#t

Nit^2:  That was the kernel patch, this is a btrfs-progs patch to use
the kernel feature.

I'm assuming kernel and userspace patches get different version
numbering...?

> 
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [PATCH 0/6] btrfs-progs: add support for LOGICAL_INO_V2 features in logical-resolve

[Anand Jain]

On 12/3/19 2:42 AM, Zygo Blaxell wrote:
> On Mon, Dec 02, 2019 at 07:02:26PM +0800, Anand Jain wrote:
>> On 27/11/19 11:55 AM, Zygo Blaxell wrote:
>>> This patch set adds support for LOGICAL_INO_V2 features:
>>>
>>>           - bigger buffer size (16M instead of 64K, default also increased to 64K)
>>>
>>>           - IGNORE_OFFSETS flag to look up references by extent instead of block
>>>
>>> If the V2 options are used, it calls the V2 ioctl; otherwise, it calls
>>> the V1 ioctl for old kernel compatibility.
>>>
>>>
>>
>> For the whole series.
>>
>> Reviewed-by: Anand Jain <anand.jain@oracle.com>
>>
>> (Nit: This should be v4 as this patch was submitted before:
>>
>> https://lore.kernel.org/linux-btrfs/20170922175847.6071-1-ce3g8jdj@umail.furryterror.org/T/#t
> 
> Nit^2:  That was the kernel patch, this is a btrfs-progs patch to use
> the kernel feature.
> 
> I'm assuming kernel and userspace patches get different version
> numbering...?

  Ah. After I commented on the kernel patch at the above link,
  I remember seeing the progs patches which I can't find now.
  And I got confused with the kernel patch. Ok this isn't v4.

Thanks, Anand

>>
>>

Re: [PATCH 0/6] btrfs-progs: add support for LOGICAL_INO_V2 features in logical-resolve

[David Sterba]

On Tue, Nov 26, 2019 at 10:55:03PM -0500, Zygo Blaxell wrote:
> This patch set adds support for LOGICAL_INO_V2 features:
> 
>         - bigger buffer size (16M instead of 64K, default also increased to 64K)
> 
>         - IGNORE_OFFSETS flag to look up references by extent instead of block
> 
> If the V2 options are used, it calls the V2 ioctl; otherwise, it calls
> the V1 ioctl for old kernel compatibility.

Thanks, I've added the patchset to devel.