[PATCH v5 0/4] btrfs: support xxhash64 checksums

[PATCH v5 0/4] btrfs: support xxhash64 checksums

[Johannes Thumshirn]

Now that Nikolay's XXHASH64 support for the Crypto API has landed and BTRFS is
prepared for an easy addition of new checksums, this patchset implements
XXHASH64 as a second, fast but not cryptographically secure checksum hash.

For changelogs, please see the individual patches.

David Sterba (1):
  btrfs: sysfs: export supported checksums

Johannes Thumshirn (3):
  btrfs: turn checksum type define into a enum
  btrfs: create structure to encode checksum type and length
  btrfs: add xxhash64 to checksumming algorithms

 fs/btrfs/Kconfig                |  1 +
 fs/btrfs/ctree.c                | 23 +++++++++++++++++++++++
 fs/btrfs/ctree.h                | 20 ++------------------
 fs/btrfs/disk-io.c              |  1 +
 fs/btrfs/super.c                |  1 +
 fs/btrfs/sysfs.c                | 29 +++++++++++++++++++++++++++++
 include/uapi/linux/btrfs_tree.h |  5 ++++-
 7 files changed, 61 insertions(+), 19 deletions(-)

-- 
2.16.4

[PATCH v5 1/4] btrfs: turn checksum type define into a enum

[Johannes Thumshirn]

Turn the checksum type definition into a enum. This eases later addition
of new checksums.

Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
---
 include/uapi/linux/btrfs_tree.h | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/include/uapi/linux/btrfs_tree.h b/include/uapi/linux/btrfs_tree.h
index 71246c1941aa..b65c7ee75bc7 100644
--- a/include/uapi/linux/btrfs_tree.h
+++ b/include/uapi/linux/btrfs_tree.h
@@ -300,7 +300,9 @@
 #define BTRFS_CSUM_SIZE 32
 
 /* csum types */
-#define BTRFS_CSUM_TYPE_CRC32	0
+enum btrfs_csum_type {
+	BTRFS_CSUM_TYPE_CRC32	= 0,
+};
 
 /*
  * flags definitions for directory entry item type
-- 
2.16.4

[PATCH v5 2/4] btrfs: create structure to encode checksum type and length

[Johannes Thumshirn]

Create a structure to encode the type and length for the known on-disk
checksums.

This makes it easier to add new checksums later.

Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>

---
Changes to v4:
- Move table and accessor functions from ctree.h to ctree.c

Changes to v2:
- Really remove initializer macro *doh*

Changes to v1:
- Remove initializer macro (David)
---
 fs/btrfs/ctree.c | 22 ++++++++++++++++++++++
 fs/btrfs/ctree.h | 20 ++------------------
 2 files changed, 24 insertions(+), 18 deletions(-)

diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c
index a2f3cd7a619c..9c36b1a878ec 100644
--- a/fs/btrfs/ctree.c
+++ b/fs/btrfs/ctree.c
@@ -29,6 +29,28 @@ static int balance_node_right(struct btrfs_trans_handle *trans,
 static void del_ptr(struct btrfs_root *root, struct btrfs_path *path,
 		    int level, int slot);
 
+static const struct btrfs_csums {
+	u16		size;
+	const char	*name;
+} btrfs_csums[] = {
+	[BTRFS_CSUM_TYPE_CRC32] = { .size = 4, .name = "crc32c" },
+};
+
+int btrfs_super_csum_size(const struct btrfs_super_block *s)
+{
+	u16 t = btrfs_super_csum_type(s);
+	/*
+	 * csum type is validated at mount time
+	 */
+	return btrfs_csums[t].size;
+}
+
+const char *btrfs_super_csum_name(u16 csum_type)
+{
+	/* csum type is validated at mount time */
+	return btrfs_csums[csum_type].name;
+}
+
 struct btrfs_path *btrfs_alloc_path(void)
 {
 	return kmem_cache_zalloc(btrfs_path_cachep, GFP_NOFS);
diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index d27b39858339..2359480749e1 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -82,10 +82,6 @@ struct btrfs_ref;
  */
 #define BTRFS_LINK_MAX 65535U
 
-/* four bytes for CRC32 */
-static const int btrfs_csum_sizes[] = { 4 };
-static const char *btrfs_csum_names[] = { "crc32c" };
-
 #define BTRFS_EMPTY_DIR_SIZE 0
 
 /* ioprio of readahead is set to idle */
@@ -2201,20 +2197,8 @@ BTRFS_SETGET_STACK_FUNCS(super_magic, struct btrfs_super_block, magic, 64);
 BTRFS_SETGET_STACK_FUNCS(super_uuid_tree_generation, struct btrfs_super_block,
 			 uuid_tree_generation, 64);
 
-static inline int btrfs_super_csum_size(const struct btrfs_super_block *s)
-{
-	u16 t = btrfs_super_csum_type(s);
-	/*
-	 * csum type is validated at mount time
-	 */
-	return btrfs_csum_sizes[t];
-}
-
-static inline const char *btrfs_super_csum_name(u16 csum_type)
-{
-	/* csum type is validated at mount time */
-	return btrfs_csum_names[csum_type];
-}
+int btrfs_super_csum_size(const struct btrfs_super_block *s);
+const char *btrfs_super_csum_name(u16 csum_type);
 
 /*
  * The leaf data grows from end-to-front in the node.
-- 
2.16.4

[PATCH v5 3/4] btrfs: add xxhash64 to checksumming algorithms

[Johannes Thumshirn]

Add xxhash64 to the list of possible checksumming algorithms used by
BTRFS.

Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>

---
Changes to v3:
- Reword Subject and add a commit message.
---
 fs/btrfs/Kconfig                | 1 +
 fs/btrfs/ctree.c                | 1 +
 fs/btrfs/disk-io.c              | 1 +
 fs/btrfs/super.c                | 1 +
 include/uapi/linux/btrfs_tree.h | 1 +
 5 files changed, 5 insertions(+)

diff --git a/fs/btrfs/Kconfig b/fs/btrfs/Kconfig
index 38651fae7f21..6d5a01c57da3 100644
--- a/fs/btrfs/Kconfig
+++ b/fs/btrfs/Kconfig
@@ -5,6 +5,7 @@ config BTRFS_FS
 	select CRYPTO
 	select CRYPTO_CRC32C
 	select LIBCRC32C
+	select CRYPTO_XXHASH
 	select ZLIB_INFLATE
 	select ZLIB_DEFLATE
 	select LZO_COMPRESS
diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c
index 9c36b1a878ec..8372d295eb5e 100644
--- a/fs/btrfs/ctree.c
+++ b/fs/btrfs/ctree.c
@@ -34,6 +34,7 @@ static const struct btrfs_csums {
 	const char	*name;
 } btrfs_csums[] = {
 	[BTRFS_CSUM_TYPE_CRC32] = { .size = 4, .name = "crc32c" },
+	[BTRFS_CSUM_TYPE_XXHASH] = { .size = 8, .name = "xxhash64" },
 };
 
 int btrfs_super_csum_size(const struct btrfs_super_block *s)
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 99dfd889b9f7..ac039a4d23ff 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -352,6 +352,7 @@ static bool btrfs_supported_super_csum(u16 csum_type)
 {
 	switch (csum_type) {
 	case BTRFS_CSUM_TYPE_CRC32:
+	case BTRFS_CSUM_TYPE_XXHASH:
 		return true;
 	default:
 		return false;
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index 1b151af25772..60116d0410e5 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -2456,3 +2456,4 @@ module_exit(exit_btrfs_fs)
 
 MODULE_LICENSE("GPL");
 MODULE_SOFTDEP("pre: crc32c");
+MODULE_SOFTDEP("pre: xxhash64");
diff --git a/include/uapi/linux/btrfs_tree.h b/include/uapi/linux/btrfs_tree.h
index b65c7ee75bc7..ba2f125a3a1c 100644
--- a/include/uapi/linux/btrfs_tree.h
+++ b/include/uapi/linux/btrfs_tree.h
@@ -302,6 +302,7 @@
 /* csum types */
 enum btrfs_csum_type {
 	BTRFS_CSUM_TYPE_CRC32	= 0,
+	BTRFS_CSUM_TYPE_XXHASH	= 1,
 };
 
 /*
-- 
2.16.4

[PATCH v5 4/4] btrfs: sysfs: export supported checksums

[Johannes Thumshirn]

From: David Sterba <dsterba@suse.com>

Export supported checksum algorithms via sysfs.

Signed-off-by: David Sterba <dsterba@suse.com>
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>

---
Changes to v2:
- Prevent possible overflow of sysfs attribute
Changes to v1:
- Removed btrfs_checksums_store() function (Nik)
- Renamed sysfs file to supported_checksums
---
 fs/btrfs/sysfs.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index f6d3c80f2e28..cae9c99253c7 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -246,6 +246,24 @@ static umode_t btrfs_feature_visible(struct kobject *kobj,
 	return mode;
 }
 
+static ssize_t btrfs_supported_checksums_show(struct kobject *kobj,
+					      struct kobj_attribute *a,
+					      char *buf)
+{
+	ssize_t ret = 0;
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(btrfs_csums); i++) {
+		ret += snprintf(buf + ret, PAGE_SIZE - ret, "%s%s",
+				(i == 0 ? "" : ", "),
+				btrfs_csums[i].name);
+
+	}
+
+	ret += snprintf(buf + ret, PAGE_SIZE - ret, "\n");
+	return ret;
+}
+
 BTRFS_FEAT_ATTR_INCOMPAT(mixed_backref, MIXED_BACKREF);
 BTRFS_FEAT_ATTR_INCOMPAT(default_subvol, DEFAULT_SUBVOL);
 BTRFS_FEAT_ATTR_INCOMPAT(mixed_groups, MIXED_GROUPS);
@@ -259,6 +277,14 @@ BTRFS_FEAT_ATTR_INCOMPAT(no_holes, NO_HOLES);
 BTRFS_FEAT_ATTR_INCOMPAT(metadata_uuid, METADATA_UUID);
 BTRFS_FEAT_ATTR_COMPAT_RO(free_space_tree, FREE_SPACE_TREE);
 
+static struct btrfs_feature_attr btrfs_attr_features_checksums_name = {
+	.kobj_attr = __INIT_KOBJ_ATTR(supported_checksums, S_IRUGO,
+				      btrfs_supported_checksums_show,
+				      NULL),
+	.feature_set	= FEAT_INCOMPAT,
+	.feature_bit	= 0,
+};
+
 static struct attribute *btrfs_supported_feature_attrs[] = {
 	BTRFS_FEAT_ATTR_PTR(mixed_backref),
 	BTRFS_FEAT_ATTR_PTR(default_subvol),
@@ -272,6 +298,9 @@ static struct attribute *btrfs_supported_feature_attrs[] = {
 	BTRFS_FEAT_ATTR_PTR(no_holes),
 	BTRFS_FEAT_ATTR_PTR(metadata_uuid),
 	BTRFS_FEAT_ATTR_PTR(free_space_tree),
+
+	&btrfs_attr_features_checksums_name.kobj_attr.attr,
+
 	NULL
 };
 
-- 
2.16.4

Re: [PATCH v5 4/4] btrfs: sysfs: export supported checksums

[David Sterba]

On Fri, Aug 30, 2019 at 01:36:11PM +0200, Johannes Thumshirn wrote:
> From: David Sterba <dsterba@suse.com>

I did not write the patch.

> Export supported checksum algorithms via sysfs.
> 
> Signed-off-by: David Sterba <dsterba@suse.com>
> Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>

And 'git am' complains that there are two s-o-b lines.

> Reviewed-by: Nikolay Borisov <nborisov@suse.com>
> 
> ---
> Changes to v2:
> - Prevent possible overflow of sysfs attribute
> Changes to v1:
> - Removed btrfs_checksums_store() function (Nik)
> - Renamed sysfs file to supported_checksums
> ---
>  fs/btrfs/sysfs.c | 29 +++++++++++++++++++++++++++++
>  1 file changed, 29 insertions(+)
> 
> diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
> index f6d3c80f2e28..cae9c99253c7 100644
> --- a/fs/btrfs/sysfs.c
> +++ b/fs/btrfs/sysfs.c
> @@ -246,6 +246,24 @@ static umode_t btrfs_feature_visible(struct kobject *kobj,
>  	return mode;
>  }
>  
> +static ssize_t btrfs_supported_checksums_show(struct kobject *kobj,
> +					      struct kobj_attribute *a,
> +					      char *buf)
> +{
> +	ssize_t ret = 0;
> +	int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(btrfs_csums); i++) {

We'll need one more helper to return the number of csums, btrfs_csums is
not visible in sysfs.c anymore (aka it does not compile).

> +		ret += snprintf(buf + ret, PAGE_SIZE - ret, "%s%s",
> +				(i == 0 ? "" : ", "),

I've checked some sysfs files what's commonly used for a value separator
and there are more examples of ' ', though ', ' is also there. I'd
prefer to use a single letter separator.

> +				btrfs_csums[i].name);
> +
> +	}
> +
> +	ret += snprintf(buf + ret, PAGE_SIZE - ret, "\n");
> +	return ret;
> +}
> +
>  BTRFS_FEAT_ATTR_INCOMPAT(mixed_backref, MIXED_BACKREF);
>  BTRFS_FEAT_ATTR_INCOMPAT(default_subvol, DEFAULT_SUBVOL);
>  BTRFS_FEAT_ATTR_INCOMPAT(mixed_groups, MIXED_GROUPS);
> @@ -259,6 +277,14 @@ BTRFS_FEAT_ATTR_INCOMPAT(no_holes, NO_HOLES);
>  BTRFS_FEAT_ATTR_INCOMPAT(metadata_uuid, METADATA_UUID);
>  BTRFS_FEAT_ATTR_COMPAT_RO(free_space_tree, FREE_SPACE_TREE);
>  
> +static struct btrfs_feature_attr btrfs_attr_features_checksums_name = {
> +	.kobj_attr = __INIT_KOBJ_ATTR(supported_checksums, S_IRUGO,
> +				      btrfs_supported_checksums_show,
> +				      NULL),
> +	.feature_set	= FEAT_INCOMPAT,
> +	.feature_bit	= 0,
> +};
> +
>  static struct attribute *btrfs_supported_feature_attrs[] = {
>  	BTRFS_FEAT_ATTR_PTR(mixed_backref),
>  	BTRFS_FEAT_ATTR_PTR(default_subvol),
> @@ -272,6 +298,9 @@ static struct attribute *btrfs_supported_feature_attrs[] = {
>  	BTRFS_FEAT_ATTR_PTR(no_holes),
>  	BTRFS_FEAT_ATTR_PTR(metadata_uuid),
>  	BTRFS_FEAT_ATTR_PTR(free_space_tree),
> +
> +	&btrfs_attr_features_checksums_name.kobj_attr.attr,

This belongs to the static features, ie. the same scheme as
rmdir_subvol. The two lists are merged together in sysfs directory, but
should be defined separately as they represent different type of
features.

> +
>  	NULL
>  };
>  
> -- 
> 2.16.4

Re: [PATCH v5 0/4] btrfs: support xxhash64 checksums

[David Sterba]

On Fri, Aug 30, 2019 at 01:36:07PM +0200, Johannes Thumshirn wrote:
> Now that Nikolay's XXHASH64 support for the Crypto API has landed and BTRFS is
> prepared for an easy addition of new checksums, this patchset implements
> XXHASH64 as a second, fast but not cryptographically secure checksum hash.
> 
> For changelogs, please see the individual patches.
> 
> David Sterba (1):
>   btrfs: sysfs: export supported checksums
> 
> Johannes Thumshirn (3):
>   btrfs: turn checksum type define into a enum
>   btrfs: create structure to encode checksum type and length
>   btrfs: add xxhash64 to checksumming algorithms

I'll add 1 and 2 to misc-next as they aren't risky, xxhash will wait
until the code freeze. The sysfs export needs more work but it's not
urgent now, would be ok with the rest of hash updates.

Re: [PATCH v5 4/4] btrfs: sysfs: export supported checksums

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 9390 bytes --]

Hi Johannes,

I love your patch! Yet something to improve:

[auto build test ERROR on linus/master]
[cannot apply to v5.3-rc6 next-20190830]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Johannes-Thumshirn/btrfs-turn-checksum-type-define-into-a-enum/20190831-103832
config: x86_64-lkp (attached as .config)
compiler: gcc-7 (Debian 7.4.0-11) 7.4.0
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

If you fix the issue, kindly add following tag
Reported-by: kbuild test robot <lkp@intel.com>

All error/warnings (new ones prefixed by >>):

   In file included from arch/x86/include/asm/percpu.h:45:0,
                    from arch/x86/include/asm/current.h:6,
                    from include/linux/sched.h:12,
                    from fs/btrfs/sysfs.c:6:
   fs/btrfs/sysfs.c: In function 'btrfs_supported_checksums_show':
>> fs/btrfs/sysfs.c:192:29: error: 'btrfs_csums' undeclared (first use in this function); did you mean 'btrfs_chunk'?
     for (i = 0; i < ARRAY_SIZE(btrfs_csums); i++) {
                                ^
   include/linux/kernel.h:47:33: note: in definition of macro 'ARRAY_SIZE'
    #define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]) + __must_be_array(arr))
                                    ^~~
   fs/btrfs/sysfs.c:192:29: note: each undeclared identifier is reported only once for each function it appears in
     for (i = 0; i < ARRAY_SIZE(btrfs_csums); i++) {
                                ^
   include/linux/kernel.h:47:33: note: in definition of macro 'ARRAY_SIZE'
    #define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]) + __must_be_array(arr))
                                    ^~~
   In file included from include/linux/kernel.h:16:0,
                    from arch/x86/include/asm/percpu.h:45,
                    from arch/x86/include/asm/current.h:6,
                    from include/linux/sched.h:12,
                    from fs/btrfs/sysfs.c:6:
   include/linux/build_bug.h:16:45: error: bit-field '<anonymous>' width not an integer constant
    #define BUILD_BUG_ON_ZERO(e) (sizeof(struct { int:(-!!(e)); }))
                                                ^
   include/linux/compiler.h:357:28: note: in expansion of macro 'BUILD_BUG_ON_ZERO'
    #define __must_be_array(a) BUILD_BUG_ON_ZERO(__same_type((a), &(a)[0]))
                               ^~~~~~~~~~~~~~~~~
   include/linux/kernel.h:47:59: note: in expansion of macro '__must_be_array'
    #define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]) + __must_be_array(arr))
                                                              ^~~~~~~~~~~~~~~
>> fs/btrfs/sysfs.c:192:18: note: in expansion of macro 'ARRAY_SIZE'
     for (i = 0; i < ARRAY_SIZE(btrfs_csums); i++) {
                     ^~~~~~~~~~

vim +192 fs/btrfs/sysfs.c

   > 6	#include <linux/sched.h>
     7	#include <linux/slab.h>
     8	#include <linux/spinlock.h>
     9	#include <linux/completion.h>
    10	#include <linux/kobject.h>
    11	#include <linux/bug.h>
    12	#include <linux/debugfs.h>
    13	
    14	#include "ctree.h"
    15	#include "disk-io.h"
    16	#include "transaction.h"
    17	#include "sysfs.h"
    18	#include "volumes.h"
    19	#include "space-info.h"
    20	
    21	static inline struct btrfs_fs_info *to_fs_info(struct kobject *kobj);
    22	static inline struct btrfs_fs_devices *to_fs_devs(struct kobject *kobj);
    23	
    24	static u64 get_features(struct btrfs_fs_info *fs_info,
    25				enum btrfs_feature_set set)
    26	{
    27		struct btrfs_super_block *disk_super = fs_info->super_copy;
    28		if (set == FEAT_COMPAT)
    29			return btrfs_super_compat_flags(disk_super);
    30		else if (set == FEAT_COMPAT_RO)
    31			return btrfs_super_compat_ro_flags(disk_super);
    32		else
    33			return btrfs_super_incompat_flags(disk_super);
    34	}
    35	
    36	static void set_features(struct btrfs_fs_info *fs_info,
    37				 enum btrfs_feature_set set, u64 features)
    38	{
    39		struct btrfs_super_block *disk_super = fs_info->super_copy;
    40		if (set == FEAT_COMPAT)
    41			btrfs_set_super_compat_flags(disk_super, features);
    42		else if (set == FEAT_COMPAT_RO)
    43			btrfs_set_super_compat_ro_flags(disk_super, features);
    44		else
    45			btrfs_set_super_incompat_flags(disk_super, features);
    46	}
    47	
    48	static int can_modify_feature(struct btrfs_feature_attr *fa)
    49	{
    50		int val = 0;
    51		u64 set, clear;
    52		switch (fa->feature_set) {
    53		case FEAT_COMPAT:
    54			set = BTRFS_FEATURE_COMPAT_SAFE_SET;
    55			clear = BTRFS_FEATURE_COMPAT_SAFE_CLEAR;
    56			break;
    57		case FEAT_COMPAT_RO:
    58			set = BTRFS_FEATURE_COMPAT_RO_SAFE_SET;
    59			clear = BTRFS_FEATURE_COMPAT_RO_SAFE_CLEAR;
    60			break;
    61		case FEAT_INCOMPAT:
    62			set = BTRFS_FEATURE_INCOMPAT_SAFE_SET;
    63			clear = BTRFS_FEATURE_INCOMPAT_SAFE_CLEAR;
    64			break;
    65		default:
    66			pr_warn("btrfs: sysfs: unknown feature set %d\n",
    67					fa->feature_set);
    68			return 0;
    69		}
    70	
    71		if (set & fa->feature_bit)
    72			val |= 1;
    73		if (clear & fa->feature_bit)
    74			val |= 2;
    75	
    76		return val;
    77	}
    78	
    79	static ssize_t btrfs_feature_attr_show(struct kobject *kobj,
    80					       struct kobj_attribute *a, char *buf)
    81	{
    82		int val = 0;
    83		struct btrfs_fs_info *fs_info = to_fs_info(kobj);
    84		struct btrfs_feature_attr *fa = to_btrfs_feature_attr(a);
    85		if (fs_info) {
    86			u64 features = get_features(fs_info, fa->feature_set);
    87			if (features & fa->feature_bit)
    88				val = 1;
    89		} else
    90			val = can_modify_feature(fa);
    91	
    92		return snprintf(buf, PAGE_SIZE, "%d\n", val);
    93	}
    94	
    95	static ssize_t btrfs_feature_attr_store(struct kobject *kobj,
    96						struct kobj_attribute *a,
    97						const char *buf, size_t count)
    98	{
    99		struct btrfs_fs_info *fs_info;
   100		struct btrfs_feature_attr *fa = to_btrfs_feature_attr(a);
   101		u64 features, set, clear;
   102		unsigned long val;
   103		int ret;
   104	
   105		fs_info = to_fs_info(kobj);
   106		if (!fs_info)
   107			return -EPERM;
   108	
   109		if (sb_rdonly(fs_info->sb))
   110			return -EROFS;
   111	
   112		ret = kstrtoul(skip_spaces(buf), 0, &val);
   113		if (ret)
   114			return ret;
   115	
   116		if (fa->feature_set == FEAT_COMPAT) {
   117			set = BTRFS_FEATURE_COMPAT_SAFE_SET;
   118			clear = BTRFS_FEATURE_COMPAT_SAFE_CLEAR;
   119		} else if (fa->feature_set == FEAT_COMPAT_RO) {
   120			set = BTRFS_FEATURE_COMPAT_RO_SAFE_SET;
   121			clear = BTRFS_FEATURE_COMPAT_RO_SAFE_CLEAR;
   122		} else {
   123			set = BTRFS_FEATURE_INCOMPAT_SAFE_SET;
   124			clear = BTRFS_FEATURE_INCOMPAT_SAFE_CLEAR;
   125		}
   126	
   127		features = get_features(fs_info, fa->feature_set);
   128	
   129		/* Nothing to do */
   130		if ((val && (features & fa->feature_bit)) ||
   131		    (!val && !(features & fa->feature_bit)))
   132			return count;
   133	
   134		if ((val && !(set & fa->feature_bit)) ||
   135		    (!val && !(clear & fa->feature_bit))) {
   136			btrfs_info(fs_info,
   137				"%sabling feature %s on mounted fs is not supported.",
   138				val ? "En" : "Dis", fa->kobj_attr.attr.name);
   139			return -EPERM;
   140		}
   141	
   142		btrfs_info(fs_info, "%s %s feature flag",
   143			   val ? "Setting" : "Clearing", fa->kobj_attr.attr.name);
   144	
   145		spin_lock(&fs_info->super_lock);
   146		features = get_features(fs_info, fa->feature_set);
   147		if (val)
   148			features |= fa->feature_bit;
   149		else
   150			features &= ~fa->feature_bit;
   151		set_features(fs_info, fa->feature_set, features);
   152		spin_unlock(&fs_info->super_lock);
   153	
   154		/*
   155		 * We don't want to do full transaction commit from inside sysfs
   156		 */
   157		btrfs_set_pending(fs_info, COMMIT);
   158		wake_up_process(fs_info->transaction_kthread);
   159	
   160		return count;
   161	}
   162	
   163	static umode_t btrfs_feature_visible(struct kobject *kobj,
   164					     struct attribute *attr, int unused)
   165	{
   166		struct btrfs_fs_info *fs_info = to_fs_info(kobj);
   167		umode_t mode = attr->mode;
   168	
   169		if (fs_info) {
   170			struct btrfs_feature_attr *fa;
   171			u64 features;
   172	
   173			fa = attr_to_btrfs_feature_attr(attr);
   174			features = get_features(fs_info, fa->feature_set);
   175	
   176			if (can_modify_feature(fa))
   177				mode |= S_IWUSR;
   178			else if (!(features & fa->feature_bit))
   179				mode = 0;
   180		}
   181	
   182		return mode;
   183	}
   184	
   185	static ssize_t btrfs_supported_checksums_show(struct kobject *kobj,
   186						      struct kobj_attribute *a,
   187						      char *buf)
   188	{
   189		ssize_t ret = 0;
   190		int i;
   191	
 > 192		for (i = 0; i < ARRAY_SIZE(btrfs_csums); i++) {
   193			ret += snprintf(buf + ret, PAGE_SIZE - ret, "%s%s",
   194					(i == 0 ? "" : ", "),
   195					btrfs_csums[i].name);
   196	
   197		}
   198	
   199		ret += snprintf(buf + ret, PAGE_SIZE - ret, "\n");
   200		return ret;
   201	}
   202	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 28161 bytes --]

Re: [PATCH v5 4/4] btrfs: sysfs: export supported checksums

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 9398 bytes --]

Hi Johannes,

I love your patch! Yet something to improve:

[auto build test ERROR on linus/master]
[cannot apply to v5.3-rc6 next-20190830]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Johannes-Thumshirn/btrfs-turn-checksum-type-define-into-a-enum/20190831-103832
config: i386-randconfig-a003-201934 (attached as .config)
compiler: gcc-6 (Debian 6.3.0-18+deb9u1) 6.3.0 20170516
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

If you fix the issue, kindly add following tag
Reported-by: kbuild test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   In file included from arch/x86/include/asm/percpu.h:45:0,
                    from arch/x86/include/asm/current.h:6,
                    from include/linux/sched.h:12,
                    from fs/btrfs/sysfs.c:6:
   fs/btrfs/sysfs.c: In function 'btrfs_supported_checksums_show':
>> fs/btrfs/sysfs.c:192:29: error: 'btrfs_csums' undeclared (first use in this function)
     for (i = 0; i < ARRAY_SIZE(btrfs_csums); i++) {
                                ^
   include/linux/kernel.h:47:33: note: in definition of macro 'ARRAY_SIZE'
    #define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]) + __must_be_array(arr))
                                    ^~~
   fs/btrfs/sysfs.c:192:29: note: each undeclared identifier is reported only once for each function it appears in
     for (i = 0; i < ARRAY_SIZE(btrfs_csums); i++) {
                                ^
   include/linux/kernel.h:47:33: note: in definition of macro 'ARRAY_SIZE'
    #define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]) + __must_be_array(arr))
                                    ^~~
   In file included from include/linux/kernel.h:16:0,
                    from arch/x86/include/asm/percpu.h:45,
                    from arch/x86/include/asm/current.h:6,
                    from include/linux/sched.h:12,
                    from fs/btrfs/sysfs.c:6:
   include/linux/build_bug.h:16:45: error: bit-field '<anonymous>' width not an integer constant
    #define BUILD_BUG_ON_ZERO(e) (sizeof(struct { int:(-!!(e)); }))
                                                ^
   include/linux/compiler.h:357:28: note: in expansion of macro 'BUILD_BUG_ON_ZERO'
    #define __must_be_array(a) BUILD_BUG_ON_ZERO(__same_type((a), &(a)[0]))
                               ^~~~~~~~~~~~~~~~~
   include/linux/kernel.h:47:59: note: in expansion of macro '__must_be_array'
    #define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]) + __must_be_array(arr))
                                                              ^~~~~~~~~~~~~~~
   fs/btrfs/sysfs.c:192:18: note: in expansion of macro 'ARRAY_SIZE'
     for (i = 0; i < ARRAY_SIZE(btrfs_csums); i++) {
                     ^~~~~~~~~~

vim +/btrfs_csums +192 fs/btrfs/sysfs.c

   > 6	#include <linux/sched.h>
     7	#include <linux/slab.h>
     8	#include <linux/spinlock.h>
     9	#include <linux/completion.h>
    10	#include <linux/kobject.h>
    11	#include <linux/bug.h>
    12	#include <linux/debugfs.h>
    13	
    14	#include "ctree.h"
    15	#include "disk-io.h"
    16	#include "transaction.h"
    17	#include "sysfs.h"
    18	#include "volumes.h"
    19	#include "space-info.h"
    20	
    21	static inline struct btrfs_fs_info *to_fs_info(struct kobject *kobj);
    22	static inline struct btrfs_fs_devices *to_fs_devs(struct kobject *kobj);
    23	
    24	static u64 get_features(struct btrfs_fs_info *fs_info,
    25				enum btrfs_feature_set set)
    26	{
    27		struct btrfs_super_block *disk_super = fs_info->super_copy;
    28		if (set == FEAT_COMPAT)
    29			return btrfs_super_compat_flags(disk_super);
    30		else if (set == FEAT_COMPAT_RO)
    31			return btrfs_super_compat_ro_flags(disk_super);
    32		else
    33			return btrfs_super_incompat_flags(disk_super);
    34	}
    35	
    36	static void set_features(struct btrfs_fs_info *fs_info,
    37				 enum btrfs_feature_set set, u64 features)
    38	{
    39		struct btrfs_super_block *disk_super = fs_info->super_copy;
    40		if (set == FEAT_COMPAT)
    41			btrfs_set_super_compat_flags(disk_super, features);
    42		else if (set == FEAT_COMPAT_RO)
    43			btrfs_set_super_compat_ro_flags(disk_super, features);
    44		else
    45			btrfs_set_super_incompat_flags(disk_super, features);
    46	}
    47	
    48	static int can_modify_feature(struct btrfs_feature_attr *fa)
    49	{
    50		int val = 0;
    51		u64 set, clear;
    52		switch (fa->feature_set) {
    53		case FEAT_COMPAT:
    54			set = BTRFS_FEATURE_COMPAT_SAFE_SET;
    55			clear = BTRFS_FEATURE_COMPAT_SAFE_CLEAR;
    56			break;
    57		case FEAT_COMPAT_RO:
    58			set = BTRFS_FEATURE_COMPAT_RO_SAFE_SET;
    59			clear = BTRFS_FEATURE_COMPAT_RO_SAFE_CLEAR;
    60			break;
    61		case FEAT_INCOMPAT:
    62			set = BTRFS_FEATURE_INCOMPAT_SAFE_SET;
    63			clear = BTRFS_FEATURE_INCOMPAT_SAFE_CLEAR;
    64			break;
    65		default:
    66			pr_warn("btrfs: sysfs: unknown feature set %d\n",
    67					fa->feature_set);
    68			return 0;
    69		}
    70	
    71		if (set & fa->feature_bit)
    72			val |= 1;
    73		if (clear & fa->feature_bit)
    74			val |= 2;
    75	
    76		return val;
    77	}
    78	
    79	static ssize_t btrfs_feature_attr_show(struct kobject *kobj,
    80					       struct kobj_attribute *a, char *buf)
    81	{
    82		int val = 0;
    83		struct btrfs_fs_info *fs_info = to_fs_info(kobj);
    84		struct btrfs_feature_attr *fa = to_btrfs_feature_attr(a);
    85		if (fs_info) {
    86			u64 features = get_features(fs_info, fa->feature_set);
    87			if (features & fa->feature_bit)
    88				val = 1;
    89		} else
    90			val = can_modify_feature(fa);
    91	
    92		return snprintf(buf, PAGE_SIZE, "%d\n", val);
    93	}
    94	
    95	static ssize_t btrfs_feature_attr_store(struct kobject *kobj,
    96						struct kobj_attribute *a,
    97						const char *buf, size_t count)
    98	{
    99		struct btrfs_fs_info *fs_info;
   100		struct btrfs_feature_attr *fa = to_btrfs_feature_attr(a);
   101		u64 features, set, clear;
   102		unsigned long val;
   103		int ret;
   104	
   105		fs_info = to_fs_info(kobj);
   106		if (!fs_info)
   107			return -EPERM;
   108	
   109		if (sb_rdonly(fs_info->sb))
   110			return -EROFS;
   111	
   112		ret = kstrtoul(skip_spaces(buf), 0, &val);
   113		if (ret)
   114			return ret;
   115	
   116		if (fa->feature_set == FEAT_COMPAT) {
   117			set = BTRFS_FEATURE_COMPAT_SAFE_SET;
   118			clear = BTRFS_FEATURE_COMPAT_SAFE_CLEAR;
   119		} else if (fa->feature_set == FEAT_COMPAT_RO) {
   120			set = BTRFS_FEATURE_COMPAT_RO_SAFE_SET;
   121			clear = BTRFS_FEATURE_COMPAT_RO_SAFE_CLEAR;
   122		} else {
   123			set = BTRFS_FEATURE_INCOMPAT_SAFE_SET;
   124			clear = BTRFS_FEATURE_INCOMPAT_SAFE_CLEAR;
   125		}
   126	
   127		features = get_features(fs_info, fa->feature_set);
   128	
   129		/* Nothing to do */
   130		if ((val && (features & fa->feature_bit)) ||
   131		    (!val && !(features & fa->feature_bit)))
   132			return count;
   133	
   134		if ((val && !(set & fa->feature_bit)) ||
   135		    (!val && !(clear & fa->feature_bit))) {
   136			btrfs_info(fs_info,
   137				"%sabling feature %s on mounted fs is not supported.",
   138				val ? "En" : "Dis", fa->kobj_attr.attr.name);
   139			return -EPERM;
   140		}
   141	
   142		btrfs_info(fs_info, "%s %s feature flag",
   143			   val ? "Setting" : "Clearing", fa->kobj_attr.attr.name);
   144	
   145		spin_lock(&fs_info->super_lock);
   146		features = get_features(fs_info, fa->feature_set);
   147		if (val)
   148			features |= fa->feature_bit;
   149		else
   150			features &= ~fa->feature_bit;
   151		set_features(fs_info, fa->feature_set, features);
   152		spin_unlock(&fs_info->super_lock);
   153	
   154		/*
   155		 * We don't want to do full transaction commit from inside sysfs
   156		 */
   157		btrfs_set_pending(fs_info, COMMIT);
   158		wake_up_process(fs_info->transaction_kthread);
   159	
   160		return count;
   161	}
   162	
   163	static umode_t btrfs_feature_visible(struct kobject *kobj,
   164					     struct attribute *attr, int unused)
   165	{
   166		struct btrfs_fs_info *fs_info = to_fs_info(kobj);
   167		umode_t mode = attr->mode;
   168	
   169		if (fs_info) {
   170			struct btrfs_feature_attr *fa;
   171			u64 features;
   172	
   173			fa = attr_to_btrfs_feature_attr(attr);
   174			features = get_features(fs_info, fa->feature_set);
   175	
   176			if (can_modify_feature(fa))
   177				mode |= S_IWUSR;
   178			else if (!(features & fa->feature_bit))
   179				mode = 0;
   180		}
   181	
   182		return mode;
   183	}
   184	
   185	static ssize_t btrfs_supported_checksums_show(struct kobject *kobj,
   186						      struct kobj_attribute *a,
   187						      char *buf)
   188	{
   189		ssize_t ret = 0;
   190		int i;
   191	
 > 192		for (i = 0; i < ARRAY_SIZE(btrfs_csums); i++) {
   193			ret += snprintf(buf + ret, PAGE_SIZE - ret, "%s%s",
   194					(i == 0 ? "" : ", "),
   195					btrfs_csums[i].name);
   196	
   197		}
   198	
   199		ret += snprintf(buf + ret, PAGE_SIZE - ret, "\n");
   200		return ret;
   201	}
   202	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 32126 bytes --]

Re: [PATCH v5 0/4] btrfs: support xxhash64 checksums

[Johannes Thumshirn]

On 30/08/2019 17:06, David Sterba wrote:
> On Fri, Aug 30, 2019 at 01:36:07PM +0200, Johannes Thumshirn wrote:
>> Now that Nikolay's XXHASH64 support for the Crypto API has landed and BTRFS is
>> prepared for an easy addition of new checksums, this patchset implements
>> XXHASH64 as a second, fast but not cryptographically secure checksum hash.
>>
>> For changelogs, please see the individual patches.
>>
>> David Sterba (1):
>>   btrfs: sysfs: export supported checksums
>>
>> Johannes Thumshirn (3):
>>   btrfs: turn checksum type define into a enum
>>   btrfs: create structure to encode checksum type and length
>>   btrfs: add xxhash64 to checksumming algorithms
> 
> I'll add 1 and 2 to misc-next as they aren't risky, xxhash will wait
> until the code freeze. The sysfs export needs more work but it's not
> urgent now, would be ok with the rest of hash updates.
> 

Yeah the sysfs stuff broke with the ctree.h -> ctree.c move, sorry for
that. I'm just curious why it didn't break with my GCC...



-- 
Johannes Thumshirn                            SUSE Labs Filesystems
jthumshirn@suse.de                                +49 911 74053 689
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5
90409 Nürnberg
Germany
(HRB 247165, AG München)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850