Re: [PATCH 0/4] Add xxhash64 and sha256 as possible new checksums

[Nikolay Borisov <nborisov@suse.com>]

On 7.10.19 г. 12:11 ч., Johannes Thumshirn wrote:
> This series adds support for two additional checksum algorithms to btrfs. These
> algorithms are xxhash64[1] and sha256[2].
> 
> xxhash64 is a fast non-cryptographic hash function with good collision resistance.
> It has a constant output length of 8 Byte (64 Bit), it provides a good
> trade-off between collision resistance and speed compared to the currently
> used crc32c.
> 
> sha256 is the 32 Byte (256 Bit) variant of the SHA-2 cryptographic hash. It
> provides cryptographically secure collision resistance with a trade off in
> speed.
> 
> Support for xxhash64 in mkfs.btrfs is in the current devel branch and sha256
> support will be sent separately after this patch-set.
> 
> In addition to adding these two hash algorithms two sysfs files are
> implemented, one being /sys/fs/btrfs/features/supported_checksums showing the
> in kernel support for different checksumming algorithms. The other one is
> /sys/fs/btrfs/$FSID/checksum showing the checksum used for a specific
> file-system and the used in-kernel driver for this checksum.
> 
> Here is an example in a qemu vm:
> host:/# cat /sys/fs/btrfs/features/supported_checksums
> crc32c, xxhash64, sha256
> host:/# cat /sys/fs/btrfs/3cf09516-5bb8-498f-834d-e9ec54043546/checksum
> sha256 (sha256-generic)
> 
> This series has survived the usual regression testing with xfstests.
> 
> I could not observe any performance differences between any of these hashes in
> my test setup 256K mixed read-write IO to a single file from a single process
> on both a 5700rpm SATA 3G Disk behind a HPE SmartArray RAID HBA and RAM Disk.
> 
> Here's the raw numbers for the spinning rust behind SATA:
> CRC32C Buffered Read (KiB/s): Avg: 7881, Min: 7495, Max: 8744, Stdev: 508
> CRC32C Buffered Write (KiB/s): Avg: 7883, Min: 7497, Max: 8746, Stdev: 508
>                                
> CRC32C Direct Read (KiB/s): Avg: 331, Min: 319, Max: 339, Stdev: 7
> CRC32C Direct Write (KiB/s): Avg: 331, Min: 319, Max: 339, Stdev: 7
> 
> XXHASH64 Buffered Read (KiB/s): Avg: 8143, Min: 7748, Max: 8721, Stdev: 355
> XXHASH64 Buffered Write (KiB/s): Avg: 8145, Min: 7750, Max: 8722, Stdev: 355
> 
> XXHASH64 Direct Read (KiB/s): Avg: 311, Min: 248, Max: 336, Stdev: 36
> XXHASH64 Direct Write (KiB/s): Avg: 311, Min: 248, Max: 336, Stdev: 36
>                                
> SHA256 Buffered Read (KiB/s): Avg: 7997, Min: 7665, Max: 8336, Stdev: 273
> SHA256 Buffered Write (KiB/s): Avg: 7998, Min: 7666, Max: 8337, Stdev: 273
> 
> SHA256 Direct Read (KiB/s): Avg: 312, Min: 248, Max: 336, Stdev: 36
> SHA256 Direct Write (KiB/s): Avg: 312, Min: 248, Max: 336, Stdev: 36
> 
> The reason I could not observe any changes in performance is the fact that the
> btrfs checksumming process takes only 0.04% of the IO path. This also explains
> the very small standard deviation in the above table as I stooped benchmarking
> after 5 benchmark runs.
> 
> The hottest call chain (according to perf) is this:
> 
> 17.08%     0.00%  kworker/u128:9-  [kernel.vmlinux]  [k] btrfs_finish_ordered_io
>  |
>  ---btrfs_finish_ordered_io
>     |          
>      --17.04%--insert_reserved_file_extent.constprop.75
>        |          
>         --17.02%--__btrfs_drop_extents
>      	   |          
>      	    --16.94%--btrfs_free_extent
>      		|          
>      		 --16.94%--btrfs_add_delayed_data_ref
>      		   |          
>      		    --16.90%--btrfs_qgroup_trace_extent_post

Yeah, we know qgroups tracing and backrefs resolv are somewhat slow. How
about benchmarking without them since I believe this to be more
representative of how people use btrfs.

>      			      btrfs_find_all_roots
>      			      |          
>      			       --16.90%--btrfs_find_all_roots_safe
>      				 |          
>      				  --16.89%--find_parent_nodes
>      				    |          
>      				     --16.68%--resolve_indirect_refs
> 					       [snip]
> 
> [1] https://cyan4973.github.io/xxHash
> [2] https://en.wikipedia.org/wiki/SHA-2
> 
> David Sterba (1):
>   btrfs: sysfs: export supported checksums
> 
> Johannes Thumshirn (3):
>   btrfs: add xxhash64 to checksumming algorithms
>   btrfs: add sha256 to checksumming algorithms
>   btrfs: show used checksum driver per filesystem in sysfs
> 
>  fs/btrfs/Kconfig                |  2 ++
>  fs/btrfs/ctree.c                |  7 ++++++
>  fs/btrfs/ctree.h                |  2 ++
>  fs/btrfs/disk-io.c              |  2 ++
>  fs/btrfs/super.c                |  2 ++
>  fs/btrfs/sysfs.c                | 48 +++++++++++++++++++++++++++++++++++++++++
>  include/uapi/linux/btrfs_tree.h |  2 ++
>  7 files changed, 65 insertions(+)
>