Linux-Fsdevel Archive on lore.kernel.org
 help / color / Atom feed
From: Andiry Xu <jix024@eng.ucsd.edu>
To: Eric Biggers <ebiggers3@gmail.com>
Cc: Nikolay Borisov <n.borisov.lkml@gmail.com>,
	Linux FS Devel <linux-fsdevel@vger.kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	"linux-nvdimm@lists.01.org" <linux-nvdimm@lists.01.org>,
	Dan Williams <dan.j.williams@intel.com>,
	"Rudoff, Andy" <andy.rudoff@intel.com>,
	coughlan@redhat.com, Steven Swanson <swanson@cs.ucsd.edu>,
	Dave Chinner <david@fromorbit.com>, Jan Kara <jack@suse.com>,
	swhiteho@redhat.com, miklos@szeredi.hu,
	Jian Xu <andiry.xu@gmail.com>, Andiry Xu <jix024@cs.ucsd.edu>,
	Herbert Xu <herbert@gondor.apana.org.au>
Subject: Re: [RFC v2 05/83] Add NOVA filesystem definitions and useful helper routines.
Date: Mon, 19 Mar 2018 14:59:49 -0700
Message-ID: <CAD4Szjvf=VfAm1VLa=0=V_76NujXp+u+qS+7A0xweSmUdUfYUA@mail.gmail.com> (raw)
In-Reply-To: <20180319203021.GA59118@gmail.com>

On Mon, Mar 19, 2018 at 1:30 PM, Eric Biggers <ebiggers3@gmail.com> wrote:
> On Mon, Mar 19, 2018 at 12:39:55PM -0700, Andiry Xu wrote:
>> On Sun, Mar 11, 2018 at 12:22 PM, Eric Biggers <ebiggers3@gmail.com> wrote:
>> > On Sun, Mar 11, 2018 at 02:00:13PM +0200, Nikolay Borisov wrote:
>> >> [Adding Herbert Xu to CC since he is the maintainer of the crypto subsys
>> >> maintainer]
>> >>
>> >> On 10.03.2018 20:17, Andiry Xu wrote:
>> >> <snip>
>> >>
>> >> > +static inline u32 nova_crc32c(u32 crc, const u8 *data, size_t len)
>> >> > +{
>> >> > +   u8 *ptr = (u8 *) data;
>> >> > +   u64 acc = crc; /* accumulator, crc32c value in lower 32b */
>> >> > +   u32 csum;
>> >> > +
>> >> > +   /* x86 instruction crc32 is part of SSE-4.2 */
>> >> > +   if (static_cpu_has(X86_FEATURE_XMM4_2)) {
>> >> > +           /* This inline assembly implementation should be equivalent
>> >> > +            * to the kernel's crc32c_intel_le_hw() function used by
>> >> > +            * crc32c(), but this performs better on test machines.
>> >> > +            */
>> >> > +           while (len > 8) {
>> >> > +                   asm volatile(/* 64b quad words */
>> >> > +                           "crc32q (%1), %0"
>> >> > +                           : "=r" (acc)
>> >> > +                           : "r"  (ptr), "0" (acc)
>> >> > +                   );
>> >> > +                   ptr += 8;
>> >> > +                   len -= 8;
>> >> > +           }
>> >> > +
>> >> > +           while (len > 0) {
>> >> > +                   asm volatile(/* trailing bytes */
>> >> > +                           "crc32b (%1), %0"
>> >> > +                           : "=r" (acc)
>> >> > +                           : "r"  (ptr), "0" (acc)
>> >> > +                   );
>> >> > +                   ptr++;
>> >> > +                   len--;
>> >> > +           }
>> >> > +
>> >> > +           csum = (u32) acc;
>> >> > +   } else {
>> >> > +           /* The kernel's crc32c() function should also detect and use the
>> >> > +            * crc32 instruction of SSE-4.2. But calling in to this function
>> >> > +            * is about 3x to 5x slower than the inline assembly version on
>> >> > +            * some test machines.
>> >>
>> >> That is really odd. Did you try to characterize why this is the case? Is
>> >> it purely the overhead of dispatching to the correct backend function?
>> >> That's a rather big performance hit.
>> >>
>> >> > +            */
>> >> > +           csum = crc32c(crc, data, len);
>> >> > +   }
>> >> > +
>> >> > +   return csum;
>> >> > +}
>> >> > +
>> >
>> > Are you sure that CONFIG_CRYPTO_CRC32C_INTEL was enabled during your tests and
>> > that the accelerated version was being called?  Or, perhaps CRC32C_PCL_BREAKEVEN
>> > (defined in arch/x86/crypto/crc32c-intel_glue.c) needs to be adjusted.  Please
>> > don't hack around performance problems like this; if they exist, they need to be
>> > fixed for everyone.
>> >
>>
>> I have performed the crc32c test on a Xeon X5647 at 2.93GHz, 14G DDR3
>> memory at 1066MHz platform.
>> You are right that enabling CONFIG_CRYPTO_CRC32C_INTEL improves the
>> performance significantly. nova_crc32c() is still slightly faster than
>> crc32c() with the flag enabled.
>>
>> Result numbers are follows: data size in bytes, latency in ns, column
>> 3 is crc32c() with  CONFIG_CRYPTO_CRC32C_INTEL enabled and column 4
>> disabled.
>>
>> data size (bytes)        nova_crc32c()        crc32c() -enabled
>> crc32c() -disabled
>> 64                              19                           21 56
>> 128                            28                           29 99
>> 256                            46                           43 182
>> 512                            82                           149 354
>> 1024                          157                         232 728
>> 2048                          305                         415 1440
>> 4096                          603                         725 2869
>>
>
> Probably CRC32C_PCL_BREAKEVEN needs to be adjusted for that CPU, as I suggested
> may be the case; notice that your measured speeds are about the same before 512
> (CRC32C_PCL_BREAKEVEN) bytes, but the crypto API version is slower at >= 512
> bytes.   It would be possible to set the breakeven point in
> crc32c_intel_mod_init() depending on the CPU.  Again, if the performance is not
> good enough you need to fix it for everyone, not hack around it.
>

We verify that by setting CRC32C_PCL_BREAKEVEN to 8192, the
performance difference between nova_crc32c() and kernel's crc32c() is
negligible. Thanks for the comments, and I will use kernel's crc32c()
in the next version.

Thanks,
Andiry

  reply index

Thread overview: 119+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-03-10 18:17 [RFC v2 00/83] NOVA: a new file system for persistent memory Andiry Xu
2018-03-10 18:17 ` [RFC v2 01/83] Introduction and documentation of NOVA filesystem Andiry Xu
2018-03-19 20:43   ` Randy Dunlap
2018-03-19 23:00     ` Andiry Xu
2018-04-22  8:05   ` Pavel Machek
2018-03-10 18:17 ` [RFC v2 02/83] Add nova_def.h Andiry Xu
2018-03-10 18:17 ` [RFC v2 03/83] Add super.h Andiry Xu
2018-03-15  4:54   ` Darrick J. Wong
2018-03-15  6:11     ` Andiry Xu
2018-03-15  9:05       ` Arnd Bergmann
2018-03-15 17:51         ` Andiry Xu
2018-03-15 20:04           ` Andreas Dilger
2018-03-15 20:38           ` Arnd Bergmann
2018-03-16  2:59             ` Theodore Y. Ts'o
2018-03-16  6:17               ` Andiry Xu
2018-03-16  6:30                 ` Darrick J. Wong
2018-03-16  9:19               ` Arnd Bergmann
2018-03-10 18:17 ` [RFC v2 04/83] NOVA inode definition Andiry Xu
2018-03-15  5:06   ` Darrick J. Wong
2018-03-15  6:16     ` Andiry Xu
2018-03-10 18:17 ` [RFC v2 05/83] Add NOVA filesystem definitions and useful helper routines Andiry Xu
2018-03-11 12:00   ` Nikolay Borisov
2018-03-11 19:22     ` Eric Biggers
2018-03-11 21:45       ` Andiry Xu
2018-03-19 19:39       ` Andiry Xu
2018-03-19 20:30         ` Eric Biggers
2018-03-19 21:59           ` Andiry Xu [this message]
2018-03-10 18:17 ` [RFC v2 06/83] Add inode get/read methods Andiry Xu
2018-04-23  6:12   ` Darrick J. Wong
2018-04-23 15:55     ` Andiry Xu
2018-03-10 18:17 ` [RFC v2 07/83] Initialize inode_info and rebuild inode information in nova_iget() Andiry Xu
2018-03-10 18:17 ` [RFC v2 08/83] NOVA superblock operations Andiry Xu
2018-03-10 18:17 ` [RFC v2 09/83] Add Kconfig and Makefile Andiry Xu
2018-03-11 12:15   ` Nikolay Borisov
2018-03-11 21:32     ` Andiry Xu
2018-03-10 18:17 ` [RFC v2 10/83] Add superblock integrity check Andiry Xu
2018-03-10 18:17 ` [RFC v2 11/83] Add timing and I/O statistics for performance analysis and profiling Andiry Xu
2018-03-10 18:17 ` [RFC v2 12/83] Add timing for mount and init Andiry Xu
2018-03-10 18:17 ` [RFC v2 13/83] Add remount_fs and show_options methods Andiry Xu
2018-03-10 18:17 ` [RFC v2 14/83] Add range node kmem cache Andiry Xu
2018-03-11 11:55   ` Nikolay Borisov
2018-03-11 21:31     ` Andiry Xu
2018-03-10 18:17 ` [RFC v2 15/83] Add free list data structure Andiry Xu
2018-03-10 18:17 ` [RFC v2 16/83] Initialize block map and free lists in nova_init() Andiry Xu
2018-03-11 12:12   ` Nikolay Borisov
2018-03-11 21:30     ` Andiry Xu
2018-03-10 18:17 ` [RFC v2 17/83] Add statfs support Andiry Xu
2018-03-10 18:17 ` [RFC v2 18/83] Add freelist statistics printing Andiry Xu
2018-03-10 18:18 ` [RFC v2 19/83] Add pmem block free routines Andiry Xu
2018-03-10 18:18 ` [RFC v2 20/83] Pmem block allocation routines Andiry Xu
2018-03-10 18:18 ` [RFC v2 21/83] Add log structure Andiry Xu
2018-03-10 18:18 ` [RFC v2 22/83] Inode log pages allocation and reclaimation Andiry Xu
2018-03-10 18:18 ` [RFC v2 23/83] Save allocator to pmem in put_super Andiry Xu
2018-03-10 18:18 ` [RFC v2 24/83] Initialize and allocate inode table Andiry Xu
2018-03-10 18:18 ` [RFC v2 25/83] Support get normal inode address and inode table extentsion Andiry Xu
2018-03-10 18:18 ` [RFC v2 26/83] Add inode_map to track inuse inodes Andiry Xu
2018-03-10 18:18 ` [RFC v2 27/83] Save the inode inuse list to pmem upon umount Andiry Xu
2018-03-10 18:18 ` [RFC v2 28/83] Add NOVA address space operations Andiry Xu
2018-03-10 18:18 ` [RFC v2 29/83] Add write_inode and dirty_inode routines Andiry Xu
2018-03-10 18:18 ` [RFC v2 30/83] New NOVA inode allocation Andiry Xu
2018-03-10 18:18 ` [RFC v2 31/83] Add new vfs " Andiry Xu
2018-03-10 18:18 ` [RFC v2 32/83] Add log entry definitions Andiry Xu
2018-03-10 18:18 ` [RFC v2 33/83] Inode log and entry printing for debug purpose Andiry Xu
2018-03-10 18:18 ` [RFC v2 34/83] Journal: NOVA light weight journal definitions Andiry Xu
2018-03-10 18:18 ` [RFC v2 35/83] Journal: Lite journal helper routines Andiry Xu
2018-03-10 18:18 ` [RFC v2 36/83] Journal: Lite journal recovery Andiry Xu
2018-03-10 18:18 ` [RFC v2 37/83] Journal: Lite journal create and commit Andiry Xu
2018-03-10 18:18 ` [RFC v2 38/83] Journal: NOVA lite journal initialization Andiry Xu
2018-03-10 18:18 ` [RFC v2 39/83] Log operation: dentry append Andiry Xu
2018-03-10 18:18 ` [RFC v2 40/83] Log operation: file write entry append Andiry Xu
2018-03-10 18:18 ` [RFC v2 41/83] Log operation: setattr " Andiry Xu
2018-03-10 18:18 ` [RFC v2 42/83] Log operation: link change append Andiry Xu
2018-03-10 18:18 ` [RFC v2 43/83] Log operation: in-place update log entry Andiry Xu
2018-03-10 18:18 ` [RFC v2 44/83] Log operation: invalidate log entries Andiry Xu
2018-03-10 18:18 ` [RFC v2 45/83] Log operation: file inode log lookup and assign Andiry Xu
2018-03-10 18:18 ` [RFC v2 46/83] Dir: Add Directory radix tree insert/remove methods Andiry Xu
2018-03-10 18:18 ` [RFC v2 47/83] Dir: Add initial dentries when initializing a directory inode log Andiry Xu
2018-03-10 18:18 ` [RFC v2 48/83] Dir: Readdir operation Andiry Xu
2018-03-10 18:18 ` [RFC v2 49/83] Dir: Append create/remove dentry Andiry Xu
2018-03-10 18:18 ` [RFC v2 50/83] Inode: Add nova_evict_inode Andiry Xu
2018-03-10 18:18 ` [RFC v2 51/83] Rebuild: directory inode Andiry Xu
2018-03-10 18:18 ` [RFC v2 52/83] Rebuild: file inode Andiry Xu
2018-03-10 18:18 ` [RFC v2 53/83] Namei: lookup Andiry Xu
2018-03-10 18:18 ` [RFC v2 54/83] Namei: create and mknod Andiry Xu
2018-03-10 18:18 ` [RFC v2 55/83] Namei: mkdir Andiry Xu
2018-03-10 18:18 ` [RFC v2 56/83] Namei: link and unlink Andiry Xu
2018-03-10 18:18 ` [RFC v2 57/83] Namei: rmdir Andiry Xu
2018-03-10 18:18 ` [RFC v2 58/83] Namei: rename Andiry Xu
2018-03-10 18:18 ` [RFC v2 59/83] Namei: setattr Andiry Xu
2018-03-10 18:18 ` [RFC v2 60/83] Add special inode operations Andiry Xu
2018-03-10 18:18 ` [RFC v2 61/83] Super: Add nova_export_ops Andiry Xu
2018-03-10 18:18 ` [RFC v2 62/83] File: getattr and file inode operations Andiry Xu
2018-03-10 18:18 ` [RFC v2 63/83] File operation: llseek Andiry Xu
2018-03-10 18:18 ` [RFC v2 64/83] File operation: open, fsync, flush Andiry Xu
2018-03-10 18:18 ` [RFC v2 65/83] File operation: read Andiry Xu
2018-03-10 18:18 ` [RFC v2 66/83] Super: Add file write item cache Andiry Xu
2018-03-10 18:18 ` [RFC v2 67/83] Dax: commit list of file write items to log Andiry Xu
2018-03-10 18:18 ` [RFC v2 68/83] File operation: copy-on-write write Andiry Xu
2018-03-10 18:18 ` [RFC v2 69/83] Super: Add module param inplace_data_updates Andiry Xu
2018-03-10 18:18 ` [RFC v2 70/83] File operation: Inplace write Andiry Xu
2018-03-10 18:18 ` [RFC v2 71/83] Symlink support Andiry Xu
2018-03-10 18:18 ` [RFC v2 72/83] File operation: fallocate Andiry Xu
2018-03-10 18:18 ` [RFC v2 73/83] Dax: Add iomap operations Andiry Xu
2018-03-10 18:18 ` [RFC v2 74/83] File operation: Mmap Andiry Xu
2018-03-10 18:18 ` [RFC v2 75/83] File operation: read/write iter Andiry Xu
2018-03-10 18:18 ` [RFC v2 76/83] Ioctl support Andiry Xu
2018-03-10 18:18 ` [RFC v2 77/83] GC: Fast garbage collection Andiry Xu
2018-03-10 18:18 ` [RFC v2 78/83] GC: Thorough " Andiry Xu
2018-03-10 18:19 ` [RFC v2 79/83] Normal recovery Andiry Xu
2018-03-10 18:19 ` [RFC v2 80/83] Failure recovery: bitmap operations Andiry Xu
2018-03-10 18:19 ` [RFC v2 81/83] Failure recovery: Inode pages recovery routines Andiry Xu
2018-03-10 18:19 ` [RFC v2 82/83] Failure recovery: Per-CPU recovery Andiry Xu
2018-03-10 18:19 ` [RFC v2 83/83] Sysfs support Andiry Xu
2018-03-15  0:33   ` Randy Dunlap
2018-03-15  6:07     ` Andiry Xu
2018-03-22 15:00   ` David Sterba
2018-03-23  0:31     ` Andiry Xu
2018-03-11  2:14 ` [RFC v2 00/83] NOVA: a new file system for persistent memory Theodore Y. Ts'o
2018-03-11  4:58   ` Andiry Xu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAD4Szjvf=VfAm1VLa=0=V_76NujXp+u+qS+7A0xweSmUdUfYUA@mail.gmail.com' \
    --to=jix024@eng.ucsd.edu \
    --cc=andiry.xu@gmail.com \
    --cc=andy.rudoff@intel.com \
    --cc=coughlan@redhat.com \
    --cc=dan.j.williams@intel.com \
    --cc=david@fromorbit.com \
    --cc=ebiggers3@gmail.com \
    --cc=herbert@gondor.apana.org.au \
    --cc=jack@suse.com \
    --cc=jix024@cs.ucsd.edu \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nvdimm@lists.01.org \
    --cc=miklos@szeredi.hu \
    --cc=n.borisov.lkml@gmail.com \
    --cc=swanson@cs.ucsd.edu \
    --cc=swhiteho@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-Fsdevel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-fsdevel/0 linux-fsdevel/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-fsdevel linux-fsdevel/ https://lore.kernel.org/linux-fsdevel \
		linux-fsdevel@vger.kernel.org
	public-inbox-index linux-fsdevel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-fsdevel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git