All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Darrick J. Wong" <djwong@us.ibm.com>
To: "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: Andi Kleen <andi@firstfloor.org>,
	Greg Freemyer <greg.freemyer@gmail.com>,
	Andreas Dilger <adilger.kernel@dilger.ca>,
	Theodore Tso <tytso@mit.edu>,
	Sunil Mushran <sunil.mushran@oracle.com>,
	Amir Goldstein <amir73il@gmail.com>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	Mingming Cao <cmm@us.ibm.com>, Joel Becker <jlbec@evilplan.org>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	linux-ext4@vger.kernel.org, Coly Li <colyli@gmail.com>
Subject: Re: [PATCH v1 00/16] ext4: Add metadata checksumming
Date: Mon, 5 Sep 2011 11:55:24 -0700	[thread overview]
Message-ID: <20110905185524.GR12086@tux1.beaverton.ibm.com> (raw)
In-Reply-To: <yq139gcymqj.fsf@sermon.lab.mkp.net>

On Sun, Sep 04, 2011 at 06:19:16PM -0400, Martin K. Petersen wrote:
> >>>>> "Andi" == Andi Kleen <andi@firstfloor.org> writes:
> 
> Andi> Doesn't have any performance numbers.
> 
> It's been a while since I read them. I thought they had some compelling
> numbers. Anyway, made a big difference in real life testing here. For
> sustained I/O we're talking an order of magnitude.
> 
> 
> Andi> You need to keep in mind that PCLMULQDQ uses FPU state, so any
> Andi> speedup for the kernel must be large enough to amortize the cost
> Andi> of saving the FPU state.
> 
> Yeah, my test cases were for bulk database I/O, not for writing a
> handful of fs metadata blocks. Plus for the DB tests the CRC was
> generated in userland.
> 
> I seem to recall Joel picking something other than the hw-accelerated
> CRC32C for ocfs2 metadata and that didn't cause any problems.

Yes, he picked regular CRC32, which has a reasonably fast slice-by-4
software implementation.  For ext4, my original choices were hw acceleration or
the slower single-byte lookup table.  With hw acceleration the overhead of
adding the checksums is about ~10% (for just the metadata operations); with the
single-byte table it was about 50%; and with the proposed slice-by-8 patch it's
about 20%.  Hopefully I can optimize this even more in the future.

> That said, I do see a difference between IP checksum and CRC on normal
> FS workloads with DIX enabled here.

I would hope so, since the IP checksum is much simpler than any CRC...

--D

> Andi> Typically that only works out for quite large buffers, but kernel
> Andi> buffers are relatively small.
> 
> *nod*
> 
> -- 
> Martin K. Petersen	Oracle Linux Engineering
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

  reply	other threads:[~2011-09-05 18:55 UTC|newest]

Thread overview: 50+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-09-01  0:30 [PATCH v1 00/16] ext4: Add metadata checksumming Darrick J. Wong
2011-09-01  0:30 ` [PATCH 01/16] ext4: ext4_dx_add_entry should dirty directory metadata with the directory inode Darrick J. Wong
2011-09-01  0:30 ` [PATCH 02/16] ext4: ext4_rename should dirty dir_bh with the correct directory Darrick J. Wong
2011-09-01  0:30 ` [PATCH 03/16] ext4: ext4_mkdir should dirty dir_block with the parent inode Darrick J. Wong
2011-09-01  0:30 ` [PATCH 04/16] ext4: Create a new BH_Verified flag to avoid unnecessary metadata validation Darrick J. Wong
2011-09-01  0:31 ` [PATCH 05/16] ext4: Create a rocompat flag for extended metadata checksumming Darrick J. Wong
2011-09-01  0:31 ` [PATCH 06/16] ext4: Calculate and verify inode checksums Darrick J. Wong
2011-09-01  2:30   ` Andreas Dilger
2011-09-02 19:32     ` Darrick J. Wong
2011-09-02 22:02       ` Andreas Dilger
2011-09-05 17:57         ` Darrick J. Wong
2011-09-01  0:31 ` [PATCH 07/16] ext4: Create bitmap checksum helper functions Darrick J. Wong
2011-09-01  0:31 ` [PATCH 08/16] ext4: Calculate and verify checksums for inode bitmaps Darrick J. Wong
2011-09-01  4:49   ` Andreas Dilger
2011-09-02 19:18     ` Darrick J. Wong
2011-09-02 21:27       ` Andreas Dilger
2011-09-05 18:22         ` Darrick J. Wong
2011-09-05 18:27           ` Andi Kleen
2011-09-05 19:45           ` James Bottomley
2011-09-05 22:12             ` Darrick J. Wong
2011-09-01  0:31 ` [PATCH 09/16] ext4: Calculate and verify block bitmap checksum Darrick J. Wong
     [not found]   ` <2492E720-3316-4561-8C9C-BBC6E8670EAD@dilger.ca>
2011-09-02 19:08     ` Darrick J. Wong
2011-09-02 21:06       ` Andreas Dilger
2011-09-01  0:31 ` [PATCH 10/16] ext4: Verify and calculate checksums for extent tree blocks Darrick J. Wong
     [not found]   ` <26584BE9-B716-40D5-B3B4-8C5912869648@dilger.ca>
2011-09-02 19:02     ` Darrick J. Wong
2011-09-01  0:31 ` [PATCH 11/16] ext4: Calculate and verify checksums for htree nodes Darrick J. Wong
2011-09-01  0:31 ` [PATCH 12/16] ext4: Calculate and verify checksums of directory leaf blocks Darrick J. Wong
     [not found]   ` <4D5D8FB2-0D8A-4D30-B6D8-51158395C1C9@dilger.ca>
2011-09-02 18:57     ` Darrick J. Wong
2011-09-02 20:52       ` Andreas Dilger
2011-09-05 18:30         ` Darrick J. Wong
2011-09-01  0:32 ` [PATCH 13/16] ext4: Calculate and verify checksums of extended attribute blocks Darrick J. Wong
     [not found]   ` <F4A42DCE-F91C-419B-9153-65A7EA91D241@dilger.ca>
2011-09-02 18:43     ` Darrick J. Wong
2011-09-01  0:32 ` [PATCH 14/16] ext4: Make inode checksum cover empty space Darrick J. Wong
     [not found]   ` <4540D5DB-53D9-4C33-BC6B-868870D42AF3@dilger.ca>
2011-09-02 18:42     ` Darrick J. Wong
2011-09-01  0:32 ` [PATCH 15/16] ext4: Calculate and verify superblock checksum Darrick J. Wong
     [not found]   ` <2882FBB2-797C-4D27-8569-B6826DD34F68@dilger.ca>
2011-09-02 18:40     ` Darrick J. Wong
2011-09-01  0:32 ` [PATCH 16/16] jbd2: Support CRC32C transaction checksums Darrick J. Wong
     [not found]   ` <99019900-30B1-450A-9620-E94371A30CC6@dilger.ca>
2011-09-02 18:31     ` Darrick J. Wong
2011-09-02 14:15 ` [PATCH v1 00/16] ext4: Add metadata checksumming Greg Freemyer
2011-09-02 14:15   ` Greg Freemyer
2011-09-02 18:22   ` Darrick J. Wong
2011-09-02 18:22     ` Darrick J. Wong
2011-09-04 11:41     ` Martin K. Petersen
2011-09-04 16:54       ` Andi Kleen
2011-09-04 17:17         ` Martin K. Petersen
2011-09-04 17:44           ` Andi Kleen
2011-09-04 22:19             ` Martin K. Petersen
2011-09-05 18:55               ` Darrick J. Wong [this message]
2011-09-05 18:45       ` Darrick J. Wong
2011-09-06 12:59         ` Martin K. Petersen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20110905185524.GR12086@tux1.beaverton.ibm.com \
    --to=djwong@us.ibm.com \
    --cc=adilger.kernel@dilger.ca \
    --cc=amir73il@gmail.com \
    --cc=andi@firstfloor.org \
    --cc=cmm@us.ibm.com \
    --cc=colyli@gmail.com \
    --cc=greg.freemyer@gmail.com \
    --cc=jlbec@evilplan.org \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=martin.petersen@oracle.com \
    --cc=sunil.mushran@oracle.com \
    --cc=tytso@mit.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.