[PATCH v6 00/10] crc32

[PATCH v6 00/10] crc32

[Bob Pearson]

Hi,

This is an attempt to resolve all the issues that were left in the last review.
There is one proposed change that is still causing a difference of opinion
which has to do with the form of the loops and their performance on x86 and ppc
This version has the change to the form that runs faster on x86 as an ifdef.

This patch series provides improved performance for computing the crc32
polynomial on common hardware by adding the Slicing-by-8 algorithm to the
existing algorithms already included. The new algorithm is very closely
related to the existing algorithm so the extension requires small changes
to implement.  Additionally it cleans up some warnings in the existing
code and adds a kernel mode optional self test to replace the existing
user mode self test.

A description of the existing and new algorithm is included in
Documentation/crc32.txt.

Re: [PATCH v6 00/10] crc32

[Andrew Morton]

On Wed, 31 Aug 2011 17:29:32 -0500
Bob Pearson <rpearson@systemfabricworks.com> wrote:

> This is an attempt to resolve all the issues that were left in the last review.
> There is one proposed change that is still causing a difference of opinion
> which has to do with the form of the loops and their performance on x86 and ppc
> This version has the change to the form that runs faster on x86 as an ifdef.
> 
> This patch series provides improved performance for computing the crc32
> polynomial on common hardware by adding the Slicing-by-8 algorithm to the
> existing algorithms already included. The new algorithm is very closely
> related to the existing algorithm so the extension requires small changes
> to implement.  Additionally it cleans up some warnings in the existing
> code and adds a kernel mode optional self test to replace the existing
> user mode self test.
> 
> A description of the existing and new algorithm is included in
> Documentation/crc32.txt.

So...  are the crc wars over yet?  I hope it's safe to look at the
patches now ;)

Please see Documentation/SubmittingPatches Section 15's discussion of
patch Subject: lines.

Is there any code in the kernel which uses the crc code so much that we
actually care about its performance?

Re: [PATCH v6 00/10] crc32

[Joakim Tjernlund]

Andrew Morton <akpm@linux-foundation.org> wrote on 2011/09/03 01:50:20:
>
> On Wed, 31 Aug 2011 17:29:32 -0500
> Bob Pearson <rpearson@systemfabricworks.com> wrote:
>
> > This is an attempt to resolve all the issues that were left in the last review.
> > There is one proposed change that is still causing a difference of opinion
> > which has to do with the form of the loops and their performance on x86 and ppc
> > This version has the change to the form that runs faster on x86 as an ifdef.
> >
> > This patch series provides improved performance for computing the crc32
> > polynomial on common hardware by adding the Slicing-by-8 algorithm to the
> > existing algorithms already included. The new algorithm is very closely
> > related to the existing algorithm so the extension requires small changes
> > to implement.  Additionally it cleans up some warnings in the existing
> > code and adds a kernel mode optional self test to replace the existing
> > user mode self test.
> >
> > A description of the existing and new algorithm is included in
> > Documentation/crc32.txt.
>
> So...  are the crc wars over yet?  I hope it's safe to look at the
> patches now ;)

Hehe, not sure :) I got side tracked with other things and haven't had time
to look into this last series.

>
> Please see Documentation/SubmittingPatches Section 15's discussion of
> patch Subject: lines.
>
> Is there any code in the kernel which uses the crc code so much that we
> actually care about its performance?

Sure, JFFS2/UBIFS/LogFS uses it heavily. SCTP too although it uses CRC32c, the
SW impl. of CRC32c should be moved to the same place as crc32.c

 Jocke

RE: [PATCH v6 00/10] crc32

[Bob Pearson]

I took the holiday weekend off so this is somewhat delayed.

> -----Original Message-----
> From: Andrew Morton [mailto:akpm@linux-foundation.org]
> Sent: Friday, September 02, 2011 6:50 PM
> To: Bob Pearson
> Cc: linux-kernel@vger.kernel.org; fzago@systemfabricworks.com; Joakim
> Tjernlund; George Spelvin
> Subject: Re: [PATCH v6 00/10] crc32
> 
> On Wed, 31 Aug 2011 17:29:32 -0500
> Bob Pearson <rpearson@systemfabricworks.com> wrote:
> 
> > This is an attempt to resolve all the issues that were left in the last
review.
> > There is one proposed change that is still causing a difference of
opinion
> > which has to do with the form of the loops and their performance on x86
> and ppc
> > This version has the change to the form that runs faster on x86 as an
ifdef.
> >
> > This patch series provides improved performance for computing the crc32
> > polynomial on common hardware by adding the Slicing-by-8 algorithm to
> the
> > existing algorithms already included. The new algorithm is very closely
> > related to the existing algorithm so the extension requires small
changes
> > to implement.  Additionally it cleans up some warnings in the existing
> > code and adds a kernel mode optional self test to replace the existing
> > user mode self test.
> >
> > A description of the existing and new algorithm is included in
> > Documentation/crc32.txt.
> 
> So...  are the crc wars over yet?  I hope it's safe to look at the
> patches now ;)

I hope so.

> 
> Please see Documentation/SubmittingPatches Section 15's discussion of
> patch Subject: lines.

Which sin am I committing? Use of 'vn' seems widespread but is not
specifically mentioned in the document. I am not sure about whether crc32
rises to the level of a 'subsystem' but chose that label. I am missing a
colon though.

> 
> Is there any code in the kernel which uses the crc code so much that we
> actually care about its performance?

I can only speak for myself. I am working on a project to write a software
emulation of an RDMA (InfiniBand like) protocol called RoCE or RDMA over
Converged Ethernet. This protocol uses CRC32 as an L3 CRC that is carried
end to end. Without this change to CRC32 the best performance on a single
core was well below 1GB/src. With it, performance is closer to the wire
speed of hardware implementations, i.e. around 1GB/sec.