From: Will Deacon <will.deacon@arm.com>
To: Robin Murphy <robin.murphy@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>,
netdev@vger.kernel.org, ilias.apalodimas@linaro.org,
Zhangshaokun <zhangshaokun@hisilicon.com>,
"huanglingyan \(A\)" <huanglingyan2@huawei.com>,
linux-arm-kernel@lists.infradead.org, steve.capper@arm.com
Subject: Re: [PATCH] arm64: do_csum: implement accelerated scalar version
Date: Wed, 15 May 2019 10:47:04 +0100 [thread overview]
Message-ID: <20190515094704.GC24357@fuggles.cambridge.arm.com> (raw)
In-Reply-To: <41b30c72-c1c5-14b2-b2e1-3507d552830d@arm.com>
On Mon, Apr 15, 2019 at 07:18:22PM +0100, Robin Murphy wrote:
> On 12/04/2019 10:52, Will Deacon wrote:
> > I'm waiting for Robin to come back with numbers for a C implementation.
> >
> > Robin -- did you get anywhere with that?
>
> Still not what I would call finished, but where I've got so far (besides an
> increasingly elaborate test rig) is as below - it still wants some unrolling
> in the middle to really fly (and actual testing on BE), but the worst-case
> performance already equals or just beats this asm version on Cortex-A53 with
> GCC 7 (by virtue of being alignment-insensitive and branchless except for
> the loop). Unfortunately, the advantage of C code being instrumentable does
> also come around to bite me...
Is there any interest from anybody in spinning a proper patch out of this?
Shaokun?
Will
> /* Looks dumb, but generates nice-ish code */
> static u64 accumulate(u64 sum, u64 data)
> {
> __uint128_t tmp = (__uint128_t)sum + data;
> return tmp + (tmp >> 64);
> }
>
> unsigned int do_csum_c(const unsigned char *buff, int len)
> {
> unsigned int offset, shift, sum, count;
> u64 data, *ptr;
> u64 sum64 = 0;
>
> offset = (unsigned long)buff & 0x7;
> /*
> * This is to all intents and purposes safe, since rounding down cannot
> * result in a different page or cache line being accessed, and @buff
> * should absolutely not be pointing to anything read-sensitive.
> * It does, however, piss off KASAN...
> */
> ptr = (u64 *)(buff - offset);
> shift = offset * 8;
>
> /*
> * Head: zero out any excess leading bytes. Shifting back by the same
> * amount should be at least as fast as any other way of handling the
> * odd/even alignment, and means we can ignore it until the very end.
> */
> data = *ptr++;
> #ifdef __LITTLE_ENDIAN
> data = (data >> shift) << shift;
> #else
> data = (data << shift) >> shift;
> #endif
> count = 8 - offset;
>
> /* Body: straightforward aligned loads from here on... */
> //TODO: fancy stuff with larger strides and uint128s?
> while(len > count) {
> sum64 = accumulate(sum64, data);
> data = *ptr++;
> count += 8;
> }
> /*
> * Tail: zero any over-read bytes similarly to the head, again
> * preserving odd/even alignment.
> */
> shift = (count - len) * 8;
> #ifdef __LITTLE_ENDIAN
> data = (data << shift) >> shift;
> #else
> data = (data >> shift) << shift;
> #endif
> sum64 = accumulate(sum64, data);
>
> /* Finally, folding */
> sum64 += (sum64 >> 32) | (sum64 << 32);
> sum = sum64 >> 32;
> sum += (sum >> 16) | (sum << 16);
> if (offset & 1)
> return (u16)swab32(sum);
>
> return sum >> 16;
> }
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2019-05-15 9:47 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-18 23:08 [PATCH] arm64: do_csum: implement accelerated scalar version Ard Biesheuvel
2019-02-19 15:08 ` Ilias Apalodimas
2019-02-28 14:16 ` Ard Biesheuvel
2019-02-28 15:13 ` Robin Murphy
2019-02-28 15:28 ` Ard Biesheuvel
2019-04-12 2:31 ` Zhangshaokun
2019-04-12 9:52 ` Will Deacon
2019-04-15 18:18 ` Robin Murphy
2019-05-15 9:47 ` Will Deacon [this message]
2019-05-15 10:15 ` David Laight
2019-05-15 10:57 ` Robin Murphy
2019-05-15 11:13 ` David Laight
2019-05-15 12:39 ` Robin Murphy
2019-05-15 13:54 ` David Laight
2019-05-15 11:02 ` Robin Murphy
2019-05-16 3:14 ` Zhangshaokun
2019-08-15 16:46 ` Will Deacon
2019-08-16 8:15 ` Shaokun Zhang
2019-08-16 14:55 ` Robin Murphy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190515094704.GC24357@fuggles.cambridge.arm.com \
--to=will.deacon@arm.com \
--cc=ard.biesheuvel@linaro.org \
--cc=huanglingyan2@huawei.com \
--cc=ilias.apalodimas@linaro.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=netdev@vger.kernel.org \
--cc=robin.murphy@arm.com \
--cc=steve.capper@arm.com \
--cc=zhangshaokun@hisilicon.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).