Does CHECKSUM_COMPLETE work at all?

Does CHECKSUM_COMPLETE work at all?

[Tom Herbert]

If a driver sets CHECKSUM_COMPLETE in a packet, I don't see anywhere
in either the Ethernet rcv path or ip_rcv where the skb->csum is being
pulled (no *pull_rcsum calls). I believe CHECKSUM_COMPLETE means the
checksum is over the whole packet (L2, L3, and L4), so it looks like
TCP would never see a usable value.

Is this interpretation correct? There's only a handful of drivers
providing CHECKSUM_COMPLETE can anyone who has access verify this?

Thanks,
Tom

RE: Does CHECKSUM_COMPLETE work at all?

[Dimitrios Michailidis]

Tom Herbert wrote:
> If a driver sets CHECKSUM_COMPLETE in a packet, I don't see anywhere
> in either the Ethernet rcv path or ip_rcv where the skb->csum is being
> pulled (no *pull_rcsum calls). I believe CHECKSUM_COMPLETE means the
> checksum is over the whole packet (L2, L3, and L4), so it looks like
> TCP would never see a usable value.
> 
> Is this interpretation correct? There's only a handful of drivers
> providing CHECKSUM_COMPLETE can anyone who has access verify this?

AFAIK CHECKSUM_COMPLETE means the checksum is calculated over the L4 part of the frame but doesn't include the pseudoheader. It is completed by adding the pseudoheader.  I use it this way and it works.

By any chance are you asking this in the context of tunneling and how to get both the inner and outer checksums from the value the HW calculates?

> 
> Thanks,
> Tom

Re: Does CHECKSUM_COMPLETE work at all?

[Tom Herbert]

On Fri, Feb 21, 2014 at 6:36 PM, Dimitrios Michailidis <dm@chelsio.com> wrote:
> Tom Herbert wrote:
>> If a driver sets CHECKSUM_COMPLETE in a packet, I don't see anywhere
>> in either the Ethernet rcv path or ip_rcv where the skb->csum is being
>> pulled (no *pull_rcsum calls). I believe CHECKSUM_COMPLETE means the
>> checksum is over the whole packet (L2, L3, and L4), so it looks like
>> TCP would never see a usable value.
>>
>> Is this interpretation correct? There's only a handful of drivers
>> providing CHECKSUM_COMPLETE can anyone who has access verify this?
>
> AFAIK CHECKSUM_COMPLETE means the checksum is calculated over the L4 part of the frame but doesn't include the pseudoheader. It is completed by adding the pseudoheader.  I use it this way and it works.
>
>From skbuff.h:

"The device supplied checksum of the _whole_packet as seen by
netif_rx() and fills out in skb->csum. Meaning, the hardware doesn't
need to parse L3/L4 headers to implement this."

> By any chance are you asking this in the context of tunneling and how to get both the inner and outer checksums from the value the HW calculates?
>
Well..., I started trying to fix the UDP encapsulation code which we
agreed was broken for checksums. This also entails fixing the new UDP
GRO code to deal with checksums correctly when going through protocol
layers. But, just looking at CHECKSUM_COMPLETE it seems like it's
fundamentally broken even without encapsulation. Between GRO and HW
checksums, it seems like we have a pretty big mess! :-)

>>
>> Thanks,
>> Tom
>

Re: Does CHECKSUM_COMPLETE work at all?

[Daniel Borkmann]

On 02/22/2014 03:11 AM, Tom Herbert wrote:
> If a driver sets CHECKSUM_COMPLETE in a packet, I don't see anywhere
> in either the Ethernet rcv path or ip_rcv where the skb->csum is being
> pulled (no *pull_rcsum calls). I believe CHECKSUM_COMPLETE means the
> checksum is over the whole packet (L2, L3, and L4), so it looks like
> TCP would never see a usable value.

Funny you ask, we had recently similar question [1]. ;) So, yes, it
should be over the whole packet as seen by Linux, and is therefore not
usable as is by protocols like TCP.

  [1] http://thread.gmane.org/gmane.linux.network/292443/

Re: Does CHECKSUM_COMPLETE work at all?

[Daniel Borkmann]

On 02/22/2014 04:08 AM, Tom Herbert wrote:
...
> Well..., I started trying to fix the UDP encapsulation code which we
> agreed was broken for checksums. This also entails fixing the new UDP
> GRO code to deal with checksums correctly when going through protocol
> layers. But, just looking at CHECKSUM_COMPLETE it seems like it's
> fundamentally broken even without encapsulation. Between GRO and HW
> checksums, it seems like we have a pretty big mess! :-)

Seems so, e.g. d97c00a32198 ("vxlan: fix receive checksum handling")
had similar issues in that regard; maybe because there seems to be
different consensus among drivers on setting CHECKSUM_COMPLETE?

Re: Does CHECKSUM_COMPLETE work at all?

[Tom Herbert]

On Sat, Feb 22, 2014 at 4:42 AM, Daniel Borkmann <dborkman@redhat.com> wrote:
> On 02/22/2014 04:08 AM, Tom Herbert wrote:
> ...
>
>> Well..., I started trying to fix the UDP encapsulation code which we
>> agreed was broken for checksums. This also entails fixing the new UDP
>> GRO code to deal with checksums correctly when going through protocol
>> layers. But, just looking at CHECKSUM_COMPLETE it seems like it's
>> fundamentally broken even without encapsulation. Between GRO and HW
>> checksums, it seems like we have a pretty big mess! :-)
>
>
> Seems so, e.g. d97c00a32198 ("vxlan: fix receive checksum handling")
> had similar issues in that regard; maybe because there seems to be
> different consensus among drivers on setting CHECKSUM_COMPLETE?

There shouldn't be, the requirements are pretty clear in the
description of CHECKSUM_COMPLETE. However, the meaning of
CHECKSUM_UNNECESSARY is ambiguous. Checksum offload for simple TCP or
UDP packets is fairly straightforward, but as devices support other
protocols and combinations of encapsulations correctness becomes
suspect. The problem is that the host can't verify
CHECKSUM_UNNECESSARY, so if it's not right or there is not agreement
between host and device as to which checksums are covered, then
packets can received with bad checksums. This will be really hard to
debug! As the comments about CHECKSUM_UNNECESSARY say "It is a bad
option" :-)

I think the fix for CHECKSUM_COMPLETE is to hold the start and length
of the data pertaining skb->csum. This way we only need to pull the
checksum when actually verifying it (don't want to pull it at every
header pull). This also relaxes the requirement that the device
provides the checksum for the whole packet, it could instead provide
partial coverage (there's really no need to have Ethernet header
checksummed for instance). No where in the stack should we be changing
CHECKSUM_COMPLETE to CHECKSUM_UNNECESSARY (that is a demotion), also
we should never be folding pseudo header checksum into skb->csum.