checksumming on non-local forward path

checksumming on non-local forward path

[Jason A. Donenfeld]

Hey netdev,

In a protocol like wireguard, if a packet arrives on the other end of
the tunnel, and the crypto checks out, then we're absolutely certain
that the packet hasn't been modified in transit. In this case, there's
no useful information that validating the inner checksums of the
various headers would give us. Every byte of the packet has arrived
intact, and we're certain of it.

Therefore, you might think in a situation like this, before calling
netif_receive_skb or the like, we can just set ip_summed to
CHECKSUM_UNNECESSARY, csum_level to ~0, and feel glad that we're not
wasting cycles unnecessarily validating the checksum.

But what if the receiving computer forwards the packet on across a
real physical network? In that case, it's probably important that the
kernel that originally produced the packet in the first place ensures
it has a valid checksum before encrypting it and sending it through
the wireguard tunnel. That's fine, but it does mean that in the case
of the packet being locally received on the other end, and not
forwarded, the checksumming by the original sender was not needed and
was therefore wasteful.

What would you think of a flag on the receiving end like,
"CHECKSUM_INVALID_BUT_UNNECESSARY"? It would be treated as
CHECKSUM_UNNECESSARY in the case that the the packet is locally
received. But if the packet is going to be forwarded instead, then
skb_checksum_help is called on it before forwarding onward.

AFAICT, wireguard isn't the only thing that could benefit from this:
virtio is another case where it's not always necessary for the sender
to call skb_checksum_help, when the receiver could just do it
conditionally based on whether it's being forwarded.

Thoughts?

Regards,
Jason

Re: checksumming on non-local forward path

[Andrew Lunn]

> What would you think of a flag on the receiving end like,
> "CHECKSUM_INVALID_BUT_UNNECESSARY"? It would be treated as
> CHECKSUM_UNNECESSARY in the case that the the packet is locally
> received. But if the packet is going to be forwarded instead, then
> skb_checksum_help is called on it before forwarding onward.
> 
> AFAICT, wireguard isn't the only thing that could benefit from this:
> virtio is another case where it's not always necessary for the sender
> to call skb_checksum_help, when the receiver could just do it
> conditionally based on whether it's being forwarded.

Hi Jason

It is the sort of thing which breaks in hard to find ways. I've run
network simulations with machine instances running in containers. It
used veth pairs to connect the instances to a central 'switching'
namespace which did the interconnect between the instances, using lots
of bridges. After a while, my simulation got bigger than a single host
could support. So i split it over multiple servers, using GRE tunnels
between the bridges. It took me a while to notice the network was
actually in two segments, because frames going over GRE were getting
tossed with checksum issues. It was not the GRE tunnel at fault. It
took a while to trace it back to where the checksumming was turned
off, a TAP interface i think, but i don't remember.

How do you reliably decide if a frame needs checksums, when you cannot
peer down the pipe of bridges, veth, GRE tunnels and TAP interfaces
the frame is about to take?

	   Andrew