Much higher CPU usage when generating UDP vs. TCP traffic

Much higher CPU usage when generating UDP vs. TCP traffic

[Ignacy Gawędzki]

Hi,

While running some network performance tests, I discovered that the
CPU usage when generating UDP traffic is much higher than when
generating TCP traffic.  Note that no significant difference of CPU
usage was observed when simply forwarding UDP vs. TCP traffic.

This happens on Ethernet, on Wi-Fi, even on veth links (but not on the
loopback interface), so it seems unrelated to any specific hardware
driver.  BTW this difference in CPU usage decreases quite notably when
generating UDP paquets of maximum size, resulting in 64 KiB IP paquest
getting immediately fragmented to fit a standard MTU of 1500 bytes.

On platforms with more modest CPU resources, such as SoCs, this
results in much lower maximum achievable throughput in UDP vs. TCP.
Even more so with some Wi-Fi 6 drivers that seem to eat a significant
portion of CPU on their own.

I observed this happening on older kernels as well as the head of
wireless.git.  Disabling any hardware offloading doesn't change much
if anything at all.

This was really unexpected to me, since I always assumed the
processing of UDP is much simpler than that of TCP.  So I eventually
resolved to write to this list, hoping that some knowledgeable person
could shed some light on the matter.

Many thanks in advance,

Ignacy

-- 
Ignacy Gawędzki
R&D Engineer
Green Communications

Re: Much higher CPU usage when generating UDP vs. TCP traffic

[Neal Cardwell]

On Mon, Jan 16, 2023 at 7:07 AM Ignacy Gawędzki
<ignacy.gawedzki@green-communications.fr> wrote:
>
> Hi,
>
> While running some network performance tests, I discovered that the
> CPU usage when generating UDP traffic is much higher than when
> generating TCP traffic.  Note that no significant difference of CPU
> usage was observed when simply forwarding UDP vs. TCP traffic.
>
> This happens on Ethernet, on Wi-Fi, even on veth links (but not on the
> loopback interface), so it seems unrelated to any specific hardware
> driver.  BTW this difference in CPU usage decreases quite notably when
> generating UDP paquets of maximum size, resulting in 64 KiB IP paquest
> getting immediately fragmented to fit a standard MTU of 1500 bytes.

Probably a large factor here is the TCP support for TSO/GSO.

There are probably mn
This article has a nice discussion of some of the main issues in UDP
vs TCP performance:
  https://www.fastly.com/blog/measuring-quic-vs-tcp-computational-efficiency

neal