How to limit TCP packet lengths given to TC egress EBPF programs?

["Fingerhut, John Andy" <john.andy.fingerhut@intel.com>]

Greetings:

I am working on a project that runs an EBPF program on the Linux
Traffic Control egress hook, which modifies selected packets to add
headers to them that we use for some network telemetry.

I know that this is _not_ what one wants to do to get maximum TCP
performance, but at least for development purposes I was hoping to
find a way to limit the length of all TCP packets that are processed
by this EBPF program to be at most one MTU.

Towards that goal, we have tried several things, but regardless of
which subset of the following things we have tried, there are some
packets processed by our EBPF program that have IPv4 Total Length
field that is some multiple of the MSS size, sometimes nearly 64
KBytes.  If it makes a difference in configuration options available,
we have primarily been testing with Ubuntu 20.04 Linux running the
Linux kernel versions near 5.8.0-50-generic distributed by Canonical.

Disable TSO and GSO on the network interface:

    ethtool -K enp0s8 tso off gso off

Configuring TCP MSS using 'ip route' command:

    ip route change 10.0.3.0/24 dev enp0s8 advmss 1424

The last command _does_ have some effect, in that many packets
processed by our EBPF program have a length affected by that advmss
value, but we still see many packets that are about twice as large,
about three times as large, etc., which fit into that MSS after being
segmented, I believe in the kernel GSO code.

Is there some other configuration option we can change that can
guarantee that when a TCP packet is given to a TC egress EBPF program,
it will always be at most a specified length?


Background:

Intel is developing and releasing some open source EBPF programs and
associated user space programs that modify packets to add INT (Inband
Network Telemetry) headers, which can be used for some kinds of
performance debugging reasons, e.g. triggering events when packet
losses are detected, or significant changes in one-way packet latency
between two hosts configured to run this Host INT code.  See the
project home page for more details if you are interested:

https://github.com/intel/host-int

Note: The code published now is an alpha release.  We know there are
bugs.  We know our development team is not what you would call EBPF
experts (at least not yet), so feel free to point out bugs and/or
anything that code is doing that might be a bad idea.

Thanks,
Andy Fingerhut
Principal Engineer
Intel Corporation