Re: AF_XDP metadata/hints - Toke Høiland-Jørgensen

From: "Toke Høiland-Jørgensen" <toke@redhat.com>
To: Jesper Dangaard Brouer <brouer@redhat.com>,
	John Fastabend <john.fastabend@gmail.com>
Cc: Alexander Lobakin <alexandr.lobakin@intel.com>,
	Saeed Mahameed <saeed@kernel.org>,
	"Raczynski, Piotr" <piotr.raczynski@intel.com>,
	"Zhang, Jessica" <jessica.zhang@intel.com>,
	"Kubiak, Marcin" <marcin.kubiak@intel.com>,
	"Joseph, Jithu" <jithu.joseph@intel.com>,
	"kurt@linutronix.de" <kurt@linutronix.de>,
	"Maloor, Kishen" <kishen.maloor@intel.com>,
	"Gomes, Vinicius" <vinicius.gomes@intel.com>,
	"Brandeburg, Jesse" <jesse.brandeburg@intel.com>,
	"Swiatkowski, Michal" <michal.swiatkowski@intel.com>,
	"Plantykow, Marta A" <marta.a.plantykow@intel.com>,
	"Ong, Boon Leong" <boon.leong.ong@intel.com>,
	"Desouza, Ederson" <ederson.desouza@intel.com>,
	"Song, Yoong Siang" <yoong.siang.song@intel.com>,
	"Czapnik, Lukasz" <lukasz.czapnik@intel.com>,
	bpf@vger.kernel.org, brouer@redhat.com
Subject: Re: AF_XDP metadata/hints
Date: Wed, 26 May 2021 15:06:55 +0200	[thread overview]
Message-ID: <87y2c1iqz4.fsf@toke.dk> (raw)
In-Reply-To: <20210526134910.1c06c5d8@carbon>

Jesper Dangaard Brouer <brouer@redhat.com> writes:

> On Tue, 25 May 2021 21:51:22 -0700
> John Fastabend <john.fastabend@gmail.com> wrote:
>
>> Separate the config of hardware from the BPF infrastructure these
>> are two separate things.
>
> I fully agree.

+1. Another reason why is the case of multiple XDP programs on a single
interface: When attaching these (using freplace as libxdp does it), the
kernel can just check the dest interface when verifying the freplace
program and any rewriting of the bytecode from the BTF format can happen
at that point. Whereas if the BPF attach needs to have side effects,
suddenly we have to copy over all the features to the dispatcher program
and do some kind of set union operation; and what happens if an freplace
program is attached after the fact (same thing with tail calls)?

So in my mind there's no doubt this needs to be:

driver is config'ed -> it changes its exposed BTF metadata -> program is
attached -> verifier rewrites program to access metadata correctly

> How should we handle existing config interfaces?
>
> Let me give some concrete examples. Today there are multiple existing
> interfaces to enable/disable NIC hardware features that change what is
> available to put in our BTF-layout.
>
> E.g. changing if VLAN is in descriptor:
>  # ethtool -K ixgbe1 rx-vlan-offload off
>  # ethtool -k ixgbe1 | grep vlan-offload
>  rx-vlan-offload: off
>  tx-vlan-offload: on
>
> The timestamping features can be listed by ethtool -T (see below
> signature), but it is a socket option that enable[1] these
> (see SO_TIMESTAMPNS or SOF_TIMESTAMPING_RX_HARDWARE).
>
> Or tuning RSS hash fields:
>  [2] https://github.com/stackpath/rxtxcpu/blob/master/Documentation/case-studies/observing-rss-on-ixgbe-advanced-rss-configuration-rss-hash-fields.md
>
> I assume we need to stay compatible and respect the existing config
> interfaces, right?
>
> Should we simple leverage existing interfaces?

Now that ethtool has moved to netlink it should be quite
straight-forward to add a separate subset of commands for configuring
metadata fields; and internally the kernel can map those to the existing
config knobs, no?

E.g., if you tell the kernel you'd like to have the VLAN field as a
metadata field that kinda implies that rx-vlan-offload should be turned
on; etc. Any reason this would break down?

-Toke