Re: [net-next V2 01/17] net/mlx5: E-Switch, Refactor setting source port

From: Vlad Buslov <vladbu@nvidia.com>
To: Marcelo Ricardo Leitner <marcelo.leitner@gmail.com>
Cc: Or Gerlitz <gerlitz.or@gmail.com>,
	Jakub Kicinski <kuba@kernel.org>,
	"Saeed Mahameed" <saeed@kernel.org>,
	"David S. Miller" <davem@davemloft.net>,
	"Linux Netdev List" <netdev@vger.kernel.org>,
	Mark Bloch <mbloch@nvidia.com>,
	"Saeed Mahameed" <saeedm@nvidia.com>
Subject: Re: [net-next V2 01/17] net/mlx5: E-Switch, Refactor setting source port
Date: Wed, 10 Feb 2021 18:44:42 +0200	[thread overview]
Message-ID: <ygnh1rdnhnyd.fsf@nvidia.com> (raw)
In-Reply-To: <20210210135605.GD2859@horizon.localdomain>

On Wed 10 Feb 2021 at 15:56, Marcelo Ricardo Leitner <marcelo.leitner@gmail.com> wrote:
> On Tue, Feb 09, 2021 at 06:10:59PM +0200, Or Gerlitz wrote:
>> On Tue, Feb 9, 2021 at 4:26 PM Vlad Buslov <vladbu@nvidia.com> wrote:
>> > On Mon 08 Feb 2021 at 22:22, Jakub Kicinski <kuba@kernel.org> wrote:
>> > > On Mon, 8 Feb 2021 10:21:21 +0200 Vlad Buslov wrote:
>> 
>> > >> > These operations imply that 7.7.7.5 is configured on some interface on
>> > >> > the host. Most likely the VF representor itself, as that aids with ARP
>> > >> > resolution. Is that so?
>> 
>> > >> The tunnel endpoint IP address is configured on VF that is represented
>> > >> by enp8s0f0_0 representor in example rules. The VF is on host.
>> 
>> > > This is very confusing, are you saying that the 7.7.7.5 is configured
>> > > both on VF and VFrep? Could you provide a full picture of the config
>> > > with IP addresses and routing?
>> 
>> > No, tunnel IP is configured on VF. That particular VF is in host [..]
>> 
>> What's the motivation for that? isn't that introducing 3x slow down?
>
> Vlad please correct me if I'm wrong.
>
> I think this boils down to not using the uplink representor as a real
> interface. This way, the host can make use of 7.7.7.5 for other stuff
> as well without passing (heavy) traffic through representor ports,
> which are not meant for it.
>
> So the host can have the IP 7.7.7.5 and also decapsulate vxlan traffic
> on it, which wouldn't be possible/recommended otherwise.
>
> Another moment that this gets visible is with VF LAG. When we bond the
> uplink representors, add an IP to it and do vxlan decap, that IP is
> meant only for the decap process and shouldn't be used for heavier
> traffic as its passing through representor ports.
>
> Then, tc config for decap need to be done on VF0rep and not on VF0
> itself because that would be a security problem: one VF (which could
> be on a netns) could steer packets to another VF at will.

While on-host VF (the one with IP 7.7.7.5 in my examples) is intended to
be used for unencapsulated control traffic as well, we don't expect
significant bandwidth of such traffic, so traffic-load on representor
wasn't the main motivation. I didn't want to go into the details in
cover letter because they are mostly OVS-specific and this series is a
groundwork for features to come.

So the main motivation is to be able to apply policy on both on underlay
network (UL) and overlay network (tunnel netdev). As that will allow us
to subject overlay and underlay traffic to different set of OVS rules,
for example underlay traffic may be subject to vlan encap/decap,
security policy or any other flow rule that the user may define.

Hope this also answers some of Or's questions from this thread.