Re: [PATCH RFC net-next 0/3] Multi-CPU DSA support

[Marek Behun <marek.behun@nic.cz>]

On Mon, 12 Apr 2021 23:22:45 +0200
Tobias Waldekranz <tobias@waldekranz.com> wrote:

> On Mon, Apr 12, 2021 at 21:30, Marek Behun <marek.behun@nic.cz> wrote:
> > On Mon, 12 Apr 2021 14:46:11 +0200
> > Tobias Waldekranz <tobias@waldekranz.com> wrote:
> >  
> >> I agree. Unless you only have a few really wideband flows, a LAG will
> >> typically do a great job with balancing. This will happen without the
> >> user having to do any configuration at all. It would also perform well
> >> in "router-on-a-stick"-setups where the incoming and outgoing port is
> >> the same.  
> >
> > TLDR: The problem with LAGs how they are currently implemented is that
> > for Turris Omnia, basically in 1/16 of configurations the traffic would
> > go via one CPU port anyway.
> >
> >
> >
> > One potencial problem that I see with using LAGs for aggregating CPU
> > ports on mv88e6xxx is how these switches determine the port for a
> > packet: only the src and dst MAC address is used for the hash that
> > chooses the port.
> >
> > The most common scenario for Turris Omnia, for example, where we have 2
> > CPU ports and 5 user ports, is that into these 5 user ports the user
> > plugs 5 simple devices (no switches, so only one peer MAC address for
> > port). So we have only 5 pairs of src + dst MAC addresses. If we simply
> > fill the LAG table as it is done now, then there is 2 * 0.5^5 = 1/16
> > chance that all packets would go through one CPU port.
> >
> > In order to have real load balancing in this scenario, we would either
> > have to recompute the LAG mask table depending on the MAC addresses, or
> > rewrite the LAG mask table somewhat randomly periodically. (This could
> > be in theory offloaded onto the Z80 internal CPU for some of the
> > switches of the mv88e6xxx family, but not for Omnia.)  
> 
> I thought that the option to associate each port netdev with a DSA
> master would only be used on transmit. Are you saying that there is a
> way to configure an mv88e6xxx chip to steer packets to different CPU
> ports depending on the incoming port?
>
> The reason that the traffic is directed towards the CPU is that some
> kind of entry in the ATU says so, and the destination of that entry will
> either be a port vector or a LAG. Of those two, only the LAG will offer
> any kind of balancing. What am I missing?

Via port vectors you can "load balance" by ports only, i.e. input port X
-> trasmit via CPU port Y.

When using LAGs, you are load balancing via hash(src MAC | dst mac)
only. This is better in some ways. But what I am saying is that if the
LAG mask table is static, as it is now implemented in mv88e6xxx code,
then for many scenarios there is a big probability of no load balancing
at all. For Turris Omnia for example there is 6.25% probability that
the switch chip will send all traffic to the CPU via one CPU port.
This is because the switch chooses the LAG port only from the hash of
dst+src MAC address. (By the 1/16 = 6.25% probability I mean that for
cca 1 in 16 customers, the switch would only use one port when sending
data to the CPU).

The round robin solution here is therefore better in this case.