Re: [PATCH net-next 1/4] mlx5: Make building eswitch configurable

From: Mohamad Haj Yahia <mohamadhajyahia.mellanox@gmail.com>
To: Alexei Starovoitov <ast@fb.com>
Cc: Saeed Mahameed <saeedm@dev.mellanox.co.il>,
	Tom Herbert <tom@herbertland.com>,
	Or Gerlitz <gerlitz.or@gmail.com>,
	Saeed Mahameed <saeedm@mellanox.com>,
	David Miller <davem@davemloft.net>,
	Linux Netdev List <netdev@vger.kernel.org>,
	Kernel Team <kernel-team@fb.com>
Subject: Re: [PATCH net-next 1/4] mlx5: Make building eswitch configurable
Date: Tue, 31 Jan 2017 16:44:14 +0200	[thread overview]
Message-ID: <CANDJFw7funFkqzvLyZuZP8V8tXeXXvvKvc8gB1=NyStnVCdPzw@mail.gmail.com> (raw)
In-Reply-To: <58900535.2050709@fb.com>

>> Is it multi host configuration or bare metal ?
>
>
> multihost
>
>> Do you have internal loopback traffic between different hosts ?
>
>
> in a multihost? how can I check that?
> Is there an ethtool command?
>

You can check that by sending traffic from one PF to another on the same port.

>>> broken firmware or expected behavior?
>>
>>
>> which driver did you test ? backported or net-next ?
>
>
> both backported and net-next with Tom's patches.
>
>> if it is backported driver please verify that on driver load the
>> following occurs :
>>
>> 1. VPORTS change events are globally enabled:
>> in mlx5_start_eqs@eq.c:
>> async_event_mask |= (1ull << MLX5_EVENT_TYPE_NIC_VPORT_CHANGE);
>
>
> this one is done.
>
>> 2. UC address change events are enabled for vport 0 (PF):
>> In eswitch_attach or on eswitch_init (depends on the kernel version)
>> @eswitch.c
>> esw_enable_vport(esw, 0, UC_ADDR_CHANGE); is called.
>
>
> this one is not. Tom's proposal to compile out eswitch.c
> removes invocation of mlx5_eswitch_attach() and
> corresponding esw_enable_vport() call as well.
> The question is why is it necessary?
> What will break if it's not done?
> so far we don't see any adverse effects in both multihost
> and baremetal setups.
>
Please pay attention that if you compile out the eswitch.c we will not
program the l2 table.
However, the FW populate the L2 table with default entries after
init_hca command (at the driver load stage).
For each PF, By default its permanent mac pointing to it in the l2 table.
This can explain why its working for you without setting the l2 entries.

Now, if you try to set another mac for that PF or try to reach mac
addresses that are not the permantent address (virtual mac addresses
over the PF) it will fail.
So for this reason its crucial that the PF program the l2 table (which
needs the UC_ADDR_CHANGE set in the PF vport context).

>> BTW folks, i am going to be on vacation for the rest of the week, so
>> please expect slow responses.
>
>
> have a great time off. I hope other mlx folks can answer.
>