Re: [PATCH net-next 2/8] devlink: add PF and VF port flavours

From: David Ahern <dsahern@gmail.com>
To: Jakub Kicinski <jakub.kicinski@netronome.com>,
	Jiri Pirko <jiri@resnulli.us>
Cc: davem@davemloft.net, oss-drivers@netronome.com, netdev@vger.kernel.org
Subject: Re: [PATCH net-next 2/8] devlink: add PF and VF port flavours
Date: Thu, 28 Feb 2019 09:24:33 -0700	[thread overview]
Message-ID: <0671e26e-2bb5-5360-e062-70abfaf1faf7@gmail.com> (raw)
In-Reply-To: <20190227144239.55481885@cakuba.netronome.com>

On 2/27/19 3:42 PM, Jakub Kicinski wrote:
> On Wed, 27 Feb 2019 21:17:27 +0100, Jiri Pirko wrote:
>> Wed, Feb 27, 2019 at 06:23:26PM CET, jakub.kicinski@netronome.com wrote:
>>> On Wed, 27 Feb 2019 13:41:35 +0100, Jiri Pirko wrote:  
>>>> Wed, Feb 27, 2019 at 01:23:27PM CET, jiri@resnulli.us wrote:  
>>>>> Tue, Feb 26, 2019 at 07:24:30PM CET, jakub.kicinski@netronome.com wrote:    
>>>>>> Current port flavours cover simple switches and DSA.  Add PF
>>>>>> and VF flavours to cover "switchdev" SR-IOV NICs.
>>>>>>
>>>>>> Example devlink user space output:
>>>>>>
>>>>>> $ devlink port
>>>>>> pci/0000:82:00.0/0: type eth netdev p4p1 flavour physical
>>>>>> pci/0000:82:00.0/10000: type eth netdev eth0 flavour pcie_pf pf 0
>>>>>> pci/0000:82:00.0/10001: type eth netdev eth1 flavour pcie_vf pf 0 vf 0
>>>>>> pci/0000:82:00.0/10002: type eth netdev eth2 flavour pcie_vf pf 0 vf 1    
>>>>>
>>>>> Wait a second, howcome pf and vfs have the same PCI address?    
>>>>
>>>> Oh, I think you have these as eswitch port representors. Confusing...  
>>>
>>> FWIW I don't like the word representor, its a port. We don't call
>>> physical ports "representors" even though from ASIC's point of view
>>> they are exactly the same.  
>>
>> My point is, they are not PFs and VFs. We have to find a way to clearly
>> see what's what.
> 
> Okay, so let me explain the way I see it, and you can explain your way
> or tell me where you disagree.  Those devlink ports and netdevs are pf
> ports and vf ports, which most refer to as "representor".  If one sends
> packets to the netdev indicated in DEVLINK_ATTR_PORT_NETDEV_*
> attributes they will _egress_ the switch from that port.  For physical
> port that means going onto the Ethernet or IB wire.  For PCIe it means
> getting DMAed over the PCIe link to host memory.
> 
> There is a netdev construct on the host which is in charge of that 
> host memory.  Maybe we shall call that host netdev?
> 
> (I said I don't like "representor" for the reason that people don't
> refer to the physical port as "representor" even though it has exactly
> the semantics we are following.  This distinction between behaviour of
> physical and PCI ports is what leads to confusion, I think.)
> 
> Let me bring out the moose :)
> 
>                     HOST A             ||          HOST B                  
>                                        ||                                  
>         PF A       | V | V | V | V     ||       PF B        | V | V | V               
>                    |*F |*F |*F |*F ... ||                   |*F |*F |*F ...           
> *port A0 |*port A1 | 0 | 1 | 2 | 3     ||*port B0 |*port B1 | 0 | 1 | 2            
>                                        ||                                  
>              PCI Express link          ||        PCI Express link          
>         \      \      \  |   |   |          |       |      /   /   /         
>          \      \      \ |   |   |          |       |     /   /   /          
>       /\  \______\______\'___|___|__________|_______'____/___/___/__    /\
>       ||  |+PF0s0|+PF0s1 |+VF0|+VF1| ...|   |+PF1s0|+PF1s1|+VF0|+VF1|   ||  
>   i   ||  |------ ------ ----- ---- ----|--- ------ ------ ---- ----|   ||   i
> d n H ||  |               <<==========                              |   || d n H
> e s O ||  |                            ==========>>                 |   || e s O
> v t S ||  |                    SR-IOV e-switch                      |   || v t S
> l a T ||  |               <<==========                              |   || l a T
> i n   ||  |                            ==========>>                 |   || i n
> n c A ||  |               ________ _________ ________               |   || n c B
> k e   ||  |              |+Phys 0 |+Phys 1  |+Phys 2 |              |   || k e
>       ||  \---------------------------------------------------------/   ||                 
>       \/                      |        |         |                      \/    
>                               |        |         |                          
>                                  ||         ||                               
>                           MAC 0  ||  MAC 1  || MAC 2                   
>                                  ||         ||                              
> 
> Things marked with + are devlink ports and have port (-repr-) netdevs
> (including physical ports).
> Things marked with * are host netdevs, don't have devlink ports.
> 

That would a good update to the commit message or cover letter.