Re: [PATCH net-next v2 4/7] devlink: allow subports on devlink PCI ports

From: Jiri Pirko <jiri@resnulli.us>
To: Jakub Kicinski <jakub.kicinski@netronome.com>
Cc: davem@davemloft.net, netdev@vger.kernel.org, oss-drivers@netronome.com
Subject: Re: [PATCH net-next v2 4/7] devlink: allow subports on devlink PCI ports
Date: Wed, 6 Mar 2019 13:20:37 +0100	[thread overview]
Message-ID: <20190306122037.GB2819@nanopsycho> (raw)
In-Reply-To: <20190305091534.36200de6@cakuba.hsd1.ca.comcast.net>

Tue, Mar 05, 2019 at 06:15:34PM CET, jakub.kicinski@netronome.com wrote:
>On Tue, 5 Mar 2019 12:06:01 +0100, Jiri Pirko wrote:
>> >> >as ports.  Can we invent a new command (say "partition"?) that'd take
>> >> >the bus info where the partition is to be spawned?    
>> >> 
>> >> Got it. But the question is how different this object would be from the
>> >> existing "port" we have today.  
>> >
>> >They'd be where "the other side of a PCI link" is represented,
>> >restricting ports to only ASIC's forwarding plane ports.  
>> 
>> Basically a "host port", right? It can still be the same port object,
>> only with different flavour and attributes. So we would have:
>> 
>> 1) pci/0000:05:00.0/0: type eth netdev enp5s0np0
>>                        flavour physical switch_id 00154d130d2f
>> 2) pci/0000:05:00.0/10000: type eth netdev enp5s0npf0s0
>>                            flavour pci_pf pf 0 subport 0
>>                            switch_id 00154d130d2f
>>                            peer pci/0000:05:00.0/1
>> 3) pci/0000:05:00.0/10001: type eth netdev enp5s0npf0vf0
>>                            flavour pci_vf pf 0 vf 0
>>                            switch_id 00154d130d2f
>>                            peer pci/0000:05:10.1/0
>> 4) pci/0000:05:00.0/10001: type eth netdev enp5s0npf0s1
>>                            flavour pci_pf pf 0 subport 1
>>                            switch_id 00154d130d2f
>>                            peer pci/0000:05:00.0/2
>> 5) pci/0000:05:00.0/1: type eth netdev enp5s0f0??
>>                        flavour host          <----------------
>>                        peer pci/0000:05:00.0/10000
>> 6) pci/0000:05:10.1/0: type eth netdev enp5s10f0 
>>                        flavour host          <----------------
>>                        peer pci/0000:05:00.0/10001
>> 7) pci/0000:05:00.0/2: type eth netdev enp5s0f0??
>>                        flavour host          <----------------
>>                        peer pci/0000:05:00.0/10001
>> 
>> I think it looks quite clear, it gives complete topology view.
>
>Okay, I have some of questions :)
>
>What do we use for port_index?

That is just a number totally in control of the driver. Driver can
assign it in any way.

>
>What are the operations one can perform on "host ports"?

That is a good question. I would start with *none* and extend it upon
needs.

>
>If we have PCI parameters, do they get set on the ASIC side of the port
>or the host side of the port?

Could you give me an example? But I believe that on switch-port side.

>
>How do those behave when device is passed to VM?

In case of VF? VF will have separate devlink instance (separate handle,
probably "aliased" to the PF handle). So it would disappear from
baremetal and appear in VM:
$ devlink dev
pci/0000:00:10.0
$ devlink dev port
pci/0000:00:10.1/0: type eth netdev enp5s10f0
                    flavour host
That's it for the VM.

There's no linkage (peer, alias) between this and the instances on
baremetal. 

>
>You have a VF devlink instance there - what ports does it show?

See above.

>
>How do those look when the PF is connected to another host?  Do they
>get spawned at all?

What do you mean by "PF is connected to another host"?

>
>Will this not be confusing to DSA folks who have a CPU port?

Why do you think so?