From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37F6CC43381 for ; Fri, 15 Mar 2019 02:15:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0947D21872 for ; Fri, 15 Mar 2019 02:15:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727383AbfCOCPx (ORCPT ); Thu, 14 Mar 2019 22:15:53 -0400 Received: from mga14.intel.com ([192.55.52.115]:65227 "EHLO mga14.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727284AbfCOCPx (ORCPT ); Thu, 14 Mar 2019 22:15:53 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 14 Mar 2019 19:15:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,480,1544515200"; d="scan'208";a="131785287" Received: from samudral-mobl1.amr.corp.intel.com (HELO [10.251.7.207]) ([10.251.7.207]) by fmsmga008.fm.intel.com with ESMTP; 14 Mar 2019 19:15:52 -0700 Subject: Re: [PATCH net-next v2 4/7] devlink: allow subports on devlink PCI ports To: Parav Pandit , Jakub Kicinski Cc: Jiri Pirko , "davem@davemloft.net" , "netdev@vger.kernel.org" , "oss-drivers@netronome.com" References: <20190308145421.GA2888@nanopsycho.orion> <20190308110943.2ee42bc0@cakuba.hsd1.ca.comcast.net> <20190311085204.GA2194@nanopsycho> <20190311191054.36b801d6@cakuba.hsd1.ca.comcast.net> <20190312140239.GA2455@nanopsycho> <20190312135628.5250135b@cakuba.hsd1.ca.comcast.net> <20190313060701.GB2384@nanopsycho.orion> <20190313091731.76129ece@cakuba.attlocal.net> <20190313162243.GB2270@nanopsycho> <20190313095555.0f4f92ca@cakuba.attlocal.net> <20190314073840.GA3034@nanopsycho> <20190314150945.031d1b08@cakuba.netronome.com> <20190314163915.24fd2481@cakuba.netronome.com> From: "Samudrala, Sridhar" Message-ID: <4436da3d-4b99-f792-8e77-695d5958794d@intel.com> Date: Thu, 14 Mar 2019 19:15:50 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.3 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On 3/14/2019 6:28 PM, Parav Pandit wrote: > > >> -----Original Message----- >> From: Jakub Kicinski >> Sent: Thursday, March 14, 2019 6:39 PM >> To: Parav Pandit >> Cc: Jiri Pirko ; davem@davemloft.net; >> netdev@vger.kernel.org; oss-drivers@netronome.com >> Subject: Re: [PATCH net-next v2 4/7] devlink: allow subports on devlink PCI >> ports >> >> On Thu, 14 Mar 2019 22:35:36 +0000, Parav Pandit wrote: >>>>> Then instances of flavour pci_vf are going to appear in the same >>>>> devlink instance. Those are the switch ports: >>>>> pci/0000:05:00.0/10002: type eth netdev enp5s0npf0pf0s0 >>>>> flavour pci_vf pf 0 vf 0 >>>>> switch_id 00154d130d2f peer >>>>> pci/0000:05:10.1/0 >>>>> pci/0000:05:00.0/10003: type eth netdev enp5s0npf0pf0s0 >>>>> flavour pci_vf pf 0 vf 0 subport 1 >>>>> switch_id 00154d130d2f peer >>>>> pci/0000:05:10.1/1 >>>>> >>>>> With that, peers are going to appear too, and those are the actual >>>>> VF/VF >>>>> subport: >>>>> pci/0000:05:10.1/0: type eth netdev ??? flavour pci_vf_host >>>>> peer pci/0000:05:00.0/10002 >>>>> pci/0000:05:10.1/1: type eth netdev ??? flavour pci_vf_host >>>>> peer pci/0000:05:00.0/10003 >>>>> >>>>> Later you can push this VF along with all subports to VM. So in >>>>> VM, you are going to see the VF like this: >>>>> $ devlink dev >>>>> pci/0000:00:08.0 >>>>> $ devlink port >>>>> pci/0000:00:08.0/0: type eth netdev ??? flavour pci_vf_host >>>>> pci/0000:00:08.0/1: type eth netdev ??? flavour pci_vf_host >>>>> >>>>> And back to your question of how are they connected in eswitch. >>>>> That is totally up to the original user John who did the creation. >>>>> He is in charge of the eswitch on baremetal, he would configure >>>>> the forwarding however he likes. >>>> >>>> Ack, so I think you're saying VM has to communicate to the cloud >>>> environment to have this provisioned using some service API, not a >>>> kernel API. That's what I wanted to confirm. >>>> >>>> I don't see any benefit to having the "host ports" under devlink, as such I >>>> think it's a matter of preference. >>> >>> We need 'host ports' to configure parameters of this host port which >>> is not exposed by the rep-netdev. >>> Such as mac address. >> >> Please look at the quote of what Jiri wrote above - the host port gets passed >> to the VM, you can't use it as a handle to set the MAC. >> >> The way to set the MAC remains: >> >> # devlink port set pci/0000:05:00.0/10002 peer mac_addr 00:11:22:33:44:55 >> > Even though it can be done, I think this is wrong model to program hostport mac address using eswitch port. > All devlink objects are control objects, so what is passed to VM is what is represented by devlink. > VF in the VM will anyway create its devlink object. > What is wrong in programming hostport? > It gives a very clear view to users of topology and objects. The VF or any subport MAC address should be configured by the orchestration layer that is running on the hypervisor and when a VF is assigned to a VF, the host port is not visible to the hypervisor. Currently we have ndo_set_vf_mac_addr api that works with PF netdev, but i think we are trying to move away from that API and do all the configuration via the port representor netdevs. As the mac address cannot be configured using this netdev, i think Jakub is suggesting creating a devlink opject for each port representor and use that interface to set peer mac address. We should be able use this to configure port vlan too. Also, instead of subport, can we call vport and support different types of vports - sr-iov, siov, vmdq etc. > > Also eswitch is flat. There is no need of pf/vf flavour for port. > It doesn't make sense to define 'mdev' flavour which we are already working. > At eswitch level it is just a port, it happen to be connected to vf or pf or other objects, it doesn't matter. > Port should be flavoured as 'hostport' or 'switchport'. > > >> (using the port ids from above)