From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS, USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5748C43381 for ; Thu, 28 Feb 2019 08:54:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5B82D2171F for ; Thu, 28 Feb 2019 08:54:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=resnulli-us.20150623.gappssmtp.com header.i=@resnulli-us.20150623.gappssmtp.com header.b="BemU601C" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731926AbfB1Iy0 (ORCPT ); Thu, 28 Feb 2019 03:54:26 -0500 Received: from mail-wm1-f66.google.com ([209.85.128.66]:39139 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726038AbfB1Iy0 (ORCPT ); Thu, 28 Feb 2019 03:54:26 -0500 Received: by mail-wm1-f66.google.com with SMTP id z84so8065406wmg.4 for ; Thu, 28 Feb 2019 00:54:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=resnulli-us.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=bCpJ1JOqMr1h5eEGoi0uxhU6VYoQgLX/1OGA346JJ2E=; b=BemU601CBTfPkMRKWDaWukbmbBGWV0NU63pjcT57aATeXaLEQVh204FlLXR3EM1f1H GVlRX8LZshI28U/iv9Ejy8u/HTKMwuw5cHr9iAka6FpCZboZK4Gj6ms/OJMfUXJthwfr zlesetrWNlm7M8nBeYM/WfZ6NlI5pQ+uncwtXhSOLL/JR3DwwihgX1l/9/zrgm3Kyh/M hROts7HsbcbatcIDAimV/TCbUi7xoIZjh5V4q035UgzbaQXes8yye6p2V0e9Oc0HiUC/ 0C8FX7QQTjiO0i1AaXDZumwhR9B8OMcqgRxGJJKHOwURN9GS0sT7fBdNSteox9FwQL9F IZgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=bCpJ1JOqMr1h5eEGoi0uxhU6VYoQgLX/1OGA346JJ2E=; b=NqfI3ceG2nxxa78tJKQUUHnnUuz+lOrVkyidYxjEWNBQmOr1RZzjApCOuV1z7gdglH iZSvORGdAQM6N7nfEY9S63y5UizRvFH7OAMTcuGhotEj5njCIA6jBNX6bebS9uU7i8oK pi/YCG5XfT/UeF7ZFWy5Pmp1bGPNYh1nlurnlJYkiz2n0fCH3Y+2OYZqbWZHfcKOrL+G fmXJnRICymmm7nz/kVyl5ScV4pQnpcKTLdyL/10mqYaFcLEpo+eiKeguET1Op/Ublf3n 0xfdq9wSQwQFQqhBGzFn+6dLFZA98WoF9xMKee3U1lidEADtSAoSBQddSt1ks/CtxKeW +bWA== X-Gm-Message-State: APjAAAV+EnvYT6+loScqBXPBqMmcsih3sb4zRNmvyAGEZGpwtqRCJUxo hRplGF7nyEc6kZv2jNhEGYfjWg== X-Google-Smtp-Source: AHgI3IYYYOiTEZJBKo339cKmqECN1x4byHHj2dsow9TbH21TsZdiExdf9NeMrAEzbwrbxgBXjsLWcw== X-Received: by 2002:a1c:16:: with SMTP id 22mr1956565wma.91.1551344063736; Thu, 28 Feb 2019 00:54:23 -0800 (PST) Received: from localhost (jirka.pirko.cz. [84.16.102.26]) by smtp.gmail.com with ESMTPSA id u12sm4362743wmf.44.2019.02.28.00.54.23 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 28 Feb 2019 00:54:23 -0800 (PST) Date: Thu, 28 Feb 2019 09:44:29 +0100 From: Jiri Pirko To: Jakub Kicinski Cc: davem@davemloft.net, oss-drivers@netronome.com, netdev@vger.kernel.org Subject: Re: [PATCH net-next 2/8] devlink: add PF and VF port flavours Message-ID: <20190228084429.GC2324@nanopsycho.orion> References: <20190226182436.23811-1-jakub.kicinski@netronome.com> <20190226182436.23811-3-jakub.kicinski@netronome.com> <20190227122327.GA10571@nanopsycho> <20190227124135.GB10571@nanopsycho> <20190227092326.5abf417b@cakuba.netronome.com> <20190227201727.GA2042@nanopsycho> <20190227144239.55481885@cakuba.netronome.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190227144239.55481885@cakuba.netronome.com> User-Agent: Mutt/1.10.1 (2018-07-13) Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Wed, Feb 27, 2019 at 11:42:39PM CET, jakub.kicinski@netronome.com wrote: >On Wed, 27 Feb 2019 21:17:27 +0100, Jiri Pirko wrote: >> Wed, Feb 27, 2019 at 06:23:26PM CET, jakub.kicinski@netronome.com wrote: >> >On Wed, 27 Feb 2019 13:41:35 +0100, Jiri Pirko wrote: >> >> Wed, Feb 27, 2019 at 01:23:27PM CET, jiri@resnulli.us wrote: >> >> >Tue, Feb 26, 2019 at 07:24:30PM CET, jakub.kicinski@netronome.com wrote: >> >> >>Current port flavours cover simple switches and DSA. Add PF >> >> >>and VF flavours to cover "switchdev" SR-IOV NICs. >> >> >> >> >> >>Example devlink user space output: >> >> >> >> >> >>$ devlink port >> >> >>pci/0000:82:00.0/0: type eth netdev p4p1 flavour physical >> >> >>pci/0000:82:00.0/10000: type eth netdev eth0 flavour pcie_pf pf 0 >> >> >>pci/0000:82:00.0/10001: type eth netdev eth1 flavour pcie_vf pf 0 vf 0 >> >> >>pci/0000:82:00.0/10002: type eth netdev eth2 flavour pcie_vf pf 0 vf 1 >> >> > >> >> >Wait a second, howcome pf and vfs have the same PCI address? >> >> >> >> Oh, I think you have these as eswitch port representors. Confusing... >> > >> >FWIW I don't like the word representor, its a port. We don't call >> >physical ports "representors" even though from ASIC's point of view >> >they are exactly the same. >> >> My point is, they are not PFs and VFs. We have to find a way to clearly >> see what's what. > >Okay, so let me explain the way I see it, and you can explain your way >or tell me where you disagree. Those devlink ports and netdevs are pf >ports and vf ports, which most refer to as "representor". If one sends >packets to the netdev indicated in DEVLINK_ATTR_PORT_NETDEV_* >attributes they will _egress_ the switch from that port. For physical >port that means going onto the Ethernet or IB wire. For PCIe it means >getting DMAed over the PCIe link to host memory. > >There is a netdev construct on the host which is in charge of that >host memory. Maybe we shall call that host netdev? > >(I said I don't like "representor" for the reason that people don't >refer to the physical port as "representor" even though it has exactly >the semantics we are following. This distinction between behaviour of >physical and PCI ports is what leads to confusion, I think.) > >Let me bring out the moose :) > > HOST A || HOST B > || > PF A | V | V | V | V || PF B | V | V | V > |*F |*F |*F |*F ... || |*F |*F |*F ... >*port A0 |*port A1 | 0 | 1 | 2 | 3 ||*port B0 |*port B1 | 0 | 1 | 2 > || > PCI Express link || PCI Express link > \ \ \ | | | | | / / / > \ \ \ | | | | | / / / > /\ \______\______\'___|___|__________|_______'____/___/___/__ /\ > || |+PF0s0|+PF0s1 |+VF0|+VF1| ...| |+PF1s0|+PF1s1|+VF0|+VF1| || > i || |------ ------ ----- ---- ----|--- ------ ------ ---- ----| || i >d n H || | <<========== | || d n H >e s O || | ==========>> | || e s O >v t S || | SR-IOV e-switch | || v t S >l a T || | <<========== | || l a T >i n || | ==========>> | || i n >n c A || | ________ _________ ________ | || n c B >k e || | |+Phys 0 |+Phys 1 |+Phys 2 | | || k e > || \---------------------------------------------------------/ || > \/ | | | \/ > | | | > || || > MAC 0 || MAC 1 || MAC 2 > || || > >Things marked with + are devlink ports and have port (-repr-) netdevs >(including physical ports). >Things marked with * are host netdevs, don't have devlink ports. Okay, I got it. So you say that devlink ports should always be only ports of eswitch. PF host netdev should have "devlink port" instance, correct? But it still "belongs" under the ASIC represented by the devlink instance...