From: Jakub Kicinski <kuba@kernel.org>
To: Saeed Mahameed <saeed@kernel.org>
Cc: "David S. Miller" <davem@davemloft.net>,
Jason Gunthorpe <jgg@nvidia.com>,
Leon Romanovsky <leonro@nvidia.com>,
netdev@vger.kernel.org, linux-rdma@vger.kernel.org,
David Ahern <dsahern@kernel.org>,
Jacob Keller <jacob.e.keller@intel.com>,
Sridhar Samudrala <sridhar.samudrala@intel.com>,
david.m.ertman@intel.com, dan.j.williams@intel.com,
kiran.patil@intel.com, gregkh@linuxfoundation.org,
Parav Pandit <parav@nvidia.com>, Jiri Pirko <jiri@nvidia.com>,
Saeed Mahameed <saeedm@nvidia.com>
Subject: Re: [net-next v5 13/15] devlink: Add devlink port documentation
Date: Tue, 15 Dec 2020 16:57:58 -0800 [thread overview]
Message-ID: <20201215165758.4ff58f85@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> (raw)
In-Reply-To: <20201215090358.240365-14-saeed@kernel.org>
On Tue, 15 Dec 2020 01:03:56 -0800 Saeed Mahameed wrote:
> +PCI controllers
> +---------------
> +In most cases a PCI device has only one controller. A controller consists of
> +potentially multiple physical and virtual functions. Such PCI function consists
> +of one or more ports.
s/Such//
you say consists in two consecutive sentences.
> This port of the function is represented by the devlink eswitch port.
"This port of the function"? Why not just "Each port"?
> +A PCI Device connected to multiple CPUs or multiple PCI root complexes or
Why is device capitalized all of the sudden?
> +SmartNIC, however, may have multiple controllers. For a device with multiple
a SmartNIC or SmartNICs
> +controllers, each controller is distinguished by a unique controller number.
> +An eswitch on the PCI device support ports of multiple controllers.
eswitch is on a PCI device?
> +An example view of a system with two controllers::
> +
> + ---------------------------------------------------------
> + | |
> + | --------- --------- ------- ------- |
> + ----------- | | vf(s) | | sf(s) | |vf(s)| |sf(s)| |
> + | server | | ------- ----/---- ---/----- ------- ---/--- ---/--- |
> + | pci rc |=== | pf0 |______/________/ | pf1 |___/_______/ |
> + | connect | | ------- ------- |
> + ----------- | | controller_num=1 (no eswitch) |
> + ------|--------------------------------------------------
> + (internal wire)
> + |
> + ---------------------------------------------------------
> + | devlink eswitch ports and reps |
> + | ----------------------------------------------------- |
> + | |ctrl-0 | ctrl-0 | ctrl-0 | ctrl-0 | ctrl-0 |ctrl-0 | |
> + | |pf0 | pf0vfN | pf0sfN | pf1 | pf1vfN |pf1sfN | |
> + | ----------------------------------------------------- |
> + | |ctrl-1 | ctrl-1 | ctrl-1 | ctrl-1 | ctrl-1 |ctrl-1 | |
> + | |pf0 | pf0vfN | pf0sfN | pf1 | pf1vfN |pf1sfN | |
> + | ----------------------------------------------------- |
> + | |
> + | |
> + ----------- | --------- --------- ------- ------- |
> + | smartNIC| | | vf(s) | | sf(s) | |vf(s)| |sf(s)| |
> + | pci rc |==| ------- ----/---- ---/----- ------- ---/--- ---/--- |
> + | connect | | | pf0 |______/________/ | pf1 |___/_______/ |
> + ----------- | ------- ------- |
> + | |
> + | local controller_num=0 (eswitch) |
> + ---------------------------------------------------------
> +
> +In above example, external controller (identified by controller number = 1)
> +doesn't have eswitch. Local controller (identified by controller number = 0)
> +has the eswitch. Devlink instance on local controller has eswitch devlink
> +ports representing ports for both the controllers.
> +
> +Port function configuration
> +===========================
> +
> +A user can configure the port function attribute before enumerating the
s/A user/User/
/port function attribute/$something_meaningful/
> +PCI function. Usually it means, user should configure port function attribute
attributes, plural
> +before a bus specific device for the function is created. However, when
> +SRIOV is enabled, virtual function devices are created on the PCI bus.
> +Hence, function attribute should be configured before binding virtual
> +function device to the driver.
> +
> +User may set the hardware address of the function represented by the devlink
> +port function. For Ethernet port function this means a MAC address.
next prev parent reply other threads:[~2020-12-16 0:58 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-15 9:03 [net-next v5 00/15] Add mlx5 subfunction support Saeed Mahameed
2020-12-15 9:03 ` [net-next v5 01/15] net/mlx5: Fix compilation warning for 32-bit platform Saeed Mahameed
2020-12-15 9:03 ` [net-next v5 02/15] devlink: Prepare code to fill multiple port function attributes Saeed Mahameed
2020-12-15 9:03 ` [net-next v5 03/15] devlink: Introduce PCI SF port flavour and port attribute Saeed Mahameed
2020-12-15 23:27 ` Jakub Kicinski
2020-12-16 3:42 ` Parav Pandit
2020-12-16 23:59 ` Jakub Kicinski
2020-12-17 4:44 ` Saeed Mahameed
2020-12-18 19:48 ` Jakub Kicinski
2020-12-19 4:43 ` Parav Pandit
2020-12-15 9:03 ` [net-next v5 04/15] devlink: Support add and delete devlink port Saeed Mahameed
2020-12-16 0:29 ` Jakub Kicinski
2020-12-16 5:06 ` Parav Pandit
2020-12-15 9:03 ` [net-next v5 05/15] devlink: Support get and set state of port function Saeed Mahameed
2020-12-16 0:37 ` Jakub Kicinski
2020-12-16 5:15 ` Parav Pandit
2020-12-16 16:15 ` David Ahern
2020-12-17 0:08 ` Jakub Kicinski
2020-12-17 5:46 ` Parav Pandit
2020-12-18 19:51 ` Jakub Kicinski
2020-12-19 5:06 ` Parav Pandit
2020-12-15 9:03 ` [net-next v5 06/15] net/mlx5: Introduce vhca state event notifier Saeed Mahameed
2020-12-15 9:03 ` [net-next v5 07/15] net/mlx5: SF, Add auxiliary device support Saeed Mahameed
2020-12-16 0:43 ` Jakub Kicinski
2020-12-16 5:19 ` Parav Pandit
2020-12-17 0:11 ` Jakub Kicinski
2020-12-17 5:23 ` Parav Pandit
2020-12-18 19:58 ` Jakub Kicinski
2020-12-19 4:53 ` Parav Pandit
2020-12-19 17:43 ` Jakub Kicinski
2020-12-15 9:03 ` [net-next v5 08/15] net/mlx5: SF, Add auxiliary device driver Saeed Mahameed
2020-12-15 9:03 ` [net-next v5 09/15] net/mlx5: E-switch, Prepare eswitch to handle SF vport Saeed Mahameed
2020-12-16 0:47 ` Jakub Kicinski
2020-12-16 5:28 ` Parav Pandit
2020-12-15 9:03 ` [net-next v5 10/15] net/mlx5: E-switch, Add eswitch helpers for " Saeed Mahameed
2020-12-15 9:03 ` [net-next v5 11/15] net/mlx5: SF, Add port add delete functionality Saeed Mahameed
2020-12-16 0:51 ` Jakub Kicinski
2020-12-16 5:31 ` Parav Pandit
2020-12-15 9:03 ` [net-next v5 12/15] net/mlx5: SF, Port function state change support Saeed Mahameed
2020-12-15 9:03 ` [net-next v5 13/15] devlink: Add devlink port documentation Saeed Mahameed
2020-12-16 0:57 ` Jakub Kicinski [this message]
2020-12-16 5:40 ` Parav Pandit
2020-12-15 9:03 ` [net-next v5 14/15] devlink: Extend devlink port documentation for subfunctions Saeed Mahameed
2020-12-16 1:00 ` Jakub Kicinski
2020-12-16 3:55 ` Parav Pandit
2020-12-15 9:03 ` [net-next v5 15/15] net/mlx5: Add devlink subfunction port documentation Saeed Mahameed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201215165758.4ff58f85@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com \
--to=kuba@kernel.org \
--cc=dan.j.williams@intel.com \
--cc=davem@davemloft.net \
--cc=david.m.ertman@intel.com \
--cc=dsahern@kernel.org \
--cc=gregkh@linuxfoundation.org \
--cc=jacob.e.keller@intel.com \
--cc=jgg@nvidia.com \
--cc=jiri@nvidia.com \
--cc=kiran.patil@intel.com \
--cc=leonro@nvidia.com \
--cc=linux-rdma@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=parav@nvidia.com \
--cc=saeed@kernel.org \
--cc=saeedm@nvidia.com \
--cc=sridhar.samudrala@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).