linux-rdma.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Parav Pandit <parav@nvidia.com>
To: Jakub Kicinski <kuba@kernel.org>, Saeed Mahameed <saeed@kernel.org>
Cc: "David S. Miller" <davem@davemloft.net>,
	Jason Gunthorpe <jgg@nvidia.com>,
	Leon Romanovsky <leonro@nvidia.com>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
	David Ahern <dsahern@kernel.org>,
	Jacob Keller <jacob.e.keller@intel.com>,
	Sridhar Samudrala <sridhar.samudrala@intel.com>,
	"david.m.ertman@intel.com" <david.m.ertman@intel.com>,
	"dan.j.williams@intel.com" <dan.j.williams@intel.com>,
	"kiran.patil@intel.com" <kiran.patil@intel.com>,
	"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
	Jiri Pirko <jiri@nvidia.com>, Saeed Mahameed <saeedm@nvidia.com>
Subject: RE: [net-next v5 13/15] devlink: Add devlink port documentation
Date: Wed, 16 Dec 2020 05:40:05 +0000	[thread overview]
Message-ID: <BY5PR12MB4322AC71674FB8DB605404C5DCC50@BY5PR12MB4322.namprd12.prod.outlook.com> (raw)
In-Reply-To: <20201215165758.4ff58f85@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>


> From: Jakub Kicinski <kuba@kernel.org>
> Sent: Wednesday, December 16, 2020 6:28 AM
> 
> On Tue, 15 Dec 2020 01:03:56 -0800 Saeed Mahameed wrote:
> > +PCI controllers
> > +---------------
> > +In most cases a PCI device has only one controller. A controller
> > +consists of potentially multiple physical and virtual functions. Such
> > +PCI function consists of one or more ports.
> 
> s/Such//
>
Ack.
 
> you say consists in two consecutive sentences.
> 
> > This port of the function is represented by the devlink eswitch port.
> 
First sentence describe controller. Second sentence describe function.
So what is wrong in that?

> "This port of the function"? Why not just "Each port"?
> 
That's fine too. Will simplify.

> > +A PCI Device connected to multiple CPUs or multiple PCI root
> > +complexes or
> 
> Why is device capitalized all of the sudden?
>
Will fix.
 
> > +SmartNIC, however, may have multiple controllers. For a device with
> > +multiple
> 
> a SmartNIC or SmartNICs
> 
> > +controllers, each controller is distinguished by a unique controller
> number.
> > +An eswitch on the PCI device support ports of multiple controllers.
> 
> eswitch is on a PCI device?
>
Will change.
 
> > +An example view of a system with two controllers::
> > +
> > +                 ---------------------------------------------------------
> > +                 |                                                       |
> > +                 |           --------- ---------         ------- ------- |
> > +    -----------  |           | vf(s) | | sf(s) |         |vf(s)| |sf(s)| |
> > +    | server  |  | -------   ----/---- ---/----- ------- ---/--- ---/--- |
> > +    | pci rc  |=== | pf0 |______/________/       | pf1 |___/_______/     |
> > +    | connect |  | -------                       -------                 |
> > +    -----------  |     | controller_num=1 (no eswitch)                   |
> > +                 ------|--------------------------------------------------
> > +                 (internal wire)
> > +                       |
> > +                 ---------------------------------------------------------
> > +                 | devlink eswitch ports and reps                        |
> > +                 | ----------------------------------------------------- |
> > +                 | |ctrl-0 | ctrl-0 | ctrl-0 | ctrl-0 | ctrl-0 |ctrl-0 | |
> > +                 | |pf0    | pf0vfN | pf0sfN | pf1    | pf1vfN |pf1sfN | |
> > +                 | ----------------------------------------------------- |
> > +                 | |ctrl-1 | ctrl-1 | ctrl-1 | ctrl-1 | ctrl-1 |ctrl-1 | |
> > +                 | |pf0    | pf0vfN | pf0sfN | pf1    | pf1vfN |pf1sfN | |
> > +                 | ----------------------------------------------------- |
> > +                 |                                                       |
> > +                 |                                                       |
> > +    -----------  |           --------- ---------         ------- ------- |
> > +    | smartNIC|  |           | vf(s) | | sf(s) |         |vf(s)| |sf(s)| |
> > +    | pci rc  |==| -------   ----/---- ---/----- ------- ---/--- ---/--- |
> > +    | connect |  | | pf0 |______/________/       | pf1 |___/_______/     |
> > +    -----------  | -------                       -------                 |
> > +                 |                                                       |
> > +                 |  local controller_num=0 (eswitch)                     |
> > +
> > + ---------------------------------------------------------
> > +
> > +In above example, external controller (identified by controller
> > +number = 1) doesn't have eswitch. Local controller (identified by
> > +controller number = 0) has the eswitch. Devlink instance on local
> > +controller has eswitch devlink ports representing ports for both the
> controllers.
> > +
> > +Port function configuration
> > +===========================
> > +
> > +A user can configure the port function attribute before enumerating
> > +the
> 
> s/A user/User/
> 
> /port function attribute/$something_meaningful/
> 
May be just say function attribute?

> > +PCI function. Usually it means, user should configure port function
> > +attribute
> 
> attributes, plural
> 
Yes, but at present there is only one i.e. mac address, so didn't use plural.

  reply	other threads:[~2020-12-16  5:40 UTC|newest]

Thread overview: 46+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-12-15  9:03 [net-next v5 00/15] Add mlx5 subfunction support Saeed Mahameed
2020-12-15  9:03 ` [net-next v5 01/15] net/mlx5: Fix compilation warning for 32-bit platform Saeed Mahameed
2020-12-15  9:03 ` [net-next v5 02/15] devlink: Prepare code to fill multiple port function attributes Saeed Mahameed
2020-12-15  9:03 ` [net-next v5 03/15] devlink: Introduce PCI SF port flavour and port attribute Saeed Mahameed
2020-12-15 23:27   ` Jakub Kicinski
2020-12-16  3:42     ` Parav Pandit
2020-12-16 23:59       ` Jakub Kicinski
2020-12-17  4:44         ` Saeed Mahameed
2020-12-18 19:48           ` Jakub Kicinski
2020-12-19  4:43             ` Parav Pandit
2020-12-15  9:03 ` [net-next v5 04/15] devlink: Support add and delete devlink port Saeed Mahameed
2020-12-16  0:29   ` Jakub Kicinski
2020-12-16  5:06     ` Parav Pandit
2020-12-15  9:03 ` [net-next v5 05/15] devlink: Support get and set state of port function Saeed Mahameed
2020-12-16  0:37   ` Jakub Kicinski
2020-12-16  5:15     ` Parav Pandit
2020-12-16 16:15       ` David Ahern
2020-12-17  0:08       ` Jakub Kicinski
2020-12-17  5:46         ` Parav Pandit
2020-12-18 19:51           ` Jakub Kicinski
2020-12-19  5:06             ` Parav Pandit
2020-12-15  9:03 ` [net-next v5 06/15] net/mlx5: Introduce vhca state event notifier Saeed Mahameed
2020-12-15  9:03 ` [net-next v5 07/15] net/mlx5: SF, Add auxiliary device support Saeed Mahameed
2020-12-16  0:43   ` Jakub Kicinski
2020-12-16  5:19     ` Parav Pandit
2020-12-17  0:11       ` Jakub Kicinski
2020-12-17  5:23         ` Parav Pandit
2020-12-18 19:58           ` Jakub Kicinski
2020-12-19  4:53             ` Parav Pandit
2020-12-19 17:43               ` Jakub Kicinski
2020-12-15  9:03 ` [net-next v5 08/15] net/mlx5: SF, Add auxiliary device driver Saeed Mahameed
2020-12-15  9:03 ` [net-next v5 09/15] net/mlx5: E-switch, Prepare eswitch to handle SF vport Saeed Mahameed
2020-12-16  0:47   ` Jakub Kicinski
2020-12-16  5:28     ` Parav Pandit
2020-12-15  9:03 ` [net-next v5 10/15] net/mlx5: E-switch, Add eswitch helpers for " Saeed Mahameed
2020-12-15  9:03 ` [net-next v5 11/15] net/mlx5: SF, Add port add delete functionality Saeed Mahameed
2020-12-16  0:51   ` Jakub Kicinski
2020-12-16  5:31     ` Parav Pandit
2020-12-15  9:03 ` [net-next v5 12/15] net/mlx5: SF, Port function state change support Saeed Mahameed
2020-12-15  9:03 ` [net-next v5 13/15] devlink: Add devlink port documentation Saeed Mahameed
2020-12-16  0:57   ` Jakub Kicinski
2020-12-16  5:40     ` Parav Pandit [this message]
2020-12-15  9:03 ` [net-next v5 14/15] devlink: Extend devlink port documentation for subfunctions Saeed Mahameed
2020-12-16  1:00   ` Jakub Kicinski
2020-12-16  3:55     ` Parav Pandit
2020-12-15  9:03 ` [net-next v5 15/15] net/mlx5: Add devlink subfunction port documentation Saeed Mahameed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=BY5PR12MB4322AC71674FB8DB605404C5DCC50@BY5PR12MB4322.namprd12.prod.outlook.com \
    --to=parav@nvidia.com \
    --cc=dan.j.williams@intel.com \
    --cc=davem@davemloft.net \
    --cc=david.m.ertman@intel.com \
    --cc=dsahern@kernel.org \
    --cc=gregkh@linuxfoundation.org \
    --cc=jacob.e.keller@intel.com \
    --cc=jgg@nvidia.com \
    --cc=jiri@nvidia.com \
    --cc=kiran.patil@intel.com \
    --cc=kuba@kernel.org \
    --cc=leonro@nvidia.com \
    --cc=linux-rdma@vger.kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=saeed@kernel.org \
    --cc=saeedm@nvidia.com \
    --cc=sridhar.samudrala@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).