From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: Leon Romanovsky <leon@kernel.org>
Cc: Bjorn Helgaas <bhelgaas@google.com>,
Saeed Mahameed <saeedm@nvidia.com>,
Jason Gunthorpe <jgg@nvidia.com>,
Alexander Duyck <alexander.duyck@gmail.com>,
Jakub Kicinski <kuba@kernel.org>,
linux-pci@vger.kernel.org, linux-rdma@vger.kernel.org,
netdev@vger.kernel.org, Don Dutile <ddutile@redhat.com>,
Alex Williamson <alex.williamson@redhat.com>,
"David S . Miller" <davem@davemloft.net>
Subject: Re: [PATCH mlx5-next v7 1/4] PCI: Add a sysfs file to change the MSI-X table size of SR-IOV VFs
Date: Mon, 1 Mar 2021 09:37:17 +0100 [thread overview]
Message-ID: <YDynvUsEGQ6bQGep@kroah.com> (raw)
In-Reply-To: <YDymifxqjvnaW3nw@unreal>
On Mon, Mar 01, 2021 at 10:32:09AM +0200, Leon Romanovsky wrote:
> On Mon, Mar 01, 2021 at 09:14:42AM +0100, Greg Kroah-Hartman wrote:
> > On Mon, Mar 01, 2021 at 09:55:21AM +0200, Leon Romanovsky wrote:
> > > From: Leon Romanovsky <leonro@nvidia.com>
> > >
> > > A typical cloud provider SR-IOV use case is to create many VFs for use by
> > > guest VMs. The VFs may not be assigned to a VM until a customer requests a
> > > VM of a certain size, e.g., number of CPUs. A VF may need MSI-X vectors
> > > proportional to the number of CPUs in the VM, but there is no standard way
> > > to change the number of MSI-X vectors supported by a VF.
> > >
> > > Some Mellanox ConnectX devices support dynamic assignment of MSI-X vectors
> > > to SR-IOV VFs. This can be done by the PF driver after VFs are enabled,
> > > and it can be done without affecting VFs that are already in use. The
> > > hardware supports a limited pool of MSI-X vectors that can be assigned to
> > > the PF or to individual VFs. This is device-specific behavior that
> > > requires support in the PF driver.
> > >
> > > Add a read-only "sriov_vf_total_msix" sysfs file for the PF and a writable
> > > "sriov_vf_msix_count" file for each VF. Management software may use these
> > > to learn how many MSI-X vectors are available and to dynamically assign
> > > them to VFs before the VFs are passed through to a VM.
> > >
> > > If the PF driver implements the ->sriov_get_vf_total_msix() callback,
> > > "sriov_vf_total_msix" contains the total number of MSI-X vectors available
> > > for distribution among VFs.
> > >
> > > If no driver is bound to the VF, writing "N" to "sriov_vf_msix_count" uses
> > > the PF driver ->sriov_set_msix_vec_count() callback to assign "N" MSI-X
> > > vectors to the VF. When a VF driver subsequently reads the MSI-X Message
> > > Control register, it will see the new Table Size "N".
> > >
> > > Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
> > > ---
> > > Documentation/ABI/testing/sysfs-bus-pci | 29 +++++++
> > > drivers/pci/iov.c | 102 ++++++++++++++++++++++--
> > > drivers/pci/pci-sysfs.c | 3 +-
> > > drivers/pci/pci.h | 3 +-
> > > include/linux/pci.h | 8 ++
> > > 5 files changed, 137 insertions(+), 8 deletions(-)
> > >
> > > diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/testing/sysfs-bus-pci
> > > index 25c9c39770c6..ebabd0d2ae88 100644
> > > --- a/Documentation/ABI/testing/sysfs-bus-pci
> > > +++ b/Documentation/ABI/testing/sysfs-bus-pci
> > > @@ -375,3 +375,32 @@ Description:
> > > The value comes from the PCI kernel device state and can be one
> > > of: "unknown", "error", "D0", D1", "D2", "D3hot", "D3cold".
> > > The file is read only.
> > > +
> > > +What: /sys/bus/pci/devices/.../sriov_vf_total_msix
> > > +Date: January 2021
> > > +Contact: Leon Romanovsky <leonro@nvidia.com>
> > > +Description:
> > > + This file is associated with a SR-IOV PF. It contains the
> > > + total number of MSI-X vectors available for assignment to
> > > + all VFs associated with PF. It will zero if the device
> >
> > "The value will be zero if the device..."
>
> Thanks, will fix when apply or resend if more fixes will be needed.
>
> >
> > And definition of "VF" and PF" are where in this file?
>
> They come from the PCI spec. It is part of SR-IOV lingo.
Yes, and the world does not have access to the PCI spec, so please
provide a hint as to what they are for those of us without access to
such things.
thanks,
greg k-h
next prev parent reply other threads:[~2021-03-01 8:38 UTC|newest]
Thread overview: 74+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-01 7:55 [PATCH mlx5-next v7 0/4] Dynamically assign MSI-X vectors count Leon Romanovsky
2021-03-01 7:55 ` [PATCH mlx5-next v7 1/4] PCI: Add a sysfs file to change the MSI-X table size of SR-IOV VFs Leon Romanovsky
2021-03-01 8:14 ` Greg Kroah-Hartman
2021-03-01 8:32 ` Leon Romanovsky
2021-03-01 8:37 ` Greg Kroah-Hartman [this message]
2021-03-01 8:53 ` Leon Romanovsky
2021-03-01 7:55 ` [PATCH mlx5-next v7 2/4] net/mlx5: Add dynamic MSI-X capabilities bits Leon Romanovsky
2021-03-01 7:55 ` [PATCH mlx5-next v7 3/4] net/mlx5: Dynamically assign MSI-X vectors count Leon Romanovsky
2021-03-01 7:55 ` [PATCH mlx5-next v7 4/4] net/mlx5: Implement sriov_get_vf_total_msix/count() callbacks Leon Romanovsky
2021-03-07 8:11 ` [PATCH mlx5-next v7 0/4] Dynamically assign MSI-X vectors count Leon Romanovsky
2021-03-07 18:55 ` Alexander Duyck
2021-03-07 19:19 ` Leon Romanovsky
2021-03-08 16:33 ` Alexander Duyck
2021-03-08 19:20 ` Leon Romanovsky
2021-03-10 19:09 ` Bjorn Helgaas
2021-03-10 20:10 ` Leon Romanovsky
2021-03-10 20:21 ` Greg Kroah-Hartman
2021-03-11 8:37 ` Leon Romanovsky
2021-03-10 23:34 ` Alexander Duyck
2021-03-11 18:17 ` Bjorn Helgaas
2021-03-11 19:16 ` Keith Busch
2021-03-11 19:21 ` Leon Romanovsky
2021-03-11 20:22 ` Jason Gunthorpe
2021-03-11 20:50 ` Keith Busch
2021-03-11 21:44 ` Jason Gunthorpe
2021-03-25 17:21 ` Bjorn Helgaas
2021-03-25 17:36 ` Jason Gunthorpe
2021-03-25 18:20 ` Bjorn Helgaas
2021-03-25 18:28 ` Jason Gunthorpe
2021-03-26 6:44 ` Leon Romanovsky
2021-03-26 16:00 ` Alexander Duyck
2021-03-26 16:56 ` Jason Gunthorpe
2021-03-26 17:08 ` Bjorn Helgaas
2021-03-26 17:12 ` Jason Gunthorpe
2021-03-27 6:00 ` Leon Romanovsky
2021-03-26 17:29 ` Keith Busch
2021-03-26 17:31 ` Jason Gunthorpe
2021-03-26 18:50 ` Alexander Duyck
2021-03-26 19:01 ` Jason Gunthorpe
2021-03-30 1:29 ` Bjorn Helgaas
2021-03-30 13:57 ` Jason Gunthorpe
2021-03-30 15:00 ` Bjorn Helgaas
2021-03-30 19:47 ` Jason Gunthorpe
2021-03-30 20:41 ` Bjorn Helgaas
2021-03-30 22:43 ` Jason Gunthorpe
2021-03-31 6:38 ` Greg Kroah-Hartman
2021-03-31 12:19 ` Jason Gunthorpe
2021-03-31 15:03 ` Greg Kroah-Hartman
2021-03-31 17:07 ` Jason Gunthorpe
2021-03-31 4:08 ` Leon Romanovsky
2021-04-01 1:23 ` Bjorn Helgaas
2021-04-01 11:49 ` Leon Romanovsky
2021-03-30 18:10 ` Keith Busch
2021-03-26 19:36 ` Bjorn Helgaas
2021-03-27 12:38 ` Greg Kroah-Hartman
2021-03-25 18:31 ` Keith Busch
2021-03-25 18:36 ` Jason Gunthorpe
2021-03-11 19:17 ` Leon Romanovsky
2021-03-11 19:37 ` Alexander Duyck
2021-03-11 19:51 ` Leon Romanovsky
2021-03-11 20:11 ` Alexander Duyck
2021-03-11 20:19 ` Jason Gunthorpe
2021-03-11 21:49 ` Alexander Duyck
2021-03-11 23:20 ` Jason Gunthorpe
2021-03-12 2:53 ` Alexander Duyck
2021-03-12 6:32 ` Leon Romanovsky
2021-03-12 16:59 ` Alexander Duyck
2021-03-12 17:03 ` Jason Gunthorpe
2021-03-12 18:34 ` Leon Romanovsky
2021-03-12 18:41 ` Leon Romanovsky
2021-03-12 13:00 ` Jason Gunthorpe
2021-03-12 13:36 ` Keith Busch
2021-03-11 20:31 ` Jason Gunthorpe
2021-03-10 5:58 ` Leon Romanovsky
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YDynvUsEGQ6bQGep@kroah.com \
--to=gregkh@linuxfoundation.org \
--cc=alex.williamson@redhat.com \
--cc=alexander.duyck@gmail.com \
--cc=bhelgaas@google.com \
--cc=davem@davemloft.net \
--cc=ddutile@redhat.com \
--cc=jgg@nvidia.com \
--cc=kuba@kernel.org \
--cc=leon@kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=saeedm@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).