From: Yan Zhao <yan.y.zhao@intel.com>
To: "Raj, Ashok" <ashok.raj@intel.com>
Cc: "Lu, Baolu" <baolu.lu@intel.com>,
"Tian, Kevin" <kevin.tian@intel.com>,
"Liu, Yi L" <yi.l.liu@intel.com>,
"alex.williamson@redhat.com" <alex.williamson@redhat.com>,
"eric.auger@redhat.com" <eric.auger@redhat.com>,
"jacob.jun.pan@linux.intel.com" <jacob.jun.pan@linux.intel.com>,
"joro@8bytes.org" <joro@8bytes.org>,
"Tian, Jun J" <jun.j.tian@intel.com>,
"Sun, Yi Y" <yi.y.sun@intel.com>,
"jean-philippe@linaro.org" <jean-philippe@linaro.org>,
"peterx@redhat.com" <peterx@redhat.com>,
"iommu@lists.linux-foundation.org"
<iommu@lists.linux-foundation.org>,
"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"Wu, Hao" <hao.wu@intel.com>
Subject: Re: [PATCH v1 0/2] vfio/pci: expose device's PASID capability to VMs
Date: Thu, 16 Apr 2020 21:13:34 -0400 [thread overview]
Message-ID: <20200417011334.GB16688@joy-OptiPlex-7040> (raw)
In-Reply-To: <20200416223353.GC45480@otc-nc-03>
On Fri, Apr 17, 2020 at 06:33:54AM +0800, Raj, Ashok wrote:
> Hi Zhao
>
>
> On Thu, Apr 16, 2020 at 06:12:26PM -0400, Yan Zhao wrote:
> > On Tue, Mar 31, 2020 at 03:08:25PM +0800, Lu, Baolu wrote:
> > > On 2020/3/31 14:35, Tian, Kevin wrote:
> > > >> From: Liu, Yi L<yi.l.liu@intel.com>
> > > >> Sent: Sunday, March 22, 2020 8:33 PM
> > > >>
> > > >> From: Liu Yi L<yi.l.liu@intel.com>
> > > >>
> > > >> Shared Virtual Addressing (SVA), a.k.a, Shared Virtual Memory (SVM) on
> > > >> Intel platforms allows address space sharing between device DMA and
> > > >> applications. SVA can reduce programming complexity and enhance security.
> > > >>
> > > >> To enable SVA, device needs to have PASID capability, which is a key
> > > >> capability for SVA. This patchset exposes the device's PASID capability
> > > >> to guest instead of hiding it from guest.
> > > >>
> > > >> The second patch emulates PASID capability for VFs (Virtual Function) since
> > > >> VFs don't implement such capability per PCIe spec. This patch emulates such
> > > >> capability and expose to VM if the capability is enabled in PF (Physical
> > > >> Function).
> > > >>
> > > >> However, there is an open for PASID emulation. If PF driver disables PASID
> > > >> capability at runtime, then it may be an issue. e.g. PF should not disable
> > > >> PASID capability if there is guest using this capability on any VF related
> > > >> to this PF. To solve it, may need to introduce a generic communication
> > > >> framework between vfio-pci driver and PF drivers. Please feel free to give
> > > >> your suggestions on it.
> > > > I'm not sure how this is addressed on bate metal today, i.e. between normal
> > > > kernel PF and VF drivers. I look at pasid enable/disable code in intel-iommu.c.
> > > > There is no check on PF/VF dependency so far. The cap is toggled when
> > > > attaching/detaching the PF to its domain. Let's see how IOMMU guys
> > > > respond, and if there is a way for VF driver to block PF driver from disabling
> > > > the pasid cap when it's being actively used by VF driver, then we may
> > > > leverage the same trick in VFIO when emulation is provided to guest.
> > >
> > > IOMMU subsystem doesn't expose any APIs for pasid enabling/disabling.
> > > The PCI subsystem does. It handles VF/PF like below.
> > >
> > > /**
> > > * pci_enable_pasid - Enable the PASID capability
> > > * @pdev: PCI device structure
> > > * @features: Features to enable
> > > *
> > > * Returns 0 on success, negative value on error. This function checks
> > > * whether the features are actually supported by the device and returns
> > > * an error if not.
> > > */
> > > int pci_enable_pasid(struct pci_dev *pdev, int features)
> > > {
> > > u16 control, supported;
> > > int pasid = pdev->pasid_cap;
> > >
> > > /*
> > > * VFs must not implement the PASID Capability, but if a PF
> > > * supports PASID, its VFs share the PF PASID configuration.
> > > */
> > > if (pdev->is_virtfn) {
> > > if (pci_physfn(pdev)->pasid_enabled)
> > > return 0;
> > > return -EINVAL;
> > > }
> > >
> > > /**
> > > * pci_disable_pasid - Disable the PASID capability
> > > * @pdev: PCI device structure
> > > */
> > > void pci_disable_pasid(struct pci_dev *pdev)
> > > {
> > > u16 control = 0;
> > > int pasid = pdev->pasid_cap;
> > >
> > > /* VFs share the PF PASID configuration */
> > > if (pdev->is_virtfn)
> > > return;
> > >
> > >
> > > It doesn't block disabling PASID on PF even VFs are possibly using it.
> > >
> > hi
> > I'm not sure, but is it possible for pci_enable_pasid() and
> > pci_disable_pasid() to do the same thing as pdev->driver->sriov_configure,
> > e.g. pci_sriov_configure_simple() below.
> >
> > It checks whether there are VFs are assigned in pci_vfs_assigned(dev).
> > and we can set the VF in assigned status if vfio_pci_open() is performed
> > on the VF.
>
> But you can still unbind the PF driver that magically causes the VF's to be
> removed from the guest image too correct?
>
> Only the IOMMU mucks with pasid_enable/disable. And it doesn't look like
> we have a path to disable without tearing down the PF binding.
>
> We originally had some refcounts and such and would do the real disable only
> when the refcount drops to 0, but we found it wasn't actually necessary
> to protect these resources like that.
>
right. now unbinding PF driver would cause VFs unplugged from guest.
if we modify vfio_pci and set VFs to be assigned, then VFs could remain
appearing in guest but it cannot function well as PF driver has been unbound.
thanks for explanation :)
> >
> >
> > int pci_sriov_configure_simple(struct pci_dev *dev, int nr_virtfn)
> > {
> > int rc;
> >
> > might_sleep();
> >
> > if (!dev->is_physfn)
> > return -ENODEV;
> >
> > if (pci_vfs_assigned(dev)) {
> > pci_warn(dev, "Cannot modify SR-IOV while VFs are assigned\n");
> > return -EPERM;
> > }
> >
> > if (nr_virtfn == 0) {
> > sriov_disable(dev);
> > return 0;
> > }
> >
> > rc = sriov_enable(dev, nr_virtfn);
> > if (rc < 0)
> > return rc;
> >
> > return nr_virtfn;
> > }
> >
> > Thanks
> > Yan
prev parent reply other threads:[~2020-04-17 1:23 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-22 12:33 [PATCH v1 0/2] vfio/pci: expose device's PASID capability to VMs Liu, Yi L
2020-03-22 12:33 ` [PATCH v1 1/2] vfio/pci: Expose PCIe PASID capability to guest Liu, Yi L
2020-03-31 6:39 ` Tian, Kevin
2020-03-31 6:42 ` Liu, Yi L
2020-03-22 12:33 ` [PATCH v1 2/2] vfio/pci: Emulate PASID/PRI capability for VFs Liu, Yi L
2020-04-02 22:59 ` Alex Williamson
2020-04-03 7:53 ` Liu, Yi L
2020-04-03 17:25 ` Alex Williamson
2020-04-07 4:26 ` Tian, Kevin
2020-04-07 15:58 ` Alex Williamson
2020-04-08 0:27 ` Tian, Kevin
2020-04-08 4:00 ` Raj, Ashok
2020-04-08 16:19 ` Alex Williamson
2020-04-08 16:33 ` Raj, Ashok
2020-04-09 7:35 ` Jean-Philippe Brucker
2020-04-13 19:44 ` Alex Williamson
2020-04-13 3:10 ` Raj, Ashok
2020-04-13 3:29 ` Raj, Ashok
2020-04-13 19:10 ` Alex Williamson
2020-04-13 7:54 ` Tian, Kevin
2020-04-13 8:05 ` Tian, Kevin
2020-04-13 19:21 ` Alex Williamson
2020-04-14 2:40 ` Tian, Kevin
2020-04-14 3:28 ` Alex Williamson
2020-04-14 3:42 ` Tian, Kevin
2020-04-14 15:24 ` Alex Williamson
2020-04-14 23:57 ` Tian, Kevin
2020-04-15 0:36 ` Alex Williamson
2020-04-15 1:01 ` Tian, Kevin
2020-04-03 11:42 ` Liu, Yi L
2020-03-31 6:35 ` [PATCH v1 0/2] vfio/pci: expose device's PASID capability to VMs Tian, Kevin
2020-03-31 7:08 ` Lu, Baolu
2020-04-16 22:12 ` Yan Zhao
2020-04-16 22:33 ` Raj, Ashok
2020-04-17 1:13 ` Yan Zhao [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200417011334.GB16688@joy-OptiPlex-7040 \
--to=yan.y.zhao@intel.com \
--cc=alex.williamson@redhat.com \
--cc=ashok.raj@intel.com \
--cc=baolu.lu@intel.com \
--cc=eric.auger@redhat.com \
--cc=hao.wu@intel.com \
--cc=iommu@lists.linux-foundation.org \
--cc=jacob.jun.pan@linux.intel.com \
--cc=jean-philippe@linaro.org \
--cc=joro@8bytes.org \
--cc=jun.j.tian@intel.com \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=peterx@redhat.com \
--cc=yi.l.liu@intel.com \
--cc=yi.y.sun@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).