From: Alex Williamson <alex.williamson@redhat.com>
To: Shenming Lu <lushenming@huawei.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
Neo Jia <cjia@nvidia.com>,
mst@redhat.com, Marc Zyngier <maz@kernel.org>,
Cornelia Huck <cohuck@redhat.com>,
qemu-devel@nongnu.org,
"Dr . David Alan Gilbert" <dgilbert@redhat.com>,
Eric Auger <eric.auger@redhat.com>,
Kirti Wankhede <kwankhede@nvidia.com>,
qemu-arm@nongnu.org, yuzenghui@huawei.com,
wanghaibin.wang@huawei.com
Subject: Re: [PATCH v3 3/3] vfio: Avoid disabling and enabling vectors repeatedly in VFIO migration
Date: Tue, 9 Mar 2021 11:12:11 -0700 [thread overview]
Message-ID: <20210309111211.7b2d7b0d@omen.home.shazbot.org> (raw)
In-Reply-To: <97a3189a-f053-cb77-0ae9-02530fb4ac89@huawei.com>
On Tue, 9 Mar 2021 15:25:52 +0800
Shenming Lu <lushenming@huawei.com> wrote:
> On 2021/3/3 5:26, Alex Williamson wrote:
> >
> > MST/Marcel,
> >
> > Do you have an Ack or objection to exporting msix_masked() as below?
>
> Or could we use msix_function_masked instead as below? :-)
>
> if (!pdev->msix_function_masked) {
> for (nr = 0; nr < msix_nr_vectors_allocated(pdev); nr++) {
> if (!msix_is_masked(pdev, nr)) {
> max_vec = nr;
> }
> }
> }
Yeah, it looks like that should work. I'll look for a v4 with that
change. Thanks,
Alex
> > On Tue, 23 Feb 2021 10:22:25 +0800
> > Shenming Lu <lushenming@huawei.com> wrote:
> >
> >> In VFIO migration resume phase and some guest startups, there are
> >> already unmasked vectors in the vector table when calling
> >> vfio_msix_enable(). So in order to avoid inefficiently disabling
> >> and enabling vectors repeatedly, let's allocate all needed vectors
> >> first and then enable these unmasked vectors one by one without
> >> disabling.
> >>
> >> Signed-off-by: Shenming Lu <lushenming@huawei.com>
> >> ---
> >> hw/pci/msix.c | 2 +-
> >> hw/vfio/pci.c | 20 +++++++++++++++++---
> >> include/hw/pci/msix.h | 1 +
> >> 3 files changed, 19 insertions(+), 4 deletions(-)
> >>
> >> diff --git a/hw/pci/msix.c b/hw/pci/msix.c
> >> index ae9331cd0b..e057958fcd 100644
> >> --- a/hw/pci/msix.c
> >> +++ b/hw/pci/msix.c
> >> @@ -131,7 +131,7 @@ static void msix_handle_mask_update(PCIDevice *dev, int vector, bool was_masked)
> >> }
> >> }
> >>
> >> -static bool msix_masked(PCIDevice *dev)
> >> +bool msix_masked(PCIDevice *dev)
> >> {
> >> return dev->config[dev->msix_cap + MSIX_CONTROL_OFFSET] & MSIX_MASKALL_MASK;
> >> }
> >> diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
> >> index f74be78209..088fd41926 100644
> >> --- a/hw/vfio/pci.c
> >> +++ b/hw/vfio/pci.c
> >> @@ -569,6 +569,9 @@ static void vfio_msix_vector_release(PCIDevice *pdev, unsigned int nr)
> >>
> >> static void vfio_msix_enable(VFIOPCIDevice *vdev)
> >> {
> >> + PCIDevice *pdev = &vdev->pdev;
> >> + unsigned int nr, max_vec = 0;
> >> +
> >> vfio_disable_interrupts(vdev);
> >>
> >> vdev->msi_vectors = g_new0(VFIOMSIVector, vdev->msix->entries);
> >> @@ -587,11 +590,22 @@ static void vfio_msix_enable(VFIOPCIDevice *vdev)
> >> * triggering to userspace, then immediately release the vector, leaving
> >> * the physical device with no vectors enabled, but MSI-X enabled, just
> >> * like the guest view.
> >> + * If there are already unmasked vectors (in migration resume phase and
> >> + * some guest startups) which will be enabled soon, we can allocate all
> >> + * of them here to avoid inefficiently disabling and enabling vectors
> >> + * repeatedly later.
> >> */
> >> - vfio_msix_vector_do_use(&vdev->pdev, 0, NULL, NULL);
> >> - vfio_msix_vector_release(&vdev->pdev, 0);
> >> + if (!msix_masked(pdev)) {
> >> + for (nr = 0; nr < msix_nr_vectors_allocated(pdev); nr++) {
> >> + if (!msix_is_masked(pdev, nr)) {
> >> + max_vec = nr;
> >> + }
> >> + }
> >> + }
> >> + vfio_msix_vector_do_use(pdev, max_vec, NULL, NULL);
> >> + vfio_msix_vector_release(pdev, max_vec);
> >>
> >> - if (msix_set_vector_notifiers(&vdev->pdev, vfio_msix_vector_use,
> >> + if (msix_set_vector_notifiers(pdev, vfio_msix_vector_use,
> >> vfio_msix_vector_release, NULL)) {
> >> error_report("vfio: msix_set_vector_notifiers failed");
> >> }
> >> diff --git a/include/hw/pci/msix.h b/include/hw/pci/msix.h
> >> index 4c4a60c739..b3cd88e262 100644
> >> --- a/include/hw/pci/msix.h
> >> +++ b/include/hw/pci/msix.h
> >> @@ -28,6 +28,7 @@ void msix_load(PCIDevice *dev, QEMUFile *f);
> >>
> >> int msix_enabled(PCIDevice *dev);
> >> int msix_present(PCIDevice *dev);
> >> +bool msix_masked(PCIDevice *dev);
> >>
> >> bool msix_is_masked(PCIDevice *dev, unsigned vector);
> >> void msix_set_pending(PCIDevice *dev, unsigned vector);
> >
> > .
> >
>
next prev parent reply other threads:[~2021-03-09 19:44 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-23 2:22 [PATCH v3 0/3] vfio: Some fixes and optimizations for VFIO migration Shenming Lu
2021-02-23 2:22 ` [PATCH v3 1/3] vfio: Move the saving of the config space to the right place in " Shenming Lu
2021-03-01 5:57 ` Kirti Wankhede
2021-02-23 2:22 ` [PATCH v3 2/3] vfio: Set the priority of the VFIO VM state change handler explicitly Shenming Lu
2021-02-23 2:22 ` [PATCH v3 3/3] vfio: Avoid disabling and enabling vectors repeatedly in VFIO migration Shenming Lu
2021-03-02 21:26 ` Alex Williamson
2021-03-09 7:25 ` Shenming Lu
2021-03-09 18:12 ` Alex Williamson [this message]
2021-03-02 5:58 ` [PATCH v3 0/3] vfio: Some fixes and optimizations for " Shenming Lu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210309111211.7b2d7b0d@omen.home.shazbot.org \
--to=alex.williamson@redhat.com \
--cc=cjia@nvidia.com \
--cc=cohuck@redhat.com \
--cc=dgilbert@redhat.com \
--cc=eric.auger@redhat.com \
--cc=kwankhede@nvidia.com \
--cc=lorenzo.pieralisi@arm.com \
--cc=lushenming@huawei.com \
--cc=maz@kernel.org \
--cc=mst@redhat.com \
--cc=qemu-arm@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=wanghaibin.wang@huawei.com \
--cc=yuzenghui@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).