From: "Tian, Kevin" <kevin.tian@intel.com>
To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>,
"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"linux-crypto@vger.kernel.org" <linux-crypto@vger.kernel.org>
Cc: "alex.williamson@redhat.com" <alex.williamson@redhat.com>,
"jgg@nvidia.com" <jgg@nvidia.com>,
"mgurtovoy@nvidia.com" <mgurtovoy@nvidia.com>,
Linuxarm <linuxarm@huawei.com>,
liulongfang <liulongfang@huawei.com>,
"Zengtao (B)" <prime.zeng@hisilicon.com>,
"Jonathan Cameron" <jonathan.cameron@huawei.com>,
"Wangzhou (B)" <wangzhou1@hisilicon.com>,
"He, Shaopeng" <shaopeng.he@intel.com>,
"Zhao, Yan Y" <yan.y.zhao@intel.com>
Subject: RE: [PATCH v3 0/6] vfio/hisilicon: add acc live migration driver
Date: Fri, 15 Oct 2021 05:53:52 +0000 [thread overview]
Message-ID: <BN9PR11MB5433164D17BB4C30EAEAC41F8CB99@BN9PR11MB5433.namprd11.prod.outlook.com> (raw)
In-Reply-To: <61274f6497424f039397677fb5d003d0@huawei.com>
> From: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> Sent: Thursday, September 30, 2021 2:35 PM
>
> > From: Tian, Kevin [mailto:kevin.tian@intel.com]
> > Sent: 30 September 2021 01:42
> >
> > > From: Shameerali Kolothum Thodi
> > > <shameerali.kolothum.thodi@huawei.com>
> > >
> > > > From: Tian, Kevin [mailto:kevin.tian@intel.com]
> > > > Sent: 29 September 2021 10:06
> > > >
> > > > > From: Shameerali Kolothum Thodi
> > > > > <shameerali.kolothum.thodi@huawei.com>
> > > > >
> > > > > Hi Kevin,
> > > > >
> > > > > > From: Tian, Kevin [mailto:kevin.tian@intel.com]
> > > > > > Sent: 29 September 2021 04:58
> > > > > >
> > > > > > Hi, Shameer,
> > > > > >
> > > > > > > From: Shameer Kolothum
> <shameerali.kolothum.thodi@huawei.com>
> > > > > > > Sent: Wednesday, September 15, 2021 5:51 PM
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > Thanks to the introduction of vfio_pci_core subsystem
> framework[0],
> > > > > > > now it is possible to provide vendor specific functionality to
> > > > > > > vfio pci devices. This series attempts to add vfio live migration
> > > > > > > support for HiSilicon ACC VF devices based on the new framework.
> > > > > > >
> > > > > > > HiSilicon ACC VF device MMIO space includes both the functional
> > > > > > > register space and migration control register space. As discussed
> > > > > > > in RFCv1[1], this may create security issues as these regions get
> > > > > > > shared between the Guest driver and the migration driver.
> > > > > > > Based on the feedback, we tried to address those concerns in
> > > > > > > this version.
> > > > > >
> > > > > > This series doesn't mention anything related to dirty page tracking.
> > > > > > Are you rely on Keqian's series for utilizing hardware iommu dirty
> > > > > > bit (e.g. SMMU HTTU)?
> > > > >
> > > > > Yes, this doesn't have dirty page tracking and the plan is to make use
> of
> > > > > Keqian's SMMU HTTU work to improve performance. We have done
> > basic
> > > > > sanity testing with those patches.
> > > > >
> > > >
> > > > Do you plan to support migration w/o HTTU as the fallback option?
> > > > Generally one would expect the basic functionality ready before talking
> > > > about optimization.
> > >
> > > Yes, the plan is to get the basic live migration working and then we can
> > > optimize
> > > it with SMMU HTTU when it is available.
> >
> > The interesting thing is that w/o HTTU vfio will just report every pinned
> > page as dirty, i.e. the entire guest memory is dirty. This completely kills
> > the benefit of precopy phase since Qemu still needs to transfer the entire
> > guest memory in the stop-copy phase. This is not a 'working' model for
> > live migration.
> >
> > So it needs to be clear whether HTTU is really an optimization or
> > a hard functional-requirement for migrating such device. If the latter
> > the migration region info is not a nice-to-have thing.
>
> Yes, agree that we have to transfer the entire Guest memory in this case.
> But don't think that is a killer here as we would still like to have the
> basic live migration enabled on these platforms and can be used
> where the constraints of memory transfer is acceptable.
>
> > btw the fallback option that I raised earlier is more like some software
> > mitigation for collecting dirty pages, e.g. analyzing the ring descriptors
> > to build software-tracked dirty info by mediating the cmd portal
> > (which requires dynamically unmapping cmd portal from the fast-path
> > to enable mediation). We are looking into this option for some platform
> > which lacks of IOMMU dirty bit support.
>
> Interesting. Is there anything available publicly so that we can take a look?
>
Not yet. Once we had an implementation based on an old approach before
vfio-pci-core is ready. Now suppose it needs rework based on the new
framework.
+Shaopeng who owns this work.
Thanks
Kevin
prev parent reply other threads:[~2021-10-15 5:54 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-15 9:50 [PATCH v3 0/6] vfio/hisilicon: add acc live migration driver Shameer Kolothum
2021-09-15 9:50 ` [PATCH v3 1/6] crypto: hisilicon/qm: Move the QM header to include/linux Shameer Kolothum
2021-09-15 12:49 ` Jason Gunthorpe
2021-09-15 9:50 ` [PATCH v3 2/6] crypto: hisilicon/qm: Move few definitions to common header Shameer Kolothum
2021-09-15 9:50 ` [PATCH v3 3/6] hisi_acc_qm: Move PCI device IDs " Shameer Kolothum
2021-09-22 15:11 ` Max Gurtovoy
2021-09-24 8:18 ` Shameerali Kolothum Thodi
2021-09-15 9:50 ` [PATCH v3 4/6] hisi-acc-vfio-pci: add new vfio_pci driver for HiSilicon ACC devices Shameer Kolothum
2021-09-15 12:51 ` Jason Gunthorpe
2021-09-15 13:35 ` Shameerali Kolothum Thodi
2021-09-15 9:50 ` [PATCH v3 5/6] hisi_acc_vfio_pci: Restrict access to VF dev BAR2 migration region Shameer Kolothum
2021-09-15 9:50 ` [PATCH v3 6/6] hisi_acc_vfio_pci: Add support for VFIO live migration Shameer Kolothum
2021-09-15 13:07 ` Jason Gunthorpe
2021-09-15 13:28 ` Shameerali Kolothum Thodi
2021-09-16 13:58 ` Jason Gunthorpe
2021-09-27 13:46 ` Shameerali Kolothum Thodi
2021-09-27 15:01 ` Jason Gunthorpe
2021-09-27 15:27 ` Shameerali Kolothum Thodi
2021-09-27 16:00 ` Leon Romanovsky
2021-09-27 16:06 ` Jason Gunthorpe
2021-09-27 18:17 ` Leon Romanovsky
2021-09-27 18:22 ` Jason Gunthorpe
2021-09-27 18:30 ` Leon Romanovsky
2021-09-27 18:32 ` Jason Gunthorpe
2021-09-29 3:57 ` [PATCH v3 0/6] vfio/hisilicon: add acc live migration driver Tian, Kevin
2021-09-29 8:34 ` Shameerali Kolothum Thodi
2021-09-29 9:05 ` Tian, Kevin
2021-09-29 9:16 ` Shameerali Kolothum Thodi
2021-09-30 0:42 ` Tian, Kevin
2021-09-30 6:34 ` Shameerali Kolothum Thodi
2021-10-15 5:53 ` Tian, Kevin [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=BN9PR11MB5433164D17BB4C30EAEAC41F8CB99@BN9PR11MB5433.namprd11.prod.outlook.com \
--to=kevin.tian@intel.com \
--cc=alex.williamson@redhat.com \
--cc=jgg@nvidia.com \
--cc=jonathan.cameron@huawei.com \
--cc=kvm@vger.kernel.org \
--cc=linux-crypto@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linuxarm@huawei.com \
--cc=liulongfang@huawei.com \
--cc=mgurtovoy@nvidia.com \
--cc=prime.zeng@hisilicon.com \
--cc=shameerali.kolothum.thodi@huawei.com \
--cc=shaopeng.he@intel.com \
--cc=wangzhou1@hisilicon.com \
--cc=yan.y.zhao@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).