All of lore.kernel.org
 help / color / mirror / Atom feed
From: Alex Williamson <alex.williamson@redhat.com>
To: Xiang Zheng <zhengxiang9@huawei.com>
Cc: Kirti Wankhede <kwankhede@nvidia.com>, <cjia@nvidia.com>,
	<kevin.tian@intel.com>, <ziye.yang@intel.com>,
	<changpeng.liu@intel.com>, <yi.l.liu@intel.com>,
	<mlevitsk@redhat.com>, <eskultet@redhat.com>, <cohuck@redhat.com>,
	<dgilbert@redhat.com>, <jonathan.davies@nutanix.com>,
	<eauger@redhat.com>, <aik@ozlabs.ru>, <pasic@linux.ibm.com>,
	<felipe@nutanix.com>, <Zhengxiao.zx@Alibaba-inc.com>,
	<shuangtai.tst@alibaba-inc.com>, <Ken.Xue@amd.com>,
	<zhi.a.wang@intel.com>, <yan.y.zhao@intel.com>,
	<qemu-devel@nongnu.org>, <kvm@vger.kernel.org>,
	"Wang Haibin" <wanghaibin.wang@huawei.com>
Subject: Re: [PATCH Kernel v24 0/8] Add UAPIs to support migration for VFIO devices
Date: Tue, 21 Jul 2020 16:43:04 -0600	[thread overview]
Message-ID: <20200721164304.0ce76b2e@x1.home> (raw)
In-Reply-To: <450612c3-2a92-9034-7958-ee7f3c1a8c52@huawei.com>

On Tue, 21 Jul 2020 10:43:21 +0800
Xiang Zheng <zhengxiang9@huawei.com> wrote:

> Hi Kirti,
> 
> Sorry to disturb you since this patch set has been merged, and I cannot
> receive the qemu-side emails about this patch set.
> 
> We are going to support migration for VFIO devices which support dirty
> pages tracking.
> 
> And we also plan to leverage SMMU HTTU feature to do the dirty pages
> tracking for the devices which don't support dirty pages tracking.
> 
> For the above two cases, which side determines to choose IOMMU driver or
> vendor driver to do dirty bitmap tracking, Qemu or VFIO?
> 
> In brief, if both IOMMU and VFIO devices support dirty pages tracking,
> we can check the capability and prefer to track dirty pages on device
> vendor driver which is more efficient.
> 
> The qusetion is which side to do the check and selection? In my opinion,
> Qemu/userspace seems more suitable.

Dirty page tracking is consolidated at the vfio container level.
Userspace has no basis for determining or interface for selecting a
dirty bitmap provider, so I would disagree that QEMU should play any
role here.  The container dirty bitmap tries to provide the finest
granularity available based on the support of all the devices/groups
managed by the container.  If there are groups attached to the
container that have not participated in page pinning, then we consider
all DMA mappings within the container as persistently dirty.  Once all
of the participants subscribe to page pinning, the dirty scope is
reduced to the pinned pages.  IOMMU support for dirty page logging would
introduce finer granularity yet, which we would probably prefer over
page pinning, but interfaces for this have not been devised.

Ideally userspace should be unaware of any of this, the benefit would
be seen transparently by having a more sparsely filled dirty bitmap,
which more accurately reflects how memory is actually being dirtied.
Thanks,

Alex


WARNING: multiple messages have this Message-ID (diff)
From: Alex Williamson <alex.williamson@redhat.com>
To: Xiang Zheng <zhengxiang9@huawei.com>
Cc: cjia@nvidia.com, kvm@vger.kernel.org, aik@ozlabs.ru,
	Zhengxiao.zx@Alibaba-inc.com, shuangtai.tst@alibaba-inc.com,
	qemu-devel@nongnu.org, Kirti Wankhede <kwankhede@nvidia.com>,
	eauger@redhat.com, yi.l.liu@intel.com, eskultet@redhat.com,
	ziye.yang@intel.com, mlevitsk@redhat.com, pasic@linux.ibm.com,
	felipe@nutanix.com, Wang Haibin <wanghaibin.wang@huawei.com>,
	zhi.a.wang@intel.com, kevin.tian@intel.com, yan.y.zhao@intel.com,
	dgilbert@redhat.com, changpeng.liu@intel.com, cohuck@redhat.com,
	Ken.Xue@amd.com, jonathan.davies@nutanix.com
Subject: Re: [PATCH Kernel v24 0/8] Add UAPIs to support migration for VFIO devices
Date: Tue, 21 Jul 2020 16:43:04 -0600	[thread overview]
Message-ID: <20200721164304.0ce76b2e@x1.home> (raw)
In-Reply-To: <450612c3-2a92-9034-7958-ee7f3c1a8c52@huawei.com>

On Tue, 21 Jul 2020 10:43:21 +0800
Xiang Zheng <zhengxiang9@huawei.com> wrote:

> Hi Kirti,
> 
> Sorry to disturb you since this patch set has been merged, and I cannot
> receive the qemu-side emails about this patch set.
> 
> We are going to support migration for VFIO devices which support dirty
> pages tracking.
> 
> And we also plan to leverage SMMU HTTU feature to do the dirty pages
> tracking for the devices which don't support dirty pages tracking.
> 
> For the above two cases, which side determines to choose IOMMU driver or
> vendor driver to do dirty bitmap tracking, Qemu or VFIO?
> 
> In brief, if both IOMMU and VFIO devices support dirty pages tracking,
> we can check the capability and prefer to track dirty pages on device
> vendor driver which is more efficient.
> 
> The qusetion is which side to do the check and selection? In my opinion,
> Qemu/userspace seems more suitable.

Dirty page tracking is consolidated at the vfio container level.
Userspace has no basis for determining or interface for selecting a
dirty bitmap provider, so I would disagree that QEMU should play any
role here.  The container dirty bitmap tries to provide the finest
granularity available based on the support of all the devices/groups
managed by the container.  If there are groups attached to the
container that have not participated in page pinning, then we consider
all DMA mappings within the container as persistently dirty.  Once all
of the participants subscribe to page pinning, the dirty scope is
reduced to the pinned pages.  IOMMU support for dirty page logging would
introduce finer granularity yet, which we would probably prefer over
page pinning, but interfaces for this have not been devised.

Ideally userspace should be unaware of any of this, the benefit would
be seen transparently by having a more sparsely filled dirty bitmap,
which more accurately reflects how memory is actually being dirtied.
Thanks,

Alex



  reply	other threads:[~2020-07-21 22:43 UTC|newest]

Thread overview: 42+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-05-28 20:30 [PATCH Kernel v24 0/8] Add UAPIs to support migration for VFIO devices Kirti Wankhede
2020-05-28 20:30 ` Kirti Wankhede
2020-05-28 20:30 ` [PATCH Kernel v24 1/8] vfio: UAPI for migration interface for device state Kirti Wankhede
2020-05-28 20:30   ` Kirti Wankhede
2020-09-28 16:51   ` Stefan Hajnoczi
2020-09-28 16:51     ` Stefan Hajnoczi
2020-05-28 20:30 ` [PATCH Kernel v24 2/8] vfio iommu: Remove atomicity of ref_count of pinned pages Kirti Wankhede
2020-05-28 20:30   ` Kirti Wankhede
2020-05-28 20:30 ` [PATCH Kernel v24 3/8] vfio iommu: Cache pgsize_bitmap in struct vfio_iommu Kirti Wankhede
2020-05-28 20:30   ` Kirti Wankhede
2020-05-28 20:30 ` [PATCH Kernel v24 4/8] vfio iommu: Add ioctl definition for dirty pages tracking Kirti Wankhede
2020-05-28 20:30   ` Kirti Wankhede
2020-09-28 17:06   ` Stefan Hajnoczi
2020-09-28 17:06     ` Stefan Hajnoczi
2020-05-28 20:30 ` [PATCH Kernel v24 5/8] vfio iommu: Implementation of ioctl " Kirti Wankhede
2020-05-28 20:30   ` Kirti Wankhede
2020-05-31  5:51   ` kbuild test robot
2020-05-31  9:53   ` kbuild test robot
2020-06-02 15:51   ` Kirti Wankhede
2020-06-02 15:51     ` Kirti Wankhede
2020-06-02 16:44     ` Alex Williamson
2020-06-02 16:44       ` Alex Williamson
2020-05-28 20:30 ` [PATCH Kernel v24 6/8] vfio iommu: Update UNMAP_DMA ioctl to get dirty bitmap before unmap Kirti Wankhede
2020-05-28 20:30   ` Kirti Wankhede
2020-05-28 20:30 ` [PATCH Kernel v24 7/8] vfio iommu: Add migration capability to report supported features Kirti Wankhede
2020-05-28 20:30   ` Kirti Wankhede
2020-05-28 20:30 ` [PATCH Kernel v24 8/8] vfio: Selective dirty page tracking if IOMMU backed device pins pages Kirti Wankhede
2020-05-28 20:30   ` Kirti Wankhede
2020-05-29 21:09 ` [PATCH Kernel v24 0/8] Add UAPIs to support migration for VFIO devices Alex Williamson
2020-05-29 21:09   ` Alex Williamson
2020-07-21  2:43 ` Xiang Zheng
2020-07-21  2:43   ` Xiang Zheng
2020-07-21 22:43   ` Alex Williamson [this message]
2020-07-21 22:43     ` Alex Williamson
2020-07-22  2:55     ` Xiang Zheng
2020-07-22  2:55       ` Xiang Zheng
2020-07-22  3:24       ` Tian, Kevin
2020-07-22  3:24         ` Tian, Kevin
2020-09-29  8:27 ` Stefan Hajnoczi
2020-09-29  8:27   ` Stefan Hajnoczi
2020-09-29 19:52   ` Alex Williamson
2020-09-29 19:52     ` Alex Williamson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200721164304.0ce76b2e@x1.home \
    --to=alex.williamson@redhat.com \
    --cc=Ken.Xue@amd.com \
    --cc=Zhengxiao.zx@Alibaba-inc.com \
    --cc=aik@ozlabs.ru \
    --cc=changpeng.liu@intel.com \
    --cc=cjia@nvidia.com \
    --cc=cohuck@redhat.com \
    --cc=dgilbert@redhat.com \
    --cc=eauger@redhat.com \
    --cc=eskultet@redhat.com \
    --cc=felipe@nutanix.com \
    --cc=jonathan.davies@nutanix.com \
    --cc=kevin.tian@intel.com \
    --cc=kvm@vger.kernel.org \
    --cc=kwankhede@nvidia.com \
    --cc=mlevitsk@redhat.com \
    --cc=pasic@linux.ibm.com \
    --cc=qemu-devel@nongnu.org \
    --cc=shuangtai.tst@alibaba-inc.com \
    --cc=wanghaibin.wang@huawei.com \
    --cc=yan.y.zhao@intel.com \
    --cc=yi.l.liu@intel.com \
    --cc=zhengxiang9@huawei.com \
    --cc=zhi.a.wang@intel.com \
    --cc=ziye.yang@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.