From: Alex Williamson <alex.williamson@redhat.com>
To: Yan Zhao <yan.y.zhao@intel.com>
Cc: "Dr. David Alan Gilbert" <dgilbert@redhat.com>,
Kirti Wankhede <kwankhede@nvidia.com>,
cjia@nvidia.com, kevin.tian@intel.com, ziye.yang@intel.com,
changpeng.liu@intel.com, yi.l.liu@intel.com, mlevitsk@redhat.com,
eskultet@redhat.com, cohuck@redhat.com,
jonathan.davies@nutanix.com, eauger@redhat.com, aik@ozlabs.ru,
pasic@linux.ibm.com, felipe@nutanix.com,
Zhengxiao.zx@alibaba-inc.com, shuangtai.tst@alibaba-inc.com,
Ken.Xue@amd.com, zhi.a.wang@intel.com, qemu-devel@nongnu.org,
kvm@vger.kernel.org
Subject: Re: [PATCH Kernel v22 0/8] Add UAPIs to support migration for VFIO devices
Date: Thu, 28 May 2020 16:53:40 -0600 [thread overview]
Message-ID: <20200528165340.53a57e22@x1.home> (raw)
In-Reply-To: <20200528080101.GD1378@joy-OptiPlex-7040>
On Thu, 28 May 2020 04:01:02 -0400
Yan Zhao <yan.y.zhao@intel.com> wrote:
> > > > This is my understanding of the protocol as well, when the device is
> > > > running, pending_bytes might drop to zero if no internal state has
> > > > changed and may be non-zero on the next iteration due to device
> > > > activity. When the device is not running, pending_bytes reporting zero
> > > > indicates the device is done, there is no further state to transmit.
> > > > Does that meet your need/expectation?
> > > >
> > > (1) on one side, as in vfio_save_pending(),
> > > vfio_save_pending()
> > > {
> > > ...
> > > ret = vfio_update_pending(vbasedev);
> > > ...
> > > *res_precopy_only += migration->pending_bytes;
> > > ...
> > > }
> > > the pending_bytes tells migration thread how much data is still hold in
> > > device side.
> > > the device data includes
> > > device internal data + running device dirty data + device state.
> > >
> > > so the pending_bytes should include device state as well, right?
> > > if so, the pending_bytes should never reach 0 if there's any device
> > > state to be sent after device is stopped.
> >
> > I hadn't expected the pending-bytes to include a fixed offset for device
> > state (If you mean a few registers etc) - I'd expect pending to drop
> > possibly to zero; the heuristic as to when to switch from iteration to
> > stop, is based on the total pending across all iterated devices; so it's
> > got to be allowed to drop otherwise you'll never transition to stop.
> >
> ok. got it.
Yeah, as I understand it, a device is not required to participate in
reporting data available while (_SAVING | _RUNNING), there will always
be an iteration while the device is !_RUNNING. Therefore if you have
fixed device state that you're always going to send, it should only be
sent once when called during !_RUNNING. The iterative phase should be
used where you have a good chance to avoid re-sending data at the
stop-and-copy phase. Thanks,
Alex
next prev parent reply other threads:[~2020-05-28 22:53 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-18 5:56 [PATCH Kernel v22 0/8] Add UAPIs to support migration for VFIO devices Kirti Wankhede
2020-05-18 5:56 ` [PATCH Kernel v22 1/8] vfio: UAPI for migration interface for device state Kirti Wankhede
2020-05-18 5:56 ` [PATCH Kernel v22 2/8] vfio iommu: Remove atomicity of ref_count of pinned pages Kirti Wankhede
2020-05-18 5:56 ` [PATCH Kernel v22 3/8] vfio iommu: Cache pgsize_bitmap in struct vfio_iommu Kirti Wankhede
2020-05-20 10:08 ` Cornelia Huck
2020-05-20 14:46 ` Kirti Wankhede
2020-05-18 5:56 ` [PATCH Kernel v22 4/8] vfio iommu: Add ioctl definition for dirty pages tracking Kirti Wankhede
2020-05-18 5:56 ` [PATCH Kernel v22 5/8] vfio iommu: Implementation of ioctl " Kirti Wankhede
2020-05-18 21:53 ` Alex Williamson
2020-05-19 7:11 ` Kirti Wankhede
2020-05-19 6:52 ` Kirti Wankhede
2020-05-18 5:56 ` [PATCH Kernel v22 6/8] vfio iommu: Update UNMAP_DMA ioctl to get dirty bitmap before unmap Kirti Wankhede
2020-05-19 6:54 ` Kirti Wankhede
2020-05-20 10:27 ` Cornelia Huck
2020-05-20 15:16 ` Kirti Wankhede
2020-05-18 5:56 ` [PATCH Kernel v22 7/8] vfio iommu: Add migration capability to report supported features Kirti Wankhede
2020-05-20 10:42 ` Cornelia Huck
2020-05-20 15:23 ` Kirti Wankhede
2020-05-18 5:56 ` [PATCH Kernel v22 8/8] vfio: Selective dirty page tracking if IOMMU backed device pins pages Kirti Wankhede
2020-05-19 6:54 ` Kirti Wankhede
2020-05-19 16:58 ` [PATCH Kernel v22 0/8] Add UAPIs to support migration for VFIO devices Alex Williamson
2020-05-20 2:55 ` Yan Zhao
2020-05-20 13:40 ` Kirti Wankhede
2020-05-20 16:46 ` Alex Williamson
2020-05-21 5:08 ` Yan Zhao
2020-05-21 7:09 ` Kirti Wankhede
2020-05-21 7:04 ` Yan Zhao
2020-05-21 7:28 ` Kirti Wankhede
2020-05-21 7:32 ` Kirti Wankhede
2020-05-25 6:59 ` Yan Zhao
2020-05-25 13:20 ` Kirti Wankhede
2020-05-26 20:19 ` Alex Williamson
2020-05-27 6:23 ` Yan Zhao
2020-05-27 8:48 ` Dr. David Alan Gilbert
2020-05-28 8:01 ` Yan Zhao
2020-05-28 22:53 ` Alex Williamson [this message]
2020-05-29 11:12 ` Dr. David Alan Gilbert
2020-05-28 22:59 ` Alex Williamson
2020-05-29 4:15 ` Yan Zhao
2020-05-29 17:57 ` Kirti Wankhede
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200528165340.53a57e22@x1.home \
--to=alex.williamson@redhat.com \
--cc=Ken.Xue@amd.com \
--cc=Zhengxiao.zx@alibaba-inc.com \
--cc=aik@ozlabs.ru \
--cc=changpeng.liu@intel.com \
--cc=cjia@nvidia.com \
--cc=cohuck@redhat.com \
--cc=dgilbert@redhat.com \
--cc=eauger@redhat.com \
--cc=eskultet@redhat.com \
--cc=felipe@nutanix.com \
--cc=jonathan.davies@nutanix.com \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=kwankhede@nvidia.com \
--cc=mlevitsk@redhat.com \
--cc=pasic@linux.ibm.com \
--cc=qemu-devel@nongnu.org \
--cc=shuangtai.tst@alibaba-inc.com \
--cc=yan.y.zhao@intel.com \
--cc=yi.l.liu@intel.com \
--cc=zhi.a.wang@intel.com \
--cc=ziye.yang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).