From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40605) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YV8aq-0002v4-HE for qemu-devel@nongnu.org; Mon, 09 Mar 2015 21:04:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YV8ao-0003ZP-Nw for qemu-devel@nongnu.org; Mon, 09 Mar 2015 21:04:20 -0400 Received: from ozlabs.org ([2401:3900:2:1::2]:59768) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YV8ao-0003YT-4z for qemu-devel@nongnu.org; Mon, 09 Mar 2015 21:04:18 -0400 Date: Tue, 10 Mar 2015 12:04:56 +1100 From: David Gibson Message-ID: <20150310010456.GC30335@voom.fritz.box> References: <1424883128-9841-1-git-send-email-dgilbert@redhat.com> <1424883128-9841-2-git-send-email-dgilbert@redhat.com> <20150305032119.GK18072@voom.fritz.box> <20150305092138.GA2381@work-vm> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="1SQmhf2mF2YjsYvc" Content-Disposition: inline In-Reply-To: <20150305092138.GA2381@work-vm> Subject: Re: [Qemu-devel] [PATCH v5 01/45] Start documenting how postcopy works. List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Dr. David Alan Gilbert" Cc: aarcange@redhat.com, yamahata@private.email.ne.jp, quintela@redhat.com, qemu-devel@nongnu.org, amit.shah@redhat.com, pbonzini@redhat.com, yanghy@cn.fujitsu.com --1SQmhf2mF2YjsYvc Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Mar 05, 2015 at 09:21:39AM +0000, Dr. David Alan Gilbert wrote: > * David Gibson (david@gibson.dropbear.id.au) wrote: > > On Wed, Feb 25, 2015 at 04:51:24PM +0000, Dr. David Alan Gilbert > (git) wrote: [snip] > > > +=3D=3D=3D Enabling postcopy =3D=3D=3D > > > + > > > +To enable postcopy (prior to the start of migration): > > > + > > > +migrate_set_capability x-postcopy-ram on > > > + > > > +The migration will still start in precopy mode, however issuing: > > > + > > > +migrate_start_postcopy > > > + > > > +will now cause the transition from precopy to postcopy. > > > +It can be issued immediately after migration is started or any > > > +time later on. Issuing it after the end of a migration is harmless. > >=20 > > It's not quite clear to me what this means. Does > > "migrate_start_postcopy" mean it will immediately transfer execution > > and transfer any remaining pages postcopy, or does it just mean it > > will start postcopying once the remaining data to transfer is small > > enough? >=20 > Yes; it will flip into postcopy soon after issuing that command irrespect= ive > of the amount of data remaining. >=20 > > What's the reason for this rather awkward two stage activation of > > postcopy? >=20 > We need to keep track of the pages that are received during the precopy p= hase, > and do some madvise and other setups on the destination RAM area before p= recopy > starts; and so we need to know we might want to do postcopy - so we need > to be told early. In the earliest posted version of my patches I had a > time-limit setting and after the time limit expired QEMU would switch into > the second phase of postcopy itself, but Paolo suggested the migrate_star= t_postcopy: >=20 > https://lists.nongnu.org/archive/html/qemu-devel/2014-07/msg00943.html >=20 > and it works out simpler anyway. Ok, that makes sense. > > > +=3D=3D=3D Postcopy device transfer =3D=3D=3D > > > + > > > +Loading of device data may cause the device emulation to access gues= t RAM > > > +that may trigger faults that have to be resolved by the source, as s= uch > > > +the migration stream has to be able to respond with page data *durin= g* the > > > +device load, and hence the device data has to be read from the strea= m completely > > > +before the device load begins to free the stream up. This is achiev= ed by > > > +'packaging' the device data into a blob that's read in one go. > > > + > > > +Source behaviour > > > + > > > +Until postcopy is entered the migration stream is identical to normal > > > +precopy, except for the addition of a 'postcopy advise' command at > > > +the beginning, to tell the destination that postcopy might happen. > > > +When postcopy starts the source sends the page discard data and then > > > +forms the 'package' containing: > > > + > > > + Command: 'postcopy ram listen' > > > + The device state > > > + A series of sections, identical to the precopy streams device = state stream > > > + containing everything except postcopiable devices (i.e. RAM) > > > + Command: 'postcopy ram run' > > > + > > > +The 'package' is sent as the data part of a Command: 'CMD_PACKAGED',= and the > > > +contents are formatted in the same way as the main migration stream. > >=20 > > It seems to me the "ram listen", "ram run" and CMD_PACKAGED really > > have to be used in conjuction this way, they don't really have any use > > on their own. So why not make it all CMD_POSTCOPY_TRANSITION and have > > the "listen" and "run" take effect implicitly at the beginning and end > > of the device data. >=20 > CMD_PACKAGED seems like something that was generally useful; it's fairly > complicated on it's own and so it seemed best to keep it separate. And can you actually think of another use case for it? The thing that bothers me is that the "listen" and "run" operations will not work correctly anywhere other than at the beginning and end of the packaged blob. --=20 David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson --1SQmhf2mF2YjsYvc Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJU/kM4AAoJEGw4ysog2bOSC0kQALZlTEZMDVRfzL27QgReIjZB RE3VAkNrcszbnUJOgWBkDuGrcKVxR0966D5gX5qaRuqK7munCFlo8/2PQJ29S0iS VnuNZVDTDYqIOesPjC1lyTg2wmPX4gmfaf8W6kwvqe2/H1NzQkjWkJvZwrHhJZkn VNlP9pZCOAcdlss9LvH6GwN7Kk1XoK75HmJltdFztGHHyVqWLEaauhGL/g/Gxukz Uz4WGooDQlM3TKERa3uNmaQMNg0gKfywau3gsBbNFFHGuGSvHtfGP7manwBZTK/i qECMG5BxEfZO/Li3WctPZnCu+bV5O/7PZkQbyqJ48z0tDpU/g6KB6MFbqAHxCOxQ 51FBXhtMTcwAW5XejN1QfPB43gpqthGtxutarV8omhCMj3j/Jr/24KmPyiW/7cSL 7ATPhNq/fEP/Xcgt9kK0anvkCNei1M8q3dn+StNYbYgWhv1KJsn7YrGpzm6zw7lx a2CnyYYMexiValtlDmLN/cR2cSt8+eykFM9/JWCI4YRl5Ddu6HBtQ12PPyW2Z4HN eApLukkG2RZS+pKAVUFiz4txjBrdiidMW/PeyVE1IBS4vKcQVvB2lJcr90D2wDDA OIhey4nUx6DpGAzxPusXS+I9l9XU86/VZlrqRw8jnnTwYNLqpJgp8OTBWh+QL+un 1ttxRwNti429C61XJ4yv =RY05 -----END PGP SIGNATURE----- --1SQmhf2mF2YjsYvc--