From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33687) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c9DkR-0003yI-VK for qemu-devel@nongnu.org; Tue, 22 Nov 2016 11:16:45 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1c9DkN-0005c8-1M for qemu-devel@nongnu.org; Tue, 22 Nov 2016 11:16:43 -0500 Received: from mx1.redhat.com ([209.132.183.28]:57844) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1c9DkM-0005ag-O9 for qemu-devel@nongnu.org; Tue, 22 Nov 2016 11:16:38 -0500 References: <2fb12281-1023-71c0-7fd9-39e27787c1e9@virtuozzo.com> <6602e519-1d25-86be-855e-d29155ec267c@redhat.com> From: Eric Blake Message-ID: <40ca4b89-f7bb-6b03-3bd2-0b177d2359ab@redhat.com> Date: Tue, 22 Nov 2016 10:16:35 -0600 MIME-Version: 1.0 In-Reply-To: <6602e519-1d25-86be-855e-d29155ec267c@redhat.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="09bG8ELeJERNRp5WiERJq3NpggvDldP4V" Subject: Re: [Qemu-devel] [RFC] dirty bitmap state uncertainty under certain conditions List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: John Snow , Nikolay Shirokovskiy , qemu-devel@nongnu.org Cc: Denis Lunev , Vladimir Sementsov-Ogievskiy , Maxim Nestratov , Jeff Cody This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --09bG8ELeJERNRp5WiERJq3NpggvDldP4V From: Eric Blake To: John Snow , Nikolay Shirokovskiy , qemu-devel@nongnu.org Cc: Denis Lunev , Vladimir Sementsov-Ogievskiy , Maxim Nestratov , Jeff Cody Message-ID: <40ca4b89-f7bb-6b03-3bd2-0b177d2359ab@redhat.com> Subject: Re: [RFC] dirty bitmap state uncertainty under certain conditions References: <2fb12281-1023-71c0-7fd9-39e27787c1e9@virtuozzo.com> <6602e519-1d25-86be-855e-d29155ec267c@redhat.com> In-Reply-To: <6602e519-1d25-86be-855e-d29155ec267c@redhat.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 11/22/2016 10:07 AM, John Snow wrote: >=20 >=20 > On 11/22/2016 07:01 AM, Nikolay Shirokovskiy wrote: >> Hi, everyone. >> >> There is a problem with current incremental backups. Imagine I ask >> qemu to >> make an incremental backup then go away and return back when backup >> job is finished. Qemu process dismisses the job completely and I misse= d >> all the events so I don't know the result of the operation and what is= >> most important I don't know the base for dirty bitmap now. In case of >> failure >> it is previous backup and in case of success it is the last backup. >> Qemu does >> not track dirty bitmap base for me so I have no choice other then clea= r >> dirty bitmap and make full backup which would be rather unexpected >> from user >> POV (The situation of going away/coming back is libvirt crash/restart >> of course.) >> >=20 > Why was the completion/failure event missed? Is there some reason why > you cannot guarantee that you will observe the completion? I think the intent of some of the on-error parameters is to make it so that the job can't go away on error, only on success. Admittedly, libvirt isn't using those policies as well as it could. >=20 >> I guess problem has wider scope. In case I miss successfull >> completion of full >> backup my only option is to drop backup file and redo the backup >> completely >> which is rather wasteful. AFAIU I can not query backup completion >> result from >> backup file itself. I guess there can be similar issues for other qemu= >> jobs. >> >> Nikolay >> >=20 > I would personally advocate for a job-neutral solution where jobs can b= e > given a parameter such that the job persists in memory in a new > "completed" state until such time that it is queried explicitly, then i= t > can be dropped. >=20 > I am not sure if we can make this the default behavior, as it might > confuse libvirt to occasionally see jobs that have already completed. >=20 > Talking to Kevin off-list, he suggested that we might be able to make > this the default behavior if we pivot to the new jobs API that I have > been proposing, accompanied by a new explicit command to put a command > to rest. Yeah, revisiting the overall job API will require some overhaul in libvirt as well, but it is probably worth it. >=20 > I can work on this for 2.9; though we may still need a "temporary" > solution for the old jobs API until we're ready to officially deprecate= > the older interface. >=20 >=20 --=20 Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org --09bG8ELeJERNRp5WiERJq3NpggvDldP4V Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCAAGBQJYNG9jAAoJEKeha0olJ0Nq7tgH/10fDG4gvdmzsTnhJdksqL4y Fn+WeCXrZfLt9W0fNtOvwzplQz712eNilTQbI1SrOn+EfulZVzr3MY+CIE4GLtK0 eI+uPluhBbV5oQDE+J6dIpCzQ3Hn/s0eQkIGGOMSy1L3eDWSRl/U+N9g4KbAOMj/ EGoZmqQkvCzelwJRo6mS3tHusoQ4Y48/FVF/Bj3wWcssaPU2kBUWOoHKHuh1Svnc 6dA+0/xo024gq4KmT+BECZrcwpb+rqlaeFMe8Z7NN+AW3h3KMgAWhBw5vRDCnbQX n+j+iD3GvCgiJIOTTTEfXGGDeB+bK8uH/OfOTmGNmI5SWBJWvRtsqsFVSOrcLVY= =shMH -----END PGP SIGNATURE----- --09bG8ELeJERNRp5WiERJq3NpggvDldP4V--