From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40424) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c9U2z-0001h4-JJ for qemu-devel@nongnu.org; Wed, 23 Nov 2016 04:40:59 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1c9U2w-0005lQ-EZ for qemu-devel@nongnu.org; Wed, 23 Nov 2016 04:40:57 -0500 Received: from mail-wj0-x243.google.com ([2a00:1450:400c:c01::243]:33616) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1c9U2w-0005kx-6t for qemu-devel@nongnu.org; Wed, 23 Nov 2016 04:40:54 -0500 Received: by mail-wj0-x243.google.com with SMTP id kp2so605115wjc.0 for ; Wed, 23 Nov 2016 01:40:53 -0800 (PST) Date: Wed, 23 Nov 2016 09:40:50 +0000 From: Stefan Hajnoczi Message-ID: <20161123094050.GC20034@stefanha-x1.localdomain> References: <2fb12281-1023-71c0-7fd9-39e27787c1e9@virtuozzo.com> <6602e519-1d25-86be-855e-d29155ec267c@redhat.com> <40ca4b89-f7bb-6b03-3bd2-0b177d2359ab@redhat.com> <4a811aba-6ede-ee4e-346a-52700d045206@redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="c3bfwLpm8qysLVxt" Content-Disposition: inline In-Reply-To: <4a811aba-6ede-ee4e-346a-52700d045206@redhat.com> Subject: Re: [Qemu-devel] [RFC] dirty bitmap state uncertainty under certain conditions List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: John Snow Cc: Eric Blake , Nikolay Shirokovskiy , qemu-devel@nongnu.org, Jeff Cody , Vladimir Sementsov-Ogievskiy , Denis Lunev , Maxim Nestratov --c3bfwLpm8qysLVxt Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Nov 22, 2016 at 12:26:34PM -0500, John Snow wrote: >=20 >=20 > On 11/22/2016 11:16 AM, Eric Blake wrote: > > On 11/22/2016 10:07 AM, John Snow wrote: > > >=20 > > >=20 > > > On 11/22/2016 07:01 AM, Nikolay Shirokovskiy wrote: > > > > Hi, everyone. > > > >=20 > > > > There is a problem with current incremental backups. Imagine I ask > > > > qemu to > > > > make an incremental backup then go away and return back when backup > > > > job is finished. Qemu process dismisses the job completely and I mi= ssed > > > > all the events so I don't know the result of the operation and what= is > > > > most important I don't know the base for dirty bitmap now. In case = of > > > > failure > > > > it is previous backup and in case of success it is the last backup. > > > > Qemu does > > > > not track dirty bitmap base for me so I have no choice other then c= lear > > > > dirty bitmap and make full backup which would be rather unexpected > > > > from user > > > > POV (The situation of going away/coming back is libvirt crash/resta= rt > > > > of course.) > > > >=20 > > >=20 > > > Why was the completion/failure event missed? Is there some reason why > > > you cannot guarantee that you will observe the completion? > >=20 > > I think the intent of some of the on-error parameters is to make it so > > that the job can't go away on error, only on success. Admittedly, > > libvirt isn't using those policies as well as it could. > >=20 > > >=20 > > > > I guess problem has wider scope. In case I miss successfull > > > > completion of full > > > > backup my only option is to drop backup file and redo the backup > > > > completely > > > > which is rather wasteful. AFAIU I can not query backup completion > > > > result from > > > > backup file itself. I guess there can be similar issues for other q= emu > > > > jobs. > > > >=20 > > > > Nikolay > > > >=20 > > >=20 > > > I would personally advocate for a job-neutral solution where jobs can= be > > > given a parameter such that the job persists in memory in a new > > > "completed" state until such time that it is queried explicitly, then= it > > > can be dropped. > > >=20 > > > I am not sure if we can make this the default behavior, as it might > > > confuse libvirt to occasionally see jobs that have already completed. > > >=20 > > > Talking to Kevin off-list, he suggested that we might be able to make > > > this the default behavior if we pivot to the new jobs API that I have > > > been proposing, accompanied by a new explicit command to put a command > > > to rest. > >=20 > > Yeah, revisiting the overall job API will require some overhaul in > > libvirt as well, but it is probably worth it. > >=20 >=20 > I wonder if I should try to rectify this temporarily for 2.9, or just jump > straight into a new interface. I suggest drafting the "proper" API fix. If it turns out to be a major undertaking then maybe a sub-problem can be solved more easily instead. But attacking the full problem first seems like a good approach - the QEMU 2.9 development cycle hasn't even opened yet :). Stefan --c3bfwLpm8qysLVxt Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJYNWQiAAoJEJykq7OBq3PIB0oIAKrJ1D6e3gZSGw6aqvTDCtpB 8DCHmI4j63sfYMrIk/PDtOupqiw+XHAnExV8zjuoTczri2FkqRsYFGUGfu8tK2Hf RFPFLhDtnJTggomdldOa6NKRZVXgO4i/+j9TozUIQSf1NVhT2VJhenQEL5EEDPpU J/UBvC2lhhusFLEHYuEP/hnZ+K7DkOXSGdWx1xlRP5/qFaevFuQ4c59jtzlPhAwI hlST3rmKNhP0OemlFwWynVFqpEJOGJAvDMIr6r98eH/DIWegzlaRgRGwFEGD2OW2 IHmTWya0tHSBxaw4T2uKfLZ505A7fjstJbqmmc5Yx8SrFEJq44l3KDebk2++VP0= =4GDj -----END PGP SIGNATURE----- --c3bfwLpm8qysLVxt--