On 11/22/2016 10:07 AM, John Snow wrote:
> 
> 
> On 11/22/2016 07:01 AM, Nikolay Shirokovskiy wrote:
>> Hi, everyone.
>>
>>   There is a problem with current incremental backups. Imagine I ask
>> qemu to
>> make an incremental backup then go away and return back when backup
>> job is finished. Qemu process dismisses the job completely and I missed
>> all the events so I don't know the result of the operation and what is
>> most important I don't know the base for dirty bitmap now. In case of
>> failure
>> it is previous backup and in case of success it is the last backup.
>> Qemu does
>> not track dirty bitmap base for me so I have no choice other then clear
>> dirty bitmap and make full backup which would be rather unexpected
>> from user
>> POV (The situation of going away/coming back is libvirt crash/restart
>> of course.)
>>
> 
> Why was the completion/failure event missed? Is there some reason why
> you cannot guarantee that you will observe the completion?

I think the intent of some of the on-error parameters is to make it so
that the job can't go away on error, only on success.  Admittedly,
libvirt isn't using those policies as well as it could.

> 
>>   I guess problem has wider scope. In case I miss successfull
>> completion of full
>> backup my only option is to drop backup file and redo the backup
>> completely
>> which is rather wasteful. AFAIU I can not query backup completion
>> result from
>> backup file itself. I guess there can be similar issues for other qemu
>> jobs.
>>
>> Nikolay
>>
> 
> I would personally advocate for a job-neutral solution where jobs can be
> given a parameter such that the job persists in memory in a new
> "completed" state until such time that it is queried explicitly, then it
> can be dropped.
> 
> I am not sure if we can make this the default behavior, as it might
> confuse libvirt to occasionally see jobs that have already completed.
> 
> Talking to Kevin off-list, he suggested that we might be able to make
> this the default behavior if we pivot to the new jobs API that I have
> been proposing, accompanied by a new explicit command to put a command
> to rest.

Yeah, revisiting the overall job API will require some overhaul in
libvirt as well, but it is probably worth it.

> 
> I can work on this for 2.9; though we may still need a "temporary"
> solution for the old jobs API until we're ready to officially deprecate
> the older interface.
> 
> 

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org