* [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to track dirty sector
@ 2015-03-03 6:52 Zhang Haoyu
2015-03-03 9:59 ` Paolo Bonzini
0 siblings, 1 reply; 11+ messages in thread
From: Zhang Haoyu @ 2015-03-03 6:52 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, Paolo Bonzini, Marcelo Tosatti, Fam Zheng, Stefan Hajnoczi
Hi,
If introducing bitmap to bdrv_commit to track dirty sector,
could we implement guest non-disruption while performing commit?
Thanks,
Zhang Haoyu
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to track dirty sector
2015-03-03 6:52 [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to track dirty sector Zhang Haoyu
@ 2015-03-03 9:59 ` Paolo Bonzini
2015-03-09 7:03 ` Zhang Haoyu
0 siblings, 1 reply; 11+ messages in thread
From: Paolo Bonzini @ 2015-03-03 9:59 UTC (permalink / raw)
To: Zhang Haoyu, qemu-devel
Cc: Kevin Wolf, Marcelo Tosatti, Fam Zheng, Stefan Hajnoczi
On 03/03/2015 07:52, Zhang Haoyu wrote:
> Hi,
> If introducing bitmap to bdrv_commit to track dirty sector,
> could we implement guest non-disruption while performing commit?
That is already implemented. It uses the same code that implements
storage migration (block/mirror.c).
Paolo
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to track dirty sector
2015-03-03 9:59 ` Paolo Bonzini
@ 2015-03-09 7:03 ` Zhang Haoyu
2015-03-09 7:38 ` Paolo Bonzini
2015-03-09 8:14 ` Zhang Haoyu
0 siblings, 2 replies; 11+ messages in thread
From: Zhang Haoyu @ 2015-03-09 7:03 UTC (permalink / raw)
To: Paolo Bonzini, qemu-devel
Cc: Kevin Wolf, Marcelo Tosatti, Fam Zheng, Stefan Hajnoczi
On 2015-03-03 18:00:09, Paolo Bonzini wrote:
>
> On 03/03/2015 07:52, Zhang Haoyu wrote:
> > Hi,
> > If introducing bitmap to bdrv_commit to track dirty sector,
> > could we implement guest non-disruption while performing commit?
>
> That is already implemented. It uses the same code that implements
> storage migration (block/mirror.c).
>
Hi Paolo,
do you mean commit_active_start()?
> Paolo
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to track dirty sector
2015-03-09 7:03 ` Zhang Haoyu
@ 2015-03-09 7:38 ` Paolo Bonzini
2015-03-09 8:14 ` Zhang Haoyu
1 sibling, 0 replies; 11+ messages in thread
From: Paolo Bonzini @ 2015-03-09 7:38 UTC (permalink / raw)
To: Zhang Haoyu, qemu-devel
Cc: Kevin Wolf, Marcelo Tosatti, Fam Zheng, Stefan Hajnoczi
On 09/03/2015 08:03, Zhang Haoyu wrote:
>
> On 2015-03-03 18:00:09, Paolo Bonzini wrote:
>>
>> On 03/03/2015 07:52, Zhang Haoyu wrote:
>>> Hi,
>>> If introducing bitmap to bdrv_commit to track dirty sector,
>>> could we implement guest non-disruption while performing commit?
>>
>> That is already implemented. It uses the same code that implements
>> storage migration (block/mirror.c).
>>
> Hi Paolo,
> do you mean commit_active_start()?
Yes.
Paolo
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to track dirty sector
2015-03-09 7:03 ` Zhang Haoyu
2015-03-09 7:38 ` Paolo Bonzini
@ 2015-03-09 8:14 ` Zhang Haoyu
2015-03-10 0:28 ` Fam Zheng
2015-03-10 1:30 ` [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to trackdirty sector Zhang Haoyu
1 sibling, 2 replies; 11+ messages in thread
From: Zhang Haoyu @ 2015-03-09 8:14 UTC (permalink / raw)
To: Paolo Bonzini, qemu-devel, jsnow, vsementsov
Cc: Kevin Wolf, Marcelo Tosatti, Fam Zheng, Stefan Hajnoczi
Hi John, Vladimir
We can using active block commit to implement incremental backup without guest disruption,
e.g.,
origin <= A <= B <= C <= current BDS,
a new external snapshot will be produced before every time backup,
we can migrate A, B, C, ... to destination,
then commit_active_start() the unneeded snapshot in source or destination end.
So, comparing with above mechanism,
what's the advantages of the incremental backup implemented by John and Vladimir?
Thanks,
Zhang Haoyu
On 2015-03-09 15:38:40, Paolo Bonzini wrote:
>
>On 09/03/2015 08:03, Zhang Haoyu wrote:
>>
>> On 2015-03-03 18:00:09, Paolo Bonzini wrote:
>>>
>>> On 03/03/2015 07:52, Zhang Haoyu wrote:
>>>> Hi,
>>>> If introducing bitmap to bdrv_commit to track dirty sector,
>>>> could we implement guest non-disruption while performing commit?
>>>
>>> That is already implemented. It uses the same code that implements
>>> storage migration (block/mirror.c).
>>>
>> Hi Paolo,
>> do you mean commit_active_start()?
>
>Yes.
>
>Paolo
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to track dirty sector
2015-03-09 8:14 ` Zhang Haoyu
@ 2015-03-10 0:28 ` Fam Zheng
2015-03-10 1:30 ` [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to trackdirty sector Zhang Haoyu
1 sibling, 0 replies; 11+ messages in thread
From: Fam Zheng @ 2015-03-10 0:28 UTC (permalink / raw)
To: Zhang Haoyu
Cc: Kevin Wolf, Marcelo Tosatti, qemu-devel, vsementsov,
Stefan Hajnoczi, Paolo Bonzini, jsnow
On Mon, 03/09 16:14, Zhang Haoyu wrote:
> Hi John, Vladimir
> We can using active block commit to implement incremental backup without guest disruption,
> e.g.,
> origin <= A <= B <= C <= current BDS,
> a new external snapshot will be produced before every time backup,
> we can migrate A, B, C, ... to destination,
> then commit_active_start() the unneeded snapshot in source or destination end.
>
> So, comparing with above mechanism,
> what's the advantages of the incremental backup implemented by John and Vladimir?
We can't migrate A, B, C because they are buried in the backing chain under
"current BDS". Even if we do that, there will be severe IO amplification
compared to the dirty bitmap way.
For the same reason we have drive-backup (write intercept) instead of a
"snapshot + drive-mirror" analogue.
Fam
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to trackdirty sector
2015-03-09 8:14 ` Zhang Haoyu
2015-03-10 0:28 ` Fam Zheng
@ 2015-03-10 1:30 ` Zhang Haoyu
2015-03-10 1:54 ` Fam Zheng
2015-03-10 2:12 ` [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to trackdirtysector Zhang Haoyu
1 sibling, 2 replies; 11+ messages in thread
From: Zhang Haoyu @ 2015-03-10 1:30 UTC (permalink / raw)
To: Fam Zheng
Cc: Kevin Wolf, Marcelo Tosatti, qemu-devel, vsementsov,
Stefan Hajnoczi, Paolo Bonzini, jsnow
On 2015-03-10 08:29:19, Fam Zheng wrote:
> On Mon, 03/09 16:14, Zhang Haoyu wrote:
> > Hi John, Vladimir
> > We can using active block commit to implement incremental backup without guest disruption,
> > e.g.,
> > origin <= A <= B <= C <= current BDS,
> > a new external snapshot will be produced before every time backup,
> > we can migrate A, B, C, ... to destination,
> > then commit_active_start() the unneeded snapshot in source or destination end.
> >
> > So, comparing with above mechanism,
> > what's the advantages of the incremental backup implemented by John and Vladimir?
>
> We can't migrate A, B, C because they are buried in the backing chain under
> "current BDS".
I think we can backup the incremental image(e.g., A, B, C) to destination in top mode,
although I haven't read the code in detail, it can work theoretically, I think.
> Even if we do that, there will be severe IO amplification
> compared to the dirty bitmap way.
>
Yes, block-commit will produce extra IO.
But regarding incremental backup, when guest IO is performed,
will the corresponding dirty bit be synchronized to qcow2 image simultaneously?
If yes, it will impact on IO throughput in two sides, I think,
1) extro IO 2) randomize the IO,
if no, if source VM is shut-down in non-normal way, like killed by force or by mistake or server poweroff suddenly,
some dirty bits maybe lost, then full copy is needed.
> For the same reason we have drive-backup (write intercept) instead of a
> "snapshot + drive-mirror" analogue.
>
drive-backup is not incremental backup, full copy is needed in every time backup,
so it dosen't meet our requirements.
Zhang Haoyu
> Fam
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to trackdirty sector
2015-03-10 1:30 ` [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to trackdirty sector Zhang Haoyu
@ 2015-03-10 1:54 ` Fam Zheng
2015-03-10 2:12 ` [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to trackdirtysector Zhang Haoyu
1 sibling, 0 replies; 11+ messages in thread
From: Fam Zheng @ 2015-03-10 1:54 UTC (permalink / raw)
To: Zhang Haoyu
Cc: Kevin Wolf, Marcelo Tosatti, qemu-devel, vsementsov,
Stefan Hajnoczi, Paolo Bonzini, jsnow
On Tue, 03/10 09:30, Zhang Haoyu wrote:
>
> On 2015-03-10 08:29:19, Fam Zheng wrote:
> > On Mon, 03/09 16:14, Zhang Haoyu wrote:
> > > Hi John, Vladimir
> > > We can using active block commit to implement incremental backup without guest disruption,
> > > e.g.,
> > > origin <= A <= B <= C <= current BDS,
> > > a new external snapshot will be produced before every time backup,
> > > we can migrate A, B, C, ... to destination,
> > > then commit_active_start() the unneeded snapshot in source or destination end.
> > >
> > > So, comparing with above mechanism,
> > > what's the advantages of the incremental backup implemented by John and Vladimir?
> >
> > We can't migrate A, B, C because they are buried in the backing chain under
> > "current BDS".
> I think we can backup the incremental image(e.g., A, B, C) to destination in top mode,
> although I haven't read the code in detail, it can work theoretically, I think.
No, we don't have any command do that.
>
> > Even if we do that, there will be severe IO amplification
> > compared to the dirty bitmap way.
> >
> Yes, block-commit will produce extra IO.
> But regarding incremental backup, when guest IO is performed,
> will the corresponding dirty bit be synchronized to qcow2 image simultaneously?
No, that would have a huge performance penalty. It will only be synced
at shutdown and or periodically, therefore it has the same implications with
other cache, such as page cache or block driver metadata cache.
> if no, if source VM is shut-down in non-normal way, like killed by force or by mistake or server poweroff suddenly,
> some dirty bits maybe lost, then full copy is needed.
Yes, it is a reasonable rule.
> >
> drive-backup is not incremental backup, full copy is needed in every time backup,
> so it dosen't meet our requirements.
I didn't mean drive-backup already provides incremental backup, but we do need
it to implement it (see the patch series posted by John Snow).
Thanks,
Fam
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to trackdirtysector
2015-03-10 1:30 ` [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to trackdirty sector Zhang Haoyu
2015-03-10 1:54 ` Fam Zheng
@ 2015-03-10 2:12 ` Zhang Haoyu
2015-03-10 2:58 ` Fam Zheng
2015-03-10 3:12 ` [Qemu-devel] [RFC] introduce bitmap to bdrv_commit totrackdirtysector Zhang Haoyu
1 sibling, 2 replies; 11+ messages in thread
From: Zhang Haoyu @ 2015-03-10 2:12 UTC (permalink / raw)
To: Fam Zheng
Cc: Kevin Wolf, Marcelo Tosatti, qemu-devel, vsementsov,
Stefan Hajnoczi, Paolo Bonzini, jsnow
On 2015-03-10 09:54:47, Fam Zheng wrote:
> On Tue, 03/10 09:30, Zhang Haoyu wrote:
> >
> > On 2015-03-10 08:29:19, Fam Zheng wrote:
> > > On Mon, 03/09 16:14, Zhang Haoyu wrote:
> > > > Hi John, Vladimir
> > > > We can using active block commit to implement incremental backup without guest disruption,
> > > > e.g.,
> > > > origin <= A <= B <= C <= current BDS,
> > > > a new external snapshot will be produced before every time backup,
> > > > we can migrate A, B, C, ... to destination,
> > > > then commit_active_start() the unneeded snapshot in source or destination end.
> > > >
> > > > So, comparing with above mechanism,
> > > > what's the advantages of the incremental backup implemented by John and Vladimir?
> > >
> > > We can't migrate A, B, C because they are buried in the backing chain under
> > > "current BDS".
> > I think we can backup the incremental image(e.g., A, B, C) to destination in top mode,
> > although I haven't read the code in detail, it can work theoretically, I think.
>
> No, we don't have any command do that.
>
Is it worthy to implement it?
And does qemu support commit any external snapshot to its backing file?
> >
> > > Even if we do that, there will be severe IO amplification
> > > compared to the dirty bitmap way.
> > >
> > Yes, block-commit will produce extra IO.
> > But regarding incremental backup, when guest IO is performed,
> > will the corresponding dirty bit be synchronized to qcow2 image simultaneously?
>
> No, that would have a huge performance penalty. It will only be synced
> at shutdown and or periodically, therefore it has the same implications with
> other cache, such as page cache or block driver metadata cache.
>
> > if no, if source VM is shut-down in non-normal way, like killed by force or by mistake or server poweroff suddenly,
> > some dirty bits maybe lost, then full copy is needed.
>
> Yes, it is a reasonable rule.
>
> > >
> > drive-backup is not incremental backup, full copy is needed in every time backup,
> > so it dosen't meet our requirements.
>
> I didn't mean drive-backup already provides incremental backup, but we do need
> it to implement it (see the patch series posted by John Snow).
>
> Thanks,
> Fam
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to trackdirtysector
2015-03-10 2:12 ` [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to trackdirtysector Zhang Haoyu
@ 2015-03-10 2:58 ` Fam Zheng
2015-03-10 3:12 ` [Qemu-devel] [RFC] introduce bitmap to bdrv_commit totrackdirtysector Zhang Haoyu
1 sibling, 0 replies; 11+ messages in thread
From: Fam Zheng @ 2015-03-10 2:58 UTC (permalink / raw)
To: Zhang Haoyu
Cc: Kevin Wolf, Marcelo Tosatti, qemu-devel, vsementsov,
Stefan Hajnoczi, Paolo Bonzini, jsnow
> Is it worthy to implement it?
Maybe, I think once we start to assign node-name to each BDS in the tree, this
will be natural to implement.
> And does qemu support commit any external snapshot to its backing file?
Yes.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Qemu-devel] [RFC] introduce bitmap to bdrv_commit totrackdirtysector
2015-03-10 2:12 ` [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to trackdirtysector Zhang Haoyu
2015-03-10 2:58 ` Fam Zheng
@ 2015-03-10 3:12 ` Zhang Haoyu
1 sibling, 0 replies; 11+ messages in thread
From: Zhang Haoyu @ 2015-03-10 3:12 UTC (permalink / raw)
To: Fam Zheng
Cc: Kevin Wolf, Marcelo Tosatti, qemu-devel, vsementsov,
Stefan Hajnoczi, Paolo Bonzini, jsnow
On 2015-03-10 10:58:44, Fam Zheng wrote:
> > Is it worthy to implement it?
>
> Maybe, I think once we start to assign node-name to each BDS in the tree, this
> will be natural to implement.
>
Once implemented, to be honest, I really don't see much advantages of
incremental backup over aforementioned mechanism(external snapshot + active block commit),
especially, regarding the case of source vm non-normal shutdown, full copy is needed,
and corresponding check mechanism is needed too.
I think so much exceptional scenarios are needed to be considered for incremental backup mechanism.
Maybe I missed something.
Thanks,
Zhang Haoyu
> > And does qemu support commit any external snapshot to its backing file?
>
> Yes.
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2015-03-10 3:12 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-03-03 6:52 [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to track dirty sector Zhang Haoyu
2015-03-03 9:59 ` Paolo Bonzini
2015-03-09 7:03 ` Zhang Haoyu
2015-03-09 7:38 ` Paolo Bonzini
2015-03-09 8:14 ` Zhang Haoyu
2015-03-10 0:28 ` Fam Zheng
2015-03-10 1:30 ` [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to trackdirty sector Zhang Haoyu
2015-03-10 1:54 ` Fam Zheng
2015-03-10 2:12 ` [Qemu-devel] [RFC] introduce bitmap to bdrv_commit to trackdirtysector Zhang Haoyu
2015-03-10 2:58 ` Fam Zheng
2015-03-10 3:12 ` [Qemu-devel] [RFC] introduce bitmap to bdrv_commit totrackdirtysector Zhang Haoyu
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.