* qemu drive-mirror to rbd storage : no sparse rbd image
[not found] <54c55fc6-b12a-42e5-b99d-9752c8a9b19b@mailpro>
@ 2014-10-08 11:15 ` Alexandre DERUMIER
0 siblings, 0 replies; 22+ messages in thread
From: Alexandre DERUMIER @ 2014-10-08 11:15 UTC (permalink / raw)
To: qemu-devel; +Cc: Ceph Devel
Hi,
I'm currently planning to migrate our storage to ceph/rbd through qemu drive-mirror
and It seem that drive-mirror with rbd block driver, don't create a sparse image. (all zeros are copied to the target rbd).
Also note, that it's working fine with "qemu-img convert" , the rbd volume is sparse after conversion.
Could it be related to the "bdrv_co_write_zeroes" missing features in block/rbd.c ?
(It's available in other block drivers (scsi,gluster,raw-aio) , and I don't have this problem with theses block drivers).
Regards,
Alexandre Derumier
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
@ 2014-10-08 11:15 ` Alexandre DERUMIER
0 siblings, 0 replies; 22+ messages in thread
From: Alexandre DERUMIER @ 2014-10-08 11:15 UTC (permalink / raw)
To: qemu-devel; +Cc: Ceph Devel
Hi,
I'm currently planning to migrate our storage to ceph/rbd through qemu drive-mirror
and It seem that drive-mirror with rbd block driver, don't create a sparse image. (all zeros are copied to the target rbd).
Also note, that it's working fine with "qemu-img convert" , the rbd volume is sparse after conversion.
Could it be related to the "bdrv_co_write_zeroes" missing features in block/rbd.c ?
(It's available in other block drivers (scsi,gluster,raw-aio) , and I don't have this problem with theses block drivers).
Regards,
Alexandre Derumier
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
2014-10-08 11:15 ` [Qemu-devel] " Alexandre DERUMIER
@ 2014-10-11 7:01 ` Fam Zheng
-1 siblings, 0 replies; 22+ messages in thread
From: Fam Zheng @ 2014-10-11 7:01 UTC (permalink / raw)
To: Alexandre DERUMIER; +Cc: qemu-devel, Ceph Devel
On Wed, 10/08 13:15, Alexandre DERUMIER wrote:
> Hi,
>
> I'm currently planning to migrate our storage to ceph/rbd through qemu drive-mirror
>
> and It seem that drive-mirror with rbd block driver, don't create a sparse image. (all zeros are copied to the target rbd).
>
> Also note, that it's working fine with "qemu-img convert" , the rbd volume is sparse after conversion.
What is the source format? If the zero clusters are actually unallocated in the
source image, drive-mirror will not write those clusters either. I.e. with
"drive-mirror sync=top", both source and target should have the same "qemu-img
map" output.
Fam
>
>
> Could it be related to the "bdrv_co_write_zeroes" missing features in block/rbd.c ?
>
> (It's available in other block drivers (scsi,gluster,raw-aio) , and I don't have this problem with theses block drivers).
>
>
>
> Regards,
>
> Alexandre Derumier
>
>
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
@ 2014-10-11 7:01 ` Fam Zheng
0 siblings, 0 replies; 22+ messages in thread
From: Fam Zheng @ 2014-10-11 7:01 UTC (permalink / raw)
To: Alexandre DERUMIER; +Cc: Ceph Devel, qemu-devel
On Wed, 10/08 13:15, Alexandre DERUMIER wrote:
> Hi,
>
> I'm currently planning to migrate our storage to ceph/rbd through qemu drive-mirror
>
> and It seem that drive-mirror with rbd block driver, don't create a sparse image. (all zeros are copied to the target rbd).
>
> Also note, that it's working fine with "qemu-img convert" , the rbd volume is sparse after conversion.
What is the source format? If the zero clusters are actually unallocated in the
source image, drive-mirror will not write those clusters either. I.e. with
"drive-mirror sync=top", both source and target should have the same "qemu-img
map" output.
Fam
>
>
> Could it be related to the "bdrv_co_write_zeroes" missing features in block/rbd.c ?
>
> (It's available in other block drivers (scsi,gluster,raw-aio) , and I don't have this problem with theses block drivers).
>
>
>
> Regards,
>
> Alexandre Derumier
>
>
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
2014-10-11 7:01 ` Fam Zheng
@ 2014-10-11 8:00 ` Alexandre DERUMIER
-1 siblings, 0 replies; 22+ messages in thread
From: Alexandre DERUMIER @ 2014-10-11 8:00 UTC (permalink / raw)
To: Fam Zheng; +Cc: qemu-devel, Ceph Devel
>>What is the source format? If the zero clusters are actually unallocated in the
>>source image, drive-mirror will not write those clusters either. I.e. with
>>"drive-mirror sync=top", both source and target should have the same "qemu-img
>>map" output.
Thanks for your reply,
I had tried drive mirror (sync=full) with
raw file (sparse) -> rbd (no sparse)
rbd (sparse) -> rbd (no sparse)
raw file (sparse) -> qcow2 on ext4 (sparse)
rbd (sparse) -> raw on ext4 (sparse)
Also I see that I have the same problem with target file format on xfs.
raw file (sparse) -> qcow2 on xfs (no sparse)
rbd (sparse) -> raw on xfs (no sparse)
I only have this problem with drive-mirror, qemu-img convert seem to simply skip zero blocks.
Or maybe this is because I'm using sync=full ?
What is the difference between full and top ?
""sync": what parts of the disk image should be copied to the destination;
possibilities include "full" for all the disk, "top" for only the sectors
allocated in the topmost image".
(what is topmost image ?)
----- Mail original -----
De: "Fam Zheng" <famz@redhat.com>
À: "Alexandre DERUMIER" <aderumier@odiso.com>
Cc: "qemu-devel" <qemu-devel@nongnu.org>, "Ceph Devel" <ceph-devel@vger.kernel.org>
Envoyé: Samedi 11 Octobre 2014 09:01:18
Objet: Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
On Wed, 10/08 13:15, Alexandre DERUMIER wrote:
> Hi,
>
> I'm currently planning to migrate our storage to ceph/rbd through qemu drive-mirror
>
> and It seem that drive-mirror with rbd block driver, don't create a sparse image. (all zeros are copied to the target rbd).
>
> Also note, that it's working fine with "qemu-img convert" , the rbd volume is sparse after conversion.
What is the source format? If the zero clusters are actually unallocated in the
source image, drive-mirror will not write those clusters either. I.e. with
"drive-mirror sync=top", both source and target should have the same "qemu-img
map" output.
Fam
>
>
> Could it be related to the "bdrv_co_write_zeroes" missing features in block/rbd.c ?
>
> (It's available in other block drivers (scsi,gluster,raw-aio) , and I don't have this problem with theses block drivers).
>
>
>
> Regards,
>
> Alexandre Derumier
>
>
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
@ 2014-10-11 8:00 ` Alexandre DERUMIER
0 siblings, 0 replies; 22+ messages in thread
From: Alexandre DERUMIER @ 2014-10-11 8:00 UTC (permalink / raw)
To: Fam Zheng; +Cc: Ceph Devel, qemu-devel
>>What is the source format? If the zero clusters are actually unallocated in the
>>source image, drive-mirror will not write those clusters either. I.e. with
>>"drive-mirror sync=top", both source and target should have the same "qemu-img
>>map" output.
Thanks for your reply,
I had tried drive mirror (sync=full) with
raw file (sparse) -> rbd (no sparse)
rbd (sparse) -> rbd (no sparse)
raw file (sparse) -> qcow2 on ext4 (sparse)
rbd (sparse) -> raw on ext4 (sparse)
Also I see that I have the same problem with target file format on xfs.
raw file (sparse) -> qcow2 on xfs (no sparse)
rbd (sparse) -> raw on xfs (no sparse)
I only have this problem with drive-mirror, qemu-img convert seem to simply skip zero blocks.
Or maybe this is because I'm using sync=full ?
What is the difference between full and top ?
""sync": what parts of the disk image should be copied to the destination;
possibilities include "full" for all the disk, "top" for only the sectors
allocated in the topmost image".
(what is topmost image ?)
----- Mail original -----
De: "Fam Zheng" <famz@redhat.com>
À: "Alexandre DERUMIER" <aderumier@odiso.com>
Cc: "qemu-devel" <qemu-devel@nongnu.org>, "Ceph Devel" <ceph-devel@vger.kernel.org>
Envoyé: Samedi 11 Octobre 2014 09:01:18
Objet: Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
On Wed, 10/08 13:15, Alexandre DERUMIER wrote:
> Hi,
>
> I'm currently planning to migrate our storage to ceph/rbd through qemu drive-mirror
>
> and It seem that drive-mirror with rbd block driver, don't create a sparse image. (all zeros are copied to the target rbd).
>
> Also note, that it's working fine with "qemu-img convert" , the rbd volume is sparse after conversion.
What is the source format? If the zero clusters are actually unallocated in the
source image, drive-mirror will not write those clusters either. I.e. with
"drive-mirror sync=top", both source and target should have the same "qemu-img
map" output.
Fam
>
>
> Could it be related to the "bdrv_co_write_zeroes" missing features in block/rbd.c ?
>
> (It's available in other block drivers (scsi,gluster,raw-aio) , and I don't have this problem with theses block drivers).
>
>
>
> Regards,
>
> Alexandre Derumier
>
>
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
2014-10-11 8:00 ` Alexandre DERUMIER
@ 2014-10-11 8:25 ` Fam Zheng
-1 siblings, 0 replies; 22+ messages in thread
From: Fam Zheng @ 2014-10-11 8:25 UTC (permalink / raw)
To: Alexandre DERUMIER; +Cc: qemu-devel, Ceph Devel
On Sat, 10/11 10:00, Alexandre DERUMIER wrote:
> >>What is the source format? If the zero clusters are actually unallocated in the
> >>source image, drive-mirror will not write those clusters either. I.e. with
> >>"drive-mirror sync=top", both source and target should have the same "qemu-img
> >>map" output.
>
> Thanks for your reply,
>
> I had tried drive mirror (sync=full) with
>
> raw file (sparse) -> rbd (no sparse)
> rbd (sparse) -> rbd (no sparse)
> raw file (sparse) -> qcow2 on ext4 (sparse)
> rbd (sparse) -> raw on ext4 (sparse)
>
> Also I see that I have the same problem with target file format on xfs.
>
> raw file (sparse) -> qcow2 on xfs (no sparse)
> rbd (sparse) -> raw on xfs (no sparse)
>
These don't tell me much. Maybe it's better to show the actual commands and how
you tell sparse from no sparse?
Does "qcow2 -> qcow2" work for you on xfs?
>
> I only have this problem with drive-mirror, qemu-img convert seem to simply skip zero blocks.
>
>
> Or maybe this is because I'm using sync=full ?
>
> What is the difference between full and top ?
>
> ""sync": what parts of the disk image should be copied to the destination;
> possibilities include "full" for all the disk, "top" for only the sectors
> allocated in the topmost image".
>
> (what is topmost image ?)
For "sync=top", only the clusters allocated in the image itself is copied; for
"full", all those clusters allocated in the image itself, and its backing
image, and it's backing's backing image, ..., are copied.
The image itself, having a backing image or not, is called the topmost image.
Fam
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
@ 2014-10-11 8:25 ` Fam Zheng
0 siblings, 0 replies; 22+ messages in thread
From: Fam Zheng @ 2014-10-11 8:25 UTC (permalink / raw)
To: Alexandre DERUMIER; +Cc: Ceph Devel, qemu-devel
On Sat, 10/11 10:00, Alexandre DERUMIER wrote:
> >>What is the source format? If the zero clusters are actually unallocated in the
> >>source image, drive-mirror will not write those clusters either. I.e. with
> >>"drive-mirror sync=top", both source and target should have the same "qemu-img
> >>map" output.
>
> Thanks for your reply,
>
> I had tried drive mirror (sync=full) with
>
> raw file (sparse) -> rbd (no sparse)
> rbd (sparse) -> rbd (no sparse)
> raw file (sparse) -> qcow2 on ext4 (sparse)
> rbd (sparse) -> raw on ext4 (sparse)
>
> Also I see that I have the same problem with target file format on xfs.
>
> raw file (sparse) -> qcow2 on xfs (no sparse)
> rbd (sparse) -> raw on xfs (no sparse)
>
These don't tell me much. Maybe it's better to show the actual commands and how
you tell sparse from no sparse?
Does "qcow2 -> qcow2" work for you on xfs?
>
> I only have this problem with drive-mirror, qemu-img convert seem to simply skip zero blocks.
>
>
> Or maybe this is because I'm using sync=full ?
>
> What is the difference between full and top ?
>
> ""sync": what parts of the disk image should be copied to the destination;
> possibilities include "full" for all the disk, "top" for only the sectors
> allocated in the topmost image".
>
> (what is topmost image ?)
For "sync=top", only the clusters allocated in the image itself is copied; for
"full", all those clusters allocated in the image itself, and its backing
image, and it's backing's backing image, ..., are copied.
The image itself, having a backing image or not, is called the topmost image.
Fam
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
2014-10-11 8:25 ` Fam Zheng
@ 2014-10-11 8:30 ` Andrey Korolyov
-1 siblings, 0 replies; 22+ messages in thread
From: Andrey Korolyov @ 2014-10-11 8:30 UTC (permalink / raw)
To: Fam Zheng; +Cc: Alexandre DERUMIER, qemu-devel, Ceph Devel
On Sat, Oct 11, 2014 at 12:25 PM, Fam Zheng <famz@redhat.com> wrote:
> On Sat, 10/11 10:00, Alexandre DERUMIER wrote:
>> >>What is the source format? If the zero clusters are actually unallocated in the
>> >>source image, drive-mirror will not write those clusters either. I.e. with
>> >>"drive-mirror sync=top", both source and target should have the same "qemu-img
>> >>map" output.
>>
>> Thanks for your reply,
>>
>> I had tried drive mirror (sync=full) with
>>
>> raw file (sparse) -> rbd (no sparse)
>> rbd (sparse) -> rbd (no sparse)
>> raw file (sparse) -> qcow2 on ext4 (sparse)
>> rbd (sparse) -> raw on ext4 (sparse)
>>
>> Also I see that I have the same problem with target file format on xfs.
>>
>> raw file (sparse) -> qcow2 on xfs (no sparse)
>> rbd (sparse) -> raw on xfs (no sparse)
>>
>
> These don't tell me much. Maybe it's better to show the actual commands and how
> you tell sparse from no sparse?
>
> Does "qcow2 -> qcow2" work for you on xfs?
>
>>
>> I only have this problem with drive-mirror, qemu-img convert seem to simply skip zero blocks.
>>
>>
>> Or maybe this is because I'm using sync=full ?
>>
>> What is the difference between full and top ?
>>
>> ""sync": what parts of the disk image should be copied to the destination;
>> possibilities include "full" for all the disk, "top" for only the sectors
>> allocated in the topmost image".
>>
>> (what is topmost image ?)
>
> For "sync=top", only the clusters allocated in the image itself is copied; for
> "full", all those clusters allocated in the image itself, and its backing
> image, and it's backing's backing image, ..., are copied.
>
> The image itself, having a backing image or not, is called the topmost image.
>
> Fam
> --
Just a wild guess - Alexandre, did you tried detect-zeroes blk option
for mirroring targets?
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
@ 2014-10-11 8:30 ` Andrey Korolyov
0 siblings, 0 replies; 22+ messages in thread
From: Andrey Korolyov @ 2014-10-11 8:30 UTC (permalink / raw)
To: Fam Zheng; +Cc: Ceph Devel, qemu-devel, Alexandre DERUMIER
On Sat, Oct 11, 2014 at 12:25 PM, Fam Zheng <famz@redhat.com> wrote:
> On Sat, 10/11 10:00, Alexandre DERUMIER wrote:
>> >>What is the source format? If the zero clusters are actually unallocated in the
>> >>source image, drive-mirror will not write those clusters either. I.e. with
>> >>"drive-mirror sync=top", both source and target should have the same "qemu-img
>> >>map" output.
>>
>> Thanks for your reply,
>>
>> I had tried drive mirror (sync=full) with
>>
>> raw file (sparse) -> rbd (no sparse)
>> rbd (sparse) -> rbd (no sparse)
>> raw file (sparse) -> qcow2 on ext4 (sparse)
>> rbd (sparse) -> raw on ext4 (sparse)
>>
>> Also I see that I have the same problem with target file format on xfs.
>>
>> raw file (sparse) -> qcow2 on xfs (no sparse)
>> rbd (sparse) -> raw on xfs (no sparse)
>>
>
> These don't tell me much. Maybe it's better to show the actual commands and how
> you tell sparse from no sparse?
>
> Does "qcow2 -> qcow2" work for you on xfs?
>
>>
>> I only have this problem with drive-mirror, qemu-img convert seem to simply skip zero blocks.
>>
>>
>> Or maybe this is because I'm using sync=full ?
>>
>> What is the difference between full and top ?
>>
>> ""sync": what parts of the disk image should be copied to the destination;
>> possibilities include "full" for all the disk, "top" for only the sectors
>> allocated in the topmost image".
>>
>> (what is topmost image ?)
>
> For "sync=top", only the clusters allocated in the image itself is copied; for
> "full", all those clusters allocated in the image itself, and its backing
> image, and it's backing's backing image, ..., are copied.
>
> The image itself, having a backing image or not, is called the topmost image.
>
> Fam
> --
Just a wild guess - Alexandre, did you tried detect-zeroes blk option
for mirroring targets?
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
2014-10-11 8:30 ` Andrey Korolyov
@ 2014-10-12 10:02 ` Alexandre DERUMIER
-1 siblings, 0 replies; 22+ messages in thread
From: Alexandre DERUMIER @ 2014-10-12 10:02 UTC (permalink / raw)
To: Andrey Korolyov; +Cc: qemu-devel, Ceph Devel, Fam Zheng
>>Just a wild guess - Alexandre, did you tried detect-zeroes blk option
>>for mirroring targets?
Hi, yes, I have also tried with detect-zeroes (on or discard) with virtio and virtio-scsi, doesn't help. (I'm not sure that is implemtend in drive-mirror).
As workaround currently, after drive-mirror, I doing fstrim inside the guest (with virtio-scsi + discard),
and like this I can free space on rbd storage.
----- Mail original -----
De: "Andrey Korolyov" <andrey@xdel.ru>
À: "Fam Zheng" <famz@redhat.com>
Cc: "Alexandre DERUMIER" <aderumier@odiso.com>, "qemu-devel" <qemu-devel@nongnu.org>, "Ceph Devel" <ceph-devel@vger.kernel.org>
Envoyé: Samedi 11 Octobre 2014 10:30:40
Objet: Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
On Sat, Oct 11, 2014 at 12:25 PM, Fam Zheng <famz@redhat.com> wrote:
> On Sat, 10/11 10:00, Alexandre DERUMIER wrote:
>> >>What is the source format? If the zero clusters are actually unallocated in the
>> >>source image, drive-mirror will not write those clusters either. I.e. with
>> >>"drive-mirror sync=top", both source and target should have the same "qemu-img
>> >>map" output.
>>
>> Thanks for your reply,
>>
>> I had tried drive mirror (sync=full) with
>>
>> raw file (sparse) -> rbd (no sparse)
>> rbd (sparse) -> rbd (no sparse)
>> raw file (sparse) -> qcow2 on ext4 (sparse)
>> rbd (sparse) -> raw on ext4 (sparse)
>>
>> Also I see that I have the same problem with target file format on xfs.
>>
>> raw file (sparse) -> qcow2 on xfs (no sparse)
>> rbd (sparse) -> raw on xfs (no sparse)
>>
>
> These don't tell me much. Maybe it's better to show the actual commands and how
> you tell sparse from no sparse?
>
> Does "qcow2 -> qcow2" work for you on xfs?
>
>>
>> I only have this problem with drive-mirror, qemu-img convert seem to simply skip zero blocks.
>>
>>
>> Or maybe this is because I'm using sync=full ?
>>
>> What is the difference between full and top ?
>>
>> ""sync": what parts of the disk image should be copied to the destination;
>> possibilities include "full" for all the disk, "top" for only the sectors
>> allocated in the topmost image".
>>
>> (what is topmost image ?)
>
> For "sync=top", only the clusters allocated in the image itself is copied; for
> "full", all those clusters allocated in the image itself, and its backing
> image, and it's backing's backing image, ..., are copied.
>
> The image itself, having a backing image or not, is called the topmost image.
>
> Fam
> --
Just a wild guess - Alexandre, did you tried detect-zeroes blk option
for mirroring targets?
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
@ 2014-10-12 10:02 ` Alexandre DERUMIER
0 siblings, 0 replies; 22+ messages in thread
From: Alexandre DERUMIER @ 2014-10-12 10:02 UTC (permalink / raw)
To: Andrey Korolyov; +Cc: Ceph Devel, Fam Zheng, qemu-devel
>>Just a wild guess - Alexandre, did you tried detect-zeroes blk option
>>for mirroring targets?
Hi, yes, I have also tried with detect-zeroes (on or discard) with virtio and virtio-scsi, doesn't help. (I'm not sure that is implemtend in drive-mirror).
As workaround currently, after drive-mirror, I doing fstrim inside the guest (with virtio-scsi + discard),
and like this I can free space on rbd storage.
----- Mail original -----
De: "Andrey Korolyov" <andrey@xdel.ru>
À: "Fam Zheng" <famz@redhat.com>
Cc: "Alexandre DERUMIER" <aderumier@odiso.com>, "qemu-devel" <qemu-devel@nongnu.org>, "Ceph Devel" <ceph-devel@vger.kernel.org>
Envoyé: Samedi 11 Octobre 2014 10:30:40
Objet: Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
On Sat, Oct 11, 2014 at 12:25 PM, Fam Zheng <famz@redhat.com> wrote:
> On Sat, 10/11 10:00, Alexandre DERUMIER wrote:
>> >>What is the source format? If the zero clusters are actually unallocated in the
>> >>source image, drive-mirror will not write those clusters either. I.e. with
>> >>"drive-mirror sync=top", both source and target should have the same "qemu-img
>> >>map" output.
>>
>> Thanks for your reply,
>>
>> I had tried drive mirror (sync=full) with
>>
>> raw file (sparse) -> rbd (no sparse)
>> rbd (sparse) -> rbd (no sparse)
>> raw file (sparse) -> qcow2 on ext4 (sparse)
>> rbd (sparse) -> raw on ext4 (sparse)
>>
>> Also I see that I have the same problem with target file format on xfs.
>>
>> raw file (sparse) -> qcow2 on xfs (no sparse)
>> rbd (sparse) -> raw on xfs (no sparse)
>>
>
> These don't tell me much. Maybe it's better to show the actual commands and how
> you tell sparse from no sparse?
>
> Does "qcow2 -> qcow2" work for you on xfs?
>
>>
>> I only have this problem with drive-mirror, qemu-img convert seem to simply skip zero blocks.
>>
>>
>> Or maybe this is because I'm using sync=full ?
>>
>> What is the difference between full and top ?
>>
>> ""sync": what parts of the disk image should be copied to the destination;
>> possibilities include "full" for all the disk, "top" for only the sectors
>> allocated in the topmost image".
>>
>> (what is topmost image ?)
>
> For "sync=top", only the clusters allocated in the image itself is copied; for
> "full", all those clusters allocated in the image itself, and its backing
> image, and it's backing's backing image, ..., are copied.
>
> The image itself, having a backing image or not, is called the topmost image.
>
> Fam
> --
Just a wild guess - Alexandre, did you tried detect-zeroes blk option
for mirroring targets?
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
2014-10-11 8:25 ` Fam Zheng
@ 2014-10-12 10:33 ` Alexandre DERUMIER
-1 siblings, 0 replies; 22+ messages in thread
From: Alexandre DERUMIER @ 2014-10-12 10:33 UTC (permalink / raw)
To: Fam Zheng; +Cc: qemu-devel, Ceph Devel
>>These don't tell me much. Maybe it's better to show the actual commands and how
>>you tell sparse from no sparse?
Well,
I create 2 empty source images files of 10G. (source.qcow2 and source.raw)
then:
du -sh source.qcow2 : 2M
du -sh source.raw : 0M
then I convert them with qemu-img convert : (source.qcow2 -> target.qcow2 , source.raw -> target.raw)
du -sh target.qcow2 : 2M
du -sh target.raw : 0M
(So it's ok here)
But If I convert them with drive-mirror:
du -sh target.qcow2 : 11G
du -sh target.raw : 11G
I have also double check with #df, and I see space allocated on filesystem when using drive-mirror.
I have the same behavior if target is a rbd storage.
Also I have done test again ext3,ext4, and I see the same problem than with xfs.
I'm pretty sure that drive-mirror copy zero block, qemu-img convert take around 2s to convert the empty file
(because it's skipping zero block), and drive mirror take around 5min.
----- Mail original -----
De: "Fam Zheng" <famz@redhat.com>
À: "Alexandre DERUMIER" <aderumier@odiso.com>
Cc: "qemu-devel" <qemu-devel@nongnu.org>, "Ceph Devel" <ceph-devel@vger.kernel.org>
Envoyé: Samedi 11 Octobre 2014 10:25:35
Objet: Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
On Sat, 10/11 10:00, Alexandre DERUMIER wrote:
> >>What is the source format? If the zero clusters are actually unallocated in the
> >>source image, drive-mirror will not write those clusters either. I.e. with
> >>"drive-mirror sync=top", both source and target should have the same "qemu-img
> >>map" output.
>
> Thanks for your reply,
>
> I had tried drive mirror (sync=full) with
>
> raw file (sparse) -> rbd (no sparse)
> rbd (sparse) -> rbd (no sparse)
> raw file (sparse) -> qcow2 on ext4 (sparse)
> rbd (sparse) -> raw on ext4 (sparse)
>
> Also I see that I have the same problem with target file format on xfs.
>
> raw file (sparse) -> qcow2 on xfs (no sparse)
> rbd (sparse) -> raw on xfs (no sparse)
>
These don't tell me much. Maybe it's better to show the actual commands and how
you tell sparse from no sparse?
Does "qcow2 -> qcow2" work for you on xfs?
>
> I only have this problem with drive-mirror, qemu-img convert seem to simply skip zero blocks.
>
>
> Or maybe this is because I'm using sync=full ?
>
> What is the difference between full and top ?
>
> ""sync": what parts of the disk image should be copied to the destination;
> possibilities include "full" for all the disk, "top" for only the sectors
> allocated in the topmost image".
>
> (what is topmost image ?)
For "sync=top", only the clusters allocated in the image itself is copied; for
"full", all those clusters allocated in the image itself, and its backing
image, and it's backing's backing image, ..., are copied.
The image itself, having a backing image or not, is called the topmost image.
Fam
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
@ 2014-10-12 10:33 ` Alexandre DERUMIER
0 siblings, 0 replies; 22+ messages in thread
From: Alexandre DERUMIER @ 2014-10-12 10:33 UTC (permalink / raw)
To: Fam Zheng; +Cc: Ceph Devel, qemu-devel
>>These don't tell me much. Maybe it's better to show the actual commands and how
>>you tell sparse from no sparse?
Well,
I create 2 empty source images files of 10G. (source.qcow2 and source.raw)
then:
du -sh source.qcow2 : 2M
du -sh source.raw : 0M
then I convert them with qemu-img convert : (source.qcow2 -> target.qcow2 , source.raw -> target.raw)
du -sh target.qcow2 : 2M
du -sh target.raw : 0M
(So it's ok here)
But If I convert them with drive-mirror:
du -sh target.qcow2 : 11G
du -sh target.raw : 11G
I have also double check with #df, and I see space allocated on filesystem when using drive-mirror.
I have the same behavior if target is a rbd storage.
Also I have done test again ext3,ext4, and I see the same problem than with xfs.
I'm pretty sure that drive-mirror copy zero block, qemu-img convert take around 2s to convert the empty file
(because it's skipping zero block), and drive mirror take around 5min.
----- Mail original -----
De: "Fam Zheng" <famz@redhat.com>
À: "Alexandre DERUMIER" <aderumier@odiso.com>
Cc: "qemu-devel" <qemu-devel@nongnu.org>, "Ceph Devel" <ceph-devel@vger.kernel.org>
Envoyé: Samedi 11 Octobre 2014 10:25:35
Objet: Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
On Sat, 10/11 10:00, Alexandre DERUMIER wrote:
> >>What is the source format? If the zero clusters are actually unallocated in the
> >>source image, drive-mirror will not write those clusters either. I.e. with
> >>"drive-mirror sync=top", both source and target should have the same "qemu-img
> >>map" output.
>
> Thanks for your reply,
>
> I had tried drive mirror (sync=full) with
>
> raw file (sparse) -> rbd (no sparse)
> rbd (sparse) -> rbd (no sparse)
> raw file (sparse) -> qcow2 on ext4 (sparse)
> rbd (sparse) -> raw on ext4 (sparse)
>
> Also I see that I have the same problem with target file format on xfs.
>
> raw file (sparse) -> qcow2 on xfs (no sparse)
> rbd (sparse) -> raw on xfs (no sparse)
>
These don't tell me much. Maybe it's better to show the actual commands and how
you tell sparse from no sparse?
Does "qcow2 -> qcow2" work for you on xfs?
>
> I only have this problem with drive-mirror, qemu-img convert seem to simply skip zero blocks.
>
>
> Or maybe this is because I'm using sync=full ?
>
> What is the difference between full and top ?
>
> ""sync": what parts of the disk image should be copied to the destination;
> possibilities include "full" for all the disk, "top" for only the sectors
> allocated in the topmost image".
>
> (what is topmost image ?)
For "sync=top", only the clusters allocated in the image itself is copied; for
"full", all those clusters allocated in the image itself, and its backing
image, and it's backing's backing image, ..., are copied.
The image itself, having a backing image or not, is called the topmost image.
Fam
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: qemu drive-mirror to rbd storage : no sparse rbd image
2014-10-08 11:15 ` [Qemu-devel] " Alexandre DERUMIER
@ 2014-10-12 13:02 ` Paolo Bonzini
-1 siblings, 0 replies; 22+ messages in thread
From: Paolo Bonzini @ 2014-10-12 13:02 UTC (permalink / raw)
To: Alexandre DERUMIER, qemu-devel; +Cc: Ceph Devel
Il 08/10/2014 13:15, Alexandre DERUMIER ha scritto:
> Hi,
>
> I'm currently planning to migrate our storage to ceph/rbd through qemu drive-mirror
>
> and It seem that drive-mirror with rbd block driver, don't create a sparse image. (all zeros are copied to the target rbd).
>
> Also note, that it's working fine with "qemu-img convert" , the rbd volume is sparse after conversion.
>
>
> Could it be related to the "bdrv_co_write_zeroes" missing features in block/rbd.c ?
>
> (It's available in other block drivers (scsi,gluster,raw-aio) , and I don't have this problem with theses block drivers).
Lack of bdrv_co_write_zeroes is why detect-zeroes does not work.
Lack of bdrv_get_block_status is why sparse->sparse does not work
without detect-zeroes.
Paolo
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
@ 2014-10-12 13:02 ` Paolo Bonzini
0 siblings, 0 replies; 22+ messages in thread
From: Paolo Bonzini @ 2014-10-12 13:02 UTC (permalink / raw)
To: Alexandre DERUMIER, qemu-devel; +Cc: Ceph Devel
Il 08/10/2014 13:15, Alexandre DERUMIER ha scritto:
> Hi,
>
> I'm currently planning to migrate our storage to ceph/rbd through qemu drive-mirror
>
> and It seem that drive-mirror with rbd block driver, don't create a sparse image. (all zeros are copied to the target rbd).
>
> Also note, that it's working fine with "qemu-img convert" , the rbd volume is sparse after conversion.
>
>
> Could it be related to the "bdrv_co_write_zeroes" missing features in block/rbd.c ?
>
> (It's available in other block drivers (scsi,gluster,raw-aio) , and I don't have this problem with theses block drivers).
Lack of bdrv_co_write_zeroes is why detect-zeroes does not work.
Lack of bdrv_get_block_status is why sparse->sparse does not work
without detect-zeroes.
Paolo
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: qemu drive-mirror to rbd storage : no sparse rbd image
2014-10-12 13:02 ` [Qemu-devel] " Paolo Bonzini
@ 2014-10-13 6:06 ` Alexandre DERUMIER
-1 siblings, 0 replies; 22+ messages in thread
From: Alexandre DERUMIER @ 2014-10-13 6:06 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: Ceph Devel, qemu-devel
>>Lack of bdrv_co_write_zeroes is why detect-zeroes does not work.
>>
>>Lack of bdrv_get_block_status is why sparse->sparse does not work
>>without detect-zeroes.
Ok, thanks Paolo !
Both are missing in rbd block driver. @ceph-devel . Could it be possible to implement them ?
Also, about drive-mirror, I had tried with detect-zeroes with simple qcow2 file,
and It don't seem to help.
I'm not sure that detect-zeroes is implement in drive-mirror.
also, the target mirrored volume don't seem to have the detect-zeroes option
# info block
drive-virtio1: /source.qcow2 (qcow2)
Detect zeroes: on
#du -sh source.qcow2 : 2M
drive-mirror source.qcow2 -> target.qcow2
# info block
drive-virtio1: /target.qcow2 (qcow2)
#du -sh target.qcow2 : 11G
----- Mail original -----
De: "Paolo Bonzini" <pbonzini@redhat.com>
À: "Alexandre DERUMIER" <aderumier@odiso.com>, "qemu-devel" <qemu-devel@nongnu.org>
Cc: "Ceph Devel" <ceph-devel@vger.kernel.org>
Envoyé: Dimanche 12 Octobre 2014 15:02:12
Objet: Re: qemu drive-mirror to rbd storage : no sparse rbd image
Il 08/10/2014 13:15, Alexandre DERUMIER ha scritto:
> Hi,
>
> I'm currently planning to migrate our storage to ceph/rbd through qemu drive-mirror
>
> and It seem that drive-mirror with rbd block driver, don't create a sparse image. (all zeros are copied to the target rbd).
>
> Also note, that it's working fine with "qemu-img convert" , the rbd volume is sparse after conversion.
>
>
> Could it be related to the "bdrv_co_write_zeroes" missing features in block/rbd.c ?
>
> (It's available in other block drivers (scsi,gluster,raw-aio) , and I don't have this problem with theses block drivers).
Lack of bdrv_co_write_zeroes is why detect-zeroes does not work.
Lack of bdrv_get_block_status is why sparse->sparse does not work
without detect-zeroes.
Paolo
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
@ 2014-10-13 6:06 ` Alexandre DERUMIER
0 siblings, 0 replies; 22+ messages in thread
From: Alexandre DERUMIER @ 2014-10-13 6:06 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: Ceph Devel, qemu-devel
>>Lack of bdrv_co_write_zeroes is why detect-zeroes does not work.
>>
>>Lack of bdrv_get_block_status is why sparse->sparse does not work
>>without detect-zeroes.
Ok, thanks Paolo !
Both are missing in rbd block driver. @ceph-devel . Could it be possible to implement them ?
Also, about drive-mirror, I had tried with detect-zeroes with simple qcow2 file,
and It don't seem to help.
I'm not sure that detect-zeroes is implement in drive-mirror.
also, the target mirrored volume don't seem to have the detect-zeroes option
# info block
drive-virtio1: /source.qcow2 (qcow2)
Detect zeroes: on
#du -sh source.qcow2 : 2M
drive-mirror source.qcow2 -> target.qcow2
# info block
drive-virtio1: /target.qcow2 (qcow2)
#du -sh target.qcow2 : 11G
----- Mail original -----
De: "Paolo Bonzini" <pbonzini@redhat.com>
À: "Alexandre DERUMIER" <aderumier@odiso.com>, "qemu-devel" <qemu-devel@nongnu.org>
Cc: "Ceph Devel" <ceph-devel@vger.kernel.org>
Envoyé: Dimanche 12 Octobre 2014 15:02:12
Objet: Re: qemu drive-mirror to rbd storage : no sparse rbd image
Il 08/10/2014 13:15, Alexandre DERUMIER ha scritto:
> Hi,
>
> I'm currently planning to migrate our storage to ceph/rbd through qemu drive-mirror
>
> and It seem that drive-mirror with rbd block driver, don't create a sparse image. (all zeros are copied to the target rbd).
>
> Also note, that it's working fine with "qemu-img convert" , the rbd volume is sparse after conversion.
>
>
> Could it be related to the "bdrv_co_write_zeroes" missing features in block/rbd.c ?
>
> (It's available in other block drivers (scsi,gluster,raw-aio) , and I don't have this problem with theses block drivers).
Lack of bdrv_co_write_zeroes is why detect-zeroes does not work.
Lack of bdrv_get_block_status is why sparse->sparse does not work
without detect-zeroes.
Paolo
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: qemu drive-mirror to rbd storage : no sparse rbd image
2014-10-13 6:06 ` [Qemu-devel] " Alexandre DERUMIER
@ 2014-10-13 7:06 ` Paolo Bonzini
-1 siblings, 0 replies; 22+ messages in thread
From: Paolo Bonzini @ 2014-10-13 7:06 UTC (permalink / raw)
To: Alexandre DERUMIER; +Cc: Ceph Devel, qemu-devel
Il 13/10/2014 08:06, Alexandre DERUMIER ha scritto:
>
> Also, about drive-mirror, I had tried with detect-zeroes with simple qcow2 file,
> and It don't seem to help.
> I'm not sure that detect-zeroes is implement in drive-mirror.
>
> also, the target mirrored volume don't seem to have the detect-zeroes option
>
>
> # info block
> drive-virtio1: /source.qcow2 (qcow2)
> Detect zeroes: on
>
> #du -sh source.qcow2 : 2M
>
> drive-mirror source.qcow2 -> target.qcow2
>
> # info block
> drive-virtio1: /target.qcow2 (qcow2)
>
> #du -sh target.qcow2 : 11G
>
Ah, you're right. We need to add an options field, or use a new
blockdev-mirror command.
Paolo
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
@ 2014-10-13 7:06 ` Paolo Bonzini
0 siblings, 0 replies; 22+ messages in thread
From: Paolo Bonzini @ 2014-10-13 7:06 UTC (permalink / raw)
To: Alexandre DERUMIER; +Cc: Ceph Devel, qemu-devel
Il 13/10/2014 08:06, Alexandre DERUMIER ha scritto:
>
> Also, about drive-mirror, I had tried with detect-zeroes with simple qcow2 file,
> and It don't seem to help.
> I'm not sure that detect-zeroes is implement in drive-mirror.
>
> also, the target mirrored volume don't seem to have the detect-zeroes option
>
>
> # info block
> drive-virtio1: /source.qcow2 (qcow2)
> Detect zeroes: on
>
> #du -sh source.qcow2 : 2M
>
> drive-mirror source.qcow2 -> target.qcow2
>
> # info block
> drive-virtio1: /target.qcow2 (qcow2)
>
> #du -sh target.qcow2 : 11G
>
Ah, you're right. We need to add an options field, or use a new
blockdev-mirror command.
Paolo
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: qemu drive-mirror to rbd storage : no sparse rbd image
2014-10-13 7:06 ` [Qemu-devel] " Paolo Bonzini
@ 2014-10-13 8:08 ` Alexandre DERUMIER
-1 siblings, 0 replies; 22+ messages in thread
From: Alexandre DERUMIER @ 2014-10-13 8:08 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: Ceph Devel, qemu-devel
>>Ah, you're right. We need to add an options field, or use a new
>>blockdev-mirror command.
Ok, thanks. Can't help to implement this, but I'll glad to help for testing.
----- Mail original -----
De: "Paolo Bonzini" <pbonzini@redhat.com>
À: "Alexandre DERUMIER" <aderumier@odiso.com>
Cc: "Ceph Devel" <ceph-devel@vger.kernel.org>, "qemu-devel" <qemu-devel@nongnu.org>
Envoyé: Lundi 13 Octobre 2014 09:06:01
Objet: Re: qemu drive-mirror to rbd storage : no sparse rbd image
Il 13/10/2014 08:06, Alexandre DERUMIER ha scritto:
>
> Also, about drive-mirror, I had tried with detect-zeroes with simple qcow2 file,
> and It don't seem to help.
> I'm not sure that detect-zeroes is implement in drive-mirror.
>
> also, the target mirrored volume don't seem to have the detect-zeroes option
>
>
> # info block
> drive-virtio1: /source.qcow2 (qcow2)
> Detect zeroes: on
>
> #du -sh source.qcow2 : 2M
>
> drive-mirror source.qcow2 -> target.qcow2
>
> # info block
> drive-virtio1: /target.qcow2 (qcow2)
>
> #du -sh target.qcow2 : 11G
>
Ah, you're right. We need to add an options field, or use a new
blockdev-mirror command.
Paolo
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] qemu drive-mirror to rbd storage : no sparse rbd image
@ 2014-10-13 8:08 ` Alexandre DERUMIER
0 siblings, 0 replies; 22+ messages in thread
From: Alexandre DERUMIER @ 2014-10-13 8:08 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: Ceph Devel, qemu-devel
>>Ah, you're right. We need to add an options field, or use a new
>>blockdev-mirror command.
Ok, thanks. Can't help to implement this, but I'll glad to help for testing.
----- Mail original -----
De: "Paolo Bonzini" <pbonzini@redhat.com>
À: "Alexandre DERUMIER" <aderumier@odiso.com>
Cc: "Ceph Devel" <ceph-devel@vger.kernel.org>, "qemu-devel" <qemu-devel@nongnu.org>
Envoyé: Lundi 13 Octobre 2014 09:06:01
Objet: Re: qemu drive-mirror to rbd storage : no sparse rbd image
Il 13/10/2014 08:06, Alexandre DERUMIER ha scritto:
>
> Also, about drive-mirror, I had tried with detect-zeroes with simple qcow2 file,
> and It don't seem to help.
> I'm not sure that detect-zeroes is implement in drive-mirror.
>
> also, the target mirrored volume don't seem to have the detect-zeroes option
>
>
> # info block
> drive-virtio1: /source.qcow2 (qcow2)
> Detect zeroes: on
>
> #du -sh source.qcow2 : 2M
>
> drive-mirror source.qcow2 -> target.qcow2
>
> # info block
> drive-virtio1: /target.qcow2 (qcow2)
>
> #du -sh target.qcow2 : 11G
>
Ah, you're right. We need to add an options field, or use a new
blockdev-mirror command.
Paolo
^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2014-10-13 8:09 UTC | newest]
Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <54c55fc6-b12a-42e5-b99d-9752c8a9b19b@mailpro>
2014-10-08 11:15 ` qemu drive-mirror to rbd storage : no sparse rbd image Alexandre DERUMIER
2014-10-08 11:15 ` [Qemu-devel] " Alexandre DERUMIER
2014-10-11 7:01 ` Fam Zheng
2014-10-11 7:01 ` Fam Zheng
2014-10-11 8:00 ` Alexandre DERUMIER
2014-10-11 8:00 ` Alexandre DERUMIER
2014-10-11 8:25 ` Fam Zheng
2014-10-11 8:25 ` Fam Zheng
2014-10-11 8:30 ` Andrey Korolyov
2014-10-11 8:30 ` Andrey Korolyov
2014-10-12 10:02 ` Alexandre DERUMIER
2014-10-12 10:02 ` Alexandre DERUMIER
2014-10-12 10:33 ` Alexandre DERUMIER
2014-10-12 10:33 ` Alexandre DERUMIER
2014-10-12 13:02 ` Paolo Bonzini
2014-10-12 13:02 ` [Qemu-devel] " Paolo Bonzini
2014-10-13 6:06 ` Alexandre DERUMIER
2014-10-13 6:06 ` [Qemu-devel] " Alexandre DERUMIER
2014-10-13 7:06 ` Paolo Bonzini
2014-10-13 7:06 ` [Qemu-devel] " Paolo Bonzini
2014-10-13 8:08 ` Alexandre DERUMIER
2014-10-13 8:08 ` [Qemu-devel] " Alexandre DERUMIER
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.