[Qemu-devel] [Bug 660366] [NEW] "qemu-img convert -O qcow2 -o backing_file" makes huge images

[Qemu-devel] [Bug 660366] [NEW] "qemu-img convert -O qcow2 -o backing_file" makes huge images

[Ildar]

Public bug reported:

$ dd if=/dev/urandom bs=1M of=1.img count=4
4+0 records in
4+0 records out
4194304 bytes (4,2 MB) copied, 1,0413 s, 4,0 MB/s
$ qemu-img create -f qcow2 -b 1.img 2.img
Formatting '2.img', fmt=qcow2 size=4194304 backing_file='1.img' encryption=off cluster_size=0 
$ qemu-img convert -O qcow2 -o backing_file=1.img 2.img 3.img
$ du -h ?.img
4,1M	1.img
144K	2.img
4,3M	3.img

The conversion result is bigger then the source!

It appears that "-o backing_file" is not applied to data (as expected).
I.e. all data is put into the resulting image: both from source image
and "backing" image.

Expected behavior is to put only data that is not present in
backing_file.

** Affects: qemu
     Importance: Undecided
         Status: New

-- 
"qemu-img convert -O qcow2 -o backing_file" makes huge images
https://bugs.launchpad.net/bugs/660366
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.

Status in QEMU: New

Bug description:
$ dd if=/dev/urandom bs=1M of=1.img count=4
4+0 records in
4+0 records out
4194304 bytes (4,2 MB) copied, 1,0413 s, 4,0 MB/s
$ qemu-img create -f qcow2 -b 1.img 2.img
Formatting '2.img', fmt=qcow2 size=4194304 backing_file='1.img' encryption=off cluster_size=0 
$ qemu-img convert -O qcow2 -o backing_file=1.img 2.img 3.img
$ du -h ?.img
4,1M	1.img
144K	2.img
4,3M	3.img

The conversion result is bigger then the source!

It appears that "-o backing_file" is not applied to data (as expected). I.e. all data is put into the resulting image: both from source image and "backing" image.

Expected behavior is to put only data that is not present in backing_file.

[Qemu-devel] [Bug 660366] Re: "qemu-img convert -O qcow2 -o backing_file" makes huge images

[Stefan Hajnoczi]

It is possible to chain backing files.  As a workaround you could do the
following:

$ qemu-img create -f qcow2 -b 2.img 4.img # from now on don't modify 2.img, instead use 4.img
$ qemu-img create -f qcow2 -b 2.img 3.img # here is the 3.img you tried to create with qemu-convert

Images 1.img and 2.img should never be modified, they are immutable
snapshots.

Images 3.img and 4.img can be modified and will contain only changes
against 2.img.

Perhaps qemu-img needs a command to drop data that is duplicated in the
base image.  This could be a new flag to commit: qemu-img commit --dedup
3.img.

-- 
"qemu-img convert -O qcow2 -o backing_file" makes huge images
https://bugs.launchpad.net/bugs/660366
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.

Status in QEMU: New

Bug description:
$ dd if=/dev/urandom bs=1M of=1.img count=4
4+0 records in
4+0 records out
4194304 bytes (4,2 MB) copied, 1,0413 s, 4,0 MB/s
$ qemu-img create -f qcow2 -b 1.img 2.img
Formatting '2.img', fmt=qcow2 size=4194304 backing_file='1.img' encryption=off cluster_size=0 
$ qemu-img convert -O qcow2 -o backing_file=1.img 2.img 3.img
$ du -h ?.img
4,1M	1.img
144K	2.img
4,3M	3.img

The conversion result is bigger then the source!

It appears that "-o backing_file" is not applied to data (as expected). I.e. all data is put into the resulting image: both from source image and "backing" image.

Expected behavior is to put only data that is not present in backing_file.

[Qemu-devel] [Bug 660366] Re: "qemu-img convert -O qcow2 -o backing_file" makes huge images

[Ildar]

Do you confirm this as a bug?

-- 
"qemu-img convert -O qcow2 -o backing_file" makes huge images
https://bugs.launchpad.net/bugs/660366
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.

Status in QEMU: New

Bug description:
$ dd if=/dev/urandom bs=1M of=1.img count=4
4+0 records in
4+0 records out
4194304 bytes (4,2 MB) copied, 1,0413 s, 4,0 MB/s
$ qemu-img create -f qcow2 -b 1.img 2.img
Formatting '2.img', fmt=qcow2 size=4194304 backing_file='1.img' encryption=off cluster_size=0 
$ qemu-img convert -O qcow2 -o backing_file=1.img 2.img 3.img
$ du -h ?.img
4,1M	1.img
144K	2.img
4,3M	3.img

The conversion result is bigger then the source!

It appears that "-o backing_file" is not applied to data (as expected). I.e. all data is put into the resulting image: both from source image and "backing" image.

Expected behavior is to put only data that is not present in backing_file.

[Qemu-devel] [Bug 660366] Re: "qemu-img convert -O qcow2 -o backing_file" makes huge images

[Stefan Hajnoczi]

Sorry I'm not a frequent Launchpad user and will leave it up to someone
more familiar to decide which status to place it in.

-- 
"qemu-img convert -O qcow2 -o backing_file" makes huge images
https://bugs.launchpad.net/bugs/660366
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.

Status in QEMU: New

Bug description:
$ dd if=/dev/urandom bs=1M of=1.img count=4
4+0 records in
4+0 records out
4194304 bytes (4,2 MB) copied, 1,0413 s, 4,0 MB/s
$ qemu-img create -f qcow2 -b 1.img 2.img
Formatting '2.img', fmt=qcow2 size=4194304 backing_file='1.img' encryption=off cluster_size=0 
$ qemu-img convert -O qcow2 -o backing_file=1.img 2.img 3.img
$ du -h ?.img
4,1M	1.img
144K	2.img
4,3M	3.img

The conversion result is bigger then the source!

It appears that "-o backing_file" is not applied to data (as expected). I.e. all data is put into the resulting image: both from source image and "backing" image.

Expected behavior is to put only data that is not present in backing_file.

[Qemu-devel] [Bug 660366] Re: "qemu-img convert -O qcow2 -o backing_file" makes huge images

[Anthony Liguori]

** Changed in: qemu
   Importance: Undecided => Wishlist

** Changed in: qemu
       Status: New => Confirmed

-- 
"qemu-img convert -O qcow2 -o backing_file" makes huge images
https://bugs.launchpad.net/bugs/660366
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.

Status in QEMU: Confirmed

Bug description:
$ dd if=/dev/urandom bs=1M of=1.img count=4
4+0 records in
4+0 records out
4194304 bytes (4,2 MB) copied, 1,0413 s, 4,0 MB/s
$ qemu-img create -f qcow2 -b 1.img 2.img
Formatting '2.img', fmt=qcow2 size=4194304 backing_file='1.img' encryption=off cluster_size=0 
$ qemu-img convert -O qcow2 -o backing_file=1.img 2.img 3.img
$ du -h ?.img
4,1M	1.img
144K	2.img
4,3M	3.img

The conversion result is bigger then the source!

It appears that "-o backing_file" is not applied to data (as expected). I.e. all data is put into the resulting image: both from source image and "backing" image.

Expected behavior is to put only data that is not present in backing_file.

Re: [Qemu-devel] [Bug 660366] Re: "qemu-img convert -O qcow2 -o backing_file" makes huge images

[Stefan Hajnoczi]

On Thu, Oct 14, 2010 at 1:26 PM, Anthony Liguori <anthony@codemonkey.ws> wrote:
> ** Changed in: qemu
>   Importance: Undecided => Wishlist
>
> ** Changed in: qemu
>       Status: New => Confirmed

Thanks for doing this Anthony.  Can I set the status myself next time
or do we have rules on who handles bugs?

Stefan

Re: [Qemu-devel] [Bug 660366] Re: "qemu-img convert -O qcow2 -o backing_file" makes huge images

[Anthony Liguori]

On 10/14/2010 07:51 AM, Stefan Hajnoczi wrote:
> On Thu, Oct 14, 2010 at 1:26 PM, Anthony Liguori<anthony@codemonkey.ws>  wrote:
>    
>> ** Changed in: qemu
>>    Importance: Undecided =>  Wishlist
>>
>> ** Changed in: qemu
>>        Status: New =>  Confirmed
>>      
> Thanks for doing this Anthony.  Can I set the status myself next time
> or do we have rules on who handles bugs?
>    

I'm pretty sure anyone can do it.  If not, I'm certainly willing to give 
people extra rights on the project if they want to help with bug triage.

Regards,

Anthony Liguori

> Stefan
>    

[Qemu-devel] [Bug 660366] Re: "qemu-img convert -O qcow2 -o backing_file" makes huge images

[Ildar]

I guess the problem is solved already.
qemu-img version 1.4.0

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/660366

Title:
  "qemu-img convert -O qcow2 -o backing_file" makes huge images

Status in QEMU:
  Confirmed

Bug description:
  $ dd if=/dev/urandom bs=1M of=1.img count=4
  4+0 records in
  4+0 records out
  4194304 bytes (4,2 MB) copied, 1,0413 s, 4,0 MB/s
  $ qemu-img create -f qcow2 -b 1.img 2.img
  Formatting '2.img', fmt=qcow2 size=4194304 backing_file='1.img' encryption=off cluster_size=0 
  $ qemu-img convert -O qcow2 -o backing_file=1.img 2.img 3.img
  $ du -h ?.img
  4,1M	1.img
  144K	2.img
  4,3M	3.img

  The conversion result is bigger then the source!

  It appears that "-o backing_file" is not applied to data (as
  expected). I.e. all data is put into the resulting image: both from
  source image and "backing" image.

  Expected behavior is to put only data that is not present in
  backing_file.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/660366/+subscriptions

[Qemu-devel] [Bug 660366] Re: "qemu-img convert -O qcow2 -o backing_file" makes huge images

[Ildar]

** Changed in: qemu
       Status: Confirmed => Fix Released

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/660366

Title:
  "qemu-img convert -O qcow2 -o backing_file" makes huge images

Status in QEMU:
  Fix Released

Bug description:
  $ dd if=/dev/urandom bs=1M of=1.img count=4
  4+0 records in
  4+0 records out
  4194304 bytes (4,2 MB) copied, 1,0413 s, 4,0 MB/s
  $ qemu-img create -f qcow2 -b 1.img 2.img
  Formatting '2.img', fmt=qcow2 size=4194304 backing_file='1.img' encryption=off cluster_size=0 
  $ qemu-img convert -O qcow2 -o backing_file=1.img 2.img 3.img
  $ du -h ?.img
  4,1M	1.img
  144K	2.img
  4,3M	3.img

  The conversion result is bigger then the source!

  It appears that "-o backing_file" is not applied to data (as
  expected). I.e. all data is put into the resulting image: both from
  source image and "backing" image.

  Expected behavior is to put only data that is not present in
  backing_file.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/660366/+subscriptions

[Qemu-devel] [Bug 660366] Re: "qemu-img convert -O qcow2 -o backing_file" makes huge images

[sakishrist]

Is this reintroduced? I am on version 2.3.0

$ dd if=/dev/urandom of=disk bs=1M count=1024

$ qemu-img convert -f raw -O qcow2 disk disk.qcow

$ qemu-img convert -f raw -O qcow2 -o backing_file=disk.qcow disk
disk1.qcow

$ ls -l
total 3146388
-rw-r--r-- 1 sakis sakis 1073741824 10 авг 15,29 disk
-rw-r--r-- 1 sakis sakis 1074135040 10 авг 15,30 disk.qcow
-rw-r--r-- 1 sakis sakis 1074135040 10 авг 15,31 disk1.qcow

All the data is copied again.

$ qemu-img info disk1.qcow
image: disk1.qcow
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 1.0G
cluster_size: 65536
*backing file: disk.qcow*
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

Qemu-img works as expected though.

$ qemu-img create -f qcow2 -o backing_file=disk1.qcow disk2.qcow

$ ls -l
total 3146584
-rw-r--r-- 1 sakis sakis 1073741824 10 авг 15,29 disk
-rw-r--r-- 1 sakis sakis 1074135040 10 авг 15,30 disk.qcow
-rw-r--r-- 1 sakis sakis 1074135040 10 авг 15,31 disk1.qcow
-rw-r--r-- 1 sakis sakis     197120 10 авг 15,33 disk2.qcow

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/660366

Title:
  "qemu-img convert -O qcow2 -o backing_file" makes huge images

Status in QEMU:
  Fix Released

Bug description:
  $ dd if=/dev/urandom bs=1M of=1.img count=4
  4+0 records in
  4+0 records out
  4194304 bytes (4,2 MB) copied, 1,0413 s, 4,0 MB/s
  $ qemu-img create -f qcow2 -b 1.img 2.img
  Formatting '2.img', fmt=qcow2 size=4194304 backing_file='1.img' encryption=off cluster_size=0 
  $ qemu-img convert -O qcow2 -o backing_file=1.img 2.img 3.img
  $ du -h ?.img
  4,1M	1.img
  144K	2.img
  4,3M	3.img

  The conversion result is bigger then the source!

  It appears that "-o backing_file" is not applied to data (as
  expected). I.e. all data is put into the resulting image: both from
  source image and "backing" image.

  Expected behavior is to put only data that is not present in
  backing_file.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/660366/+subscriptions

[Qemu-devel] [Bug 660366] Re: "qemu-img convert -O qcow2 -o backing_file" makes huge images

[Kevin Wolf]

This is a different case. The original report used "qemu-img create" in
step 2, which results in a sparse image that refers to the backing file
for all data. Your sequence has "qemu-img convert" instead, which fully
populates disk.qcow. Therefore, in step 3, "qemu-img convert" leaves the
full allocation intact.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/660366

Title:
  "qemu-img convert -O qcow2 -o backing_file" makes huge images

Status in QEMU:
  Fix Released

Bug description:
  $ dd if=/dev/urandom bs=1M of=1.img count=4
  4+0 records in
  4+0 records out
  4194304 bytes (4,2 MB) copied, 1,0413 s, 4,0 MB/s
  $ qemu-img create -f qcow2 -b 1.img 2.img
  Formatting '2.img', fmt=qcow2 size=4194304 backing_file='1.img' encryption=off cluster_size=0 
  $ qemu-img convert -O qcow2 -o backing_file=1.img 2.img 3.img
  $ du -h ?.img
  4,1M	1.img
  144K	2.img
  4,3M	3.img

  The conversion result is bigger then the source!

  It appears that "-o backing_file" is not applied to data (as
  expected). I.e. all data is put into the resulting image: both from
  source image and "backing" image.

  Expected behavior is to put only data that is not present in
  backing_file.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/660366/+subscriptions

[Qemu-devel] [Bug 660366] Re: "qemu-img convert -O qcow2 -o backing_file" makes huge images

[sakishrist]

My mistake. It's different case than mine. Above sequence (original
report) works fine.

But I do not really understand why the same is not achieved in my case.
I use the convert instead of the create to get a full image in qcow
format. From that point, the desired behaviour is to create a qcow that
is based on the one created from the first convert and contain only the
changes. Why would the second convert populate the whole image again?

Thanks in advance.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/660366

Title:
  "qemu-img convert -O qcow2 -o backing_file" makes huge images

Status in QEMU:
  Fix Released

Bug description:
  $ dd if=/dev/urandom bs=1M of=1.img count=4
  4+0 records in
  4+0 records out
  4194304 bytes (4,2 MB) copied, 1,0413 s, 4,0 MB/s
  $ qemu-img create -f qcow2 -b 1.img 2.img
  Formatting '2.img', fmt=qcow2 size=4194304 backing_file='1.img' encryption=off cluster_size=0 
  $ qemu-img convert -O qcow2 -o backing_file=1.img 2.img 3.img
  $ du -h ?.img
  4,1M	1.img
  144K	2.img
  4,3M	3.img

  The conversion result is bigger then the source!

  It appears that "-o backing_file" is not applied to data (as
  expected). I.e. all data is put into the resulting image: both from
  source image and "backing" image.

  Expected behavior is to put only data that is not present in
  backing_file.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/660366/+subscriptions