[Qemu-devel] The image size of instance VM keeps growing

[Qemu-devel] The image size of instance VM keeps growing

[Charles.Tsai-蔡清海-研究發展部]

[-- Attachment #1: Type: text/plain, Size: 828 bytes --]



We created a qcow2 based-image for a VM whose size is big when it is created. When a VM is launched, it is a instance VM image which refers to the based-image. Initially, the image size of instance VM is smaller than that of the based-mage. But we found the image size of the instance VM kept growing, when a VM ran in idle state. We doubted this issue is caused by the Windows swap file(virtual memory) that Windows OS pages the memory contents to it during the run-time. We did disable the swap file setting from Windows but the image size of the instance VM still kept growing.



Ideally, the image size of instance VM is supposed to remain in a smaller size than the based-image. Why does it keep growing in size which is even bigger than that of based-image? Our VM, in fact, did nothing during the idle state.


[-- Attachment #2: Type: text/html, Size: 3160 bytes --]

Re: [Qemu-devel] The image size of instance VM keeps growing

[Stefan Hajnoczi]

On Mon, May 21, 2012 at 8:47 AM, Charles.Tsai-蔡清海-研究發展部
<charles.tsai@cloudena.com> wrote:
> Ideally, the image size of instance VM is supposed to remain in a smaller
> size than the based-image. Why does it keep growing in size which is even
> bigger than that of based-image? Our VM, in fact, did nothing during the
> idle state.

The guest is probably issuing I/O.  Are there Windows tools you can
use to see which process or kernel component is issuing I/O?

You can try running qemu-img check <image-file> to ensure that the
qcow2 file is consistent.  Error messages will be printed if there are
leaks or corruptions in the qcow2 file.

Stefan

Re: [Qemu-devel] The image size of instance VM keeps growing

[Charles.Tsai-蔡清海-研究發展部]

Stefan,

We can run a tool to see if there is any active I/O or not. Does the image size shrink back if an I/O activity stops?
What we found here is that the image size kept growing. If this problem persists, it could eat up the entire disk space.


-----Original Message-----
From: Stefan Hajnoczi [mailto:stefanha@gmail.com] 
Sent: Monday, May 21, 2012 4:44 PM
To: Charles.Tsai-蔡清海-研究發展部
Cc: qemu-devel@nongnu.org; Jonah.Wu-吳君勉-研究發展部
Subject: Re: [Qemu-devel] The image size of instance VM keeps growing

On Mon, May 21, 2012 at 8:47 AM, Charles.Tsai-蔡清海-研究發展部
<charles.tsai@cloudena.com> wrote:
> Ideally, the image size of instance VM is supposed to remain in a 
> smaller size than the based-image. Why does it keep growing in size 
> which is even bigger than that of based-image? Our VM, in fact, did 
> nothing during the idle state.

The guest is probably issuing I/O.  Are there Windows tools you can use to see which process or kernel component is issuing I/O?

You can try running qemu-img check <image-file> to ensure that the
qcow2 file is consistent.  Error messages will be printed if there are leaks or corruptions in the qcow2 file.

Stefan

Re: [Qemu-devel] The image size of instance VM keeps growing

[Stefan Hajnoczi]

On Mon, May 21, 2012 at 10:08 AM, Charles.Tsai-蔡清海-研究發展部
<charles.tsai@cloudena.com> wrote:
> We can run a tool to see if there is any active I/O or not. Does the image size shrink back if an I/O activity stops?
> What we found here is that the image size kept growing. If this problem persists, it could eat up the entire disk space.

It's possible for an image to grow larger than its backing file due to
the layout and metadata of qcow2 (or any image file format).  But this
overhead should be fixed to maximum 1 MB, maybe (assuming you are not
using qcow2 snapshots where the file can grow arbitrarily).

If the image file is growing endlessly then there is a problem :).
The starting point is figuring out which I/O request causes this to
happen so it can be reproduced and debugged.  Can you isolate the
problem to a few steps that can be reproduced?

Stefan

Re: [Qemu-devel] The image size of instance VM keeps growing

[Charles.Tsai-蔡清海-研究發展部]

Stefan,

Thank you for information. We will try to isolate the issue here and give you an update.


-----Original Message-----
From: Stefan Hajnoczi [mailto:stefanha@gmail.com] 
Sent: Monday, May 21, 2012 9:24 PM
To: Charles.Tsai-蔡清海-研究發展部
Cc: qemu-devel@nongnu.org; Jonah.Wu-吳君勉-研究發展部
Subject: Re: [Qemu-devel] The image size of instance VM keeps growing

On Mon, May 21, 2012 at 10:08 AM, Charles.Tsai-蔡清海-研究發展部
<charles.tsai@cloudena.com> wrote:
> We can run a tool to see if there is any active I/O or not. Does the image size shrink back if an I/O activity stops?
> What we found here is that the image size kept growing. If this problem persists, it could eat up the entire disk space.

It's possible for an image to grow larger than its backing file due to the layout and metadata of qcow2 (or any image file format).  But this overhead should be fixed to maximum 1 MB, maybe (assuming you are not using qcow2 snapshots where the file can grow arbitrarily).

If the image file is growing endlessly then there is a problem :).
The starting point is figuring out which I/O request causes this to happen so it can be reproduced and debugged.  Can you isolate the problem to a few steps that can be reproduced?

Stefan

Re: [Qemu-devel] The image size of instance VM keeps growing

[Charles.Tsai-蔡清海-研究發展部]

Stefan,

	Here are what we tested and found  for this issue.

	1.	We checked the integrity of the qcow2 image by running “qemu-img check” and it looked good.
	2.	We monitored the I/O activities of VM using 
	“Process monitor”, “Process explorer” and could not find a process running an aggressive I/O activity in the background.

	In the guest OS, the following configurations are turn off to cut down I/O activities.
	1.	Disable the NIC.
	2.	Disable Windows automatic update.
	3.	Disable the page file of the virtual memory.

	After the VM ran for overnight, the image size for the running VM grew up to 3G bytes.


	This issue is very easy to be duplicated. Here are the steps.

	1.	Create a qcow2 image of 64-bit Windows 7 VM from the CD.
	2.	Launch the VM
	3.	Turn off the VM configurations stated above.
	4.	Let the VM run.
	5.	Watch the growth of the VM image size. The longer the VM runs, the bigger the size of the VM is.

	During the run-time, you can run “process monitor” to watch guest OS’s I/O activities. You can see File I/O, registry I/O from the 	system during the run-time in this test case. But the growth rate of the image size seems to be greater than default system I/O.


-----Original Message-----
From: Stefan Hajnoczi [mailto:stefanha@gmail.com] 
Sent: Monday, May 21, 2012 9:24 PM
To: Charles.Tsai-蔡清海-研究發展部
Cc: qemu-devel@nongnu.org; Jonah.Wu-吳君勉-研究發展部
Subject: Re: [Qemu-devel] The image size of instance VM keeps growing

On Mon, May 21, 2012 at 10:08 AM, Charles.Tsai-蔡清海-研究發展部
<charles.tsai@cloudena.com> wrote:
> We can run a tool to see if there is any active I/O or not. Does the image size shrink back if an I/O activity stops?
> What we found here is that the image size kept growing. If this problem persists, it could eat up the entire disk space.

It's possible for an image to grow larger than its backing file due to the layout and metadata of qcow2 (or any image file format).  But this overhead should be fixed to maximum 1 MB, maybe (assuming you are not using qcow2 snapshots where the file can grow arbitrarily).

If the image file is growing endlessly then there is a problem :).
The starting point is figuring out which I/O request causes this to happen so it can be reproduced and debugged.  Can you isolate the problem to a few steps that can be reproduced?

Stefan

Re: [Qemu-devel] The image size of instance VM keeps growing

[nicolas prochazka]

[-- Attachment #1: Type: text/plain, Size: 3073 bytes --]

hello,
I think it's not qemu relative but windows relative
Windows do a lot of things as auto defrag disk ( without monitoring) when
idle time
read this document:
http://www.vmware.com/files/pdf/VMware-View-OptimizationGuideWindows7-EN.pdfand
optimize your guest.

Regards,
Nicolas P.

2012/5/23 Charles.Tsai-蔡清海-研究發展部 <charles.tsai@cloudena.com>

> Stefan,
>
>        Here are what we tested and found  for this issue.
>
>        1.      We checked the integrity of the qcow2 image by running
> “qemu-img check” and it looked good.
>        2.      We monitored the I/O activities of VM using
>        “Process monitor”, “Process explorer” and could not find a process
> running an aggressive I/O activity in the background.
>
>        In the guest OS, the following configurations are turn off to cut
> down I/O activities.
>        1.      Disable the NIC.
>        2.      Disable Windows automatic update.
>        3.      Disable the page file of the virtual memory.
>
>        After the VM ran for overnight, the image size for the running VM
> grew up to 3G bytes.
>
>
>        This issue is very easy to be duplicated. Here are the steps.
>
>        1.      Create a qcow2 image of 64-bit Windows 7 VM from the CD.
>        2.      Launch the VM
>        3.      Turn off the VM configurations stated above.
>        4.      Let the VM run.
>        5.      Watch the growth of the VM image size. The longer the VM
> runs, the bigger the size of the VM is.
>
>        During the run-time, you can run “process monitor” to watch guest
> OS’s I/O activities. You can see File I/O, registry I/O from the
>  system during the run-time in this test case. But the growth rate of the
> image size seems to be greater than default system I/O.
>
>
> -----Original Message-----
> From: Stefan Hajnoczi [mailto:stefanha@gmail.com]
> Sent: Monday, May 21, 2012 9:24 PM
> To: Charles.Tsai-蔡清海-研究發展部
> Cc: qemu-devel@nongnu.org; Jonah.Wu-吳君勉-研究發展部
> Subject: Re: [Qemu-devel] The image size of instance VM keeps growing
>
> On Mon, May 21, 2012 at 10:08 AM, Charles.Tsai-蔡清海-研究發展部
> <charles.tsai@cloudena.com> wrote:
> > We can run a tool to see if there is any active I/O or not. Does the
> image size shrink back if an I/O activity stops?
> > What we found here is that the image size kept growing. If this problem
> persists, it could eat up the entire disk space.
>
> It's possible for an image to grow larger than its backing file due to the
> layout and metadata of qcow2 (or any image file format).  But this overhead
> should be fixed to maximum 1 MB, maybe (assuming you are not using qcow2
> snapshots where the file can grow arbitrarily).
>
> If the image file is growing endlessly then there is a problem :).
> The starting point is figuring out which I/O request causes this to happen
> so it can be reproduced and debugged.  Can you isolate the problem to a few
> steps that can be reproduced?
>
> Stefan
>

[-- Attachment #2: Type: text/html, Size: 3913 bytes --]

Re: [Qemu-devel] The image size of instance VM keeps growing

[Stefan Hajnoczi]

On Wed, May 23, 2012 at 11:47 AM, Charles.Tsai-蔡清海-研究發展部
<charles.tsai@cloudena.com> wrote:
>        After the VM ran for overnight, the image size for the running VM grew up to 3G bytes.

What is the size of the backing file?  3G is small for a Windows
guest.  Any sector that is written to could cause 64 KB to be
allocated in the image file.

If qemu-img check sees no inconsistencies then it seems the guest is
writing to previously untouched regions of the disk...causing the
qcow2 to grow.

However, the qcow2 file should not grow much beyond its virtual disk
size.  So if you find the qcow2 is 3G but the virtual disk size is 1G,
then there is a bug.  But from what you've posted so far I suspect the
guest is simply writing to the disk.

Stefan

Re: [Qemu-devel] The image size of instance VM keeps growing

[Michael Tokarev]

On 23.05.2012 16:57, Stefan Hajnoczi wrote:
[]
> However, the qcow2 file should not grow much beyond its virtual disk
> size.  So if you find the qcow2 is 3G but the virtual disk size is 1G,
> then there is a bug.  But from what you've posted so far I suspect the
> guest is simply writing to the disk.

Dont't forget about possible snapshots inside the qcow2 file.  So it
is quite well possible to have qcow2 file sized much larger than virtual
disk size :)

/mjt

Re: [Qemu-devel] The image size of instance VM keeps growing

[Michael Roth]

On Mon, May 21, 2012 at 07:47:49AM +0000, Charles.Tsai-蔡清海-研究發展部 wrote:
> 
> 
> We created a qcow2 based-image for a VM whose size is big when it is created. When a VM is launched, it is a instance VM image which refers to the based-image. Initially, the image size of instance VM is smaller than that of the based-mage. But we found the image size of the instance VM kept growing, when a VM ran in idle state. We doubted this issue is caused by the Windows swap file(virtual memory) that Windows OS pages the memory contents to it during the run-time. We did disable the swap file setting from Windows but the image size of the instance VM still kept growing.
> 
> 
> 
> Ideally, the image size of instance VM is supposed to remain in a smaller size than the based-image. Why does it keep growing in size which is even bigger than that of based-image? Our VM, in fact, did nothing during the idle state.
> 

Do you have automatic indexing turned on in the guest? Not sure of the
specifics WRT to NTFS, but indexing might be causing metadata updates
(particularly last access time) that might be causing a COW on a lot of
blocks in a relatively short amount of time.

Re: [Qemu-devel] The image size of instance VM keeps growing

[Charles.Tsai-蔡清海-研究發展部]

Stefan,

Thank you for information. What we worried is an endless growth of the image size.
If virtual disk size is the ceiling of the max. image size to be grew, we should not have to worry about this issue.


-----Original Message-----
From: Stefan Hajnoczi [mailto:stefanha@gmail.com] 
Sent: Wednesday, May 23, 2012 8:57 PM
To: Charles.Tsai-蔡清海-研究發展部
Cc: qemu-devel@nongnu.org; Jonah.Wu-吳君勉-研究發展部
Subject: Re: [Qemu-devel] The image size of instance VM keeps growing

On Wed, May 23, 2012 at 11:47 AM, Charles.Tsai-蔡清海-研究發展部
<charles.tsai@cloudena.com> wrote:
>        After the VM ran for overnight, the image size for the running VM grew up to 3G bytes.

What is the size of the backing file?  3G is small for a Windows guest.  Any sector that is written to could cause 64 KB to be allocated in the image file.

If qemu-img check sees no inconsistencies then it seems the guest is writing to previously untouched regions of the disk...causing the
qcow2 to grow.

However, the qcow2 file should not grow much beyond its virtual disk size.  So if you find the qcow2 is 3G but the virtual disk size is 1G, then there is a bug.  But from what you've posted so far I suspect the guest is simply writing to the disk.

Stefan

Re: [Qemu-devel] The image size of instance VM keeps growing

[Charles.Tsai-蔡清海-研究發展部]

Thanks. Good alternative view.

-----Original Message-----
From: Michael Tokarev [mailto:mjt@tls.msk.ru] 
Sent: Wednesday, May 23, 2012 9:11 PM
To: Stefan Hajnoczi
Cc: Charles.Tsai-蔡清海-研究發展部; Jonah.Wu-吳君勉-研究發展部; qemu-devel@nongnu.org
Subject: Re: [Qemu-devel] The image size of instance VM keeps growing

On 23.05.2012 16:57, Stefan Hajnoczi wrote:
[]
> However, the qcow2 file should not grow much beyond its virtual disk 
> size.  So if you find the qcow2 is 3G but the virtual disk size is 1G, 
> then there is a bug.  But from what you've posted so far I suspect the 
> guest is simply writing to the disk.

Dont't forget about possible snapshots inside the qcow2 file.  So it is quite well possible to have qcow2 file sized much larger than virtual disk size :)

/mjt

Re: [Qemu-devel] The image size of instance VM keeps growing

[Charles.Tsai-蔡清海-研究發展部]

Stefan,

	Can you explain why a 64KB could be allocated for each sector write?

	Right now, I am testing a VM with 8G virtual disk space.
	The disk only has 12.6 MB free space left when I run the test.
	I am going to see what will happen in this test case.



-----Original Message-----
From: Stefan Hajnoczi [mailto:stefanha@gmail.com] 
Sent: Wednesday, May 23, 2012 8:57 PM
To: Charles.Tsai-蔡清海-研究發展部
Cc: qemu-devel@nongnu.org; Jonah.Wu-吳君勉-研究發展部
Subject: Re: [Qemu-devel] The image size of instance VM keeps growing

On Wed, May 23, 2012 at 11:47 AM, Charles.Tsai-蔡清海-研究發展部
<charles.tsai@cloudena.com> wrote:
>        After the VM ran for overnight, the image size for the running VM grew up to 3G bytes.

What is the size of the backing file?  3G is small for a Windows guest.  Any sector that is written to could cause 64 KB to be allocated in the image file.

If qemu-img check sees no inconsistencies then it seems the guest is writing to previously untouched regions of the disk...causing the
qcow2 to grow.

However, the qcow2 file should not grow much beyond its virtual disk size.  So if you find the qcow2 is 3G but the virtual disk size is 1G, then there is a bug.  But from what you've posted so far I suspect the guest is simply writing to the disk.

Stefan

[Qemu-devel] Tracing message for mu

[Charles.Tsai-蔡清海-研究發展部]

Hi Stefan,

I had a problem to launch the VM using the script that contains the Qemu options. My script used to work before updating the latest Qemu code suggested by Hans. I also tried to latest code from Freedesktop, it still failed to work either.
Obviously, Qemu was broken somewhere in the code. I did try to check the log file of the Qemu and it seemed to be not quite helpful.

In this case, how do I turn on the Qemu trace to see where the code failed? From the Qemu document, I cannot find anything talking about this. Please give me a suggestion so that I can probe this issue here.

Thanks.

======================================================================
On 05/24/2012 04:36 PM, Charles.Tsai-蔡清海-研究發展部 wrote:
> Hans,
>
> I was talking about the bug of the Qemu command options found in your source code.
> Do you have fix for that?

I know you are having issues, but I've been unable to reproduce using the exact commandline you've given me, so I think something is wrong with your installation.

Have you tried stracing qemu? I think it simply cannot find a file.

Regards,

Hans

Re: [Qemu-devel] Tracing message for mu

[Stefan Hajnoczi]

On Fri, May 25, 2012 at 3:42 AM, Charles.Tsai-蔡清海-研究發展部
<charles.tsai@cloudena.com> wrote:
> I had a problem to launch the VM using the script that contains the Qemu options. My script used to work before updating the latest Qemu code suggested by Hans. I also tried to latest code from Freedesktop, it still failed to work either.
> Obviously, Qemu was broken somewhere in the code. I did try to check the log file of the Qemu and it seemed to be not quite helpful.
>
> In this case, how do I turn on the Qemu trace to see where the code failed? From the Qemu document, I cannot find anything talking about this. Please give me a suggestion so that I can probe this issue here.

Please provide details when asking for help:
1. The full QEMU command-line used to launch the guest.
2. The exact error message and where it was printed (inside the guest
or by QEMU itself).
3. Which version of QEMU or which git commit you are testing.

Since you mentioned Hans and Freedesktop I guess this has something to
do with SPICE and QXL.

QEMU tracing does not provide much debug info on startup - it's more
useful once the guest is running and you need to understand the I/O
requests and other operations that are going on.  There is a
quickstart guide in docs/tracing.txt if you want to try it.

Stefan

Re: [Qemu-devel] The image size of instance VM keeps growing

[Charles.Tsai-蔡清海-研究發展部]

Stefan,

	I made two test cases here.

	1) case one:
		A 32-bit Windows 7 VM with 8G virtual disk size.
		In this test, the virtual disk is almost full.
		Like what you said, the instance VM size only increased 2% initially and only a little when virtual disk size is almost 			full.

	2) case two: (we think it is a bug)

		A 64-bit Windows 7 VM with 64G virtual disk size.
		Base image size: 19G

		Here is what I found

		The increase size of instance VM is significantly different from the decrease size of virtual disk.
		The decrease size of virtual disk stands for the new data written into the disk by the OS.

		Here are the readings I collected at differ time slot.

		1)
		Instance VM size: 1.6G
		Virtual disk free space: 45792534528 (read from guest OS)

		2)
		Instance VM size: 2.1 G
		Virtual disk free space: 45786300416 (read from guest OS)	

		You can find that the instance VM increase 0.5G but virtual disk space only reduces around 6MB.
		This process will continue as long as the VM is running.

		What do you think about this? Let me know if you need more explanations from me.



-----Original Message-----
From: Charles.Tsai-蔡清海-研究發展部 
Sent: Thursday, May 24, 2012 6:41 PM
To: 'Stefan Hajnoczi'
Cc: qemu-devel@nongnu.org; Jonah.Wu-吳君勉-研究發展部
Subject: RE: [Qemu-devel] The image size of instance VM keeps growing

Stefan,

	Can you explain why a 64KB could be allocated for each sector write?

	Right now, I am testing a VM with 8G virtual disk space.
	The disk only has 12.6 MB free space left when I run the test.
	I am going to see what will happen in this test case.



-----Original Message-----
From: Stefan Hajnoczi [mailto:stefanha@gmail.com] 
Sent: Wednesday, May 23, 2012 8:57 PM
To: Charles.Tsai-蔡清海-研究發展部
Cc: qemu-devel@nongnu.org; Jonah.Wu-吳君勉-研究發展部
Subject: Re: [Qemu-devel] The image size of instance VM keeps growing

On Wed, May 23, 2012 at 11:47 AM, Charles.Tsai-蔡清海-研究發展部
<charles.tsai@cloudena.com> wrote:
>        After the VM ran for overnight, the image size for the running VM grew up to 3G bytes.

What is the size of the backing file?  3G is small for a Windows guest.  Any sector that is written to could cause 64 KB to be allocated in the image file.

If qemu-img check sees no inconsistencies then it seems the guest is writing to previously untouched regions of the disk...causing the
qcow2 to grow.

However, the qcow2 file should not grow much beyond its virtual disk size.  So if you find the qcow2 is 3G but the virtual disk size is 1G, then there is a bug.  But from what you've posted so far I suspect the guest is simply writing to the disk.

Stefan

Re: [Qemu-devel] The image size of instance VM keeps growing

[Stefan Hajnoczi]

On Wed, May 23, 2012 at 2:11 PM, Michael Tokarev <mjt@tls.msk.ru> wrote:
> On 23.05.2012 16:57, Stefan Hajnoczi wrote:
> []
>> However, the qcow2 file should not grow much beyond its virtual disk
>> size.  So if you find the qcow2 is 3G but the virtual disk size is 1G,
>> then there is a bug.  But from what you've posted so far I suspect the
>> guest is simply writing to the disk.
>
> Dont't forget about possible snapshots inside the qcow2 file.  So it
> is quite well possible to have qcow2 file sized much larger than virtual
> disk size :)

Right.  I said this earlier: "(assuming you are not using qcow2
snapshots where the file can grow arbitrarily)"

Re: [Qemu-devel] The image size of instance VM keeps growing

[Stefan Hajnoczi]

On Thu, May 24, 2012 at 11:40 AM, Charles.Tsai-蔡清海-研究發展部
<charles.tsai@cloudena.com> wrote:
> Stefan,
>
>        Can you explain why a 64KB could be allocated for each sector write?

The default cluster size in qcow2 is 64 KB.  qcow2 manages space in 64
KB pieces.  Therefore, when you write to a region of the image that
has not been allocated yet, you incur the cost of allocating a new 64
KB cluster.

>        Right now, I am testing a VM with 8G virtual disk space.
>        The disk only has 12.6 MB free space left when I run the test.
>        I am going to see what will happen in this test case.

How much space is free on the file system inside the disk image has
nothing to do with the growth of the qcow2 image file on the host.

When you using a backing file in qocw2 you start like this:

backing.img <-- vm.qcow2 (empty)

The qcow2 file is empty, it should be very small (<1 MB) when you boot
the guest for the first time.

Now that the guest is running, every disk write does the following:

if this 64 KB cluster in the VM disk has not been touched yet:
    allocate the 64 KB cluster and write the data (the image file
grows by 64 KB)
else:
    write the data in-place because we already have a cluster (no
allocation necessary and the image file does not grow)

The important thing to understand is that "has this cluster been
touched?" means "have we written to this cluster in vm.qcow2
previously?".  When you start the guest for the first time vm.qcow2 is
empty, so the first write is guaranteed to allocate a cluster
(remember we never write into the backing file, so we need to write
into vm.qcow2).

Stefan

Re: [Qemu-devel] The image size of instance VM keeps growing

[Stefan Hajnoczi]

On Fri, May 25, 2012 at 10:20 AM, Charles.Tsai-蔡清海-研究發展部
<charles.tsai@cloudena.com> wrote:
> Stefan,
>
>        I made two test cases here.
>
>        1) case one:
>                A 32-bit Windows 7 VM with 8G virtual disk size.
>                In this test, the virtual disk is almost full.
>                Like what you said, the instance VM size only increased 2% initially and only a little when virtual disk size is almost                         full.
>
>        2) case two: (we think it is a bug)
>
>                A 64-bit Windows 7 VM with 64G virtual disk size.
>                Base image size: 19G
>
>                Here is what I found
>
>                The increase size of instance VM is significantly different from the decrease size of virtual disk.
>                The decrease size of virtual disk stands for the new data written into the disk by the OS.
>
>                Here are the readings I collected at differ time slot.
>
>                1)
>                Instance VM size: 1.6G
>                Virtual disk free space: 45792534528 (read from guest OS)
>
>                2)
>                Instance VM size: 2.1 G
>                Virtual disk free space: 45786300416 (read from guest OS)
>
>                You can find that the instance VM increase 0.5G but virtual disk space only reduces around 6MB.
>                This process will continue as long as the VM is running.

This seems normal.  The relationship between free space inside the
guest and qcow2 image file size is not 1:1 - in fact it's arbitrary
and depends completely on your file system, partitioning, etc.

But let's assume for a second that free space inside the guest
indicates the amount of data written into the image file:

free space consumed = 45792534528 - 45786300416 = 6234112 = ~6 MB

In the worst case each 512-byte sector of those 6 MB is in a separate
64 KB cluster region.  In other words:

for (i = 0; i < 6234112 / 512; i++) {
    offset = i * 65536;
    pwrite(disk, buf, 512, offset);
}

Let's calculate how much allocation this causes in the qcow2 image file:

(6234112 / 512) * 65536 = 797966336 = 761 MB

This is the worst case.  In practice the I/O pattern will probably not
touch a new cluster every time and the qcow2 file will grow a lot
less.

So we have estimated the worst case qcow2 image file growth for 6 MB
of writes.  The result is 761 MB.  It is larger than what you observed
so your results can be explained as expected behavior.

Stefan

Re: [Qemu-devel] Tracing message for mu

[Charles.Tsai-蔡清海-研究發展部]

Stefan,

Hans replied me back and he suggested me to use strace.
Hopefully, this issue can be resolved.

Additionally, we encountered a tough issue right now.
When we ran a 64-bit Windows 7 VM overnight, the kvm process was killed for some reasons.

Here is the qemu version I copied from the screen

"QEMU emulator version 1.0 (qemu-kvm-devel), Copyright (c) 2003-2008 Fabrice Bellard"

The Qemu source is from Hans. Any suggestion to attack this issue?


-----Original Message-----
From: Stefan Hajnoczi [mailto:stefanha@gmail.com] 
Sent: Friday, May 25, 2012 4:13 PM
To: Charles.Tsai-蔡清海-研究發展部
Cc: qemu-devel@nongnu.org; Jonah.Wu-吳君勉-研究發展部; Hans de Goede
Subject: Re: Tracing message for mu

On Fri, May 25, 2012 at 3:42 AM, Charles.Tsai-蔡清海-研究發展部
<charles.tsai@cloudena.com> wrote:
> I had a problem to launch the VM using the script that contains the Qemu options. My script used to work before updating the latest Qemu code suggested by Hans. I also tried to latest code from Freedesktop, it still failed to work either.
> Obviously, Qemu was broken somewhere in the code. I did try to check the log file of the Qemu and it seemed to be not quite helpful.
>
> In this case, how do I turn on the Qemu trace to see where the code failed? From the Qemu document, I cannot find anything talking about this. Please give me a suggestion so that I can probe this issue here.

Please provide details when asking for help:
1. The full QEMU command-line used to launch the guest.
2. The exact error message and where it was printed (inside the guest or by QEMU itself).
3. Which version of QEMU or which git commit you are testing.

Since you mentioned Hans and Freedesktop I guess this has something to do with SPICE and QXL.

QEMU tracing does not provide much debug info on startup - it's more useful once the guest is running and you need to understand the I/O requests and other operations that are going on.  There is a quickstart guide in docs/tracing.txt if you want to try it.

Stefan

Re: [Qemu-devel] Tracing message for mu

[Stefan Hajnoczi]

On Sat, May 26, 2012 at 7:50 AM, Charles.Tsai-蔡清海-研究發展部
<charles.tsai@cloudena.com> wrote:
> Hans replied me back and he suggested me to use strace.
> Hopefully, this issue can be resolved.
>
> Additionally, we encountered a tough issue right now.
> When we ran a 64-bit Windows 7 VM overnight, the kvm process was killed for some reasons.
>
> Here is the qemu version I copied from the screen
>
> "QEMU emulator version 1.0 (qemu-kvm-devel), Copyright (c) 2003-2008 Fabrice Bellard"
>
> The Qemu source is from Hans. Any suggestion to attack this issue?

Next time ensure that core dumps are enabled:
$ ulimit -c unlimited  # enable core dumps
$ qemu-kvm ...

If it crashes you'll have a core file that the gdb debugger can process.

It's also worth checking dmesg(1) output to see if your system ran out
of memory and the kernel killed the qemu-kvm process to free memory.

Hope this helps,
Stefan