* Problems KVM-84
@ 2009-02-27 0:58 Jay Mann
2009-02-27 6:52 ` Thomas Mueller
` (2 more replies)
0 siblings, 3 replies; 20+ messages in thread
From: Jay Mann @ 2009-02-27 0:58 UTC (permalink / raw)
To: kvm
Hi,
I just downloaded built and installed kvm-84 on ubuntu Hardy x64 2.6.24-23-
server and I’m getting the following 2 problems that did not exists in kvm-83.
1. The qemu emulater (bios screen) takes a long time to start (~10
seconds), and subsequently Libvirt times out when I try to start a guest VM
from virsh.
2. In my windows xp guest, a new PCI RAM (Memory) Controller is found, but
windows doesn’t know what driver to install (and neither do i)
Is anyone else experiencing these issues or are there any workarounds?
Thanks,
-J
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Problems KVM-84
2009-02-27 0:58 Problems KVM-84 Jay Mann
@ 2009-02-27 6:52 ` Thomas Mueller
2009-03-09 10:22 ` Avi Kivity
2009-03-11 11:15 ` [Qemu-devel] " Avi Kivity
2 siblings, 0 replies; 20+ messages in thread
From: Thomas Mueller @ 2009-02-27 6:52 UTC (permalink / raw)
To: kvm
>
> I just downloaded built and installed kvm-84 on ubuntu Hardy x64
> 2.6.24-23- server and I’m getting the following 2 problems that did not
> exists in kvm-83.
>
> 1. The qemu emulater (bios screen) takes a long time to start (~10
> seconds), and subsequently Libvirt times out when I try to start a guest
> VM from virsh.
also seen on debian lenny with kvm-84 and kernel 2.6.26. starting a vm
takes about 10s (with kvm <= 83 ~1s) until the bios screen shows up.
- Thomas
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Problems KVM-84
2009-02-27 0:58 Problems KVM-84 Jay Mann
2009-02-27 6:52 ` Thomas Mueller
@ 2009-03-09 10:22 ` Avi Kivity
2009-03-09 12:19 ` jmandawg
2009-03-11 11:15 ` [Qemu-devel] " Avi Kivity
2 siblings, 1 reply; 20+ messages in thread
From: Avi Kivity @ 2009-03-09 10:22 UTC (permalink / raw)
To: Jay Mann; +Cc: kvm
Jay Mann wrote:
> Hi,
>
> I just downloaded built and installed kvm-84 on ubuntu Hardy x64 2.6.24-23-
> server and I’m getting the following 2 problems that did not exists in kvm-83.
>
> 1. The qemu emulater (bios screen) takes a long time to start (~10
> seconds), and subsequently Libvirt times out when I try to start a guest VM
> from virsh.
>
Are you using a qcow image? How large is it?
> 2. In my windows xp guest, a new PCI RAM (Memory) Controller is found, but
> windows doesn’t know what driver to install (and neither do i)
>
That's the virtio-console device. I guess we should disable it by default.
--
error compiling committee.c: too many arguments to function
^ permalink raw reply [flat|nested] 20+ messages in thread
* RE: Problems KVM-84
@ 2009-03-09 12:19 ` jmandawg
2009-03-09 12:37 ` Avi Kivity
0 siblings, 1 reply; 20+ messages in thread
From: jmandawg @ 2009-03-09 12:19 UTC (permalink / raw)
To: 'Avi Kivity'; +Cc: kvm
> Are you using a qcow image? How large is it?
Yes, it's 33GB. But with kvm-83 it starts right up with no delay.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Problems KVM-84
2009-03-09 12:19 ` jmandawg
@ 2009-03-09 12:37 ` Avi Kivity
[not found] ` <BAY102-W25DF72B635B00AD337130D2A00@phx.gbl>
0 siblings, 1 reply; 20+ messages in thread
From: Avi Kivity @ 2009-03-09 12:37 UTC (permalink / raw)
To: jmandawg; +Cc: kvm
jmandawg wrote:
>> Are you using a qcow image? How large is it?
>>
>
> Yes, it's 33GB. But with kvm-83 it starts right up with no delay.
>
Can you run qemu like this:
strace -c -fF qemu ...
And kill qemu immediately when the screen shows up? Then post the results.
--
error compiling committee.c: too many arguments to function
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Problems KVM-84
[not found] ` <BAY102-W25DF72B635B00AD337130D2A00@phx.gbl>
@ 2009-03-10 8:39 ` Avi Kivity
[not found] ` <BAY102-W2139785413083807492C85D2A10@phx.gbl>
0 siblings, 1 reply; 20+ messages in thread
From: Avi Kivity @ 2009-03-10 8:39 UTC (permalink / raw)
To: J- MAN; +Cc: kvm
J- MAN wrote:
> Also the second time i run the command, it only takes around 2-3
> seconds for the window to open.
> ------------------------------------------------------------------------
I don't think this is a regression from kvm-83. Large qcow2 files have
lots of metadata which needs to be loaded up front. If the kernel
doesn't have this cached, it will take a long time.
--
error compiling committee.c: too many arguments to function
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Problems KVM-84
[not found] ` <BAY102-W2139785413083807492C85D2A10@phx.gbl>
@ 2009-03-10 13:47 ` Avi Kivity
0 siblings, 0 replies; 20+ messages in thread
From: Avi Kivity @ 2009-03-10 13:47 UTC (permalink / raw)
To: J- MAN; +Cc: kvm
J- MAN wrote:
>
> > I don't think this is a regression from kvm-83. Large qcow2 files have
> > lots of metadata which needs to be loaded up front. If the kernel
> > doesn't have this cached, it will take a long time.
>
> I'm just wondering what happend between 83 and 84, because there is no
> startup delay in 83.
> My real problem is that libvirt times out when waiting for my virtual
> machine to start using 84, unless i manually start the machine first
> to get it cached.
>
> This makes using libvirt & virt-manager useless with 84 unless i
> pre-start all my VMs manually to get them cached.
>
> I'm suprised nobody else is having this issue. If there is anything
> else i can do to help please let me know.
Try (with kvm-83):
echo 255 > /proc/sys/vm/drop-caches
qemu ...
--
error compiling committee.c: too many arguments to function
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Problems KVM-84
2009-02-27 0:58 Problems KVM-84 Jay Mann
@ 2009-03-11 11:15 ` Avi Kivity
2009-03-09 10:22 ` Avi Kivity
2009-03-11 11:15 ` [Qemu-devel] " Avi Kivity
2 siblings, 0 replies; 20+ messages in thread
From: Avi Kivity @ 2009-03-11 11:15 UTC (permalink / raw)
To: Jay Mann; +Cc: kvm, Uri Lublin, qemu-devel, Anthony Liguori
Jay Mann wrote:
> Hi,
>
> I just downloaded built and installed kvm-84 on ubuntu Hardy x64 2.6.24-23-
> server and I’m getting the following 2 problems that did not exists in kvm-83.
>
> 1. The qemu emulater (bios screen) takes a long time to start (~10
> seconds), and subsequently Libvirt times out when I try to start a guest VM
> from virsh.
>
This is caused by qemu r6404:
commit 5d4cbd78aa33f6d034a62207c99ad0b64af44621
Author: aliguori <aliguori@c046a42c-6fe2-441c-8c8c-71466251a162>
Date: Thu Jan 22 18:57:22 2009 +0000
block-qcow2: keep highest allocated byte (Uri Lublin)
We want to know the highest written offset for qcow2 images.
This gives a pretty good (and easy to calculate) estimation to how
much more allocation can be done for the block device.
It can be usefull for allocating more diskspace for that image
(if possible, e.g. lvm) before we run out-of-disk-space
Signed-off-by: Uri Lublin <uril@redhat.com>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
Scanning the file at startup is slow. We need to find a better way.
--
error compiling committee.c: too many arguments to function
^ permalink raw reply [flat|nested] 20+ messages in thread
* [Qemu-devel] Re: Problems KVM-84
@ 2009-03-11 11:15 ` Avi Kivity
0 siblings, 0 replies; 20+ messages in thread
From: Avi Kivity @ 2009-03-11 11:15 UTC (permalink / raw)
To: Jay Mann; +Cc: Uri Lublin, qemu-devel, kvm
Jay Mann wrote:
> Hi,
>
> I just downloaded built and installed kvm-84 on ubuntu Hardy x64 2.6.24-23-
> server and I’m getting the following 2 problems that did not exists in kvm-83.
>
> 1. The qemu emulater (bios screen) takes a long time to start (~10
> seconds), and subsequently Libvirt times out when I try to start a guest VM
> from virsh.
>
This is caused by qemu r6404:
commit 5d4cbd78aa33f6d034a62207c99ad0b64af44621
Author: aliguori <aliguori@c046a42c-6fe2-441c-8c8c-71466251a162>
Date: Thu Jan 22 18:57:22 2009 +0000
block-qcow2: keep highest allocated byte (Uri Lublin)
We want to know the highest written offset for qcow2 images.
This gives a pretty good (and easy to calculate) estimation to how
much more allocation can be done for the block device.
It can be usefull for allocating more diskspace for that image
(if possible, e.g. lvm) before we run out-of-disk-space
Signed-off-by: Uri Lublin <uril@redhat.com>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
Scanning the file at startup is slow. We need to find a better way.
--
error compiling committee.c: too many arguments to function
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Problems KVM-84
2009-03-11 11:15 ` [Qemu-devel] " Avi Kivity
@ 2009-03-11 14:12 ` Anthony Liguori
-1 siblings, 0 replies; 20+ messages in thread
From: Anthony Liguori @ 2009-03-11 14:12 UTC (permalink / raw)
To: Avi Kivity; +Cc: Jay Mann, kvm, Uri Lublin, qemu-devel
Avi Kivity wrote:
> Jay Mann wrote:
>> Hi,
>>
>> I just downloaded built and installed kvm-84 on ubuntu Hardy x64
>> 2.6.24-23-
>> server and I’m getting the following 2 problems that did not exists
>> in kvm-83.
>>
>> 1. The qemu emulater (bios screen) takes a long time to start (~10
>> seconds), and subsequently Libvirt times out when I try to start a
>> guest VM from virsh.
>
> This is caused by qemu r6404:
>
> commit 5d4cbd78aa33f6d034a62207c99ad0b64af44621
> Author: aliguori <aliguori@c046a42c-6fe2-441c-8c8c-71466251a162>
> Date: Thu Jan 22 18:57:22 2009 +0000
>
> block-qcow2: keep highest allocated byte (Uri Lublin)
> We want to know the highest written offset for qcow2 images.
> This gives a pretty good (and easy to calculate) estimation to how
> much more allocation can be done for the block device.
> It can be usefull for allocating more diskspace for that image
> (if possible, e.g. lvm) before we run out-of-disk-space
> Signed-off-by: Uri Lublin <uril@redhat.com>
> Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
>
> Scanning the file at startup is slow. We need to find a better way.
Any quick ideas? Seems like this is broken by design. Unless we can
find a quick fix, I'm going to revert this in the stable tree.
Regards,
Anthony Liguori
^ permalink raw reply [flat|nested] 20+ messages in thread
* [Qemu-devel] Re: Problems KVM-84
@ 2009-03-11 14:12 ` Anthony Liguori
0 siblings, 0 replies; 20+ messages in thread
From: Anthony Liguori @ 2009-03-11 14:12 UTC (permalink / raw)
To: Avi Kivity; +Cc: Jay Mann, Uri Lublin, qemu-devel, kvm
Avi Kivity wrote:
> Jay Mann wrote:
>> Hi,
>>
>> I just downloaded built and installed kvm-84 on ubuntu Hardy x64
>> 2.6.24-23-
>> server and I’m getting the following 2 problems that did not exists
>> in kvm-83.
>>
>> 1. The qemu emulater (bios screen) takes a long time to start (~10
>> seconds), and subsequently Libvirt times out when I try to start a
>> guest VM from virsh.
>
> This is caused by qemu r6404:
>
> commit 5d4cbd78aa33f6d034a62207c99ad0b64af44621
> Author: aliguori <aliguori@c046a42c-6fe2-441c-8c8c-71466251a162>
> Date: Thu Jan 22 18:57:22 2009 +0000
>
> block-qcow2: keep highest allocated byte (Uri Lublin)
> We want to know the highest written offset for qcow2 images.
> This gives a pretty good (and easy to calculate) estimation to how
> much more allocation can be done for the block device.
> It can be usefull for allocating more diskspace for that image
> (if possible, e.g. lvm) before we run out-of-disk-space
> Signed-off-by: Uri Lublin <uril@redhat.com>
> Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
>
> Scanning the file at startup is slow. We need to find a better way.
Any quick ideas? Seems like this is broken by design. Unless we can
find a quick fix, I'm going to revert this in the stable tree.
Regards,
Anthony Liguori
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] Re: Problems KVM-84
2009-03-11 14:12 ` [Qemu-devel] " Anthony Liguori
(?)
@ 2009-03-11 14:52 ` Avi Kivity
2009-03-11 20:19 ` Anthony Liguori
-1 siblings, 1 reply; 20+ messages in thread
From: Avi Kivity @ 2009-03-11 14:52 UTC (permalink / raw)
To: qemu-devel; +Cc: Jay Mann, Uri Lublin, kvm
Anthony Liguori wrote:
>>
>> Scanning the file at startup is slow. We need to find a better way.
>
> Any quick ideas? Seems like this is broken by design.
I agree.
> Unless we can find a quick fix, I'm going to revert this in the stable
> tree.
We could cache the value in some unused area. Set it up in qemu-img.
Don't support it on older images.
But that's not a quick fix; I'd suggest reverting it in both stable and
mainline, and re-applying in mainline when there's a fix.
--
error compiling committee.c: too many arguments to function
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] Re: Problems KVM-84
2009-03-11 14:12 ` [Qemu-devel] " Anthony Liguori
@ 2009-03-11 16:06 ` Jamie Lokier
-1 siblings, 0 replies; 20+ messages in thread
From: Jamie Lokier @ 2009-03-11 16:06 UTC (permalink / raw)
To: qemu-devel; +Cc: Avi Kivity, Jay Mann, Uri Lublin, kvm
Anthony Liguori wrote:
> > block-qcow2: keep highest allocated byte (Uri Lublin)
> > We want to know the highest written offset for qcow2 images.
> > This gives a pretty good (and easy to calculate) estimation to how
> > much more allocation can be done for the block device.
> > It can be usefull for allocating more diskspace for that image
> > (if possible, e.g. lvm) before we run out-of-disk-space
> > Signed-off-by: Uri Lublin <uril@redhat.com>
> > Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
> >
> >Scanning the file at startup is slow. We need to find a better way.
>
> Any quick ideas? Seems like this is broken by design. Unless we can
> find a quick fix, I'm going to revert this in the stable tree.
I still don't understand the point in that feature.
You have the size of the qcow2 file (how much data is used), and the
size of the image it's representing (how much it could expand to).
What does the highest written offset help you anticipate?
Many guest filesystems write data all over the block device, not
concentrated at the start. (E.g. ext2/3/4 and it's block groups).
-- Jamie
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] Re: Problems KVM-84
@ 2009-03-11 16:06 ` Jamie Lokier
0 siblings, 0 replies; 20+ messages in thread
From: Jamie Lokier @ 2009-03-11 16:06 UTC (permalink / raw)
To: qemu-devel; +Cc: Jay Mann, Uri Lublin, Avi Kivity, kvm
Anthony Liguori wrote:
> > block-qcow2: keep highest allocated byte (Uri Lublin)
> > We want to know the highest written offset for qcow2 images.
> > This gives a pretty good (and easy to calculate) estimation to how
> > much more allocation can be done for the block device.
> > It can be usefull for allocating more diskspace for that image
> > (if possible, e.g. lvm) before we run out-of-disk-space
> > Signed-off-by: Uri Lublin <uril@redhat.com>
> > Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
> >
> >Scanning the file at startup is slow. We need to find a better way.
>
> Any quick ideas? Seems like this is broken by design. Unless we can
> find a quick fix, I'm going to revert this in the stable tree.
I still don't understand the point in that feature.
You have the size of the qcow2 file (how much data is used), and the
size of the image it's representing (how much it could expand to).
What does the highest written offset help you anticipate?
Many guest filesystems write data all over the block device, not
concentrated at the start. (E.g. ext2/3/4 and it's block groups).
-- Jamie
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Problems KVM-84
2009-03-11 14:12 ` [Qemu-devel] " Anthony Liguori
@ 2009-03-11 18:37 ` Uri Lublin
-1 siblings, 0 replies; 20+ messages in thread
From: Uri Lublin @ 2009-03-11 18:37 UTC (permalink / raw)
To: Anthony Liguori; +Cc: Avi Kivity, Jay Mann, kvm, Uri Lublin, qemu-devel
[-- Attachment #1: Type: text/plain, Size: 2041 bytes --]
Anthony Liguori wrote:
> Avi Kivity wrote:
>> Jay Mann wrote:
>>> Hi,
>>>
>>> I just downloaded built and installed kvm-84 on ubuntu Hardy x64
>>> 2.6.24-23-
>>> server and I’m getting the following 2 problems that did not exists
>>> in kvm-83.
>>>
>>> 1. The qemu emulater (bios screen) takes a long time to start (~10
>>> seconds), and subsequently Libvirt times out when I try to start a
>>> guest VM from virsh.
>>
>> This is caused by qemu r6404:
>>
>> commit 5d4cbd78aa33f6d034a62207c99ad0b64af44621
>> Author: aliguori <aliguori@c046a42c-6fe2-441c-8c8c-71466251a162>
>> Date: Thu Jan 22 18:57:22 2009 +0000
>>
>> block-qcow2: keep highest allocated byte (Uri Lublin)
>> We want to know the highest written offset for qcow2 images.
>> This gives a pretty good (and easy to calculate) estimation to how
>> much more allocation can be done for the block device.
>> It can be usefull for allocating more diskspace for that image
>> (if possible, e.g. lvm) before we run out-of-disk-space
>> Signed-off-by: Uri Lublin <uril@redhat.com>
>> Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
>>
>> Scanning the file at startup is slow. We need to find a better way.
>
> Any quick ideas? Seems like this is broken by design. Unless we can
> find a quick fix, I'm going to revert this in the stable tree.
>
> Regards,
>
> Anthony Liguori
>
We only need to scan the given filename (no backing files). We can pass a flag
to qcow_open (bdrv_open of bs->backing_hd) specifying it's a backing file.
(attached 2 patches). That makes things better for small images with big backing
files. It does not fix the problem for very large images.
A better solution may we to allocate a qcow2-extension to keep highest-alloc and
num-free-bytes so we don't have to scan refcount table of the image upon open.
With that solution qcow_create initializes them, qcow_open reads them and
bdrv_close updates them. We can also add a qemu-img command to scan and update
those values.
Regards,
Uri.
[-- Attachment #2: 0001-block-pass-BDRV_BACKING-flag-to-backing-file-open.patch --]
[-- Type: text/x-patch, Size: 1782 bytes --]
>From 1ccf1940e0b45a9001b916bc6160c03a098a5d1d Mon Sep 17 00:00:00 2001
From: Uri Lublin <uril@redhat.com>
Date: Wed, 11 Mar 2009 20:16:23 +0200
Subject: [PATCH] block: pass BDRV_BACKING flag to backing-file open
With this information, open can behave differently for the image
filename and its backing files.
Signed-off-by: Uri Lublin <uril@redhat.com>
---
qemu/block.c | 3 ++-
qemu/block.h | 2 ++
2 files changed, 4 insertions(+), 1 deletions(-)
diff --git a/qemu/block.c b/qemu/block.c
index 4c556ec..45fa5f4 100644
--- a/qemu/block.c
+++ b/qemu/block.c
@@ -397,7 +397,7 @@ int bdrv_open2(BlockDriverState *bs, const char *filename, int flags,
/* Note: for compatibility, we open disk image files as RDWR, and
RDONLY as fallback */
if (!(flags & BDRV_O_FILE))
- open_flags = BDRV_O_RDWR | (flags & BDRV_O_CACHE_MASK);
+ open_flags = BDRV_O_RDWR | (flags & (BDRV_O_CACHE_MASK | BDRV_O_BACKING));
else
open_flags = flags & ~(BDRV_O_FILE | BDRV_O_SNAPSHOT);
ret = drv->bdrv_open(bs, filename, open_flags);
@@ -429,6 +429,7 @@ int bdrv_open2(BlockDriverState *bs, const char *filename, int flags,
}
path_combine(backing_filename, sizeof(backing_filename),
filename, bs->backing_file);
+ open_flags |= BDRV_O_BACKING;
if (bdrv_open(bs->backing_hd, backing_filename, open_flags) < 0)
goto fail;
}
diff --git a/qemu/block.h b/qemu/block.h
index e1927dd..ff70d64 100644
--- a/qemu/block.h
+++ b/qemu/block.h
@@ -56,6 +56,8 @@ typedef struct QEMUSnapshotInfo {
#define BDRV_O_CACHE_MASK (BDRV_O_NOCACHE | BDRV_O_CACHE_WB | BDRV_O_CACHE_DEF)
+#define BDRV_O_BACKING 0x0100
+
void bdrv_info(void);
void bdrv_info_stats(void);
--
1.6.0.6
[-- Attachment #3: 0002-block-qcow2-do-not-scan-refcounts-when-opening-a-ba.patch --]
[-- Type: text/x-patch, Size: 930 bytes --]
>From c618b293a64d1690105505d7cb28ba1ca8dd33c7 Mon Sep 17 00:00:00 2001
From: Uri Lublin <uril@redhat.com>
Date: Wed, 11 Mar 2009 20:18:23 +0200
Subject: [PATCH] block-qcow2: do not scan refcounts when opening a backing file
It takes too long and is not needed.
Signed-off-by: Uri Lublin <uril@redhat.com>
---
qemu/block-qcow2.c | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/qemu/block-qcow2.c b/qemu/block-qcow2.c
index 465dcd6..41cdbe9 100644
--- a/qemu/block-qcow2.c
+++ b/qemu/block-qcow2.c
@@ -276,7 +276,8 @@ static int qcow_open(BlockDriverState *bs, const char *filename, int flags)
if (refcount_init(bs) < 0)
goto fail;
- scan_refcount(bs, &s->highest_alloc, &s->nc_free);
+ if ((flags & BDRV_O_BACKING) == 0)
+ scan_refcount(bs, &s->highest_alloc, &s->nc_free);
/* read the backing file name */
if (header.backing_file_offset != 0) {
--
1.6.0.6
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [Qemu-devel] Re: Problems KVM-84
@ 2009-03-11 18:37 ` Uri Lublin
0 siblings, 0 replies; 20+ messages in thread
From: Uri Lublin @ 2009-03-11 18:37 UTC (permalink / raw)
To: Anthony Liguori; +Cc: Jay Mann, Uri Lublin, Avi Kivity, kvm, qemu-devel
[-- Attachment #1: Type: text/plain, Size: 2093 bytes --]
Anthony Liguori wrote:
> Avi Kivity wrote:
>> Jay Mann wrote:
>>> Hi,
>>>
>>> I just downloaded built and installed kvm-84 on ubuntu Hardy x64
>>> 2.6.24-23-
>>> server and I’m getting the following 2 problems that did not exists
>>> in kvm-83.
>>>
>>> 1. The qemu emulater (bios screen) takes a long time to start (~10
>>> seconds), and subsequently Libvirt times out when I try to start a
>>> guest VM from virsh.
>>
>> This is caused by qemu r6404:
>>
>> commit 5d4cbd78aa33f6d034a62207c99ad0b64af44621
>> Author: aliguori <aliguori@c046a42c-6fe2-441c-8c8c-71466251a162>
>> Date: Thu Jan 22 18:57:22 2009 +0000
>>
>> block-qcow2: keep highest allocated byte (Uri Lublin)
>> We want to know the highest written offset for qcow2 images.
>> This gives a pretty good (and easy to calculate) estimation to how
>> much more allocation can be done for the block device.
>> It can be usefull for allocating more diskspace for that image
>> (if possible, e.g. lvm) before we run out-of-disk-space
>> Signed-off-by: Uri Lublin <uril@redhat.com>
>> Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
>>
>> Scanning the file at startup is slow. We need to find a better way.
>
> Any quick ideas? Seems like this is broken by design. Unless we can
> find a quick fix, I'm going to revert this in the stable tree.
>
> Regards,
>
> Anthony Liguori
>
We only need to scan the given filename (no backing files). We can pass a flag
to qcow_open (bdrv_open of bs->backing_hd) specifying it's a backing file.
(attached 2 patches). That makes things better for small images with big backing
files. It does not fix the problem for very large images.
A better solution may we to allocate a qcow2-extension to keep highest-alloc and
num-free-bytes so we don't have to scan refcount table of the image upon open.
With that solution qcow_create initializes them, qcow_open reads them and
bdrv_close updates them. We can also add a qemu-img command to scan and update
those values.
Regards,
Uri.
[-- Attachment #2: 0001-block-pass-BDRV_BACKING-flag-to-backing-file-open.patch --]
[-- Type: text/x-patch, Size: 1782 bytes --]
>From 1ccf1940e0b45a9001b916bc6160c03a098a5d1d Mon Sep 17 00:00:00 2001
From: Uri Lublin <uril@redhat.com>
Date: Wed, 11 Mar 2009 20:16:23 +0200
Subject: [PATCH] block: pass BDRV_BACKING flag to backing-file open
With this information, open can behave differently for the image
filename and its backing files.
Signed-off-by: Uri Lublin <uril@redhat.com>
---
qemu/block.c | 3 ++-
qemu/block.h | 2 ++
2 files changed, 4 insertions(+), 1 deletions(-)
diff --git a/qemu/block.c b/qemu/block.c
index 4c556ec..45fa5f4 100644
--- a/qemu/block.c
+++ b/qemu/block.c
@@ -397,7 +397,7 @@ int bdrv_open2(BlockDriverState *bs, const char *filename, int flags,
/* Note: for compatibility, we open disk image files as RDWR, and
RDONLY as fallback */
if (!(flags & BDRV_O_FILE))
- open_flags = BDRV_O_RDWR | (flags & BDRV_O_CACHE_MASK);
+ open_flags = BDRV_O_RDWR | (flags & (BDRV_O_CACHE_MASK | BDRV_O_BACKING));
else
open_flags = flags & ~(BDRV_O_FILE | BDRV_O_SNAPSHOT);
ret = drv->bdrv_open(bs, filename, open_flags);
@@ -429,6 +429,7 @@ int bdrv_open2(BlockDriverState *bs, const char *filename, int flags,
}
path_combine(backing_filename, sizeof(backing_filename),
filename, bs->backing_file);
+ open_flags |= BDRV_O_BACKING;
if (bdrv_open(bs->backing_hd, backing_filename, open_flags) < 0)
goto fail;
}
diff --git a/qemu/block.h b/qemu/block.h
index e1927dd..ff70d64 100644
--- a/qemu/block.h
+++ b/qemu/block.h
@@ -56,6 +56,8 @@ typedef struct QEMUSnapshotInfo {
#define BDRV_O_CACHE_MASK (BDRV_O_NOCACHE | BDRV_O_CACHE_WB | BDRV_O_CACHE_DEF)
+#define BDRV_O_BACKING 0x0100
+
void bdrv_info(void);
void bdrv_info_stats(void);
--
1.6.0.6
[-- Attachment #3: 0002-block-qcow2-do-not-scan-refcounts-when-opening-a-ba.patch --]
[-- Type: text/x-patch, Size: 930 bytes --]
>From c618b293a64d1690105505d7cb28ba1ca8dd33c7 Mon Sep 17 00:00:00 2001
From: Uri Lublin <uril@redhat.com>
Date: Wed, 11 Mar 2009 20:18:23 +0200
Subject: [PATCH] block-qcow2: do not scan refcounts when opening a backing file
It takes too long and is not needed.
Signed-off-by: Uri Lublin <uril@redhat.com>
---
qemu/block-qcow2.c | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/qemu/block-qcow2.c b/qemu/block-qcow2.c
index 465dcd6..41cdbe9 100644
--- a/qemu/block-qcow2.c
+++ b/qemu/block-qcow2.c
@@ -276,7 +276,8 @@ static int qcow_open(BlockDriverState *bs, const char *filename, int flags)
if (refcount_init(bs) < 0)
goto fail;
- scan_refcount(bs, &s->highest_alloc, &s->nc_free);
+ if ((flags & BDRV_O_BACKING) == 0)
+ scan_refcount(bs, &s->highest_alloc, &s->nc_free);
/* read the backing file name */
if (header.backing_file_offset != 0) {
--
1.6.0.6
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] Re: Problems KVM-84
2009-03-11 14:52 ` Avi Kivity
@ 2009-03-11 20:19 ` Anthony Liguori
0 siblings, 0 replies; 20+ messages in thread
From: Anthony Liguori @ 2009-03-11 20:19 UTC (permalink / raw)
To: Avi Kivity; +Cc: qemu-devel, Jay Mann, Uri Lublin, kvm
Avi Kivity wrote:
> Anthony Liguori wrote:
>>>
>>> Scanning the file at startup is slow. We need to find a better way.
>>
>> Any quick ideas? Seems like this is broken by design.
>
> I agree.
>
>> Unless we can find a quick fix, I'm going to revert this in the
>> stable tree.
>
> We could cache the value in some unused area. Set it up in qemu-img.
> Don't support it on older images.
>
> But that's not a quick fix; I'd suggest reverting it in both stable
> and mainline, and re-applying in mainline when there's a fix.
Done. Uri, please submit the series again when you've found a solution
to these problems.
Regards,
Anthony Liguori
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] Re: Problems KVM-84
@ 2009-03-11 20:19 ` Anthony Liguori
0 siblings, 0 replies; 20+ messages in thread
From: Anthony Liguori @ 2009-03-11 20:19 UTC (permalink / raw)
To: Avi Kivity; +Cc: Jay Mann, Uri Lublin, qemu-devel, kvm
Avi Kivity wrote:
> Anthony Liguori wrote:
>>>
>>> Scanning the file at startup is slow. We need to find a better way.
>>
>> Any quick ideas? Seems like this is broken by design.
>
> I agree.
>
>> Unless we can find a quick fix, I'm going to revert this in the
>> stable tree.
>
> We could cache the value in some unused area. Set it up in qemu-img.
> Don't support it on older images.
>
> But that's not a quick fix; I'd suggest reverting it in both stable
> and mainline, and re-applying in mainline when there's a fix.
Done. Uri, please submit the series again when you've found a solution
to these problems.
Regards,
Anthony Liguori
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Re: Problems KVM-84
2009-03-11 16:06 ` Jamie Lokier
@ 2009-03-11 22:59 ` Uri Lublin
-1 siblings, 0 replies; 20+ messages in thread
From: Uri Lublin @ 2009-03-11 22:59 UTC (permalink / raw)
To: qemu-devel; +Cc: Jay Mann, Uri Lublin, Avi Kivity, kvm
Jamie Lokier wrote:
> Anthony Liguori wrote:
>>> block-qcow2: keep highest allocated byte (Uri Lublin)
>>> We want to know the highest written offset for qcow2 images.
>>> This gives a pretty good (and easy to calculate) estimation to how
>>> much more allocation can be done for the block device.
>>> It can be usefull for allocating more diskspace for that image
>>> (if possible, e.g. lvm) before we run out-of-disk-space
>>> Signed-off-by: Uri Lublin <uril@redhat.com>
>>> Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
>>>
>>> Scanning the file at startup is slow. We need to find a better way.
>> Any quick ideas? Seems like this is broken by design. Unless we can
>> find a quick fix, I'm going to revert this in the stable tree.
>
> I still don't understand the point in that feature.
>
> You have the size of the qcow2 file (how much data is used), and the
> size of the image it's representing (how much it could expand to).
>
> What does the highest written offset help you anticipate?
You assume there is a file system installed. If I want to use e.g. LVM's logical
volume, I can not tell the size of the qcow2 image. I want to be able to set a
watermark and handle out-of-disk-space before it actually happens.
>
> Many guest filesystems write data all over the block device, not
> concentrated at the start. (E.g. ext2/3/4 and it's block groups).
qcow2 allocates disk space regardless of the guest write offset.
And if it happen that the image is sparse (or fragmented), num_free_bytes can
tell us that.
Regards,
Uri.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] Re: Problems KVM-84
@ 2009-03-11 22:59 ` Uri Lublin
0 siblings, 0 replies; 20+ messages in thread
From: Uri Lublin @ 2009-03-11 22:59 UTC (permalink / raw)
To: qemu-devel; +Cc: Jay Mann, Uri Lublin, Avi Kivity, kvm
Jamie Lokier wrote:
> Anthony Liguori wrote:
>>> block-qcow2: keep highest allocated byte (Uri Lublin)
>>> We want to know the highest written offset for qcow2 images.
>>> This gives a pretty good (and easy to calculate) estimation to how
>>> much more allocation can be done for the block device.
>>> It can be usefull for allocating more diskspace for that image
>>> (if possible, e.g. lvm) before we run out-of-disk-space
>>> Signed-off-by: Uri Lublin <uril@redhat.com>
>>> Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
>>>
>>> Scanning the file at startup is slow. We need to find a better way.
>> Any quick ideas? Seems like this is broken by design. Unless we can
>> find a quick fix, I'm going to revert this in the stable tree.
>
> I still don't understand the point in that feature.
>
> You have the size of the qcow2 file (how much data is used), and the
> size of the image it's representing (how much it could expand to).
>
> What does the highest written offset help you anticipate?
You assume there is a file system installed. If I want to use e.g. LVM's logical
volume, I can not tell the size of the qcow2 image. I want to be able to set a
watermark and handle out-of-disk-space before it actually happens.
>
> Many guest filesystems write data all over the block device, not
> concentrated at the start. (E.g. ext2/3/4 and it's block groups).
qcow2 allocates disk space regardless of the guest write offset.
And if it happen that the image is sparse (or fragmented), num_free_bytes can
tell us that.
Regards,
Uri.
^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2009-03-11 23:00 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-02-27 0:58 Problems KVM-84 Jay Mann
2009-02-27 6:52 ` Thomas Mueller
2009-03-09 10:22 ` Avi Kivity
2009-03-09 12:19 ` jmandawg
2009-03-09 12:19 ` jmandawg
2009-03-09 12:37 ` Avi Kivity
[not found] ` <BAY102-W25DF72B635B00AD337130D2A00@phx.gbl>
2009-03-10 8:39 ` Avi Kivity
[not found] ` <BAY102-W2139785413083807492C85D2A10@phx.gbl>
2009-03-10 13:47 ` Avi Kivity
2009-03-11 11:15 ` Avi Kivity
2009-03-11 11:15 ` [Qemu-devel] " Avi Kivity
2009-03-11 14:12 ` Anthony Liguori
2009-03-11 14:12 ` [Qemu-devel] " Anthony Liguori
2009-03-11 14:52 ` Avi Kivity
2009-03-11 20:19 ` Anthony Liguori
2009-03-11 20:19 ` Anthony Liguori
2009-03-11 16:06 ` Jamie Lokier
2009-03-11 16:06 ` Jamie Lokier
2009-03-11 22:59 ` Uri Lublin
2009-03-11 22:59 ` [Qemu-devel] " Uri Lublin
2009-03-11 18:37 ` Uri Lublin
2009-03-11 18:37 ` [Qemu-devel] " Uri Lublin
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.