[Qemu-devel] qemu-img behavior for locating backing files

[Qemu-devel] qemu-img behavior for locating backing files

[John Snow]

Kevin, what's the correct behavior for qemu-img and relative paths when 
creating a new qcow2 file?

Example:

(in e.g. /home/qemu/build/ or anywhere not /home: )
qemu-img create -f qcow2 base.qcow2 32G
qemu-img create -f qcow2 -F qcow2 -b base.qcow2 /home/overlay.qcow2

In 1.7.0., this produces a warning that the base object cannot be found 
(because it does not exist at that location relative to overlay.qcow2), 
but qemu-img will create the qcow2 for you regardless.

2.0, 2.1 and 2.2 all will create the image successfully, with no warnings.

2.3-rc1/master as they exist now will emit an error message and create 
no image.

Since this is a change in behavior for the pending release, is this the 
correct/desired behavior?

Re: [Qemu-devel] qemu-img behavior for locating backing files

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1857 bytes --]

On 04/01/2015 10:16 AM, John Snow wrote:
> Kevin, what's the correct behavior for qemu-img and relative paths when
> creating a new qcow2 file?
> 
> Example:
> 
> (in e.g. /home/qemu/build/ or anywhere not /home: )
> qemu-img create -f qcow2 base.qcow2 32G

creates /home/qemu/build/base.qcow2

> qemu-img create -f qcow2 -F qcow2 -b base.qcow2 /home/overlay.qcow2

Tries to create /home/overlay.qcow2; requires /home/base.qcow2 to exist
for the creation to be well-formed.  (Any use of
/home/qemu/build/base.qcow2 should be wrong)

If you want, you could do:

qemu-img create -f qcow2 /home/overlay.qcow2 $size
qemu-img rebase -u -f qcow2 -F qcow2 -b base.qcow2 /home/overlay.qcow2

to create the file that would relatively point to /home/base.qcow2,
whether or not that file already exists; and it could be argued that we
may even want to support that via a single create command (that is,
'create an image with this string as the backing file, but without
actually chasing through that string')

> 
> In 1.7.0., this produces a warning that the base object cannot be found
> (because it does not exist at that location relative to overlay.qcow2),
> but qemu-img will create the qcow2 for you regardless.

Sounds almost ideal (or at least an argument for the 'create with an
unsafe string for backing) - but how did it pick the size for that image?

> 
> 2.0, 2.1 and 2.2 all will create the image successfully, with no warnings.

Oops.

> 
> 2.3-rc1/master as they exist now will emit an error message and create
> no image.

Sounds like a bug fix, not a regression.

> 
> Since this is a change in behavior for the pending release, is this the
> correct/desired behavior?

Yes, I think so.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] qemu-img behavior for locating backing files

[Kevin Wolf]

Am 01.04.2015 um 18:16 hat John Snow geschrieben:
> Kevin, what's the correct behavior for qemu-img and relative paths
> when creating a new qcow2 file?
> 
> Example:
> 
> (in e.g. /home/qemu/build/ or anywhere not /home: )
> qemu-img create -f qcow2 base.qcow2 32G
> qemu-img create -f qcow2 -F qcow2 -b base.qcow2 /home/overlay.qcow2
> 
> In 1.7.0., this produces a warning that the base object cannot be
> found (because it does not exist at that location relative to
> overlay.qcow2), but qemu-img will create the qcow2 for you
> regardless.
> 
> 2.0, 2.1 and 2.2 all will create the image successfully, with no warnings.
> 
> 2.3-rc1/master as they exist now will emit an error message and
> create no image.
> 
> Since this is a change in behavior for the pending release, is this
> the correct/desired behavior?

Part one of the answer is easy: qemu-img create should succeed if, and
only if, a usable image is created. This requires that the backing file
exists.

Part two is a bit harder: Should base.qcow2 be found in the current
directory even if the new image is somewhere else? We must give
preference to an existing base.qcow2 relative to the new image path, but
if it doesn't exist, we could in theory try to find it relative to the
working directory.

If we then find it, we have two options: Either we use that image
(probably with an absolute path then?) or we print a useful error
message that instructs the user how relative paths work with images.
I think the latter is better because the other option feels like too
much magic.

In any case, the behaviour you describe for 2.3-rc1 seems to be the best
that we've had until now; 1.7.0 looks like the second best. We should
probably "document" the 2.3-rc1 behaviour with a qemu-iotests case.

Oh, and we still have a bug: If you specify an image size, qemu-img
doesn't check at all whether the backing file exists.

Kevin

Re: [Qemu-devel] qemu-img behavior for locating backing files

[John Snow]

On 04/02/2015 05:38 AM, Kevin Wolf wrote:
> Am 01.04.2015 um 18:16 hat John Snow geschrieben:
>> Kevin, what's the correct behavior for qemu-img and relative paths
>> when creating a new qcow2 file?
>>
>> Example:
>>
>> (in e.g. /home/qemu/build/ or anywhere not /home: )
>> qemu-img create -f qcow2 base.qcow2 32G
>> qemu-img create -f qcow2 -F qcow2 -b base.qcow2 /home/overlay.qcow2
>>
>> In 1.7.0., this produces a warning that the base object cannot be
>> found (because it does not exist at that location relative to
>> overlay.qcow2), but qemu-img will create the qcow2 for you
>> regardless.
>>
>> 2.0, 2.1 and 2.2 all will create the image successfully, with no warnings.
>>
>> 2.3-rc1/master as they exist now will emit an error message and
>> create no image.
>>
>> Since this is a change in behavior for the pending release, is this
>> the correct/desired behavior?
>
> Part one of the answer is easy: qemu-img create should succeed if, and
> only if, a usable image is created. This requires that the backing file
> exists.
>

So far so good.

> Part two is a bit harder: Should base.qcow2 be found in the current
> directory even if the new image is somewhere else? We must give
> preference to an existing base.qcow2 relative to the new image path, but
> if it doesn't exist, we could in theory try to find it relative to the
> working directory.
>

Nack. This seems like inviting heartbreak unnecessarily.

> If we then find it, we have two options: Either we use that image
> (probably with an absolute path then?) or we print a useful error
> message that instructs the user how relative paths work with images.
> I think the latter is better because the other option feels like too
> much magic.
>

Too much magic indeed. I think where ambiguity of paths is concerned, it 
is best to stick to one particular path and make it very explicit.

In this case, if we cannot find some relative path offered by the user, 
an error message such as:

"Hey! We can't find /absolute/path/to/../your/relative/file.qcow2"

should be sufficient to clue the user in to where qemu-img is looking 
for this backing file.

A usability bonus might be when we go to whine at the user, if the file 
exists relative to the PWD:

"Qemu noticed a file at /your/pwd/.../your/relative/file.qcow2, but Qemu 
expects relative paths for backing files to be relative to the image 
referencing them, not your current PWD"

but this is just a Deluxe Niceness.

> In any case, the behaviour you describe for 2.3-rc1 seems to be the best
> that we've had until now; 1.7.0 looks like the second best. We should
> probably "document" the 2.3-rc1 behaviour with a qemu-iotests case.
>

Absolutely.

> Oh, and we still have a bug: If you specify an image size, qemu-img
> doesn't check at all whether the backing file exists.
>
> Kevin
>

Thanks,
--js

Re: [Qemu-devel] qemu-img behavior for locating backing files

[Kevin Wolf]

Am 07.04.2015 um 02:31 hat John Snow geschrieben:
> 
> 
> On 04/02/2015 05:38 AM, Kevin Wolf wrote:
> >Am 01.04.2015 um 18:16 hat John Snow geschrieben:
> >>Kevin, what's the correct behavior for qemu-img and relative paths
> >>when creating a new qcow2 file?
> >>
> >>Example:
> >>
> >>(in e.g. /home/qemu/build/ or anywhere not /home: )
> >>qemu-img create -f qcow2 base.qcow2 32G
> >>qemu-img create -f qcow2 -F qcow2 -b base.qcow2 /home/overlay.qcow2
> >>
> >>In 1.7.0., this produces a warning that the base object cannot be
> >>found (because it does not exist at that location relative to
> >>overlay.qcow2), but qemu-img will create the qcow2 for you
> >>regardless.
> >>
> >>2.0, 2.1 and 2.2 all will create the image successfully, with no warnings.
> >>
> >>2.3-rc1/master as they exist now will emit an error message and
> >>create no image.
> >>
> >>Since this is a change in behavior for the pending release, is this
> >>the correct/desired behavior?
> >
> >Part one of the answer is easy: qemu-img create should succeed if, and
> >only if, a usable image is created. This requires that the backing file
> >exists.
> >
> 
> So far so good.
> 
> >Part two is a bit harder: Should base.qcow2 be found in the current
> >directory even if the new image is somewhere else? We must give
> >preference to an existing base.qcow2 relative to the new image path, but
> >if it doesn't exist, we could in theory try to find it relative to the
> >working directory.
> >
> 
> Nack. This seems like inviting heartbreak unnecessarily.
> 
> >If we then find it, we have two options: Either we use that image
> >(probably with an absolute path then?) or we print a useful error
> >message that instructs the user how relative paths work with images.
> >I think the latter is better because the other option feels like too
> >much magic.
> >
> 
> Too much magic indeed. I think where ambiguity of paths is
> concerned, it is best to stick to one particular path and make it
> very explicit.
> 
> In this case, if we cannot find some relative path offered by the
> user, an error message such as:
> 
> "Hey! We can't find /absolute/path/to/../your/relative/file.qcow2"
> 
> should be sufficient to clue the user in to where qemu-img is
> looking for this backing file.

Yes, printing the combined path sounds like a good option.

> A usability bonus might be when we go to whine at the user, if the
> file exists relative to the PWD:
> 
> "Qemu noticed a file at /your/pwd/.../your/relative/file.qcow2, but
> Qemu expects relative paths for backing files to be relative to the
> image referencing them, not your current PWD"
> 
> but this is just a Deluxe Niceness.

If we touch it anyway, why not have Deluxe Niceness?

> >In any case, the behaviour you describe for 2.3-rc1 seems to be the best
> >that we've had until now; 1.7.0 looks like the second best. We should
> >probably "document" the 2.3-rc1 behaviour with a qemu-iotests case.
> >
> 
> Absolutely.

Are you going to take care of that, or should I write one?

> >Oh, and we still have a bug: If you specify an image size, qemu-img
> >doesn't check at all whether the backing file exists.

Same question here.

Kevin

Re: [Qemu-devel] qemu-img behavior for locating backing files

[John Snow]

On 04/07/2015 04:44 AM, Kevin Wolf wrote:
> Am 07.04.2015 um 02:31 hat John Snow geschrieben:
>>
>>
>> On 04/02/2015 05:38 AM, Kevin Wolf wrote:
>>> Am 01.04.2015 um 18:16 hat John Snow geschrieben:
>>>> Kevin, what's the correct behavior for qemu-img and relative paths
>>>> when creating a new qcow2 file?
>>>>
>>>> Example:
>>>>
>>>> (in e.g. /home/qemu/build/ or anywhere not /home: )
>>>> qemu-img create -f qcow2 base.qcow2 32G
>>>> qemu-img create -f qcow2 -F qcow2 -b base.qcow2 /home/overlay.qcow2
>>>>
>>>> In 1.7.0., this produces a warning that the base object cannot be
>>>> found (because it does not exist at that location relative to
>>>> overlay.qcow2), but qemu-img will create the qcow2 for you
>>>> regardless.
>>>>
>>>> 2.0, 2.1 and 2.2 all will create the image successfully, with no warnings.
>>>>
>>>> 2.3-rc1/master as they exist now will emit an error message and
>>>> create no image.
>>>>
Are you going to take care of that, or should I write one?


>>>> Since this is a change in behavior for the pending release, is this
>>>> the correct/desired behavior?
>>>
>>> Part one of the answer is easy: qemu-img create should succeed if, and
>>> only if, a usable image is created. This requires that the backing file
>>> exists.
>>>
>>
>> So far so good.
>>
>>> Part two is a bit harder: Should base.qcow2 be found in the current
>>> directory even if the new image is somewhere else? We must give
>>> preference to an existing base.qcow2 relative to the new image path, but
>>> if it doesn't exist, we could in theory try to find it relative to the
>>> working directory.
>>>
>>
>> Nack. This seems like inviting heartbreak unnecessarily.
>>
>>> If we then find it, we have two options: Either we use that image
>>> (probably with an absolute path then?) or we print a useful error
>>> message that instructs the user how relative paths work with images.
>>> I think the latter is better because the other option feels like too
>>> much magic.
>>>
>>
>> Too much magic indeed. I think where ambiguity of paths is
>> concerned, it is best to stick to one particular path and make it
>> very explicit.
>>
>> In this case, if we cannot find some relative path offered by the
>> user, an error message such as:
>>
>> "Hey! We can't find /absolute/path/to/../your/relative/file.qcow2"
>>
>> should be sufficient to clue the user in to where qemu-img is
>> looking for this backing file.
>
> Yes, printing the combined path sounds like a good option.
>
>> A usability bonus might be when we go to whine at the user, if the
>> file exists relative to the PWD:
>>
>> "Qemu noticed a file at /your/pwd/.../your/relative/file.qcow2, but
>> Qemu expects relative paths for backing files to be relative to the
>> image referencing them, not your current PWD"
>>
>> but this is just a Deluxe Niceness.
>
> If we touch it anyway, why not have Deluxe Niceness?
>

Your wish is my command!

>>> In any case, the behaviour you describe for 2.3-rc1 seems to be the best
>>> that we've had until now; 1.7.0 looks like the second best. We should
>>> probably "document" the 2.3-rc1 behaviour with a qemu-iotests case.
>>>
>>
>> Absolutely.
>
> Are you going to take care of that, or should I write one?
>
>>> Oh, and we still have a bug: If you specify an image size, qemu-img
>>> doesn't check at all whether the backing file exists.
>
> Same question here.
>
> Kevin
>

I meant to imply I'd do it, thanks!

--js