Re: [Qemu-devel] [PATCH v6 37/42] block: Leave BDS.backing_file constant

[Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>]

09.08.2019 19:14, Max Reitz wrote:
> Parts of the block layer treat BDS.backing_file as if it were whatever
> the image header says (i.e., if it is a relative path, it is relative to
> the overlay), other parts treat it like a cache for
> bs->backing->bs->filename (relative paths are relative to the CWD).
> Considering bs->backing->bs->filename exists, let us make it mean the
> former.
> 
> Among other things, this now allows the user to specify a base when
> using qemu-img to commit an image file in a directory that is not the
> CWD (assuming, everything uses relative filenames).
> 
> Before this patch:
> 
> $ ./qemu-img create -f qcow2 foo/bot.qcow2 1M
> $ ./qemu-img create -f qcow2 -b bot.qcow2 foo/mid.qcow2
> $ ./qemu-img create -f qcow2 -b mid.qcow2 foo/top.qcow2
> $ ./qemu-img commit -b mid.qcow2 foo/top.qcow2
> qemu-img: Did not find 'mid.qcow2' in the backing chain of 'foo/top.qcow2'
> $ ./qemu-img commit -b foo/mid.qcow2 foo/top.qcow2
> qemu-img: Did not find 'foo/mid.qcow2' in the backing chain of 'foo/top.qcow2'
> $ ./qemu-img commit -b $PWD/foo/mid.qcow2 foo/top.qcow2
> qemu-img: Did not find '[...]/foo/mid.qcow2' in the backing chain of 'foo/top.qcow2'

nothing works

> 
> After this patch:
> 
> $ ./qemu-img commit -b mid.qcow2 foo/top.qcow2
> Image committed.
> $ ./qemu-img commit -b foo/mid.qcow2 foo/top.qcow2
> qemu-img: Did not find 'foo/mid.qcow2' in the backing chain of 'foo/top.qcow2'
> $ ./qemu-img commit -b $PWD/foo/mid.qcow2 foo/top.qcow2
> Image committed.

something works.. However it seems that not working one is actually most probable
to be called by user. Anyway something is better than nothing.

> 
> With this change, bdrv_find_backing_image() must look at whether the
> user has overridden a BDS's backing file.  If so, it can no longer use
> bs->backing_file, but must instead compare the given filename against
> the backing node's filename directly.
> 
> Note that this changes the QAPI output for a node's backing_file.  We
> had very inconsistent output there (sometimes what the image header
> said, sometimes the actual filename of the backing image).  This
> inconsistent output was effectively useless, so we have to decide one
> way or the other.  Considering that bs->backing_file usually at runtime
> contained the path to the image relative to qemu's CWD (or absolute),
> this patch changes QAPI's backing_file to always report the
> bs->backing->bs->filename from now on.  If you want to receive the image
> header information, you have to refer to full-backing-filename.
> 
> This necessitates a change to iotest 228.  The interesting information
> it really wanted is the image header, and it can get that now, but it
> has to use full-backing-filename instead of backing_file.  Because of
> this patch's changes to bs->backing_file's behavior, we also need some
> reference output changes.
> 
> Along with the changes to bs->backing_file, stop updating
> BDS.backing_format in bdrv_backing_attach() as well.  This necessitates
> a change to the reference output of iotest 191.
> 
> iotest 245 changes in behavior: With the backing node no longer
> overriding the parent node's backing_file string, you can now omit the
> @backing option when reopening a node with neither a default nor a
> current backing file even if it used to have a backing node at some
> point.
> 
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
>   include/block/block_int.h  | 19 ++++++++++++++-----
>   block.c                    | 35 ++++++++++++++++++++++++++++-------
>   block/qapi.c               |  7 ++++---
>   tests/qemu-iotests/191.out |  1 -
>   tests/qemu-iotests/228     |  6 +++---
>   tests/qemu-iotests/228.out |  6 +++---
>   tests/qemu-iotests/245     |  4 +++-
>   7 files changed, 55 insertions(+), 23 deletions(-)
> 
> diff --git a/include/block/block_int.h b/include/block/block_int.h
> index 42ee2fcf7f..993bafc090 100644
> --- a/include/block/block_int.h
> +++ b/include/block/block_int.h
> @@ -784,11 +784,20 @@ struct BlockDriverState {
>       bool walking_aio_notifiers; /* to make removal during iteration safe */
>   
>       char filename[PATH_MAX];
> -    char backing_file[PATH_MAX]; /* if non zero, the image is a diff of
> -                                    this file image */
> -    /* The backing filename indicated by the image header; if we ever
> -     * open this file, then this is replaced by the resulting BDS's
> -     * filename (i.e. after a bdrv_refresh_filename() run). */
> +    /*
> +     * If not empty, this image is a diff in relation to backing_file.
> +     * Note that this is the name given in the image header

Is it synced when image header is updated? If yes, it's not constant, if not it's just wrong.

> and
> +     * therefore may or may not be equal to .backing->bs->filename.
> +     * If this field contains a relative path, it is to be resolved
> +     * relatively to the overlay's location.
> +     */
> +    char backing_file[PATH_MAX];
> +    /*
> +     * The backing filename indicated by the image header.  Contrary
> +     * to backing_file, if we ever open this file, auto_backing_file

Preexisting, but for a fresh look (I don't know backing_file and auto_backing_file related
logic and assume that these are strange fields which would better not exist) this is
hard to understand.

open this file - which one? this bds? or it's backing?

> +     * is replaced by the resulting BDS's filename (i.e. after a
> +     * bdrv_refresh_filename() run).
> +     */
>       char auto_backing_file[PATH_MAX];
>       char backing_format[16]; /* if non-zero and backing_file exists */
>   
> diff --git a/block.c b/block.c
> index 4858d3e718..88533fa0d3 100644
> --- a/block.c
> +++ b/block.c
> @@ -78,6 +78,8 @@ static BlockDriverState *bdrv_open_inherit(const char *filename,
>                                              const BdrvChildRole *child_role,
>                                              Error **errp);
>   
> +static bool bdrv_backing_overridden(BlockDriverState *bs);
> +
>   /* If non-zero, use only whitelisted block drivers */
>   static int use_bdrv_whitelist;
>   
> @@ -1065,10 +1067,6 @@ static void bdrv_backing_attach(BdrvChild *c)
>       bdrv_refresh_filename(backing_hd);
>   
>       parent->open_flags &= ~BDRV_O_NO_BACKING;
> -    pstrcpy(parent->backing_file, sizeof(parent->backing_file),
> -            backing_hd->filename);
> -    pstrcpy(parent->backing_format, sizeof(parent->backing_format),
> -            backing_hd->drv ? backing_hd->drv->format_name : "");
>   
>       bdrv_op_block_all(backing_hd, parent->backing_blocker);
>       /* Otherwise we won't be able to commit or stream */
> @@ -5294,6 +5292,7 @@ BlockDriverState *bdrv_find_backing_image(BlockDriverState *bs,
>       char *backing_file_full = NULL;
>       char *filename_tmp = NULL;
>       int is_protocol = 0;
> +    bool filenames_refreshed = false;
>       BlockDriverState *curr_bs = NULL;
>       BlockDriverState *retval = NULL;
>   
> @@ -5318,9 +5317,31 @@ BlockDriverState *bdrv_find_backing_image(BlockDriverState *bs,
>       {
>           BlockDriverState *bs_below = bdrv_backing_chain_next(curr_bs);
>   
> -        /* If either of the filename paths is actually a protocol, then
> -         * compare unmodified paths; otherwise make paths relative */
> -        if (is_protocol || path_has_protocol(curr_bs->backing_file)) {
> +        if (bdrv_backing_overridden(curr_bs)) {
> +            /*
> +             * If the backing file was overridden, we can only compare
> +             * directly against the backing node's filename.
> +             */
> +
> +            if (!filenames_refreshed) {
> +                /*
> +                 * This will automatically refresh all of the
> +                 * filenames in the rest of the backing chain, so we
> +                 * only need to do this once.
> +                 */
> +                bdrv_refresh_filename(bs_below);
> +                filenames_refreshed = true;
> +            }
> +
> +            if (strcmp(backing_file, bs_below->filename) == 0) {

Don't you want try realpath() here too?

And I doubt, why we can't always assume bdrv_backing_overridden() = true? (keeping in mind
that it may give false positives, as comment says)?

Just go through backing chain and check

searched_file == bs->filename || realpath(searched_file) == realpath(bs->filename) ?

Do we need to cover case when something is relative to parent really? I'm not sure
but it seems like in libvirt absolute paths are used always.. And if we need just add
to this condition:

|| realpath(joinpath(prev_bs->filename, searched_file)) == realpath(bs->filename))


> +                retval = bs_below;
> +                break;
> +            }
> +        } else if (is_protocol || path_has_protocol(curr_bs->backing_file)) {
> +            /*
> +             * If either of the filename paths is actually a protocol, then
> +             * compare unmodified paths; otherwise make paths relative.
> +             */
>               char *backing_file_full_ret;
>   
>               if (strcmp(backing_file, curr_bs->backing_file) == 0) {
> diff --git a/block/qapi.c b/block/qapi.c
> index 4f59ac1c0f..751c3e695a 100644
> --- a/block/qapi.c
> +++ b/block/qapi.c
> @@ -45,7 +45,7 @@ BlockDeviceInfo *bdrv_block_device_info(BlockBackend *blk,
>                                           BlockDriverState *bs, Error **errp)
>   {
>       ImageInfo **p_image_info;
> -    BlockDriverState *bs0;
> +    BlockDriverState *bs0, *backing;
>       BlockDeviceInfo *info;
>   
>       if (!bs->drv) {
> @@ -74,9 +74,10 @@ BlockDeviceInfo *bdrv_block_device_info(BlockBackend *blk,
>           info->node_name = g_strdup(bs->node_name);
>       }
>   
> -    if (bs->backing_file[0]) {
> +    backing = bdrv_filtered_cow_bs(bs);
> +    if (backing) {
>           info->has_backing_file = true;
> -        info->backing_file = g_strdup(bs->backing_file);
> +        info->backing_file = g_strdup(backing->filename);
>       }
>   
>       if (!QLIST_EMPTY(&bs->dirty_bitmaps)) {
> diff --git a/tests/qemu-iotests/191.out b/tests/qemu-iotests/191.out
> index 3fc92bb56e..0b3c216b0c 100644
> --- a/tests/qemu-iotests/191.out
> +++ b/tests/qemu-iotests/191.out
> @@ -605,7 +605,6 @@ wrote 65536/65536 bytes at offset 1048576
>                       "backing-filename": "TEST_DIR/t.IMGFMT.base",
>                       "dirty-flag": false
>                   },
> -                "backing-filename-format": "IMGFMT",
>                   "virtual-size": 67108864,
>                   "filename": "TEST_DIR/t.IMGFMT.ovl3",
>                   "cluster-size": 65536,
> diff --git a/tests/qemu-iotests/228 b/tests/qemu-iotests/228
> index 9a50afd205..a1f3187212 100755
> --- a/tests/qemu-iotests/228
> +++ b/tests/qemu-iotests/228
> @@ -34,7 +34,7 @@ def log_node_info(node):
>   
>       log('bs->filename: ' + node['image']['filename'],
>           filters=[filter_testfiles, filter_imgfmt])
> -    log('bs->backing_file: ' + node['backing_file'],
> +    log('bs->backing_file: ' + node['image']['full-backing-filename'],
>           filters=[filter_testfiles, filter_imgfmt])
>   
>       if 'backing-image' in node['image']:
> @@ -70,8 +70,8 @@ with iotests.FilePath('base.img') as base_img_path, \
>                   },
>                   filters=[filter_qmp_testfiles, filter_qmp_imgfmt])
>   
> -    # Filename should be plain, and the backing filename should not
> -    # contain the "file:" prefix
> +    # Filename should be plain, and the backing node filename should
> +    # not contain the "file:" prefix
>       log_node_info(vm.node_info('node0'))
>   
>       vm.qmp_log('blockdev-del', node_name='node0')
> diff --git a/tests/qemu-iotests/228.out b/tests/qemu-iotests/228.out
> index 4217df24fe..8c82009abe 100644
> --- a/tests/qemu-iotests/228.out
> +++ b/tests/qemu-iotests/228.out
> @@ -4,7 +4,7 @@
>   {"return": {}}
>   
>   bs->filename: TEST_DIR/PID-top.img
> -bs->backing_file: TEST_DIR/PID-base.img
> +bs->backing_file: file:TEST_DIR/PID-base.img
>   bs->backing->bs->filename: TEST_DIR/PID-base.img
>   
>   {"execute": "blockdev-del", "arguments": {"node-name": "node0"}}
> @@ -41,7 +41,7 @@ bs->backing->bs->filename: TEST_DIR/PID-base.img
>   {"return": {}}
>   
>   bs->filename: TEST_DIR/PID-top.img
> -bs->backing_file: TEST_DIR/PID-base.img
> +bs->backing_file: file:TEST_DIR/PID-base.img
>   bs->backing->bs->filename: TEST_DIR/PID-base.img
>   
>   {"execute": "blockdev-del", "arguments": {"node-name": "node0"}}
> @@ -55,7 +55,7 @@ bs->backing->bs->filename: TEST_DIR/PID-base.img
>   {"return": {}}
>   
>   bs->filename: json:{"backing": {"driver": "null-co"}, "driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/PID-top.img"}}
> -bs->backing_file: null-co://
> +bs->backing_file: TEST_DIR/PID-base.img
>   bs->backing->bs->filename: null-co://
>   
>   {"execute": "blockdev-del", "arguments": {"node-name": "node0"}}
> diff --git a/tests/qemu-iotests/245 b/tests/qemu-iotests/245
> index bc1ceb9792..049ef6a71f 100644
> --- a/tests/qemu-iotests/245
> +++ b/tests/qemu-iotests/245
> @@ -722,7 +722,9 @@ class TestBlockdevReopen(iotests.QMPTestCase):
>   
>           # Detach hd2 from hd0.
>           self.reopen(opts, {'backing': None})
> -        self.reopen(opts, {}, "backing is missing for 'hd0'")
> +
> +        # Without a backing file, we can omit 'backing' again
> +        self.reopen(opts)
>   
>           # Remove both hd0 and hd2
>           result = self.vm.qmp('blockdev-del', conv_keys = True, node_name = 'hd0')
> 


-- 
Best regards,
Vladimir