[Bug 1865048] [NEW] qemu-img --force-share does not disable file locking

[Bug 1865048] [NEW] qemu-img --force-share does not disable file locking

[Olaf Seibert]

Public bug reported:

The new option "--force-share" for qemu-img does not disable file
locking.

I tried it with version qemu-img version 2.11.1(Debian 1:2.11+dfsg-
1ubuntu7.21~cloud0) and I traced the source code of the current git
trunk.

Sample to demonstrate:

# strace qemu-img info --force-share testfile.qcow2   2>&1 | grep F_RDLCK
fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0

I traced the passing of the --force-share option through the source code
(I used commit 6c599282f8 as of Mon Feb 17 13:32:25 2020 +0000)

qemu-img.c:img_info()
        force_share = true;
qemu-img.c:collect_image_info_list(force_share)
qemu-img.c:img_open(force_share)
qemu-img.c:img_open_file(force_share)
        qdict_put_bool(options, BDRV_OPT_FORCE_SHARE, true);
block/block-backend.c:blk_new_open(options)
block.c:bdrv_open(options)
block.c:bdrv_open_inheritoptions()
block.c:bdrv_open_common(options)
        bs->force_share = qemu_opt_get_bool(opts, BDRV_OPT_FORCE_SHARE, false);
block.c:bdrv_open_driver(bs)
include/block/block_int.h:int (*bdrv_file_open)(BlockDriverState *bs, QDict *options, int flags,
block/file-posix.c:.bdrv_file_open = raw_open,
block/file-posix.c:raw_open_common(bs)
        locking = qapi_enum_parse(&OnOffAuto_lookup,
                              qemu_opt_get(opts, "locking"),
                              ON_OFF_AUTO_AUTO, &local_err);
        ignoring bs->force_share

At the end, bs->force_share is ignored in determining the locking value.

** Affects: qemu
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1865048

Title:
  qemu-img --force-share does not disable file locking

Status in QEMU:
  New

Bug description:
  The new option "--force-share" for qemu-img does not disable file
  locking.

  I tried it with version qemu-img version 2.11.1(Debian 1:2.11+dfsg-
  1ubuntu7.21~cloud0) and I traced the source code of the current git
  trunk.

  Sample to demonstrate:

  # strace qemu-img info --force-share testfile.qcow2   2>&1 | grep F_RDLCK
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0

  I traced the passing of the --force-share option through the source
  code (I used commit 6c599282f8 as of Mon Feb 17 13:32:25 2020 +0000)

  qemu-img.c:img_info()
          force_share = true;
  qemu-img.c:collect_image_info_list(force_share)
  qemu-img.c:img_open(force_share)
  qemu-img.c:img_open_file(force_share)
          qdict_put_bool(options, BDRV_OPT_FORCE_SHARE, true);
  block/block-backend.c:blk_new_open(options)
  block.c:bdrv_open(options)
  block.c:bdrv_open_inheritoptions()
  block.c:bdrv_open_common(options)
          bs->force_share = qemu_opt_get_bool(opts, BDRV_OPT_FORCE_SHARE, false);
  block.c:bdrv_open_driver(bs)
  include/block/block_int.h:int (*bdrv_file_open)(BlockDriverState *bs, QDict *options, int flags,
  block/file-posix.c:.bdrv_file_open = raw_open,
  block/file-posix.c:raw_open_common(bs)
          locking = qapi_enum_parse(&OnOffAuto_lookup,
                                qemu_opt_get(opts, "locking"),
                                ON_OFF_AUTO_AUTO, &local_err);
          ignoring bs->force_share

  At the end, bs->force_share is ignored in determining the locking
  value.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1865048/+subscriptions

[Bug 1865048] Re: qemu-img --force-share does not disable file locking

[Max Reitz]

Hi,

That’s intentional.  The man page says this:

       --force-share (-U)
           If specified, "qemu-img" will open the image in shared mode,
           allowing other QEMU processes to open it in write mode. For
           example, this can be used to get the image information (with
           'info' subcommand) when the image is used by a running guest.

It says nothing about not using file locks.  All it will do is cause
qemu-img to signal to other processes that it’s OK for them to use the
image in any way, or if there already is another process having opened
the image for any access, qemu-img will not complain.

For example, open a qemu-io process on some image:

$ qemu-io foo.qcow2

And in another shell, invoke qemu-img:

$ qemu-img info foo.qcow2
qemu-img: Could not open 'foo.qcow2': Failed to get shared "write" lock
Is another process using the image [foo.qcow2]?

$ qemu-img info --force-share foo.qcow2
image: foo.qcow2
file format: qcow2
[...]


force-share is evaluated in bdrv_child_perm in block.c.  The file locks qemu sets are an extension of the internal “permission” system we use for block nodes: For example, when some guest device wants write access to an image, it has to take the WRITE permission.  That will be disallowed if there is some other user of the image that does not allow taking the WRITE permission (we say: it “unshares” the WRITE permission).  force-share enforces sharing all permissions, but it does not disable the permission system.

The file locks are used to transmit that internal mechanism of
taking/sharing permissions across different processes.  Unshared
permissions are reflected by locks between offset 200 and 299. Taken
permissions are reflected by locks between offset 100 and 199.  As you
can see, qemu-img with --force-share does not unshare any permissions
(it does not take any locks after offset 200).  The only lock it takes
is offset 100, which corresponds to CONSISTENT_READ.  That permission
means “I want to read from the image and get back something sane”.  So
if any other process uses the image in such a way that this is
impossible (i.e., it has to unshare CONSISTENT_READ), qemu-img info will
complain, regardless of --force-share.


File locks can only be completely disabled through file-posix’s @locking option (locking=false), e.g. like so:

$ qemu-img info --image-opts file.filename=foo.qcow2,file.locking=off

But that is strongly discouraged, and I have yet to see a case where
this would be the right thing to do.

Max

** Changed in: qemu
       Status: New => Invalid

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1865048

Title:
  qemu-img --force-share does not disable file locking

Status in QEMU:
  Invalid

Bug description:
  The new option "--force-share" for qemu-img does not disable file
  locking.

  I tried it with version qemu-img version 2.11.1(Debian 1:2.11+dfsg-
  1ubuntu7.21~cloud0) and I traced the source code of the current git
  trunk.

  Sample to demonstrate:

  # strace qemu-img info --force-share testfile.qcow2   2>&1 | grep F_RDLCK
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0

  I traced the passing of the --force-share option through the source
  code (I used commit 6c599282f8 as of Mon Feb 17 13:32:25 2020 +0000)

  qemu-img.c:img_info()
          force_share = true;
  qemu-img.c:collect_image_info_list(force_share)
  qemu-img.c:img_open(force_share)
  qemu-img.c:img_open_file(force_share)
          qdict_put_bool(options, BDRV_OPT_FORCE_SHARE, true);
  block/block-backend.c:blk_new_open(options)
  block.c:bdrv_open(options)
  block.c:bdrv_open_inheritoptions()
  block.c:bdrv_open_common(options)
          bs->force_share = qemu_opt_get_bool(opts, BDRV_OPT_FORCE_SHARE, false);
  block.c:bdrv_open_driver(bs)
  include/block/block_int.h:int (*bdrv_file_open)(BlockDriverState *bs, QDict *options, int flags,
  block/file-posix.c:.bdrv_file_open = raw_open,
  block/file-posix.c:raw_open_common(bs)
          locking = qapi_enum_parse(&OnOffAuto_lookup,
                                qemu_opt_get(opts, "locking"),
                                ON_OFF_AUTO_AUTO, &local_err);
          ignoring bs->force_share

  At the end, bs->force_share is ignored in determining the locking
  value.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1865048/+subscriptions

[Bug 1865048] Re: qemu-img --force-share does not disable file locking

[Olaf Seibert]

Hi Maz,

thanks for the information!

The situation we're in is where we are suspecting the file locking on a
shared network file system to be broken, so we were looking for ways to
avoid any locking. I had tried some variations on your image-opts style
invocation, but did not find any variant where the automatic file format
detection would be preserved. For instance, with --image-opts
driver=file,filename=foo.qcow2,locking=off it would think the file is
raw. But the one you give seems to do what I want to experiment with, so
thanks again!

-Olaf.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1865048

Title:
  qemu-img --force-share does not disable file locking

Status in QEMU:
  Invalid

Bug description:
  The new option "--force-share" for qemu-img does not disable file
  locking.

  I tried it with version qemu-img version 2.11.1(Debian 1:2.11+dfsg-
  1ubuntu7.21~cloud0) and I traced the source code of the current git
  trunk.

  Sample to demonstrate:

  # strace qemu-img info --force-share testfile.qcow2   2>&1 | grep F_RDLCK
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0

  I traced the passing of the --force-share option through the source
  code (I used commit 6c599282f8 as of Mon Feb 17 13:32:25 2020 +0000)

  qemu-img.c:img_info()
          force_share = true;
  qemu-img.c:collect_image_info_list(force_share)
  qemu-img.c:img_open(force_share)
  qemu-img.c:img_open_file(force_share)
          qdict_put_bool(options, BDRV_OPT_FORCE_SHARE, true);
  block/block-backend.c:blk_new_open(options)
  block.c:bdrv_open(options)
  block.c:bdrv_open_inheritoptions()
  block.c:bdrv_open_common(options)
          bs->force_share = qemu_opt_get_bool(opts, BDRV_OPT_FORCE_SHARE, false);
  block.c:bdrv_open_driver(bs)
  include/block/block_int.h:int (*bdrv_file_open)(BlockDriverState *bs, QDict *options, int flags,
  block/file-posix.c:.bdrv_file_open = raw_open,
  block/file-posix.c:raw_open_common(bs)
          locking = qapi_enum_parse(&OnOffAuto_lookup,
                                qemu_opt_get(opts, "locking"),
                                ON_OFF_AUTO_AUTO, &local_err);
          ignoring bs->force_share

  At the end, bs->force_share is ignored in determining the locking
  value.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1865048/+subscriptions

[Bug 1865048] Re: qemu-img --force-share does not disable file locking

[Max Reitz]

Hi Olaf,

Every “node” in the block graph corresponds to some driver.  A driver
can be a protocol or a format driver (or a filter driver, but that isn’t
important here).  In your example, there is only a single node, for a
protocol driver (namely “file”).  You need a format driver node on top
to interpret the image format.

If you use file.driver=file,file.filename=foo.qcow2,file.locking=off,
then that specifies the options driver, filename, and locking for a node
under another node’s “file” link.  So this has to create two nodes.  The
node on top (for which no options are specified) should default to being
a format node whose format is probed.

Of course you can also give options to the top (format) node, like e.g.
the driver.  (In fact, you should probably give the driver, because
format probing is considered dangerous.)

Then it would look like this:
driver=qcow2,file.driver=file,file.filename=foo.qcow2,file.locking=off

(Or, in JSON, but that only works with qemu’s -blockdev (but I think it’s better for visualizing the resulting block graph:
 {"node-name": "some-node-name",
  "driver": "qcow2",
  "file": {
      "driver": "file",
      "filename": "foo.qcow2",
      "locking": false
  } })


Hope that helps,

Max

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1865048

Title:
  qemu-img --force-share does not disable file locking

Status in QEMU:
  Invalid

Bug description:
  The new option "--force-share" for qemu-img does not disable file
  locking.

  I tried it with version qemu-img version 2.11.1(Debian 1:2.11+dfsg-
  1ubuntu7.21~cloud0) and I traced the source code of the current git
  trunk.

  Sample to demonstrate:

  # strace qemu-img info --force-share testfile.qcow2   2>&1 | grep F_RDLCK
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0
  fcntl(11, F_OFD_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=100, l_len=1}) = 0

  I traced the passing of the --force-share option through the source
  code (I used commit 6c599282f8 as of Mon Feb 17 13:32:25 2020 +0000)

  qemu-img.c:img_info()
          force_share = true;
  qemu-img.c:collect_image_info_list(force_share)
  qemu-img.c:img_open(force_share)
  qemu-img.c:img_open_file(force_share)
          qdict_put_bool(options, BDRV_OPT_FORCE_SHARE, true);
  block/block-backend.c:blk_new_open(options)
  block.c:bdrv_open(options)
  block.c:bdrv_open_inheritoptions()
  block.c:bdrv_open_common(options)
          bs->force_share = qemu_opt_get_bool(opts, BDRV_OPT_FORCE_SHARE, false);
  block.c:bdrv_open_driver(bs)
  include/block/block_int.h:int (*bdrv_file_open)(BlockDriverState *bs, QDict *options, int flags,
  block/file-posix.c:.bdrv_file_open = raw_open,
  block/file-posix.c:raw_open_common(bs)
          locking = qapi_enum_parse(&OnOffAuto_lookup,
                                qemu_opt_get(opts, "locking"),
                                ON_OFF_AUTO_AUTO, &local_err);
          ignoring bs->force_share

  At the end, bs->force_share is ignored in determining the locking
  value.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1865048/+subscriptions