All of lore.kernel.org
 help / color / mirror / Atom feed
* [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images
@ 2014-09-12 14:58 Michael Steffens
  2014-09-12 15:08 ` [Qemu-devel] [Bug 1368815] " Michael Steffens
                   ` (57 more replies)
  0 siblings, 58 replies; 59+ messages in thread
From: Michael Steffens @ 2014-09-12 14:58 UTC (permalink / raw)
  To: qemu-devel

Public bug reported:

-- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
Ubuntu 14.04 using Ext4 filesystems.

The command

  qemu-img convert -O raw inputimage.qcow2 outputimage.raw

intermittently creates corrupted output images, when the input image is
not yet fully synchronized to disk. While the issue has actually been
discovered in operation of of OpenStack nova, it can be reproduced
"easily" on command line using

  cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
$DST_PATH && cksum $DST_PATH

on filesystems exposing this behavior. (The difficult part of this
exercise is to prepare a filesystem to reliably trigger this race. On my
test machine some filesystems are affected while other aren't, and
unfortunately I haven't found the relevant difference between them, yet.
Possible it's timing issues completely out of userspace control ...)

The root cause, however, is the same as in

  http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

and it can be solved the same way as suggested in

  http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

    f.fm.fm_flags = 0;

to

    f.fm.fm_flags = FIEMAP_FLAG_SYNC;

As discussed in the thread mentioned above, retrieving a page cache
coherent map of file extents is possible only after fsync on that file.

See also

  https://bugs.launchpad.net/nova/+bug/1350766

In that bug report filed against nova, fsync had been suggested to be
performed by the framework invoking qemu-img. However, as the choice of
fiemap -- implying this otherwise unneeded fsync of a temporary file  --
is not made by the caller but by qemu-img, I agree with the nova bug
reviewer's objection to put it into nova. The fsync should instead be
triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC, specifically
intended for that purpose.

** Affects: qemu
     Importance: Undecided
         Status: New


** Tags: fiemap nova openstack qemu-img

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in QEMU:
  New

Bug description:
  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
@ 2014-09-12 15:08 ` Michael Steffens
  2014-09-12 15:13 ` Matt Riedemann
                   ` (56 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Michael Steffens @ 2014-09-12 15:08 UTC (permalink / raw)
  To: qemu-devel

** Also affects: nova
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  Triaged
Status in QEMU:
  New

Bug description:
  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
  2014-09-12 15:08 ` [Qemu-devel] [Bug 1368815] " Michael Steffens
@ 2014-09-12 15:13 ` Matt Riedemann
  2014-09-14 21:04 ` Michael Steffens
                   ` (55 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Matt Riedemann @ 2014-09-12 15:13 UTC (permalink / raw)
  To: qemu-devel

Is there a minimum version of qemu that would be required to use the
FIEMAP_FLAG_SYNC flag?

** Changed in: nova
       Status: New => Triaged

** Changed in: nova
   Importance: Undecided => Medium

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  Triaged
Status in QEMU:
  New

Bug description:
  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
  2014-09-12 15:08 ` [Qemu-devel] [Bug 1368815] " Michael Steffens
  2014-09-12 15:13 ` Matt Riedemann
@ 2014-09-14 21:04 ` Michael Steffens
  2014-09-15 10:08 ` Michael Steffens
                   ` (54 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Michael Steffens @ 2014-09-14 21:04 UTC (permalink / raw)
  To: qemu-devel

The affected code was introduced with version 1.2.0. However, due to
https://bugs.launchpad.net/qemu/+bug/1193628 I can't build these old
releases to verify whether they actually expose the same behaviour.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  Triaged
Status in QEMU:
  New

Bug description:
  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (2 preceding siblings ...)
  2014-09-14 21:04 ` Michael Steffens
@ 2014-09-15 10:08 ` Michael Steffens
  2014-09-15 10:30 ` Pádraig Brady
                   ` (53 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Michael Steffens @ 2014-09-15 10:08 UTC (permalink / raw)
  To: qemu-devel

It seems the dust settles a bit: Found the relevant difference between
my various filesystems, and how to reproduce the failure: Susceptible
filesystems don't have the extent feature of ext4 enabled.

You can create such a filesystem using

  mke2fs -t ext4 -O ^extent /dev/...
  mount /mnt /dev/...
 
Adapting the command line example provided above you can see

  rm -f /mnt/tmp.qcow2
  cat $SRC_PATH > /mnt/tmp.qcow2 && qemu-img convert -O raw  /mnt/tmp.qcow /mnt/tmp.qcow
  cksum  /mnt/tmp.qcow

creating corrupt (usually nullified) result images. By inserting a sleep
of at least 33 seconds between the cat command and the qemu-img
invocation I'm getting proper output.

To me it's unclear now, where the actual defect is located. Creating
ext4 filesystems with certain features disabled (such as the exetent
tree) is apparently supported and ok. Is the fiemap ioctl supposed to
handle this gracefully, for example by assuming FIEMAP_FLAG_SYNC in
absence of an extent tree? Or are clients such as qemu-img supposed to
always FIEMAP_FLAG_SYNC to be safe?

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  Triaged
Status in QEMU:
  New

Bug description:
  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (3 preceding siblings ...)
  2014-09-15 10:08 ` Michael Steffens
@ 2014-09-15 10:30 ` Pádraig Brady
  2014-09-15 12:52 ` Michael Steffens
                   ` (52 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Pádraig Brady @ 2014-09-15 10:30 UTC (permalink / raw)
  To: qemu-devel

I see seek hole is supported in the latest qemu-img so I would reorder
so that's tried first like:

    if lseek(SEEK_HOLE) == ENOTSUP
        use_that
        if fiemap(FIEMAP_FLAG_SYNC)
            use_that

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  Triaged
Status in QEMU:
  New

Bug description:
  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (4 preceding siblings ...)
  2014-09-15 10:30 ` Pádraig Brady
@ 2014-09-15 12:52 ` Michael Steffens
  2014-09-15 15:16 ` Matt Riedemann
                   ` (51 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Michael Steffens @ 2014-09-15 12:52 UTC (permalink / raw)
  To: qemu-devel

The fallback cascade Pádraig mentions is already implemented in
qemu-2.1.0, in function raw_co_get_block_status. Just swap

  ret = try_fiemap( ... )

and

  ret = try_seek_hole( ... )

to reverse the order. I can confirm that it works just fine on 3.13
kernel (all version since 3.1, according to lseek(2)), while older
versions will fall back to fiemap, which needs to be protected with
FIEMAP_FLAG_SYNC in try_fiemap, to be safe.

This should work under all conditions, and avoid redundant syncs where
possible, right?

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  Triaged
Status in QEMU:
  New

Bug description:
  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (5 preceding siblings ...)
  2014-09-15 12:52 ` Michael Steffens
@ 2014-09-15 15:16 ` Matt Riedemann
  2014-09-15 18:27 ` Serge Hallyn
                   ` (50 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Matt Riedemann @ 2014-09-15 15:16 UTC (permalink / raw)
  To: qemu-devel

Marking as High since duplicate bug 1350766 was marked High.

** Changed in: nova
   Importance: Medium => High

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  Triaged
Status in QEMU:
  New

Bug description:
  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (6 preceding siblings ...)
  2014-09-15 15:16 ` Matt Riedemann
@ 2014-09-15 18:27 ` Serge Hallyn
  2014-09-30  4:51 ` Tony Breeds
                   ` (49 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Serge Hallyn @ 2014-09-15 18:27 UTC (permalink / raw)
  To: qemu-devel

** Also affects: qemu (Ubuntu)
   Importance: Undecided
       Status: New

** Changed in: qemu (Ubuntu)
       Status: New => Triaged

** Changed in: qemu (Ubuntu)
   Importance: Undecided => High

** Also affects: qemu (Ubuntu Trusty)
   Importance: Undecided
       Status: New

** Changed in: qemu (Ubuntu Trusty)
       Status: New => Triaged

** Changed in: qemu (Ubuntu Trusty)
   Importance: Undecided => High

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  Triaged
Status in QEMU:
  New
Status in “qemu” package in Ubuntu:
  Triaged
Status in “qemu” source package in Trusty:
  Triaged

Bug description:
  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (7 preceding siblings ...)
  2014-09-15 18:27 ` Serge Hallyn
@ 2014-09-30  4:51 ` Tony Breeds
  2014-10-22 23:21 ` Tony Breeds
                   ` (48 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Tony Breeds @ 2014-09-30  4:51 UTC (permalink / raw)
  To: qemu-devel

openstack review at:
  https://review.openstack.org/#/c/123957/

Qemu patches at:
  http://patchwork.ozlabs.org/patch/393494/ ; and
  http://patchwork.ozlabs.org/patch/393495/

** Changed in: nova
     Assignee: (unassigned) => Tony Breeds (o-tony)

** Changed in: qemu
       Status: New => In Progress

** Changed in: qemu
     Assignee: (unassigned) => Tony Breeds (o-tony)

** Changed in: nova
       Status: Triaged => In Progress

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in “qemu” package in Ubuntu:
  Triaged
Status in “qemu” source package in Trusty:
  Triaged

Bug description:
  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (8 preceding siblings ...)
  2014-09-30  4:51 ` Tony Breeds
@ 2014-10-22 23:21 ` Tony Breeds
  2014-10-23 18:22 ` Serge Hallyn
                   ` (47 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Tony Breeds @ 2014-10-22 23:21 UTC (permalink / raw)
  To: qemu-devel

FWIW the following 2 commits in qemu master resolve the issue for qemu-
img.

  http://git.qemu.org/?p=qemu.git;a=commit;h=38c4d0aea3e1264c86e282d99560330adf2b6e25
  http://git.qemu.org/?p=qemu.git;a=commit;h=7c15903789953ead14a417882657d52dc0c19a24

If possible they should be back ported to trusty and utopic.

You'll also need something like:

http://git.qemu.org/?p=qemu.git;a=commit;h=4f11aa8a40351b28c0e67c7276e0003b38cc46ac

before my 2 patches.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in “qemu” package in Ubuntu:
  Triaged
Status in “qemu” source package in Trusty:
  Triaged

Bug description:
  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (9 preceding siblings ...)
  2014-10-22 23:21 ` Tony Breeds
@ 2014-10-23 18:22 ` Serge Hallyn
  2014-10-27  5:27 ` Launchpad Bug Tracker
                   ` (46 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Serge Hallyn @ 2014-10-23 18:22 UTC (permalink / raw)
  To: qemu-devel

Thanks for the information.  Looks like we can apply these in debian
too.

** Also affects: qemu (Ubuntu Utopic)
   Importance: High
       Status: Triaged

** Also affects: qemu (Ubuntu Vivid)
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in “qemu” package in Ubuntu:
  Triaged
Status in “qemu” source package in Trusty:
  Triaged
Status in “qemu” source package in Utopic:
  Triaged
Status in “qemu” source package in Vivid:
  New

Bug description:
  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (10 preceding siblings ...)
  2014-10-23 18:22 ` Serge Hallyn
@ 2014-10-27  5:27 ` Launchpad Bug Tracker
  2014-10-30  6:40 ` Launchpad Bug Tracker
                   ` (45 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Launchpad Bug Tracker @ 2014-10-27  5:27 UTC (permalink / raw)
  To: qemu-devel

Status changed to 'Confirmed' because the bug affects multiple users.

** Changed in: qemu (Ubuntu Vivid)
       Status: New => Confirmed

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in “qemu” package in Ubuntu:
  Confirmed
Status in “qemu” source package in Trusty:
  Triaged
Status in “qemu” source package in Utopic:
  Triaged
Status in “qemu” source package in Vivid:
  Confirmed

Bug description:
  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (11 preceding siblings ...)
  2014-10-27  5:27 ` Launchpad Bug Tracker
@ 2014-10-30  6:40 ` Launchpad Bug Tracker
  2014-11-20 22:23 ` Serge Hallyn
                   ` (44 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Launchpad Bug Tracker @ 2014-10-30  6:40 UTC (permalink / raw)
  To: qemu-devel

This bug was fixed in the package qemu - 2.1+dfsg-4ubuntu7

---------------
qemu (2.1+dfsg-4ubuntu7) vivid; urgency=medium

  * Apply two patches to fix intermittent qemu-img corruption
    (LP: #1368815)
    - 501-block-raw-posix-fix-disk-corruption-in-try-fiemap
    - 502-block-raw-posic-use-seek-hole-ahead-of-fiemap
 -- Serge Hallyn <serge.hallyn@ubuntu.com>   Wed, 29 Oct 2014 22:31:43 -0500

** Changed in: qemu (Ubuntu Vivid)
       Status: Confirmed => Fix Released

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in “qemu” package in Ubuntu:
  Fix Released
Status in “qemu” source package in Trusty:
  Triaged
Status in “qemu” source package in Utopic:
  Triaged
Status in “qemu” source package in Vivid:
  Fix Released

Bug description:
  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (12 preceding siblings ...)
  2014-10-30  6:40 ` Launchpad Bug Tracker
@ 2014-11-20 22:23 ` Serge Hallyn
  2014-11-20 22:42 ` Tony Breeds
                   ` (43 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Serge Hallyn @ 2014-11-20 22:23 UTC (permalink / raw)
  To: qemu-devel

** Description changed:

+ ==========================================================
+ Impact: occasional qcow2 corruption
+ Test case: see the qemu-img command below
+ Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
+ ==========================================================
+ 
  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.
  
  The command
  
    qemu-img convert -O raw inputimage.qcow2 outputimage.raw
  
  intermittently creates corrupted output images, when the input image is
  not yet fully synchronized to disk. While the issue has actually been
  discovered in operation of of OpenStack nova, it can be reproduced
  "easily" on command line using
  
    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH
  
  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On my
  test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them, yet.
  Possible it's timing issues completely out of userspace control ...)
  
  The root cause, however, is the same as in
  
    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html
  
  and it can be solved the same way as suggested in
  
    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html
  
  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change
  
      f.fm.fm_flags = 0;
  
  to
  
      f.fm.fm_flags = FIEMAP_FLAG_SYNC;
  
  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that file.
  
  See also
  
    https://bugs.launchpad.net/nova/+bug/1350766
  
  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice of
  fiemap -- implying this otherwise unneeded fsync of a temporary file  --
  is not made by the caller but by qemu-img, I agree with the nova bug
  reviewer's objection to put it into nova. The fsync should instead be
  triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC, specifically
  intended for that purpose.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in “qemu” package in Ubuntu:
  Fix Released
Status in “qemu” source package in Trusty:
  Triaged
Status in “qemu” source package in Utopic:
  Triaged
Status in “qemu” source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional qcow2 corruption
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (13 preceding siblings ...)
  2014-11-20 22:23 ` Serge Hallyn
@ 2014-11-20 22:42 ` Tony Breeds
  2014-11-21  4:17 ` Serge Hallyn
                   ` (42 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Tony Breeds @ 2014-11-20 22:42 UTC (permalink / raw)
  To: qemu-devel

Hi Serge,


Is there any chance these fixes will go into trusty?

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in “qemu” package in Ubuntu:
  Fix Released
Status in “qemu” source package in Trusty:
  Triaged
Status in “qemu” source package in Utopic:
  Triaged
Status in “qemu” source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional qcow2 corruption
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (14 preceding siblings ...)
  2014-11-20 22:42 ` Tony Breeds
@ 2014-11-21  4:17 ` Serge Hallyn
  2014-11-21  4:47 ` Tony Breeds
                   ` (41 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Serge Hallyn @ 2014-11-21  4:17 UTC (permalink / raw)
  To: qemu-devel

Hi Tony,

yes, I've uploaded a proposed fix for trusty-proposed earlier today.  It
should be available for testing as soon as it is accepted.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in “qemu” package in Ubuntu:
  Fix Released
Status in “qemu” source package in Trusty:
  Triaged
Status in “qemu” source package in Utopic:
  Triaged
Status in “qemu” source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional qcow2 corruption
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (15 preceding siblings ...)
  2014-11-21  4:17 ` Serge Hallyn
@ 2014-11-21  4:47 ` Tony Breeds
  2014-11-25 11:44 ` Kevin Wolf
                   ` (40 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Tony Breeds @ 2014-11-21  4:47 UTC (permalink / raw)
  To: qemu-devel

Awesome.

Thanks!

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in “qemu” package in Ubuntu:
  Fix Released
Status in “qemu” source package in Trusty:
  Triaged
Status in “qemu” source package in Utopic:
  Triaged
Status in “qemu” source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional qcow2 corruption
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (16 preceding siblings ...)
  2014-11-21  4:47 ` Tony Breeds
@ 2014-11-25 11:44 ` Kevin Wolf
  2014-11-25 17:22 ` Chris J Arges
                   ` (39 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Kevin Wolf @ 2014-11-25 11:44 UTC (permalink / raw)
  To: qemu-devel

** Description changed:

  ==========================================================
- Impact: occasional qcow2 corruption
+ Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================
  
  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.
  
  The command
  
-   qemu-img convert -O raw inputimage.qcow2 outputimage.raw
+   qemu-img convert -O raw inputimage.qcow2 outputimage.raw
  
  intermittently creates corrupted output images, when the input image is
  not yet fully synchronized to disk. While the issue has actually been
  discovered in operation of of OpenStack nova, it can be reproduced
  "easily" on command line using
  
-   cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
+   cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH
  
  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On my
  test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them, yet.
  Possible it's timing issues completely out of userspace control ...)
  
  The root cause, however, is the same as in
  
-   http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html
+   http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html
  
  and it can be solved the same way as suggested in
  
-   http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html
+   http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html
  
  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change
  
-     f.fm.fm_flags = 0;
+     f.fm.fm_flags = 0;
  
  to
  
-     f.fm.fm_flags = FIEMAP_FLAG_SYNC;
+     f.fm.fm_flags = FIEMAP_FLAG_SYNC;
  
  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that file.
  
  See also
  
-   https://bugs.launchpad.net/nova/+bug/1350766
+   https://bugs.launchpad.net/nova/+bug/1350766
  
  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice of
  fiemap -- implying this otherwise unneeded fsync of a temporary file  --
  is not made by the caller but by qemu-img, I agree with the nova bug
  reviewer's objection to put it into nova. The fsync should instead be
  triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC, specifically
  intended for that purpose.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in “qemu” package in Ubuntu:
  Fix Released
Status in “qemu” source package in Trusty:
  Triaged
Status in “qemu” source package in Utopic:
  Triaged
Status in “qemu” source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (17 preceding siblings ...)
  2014-11-25 11:44 ` Kevin Wolf
@ 2014-11-25 17:22 ` Chris J Arges
  2014-11-25 17:24 ` [Qemu-devel] [Bug 1368815] Please test proposed package Chris J Arges
                   ` (38 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Chris J Arges @ 2014-11-25 17:22 UTC (permalink / raw)
  To: qemu-devel

Hello Michael, or anyone else affected,

Accepted qemu into utopic-proposed. The package will build now and be
available at http://launchpad.net/ubuntu/+source/qemu/2.1+dfsg-
4ubuntu6.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to
enable and use -proposed.  Your feedback will aid us getting this update
out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, and change the tag
from verification-needed to verification-done. If it does not fix the
bug for you, please add a comment stating that, and change the tag to
verification-failed.  In either case, details of your testing will help
us make a better decision.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance!

** Changed in: qemu (Ubuntu Utopic)
       Status: Triaged => Fix Committed

** Tags added: verification-needed

** Changed in: qemu (Ubuntu Trusty)
       Status: Triaged => Fix Committed

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in “qemu” package in Ubuntu:
  Fix Released
Status in “qemu” source package in Trusty:
  Fix Committed
Status in “qemu” source package in Utopic:
  Fix Committed
Status in “qemu” source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Please test proposed package
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (18 preceding siblings ...)
  2014-11-25 17:22 ` Chris J Arges
@ 2014-11-25 17:24 ` Chris J Arges
  2014-12-01  8:23 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (37 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Chris J Arges @ 2014-11-25 17:24 UTC (permalink / raw)
  To: qemu-devel

Hello Michael, or anyone else affected,

Accepted qemu into trusty-proposed. The package will build now and be
available at http://launchpad.net/ubuntu/+source/qemu/2.0.0+dfsg-
2ubuntu1.8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to
enable and use -proposed.  Your feedback will aid us getting this update
out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, and change the tag
from verification-needed to verification-done. If it does not fix the
bug for you, please add a comment stating that, and change the tag to
verification-failed.  In either case, details of your testing will help
us make a better decision.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance!

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in “qemu” package in Ubuntu:
  Fix Released
Status in “qemu” source package in Trusty:
  Fix Committed
Status in “qemu” source package in Utopic:
  Fix Committed
Status in “qemu” source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (19 preceding siblings ...)
  2014-11-25 17:24 ` [Qemu-devel] [Bug 1368815] Please test proposed package Chris J Arges
@ 2014-12-01  8:23 ` Michael Steffens
  2014-12-01  8:32 ` Michael Steffens
                   ` (36 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Michael Steffens @ 2014-12-01  8:23 UTC (permalink / raw)
  To: qemu-devel

Tested qemu-utils  2.0.0+dfsg-2ubuntu1.8. Successful.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Committed
Status in qemu source package in Utopic:
  Fix Committed
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (20 preceding siblings ...)
  2014-12-01  8:23 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Michael Steffens
@ 2014-12-01  8:32 ` Michael Steffens
  2014-12-03  5:14 ` Mathew Hodson
                   ` (35 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Michael Steffens @ 2014-12-01  8:32 UTC (permalink / raw)
  To: qemu-devel

** Tags removed: verification-needed
** Tags added: verification-done-trusty

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Committed
Status in qemu source package in Utopic:
  Fix Committed
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (21 preceding siblings ...)
  2014-12-01  8:32 ` Michael Steffens
@ 2014-12-03  5:14 ` Mathew Hodson
  2014-12-08 18:40 ` [Qemu-devel] [Bug 1368815] Update Released Brian Murray
                   ` (34 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Mathew Hodson @ 2014-12-03  5:14 UTC (permalink / raw)
  To: qemu-devel

** Tags added: verification-needed-utopic

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Committed
Status in qemu source package in Utopic:
  Fix Committed
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Update Released
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (22 preceding siblings ...)
  2014-12-03  5:14 ` Mathew Hodson
@ 2014-12-08 18:40 ` Brian Murray
  2014-12-08 18:46 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Launchpad Bug Tracker
                   ` (33 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Brian Murray @ 2014-12-08 18:40 UTC (permalink / raw)
  To: qemu-devel

The verification of the Stable Release Update for qemu has completed
successfully and the package has now been released to -updates.
Subsequently, the Ubuntu Stable Release Updates Team is being
unsubscribed and will not receive messages about this bug report.  In
the event that you encounter a regression using the package from
-updates please report a new bug using ubuntu-bug and tag the bug report
regression-update so we can easily find any regressions.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Committed
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (23 preceding siblings ...)
  2014-12-08 18:40 ` [Qemu-devel] [Bug 1368815] Update Released Brian Murray
@ 2014-12-08 18:46 ` Launchpad Bug Tracker
  2014-12-10 16:38 ` Serge Hallyn
                   ` (32 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Launchpad Bug Tracker @ 2014-12-08 18:46 UTC (permalink / raw)
  To: qemu-devel

This bug was fixed in the package qemu - 2.0.0+dfsg-2ubuntu1.8

---------------
qemu (2.0.0+dfsg-2ubuntu1.8) trusty-proposed; urgency=medium

  * debian/qemu-system-x86.qemu-kvm.upstart: create /dev/kvm in a
    container. (LP: #1370199)
  * Cherrypick upstream patch to fix intermittent qemu-img corruption
    (LP: #1368815)
    - 501-block-raw-posix-fix-disk-corruption-in-try-fiemap
    - (note - 502-block-raw-posic-use-seek-hole-ahead-of-fiemap (which was
      also needed in utopic) appears to be unneeded here as the code being
      changed has not yet been switched to using try_fiemap)
 -- Serge Hallyn <serge.hallyn@ubuntu.com>   Thu, 20 Nov 2014 11:24:51 -0600

** Changed in: qemu (Ubuntu Trusty)
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Committed
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (24 preceding siblings ...)
  2014-12-08 18:46 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Launchpad Bug Tracker
@ 2014-12-10 16:38 ` Serge Hallyn
  2014-12-10 18:06 ` Vladimir Kuklin
                   ` (31 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Serge Hallyn @ 2014-12-10 16:38 UTC (permalink / raw)
  To: qemu-devel

@Michael,

by any chance would you be albe to test on utopic?

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Committed
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (25 preceding siblings ...)
  2014-12-10 16:38 ` Serge Hallyn
@ 2014-12-10 18:06 ` Vladimir Kuklin
  2014-12-10 18:32 ` Serge Hallyn
                   ` (30 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Vladimir Kuklin @ 2014-12-10 18:06 UTC (permalink / raw)
  To: qemu-devel

** Also affects: mos
   Importance: Undecided
       Status: New

** Changed in: mos
       Status: New => Triaged

** Changed in: mos
   Importance: Undecided => Critical

** Changed in: mos
     Assignee: (unassigned) => MOS Linux (mos-linux)

** Changed in: mos
    Milestone: None => 6.0

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Mirantis OpenStack:
  Triaged
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Committed
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/mos/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (26 preceding siblings ...)
  2014-12-10 18:06 ` Vladimir Kuklin
@ 2014-12-10 18:32 ` Serge Hallyn
  2014-12-10 20:34 ` Chris J Arges
                   ` (29 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Serge Hallyn @ 2014-12-10 18:32 UTC (permalink / raw)
  To: qemu-devel

I couldn't reproduce the bug on the old qemu myself, however Michael has
verified the (same) fix on trusty, and the full qa-regression-test
passed for me on utopic-proposed.  So I would request that we call this
verification-done.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Mirantis OpenStack:
  Triaged
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Committed
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/mos/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (27 preceding siblings ...)
  2014-12-10 18:32 ` Serge Hallyn
@ 2014-12-10 20:34 ` Chris J Arges
  2014-12-10 20:49 ` Chris J Arges
                   ` (28 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Chris J Arges @ 2014-12-10 20:34 UTC (permalink / raw)
  To: qemu-devel

Looking at the fixes, I also see the following commits remove the above changes, which could mean we might encounter this again:
c4875e5 raw-posix: SEEK_HOLE suffices, get rid of FIEMAP
d1f06fe raw-posix: The SEEK_HOLE code is flawed, rewrite it

Note there is also a related issue:
bug 1292234
So far testing with the proposed qemu version or upstream I still encounter issues on ext4 w/ ^extent and ext3 filesystems.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Mirantis OpenStack:
  Triaged
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Committed
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/mos/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (28 preceding siblings ...)
  2014-12-10 20:34 ` Chris J Arges
@ 2014-12-10 20:49 ` Chris J Arges
  2014-12-10 20:50 ` Dmitry Mescheryakov
                   ` (27 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Chris J Arges @ 2014-12-10 20:49 UTC (permalink / raw)
  To: qemu-devel

** Tags removed: verification-needed-utopic
** Tags added: verification-done-utopic

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Committed
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (29 preceding siblings ...)
  2014-12-10 20:49 ` Chris J Arges
@ 2014-12-10 20:50 ` Dmitry Mescheryakov
  2014-12-10 21:54 ` Tony Breeds
                   ` (26 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Dmitry Mescheryakov @ 2014-12-10 20:50 UTC (permalink / raw)
  To: qemu-devel

Filed a separate issue for MOS
https://bugs.launchpad.net/mos/+bug/1401261

** No longer affects: mos

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Committed
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (30 preceding siblings ...)
  2014-12-10 20:50 ` Dmitry Mescheryakov
@ 2014-12-10 21:54 ` Tony Breeds
  2014-12-11  3:54 ` Chris J Arges
                   ` (25 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Tony Breeds @ 2014-12-10 21:54 UTC (permalink / raw)
  To: qemu-devel

Hi Chris,
Markus' rework will not reintroduce this bug as it completely removes all fiemap code.

bug 129224 is a different issue, I'll comment on that bug.

You say: you encounter issues with upstream with ^extent and ext3
filesystems.  Just to be clear: Are you saying that *this* bug is still
a problem for you?

if it's a different bug then I write it up and I'll take a look.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Committed
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (31 preceding siblings ...)
  2014-12-10 21:54 ` Tony Breeds
@ 2014-12-11  3:54 ` Chris J Arges
  2014-12-11  3:57 ` Chris J Arges
                   ` (24 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Chris J Arges @ 2014-12-11  3:54 UTC (permalink / raw)
  To: qemu-devel

Tony,

Yea, its a different bug. I tested with the above patched package and
upstream qemu from git, and I can still hit bug 129224. I was hoping
this also fixed my issue, but unfortunately it seems to be a different
issue that occurs when using the same types of filesystems. I have a
solid reproducer on my desk so let me know which experiments / areas of
code / etc I should look at.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Committed
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (32 preceding siblings ...)
  2014-12-11  3:54 ` Chris J Arges
@ 2014-12-11  3:57 ` Chris J Arges
  2014-12-11  4:47 ` Tony Breeds
                   ` (23 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Chris J Arges @ 2014-12-11  3:57 UTC (permalink / raw)
  To: qemu-devel

Just to clarify it's bug 1292234 in the previous comment.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Committed
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (33 preceding siblings ...)
  2014-12-11  3:57 ` Chris J Arges
@ 2014-12-11  4:47 ` Tony Breeds
  2014-12-11 17:53 ` Launchpad Bug Tracker
                   ` (22 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Tony Breeds @ 2014-12-11  4:47 UTC (permalink / raw)
  To: qemu-devel

Chris,
I've read through 1292234 and I'll have a play with your reproducer locally and see if I can gain any insight.

I'm sorry my fix didn't help 1292234, but glad you can't hit 1368815
with upstream, I was kinda having kittens here ;P

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Committed
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (34 preceding siblings ...)
  2014-12-11  4:47 ` Tony Breeds
@ 2014-12-11 17:53 ` Launchpad Bug Tracker
  2014-12-11 20:11 ` Vish Ishaya
                   ` (21 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Launchpad Bug Tracker @ 2014-12-11 17:53 UTC (permalink / raw)
  To: qemu-devel

This bug was fixed in the package qemu - 2.1+dfsg-4ubuntu6.2

---------------
qemu (2.1+dfsg-4ubuntu6.2) utopic-proposed; urgency=medium

  * Apply two patches to fix intermittent qemu-img corruption
    (LP: #1368815)
    - 501-block-raw-posix-fix-disk-corruption-in-try-fiemap
    - 502-block-raw-posic-use-seek-hole-ahead-of-fiemap
 -- Serge Hallyn <serge.hallyn@ubuntu.com>   Thu, 20 Nov 2014 16:33:09 -0600

** Changed in: qemu (Ubuntu Utopic)
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (35 preceding siblings ...)
  2014-12-11 17:53 ` Launchpad Bug Tracker
@ 2014-12-11 20:11 ` Vish Ishaya
  2014-12-12  1:56 ` Tony Breeds
                   ` (20 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Vish Ishaya @ 2014-12-11 20:11 UTC (permalink / raw)
  To: qemu-devel

** Also affects: cinder
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  New
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (36 preceding siblings ...)
  2014-12-11 20:11 ` Vish Ishaya
@ 2014-12-12  1:56 ` Tony Breeds
  2014-12-12  1:58 ` Tony Breeds
                   ` (19 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Tony Breeds @ 2014-12-12  1:56 UTC (permalink / raw)
  To: qemu-devel

I'm happy to tackle to also fix cinder with a version of the nova fix
(for consistency).  I propose waiting until the nova fix lands

** Changed in: cinder
     Assignee: (unassigned) => Tony Breeds (o-tony)

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  In Progress
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (37 preceding siblings ...)
  2014-12-12  1:56 ` Tony Breeds
@ 2014-12-12  1:58 ` Tony Breeds
  2014-12-12  3:28 ` OpenStack Infra
                   ` (18 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Tony Breeds @ 2014-12-12  1:58 UTC (permalink / raw)
  To: qemu-devel

I'd elevate this to high so it matches nova and ubuntu but I don't have
permissions to do so.

** Changed in: cinder
       Status: New => In Progress

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  In Progress
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (38 preceding siblings ...)
  2014-12-12  1:58 ` Tony Breeds
@ 2014-12-12  3:28 ` OpenStack Infra
  2014-12-15 16:26 ` Jay Bryant
                   ` (17 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: OpenStack Infra @ 2014-12-12  3:28 UTC (permalink / raw)
  To: qemu-devel

Fix proposed to branch: master
Review: https://review.openstack.org/141259

** Changed in: cinder
     Assignee: Tony Breeds (o-tony) => John Griffith (john-griffith)

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  In Progress
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (39 preceding siblings ...)
  2014-12-12  3:28 ` OpenStack Infra
@ 2014-12-15 16:26 ` Jay Bryant
  2014-12-16 12:31 ` Alexei Sheplyakov
                   ` (16 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Jay Bryant @ 2014-12-15 16:26 UTC (permalink / raw)
  To: qemu-devel

** Changed in: cinder
   Importance: Undecided => High

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  In Progress
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (40 preceding siblings ...)
  2014-12-15 16:26 ` Jay Bryant
@ 2014-12-16 12:31 ` Alexei Sheplyakov
  2014-12-16 12:33 ` Alexei Sheplyakov
                   ` (15 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Alexei Sheplyakov @ 2014-12-16 12:31 UTC (permalink / raw)
  To: qemu-devel

> - 501-block-raw-posix-fix-disk-corruption-in-try-fiemap
>   - (note - 502-block-raw-posic-use-seek-hole-ahead-of-fiemap (which was
>     also needed in utopic) appears to be unneeded here as the code being
>      changed has not yet been switched to using try_fiemap)

Actually such a enforces fsync and drastically reduces the performance of conversion.
I propose to use seek_hole instead of FIEMAP (which is basically what 
 502-block-raw-posic-use-seek-hole-ahead-of-fiemap does). 


** Patch added: "0500-block-raw-posix-Try-both-FIEMAP-and-SEEK_HOLE.patch"
   https://bugs.launchpad.net/qemu/+bug/1368815/+attachment/4282118/+files/0500-block-raw-posix-Try-both-FIEMAP-and-SEEK_HOLE.patch

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  In Progress
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (41 preceding siblings ...)
  2014-12-16 12:31 ` Alexei Sheplyakov
@ 2014-12-16 12:33 ` Alexei Sheplyakov
  2014-12-16 22:00 ` Tony Breeds
                   ` (14 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Alexei Sheplyakov @ 2014-12-16 12:33 UTC (permalink / raw)
  To: qemu-devel

The second part of the fix (which does not reduce the performance) for
qemu 2.0 (apparently uploading two patches at once is not so easy)

** Patch added: "0502-block-raw-posix-use-seek_hole-ahead-of-fiemap.patch"
   https://bugs.launchpad.net/qemu/+bug/1368815/+attachment/4282119/+files/0502-block-raw-posix-use-seek_hole-ahead-of-fiemap.patch

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  In Progress
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (42 preceding siblings ...)
  2014-12-16 12:33 ` Alexei Sheplyakov
@ 2014-12-16 22:00 ` Tony Breeds
  2014-12-17  6:44 ` Alexei Sheplyakov
                   ` (13 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Tony Breeds @ 2014-12-16 22:00 UTC (permalink / raw)
  To: qemu-devel

Patchg 0500-block-raw-posix-Try-both-FIEMAP-and-SEEK_HOLE.patch appears
to be part of a bigger re-write of the related code.   and is ON TOP of
the patches already applied in this bug.


No doubt the rewirtten code is "better" but backporting it contains more risk than the 2 simple fixes I already nominated.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  In Progress
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (43 preceding siblings ...)
  2014-12-16 22:00 ` Tony Breeds
@ 2014-12-17  6:44 ` Alexei Sheplyakov
  2014-12-21 19:29 ` [Qemu-devel] [Bug 1368815] Change abandoned on cinder (master) OpenStack Infra
                   ` (12 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Alexei Sheplyakov @ 2014-12-17  6:44 UTC (permalink / raw)
  To: qemu-devel

> Patch 0500-block-raw-posix-Try-both-FIEMAP-and-SEEK_HOLE.patch appears to be part of a bigger re-write
> of the related code. and is ON TOP of the patches already applied in this bug.

Yep, sorry for not mentioning this. As far as I understand qemu-2.1 package contains this partially rewritten
code too (without any recent changes like disabling FIEMAP completely and rewriting the code using SEEK_HOLE).

> No doubt the rewirtten code is "better" but backporting it contains
more risk than the 2 simple fixes I already nominated.

Can we completely disable the FIEMAP code and pretend that all blocks are allocated? I'm afraid fsync'ing 100+ GB
files might be even slower than ignoring the sparseness.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  In Progress
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Change abandoned on cinder (master)
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (44 preceding siblings ...)
  2014-12-17  6:44 ` Alexei Sheplyakov
@ 2014-12-21 19:29 ` OpenStack Infra
  2014-12-23  0:44 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images OpenStack Infra
                   ` (11 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: OpenStack Infra @ 2014-12-21 19:29 UTC (permalink / raw)
  To: qemu-devel

Change abandoned by John Griffith (john.griffith8@gmail.com) on branch: master
Review: https://review.openstack.org/141259

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  In Progress
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (45 preceding siblings ...)
  2014-12-21 19:29 ` [Qemu-devel] [Bug 1368815] Change abandoned on cinder (master) OpenStack Infra
@ 2014-12-23  0:44 ` OpenStack Infra
  2015-01-26 15:27 ` [Qemu-devel] [Bug 1368815] Change abandoned on cinder (master) OpenStack Infra
                   ` (10 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: OpenStack Infra @ 2014-12-23  0:44 UTC (permalink / raw)
  To: qemu-devel

Fix proposed to branch: master
Review: https://review.openstack.org/143575

** Changed in: cinder
     Assignee: John Griffith (john-griffith) => Tony Breeds (o-tony)

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  In Progress
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Change abandoned on cinder (master)
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (46 preceding siblings ...)
  2014-12-23  0:44 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images OpenStack Infra
@ 2015-01-26 15:27 ` OpenStack Infra
  2015-01-26 20:32 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Mike Perez
                   ` (9 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: OpenStack Infra @ 2015-01-26 15:27 UTC (permalink / raw)
  To: qemu-devel

Change abandoned by Mike Perez (thingee@gmail.com) on branch: master
Review: https://review.openstack.org/143575
Reason: 1 month, no update.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  In Progress
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (47 preceding siblings ...)
  2015-01-26 15:27 ` [Qemu-devel] [Bug 1368815] Change abandoned on cinder (master) OpenStack Infra
@ 2015-01-26 20:32 ` Mike Perez
  2015-01-26 22:27 ` Tony Breeds
                   ` (8 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Mike Perez @ 2015-01-26 20:32 UTC (permalink / raw)
  To: qemu-devel

** Changed in: cinder
       Status: In Progress => Triaged

** Changed in: cinder
     Assignee: Tony Breeds (o-tony) => (unassigned)

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  Triaged
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (48 preceding siblings ...)
  2015-01-26 20:32 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Mike Perez
@ 2015-01-26 22:27 ` Tony Breeds
  2015-01-27 17:23 ` Tony Breeds
                   ` (7 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Tony Breeds @ 2015-01-26 22:27 UTC (permalink / raw)
  To: qemu-devel

** Changed in: cinder
     Assignee: (unassigned) => Tony Breeds (o-tony)

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  Triaged
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (49 preceding siblings ...)
  2015-01-26 22:27 ` Tony Breeds
@ 2015-01-27 17:23 ` Tony Breeds
  2015-01-28  0:41 ` [Qemu-devel] [Bug 1368815] Change abandoned on nova (master) OpenStack Infra
                   ` (6 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Tony Breeds @ 2015-01-27 17:23 UTC (permalink / raw)
  To: qemu-devel

** Changed in: nova
    Milestone: None => kilo-2

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  Triaged
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Change abandoned on nova (master)
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (50 preceding siblings ...)
  2015-01-27 17:23 ` Tony Breeds
@ 2015-01-28  0:41 ` OpenStack Infra
  2015-02-05 10:43 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Thierry Carrez
                   ` (5 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: OpenStack Infra @ 2015-01-28  0:41 UTC (permalink / raw)
  To: qemu-devel

Change abandoned by Tony Breeds (tony@bakeyournoodle.com) on branch: master
Review: https://review.openstack.org/123957
Reason: The main distros we care about have landed or are in progress.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  Triaged
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (51 preceding siblings ...)
  2015-01-28  0:41 ` [Qemu-devel] [Bug 1368815] Change abandoned on nova (master) OpenStack Infra
@ 2015-02-05 10:43 ` Thierry Carrez
  2015-02-12 15:09 ` Davanum Srinivas (DIMS)
                   ` (4 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Thierry Carrez @ 2015-02-05 10:43 UTC (permalink / raw)
  To: qemu-devel

** Changed in: nova
    Milestone: kilo-2 => kilo-3

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  Triaged
Status in OpenStack Compute (Nova):
  In Progress
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (52 preceding siblings ...)
  2015-02-05 10:43 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Thierry Carrez
@ 2015-02-12 15:09 ` Davanum Srinivas (DIMS)
  2015-02-24  9:25 ` Thierry Carrez
                   ` (3 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Davanum Srinivas (DIMS) @ 2015-02-12 15:09 UTC (permalink / raw)
  To: qemu-devel

Marking as Wont-Fix.

** Changed in: nova
       Status: In Progress => Won't Fix

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  Triaged
Status in OpenStack Compute (Nova):
  Won't Fix
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (53 preceding siblings ...)
  2015-02-12 15:09 ` Davanum Srinivas (DIMS)
@ 2015-02-24  9:25 ` Thierry Carrez
  2015-05-08 22:29 ` [Qemu-devel] [Bug 1368815] Change abandoned on cinder (master) OpenStack Infra
                   ` (2 subsequent siblings)
  57 siblings, 0 replies; 59+ messages in thread
From: Thierry Carrez @ 2015-02-24  9:25 UTC (permalink / raw)
  To: qemu-devel

** Changed in: nova
    Milestone: kilo-3 => None

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  Triaged
Status in OpenStack Compute (Nova):
  Won't Fix
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Change abandoned on cinder (master)
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (54 preceding siblings ...)
  2015-02-24  9:25 ` Thierry Carrez
@ 2015-05-08 22:29 ` OpenStack Infra
  2015-10-30  1:03 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Eric Harney
  2016-07-19  8:14 ` T. Huth
  57 siblings, 0 replies; 59+ messages in thread
From: OpenStack Infra @ 2015-05-08 22:29 UTC (permalink / raw)
  To: qemu-devel

Change abandoned by Mike Perez (thingee@gmail.com) on branch: master
Review: https://review.openstack.org/143575
Reason: No activity for over a month.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  Triaged
Status in OpenStack Compute (Nova):
  Won't Fix
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (55 preceding siblings ...)
  2015-05-08 22:29 ` [Qemu-devel] [Bug 1368815] Change abandoned on cinder (master) OpenStack Infra
@ 2015-10-30  1:03 ` Eric Harney
  2016-07-19  8:14 ` T. Huth
  57 siblings, 0 replies; 59+ messages in thread
From: Eric Harney @ 2015-10-30  1:03 UTC (permalink / raw)
  To: qemu-devel

Closing based on the assumption that a working qemu-img is available
now.

** Changed in: cinder
       Status: Triaged => Won't Fix

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  Won't Fix
Status in OpenStack Compute (nova):
  Won't Fix
Status in QEMU:
  In Progress
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

* [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
  2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
                   ` (56 preceding siblings ...)
  2015-10-30  1:03 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Eric Harney
@ 2016-07-19  8:14 ` T. Huth
  57 siblings, 0 replies; 59+ messages in thread
From: T. Huth @ 2016-07-19  8:14 UTC (permalink / raw)
  To: qemu-devel

According to comment #8 the fixes have been included in the upstream
QEMU repository, so setting the status to "Fix released" now.

** Changed in: qemu
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815

Title:
  qemu-img convert intermittently corrupts output images

Status in Cinder:
  Won't Fix
Status in OpenStack Compute (nova):
  Won't Fix
Status in QEMU:
  Fix Released
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Trusty:
  Fix Released
Status in qemu source package in Utopic:
  Fix Released
Status in qemu source package in Vivid:
  Fix Released

Bug description:
  ==========================================================
  Impact: occasional image corruption (any format on local filesystem)
  Test case: see the qemu-img command below
  Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree.  While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta.  I'd really like for a full qa-regression-test qemu testcase to be run against this package.
  ==========================================================

  -- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
  Ubuntu 14.04 using Ext4 filesystems.

  The command

    qemu-img convert -O raw inputimage.qcow2 outputimage.raw

  intermittently creates corrupted output images, when the input image
  is not yet fully synchronized to disk. While the issue has actually
  been discovered in operation of of OpenStack nova, it can be
  reproduced "easily" on command line using

    cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
  $DST_PATH && cksum $DST_PATH

  on filesystems exposing this behavior. (The difficult part of this
  exercise is to prepare a filesystem to reliably trigger this race. On
  my test machine some filesystems are affected while other aren't, and
  unfortunately I haven't found the relevant difference between them,
  yet. Possible it's timing issues completely out of userspace control
  ...)

  The root cause, however, is the same as in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html

  and it can be solved the same way as suggested in

    http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html

  In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change

      f.fm.fm_flags = 0;

  to

      f.fm.fm_flags = FIEMAP_FLAG_SYNC;

  As discussed in the thread mentioned above, retrieving a page cache
  coherent map of file extents is possible only after fsync on that
  file.

  See also

    https://bugs.launchpad.net/nova/+bug/1350766

  In that bug report filed against nova, fsync had been suggested to be
  performed by the framework invoking qemu-img. However, as the choice
  of fiemap -- implying this otherwise unneeded fsync of a temporary
  file  -- is not made by the caller but by qemu-img, I agree with the
  nova bug reviewer's objection to put it into nova. The fsync should
  instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
  specifically intended for that purpose.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1368815/+subscriptions

^ permalink raw reply	[flat|nested] 59+ messages in thread

end of thread, other threads:[~2016-07-19  8:21 UTC | newest]

Thread overview: 59+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
2014-09-12 15:08 ` [Qemu-devel] [Bug 1368815] " Michael Steffens
2014-09-12 15:13 ` Matt Riedemann
2014-09-14 21:04 ` Michael Steffens
2014-09-15 10:08 ` Michael Steffens
2014-09-15 10:30 ` Pádraig Brady
2014-09-15 12:52 ` Michael Steffens
2014-09-15 15:16 ` Matt Riedemann
2014-09-15 18:27 ` Serge Hallyn
2014-09-30  4:51 ` Tony Breeds
2014-10-22 23:21 ` Tony Breeds
2014-10-23 18:22 ` Serge Hallyn
2014-10-27  5:27 ` Launchpad Bug Tracker
2014-10-30  6:40 ` Launchpad Bug Tracker
2014-11-20 22:23 ` Serge Hallyn
2014-11-20 22:42 ` Tony Breeds
2014-11-21  4:17 ` Serge Hallyn
2014-11-21  4:47 ` Tony Breeds
2014-11-25 11:44 ` Kevin Wolf
2014-11-25 17:22 ` Chris J Arges
2014-11-25 17:24 ` [Qemu-devel] [Bug 1368815] Please test proposed package Chris J Arges
2014-12-01  8:23 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Michael Steffens
2014-12-01  8:32 ` Michael Steffens
2014-12-03  5:14 ` Mathew Hodson
2014-12-08 18:40 ` [Qemu-devel] [Bug 1368815] Update Released Brian Murray
2014-12-08 18:46 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Launchpad Bug Tracker
2014-12-10 16:38 ` Serge Hallyn
2014-12-10 18:06 ` Vladimir Kuklin
2014-12-10 18:32 ` Serge Hallyn
2014-12-10 20:34 ` Chris J Arges
2014-12-10 20:49 ` Chris J Arges
2014-12-10 20:50 ` Dmitry Mescheryakov
2014-12-10 21:54 ` Tony Breeds
2014-12-11  3:54 ` Chris J Arges
2014-12-11  3:57 ` Chris J Arges
2014-12-11  4:47 ` Tony Breeds
2014-12-11 17:53 ` Launchpad Bug Tracker
2014-12-11 20:11 ` Vish Ishaya
2014-12-12  1:56 ` Tony Breeds
2014-12-12  1:58 ` Tony Breeds
2014-12-12  3:28 ` OpenStack Infra
2014-12-15 16:26 ` Jay Bryant
2014-12-16 12:31 ` Alexei Sheplyakov
2014-12-16 12:33 ` Alexei Sheplyakov
2014-12-16 22:00 ` Tony Breeds
2014-12-17  6:44 ` Alexei Sheplyakov
2014-12-21 19:29 ` [Qemu-devel] [Bug 1368815] Change abandoned on cinder (master) OpenStack Infra
2014-12-23  0:44 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images OpenStack Infra
2015-01-26 15:27 ` [Qemu-devel] [Bug 1368815] Change abandoned on cinder (master) OpenStack Infra
2015-01-26 20:32 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Mike Perez
2015-01-26 22:27 ` Tony Breeds
2015-01-27 17:23 ` Tony Breeds
2015-01-28  0:41 ` [Qemu-devel] [Bug 1368815] Change abandoned on nova (master) OpenStack Infra
2015-02-05 10:43 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Thierry Carrez
2015-02-12 15:09 ` Davanum Srinivas (DIMS)
2015-02-24  9:25 ` Thierry Carrez
2015-05-08 22:29 ` [Qemu-devel] [Bug 1368815] Change abandoned on cinder (master) OpenStack Infra
2015-10-30  1:03 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Eric Harney
2016-07-19  8:14 ` T. Huth

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.