From: Tony Breeds <tony@bakeyournoodle.com>
To: qemu-devel@nongnu.org
Subject: [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images
Date: Wed, 10 Dec 2014 21:54:12 -0000 [thread overview]
Message-ID: <20141210215412.19368.72760.malone@soybean.canonical.com> (raw)
In-Reply-To: 20140912145823.442.8812.malonedeb@gac.canonical.com
Hi Chris,
Markus' rework will not reintroduce this bug as it completely removes all fiemap code.
bug 129224 is a different issue, I'll comment on that bug.
You say: you encounter issues with upstream with ^extent and ext3
filesystems. Just to be clear: Are you saying that *this* bug is still
a problem for you?
if it's a different bug then I write it up and I'll take a look.
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1368815
Title:
qemu-img convert intermittently corrupts output images
Status in OpenStack Compute (Nova):
In Progress
Status in QEMU:
In Progress
Status in qemu package in Ubuntu:
Fix Released
Status in qemu source package in Trusty:
Fix Released
Status in qemu source package in Utopic:
Fix Committed
Status in qemu source package in Vivid:
Fix Released
Bug description:
==========================================================
Impact: occasional image corruption (any format on local filesystem)
Test case: see the qemu-img command below
Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree. While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta. I'd really like for a full qa-regression-test qemu testcase to be run against this package.
==========================================================
-- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on
Ubuntu 14.04 using Ext4 filesystems.
The command
qemu-img convert -O raw inputimage.qcow2 outputimage.raw
intermittently creates corrupted output images, when the input image
is not yet fully synchronized to disk. While the issue has actually
been discovered in operation of of OpenStack nova, it can be
reproduced "easily" on command line using
cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH
$DST_PATH && cksum $DST_PATH
on filesystems exposing this behavior. (The difficult part of this
exercise is to prepare a filesystem to reliably trigger this race. On
my test machine some filesystems are affected while other aren't, and
unfortunately I haven't found the relevant difference between them,
yet. Possible it's timing issues completely out of userspace control
...)
The root cause, however, is the same as in
http://lists.gnu.org/archive/html/coreutils/2011-04/msg00069.html
and it can be solved the same way as suggested in
http://lists.gnu.org/archive/html/coreutils/2011-04/msg00102.html
In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change
f.fm.fm_flags = 0;
to
f.fm.fm_flags = FIEMAP_FLAG_SYNC;
As discussed in the thread mentioned above, retrieving a page cache
coherent map of file extents is possible only after fsync on that
file.
See also
https://bugs.launchpad.net/nova/+bug/1350766
In that bug report filed against nova, fsync had been suggested to be
performed by the framework invoking qemu-img. However, as the choice
of fiemap -- implying this otherwise unneeded fsync of a temporary
file -- is not made by the caller but by qemu-img, I agree with the
nova bug reviewer's objection to put it into nova. The fsync should
instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC,
specifically intended for that purpose.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1368815/+subscriptions
next prev parent reply other threads:[~2014-12-10 22:01 UTC|newest]
Thread overview: 59+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-09-12 14:58 [Qemu-devel] [Bug 1368815] [NEW] qemu-img convert intermittently corrupts output images Michael Steffens
2014-09-12 15:08 ` [Qemu-devel] [Bug 1368815] " Michael Steffens
2014-09-12 15:13 ` Matt Riedemann
2014-09-14 21:04 ` Michael Steffens
2014-09-15 10:08 ` Michael Steffens
2014-09-15 10:30 ` Pádraig Brady
2014-09-15 12:52 ` Michael Steffens
2014-09-15 15:16 ` Matt Riedemann
2014-09-15 18:27 ` Serge Hallyn
2014-09-30 4:51 ` Tony Breeds
2014-10-22 23:21 ` Tony Breeds
2014-10-23 18:22 ` Serge Hallyn
2014-10-27 5:27 ` Launchpad Bug Tracker
2014-10-30 6:40 ` Launchpad Bug Tracker
2014-11-20 22:23 ` Serge Hallyn
2014-11-20 22:42 ` Tony Breeds
2014-11-21 4:17 ` Serge Hallyn
2014-11-21 4:47 ` Tony Breeds
2014-11-25 11:44 ` Kevin Wolf
2014-11-25 17:22 ` Chris J Arges
2014-11-25 17:24 ` [Qemu-devel] [Bug 1368815] Please test proposed package Chris J Arges
2014-12-01 8:23 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Michael Steffens
2014-12-01 8:32 ` Michael Steffens
2014-12-03 5:14 ` Mathew Hodson
2014-12-08 18:40 ` [Qemu-devel] [Bug 1368815] Update Released Brian Murray
2014-12-08 18:46 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Launchpad Bug Tracker
2014-12-10 16:38 ` Serge Hallyn
2014-12-10 18:06 ` Vladimir Kuklin
2014-12-10 18:32 ` Serge Hallyn
2014-12-10 20:34 ` Chris J Arges
2014-12-10 20:49 ` Chris J Arges
2014-12-10 20:50 ` Dmitry Mescheryakov
2014-12-10 21:54 ` Tony Breeds [this message]
2014-12-11 3:54 ` Chris J Arges
2014-12-11 3:57 ` Chris J Arges
2014-12-11 4:47 ` Tony Breeds
2014-12-11 17:53 ` Launchpad Bug Tracker
2014-12-11 20:11 ` Vish Ishaya
2014-12-12 1:56 ` Tony Breeds
2014-12-12 1:58 ` Tony Breeds
2014-12-12 3:28 ` OpenStack Infra
2014-12-15 16:26 ` Jay Bryant
2014-12-16 12:31 ` Alexei Sheplyakov
2014-12-16 12:33 ` Alexei Sheplyakov
2014-12-16 22:00 ` Tony Breeds
2014-12-17 6:44 ` Alexei Sheplyakov
2014-12-21 19:29 ` [Qemu-devel] [Bug 1368815] Change abandoned on cinder (master) OpenStack Infra
2014-12-23 0:44 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images OpenStack Infra
2015-01-26 15:27 ` [Qemu-devel] [Bug 1368815] Change abandoned on cinder (master) OpenStack Infra
2015-01-26 20:32 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Mike Perez
2015-01-26 22:27 ` Tony Breeds
2015-01-27 17:23 ` Tony Breeds
2015-01-28 0:41 ` [Qemu-devel] [Bug 1368815] Change abandoned on nova (master) OpenStack Infra
2015-02-05 10:43 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Thierry Carrez
2015-02-12 15:09 ` Davanum Srinivas (DIMS)
2015-02-24 9:25 ` Thierry Carrez
2015-05-08 22:29 ` [Qemu-devel] [Bug 1368815] Change abandoned on cinder (master) OpenStack Infra
2015-10-30 1:03 ` [Qemu-devel] [Bug 1368815] Re: qemu-img convert intermittently corrupts output images Eric Harney
2016-07-19 8:14 ` T. Huth
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20141210215412.19368.72760.malone@soybean.canonical.com \
--to=tony@bakeyournoodle.com \
--cc=1368815@bugs.launchpad.net \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.