From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-yw0-f172.google.com ([209.85.161.172]:34767 "EHLO mail-yw0-f172.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752806AbdICNjv (ORCPT ); Sun, 3 Sep 2017 09:39:51 -0400 Received: by mail-yw0-f172.google.com with SMTP id t188so16291533ywb.1 for ; Sun, 03 Sep 2017 06:39:51 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <20170901200658.txwbtntrtqwqdvcq@destiny> References: <20170830152326.vil3fhsrecp2ccql@destiny> <20170830185512.7q5mnh5ja6o4mpds@destiny> <20170831134320.lnyu4jibsm3amuk7@destiny> <20170831205403.2tene34ccvw55yo7@destiny> <20170901200658.txwbtntrtqwqdvcq@destiny> From: Amir Goldstein Date: Sun, 3 Sep 2017 16:39:49 +0300 Message-ID: Subject: Re: [PATCH v2 00/14] Crash consistency xfstest using dm-log-writes Content-Type: text/plain; charset="UTF-8" Sender: fstests-owner@vger.kernel.org To: Josef Bacik Cc: fstests , Theodore Tso , Eryu Guan List-ID: On Fri, Sep 1, 2017 at 11:07 PM, Josef Bacik wrote: > On Fri, Sep 01, 2017 at 09:52:18AM +0300, Amir Goldstein wrote: >> [CC list, Ted] >> >> On Thu, Aug 31, 2017 at 11:54 PM, Josef Bacik wrote: >> > On Thu, Aug 31, 2017 at 05:02:46PM +0300, Amir Goldstein wrote: >> >> On Thu, Aug 31, 2017 at 4:43 PM, Josef Bacik wrote: >> >> > On Thu, Aug 31, 2017 at 03:48:44PM +0300, Amir Goldstein wrote: >> >> >> >> >> >> Josef, >> >> >> >> >> >> I am at lost with these log corruptions. >> >> >> I see log entry bios submitted and log_end_io report success, >> >> >> but then in the log I see old data on disk where that entry should be. >> >> >> This happens quite randomly and I assume it also happens on >> >> >> logged data, because tests sometime fail on checksum on ext4. >> >> >> >> >> >> Mean while I added some more log entry sanity checks and debug >> >> >> prints to replay-log to debug the corruption: >> >> >> https://github.com/amir73il/xfstests/commit/bb946deb0dc285867be394613ddb19ce281392cc >> >> >> >> >> >> This only happens to me when running in kvm, so maybe something >> >> >> with the virtio devices is fishy. >> >> >> >> >> >> Anyway, I ran out of time to work on this for now, so if you have >> >> >> any ideas and/or time to test this issue, let me know. >> >> >> >> >> > >> ... >> >> >> > >> > Alright I tested it and it's working fine for me. I'm creating three lv's and >> > then doing >> > >> > -drive file=/dev/mapper/whatever,format=raw,cache=none,if=virtio,aio=native >> > >> > And I get /dev/vd[bcd] which I use for my test/scratch/log dev and it works out >> > fine. What is your -drive option line and I'll duplicate what you are doing. >> > Thanks, >> > >> >> I am using Ted's kvm-xfstests, so this is the qemu command line: >> https://github.com/tytso/xfstests-bld/blob/master/kvm-xfstests/kvm-xfstests#L104 >> >> The only difference in -drive command is no aio=native. >> BINGO! when I add aio-native there are no more log corruptions :) >> Please try to use aio=threads to see if you also get log corruptions. >> >> Thing is we cannot change kvm-xfstests to always use aio=native because >> it is not recommended for sparse images: >> https://access.redhat.com/articles/41313 >> I will try to work something out so that kvm-xfstest will use aio=native >> when using the recommended (by not default) LV setup. >> >> However, why would aio=threads cause log corruption? >> Does it indicate a bug in kvm-qemu or in dm-log-writes?? > > So I've been running this in a loop all day with aio=threads and it's not > blowing up. This is my qemu version > > QEMU emulator version 2.9.0(qemu-2.9.0-1.fb1) > > Maybe it has to do with the version of qemu? Thanks, > Maybe. I am running QEMU 2.5.0