Re: DAX can not work on virtual nvdimm device

From: Jan Kara <jack@suse.cz>
To: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Xiao Guangrong <guangrong.xiao@linux.intel.com>,
	linux-ext4@vger.kernel.org, Jan Kara <jack@suse.cz>,
	"Theodore Ts'o" <tytso@mit.edu>,
	Dan Williams <dan.j.williams@intel.com>,
	Yumei Huang <yuhuang@redhat.com>, KVM <kvm@vger.kernel.org>,
	"linux-nvdimm@lists.01.org" <linux-nvdimm@ml01.01.org>,
	"qemu-devel@nongnu.org" <qemu-devel@nongnu.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Linux ACPI <linux-acpi@vger.kernel.org>,
	Stefan Hajnoczi <stefanha@redhat.com>
Subject: Re: DAX can not work on virtual nvdimm device
Date: Tue, 6 Sep 2016 17:06:20 +0200	[thread overview]
Message-ID: <20160906150620.GJ28922@quack2.suse.cz> (raw)
In-Reply-To: <20160902025738.GA26108@linux.intel.com>

On Thu 01-09-16 20:57:38, Ross Zwisler wrote:
> On Wed, Aug 31, 2016 at 04:44:47PM +0800, Xiao Guangrong wrote:
> > On 08/31/2016 01:09 AM, Dan Williams wrote:
> > > 
> > > Can you post your exact reproduction steps?  This test is not failing for me.
> > > 
> > 
> > Sure.
> > 
> > 1. make the guest kernel based on your tree, the top commit is
> >    10d7902fa0e82b (dax: unmap/truncate on device shutdown) and
> >    the config file can be found in this thread.
> > 
> > 2. add guest kernel command line: memmap=6G!10G
> > 
> > 3: start the guest:
> >    x86_64-softmmu/qemu-system-x86_64 -machine pc,nvdimm --enable-kvm \
> >    -smp 16 -m 32G,maxmem=100G,slots=100 /other/VMs/centos6.img -monitor stdio
> > 
> > 4: in guest:
> >    mkfs.ext4 /dev/pmem0
> >    mount -o dax /dev/pmem0  /mnt/pmem/
> >    echo > /mnt/pmem/xxx
> >    ./mmap /mnt/pmem/xxx
> >    ./read /mnt/pmem/xxx
> > 
> >   The source code of mmap and read has been attached in this mail.
> > 
> >   Hopefully, you can detect the error triggered by read test.
> > 
> > Thanks!
> 
> Okay, I think I've isolated this issue.  Xiao's VM was an old CentOS 6 system,
> and for some reason ext4+DAX with the old tools found in that VM fails.  I was
> able to reproduce this failure with a freshly installed CentOS 6.8 VM.
> 
> You can see the failure with his tests, or perhaps more easily with this
> series of commands:
> 
>   # mkfs.ext4 /dev/pmem0
>   # mount -o dax /dev/pmem0  /mnt/pmem/
>   # touch /mnt/pmem/x
>   # md5sum /mnt/pmem/x
>   md5sum: /mnt/pmem/x: Bad address
> 
> This sequence of commands works fine in the old CentOS 6 system if you use XFS
> instead of ext4, and it works fine with both ext4 and XFS in CentOS 7 and
> with recent versions of Fedora.
> 
> I've added the ext4 folks to this mail in case they care, but my guess is that
> the tools in CentOS 6 are so old that it's not worth worrying about.  For
> reference, the kernel in CentOS 6 is based on 2.6.32.  :)  DAX was introduced
> in v4.0.

Hum, can you post 'dumpe2fs -h /dev/pmem0' output from that system when the
md5sum fails? Because the only idea I have is that mkfs.ext4 in CentOS 6
creates the filesystem with a different set of features than more recent
e2fsprogs and so we hit some untested path...

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR