From: Yigal Korman <yigal@plexistor.com>
To: linux-fsdevel <linux-fsdevel@vger.kernel.org>,
kvm@vger.kernel.org, qemu-devel@nongnu.org,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Cc: Dan Williams <dan.j.williams@intel.com>
Subject: qemu-kvm hangs with DAX
Date: Mon, 24 Apr 2017 09:16:37 +0300 [thread overview]
Message-ID: <CACTTzNbu0NwXBw+Jj=8L2Nw+SBA3jLQ1-WDSa6-TgBuZwHj4ZQ@mail.gmail.com> (raw)
This is a re-post, I didn't send it to all relevant mailing lists before...
Original below.
Hi everyone,
I have an interesting issue with DAX and KVM - I'm trying to boot a VM
with its memory mapped to a DAX-mounted file (kernel 4.9).
The use case is a bit wacky but I'm trying to recreate something
similar to what clearlinux[1] described (although they don't use this
method anymore).
When mapping the memory to a regular ext4 file, the VM boots fine.
But when mapping to ext4+dax, the VM won't boot or perhaps boots
extremely slowly.
In both cases the FS is on a memory pmem device.
Here's a snippet of how I load things:
mkfs.ext4 /dev/pmem0
mount /dev/pmem0 /mnt
fallocate -l 512M /mnt/mem
qemu-system-x86_64 -nodefconfig -nodefaults \
-drive if=virtio,file=centos7.qcow2,index=0,media=disk \
--enable-kvm -serial telnet:localhost:4443,server,nowait \
-device sga -m 512 -smp 1,sockets=1,cores=1,threads=1 \
-object memory-backend-file,prealloc=yes,mem-path=/mnt/mem,share=on,size=512M,id=ram
\
-numa node,nodeid=0,cpus=0,memdev=ram \
-net nic,model=virtio,vlan=0 \
-net user,vlan=0,hostname=vm,hostfwd=tcp:127.0.0.1:8001-:22 \
-name test -monitor telnet:localhost:4444,server,nowait
I use a headless host so I usually connect to the VM with 'telnet
localhost 4443'.
The above works and the VM boots in seconds.
When adding '-o dax' to the mount command, I can catch the grub menu
during boot but it gets stuck.
Sometimes if I wait about 20 minutes, I see some kernel boot messages
appear, but no errors.
I've already tried something Dan Williams suggested - using 'dd'
instead of 'fallocate', but it didn't seem to help.
Also tried profiling the first 30s of qemu boot with 'perf stat' -
doesn't seem any clearer to me but here are the results:
for ext4 w/o DAX:
4804.688402 task-clock (msec) # 0.160 CPUs
utilized
22,389 context-switches # 0.005 M/sec
144 cpu-migrations # 0.030 K/sec
158,611 page-faults # 0.033 M/sec
7,537,184,564 cycles # 1.569 GHz
8,034,998,998 instructions # 1.07 insn per
cycle
1,612,266,593 branches # 335.561 M/sec
8,574,733 branch-misses # 0.53% of all
branches
for ext4 w/ DAX:
30001.643354 task-clock (msec) # 1.000 CPUs
utilized
584 context-switches # 0.019 K/sec
12 cpu-migrations # 0.000 K/sec
274,575 page-faults # 0.009 M/sec
2,131,506,685 cycles # 0.071 GHz
2,252,004,361 instructions # 1.06 insn per
cycle
439,086,052 branches # 14.635 M/sec
2,663,760 branch-misses # 0.61% of all
branches
Seems like w/o DAX, the boot will complete in seconds and the CPU will
remain idle and w/ DAX the CPU is working very hard and there much
more page-faults.
Any thoughts?
Regards,
Yigal
next reply other threads:[~2017-04-24 6:17 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-04-24 6:16 Yigal Korman [this message]
2017-04-26 13:43 ` qemu-kvm hangs with DAX Paolo Bonzini
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CACTTzNbu0NwXBw+Jj=8L2Nw+SBA3jLQ1-WDSa6-TgBuZwHj4ZQ@mail.gmail.com' \
--to=yigal@plexistor.com \
--cc=dan.j.williams@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).