[Bug 42755] New: KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[Bug 42755] New: KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755

           Summary: KVM is being extremely slow on AMD Athlon64 4000+ Dual
                    Core 2.1GHz Brisbane
           Product: Virtualization
           Version: unspecified
    Kernel Version: 3.2.2
          Platform: All
        OS/Version: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: kvm
        AssignedTo: virtualization_kvm@kernel-bugs.osdl.org
        ReportedBy: sandikata@yandex.ru
        Regression: No


Hello.

Kvm seems to be broken on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane. A
kernel build takes over
2 hours, compared to 15 minutes on the host. I'm running with virtio
for storage, and this does not seem to be related to IO. A simple test
with dd also demonstrates that the problem is cpu-ralated:

Host test:
dd if=/dev/zero of=/dev/null bs=1k count=8M
8388608+0 записей считано (read)
8388608+0 записей написано (write)
 скопировано (copyed) 8589934592 байта (8,6 GB), 7,04456 c, 1,2 GB/c

This also does not seem to be a cpu bug, as both VirtualBox and VMware
work fine.

Vmware/Virtualbox test:
dd if=/dev/zero of=/dev/null bs=1k count=8M
8388608+0 записей считано (read)
8388608+0 записей написано (write)
 скопировано (copyed) 8589934592 байта (8,6 GB), 5,02745 c, 1,7 GB/c

I've tried different kernels on host, between 2.6.22 and current git,
with the same results. The guest is running Gentoo Hardened kernel 3.2.2
, but I could try with a an other distro and a vanilla kernel
if required.

My kvm version is kvm 1.0

Help would be much appreciated.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #1 from Rosen <sandikata@yandex.ru>  2012-02-10 08:06:25 ---
in kvm guest this speed are about 2MB/s and is slow like a hell. With it
booting for about 15 minutes.

Same guest machine boot for about 4-7 seconds on virtualbox/vmware

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #2 from Michael Tokarev <mjt@tls.msk.ru>  2012-02-10 09:18:35 ---
This is very unlikely to do with the kernel.  Qemu (which you're using to run
your guest -- kvm is just in-kernel module) will emulate whole machine
including CPU in case it can't access /dev/kvm or if you requested to not use
it in the first place.  If the problem is due to lack of access to /dev/kvm,
qemu tells you about that on stderr.

So far, you haven't provided any of:

 - qemu version
 - qemu command line
 - messages generated by qemu

And for this issue, please also provide output of

 info kvm

from qemu monitor.  If kvm is enabled, it should tell you just that --
"enabled". If not, it'll tell you "disabled" and this means that your guest is
running in full emulation mode.

Without that, this bugreport is not very useful.  Especially having in mind
that this setup works for lots of other people just fine.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #3 from Rosen <sandikata@yandex.ru>  2012-02-10 09:36:35 ---
cmdline:

sudo kvm -enable-kvm -cpu 'host' -smp '2,cores=2,threads=2,sockets=1' -m 512M
-vga vmware -usbdevice tablet -soundhw es1370 -drive
file=test-amd64.img,if=virtio,cache=none -netdev
type=tap,script=/etc/qemu-ifup,id=net0 -device virtio-net-pci,netdev=net0
-balloon virtio -daemonize -name 'Gentoo Hardened 64 20120207'

version:

qemu -version
QEMU emulator version 1.0 (qemu-kvm-1.0), Copyright (c) 2003-2008 Fabrice
Bellard

info:

(qemu) info kvm
kvm support: enabled
(qemu)

Host dmesg kvm only :

roko__@ArchXFCE64(11:34:21-12-02-10)( ~ pts/1 )# dmesg | grep kvm
[12347.567390] kvm: Nested Virtualization enabled
[12374.734909] kvm: unreliable cycle conversion on adjustable rate TSC
[12451.916934] kvm: 13119: cpu0 unhandled rdmsr: 0xc001100d
[12451.916959] kvm: 13119: cpu0 unhandled rdmsr: 0xc0010112
[12452.286858] kvm: 13119: cpu0 unhandled rdmsr: 0xc0010001
[16345.719346] kvm: 23604: cpu0 unhandled rdmsr: 0xc0010112
[16346.104688] kvm: 23604: cpu0 unhandled rdmsr: 0xc0010001
[23694.763207] kvm: 4941: cpu0 unhandled rdmsr: 0xc0010112
[23694.969082] kvm: 4941: cpu0 unhandled rdmsr: 0xc0010001
[23830.158470] kvm: 4941: cpu0 unhandled rdmsr: 0xc0010112
[23830.360768] kvm: 4941: cpu0 unhandled rdmsr: 0xc0010001
[23882.579070] kvm: 5419: cpu0 unhandled rdmsr: 0xc0010112
[23882.779127] kvm: 5419: cpu0 unhandled rdmsr: 0xc0010001
[24081.106328] kvm: 5770: cpu0 unhandled rdmsr: 0xc0010112
[24081.439772] kvm: 5770: cpu0 unhandled rdmsr: 0xc0010001
[24478.792921] kvm: 7706: cpu0 unhandled rdmsr: 0xc0010112
[24478.986782] kvm: 7706: cpu0 unhandled rdmsr: 0xc0010001
[24931.920782] kvm: SMP vm created on host with unstable TSC; guest TSC will
not be reliable
[24942.147749] kvm: 8966: cpu0 unhandled rdmsr: 0xc0010112
[24942.301014] kvm: 8966: cpu0 unhandled rdmsr: 0xc0010001
[27046.175073] kvm: 15606: cpu0 unhandled rdmsr: 0xc0010112
[27046.332410] kvm: 15606: cpu0 unhandled rdmsr: 0xc0010001
[28459.859245] kvm: 23994: cpu0 unhandled rdmsr: 0xc0010112
[28460.058113] kvm: 23994: cpu0 unhandled rdmsr: 0xc0010001
[30892.864676] kvm: 6341: cpu0 unhandled rdmsr: 0xc0010001
[31438.995057] kvm: 11147: cpu0 unhandled rdmsr: 0xc0010112
[31439.214065] kvm: 11147: cpu0 unhandled rdmsr: 0xc0010001
[32561.185944] kvm: 20349: cpu0 unhandled rdmsr: 0xc0010112
[32561.448391] kvm: 20349: cpu0 unhandled rdmsr: 0xc0010001
[33633.457536] kvm: 28893: cpu0 unhandled rdmsr: 0xc0010112
[33633.801818] kvm: 28893: cpu0 unhandled rdmsr: 0xc0010001
[33879.382766] kvm: 30299: cpu0 unhandled rdmsr: 0xc0010112
[33879.577796] kvm: 30299: cpu0 unhandled rdmsr: 0xc0010001
[34654.862032] kvm: 32668: cpu0 unhandled rdmsr: 0xc0010112
[34655.234667] kvm: 32668: cpu0 unhandled rdmsr: 0xc0010001
[36724.031803] kvm: 32668: cpu0 unhandled rdmsr: 0xc0010112
[36724.147844] kvm: 32668: cpu0 unhandled rdmsr: 0xc0010001
[36836.657695] kvm: 32668: cpu0 unhandled rdmsr: 0xc0010112
[36836.896359] kvm: 32668: cpu0 unhandled rdmsr: 0xc0010001
[36891.644691] kvm: 8665: cpu0 unhandled rdmsr: 0xc0010112
[36891.778808] kvm: 8665: cpu0 unhandled rdmsr: 0xc0010001
[37065.796742] kvm: 9545: cpu0 unhandled rdmsr: 0xc0010112
[37065.951394] kvm: 9545: cpu0 unhandled rdmsr: 0xc0010001
[37198.255255] kvm: 10184: cpu0 unhandled rdmsr: 0xc0010112
[37198.385243] kvm: 10184: cpu0 unhandled rdmsr: 0xc0010001
[37531.276399] kvm: 11308: cpu0 unhandled rdmsr: 0xc0010112
[37531.404946] kvm: 11308: cpu0 unhandled rdmsr: 0xc0010001
[37633.200338] kvm: 12026: cpu0 unhandled rdmsr: 0xc0010112
[37633.335279] kvm: 12026: cpu0 unhandled rdmsr: 0xc0010001
[37974.339382] kvm: 13679: cpu0 unhandled rdmsr: 0xc0010112
[37974.463314] kvm: 13679: cpu0 unhandled rdmsr: 0xc0010001
[37998.836470] kvm: 14017: cpu0 unhandled rdmsr: 0xc0010112
[37998.952253] kvm: 14017: cpu0 unhandled rdmsr: 0xc0010001
[38194.260773] kvm: 14756: cpu0 unhandled rdmsr: 0xc0010112
[38194.400660] kvm: 14756: cpu0 unhandled rdmsr: 0xc0010001
[40609.026332] kvm: 23045: cpu0 unhandled rdmsr: 0xc0010001
[41580.924327] kvm: 28276: cpu0 unhandled rdmsr: 0xc0010112
[41581.040324] kvm: 28276: cpu0 unhandled rdmsr: 0xc0010001
[89588.419228] kvm: 5045: cpu0 unhandled rdmsr: 0xc0010112
[89588.558034] kvm: 5045: cpu0 unhandled rdmsr: 0xc0010001
[126250.670121] kvm: 27098: cpu0 unhandled rdmsr: 0xc0010112
[126250.800737] kvm: 27098: cpu0 unhandled rdmsr: 0xc0010001
[128187.690739] kvm: 27098: cpu0 unhandled rdmsr: 0xc0010112
[128187.821740] kvm: 27098: cpu0 unhandled rdmsr: 0xc0010001
[128452.392446] kvm: 1746: cpu0 unhandled rdmsr: 0xc0010112
[128452.524364] kvm: 1746: cpu0 unhandled rdmsr: 0xc0010001
[128642.028625] kvm: 1746: cpu0 unhandled rdmsr: 0xc0010112
[128642.165803] kvm: 1746: cpu0 unhandled rdmsr: 0xc0010001
[129900.176718] kvm: 6916: cpu0 unhandled rdmsr: 0xc0010112
[129900.301721] kvm: 6916: cpu0 unhandled rdmsr: 0xc0010001
[130077.225227] kvm: 8083: cpu0 unhandled rdmsr: 0xc0010112
[130077.355747] kvm: 8083: cpu0 unhandled rdmsr: 0xc0010001
[130251.383970] kvm: 8899: cpu0 unhandled rdmsr: 0xc001100d
[130251.383995] kvm: 8899: cpu0 unhandled rdmsr: 0xc0010112
[130251.526014] kvm: 8899: cpu0 unhandled rdmsr: 0xc0010001
[130251.552576] kvm: 8899: cpu1 unhandled rdmsr: 0xc001100d
[172942.439354] kvm: 6960: cpu0 unhandled rdmsr: 0xc001100d
[172942.439379] kvm: 6960: cpu0 unhandled rdmsr: 0xc0010112
[172942.559674] kvm: 6960: cpu0 unhandled rdmsr: 0xc0010001
[172942.584961] kvm: 6960: cpu1 unhandled rdmsr: 0xc001100d
[173128.821234] kvm: 7982: cpu0 unhandled rdmsr: 0xc0010112
[173128.952489] kvm: 7982: cpu0 unhandled rdmsr: 0xc0010001
[202490.969575] kvm: 26305: cpu0 unhandled rdmsr: 0xc0010112
[202491.091029] kvm: 26305: cpu0 unhandled rdmsr: 0xc0010001
[203118.612164] kvm: 28617: cpu0 unhandled rdmsr: 0xc0010112
[203118.753413] kvm: 28617: cpu0 unhandled rdmsr: 0xc0010001
[203530.933754] kvm: 30072: cpu0 unhandled rdmsr: 0xc0010112
[203531.058048] kvm: 30072: cpu0 unhandled rdmsr: 0xc0010001
[203966.434833] kvm: 32198: cpu0 unhandled rdmsr: 0xc0010112
[203966.561245] kvm: 32198: cpu0 unhandled rdmsr: 0xc0010001
[204582.016825] kvm: 32198: cpu0 unhandled rdmsr: 0xc0010112
[204582.152043] kvm: 32198: cpu0 unhandled rdmsr: 0xc0010001
[204863.395916] kvm: 3269: cpu0 unhandled rdmsr: 0xc0010112
[204863.530150] kvm: 3269: cpu0 unhandled rdmsr: 0xc0010001
[207390.936205] kvm: 3269: cpu0 unhandled rdmsr: 0xc0010112
[207391.077543] kvm: 3269: cpu0 unhandled rdmsr: 0xc0010001
[255644.169559] kvm: 3269: cpu0 unhandled rdmsr: 0xc0010112
[255644.310790] kvm: 3269: cpu0 unhandled rdmsr: 0xc0010001
[256444.141527] kvm: 3269: cpu0 unhandled rdmsr: 0xc0010112
[256444.270798] kvm: 3269: cpu0 unhandled rdmsr: 0xc0010001
[256684.553828] kvm: 30582: cpu0 unhandled rdmsr: 0xc0010112
[256684.678749] kvm: 30582: cpu0 unhandled rdmsr: 0xc0010001
[259699.726226] kvm: 30582: cpu0 unhandled rdmsr: 0xc0010112
[259699.934852] kvm: 30582: cpu0 unhandled rdmsr: 0xc0010001
roko__@ArchXFCE64(11:34:28-12-02-10)( ~ pts/1 )#

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #4 from Rosen <sandikata@yandex.ru>  2012-02-11 10:50:46 ---
Still no response there ..

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755


Gleb <gleb@redhat.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |gleb@redhat.com




--- Comment #5 from Gleb <gleb@redhat.com>  2012-02-12 08:27:12 ---
Try without -vga vmware. If still slow provide kvm trace please
(http://www.linux-kvm.org/page/Tracing)

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #6 from Rosen <sandikata@yandex.ru>  2012-02-12 09:52:13 ---
@Gleb

i tryed many variants

cirrus std vmware vga-s no diferences.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #7 from Rosen <sandikata@yandex.ru>  2012-02-12 10:10:14 ---
@Gleb

trace-cmd cannot install

*********************************
** No docbook.xsl is installed **
** Can't make man pages        **
*********************************

extra/docbook-xsl 1.76.1-2 [installed]
    XML stylesheets for Docbook-xml transformations

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #8 from Gleb <gleb@redhat.com>  2012-02-12 10:13:24 ---
(In reply to comment #7)
> @Gleb
> 
> trace-cmd cannot install
> 
> *********************************
> ** No docbook.xsl is installed **
> ** Can't make man pages        **
> *********************************
> 
> extra/docbook-xsl 1.76.1-2 [installed]
>     XML stylesheets for Docbook-xml transformations

Please resolve the problem and try again. Which host distribution are you
using?

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #9 from Rosen <sandikata@yandex.ru>  2012-02-12 10:16:07 ---
Hm ..

trace-cmd record -b 20000 -e kvm
trace-cmd: No such file or directory
  opening '/sys/kernel/debug/tracing/tracing_on'

this directory doesn't exist.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #10 from Rosen <sandikata@yandex.ru>  2012-02-12 10:16:44 ---
(In reply to comment #8)
> (In reply to comment #7)
> > @Gleb
> > 
> > trace-cmd cannot install
> > 
> > *********************************
> > ** No docbook.xsl is installed **
> > ** Can't make man pages        **
> > *********************************
> > 
> > extra/docbook-xsl 1.76.1-2 [installed]
> >     XML stylesheets for Docbook-xml transformations
> 
> Please resolve the problem and try again. Which host distribution are you
> using?

Host distribution is archlinux.
I write answer to trace-cmd, missing directory.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #11 from Gleb <gleb@redhat.com>  2012-02-12 10:18:48 ---
(In reply to comment #9)
> Hm ..
> 
> trace-cmd record -b 20000 -e kvm
> trace-cmd: No such file or directory
>   opening '/sys/kernel/debug/tracing/tracing_on'
> 
> this directory doesn't exist.

mount -t debugfs none /sys/kernel/debug/

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #12 from Rosen <sandikata@yandex.ru>  2012-02-12 10:21:36 ---
(In reply to comment #11)
> (In reply to comment #9)
> > Hm ..
> > 
> > trace-cmd record -b 20000 -e kvm
> > trace-cmd: No such file or directory
> >   opening '/sys/kernel/debug/tracing/tracing_on'
> > 
> > this directory doesn't exist.
> 
> mount -t debugfs none /sys/kernel/debug/

mount -t debugfs none /sys/kernel/debug/
mount: none already mounted or /sys/kernel/debug/ busy
mount: according to mtab, nodev is already mounted on /sys/kernel/debug

Allready mounted. Bu this directory doesn't exist, i have those dirs:

/sys/kernel/debug 
acpi  bdi  dri  hid  ieee80211  kvm  mce  suspend_stats  usb  virtio-ports 
wakeup_sources  x86

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #13 from Gleb <gleb@redhat.com>  2012-02-12 10:30:30 ---
(In reply to comment #12)
> Allready mounted. Bu this directory doesn't exist, i have those dirs:
> 
> /sys/kernel/debug 
> acpi  bdi  dri  hid  ieee80211  kvm  mce  suspend_stats  usb  virtio-ports 
> wakeup_sources  x86

Is this ancient kernel? Or have you disabled tracing during kernel compile? If
later you will have to rebuild the kernel with tracing enabled. This is the
only way to see what is happening.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #14 from Rosen <sandikata@yandex.ru>  2012-02-12 10:40:56 ---
(In reply to comment #13)
> (In reply to comment #12)
> > Allready mounted. Bu this directory doesn't exist, i have those dirs:
> > 
> > /sys/kernel/debug 
> > acpi  bdi  dri  hid  ieee80211  kvm  mce  suspend_stats  usb  virtio-ports 
> > wakeup_sources  x86
> 
> Is this ancient kernel? Or have you disabled tracing during kernel compile? If
> later you will have to rebuild the kernel with tracing enabled. This is the
> only way to see what is happening.

Its my kernel not default ARCH kernel.

zgrep TRACE /proc/config.gz 
CONFIG_STACKTRACE_SUPPORT=y
# CONFIG_RCU_TRACE is not set
# CONFIG_TREE_RCU_TRACE is not set
CONFIG_HAVE_ARCH_TRACEHOOK=y
# CONFIG_TRACE_SINK is not set
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
CONFIG_USER_STACKTRACE_SUPPORT=y
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
# CONFIG_FTRACE is not set
CONFIG_HAVE_MMIOTRACE_SUPPORT=y

i'm not sure which option need to be enabled?!

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #15 from Rosen <sandikata@yandex.ru>  2012-02-12 10:41:31 ---
(In reply to comment #14)
> (In reply to comment #13)
> > (In reply to comment #12)
> > > Allready mounted. Bu this directory doesn't exist, i have those dirs:
> > > 
> > > /sys/kernel/debug 
> > > acpi  bdi  dri  hid  ieee80211  kvm  mce  suspend_stats  usb  virtio-ports 
> > > wakeup_sources  x86
> > 
> > Is this ancient kernel? Or have you disabled tracing during kernel compile? If
> > later you will have to rebuild the kernel with tracing enabled. This is the
> > only way to see what is happening.
> 
> Its my kernel not default ARCH kernel.
> 
> zgrep TRACE /proc/config.gz 
> CONFIG_STACKTRACE_SUPPORT=y
> # CONFIG_RCU_TRACE is not set
> # CONFIG_TREE_RCU_TRACE is not set
> CONFIG_HAVE_ARCH_TRACEHOOK=y
> # CONFIG_TRACE_SINK is not set
> CONFIG_TRACE_IRQFLAGS_SUPPORT=y
> CONFIG_USER_STACKTRACE_SUPPORT=y
> CONFIG_HAVE_FUNCTION_TRACER=y
> CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
> CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST=y
> CONFIG_HAVE_DYNAMIC_FTRACE=y
> CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
> CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
> # CONFIG_FTRACE is not set
> CONFIG_HAVE_MMIOTRACE_SUPPORT=y
> 

zgrep TRACING /proc/config.gz
CONFIG_TRACING_SUPPORT=y


> i'm not sure which option need to be enabled?!

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #16 from Gleb <gleb@redhat.com>  2012-02-12 10:49:04 ---
(In reply to comment #15)
>
> zgrep TRACING /proc/config.gz
> CONFIG_TRACING_SUPPORT=y
> 
You need CONFIG_TRACING/CONFIG_FTRACE and others.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #17 from Rosen <sandikata@yandex.ru>  2012-02-12 14:49:41 ---
hello again!

roko__@ArchXFCE64(16:47:28-12-02-12)( /mnt/home/roko__-backup/kvm pts/6 )#
trace-cmd report
version = 6
  cound not load plugin '/usr/local/share/trace-cmd/plugins/plugin_python.so'
/usr/local/share/trace-cmd/plugins/plugin_python.so: undefined symbol:
PyString_FromString

File possibly truncated. Need at least 1654784, but file size is 1654292.
trace-cmd: Invalid argument
  failed to init data

And the trace.dat file is in attachments.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #18 from Rosen <sandikata@yandex.ru>  2012-02-12 14:55:25 ---
Created an attachment (id=72360)
 --> (https://bugzilla.kernel.org/attachment.cgi?id=72360)
sudo trace-cmd record -b 20000 -e kvm

trace.dat from 

roko__@ArchXFCE64(16:46:25-12-02-12)( /mnt/home/roko__-backup/kvm pts/6 )# sudo
trace-cmd record -b 20000 -e kvm
/sys/kernel/debug/tracing/events/kvm/filter
/sys/kernel/debug/tracing/events/*/kvm/filter
Hit Ctrl^C to stop recording
^CKernel buffer statistics:
  Note: "entries" are the entries left in the kernel ring buffer and are not
        recorded in the trace data. They should all be zero.

CPU: 0
entries: 0
overrun: 0
commit overrun: 0
bytes: 0
oldest event ts: 18446612134.805802
now ts:  3383.418379

CPU: 1
entries: 0
overrun: 0
commit overrun: 0
bytes: 0
oldest event ts: 142390.782329
now ts:  3383.418586

CPU0 data recorded at offset=0x194000
    0 bytes in size
CPU1 data recorded at offset=0x194000
    0 bytes in size
roko__@ArchXFCE64(16:47:23-12-02-12)( /mnt/home/roko__-backup/kvm pts/6 )#

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #19 from Gleb <gleb@redhat.com>  2012-02-12 15:09:04 ---
(In reply to comment #17)
> hello again!
> 
> roko__@ArchXFCE64(16:47:28-12-02-12)( /mnt/home/roko__-backup/kvm pts/6 )#
> trace-cmd report
> version = 6
>   cound not load plugin '/usr/local/share/trace-cmd/plugins/plugin_python.so'
> /usr/local/share/trace-cmd/plugins/plugin_python.so: undefined symbol:
> PyString_FromString
> 
> File possibly truncated. Need at least 1654784, but file size is 1654292.
> trace-cmd: Invalid argument
>   failed to init data
> 
> And the trace.dat file is in attachments.

It is corrupted for me too. Have you compiled trace-cmd by yourself? If yes
have you ran "make install" before using it?

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #20 from Rosen <sandikata@yandex.ru>  2012-02-12 16:14:21 ---
i compilled it from AUR, maybe package is broken

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #21 from Gleb <gleb@redhat.com>  2012-02-13 08:53:28 ---
Can you please compile trace-cmd from its git [1] (do "make all_cmd
install_cmd"; install part is important) and try getting trace with it? If this
will not work I will guide you how to take a trace using debugfs directly.

[1] git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/trace-cmd.git

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #22 from Rosen <sandikata@yandex.ru>  2012-02-13 13:19:29 ---
Created an attachment (id=72363)
 --> (https://bugzilla.kernel.org/attachment.cgi?id=72363)
trace-cmd report

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #23 from Gleb <gleb@redhat.com>  2012-02-13 13:30:01 ---
What guest did during this trace? Can you provide "info pci" monitor output
pls?

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755


Avi Kivity <avi@redhat.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |avi@redhat.com




--- Comment #24 from Avi Kivity <avi@redhat.com>  2012-02-13 13:43:35 ---
Please run 'perf top' in the host and report the output (while tracing is
disabled).

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #25 from Rosen <sandikata@yandex.ru>  2012-02-13 14:14:17 ---
(In reply to comment #23)
> What guest did during this trace? Can you provide "info pci" monitor output
> pls?

can't see full output from this command

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #26 from Rosen <sandikata@yandex.ru>  2012-02-13 14:24:36 ---
Created an attachment (id=72365)
 --> (https://bugzilla.kernel.org/attachment.cgi?id=72365)
info pci

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #27 from Rosen <sandikata@yandex.ru>  2012-02-13 14:37:36 ---
and there soon will be video capture with 'perf top'

http://vbox7.com/play:199e9ede30

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #28 from Avi Kivity <avi@redhat.com>  2012-02-14 14:47:38 ---
(In reply to comment #27)
> and there soon will be video capture with 'perf top'
> 
> http://vbox7.com/play:199e9ede30

Run it while the guest is also running.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #29 from Rosen <sandikata@yandex.ru>  2012-02-15 07:49:35 ---
(In reply to comment #28)
> (In reply to comment #27)
> > and there soon will be video capture with 'perf top'
> > 
> > http://vbox7.com/play:199e9ede30
> 
> Run it while the guest is also running.

Good Morning!
There will be video http://vbox7.com/play:7128f03f1f after some momments.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #30 from Avi Kivity <avi@redhat.com>  2012-02-15 09:28:12 ---
Disable ksm, and build with debug information so we get useful information
instead of hex addresses.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #31 from Rosen <sandikata@yandex.ru>  2012-02-21 08:55:31 ---
any responses ?
with/without ksm no change.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #32 from Gleb <gleb@redhat.com>  2012-02-21 09:10:37 ---
Can you run perf without ksm? In your previous perf invocation ksm was at the
top, so lets see what is it there now.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755


Kurk <kurk@shiftmail.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |kurk@shiftmail.org




--- Comment #33 from Kurk <kurk@shiftmail.org>  2012-02-21 11:53:33 ---
Supposing it was working good with an old version of kvm: 
Why don't you just do a git bisect? qemu-kvm is very quick to compile and
install. Start kvm with -snapshot so you can kill it anytime; and just by
visual inspection at boot you should be able to determine if it is fast or slow
and proceed with next step of bisect. In a short time you would probably solve
this.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #34 from Rosen <sandikata@yandex.ru>  2012-02-21 12:45:15 ---
(In reply to comment #33)
> Supposing it was working good with an old version of kvm: 
> Why don't you just do a git bisect? qemu-kvm is very quick to compile and
> install. Start kvm with -snapshot so you can kill it anytime; and just by
> visual inspection at boot you should be able to determine if it is fast or slow
> and proceed with next step of bisect. In a short time you would probably solve
> this.

not that fast compilling, it eat my memory durring compille process and going
to megaswapping..

100% CPU RAM SWAP usage when compilling kvm.

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

[Bug 42755] KVM is being extremely slow on AMD Athlon64 4000+ Dual Core 2.1GHz Brisbane

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=42755





--- Comment #35 from Rosen <sandikata@yandex.ru>  2012-02-26 09:53:22 ---
No changes with git version of qemu-kvm
Only see when work guest cpu usage is out of range, about 120% - 150%

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.