xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
From: Andrii Anisov <andrii.anisov@gmail.com>
To: Julien Grall <julien.grall@arm.com>,
	xen-devel <xen-devel@lists.xenproject.org>
Subject: Re: arm: xl vcpu-pin leads to oom-killer slashing processes
Date: Wed, 5 Dec 2018 16:46:37 +0200	[thread overview]
Message-ID: <973a7ae9-2c78-690d-cf70-913a99767720@gmail.com> (raw)
In-Reply-To: <69b378ad-3710-afe9-25a0-8825caf848a7@arm.com>



On 05.12.18 15:13, Julien Grall wrote:
> I need at least some sort of proof that Xen might corrupt the kernel. I don't believe we manage to just corrupt the kernel memory subsystem with good enough value reliably. So maybe we should start looking at more plausible cause.
I think I would be able to look deeper into it by the end of the week.

> It does not mean this is because of Xen. It might just be because of Xen drivers that does not free memory.
Totally agree. I did not dig into it yet. Just made a notification, so maybe other interested parties can check/verify the issue on different setups.

> I tried and can't reproduce it.
Great. So it might be the problem on my setup.

> But I am using 4.20-rc4 and not 4.14.35. If you think the bug was introduced in recent Xen, then the first step is to downgrade Xen. If it does not happen on the downgraded version, then you can bisect it.
It definitely does not reproduce with XEN 4.10 release. But it is pretty old.

> How about after vCPU pining? Do you see the memory free going down?
I've checked, and seen a strange thing: memtotal is shrinked down.
Meminfo before vcpu pin:

MemTotal:        2995828 kB
MemFree:         2810360 kB
MemAvailable:    2758420 kB
Buffers:               0 kB
Cached:            53092 kB
SwapCached:            0 kB
Active:            26716 kB
Inactive:          40980 kB
Active(anon):      14976 kB
Inactive(anon):    20648 kB
Active(file):      11740 kB
Inactive(file):    20332 kB
Unevictable:          12 kB
Mlocked:              12 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:         14644 kB
Mapped:            31632 kB
Shmem:             21020 kB
Slab:              29204 kB
SReclaimable:       9924 kB
SUnreclaim:        19280 kB
KernelStack:        2848 kB
PageTables:          828 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     1497912 kB
Committed_AS:      58556 kB
VmallocTotal:   135290290112 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
CmaTotal:         393216 kB
CmaFree:          360488 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB


Meminfo before oom-killer rose:

root@salvator-x:~# cat /proc/meminfo
MemTotal:         549000 kB
MemFree:          412108 kB
MemAvailable:     347472 kB
Buffers:               0 kB
Cached:            19356 kB
SwapCached:            0 kB
Active:            15920 kB
Inactive:          17512 kB
Active(anon):      14516 kB
Inactive(anon):     9104 kB
Active(file):       1404 kB
Inactive(file):     8408 kB
Unevictable:          12 kB
Mlocked:              12 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:         14160 kB
Mapped:             9016 kB
Shmem:              9496 kB
Slab:              26412 kB
SReclaimable:       6792 kB
SUnreclaim:        19620 kB
KernelStack:        2880 kB
PageTables:          776 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      274500 kB
Committed_AS:      49264 kB
VmallocTotal:   135290290112 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
CmaTotal:         393216 kB
CmaFree:          360488 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB


> So that's reliably happening?
It is 100% reproducible on my setup.

> Are you sure there are nothing else on the system using memory? For instance you seem to have nfs in place.
Yes, Dom0 root is nfs.

-- 
Sincerely,
Andrii Anisov.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

  reply	other threads:[~2018-12-05 14:46 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-12-05 10:26 arm: xl vcpu-pin leads to oom-killer slashing processes Andrii Anisov
2018-12-05 10:28 ` Andrii Anisov
2018-12-05 10:49 ` Julien Grall
2018-12-05 10:59   ` Andrii Anisov
2018-12-05 11:45     ` Julien Grall
2018-12-05 11:59       ` Andrii Anisov
2018-12-05 12:15         ` Julien Grall
2018-12-05 12:40           ` Andrii Anisov
2018-12-05 13:13             ` Julien Grall
2018-12-05 14:46               ` Andrii Anisov [this message]
2018-12-13 14:13                 ` Andrii Anisov
2018-12-13 16:37                   ` Juergen Gross
2018-12-14 10:36                     ` Andrii Anisov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=973a7ae9-2c78-690d-cf70-913a99767720@gmail.com \
    --to=andrii.anisov@gmail.com \
    --cc=julien.grall@arm.com \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).