kvm.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Michal Hocko <mhocko@suse.com>
To: Jiri Slaby <jirislaby@kernel.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>,
	kvm@vger.kernel.org, Andrew Morton <akpm@linux-foundation.org>,
	mm <linux-mm@kvack.org>,
	yuzhao@google.com, Vlastimil Babka <vbabka@suse.cz>,
	shy828301@gmail.com
Subject: Re: Stalls in qemu with host running 6.1 (everything stuck at mmap_read_lock())
Date: Wed, 11 Jan 2023 09:31:21 +0100	[thread overview]
Message-ID: <Y75z2Ran1+Iw+L6+@dhcp22.suse.cz> (raw)
In-Reply-To: <b8017e09-f336-3035-8344-c549086c2340@kernel.org>

On Wed 11-01-23 09:00:04, Jiri Slaby wrote:
> Hi,
> 
> after I updated the host from 6.0 to 6.1 (being at 6.1.4 ATM), my qemu VMs
> started stalling (and the host at the same point too). It doesn't happen
> right after boot, maybe a suspend-resume cycle is needed (or longer uptime,
> or a couple of qemu VM starts, or ...). But when it happens, it happens all
> the time till the next reboot.
> 
> Older guest's kernels/distros are affected as well as Win10.
> 
> In guests, I see for example stalls in memset_orig or
> smp_call_function_many_cond -- traces below.
> 
> qemu-kvm-7.1.0-13.34.x86_64 from openSUSE.
> 
> It's quite interesting that:
>   $ cat /proc/<PID_OF_QEMU>/cmdline
> is stuck at read:
> 
> openat(AT_FDCWD, "/proc/12239/cmdline", O_RDONLY) = 3
> newfstatat(3, "", {st_mode=S_IFREG|0444, st_size=0, ...}, AT_EMPTY_PATH) = 0
> fadvise64(3, 0, 0, POSIX_FADV_SEQUENTIAL) = 0
> mmap(NULL, 139264, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
> 0x7f22f0487000
> read(3, ^C^C^C^\^C

This will require mmap_lock for read as well. And that is blocked
because there is a pending writer as well.
[...]
 
> Is this known? Any idea how to debug this?

I would recommend taking a crashdump when the host is in that state and
examine the state of the blocked lock. Dump will hopefully give you more
information about potential holder of the lock. If it is blocked on
writer then you should get the owner task. If it is in read mode then
this can get more tricky because the exact owner might be different from
the recorded one. Anyway the full list of tasks and their backtraces
could tell more.

HTH

-- 
Michal Hocko
SUSE Labs

  parent reply	other threads:[~2023-01-11  8:31 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-01-11  8:00 Stalls in qemu with host running 6.1 (everything stuck at mmap_read_lock()) Jiri Slaby
2023-01-11  8:23 ` Vlastimil Babka
2023-01-11  8:31 ` Michal Hocko [this message]
2023-01-11  9:21 ` Linux kernel regression tracking (#adding)
2023-01-19 14:58   ` Vlastimil Babka
2023-01-11 22:50 ` Paolo Bonzini
2023-01-12  0:37 ` Pedro Falcato
2023-01-12  6:07   ` Jiri Slaby
2023-01-12 10:31     ` Maxim Levitsky
2023-01-12 10:35       ` Vlastimil Babka
2023-01-12 15:43     ` Jiri Slaby
2023-01-12 16:57       ` Jiri Slaby
2023-01-13 17:14         ` Liam Howlett
2023-01-12  8:23   ` Yu Zhao
2023-01-13 17:27     ` Liam Howlett

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=Y75z2Ran1+Iw+L6+@dhcp22.suse.cz \
    --to=mhocko@suse.com \
    --cc=akpm@linux-foundation.org \
    --cc=jirislaby@kernel.org \
    --cc=kvm@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=pbonzini@redhat.com \
    --cc=shy828301@gmail.com \
    --cc=vbabka@suse.cz \
    --cc=yuzhao@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).