All of lore.kernel.org
 help / color / mirror / Atom feed
From: Michal Hocko <mhocko@suse.com>
To: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: akpm@linux-foundation.org, linux-mm@kvack.org,
	xiyou.wangcong@gmail.com, dave.hansen@intel.com,
	hannes@cmpxchg.org, mgorman@suse.de, vbabka@suse.cz
Subject: Re: [PATCH] mm,page_alloc: Serialize warn_alloc() if schedulable.
Date: Thu, 1 Jun 2017 15:28:08 +0200	[thread overview]
Message-ID: <20170601132808.GD9091@dhcp22.suse.cz> (raw)
In-Reply-To: <201706012211.GHI18267.JFOVMSOLFFQHOt@I-love.SAKURA.ne.jp>

On Thu 01-06-17 22:11:13, Tetsuo Handa wrote:
> Michal Hocko wrote:
> > On Thu 01-06-17 20:43:47, Tetsuo Handa wrote:
> > > Cong Wang has reported a lockup when running LTP memcg_stress test [1].
> >
> > This seems to be on an old and not pristine kernel. Does it happen also
> > on the vanilla up-to-date kernel?
> 
> 4.9 is not an old kernel! It might be close to the kernel version which
> enterprise distributions would choose for their next long term supported
> version.
> 
> And please stop saying "can you reproduce your problem with latest
> linux-next (or at least latest linux)?" Not everybody can use the vanilla
> up-to-date kernel!

The changelog mentioned that the source of stalls is not clear so this
might be out-of-tree patches doing something wrong and dump_stack
showing up just because it is called often. This wouldn't be the first
time I have seen something like that. I am not really keen on adding
heavy lifting for something that is not clearly debugged and based on
hand waving and speculations.

> What I'm pushing via kmallocwd patch is to prepare for overlooked problems
> so that enterprise distributors can collect information and identify what
> changes are needed to be backported.
> 
> As long as you ignore problems not happened with latest linux-next (or
> at least latest linux), enterprise distribution users can do nothing.
> 
> >
> > [...]
> > > Therefore, this patch uses a mutex dedicated for warn_alloc() like
> > > suggested in [3].
> >
> > As I've said previously. We have rate limiting and if that doesn't work
> > out well, let's tune it. The lock should be the last resort to go with.
> > We already throttle show_mem, maybe we can throttle dump_stack as well,
> > although it sounds a bit strange that this adds so much to the picture.
> 
> Ratelimiting never works well. It randomly drops information which is
> useful for debugging. Uncontrolled concurrent dump_stack() causes lockups.
> And restricting dump_stack() drops more information.

As long as the dump_stack can be a source of the stalls, which I am not
so sure about, then we should rate limit it.

> What we should do is to yield CPU time to operations which might do useful
> things (let threads not doing memory allocation; e.g. let printk kernel
> threads to flush pending buffer, let console drivers write the output to
> consoles, let watchdog kernel threads report what is happening).

yes we call that preemptive kernel...

> When memory allocation request is stalling, serialization via waiting
> for a lock does help.

Which will mean that those unlucky ones which stall will stall even more
because they will wait on a lock with potentially many others. While
this certainly is a throttling mechanism it is also a big hammer.
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2017-06-01 13:28 UTC|newest]

Thread overview: 43+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-06-01 11:43 [PATCH] mm,page_alloc: Serialize warn_alloc() if schedulable Tetsuo Handa
2017-06-01 11:59 ` Michal Hocko
2017-06-01 13:11   ` Tetsuo Handa
2017-06-01 13:28     ` Michal Hocko [this message]
2017-06-01 22:10       ` Andrew Morton
2017-06-02  7:18         ` Michal Hocko
2017-06-02 11:13           ` Tetsuo Handa
2017-06-02 12:15             ` Michal Hocko
2017-06-02 17:13               ` Tetsuo Handa
2017-06-02 21:57             ` Cong Wang
2017-06-04  8:58               ` Tetsuo Handa
2017-06-04 15:05                 ` Michal Hocko
2017-06-04 21:43                   ` Tetsuo Handa
2017-06-05  5:37                     ` Michal Hocko
2017-06-05 18:15                       ` Cong Wang
2017-06-06  9:17                         ` Michal Hocko
2017-06-05 18:25                 ` Cong Wang
2017-06-22 10:35                   ` Tetsuo Handa
2017-06-22 22:53                     ` Cong Wang
2017-06-02 16:59           ` Cong Wang
2017-06-02 19:59           ` Andrew Morton
2017-06-03  2:57             ` Tetsuo Handa
2017-06-03  7:32             ` Michal Hocko
2017-06-03  8:36               ` Tetsuo Handa
2017-06-05  7:10                 ` Sergey Senozhatsky
2017-06-05  9:36                   ` Sergey Senozhatsky
2017-06-05 15:02                     ` Tetsuo Handa
2017-06-03 13:21               ` Tetsuo Handa
2017-07-08  4:59           ` Tetsuo Handa
2017-07-10 13:21             ` Michal Hocko
2017-07-10 13:54               ` Tetsuo Handa
2017-07-10 14:14                 ` Michal Hocko
2017-07-11 13:10                   ` Tetsuo Handa
2017-07-11 13:49                     ` Michal Hocko
2017-07-11 14:58                       ` Petr Mladek
2017-07-11 22:06                       ` Tetsuo Handa
2017-07-12  8:54                         ` Michal Hocko
2017-07-12 12:23                           ` Tetsuo Handa
2017-07-12 12:41                             ` Michal Hocko
2017-07-14 12:30                               ` Tetsuo Handa
2017-07-14 12:48                                 ` Michal Hocko
2017-08-09  6:14                                   ` Tetsuo Handa
2017-08-09 13:01                                     ` Tetsuo Handa

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170601132808.GD9091@dhcp22.suse.cz \
    --to=mhocko@suse.com \
    --cc=akpm@linux-foundation.org \
    --cc=dave.hansen@intel.com \
    --cc=hannes@cmpxchg.org \
    --cc=linux-mm@kvack.org \
    --cc=mgorman@suse.de \
    --cc=penguin-kernel@I-love.SAKURA.ne.jp \
    --cc=vbabka@suse.cz \
    --cc=xiyou.wangcong@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.