LKML Archive on lore.kernel.org
 help / color / Atom feed
From: Steven Rostedt <rostedt@goodmis.org>
To: Matthew Wilcox <willy@infradead.org>
Cc: Joel Fernandes <joelaf@google.com>,
	Zhaoyang Huang <huangzhaoyang@gmail.com>,
	Ingo Molnar <mingo@kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	kernel-patch-test@lists.linaro.org,
	Andrew Morton <akpm@linux-foundation.org>,
	Michal Hocko <mhocko@suse.com>,
	"open list:MEMORY MANAGEMENT" <linux-mm@kvack.org>,
	Vlastimil Babka <vbabka@suse.cz>,
	Michal Hocko <mhocko@kernel.org>
Subject: Re: [PATCH v1] kernel/trace:check the val against the available mem
Date: Fri, 30 Mar 2018 23:07:33 -0400
Message-ID: <20180330230733.2bf010f2@gandalf.local.home> (raw)
In-Reply-To: <20180331021857.GD13332@bombadil.infradead.org>

On Fri, 30 Mar 2018 19:18:57 -0700
Matthew Wilcox <willy@infradead.org> wrote:

> Again though, this is the same pattern as vmalloc.  There are any number
> of places where userspace can cause an arbitrarily large vmalloc to be
> attempted (grep for kvmalloc_array for a list of promising candidates).
> I'm pretty sure that just changing your GFP flags to GFP_KERNEL |
> __GFP_NOWARN will give you the exact behaviour that you want with no
> need to grub around in the VM to find out if your huge allocation is
> likely to succeed.

Not sure how this helps. Note, I don't care about consecutive pages, so
this is not an array. It's a link list of thousands of pages. How do
you suggest allocating them? The ring buffer is a link list of pages.

What I currently do is to see how many more pages I need. Allocate them
one at a time and put them in a temporary list, if it succeeds I add
them to the ring buffer, if not, I free the entire list (it's an all or
nothing operation).

The allocation I'm making doesn't warn. The problem is the
GFP_RETRY_MAYFAIL, which will try to allocate any possible memory in
the system. When it succeeds, the ring buffer allocation logic will
then try to allocate another page. If we add too many pages, we will
allocate all possible pages and then try to allocate more. This
allocation will fail without causing an OOM. That's not the problem.
The problem is if the system is active during this time, and something
else tries to do any allocation, after all memory has been consumed,
that allocation will fail. Then it will trigger an OOM.

I showed this in my Call Trace, that the allocation that failed during
my test was something completely unrelated, and that failure caused an
OOM.

What this last patch does is see if there's space available before it
even starts the process.

Maybe I'm missing something, but I don't see how NOWARN can help. My
allocations are not what is giving the warning.

-- Steve

  reply index

Thread overview: 64+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-03-29 10:41 Zhaoyang Huang
2018-03-29 16:05 ` Steven Rostedt
2018-03-30  3:32   ` Zhaoyang Huang
2018-03-30 14:07     ` Steven Rostedt
2018-03-30  6:53 ` [Kernel-patch-test] " kbuild test robot
2018-03-30  6:54 ` kbuild test robot
2018-03-30 14:20 ` Steven Rostedt
2018-03-30 16:37   ` Joel Fernandes
2018-03-30 19:10     ` Steven Rostedt
2018-03-30 20:37       ` Joel Fernandes
2018-03-30 20:53   ` Matthew Wilcox
2018-03-30 21:30     ` Steven Rostedt
2018-03-30 21:42       ` Steven Rostedt
2018-03-30 23:38         ` Joel Fernandes
2018-03-31  1:41           ` Steven Rostedt
2018-03-31  2:18             ` Matthew Wilcox
2018-03-31  3:07               ` Steven Rostedt [this message]
2018-03-31  5:44                 ` Joel Fernandes
2018-04-02  0:52         ` Zhaoyang Huang
2018-04-03 11:06   ` Michal Hocko
2018-04-03 11:51     ` Steven Rostedt
2018-04-03 12:16       ` Michal Hocko
2018-04-03 12:23         ` Steven Rostedt
2018-04-03 12:35           ` Michal Hocko
2018-04-03 13:32             ` Steven Rostedt
2018-04-03 13:56               ` Michal Hocko
2018-04-03 14:17                 ` Steven Rostedt
2018-04-03 16:11                   ` Michal Hocko
2018-04-03 16:59                     ` Steven Rostedt
2018-04-03 22:56                     ` Steven Rostedt
2018-04-04  6:20                       ` Michal Hocko
2018-04-04 12:21                         ` Joel Fernandes
2018-04-04 12:59                         ` Steven Rostedt
2018-04-04 14:10                           ` Michal Hocko
2018-04-04 14:25                             ` Steven Rostedt
2018-04-04 14:42                               ` Michal Hocko
2018-04-04 15:04                                 ` Steven Rostedt
2018-04-04 15:27                                   ` Michal Hocko
2018-04-04 15:38                                     ` Steven Rostedt
2018-04-04  2:58                 ` Zhaoyang Huang
2018-04-04  6:23                   ` Michal Hocko
2018-04-04  9:29                     ` Zhaoyang Huang
2018-04-04 14:11                     ` Steven Rostedt
2018-04-04 14:23                       ` Michal Hocko
2018-04-04 14:31                         ` Steven Rostedt
2018-04-04 14:47                           ` Michal Hocko
2018-04-04 15:47                         ` Steven Rostedt
2018-04-05  2:58                           ` Matthew Wilcox
2018-04-05  4:12                             ` Joel Fernandes
2018-04-05 14:22                               ` Matthew Wilcox
2018-04-05 14:27                                 ` Michal Hocko
2018-04-05 14:34                                   ` Steven Rostedt
2018-04-05 15:13                                   ` Matthew Wilcox
2018-04-05 15:32                                     ` Michal Hocko
2018-04-05 16:15                                       ` Matthew Wilcox
2018-04-05 18:54                                         ` Michal Hocko
2018-04-05 20:15                                           ` __GFP_LOW Matthew Wilcox
2018-04-06  6:09                                             ` __GFP_LOW Michal Hocko
2018-04-08  4:27                                               ` __GFP_LOW Matthew Wilcox
2018-04-09  7:34                                                 ` __GFP_LOW Michal Hocko
2018-04-09 15:51                                                   ` __GFP_LOW Matthew Wilcox
2018-04-09 18:14                                                     ` __GFP_LOW Michal Hocko
     [not found]                                                       ` <CA+JonM0HG9kWb6-0iyDQ8UMxTeR-f=+ZL89t5DvvDULDC8Sfyw@mail.gmail.com>
2018-04-10 12:19                                                         ` __GFP_LOW Matthew Wilcox
2018-04-05 14:30                                 ` [PATCH v1] kernel/trace:check the val against the available mem Steven Rostedt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180330230733.2bf010f2@gandalf.local.home \
    --to=rostedt@goodmis.org \
    --cc=akpm@linux-foundation.org \
    --cc=huangzhaoyang@gmail.com \
    --cc=joelaf@google.com \
    --cc=kernel-patch-test@lists.linaro.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@kernel.org \
    --cc=mhocko@suse.com \
    --cc=mingo@kernel.org \
    --cc=vbabka@suse.cz \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
	git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git
	git clone --mirror https://lore.kernel.org/lkml/10 lkml/git/10.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git