From: Uladzislau Rezki <urezki@gmail.com>
To: Michal Hocko <mhocko@suse.com>
Cc: Uladzislau Rezki <urezki@gmail.com>,
Peter Zijlstra <peterz@infradead.org>,
"Paul E. McKenney" <paulmck@kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
LKML <linux-kernel@vger.kernel.org>, RCU <rcu@vger.kernel.org>,
linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>,
Vlastimil Babka <vbabka@suse.cz>,
Matthew Wilcox <willy@infradead.org>,
"Theodore Y . Ts'o" <tytso@mit.edu>,
Joel Fernandes <joel@joelfernandes.org>,
Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
Oleksiy Avramchenko <oleksiy.avramchenko@sonymobile.com>
Subject: Re: [RFC-PATCH 1/2] mm: Add __GFP_NO_LOCKS flag
Date: Mon, 17 Aug 2020 12:36:00 +0200 [thread overview]
Message-ID: <20200817103600.GA21301@pc636> (raw)
In-Reply-To: <20200817082849.GA28270@dhcp22.suse.cz>
On Mon, Aug 17, 2020 at 10:28:49AM +0200, Michal Hocko wrote:
> On Mon 17-08-20 00:56:55, Uladzislau Rezki wrote:
> [...]
> > Michal asked to provide some data regarding how many pages we need and how
> > "lockless allocation" behaves when it comes to success vs failed scenarios.
> >
> > Please see below some results. The test case is a tight loop of 1 000 000 allocations
> > doing kmalloc() and kfree_rcu():
>
> It would be nice to cover some more realistic workloads as well.
>
Hmm.. I tried to show syntactic worst case when a "flood" occurs.
In such conditions we can get fails what is expectable and we have
fallback mechanism for it.
> > sudo ./test_vmalloc.sh run_test_mask=2048 single_cpu_test=1
> >
> > <snip>
> > for (i = 0; i < 1 000 000; i++) {
> > p = kmalloc(sizeof(*p), GFP_KERNEL);
> > if (!p)
> > return -1;
> >
> > p->array[0] = 'a';
> > kvfree_rcu(p, rcu);
> > }
> > <snip>
> >
> > wget ftp://vps418301.ovh.net/incoming/1000000_kmalloc_kfree_rcu_proc_percpu_pagelist_fractio_is_0.png
>
> If I understand this correctly then this means that failures happen very
> often because pcp pages are not recycled quicklly enough.
>
Yep, it happens and that is kind of worst scenario(flood one). Therefore we
have a fallback and is expectable. Also, i did not provide the number of pages
in a loop. On my test machine we need approximately ~300/400 pages to cover
that flood case until we recycles or return back the pages to the pcp.
Please note, as i mentioned before. Our drain part is not optimal for sure,
it means that we can rework it a bit making it more efficient. For example,
when a flood occurs, instead of delaying "reclaimer logic" thread, it can be
placed to a run-queue right away. We can use separate "flush workqueue"
that is tagged with WQ_MEM_RECLAIM raising a priority of drain context.
i.e. there is a room for reducing such page footprint.
> > wget ftp://vps418301.ovh.net/incoming/1000000_kmalloc_kfree_rcu_proc_percpu_pagelist_fractio_is_8.png
>
> 1/8 of the memory in pcp lists is quite large and likely not something
> used very often.
>
Just for illustration. When percpu_pagelist_fractio is set to 8, i do
not see any page fail on a single CPU flood case. If i run simultaneously
such flood on all available CPUs there will be fails for sure.
> Both these numbers just make me think that a dedicated pool of page
> pre-allocated for RCU specifically might be a better solution. I still
> haven't read through that branch of the email thread though so there
> might be some pretty convincing argments to not do that.
>
> > Also i would like to underline, that kfree_rcu() reclaim logic can be improved further,
> > making the drain logic more efficient when it comes to time, thus to reduce a footprint
> > as a result number of required pages.
> >
> > --
> > Vlad Rezki
>
> --
> Michal Hocko
> SUSE Labs
next prev parent reply other threads:[~2020-08-17 10:36 UTC|newest]
Thread overview: 111+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-09 20:43 [RFC-PATCH 0/2] __GFP_NO_LOCKS Uladzislau Rezki (Sony)
2020-08-09 20:43 ` [RFC-PATCH 1/2] mm: Add __GFP_NO_LOCKS flag Uladzislau Rezki (Sony)
2020-08-10 12:31 ` Michal Hocko
2020-08-10 16:07 ` Uladzislau Rezki
2020-08-10 19:25 ` Michal Hocko
2020-08-11 8:19 ` Michal Hocko
2020-08-11 9:37 ` Uladzislau Rezki
2020-08-11 9:42 ` Uladzislau Rezki
2020-08-11 10:28 ` Michal Hocko
2020-08-11 10:45 ` Uladzislau Rezki
2020-08-11 10:26 ` Michal Hocko
2020-08-11 11:33 ` Uladzislau Rezki
2020-08-11 9:18 ` Uladzislau Rezki
2020-08-11 10:21 ` Michal Hocko
2020-08-11 11:10 ` Uladzislau Rezki
2020-08-11 14:44 ` Thomas Gleixner
2020-08-11 15:22 ` Thomas Gleixner
2020-08-12 11:38 ` Thomas Gleixner
2020-08-12 12:01 ` Uladzislau Rezki
2020-08-13 7:18 ` Michal Hocko
2020-08-11 15:33 ` Paul E. McKenney
2020-08-11 15:43 ` Thomas Gleixner
2020-08-11 15:56 ` Sebastian Andrzej Siewior
2020-08-11 16:02 ` Paul E. McKenney
2020-08-11 16:19 ` Paul E. McKenney
2020-08-11 19:39 ` Thomas Gleixner
2020-08-11 21:09 ` Paul E. McKenney
2020-08-12 0:13 ` Thomas Gleixner
2020-08-12 4:29 ` Paul E. McKenney
2020-08-12 8:32 ` Thomas Gleixner
2020-08-12 13:30 ` Paul E. McKenney
2020-08-13 7:50 ` Michal Hocko
2020-08-13 9:58 ` Uladzislau Rezki
2020-08-13 11:15 ` Michal Hocko
2020-08-13 13:27 ` Thomas Gleixner
2020-08-13 13:45 ` Michal Hocko
2020-08-13 14:32 ` Matthew Wilcox
2020-08-13 16:14 ` Thomas Gleixner
2020-08-13 16:22 ` Matthew Wilcox
2020-08-13 13:22 ` Thomas Gleixner
2020-08-13 13:33 ` Michal Hocko
2020-08-13 14:34 ` Thomas Gleixner
2020-08-13 14:53 ` Michal Hocko
2020-08-13 15:41 ` Paul E. McKenney
2020-08-13 15:54 ` Michal Hocko
2020-08-13 16:04 ` Paul E. McKenney
2020-08-13 16:13 ` Michal Hocko
2020-08-13 16:29 ` Paul E. McKenney
2020-08-13 17:12 ` Michal Hocko
2020-08-13 17:27 ` Paul E. McKenney
2020-08-13 18:31 ` peterz
2020-08-13 19:13 ` Michal Hocko
2020-08-13 16:20 ` Uladzislau Rezki
2020-08-13 16:36 ` Michal Hocko
2020-08-14 11:54 ` Uladzislau Rezki
2020-08-13 17:09 ` Thomas Gleixner
2020-08-13 17:22 ` Michal Hocko
2020-08-14 7:17 ` Michal Hocko
2020-08-14 12:15 ` Uladzislau Rezki
2020-08-14 12:48 ` Michal Hocko
2020-08-14 13:34 ` Paul E. McKenney
2020-08-14 14:06 ` Michal Hocko
2020-08-14 18:01 ` Paul E. McKenney
2020-08-14 23:14 ` Thomas Gleixner
2020-08-14 23:41 ` Paul E. McKenney
2020-08-15 0:43 ` Thomas Gleixner
2020-08-15 3:01 ` Paul E. McKenney
2020-08-15 8:27 ` Peter Zijlstra
2020-08-15 13:03 ` Paul E. McKenney
2020-08-15 8:42 ` Peter Zijlstra
2020-08-15 14:18 ` Paul E. McKenney
2020-08-15 14:23 ` Paul E. McKenney
2020-08-17 8:47 ` Michal Hocko
2020-08-13 18:26 ` peterz
2020-08-13 18:52 ` Paul E. McKenney
2020-08-13 22:06 ` peterz
2020-08-13 23:23 ` Paul E. McKenney
2020-08-13 23:59 ` Thomas Gleixner
2020-08-14 8:30 ` Peter Zijlstra
2020-08-14 10:23 ` peterz
2020-08-14 15:26 ` Paul E. McKenney
2020-08-14 14:14 ` Paul E. McKenney
2020-08-14 16:11 ` Paul E. McKenney
2020-08-14 17:49 ` Peter Zijlstra
2020-08-14 18:02 ` Paul E. McKenney
2020-08-14 19:33 ` Thomas Gleixner
2020-08-14 20:41 ` Paul E. McKenney
2020-08-14 21:52 ` Peter Zijlstra
2020-08-14 23:27 ` Paul E. McKenney
2020-08-14 23:40 ` Thomas Gleixner
2020-08-16 22:56 ` Uladzislau Rezki
2020-08-17 8:28 ` Michal Hocko
2020-08-17 10:36 ` Uladzislau Rezki [this message]
2020-08-17 22:28 ` Paul E. McKenney
2020-08-18 7:43 ` Michal Hocko
2020-08-18 13:53 ` Paul E. McKenney
2020-08-18 14:43 ` Thomas Gleixner
2020-08-18 16:13 ` Paul E. McKenney
2020-08-18 16:55 ` Thomas Gleixner
2020-08-18 17:13 ` Paul E. McKenney
2020-08-18 23:26 ` Thomas Gleixner
2020-08-19 23:07 ` Paul E. McKenney
2020-08-18 15:02 ` Michal Hocko
2020-08-18 15:45 ` Uladzislau Rezki
2020-08-18 16:18 ` Paul E. McKenney
2020-08-14 16:19 ` peterz
2020-08-14 18:15 ` Paul E. McKenney
2020-08-13 13:29 ` Uladzislau Rezki
2020-08-13 13:41 ` Michal Hocko
2020-08-13 14:22 ` Uladzislau Rezki
2020-08-09 20:43 ` [PATCH 2/2] rcu/tree: use " Uladzislau Rezki (Sony)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200817103600.GA21301@pc636 \
--to=urezki@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=bigeasy@linutronix.de \
--cc=joel@joelfernandes.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.com \
--cc=oleksiy.avramchenko@sonymobile.com \
--cc=paulmck@kernel.org \
--cc=peterz@infradead.org \
--cc=rcu@vger.kernel.org \
--cc=tglx@linutronix.de \
--cc=tytso@mit.edu \
--cc=vbabka@suse.cz \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).