Re: [PATCH 1/3] kvfree_rcu: Allocate a page for a single argument

From: Uladzislau Rezki <urezki@gmail.com>
To: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Uladzislau Rezki <urezki@gmail.com>,
	LKML <linux-kernel@vger.kernel.org>, RCU <rcu@vger.kernel.org>,
	"Paul E . McKenney" <paulmck@kernel.org>,
	Michael Ellerman <mpe@ellerman.id.au>,
	Andrew Morton <akpm@linux-foundation.org>,
	Daniel Axtens <dja@axtens.net>,
	Frederic Weisbecker <frederic@kernel.org>,
	Neeraj Upadhyay <neeraju@codeaurora.org>,
	Joel Fernandes <joel@joelfernandes.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Michal Hocko <mhocko@suse.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	"Theodore Y . Ts'o" <tytso@mit.edu>,
	Oleksiy Avramchenko <oleksiy.avramchenko@sonymobile.com>
Subject: Re: [PATCH 1/3] kvfree_rcu: Allocate a page for a single argument
Date: Fri, 22 Jan 2021 15:21:13 +0100	[thread overview]
Message-ID: <20210122142113.GA1873@pc638.lan> (raw)
In-Reply-To: <20210122113409.w55kul5bpjxhkgm4@linutronix.de>

> On 2021-01-21 13:38:34 [+0100], Uladzislau Rezki wrote:
> > __get_free_page() returns "unsigned long" whereas a bnode is a pointer
> > to kvfree_rcu_bulk_data struct, without a casting the compiler will
> > emit a warning.
> 
> Yes, learned about it, sorry.
> 
> > >> You think that a CPU migration is a bad thing. But why?
> > >>
> > It is not a bad thing. But if it happens we might queue a new bnode
> > to a drain list of another CPU where a previous element of a new
> > bnode may be just underutilized. So that is why i use migrate_disable()/enable()
> > to prevent it.
> 
> If you allocate a node for queueing and then you realize that you
> already have one then you either free it or queue it for later.
> Given that all this is a slower-path and that this is used on every-CPU,
> sooner or later that page will be used avoids a later allocation, right?
> 
To free it right away is a bit problematic, because we need at least one more
time to drop the lock what would introduce more complexity. Adding to the cache 
for later reuse is possible but would require an extra decay cache logic.

I think, since it is a corner case and i do not consider it as a big issue,
we can just queue it to the drain list so the previous node can be half filled
due to migration.

> > If there are some hidden issues with migrate_disable()/enable() or you
> > think it is a bad idea to use it, it would be appreciated if you could
> > describe your view in more detail.
> 
> Just what I mentioned in my previous email:
> - the whole operation isn't cheap but it is more efficient in tree
>   compared to what we used to have in RT.
> - the task can no be freely placed while it could run. So if the task
>   gets preempted, it will have to wait until it can run on the CPU
>   again.
> 
Yep, that is obvious. From scheduling point of view the task can wait more
time because the migration is prohibited. From the other hand, the page is
obtained in the path that is not considered as a hot one. One page can serve 
~500 pointers.

I do not say that we should keep: [PATCH 3/3] kvfree_rcu: use migrate_disable/enable()

without it, a migration can occur, what is really rare according to my tests
therefore i do not have a strong opinion about keeping it. If you or someone
else has some concern we can drop it.

--
Vlad Rezki