Re: Is there a reason we don't have kvfree_rcu()?

[Joel Fernandes <joel@joelfernandes.org>]

On Fri, Mar 13, 2020 at 02:21:24PM +0100, Uladzislau Rezki wrote:
> On Thu, Mar 12, 2020 at 11:58:09PM -0400, Joel Fernandes wrote:
> > On Thu, Mar 12, 2020 at 08:10:09PM +0100, Uladzislau Rezki wrote:
> > > On Thu, Mar 12, 2020 at 11:11:38AM -0700, Paul E. McKenney wrote:
> > > > On Thu, Mar 12, 2020 at 04:27:30PM +0000, George Spelvin wrote:
> > > > > kvfree() is a superset of kfree(), so there's nothing obvious stopping 
> > > > > kfree_rcu() from simply changing to kvfree() and everything will keep 
> > > > > working.
> > > > > 
> > > > > I'd probably add a kvfree_rcu() alias, just for documentation's sake and
> > > > > to make code that depends on the new feature explode at compile time, but
> > > > > it could be identical behind the scenes.
> > > > > 
> > > > > There's an existing user in mm/list_lru.c already.
> > > > > 
> > > > > I was just thinking of using kvmalloc() in a module, and realized that the 
> > > > > lack of a core kvfree_rcu() helper meant I'd have to synchronize_rcu() on 
> > > > > module unload.
> > > > 
> > > > There was a recent proposal to do just that, but current patches in -rcu
> > > > use kfree_bulk().  It doesn't look to me that this works for kfvree()
> > > > under the covers in its current form.  Could it be upgraded to handle
> > > > this case?
> > > > 
> > > > Adding Vlad on CC for his thoughts.
> > > > 
> > > Paul, see below my view:
> > > 
> > > Answering to topic's question it looks like we need kvfree_rcu() support :)
> > > 
> > > It is easy to add it actually. But if we are talking about the case when
> > > an object has rcu_head inside. From the other hand recent discussion showed
> > > that we would like to have head-less variant of the kvfree_rcu() functionality:
> > > 
> > > https://lkml.org/lkml/2020/2/18/566
> > > 
> > > for example, as Ted pointed, he would go with head-less case(for ext4) only.
> > > The reason is nobody wants to modify internal structures injecting rcu_head
> > > there. Also there are many other places in the kernel where it would be good
> > > to have kfree_rcu() head-less variant as well.
> > > 
> > > I spent some time implementing it together with Joel. It is ready from my
> > > side but only for RCU-tree case. Next step is RCU-tiny support, so i am
> > > working on it.  
> > > 
> > > I can send out an RFC for RCU-tree only support, so we can discuss it
> > > and agree on how to move forward. After that i or Joel or together can
> > > update RCU-tine.
> > > 
> > > Joel: What do you think?
> > 
> > Yes, your sending an RFC with what you have sounds good. I can prepare a tree
> > for both of us then and we can develop on that. I was actually waiting on
> > your patches so I can add more on top.
> > 
> OK. Let's setup something. Currently i am working based on dev.2020.02.16a
> branch. If you can, please do it and let me know so we can avoid of conflicts.

Ok, I based the kfree_rcu shrinker patches on an 'rcu/kfree' branch in my git
tree: https://github.com/joelagnel/linux-kernel/tree/rcu/kfree

I will apply your patches to this branch once I have them.

> > One more thing I want to add is the shrinker interface to prevent OOM during
> > kfree_rcu() flood. I sent patches to fix that. It works well. We can prepare
> > a tree with all these features and develop on that so there's no conflict.
> > 
> > For -tiny and lack of rcu_head, I think we discussed that we would
> > always dynamically allocate rcu_head for that case.
> > 
> We can do dynamic attaching. Later on, i think we can implement "arrays" logic
> like we do for RCU-tree but without any batching. Simple array pointer store  
> mechanism for head-less case.

Yep, sounds good.

> > > Another thought. We can add kvfree_rcu(ptr, rcu) first, because it is
> > > easy and after that implement head-less case.
> > 
> > Yes, that is also fine. We can start simple and then keep improving it. I
> > think we have now 3 users who want head-less interface so ultimately we can
> > shoot for that goal (at later stage).
> > 
> OK. Then, i think i will make kvfree_rcu(ptr, rcu) interface and all the
> rest on top.

Ok, sounds good.

thanks,

 - Joel

> 
> --
> Vlad Rezki
>