Re: [PATCH v2 09/16] rcu/tree: Maintain separate array for vmalloc ptrs

[Uladzislau Rezki <urezki@gmail.com>]

On Thu, Jun 18, 2020 at 10:32:06AM -0700, Paul E. McKenney wrote:
> On Thu, Jun 18, 2020 at 07:25:04PM +0200, Uladzislau Rezki wrote:
> > > > +	// Handle two first channels.
> > > > +	for (i = 0; i < FREE_N_CHANNELS; i++) {
> > > > +		for (; bkvhead[i]; bkvhead[i] = bnext) {
> > > > +			bnext = bkvhead[i]->next;
> > > > +			debug_rcu_bhead_unqueue(bkvhead[i]);
> > > > +
> > > > +			rcu_lock_acquire(&rcu_callback_map);
> > > > +			if (i == 0) { // kmalloc() / kfree().
> > > > +				trace_rcu_invoke_kfree_bulk_callback(
> > > > +					rcu_state.name, bkvhead[i]->nr_records,
> > > > +					bkvhead[i]->records);
> > > > +
> > > > +				kfree_bulk(bkvhead[i]->nr_records,
> > > > +					bkvhead[i]->records);
> > > > +			} else { // vmalloc() / vfree().
> > > > +				for (j = 0; j < bkvhead[i]->nr_records; j++) {
> > > > +					trace_rcu_invoke_kfree_callback(
> > > > +						rcu_state.name,
> > > > +						bkvhead[i]->records[j], 0);
> > > > +
> > > > +					vfree(bkvhead[i]->records[j]);
> > > > +				}
> > > > +			}
> > > > +			rcu_lock_release(&rcu_callback_map);
> > > 
> > > Not an emergency, but did you look into replacing this "if" statement
> > > with an array of pointers to functions implementing the legs of the
> > > "if" statement?  If nothing else, this would greatly reduced indentation.
> > > 
> > >
> > > I am taking this as is, but if you have not already done so, could you
> > > please look into this for a follow-up patch?
> > > 
> > I do not think it makes sense, because it would require to check each
> > pointer in the array, what can lead to many branching, i.e. "if-else"
> > instructions.
> 
> Mightn't the compiler simply unroll the outer loop?  Then the first
> unrolled iteration of that loop would contain the then-clause and
> the second unrolled iteration would contain the else-clause.  At that
> point, there would be no checking, just direct calls.
> 
> Or am I missing something?
> 
If we mix pointers, then we can do free per pointer only. I mean in that
case we will not be able to use kfree_bulk() interface for freeing SLAB
memory and the code would converted to something like:

<snip>
while (nr_objects_in_array > 0) {
    if (is_vmalloc_addr(array[X]))
       vfree(array[X]);
    else
       kfree(array[X]);
}
<snip>

> > Paul, thank you to take it in!
> 
> Thank you for persisting!
> 
Welcome :)

--
Vlad Rezki