smp_processor_id used in preemptable context ?

smp_processor_id used in preemptable context ?

[Vincent Ray]

[-- Attachment #1.1: Type: text/plain, Size: 925 bytes --]

Hi Linux hackers ! 

I'm reading some code in net/core/dev.c and something puzzles me : somewhere in __dev_queue_xmit, we have : 
int cpu = smp_processor_id(); /* ok because BHs are off */ 

and, indeed, a few lines up there is : 
/* Disable soft irqs for various locks below. Also 
* stops preemption for RCU. 
*/ 
rcu_read_lock_bh(); 

Now I'm wondering : is this really ok ? 
From what I understand, this code can very well run in user context, with hard IRQs ON, and in fact it should, according to the following comment : 

* When calling this method, interrupts MUST be enabled. This is because 
* the BH enable code must have IRQs enabled so that it will not deadlock. 
* --BLG 

Then I guess it could be preempted at any time, especially with aggressive versions of preemptions ? 
And if so, are we not at risk that our thread is migrated to an other CPU just after smp_processor_id returned ? 

Cheers, 

V Ray 





[-- Attachment #1.2: Type: text/html, Size: 1573 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: smp_processor_id used in preemptable context ?

[Valdis Klētnieks]

On Thu, 28 Apr 2022 11:15:27 +0200, Vincent Ray said:
> Then I guess it could be preempted at any time, especially with aggressive versions of preemptions ? 
> And if so, are we not at risk that our thread is migrated to an other CPU just after smp_processor_id returned ? 

Often, we don't actually *care* if it gets migrated.  A *lot* of uses of
smp_processor_id() are just to make statistic gathering more efficient.

Rather than all the CPUs do all sorts of locking to avoid race conditions
between different threads updating the same variable, and avoid massive cache
line ping-ponging, the code just gets a pointer to a per_cpu area where we
update some statistic or counter without worrying about that stuff because we
know no other processor should be updating *this* processor's per_cpu area.
Then anything that cares about the *total* across all CPUs can iterate across
all the per_cpu numbers - and that (a) usually happens much less frequently
than updates and (b) only needs to read all the per_cpu area without updating
them.

Yes, there's a very small chance that the "wrong" CPU's stats will be updated. But
let's be realistic - for a lot of statistics, you'll never notice. 

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: smp_processor_id used in preemptable context ?

[Vincent Ray]

Thanks for your answer Valdis.

So, if I understand well :

1) the comment in net/core/dev.c:__dev_queue_xmit is indeed misleading :

		int cpu = smp_processor_id(); /* ok because BHs are off */

as there is a small chance that we get the wrong info. Disabled BHs are not enough. 
Disabling preemption would be needed to be 100% correct. 
You agree ?

2) You're saying that, most of the time it's ok, as, at worst, we will be a little wrong on some per-cpu stats.
I'm sure it does not matter if cpu0 gets 45 'events' instead of 46, and cpu1 gets 1 extra for free, but that's only if atomic_inc
or something similar is used. If not, can't we end up with completely broken values, with e.g. cpu0 more or less writing its own count, 
totally different from that of cpu1, to cpu1's area ?

3) Finally it looks to me that this particular usage of smp_processor_id() in __dev_queue_xmit() 
is more about preventing a dead loop than just gathering statistics.

Although I don't understand everything going on here, it looks like cpu id is directly used to detect
a bad scenario, leading to a critical message :

	if (dev->flags & IFF_UP) {
		int cpu = smp_processor_id(); /* ok because BHs are off */

		/* Other cpus might concurrently change txq->xmit_lock_owner
		 * to -1 or to their cpu id, but not to our id.
		 */
		if (READ_ONCE(txq->xmit_lock_owner) != cpu) {
[...]
		} else {
			/* Recursion is detected! It is possible,
			 * unfortunately
			 */
recursion_alert:
			net_crit_ratelimited("Dead loop on virtual device %s, fix it urgently!\n",
					     dev->name);

So if our thread is migrated just after smp_processor_id() and lands on the cpu occupied by the thread that is the txq->xmit_lock_owner,
itself just scheduled out, can't we go to the net_crit_ratelimited() ?


Thanks,

V






_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies