Re: [patch] cpufreq: mark cpufreq_tsc() as core_initcall_sync

From: Oleg Nesterov <oleg@tv-sign.ru>
To: Jens Axboe <jens.axboe@oracle.com>
Cc: "Paul E. McKenney" <paulmck@us.ibm.com>,
	Alan Stern <stern@rowland.harvard.edu>,
	Linus Torvalds <torvalds@osdl.org>,
	Thomas Gleixner <tglx@timesys.com>, Ingo Molnar <mingo@elte.hu>,
	LKML <linux-kernel@vger.kernel.org>,
	john stultz <johnstul@us.ibm.com>,
	David Miller <davem@davemloft.net>,
	Arjan van de Ven <arjan@infradead.org>,
	Andrew Morton <akpm@osdl.org>, Andi Kleen <ak@suse.de>,
	manfred@colorfullife.com
Subject: Re: [patch] cpufreq: mark cpufreq_tsc() as core_initcall_sync
Date: Fri, 17 Nov 2006 21:39:45 +0300	[thread overview]
Message-ID: <20061117183945.GA367@oleg> (raw)
In-Reply-To: <20061117092925.GT7164@kernel.dk>

Paul E. McKenney wrote:
>
>  int srcu_read_lock(struct srcu_struct *sp)
>  {
> @@ -112,11 +126,24 @@ int srcu_read_lock(struct srcu_struct *s
>  
>  	preempt_disable();
>  	idx = sp->completed & 0x1;
> -	barrier();  /* ensure compiler looks -once- at sp->completed. */
> -	per_cpu_ptr(sp->per_cpu_ref, smp_processor_id())->c[idx]++;
> -	srcu_barrier();  /* ensure compiler won't misorder critical section. */
> +	if (likely(sp->per_cpu_ref != NULL)) {
> +		barrier();  /* ensure compiler looks -once- at sp->completed. */
> +		per_cpu_ptr(rcu_dereference(sp->per_cpu_ref),
> +			    smp_processor_id())->c[idx]++;
> +		smp_mb();
> +		preempt_enable();
> +		return idx;
> +	}
>  	preempt_enable();
> -	return idx;
> +	mutex_lock(&sp->mutex);
> +	sp->per_cpu_ref = alloc_srcu_struct_percpu();

We should re-check sp->per_cpu_ref != NULL after taking sp->mutex,
it was probably allocated by another thread.

>  void srcu_read_unlock(struct srcu_struct *sp, int idx)
>  {
> -	preempt_disable();
> -	srcu_barrier();  /* ensure compiler won't misorder critical section. */
> -	per_cpu_ptr(sp->per_cpu_ref, smp_processor_id())->c[idx]--;
> -	preempt_enable();
> +	if (likely(idx != -1)) {
> +		preempt_disable();
> +		smp_mb();
> +		per_cpu_ptr(sp->per_cpu_ref, smp_processor_id())->c[idx]--;
> +		preempt_enable();
> +		return;
> +	}
> +	mutex_lock(&sp->mutex);
> +	sp->hardluckref--;
> +	mutex_unlock(&sp->mutex);
>  }

I think this is deadlockable, synchronize_srcu() does

  	while (srcu_readers_active_idx(sp, idx))
  		schedule_timeout_interruptible(1);

under sp->mutex, so the loop above may spin forever while the reader
waits for sp->mutex in srcu_read_unlock(sp, -1).

Oleg.