From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753537AbbBTFH6 (ORCPT ); Fri, 20 Feb 2015 00:07:58 -0500 Received: from mail-ie0-f170.google.com ([209.85.223.170]:44235 "EHLO mail-ie0-f170.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751019AbbBTFH4 (ORCPT ); Fri, 20 Feb 2015 00:07:56 -0500 MIME-Version: 1.0 In-Reply-To: <20150218151352.2968cf06@grimm.local.home> References: <20140409024700.702797305@goodmis.org> <20140409025231.998774075@goodmis.org> <20150218195710.GH28763@linutronix.de> <20150218151352.2968cf06@grimm.local.home> Date: Thu, 19 Feb 2015 21:07:56 -0800 X-Google-Sender-Auth: 1Ml2-48xCvbcf0uqDV_6HkZKuhM Message-ID: Subject: Re: [PATCH RT 1/2] rwsem-rt: Do not allow readers to nest From: Jason Low To: Steven Rostedt Cc: Sebastian Andrzej Siewior , Linux Kernel Mailing List , linux-rt-users , Thomas Gleixner , Carsten Emde , John Kacur , Paul Gortmaker , Peter Zijlstra , "H. Peter Anvin" , Jason Low , juerg.haefliger@hp.com Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Feb 18, 2015 at 12:13 PM, Steven Rostedt wrote: > On Wed, 18 Feb 2015 20:57:10 +0100 > Sebastian Andrzej Siewior wrote: > >> * Steven Rostedt | 2014-04-08 22:47:01 [-0400]: >> >> >From: "Steven Rostedt (Red Hat)" >> > >> >The readers of mainline rwsems are not allowed to nest, the rwsems in the >> >PREEMPT_RT kernel should not nest either. >> >> I applied this and this is the reason why cpufreq isn't working. What I >> see in cpufreq is: >> | test.sh-788 [004] ....... 61.416288: store: down_read_try >> | test.sh-788 [004] ....... 61.416296: cpufreq_cpu_get: down_read_try >> | test.sh-788 [004] ....... 61.416301: cpufreq_cpu_put.part.6: up_read >> | test.sh-788 [004] ....... 61.416332: store: up_read >> >> as you see, one code path takes the read path of rw_sema twice. >> >> Looking at the generic implementation, we have: >> |#define RWSEM_UNLOCKED_VALUE 0x00000000L >> |#define RWSEM_ACTIVE_BIAS 0x00000001L >> |#define RWSEM_WAITING_BIAS (-RWSEM_ACTIVE_MASK-1) >> >> | static inline int __down_read_trylock(struct rw_semaphore *sem) >> | { >> | long tmp; >> | >> | while ((tmp = sem->count) >= 0) { >> | if (tmp == cmpxchg(&sem->count, tmp, >> | tmp + RWSEM_ACTIVE_READ_BIAS)) { >> | return 1; >> | } >> | } >> | return 0; >> | } >> >> While sem->count is >= 0 we loop and take the semaphore. So we can have >> five readers at once. The first writer would set count to a negative >> value resulting in trylock failure. >> >> |static inline void __down_read(struct rw_semaphore *sem) >> |{ >> | if (unlikely(atomic_long_inc_return((atomic_long_t*)&sem->count) <= 0)) >> | rwsem_down_read_failed(sem); >> |} >> >> Here the same thing but without cmpxchg(). _If_ after an increment the >> value is negative then we take slowpath. Otherwise we have the lock. > > OK, so I need to make it so it can nest with trylock. I have to look at > the patch again because it has been a while. When we reported this a few months ago, Thomas provided the following patch to fix the issue (which essentially reverted the patch) and appeared to be agreed on: https://lkml.org/lkml/2014/11/5/844