From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760189AbdEOIgf (ORCPT ); Mon, 15 May 2017 04:36:35 -0400 Received: from foss.arm.com ([217.140.101.70]:48486 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760065AbdEOIge (ORCPT ); Mon, 15 May 2017 04:36:34 -0400 Date: Mon, 15 May 2017 09:36:29 +0100 From: Juri Lelli To: Steven Rostedt Cc: Byungchul Park , peterz@infradead.org, mingo@kernel.org, linux-kernel@vger.kernel.org, juri.lelli@gmail.com, bristot@redhat.com, kernel-team@lge.com Subject: Re: [PATCH v4 1/5] sched/deadline: Refer to cpudl.elements atomically Message-ID: <20170515083629.kpowe7tcbnfvg6wk@e106622-lin> References: <1494568129-9985-1-git-send-email-byungchul.park@lge.com> <1494568129-9985-2-git-send-email-byungchul.park@lge.com> <20170512102530.50b85979@gandalf.local.home> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170512102530.50b85979@gandalf.local.home> User-Agent: NeoMutt/20170113 (1.7.2) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi, On 12/05/17 10:25, Steven Rostedt wrote: > On Fri, 12 May 2017 14:48:45 +0900 > Byungchul Park wrote: > > > cpudl.elements is an instance that should be protected with a spin lock. > > Without it, the code would be insane. > > And how much contention will this add? Spin locks in the scheduler code > that are shared among a domain can cause huge latency. This was why I > worked hard not to add any in the cpupri code. > > > > > > Current cpudl_find() has problems like, > > > > 1. cpudl.elements[0].cpu might not match with cpudl.elements[0].dl. > > 2. cpudl.elements[0].dl(u64) might not be referred atomically. > > 3. Two cpudl_maximum()s might return different values. > > 4. It's just insane. > > And lockless algorithms usually are insane. But locks come with a huge > cost. What happens when we have 32 core domains. This can cause > tremendous contention and makes the entire cpu priority for deadlines > useless. Might as well rip out the code. > Right. So, rationale for not taking any lock in the find() path (at the risk of getting bogus values) is that we don't want to pay to much in terms of contention, when also considering the fact that find_lock_later_ rq() might then release the rq lock, possibly making the search useless (if things change in the meantime anyway). The update path is instead guarded by a lock, to ensure consistency. Experiments on reasonably big machines (48-cores IIRC) showed that the approach was "good enough", so we looked somewhere else to improve things (as there are many to improve :). This of course doesn't prevent us to look at this again now and see if we need to do something about it. Having numbers about introduced overhead and wrong decisions caused by the lockless find() path would help a lot understanding what (and can) be done. Thanks! - Juri