From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751258AbdEPHB0 (ORCPT ); Tue, 16 May 2017 03:01:26 -0400 Received: from LGEAMRELO13.lge.com ([156.147.23.53]:52830 "EHLO lgeamrelo13.lge.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750832AbdEPHBY (ORCPT ); Tue, 16 May 2017 03:01:24 -0400 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: byungchul.park@lge.com X-Original-SENDERIP: 165.244.249.26 X-Original-MAILFROM: byungchul.park@lge.com X-Original-SENDERIP: 10.177.222.33 X-Original-MAILFROM: byungchul.park@lge.com Date: Tue, 16 May 2017 16:00:26 +0900 From: Byungchul Park To: Juri Lelli CC: Steven Rostedt , , , , , , Subject: Re: [PATCH v4 1/5] sched/deadline: Refer to cpudl.elements atomically Message-ID: <20170516070026.GB24127@X58A-UD3R> References: <1494568129-9985-1-git-send-email-byungchul.park@lge.com> <1494568129-9985-2-git-send-email-byungchul.park@lge.com> <20170512102530.50b85979@gandalf.local.home> <20170515083629.kpowe7tcbnfvg6wk@e106622-lin> MIME-Version: 1.0 In-Reply-To: <20170515083629.kpowe7tcbnfvg6wk@e106622-lin> User-Agent: Mutt/1.5.21 (2010-09-15) X-MIMETrack: Itemize by SMTP Server on LGEKRMHUB03/LGE/LG Group(Release 8.5.3FP6|November 21, 2013) at 2017/05/16 16:00:33, Serialize by Router on LGEKRMHUB03/LGE/LG Group(Release 8.5.3FP6|November 21, 2013) at 2017/05/16 16:00:33, Serialize complete at 2017/05/16 16:00:33 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, May 15, 2017 at 09:36:29AM +0100, Juri Lelli wrote: > Hi, > > On 12/05/17 10:25, Steven Rostedt wrote: > > On Fri, 12 May 2017 14:48:45 +0900 > > Byungchul Park wrote: > > > > > cpudl.elements is an instance that should be protected with a spin lock. > > > Without it, the code would be insane. > > > > And how much contention will this add? Spin locks in the scheduler code > > that are shared among a domain can cause huge latency. This was why I > > worked hard not to add any in the cpupri code. > > > > > > > > > > Current cpudl_find() has problems like, > > > > > > 1. cpudl.elements[0].cpu might not match with cpudl.elements[0].dl. > > > 2. cpudl.elements[0].dl(u64) might not be referred atomically. > > > 3. Two cpudl_maximum()s might return different values. > > > 4. It's just insane. > > > > And lockless algorithms usually are insane. But locks come with a huge > > cost. What happens when we have 32 core domains. This can cause > > tremendous contention and makes the entire cpu priority for deadlines > > useless. Might as well rip out the code. > > > > Right. So, rationale for not taking any lock in the find() path (at the > risk of getting bogus values) is that we don't want to pay to much in > terms of contention, when also considering the fact that find_lock_later_ > rq() might then release the rq lock, possibly making the search useless > (if things change in the meantime anyway). The update path is instead > guarded by a lock, to ensure consistency. > > Experiments on reasonably big machines (48-cores IIRC) showed that the > approach was "good enough", so we looked somewhere else to improve > things (as there are many to improve :). This of course doesn't prevent > us to look at this again now and see if we need to do something about it. > > Having numbers about introduced overhead and wrong decisions caused by > the lockless find() path would help a lot understanding what (and can) > be done. I see what you say. Agree.. Hm.. Before that, what do you think about my suggestions in my reply to steven? > > Thanks! > > - Juri