From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2781C77B60 for ; Thu, 30 Mar 2023 19:19:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231514AbjC3TTk (ORCPT ); Thu, 30 Mar 2023 15:19:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47998 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232076AbjC3TTK (ORCPT ); Thu, 30 Mar 2023 15:19:10 -0400 Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D55FA11158; Thu, 30 Mar 2023 12:18:49 -0700 (PDT) Received: by mail-ed1-x529.google.com with SMTP id cn12so80690560edb.4; Thu, 30 Mar 2023 12:18:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680203928; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:date:from:from:to :cc:subject:date:message-id:reply-to; bh=4q03OZ6Bg+GlvpSvY95+tuvETzAF1mWv6O6MJqkSOes=; b=O28g3NA8Tb2ES1tEbLmVheLCFjUEgDzVflxrsrNJwK1AdjLkgJjOE/tVKCDdaxXwZp WCCzlOKVnTYxDR0HmhVnIXFreE08BxTQJmc+dawdjtYyiD4iiYk78b654B4auVB/dslq 7DG48ByHnzkwesNqh7Ya3zL1QwK5gLt5lZ2AT4OfGaN5TDHv7qKi/n7YatPRh2BU2CrN GSbJhL5cpyaywuayFC9N8fb9+ordoHFCZE8D/VxBG46PcFNWtnrviOvEOklA0b34kLN7 e48ViFQuhHD7bJ/psalH7wvJXbbkEiZO89lq0Xrw9rA9bmkJTqRF1wb8dQFjNsR1oD52 iNJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680203928; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:date:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=4q03OZ6Bg+GlvpSvY95+tuvETzAF1mWv6O6MJqkSOes=; b=rPtsllWO/vvkjOizn2uAp4v5Ty05fvWc2ba7PqU8xN3KW+x9YT/YB40LrSRt84KCHV w+AWNhjHxnPVXx9xTrxxFaAuv8A4hTabgGimkdj9eXV2kv5xjbPt0KfGO5drAccf40Qh 1vifHAesTdLfXGd5llFaNCe5zXOhxTlLufTMwOj5xPgFwjmSlcbSMJmW+F7aF0SiErUZ e1vnzUGitZZYN+DPtfSrjlGvVl2ICZ0WUTVW/YFBzSLtql8UQJ01BE5GNr8ZXNR8d38n nDoHy4f2VzznTjglwzhM0VjScu+zIO6ZpgjPkD0BjyFdOY6ir7Maj5LA9kBcwrrMm+TW 8iFg== X-Gm-Message-State: AAQBX9eB97ptDTicu8cgiufnHBA9FL4/zZYsprjItnlwKb2RXHIkwUex fAnSgoLpXkqOnVeRqOX43rs= X-Google-Smtp-Source: AKy350ZdUzEUad/4w3uOLqChw1DiPg/T2noGNa6IvE9m7OI0oF76V4MwVBeh2pIG9pR/FXdp1KIMnQ== X-Received: by 2002:a17:907:9d1a:b0:92e:eecf:b742 with SMTP id kt26-20020a1709079d1a00b0092eeecfb742mr24362906ejc.2.1680203927685; Thu, 30 Mar 2023 12:18:47 -0700 (PDT) Received: from pc636 ([155.137.26.201]) by smtp.gmail.com with ESMTPSA id gx20-20020a1709068a5400b00931faf03db0sm139005ejc.27.2023.03.30.12.18.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Mar 2023 12:18:47 -0700 (PDT) From: Uladzislau Rezki X-Google-Original-From: Uladzislau Rezki Date: Thu, 30 Mar 2023 21:18:44 +0200 To: "Paul E. McKenney" Cc: Uladzislau Rezki , Joel Fernandes , "Zhang, Qiang1" , "Zhuo, Qiuxu" , RCU , quic_neeraju@quicinc.com, Boqun Feng , LKML , Oleksiy Avramchenko , Steven Rostedt , Frederic Weisbecker Subject: Re: [PATCH 1/1] Reduce synchronize_rcu() waiting time Message-ID: References: <2cd8f407-2b77-48b1-9f17-9aa8e4ce9c64@paulmck-laptop> <20230330150933.GB2114899@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Mar 30, 2023 at 11:58:41AM -0700, Paul E. McKenney wrote: > On Thu, Mar 30, 2023 at 05:43:15PM +0200, Uladzislau Rezki wrote: > > On Thu, Mar 30, 2023 at 03:09:33PM +0000, Joel Fernandes wrote: > > > On Tue, Mar 28, 2023 at 08:26:13AM -0700, Paul E. McKenney wrote: > > > > On Mon, Mar 27, 2023 at 10:29:31PM -0400, Joel Fernandes wrote: > > > > > Hello, > > > > > > > > > > > On Mar 27, 2023, at 9:06 PM, Paul E. McKenney wrote: > > > > > > > > > > > > On Mon, Mar 27, 2023 at 11:21:23AM +0000, Zhang, Qiang1 wrote: > > > > > >>>> From: Uladzislau Rezki (Sony) > > > > > >>>> Sent: Tuesday, March 21, 2023 6:28 PM > > > > > >>>> [...] > > > > > >>>> Subject: [PATCH 1/1] Reduce synchronize_rcu() waiting time > > > > > >>>> > > > > > >>>> A call to a synchronize_rcu() can be expensive from time point of view. > > > > > >>>> Different workloads can be affected by this especially the ones which use this > > > > > >>>> API in its time critical sections. > > > > > >>>> > > > > > >>> > > > > > >>> This is interesting and meaningful research. ;-) > > > > > >>> > > > > > >>>> For example in case of NOCB scenario the wakeme_after_rcu() callback > > > > > >>>> invocation depends on where in a nocb-list it is located. Below is an example > > > > > >>>> when it was the last out of ~3600 callbacks: > > > > > >>> > > > > > >> > > > > > >> > > > > > >> > > > > > >> Can it be implemented separately as follows? it seems that the code is simpler > > > > > >> (only personal opinion) 😊. > > > > > >> > > > > > >> But I didn't test whether this reduce synchronize_rcu() waiting time > > > > > >> > > > > > >> +static void rcu_poll_wait_gp(struct rcu_tasks *rtp) > > > > > >> +{ > > > > > >> + unsigned long gp_snap; > > > > > >> + > > > > > >> + gp_snap = start_poll_synchronize_rcu(); > > > > > >> + while (!poll_state_synchronize_rcu(gp_snap)) > > > > > >> + schedule_timeout_idle(1); > > > > > > > > > > > > I could be wrong, but my guess is that the guys working with > > > > > > battery-powered devices are not going to be very happy with this loop. > > > > > > > > > > > > All those wakeups by all tasks waiting for a grace period end up > > > > > > consuming a surprisingly large amount of energy. > > > > > > > > > > Is that really the common case? On the general topic of wake-ups: > > > > > Most of the time there should be only one > > > > > task waiting synchronously on a GP to end. If that is > > > > > true, then it feels like waking > > > > > up nocb Kthreads which indirectly wake other threads is doing more work than usual? > > > > > > > > A good question, and the number of outstanding synchronize_rcu() > > > > calls will of course be limited by the number of tasks in the system. > > > > But I myself have raised the ire of battery-powered embedded folks with > > > > a rather small number of wakeups, so... > > > > > > But unless I am missing something, even if there is single synchronize_rcu(), > > > you have a flurry of potential wakeups right now, instead of the bare minimum > > > I think. I have not measured how many wake ups, but I'd love to when I get > > > time. Maybe Vlad has some numbers. > > > > > I will measure and have a look at wake-ups. But, what we have for now is > > if there are two callers of synchronize_rcu() on different CPUs, i guess > > two nocb-kthreads have to handle it, thus two nocb-kthreads have to be > > awaken to do the work. This patch needs only one wake-up to serve all > > users. > > One wakeup per synchronize_rcu(), right? > The gp-kthread wake-ups only one work, in its turn a worker wake-ups all registered users of synchronize_rcu() for which a gp was passed. How many users of synchonize_rcu() awaken by one worker depends on how many were registered before initiating a new GP by the gp-kthread. > > Anyway, i will provide some data and analysis of it. > > Looking forward to seeing it! > Good. I will switch fully on it soon. I need to sort out some perf. issues at work. -- Uladzislau Rezki