From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2A7BC6FD1D for ; Thu, 30 Mar 2023 15:44:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233240AbjC3Pog (ORCPT ); Thu, 30 Mar 2023 11:44:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55998 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233179AbjC3Pof (ORCPT ); Thu, 30 Mar 2023 11:44:35 -0400 Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com [IPv6:2a00:1450:4864:20::12c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC579CA3D; Thu, 30 Mar 2023 08:43:45 -0700 (PDT) Received: by mail-lf1-x12c.google.com with SMTP id k37so25132347lfv.0; Thu, 30 Mar 2023 08:43:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680190998; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:date:from:from:to :cc:subject:date:message-id:reply-to; bh=K+snN28zGpqdGdh0F8mLob3cT7QxGO9Mm/LKoyV1UaQ=; b=gagKM4RTkR+kg03jLaMHmV8lXtW8Fy4u6OS1fiDqgdrX4jV/HJqnlFRQZ86miITzAP TizipNShN1NIZPKhOHdDvII94p0PFcZMcxVRH8+IxXZJD91N0VCFV4qNpMYDRMnXdmJV L+30Kz4hIap3S36Jk7WwNgI717IjtSqc9Cj7IK6aBBP1V2XhnTvuzdSylkWKGMnyGl1L fRrQrLKuQ69EegT5nEgWraOcTVBJMeD3rxBgMlyEbmy4JkCv02N/HNcVanc8e12kG11w c9OduTlTMmdeYoslWRTgUv63iRVcwaDBKtozFqqvQMBrbbqt9k9DWn1oMWItHLK7kqx5 Xw7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680190998; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:date:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=K+snN28zGpqdGdh0F8mLob3cT7QxGO9Mm/LKoyV1UaQ=; b=yJlZqCwb5g6Wn7Q1xEEJRIv4G+m5yHB283qf7ZdsH+w5keBun4OhK3fXZHJp26f2Nt zAqQFNLl70CGF2f4AyWzmh1vFtYGO7dnAKqGFQG6HpIdvK8JvZzK2dCy+pzLSUPbamCZ 09OvQRovuUogEjvYz9C1eH/RAvGMjwHXYR6y4cMD0420/1zP5myP0G3Zfndl9rH4ZfUP JrDTrDwaAMIc76Ohz6j9Bfch7UmqE6iTL1uJXcSVu9WsesfXwX3vefM7+PuCCiCQEJWv XbmNby7uHvwIub/kcXEqPi358yuctnF8o6Pc7i5mLnY5HcvEPd91sE2DEOZSBsNy2zdE vNAw== X-Gm-Message-State: AAQBX9eB+ONvSD8GQnPTbp8kicRZMjEkhFv1j31Y9VNUcszSD6BcwYLx 5cy6TSeukZ8u40O3ciAr7t0= X-Google-Smtp-Source: AKy350YWo9ixOeCQIogabUOy4+LcErTfIgi6CAX9/qN9OKu0g/kQngikDEc8YgB6bBXuVsDRuI1fHA== X-Received: by 2002:ac2:5e88:0:b0:4eb:c4e:bd87 with SMTP id b8-20020ac25e88000000b004eb0c4ebd87mr5439021lfq.58.1680190998199; Thu, 30 Mar 2023 08:43:18 -0700 (PDT) Received: from pc636 (host-90-233-209-50.mobileonline.telia.com. [90.233.209.50]) by smtp.gmail.com with ESMTPSA id w6-20020a2e9986000000b00290679ebac1sm5962840lji.9.2023.03.30.08.43.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Mar 2023 08:43:17 -0700 (PDT) From: Uladzislau Rezki X-Google-Original-From: Uladzislau Rezki Date: Thu, 30 Mar 2023 17:43:15 +0200 To: Joel Fernandes Cc: "Paul E. McKenney" , "Zhang, Qiang1" , Uladzislau Rezki , "Zhuo, Qiuxu" , RCU , quic_neeraju@quicinc.com, Boqun Feng , LKML , Oleksiy Avramchenko , Steven Rostedt , Frederic Weisbecker Subject: Re: [PATCH 1/1] Reduce synchronize_rcu() waiting time Message-ID: References: <2cd8f407-2b77-48b1-9f17-9aa8e4ce9c64@paulmck-laptop> <20230330150933.GB2114899@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20230330150933.GB2114899@google.com> Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org On Thu, Mar 30, 2023 at 03:09:33PM +0000, Joel Fernandes wrote: > On Tue, Mar 28, 2023 at 08:26:13AM -0700, Paul E. McKenney wrote: > > On Mon, Mar 27, 2023 at 10:29:31PM -0400, Joel Fernandes wrote: > > > Hello, > > > > > > > On Mar 27, 2023, at 9:06 PM, Paul E. McKenney wrote: > > > > > > > > On Mon, Mar 27, 2023 at 11:21:23AM +0000, Zhang, Qiang1 wrote: > > > >>>> From: Uladzislau Rezki (Sony) > > > >>>> Sent: Tuesday, March 21, 2023 6:28 PM > > > >>>> [...] > > > >>>> Subject: [PATCH 1/1] Reduce synchronize_rcu() waiting time > > > >>>> > > > >>>> A call to a synchronize_rcu() can be expensive from time point of view. > > > >>>> Different workloads can be affected by this especially the ones which use this > > > >>>> API in its time critical sections. > > > >>>> > > > >>> > > > >>> This is interesting and meaningful research. ;-) > > > >>> > > > >>>> For example in case of NOCB scenario the wakeme_after_rcu() callback > > > >>>> invocation depends on where in a nocb-list it is located. Below is an example > > > >>>> when it was the last out of ~3600 callbacks: > > > >>> > > > >> > > > >> > > > >> > > > >> Can it be implemented separately as follows? it seems that the code is simpler > > > >> (only personal opinion) 😊. > > > >> > > > >> But I didn't test whether this reduce synchronize_rcu() waiting time > > > >> > > > >> +static void rcu_poll_wait_gp(struct rcu_tasks *rtp) > > > >> +{ > > > >> + unsigned long gp_snap; > > > >> + > > > >> + gp_snap = start_poll_synchronize_rcu(); > > > >> + while (!poll_state_synchronize_rcu(gp_snap)) > > > >> + schedule_timeout_idle(1); > > > > > > > > I could be wrong, but my guess is that the guys working with > > > > battery-powered devices are not going to be very happy with this loop. > > > > > > > > All those wakeups by all tasks waiting for a grace period end up > > > > consuming a surprisingly large amount of energy. > > > > > > Is that really the common case? On the general topic of wake-ups: > > > Most of the time there should be only one > > > task waiting synchronously on a GP to end. If that is > > > true, then it feels like waking > > > up nocb Kthreads which indirectly wake other threads is doing more work than usual? > > > > A good question, and the number of outstanding synchronize_rcu() > > calls will of course be limited by the number of tasks in the system. > > But I myself have raised the ire of battery-powered embedded folks with > > a rather small number of wakeups, so... > > But unless I am missing something, even if there is single synchronize_rcu(), > you have a flurry of potential wakeups right now, instead of the bare minimum > I think. I have not measured how many wake ups, but I'd love to when I get > time. Maybe Vlad has some numbers. > I will measure and have a look at wake-ups. But, what we have for now is if there are two callers of synchronize_rcu() on different CPUs, i guess two nocb-kthreads have to handle it, thus two nocb-kthreads have to be awaken to do the work. This patch needs only one wake-up to serve all users. Anyway, i will provide some data and analysis of it. -- Uladzislau Rezki