From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.4 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5CB21C3A5A6 for ; Fri, 30 Aug 2019 01:20:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 255CA21897 for ; Fri, 30 Aug 2019 01:20:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=joelfernandes.org header.i=@joelfernandes.org header.b="KiQBSMSh" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727222AbfH3BUj (ORCPT ); Thu, 29 Aug 2019 21:20:39 -0400 Received: from mail-pf1-f195.google.com ([209.85.210.195]:45757 "EHLO mail-pf1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727066AbfH3BUj (ORCPT ); Thu, 29 Aug 2019 21:20:39 -0400 Received: by mail-pf1-f195.google.com with SMTP id w26so3373934pfq.12 for ; Thu, 29 Aug 2019 18:20:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=VAQRTw/5GtO2IVjhxbnhbmA6jHoOszIxdzX2lqnMQMg=; b=KiQBSMShDGj9y6Xiii40u6wmUwaPzvCmLBrsQiiSeEfxEgpqCJWUuqzMHmO/Q73wx5 r5riO7kqXyCHlrCGrBMtmwN2qlKAcIX9GLH7grhaEilxxO8KtWz0gbBbCVXi2z6BsmQX aYI3FaS4XY/k3gNyDLvPelJslB3mROrAPEU+4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=VAQRTw/5GtO2IVjhxbnhbmA6jHoOszIxdzX2lqnMQMg=; b=CmSJucVb9BfocRKcCei9n5opnIy+eq6V9IQpU86L1LAZsV4MDAD+zG0tTJX0h5Mgb8 OI6sn/FbzOg3oHsmSZTZ9ks4KXWVNlHKIE4FUJtTvjglj2WlCC+dJVcEArBzW/zHXB4G QoC9SHMBrZNLTGqx9Kv5vU3Xr4OqmE7FvUXIEvsSORGZCfAqWMcepJdsEuMn2H6Ow9LC gfb9yMYu96nKUCcvYn+vBtpX4T4ZNvSDGjTfg011YZmCCojavEkhkB6e25+enrPACv1r O+bAosBTk9bBgZJd+6KPJFbO9iINbmZrFklFTN98c95bI1jzxhj+o6qdOSzjrwlktZYA VF3A== X-Gm-Message-State: APjAAAWLQV/5DnSFhqtjobO6YDYV5shYwy0P0BQoRr7oIRTT9/d3jhLc gSX9pfyqpJ5ncYeARPfWdTlUQ9bCYvw= X-Google-Smtp-Source: APXvYqykfjS84Wch2NctDuoxhRCl1Y8YxQnTMAp2BwXlvT5nVnlAr/uCcXGqkIIx/qsACUKgYHMMvg== X-Received: by 2002:a62:2d3:: with SMTP id 202mr1245085pfc.141.1567128038716; Thu, 29 Aug 2019 18:20:38 -0700 (PDT) Received: from localhost ([2620:15c:6:12:9c46:e0da:efbf:69cc]) by smtp.gmail.com with ESMTPSA id d11sm5473663pfh.59.2019.08.29.18.20.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Aug 2019 18:20:37 -0700 (PDT) Date: Thu, 29 Aug 2019 21:20:36 -0400 From: Joel Fernandes To: "Paul E. McKenney" Cc: linux-kernel@vger.kernel.org, Frederic Weisbecker , Jonathan Corbet , Josh Triplett , kernel-team@android.com, Lai Jiangshan , linux-doc@vger.kernel.org, Mathieu Desnoyers , Mauro Carvalho Chehab , rcu@vger.kernel.org, Steven Rostedt Subject: Re: [RFC v1 2/2] rcu/tree: Remove dynticks_nmi_nesting counter Message-ID: <20190830012036.GA184995@google.com> References: <20190828220108.GC26530@linux.ibm.com> <20190828221444.GA100789@google.com> <20190828231247.GE26530@linux.ibm.com> <20190829015155.GB100789@google.com> <20190829034336.GD4125@linux.ibm.com> <20190829144355.GE63638@google.com> <20190829151325.GF63638@google.com> <20190829161301.GQ4125@linux.ibm.com> <20190829171454.GA115245@google.com> <20190830004756.GW4125@linux.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190830004756.GW4125@linux.ibm.com> User-Agent: Mutt/1.10.1 (2018-07-13) Sender: rcu-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org On Thu, Aug 29, 2019 at 05:47:56PM -0700, Paul E. McKenney wrote: [snip] > > > > Paul, also what what happens in the following scenario: > > > > > > > > CPU0 CPU1 > > > > > > > > A syscall causes rcu_eqs_exit() > > > > rcu_read_lock(); > > > > ---> FQS loop waiting on > > > > dyntick_snap > > > > usermode-upcall entry -->causes rcu_eqs_enter(); > > > > > > > > usermode-upcall exit -->causes rcu_eqs_exit(); > > > > > > > > ---> FQS loop sees > > > > dyntick snap > > > > increment and > > > > declares CPU0 is > > > > in a QS state > > > > before the > > > > rcu_read_unlock! > > > > > > > > rcu_read_unlock(); > > > > --- > > > > > > > > Does the context tracking not call rcu_user_enter() in this case, or did I > > > > really miss something? > > > > > > Holding rcu_read_lock() across usermode execution (in this case, > > > the usermode upcall) is a bad idea. Why is CPU 0 doing that? > > > > Oh, ok. I was just hypothesizing that since usermode upcalls from > > something as heavy as interrupts, it could also mean we had the same from > > some path that held an rcu_read_lock() as well. It was just a theoretical > > concern, if it is not an issue, no problem. > > Are there the usual lockdep checks in the upcall code? Holding a spinlock > across them would seem to be at least as bad as holding an rcu_read_lock() > across them. Great point, I'll take a look. > > The other question I had was, in which cases would dyntick_nesting in current > > RCU code be > 1 (after removing the lower bit and any crowbarring) ? In the > > scenarios I worked out on paper, I can only see this as 1 or 0. But the > > wording of it is 'dynticks_nesting'. May be I am missing a nesting scenario? > > We can exit RCU-idleness into process context only once (either exiting idle > > mode or user mode). Both cases would imply a value of 1. > > Interrrupt -> NMI -> certain types of tracing. I believe that can get > it to 5. There might be even more elaborate sequences of events. I am only talking about dynticks_nesting, not dynticks_nmi_nesting. In current mainline, I see this only 0 or 1. I am running the below patch overnight on all RCU configurations to see if it is ever any other value. And, please feel free to ignore my emails as you mentioned you are supposed to be out next 2 days! Thanks for the replies though! ---8<----------------------- diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 68ebf0eb64c8..8c8ddb6457d5 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -571,6 +571,9 @@ static void rcu_eqs_enter(bool user) WRITE_ONCE(rdp->dynticks_nmi_nesting, 0); WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && rdp->dynticks_nesting == 0); + + WARN_ON_ONCE(rdp->dynticks_nesting != 1); + if (rdp->dynticks_nesting != 1) { rdp->dynticks_nesting--; return; @@ -736,6 +739,9 @@ static void rcu_eqs_exit(bool user) lockdep_assert_irqs_disabled(); rdp = this_cpu_ptr(&rcu_data); oldval = rdp->dynticks_nesting; + + WARN_ON_ONCE(rdp->dynticks_nesting != 0); + WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && oldval < 0); if (oldval) { rdp->dynticks_nesting++; -- 2.23.0.187.g17f5b7556c-goog