linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Paul E. McKenney" <paulmck@kernel.org>
To: Frederic Weisbecker <frederic@kernel.org>
Cc: LKML <linux-kernel@vger.kernel.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
	Lai Jiangshan <jiangshanlai@gmail.com>,
	Joel Fernandes <joel@joelfernandes.org>,
	Josh Triplett <josh@joshtriplett.org>
Subject: Re: [RFC PATCH 03/12] rcu: Provide basic callback offloading state machine bits
Date: Mon, 21 Sep 2020 16:50:03 -0700	[thread overview]
Message-ID: <20200921235003.GM29330@paulmck-ThinkPad-P72> (raw)
In-Reply-To: <20200921124351.24035-4-frederic@kernel.org>

On Mon, Sep 21, 2020 at 02:43:42PM +0200, Frederic Weisbecker wrote:
> We'll need to be able to runtime offload and de-offload the processing
> of callback for a given CPU. In order to support a smooth transition
> from unlocked local processing (softirq/rcuc) to locked offloaded
> processing (rcuop/rcuog) and the reverse, provide the necessary bits and
> documentation for the state machine that will carry up all the steps to
> enforce correctness while serving callbacks processing all along.
> 
> Inspired-by: Paul E. McKenney <paulmck@kernel.org>
> Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
> Cc: Paul E. McKenney <paulmck@kernel.org>
> Cc: Josh Triplett <josh@joshtriplett.org>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> Cc: Lai Jiangshan <jiangshanlai@gmail.com>
> Cc: Joel Fernandes <joel@joelfernandes.org>
> ---
>  include/linux/rcu_segcblist.h | 115 +++++++++++++++++++++++++++++++++-
>  kernel/rcu/rcu_segcblist.c    |   1 +
>  kernel/rcu/rcu_segcblist.h    |  12 +++-
>  kernel/rcu/tree.c             |   3 +
>  4 files changed, 128 insertions(+), 3 deletions(-)
> 
> diff --git a/include/linux/rcu_segcblist.h b/include/linux/rcu_segcblist.h
> index dca2f39ee67f..67f09a912f96 100644
> --- a/include/linux/rcu_segcblist.h
> +++ b/include/linux/rcu_segcblist.h
> @@ -63,8 +63,121 @@ struct rcu_cblist {
>  #define RCU_NEXT_TAIL		3
>  #define RCU_CBLIST_NSEGS	4
>  
> +
> +/*
> + *                     ==NOCB Offloading state machine==
> + *
> + *
> + *  ----------------------------------------------------------------------------
> + *  |                         SEGCBLIST_SOFTIRQ_ONLY                           |
> + *  |                                                                          |
> + *  |  Callbacks processed by rcu_core() from softirqs or local                |
> + *  |  rcuc kthread, without holding nocb_lock.                                |
> + *  ----------------------------------------------------------------------------
> + *                                         |
> + *                                         v
> + *  ----------------------------------------------------------------------------
> + *  |                        SEGCBLIST_OFFLOADED                               |
> + *  |                                                                          |
> + *  | Callbacks processed by rcu_core() from softirqs or local                 |
> + *  | rcuc kthread, while holding nocb_lock. Waking up CB and GP kthreads,     |
> + *  |  allowing nocb_timer to be armed.                                        |

Whitespace nit before "allowing", just to show that I am paying attention.  ;-)

Don't we need to acquire rcu_state.barrier_mutex at this point?  Otherwise
rcu_barrier() could fail at this point.

> + *  ----------------------------------------------------------------------------
> + *                                         |
> + *                                         v
> + *                        -----------------------------------
> + *                        |                                 |
> + *                        v                                 v
> + *  ---------------------------------------  ----------------------------------|
> + *  |        SEGCBLIST_OFFLOADED |        |  |     SEGCBLIST_OFFLOADED |       |
> + *  |        SEGCBLIST_KTHREAD_CB         |  |     SEGCBLIST_KTHREAD_GP        |
> + *  |                                     |  |                                 |
> + *  |                                     |  |                                 |
> + *  | CB kthread woke up and              |  | GP kthread woke up and          |
> + *  | acknowledged SEGCBLIST_OFFLOADED.   |  | acknowledged SEGCBLIST_OFFLOADED|
> + *  | Processes callbacks concurrently    |  |                                 |
> + *  | with rcu_core(), holding            |  |                                 |
> + *  | nocb_lock.                          |  |                                 |
> + *  ---------------------------------------  -----------------------------------
> + *                        |                                 |
> + *                        -----------------------------------
> + *                                         |
> + *                                         v
> + *  |--------------------------------------------------------------------------|
> + *  |                           SEGCBLIST_OFFLOADED |                          |
> + *  |                           SEGCBLIST_KTHREAD_CB |                         |
> + *  |                           SEGCBLIST_KTHREAD_GP                           |
> + *  |                                                                          |
> + *  |   Kthreads handle callbacks holding nocb_lock, local rcu_core() stops    |
> + *  |   handling callbacks.                                                    |

And rcu_state.barrier_mutex is dropped here.

Or am I missing a trick here?  I guess I will look at the later patches to
get an initial estimate of an answer to this question.

> + *  ----------------------------------------------------------------------------
> + */
> +
> +
> +
> +/*
> + *                       ==NOCB De-Offloading state machine==
> + *
> + *
> + *  |--------------------------------------------------------------------------|
> + *  |                           SEGCBLIST_OFFLOADED |                          |
> + *  |                           SEGCBLIST_KTHREAD_CB |                         |
> + *  |                           SEGCBLIST_KTHREAD_GP                           |
> + *  |                                                                          |
> + *  |   CB/GP kthreads handle callbacks holding nocb_lock, local rcu_core()    |
> + *  |   ignores callbacks.                                                     |

And don't we also need to acquire rcu_state.barrier_mutex here?

> + *  ----------------------------------------------------------------------------
> + *                                      |
> + *                                      v
> + *  |--------------------------------------------------------------------------|
> + *  |                           SEGCBLIST_KTHREAD_CB |                         |
> + *  |                           SEGCBLIST_KTHREAD_GP                           |
> + *  |                                                                          |
> + *  |   CB/GP kthreads and local rcu_core() handle callbacks concurrently      |
> + *  |   holding nocb_lock. Wake up CB and GP kthreads if necessary.            |
> + *  ----------------------------------------------------------------------------
> + *                                      |
> + *                                      v
> + *                     -----------------------------------
> + *                     |                                 |
> + *                     v                                 v
> + *  ---------------------------------------------------------------------------|
> + *  |                                                                          |
> + *  |        SEGCBLIST_KTHREAD_CB         |       SEGCBLIST_KTHREAD_GP         |
> + *  |                                     |                                    |
> + *  | GP kthread woke up and              |   CB kthread woke up and           |
> + *  | acknowledged the fact that          |   acknowledged the fact that       |
> + *  | SEGCBLIST_OFFLOADED got cleared.    |   SEGCBLIST_OFFLOADED got cleared. |
> + *  |                                     |   The CB kthread goes to sleep     |
> + *  | The callbacks from the target CPU   |   until it ever gets re-offloaded. |
> + *  | will be ignored from the GP kthread |                                    |
> + *  | loop.                               |                                    |
> + *  ----------------------------------------------------------------------------
> + *                      |                                 |
> + *                      -----------------------------------
> + *                                      |
> + *                                      v
> + *  ----------------------------------------------------------------------------
> + *  |                                   0                                      |
> + *  |                                                                          |
> + *  | Callbacks processed by rcu_core() from softirqs or local                 |
> + *  | rcuc kthread, while holding nocb_lock. Forbid nocb_timer to be armed.    |
> + *  | Flush pending nocb_timer. Flush nocb bypass callbacks.                   |

And release rcu_state.barrier_mutex here?

							Thanx, Paul

> + *  ----------------------------------------------------------------------------
> + *                                      |
> + *                                      v
> + *  ----------------------------------------------------------------------------
> + *  |                         SEGCBLIST_SOFTIRQ_ONLY                           |
> + *  |                                                                          |
> + *  |  Callbacks processed by rcu_core() from softirqs or local                |
> + *  |  rcuc kthread, without holding nocb_lock.                                |
> + *  ----------------------------------------------------------------------------
> + */
>  #define SEGCBLIST_ENABLED	BIT(0)
> -#define SEGCBLIST_OFFLOADED	BIT(1)
> +#define SEGCBLIST_SOFTIRQ_ONLY	BIT(1)
> +#define SEGCBLIST_KTHREAD_CB	BIT(2)
> +#define SEGCBLIST_KTHREAD_GP	BIT(3)
> +#define SEGCBLIST_OFFLOADED	BIT(4)
>  
>  struct rcu_segcblist {
>  	struct rcu_head *head;
> diff --git a/kernel/rcu/rcu_segcblist.c b/kernel/rcu/rcu_segcblist.c
> index d131ef8940a0..31cc27ee98d8 100644
> --- a/kernel/rcu/rcu_segcblist.c
> +++ b/kernel/rcu/rcu_segcblist.c
> @@ -172,6 +172,7 @@ void rcu_segcblist_disable(struct rcu_segcblist *rsclp)
>   */
>  void rcu_segcblist_offload(struct rcu_segcblist *rsclp)
>  {
> +	rcu_segcblist_clear_flags(rsclp, SEGCBLIST_SOFTIRQ_ONLY);
>  	rcu_segcblist_set_flags(rsclp, SEGCBLIST_OFFLOADED);
>  }
>  
> diff --git a/kernel/rcu/rcu_segcblist.h b/kernel/rcu/rcu_segcblist.h
> index fc98761e3ee9..575896a2518b 100644
> --- a/kernel/rcu/rcu_segcblist.h
> +++ b/kernel/rcu/rcu_segcblist.h
> @@ -80,8 +80,16 @@ static inline bool rcu_segcblist_is_enabled(struct rcu_segcblist *rsclp)
>  /* Is the specified rcu_segcblist offloaded?  */
>  static inline bool rcu_segcblist_is_offloaded(struct rcu_segcblist *rsclp)
>  {
> -	return IS_ENABLED(CONFIG_RCU_NOCB_CPU) &&
> -		rcu_segcblist_test_flags(rsclp, SEGCBLIST_OFFLOADED);
> +	if (IS_ENABLED(CONFIG_RCU_NOCB_CPU)) {
> +		/*
> +		 * Complete de-offloading happens only when SEGCBLIST_SOFTIRQ_ONLY
> +		 * is set.
> +		 */
> +		if (!rcu_segcblist_test_flags(rsclp, SEGCBLIST_SOFTIRQ_ONLY))
> +			return true;
> +	}
> +
> +	return false;
>  }
>  
>  /*
> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> index c0286ce8fc03..b4292489db0c 100644
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -96,6 +96,9 @@ static DEFINE_PER_CPU_SHARED_ALIGNED(struct rcu_data, rcu_data) = {
>  	.dynticks_nesting = 1,
>  	.dynticks_nmi_nesting = DYNTICK_IRQ_NONIDLE,
>  	.dynticks = ATOMIC_INIT(RCU_DYNTICK_CTRL_CTR),
> +#ifdef CONFIG_RCU_NOCB_CPU
> +	.cblist.flags = SEGCBLIST_SOFTIRQ_ONLY,
> +#endif
>  };
>  static struct rcu_state rcu_state = {
>  	.level = { &rcu_state.node[0] },
> -- 
> 2.28.0
> 

  reply	other threads:[~2020-09-21 23:50 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-09-21 12:43 [RFC PATCH 00/12] rcu/nocb: De-offload and re-offload support v2 Frederic Weisbecker
2020-09-21 12:43 ` [RFC PATCH 01/12] rcu: Implement rcu_segcblist_is_offloaded() config dependent Frederic Weisbecker
2020-09-22  0:27   ` Paul E. McKenney
2020-09-22 21:43     ` Frederic Weisbecker
2020-09-22 23:11       ` Paul E. McKenney
2020-09-23 15:25         ` Frederic Weisbecker
2020-09-21 12:43 ` [RFC PATCH 02/12] rcu: Turn enabled/offload states into a common flag Frederic Weisbecker
2020-09-21 12:43 ` [RFC PATCH 03/12] rcu: Provide basic callback offloading state machine bits Frederic Weisbecker
2020-09-21 23:50   ` Paul E. McKenney [this message]
2020-09-21 12:43 ` [RFC PATCH 04/12] rcu: De-offloading CB kthread Frederic Weisbecker
2020-09-22  0:07   ` Paul E. McKenney
2020-09-21 12:43 ` [RFC PATCH 05/12] rcu: De-offloading GP kthread Frederic Weisbecker
2020-09-22  0:10   ` Paul E. McKenney
2020-09-23 15:31     ` Frederic Weisbecker
2020-09-23 15:48       ` Paul E. McKenney
2020-09-21 12:43 ` [RFC PATCH 06/12] rcu: Re-offload support Frederic Weisbecker
2020-09-21 12:43 ` [RFC PATCH 07/12] rcu: Shutdown nocb timer on de-offloading Frederic Weisbecker
2020-09-22  0:17   ` Paul E. McKenney
2020-09-23 15:29     ` Frederic Weisbecker
2020-09-23 15:47       ` Paul E. McKenney
2020-09-21 12:43 ` [RFC PATCH 08/12] rcu: Flush bypass before setting SEGCBLIST_SOFTIRQ_ONLY Frederic Weisbecker
2020-09-21 12:43 ` [RFC PATCH 09/12] rcu: Set SEGCBLIST_SOFTIRQ_ONLY at the very last stage of de-offloading Frederic Weisbecker
2020-09-21 12:43 ` [RFC PATCH 10/12] rcu: Process batch locally as long as offloading isn't complete Frederic Weisbecker
2020-09-22  0:22   ` Paul E. McKenney
2020-09-21 12:43 ` [RFC PATCH 11/12] rcu: Locally accelerate callbacks " Frederic Weisbecker
2020-09-22  0:24   ` Paul E. McKenney
2020-09-21 12:43 ` [RFC PATCH 12/12] rcu: Nocb (de)activate through sysfs Frederic Weisbecker
2020-09-22  0:26   ` Paul E. McKenney
2020-09-23 15:27     ` Frederic Weisbecker
2020-09-23 15:47       ` Paul E. McKenney
2020-09-23 19:00         ` Frederic Weisbecker
2020-09-24  0:38           ` Paul E. McKenney

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200921235003.GM29330@paulmck-ThinkPad-P72 \
    --to=paulmck@kernel.org \
    --cc=frederic@kernel.org \
    --cc=jiangshanlai@gmail.com \
    --cc=joel@joelfernandes.org \
    --cc=josh@joshtriplett.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mathieu.desnoyers@efficios.com \
    --cc=rostedt@goodmis.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).