From: Marcelo Tosatti <mtosatti@redhat.com>
To: atomlin@atomlin.com, frederic@kernel.org
Cc: cl@linux.com, tglx@linutronix.de, mingo@kernel.org,
peterz@infradead.org, pauld@redhat.com, neelx@redhat.com,
oleksandr@natalenko.name, linux-kernel@vger.kernel.org,
linux-mm@kvack.org, Marcelo Tosatti <mtosatti@redhat.com>
Subject: [PATCH v13 4/6] tick/nohz_full: Ensure quiet_vmstat() is called on exit to user-mode when the idle tick is stopped
Date: Thu, 05 Jan 2023 09:52:22 -0300 [thread overview]
Message-ID: <20230105125248.892336104@redhat.com> (raw)
In-Reply-To: 20230105125218.031928326@redhat.com
From: Aaron Tomlin <atomlin@atomlin.com>
For nohz full CPUs, we'd like the per-CPU vm statistics to be
synchronized when userspace is executing. Otherwise, the vmstat_shepherd
might queue a work item to synchronize them, which is undesired
intereference for isolated CPUs.
This patch syncs CPU-specific vmstat differentials, on return to
userspace, if CONFIG_FLUSH_WORK_ON_RESUME_USER is enabled and the tick
is stopped.
A trivial test program was used to determine the impact of the proposed
changes and under vanilla. The mlock(2) and munlock(2) system calls
was used solely to modify vmstat item 'NR_MLOCK'. The following is an
average count of CPU-cycles across the aforementioned system calls:
Vanilla Modified
Cycles per syscall 8461 8690 (+2.6%)
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
---
include/linux/tick.h | 5 +++--
kernel/time/tick-sched.c | 15 +++++++++++++++
2 files changed, 18 insertions(+), 2 deletions(-)
Index: linux-2.6/include/linux/tick.h
===================================================================
--- linux-2.6.orig/include/linux/tick.h
+++ linux-2.6/include/linux/tick.h
@@ -11,7 +11,6 @@
#include <linux/context_tracking_state.h>
#include <linux/cpumask.h>
#include <linux/sched.h>
-#include <linux/rcupdate.h>
#ifdef CONFIG_GENERIC_CLOCKEVENTS
extern void __init tick_init(void);
@@ -272,6 +271,7 @@ static inline void tick_dep_clear_signal
extern void tick_nohz_full_kick_cpu(int cpu);
extern void __tick_nohz_task_switch(void);
+void __tick_nohz_user_enter_prepare(void);
extern void __init tick_nohz_full_setup(cpumask_var_t cpumask);
#else
static inline bool tick_nohz_full_enabled(void) { return false; }
@@ -296,6 +296,7 @@ static inline void tick_dep_clear_signal
static inline void tick_nohz_full_kick_cpu(int cpu) { }
static inline void __tick_nohz_task_switch(void) { }
+static inline void __tick_nohz_user_enter_prepare(void) { }
static inline void tick_nohz_full_setup(cpumask_var_t cpumask) { }
#endif
@@ -308,7 +309,7 @@ static inline void tick_nohz_task_switch
static inline void tick_nohz_user_enter_prepare(void)
{
if (tick_nohz_full_cpu(smp_processor_id()))
- rcu_nocb_flush_deferred_wakeup();
+ __tick_nohz_user_enter_prepare();
}
#endif
Index: linux-2.6/kernel/time/tick-sched.c
===================================================================
--- linux-2.6.orig/kernel/time/tick-sched.c
+++ linux-2.6/kernel/time/tick-sched.c
@@ -26,6 +26,7 @@
#include <linux/posix-timers.h>
#include <linux/context_tracking.h>
#include <linux/mm.h>
+#include <linux/rcupdate.h>
#include <asm/irq_regs.h>
@@ -519,6 +520,23 @@ void __tick_nohz_task_switch(void)
}
}
+void __tick_nohz_user_enter_prepare(void)
+{
+ if (tick_nohz_full_cpu(smp_processor_id())) {
+ if (IS_ENABLED(CONFIG_FLUSH_WORK_ON_RESUME_USER)) {
+ struct tick_sched *ts;
+
+ ts = this_cpu_ptr(&tick_cpu_sched);
+
+ if (ts->tick_stopped)
+ quiet_vmstat(true);
+ }
+
+ rcu_nocb_flush_deferred_wakeup();
+ }
+}
+EXPORT_SYMBOL_GPL(__tick_nohz_user_enter_prepare);
+
/* Get the boot-time nohz CPU list from the kernel parameters. */
void __init tick_nohz_full_setup(cpumask_var_t cpumask)
{
next prev parent reply other threads:[~2023-01-05 12:57 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-05 12:52 [PATCH v13 0/6] Ensure quiet_vmstat() is called when returning to userpace and when idle tick is stopped Marcelo Tosatti
2023-01-05 12:52 ` [PATCH v13 1/6] mm/vmstat: Add CPU-specific variable to track a vmstat discrepancy Marcelo Tosatti
2023-01-10 11:58 ` Christoph Lameter
2023-01-10 12:12 ` Frederic Weisbecker
2023-01-05 12:52 ` [PATCH v13 2/6] mm/vmstat: Use vmstat_dirty to track CPU-specific vmstat discrepancies Marcelo Tosatti
2023-01-10 12:06 ` Christoph Lameter
2023-01-10 12:18 ` Frederic Weisbecker
2023-01-10 13:39 ` Christoph Lameter
2023-01-10 20:09 ` Marcelo Tosatti
2023-01-11 8:42 ` Christoph Lameter
2023-01-11 17:07 ` Marcelo Tosatti
2023-01-16 9:51 ` Christoph Lameter
2023-01-16 16:11 ` Marcelo Tosatti
2023-01-17 12:52 ` Christoph Lameter
2023-01-05 12:52 ` [PATCH v13 3/6] mm/vmstat: manage per-CPU stats from CPU context when NOHZ full Marcelo Tosatti
2023-01-05 12:52 ` Marcelo Tosatti [this message]
2023-01-05 12:52 ` [PATCH v13 5/6] tick/sched: Ensure quiet_vmstat() is called when the idle tick was stopped too Marcelo Tosatti
2023-01-05 12:52 ` [PATCH v13 6/6] mm/vmstat: avoid queueing work item if cpu stats are clean Marcelo Tosatti
[not found] ` <20230106001244.4463-1-hdanton@sina.com>
2023-01-06 12:51 ` [PATCH v13 3/6] mm/vmstat: manage per-CPU stats from CPU context when NOHZ full Marcelo Tosatti
[not found] ` <20230106150154.4560-1-hdanton@sina.com>
2023-01-06 18:16 ` Marcelo Tosatti
[not found] ` <20230107001529.4617-1-hdanton@sina.com>
2023-01-09 14:12 ` Marcelo Tosatti
[not found] ` <20230110024356.336-1-hdanton@sina.com>
2023-01-10 11:50 ` Marcelo Tosatti
[not found] ` <20230110151901.402-1-hdanton@sina.com>
2023-01-10 16:12 ` Frederic Weisbecker
[not found] ` <20230110235822.456-1-hdanton@sina.com>
2023-01-11 0:09 ` Frederic Weisbecker
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230105125248.892336104@redhat.com \
--to=mtosatti@redhat.com \
--cc=atomlin@atomlin.com \
--cc=cl@linux.com \
--cc=frederic@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mingo@kernel.org \
--cc=neelx@redhat.com \
--cc=oleksandr@natalenko.name \
--cc=pauld@redhat.com \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).