From: Marcelo Tosatti <mtosatti@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: Nitesh Lal <nilal@redhat.com>,
Nicolas Saenz Julienne <nsaenzju@redhat.com>,
Frederic Weisbecker <frederic@kernel.org>,
Christoph Lameter <cl@linux.com>,
Juri Lelli <juri.lelli@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
Alex Belits <abelits@belits.com>, Peter Xu <peterx@redhat.com>,
Thomas Gleixner <tglx@linutronix.de>,
Daniel Bristot de Oliveira <bristot@redhat.com>,
Oscar Shiang <oscar0225@livemail.tw>,
Marcelo Tosatti <mtosatti@redhat.com>
Subject: [patch v11 07/13] task isolation: sync vmstats conditional on changes
Date: Fri, 04 Feb 2022 14:35:44 -0300 [thread overview]
Message-ID: <20220204173554.763888172@fedora.localdomain> (raw)
In-Reply-To: 20220204173537.429902988@fedora.localdomain
Rather than syncing VM-stats on every return to userspace
(or VM-entry), keep track of changes through a per-CPU bool.
This improves performance when enabling task isolated
for vcpu VMs.
Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
---
v11:
- Add TIF_TASK_ISOL bit to thread info flags and use it
to decide whether to perform task isolation work on
return to userspace
include/linux/vmstat.h | 13 ++++++++++++-
mm/vmstat.c | 29 ++++++++++++++++++++++++++++-
2 files changed, 40 insertions(+), 2 deletions(-)
Index: linux-2.6/include/linux/vmstat.h
===================================================================
--- linux-2.6.orig/include/linux/vmstat.h
+++ linux-2.6/include/linux/vmstat.h
@@ -22,7 +22,18 @@ int sysctl_vm_numa_stat_handler(struct c
#endif
#if defined(CONFIG_SMP) && defined(CONFIG_TASK_ISOLATION)
-void sync_vmstat(void);
+DECLARE_PER_CPU_ALIGNED(bool, vmstat_dirty);
+
+extern struct static_key vmstat_sync_enabled;
+
+void __sync_vmstat(void);
+static inline void sync_vmstat(void)
+{
+ if (static_key_false(&vmstat_sync_enabled))
+ __sync_vmstat();
+}
+
+void init_sync_vmstat(void);
#else
static inline void sync_vmstat(void)
{
Index: linux-2.6/mm/vmstat.c
===================================================================
--- linux-2.6.orig/mm/vmstat.c
+++ linux-2.6/mm/vmstat.c
@@ -334,6 +334,31 @@ void set_pgdat_percpu_threshold(pg_data_
}
}
+#ifdef CONFIG_TASK_ISOLATION
+struct static_key vmstat_sync_enabled;
+DEFINE_PER_CPU_ALIGNED(bool, vmstat_dirty);
+
+static inline void mark_vmstat_dirty(void)
+{
+ if (!static_key_false(&vmstat_sync_enabled))
+ return;
+
+ raw_cpu_write(vmstat_dirty, true);
+ set_thread_flag(TIF_TASK_ISOL);
+}
+
+void init_sync_vmstat(void)
+{
+ raw_cpu_write(vmstat_dirty, true);
+ set_thread_flag(TIF_TASK_ISOL);
+}
+EXPORT_SYMBOL_GPL(vmstat_dirty);
+#else
+static inline void mark_vmstat_dirty(void)
+{
+}
+#endif
+
/*
* For use when we know that interrupts are disabled,
* or when we know that preemption is disabled and that
@@ -366,6 +391,7 @@ void __mod_zone_page_state(struct zone *
x = 0;
}
__this_cpu_write(*p, x);
+ mark_vmstat_dirty();
if (IS_ENABLED(CONFIG_PREEMPT_RT))
preempt_enable();
@@ -404,6 +430,7 @@ void __mod_node_page_state(struct pglist
x = 0;
}
__this_cpu_write(*p, x);
+ mark_vmstat_dirty();
if (IS_ENABLED(CONFIG_PREEMPT_RT))
preempt_enable();
@@ -602,6 +629,7 @@ static inline void mod_zone_state(struct
if (z)
zone_page_state_add(z, zone, item);
+ mark_vmstat_dirty();
}
void mod_zone_page_state(struct zone *zone, enum zone_stat_item item,
@@ -670,6 +698,7 @@ static inline void mod_node_state(struct
if (z)
node_page_state_add(z, pgdat, item);
+ mark_vmstat_dirty();
}
void mod_node_page_state(struct pglist_data *pgdat, enum node_stat_item item,
@@ -1087,6 +1116,7 @@ static void fill_contig_page_info(struct
info->free_blocks_suitable += blocks <<
(order - suitable_order);
}
+ mark_vmstat_dirty();
}
/*
@@ -1443,6 +1473,7 @@ static void walk_zones_in_node(struct se
if (!nolock)
spin_unlock_irqrestore(&zone->lock, flags);
}
+ mark_vmstat_dirty();
}
#endif
@@ -1512,6 +1543,7 @@ static void pagetypeinfo_showfree_print(
}
seq_putc(m, '\n');
}
+ mark_vmstat_dirty();
}
/* Print out the free pages at each order for each migatetype */
@@ -1932,6 +1964,7 @@ static void vmstat_update(struct work_st
this_cpu_ptr(&vmstat_work),
round_jiffies_relative(sysctl_stat_interval));
}
+ mark_vmstat_dirty();
}
/*
@@ -2019,13 +2052,14 @@ static void vmstat_shepherd(struct work_
}
#ifdef CONFIG_TASK_ISOLATION
-void sync_vmstat(void)
+void __sync_vmstat(void)
{
int cpu;
cpu = get_cpu();
refresh_cpu_vm_stats(false);
+ raw_cpu_write(vmstat_dirty, false);
put_cpu();
/*
next prev parent reply other threads:[~2022-02-04 17:37 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-04 17:35 [patch v11 00/13] extensible prctl task isolation interface and vmstat sync Marcelo Tosatti
2022-02-04 17:35 ` [patch v11 01/13] s390: add support for TIF_TASK_ISOL Marcelo Tosatti
2022-02-07 11:40 ` Sven Schnelle
2022-02-04 17:35 ` [patch v11 02/13] x86: " Marcelo Tosatti
2022-02-04 17:35 ` [patch v11 03/13] add basic task isolation prctl interface Marcelo Tosatti
2022-02-04 17:35 ` [patch v11 04/13] add prctl task isolation prctl docs and samples Marcelo Tosatti
2022-02-04 17:35 ` [patch v11 05/13] task isolation: sync vmstats on return to userspace Marcelo Tosatti
2022-02-07 14:57 ` Frederic Weisbecker
2022-02-07 15:16 ` Marcelo Tosatti
2022-02-04 17:35 ` [patch v11 06/13] procfs: add per-pid task isolation state Marcelo Tosatti
2022-02-04 17:35 ` Marcelo Tosatti [this message]
2022-02-07 15:19 ` [patch v11 07/13] task isolation: sync vmstats conditional on changes Frederic Weisbecker
2022-03-10 16:30 ` Marcelo Tosatti
2022-02-07 15:29 ` Frederic Weisbecker
2022-02-04 17:35 ` [patch v11 08/13] task isolation: enable return to userspace processing Marcelo Tosatti
2022-02-04 17:35 ` [patch v11 09/13] task isolation: add preempt notifier to sync per-CPU vmstat dirty info to thread info Marcelo Tosatti
2022-02-07 15:36 ` Frederic Weisbecker
2022-02-04 17:35 ` [patch v11 10/13] KVM: x86: process isolation work from VM-entry code path Marcelo Tosatti
2022-02-07 15:47 ` Frederic Weisbecker
2022-03-10 16:35 ` Marcelo Tosatti
2022-02-04 17:35 ` [patch v11 11/13] mm: vmstat: move need_update Marcelo Tosatti
2022-02-04 17:35 ` [patch v11 12/13] mm: vmstat_refresh: avoid queueing work item if cpu stats are clean Marcelo Tosatti
2022-02-04 17:35 ` [patch v11 13/13] task isolation: only TIF_TASK_ISOL if task isolation is enabled Marcelo Tosatti
2022-02-07 15:38 ` Frederic Weisbecker
2022-02-19 8:02 ` [patch v11 00/13] extensible prctl task isolation interface and vmstat sync Oscar Shiang
2022-02-23 17:31 ` Marcelo Tosatti
2022-03-08 6:32 ` Oscar Shiang
2022-03-08 13:12 ` Marcelo Tosatti
2022-03-09 15:31 ` Oscar Shiang
2022-03-08 7:20 ` Oscar Shiang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220204173554.763888172@fedora.localdomain \
--to=mtosatti@redhat.com \
--cc=abelits@belits.com \
--cc=bristot@redhat.com \
--cc=cl@linux.com \
--cc=frederic@kernel.org \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=nilal@redhat.com \
--cc=nsaenzju@redhat.com \
--cc=oscar0225@livemail.tw \
--cc=peterx@redhat.com \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).