From: Mark Salyzyn <salyzyn@android.com>
To: linux-kernel@vger.kernel.org
Cc: Mark Salyzyn <salyzyn@android.com>,
James Morse <james.morse@arm.com>,
Russell King <linux@armlinux.org.uk>,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will.deacon@arm.com>,
Andy Lutomirski <luto@amacapital.net>,
Dmitry Safonov <dsafonov@virtuozzo.com>,
John Stultz <john.stultz@linaro.org>,
Mark Rutland <mark.rutland@arm.com>,
Laura Abbott <labbott@redhat.com>,
Kees Cook <keescook@chromium.org>,
Ard Biesheuvel <ard.biesheuvel@linaro.org>,
Andy Gross <andy.gross@linaro.org>,
Kevin Brodsky <kevin.brodsky@arm.com>,
Andrew Pinski <apinski@cavium.com>,
Thomas Gleixner <tglx@linutronix.de>,
linux-arm-kernel@lists.infradead.org,
Jeremy Linton <Jeremy.Linton@arm.com>,
android-kernel@android.com, Jinbum Park <jinb.park7@gmail.com>,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
Ingo Molnar <mingo@kernel.org>
Subject: RESEND [PATCH v5 01/12] arm: vdso: rename vdso_datapage variables
Date: Mon, 1 Oct 2018 10:58:21 -0700 [thread overview]
Message-ID: <20181001175845.168430-2-salyzyn@android.com> (raw)
In-Reply-To: <20181001175845.168430-1-salyzyn@android.com>
Take an effort to recode the arm64 vdso code from assembler to C
previously submitted by Andrew Pinski <apinski@cavium.com>, rework
it for use in both arm and arm64, overlapping any optimizations
for each architecture. But instead of landing it in arm64, land the
result into lib/vdso and unify both implementations to simplify
future maintenance.
Rename seq_count to tb_seq_count. Rename tk_is_cntvct to use_syscall.
Rename cs_mult to cs_mono_mult. All to align with the variables in the
arm64 vdso datapage. Rework vdso_read_begin() and vdso_read_retry()
functions to reflect modern access patterns for tb_seq_count field.
Update copyright message to reflect the start of the contributions in
this series.
Signed-off-by: Mark Salyzyn <salyzyn@android.com>
Tested-by: Mark Salyzyn <salyzyn@android.com>
Cc: James Morse <james.morse@arm.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Dmitry Safonov <dsafonov@virtuozzo.com>
Cc: John Stultz <john.stultz@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Laura Abbott <labbott@redhat.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Andy Gross <andy.gross@linaro.org>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Andrew Pinski <apinski@cavium.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: Jeremy Linton <Jeremy.Linton@arm.com>
Cc: android-kernel@android.com
v2:
- split first CL into 1 of 7 pieces
v4:
- update commit message to reflect reasoning
v5:
- rebase
---
arch/arm/include/asm/vdso_datapage.h | 6 +--
arch/arm/kernel/vdso.c | 17 ++++----
arch/arm/vdso/vgettimeofday.c | 61 +++++++++++++++-------------
3 files changed, 45 insertions(+), 39 deletions(-)
diff --git a/arch/arm/include/asm/vdso_datapage.h b/arch/arm/include/asm/vdso_datapage.h
index 9be259442fca..fa3e1856244d 100644
--- a/arch/arm/include/asm/vdso_datapage.h
+++ b/arch/arm/include/asm/vdso_datapage.h
@@ -29,8 +29,8 @@
* 32 bytes.
*/
struct vdso_data {
- u32 seq_count; /* sequence count - odd during updates */
- u16 tk_is_cntvct; /* fall back to syscall if false */
+ u32 tb_seq_count; /* sequence count - odd during updates */
+ u16 use_syscall; /* fall back to syscall if true */
u16 cs_shift; /* clocksource shift */
u32 xtime_coarse_sec; /* coarse time */
u32 xtime_coarse_nsec;
@@ -38,7 +38,7 @@ struct vdso_data {
u32 wtm_clock_sec; /* wall to monotonic offset */
u32 wtm_clock_nsec;
u32 xtime_clock_sec; /* CLOCK_REALTIME - seconds */
- u32 cs_mult; /* clocksource multiplier */
+ u32 cs_mono_mult; /* clocksource multiplier */
u64 cs_cycle_last; /* last cycle value */
u64 cs_mask; /* clocksource mask */
diff --git a/arch/arm/kernel/vdso.c b/arch/arm/kernel/vdso.c
index f4dd7f9663c1..c2c57f6b8c60 100644
--- a/arch/arm/kernel/vdso.c
+++ b/arch/arm/kernel/vdso.c
@@ -276,14 +276,14 @@ void arm_install_vdso(struct mm_struct *mm, unsigned long addr)
static void vdso_write_begin(struct vdso_data *vdata)
{
- ++vdso_data->seq_count;
+ ++vdso_data->tb_seq_count;
smp_wmb(); /* Pairs with smp_rmb in vdso_read_retry */
}
static void vdso_write_end(struct vdso_data *vdata)
{
smp_wmb(); /* Pairs with smp_rmb in vdso_read_begin */
- ++vdso_data->seq_count;
+ ++vdso_data->tb_seq_count;
}
static bool tk_is_cntvct(const struct timekeeper *tk)
@@ -307,10 +307,10 @@ static bool tk_is_cntvct(const struct timekeeper *tk)
* counter again, making it even, indicating to userspace that the
* update is finished.
*
- * Userspace is expected to sample seq_count before reading any other
- * fields from the data page. If seq_count is odd, userspace is
+ * Userspace is expected to sample tb_seq_count before reading any other
+ * fields from the data page. If tb_seq_count is odd, userspace is
* expected to wait until it becomes even. After copying data from
- * the page, userspace must sample seq_count again; if it has changed
+ * the page, userspace must sample tb_seq_count again; if it has changed
* from its previous value, userspace must retry the whole sequence.
*
* Calls to update_vsyscall are serialized by the timekeeping core.
@@ -328,18 +328,19 @@ void update_vsyscall(struct timekeeper *tk)
vdso_write_begin(vdso_data);
- vdso_data->tk_is_cntvct = tk_is_cntvct(tk);
+ vdso_data->use_syscall = !tk_is_cntvct(tk);
vdso_data->xtime_coarse_sec = tk->xtime_sec;
vdso_data->xtime_coarse_nsec = (u32)(tk->tkr_mono.xtime_nsec >>
tk->tkr_mono.shift);
vdso_data->wtm_clock_sec = wtm->tv_sec;
vdso_data->wtm_clock_nsec = wtm->tv_nsec;
- if (vdso_data->tk_is_cntvct) {
+ if (!vdso_data->use_syscall) {
vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
vdso_data->xtime_clock_sec = tk->xtime_sec;
vdso_data->xtime_clock_snsec = tk->tkr_mono.xtime_nsec;
- vdso_data->cs_mult = tk->tkr_mono.mult;
+ vdso_data->cs_mono_mult = tk->tkr_mono.mult;
+ /* tkr_mono.shift == tkr_raw.shift */
vdso_data->cs_shift = tk->tkr_mono.shift;
vdso_data->cs_mask = tk->tkr_mono.mask;
}
diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c
index a9dd619c6c29..8cf13af1323c 100644
--- a/arch/arm/vdso/vgettimeofday.c
+++ b/arch/arm/vdso/vgettimeofday.c
@@ -1,18 +1,25 @@
/*
- * Copyright 2015 Mentor Graphics Corporation.
+ * Userspace implementations of gettimeofday() and friends.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; version 2 of the
- * License.
+ * Copyright (C) 2017 Cavium, Inc.
+ * Copyright (C) 2015 Mentor Graphics Corporation
+ * Copyright (C) 2012 ARM Limited
*
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Author: Will Deacon <will.deacon@arm.com>
+ * Rewriten from arch64 version into C by: Andrew Pinski <apinski@cavium.com>
+ * Reworked and rebased over arm version by: Mark Salyzyn <salyzyn@android.com>
*/
#include <linux/compiler.h>
@@ -31,32 +38,30 @@
extern struct vdso_data *__get_datapage(void);
-static notrace u32 __vdso_read_begin(const struct vdso_data *vdata)
-{
- u32 seq;
-repeat:
- seq = READ_ONCE(vdata->seq_count);
- if (seq & 1) {
- cpu_relax();
- goto repeat;
- }
- return seq;
-}
-
static notrace u32 vdso_read_begin(const struct vdso_data *vdata)
{
u32 seq;
- seq = __vdso_read_begin(vdata);
+ do {
+ seq = READ_ONCE(vdata->tb_seq_count);
+
+ if ((seq & 1) == 0)
+ break;
- smp_rmb(); /* Pairs with smp_wmb in vdso_write_end */
+ cpu_relax();
+ } while (true);
+
+ smp_rmb(); /* Pairs with second smp_wmb in update_vsyscall */
return seq;
}
static notrace int vdso_read_retry(const struct vdso_data *vdata, u32 start)
{
- smp_rmb(); /* Pairs with smp_wmb in vdso_write_begin */
- return vdata->seq_count != start;
+ u32 seq;
+
+ smp_rmb(); /* Pairs with first smp_wmb in update_vsyscall */
+ seq = READ_ONCE(vdata->tb_seq_count);
+ return seq != start;
}
static notrace long clock_gettime_fallback(clockid_t _clkid,
@@ -127,7 +132,7 @@ static notrace u64 get_ns(struct vdso_data *vdata)
cycle_delta = (cycle_now - vdata->cs_cycle_last) & vdata->cs_mask;
- nsec = (cycle_delta * vdata->cs_mult) + vdata->xtime_clock_snsec;
+ nsec = (cycle_delta * vdata->cs_mono_mult) + vdata->xtime_clock_snsec;
nsec >>= vdata->cs_shift;
return nsec;
@@ -141,7 +146,7 @@ static notrace int do_realtime(struct timespec *ts, struct vdso_data *vdata)
do {
seq = vdso_read_begin(vdata);
- if (!vdata->tk_is_cntvct)
+ if (vdata->use_syscall)
return -1;
ts->tv_sec = vdata->xtime_clock_sec;
@@ -164,7 +169,7 @@ static notrace int do_monotonic(struct timespec *ts, struct vdso_data *vdata)
do {
seq = vdso_read_begin(vdata);
- if (!vdata->tk_is_cntvct)
+ if (vdata->use_syscall)
return -1;
ts->tv_sec = vdata->xtime_clock_sec;
--
2.19.0.605.g01d371f741-goog
next prev parent reply other threads:[~2018-10-01 17:59 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-10-01 17:58 RESEND and REBASE arm+arm64+aarch32 vdso rewrite Mark Salyzyn
2018-10-01 17:58 ` Mark Salyzyn [this message]
2018-10-01 17:58 ` RESEND [PATCH v5 02/12] arm: vdso: add include file defining __get_datapage() Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH v5 03/12] arm: vdso: inline assembler operations to compiler.h Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH v5 04/12] arm: vdso: do calculations outside reader loops Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH v6 05/12] arm: vdso: Add support for CLOCK_MONOTONIC_RAW Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH v5 06/12] arm: vdso: add support for clock_getres Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH v5 07/12] arm: vdso: disable profiling Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH v5 08/12] arm: vdso: Add ARCH_CLOCK_FIXED_MASK Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH v5 09/12] arm: vdso: move vgettimeofday.c to lib/vdso/ Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH v5 10/12] arm64: vdso: replace gettimeofday.S with global vgettimeofday.C Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH v6 11/12] lib: vdso: Add support for CLOCK_BOOTTIME Mark Salyzyn
2018-10-01 18:15 ` Thomas Gleixner
2018-10-01 20:52 ` Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH v5 12/12] lib: vdso: do not expose gettimeofday, if no arch supported timer Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH] lib: vdso: add support for time Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH v3 1/3] arm64: compat: Split the sigreturn trampolines and kuser helpers (C sources) Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH v3 2/3] arm64: compat: Split the sigreturn trampolines and kuser helpers (assembler sources) Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH v3 3/3] arm64: compat: Add CONFIG_KUSER_HELPERS Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH] arm64: compat: Expose offset to registers in sigframes Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH 1/6] arm64: compat: Use vDSO sigreturn trampolines if available Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH 2/6] arm64: elf: Set AT_SYSINFO_EHDR in compat processes Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH 3/6] arm64: Refactor vDSO init/setup Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH v2 4/6] arm64: compat: Add a 32-bit vDSO Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH 5/6] arm64: compat: 32-bit vDSO setup Mark Salyzyn
2018-10-01 17:58 ` RESEND [PATCH v2 6/6] arm64: Wire up and expose the new compat vDSO Mark Salyzyn
2018-10-01 18:49 ` RESEND and REBASE arm+arm64+aarch32 vdso rewrite John Stultz
2018-10-01 20:44 ` Mark Salyzyn
2018-10-01 21:28 ` John Stultz
2018-10-02 8:50 ` Russell King - ARM Linux
2018-10-02 14:53 ` Mark Salyzyn
2018-10-02 10:00 ` Catalin Marinas
2018-10-02 15:09 ` Mark Salyzyn
-- strict thread matches above, loose matches on Subject: below --
2018-06-18 15:05 RESEND " Mark Salyzyn
2018-06-18 15:05 ` RESEND [PATCH v5 01/12] arm: vdso: rename vdso_datapage variables Mark Salyzyn
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20181001175845.168430-2-salyzyn@android.com \
--to=salyzyn@android.com \
--cc=Jeremy.Linton@arm.com \
--cc=android-kernel@android.com \
--cc=andy.gross@linaro.org \
--cc=apinski@cavium.com \
--cc=ard.biesheuvel@linaro.org \
--cc=catalin.marinas@arm.com \
--cc=dsafonov@virtuozzo.com \
--cc=james.morse@arm.com \
--cc=jinb.park7@gmail.com \
--cc=john.stultz@linaro.org \
--cc=keescook@chromium.org \
--cc=kevin.brodsky@arm.com \
--cc=labbott@redhat.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@armlinux.org.uk \
--cc=luto@amacapital.net \
--cc=mark.rutland@arm.com \
--cc=mingo@kernel.org \
--cc=paulmck@linux.vnet.ibm.com \
--cc=tglx@linutronix.de \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).