linux-mips.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Vincenzo Frascino <vincenzo.frascino@arm.com>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org,
	LAK <linux-arm-kernel@lists.infradead.org>,
	LKML <linux-kernel@vger.kernel.org>,
	linux-mips@vger.kernel.org, linux-kselftest@vger.kernel.org,
	catalin.marinas@arm.com, Will Deacon <will.deacon@arm.com>,
	Arnd Bergmann <arnd@arndb.de>,
	linux@armlinux.org.uk, Ralf Baechle <ralf@linux-mips.org>,
	paul.burton@mips.com, Daniel Lezcano <daniel.lezcano@linaro.org>,
	salyzyn@android.com, pcc@google.com, shuah@kernel.org,
	0x7f454c46@gmail.com, linux@rasmusvillemoes.dk,
	huw@codeweavers.com, sthotton@marvell.com,
	andre.przywara@arm.com, Andy Lutomirski <luto@kernel.org>
Subject: Re: lib/vdso: Make delta calculation work correctly
Date: Wed, 26 Jun 2019 12:08:29 +0100	[thread overview]
Message-ID: <601481b9-aced-d03c-03a3-3de36bc58ffb@arm.com> (raw)
In-Reply-To: <alpine.DEB.2.21.1906261159230.32342@nanos.tec.linutronix.de>

Hi Thomas,

On 26/06/2019 11:02, Thomas Gleixner wrote:
> The x86 vdso implementation on which the generic vdso library is based on
> has subtle (unfortunately undocumented) twists:
> 
>  1) The code assumes that the clocksource mask is U64_MAX which means that
>     no bits are masked. Which is true for any valid x86 VDSO clocksource.
>     Stupidly it still did the mask operation for no reason and at the wrong
>     place right after reading the clocksource.
> 
>  2) It contains a sanity check to catch the case where slightly
>     unsynchronized TSC values can be overserved which would cause the delta
>     calculation to make a huge jump. It therefore checks whether the
>     current TSC value is larger than the value on which the current
>     conversion is based on. If it's not larger the base value is used to
>     prevent time jumps.
> 
> #1 Is not only stupid for the X86 case because it does the masking for no
> reason it is also completely wrong for clocksources with a smaller mask
> which can legitimately wrap around during a conversion period. The core
> timekeeping code does it correct by applying the mask after the delta
> calculation:
> 
> 	(now - base) & mask
> 
> #2 is equally broken for clocksources which have smaller masks and can wrap
> around during a conversion period because there the now > base check is
> just wrong and causes stale time stamps and time going backwards issues.
> 
> Unbreak it by:
> 
>   1) Removing the mask operation from the clocksource read which makes the
>      fallback detection work for all clocksources
> 
>   2) Replacing the conditional delta calculation with a overrideable inline
>      function.
> 
> #2 could reuse clocksource_delta() from the timekeeping code but that
> results in a significant performance hit for the x86 VSDO. The timekeeping
> core code must have the non optimized version as it has to operate
> correctly with clocksources which have smaller masks as well to handle the
> case where TSC is discarded as timekeeper clocksource and replaced by HPET
> or pmtimer. For the VDSO there is no replacement clocksource. If TSC is
> unusable the syscall is enforced which does the right thing.
> 
> To accomodate to the needs of various architectures provide an overrideable
> inline function which defaults to the regular delta calculation with
> masking:
> 
> 	(now - base) & mask
> 
> Override it for x86 with the non-masking and checking version.
> 
> This unbreaks the ARM64 syscall fallback operation, allows to use
> clocksources with arbitrary width and preserves the performance
> optimization for x86.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>

A part a typo that leads to compilation errors on non-x86 platforms the rest
looks fine by me.

I tested it on arm64 and behaves correctly.

With this:

Reviewed-by: Vincenzo Frascino <vincenzo.frascino@arm.com>

> ---
>  arch/x86/include/asm/vdso/gettimeofday.h |   27 +++++++++++++++++++++++++++
>  lib/vdso/gettimeofday.c                  |   19 +++++++++++++++----
>  2 files changed, 42 insertions(+), 4 deletions(-)
> 
> --- a/arch/x86/include/asm/vdso/gettimeofday.h
> +++ b/arch/x86/include/asm/vdso/gettimeofday.h
> @@ -229,6 +229,33 @@ static __always_inline const struct vdso
>  	return __vdso_data;
>  }
>  
> +/*
> + * x86 specific delta calculation.
> + *
> + * The regular implementation assumes that clocksource reads are globally
> + * monotonic. The TSC can be slightly off across sockets which can cause
> + * the regular delta calculation (@cycles - @last) to return a huge time
> + * jump.
> + *
> + * Therefore it needs to be verified that @cycles are greater than
> + * @last. If not then use @last, which is the base time of the current
> + * conversion period.
> + *
> + * This variant also removes the masking of the subtraction because the
> + * clocksource mask of all VDSO capable clocksources on x86 is U64_MAX
> + * which would result in a pointless operation. The compiler cannot
> + * optimize it away as the mask comes from the vdso data and is not compile
> + * time constant.
> + */
> +static __always_inline
> +u64 vdso_calc_delta(u64 cycles, u64 last, u64 mask, u32 mult)
> +{
> +	if (cycles > last)
> +		return (cycles - last) * mult;
> +	return 0;
> +}
> +#define vdso_calc_delta vdso_calc_delta
> +
>  #endif /* !__ASSEMBLY__ */
>  
>  #endif /* __ASM_VDSO_GETTIMEOFDAY_H */
> --- a/lib/vdso/gettimeofday.c
> +++ b/lib/vdso/gettimeofday.c
> @@ -26,6 +26,18 @@
>  #include <asm/vdso/gettimeofday.h>
>  #endif /* ENABLE_COMPAT_VDSO */
>  
> +#ifndef vdso_calc_delta
> +/*
> + * Default implementation which works for all sane clocksources. That
> + * obviously excludes x86/TSC.
> + */
> +static __always_inline
> +u64 vdso_calc_delta(u64 cycles, u64 last, u64 mask, u32 mult)
> +{
> +	return ((cyles - last) & mask) * mult;

Typo here:

s/cyles/cycles/

> +}
> +#endif
> +
>  static int do_hres(const struct vdso_data *vd, clockid_t clk,
>  		   struct __kernel_timespec *ts)
>  {
> @@ -35,14 +47,13 @@ static int do_hres(const struct vdso_dat
>  
>  	do {
>  		seq = vdso_read_begin(vd);
> -		cycles = __arch_get_hw_counter(vd->clock_mode) &
> -			vd->mask;
> +		cycles = __arch_get_hw_counter(vd->clock_mode);
>  		ns = vdso_ts->nsec;
>  		last = vd->cycle_last;
>  		if (unlikely((s64)cycles < 0))
>  			return clock_gettime_fallback(clk, ts);
> -		if (cycles > last)
> -			ns += (cycles - last) * vd->mult;
> +
> +		ns += vdso_calc_delta(cycles, last, vd->mask, vd->mult);
>  		ns >>= vd->shift;
>  		sec = vdso_ts->sec;
>  	} while (unlikely(vdso_read_retry(vd, seq)));
> 

-- 
Regards,
Vincenzo

  reply	other threads:[~2019-06-26 11:08 UTC|newest]

Thread overview: 108+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-06-21  9:52 [PATCH v7 00/25] Unify vDSOs across more architectures Vincenzo Frascino
2019-06-21  9:52 ` [PATCH v7 01/25] kernel: Standardize vdso_datapage Vincenzo Frascino
2019-06-24 13:56   ` Catalin Marinas
2019-06-21  9:52 ` [PATCH v7 02/25] kernel: Define gettimeofday vdso common code Vincenzo Frascino
2019-06-21  9:52 ` [PATCH v7 03/25] kernel: Unify update_vsyscall implementation Vincenzo Frascino
2019-06-21 10:49   ` Huw Davies
2019-06-21  9:52 ` [PATCH v7 04/25] arm64: Substitute gettimeofday with C implementation Vincenzo Frascino
2019-06-24 13:36   ` Will Deacon
2019-06-24 13:59     ` Vincenzo Frascino
2019-06-25 16:18     ` [PATCH 1/3] lib/vdso: Delay mask application in do_hres() Vincenzo Frascino
2019-06-25 16:18       ` [PATCH 2/3] arm64: Fix __arch_get_hw_counter() implementation Vincenzo Frascino
2019-06-25 16:18       ` [PATCH 3/3] arm64: compat: " Vincenzo Frascino
2019-06-25 17:02       ` [PATCH 1/3] lib/vdso: Delay mask application in do_hres() Thomas Gleixner
2019-06-25 18:27         ` Thomas Gleixner
2019-06-25 20:15           ` Andy Lutomirski
2019-06-25 22:24             ` Thomas Gleixner
2019-06-26  6:38         ` Thomas Gleixner
2019-06-26  9:25           ` Vincenzo Frascino
2019-06-26 10:02             ` lib/vdso: Make delta calculation work correctly Thomas Gleixner
2019-06-26 11:08               ` Vincenzo Frascino [this message]
2019-06-24 13:58   ` [PATCH v7 04/25] arm64: Substitute gettimeofday with C implementation Catalin Marinas
2019-06-25 15:33   ` Dave Martin
2019-06-26 13:27     ` Vincenzo Frascino
2019-06-26 16:14       ` Dave Martin
2019-06-26 19:01         ` Vincenzo Frascino
2019-06-27 10:01           ` Dave Martin
2019-06-27 10:57             ` Vincenzo Frascino
2019-06-27 11:27               ` Dave Martin
2019-06-27 11:59                 ` Vincenzo Frascino
2019-06-27 14:38                   ` Dave Martin
2019-06-27 15:34                     ` Vincenzo Frascino
2019-06-25 17:43   ` [PATCH] arm64: vdso: Fix compilation with clang < 8 Vincenzo Frascino
2019-06-26 11:36   ` [PATCH v2] arm64: vdso: Fix compilation with clang older then 8 Vincenzo Frascino
     [not found]   ` <CGME20190628130921eucas1p239935b0771032c331911eacc1a69dd2e@eucas1p2.samsung.com>
2019-06-28 13:09     ` [PATCH v7 04/25] arm64: Substitute gettimeofday with C implementation Marek Szyprowski
2019-06-28 14:32       ` Vincenzo Frascino
2019-06-28 16:50         ` Sylwester Nawrocki
2019-06-29  6:58           ` Vincenzo Frascino
2019-07-08 12:57             ` Sylwester Nawrocki
2019-07-08 13:09               ` Vincenzo Frascino
2019-06-21  9:52 ` [PATCH v7 05/25] arm64: Build vDSO with -ffixed-x18 Vincenzo Frascino
2019-06-21  9:52 ` [PATCH v7 06/25] arm64: compat: Add missing syscall numbers Vincenzo Frascino
2019-06-21  9:52 ` [PATCH v7 07/25] arm64: compat: Expose signal related structures Vincenzo Frascino
2019-06-21  9:52 ` [PATCH v7 08/25] arm64: compat: Generate asm offsets for signals Vincenzo Frascino
2019-06-21  9:52 ` [PATCH v7 09/25] lib: vdso: Add compat support Vincenzo Frascino
2019-06-21  9:52 ` [PATCH v7 10/25] arm64: compat: Add vDSO Vincenzo Frascino
2019-06-24 14:00   ` Catalin Marinas
2019-07-10  4:02   ` John Stultz
2019-07-10  6:12     ` Thomas Gleixner
2019-07-10  9:48       ` Vincenzo Frascino
2019-07-10  8:27     ` Will Deacon
2019-07-10  8:58       ` Thomas Gleixner
2019-07-10  9:12         ` Will Deacon
2019-07-10  9:47     ` Vincenzo Frascino
2019-07-10 13:41       ` Vincenzo Frascino
2019-07-10 13:04   ` [PATCH] arm64: vdso: Fix ABI regression in compat vdso Vincenzo Frascino
2019-07-10 13:25     ` Will Deacon
2019-07-10 13:42       ` Vincenzo Frascino
2019-07-10 14:01   ` [PATCH v2] " Vincenzo Frascino
2019-07-10 15:44     ` John Stultz
2019-07-10 15:53       ` Vincenzo Frascino
2019-07-11  9:45     ` Will Deacon
2019-07-11 10:34       ` Thomas Gleixner
2019-07-11 11:32         ` Will Deacon
2019-06-21  9:52 ` [PATCH v7 11/25] arm64: Refactor vDSO code Vincenzo Frascino
2019-06-21  9:52 ` [PATCH v7 12/25] arm64: compat: vDSO setup for compat layer Vincenzo Frascino
2019-06-21  9:52 ` [PATCH v7 13/25] arm64: elf: vDSO code page discovery Vincenzo Frascino
2019-06-21  9:52 ` [PATCH v7 14/25] arm64: compat: Get sigreturn trampolines from vDSO Vincenzo Frascino
2019-06-21  9:52 ` [PATCH v7 15/25] arm64: Add vDSO compat support Vincenzo Frascino
2019-06-21  9:52 ` [PATCH v7 16/25] arm: Add support for generic vDSO Vincenzo Frascino
2019-12-04 13:51   ` [PATCH v7 16/25] arm: Add support for generic vDSO (causing crash) Guenter Roeck
2019-12-04 13:58     ` Vincenzo Frascino
2019-12-04 16:16       ` Guenter Roeck
2019-12-04 17:15         ` Vincenzo Frascino
2019-12-04 19:39           ` Guenter Roeck
2019-12-05  9:42           ` Philippe Mathieu-Daudé
2019-12-05 10:00             ` Vincenzo Frascino
2019-12-05 11:02               ` Arnd Bergmann
2019-12-05 14:56                 ` Philippe Mathieu-Daudé
2019-06-21  9:52 ` [PATCH v7 17/25] arm: Add clock_getres entry point Vincenzo Frascino
2019-06-21  9:52 ` [PATCH v7 18/25] arm: Add clock_gettime64 " Vincenzo Frascino
2019-06-21  9:52 ` [PATCH v7 19/25] mips: Add support for generic vDSO Vincenzo Frascino
2019-07-26  5:15   ` Paul Burton
2019-07-26 16:29     ` [PATCH 0/2] mips: vdso: Fix Makefile Vincenzo Frascino
2019-07-26 16:29       ` [PATCH 1/2] mips: vdso: Fix source path Vincenzo Frascino
2019-07-26 16:29       ` [PATCH 2/2] mips: vdso: Fix flip/flop vdso building bug Vincenzo Frascino
2019-07-28 22:20       ` [PATCH 0/2] mips: vdso: Fix Makefile Paul Burton
2019-06-21  9:52 ` [PATCH v7 20/25] mips: Add clock_getres entry point Vincenzo Frascino
2019-07-26  5:15   ` Paul Burton
2019-06-21  9:52 ` [PATCH v7 21/25] mips: Add clock_gettime64 " Vincenzo Frascino
2019-07-26  5:15   ` Paul Burton
2019-06-21  9:52 ` [PATCH v7 22/25] x86: Add support for generic vDSO Vincenzo Frascino
2019-06-21  9:52 ` [PATCH v7 23/25] x86: Add clock_getres entry point Vincenzo Frascino
2019-06-21  9:52 ` [PATCH v7 24/25] x86: Add clock_gettime64 " Vincenzo Frascino
2019-06-21  9:52 ` [PATCH v7 25/25] kselftest: Extend vDSO selftest Vincenzo Frascino
2019-06-24  0:34 ` [PATCH v7 00/25] Unify vDSOs across more architectures Thomas Gleixner
2019-06-24  1:15   ` Andy Lutomirski
2019-06-24  7:42     ` Thomas Gleixner
2019-06-24 13:21   ` Vincenzo Frascino
2019-06-24 14:18   ` Thomas Gleixner
2019-06-24 14:23     ` Russell King - ARM Linux admin
2019-06-24 14:49       ` Catalin Marinas
2019-06-24 16:20         ` Vincenzo Frascino
2019-10-25 11:42         ` Geert Uytterhoeven
2019-06-24 18:41   ` Paul Burton
2019-06-24 23:16     ` Vincenzo Frascino
2019-06-25 17:11       ` Paul Burton
2019-06-25 17:17         ` Vincenzo Frascino
2019-06-24 12:50 ` Andre Przywara

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=601481b9-aced-d03c-03a3-3de36bc58ffb@arm.com \
    --to=vincenzo.frascino@arm.com \
    --cc=0x7f454c46@gmail.com \
    --cc=andre.przywara@arm.com \
    --cc=arnd@arndb.de \
    --cc=catalin.marinas@arm.com \
    --cc=daniel.lezcano@linaro.org \
    --cc=huw@codeweavers.com \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=linux-mips@vger.kernel.org \
    --cc=linux@armlinux.org.uk \
    --cc=linux@rasmusvillemoes.dk \
    --cc=luto@kernel.org \
    --cc=paul.burton@mips.com \
    --cc=pcc@google.com \
    --cc=ralf@linux-mips.org \
    --cc=salyzyn@android.com \
    --cc=shuah@kernel.org \
    --cc=sthotton@marvell.com \
    --cc=tglx@linutronix.de \
    --cc=will.deacon@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).