From: Vincenzo Frascino <vincenzo.frascino@arm.com> To: Thomas Gleixner <tglx@linutronix.de> Cc: linux-arch@vger.kernel.org, LAK <linux-arm-kernel@lists.infradead.org>, LKML <linux-kernel@vger.kernel.org>, linux-mips@vger.kernel.org, linux-kselftest@vger.kernel.org, catalin.marinas@arm.com, Will Deacon <will.deacon@arm.com>, Arnd Bergmann <arnd@arndb.de>, linux@armlinux.org.uk, Ralf Baechle <ralf@linux-mips.org>, paul.burton@mips.com, Daniel Lezcano <daniel.lezcano@linaro.org>, salyzyn@android.com, pcc@google.com, shuah@kernel.org, 0x7f454c46@gmail.com, linux@rasmusvillemoes.dk, huw@codeweavers.com, sthotton@marvell.com, andre.przywara@arm.com, Andy Lutomirski <luto@kernel.org> Subject: Re: lib/vdso: Make delta calculation work correctly Date: Wed, 26 Jun 2019 12:08:29 +0100 Message-ID: <601481b9-aced-d03c-03a3-3de36bc58ffb@arm.com> (raw) In-Reply-To: <alpine.DEB.2.21.1906261159230.32342@nanos.tec.linutronix.de> Hi Thomas, On 26/06/2019 11:02, Thomas Gleixner wrote: > The x86 vdso implementation on which the generic vdso library is based on > has subtle (unfortunately undocumented) twists: > > 1) The code assumes that the clocksource mask is U64_MAX which means that > no bits are masked. Which is true for any valid x86 VDSO clocksource. > Stupidly it still did the mask operation for no reason and at the wrong > place right after reading the clocksource. > > 2) It contains a sanity check to catch the case where slightly > unsynchronized TSC values can be overserved which would cause the delta > calculation to make a huge jump. It therefore checks whether the > current TSC value is larger than the value on which the current > conversion is based on. If it's not larger the base value is used to > prevent time jumps. > > #1 Is not only stupid for the X86 case because it does the masking for no > reason it is also completely wrong for clocksources with a smaller mask > which can legitimately wrap around during a conversion period. The core > timekeeping code does it correct by applying the mask after the delta > calculation: > > (now - base) & mask > > #2 is equally broken for clocksources which have smaller masks and can wrap > around during a conversion period because there the now > base check is > just wrong and causes stale time stamps and time going backwards issues. > > Unbreak it by: > > 1) Removing the mask operation from the clocksource read which makes the > fallback detection work for all clocksources > > 2) Replacing the conditional delta calculation with a overrideable inline > function. > > #2 could reuse clocksource_delta() from the timekeeping code but that > results in a significant performance hit for the x86 VSDO. The timekeeping > core code must have the non optimized version as it has to operate > correctly with clocksources which have smaller masks as well to handle the > case where TSC is discarded as timekeeper clocksource and replaced by HPET > or pmtimer. For the VDSO there is no replacement clocksource. If TSC is > unusable the syscall is enforced which does the right thing. > > To accomodate to the needs of various architectures provide an overrideable > inline function which defaults to the regular delta calculation with > masking: > > (now - base) & mask > > Override it for x86 with the non-masking and checking version. > > This unbreaks the ARM64 syscall fallback operation, allows to use > clocksources with arbitrary width and preserves the performance > optimization for x86. > > Signed-off-by: Thomas Gleixner <tglx@linutronix.de> A part a typo that leads to compilation errors on non-x86 platforms the rest looks fine by me. I tested it on arm64 and behaves correctly. With this: Reviewed-by: Vincenzo Frascino <vincenzo.frascino@arm.com> > --- > arch/x86/include/asm/vdso/gettimeofday.h | 27 +++++++++++++++++++++++++++ > lib/vdso/gettimeofday.c | 19 +++++++++++++++---- > 2 files changed, 42 insertions(+), 4 deletions(-) > > --- a/arch/x86/include/asm/vdso/gettimeofday.h > +++ b/arch/x86/include/asm/vdso/gettimeofday.h > @@ -229,6 +229,33 @@ static __always_inline const struct vdso > return __vdso_data; > } > > +/* > + * x86 specific delta calculation. > + * > + * The regular implementation assumes that clocksource reads are globally > + * monotonic. The TSC can be slightly off across sockets which can cause > + * the regular delta calculation (@cycles - @last) to return a huge time > + * jump. > + * > + * Therefore it needs to be verified that @cycles are greater than > + * @last. If not then use @last, which is the base time of the current > + * conversion period. > + * > + * This variant also removes the masking of the subtraction because the > + * clocksource mask of all VDSO capable clocksources on x86 is U64_MAX > + * which would result in a pointless operation. The compiler cannot > + * optimize it away as the mask comes from the vdso data and is not compile > + * time constant. > + */ > +static __always_inline > +u64 vdso_calc_delta(u64 cycles, u64 last, u64 mask, u32 mult) > +{ > + if (cycles > last) > + return (cycles - last) * mult; > + return 0; > +} > +#define vdso_calc_delta vdso_calc_delta > + > #endif /* !__ASSEMBLY__ */ > > #endif /* __ASM_VDSO_GETTIMEOFDAY_H */ > --- a/lib/vdso/gettimeofday.c > +++ b/lib/vdso/gettimeofday.c > @@ -26,6 +26,18 @@ > #include <asm/vdso/gettimeofday.h> > #endif /* ENABLE_COMPAT_VDSO */ > > +#ifndef vdso_calc_delta > +/* > + * Default implementation which works for all sane clocksources. That > + * obviously excludes x86/TSC. > + */ > +static __always_inline > +u64 vdso_calc_delta(u64 cycles, u64 last, u64 mask, u32 mult) > +{ > + return ((cyles - last) & mask) * mult; Typo here: s/cyles/cycles/ > +} > +#endif > + > static int do_hres(const struct vdso_data *vd, clockid_t clk, > struct __kernel_timespec *ts) > { > @@ -35,14 +47,13 @@ static int do_hres(const struct vdso_dat > > do { > seq = vdso_read_begin(vd); > - cycles = __arch_get_hw_counter(vd->clock_mode) & > - vd->mask; > + cycles = __arch_get_hw_counter(vd->clock_mode); > ns = vdso_ts->nsec; > last = vd->cycle_last; > if (unlikely((s64)cycles < 0)) > return clock_gettime_fallback(clk, ts); > - if (cycles > last) > - ns += (cycles - last) * vd->mult; > + > + ns += vdso_calc_delta(cycles, last, vd->mask, vd->mult); > ns >>= vd->shift; > sec = vdso_ts->sec; > } while (unlikely(vdso_read_retry(vd, seq))); > -- Regards, Vincenzo
next prev parent reply index Thread overview: 108+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-06-21 9:52 [PATCH v7 00/25] Unify vDSOs across more architectures Vincenzo Frascino 2019-06-21 9:52 ` [PATCH v7 01/25] kernel: Standardize vdso_datapage Vincenzo Frascino 2019-06-24 13:56 ` Catalin Marinas 2019-06-21 9:52 ` [PATCH v7 02/25] kernel: Define gettimeofday vdso common code Vincenzo Frascino 2019-06-21 9:52 ` [PATCH v7 03/25] kernel: Unify update_vsyscall implementation Vincenzo Frascino 2019-06-21 10:49 ` Huw Davies 2019-06-21 9:52 ` [PATCH v7 04/25] arm64: Substitute gettimeofday with C implementation Vincenzo Frascino 2019-06-24 13:36 ` Will Deacon 2019-06-24 13:59 ` Vincenzo Frascino 2019-06-25 16:18 ` [PATCH 1/3] lib/vdso: Delay mask application in do_hres() Vincenzo Frascino 2019-06-25 16:18 ` [PATCH 2/3] arm64: Fix __arch_get_hw_counter() implementation Vincenzo Frascino 2019-06-25 16:18 ` [PATCH 3/3] arm64: compat: " Vincenzo Frascino 2019-06-25 17:02 ` [PATCH 1/3] lib/vdso: Delay mask application in do_hres() Thomas Gleixner 2019-06-25 18:27 ` Thomas Gleixner 2019-06-25 20:15 ` Andy Lutomirski 2019-06-25 22:24 ` Thomas Gleixner 2019-06-26 6:38 ` Thomas Gleixner 2019-06-26 9:25 ` Vincenzo Frascino 2019-06-26 10:02 ` lib/vdso: Make delta calculation work correctly Thomas Gleixner 2019-06-26 11:08 ` Vincenzo Frascino [this message] 2019-06-24 13:58 ` [PATCH v7 04/25] arm64: Substitute gettimeofday with C implementation Catalin Marinas 2019-06-25 15:33 ` Dave Martin 2019-06-26 13:27 ` Vincenzo Frascino 2019-06-26 16:14 ` Dave Martin 2019-06-26 19:01 ` Vincenzo Frascino 2019-06-27 10:01 ` Dave Martin 2019-06-27 10:57 ` Vincenzo Frascino 2019-06-27 11:27 ` Dave Martin 2019-06-27 11:59 ` Vincenzo Frascino 2019-06-27 14:38 ` Dave Martin 2019-06-27 15:34 ` Vincenzo Frascino 2019-06-25 17:43 ` [PATCH] arm64: vdso: Fix compilation with clang < 8 Vincenzo Frascino 2019-06-26 11:36 ` [PATCH v2] arm64: vdso: Fix compilation with clang older then 8 Vincenzo Frascino [not found] ` <CGME20190628130921eucas1p239935b0771032c331911eacc1a69dd2e@eucas1p2.samsung.com> 2019-06-28 13:09 ` [PATCH v7 04/25] arm64: Substitute gettimeofday with C implementation Marek Szyprowski 2019-06-28 14:32 ` Vincenzo Frascino 2019-06-28 16:50 ` Sylwester Nawrocki 2019-06-29 6:58 ` Vincenzo Frascino 2019-07-08 12:57 ` Sylwester Nawrocki 2019-07-08 13:09 ` Vincenzo Frascino 2019-06-21 9:52 ` [PATCH v7 05/25] arm64: Build vDSO with -ffixed-x18 Vincenzo Frascino 2019-06-21 9:52 ` [PATCH v7 06/25] arm64: compat: Add missing syscall numbers Vincenzo Frascino 2019-06-21 9:52 ` [PATCH v7 07/25] arm64: compat: Expose signal related structures Vincenzo Frascino 2019-06-21 9:52 ` [PATCH v7 08/25] arm64: compat: Generate asm offsets for signals Vincenzo Frascino 2019-06-21 9:52 ` [PATCH v7 09/25] lib: vdso: Add compat support Vincenzo Frascino 2019-06-21 9:52 ` [PATCH v7 10/25] arm64: compat: Add vDSO Vincenzo Frascino 2019-06-24 14:00 ` Catalin Marinas 2019-07-10 4:02 ` John Stultz 2019-07-10 6:12 ` Thomas Gleixner 2019-07-10 9:48 ` Vincenzo Frascino 2019-07-10 8:27 ` Will Deacon 2019-07-10 8:58 ` Thomas Gleixner 2019-07-10 9:12 ` Will Deacon 2019-07-10 9:47 ` Vincenzo Frascino 2019-07-10 13:41 ` Vincenzo Frascino 2019-07-10 13:04 ` [PATCH] arm64: vdso: Fix ABI regression in compat vdso Vincenzo Frascino 2019-07-10 13:25 ` Will Deacon 2019-07-10 13:42 ` Vincenzo Frascino 2019-07-10 14:01 ` [PATCH v2] " Vincenzo Frascino 2019-07-10 15:44 ` John Stultz 2019-07-10 15:53 ` Vincenzo Frascino 2019-07-11 9:45 ` Will Deacon 2019-07-11 10:34 ` Thomas Gleixner 2019-07-11 11:32 ` Will Deacon 2019-06-21 9:52 ` [PATCH v7 11/25] arm64: Refactor vDSO code Vincenzo Frascino 2019-06-21 9:52 ` [PATCH v7 12/25] arm64: compat: vDSO setup for compat layer Vincenzo Frascino 2019-06-21 9:52 ` [PATCH v7 13/25] arm64: elf: vDSO code page discovery Vincenzo Frascino 2019-06-21 9:52 ` [PATCH v7 14/25] arm64: compat: Get sigreturn trampolines from vDSO Vincenzo Frascino 2019-06-21 9:52 ` [PATCH v7 15/25] arm64: Add vDSO compat support Vincenzo Frascino 2019-06-21 9:52 ` [PATCH v7 16/25] arm: Add support for generic vDSO Vincenzo Frascino 2019-12-04 13:51 ` [PATCH v7 16/25] arm: Add support for generic vDSO (causing crash) Guenter Roeck 2019-12-04 13:58 ` Vincenzo Frascino 2019-12-04 16:16 ` Guenter Roeck 2019-12-04 17:15 ` Vincenzo Frascino 2019-12-04 19:39 ` Guenter Roeck 2019-12-05 9:42 ` Philippe Mathieu-Daudé 2019-12-05 10:00 ` Vincenzo Frascino 2019-12-05 11:02 ` Arnd Bergmann 2019-12-05 14:56 ` Philippe Mathieu-Daudé 2019-06-21 9:52 ` [PATCH v7 17/25] arm: Add clock_getres entry point Vincenzo Frascino 2019-06-21 9:52 ` [PATCH v7 18/25] arm: Add clock_gettime64 " Vincenzo Frascino 2019-06-21 9:52 ` [PATCH v7 19/25] mips: Add support for generic vDSO Vincenzo Frascino 2019-07-26 5:15 ` Paul Burton 2019-07-26 16:29 ` [PATCH 0/2] mips: vdso: Fix Makefile Vincenzo Frascino 2019-07-26 16:29 ` [PATCH 1/2] mips: vdso: Fix source path Vincenzo Frascino 2019-07-26 16:29 ` [PATCH 2/2] mips: vdso: Fix flip/flop vdso building bug Vincenzo Frascino 2019-07-28 22:20 ` [PATCH 0/2] mips: vdso: Fix Makefile Paul Burton 2019-06-21 9:52 ` [PATCH v7 20/25] mips: Add clock_getres entry point Vincenzo Frascino 2019-07-26 5:15 ` Paul Burton 2019-06-21 9:52 ` [PATCH v7 21/25] mips: Add clock_gettime64 " Vincenzo Frascino 2019-07-26 5:15 ` Paul Burton 2019-06-21 9:52 ` [PATCH v7 22/25] x86: Add support for generic vDSO Vincenzo Frascino 2019-06-21 9:52 ` [PATCH v7 23/25] x86: Add clock_getres entry point Vincenzo Frascino 2019-06-21 9:52 ` [PATCH v7 24/25] x86: Add clock_gettime64 " Vincenzo Frascino 2019-06-21 9:52 ` [PATCH v7 25/25] kselftest: Extend vDSO selftest Vincenzo Frascino 2019-06-24 0:34 ` [PATCH v7 00/25] Unify vDSOs across more architectures Thomas Gleixner 2019-06-24 1:15 ` Andy Lutomirski 2019-06-24 7:42 ` Thomas Gleixner 2019-06-24 13:21 ` Vincenzo Frascino 2019-06-24 14:18 ` Thomas Gleixner 2019-06-24 14:23 ` Russell King - ARM Linux admin 2019-06-24 14:49 ` Catalin Marinas 2019-06-24 16:20 ` Vincenzo Frascino 2019-10-25 11:42 ` Geert Uytterhoeven 2019-06-24 18:41 ` Paul Burton 2019-06-24 23:16 ` Vincenzo Frascino 2019-06-25 17:11 ` Paul Burton 2019-06-25 17:17 ` Vincenzo Frascino 2019-06-24 12:50 ` Andre Przywara
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=601481b9-aced-d03c-03a3-3de36bc58ffb@arm.com \ --to=vincenzo.frascino@arm.com \ --cc=0x7f454c46@gmail.com \ --cc=andre.przywara@arm.com \ --cc=arnd@arndb.de \ --cc=catalin.marinas@arm.com \ --cc=daniel.lezcano@linaro.org \ --cc=huw@codeweavers.com \ --cc=linux-arch@vger.kernel.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-kselftest@vger.kernel.org \ --cc=linux-mips@vger.kernel.org \ --cc=linux@armlinux.org.uk \ --cc=linux@rasmusvillemoes.dk \ --cc=luto@kernel.org \ --cc=paul.burton@mips.com \ --cc=pcc@google.com \ --cc=ralf@linux-mips.org \ --cc=salyzyn@android.com \ --cc=shuah@kernel.org \ --cc=sthotton@marvell.com \ --cc=tglx@linutronix.de \ --cc=will.deacon@arm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Linux-MIPS Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/linux-mips/0 linux-mips/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 linux-mips linux-mips/ https://lore.kernel.org/linux-mips \ linux-mips@vger.kernel.org public-inbox-index linux-mips Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.linux-mips AGPL code for this site: git clone https://public-inbox.org/public-inbox.git