From: Dmitry Safonov <dima@arista.com> To: linux-kernel@vger.kernel.org Cc: Dmitry Safonov <0x7f454c46@gmail.com>, Andrei Vagin <avagin@gmail.com>, Dmitry Safonov <dima@arista.com>, Adrian Reber <adrian@lisas.de>, Andrei Vagin <avagin@openvz.org>, Andy Lutomirski <luto@kernel.org>, Arnd Bergmann <arnd@arndb.de>, Christian Brauner <christian.brauner@ubuntu.com>, Cyrill Gorcunov <gorcunov@openvz.org>, "Eric W. Biederman" <ebiederm@xmission.com>, "H. Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>, Jann Horn <jannh@google.com>, Jeff Dike <jdike@addtoit.com>, Oleg Nesterov <oleg@redhat.com>, Pavel Emelyanov <xemul@virtuozzo.com>, Shuah Khan <shuah@kernel.org>, Thomas Gleixner <tglx@linutronix.de>, Vincenzo Frascino <vincenzo.frascino@arm.com>, containers@lists.linux-foundation.org, criu@openvz.org, linux-api@vger.kernel.org, x86@kernel.org Subject: [PATCHv8 01/34] lib/vdso: Add unlikely() hint into vdso_read_begin() Date: Tue, 12 Nov 2019 01:26:50 +0000 [thread overview] Message-ID: <20191112012724.250792-2-dima@arista.com> (raw) In-Reply-To: <20191112012724.250792-1-dima@arista.com> From: Andrei Vagin <avagin@gmail.com> Place the branch with no concurrent write before contended case. Performance numbers for Intel(R) Core(TM) i5-6300U CPU @ 2.40GHz (more clock_gettime() cycles - the better): | before | after ----------------------------------- | 150252214 | 153242367 | 150301112 | 153324800 | 150392773 | 153125401 | 150373957 | 153399355 | 150303157 | 153489417 | 150365237 | 153494270 ----------------------------------- avg | 150331408 | 153345935 diff % | 2 | 0 ----------------------------------- stdev % | 0.3 | 0.1 Signed-off-by: Andrei Vagin <avagin@gmail.com> Co-developed-by: Dmitry Safonov <dima@arista.com> Reviewed-by: Vincenzo Frascino <vincenzo.frascino@arm.com> Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com> Signed-off-by: Dmitry Safonov <dima@arista.com> --- include/vdso/helpers.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/vdso/helpers.h b/include/vdso/helpers.h index 01641dbb68ef..9a2af9fca45e 100644 --- a/include/vdso/helpers.h +++ b/include/vdso/helpers.h @@ -10,7 +10,7 @@ static __always_inline u32 vdso_read_begin(const struct vdso_data *vd) { u32 seq; - while ((seq = READ_ONCE(vd->seq)) & 1) + while (unlikely((seq = READ_ONCE(vd->seq)) & 1)) cpu_relax(); smp_rmb(); -- 2.24.0
WARNING: multiple messages have this Message-ID (diff)
From: Dmitry Safonov <dima@arista.com> To: linux-kernel@vger.kernel.org Cc: Dmitry Safonov <0x7f454c46@gmail.com>, Andrei Vagin <avagin@gmail.com>, Dmitry Safonov <dima@arista.com>, Adrian Reber <adrian@lisas.de>, Andrei Vagin <avagin@openvz.org>, Andy Lutomirski <luto@kernel.org>, Arnd Bergmann <arnd@arndb.de>, Christian Brauner <christian.brauner@ubuntu.com>, Cyrill Gorcunov <gorcunov@openvz.org>, "Eric W. Biederman" <ebiederm@xmission.com>, "H. Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>, Jann Horn <jannh@google.com>, Jeff Dike <jdike@addtoit.com>, Oleg Nesterov <oleg@redhat.com>, Pavel Emelyanov <xemul@virtuozzo.com>, Shuah Khan <shuah@kernel.org>, Thomas Gleixner <tglx@linutronix.de>, Vincenzo Frascino <vincenzo.frascino@arm.com>, containers@lists.linux-foundation.org, criu@openvz.org, linux-api Subject: [PATCHv8 01/34] lib/vdso: Add unlikely() hint into vdso_read_begin() Date: Tue, 12 Nov 2019 01:26:50 +0000 [thread overview] Message-ID: <20191112012724.250792-2-dima@arista.com> (raw) In-Reply-To: <20191112012724.250792-1-dima@arista.com> From: Andrei Vagin <avagin@gmail.com> Place the branch with no concurrent write before contended case. Performance numbers for Intel(R) Core(TM) i5-6300U CPU @ 2.40GHz (more clock_gettime() cycles - the better): | before | after ----------------------------------- | 150252214 | 153242367 | 150301112 | 153324800 | 150392773 | 153125401 | 150373957 | 153399355 | 150303157 | 153489417 | 150365237 | 153494270 ----------------------------------- avg | 150331408 | 153345935 diff % | 2 | 0 ----------------------------------- stdev % | 0.3 | 0.1 Signed-off-by: Andrei Vagin <avagin@gmail.com> Co-developed-by: Dmitry Safonov <dima@arista.com> Reviewed-by: Vincenzo Frascino <vincenzo.frascino@arm.com> Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com> Signed-off-by: Dmitry Safonov <dima@arista.com> --- include/vdso/helpers.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/vdso/helpers.h b/include/vdso/helpers.h index 01641dbb68ef..9a2af9fca45e 100644 --- a/include/vdso/helpers.h +++ b/include/vdso/helpers.h @@ -10,7 +10,7 @@ static __always_inline u32 vdso_read_begin(const struct vdso_data *vd) { u32 seq; - while ((seq = READ_ONCE(vd->seq)) & 1) + while (unlikely((seq = READ_ONCE(vd->seq)) & 1)) cpu_relax(); smp_rmb(); -- 2.24.0
next prev parent reply other threads:[~2019-11-12 1:27 UTC|newest] Thread overview: 160+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-11-12 1:26 [PATCHv8 00/34] kernel: Introduce Time Namespace Dmitry Safonov 2019-11-12 1:26 ` Dmitry Safonov 2019-11-12 1:26 ` Dmitry Safonov [this message] 2019-11-12 1:26 ` [PATCHv8 01/34] lib/vdso: Add unlikely() hint into vdso_read_begin() Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:26 ` [PATCHv8 02/34] lib/vdso: make do_hres and do_coarse as __always_inline Dmitry Safonov 2019-11-12 1:26 ` Dmitry Safonov 2020-01-10 9:45 ` Vincenzo Frascino 2020-01-10 9:45 ` Vincenzo Frascino 2020-01-10 11:42 ` Thomas Gleixner 2020-01-10 11:47 ` Vincenzo Frascino 2020-01-10 11:47 ` Vincenzo Frascino 2020-01-10 12:02 ` Thomas Gleixner 2020-01-10 12:18 ` Vincenzo Frascino 2020-01-10 12:18 ` Vincenzo Frascino 2020-01-13 5:27 ` Andrei Vagin 2020-01-13 5:27 ` Andrei Vagin 2020-01-13 19:09 ` [tip: timers/core] lib/vdso: Mark do_hres() and do_coarse() " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:26 ` [PATCHv8 03/34] ns: Introduce Time Namespace Dmitry Safonov 2019-11-12 1:26 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2020-01-27 14:12 ` [PATCHv8 03/34] " Dmitry Vyukov 2020-01-27 14:12 ` Dmitry Vyukov 2020-01-27 14:19 ` Dmitry Safonov 2020-01-27 14:19 ` Dmitry Safonov 2020-02-17 14:20 ` Time Namespaces: CLONE_NEWTIME and clone3()? Michael Kerrisk 2020-02-17 14:59 ` Christian Brauner 2020-02-17 21:47 ` Michael Kerrisk (man-pages) 2020-02-17 23:03 ` Christian Brauner 2020-02-17 23:29 ` Thomas Gleixner 2020-02-18 2:37 ` Eric W. Biederman 2020-02-18 17:11 ` Adrian Reber 2020-02-18 17:26 ` Christian Brauner 2019-11-12 1:26 ` [PATCHv8 04/34] time: Add timens_offsets to be used for tasks in timens Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] time: Add timens_offsets to be used for tasks in time namespace tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:26 ` [PATCHv8 05/34] posix-clocks: Rename the clock_get() callback to clock_get_timespec() Dmitry Safonov 2019-11-12 1:26 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:26 ` [PATCHv8 06/34] posix-clocks: Rename .clock_get_timespec() callbacks accordingly Dmitry Safonov 2019-11-12 1:26 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:26 ` [PATCHv8 07/34] alarmtimer: Rename gettime() callback to get_ktime() Dmitry Safonov 2019-11-12 1:26 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:26 ` [PATCHv8 08/34] alarmtimer: Provide get_timespec() callback Dmitry Safonov 2019-11-12 1:26 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:26 ` [PATCHv8 09/34] posix-clocks: Introduce clock_get_ktime() callback Dmitry Safonov 2019-11-12 1:26 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:26 ` [PATCHv8 10/34] posix-timers: Use clock_get_ktime() in common_timer_get() Dmitry Safonov 2019-11-12 1:26 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:27 ` [PATCHv8 11/34] posix-clocks: Wire up clock_gettime() with timens offsets Dmitry Safonov 2019-11-12 1:27 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:27 ` [PATCHv8 12/34] kernel: Add do_timens_ktime_to_host() helper Dmitry Safonov 2019-11-12 1:27 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] time: " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:27 ` [PATCHv8 13/34] timerfd: Make timerfd_settime() time namespace aware Dmitry Safonov 2019-11-12 1:27 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:27 ` [PATCHv8 14/34] posix-timers: Make timer_settime() " Dmitry Safonov 2019-11-12 1:27 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:27 ` [PATCHv8 15/34] alarmtimer: Make nanosleep " Dmitry Safonov 2019-11-12 1:27 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] alarmtimer: Make nanosleep() " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:27 ` [PATCHv8 16/34] hrtimers: Prepare hrtimer_nanosleep() for time namespaces Dmitry Safonov 2019-11-12 1:27 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:27 ` [PATCHv8 17/34] posix-timers: Make clock_nanosleep() time namespace aware Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:27 ` [PATCHv8 18/34] fs/proc: Respect boottime inside time namespace for /proc/uptime Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Dmitry Safonov 2020-01-14 13:02 ` tip-bot2 for Dmitry Safonov 2019-11-12 1:27 ` [PATCHv8 19/34] x86/vdso: Restrict splitting VVAR VMA Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Dmitry Safonov 2020-01-14 13:02 ` tip-bot2 for Dmitry Safonov 2019-11-12 1:27 ` [PATCHv8 20/34] lib/vdso: Prepare for time namespace support Dmitry Safonov 2019-11-12 1:27 ` Dmitry Safonov 2020-01-12 10:32 ` Thomas Gleixner 2020-01-12 10:32 ` Thomas Gleixner 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Thomas Gleixner 2020-01-14 13:02 ` tip-bot2 for Thomas Gleixner 2019-11-12 1:27 ` [PATCHv8 21/34] x86/vdso: Provide vdso_data offset on vvar_page Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Dmitry Safonov 2020-01-14 13:02 ` tip-bot2 for Dmitry Safonov 2019-11-12 1:27 ` [PATCHv8 22/34] x86/vdso: Add timens page Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] x86/vdso: Add time napespace page tip-bot2 for Dmitry Safonov 2020-01-14 13:02 ` tip-bot2 for Dmitry Safonov 2019-11-12 1:27 ` [PATCHv8 23/34] time: Allocate per-timens vvar page Dmitry Safonov 2019-11-12 1:27 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Dmitry Safonov 2020-01-14 13:02 ` tip-bot2 for Dmitry Safonov 2019-11-12 1:27 ` [PATCHv8 24/34] x86/vdso: Handle faults on timens page Dmitry Safonov 2019-11-12 1:27 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Dmitry Safonov 2020-01-14 13:02 ` tip-bot2 for Dmitry Safonov 2019-11-12 1:27 ` [PATCHv8 25/34] x86/vdso: On timens page fault prefault also VVAR page Dmitry Safonov 2019-11-12 1:27 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Dmitry Safonov 2020-01-14 13:02 ` tip-bot2 for Dmitry Safonov 2019-11-12 1:27 ` [PATCHv8 26/34] x86/vdso: Zap vvar pages on switch a time namspace Dmitry Safonov 2019-11-12 1:27 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] x86/vdso: Zap vvar pages when switching to a time namespace tip-bot2 for Dmitry Safonov 2020-01-14 13:02 ` tip-bot2 for Dmitry Safonov 2019-11-12 1:27 ` [PATCHv8 27/34] fs/proc: Introduce /proc/pid/timens_offsets Dmitry Safonov 2019-11-12 1:27 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:27 ` [PATCHv8 28/34] selftests/timens: Add Time Namespace test for supported clocks Dmitry Safonov 2019-11-12 1:27 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Dmitry Safonov 2020-01-14 13:02 ` tip-bot2 for Dmitry Safonov 2019-11-12 1:27 ` [PATCHv8 29/34] selftests/timens: Add a test for timerfd Dmitry Safonov 2019-11-12 1:27 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:27 ` [PATCHv8 30/34] selftests/timens: Add a test for clock_nanosleep() Dmitry Safonov 2019-11-12 1:27 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:27 ` [PATCHv8 31/34] selftests/timens: Add procfs selftest Dmitry Safonov 2019-11-12 1:27 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Dmitry Safonov 2020-01-14 13:02 ` tip-bot2 for Dmitry Safonov 2019-11-12 1:27 ` [PATCHv8 32/34] selftests/timens: Add timer offsets test Dmitry Safonov 2019-11-12 1:27 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:27 ` [PATCHv8 33/34] selftests/timens: Add a simple perf test for clock_gettime() Dmitry Safonov 2019-11-12 1:27 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-12 1:27 ` [PATCHv8 34/34] selftests/timens: Check for right timens offsets after fork and exec Dmitry Safonov 2019-11-12 1:27 ` Dmitry Safonov 2020-01-13 19:09 ` [tip: timers/core] " tip-bot2 for Andrei Vagin 2020-01-14 13:02 ` tip-bot2 for Andrei Vagin 2019-11-21 18:05 ` [PATCHv8 00/34] kernel: Introduce Time Namespace Andrei Vagin 2019-12-11 20:38 ` Dmitry Safonov 2020-01-09 21:09 ` Thomas Gleixner 2020-01-10 9:52 ` Vincenzo Frascino
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20191112012724.250792-2-dima@arista.com \ --to=dima@arista.com \ --cc=0x7f454c46@gmail.com \ --cc=adrian@lisas.de \ --cc=arnd@arndb.de \ --cc=avagin@gmail.com \ --cc=avagin@openvz.org \ --cc=christian.brauner@ubuntu.com \ --cc=containers@lists.linux-foundation.org \ --cc=criu@openvz.org \ --cc=ebiederm@xmission.com \ --cc=gorcunov@openvz.org \ --cc=hpa@zytor.com \ --cc=jannh@google.com \ --cc=jdike@addtoit.com \ --cc=linux-api@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=luto@kernel.org \ --cc=mingo@redhat.com \ --cc=oleg@redhat.com \ --cc=shuah@kernel.org \ --cc=tglx@linutronix.de \ --cc=vincenzo.frascino@arm.com \ --cc=x86@kernel.org \ --cc=xemul@virtuozzo.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.