From: Andrei Vagin <avagin@gmail.com>
To: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Dmitry Safonov <dima@arista.com>,
linux-kernel@vger.kernel.org,
Dmitry Safonov <0x7f454c46@gmail.com>,
Adrian Reber <adrian@lisas.de>, Andrei Vagin <avagin@openvz.org>,
Andy Lutomirski <luto@kernel.org>, Arnd Bergmann <arnd@arndb.de>,
Christian Brauner <christian.brauner@ubuntu.com>,
Cyrill Gorcunov <gorcunov@openvz.org>,
"Eric W. Biederman" <ebiederm@xmission.com>,
"H. Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>,
Jann Horn <jannh@google.com>, Jeff Dike <jdike@addtoit.com>,
Oleg Nesterov <oleg@redhat.com>,
Pavel Emelyanov <xemul@virtuozzo.com>,
Shuah Khan <shuah@kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
containers@lists.linux-foundation.org, criu@openvz.org,
linux-api@vger.kernel.org, x86@kernel.org
Subject: Re: [PATCHv7 18/33] lib/vdso: Add unlikely() hint into vdso_read_begin()
Date: Wed, 23 Oct 2019 23:13:11 -0700 [thread overview]
Message-ID: <20191024061311.GA4541@gmail.com> (raw)
In-Reply-To: <100f6921-9081-7eb0-7acc-f10cfb647c21@arm.com>
[-- Attachment #1: Type: text/plain, Size: 1415 bytes --]
On Wed, Oct 16, 2019 at 12:24:14PM +0100, Vincenzo Frascino wrote:
> On 10/11/19 2:23 AM, Dmitry Safonov wrote:
> > From: Andrei Vagin <avagin@gmail.com>
> >
> > Place the branch with no concurrent write before contended case.
> >
> > Performance numbers for Intel(R) Core(TM) i5-6300U CPU @ 2.40GHz
> > (more clock_gettime() cycles - the better):
> > | before | after
> > -----------------------------------
> > | 150252214 | 153242367
> > | 150301112 | 153324800
> > | 150392773 | 153125401
> > | 150373957 | 153399355
> > | 150303157 | 153489417
> > | 150365237 | 153494270
> > -----------------------------------
> > avg | 150331408 | 153345935
> > diff % | 2 | 0
> > -----------------------------------
> > stdev % | 0.3 | 0.1
> >
> > Signed-off-by: Andrei Vagin <avagin@gmail.com>
> > Co-developed-by: Dmitry Safonov <dima@arista.com>
> > Signed-off-by: Dmitry Safonov <dima@arista.com>
>
> Reviewed-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Hello Vincenzo,
Could you test the attached patch on aarch64? On x86, it gives about 9%
performance improvement for CLOCK_MONOTONIC and CLOCK_BOOTTIME.
Here is my test:
https://github.com/avagin/vdso-perf
It is calling clock_gettime() in a loop for three seconds and then
reports a number of iterations.
Thanks,
Andrei
[-- Attachment #2: 0001-lib-vdso-make-do_hres-and-do_coarse-as-__always_inli.patch --]
[-- Type: text/plain, Size: 2207 bytes --]
From 5252093fec4c74802e5ef501b9f1db3369430c80 Mon Sep 17 00:00:00 2001
From: Andrei Vagin <avagin@gmail.com>
Date: Tue, 22 Oct 2019 18:23:17 -0700
Subject: [PATCH] lib/vdso: make do_hres and do_coarse as __always_inline
Performance numbers for Intel(R) Core(TM) i5-6300U CPU @ 2.40GHz
(more clock_gettime() cycles - the better):
clock | before | after | diff
----------------------------------------------------------
monotonic | 153222105 | 166775025 | 8.8%
monotonic-coarse | 671557054 | 691513017 | 3.0%
monotonic-raw | 147116067 | 161057395 | 9.5%
boottime | 153446224 | 166962668 | 9.1%
Signed-off-by: Andrei Vagin <avagin@gmail.com>
---
lib/vdso/gettimeofday.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
index e630e7ff57f1..b4f7f0f246af 100644
--- a/lib/vdso/gettimeofday.c
+++ b/lib/vdso/gettimeofday.c
@@ -38,7 +38,7 @@ u64 vdso_calc_delta(u64 cycles, u64 last, u64 mask, u32 mult)
}
#endif
-static int do_hres(const struct vdso_data *vd, clockid_t clk,
+static __always_inline int do_hres(const struct vdso_data *vd, clockid_t clk,
struct __kernel_timespec *ts)
{
const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
@@ -68,7 +68,7 @@ static int do_hres(const struct vdso_data *vd, clockid_t clk,
return 0;
}
-static void do_coarse(const struct vdso_data *vd, clockid_t clk,
+static __always_inline void do_coarse(const struct vdso_data *vd, clockid_t clk,
struct __kernel_timespec *ts)
{
const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
@@ -97,12 +97,16 @@ __cvdso_clock_gettime_common(clockid_t clock, struct __kernel_timespec *ts)
*/
msk = 1U << clock;
if (likely(msk & VDSO_HRES)) {
- return do_hres(&vd[CS_HRES_COARSE], clock, ts);
+ vd = &vd[CS_HRES_COARSE];
+out_hres:
+ return do_hres(vd, clock, ts);
} else if (msk & VDSO_COARSE) {
do_coarse(&vd[CS_HRES_COARSE], clock, ts);
return 0;
} else if (msk & VDSO_RAW) {
- return do_hres(&vd[CS_RAW], clock, ts);
+ vd = &vd[CS_RAW];
+ /* goto allows to avoid extra inlining of do_hres. */
+ goto out_hres;
}
return -1;
}
--
2.21.0
next prev parent reply other threads:[~2019-10-24 6:13 UTC|newest]
Thread overview: 60+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-11 1:23 [PATCHv7 00/33] kernel: Introduce Time Namespace Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 01/33] ns: " Dmitry Safonov
2019-10-16 10:27 ` Vincenzo Frascino
2019-10-16 10:39 ` Thomas Gleixner
2019-10-16 10:44 ` Vincenzo Frascino
2019-10-16 13:57 ` Dmitry Safonov
2019-10-16 23:33 ` Andrei Vagin
2019-10-17 9:20 ` Thomas Gleixner
2019-10-17 9:47 ` Vincenzo Frascino
2019-10-17 9:23 ` Vincenzo Frascino
2019-10-11 1:23 ` [PATCHv7 02/33] time: Add timens_offsets to be used for tasks in timens Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 03/33] posix-clocks: Rename the clock_get() callback to clock_get_timespec() Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 04/33] posix-clocks: Rename .clock_get_timespec() callbacks accordingly Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 05/33] alarmtimer: Rename gettime() callback to get_ktime() Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 06/33] alarmtimer: Provide get_timespec() callback Dmitry Safonov
2019-10-14 0:36 ` kbuild test robot
2019-10-11 1:23 ` [PATCHv7 07/33] posix-clocks: Introduce clock_get_ktime() callback Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 08/33] posix-timers: Use clock_get_ktime() in common_timer_get() Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 09/33] posix-clocks: Wire up clock_gettime() with timens offsets Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 10/33] kernel: Add do_timens_ktime_to_host() helper Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 11/33] timerfd: Make timerfd_settime() time namespace aware Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 12/33] posix-timers: Make timer_settime() " Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 13/33] alarmtimer: Make nanosleep " Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 14/33] hrtimers: Prepare hrtimer_nanosleep() for time namespaces Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 15/33] posix-timers: Make clock_nanosleep() time namespace aware Dmitry Safonov
2019-10-14 0:50 ` kbuild test robot
2019-10-14 4:10 ` kbuild test robot
2019-10-14 19:58 ` Andrey Vagin
2019-10-11 1:23 ` [PATCHv7 16/33] fs/proc: Respect boottime inside time namespace for /proc/uptime Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 17/33] x86/vdso: Restrict splitting VVAR VMA Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 18/33] lib/vdso: Add unlikely() hint into vdso_read_begin() Dmitry Safonov
2019-10-16 11:24 ` Vincenzo Frascino
2019-10-24 6:13 ` Andrei Vagin [this message]
2019-10-24 9:30 ` Vincenzo Frascino
2019-10-24 13:14 ` Vincenzo Frascino
2019-10-11 1:23 ` [PATCHv7 19/33] lib/vdso: Prepare for time namespace support Dmitry Safonov
2019-10-16 14:37 ` Vincenzo Frascino
2019-10-16 15:07 ` Thomas Gleixner
2019-10-16 16:36 ` Vincenzo Frascino
2019-10-11 1:23 ` [PATCHv7 20/33] x86/vdso: Provide vdso_data offset on vvar_page Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 21/33] x86/vdso: Add timens page Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 22/33] time: Allocate per-timens vvar page Dmitry Safonov
2019-10-14 2:22 ` kbuild test robot
2019-10-14 2:34 ` kbuild test robot
2019-10-11 1:23 ` [PATCHv7 23/33] x86/vdso: Handle faults on timens page Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 24/33] x86/vdso: On timens page fault prefault also VVAR page Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 25/33] x86/vdso: Zap vvar pages on switch a time namspace Dmitry Safonov
2019-10-14 2:47 ` kbuild test robot
2019-10-14 3:11 ` kbuild test robot
2019-10-11 1:23 ` [PATCHv7 26/33] fs/proc: Introduce /proc/pid/timens_offsets Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 27/33] selftests/timens: Add Time Namespace test for supported clocks Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 28/33] selftests/timens: Add a test for timerfd Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 29/33] selftests/timens: Add a test for clock_nanosleep() Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 30/33] selftests/timens: Add procfs selftest Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 31/33] selftests/timens: Add timer offsets test Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 32/33] selftests/timens: Add a simple perf test for clock_gettime() Dmitry Safonov
2019-10-11 1:23 ` [PATCHv7 33/33] selftests/timens: Check for right timens offsets after fork and exec Dmitry Safonov
2019-10-17 9:24 ` [PATCHv7 00/33] kernel: Introduce Time Namespace Thomas Gleixner
2019-10-17 23:47 ` Andrei Vagin
2019-10-22 8:45 ` Andrei Vagin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191024061311.GA4541@gmail.com \
--to=avagin@gmail.com \
--cc=0x7f454c46@gmail.com \
--cc=adrian@lisas.de \
--cc=arnd@arndb.de \
--cc=avagin@openvz.org \
--cc=christian.brauner@ubuntu.com \
--cc=containers@lists.linux-foundation.org \
--cc=criu@openvz.org \
--cc=dima@arista.com \
--cc=ebiederm@xmission.com \
--cc=gorcunov@openvz.org \
--cc=hpa@zytor.com \
--cc=jannh@google.com \
--cc=jdike@addtoit.com \
--cc=linux-api@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=luto@kernel.org \
--cc=mingo@redhat.com \
--cc=oleg@redhat.com \
--cc=shuah@kernel.org \
--cc=tglx@linutronix.de \
--cc=vincenzo.frascino@arm.com \
--cc=x86@kernel.org \
--cc=xemul@virtuozzo.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).