All of lore.kernel.org
 help / color / mirror / Atom feed
From: Christophe Leroy <christophe.leroy@c-s.fr>
To: Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Paul Mackerras <paulus@samba.org>,
	Michael Ellerman <mpe@ellerman.id.au>,
	nathanl@linux.ibm.com
Cc: linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org,
	arnd@arndb.de, tglx@linutronix.de, vincenzo.frascino@arm.com,
	luto@kernel.org, linux-arch@vger.kernel.org
Subject: [PATCH v8 7/8] lib/vdso: force inlining of __cvdso_clock_gettime_common()
Date: Tue, 28 Apr 2020 13:16:53 +0000 (UTC)	[thread overview]
Message-ID: <1ab6a62c356c3bec35d1623563ef9c636205bcda.1588079622.git.christophe.leroy@c-s.fr> (raw)
In-Reply-To: <cover.1588079622.git.christophe.leroy@c-s.fr>

When adding gettime64() to a 32 bit architecture (namely powerpc/32)
it has been noticed that GCC doesn't inline anymore
__cvdso_clock_gettime_common() because it is called twice
(Once by __cvdso_clock_gettime() and once by
__cvdso_clock_gettime32).

This has the effect of seriously degrading the performance:

Before the implementation of gettime64(), gettime() runs in:

	clock-gettime-monotonic-raw:    vdso: 1003 nsec/call
	clock-gettime-monotonic-coarse:    vdso: 592 nsec/call
	clock-gettime-monotonic:    vdso: 942 nsec/call

When adding a gettime64() entry point, the standard gettime()
performance is degraded by 30% to 50%:

	clock-gettime-monotonic-raw:    vdso: 1300 nsec/call
	clock-gettime-monotonic-coarse:    vdso: 900 nsec/call
	clock-gettime-monotonic:    vdso: 1232 nsec/call

Adding __always_inline() to __cvdso_clock_gettime_common()
regains the original performance.

In terms of code size, the inlining increases the code size
by only 176 bytes. This is in the noise for a kernel image.

Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
---
 lib/vdso/gettimeofday.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
index a2909af4b924..7938d3c4901d 100644
--- a/lib/vdso/gettimeofday.c
+++ b/lib/vdso/gettimeofday.c
@@ -210,7 +210,7 @@ static __always_inline int do_coarse(const struct vdso_data *vd, clockid_t clk,
 	return 0;
 }
 
-static __maybe_unused int
+static __always_inline int
 __cvdso_clock_gettime_common(const struct vdso_data *vd, clockid_t clock,
 			     struct __kernel_timespec *ts)
 {
-- 
2.25.0


WARNING: multiple messages have this Message-ID (diff)
From: Christophe Leroy <christophe.leroy@c-s.fr>
To: Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Paul Mackerras <paulus@samba.org>,
	Michael Ellerman <mpe@ellerman.id.au>,
	 nathanl@linux.ibm.com
Cc: linux-arch@vger.kernel.org, arnd@arndb.de,
	linux-kernel@vger.kernel.org, luto@kernel.org,
	tglx@linutronix.de, vincenzo.frascino@arm.com,
	linuxppc-dev@lists.ozlabs.org
Subject: [PATCH v8 7/8] lib/vdso: force inlining of __cvdso_clock_gettime_common()
Date: Tue, 28 Apr 2020 13:16:53 +0000 (UTC)	[thread overview]
Message-ID: <1ab6a62c356c3bec35d1623563ef9c636205bcda.1588079622.git.christophe.leroy@c-s.fr> (raw)
In-Reply-To: <cover.1588079622.git.christophe.leroy@c-s.fr>

When adding gettime64() to a 32 bit architecture (namely powerpc/32)
it has been noticed that GCC doesn't inline anymore
__cvdso_clock_gettime_common() because it is called twice
(Once by __cvdso_clock_gettime() and once by
__cvdso_clock_gettime32).

This has the effect of seriously degrading the performance:

Before the implementation of gettime64(), gettime() runs in:

	clock-gettime-monotonic-raw:    vdso: 1003 nsec/call
	clock-gettime-monotonic-coarse:    vdso: 592 nsec/call
	clock-gettime-monotonic:    vdso: 942 nsec/call

When adding a gettime64() entry point, the standard gettime()
performance is degraded by 30% to 50%:

	clock-gettime-monotonic-raw:    vdso: 1300 nsec/call
	clock-gettime-monotonic-coarse:    vdso: 900 nsec/call
	clock-gettime-monotonic:    vdso: 1232 nsec/call

Adding __always_inline() to __cvdso_clock_gettime_common()
regains the original performance.

In terms of code size, the inlining increases the code size
by only 176 bytes. This is in the noise for a kernel image.

Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
---
 lib/vdso/gettimeofday.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
index a2909af4b924..7938d3c4901d 100644
--- a/lib/vdso/gettimeofday.c
+++ b/lib/vdso/gettimeofday.c
@@ -210,7 +210,7 @@ static __always_inline int do_coarse(const struct vdso_data *vd, clockid_t clk,
 	return 0;
 }
 
-static __maybe_unused int
+static __always_inline int
 __cvdso_clock_gettime_common(const struct vdso_data *vd, clockid_t clock,
 			     struct __kernel_timespec *ts)
 {
-- 
2.25.0


  parent reply	other threads:[~2020-04-28 13:17 UTC|newest]

Thread overview: 71+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-28 13:16 [PATCH v8 0/8] powerpc: switch VDSO to C implementation Christophe Leroy
2020-04-28 13:16 ` Christophe Leroy
2020-04-28 13:16 ` [PATCH v8 1/8] powerpc/vdso64: Switch from __get_datapage() to get_datapage inline macro Christophe Leroy
2020-04-28 13:16   ` Christophe Leroy
2020-04-28 13:16 ` [PATCH v8 2/8] powerpc/vdso: Remove __kernel_datapage_offset and simplify __get_datapage() Christophe Leroy
2020-04-28 13:16   ` Christophe Leroy
2020-07-16  2:59   ` Michael Ellerman
2020-07-16  2:59     ` Michael Ellerman
2020-08-04 11:17     ` Christophe Leroy
2020-08-04 11:17       ` Christophe Leroy
2020-08-25 14:15       ` Christophe Leroy
2020-08-26 13:58         ` Michael Ellerman
2020-08-26 13:58           ` Michael Ellerman
2020-08-27 20:34           ` Dmitry Safonov
2020-08-27 20:34             ` Dmitry Safonov
2020-08-28  2:14             ` Michael Ellerman
2020-08-28  2:14               ` Michael Ellerman
2020-09-21 11:26               ` Will Deacon
2020-09-21 11:26                 ` Will Deacon
2020-09-27  7:43                 ` Christophe Leroy
2020-09-27  7:43                   ` Christophe Leroy
2020-09-28 15:08                   ` Dmitry Safonov
2020-09-28 15:08                     ` Dmitry Safonov
2020-10-23 11:22                     ` Christophe Leroy
2020-10-23 11:22                       ` Christophe Leroy
2020-10-23 11:25                       ` Will Deacon
2020-10-23 11:25                         ` Will Deacon
2020-10-23 11:57                         ` Christophe Leroy
2020-10-23 11:57                           ` Christophe Leroy
2020-10-23 13:29                           ` Dmitry Safonov
2020-10-23 13:29                             ` Dmitry Safonov
2020-04-28 13:16 ` [PATCH v8 3/8] powerpc/vdso: Remove unused \tmp param in __get_datapage() Christophe Leroy
2020-04-28 13:16   ` Christophe Leroy
2020-04-28 13:16 ` [PATCH v8 4/8] powerpc/processor: Move cpu_relax() into asm/vdso/processor.h Christophe Leroy
2020-04-28 13:16   ` Christophe Leroy
2020-04-28 13:16 ` [PATCH v8 5/8] powerpc/vdso: Prepare for switching VDSO to generic C implementation Christophe Leroy
2020-04-28 13:16   ` Christophe Leroy
2020-07-15  1:04   ` Michael Ellerman
2020-07-15  1:04     ` Michael Ellerman
2020-07-15 18:47     ` Christophe Leroy
2020-07-15 18:47       ` Christophe Leroy
2020-07-16 23:18       ` Tulio Magno Quites Machado Filho
2020-07-16 23:18         ` Tulio Magno Quites Machado Filho
2020-08-04 11:14     ` Christophe Leroy
2020-08-04 11:14       ` Christophe Leroy
2020-08-05  6:24       ` Michael Ellerman
2020-08-05  6:24         ` Michael Ellerman
2020-08-05 13:35         ` Segher Boessenkool
2020-08-05 13:35           ` Segher Boessenkool
2020-08-06  2:03           ` Michael Ellerman
2020-08-06  2:03             ` Michael Ellerman
2020-08-06 18:33             ` Segher Boessenkool
2020-08-06 18:33               ` Segher Boessenkool
2020-08-07  2:44               ` Michael Ellerman
2020-08-07  2:44                 ` Michael Ellerman
2020-04-28 13:16 ` [PATCH v8 6/8] powerpc/vdso: Switch " Christophe Leroy
2020-04-28 13:16   ` Christophe Leroy
2020-04-28 13:16 ` Christophe Leroy [this message]
2020-04-28 13:16   ` [PATCH v8 7/8] lib/vdso: force inlining of __cvdso_clock_gettime_common() Christophe Leroy
2020-04-28 13:16 ` [PATCH v8 8/8] powerpc/vdso: Provide __kernel_clock_gettime64() on vdso32 Christophe Leroy
2020-04-28 13:16   ` Christophe Leroy
2020-04-28 15:03   ` Christophe Leroy
2020-04-28 16:05   ` Arnd Bergmann
2020-04-28 16:05     ` Arnd Bergmann
2020-05-09 15:54     ` Christophe Leroy
2020-05-09 15:54       ` Christophe Leroy
2020-05-09 18:48       ` Christophe Leroy
2020-05-29 18:56 ` [PATCH v8 0/8] powerpc: switch VDSO to C implementation Christophe Leroy
2020-06-03 10:04   ` Michael Ellerman
2020-07-16 12:55 ` Michael Ellerman
2020-07-16 12:55   ` Michael Ellerman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1ab6a62c356c3bec35d1623563ef9c636205bcda.1588079622.git.christophe.leroy@c-s.fr \
    --to=christophe.leroy@c-s.fr \
    --cc=arnd@arndb.de \
    --cc=benh@kernel.crashing.org \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linuxppc-dev@lists.ozlabs.org \
    --cc=luto@kernel.org \
    --cc=mpe@ellerman.id.au \
    --cc=nathanl@linux.ibm.com \
    --cc=paulus@samba.org \
    --cc=tglx@linutronix.de \
    --cc=vincenzo.frascino@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.