From: Andi Kleen <andi@firstfloor.org>
To: akpm@linux-foundation.org
Cc: linux-kernel@vger.kernel.org, Andi Kleen <ak@linux.intel.com>,
x86@kernel.org
Subject: [PATCH 2/7] x86/atomic: Move __atomic_add_unless out of line
Date: Tue, 14 Mar 2017 19:14:26 -0700 [thread overview]
Message-ID: <20170315021431.13107-3-andi@firstfloor.org> (raw)
In-Reply-To: <20170315021431.13107-1-andi@firstfloor.org>
From: Andi Kleen <ak@linux.intel.com>
__atomic_add_unless is fairly big and often used, so it's quite expensive
to inline it. But it's unlikely that a call makes much difference for
such a complex function doing an expensive atomic. So out of line it.
This saves around 12k of text.
text data bss dec hex filename
9084246 5367600 11116544 25568390 1862486 vmlinux-atomic-add
9096494 5367568 11116544 25580606 186543e vmlinux-before-atomic-add
Cc: x86@kernel.org
Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
arch/x86/include/asm/atomic.h | 24 +-----------------------
arch/x86/lib/Makefile | 1 +
arch/x86/lib/atomic.c | 27 +++++++++++++++++++++++++++
3 files changed, 29 insertions(+), 23 deletions(-)
create mode 100644 arch/x86/lib/atomic.c
diff --git a/arch/x86/include/asm/atomic.h b/arch/x86/include/asm/atomic.h
index 14635c5ea025..069d69712275 100644
--- a/arch/x86/include/asm/atomic.h
+++ b/arch/x86/include/asm/atomic.h
@@ -225,29 +225,7 @@ ATOMIC_OPS(xor, ^)
#undef ATOMIC_FETCH_OP
#undef ATOMIC_OP
-/**
- * __atomic_add_unless - add unless the number is already a given value
- * @v: pointer of type atomic_t
- * @a: the amount to add to v...
- * @u: ...unless v is equal to u.
- *
- * Atomically adds @a to @v, so long as @v was not already @u.
- * Returns the old value of @v.
- */
-static __always_inline int __atomic_add_unless(atomic_t *v, int a, int u)
-{
- int c, old;
- c = atomic_read(v);
- for (;;) {
- if (unlikely(c == (u)))
- break;
- old = atomic_cmpxchg((v), c, c + (a));
- if (likely(old == c))
- break;
- c = old;
- }
- return c;
-}
+int __atomic_add_unless(atomic_t *v, int a, int u);
/**
* atomic_inc_short - increment of a short integer
diff --git a/arch/x86/lib/Makefile b/arch/x86/lib/Makefile
index 34a74131a12c..81303cefa9f4 100644
--- a/arch/x86/lib/Makefile
+++ b/arch/x86/lib/Makefile
@@ -25,6 +25,7 @@ lib-y += memcpy_$(BITS).o
lib-$(CONFIG_RWSEM_XCHGADD_ALGORITHM) += rwsem.o
lib-$(CONFIG_INSTRUCTION_DECODER) += insn.o inat.o
lib-$(CONFIG_RANDOMIZE_BASE) += kaslr.o
+lib-y += atomic.o
obj-y += msr.o msr-reg.o msr-reg-export.o hweight.o
diff --git a/arch/x86/lib/atomic.c b/arch/x86/lib/atomic.c
new file mode 100644
index 000000000000..dde7ddf67698
--- /dev/null
+++ b/arch/x86/lib/atomic.c
@@ -0,0 +1,27 @@
+#include <linux/module.h>
+#include <asm/atomic.h>
+
+/**
+ * __atomic_add_unless - add unless the number is already a given value
+ * @v: pointer of type atomic_t
+ * @a: the amount to add to v...
+ * @u: ...unless v is equal to u.
+ *
+ * Atomically adds @a to @v, so long as @v was not already @u.
+ * Returns the old value of @v.
+ */
+int __atomic_add_unless(atomic_t *v, int a, int u)
+{
+ int c, old;
+ c = atomic_read(v);
+ for (;;) {
+ if (unlikely(c == (u)))
+ break;
+ old = atomic_cmpxchg((v), c, c + (a));
+ if (likely(old == c))
+ break;
+ c = old;
+ }
+ return c;
+}
+EXPORT_SYMBOL(__atomic_add_unless);
--
2.9.3
next prev parent reply other threads:[~2017-03-15 2:15 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-03-15 2:14 Some inline debloating, 4.11 edition Andi Kleen
2017-03-15 2:14 ` [PATCH 1/7] trace: Move trace_seq_overflowed out of line Andi Kleen
2017-03-16 0:54 ` Steven Rostedt
2017-03-16 2:27 ` Andi Kleen
2017-03-16 3:20 ` Steven Rostedt
2017-03-16 3:41 ` Steven Rostedt
2017-03-15 2:14 ` Andi Kleen [this message]
2017-03-15 2:14 ` [PATCH 3/7] sched: Out of line __update_load_avg Andi Kleen
2017-03-15 2:14 ` [PATCH 4/7] kref: Remove WARN_ON for NULL release functions Andi Kleen
2017-03-15 2:46 ` Greg KH
2017-03-15 12:27 ` Peter Zijlstra
2017-03-15 2:14 ` [PATCH 5/7] Out of line dma_alloc/free_attrs Andi Kleen
2017-03-15 2:14 ` [PATCH 6/7] megasas: Remove expensive inline from megasas_return_cmd Andi Kleen
2017-03-15 12:17 ` Sumit Saxena
2017-03-15 2:14 ` [PATCH 7/7] Remove expensive WARN_ON in pagefault_disabled_dec Andi Kleen
2017-03-15 21:49 ` Andrew Morton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170315021431.13107-3-andi@firstfloor.org \
--to=andi@firstfloor.org \
--cc=ak@linux.intel.com \
--cc=akpm@linux-foundation.org \
--cc=linux-kernel@vger.kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).