From: Benjamin Herrenschmidt <benh@kernel.crashing.org>
To: Anton Blanchard <anton@samba.org>
Cc: Michael Neuling <mikey@neuling.org>,
paulus@samba.org, sukadev@linux.vnet.ibm.com,
linuxppc-dev@lists.ozlabs.org
Subject: Re: [PATCH] powerpc: POWER7 optimised copy_to_user/copy_from_user using VMX
Date: Mon, 19 Dec 2011 14:00:52 +1100 [thread overview]
Message-ID: <1324263652.30454.13.camel@pasglop> (raw)
In-Reply-To: <20111208171145.3ab3c423@kryten>
On Thu, 2011-12-08 at 17:11 +1100, Anton Blanchard wrote:
> Implement a POWER7 optimised copy_to_user/copy_from_user using VMX.
> For large aligned copies this new loop is over 10% faster, and for
> large unaligned copies it is over 200% faster.
Breaks !CONFIG_ALTIVEC build an pops some WARN's with preempt & lockdep,
this seems to fix them:
diff --git a/arch/powerpc/lib/Makefile b/arch/powerpc/lib/Makefile
index b90b3e7..7735a2c 100644
--- a/arch/powerpc/lib/Makefile
+++ b/arch/powerpc/lib/Makefile
@@ -17,13 +17,14 @@ obj-$(CONFIG_HAS_IOMEM) += devres.o
obj-$(CONFIG_PPC64) += copypage_64.o copyuser_64.o \
memcpy_64.o usercopy_64.o mem_64.o string.o \
checksum_wrappers_64.o hweight_64.o \
- copyuser_power7.o copyuser_power7_vmx.o
+ copyuser_power7.o
obj-$(CONFIG_XMON) += sstep.o ldstfp.o
obj-$(CONFIG_KPROBES) += sstep.o ldstfp.o
obj-$(CONFIG_HAVE_HW_BREAKPOINT) += sstep.o ldstfp.o
ifeq ($(CONFIG_PPC64),y)
obj-$(CONFIG_SMP) += locks.o
+obj-$(CONFIG_ALTIVEC) += copyuser_power7_vmx.o
endif
obj-$(CONFIG_PPC_LIB_RHEAP) += rheap.o
diff --git a/arch/powerpc/lib/copyuser_power7.S b/arch/powerpc/lib/copyuser_power7.S
index 4395939..9a21b08 100644
--- a/arch/powerpc/lib/copyuser_power7.S
+++ b/arch/powerpc/lib/copyuser_power7.S
@@ -85,6 +85,7 @@
_GLOBAL(__copy_tofrom_user_power7)
+#ifdef CONFIG_ALTIVEC
cmpldi r5,16
cmpldi cr1,r5,4096
@@ -94,6 +95,15 @@ _GLOBAL(__copy_tofrom_user_power7)
blt .Lshort_copy
bgt cr1,.Lvmx_copy
+#else
+ cmpldi r5,16
+
+ std r3,48(r1)
+ std r4,56(r1)
+ std r5,64(r1)
+
+ blt .Lshort_copy
+#endif
.Lnonvmx_copy:
/* Get the source 8B aligned */
@@ -273,6 +283,7 @@ err1; stb r0,0(r3)
addi r1,r1,STACKFRAMESIZE
b .Lnonvmx_copy
+#ifdef CONFIG_ALTIVEC
.Lvmx_copy:
mflr r0
std r0,16(r1)
@@ -667,3 +678,4 @@ err3; stb r0,0(r3)
15: addi r1,r1,STACKFRAMESIZE
b .exit_vmx_copy /* tail call optimise */
+#endif /* CONFiG_ALTIVEC */
diff --git a/arch/powerpc/lib/copyuser_power7_vmx.c b/arch/powerpc/lib/copyuser_power7_vmx.c
index c37b949..6e1efad 100644
--- a/arch/powerpc/lib/copyuser_power7_vmx.c
+++ b/arch/powerpc/lib/copyuser_power7_vmx.c
@@ -26,10 +26,16 @@ int enter_vmx_copy(void)
if (in_interrupt())
return 0;
- enable_kernel_altivec();
-
+ /* This acts as preempt_disable() as well and will make
+ * enable_kernel_altivec(). We need to disable page faults
+ * as they can call schedule and thus make us lose the VMX
+ * context. So on page faults, we just fail which will cause
+ * a fallback to the normal non-vmx copy.
+ */
pagefault_disable();
+ enable_kernel_altivec();
+
return 1;
}
next prev parent reply other threads:[~2011-12-19 3:01 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-12-08 5:02 [PATCH] powerpc: POWER7 optimised copy_to_user/copy_from_user using VMX Anton Blanchard
2011-12-08 5:44 ` Kumar Gala
2011-12-08 5:54 ` Michael Neuling
2011-12-08 6:04 ` Anton Blanchard
2011-12-08 13:31 ` Segher Boessenkool
2011-12-08 14:02 ` David Laight
2011-12-08 14:12 ` Segher Boessenkool
2011-12-08 19:40 ` Benjamin Herrenschmidt
2011-12-08 6:11 ` Anton Blanchard
2011-12-19 3:00 ` Benjamin Herrenschmidt [this message]
2011-12-19 3:19 ` Benjamin Herrenschmidt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1324263652.30454.13.camel@pasglop \
--to=benh@kernel.crashing.org \
--cc=anton@samba.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mikey@neuling.org \
--cc=paulus@samba.org \
--cc=sukadev@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).