Cannot unwind through MIPS signal frames with ICACHE_REFILLS_WORKAROUND_WAR

Cannot unwind through MIPS signal frames with ICACHE_REFILLS_WORKAROUND_WAR

[David Daney]

With the current kernel (2.6.23.1) in my R5000 based O2 it seems 
impossible for GCC's exception unwinding machinery to unwind through 
signal frames.  The cause of the problems is the 
ICACHE_REFILLS_WORKAROUND_WAR which puts the sigcontext at an almost 
impossible to determine offset from the signal return trampoline.  The 
unwinder depends on being able to find the sigcontext given a known 
location of the trampoline.

It seems there are a couple of possible solutions:

1) The comments in war.h indicate the problem only exists in R7000 and 
E9000 processors.  We could turn off the workaround if the kernel is 
configured for R5000.  That would help me, but not those with the 
effected systems.

2) In the non-workaround case, the siginfo immediately follows the 
trampoline and the first member is the signal number.  For the 
workaround case the first word following the trampoline is zero.  We 
could replace this with the offset to the sigcontext which is always a 
small negative value.  The unwinder could then distinguish the two cases 
(signal numbers are positive and the offset negative).  If we did this, 
the change would have to be coordinated with GCC's unwinder (in 
libgcc_s.so.1).

Thoughts?

David Daney

Re: Cannot unwind through MIPS signal frames with ICACHE_REFILLS_WORKAROUND_WAR

[Andrew Haley]

David Daney writes:
 > With the current kernel (2.6.23.1) in my R5000 based O2 it seems 
 > impossible for GCC's exception unwinding machinery to unwind through 
 > signal frames.  The cause of the problems is the 
 > ICACHE_REFILLS_WORKAROUND_WAR which puts the sigcontext at an almost 
 > impossible to determine offset from the signal return trampoline.  The 
 > unwinder depends on being able to find the sigcontext given a known 
 > location of the trampoline.
 > 
 > It seems there are a couple of possible solutions:
 > 
 > 1) The comments in war.h indicate the problem only exists in R7000
 > and E9000 processors.  We could turn off the workaround if the
 > kernel is configured for R5000.  That would help me, but not those
 > with the effected systems.
 > 
 > 2) In the non-workaround case, the siginfo immediately follows the
 > trampoline and the first member is the signal number.  For the
 > workaround case the first word following the trampoline is zero.
 > We could replace this with the offset to the sigcontext which is
 > always a small negative value.  The unwinder could then distinguish
 > the two cases (signal numbers are positive and the offset
 > negative).  If we did this, the change would have to be coordinated
 > with GCC's unwinder (in libgcc_s.so.1).
 > 
 > Thoughts?

The best solution is to put the unwinder info in the kernel.  Does
MIPS use a vDSO ?

Andrew.

Re: Cannot unwind through MIPS signal frames with ICACHE_REFILLS_WORKAROUND_WAR

[Ralf Baechle]

On Tue, Nov 13, 2007 at 11:48:53AM +0000, Andrew Haley wrote:

> David Daney writes:
>  > With the current kernel (2.6.23.1) in my R5000 based O2 it seems 
>  > impossible for GCC's exception unwinding machinery to unwind through 
>  > signal frames.  The cause of the problems is the 
>  > ICACHE_REFILLS_WORKAROUND_WAR which puts the sigcontext at an almost 
>  > impossible to determine offset from the signal return trampoline.  The 
>  > unwinder depends on being able to find the sigcontext given a known 
>  > location of the trampoline.
>  > 
>  > It seems there are a couple of possible solutions:
>  > 
>  > 1) The comments in war.h indicate the problem only exists in R7000
>  > and E9000 processors.  We could turn off the workaround if the
>  > kernel is configured for R5000.  That would help me, but not those
>  > with the effected systems.
>  > 
>  > 2) In the non-workaround case, the siginfo immediately follows the
>  > trampoline and the first member is the signal number.  For the
>  > workaround case the first word following the trampoline is zero.
>  > We could replace this with the offset to the sigcontext which is
>  > always a small negative value.  The unwinder could then distinguish
>  > the two cases (signal numbers are positive and the offset
>  > negative).  If we did this, the change would have to be coordinated
>  > with GCC's unwinder (in libgcc_s.so.1).
>  > 
>  > Thoughts?
> 
> The best solution is to put the unwinder info in the kernel.  Does
> MIPS use a vDSO ?

No though we should.

Another reason is to get rid of the classic trampoline the kernel installs
on the stack.  On some multiprocessor systems it requires a cacheflush
operation to be performed on all processors which is expensive.  Having
the trampoline in a vDSO would solve that.

I need to look into it, not sure what it would take.

  Ralf

Re: Cannot unwind through MIPS signal frames with ICACHE_REFILLS_WORKAROUND_WAR

[Franck Bui-Huu]

On Nov 13, 2007 1:10 PM, Ralf Baechle <ralf@linux-mips.org> wrote:
>
> On Tue, Nov 13, 2007 at 11:48:53AM +0000, Andrew Haley wrote:
>
> > David Daney writes:
> >  > With the current kernel (2.6.23.1) in my R5000 based O2 it seems
> >  > impossible for GCC's exception unwinding machinery to unwind through
> >  > signal frames.  The cause of the problems is the
> >  > ICACHE_REFILLS_WORKAROUND_WAR which puts the sigcontext at an almost
> >  > impossible to determine offset from the signal return trampoline.  The
> >  > unwinder depends on being able to find the sigcontext given a known
> >  > location of the trampoline.
> >  >
> >  > It seems there are a couple of possible solutions:
> >  >
> >  > 1) The comments in war.h indicate the problem only exists in R7000
> >  > and E9000 processors.  We could turn off the workaround if the
> >  > kernel is configured for R5000.  That would help me, but not those
> >  > with the effected systems.
> >  >
> >  > 2) In the non-workaround case, the siginfo immediately follows the
> >  > trampoline and the first member is the signal number.  For the
> >  > workaround case the first word following the trampoline is zero.
> >  > We could replace this with the offset to the sigcontext which is
> >  > always a small negative value.  The unwinder could then distinguish
> >  > the two cases (signal numbers are positive and the offset
> >  > negative).  If we did this, the change would have to be coordinated
> >  > with GCC's unwinder (in libgcc_s.so.1).
> >  >
> >  > Thoughts?
> >
> > The best solution is to put the unwinder info in the kernel.  Does
> > MIPS use a vDSO ?
>
> No though we should.
>
> Another reason is to get rid of the classic trampoline the kernel installs
> on the stack.  On some multiprocessor systems it requires a cacheflush
> operation to be performed on all processors which is expensive.  Having
> the trampoline in a vDSO would solve that.
>

And the stack wouldn't need to have exec permission anymore.

> I need to look into it, not sure what it would take.
>

I started to add vdso support for MIPS a couple months ago, but
it's in a very early stage and I unfortunately haven't time to finish
it. I can send it to you if you want.

-- 
               Franck

Re: Cannot unwind through MIPS signal frames with ICACHE_REFILLS_WORKAROUND_WAR

[Ralf Baechle]

On Tue, Nov 13, 2007 at 02:14:58PM +0100, Franck Bui-Huu wrote:

> > > David Daney writes:
> > >  > With the current kernel (2.6.23.1) in my R5000 based O2 it seems
> > >  > impossible for GCC's exception unwinding machinery to unwind through
> > >  > signal frames.  The cause of the problems is the
> > >  > ICACHE_REFILLS_WORKAROUND_WAR which puts the sigcontext at an almost
> > >  > impossible to determine offset from the signal return trampoline.  The
> > >  > unwinder depends on being able to find the sigcontext given a known
> > >  > location of the trampoline.
> > >  >
> > >  > It seems there are a couple of possible solutions:
> > >  >
> > >  > 1) The comments in war.h indicate the problem only exists in R7000
> > >  > and E9000 processors.  We could turn off the workaround if the
> > >  > kernel is configured for R5000.  That would help me, but not those
> > >  > with the effected systems.
> > >  >
> > >  > 2) In the non-workaround case, the siginfo immediately follows the
> > >  > trampoline and the first member is the signal number.  For the
> > >  > workaround case the first word following the trampoline is zero.
> > >  > We could replace this with the offset to the sigcontext which is
> > >  > always a small negative value.  The unwinder could then distinguish
> > >  > the two cases (signal numbers are positive and the offset
> > >  > negative).  If we did this, the change would have to be coordinated
> > >  > with GCC's unwinder (in libgcc_s.so.1).
> > >  >
> > >  > Thoughts?
> > >
> > > The best solution is to put the unwinder info in the kernel.  Does
> > > MIPS use a vDSO ?
> >
> > No though we should.
> >
> > Another reason is to get rid of the classic trampoline the kernel installs
> > on the stack.  On some multiprocessor systems it requires a cacheflush
> > operation to be performed on all processors which is expensive.  Having
> > the trampoline in a vDSO would solve that.
> >
> 
> And the stack wouldn't need to have exec permission anymore.

Oh?

extern void frob(void (*)(void));

int foo(void)
{
	int x;

	void bar(void)
	{
		x++;
	}

	frob(&bar);
	print("x is %d\n", x);
}

Compile and enjoy.

  Ralf

Re: Cannot unwind through MIPS signal frames with ICACHE_REFILLS_WORKAROUND_WAR

[Franck Bui-Huu]

On Nov 13, 2007 3:00 PM, Ralf Baechle <ralf@linux-mips.org> wrote:
>
> On Tue, Nov 13, 2007 at 02:14:58PM +0100, Franck Bui-Huu wrote:
>
> > > > David Daney writes:
> > > >  > With the current kernel (2.6.23.1) in my R5000 based O2 it seems
> > > >  > impossible for GCC's exception unwinding machinery to unwind through
> > > >  > signal frames.  The cause of the problems is the
> > > >  > ICACHE_REFILLS_WORKAROUND_WAR which puts the sigcontext at an almost
> > > >  > impossible to determine offset from the signal return trampoline.  The
> > > >  > unwinder depends on being able to find the sigcontext given a known
> > > >  > location of the trampoline.
> > > >  >
> > > >  > It seems there are a couple of possible solutions:
> > > >  >
> > > >  > 1) The comments in war.h indicate the problem only exists in R7000
> > > >  > and E9000 processors.  We could turn off the workaround if the
> > > >  > kernel is configured for R5000.  That would help me, but not those
> > > >  > with the effected systems.
> > > >  >
> > > >  > 2) In the non-workaround case, the siginfo immediately follows the
> > > >  > trampoline and the first member is the signal number.  For the
> > > >  > workaround case the first word following the trampoline is zero.
> > > >  > We could replace this with the offset to the sigcontext which is
> > > >  > always a small negative value.  The unwinder could then distinguish
> > > >  > the two cases (signal numbers are positive and the offset
> > > >  > negative).  If we did this, the change would have to be coordinated
> > > >  > with GCC's unwinder (in libgcc_s.so.1).
> > > >  >
> > > >  > Thoughts?
> > > >
> > > > The best solution is to put the unwinder info in the kernel.  Does
> > > > MIPS use a vDSO ?
> > >
> > > No though we should.
> > >
> > > Another reason is to get rid of the classic trampoline the kernel installs
> > > on the stack.  On some multiprocessor systems it requires a cacheflush
> > > operation to be performed on all processors which is expensive.  Having
> > > the trampoline in a vDSO would solve that.
> > >
> >
> > And the stack wouldn't need to have exec permission anymore.
>
> Oh?
>
> extern void frob(void (*)(void));
>
> int foo(void)
> {
>         int x;
>
>         void bar(void)
>         {
>                 x++;
>         }
>
>         frob(&bar);
>         print("x is %d\n", x);
> }
>
> Compile and enjoy.
>

Sorry Ralf, I missed your point.

-- 
               Franck

Re: Cannot unwind through MIPS signal frames with ICACHE_REFILLS_WORKAROUND_WAR

[Kevin D. Kissell]

Franck a dit:
> > Another reason is to get rid of the classic trampoline the kernel installs
> > on the stack.  On some multiprocessor systems it requires a cacheflush
> > operation to be performed on all processors which is expensive.  Having
> > the trampoline in a vDSO would solve that.
> >
> 
> And the stack wouldn't need to have exec permission anymore.

True, though it should perhaps be noted that currently it's only on 4KSc/Sd
systems (which I know you work on) where it's even possible for the stack
*not* to have exec permissions, since the classical MIPS MMU gives
execute permission to any page that is readable.

            Regards,

            Kevin K.

Re: Cannot unwind through MIPS signal frames with ICACHE_REFILLS_WORKAROUND_WAR

[Kevin D. Kissell]

Franck a dit:
> > Another reason is to get rid of the classic trampoline the kernel installs
> > on the stack.  On some multiprocessor systems it requires a cacheflush
> > operation to be performed on all processors which is expensive.  Having
> > the trampoline in a vDSO would solve that.
> >
> 
> And the stack wouldn't need to have exec permission anymore.

True, though it should perhaps be noted that currently it's only on 4KSc/Sd
systems (which I know you work on) where it's even possible for the stack
*not* to have exec permissions, since the classical MIPS MMU gives
execute permission to any page that is readable.

            Regards,

            Kevin K.

Re: Cannot unwind through MIPS signal frames with ICACHE_REFILLS_WORKAROUND_WAR

[Franck Bui-Huu]

On Nov 13, 2007 3:37 PM, Kevin D. Kissell <kevink@mips.com> wrote:
> Franck a dit:
> > > Another reason is to get rid of the classic trampoline the kernel installs
> > > on the stack.  On some multiprocessor systems it requires a cacheflush
> > > operation to be performed on all processors which is expensive.  Having
> > > the trampoline in a vDSO would solve that.
> > >
> >
> > And the stack wouldn't need to have exec permission anymore.
>
> True, though it should perhaps be noted that currently it's only on 4KSc/Sd
> systems (which I know you work on) where it's even possible for the stack
> *not* to have exec permissions, since the classical MIPS MMU gives
> execute permission to any page that is readable.
>

Well, the noexec stack is pretty usefull I think. I can't believe it
will be limited to these 2 systems in the near future...

-- 
               Franck

Re: Cannot unwind through MIPS signal frames with ICACHE_REFILLS_WORKAROUND_WAR

[Ralf Baechle]

On Tue, Nov 13, 2007 at 03:22:33PM +0100, Franck Bui-Huu wrote:

> > > And the stack wouldn't need to have exec permission anymore.
> >
> > Oh?
> >
> > extern void frob(void (*)(void));
> >
> > int foo(void)
> > {
> >         int x;
> >
> >         void bar(void)
> >         {
> >                 x++;
> >         }
> >
> >         frob(&bar);
> >         print("x is %d\n", x);
> > }
> >
> > Compile and enjoy.
> >
> 
> Sorry Ralf, I missed your point.

This piece of code compiles to something that copies a trampoline to the
stack.  The address of that trampoline is what is then passed as argument
to frob().

Old versions of glibc were probable the most notorious users of trampolines.
Objective C also generates them.  Since a cacheflush that is a syscall is
required performance is less than great.

Which means the libc() cacheflush() function is another candidate for a
vDSO, it can be optimized by using SYNCI on some configurations.

  Ralf

Re: Cannot unwind through MIPS signal frames with ICACHE_REFILLS_WORKAROUND_WAR

[Ralf Baechle]

On Tue, Nov 13, 2007 at 03:37:39PM +0100, Kevin D. Kissell wrote:

> True, though it should perhaps be noted that currently it's only on 4KSc/Sd
> systems (which I know you work on) where it's even possible for the stack
> *not* to have exec permissions, since the classical MIPS MMU gives
> execute permission to any page that is readable.

Disabling PROT_EXEC on a mapping tells the kernel it doesn't need to take
care of I-cache coherency.  So it's somewhat beneficial even in absence of
a protection bit in the actual TLB hardware.

Some of these performance optimizations are impossible because the kernel
can't have definate knowledge that certain addresses have never entered the
I-cache.

  Ralf

Re: Cannot unwind through MIPS signal frames with ICACHE_REFILLS_WORKAROUND_WAR

[David Daney]

David Daney wrote:
> With the current kernel (2.6.23.1) in my R5000 based O2 it seems 
> impossible for GCC's exception unwinding machinery to unwind through 
> signal frames.  The cause of the problems is the 
> ICACHE_REFILLS_WORKAROUND_WAR which puts the sigcontext at an almost 
> impossible to determine offset from the signal return trampoline.  The 
> unwinder depends on being able to find the sigcontext given a known 
> location of the trampoline.
>
> It seems there are a couple of possible solutions:
>
> 1) The comments in war.h indicate the problem only exists in R7000 and 
> E9000 processors.  We could turn off the workaround if the kernel is 
> configured for R5000.  That would help me, but not those with the 
> effected systems.
>
> 2) In the non-workaround case, the siginfo immediately follows the 
> trampoline and the first member is the signal number.  For the 
> workaround case the first word following the trampoline is zero.  We 
> could replace this with the offset to the sigcontext which is always a 
> small negative value.  The unwinder could then distinguish the two 
> cases (signal numbers are positive and the offset negative).  If we 
> did this, the change would have to be coordinated with GCC's unwinder 
> (in libgcc_s.so.1).
>
I think I have found a solution that doesn't require kernel changes.

The CFA (i.e. the stack pointer of the signal handler) of the the 
context when calling mips_fallback_frame_state is at a constant offset 
from the sigcontext.  I can just use the CFA instead of the trampoline's 
address.

Does this seem plausible?

Thanks,
David Daney

VDSO on mips (was Re: Cannot unwind through MIPS signal frames with ICACHE_REFILLS_WORKAROUND_WAR)

[Franck Bui-Huu]

Franck Bui-Huu wrote:
> 
> I started to add vdso support for MIPS a couple months ago, but
> it's in a very early stage and I unfortunately haven't time to finish
> it. I can send it to you if you want.
> 

Here it is. As I said it far from complete but it might help.

		Franck

--- 8< ---

diff --git a/arch/mips/kernel/Makefile b/arch/mips/kernel/Makefile
index 2fd96d9..01d700c 100644
--- a/arch/mips/kernel/Makefile
+++ b/arch/mips/kernel/Makefile
@@ -6,11 +6,13 @@ extra-y		:= head.o init_task.o vmlinux.lds
 
 obj-y		+= cpu-probe.o branch.o entry.o genex.o irq.o process.o \
 		   ptrace.o reset.o semaphore.o setup.o signal.o syscall.o \
-		   time.o topology.o traps.o unaligned.o
+		   time.o topology.o traps.o unaligned.o vdso.o
 
 binfmt_irix-objs	:= irixelf.o irixinv.o irixioctl.o irixsig.o	\
 			   irix5sys.o sysirix.o
 
+obj-$(CONFIG_32BIT)		+= vdso32/
+
 obj-$(CONFIG_STACKTRACE)	+= stacktrace.o
 obj-$(CONFIG_MODULES)		+= mips_ksyms.o module.o
 
diff --git a/arch/mips/kernel/vdso.c b/arch/mips/kernel/vdso.c
new file mode 100644
index 0000000..281b7ce
--- /dev/null
+++ b/arch/mips/kernel/vdso.c
@@ -0,0 +1,52 @@
+#include <linux/init.h>
+
+typedef struct {
+        unsigned long id;
+        unsigned long vdso_base;
+} mm_context_t;
+
+
+static int vdso_enabled __read_mostly = 1;
+
+static int __init vdso_setup(char *s)
+{
+	vdso_enabled = simple_strtol(s, NULL, 0);
+	return 1;
+}
+__setup("vdso=", vdso_setup);
+
+
+int arch_setup_additional_pages(struct linux_binprm *bprm, int exec_stack)
+{
+	struct mm_struct *mm = current->mm;
+	unsigned long vdso_pages;
+	unsigned long vdso_base;
+	int rv;
+
+	if (!vdso_enabled)
+		return 0;
+
+	down_write(&mm->mmap_sem);
+
+	rv = get_unmapped_area(NULL, vdso_base, vdso_pages << PAGE_SHIFT, 0, 0);
+	if (IS_ERR_VALUE(rv))
+		goto out;
+	vdso_base = rv;
+
+	rv = install_special_mapping(mm, vdso_base, vdso_pages << PAGE_SHIFT,
+				     VM_READ|VM_EXEC|
+				     VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC|
+				     VM_ALWAYSDUMP,
+				     vdso_pagelist);
+	if (rv)
+		goto out;
+out:
+	up_write(&mm->mmap_sem);
+	return rv;
+}
+
+static int __init vdso_init(void)
+{
+        return 0;
+}
+arch_initcall(vdso_init);
diff --git a/arch/mips/kernel/vdso32/.gitignore b/arch/mips/kernel/vdso32/.gitignore
new file mode 100644
index 0000000..e45fba9
--- /dev/null
+++ b/arch/mips/kernel/vdso32/.gitignore
@@ -0,0 +1 @@
+vdso32.lds
diff --git a/arch/mips/kernel/vdso32/Makefile b/arch/mips/kernel/vdso32/Makefile
new file mode 100644
index 0000000..b1ea645
--- /dev/null
+++ b/arch/mips/kernel/vdso32/Makefile
@@ -0,0 +1,35 @@
+# List of files in the vdso
+obj-vdso32 = sigtramp.o
+
+# Build rules
+targets := $(obj-vdso32) vdso32.so
+obj-vdso32 := $(addprefix $(obj)/, $(obj-vdso32))
+
+
+EXTRA_CFLAGS := -shared -s -fno-common -fno-builtin
+EXTRA_CFLAGS += -nostdlib -Wl,-soname=linux-vdso32.so.1
+EXTRA_CFLAGS +=	$(call ld-option, -Wl$(comma)--hash-style=sysv)
+
+EXTRA_AFLAGS := -D__VDSO32__ -s
+
+obj-y += vdso32.o
+extra-y += vdso32.lds
+CPPFLAGS_vdso32.lds += -P -C -U$(ARCH)
+
+# kbuild does not track this dependency due to usage of .incbin
+$(obj)/vdso32.o : $(obj)/vdso32.so
+
+# link rule for the .so file, .lds has to be first
+$(obj)/vdso32.so: $(src)/vdso32.lds $(obj-vdso32)
+	$(call if_changed,vdso32ld)
+
+# assembly rules for the .S files
+$(obj-vdso32): %.o: %.S
+	$(call if_changed_dep,vdso32as)
+
+# actual build commands
+quiet_cmd_vdso32ld = VDSO32_LD $@
+      cmd_vdso32ld = $(CC) $(c_flags) -Wl,-T $^ -o $@
+quiet_cmd_vdso32as = VDSO32_AS $@
+      cmd_vdso32as = $(CC) $(a_flags) -c -o $@ $<
+
diff --git a/arch/mips/kernel/vdso32/sigtramp.S b/arch/mips/kernel/vdso32/sigtramp.S
new file mode 100644
index 0000000..4f83203
--- /dev/null
+++ b/arch/mips/kernel/vdso32/sigtramp.S
@@ -0,0 +1,13 @@
+#include <asm/asm.h>
+#include <asm/regdef.h>
+#include <asm/unistd.h>
+
+LEAF(__kernel_sigtramp_32)
+	li	v0, __NR_sigreturn
+	syscall
+END(__kernel_sigtramp_32)
+
+LEAF(__kernel_sigtramp_rt32)
+	li	v0, __NR_rt_sigreturn
+	syscall
+END(__kernel_sigtramp_rt32)
diff --git a/arch/mips/kernel/vdso32/vdso32.S b/arch/mips/kernel/vdso32/vdso32.S
new file mode 100644
index 0000000..9548930
--- /dev/null
+++ b/arch/mips/kernel/vdso32/vdso32.S
@@ -0,0 +1,11 @@
+#include <linux/init.h>
+
+__INITDATA
+
+	.globl vdso32_start
+	.globl vdso32_end
+vdso32_start:
+	.incbin "arch/mips/kernel/vdso32/vdso32.so"
+vdso32_end:
+
+__FINIT
diff --git a/arch/mips/kernel/vdso32/vdso32.lds.S b/arch/mips/kernel/vdso32/vdso32.lds.S
new file mode 100644
index 0000000..250a03d
--- /dev/null
+++ b/arch/mips/kernel/vdso32/vdso32.lds.S
@@ -0,0 +1,73 @@
+/*
+ * Linker script for vsyscall DSO.  The vsyscall page is an ELF shared
+ * object prelinked to its virtual address, and with only one read-only
+ * segment (that fits in one page).  This script controls its layout.
+ */
+#include <asm/asm-offsets.h>
+
+/* Default link addresses for the vDSOs */
+#define VDSO32_LBASE	0x100000
+#define VDSO64_LBASE	0x100000
+
+/* Default map addresses */
+#define VDSO32_MBASE	VDSO32_LBASE
+#define VDSO64_MBASE	VDSO64_LBASE
+
+OUTPUT_ARCH(mips)
+ENTRY(__kernel_sigtramp_32);
+
+SECTIONS
+{
+  . = VDSO32_LBASE + SIZEOF_HEADERS;
+
+  .hash           : { *(.hash) }		:text
+  .gnu.hash       : { *(.gnu.hash) }
+  .dynsym         : { *(.dynsym) }
+  .dynstr         : { *(.dynstr) }
+  .gnu.version    : { *(.gnu.version) }
+  .gnu.version_d  : { *(.gnu.version_d) }
+  .gnu.version_r  : { *(.gnu.version_r) }
+
+  . = ALIGN(16);
+
+  .text           : { *(.text) }		:text
+  .note           : { *(.note.*) }		:text :note
+  .eh_frame_hdr   : { *(.eh_frame_hdr) }	:text :eh_frame_hdr
+  .eh_frame       : { KEEP (*(.eh_frame)) }	:text
+
+  .dynamic        : { *(.dynamic) }		:text :dynamic
+  .got            : { *(.got) }
+  .plt            : { *(.plt) }
+
+  /DISCARD/       : {
+	*(.got.plt) *(.got)
+	*(.data .data.* .gnu.linkonce.d.*)
+	*(.dynbss)
+	*(.bss .bss.* .gnu.linkonce.b.*)
+  }						:text
+}
+
+/*
+ * We must supply the ELF program headers explicitly to get just one
+ * PT_LOAD segment, and set the flags explicitly to make segments read-only.
+ */
+PHDRS
+{
+  text PT_LOAD FILEHDR PHDRS FLAGS(5);	/* PF_R|PF_X */
+  dynamic PT_DYNAMIC FLAGS(4);		/* PF_R */
+  note PT_NOTE FLAGS(4);		/* PF_R */
+  eh_frame_hdr 0x6474e550; /* PT_GNU_EH_FRAME, but ld doesn't match the name */
+}
+
+/*
+ * This controls what symbols we export from the DSO.
+ */
+VERSION
+{
+  LINUX_2.6.24 {
+    global:
+	__kernel_sigtramp_32;
+	__kernel_sigtramp_rt32;
+    local: *;
+  };
+}

Re: Cannot unwind through MIPS signal frames with ICACHE_REFILLS_WORKAROUND_WAR

[Andrew Pinski]

On 11/13/07, Ralf Baechle <ralf@linux-mips.org> wrote:
> Old versions of glibc were probable the most notorious users of trampolines.
> Objective C also generates them.  Since a cacheflush that is a syscall is
> required performance is less than great.

No Objective-C does not generate them.  Objective-C returns the exact
function pointer back.  Now libffi generates trampolines.

-- Pinski

Re: Cannot unwind through MIPS signal frames withICACHE_REFILLS_WORKAROUND_WAR

[Kevin D. Kissell]

> > True, though it should perhaps be noted that currently it's only on 4KSc/Sd
> > systems (which I know you work on) where it's even possible for the stack
> > *not* to have exec permissions, since the classical MIPS MMU gives
> > execute permission to any page that is readable.
> 
> Disabling PROT_EXEC on a mapping tells the kernel it doesn't need to take
> care of I-cache coherency.  So it's somewhat beneficial even in absence of
> a protection bit in the actual TLB hardware.

That depends on just what the consequences of I-cache incoherence might be.
Without help from the MMU, the kernel cannot *know* that a given location
isn't in the I-cache, because a program can always compute a pointer-to-function
to an arbitrary address and dereference it successfully so long as the location
is readable.  If it's only the user who does this that will suffer as a result of
I-cache incoherence, one can argue that it serves him right.  But if it can screw
up the execution of the kernel, or other user processes, I think we have to 
assume the worst.

> Some of these performance optimizations are impossible because the kernel
> can't have definate knowledge that certain addresses have never entered the
> I-cache.

Sad but true.

            Regards,

            Kevin K.

Re: Cannot unwind through MIPS signal frames withICACHE_REFILLS_WORKAROUND_WAR

[Kevin D. Kissell]

> > True, though it should perhaps be noted that currently it's only on 4KSc/Sd
> > systems (which I know you work on) where it's even possible for the stack
> > *not* to have exec permissions, since the classical MIPS MMU gives
> > execute permission to any page that is readable.
> 
> Disabling PROT_EXEC on a mapping tells the kernel it doesn't need to take
> care of I-cache coherency.  So it's somewhat beneficial even in absence of
> a protection bit in the actual TLB hardware.

That depends on just what the consequences of I-cache incoherence might be.
Without help from the MMU, the kernel cannot *know* that a given location
isn't in the I-cache, because a program can always compute a pointer-to-function
to an arbitrary address and dereference it successfully so long as the location
is readable.  If it's only the user who does this that will suffer as a result of
I-cache incoherence, one can argue that it serves him right.  But if it can screw
up the execution of the kernel, or other user processes, I think we have to 
assume the worst.

> Some of these performance optimizations are impossible because the kernel
> can't have definate knowledge that certain addresses have never entered the
> I-cache.

Sad but true.

            Regards,

            Kevin K.

Re: Cannot unwind through MIPS signal frames with ICACHE_REFILLS_WORKAROUND_WAR

[Franck Bui-Huu]

Ralf Baechle wrote:
> Another reason is to get rid of the classic trampoline the kernel installs
> on the stack.  On some multiprocessor systems it requires a cacheflush

BTW, could we get rid of the trampoline so easily ? I mean won't we have
to keep it for backward compatibility reasons ?

		Franck

Re: Cannot unwind through MIPS signal frames with ICACHE_REFILLS_WORKAROUND_WAR

[Ralf Baechle]

On Thu, Nov 15, 2007 at 09:46:25AM +0100, Franck Bui-Huu wrote:

> Ralf Baechle wrote:
> > Another reason is to get rid of the classic trampoline the kernel installs
> > on the stack.  On some multiprocessor systems it requires a cacheflush
> 
> BTW, could we get rid of the trampoline so easily ? I mean won't we have
> to keep it for backward compatibility reasons ?

The trampolines are an implementation detail.  Little software needs to
know about it, so while I expect some slight colateral damage from getting
rid of trampolines it's not going to be painful.  GDB is the primary piece
of software that will need to change.

Some of the other architectures have an sa_restorer field in struct
sigaction but we don't have that on MIPS.

One way to deal with this would be to do a similar as IRIX where the
sigaction(2) takes a 4th argument which takes the role of sa_restorer.
For backward compatibility an SA_RESTORER field.  So if the SA_RESTORER
is clear we'd be using a classic trampoline, if it's set the value of
the 4th argument.

Or slightly crazier, put a kernel address into the $ra register of the
invoked signal handler.  So the signal handler will cause an address
error exception which then can be trapped.  Additional advantage - some of
the "Don't let your children do this ..." sections of code can go away ;-)

  Ralf

Re: Cannot unwind through MIPS signal frames with ICACHE_REFILLS_WORKAROUND_WAR

[David Daney]

Ralf Baechle wrote:
> On Thu, Nov 15, 2007 at 09:46:25AM +0100, Franck Bui-Huu wrote:
> 
>> Ralf Baechle wrote:
>>> Another reason is to get rid of the classic trampoline the kernel installs
>>> on the stack.  On some multiprocessor systems it requires a cacheflush
>> BTW, could we get rid of the trampoline so easily ? I mean won't we have
>> to keep it for backward compatibility reasons ?

There has to be some way for user code to unwind the call stack through 
signal handler frames.  For Linux/MIPS systems that use GCC as their 
system compiler, this is done by code in libgcc which is part of the GCC 
runtime.

Currently the unwinder examines the code at the signal handler return 
address and if it detects one of the several know trampoline code 
sequences, it makes assumptions about the way the kernel lays out the 
stack.  In particular, it assumes that there is a struct sigcontext at a 
  well known offset from the stack pointer at entry to the signal handler.

The code that handles this is in gcc/config/mips/linux-unwind.h

As of GCC-4.3 the location of the trampoline is unimportant, but the 
location of the sigcontext relative to the value in $sp is important and 
should not be changed unless there really is no other choice.


> 
> The trampolines are an implementation detail.  Little software needs to
> know about it, so while I expect some slight colateral damage from getting
> rid of trampolines it's not going to be painful.  GDB is the primary piece
> of software that will need to change.

And any user code that uses SIGSEGV to detect and handle dereferencing 
null pointers.

> 
> Some of the other architectures have an sa_restorer field in struct
> sigaction but we don't have that on MIPS.
> 
> One way to deal with this would be to do a similar as IRIX where the
> sigaction(2) takes a 4th argument which takes the role of sa_restorer.
> For backward compatibility an SA_RESTORER field.  So if the SA_RESTORER
> is clear we'd be using a classic trampoline, if it's set the value of
> the 4th argument.
> 
> Or slightly crazier, put a kernel address into the $ra register of the
> invoked signal handler.  So the signal handler will cause an address
> error exception which then can be trapped.  Additional advantage - some of
> the "Don't let your children do this ..." sections of code can go away ;-)

I am liking the idea of putting the trampoline code in the (as of yet 
non-existent vdso).  This eliminates the need to flush the icache, but 
maintains the ability of user code to unwind through signal frames.

Putting a .eh_frame section in the vdso would be even better as that 
would eliminate the need for libgcc to do code reading and let the 
kernel use any means desired to return from signal handlers.

David Daney

Re: Cannot unwind through MIPS signal frames with ICACHE_REFILLS_WORKAROUND_WAR

[Franck Bui-Huu]

David Daney wrote:
> I am liking the idea of putting the trampoline code in the (as of yet
> non-existent vdso).  This eliminates the need to flush the icache, but
> maintains the ability of user code to unwind through signal frames.
> 
> Putting a .eh_frame section in the vdso would be even better as that
> would eliminate the need for libgcc to do code reading and let the
> kernel use any means desired to return from signal handlers.
> 

You're very welcome to improve the ld script for VDSO generation, I
sent a couple days ago. I would be happy to integrate your
improvements in my patch for the future version.

thanks,
		Franck