vfork test case.

vfork test case.

[Carlos O'Donell]

[-- Attachment #1: Type: text/plain, Size: 4938 bytes --]

Kyle, James,

I have constructed a vfork test case which shows some of the problems
I have using vfork reliably. This fails every time on my PA8700 system
running 2.6.32-rc6. It appears as though r28 (ret0) in the parent is
being corrupted.

The intent of the testcase is to do the following:
(a) vfork
(b) Launch "ls -l" in the vfork'd child.
(c) Print some information in the parent.

~~~ vfork.c
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>

int main (void)
{
  pid_t child;
  char *cmd[] = { "ls", "-l", (char *)0 };
  char *env[] = { "HOME=/tmp", (char *)0 };

  child = vfork();

  if (child == 0)
    {
      execve("/bin/ls", cmd, env);
    }
  else
    {
      printf("child != 0\n");
    }

  printf("child is 0x%x\n", (unsigned int)child);

  return 0;
}
~~~
Compile this test case twice:
gcc -O1 -g -o vfork-O1 vfork.c
gcc -O0 -g -o vfork-O0 vfork.c

When run on x86, the following are the results:
~~~ vfork-O1
child != 0
child is 0x34e3
total 4824848
-rw-r--r--  1 carlos carlos      25135 Sep  2 10:41
"${BuildArtifactFileName}.map"
... [snip]
~~~
~~~ vfork-O0
child != 0
child is 0x3515
total 4824880
-rw-r--r--  1 carlos carlos      25135 Sep  2 10:41
"${BuildArtifactFileName}.map"
~~~

This x86 runs are correct.

However, on hppa compiling with gcc version 4.3.4 (Debian 4.3.4-6) I
get the following:
~~~ vfork-O1
./vfork-O1
child is 0x40552aac
carlos@firin:~/fsrc$ total 284
drwxr-xr-x  8 carlos carlos  4096 Jul 14  2005 binutils-old-work
... [snip]
~~~

The return from vfork is corrupted in the parent.

This gets worse at -O0.

~~~ vfork-O0
child is 0x405551a0
Illegal instruction
carlos@firin:~/fsrc$ total 284
drwxr-xr-x  8 carlos carlos  4096 Jul 14  2005 binutils-old-work
... [snip file list]
~~~
The kernel says:
vfork-O0 (pid 16313): Illegal instruction (code 8) at 000000004054ec77

     YZrvWESTHLNXBCVMcbcbcbcbOGFRQPDI
PSW: 00000000000001101111111100001111 Not tainted
r00-03  000000ff0006ff0f 00000000c06be968 000000004054ec70 00000000405551a4
r04-07  0000000040552aac 0000000000011b8a 0000000040553aac 00000000000f8c48
r08-11  00000000000f8a48 0000000000000000 00000000000c0000 00000000000f5808
r12-15  00000000000e8404 00000000000e8408 00000000ffffffff 0000000000000000
r16-19  0000000000000000 00000000000e6c40 00000000000e3fbc 00000000401f07c0
r20-23  0000000000000000 0000000040001900 00000000401e65f4 0000000000000000
r24-27  fffffffffffffff5 0000000000000000 00000000c06be42c 0000000000011b50
r28-31  0000000000000000 0000000000000000 0000000040552aac 000000004043d053
sr00-03  00000000053b5800 0000000000000000 0000000000000000 00000000053b5800
sr04-07  00000000053b5800 00000000053b5800 00000000053b5800 00000000053b5800

      VZOUICununcqcqcqcqcqcrmunTDVZOUI
FPSR: 00000000000000000000000000000000
FPER1: 00000000
fr00-03  0000000000000000 0000000000000000 0000000000000000 0000000000000000
fr04-07  0000000040197fac 00000000405628cc 000000004060b9a0 4040000000000000
fr08-11  00000000401fd7ac 0000000000000000 000000009f81ebb8 000000009f8403e0
fr12-15  0000000000000002 000000009f81ebb0 000000009f81ebc0 000000004060b9a0
fr16-19  0000000000000002 000000009f80eac0 0000000040552b48 00000000f000022c
fr20-23  0000000040657140 00000000404e32f8 0000000000000700 00001c8c00000000
fr24-27  0000000000000000 000000004011c8ac 0000000040652ca0 fffffffffffff000
fr28-31  0000000000000000 ffffffffffffff9c 00000000401d4a68 0000000000000000

IASQ: 00000000053b5800 00000000053b5800 IAOQ: 000000004054ec77 000000004054ec7b
 IIR: 0015e5b6    ISR: 0000000000000000  IOR: 0000000000000000
 CPU:        0   CR30: 00000000458a8000 CR31: ffffffffffffffff
 ORIG_R28: 0000000000000000
 IAOQ[0]: 000000004054ec77
 IAOQ[1]: 000000004054ec7b
 RP(r2): 000000004054ec70
~~~

However, when running the -O1 version under ptrace, it works:
~~~
strace -o strace.log -ff ./vfork-O1
child != 0
child is 0x40b7
total 340
drwxr-xr-x  8 carlos carlos  4096 Jul 14  2005 binutils-old-work
... [snip]
~~~

This indicates to me that the kernel is corrupting the parent process
after vfork, and the testcase shows it. I have reviewed the assembly
generated by the compiler and I can't show that anything is wrong.

To remove the C library from the loop I attach a complete vfork
implementation as used by glibc.

You can compile the test case using:
gcc -O1 -g -o vfork-O1 vfork.c pt-vfork.s

This will call the vfork in pt-vfork.s and allow you to adjust the
instructions stream up to and including removing the vfork system call
e.g. ble 0x100(%sr2,%r0).

For example I use "iitlbp %r0,(%sr0, %r0)" to force a fault in either
the parent or the child.

Please note that this pt-vfork.s isn't exactly as used in glibc, I
have expanded the return sequence for both child and parent so you can
cause a fault in one the other or both independently.

In summary:
* Test case works on x86.
* Test case fails on hppa.
* Test case works on hppa under strace.

What are we doing wrong and where is the bug?

Cheers,
Carlos.

[-- Attachment #2: pt-vfork.s --]
[-- Type: application/octet-stream, Size: 2669 bytes --]

# 1 "../ports/sysdeps/unix/sysv/linux/hppa/nptl/pt-vfork.S"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "./../include/libc-symbols.h" 1
# 54 "./../include/libc-symbols.h"
# 1 "/home/carlos/fsrc/glibc-work/builds/glibc/config.h" 1
# 55 "./../include/libc-symbols.h" 2
# 823 "./../include/libc-symbols.h"
# 1 "../sysdeps/wordsize-32/symbol-hacks.h" 1
# 824 "./../include/libc-symbols.h" 2
# 1 "<command-line>" 2
# 1 "../ports/sysdeps/unix/sysv/linux/hppa/nptl/pt-vfork.S"
# 19 "../ports/sysdeps/unix/sysv/linux/hppa/nptl/pt-vfork.S"
# 1 "../ports/sysdeps/unix/sysv/linux/hppa/sysdep.h" 1
# 23 "../ports/sysdeps/unix/sysv/linux/hppa/sysdep.h"
# 1 "/usr/include/asm/unistd.h" 1 3 4
# 24 "../ports/sysdeps/unix/sysv/linux/hppa/sysdep.h" 2
# 1 "../sysdeps/generic/sysdep.h" 1
# 25 "../ports/sysdeps/unix/sysv/linux/hppa/sysdep.h" 2
# 1 "../sysdeps/unix/sysv/linux/sys/syscall.h" 1
# 26 "../ports/sysdeps/unix/sysv/linux/hppa/sysdep.h" 2
# 20 "../ports/sysdeps/unix/sysv/linux/hppa/nptl/pt-vfork.S" 2

# 1 "../ports/sysdeps/unix/sysv/linux/hppa/bits/errno.h" 1
# 25 "../ports/sysdeps/unix/sysv/linux/hppa/bits/errno.h"
# 1 "/usr/include/linux/errno.h" 1 3 4



# 1 "/usr/include/asm/errno.h" 1 3 4



# 1 "/usr/include/asm-generic/errno-base.h" 1 3 4
# 5 "/usr/include/asm/errno.h" 2 3 4
# 5 "/usr/include/linux/errno.h" 2 3 4
# 26 "../ports/sysdeps/unix/sysv/linux/hppa/bits/errno.h" 2
# 22 "../ports/sysdeps/unix/sysv/linux/hppa/nptl/pt-vfork.S" 2
# 1 "/home/carlos/fsrc/glibc-work/builds/glibc/tcb-offsets.h" 1
# 23 "../ports/sysdeps/unix/sysv/linux/hppa/nptl/pt-vfork.S" 2
# 52 "../ports/sysdeps/unix/sysv/linux/hppa/nptl/pt-vfork.S"
.text ! 
.align 4 ! 
.export __vfork ! 
.type __vfork,@function ! 
__vfork: ! 
.PROC ! 
.CALLINFO FRAME=64,CALLS,SAVE_RP,ENTRY_GR=3 ! 
.ENTRY ! ! 
 stw %rp, -20(%sr0,%sp) !

 stwm %r3, 64(%sp)
 stw %sp, -4(%sp)
 stw %r19, -32(%sp)



 copy %r19, %r25



 mfctl %cr27, %r26 ! 
 ldw -1044(%r26),%r1 ! 
 sub %r0,%r1,%r1 ! 
 stw %r1,-1044(%r26) !


 ble 0x100(%sr2,%r0)
 ldi (0 + 113),%r20


 cmpb,=,n %r0,%ret0,.Lthread_start ! 
 mfctl %cr27, %r26 ! 
 ldw -1044(%r26),%r1 ! 
 sub %r0,%r1,%r1 ! 
 stw %r1,-1044(%r26) ! 

 ldi -4096,%r1
 comclr,>>= %r1,%ret0,%r0
 b,n .Lerror

 /* iitlbp %r0,(%sr0, %r0) */
 ldw -84(%sp), %rp
 bv %r0(%rp)
 ldwm -64(%sp), %r3

.Lthread_start: !

 ldi -4096,%r1
 comclr,>>= %r1,%ret0,%r0
 b,n .Lerror

 ldw -84(%sp), %rp
 bv %r0(%rp)
 ldwm -64(%sp), %r3

.Lerror:
 sub %r0,%ret0,%r3
 .import __errno_location,code ! ! 
 bl __errno_location,%rp !

 copy %r25, %r19

 stw %r3, 0(%ret0)
 ldw -84(%sp), %rp
 bv %r0(%rp)
 ldwm -64(%sp), %r3
.EXIT ! .PROCEND ! .size __vfork, .-__vfork !

.weak vfork ! vfork = __vfork

Re: vfork test case.

[John David Anglin]

> I have constructed a vfork test case which shows some of the problems
> I have using vfork reliably. This fails every time on my PA8700 system
> running 2.6.32-rc6. It appears as though r28 (ret0) in the parent is
> being corrupted.

The test doesn't fail on two of my builds:
Linux mx3210 2.6.31.7 #5 Wed Dec 9 22:49:53 EST 2009 parisc64
Linux hiauly6 2.6.31.7 #18 Wed Dec 9 21:34:36 EST 2009 parisc

Are you running a SMP config?

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: vfork test case.

[Carlos O'Donell]

On Tue, Dec 15, 2009 at 5:21 PM, John David Anglin
<dave@hiauly1.hia.nrc.ca> wrote:
>> I have constructed a vfork test case which shows some of the problems
>> I have using vfork reliably. This fails every time on my PA8700 system
>> running 2.6.32-rc6. It appears as though r28 (ret0) in the parent is
>> being corrupted.
>
> The test doesn't fail on two of my builds:
> Linux mx3210 2.6.31.7 #5 Wed Dec 9 22:49:53 EST 2009 parisc64
> Linux hiauly6 2.6.31.7 #18 Wed Dec 9 21:34:36 EST 2009 parisc

Interesting.

> Are you running a SMP config?

Yes, 64-bit 2.6.32-rc6.

I recently discovered that execve is actually *returning* from the
kernel, and this is a huge problem. I'm instrumenting the kernel to
determine why this is happening.

Cheers,
Carlos.

Re: vfork test case.

[Helge Deller]

On 12/15/2009 11:21 PM, John David Anglin wrote:
>> I have constructed a vfork test case which shows some of the problems
>> I have using vfork reliably. This fails every time on my PA8700 system
>> running 2.6.32-rc6. It appears as though r28 (ret0) in the parent is
>> being corrupted.
>
> The test doesn't fail on two of my builds:
> Linux mx3210 2.6.31.7 #5 Wed Dec 9 22:49:53 EST 2009 parisc64
> Linux hiauly6 2.6.31.7 #18 Wed Dec 9 21:34:36 EST 2009 parisc

I did tested your testcase on my c3000 too but it didn't failed for me either.
This was on some 2.6.32-rc candidate.

Helge

Re: vfork test case.

[John David Anglin]

> On 12/15/2009 11:21 PM, John David Anglin wrote:
> >> I have constructed a vfork test case which shows some of the problems
> >> I have using vfork reliably. This fails every time on my PA8700 system
> >> running 2.6.32-rc6. It appears as though r28 (ret0) in the parent is
> >> being corrupted.
> >
> > The test doesn't fail on two of my builds:
> > Linux mx3210 2.6.31.7 #5 Wed Dec 9 22:49:53 EST 2009 parisc64
> > Linux hiauly6 2.6.31.7 #18 Wed Dec 9 21:34:36 EST 2009 parisc
> 
> I did tested your testcase on my c3000 too but it didn't failed for me either.
> This was on some 2.6.32-rc candidate.

I think it must be because Carlos is building SMP kernels.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: vfork test case.

[Carlos O'Donell]

On Wed, Dec 16, 2009 at 4:35 PM, John David Anglin
<dave@hiauly1.hia.nrc.ca> wrote:
>> On 12/15/2009 11:21 PM, John David Anglin wrote:
>> >> I have constructed a vfork test case which shows some of the problems
>> >> I have using vfork reliably. This fails every time on my PA8700 system
>> >> running 2.6.32-rc6. It appears as though r28 (ret0) in the parent is
>> >> being corrupted.
>> >
>> > The test doesn't fail on two of my builds:
>> > Linux mx3210 2.6.31.7 #5 Wed Dec 9 22:49:53 EST 2009 parisc64
>> > Linux hiauly6 2.6.31.7 #18 Wed Dec 9 21:34:36 EST 2009 parisc
>>
>> I did tested your testcase on my c3000 too but it didn't failed for me either.
>> This was on some 2.6.32-rc candidate.
>
> I think it must be because Carlos is building SMP kernels.

Yes, I'm trying to shake out some SMP bugs.

Dan, you said that the emacs32 failure went away when you upgraded to
2.6.32 stable on the buildds. Are they running SMP kernels? Could you
try the vfork test case there?

Test case is here:
http://article.gmane.org/gmane.linux.ports.parisc/2403

Cheers,
Carlos.

Re: vfork test case.

[dann frazier]

On Thu, Dec 17, 2009 at 09:24:05AM -0500, Carlos O'Donell wrote:
> On Wed, Dec 16, 2009 at 4:35 PM, John David Anglin
> <dave@hiauly1.hia.nrc.ca> wrote:
> >> On 12/15/2009 11:21 PM, John David Anglin wrote:
> >> >> I have constructed a vfork test case which shows some of the problems
> >> >> I have using vfork reliably. This fails every time on my PA8700 system
> >> >> running 2.6.32-rc6. It appears as though r28 (ret0) in the parent is
> >> >> being corrupted.
> >> >
> >> > The test doesn't fail on two of my builds:
> >> > Linux mx3210 2.6.31.7 #5 Wed Dec 9 22:49:53 EST 2009 parisc64
> >> > Linux hiauly6 2.6.31.7 #18 Wed Dec 9 21:34:36 EST 2009 parisc
> >>
> >> I did tested your testcase on my c3000 too but it didn't failed for me either.
> >> This was on some 2.6.32-rc candidate.
> >
> > I think it must be because Carlos is building SMP kernels.
> 
> Yes, I'm trying to shake out some SMP bugs.
> 
> Dan, you said that the emacs32 failure went away when you upgraded to
> 2.6.32 stable on the buildds. Are they running SMP kernels?

Nope. When I initially took over these buildds I tried SMP kernels and
found them to be far less stable, so we're sticking w/ UP.

That means that some emacs23 build attempts may have been on SMP, but
most were not. Kernel versions should be available in the build logs.
smp kernels should all have 'smp' in their uname output.

> Could you try the vfork test case there?

dannf@penalosa:/tmp$ cat /proc/version
Linux version 2.6.32-trunk-parisc64 (Debian 2.6.32-1) (waldi@debian.org) (gcc version 4.3.4 (GCC) ) #1 Tue Dec 8 04:07:32 UTC 2009
dannf@penalosa:/tmp$ gcc -O1 -g -o vfork-01 vfork.c
dannf@penalosa:/tmp$ gcc -O0 -g -o vfork-00 vfork.c
dannf@penalosa:/tmp$ ./vfork-01
child != 0
child is 0x10c8
dannf@penalosa:/tmp$ total 28
-rwxr-xr-x 1 dannf Debian  9333 Dec 17 16:41 vfork-00
-rwxr-xr-x 1 dannf Debian 10383 Dec 17 16:40 vfork-01
-rw-r--r-- 1 dannf Debian   380 Dec 17 16:40 vfork.c

dannf@penalosa:/tmp$ ./vfork-00 
child != 0
child is 0x1101
dannf@penalosa:/tmp$ total 28
-rwxr-xr-x 1 dannf Debian  9333 Dec 17 16:41 vfork-00
-rwxr-xr-x 1 dannf Debian 10383 Dec 17 16:40 vfork-01
-rw-r--r-- 1 dannf Debian   380 Dec 17 16:40 vfork.c

> Test case is here:
> http://article.gmane.org/gmane.linux.ports.parisc/2403
> 
> Cheers,
> Carlos.
> 

-- 
dann frazier

Re: vfork test case.

[Helge Deller]

On 12/16/2009 10:35 PM, John David Anglin wrote:
>> On 12/15/2009 11:21 PM, John David Anglin wrote:
>>>> I have constructed a vfork test case which shows some of the problems
>>>> I have using vfork reliably. This fails every time on my PA8700 system
>>>> running 2.6.32-rc6. It appears as though r28 (ret0) in the parent is
>>>> being corrupted.
>>>
>>> The test doesn't fail on two of my builds:
>>> Linux mx3210 2.6.31.7 #5 Wed Dec 9 22:49:53 EST 2009 parisc64
>>> Linux hiauly6 2.6.31.7 #18 Wed Dec 9 21:34:36 EST 2009 parisc
>>
>> I did tested your testcase on my c3000 too but it didn't failed for me either.
>> This was on some 2.6.32-rc candidate.
>
> I think it must be because Carlos is building SMP kernels.

I couldn't reproduce on my SMP box either (but it's PA8500 instead of PA8700):
cpu             : PA8500 (PCX-W)
model           : 9000/785/J5000
Linux ls3017 2.6.32-32bit #79 SMP Thu Dec 3 14:04:53 CET 2009 parisc GNU/Linux

Helge

Re: vfork test case.

[Carlos O'Donell]

On Thu, Dec 17, 2009 at 2:12 PM, Helge Deller <deller@gmx.de> wrote:
> I couldn't reproduce on my SMP box either (but it's PA8500 instead of
> PA8700):
> cpu =A0 =A0 =A0 =A0 =A0 =A0 : PA8500 (PCX-W)
> model =A0 =A0 =A0 =A0 =A0 : 9000/785/J5000
> Linux ls3017 2.6.32-32bit #79 SMP Thu Dec 3 14:04:53 CET 2009 parisc
> GNU/Linux

Thanks Helge. This was a 64-bit box with SMP. It's possible part of
the 64-bit kernel is being incorrectly compiled.

I am not yet able to boot 2.6.32.

I'm getting problems in the initrd while waiting for /dev/sda1 to
appear, but it never does.

Once I resolve that I will see if the testcase works with 2.6.32.

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: vfork test case.

[Berthold Gunreben]

Hi Carlos,

On Sunday 06 December 2009 22:19:13 Carlos O'Donell wrote:
> Kyle, James,
> 
> I have constructed a vfork test case which shows some of the problems
> I have using vfork reliably. This fails every time on my PA8700 system
> running 2.6.32-rc6. It appears as though r28 (ret0) in the parent is
> being corrupted.

this reminds me of a problem with vfork that I encountered some years ago. The 
problem as described in 
http://sourceware.org/ml/libc-alpha/2003-02/msg00042.html
was fixed for many architectures, but not for hppa as far as I remember. See 
also 
http://sourceware.org/ml/libc-hacker/2003-02/msg00015.html

I have no idea if this is related, but thought it won't hurt to mention this.

Berthold

Re: vfork test case.

[Carlos O'Donell]

On Thu, Dec 17, 2009 at 4:40 PM, Berthold Gunreben <b.gunreben@web.de> wrote:
> Hi Carlos,
>
> On Sunday 06 December 2009 22:19:13 Carlos O'Donell wrote:
>> Kyle, James,
>>
>> I have constructed a vfork test case which shows some of the problems
>> I have using vfork reliably. This fails every time on my PA8700 system
>> running 2.6.32-rc6. It appears as though r28 (ret0) in the parent is
>> being corrupted.
>
> this reminds me of a problem with vfork that I encountered some years ago. The
> problem as described in
> http://sourceware.org/ml/libc-alpha/2003-02/msg00042.html
> was fixed for many architectures, but not for hppa as far as I remember. See
> also
> http://sourceware.org/ml/libc-hacker/2003-02/msg00015.html
>
> I have no idea if this is related, but thought it won't hurt to mention this.

That has to do with the assembly not checking the pthread function
lookup table to determine if it should call vfork or pthread's vfork.

In my test case there are no pthread functions calls.

Thanks for the pointer, the fix you point out is no longer needed in nptl.

Cheers,
Carlos.

Re: vfork test case.

[Grant Grundler]

On Thu, Dec 17, 2009 at 03:39:02PM -0500, Carlos O'Donell wrote:
> On Thu, Dec 17, 2009 at 2:12 PM, Helge Deller <deller@gmx.de> wrote:
> > I couldn't reproduce on my SMP box either (but it's PA8500 instead =
of
> > PA8700):
> > cpu =A0 =A0 =A0 =A0 =A0 =A0 : PA8500 (PCX-W)
> > model =A0 =A0 =A0 =A0 =A0 : 9000/785/J5000
> > Linux ls3017 2.6.32-32bit #79 SMP Thu Dec 3 14:04:53 CET 2009 paris=
c
> > GNU/Linux
>=20
> Thanks Helge. This was a 64-bit box with SMP. It's possible part of
> the 64-bit kernel is being incorrectly compiled.
>=20
> I am not yet able to boot 2.6.32.
>=20
> I'm getting problems in the initrd while waiting for /dev/sda1 to
> appear, but it never does.

hrm..there is a module option to force SCSI Async scan to complete
before calling into user space. Perhaps that is the problem?

=46WIW, I've been moving all machine to "mount by Label" or UUID.

cheers,
grant


>=20
> Once I resolve that I will see if the testcase works with 2.6.32.
>=20
> Cheers,
> Carlos.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-paris=
c" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: vfork test case.

[John David Anglin]

Carlos,

> I have constructed a vfork test case which shows some of the problems
> I have using vfork reliably. This fails every time on my PA8700 system
> running 2.6.32-rc6. It appears as though r28 (ret0) in the parent is
> being corrupted.

It seems the vfork implementation has changed in 2.11.1 from
using clone to the vfork syscall.  I tried to build gdb head
today, but it dies at startup due to a problem with vfork.  This
is potentially also an issue with gcc as it uses the same
libiberty code.

I filed a sourceware bug, but closed it when I realized that
this is a glibc/kernel bug:
http://sourceware.org/bugzilla/show_bug.cgi?id=11730

After spending some time looking at this, I realized that __vfork
is broken.  It should not allocate a frame as the child may clobber
the parent's frame when it does a call, etc.  A frame should only
be allocated and popped around the call to the syscall error handler.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: vfork test case.

[Carlos O'Donell]

On Sun, Jun 20, 2010 at 12:28 AM, John David Anglin
<dave@hiauly1.hia.nrc.ca> wrote:
> Carlos,
>
>> I have constructed a vfork test case which shows some of the problem=
s
>> I have using vfork reliably. This fails every time on my PA8700 syst=
em
>> running 2.6.32-rc6. It appears as though r28 (ret0) in the parent is
>> being corrupted.
>
> It seems the vfork implementation has changed in 2.11.1 from
> using clone to the vfork syscall. =A0I tried to build gdb head
> today, but it dies at startup due to a problem with vfork. =A0This
> is potentially also an issue with gcc as it uses the same
> libiberty code.

On hppa calling vfork in a non-multithreaded environment does the follo=
wing:

vfork [vfork.c] ->
  __fork (__libc_fork) [nptl/fork.c, linux/fork.c] ->
    ARCH_FORK (clone syscall) []

On hppa in a multithreaded environment (linked against libpthread.so)
vfork does the following:

vfork ->
  vfork syscall (pt-vfork.S)

This has not changed in 2.11.1.

Could you please elaborate on the change that you believe occurred in 2=
=2E11.1?

> I filed a sourceware bug, but closed it when I realized that
> this is a glibc/kernel bug:
> http://sourceware.org/bugzilla/show_bug.cgi?id=3D11730
>
> After spending some time looking at this, I realized that __vfork
> is broken. =A0It should not allocate a frame as the child may clobber
> the parent's frame when it does a call, etc. =A0A frame should only
> be allocated and popped around the call to the syscall error handler.

Oh dear, yes I see. After the child returns from vfork, it calls exec,
clobbering the vfork frame that the parent must still unwind from. Is
that what you mean?

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: vfork test case.

[Carlos O'Donell]

On Tue, Jun 22, 2010 at 11:17 AM, Carlos O'Donell
<carlos@systemhalted.org> wrote:
> Oh dear, yes I see. After the child returns from vfork, it calls exec,
> clobbering the vfork frame that the parent must still unwind from. Is
> that what you mean?

Is this a complete fix?

diff --git a/sysdeps/unix/sysv/linux/hppa/nptl/pt-vfork.S
b/sysdeps/unix/sysv/linux/hppa/nptl/pt-vfork.S
index 83a70b7..3c685cb 100644
--- a/sysdeps/unix/sysv/linux/hppa/nptl/pt-vfork.S
+++ b/sysdeps/unix/sysv/linux/hppa/nptl/pt-vfork.S
@@ -50,10 +50,9 @@

        /* r26, r25, r24, r23 are free since vfork has no arguments */
 ENTRY(__vfork)
-       /* Prologue */
-       stwm    %r3, 64(%sp)
-       stw     %sp, -4(%sp)
-       stw     %r19, -32(%sp)
+       /* We must not create a frame, otherwise when the child unwinds
+          to call exec it will clobber the same frame that the parent
+          needs to unwind.  */

        /* Save the PIC register. */
 #ifdef PIC
@@ -76,11 +75,16 @@ ENTRY(__vfork)
        b,n     .Lerror

        /* Return, no need to restore the PIC register. */
-       ldw     -84(%sp), %rp
-       bv      %r0(%rp)
-       ldwm    -64(%sp), %r3
+       ldw     -20(%sp), %rp
+       bv,n    %r0(%rp)

 .Lerror:
+       /* Now we need a stack to record the error. We are assured
+          that there is no child now, so it's safe to create
+          a frame.  */
+       stwm    %r3, 64(%sp)
+       stw     %sp, -4(%sp)
+
        sub     %r0,%ret0,%r3
        SYSCALL_ERROR_HANDLER
        /* Restore the PIC register (in delay slot) on error */

I'm building/testing this and I will tell you how it goes.

Cheers,
Carlos.

Re: vfork test case.

[John David Anglin]

> vfork ->
>   vfork syscall (pt-vfork.S)
> 
> This has not changed in 2.11.1.

I just assumed that a change must have occurred because of the behavior
of pex_unix_exec_child in libiberty/pex-unix.c.  I didn't investigate
history.

> Could you please elaborate on the change that you believe occurred in 2.11.=
> 1?
> 
> > I filed a sourceware bug, but closed it when I realized that
> > this is a glibc/kernel bug:
> > http://sourceware.org/bugzilla/show_bug.cgi?id=3D11730
> >
> > After spending some time looking at this, I realized that __vfork
> > is broken. =A0It should not allocate a frame as the child may clobber
> > the parent's frame when it does a call, etc. =A0A frame should only
> > be allocated and popped around the call to the syscall error handler.
> 
> Oh dear, yes I see. After the child returns from vfork, it calls exec,
> clobbering the vfork frame that the parent must still unwind from. Is
> that what you mean?

Yes.  Actually, pex_unix_exec_child may call dup2 and close as well.
In my gdb build, the parent's return address in the frame is clobbered
resulting in pex_child_error being called by the parent.

I looked at the hpux implementation and it doesn't create a frame.
Of course, the vfork syscall then must not clobber the return address,
etc.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: vfork test case.

[Carlos O'Donell]

On Tue, Jun 22, 2010 at 12:26 PM, John David Anglin
<dave@hiauly1.hia.nrc.ca> wrote:
>> vfork ->
>> =A0 vfork syscall (pt-vfork.S)
>>
>> This has not changed in 2.11.1.
>
> I just assumed that a change must have occurred because of the behavi=
or
> of pex_unix_exec_child in libiberty/pex-unix.c. =A0I didn't investiga=
te
> history.

Thank you, I don't expect anything has changed, but even if it hasn't
there is still a bug in glibc.

> Yes. =A0Actually, pex_unix_exec_child may call dup2 and close as well=
=2E
> In my gdb build, the parent's return address in the frame is clobbere=
d
> resulting in pex_child_error being called by the parent.

Technically POSIX says it can't call dup2/close, but under Linux the
vfork implementation duplicates file descriptors and therefore
dup2/close are OK to use before execve.

The important part is that vfork *must not* create a frame in the
parent that the child unwinds.

> I looked at the hpux implementation and it doesn't create a frame.
> Of course, the vfork syscall then must not clobber the return address=
,
> etc.

Yes, the vfork syscall *must not* clobber r2, without a stack we have
nowhere to save it. I'm pretty sure the kernel preserves r2 across a
vfork. The next step is to verify that :-)

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: vfork test case.

[John David Anglin]

> > I looked at the hpux implementation and it doesn't create a frame.
> > Of course, the vfork syscall then must not clobber the return address,
> > etc.
> 
> Yes, the vfork syscall *must not* clobber r2, without a stack we have
> nowhere to save it. I'm pretty sure the kernel preserves r2 across a
> vfork. The next step is to verify that :-)

Actually, it's ok to save it in the __fork caller's frame at the standard
location.  r2 is always saved there by the callee.  The argument slots for
r26 thru r23 and the PIC register slot also can be used.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: vfork test case.

[Carlos O'Donell]

On Tue, Jun 22, 2010 at 2:44 PM, John David Anglin
<dave@hiauly1.hia.nrc.ca> wrote:
>> > I looked at the hpux implementation and it doesn't create a frame.
>> > Of course, the vfork syscall then must not clobber the return addr=
ess,
>> > etc.
>>
>> Yes, the vfork syscall *must not* clobber r2, without a stack we hav=
e
>> nowhere to save it. I'm pretty sure the kernel preserves r2 across a
>> vfork. The next step is to verify that :-)
>
> Actually, it's ok to save it in the __fork caller's frame at the stan=
dard
> location. =A0r2 is always saved there by the callee. =A0The argument =
slots for
> r26 thru r23 and the PIC register slot also can be used.

I disagree with you.

What about this scenario:
* Parent call vfork, saves rp in -20(%sp) (parents frame)
* Parent suspends upon entry to vfork syscall
* Child returns from vfork syscall
* Child restores rp from -20(%sp) (parents frame)
* Child calls close which saves rp in -20(%sp) (parents frame)
* Child calls execve
* Parent resumes, restores rp from -20(%sp) and incorrectly returns to
just after the call to close.

At this point if the parent restores rp from the parents frame it will
get the rp the child's call to close saved.

The only way to get around this is to *ensure* the kernel
saves/restores r2 across the syscall.

I see no other way around it.

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: vfork test case.

[John David Anglin]

> I disagree with you.
> 
> What about this scenario:
> * Parent call vfork, saves rp in -20(%sp) (parents frame)
> * Parent suspends upon entry to vfork syscall
> * Child returns from vfork syscall
> * Child restores rp from -20(%sp) (parents frame)
> * Child calls close which saves rp in -20(%sp) (parents frame)
> * Child calls execve
> * Parent resumes, restores rp from -20(%sp) and incorrectly returns to
> just after the call to close.
> 
> At this point if the parent restores rp from the parents frame it will
> get the rp the child's call to close saved.

You are right.

> 
> The only way to get around this is to *ensure* the kernel
> saves/restores r2 across the syscall.
> 
> I see no other way around it.

It looks as if r2 is saved for both parent and child.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: vfork test case.

[Carlos O'Donell]

On Tue, Jun 22, 2010 at 4:20 PM, John David Anglin
<dave@hiauly1.hia.nrc.ca> wrote:
>> At this point if the parent restores rp from the parents frame it will
>> get the rp the child's call to close saved.
>
> You are right.
>
>>
>> The only way to get around this is to *ensure* the kernel
>> saves/restores r2 across the syscall.
>>
>> I see no other way around it.
>
> It looks as if r2 is saved for both parent and child.

Perfect, in that case we have all the bases covered.

I have seen no regressions in my testing, so I will push this patch
into glibc-ports, and notify debian.

I still saw a hang in tst-vfork2, but I haven't investigated this yet.

Cheers,
Carlos.

Re: vfork test case.

[John David Anglin]

On Tue, 22 Jun 2010, Carlos O'Donell wrote:

>         /* Return, no need to restore the PIC register. */
> -       ldw     -84(%sp), %rp
> -       bv      %r0(%rp)
> -       ldwm    -64(%sp), %r3
> +       ldw     -20(%sp), %rp
> +       bv,n    %r0(%rp)

For the reasons discussed, I don't believe that %rp should be restored
from the stack.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: vfork test case.

[Carlos O'Donell]

On Tue, Jun 22, 2010 at 6:50 PM, John David Anglin
<dave@hiauly1.hia.nrc.ca> wrote:
> On Tue, 22 Jun 2010, Carlos O'Donell wrote:
>
>> =A0 =A0 =A0 =A0 /* Return, no need to restore the PIC register. */
>> - =A0 =A0 =A0 ldw =A0 =A0 -84(%sp), %rp
>> - =A0 =A0 =A0 bv =A0 =A0 =A0%r0(%rp)
>> - =A0 =A0 =A0 ldwm =A0 =A0-64(%sp), %r3
>> + =A0 =A0 =A0 ldw =A0 =A0 -20(%sp), %rp
>> + =A0 =A0 =A0 bv,n =A0 =A0%r0(%rp)
>
> For the reasons discussed, I don't believe that %rp should be restore=
d
> from the stack.

The final version of the patch does not restore %sp from the callers st=
ack.

This is now fixed upstream here:
http://sourceware.org/git/?p=3Dglibc-ports.git;a=3Dcommit;h=3D3680f14a7=
d12a9faa86e09aaea1b3aa20713355e

I have pushed this fix into debian-glibc, so it should be in the next
release of the library.

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html