[uml-devel] UML Kernel 2.6.7 Nothing happens

[uml-devel] UML Kernel 2.6.7 Nothing happens

[Roland Kaeser]

Hi all

I'm a newby to UML but an very oldy to linux. I want to try to setup up several uml
jailed services on one physical machine.

I'm using Kernel 2.6.7 with Host SKS patch and also Kernel 2.6.7 for the uml
instances. I'm calling the uml kernel but then nothing happens:

root@roland linux]# /install/uml/kernel/linux initrd=/install/uml/files/initrd.img 
lang= devfs=nomount ramdisk_size=10240 mem=128M eth0=tuntap,,,195.49.21.28
ubd0=/install/uml/instances/fc2install fakehd con=xterm ubd=0
Checking for the skas3 patch in the host...found
Checking for /proc/mm...found

[1]+  Stopped                 /install/uml/kernel/linux
initrd=/install/uml/files/initrd.img lang= devfs=nomount ramdisk_size=10240 mem=128M
eth0=tuntap,,,195.49.21.28 ubd0=/install/uml/instances/fc2install fakehd con=xterm
ubd=0

I've made the nodes in /dev/ compiled and installed the uml utilities. I really have
no idea whats wrong.
Please help

Roland Kaeser
Systems Administrator


	

	
		
___________________________________________________________
Gesendet von Yahoo! Mail - Jetzt mit 100MB Speicher kostenlos - Hier anmelden: http://mail.yahoo.de


-------------------------------------------------------
This Newsletter Sponsored by: Macrovision 
For reliable Linux application installations, use the industry's leading
setup authoring tool, InstallShield X. Learn more and evaluate 
today. http://clk.atdmt.com/MSI/go/ins0030000001msi/direct/01/
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Stop at startup on 2.6 NPTL hosts (was: Re: [uml-devel] UML Kernel 2.6.7 Nothing happens)

[Blaisorblade]

On Friday 29 October 2004 11:26, Roland Kaeser wrote:
> Hi all

> I'm a newby to UML but an very oldy to linux. I want to try to setup up
> several uml jailed services on one physical machine.

> I'm using Kernel 2.6.7 with Host SKS patch and also Kernel 2.6.7 for the
> uml instances. I'm calling the uml kernel but then nothing happens:

> root@roland linux]# /install/uml/kernel/linux
> initrd=/install/uml/files/initrd.img lang= devfs=nomount ramdisk_size=10240
> mem=128M eth0=tuntap,,,195.49.21.28 ubd0=/install/uml/instances/fc2install
> fakehd con=xterm ubd=0
> Checking for the skas3 patch in the host...found
> Checking for /proc/mm...found

> [1]+  Stopped                 /install/uml/kernel/linux
> initrd=/install/uml/files/initrd.img lang= devfs=nomount ramdisk_size=10240
> mem=128M eth0=tuntap,,,195.49.21.28 ubd0=/install/uml/instances/fc2install
> fakehd con=xterm ubd=0

> I've made the nodes in /dev/ compiled and installed the uml utilities. I
> really have no idea whats wrong.
> Please help

I'm CC:ing everybody who I found reported this problem. And with this problem, 
I mean this:

$ ./vmlinux ubd0=~/Uml/root_fs_toms1.7.205
Checking for the skas3 patch in the host...found
Checking for /proc/mm...found

[1]+  Stopped                 ./vmlinux ubd0=~/Uml/root_fs_toms1.7.205

I'm experiencing this problem, too, so I'm trying to fix it. I'm getting some 
success, but I'll have to fix a lot of places before UML start working again.

I have found out what happens. I simply ran:
gdb ./vmlinux
r
bt
and went watching the source in the backtrace point.

This happens if and only if UML is using NPTL thread libraries. This, in turn, 
happens in normal config with a NPTL-enabled glibc on the host, or in the 
special situation of CONFIG_MODE_TT and CONFIG_STATIC_LINK off. This is why 
it's not normally noticed and it's not yet debugged (it was reported the 1st 
time by Nuno Silva around 2.6.0).

Usually compiling with CONFIG_MODE_TT enabled appears to workaround the 
problem on most distros. The exception are Gentoo and recent LinuxFromScratch 
systems, where it would cause the compilation to fail.

It is one UML bug I've just identified and that I'm trying to fix. This is the 
faulty code, from arch/um/kernel/skas/process.c:

static int userspace_tramp(void *arg)
{
        init_new_thread_signals(0);
        enable_timer();
        ptrace(PTRACE_TRACEME, 0, 0, 0);
        tkill(gettid(), SIGSTOP);
        os_stop_process(os_getpid());
        return(0);
}

This is executed by a clone() child. What happens is that, with NPTL, both the 
father and the son have the same pid, so the SIGSTOP is routed to the wrong 
thread. However, this is not expected: having the same pid should be reserved 
to when clone is called with CLONE_THREAD in the flags. I've verified that 
this is not happening in this case, even with strace (to make sure glibc is 
not playing any dirty tricks). But for some reasons, the kernel is behaving 
as if this happened.

What makes me suspicious is that the NPTL library calls set_thread_area(). 
Even by checking the source, it should not interfere. But something wrong may 
be happening.

Best regards.
-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729



-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

[uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Sven Köhler]

> It is one UML bug I've just identified and that I'm trying to fix. This is the 
> faulty code, from arch/um/kernel/skas/process.c:
> 
> static int userspace_tramp(void *arg)
> {
>         init_new_thread_signals(0);
>         enable_timer();
>         ptrace(PTRACE_TRACEME, 0, 0, 0);
>         tkill(gettid(), SIGSTOP);
>         os_stop_process(os_getpid());
>         return(0);
> }
> 
> This is executed by a clone() child. What happens is that, with NPTL, both the 
> father and the son have the same pid, so the SIGSTOP is routed to the wrong 
> thread. However, this is not expected: having the same pid should be reserved 
> to when clone is called with CLONE_THREAD in the flags. I've verified that 
> this is not happening in this case, even with strace (to make sure glibc is 
> not playing any dirty tricks). But for some reasons, the kernel is behaving 
> as if this happened.

This does remind me of something. I've already read the whole thing with 
CLONE_THREAD and clone() somewhere else, but i can't remember where it 
was. Was it on UML-MailingLists? Or was it even on the LKML? If my brain 
would just be a bit more reliable :-(


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

[uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Nuno Silva]

Blaisorblade wrote:

...

> This happens if and only if UML is using NPTL thread libraries. This, in turn, 
> happens in normal config with a NPTL-enabled glibc on the host, or in the 
> special situation of CONFIG_MODE_TT and CONFIG_STATIC_LINK off. This is why 
> it's not normally noticed and it's not yet debugged (it was reported the 1st 
> time by Nuno Silva around 2.6.0).

Yes, maybe it was me and, FWIW, I never found a solution to make a 
dynamic skas uml run under 2.6+libcNTPL.

Regards,
Nuno Silva


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Gerd Knorr]

Nuno Silva <nuno.silva@vgertech.com> writes:

> Yes, maybe it was me and, FWIW, I never found a solution to make a
> dynamic skas uml run under 2.6+libcNTPL.

The usual workaround for any nptl issues works here as well:
export LD_ASSUME_KERNEL=2.4.21

  Gerd

-- 
#define printk(args...) fprintf(stderr, ## args)


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Blaisorblade]

On Wednesday 03 November 2004 09:32, Gerd Knorr wrote:
> Nuno Silva <nuno.silva@vgertech.com> writes:
> > Yes, maybe it was me and, FWIW, I never found a solution to make a
> > dynamic skas uml run under 2.6+libcNTPL.
>
> The usual workaround for any nptl issues works here as well:
> export LD_ASSUME_KERNEL=2.4.21
You mean 2.4.1 IIRC, right? However, that does not work on LinuxFromScratch 
and Gentoo systems, mostly, which miss a non-NPTL glibc version.

-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Gerd Knorr]

On Wed, Nov 03, 2004 at 04:59:56PM +0100, Blaisorblade wrote:
> On Wednesday 03 November 2004 09:32, Gerd Knorr wrote:
> > Nuno Silva <nuno.silva@vgertech.com> writes:
> > > Yes, maybe it was me and, FWIW, I never found a solution to make a
> > > dynamic skas uml run under 2.6+libcNTPL.
> >
> > The usual workaround for any nptl issues works here as well:
> > export LD_ASSUME_KERNEL=2.4.21
> You mean 2.4.1 IIRC, right?

The exact version doesn't really matter I think.

> However, that does not work on LinuxFromScratch and Gentoo systems,
> mostly, which miss a non-NPTL glibc version.

Yes, you need a non-nptl glibc version for that, otherwise the dynamic
linker can't use it obviously ...

  Gerd

-- 
#define printk(args...) fprintf(stderr, ## args)


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Blaisorblade]

On Wednesday 03 November 2004 21:10, Gerd Knorr wrote:
> On Wed, Nov 03, 2004 at 04:59:56PM +0100, Blaisorblade wrote:
> > On Wednesday 03 November 2004 09:32, Gerd Knorr wrote:
> > > Nuno Silva <nuno.silva@vgertech.com> writes:
> > > > Yes, maybe it was me and, FWIW, I never found a solution to make a
> > > > dynamic skas uml run under 2.6+libcNTPL.
> > >
> > > The usual workaround for any nptl issues works here as well:
> > > export LD_ASSUME_KERNEL=2.4.21
> >
> > You mean 2.4.1 IIRC, right?
>
> The exact version doesn't really matter I think.
More or less, it's true. The value is checked against 2.2.5 for base , 2.4.1 
and 2.6.0 for using NPTL.

On a RH 9, instead, IIRC 2.4.20 is the minimum for using NPTL. More details 
can be found at http://people.redhat.com/drepper/  (there is a 
LD_ASSUME_KERNEL link).

> > However, that does not work on LinuxFromScratch and Gentoo systems,
> > mostly, which miss a non-NPTL glibc version.

> Yes, you need a non-nptl glibc version for that, otherwise the dynamic
> linker can't use it obviously ...

And this means that we must fix UML for NPTL. Also, the problem is strange: 
the child and the father, when created linking with NPTL, get the same pid, 
so os_stop_process (i.e. kill(SIGSTOP, child_pid)) kills the whole process. 
In fact, replacing those with gettid() and tkill() appears to fix the 
problem.

But:

1) This should not happen, since clone() is not called with CLONE_THREAD (I 
verified that CLONE_THREAD is not used even with NPTL, so glibc is not doing 
anything strange here).

2) So, we get a host bug.

3) In fact, on my current Gentoo, Mplayer appears to experience a similar 
behaviour (it stops itself on entry and exit). It could be something 
unrelated, but it seems to me a host bug.

Bye
-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729

Re: [uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Blaisorblade]

On Wednesday 03 November 2004 21:10, Gerd Knorr wrote:
> On Wed, Nov 03, 2004 at 04:59:56PM +0100, Blaisorblade wrote:
> > On Wednesday 03 November 2004 09:32, Gerd Knorr wrote:
> > > Nuno Silva <nuno.silva@vgertech.com> writes:
> > > > Yes, maybe it was me and, FWIW, I never found a solution to make a
> > > > dynamic skas uml run under 2.6+libcNTPL.
> > >
> > > The usual workaround for any nptl issues works here as well:
> > > export LD_ASSUME_KERNEL=2.4.21
> >
> > You mean 2.4.1 IIRC, right?
>
> The exact version doesn't really matter I think.
More or less, it's true. The value is checked against 2.2.5 for base , 2.4.1 
and 2.6.0 for using NPTL.

On a RH 9, instead, IIRC 2.4.20 is the minimum for using NPTL. More details 
can be found at http://people.redhat.com/drepper/  (there is a 
LD_ASSUME_KERNEL link).

> > However, that does not work on LinuxFromScratch and Gentoo systems,
> > mostly, which miss a non-NPTL glibc version.

> Yes, you need a non-nptl glibc version for that, otherwise the dynamic
> linker can't use it obviously ...

And this means that we must fix UML for NPTL. Also, the problem is strange: 
the child and the father, when created linking with NPTL, get the same pid, 
so os_stop_process (i.e. kill(SIGSTOP, child_pid)) kills the whole process. 
In fact, replacing those with gettid() and tkill() appears to fix the 
problem.

But:

1) This should not happen, since clone() is not called with CLONE_THREAD (I 
verified that CLONE_THREAD is not used even with NPTL, so glibc is not doing 
anything strange here).

2) So, we get a host bug.

3) In fact, on my current Gentoo, Mplayer appears to experience a similar 
behaviour (it stops itself on entry and exit). It could be something 
unrelated, but it seems to me a host bug.

Bye
-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Nuno Silva]

Gerd Knorr wrote:
> Nuno Silva <nuno.silva@vgertech.com> writes:
> 
> 
>>Yes, maybe it was me and, FWIW, I never found a solution to make a
>>dynamic skas uml run under 2.6+libcNTPL.
> 
> 
> The usual workaround for any nptl issues works here as well:
> export LD_ASSUME_KERNEL=2.4.21

I know that, but thanks for the info anyway. The reason to try UML with 
newer glibcs was to stop using int80 and use the vsyscall thing. Maybe I 
could save a few cycles! :-)

So using another glibc (by setting LD_ASSUME_KERNEL) is not what I want ;)

Regards,
Nuno Silva


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

[uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Bodo Stroesser]

[-- Attachment #1: Type: text/plain, Size: 2318 bytes --]

Blaisorblade wrote:
> On Friday 29 October 2004 11:26, Roland Kaeser wrote:
> This is executed by a clone() child. What happens is that, with NPTL, both the 
> father and the son have the same pid, so the SIGSTOP is routed to the wrong 
> thread. However, this is not expected: having the same pid should be reserved 
> to when clone is called with CLONE_THREAD in the flags. I've verified that 
> this is not happening in this case, even with strace (to make sure glibc is 
> not playing any dirty tricks). But for some reasons, the kernel is behaving 
> as if this happened.
Sorry, I have to oppose. The threads don't have the same pid! Only the getpid()-
call to the lib returns the pid of the father.

I wrote a small test program (attached). Please compile it with:

    gcc -static -o test_getpid_static test_getpid.c        and
    gcc -o test_getpid_nptl test_getpid.c

Using the two programs, you can see the following:
1) having linked my test with -static, each "getpid()" in the test results in a
    syscall (try "strace test_getpid_static")
2) linking without -static (I assume, this means using NPTL), only the first
    getpid() does a syscall, I guess, the further calls deliver a pid-value
    buffered in the lib! (try "strace test_getpid_static")
The test here requests and prints out its pid twice, then it exits. But strace will
show you two getpid()-calls only in case of the _static program.

3) the pid-history seen in 1 and 2 is used even for a child created with "clone()",
    regardless which clone-flags are used! Try "test_getpid_static clone" and
    "test_getpid_nptl clone" to see, what happens.
    After printing its pid twice, the program now creates a child via "clone()". The
    child requests its *real* pid via a "by-hand-syscall". Than it stops itself and
    is ptraced by the father, which prints out a message, if the child does a *real*
    getpid()-syscall.
    Note: If you remove the two getpid()-calls at the beginning of main(), the
    child will work correctly even with NPTL, since there isn't yet a buffered pid ...

Summary: I guess, the behavior of NPTL is a bug. Do you agree? To which list should
a bugreport be mailed?
To work around, we could use the by-hand-syscall for os_getpid(). I didn't test it,
but I'm quite shure, that it fixes the problem.

Bodo

[-- Attachment #2: test_getpid.c --]
[-- Type: text/plain, Size: 2066 bytes --]

#include <stdio.h>
#include <signal.h>
#include <sched.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <sys/mman.h>
#include <sys/ptrace.h>
#include <asm/ptrace.h>
#include <unistd.h>
#include <asm/unistd.h>
#include <errno.h>

char childstack[ 8*1024];


int my_getpid()
{
	long res;
	__asm__ volatile ("	int $0x80\n\t"
				: "=a" (res)
				: "0" (__NR_getpid));
	return res;
}


int
child_fn( void * unused)
{
	pid_t me = my_getpid();

	printf("Child: my PID via 'int $0x80' is %d\n", me);

	ptrace(PTRACE_TRACEME, 0, 0, 0);
	kill( me, SIGSTOP);

	printf("Child: my PID via 'getpid()' is %d\n", getpid());

	return 0;
}


int
main( int argc, char ** argv)
{
	int ret, status, suppress=0;
	pid_t child;

	printf("Parent: my PID is %d\n", getpid());
	printf("Parent: my PID is %d\n", getpid());

	if ( argc < 2 || strcmp( argv[1], "clone") )
		return 0;

	child = clone( child_fn, childstack+8*1024-4, SIGCHLD, NULL);

	if ( child < 0 ) {
		perror("clone");
		exit(1);
	}
	printf("Parent: childs PID is %d\n", child);

	do {
		ret = waitpid(child, &status, WUNTRACED);
		if(ret < 0) {
			perror("waitpid()");
			exit(1);
		}
	} while(WIFSTOPPED(status) && (WSTOPSIG(status) == SIGVTALRM));

	if(!WIFSTOPPED(status) || (WSTOPSIG(status) != SIGSTOP)) {
		printf("Parent: waitpid(): expected SIGSTOP, got status = %d\n", status);
		return 1;
	}

	while (1) {
		if ( ptrace( PTRACE_SYSCALL, child, (void *)0, 0) < 0 ) {
			perror("ptrace( PTRACE_SYSCALL, child, 0, 0)");
			exit(1);
		}
		ret = waitpid( child, &status, 0);
		if ( ret != child ) {
			perror("waitpid");
			return 1;
		}
		if ( WIFSTOPPED(status) && WSTOPSIG(status) == SIGTRAP ) {
			errno = 0;
			ret = ptrace( PTRACE_PEEKUSER, child, (void *)(ORIG_EAX*4), NULL);
			if ( errno ) {
				perror("ptrace( PTRACE_PEEKUSER, child, ORIG_EAX, NULL)");
				return 1;
			}
			if ( ret == __NR_getpid ) {
				if ( (suppress ^= 1) )
					printf("Parent: Child does a getpid() syscall!\n");
			}
		}
		else {
			printf("Parent: Child's status is %x: exiting\n", status);
			return (status != 0);
		}
	}
}

Re: [uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Adam Heath]

On Tue, 9 Nov 2004, Bodo Stroesser wrote:

> To work around, we could use the by-hand-syscall for os_getpid(). I didn't test it,
> but I'm quite shure, that it fixes the problem.

That would mean each arch has separate asm blocks.  Why not just use the
_syscall macros?


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Bodo Stroesser]

Adam Heath wrote:
> On Tue, 9 Nov 2004, Bodo Stroesser wrote:
> 
> 
>>To work around, we could use the by-hand-syscall for os_getpid(). I didn't test it,
>>but I'm quite shure, that it fixes the problem.
> 
> 
> That would mean each arch has separate asm blocks.  Why not just use the
> _syscall macros?
Yes, I agree.
But before doing any workaround, we should try to get a fixed lib!


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Bodo Stroesser]

Bodo Stroesser wrote:
> Adam Heath wrote:
> 
>> On Tue, 9 Nov 2004, Bodo Stroesser wrote:
>>
>>
>>> To work around, we could use the by-hand-syscall for os_getpid(). I 
>>> didn't test it,
>>> but I'm quite shure, that it fixes the problem.
>>
>>
>>
>> That would mean each arch has separate asm blocks.  Why not just use the
>> _syscall macros?
> 
> Yes, I agree.
> But before doing any workaround, we should try to get a fixed lib!
> 
Meanwhile I did a test and created a patch for the problem. It's a
workaround only, since I think, the lib should be fixed. But it
works fine linked with NPTL and static, so it could stay with a fixed
lib anyway.

#####################

From: Bodo Stroesser <bstroesser@fujitsu-siemens.com>

Using NPTL, getpid() sometimes delivers the wrong pid, since it
uses a buffered one from previous calls. This buffered pid isn't
discarded, when a child is created by a clone().
So, as a workaround, UML should use a direct kernel call to bypass
the lib.

Signed-off-by: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
---

--- a/arch/um/os-Linux/process.c	2004-11-09 18:16:07.149159600 +0100
+++ b/arch/um/os-Linux/process.c	2004-11-09 18:40:43.723686152 +0100
@@ -107,6 +107,8 @@ void os_usr1_process(int pid)
  	kill(pid, SIGUSR1);
  }

+inline _syscall0( pid_t, getpid)
+
  int os_getpid(void)
  {
  	return(getpid());


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Blaisorblade]

[-- Attachment #1: Type: text/plain, Size: 3840 bytes --]

On Tuesday 09 November 2004 17:42, Bodo Stroesser wrote:
> Blaisorblade wrote:
> > On Friday 29 October 2004 11:26, Roland Kaeser wrote:
> > This is executed by a clone() child. What happens is that, with NPTL,
> > both the father and the son have the same pid, so the SIGSTOP is routed
> > to the wrong thread. However, this is not expected: having the same pid
> > should be reserved to when clone is called with CLONE_THREAD in the
> > flags. I've verified that this is not happening in this case, even with
> > strace (to make sure glibc is not playing any dirty tricks). But for some
> > reasons, the kernel is behaving as if this happened.

> Sorry, I have to oppose. The threads don't have the same pid! Only the
> getpid()- call to the lib returns the pid of the father.

This makes much more sense than my thoughts. I said I was going to investigate 
on that, but had no time yet.

This fact (glibc caches getpid() results) was even known because already 
posted, and I guessed that this could happen only with TLS support.

Also, not a lot of days ago, I was thinking to this issue for the UML startup 
tests (they rely on ptrace() catching the syscall done by the child and 
modifying it, so they would fail with the caching).

Still, I didn't realize this was the problem. Thanks for your help, as always.

> I wrote a small test program (attached). Please compile it with:
>
>     gcc -static -o test_getpid_static test_getpid.c        and
>     gcc -o test_getpid_nptl test_getpid.c

> Using the two programs, you can see the following:

> 1) having linked my test with -static, each "getpid()" in the test results
> in a syscall (try "strace test_getpid_static")
> 2) linking without -static (I assume, this means using NPTL), only the
> first getpid() does a syscall, I guess, the further calls deliver a
> pid-value buffered in the lib! (try "strace test_getpid_static")
> The test here requests and prints out its pid twice, then it exits. But
> strace will show you two getpid()-calls only in case of the _static
> program.

> 3) the pid-history seen in 1 and 2 is used even for a child created with
> "clone()", regardless which clone-flags are used! Try "test_getpid_static
> clone" and "test_getpid_nptl clone" to see, what happens.

>     After printing its pid twice, the program now creates a child via
> "clone()". The child requests its *real* pid via a "by-hand-syscall". Than
> it stops itself and is ptraced by the father, which prints out a message,
> if the child does a *real* getpid()-syscall.

>     Note: If you remove the two getpid()-calls at the beginning of main(),
> the child will work correctly even with NPTL, since there isn't yet a
> buffered pid ...

> Summary: I guess, the behavior of NPTL is a bug. Do you agree? To which
> list should a bugreport be mailed?

I don't think it's exactly a bug, but it could be more of a pitfall. Why? When 
using pthread_create, it will also setup a different per-thread TLS area.

Now, the question is: clone() is a syscall, so no such setup is done. Is it 
correct to wrap it to do this?

However, even fork() is a syscall, but I think that it does not have this 
problem. Would you mind testing if fork() correctly works?

I must also say, that I saw, in fact, a clone() wrapper in glibc; only I think 
it's not used when issuing clone() calls, which only do the syscall and a 
little setup to jump to the provided function.

Anyway, it must documented in getpid() and clone() man-pages, at least.

> To work around, we could use the by-hand-syscall for os_getpid(). I didn't
> test it, but I'm quite shure, that it fixes the problem.

I'm on the same position. I'm going to test that on my Gentoo host.
-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[stian]

> This fact (glibc caches getpid() results) was even known because already
> posted, and I guessed that this could happen only with TLS support.

That glibc caches getpid() has been a known issue before when people
wanted to benchmark uml syscall speeds inside vs outside of UML using a
simple libc-function. A clone() call can't easily invalidate caches around
in libs that are stored in statics. But it should be mentioned on
getpid()'s man/info page that the value might be cached.

Perhaps sende the glibc dudes an email with the problem, and either get
them to remove the cache or document it. Either way it will be a problem,
since people might use a version of glibc then that caches getpid.. only
work-around  is to use syscall() directly, or avoid using getpid() before
clone().


Stian


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Geert Uytterhoeven]

On Wed, 10 Nov 2004 stian@nixia.no wrote:
> > This fact (glibc caches getpid() results) was even known because already
> > posted, and I guessed that this could happen only with TLS support.
> 
> That glibc caches getpid() has been a known issue before when people
> wanted to benchmark uml syscall speeds inside vs outside of UML using a
> simple libc-function. A clone() call can't easily invalidate caches around
> in libs that are stored in statics. But it should be mentioned on
> getpid()'s man/info page that the value might be cached.

Why? getpid() is just an API call. No one guarantees it's actually a syscall.
And IMHO caching getpid() is allowed, since the returned value won't change
anyway.

If you want to be 100% sure you use a syscall, why not call the syscall
directly?

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Bodo Stroesser]

Geert Uytterhoeven wrote:
> On Wed, 10 Nov 2004 stian@nixia.no wrote:
> 
>>>This fact (glibc caches getpid() results) was even known because already
>>>posted, and I guessed that this could happen only with TLS support.
>>
>>That glibc caches getpid() has been a known issue before when people
>>wanted to benchmark uml syscall speeds inside vs outside of UML using a
>>simple libc-function. A clone() call can't easily invalidate caches around
>>in libs that are stored in statics. But it should be mentioned on
>>getpid()'s man/info page that the value might be cached.
> 
> 
> Why? getpid() is just an API call. No one guarantees it's actually a syscall.
> And IMHO caching getpid() is allowed, since the returned value won't change
> anyway.
> 
> If you want to be 100% sure you use a syscall, why not call the syscall
> directly?
OK. It's an API call. But what's this API call defined to return? Isn't it the
pid of the process calling? If it is, the behavior of the lib is a bug. If it
isn't, the descriptions are wrong. Do you agree?

Bodo


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Geert Uytterhoeven]

On Wed, 10 Nov 2004, Bodo Stroesser wrote:
> Geert Uytterhoeven wrote:
> > On Wed, 10 Nov 2004 stian@nixia.no wrote:
> > > > This fact (glibc caches getpid() results) was even known because already
> > > > posted, and I guessed that this could happen only with TLS support.
> > > 
> > > That glibc caches getpid() has been a known issue before when people
> > > wanted to benchmark uml syscall speeds inside vs outside of UML using a
> > > simple libc-function. A clone() call can't easily invalidate caches around
> > > in libs that are stored in statics. But it should be mentioned on
> > > getpid()'s man/info page that the value might be cached.
> > 
> > 
> > Why? getpid() is just an API call. No one guarantees it's actually a
> > syscall.
> > And IMHO caching getpid() is allowed, since the returned value won't change
> > anyway.
> > 
> > If you want to be 100% sure you use a syscall, why not call the syscall
> > directly?
> OK. It's an API call. But what's this API call defined to return? Isn't it the
> pid of the process calling? If it is, the behavior of the lib is a bug. If it

Yes, according to the man page getpid returns the process ID of the current
process. Doesn't it do that?

[ digging into mail archives ]

Sorry, I misunderstood. Yes, this must be a glibc bug.
Sorry for the confusion...

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Blaisorblade]

On Wednesday 10 November 2004 13:47, Geert Uytterhoeven wrote:
> On Wed, 10 Nov 2004, Bodo Stroesser wrote:
> > Geert Uytterhoeven wrote:
> > > On Wed, 10 Nov 2004 stian@nixia.no wrote:

> Yes, according to the man page getpid returns the process ID of the current
> process. Doesn't it do that?

> [ digging into mail archives ]

> Sorry, I misunderstood. Yes, this must be a glibc bug.
> Sorry for the confusion...

> Gr{oetje,eeting}s,

>       Geert
What's more, I think that fork() cannot be a plain syscall or it would 
experience the same bug.

So, the same thing applies to clone().

Btw, the issue is not only with getpid(), I guess, but rather with any 
__thread variable (the so-called Thread Local Storage).

Bye
-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Sven Köhler]

>> If you want to be 100% sure you use a syscall, why not call the syscall
>> directly?
> 
> OK. It's an API call. But what's this API call defined to return? Isn't 
> it the
> pid of the process calling? If it is, the behavior of the lib is a bug. 
> If it
> isn't, the descriptions are wrong. Do you agree?

This glibc-Bug has been discussed on the LKML already. Linus was 
somewhat angry about it, since glibc should not cache. He said that this 
caching-stuff may be, because somewhat wanted to tune glibc for some 
synthetic benchmark that uses getpid() as a speed measurement. He also 
recommed to get rid of that cache - at least as far as i read the 
discussion.

So the conclusion is: override getpid() with something that does not 
cache anything since there are buggy glibc's out there, or use our own 
cache for the PID if syscalls cause to much overhead.

Thx
  Sven


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Henrik Nordstrom]

On Wed, 10 Nov 2004, Geert Uytterhoeven wrote:

> Why? getpid() is just an API call. No one guarantees it's actually a syscall.
> And IMHO caching getpid() is allowed, since the returned value won't change
> anyway.

And in case of pthreads SHOULD be different from the Linux process ID.

Regards
Henrik


-------------------------------------------------------
This SF.Net email is sponsored by: InterSystems CACHE
FREE OODBMS DOWNLOAD - A multidimensional database that combines
robust object and relational technologies, making it a perfect match
for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Blaisorblade]

On Wednesday 10 November 2004 22:19, Henrik Nordstrom wrote:
> On Wed, 10 Nov 2004, Geert Uytterhoeven wrote:
> > Why? getpid() is just an API call. No one guarantees it's actually a
> > syscall. And IMHO caching getpid() is allowed, since the returned value
> > won't change anyway.
The cache must be cleared on fork() (which is done) and on clone().

> And in case of pthreads SHOULD be different from the Linux process ID.
He was misunderstanding the thread - UML calls clone() without CLONE_THREAD, 
so that on this aspect it is fork()-like, so no thread group is created, just 
a child process. Which has a different PID. But glibc does not clean the 
thread-local-storage (or just the getpid() cached value in it) appropriately 
when using clone().
-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729


-------------------------------------------------------
This SF.Net email is sponsored by: InterSystems CACHE
FREE OODBMS DOWNLOAD - A multidimensional database that combines
robust object and relational technologies, making it a perfect match
for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] Re: Stop at startup on 2.6 NPTL hosts

[Blaisorblade]

On Wednesday 10 November 2004 09:36, stian@nixia.no wrote:
> > This fact (glibc caches getpid() results) was even known because already
> > posted, and I guessed that this could happen only with TLS support.
>
> That glibc caches getpid() has been a known issue before when people
> wanted to benchmark uml syscall speeds inside vs outside of UML using a
> simple libc-function.
In fact, it has been also said against SYSEMU benchmarks. Actually, it was 
obvious it didn't apply: in fact, this only happens with TLS, which is always 
disabled inside UML.

> A clone() call can't easily invalidate caches around 
> in libs that are stored in statics. But it should be mentioned on
> getpid()'s man/info page that the value might be cached.

> Perhaps sende the glibc dudes an email with the problem, and either get
> them to remove the cache or document it. Either way it will be a problem,
> since people might use a version of glibc then that caches getpid.. only
> work-around  is to use syscall() directly, or avoid using getpid() before
> clone().
The first one is better, I'm going to use it.

Bye
-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel