Accessing the user stack inside system call service routine

Accessing the user stack inside system call service routine

[Steffen Rumler]

Hello,

I want to access/record the user stack within the system call
service routine sys_sigsuspend() (arch/ppc/signal.c),
in order to debug hanging threads.

The user register context is passed to sys_sigsuspend() via
the 'struct pt_regs *regs' argument.

In this way I can obtain the stack pointer r1
(via regs->gpr[1]) and catch the PPC EABI stack frames.

But I see all zeros behind 'regs->gpr[1]' instead of proper
stack frames. What is wrong ?

How can I access the user stack within a
system call service routine (sys_XXX) ?

Thank you for help,


Steffen

--


--------------------------------------------------------------

Steffen Rumler
ICN ON PNE SB 4
Siemens AG
Hofmannstr. 51                 Email: Steffen.Rumler@icn.siemens.de
D-81359 Munich                 Phone: +49 89 722-44061
Germany                        Fax  : +49 89 722-36703

--------------------------------------------------------------

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Accessing the user stack inside system call service routine

[Neil Horman]

	Since the process virtual address which are used on the user space stack are not
directly available from the context of the kernel, you will need to do some
virtual address translation to access teh stack frames you are interested in.  I
would suggest that you look at the source code for the copy_to_user and
copy_from_user functions to see how they access a process address space from
kernel space.  That will probably start you in the right direction.
hope that helps!
Neil :)


Steffen Rumler wrote:

> Hello,
>
> I want to access/record the user stack within the system call
> service routine sys_sigsuspend() (arch/ppc/signal.c),
> in order to debug hanging threads.
>
> The user register context is passed to sys_sigsuspend() via
> the 'struct pt_regs *regs' argument.
>
> In this way I can obtain the stack pointer r1
> (via regs->gpr[1]) and catch the PPC EABI stack frames.
>
> But I see all zeros behind 'regs->gpr[1]' instead of proper
> stack frames. What is wrong ?
>
> How can I access the user stack within a
> system call service routine (sys_XXX) ?
>
> Thank you for help,
>
>
> Steffen
>
> --
>
>
> --------------------------------------------------------------
>
> Steffen Rumler
> ICN ON PNE SB 4
> Siemens AG
> Hofmannstr. 51                 Email: Steffen.Rumler@icn.siemens.de
> D-81359 Munich                 Phone: +49 89 722-44061
> Germany                        Fax  : +49 89 722-36703
>
> --------------------------------------------------------------
>
>


--
/******************************************************************
*Neil Horman
*Software Engineer
*LVL7 Systems
*13000 Weston Pkwy.
*Cary, NC 27513
*(919)-865-2915
*nhorman@lvl7.com
*PGP keyID 0xB5E1020A
*http://www.keyserver.net/en
*******************************************************************/


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Accessing the user stack inside system call service routine

[Steffen Rumler]

>         Since the process virtual address which are used on the user space stack are not
> directly available from the context of the kernel, you will need to do some
> virtual address translation to access teh stack frames you are interested in.  I
> would suggest that you look at the source code for the copy_to_user and
> copy_from_user functions to see how they access a process address space from
> kernel space.  That will probably start you in the right direction.
> hope that helps!
> Neil :)
>

Thank you for the answer.

I have not accessed the user  space directly.
Instead I have just used copy_from_user():


int
sys_sigsuspend(old_sigset_t mask, int p2, int p3, int p4, int p6, int
p7,
	       struct pt_regs *regs)
{
  ...
  while (1) {

    current->state = TASK_INTERRUPTIBLE;

    current->in_suspend = 1;     /* XXX my stuff for debugging */
    current->user_regs  = regs;
    if(copy_from_user(current->user_stack_xxx, regs->gpr[1], 128)){

      printk("\n------------- copy_from_user failed: %d\n", ret);
    }

    schedule();

    current->in_suspend = 0;    /* XXX my stuff for debugging */
    current->user_regs  = NULL;
    ...
}

The copy_from_user() works fine. The printk() message does not
appear.

Later, when I inspect 'current->user_stack_xxx' (temporary
added to struct_task) for all processes with 'current->in_suspend'
set, all seems to be zero. I do this check within a kernel
module, I can load when the threads are hanging.

Steffen

--


--------------------------------------------------------------

Steffen Rumler
ICN ON PNE SB 4
Siemens AG
Hofmannstr. 51                 Email: Steffen.Rumler@icn.siemens.de
D-81359 Munich                 Phone: +49 89 722-44061
Germany                        Fax  : +49 89 722-36703

--------------------------------------------------------------

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Accessing the user stack inside system call service routine

[Neil Horman]

Ahh, so you had already covered the addressing point.  sorry.  Well, the next
thing that would occur to me is that 128 bytes of all zero's may be ok.  I'm not
quite sure of the details of the inner workings of the standard C library for
this situation, but right before the code in the C library executes a trap
instruction to make the reuqested system call, it may append extra data to the
last user space stack frame.  I know this happens during signal handling.
Granted that call stack is transitioning the other way, but you see the same
sort of behavior, in which the first "stack frame" is not really a stack frame,
but rather a block of data which is removed before a return to the application
at the point where it called the library function.  I would suggest that you try
   copying more than 128 bytes to see if you start seeing valid stack frames, or
more directly, invistigate the C library code which wraps around the system call
in user space to see if it makes and strange manipulations to the stack frame
before executing a trap.  Good luck!
Neil

Steffen Rumler wrote:

>>        Since the process virtual address which are used on the user space stack are not
>>directly available from the context of the kernel, you will need to do some
>>virtual address translation to access teh stack frames you are interested in.  I
>>would suggest that you look at the source code for the copy_to_user and
>>copy_from_user functions to see how they access a process address space from
>>kernel space.  That will probably start you in the right direction.
>>hope that helps!
>>Neil :)
>>
>>
>
> Thank you for the answer.
>
> I have not accessed the user  space directly.
> Instead I have just used copy_from_user():
>
>
> int
> sys_sigsuspend(old_sigset_t mask, int p2, int p3, int p4, int p6, int
> p7,
> 	       struct pt_regs *regs)
> {
>   ...
>   while (1) {
>
>     current->state = TASK_INTERRUPTIBLE;
>
>     current->in_suspend = 1;     /* XXX my stuff for debugging */
>     current->user_regs  = regs;
>     if(copy_from_user(current->user_stack_xxx, regs->gpr[1], 128)){
>
>       printk("\n------------- copy_from_user failed: %d\n", ret);
>     }
>
>     schedule();
>
>     current->in_suspend = 0;    /* XXX my stuff for debugging */
>     current->user_regs  = NULL;
>     ...
> }
>
> The copy_from_user() works fine. The printk() message does not
> appear.
>
> Later, when I inspect 'current->user_stack_xxx' (temporary
> added to struct_task) for all processes with 'current->in_suspend'
> set, all seems to be zero. I do this check within a kernel
> module, I can load when the threads are hanging.
>
> Steffen
>
>


--
/******************************************************************
*Neil Horman
*Software Engineer
*LVL7 Systems
*13000 Weston Pkwy.
*Cary, NC 27513
*(919)-865-2915
*nhorman@lvl7.com
*PGP keyID 0xB5E1020A
*http://www.keyserver.net/en
*******************************************************************/

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Accessing the user stack inside system call service routine

[Paul Mackerras]

Steffen Rumler writes:

> The copy_from_user() works fine. The printk() message does not
> appear.

The copy_from_user call looks correct to me.

> Later, when I inspect 'current->user_stack_xxx' (temporary
> added to struct_task) for all processes with 'current->in_suspend'
> set, all seems to be zero. I do this check within a kernel
> module, I can load when the threads are hanging.

That is indeed very strange.  The first word should be non-zero at the
very least.  Is it possible that your program has in fact trashed its
stack?  Another thing to try would be to check the contents of
current->user_stack_xxx immediately after you copy it to see whether
it is all zero at that point.

Paul.

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Accessing the user stack inside system call service routine

[Frank Rowand]

Paul Mackerras wrote:
>
> Steffen Rumler writes:
>
> > The copy_from_user() works fine. The printk() message does not
> > appear.
>
> The copy_from_user call looks correct to me.
>
> > Later, when I inspect 'current->user_stack_xxx' (temporary
> > added to struct_task) for all processes with 'current->in_suspend'
> > set, all seems to be zero. I do this check within a kernel
> > module, I can load when the threads are hanging.
>
> That is indeed very strange.  The first word should be non-zero at the
> very least.  Is it possible that your program has in fact trashed its
> stack?  Another thing to try would be to check the contents of
> current->user_stack_xxx immediately after you copy it to see whether
> it is all zero at that point.

Here is how the Linux Trace Toolkit (LTT) kernel patch searches back
through the stack, in the case of a syscall, to find the caller of
the library function that resulted in a syscall (for example, the
place that printf() was called from, not the place that the resulting
syscall was called from in the library).  It sounds like you don't
need to dig that far back in the calling sequence.  Hopefully this example
can point you in the right direction.



#if (CONFIG_TRACE || CONFIG_TRACE_MODULE)
asmlinkage void trace_real_syscall_entry(struct pt_regs * regs)
{
        int                 use_depth;
        int                 use_bounds;
        int                 depth = 0;
        int                 seek_depth;
        unsigned long       lower_bound;
        unsigned long       upper_bound;
        unsigned long       addr;
        unsigned long*      stack;
        trace_syscall_entry trace_syscall_event;

        /* Set the syscall ID */
        trace_syscall_event.syscall_id = (uint8_t) regs->gpr[0];

        /* Set the address in any case */
        trace_syscall_event.address  = instruction_pointer(regs);

        /* Are we in the kernel (This is a kernel thread)? */
        if(!user_mode(regs))
          /* Don't go digining anywhere */
          goto trace_syscall_end;

        /* Get the trace configuration */
        if(trace_get_config(&use_depth,
                            &use_bounds,
                            &seek_depth,
                            (void*)&lower_bound,
                            (void*)&upper_bound) < 0)
          goto trace_syscall_end;

        /* Do we have to search for an eip address range */
        if((use_depth == 1) || (use_bounds == 1))
          {
          /* Start at the top of the stack (bottom address since stacks grow downward) */
          stack = (unsigned long*) regs->gpr[1];

          /* Skip over first stack frame as the return address isn't valid */
          if(get_user(addr, stack))
            goto trace_syscall_end;
          stack = (unsigned long*) addr;

          /* Keep on going until we reach the end of the process' stack limit (wherever it may be) */
          while(!get_user(addr, stack + 1)) /* "stack + 1", since this is where the IP is */
            {
            /* Does this LOOK LIKE an address in the program */
            if((addr > current->mm->start_code)
             &&(addr < current->mm->end_code))
              {
              /* Does this address fit the description */
              if(((use_depth == 1) && (depth == seek_depth))
               ||((use_bounds == 1) && (addr > lower_bound) && (addr < upper_bound)))
                {
                /* Set the address */
                trace_syscall_event.address = addr;

                /* We're done */
                goto trace_syscall_end;
                }
              else
                /* We're one depth more */
                depth++;
              }
            /* Go on to the next address */
            if(get_user(addr, stack))
              goto trace_syscall_end;
            stack = (unsigned long*) addr;
            }
          }

trace_syscall_end:
        /* Trace the event */
        trace_event(TRACE_EV_SYSCALL_ENTRY, &trace_syscall_event);
}





-Frank
--
Frank Rowand <frank_rowand@mvista.com>
MontaVista Software, Inc

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Accessing the user stack inside system call service routine

[Steffen Rumler]

> That is indeed very strange.  The first word should be non-zero at the
> very least.  Is it possible that your program has in fact trashed its
> stack?  Another thing to try would be to check the contents of
> current->user_stack_xxx immediately after you copy it to see whether
> it is all zero at that point.

Thank you all for the answer, it is now working fine.

I have basically used the LTT routine to analyze the EABI stack
frames, provided by Frank.

When I copy the relevant addresses (backchain pointer, saved link
register)
to separate static buffer and not to the task structure it is
working.


Steffen


--


--------------------------------------------------------------

Steffen Rumler
ICN ON PNE SB 4
Siemens AG
Hofmannstr. 51                 Email: Steffen.Rumler@icn.siemens.de
D-81359 Munich                 Phone: +49 89 722-44061
Germany                        Fax  : +49 89 722-36703

--------------------------------------------------------------

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/