Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Jonathan Andrews]

Sorry all me again.

I've written a program that picks up UDP broadcast packets containing
audio from a number of sources, mixes it together and presents it to
alsa for playback.

I'm experiencing random freezes.  I cant reproduce then running the
process under strace as the process on the ARM board slows down to the
point it underruns with ALSA all the time and the symptom doesn't show.

If I attach strace after the process has frozen I get this and nothing
else:

# strace -p 443
Process 443 attached - interrupt to quit
futex(0x4030e144, FUTEX_WAIT_PRIVATE, 2, NULL


This stop happens at random at any point typically after minutes but run
time can be over an hour.

I need to work around this as I need the board for a demo.

Im using, alsa-lib-1.0.25
kernel 2.6.36.4
BusyBox v1.19.2

I also compiled strace for arm, but this and a busybox shell is the only
environment I have on the board.

The problem showed when I starting using this function to poll alsa:

The same code on my PC dual core AMD does not seem to suffer the
problem  : alsa-lib-1.0.23, Linux jonspc 2.6.32.26-175.fc12.i686 #1 SMP

Before anyone points it out I want to poll as alsa is not the only sound
output supported by the code.

int sound_system_needs_data_now()
{
        int framesleft;
        snd_pcm_status_t *status;
        snd_pcm_status_alloca( &status );

        // If the alsa playback has stalled then it definately needs data
        snd_pcm_status(playback_handle,status);
        if( snd_pcm_status_get_state( status ) != SND_PCM_STATE_RUNNING )
                return(TRUE);

        framesleft=snd_pcm_avail (playback_handle);
	//printf("FL=%d\n",framesleft); fflush(stdout);

        if (framesleft>1000)
                return(TRUE);
        else
                return(FALSE);
}


Stuck process

# cat /proc/443/status 
Name:   udp-many-way-au
State:  S (sleeping)
Tgid:   443
Pid:    443
PPid:   392
TracerPid:      0
Uid:    0       0       0       0
Gid:    0       0       0       0
FDSize: 32
Groups: 
VmPeak:     4356 kB
VmSize:     4292 kB
VmLck:       132 kB
VmHWM:      1788 kB
VmRSS:      1788 kB
VmData:      396 kB
VmStk:       136 kB
VmExe:        28 kB
VmLib:      3348 kB
VmPTE:        12 kB
VmSwap:        0 kB
Threads:        1
SigQ:   0/471
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000002000
SigIgn: 0000000000000006
SigCgt: 0000000180002000
CapInh: 0000000000000000
CapPrm: ffffffffffffffff
CapEff: ffffffffffffffff
CapBnd: ffffffffffffffff
Cpus_allowed:   1
Cpus_allowed_list:      0
Mems_allowed:   1
Mems_allowed_list:      0
voluntary_ctxt_switches:        5908
nonvoluntary_ctxt_switches:     1766


Any useful suggestions welcome.

Thanks,
Jon

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Jonathan Andrews]

Reposting this:

# strace -p 443
> Process 443 attached - interrupt to quit
> futex(0x4030e144, FUTEX_WAIT_PRIVATE, 2, NULL
> 
locks up.  Stops, stalls ... fails to continue ....


Now tried several alsa-lib versions.  A clue would be nice?
What do I replace to fix this ?

Anyone ?


Thanks,
Jon



On Thu, 2012-02-16 at 12:01 +0000, Jonathan Andrews wrote:
> Sorry all me again.
> 
> I've written a program that picks up UDP broadcast packets containing
> audio from a number of sources, mixes it together and presents it to
> alsa for playback.
> 
> I'm experiencing random freezes.  I cant reproduce then running the
> process under strace as the process on the ARM board slows down to the
> point it underruns with ALSA all the time and the symptom doesn't show.
> 
> If I attach strace after the process has frozen I get this and nothing
> else:
> 
> # strace -p 443
> Process 443 attached - interrupt to quit
> futex(0x4030e144, FUTEX_WAIT_PRIVATE, 2, NULL
> 
> 
> This stop happens at random at any point typically after minutes but run
> time can be over an hour.
> 
> I need to work around this as I need the board for a demo.
> 
> Im using, alsa-lib-1.0.25
> kernel 2.6.36.4
> BusyBox v1.19.2
> 
> I also compiled strace for arm, but this and a busybox shell is the only
> environment I have on the board.
> 
> The problem showed when I starting using this function to poll alsa:
> 
> The same code on my PC dual core AMD does not seem to suffer the
> problem  : alsa-lib-1.0.23, Linux jonspc 2.6.32.26-175.fc12.i686 #1 SMP
> 
> Before anyone points it out I want to poll as alsa is not the only sound
> output supported by the code.
> 
> int sound_system_needs_data_now()
> {
>         int framesleft;
>         snd_pcm_status_t *status;
>         snd_pcm_status_alloca( &status );
> 
>         // If the alsa playback has stalled then it definately needs data
>         snd_pcm_status(playback_handle,status);
>         if( snd_pcm_status_get_state( status ) != SND_PCM_STATE_RUNNING )
>                 return(TRUE);
> 
>         framesleft=snd_pcm_avail (playback_handle);
> 	//printf("FL=%d\n",framesleft); fflush(stdout);
> 
>         if (framesleft>1000)
>                 return(TRUE);
>         else
>                 return(FALSE);
> }
> 
> 
> Stuck process
> 
> # cat /proc/443/status 
> Name:   udp-many-way-au
> State:  S (sleeping)
> Tgid:   443
> Pid:    443
> PPid:   392
> TracerPid:      0
> Uid:    0       0       0       0
> Gid:    0       0       0       0
> FDSize: 32
> Groups: 
> VmPeak:     4356 kB
> VmSize:     4292 kB
> VmLck:       132 kB
> VmHWM:      1788 kB
> VmRSS:      1788 kB
> VmData:      396 kB
> VmStk:       136 kB
> VmExe:        28 kB
> VmLib:      3348 kB
> VmPTE:        12 kB
> VmSwap:        0 kB
> Threads:        1
> SigQ:   0/471
> SigPnd: 0000000000000000
> ShdPnd: 0000000000000000
> SigBlk: 0000000000002000
> SigIgn: 0000000000000006
> SigCgt: 0000000180002000
> CapInh: 0000000000000000
> CapPrm: ffffffffffffffff
> CapEff: ffffffffffffffff
> CapBnd: ffffffffffffffff
> Cpus_allowed:   1
> Cpus_allowed_list:      0
> Mems_allowed:   1
> Mems_allowed_list:      0
> voluntary_ctxt_switches:        5908
> nonvoluntary_ctxt_switches:     1766
> 
> 
> Any useful suggestions welcome.
> 
> Thanks,
> Jon
> 
> 
> _______________________________________________
> Alsa-devel mailing list
> Alsa-devel@alsa-project.org
> http://mailman.alsa-project.org/mailman/listinfo/alsa-devel

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Jonathan Andrews]

On Fri, 2012-02-17 at 21:04 +0000, Jonathan Andrews wrote:
> Reposting this:
> 
> # strace -p 443
> > Process 443 attached - interrupt to quit
> > futex(0x4030e144, FUTEX_WAIT_PRIVATE, 2, NULL
> > 
> locks up.  Stops, stalls ... fails to continue ....
> 
> 
> Now tried several alsa-lib versions.  A clue would be nice?
> What do I replace to fix this ?
> 
Now tried 2 kernels, 3 versions of alsalib still stalls.

If I remove alsa code it works forever.

Thanks to everyone who helped (no one at all).

FUCK OPEN SOURCE !!!!!!!!!!!

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Clemens Ladisch]

Jonathan Andrews wrote:
> On Fri, 2012-02-17 at 21:04 +0000, Jonathan Andrews wrote:
>> # strace -p 443
>>> futex(0x4030e144, FUTEX_WAIT_PRIVATE, 2, NULL
>>
>> locks up.  Stops, stalls ... fails to continue ....
>>
>> Now tried several alsa-lib versions.  A clue would be nice?
>> What do I replace to fix this ?
>>
> Now tried 2 kernels, 3 versions of alsalib still stalls.

The synchronization calls in alsa-lib shouldn't block unless you're
actually using several threads, and even then you wouldn't be able
to reproduce it so reliably.

Do you have a stack trace to find out where this is called from?

Can you confirm that pthread mutexes work on your board?


Regards,
Clemens

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Clemens Ladisch]

Jonathan Andrews wrote:
> I've tried building alsalib in many ways, at the moment its :
> $ ./configure --target=arm-linux --host=i686-linux --enable-shared --disable-static --without-pthread

Try adding --disable-ucm and --disable-python.


Regards,
Clemens

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Clemens Ladisch]

Jonathan Andrews wrote:
> On Mon, 2012-02-20 at 08:54 +0100, Clemens Ladisch wrote:
>> Jonathan Andrews wrote:
>>> I've tried building alsalib in many ways, at the moment its :
>>> $ ./configure --target=arm-linux --host=i686-linux --enable-shared --disable-static --without-pthread
>>
>> Try adding --disable-ucm and --disable-python.
>
> Err ok, alsa-lib still builds ok.

Now it doesn't use any pthread functions.

> My application runs for longer, but still freezes in the end. With the
> same futex locking problem.

I'd guess that this is not an alsa-lib problem.
Now you really need a stack trace.


Regards,
Clemens

Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Jonathan Andrews]

Using kernel 3.2.5 with alsa-lib 1.0.25, all compiled with generic
Debian arm-linux-gnueabi toolchain.

arm-linux-gnueabi-gcc (Debian 4.3.2-1.1) 4.3.2
Was used to build kernel, alsa-lib and application.

Changing gcc version, kernel version or alsa-lib version makes the
problem worse or better, but ALL versions seem to suffer this problem. I
have also seen it once on Intel (but only once so far).

Something seeks broken at a lower layer than im using.  I simply don't
have the skill to debug it.

The hardware is a USB cm109 audio adapter, but the problem seems to show
on more than this one driver.

The audio application writing to alsa will freezes at random intervals,
infrequent at the moment, last one was after runtime 20H 37M 29S.  Two
processes are running, one reading from the sound device and one writing
to the sound device. I am not using threading or anything very clever
just generic alsa functions.

This is the only diagnostic I can generate so far as running the
application under strace slows it to the point it no longer functions
enough to generate the problem.

ARM / # strace -p 417
Process 417 attached - interrupt to quit
futex(0x175734, FUTEX_WAIT_PRIVATE, 2, NULL^C <unfinished ...>
Process 417 detached

ARM / # uname -a
Linux (none) 3.2.5 #2 Wed Feb 22 17:11:52 GMT 2012 armv4tl GNU/Linux
ARM / # uptime
 22:36:19 up 22:36,  0 users,  load average: 0.15, 0.16, 0.18
ARM / # cat /proc/cpuinfo 
Processor       : ARM920T rev 0 (v4l)
BogoMIPS        : 199.06
Features        : swp half thumb crunch 
CPU implementer : 0x41
CPU architecture: 4T
CPU variant     : 0x1
CPU part        : 0x920
CPU revision    : 0


Any help welcome.

Thanks,
Jon

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Huang Shijie]

Hi ,

I meet a similar problem with the latest futex code.

I play the video and the processes will hang at the futex.

BR
Huang Shijie

On Fri, Feb 24, 2012 at 8:29 AM, Jonathan Andrews <jon@jonshouse.co.uk> wrote:
> Using kernel 3.2.5 with alsa-lib 1.0.25, all compiled with generic
> Debian arm-linux-gnueabi toolchain.
>
> arm-linux-gnueabi-gcc (Debian 4.3.2-1.1) 4.3.2
> Was used to build kernel, alsa-lib and application.
>
> Changing gcc version, kernel version or alsa-lib version makes the
> problem worse or better, but ALL versions seem to suffer this problem. I
> have also seen it once on Intel (but only once so far).
>
> Something seeks broken at a lower layer than im using.  I simply don't
> have the skill to debug it.
>
> The hardware is a USB cm109 audio adapter, but the problem seems to show
> on more than this one driver.
>
> The audio application writing to alsa will freezes at random intervals,
> infrequent at the moment, last one was after runtime 20H 37M 29S.  Two
> processes are running, one reading from the sound device and one writing
> to the sound device. I am not using threading or anything very clever
> just generic alsa functions.
>
> This is the only diagnostic I can generate so far as running the
> application under strace slows it to the point it no longer functions
> enough to generate the problem.
>
> ARM / # strace -p 417
> Process 417 attached - interrupt to quit
> futex(0x175734, FUTEX_WAIT_PRIVATE, 2, NULL^C <unfinished ...>
> Process 417 detached
>
> ARM / # uname -a
> Linux (none) 3.2.5 #2 Wed Feb 22 17:11:52 GMT 2012 armv4tl GNU/Linux
> ARM / # uptime
>  22:36:19 up 22:36,  0 users,  load average: 0.15, 0.16, 0.18
> ARM / # cat /proc/cpuinfo
> Processor       : ARM920T rev 0 (v4l)
> BogoMIPS        : 199.06
> Features        : swp half thumb crunch
> CPU implementer : 0x41
> CPU architecture: 4T
> CPU variant     : 0x1
> CPU part        : 0x920
> CPU revision    : 0
>
>
> Any help welcome.
>
> Thanks,
> Jon
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Darren Hart]

On 02/29/2012 01:12 AM, Huang Shijie wrote:
> Hi ,
> 
> I meet a similar problem with the latest futex code.
> 
> I play the video and the processes will hang at the futex.

Are either of you able to bisect the kernel? At the very least can you
find two kernels where it works and where it does not?

Hanging on FUTEX_WAIT_PRIVATE can be the symptom for higher level
problems including userspace locking issues and race conditions.

Huang, are you also on ARM?

--
Darren

> 
> BR
> Huang Shijie
> 
> On Fri, Feb 24, 2012 at 8:29 AM, Jonathan Andrews <jon@jonshouse.co.uk> wrote:
>> Using kernel 3.2.5 with alsa-lib 1.0.25, all compiled with generic
>> Debian arm-linux-gnueabi toolchain.
>>
>> arm-linux-gnueabi-gcc (Debian 4.3.2-1.1) 4.3.2
>> Was used to build kernel, alsa-lib and application.
>>
>> Changing gcc version, kernel version or alsa-lib version makes the
>> problem worse or better, but ALL versions seem to suffer this problem. I
>> have also seen it once on Intel (but only once so far).
>>
>> Something seeks broken at a lower layer than im using.  I simply don't
>> have the skill to debug it.
>>
>> The hardware is a USB cm109 audio adapter, but the problem seems to show
>> on more than this one driver.
>>
>> The audio application writing to alsa will freezes at random intervals,
>> infrequent at the moment, last one was after runtime 20H 37M 29S.  Two
>> processes are running, one reading from the sound device and one writing
>> to the sound device. I am not using threading or anything very clever
>> just generic alsa functions.
>>
>> This is the only diagnostic I can generate so far as running the
>> application under strace slows it to the point it no longer functions
>> enough to generate the problem.
>>
>> ARM / # strace -p 417
>> Process 417 attached - interrupt to quit
>> futex(0x175734, FUTEX_WAIT_PRIVATE, 2, NULL^C <unfinished ...>
>> Process 417 detached
>>
>> ARM / # uname -a
>> Linux (none) 3.2.5 #2 Wed Feb 22 17:11:52 GMT 2012 armv4tl GNU/Linux
>> ARM / # uptime
>>  22:36:19 up 22:36,  0 users,  load average: 0.15, 0.16, 0.18
>> ARM / # cat /proc/cpuinfo
>> Processor       : ARM920T rev 0 (v4l)
>> BogoMIPS        : 199.06
>> Features        : swp half thumb crunch
>> CPU implementer : 0x41
>> CPU architecture: 4T
>> CPU variant     : 0x1
>> CPU part        : 0x920
>> CPU revision    : 0
>>
>>
>> Any help welcome.
>>
>> Thanks,
>> Jon
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at  http://www.tux.org/lkml/
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Jonathan Andrews]

On Wed, 2012-03-07 at 12:07 -0800, Darren Hart wrote:
> 
> On 02/29/2012 01:12 AM, Huang Shijie wrote:
> > Hi ,
> > 
> > I meet a similar problem with the latest futex code.
> > 
> > I play the video and the processes will hang at the futex.
> 
> Are either of you able to bisect the kernel?
I'm not a kernel hacker what do you mean ?

>  At the very least can you
> find two kernels where it works and where it does not?
> 
> Hanging on FUTEX_WAIT_PRIVATE can be the symptom for higher level
> problems including userspace locking issues and race conditions.


My workload is UDP network audio.  I have compiled my code with and
without ALSA support. The version without ALSA seems to run forever, the
version with ALSA works on ARM for between a few minutes and a few
hours. On Intel the same futex stall problem occurs, but it may take
runtime of days.

I have two processes running. One RX process that takes UDP packets from
the network mixes them and presents them to ALSA as an audio stream, the
second process takes audio from the sound device and transmits it as a
UDP audio stream.  The two processes are independent. 

My workload is atypical as I need to both transmit and receive audio via
UDP on a 27/7 basis.

So far I have experienced the problem on 3 kernels, but I have tried
only 3 kernels it may be all 2.6 kernels that suffer.

My development PC is "Linux jonspc 2.6.32.26-175.fc12.i686 #1 SMP Wed
Dec 1 21:52:04 UTC 2010 i686 athlon i386 GNU/Linux"

My ARM board target:
ARM / # uname -a
Linux (none) 3.2.5 #2 Wed Feb 22 17:11:52 GMT 2012 armv4tl GNU/Linux

And my ARM target running its older kernel was (2.6.36).
 
I have an strace of the process running and stalling on the PC.
The file is 2GB, its not a fast link sorry.
 
http://www.jonshouse.co.uk/download/a_stop.txt


Many thanks,
Jon

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Huang Shijie]

Hi,

On Thu, Mar 8, 2012 at 4:07 AM, Darren Hart <dvhart@linux.intel.com> wrote:
>
>
> On 02/29/2012 01:12 AM, Huang Shijie wrote:
>> Hi ,
>>
>> I meet a similar problem with the latest futex code.
>>
>> I play the video and the processes will hang at the futex.
>
> Are either of you able to bisect the kernel? At the very least can you

I finially found my arch/arm/include/asm/futex.h is not the
latest, so i updated the header.

And the futex issue is gone now. But a dataAbort issue appears, I am
not sure whether it caused by the futex patch.
I am debugging it now.


BR
Huang Shijie
> find two kernels where it works and where it does not?
>
> Hanging on FUTEX_WAIT_PRIVATE can be the symptom for higher level
> problems including userspace locking issues and race conditions.
>
> Huang, are you also on ARM?

Yes, FREESCALE imx6q platform.

BR
Huang Shijie
>
> --
> Darren
>
>>
>> BR
>> Huang Shijie
>>
>> On Fri, Feb 24, 2012 at 8:29 AM, Jonathan Andrews <jon@jonshouse.co.uk> wrote:
>>> Using kernel 3.2.5 with alsa-lib 1.0.25, all compiled with generic
>>> Debian arm-linux-gnueabi toolchain.
>>>
>>> arm-linux-gnueabi-gcc (Debian 4.3.2-1.1) 4.3.2
>>> Was used to build kernel, alsa-lib and application.
>>>
>>> Changing gcc version, kernel version or alsa-lib version makes the
>>> problem worse or better, but ALL versions seem to suffer this problem. I
>>> have also seen it once on Intel (but only once so far).
>>>
>>> Something seeks broken at a lower layer than im using.  I simply don't
>>> have the skill to debug it.
>>>
>>> The hardware is a USB cm109 audio adapter, but the problem seems to show
>>> on more than this one driver.
>>>
>>> The audio application writing to alsa will freezes at random intervals,
>>> infrequent at the moment, last one was after runtime 20H 37M 29S.  Two
>>> processes are running, one reading from the sound device and one writing
>>> to the sound device. I am not using threading or anything very clever
>>> just generic alsa functions.
>>>
>>> This is the only diagnostic I can generate so far as running the
>>> application under strace slows it to the point it no longer functions
>>> enough to generate the problem.
>>>
>>> ARM / # strace -p 417
>>> Process 417 attached - interrupt to quit
>>> futex(0x175734, FUTEX_WAIT_PRIVATE, 2, NULL^C <unfinished ...>
>>> Process 417 detached
>>>
>>> ARM / # uname -a
>>> Linux (none) 3.2.5 #2 Wed Feb 22 17:11:52 GMT 2012 armv4tl GNU/Linux
>>> ARM / # uptime
>>>  22:36:19 up 22:36,  0 users,  load average: 0.15, 0.16, 0.18
>>> ARM / # cat /proc/cpuinfo
>>> Processor       : ARM920T rev 0 (v4l)
>>> BogoMIPS        : 199.06
>>> Features        : swp half thumb crunch
>>> CPU implementer : 0x41
>>> CPU architecture: 4T
>>> CPU variant     : 0x1
>>> CPU part        : 0x920
>>> CPU revision    : 0
>>>
>>>
>>> Any help welcome.
>>>
>>> Thanks,
>>> Jon
>>>
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> Please read the FAQ at  http://www.tux.org/lkml/
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at  http://www.tux.org/lkml/
>
> --
> Darren Hart
> Intel Open Source Technology Center
> Yocto Project - Linux Kernel

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Darren Hart]

On 03/07/2012 06:28 PM, Huang Shijie wrote:
> Hi,
> 
> On Thu, Mar 8, 2012 at 4:07 AM, Darren Hart <dvhart@linux.intel.com> wrote:
>>
>>
>> On 02/29/2012 01:12 AM, Huang Shijie wrote:
>>> Hi ,
>>>
>>> I meet a similar problem with the latest futex code.
>>>
>>> I play the video and the processes will hang at the futex.
>>
>> Are either of you able to bisect the kernel? At the very least can you
> 
> I finially found my arch/arm/include/asm/futex.h is not the
> latest, so i updated the header.

Just make sure it matches your kernel version.

> 
> And the futex issue is gone now. But a dataAbort issue appears, I am
> not sure whether it caused by the futex patch.
> I am debugging it now.

Which APIs are you using that make the futex syscall?

--
Darren

> 
> 
> BR
> Huang Shijie
>> find two kernels where it works and where it does not?
>>
>> Hanging on FUTEX_WAIT_PRIVATE can be the symptom for higher level
>> problems including userspace locking issues and race conditions.
>>
>> Huang, are you also on ARM?
> 
> Yes, FREESCALE imx6q platform.
> 
> BR
> Huang Shijie
>>
>> --
>> Darren
>>
>>>
>>> BR
>>> Huang Shijie
>>>
>>> On Fri, Feb 24, 2012 at 8:29 AM, Jonathan Andrews <jon@jonshouse.co.uk> wrote:
>>>> Using kernel 3.2.5 with alsa-lib 1.0.25, all compiled with generic
>>>> Debian arm-linux-gnueabi toolchain.
>>>>
>>>> arm-linux-gnueabi-gcc (Debian 4.3.2-1.1) 4.3.2
>>>> Was used to build kernel, alsa-lib and application.
>>>>
>>>> Changing gcc version, kernel version or alsa-lib version makes the
>>>> problem worse or better, but ALL versions seem to suffer this problem. I
>>>> have also seen it once on Intel (but only once so far).
>>>>
>>>> Something seeks broken at a lower layer than im using.  I simply don't
>>>> have the skill to debug it.
>>>>
>>>> The hardware is a USB cm109 audio adapter, but the problem seems to show
>>>> on more than this one driver.
>>>>
>>>> The audio application writing to alsa will freezes at random intervals,
>>>> infrequent at the moment, last one was after runtime 20H 37M 29S.  Two
>>>> processes are running, one reading from the sound device and one writing
>>>> to the sound device. I am not using threading or anything very clever
>>>> just generic alsa functions.
>>>>
>>>> This is the only diagnostic I can generate so far as running the
>>>> application under strace slows it to the point it no longer functions
>>>> enough to generate the problem.
>>>>
>>>> ARM / # strace -p 417
>>>> Process 417 attached - interrupt to quit
>>>> futex(0x175734, FUTEX_WAIT_PRIVATE, 2, NULL^C <unfinished ...>
>>>> Process 417 detached
>>>>
>>>> ARM / # uname -a
>>>> Linux (none) 3.2.5 #2 Wed Feb 22 17:11:52 GMT 2012 armv4tl GNU/Linux
>>>> ARM / # uptime
>>>>  22:36:19 up 22:36,  0 users,  load average: 0.15, 0.16, 0.18
>>>> ARM / # cat /proc/cpuinfo
>>>> Processor       : ARM920T rev 0 (v4l)
>>>> BogoMIPS        : 199.06
>>>> Features        : swp half thumb crunch
>>>> CPU implementer : 0x41
>>>> CPU architecture: 4T
>>>> CPU variant     : 0x1
>>>> CPU part        : 0x920
>>>> CPU revision    : 0
>>>>
>>>>
>>>> Any help welcome.
>>>>
>>>> Thanks,
>>>> Jon
>>>>
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>> Please read the FAQ at  http://www.tux.org/lkml/
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> Please read the FAQ at  http://www.tux.org/lkml/
>>
>> --
>> Darren Hart
>> Intel Open Source Technology Center
>> Yocto Project - Linux Kernel

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Darren Hart]

On 03/07/2012 01:22 PM, Jonathan Andrews wrote:
> On Wed, 2012-03-07 at 12:07 -0800, Darren Hart wrote:
>>
>> On 02/29/2012 01:12 AM, Huang Shijie wrote:
>>> Hi ,
>>>
>>> I meet a similar problem with the latest futex code.
>>>
>>> I play the video and the processes will hang at the futex.
>>
>> Are either of you able to bisect the kernel?
> I'm not a kernel hacker what do you mean ?

Google for "git bisect". It's a way to divide-and-conquer and find
exactly where the kernel stops working for you. You would need a known
working kernel though. If you don't have a known working kernel, then
this is either a long standing bug (not a regression) or, I suspect more
likely, a locking bug or race in the userspace code.

When you see your thread stuck in FUTEX_WAIT_PRIVATE, that in and of
itself is not indication of a problem. It's waiting there to be woken
(FUTEX_WAKE_PRIVATE) by another thread in your application. So the
questions you should be asking are:

Why is it blocked?
	pthread_cond_wait()?
	pthread_mutex_lock()?

Why isn't it being woken up?
	Did you call pthread_cond_broadcast() without taking the mutex
	first?

There are other API that use futexes under the covers. You should be
able to sort out where in your application your threads are blocked and
thus determine which API you are using - and from there how you were
expecting that to get woken up. These calls may be being made from
within ALSA, in which case you may need to elicit the help of the ALSA
developers.

--
Darren

> 
>>  At the very least can you
>> find two kernels where it works and where it does not?
>>
>> Hanging on FUTEX_WAIT_PRIVATE can be the symptom for higher level
>> problems including userspace locking issues and race conditions.
> 
> 
> My workload is UDP network audio.  I have compiled my code with and
> without ALSA support. The version without ALSA seems to run forever, the
> version with ALSA works on ARM for between a few minutes and a few
> hours. On Intel the same futex stall problem occurs, but it may take
> runtime of days.
> 
> I have two processes running. One RX process that takes UDP packets from
> the network mixes them and presents them to ALSA as an audio stream, the
> second process takes audio from the sound device and transmits it as a
> UDP audio stream.  The two processes are independent. 
> 
> My workload is atypical as I need to both transmit and receive audio via
> UDP on a 27/7 basis.
> 
> So far I have experienced the problem on 3 kernels, but I have tried
> only 3 kernels it may be all 2.6 kernels that suffer.
> 
> My development PC is "Linux jonspc 2.6.32.26-175.fc12.i686 #1 SMP Wed
> Dec 1 21:52:04 UTC 2010 i686 athlon i386 GNU/Linux"
> 
> My ARM board target:
> ARM / # uname -a
> Linux (none) 3.2.5 #2 Wed Feb 22 17:11:52 GMT 2012 armv4tl GNU/Linux
> 
> And my ARM target running its older kernel was (2.6.36).
>  
> I have an strace of the process running and stalling on the PC.
> The file is 2GB, its not a fast link sorry.
>  
> http://www.jonshouse.co.uk/download/a_stop.txt
> 
> 
> Many thanks,
> Jon
> 
> 
> 

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Huang Shijie]

hi,

On Thu, Mar 8, 2012 at 11:36 AM, Darren Hart <dvhart@linux.intel.com> wrote:
>
>
> On 03/07/2012 06:28 PM, Huang Shijie wrote:
>> Hi,
>>
>> On Thu, Mar 8, 2012 at 4:07 AM, Darren Hart <dvhart@linux.intel.com> wrote:
>>>
>>>
>>> On 02/29/2012 01:12 AM, Huang Shijie wrote:
>>>> Hi ,
>>>>
>>>> I meet a similar problem with the latest futex code.
>>>>
>>>> I play the video and the processes will hang at the futex.
>>>
>>> Are either of you able to bisect the kernel? At the very least can you
>>
>> I finially found my arch/arm/include/asm/futex.h is not the
>> latest, so i updated the header.
>
> Just make sure it matches your kernel version.
>
>>
>> And the futex issue is gone now. But a dataAbort issue appears, I am
>> not sure whether it caused by the futex patch.
>> I am debugging it now.
>
> Which APIs are you using that make the futex syscall?
>

futex_wait().

I cherry-pick the
" df77abc ARM: 7099/1: futex: preserve oldval in SMP
__futex_atomic_op" to the arch/arm/include/asm/futex.h.


BR
Huang Shijie

> --
> Darren
>
>>
>>
>> BR
>> Huang Shijie
>>> find two kernels where it works and where it does not?
>>>
>>> Hanging on FUTEX_WAIT_PRIVATE can be the symptom for higher level
>>> problems including userspace locking issues and race conditions.
>>>
>>> Huang, are you also on ARM?
>>
>> Yes, FREESCALE imx6q platform.
>>
>> BR
>> Huang Shijie
>>>
>>> --
>>> Darren
>>>
>>>>
>>>> BR
>>>> Huang Shijie
>>>>
>>>> On Fri, Feb 24, 2012 at 8:29 AM, Jonathan Andrews <jon@jonshouse.co.uk> wrote:
>>>>> Using kernel 3.2.5 with alsa-lib 1.0.25, all compiled with generic
>>>>> Debian arm-linux-gnueabi toolchain.
>>>>>
>>>>> arm-linux-gnueabi-gcc (Debian 4.3.2-1.1) 4.3.2
>>>>> Was used to build kernel, alsa-lib and application.
>>>>>
>>>>> Changing gcc version, kernel version or alsa-lib version makes the
>>>>> problem worse or better, but ALL versions seem to suffer this problem. I
>>>>> have also seen it once on Intel (but only once so far).
>>>>>
>>>>> Something seeks broken at a lower layer than im using.  I simply don't
>>>>> have the skill to debug it.
>>>>>
>>>>> The hardware is a USB cm109 audio adapter, but the problem seems to show
>>>>> on more than this one driver.
>>>>>
>>>>> The audio application writing to alsa will freezes at random intervals,
>>>>> infrequent at the moment, last one was after runtime 20H 37M 29S.  Two
>>>>> processes are running, one reading from the sound device and one writing
>>>>> to the sound device. I am not using threading or anything very clever
>>>>> just generic alsa functions.
>>>>>
>>>>> This is the only diagnostic I can generate so far as running the
>>>>> application under strace slows it to the point it no longer functions
>>>>> enough to generate the problem.
>>>>>
>>>>> ARM / # strace -p 417
>>>>> Process 417 attached - interrupt to quit
>>>>> futex(0x175734, FUTEX_WAIT_PRIVATE, 2, NULL^C <unfinished ...>
>>>>> Process 417 detached
>>>>>
>>>>> ARM / # uname -a
>>>>> Linux (none) 3.2.5 #2 Wed Feb 22 17:11:52 GMT 2012 armv4tl GNU/Linux
>>>>> ARM / # uptime
>>>>>  22:36:19 up 22:36,  0 users,  load average: 0.15, 0.16, 0.18
>>>>> ARM / # cat /proc/cpuinfo
>>>>> Processor       : ARM920T rev 0 (v4l)
>>>>> BogoMIPS        : 199.06
>>>>> Features        : swp half thumb crunch
>>>>> CPU implementer : 0x41
>>>>> CPU architecture: 4T
>>>>> CPU variant     : 0x1
>>>>> CPU part        : 0x920
>>>>> CPU revision    : 0
>>>>>
>>>>>
>>>>> Any help welcome.
>>>>>
>>>>> Thanks,
>>>>> Jon
>>>>>
>>>>>
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>> Please read the FAQ at  http://www.tux.org/lkml/
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>> Please read the FAQ at  http://www.tux.org/lkml/
>>>
>>> --
>>> Darren Hart
>>> Intel Open Source Technology Center
>>> Yocto Project - Linux Kernel
>
> --
> Darren Hart
> Intel Open Source Technology Center
> Yocto Project - Linux Kernel

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Darren Hart]

On 03/07/2012 08:24 PM, Huang Shijie wrote:
> hi,
> 
> On Thu, Mar 8, 2012 at 11:36 AM, Darren Hart <dvhart@linux.intel.com> wrote:
>>
>>
>> On 03/07/2012 06:28 PM, Huang Shijie wrote:
>>> Hi,
>>>
>>> On Thu, Mar 8, 2012 at 4:07 AM, Darren Hart <dvhart@linux.intel.com> wrote:
>>>>
>>>>
>>>> On 02/29/2012 01:12 AM, Huang Shijie wrote:
>>>>> Hi ,
>>>>>
>>>>> I meet a similar problem with the latest futex code.
>>>>>
>>>>> I play the video and the processes will hang at the futex.
>>>>
>>>> Are either of you able to bisect the kernel? At the very least can you
>>>
>>> I finially found my arch/arm/include/asm/futex.h is not the
>>> latest, so i updated the header.
>>
>> Just make sure it matches your kernel version.
>>
>>>
>>> And the futex issue is gone now. But a dataAbort issue appears, I am
>>> not sure whether it caused by the futex patch.
>>> I am debugging it now.
>>
>> Which APIs are you using that make the futex syscall?
>>
> 
> futex_wait().

You are calling the futex syscall directly from your application?

--
Darren

> 
> I cherry-pick the
> " df77abc ARM: 7099/1: futex: preserve oldval in SMP
> __futex_atomic_op" to the arch/arm/include/asm/futex.h.
> 

> 
> BR
> Huang Shijie
> 
>> --
>> Darren
>>
>>>
>>>
>>> BR
>>> Huang Shijie
>>>> find two kernels where it works and where it does not?
>>>>
>>>> Hanging on FUTEX_WAIT_PRIVATE can be the symptom for higher level
>>>> problems including userspace locking issues and race conditions.
>>>>
>>>> Huang, are you also on ARM?
>>>
>>> Yes, FREESCALE imx6q platform.
>>>
>>> BR
>>> Huang Shijie
>>>>
>>>> --
>>>> Darren
>>>>
>>>>>
>>>>> BR
>>>>> Huang Shijie
>>>>>
>>>>> On Fri, Feb 24, 2012 at 8:29 AM, Jonathan Andrews <jon@jonshouse.co.uk> wrote:
>>>>>> Using kernel 3.2.5 with alsa-lib 1.0.25, all compiled with generic
>>>>>> Debian arm-linux-gnueabi toolchain.
>>>>>>
>>>>>> arm-linux-gnueabi-gcc (Debian 4.3.2-1.1) 4.3.2
>>>>>> Was used to build kernel, alsa-lib and application.
>>>>>>
>>>>>> Changing gcc version, kernel version or alsa-lib version makes the
>>>>>> problem worse or better, but ALL versions seem to suffer this problem. I
>>>>>> have also seen it once on Intel (but only once so far).
>>>>>>
>>>>>> Something seeks broken at a lower layer than im using.  I simply don't
>>>>>> have the skill to debug it.
>>>>>>
>>>>>> The hardware is a USB cm109 audio adapter, but the problem seems to show
>>>>>> on more than this one driver.
>>>>>>
>>>>>> The audio application writing to alsa will freezes at random intervals,
>>>>>> infrequent at the moment, last one was after runtime 20H 37M 29S.  Two
>>>>>> processes are running, one reading from the sound device and one writing
>>>>>> to the sound device. I am not using threading or anything very clever
>>>>>> just generic alsa functions.
>>>>>>
>>>>>> This is the only diagnostic I can generate so far as running the
>>>>>> application under strace slows it to the point it no longer functions
>>>>>> enough to generate the problem.
>>>>>>
>>>>>> ARM / # strace -p 417
>>>>>> Process 417 attached - interrupt to quit
>>>>>> futex(0x175734, FUTEX_WAIT_PRIVATE, 2, NULL^C <unfinished ...>
>>>>>> Process 417 detached
>>>>>>
>>>>>> ARM / # uname -a
>>>>>> Linux (none) 3.2.5 #2 Wed Feb 22 17:11:52 GMT 2012 armv4tl GNU/Linux
>>>>>> ARM / # uptime
>>>>>>  22:36:19 up 22:36,  0 users,  load average: 0.15, 0.16, 0.18
>>>>>> ARM / # cat /proc/cpuinfo
>>>>>> Processor       : ARM920T rev 0 (v4l)
>>>>>> BogoMIPS        : 199.06
>>>>>> Features        : swp half thumb crunch
>>>>>> CPU implementer : 0x41
>>>>>> CPU architecture: 4T
>>>>>> CPU variant     : 0x1
>>>>>> CPU part        : 0x920
>>>>>> CPU revision    : 0
>>>>>>
>>>>>>
>>>>>> Any help welcome.
>>>>>>
>>>>>> Thanks,
>>>>>> Jon
>>>>>>
>>>>>>
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>> Please read the FAQ at  http://www.tux.org/lkml/
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>> Please read the FAQ at  http://www.tux.org/lkml/
>>>>
>>>> --
>>>> Darren Hart
>>>> Intel Open Source Technology Center
>>>> Yocto Project - Linux Kernel
>>
>> --
>> Darren Hart
>> Intel Open Source Technology Center
>> Yocto Project - Linux Kernel

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Huang Shijie]

On Thu, Mar 8, 2012 at 3:40 PM, Darren Hart <dvhart@linux.intel.com> wrote:
>
>
> On 03/07/2012 08:24 PM, Huang Shijie wrote:
>> hi,
>>
>> On Thu, Mar 8, 2012 at 11:36 AM, Darren Hart <dvhart@linux.intel.com> wrote:
>>>
>>>
>>> On 03/07/2012 06:28 PM, Huang Shijie wrote:
>>>> Hi,
>>>>
>>>> On Thu, Mar 8, 2012 at 4:07 AM, Darren Hart <dvhart@linux.intel.com> wrote:
>>>>>
>>>>>
>>>>> On 02/29/2012 01:12 AM, Huang Shijie wrote:
>>>>>> Hi ,
>>>>>>
>>>>>> I meet a similar problem with the latest futex code.
>>>>>>
>>>>>> I play the video and the processes will hang at the futex.
>>>>>
>>>>> Are either of you able to bisect the kernel? At the very least can you
>>>>
>>>> I finially found my arch/arm/include/asm/futex.h is not the
>>>> latest, so i updated the header.
>>>
>>> Just make sure it matches your kernel version.
>>>
>>>>
>>>> And the futex issue is gone now. But a dataAbort issue appears, I am
>>>> not sure whether it caused by the futex patch.
>>>> I am debugging it now.
>>>
>>> Which APIs are you using that make the futex syscall?
>>>
>>
>> futex_wait().
>
> You are calling the futex syscall directly from your application?
>

The application call futex(), but i ever hung at futex_wait().

Huang Shijie

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Clemens Ladisch]

Jonathan Andrews wrote:
> On Wed, 2012-02-22 at 17:23 +0100, Clemens Ladisch wrote:
>> Now it doesn't use any pthread functions.
>
> Thanks, use or require ? Still complains if I static compile my
> application but don't link with pthreads.

Please try with the patch below; this should remove the last
occurence of pthreads in alsa-lib.


Regards,
Clemens


--- alsa-lib/configure.in
+++ alsa-lib/configure.in
@@ -512,6 +512,7 @@ if test "$HAVE_LIBDL" != "yes"; then
 fi

 if test "$HAVE_LIBPTHREAD" != "yes"; then
+  build_pcm_meter="no"
   build_pcm_share="no"
 fi

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Clemens Ladisch]

Jonathan Andrews wrote:
> On Thu, 2012-03-08 at 10:55 +0100, Clemens Ladisch wrote:
>> Jonathan Andrews wrote:
>>> I'm using this script to build both static and dynamic versions of the
>>> alsalib. Can you check the configure arguments are correct.
>>
>> They look correct.
>
> Tried to compile my application, still fails asking for pthreads.

The configure.in file contains the patch, but the generated configure
script does not have it (although its time stamp is newer).
Please ensure that you've run autoconf (although make should do this
automatically).


Regards,
Clemens

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Jonathan Andrews]

On Thu, 2012-03-08 at 12:43 +0100, Clemens Ladisch wrote:
> Jonathan Andrews wrote:
> > On Thu, 2012-03-08 at 10:55 +0100, Clemens Ladisch wrote:
> >> Jonathan Andrews wrote:
> >>> I'm using this script to build both static and dynamic versions of the
> >>> alsalib. Can you check the configure arguments are correct.
> >>
> >> They look correct.
> >
> > Tried to compile my application, still fails asking for pthreads.
> 
> The configure.in file contains the patch, but the generated configure
> script does not have it (although its time stamp is newer).
> Please ensure that you've run autoconf (although make should do this
> automatically).
> 
> 
> Regards,
> Clemens

aclocal.m4:16: warning: this file was generated for autoconf 2.68.
You have another version of autoconf.  It may work, but is not
guaranteed to.
If you have problems, you may need to regenerate the build system
entirely.
To do so, use the procedure documented by the package, typically
`autoreconf'.
configure.in:15: error: Autoconf version 2.62 or higher is required
aclocal.m4:6823: AM_INIT_AUTOMAKE is expanded from...
configure.in:15: the top level
autom4te: /usr/bin/m4 failed with exit status: 63


Thanks for finding that, seems its a version problem.  My machine has
version 2.61

Upgrading now.

Thanks,
Jon

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Jonathan Andrews]

On Thu, 2012-03-08 at 12:43 +0100, Clemens Ladisch wrote:
> Jonathan Andrews wrote:
> > On Thu, 2012-03-08 at 10:55 +0100, Clemens Ladisch wrote:
> >> Jonathan Andrews wrote:
> >>> I'm using this script to build both static and dynamic versions of the
> >>> alsalib. Can you check the configure arguments are correct.
> >>
> >> They look correct.
> >
> > Tried to compile my application, still fails asking for pthreads.
> 
> The configure.in file contains the patch, but the generated configure
> script does not have it (although its time stamp is newer).
> Please ensure that you've run autoconf (although make should do this
> automatically).
> 
> 

Getting closer, alsalib now builds without the need for pthreads but
seems to insist on dlopen dlclose even in static lib ?


Compiling udp-many-way-audio-rx.c                        -> udp-many-way-audio-rx_alsa_arms (static)
/usr/lib/gcc/arm-linux-gnueabi/4.3.2/../../../../arm-linux-gnueabi/lib/libasound.a(pcm_dshare.o): In function `snd_pcm_dshare_sync_ptr':
pcm_dshare.c:(.text+0x6a4): warning: Warning: snd_pcm_hwsync() is deprecated, consider to use snd_pcm_avail()
/usr/lib/gcc/arm-linux-gnueabi/4.3.2/../../../../arm-linux-gnueabi/lib/libasound.a(pcm_direct.o): In function `snd1_pcm_direct_parse_open_conf':
pcm_direct.c:(.text+0x58c8): warning: Using 'getgrnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/arm-linux-gnueabi/4.3.2/../../../../arm-linux-gnueabi/lib/libasound.a(pcm_shm.o): In function `_snd_pcm_shm_open':
pcm_shm.c:(.text+0x27dc): warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/arm-linux-gnueabi/4.3.2/../../../../arm-linux-gnueabi/lib/libasound.a(dlmisc.o): In function `snd_dlopen':
dlmisc.c:(.text+0x38): undefined reference to `dlopen'
/usr/lib/gcc/arm-linux-gnueabi/4.3.2/../../../../arm-linux-gnueabi/lib/libasound.a(dlmisc.o): In function `snd_dlclose':
dlmisc.c:(.text+0x90): undefined reference to `dlclose'
/usr/lib/gcc/arm-linux-gnueabi/4.3.2/../../../../arm-linux-gnueabi/lib/libasound.a(dlmisc.o): In function `snd_dlsym_verify':
dlmisc.c:(.text+0x190): undefined reference to `dlsym'
/usr/lib/gcc/arm-linux-gnueabi/4.3.2/../../../../arm-linux-gnueabi/lib/libasound.a(dlmisc.o): In function `snd_dlsym':
dlmisc.c:(.text+0x2f8): undefined reference to `dlsym'
/usr/lib/gcc/arm-linux-gnueabi/4.3.2/../../../../arm-linux-gnueabi/lib/libasound.a(pcm_ladspa.o): In function `snd_pcm_ladspa_free_plugins':
pcm_ladspa.c:(.text+0x3cc): undefined reference to `dlclose'
/usr/lib/gcc/arm-linux-gnueabi/4.3.2/../../../../arm-linux-gnueabi/lib/libasound.a(pcm_ladspa.o): In function `snd_pcm_ladspa_check_file':
pcm_ladspa.c:(.text+0x41c4): undefined reference to `dlopen'
pcm_ladspa.c:(.text+0x41e4): undefined reference to `dlsym'
pcm_ladspa.c:(.text+0x4250): undefined reference to `dlclose'
pcm_ladspa.c:(.text+0x4344): undefined reference to `dlclose'
pcm_ladspa.c:(.text+0x43b0): undefined reference to `dlclose'
collect2: ld returned 1 exit status
~                                                 

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Clemens Ladisch]

Jonathan Andrews wrote:
> Getting closer, alsalib now builds without the need for pthreads but
> seems to insist on dlopen dlclose even in static lib ?

Add --without-libdl.


Regards,
Clemens

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Jonathan Andrews]

On Thu, 2012-03-08 at 15:35 +0100, Clemens Ladisch wrote:
> Jonathan Andrews wrote:
> > Getting closer, alsalib now builds without the need for pthreads but
> > seems to insist on dlopen dlclose even in static lib ?
> 
> Add --without-libdl.
> 

Ok, that fixed it thanks.

A couple of comments. 

1) A lot of my trouble was this :
configure.in:15: error: Autoconf version 2.62 or higher is required

If autoconf exists, is executed,  but does not complete should configure
not exit with an error ?

I checked the output of the configure/make carefully. Autoconf was
silently failing, gives no clues at all. As the datetime is in the
binary created each library had a unique file hash so I (reasonably)
assumed it to be doing the job :-(

2)  "Add --without-libdl", 
   ok ... should "--static" not turn this on then ?


Thanks,
Jon

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Takashi Iwai]

At Thu, 08 Mar 2012 15:14:50 +0000,
Jonathan Andrews wrote:
> 
> On Thu, 2012-03-08 at 15:35 +0100, Clemens Ladisch wrote:
> > Jonathan Andrews wrote:
> > > Getting closer, alsalib now builds without the need for pthreads but
> > > seems to insist on dlopen dlclose even in static lib ?
> > 
> > Add --without-libdl.
> > 
> 
> Ok, that fixed it thanks.
> 
> A couple of comments. 
> 
> 1) A lot of my trouble was this :
> configure.in:15: error: Autoconf version 2.62 or higher is required
> 
> If autoconf exists, is executed,  but does not complete should configure
> not exit with an error ?

Blame your autoconf :)

> I checked the output of the configure/make carefully. Autoconf was
> silently failing, gives no clues at all. As the datetime is in the
> binary created each library had a unique file hash so I (reasonably)
> assumed it to be doing the job :-(
> 
> 2)  "Add --without-libdl", 
>    ok ... should "--static" not turn this on then ?

dlopen() can be still used for taking other objects like LADSPA or
external io or pcm plugins.  There is no reason to prohibit dlopen()
in a static library.


Takashi

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Jonathan Andrews]

On Thu, 2012-03-08 at 16:34 +0100, Takashi Iwai wrote:
> At Thu, 08 Mar 2012 15:14:50 +0000,
> Jonathan Andrews wrote:
> > 
> > On Thu, 2012-03-08 at 15:35 +0100, Clemens Ladisch wrote:
> > > Jonathan Andrews wrote:
> > > > Getting closer, alsalib now builds without the need for pthreads but
> > > > seems to insist on dlopen dlclose even in static lib ?
> > > 
> > > Add --without-libdl.
> > > 
> > 
> > Ok, that fixed it thanks.
> > 
> > A couple of comments. 
> > 
> > 1) A lot of my trouble was this :
> > configure.in:15: error: Autoconf version 2.62 or higher is required
> > 
> > If autoconf exists, is executed,  but does not complete should configure
> > not exit with an error ?
> 
Blame your autoconf :)
Hmmmm .... true, but still not convinced this could not have been
prevented by reading the errorlevel and crapping out with a nice fluffy
error message for me !


> > I checked the output of the configure/make carefully. Autoconf was
> > silently failing, gives no clues at all. As the datetime is in the
> > binary created each library had a unique file hash so I (reasonably)
> > assumed it to be doing the job :-(
> > 
> > 2)  "Add --without-libdl", 
> >    ok ... should "--static" not turn this on then ?
> 
> dlopen() can be still used for taking other objects like LADSPA or
> external io or pcm plugins.  There is no reason to prohibit dlopen()
> in a static library.

Ok, makes sense.

Thanks,
Jon

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Jonathan Andrews]

On Thu, 2012-03-08 at 16:34 +0100, Takashi Iwai wrote:
> At Thu, 08 Mar 2012 15:14:50 +0000,
> Jonathan Andrews wrote:
> > 
> > On Thu, 2012-03-08 at 15:35 +0100, Clemens Ladisch wrote:
> > > Jonathan Andrews wrote:
> > > > Getting closer, alsalib now builds without the need for pthreads but
> > > > seems to insist on dlopen dlclose even in static lib ?
> > > 
> > > Add --without-libdl.
> > > 
Using the static libasound without pthreads now.

Process still stalls in same way:

ARM / # strace -p 408
Process 408 attached - interrupt to quit
futex(0x15a684, FUTEX_WAIT, 2, NULL
^C <unfinished ...>
Process 408 detached

Took 3 hours 1 min to stall. It was decoding 8 udp audio streams, a
second tx process reading from the same sound device was also running.


An strace of a stall (intel this time) is here.  Its 2GB and on a slow
link (sorry).
http://www.jonshouse.co.uk/download/a_stop.txt

futex(0xae5320, FUTEX_WAIT_PRIVATE, 2, NULL) = ? ERESTARTSYS (To be
restarted)
--- SIGINT (Interrupt) @ 0 (0) ---

The futex address 0xae5320 occurs only once at the end of the text file
so I have no idea what sets it up.

Thanks,
Jon

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Clemens Ladisch]

Jonathan Andrews wrote:
> Using the static libasound without pthreads now.
>
> Process still stalls in same way:
>
> ARM / # strace -p 408
> Process 408 attached - interrupt to quit
> futex(0x15a684, FUTEX_WAIT, 2, NULL

So it's not a call from alsa-lib.

> An strace of a stall (intel this time) is here.  Its 2GB and on a slow
> link (sorry).
> http://www.jonshouse.co.uk/download/a_stop.txt

If only there were a tool to compress files ...

> The futex address 0xae5320 occurs only once at the end of the text file
> so I have no idea what sets it up.

Can't you attach gdb to your process?
Or at least log where in your program you are?
(If strace has too much overhead, write to a variable, and run a thread
that waits for keyboard input and prints the variable.)


Regards,
Clemens

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Jonathan Andrews]

On Thu, 2012-03-08 at 21:40 +0100, Clemens Ladisch wrote:
> Jonathan Andrews wrote:
> > Using the static libasound without pthreads now.
> >
> > Process still stalls in same way:
> >
> > ARM / # strace -p 408
> > Process 408 attached - interrupt to quit
> > futex(0x15a684, FUTEX_WAIT, 2, NULL
> 
> So it's not a call from alsa-lib.
The alsa code is a conditional compile, as is that for the /dev/dsp
device.  Only the alsa version fails, the version that just takes data
from the network and mixes it does not fail.

cc udp-many-way-audio-rx.c -DARM -DALSA -lasound -lrt -lm -o udp-many-way-audio-rx_alsa_arms --static
Fails randomly

cc udp-many-way-audio-rx.c -DARM -DDUMMY -DSTATIC -o udp-many-way-audio-rx_dummy_arms --static
Dummy version, never fails.  Still has network code and my audio mixer.

Its not as if I'm linking in a lot of libs...

> > An strace of a stall (intel this time) is here.  Its 2GB and on a slow
> > link (sorry).
> > http://www.jonshouse.co.uk/download/a_stop.txt
> 
> If only there were a tool to compress files ...
Ok, i'm pretty tired I admit it .... 30MB
http://www.jonshouse.co.uk/download/a_stop.txt.gz


> 
> > The futex address 0xae5320 occurs only once at the end of the text file
> > so I have no idea what sets it up.
> 
> Can't you attach gdb to your process?
I need to build an arm version, will work on that next.

> Or at least log where in your program you are?
I did that, best I can tell it enters one of these two functions but
never exists. Makes sense as these are the two conditional compiles
wrapped around #ifdef ALSA in the main loop.
http://www.jonshouse.co.uk/download/udp-many-way-audio-playsample-alsa.c


> (If strace has too much overhead, write to a variable, and run a thread
> that waits for keyboard input and prints the variable.)
The strace (now gzipped) is the same process as the arm running (and
then stalling) on the PC. It still stalls, just takes (on average)
longer to stall on the faster machine ......

Thanks,
Jon

Re: Random process lockup on ARM board: alsa-lib-1.0.25, FUTEX_WAIT_PRIVATE

[Jonathan Andrews]

Just to finish this thread.

Maybe its not an alsa bug, but only happens to my code when using alsa.

Report filed here.
http://sourceware.org/bugzilla/show_bug.cgi?id=13845

Thanks,
Jon