[cip-dev] Watchdog triggered do_futex() on 4.4.277-cip60-rt35 #cip

[cip-dev] Watchdog triggered do_futex() on 4.4.277-cip60-rt35 #cip

[Sebastian Holzgreve]

[-- Attachment #1.1: Type: text/plain, Size: 1697 bytes --]

Hi,

we were using the 4.4.277-cip60-rt35 kernel and it happens from time to time that the kernel watchdog is triggered and our embedded devices gets resetted.

This message specially appers when we run the CodeSYS Runtime (PLC runtime)

> 
> [  536.133081] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s!
> [BlkDrvUdp:330]
> [  536.140637] Modules linked in:
> [  536.143756] CPU: 0 PID: 330 Comm: BlkDrvUdp Not tainted
> 4.4.277-cip60-rt35-ohp+gd23c00f1c39e #1
> [  536.152489] Hardware name: Freescale Vybrid VF5xx/VF6xx (Device Tree)
> [  536.158967] task: 86bd7700 ti: 841aa000 task.ti: 841aa000
> [  536.164408] PC is at do_futex+0x458/0xa98
> [  536.168450] LR is at do_futex+0x428/0xa98
> [  536.172487] pc : [<80069a40>]    lr : [<80069a10>]    psr: a0070013
> [  536.172487] sp : 841abe90  ip : 86896244  fp : 841abf4c
> 

I contacted the support and they told me following about usage of mutexes of the runtime:

The runtime is using pi mutexes
> 
> pthread_mutexattr_setprotocol(&attr, PTHREAD_PRIO_INHERIT);

and also with recursive attribute.

> 
> pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE_NP);

and that they got other customers that had also problems with this constellation.

My current knowledge of the linux kernel is not big enough to solve that problem by myself. But i'm willed to learn more about the kernel and maybe solve the problem by own.

Can anyone give me a hint how to continue investigation of that problem?
Maybe someone has solved that problem for another target hardware?

Every RTFM hint is also welcome, when it least tell me the manual to dig trough :)

Kind regards,
Sebastian

[-- Attachment #1.2: Type: text/html, Size: 2281 bytes --]

[-- Attachment #2: Type: text/plain, Size: 496 bytes --]


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#6736): https://lists.cip-project.org/g/cip-dev/message/6736
Mute This Topic: https://lists.cip-project.org/mt/85625711/4520388
Mute #cip:https://lists.cip-project.org/g/cip-dev/mutehashtag/cip
Group Owner: cip-dev+owner@lists.cip-project.org
Unsubscribe: https://lists.cip-project.org/g/cip-dev/leave/10495289/4520388/727948398/xyzzy [cip-dev@archiver.kernel.org]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [cip-dev] Watchdog triggered do_futex() on 4.4.277-cip60-rt35 #cip

[Sebastian Holzgreve]

[-- Attachment #1.1.1: Type: text/plain, Size: 1949 bytes --]

> Hi,
>
> we were using the 4.4.277-cip60-rt35 kernel and it happens from time 
> to time that the kernel watchdog is triggered and our embedded devices 
> gets resetted.
>
> This message specially appers when we run the CodeSYS Runtime (PLC 
> runtime)
>
>     [  536.133081] NMI watchdog: BUG: soft lockup - CPU#0 stuck for
>     23s! [BlkDrvUdp:330]
>     [  536.140637] Modules linked in:
>     [  536.143756] CPU: 0 PID: 330 Comm: BlkDrvUdp Not tainted
>     4.4.277-cip60-rt35-ohp+gd23c00f1c39e #1
>     [  536.152489] Hardware name: Freescale Vybrid VF5xx/VF6xx (Device
>     Tree)
>     [  536.158967] task: 86bd7700 ti: 841aa000 task.ti: 841aa000
>     [  536.164408] PC is at do_futex+0x458/0xa98
>     [  536.168450] LR is at do_futex+0x428/0xa98
>     [  536.172487] pc : [<80069a40>]    lr : [<80069a10>]    psr: a0070013
>     [  536.172487] sp : 841abe90 ip : 86896244  fp : 841abf4c
>
>
> I contacted the support and they told me following about usage of 
> mutexes of the runtime:
>
> The runtime is using pi mutexes
>
>     pthread_mutexattr_setprotocol(&attr, PTHREAD_PRIO_INHERIT);
>
> and also with recursive attribute.
>
>     pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE_NP);
>
> and that they got other customers that had also problems with this 
> constellation.
>
>
> My current knowledge of the linux kernel is not big enough to solve 
> that problem by myself. But i'm willed to learn more about the kernel 
> and maybe solve the problem by own.
>
>
> Can anyone give me a hint how to continue investigation of that problem?
> Maybe someone has solved that problem for another target hardware?
>
> Every RTFM hint is also welcome, when it least tell me the manual to 
> dig trough :)
>
> Kind regards,
> Sebastian


I tried the latest 4.19.206-cip57-rt22-kernel. In this kernel version 
the problem reported above doesn't occur.

For now we will use the 4.19 kernel.


Kind regards,

Sebastian

[-- Attachment #1.1.2: Type: text/html, Size: 3173 bytes --]

[-- Attachment #1.2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4071 bytes --]

[-- Attachment #2: Type: text/plain, Size: 496 bytes --]


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#6737): https://lists.cip-project.org/g/cip-dev/message/6737
Mute This Topic: https://lists.cip-project.org/mt/85625711/4520388
Mute #cip:https://lists.cip-project.org/g/cip-dev/mutehashtag/cip
Group Owner: cip-dev+owner@lists.cip-project.org
Unsubscribe: https://lists.cip-project.org/g/cip-dev/leave/10495289/4520388/727948398/xyzzy [cip-dev@archiver.kernel.org]
-=-=-=-=-=-=-=-=-=-=-=-