All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] telnetd:Fix deadlock on cleanup
@ 2018-05-28  6:27 Seiichi Ishitsuka
  2018-05-28  8:39 ` Alexander Kanavin
  2018-05-29 17:38 ` [OE-core] " Khem Raj
  0 siblings, 2 replies; 7+ messages in thread
From: Seiichi Ishitsuka @ 2018-05-28  6:27 UTC (permalink / raw)
  To: Openembedded-core

[-- Attachment #1: Type: text/plain, Size: 700 bytes --]

Hi all,

I was able to reproduce the old problems shown below with netkit-telnet_0.17.bb.

https://patchwork.openembedded.org/patch/96727/
https://bugs.launchpad.net/ubuntu/+source/netkit-telnet/+bug/507455

The evaluated environment is,
 glibc  :2.2 
 kernel :4.4.32

How to reproduce:

Run the script more than once in the background, and
please adjust CPU usage to 100%.

#!/bin/sh

HOST="host.target"
USER="root"
PASS="password"

while :
  do
  expect -c "
  set timeout 10
  spawn telnet $HOST
  expect login:\  ; send \"$USER\r\"
  expect sword:\  ; send \"$PASS\r\"
  expect \"# \" ; send \"exit\r\"
  " > /dev/null
  done


Best regards,
Seiichi Ishitsuka

[-- Attachment #2: 0001-telnetd-Fix-deadlock-on-cleanup.patch --]
[-- Type: application/octet-stream, Size: 2957 bytes --]

From cdf8829f743adb3ec72c231ee5843dabed91b30f Mon Sep 17 00:00:00 2001
From: Seiichi Ishitsuka <ishitsuka.sc@ncos.nec.co.jp>
Date: Tue, 8 May 2018 09:53:34 +0900
Subject: [PATCH] telnetd:Fix deadlock on cleanup

the patch comes from:
https://launchpadlibrarian.net/37882973/0001-telnetd-Fix-deadlock-on-cleanup.patch

The cleanup function in telnetd is called both directly and on SIGCHLD
signals. This, unfortunately, triggered a deadlock in glibc 2.2 while
running on a 4.4.32 kernel.

What we were seeing is hangs like these:

(gdb) bt
 #0  0x489fee6c in __lll_lock_wait_private (futex=0x48a5e9a4 <__libc_utmp_lock>) at ./lowlevellock.c:33
 #1  0x48a27934 in __utmpname (file=0x48ac1278 "/var/run/utmp") at utmpname.c:42
 #2  0x48ac0ec0 in logout (line=0x27e9d "pts/11") at logout.c:32

How to reproduce:

Run the script more than once in the background.

 #!/bin/sh

 HOST="host.target"
 USER="root"
 PASS="password"

 while :
 do
   expect -c "
   set timeout 10
   spawn telnet $HOST
   expect login:\  ; send \"$USER\r\"
   expect sword:\  ; send \"$PASS\r\"
   expect \"# \" ; send \"exit\r\"
 " > /dev/null
 done

Signed-off-by: Seiichi Ishitsuka <ishitsuka.sc@ncos.nec.co.jp>
---
 telnetd/ext.h      |  1 +
 telnetd/sys_term.c | 17 ++++++++++++++++-
 telnetd/telnetd.c  |  2 +-
 3 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/telnetd/ext.h b/telnetd/ext.h
index b98d6ec..08f9d07 100644
--- a/telnetd/ext.h
+++ b/telnetd/ext.h
@@ -97,6 +97,7 @@ void add_slc(int, int, int);
 void check_slc(void);
 void change_slc(int, int, int);
 void cleanup(int);
+void cleanup_sighandler(int);
 void clientstat(int, int, int);
 void copy_termbuf(char *, int);
 void deferslc(void);
diff --git a/telnetd/sys_term.c b/telnetd/sys_term.c
index 5b4aa84..c4fb0f7 100644
--- a/telnetd/sys_term.c
+++ b/telnetd/sys_term.c
@@ -719,7 +719,7 @@ static void addarg(struct argv_stuff *avs, const char *val) {
  * This is the routine to call when we are all through, to
  * clean up anything that needs to be cleaned up.
  */
-void cleanup(int sig) {
+void cleanup_sighandler(int sig) {
     char *p;
     (void)sig;
 
@@ -742,3 +742,18 @@ void cleanup(int sig) {
     shutdown(net, 2);
     exit(0);
 }
+
+void cleanup(int sig) {
+    sigset_t mask, oldmask;
+
+    /* Set up the mask of signals to temporarily block. */
+    sigemptyset (&mask);
+    sigaddset (&mask, SIGCHLD);
+
+    /* Block SIGCHLD while running cleanup */
+    sigprocmask (SIG_BLOCK, &mask, &oldmask);
+
+    cleanup_sighandler(sig);
+    /* Technically not needed since cleanup_sighandler exits */
+    sigprocmask (SIG_UNBLOCK, &mask, NULL);
+}
diff --git a/telnetd/telnetd.c b/telnetd/telnetd.c
index 9ace838..788919c 100644
--- a/telnetd/telnetd.c
+++ b/telnetd/telnetd.c
@@ -833,7 +833,7 @@ void telnet(int f, int p)
     signal(SIGTTOU, SIG_IGN);
 #endif
     
-    signal(SIGCHLD, cleanup);
+    signal(SIGCHLD, cleanup_sighandler);
     
 #ifdef TIOCNOTTY
     {
-- 
2.6.4


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [PATCH] telnetd:Fix deadlock on cleanup
  2018-05-28  6:27 [PATCH] telnetd:Fix deadlock on cleanup Seiichi Ishitsuka
@ 2018-05-28  8:39 ` Alexander Kanavin
  2018-05-29  5:44   ` Seiichi Ishitsuka
  2018-05-29 17:38 ` [OE-core] " Khem Raj
  1 sibling, 1 reply; 7+ messages in thread
From: Alexander Kanavin @ 2018-05-28  8:39 UTC (permalink / raw)
  To: Seiichi Ishitsuka, Openembedded-core

On 05/28/2018 09:27 AM, Seiichi Ishitsuka wrote:
> I was able to reproduce the old problems shown below with netkit-telnet_0.17.bb.
> 
> https://patchwork.openembedded.org/patch/96727/
> https://bugs.launchpad.net/ubuntu/+source/netkit-telnet/+bug/507455

The patch should go to meta-openembedded list, and it should be created  
against the layer source tree, not against the component source tree.

Alex


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH] telnetd:Fix deadlock on cleanup
  2018-05-28  8:39 ` Alexander Kanavin
@ 2018-05-29  5:44   ` Seiichi Ishitsuka
  2018-05-29 13:21     ` Alexander Kanavin
  0 siblings, 1 reply; 7+ messages in thread
From: Seiichi Ishitsuka @ 2018-05-29  5:44 UTC (permalink / raw)
  To: Alexander Kanavin, Openembedded-core

[-- Attachment #1: Type: text/plain, Size: 345 bytes --]

Dear Alex,

> The patch should go to meta-openembedded list, and it should be created against the layer source tree, not against the component source
> tree.

Thank you for all your kindness.
I have created a patch for meta-networking/recipes-netkit/netkit-telnet.
Please refer to the attached patch.

Best regards,
Seiichi Ishitsuka

[-- Attachment #2: 0001-netkit-telnet-Fix-deadlock-on-cleanup.patch --]
[-- Type: application/octet-stream, Size: 4752 bytes --]

From 59e038c9d15770323be23f4f12087bdf43db7a7b Mon Sep 17 00:00:00 2001
From: Seiichi Ishitsuka <ishitsuka.sc@ncos.nec.co.jp>
Date: Tue, 29 May 2018 11:43:25 +0900
Subject: [PATCH] netkit-telnet: Fix deadlock on cleanup

Signed-off-by: Seiichi Ishitsuka <ishitsuka.sc@ncos.nec.co.jp>
---
 .../0001-telnetd-Fix-deadlock-on-cleanup.patch     | 107 +++++++++++++++++++++
 .../netkit-telnet/netkit-telnet_0.17.bb            |   1 +
 2 files changed, 108 insertions(+)
 create mode 100644 meta-networking/recipes-netkit/netkit-telnet/files/0001-telnetd-Fix-deadlock-on-cleanup.patch

diff --git a/meta-networking/recipes-netkit/netkit-telnet/files/0001-telnetd-Fix-deadlock-on-cleanup.patch b/meta-networking/recipes-netkit/netkit-telnet/files/0001-telnetd-Fix-deadlock-on-cleanup.patch
new file mode 100644
index 0000000..d6873a5
--- /dev/null
+++ b/meta-networking/recipes-netkit/netkit-telnet/files/0001-telnetd-Fix-deadlock-on-cleanup.patch
@@ -0,0 +1,107 @@
+From cdf8829f743adb3ec72c231ee5843dabed91b30f Mon Sep 17 00:00:00 2001
+From: Seiichi Ishitsuka <ishitsuka.sc@ncos.nec.co.jp>
+Date: Tue, 8 May 2018 09:53:34 +0900
+Subject: [PATCH] telnetd:Fix deadlock on cleanup
+
+the patch comes from:
+https://launchpadlibrarian.net/37882973/0001-telnetd-Fix-deadlock-on-cleanup.patch
+
+The cleanup function in telnetd is called both directly and on SIGCHLD
+signals. This, unfortunately, triggered a deadlock in glibc 2.2 while
+running on a 4.4.32 kernel.
+
+What we were seeing is hangs like these:
+
+(gdb) bt
+ #0  0x489fee6c in __lll_lock_wait_private (futex=0x48a5e9a4 <__libc_utmp_lock>) at ./lowlevellock.c:33
+ #1  0x48a27934 in __utmpname (file=0x48ac1278 "/var/run/utmp") at utmpname.c:42
+ #2  0x48ac0ec0 in logout (line=0x27e9d "pts/11") at logout.c:32
+
+How to reproduce:
+
+Run the script more than once in the background.
+
+ #!/bin/sh
+
+ HOST="host.target"
+ USER="root"
+ PASS="password"
+
+ while :
+ do
+   expect -c "
+   set timeout 10
+   spawn telnet $HOST
+   expect login:\  ; send \"$USER\r\"
+   expect sword:\  ; send \"$PASS\r\"
+   expect \"# \" ; send \"exit\r\"
+ " > /dev/null
+ done
+
+Signed-off-by: Seiichi Ishitsuka <ishitsuka.sc@ncos.nec.co.jp>
+---
+ telnetd/ext.h      |  1 +
+ telnetd/sys_term.c | 17 ++++++++++++++++-
+ telnetd/telnetd.c  |  2 +-
+ 3 files changed, 18 insertions(+), 2 deletions(-)
+
+diff --git a/telnetd/ext.h b/telnetd/ext.h
+index b98d6ec..08f9d07 100644
+--- a/telnetd/ext.h
++++ b/telnetd/ext.h
+@@ -97,6 +97,7 @@ void add_slc(int, int, int);
+ void check_slc(void);
+ void change_slc(int, int, int);
+ void cleanup(int);
++void cleanup_sighandler(int);
+ void clientstat(int, int, int);
+ void copy_termbuf(char *, int);
+ void deferslc(void);
+diff --git a/telnetd/sys_term.c b/telnetd/sys_term.c
+index 5b4aa84..c4fb0f7 100644
+--- a/telnetd/sys_term.c
++++ b/telnetd/sys_term.c
+@@ -719,7 +719,7 @@ static void addarg(struct argv_stuff *avs, const char *val) {
+  * This is the routine to call when we are all through, to
+  * clean up anything that needs to be cleaned up.
+  */
+-void cleanup(int sig) {
++void cleanup_sighandler(int sig) {
+     char *p;
+     (void)sig;
+ 
+@@ -742,3 +742,18 @@ void cleanup(int sig) {
+     shutdown(net, 2);
+     exit(0);
+ }
++
++void cleanup(int sig) {
++    sigset_t mask, oldmask;
++
++    /* Set up the mask of signals to temporarily block. */
++    sigemptyset (&mask);
++    sigaddset (&mask, SIGCHLD);
++
++    /* Block SIGCHLD while running cleanup */
++    sigprocmask (SIG_BLOCK, &mask, &oldmask);
++
++    cleanup_sighandler(sig);
++    /* Technically not needed since cleanup_sighandler exits */
++    sigprocmask (SIG_UNBLOCK, &mask, NULL);
++}
+diff --git a/telnetd/telnetd.c b/telnetd/telnetd.c
+index 9ace838..788919c 100644
+--- a/telnetd/telnetd.c
++++ b/telnetd/telnetd.c
+@@ -833,7 +833,7 @@ void telnet(int f, int p)
+     signal(SIGTTOU, SIG_IGN);
+ #endif
+     
+-    signal(SIGCHLD, cleanup);
++    signal(SIGCHLD, cleanup_sighandler);
+     
+ #ifdef TIOCNOTTY
+     {
+-- 
+2.6.4
+
diff --git a/meta-networking/recipes-netkit/netkit-telnet/netkit-telnet_0.17.bb b/meta-networking/recipes-netkit/netkit-telnet/netkit-telnet_0.17.bb
index 807b566..e1566f8 100644
--- a/meta-networking/recipes-netkit/netkit-telnet/netkit-telnet_0.17.bb
+++ b/meta-networking/recipes-netkit/netkit-telnet/netkit-telnet_0.17.bb
@@ -11,6 +11,7 @@ SRC_URI = "ftp://ftp.uk.linux.org/pub/linux/Networking/netkit/${BP}.tar.gz \
            file://telnet-xinetd \
            file://cross-compile.patch \
            file://0001-telnet-telnetd-Fix-print-format-strings.patch \
+           file://0001-telnetd-Fix-deadlock-on-cleanup.patch \
            "
 
 EXTRA_OEMAKE = "INSTALLROOT=${D} SBINDIR=${sbindir} DAEMONMODE=755 \
-- 
1.8.3.1


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [PATCH] telnetd:Fix deadlock on cleanup
  2018-05-29  5:44   ` Seiichi Ishitsuka
@ 2018-05-29 13:21     ` Alexander Kanavin
  0 siblings, 0 replies; 7+ messages in thread
From: Alexander Kanavin @ 2018-05-29 13:21 UTC (permalink / raw)
  To: Seiichi Ishitsuka; +Cc: Openembedded-core

2018-05-29 8:44 GMT+03:00 Seiichi Ishitsuka <ishitsuka.sc@ncos.nec.co.jp>:
> Dear Alex,
>
>> The patch should go to meta-openembedded list, and it should be created against the layer source tree, not against the component source
>> tree.
>
> Thank you for all your kindness.
> I have created a patch for meta-networking/recipes-netkit/netkit-telnet.
> Please refer to the attached patch.

I must clarify: the patch should be sent to the openembedded-devel
mailing list, not to openembedded-core list. Only patches for the
oe-core layer should be sent here; patches for the meta-oe layers go
to oe-devel list.

Also, please use 'git send-email' for sending patches.

Thank you,
Alex


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] [PATCH] telnetd:Fix deadlock on cleanup
  2018-05-28  6:27 [PATCH] telnetd:Fix deadlock on cleanup Seiichi Ishitsuka
  2018-05-28  8:39 ` Alexander Kanavin
@ 2018-05-29 17:38 ` Khem Raj
  1 sibling, 0 replies; 7+ messages in thread
From: Khem Raj @ 2018-05-29 17:38 UTC (permalink / raw)
  To: Seiichi Ishitsuka; +Cc: openembeded-devel

this belongs to oe-devel list

Please check will latest master, if you can reporduce the issue with master
then we can revive this discussion on the patch.

On Sun, May 27, 2018 at 11:27 PM, Seiichi Ishitsuka
<ishitsuka.sc@ncos.nec.co.jp> wrote:
> Hi all,
>
> I was able to reproduce the old problems shown below with netkit-telnet_0.17.bb.
>
> https://patchwork.openembedded.org/patch/96727/
> https://bugs.launchpad.net/ubuntu/+source/netkit-telnet/+bug/507455
>
> The evaluated environment is,
>  glibc  :2.2
>  kernel :4.4.32
>
> How to reproduce:
>
> Run the script more than once in the background, and
> please adjust CPU usage to 100%.
>
> #!/bin/sh
>
> HOST="host.target"
> USER="root"
> PASS="password"
>
> while :
>   do
>   expect -c "
>   set timeout 10
>   spawn telnet $HOST
>   expect login:\  ; send \"$USER\r\"
>   expect sword:\  ; send \"$PASS\r\"
>   expect \"# \" ; send \"exit\r\"
>   " > /dev/null
>   done
>
>
> Best regards,
> Seiichi Ishitsuka
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] [PATCH] telnetd:Fix deadlock on cleanup
  2018-05-31  5:49 Seiichi Ishitsuka
@ 2018-05-31 16:22 ` Khem Raj
  0 siblings, 0 replies; 7+ messages in thread
From: Khem Raj @ 2018-05-31 16:22 UTC (permalink / raw)
  To: Seiichi Ishitsuka; +Cc: openembeded-devel

On Wed, May 30, 2018 at 10:49 PM, Seiichi Ishitsuka
<ishitsuka.sc@ncos.nec.co.jp> wrote:
>> Please check will latest master, if you can reporduce the issue with master then we can revive this discussion on the patch.
>
> I was able to reproduce this issue on qemux86 with the latest version.
>
> Build Configuration:
> BB_VERSION           = "1.37.0"
> BUILD_SYS            = "x86_64-linux"
> NATIVELSBSTRING      = "universal"
> TARGET_SYS           = "i586-poky-linux"
> MACHINE              = "qemux86"
> DISTRO               = "poky"
> DISTRO_VERSION       = "2.5"
> TUNE_FEATURES        = "m32 i586"
> TARGET_FPU           = ""
> meta
> meta-poky
> meta-yocto-bsp       = "master:719d068bde55ef29a3468bc0779d4cb0c11e8c1d"
> meta-oe
> meta-python
> meta-networking      = "master:bb57bac845f3cd1634862fa9868bc8e294ba74a9"
>

OK Please revive/refresh the patch and send out to ml for discussion

> How to reproduce:
>
> 1. telnet from host(192.168.7.1) to qemux86(192.168.7.2)
>
> $ cat ./telnetd.sh
>  #!/bin/sh
>
>  HOST="192.168.7.2"
>  USER="root"
>
>  while :
>  do
>    expect -c "
>    set timeout 10
>    spawn telnet $HOST
>    expect login:\  ; send \"$USER\r\"
>    expect \"# \" ; send \"exit\r\"
>  "> /dev/null
>  done
>
> $ ./telnetd.sh &
> $ ./telnetd.sh &
>
> 2. Check process status on qemux86
>
> root       325     1  0 04:41 ?        00:00:01 /usr/sbin/xinetd -pidfile /var/r
> root      3328   325  0 04:43 ?        00:00:00 in.telnetd: 192.168.7.1
> root      3330  3328  0 04:43 ?        00:00:00 [sh] <defunct>
> root      3374   325  0 04:43 ?        00:00:00 in.telnetd: 192.168.7.1
> root      3376  3374  0 04:43 ?        00:00:00 [sh] <defunct>
> root      3417   306  0 04:44 ttyS0    00:00:00 ps -ef
>
> -> Found defunct sh and in.telnetd(waiting futex).
>
> root@qemux86:~# strace -p 3328
> strace: Process 3328 attached
> futex(0x45055a18, FUTEX_WAIT_PRIVATE, 2, NULL^Cstrace: Process 3328 detached
>  <detached ...>
>
> root@qemux86:~# strace -p 3374
> strace: Process 3374 attached
> futex(0x45055a18, FUTEX_WAIT_PRIVATE, 2, NULL^Cstrace: Process 3374 detached
>  <detached ...>
>
> Best regards,
> Seiichi Ishitsuka
> --
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-devel


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] [PATCH] telnetd:Fix deadlock on cleanup
@ 2018-05-31  5:49 Seiichi Ishitsuka
  2018-05-31 16:22 ` Khem Raj
  0 siblings, 1 reply; 7+ messages in thread
From: Seiichi Ishitsuka @ 2018-05-31  5:49 UTC (permalink / raw)
  To: openembeded-devel

> Please check will latest master, if you can reporduce the issue with master then we can revive this discussion on the patch.

I was able to reproduce this issue on qemux86 with the latest version.

Build Configuration:
BB_VERSION           = "1.37.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "i586-poky-linux"
MACHINE              = "qemux86"
DISTRO               = "poky"
DISTRO_VERSION       = "2.5"
TUNE_FEATURES        = "m32 i586"
TARGET_FPU           = ""
meta
meta-poky
meta-yocto-bsp       = "master:719d068bde55ef29a3468bc0779d4cb0c11e8c1d"
meta-oe
meta-python
meta-networking      = "master:bb57bac845f3cd1634862fa9868bc8e294ba74a9"

How to reproduce:

1. telnet from host(192.168.7.1) to qemux86(192.168.7.2)

$ cat ./telnetd.sh
 #!/bin/sh

 HOST="192.168.7.2"
 USER="root"

 while :
 do
   expect -c "
   set timeout 10
   spawn telnet $HOST
   expect login:\  ; send \"$USER\r\"
   expect \"# \" ; send \"exit\r\"
 "> /dev/null
 done

$ ./telnetd.sh &
$ ./telnetd.sh &

2. Check process status on qemux86

root       325     1  0 04:41 ?        00:00:01 /usr/sbin/xinetd -pidfile /var/r
root      3328   325  0 04:43 ?        00:00:00 in.telnetd: 192.168.7.1
root      3330  3328  0 04:43 ?        00:00:00 [sh] <defunct>
root      3374   325  0 04:43 ?        00:00:00 in.telnetd: 192.168.7.1
root      3376  3374  0 04:43 ?        00:00:00 [sh] <defunct>
root      3417   306  0 04:44 ttyS0    00:00:00 ps -ef

-> Found defunct sh and in.telnetd(waiting futex). 

root@qemux86:~# strace -p 3328
strace: Process 3328 attached
futex(0x45055a18, FUTEX_WAIT_PRIVATE, 2, NULL^Cstrace: Process 3328 detached
 <detached ...>

root@qemux86:~# strace -p 3374
strace: Process 3374 attached
futex(0x45055a18, FUTEX_WAIT_PRIVATE, 2, NULL^Cstrace: Process 3374 detached
 <detached ...>

Best regards,
Seiichi Ishitsuka

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2018-05-31 16:23 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-05-28  6:27 [PATCH] telnetd:Fix deadlock on cleanup Seiichi Ishitsuka
2018-05-28  8:39 ` Alexander Kanavin
2018-05-29  5:44   ` Seiichi Ishitsuka
2018-05-29 13:21     ` Alexander Kanavin
2018-05-29 17:38 ` [OE-core] " Khem Raj
2018-05-31  5:49 Seiichi Ishitsuka
2018-05-31 16:22 ` Khem Raj

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.