[PATCH] Fix freezing bug in curses console

[Matthew Bloch <matthew@bytemark.co.uk>]

Hi there,

We are running lots of kvm processes in screen and found that about 1 in
5 froze shortly after startup startup with a backtrace like this one:

#0  0xf7c7fcd9 in pthread_exit () from /lib/tls/libc.so.6
#1  0xf7cfbe62 in wresize () from /lib/libncurses.so.5
#2  0xf7cfb7ab in is_term_resized () from /lib/libncurses.so.5
#3  0xf7cfb877 in is_term_resized () from /lib/libncurses.so.5
#4  0xf7cfba31 in resize_term () from /lib/libncurses.so.5
#5  0x080d3dd9 in vga_init ()
#6  <signal handler called>
#7  0xf7c0da5b in free () from /lib/tls/libc.so.6
#8  0xf7c0effe in calloc () from /lib/tls/libc.so.6
#9  0xf7cf222e in newpad () from /lib/libncurses.so.5
#10 0x080d3549 in vga_init ()

We're just using the lenny version of kvm from 2008-12-16.

On casual inspection, the SIGWINCH signal handling looked ropey to me -
grandpa always told me not to do any real work in a signal handler, and
the backtrace suggested re-entrancy problems in curses, so I changed the
behaviour to set a flag and do the work in the main loop instead.  Maybe
I'm reading the backtrace wrong.

So far that means that when you resize the window, the display is
corrupt until the VM outputs some text, or the user hits a key.  But I
think it has solved the freezing / crashing bug too - would appreciate
any comments on my analysis or proposed solution.

Index: curses.c
===================================================================
--- curses.c    (revision 6374)
+++ curses.c    (working copy)
@@ -41,6 +41,7 @@
 #define FONT_HEIGHT 16
 #define FONT_WIDTH 8

+static int winch_flag = 0;
 static console_ch_t screen[160 * 100];
 static WINDOW *screenpad = NULL;
 static int width, height, gwidth, gheight, invalidate;
@@ -110,7 +111,7 @@

 #ifndef _WIN32
 #if defined(SIGWINCH) && defined(KEY_RESIZE)
-static void curses_winch_handler(int signum)
+static void curses_winch_handler_real(void)
 {
     struct winsize {
         unsigned short ws_row;
@@ -126,7 +127,13 @@
     resize_term(ws.ws_row, ws.ws_col);
     curses_calc_pad();
     invalidate = 1;
+    winch_flag = 0;
+}

+static void curses_winch_handler(int sig)
+{
+    winch_flag = 1;
+
     /* some systems require this */
     signal(SIGWINCH, curses_winch_handler);
 }
@@ -179,6 +186,12 @@
s
     nextchr = ERR;
     while (1) {
+
+#if !defined(_WIN32) && defined(SIGWINCH) && defined(KEY_RESIZE)
+        if (winch_flag)
+            curses_winch_handler_real();
+#endif
+
         /* while there are any pending key strokes to process */
         if (nextchr == ERR)
             chr = getch();

-- 
Matthew