From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753027Ab1HAMwt (ORCPT ); Mon, 1 Aug 2011 08:52:49 -0400 Received: from mx1.redhat.com ([209.132.183.28]:27844 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752958Ab1HAMwo (ORCPT ); Mon, 1 Aug 2011 08:52:44 -0400 Date: Mon, 1 Aug 2011 08:52:34 -0400 From: Don Zickus To: ZAK Magnus Cc: linux-kernel@vger.kernel.org, Ingo Molnar , Mandeep Singh Baines Subject: Re: [PATCH v3 2/2] Make hard lockup detection use timestamps Message-ID: <20110801125234.GE14343@redhat.com> References: <1311271873-10879-1-git-send-email-zakmagnus@google.com> <20110722195340.GF3765@redhat.com> <20110725124451.GA2866@redhat.com> <20110729205538.GD14343@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Jul 29, 2011 at 04:12:32PM -0700, ZAK Magnus wrote: > Are you saying that any call to printk() will touch the watchdogs? I > wasn't aware of that and it doesn't seem to comply with my > observations too well, either. Then again, at the moment I don't > understand some of the things I'm currently seeing so I could just be > wrong. I believe the serial console write is the source of all the touch_nmi_watchdogs that result from a printk. drivers/tty/serial/8250.c::serial8250_console_write() Cheers, Don > > On Fri, Jul 29, 2011 at 1:55 PM, Don Zickus wrote: > > On Thu, Jul 28, 2011 at 05:16:00PM -0700, ZAK Magnus wrote: > >> No news? > >> > >> I've been testing and looking into issues and I realized dump_stack() > >> calls touch_nmi_watchdog(). That wrecks what the patch is trying to do > >> so I'm changing it to save the trace and print it later after the > >> stall has completed. This would also resolve some other things you > >> were saying weren't so good. Hopefully the logic is similar enough > >> that some things you may have learned still apply. > > > > So yeah, the acting of printing was resesting the softlockup counter and > > delaying it forever.  In parallel, rcu has its own stall detector that was > > going off after a minute or two. > > > > Once I routed the printk to trace_printk and disabled dump_stack, > > everything started working as expected. > > > > Now the question is how to avoid shooting ourselves in the foot by > > printk'ing a message without resetting the hard/soft lock watchdogs. > > > > I'll have to think about how to do that.  If you can come up with any > > ideas let me know. > > > > We almost need a quiet dump_stack that dumps to a buffer instead of the > > console.  But I am not sure that is worth the effort. > > > > Hmm. > > > > Cheers, > > Don > >