linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH RFC] Debug handling of early spurious interrupts
@ 2007-07-17 10:09 Fernando Luis Vázquez Cao
  2007-07-18 22:46 ` Andrew Morton
  0 siblings, 1 reply; 13+ messages in thread
From: Fernando Luis Vázquez Cao @ 2007-07-17 10:09 UTC (permalink / raw)
  To: Andrew Morton; +Cc: linux-kernel, kexec

With the advent of kdump it is possible that device drivers receive
interrupts generated in the context of a previous kernel. Ideally
quiescing the underlying devices should suffice but not all drivers
do this, either because it is not possible or because they did not
contemplate this case. Thus drivers ought to be able to handle
interrupts coming in as soon as the interrupt handler is registered.

Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp>
---

diff -urNp linux-2.6.22-orig/kernel/irq/manage.c linux-2.6.22/kernel/irq/manage.c
--- linux-2.6.22-orig/kernel/irq/manage.c	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22/kernel/irq/manage.c	2007-07-17 18:37:24.000000000 +0900
@@ -537,6 +537,29 @@ int request_irq(unsigned int irq, irq_ha
 
 	select_smp_affinity(irq);
 
+#if defined(CONFIG_DEBUG_PENDING_IRQ) || defined(CONFIG_DEBUG_SHIRQ)
+#ifndef CONFIG_DEBUG_PENDING_IRQ
+	if (irqflags & IRQF_SHARED) {
+		/*
+		 * It's a shared IRQ -- the driver ought to be prepared for it
+		 * to happen immediately, so let's make sure....
+		 * We do this before actually registering it, to make sure that
+		 * a 'real' IRQ doesn't run in parallel with our fake.
+		 */
+#endif /* !CONFIG_DEBUG_PENDING_IRQ */
+		if (irqflags & IRQF_DISABLED) {
+			unsigned long flags;
+
+			local_irq_save(flags);
+			handler(irq, dev_id);
+			local_irq_restore(flags);
+		} else
+			handler(irq, dev_id);
+#ifndef CONFIG_DEBUG_PENDING_IRQ
+	}
+#endif /* !CONFIG_DEBUG_PENDING_IRQ */
+#endif /* CONFIG_DEBUG_PENDING_IRQ || CONFIG_DEBUG_SHIRQ */
+
 #ifdef CONFIG_DEBUG_SHIRQ
 	if (irqflags & IRQF_SHARED) {
 		/*
diff -urNp linux-2.6.22-orig/lib/Kconfig.debug linux-2.6.22/lib/Kconfig.debug
--- linux-2.6.22-orig/lib/Kconfig.debug	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22/lib/Kconfig.debug	2007-07-17 18:52:00.000000000 +0900
@@ -77,6 +77,17 @@ config DEBUG_KERNEL
 	  Say Y here if you are developing drivers or trying to debug and
 	  identify kernel problems.
 
+config DEBUG_PENDING_IRQ
+	bool "Debug handling of early spurious interrupts"
+	depends on DEBUG_KERNEL && GENERIC_HARDIRQS
+	help
+	  With the advent of kdump it is possible that device drivers receive
+	  interrupts generated in the context of a previous kernel. Ideally
+	  quiescing the underlying devices should suffice but not all drivers
+	  do this, either because it is not possible or because they did not
+	  contemplate this case. Thus drivers ought to be able to handle
+	  interrupts coming in as soon as the interrupt handler is registered.
+
 config DEBUG_SHIRQ
 	bool "Debug shared IRQ handlers"
 	depends on DEBUG_KERNEL && GENERIC_HARDIRQS



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH RFC] Debug handling of early spurious interrupts
  2007-07-17 10:09 [PATCH RFC] Debug handling of early spurious interrupts Fernando Luis Vázquez Cao
@ 2007-07-18 22:46 ` Andrew Morton
  2007-07-20  1:54   ` Fernando Luis Vázquez Cao
  0 siblings, 1 reply; 13+ messages in thread
From: Andrew Morton @ 2007-07-18 22:46 UTC (permalink / raw)
  To: Fernando Luis Vázquez Cao; +Cc: linux-kernel, kexec

On Tue, 17 Jul 2007 19:09:57 +0900
Fernando Luis V__zquez Cao <fernando@oss.ntt.co.jp> wrote:

> With the advent of kdump it is possible that device drivers receive
> interrupts generated in the context of a previous kernel. Ideally
> quiescing the underlying devices should suffice but not all drivers
> do this, either because it is not possible or because they did not
> contemplate this case. Thus drivers ought to be able to handle
> interrupts coming in as soon as the interrupt handler is registered.
> 
> Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp>
> ---
> 
> diff -urNp linux-2.6.22-orig/kernel/irq/manage.c linux-2.6.22/kernel/irq/manage.c
> --- linux-2.6.22-orig/kernel/irq/manage.c	2007-07-09 08:32:17.000000000 +0900
> +++ linux-2.6.22/kernel/irq/manage.c	2007-07-17 18:37:24.000000000 +0900
> @@ -537,6 +537,29 @@ int request_irq(unsigned int irq, irq_ha
>  
>  	select_smp_affinity(irq);
>  
> +#if defined(CONFIG_DEBUG_PENDING_IRQ) || defined(CONFIG_DEBUG_SHIRQ)
> +#ifndef CONFIG_DEBUG_PENDING_IRQ
> +	if (irqflags & IRQF_SHARED) {
> +		/*
> +		 * It's a shared IRQ -- the driver ought to be prepared for it
> +		 * to happen immediately, so let's make sure....
> +		 * We do this before actually registering it, to make sure that
> +		 * a 'real' IRQ doesn't run in parallel with our fake.
> +		 */
> +#endif /* !CONFIG_DEBUG_PENDING_IRQ */
> +		if (irqflags & IRQF_DISABLED) {
> +			unsigned long flags;
> +
> +			local_irq_save(flags);
> +			handler(irq, dev_id);
> +			local_irq_restore(flags);
> +		} else
> +			handler(irq, dev_id);
> +#ifndef CONFIG_DEBUG_PENDING_IRQ
> +	}
> +#endif /* !CONFIG_DEBUG_PENDING_IRQ */
> +#endif /* CONFIG_DEBUG_PENDING_IRQ || CONFIG_DEBUG_SHIRQ */

Even if we were going to merge this functionality as-is, I'd ask for some
sort of refactoring to fix up that ifdef maze.

But more substantial issues:

- This is presented as a "debug" feature, but it isn't a debug feature at
  all - it is new functionality which is unrelated to kernel development.

  Also, it is a "debug" feature which provides no debugging!  At the very
  least, one would expect to see it emit a printk to tell people that we
  have some driver which needs fixing.

  Also, this not-really-a-debug-feature is undesirably coupled with a
  real debugging feature: CONFIG_DEBUG_PENDING_IRQ.

- Does this new feature really need its own Kconfig setting?  Why not enable
  it unconditionally?  request_irq() isn't exactly performance-critical.

- If poss, we really do want to find some way of emitting a warning when
  we detect such a device driver.  Like, call the handler and if it
  returned IRQ_HANDLED, start shouting.



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH RFC] Debug handling of early spurious interrupts
  2007-07-18 22:46 ` Andrew Morton
@ 2007-07-20  1:54   ` Fernando Luis Vázquez Cao
  2007-07-20  2:02     ` [PATCH 1/2] Remove Kconfig setting CONFIG_DEBUG_SHIRQ Fernando Luis Vázquez Cao
  0 siblings, 1 reply; 13+ messages in thread
From: Fernando Luis Vázquez Cao @ 2007-07-20  1:54 UTC (permalink / raw)
  To: Andrew Morton; +Cc: linux-kernel, kexec

On Wed, 2007-07-18 at 15:46 -0700, Andrew Morton wrote:
> On Tue, 17 Jul 2007 19:09:57 +0900
> Fernando Luis V__zquez Cao <fernando@oss.ntt.co.jp> wrote:
> 
> > With the advent of kdump it is possible that device drivers receive
> > interrupts generated in the context of a previous kernel. Ideally
> > quiescing the underlying devices should suffice but not all drivers
> > do this, either because it is not possible or because they did not
> > contemplate this case. Thus drivers ought to be able to handle
> > interrupts coming in as soon as the interrupt handler is registered.
> > 
> > Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp>
> > ---
> > 
> > diff -urNp linux-2.6.22-orig/kernel/irq/manage.c linux-2.6.22/kernel/irq/manage.c
> > --- linux-2.6.22-orig/kernel/irq/manage.c	2007-07-09 08:32:17.000000000 +0900
> > +++ linux-2.6.22/kernel/irq/manage.c	2007-07-17 18:37:24.000000000 +0900
> > @@ -537,6 +537,29 @@ int request_irq(unsigned int irq, irq_ha
> >  
> >  	select_smp_affinity(irq);
> >  
> > +#if defined(CONFIG_DEBUG_PENDING_IRQ) || defined(CONFIG_DEBUG_SHIRQ)
> > +#ifndef CONFIG_DEBUG_PENDING_IRQ
> > +	if (irqflags & IRQF_SHARED) {
> > +		/*
> > +		 * It's a shared IRQ -- the driver ought to be prepared for it
> > +		 * to happen immediately, so let's make sure....
> > +		 * We do this before actually registering it, to make sure that
> > +		 * a 'real' IRQ doesn't run in parallel with our fake.
> > +		 */
> > +#endif /* !CONFIG_DEBUG_PENDING_IRQ */
> > +		if (irqflags & IRQF_DISABLED) {
> > +			unsigned long flags;
> > +
> > +			local_irq_save(flags);
> > +			handler(irq, dev_id);
> > +			local_irq_restore(flags);
> > +		} else
> > +			handler(irq, dev_id);
> > +#ifndef CONFIG_DEBUG_PENDING_IRQ
> > +	}
> > +#endif /* !CONFIG_DEBUG_PENDING_IRQ */
> > +#endif /* CONFIG_DEBUG_PENDING_IRQ || CONFIG_DEBUG_SHIRQ */
> 
> Even if we were going to merge this functionality as-is, I'd ask for some
> sort of refactoring to fix up that ifdef maze.
I a absolutely agree. My first impulse was to get rid of all the cpp
kludge including the Kconfig setting CONFIG_DEBUG_SHIRQ, since, as you
pointed out, request_irq() is not really performance critical.
Unfortunately for the RFC I decided to be conservative and ended up with
an "ifdef maze". Thank you for the heads-up.

> But more substantial issues:
> 
> - This is presented as a "debug" feature, but it isn't a debug feature at
>   all - it is new functionality which is unrelated to kernel development.
Yup.

>   Also, it is a "debug" feature which provides no debugging!  At the very
>   least, one would expect to see it emit a printk to tell people that we
>   have some driver which needs fixing.
I am afraid that in some occasions the kernel may panic inside the
interrupt handler, but I agree that we need to print a meaningful
message for the general case (i.e. something goes wrong but we can
recover). I will do that.

>   Also, this not-really-a-debug-feature is undesirably coupled with a
>   real debugging feature: CONFIG_DEBUG_PENDING_IRQ.
In the new version of the patch I will remove both
CONFIG_DEBUG_PENDING_IRQ and CONFIG_DEBUG_SHIRQ. request_irq() and
setup_irq() are not fast paths and free_irq() much less so.

> - Does this new feature really need its own Kconfig setting?  Why not enable
>   it unconditionally?  request_irq() isn't exactly performance-critical.
I guess the Kconfig setting is not needed. In fact, by enabling this
feature unconditionally we would have _everyone_ (unknowing) testing an
area which is a major pain point for kdump. I am not sure this is an
acceptable default for all systems though. Opinions welcome.

> - If poss, we really do want to find some way of emitting a warning when
>   we detect such a device driver.  Like, call the handler and if it
>   returned IRQ_HANDLED, start shouting.
I will do that and submit updated patches.

Thank you for your feedback, Andrew!


^ permalink raw reply	[flat|nested] 13+ messages in thread

* [PATCH 1/2] Remove Kconfig setting CONFIG_DEBUG_SHIRQ
  2007-07-20  1:54   ` Fernando Luis Vázquez Cao
@ 2007-07-20  2:02     ` Fernando Luis Vázquez Cao
  2007-07-20  2:20       ` [PATCH 2/2] Debug handling of early spurious interrupts Fernando Luis Vázquez Cao
  0 siblings, 1 reply; 13+ messages in thread
From: Fernando Luis Vázquez Cao @ 2007-07-20  2:02 UTC (permalink / raw)
  To: Andrew Morton; +Cc: kexec, linux-kernel

request_irq() and setup_irq() are not fast paths and free_irq() much
less so. In fact, by enabling this feature unconditionally we would have
_everyone_ (unknowingly) testing devices drivers, which hopefully will
result in more bug-reports and, in turn, better drivers.

Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp>
---

diff -urNp linux-2.6.22-orig/arch/arm/configs/at91sam9rlek_defconfig linux-2.6.22-pendirq/arch/arm/configs/at91sam9rlek_defconfig
--- linux-2.6.22-orig/arch/arm/configs/at91sam9rlek_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/arm/configs/at91sam9rlek_defconfig	2007-07-19 16:49:54.000000000 +0900
@@ -906,7 +906,6 @@ CONFIG_ENABLE_MUST_CHECK=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_LOG_BUF_SHIFT=14
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
diff -urNp linux-2.6.22-orig/arch/arm/configs/picotux200_defconfig linux-2.6.22-pendirq/arch/arm/configs/picotux200_defconfig
--- linux-2.6.22-orig/arch/arm/configs/picotux200_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/arm/configs/picotux200_defconfig	2007-07-19 16:50:00.000000000 +0900
@@ -1293,7 +1293,6 @@ CONFIG_ENABLE_MUST_CHECK=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_LOG_BUF_SHIFT=14
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
diff -urNp linux-2.6.22-orig/arch/arm/configs/s3c2410_defconfig linux-2.6.22-pendirq/arch/arm/configs/s3c2410_defconfig
--- linux-2.6.22-orig/arch/arm/configs/s3c2410_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/arm/configs/s3c2410_defconfig	2007-07-19 16:50:06.000000000 +0900
@@ -1368,7 +1368,6 @@ CONFIG_MAGIC_SYSRQ=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_LOG_BUF_SHIFT=16
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
diff -urNp linux-2.6.22-orig/arch/avr32/configs/atngw100_defconfig linux-2.6.22-pendirq/arch/avr32/configs/atngw100_defconfig
--- linux-2.6.22-orig/arch/avr32/configs/atngw100_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/avr32/configs/atngw100_defconfig	2007-07-19 16:50:11.000000000 +0900
@@ -925,7 +925,6 @@ CONFIG_MAGIC_SYSRQ=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/avr32/configs/atstk1002_defconfig linux-2.6.22-pendirq/arch/avr32/configs/atstk1002_defconfig
--- linux-2.6.22-orig/arch/avr32/configs/atstk1002_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/avr32/configs/atstk1002_defconfig	2007-07-19 16:50:16.000000000 +0900
@@ -756,7 +756,6 @@ CONFIG_MAGIC_SYSRQ=y
 CONFIG_DEBUG_FS=y
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/i386/defconfig linux-2.6.22-pendirq/arch/i386/defconfig
--- linux-2.6.22-orig/arch/i386/defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/i386/defconfig	2007-07-19 16:50:21.000000000 +0900
@@ -1424,7 +1424,6 @@ CONFIG_UNUSED_SYMBOLS=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/ia64/configs/tiger_defconfig linux-2.6.22-pendirq/arch/ia64/configs/tiger_defconfig
--- linux-2.6.22-orig/arch/ia64/configs/tiger_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/ia64/configs/tiger_defconfig	2007-07-19 16:50:26.000000000 +0900
@@ -1318,7 +1318,6 @@ CONFIG_MAGIC_SYSRQ=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_LOG_BUF_SHIFT=20
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
diff -urNp linux-2.6.22-orig/arch/ia64/configs/zx1_defconfig linux-2.6.22-pendirq/arch/ia64/configs/zx1_defconfig
--- linux-2.6.22-orig/arch/ia64/configs/zx1_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/ia64/configs/zx1_defconfig	2007-07-19 16:50:31.000000000 +0900
@@ -1533,7 +1533,6 @@ CONFIG_MAGIC_SYSRQ=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_LOG_BUF_SHIFT=17
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
diff -urNp linux-2.6.22-orig/arch/ia64/defconfig linux-2.6.22-pendirq/arch/ia64/defconfig
--- linux-2.6.22-orig/arch/ia64/defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/ia64/defconfig	2007-07-19 16:50:36.000000000 +0900
@@ -1482,7 +1482,6 @@ CONFIG_MAGIC_SYSRQ=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_LOG_BUF_SHIFT=20
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
diff -urNp linux-2.6.22-orig/arch/mips/configs/bigsur_defconfig linux-2.6.22-pendirq/arch/mips/configs/bigsur_defconfig
--- linux-2.6.22-orig/arch/mips/configs/bigsur_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/mips/configs/bigsur_defconfig	2007-07-19 16:50:42.000000000 +0900
@@ -1003,7 +1003,6 @@ CONFIG_MAGIC_SYSRQ=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_LOG_BUF_SHIFT=16
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
diff -urNp linux-2.6.22-orig/arch/mips/configs/mipssim_defconfig linux-2.6.22-pendirq/arch/mips/configs/mipssim_defconfig
--- linux-2.6.22-orig/arch/mips/configs/mipssim_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/mips/configs/mipssim_defconfig	2007-07-19 16:50:54.000000000 +0900
@@ -855,7 +855,6 @@ CONFIG_ENABLE_MUST_CHECK=y
 # CONFIG_UNUSED_SYMBOLS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_LOG_BUF_SHIFT=14
 # CONFIG_DETECT_SOFTLOCKUP is not set
 # CONFIG_SCHEDSTATS is not set
diff -urNp linux-2.6.22-orig/arch/mips/configs/pnx8550-jbs_defconfig linux-2.6.22-pendirq/arch/mips/configs/pnx8550-jbs_defconfig
--- linux-2.6.22-orig/arch/mips/configs/pnx8550-jbs_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/mips/configs/pnx8550-jbs_defconfig	2007-07-19 16:51:08.000000000 +0900
@@ -1205,7 +1205,6 @@ CONFIG_MAGIC_SYSRQ=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_LOG_BUF_SHIFT=14
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
diff -urNp linux-2.6.22-orig/arch/mips/configs/pnx8550-stb810_defconfig linux-2.6.22-pendirq/arch/mips/configs/pnx8550-stb810_defconfig
--- linux-2.6.22-orig/arch/mips/configs/pnx8550-stb810_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/mips/configs/pnx8550-stb810_defconfig	2007-07-19 16:51:33.000000000 +0900
@@ -1195,7 +1195,6 @@ CONFIG_MAGIC_SYSRQ=y
 # CONFIG_DEBUG_FS is not set
 CONFIG_HEADERS_CHECK=y
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_LOG_BUF_SHIFT=14
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
diff -urNp linux-2.6.22-orig/arch/mips/configs/yosemite_defconfig linux-2.6.22-pendirq/arch/mips/configs/yosemite_defconfig
--- linux-2.6.22-orig/arch/mips/configs/yosemite_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/mips/configs/yosemite_defconfig	2007-07-19 16:51:38.000000000 +0900
@@ -816,7 +816,6 @@ CONFIG_ENABLE_MUST_CHECK=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_LOG_BUF_SHIFT=14
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/cell_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/cell_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/cell_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/cell_defconfig	2007-07-19 16:51:59.000000000 +0900
@@ -1468,7 +1468,6 @@ CONFIG_MAGIC_SYSRQ=y
 CONFIG_DEBUG_FS=y
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 # CONFIG_DETECT_SOFTLOCKUP is not set
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/celleb_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/celleb_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/celleb_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/celleb_defconfig	2007-07-19 16:52:06.000000000 +0900
@@ -1295,7 +1295,6 @@ CONFIG_MAGIC_SYSRQ=y
 CONFIG_DEBUG_FS=y
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/chrp32_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/chrp32_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/chrp32_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/chrp32_defconfig	2007-07-19 16:52:12.000000000 +0900
@@ -1454,7 +1454,6 @@ CONFIG_MAGIC_SYSRQ=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/ebony_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/ebony_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/ebony_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/ebony_defconfig	2007-07-19 16:52:16.000000000 +0900
@@ -772,7 +772,6 @@ CONFIG_MAGIC_SYSRQ=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/g5_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/g5_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/g5_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/g5_defconfig	2007-07-19 16:52:22.000000000 +0900
@@ -1645,7 +1645,6 @@ CONFIG_MAGIC_SYSRQ=y
 CONFIG_DEBUG_FS=y
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/holly_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/holly_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/holly_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/holly_defconfig	2007-07-19 16:52:26.000000000 +0900
@@ -985,7 +985,6 @@ CONFIG_MAGIC_SYSRQ=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/iseries_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/iseries_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/iseries_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/iseries_defconfig	2007-07-19 16:52:30.000000000 +0900
@@ -1147,7 +1147,6 @@ CONFIG_MAGIC_SYSRQ=y
 CONFIG_DEBUG_FS=y
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/linkstation_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/linkstation_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/linkstation_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/linkstation_defconfig	2007-07-19 16:52:34.000000000 +0900
@@ -1530,7 +1530,6 @@ CONFIG_MAGIC_SYSRQ=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/lite5200_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/lite5200_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/lite5200_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/lite5200_defconfig	2007-07-19 16:52:38.000000000 +0900
@@ -893,7 +893,6 @@ CONFIG_ENABLE_MUST_CHECK=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/maple_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/maple_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/maple_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/maple_defconfig	2007-07-19 16:52:43.000000000 +0900
@@ -1169,7 +1169,6 @@ CONFIG_MAGIC_SYSRQ=y
 CONFIG_DEBUG_FS=y
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/mpc8272_ads_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/mpc8272_ads_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/mpc8272_ads_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/mpc8272_ads_defconfig	2007-07-19 16:52:47.000000000 +0900
@@ -844,7 +844,6 @@ CONFIG_ENABLE_MUST_CHECK=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/mpc8313_rdb_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/mpc8313_rdb_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/mpc8313_rdb_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/mpc8313_rdb_defconfig	2007-07-19 16:52:51.000000000 +0900
@@ -1336,7 +1336,6 @@ CONFIG_ENABLE_MUST_CHECK=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/mpc8540_ads_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/mpc8540_ads_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/mpc8540_ads_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/mpc8540_ads_defconfig	2007-07-19 16:52:56.000000000 +0900
@@ -772,7 +772,6 @@ CONFIG_ENABLE_MUST_CHECK=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/mpc8544_ds_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/mpc8544_ds_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/mpc8544_ds_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/mpc8544_ds_defconfig	2007-07-19 16:53:00.000000000 +0900
@@ -963,7 +963,6 @@ CONFIG_ENABLE_MUST_CHECK=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/mpc8560_ads_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/mpc8560_ads_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/mpc8560_ads_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/mpc8560_ads_defconfig	2007-07-19 16:53:04.000000000 +0900
@@ -852,7 +852,6 @@ CONFIG_ENABLE_MUST_CHECK=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/mpc8568mds_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/mpc8568mds_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/mpc8568mds_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/mpc8568mds_defconfig	2007-07-19 16:53:08.000000000 +0900
@@ -890,7 +890,6 @@ CONFIG_ENABLE_MUST_CHECK=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/mpc85xx_cds_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/mpc85xx_cds_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/mpc85xx_cds_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/mpc85xx_cds_defconfig	2007-07-19 16:53:11.000000000 +0900
@@ -905,7 +905,6 @@ CONFIG_ENABLE_MUST_CHECK=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/mpc8641_hpcn_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/mpc8641_hpcn_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/mpc8641_hpcn_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/mpc8641_hpcn_defconfig	2007-07-19 16:53:15.000000000 +0900
@@ -962,7 +962,6 @@ CONFIG_ENABLE_MUST_CHECK=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/pasemi_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/pasemi_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/pasemi_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/pasemi_defconfig	2007-07-19 16:53:27.000000000 +0900
@@ -1646,7 +1646,6 @@ CONFIG_MAGIC_SYSRQ=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/pmac32_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/pmac32_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/pmac32_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/pmac32_defconfig	2007-07-19 16:53:30.000000000 +0900
@@ -1885,7 +1885,6 @@ CONFIG_MAGIC_SYSRQ=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/ppc64_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/ppc64_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/ppc64_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/ppc64_defconfig	2007-07-19 16:53:34.000000000 +0900
@@ -1801,7 +1801,6 @@ CONFIG_MAGIC_SYSRQ=y
 CONFIG_DEBUG_FS=y
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/ps3_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/ps3_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/ps3_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/ps3_defconfig	2007-07-19 16:53:40.000000000 +0900
@@ -1091,7 +1091,6 @@ CONFIG_ENABLE_MUST_CHECK=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/powerpc/configs/pseries_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/pseries_defconfig
--- linux-2.6.22-orig/arch/powerpc/configs/pseries_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/powerpc/configs/pseries_defconfig	2007-07-19 16:54:16.000000000 +0900
@@ -1580,7 +1580,6 @@ CONFIG_MAGIC_SYSRQ=y
 CONFIG_DEBUG_FS=y
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/sh/configs/r7780rp_defconfig linux-2.6.22-pendirq/arch/sh/configs/r7780rp_defconfig
--- linux-2.6.22-orig/arch/sh/configs/r7780rp_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/sh/configs/r7780rp_defconfig	2007-07-19 16:54:27.000000000 +0900
@@ -1235,7 +1235,6 @@ CONFIG_MAGIC_SYSRQ=y
 CONFIG_DEBUG_FS=y
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_LOG_BUF_SHIFT=14
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
diff -urNp linux-2.6.22-orig/arch/sh/configs/r7785rp_defconfig linux-2.6.22-pendirq/arch/sh/configs/r7785rp_defconfig
--- linux-2.6.22-orig/arch/sh/configs/r7785rp_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/sh/configs/r7785rp_defconfig	2007-07-19 16:54:31.000000000 +0900
@@ -1238,7 +1238,6 @@ CONFIG_MAGIC_SYSRQ=y
 CONFIG_DEBUG_FS=y
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_LOG_BUF_SHIFT=14
 # CONFIG_DETECT_SOFTLOCKUP is not set
 # CONFIG_SCHEDSTATS is not set
diff -urNp linux-2.6.22-orig/arch/sh/configs/se7712_defconfig linux-2.6.22-pendirq/arch/sh/configs/se7712_defconfig
--- linux-2.6.22-orig/arch/sh/configs/se7712_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/sh/configs/se7712_defconfig	2007-07-19 16:54:34.000000000 +0900
@@ -993,7 +993,6 @@ CONFIG_ENABLE_MUST_CHECK=y
 # CONFIG_DEBUG_FS is not set
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_LOG_BUF_SHIFT=14
 # CONFIG_DETECT_SOFTLOCKUP is not set
 # CONFIG_SCHEDSTATS is not set
diff -urNp linux-2.6.22-orig/arch/sh64/configs/cayman_defconfig linux-2.6.22-pendirq/arch/sh64/configs/cayman_defconfig
--- linux-2.6.22-orig/arch/sh64/configs/cayman_defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/sh64/configs/cayman_defconfig	2007-07-19 16:54:22.000000000 +0900
@@ -999,7 +999,6 @@ CONFIG_MAGIC_SYSRQ=y
 CONFIG_DEBUG_FS=y
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 CONFIG_SCHEDSTATS=y
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/sparc64/defconfig linux-2.6.22-pendirq/arch/sparc64/defconfig
--- linux-2.6.22-orig/arch/sparc64/defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/sparc64/defconfig	2007-07-19 16:54:38.000000000 +0900
@@ -1463,7 +1463,6 @@ CONFIG_MAGIC_SYSRQ=y
 CONFIG_DEBUG_FS=y
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 CONFIG_SCHEDSTATS=y
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/arch/x86_64/defconfig linux-2.6.22-pendirq/arch/x86_64/defconfig
--- linux-2.6.22-orig/arch/x86_64/defconfig	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/arch/x86_64/defconfig	2007-07-19 16:54:43.000000000 +0900
@@ -1522,7 +1522,6 @@ CONFIG_UNUSED_SYMBOLS=y
 CONFIG_DEBUG_FS=y
 # CONFIG_HEADERS_CHECK is not set
 CONFIG_DEBUG_KERNEL=y
-# CONFIG_DEBUG_SHIRQ is not set
 CONFIG_DETECT_SOFTLOCKUP=y
 # CONFIG_SCHEDSTATS is not set
 # CONFIG_TIMER_STATS is not set
diff -urNp linux-2.6.22-orig/drivers/char/cyclades.c linux-2.6.22-pendirq/drivers/char/cyclades.c
--- linux-2.6.22-orig/drivers/char/cyclades.c	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/drivers/char/cyclades.c	2007-07-19 17:11:48.000000000 +0900
@@ -1408,7 +1408,7 @@ static irqreturn_t cyy_interrupt(int irq
 	card_base_addr = cinfo->base_addr;
 	index = cinfo->bus_index;
 
-	/* card was not initialized yet (e.g. DEBUG_SHIRQ) */
+	/* card was not initialized yet (e.g. shared IRQ) */
 	if (unlikely(card_base_addr == NULL))
 		return IRQ_HANDLED;
 
diff -urNp linux-2.6.22-orig/drivers/net/e1000/e1000_main.c linux-2.6.22-pendirq/drivers/net/e1000/e1000_main.c
--- linux-2.6.22-orig/drivers/net/e1000/e1000_main.c	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/drivers/net/e1000/e1000_main.c	2007-07-19 17:23:55.000000000 +0900
@@ -1424,10 +1424,11 @@ e1000_open(struct net_device *netdev)
 	    e1000_check_mng_mode(&adapter->hw))
 		e1000_get_hw_control(adapter);
 
-	/* before we allocate an interrupt, we must be ready to handle it.
-	 * Setting DEBUG_SHIRQ in the kernel makes it fire an interrupt
-	 * as soon as we call pci_request_irq, so we have to setup our
-	 * clean_rx handler before we do so.  */
+	/* Before we allocate an interrupt, we must be ready to handle it.
+	 * Any pending interrupts (coming from a device sharing our IRQ line or
+	 * generated in the context of a previous kernel if this is a kdump
+	 * kernel, for example) will come in as soon as we call pci_request_irq,
+	 * so we have to setup our clean_rx handler before we do so.  */
 	e1000_configure(adapter);
 
 	err = e1000_request_irq(adapter);
diff -urNp linux-2.6.22-orig/kernel/irq/manage.c linux-2.6.22-pendirq/kernel/irq/manage.c
--- linux-2.6.22-orig/kernel/irq/manage.c	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/kernel/irq/manage.c	2007-07-19 17:04:50.000000000 +0900
@@ -374,14 +374,12 @@ int setup_irq(unsigned int irq, struct i
 	return 0;
 
 mismatch:
-#ifdef CONFIG_DEBUG_SHIRQ
 	if (!(new->flags & IRQF_PROBE_SHARED)) {
 		printk(KERN_ERR "IRQ handler type mismatch for IRQ %d\n", irq);
 		if (old_name)
 			printk(KERN_ERR "current handler: %s\n", old_name);
 		dump_stack();
 	}
-#endif
 	spin_unlock_irqrestore(&desc->lock, flags);
 	return -EBUSY;
 }
@@ -454,7 +452,6 @@ void free_irq(unsigned int irq, void *de
 		spin_unlock_irqrestore(&desc->lock, flags);
 		return;
 	}
-#ifdef CONFIG_DEBUG_SHIRQ
 	if (handler) {
 		/*
 		 * It's a shared IRQ -- the driver ought to be prepared for it
@@ -464,7 +461,6 @@ void free_irq(unsigned int irq, void *de
 		 */
 		handler(irq, dev_id);
 	}
-#endif
 }
 EXPORT_SYMBOL(free_irq);
 
@@ -537,7 +533,6 @@ int request_irq(unsigned int irq, irq_ha
 
 	select_smp_affinity(irq);
 
-#ifdef CONFIG_DEBUG_SHIRQ
 	if (irqflags & IRQF_SHARED) {
 		/*
 		 * It's a shared IRQ -- the driver ought to be prepared for it
@@ -554,7 +549,6 @@ int request_irq(unsigned int irq, irq_ha
 		} else
 			handler(irq, dev_id);
 	}
-#endif
 
 	retval = setup_irq(irq, action);
 	if (retval)
diff -urNp linux-2.6.22-orig/lib/Kconfig.debug linux-2.6.22-pendirq/lib/Kconfig.debug
--- linux-2.6.22-orig/lib/Kconfig.debug	2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22-pendirq/lib/Kconfig.debug	2007-07-19 17:05:52.000000000 +0900
@@ -77,15 +77,6 @@ config DEBUG_KERNEL
 	  Say Y here if you are developing drivers or trying to debug and
 	  identify kernel problems.
 
-config DEBUG_SHIRQ
-	bool "Debug shared IRQ handlers"
-	depends on DEBUG_KERNEL && GENERIC_HARDIRQS
-	help
-	  Enable this to generate a spurious interrupt as soon as a shared
-	  interrupt handler is registered, and just before one is deregistered.
-	  Drivers ought to be able to handle interrupts coming in at those
-	  points; some don't and need to be caught.
-
 config DETECT_SOFTLOCKUP
 	bool "Detect Soft Lockups"
 	depends on DEBUG_KERNEL && !S390



^ permalink raw reply	[flat|nested] 13+ messages in thread

* [PATCH 2/2] Debug handling of early spurious interrupts
  2007-07-20  2:02     ` [PATCH 1/2] Remove Kconfig setting CONFIG_DEBUG_SHIRQ Fernando Luis Vázquez Cao
@ 2007-07-20  2:20       ` Fernando Luis Vázquez Cao
  2007-07-20 21:43         ` Andrew Morton
  2007-07-25  9:18         ` [PATCH RFC] e1000: clear ICR before requesting an IRQ line Fernando Luis Vázquez Cao
  0 siblings, 2 replies; 13+ messages in thread
From: Fernando Luis Vázquez Cao @ 2007-07-20  2:20 UTC (permalink / raw)
  To: Andrew Morton; +Cc: kexec, linux-kernel

With the advent of kdump it is possible that device drivers receive
interrupts generated in the context of a previous kernel. Ideally
quiescing the underlying devices should suffice but not all drivers do
this, either because it is not possible or because they did not
contemplate this case. Thus drivers ought to be able to handle
interrupts coming in as soon as the interrupt handler is registered.

Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp>
---

diff -urNp linux-2.6.22-orig/kernel/irq/manage.c linux-2.6.22-pendirq/kernel/irq/manage.c
--- linux-2.6.22-orig/kernel/irq/manage.c	2007-07-19 18:18:53.000000000 +0900
+++ linux-2.6.22-pendirq/kernel/irq/manage.c	2007-07-19 19:43:41.000000000 +0900
@@ -533,21 +533,32 @@ int request_irq(unsigned int irq, irq_ha
 
 	select_smp_affinity(irq);
 
-	if (irqflags & IRQF_SHARED) {
-		/*
-		 * It's a shared IRQ -- the driver ought to be prepared for it
-		 * to happen immediately, so let's make sure....
-		 * We do this before actually registering it, to make sure that
-		 * a 'real' IRQ doesn't run in parallel with our fake
-		 */
-		if (irqflags & IRQF_DISABLED) {
-			unsigned long flags;
+	/*
+	 * With the advent of kdump it possible that device drivers receive
+	 * interrupts generated in the context of a previous kernel. Ideally
+	 * quiescing the underlying devices should suffice but not all drivers
+	 * do this, either because it is not possible or because they did not
+	 * contemplate this case. Thus drivers ought to be able to handle
+	 * interrupts coming in as soon as the interrupt handler is registered.
+	 *
+	 * Besides, if it is a shared IRQ the driver ought to be prepared for
+	 * it to happen immediately too.
+	 *
+	 * We do this before actually registering it, to make sure that
+	 * a 'real' IRQ doesn't run in parallel with our fake.
+	 */
+	if (irqflags & IRQF_DISABLED) {
+		unsigned long flags;
 
-			local_irq_save(flags);
-			handler(irq, dev_id);
-			local_irq_restore(flags);
-		} else
-			handler(irq, dev_id);
+		local_irq_save(flags);
+		retval = handler(irq, dev_id);
+		local_irq_restore(flags);
+	} else
+		retval = handler(irq, dev_id);
+	if (retval == IRQ_HANDLED) {
+		printk(KERN_WARNING
+		       "%s (IRQ %d) handled a spurious interrupt\n",
+		       devname, irq);
 	}
 
 	retval = setup_irq(irq, action);



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 2/2] Debug handling of early spurious interrupts
  2007-07-20  2:20       ` [PATCH 2/2] Debug handling of early spurious interrupts Fernando Luis Vázquez Cao
@ 2007-07-20 21:43         ` Andrew Morton
  2007-07-30  9:58           ` Fernando Luis Vázquez Cao
  2007-07-25  9:18         ` [PATCH RFC] e1000: clear ICR before requesting an IRQ line Fernando Luis Vázquez Cao
  1 sibling, 1 reply; 13+ messages in thread
From: Andrew Morton @ 2007-07-20 21:43 UTC (permalink / raw)
  To: Fernando Luis Vázquez Cao; +Cc: kexec, linux-kernel

On Fri, 20 Jul 2007 11:20:43 +0900
Fernando Luis V__zquez Cao <fernando@oss.ntt.co.jp> wrote:

> With the advent of kdump it is possible that device drivers receive
> interrupts generated in the context of a previous kernel. Ideally
> quiescing the underlying devices should suffice but not all drivers do
> this, either because it is not possible or because they did not
> contemplate this case. Thus drivers ought to be able to handle
> interrupts coming in as soon as the interrupt handler is registered.
> 
> Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp>
> ---
> 
> diff -urNp linux-2.6.22-orig/kernel/irq/manage.c linux-2.6.22-pendirq/kernel/irq/manage.c
> --- linux-2.6.22-orig/kernel/irq/manage.c	2007-07-19 18:18:53.000000000 +0900
> +++ linux-2.6.22-pendirq/kernel/irq/manage.c	2007-07-19 19:43:41.000000000 +0900
> @@ -533,21 +533,32 @@ int request_irq(unsigned int irq, irq_ha
>  
>  	select_smp_affinity(irq);
>  
> -	if (irqflags & IRQF_SHARED) {
> -		/*
> -		 * It's a shared IRQ -- the driver ought to be prepared for it
> -		 * to happen immediately, so let's make sure....
> -		 * We do this before actually registering it, to make sure that
> -		 * a 'real' IRQ doesn't run in parallel with our fake
> -		 */
> -		if (irqflags & IRQF_DISABLED) {
> -			unsigned long flags;
> +	/*
> +	 * With the advent of kdump it possible that device drivers receive
> +	 * interrupts generated in the context of a previous kernel. Ideally
> +	 * quiescing the underlying devices should suffice but not all drivers
> +	 * do this, either because it is not possible or because they did not
> +	 * contemplate this case. Thus drivers ought to be able to handle
> +	 * interrupts coming in as soon as the interrupt handler is registered.
> +	 *
> +	 * Besides, if it is a shared IRQ the driver ought to be prepared for
> +	 * it to happen immediately too.
> +	 *
> +	 * We do this before actually registering it, to make sure that
> +	 * a 'real' IRQ doesn't run in parallel with our fake.
> +	 */
> +	if (irqflags & IRQF_DISABLED) {
> +		unsigned long flags;
>  
> -			local_irq_save(flags);
> -			handler(irq, dev_id);
> -			local_irq_restore(flags);
> -		} else
> -			handler(irq, dev_id);
> +		local_irq_save(flags);
> +		retval = handler(irq, dev_id);
> +		local_irq_restore(flags);
> +	} else
> +		retval = handler(irq, dev_id);
> +	if (retval == IRQ_HANDLED) {
> +		printk(KERN_WARNING
> +		       "%s (IRQ %d) handled a spurious interrupt\n",
> +		       devname, irq);
>  	}
>  

This change means that we'll run the irq handler at request_irq()-time even
for non-shared interrupts.

This is a bit of a worry.  See, shared-interrupt handlers are required to
be able to cope with being called when their device isn't interrupting. 
But nobody ever said that non-shared interrupt handlers need to be able to
cope with that.

Hence these non-shared handlers are within their rights to emit warning
printks, go BUG or accuse the operator of having busted hardware.

So bad things might happen because of this change.  And if they do, they
will take a loooong time to be discovered, because non-shared interrupt
handlers tend to dwell in crufty old drivers which not many people use.

So I'm wondering if it would be more prudent to only do all this for shared
handlers?



^ permalink raw reply	[flat|nested] 13+ messages in thread

* [PATCH RFC] e1000: clear ICR before requesting an IRQ line
  2007-07-20  2:20       ` [PATCH 2/2] Debug handling of early spurious interrupts Fernando Luis Vázquez Cao
  2007-07-20 21:43         ` Andrew Morton
@ 2007-07-25  9:18         ` Fernando Luis Vázquez Cao
  2007-07-25 15:27           ` Kok, Auke
  1 sibling, 1 reply; 13+ messages in thread
From: Fernando Luis Vázquez Cao @ 2007-07-25  9:18 UTC (permalink / raw)
  To: e1000-devel
  Cc: Andrew Morton, linux-kernel, john.ronciak, jesse.brandeburg,
	jeffrey.t.kirsher, auke-jan.h.kok

I made an interesting finding while testing the two patches below.

http://lkml.org/lkml/2007/7/19/685
http://lkml.org/lkml/2007/7/19/687

These patches modify the traditional CONFIG_DEBUG_KERNEL in such a way
that the request_irq prints a warning if after calling the handler it
returned IRQ_HANDLED .

The code looks like this:

int request_irq(unsigned int irq, irq_handler_t handler,
                unsigned long irqflags, const char *devname, void *dev_id)
.....
        if (irqflags & IRQF_DISABLED) {
                unsigned long flags;

                local_irq_save(flags);
                retval = handler(irq, dev_id);
                local_irq_restore(flags);
        } else
                retval = handler(irq, dev_id);
        if (retval == IRQ_HANDLED) {
                printk(KERN_WARNING
                       "%s (IRQ %d) handled a spurious interrupt\n",
                       devname, irq);
        }
.....

I discovered that the e1000 driver handles the "fake" interrupt, which,
in principle, is not correct because it obviously isn't a real interrupt
and it could have been an interrupt coming from another device that is
sharing the IRQ line.

The problem is that the interrupt handler assumes that if ICR!=0 it was
its device who generated the interrupt and, consequently, it should be
handled. But, unfortunately, that is not always the case. If the network
link is active when we open the device (e1000_open) the ICR will have
the E1000_ICR_LSC bit set (by the way, is this the expected behavior?).
This means that _any_ interrupt coming in after allocating our interrupt
(e1000_request_irq) will be handled, no matter where it came from.

The solution I came up with is clearing the ICR before calling
request_irq. I have to admit that I am not familiar enough with this
driver, so it is quite likely that this is not the right fix. I would
appreciate your comments on this.

Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp>
---

diff -urNp linux-2.6.22-orig/drivers/net/e1000/e1000_main.c linux-2.6.22-pendirq/drivers/net/e1000/e1000_main.c
--- linux-2.6.22-orig/drivers/net/e1000/e1000_main.c	2007-07-19 18:18:53.000000000 +0900
+++ linux-2.6.22-pendirq/drivers/net/e1000/e1000_main.c	2007-07-25 17:22:54.000000000 +0900
@@ -1378,6 +1378,17 @@ e1000_alloc_queues(struct e1000_adapter 
 }
 
 /**
+ * e1000_clear_interrupts
+ * @adapter: address of board private structure
+ *
+ * Mask interrupts
+ **/
+static void
+e1000_clear_interrupts(struct e1000_adapter *adapter) {
+	E1000_READ_REG(&adapter->hw, ICR);
+}
+
+/**
  * e1000_open - Called when a network interface is made active
  * @netdev: network interface device structure
  *
@@ -1431,6 +1442,9 @@ e1000_open(struct net_device *netdev)
 	 * so we have to setup our clean_rx handler before we do so.  */
 	e1000_configure(adapter);
 
+	/* Discard any possible pending interrupts. */
+	e1000_clear_interrupts(adapter);
+
 	err = e1000_request_irq(adapter);
 	if (err)
 		goto err_req_irq;



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH RFC] e1000: clear ICR before requesting an IRQ line
  2007-07-25  9:18         ` [PATCH RFC] e1000: clear ICR before requesting an IRQ line Fernando Luis Vázquez Cao
@ 2007-07-25 15:27           ` Kok, Auke
  2007-07-26  1:34             ` Fernando Luis Vázquez Cao
  0 siblings, 1 reply; 13+ messages in thread
From: Kok, Auke @ 2007-07-25 15:27 UTC (permalink / raw)
  To: Fernando Luis Vázquez Cao
  Cc: e1000-devel, Andrew Morton, linux-kernel, john.ronciak,
	jesse.brandeburg, jeffrey.t.kirsher

Fernando Luis Vázquez Cao wrote:
> I made an interesting finding while testing the two patches below.
> 
> http://lkml.org/lkml/2007/7/19/685
> http://lkml.org/lkml/2007/7/19/687
> 
> These patches modify the traditional CONFIG_DEBUG_KERNEL in such a way
> that the request_irq prints a warning if after calling the handler it
> returned IRQ_HANDLED .
> 
> The code looks like this:
> 
> int request_irq(unsigned int irq, irq_handler_t handler,
>                 unsigned long irqflags, const char *devname, void *dev_id)
> .....
>         if (irqflags & IRQF_DISABLED) {
>                 unsigned long flags;
> 
>                 local_irq_save(flags);
>                 retval = handler(irq, dev_id);
>                 local_irq_restore(flags);
>         } else
>                 retval = handler(irq, dev_id);
>         if (retval == IRQ_HANDLED) {
>                 printk(KERN_WARNING
>                        "%s (IRQ %d) handled a spurious interrupt\n",
>                        devname, irq);
>         }
> .....
> 
> I discovered that the e1000 driver handles the "fake" interrupt, which,
> in principle, is not correct because it obviously isn't a real interrupt
> and it could have been an interrupt coming from another device that is
> sharing the IRQ line.
> 
> The problem is that the interrupt handler assumes that if ICR!=0 it was
> its device who generated the interrupt and, consequently, it should be
> handled. But, unfortunately, that is not always the case. If the network
> link is active when we open the device (e1000_open) the ICR will have
> the E1000_ICR_LSC bit set (by the way, is this the expected behavior?).

yes. is it really a problem though?

> This means that _any_ interrupt coming in after allocating our interrupt
> (e1000_request_irq) will be handled, no matter where it came from.

we actually generate this LSC interrupt ourselves in the driver, to make sure 
that we cascade into the watchdog which then enables or disables the link code 
based on the link status change. This allows us to _not_ do any link checking in 
_open and makes things a bit more simple.

> The solution I came up with is clearing the ICR before calling
> request_irq. I have to admit that I am not familiar enough with this
> driver, so it is quite likely that this is not the right fix. I would
> appreciate your comments on this.

Clearing the ICR before requesting an irq might not work - at the same time the 
device could generate another LSC irq...

Of course, we probably should just schedule some delayed work to run our 
watchdog in e1000_open, but I haven't checked if that actually works.


Auke

> Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp>
> ---
> 
> diff -urNp linux-2.6.22-orig/drivers/net/e1000/e1000_main.c linux-2.6.22-pendirq/drivers/net/e1000/e1000_main.c
> --- linux-2.6.22-orig/drivers/net/e1000/e1000_main.c	2007-07-19 18:18:53.000000000 +0900
> +++ linux-2.6.22-pendirq/drivers/net/e1000/e1000_main.c	2007-07-25 17:22:54.000000000 +0900
> @@ -1378,6 +1378,17 @@ e1000_alloc_queues(struct e1000_adapter 
>  }
>  
>  /**
> + * e1000_clear_interrupts
> + * @adapter: address of board private structure
> + *
> + * Mask interrupts
> + **/
> +static void
> +e1000_clear_interrupts(struct e1000_adapter *adapter) {
> +	E1000_READ_REG(&adapter->hw, ICR);
> +}
> +
> +/**
>   * e1000_open - Called when a network interface is made active
>   * @netdev: network interface device structure
>   *
> @@ -1431,6 +1442,9 @@ e1000_open(struct net_device *netdev)
>  	 * so we have to setup our clean_rx handler before we do so.  */
>  	e1000_configure(adapter);
>  
> +	/* Discard any possible pending interrupts. */
> +	e1000_clear_interrupts(adapter);
> +
>  	err = e1000_request_irq(adapter);
>  	if (err)
>  		goto err_req_irq;

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH RFC] e1000: clear ICR before requesting an IRQ line
  2007-07-25 15:27           ` Kok, Auke
@ 2007-07-26  1:34             ` Fernando Luis Vázquez Cao
  0 siblings, 0 replies; 13+ messages in thread
From: Fernando Luis Vázquez Cao @ 2007-07-26  1:34 UTC (permalink / raw)
  To: Kok, Auke
  Cc: e1000-devel, Andrew Morton, linux-kernel, john.ronciak,
	jesse.brandeburg, jeffrey.t.kirsher

On Wed, 2007-07-25 at 08:27 -0700, Kok, Auke wrote:
> Fernando Luis Vázquez Cao wrote:
> > I made an interesting finding while testing the two patches below.
> > 
> > http://lkml.org/lkml/2007/7/19/685
> > http://lkml.org/lkml/2007/7/19/687
> > 
> > These patches modify the traditional CONFIG_DEBUG_KERNEL in such a way
> > that the request_irq prints a warning if after calling the handler it
> > returned IRQ_HANDLED .
> > 
> > The code looks like this:
> > 
> > int request_irq(unsigned int irq, irq_handler_t handler,
> >                 unsigned long irqflags, const char *devname, void *dev_id)
> > .....
> >         if (irqflags & IRQF_DISABLED) {
> >                 unsigned long flags;
> > 
> >                 local_irq_save(flags);
> >                 retval = handler(irq, dev_id);
> >                 local_irq_restore(flags);
> >         } else
> >                 retval = handler(irq, dev_id);
> >         if (retval == IRQ_HANDLED) {
> >                 printk(KERN_WARNING
> >                        "%s (IRQ %d) handled a spurious interrupt\n",
> >                        devname, irq);
> >         }
> > .....
> > 
> > I discovered that the e1000 driver handles the "fake" interrupt, which,
> > in principle, is not correct because it obviously isn't a real interrupt
> > and it could have been an interrupt coming from another device that is
> > sharing the IRQ line.
> > 
> > The problem is that the interrupt handler assumes that if ICR!=0 it was
> > its device who generated the interrupt and, consequently, it should be
> > handled. But, unfortunately, that is not always the case. If the network
> > link is active when we open the device (e1000_open) the ICR will have
> > the E1000_ICR_LSC bit set (by the way, is this the expected behavior?).
> 
> yes. is it really a problem though?
It seems we may end up handling spurious interrupts or interrupts coming
from another devices.

> > This means that _any_ interrupt coming in after allocating our interrupt
> > (e1000_request_irq) will be handled, no matter where it came from.
> 
> we actually generate this LSC interrupt ourselves in the driver, to make sure 
> that we cascade into the watchdog which then enables or disables the link code 
> based on the link status change. This allows us to _not_ do any link checking in 
> _open and makes things a bit more simple.
I am not referring to the LSC interrupt the driver itself generates in
e1000_open just before returning. The ICR is masked (ICR==0) after
executing the driver probe (e1000_probe), but when we enter e1000_open
the E1000_ICR_LSC bit will already be set, before the function even
starts executing. I also observed that when the link is active the line

  /* fire a link status change interrupt to start the watchdog */
  E1000_WRITE_REG(&adapter->hw, ICS, E1000_ICS_LSC);

is redundant because the E1000_ICS_LSC bit is already set. In fact, the
irq handler gets invoked twice in a row with the interrupt cause being a
link status change.

> > The solution I came up with is clearing the ICR before calling
> > request_irq. I have to admit that I am not familiar enough with this
> > driver, so it is quite likely that this is not the right fix. I would
> > appreciate your comments on this.
> 
> Clearing the ICR before requesting an irq might not work - at the same time the 
> device could generate another LSC irq...
Is it not possible to prevent the device from generating interrupts until we call request_irq?

Thank you for your feedback!

  - Fernando

> Of course, we probably should just schedule some delayed work to run our 
> watchdog in e1000_open, but I haven't checked if that actually works.
> 
> 
> Auke
> 
> > Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp>
> > ---
> > 
> > diff -urNp linux-2.6.22-orig/drivers/net/e1000/e1000_main.c linux-2.6.22-pendirq/drivers/net/e1000/e1000_main.c
> > --- linux-2.6.22-orig/drivers/net/e1000/e1000_main.c	2007-07-19 18:18:53.000000000 +0900
> > +++ linux-2.6.22-pendirq/drivers/net/e1000/e1000_main.c	2007-07-25 17:22:54.000000000 +0900
> > @@ -1378,6 +1378,17 @@ e1000_alloc_queues(struct e1000_adapter 
> >  }
> >  
> >  /**
> > + * e1000_clear_interrupts
> > + * @adapter: address of board private structure
> > + *
> > + * Mask interrupts
> > + **/
> > +static void
> > +e1000_clear_interrupts(struct e1000_adapter *adapter) {
> > +	E1000_READ_REG(&adapter->hw, ICR);
> > +}
> > +
> > +/**
> >   * e1000_open - Called when a network interface is made active
> >   * @netdev: network interface device structure
> >   *
> > @@ -1431,6 +1442,9 @@ e1000_open(struct net_device *netdev)
> >  	 * so we have to setup our clean_rx handler before we do so.  */
> >  	e1000_configure(adapter);
> >  
> > +	/* Discard any possible pending interrupts. */
> > +	e1000_clear_interrupts(adapter);
> > +
> >  	err = e1000_request_irq(adapter);
> >  	if (err)
> >  		goto err_req_irq;


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 2/2] Debug handling of early spurious interrupts
  2007-07-20 21:43         ` Andrew Morton
@ 2007-07-30  9:58           ` Fernando Luis Vázquez Cao
  2007-07-30 18:22             ` Andrew Morton
  0 siblings, 1 reply; 13+ messages in thread
From: Fernando Luis Vázquez Cao @ 2007-07-30  9:58 UTC (permalink / raw)
  To: Andrew Morton; +Cc: kexec, linux-kernel

On Fri, 2007-07-20 at 14:43 -0700, Andrew Morton wrote: 
> On Fri, 20 Jul 2007 11:20:43 +0900
> Fernando Luis V__zquez Cao <fernando@oss.ntt.co.jp> wrote:
> 
> > With the advent of kdump it is possible that device drivers receive
> > interrupts generated in the context of a previous kernel. Ideally
> > quiescing the underlying devices should suffice but not all drivers do
> > this, either because it is not possible or because they did not
> > contemplate this case. Thus drivers ought to be able to handle
> > interrupts coming in as soon as the interrupt handler is registered.
> > 
> > Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp>
> > ---
> > 
> > diff -urNp linux-2.6.22-orig/kernel/irq/manage.c linux-2.6.22-pendirq/kernel/irq/manage.c
> > --- linux-2.6.22-orig/kernel/irq/manage.c	2007-07-19 18:18:53.000000000 +0900
> > +++ linux-2.6.22-pendirq/kernel/irq/manage.c	2007-07-19 19:43:41.000000000 +0900
> > @@ -533,21 +533,32 @@ int request_irq(unsigned int irq, irq_ha
> >  
> >  	select_smp_affinity(irq);
> >  
> > -	if (irqflags & IRQF_SHARED) {
> > -		/*
> > -		 * It's a shared IRQ -- the driver ought to be prepared for it
> > -		 * to happen immediately, so let's make sure....
> > -		 * We do this before actually registering it, to make sure that
> > -		 * a 'real' IRQ doesn't run in parallel with our fake
> > -		 */
> > -		if (irqflags & IRQF_DISABLED) {
> > -			unsigned long flags;
> > +	/*
> > +	 * With the advent of kdump it possible that device drivers receive
> > +	 * interrupts generated in the context of a previous kernel. Ideally
> > +	 * quiescing the underlying devices should suffice but not all drivers
> > +	 * do this, either because it is not possible or because they did not
> > +	 * contemplate this case. Thus drivers ought to be able to handle
> > +	 * interrupts coming in as soon as the interrupt handler is registered.
> > +	 *
> > +	 * Besides, if it is a shared IRQ the driver ought to be prepared for
> > +	 * it to happen immediately too.
> > +	 *
> > +	 * We do this before actually registering it, to make sure that
> > +	 * a 'real' IRQ doesn't run in parallel with our fake.
> > +	 */
> > +	if (irqflags & IRQF_DISABLED) {
> > +		unsigned long flags;
> >  
> > -			local_irq_save(flags);
> > -			handler(irq, dev_id);
> > -			local_irq_restore(flags);
> > -		} else
> > -			handler(irq, dev_id);
> > +		local_irq_save(flags);
> > +		retval = handler(irq, dev_id);
> > +		local_irq_restore(flags);
> > +	} else
> > +		retval = handler(irq, dev_id);
> > +	if (retval == IRQ_HANDLED) {
> > +		printk(KERN_WARNING
> > +		       "%s (IRQ %d) handled a spurious interrupt\n",
> > +		       devname, irq);
> >  	}
> >  
> 
> This change means that we'll run the irq handler at request_irq()-time even
> for non-shared interrupts.
> 
> This is a bit of a worry.  See, shared-interrupt handlers are required to
> be able to cope with being called when their device isn't interrupting. 
> But nobody ever said that non-shared interrupt handlers need to be able to
> cope with that.
> 
> Hence these non-shared handlers are within their rights to emit warning
> printks, go BUG or accuse the operator of having busted hardware.
> 
> So bad things might happen because of this change.  And if they do, they
> will take a loooong time to be discovered, because non-shared interrupt
> handlers tend to dwell in crufty old drivers which not many people use.
> 
> So I'm wondering if it would be more prudent to only do all this for shared
> handlers?

I have been testing this patches on all my machines, which spans three
different architectures: i386, x86_64 (EM64T and Opteron), and ia64.

The good news is that nothing catastrophic happened and, in the process,
I managed to find some somewhat buggy interrupt handlers.

I will present a brief summary of my findings here. That said, I have to
admit that these are still preliminary results and have not had time to
dig deep into all the issues. I would like hear from you first.

- Test environment

-- x86_64 [EM64T]
   CPU: Intel(R) Xeon(TM) CPU 3.20GHz (2 cpus)
   SCSI: Adaptec AIC-7902B U320
   IDE: Intel Corporation 82801EB/ER (ICH5/ICH5R) IDE Controller

-- x86_64 [Opteron]
   CPU: AMD Opteron(tm) Processor 240 (2 cpus)
   IDE: Advanced Micro Devices [AMD] AMD-8111 IDE (rev 03)

-- ia64
   CPU: Itanium 2 Madison (2 cpus)
   SCSI: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual
Ultra320 SCSI
   IDE: Intel Corporation 82801DB (ICH4) IDE Controller

-- i386
   CPU: Intel(R) Xeon(TM) CPU 2.40GHz (2 cpus)
   IDE: Intel Corporation 82801EB/ER (ICH5/ICH5R) IDE Controller (rev
02)

- Test results for i386

--i8042 (IRQ 12) handled a spurious interrupt -> i8042
The culprit here was the i8042_aux_test_irq interrupt handler
unconditionally returning IRQ_HANDLED despite being marked as
IRQF_SHARED. I submitted a fix and it is currently sitting in the
'for-linus' branch of Dmitry Torokhov's input tree (Dmitry, Alan,
Vojtech, thank you for reviewing the patch!).

-- eth0 (IRQ 16) handled a spurious interrupt -> e1000
The problem here is that the network card is active before it has been
assigned an interrupt line. This means that if the card detects an event
such as a link status change it will update the status register (ICR) by
setting the corresponding flag, even if request_irq has not been invoked
yet. In such a case, once the interrupt handler is set if a spurious
interrupt comes in it will be handled. The reason is that the handler
checks ICR to determine whether the underlying the device has generated
an interrupt or not, and the flag that indicates a link status change
will happen to be set.

I have sent a patch that clears the ICR before requesting an IRQ, but
according to Auke Kok there is a small window during which the device
could generate another link status change interrupt. I am looking for a
way to prevent the device from reacting to such events before an IRQ has
been allocated, but still do not know if it is possible.

-- rtc (IRQ 8) handled a spurious interrupt
-- parport0 (IRQ 7) handled a spurious interrupt
-- MPU401 UART (IRQ 10) handled a spurious interrupt
I have not looked into these three yet but it seems that in some cases
it is not possible to determine whether an interrupt has actually
occurred.

- Test results for x86_64 [EM64T]

-- i8042 (IRQ 12) handled a spurious interrupt -> i8042
Ditto.
-- eth0 (IRQ 54) handled a spurious interrupt -> e1000
Ditto.
-- rtc (IRQ 8) handled a spurious interrupt
Ditto.

- Test results for x86_64 [Opteron]
-- i8042 (IRQ 12) handled a spurious interrupt -> i8042
Ditto.
-- rtc (IRQ 8) handled a spurious interrupt
Ditto.
-- parport0 (IRQ 7) handled a spurious interrupt
Ditto.

- Test results for ia64
-- eth0 (IRQ 48) handled a spurious interrupt -> e1000
Ditto.

Besides these patches have exposed bugs in other people's machines too.
Valdis Kletnieks discovered a problem in his tpm driver and it seems
that the TPM guys are already looking into it.

- Conclusion

These patches can affect device drivers in several ways and, so far, I
have identified 5 possible scenarios:

  1. Shared handlers that behave as expected and ignore spurious
interrupts.
  2. Shared handlers that unconditionally handle any incoming interrupt.
  3. Shared handlers that cannot distinguish between interrupts that it
needs to handle and interrupts generated by other devices in certain
situations.
  4. Non-shared handlers that behave as expected and ignore spurious
interrupts.
  5. Drivers for legacy devices in which it is not possible to determine
whether a given interrupt has actually come from the underlying device.

Obviously, 1) and 4) are the ideal cases and (probably) the majority of
the drivers fall into this group.

2) means that the driver cannot cohabit harmoniously with other drivers
and it would end up handling any interrupt coming in from that interrupt
line regardless of its origin. i8042 used to be belong to this group,
but the aforementioned patch should have solved it.

Most devices have a interrupt pending bit in some port or an interrupt
cause register (ICR) that can be checked by the driver to determine what
interrupts it needs to handle. The problem is that some devices are
operational before request_irq has been called and react to external
events by updating the interrupt pending bit or the ICR. This means that
after actually registering the interrupt handler, any interrupt coming
in would be handled. 3) refers to such cases. By the way, the e1000
driver seems to be suffering this issue which causes the error message
above.

When it comes to 5) there is not much we can do. Sometimes you simply
cannot tell.

The bottom line is that we have to decide how two tackle 2), 3) and 5).
My gut feeling is that in most cases 2) and 3) constitute a bug, because
if the driver is not fixable then IRQF_SHARED should not have been set
in the first place. Regarding 5), there is no choice, so it seems we
would need to mark such drivers somehow (another possible solution would
be calling the interrupt handler from request_irq only when IRQF_SHARED
is set, because shared handlers should not be affected by this problem).

If we find drivers that are not fixable we should probably consider this
new behaviour on a per-driver basis, as Andrew suggested. This would
probably require passing a new flag into request_irq. Besides, when such
a driver is detected we should emit a warning that it may not work
properly in a kdump kernel.

I would appreciate your comments on this.

- Fernando


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 2/2] Debug handling of early spurious interrupts
  2007-07-30  9:58           ` Fernando Luis Vázquez Cao
@ 2007-07-30 18:22             ` Andrew Morton
  2007-07-31  2:25               ` Fernando Luis Vázquez Cao
  0 siblings, 1 reply; 13+ messages in thread
From: Andrew Morton @ 2007-07-30 18:22 UTC (permalink / raw)
  To: Fernando Luis Vázquez Cao; +Cc: kexec, linux-kernel

On Mon, 30 Jul 2007 18:58:14 +0900
Fernando Luis V__zquez Cao <fernando@oss.ntt.co.jp> wrote:

> > 
> > So bad things might happen because of this change.  And if they do, they
> > will take a loooong time to be discovered, because non-shared interrupt
> > handlers tend to dwell in crufty old drivers which not many people use.
> > 
> > So I'm wondering if it would be more prudent to only do all this for shared
> > handlers?
> 
> I have been testing this patches on all my machines, which spans three
> different architectures: i386, x86_64 (EM64T and Opteron), and ia64.
> 
> The good news is that nothing catastrophic happened and, in the process,
> I managed to find some somewhat buggy interrupt handlers.
> 
> I will present a brief summary of my findings here.

That's quite a lot of breakage for such a small sample of machines.  I do
suspect that if we were to merge this lot into mainline, all sorts of bad
stuff would happen.

otoh, as you point out, pretty much everthing which goes wrong is due to
incorrect or dubious driver behaviour, and there is value in weeding these
things out. 

But the problem with this process is that we're weeding things out at
runtime.  Some drivers don't get used by many people and users of some
architectures (esp embedded) tend to lag kernel.org by a long time.  So it
could be years before all the fallout from this change is finally wrapped
up.

> 
> If we find drivers that are not fixable we should probably consider this
> new behaviour on a per-driver basis, as Andrew suggested.

We haven't found any such yet, have we?

> This would
> probably require passing a new flag into request_irq. Besides, when such
> a driver is detected we should emit a warning that it may not work
> properly in a kdump kernel.
> 
> I would appreciate your comments on this.

Oh well, let's just keep maintaining it in -mm for now, gather more
information so that we can make a more informed decision later on.

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 2/2] Debug handling of early spurious interrupts
  2007-07-30 18:22             ` Andrew Morton
@ 2007-07-31  2:25               ` Fernando Luis Vázquez Cao
  2007-07-31  4:46                 ` Andrew Morton
  0 siblings, 1 reply; 13+ messages in thread
From: Fernando Luis Vázquez Cao @ 2007-07-31  2:25 UTC (permalink / raw)
  To: Andrew Morton; +Cc: kexec, linux-kernel

On Mon, 2007-07-30 at 11:22 -0700, Andrew Morton wrote: 
> On Mon, 30 Jul 2007 18:58:14 +0900
> Fernando Luis V__zquez Cao <fernando@oss.ntt.co.jp> wrote:
> 
> > > 
> > > So bad things might happen because of this change.  And if they do, they
> > > will take a loooong time to be discovered, because non-shared interrupt
> > > handlers tend to dwell in crufty old drivers which not many people use.
> > > 
> > > So I'm wondering if it would be more prudent to only do all this for shared
> > > handlers?
> > 
> > I have been testing this patches on all my machines, which spans three
> > different architectures: i386, x86_64 (EM64T and Opteron), and ia64.
> > 
> > The good news is that nothing catastrophic happened and, in the process,
> > I managed to find some somewhat buggy interrupt handlers.
> > 
> > I will present a brief summary of my findings here.
> 
> That's quite a lot of breakage for such a small sample of machines.  I do
> suspect that if we were to merge this lot into mainline, all sorts of bad
> stuff would happen.
Yup.

> otoh, as you point out, pretty much everthing which goes wrong is due to
> incorrect or dubious driver behaviour, and there is value in weeding these
> things out. 
> 
> But the problem with this process is that we're weeding things out at
> runtime.  Some drivers don't get used by many people and users of some
> architectures (esp embedded) tend to lag kernel.org by a long time.  So it
> could be years before all the fallout from this change is finally wrapped
> up.
Yes, that is a big concern. However, the same embedded people is
starting to use both kexec and kdump, so they may suffer the issues we
are trying to weed out anyway, even if these patches are not applied.
The difference is that with this new functionality it is possible to
catch potential problems relatively easily, because any incorrect
behaviour this may cause will be easily reproducible and, in most cases,
will reveal itself early at boot time.

As things stand now, I guess we will keep seeing occasional crashes and
strange behaviour in kexec-booted kernels, which in some cases will be
due to incorrect handling of spurious interrupts. Besides, such problems
are really difficult to reproduce because, commonly, we would need to
hit an obscure corner case.

> > If we find drivers that are not fixable we should probably consider this
> > new behaviour on a per-driver basis, as Andrew suggested.
> 
> We haven't found any such yet, have we?
Not yet, fortunately.

> > This would
> > probably require passing a new flag into request_irq. Besides, when such
> > a driver is detected we should emit a warning that it may not work
> > properly in a kdump kernel.
> > 
> > I would appreciate your comments on this.
> 
> Oh well, let's just keep maintaining it in -mm for now, gather more
> information so that we can make a more informed decision later on.
Makes sense. I will look into all the issues I have found so far, do
some more testing (I think a new approach is needed to speed up testing
of these issues...), and get back to you.

Thank you.

Fernando


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 2/2] Debug handling of early spurious interrupts
  2007-07-31  2:25               ` Fernando Luis Vázquez Cao
@ 2007-07-31  4:46                 ` Andrew Morton
  0 siblings, 0 replies; 13+ messages in thread
From: Andrew Morton @ 2007-07-31  4:46 UTC (permalink / raw)
  To: Fernando Luis Vázquez Cao; +Cc: kexec, linux-kernel

On Tue, 31 Jul 2007 11:25:26 +0900 Fernando Luis Vázquez Cao <fernando@oss.ntt.co.jp> wrote:

> > runtime.  Some drivers don't get used by many people and users of some
> > architectures (esp embedded) tend to lag kernel.org by a long time.  So it
> > could be years before all the fallout from this change is finally wrapped
> > up.
> Yes, that is a big concern. However, the same embedded people is
> starting to use both kexec and kdump, so they may suffer the issues we
> are trying to weed out anyway, even if these patches are not applied.
> The difference is that with this new functionality it is possible to
> catch potential problems relatively easily, because any incorrect
> behaviour this may cause will be easily reproducible and, in most cases,
> will reveal itself early at boot time.
> 
> As things stand now, I guess we will keep seeing occasional crashes and
> strange behaviour in kexec-booted kernels, which in some cases will be
> due to incorrect handling of spurious interrupts. Besides, such problems
> are really difficult to reproduce because, commonly, we would need to
> hit an obscure corner case.

Please have a think about what we can do to aid this debugging.  For
example, add an uncondtional printk into request_irq() for now which tells
us what irq is being registered - a print_symbol() of the irq_handler_t
would be pretty good, although I suspect there are a lot of interrupt
handlers with the same name..



^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2007-07-31  4:47 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-07-17 10:09 [PATCH RFC] Debug handling of early spurious interrupts Fernando Luis Vázquez Cao
2007-07-18 22:46 ` Andrew Morton
2007-07-20  1:54   ` Fernando Luis Vázquez Cao
2007-07-20  2:02     ` [PATCH 1/2] Remove Kconfig setting CONFIG_DEBUG_SHIRQ Fernando Luis Vázquez Cao
2007-07-20  2:20       ` [PATCH 2/2] Debug handling of early spurious interrupts Fernando Luis Vázquez Cao
2007-07-20 21:43         ` Andrew Morton
2007-07-30  9:58           ` Fernando Luis Vázquez Cao
2007-07-30 18:22             ` Andrew Morton
2007-07-31  2:25               ` Fernando Luis Vázquez Cao
2007-07-31  4:46                 ` Andrew Morton
2007-07-25  9:18         ` [PATCH RFC] e1000: clear ICR before requesting an IRQ line Fernando Luis Vázquez Cao
2007-07-25 15:27           ` Kok, Auke
2007-07-26  1:34             ` Fernando Luis Vázquez Cao

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).