SMP+irq handling broken in current git?

SMP+irq handling broken in current git?

[Jens Axboe]

Hi,

Something strange is going on with current git as of this morning (head
ee98689be1b054897ff17655008c3048fe88be94). On an old test box (dual p3
800MHz), using the same old config I always do on this box has very
broken interrupt handling:

[...]

Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with
idebus=xx
PIIX4: IDE controller at PCI slot 0000:00:04.1
PIIX4: chipset revision 1
PIIX4: not 100% native mode: will probe irqs later
    ide0: BM-DMA at 0xd800-0xd807, BIOS settings: hda:DMA, hdb:DMA
    ide1: BM-DMA at 0xd808-0xd80f, BIOS settings: hdc:DMA, hdd:pio
hda: QUANTUM FIREBALLP AS40.0, ATA DISK drive
hdb: IOMEGA DVDRW8824E2Q-D, ATAPI CD/DVD-ROM drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
hdc: IBM-DTLA-307030, ATA DISK drive
ide1 at 0x170-0x177,0x376 on irq 15
irq 14: nobody cared!
 [<c01039e7>] dump_stack+0x17/0x20
 [<c0134ec7>] __report_bad_irq+0x27/0x90
 [<c0134fd2>] note_interrupt+0x72/0x90
 [<c01349cb>] __do_IRQ+0x11b/0x130
 [<c0104f9c>] do_IRQ+0x1c/0x30
 [<c01034ca>] common_interrupt+0x1a/0x20
 [<c0100a9c>] cpu_idle+0x6c/0x80
 [<c040c97a>] start_kernel+0x14a/0x170
 [<c010020e>] 0xc010020e
handlers:
[<c02d12b0>] (ide_intr+0x0/0x150)
Disabling IRQ #14
irq 15: nobody cared!
 [<c01039e7>] dump_stack+0x17/0x20
 [<c0134ec7>] __report_bad_irq+0x27/0x90
 [<c0134fd2>] note_interrupt+0x72/0x90
 [<c01349cb>] __do_IRQ+0x11b/0x130
 [<c0104f9c>] do_IRQ+0x1c/0x30
 [<c01034ca>] common_interrupt+0x1a/0x20
 [<c0100a9c>] cpu_idle+0x6c/0x80
 [<c040c97a>] start_kernel+0x14a/0x170
 [<c010020e>] 0xc010020e
handlers:
[<c02d12b0>] (ide_intr+0x0/0x150)
Disabling IRQ #15
hda: max request size: 128KiB
hda: lost interrupt
hda: lost interrupt
hda: 78177792 sectors (40027 MB) w/1902KiB Cache, CHS=65535/16/63,
UDMA(33)
hda: cache flushes not supported
 hda:<4>hda: dma_timer_expiry: dma status == 0x64
hda: DMA interrupt recovery
hda: lost interrupt
 hda1 hda2 hda3 hda4
hdc: max request size: 128KiB
irq 14: nobody cared!
 [<c01039e7>] dump_stack+0x17/0x20
 [<c0134ec7>] __report_bad_irq+0x27/0x90
 [<c0134fd2>] note_interrupt+0x72/0x90
 [<c01349cb>] __do_IRQ+0x11b/0x130
 [<c0104f9c>] do_IRQ+0x1c/0x30
 [<c01034ca>] common_interrupt+0x1a/0x20
 [<c0100a9c>] cpu_idle+0x6c/0x80
 [<c040c97a>] start_kernel+0x14a/0x170
 [<c010020e>] 0xc010020e
handlers:
[<c02d12b0>] (ide_intr+0x0/0x150)
Disabling IRQ #14
hdc: lost interrupt

Going to UP makes it work just fine for me. Does anyone know what is
going on here?!

-- 
Jens Axboe

Re: SMP+irq handling broken in current git?

[Jeff Garzik]

Jens Axboe wrote:
> Hi,
> 
> Something strange is going on with current git as of this morning (head
> ee98689be1b054897ff17655008c3048fe88be94). On an old test box (dual p3
> 800MHz), using the same old config I always do on this box has very
> broken interrupt handling:

Does 2.6.12 work for you?
2.6.11?

I noticed a few "2.6.12 is broken, 2.6.11 works" bug reports with 
vaguely similar circumstances -- irq handling being a culprit.

	Jeff

Re: SMP+irq handling broken in current git?

[Jens Axboe]

On Thu, Jun 23 2005, Jeff Garzik wrote:
> Jens Axboe wrote:
> >Hi,
> >
> >Something strange is going on with current git as of this morning (head
> >ee98689be1b054897ff17655008c3048fe88be94). On an old test box (dual p3
> >800MHz), using the same old config I always do on this box has very
> >broken interrupt handling:
> 
> Does 2.6.12 work for you?
> 2.6.11?

2.6.11 works, 2.6.12 does not.


> I noticed a few "2.6.12 is broken, 2.6.11 works" bug reports with 
> vaguely similar circumstances -- irq handling being a culprit.

One boot kept looping with screaming floppy interrupts, so something has
gone really bad there.

-- 
Jens Axboe

Re: SMP+irq handling broken in current git?

[Bjorn Helgaas]

[-- Attachment #1: Type: text/plain, Size: 620 bytes --]

On Thursday 23 June 2005 12:42 pm, Jens Axboe wrote:
> On Thu, Jun 23 2005, Jeff Garzik wrote:
> > Jens Axboe wrote:
> > >Hi,
> > >
> > >Something strange is going on with current git as of this morning (head
> > >ee98689be1b054897ff17655008c3048fe88be94). On an old test box (dual p3
> > >800MHz), using the same old config I always do on this box has very
> > >broken interrupt handling:
> > 
> > Does 2.6.12 work for you?
> > 2.6.11?
> 
> 2.6.11 works, 2.6.12 does not.

Do you have any VIA devices?  If so, you might try the attached.
(Just for debugging; if the patch helps, I have no idea how to
do it correctly.)

[-- Attachment #2: via-irq --]
[-- Type: text/x-diff, Size: 528 bytes --]

Index: work/drivers/pci/quirks.c
===================================================================
--- work.orig/drivers/pci/quirks.c	2005-06-21 13:43:29.000000000 -0600
+++ work/drivers/pci/quirks.c	2005-06-23 10:40:55.000000000 -0600
@@ -510,7 +510,7 @@
 		pci_write_config_byte(dev, PCI_INTERRUPT_LINE, new_irq);
 	}
 }
-DECLARE_PCI_FIXUP_ENABLE(PCI_VENDOR_ID_VIA, PCI_ANY_ID, quirk_via_irq);
+DECLARE_PCI_FIXUP_ENABLE(PCI_ANY_ID, PCI_ANY_ID, quirk_via_irq);
 
 /*
  * PIIX3 USB: We have to disable USB interrupts that are

Re: SMP+irq handling broken in current git?

[Jens Axboe]

On Thu, Jun 23 2005, Bjorn Helgaas wrote:
> On Thursday 23 June 2005 12:42 pm, Jens Axboe wrote:
> > On Thu, Jun 23 2005, Jeff Garzik wrote:
> > > Jens Axboe wrote:
> > > >Hi,
> > > >
> > > >Something strange is going on with current git as of this morning (head
> > > >ee98689be1b054897ff17655008c3048fe88be94). On an old test box (dual p3
> > > >800MHz), using the same old config I always do on this box has very
> > > >broken interrupt handling:
> > > 
> > > Does 2.6.12 work for you?
> > > 2.6.11?
> > 
> > 2.6.11 works, 2.6.12 does not.
> 
> Do you have any VIA devices?  If so, you might try the attached.
> (Just for debugging; if the patch helps, I have no idea how to
> do it correctly.)

No VIA devices, it's an intel board with intel chipset. Do you still
want me to test it?


> Index: work/drivers/pci/quirks.c
> ===================================================================
> --- work.orig/drivers/pci/quirks.c	2005-06-21 13:43:29.000000000 -0600
> +++ work/drivers/pci/quirks.c	2005-06-23 10:40:55.000000000 -0600
> @@ -510,7 +510,7 @@
>  		pci_write_config_byte(dev, PCI_INTERRUPT_LINE, new_irq);
>  	}
>  }
> -DECLARE_PCI_FIXUP_ENABLE(PCI_VENDOR_ID_VIA, PCI_ANY_ID, quirk_via_irq);
> +DECLARE_PCI_FIXUP_ENABLE(PCI_ANY_ID, PCI_ANY_ID, quirk_via_irq);
>  
>  /*
>   * PIIX3 USB: We have to disable USB interrupts that are


-- 
Jens Axboe

Re: SMP+irq handling broken in current git?

[Bjorn Helgaas]

On Thursday 23 June 2005 1:11 pm, Jens Axboe wrote:
> On Thu, Jun 23 2005, Bjorn Helgaas wrote:
> > On Thursday 23 June 2005 12:42 pm, Jens Axboe wrote:
> > > On Thu, Jun 23 2005, Jeff Garzik wrote:
> > > > Jens Axboe wrote:
> > > > >Hi,
> > > > >
> > > > >Something strange is going on with current git as of this morning (head
> > > > >ee98689be1b054897ff17655008c3048fe88be94). On an old test box (dual p3
> > > > >800MHz), using the same old config I always do on this box has very
> > > > >broken interrupt handling:
> > > > 
> > > > Does 2.6.12 work for you?
> > > > 2.6.11?
> > > 
> > > 2.6.11 works, 2.6.12 does not.
> > 
> > Do you have any VIA devices?  If so, you might try the attached.
> > (Just for debugging; if the patch helps, I have no idea how to
> > do it correctly.)
> 
> No VIA devices, it's an intel board with intel chipset. Do you still
> want me to test it?

Nope, I don't think it will make any difference then.

Re: SMP+irq handling broken in current git?

[Alan Cox]

On Iau, 2005-06-23 at 19:58, Bjorn Helgaas wrote:
> Do you have any VIA devices?  If so, you might try the attached.
> (Just for debugging; if the patch helps, I have no idea how to
> do it correctly.)

The VIA routing is documented in the bridge chipset docs. The older ones
are/were on the ftp site. The newer ones are available under NDA or by
asking nicely and having a reason.

Re: SMP+irq handling broken in current git?

[Jens Axboe]

On Thu, Jun 23 2005, Jeff Garzik wrote:
> Jens Axboe wrote:
> >Hi,
> >
> >Something strange is going on with current git as of this morning (head
> >ee98689be1b054897ff17655008c3048fe88be94). On an old test box (dual p3
> >800MHz), using the same old config I always do on this box has very
> >broken interrupt handling:
> 
> Does 2.6.12 work for you?
> 2.6.11?
> 
> I noticed a few "2.6.12 is broken, 2.6.11 works" bug reports with 
> vaguely similar circumstances -- irq handling being a culprit.

To follow up on this on the list, the culprit appears to be the attached
string.h update introduced between -rc3 and -rc4. Backing it out makes
the kernel work, making it clobber memory (as suggested by Linus) makes
it work as well.

The failed test was in mpparse.c, it didn't recognise the pci bus.

-- 
Jens Axboe