RE: [NOT solved] Kernel 2.6.12 + IO-APIC + uhci_hcd = Trouble

RE: [NOT solved] Kernel 2.6.12 + IO-APIC + uhci_hcd = Trouble

[Protasevich, Natalie]

> >Michel,
> >When you get chance, maybe you could boot the OS that used to work
for you (you mentioned 2.4) and provide the boot trace and
/proc/interrupts for comparison.
> # cat /proc/interrupts - 2.4:
>            CPU0
>   0:      32095    IO-APIC-edge  timer
>   1:        968    IO-APIC-edge  keyboard
>   2:          0          XT-PIC  cascade
>   4:        890    IO-APIC-edge  serial
>   7:          2    IO-APIC-edge  parport0
>   8:          1    IO-APIC-edge  rtc
>  14:         10    IO-APIC-edge  ide4
>  15:         42    IO-APIC-edge  ide5
>  18:       1714   IO-APIC-level  eth0, eth1
>  19:      13108   IO-APIC-level  ide0, ide1, ide2, ide3, ehci_hcd
>  21:        751   IO-APIC-level  usb-uhci, usb-uhci, usb-uhci
>  22:          0   IO-APIC-level  VIA8233
> NMI:          0
> LOC:      32049
> ERR:          0
> MIS:         33
> 

One difference between the above and the 2.6 one you sent before is that
you don't seem to have rtc employed on 2.6:
           CPU0
  0:     626375    IO-APIC-edge  timer
  1:       1599    IO-APIC-edge  i8042
  2:          0          XT-PIC  cascade
  4:       1708    IO-APIC-edge  serial
  7:          2    IO-APIC-edge  parport0
 14:      19858    IO-APIC-edge  ide2
 15:       5220    IO-APIC-edge  ide3
 16:      30711   IO-APIC-level  nvidia
 18:       1799   IO-APIC-level  eth0, eth1
 19:     103365   IO-APIC-level  ide0, ide1, ehci_hcd:usb4, aic7xxx
 21:      47273   IO-APIC-level  uhci_hcd:usb1, uhci_hcd:usb2,
uhci_hcd:usb3
 22:       2782   IO-APIC-level  VIA8233

I don't expect this to be of any significance, but as Alan said you
never know... Another thing is that you are getting large number of
interrupts on the VIA device, whereas there is no any on 2.4. Does the
chipset get enabled differently? I wish I new VIA chipset, or had it,
will probably have to finally get some documentation on it. 

> And what should be relevant to USB in the boot log...:
> 
> usb.c: registered new driver usbdevfs
> usb.c: registered new driver hub
> usb-uhci.c: $Revision: 1.275 $ time 11:23:26 Jan  2 2005
> usb-uhci.c: High bandwidth mode enabled
> usb-uhci.c: USB UHCI at I/O 0xcc00, IRQ 21
> usb-uhci.c: Detected 2 ports
> usb.c: new USB bus registered, assigned bus number 1
> hub.c: USB hub found
> hub.c: 2 ports detected
> usb-uhci.c: USB UHCI at I/O 0xd000, IRQ 21
> usb-uhci.c: Detected 2 ports
> usb.c: new USB bus registered, assigned bus number 2
> hub.c: USB hub found
> hub.c: 2 ports detected
> usb-uhci.c: USB UHCI at I/O 0xd400, IRQ 21
> usb-uhci.c: Detected 2 ports

Here I see that no fixups were applied to the chipset, and it seemed to
work just fine without them. I would experiment and turn fixups off, but
since this is a production system... Hmmm. 

Your previous trace for 2.6 with fixups:

USB Universal Host Controller Interface driver v2.2
PCI: Via IRQ fixup for 0000:00:10.0, from 10 to 5 uhci_hcd 0000:00:10.0:
VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller uhci_hcd
0000:00:10.0: new USB bus registered, assigned bus number 1 uhci_hcd
0000:00:10.0: irq 21, io base 0x0000cc00 hub 1-0:1.0: USB hub found hub
1-0:1.0: 2 ports detected
PCI: Via IRQ fixup for 0000:00:10.1, from 10 to 5 uhci_hcd 0000:00:10.1:
VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (#2) uhci_hcd
0000:00:10.1: new USB bus registered, assigned bus number 2 uhci_hcd
0000:00:10.1: irq 21, io base 0x0000d000 hub 2-0:1.0: USB hub found hub
2-0:1.0: 2 ports detected
PCI: Via IRQ fixup for 0000:00:10.2, from 10 to 5 uhci_hcd 0000:00:10.2:
VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (#3) usb 1-1:
new low speed USB device using uhci_hcd and address 2 uhci_hcd
0000:00:10.2: new USB bus registered, assigned bus number 3 uhci_hcd
0000:00:10.2: irq 21, io base 0x0000d400 hub 3-0:1.0: USB hub found hub
3-0:1.0: 2 ports detected
PCI: Via IRQ fixup for 0000:00:10.3, from 11 to 3 ehci_hcd 0000:00:10.3:
VIA Technologies, Inc. USB 2.0 ehci_hcd 0000:00:10.3: new USB bus
registered, assigned bus number 4 ehci_hcd 0000:00:10.3: irq 19, io mem
0xe3009000 ehci_hcd 0000:00:10.3: USB 2.0 initialized, EHCI 1.00, driver
10 Dec 2004
usbcore: registered new driver hiddev
hub 4-0:1.0: USB hub found
hub 4-0:1.0: 6 ports detected
usbhid: probe of 1-1:1.0 failed with error -5
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.01:USB HID core driver irq 21: nobody
cared!

Seeing actual IO-APIC setup in both cases would help, the ones you get
with apic=verbose, and you might have to provide full traces (as
attachment for example). It is somewhat comforting for me to know that
my patch is not affecting the outcome. But it is important to crack this
case of course. I think we need higher authority here, such as Bjorn, or
Alan...

Cheers,
--Natalie
> --
> Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xDDE8AC6E
> 

Re: [NOT solved] Kernel 2.6.12 + IO-APIC + uhci_hcd = Trouble

[Michel Bouissou]

Le Mardi 12 Juillet 2005 01:41, Protasevich, Natalie a écrit :
>
> Another thing is that you are getting large number of
> interrupts on the VIA device, whereas there is no any on 2.4. Does the
> chipset get enabled differently?

I believe this interruption is for the sound module. The /proc/interrupts I 
sent you for the 2.6 kernel had been taken with a system that had been 
running X11 (with nice sounds ;-) for a while, where the /proc/interrupts I 
sent you for the 2.4 kernel had been taken just after having booted in 
runlevel 3 for a short while, without having produced any sound. This might 
explain this difference.

> > And what should be relevant to USB in the boot log...:
[...]
> Here I see that no fixups were applied to the chipset, and it seemed to
> work just fine without them.

I'm not sure that the 2.4 series kernel logged the IRQ fixups the same way 2.6 
does... I'm no kernel specialist.
But I remember that long ago (with early 2.4 kernels) I had had interrupts 
problems with the VIA chipset, until some specific VIA fixes were integrated 
into the 2.4 kernel, and since this day everything was OK -- until I tried 
2.6 kernels...

> I would experiment and turn fixups off, but 
> since this is a production system... Hmmm.

Yes, and I'm really afraid of breaking my storage, as the EVMS snapshots I use 
to perform backups seems to be severely broke with the 2.6 kernel and my 
system -- which means I don't have recent and reliable backups until this 
gets fixed, but this is another issue.

> Seeing actual IO-APIC setup in both cases would help, the ones you get
> with apic=verbose, and you might have to provide full traces (as
> attachment for example). It is somewhat comforting for me to know that
> my patch is not affecting the outcome. But it is important to crack this
> case of course. I think we need higher authority here, such as Bjorn, or
> Alan...

Please tell me how you would like me to boot ;-)

Another little issue is that, with kernel 2.4, the activation of EVMS and RAID 
sets from my initrd produces very verbose output, so verbose that the very 
beginning of the dmesg messages gets lost before true loggin is started and 
they have a chance to get copied to /var/log/messages... It has never been an 
issue to me as 2.4 is running fine... But if you want my 2.4 kernel to 
produce yet more verbose output, it might be difficult ;-)

I don't have this problem in 2.6 as the EVMS/RAID startup is far less verbose.

Cheers.

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xDDE8AC6E

RE: [NOT solved] Kernel 2.6.12 + IO-APIC + uhci_hcd = Trouble

[Protasevich, Natalie]

> Le Lundi 11 Juillet 2005 22:58, Michel Bouissou a écrit :
> >
> > Oh no :-(
> 
> Well, I give up for tonight :-(
> 
> This time I rebooted with the mouse disabled in BIOS, with 
> the usb-handoff option, with the scanner unplugged... And it 
> went wrong simply by itself. 
> "irq 21: nobody cared!"
> 
> The only thing I'm sure about is that there is something 
> either with UP IO-APIC support, or with the uhci_hcd module.
> 
> When both are combined, as you can see, this is completely 
> unstable. One time it works, one time it doesn't.
> But if I use a kernel compiled without UP IO-APIC, or if I 
> boot an IO-APIC-capable kernel with the "noapic" option, then 
> the problem is gone, and it is stable (really, this time).
> 
> But of course, I don't have no IO-APIC anymore...
> 
> [root@totor etc]# cat /proc/interrupts
>            CPU0
>   0:     423482          XT-PIC  timer
>   1:       1083          XT-PIC  i8042
>   2:          0          XT-PIC  cascade
>   4:       1106          XT-PIC  serial
>   7:       1543          XT-PIC  parport0
>  10:       3527          XT-PIC  uhci_hcd:usb3, eth0, eth1, VIA8233
>  11:      24934          XT-PIC  ide0, ide1, ide2, ide3, 
> uhci_hcd:usb2, 
> ehci_hcd:usb4
>  12:      13809          XT-PIC  uhci_hcd:usb1, nvidia
>  14:       3245          XT-PIC  ide4
>  15:       3254          XT-PIC  ide5
> NMI:          0
> LOC:     423403
> ERR:        270
> 
> Cheers.

Michel,
When you get chance, maybe you could boot the OS that used to work for you (you mentioned 2.4) and provide the boot trace and /proc/interrupts for comparison.
Thanks,
--Natalie

Re: [NOT solved] Kernel 2.6.12 + IO-APIC + uhci_hcd = Trouble

[Michel Bouissou]

Le Lundi 11 Juillet 2005 23:25, Protasevich, Natalie a écrit :
> When you get chance, maybe you could boot the OS that used to work for you
> (you mentioned 2.4) and provide the boot trace and /proc/interrupts for
> comparison.

Here it goes. Kernel is a somewhat custom ;-) 2.4.28:

# uname -r
2.4.28-0.rc1.1.1MiBevmsmdkc5

# cat /proc/interrupts
           CPU0
  0:      32095    IO-APIC-edge  timer
  1:        968    IO-APIC-edge  keyboard
  2:          0          XT-PIC  cascade
  4:        890    IO-APIC-edge  serial
  7:          2    IO-APIC-edge  parport0
  8:          1    IO-APIC-edge  rtc
 14:         10    IO-APIC-edge  ide4
 15:         42    IO-APIC-edge  ide5
 18:       1714   IO-APIC-level  eth0, eth1
 19:      13108   IO-APIC-level  ide0, ide1, ide2, ide3, ehci_hcd
 21:        751   IO-APIC-level  usb-uhci, usb-uhci, usb-uhci
 22:          0   IO-APIC-level  VIA8233
NMI:          0
LOC:      32049
ERR:          0
MIS:         33


And what should be relevant to USB in the boot log...:

usb.c: registered new driver usbdevfs
usb.c: registered new driver hub
usb-uhci.c: $Revision: 1.275 $ time 11:23:26 Jan  2 2005
usb-uhci.c: High bandwidth mode enabled
usb-uhci.c: USB UHCI at I/O 0xcc00, IRQ 21
usb-uhci.c: Detected 2 ports
usb.c: new USB bus registered, assigned bus number 1
hub.c: USB hub found
hub.c: 2 ports detected
usb-uhci.c: USB UHCI at I/O 0xd000, IRQ 21
usb-uhci.c: Detected 2 ports
usb.c: new USB bus registered, assigned bus number 2
hub.c: USB hub found
hub.c: 2 ports detected
usb-uhci.c: USB UHCI at I/O 0xd400, IRQ 21
usb-uhci.c: Detected 2 ports
usb.c: new USB bus registered, assigned bus number 3
hub.c: USB hub found
hub.c: 2 ports detected
usb-uhci.c: v1.275:USB Universal Host Controller Interface driver
ehci_hcd 00:10.3: VIA Technologies, Inc. USB 2.0
ehci_hcd 00:10.3: irq 19, pci mem f8b32000
usb.c: new USB bus registered, assigned bus number 4
ehci_hcd 00:10.3: USB 2.0 enabled, EHCI 1.00, driver 2003-Dec-29/2.4
hub.c: USB hub found
hub.c: 6 ports detected
usbdevfs: remount parameter error
usb.c: registered new driver hiddev
usb.c: registered new driver hid
hid-core.c: v1.8.1 Andreas Gal, Vojtech Pavlik <vojtech@suse.cz>
hid-core.c: USB HID support drivers
mice: PS/2 mouse device common for all mice
[...]
hub.c: connect-debounce failed, port 1 disabled
[...]
hub.c: new USB device 00:10.2-2, assigned address 2
usb.c: USB device 2 (vend/prod 0x5d8/0x4002) is not claimed by any active 
driver.
hub.c: new USB device 00:10.0-1, assigned address 2
input: USB HID v1.10 Mouse [Logitech USB Receiver] on usb1:2.0
usb.c: registered new driver usbscanner
scanner.c: USB scanner device (0x05d8/0x4002) now attached to scanner0
scanner.c: 0.4.16:USB Scanner Driver
usb.c: registered new driver usbmouse
usbmouse.c: v1.6:USB HID Boot Protocol mouse driver

Hope this may help...?

Cheers.

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xDDE8AC6E

Re: [SOLVED ??] Kernel 2.6.12 + IO-APIC + uhci_hcd = Trouble

[Alan Stern]

On Mon, 11 Jul 2005, Michel Bouissou wrote:

> Le Lundi 11 Juillet 2005 21:43, Alan Stern a écrit :
> > >
> > > Enable USB mouse support: YES	(Well, I have one ;-)
> >
> > That's what I was talking about.  BIOS support for keyboard and mouse is
> > called "Legacy" support, because it emulates plain old non-USB AT-type
> > devices.  I bet if you turned off the "Enable USB mouse support" option
> > then everything would work.
> 
> Aha. And what would be your advice ? Rather leave the BIOS mouse option ON and 
> use "usb-handoff", or remove both ?

Well, some people don't have a choice.  They need to leave BIOS USB 
keyboard/mouse support turned on in order to use their USB keyboard/mouse 
before the Linux kernel has loaded (within Grub, for instance).

If you don't care about that, then it's cleaner to turn off the BIOS 
support.  Of course, you may find that you still need to use 
"usb-handoff" anyway!

In the end it doesn't make much difference.  Do whatever works best for 
you.

Alan Stern

Re: [SOLVED ??] Kernel 2.6.12 + IO-APIC + uhci_hcd = Trouble

[Michel Bouissou]

Le Lundi 11 Juillet 2005 22:16, Alan Stern a écrit :
> > Aha. And what would be your advice ? Rather leave the BIOS mouse option
> > ON and use "usb-handoff", or remove both ?
>
> Well, some people don't have a choice.  They need to leave BIOS USB
> keyboard/mouse support turned on in order to use their USB keyboard/mouse
> before the Linux kernel has loaded (within Grub, for instance).
>
> If you don't care about that, then it's cleaner to turn off the BIOS
> support.  Of course, you may find that you still need to use
> "usb-handoff" anyway!

Well, I'm afraid I spoke too fast :-(

I rebooted my system with "usb-handoff" again, but still the USB mouse enabled 
in BIOS, and this time got the punishent again :-(

usbcore: registered new driver usbfs
usbcore: registered new driver hub
USB Universal Host Controller Interface driver v2.2
PCI: Via IRQ fixup for 0000:00:10.0, from 10 to 5
uhci_hcd 0000:00:10.0: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 
Controller
uhci_hcd 0000:00:10.0: new USB bus registered, assigned bus number 1
uhci_hcd 0000:00:10.0: irq 21, io base 0x0000cc00
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 2 ports detected
PCI: Via IRQ fixup for 0000:00:10.1, from 10 to 5
uhci_hcd 0000:00:10.1: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 
Controller (#2)
uhci_hcd 0000:00:10.1: new USB bus registered, assigned bus number 2
uhci_hcd 0000:00:10.1: irq 21, io base 0x0000d000
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 2 ports detected
PCI: Via IRQ fixup for 0000:00:10.2, from 10 to 5
uhci_hcd 0000:00:10.2: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 
Controller (#3)
usb 1-1: new low speed USB device using uhci_hcd and address 2
uhci_hcd 0000:00:10.2: new USB bus registered, assigned bus number 3
uhci_hcd 0000:00:10.2: irq 21, io base 0x0000d400
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 2 ports detected
PCI: Via IRQ fixup for 0000:00:10.3, from 11 to 3
ehci_hcd 0000:00:10.3: VIA Technologies, Inc. USB 2.0
ehci_hcd 0000:00:10.3: new USB bus registered, assigned bus number 4
ehci_hcd 0000:00:10.3: irq 19, io mem 0xe3009000
ehci_hcd 0000:00:10.3: USB 2.0 initialized, EHCI 1.00, driver 10 Dec 2004
usbcore: registered new driver hiddev
hub 4-0:1.0: USB hub found
hub 4-0:1.0: 6 ports detected
usbhid: probe of 1-1:1.0 failed with error -5
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.01:USB HID core driver
irq 21: nobody cared!

...etc :-(

Well, this time, I deactivated the mouse in BIOS, rebooted again, also with 
"usb-handoff", and "again it works"...

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xDDE8AC6E

Re: [SOLVED ??] Kernel 2.6.12 + IO-APIC + uhci_hcd = Trouble

[Michel Bouissou]

Le Lundi 11 Juillet 2005 22:46, Michel Bouissou a écrit :
>
> Well, this time, I deactivated the mouse in BIOS, rebooted again, also with
> "usb-handoff", and "again it works"...

...at least, it worked until I unplugged my USB scanner...

Jul 11 22:52:54 totor kernel: usb 3-2: USB disconnect, address 2

...then replugged my USB scanner...

Jul 11 22:53:08 totor kernel: usb 3-2: new full speed USB device using 
uhci_hcd and address 3
Jul 11 22:53:08 totor kernel: irq 21: nobody cared!

Oh no :-(

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xDDE8AC6E

Re: [NOT solved] Kernel 2.6.12 + IO-APIC + uhci_hcd = Trouble

[Michel Bouissou]

Le Lundi 11 Juillet 2005 22:58, Michel Bouissou a écrit :
>
> Oh no :-(

Well, I give up for tonight :-(

This time I rebooted with the mouse disabled in BIOS, with the usb-handoff 
option, with the scanner unplugged... And it went wrong simply by itself. 
"irq 21: nobody cared!"

The only thing I'm sure about is that there is something either with UP 
IO-APIC support, or with the uhci_hcd module.

When both are combined, as you can see, this is completely unstable. One time 
it works, one time it doesn't.
But if I use a kernel compiled without UP IO-APIC, or if I boot an 
IO-APIC-capable kernel with the "noapic" option, then the problem is gone, 
and it is stable (really, this time).

But of course, I don't have no IO-APIC anymore...

[root@totor etc]# cat /proc/interrupts
           CPU0
  0:     423482          XT-PIC  timer
  1:       1083          XT-PIC  i8042
  2:          0          XT-PIC  cascade
  4:       1106          XT-PIC  serial
  7:       1543          XT-PIC  parport0
 10:       3527          XT-PIC  uhci_hcd:usb3, eth0, eth1, VIA8233
 11:      24934          XT-PIC  ide0, ide1, ide2, ide3, uhci_hcd:usb2, 
ehci_hcd:usb4
 12:      13809          XT-PIC  uhci_hcd:usb1, nvidia
 14:       3245          XT-PIC  ide4
 15:       3254          XT-PIC  ide5
NMI:          0
LOC:     423403
ERR:        270

Cheers.

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xDDE8AC6E