* RE: [patch] PCI: disable MSI on more ATI NorthBridges
@ 2007-10-19 13:17 Shane Huang
[not found] ` <9783dadb0710190706i499b4edbh9736019b0384f9b1@mail.gmail.com>
2007-10-19 19:57 ` Linas Vepstas
0 siblings, 2 replies; 31+ messages in thread
From: Shane Huang @ 2007-10-19 13:17 UTC (permalink / raw)
To: davem, gregkh, htejun, brice.goglin, david.gaarenstroom
Cc: linux-kernel, linux-pci, shane.huang, chunhao.huang
Hi Guys:
> This logic seems backwards, to me. "shoot first, ask questions later"
> To me this it not how to approach this problem.
>
> Once you turn MSI off, there is next to no incentive to fix the
> problem because users aren't running into it any longer.
>
> The only two devices in that bug report which should be using MSI
> would be the SATA controller and the broadcom ethernet NIC. And by
> the failed bootup logs provided by the user the problem is clearly
> with the SATA controller.
>
> One common problem we're finding is that some devices have a hardware
> bug where setting INTX_DISABLE in the PCI COMMAND register masks MSI
> interrupts too.
>
> I mention this because the user in that report mentions that the
> kernel upgrade causes the failure, and one thing we started doing not
> too long ago was to set the INTX_DISABLE bit when MSI is enabled for a
> device.
Yes, you are right, to find out the root cause is better. Thank you
for all your suggestion and information to us.
Since we have little experience on PCI and MSI here, we had to try to
disable MSI before we find a better solution. But as you are giving
us help now on this case, it's great! Then let's go...
> So maybe this SATA controller has this problem too. It is easy to
> test, simply comment out all of the pci_intx() function calls in
> drivers/pci/msi.c and perform a test boot with MSI enabled.
>
> I would rather you approach analysis of these kinds of MSI bugs in
> this manner, instead of disabling MSI wholesale. Because with the
> latter approach it is nearly guarenteed that the real reason will only
> be discovered with an extremely low priority.
I'm using kernel 2.6.23-rc5 to debug this MSI problem, which can NOT
boot on our Trevally board(RS690+SB700) without any kernel modification.
But if I comment out all the pci_intx() function calls in
drivers/pci/msi.c, it can boot now with MSI enabled as you expected!
# cat /proc/interrupts
CPU0 CPU1
0: 318 174060 IO-APIC-edge timer
8: 0 1 IO-APIC-edge rtc
9: 0 0 IO-APIC-fasteoi acpi
16: 0 204 IO-APIC-fasteoi HDA Intel
17: 0 479 IO-APIC-fasteoi ohci_hcd:usb1, ohci_hcd:usb2, ehci_hcd:usb6
18: 1 2 IO-APIC-fasteoi ohci_hcd:usb3, ohci_hcd:usb4, ohci_hcd:usb5
19: 0 0 IO-APIC-fasteoi ehci_hcd:usb7
22: 4 1 IO-APIC-fasteoi yenta
8412: 0 1315 PCI-MSI-edge eth0
8413: 381 4858 PCI-MSI-edge ahci
NMI: 0 0
LOC: 174285 174210
ERR: 0
Also if I keep the pci_intx() calls in drivers/pci/msi.c and ONLY
comment out the pci_intx() call in drivers/ata/ahci.c
My system can boot up too with MSI enabled!
So does it mean that the root cause is our SB700 SATA controller
has a hardware bug where setting INTX_DISABLE in the PCI COMMAND
register masks MSI interrupts too?
And what is the software solution or workaround?
I will continue debug this MSI problem next week. Any suggestions,
please don't hesitate to tell us.
Thanks
Best Regards
Shane
_________________________________________________________________
News, entertainment and everything you care about at Live.com. Get it now!
http://www.live.com/getstarted.aspx
^ permalink raw reply [flat|nested] 31+ messages in thread
* Fwd: [patch] PCI: disable MSI on more ATI NorthBridges
[not found] ` <9783dadb0710190706i499b4edbh9736019b0384f9b1@mail.gmail.com>
@ 2007-10-19 14:10 ` David Gaarenstroom
0 siblings, 0 replies; 31+ messages in thread
From: David Gaarenstroom @ 2007-10-19 14:10 UTC (permalink / raw)
To: linux-kernel
2007/10/19, Shane Huang <chunhao.huang@hotmail.com>:
>
> Since we have little experience on PCI and MSI here,
"Here" as in "here at AMD"?! I'm stunned...
Both AMD and the former ATI should have quite some experience?!
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-19 13:17 [patch] PCI: disable MSI on more ATI NorthBridges Shane Huang
[not found] ` <9783dadb0710190706i499b4edbh9736019b0384f9b1@mail.gmail.com>
@ 2007-10-19 19:57 ` Linas Vepstas
2007-10-19 20:21 ` Jeff Garzik
2007-10-20 14:50 ` Shane Huang
1 sibling, 2 replies; 31+ messages in thread
From: Linas Vepstas @ 2007-10-19 19:57 UTC (permalink / raw)
To: Shane Huang
Cc: davem, gregkh, htejun, brice.goglin, david.gaarenstroom,
linux-kernel, linux-pci, shane.huang, jgarzik, linux-ide
On Fri, Oct 19, 2007 at 09:17:23PM +0800, Shane Huang wrote:
> Since we have little experience on PCI and MSI here, we had to try to
As someone else pointed out, AMD should have *lots* of people with
pci and msi experience on the payroll. (Folks here buy AMD-designed
pci chips ...)
> ONLY
> comment out the pci_intx() call in drivers/ata/ahci.c
> My system can boot up too with MSI enabled!
>
> So does it mean that the root cause is our SB700 SATA controller
> has a hardware bug where setting INTX_DISABLE in the PCI COMMAND
> register masks MSI interrupts too?
That's what it sounds like, to me.
> And what is the software solution or workaround?
Not sure. Sounds like the device driver needs a quirk for this part.
The over-worked Jeff Garzik is the maintainer for that driver.
You should probably provide the pci device id for this beast.
--linas
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-19 19:57 ` Linas Vepstas
@ 2007-10-19 20:21 ` Jeff Garzik
2007-10-20 22:03 ` Benjamin Herrenschmidt
2007-10-22 20:26 ` Daniel Barkalow
2007-10-20 14:50 ` Shane Huang
1 sibling, 2 replies; 31+ messages in thread
From: Jeff Garzik @ 2007-10-19 20:21 UTC (permalink / raw)
To: Linas Vepstas
Cc: Shane Huang, davem, gregkh, htejun, brice.goglin,
david.gaarenstroom, linux-kernel, linux-pci, shane.huang,
linux-ide, Brice Goglin
Linas Vepstas wrote:
> On Fri, Oct 19, 2007 at 09:17:23PM +0800, Shane Huang wrote:
>> Since we have little experience on PCI and MSI here, we had to try to
>
> As someone else pointed out, AMD should have *lots* of people with
> pci and msi experience on the payroll. (Folks here buy AMD-designed
> pci chips ...)
>
>> ONLY
>> comment out the pci_intx() call in drivers/ata/ahci.c
>> My system can boot up too with MSI enabled!
>>
>> So does it mean that the root cause is our SB700 SATA controller
>> has a hardware bug where setting INTX_DISABLE in the PCI COMMAND
>> register masks MSI interrupts too?
>
> That's what it sounds like, to me.
>
>> And what is the software solution or workaround?
>
> Not sure. Sounds like the device driver needs a quirk for this part.
Take a look at tg3.c net driver change
2fbe43f6f631dd7ce19fb1499d6164a5bdb34568 which is a similar situation.
However, it may turn out that removing the pci_intx() stuff as a general
rule is easier than quirking these devices, if enough of them turn out
to have this hardware bug.
The tg3.c change should illustrate how to fix immediately, though.
Jeff
^ permalink raw reply [flat|nested] 31+ messages in thread
* RE: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-19 19:57 ` Linas Vepstas
2007-10-19 20:21 ` Jeff Garzik
@ 2007-10-20 14:50 ` Shane Huang
2007-10-20 20:52 ` David Gaarenstroom
1 sibling, 1 reply; 31+ messages in thread
From: Shane Huang @ 2007-10-20 14:50 UTC (permalink / raw)
To: Linas Vepstas, davem, gregkh, htejun, brice.goglin, david.gaarenstroom
Cc: linux-kernel, linux-pci, shane.huang, jgarzik, linux-ide, chunhao.huang
Quoting "David G"
> "Here" as in "here at AMD"?! I'm stunned...
> Both AMD and the former ATI should have quite some experience?!
When I used "here", I was just meaning our youthful linux southbridge
drivers team instead of the whole AMD. Sorry for the confusion to you.
Quoting "Linas"
> As someone else pointed out, AMD should have *lots* of people with
> pci and msi experience on the payroll. (Folks here buy AMD-designed
> pci chips ...)
Yes, absolutely AMD has lots of people with PCI and MSI experience,
but this MSI issue is mainly under the debug of our team now.
I think our team will cooperate with other teams more closely to provide
linux chipset support besides fixing the MSI problem in the future.
Quoting "Jeff"
> Take a look at tg3.c net driver change
> 2fbe43f6f631dd7ce19fb1499d6164a5bdb34568 which is a similar
> situation. However, it may turn out that removing the pci_intx()
> stuff as a general rule is easier than quirking these devices, if enough
> of them turn out to have this hardware bug. The tg3.c change should
> illustrate how to fix immediately, though.
Thanks for your suggestion. I will continue to debug this problem next
Monday when I'm back to the office.
Thanks
Best Regards
Shane
_________________________________________________________________
Invite your mail contacts to join your friends list with Windows Live Spaces. It's easy!
http://spaces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mkt=en-us
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-20 14:50 ` Shane Huang
@ 2007-10-20 20:52 ` David Gaarenstroom
2007-10-21 6:00 ` Shane Huang
2007-10-23 10:56 ` Shane Huang
0 siblings, 2 replies; 31+ messages in thread
From: David Gaarenstroom @ 2007-10-20 20:52 UTC (permalink / raw)
To: Shane Huang, linux-kernel
Cc: Linas Vepstas, davem, gregkh, htejun, brice.goglin, linux-pci,
shane.huang, jgarzik
If the pci_intx change will be applied to the SATA driver, can it be
applied for the ATI USB-HCDs too? See
http://lkml.org/lkml/2006/12/21/47 for more details. That should help
most of the ATI MSI quirks. It helped me (Acer Aspire 502x laptop with
ATI RS480/SB400).
> When I used "here", I was just meaning our youthful linux southbridge
> drivers team instead of the whole AMD. Sorry for the confusion to you.
I am still somewhat confused: most people developing the Linux kernel
are limited to look at this hardware from the outside. They had to
find out by trial and error. I really hope your colleagues will be
more actively helping your team and (other) Linux kernel developers.
But I'm glad AMD has such a team.
(So while you're at it, ask about the ATI USB HCDs and any other MSI quirk too)
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-19 20:21 ` Jeff Garzik
@ 2007-10-20 22:03 ` Benjamin Herrenschmidt
2007-10-22 20:26 ` Daniel Barkalow
1 sibling, 0 replies; 31+ messages in thread
From: Benjamin Herrenschmidt @ 2007-10-20 22:03 UTC (permalink / raw)
To: Jeff Garzik
Cc: Linas Vepstas, Shane Huang, davem, gregkh, htejun, brice.goglin,
david.gaarenstroom, linux-kernel, linux-pci, shane.huang,
linux-ide, Brice Goglin
On Fri, 2007-10-19 at 16:21 -0400, Jeff Garzik wrote:
> Take a look at tg3.c net driver change
> 2fbe43f6f631dd7ce19fb1499d6164a5bdb34568 which is a similar situation.
>
> However, it may turn out that removing the pci_intx() stuff as a
> general
> rule is easier than quirking these devices, if enough of them turn
> out
> to have this hardware bug.
We'd have to count how many have this bug vs. how many will emit both
intx and msi unless pci_intx is cleared, and then how many do that
regardless of pci_intx :-)
yuck
Ben.
^ permalink raw reply [flat|nested] 31+ messages in thread
* RE: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-20 20:52 ` David Gaarenstroom
@ 2007-10-21 6:00 ` Shane Huang
2007-10-23 10:56 ` Shane Huang
1 sibling, 0 replies; 31+ messages in thread
From: Shane Huang @ 2007-10-21 6:00 UTC (permalink / raw)
To: David Gaarenstroom
Cc: linux-kernel, Linas Vepstas, davem, gregkh, htejun, brice.goglin,
linux-pci, shane.huang, jgarzik, chunhao.huang
Hi David
>> When I used "here", I was just meaning our youthful linux southbridge
>> drivers team instead of the whole AMD. Sorry for the confusion to you.
>
> I am still somewhat confused: most people developing the Linux kernel
> are limited to look at this hardware from the outside. They had to
> find out by trial and error. I really hope your colleagues will be
> more actively helping your team and (other) Linux kernel developers.
> But I'm glad AMD has such a team.
Well... I think the cooperation will become more and more smoothly as
time goes by. In fact, other teams are always helping us.
As to the other problems such as USB-HCD, we need some investigation
before I can send you any response.
Thanks
Best Regards
Shane
_________________________________________________________________
Discover the new Windows Vista
http://search.msn.com/results.aspx?q=windows+vista&mkt=en-US&form=QBRE
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-19 20:21 ` Jeff Garzik
2007-10-20 22:03 ` Benjamin Herrenschmidt
@ 2007-10-22 20:26 ` Daniel Barkalow
2007-10-22 20:41 ` Jeff Garzik
1 sibling, 1 reply; 31+ messages in thread
From: Daniel Barkalow @ 2007-10-22 20:26 UTC (permalink / raw)
To: Jeff Garzik
Cc: Linas Vepstas, Shane Huang, davem, gregkh, htejun, brice.goglin,
david.gaarenstroom, linux-kernel, linux-pci, shane.huang,
linux-ide, Brice Goglin
On Fri, 19 Oct 2007, Jeff Garzik wrote:
> Linas Vepstas wrote:
> > On Fri, Oct 19, 2007 at 09:17:23PM +0800, Shane Huang wrote:
> > > Since we have little experience on PCI and MSI here, we had to try to
> >
> > As someone else pointed out, AMD should have *lots* of people with
> > pci and msi experience on the payroll. (Folks here buy AMD-designed pci
> > chips ...)
> >
> > > ONLY
> > > comment out the pci_intx() call in drivers/ata/ahci.c
> > > My system can boot up too with MSI enabled!
> > >
> > > So does it mean that the root cause is our SB700 SATA controller
> > > has a hardware bug where setting INTX_DISABLE in the PCI COMMAND
> > > register masks MSI interrupts too?
> >
> > That's what it sounds like, to me.
> >
> > > And what is the software solution or workaround?
> >
> > Not sure. Sounds like the device driver needs a quirk for this part.
>
>
> Take a look at tg3.c net driver change
> 2fbe43f6f631dd7ce19fb1499d6164a5bdb34568 which is a similar situation.
>
> However, it may turn out that removing the pci_intx() stuff as a general rule
> is easier than quirking these devices, if enough of them turn out to have this
> hardware bug.
At a first approximation, ATI/AMD devices don't send any interrupts if
intx is disabled, nVidia devices send legacy interrupts in addition to MSI
ones if intx isn't disabled, and Intel devices actually work correctly. So
we need at least one kind of device quirk for intx and msi. (And doing it
in the drivers doesn't work, since everybody is making things driven by
snd_hda_intel and would like msi, afaict)
-Daniel
*This .sig left intentionally blank*
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-22 20:26 ` Daniel Barkalow
@ 2007-10-22 20:41 ` Jeff Garzik
2007-10-22 21:31 ` Daniel Barkalow
2007-10-22 23:40 ` Krzysztof Halasa
0 siblings, 2 replies; 31+ messages in thread
From: Jeff Garzik @ 2007-10-22 20:41 UTC (permalink / raw)
To: Daniel Barkalow
Cc: Linas Vepstas, Shane Huang, davem, gregkh, htejun, brice.goglin,
david.gaarenstroom, linux-kernel, linux-pci, shane.huang,
linux-ide, Brice Goglin
Daniel Barkalow wrote:
> On Fri, 19 Oct 2007, Jeff Garzik wrote:
>
>> Linas Vepstas wrote:
>>> On Fri, Oct 19, 2007 at 09:17:23PM +0800, Shane Huang wrote:
>>>> Since we have little experience on PCI and MSI here, we had to try to
>>> As someone else pointed out, AMD should have *lots* of people with
>>> pci and msi experience on the payroll. (Folks here buy AMD-designed pci
>>> chips ...)
>>>
>>>> ONLY
>>>> comment out the pci_intx() call in drivers/ata/ahci.c
>>>> My system can boot up too with MSI enabled!
>>>>
>>>> So does it mean that the root cause is our SB700 SATA controller
>>>> has a hardware bug where setting INTX_DISABLE in the PCI COMMAND
>>>> register masks MSI interrupts too?
>>> That's what it sounds like, to me.
>>>
>>>> And what is the software solution or workaround?
>>> Not sure. Sounds like the device driver needs a quirk for this part.
>>
>> Take a look at tg3.c net driver change
>> 2fbe43f6f631dd7ce19fb1499d6164a5bdb34568 which is a similar situation.
>>
>> However, it may turn out that removing the pci_intx() stuff as a general rule
>> is easier than quirking these devices, if enough of them turn out to have this
>> hardware bug.
>
> At a first approximation, ATI/AMD devices don't send any interrupts if
> intx is disabled, nVidia devices send legacy interrupts in addition to MSI
> ones if intx isn't disabled, and Intel devices actually work correctly. So
> we need at least one kind of device quirk for intx and msi. (And doing it
> in the drivers doesn't work, since everybody is making things driven by
> snd_hda_intel and would like msi, afaict)
Note that INTX_DISABLE is a recent addition to PCI. Older PCI devices
support neither MSI nor INTX-disable, so make sure such devices don't
creep into your sample.
In general it is documented that INTX_DISABLE should apply only to INTx#
so devices that disable MSI based on that bit are out of spec. But
unfortunately that is rather irrelevant, since we see these out-of-spec
devices in the field today.
Jeff
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-22 20:41 ` Jeff Garzik
@ 2007-10-22 21:31 ` Daniel Barkalow
2007-10-22 23:48 ` Krzysztof Halasa
` (2 more replies)
2007-10-22 23:40 ` Krzysztof Halasa
1 sibling, 3 replies; 31+ messages in thread
From: Daniel Barkalow @ 2007-10-22 21:31 UTC (permalink / raw)
To: Jeff Garzik
Cc: Linas Vepstas, Shane Huang, davem, gregkh, htejun, brice.goglin,
david.gaarenstroom, linux-kernel, linux-pci, shane.huang,
linux-ide, Brice Goglin
On Mon, 22 Oct 2007, Jeff Garzik wrote:
> Daniel Barkalow wrote:
> > On Fri, 19 Oct 2007, Jeff Garzik wrote:
> >
> > > Linas Vepstas wrote:
> > > > On Fri, Oct 19, 2007 at 09:17:23PM +0800, Shane Huang wrote:
> > > > > Since we have little experience on PCI and MSI here, we had to try to
> > > > As someone else pointed out, AMD should have *lots* of people with
> > > > pci and msi experience on the payroll. (Folks here buy AMD-designed pci
> > > > chips ...)
> > > >
> > > > > ONLY
> > > > > comment out the pci_intx() call in drivers/ata/ahci.c
> > > > > My system can boot up too with MSI enabled!
> > > > >
> > > > > So does it mean that the root cause is our SB700 SATA controller
> > > > > has a hardware bug where setting INTX_DISABLE in the PCI COMMAND
> > > > > register masks MSI interrupts too?
> > > > That's what it sounds like, to me.
> > > >
> > > > > And what is the software solution or workaround?
> > > > Not sure. Sounds like the device driver needs a quirk for this part.
> > >
> > > Take a look at tg3.c net driver change
> > > 2fbe43f6f631dd7ce19fb1499d6164a5bdb34568 which is a similar situation.
> > >
> > > However, it may turn out that removing the pci_intx() stuff as a general
> > > rule
> > > is easier than quirking these devices, if enough of them turn out to have
> > > this
> > > hardware bug.
> >
> > At a first approximation, ATI/AMD devices don't send any interrupts if intx
> > is disabled, nVidia devices send legacy interrupts in addition to MSI ones
> > if intx isn't disabled, and Intel devices actually work correctly. So we
> > need at least one kind of device quirk for intx and msi. (And doing it in
> > the drivers doesn't work, since everybody is making things driven by
> > snd_hda_intel and would like msi, afaict)
>
> Note that INTX_DISABLE is a recent addition to PCI. Older PCI devices support
> neither MSI nor INTX-disable, so make sure such devices don't creep into your
> sample.
I have a device that supports MSI and INTX-disable, and, with MSI on (and
delivering interrupts successfully) also sends legacy interrupts (on
the IRQ that is no longer associated with the device) unless INTX is
disabled. Without the intx_disable(), the kernel disables the IRQ
entirely and breaks a random other device in my system.
It's:
00:07.0 Bridge: nVidia Corporation MCP61 Ethernet (rev a2)
I haven't tried MSI with the other devices in the system, but I expect
that this:
00:05.0 Audio device: nVidia Corporation MCP61 High Definition Audio (rev a2)
will have the same issue, and use a multi-vendor driver.
> In general it is documented that INTX_DISABLE should apply only to INTx# so
> devices that disable MSI based on that bit are out of spec. But unfortunately
> that is rather irrelevant, since we see these out-of-spec devices in the field
> today.
It's likewise documented (although maybe arguable in wording) that the
device shouldn't send legacy interrupts if MSI is in use, regardless of
INTX_DISABLE, but this also happens in the field.
I think that the current Linux behavior with respect to INTX_DISABLE is
simply due to which hardware bug was present in the device whose driver
first got Linux support, but one or the other or both needs a quirk, since
there's no behavior that works with everything. And it's still impossible
to tell which bug is more common, since MSI isn't used most of the time,
even if the hardware supports it, so it's pretty arbitrary which way Linux
goes in the non-quirk case.
-Daniel
*This .sig left intentionally blank*
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-22 20:41 ` Jeff Garzik
2007-10-22 21:31 ` Daniel Barkalow
@ 2007-10-22 23:40 ` Krzysztof Halasa
2007-10-22 23:58 ` David Miller
2007-10-23 10:13 ` Jeff Garzik
1 sibling, 2 replies; 31+ messages in thread
From: Krzysztof Halasa @ 2007-10-22 23:40 UTC (permalink / raw)
To: Jeff Garzik
Cc: Daniel Barkalow, Linas Vepstas, Shane Huang, davem, gregkh,
htejun, brice.goglin, david.gaarenstroom, linux-kernel,
linux-pci, shane.huang, linux-ide, Brice Goglin
Jeff Garzik <jeff@garzik.org> writes:
> Note that INTX_DISABLE is a recent addition to PCI.
It's PCI 2.3.
> Older PCI devices
> support neither MSI nor INTX-disable, so make sure such devices don't
> creep into your sample.
MSI has been introduced by PCI 2.2 (and thus PCI-X 1.0) so there may
be devices with MSI but without INTx-disable bit. I guess I have some
early PCI-X hardware with MSI but I don't know if they have INTx-disable
bit and I can't currently test that.
And it probably doesn't matter.
> In general it is documented that INTX_DISABLE should apply only to
> INTx# so devices that disable MSI based on that bit are out of spec.
The wording is:
10: This bit disables the device from asserting INTx#. A value of 0
enables the assertion of its INTx# signal. A value of 1 disables the
assertion of its INTx# signal. This bit's state after RST# is 0. Refer
to Section 6.8.1.3 for control of MSI.
So strictly speaking it mandates disabling/enabling INTx but says
nothing about other things (e.g. MSI). Some common sense dictates
it shouldn't disable MSI, I guess.
The "MSI Enable" description doesn't leave any doubt:
0: MSI Enable: If 1, the function is permitted to use MSI to request
service and is prohibited from using its INTx# pin [...]
> But unfortunately that is rather irrelevant, since we see these
> out-of-spec devices in the field today.
Right.
--
Krzysztof Halasa
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-22 21:31 ` Daniel Barkalow
@ 2007-10-22 23:48 ` Krzysztof Halasa
2007-10-23 0:13 ` David Miller
2007-10-23 10:15 ` Jeff Garzik
2 siblings, 0 replies; 31+ messages in thread
From: Krzysztof Halasa @ 2007-10-22 23:48 UTC (permalink / raw)
To: Daniel Barkalow
Cc: Jeff Garzik, Linas Vepstas, Shane Huang, davem, gregkh, htejun,
brice.goglin, david.gaarenstroom, linux-kernel, linux-pci,
shane.huang, linux-ide, Brice Goglin
Daniel Barkalow <barkalow@iabervon.org> writes:
> 00:05.0 Audio device: nVidia Corporation MCP61 High Definition Audio (rev a2)
>
> will have the same issue, and use a multi-vendor driver.
I think MCP55 HDA did not have such problem, though I may be wrong
(AFAIR it worked with shared IRQ and with MSI).
--
Krzysztof Halasa
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-22 23:40 ` Krzysztof Halasa
@ 2007-10-22 23:58 ` David Miller
2007-10-23 10:13 ` Jeff Garzik
1 sibling, 0 replies; 31+ messages in thread
From: David Miller @ 2007-10-22 23:58 UTC (permalink / raw)
To: khc
Cc: jeff, barkalow, linas, chunhao.huang, gregkh, htejun,
brice.goglin, david.gaarenstroom, linux-kernel, linux-pci,
shane.huang, linux-ide, brice
From: Krzysztof Halasa <khc@pm.waw.pl>
Date: Tue, 23 Oct 2007 01:40:18 +0200
> Jeff Garzik <jeff@garzik.org> writes:
>
> > In general it is documented that INTX_DISABLE should apply only to
> > INTx# so devices that disable MSI based on that bit are out of spec.
>
> The wording is:
> 10: This bit disables the device from asserting INTx#. A value of 0
> enables the assertion of its INTx# signal. A value of 1 disables the
> assertion of its INTx# signal. This bit's state after RST# is 0. Refer
> to Section 6.8.1.3 for control of MSI.
>
> So strictly speaking it mandates disabling/enabling INTx but says
> nothing about other things (e.g. MSI). Some common sense dictates
> it shouldn't disable MSI, I guess.
Right, and every vendor I've spoken to who had the INTX_DISABLE
bug clearly acknowledged that it was a bug in their RTL design
and that they considered the spec to be clear on this matter
in that INTX_DISABLE should not influence MSI in any way.
> The "MSI Enable" description doesn't leave any doubt:
> 0: MSI Enable: If 1, the function is permitted to use MSI to request
> service and is prohibited from using its INTx# pin [...]
Things get more complicated with PCI-Express because INTx# isn't an
out-of-band "pin", but rather a message sent over the bus :-)
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-22 21:31 ` Daniel Barkalow
2007-10-22 23:48 ` Krzysztof Halasa
@ 2007-10-23 0:13 ` David Miller
2007-10-23 5:52 ` Daniel Barkalow
` (2 more replies)
2007-10-23 10:15 ` Jeff Garzik
2 siblings, 3 replies; 31+ messages in thread
From: David Miller @ 2007-10-23 0:13 UTC (permalink / raw)
To: barkalow
Cc: jeff, linas, chunhao.huang, gregkh, htejun, brice.goglin,
david.gaarenstroom, linux-kernel, linux-pci, shane.huang,
linux-ide, brice
From: Daniel Barkalow <barkalow@iabervon.org>
Date: Mon, 22 Oct 2007 17:31:04 -0400 (EDT)
> It's likewise documented (although maybe arguable in wording) that the
> device shouldn't send legacy interrupts if MSI is in use, regardless of
> INTX_DISABLE, but this also happens in the field.
>
> I think that the current Linux behavior with respect to INTX_DISABLE is
> simply due to which hardware bug was present in the device whose driver
> first got Linux support, but one or the other or both needs a quirk, since
> there's no behavior that works with everything. And it's still impossible
> to tell which bug is more common, since MSI isn't used most of the time,
> even if the hardware supports it, so it's pretty arbitrary which way Linux
> goes in the non-quirk case.
I think this pretty much sums up the situation accurately.
My suggestion is:
1) Leave the pci_intx() twiddling code in drivers/pci/msi.c
2) Add quirks for "INTX_DISABLE turns off MSI too", this sets
a flag in the pci_dev.
3) The pci_intx() calls in drivers/pci/msi.c are skipped if this
flag from #2 is set.
4) Add quirk entries for drivers/net/tg3.c chips and these SATA
devices we are learning about here, as well as any others we
are aware of right now.
5) Remove the pci_intx() workaround code from drivers/net/tg3.c
and elsewhere.
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-23 0:13 ` David Miller
@ 2007-10-23 5:52 ` Daniel Barkalow
2007-10-23 9:39 ` Shane Huang
2007-10-23 10:01 ` Jeff Garzik
2 siblings, 0 replies; 31+ messages in thread
From: Daniel Barkalow @ 2007-10-23 5:52 UTC (permalink / raw)
To: David Miller
Cc: jeff, linas, chunhao.huang, gregkh, htejun, brice.goglin,
david.gaarenstroom, linux-kernel, linux-pci, shane.huang,
linux-ide, brice
On Mon, 22 Oct 2007, David Miller wrote:
> My suggestion is:
>
> 1) Leave the pci_intx() twiddling code in drivers/pci/msi.c
>
> 2) Add quirks for "INTX_DISABLE turns off MSI too", this sets
> a flag in the pci_dev.
>
> 3) The pci_intx() calls in drivers/pci/msi.c are skipped if this
> flag from #2 is set.
>
> 4) Add quirk entries for drivers/net/tg3.c chips and these SATA
> devices we are learning about here, as well as any others we
> are aware of right now.
>
> 5) Remove the pci_intx() workaround code from drivers/net/tg3.c
> and elsewhere.
Seems right to me, and pretty straightforward, except that I don't really
understand the pm-related logic in there to know how that should work and
whether intx will need to be enabled somewhere in addition to not
disabling it in the msi enable code.
-Daniel
*This .sig left intentionally blank*
^ permalink raw reply [flat|nested] 31+ messages in thread
* RE: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-23 0:13 ` David Miller
2007-10-23 5:52 ` Daniel Barkalow
@ 2007-10-23 9:39 ` Shane Huang
2007-10-23 10:01 ` Jeff Garzik
2 siblings, 0 replies; 31+ messages in thread
From: Shane Huang @ 2007-10-23 9:39 UTC (permalink / raw)
To: David Miller, barkalow
Cc: jeff, linas, gregkh, htejun, brice.goglin, david.gaarenstroom,
linux-kernel, linux-pci, linux-ide, brice, Shane Huang
Hi David:
> I think this pretty much sums up the situation accurately.
>
> My suggestion is:
>
> 1) Leave the pci_intx() twiddling code in drivers/pci/msi.c
>
> 2) Add quirks for "INTX_DISABLE turns off MSI too", this sets
> a flag in the pci_dev.
>
> 3) The pci_intx() calls in drivers/pci/msi.c are skipped if this
> flag from #2 is set.
>
> 4) Add quirk entries for drivers/net/tg3.c chips and these SATA
> devices we are learning about here, as well as any others we
> are aware of right now.
>
> 5) Remove the pci_intx() workaround code from drivers/net/tg3.c
> and elsewhere.
This quirk seems good to me. Waiting for your final decision....
This SB700 SATA controller MSI/INTx problem has been reported to our
hardware team. I will forward the update information or response to
you when I get any from HW team.
Thanks
Shane
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-23 0:13 ` David Miller
2007-10-23 5:52 ` Daniel Barkalow
2007-10-23 9:39 ` Shane Huang
@ 2007-10-23 10:01 ` Jeff Garzik
2007-10-23 10:06 ` David Miller
2 siblings, 1 reply; 31+ messages in thread
From: Jeff Garzik @ 2007-10-23 10:01 UTC (permalink / raw)
To: David Miller
Cc: barkalow, linas, chunhao.huang, gregkh, htejun, brice.goglin,
david.gaarenstroom, linux-kernel, linux-pci, shane.huang,
linux-ide, brice
David Miller wrote:
> My suggestion is:
>
> 1) Leave the pci_intx() twiddling code in drivers/pci/msi.c
>
> 2) Add quirks for "INTX_DISABLE turns off MSI too", this sets
> a flag in the pci_dev.
>
> 3) The pci_intx() calls in drivers/pci/msi.c are skipped if this
> flag from #2 is set.
>
> 4) Add quirk entries for drivers/net/tg3.c chips and these SATA
> devices we are learning about here, as well as any others we
> are aware of right now.
>
> 5) Remove the pci_intx() workaround code from drivers/net/tg3.c
> and elsewhere.
Sounds good to me also.
Jeff
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-23 10:01 ` Jeff Garzik
@ 2007-10-23 10:06 ` David Miller
2007-10-24 2:46 ` David Miller
0 siblings, 1 reply; 31+ messages in thread
From: David Miller @ 2007-10-23 10:06 UTC (permalink / raw)
To: jeff
Cc: barkalow, linas, chunhao.huang, gregkh, htejun, brice.goglin,
david.gaarenstroom, linux-kernel, linux-pci, shane.huang,
linux-ide, brice
From: Jeff Garzik <jeff@garzik.org>
Date: Tue, 23 Oct 2007 06:01:17 -0400
> David Miller wrote:
> > My suggestion is:
...
> Sounds good to me also.
Ok, it seems I've sort-of self-nominated myself to implement
this so I'll try to work on it tomorrow :-)
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-22 23:40 ` Krzysztof Halasa
2007-10-22 23:58 ` David Miller
@ 2007-10-23 10:13 ` Jeff Garzik
1 sibling, 0 replies; 31+ messages in thread
From: Jeff Garzik @ 2007-10-23 10:13 UTC (permalink / raw)
To: Krzysztof Halasa
Cc: Daniel Barkalow, Linas Vepstas, Shane Huang, davem, gregkh,
htejun, brice.goglin, david.gaarenstroom, linux-kernel,
linux-pci, shane.huang, linux-ide, Brice Goglin
Krzysztof Halasa wrote:
> Jeff Garzik <jeff@garzik.org> writes:
>
>> Note that INTX_DISABLE is a recent addition to PCI.
>
> It's PCI 2.3.
Yes.
>> Older PCI devices
>> support neither MSI nor INTX-disable, so make sure such devices don't
>> creep into your sample.
>
> MSI has been introduced by PCI 2.2 (and thus PCI-X 1.0) so there may
> be devices with MSI but without INTx-disable bit. I guess I have some
> early PCI-X hardware with MSI but I don't know if they have INTx-disable
> bit and I can't currently test that.
> And it probably doesn't matter.
Time will tell :)
>> In general it is documented that INTX_DISABLE should apply only to
>> INTx# so devices that disable MSI based on that bit are out of spec.
>
> The wording is:
> 10: This bit disables the device from asserting INTx#. A value of 0
> enables the assertion of its INTx# signal. A value of 1 disables the
> assertion of its INTx# signal. This bit's state after RST# is 0. Refer
> to Section 6.8.1.3 for control of MSI.
>
> So strictly speaking it mandates disabling/enabling INTx but says
> nothing about other things (e.g. MSI). Some common sense dictates
> it shouldn't disable MSI, I guess.
>
> The "MSI Enable" description doesn't leave any doubt:
> 0: MSI Enable: If 1, the function is permitted to use MSI to request
> service and is prohibited from using its INTx# pin [...]
Right. I was merely describing the end result, the union of that
language as it applies to the kernel.
Jeff
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-22 21:31 ` Daniel Barkalow
2007-10-22 23:48 ` Krzysztof Halasa
2007-10-23 0:13 ` David Miller
@ 2007-10-23 10:15 ` Jeff Garzik
2 siblings, 0 replies; 31+ messages in thread
From: Jeff Garzik @ 2007-10-23 10:15 UTC (permalink / raw)
To: Daniel Barkalow
Cc: Linas Vepstas, Shane Huang, davem, gregkh, htejun, brice.goglin,
david.gaarenstroom, linux-kernel, linux-pci, shane.huang,
linux-ide, Brice Goglin
Daniel Barkalow wrote:
> I have a device that supports MSI and INTX-disable, and, with MSI on (and
> delivering interrupts successfully) also sends legacy interrupts (on
> the IRQ that is no longer associated with the device) unless INTX is
> disabled. Without the intx_disable(), the kernel disables the IRQ
> entirely and breaks a random other device in my system.
That sort of behavior is an example of why I wrote pci_intx() in the
first place, and employed it by default throughout the ATA drivers
(before it migrated into PCI core).
Jeff
^ permalink raw reply [flat|nested] 31+ messages in thread
* RE: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-20 20:52 ` David Gaarenstroom
2007-10-21 6:00 ` Shane Huang
@ 2007-10-23 10:56 ` Shane Huang
2007-10-24 2:41 ` David Miller
1 sibling, 1 reply; 31+ messages in thread
From: Shane Huang @ 2007-10-23 10:56 UTC (permalink / raw)
To: David Gaarenstroom, linux-kernel
Cc: Linas Vepstas, davem, gregkh, htejun, brice.goglin, linux-pci,
jgarzik, Shane Huang
> If the pci_intx change will be applied to the SATA driver, can it be
> applied for the ATI USB-HCDs too? See
> http://lkml.org/lkml/2006/12/21/47 for more details. That should help
> most of the ATI MSI quirks. It helped me (Acer Aspire 502x laptop with
> ATI RS480/SB400).
I checked the USB MSI problem above(also has reported to our hardware
team),
the cause seems same as our SB700 SATA controller MSI problem.
I also did some small USB MSI debug on our SB700 board with 2.6.23-rc5:
The USB part of the second patch(attached at the end of this mail)
does not work for SB700 USB EHCI/OHCI controllers, no matter INTx is
enabled or disabled before enter MSI. The USB host controllers are
always using IO-APIC, which is different from SB400. I don't know why.
[root@localhost ~]# cat /proc/interrupts
CPU0 CPU1
17: 0 133 IO-APIC-fasteoi ohci_hcd:usb1,
ohci_hcd:usb2, ehci_hcd:usb6
18: 1 2 IO-APIC-fasteoi ohci_hcd:usb3,
ohci_hcd:usb4, ohci_hcd:usb5
19: 0 0 IO-APIC-fasteoi ehci_hcd:usb7
Also I wonder why the USB MSI patch is not added into kernel at last?
Will it lead to other bugs?
Thanks
Best Regards
Shane
======> USB part of the second patch in lkml.org/lkml/2006/12/21/47
diff -uprdN linux/drivers/usb/core/hcd-pci.c
linux/drivers/usb/core/hcd-pci.c
--- linux/drivers/usb/core/hcd-pci.c 2006-12-16 13:34:57.000000000
-0800
+++ linux/drivers/usb/core/hcd-pci.c 2006-12-16 13:57:09.000000000
-0800
@@ -69,6 +69,7 @@ int usb_hcd_pci_probe (struct pci_dev *d
if (pci_enable_device (dev) < 0)
return -ENODEV;
+ pci_enable_msi(dev);
dev->current_state = PCI_D0;
dev->dev.power.power_state = PMSG_ON;
@@ -139,6 +140,7 @@ int usb_hcd_pci_probe (struct pci_dev *d
release_region (hcd->rsrc_start, hcd->rsrc_len);
err2:
usb_put_hcd (hcd);
+ pci_disable_msi (dev);
err1:
pci_disable_device (dev);
dev_err (&dev->dev, "init %s fail, %d\n", pci_name(dev),
retval);
@@ -177,6 +179,7 @@ void usb_hcd_pci_remove (struct pci_dev
release_region (hcd->rsrc_start, hcd->rsrc_len);
}
usb_put_hcd (hcd);
+ pci_disable_msi(dev);
pci_disable_device(dev);
}
EXPORT_SYMBOL (usb_hcd_pci_remove);
@@ -391,6 +394,7 @@ int usb_hcd_pci_resume (struct pci_dev *
"can't re-enable after resume, %d!\n", retval);
return retval;
}
+ pci_enable_msi (dev);
pci_set_master (dev);
pci_restore_state (dev);
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-23 10:56 ` Shane Huang
@ 2007-10-24 2:41 ` David Miller
2007-10-24 6:53 ` Greg KH
0 siblings, 1 reply; 31+ messages in thread
From: David Miller @ 2007-10-24 2:41 UTC (permalink / raw)
To: Shane.Huang
Cc: david.gaarenstroom, linux-kernel, linas, gregkh, htejun,
brice.goglin, linux-pci, jgarzik
From: "Shane Huang" <Shane.Huang@amd.com>
Date: Tue, 23 Oct 2007 18:56:03 +0800
> Also I wonder why the USB MSI patch is not added into kernel at last?
> Will it lead to other bugs?
Probably someone just needs to be more vocal and active in pushing it
to the USB subsystem maintainer(s). I've even had trouble getting
even simple bug fixes integrated recently, so perhaps it will take a
few retransmits and some patience to get it included.
Anyways, thanks for bringing it to our attention.
Greg, can you at least devote a few minutes to going over that USB MSI
patch, giving it any obvious things it needs (perhaps some
pci_msi_enable() return value checks, for example, but may not be
needed at all in this case) and then stash it somewhere so it doesn't
get lost in the void?
Thanks.
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-23 10:06 ` David Miller
@ 2007-10-24 2:46 ` David Miller
0 siblings, 0 replies; 31+ messages in thread
From: David Miller @ 2007-10-24 2:46 UTC (permalink / raw)
To: jeff
Cc: barkalow, linas, chunhao.huang, gregkh, htejun, brice.goglin,
david.gaarenstroom, linux-kernel, linux-pci, shane.huang,
linux-ide, brice
From: David Miller <davem@davemloft.net>
Date: Tue, 23 Oct 2007 03:06:22 -0700 (PDT)
> Ok, it seems I've sort-of self-nominated myself to implement
> this so I'll try to work on it tomorrow :-)
I have a working implementation, fully tested on a machine
with Tigon3 ethernet chips that have the quirk in question.
Patch set coming up next.
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-24 2:41 ` David Miller
@ 2007-10-24 6:53 ` Greg KH
0 siblings, 0 replies; 31+ messages in thread
From: Greg KH @ 2007-10-24 6:53 UTC (permalink / raw)
To: David Miller
Cc: Shane.Huang, david.gaarenstroom, linux-kernel, linas, gregkh,
htejun, brice.goglin, linux-pci, jgarzik
On Tue, Oct 23, 2007 at 07:41:55PM -0700, David Miller wrote:
> From: "Shane Huang" <Shane.Huang@amd.com>
> Date: Tue, 23 Oct 2007 18:56:03 +0800
>
> > Also I wonder why the USB MSI patch is not added into kernel at last?
> > Will it lead to other bugs?
>
> Probably someone just needs to be more vocal and active in pushing it
> to the USB subsystem maintainer(s). I've even had trouble getting
> even simple bug fixes integrated recently, so perhaps it will take a
> few retransmits and some patience to get it included.
Yeah, I appologize for some of our developers, they seem a bit grumpy at
times :(
> Greg, can you at least devote a few minutes to going over that USB MSI
> patch, giving it any obvious things it needs (perhaps some
> pci_msi_enable() return value checks, for example, but may not be
> needed at all in this case) and then stash it somewhere so it doesn't
> get lost in the void?
Can someone forward it to me so that I can see it? I can't seem to
locate it at the moment, was I copied on it?
thanks,
greg k-h
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-18 11:46 ` David Miller
2007-10-18 15:24 ` Greg KH
@ 2007-10-19 17:42 ` Daniel Barkalow
1 sibling, 0 replies; 31+ messages in thread
From: Daniel Barkalow @ 2007-10-19 17:42 UTC (permalink / raw)
To: David Miller
Cc: Shane.Huang, gregkh, htejun, linux-kernel, linux-pci, Henry.Su,
Libin.Yang
On Thu, 18 Oct 2007, David Miller wrote:
> From: "Shane Huang" <Shane.Huang@amd.com>
> Date: Thu, 18 Oct 2007 18:37:59 +0800
>
> > Hi Miller:
> >
> > Thank you for your response.
> >
> > The reason why MSIs of these northbridges do not work is still under
> > further debug, we are NOT able to tell its hardware issue or software
> > issue at this time. But enablement of them will lead to the OS
> > installation failure in many distributions like openSUSE, Ubuntu etc:
> > https://bugzilla.novell.com/show_bug.cgi?id=302016
> >
> > So we have to disable them firstly before we find out the root cause,
> > maybe they are just workarounds.
>
> This logic seems backwards, to me. "shoot first, ask questions later"
> To me this it not how to approach this problem.
>
> Once you turn MSI off, there is next to no incentive to fix the
> problem because users aren't running into it any longer.
>
> The only two devices in that bug report which should be using MSI
> would be the SATA controller and the broadcom ethernet NIC. And by
> the failed bootup logs provided by the user the problem is clearly
> with the SATA controller.
And the same SATA controller could show up behind a different northbridge.
It would be unfortunate to hit the same device bug independantly on each
system and work around it by doing something that won't help the next
user.
> One common problem we're finding is that some devices have a hardware
> bug where setting INTX_DISABLE in the PCI COMMAND register masks MSI
> interrupts too.
>
> I mention this because the user in that report mentions that the
> kernel upgrade causes the failure, and one thing we started doing not
> too long ago was to set the INTX_DISABLE bit when MSI is enabled for a
> device.
>
> So maybe this SATA controller has this problem too. It is easy to
> test, simply comment out all of the pci_intx() function calls in
> drivers/pci/msi.c and perform a test boot with MSI enabled.
Have we gotten around to having a device quirk for this? I bet it won't be
too long before we see a system where the SATA controller doesn't work
with INTX disabled and the ethernet controller doesn't work with it
enabled, since we've seen devices with each of these bugs.
-Daniel
*This .sig left intentionally blank*
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-18 11:46 ` David Miller
@ 2007-10-18 15:24 ` Greg KH
2007-10-19 17:42 ` Daniel Barkalow
1 sibling, 0 replies; 31+ messages in thread
From: Greg KH @ 2007-10-18 15:24 UTC (permalink / raw)
To: David Miller
Cc: Shane.Huang, htejun, linux-kernel, linux-pci, Henry.Su, Libin.Yang
On Thu, Oct 18, 2007 at 04:46:11AM -0700, David Miller wrote:
> From: "Shane Huang" <Shane.Huang@amd.com>
> Date: Thu, 18 Oct 2007 18:37:59 +0800
>
> > Hi Miller:
> >
> > Thank you for your response.
> >
> > The reason why MSIs of these northbridges do not work is still under
> > further debug, we are NOT able to tell its hardware issue or software
> > issue at this time. But enablement of them will lead to the OS
> > installation failure in many distributions like openSUSE, Ubuntu etc:
> > https://bugzilla.novell.com/show_bug.cgi?id=302016
> >
> > So we have to disable them firstly before we find out the root cause,
> > maybe they are just workarounds.
>
> This logic seems backwards, to me. "shoot first, ask questions later"
> To me this it not how to approach this problem.
>
> Once you turn MSI off, there is next to no incentive to fix the
> problem because users aren't running into it any longer.
I agree with David here. Please work to find the root cause of this
problem. If it turns out that all of these chipsets have broken MSI and
can't handle it at all (oops, that's a major bug...) then we will accept
this patch.
But for now, let's not take a band-aid that prevents others from working
to solve the real issues here.
thanks,
greg k-h
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-18 10:37 ` Shane Huang
@ 2007-10-18 11:46 ` David Miller
2007-10-18 15:24 ` Greg KH
2007-10-19 17:42 ` Daniel Barkalow
0 siblings, 2 replies; 31+ messages in thread
From: David Miller @ 2007-10-18 11:46 UTC (permalink / raw)
To: Shane.Huang; +Cc: gregkh, htejun, linux-kernel, linux-pci, Henry.Su, Libin.Yang
From: "Shane Huang" <Shane.Huang@amd.com>
Date: Thu, 18 Oct 2007 18:37:59 +0800
> Hi Miller:
>
> Thank you for your response.
>
> The reason why MSIs of these northbridges do not work is still under
> further debug, we are NOT able to tell its hardware issue or software
> issue at this time. But enablement of them will lead to the OS
> installation failure in many distributions like openSUSE, Ubuntu etc:
> https://bugzilla.novell.com/show_bug.cgi?id=302016
>
> So we have to disable them firstly before we find out the root cause,
> maybe they are just workarounds.
This logic seems backwards, to me. "shoot first, ask questions later"
To me this it not how to approach this problem.
Once you turn MSI off, there is next to no incentive to fix the
problem because users aren't running into it any longer.
The only two devices in that bug report which should be using MSI
would be the SATA controller and the broadcom ethernet NIC. And by
the failed bootup logs provided by the user the problem is clearly
with the SATA controller.
One common problem we're finding is that some devices have a hardware
bug where setting INTX_DISABLE in the PCI COMMAND register masks MSI
interrupts too.
I mention this because the user in that report mentions that the
kernel upgrade causes the failure, and one thing we started doing not
too long ago was to set the INTX_DISABLE bit when MSI is enabled for a
device.
So maybe this SATA controller has this problem too. It is easy to
test, simply comment out all of the pci_intx() function calls in
drivers/pci/msi.c and perform a test boot with MSI enabled.
I would rather you approach analysis of these kinds of MSI bugs in
this manner, instead of disabling MSI wholesale. Because with the
latter approach it is nearly guarenteed that the real reason will only
be discovered with an extremely low priority.
Thank you.
^ permalink raw reply [flat|nested] 31+ messages in thread
* RE: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-18 10:19 ` David Miller
@ 2007-10-18 10:37 ` Shane Huang
2007-10-18 11:46 ` David Miller
0 siblings, 1 reply; 31+ messages in thread
From: Shane Huang @ 2007-10-18 10:37 UTC (permalink / raw)
To: David Miller
Cc: gregkh, htejun, linux-kernel, linux-pci, Su, Henry, Yang, Libin,
Shane Huang
Hi Miller:
Thank you for your response.
The reason why MSIs of these northbridges do not work is still under
further debug, we are NOT able to tell its hardware issue or software
issue at this time. But enablement of them will lead to the OS
installation failure in many distributions like openSUSE, Ubuntu etc:
https://bugzilla.novell.com/show_bug.cgi?id=302016
So we have to disable them firstly before we find out the root cause,
maybe they are just workarounds.
If you guys know much more about this MSI problem, don't hesitate to
tell us, we can debug on some of the hardware platforms.
BTW, There already some disablements to some ATI NB MSIs in the kernel,
the reasons are similar.
Thanks
Shane
> -----Original Message-----
> From: David Miller [mailto:davem@davemloft.net]
> Sent: Thursday, October 18, 2007 6:20 PM
> To: Shane Huang
> Cc: gregkh@suse.de; htejun@gmail.com; linux-kernel@vger.kernel.org;
linux-pci@atrey.karlin.mff.cuni.cz; Su, Henry; Yang, Libin
> Subject: Re: [patch] PCI: disable MSI on more ATI NorthBridges
>
> From: "Shane Huang" <Shane.Huang@amd.com>
> Date: Thu, 18 Oct 2007 17:14:36 +0800
>
> > More ATI North Bridges like RS780 can't do MSI like its predecessors
> > in linux. Disable MSIs on them.
> >
> > Signed-off-by: Shane Huang <shane.huang@amd.com>
>
> Can we get some detail as to why these north bridges have to have MSI
> disabled completely? I can't believe that all 6 of these ATI
> controllers which will now be listed in the quirk table cannot use MSI
> at all.
>
> I've discovered several cases where it was a buggy device or a driver
> bug that caused someone to erroneously submit patches that disable MSI
> completely for the bridge they were behind.
>
> I don't want that to happen any more. One way to prevent that is to
> have a full detailed justification for the MSI disabling in the
> changelog or in the comments for the MSI quirk.
>
> Thank you.
>
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [patch] PCI: disable MSI on more ATI NorthBridges
2007-10-18 9:14 Shane Huang
@ 2007-10-18 10:19 ` David Miller
2007-10-18 10:37 ` Shane Huang
0 siblings, 1 reply; 31+ messages in thread
From: David Miller @ 2007-10-18 10:19 UTC (permalink / raw)
To: Shane.Huang; +Cc: gregkh, htejun, linux-kernel, linux-pci, Henry.Su, Libin.Yang
From: "Shane Huang" <Shane.Huang@amd.com>
Date: Thu, 18 Oct 2007 17:14:36 +0800
> More ATI North Bridges like RS780 can't do MSI like its predecessors
> in linux. Disable MSIs on them.
>
> Signed-off-by: Shane Huang <shane.huang@amd.com>
Can we get some detail as to why these north bridges have to have MSI
disabled completely? I can't believe that all 6 of these ATI
controllers which will now be listed in the quirk table cannot use MSI
at all.
I've discovered several cases where it was a buggy device or a driver
bug that caused someone to erroneously submit patches that disable MSI
completely for the bridge they were behind.
I don't want that to happen any more. One way to prevent that is to
have a full detailed justification for the MSI disabling in the
changelog or in the comments for the MSI quirk.
Thank you.
^ permalink raw reply [flat|nested] 31+ messages in thread
* [patch] PCI: disable MSI on more ATI NorthBridges
@ 2007-10-18 9:14 Shane Huang
2007-10-18 10:19 ` David Miller
0 siblings, 1 reply; 31+ messages in thread
From: Shane Huang @ 2007-10-18 9:14 UTC (permalink / raw)
To: gregkh, htejun
Cc: linux-kernel, linux-pci, Su, Henry, Yang, Libin, Shane Huang
[-- Attachment #1: Type: text/plain, Size: 371 bytes --]
More ATI North Bridges like RS780 can't do MSI like its predecessors
in linux. Disable MSIs on them.
Signed-off-by: Shane Huang <shane.huang@amd.com>
Since there is some word wrapping problem with my mail client MS outlook
if I copy the patch into the text, so I'll have to attach the patch as
an attachment. Please check it.
Thanks
Best Regards
Shane
[-- Attachment #2: ATI_NBs_MSI_disable.patch --]
[-- Type: application/octet-stream, Size: 2854 bytes --]
diff -ruN linux-2.6.23_orig/drivers/pci/quirks.c linux-2.6.23/drivers/pci/quirks.c
--- linux-2.6.23_orig/drivers/pci/quirks.c 2007-10-10 04:31:38.000000000 +0800
+++ linux-2.6.23/drivers/pci/quirks.c 2007-10-18 16:51:30.000000000 +0800
@@ -1662,8 +1662,18 @@
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, PCI_DEVICE_ID_ATI_RS400_200, quirk_disable_all_msi);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, PCI_DEVICE_ID_ATI_RS480, quirk_disable_all_msi);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, PCI_DEVICE_ID_ATI_RD580, quirk_disable_all_msi);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, PCI_DEVICE_ID_ATI_RD790, quirk_disable_all_msi);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, PCI_DEVICE_ID_ATI_RX790, quirk_disable_all_msi);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, PCI_DEVICE_ID_ATI_RS690, quirk_disable_all_msi);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, PCI_DEVICE_ID_ATI_RS690_ALT, quirk_disable_all_msi);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, PCI_DEVICE_ID_ATI_RS740, quirk_disable_all_msi);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, PCI_DEVICE_ID_ATI_RS740_ALT, quirk_disable_all_msi);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, PCI_DEVICE_ID_ATI_RS780, quirk_disable_all_msi);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, PCI_DEVICE_ID_ATI_RS780_ALT, quirk_disable_all_msi);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, PCI_DEVICE_ID_ATI_RD890S, quirk_disable_all_msi);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, PCI_DEVICE_ID_ATI_RD890, quirk_disable_all_msi);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, PCI_DEVICE_ID_ATI_RD870S, quirk_disable_all_msi);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, PCI_DEVICE_ID_ATI_RX880, quirk_disable_all_msi);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_VIA, PCI_DEVICE_ID_VIA_VT3351, quirk_disable_all_msi);
/* Disable MSI on chipsets that are known to not support it */
diff -ruN linux-2.6.23_orig/include/linux/pci_ids.h linux-2.6.23/include/linux/pci_ids.h
--- linux-2.6.23_orig/include/linux/pci_ids.h 2007-10-10 04:31:38.000000000 +0800
+++ linux-2.6.23/include/linux/pci_ids.h 2007-10-18 16:41:57.000000000 +0800
@@ -361,8 +361,18 @@
#define PCI_DEVICE_ID_ATI_RS400_200 0x5a33
#define PCI_DEVICE_ID_ATI_RS480 0x5950
#define PCI_DEVICE_ID_ATI_RD580 0x5952
+#define PCI_DEVICE_ID_ATI_RD790 0x5956
#define PCI_DEVICE_ID_ATI_RX790 0x5957
#define PCI_DEVICE_ID_ATI_RS690 0x7910
+#define PCI_DEVICE_ID_ATI_RS690_ALT 0x7911
+#define PCI_DEVICE_ID_ATI_RS740 0x7960
+#define PCI_DEVICE_ID_ATI_RS740_ALT 0x7961
+#define PCI_DEVICE_ID_ATI_RS780 0x9600
+#define PCI_DEVICE_ID_ATI_RS780_ALT 0x9601
+#define PCI_DEVICE_ID_ATI_RD890S 0x5a10
+#define PCI_DEVICE_ID_ATI_RD890 0x5a11
+#define PCI_DEVICE_ID_ATI_RD870S 0x5a12
+#define PCI_DEVICE_ID_ATI_RX880 0x5a13
/* ATI IXP Chipset */
#define PCI_DEVICE_ID_ATI_IXP200_IDE 0x4349
#define PCI_DEVICE_ID_ATI_IXP200_SMBUS 0x4353
^ permalink raw reply [flat|nested] 31+ messages in thread
end of thread, other threads:[~2007-10-24 6:55 UTC | newest]
Thread overview: 31+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-10-19 13:17 [patch] PCI: disable MSI on more ATI NorthBridges Shane Huang
[not found] ` <9783dadb0710190706i499b4edbh9736019b0384f9b1@mail.gmail.com>
2007-10-19 14:10 ` Fwd: " David Gaarenstroom
2007-10-19 19:57 ` Linas Vepstas
2007-10-19 20:21 ` Jeff Garzik
2007-10-20 22:03 ` Benjamin Herrenschmidt
2007-10-22 20:26 ` Daniel Barkalow
2007-10-22 20:41 ` Jeff Garzik
2007-10-22 21:31 ` Daniel Barkalow
2007-10-22 23:48 ` Krzysztof Halasa
2007-10-23 0:13 ` David Miller
2007-10-23 5:52 ` Daniel Barkalow
2007-10-23 9:39 ` Shane Huang
2007-10-23 10:01 ` Jeff Garzik
2007-10-23 10:06 ` David Miller
2007-10-24 2:46 ` David Miller
2007-10-23 10:15 ` Jeff Garzik
2007-10-22 23:40 ` Krzysztof Halasa
2007-10-22 23:58 ` David Miller
2007-10-23 10:13 ` Jeff Garzik
2007-10-20 14:50 ` Shane Huang
2007-10-20 20:52 ` David Gaarenstroom
2007-10-21 6:00 ` Shane Huang
2007-10-23 10:56 ` Shane Huang
2007-10-24 2:41 ` David Miller
2007-10-24 6:53 ` Greg KH
-- strict thread matches above, loose matches on Subject: below --
2007-10-18 9:14 Shane Huang
2007-10-18 10:19 ` David Miller
2007-10-18 10:37 ` Shane Huang
2007-10-18 11:46 ` David Miller
2007-10-18 15:24 ` Greg KH
2007-10-19 17:42 ` Daniel Barkalow
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).