From: Bjorn Helgaas <bhelgaas@google.com>
To: Chris Boot <bootc@bootc.net>
Cc: Nix <nix@esperi.org.uk>,
"Wyborny, Carolyn" <carolyn.wyborny@intel.com>,
e1000-devel@lists.sourceforge.net,
netdev <netdev@vger.kernel.org>,
lkml <linux-kernel@vger.kernel.org>,
linux-pci@vger.kernel.org, Matthew Garrett <mjg@redhat.com>
Subject: Re: [E1000-devel] e1000e interface hang on 82574L
Date: Fri, 6 Apr 2012 06:12:58 -0600 [thread overview]
Message-ID: <CAErSpo4TJ-9NsD5QTpFi9fsnD9qHt7+Tr5jJHQPuS7vg6ghq_w@mail.gmail.com> (raw)
In-Reply-To: <1590F833-7D40-42FE-8FA2-6DCCADF9C6B0@bootc.net>
On Fri, Apr 6, 2012 at 4:17 AM, Chris Boot <bootc@bootc.net> wrote:
> On 19 Mar 2012, at 17:31, Nix wrote:
>
>> On 19 Mar 2012, Carolyn Wyborny said:
>>
>>>> you'll see that I tested that, and it doesn't work :( even if it did
>>>> work, it shouldn't be needed: the driver attempts to turn off PCIe ASPM
>>>> on affected NICs, and fails, apparently because *something* turns it
>>>> back on again.
>>>>
>>> The driver attempts to disable L0s state, not the entire feature. It
>>
>> It tries to disable L1 state as well (or it did when I tested this last,
>> although I suspect you're right and it may leave L1 turned on these
>> days: judging by the contents of e1000_82574_info, anyway.)
>>
>>> is also required that the device upstream on the bus from the 82574L
>>> have this disabled. Yes, I agree there appears to be something in the
>>> os that either ren-enables or fails to disable the feature on the
>>> upstream device, as desired. Platforms/systems also appear to vary in
>>> this regard, so the solutions may vary a bit as well.
>>>
>>> Its worth trying your solution as well if what I suggested doesn't
>>> work, but there is not one solution that fits all, unfortunately.
>>
>> I don't *have* a solution. :( 'setpci by hand some unknown amount of
>> time after booting once the interface has stabilized' hardly counts as a
>> solution of any sort. It's, at best, a workaround that lets me use my
>> systems without hourly lockups until a real solution is found.
>>
>> (To clarify: manual setpci to force off the ASPM bits is the only thing
>> that works for me. The driver's automatic disabling of L0s and L1
>> doesn't work: nor does booting with pcie_aspm=off. In both cases, I end
>> up with both L0s and L1 turned on, and a lockup some time later, unless
>> I setpci the bits off by hand.)
>
>
> Well, with that setpci incantation run against the NIC and its upstream device to disable ASPM L1s (setpci -s <dev> CAP_EXP+10.b=40), everything has been working very well indeed. Is there something the e1000e driver could do to disable L1s as well as L0s if we know there's a problem with them for these devices?
>
> Adding Bjorn Helgaas and linux-pci to CCs to try to get the ball rolling some more, as this is crippling without the fixes.
[+cc Matthew Garrett for ASPM stuff]
If I understand correctly, e1000e attempts to disable ASPM to work
around an 82574L hardware erratum, but the PCI core either doesn't
disable ASPM or it gets re-enabled somehow.
next prev parent reply other threads:[~2012-04-06 12:13 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-12-27 22:01 e1000e interface hang on 82574L Chris Boot
2011-12-27 22:33 ` Dave Taht
2011-12-31 9:31 ` Chris Boot
2012-01-03 0:02 ` Wyborny, Carolyn
2012-01-04 17:12 ` Chris Boot
2012-01-15 11:10 ` Chris Boot
2012-01-16 15:56 ` Wyborny, Carolyn
2012-01-16 16:04 ` Chris Boot
2012-03-17 15:59 ` Chris Boot
2012-03-17 17:54 ` Chris Boot
2012-03-17 23:50 ` [E1000-devel] " Nix
2012-03-19 14:59 ` Wyborny, Carolyn
2012-03-19 16:19 ` [E1000-devel] " Nix
2012-03-19 16:29 ` Wyborny, Carolyn
2012-03-19 17:31 ` Nix
2012-04-06 10:17 ` Chris Boot
2012-04-06 12:12 ` Bjorn Helgaas [this message]
2012-04-06 13:41 ` Henrique de Moraes Holschuh
2012-04-06 13:48 ` Chris Boot
2012-04-06 16:05 ` Nix
2012-04-06 16:04 ` Nix
2012-04-23 21:29 ` [PATCH RFC 0/2] e1000e: 82574 also needs ASPM L1 completely disabled Chris Boot
2012-04-23 21:29 ` [PATCH 1/2] e1000e: Disable ASPM L1 on 82574 Chris Boot
2012-04-23 23:18 ` [E1000-devel] " Jeff Kirsher
2012-04-24 11:08 ` Nix
2012-06-01 21:17 ` Chris Boot
2012-06-07 1:41 ` Greg KH
2012-04-23 21:29 ` [PATCH 2/2] e1000e: Remove special case for 82573/82574 ASPM L1 disablement Chris Boot
2012-04-23 23:18 ` [E1000-devel] " Jeff Kirsher
2012-04-23 23:11 ` [PATCH RFC 0/2] e1000e: 82574 also needs ASPM L1 completely disabled Jesse Brandeburg
2012-04-29 16:45 ` Nix
2012-04-29 18:03 ` Chris Boot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAErSpo4TJ-9NsD5QTpFi9fsnD9qHt7+Tr5jJHQPuS7vg6ghq_w@mail.gmail.com \
--to=bhelgaas@google.com \
--cc=bootc@bootc.net \
--cc=carolyn.wyborny@intel.com \
--cc=e1000-devel@lists.sourceforge.net \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=mjg@redhat.com \
--cc=netdev@vger.kernel.org \
--cc=nix@esperi.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).