From: Benjamin Herrenschmidt <benh@kernel.crashing.org>
To: Alexey Kardashevskiy <aik@ozlabs.ru>,
Jason Gunthorpe <jgg@ziepe.ca>,
David Gibson <david@gibson.dropbear.id.au>
Cc: Leon Romanovsky <leon@kernel.org>,
linux-rdma@vger.kernel.org, netdev@vger.kernel.org,
linux-kernel@vger.kernel.org, sbest@redhat.com,
saeedm@mellanox.com, alex.williamson@redhat.com,
paulus@samba.org, linux-pci@vger.kernel.org, bhelgaas@google.com,
ogerlitz@mellanox.com, linuxppc-dev@lists.ozlabs.org,
davem@davemloft.net, tariqt@mellanox.com
Subject: Re: [PATCH] PCI: Add no-D3 quirk for Mellanox ConnectX-[45]
Date: Wed, 09 Jan 2019 18:24:00 +1100 [thread overview]
Message-ID: <f06efdd89b418ad831ee3a5da766f3f0a584050c.camel@kernel.crashing.org> (raw)
In-Reply-To: <06c4612c-8409-ea7d-4f7c-4c010d8ecc01@ozlabs.ru>
On Wed, 2019-01-09 at 15:53 +1100, Alexey Kardashevskiy wrote:
> "A PCI completion timeout occurred for an outstanding PCI-E transaction"
> it is.
>
> This is how I bind the device to vfio:
>
> echo vfio-pci > '/sys/bus/pci/devices/0000:01:00.0/driver_override'
> echo vfio-pci > '/sys/bus/pci/devices/0000:01:00.1/driver_override'
> echo '0000:01:00.0' > '/sys/bus/pci/devices/0000:01:00.0/driver/unbind'
> echo '0000:01:00.1' > '/sys/bus/pci/devices/0000:01:00.1/driver/unbind'
> echo '0000:01:00.0' > /sys/bus/pci/drivers/vfio-pci/bind
> echo '0000:01:00.1' > /sys/bus/pci/drivers/vfio-pci/bind
>
>
> and I noticed that EEH only happens with the last command. The order
> (.0,.1 or .1,.0) does not matter, it seems that putting one function to
> D3 is fine but putting another one when the first one is already in D3 -
> produces EEH. And I do not recall ever seeing this on the firestone
> machine. Weird.
Putting all functions into D3 is what allows the device to actually go
into D3.
Does it work with other devices ? We do have that bug on early P9
revisions where the attempt of bringing the link to L1 as part of the
D3 process fails in horrible ways, I thought P8 would be ok but maybe
not ...
Otherwise, it might be that our timeouts are too low (you may want to
talk to our PCIe guys internally)
Cheers,
Ben.
next prev parent reply other threads:[~2019-01-09 7:26 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-06 4:19 [PATCH] PCI: Add no-D3 quirk for Mellanox ConnectX-[45] David Gibson
2018-12-06 6:45 ` Leon Romanovsky
2018-12-11 2:31 ` David Gibson
2019-01-04 3:44 ` David Gibson
2019-01-05 17:51 ` Jason Gunthorpe
2019-01-05 22:43 ` Benjamin Herrenschmidt
2019-01-08 4:01 ` Jason Gunthorpe
2019-01-08 6:07 ` Leon Romanovsky
2019-01-09 5:09 ` Benjamin Herrenschmidt
2019-01-09 5:30 ` David Gibson
2019-01-09 6:32 ` Alexey Kardashevskiy
2019-01-09 7:25 ` Benjamin Herrenschmidt
2019-01-09 8:14 ` Alexey Kardashevskiy
2019-01-09 15:27 ` Jason Gunthorpe
2019-01-09 4:53 ` Alexey Kardashevskiy
2019-01-09 7:24 ` Benjamin Herrenschmidt [this message]
2019-01-09 8:20 ` Alexey Kardashevskiy
2018-12-11 14:01 ` Bjorn Helgaas
2018-12-12 0:22 ` David Gibson
2018-12-12 3:04 ` Bjorn Helgaas
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f06efdd89b418ad831ee3a5da766f3f0a584050c.camel@kernel.crashing.org \
--to=benh@kernel.crashing.org \
--cc=aik@ozlabs.ru \
--cc=alex.williamson@redhat.com \
--cc=bhelgaas@google.com \
--cc=davem@davemloft.net \
--cc=david@gibson.dropbear.id.au \
--cc=jgg@ziepe.ca \
--cc=leon@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=netdev@vger.kernel.org \
--cc=ogerlitz@mellanox.com \
--cc=paulus@samba.org \
--cc=saeedm@mellanox.com \
--cc=sbest@redhat.com \
--cc=tariqt@mellanox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).