From: Alex Williamson <alex.williamson@redhat.com>
To: Andreas Hartmann <andihartmann@freenet.de>
Cc: Bjorn Helgaas <bhelgaas@google.com>,
linux-pci <linux-pci@vger.kernel.org>
Subject: Re: Hard and silent lock up since linux 3.14 with PCIe pass through (vfio)
Date: Thu, 23 Oct 2014 10:33:42 -0600 [thread overview]
Message-ID: <1414082022.27420.39.camel@ul30vt.home> (raw)
In-Reply-To: <54492606.5090308@maya.org>
On Thu, 2014-10-23 at 18:00 +0200, Andreas Hartmann wrote:
> Alex Williamson wrote:
> > On Wed, 2014-10-22 at 18:22 +0200, Andreas Hartmann wrote:
> >> Alex Williamson wrote:
> >>> --- a/drivers/pci/pci.c
> >>> +++ b/drivers/pci/pci.c
> >>> @@ -3308,15 +3308,15 @@ static int __pci_dev_reset(struct pci_dev *dev, int prob
> >>> if (rc != -ENOTTY)
> >>> goto done;
> >>>
> >>> - rc = pci_pm_reset(dev, probe);
> >>> + rc = pci_dev_reset_slot_function(dev, probe);
> >>> if (rc != -ENOTTY)
> >>> goto done;
> >>>
> >>> - rc = pci_dev_reset_slot_function(dev, probe);
> >>> + rc = pci_parent_bus_reset(dev, probe);
> >>> if (rc != -ENOTTY)
> >>> goto done;
> >>>
> >>> - rc = pci_parent_bus_reset(dev, probe);
> >>> + rc = pci_pm_reset(dev, probe);
> >>> done:
> >>> return rc;
> >>> }
> >>
> >> This way it's crashing with echo 1 > reset, too.
> >
> > Ok, so it's somehow related to doing a bus reset with virtual channel
> > save/restore while PM reset with VC save/restore works ok as apparently
> > does bus reset without VC save/restore. Let's try to do a manual bus
> > reset so we can look at the post reset state of the device before the
> > kernel tries to restore it.
> >
> > First bind the target device 03:00.0 to pci-stub or vfio-pci so that we
> > know it's not being used.
> >
> > Next capture lspci -xxxx -s 3:00.0 so we have the starting state.
> >
> > Then we'll do a bus reset using setpci:
> > # setpci -s 00:05.0 3e.w=40:40
> > <if you script this, wait at least 2ms here>
> > # setpci -s 00:05.0 3e.w=00:40
> > <wait 1 second here>
> >
> > Now re-capture lspci -xxxx -s 3:00.0
>
> The machine is booted w/ vfio bound to 3:00.0 as usual (now for testing
> linux 3.14)
>
> lspci -xxxx -s 3:00.0
> setpci -s 00:05.0 3e.w=40:40
> usleep 10
> setpci -s 00:05.0 3e.w=00:40
> sleep 1
> lspci -xxxx -s 3:00.0
>
> I didn't get the second lspci because the machine already was hanging.
> The first output is attached completely.
Hmm, that doesn't make much sense. You had found that if you disabled
the VC save/restore then QEMU works. That should have still been using
secondary bus reset as we're trying to do here, so I don't understand
why we can't do a manual secondary bus reset now.
If you use Bjorn's previous patch to disable VC save/restore and my
patch to reorder the reset mechanisms, does echo 1 > reset for the sysfs
entry for the device also still cause a hang?
Can you provide a link to the specific model for this card? Thanks,
Alex
next prev parent reply other threads:[~2014-10-23 16:33 UTC|newest]
Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-09-23 19:03 Hard and silent lock up since linux 3.14 with PCIe pass through (vfio) Andreas Hartmann
2014-09-23 20:07 ` Alex Williamson
2014-09-24 14:54 ` Andreas Hartmann
2014-09-24 17:16 ` Andreas Hartmann
2014-10-10 9:39 ` Andreas Hartmann
2014-10-10 14:37 ` Bjorn Helgaas
2014-10-10 14:49 ` Andreas Hartmann
2014-10-10 15:55 ` Bjorn Helgaas
2014-10-10 16:09 ` Andreas Hartmann
2014-10-10 16:41 ` Bjorn Helgaas
2014-10-10 22:32 ` Andreas Hartmann
2014-10-10 22:54 ` Bjorn Helgaas
2014-10-11 6:20 ` Andreas Hartmann
2014-10-15 8:04 ` Alex Williamson
2014-10-17 1:04 ` Andreas Hartmann
2014-10-21 21:06 ` Alex Williamson
2014-10-21 21:32 ` Alex Williamson
2014-10-22 16:22 ` Andreas Hartmann
2014-10-22 20:36 ` Alex Williamson
2014-10-23 16:00 ` Andreas Hartmann
2014-10-23 16:33 ` Alex Williamson [this message]
2014-10-23 17:12 ` Andreas Hartmann
2014-10-23 17:33 ` Andreas Hartmann
2014-10-23 19:37 ` Alex Williamson
2014-10-24 14:21 ` Andreas Hartmann
2014-10-25 6:03 ` Andreas Hartmann
2014-10-28 21:51 ` Alex Williamson
2014-10-29 16:47 ` Andreas Hartmann
2014-10-29 17:44 ` Alex Williamson
2014-10-29 17:57 ` Andreas Hartmann
2014-10-29 18:16 ` Alex Williamson
2014-10-29 19:43 ` Andreas Hartmann
2014-10-29 20:50 ` Alex Williamson
2014-10-29 21:35 ` Andreas Hartmann
2014-10-30 16:35 ` Andreas Hartmann
2014-10-30 16:58 ` Alex Williamson
2014-10-30 19:09 ` Andreas Hartmann
2014-10-30 19:45 ` Alex Williamson
2014-10-30 20:21 ` Andreas Hartmann
2014-10-22 15:34 ` Andreas Hartmann
2014-10-22 16:02 ` Alex Williamson
2014-10-22 16:20 ` Andreas Hartmann
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1414082022.27420.39.camel@ul30vt.home \
--to=alex.williamson@redhat.com \
--cc=andihartmann@freenet.de \
--cc=bhelgaas@google.com \
--cc=linux-pci@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).