From: Bjorn Helgaas <bhelgaas@google.com>
To: Huang Ying <ying.huang@intel.com>
Cc: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org,
linux-pm@vger.kernel.org, "Rafael J. Wysocki" <rjw@sisk.pl>,
Zhang Yanmin <yanmin.zhang@intel.com>
Subject: Re: [BUGFIX 1/2] PCI/PM: Fix deadlock when unbind device if its parent in D3cold
Date: Sat, 3 Nov 2012 11:22:10 -0600 [thread overview]
Message-ID: <CAErSpo4k+SjqdBt3w3POkg=toCnZufH4gnwJpV8BW8DH5abkqw@mail.gmail.com> (raw)
In-Reply-To: <1351919165.13106.2.camel@yhuang-mobile.sh.intel.com>
On Fri, Nov 2, 2012 at 11:06 PM, Huang Ying <ying.huang@intel.com> wrote:
> On Fri, 2012-11-02 at 10:50 -0600, Bjorn Helgaas wrote:
>> On Wed, Oct 24, 2012 at 12:54 AM, Huang Ying <ying.huang@intel.com> wrote:
>> > If a PCI device and its parents are put into D3cold, unbinding the
>> > device will trigger deadlock as follow:
>> >
>> > - driver_unbind
>> > - device_release_driver
>> > - device_lock(dev) <--- previous lock here
>> > - __device_release_driver
>> > - pm_runtime_get_sync
>> > ...
>> > - rpm_resume(dev)
>> > - rpm_resume(dev->parent)
>> > ...
>> > - pci_pm_runtime_resume
>> > ...
>> > - pci_set_power_state
>> > - __pci_start_power_transition
>> > - pci_wakeup_bus(dev->parent->subordinate)
>> > - pci_walk_bus
>> > - device_lock(dev) <--- dead lock here
>> >
>> >
>> > If we do not do device_lock in pci_walk_bus, we can avoid dead lock.
>> > Device_lock in pci_walk_bus is introduced in commit:
>> > d71374dafbba7ec3f67371d3b7e9f6310a588808, corresponding email thread
>> > is: https://lkml.org/lkml/2006/5/26/38. The patch author Zhang Yanmin
>> > said device_lock is added to pci_walk_bus because:
>> >
>> > Some error handling functions call pci_walk_bus. For example, PCIe
>> > aer. Here we lock the device, so the driver wouldn't detach from the
>> > device, as the cb might call driver's callback function.
>> >
>> > So I fixed the dead lock as follow:
>> >
>> > - remove device_lock from pci_walk_bus
>> > - add device_lock into callback if callback will call driver's callback
>> >
>> > I checked pci_walk_bus users one by one, and found only PCIe aer needs
>> > device lock.
>>
>> Is there a problem report or bugzilla for this issue?
>
> No. I found this issue when I developed kexec fixing.
>
> Best Regards,
> Huang Ying
>
>> > Signed-off-by: Huang Ying <ying.huang@intel.com>
>> > Cc: Zhang Yanmin <yanmin.zhang@intel.com>
>>
>> Should this go to stable as well? The D3cold support appeared in
>> v3.6, so my guess is that this fix could go to v3.6.x.
What about the stable tree?
>> > ---
>> > drivers/pci/bus.c | 3 ---
>> > drivers/pci/pcie/aer/aerdrv_core.c | 20 ++++++++++++++++----
>> > 2 files changed, 16 insertions(+), 7 deletions(-)
>> >
>> > --- a/drivers/pci/bus.c
>> > +++ b/drivers/pci/bus.c
>> > @@ -320,10 +320,7 @@ void pci_walk_bus(struct pci_bus *top, i
>> > } else
>> > next = dev->bus_list.next;
>> >
>> > - /* Run device routines with the device locked */
>> > - device_lock(&dev->dev);
>> > retval = cb(dev, userdata);
>> > - device_unlock(&dev->dev);
>> > if (retval)
>> > break;
>> > }
>> > --- a/drivers/pci/pcie/aer/aerdrv_core.c
>> > +++ b/drivers/pci/pcie/aer/aerdrv_core.c
>> > @@ -213,6 +213,7 @@ static int report_error_detected(struct
>> > struct aer_broadcast_data *result_data;
>> > result_data = (struct aer_broadcast_data *) data;
>> >
>> > + device_lock(&dev->dev);
>> > dev->error_state = result_data->state;
>> >
>> > if (!dev->driver ||
>> > @@ -231,12 +232,14 @@ static int report_error_detected(struct
>> > dev->driver ?
>> > "no AER-aware driver" : "no driver");
>> > }
>> > - return 0;
>> > + goto out;
>> > }
>> >
>> > err_handler = dev->driver->err_handler;
>> > vote = err_handler->error_detected(dev, result_data->state);
>> > result_data->result = merge_result(result_data->result, vote);
>> > +out:
>> > + device_unlock(&dev->dev);
>> > return 0;
>> > }
>> >
>> > @@ -247,14 +250,17 @@ static int report_mmio_enabled(struct pc
>> > struct aer_broadcast_data *result_data;
>> > result_data = (struct aer_broadcast_data *) data;
>> >
>> > + device_lock(&dev->dev);
>> > if (!dev->driver ||
>> > !dev->driver->err_handler ||
>> > !dev->driver->err_handler->mmio_enabled)
>> > - return 0;
>> > + goto out;
>> >
>> > err_handler = dev->driver->err_handler;
>> > vote = err_handler->mmio_enabled(dev);
>> > result_data->result = merge_result(result_data->result, vote);
>> > +out:
>> > + device_unlock(&dev->dev);
>> > return 0;
>> > }
>> >
>> > @@ -265,14 +271,17 @@ static int report_slot_reset(struct pci_
>> > struct aer_broadcast_data *result_data;
>> > result_data = (struct aer_broadcast_data *) data;
>> >
>> > + device_lock(&dev->dev);
>> > if (!dev->driver ||
>> > !dev->driver->err_handler ||
>> > !dev->driver->err_handler->slot_reset)
>> > - return 0;
>> > + goto out;
>> >
>> > err_handler = dev->driver->err_handler;
>> > vote = err_handler->slot_reset(dev);
>> > result_data->result = merge_result(result_data->result, vote);
>> > +out:
>> > + device_unlock(&dev->dev);
>> > return 0;
>> > }
>> >
>> > @@ -280,15 +289,18 @@ static int report_resume(struct pci_dev
>> > {
>> > const struct pci_error_handlers *err_handler;
>> >
>> > + device_lock(&dev->dev);
>> > dev->error_state = pci_channel_io_normal;
>> >
>> > if (!dev->driver ||
>> > !dev->driver->err_handler ||
>> > !dev->driver->err_handler->resume)
>> > - return 0;
>> > + goto out;
>> >
>> > err_handler = dev->driver->err_handler;
>> > err_handler->resume(dev);
>> > +out:
>> > + device_unlock(&dev->dev);
>> > return 0;
>> > }
>> >
>
>
next prev parent reply other threads:[~2012-11-03 17:22 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-10-24 6:54 [BUGFIX 1/2] PCI/PM: Fix deadlock when unbind device if its parent in D3cold Huang Ying
2012-10-24 6:54 ` [BUGFIX 2/2] PCI/PM: Resume device before shutdown Huang Ying
2012-10-24 21:01 ` Rafael J. Wysocki
2012-11-02 16:52 ` Bjorn Helgaas
2012-11-02 20:26 ` Rafael J. Wysocki
2012-11-03 5:05 ` Huang Ying
2012-11-03 17:21 ` Bjorn Helgaas
2012-11-04 3:44 ` Huang Ying
2012-10-24 21:01 ` [BUGFIX 1/2] PCI/PM: Fix deadlock when unbind device if its parent in D3cold Rafael J. Wysocki
2012-11-02 16:50 ` Bjorn Helgaas
2012-11-02 20:27 ` Rafael J. Wysocki
2012-11-03 5:06 ` Huang Ying
2012-11-03 17:22 ` Bjorn Helgaas [this message]
2012-11-04 3:38 ` Huang Ying
2012-11-05 22:11 ` Bjorn Helgaas
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAErSpo4k+SjqdBt3w3POkg=toCnZufH4gnwJpV8BW8DH5abkqw@mail.gmail.com' \
--to=bhelgaas@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=linux-pm@vger.kernel.org \
--cc=rjw@sisk.pl \
--cc=yanmin.zhang@intel.com \
--cc=ying.huang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).