Unhandled fault during system suspend in sky2_shutdown

Unhandled fault during system suspend in sky2_shutdown

[Sudeep Holla]

Hi,

I am seeing unhandled fault during system suspend in sky2_shutdown.
I am not sure if it's something missing in the firmware, but just wanted
to check. I see that networkmanager is invoking calling to 
netlink_sendmsg which calls sky2_get_stats after the device is shutdown.

Unhandled fault: synchronous external abort (0x96000210) at 
0xffff0000091c2918
Internal error: : 96000210 [#1] PREEMPT SMP
Modules linked in:
CPU: 3 PID: 2029 Comm: NetworkManager Not tainted 4.6.0-rc3 #126
Hardware name: ARM Juno development board (r2) (DT)
task: ffff80007a673000 ti: ffff800940b5c000 task.ti: ffff800940b5c000
PC is at sky2_get_stats+0x44/0x3b8
LR is at dev_get_stats+0x58/0xc8
  sky2_get_stats+0x44/0x3b8
  rtnl_fill_stats+0x20/0x138
  rtnl_fill_ifinfo+0x440/0xb38
  rtnl_getlink+0xe8/0x198
  rtnetlink_rcv_msg+0xe4/0x220
  netlink_rcv_skb+0xc4/0xf8
  rtnetlink_rcv+0x2c/0x40
  netlink_unicast+0x160/0x238
  netlink_sendmsg+0x2f0/0x358
  sock_sendmsg+0x18/0x30
  ___sys_sendmsg+0x204/0x218
  __sys_sendmsg+0x44/0x88
  SyS_sendmsg+0xc/0x18
  el0_svc_naked+0x24/0x28

The below patch is the hack I came up to check if the netdev is detached 
and unregistered, I no longer see the issue.

Regards,
Sudeep

-->8

diff --git i/drivers/net/ethernet/marvell/sky2.c 
w/drivers/net/ethernet/marvell/sky2.c
index ec0a22119e09..0ff0434e32fc 100644
--- i/drivers/net/ethernet/marvell/sky2.c
+++ w/drivers/net/ethernet/marvell/sky2.c
@@ -5220,6 +5220,13 @@ static SIMPLE_DEV_PM_OPS(sky2_pm_ops, 
sky2_suspend, sky2_resume);

  static void sky2_shutdown(struct pci_dev *pdev)
  {
+       struct sky2_hw *hw = pci_get_drvdata(pdev);
+       int i;
+
+       for (i = hw->ports - 1; i >= 0; --i) {
+               sky2_detach(hw->dev[i]);
+               unregister_netdev(hw->dev[i]);
+       }
         sky2_suspend(&pdev->dev);
         pci_wake_from_d3(pdev, device_may_wakeup(&pdev->dev));
         pci_set_power_state(pdev, PCI_D3hot);

Re: Unhandled fault during system suspend in sky2_shutdown

[Stephen Hemminger]

On Mon, 11 Apr 2016 17:24:37 +0100
Sudeep Holla <sudeep.holla@arm.com> wrote:

> Hi,
> 
> I am seeing unhandled fault during system suspend in sky2_shutdown.
> I am not sure if it's something missing in the firmware, but just wanted
> to check. I see that networkmanager is invoking calling to 
> netlink_sendmsg which calls sky2_get_stats after the device is shutdown.
> 
> Unhandled fault: synchronous external abort (0x96000210) at 
> 0xffff0000091c2918
> Internal error: : 96000210 [#1] PREEMPT SMP
> Modules linked in:
> CPU: 3 PID: 2029 Comm: NetworkManager Not tainted 4.6.0-rc3 #126
> Hardware name: ARM Juno development board (r2) (DT)
> task: ffff80007a673000 ti: ffff800940b5c000 task.ti: ffff800940b5c000
> PC is at sky2_get_stats+0x44/0x3b8
> LR is at dev_get_stats+0x58/0xc8
>   sky2_get_stats+0x44/0x3b8
>   rtnl_fill_stats+0x20/0x138
>   rtnl_fill_ifinfo+0x440/0xb38
>   rtnl_getlink+0xe8/0x198
>   rtnetlink_rcv_msg+0xe4/0x220
>   netlink_rcv_skb+0xc4/0xf8
>   rtnetlink_rcv+0x2c/0x40
>   netlink_unicast+0x160/0x238
>   netlink_sendmsg+0x2f0/0x358
>   sock_sendmsg+0x18/0x30
>   ___sys_sendmsg+0x204/0x218
>   __sys_sendmsg+0x44/0x88
>   SyS_sendmsg+0xc/0x18
>   el0_svc_naked+0x24/0x28
> 
> The below patch is the hack I came up to check if the netdev is detached 
> and unregistered, I no longer see the issue.
> 
> Regards,
> Sudeep
> 
> -->8  
> 
> diff --git i/drivers/net/ethernet/marvell/sky2.c 
> w/drivers/net/ethernet/marvell/sky2.c
> index ec0a22119e09..0ff0434e32fc 100644
> --- i/drivers/net/ethernet/marvell/sky2.c
> +++ w/drivers/net/ethernet/marvell/sky2.c
> @@ -5220,6 +5220,13 @@ static SIMPLE_DEV_PM_OPS(sky2_pm_ops, 
> sky2_suspend, sky2_resume);
> 
>   static void sky2_shutdown(struct pci_dev *pdev)
>   {
> +       struct sky2_hw *hw = pci_get_drvdata(pdev);
> +       int i;
> +
> +       for (i = hw->ports - 1; i >= 0; --i) {
> +               sky2_detach(hw->dev[i]);
> +               unregister_netdev(hw->dev[i]);
> +       }
>          sky2_suspend(&pdev->dev);
>          pci_wake_from_d3(pdev, device_may_wakeup(&pdev->dev));
>          pci_set_power_state(pdev, PCI_D3hot);

This is not the correct fix, the device is supposed to stay registered.
The correct way to fix this would be to make get_stats ignore requests for device
when suspended.

Re: Unhandled fault during system suspend in sky2_shutdown

[Sudeep Holla]

On 11/04/16 19:24, Stephen Hemminger wrote:
> On Mon, 11 Apr 2016 17:24:37 +0100
> Sudeep Holla <sudeep.holla@arm.com> wrote:
>

[...]

>>
>> diff --git i/drivers/net/ethernet/marvell/sky2.c
>> w/drivers/net/ethernet/marvell/sky2.c
>> index ec0a22119e09..0ff0434e32fc 100644
>> --- i/drivers/net/ethernet/marvell/sky2.c
>> +++ w/drivers/net/ethernet/marvell/sky2.c
>> @@ -5220,6 +5220,13 @@ static SIMPLE_DEV_PM_OPS(sky2_pm_ops,
>> sky2_suspend, sky2_resume);
>>
>>    static void sky2_shutdown(struct pci_dev *pdev)
>>    {
>> +       struct sky2_hw *hw = pci_get_drvdata(pdev);
>> +       int i;
>> +
>> +       for (i = hw->ports - 1; i >= 0; --i) {
>> +               sky2_detach(hw->dev[i]);
>> +               unregister_netdev(hw->dev[i]);
>> +       }
>>           sky2_suspend(&pdev->dev);
>>           pci_wake_from_d3(pdev, device_may_wakeup(&pdev->dev));
>>           pci_set_power_state(pdev, PCI_D3hot);
>
> This is not the correct fix, the device is supposed to stay
> registered. The correct way to fix this would be to make get_stats
> ignore requests  for device when suspended.

Yes I agree it's not correct fix. But I tried ignoring it in get_stat32
but the crash just moves the bug elsewhere. IMO patching all the places
to check the suspended state might be bit heavy ?

E.g. with something like below the crash moves to sky2_get_eeprom_len
function.

sky2_get_eeprom_len+0x10/0x30
dev_ethtool+0x29c/0x1d78
dev_ioctl+0x31c/0x5a8
sock_ioctl+0x2ac/0x310
do_vfs_ioctl+0xa4/0x750
SyS_ioctl+0x8c/0xa0
el0_svc_naked+0x24/0x2

Sorry if I am missing something fundamental, I am bit new to net drivers

Regards,
Sudeep

-->8

diff --git i/drivers/net/ethernet/marvell/sky2.c 
w/drivers/net/ethernet/marvell/sky2.c
index ec0a22119e09..d4cfcd89e7e5 100644
--- i/drivers/net/ethernet/marvell/sky2.c
+++ w/drivers/net/ethernet/marvell/sky2.c
@@ -5175,6 +5175,7 @@ static int sky2_suspend(struct device *dev)
         }

         sky2_power_aux(hw);
+       hw->suspended = true;
         rtnl_unlock();

         return 0;
@@ -5198,6 +5199,7 @@ static int sky2_resume(struct device *dev)
         }

         rtnl_lock();
+       hw->suspended = false;
         sky2_reset(hw);
         sky2_all_up(hw);
         rtnl_unlock();
diff --git i/drivers/net/ethernet/marvell/sky2.h 
w/drivers/net/ethernet/marvell/sky2.h
index ec6dcd80152b..1386e5b635ff 100644
--- i/drivers/net/ethernet/marvell/sky2.h
+++ w/drivers/net/ethernet/marvell/sky2.h
@@ -2308,6 +2308,7 @@ struct sky2_hw {
         wait_queue_head_t    msi_wait;

         char                 irq_name[0];
+       bool                 suspended;
  };

  static inline int sky2_is_copper(const struct sky2_hw *hw)
@@ -2378,6 +2379,9 @@ static inline u32 get_stats32(struct sky2_hw *hw, 
unsigned port, unsigned reg)
  {
         u32 val;

+       if (hw->suspended)
+               return 0;
+
         do {
                 val = gma_read32(hw, port, reg);
         } while (gma_read32(hw, port, reg) != val);