From: Aaron Ma <aaron.ma@canonical.com>
To: "Pali Rohár" <pali@kernel.org>,
"Bjorn Helgaas" <bhelgaas@google.com>,
"Krzysztof Wilczyński" <kw@linux.com>
Cc: jesse.brandeburg@intel.com, anthony.l.nguyen@intel.com,
davem@davemloft.net, kuba@kernel.org,
intel-wired-lan@lists.osuosl.org, netdev@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org,
"Marek Behún" <kabel@kernel.org>
Subject: Re: [PATCH 1/2] igc: don't rd/wr iomem when PCI is removed
Date: Mon, 5 Jul 2021 15:23:21 +0800 [thread overview]
Message-ID: <3bc507f7-3eb9-1bef-d47d-cad42fcb1c48@canonical.com> (raw)
In-Reply-To: <20210704142808.f43jbcufk37hundo@pali>
On 7/4/21 10:28 PM, Pali Rohár wrote:
> + Bjorn, Krzysztof and linux-pci
>
> On Friday 02 July 2021 12:51:19 Aaron Ma wrote:
>> Check PCI state when rd/wr iomem.
>> Implement wr32 function as rd32 too.
>>
>> When unplug TBT dock with i225, rd/wr PCI iomem will cause error log:
>> Trace:
>> BUG: unable to handle page fault for address: 000000000000b604
>> Oops: 0000 [#1] SMP NOPTI
>> RIP: 0010:igc_rd32+0x1c/0x90 [igc]
>> Call Trace:
>> igc_ptp_suspend+0x6c/0xa0 [igc]
>> igc_ptp_stop+0x12/0x50 [igc]
>> igc_remove+0x7f/0x1c0 [igc]
>> pci_device_remove+0x3e/0xb0
>> __device_release_driver+0x181/0x240
>>
>> Signed-off-by: Aaron Ma <aaron.ma@canonical.com>
>> ---
>> drivers/net/ethernet/intel/igc/igc_main.c | 16 ++++++++++++++++
>> drivers/net/ethernet/intel/igc/igc_regs.h | 7 ++-----
>> 2 files changed, 18 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c
>> index f1adf154ec4a..606b72cb6193 100644
>> --- a/drivers/net/ethernet/intel/igc/igc_main.c
>> +++ b/drivers/net/ethernet/intel/igc/igc_main.c
>> @@ -5292,6 +5292,10 @@ u32 igc_rd32(struct igc_hw *hw, u32 reg)
>> u8 __iomem *hw_addr = READ_ONCE(hw->hw_addr);
>> u32 value = 0;
>>
>> + if (igc->pdev &&
>> + igc->pdev->error_state == pci_channel_io_perm_failure)
>
> Hello! This code pattern and commit message looks like that we could use
> pci_dev_is_disconnected() helper function for checking if device is
> still connected or was disconnected.
>
> Apparently pci_dev_is_disconnected() is defined only in private header
> file drivers/pci/pci.h and not in public include/linux/pci.h.
>
> Aaron: can you check if pci_dev_is_disconnected() is really something
> which should be used and it helps you?
>
Hi Pali,
How about using pci_channel_offline instead?
It's ready and also safe for frozen state, and verified on hw.
> Bjorn, Krzysztof: what do you think about lifting helper function
> pci_dev_is_disconnected() to be available to all drivers and not only in
> PCI subsystem?
>
> I think that such helper function makes driver code more readable and
> can be useful also for other drivers which are checking if return value
> is all F's.
>
>> + return 0;
>
> Aaron: should not you return all F's on error? Because few lines below
> in this function is returned value with all F's when PCIe link lost.
>
If you agree with the above change, I can fix it to "return -1" in v2.
Thanks for your comments,
Aaron
>> +
>> value = readl(&hw_addr[reg]);
>
> Anyway, this code looks to be racy. When pci_channel_io_perm_failure is
> set (e.g. by hotplug interrupt) after checking for pdev->error_state and
> prior executing above readl() then mentioned fatal error still occurs.
>
>>
>> /* reads should not return all F's */
>> @@ -5308,6 +5312,18 @@ u32 igc_rd32(struct igc_hw *hw, u32 reg)
>> return value;
>> }
>>
>> +void igc_wr32(struct igc_hw *hw, u32 reg, u32 val)
>> +{
>> + struct igc_adapter *igc = container_of(hw, struct igc_adapter, hw);
>> + u8 __iomem *hw_addr = READ_ONCE(hw->hw_addr);
>> +
>> + if (igc->pdev &&
>> + igc->pdev->error_state == pci_channel_io_perm_failure)
>> + return;
>> +
>> + writel((val), &hw_addr[(reg)]);
>> +}
>> +
>> int igc_set_spd_dplx(struct igc_adapter *adapter, u32 spd, u8 dplx)
>> {
>> struct igc_mac_info *mac = &adapter->hw.mac;
>> diff --git a/drivers/net/ethernet/intel/igc/igc_regs.h b/drivers/net/ethernet/intel/igc/igc_regs.h
>> index cc174853554b..eb4be87d0e8b 100644
>> --- a/drivers/net/ethernet/intel/igc/igc_regs.h
>> +++ b/drivers/net/ethernet/intel/igc/igc_regs.h
>> @@ -260,13 +260,10 @@ struct igc_hw;
>> u32 igc_rd32(struct igc_hw *hw, u32 reg);
>>
>> /* write operations, indexed using DWORDS */
>> -#define wr32(reg, val) \
>> -do { \
>> - u8 __iomem *hw_addr = READ_ONCE((hw)->hw_addr); \
>> - writel((val), &hw_addr[(reg)]); \
>> -} while (0)
>> +void igc_wr32(struct igc_hw *hw, u32 reg, u32 val);
>>
>> #define rd32(reg) (igc_rd32(hw, reg))
>> +#define wr32(reg, val) (igc_wr32(hw, reg, val))
>>
>> #define wrfl() ((void)rd32(IGC_STATUS))
>>
>> --
>> 2.30.2
>>
next prev parent reply other threads:[~2021-07-05 7:23 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-02 4:51 [PATCH 1/2] igc: don't rd/wr iomem when PCI is removed Aaron Ma
2021-07-02 4:51 ` [PATCH 2/2] igc: wait for the MAC copy when enabled MAC passthrough Aaron Ma
2021-07-04 5:36 ` [Intel-wired-lan] " Neftin, Sasha
2021-07-05 7:38 ` Aaron Ma
2021-07-05 11:54 ` Neftin, Sasha
2021-07-06 6:46 ` Aaron Ma
2021-07-08 4:24 ` Neftin, Sasha
2021-07-13 13:45 ` Aaron Ma
2021-07-14 9:13 ` Ruinskiy, Dima
2021-07-04 14:28 ` [PATCH 1/2] igc: don't rd/wr iomem when PCI is removed Pali Rohár
2021-07-05 7:23 ` Aaron Ma [this message]
2021-07-05 23:02 ` Krzysztof Wilczyński
2021-07-06 14:23 ` Pali Rohár
2021-07-05 7:47 ` Dave Airlie
2021-07-06 6:42 ` Aaron Ma
2021-07-06 20:12 ` Bjorn Helgaas
2021-07-07 21:53 ` Pali Rohár
2021-07-07 22:10 ` Bjorn Helgaas
2021-07-08 2:04 ` Oliver O'Halloran
2021-07-08 15:45 ` Bjorn Helgaas
2021-07-18 16:31 ` Oliver O'Halloran
2021-07-18 22:50 ` Pali Rohár
2021-07-19 2:49 ` Oliver O'Halloran
2021-07-19 8:13 ` Pali Rohár
2021-07-20 0:17 ` Bjorn Helgaas
2021-07-13 13:00 ` [PATCH v2] igc: fix page fault when thunderbolt is unplugged Aaron Ma
2021-08-04 12:06 ` [Intel-wired-lan] " Fuxbrumer, Dvora
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3bc507f7-3eb9-1bef-d47d-cad42fcb1c48@canonical.com \
--to=aaron.ma@canonical.com \
--cc=anthony.l.nguyen@intel.com \
--cc=bhelgaas@google.com \
--cc=davem@davemloft.net \
--cc=intel-wired-lan@lists.osuosl.org \
--cc=jesse.brandeburg@intel.com \
--cc=kabel@kernel.org \
--cc=kuba@kernel.org \
--cc=kw@linux.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pali@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).