From: Aaron Ma <aaron.ma@canonical.com> To: "Pali Rohár" <pali@kernel.org>, "Bjorn Helgaas" <bhelgaas@google.com>, "Krzysztof Wilczyński" <kw@linux.com> Cc: jesse.brandeburg@intel.com, anthony.l.nguyen@intel.com, davem@davemloft.net, kuba@kernel.org, intel-wired-lan@lists.osuosl.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, "Marek Behún" <kabel@kernel.org> Subject: Re: [PATCH 1/2] igc: don't rd/wr iomem when PCI is removed Date: Mon, 5 Jul 2021 15:23:21 +0800 [thread overview] Message-ID: <3bc507f7-3eb9-1bef-d47d-cad42fcb1c48@canonical.com> (raw) In-Reply-To: <20210704142808.f43jbcufk37hundo@pali> On 7/4/21 10:28 PM, Pali Rohár wrote: > + Bjorn, Krzysztof and linux-pci > > On Friday 02 July 2021 12:51:19 Aaron Ma wrote: >> Check PCI state when rd/wr iomem. >> Implement wr32 function as rd32 too. >> >> When unplug TBT dock with i225, rd/wr PCI iomem will cause error log: >> Trace: >> BUG: unable to handle page fault for address: 000000000000b604 >> Oops: 0000 [#1] SMP NOPTI >> RIP: 0010:igc_rd32+0x1c/0x90 [igc] >> Call Trace: >> igc_ptp_suspend+0x6c/0xa0 [igc] >> igc_ptp_stop+0x12/0x50 [igc] >> igc_remove+0x7f/0x1c0 [igc] >> pci_device_remove+0x3e/0xb0 >> __device_release_driver+0x181/0x240 >> >> Signed-off-by: Aaron Ma <aaron.ma@canonical.com> >> --- >> drivers/net/ethernet/intel/igc/igc_main.c | 16 ++++++++++++++++ >> drivers/net/ethernet/intel/igc/igc_regs.h | 7 ++----- >> 2 files changed, 18 insertions(+), 5 deletions(-) >> >> diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c >> index f1adf154ec4a..606b72cb6193 100644 >> --- a/drivers/net/ethernet/intel/igc/igc_main.c >> +++ b/drivers/net/ethernet/intel/igc/igc_main.c >> @@ -5292,6 +5292,10 @@ u32 igc_rd32(struct igc_hw *hw, u32 reg) >> u8 __iomem *hw_addr = READ_ONCE(hw->hw_addr); >> u32 value = 0; >> >> + if (igc->pdev && >> + igc->pdev->error_state == pci_channel_io_perm_failure) > > Hello! This code pattern and commit message looks like that we could use > pci_dev_is_disconnected() helper function for checking if device is > still connected or was disconnected. > > Apparently pci_dev_is_disconnected() is defined only in private header > file drivers/pci/pci.h and not in public include/linux/pci.h. > > Aaron: can you check if pci_dev_is_disconnected() is really something > which should be used and it helps you? > Hi Pali, How about using pci_channel_offline instead? It's ready and also safe for frozen state, and verified on hw. > Bjorn, Krzysztof: what do you think about lifting helper function > pci_dev_is_disconnected() to be available to all drivers and not only in > PCI subsystem? > > I think that such helper function makes driver code more readable and > can be useful also for other drivers which are checking if return value > is all F's. > >> + return 0; > > Aaron: should not you return all F's on error? Because few lines below > in this function is returned value with all F's when PCIe link lost. > If you agree with the above change, I can fix it to "return -1" in v2. Thanks for your comments, Aaron >> + >> value = readl(&hw_addr[reg]); > > Anyway, this code looks to be racy. When pci_channel_io_perm_failure is > set (e.g. by hotplug interrupt) after checking for pdev->error_state and > prior executing above readl() then mentioned fatal error still occurs. > >> >> /* reads should not return all F's */ >> @@ -5308,6 +5312,18 @@ u32 igc_rd32(struct igc_hw *hw, u32 reg) >> return value; >> } >> >> +void igc_wr32(struct igc_hw *hw, u32 reg, u32 val) >> +{ >> + struct igc_adapter *igc = container_of(hw, struct igc_adapter, hw); >> + u8 __iomem *hw_addr = READ_ONCE(hw->hw_addr); >> + >> + if (igc->pdev && >> + igc->pdev->error_state == pci_channel_io_perm_failure) >> + return; >> + >> + writel((val), &hw_addr[(reg)]); >> +} >> + >> int igc_set_spd_dplx(struct igc_adapter *adapter, u32 spd, u8 dplx) >> { >> struct igc_mac_info *mac = &adapter->hw.mac; >> diff --git a/drivers/net/ethernet/intel/igc/igc_regs.h b/drivers/net/ethernet/intel/igc/igc_regs.h >> index cc174853554b..eb4be87d0e8b 100644 >> --- a/drivers/net/ethernet/intel/igc/igc_regs.h >> +++ b/drivers/net/ethernet/intel/igc/igc_regs.h >> @@ -260,13 +260,10 @@ struct igc_hw; >> u32 igc_rd32(struct igc_hw *hw, u32 reg); >> >> /* write operations, indexed using DWORDS */ >> -#define wr32(reg, val) \ >> -do { \ >> - u8 __iomem *hw_addr = READ_ONCE((hw)->hw_addr); \ >> - writel((val), &hw_addr[(reg)]); \ >> -} while (0) >> +void igc_wr32(struct igc_hw *hw, u32 reg, u32 val); >> >> #define rd32(reg) (igc_rd32(hw, reg)) >> +#define wr32(reg, val) (igc_wr32(hw, reg, val)) >> >> #define wrfl() ((void)rd32(IGC_STATUS)) >> >> -- >> 2.30.2 >>
WARNING: multiple messages have this Message-ID (diff)
From: Aaron Ma <aaron.ma@canonical.com> To: intel-wired-lan@osuosl.org Subject: [Intel-wired-lan] [PATCH 1/2] igc: don't rd/wr iomem when PCI is removed Date: Mon, 5 Jul 2021 15:23:21 +0800 [thread overview] Message-ID: <3bc507f7-3eb9-1bef-d47d-cad42fcb1c48@canonical.com> (raw) In-Reply-To: <20210704142808.f43jbcufk37hundo@pali> On 7/4/21 10:28 PM, Pali Roh?r wrote: > + Bjorn, Krzysztof and linux-pci > > On Friday 02 July 2021 12:51:19 Aaron Ma wrote: >> Check PCI state when rd/wr iomem. >> Implement wr32 function as rd32 too. >> >> When unplug TBT dock with i225, rd/wr PCI iomem will cause error log: >> Trace: >> BUG: unable to handle page fault for address: 000000000000b604 >> Oops: 0000 [#1] SMP NOPTI >> RIP: 0010:igc_rd32+0x1c/0x90 [igc] >> Call Trace: >> igc_ptp_suspend+0x6c/0xa0 [igc] >> igc_ptp_stop+0x12/0x50 [igc] >> igc_remove+0x7f/0x1c0 [igc] >> pci_device_remove+0x3e/0xb0 >> __device_release_driver+0x181/0x240 >> >> Signed-off-by: Aaron Ma <aaron.ma@canonical.com> >> --- >> drivers/net/ethernet/intel/igc/igc_main.c | 16 ++++++++++++++++ >> drivers/net/ethernet/intel/igc/igc_regs.h | 7 ++----- >> 2 files changed, 18 insertions(+), 5 deletions(-) >> >> diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c >> index f1adf154ec4a..606b72cb6193 100644 >> --- a/drivers/net/ethernet/intel/igc/igc_main.c >> +++ b/drivers/net/ethernet/intel/igc/igc_main.c >> @@ -5292,6 +5292,10 @@ u32 igc_rd32(struct igc_hw *hw, u32 reg) >> u8 __iomem *hw_addr = READ_ONCE(hw->hw_addr); >> u32 value = 0; >> >> + if (igc->pdev && >> + igc->pdev->error_state == pci_channel_io_perm_failure) > > Hello! This code pattern and commit message looks like that we could use > pci_dev_is_disconnected() helper function for checking if device is > still connected or was disconnected. > > Apparently pci_dev_is_disconnected() is defined only in private header > file drivers/pci/pci.h and not in public include/linux/pci.h. > > Aaron: can you check if pci_dev_is_disconnected() is really something > which should be used and it helps you? > Hi Pali, How about using pci_channel_offline instead? It's ready and also safe for frozen state, and verified on hw. > Bjorn, Krzysztof: what do you think about lifting helper function > pci_dev_is_disconnected() to be available to all drivers and not only in > PCI subsystem? > > I think that such helper function makes driver code more readable and > can be useful also for other drivers which are checking if return value > is all F's. > >> + return 0; > > Aaron: should not you return all F's on error? Because few lines below > in this function is returned value with all F's when PCIe link lost. > If you agree with the above change, I can fix it to "return -1" in v2. Thanks for your comments, Aaron >> + >> value = readl(&hw_addr[reg]); > > Anyway, this code looks to be racy. When pci_channel_io_perm_failure is > set (e.g. by hotplug interrupt) after checking for pdev->error_state and > prior executing above readl() then mentioned fatal error still occurs. > >> >> /* reads should not return all F's */ >> @@ -5308,6 +5312,18 @@ u32 igc_rd32(struct igc_hw *hw, u32 reg) >> return value; >> } >> >> +void igc_wr32(struct igc_hw *hw, u32 reg, u32 val) >> +{ >> + struct igc_adapter *igc = container_of(hw, struct igc_adapter, hw); >> + u8 __iomem *hw_addr = READ_ONCE(hw->hw_addr); >> + >> + if (igc->pdev && >> + igc->pdev->error_state == pci_channel_io_perm_failure) >> + return; >> + >> + writel((val), &hw_addr[(reg)]); >> +} >> + >> int igc_set_spd_dplx(struct igc_adapter *adapter, u32 spd, u8 dplx) >> { >> struct igc_mac_info *mac = &adapter->hw.mac; >> diff --git a/drivers/net/ethernet/intel/igc/igc_regs.h b/drivers/net/ethernet/intel/igc/igc_regs.h >> index cc174853554b..eb4be87d0e8b 100644 >> --- a/drivers/net/ethernet/intel/igc/igc_regs.h >> +++ b/drivers/net/ethernet/intel/igc/igc_regs.h >> @@ -260,13 +260,10 @@ struct igc_hw; >> u32 igc_rd32(struct igc_hw *hw, u32 reg); >> >> /* write operations, indexed using DWORDS */ >> -#define wr32(reg, val) \ >> -do { \ >> - u8 __iomem *hw_addr = READ_ONCE((hw)->hw_addr); \ >> - writel((val), &hw_addr[(reg)]); \ >> -} while (0) >> +void igc_wr32(struct igc_hw *hw, u32 reg, u32 val); >> >> #define rd32(reg) (igc_rd32(hw, reg)) >> +#define wr32(reg, val) (igc_wr32(hw, reg, val)) >> >> #define wrfl() ((void)rd32(IGC_STATUS)) >> >> -- >> 2.30.2 >>
next prev parent reply other threads:[~2021-07-05 7:23 UTC|newest] Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-07-02 4:51 [PATCH 1/2] igc: don't rd/wr iomem when PCI is removed Aaron Ma 2021-07-02 4:51 ` [Intel-wired-lan] " Aaron Ma 2021-07-02 4:51 ` [PATCH 2/2] igc: wait for the MAC copy when enabled MAC passthrough Aaron Ma 2021-07-02 4:51 ` [Intel-wired-lan] " Aaron Ma 2021-07-04 5:36 ` Neftin, Sasha 2021-07-04 5:36 ` Neftin, Sasha 2021-07-05 7:38 ` Aaron Ma 2021-07-05 7:38 ` Aaron Ma 2021-07-05 11:54 ` Neftin, Sasha 2021-07-05 11:54 ` Neftin, Sasha 2021-07-06 6:46 ` Aaron Ma 2021-07-06 6:46 ` Aaron Ma 2021-07-08 4:24 ` Neftin, Sasha 2021-07-08 4:24 ` Neftin, Sasha 2021-07-13 13:45 ` Aaron Ma 2021-07-13 13:45 ` Aaron Ma 2021-07-14 9:13 ` Ruinskiy, Dima 2021-07-14 9:13 ` Ruinskiy, Dima 2021-07-04 14:28 ` [PATCH 1/2] igc: don't rd/wr iomem when PCI is removed Pali Rohár 2021-07-04 14:28 ` [Intel-wired-lan] " Pali =?unknown-8bit?q?Roh=C3=A1r?= 2021-07-05 7:23 ` Aaron Ma [this message] 2021-07-05 7:23 ` Aaron Ma 2021-07-05 23:02 ` Krzysztof Wilczyński 2021-07-05 23:02 ` [Intel-wired-lan] " Krzysztof =?unknown-8bit?q?Wilczy=C5=84ski?= 2021-07-06 14:23 ` Pali Rohár 2021-07-06 14:23 ` [Intel-wired-lan] " Pali =?unknown-8bit?q?Roh=C3=A1r?= 2021-07-05 7:47 ` Dave Airlie 2021-07-05 7:47 ` [Intel-wired-lan] " Dave Airlie 2021-07-06 6:42 ` Aaron Ma 2021-07-06 6:42 ` [Intel-wired-lan] " Aaron Ma 2021-07-06 20:12 ` Bjorn Helgaas 2021-07-06 20:12 ` [Intel-wired-lan] " Bjorn Helgaas 2021-07-07 21:53 ` Pali Rohár 2021-07-07 21:53 ` [Intel-wired-lan] " Pali =?unknown-8bit?q?Roh=C3=A1r?= 2021-07-07 22:10 ` Bjorn Helgaas 2021-07-07 22:10 ` [Intel-wired-lan] " Bjorn Helgaas 2021-07-08 2:04 ` Oliver O'Halloran 2021-07-08 2:04 ` [Intel-wired-lan] " Oliver O'Halloran 2021-07-08 15:45 ` Bjorn Helgaas 2021-07-08 15:45 ` [Intel-wired-lan] " Bjorn Helgaas 2021-07-18 16:31 ` Oliver O'Halloran 2021-07-18 16:31 ` [Intel-wired-lan] " Oliver O'Halloran 2021-07-18 22:50 ` Pali Rohár 2021-07-18 22:50 ` [Intel-wired-lan] " Pali =?unknown-8bit?q?Roh=C3=A1r?= 2021-07-19 2:49 ` Oliver O'Halloran 2021-07-19 2:49 ` [Intel-wired-lan] " Oliver O'Halloran 2021-07-19 8:13 ` Pali Rohár 2021-07-19 8:13 ` [Intel-wired-lan] " Pali =?unknown-8bit?q?Roh=C3=A1r?= 2021-07-20 0:17 ` Bjorn Helgaas 2021-07-20 0:17 ` [Intel-wired-lan] " Bjorn Helgaas 2021-07-13 13:00 ` [PATCH v2] igc: fix page fault when thunderbolt is unplugged Aaron Ma 2021-07-13 13:00 ` [Intel-wired-lan] " Aaron Ma 2021-08-04 12:06 ` Fuxbrumer, Dvora 2021-08-04 12:06 ` Fuxbrumer, Dvora
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=3bc507f7-3eb9-1bef-d47d-cad42fcb1c48@canonical.com \ --to=aaron.ma@canonical.com \ --cc=anthony.l.nguyen@intel.com \ --cc=bhelgaas@google.com \ --cc=davem@davemloft.net \ --cc=intel-wired-lan@lists.osuosl.org \ --cc=jesse.brandeburg@intel.com \ --cc=kabel@kernel.org \ --cc=kuba@kernel.org \ --cc=kw@linux.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-pci@vger.kernel.org \ --cc=netdev@vger.kernel.org \ --cc=pali@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.