* [PATCH v2 0/2] Fully enable AER @ 2022-01-17 8:03 Stefan Roese 2022-01-17 8:03 ` [PATCH v2 1/2] PCI/portdrv: Don't disable AER reporting in get_port_device_capability() Stefan Roese 2022-01-17 8:03 ` [PATCH v2 2/2] PCI/AER: Enable AER on Endpoints as well Stefan Roese 0 siblings, 2 replies; 6+ messages in thread From: Stefan Roese @ 2022-01-17 8:03 UTC (permalink / raw) To: linux-pci While working on AER support on a ZynqMP based system, which has some PCIe Device connected via a PCIe switch, problems with AER enabling in the Device Control registers of all PCIe devices but the Root Port. In fact, only the Root Port has AER enabled right now. This patch set now fixes this problem by first fixing the AER enabing in the interconnected PCIe switches between the Root Port and the PCIe devices and in a 2nd patch, also enabling AER in the PCIe Endpoints. Please note that these changes are quite invasie, as with these patches applied, AER now will be enabled in the Device Control registers of all available PCIe Endpoints, which currently is not the case. Stefan Roese (2): PCI/portdrv: Don't disable AER reporting in get_port_device_capability() PCI/AER: Enable AER on Endpoints as well drivers/pci/pcie/aer.c | 3 ++- drivers/pci/pcie/portdrv_core.c | 9 +-------- 2 files changed, 3 insertions(+), 9 deletions(-) -- 2.34.1 ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v2 1/2] PCI/portdrv: Don't disable AER reporting in get_port_device_capability() 2022-01-17 8:03 [PATCH v2 0/2] Fully enable AER Stefan Roese @ 2022-01-17 8:03 ` Stefan Roese 2022-01-17 9:21 ` Pali Rohár 2022-01-17 8:03 ` [PATCH v2 2/2] PCI/AER: Enable AER on Endpoints as well Stefan Roese 1 sibling, 1 reply; 6+ messages in thread From: Stefan Roese @ 2022-01-17 8:03 UTC (permalink / raw) To: linux-pci Cc: Rafael J . Wysocki, Bjorn Helgaas, Pali Rohár, Bharat Kumar Gogada, Michal Simek, Yao Hongbo, Naveen Naidu Testing has shown, that AER reporting is currently disabled in the DevCtl registers of all non Root Port PCIe devices on systems using pcie_ports_native || host->native_aer. Practically disabling AER completely in such systems. This is due to the fact that with commit 2bd50dd800b5 ("PCI: PCIe: Disable PCIe port services during port initialization"), a call to pci_disable_pcie_error_reporting() was added *after* the PCIe AER setup was completed for the PCIe device tree. Here a longer analysis about the currect status of AER enaling / disabling upon bootup provided by Bjorn: pcie_portdrv_probe pcie_port_device_register get_port_device_capability pci_disable_pcie_error_reporting clear CERE NFERE FERE URRE # <-- disable for RP USP DSP pcie_device_init device_register # new AER service device aer_probe aer_enable_rootport # RP only set_downstream_devices_error_reporting set_device_error_reporting # self (RP) if (RP || USP || DSP) pci_enable_pcie_error_reporting set CERE NFERE FERE URRE # <-- enable for RP pci_walk_bus set_device_error_reporting if (RP || USP || DSP) pci_enable_pcie_error_reporting set CERE NFERE FERE URRE # <-- enable for USP DSP In a typical Root Port -> Endpoint hierarchy, the above: - Disables Error Reporting for the Root Port, - Enables Error Reporting for the Root Port, - Does NOT enable Error Reporting for the Endpoint because it is not a Root Port or Switch Port. In a deeper Root Port -> Upstream Switch Port -> Downstream Switch Port -> Endpoint hierarchy: - Disables Error Reporting for the Root Port, - Enables Error Reporting for the Root Port, - Enables Error Reporting for both Switch Ports, - Does NOT enable Error Reporting for the Endpoint because it is not a Root Port or Switch Port, - Disables Error Reporting for the Switch Ports when pcie_portdrv_probe() claims them. AER does not re-enable it because these are not Root Ports. This patch now removes this call to pci_disable_pcie_error_reporting() from get_port_device_capability(), leaving the already enabled AER configuration intact. With this change, AER is enabled in the Root Port and the PCIe switch upstream and downstream ports. Only the PCIe Endpoints don't have AER enabled yet. A follow-up patch will take care of this Endpoint enabling. Fixes: 2bd50dd800b5 ("PCI: PCIe: Disable PCIe port services during port initialization") Signed-off-by: Stefan Roese <sr@denx.de> Cc: Rafael J. Wysocki <rjw@rjwysocki.net> Cc: Bjorn Helgaas <helgaas@kernel.org> Cc: Pali Rohár <pali@kernel.org> Cc: Bharat Kumar Gogada <bharat.kumar.gogada@xilinx.com> Cc: Michal Simek <michal.simek@xilinx.com> Cc: Yao Hongbo <yaohongbo@linux.alibaba.com> Cc: Naveen Naidu <naveennaidu479@gmail.com> --- v2: - Enhance commit message as suggested by Bjorn drivers/pci/pcie/portdrv_core.c | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/drivers/pci/pcie/portdrv_core.c b/drivers/pci/pcie/portdrv_core.c index f81c7be4d7d8..27b990cedb4c 100644 --- a/drivers/pci/pcie/portdrv_core.c +++ b/drivers/pci/pcie/portdrv_core.c @@ -244,15 +244,8 @@ static int get_port_device_capability(struct pci_dev *dev) #ifdef CONFIG_PCIEAER if (dev->aer_cap && pci_aer_available() && - (pcie_ports_native || host->native_aer)) { + (pcie_ports_native || host->native_aer)) services |= PCIE_PORT_SERVICE_AER; - - /* - * Disable AER on this port in case it's been enabled by the - * BIOS (the AER service driver will enable it when necessary). - */ - pci_disable_pcie_error_reporting(dev); - } #endif /* Root Ports and Root Complex Event Collectors may generate PMEs */ -- 2.34.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v2 1/2] PCI/portdrv: Don't disable AER reporting in get_port_device_capability() 2022-01-17 8:03 ` [PATCH v2 1/2] PCI/portdrv: Don't disable AER reporting in get_port_device_capability() Stefan Roese @ 2022-01-17 9:21 ` Pali Rohár 0 siblings, 0 replies; 6+ messages in thread From: Pali Rohár @ 2022-01-17 9:21 UTC (permalink / raw) To: Stefan Roese Cc: linux-pci, Rafael J . Wysocki, Bjorn Helgaas, Bharat Kumar Gogada, Michal Simek, Yao Hongbo, Naveen Naidu On Monday 17 January 2022 09:03:47 Stefan Roese wrote: > Testing has shown, that AER reporting is currently disabled in the > DevCtl registers of all non Root Port PCIe devices on systems using > pcie_ports_native || host->native_aer. Practically disabling AER > completely in such systems. This is due to the fact that with commit > 2bd50dd800b5 ("PCI: PCIe: Disable PCIe port services during port > initialization"), a call to pci_disable_pcie_error_reporting() was > added *after* the PCIe AER setup was completed for the PCIe device > tree. > > Here a longer analysis about the currect status of AER enaling / > disabling upon bootup provided by Bjorn: > > pcie_portdrv_probe > pcie_port_device_register > get_port_device_capability > pci_disable_pcie_error_reporting > clear CERE NFERE FERE URRE # <-- disable for RP USP DSP > pcie_device_init > device_register # new AER service device > aer_probe > aer_enable_rootport # RP only > set_downstream_devices_error_reporting > set_device_error_reporting # self (RP) > if (RP || USP || DSP) > pci_enable_pcie_error_reporting > set CERE NFERE FERE URRE # <-- enable for RP > pci_walk_bus > set_device_error_reporting > if (RP || USP || DSP) > pci_enable_pcie_error_reporting > set CERE NFERE FERE URRE # <-- enable for USP DSP > > In a typical Root Port -> Endpoint hierarchy, the above: > - Disables Error Reporting for the Root Port, > - Enables Error Reporting for the Root Port, > - Does NOT enable Error Reporting for the Endpoint because it is not > a Root Port or Switch Port. > > In a deeper Root Port -> Upstream Switch Port -> Downstream Switch > Port -> Endpoint hierarchy: > - Disables Error Reporting for the Root Port, > - Enables Error Reporting for the Root Port, > - Enables Error Reporting for both Switch Ports, > - Does NOT enable Error Reporting for the Endpoint because it is not > a Root Port or Switch Port, > - Disables Error Reporting for the Switch Ports when > pcie_portdrv_probe() claims them. AER does not re-enable it > because these are not Root Ports. > > This patch now removes this call to pci_disable_pcie_error_reporting() > from get_port_device_capability(), leaving the already enabled AER > configuration intact. With this change, AER is enabled in the Root Port > and the PCIe switch upstream and downstream ports. Only the PCIe > Endpoints don't have AER enabled yet. A follow-up patch will take > care of this Endpoint enabling. > > Fixes: 2bd50dd800b5 ("PCI: PCIe: Disable PCIe port services during port initialization") > Signed-off-by: Stefan Roese <sr@denx.de> > Cc: Rafael J. Wysocki <rjw@rjwysocki.net> > Cc: Bjorn Helgaas <helgaas@kernel.org> > Cc: Pali Rohár <pali@kernel.org> > Cc: Bharat Kumar Gogada <bharat.kumar.gogada@xilinx.com> > Cc: Michal Simek <michal.simek@xilinx.com> > Cc: Yao Hongbo <yaohongbo@linux.alibaba.com> > Cc: Naveen Naidu <naveennaidu479@gmail.com> Reviewed-by: Pali Rohár <pali@kernel.org> > --- > v2: > - Enhance commit message as suggested by Bjorn > > drivers/pci/pcie/portdrv_core.c | 9 +-------- > 1 file changed, 1 insertion(+), 8 deletions(-) > > diff --git a/drivers/pci/pcie/portdrv_core.c b/drivers/pci/pcie/portdrv_core.c > index f81c7be4d7d8..27b990cedb4c 100644 > --- a/drivers/pci/pcie/portdrv_core.c > +++ b/drivers/pci/pcie/portdrv_core.c > @@ -244,15 +244,8 @@ static int get_port_device_capability(struct pci_dev *dev) > > #ifdef CONFIG_PCIEAER > if (dev->aer_cap && pci_aer_available() && > - (pcie_ports_native || host->native_aer)) { > + (pcie_ports_native || host->native_aer)) > services |= PCIE_PORT_SERVICE_AER; > - > - /* > - * Disable AER on this port in case it's been enabled by the > - * BIOS (the AER service driver will enable it when necessary). > - */ > - pci_disable_pcie_error_reporting(dev); > - } > #endif > > /* Root Ports and Root Complex Event Collectors may generate PMEs */ > -- > 2.34.1 > ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v2 2/2] PCI/AER: Enable AER on Endpoints as well 2022-01-17 8:03 [PATCH v2 0/2] Fully enable AER Stefan Roese 2022-01-17 8:03 ` [PATCH v2 1/2] PCI/portdrv: Don't disable AER reporting in get_port_device_capability() Stefan Roese @ 2022-01-17 8:03 ` Stefan Roese 2022-01-17 9:30 ` Pali Rohár 1 sibling, 1 reply; 6+ messages in thread From: Stefan Roese @ 2022-01-17 8:03 UTC (permalink / raw) To: linux-pci Cc: Bjorn Helgaas, Pali Rohár, Bharat Kumar Gogada, Michal Simek, Yao Hongbo, Naveen Naidu Currently, the PCIe AER subsystem does not enable AER in the PCIe Endpoints via the Device Control register. It's only done for the Root Port and all PCIe Ports in between the Root Port and the Endpoint(s). Some device drivers enable AER in their PCIe device by directly calling pci_enable_pcie_error_reporting(). But in most cases, AER is currently disabled in the PCIe Endpoints. This patch enables AER on PCIe Endpoints now as well in set_device_error_reporting(). This will make the ad-hoc calls to pci_enable_pcie_error_reporting() superfluous. Signed-off-by: Stefan Roese <sr@denx.de> Cc: Bjorn Helgaas <helgaas@kernel.org> Cc: Pali Rohár <pali@kernel.org> Cc: Bharat Kumar Gogada <bharat.kumar.gogada@xilinx.com> Cc: Michal Simek <michal.simek@xilinx.com> Cc: Yao Hongbo <yaohongbo@linux.alibaba.com> Cc: Naveen Naidu <naveennaidu479@gmail.com> --- v2: - New patch drivers/pci/pcie/aer.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index 9fa1f97e5b27..385e2033d7b5 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -1216,7 +1216,8 @@ static int set_device_error_reporting(struct pci_dev *dev, void *data) if ((type == PCI_EXP_TYPE_ROOT_PORT) || (type == PCI_EXP_TYPE_RC_EC) || (type == PCI_EXP_TYPE_UPSTREAM) || - (type == PCI_EXP_TYPE_DOWNSTREAM)) { + (type == PCI_EXP_TYPE_DOWNSTREAM) || + (type == PCI_EXP_TYPE_ENDPOINT)) { if (enable) pci_enable_pcie_error_reporting(dev); else -- 2.34.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v2 2/2] PCI/AER: Enable AER on Endpoints as well 2022-01-17 8:03 ` [PATCH v2 2/2] PCI/AER: Enable AER on Endpoints as well Stefan Roese @ 2022-01-17 9:30 ` Pali Rohár 2022-01-17 9:39 ` Stefan Roese 0 siblings, 1 reply; 6+ messages in thread From: Pali Rohár @ 2022-01-17 9:30 UTC (permalink / raw) To: Stefan Roese Cc: linux-pci, Bjorn Helgaas, Bharat Kumar Gogada, Michal Simek, Yao Hongbo, Naveen Naidu On Monday 17 January 2022 09:03:48 Stefan Roese wrote: > Currently, the PCIe AER subsystem does not enable AER in the PCIe > Endpoints via the Device Control register. It's only done for the > Root Port and all PCIe Ports in between the Root Port and the > Endpoint(s). Some device drivers enable AER in their PCIe device by > directly calling pci_enable_pcie_error_reporting(). But in most > cases, AER is currently disabled in the PCIe Endpoints. > > This patch enables AER on PCIe Endpoints now as well in > set_device_error_reporting(). This will make the ad-hoc calls to > pci_enable_pcie_error_reporting() superfluous. > > Signed-off-by: Stefan Roese <sr@denx.de> > Cc: Bjorn Helgaas <helgaas@kernel.org> > Cc: Pali Rohár <pali@kernel.org> > Cc: Bharat Kumar Gogada <bharat.kumar.gogada@xilinx.com> > Cc: Michal Simek <michal.simek@xilinx.com> > Cc: Yao Hongbo <yaohongbo@linux.alibaba.com> > Cc: Naveen Naidu <naveennaidu479@gmail.com> Reviewed-by: Pali Rohár <pali@kernel.org> > --- > v2: > - New patch > > drivers/pci/pcie/aer.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c > index 9fa1f97e5b27..385e2033d7b5 100644 > --- a/drivers/pci/pcie/aer.c > +++ b/drivers/pci/pcie/aer.c > @@ -1216,7 +1216,8 @@ static int set_device_error_reporting(struct pci_dev *dev, void *data) > if ((type == PCI_EXP_TYPE_ROOT_PORT) || > (type == PCI_EXP_TYPE_RC_EC) || > (type == PCI_EXP_TYPE_UPSTREAM) || > - (type == PCI_EXP_TYPE_DOWNSTREAM)) { > + (type == PCI_EXP_TYPE_DOWNSTREAM) || > + (type == PCI_EXP_TYPE_ENDPOINT)) { Hm... maybe another question to discussion: Why enabling of AER is limited just to above PCIe port types? Why we do not want to enable it for _all_ PCIe devices? Currently in the above list are missing Legacy endpoints (which probably do not support AER and so do not have AER capability in config space), Root Complex Integrated Endpoints (these should provide AER supports too, right?), PCIe to PCI/X Bridges (these may generate its own AER errors) and PCI to PCIe Bridges (these are maybe complicated as subtree behind such bridges are regular PCIe devices and so could fully support AER but on legacy PCI bus there is probably no access to extended config space where is AER). But in all of these cases, are there any issues with enabling AER via function pci_enable_pcie_error_reporting()? For me it looks like that in the worst case dev just does not have AER capability in config space or extended config space is not accessible (which is same as no AER capability). > if (enable) > pci_enable_pcie_error_reporting(dev); > else > -- > 2.34.1 > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2 2/2] PCI/AER: Enable AER on Endpoints as well 2022-01-17 9:30 ` Pali Rohár @ 2022-01-17 9:39 ` Stefan Roese 0 siblings, 0 replies; 6+ messages in thread From: Stefan Roese @ 2022-01-17 9:39 UTC (permalink / raw) To: Pali Rohár Cc: linux-pci, Bjorn Helgaas, Bharat Kumar Gogada, Michal Simek, Yao Hongbo, Naveen Naidu On 1/17/22 10:30, Pali Rohár wrote: > On Monday 17 January 2022 09:03:48 Stefan Roese wrote: >> Currently, the PCIe AER subsystem does not enable AER in the PCIe >> Endpoints via the Device Control register. It's only done for the >> Root Port and all PCIe Ports in between the Root Port and the >> Endpoint(s). Some device drivers enable AER in their PCIe device by >> directly calling pci_enable_pcie_error_reporting(). But in most >> cases, AER is currently disabled in the PCIe Endpoints. >> >> This patch enables AER on PCIe Endpoints now as well in >> set_device_error_reporting(). This will make the ad-hoc calls to >> pci_enable_pcie_error_reporting() superfluous. >> >> Signed-off-by: Stefan Roese <sr@denx.de> >> Cc: Bjorn Helgaas <helgaas@kernel.org> >> Cc: Pali Rohár <pali@kernel.org> >> Cc: Bharat Kumar Gogada <bharat.kumar.gogada@xilinx.com> >> Cc: Michal Simek <michal.simek@xilinx.com> >> Cc: Yao Hongbo <yaohongbo@linux.alibaba.com> >> Cc: Naveen Naidu <naveennaidu479@gmail.com> > > Reviewed-by: Pali Rohár <pali@kernel.org> > >> --- >> v2: >> - New patch >> >> drivers/pci/pcie/aer.c | 3 ++- >> 1 file changed, 2 insertions(+), 1 deletion(-) >> >> diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c >> index 9fa1f97e5b27..385e2033d7b5 100644 >> --- a/drivers/pci/pcie/aer.c >> +++ b/drivers/pci/pcie/aer.c >> @@ -1216,7 +1216,8 @@ static int set_device_error_reporting(struct pci_dev *dev, void *data) >> if ((type == PCI_EXP_TYPE_ROOT_PORT) || >> (type == PCI_EXP_TYPE_RC_EC) || >> (type == PCI_EXP_TYPE_UPSTREAM) || >> - (type == PCI_EXP_TYPE_DOWNSTREAM)) { >> + (type == PCI_EXP_TYPE_DOWNSTREAM) || >> + (type == PCI_EXP_TYPE_ENDPOINT)) { > > Hm... maybe another question to discussion: Why enabling of AER is > limited just to above PCIe port types? Why we do not want to enable it > for _all_ PCIe devices? Currently in the above list are missing Legacy > endpoints (which probably do not support AER and so do not have AER > capability in config space), Root Complex Integrated Endpoints (these > should provide AER supports too, right?), PCIe to PCI/X Bridges (these > may generate its own AER errors) and PCI to PCIe Bridges (these are > maybe complicated as subtree behind such bridges are regular PCIe > devices and so could fully support AER but on legacy PCI bus there is > probably no access to extended config space where is AER). But in all of > these cases, are there any issues with enabling AER via function > pci_enable_pcie_error_reporting()? For me it looks like that in the > worst case dev just does not have AER capability in config space or > extended config space is not accessible (which is same as no AER > capability). I also had similar thoughts on this and was a bit unsure here. Perhaps Bjorn can also comment. But... ... I noticed that AER is still disabled for hot-plugged PCIe devices. As this code patch will not get called in this HP case. Right now I'm testing with this patch here, which could be used instead of this one from this mail: PCI/AER: Enable AER on all PCIe devices supporting it With this change, AER is now also enabled for hot-plugged PCIe devices as pci_aer_init() is also called upon hot-plugging of a PCIe device. When "pci=noaer" is selected, AER stays disabled of course. Signed-off-by: Stefan Roese <sr@denx.de> diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index 385e2033d7b5..94107309ef2d 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -387,6 +387,10 @@ void pci_aer_init(struct pci_dev *dev) pci_add_ext_cap_save_buffer(dev, PCI_EXT_CAP_ID_ERR, sizeof(u32) * n); pci_aer_clear_status(dev); + + /* Enable AER if requested */ + if (pci_aer_available()) + pci_enable_pcie_error_reporting(dev); } This has the same effect of enabling AER in each PCIe Endpoint and also is available for hot-plugged Endpoints. If nobody objects, I'll probably use this patch as patch 2/2 in the next patchset version. Comments welcome as always. Thanks, Stefan ^ permalink raw reply related [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-01-17 9:39 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-01-17 8:03 [PATCH v2 0/2] Fully enable AER Stefan Roese 2022-01-17 8:03 ` [PATCH v2 1/2] PCI/portdrv: Don't disable AER reporting in get_port_device_capability() Stefan Roese 2022-01-17 9:21 ` Pali Rohár 2022-01-17 8:03 ` [PATCH v2 2/2] PCI/AER: Enable AER on Endpoints as well Stefan Roese 2022-01-17 9:30 ` Pali Rohár 2022-01-17 9:39 ` Stefan Roese
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.