[5.2 regression] rtwpci + amd iommu

[5.2 regression] rtwpci + amd iommu

[Ján Veselý]

Hi,

after updating to 5.2 the wi-fi driver stopped working when the iommu
is enabled.
It fails to list the available wi-fi networks or connect to a known one.
booting with amd_iommu=off works around the problem.
The staging version in 5.1 and older worked OK with the iommu enabled

The device is:
04:00.0 Network controller: Realtek Semiconductor Co., Ltd. RTL8822BE
802.11a/b/g/n/ac WiFi adapter

I do see an IOMMU error in dmesg, but the originating device does not
match the nics pci location:
Jul 08 15:03:14 host kernel: rtw_pci 0000:04:00.0: start vif
xx:xx:xx:xx:xx:xx on port 0
Jul 08 15:03:14 host kernel: iommu ivhd0: AMD-Vi: Event logged
[INVALID_DEVICE_REQUEST device=00:00.1 pasid=0x00000
address=0xfffffffdf8140200 flags=0x0a00]
Jul 08 15:03:14 host kernel: rtw_pci 0000:04:00.0: stop vif
xx:xx:xx:xx:xx:xx on port 0
Jul 08 15:03:14 host NetworkManager[790]: <info>  [1562612594.8992]
device (wlp4s0): set-hw-addr: set MAC address to yy:yy:yy:yy:yy:yy
(scanning)
Jul 08 15:03:15 host kernel: rtw_pci 0000:04:00.0: start vif
yy:yy:yy:yy:yy:yy on port 0

let me know if I should provide any further info.
thanks,
Jan

RE: [5.2 regression] rtwpci + amd iommu

[Tony Chuang]

[-- Attachment #1: Type: text/plain, Size: 1390 bytes --]

> Hi,
> 
> after updating to 5.2 the wi-fi driver stopped working when the iommu
> is enabled.
> It fails to list the available wi-fi networks or connect to a known one.
> booting with amd_iommu=off works around the problem.
> The staging version in 5.1 and older worked OK with the iommu enabled
> 
> The device is:
> 04:00.0 Network controller: Realtek Semiconductor Co., Ltd. RTL8822BE
> 802.11a/b/g/n/ac WiFi adapter
> 
> I do see an IOMMU error in dmesg, but the originating device does not
> match the nics pci location:
> Jul 08 15:03:14 host kernel: rtw_pci 0000:04:00.0: start vif
> xx:xx:xx:xx:xx:xx on port 0
> Jul 08 15:03:14 host kernel: iommu ivhd0: AMD-Vi: Event logged
> [INVALID_DEVICE_REQUEST device=00:00.1 pasid=0x00000
> address=0xfffffffdf8140200 flags=0x0a00]
> Jul 08 15:03:14 host kernel: rtw_pci 0000:04:00.0: stop vif
> xx:xx:xx:xx:xx:xx on port 0
> Jul 08 15:03:14 host NetworkManager[790]: <info>  [1562612594.8992]
> device (wlp4s0): set-hw-addr: set MAC address to yy:yy:yy:yy:yy:yy
> (scanning)
> Jul 08 15:03:15 host kernel: rtw_pci 0000:04:00.0: start vif
> yy:yy:yy:yy:yy:yy on port 0
> 
> let me know if I should provide any further info.
> thanks,
> Jan
> 


Hi,

I am not sure if enabling MSI interrupt is going to help.
You can try the patch attached, if it works, I will send it to upstream.
Thanks

Yan-Hsuan

[-- Attachment #2: 0001-rtw88-pci-add-MSI-interrupt-support.patch --]
[-- Type: application/octet-stream, Size: 5082 bytes --]

From 570d6ed8585855a8c6aee20e6d319829ebef6bc7 Mon Sep 17 00:00:00 2001
From: Yu-Yen Ting <steventing@realtek.com>
Date: Wed, 29 May 2019 16:31:27 +0800
Subject: [PATCH] rtw88: pci: add MSI interrupt support

The MSI interrupt should be enabled on certain platform.
Add the module parameter disable_msi to force disable the MSI interrupt

Signed-off-by: Yu-Yen Ting <steventing@realtek.com>
Signed-off-by: Yan-Hsuan Chuang <yhchuang@realtek.com>
---
 drivers/net/wireless/realtek/rtw88/main.c |  5 ++++
 drivers/net/wireless/realtek/rtw88/main.h |  1 +
 drivers/net/wireless/realtek/rtw88/pci.c  | 47 +++++++++++++++++++++++++++++--
 drivers/net/wireless/realtek/rtw88/pci.h  |  1 +
 4 files changed, 51 insertions(+), 3 deletions(-)

diff --git a/drivers/net/wireless/realtek/rtw88/main.c b/drivers/net/wireless/realtek/rtw88/main.c
index b8e0519..d86e05e 100644
--- a/drivers/net/wireless/realtek/rtw88/main.c
+++ b/drivers/net/wireless/realtek/rtw88/main.c
@@ -15,13 +15,18 @@
 
 static bool rtw_fw_support_lps;
 unsigned int rtw_debug_mask;
+bool rtw_disable_msi;
+
 EXPORT_SYMBOL(rtw_debug_mask);
+EXPORT_SYMBOL(rtw_disable_msi);
 
 module_param_named(support_lps, rtw_fw_support_lps, bool, 0644);
 module_param_named(debug_mask, rtw_debug_mask, uint, 0644);
+module_param_named(disable_msi, rtw_disable_msi, bool, 0644);
 
 MODULE_PARM_DESC(support_lps, "Set Y to enable Leisure Power Save support, to turn radio off between beacons");
 MODULE_PARM_DESC(debug_mask, "Debugging mask");
+MODULE_PARM_DESC(disable_msi, "Set Y to disable MSI interrupt support");
 
 static struct ieee80211_channel rtw_channeltable_2g[] = {
 	{.center_freq = 2412, .hw_value = 1,},
diff --git a/drivers/net/wireless/realtek/rtw88/main.h b/drivers/net/wireless/realtek/rtw88/main.h
index 92730507..ca603ba 100644
--- a/drivers/net/wireless/realtek/rtw88/main.h
+++ b/drivers/net/wireless/realtek/rtw88/main.h
@@ -28,6 +28,7 @@
 #define HW_FEATURE_LEN			13
 
 extern unsigned int rtw_debug_mask;
+extern bool rtw_disable_msi;
 extern const struct ieee80211_ops rtw_ops;
 extern struct rtw_chip_info rtw8822b_hw_spec;
 extern struct rtw_chip_info rtw8822c_hw_spec;
diff --git a/drivers/net/wireless/realtek/rtw88/pci.c b/drivers/net/wireless/realtek/rtw88/pci.c
index 353871c..817b4ab 100644
--- a/drivers/net/wireless/realtek/rtw88/pci.c
+++ b/drivers/net/wireless/realtek/rtw88/pci.c
@@ -861,6 +861,7 @@ static irqreturn_t rtw_pci_interrupt_handler(int irq, void *dev)
 	if (!rtwpci->irq_enabled)
 		goto out;
 
+	rtw_pci_disable_interrupt(rtwdev, rtwpci);
 	rtw_pci_irq_recognized(rtwdev, rtwpci, irq_status);
 
 	if (irq_status[0] & IMR_MGNTDOK)
@@ -880,6 +881,8 @@ static irqreturn_t rtw_pci_interrupt_handler(int irq, void *dev)
 	if (irq_status[0] & IMR_ROK)
 		rtw_pci_rx_isr(rtwdev, rtwpci, RTW_RX_QUEUE_MPDU);
 
+	rtw_pci_enable_interrupt(rtwdev, rtwpci);
+
 out:
 	spin_unlock(&rtwpci->irq_lock);
 
@@ -1090,6 +1093,45 @@ static struct rtw_hci_ops rtw_pci_ops = {
 	.write_data_h2c = rtw_pci_write_data_h2c,
 };
 
+static int rtw_pci_request_irq(struct rtw_dev *rtwdev, struct pci_dev *pdev)
+{
+	struct rtw_pci *rtwpci = (struct rtw_pci *)rtwdev->priv;
+	int ret;
+
+	if (!rtw_disable_msi) {
+		ret = pci_enable_msi(pdev);
+		if (ret) {
+			rtw_warn(rtwdev, "failed to enable msi, using legacy irq\n");
+		} else {
+			rtw_warn(rtwdev, "pci msi enabled\n");
+			rtwpci->msi_enabled = true;
+		}
+	}
+
+	ret = request_irq(pdev->irq, &rtw_pci_interrupt_handler, IRQF_SHARED,
+			  KBUILD_MODNAME, rtwdev);
+	if (ret) {
+		rtw_err(rtwdev, "failed to request irq\n");
+		if (rtwpci->msi_enabled) {
+			pci_disable_msi(pdev);
+			rtwpci->msi_enabled = false;
+		}
+	}
+
+	return ret;
+}
+
+static void rtw_pci_free_irq(struct rtw_dev *rtwdev, struct pci_dev *pdev)
+{
+	struct rtw_pci *rtwpci = (struct rtw_pci *)rtwdev->priv;
+
+	free_irq(pdev->irq, rtwdev);
+	if (rtwpci->msi_enabled) {
+		pci_disable_msi(pdev);
+		rtwpci->msi_enabled = false;
+	}
+}
+
 static int rtw_pci_probe(struct pci_dev *pdev,
 			 const struct pci_device_id *id)
 {
@@ -1144,8 +1186,7 @@ static int rtw_pci_probe(struct pci_dev *pdev,
 		goto err_destroy_pci;
 	}
 
-	ret = request_irq(pdev->irq, &rtw_pci_interrupt_handler,
-			  IRQF_SHARED, KBUILD_MODNAME, rtwdev);
+	ret = rtw_pci_request_irq(rtwdev, pdev);
 	if (ret) {
 		ieee80211_unregister_hw(hw);
 		goto err_destroy_pci;
@@ -1184,7 +1225,7 @@ static void rtw_pci_remove(struct pci_dev *pdev)
 	rtw_pci_disable_interrupt(rtwdev, rtwpci);
 	rtw_pci_destroy(rtwdev, pdev);
 	rtw_pci_declaim(rtwdev, pdev);
-	free_irq(rtwpci->pdev->irq, rtwdev);
+	rtw_pci_free_irq(rtwdev, pdev);
 	rtw_core_deinit(rtwdev);
 	ieee80211_free_hw(hw);
 }
diff --git a/drivers/net/wireless/realtek/rtw88/pci.h b/drivers/net/wireless/realtek/rtw88/pci.h
index 87824a4..a8e369c 100644
--- a/drivers/net/wireless/realtek/rtw88/pci.h
+++ b/drivers/net/wireless/realtek/rtw88/pci.h
@@ -186,6 +186,7 @@ struct rtw_pci {
 	spinlock_t irq_lock;
 	u32 irq_mask[4];
 	bool irq_enabled;
+	bool msi_enabled;
 
 	u16 rx_tag;
 	struct rtw_pci_tx_ring tx_rings[RTK_MAX_TX_QUEUE_NUM];
-- 
2.7.4

Re: [5.2 regression] rtwpci + amd iommu

[Ján Veselý]

Hi,

thanks for a prompt reply.
the patch applies with minor changes (the LPS mod param message and -2
lines offsets), but it works.
The system now lists available wi-fi networks and connects to known
networks all while the iommu is enabled.

feel free to add my tested-by,
regards,
Jan

On Wed, Jul 10, 2019 at 1:12 AM Tony Chuang <yhchuang@realtek.com> wrote:
>
> > Hi,
> >
> > after updating to 5.2 the wi-fi driver stopped working when the iommu
> > is enabled.
> > It fails to list the available wi-fi networks or connect to a known one.
> > booting with amd_iommu=off works around the problem.
> > The staging version in 5.1 and older worked OK with the iommu enabled
> >
> > The device is:
> > 04:00.0 Network controller: Realtek Semiconductor Co., Ltd. RTL8822BE
> > 802.11a/b/g/n/ac WiFi adapter
> >
> > I do see an IOMMU error in dmesg, but the originating device does not
> > match the nics pci location:
> > Jul 08 15:03:14 host kernel: rtw_pci 0000:04:00.0: start vif
> > xx:xx:xx:xx:xx:xx on port 0
> > Jul 08 15:03:14 host kernel: iommu ivhd0: AMD-Vi: Event logged
> > [INVALID_DEVICE_REQUEST device=00:00.1 pasid=0x00000
> > address=0xfffffffdf8140200 flags=0x0a00]
> > Jul 08 15:03:14 host kernel: rtw_pci 0000:04:00.0: stop vif
> > xx:xx:xx:xx:xx:xx on port 0
> > Jul 08 15:03:14 host NetworkManager[790]: <info>  [1562612594.8992]
> > device (wlp4s0): set-hw-addr: set MAC address to yy:yy:yy:yy:yy:yy
> > (scanning)
> > Jul 08 15:03:15 host kernel: rtw_pci 0000:04:00.0: start vif
> > yy:yy:yy:yy:yy:yy on port 0
> >
> > let me know if I should provide any further info.
> > thanks,
> > Jan
> >
>
>
> Hi,
>
> I am not sure if enabling MSI interrupt is going to help.
> You can try the patch attached, if it works, I will send it to upstream.
> Thanks
>
> Yan-Hsuan

Re: [5.2 regression] rtwpci + amd iommu

[Brian Norris]

Hi Tony,

Can you please submit that patch upstream soon?

Thanks,
Brian

On Tue, Jul 9, 2019 at 11:10 PM Ján Veselý <jano.vesely@gmail.com> wrote:
...
> feel free to add my tested-by,
...

> On Wed, Jul 10, 2019 at 1:12 AM Tony Chuang <yhchuang@realtek.com> wrote:
...
> > I am not sure if enabling MSI interrupt is going to help.
> > You can try the patch attached, if it works, I will send it to upstream.
> > Thanks
> >
> > Yan-Hsuan

RE: [5.2 regression] rtwpci + amd iommu

[Tony Chuang]

> Hi Tony,
> 
> Can you please submit that patch upstream soon?
> 
> Thanks,
> Brian
> 

Sure! I can submit it today.

Yan-Hsuan