linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [v4] PCI: Add reset quirk for Huawei Intelligent NIC virtual function
@ 2021-03-13  7:29 Chiqijun
  2021-03-13  9:28 ` kernel test robot
  0 siblings, 1 reply; 2+ messages in thread
From: Chiqijun @ 2021-03-13  7:29 UTC (permalink / raw)
  To: bhelgaas
  Cc: linux-pci, linux-kernel, alex.williamson, yin.yinshi,
	cloud.wangxiaoyun, zengweiliang.zengweiliang, chenlizhong

When multiple VFs do FLR at the same time, the firmware is
processed serially, resulting in some VF FLRs being delayed more
than 100ms, when the virtual machine restarts and the device
driver is loaded, the firmware is doing the corresponding VF
FLR, causing the driver to fail to load.

To solve this problem, add host and firmware status synchronization
during FLR.

Signed-off-by: Chiqijun <chiqijun@huawei.com>
---
v4:
 - Addressed Bjorn's review comments

v3:
 - The MSE bit in the VF configuration space is hardwired to zero,
   remove the setting of PCI_COMMAND_MEMORY bit. Add comment for
   set PCI_COMMAND register.

v2:
 - Update comments
 - Use the HINIC_VF_FLR_CAP_BIT_SHIFT and HINIC_VF_FLR_PROC_BIT_SHIFT
   macro instead of the magic number
---
 drivers/pci/quirks.c | 69 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 69 insertions(+)

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 653660e3ba9e..4623125098e0 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -3913,6 +3913,73 @@ static int delay_250ms_after_flr(struct pci_dev *dev, int probe)
 	return 0;
 }
 
+#define PCI_DEVICE_ID_HINIC_VF      0x375E
+#define HINIC_VF_FLR_TYPE           0x1000
+#define HINIC_VF_FLR_CAP_BIT_SHIFT  30
+#define HINIC_VF_OP                 0xE80
+#define HINIC_VF_FLR_PROC_BIT_SHIFT 18
+#define HINIC_OPERATION_TIMEOUT     15000	/* 15 seconds */
+
+/* Device-specific reset method for Huawei Intelligent NIC virtual functions */
+static int reset_hinic_vf_dev(struct pci_dev *pdev, int probe)
+{
+	unsigned long timeout;
+	void __iomem *bar;
+	u32 val;
+
+	if (probe)
+		return 0;
+
+	bar = pci_iomap(pdev, 0, 0);
+	if (!bar)
+		return -ENOTTY;
+
+	/* Get and check firmware capabilities. */
+	val = be32_to_cpu(readl(bar + HINIC_VF_FLR_TYPE));
+	if (!(val & (1UL << HINIC_VF_FLR_CAP_BIT_SHIFT))) {
+		pci_iounmap(pdev, bar);
+		return -ENOTTY;
+	}
+
+	/*
+	 * Set the processing bit for the start of FLR, which will be cleared
+	 * by the firmware after FLR is completed.
+	 */
+	val = be32_to_cpu(readl(bar + HINIC_VF_OP));
+	val = val | (1UL << HINIC_VF_FLR_PROC_BIT_SHIFT);
+	writel(cpu_to_be32(val), bar + HINIC_VF_OP);
+
+	/* Perform the actual device function reset */
+	pcie_flr(pdev);
+
+	/*
+	 * The device must learn BDF after FLR in order to respond to BAR's
+	 * read request, therefore, we issue a configure write request to let
+	 * the device capture BDF.
+	 */
+	pci_write_config_word(pdev, PCI_VENDOR_ID, 0);
+
+	/* Waiting for device reset complete */
+	timeout = jiffies + msecs_to_jiffies(HINIC_OPERATION_TIMEOUT);
+	do {
+		val = be32_to_cpu(readl(bar + HINIC_VF_OP));
+		if (!(val & (1UL << HINIC_VF_FLR_PROC_BIT_SHIFT)))
+			goto reset_complete;
+		msleep(20);
+	} while (time_before(jiffies, timeout));
+
+	val = be32_to_cpu(readl(bar + HINIC_VF_OP));
+	if (!(val & (1UL << HINIC_VF_FLR_PROC_BIT_SHIFT)))
+		goto reset_complete;
+
+	pci_warn(pdev, "Reset dev timeout, flr ack reg: %#010x\n", val);
+
+reset_complete:
+	pci_iounmap(pdev, bar);
+
+	return 0;
+}
+
 static const struct pci_dev_reset_methods pci_dev_reset_methods[] = {
 	{ PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82599_SFP_VF,
 		 reset_intel_82599_sfp_virtfn },
@@ -3924,6 +3991,8 @@ static const struct pci_dev_reset_methods pci_dev_reset_methods[] = {
 	{ PCI_VENDOR_ID_INTEL, 0x0953, delay_250ms_after_flr },
 	{ PCI_VENDOR_ID_CHELSIO, PCI_ANY_ID,
 		reset_chelsio_generic_dev },
+	{ PCI_VENDOR_ID_HUAWEI, PCI_DEVICE_ID_HINIC_VF,
+		reset_hinic_vf_dev },
 	{ 0 }
 };
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 2+ messages in thread

* Re: [v4] PCI: Add reset quirk for Huawei Intelligent NIC virtual function
  2021-03-13  7:29 [v4] PCI: Add reset quirk for Huawei Intelligent NIC virtual function Chiqijun
@ 2021-03-13  9:28 ` kernel test robot
  0 siblings, 0 replies; 2+ messages in thread
From: kernel test robot @ 2021-03-13  9:28 UTC (permalink / raw)
  To: Chiqijun, bhelgaas
  Cc: kbuild-all, linux-pci, linux-kernel, alex.williamson, yin.yinshi,
	cloud.wangxiaoyun, zengweiliang.zengweiliang, chenlizhong

[-- Attachment #1: Type: text/plain, Size: 4487 bytes --]

Hi Chiqijun,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on pci/next]
[also build test WARNING on v5.12-rc2 next-20210312]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Chiqijun/PCI-Add-reset-quirk-for-Huawei-Intelligent-NIC-virtual-function/20210313-152632
base:   https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next
config: sparc-randconfig-s031-20210313 (attached as .config)
compiler: sparc64-linux-gcc (GCC) 9.3.0
reproduce:
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # apt-get install sparse
        # sparse version: v0.6.3-262-g5e674421-dirty
        # https://github.com/0day-ci/linux/commit/2ee50cef940514cc4b80bf8d550cb4f28e257d7a
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Chiqijun/PCI-Add-reset-quirk-for-Huawei-Intelligent-NIC-virtual-function/20210313-152632
        git checkout 2ee50cef940514cc4b80bf8d550cb4f28e257d7a
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' ARCH=sparc 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>


"sparse warnings: (new ones prefixed by >>)"
   drivers/pci/quirks.c:2287:57: sparse: sparse: restricted pci_power_t degrades to integer
   drivers/pci/quirks.c:3938:15: sparse: sparse: cast to restricted __be32
   drivers/pci/quirks.c:3948:15: sparse: sparse: cast to restricted __be32
>> drivers/pci/quirks.c:3950:16: sparse: sparse: incorrect type in argument 1 (different base types) @@     expected unsigned int [usertype] l @@     got restricted __be32 [usertype] @@
   drivers/pci/quirks.c:3950:16: sparse:     expected unsigned int [usertype] l
   drivers/pci/quirks.c:3950:16: sparse:     got restricted __be32 [usertype]
   drivers/pci/quirks.c:3965:23: sparse: sparse: cast to restricted __be32
   drivers/pci/quirks.c:3971:15: sparse: sparse: cast to restricted __be32

vim +3950 drivers/pci/quirks.c

  3922	
  3923	/* Device-specific reset method for Huawei Intelligent NIC virtual functions */
  3924	static int reset_hinic_vf_dev(struct pci_dev *pdev, int probe)
  3925	{
  3926		unsigned long timeout;
  3927		void __iomem *bar;
  3928		u32 val;
  3929	
  3930		if (probe)
  3931			return 0;
  3932	
  3933		bar = pci_iomap(pdev, 0, 0);
  3934		if (!bar)
  3935			return -ENOTTY;
  3936	
  3937		/* Get and check firmware capabilities. */
  3938		val = be32_to_cpu(readl(bar + HINIC_VF_FLR_TYPE));
  3939		if (!(val & (1UL << HINIC_VF_FLR_CAP_BIT_SHIFT))) {
  3940			pci_iounmap(pdev, bar);
  3941			return -ENOTTY;
  3942		}
  3943	
  3944		/*
  3945		 * Set the processing bit for the start of FLR, which will be cleared
  3946		 * by the firmware after FLR is completed.
  3947		 */
  3948		val = be32_to_cpu(readl(bar + HINIC_VF_OP));
  3949		val = val | (1UL << HINIC_VF_FLR_PROC_BIT_SHIFT);
> 3950		writel(cpu_to_be32(val), bar + HINIC_VF_OP);
  3951	
  3952		/* Perform the actual device function reset */
  3953		pcie_flr(pdev);
  3954	
  3955		/*
  3956		 * The device must learn BDF after FLR in order to respond to BAR's
  3957		 * read request, therefore, we issue a configure write request to let
  3958		 * the device capture BDF.
  3959		 */
  3960		pci_write_config_word(pdev, PCI_VENDOR_ID, 0);
  3961	
  3962		/* Waiting for device reset complete */
  3963		timeout = jiffies + msecs_to_jiffies(HINIC_OPERATION_TIMEOUT);
  3964		do {
  3965			val = be32_to_cpu(readl(bar + HINIC_VF_OP));
  3966			if (!(val & (1UL << HINIC_VF_FLR_PROC_BIT_SHIFT)))
  3967				goto reset_complete;
  3968			msleep(20);
  3969		} while (time_before(jiffies, timeout));
  3970	
  3971		val = be32_to_cpu(readl(bar + HINIC_VF_OP));
  3972		if (!(val & (1UL << HINIC_VF_FLR_PROC_BIT_SHIFT)))
  3973			goto reset_complete;
  3974	
  3975		pci_warn(pdev, "Reset dev timeout, flr ack reg: %#010x\n", val);
  3976	
  3977	reset_complete:
  3978		pci_iounmap(pdev, bar);
  3979	
  3980		return 0;
  3981	}
  3982	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 25836 bytes --]

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2021-03-13  9:31 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-13  7:29 [v4] PCI: Add reset quirk for Huawei Intelligent NIC virtual function Chiqijun
2021-03-13  9:28 ` kernel test robot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).