From: Kishon Vijay Abraham I <kishon@ti.com> To: Gustavo Pimentel <Gustavo.Pimentel@synopsys.com>, Andrew Murray <andrew.murray@arm.com>, Xiaowei Bao <xiaowei.bao@nxp.com> Cc: "mark.rutland@arm.com" <mark.rutland@arm.com>, Roy Zang <roy.zang@nxp.com>, "lorenzo.pieralisi@arm.co" <lorenzo.pieralisi@arm.co>, "arnd@arndb.de" <arnd@arndb.de>, "devicetree@vger.kernel.org" <devicetree@vger.kernel.org>, "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>, "linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>, "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, Leo Li <leoyang.li@nxp.com>, "M.h. Lian" <minghuan.lian@nxp.com>, "robh+dt@kernel.org" <robh+dt@kernel.org>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>, "jingoohan1@gmail.com" <jingoohan1@gmail.com>, "bhelgaas@google.com" <bhelgaas@google.com>, "shawnguo@kernel.org" <shawnguo@kernel.org>, Mingkai Hu <mingkai.hu@nxp.com> Subject: Re: [PATCH v2 07/10] PCI: layerscape: Modify the MSIX to the doorbell way Date: Wed, 6 Nov 2019 19:09:05 +0530 Message-ID: <e34708a1-1116-89f9-c3f8-7f21b63c9d9c@ti.com> (raw) In-Reply-To: <DM6PR12MB40107A9B97A8DAF32A4C651EDA790@DM6PR12MB4010.namprd12.prod.outlook.com> Gustavo, On 06/11/19 3:10 PM, Gustavo Pimentel wrote: > On Thu, Aug 29, 2019 at 6:13:18, Kishon Vijay Abraham I <kishon@ti.com> > wrote: > > Hi, this email slip away from my attention... > >> Gustavo, >> >> On 27/08/19 6:55 PM, Andrew Murray wrote: >>> On Sat, Aug 24, 2019 at 12:08:40AM +0000, Xiaowei Bao wrote: >>>> >>>> >>>>> -----Original Message----- >>>>> From: Andrew Murray <andrew.murray@arm.com> >>>>> Sent: 2019年8月23日 21:58 >>>>> To: Xiaowei Bao <xiaowei.bao@nxp.com> >>>>> Cc: bhelgaas@google.com; robh+dt@kernel.org; mark.rutland@arm.com; >>>>> shawnguo@kernel.org; Leo Li <leoyang.li@nxp.com>; kishon@ti.com; >>>>> lorenzo.pieralisi@arm.co; arnd@arndb.de; gregkh@linuxfoundation.org; M.h. >>>>> Lian <minghuan.lian@nxp.com>; Mingkai Hu <mingkai.hu@nxp.com>; Roy >>>>> Zang <roy.zang@nxp.com>; jingoohan1@gmail.com; >>>>> gustavo.pimentel@synopsys.com; linux-pci@vger.kernel.org; >>>>> devicetree@vger.kernel.org; linux-kernel@vger.kernel.org; >>>>> linux-arm-kernel@lists.infradead.org; linuxppc-dev@lists.ozlabs.org >>>>> Subject: Re: [PATCH v2 07/10] PCI: layerscape: Modify the MSIX to the >>>>> doorbell way >>>>> >>>>> On Thu, Aug 22, 2019 at 07:22:39PM +0800, Xiaowei Bao wrote: >>>>>> The layerscape platform use the doorbell way to trigger MSIX interrupt >>>>>> in EP mode. >>>>>> >>>>> >>>>> I have no problems with this patch, however... >>>>> >>>>> Are you able to add to this message a reason for why you are making this >>>>> change? Did dw_pcie_ep_raise_msix_irq not work when func_no != 0? Or did >>>>> it work yet dw_pcie_ep_raise_msix_irq_doorbell is more efficient? >>>> >>>> The fact is that, this driver is verified in ls1046a platform of NXP before, and ls1046a don't >>>> support MSIX feature, so I set the msix_capable of pci_epc_features struct is false, >>>> but in other platform, e.g. ls1088a, it support the MSIX feature, I verified the MSIX >>>> feature in ls1088a, it is not OK, so I changed to another way. Thanks. >>> >>> Right, so the existing pci-layerscape-ep.c driver never supported MSIX yet it >>> erroneously had a switch case statement to call dw_pcie_ep_raise_msix_irq which >>> would never get used. >>> >>> Now that we're adding a platform with MSIX support the existing >>> dw_pcie_ep_raise_msix_irq doesn't work (for this platform) so we are adding a >>> different method. >> >> Gustavo, can you confirm dw_pcie_ep_raise_msix_irq() works for designware as it >> didn't work for both me and Xiaowei? > > When I implemented the dw_pcie_ep_raise_msix_irq(), the implementation > was working quite fine on DesignWare solution. Otherwise, I wouldn't > submit it to the kernel. > From what I have seen and if I recall well, Xiaowei implementation was > done having PF's configurated on his solution, which is a configuration > that I don't have in my solution, I believe this could be the missing > piece that differs between our 2 implementations. I haven't debugged the issue yet but in my understanding the MSI-X table should be in the memory (DDR) of EP system. This table will be populated by RC while configuring MSI-X (with msg address and msg data). The EP will use the populated msg address and msg data for raising MSI-X interrupt. From the dw_pcie_ep_raise_msix_irq() (copied below), nowhere the MSI-X table is being read from the memory of EP system. I've given my comments below. int dw_pcie_ep_raise_msix_irq(struct dw_pcie_ep *ep, u8 func_no, u16 interrupt_num) { . . reg = PCI_BASE_ADDRESS_0 + (4 * bir); bar_addr_upper = 0; bar_addr_lower = dw_pcie_readl_dbi(pci, reg); BAR register will hold the "PCI address" programmed by the host. So "bar_addr_lower" will have PCI address. reg_u64 = (bar_addr_lower & PCI_BASE_ADDRESS_MEM_TYPE_MASK); if (reg_u64 == PCI_BASE_ADDRESS_MEM_TYPE_64) bar_addr_upper = dw_pcie_readl_dbi(pci, reg + 4); tbl_addr = ((u64) bar_addr_upper) << 32 | bar_addr_lower; The "tbl_addr" now has the PCI address programmed by the host. tbl_addr += (tbl_offset + ((interrupt_num - 1) * PCI_MSIX_ENTRY_SIZE)); tbl_addr &= PCI_BASE_ADDRESS_MEM_MASK; msix_tbl = ioremap_nocache(ep->phys_base + tbl_addr, PCI_MSIX_ENTRY_SIZE); "ep->phys_base" will have EPs outbound memory address and "tbl_addr" will have PCI address. So msix_tbl points to the EPs outbound memory region. if (!msix_tbl) return -EINVAL; msg_addr_lower = readl(msix_tbl + PCI_MSIX_ENTRY_LOWER_ADDR); msg_addr_upper = readl(msix_tbl + PCI_MSIX_ENTRY_UPPER_ADDR); Here an access to the EP outbound region is made (and the transaction will be based on ATU configuration). The message address should ideally be obtained from the MSI-X table present in the EP system. There need not be any access to the OB region for getting data from MSI-X table. msg_addr = ((u64) msg_addr_upper) << 32 | msg_addr_lower; msg_data = readl(msix_tbl + PCI_MSIX_ENTRY_DATA); vec_ctrl = readl(msix_tbl + PCI_MSIX_ENTRY_VECTOR_CTRL); All this should be obtained from the memory of EP. . . } I'm not sure how this worked for you. Thanks Kishon > > Since patch submission into the kernel related to msix feature on pcitest > tool, I didn't touch or re-tested the msix feature by lack of time (other > projects requires my full attention for now). However is on my roadmap to > came back to add some other features on DesignWare eDMA driver and I can > do at that time some tests to see if the > dw_pcie_ep_raise_msix_irq_doorbell() is compatible or not with my > solution. If so, I can do some patch to simplify and use the > dw_pcie_ep_raise_msix_irq_doorbell() if it still works as expected like > on dw_pcie_ep_raise_msix_irq(). Agree? > > Gustavo > >> >> Thanks >> Kishon > > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply index Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-08-22 11:22 [PATCH v2 01/10] PCI: designware-ep: Add multiple PFs support for DWC Xiaowei Bao 2019-08-22 11:22 ` [PATCH v2 02/10] PCI: designware-ep: Add the doorbell mode of MSI-X in EP mode Xiaowei Bao 2019-08-23 13:35 ` Andrew Murray 2019-08-23 23:51 ` Xiaowei Bao 2019-08-22 11:22 ` [PATCH v2 03/10] PCI: designware-ep: Move the function of getting MSI capability forward Xiaowei Bao 2019-08-23 13:38 ` Andrew Murray 2019-08-24 0:20 ` Xiaowei Bao 2019-08-22 11:22 ` [PATCH v2 04/10] dt-bindings: pci: layerscape-pci: add compatible strings for ls1088a and ls2088a Xiaowei Bao 2019-08-27 22:26 ` Rob Herring 2019-08-29 9:19 ` Xiaowei Bao 2019-08-22 11:22 ` [PATCH v2 05/10] PCI: layerscape: Fix some format issue of the code Xiaowei Bao 2019-08-23 13:45 ` Andrew Murray 2019-08-24 0:00 ` Xiaowei Bao 2019-08-22 11:22 ` [PATCH v2 06/10] PCI: layerscape: Modify the way of getting capability with different PEX Xiaowei Bao 2019-08-22 11:43 ` Kishon Vijay Abraham I 2019-08-23 2:39 ` Xiaowei Bao 2019-08-23 3:39 ` Kishon Vijay Abraham I 2019-08-23 4:13 ` Xiaowei Bao 2019-09-02 13:36 ` Andrew Murray 2019-09-03 2:11 ` Xiaowei Bao 2019-08-22 11:22 ` [PATCH v2 07/10] PCI: layerscape: Modify the MSIX to the doorbell way Xiaowei Bao 2019-08-23 13:58 ` Andrew Murray 2019-08-24 0:08 ` Xiaowei Bao 2019-08-27 13:25 ` Andrew Murray 2019-08-28 2:49 ` Xiaowei Bao 2019-08-29 5:13 ` Kishon Vijay Abraham I 2019-11-05 12:37 ` Lorenzo Pieralisi 2019-11-06 9:33 ` Xiaowei Bao 2019-11-06 9:40 ` Gustavo Pimentel 2019-11-06 10:03 ` Xiaowei Bao 2019-11-06 13:39 ` Kishon Vijay Abraham I [this message] 2019-11-06 15:40 ` Gustavo Pimentel 2019-08-22 11:22 ` [PATCH v2 08/10] PCI: layerscape: Add EP mode support for ls1088a and ls2088a Xiaowei Bao 2019-08-23 14:27 ` Andrew Murray 2019-08-24 0:18 ` Xiaowei Bao 2019-08-24 6:45 ` christophe leroy 2019-08-25 3:07 ` Xiaowei Bao 2019-08-27 14:48 ` Andrew Murray 2019-08-28 3:25 ` Xiaowei Bao 2019-08-26 9:49 ` Xiaowei Bao 2019-08-27 13:34 ` Andrew Murray 2019-08-28 4:29 ` Xiaowei Bao 2019-08-28 9:01 ` Andrew Murray 2019-08-29 2:03 ` Xiaowei Bao 2019-08-22 11:22 ` [PATCH v2 09/10] arm64: dts: layerscape: Add PCIe EP node for ls1088a Xiaowei Bao 2019-08-22 11:22 ` [PATCH v2 10/10] misc: pci_endpoint_test: Add LS1088a in pci_device_id table Xiaowei Bao 2019-08-23 13:25 ` [PATCH v2 01/10] PCI: designware-ep: Add multiple PFs support for DWC Andrew Murray 2019-08-23 23:50 ` Xiaowei Bao 2019-08-27 13:10 ` Andrew Murray 2019-08-28 7:22 ` Xiaowei Bao
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=e34708a1-1116-89f9-c3f8-7f21b63c9d9c@ti.com \ --to=kishon@ti.com \ --cc=Gustavo.Pimentel@synopsys.com \ --cc=andrew.murray@arm.com \ --cc=arnd@arndb.de \ --cc=bhelgaas@google.com \ --cc=devicetree@vger.kernel.org \ --cc=gregkh@linuxfoundation.org \ --cc=jingoohan1@gmail.com \ --cc=leoyang.li@nxp.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-pci@vger.kernel.org \ --cc=linuxppc-dev@lists.ozlabs.org \ --cc=lorenzo.pieralisi@arm.co \ --cc=mark.rutland@arm.com \ --cc=minghuan.lian@nxp.com \ --cc=mingkai.hu@nxp.com \ --cc=robh+dt@kernel.org \ --cc=roy.zang@nxp.com \ --cc=shawnguo@kernel.org \ --cc=xiaowei.bao@nxp.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Linux-ARM-Kernel Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/linux-arm-kernel/0 linux-arm-kernel/git/0.git git clone --mirror https://lore.kernel.org/linux-arm-kernel/1 linux-arm-kernel/git/1.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 linux-arm-kernel linux-arm-kernel/ https://lore.kernel.org/linux-arm-kernel \ linux-arm-kernel@lists.infradead.org public-inbox-index linux-arm-kernel Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.infradead.lists.linux-arm-kernel AGPL code for this site: git clone https://public-inbox.org/public-inbox.git