From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.1 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68581C43603 for ; Wed, 4 Dec 2019 08:14:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 332DC20675 for ; Wed, 4 Dec 2019 08:14:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575447242; bh=yguauI0IayAymFt04WWVObdEb9zbkaHw3KCapf9TULg=; h=Date:From:To:Cc:Subject:In-Reply-To:References:List-ID:From; b=jjP6NmSVs5hL44SsEr9fKnLOLPo5MdMEneWRMSGID1kMyvhYGYHhJuSWThFR/1Elw 9JXvyH54dwBiYO3tXZ+Xsu/oBRDPVKwm+FQ1AkpMV/ab6fg44Q/v0+fDHE9SxR4oWZ sKQJl+j8nbmHYMZO5dfhOU68ATm5CfVWT3MzCGCQ= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726679AbfLDIN6 convert rfc822-to-8bit (ORCPT ); Wed, 4 Dec 2019 03:13:58 -0500 Received: from inca-roads.misterjones.org ([213.251.177.50]:45862 "EHLO inca-roads.misterjones.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726053AbfLDIN6 (ORCPT ); Wed, 4 Dec 2019 03:13:58 -0500 Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=big-swifty.misterjones.org) by cheepnis.misterjones.org with esmtpsa (TLSv1.2:AES256-GCM-SHA384:256) (Exim 4.80) (envelope-from ) id 1icPnH-000632-3q; Wed, 04 Dec 2019 09:13:55 +0100 Date: Wed, 04 Dec 2019 08:13:53 +0000 Message-ID: <86k17czewu.wl-maz@kernel.org> From: Marc Zyngier To: Xiaowei Bao Cc: Robin Murphy , Roy Zang , "lorenzo.pieralisi@arm.com" , "devicetree@vger.kernel.org" , "linux-pci@vger.kernel.org" , "Z.q. Hou" , "linux-kernel@vger.kernel.org" , "M.h. Lian" , "robh+dt@kernel.org" , "linux-arm-kernel@lists.infradead.org" , "bhelgaas@google.com" , "andrew.murray@arm.com" , "frowand.list@gmail.com" , Mingkai Hu Subject: Re: [PATCH] PCI: layerscape: Add the SRIOV support in host side In-Reply-To: References: <20191202104506.27916-1-xiaowei.bao@nxp.com> <606a00a2edcf077aa868319e0daa4dbc@www.loen.fr> <3dcdf44eb76390730658e3f4d932620c@www.loen.fr> <8f56c2d9-ab01-a91e-902f-a61def0e8ce8@arm.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL/10.8 EasyPG/1.0.0 Emacs/26 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: xiaowei.bao@nxp.com, robin.murphy@arm.com, roy.zang@nxp.com, lorenzo.pieralisi@arm.com, devicetree@vger.kernel.org, linux-pci@vger.kernel.org, zhiqiang.hou@nxp.com, linux-kernel@vger.kernel.org, minghuan.lian@nxp.com, robh+dt@kernel.org, linux-arm-kernel@lists.infradead.org, bhelgaas@google.com, andrew.murray@arm.com, frowand.list@gmail.com, mingkai.hu@nxp.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on cheepnis.misterjones.org); SAEximRunCond expanded to false Sender: devicetree-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org On Wed, 04 Dec 2019 04:34:32 +0000, Xiaowei Bao wrote: > > > > > -----Original Message----- > > From: Robin Murphy > > Sent: 2019年12月3日 23:20 > > To: Marc Zyngier ; Xiaowei Bao > > Cc: Roy Zang ; lorenzo.pieralisi@arm.com; > > devicetree@vger.kernel.org; linux-pci@vger.kernel.org; Z.q. Hou > > ; linux-kernel@vger.kernel.org; M.h. Lian > > ; robh+dt@kernel.org; > > linux-arm-kernel@lists.infradead.org; bhelgaas@google.com; > > andrew.murray@arm.com; frowand.list@gmail.com; Mingkai Hu > > > > Subject: Re: [PATCH] PCI: layerscape: Add the SRIOV support in host side > > > > On 03/12/2019 11:51 am, Marc Zyngier wrote: > > > On 2019-12-03 01:42, Xiaowei Bao wrote: > > >>> -----Original Message----- > > >>> From: Marc Zyngier > > >>> Sent: 2019年12月2日 20:48 > > >>> To: Xiaowei Bao > > >>> Cc: robh+dt@kernel.org; frowand.list@gmail.com; M.h. Lian > > >>> ; Mingkai Hu ; Roy > > Zang > > >>> ; lorenzo.pieralisi@arm.com; > > >>> andrew.murray@arm.com; bhelgaas@google.com; > > >>> devicetree@vger.kernel.org; linux-kernel@vger.kernel.org; > > >>> linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org; > > >>> Z.q. Hou > > >>> Subject: Re: [PATCH] PCI: layerscape: Add the SRIOV support in host > > >>> side > > >>> > > >>> On 2019-12-02 10:45, Xiaowei Bao wrote: > > >>> > GIC get the map relations of devid and stream id from the msi-map > > >>> > property of DTS, our platform add this property in u-boot base on > > >>> > the PCIe device in the bus, but if enable the vf device in kernel, > > >>> > the vf device msi-map will not set, so the vf device can't work, > > >>> > this patch purpose is that manage the stream id and device id map > > >>> > relations dynamically in kernel, and make the new PCIe device work in > > kernel. > > >>> > > > >>> > Signed-off-by: Xiaowei Bao > > >>> > --- > > >>> >  drivers/of/irq.c                            |  9 +++ > > >>> >  drivers/pci/controller/dwc/pci-layerscape.c | 94 > > >>> > +++++++++++++++++++++++++++++ > > >>> >  drivers/pci/probe.c                         |  6 ++ > > >>> >  drivers/pci/remove.c                        |  6 ++ > > >>> >  4 files changed, 115 insertions(+) > > >>> > > > >>> > diff --git a/drivers/of/irq.c b/drivers/of/irq.c index > > >>> > a296eaf..791e609 100644 > > >>> > --- a/drivers/of/irq.c > > >>> > +++ b/drivers/of/irq.c > > >>> > @@ -576,6 +576,11 @@ void __init of_irq_init(const struct > > >>> >of_device_id > > >>> > *matches) > > >>> >      } > > >>> >  } > > >>> > > > >>> > +u32 __weak ls_pcie_streamid_fix(struct device *dev, u32 rid) { > > >>> > +    return rid; > > >>> > +} > > >>> > + > > >>> >  static u32 __of_msi_map_rid(struct device *dev, struct > > >>> >device_node **np, > > >>> >                  u32 rid_in) > > >>> >  { > > >>> > @@ -590,6 +595,10 @@ static u32 __of_msi_map_rid(struct device > > >>> >*dev, struct device_node **np, > > >>> >          if (!of_map_rid(parent_dev->of_node, rid_in, "msi-map", > > >>> >                  "msi-map-mask", np, &rid_out)) > > >>> >              break; > > >>> > + > > >>> > +    if (rid_out == rid_in) > > >>> > +        rid_out = ls_pcie_streamid_fix(parent_dev, rid_in); > > >>> > > >>> Over my dead body. Get your firmware to properly program the LUT so > > >>> that it presents the ITS with a reasonable topology. There is > > >>> absolutely no way this kind of change makes it into the kernel. > > >> > > >> Sorry for this, I know it is not reasonable, but I have no other way, > > >> as I know, ARM get the mapping of stream ID to request ID from the > > >> msi-map property of DTS, if add a new device which need the stream ID > > >> and try to get it from the msi-map of DTS, it will failed and not > > >> work, yes? So could you give me a better advice to fix this issue, I > > >> would really appreciate any comments or suggestions, thanks a lot. > > > > > > Why can't firmware expose an msi-map/msi-map-mask that has a large > > > enough range to ensure mapping of VFs? What are the limitations of the > > > LUT that would prevent this from being configured before the kernel > > > boots? > > Thanks for your comments, yes, this is the root cause, we only have > 16 stream IDs for PCIe domain, this is the hardware limitation, if > there have enough stream IDs, we can expose an msi-map/msi-map-mask > for all PCIe devices in system, unfortunately, the stream IDs is not > enough, I think other ARM vendor have same issue that they don't > have enough stream IDs. Not that I know off. I'm using a number of ARM-based, SMMU-equipped HW that works just fine. SR-IOV is perfectly functional on these platforms, and it seems that only FSL/NXP HW requires hacks of this sort. M. -- Jazz is not dead, it just smells funny.