From: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> To: Pankaj Bansal <pankaj.bansal@nxp.com>, Will Deacon <will.deacon@arm.com>, Robin Murphy <robin.murphy@arm.com>, Joerg Roedel <joro@8bytes.org> Cc: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>, Varun Sethi <V.Sethi@nxp.com>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org> Subject: Re: [ARM SMMU] Dynamic StreamID allocation Date: Mon, 13 May 2019 13:42:27 +0100 [thread overview] Message-ID: <662f966d-aad9-10a4-7326-374c0bb26020@arm.com> (raw) In-Reply-To: <VI1PR0401MB249699B9BA183B18997532D0F10F0@VI1PR0401MB2496.eurprd04.prod.outlook.com> On 13/05/2019 08:09, Pankaj Bansal wrote: > Hi Jean, > >> -----Original Message----- >> From: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> >> Sent: Friday, 10 May, 2019 07:07 PM >> To: Pankaj Bansal <pankaj.bansal@nxp.com>; Will Deacon >> <will.deacon@arm.com>; Robin Murphy <robin.murphy@arm.com>; Joerg >> Roedel <joro@8bytes.org> >> Cc: iommu@lists.linux-foundation.org; Varun Sethi <V.Sethi@nxp.com>; linux- >> arm-kernel@lists.infradead.org; Nipun Gupta <nipun.gupta@nxp.com> >> Subject: Re: [ARM SMMU] Dynamic StreamID allocation >> >> On 10/05/2019 13:33, Pankaj Bansal wrote: >>> Hi Will/Robin/Joerg, >>> >>> I am s/w engineer from NXP India Pvt. Ltd. >>> We are using SMMU-V3 in one of NXP SOC. >>> I have a question about the SMMU Stream ID allocation in linux. >>> >>> Right now the Stream IDs allocated to a device are mapped via device tree to >> the device. >>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Felix >>> ir.bootlin.com%2Flinux%2Flatest%2Fsource%2FDocumentation%2Fdevicetree% >>> 2Fbindings%2Fiommu%2Farm%2Csmmu- >> v3.txt%23L39&data=02%7C01%7Cpankaj >>> .bansal%40nxp.com%7C3cbe8bd4827e425afd0f08d6d54c925e%7C686ea1d3b >> c2b4c6 >>> >> fa92cd99c5c301635%7C0%7C0%7C636930922220665343&sdata=vIG5u5n >> XR5iRp >>> uuuGjeFxKBtA5f5ohf91znXX0QWm1c%3D&reserved=0 >>> >>> As the device tree is passed from bootloader to linux, we detect all the stream >> IDs needed by a device in bootloader and add their IDs in respective device >> nodes. >>> For each PCIE Endpoint (a unique BDF (Bus Device Function)) on PCIE bus, we >> are assigning a unique Stream ID in bootloader. >>> >>> However, this poses an issue with PCIE hot plug. >>> If we plug in a pcie device while linux is running, a unique BDF is assigned to >> the device, for which there is no stream ID in device tree. >>> >>> How can this problem be solved in linux? >> >> Assuming the streamID associated to a BDF is predictable (streamID = BDF >> + constant), using the iommu-map property should just work: >> >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.boo >> tlin.com%2Flinux%2Flatest%2Fsource%2FDocumentation%2Fdevicetree%2Fbind >> ings%2Fpci%2Fpci- >> iommu.txt&data=02%7C01%7Cpankaj.bansal%40nxp.com%7C3cbe8bd482 >> 7e425afd0f08d6d54c925e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0 >> %7C636930922220665343&sdata=GkkovEnvhd5dN%2BGdh%2FnKCyW5Cd >> EnLDP3cWTrk%2B%2FO7EQ%3D&reserved=0 >> >> It describes the streamIDs of all possible BDFs, including hotplugged functions. > > You mean that we should increase the "length" parameter (in (rid-base,iommu,iommu-base,length) touple) ? > This would cater to any *new* Bus Device Function being detected on PCIE bus? > Is that right ? No, the iommu-map solution only works when you can predict at boot time which streamID will correspond to any BDF, for example if the PCIe controller or firmware automatically assigns streamID = BDF. > Right now when we make iommu-map in bootloader, we are giving one RID per BDF: > https://elixir.bootlin.com/u-boot/latest/source/drivers/pci/pcie_layerscape_fixup.c#L168 Right, that won't work with hotplug. You can't derive a unique streamID from any BDF at boot, if the streamID range is limited to 16 values on this hardware. Maybe you could move this allocator to the Linux Layerscape driver, and call iommu_fwspec_add_ids() from there, rather than using iommu-map? Not sure how feasible that is, but it might still be the simplest. Thanks, Jean > > But isn't the better approach to make it dynamic in linux? > i.e. as soon as a new device is detected "requester id" is allocated to it from available pool. > When device is removed, return the "requester id" to pool. > is there any h/w limitation which prevents it? > > Regards, > Pankaj Bansal >> >> Thanks, >> Jean >> >>> >>> Is there a way to assign (and revoke) stream IDs at run time? >>> >>> Regards, >>> Pankaj Bansal >>> >>> >>> _______________________________________________ >>> linux-arm-kernel mailing list >>> linux-arm-kernel@lists.infradead.org >>> https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists >>> .infradead.org%2Fmailman%2Flistinfo%2Flinux-arm- >> kernel&data=02%7C0 >>> >> 1%7Cpankaj.bansal%40nxp.com%7C3cbe8bd4827e425afd0f08d6d54c925e%7C6 >> 86ea >>> >> 1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C636930922220665343&sda >> ta=2La >>> GBHO2%2Bbqk519uJvCatlHyRCtAPPjKO8Gxu1bQHBM%3D&reserved=0 >>> > > _______________________________________________ > iommu mailing list > iommu@lists.linux-foundation.org > https://lists.linuxfoundation.org/mailman/listinfo/iommu > _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
WARNING: multiple messages have this Message-ID (diff)
From: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> To: Pankaj Bansal <pankaj.bansal@nxp.com>, Will Deacon <will.deacon@arm.com>, Robin Murphy <robin.murphy@arm.com>, Joerg Roedel <joro@8bytes.org> Cc: "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>, Varun Sethi <V.Sethi@nxp.com>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org> Subject: Re: [ARM SMMU] Dynamic StreamID allocation Date: Mon, 13 May 2019 13:42:27 +0100 [thread overview] Message-ID: <662f966d-aad9-10a4-7326-374c0bb26020@arm.com> (raw) In-Reply-To: <VI1PR0401MB249699B9BA183B18997532D0F10F0@VI1PR0401MB2496.eurprd04.prod.outlook.com> On 13/05/2019 08:09, Pankaj Bansal wrote: > Hi Jean, > >> -----Original Message----- >> From: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> >> Sent: Friday, 10 May, 2019 07:07 PM >> To: Pankaj Bansal <pankaj.bansal@nxp.com>; Will Deacon >> <will.deacon@arm.com>; Robin Murphy <robin.murphy@arm.com>; Joerg >> Roedel <joro@8bytes.org> >> Cc: iommu@lists.linux-foundation.org; Varun Sethi <V.Sethi@nxp.com>; linux- >> arm-kernel@lists.infradead.org; Nipun Gupta <nipun.gupta@nxp.com> >> Subject: Re: [ARM SMMU] Dynamic StreamID allocation >> >> On 10/05/2019 13:33, Pankaj Bansal wrote: >>> Hi Will/Robin/Joerg, >>> >>> I am s/w engineer from NXP India Pvt. Ltd. >>> We are using SMMU-V3 in one of NXP SOC. >>> I have a question about the SMMU Stream ID allocation in linux. >>> >>> Right now the Stream IDs allocated to a device are mapped via device tree to >> the device. >>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Felix >>> ir.bootlin.com%2Flinux%2Flatest%2Fsource%2FDocumentation%2Fdevicetree% >>> 2Fbindings%2Fiommu%2Farm%2Csmmu- >> v3.txt%23L39&data=02%7C01%7Cpankaj >>> .bansal%40nxp.com%7C3cbe8bd4827e425afd0f08d6d54c925e%7C686ea1d3b >> c2b4c6 >>> >> fa92cd99c5c301635%7C0%7C0%7C636930922220665343&sdata=vIG5u5n >> XR5iRp >>> uuuGjeFxKBtA5f5ohf91znXX0QWm1c%3D&reserved=0 >>> >>> As the device tree is passed from bootloader to linux, we detect all the stream >> IDs needed by a device in bootloader and add their IDs in respective device >> nodes. >>> For each PCIE Endpoint (a unique BDF (Bus Device Function)) on PCIE bus, we >> are assigning a unique Stream ID in bootloader. >>> >>> However, this poses an issue with PCIE hot plug. >>> If we plug in a pcie device while linux is running, a unique BDF is assigned to >> the device, for which there is no stream ID in device tree. >>> >>> How can this problem be solved in linux? >> >> Assuming the streamID associated to a BDF is predictable (streamID = BDF >> + constant), using the iommu-map property should just work: >> >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.boo >> tlin.com%2Flinux%2Flatest%2Fsource%2FDocumentation%2Fdevicetree%2Fbind >> ings%2Fpci%2Fpci- >> iommu.txt&data=02%7C01%7Cpankaj.bansal%40nxp.com%7C3cbe8bd482 >> 7e425afd0f08d6d54c925e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0 >> %7C636930922220665343&sdata=GkkovEnvhd5dN%2BGdh%2FnKCyW5Cd >> EnLDP3cWTrk%2B%2FO7EQ%3D&reserved=0 >> >> It describes the streamIDs of all possible BDFs, including hotplugged functions. > > You mean that we should increase the "length" parameter (in (rid-base,iommu,iommu-base,length) touple) ? > This would cater to any *new* Bus Device Function being detected on PCIE bus? > Is that right ? No, the iommu-map solution only works when you can predict at boot time which streamID will correspond to any BDF, for example if the PCIe controller or firmware automatically assigns streamID = BDF. > Right now when we make iommu-map in bootloader, we are giving one RID per BDF: > https://elixir.bootlin.com/u-boot/latest/source/drivers/pci/pcie_layerscape_fixup.c#L168 Right, that won't work with hotplug. You can't derive a unique streamID from any BDF at boot, if the streamID range is limited to 16 values on this hardware. Maybe you could move this allocator to the Linux Layerscape driver, and call iommu_fwspec_add_ids() from there, rather than using iommu-map? Not sure how feasible that is, but it might still be the simplest. Thanks, Jean > > But isn't the better approach to make it dynamic in linux? > i.e. as soon as a new device is detected "requester id" is allocated to it from available pool. > When device is removed, return the "requester id" to pool. > is there any h/w limitation which prevents it? > > Regards, > Pankaj Bansal >> >> Thanks, >> Jean >> >>> >>> Is there a way to assign (and revoke) stream IDs at run time? >>> >>> Regards, >>> Pankaj Bansal >>> >>> >>> _______________________________________________ >>> linux-arm-kernel mailing list >>> linux-arm-kernel@lists.infradead.org >>> https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists >>> .infradead.org%2Fmailman%2Flistinfo%2Flinux-arm- >> kernel&data=02%7C0 >>> >> 1%7Cpankaj.bansal%40nxp.com%7C3cbe8bd4827e425afd0f08d6d54c925e%7C6 >> 86ea >>> >> 1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C636930922220665343&sda >> ta=2La >>> GBHO2%2Bbqk519uJvCatlHyRCtAPPjKO8Gxu1bQHBM%3D&reserved=0 >>> > > _______________________________________________ > iommu mailing list > iommu@lists.linux-foundation.org > https://lists.linuxfoundation.org/mailman/listinfo/iommu > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2019-05-13 12:42 UTC|newest] Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-05-10 12:33 [ARM SMMU] Dynamic StreamID allocation Pankaj Bansal 2019-05-10 12:33 ` Pankaj Bansal 2019-05-10 13:19 ` Pankaj Bansal 2019-05-10 13:19 ` Pankaj Bansal 2019-05-10 13:36 ` Jean-Philippe Brucker 2019-05-10 13:36 ` Jean-Philippe Brucker 2019-05-13 7:09 ` Pankaj Bansal 2019-05-13 7:09 ` Pankaj Bansal 2019-05-13 12:42 ` Jean-Philippe Brucker [this message] 2019-05-13 12:42 ` Jean-Philippe Brucker 2019-05-13 13:36 ` Robin Murphy 2019-05-13 13:36 ` Robin Murphy 2019-05-13 13:34 ` Laurentiu Tudor 2019-05-13 13:34 ` Laurentiu Tudor
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=662f966d-aad9-10a4-7326-374c0bb26020@arm.com \ --to=jean-philippe.brucker@arm.com \ --cc=V.Sethi@nxp.com \ --cc=iommu@lists.linux-foundation.org \ --cc=joro@8bytes.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=pankaj.bansal@nxp.com \ --cc=robin.murphy@arm.com \ --cc=will.deacon@arm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.