[PATCH 0/3] Request direct mapping for modem firmware subdevice

[PATCH 0/3] Request direct mapping for modem firmware subdevice

[Sibi Sankar]

The Q6 modem sub-system has direct access to DDR through memnoc and
an indirect access routed through a SMMU which MSS CE (crypto engine
sub-component of MSS) uses during out of reset sequence. Request direct
mapping for the modem-firmware subdevice since smmu is not expected
to provide access control/translation for these SIDs (sandboxing of the
modem is achieved through XPUs engaged using SMC calls).

Sibi Sankar (3):
  iommu: Export "iommu_request_dm_for_dev"
  dt-bindings: remoteproc: qcom: Add modem-firmware bindings
  remoteproc: qcom_q6v5_mss: Request direct mapping for firmware
    subdevice

 .../bindings/remoteproc/qcom,q6v5.txt         |  4 ++
 drivers/iommu/iommu.c                         |  1 +
 drivers/remoteproc/qcom_q6v5_mss.c            | 68 +++++++++++++++++++
 3 files changed, 73 insertions(+)

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

[PATCH 1/3] iommu: Export "iommu_request_dm_for_dev"

[Sibi Sankar]

Export "iommu_request_dm_for_dev" to allow for modular builds of
drivers requesting for direct mapping.

Signed-off-by: Sibi Sankar <sibis@codeaurora.org>
---
 drivers/iommu/iommu.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 3e3528436e0b2..06a36230fe22d 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -2353,6 +2353,7 @@ int iommu_request_dm_for_dev(struct device *dev)
 {
 	return request_default_domain_for_dev(dev, IOMMU_DOMAIN_IDENTITY);
 }
+EXPORT_SYMBOL_GPL(iommu_request_dm_for_dev);
 
 /* Request that a device can't be direct mapped by the IOMMU */
 int iommu_request_dma_domain_for_dev(struct device *dev)
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

[PATCH 2/3] dt-bindings: remoteproc: qcom: Add modem-firmware bindings

[Sibi Sankar]

Introduce modem-firmware binding to allow Q6 modem to boot on platforms
which do not have trustZone.

Signed-off-by: Sibi Sankar <sibis@codeaurora.org>
---
 Documentation/devicetree/bindings/remoteproc/qcom,q6v5.txt | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/Documentation/devicetree/bindings/remoteproc/qcom,q6v5.txt b/Documentation/devicetree/bindings/remoteproc/qcom,q6v5.txt
index 88dfa3fc15f78..b1715bc90fb05 100644
--- a/Documentation/devicetree/bindings/remoteproc/qcom,q6v5.txt
+++ b/Documentation/devicetree/bindings/remoteproc/qcom,q6v5.txt
@@ -198,6 +198,10 @@ The Hexagon node may also have an subnode named either "smd-edge" or
 related to the Hexagon.  See ../soc/qcom/qcom,smd.txt and
 ../soc/qcom/qcom,glink.txt for details on how to describe these.
 
+The Hexagon node must contain a subnode named "modem-firmware" on
+platforms which do not have TrustZone. The sub-node should have the
+iommu property as described in ../iommu/iommu.txt.
+
 = EXAMPLE
 The following example describes the resources needed to boot control the
 Hexagon, as it is found on MSM8974 boards.
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

[PATCH 3/3] remoteproc: qcom_q6v5_mss: Request direct mapping for firmware subdevice

[Sibi Sankar]

The Q6 modem sub-system has direct access to DDR through memnoc and
an indirect access routed through a SMMU which MSS CE (crypto engine
sub-component of MSS) uses during out of reset sequence. Request direct
mapping for the modem-firmware subdevice since smmu is not expected
to provide access control/translation for these SIDs (sandboxing of the
modem is achieved through XPUs engaged using SMC calls).

Signed-off-by: Sibi Sankar <sibis@codeaurora.org>
---
 drivers/remoteproc/qcom_q6v5_mss.c | 68 ++++++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)

diff --git a/drivers/remoteproc/qcom_q6v5_mss.c b/drivers/remoteproc/qcom_q6v5_mss.c
index d7667418a62f4..ceb7f71dd17df 100644
--- a/drivers/remoteproc/qcom_q6v5_mss.c
+++ b/drivers/remoteproc/qcom_q6v5_mss.c
@@ -10,6 +10,7 @@
 #include <linux/clk.h>
 #include <linux/delay.h>
 #include <linux/dma-mapping.h>
+#include <linux/iommu.h>
 #include <linux/interrupt.h>
 #include <linux/kernel.h>
 #include <linux/mfd/syscon.h>
@@ -202,6 +203,7 @@ struct q6v5 {
 	struct qcom_rproc_subdev smd_subdev;
 	struct qcom_rproc_ssr ssr_subdev;
 	struct qcom_sysmon *sysmon;
+	struct device *fw_subdev;
 	bool need_mem_protection;
 	bool has_alt_reset;
 	bool has_halt_nav;
@@ -378,6 +380,67 @@ static void q6v5_pds_disable(struct q6v5 *qproc, struct device **pds,
 	}
 }
 
+static int qcom_init_firmware(struct q6v5 *qproc)
+{
+	struct platform_device_info info;
+	struct platform_device *pdev;
+	struct device_node *np;
+	int ret;
+
+	np = of_get_child_by_name(qproc->dev->of_node, "modem-firmware");
+	if (!np)
+		return 0;
+
+	memset(&info, 0, sizeof(info));
+	info.fwnode = &np->fwnode;
+	info.parent = qproc->dev;
+	info.name = np->name;
+
+	pdev = platform_device_register_full(&info);
+	if (IS_ERR(pdev)) {
+		of_node_put(np);
+		return PTR_ERR(pdev);
+	}
+
+	pdev->dev.of_node = np;
+	ret = of_dma_configure(&pdev->dev, np, true);
+	if (ret) {
+		dev_err(&pdev->dev, "Failed to configure DMA\n");
+		goto err_unregister;
+	}
+
+	ret = iommu_request_dm_for_dev(&pdev->dev);
+	if (ret) {
+		dev_err(&pdev->dev, "Failed to request direct mapping\n");
+		goto err_unregister;
+	}
+
+	qproc->fw_subdev = &pdev->dev;
+	of_node_put(np);
+	return 0;
+
+err_unregister:
+	platform_device_unregister(pdev);
+	of_node_put(np);
+	return ret;
+}
+
+static void qcom_remove_firmware(struct q6v5 *qproc)
+{
+	struct iommu_domain *iommu;
+
+	if (!qproc->fw_subdev)
+		return;
+
+	iommu = iommu_get_domain_for_dev(qproc->fw_subdev);
+	if (!iommu)
+		return;
+
+	iommu_detach_device(iommu, qproc->fw_subdev);
+	iommu_domain_free(iommu);
+	platform_device_unregister(to_platform_device(qproc->fw_subdev));
+}
+
 static int q6v5_xfer_mem_ownership(struct q6v5 *qproc, int *current_perm,
 				   bool local, bool remote, phys_addr_t addr,
 				   size_t size)
@@ -1722,6 +1785,10 @@ static int q6v5_probe(struct platform_device *pdev)
 	if (ret)
 		goto detach_proxy_pds;
 
+	ret = qcom_init_firmware(qproc);
+	if (ret)
+		goto detach_proxy_pds;
+
 	qproc->mpss_perm = BIT(QCOM_SCM_VMID_HLOS);
 	qproc->mba_perm = BIT(QCOM_SCM_VMID_HLOS);
 	qcom_add_glink_subdev(rproc, &qproc->glink_subdev);
@@ -1759,6 +1826,7 @@ static int q6v5_remove(struct platform_device *pdev)
 	qcom_remove_glink_subdev(qproc->rproc, &qproc->glink_subdev);
 	qcom_remove_smd_subdev(qproc->rproc, &qproc->smd_subdev);
 	qcom_remove_ssr_subdev(qproc->rproc, &qproc->ssr_subdev);
+	qcom_remove_firmware(qproc);
 
 	q6v5_pds_detach(qproc, qproc->active_pds, qproc->active_pd_count);
 	q6v5_pds_detach(qproc, qproc->proxy_pds, qproc->proxy_pd_count);
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [PATCH 0/3] Request direct mapping for modem firmware subdevice

[Joerg Roedel]

On Mon, Mar 09, 2020 at 11:52:52PM +0530, Sibi Sankar wrote:
> The Q6 modem sub-system has direct access to DDR through memnoc and
> an indirect access routed through a SMMU which MSS CE (crypto engine
> sub-component of MSS) uses during out of reset sequence. Request direct
> mapping for the modem-firmware subdevice since smmu is not expected
> to provide access control/translation for these SIDs (sandboxing of the
> modem is achieved through XPUs engaged using SMC calls).

So the DMA accesses are initiated by the firmware and need to be direct
mapped? Which memory region do they access?

Regards,

	Joerg

Re: [PATCH 0/3] Request direct mapping for modem firmware subdevice

[Sibi Sankar]

Hey Joerg,
Thanks for taking time to review
the series!

On 2020-03-10 16:53, Joerg Roedel wrote:
> On Mon, Mar 09, 2020 at 11:52:52PM +0530, Sibi Sankar wrote:
>> The Q6 modem sub-system has direct access to DDR through memnoc and
>> an indirect access routed through a SMMU which MSS CE (crypto engine
>> sub-component of MSS) uses during out of reset sequence. Request 
>> direct
>> mapping for the modem-firmware subdevice since smmu is not expected
>> to provide access control/translation for these SIDs (sandboxing of 
>> the
>> modem is achieved through XPUs engaged using SMC calls).
> 
> So the DMA accesses are initiated by the firmware and need to be direct
> mapped? Which memory region do they access?

The accesses are initiated by the firmware
and they access modem reserved regions.
However as explained in ^^ any accesses
outside the region will result in a violation
and is controlled through XPUs (protection units).

With ARM_SMMU_DISABLE_BYPASS_BY_DEFAULT y
the firmware SIDs will explicitly need to
be directly mapped to avoid observing
global faults in the absence of secure
firmware (TrustZone) programming the modem
SIDs as S2CR-Bypass.

> 
> Regards,
> 
> 	Joerg

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project.

Re: [PATCH 0/3] Request direct mapping for modem firmware subdevice

[Joerg Roedel]

On Tue, Mar 10, 2020 at 07:30:50PM +0530, Sibi Sankar wrote:
> The accesses are initiated by the firmware
> and they access modem reserved regions.
> However as explained in ^^ any accesses
> outside the region will result in a violation
> and is controlled through XPUs (protection units).

Okay, this sounds like a case for arm_smmu_get_resv_region(). It should
return an entry for the reserved memory region the firmware needs to
access, so that generic iommu can setup this mapping.

Note that it should return that entry only for your device, not for all
devices. Maybe there is a property in DT or IORT you can set to
transport this information into the arm-smmu driver.

This is pretty similar to RMRR mapping on the Intel VT-d IOMMU or
Unity-mapped ranges in the AMD-Vi IOMMU.

Regards,

	Joerg

Re: [PATCH 0/3] Request direct mapping for modem firmware subdevice

[Robin Murphy]

On 10/03/2020 4:23 pm, Joerg Roedel wrote:
> On Tue, Mar 10, 2020 at 07:30:50PM +0530, Sibi Sankar wrote:
>> The accesses are initiated by the firmware
>> and they access modem reserved regions.
>> However as explained in ^^ any accesses
>> outside the region will result in a violation
>> and is controlled through XPUs (protection units).
> 
> Okay, this sounds like a case for arm_smmu_get_resv_region(). It should
> return an entry for the reserved memory region the firmware needs to
> access, so that generic iommu can setup this mapping.
> 
> Note that it should return that entry only for your device, not for all
> devices. Maybe there is a property in DT or IORT you can set to
> transport this information into the arm-smmu driver.
> 
> This is pretty similar to RMRR mapping on the Intel VT-d IOMMU or
> Unity-mapped ranges in the AMD-Vi IOMMU.

Yup, a way to describe boot-time memory regions in IORT is in the 
process of being specced out; the first attempt at an equivalent for DT 
is here:

https://lore.kernel.org/linux-iommu/20191209150748.2471814-1-thierry.reding@gmail.com/

If that's not enough and the SMMU still needs to treat certain Stream 
IDs specially because they may be untranslatable (due to having direct 
access to memory as a side-channel), then that should be handled in the 
SoC-specific corner of the SMMU driver, not delegated to individual 
endpoint drivers.

Robin.

Re: [PATCH 0/3] Request direct mapping for modem firmware subdevice

[Sai Prakash Ranjan]

Hi Robin,

On 2020-03-10 22:14, Robin Murphy wrote:
> On 10/03/2020 4:23 pm, Joerg Roedel wrote:
>> On Tue, Mar 10, 2020 at 07:30:50PM +0530, Sibi Sankar wrote:
>>> The accesses are initiated by the firmware
>>> and they access modem reserved regions.
>>> However as explained in ^^ any accesses
>>> outside the region will result in a violation
>>> and is controlled through XPUs (protection units).
>> 
>> Okay, this sounds like a case for arm_smmu_get_resv_region(). It 
>> should
>> return an entry for the reserved memory region the firmware needs to
>> access, so that generic iommu can setup this mapping.
>> 
>> Note that it should return that entry only for your device, not for 
>> all
>> devices. Maybe there is a property in DT or IORT you can set to
>> transport this information into the arm-smmu driver.
>> 
>> This is pretty similar to RMRR mapping on the Intel VT-d IOMMU or
>> Unity-mapped ranges in the AMD-Vi IOMMU.
> 
> Yup, a way to describe boot-time memory regions in IORT is in the
> process of being specced out; the first attempt at an equivalent for
> DT is here:
> 
> https://lore.kernel.org/linux-iommu/20191209150748.2471814-1-thierry.reding@gmail.com/
> 
> If that's not enough and the SMMU still needs to treat certain Stream
> IDs specially because they may be untranslatable (due to having direct
> access to memory as a side-channel), then that should be handled in
> the SoC-specific corner of the SMMU driver, not delegated to
> individual endpoint drivers.
> 

Are you talking about this one for SoC specific change - 
https://lore.kernel.org/patchwork/patch/1183530/

Thanks,
Sai

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a 
member
of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH 0/3] Request direct mapping for modem firmware subdevice

[Robin Murphy]

On 2020-03-12 6:28 am, Sai Prakash Ranjan wrote:
> Hi Robin,
> 
> On 2020-03-10 22:14, Robin Murphy wrote:
>> On 10/03/2020 4:23 pm, Joerg Roedel wrote:
>>> On Tue, Mar 10, 2020 at 07:30:50PM +0530, Sibi Sankar wrote:
>>>> The accesses are initiated by the firmware
>>>> and they access modem reserved regions.
>>>> However as explained in ^^ any accesses
>>>> outside the region will result in a violation
>>>> and is controlled through XPUs (protection units).
>>>
>>> Okay, this sounds like a case for arm_smmu_get_resv_region(). It should
>>> return an entry for the reserved memory region the firmware needs to
>>> access, so that generic iommu can setup this mapping.
>>>
>>> Note that it should return that entry only for your device, not for all
>>> devices. Maybe there is a property in DT or IORT you can set to
>>> transport this information into the arm-smmu driver.
>>>
>>> This is pretty similar to RMRR mapping on the Intel VT-d IOMMU or
>>> Unity-mapped ranges in the AMD-Vi IOMMU.
>>
>> Yup, a way to describe boot-time memory regions in IORT is in the
>> process of being specced out; the first attempt at an equivalent for
>> DT is here:
>>
>> https://lore.kernel.org/linux-iommu/20191209150748.2471814-1-thierry.reding@gmail.com/ 
>>
>>
>> If that's not enough and the SMMU still needs to treat certain Stream
>> IDs specially because they may be untranslatable (due to having direct
>> access to memory as a side-channel), then that should be handled in
>> the SoC-specific corner of the SMMU driver, not delegated to
>> individual endpoint drivers.
>>
> 
> Are you talking about this one for SoC specific change - 
> https://lore.kernel.org/patchwork/patch/1183530/

Exactly - this particular wheel needs no reinventing at all.

[ I guess I should go review those patches properly... :) ]

Robin.

Re: [PATCH 0/3] Request direct mapping for modem firmware subdevice

[Christoph Hellwig]

On Mon, Mar 09, 2020 at 11:52:52PM +0530, Sibi Sankar wrote:
> The Q6 modem sub-system has direct access to DDR through memnoc and
> an indirect access routed through a SMMU which MSS CE (crypto engine
> sub-component of MSS) uses during out of reset sequence. Request direct
> mapping for the modem-firmware subdevice since smmu is not expected
> to provide access control/translation for these SIDs (sandboxing of the
> modem is achieved through XPUs engaged using SMC calls).

Please fix your device tree so that the device isn't bound to an
IOMMU.

Re: [PATCH 0/3] Request direct mapping for modem firmware subdevice

[Sibi Sankar]

Hey Christoph,
Thanks for taking time to review
the series.

On 2020-03-16 21:20, Christoph Hellwig wrote:
> On Mon, Mar 09, 2020 at 11:52:52PM +0530, Sibi Sankar wrote:
>> The Q6 modem sub-system has direct access to DDR through memnoc and
>> an indirect access routed through a SMMU which MSS CE (crypto engine
>> sub-component of MSS) uses during out of reset sequence. Request 
>> direct
>> mapping for the modem-firmware subdevice since smmu is not expected
>> to provide access control/translation for these SIDs (sandboxing of 
>> the
>> modem is achieved through XPUs engaged using SMC calls).
> 
> Please fix your device tree so that the device isn't bound to an
> IOMMU.

the bindings proposed in the series
would add a sub-device with an iommu
property.

modem_pil: remoteproc@xxxxx {
...
    modem-firmware {
         iommus = <&apps_smmu 0x460 0x1>;
    };
...
};

Remoteproc device will not have a iommu
property but modem-firmware sub-device
will.

With ARM_SMMU_DISABLE_BYPASS_BY_DEFAULT y,
we would want to configure the SID either
in direct mapping or bypass (either will
do since protection is achieved through
other means)

https://lore.kernel.org/lkml/497e40b8-300f-1b83-4312-93a58c459d1d@arm.com/

Currently the restructuring is trending
towards whats discussed in the ^^ thread.
i.e either direct mapping/bypass will be
done in the SoC specific corner of the
SMMU driver.

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project.

Re: [PATCH 0/3] Request direct mapping for modem firmware subdevice

[Sai Prakash Ranjan]

Hi Robin,

On 2020-03-12 17:35, Robin Murphy wrote:
> On 2020-03-12 6:28 am, Sai Prakash Ranjan wrote:
>> Hi Robin,
>> 
>> 
>> Are you talking about this one for SoC specific change - 
>> https://lore.kernel.org/patchwork/patch/1183530/
> 
> Exactly - this particular wheel needs no reinventing at all.
> 
> [ I guess I should go review those patches properly... :) ]
> 

It would be great if you could review the patch - 
https://lore.kernel.org/patchwork/patch/1183530/
Sibi has posted a v2 of this series based on that patch.

Thanks,
Sai

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a 
member
of Code Aurora Forum, hosted by The Linux Foundation