* xen/arm: Hiding SMMUs from Dom0 when using ACPI on Xen
@ 2017-02-22 14:10 Julien Grall
2017-02-27 13:23 ` Vijay Kilari
0 siblings, 1 reply; 10+ messages in thread
From: Julien Grall @ 2017-02-22 14:10 UTC (permalink / raw)
To: Stefano Stabellini, xen-devel
Cc: Steve Capper, Andre Przywara, Jiandi An, Punit Agrawal,
Vijaya Kumar K, sgoel, nd, manish.jaggi, Shanker Donthineni
Hello,
There was few discussions recently about hiding SMMUs from DOM0 when
using ACPI. I thought it would be good to have a separate thread for this.
When using ACPI, the SMMUs will be described in the IO Remapping Table
(IORT). The specification can be found on the ARM website [1].
For a brief summary, the IORT can be used to discover the SMMUs present
on the platform and find for a given device the ID to configure
components such as ITS (DeviceID) and SMMU (StreamID).
The appendix A in the specification gives an example how DeviceID and
StreamID can be found. For instance, when a PCI device is both protected
by an SMMU and MSI-capable the following translation will happen:
RID -> StreamID -> DeviceID
Currently, SMMUs are hidden from DOM0 because they are been used by Xen
and we don't support stage-1 SMMU. If we pass the IORT as it is, DOM0
will try to initialize SMMU and crash.
I first thought about using a Xen specific way (STAO) or extending a
flag in IORT. But that is not ideal.
So we would have to rewrite the IORT for DOM0. Given that a range of RID
can mapped to multiple ranges of DeviceID, we would have to translate
RID one by one to find the associated DeviceID. I think this may end up
to complex code and have a big IORT table.
However, given that DeviceID will be used by DOM0 to only configure the
ITS. We have no need to use to have the DOM0 DeviceID equal to the host
DeviceID. So I think we could simplify our life by generating DeviceID
for each RID range.
Any opinions?
Cheers,
[1]
http://infocenter.arm.com/help/topic/com.arm.doc.den0049b/DEN0049B_IO_Remapping_Table.pdf
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: xen/arm: Hiding SMMUs from Dom0 when using ACPI on Xen
2017-02-22 14:10 xen/arm: Hiding SMMUs from Dom0 when using ACPI on Xen Julien Grall
@ 2017-02-27 13:23 ` Vijay Kilari
2017-02-27 14:12 ` Julien Grall
0 siblings, 1 reply; 10+ messages in thread
From: Vijay Kilari @ 2017-02-27 13:23 UTC (permalink / raw)
To: Julien Grall
Cc: Stefano Stabellini, Steve Capper, Andre Przywara, Jiandi An,
Punit Agrawal, Vijaya Kumar K, sgoel, xen-devel, nd,
manish.jaggi, Shanker Donthineni
Hi Julien,
On Wed, Feb 22, 2017 at 7:40 PM, Julien Grall <julien.grall@arm.com> wrote:
> Hello,
>
> There was few discussions recently about hiding SMMUs from DOM0 when using
> ACPI. I thought it would be good to have a separate thread for this.
>
> When using ACPI, the SMMUs will be described in the IO Remapping Table
> (IORT). The specification can be found on the ARM website [1].
>
> For a brief summary, the IORT can be used to discover the SMMUs present on
> the platform and find for a given device the ID to configure components such
> as ITS (DeviceID) and SMMU (StreamID).
>
> The appendix A in the specification gives an example how DeviceID and
> StreamID can be found. For instance, when a PCI device is both protected by
> an SMMU and MSI-capable the following translation will happen:
> RID -> StreamID -> DeviceID
>
> Currently, SMMUs are hidden from DOM0 because they are been used by Xen and
> we don't support stage-1 SMMU. If we pass the IORT as it is, DOM0 will try
> to initialize SMMU and crash.
>
> I first thought about using a Xen specific way (STAO) or extending a flag in
> IORT. But that is not ideal.
>
> So we would have to rewrite the IORT for DOM0. Given that a range of RID can
> mapped to multiple ranges of DeviceID, we would have to translate RID one by
> one to find the associated DeviceID. I think this may end up to complex code
> and have a big IORT table.
Why can't we replace Output base of IORT of PCI node with SMMU output base?.
I mean similar to PCI node without SMMU, why can't replace output base
of PCI node with
SMMU's output base?.
The issue I see is RID is [15:0] where is DeviceID is [17:0].
>
> However, given that DeviceID will be used by DOM0 to only configure the ITS.
> We have no need to use to have the DOM0 DeviceID equal to the host DeviceID.
> So I think we could simplify our life by generating DeviceID for each RID
> range.
If DOM0 DeviceID != host Device ID, then we cannot initialize ITS using DOM0
ITS commands (MAPD). So, is it concluded that ITS initializes all the devices
with platform specific Device ID's in Xen?.
>
> Any opinions?
>
> Cheers,
>
> [1]
> http://infocenter.arm.com/help/topic/com.arm.doc.den0049b/DEN0049B_IO_Remapping_Table.pdf
>
> --
> Julien Grall
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: xen/arm: Hiding SMMUs from Dom0 when using ACPI on Xen
2017-02-27 13:23 ` Vijay Kilari
@ 2017-02-27 14:12 ` Julien Grall
2017-02-27 16:58 ` Shanker Donthineni
0 siblings, 1 reply; 10+ messages in thread
From: Julien Grall @ 2017-02-27 14:12 UTC (permalink / raw)
To: Vijay Kilari
Cc: Stefano Stabellini, Steve Capper, Andre Przywara, Jiandi An,
Punit Agrawal, Vijaya Kumar K, sgoel, xen-devel, nd,
manish.jaggi, Shanker Donthineni
On 27/02/17 13:23, Vijay Kilari wrote:
> Hi Julien,
Hello Vijay,
> On Wed, Feb 22, 2017 at 7:40 PM, Julien Grall <julien.grall@arm.com> wrote:
>> Hello,
>>
>> There was few discussions recently about hiding SMMUs from DOM0 when using
>> ACPI. I thought it would be good to have a separate thread for this.
>>
>> When using ACPI, the SMMUs will be described in the IO Remapping Table
>> (IORT). The specification can be found on the ARM website [1].
>>
>> For a brief summary, the IORT can be used to discover the SMMUs present on
>> the platform and find for a given device the ID to configure components such
>> as ITS (DeviceID) and SMMU (StreamID).
>>
>> The appendix A in the specification gives an example how DeviceID and
>> StreamID can be found. For instance, when a PCI device is both protected by
>> an SMMU and MSI-capable the following translation will happen:
>> RID -> StreamID -> DeviceID
>>
>> Currently, SMMUs are hidden from DOM0 because they are been used by Xen and
>> we don't support stage-1 SMMU. If we pass the IORT as it is, DOM0 will try
>> to initialize SMMU and crash.
>>
>> I first thought about using a Xen specific way (STAO) or extending a flag in
>> IORT. But that is not ideal.
>>
>> So we would have to rewrite the IORT for DOM0. Given that a range of RID can
>> mapped to multiple ranges of DeviceID, we would have to translate RID one by
>> one to find the associated DeviceID. I think this may end up to complex code
>> and have a big IORT table.
>
> Why can't we replace Output base of IORT of PCI node with SMMU output base?.
> I mean similar to PCI node without SMMU, why can't replace output base
> of PCI node with
> SMMU's output base?.
Because I don't see anything in the spec preventing one RC ID mapping to
produce multiple SMMU ID mapping. So which output base would you use?
>
> The issue I see is RID is [15:0] where is DeviceID is [17:0].
>
>>
>> However, given that DeviceID will be used by DOM0 to only configure the ITS.
>> We have no need to use to have the DOM0 DeviceID equal to the host DeviceID.
>> So I think we could simplify our life by generating DeviceID for each RID
>> range.
>
> If DOM0 DeviceID != host Device ID, then we cannot initialize ITS using DOM0
> ITS commands (MAPD). So, is it concluded that ITS initializes all the devices
> with platform specific Device ID's in Xen?.
Initializing ITS using DOM0 ITS command is a workaround until we get PCI
passthrough done. It would still be possible to implement that with
vDeviceID != pDeviceID as Xen would likely have the mapping between the
2 DeviceID.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: xen/arm: Hiding SMMUs from Dom0 when using ACPI on Xen
2017-02-27 14:12 ` Julien Grall
@ 2017-02-27 16:58 ` Shanker Donthineni
2017-02-27 18:12 ` Julien Grall
0 siblings, 1 reply; 10+ messages in thread
From: Shanker Donthineni @ 2017-02-27 16:58 UTC (permalink / raw)
To: xen-devel
Hi Julien,
On 02/27/2017 08:12 AM, Julien Grall wrote:
>
>
> On 27/02/17 13:23, Vijay Kilari wrote:
>> Hi Julien,
>
> Hello Vijay,
>
>> On Wed, Feb 22, 2017 at 7:40 PM, Julien Grall <julien.grall@arm.com>
>> wrote:
>>> Hello,
>>>
>>> There was few discussions recently about hiding SMMUs from DOM0 when
>>> using
>>> ACPI. I thought it would be good to have a separate thread for this.
>>>
>>> When using ACPI, the SMMUs will be described in the IO Remapping Table
>>> (IORT). The specification can be found on the ARM website [1].
>>>
>>> For a brief summary, the IORT can be used to discover the SMMUs
>>> present on
>>> the platform and find for a given device the ID to configure
>>> components such
>>> as ITS (DeviceID) and SMMU (StreamID).
>>>
>>> The appendix A in the specification gives an example how DeviceID and
>>> StreamID can be found. For instance, when a PCI device is both
>>> protected by
>>> an SMMU and MSI-capable the following translation will happen:
>>> RID -> StreamID -> DeviceID
>>>
>>> Currently, SMMUs are hidden from DOM0 because they are been used by
>>> Xen and
>>> we don't support stage-1 SMMU. If we pass the IORT as it is, DOM0
>>> will try
>>> to initialize SMMU and crash.
>>>
>>> I first thought about using a Xen specific way (STAO) or extending a
>>> flag in
>>> IORT. But that is not ideal.
>>>
>>> So we would have to rewrite the IORT for DOM0. Given that a range of
>>> RID can
>>> mapped to multiple ranges of DeviceID, we would have to translate
>>> RID one by
>>> one to find the associated DeviceID. I think this may end up to
>>> complex code
>>> and have a big IORT table.
>>
>> Why can't we replace Output base of IORT of PCI node with SMMU output
>> base?.
>> I mean similar to PCI node without SMMU, why can't replace output base
>> of PCI node with
>> SMMU's output base?.
>
> Because I don't see anything in the spec preventing one RC ID mapping
> to produce multiple SMMU ID mapping. So which output base would you use?
>
Basically, remove SMMU nodes, and replaces output of the PCIe and named
nodes ID mappings with ITS nodes.
RID --> StreamID --> dviceID --> ITS device id = RID --> dviceID -->
ITS device id
>>
>> The issue I see is RID is [15:0] where is DeviceID is [17:0].
>>
Actuality device id is 32bit field.
>>>
>>> However, given that DeviceID will be used by DOM0 to only configure
>>> the ITS.
>>> We have no need to use to have the DOM0 DeviceID equal to the host
>>> DeviceID.
>>> So I think we could simplify our life by generating DeviceID for
>>> each RID
>>> range.
>>
>> If DOM0 DeviceID != host Device ID, then we cannot initialize ITS
>> using DOM0
>> ITS commands (MAPD). So, is it concluded that ITS initializes all the
>> devices
>> with platform specific Device ID's in Xen?.
>
> Initializing ITS using DOM0 ITS command is a workaround until we get
> PCI passthrough done. It would still be possible to implement that
> with vDeviceID != pDeviceID as Xen would likely have the mapping
> between the 2 DeviceID.
>
I believe mapping dom0 ITS commands to XEN ITS commands one to one is
the better approach. Physical DeviceID is unique per ITS group, not a
system wide unique ID. In case of direct VLPI, LPI number has to be
programmed whenever dom0/domU calls the MAPTI command but not at the
time of PCIe device creation.
--
Shanker Donthineni
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: xen/arm: Hiding SMMUs from Dom0 when using ACPI on Xen
2017-02-27 16:58 ` Shanker Donthineni
@ 2017-02-27 18:12 ` Julien Grall
2017-05-18 11:59 ` Manish Jaggi
0 siblings, 1 reply; 10+ messages in thread
From: Julien Grall @ 2017-02-27 18:12 UTC (permalink / raw)
To: shankerd, xen-devel
Cc: Stefano Stabellini, Steve Capper, Andre Przywara, Jiandi An,
Punit Agrawal, Sameer Goel, nd
On 02/27/2017 04:58 PM, Shanker Donthineni wrote:
> Hi Julien,
Hi Shanker,
Please don't drop people in CC. In my case, any e-mail I am not CCed are
skipping my inbox and I may not read them for a while.
>
> On 02/27/2017 08:12 AM, Julien Grall wrote:
>>
>>
>> On 27/02/17 13:23, Vijay Kilari wrote:
>>> Hi Julien,
>>
>> Hello Vijay,
>>
>>> On Wed, Feb 22, 2017 at 7:40 PM, Julien Grall <julien.grall@arm.com>
>>> wrote:
>>>> Hello,
>>>>
>>>> There was few discussions recently about hiding SMMUs from DOM0 when
>>>> using
>>>> ACPI. I thought it would be good to have a separate thread for this.
>>>>
>>>> When using ACPI, the SMMUs will be described in the IO Remapping Table
>>>> (IORT). The specification can be found on the ARM website [1].
>>>>
>>>> For a brief summary, the IORT can be used to discover the SMMUs
>>>> present on
>>>> the platform and find for a given device the ID to configure
>>>> components such
>>>> as ITS (DeviceID) and SMMU (StreamID).
>>>>
>>>> The appendix A in the specification gives an example how DeviceID and
>>>> StreamID can be found. For instance, when a PCI device is both
>>>> protected by
>>>> an SMMU and MSI-capable the following translation will happen:
>>>> RID -> StreamID -> DeviceID
>>>>
>>>> Currently, SMMUs are hidden from DOM0 because they are been used by
>>>> Xen and
>>>> we don't support stage-1 SMMU. If we pass the IORT as it is, DOM0
>>>> will try
>>>> to initialize SMMU and crash.
>>>>
>>>> I first thought about using a Xen specific way (STAO) or extending a
>>>> flag in
>>>> IORT. But that is not ideal.
>>>>
>>>> So we would have to rewrite the IORT for DOM0. Given that a range of
>>>> RID can
>>>> mapped to multiple ranges of DeviceID, we would have to translate
>>>> RID one by
>>>> one to find the associated DeviceID. I think this may end up to
>>>> complex code
>>>> and have a big IORT table.
>>>
>>> Why can't we replace Output base of IORT of PCI node with SMMU output
>>> base?.
>>> I mean similar to PCI node without SMMU, why can't replace output base
>>> of PCI node with
>>> SMMU's output base?.
>>
>> Because I don't see anything in the spec preventing one RC ID mapping
>> to produce multiple SMMU ID mapping. So which output base would you use?
>>
>
> Basically, remove SMMU nodes, and replaces output of the PCIe and named
> nodes ID mappings with ITS nodes.
>
> RID --> StreamID --> dviceID --> ITS device id = RID --> dviceID -->
> ITS device id
Can you detail it? You seem to assume that one RC ID mapping range will
only produce ID mapping range. AFAICT, this is not mandated by the spec.
>
>>>
>>> The issue I see is RID is [15:0] where is DeviceID is [17:0].
>>>
> Actuality device id is 32bit field.
>
>>>>
>>>> However, given that DeviceID will be used by DOM0 to only configure
>>>> the ITS.
>>>> We have no need to use to have the DOM0 DeviceID equal to the host
>>>> DeviceID.
>>>> So I think we could simplify our life by generating DeviceID for
>>>> each RID
>>>> range.
>>>
>>> If DOM0 DeviceID != host Device ID, then we cannot initialize ITS
>>> using DOM0
>>> ITS commands (MAPD). So, is it concluded that ITS initializes all the
>>> devices
>>> with platform specific Device ID's in Xen?.
>>
>> Initializing ITS using DOM0 ITS command is a workaround until we get
>> PCI passthrough done. It would still be possible to implement that
>> with vDeviceID != pDeviceID as Xen would likely have the mapping
>> between the 2 DeviceID.
>>
>
> I believe mapping dom0 ITS commands to XEN ITS commands one to one is
> the better approach. Physical DeviceID is unique per ITS group, not a
> system wide unique ID.
As for guest, you don't care about the virtual DeviceID for DOM0 as long
as you are able to map it to the host ITS and host DeviceID.
> In case of direct VLPI, LPI number has to be
> programmed whenever dom0/domU calls the MAPTI command but not at the
> time of PCIe device creation.
I am a bit confused. Why are you speaking about direct vLPI here? This
has no relation with the IORT.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: xen/arm: Hiding SMMUs from Dom0 when using ACPI on Xen
2017-02-27 18:12 ` Julien Grall
@ 2017-05-18 11:59 ` Manish Jaggi
2017-05-18 14:57 ` Julien Grall
0 siblings, 1 reply; 10+ messages in thread
From: Manish Jaggi @ 2017-05-18 11:59 UTC (permalink / raw)
To: Julien Grall, shankerd, xen-devel
Cc: Stefano Stabellini, Steve Capper, Andre Przywara, Jiandi An,
Punit Agrawal, Sameer Goel, nd, Charles Garcia-Tobin
+Chales.
Hi Julien,
On 2/27/2017 11:42 PM, Julien Grall wrote:
> On 02/27/2017 04:58 PM, Shanker Donthineni wrote:
>> Hi Julien,
>
> Hi Shanker,
>
> Please don't drop people in CC. In my case, any e-mail I am not CCed
> are skipping my inbox and I may not read them for a while.
>
>>
>> On 02/27/2017 08:12 AM, Julien Grall wrote:
>>>
>>>
>>> On 27/02/17 13:23, Vijay Kilari wrote:
>>>> Hi Julien,
>>>
>>> Hello Vijay,
>>>
>>>> On Wed, Feb 22, 2017 at 7:40 PM, Julien Grall <julien.grall@arm.com>
>>>> wrote:
>>>>> Hello,
>>>>>
>>>>> There was few discussions recently about hiding SMMUs from DOM0 when
>>>>> using
>>>>> ACPI. I thought it would be good to have a separate thread for this.
>>>>>
>>>>> When using ACPI, the SMMUs will be described in the IO Remapping
>>>>> Table
>>>>> (IORT). The specification can be found on the ARM website [1].
>>>>>
>>>>> For a brief summary, the IORT can be used to discover the SMMUs
>>>>> present on
>>>>> the platform and find for a given device the ID to configure
>>>>> components such
>>>>> as ITS (DeviceID) and SMMU (StreamID).
>>>>>
>>>>> The appendix A in the specification gives an example how DeviceID and
>>>>> StreamID can be found. For instance, when a PCI device is both
>>>>> protected by
>>>>> an SMMU and MSI-capable the following translation will happen:
>>>>> RID -> StreamID -> DeviceID
>>>>>
>>>>> Currently, SMMUs are hidden from DOM0 because they are been used by
>>>>> Xen and
>>>>> we don't support stage-1 SMMU. If we pass the IORT as it is, DOM0
>>>>> will try
>>>>> to initialize SMMU and crash.
>>>>>
>>>>> I first thought about using a Xen specific way (STAO) or extending a
>>>>> flag in
>>>>> IORT. But that is not ideal.
>>>>>
>>>>> So we would have to rewrite the IORT for DOM0. Given that a range of
>>>>> RID can
>>>>> mapped to multiple ranges of DeviceID, we would have to translate
>>>>> RID one by
>>>>> one to find the associated DeviceID. I think this may end up to
>>>>> complex code
>>>>> and have a big IORT table.
>>>>
>>>> Why can't we replace Output base of IORT of PCI node with SMMU output
>>>> base?.
>>>> I mean similar to PCI node without SMMU, why can't replace output base
>>>> of PCI node with
>>>> SMMU's output base?.
>>>
>>> Because I don't see anything in the spec preventing one RC ID mapping
>>> to produce multiple SMMU ID mapping. So which output base would you
>>> use?
>>>
>>
>> Basically, remove SMMU nodes, and replaces output of the PCIe and named
>> nodes ID mappings with ITS nodes.
>>
>> RID --> StreamID --> dviceID --> ITS device id = RID --> dviceID -->
>> ITS device id
>
> Can you detail it? You seem to assume that one RC ID mapping range
> will only produce ID mapping range. AFAICT, this is not mandated by
> the spec.
>
You are correct that it is not mandated by the spec, but AFAIK there
seems to be no valid use case for that.
RID range should not overlap between ID Array entries.
I believe this would be updated in the next IORT spec revision.
I have started working on recreating iort for dom0 with this restriction.
>>
>>>>
>>>> The issue I see is RID is [15:0] where is DeviceID is [17:0].
>>>>
>> Actuality device id is 32bit field.
>>
>>>>>
>>>>> However, given that DeviceID will be used by DOM0 to only configure
>>>>> the ITS.
>>>>> We have no need to use to have the DOM0 DeviceID equal to the host
>>>>> DeviceID.
>>>>> So I think we could simplify our life by generating DeviceID for
>>>>> each RID
>>>>> range.
>>>>
>>>> If DOM0 DeviceID != host Device ID, then we cannot initialize ITS
>>>> using DOM0
>>>> ITS commands (MAPD). So, is it concluded that ITS initializes all the
>>>> devices
>>>> with platform specific Device ID's in Xen?.
>>>
>>> Initializing ITS using DOM0 ITS command is a workaround until we get
>>> PCI passthrough done. It would still be possible to implement that
>>> with vDeviceID != pDeviceID as Xen would likely have the mapping
>>> between the 2 DeviceID.
>>>
>>
>> I believe mapping dom0 ITS commands to XEN ITS commands one to one is
>> the better approach. Physical DeviceID is unique per ITS group, not a
>> system wide unique ID.
>
> As for guest, you don't care about the virtual DeviceID for DOM0 as
> long as you are able to map it to the host ITS and host DeviceID.
>
> > In case of direct VLPI, LPI number has to be
>> programmed whenever dom0/domU calls the MAPTI command but not at the
>> time of PCIe device creation.
>
> I am a bit confused. Why are you speaking about direct vLPI here? This
> has no relation with the IORT.
>
> Cheers,
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: xen/arm: Hiding SMMUs from Dom0 when using ACPI on Xen
2017-05-18 11:59 ` Manish Jaggi
@ 2017-05-18 14:57 ` Julien Grall
2017-05-18 20:02 ` Manish Jaggi
0 siblings, 1 reply; 10+ messages in thread
From: Julien Grall @ 2017-05-18 14:57 UTC (permalink / raw)
To: Manish Jaggi, shankerd, xen-devel
Cc: Stefano Stabellini, Steve Capper, Andre Przywara, Jiandi An,
Punit Agrawal, Sameer Goel, nd, Charles Garcia-Tobin
Hello,
On 18/05/17 12:59, Manish Jaggi wrote:
> On 2/27/2017 11:42 PM, Julien Grall wrote:
>> On 02/27/2017 04:58 PM, Shanker Donthineni wrote:
>>> Hi Julien,
>>
>> Hi Shanker,
>>
>> Please don't drop people in CC. In my case, any e-mail I am not CCed
>> are skipping my inbox and I may not read them for a while.
>>
>>>
>>> On 02/27/2017 08:12 AM, Julien Grall wrote:
>>>>
>>>>
>>>> On 27/02/17 13:23, Vijay Kilari wrote:
>>>>> Hi Julien,
>>>>
>>>> Hello Vijay,
>>>>
>>>>> On Wed, Feb 22, 2017 at 7:40 PM, Julien Grall <julien.grall@arm.com>
>>>>> wrote:
>>>>>> Hello,
>>>>>>
>>>>>> There was few discussions recently about hiding SMMUs from DOM0 when
>>>>>> using
>>>>>> ACPI. I thought it would be good to have a separate thread for this.
>>>>>>
>>>>>> When using ACPI, the SMMUs will be described in the IO Remapping
>>>>>> Table
>>>>>> (IORT). The specification can be found on the ARM website [1].
>>>>>>
>>>>>> For a brief summary, the IORT can be used to discover the SMMUs
>>>>>> present on
>>>>>> the platform and find for a given device the ID to configure
>>>>>> components such
>>>>>> as ITS (DeviceID) and SMMU (StreamID).
>>>>>>
>>>>>> The appendix A in the specification gives an example how DeviceID and
>>>>>> StreamID can be found. For instance, when a PCI device is both
>>>>>> protected by
>>>>>> an SMMU and MSI-capable the following translation will happen:
>>>>>> RID -> StreamID -> DeviceID
>>>>>>
>>>>>> Currently, SMMUs are hidden from DOM0 because they are been used by
>>>>>> Xen and
>>>>>> we don't support stage-1 SMMU. If we pass the IORT as it is, DOM0
>>>>>> will try
>>>>>> to initialize SMMU and crash.
>>>>>>
>>>>>> I first thought about using a Xen specific way (STAO) or extending a
>>>>>> flag in
>>>>>> IORT. But that is not ideal.
>>>>>>
>>>>>> So we would have to rewrite the IORT for DOM0. Given that a range of
>>>>>> RID can
>>>>>> mapped to multiple ranges of DeviceID, we would have to translate
>>>>>> RID one by
>>>>>> one to find the associated DeviceID. I think this may end up to
>>>>>> complex code
>>>>>> and have a big IORT table.
>>>>>
>>>>> Why can't we replace Output base of IORT of PCI node with SMMU output
>>>>> base?.
>>>>> I mean similar to PCI node without SMMU, why can't replace output base
>>>>> of PCI node with
>>>>> SMMU's output base?.
>>>>
>>>> Because I don't see anything in the spec preventing one RC ID mapping
>>>> to produce multiple SMMU ID mapping. So which output base would you
>>>> use?
>>>>
>>>
>>> Basically, remove SMMU nodes, and replaces output of the PCIe and named
>>> nodes ID mappings with ITS nodes.
>>>
>>> RID --> StreamID --> dviceID --> ITS device id = RID --> dviceID -->
>>> ITS device id
>>
>> Can you detail it? You seem to assume that one RC ID mapping range
>> will only produce ID mapping range. AFAICT, this is not mandated by
>> the spec.
>>
> You are correct that it is not mandated by the spec, but AFAIK there
> seems to be no valid use case for that.
Xen has to be compliant with the spec, if the spec says something then
we should do it unless there is a strong reason not to.
In this case, it is not too difficult to implement the suggestion I
wrote a couple of months ago. So why would we try to put us in a corner?
>
> RID range should not overlap between ID Array entries.
I believe you misunderstood my point here. So let me give an example. My
understanding of the spec is it is possible to have:
RC A
// doesn't use SMMU 0 so just outputs DeviceIDs to ITS GROUP 0
// Input ID --> Output reference: Output ID
0x0000-0xffff --> ITS GROUP 0 : 0x0000->0xffff
SMMU 0
// Note that range of StreamIDs that map to DeviceIDs excludes
// the NIC 0 DeviceID as it does not generate MSIs
// Input ID --> Output reference: Output ID
0x0000-0x01ff --> ITS GROUP 0 : 0x10000->0x101ff
0x0200-0xffff --> ITS GROUP 0 : 0x20000->0x207ff
// SMMU 0 Control interrupt is MSI based
// Input ID --> Output reference: Output ID
N/A --> ITS GROUP 0 : 0x200001
> I believe this would be updated in the next IORT spec revision.
Well, Xen should still support current revision of IORT even if the next
version add more restriction.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: xen/arm: Hiding SMMUs from Dom0 when using ACPI on Xen
2017-05-18 14:57 ` Julien Grall
@ 2017-05-18 20:02 ` Manish Jaggi
2017-05-18 20:09 ` Julien Grall
0 siblings, 1 reply; 10+ messages in thread
From: Manish Jaggi @ 2017-05-18 20:02 UTC (permalink / raw)
To: Julien Grall, shankerd, xen-devel
Cc: Stefano Stabellini, Steve Capper, Andre Przywara, Jiandi An,
Punit Agrawal, Sameer Goel, nd, Charles Garcia-Tobin
Hi Julien,
On 5/18/2017 8:27 PM, Julien Grall wrote:
> Hello,
>
> On 18/05/17 12:59, Manish Jaggi wrote:
>> On 2/27/2017 11:42 PM, Julien Grall wrote:
>>> On 02/27/2017 04:58 PM, Shanker Donthineni wrote:
>>>> Hi Julien,
>>>
>>> Hi Shanker,
>>>
>>> Please don't drop people in CC. In my case, any e-mail I am not CCed
>>> are skipping my inbox and I may not read them for a while.
>>>
>>>>
>>>> On 02/27/2017 08:12 AM, Julien Grall wrote:
>>>>>
>>>>>
>>>>> On 27/02/17 13:23, Vijay Kilari wrote:
>>>>>> Hi Julien,
>>>>>
>>>>> Hello Vijay,
>>>>>
>>>>>> On Wed, Feb 22, 2017 at 7:40 PM, Julien Grall <julien.grall@arm.com>
>>>>>> wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> There was few discussions recently about hiding SMMUs from DOM0
>>>>>>> when
>>>>>>> using
>>>>>>> ACPI. I thought it would be good to have a separate thread for
>>>>>>> this.
>>>>>>>
>>>>>>> When using ACPI, the SMMUs will be described in the IO Remapping
>>>>>>> Table
>>>>>>> (IORT). The specification can be found on the ARM website [1].
>>>>>>>
>>>>>>> For a brief summary, the IORT can be used to discover the SMMUs
>>>>>>> present on
>>>>>>> the platform and find for a given device the ID to configure
>>>>>>> components such
>>>>>>> as ITS (DeviceID) and SMMU (StreamID).
>>>>>>>
>>>>>>> The appendix A in the specification gives an example how
>>>>>>> DeviceID and
>>>>>>> StreamID can be found. For instance, when a PCI device is both
>>>>>>> protected by
>>>>>>> an SMMU and MSI-capable the following translation will happen:
>>>>>>> RID -> StreamID -> DeviceID
>>>>>>>
>>>>>>> Currently, SMMUs are hidden from DOM0 because they are been used by
>>>>>>> Xen and
>>>>>>> we don't support stage-1 SMMU. If we pass the IORT as it is, DOM0
>>>>>>> will try
>>>>>>> to initialize SMMU and crash.
>>>>>>>
>>>>>>> I first thought about using a Xen specific way (STAO) or
>>>>>>> extending a
>>>>>>> flag in
>>>>>>> IORT. But that is not ideal.
>>>>>>>
>>>>>>> So we would have to rewrite the IORT for DOM0. Given that a
>>>>>>> range of
>>>>>>> RID can
>>>>>>> mapped to multiple ranges of DeviceID,
Do you envisage a scenario where same RID can map to multiple StreamIDs
belonging to different SMMUs ?
>>>>>>> we would have to translate
>>>>>>> RID one by
>>>>>>> one to find the associated DeviceID. I think this may end up to
>>>>>>> complex code
>>>>>>> and have a big IORT table.
>>>>>>
>>>>>> Why can't we replace Output base of IORT of PCI node with SMMU
>>>>>> output
>>>>>> base?.
>>>>>> I mean similar to PCI node without SMMU, why can't replace output
>>>>>> base
>>>>>> of PCI node with
>>>>>> SMMU's output base?.
>>>>>
>>>>> Because I don't see anything in the spec preventing one RC ID mapping
>>>>> to produce multiple SMMU ID mapping. So which output base would you
>>>>> use?
>>>>>
>>>>
>>>> Basically, remove SMMU nodes, and replaces output of the PCIe and
>>>> named
>>>> nodes ID mappings with ITS nodes.
>>>>
>>>> RID --> StreamID --> dviceID --> ITS device id = RID --> dviceID
>>>> -->
>>>> ITS device id
>>>
>>> Can you detail it? You seem to assume that one RC ID mapping range
>>> will only produce ID mapping range. AFAICT, this is not mandated by
>>> the spec.
>>>
>> You are correct that it is not mandated by the spec, but AFAIK there
>> seems to be no valid use case for that.
>
> Xen has to be compliant with the spec, if the spec says something then
> we should do it unless there is a strong reason not to.
>
> In this case, it is not too difficult to implement the suggestion I
> wrote a couple of months ago. So why would we try to put us in a corner?
>
See below
>>
>> RID range should not overlap between ID Array entries.
>
> I believe you misunderstood my point here. So let me give an example.
> My understanding of the spec is it is possible to have:
>
> RC A
> // doesn't use SMMU 0 so just outputs DeviceIDs to ITS GROUP 0
> // Input ID --> Output reference: Output ID
> 0x0000-0xffff --> ITS GROUP 0 : 0x0000->0xffff
>
> SMMU 0
> // Note that range of StreamIDs that map to DeviceIDs excludes
> // the NIC 0 DeviceID as it does not generate MSIs
> // Input ID --> Output reference: Output ID
> 0x0000-0x01ff --> ITS GROUP 0 : 0x10000->0x101ff
> 0x0200-0xffff --> ITS GROUP 0 : 0x20000->0x207ff
>
> // SMMU 0 Control interrupt is MSI based
> // Input ID --> Output reference: Output ID
> N/A --> ITS GROUP 0 : 0x200001
>
I could have misunderstood so I am stating my understanding so far ..
please feel free to correct me :)
In the IORT table using the PCI-RC node, SMMU node and ITS node,
RID->StreamID->Device-ID mapping can be generated.
As per IORT spec toady, same RID can be mapped to different StreamIDs
using two ID Array elements with same RID range but different output
reference.
There exists no use case for such a scenario hence a clarification is
required in IORT spec which states that RID range cannot overlap in the
ID array.
with this clarification in place, it is straight-forward to map RID to a
device-ID by replacing output of SMMU to output of RCI-RC
>> I believe this would be updated in the next IORT spec revision.
>
> Well, Xen should still support current revision of IORT even if the
> next version add more restriction.
>
> Cheers,
>
-Manish
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: xen/arm: Hiding SMMUs from Dom0 when using ACPI on Xen
2017-05-18 20:02 ` Manish Jaggi
@ 2017-05-18 20:09 ` Julien Grall
2017-06-08 12:38 ` Manish Jaggi
0 siblings, 1 reply; 10+ messages in thread
From: Julien Grall @ 2017-05-18 20:09 UTC (permalink / raw)
To: Manish Jaggi, shankerd, xen-devel
Cc: Stefano Stabellini, Steve Capper, Andre Przywara, Jiandi An,
Punit Agrawal, Sameer Goel, nd, Charles Garcia-Tobin
On 18/05/2017 21:02, Manish Jaggi wrote:
> In the IORT table using the PCI-RC node, SMMU node and ITS node,
> RID->StreamID->Device-ID mapping can be generated.
> As per IORT spec toady, same RID can be mapped to different StreamIDs
> using two ID Array elements with same RID range but different output
> reference.
> There exists no use case for such a scenario hence a clarification is
> required in IORT spec which states that RID range cannot overlap in the
> ID array.
I understand that.
>
> with this clarification in place, it is straight-forward to map RID to a
> device-ID by replacing output of SMMU to output of RCI-RC
I am not sure to follow your suggestion here. But I will wait a patch
before commenting.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: xen/arm: Hiding SMMUs from Dom0 when using ACPI on Xen
2017-05-18 20:09 ` Julien Grall
@ 2017-06-08 12:38 ` Manish Jaggi
0 siblings, 0 replies; 10+ messages in thread
From: Manish Jaggi @ 2017-06-08 12:38 UTC (permalink / raw)
To: Julien Grall, shankerd, xen-devel
Cc: Stefano Stabellini, Steve Capper, Andre Przywara, Jiandi An,
Punit Agrawal, Sameer Goel, nd, Charles Garcia-Tobin
On 5/19/2017 1:39 AM, Julien Grall wrote:
>
>
> On 18/05/2017 21:02, Manish Jaggi wrote:
>> In the IORT table using the PCI-RC node, SMMU node and ITS node,
>> RID->StreamID->Device-ID mapping can be generated.
>> As per IORT spec toady, same RID can be mapped to different StreamIDs
>> using two ID Array elements with same RID range but different output
>> reference.
>> There exists no use case for such a scenario hence a clarification is
>> required in IORT spec which states that RID range cannot overlap in the
>> ID array.
>
> I understand that.
>
>>
>> with this clarification in place, it is straight-forward to map RID to a
>> device-ID by replacing output of SMMU to output of RCI-RC
>
> I am not sure to follow your suggestion here. But I will wait a patch
> before commenting.
>
Please see [RFC] [PATCH] arm-acpi: Hide SMMU from IORT for hardware domain
> Cheers,
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2017-06-08 12:38 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-02-22 14:10 xen/arm: Hiding SMMUs from Dom0 when using ACPI on Xen Julien Grall
2017-02-27 13:23 ` Vijay Kilari
2017-02-27 14:12 ` Julien Grall
2017-02-27 16:58 ` Shanker Donthineni
2017-02-27 18:12 ` Julien Grall
2017-05-18 11:59 ` Manish Jaggi
2017-05-18 14:57 ` Julien Grall
2017-05-18 20:02 ` Manish Jaggi
2017-05-18 20:09 ` Julien Grall
2017-06-08 12:38 ` Manish Jaggi
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.