A question about the translation granule size supported by the vSMMU

A question about the translation granule size supported by the vSMMU

[Kunkun Jiang]

Hi all,

Recently, I did some tests on SMMU nested mode. Here is
a question about the translation granule size supported by
vSMMU.

There is such a code in SMMUv3_init_regs():

>    /* 4K and 64K granule support */
>     s->idr[5] = FIELD_DP32(s->idr[5], IDR5, GRAN4K, 1);
>     s->idr[5] = FIELD_DP32(s->idr[5], IDR5, GRAN64K, 1);
>     s->idr[5] = FIELD_DP32(s->idr[5], IDR5, OAS, SMMU_IDR5_OAS); /* 44 
> bits */
Why is the 16K granule not supported? I modified the code
to support it and did not encounter any problems in the
test. Although 4K and 64K minimal granules are "strongly
recommended", I think vSMMU should still support 16K.😉
Are there other reasons why 16K is not supported here?

When in SMMU nested mode, it may get errors if pSMMU
doesn't support 16K but vSMMU supports 16K. But we
can get some settings of pSMMU to avoid this situation.
I found some discussions between Eric and Linu about
this [1], but this idea does not seem to be implemented.

[1] https://lists.gnu.org/archive/html/qemu-arm/2017-09/msg00149.html

Best regards,
Kunkun Jiang

Re: A question about the translation granule size supported by the vSMMU

[Auger Eric]

Hi Kunkun,

On 3/27/21 3:24 AM, Kunkun Jiang wrote:
> Hi all,
> 
> Recently, I did some tests on SMMU nested mode. Here is
> a question about the translation granule size supported by
> vSMMU.
> 
> There is such a code in SMMUv3_init_regs():
> 
>>    /* 4K and 64K granule support */
>>     s->idr[5] = FIELD_DP32(s->idr[5], IDR5, GRAN4K, 1);
>>     s->idr[5] = FIELD_DP32(s->idr[5], IDR5, GRAN64K, 1);
>>     s->idr[5] = FIELD_DP32(s->idr[5], IDR5, OAS, SMMU_IDR5_OAS); /* 44
>> bits */
> Why is the 16K granule not supported? I modified the code
> to support it and did not encounter any problems in the
> test. Although 4K and 64K minimal granules are "strongly
> recommended", I think vSMMU should still support 16K.😉
> Are there other reasons why 16K is not supported here?
no there aren't any. The main reasons were 16KB support is optional and
supporting it increases the test matrix. Also it seems quite a few
machines I have access to do support 16KB granule. On the others I get

"EFI stub: ERROR: This 16 KB granular kernel is not supported by your CPU".

Nevertheless I am not opposed to support it as it seems to work without
trouble. Just need to have an extra look at implied validity checks but
there shouldn't be much.

Thanks

Eric
> 
> When in SMMU nested mode, it may get errors if pSMMU
> doesn't support 16K but vSMMU supports 16K. But we
> can get some settings of pSMMU to avoid this situation.
> I found some discussions between Eric and Linu about
> this [1], but this idea does not seem to be implemented.
> 
> [1] https://lists.gnu.org/archive/html/qemu-arm/2017-09/msg00149.html
> 
> Best regards,
> Kunkun Jiang
> 

Re: A question about the translation granule size supported by the vSMMU

[Kunkun Jiang]

Hi Eric,

On 2021/4/7 3:50, Auger Eric wrote:
> Hi Kunkun,
>
> On 3/27/21 3:24 AM, Kunkun Jiang wrote:
>> Hi all,
>>
>> Recently, I did some tests on SMMU nested mode. Here is
>> a question about the translation granule size supported by
>> vSMMU.
>>
>> There is such a code in SMMUv3_init_regs():
>>
>>>     /* 4K and 64K granule support */
>>>      s->idr[5] = FIELD_DP32(s->idr[5], IDR5, GRAN4K, 1);
>>>      s->idr[5] = FIELD_DP32(s->idr[5], IDR5, GRAN64K, 1);
>>>      s->idr[5] = FIELD_DP32(s->idr[5], IDR5, OAS, SMMU_IDR5_OAS); /* 44
>>> bits */
>> Why is the 16K granule not supported? I modified the code
>> to support it and did not encounter any problems in the
>> test. Although 4K and 64K minimal granules are "strongly
>> recommended", I think vSMMU should still support 16K.😉
>> Are there other reasons why 16K is not supported here?
> no there aren't any. The main reasons were 16KB support is optional and
> supporting it increases the test matrix. Also it seems quite a few
> machines I have access to do support 16KB granule. On the others I get
>
> "EFI stub: ERROR: This 16 KB granular kernel is not supported by your CPU".
>
> Nevertheless I am not opposed to support it as it seems to work without
> trouble. Just need to have an extra look at implied validity checks but
> there shouldn't be much.
>
> Thanks
>
> Eric
Yes, you are right. In my opinion, it is necessary to check whether pSMMU
supports 16K to avoid the situation I mentioned below.
In SMMU nested mode, if vSMMU supports 16K and set pasid table to
pSMMU, it may get errors when pSMMU does translation table walk if
pSMMU doesn't support 16K (not tested). Do you think we need to add
an interface to get some pSMMU info?

Maybe my consideration was superfluous.😁

Thanks,
Kunkun Jiang
>> When in SMMU nested mode, it may get errors if pSMMU
>> doesn't support 16K but vSMMU supports 16K. But we
>> can get some settings of pSMMU to avoid this situation.
>> I found some discussions between Eric and Linu about
>> this [1], but this idea does not seem to be implemented.
>>
>> [1] https://lists.gnu.org/archive/html/qemu-arm/2017-09/msg00149.html
>>
>> Best regards,
>> Kunkun Jiang
>>
>
> .

Re: A question about the translation granule size supported by the vSMMU

[Auger Eric]

Hi Kunkun,

On 4/7/21 11:26 AM, Kunkun Jiang wrote:
> Hi Eric,
> 
> On 2021/4/7 3:50, Auger Eric wrote:
>> Hi Kunkun,
>>
>> On 3/27/21 3:24 AM, Kunkun Jiang wrote:
>>> Hi all,
>>>
>>> Recently, I did some tests on SMMU nested mode. Here is
>>> a question about the translation granule size supported by
>>> vSMMU.
>>>
>>> There is such a code in SMMUv3_init_regs():
>>>
>>>>     /* 4K and 64K granule support */
>>>>      s->idr[5] = FIELD_DP32(s->idr[5], IDR5, GRAN4K, 1);
>>>>      s->idr[5] = FIELD_DP32(s->idr[5], IDR5, GRAN64K, 1);
>>>>      s->idr[5] = FIELD_DP32(s->idr[5], IDR5, OAS, SMMU_IDR5_OAS); /* 44
>>>> bits */
>>> Why is the 16K granule not supported? I modified the code
>>> to support it and did not encounter any problems in the
>>> test. Although 4K and 64K minimal granules are "strongly
>>> recommended", I think vSMMU should still support 16K.😉
>>> Are there other reasons why 16K is not supported here?
>> no there aren't any. The main reasons were 16KB support is optional and
>> supporting it increases the test matrix. Also it seems quite a few
>> machines I have access to do support 16KB granule. On the others I get
>>
>> "EFI stub: ERROR: This 16 KB granular kernel is not supported by your
>> CPU".
>>
>> Nevertheless I am not opposed to support it as it seems to work without
>> trouble. Just need to have an extra look at implied validity checks but
>> there shouldn't be much.
>>
>> Thanks
>>
>> Eric
> Yes, you are right. In my opinion, it is necessary to check whether pSMMU
> supports 16K to avoid the situation I mentioned below.
> In SMMU nested mode, if vSMMU supports 16K and set pasid table to
> pSMMU, it may get errors when pSMMU does translation table walk if
> pSMMU doesn't support 16K (not tested). Do you think we need to add
> an interface to get some pSMMU info?>
> Maybe my consideration was superfluous.😁
No it is not. At qemu level we have
memory_region_iommu_set_page_size_mask() that is called from the VFIO
device. It allows to pass such info to the IOMMU device (qemu
b91774984249).

iommu_set_page_size_mask() cb needs to be implemented at SMMU QEMU
device level. Also [PATCH 0/2] Domain nesting info for arm-smmu may
allow to return other constraints from the pSMMU.

Thanks

Eric
> 
> Thanks,
> Kunkun Jiang
>>> When in SMMU nested mode, it may get errors if pSMMU
>>> doesn't support 16K but vSMMU supports 16K. But we
>>> can get some settings of pSMMU to avoid this situation.
>>> I found some discussions between Eric and Linu about
>>> this [1], but this idea does not seem to be implemented.
>>>
>>> [1] https://lists.gnu.org/archive/html/qemu-arm/2017-09/msg00149.html
>>>
>>> Best regards,
>>> Kunkun Jiang
>>>
>>
>> .
> 
> 

Re: A question about the translation granule size supported by the vSMMU

[Kunkun Jiang]

Hi Eric,

On 2021/4/8 15:27, Auger Eric wrote:
> Hi Kunkun,
>
> On 4/7/21 11:26 AM, Kunkun Jiang wrote:
>> Hi Eric,
>>
>> On 2021/4/7 3:50, Auger Eric wrote:
>>> Hi Kunkun,
>>>
>>> On 3/27/21 3:24 AM, Kunkun Jiang wrote:
>>>> Hi all,
>>>>
>>>> Recently, I did some tests on SMMU nested mode. Here is
>>>> a question about the translation granule size supported by
>>>> vSMMU.
>>>>
>>>> There is such a code in SMMUv3_init_regs():
>>>>
>>>>>      /* 4K and 64K granule support */
>>>>>       s->idr[5] = FIELD_DP32(s->idr[5], IDR5, GRAN4K, 1);
>>>>>       s->idr[5] = FIELD_DP32(s->idr[5], IDR5, GRAN64K, 1);
>>>>>       s->idr[5] = FIELD_DP32(s->idr[5], IDR5, OAS, SMMU_IDR5_OAS); /* 44
>>>>> bits */
>>>> Why is the 16K granule not supported? I modified the code
>>>> to support it and did not encounter any problems in the
>>>> test. Although 4K and 64K minimal granules are "strongly
>>>> recommended", I think vSMMU should still support 16K.😉
>>>> Are there other reasons why 16K is not supported here?
>>> no there aren't any. The main reasons were 16KB support is optional and
>>> supporting it increases the test matrix. Also it seems quite a few
>>> machines I have access to do support 16KB granule. On the others I get
>>>
>>> "EFI stub: ERROR: This 16 KB granular kernel is not supported by your
>>> CPU".
>>>
>>> Nevertheless I am not opposed to support it as it seems to work without
>>> trouble. Just need to have an extra look at implied validity checks but
>>> there shouldn't be much.
>>>
>>> Thanks
>>>
>>> Eric
>> Yes, you are right. In my opinion, it is necessary to check whether pSMMU
>> supports 16K to avoid the situation I mentioned below.
>> In SMMU nested mode, if vSMMU supports 16K and set pasid table to
>> pSMMU, it may get errors when pSMMU does translation table walk if
>> pSMMU doesn't support 16K (not tested). Do you think we need to add
>> an interface to get some pSMMU info?>
>> Maybe my consideration was superfluous.😁
> No it is not. At qemu level we have
> memory_region_iommu_set_page_size_mask() that is called from the VFIO
> device. It allows to pass such info to the IOMMU device (qemu
> b91774984249).
>
> iommu_set_page_size_mask() cb needs to be implemented at SMMU QEMU
> device level. Also [PATCH 0/2] Domain nesting info for arm-smmu may
> allow to return other constraints from the pSMMU.
>
> Thanks
>
> Eric
Ok, it makes sense to me.
I am glad to test them if these patches are ready.

Thanks,
Kunkun Jiang
>> Thanks,
>> Kunkun Jiang
>>>> When in SMMU nested mode, it may get errors if pSMMU
>>>> doesn't support 16K but vSMMU supports 16K. But we
>>>> can get some settings of pSMMU to avoid this situation.
>>>> I found some discussions between Eric and Linu about
>>>> this [1], but this idea does not seem to be implemented.
>>>>
>>>> [1] https://lists.gnu.org/archive/html/qemu-arm/2017-09/msg00149.html
>>>>
>>>> Best regards,
>>>> Kunkun Jiang
>>>>
>>> .
>>
> .