[PATCH 0/2] Skip IH re-route on Vega SR-IOV

[PATCH 0/2] Skip IH re-route on Vega SR-IOV

[Trigger Huang]

IH re-route is not supported on Vega SR-IOV, need to be skipped

Trigger Huang (2):
  drm/amdgpu: Skip IH reroute in Vega10 SR-IOV VF
  drm/amdgpu: Skip IH reroute in Vega20 SR-IOV VF

 drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 4 ++++
 drivers/gpu/drm/amd/amdgpu/psp_v3_1.c  | 4 ++++
 2 files changed, 8 insertions(+)

-- 
2.7.4

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 1/2] drm/amdgpu: Skip IH reroute in Vega10 SR-IOV VF

[Trigger Huang]

IH reroute commands are not supported on Vega10 VF

Signed-off-by: Trigger Huang <Trigger.Huang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/psp_v3_1.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c
index 143f0fa..9d6e603 100644
--- a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c
@@ -260,6 +260,10 @@ static void psp_v3_1_reroute_ih(struct psp_context *psp)
 	struct amdgpu_device *adev = psp->adev;
 	uint32_t tmp;
 
+	/* reroute_ih is not supported on SR_IOV VF */
+	if (amdgpu_sriov_vf(adev))
+		return;
+
 	/* Change IH ring for VMC */
 	tmp = REG_SET_FIELD(0, IH_CLIENT_CFG_DATA, CREDIT_RETURN_ADDR, 0x1244b);
 	tmp = REG_SET_FIELD(tmp, IH_CLIENT_CFG_DATA, CLIENT_TYPE, 1);
-- 
2.7.4

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 2/2] drm/amdgpu: Skip IH reroute in Vega20 SR-IOV VF

[Trigger Huang]

IH reroute commands are not supported on Vega20 VF

Signed-off-by: Trigger Huang <Trigger.Huang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c
index b91df7b..4bdd70a 100644
--- a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c
@@ -232,6 +232,10 @@ static void psp_v11_0_reroute_ih(struct psp_context *psp)
 	struct amdgpu_device *adev = psp->adev;
 	uint32_t tmp;
 
+	/* reroute_ih is not supported on SR_IOV VF */
+	if (amdgpu_sriov_vf(adev))
+		return;
+
 	/* Change IH ring for VMC */
 	tmp = REG_SET_FIELD(0, IH_CLIENT_CFG_DATA, CREDIT_RETURN_ADDR, 0x1244b);
 	tmp = REG_SET_FIELD(tmp, IH_CLIENT_CFG_DATA, CLIENT_TYPE, 1);
-- 
2.7.4

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 0/2] Skip IH re-route on Vega SR-IOV

[Christian König]

We intentionally didn't do this to make sure that the commands are 
ignored by the PSP firmware.

I have no strong opinion on if we should do this or not, but the PSP 
firmware guys might have.

Christian.

Am 07.05.19 um 06:08 schrieb Trigger Huang:
> IH re-route is not supported on Vega SR-IOV, need to be skipped
>
> Trigger Huang (2):
>    drm/amdgpu: Skip IH reroute in Vega10 SR-IOV VF
>    drm/amdgpu: Skip IH reroute in Vega20 SR-IOV VF
>
>   drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 4 ++++
>   drivers/gpu/drm/amd/amdgpu/psp_v3_1.c  | 4 ++++
>   2 files changed, 8 insertions(+)
>

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

RE: [PATCH 0/2] Skip IH re-route on Vega SR-IOV

[Huang, Trigger]

Hi Christian,

On Vega10 SR-IOV VF, I injected a 'real' VMC page fault from user space, using the modified amdgpu_test.
[   19.127874] amdgpu 0000:00:08.0: [gfxhub] no-retry page fault (src_id:0 ring:174 vmid:1 pasid:32768, for process amdgpu_test pid 1071 thread amdgpu_test pid 1071)
[   19.130037] amdgpu 0000:00:08.0:   in page starting at address 0x0000000800000000 from 27

And see this interrupt is still from IH0 amdgpu_irq_handler, which can prove this feature is not working under SR-IOV.

I suggest to remove this feature from SR-IOV, as my concern is,  some weird bugs may be cased by it in the Virtualization heavy stress test.
In the future, maybe we can request PSP team to add this support for SR-IOV.

Thanks & Best Wishes,
Trigger Huang

-----Original Message-----
From: Christian König <ckoenig.leichtzumerken@gmail.com> 
Sent: Tuesday, May 07, 2019 3:37 PM
To: Huang, Trigger <Trigger.Huang@amd.com>; amd-gfx@lists.freedesktop.org
Subject: Re: [PATCH 0/2] Skip IH re-route on Vega SR-IOV

[CAUTION: External Email]

We intentionally didn't do this to make sure that the commands are ignored by the PSP firmware.

I have no strong opinion on if we should do this or not, but the PSP firmware guys might have.

Christian.

Am 07.05.19 um 06:08 schrieb Trigger Huang:
> IH re-route is not supported on Vega SR-IOV, need to be skipped
>
> Trigger Huang (2):
>    drm/amdgpu: Skip IH reroute in Vega10 SR-IOV VF
>    drm/amdgpu: Skip IH reroute in Vega20 SR-IOV VF
>
>   drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 4 ++++
>   drivers/gpu/drm/amd/amdgpu/psp_v3_1.c  | 4 ++++
>   2 files changed, 8 insertions(+)
>

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 0/2] Skip IH re-route on Vega SR-IOV

[Christian König]

Hi Trigger,

> And see this interrupt is still from IH0 amdgpu_irq_handler, which can prove this feature is not working under SR-IOV.
In this case this change is a clear NAK.

> I suggest to remove this feature from SR-IOV, as my concern is,  some weird bugs may be cased by it in the Virtualization heavy stress test.
And I really think we should keep it to make sure that we have the same 
handling for bare metal as for SRIOV.

> In the future, maybe we can request PSP team to add this support for SR-IOV.
We will never be able to use this under SRIOV because of limitation in 
the hardware.

What we could maybe do is check the response code from the PSP firmware 
if it correctly ignored the commands under SR-IOV, but I think the 
response code is the same for ignoring as for executing the commands.

Regards,
Christian.

Am 07.05.19 um 10:54 schrieb Huang, Trigger:
> Hi Christian,
>
> On Vega10 SR-IOV VF, I injected a 'real' VMC page fault from user space, using the modified amdgpu_test.
> [   19.127874] amdgpu 0000:00:08.0: [gfxhub] no-retry page fault (src_id:0 ring:174 vmid:1 pasid:32768, for process amdgpu_test pid 1071 thread amdgpu_test pid 1071)
> [   19.130037] amdgpu 0000:00:08.0:   in page starting at address 0x0000000800000000 from 27
>
> And see this interrupt is still from IH0 amdgpu_irq_handler, which can prove this feature is not working under SR-IOV.
>
> I suggest to remove this feature from SR-IOV, as my concern is,  some weird bugs may be cased by it in the Virtualization heavy stress test.
> In the future, maybe we can request PSP team to add this support for SR-IOV.
>
> Thanks & Best Wishes,
> Trigger Huang
>
> -----Original Message-----
> From: Christian König <ckoenig.leichtzumerken@gmail.com>
> Sent: Tuesday, May 07, 2019 3:37 PM
> To: Huang, Trigger <Trigger.Huang@amd.com>; amd-gfx@lists.freedesktop.org
> Subject: Re: [PATCH 0/2] Skip IH re-route on Vega SR-IOV
>
> [CAUTION: External Email]
>
> We intentionally didn't do this to make sure that the commands are ignored by the PSP firmware.
>
> I have no strong opinion on if we should do this or not, but the PSP firmware guys might have.
>
> Christian.
>
> Am 07.05.19 um 06:08 schrieb Trigger Huang:
>> IH re-route is not supported on Vega SR-IOV, need to be skipped
>>
>> Trigger Huang (2):
>>     drm/amdgpu: Skip IH reroute in Vega10 SR-IOV VF
>>     drm/amdgpu: Skip IH reroute in Vega20 SR-IOV VF
>>
>>    drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 4 ++++
>>    drivers/gpu/drm/amd/amdgpu/psp_v3_1.c  | 4 ++++
>>    2 files changed, 8 insertions(+)
>>
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

RE: [PATCH 0/2] Skip IH re-route on Vega SR-IOV

[Huang, Trigger]

OK, thanks for the detailed background,  before I didn't  know the limitation in the hardware.

Thanks & Best Wishes,
Trigger Huang

-----Original Message-----
From: Christian König <ckoenig.leichtzumerken@gmail.com> 
Sent: Tuesday, May 07, 2019 5:04 PM
To: Huang, Trigger <Trigger.Huang@amd.com>; Koenig, Christian <Christian.Koenig@amd.com>; amd-gfx@lists.freedesktop.org
Subject: Re: [PATCH 0/2] Skip IH re-route on Vega SR-IOV

[CAUTION: External Email]

Hi Trigger,

> And see this interrupt is still from IH0 amdgpu_irq_handler, which can prove this feature is not working under SR-IOV.
In this case this change is a clear NAK.

> I suggest to remove this feature from SR-IOV, as my concern is,  some weird bugs may be cased by it in the Virtualization heavy stress test.
And I really think we should keep it to make sure that we have the same handling for bare metal as for SRIOV.

> In the future, maybe we can request PSP team to add this support for SR-IOV.
We will never be able to use this under SRIOV because of limitation in the hardware.

What we could maybe do is check the response code from the PSP firmware if it correctly ignored the commands under SR-IOV, but I think the response code is the same for ignoring as for executing the commands.

Regards,
Christian.

Am 07.05.19 um 10:54 schrieb Huang, Trigger:
> Hi Christian,
>
> On Vega10 SR-IOV VF, I injected a 'real' VMC page fault from user space, using the modified amdgpu_test.
> [   19.127874] amdgpu 0000:00:08.0: [gfxhub] no-retry page fault (src_id:0 ring:174 vmid:1 pasid:32768, for process amdgpu_test pid 1071 thread amdgpu_test pid 1071)
> [   19.130037] amdgpu 0000:00:08.0:   in page starting at address 0x0000000800000000 from 27
>
> And see this interrupt is still from IH0 amdgpu_irq_handler, which can prove this feature is not working under SR-IOV.
>
> I suggest to remove this feature from SR-IOV, as my concern is,  some weird bugs may be cased by it in the Virtualization heavy stress test.
> In the future, maybe we can request PSP team to add this support for SR-IOV.
>
> Thanks & Best Wishes,
> Trigger Huang
>
> -----Original Message-----
> From: Christian König <ckoenig.leichtzumerken@gmail.com>
> Sent: Tuesday, May 07, 2019 3:37 PM
> To: Huang, Trigger <Trigger.Huang@amd.com>; 
> amd-gfx@lists.freedesktop.org
> Subject: Re: [PATCH 0/2] Skip IH re-route on Vega SR-IOV
>
> [CAUTION: External Email]
>
> We intentionally didn't do this to make sure that the commands are ignored by the PSP firmware.
>
> I have no strong opinion on if we should do this or not, but the PSP firmware guys might have.
>
> Christian.
>
> Am 07.05.19 um 06:08 schrieb Trigger Huang:
>> IH re-route is not supported on Vega SR-IOV, need to be skipped
>>
>> Trigger Huang (2):
>>     drm/amdgpu: Skip IH reroute in Vega10 SR-IOV VF
>>     drm/amdgpu: Skip IH reroute in Vega20 SR-IOV VF
>>
>>    drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 4 ++++
>>    drivers/gpu/drm/amd/amdgpu/psp_v3_1.c  | 4 ++++
>>    2 files changed, 8 insertions(+)
>>
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx