From: "Christian König" <christian.koenig@amd.com>
To: Felix Kuehling <felix.kuehling@amd.com>, amd-gfx@lists.freedesktop.org
Subject: Re: [PATCH 2/2] drm/amdgpu: Let KFD use more VMIDs on Arcturus
Date: Fri, 26 Jun 2020 08:57:41 +0200 [thread overview]
Message-ID: <927d4503-536d-53e9-0a13-9a01b6ab2f0a@amd.com> (raw)
In-Reply-To: <bcca1d9b-a260-347e-be7c-b57ab4b0a700@amd.com>
Am 25.06.20 um 22:39 schrieb Felix Kuehling:
>
> On 2020-06-25 12:01 p.m., Christian König wrote:
>> Am 25.06.20 um 17:58 schrieb Felix Kuehling:
>>> Am 2020-06-25 um 11:50 a.m. schrieb Christian König:
>>>> Am 25.06.20 um 17:43 schrieb Felix Kuehling:
>>>>> Am 2020-06-25 um 11:38 a.m. schrieb Christian König:
>>>>>> Am 25.06.20 um 17:15 schrieb Felix Kuehling:
>>>>>>> Am 2020-06-25 um 4:19 a.m. schrieb Christian König:
>>>>>>>> Am 25.06.20 um 05:18 schrieb Felix Kuehling:
>>>>>>>>> When there is no graphics support, KFD can use more of the VMIDs.
>>>>>>>>> Graphics
>>>>>>>>> VMIDs are only used for video decoding/encoding and post
>>>>>>>>> processing.
>>>>>>>>> With
>>>>>>>>> two VCE engines, there is no reason to reserve more than 2
>>>>>>>>> VMIDs for
>>>>>>>>> that.
>>>>>>>> IIRC the expectation is that we still use the compute queues
>>>>>>>> for post
>>>>>>>> processing and not the KFD.
>>>>>>>>
>>>>>>>> So we will need at least VMIDs for that as well.
>>>>>>> Correct. Post processing uses compute queues and VMIDs in the
>>>>>>> GFXHUB.
>>>>>>> VCE uses VMIDs in the MMHUB. I believe in Mesa they use the same VM
>>>>>>> context. So can't they share the same VMIDs?
>>>>>> Ah! Good point, But we still need at least 3 VMID when VMID
>>>>>> reservation is active.
>>>>> I don't know anything about that VMID reservation feature. What is it
>>>>> used for? Who is using it? How many VMIDs can be reserved?
>>>>>
>>>>> If one VMID is reserved, there would still be one VMID left for video
>>>>> post processing. That's not ideal, but I don't think it would be
>>>>> fatal.
>>>>> But is it a realistic use case that VMID reservation and ROCm+video
>>>>> processing would happen on the same system at the same time?
>>>> VMID reservation is used for debugging and yes there can only be one
>>>> reserved.
>>>>
>>>> But I think we need at least two for dynamic assignment or we might
>>>> run into a BUG_ON() while giving VMIDs to jobs.
>>> I don't see any BUGs or BUG_ONs in amdgpu_ids.c. What should I be
>>> looking out for?
>>
>> We used to have a BUG_ON() when we couldn't find a VMID as
>> alternative if the process already has one but needs to flush it.
>>
>> But it's a long time ago that I last looked into this.
>>
>>>> But I certainly need to test this as well. It's possible that 1 VMID
>>>> indeed works as expected.
>>> I could run the test, if you describe the problematic scenario you have
>>> in mind.
>>
>> Just try to set the available VMIDs to 1 and see if GFX/Compute and
>> MM submission work at the same time from multiple processes.
>>
>> A few UVD video playbacks at the same time should do the job.
>
> I tested it on Fiji with first_kfd_vmid=2, running 3 instances of VLC
> playing a 1080p h.264 video using VDPAU. (For some reason VA-API is
> broken: vaDeriveImage: operation failed). Just to be sure it was
> really using UVD, I disabled HW acceleration in VLC and saw CPU usage
> increase. Everything seems to be working fine.
In this case the patch is Reviewed-by: Christian König
<christian.koenig@amd.com>
>
> Regards,
> Felix
>
>
>>
>> Regards,
>> Christian.
>>
>>>
>>> Thanks,
>>> Felix
>>>
>>>
>>>> Regards,
>>>> Christian.
>>>>
>>>>> Thanks,
>>>>> Felix
>>>>>
>>>>>
>>>>>> I don't think you can go below this.
>>>>>>
>>>>>> Regards,
>>>>>> Christian.
>>>>>>
>>>>>>> Regards,
>>>>>>> Felix
>>>>>>>
>>>>>>>
>>>>>>>>> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
>>>>>>>>> ---
>>>>>>>>> drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 11 ++++++++---
>>>>>>>>> 1 file changed, 8 insertions(+), 3 deletions(-)
>>>>>>>>>
>>>>>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
>>>>>>>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
>>>>>>>>> index 6e10b42c57e5..3470929e5b8e 100644
>>>>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
>>>>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
>>>>>>>>> @@ -1245,10 +1245,15 @@ static int gmc_v9_0_sw_init(void *handle)
>>>>>>>>> /*
>>>>>>>>> * number of VMs
>>>>>>>>> * VMID 0 is reserved for System
>>>>>>>>> - * amdgpu graphics/compute will use VMIDs 1-7
>>>>>>>>> - * amdkfd will use VMIDs 8-15
>>>>>>>>> + * amdgpu graphics/compute will use VMIDs 1..n-1
>>>>>>>>> + * amdkfd will use VMIDs n..15
>>>>>>>>> + *
>>>>>>>>> + * The first KFD VMID is 8 for GPUs with graphics, 3 for
>>>>>>>>> + * compute-only GPUs. On compute-only GPUs that leaves 2
>>>>>>>>> VMIDs
>>>>>>>>> + * for video processing.
>>>>>>>>> */
>>>>>>>>> - adev->vm_manager.first_kfd_vmid = 8;
>>>>>>>>> + adev->vm_manager.first_kfd_vmid =
>>>>>>>>> + adev->asic_type == CHIP_ARCTURUS ? 3 : 8;
>>>>>>>>> amdgpu_vm_manager_init(adev);
>>
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
next prev parent reply other threads:[~2020-06-26 6:57 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-06-25 3:18 [PATCH 1/2] drm/amdgpu: Clean up KFD VMID assignment Felix Kuehling
2020-06-25 3:18 ` [PATCH 2/2] drm/amdgpu: Let KFD use more VMIDs on Arcturus Felix Kuehling
2020-06-25 8:19 ` Christian König
2020-06-25 15:15 ` Felix Kuehling
2020-06-25 15:38 ` Christian König
2020-06-25 15:43 ` Felix Kuehling
2020-06-25 15:50 ` Christian König
2020-06-25 15:58 ` Felix Kuehling
2020-06-25 16:01 ` Christian König
2020-06-25 20:39 ` Felix Kuehling
2020-06-26 6:57 ` Christian König [this message]
2020-06-25 8:18 ` [PATCH 1/2] drm/amdgpu: Clean up KFD VMID assignment Christian König
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=927d4503-536d-53e9-0a13-9a01b6ab2f0a@amd.com \
--to=christian.koenig@amd.com \
--cc=amd-gfx@lists.freedesktop.org \
--cc=felix.kuehling@amd.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).