Re: [PATCH 1/2] drm/amdgpu: track MQD size for gfx and compute

[Felix Kuehling <felix.kuehling@amd.com>]

MQDs are smaller than a page. The BO size will always be exactly be one 
page.

KFD can allocate MQDs with a suballocator. On some GPUs we allocate MQDs 
together with the queue's control stack in a single BO. And on some GPUs 
we allocate SDMA "MQDs" in bulk together with the HIQ MQD. So relying on 
the BO size would not work for us.

Regards,
   Felix


Am 2023-03-22 um 09:58 schrieb Christian König:
> Am 22.03.23 um 14:26 schrieb Alex Deucher:
>> On Wed, Mar 22, 2023 at 4:48 AM Christian König
>> <ckoenig.leichtzumerken@gmail.com> wrote:
>>> Am 21.03.23 um 20:39 schrieb Alex Deucher:
>>>> It varies by generation and we need to know the size
>>>> to expose this via debugfs.
>>> I suspect we can't just use the BO size for this?
>> We could, but it may be larger than the actual MQD.  Maybe that's not
>> a big deal?
>
> I don't really know either. Maybe just go ahead with this approach 
> here, but I usually try to avoid stuff like that because it can be an 
> additional source of errors when the allocation size is not correct.
>
> Christian.
>
>>
>> Alex
>>
>>
>>> If yes the series is Reviewed-by: Christian König 
>>> <christian.koenig@amd.com>
>>>
>>>> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
>>>> ---
>>>>    drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c  | 2 ++
>>>>    drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 1 +
>>>>    2 files changed, 3 insertions(+)
>>>>
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c 
>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
>>>> index c50d59855011..5435f41a3b7f 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
>>>> @@ -404,6 +404,7 @@ int amdgpu_gfx_mqd_sw_init(struct amdgpu_device 
>>>> *adev,
>>>>                                        return r;
>>>>                                }
>>>>
>>>> +                             ring->mqd_size = mqd_size;
>>>>                                /* prepare MQD backup */
>>>>                                adev->gfx.me.mqd_backup[i] = 
>>>> kmalloc(mqd_size, GFP_KERNEL);
>>>>                                if (!adev->gfx.me.mqd_backup[i])
>>>> @@ -424,6 +425,7 @@ int amdgpu_gfx_mqd_sw_init(struct amdgpu_device 
>>>> *adev,
>>>>                                return r;
>>>>                        }
>>>>
>>>> +                     ring->mqd_size = mqd_size;
>>>>                        /* prepare MQD backup */
>>>>                        adev->gfx.mec.mqd_backup[i] = 
>>>> kmalloc(mqd_size, GFP_KERNEL);
>>>>                        if (!adev->gfx.mec.mqd_backup[i])
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h 
>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
>>>> index 7942cb62e52c..deb9f7bead02 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
>>>> @@ -257,6 +257,7 @@ struct amdgpu_ring {
>>>>        struct amdgpu_bo        *mqd_obj;
>>>>        uint64_t                mqd_gpu_addr;
>>>>        void                    *mqd_ptr;
>>>> +     unsigned                mqd_size;
>>>>        uint64_t                eop_gpu_addr;
>>>>        u32                     doorbell_index;
>>>>        bool                    use_doorbell;
>