Re: [PATCH 2/2] drm/amdgpu: add shared fdinfo stats

From: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
To: "Christian König" <ckoenig.leichtzumerken@gmail.com>,
	"Daniel Vetter" <daniel@ffwll.ch>
Cc: Alex Deucher <alexander.deucher@amd.com>,
	dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org
Subject: Re: [PATCH 2/2] drm/amdgpu: add shared fdinfo stats
Date: Tue, 9 Jan 2024 15:09:18 +0000	[thread overview]
Message-ID: <d0f89a0e-4670-40fd-a4b8-a1fffc6da225@linux.intel.com> (raw)
In-Reply-To: <e9fdf33c-d93e-4120-ba9b-f494c504e047@gmail.com>

On 09/01/2024 14:57, Christian König wrote:
> Am 09.01.24 um 15:26 schrieb Daniel Vetter:
>> On Tue, 9 Jan 2024 at 14:25, Tvrtko Ursulin
>> <tvrtko.ursulin@linux.intel.com> wrote:
>>>
>>> On 09/01/2024 12:54, Daniel Vetter wrote:
>>>> On Tue, Jan 09, 2024 at 09:30:15AM +0000, Tvrtko Ursulin wrote:
>>>>> On 09/01/2024 07:56, Christian König wrote:
>>>>>> Am 07.12.23 um 19:02 schrieb Alex Deucher:
>>>>>>> Add shared stats.  Useful for seeing shared memory.
>>>>>>>
>>>>>>> v2: take dma-buf into account as well
>>>>>>>
>>>>>>> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
>>>>>>> Cc: Rob Clark <robdclark@gmail.com>
>>>>>>> ---
>>>>>>>     drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.c |  4 ++++
>>>>>>>     drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 11 +++++++++++
>>>>>>>     drivers/gpu/drm/amd/amdgpu/amdgpu_object.h |  6 ++++++
>>>>>>>     3 files changed, 21 insertions(+)
>>>>>>>
>>>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.c
>>>>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.c
>>>>>>> index 5706b282a0c7..c7df7fa3459f 100644
>>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.c
>>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.c
>>>>>>> @@ -97,6 +97,10 @@ void amdgpu_show_fdinfo(struct drm_printer *p,
>>>>>>> struct drm_file *file)
>>>>>>>                stats.requested_visible_vram/1024UL);
>>>>>>>         drm_printf(p, "amd-requested-gtt:\t%llu KiB\n",
>>>>>>>                stats.requested_gtt/1024UL);
>>>>>>> +    drm_printf(p, "drm-shared-vram:\t%llu KiB\n",
>>>>>>> stats.vram_shared/1024UL);
>>>>>>> +    drm_printf(p, "drm-shared-gtt:\t%llu KiB\n",
>>>>>>> stats.gtt_shared/1024UL);
>>>>>>> +    drm_printf(p, "drm-shared-cpu:\t%llu KiB\n",
>>>>>>> stats.cpu_shared/1024UL);
>>>>>>> +
>>>>>>>         for (hw_ip = 0; hw_ip < AMDGPU_HW_IP_NUM; ++hw_ip) {
>>>>>>>             if (!usage[hw_ip])
>>>>>>>                 continue;
>>>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>>>>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>>>>>>> index d79b4ca1ecfc..1b37d95475b8 100644
>>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>>>>>>> @@ -1287,25 +1287,36 @@ void amdgpu_bo_get_memory(struct 
>>>>>>> amdgpu_bo *bo,
>>>>>>>                   struct amdgpu_mem_stats *stats)
>>>>>>>     {
>>>>>>>         uint64_t size = amdgpu_bo_size(bo);
>>>>>>> +    struct drm_gem_object *obj;
>>>>>>>         unsigned int domain;
>>>>>>> +    bool shared;
>>>>>>>         /* Abort if the BO doesn't currently have a backing store */
>>>>>>>         if (!bo->tbo.resource)
>>>>>>>             return;
>>>>>>> +    obj = &bo->tbo.base;
>>>>>>> +    shared = (obj->handle_count > 1) || obj->dma_buf;
>>>>>> I still think that looking at handle_count is the completely wrong
>>>>>> approach, we should really only look at obj->dma_buf.
>>>>> Yeah it is all a bit tricky with the handle table walk. I don't 
>>>>> think it is
>>>>> even possible to claim it is shared with obj->dma_buf could be the 
>>>>> same
>>>>> process creating say via udmabuf and importing into drm. It is a wild
>>>>> scenario yes, but it could be private memory in that case. Not sure 
>>>>> where it
>>>>> would leave us if we said this is just a limitation of a BO based 
>>>>> tracking.
>>>>>
>>>>> Would adding a new category "imported" help?
>>>>>
>>>>> Hmm or we simply change drm-usage-stats.rst:
>>>>>
>>>>> """
>>>>> - drm-shared-<region>: <uint> [KiB|MiB]
>>>>>
>>>>> The total size of buffers that are shared with another file (ie. 
>>>>> have more
>>>>> than than a single handle).
>>>>> """
>>>>>
>>>>> Changing ie into eg coule be get our of jail free card to allow the
>>>>> "(obj->handle_count > 1) || obj->dma_buf;" condition?
>>>>>
>>>>> Because of the shared with another _file_ wording would cover my wild
>>>>> udmabuf self-import case. Unless there are more such creative 
>>>>> private import
>>>>> options.
>>>> Yeah I think clarifying that we can only track sharing with other fd 
>>>> and
>>>> have no idea whether this means sharing with another process or not is
>>>> probably simplest. Maybe not exactly what users want, but still the
>>>> roughly best-case approximation we can deliver somewhat cheaply.
>>>>
>>>> Also maybe time for a drm_gem_buffer_object_is_shared() helper so we 
>>>> don't
>>>> copypaste this all over and then end up in divergent conditions? I'm
>>>> guessing that there's going to be a bunch of drivers which needs this
>>>> little helper to add drm-shared-* stats to their fdinfo ...
>>> Yeah I agree that works and i915 would need to use the helper too.
>>>
>>> I would only suggest to name it so the meaning of shared is obviously
>>> only about the fdinfo memory stats and no one gets a more meaningful
>>> idea about its semantics.
>>>
>>> We have drm_show_memory_stats and drm_print_memory_stats exported so
>>> perhaps something like drm_object_is_shared_for_memory_stats,
>>> drm_object_is_memory_stats_shared, drm_memory_stats_object_is_shared?
> 
> + for drm_object_is_shared_for_memory_stats().

Hmmm although I probably meant to write 
drm_gem_object_is_shared_for_memory_stats. Since drm_mode_object seems 
to have claimed the drm_object_ function name space.

>>> And s/ie/eg/ in the above quoted drm-usage-stats.rst.
>> Ack on making it clear this helper would be for fdinfo memory stats
>> only. Sounds like a good idea to stop people from finding really
>> creative uses ...
> 
> It's astonishing how defensive the development process has become :)

A bit. :) But probably justified in this case.

Regards,

Tvrtko

> 
> Cheers,
> Christian.
> 
>> -Sima
>>
>>> Regards,
>>>
>>> Tvrtko
>>>
>>>> Cheers, Sima
>>>>> Regards,
>>>>>
>>>>> Tvrtko
>>>>>
>>>>>> Regards,
>>>>>> Christian.
>>>>>>
>>>>>>> +
>>>>>>>         domain = 
>>>>>>> amdgpu_mem_type_to_domain(bo->tbo.resource->mem_type);
>>>>>>>         switch (domain) {
>>>>>>>         case AMDGPU_GEM_DOMAIN_VRAM:
>>>>>>>             stats->vram += size;
>>>>>>>             if (amdgpu_bo_in_cpu_visible_vram(bo))
>>>>>>>                 stats->visible_vram += size;
>>>>>>> +        if (shared)
>>>>>>> +            stats->vram_shared += size;
>>>>>>>             break;
>>>>>>>         case AMDGPU_GEM_DOMAIN_GTT:
>>>>>>>             stats->gtt += size;
>>>>>>> +        if (shared)
>>>>>>> +            stats->gtt_shared += size;
>>>>>>>             break;
>>>>>>>         case AMDGPU_GEM_DOMAIN_CPU:
>>>>>>>         default:
>>>>>>>             stats->cpu += size;
>>>>>>> +        if (shared)
>>>>>>> +            stats->cpu_shared += size;
>>>>>>>             break;
>>>>>>>         }
>>>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>>>>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>>>>>>> index d28e21baef16..0503af75dc26 100644
>>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>>>>>>> @@ -138,12 +138,18 @@ struct amdgpu_bo_vm {
>>>>>>>     struct amdgpu_mem_stats {
>>>>>>>         /* current VRAM usage, includes visible VRAM */
>>>>>>>         uint64_t vram;
>>>>>>> +    /* current shared VRAM usage, includes visible VRAM */
>>>>>>> +    uint64_t vram_shared;
>>>>>>>         /* current visible VRAM usage */
>>>>>>>         uint64_t visible_vram;
>>>>>>>         /* current GTT usage */
>>>>>>>         uint64_t gtt;
>>>>>>> +    /* current shared GTT usage */
>>>>>>> +    uint64_t gtt_shared;
>>>>>>>         /* current system memory usage */
>>>>>>>         uint64_t cpu;
>>>>>>> +    /* current shared system memory usage */
>>>>>>> +    uint64_t cpu_shared;
>>>>>>>         /* sum of evicted buffers, includes visible VRAM */
>>>>>>>         uint64_t evicted_vram;
>>>>>>>         /* sum of evicted buffers due to CPU access */
>>
>>
> 