From: Dmitry Osipenko <dmitry.osipenko@collabora.com>
To: Boris Brezillon <boris.brezillon@collabora.com>
Cc: "David Airlie" <airlied@gmail.com>,
"Gerd Hoffmann" <kraxel@redhat.com>,
"Gurchetan Singh" <gurchetansingh@chromium.org>,
"Chia-I Wu" <olvaffe@gmail.com>,
"Daniel Vetter" <daniel@ffwll.ch>,
"Maarten Lankhorst" <maarten.lankhorst@linux.intel.com>,
"Maxime Ripard" <mripard@kernel.org>,
"Thomas Zimmermann" <tzimmermann@suse.de>,
"Christian König" <christian.koenig@amd.com>,
"Qiang Yu" <yuq825@gmail.com>,
"Steven Price" <steven.price@arm.com>,
"Emma Anholt" <emma@anholt.net>, "Melissa Wen" <mwen@igalia.com>,
dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org,
kernel@collabora.com, virtualization@lists.linux-foundation.org
Subject: Re: [PATCH v19 22/30] drm/shmem-helper: Add common memory shrinker
Date: Mon, 29 Jan 2024 09:16:04 +0300 [thread overview]
Message-ID: <ea74a26d-d97f-4d09-bc24-67484ee45b76@collabora.com> (raw)
In-Reply-To: <20240126191230.0ee6f99f@collabora.com>
On 1/26/24 21:12, Boris Brezillon wrote:
> On Fri, 26 Jan 2024 19:27:49 +0300
> Dmitry Osipenko <dmitry.osipenko@collabora.com> wrote:
>
>> On 1/26/24 12:55, Boris Brezillon wrote:
>>> On Fri, 26 Jan 2024 00:56:47 +0300
>>> Dmitry Osipenko <dmitry.osipenko@collabora.com> wrote:
>>>
>>>> On 1/25/24 13:19, Boris Brezillon wrote:
>>>>> On Fri, 5 Jan 2024 21:46:16 +0300
>>>>> Dmitry Osipenko <dmitry.osipenko@collabora.com> wrote:
>>>>>
>>>>>> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
>>>>>> +{
>>>>>> + return (shmem->madv >= 0) && shmem->base.funcs->evict &&
>>>>>> + refcount_read(&shmem->pages_use_count) &&
>>>>>> + !refcount_read(&shmem->pages_pin_count) &&
>>>>>> + !shmem->base.dma_buf && !shmem->base.import_attach &&
>>>>>> + !shmem->evicted;
>>>>>
>>>>> Are we missing
>>>>>
>>>>> && dma_resv_test_signaled(shmem->base.resv,
>>>>> DMA_RESV_USAGE_BOOKKEEP)
>>>>>
>>>>> to make sure the GPU is done using the BO?
>>>>> The same applies to drm_gem_shmem_is_purgeable() BTW.
>>>>>
>>>>> If you don't want to do this test here, we need a way to let drivers
>>>>> provide a custom is_{evictable,purgeable}() test.
>>>>>
>>>>> I guess we should also expose drm_gem_shmem_shrinker_update_lru_locked()
>>>>> to let drivers move the GEMs that were used most recently (those
>>>>> referenced by a GPU job) at the end of the evictable LRU.
>>>>
>>>> We have the signaled-check in the common drm_gem_evict() helper:
>>>>
>>>> https://elixir.bootlin.com/linux/v6.8-rc1/source/drivers/gpu/drm/drm_gem.c#L1496
>>>
>>> Ah, indeed. I'll need DMA_RESV_USAGE_BOOKKEEP instead of
>>> DMA_RESV_USAGE_READ in panthor, but I can add it in the driver specific
>>> ->evict() hook (though that means calling dma_resv_test_signaled()
>>> twice, which is not great, oh well).
>>
>> Maybe we should change drm_gem_evict() to use BOOKKEEP. The
>> test_signaled(BOOKKEEP) should be a "stronger" check than
>> test_signaled(READ)?
>
> It is, just wondering if some users have a good reason to want
> READ here.
>
>>
>>> The problem about the evictable LRU remains though: we need a way to let
>>> drivers put their BOs at the end of the list when the BO has been used
>>> by the GPU, don't we?
>>
>> If BO is use, then it won't be evicted, while idling BOs will be
>> evicted. Hence, the used BOs will be naturally moved down the LRU list
>> each time shrinker is invoked.
>>
>
> That only do the trick if the BOs being used most often are busy when
> the shrinker kicks in though. Let's take this scenario:
>
>
> BO 1 BO 2 shinker
>
> busy
> idle (first-pos-in-evictable-LRU)
>
> busy
> idle (second-pos-in-evictable-LRU)
>
> busy
> idle
>
> busy
> idle
>
> busy
> idle
>
> find a BO to evict
> pick BO 2
>
> busy (swapin)
> idle
>
> If the LRU had been updated at each busy event, BO 1 should have
> been picked for eviction. But we evicted the BO that was first
> recorded idle instead of the one that was least recently
> recorded busy.
You have to swapin(BO) every time BO goes to busy state, and swapin does drm_gem_lru_move_tail(BO). Hence, each time BO goes idle->busy, it's moved down the LRU list.
For example, please see patch #29 where virtio-gpu invokes swapin for each job's BO in the submit()->virtio_gpu_array_prepare() code path.
--
Best regards,
Dmitry
next prev parent reply other threads:[~2024-01-29 6:16 UTC|newest]
Thread overview: 70+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-01-05 18:45 [PATCH v19 00/30] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers Dmitry Osipenko
2024-01-05 18:45 ` [PATCH v19 01/30] drm/gem: Change locked/unlocked postfix of drm_gem_v/unmap() function names Dmitry Osipenko
2024-01-05 18:45 ` [PATCH v19 02/30] drm/gem: Add _locked postfix to functions that have unlocked counterpart Dmitry Osipenko
2024-01-05 18:45 ` [PATCH v19 03/30] drm/gem: Document locking rule of vmap and evict callbacks Dmitry Osipenko
2024-01-25 7:55 ` Boris Brezillon
2024-01-05 18:45 ` [PATCH v19 04/30] drm/shmem-helper: Make all exported symbols GPL Dmitry Osipenko
2024-01-05 18:45 ` [PATCH v19 05/30] drm/shmem-helper: Refactor locked/unlocked functions Dmitry Osipenko
2024-01-05 18:46 ` [PATCH v19 06/30] drm/shmem-helper: Remove obsoleted is_iomem test Dmitry Osipenko
2024-01-05 18:46 ` [PATCH v19 07/30] drm/shmem-helper: Add and use pages_pin_count Dmitry Osipenko
2024-01-05 18:46 ` [PATCH v19 08/30] drm/shmem-helper: Use refcount_t for pages_use_count Dmitry Osipenko
2024-01-05 18:46 ` [PATCH v19 09/30] drm/shmem-helper: Add and use lockless drm_gem_shmem_get_pages() Dmitry Osipenko
2024-01-25 17:24 ` Daniel Vetter
2024-01-25 21:52 ` Dmitry Osipenko
2024-01-26 10:18 ` Boris Brezillon
2024-01-26 16:43 ` Dmitry Osipenko
2024-01-30 8:34 ` Daniel Vetter
2024-01-30 10:05 ` Dmitry Osipenko
2024-01-30 10:10 ` Boris Brezillon
2024-02-01 18:53 ` Dmitry Osipenko
2024-01-05 18:46 ` [PATCH v19 10/30] drm/shmem-helper: Switch drm_gem_shmem_vmap/vunmap to use pin/unpin Dmitry Osipenko
2024-01-05 18:46 ` [PATCH v19 11/30] drm/shmem-helper: Use refcount_t for vmap_use_count Dmitry Osipenko
2024-01-05 18:46 ` [PATCH v19 12/30] drm/shmem-helper: Prepare drm_gem_shmem_free() to shrinker addition Dmitry Osipenko
2024-01-25 8:01 ` Boris Brezillon
2024-01-05 18:46 ` [PATCH v19 13/30] drm/shmem-helper: Make drm_gem_shmem_get_pages() public Dmitry Osipenko
2024-01-25 8:02 ` Boris Brezillon
2024-01-05 18:46 ` [PATCH v19 14/30] drm/shmem-helper: Add drm_gem_shmem_put_pages() Dmitry Osipenko
2024-01-25 8:02 ` Boris Brezillon
2024-01-05 18:46 ` [PATCH v19 15/30] drm/shmem-helper: Avoid lockdep warning when pages are released Dmitry Osipenko
2024-01-25 8:02 ` Boris Brezillon
2024-01-05 18:46 ` [PATCH v19 16/30] drm/lima: Explicitly get and put drm-shmem pages Dmitry Osipenko
2024-01-05 18:46 ` [PATCH v19 17/30] drm/panfrost: Fix the error path in panfrost_mmu_map_fault_addr() Dmitry Osipenko
2024-01-25 16:47 ` Steven Price
2024-01-25 21:41 ` Dmitry Osipenko
2024-01-25 21:44 ` Dmitry Osipenko
2024-01-26 9:08 ` AngeloGioacchino Del Regno
2024-04-04 15:23 ` Dmitry Osipenko
2024-01-05 18:46 ` [PATCH v19 18/30] drm/panfrost: Explicitly get and put drm-shmem pages Dmitry Osipenko
2024-01-25 8:10 ` Boris Brezillon
2024-01-25 16:47 ` Steven Price
2024-01-26 9:39 ` Boris Brezillon
2024-01-26 11:26 ` Steven Price
2024-01-05 18:46 ` [PATCH v19 19/30] drm/virtio: " Dmitry Osipenko
2024-01-05 18:46 ` [PATCH v19 20/30] drm/v3d: " Dmitry Osipenko
2024-01-05 18:46 ` [PATCH v19 21/30] drm/shmem-helper: Change sgt allocation policy Dmitry Osipenko
2024-01-05 18:46 ` [PATCH v19 22/30] drm/shmem-helper: Add common memory shrinker Dmitry Osipenko
2024-01-25 9:07 ` Boris Brezillon
2024-01-30 8:39 ` Daniel Vetter
2024-01-30 10:04 ` Dmitry Osipenko
2024-01-25 10:19 ` Boris Brezillon
2024-01-25 21:56 ` Dmitry Osipenko
2024-01-26 9:55 ` Boris Brezillon
2024-01-26 16:27 ` Dmitry Osipenko
2024-01-26 18:12 ` Boris Brezillon
2024-01-29 6:16 ` Dmitry Osipenko [this message]
2024-01-29 8:55 ` Boris Brezillon
2024-01-29 9:06 ` Boris Brezillon
2024-01-29 9:01 ` Boris Brezillon
2024-01-29 9:25 ` Dmitry Osipenko
2024-01-05 18:46 ` [PATCH v19 23/30] drm/shmem-helper: Export drm_gem_shmem_get_pages_sgt_locked() Dmitry Osipenko
2024-01-25 9:09 ` Boris Brezillon
2024-01-05 18:46 ` [PATCH v19 24/30] drm/shmem-helper: Optimize unlocked get_pages_sgt() Dmitry Osipenko
2024-01-25 9:28 ` Boris Brezillon
2024-01-05 18:46 ` [PATCH v19 25/30] drm/shmem-helper: Don't free refcounted GEM Dmitry Osipenko
2024-01-05 18:46 ` [PATCH v19 26/30] drm/shmem-helper: Turn warnings about imported GEM into errors Dmitry Osipenko
2024-01-05 18:46 ` [PATCH v19 27/30] drm/virtio: Pin display framebuffer BO Dmitry Osipenko
2024-01-05 18:46 ` [PATCH v19 28/30] drm/virtio: Attach shmem BOs dynamically Dmitry Osipenko
2024-01-05 18:46 ` [PATCH v19 29/30] drm/virtio: Support shmem shrinking Dmitry Osipenko
2024-01-05 18:46 ` [PATCH v19 30/30] drm/panfrost: Switch to generic memory shrinker Dmitry Osipenko
2024-01-25 9:49 ` Boris Brezillon
2024-01-25 12:04 ` Dmitry Osipenko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ea74a26d-d97f-4d09-bc24-67484ee45b76@collabora.com \
--to=dmitry.osipenko@collabora.com \
--cc=airlied@gmail.com \
--cc=boris.brezillon@collabora.com \
--cc=christian.koenig@amd.com \
--cc=daniel@ffwll.ch \
--cc=dri-devel@lists.freedesktop.org \
--cc=emma@anholt.net \
--cc=gurchetansingh@chromium.org \
--cc=kernel@collabora.com \
--cc=kraxel@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=maarten.lankhorst@linux.intel.com \
--cc=mripard@kernel.org \
--cc=mwen@igalia.com \
--cc=olvaffe@gmail.com \
--cc=steven.price@arm.com \
--cc=tzimmermann@suse.de \
--cc=virtualization@lists.linux-foundation.org \
--cc=yuq825@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).