From: Dmitry Osipenko <dmitry.osipenko@collabora.com>
To: Thomas Zimmermann <tzimmermann@suse.de>
Cc: "David Airlie" <airlied@linux.ie>,
dri-devel@lists.freedesktop.org,
virtualization@lists.linux-foundation.org,
"Thierry Reding" <thierry.reding@gmail.com>,
"Gerd Hoffmann" <kraxel@redhat.com>,
"Dmitry Osipenko" <digetx@gmail.com>,
kernel@collabora.com, "Sumit Semwal" <sumit.semwal@linaro.org>,
"Marek Szyprowski" <m.szyprowski@samsung.com>,
"Mauro Carvalho Chehab" <mchehab@kernel.org>,
"Steven Price" <steven.price@arm.com>,
"Gustavo Padovan" <gustavo.padovan@collabora.com>,
"Alyssa Rosenzweig" <alyssa.rosenzweig@collabora.com>,
linux-media@vger.kernel.org, intel-gfx@lists.freedesktop.org,
linaro-mm-sig@lists.linaro.org,
"Rodrigo Vivi" <rodrigo.vivi@intel.com>,
linux-tegra@vger.kernel.org,
"Gurchetan Singh" <gurchetansingh@chromium.org>,
"Tvrtko Ursulin" <tvrtko.ursulin@linux.intel.com>,
"Daniel Almeida" <daniel.almeida@collabora.com>,
amd-gfx@lists.freedesktop.org,
"Tomeu Vizoso" <tomeu.vizoso@collabora.com>,
"Gert Wollny" <gert.wollny@collabora.com>,
"Pan, Xinhui" <Xinhui.Pan@amd.com>,
"Emil Velikov" <emil.l.velikov@gmail.com>,
linux-kernel@vger.kernel.org, "Tomasz Figa" <tfiga@chromium.org>,
"Qiang Yu" <yuq825@gmail.com>,
"Alex Deucher" <alexander.deucher@amd.com>,
"Robin Murphy" <robin.murphy@arm.com>,
"Christian König" <christian.koenig@amd.com>
Subject: Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers
Date: Fri, 10 Jun 2022 17:40:18 +0300 [thread overview]
Message-ID: <3cbc42df-fff1-5325-6c6f-17dfc6899283@collabora.com> (raw)
In-Reply-To: <20220526235040.678984-1-dmitry.osipenko@collabora.com>
On 5/27/22 02:50, Dmitry Osipenko wrote:
> Hello,
>
> This patchset introduces memory shrinker for the VirtIO-GPU DRM driver
> and adds memory purging and eviction support to VirtIO-GPU driver.
>
> The new dma-buf locking convention is introduced here as well.
>
> During OOM, the shrinker will release BOs that are marked as "not needed"
> by userspace using the new madvise IOCTL, it will also evict idling BOs
> to SWAP. The userspace in this case is the Mesa VirGL driver, it will mark
> the cached BOs as "not needed", allowing kernel driver to release memory
> of the cached shmem BOs on lowmem situations, preventing OOM kills.
>
> The Panfrost driver is switched to use generic memory shrinker.
>
> This patchset includes improvements and fixes for various things that
> I found while was working on the shrinker.
>
> The Mesa and IGT patches will be kept on hold until this kernel series
> will be approved and merged.
>
> This patchset was tested using Qemu and crosvm, including both cases of
> IOMMU off/on.
>
> Mesa: https://gitlab.freedesktop.org/digetx/mesa/-/commits/virgl-madvise
> IGT: https://gitlab.freedesktop.org/digetx/igt-gpu-tools/-/commits/virtio-madvise
> https://gitlab.freedesktop.org/digetx/igt-gpu-tools/-/commits/panfrost-madvise
>
> Changelog:
>
> v6: - Added new VirtIO-related fix patch that previously was sent separately
> and didn't get much attention:
>
> drm/gem: Properly annotate WW context on drm_gem_lock_reservations() error
>
> - Added new patch that fixes mapping of imported dma-bufs for
> Tegra DRM and other affected drivers. It's also handy to have it
> for switching to the new dma-buf locking convention scheme:
>
> drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()
>
> - Added new patch that fixes shrinker list corruption for stable Panfrost
> driver:
>
> drm/panfrost: Fix shrinker list corruption by madvise IOCTL
>
> - Added new minor patch-fix for drm-shmem:
>
> drm/shmem-helper: Add missing vunmap on error
>
> - Added fixes tag to the "Put mapping ..." patch like was suggested by
> Steven Price.
>
> - Added new VirtIO-GPU driver improvement patch:
>
> drm/virtio: Return proper error codes instead of -1
>
> - Reworked shrinker patches like was suggested by Daniel Vetter:
>
> - Introduced the new locking convention for dma-bufs. Tested on
> VirtIO-GPU, Panfrost, Lima, Tegra and Intel selftests.
>
> - Dropped separate purge() callback. Now single evict() does
> everything.
>
> - Dropped swap_in() callback from drm-shmem objects. DRM drivers
> now could and should restore only the required mappings.
>
> - Dropped dynamic counting of evictable pages. This simplifies
> code in exchange to *potentially* burning more CPU time on OOM.
>
> v5: - Added new for-stable patch "drm/panfrost: Put mapping instead of
> shmem obj on panfrost_mmu_map_fault_addr() error" that corrects GEM's
> refcounting in case of error.
>
> - The drm_gem_shmem_v[un]map() now takes a separate vmap_lock for
> imported GEMs to avoid recursive locking of DMA reservations.
> This addresses v4 comment from Thomas Zimmermann about the potential
> deadlocking of vmapping.
>
> - Added ack from Thomas Zimmermann to "drm/shmem-helper: Correct
> doc-comment of drm_gem_shmem_get_sg_table()" patch.
>
> - Dropped explicit shmem states from the generic shrinker patch as
> was requested by Thomas Zimmermann.
>
> - Improved variable names and comments of the generic shrinker code.
>
> - Extended drm_gem_shmem_print_info() with the shrinker-state info in
> the "drm/virtio: Support memory shrinking" patch.
>
> - Moved evict()/swap_in()/purge() callbacks from drm_gem_object_funcs
> to drm_gem_shmem_object in the generic shrinker patch, for more
> consistency.
>
> - Corrected bisectability of the patches that was broken in v4
> by accident.
>
> - The virtio_gpu_plane_prepare_fb() now uses drm_gem_shmem_pin() instead
> of drm_gem_shmem_set_unpurgeable_and_unevictable() and does it only for
> shmem BOs in the "drm/virtio: Support memory shrinking" patch.
>
> - Made more functions private to drm_gem_shmem_helper.c as was requested
> by Thomas Zimmermann. This minimizes number of the public shmem helpers.
>
> v4: - Corrected minor W=1 warnings reported by kernel test robot for v3.
>
> - Renamed DRM_GEM_SHMEM_PAGES_STATE_ACTIVE/INACTIVE to PINNED/UNPINNED,
> for more clarity.
>
> v3: - Hardened shrinker's count() with usage of READ_ONCE() since we don't
> use atomic type for counting and technically compiler is free to
> re-fetch counter's variable.
>
> - "Correct drm_gem_shmem_get_sg_table() error handling" now uses
> PTR_ERR_OR_ZERO(), fixing typo that was made in v2.
>
> - Removed obsoleted shrinker from the Panfrost driver, which I missed to
> do in v2 by accident and Alyssa Rosenzweig managed to notice it.
>
> - CCed stable kernels in all patches that make fixes, even the minor ones,
> like was suggested by Emil Velikov and added his r-b to the patches.
>
> - Added t-b from Steven Price to the Panfrost's shrinker patch.
>
> - Corrected doc-comment of drm_gem_shmem_object.madv, like was suggested
> by Steven Price. Comment now says that madv=1 means "object is purged"
> instead of saying that value is unused.
>
> - Added more doc-comments to the new shmem shrinker API.
>
> - The "Improve DMA API usage for shmem BOs" patch got more improvements
> by removing the obsoleted drm_dev_set_unique() quirk and its comment.
>
> - Added patch that makes Virtio-GPU driver to use common dev_is_pci()
> helper, which was suggested by Robin Murphy.
>
> - Added new "drm/shmem-helper: Take GEM reservation lock instead of
> drm_gem_shmem locks" patch, which was suggested by Daniel Vetter.
>
> - Added new "drm/virtio: Simplify error handling of
> virtio_gpu_object_create()" patch.
>
> - Improved "Correct doc-comment of drm_gem_shmem_get_sg_table()" patch,
> like was suggested by Daniel Vetter, by saying that function returns
> ERR_PTR() and not errno.
>
> - virtio_gpu_purge_object() is fenced properly now, turned out
> virtio_gpu_notify() doesn't do fencing as I was supposing before.
> Stress testing of memory eviction revealed that.
>
> - Added new patch that corrects virtio_gpu_plane_cleanup_fb() to use
> appropriate atomic plane state.
>
> - SHMEM shrinker got eviction support.
>
> - VirtIO-GPU driver now supports memory eviction. It's enabled for a
> non-blob GEMs only, i.e. for VirGL. The blobs don't support dynamic
> attaching/detaching of guest's memory, so it's not trivial to enable
> them.
>
> - Added patch that removes obsoleted drm_gem_shmem_purge()
>
> - Added patch that makes drm_gem_shmem_get_pages() private.
>
> - Added patch that fixes lockup on dma_resv_reserve_fences() error.
>
> v2: - Improved shrinker by using a more fine-grained locking to reduce
> contention during scan of objects and dropped locking from the
> 'counting' callback by tracking count of shrinkable pages. This
> was suggested by Rob Clark in the comment to v1.
>
> - Factored out common shrinker code into drm_gem_shmem_helper.c
> and switched Panfrost driver to use the new common memory shrinker.
> This was proposed by Thomas Zimmermann in his prototype series that
> he shared with us in the comment to v1. Note that I only compile-tested
> the Panfrost driver.
>
> - Shrinker now takes object_name_lock during scan to prevent racing
> with dma-buf exporting.
>
> - Shrinker now takes vmap_lock during scan to prevent racing with shmem
> vmap/unmap code.
>
> - Added "Correct doc-comment of drm_gem_shmem_get_sg_table()" patch,
> which I sent out previously as a standalone change, since the
> drm_gem_shmem_helper.c is now touched by this patchset anyways and
> it doesn't hurt to group all the patches together.
>
> Dmitry Osipenko (22):
> drm/gem: Properly annotate WW context on drm_gem_lock_reservations()
> error
> drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()
> drm/panfrost: Put mapping instead of shmem obj on
> panfrost_mmu_map_fault_addr() error
> drm/panfrost: Fix shrinker list corruption by madvise IOCTL
> drm/virtio: Correct drm_gem_shmem_get_sg_table() error handling
> drm/virtio: Check whether transferred 2D BO is shmem
> drm/virtio: Unlock reservations on virtio_gpu_object_shmem_init()
> error
> drm/virtio: Unlock reservations on dma_resv_reserve_fences() error
> drm/virtio: Use appropriate atomic state in
> virtio_gpu_plane_cleanup_fb()
> drm/shmem-helper: Add missing vunmap on error
> drm/shmem-helper: Correct doc-comment of drm_gem_shmem_get_sg_table()
...
Thomas, do you think it will be possible for you to take the fix patches
1-11 into the drm-fixes or would you prefer me to re-send them separately?
The VirtIO patches 12-13 also are good to go into drm-next, IMO.
I'm going to factor out the new dma-buf convention into a separate
patchset, like was suggested by Christian. But it will take me some time
to get the dma-buf patches ready and I also will be on a vacation soon.
At minimum nothing should hold the fixes, so will be great if they could
land sooner.
Thank you!
--
Best regards,
Dmitry
next prev parent reply other threads:[~2022-06-10 14:40 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-05-26 23:50 [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 01/22] drm/gem: Properly annotate WW context on drm_gem_lock_reservations() error Dmitry Osipenko
2022-06-28 20:12 ` Thomas Hellström (Intel)
2022-06-29 8:23 ` Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj() Dmitry Osipenko
2022-06-29 6:40 ` Thomas Hellström (Intel)
2022-06-29 8:22 ` Dmitry Osipenko
2022-06-29 8:43 ` Thomas Hellström (Intel)
2022-06-29 23:06 ` Dmitry Osipenko
2022-07-04 12:33 ` [Linaro-mm-sig] " Christian König
2022-07-04 22:44 ` Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 03/22] drm/panfrost: Put mapping instead of shmem obj on panfrost_mmu_map_fault_addr() error Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 04/22] drm/panfrost: Fix shrinker list corruption by madvise IOCTL Dmitry Osipenko
2022-05-27 15:56 ` Alyssa Rosenzweig
2022-05-30 9:41 ` Steven Price
2022-05-26 23:50 ` [PATCH v6 05/22] drm/virtio: Correct drm_gem_shmem_get_sg_table() error handling Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 06/22] drm/virtio: Check whether transferred 2D BO is shmem Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 07/22] drm/virtio: Unlock reservations on virtio_gpu_object_shmem_init() error Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 08/22] drm/virtio: Unlock reservations on dma_resv_reserve_fences() error Dmitry Osipenko
2022-06-29 6:43 ` Thomas Hellström (Intel)
2022-05-26 23:50 ` [PATCH v6 09/22] drm/virtio: Use appropriate atomic state in virtio_gpu_plane_cleanup_fb() Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 10/22] drm/shmem-helper: Add missing vunmap on error Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 11/22] drm/shmem-helper: Correct doc-comment of drm_gem_shmem_get_sg_table() Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 12/22] drm/virtio: Simplify error handling of virtio_gpu_object_create() Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 13/22] drm/virtio: Improve DMA API usage for shmem BOs Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 14/22] dma-buf: Introduce new locking convention Dmitry Osipenko
2022-05-30 6:50 ` Christian König
2022-05-30 13:26 ` Dmitry Osipenko
2022-05-30 13:41 ` Christian König
2022-05-30 13:57 ` Dmitry Osipenko
2022-06-28 21:26 ` Thomas Hellström (Intel)
2022-07-01 10:43 ` Dmitry Osipenko
2022-07-04 22:38 ` Dmitry Osipenko
2022-07-05 10:52 ` Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 15/22] drm/shmem-helper: Don't use vmap_use_count for dma-bufs Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 16/22] drm/shmem-helper: Use reservation lock Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker Dmitry Osipenko
2022-06-05 16:47 ` Daniel Vetter
2022-06-05 18:32 ` Rob Clark
2022-06-05 18:45 ` Daniel Vetter
2022-06-06 10:57 ` Christian König
2022-06-07 15:33 ` Dmitry Osipenko
2022-06-19 17:53 ` Rob Clark
2022-06-20 14:08 ` Dmitry Osipenko
2022-06-20 15:18 ` Rob Clark
2022-06-24 20:23 ` Daniel Vetter
2022-06-24 20:21 ` Daniel Vetter
2022-06-20 15:37 ` Rob Clark
2022-06-21 16:39 ` Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 18/22] drm/gem: Add drm_gem_pin_unlocked() Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 19/22] drm/virtio: Support memory shrinking Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 20/22] drm/virtio: Use dev_is_pci() Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 21/22] drm/virtio: Return proper error codes instead of -1 Dmitry Osipenko
2022-05-26 23:50 ` [PATCH v6 22/22] drm/panfrost: Switch to generic memory shrinker Dmitry Osipenko
2022-05-27 11:48 ` Alyssa Rosenzweig
2022-06-10 14:40 ` Dmitry Osipenko [this message]
2022-06-28 12:31 ` [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers Robin Murphy
2022-06-28 12:50 ` Dmitry Osipenko
2022-06-28 16:48 ` Rob Clark
2022-06-28 23:11 ` Dmitry Osipenko
2022-06-28 12:51 ` Dmitry Osipenko
2022-06-28 13:11 ` Dmitry Osipenko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3cbc42df-fff1-5325-6c6f-17dfc6899283@collabora.com \
--to=dmitry.osipenko@collabora.com \
--cc=Xinhui.Pan@amd.com \
--cc=airlied@linux.ie \
--cc=alexander.deucher@amd.com \
--cc=alyssa.rosenzweig@collabora.com \
--cc=amd-gfx@lists.freedesktop.org \
--cc=christian.koenig@amd.com \
--cc=daniel.almeida@collabora.com \
--cc=digetx@gmail.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=emil.l.velikov@gmail.com \
--cc=gert.wollny@collabora.com \
--cc=gurchetansingh@chromium.org \
--cc=gustavo.padovan@collabora.com \
--cc=intel-gfx@lists.freedesktop.org \
--cc=kernel@collabora.com \
--cc=kraxel@redhat.com \
--cc=linaro-mm-sig@lists.linaro.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=linux-tegra@vger.kernel.org \
--cc=m.szyprowski@samsung.com \
--cc=mchehab@kernel.org \
--cc=robin.murphy@arm.com \
--cc=rodrigo.vivi@intel.com \
--cc=steven.price@arm.com \
--cc=sumit.semwal@linaro.org \
--cc=tfiga@chromium.org \
--cc=thierry.reding@gmail.com \
--cc=tomeu.vizoso@collabora.com \
--cc=tvrtko.ursulin@linux.intel.com \
--cc=tzimmermann@suse.de \
--cc=virtualization@lists.linux-foundation.org \
--cc=yuq825@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).