[PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Dmitry Osipenko]

Hello,

This patchset introduces memory shrinker for the VirtIO-GPU DRM driver
and adds memory purging and eviction support to VirtIO-GPU driver.

The new dma-buf locking convention is introduced here as well.

During OOM, the shrinker will release BOs that are marked as "not needed"
by userspace using the new madvise IOCTL, it will also evict idling BOs
to SWAP. The userspace in this case is the Mesa VirGL driver, it will mark
the cached BOs as "not needed", allowing kernel driver to release memory
of the cached shmem BOs on lowmem situations, preventing OOM kills.

The Panfrost driver is switched to use generic memory shrinker.

This patchset includes improvements and fixes for various things that
I found while was working on the shrinker.

The Mesa and IGT patches will be kept on hold until this kernel series
will be approved and merged.

This patchset was tested using Qemu and crosvm, including both cases of
IOMMU off/on.

Mesa: https://gitlab.freedesktop.org/digetx/mesa/-/commits/virgl-madvise
IGT:  https://gitlab.freedesktop.org/digetx/igt-gpu-tools/-/commits/virtio-madvise
      https://gitlab.freedesktop.org/digetx/igt-gpu-tools/-/commits/panfrost-madvise

Changelog:

v6: - Added new VirtIO-related fix patch that previously was sent separately
      and didn't get much attention:

        drm/gem: Properly annotate WW context on drm_gem_lock_reservations() error

    - Added new patch that fixes mapping of imported dma-bufs for
      Tegra DRM and other affected drivers. It's also handy to have it
      for switching to the new dma-buf locking convention scheme:

        drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

    - Added new patch that fixes shrinker list corruption for stable Panfrost
      driver:

        drm/panfrost: Fix shrinker list corruption by madvise IOCTL

    - Added new minor patch-fix for drm-shmem:

        drm/shmem-helper: Add missing vunmap on error

    - Added fixes tag to the "Put mapping ..." patch like was suggested by
      Steven Price.

    - Added new VirtIO-GPU driver improvement patch:

        drm/virtio: Return proper error codes instead of -1

    - Reworked shrinker patches like was suggested by Daniel Vetter:

        - Introduced the new locking convention for dma-bufs. Tested on
          VirtIO-GPU, Panfrost, Lima, Tegra and Intel selftests.

        - Dropped separate purge() callback. Now single evict() does
          everything.

        - Dropped swap_in() callback from drm-shmem objects. DRM drivers
          now could and should restore only the required mappings.

        - Dropped dynamic counting of evictable pages. This simplifies
          code in exchange to *potentially* burning more CPU time on OOM.

v5: - Added new for-stable patch "drm/panfrost: Put mapping instead of
      shmem obj on panfrost_mmu_map_fault_addr() error" that corrects GEM's
      refcounting in case of error.

    - The drm_gem_shmem_v[un]map() now takes a separate vmap_lock for
      imported GEMs to avoid recursive locking of DMA reservations.
      This addresses v4 comment from Thomas Zimmermann about the potential
      deadlocking of vmapping.

    - Added ack from Thomas Zimmermann to "drm/shmem-helper: Correct
      doc-comment of drm_gem_shmem_get_sg_table()" patch.

    - Dropped explicit shmem states from the generic shrinker patch as
      was requested by Thomas Zimmermann.

    - Improved variable names and comments of the generic shrinker code.

    - Extended drm_gem_shmem_print_info() with the shrinker-state info in
      the "drm/virtio: Support memory shrinking" patch.

    - Moved evict()/swap_in()/purge() callbacks from drm_gem_object_funcs
      to drm_gem_shmem_object in the generic shrinker patch, for more
      consistency.

    - Corrected bisectability of the patches that was broken in v4
      by accident.

    - The virtio_gpu_plane_prepare_fb() now uses drm_gem_shmem_pin() instead
      of drm_gem_shmem_set_unpurgeable_and_unevictable() and does it only for
      shmem BOs in the "drm/virtio: Support memory shrinking" patch.

    - Made more functions private to drm_gem_shmem_helper.c as was requested
      by Thomas Zimmermann. This minimizes number of the public shmem helpers.

v4: - Corrected minor W=1 warnings reported by kernel test robot for v3.

    - Renamed DRM_GEM_SHMEM_PAGES_STATE_ACTIVE/INACTIVE to PINNED/UNPINNED,
      for more clarity.

v3: - Hardened shrinker's count() with usage of READ_ONCE() since we don't
      use atomic type for counting and technically compiler is free to
      re-fetch counter's variable.

    - "Correct drm_gem_shmem_get_sg_table() error handling" now uses
      PTR_ERR_OR_ZERO(), fixing typo that was made in v2.

    - Removed obsoleted shrinker from the Panfrost driver, which I missed to
      do in v2 by accident and Alyssa Rosenzweig managed to notice it.

    - CCed stable kernels in all patches that make fixes, even the minor ones,
      like was suggested by Emil Velikov and added his r-b to the patches.

    - Added t-b from Steven Price to the Panfrost's shrinker patch.

    - Corrected doc-comment of drm_gem_shmem_object.madv, like was suggested
      by Steven Price. Comment now says that madv=1 means "object is purged"
      instead of saying that value is unused.

    - Added more doc-comments to the new shmem shrinker API.

    - The "Improve DMA API usage for shmem BOs" patch got more improvements
      by removing the obsoleted drm_dev_set_unique() quirk and its comment.

    - Added patch that makes Virtio-GPU driver to use common dev_is_pci()
      helper, which was suggested by Robin Murphy.

    - Added new "drm/shmem-helper: Take GEM reservation lock instead of
      drm_gem_shmem locks" patch, which was suggested by Daniel Vetter.

    - Added new "drm/virtio: Simplify error handling of
      virtio_gpu_object_create()" patch.

    - Improved "Correct doc-comment of drm_gem_shmem_get_sg_table()" patch,
      like was suggested by Daniel Vetter, by saying that function returns
      ERR_PTR() and not errno.

    - virtio_gpu_purge_object() is fenced properly now, turned out
      virtio_gpu_notify() doesn't do fencing as I was supposing before.
      Stress testing of memory eviction revealed that.

    - Added new patch that corrects virtio_gpu_plane_cleanup_fb() to use
      appropriate atomic plane state.

    - SHMEM shrinker got eviction support.

    - VirtIO-GPU driver now supports memory eviction. It's enabled for a
      non-blob GEMs only, i.e. for VirGL. The blobs don't support dynamic
      attaching/detaching of guest's memory, so it's not trivial to enable
      them.

    - Added patch that removes obsoleted drm_gem_shmem_purge()

    - Added patch that makes drm_gem_shmem_get_pages() private.

    - Added patch that fixes lockup on dma_resv_reserve_fences() error.

v2: - Improved shrinker by using a more fine-grained locking to reduce
      contention during scan of objects and dropped locking from the
      'counting' callback by tracking count of shrinkable pages. This
      was suggested by Rob Clark in the comment to v1.

    - Factored out common shrinker code into drm_gem_shmem_helper.c
      and switched Panfrost driver to use the new common memory shrinker.
      This was proposed by Thomas Zimmermann in his prototype series that
      he shared with us in the comment to v1. Note that I only compile-tested
      the Panfrost driver.

    - Shrinker now takes object_name_lock during scan to prevent racing
      with dma-buf exporting.

    - Shrinker now takes vmap_lock during scan to prevent racing with shmem
      vmap/unmap code.

    - Added "Correct doc-comment of drm_gem_shmem_get_sg_table()" patch,
      which I sent out previously as a standalone change, since the
      drm_gem_shmem_helper.c is now touched by this patchset anyways and
      it doesn't hurt to group all the patches together.

Dmitry Osipenko (22):
  drm/gem: Properly annotate WW context on drm_gem_lock_reservations()
    error
  drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()
  drm/panfrost: Put mapping instead of shmem obj on
    panfrost_mmu_map_fault_addr() error
  drm/panfrost: Fix shrinker list corruption by madvise IOCTL
  drm/virtio: Correct drm_gem_shmem_get_sg_table() error handling
  drm/virtio: Check whether transferred 2D BO is shmem
  drm/virtio: Unlock reservations on virtio_gpu_object_shmem_init()
    error
  drm/virtio: Unlock reservations on dma_resv_reserve_fences() error
  drm/virtio: Use appropriate atomic state in
    virtio_gpu_plane_cleanup_fb()
  drm/shmem-helper: Add missing vunmap on error
  drm/shmem-helper: Correct doc-comment of drm_gem_shmem_get_sg_table()
  drm/virtio: Simplify error handling of virtio_gpu_object_create()
  drm/virtio: Improve DMA API usage for shmem BOs
  dma-buf: Introduce new locking convention
  drm/shmem-helper: Don't use vmap_use_count for dma-bufs
  drm/shmem-helper: Use reservation lock
  drm/shmem-helper: Add generic memory shrinker
  drm/gem: Add drm_gem_pin_unlocked()
  drm/virtio: Support memory shrinking
  drm/virtio: Use dev_is_pci()
  drm/virtio: Return proper error codes instead of -1
  drm/panfrost: Switch to generic memory shrinker

 drivers/dma-buf/dma-buf.c                     | 270 ++++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
 drivers/gpu/drm/drm_client.c                  |   4 +-
 drivers/gpu/drm/drm_gem.c                     |  69 +-
 drivers/gpu/drm/drm_gem_framebuffer_helper.c  |   6 +-
 drivers/gpu/drm/drm_gem_shmem_helper.c        | 718 ++++++++++++++----
 drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    |  10 +-
 drivers/gpu/drm/lima/lima_gem.c               |   8 +-
 drivers/gpu/drm/lima/lima_sched.c             |   4 +-
 drivers/gpu/drm/panfrost/Makefile             |   1 -
 drivers/gpu/drm/panfrost/panfrost_device.h    |   4 -
 drivers/gpu/drm/panfrost/panfrost_drv.c       |  26 +-
 drivers/gpu/drm/panfrost/panfrost_gem.c       |  33 +-
 drivers/gpu/drm/panfrost/panfrost_gem.h       |   9 -
 .../gpu/drm/panfrost/panfrost_gem_shrinker.c  | 122 ---
 drivers/gpu/drm/panfrost/panfrost_job.c       |  18 +-
 drivers/gpu/drm/panfrost/panfrost_mmu.c       |  21 +-
 drivers/gpu/drm/panfrost/panfrost_perfcnt.c   |   6 +-
 drivers/gpu/drm/qxl/qxl_object.c              |  17 +-
 drivers/gpu/drm/qxl/qxl_prime.c               |   4 +-
 drivers/gpu/drm/tegra/gem.c                   |   4 +
 drivers/gpu/drm/virtio/virtgpu_drv.c          |  53 +-
 drivers/gpu/drm/virtio/virtgpu_drv.h          |  23 +-
 drivers/gpu/drm/virtio/virtgpu_gem.c          |  59 +-
 drivers/gpu/drm/virtio/virtgpu_ioctl.c        |  37 +
 drivers/gpu/drm/virtio/virtgpu_kms.c          |  16 +-
 drivers/gpu/drm/virtio/virtgpu_object.c       | 203 +++--
 drivers/gpu/drm/virtio/virtgpu_plane.c        |  28 +-
 drivers/gpu/drm/virtio/virtgpu_vq.c           |  61 +-
 .../common/videobuf2/videobuf2-dma-contig.c   |  11 +-
 .../media/common/videobuf2/videobuf2-dma-sg.c |  11 +-
 .../common/videobuf2/videobuf2-vmalloc.c      |  11 +-
 include/drm/drm_device.h                      |   4 +
 include/drm/drm_gem.h                         |   6 +
 include/drm/drm_gem_shmem_helper.h            |  99 ++-
 include/linux/dma-buf.h                       |  14 +-
 include/uapi/drm/virtgpu_drm.h                |  14 +
 37 files changed, 1349 insertions(+), 661 deletions(-)
 delete mode 100644 drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c

-- 
2.35.3

[PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Dmitry Osipenko]

Hello,

This patchset introduces memory shrinker for the VirtIO-GPU DRM driver
and adds memory purging and eviction support to VirtIO-GPU driver.

The new dma-buf locking convention is introduced here as well.

During OOM, the shrinker will release BOs that are marked as "not needed"
by userspace using the new madvise IOCTL, it will also evict idling BOs
to SWAP. The userspace in this case is the Mesa VirGL driver, it will mark
the cached BOs as "not needed", allowing kernel driver to release memory
of the cached shmem BOs on lowmem situations, preventing OOM kills.

The Panfrost driver is switched to use generic memory shrinker.

This patchset includes improvements and fixes for various things that
I found while was working on the shrinker.

The Mesa and IGT patches will be kept on hold until this kernel series
will be approved and merged.

This patchset was tested using Qemu and crosvm, including both cases of
IOMMU off/on.

Mesa: https://gitlab.freedesktop.org/digetx/mesa/-/commits/virgl-madvise
IGT:  https://gitlab.freedesktop.org/digetx/igt-gpu-tools/-/commits/virtio-madvise
      https://gitlab.freedesktop.org/digetx/igt-gpu-tools/-/commits/panfrost-madvise

Changelog:

v6: - Added new VirtIO-related fix patch that previously was sent separately
      and didn't get much attention:

        drm/gem: Properly annotate WW context on drm_gem_lock_reservations() error

    - Added new patch that fixes mapping of imported dma-bufs for
      Tegra DRM and other affected drivers. It's also handy to have it
      for switching to the new dma-buf locking convention scheme:

        drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

    - Added new patch that fixes shrinker list corruption for stable Panfrost
      driver:

        drm/panfrost: Fix shrinker list corruption by madvise IOCTL

    - Added new minor patch-fix for drm-shmem:

        drm/shmem-helper: Add missing vunmap on error

    - Added fixes tag to the "Put mapping ..." patch like was suggested by
      Steven Price.

    - Added new VirtIO-GPU driver improvement patch:

        drm/virtio: Return proper error codes instead of -1

    - Reworked shrinker patches like was suggested by Daniel Vetter:

        - Introduced the new locking convention for dma-bufs. Tested on
          VirtIO-GPU, Panfrost, Lima, Tegra and Intel selftests.

        - Dropped separate purge() callback. Now single evict() does
          everything.

        - Dropped swap_in() callback from drm-shmem objects. DRM drivers
          now could and should restore only the required mappings.

        - Dropped dynamic counting of evictable pages. This simplifies
          code in exchange to *potentially* burning more CPU time on OOM.

v5: - Added new for-stable patch "drm/panfrost: Put mapping instead of
      shmem obj on panfrost_mmu_map_fault_addr() error" that corrects GEM's
      refcounting in case of error.

    - The drm_gem_shmem_v[un]map() now takes a separate vmap_lock for
      imported GEMs to avoid recursive locking of DMA reservations.
      This addresses v4 comment from Thomas Zimmermann about the potential
      deadlocking of vmapping.

    - Added ack from Thomas Zimmermann to "drm/shmem-helper: Correct
      doc-comment of drm_gem_shmem_get_sg_table()" patch.

    - Dropped explicit shmem states from the generic shrinker patch as
      was requested by Thomas Zimmermann.

    - Improved variable names and comments of the generic shrinker code.

    - Extended drm_gem_shmem_print_info() with the shrinker-state info in
      the "drm/virtio: Support memory shrinking" patch.

    - Moved evict()/swap_in()/purge() callbacks from drm_gem_object_funcs
      to drm_gem_shmem_object in the generic shrinker patch, for more
      consistency.

    - Corrected bisectability of the patches that was broken in v4
      by accident.

    - The virtio_gpu_plane_prepare_fb() now uses drm_gem_shmem_pin() instead
      of drm_gem_shmem_set_unpurgeable_and_unevictable() and does it only for
      shmem BOs in the "drm/virtio: Support memory shrinking" patch.

    - Made more functions private to drm_gem_shmem_helper.c as was requested
      by Thomas Zimmermann. This minimizes number of the public shmem helpers.

v4: - Corrected minor W=1 warnings reported by kernel test robot for v3.

    - Renamed DRM_GEM_SHMEM_PAGES_STATE_ACTIVE/INACTIVE to PINNED/UNPINNED,
      for more clarity.

v3: - Hardened shrinker's count() with usage of READ_ONCE() since we don't
      use atomic type for counting and technically compiler is free to
      re-fetch counter's variable.

    - "Correct drm_gem_shmem_get_sg_table() error handling" now uses
      PTR_ERR_OR_ZERO(), fixing typo that was made in v2.

    - Removed obsoleted shrinker from the Panfrost driver, which I missed to
      do in v2 by accident and Alyssa Rosenzweig managed to notice it.

    - CCed stable kernels in all patches that make fixes, even the minor ones,
      like was suggested by Emil Velikov and added his r-b to the patches.

    - Added t-b from Steven Price to the Panfrost's shrinker patch.

    - Corrected doc-comment of drm_gem_shmem_object.madv, like was suggested
      by Steven Price. Comment now says that madv=1 means "object is purged"
      instead of saying that value is unused.

    - Added more doc-comments to the new shmem shrinker API.

    - The "Improve DMA API usage for shmem BOs" patch got more improvements
      by removing the obsoleted drm_dev_set_unique() quirk and its comment.

    - Added patch that makes Virtio-GPU driver to use common dev_is_pci()
      helper, which was suggested by Robin Murphy.

    - Added new "drm/shmem-helper: Take GEM reservation lock instead of
      drm_gem_shmem locks" patch, which was suggested by Daniel Vetter.

    - Added new "drm/virtio: Simplify error handling of
      virtio_gpu_object_create()" patch.

    - Improved "Correct doc-comment of drm_gem_shmem_get_sg_table()" patch,
      like was suggested by Daniel Vetter, by saying that function returns
      ERR_PTR() and not errno.

    - virtio_gpu_purge_object() is fenced properly now, turned out
      virtio_gpu_notify() doesn't do fencing as I was supposing before.
      Stress testing of memory eviction revealed that.

    - Added new patch that corrects virtio_gpu_plane_cleanup_fb() to use
      appropriate atomic plane state.

    - SHMEM shrinker got eviction support.

    - VirtIO-GPU driver now supports memory eviction. It's enabled for a
      non-blob GEMs only, i.e. for VirGL. The blobs don't support dynamic
      attaching/detaching of guest's memory, so it's not trivial to enable
      them.

    - Added patch that removes obsoleted drm_gem_shmem_purge()

    - Added patch that makes drm_gem_shmem_get_pages() private.

    - Added patch that fixes lockup on dma_resv_reserve_fences() error.

v2: - Improved shrinker by using a more fine-grained locking to reduce
      contention during scan of objects and dropped locking from the
      'counting' callback by tracking count of shrinkable pages. This
      was suggested by Rob Clark in the comment to v1.

    - Factored out common shrinker code into drm_gem_shmem_helper.c
      and switched Panfrost driver to use the new common memory shrinker.
      This was proposed by Thomas Zimmermann in his prototype series that
      he shared with us in the comment to v1. Note that I only compile-tested
      the Panfrost driver.

    - Shrinker now takes object_name_lock during scan to prevent racing
      with dma-buf exporting.

    - Shrinker now takes vmap_lock during scan to prevent racing with shmem
      vmap/unmap code.

    - Added "Correct doc-comment of drm_gem_shmem_get_sg_table()" patch,
      which I sent out previously as a standalone change, since the
      drm_gem_shmem_helper.c is now touched by this patchset anyways and
      it doesn't hurt to group all the patches together.

Dmitry Osipenko (22):
  drm/gem: Properly annotate WW context on drm_gem_lock_reservations()
    error
  drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()
  drm/panfrost: Put mapping instead of shmem obj on
    panfrost_mmu_map_fault_addr() error
  drm/panfrost: Fix shrinker list corruption by madvise IOCTL
  drm/virtio: Correct drm_gem_shmem_get_sg_table() error handling
  drm/virtio: Check whether transferred 2D BO is shmem
  drm/virtio: Unlock reservations on virtio_gpu_object_shmem_init()
    error
  drm/virtio: Unlock reservations on dma_resv_reserve_fences() error
  drm/virtio: Use appropriate atomic state in
    virtio_gpu_plane_cleanup_fb()
  drm/shmem-helper: Add missing vunmap on error
  drm/shmem-helper: Correct doc-comment of drm_gem_shmem_get_sg_table()
  drm/virtio: Simplify error handling of virtio_gpu_object_create()
  drm/virtio: Improve DMA API usage for shmem BOs
  dma-buf: Introduce new locking convention
  drm/shmem-helper: Don't use vmap_use_count for dma-bufs
  drm/shmem-helper: Use reservation lock
  drm/shmem-helper: Add generic memory shrinker
  drm/gem: Add drm_gem_pin_unlocked()
  drm/virtio: Support memory shrinking
  drm/virtio: Use dev_is_pci()
  drm/virtio: Return proper error codes instead of -1
  drm/panfrost: Switch to generic memory shrinker

 drivers/dma-buf/dma-buf.c                     | 270 ++++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
 drivers/gpu/drm/drm_client.c                  |   4 +-
 drivers/gpu/drm/drm_gem.c                     |  69 +-
 drivers/gpu/drm/drm_gem_framebuffer_helper.c  |   6 +-
 drivers/gpu/drm/drm_gem_shmem_helper.c        | 718 ++++++++++++++----
 drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    |  10 +-
 drivers/gpu/drm/lima/lima_gem.c               |   8 +-
 drivers/gpu/drm/lima/lima_sched.c             |   4 +-
 drivers/gpu/drm/panfrost/Makefile             |   1 -
 drivers/gpu/drm/panfrost/panfrost_device.h    |   4 -
 drivers/gpu/drm/panfrost/panfrost_drv.c       |  26 +-
 drivers/gpu/drm/panfrost/panfrost_gem.c       |  33 +-
 drivers/gpu/drm/panfrost/panfrost_gem.h       |   9 -
 .../gpu/drm/panfrost/panfrost_gem_shrinker.c  | 122 ---
 drivers/gpu/drm/panfrost/panfrost_job.c       |  18 +-
 drivers/gpu/drm/panfrost/panfrost_mmu.c       |  21 +-
 drivers/gpu/drm/panfrost/panfrost_perfcnt.c   |   6 +-
 drivers/gpu/drm/qxl/qxl_object.c              |  17 +-
 drivers/gpu/drm/qxl/qxl_prime.c               |   4 +-
 drivers/gpu/drm/tegra/gem.c                   |   4 +
 drivers/gpu/drm/virtio/virtgpu_drv.c          |  53 +-
 drivers/gpu/drm/virtio/virtgpu_drv.h          |  23 +-
 drivers/gpu/drm/virtio/virtgpu_gem.c          |  59 +-
 drivers/gpu/drm/virtio/virtgpu_ioctl.c        |  37 +
 drivers/gpu/drm/virtio/virtgpu_kms.c          |  16 +-
 drivers/gpu/drm/virtio/virtgpu_object.c       | 203 +++--
 drivers/gpu/drm/virtio/virtgpu_plane.c        |  28 +-
 drivers/gpu/drm/virtio/virtgpu_vq.c           |  61 +-
 .../common/videobuf2/videobuf2-dma-contig.c   |  11 +-
 .../media/common/videobuf2/videobuf2-dma-sg.c |  11 +-
 .../common/videobuf2/videobuf2-vmalloc.c      |  11 +-
 include/drm/drm_device.h                      |   4 +
 include/drm/drm_gem.h                         |   6 +
 include/drm/drm_gem_shmem_helper.h            |  99 ++-
 include/linux/dma-buf.h                       |  14 +-
 include/uapi/drm/virtgpu_drm.h                |  14 +
 37 files changed, 1349 insertions(+), 661 deletions(-)
 delete mode 100644 drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c

-- 
2.35.3

[PATCH v6 01/22] drm/gem: Properly annotate WW context on drm_gem_lock_reservations() error

[Dmitry Osipenko]

Use ww_acquire_fini() in the error code paths. Otherwise lockdep
thinks that lock is held when lock's memory is freed after the
drm_gem_lock_reservations() error. The WW needs to be annotated
as "freed", which fixes the noisy "WARNING: held lock freed!" splat
of VirtIO-GPU driver with CONFIG_DEBUG_MUTEXES=y and enabled lockdep.

Cc: stable@vger.kernel.org
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/drm_gem.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index eb0c2d041f13..86d670c71286 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -1226,7 +1226,7 @@ drm_gem_lock_reservations(struct drm_gem_object **objs, int count,
 		ret = dma_resv_lock_slow_interruptible(obj->resv,
 								 acquire_ctx);
 		if (ret) {
-			ww_acquire_done(acquire_ctx);
+			ww_acquire_fini(acquire_ctx);
 			return ret;
 		}
 	}
@@ -1251,7 +1251,7 @@ drm_gem_lock_reservations(struct drm_gem_object **objs, int count,
 				goto retry;
 			}
 
-			ww_acquire_done(acquire_ctx);
+			ww_acquire_fini(acquire_ctx);
 			return ret;
 		}
 	}
-- 
2.35.3

[PATCH v6 01/22] drm/gem: Properly annotate WW context on drm_gem_lock_reservations() error

[Dmitry Osipenko]

Use ww_acquire_fini() in the error code paths. Otherwise lockdep
thinks that lock is held when lock's memory is freed after the
drm_gem_lock_reservations() error. The WW needs to be annotated
as "freed", which fixes the noisy "WARNING: held lock freed!" splat
of VirtIO-GPU driver with CONFIG_DEBUG_MUTEXES=y and enabled lockdep.

Cc: stable@vger.kernel.org
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/drm_gem.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index eb0c2d041f13..86d670c71286 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -1226,7 +1226,7 @@ drm_gem_lock_reservations(struct drm_gem_object **objs, int count,
 		ret = dma_resv_lock_slow_interruptible(obj->resv,
 								 acquire_ctx);
 		if (ret) {
-			ww_acquire_done(acquire_ctx);
+			ww_acquire_fini(acquire_ctx);
 			return ret;
 		}
 	}
@@ -1251,7 +1251,7 @@ drm_gem_lock_reservations(struct drm_gem_object **objs, int count,
 				goto retry;
 			}
 
-			ww_acquire_done(acquire_ctx);
+			ww_acquire_fini(acquire_ctx);
 			return ret;
 		}
 	}
-- 
2.35.3

[PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Dmitry Osipenko]

Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
handle imported dma-bufs properly, which results in mapping of something
else than the imported dma-buf. For example, on NVIDIA Tegra we get a hard
lockup when userspace writes to the memory mapping of a dma-buf that was
imported into Tegra's DRM GEM.

To fix this bug, move mapping of imported dma-bufs to drm_gem_mmap_obj().
Now mmaping of imported dma-bufs works properly for all DRM drivers.

Cc: stable@vger.kernel.org
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/drm_gem.c              | 3 +++
 drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
 drivers/gpu/drm/tegra/gem.c            | 4 ++++
 3 files changed, 7 insertions(+), 9 deletions(-)

diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index 86d670c71286..7c0b025508e4 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,
 	if (obj_size < vma->vm_end - vma->vm_start)
 		return -EINVAL;
 
+	if (obj->import_attach)
+		return dma_buf_mmap(obj->dma_buf, vma, 0);
+
 	/* Take a ref for this mapping of the object, so that the fault
 	 * handler can dereference the mmap offset's pointer to the object.
 	 * This reference is cleaned up by the corresponding vm_close
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 8ad0e02991ca..6190f5018986 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -609,17 +609,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_vm_ops);
  */
 int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct *vma)
 {
-	struct drm_gem_object *obj = &shmem->base;
 	int ret;
 
-	if (obj->import_attach) {
-		/* Drop the reference drm_gem_mmap_obj() acquired.*/
-		drm_gem_object_put(obj);
-		vma->vm_private_data = NULL;
-
-		return dma_buf_mmap(obj->dma_buf, vma, 0);
-	}
-
 	ret = drm_gem_shmem_get_pages(shmem);
 	if (ret) {
 		drm_gem_vm_close(vma);
diff --git a/drivers/gpu/drm/tegra/gem.c b/drivers/gpu/drm/tegra/gem.c
index 7c7dd84e6db8..f92aa20d63bb 100644
--- a/drivers/gpu/drm/tegra/gem.c
+++ b/drivers/gpu/drm/tegra/gem.c
@@ -564,6 +564,10 @@ int __tegra_gem_mmap(struct drm_gem_object *gem, struct vm_area_struct *vma)
 {
 	struct tegra_bo *bo = to_tegra_bo(gem);
 
+	/* imported dmu-buf is mapped by drm_gem_mmap_obj()  */
+	if (gem->import_attach)
+		return 0;
+
 	if (!bo->pages) {
 		unsigned long vm_pgoff = vma->vm_pgoff;
 		int err;
-- 
2.35.3

[PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Dmitry Osipenko]

Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
handle imported dma-bufs properly, which results in mapping of something
else than the imported dma-buf. For example, on NVIDIA Tegra we get a hard
lockup when userspace writes to the memory mapping of a dma-buf that was
imported into Tegra's DRM GEM.

To fix this bug, move mapping of imported dma-bufs to drm_gem_mmap_obj().
Now mmaping of imported dma-bufs works properly for all DRM drivers.

Cc: stable@vger.kernel.org
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/drm_gem.c              | 3 +++
 drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
 drivers/gpu/drm/tegra/gem.c            | 4 ++++
 3 files changed, 7 insertions(+), 9 deletions(-)

diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index 86d670c71286..7c0b025508e4 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,
 	if (obj_size < vma->vm_end - vma->vm_start)
 		return -EINVAL;
 
+	if (obj->import_attach)
+		return dma_buf_mmap(obj->dma_buf, vma, 0);
+
 	/* Take a ref for this mapping of the object, so that the fault
 	 * handler can dereference the mmap offset's pointer to the object.
 	 * This reference is cleaned up by the corresponding vm_close
diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 8ad0e02991ca..6190f5018986 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -609,17 +609,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_vm_ops);
  */
 int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct *vma)
 {
-	struct drm_gem_object *obj = &shmem->base;
 	int ret;
 
-	if (obj->import_attach) {
-		/* Drop the reference drm_gem_mmap_obj() acquired.*/
-		drm_gem_object_put(obj);
-		vma->vm_private_data = NULL;
-
-		return dma_buf_mmap(obj->dma_buf, vma, 0);
-	}
-
 	ret = drm_gem_shmem_get_pages(shmem);
 	if (ret) {
 		drm_gem_vm_close(vma);
diff --git a/drivers/gpu/drm/tegra/gem.c b/drivers/gpu/drm/tegra/gem.c
index 7c7dd84e6db8..f92aa20d63bb 100644
--- a/drivers/gpu/drm/tegra/gem.c
+++ b/drivers/gpu/drm/tegra/gem.c
@@ -564,6 +564,10 @@ int __tegra_gem_mmap(struct drm_gem_object *gem, struct vm_area_struct *vma)
 {
 	struct tegra_bo *bo = to_tegra_bo(gem);
 
+	/* imported dmu-buf is mapped by drm_gem_mmap_obj()  */
+	if (gem->import_attach)
+		return 0;
+
 	if (!bo->pages) {
 		unsigned long vm_pgoff = vma->vm_pgoff;
 		int err;
-- 
2.35.3

[PATCH v6 03/22] drm/panfrost: Put mapping instead of shmem obj on panfrost_mmu_map_fault_addr() error

[Dmitry Osipenko]

When panfrost_mmu_map_fault_addr() fails, the BO's mapping should be
unreferenced and not the shmem object which backs the mapping.

Cc: stable@vger.kernel.org
Fixes: bdefca2d8dc0 ("drm/panfrost: Add the panfrost_gem_mapping concept")
Reviewed-by: Steven Price <steven.price@arm.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/panfrost/panfrost_mmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/panfrost/panfrost_mmu.c b/drivers/gpu/drm/panfrost/panfrost_mmu.c
index d3f82b26a631..b285a8001b1d 100644
--- a/drivers/gpu/drm/panfrost/panfrost_mmu.c
+++ b/drivers/gpu/drm/panfrost/panfrost_mmu.c
@@ -518,7 +518,7 @@ static int panfrost_mmu_map_fault_addr(struct panfrost_device *pfdev, int as,
 err_pages:
 	drm_gem_shmem_put_pages(&bo->base);
 err_bo:
-	drm_gem_object_put(&bo->base.base);
+	panfrost_gem_mapping_put(bomapping);
 	return ret;
 }
 
-- 
2.35.3

[PATCH v6 03/22] drm/panfrost: Put mapping instead of shmem obj on panfrost_mmu_map_fault_addr() error

[Dmitry Osipenko]

When panfrost_mmu_map_fault_addr() fails, the BO's mapping should be
unreferenced and not the shmem object which backs the mapping.

Cc: stable@vger.kernel.org
Fixes: bdefca2d8dc0 ("drm/panfrost: Add the panfrost_gem_mapping concept")
Reviewed-by: Steven Price <steven.price@arm.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/panfrost/panfrost_mmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/panfrost/panfrost_mmu.c b/drivers/gpu/drm/panfrost/panfrost_mmu.c
index d3f82b26a631..b285a8001b1d 100644
--- a/drivers/gpu/drm/panfrost/panfrost_mmu.c
+++ b/drivers/gpu/drm/panfrost/panfrost_mmu.c
@@ -518,7 +518,7 @@ static int panfrost_mmu_map_fault_addr(struct panfrost_device *pfdev, int as,
 err_pages:
 	drm_gem_shmem_put_pages(&bo->base);
 err_bo:
-	drm_gem_object_put(&bo->base.base);
+	panfrost_gem_mapping_put(bomapping);
 	return ret;
 }
 
-- 
2.35.3

[PATCH v6 04/22] drm/panfrost: Fix shrinker list corruption by madvise IOCTL

[Dmitry Osipenko]

Calling madvise IOCTL twice on BO causes memory shrinker list corruption
and crashes kernel because BO is already on the list and it's added to
the list again, while BO should be removed from from the list before it's
re-added. Fix it.

Cc: stable@vger.kernel.org
Fixes: 013b65101315 ("drm/panfrost: Add madvise and shrinker support")
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/panfrost/panfrost_drv.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
index 087e69b98d06..b1e6d238674f 100644
--- a/drivers/gpu/drm/panfrost/panfrost_drv.c
+++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
@@ -433,8 +433,8 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
 
 	if (args->retained) {
 		if (args->madv == PANFROST_MADV_DONTNEED)
-			list_add_tail(&bo->base.madv_list,
-				      &pfdev->shrinker_list);
+			list_move_tail(&bo->base.madv_list,
+				       &pfdev->shrinker_list);
 		else if (args->madv == PANFROST_MADV_WILLNEED)
 			list_del_init(&bo->base.madv_list);
 	}
-- 
2.35.3

[PATCH v6 04/22] drm/panfrost: Fix shrinker list corruption by madvise IOCTL

[Dmitry Osipenko]

Calling madvise IOCTL twice on BO causes memory shrinker list corruption
and crashes kernel because BO is already on the list and it's added to
the list again, while BO should be removed from from the list before it's
re-added. Fix it.

Cc: stable@vger.kernel.org
Fixes: 013b65101315 ("drm/panfrost: Add madvise and shrinker support")
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/panfrost/panfrost_drv.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
index 087e69b98d06..b1e6d238674f 100644
--- a/drivers/gpu/drm/panfrost/panfrost_drv.c
+++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
@@ -433,8 +433,8 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
 
 	if (args->retained) {
 		if (args->madv == PANFROST_MADV_DONTNEED)
-			list_add_tail(&bo->base.madv_list,
-				      &pfdev->shrinker_list);
+			list_move_tail(&bo->base.madv_list,
+				       &pfdev->shrinker_list);
 		else if (args->madv == PANFROST_MADV_WILLNEED)
 			list_del_init(&bo->base.madv_list);
 	}
-- 
2.35.3

[PATCH v6 05/22] drm/virtio: Correct drm_gem_shmem_get_sg_table() error handling

[Dmitry Osipenko]

drm_gem_shmem_get_sg_table() never ever returned NULL on error. Correct
the error handling to avoid crash on OOM.

Cc: stable@vger.kernel.org
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_object.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c b/drivers/gpu/drm/virtio/virtgpu_object.c
index f293e6ad52da..3d0c8d4d1c20 100644
--- a/drivers/gpu/drm/virtio/virtgpu_object.c
+++ b/drivers/gpu/drm/virtio/virtgpu_object.c
@@ -168,9 +168,11 @@ static int virtio_gpu_object_shmem_init(struct virtio_gpu_device *vgdev,
 	 * since virtio_gpu doesn't support dma-buf import from other devices.
 	 */
 	shmem->pages = drm_gem_shmem_get_sg_table(&bo->base);
-	if (!shmem->pages) {
+	ret = PTR_ERR_OR_ZERO(shmem->pages);
+	if (ret) {
 		drm_gem_shmem_unpin(&bo->base);
-		return -EINVAL;
+		shmem->pages = NULL;
+		return ret;
 	}
 
 	if (use_dma_api) {
-- 
2.35.3

[PATCH v6 05/22] drm/virtio: Correct drm_gem_shmem_get_sg_table() error handling

[Dmitry Osipenko]

drm_gem_shmem_get_sg_table() never ever returned NULL on error. Correct
the error handling to avoid crash on OOM.

Cc: stable@vger.kernel.org
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_object.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c b/drivers/gpu/drm/virtio/virtgpu_object.c
index f293e6ad52da..3d0c8d4d1c20 100644
--- a/drivers/gpu/drm/virtio/virtgpu_object.c
+++ b/drivers/gpu/drm/virtio/virtgpu_object.c
@@ -168,9 +168,11 @@ static int virtio_gpu_object_shmem_init(struct virtio_gpu_device *vgdev,
 	 * since virtio_gpu doesn't support dma-buf import from other devices.
 	 */
 	shmem->pages = drm_gem_shmem_get_sg_table(&bo->base);
-	if (!shmem->pages) {
+	ret = PTR_ERR_OR_ZERO(shmem->pages);
+	if (ret) {
 		drm_gem_shmem_unpin(&bo->base);
-		return -EINVAL;
+		shmem->pages = NULL;
+		return ret;
 	}
 
 	if (use_dma_api) {
-- 
2.35.3

[PATCH v6 06/22] drm/virtio: Check whether transferred 2D BO is shmem

[Dmitry Osipenko]

Transferred 2D BO always must be a shmem BO. Add check for that to prevent
NULL dereference if userspace passes a VRAM BO.

Cc: stable@vger.kernel.org
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_vq.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_vq.c b/drivers/gpu/drm/virtio/virtgpu_vq.c
index 7c052efe8836..2edf31806b74 100644
--- a/drivers/gpu/drm/virtio/virtgpu_vq.c
+++ b/drivers/gpu/drm/virtio/virtgpu_vq.c
@@ -595,7 +595,7 @@ void virtio_gpu_cmd_transfer_to_host_2d(struct virtio_gpu_device *vgdev,
 	bool use_dma_api = !virtio_has_dma_quirk(vgdev->vdev);
 	struct virtio_gpu_object_shmem *shmem = to_virtio_gpu_shmem(bo);
 
-	if (use_dma_api)
+	if (virtio_gpu_is_shmem(bo) && use_dma_api)
 		dma_sync_sgtable_for_device(vgdev->vdev->dev.parent,
 					    shmem->pages, DMA_TO_DEVICE);
 
-- 
2.35.3

[PATCH v6 06/22] drm/virtio: Check whether transferred 2D BO is shmem

[Dmitry Osipenko]

Transferred 2D BO always must be a shmem BO. Add check for that to prevent
NULL dereference if userspace passes a VRAM BO.

Cc: stable@vger.kernel.org
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_vq.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_vq.c b/drivers/gpu/drm/virtio/virtgpu_vq.c
index 7c052efe8836..2edf31806b74 100644
--- a/drivers/gpu/drm/virtio/virtgpu_vq.c
+++ b/drivers/gpu/drm/virtio/virtgpu_vq.c
@@ -595,7 +595,7 @@ void virtio_gpu_cmd_transfer_to_host_2d(struct virtio_gpu_device *vgdev,
 	bool use_dma_api = !virtio_has_dma_quirk(vgdev->vdev);
 	struct virtio_gpu_object_shmem *shmem = to_virtio_gpu_shmem(bo);
 
-	if (use_dma_api)
+	if (virtio_gpu_is_shmem(bo) && use_dma_api)
 		dma_sync_sgtable_for_device(vgdev->vdev->dev.parent,
 					    shmem->pages, DMA_TO_DEVICE);
 
-- 
2.35.3

[PATCH v6 07/22] drm/virtio: Unlock reservations on virtio_gpu_object_shmem_init() error

[Dmitry Osipenko]

Unlock reservations in the error code path of virtio_gpu_object_create()
to silence debug warning splat produced by ww_mutex_destroy(&obj->lock)
when GEM is released with the held lock.

Cc: stable@vger.kernel.org
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_object.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c b/drivers/gpu/drm/virtio/virtgpu_object.c
index 3d0c8d4d1c20..21c19cdedce0 100644
--- a/drivers/gpu/drm/virtio/virtgpu_object.c
+++ b/drivers/gpu/drm/virtio/virtgpu_object.c
@@ -250,6 +250,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
 
 	ret = virtio_gpu_object_shmem_init(vgdev, bo, &ents, &nents);
 	if (ret != 0) {
+		if (fence)
+			virtio_gpu_array_unlock_resv(objs);
 		virtio_gpu_array_put_free(objs);
 		virtio_gpu_free_object(&shmem_obj->base);
 		return ret;
-- 
2.35.3

[PATCH v6 07/22] drm/virtio: Unlock reservations on virtio_gpu_object_shmem_init() error

[Dmitry Osipenko]

Unlock reservations in the error code path of virtio_gpu_object_create()
to silence debug warning splat produced by ww_mutex_destroy(&obj->lock)
when GEM is released with the held lock.

Cc: stable@vger.kernel.org
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_object.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c b/drivers/gpu/drm/virtio/virtgpu_object.c
index 3d0c8d4d1c20..21c19cdedce0 100644
--- a/drivers/gpu/drm/virtio/virtgpu_object.c
+++ b/drivers/gpu/drm/virtio/virtgpu_object.c
@@ -250,6 +250,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
 
 	ret = virtio_gpu_object_shmem_init(vgdev, bo, &ents, &nents);
 	if (ret != 0) {
+		if (fence)
+			virtio_gpu_array_unlock_resv(objs);
 		virtio_gpu_array_put_free(objs);
 		virtio_gpu_free_object(&shmem_obj->base);
 		return ret;
-- 
2.35.3

[PATCH v6 08/22] drm/virtio: Unlock reservations on dma_resv_reserve_fences() error

[Dmitry Osipenko]

Unlock reservations on dma_resv_reserve_fences() error to fix recursive
locking of the reservations when this error happens.

Cc: stable@vger.kernel.org
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_gem.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_gem.c b/drivers/gpu/drm/virtio/virtgpu_gem.c
index 580a78809836..7db48d17ee3a 100644
--- a/drivers/gpu/drm/virtio/virtgpu_gem.c
+++ b/drivers/gpu/drm/virtio/virtgpu_gem.c
@@ -228,8 +228,10 @@ int virtio_gpu_array_lock_resv(struct virtio_gpu_object_array *objs)
 
 	for (i = 0; i < objs->nents; ++i) {
 		ret = dma_resv_reserve_fences(objs->objs[i]->resv, 1);
-		if (ret)
+		if (ret) {
+			virtio_gpu_array_unlock_resv(objs);
 			return ret;
+		}
 	}
 	return ret;
 }
-- 
2.35.3

[PATCH v6 08/22] drm/virtio: Unlock reservations on dma_resv_reserve_fences() error

[Dmitry Osipenko]

Unlock reservations on dma_resv_reserve_fences() error to fix recursive
locking of the reservations when this error happens.

Cc: stable@vger.kernel.org
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_gem.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_gem.c b/drivers/gpu/drm/virtio/virtgpu_gem.c
index 580a78809836..7db48d17ee3a 100644
--- a/drivers/gpu/drm/virtio/virtgpu_gem.c
+++ b/drivers/gpu/drm/virtio/virtgpu_gem.c
@@ -228,8 +228,10 @@ int virtio_gpu_array_lock_resv(struct virtio_gpu_object_array *objs)
 
 	for (i = 0; i < objs->nents; ++i) {
 		ret = dma_resv_reserve_fences(objs->objs[i]->resv, 1);
-		if (ret)
+		if (ret) {
+			virtio_gpu_array_unlock_resv(objs);
 			return ret;
+		}
 	}
 	return ret;
 }
-- 
2.35.3

[PATCH v6 09/22] drm/virtio: Use appropriate atomic state in virtio_gpu_plane_cleanup_fb()

[Dmitry Osipenko]

Make virtio_gpu_plane_cleanup_fb() to clean the state which DRM core
wants to clean up and not the current plane's state. Normally the older
atomic state is cleaned up, but the newer state could also be cleaned up
in case of aborted commits.

Cc: stable@vger.kernel.org
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_plane.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_plane.c b/drivers/gpu/drm/virtio/virtgpu_plane.c
index 6d3cc9e238a4..7148f3813d8b 100644
--- a/drivers/gpu/drm/virtio/virtgpu_plane.c
+++ b/drivers/gpu/drm/virtio/virtgpu_plane.c
@@ -266,14 +266,14 @@ static int virtio_gpu_plane_prepare_fb(struct drm_plane *plane,
 }
 
 static void virtio_gpu_plane_cleanup_fb(struct drm_plane *plane,
-					struct drm_plane_state *old_state)
+					struct drm_plane_state *state)
 {
 	struct virtio_gpu_framebuffer *vgfb;
 
-	if (!plane->state->fb)
+	if (!state->fb)
 		return;
 
-	vgfb = to_virtio_gpu_framebuffer(plane->state->fb);
+	vgfb = to_virtio_gpu_framebuffer(state->fb);
 	if (vgfb->fence) {
 		dma_fence_put(&vgfb->fence->f);
 		vgfb->fence = NULL;
-- 
2.35.3

[PATCH v6 09/22] drm/virtio: Use appropriate atomic state in virtio_gpu_plane_cleanup_fb()

[Dmitry Osipenko]

Make virtio_gpu_plane_cleanup_fb() to clean the state which DRM core
wants to clean up and not the current plane's state. Normally the older
atomic state is cleaned up, but the newer state could also be cleaned up
in case of aborted commits.

Cc: stable@vger.kernel.org
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_plane.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_plane.c b/drivers/gpu/drm/virtio/virtgpu_plane.c
index 6d3cc9e238a4..7148f3813d8b 100644
--- a/drivers/gpu/drm/virtio/virtgpu_plane.c
+++ b/drivers/gpu/drm/virtio/virtgpu_plane.c
@@ -266,14 +266,14 @@ static int virtio_gpu_plane_prepare_fb(struct drm_plane *plane,
 }
 
 static void virtio_gpu_plane_cleanup_fb(struct drm_plane *plane,
-					struct drm_plane_state *old_state)
+					struct drm_plane_state *state)
 {
 	struct virtio_gpu_framebuffer *vgfb;
 
-	if (!plane->state->fb)
+	if (!state->fb)
 		return;
 
-	vgfb = to_virtio_gpu_framebuffer(plane->state->fb);
+	vgfb = to_virtio_gpu_framebuffer(state->fb);
 	if (vgfb->fence) {
 		dma_fence_put(&vgfb->fence->f);
 		vgfb->fence = NULL;
-- 
2.35.3

[PATCH v6 10/22] drm/shmem-helper: Add missing vunmap on error

[Dmitry Osipenko]

The vmapping of dma-buf may succeed, but DRM SHMEM rejects the iomem
mappings, and thus, drm_gem_shmem_vmap_locked() should unvmap the iomem
before erroring out.

Cc: stable@vger.kernel.org
Fixes: 49a3f51dfeee ("drm/gem: Use struct dma_buf_map in GEM vmap ops and convert GEM backends")
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/drm_gem_shmem_helper.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 6190f5018986..54b0ba28aa0a 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -302,6 +302,7 @@ static int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object *shmem,
 		ret = dma_buf_vmap(obj->import_attach->dmabuf, map);
 		if (!ret) {
 			if (WARN_ON(map->is_iomem)) {
+				dma_buf_vunmap(obj->import_attach->dmabuf, map);
 				ret = -EIO;
 				goto err_put_pages;
 			}
-- 
2.35.3

[PATCH v6 10/22] drm/shmem-helper: Add missing vunmap on error

[Dmitry Osipenko]

The vmapping of dma-buf may succeed, but DRM SHMEM rejects the iomem
mappings, and thus, drm_gem_shmem_vmap_locked() should unvmap the iomem
before erroring out.

Cc: stable@vger.kernel.org
Fixes: 49a3f51dfeee ("drm/gem: Use struct dma_buf_map in GEM vmap ops and convert GEM backends")
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/drm_gem_shmem_helper.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 6190f5018986..54b0ba28aa0a 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -302,6 +302,7 @@ static int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object *shmem,
 		ret = dma_buf_vmap(obj->import_attach->dmabuf, map);
 		if (!ret) {
 			if (WARN_ON(map->is_iomem)) {
+				dma_buf_vunmap(obj->import_attach->dmabuf, map);
 				ret = -EIO;
 				goto err_put_pages;
 			}
-- 
2.35.3

[PATCH v6 11/22] drm/shmem-helper: Correct doc-comment of drm_gem_shmem_get_sg_table()

[Dmitry Osipenko]

drm_gem_shmem_get_sg_table() never returns NULL on error, but a ERR_PTR.
Correct the doc comment which says that it returns NULL on error.

Acked-by: Thomas Zimmermann <tzimmermann@suse.de>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/drm_gem_shmem_helper.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 54b0ba28aa0a..7232e321fdb4 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -654,7 +654,8 @@ EXPORT_SYMBOL(drm_gem_shmem_print_info);
  * drm_gem_shmem_get_pages_sgt() instead.
  *
  * Returns:
- * A pointer to the scatter/gather table of pinned pages or NULL on failure.
+ * A pointer to the scatter/gather table of pinned pages or an ERR_PTR()-encoded
+ * error code on failure.
  */
 struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem)
 {
@@ -680,7 +681,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_get_sg_table);
  * drm_gem_shmem_get_sg_table() should not be directly called by drivers.
  *
  * Returns:
- * A pointer to the scatter/gather table of pinned pages or errno on failure.
+ * A pointer to the scatter/gather table of pinned pages or an ERR_PTR()-encoded
+ * error code on failure.
  */
 struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
 {
-- 
2.35.3

[PATCH v6 11/22] drm/shmem-helper: Correct doc-comment of drm_gem_shmem_get_sg_table()

[Dmitry Osipenko]

drm_gem_shmem_get_sg_table() never returns NULL on error, but a ERR_PTR.
Correct the doc comment which says that it returns NULL on error.

Acked-by: Thomas Zimmermann <tzimmermann@suse.de>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/drm_gem_shmem_helper.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 54b0ba28aa0a..7232e321fdb4 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -654,7 +654,8 @@ EXPORT_SYMBOL(drm_gem_shmem_print_info);
  * drm_gem_shmem_get_pages_sgt() instead.
  *
  * Returns:
- * A pointer to the scatter/gather table of pinned pages or NULL on failure.
+ * A pointer to the scatter/gather table of pinned pages or an ERR_PTR()-encoded
+ * error code on failure.
  */
 struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem)
 {
@@ -680,7 +681,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_get_sg_table);
  * drm_gem_shmem_get_sg_table() should not be directly called by drivers.
  *
  * Returns:
- * A pointer to the scatter/gather table of pinned pages or errno on failure.
+ * A pointer to the scatter/gather table of pinned pages or an ERR_PTR()-encoded
+ * error code on failure.
  */
 struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
 {
-- 
2.35.3

[PATCH v6 12/22] drm/virtio: Simplify error handling of virtio_gpu_object_create()

[Dmitry Osipenko]

Change the order of SHMEM initialization and reservation locking
to make code cleaner and to prepare for transitioning of the common
GEM SHMEM code to use the GEM's reservation lock instead of the
shmem.page_lock.

There is no need to lock reservation during allocation of the SHMEM pages
because the lock is needed only to avoid racing with the async host-side
allocation. Hence we can safely move the SHMEM initialization out of the
reservation lock.

Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_object.c | 13 ++++---------
 1 file changed, 4 insertions(+), 9 deletions(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c b/drivers/gpu/drm/virtio/virtgpu_object.c
index 21c19cdedce0..18f70ef6b4d0 100644
--- a/drivers/gpu/drm/virtio/virtgpu_object.c
+++ b/drivers/gpu/drm/virtio/virtgpu_object.c
@@ -236,6 +236,10 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
 
 	bo->dumb = params->dumb;
 
+	ret = virtio_gpu_object_shmem_init(vgdev, bo, &ents, &nents);
+	if (ret != 0)
+		goto err_put_id;
+
 	if (fence) {
 		ret = -ENOMEM;
 		objs = virtio_gpu_array_alloc(1);
@@ -248,15 +252,6 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
 			goto err_put_objs;
 	}
 
-	ret = virtio_gpu_object_shmem_init(vgdev, bo, &ents, &nents);
-	if (ret != 0) {
-		if (fence)
-			virtio_gpu_array_unlock_resv(objs);
-		virtio_gpu_array_put_free(objs);
-		virtio_gpu_free_object(&shmem_obj->base);
-		return ret;
-	}
-
 	if (params->blob) {
 		if (params->blob_mem == VIRTGPU_BLOB_MEM_GUEST)
 			bo->guest_blob = true;
-- 
2.35.3

[PATCH v6 12/22] drm/virtio: Simplify error handling of virtio_gpu_object_create()

[Dmitry Osipenko]

Change the order of SHMEM initialization and reservation locking
to make code cleaner and to prepare for transitioning of the common
GEM SHMEM code to use the GEM's reservation lock instead of the
shmem.page_lock.

There is no need to lock reservation during allocation of the SHMEM pages
because the lock is needed only to avoid racing with the async host-side
allocation. Hence we can safely move the SHMEM initialization out of the
reservation lock.

Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_object.c | 13 ++++---------
 1 file changed, 4 insertions(+), 9 deletions(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c b/drivers/gpu/drm/virtio/virtgpu_object.c
index 21c19cdedce0..18f70ef6b4d0 100644
--- a/drivers/gpu/drm/virtio/virtgpu_object.c
+++ b/drivers/gpu/drm/virtio/virtgpu_object.c
@@ -236,6 +236,10 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
 
 	bo->dumb = params->dumb;
 
+	ret = virtio_gpu_object_shmem_init(vgdev, bo, &ents, &nents);
+	if (ret != 0)
+		goto err_put_id;
+
 	if (fence) {
 		ret = -ENOMEM;
 		objs = virtio_gpu_array_alloc(1);
@@ -248,15 +252,6 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
 			goto err_put_objs;
 	}
 
-	ret = virtio_gpu_object_shmem_init(vgdev, bo, &ents, &nents);
-	if (ret != 0) {
-		if (fence)
-			virtio_gpu_array_unlock_resv(objs);
-		virtio_gpu_array_put_free(objs);
-		virtio_gpu_free_object(&shmem_obj->base);
-		return ret;
-	}
-
 	if (params->blob) {
 		if (params->blob_mem == VIRTGPU_BLOB_MEM_GUEST)
 			bo->guest_blob = true;
-- 
2.35.3

[PATCH v6 13/22] drm/virtio: Improve DMA API usage for shmem BOs

[Dmitry Osipenko]

DRM API requires the DRM's driver to be backed with the device that can
be used for generic DMA operations. The VirtIO-GPU device can't perform
DMA operations if it uses PCI transport because PCI device driver creates
a virtual VirtIO-GPU device that isn't associated with the PCI. Use PCI's
GPU device for the DRM's device instead of the VirtIO-GPU device and drop
DMA-related hacks from the VirtIO-GPU driver.

Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_drv.c    | 51 ++++++----------------
 drivers/gpu/drm/virtio/virtgpu_drv.h    |  5 +--
 drivers/gpu/drm/virtio/virtgpu_kms.c    |  7 ++--
 drivers/gpu/drm/virtio/virtgpu_object.c | 56 +++++--------------------
 drivers/gpu/drm/virtio/virtgpu_vq.c     | 13 +++---
 5 files changed, 32 insertions(+), 100 deletions(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.c b/drivers/gpu/drm/virtio/virtgpu_drv.c
index 5f25a8d15464..0141b7df97ec 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.c
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.c
@@ -46,12 +46,11 @@ static int virtio_gpu_modeset = -1;
 MODULE_PARM_DESC(modeset, "Disable/Enable modesetting");
 module_param_named(modeset, virtio_gpu_modeset, int, 0400);
 
-static int virtio_gpu_pci_quirk(struct drm_device *dev, struct virtio_device *vdev)
+static int virtio_gpu_pci_quirk(struct drm_device *dev)
 {
-	struct pci_dev *pdev = to_pci_dev(vdev->dev.parent);
+	struct pci_dev *pdev = to_pci_dev(dev->dev);
 	const char *pname = dev_name(&pdev->dev);
 	bool vga = (pdev->class >> 8) == PCI_CLASS_DISPLAY_VGA;
-	char unique[20];
 	int ret;
 
 	DRM_INFO("pci: %s detected at %s\n",
@@ -63,39 +62,7 @@ static int virtio_gpu_pci_quirk(struct drm_device *dev, struct virtio_device *vd
 			return ret;
 	}
 
-	/*
-	 * Normally the drm_dev_set_unique() call is done by core DRM.
-	 * The following comment covers, why virtio cannot rely on it.
-	 *
-	 * Unlike the other virtual GPU drivers, virtio abstracts the
-	 * underlying bus type by using struct virtio_device.
-	 *
-	 * Hence the dev_is_pci() check, used in core DRM, will fail
-	 * and the unique returned will be the virtio_device "virtio0",
-	 * while a "pci:..." one is required.
-	 *
-	 * A few other ideas were considered:
-	 * - Extend the dev_is_pci() check [in drm_set_busid] to
-	 *   consider virtio.
-	 *   Seems like a bigger hack than what we have already.
-	 *
-	 * - Point drm_device::dev to the parent of the virtio_device
-	 *   Semantic changes:
-	 *   * Using the wrong device for i2c, framebuffer_alloc and
-	 *     prime import.
-	 *   Visual changes:
-	 *   * Helpers such as DRM_DEV_ERROR, dev_info, drm_printer,
-	 *     will print the wrong information.
-	 *
-	 * We could address the latter issues, by introducing
-	 * drm_device::bus_dev, ... which would be used solely for this.
-	 *
-	 * So for the moment keep things as-is, with a bulky comment
-	 * for the next person who feels like removing this
-	 * drm_dev_set_unique() quirk.
-	 */
-	snprintf(unique, sizeof(unique), "pci:%s", pname);
-	return drm_dev_set_unique(dev, unique);
+	return 0;
 }
 
 static int virtio_gpu_probe(struct virtio_device *vdev)
@@ -109,18 +76,24 @@ static int virtio_gpu_probe(struct virtio_device *vdev)
 	if (virtio_gpu_modeset == 0)
 		return -EINVAL;
 
-	dev = drm_dev_alloc(&driver, &vdev->dev);
+	/*
+	 * The virtio-gpu device is a virtual device that doesn't have DMA
+	 * ops assigned to it, nor DMA mask set and etc. Its parent device
+	 * is actual GPU device we want to use it for the DRM's device in
+	 * order to benefit from using generic DRM APIs.
+	 */
+	dev = drm_dev_alloc(&driver, vdev->dev.parent);
 	if (IS_ERR(dev))
 		return PTR_ERR(dev);
 	vdev->priv = dev;
 
 	if (!strcmp(vdev->dev.parent->bus->name, "pci")) {
-		ret = virtio_gpu_pci_quirk(dev, vdev);
+		ret = virtio_gpu_pci_quirk(dev);
 		if (ret)
 			goto err_free;
 	}
 
-	ret = virtio_gpu_init(dev);
+	ret = virtio_gpu_init(vdev, dev);
 	if (ret)
 		goto err_free;
 
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
index 0a194aaad419..b2d93cb12ebf 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -100,8 +100,6 @@ struct virtio_gpu_object {
 
 struct virtio_gpu_object_shmem {
 	struct virtio_gpu_object base;
-	struct sg_table *pages;
-	uint32_t mapped;
 };
 
 struct virtio_gpu_object_vram {
@@ -214,7 +212,6 @@ struct virtio_gpu_drv_cap_cache {
 };
 
 struct virtio_gpu_device {
-	struct device *dev;
 	struct drm_device *ddev;
 
 	struct virtio_device *vdev;
@@ -282,7 +279,7 @@ extern struct drm_ioctl_desc virtio_gpu_ioctls[DRM_VIRTIO_NUM_IOCTLS];
 void virtio_gpu_create_context(struct drm_device *dev, struct drm_file *file);
 
 /* virtgpu_kms.c */
-int virtio_gpu_init(struct drm_device *dev);
+int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev);
 void virtio_gpu_deinit(struct drm_device *dev);
 void virtio_gpu_release(struct drm_device *dev);
 int virtio_gpu_driver_open(struct drm_device *dev, struct drm_file *file);
diff --git a/drivers/gpu/drm/virtio/virtgpu_kms.c b/drivers/gpu/drm/virtio/virtgpu_kms.c
index 3313b92db531..0d1e3eb61bee 100644
--- a/drivers/gpu/drm/virtio/virtgpu_kms.c
+++ b/drivers/gpu/drm/virtio/virtgpu_kms.c
@@ -110,7 +110,7 @@ static void virtio_gpu_get_capsets(struct virtio_gpu_device *vgdev,
 	vgdev->num_capsets = num_capsets;
 }
 
-int virtio_gpu_init(struct drm_device *dev)
+int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)
 {
 	static vq_callback_t *callbacks[] = {
 		virtio_gpu_ctrl_ack, virtio_gpu_cursor_ack
@@ -123,7 +123,7 @@ int virtio_gpu_init(struct drm_device *dev)
 	u32 num_scanouts, num_capsets;
 	int ret = 0;
 
-	if (!virtio_has_feature(dev_to_virtio(dev->dev), VIRTIO_F_VERSION_1))
+	if (!virtio_has_feature(vdev, VIRTIO_F_VERSION_1))
 		return -ENODEV;
 
 	vgdev = kzalloc(sizeof(struct virtio_gpu_device), GFP_KERNEL);
@@ -132,8 +132,7 @@ int virtio_gpu_init(struct drm_device *dev)
 
 	vgdev->ddev = dev;
 	dev->dev_private = vgdev;
-	vgdev->vdev = dev_to_virtio(dev->dev);
-	vgdev->dev = dev->dev;
+	vgdev->vdev = vdev;
 
 	spin_lock_init(&vgdev->display_info_lock);
 	spin_lock_init(&vgdev->resource_export_lock);
diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c b/drivers/gpu/drm/virtio/virtgpu_object.c
index 18f70ef6b4d0..8d7728181de0 100644
--- a/drivers/gpu/drm/virtio/virtgpu_object.c
+++ b/drivers/gpu/drm/virtio/virtgpu_object.c
@@ -67,21 +67,6 @@ void virtio_gpu_cleanup_object(struct virtio_gpu_object *bo)
 
 	virtio_gpu_resource_id_put(vgdev, bo->hw_res_handle);
 	if (virtio_gpu_is_shmem(bo)) {
-		struct virtio_gpu_object_shmem *shmem = to_virtio_gpu_shmem(bo);
-
-		if (shmem->pages) {
-			if (shmem->mapped) {
-				dma_unmap_sgtable(vgdev->vdev->dev.parent,
-					     shmem->pages, DMA_TO_DEVICE, 0);
-				shmem->mapped = 0;
-			}
-
-			sg_free_table(shmem->pages);
-			kfree(shmem->pages);
-			shmem->pages = NULL;
-			drm_gem_shmem_unpin(&bo->base);
-		}
-
 		drm_gem_shmem_free(&bo->base);
 	} else if (virtio_gpu_is_vram(bo)) {
 		struct virtio_gpu_object_vram *vram = to_virtio_gpu_vram(bo);
@@ -153,37 +138,18 @@ static int virtio_gpu_object_shmem_init(struct virtio_gpu_device *vgdev,
 					unsigned int *nents)
 {
 	bool use_dma_api = !virtio_has_dma_quirk(vgdev->vdev);
-	struct virtio_gpu_object_shmem *shmem = to_virtio_gpu_shmem(bo);
 	struct scatterlist *sg;
-	int si, ret;
+	struct sg_table *pages;
+	int si;
 
-	ret = drm_gem_shmem_pin(&bo->base);
-	if (ret < 0)
-		return -EINVAL;
-
-	/*
-	 * virtio_gpu uses drm_gem_shmem_get_sg_table instead of
-	 * drm_gem_shmem_get_pages_sgt because virtio has it's own set of
-	 * dma-ops. This is discouraged for other drivers, but should be fine
-	 * since virtio_gpu doesn't support dma-buf import from other devices.
-	 */
-	shmem->pages = drm_gem_shmem_get_sg_table(&bo->base);
-	ret = PTR_ERR_OR_ZERO(shmem->pages);
-	if (ret) {
-		drm_gem_shmem_unpin(&bo->base);
-		shmem->pages = NULL;
-		return ret;
-	}
+	pages = drm_gem_shmem_get_pages_sgt(&bo->base);
+	if (IS_ERR(pages))
+		return PTR_ERR(pages);
 
-	if (use_dma_api) {
-		ret = dma_map_sgtable(vgdev->vdev->dev.parent,
-				      shmem->pages, DMA_TO_DEVICE, 0);
-		if (ret)
-			return ret;
-		*nents = shmem->mapped = shmem->pages->nents;
-	} else {
-		*nents = shmem->pages->orig_nents;
-	}
+	if (use_dma_api)
+		*nents = pages->nents;
+	else
+		*nents = pages->orig_nents;
 
 	*ents = kvmalloc_array(*nents,
 			       sizeof(struct virtio_gpu_mem_entry),
@@ -194,13 +160,13 @@ static int virtio_gpu_object_shmem_init(struct virtio_gpu_device *vgdev,
 	}
 
 	if (use_dma_api) {
-		for_each_sgtable_dma_sg(shmem->pages, sg, si) {
+		for_each_sgtable_dma_sg(pages, sg, si) {
 			(*ents)[si].addr = cpu_to_le64(sg_dma_address(sg));
 			(*ents)[si].length = cpu_to_le32(sg_dma_len(sg));
 			(*ents)[si].padding = 0;
 		}
 	} else {
-		for_each_sgtable_sg(shmem->pages, sg, si) {
+		for_each_sgtable_sg(pages, sg, si) {
 			(*ents)[si].addr = cpu_to_le64(sg_phys(sg));
 			(*ents)[si].length = cpu_to_le32(sg->length);
 			(*ents)[si].padding = 0;
diff --git a/drivers/gpu/drm/virtio/virtgpu_vq.c b/drivers/gpu/drm/virtio/virtgpu_vq.c
index 2edf31806b74..06566e44307d 100644
--- a/drivers/gpu/drm/virtio/virtgpu_vq.c
+++ b/drivers/gpu/drm/virtio/virtgpu_vq.c
@@ -593,11 +593,10 @@ void virtio_gpu_cmd_transfer_to_host_2d(struct virtio_gpu_device *vgdev,
 	struct virtio_gpu_transfer_to_host_2d *cmd_p;
 	struct virtio_gpu_vbuffer *vbuf;
 	bool use_dma_api = !virtio_has_dma_quirk(vgdev->vdev);
-	struct virtio_gpu_object_shmem *shmem = to_virtio_gpu_shmem(bo);
 
 	if (virtio_gpu_is_shmem(bo) && use_dma_api)
-		dma_sync_sgtable_for_device(vgdev->vdev->dev.parent,
-					    shmem->pages, DMA_TO_DEVICE);
+		dma_sync_sgtable_for_device(&vgdev->vdev->dev,
+					    bo->base.sgt, DMA_TO_DEVICE);
 
 	cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
 	memset(cmd_p, 0, sizeof(*cmd_p));
@@ -1017,11 +1016,9 @@ void virtio_gpu_cmd_transfer_to_host_3d(struct virtio_gpu_device *vgdev,
 	struct virtio_gpu_vbuffer *vbuf;
 	bool use_dma_api = !virtio_has_dma_quirk(vgdev->vdev);
 
-	if (virtio_gpu_is_shmem(bo) && use_dma_api) {
-		struct virtio_gpu_object_shmem *shmem = to_virtio_gpu_shmem(bo);
-		dma_sync_sgtable_for_device(vgdev->vdev->dev.parent,
-					    shmem->pages, DMA_TO_DEVICE);
-	}
+	if (virtio_gpu_is_shmem(bo) && use_dma_api)
+		dma_sync_sgtable_for_device(&vgdev->vdev->dev,
+					    bo->base.sgt, DMA_TO_DEVICE);
 
 	cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
 	memset(cmd_p, 0, sizeof(*cmd_p));
-- 
2.35.3

[PATCH v6 13/22] drm/virtio: Improve DMA API usage for shmem BOs

[Dmitry Osipenko]

DRM API requires the DRM's driver to be backed with the device that can
be used for generic DMA operations. The VirtIO-GPU device can't perform
DMA operations if it uses PCI transport because PCI device driver creates
a virtual VirtIO-GPU device that isn't associated with the PCI. Use PCI's
GPU device for the DRM's device instead of the VirtIO-GPU device and drop
DMA-related hacks from the VirtIO-GPU driver.

Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_drv.c    | 51 ++++++----------------
 drivers/gpu/drm/virtio/virtgpu_drv.h    |  5 +--
 drivers/gpu/drm/virtio/virtgpu_kms.c    |  7 ++--
 drivers/gpu/drm/virtio/virtgpu_object.c | 56 +++++--------------------
 drivers/gpu/drm/virtio/virtgpu_vq.c     | 13 +++---
 5 files changed, 32 insertions(+), 100 deletions(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.c b/drivers/gpu/drm/virtio/virtgpu_drv.c
index 5f25a8d15464..0141b7df97ec 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.c
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.c
@@ -46,12 +46,11 @@ static int virtio_gpu_modeset = -1;
 MODULE_PARM_DESC(modeset, "Disable/Enable modesetting");
 module_param_named(modeset, virtio_gpu_modeset, int, 0400);
 
-static int virtio_gpu_pci_quirk(struct drm_device *dev, struct virtio_device *vdev)
+static int virtio_gpu_pci_quirk(struct drm_device *dev)
 {
-	struct pci_dev *pdev = to_pci_dev(vdev->dev.parent);
+	struct pci_dev *pdev = to_pci_dev(dev->dev);
 	const char *pname = dev_name(&pdev->dev);
 	bool vga = (pdev->class >> 8) == PCI_CLASS_DISPLAY_VGA;
-	char unique[20];
 	int ret;
 
 	DRM_INFO("pci: %s detected at %s\n",
@@ -63,39 +62,7 @@ static int virtio_gpu_pci_quirk(struct drm_device *dev, struct virtio_device *vd
 			return ret;
 	}
 
-	/*
-	 * Normally the drm_dev_set_unique() call is done by core DRM.
-	 * The following comment covers, why virtio cannot rely on it.
-	 *
-	 * Unlike the other virtual GPU drivers, virtio abstracts the
-	 * underlying bus type by using struct virtio_device.
-	 *
-	 * Hence the dev_is_pci() check, used in core DRM, will fail
-	 * and the unique returned will be the virtio_device "virtio0",
-	 * while a "pci:..." one is required.
-	 *
-	 * A few other ideas were considered:
-	 * - Extend the dev_is_pci() check [in drm_set_busid] to
-	 *   consider virtio.
-	 *   Seems like a bigger hack than what we have already.
-	 *
-	 * - Point drm_device::dev to the parent of the virtio_device
-	 *   Semantic changes:
-	 *   * Using the wrong device for i2c, framebuffer_alloc and
-	 *     prime import.
-	 *   Visual changes:
-	 *   * Helpers such as DRM_DEV_ERROR, dev_info, drm_printer,
-	 *     will print the wrong information.
-	 *
-	 * We could address the latter issues, by introducing
-	 * drm_device::bus_dev, ... which would be used solely for this.
-	 *
-	 * So for the moment keep things as-is, with a bulky comment
-	 * for the next person who feels like removing this
-	 * drm_dev_set_unique() quirk.
-	 */
-	snprintf(unique, sizeof(unique), "pci:%s", pname);
-	return drm_dev_set_unique(dev, unique);
+	return 0;
 }
 
 static int virtio_gpu_probe(struct virtio_device *vdev)
@@ -109,18 +76,24 @@ static int virtio_gpu_probe(struct virtio_device *vdev)
 	if (virtio_gpu_modeset == 0)
 		return -EINVAL;
 
-	dev = drm_dev_alloc(&driver, &vdev->dev);
+	/*
+	 * The virtio-gpu device is a virtual device that doesn't have DMA
+	 * ops assigned to it, nor DMA mask set and etc. Its parent device
+	 * is actual GPU device we want to use it for the DRM's device in
+	 * order to benefit from using generic DRM APIs.
+	 */
+	dev = drm_dev_alloc(&driver, vdev->dev.parent);
 	if (IS_ERR(dev))
 		return PTR_ERR(dev);
 	vdev->priv = dev;
 
 	if (!strcmp(vdev->dev.parent->bus->name, "pci")) {
-		ret = virtio_gpu_pci_quirk(dev, vdev);
+		ret = virtio_gpu_pci_quirk(dev);
 		if (ret)
 			goto err_free;
 	}
 
-	ret = virtio_gpu_init(dev);
+	ret = virtio_gpu_init(vdev, dev);
 	if (ret)
 		goto err_free;
 
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
index 0a194aaad419..b2d93cb12ebf 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -100,8 +100,6 @@ struct virtio_gpu_object {
 
 struct virtio_gpu_object_shmem {
 	struct virtio_gpu_object base;
-	struct sg_table *pages;
-	uint32_t mapped;
 };
 
 struct virtio_gpu_object_vram {
@@ -214,7 +212,6 @@ struct virtio_gpu_drv_cap_cache {
 };
 
 struct virtio_gpu_device {
-	struct device *dev;
 	struct drm_device *ddev;
 
 	struct virtio_device *vdev;
@@ -282,7 +279,7 @@ extern struct drm_ioctl_desc virtio_gpu_ioctls[DRM_VIRTIO_NUM_IOCTLS];
 void virtio_gpu_create_context(struct drm_device *dev, struct drm_file *file);
 
 /* virtgpu_kms.c */
-int virtio_gpu_init(struct drm_device *dev);
+int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev);
 void virtio_gpu_deinit(struct drm_device *dev);
 void virtio_gpu_release(struct drm_device *dev);
 int virtio_gpu_driver_open(struct drm_device *dev, struct drm_file *file);
diff --git a/drivers/gpu/drm/virtio/virtgpu_kms.c b/drivers/gpu/drm/virtio/virtgpu_kms.c
index 3313b92db531..0d1e3eb61bee 100644
--- a/drivers/gpu/drm/virtio/virtgpu_kms.c
+++ b/drivers/gpu/drm/virtio/virtgpu_kms.c
@@ -110,7 +110,7 @@ static void virtio_gpu_get_capsets(struct virtio_gpu_device *vgdev,
 	vgdev->num_capsets = num_capsets;
 }
 
-int virtio_gpu_init(struct drm_device *dev)
+int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)
 {
 	static vq_callback_t *callbacks[] = {
 		virtio_gpu_ctrl_ack, virtio_gpu_cursor_ack
@@ -123,7 +123,7 @@ int virtio_gpu_init(struct drm_device *dev)
 	u32 num_scanouts, num_capsets;
 	int ret = 0;
 
-	if (!virtio_has_feature(dev_to_virtio(dev->dev), VIRTIO_F_VERSION_1))
+	if (!virtio_has_feature(vdev, VIRTIO_F_VERSION_1))
 		return -ENODEV;
 
 	vgdev = kzalloc(sizeof(struct virtio_gpu_device), GFP_KERNEL);
@@ -132,8 +132,7 @@ int virtio_gpu_init(struct drm_device *dev)
 
 	vgdev->ddev = dev;
 	dev->dev_private = vgdev;
-	vgdev->vdev = dev_to_virtio(dev->dev);
-	vgdev->dev = dev->dev;
+	vgdev->vdev = vdev;
 
 	spin_lock_init(&vgdev->display_info_lock);
 	spin_lock_init(&vgdev->resource_export_lock);
diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c b/drivers/gpu/drm/virtio/virtgpu_object.c
index 18f70ef6b4d0..8d7728181de0 100644
--- a/drivers/gpu/drm/virtio/virtgpu_object.c
+++ b/drivers/gpu/drm/virtio/virtgpu_object.c
@@ -67,21 +67,6 @@ void virtio_gpu_cleanup_object(struct virtio_gpu_object *bo)
 
 	virtio_gpu_resource_id_put(vgdev, bo->hw_res_handle);
 	if (virtio_gpu_is_shmem(bo)) {
-		struct virtio_gpu_object_shmem *shmem = to_virtio_gpu_shmem(bo);
-
-		if (shmem->pages) {
-			if (shmem->mapped) {
-				dma_unmap_sgtable(vgdev->vdev->dev.parent,
-					     shmem->pages, DMA_TO_DEVICE, 0);
-				shmem->mapped = 0;
-			}
-
-			sg_free_table(shmem->pages);
-			kfree(shmem->pages);
-			shmem->pages = NULL;
-			drm_gem_shmem_unpin(&bo->base);
-		}
-
 		drm_gem_shmem_free(&bo->base);
 	} else if (virtio_gpu_is_vram(bo)) {
 		struct virtio_gpu_object_vram *vram = to_virtio_gpu_vram(bo);
@@ -153,37 +138,18 @@ static int virtio_gpu_object_shmem_init(struct virtio_gpu_device *vgdev,
 					unsigned int *nents)
 {
 	bool use_dma_api = !virtio_has_dma_quirk(vgdev->vdev);
-	struct virtio_gpu_object_shmem *shmem = to_virtio_gpu_shmem(bo);
 	struct scatterlist *sg;
-	int si, ret;
+	struct sg_table *pages;
+	int si;
 
-	ret = drm_gem_shmem_pin(&bo->base);
-	if (ret < 0)
-		return -EINVAL;
-
-	/*
-	 * virtio_gpu uses drm_gem_shmem_get_sg_table instead of
-	 * drm_gem_shmem_get_pages_sgt because virtio has it's own set of
-	 * dma-ops. This is discouraged for other drivers, but should be fine
-	 * since virtio_gpu doesn't support dma-buf import from other devices.
-	 */
-	shmem->pages = drm_gem_shmem_get_sg_table(&bo->base);
-	ret = PTR_ERR_OR_ZERO(shmem->pages);
-	if (ret) {
-		drm_gem_shmem_unpin(&bo->base);
-		shmem->pages = NULL;
-		return ret;
-	}
+	pages = drm_gem_shmem_get_pages_sgt(&bo->base);
+	if (IS_ERR(pages))
+		return PTR_ERR(pages);
 
-	if (use_dma_api) {
-		ret = dma_map_sgtable(vgdev->vdev->dev.parent,
-				      shmem->pages, DMA_TO_DEVICE, 0);
-		if (ret)
-			return ret;
-		*nents = shmem->mapped = shmem->pages->nents;
-	} else {
-		*nents = shmem->pages->orig_nents;
-	}
+	if (use_dma_api)
+		*nents = pages->nents;
+	else
+		*nents = pages->orig_nents;
 
 	*ents = kvmalloc_array(*nents,
 			       sizeof(struct virtio_gpu_mem_entry),
@@ -194,13 +160,13 @@ static int virtio_gpu_object_shmem_init(struct virtio_gpu_device *vgdev,
 	}
 
 	if (use_dma_api) {
-		for_each_sgtable_dma_sg(shmem->pages, sg, si) {
+		for_each_sgtable_dma_sg(pages, sg, si) {
 			(*ents)[si].addr = cpu_to_le64(sg_dma_address(sg));
 			(*ents)[si].length = cpu_to_le32(sg_dma_len(sg));
 			(*ents)[si].padding = 0;
 		}
 	} else {
-		for_each_sgtable_sg(shmem->pages, sg, si) {
+		for_each_sgtable_sg(pages, sg, si) {
 			(*ents)[si].addr = cpu_to_le64(sg_phys(sg));
 			(*ents)[si].length = cpu_to_le32(sg->length);
 			(*ents)[si].padding = 0;
diff --git a/drivers/gpu/drm/virtio/virtgpu_vq.c b/drivers/gpu/drm/virtio/virtgpu_vq.c
index 2edf31806b74..06566e44307d 100644
--- a/drivers/gpu/drm/virtio/virtgpu_vq.c
+++ b/drivers/gpu/drm/virtio/virtgpu_vq.c
@@ -593,11 +593,10 @@ void virtio_gpu_cmd_transfer_to_host_2d(struct virtio_gpu_device *vgdev,
 	struct virtio_gpu_transfer_to_host_2d *cmd_p;
 	struct virtio_gpu_vbuffer *vbuf;
 	bool use_dma_api = !virtio_has_dma_quirk(vgdev->vdev);
-	struct virtio_gpu_object_shmem *shmem = to_virtio_gpu_shmem(bo);
 
 	if (virtio_gpu_is_shmem(bo) && use_dma_api)
-		dma_sync_sgtable_for_device(vgdev->vdev->dev.parent,
-					    shmem->pages, DMA_TO_DEVICE);
+		dma_sync_sgtable_for_device(&vgdev->vdev->dev,
+					    bo->base.sgt, DMA_TO_DEVICE);
 
 	cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
 	memset(cmd_p, 0, sizeof(*cmd_p));
@@ -1017,11 +1016,9 @@ void virtio_gpu_cmd_transfer_to_host_3d(struct virtio_gpu_device *vgdev,
 	struct virtio_gpu_vbuffer *vbuf;
 	bool use_dma_api = !virtio_has_dma_quirk(vgdev->vdev);
 
-	if (virtio_gpu_is_shmem(bo) && use_dma_api) {
-		struct virtio_gpu_object_shmem *shmem = to_virtio_gpu_shmem(bo);
-		dma_sync_sgtable_for_device(vgdev->vdev->dev.parent,
-					    shmem->pages, DMA_TO_DEVICE);
-	}
+	if (virtio_gpu_is_shmem(bo) && use_dma_api)
+		dma_sync_sgtable_for_device(&vgdev->vdev->dev,
+					    bo->base.sgt, DMA_TO_DEVICE);
 
 	cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
 	memset(cmd_p, 0, sizeof(*cmd_p));
-- 
2.35.3

[PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dmitry Osipenko]

All dma-bufs have dma-reservation lock that allows drivers to perform
exclusive operations over shared dma-bufs. Today's dma-buf API has
incomplete locking specification, which creates dead lock situation
for dma-buf importers and exporters that don't coordinate theirs locks.

This patch introduces new locking convention for dma-buf users. From now
on all dma-buf importers are responsible for holding dma-buf reservation
lock around operations performed over dma-bufs.

This patch implements the new dma-buf locking convention by:

  1. Making dma-buf API functions to take the reservation lock.

  2. Adding new locked variants of the dma-buf API functions for drivers
     that need to manage imported dma-bufs under the held lock.

  3. Converting all drivers to the new locking scheme.

Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/dma-buf/dma-buf.c                     | 270 +++++++++++-------
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
 drivers/gpu/drm/drm_client.c                  |   4 +-
 drivers/gpu/drm/drm_gem.c                     |  33 +++
 drivers/gpu/drm/drm_gem_framebuffer_helper.c  |   6 +-
 drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    |  10 +-
 drivers/gpu/drm/qxl/qxl_object.c              |  17 +-
 drivers/gpu/drm/qxl/qxl_prime.c               |   4 +-
 .../common/videobuf2/videobuf2-dma-contig.c   |  11 +-
 .../media/common/videobuf2/videobuf2-dma-sg.c |  11 +-
 .../common/videobuf2/videobuf2-vmalloc.c      |  11 +-
 include/drm/drm_gem.h                         |   3 +
 include/linux/dma-buf.h                       |  14 +-
 13 files changed, 241 insertions(+), 159 deletions(-)

diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
index 32f55640890c..64a9909ccfa2 100644
--- a/drivers/dma-buf/dma-buf.c
+++ b/drivers/dma-buf/dma-buf.c
@@ -552,7 +552,6 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)
 	file->f_mode |= FMODE_LSEEK;
 	dmabuf->file = file;
 
-	mutex_init(&dmabuf->lock);
 	INIT_LIST_HEAD(&dmabuf->attachments);
 
 	mutex_lock(&db_list.lock);
@@ -737,14 +736,14 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
 	attach->importer_ops = importer_ops;
 	attach->importer_priv = importer_priv;
 
+	dma_resv_lock(dmabuf->resv, NULL);
+
 	if (dmabuf->ops->attach) {
 		ret = dmabuf->ops->attach(dmabuf, attach);
 		if (ret)
 			goto err_attach;
 	}
-	dma_resv_lock(dmabuf->resv, NULL);
 	list_add(&attach->node, &dmabuf->attachments);
-	dma_resv_unlock(dmabuf->resv);
 
 	/* When either the importer or the exporter can't handle dynamic
 	 * mappings we cache the mapping here to avoid issues with the
@@ -755,7 +754,6 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
 		struct sg_table *sgt;
 
 		if (dma_buf_is_dynamic(attach->dmabuf)) {
-			dma_resv_lock(attach->dmabuf->resv, NULL);
 			ret = dmabuf->ops->pin(attach);
 			if (ret)
 				goto err_unlock;
@@ -768,15 +766,16 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
 			ret = PTR_ERR(sgt);
 			goto err_unpin;
 		}
-		if (dma_buf_is_dynamic(attach->dmabuf))
-			dma_resv_unlock(attach->dmabuf->resv);
 		attach->sgt = sgt;
 		attach->dir = DMA_BIDIRECTIONAL;
 	}
 
+	dma_resv_unlock(dmabuf->resv);
+
 	return attach;
 
 err_attach:
+	dma_resv_unlock(attach->dmabuf->resv);
 	kfree(attach);
 	return ERR_PTR(ret);
 
@@ -785,10 +784,10 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
 		dmabuf->ops->unpin(attach);
 
 err_unlock:
-	if (dma_buf_is_dynamic(attach->dmabuf))
-		dma_resv_unlock(attach->dmabuf->resv);
+	dma_resv_unlock(dmabuf->resv);
 
 	dma_buf_detach(dmabuf, attach);
+
 	return ERR_PTR(ret);
 }
 EXPORT_SYMBOL_NS_GPL(dma_buf_dynamic_attach, DMA_BUF);
@@ -832,24 +831,23 @@ void dma_buf_detach(struct dma_buf *dmabuf, struct dma_buf_attachment *attach)
 	if (WARN_ON(!dmabuf || !attach))
 		return;
 
-	if (attach->sgt) {
-		if (dma_buf_is_dynamic(attach->dmabuf))
-			dma_resv_lock(attach->dmabuf->resv, NULL);
+	if (WARN_ON(dmabuf != attach->dmabuf))
+		return;
 
+	dma_resv_lock(dmabuf->resv, NULL);
+
+	if (attach->sgt) {
 		__unmap_dma_buf(attach, attach->sgt, attach->dir);
 
-		if (dma_buf_is_dynamic(attach->dmabuf)) {
+		if (dma_buf_is_dynamic(attach->dmabuf))
 			dmabuf->ops->unpin(attach);
-			dma_resv_unlock(attach->dmabuf->resv);
-		}
 	}
 
-	dma_resv_lock(dmabuf->resv, NULL);
 	list_del(&attach->node);
-	dma_resv_unlock(dmabuf->resv);
 	if (dmabuf->ops->detach)
 		dmabuf->ops->detach(dmabuf, attach);
 
+	dma_resv_unlock(dmabuf->resv);
 	kfree(attach);
 }
 EXPORT_SYMBOL_NS_GPL(dma_buf_detach, DMA_BUF);
@@ -906,28 +904,18 @@ void dma_buf_unpin(struct dma_buf_attachment *attach)
 EXPORT_SYMBOL_NS_GPL(dma_buf_unpin, DMA_BUF);
 
 /**
- * dma_buf_map_attachment - Returns the scatterlist table of the attachment;
+ * dma_buf_map_attachment_locked - Returns the scatterlist table of the attachment;
  * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
  * dma_buf_ops.
  * @attach:	[in]	attachment whose scatterlist is to be returned
  * @direction:	[in]	direction of DMA transfer
  *
- * Returns sg_table containing the scatterlist to be returned; returns ERR_PTR
- * on error. May return -EINTR if it is interrupted by a signal.
- *
- * On success, the DMA addresses and lengths in the returned scatterlist are
- * PAGE_SIZE aligned.
- *
- * A mapping must be unmapped by using dma_buf_unmap_attachment(). Note that
- * the underlying backing storage is pinned for as long as a mapping exists,
- * therefore users/importers should not hold onto a mapping for undue amounts of
- * time.
+ * Locked variant of dma_buf_map_attachment().
  *
- * Important: Dynamic importers must wait for the exclusive fence of the struct
- * dma_resv attached to the DMA-BUF first.
+ * Caller is responsible for holding dmabuf's reservation lock.
  */
-struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
-					enum dma_data_direction direction)
+struct sg_table *dma_buf_map_attachment_locked(struct dma_buf_attachment *attach,
+					       enum dma_data_direction direction)
 {
 	struct sg_table *sg_table;
 	int r;
@@ -937,8 +925,7 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
 	if (WARN_ON(!attach || !attach->dmabuf))
 		return ERR_PTR(-EINVAL);
 
-	if (dma_buf_attachment_is_dynamic(attach))
-		dma_resv_assert_held(attach->dmabuf->resv);
+	dma_resv_assert_held(attach->dmabuf->resv);
 
 	if (attach->sgt) {
 		/*
@@ -953,7 +940,6 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
 	}
 
 	if (dma_buf_is_dynamic(attach->dmabuf)) {
-		dma_resv_assert_held(attach->dmabuf->resv);
 		if (!IS_ENABLED(CONFIG_DMABUF_MOVE_NOTIFY)) {
 			r = attach->dmabuf->ops->pin(attach);
 			if (r)
@@ -993,42 +979,101 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
 #endif /* CONFIG_DMA_API_DEBUG */
 	return sg_table;
 }
-EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment, DMA_BUF);
+EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment_locked, DMA_BUF);
 
 /**
- * dma_buf_unmap_attachment - unmaps and decreases usecount of the buffer;might
- * deallocate the scatterlist associated. Is a wrapper for unmap_dma_buf() of
+ * dma_buf_map_attachment - Returns the scatterlist table of the attachment;
+ * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
  * dma_buf_ops.
- * @attach:	[in]	attachment to unmap buffer from
- * @sg_table:	[in]	scatterlist info of the buffer to unmap
- * @direction:  [in]    direction of DMA transfer
+ * @attach:	[in]	attachment whose scatterlist is to be returned
+ * @direction:	[in]	direction of DMA transfer
  *
- * This unmaps a DMA mapping for @attached obtained by dma_buf_map_attachment().
+ * Returns sg_table containing the scatterlist to be returned; returns ERR_PTR
+ * on error. May return -EINTR if it is interrupted by a signal.
+ *
+ * On success, the DMA addresses and lengths in the returned scatterlist are
+ * PAGE_SIZE aligned.
+ *
+ * A mapping must be unmapped by using dma_buf_unmap_attachment(). Note that
+ * the underlying backing storage is pinned for as long as a mapping exists,
+ * therefore users/importers should not hold onto a mapping for undue amounts of
+ * time.
+ *
+ * Important: Dynamic importers must wait for the exclusive fence of the struct
+ * dma_resv attached to the DMA-BUF first.
  */
-void dma_buf_unmap_attachment(struct dma_buf_attachment *attach,
-				struct sg_table *sg_table,
+struct sg_table *
+dma_buf_map_attachment(struct dma_buf_attachment *attach,
 				enum dma_data_direction direction)
 {
+	struct sg_table *sg_table;
+
 	might_sleep();
 
-	if (WARN_ON(!attach || !attach->dmabuf || !sg_table))
-		return;
+	if (WARN_ON(!attach || !attach->dmabuf))
+		return ERR_PTR(-EINVAL);
+
+	dma_resv_lock(attach->dmabuf->resv, NULL);
+	sg_table = dma_buf_map_attachment_locked(attach, direction);
+	dma_resv_unlock(attach->dmabuf->resv);
 
-	if (dma_buf_attachment_is_dynamic(attach))
-		dma_resv_assert_held(attach->dmabuf->resv);
+	return sg_table;
+}
+EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment, DMA_BUF);
+
+/**
+ * dma_buf_unmap_attachment_locked - Returns the scatterlist table of the attachment;
+ * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
+ * dma_buf_ops.
+ * @attach:	[in]	attachment whose scatterlist is to be returned
+ * @direction:	[in]	direction of DMA transfer
+ *
+ * Locked variant of dma_buf_unmap_attachment().
+ *
+ * Caller is responsible for holding dmabuf's reservation lock.
+ */
+void dma_buf_unmap_attachment_locked(struct dma_buf_attachment *attach,
+				     struct sg_table *sg_table,
+				     enum dma_data_direction direction)
+{
+	might_sleep();
+
+	dma_resv_assert_held(attach->dmabuf->resv);
 
 	if (attach->sgt == sg_table)
 		return;
 
-	if (dma_buf_is_dynamic(attach->dmabuf))
-		dma_resv_assert_held(attach->dmabuf->resv);
-
 	__unmap_dma_buf(attach, sg_table, direction);
 
 	if (dma_buf_is_dynamic(attach->dmabuf) &&
 	    !IS_ENABLED(CONFIG_DMABUF_MOVE_NOTIFY))
 		dma_buf_unpin(attach);
 }
+EXPORT_SYMBOL_NS_GPL(dma_buf_unmap_attachment_locked, DMA_BUF);
+
+/**
+ * dma_buf_unmap_attachment - unmaps and decreases usecount of the buffer;might
+ * deallocate the scatterlist associated. Is a wrapper for unmap_dma_buf() of
+ * dma_buf_ops.
+ * @attach:	[in]	attachment to unmap buffer from
+ * @sg_table:	[in]	scatterlist info of the buffer to unmap
+ * @direction:  [in]    direction of DMA transfer
+ *
+ * This unmaps a DMA mapping for @attached obtained by dma_buf_map_attachment().
+ */
+void dma_buf_unmap_attachment(struct dma_buf_attachment *attach,
+			      struct sg_table *sg_table,
+			      enum dma_data_direction direction)
+{
+	might_sleep();
+
+	if (WARN_ON(!attach || !attach->dmabuf || !sg_table))
+		return;
+
+	dma_resv_lock(attach->dmabuf->resv, NULL);
+	dma_buf_unmap_attachment_locked(attach, sg_table, direction);
+	dma_resv_unlock(attach->dmabuf->resv);
+}
 EXPORT_SYMBOL_NS_GPL(dma_buf_unmap_attachment, DMA_BUF);
 
 /**
@@ -1224,6 +1269,31 @@ int dma_buf_end_cpu_access(struct dma_buf *dmabuf,
 }
 EXPORT_SYMBOL_NS_GPL(dma_buf_end_cpu_access, DMA_BUF);
 
+static int dma_buf_mmap_locked(struct dma_buf *dmabuf,
+			       struct vm_area_struct *vma,
+			       unsigned long pgoff)
+{
+	dma_resv_assert_held(dmabuf->resv);
+
+	/* check if buffer supports mmap */
+	if (!dmabuf->ops->mmap)
+		return -EINVAL;
+
+	/* check for offset overflow */
+	if (pgoff + vma_pages(vma) < pgoff)
+		return -EOVERFLOW;
+
+	/* check for overflowing the buffer's size */
+	if (pgoff + vma_pages(vma) >
+	    dmabuf->size >> PAGE_SHIFT)
+		return -EINVAL;
+
+	/* readjust the vma */
+	vma_set_file(vma, dmabuf->file);
+	vma->vm_pgoff = pgoff;
+
+	return dmabuf->ops->mmap(dmabuf, vma);
+}
 
 /**
  * dma_buf_mmap - Setup up a userspace mmap with the given vma
@@ -1242,29 +1312,46 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_end_cpu_access, DMA_BUF);
 int dma_buf_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma,
 		 unsigned long pgoff)
 {
+	int ret;
+
 	if (WARN_ON(!dmabuf || !vma))
 		return -EINVAL;
 
-	/* check if buffer supports mmap */
-	if (!dmabuf->ops->mmap)
-		return -EINVAL;
+	dma_resv_lock(dmabuf->resv, NULL);
+	ret = dma_buf_mmap_locked(dmabuf, vma, pgoff);
+	dma_resv_unlock(dmabuf->resv);
 
-	/* check for offset overflow */
-	if (pgoff + vma_pages(vma) < pgoff)
-		return -EOVERFLOW;
+	return ret;
+}
+EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
 
-	/* check for overflowing the buffer's size */
-	if (pgoff + vma_pages(vma) >
-	    dmabuf->size >> PAGE_SHIFT)
-		return -EINVAL;
+static int dma_buf_vmap_locked(struct dma_buf *dmabuf, struct iosys_map *map)
+{
+	struct iosys_map ptr;
+	int ret;
 
-	/* readjust the vma */
-	vma_set_file(vma, dmabuf->file);
-	vma->vm_pgoff = pgoff;
+	dma_resv_assert_held(dmabuf->resv);
 
-	return dmabuf->ops->mmap(dmabuf, vma);
+	if (dmabuf->vmapping_counter) {
+		dmabuf->vmapping_counter++;
+		BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
+		*map = dmabuf->vmap_ptr;
+		return ret;
+	}
+
+	BUG_ON(iosys_map_is_set(&dmabuf->vmap_ptr));
+
+	ret = dmabuf->ops->vmap(dmabuf, &ptr);
+	if (WARN_ON_ONCE(ret))
+		return ret;
+
+	dmabuf->vmap_ptr = ptr;
+	dmabuf->vmapping_counter = 1;
+
+	*map = dmabuf->vmap_ptr;
+
+	return 0;
 }
-EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
 
 /**
  * dma_buf_vmap - Create virtual mapping for the buffer object into kernel
@@ -1284,8 +1371,7 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
  */
 int dma_buf_vmap(struct dma_buf *dmabuf, struct iosys_map *map)
 {
-	struct iosys_map ptr;
-	int ret = 0;
+	int ret;
 
 	iosys_map_clear(map);
 
@@ -1295,52 +1381,40 @@ int dma_buf_vmap(struct dma_buf *dmabuf, struct iosys_map *map)
 	if (!dmabuf->ops->vmap)
 		return -EINVAL;
 
-	mutex_lock(&dmabuf->lock);
-	if (dmabuf->vmapping_counter) {
-		dmabuf->vmapping_counter++;
-		BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
-		*map = dmabuf->vmap_ptr;
-		goto out_unlock;
-	}
-
-	BUG_ON(iosys_map_is_set(&dmabuf->vmap_ptr));
-
-	ret = dmabuf->ops->vmap(dmabuf, &ptr);
-	if (WARN_ON_ONCE(ret))
-		goto out_unlock;
-
-	dmabuf->vmap_ptr = ptr;
-	dmabuf->vmapping_counter = 1;
-
-	*map = dmabuf->vmap_ptr;
+	dma_resv_lock(dmabuf->resv, NULL);
+	ret = dma_buf_vmap_locked(dmabuf, map);
+	dma_resv_unlock(dmabuf->resv);
 
-out_unlock:
-	mutex_unlock(&dmabuf->lock);
 	return ret;
 }
 EXPORT_SYMBOL_NS_GPL(dma_buf_vmap, DMA_BUF);
 
-/**
- * dma_buf_vunmap - Unmap a vmap obtained by dma_buf_vmap.
- * @dmabuf:	[in]	buffer to vunmap
- * @map:	[in]	vmap pointer to vunmap
- */
-void dma_buf_vunmap(struct dma_buf *dmabuf, struct iosys_map *map)
+static void dma_buf_vunmap_locked(struct dma_buf *dmabuf, struct iosys_map *map)
 {
-	if (WARN_ON(!dmabuf))
-		return;
-
 	BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
 	BUG_ON(dmabuf->vmapping_counter == 0);
 	BUG_ON(!iosys_map_is_equal(&dmabuf->vmap_ptr, map));
 
-	mutex_lock(&dmabuf->lock);
 	if (--dmabuf->vmapping_counter == 0) {
 		if (dmabuf->ops->vunmap)
 			dmabuf->ops->vunmap(dmabuf, map);
 		iosys_map_clear(&dmabuf->vmap_ptr);
 	}
-	mutex_unlock(&dmabuf->lock);
+}
+
+/**
+ * dma_buf_vunmap - Unmap a vmap obtained by dma_buf_vmap.
+ * @dmabuf:	[in]	buffer to vunmap
+ * @map:	[in]	vmap pointer to vunmap
+ */
+void dma_buf_vunmap(struct dma_buf *dmabuf, struct iosys_map *map)
+{
+	if (WARN_ON(!dmabuf))
+		return;
+
+	dma_resv_lock(dmabuf->resv, NULL);
+	dma_buf_vunmap_locked(dmabuf, map);
+	dma_resv_unlock(dmabuf->resv);
 }
 EXPORT_SYMBOL_NS_GPL(dma_buf_vunmap, DMA_BUF);
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index be6f76a30ac6..b704bdf5601a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -882,7 +882,8 @@ static int amdgpu_ttm_backend_bind(struct ttm_device *bdev,
 			struct sg_table *sgt;
 
 			attach = gtt->gobj->import_attach;
-			sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL);
+			sgt = dma_buf_map_attachment_locked(attach,
+							    DMA_BIDIRECTIONAL);
 			if (IS_ERR(sgt))
 				return PTR_ERR(sgt);
 
@@ -1007,7 +1008,8 @@ static void amdgpu_ttm_backend_unbind(struct ttm_device *bdev,
 		struct dma_buf_attachment *attach;
 
 		attach = gtt->gobj->import_attach;
-		dma_buf_unmap_attachment(attach, ttm->sg, DMA_BIDIRECTIONAL);
+		dma_buf_unmap_attachment_locked(attach, ttm->sg,
+						DMA_BIDIRECTIONAL);
 		ttm->sg = NULL;
 	}
 
diff --git a/drivers/gpu/drm/drm_client.c b/drivers/gpu/drm/drm_client.c
index af3b7395bf69..e9a1cd310352 100644
--- a/drivers/gpu/drm/drm_client.c
+++ b/drivers/gpu/drm/drm_client.c
@@ -323,7 +323,7 @@ drm_client_buffer_vmap(struct drm_client_buffer *buffer,
 	 * fd_install step out of the driver backend hooks, to make that
 	 * final step optional for internal users.
 	 */
-	ret = drm_gem_vmap(buffer->gem, map);
+	ret = drm_gem_vmap_unlocked(buffer->gem, map);
 	if (ret)
 		return ret;
 
@@ -345,7 +345,7 @@ void drm_client_buffer_vunmap(struct drm_client_buffer *buffer)
 {
 	struct iosys_map *map = &buffer->map;
 
-	drm_gem_vunmap(buffer->gem, map);
+	drm_gem_vunmap_unlocked(buffer->gem, map);
 }
 EXPORT_SYMBOL(drm_client_buffer_vunmap);
 
diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index 7c0b025508e4..c61674887582 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -1053,7 +1053,12 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,
 	vma->vm_ops = obj->funcs->vm_ops;
 
 	if (obj->funcs->mmap) {
+		ret = dma_resv_lock_interruptible(obj->resv, NULL);
+		if (ret)
+			goto err_drm_gem_object_put;
+
 		ret = obj->funcs->mmap(obj, vma);
+		dma_resv_unlock(obj->resv);
 		if (ret)
 			goto err_drm_gem_object_put;
 		WARN_ON(!(vma->vm_flags & VM_DONTEXPAND));
@@ -1158,6 +1163,8 @@ void drm_gem_print_info(struct drm_printer *p, unsigned int indent,
 
 int drm_gem_pin(struct drm_gem_object *obj)
 {
+	dma_resv_assert_held(obj->resv);
+
 	if (obj->funcs->pin)
 		return obj->funcs->pin(obj);
 	else
@@ -1166,6 +1173,8 @@ int drm_gem_pin(struct drm_gem_object *obj)
 
 void drm_gem_unpin(struct drm_gem_object *obj)
 {
+	dma_resv_assert_held(obj->resv);
+
 	if (obj->funcs->unpin)
 		obj->funcs->unpin(obj);
 }
@@ -1174,6 +1183,8 @@ int drm_gem_vmap(struct drm_gem_object *obj, struct iosys_map *map)
 {
 	int ret;
 
+	dma_resv_assert_held(obj->resv);
+
 	if (!obj->funcs->vmap)
 		return -EOPNOTSUPP;
 
@@ -1189,6 +1200,8 @@ EXPORT_SYMBOL(drm_gem_vmap);
 
 void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
 {
+	dma_resv_assert_held(obj->resv);
+
 	if (iosys_map_is_null(map))
 		return;
 
@@ -1200,6 +1213,26 @@ void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
 }
 EXPORT_SYMBOL(drm_gem_vunmap);
 
+int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
+{
+	int ret;
+
+	dma_resv_lock(obj->resv, NULL);
+	ret = drm_gem_vmap(obj, map);
+	dma_resv_unlock(obj->resv);
+
+	return ret;
+}
+EXPORT_SYMBOL(drm_gem_vmap_unlocked);
+
+void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
+{
+	dma_resv_lock(obj->resv, NULL);
+	drm_gem_vunmap(obj, map);
+	dma_resv_unlock(obj->resv);
+}
+EXPORT_SYMBOL(drm_gem_vunmap_unlocked);
+
 /**
  * drm_gem_lock_reservations - Sets up the ww context and acquires
  * the lock on an array of GEM objects.
diff --git a/drivers/gpu/drm/drm_gem_framebuffer_helper.c b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
index f4619803acd0..a0bff53b158e 100644
--- a/drivers/gpu/drm/drm_gem_framebuffer_helper.c
+++ b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
@@ -348,7 +348,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb,
 			iosys_map_clear(&map[i]);
 			continue;
 		}
-		ret = drm_gem_vmap(obj, &map[i]);
+		ret = drm_gem_vmap_unlocked(obj, &map[i]);
 		if (ret)
 			goto err_drm_gem_vunmap;
 	}
@@ -370,7 +370,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb,
 		obj = drm_gem_fb_get_obj(fb, i);
 		if (!obj)
 			continue;
-		drm_gem_vunmap(obj, &map[i]);
+		drm_gem_vunmap_unlocked(obj, &map[i]);
 	}
 	return ret;
 }
@@ -398,7 +398,7 @@ void drm_gem_fb_vunmap(struct drm_framebuffer *fb,
 			continue;
 		if (iosys_map_is_null(&map[i]))
 			continue;
-		drm_gem_vunmap(obj, &map[i]);
+		drm_gem_vunmap_unlocked(obj, &map[i]);
 	}
 }
 EXPORT_SYMBOL(drm_gem_fb_vunmap);
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
index f5062d0c6333..09502d490da8 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
@@ -72,7 +72,7 @@ static int i915_gem_dmabuf_vmap(struct dma_buf *dma_buf,
 	struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);
 	void *vaddr;
 
-	vaddr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
+	vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
 	if (IS_ERR(vaddr))
 		return PTR_ERR(vaddr);
 
@@ -241,8 +241,8 @@ static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
 
 	assert_object_held(obj);
 
-	pages = dma_buf_map_attachment(obj->base.import_attach,
-				       DMA_BIDIRECTIONAL);
+	pages = dma_buf_map_attachment_locked(obj->base.import_attach,
+					      DMA_BIDIRECTIONAL);
 	if (IS_ERR(pages))
 		return PTR_ERR(pages);
 
@@ -270,8 +270,8 @@ static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
 static void i915_gem_object_put_pages_dmabuf(struct drm_i915_gem_object *obj,
 					     struct sg_table *pages)
 {
-	dma_buf_unmap_attachment(obj->base.import_attach, pages,
-				 DMA_BIDIRECTIONAL);
+	dma_buf_unmap_attachment_locked(obj->base.import_attach, pages,
+					DMA_BIDIRECTIONAL);
 }
 
 static const struct drm_i915_gem_object_ops i915_gem_object_dmabuf_ops = {
diff --git a/drivers/gpu/drm/qxl/qxl_object.c b/drivers/gpu/drm/qxl/qxl_object.c
index b42a657e4c2f..a64cd635fbc0 100644
--- a/drivers/gpu/drm/qxl/qxl_object.c
+++ b/drivers/gpu/drm/qxl/qxl_object.c
@@ -168,9 +168,16 @@ int qxl_bo_vmap_locked(struct qxl_bo *bo, struct iosys_map *map)
 		bo->map_count++;
 		goto out;
 	}
-	r = ttm_bo_vmap(&bo->tbo, &bo->map);
+
+	r = __qxl_bo_pin(bo);
 	if (r)
 		return r;
+
+	r = ttm_bo_vmap(&bo->tbo, &bo->map);
+	if (r) {
+		__qxl_bo_unpin(bo);
+		return r;
+	}
 	bo->map_count = 1;
 
 	/* TODO: Remove kptr in favor of map everywhere. */
@@ -192,12 +199,6 @@ int qxl_bo_vmap(struct qxl_bo *bo, struct iosys_map *map)
 	if (r)
 		return r;
 
-	r = __qxl_bo_pin(bo);
-	if (r) {
-		qxl_bo_unreserve(bo);
-		return r;
-	}
-
 	r = qxl_bo_vmap_locked(bo, map);
 	qxl_bo_unreserve(bo);
 	return r;
@@ -247,6 +248,7 @@ void qxl_bo_vunmap_locked(struct qxl_bo *bo)
 		return;
 	bo->kptr = NULL;
 	ttm_bo_vunmap(&bo->tbo, &bo->map);
+	__qxl_bo_unpin(bo);
 }
 
 int qxl_bo_vunmap(struct qxl_bo *bo)
@@ -258,7 +260,6 @@ int qxl_bo_vunmap(struct qxl_bo *bo)
 		return r;
 
 	qxl_bo_vunmap_locked(bo);
-	__qxl_bo_unpin(bo);
 	qxl_bo_unreserve(bo);
 	return 0;
 }
diff --git a/drivers/gpu/drm/qxl/qxl_prime.c b/drivers/gpu/drm/qxl/qxl_prime.c
index 142d01415acb..9169c26357d3 100644
--- a/drivers/gpu/drm/qxl/qxl_prime.c
+++ b/drivers/gpu/drm/qxl/qxl_prime.c
@@ -59,7 +59,7 @@ int qxl_gem_prime_vmap(struct drm_gem_object *obj, struct iosys_map *map)
 	struct qxl_bo *bo = gem_to_qxl_bo(obj);
 	int ret;
 
-	ret = qxl_bo_vmap(bo, map);
+	ret = qxl_bo_vmap_locked(bo, map);
 	if (ret < 0)
 		return ret;
 
@@ -71,5 +71,5 @@ void qxl_gem_prime_vunmap(struct drm_gem_object *obj,
 {
 	struct qxl_bo *bo = gem_to_qxl_bo(obj);
 
-	qxl_bo_vunmap(bo);
+	qxl_bo_vunmap_locked(bo);
 }
diff --git a/drivers/media/common/videobuf2/videobuf2-dma-contig.c b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
index 678b359717c4..617062076370 100644
--- a/drivers/media/common/videobuf2/videobuf2-dma-contig.c
+++ b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
@@ -382,18 +382,12 @@ static struct sg_table *vb2_dc_dmabuf_ops_map(
 	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
 {
 	struct vb2_dc_attachment *attach = db_attach->priv;
-	/* stealing dmabuf mutex to serialize map/unmap operations */
-	struct mutex *lock = &db_attach->dmabuf->lock;
 	struct sg_table *sgt;
 
-	mutex_lock(lock);
-
 	sgt = &attach->sgt;
 	/* return previously mapped sg table */
-	if (attach->dma_dir == dma_dir) {
-		mutex_unlock(lock);
+	if (attach->dma_dir == dma_dir)
 		return sgt;
-	}
 
 	/* release any previous cache */
 	if (attach->dma_dir != DMA_NONE) {
@@ -409,14 +403,11 @@ static struct sg_table *vb2_dc_dmabuf_ops_map(
 	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir,
 			    DMA_ATTR_SKIP_CPU_SYNC)) {
 		pr_err("failed to map scatterlist\n");
-		mutex_unlock(lock);
 		return ERR_PTR(-EIO);
 	}
 
 	attach->dma_dir = dma_dir;
 
-	mutex_unlock(lock);
-
 	return sgt;
 }
 
diff --git a/drivers/media/common/videobuf2/videobuf2-dma-sg.c b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
index fa69158a65b1..d2075e7078cd 100644
--- a/drivers/media/common/videobuf2/videobuf2-dma-sg.c
+++ b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
@@ -424,18 +424,12 @@ static struct sg_table *vb2_dma_sg_dmabuf_ops_map(
 	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
 {
 	struct vb2_dma_sg_attachment *attach = db_attach->priv;
-	/* stealing dmabuf mutex to serialize map/unmap operations */
-	struct mutex *lock = &db_attach->dmabuf->lock;
 	struct sg_table *sgt;
 
-	mutex_lock(lock);
-
 	sgt = &attach->sgt;
 	/* return previously mapped sg table */
-	if (attach->dma_dir == dma_dir) {
-		mutex_unlock(lock);
+	if (attach->dma_dir == dma_dir)
 		return sgt;
-	}
 
 	/* release any previous cache */
 	if (attach->dma_dir != DMA_NONE) {
@@ -446,14 +440,11 @@ static struct sg_table *vb2_dma_sg_dmabuf_ops_map(
 	/* mapping to the client with new direction */
 	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir, 0)) {
 		pr_err("failed to map scatterlist\n");
-		mutex_unlock(lock);
 		return ERR_PTR(-EIO);
 	}
 
 	attach->dma_dir = dma_dir;
 
-	mutex_unlock(lock);
-
 	return sgt;
 }
 
diff --git a/drivers/media/common/videobuf2/videobuf2-vmalloc.c b/drivers/media/common/videobuf2/videobuf2-vmalloc.c
index 948152f1596b..3d00a7f0aac1 100644
--- a/drivers/media/common/videobuf2/videobuf2-vmalloc.c
+++ b/drivers/media/common/videobuf2/videobuf2-vmalloc.c
@@ -267,18 +267,12 @@ static struct sg_table *vb2_vmalloc_dmabuf_ops_map(
 	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
 {
 	struct vb2_vmalloc_attachment *attach = db_attach->priv;
-	/* stealing dmabuf mutex to serialize map/unmap operations */
-	struct mutex *lock = &db_attach->dmabuf->lock;
 	struct sg_table *sgt;
 
-	mutex_lock(lock);
-
 	sgt = &attach->sgt;
 	/* return previously mapped sg table */
-	if (attach->dma_dir == dma_dir) {
-		mutex_unlock(lock);
+	if (attach->dma_dir == dma_dir)
 		return sgt;
-	}
 
 	/* release any previous cache */
 	if (attach->dma_dir != DMA_NONE) {
@@ -289,14 +283,11 @@ static struct sg_table *vb2_vmalloc_dmabuf_ops_map(
 	/* mapping to the client with new direction */
 	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir, 0)) {
 		pr_err("failed to map scatterlist\n");
-		mutex_unlock(lock);
 		return ERR_PTR(-EIO);
 	}
 
 	attach->dma_dir = dma_dir;
 
-	mutex_unlock(lock);
-
 	return sgt;
 }
 
diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h
index 9d7c61a122dc..0b427939f466 100644
--- a/include/drm/drm_gem.h
+++ b/include/drm/drm_gem.h
@@ -410,4 +410,7 @@ void drm_gem_unlock_reservations(struct drm_gem_object **objs, int count,
 int drm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev,
 			    u32 handle, u64 *offset);
 
+int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
+void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
+
 #endif /* __DRM_GEM_H__ */
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
index 71731796c8c3..23698c6b1d1e 100644
--- a/include/linux/dma-buf.h
+++ b/include/linux/dma-buf.h
@@ -326,15 +326,6 @@ struct dma_buf {
 	/** @ops: dma_buf_ops associated with this buffer object. */
 	const struct dma_buf_ops *ops;
 
-	/**
-	 * @lock:
-	 *
-	 * Used internally to serialize list manipulation, attach/detach and
-	 * vmap/unmap. Note that in many cases this is superseeded by
-	 * dma_resv_lock() on @resv.
-	 */
-	struct mutex lock;
-
 	/**
 	 * @vmapping_counter:
 	 *
@@ -618,6 +609,11 @@ int dma_buf_fd(struct dma_buf *dmabuf, int flags);
 struct dma_buf *dma_buf_get(int fd);
 void dma_buf_put(struct dma_buf *dmabuf);
 
+struct sg_table *dma_buf_map_attachment_locked(struct dma_buf_attachment *,
+					       enum dma_data_direction);
+void dma_buf_unmap_attachment_locked(struct dma_buf_attachment *,
+				     struct sg_table *,
+				     enum dma_data_direction);
 struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *,
 					enum dma_data_direction);
 void dma_buf_unmap_attachment(struct dma_buf_attachment *, struct sg_table *,
-- 
2.35.3

[PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dmitry Osipenko]

All dma-bufs have dma-reservation lock that allows drivers to perform
exclusive operations over shared dma-bufs. Today's dma-buf API has
incomplete locking specification, which creates dead lock situation
for dma-buf importers and exporters that don't coordinate theirs locks.

This patch introduces new locking convention for dma-buf users. From now
on all dma-buf importers are responsible for holding dma-buf reservation
lock around operations performed over dma-bufs.

This patch implements the new dma-buf locking convention by:

  1. Making dma-buf API functions to take the reservation lock.

  2. Adding new locked variants of the dma-buf API functions for drivers
     that need to manage imported dma-bufs under the held lock.

  3. Converting all drivers to the new locking scheme.

Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/dma-buf/dma-buf.c                     | 270 +++++++++++-------
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
 drivers/gpu/drm/drm_client.c                  |   4 +-
 drivers/gpu/drm/drm_gem.c                     |  33 +++
 drivers/gpu/drm/drm_gem_framebuffer_helper.c  |   6 +-
 drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    |  10 +-
 drivers/gpu/drm/qxl/qxl_object.c              |  17 +-
 drivers/gpu/drm/qxl/qxl_prime.c               |   4 +-
 .../common/videobuf2/videobuf2-dma-contig.c   |  11 +-
 .../media/common/videobuf2/videobuf2-dma-sg.c |  11 +-
 .../common/videobuf2/videobuf2-vmalloc.c      |  11 +-
 include/drm/drm_gem.h                         |   3 +
 include/linux/dma-buf.h                       |  14 +-
 13 files changed, 241 insertions(+), 159 deletions(-)

diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
index 32f55640890c..64a9909ccfa2 100644
--- a/drivers/dma-buf/dma-buf.c
+++ b/drivers/dma-buf/dma-buf.c
@@ -552,7 +552,6 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)
 	file->f_mode |= FMODE_LSEEK;
 	dmabuf->file = file;
 
-	mutex_init(&dmabuf->lock);
 	INIT_LIST_HEAD(&dmabuf->attachments);
 
 	mutex_lock(&db_list.lock);
@@ -737,14 +736,14 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
 	attach->importer_ops = importer_ops;
 	attach->importer_priv = importer_priv;
 
+	dma_resv_lock(dmabuf->resv, NULL);
+
 	if (dmabuf->ops->attach) {
 		ret = dmabuf->ops->attach(dmabuf, attach);
 		if (ret)
 			goto err_attach;
 	}
-	dma_resv_lock(dmabuf->resv, NULL);
 	list_add(&attach->node, &dmabuf->attachments);
-	dma_resv_unlock(dmabuf->resv);
 
 	/* When either the importer or the exporter can't handle dynamic
 	 * mappings we cache the mapping here to avoid issues with the
@@ -755,7 +754,6 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
 		struct sg_table *sgt;
 
 		if (dma_buf_is_dynamic(attach->dmabuf)) {
-			dma_resv_lock(attach->dmabuf->resv, NULL);
 			ret = dmabuf->ops->pin(attach);
 			if (ret)
 				goto err_unlock;
@@ -768,15 +766,16 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
 			ret = PTR_ERR(sgt);
 			goto err_unpin;
 		}
-		if (dma_buf_is_dynamic(attach->dmabuf))
-			dma_resv_unlock(attach->dmabuf->resv);
 		attach->sgt = sgt;
 		attach->dir = DMA_BIDIRECTIONAL;
 	}
 
+	dma_resv_unlock(dmabuf->resv);
+
 	return attach;
 
 err_attach:
+	dma_resv_unlock(attach->dmabuf->resv);
 	kfree(attach);
 	return ERR_PTR(ret);
 
@@ -785,10 +784,10 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
 		dmabuf->ops->unpin(attach);
 
 err_unlock:
-	if (dma_buf_is_dynamic(attach->dmabuf))
-		dma_resv_unlock(attach->dmabuf->resv);
+	dma_resv_unlock(dmabuf->resv);
 
 	dma_buf_detach(dmabuf, attach);
+
 	return ERR_PTR(ret);
 }
 EXPORT_SYMBOL_NS_GPL(dma_buf_dynamic_attach, DMA_BUF);
@@ -832,24 +831,23 @@ void dma_buf_detach(struct dma_buf *dmabuf, struct dma_buf_attachment *attach)
 	if (WARN_ON(!dmabuf || !attach))
 		return;
 
-	if (attach->sgt) {
-		if (dma_buf_is_dynamic(attach->dmabuf))
-			dma_resv_lock(attach->dmabuf->resv, NULL);
+	if (WARN_ON(dmabuf != attach->dmabuf))
+		return;
 
+	dma_resv_lock(dmabuf->resv, NULL);
+
+	if (attach->sgt) {
 		__unmap_dma_buf(attach, attach->sgt, attach->dir);
 
-		if (dma_buf_is_dynamic(attach->dmabuf)) {
+		if (dma_buf_is_dynamic(attach->dmabuf))
 			dmabuf->ops->unpin(attach);
-			dma_resv_unlock(attach->dmabuf->resv);
-		}
 	}
 
-	dma_resv_lock(dmabuf->resv, NULL);
 	list_del(&attach->node);
-	dma_resv_unlock(dmabuf->resv);
 	if (dmabuf->ops->detach)
 		dmabuf->ops->detach(dmabuf, attach);
 
+	dma_resv_unlock(dmabuf->resv);
 	kfree(attach);
 }
 EXPORT_SYMBOL_NS_GPL(dma_buf_detach, DMA_BUF);
@@ -906,28 +904,18 @@ void dma_buf_unpin(struct dma_buf_attachment *attach)
 EXPORT_SYMBOL_NS_GPL(dma_buf_unpin, DMA_BUF);
 
 /**
- * dma_buf_map_attachment - Returns the scatterlist table of the attachment;
+ * dma_buf_map_attachment_locked - Returns the scatterlist table of the attachment;
  * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
  * dma_buf_ops.
  * @attach:	[in]	attachment whose scatterlist is to be returned
  * @direction:	[in]	direction of DMA transfer
  *
- * Returns sg_table containing the scatterlist to be returned; returns ERR_PTR
- * on error. May return -EINTR if it is interrupted by a signal.
- *
- * On success, the DMA addresses and lengths in the returned scatterlist are
- * PAGE_SIZE aligned.
- *
- * A mapping must be unmapped by using dma_buf_unmap_attachment(). Note that
- * the underlying backing storage is pinned for as long as a mapping exists,
- * therefore users/importers should not hold onto a mapping for undue amounts of
- * time.
+ * Locked variant of dma_buf_map_attachment().
  *
- * Important: Dynamic importers must wait for the exclusive fence of the struct
- * dma_resv attached to the DMA-BUF first.
+ * Caller is responsible for holding dmabuf's reservation lock.
  */
-struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
-					enum dma_data_direction direction)
+struct sg_table *dma_buf_map_attachment_locked(struct dma_buf_attachment *attach,
+					       enum dma_data_direction direction)
 {
 	struct sg_table *sg_table;
 	int r;
@@ -937,8 +925,7 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
 	if (WARN_ON(!attach || !attach->dmabuf))
 		return ERR_PTR(-EINVAL);
 
-	if (dma_buf_attachment_is_dynamic(attach))
-		dma_resv_assert_held(attach->dmabuf->resv);
+	dma_resv_assert_held(attach->dmabuf->resv);
 
 	if (attach->sgt) {
 		/*
@@ -953,7 +940,6 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
 	}
 
 	if (dma_buf_is_dynamic(attach->dmabuf)) {
-		dma_resv_assert_held(attach->dmabuf->resv);
 		if (!IS_ENABLED(CONFIG_DMABUF_MOVE_NOTIFY)) {
 			r = attach->dmabuf->ops->pin(attach);
 			if (r)
@@ -993,42 +979,101 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
 #endif /* CONFIG_DMA_API_DEBUG */
 	return sg_table;
 }
-EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment, DMA_BUF);
+EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment_locked, DMA_BUF);
 
 /**
- * dma_buf_unmap_attachment - unmaps and decreases usecount of the buffer;might
- * deallocate the scatterlist associated. Is a wrapper for unmap_dma_buf() of
+ * dma_buf_map_attachment - Returns the scatterlist table of the attachment;
+ * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
  * dma_buf_ops.
- * @attach:	[in]	attachment to unmap buffer from
- * @sg_table:	[in]	scatterlist info of the buffer to unmap
- * @direction:  [in]    direction of DMA transfer
+ * @attach:	[in]	attachment whose scatterlist is to be returned
+ * @direction:	[in]	direction of DMA transfer
  *
- * This unmaps a DMA mapping for @attached obtained by dma_buf_map_attachment().
+ * Returns sg_table containing the scatterlist to be returned; returns ERR_PTR
+ * on error. May return -EINTR if it is interrupted by a signal.
+ *
+ * On success, the DMA addresses and lengths in the returned scatterlist are
+ * PAGE_SIZE aligned.
+ *
+ * A mapping must be unmapped by using dma_buf_unmap_attachment(). Note that
+ * the underlying backing storage is pinned for as long as a mapping exists,
+ * therefore users/importers should not hold onto a mapping for undue amounts of
+ * time.
+ *
+ * Important: Dynamic importers must wait for the exclusive fence of the struct
+ * dma_resv attached to the DMA-BUF first.
  */
-void dma_buf_unmap_attachment(struct dma_buf_attachment *attach,
-				struct sg_table *sg_table,
+struct sg_table *
+dma_buf_map_attachment(struct dma_buf_attachment *attach,
 				enum dma_data_direction direction)
 {
+	struct sg_table *sg_table;
+
 	might_sleep();
 
-	if (WARN_ON(!attach || !attach->dmabuf || !sg_table))
-		return;
+	if (WARN_ON(!attach || !attach->dmabuf))
+		return ERR_PTR(-EINVAL);
+
+	dma_resv_lock(attach->dmabuf->resv, NULL);
+	sg_table = dma_buf_map_attachment_locked(attach, direction);
+	dma_resv_unlock(attach->dmabuf->resv);
 
-	if (dma_buf_attachment_is_dynamic(attach))
-		dma_resv_assert_held(attach->dmabuf->resv);
+	return sg_table;
+}
+EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment, DMA_BUF);
+
+/**
+ * dma_buf_unmap_attachment_locked - Returns the scatterlist table of the attachment;
+ * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
+ * dma_buf_ops.
+ * @attach:	[in]	attachment whose scatterlist is to be returned
+ * @direction:	[in]	direction of DMA transfer
+ *
+ * Locked variant of dma_buf_unmap_attachment().
+ *
+ * Caller is responsible for holding dmabuf's reservation lock.
+ */
+void dma_buf_unmap_attachment_locked(struct dma_buf_attachment *attach,
+				     struct sg_table *sg_table,
+				     enum dma_data_direction direction)
+{
+	might_sleep();
+
+	dma_resv_assert_held(attach->dmabuf->resv);
 
 	if (attach->sgt == sg_table)
 		return;
 
-	if (dma_buf_is_dynamic(attach->dmabuf))
-		dma_resv_assert_held(attach->dmabuf->resv);
-
 	__unmap_dma_buf(attach, sg_table, direction);
 
 	if (dma_buf_is_dynamic(attach->dmabuf) &&
 	    !IS_ENABLED(CONFIG_DMABUF_MOVE_NOTIFY))
 		dma_buf_unpin(attach);
 }
+EXPORT_SYMBOL_NS_GPL(dma_buf_unmap_attachment_locked, DMA_BUF);
+
+/**
+ * dma_buf_unmap_attachment - unmaps and decreases usecount of the buffer;might
+ * deallocate the scatterlist associated. Is a wrapper for unmap_dma_buf() of
+ * dma_buf_ops.
+ * @attach:	[in]	attachment to unmap buffer from
+ * @sg_table:	[in]	scatterlist info of the buffer to unmap
+ * @direction:  [in]    direction of DMA transfer
+ *
+ * This unmaps a DMA mapping for @attached obtained by dma_buf_map_attachment().
+ */
+void dma_buf_unmap_attachment(struct dma_buf_attachment *attach,
+			      struct sg_table *sg_table,
+			      enum dma_data_direction direction)
+{
+	might_sleep();
+
+	if (WARN_ON(!attach || !attach->dmabuf || !sg_table))
+		return;
+
+	dma_resv_lock(attach->dmabuf->resv, NULL);
+	dma_buf_unmap_attachment_locked(attach, sg_table, direction);
+	dma_resv_unlock(attach->dmabuf->resv);
+}
 EXPORT_SYMBOL_NS_GPL(dma_buf_unmap_attachment, DMA_BUF);
 
 /**
@@ -1224,6 +1269,31 @@ int dma_buf_end_cpu_access(struct dma_buf *dmabuf,
 }
 EXPORT_SYMBOL_NS_GPL(dma_buf_end_cpu_access, DMA_BUF);
 
+static int dma_buf_mmap_locked(struct dma_buf *dmabuf,
+			       struct vm_area_struct *vma,
+			       unsigned long pgoff)
+{
+	dma_resv_assert_held(dmabuf->resv);
+
+	/* check if buffer supports mmap */
+	if (!dmabuf->ops->mmap)
+		return -EINVAL;
+
+	/* check for offset overflow */
+	if (pgoff + vma_pages(vma) < pgoff)
+		return -EOVERFLOW;
+
+	/* check for overflowing the buffer's size */
+	if (pgoff + vma_pages(vma) >
+	    dmabuf->size >> PAGE_SHIFT)
+		return -EINVAL;
+
+	/* readjust the vma */
+	vma_set_file(vma, dmabuf->file);
+	vma->vm_pgoff = pgoff;
+
+	return dmabuf->ops->mmap(dmabuf, vma);
+}
 
 /**
  * dma_buf_mmap - Setup up a userspace mmap with the given vma
@@ -1242,29 +1312,46 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_end_cpu_access, DMA_BUF);
 int dma_buf_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma,
 		 unsigned long pgoff)
 {
+	int ret;
+
 	if (WARN_ON(!dmabuf || !vma))
 		return -EINVAL;
 
-	/* check if buffer supports mmap */
-	if (!dmabuf->ops->mmap)
-		return -EINVAL;
+	dma_resv_lock(dmabuf->resv, NULL);
+	ret = dma_buf_mmap_locked(dmabuf, vma, pgoff);
+	dma_resv_unlock(dmabuf->resv);
 
-	/* check for offset overflow */
-	if (pgoff + vma_pages(vma) < pgoff)
-		return -EOVERFLOW;
+	return ret;
+}
+EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
 
-	/* check for overflowing the buffer's size */
-	if (pgoff + vma_pages(vma) >
-	    dmabuf->size >> PAGE_SHIFT)
-		return -EINVAL;
+static int dma_buf_vmap_locked(struct dma_buf *dmabuf, struct iosys_map *map)
+{
+	struct iosys_map ptr;
+	int ret;
 
-	/* readjust the vma */
-	vma_set_file(vma, dmabuf->file);
-	vma->vm_pgoff = pgoff;
+	dma_resv_assert_held(dmabuf->resv);
 
-	return dmabuf->ops->mmap(dmabuf, vma);
+	if (dmabuf->vmapping_counter) {
+		dmabuf->vmapping_counter++;
+		BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
+		*map = dmabuf->vmap_ptr;
+		return ret;
+	}
+
+	BUG_ON(iosys_map_is_set(&dmabuf->vmap_ptr));
+
+	ret = dmabuf->ops->vmap(dmabuf, &ptr);
+	if (WARN_ON_ONCE(ret))
+		return ret;
+
+	dmabuf->vmap_ptr = ptr;
+	dmabuf->vmapping_counter = 1;
+
+	*map = dmabuf->vmap_ptr;
+
+	return 0;
 }
-EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
 
 /**
  * dma_buf_vmap - Create virtual mapping for the buffer object into kernel
@@ -1284,8 +1371,7 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
  */
 int dma_buf_vmap(struct dma_buf *dmabuf, struct iosys_map *map)
 {
-	struct iosys_map ptr;
-	int ret = 0;
+	int ret;
 
 	iosys_map_clear(map);
 
@@ -1295,52 +1381,40 @@ int dma_buf_vmap(struct dma_buf *dmabuf, struct iosys_map *map)
 	if (!dmabuf->ops->vmap)
 		return -EINVAL;
 
-	mutex_lock(&dmabuf->lock);
-	if (dmabuf->vmapping_counter) {
-		dmabuf->vmapping_counter++;
-		BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
-		*map = dmabuf->vmap_ptr;
-		goto out_unlock;
-	}
-
-	BUG_ON(iosys_map_is_set(&dmabuf->vmap_ptr));
-
-	ret = dmabuf->ops->vmap(dmabuf, &ptr);
-	if (WARN_ON_ONCE(ret))
-		goto out_unlock;
-
-	dmabuf->vmap_ptr = ptr;
-	dmabuf->vmapping_counter = 1;
-
-	*map = dmabuf->vmap_ptr;
+	dma_resv_lock(dmabuf->resv, NULL);
+	ret = dma_buf_vmap_locked(dmabuf, map);
+	dma_resv_unlock(dmabuf->resv);
 
-out_unlock:
-	mutex_unlock(&dmabuf->lock);
 	return ret;
 }
 EXPORT_SYMBOL_NS_GPL(dma_buf_vmap, DMA_BUF);
 
-/**
- * dma_buf_vunmap - Unmap a vmap obtained by dma_buf_vmap.
- * @dmabuf:	[in]	buffer to vunmap
- * @map:	[in]	vmap pointer to vunmap
- */
-void dma_buf_vunmap(struct dma_buf *dmabuf, struct iosys_map *map)
+static void dma_buf_vunmap_locked(struct dma_buf *dmabuf, struct iosys_map *map)
 {
-	if (WARN_ON(!dmabuf))
-		return;
-
 	BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
 	BUG_ON(dmabuf->vmapping_counter == 0);
 	BUG_ON(!iosys_map_is_equal(&dmabuf->vmap_ptr, map));
 
-	mutex_lock(&dmabuf->lock);
 	if (--dmabuf->vmapping_counter == 0) {
 		if (dmabuf->ops->vunmap)
 			dmabuf->ops->vunmap(dmabuf, map);
 		iosys_map_clear(&dmabuf->vmap_ptr);
 	}
-	mutex_unlock(&dmabuf->lock);
+}
+
+/**
+ * dma_buf_vunmap - Unmap a vmap obtained by dma_buf_vmap.
+ * @dmabuf:	[in]	buffer to vunmap
+ * @map:	[in]	vmap pointer to vunmap
+ */
+void dma_buf_vunmap(struct dma_buf *dmabuf, struct iosys_map *map)
+{
+	if (WARN_ON(!dmabuf))
+		return;
+
+	dma_resv_lock(dmabuf->resv, NULL);
+	dma_buf_vunmap_locked(dmabuf, map);
+	dma_resv_unlock(dmabuf->resv);
 }
 EXPORT_SYMBOL_NS_GPL(dma_buf_vunmap, DMA_BUF);
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index be6f76a30ac6..b704bdf5601a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -882,7 +882,8 @@ static int amdgpu_ttm_backend_bind(struct ttm_device *bdev,
 			struct sg_table *sgt;
 
 			attach = gtt->gobj->import_attach;
-			sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL);
+			sgt = dma_buf_map_attachment_locked(attach,
+							    DMA_BIDIRECTIONAL);
 			if (IS_ERR(sgt))
 				return PTR_ERR(sgt);
 
@@ -1007,7 +1008,8 @@ static void amdgpu_ttm_backend_unbind(struct ttm_device *bdev,
 		struct dma_buf_attachment *attach;
 
 		attach = gtt->gobj->import_attach;
-		dma_buf_unmap_attachment(attach, ttm->sg, DMA_BIDIRECTIONAL);
+		dma_buf_unmap_attachment_locked(attach, ttm->sg,
+						DMA_BIDIRECTIONAL);
 		ttm->sg = NULL;
 	}
 
diff --git a/drivers/gpu/drm/drm_client.c b/drivers/gpu/drm/drm_client.c
index af3b7395bf69..e9a1cd310352 100644
--- a/drivers/gpu/drm/drm_client.c
+++ b/drivers/gpu/drm/drm_client.c
@@ -323,7 +323,7 @@ drm_client_buffer_vmap(struct drm_client_buffer *buffer,
 	 * fd_install step out of the driver backend hooks, to make that
 	 * final step optional for internal users.
 	 */
-	ret = drm_gem_vmap(buffer->gem, map);
+	ret = drm_gem_vmap_unlocked(buffer->gem, map);
 	if (ret)
 		return ret;
 
@@ -345,7 +345,7 @@ void drm_client_buffer_vunmap(struct drm_client_buffer *buffer)
 {
 	struct iosys_map *map = &buffer->map;
 
-	drm_gem_vunmap(buffer->gem, map);
+	drm_gem_vunmap_unlocked(buffer->gem, map);
 }
 EXPORT_SYMBOL(drm_client_buffer_vunmap);
 
diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index 7c0b025508e4..c61674887582 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -1053,7 +1053,12 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,
 	vma->vm_ops = obj->funcs->vm_ops;
 
 	if (obj->funcs->mmap) {
+		ret = dma_resv_lock_interruptible(obj->resv, NULL);
+		if (ret)
+			goto err_drm_gem_object_put;
+
 		ret = obj->funcs->mmap(obj, vma);
+		dma_resv_unlock(obj->resv);
 		if (ret)
 			goto err_drm_gem_object_put;
 		WARN_ON(!(vma->vm_flags & VM_DONTEXPAND));
@@ -1158,6 +1163,8 @@ void drm_gem_print_info(struct drm_printer *p, unsigned int indent,
 
 int drm_gem_pin(struct drm_gem_object *obj)
 {
+	dma_resv_assert_held(obj->resv);
+
 	if (obj->funcs->pin)
 		return obj->funcs->pin(obj);
 	else
@@ -1166,6 +1173,8 @@ int drm_gem_pin(struct drm_gem_object *obj)
 
 void drm_gem_unpin(struct drm_gem_object *obj)
 {
+	dma_resv_assert_held(obj->resv);
+
 	if (obj->funcs->unpin)
 		obj->funcs->unpin(obj);
 }
@@ -1174,6 +1183,8 @@ int drm_gem_vmap(struct drm_gem_object *obj, struct iosys_map *map)
 {
 	int ret;
 
+	dma_resv_assert_held(obj->resv);
+
 	if (!obj->funcs->vmap)
 		return -EOPNOTSUPP;
 
@@ -1189,6 +1200,8 @@ EXPORT_SYMBOL(drm_gem_vmap);
 
 void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
 {
+	dma_resv_assert_held(obj->resv);
+
 	if (iosys_map_is_null(map))
 		return;
 
@@ -1200,6 +1213,26 @@ void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
 }
 EXPORT_SYMBOL(drm_gem_vunmap);
 
+int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
+{
+	int ret;
+
+	dma_resv_lock(obj->resv, NULL);
+	ret = drm_gem_vmap(obj, map);
+	dma_resv_unlock(obj->resv);
+
+	return ret;
+}
+EXPORT_SYMBOL(drm_gem_vmap_unlocked);
+
+void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
+{
+	dma_resv_lock(obj->resv, NULL);
+	drm_gem_vunmap(obj, map);
+	dma_resv_unlock(obj->resv);
+}
+EXPORT_SYMBOL(drm_gem_vunmap_unlocked);
+
 /**
  * drm_gem_lock_reservations - Sets up the ww context and acquires
  * the lock on an array of GEM objects.
diff --git a/drivers/gpu/drm/drm_gem_framebuffer_helper.c b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
index f4619803acd0..a0bff53b158e 100644
--- a/drivers/gpu/drm/drm_gem_framebuffer_helper.c
+++ b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
@@ -348,7 +348,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb,
 			iosys_map_clear(&map[i]);
 			continue;
 		}
-		ret = drm_gem_vmap(obj, &map[i]);
+		ret = drm_gem_vmap_unlocked(obj, &map[i]);
 		if (ret)
 			goto err_drm_gem_vunmap;
 	}
@@ -370,7 +370,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb,
 		obj = drm_gem_fb_get_obj(fb, i);
 		if (!obj)
 			continue;
-		drm_gem_vunmap(obj, &map[i]);
+		drm_gem_vunmap_unlocked(obj, &map[i]);
 	}
 	return ret;
 }
@@ -398,7 +398,7 @@ void drm_gem_fb_vunmap(struct drm_framebuffer *fb,
 			continue;
 		if (iosys_map_is_null(&map[i]))
 			continue;
-		drm_gem_vunmap(obj, &map[i]);
+		drm_gem_vunmap_unlocked(obj, &map[i]);
 	}
 }
 EXPORT_SYMBOL(drm_gem_fb_vunmap);
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
index f5062d0c6333..09502d490da8 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
@@ -72,7 +72,7 @@ static int i915_gem_dmabuf_vmap(struct dma_buf *dma_buf,
 	struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);
 	void *vaddr;
 
-	vaddr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
+	vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
 	if (IS_ERR(vaddr))
 		return PTR_ERR(vaddr);
 
@@ -241,8 +241,8 @@ static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
 
 	assert_object_held(obj);
 
-	pages = dma_buf_map_attachment(obj->base.import_attach,
-				       DMA_BIDIRECTIONAL);
+	pages = dma_buf_map_attachment_locked(obj->base.import_attach,
+					      DMA_BIDIRECTIONAL);
 	if (IS_ERR(pages))
 		return PTR_ERR(pages);
 
@@ -270,8 +270,8 @@ static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
 static void i915_gem_object_put_pages_dmabuf(struct drm_i915_gem_object *obj,
 					     struct sg_table *pages)
 {
-	dma_buf_unmap_attachment(obj->base.import_attach, pages,
-				 DMA_BIDIRECTIONAL);
+	dma_buf_unmap_attachment_locked(obj->base.import_attach, pages,
+					DMA_BIDIRECTIONAL);
 }
 
 static const struct drm_i915_gem_object_ops i915_gem_object_dmabuf_ops = {
diff --git a/drivers/gpu/drm/qxl/qxl_object.c b/drivers/gpu/drm/qxl/qxl_object.c
index b42a657e4c2f..a64cd635fbc0 100644
--- a/drivers/gpu/drm/qxl/qxl_object.c
+++ b/drivers/gpu/drm/qxl/qxl_object.c
@@ -168,9 +168,16 @@ int qxl_bo_vmap_locked(struct qxl_bo *bo, struct iosys_map *map)
 		bo->map_count++;
 		goto out;
 	}
-	r = ttm_bo_vmap(&bo->tbo, &bo->map);
+
+	r = __qxl_bo_pin(bo);
 	if (r)
 		return r;
+
+	r = ttm_bo_vmap(&bo->tbo, &bo->map);
+	if (r) {
+		__qxl_bo_unpin(bo);
+		return r;
+	}
 	bo->map_count = 1;
 
 	/* TODO: Remove kptr in favor of map everywhere. */
@@ -192,12 +199,6 @@ int qxl_bo_vmap(struct qxl_bo *bo, struct iosys_map *map)
 	if (r)
 		return r;
 
-	r = __qxl_bo_pin(bo);
-	if (r) {
-		qxl_bo_unreserve(bo);
-		return r;
-	}
-
 	r = qxl_bo_vmap_locked(bo, map);
 	qxl_bo_unreserve(bo);
 	return r;
@@ -247,6 +248,7 @@ void qxl_bo_vunmap_locked(struct qxl_bo *bo)
 		return;
 	bo->kptr = NULL;
 	ttm_bo_vunmap(&bo->tbo, &bo->map);
+	__qxl_bo_unpin(bo);
 }
 
 int qxl_bo_vunmap(struct qxl_bo *bo)
@@ -258,7 +260,6 @@ int qxl_bo_vunmap(struct qxl_bo *bo)
 		return r;
 
 	qxl_bo_vunmap_locked(bo);
-	__qxl_bo_unpin(bo);
 	qxl_bo_unreserve(bo);
 	return 0;
 }
diff --git a/drivers/gpu/drm/qxl/qxl_prime.c b/drivers/gpu/drm/qxl/qxl_prime.c
index 142d01415acb..9169c26357d3 100644
--- a/drivers/gpu/drm/qxl/qxl_prime.c
+++ b/drivers/gpu/drm/qxl/qxl_prime.c
@@ -59,7 +59,7 @@ int qxl_gem_prime_vmap(struct drm_gem_object *obj, struct iosys_map *map)
 	struct qxl_bo *bo = gem_to_qxl_bo(obj);
 	int ret;
 
-	ret = qxl_bo_vmap(bo, map);
+	ret = qxl_bo_vmap_locked(bo, map);
 	if (ret < 0)
 		return ret;
 
@@ -71,5 +71,5 @@ void qxl_gem_prime_vunmap(struct drm_gem_object *obj,
 {
 	struct qxl_bo *bo = gem_to_qxl_bo(obj);
 
-	qxl_bo_vunmap(bo);
+	qxl_bo_vunmap_locked(bo);
 }
diff --git a/drivers/media/common/videobuf2/videobuf2-dma-contig.c b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
index 678b359717c4..617062076370 100644
--- a/drivers/media/common/videobuf2/videobuf2-dma-contig.c
+++ b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
@@ -382,18 +382,12 @@ static struct sg_table *vb2_dc_dmabuf_ops_map(
 	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
 {
 	struct vb2_dc_attachment *attach = db_attach->priv;
-	/* stealing dmabuf mutex to serialize map/unmap operations */
-	struct mutex *lock = &db_attach->dmabuf->lock;
 	struct sg_table *sgt;
 
-	mutex_lock(lock);
-
 	sgt = &attach->sgt;
 	/* return previously mapped sg table */
-	if (attach->dma_dir == dma_dir) {
-		mutex_unlock(lock);
+	if (attach->dma_dir == dma_dir)
 		return sgt;
-	}
 
 	/* release any previous cache */
 	if (attach->dma_dir != DMA_NONE) {
@@ -409,14 +403,11 @@ static struct sg_table *vb2_dc_dmabuf_ops_map(
 	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir,
 			    DMA_ATTR_SKIP_CPU_SYNC)) {
 		pr_err("failed to map scatterlist\n");
-		mutex_unlock(lock);
 		return ERR_PTR(-EIO);
 	}
 
 	attach->dma_dir = dma_dir;
 
-	mutex_unlock(lock);
-
 	return sgt;
 }
 
diff --git a/drivers/media/common/videobuf2/videobuf2-dma-sg.c b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
index fa69158a65b1..d2075e7078cd 100644
--- a/drivers/media/common/videobuf2/videobuf2-dma-sg.c
+++ b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
@@ -424,18 +424,12 @@ static struct sg_table *vb2_dma_sg_dmabuf_ops_map(
 	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
 {
 	struct vb2_dma_sg_attachment *attach = db_attach->priv;
-	/* stealing dmabuf mutex to serialize map/unmap operations */
-	struct mutex *lock = &db_attach->dmabuf->lock;
 	struct sg_table *sgt;
 
-	mutex_lock(lock);
-
 	sgt = &attach->sgt;
 	/* return previously mapped sg table */
-	if (attach->dma_dir == dma_dir) {
-		mutex_unlock(lock);
+	if (attach->dma_dir == dma_dir)
 		return sgt;
-	}
 
 	/* release any previous cache */
 	if (attach->dma_dir != DMA_NONE) {
@@ -446,14 +440,11 @@ static struct sg_table *vb2_dma_sg_dmabuf_ops_map(
 	/* mapping to the client with new direction */
 	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir, 0)) {
 		pr_err("failed to map scatterlist\n");
-		mutex_unlock(lock);
 		return ERR_PTR(-EIO);
 	}
 
 	attach->dma_dir = dma_dir;
 
-	mutex_unlock(lock);
-
 	return sgt;
 }
 
diff --git a/drivers/media/common/videobuf2/videobuf2-vmalloc.c b/drivers/media/common/videobuf2/videobuf2-vmalloc.c
index 948152f1596b..3d00a7f0aac1 100644
--- a/drivers/media/common/videobuf2/videobuf2-vmalloc.c
+++ b/drivers/media/common/videobuf2/videobuf2-vmalloc.c
@@ -267,18 +267,12 @@ static struct sg_table *vb2_vmalloc_dmabuf_ops_map(
 	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
 {
 	struct vb2_vmalloc_attachment *attach = db_attach->priv;
-	/* stealing dmabuf mutex to serialize map/unmap operations */
-	struct mutex *lock = &db_attach->dmabuf->lock;
 	struct sg_table *sgt;
 
-	mutex_lock(lock);
-
 	sgt = &attach->sgt;
 	/* return previously mapped sg table */
-	if (attach->dma_dir == dma_dir) {
-		mutex_unlock(lock);
+	if (attach->dma_dir == dma_dir)
 		return sgt;
-	}
 
 	/* release any previous cache */
 	if (attach->dma_dir != DMA_NONE) {
@@ -289,14 +283,11 @@ static struct sg_table *vb2_vmalloc_dmabuf_ops_map(
 	/* mapping to the client with new direction */
 	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir, 0)) {
 		pr_err("failed to map scatterlist\n");
-		mutex_unlock(lock);
 		return ERR_PTR(-EIO);
 	}
 
 	attach->dma_dir = dma_dir;
 
-	mutex_unlock(lock);
-
 	return sgt;
 }
 
diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h
index 9d7c61a122dc..0b427939f466 100644
--- a/include/drm/drm_gem.h
+++ b/include/drm/drm_gem.h
@@ -410,4 +410,7 @@ void drm_gem_unlock_reservations(struct drm_gem_object **objs, int count,
 int drm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev,
 			    u32 handle, u64 *offset);
 
+int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
+void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
+
 #endif /* __DRM_GEM_H__ */
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
index 71731796c8c3..23698c6b1d1e 100644
--- a/include/linux/dma-buf.h
+++ b/include/linux/dma-buf.h
@@ -326,15 +326,6 @@ struct dma_buf {
 	/** @ops: dma_buf_ops associated with this buffer object. */
 	const struct dma_buf_ops *ops;
 
-	/**
-	 * @lock:
-	 *
-	 * Used internally to serialize list manipulation, attach/detach and
-	 * vmap/unmap. Note that in many cases this is superseeded by
-	 * dma_resv_lock() on @resv.
-	 */
-	struct mutex lock;
-
 	/**
 	 * @vmapping_counter:
 	 *
@@ -618,6 +609,11 @@ int dma_buf_fd(struct dma_buf *dmabuf, int flags);
 struct dma_buf *dma_buf_get(int fd);
 void dma_buf_put(struct dma_buf *dmabuf);
 
+struct sg_table *dma_buf_map_attachment_locked(struct dma_buf_attachment *,
+					       enum dma_data_direction);
+void dma_buf_unmap_attachment_locked(struct dma_buf_attachment *,
+				     struct sg_table *,
+				     enum dma_data_direction);
 struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *,
 					enum dma_data_direction);
 void dma_buf_unmap_attachment(struct dma_buf_attachment *, struct sg_table *,
-- 
2.35.3

[PATCH v6 15/22] drm/shmem-helper: Don't use vmap_use_count for dma-bufs

[Dmitry Osipenko]

There is no need to refcount vmappings of dma-bufs because dma-buf core
has its own refcounting. Drop the refcounting of dma-bufs. This will ease
replacing of all drm-shmem locks with a single dma-buf reservation lock,
preparing drm-shmem code for addition of the generic drm-shmem shrinker.

Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/drm_gem_shmem_helper.c | 35 +++++++++++++++-----------
 1 file changed, 20 insertions(+), 15 deletions(-)

diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 7232e321fdb4..fd2647690bf7 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -293,24 +293,22 @@ static int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object *shmem,
 	struct drm_gem_object *obj = &shmem->base;
 	int ret = 0;
 
-	if (shmem->vmap_use_count++ > 0) {
-		iosys_map_set_vaddr(map, shmem->vaddr);
-		return 0;
-	}
-
 	if (obj->import_attach) {
 		ret = dma_buf_vmap(obj->import_attach->dmabuf, map);
 		if (!ret) {
 			if (WARN_ON(map->is_iomem)) {
 				dma_buf_vunmap(obj->import_attach->dmabuf, map);
-				ret = -EIO;
-				goto err_put_pages;
+				return -EIO;
 			}
-			shmem->vaddr = map->vaddr;
 		}
 	} else {
 		pgprot_t prot = PAGE_KERNEL;
 
+		if (shmem->vmap_use_count++ > 0) {
+			iosys_map_set_vaddr(map, shmem->vaddr);
+			return 0;
+		}
+
 		ret = drm_gem_shmem_get_pages(shmem);
 		if (ret)
 			goto err_zero_use;
@@ -376,15 +374,15 @@ static void drm_gem_shmem_vunmap_locked(struct drm_gem_shmem_object *shmem,
 {
 	struct drm_gem_object *obj = &shmem->base;
 
-	if (WARN_ON_ONCE(!shmem->vmap_use_count))
-		return;
-
-	if (--shmem->vmap_use_count > 0)
-		return;
-
 	if (obj->import_attach) {
 		dma_buf_vunmap(obj->import_attach->dmabuf, map);
 	} else {
+		if (WARN_ON_ONCE(!shmem->vmap_use_count))
+			return;
+
+		if (--shmem->vmap_use_count > 0)
+			return;
+
 		vunmap(shmem->vaddr);
 		drm_gem_shmem_put_pages(shmem);
 	}
@@ -637,7 +635,14 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
 			      struct drm_printer *p, unsigned int indent)
 {
 	drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
-	drm_printf_indent(p, indent, "vmap_use_count=%u\n", shmem->vmap_use_count);
+
+	if (shmem->base.import_attach)
+		drm_printf_indent(p, indent, "vmap_use_count=%u\n",
+				  shmem->base.dma_buf->vmapping_counter);
+	else
+		drm_printf_indent(p, indent, "vmap_use_count=%u\n",
+				  shmem->vmap_use_count);
+
 	drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
 }
 EXPORT_SYMBOL(drm_gem_shmem_print_info);
-- 
2.35.3

[PATCH v6 15/22] drm/shmem-helper: Don't use vmap_use_count for dma-bufs

[Dmitry Osipenko]

There is no need to refcount vmappings of dma-bufs because dma-buf core
has its own refcounting. Drop the refcounting of dma-bufs. This will ease
replacing of all drm-shmem locks with a single dma-buf reservation lock,
preparing drm-shmem code for addition of the generic drm-shmem shrinker.

Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/drm_gem_shmem_helper.c | 35 +++++++++++++++-----------
 1 file changed, 20 insertions(+), 15 deletions(-)

diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 7232e321fdb4..fd2647690bf7 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -293,24 +293,22 @@ static int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object *shmem,
 	struct drm_gem_object *obj = &shmem->base;
 	int ret = 0;
 
-	if (shmem->vmap_use_count++ > 0) {
-		iosys_map_set_vaddr(map, shmem->vaddr);
-		return 0;
-	}
-
 	if (obj->import_attach) {
 		ret = dma_buf_vmap(obj->import_attach->dmabuf, map);
 		if (!ret) {
 			if (WARN_ON(map->is_iomem)) {
 				dma_buf_vunmap(obj->import_attach->dmabuf, map);
-				ret = -EIO;
-				goto err_put_pages;
+				return -EIO;
 			}
-			shmem->vaddr = map->vaddr;
 		}
 	} else {
 		pgprot_t prot = PAGE_KERNEL;
 
+		if (shmem->vmap_use_count++ > 0) {
+			iosys_map_set_vaddr(map, shmem->vaddr);
+			return 0;
+		}
+
 		ret = drm_gem_shmem_get_pages(shmem);
 		if (ret)
 			goto err_zero_use;
@@ -376,15 +374,15 @@ static void drm_gem_shmem_vunmap_locked(struct drm_gem_shmem_object *shmem,
 {
 	struct drm_gem_object *obj = &shmem->base;
 
-	if (WARN_ON_ONCE(!shmem->vmap_use_count))
-		return;
-
-	if (--shmem->vmap_use_count > 0)
-		return;
-
 	if (obj->import_attach) {
 		dma_buf_vunmap(obj->import_attach->dmabuf, map);
 	} else {
+		if (WARN_ON_ONCE(!shmem->vmap_use_count))
+			return;
+
+		if (--shmem->vmap_use_count > 0)
+			return;
+
 		vunmap(shmem->vaddr);
 		drm_gem_shmem_put_pages(shmem);
 	}
@@ -637,7 +635,14 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
 			      struct drm_printer *p, unsigned int indent)
 {
 	drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
-	drm_printf_indent(p, indent, "vmap_use_count=%u\n", shmem->vmap_use_count);
+
+	if (shmem->base.import_attach)
+		drm_printf_indent(p, indent, "vmap_use_count=%u\n",
+				  shmem->base.dma_buf->vmapping_counter);
+	else
+		drm_printf_indent(p, indent, "vmap_use_count=%u\n",
+				  shmem->vmap_use_count);
+
 	drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
 }
 EXPORT_SYMBOL(drm_gem_shmem_print_info);
-- 
2.35.3

[PATCH v6 16/22] drm/shmem-helper: Use reservation lock

[Dmitry Osipenko]

Replace drm_gem_shmem locks with GEM reservation lock to make drm-shmem
locks consistent with the new locking convention of dma-bufs which tells
that dma-buf importers are responsible for holding reservation lock for
all operations performed over dma-bufs. This prepares drm-shmem code for
addition of the generic shmem shrinker framework.

Suggested-by: Daniel Vetter <daniel@ffwll.ch>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/drm_gem_shmem_helper.c        | 181 +++++++-----------
 drivers/gpu/drm/lima/lima_gem.c               |   8 +-
 drivers/gpu/drm/lima/lima_sched.c             |   4 +-
 drivers/gpu/drm/panfrost/panfrost_drv.c       |   7 +-
 .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   6 +-
 drivers/gpu/drm/panfrost/panfrost_mmu.c       |  19 +-
 drivers/gpu/drm/panfrost/panfrost_perfcnt.c   |   6 +-
 include/drm/drm_gem_shmem_helper.h            |  14 +-
 8 files changed, 97 insertions(+), 148 deletions(-)

diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
index fd2647690bf7..555fe212bd98 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -86,8 +86,6 @@ __drm_gem_shmem_create(struct drm_device *dev, size_t size, bool private)
 	if (ret)
 		goto err_release;
 
-	mutex_init(&shmem->pages_lock);
-	mutex_init(&shmem->vmap_lock);
 	INIT_LIST_HEAD(&shmem->madv_list);
 
 	if (!private) {
@@ -139,11 +137,13 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
 {
 	struct drm_gem_object *obj = &shmem->base;
 
-	WARN_ON(shmem->vmap_use_count);
-
 	if (obj->import_attach) {
 		drm_prime_gem_destroy(obj, shmem->sgt);
 	} else {
+		dma_resv_lock(shmem->base.resv, NULL);
+
+		WARN_ON(shmem->vmap_use_count);
+
 		if (shmem->sgt) {
 			dma_unmap_sgtable(obj->dev->dev, shmem->sgt,
 					  DMA_BIDIRECTIONAL, 0);
@@ -152,18 +152,18 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
 		}
 		if (shmem->pages)
 			drm_gem_shmem_put_pages(shmem);
-	}
 
-	WARN_ON(shmem->pages_use_count);
+		WARN_ON(shmem->pages_use_count);
+
+		dma_resv_unlock(shmem->base.resv);
+	}
 
 	drm_gem_object_release(obj);
-	mutex_destroy(&shmem->pages_lock);
-	mutex_destroy(&shmem->vmap_lock);
 	kfree(shmem);
 }
 EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
 
-static int drm_gem_shmem_get_pages_locked(struct drm_gem_shmem_object *shmem)
+static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
 {
 	struct drm_gem_object *obj = &shmem->base;
 	struct page **pages;
@@ -194,35 +194,17 @@ static int drm_gem_shmem_get_pages_locked(struct drm_gem_shmem_object *shmem)
 }
 
 /*
- * drm_gem_shmem_get_pages - Allocate backing pages for a shmem GEM object
+ * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
  * @shmem: shmem GEM object
  *
- * This function makes sure that backing pages exists for the shmem GEM object
- * and increases the use count.
- *
- * Returns:
- * 0 on success or a negative error code on failure.
+ * This function decreases the use count and puts the backing pages when use drops to zero.
  */
-int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
-{
-	int ret;
-
-	WARN_ON(shmem->base.import_attach);
-
-	ret = mutex_lock_interruptible(&shmem->pages_lock);
-	if (ret)
-		return ret;
-	ret = drm_gem_shmem_get_pages_locked(shmem);
-	mutex_unlock(&shmem->pages_lock);
-
-	return ret;
-}
-EXPORT_SYMBOL(drm_gem_shmem_get_pages);
-
-static void drm_gem_shmem_put_pages_locked(struct drm_gem_shmem_object *shmem)
+void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
 {
 	struct drm_gem_object *obj = &shmem->base;
 
+	dma_resv_assert_held(shmem->base.resv);
+
 	if (WARN_ON_ONCE(!shmem->pages_use_count))
 		return;
 
@@ -239,19 +221,6 @@ static void drm_gem_shmem_put_pages_locked(struct drm_gem_shmem_object *shmem)
 			  shmem->pages_mark_accessed_on_put);
 	shmem->pages = NULL;
 }
-
-/*
- * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
- * @shmem: shmem GEM object
- *
- * This function decreases the use count and puts the backing pages when use drops to zero.
- */
-void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
-{
-	mutex_lock(&shmem->pages_lock);
-	drm_gem_shmem_put_pages_locked(shmem);
-	mutex_unlock(&shmem->pages_lock);
-}
 EXPORT_SYMBOL(drm_gem_shmem_put_pages);
 
 /**
@@ -266,6 +235,8 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
  */
 int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
 {
+	dma_resv_assert_held(shmem->base.resv);
+
 	WARN_ON(shmem->base.import_attach);
 
 	return drm_gem_shmem_get_pages(shmem);
@@ -281,14 +252,31 @@ EXPORT_SYMBOL(drm_gem_shmem_pin);
  */
 void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
 {
+	dma_resv_assert_held(shmem->base.resv);
+
 	WARN_ON(shmem->base.import_attach);
 
 	drm_gem_shmem_put_pages(shmem);
 }
 EXPORT_SYMBOL(drm_gem_shmem_unpin);
 
-static int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object *shmem,
-				     struct iosys_map *map)
+/*
+ * drm_gem_shmem_vmap - Create a virtual mapping for a shmem GEM object
+ * @shmem: shmem GEM object
+ * @map: Returns the kernel virtual address of the SHMEM GEM object's backing
+ *       store.
+ *
+ * This function makes sure that a contiguous kernel virtual address mapping
+ * exists for the buffer backing the shmem GEM object. It hides the differences
+ * between dma-buf imported and natively allocated objects.
+ *
+ * Acquired mappings should be cleaned up by calling drm_gem_shmem_vunmap().
+ *
+ * Returns:
+ * 0 on success or a negative error code on failure.
+ */
+int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
+		       struct iosys_map *map)
 {
 	struct drm_gem_object *obj = &shmem->base;
 	int ret = 0;
@@ -304,6 +292,8 @@ static int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object *shmem,
 	} else {
 		pgprot_t prot = PAGE_KERNEL;
 
+		dma_resv_assert_held(shmem->base.resv);
+
 		if (shmem->vmap_use_count++ > 0) {
 			iosys_map_set_vaddr(map, shmem->vaddr);
 			return 0;
@@ -338,45 +328,30 @@ static int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object *shmem,
 
 	return ret;
 }
+EXPORT_SYMBOL(drm_gem_shmem_vmap);
 
 /*
- * drm_gem_shmem_vmap - Create a virtual mapping for a shmem GEM object
+ * drm_gem_shmem_vunmap - Unmap a virtual mapping for a shmem GEM object
  * @shmem: shmem GEM object
- * @map: Returns the kernel virtual address of the SHMEM GEM object's backing
- *       store.
- *
- * This function makes sure that a contiguous kernel virtual address mapping
- * exists for the buffer backing the shmem GEM object. It hides the differences
- * between dma-buf imported and natively allocated objects.
+ * @map: Kernel virtual address where the SHMEM GEM object was mapped
  *
- * Acquired mappings should be cleaned up by calling drm_gem_shmem_vunmap().
+ * This function cleans up a kernel virtual address mapping acquired by
+ * drm_gem_shmem_vmap(). The mapping is only removed when the use count drops to
+ * zero.
  *
- * Returns:
- * 0 on success or a negative error code on failure.
+ * This function hides the differences between dma-buf imported and natively
+ * allocated objects.
  */
-int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
-		       struct iosys_map *map)
-{
-	int ret;
-
-	ret = mutex_lock_interruptible(&shmem->vmap_lock);
-	if (ret)
-		return ret;
-	ret = drm_gem_shmem_vmap_locked(shmem, map);
-	mutex_unlock(&shmem->vmap_lock);
-
-	return ret;
-}
-EXPORT_SYMBOL(drm_gem_shmem_vmap);
-
-static void drm_gem_shmem_vunmap_locked(struct drm_gem_shmem_object *shmem,
-					struct iosys_map *map)
+void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
+			  struct iosys_map *map)
 {
 	struct drm_gem_object *obj = &shmem->base;
 
 	if (obj->import_attach) {
 		dma_buf_vunmap(obj->import_attach->dmabuf, map);
 	} else {
+		dma_resv_assert_held(shmem->base.resv);
+
 		if (WARN_ON_ONCE(!shmem->vmap_use_count))
 			return;
 
@@ -389,26 +364,6 @@ static void drm_gem_shmem_vunmap_locked(struct drm_gem_shmem_object *shmem,
 
 	shmem->vaddr = NULL;
 }
-
-/*
- * drm_gem_shmem_vunmap - Unmap a virtual mapping for a shmem GEM object
- * @shmem: shmem GEM object
- * @map: Kernel virtual address where the SHMEM GEM object was mapped
- *
- * This function cleans up a kernel virtual address mapping acquired by
- * drm_gem_shmem_vmap(). The mapping is only removed when the use count drops to
- * zero.
- *
- * This function hides the differences between dma-buf imported and natively
- * allocated objects.
- */
-void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
-			  struct iosys_map *map)
-{
-	mutex_lock(&shmem->vmap_lock);
-	drm_gem_shmem_vunmap_locked(shmem, map);
-	mutex_unlock(&shmem->vmap_lock);
-}
 EXPORT_SYMBOL(drm_gem_shmem_vunmap);
 
 static struct drm_gem_shmem_object *
@@ -441,24 +396,24 @@ drm_gem_shmem_create_with_handle(struct drm_file *file_priv,
  */
 int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
 {
-	mutex_lock(&shmem->pages_lock);
+	dma_resv_assert_held(shmem->base.resv);
 
 	if (shmem->madv >= 0)
 		shmem->madv = madv;
 
 	madv = shmem->madv;
 
-	mutex_unlock(&shmem->pages_lock);
-
 	return (madv >= 0);
 }
 EXPORT_SYMBOL(drm_gem_shmem_madvise);
 
-void drm_gem_shmem_purge_locked(struct drm_gem_shmem_object *shmem)
+void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
 {
 	struct drm_gem_object *obj = &shmem->base;
 	struct drm_device *dev = obj->dev;
 
+	dma_resv_assert_held(shmem->base.resv);
+
 	WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
 
 	dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
@@ -466,7 +421,7 @@ void drm_gem_shmem_purge_locked(struct drm_gem_shmem_object *shmem)
 	kfree(shmem->sgt);
 	shmem->sgt = NULL;
 
-	drm_gem_shmem_put_pages_locked(shmem);
+	drm_gem_shmem_put_pages(shmem);
 
 	shmem->madv = -1;
 
@@ -482,17 +437,6 @@ void drm_gem_shmem_purge_locked(struct drm_gem_shmem_object *shmem)
 
 	invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
 }
-EXPORT_SYMBOL(drm_gem_shmem_purge_locked);
-
-bool drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
-{
-	if (!mutex_trylock(&shmem->pages_lock))
-		return false;
-	drm_gem_shmem_purge_locked(shmem);
-	mutex_unlock(&shmem->pages_lock);
-
-	return true;
-}
 EXPORT_SYMBOL(drm_gem_shmem_purge);
 
 /**
@@ -548,7 +492,7 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
 	/* We don't use vmf->pgoff since that has the fake offset */
 	page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
 
-	mutex_lock(&shmem->pages_lock);
+	dma_resv_lock(shmem->base.resv, NULL);
 
 	if (page_offset >= num_pages ||
 	    WARN_ON_ONCE(!shmem->pages) ||
@@ -560,7 +504,7 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
 		ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
 	}
 
-	mutex_unlock(&shmem->pages_lock);
+	dma_resv_unlock(shmem->base.resv);
 
 	return ret;
 }
@@ -573,8 +517,10 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
 
 	WARN_ON(shmem->base.import_attach);
 
+	dma_resv_lock(shmem->base.resv, NULL);
 	ret = drm_gem_shmem_get_pages(shmem);
 	WARN_ON_ONCE(ret != 0);
+	dma_resv_unlock(shmem->base.resv);
 
 	drm_gem_vm_open(vma);
 }
@@ -584,7 +530,10 @@ static void drm_gem_shmem_vm_close(struct vm_area_struct *vma)
 	struct drm_gem_object *obj = vma->vm_private_data;
 	struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
 
+	dma_resv_lock(shmem->base.resv, NULL);
 	drm_gem_shmem_put_pages(shmem);
+	dma_resv_unlock(shmem->base.resv);
+
 	drm_gem_vm_close(vma);
 }
 
@@ -700,9 +649,11 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
 
 	WARN_ON(obj->import_attach);
 
+	dma_resv_lock(shmem->base.resv, NULL);
+
 	ret = drm_gem_shmem_get_pages(shmem);
 	if (ret)
-		return ERR_PTR(ret);
+		goto err_unlock;
 
 	sgt = drm_gem_shmem_get_sg_table(shmem);
 	if (IS_ERR(sgt)) {
@@ -716,6 +667,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
 
 	shmem->sgt = sgt;
 
+	dma_resv_unlock(shmem->base.resv);
+
 	return sgt;
 
 err_free_sgt:
@@ -723,6 +676,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
 	kfree(sgt);
 err_put_pages:
 	drm_gem_shmem_put_pages(shmem);
+err_unlock:
+	dma_resv_unlock(shmem->base.resv);
 	return ERR_PTR(ret);
 }
 EXPORT_SYMBOL_GPL(drm_gem_shmem_get_pages_sgt);
diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c
index 0f1ca0b0db49..5008f0c2428f 100644
--- a/drivers/gpu/drm/lima/lima_gem.c
+++ b/drivers/gpu/drm/lima/lima_gem.c
@@ -34,7 +34,7 @@ int lima_heap_alloc(struct lima_bo *bo, struct lima_vm *vm)
 
 	new_size = min(new_size, bo->base.base.size);
 
-	mutex_lock(&bo->base.pages_lock);
+	dma_resv_lock(bo->base.base.resv, NULL);
 
 	if (bo->base.pages) {
 		pages = bo->base.pages;
@@ -42,7 +42,7 @@ int lima_heap_alloc(struct lima_bo *bo, struct lima_vm *vm)
 		pages = kvmalloc_array(bo->base.base.size >> PAGE_SHIFT,
 				       sizeof(*pages), GFP_KERNEL | __GFP_ZERO);
 		if (!pages) {
-			mutex_unlock(&bo->base.pages_lock);
+			dma_resv_unlock(bo->base.base.resv);
 			return -ENOMEM;
 		}
 
@@ -56,13 +56,13 @@ int lima_heap_alloc(struct lima_bo *bo, struct lima_vm *vm)
 		struct page *page = shmem_read_mapping_page(mapping, i);
 
 		if (IS_ERR(page)) {
-			mutex_unlock(&bo->base.pages_lock);
+			dma_resv_unlock(bo->base.base.resv);
 			return PTR_ERR(page);
 		}
 		pages[i] = page;
 	}
 
-	mutex_unlock(&bo->base.pages_lock);
+	dma_resv_unlock(bo->base.base.resv);
 
 	ret = sg_alloc_table_from_pages(&sgt, pages, i, 0,
 					new_size, GFP_KERNEL);
diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
index e82931712d8a..ff003403fbbc 100644
--- a/drivers/gpu/drm/lima/lima_sched.c
+++ b/drivers/gpu/drm/lima/lima_sched.c
@@ -371,7 +371,7 @@ static void lima_sched_build_error_task_list(struct lima_sched_task *task)
 		} else {
 			buffer_chunk->size = lima_bo_size(bo);
 
-			ret = drm_gem_shmem_vmap(&bo->base, &map);
+			ret = drm_gem_vmap_unlocked(&bo->base.base, &map);
 			if (ret) {
 				kvfree(et);
 				goto out;
@@ -379,7 +379,7 @@ static void lima_sched_build_error_task_list(struct lima_sched_task *task)
 
 			memcpy(buffer_chunk + 1, map.vaddr, buffer_chunk->size);
 
-			drm_gem_shmem_vunmap(&bo->base, &map);
+			drm_gem_vunmap_unlocked(&bo->base.base, &map);
 		}
 
 		buffer_chunk = (void *)(buffer_chunk + 1) + buffer_chunk->size;
diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
index b1e6d238674f..859e240161d1 100644
--- a/drivers/gpu/drm/panfrost/panfrost_drv.c
+++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
@@ -405,6 +405,10 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
 
 	bo = to_panfrost_bo(gem_obj);
 
+	ret = dma_resv_lock_interruptible(bo->base.base.resv, NULL);
+	if (ret)
+		goto out_put_object;
+
 	mutex_lock(&pfdev->shrinker_lock);
 	mutex_lock(&bo->mappings.lock);
 	if (args->madv == PANFROST_MADV_DONTNEED) {
@@ -442,7 +446,8 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
 out_unlock_mappings:
 	mutex_unlock(&bo->mappings.lock);
 	mutex_unlock(&pfdev->shrinker_lock);
-
+	dma_resv_unlock(bo->base.base.resv);
+out_put_object:
 	drm_gem_object_put(gem_obj);
 	return ret;
 }
diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
index 77e7cb6d1ae3..a4bedfeb2ec4 100644
--- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
+++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
@@ -48,14 +48,14 @@ static bool panfrost_gem_purge(struct drm_gem_object *obj)
 	if (!mutex_trylock(&bo->mappings.lock))
 		return false;
 
-	if (!mutex_trylock(&shmem->pages_lock))
+	if (!dma_resv_trylock(shmem->base.resv))
 		goto unlock_mappings;
 
 	panfrost_gem_teardown_mappings_locked(bo);
-	drm_gem_shmem_purge_locked(&bo->base);
+	drm_gem_shmem_purge(&bo->base);
 	ret = true;
 
-	mutex_unlock(&shmem->pages_lock);
+	dma_resv_unlock(shmem->base.resv);
 
 unlock_mappings:
 	mutex_unlock(&bo->mappings.lock);
diff --git a/drivers/gpu/drm/panfrost/panfrost_mmu.c b/drivers/gpu/drm/panfrost/panfrost_mmu.c
index b285a8001b1d..e164017e84cd 100644
--- a/drivers/gpu/drm/panfrost/panfrost_mmu.c
+++ b/drivers/gpu/drm/panfrost/panfrost_mmu.c
@@ -424,6 +424,7 @@ static int panfrost_mmu_map_fault_addr(struct panfrost_device *pfdev, int as,
 	struct panfrost_gem_mapping *bomapping;
 	struct panfrost_gem_object *bo;
 	struct address_space *mapping;
+	struct drm_gem_object *obj;
 	pgoff_t page_offset;
 	struct sg_table *sgt;
 	struct page **pages;
@@ -446,15 +447,16 @@ static int panfrost_mmu_map_fault_addr(struct panfrost_device *pfdev, int as,
 	page_offset = addr >> PAGE_SHIFT;
 	page_offset -= bomapping->mmnode.start;
 
-	mutex_lock(&bo->base.pages_lock);
+	obj = &bo->base.base;
+
+	dma_resv_lock(obj->resv, NULL);
 
 	if (!bo->base.pages) {
 		bo->sgts = kvmalloc_array(bo->base.base.size / SZ_2M,
 				     sizeof(struct sg_table), GFP_KERNEL | __GFP_ZERO);
 		if (!bo->sgts) {
-			mutex_unlock(&bo->base.pages_lock);
 			ret = -ENOMEM;
-			goto err_bo;
+			goto err_unlock;
 		}
 
 		pages = kvmalloc_array(bo->base.base.size >> PAGE_SHIFT,
@@ -462,9 +464,8 @@ static int panfrost_mmu_map_fault_addr(struct panfrost_device *pfdev, int as,
 		if (!pages) {
 			kvfree(bo->sgts);
 			bo->sgts = NULL;
-			mutex_unlock(&bo->base.pages_lock);
 			ret = -ENOMEM;
-			goto err_bo;
+			goto err_unlock;
 		}
 		bo->base.pages = pages;
 		bo->base.pages_use_count = 1;
@@ -472,7 +473,6 @@ static int panfrost_mmu_map_fault_addr(struct panfrost_device *pfdev, int as,
 		pages = bo->base.pages;
 		if (pages[page_offset]) {
 			/* Pages are already mapped, bail out. */
-			mutex_unlock(&bo->base.pages_lock);
 			goto out;
 		}
 	}
@@ -483,14 +483,11 @@ static int panfrost_mmu_map_fault_addr(struct panfrost_device *pfdev, int as,
 	for (i = page_offset; i < page_offset + NUM_FAULT_PAGES; i++) {
 		pages[i] = shmem_read_mapping_page(mapping, i);
 		if (IS_ERR(pages[i])) {
-			mutex_unlock(&bo->base.pages_lock);
 			ret = PTR_ERR(pages[i]);
 			goto err_pages;
 		}
 	}
 
-	mutex_unlock(&bo->base.pages_lock);
-
 	sgt = &bo->sgts[page_offset / (SZ_2M / PAGE_SIZE)];
 	ret = sg_alloc_table_from_pages(sgt, pages + page_offset,
 					NUM_FAULT_PAGES, 0, SZ_2M, GFP_KERNEL);
@@ -509,6 +506,8 @@ static int panfrost_mmu_map_fault_addr(struct panfrost_device *pfdev, int as,
 	dev_dbg(pfdev->dev, "mapped page fault @ AS%d %llx", as, addr);
 
 out:
+	dma_resv_unlock(obj->resv);
+
 	panfrost_gem_mapping_put(bomapping);
 
 	return 0;
@@ -517,6 +516,8 @@ static int panfrost_mmu_map_fault_addr(struct panfrost_device *pfdev, int as,
 	sg_free_table(sgt);
 err_pages:
 	drm_gem_shmem_put_pages(&bo->base);
+err_unlock:
+	dma_resv_unlock(obj->resv);
 err_bo:
 	panfrost_gem_mapping_put(bomapping);
 	return ret;
diff --git a/drivers/gpu/drm/panfrost/panfrost_perfcnt.c b/drivers/gpu/drm/panfrost/panfrost_perfcnt.c
index bc0df93f7f21..ba9b6e2b2636 100644
--- a/drivers/gpu/drm/panfrost/panfrost_perfcnt.c
+++ b/drivers/gpu/drm/panfrost/panfrost_perfcnt.c
@@ -106,7 +106,7 @@ static int panfrost_perfcnt_enable_locked(struct panfrost_device *pfdev,
 		goto err_close_bo;
 	}
 
-	ret = drm_gem_shmem_vmap(bo, &map);
+	ret = drm_gem_vmap_unlocked(&bo->base, &map);
 	if (ret)
 		goto err_put_mapping;
 	perfcnt->buf = map.vaddr;
@@ -165,7 +165,7 @@ static int panfrost_perfcnt_enable_locked(struct panfrost_device *pfdev,
 	return 0;
 
 err_vunmap:
-	drm_gem_shmem_vunmap(bo, &map);
+	drm_gem_vunmap_unlocked(&bo->base, &map);
 err_put_mapping:
 	panfrost_gem_mapping_put(perfcnt->mapping);
 err_close_bo:
@@ -195,7 +195,7 @@ static int panfrost_perfcnt_disable_locked(struct panfrost_device *pfdev,
 		  GPU_PERFCNT_CFG_MODE(GPU_PERFCNT_CFG_MODE_OFF));
 
 	perfcnt->user = NULL;
-	drm_gem_shmem_vunmap(&perfcnt->mapping->obj->base, &map);
+	drm_gem_vunmap_unlocked(&perfcnt->mapping->obj->base.base, &map);
 	perfcnt->buf = NULL;
 	panfrost_gem_close(&perfcnt->mapping->obj->base.base, file_priv);
 	panfrost_mmu_as_put(pfdev, perfcnt->mapping->mmu);
diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
index d0a57853c188..9a8983ee8abe 100644
--- a/include/drm/drm_gem_shmem_helper.h
+++ b/include/drm/drm_gem_shmem_helper.h
@@ -26,11 +26,6 @@ struct drm_gem_shmem_object {
 	 */
 	struct drm_gem_object base;
 
-	/**
-	 * @pages_lock: Protects the page table and use count
-	 */
-	struct mutex pages_lock;
-
 	/**
 	 * @pages: Page table
 	 */
@@ -79,11 +74,6 @@ struct drm_gem_shmem_object {
 	 */
 	struct sg_table *sgt;
 
-	/**
-	 * @vmap_lock: Protects the vmap address and use count
-	 */
-	struct mutex vmap_lock;
-
 	/**
 	 * @vaddr: Kernel virtual address of the backing memory
 	 */
@@ -109,7 +99,6 @@ struct drm_gem_shmem_object {
 struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t size);
 void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem);
 
-int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem);
 void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem);
 int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem);
 void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem);
@@ -128,8 +117,7 @@ static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem
 		!shmem->base.dma_buf && !shmem->base.import_attach;
 }
 
-void drm_gem_shmem_purge_locked(struct drm_gem_shmem_object *shmem);
-bool drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
+void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
 
 struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
 struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
-- 
2.35.3

[PATCH v6 16/22] drm/shmem-helper: Use reservation lock

[Dmitry Osipenko]

Replace drm_gem_shmem locks with GEM reservation lock to make drm-shmem
locks consistent with the new locking convention of dma-bufs which tells
that dma-buf importers are responsible for holding reservation lock for
all operations performed over dma-bufs. This prepares drm-shmem code for
addition of the generic shmem shrinker framework.

Suggested-by: Daniel Vetter <daniel@ffwll.ch>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/drm_gem_shmem_helper.c        | 181 +++++++-----------
 drivers/gpu/drm/lima/lima_gem.c               |   8 +-
 drivers/gpu/drm/lima/lima_sched.c             |   4 +-
 drivers/gpu/drm/panfrost/panfrost_drv.c       |   7 +-
 .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   6 +-
 drivers/gpu/drm/panfrost/panfrost_mmu.c       |  19 +-
 drivers/gpu/drm/panfrost/panfrost_perfcnt.c   |   6 +-
 include/drm/drm_gem_shmem_helper.h            |  14 +-
 8 files changed, 97 insertions(+), 148 deletions(-)

diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
index fd2647690bf7..555fe212bd98 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -86,8 +86,6 @@ __drm_gem_shmem_create(struct drm_device *dev, size_t size, bool private)
 	if (ret)
 		goto err_release;
 
-	mutex_init(&shmem->pages_lock);
-	mutex_init(&shmem->vmap_lock);
 	INIT_LIST_HEAD(&shmem->madv_list);
 
 	if (!private) {
@@ -139,11 +137,13 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
 {
 	struct drm_gem_object *obj = &shmem->base;
 
-	WARN_ON(shmem->vmap_use_count);
-
 	if (obj->import_attach) {
 		drm_prime_gem_destroy(obj, shmem->sgt);
 	} else {
+		dma_resv_lock(shmem->base.resv, NULL);
+
+		WARN_ON(shmem->vmap_use_count);
+
 		if (shmem->sgt) {
 			dma_unmap_sgtable(obj->dev->dev, shmem->sgt,
 					  DMA_BIDIRECTIONAL, 0);
@@ -152,18 +152,18 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
 		}
 		if (shmem->pages)
 			drm_gem_shmem_put_pages(shmem);
-	}
 
-	WARN_ON(shmem->pages_use_count);
+		WARN_ON(shmem->pages_use_count);
+
+		dma_resv_unlock(shmem->base.resv);
+	}
 
 	drm_gem_object_release(obj);
-	mutex_destroy(&shmem->pages_lock);
-	mutex_destroy(&shmem->vmap_lock);
 	kfree(shmem);
 }
 EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
 
-static int drm_gem_shmem_get_pages_locked(struct drm_gem_shmem_object *shmem)
+static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
 {
 	struct drm_gem_object *obj = &shmem->base;
 	struct page **pages;
@@ -194,35 +194,17 @@ static int drm_gem_shmem_get_pages_locked(struct drm_gem_shmem_object *shmem)
 }
 
 /*
- * drm_gem_shmem_get_pages - Allocate backing pages for a shmem GEM object
+ * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
  * @shmem: shmem GEM object
  *
- * This function makes sure that backing pages exists for the shmem GEM object
- * and increases the use count.
- *
- * Returns:
- * 0 on success or a negative error code on failure.
+ * This function decreases the use count and puts the backing pages when use drops to zero.
  */
-int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
-{
-	int ret;
-
-	WARN_ON(shmem->base.import_attach);
-
-	ret = mutex_lock_interruptible(&shmem->pages_lock);
-	if (ret)
-		return ret;
-	ret = drm_gem_shmem_get_pages_locked(shmem);
-	mutex_unlock(&shmem->pages_lock);
-
-	return ret;
-}
-EXPORT_SYMBOL(drm_gem_shmem_get_pages);
-
-static void drm_gem_shmem_put_pages_locked(struct drm_gem_shmem_object *shmem)
+void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
 {
 	struct drm_gem_object *obj = &shmem->base;
 
+	dma_resv_assert_held(shmem->base.resv);
+
 	if (WARN_ON_ONCE(!shmem->pages_use_count))
 		return;
 
@@ -239,19 +221,6 @@ static void drm_gem_shmem_put_pages_locked(struct drm_gem_shmem_object *shmem)
 			  shmem->pages_mark_accessed_on_put);
 	shmem->pages = NULL;
 }
-
-/*
- * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
- * @shmem: shmem GEM object
- *
- * This function decreases the use count and puts the backing pages when use drops to zero.
- */
-void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
-{
-	mutex_lock(&shmem->pages_lock);
-	drm_gem_shmem_put_pages_locked(shmem);
-	mutex_unlock(&shmem->pages_lock);
-}
 EXPORT_SYMBOL(drm_gem_shmem_put_pages);
 
 /**
@@ -266,6 +235,8 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
  */
 int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
 {
+	dma_resv_assert_held(shmem->base.resv);
+
 	WARN_ON(shmem->base.import_attach);
 
 	return drm_gem_shmem_get_pages(shmem);
@@ -281,14 +252,31 @@ EXPORT_SYMBOL(drm_gem_shmem_pin);
  */
 void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
 {
+	dma_resv_assert_held(shmem->base.resv);
+
 	WARN_ON(shmem->base.import_attach);
 
 	drm_gem_shmem_put_pages(shmem);
 }
 EXPORT_SYMBOL(drm_gem_shmem_unpin);
 
-static int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object *shmem,
-				     struct iosys_map *map)
+/*
+ * drm_gem_shmem_vmap - Create a virtual mapping for a shmem GEM object
+ * @shmem: shmem GEM object
+ * @map: Returns the kernel virtual address of the SHMEM GEM object's backing
+ *       store.
+ *
+ * This function makes sure that a contiguous kernel virtual address mapping
+ * exists for the buffer backing the shmem GEM object. It hides the differences
+ * between dma-buf imported and natively allocated objects.
+ *
+ * Acquired mappings should be cleaned up by calling drm_gem_shmem_vunmap().
+ *
+ * Returns:
+ * 0 on success or a negative error code on failure.
+ */
+int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
+		       struct iosys_map *map)
 {
 	struct drm_gem_object *obj = &shmem->base;
 	int ret = 0;
@@ -304,6 +292,8 @@ static int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object *shmem,
 	} else {
 		pgprot_t prot = PAGE_KERNEL;
 
+		dma_resv_assert_held(shmem->base.resv);
+
 		if (shmem->vmap_use_count++ > 0) {
 			iosys_map_set_vaddr(map, shmem->vaddr);
 			return 0;
@@ -338,45 +328,30 @@ static int drm_gem_shmem_vmap_locked(struct drm_gem_shmem_object *shmem,
 
 	return ret;
 }
+EXPORT_SYMBOL(drm_gem_shmem_vmap);
 
 /*
- * drm_gem_shmem_vmap - Create a virtual mapping for a shmem GEM object
+ * drm_gem_shmem_vunmap - Unmap a virtual mapping for a shmem GEM object
  * @shmem: shmem GEM object
- * @map: Returns the kernel virtual address of the SHMEM GEM object's backing
- *       store.
- *
- * This function makes sure that a contiguous kernel virtual address mapping
- * exists for the buffer backing the shmem GEM object. It hides the differences
- * between dma-buf imported and natively allocated objects.
+ * @map: Kernel virtual address where the SHMEM GEM object was mapped
  *
- * Acquired mappings should be cleaned up by calling drm_gem_shmem_vunmap().
+ * This function cleans up a kernel virtual address mapping acquired by
+ * drm_gem_shmem_vmap(). The mapping is only removed when the use count drops to
+ * zero.
  *
- * Returns:
- * 0 on success or a negative error code on failure.
+ * This function hides the differences between dma-buf imported and natively
+ * allocated objects.
  */
-int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
-		       struct iosys_map *map)
-{
-	int ret;
-
-	ret = mutex_lock_interruptible(&shmem->vmap_lock);
-	if (ret)
-		return ret;
-	ret = drm_gem_shmem_vmap_locked(shmem, map);
-	mutex_unlock(&shmem->vmap_lock);
-
-	return ret;
-}
-EXPORT_SYMBOL(drm_gem_shmem_vmap);
-
-static void drm_gem_shmem_vunmap_locked(struct drm_gem_shmem_object *shmem,
-					struct iosys_map *map)
+void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
+			  struct iosys_map *map)
 {
 	struct drm_gem_object *obj = &shmem->base;
 
 	if (obj->import_attach) {
 		dma_buf_vunmap(obj->import_attach->dmabuf, map);
 	} else {
+		dma_resv_assert_held(shmem->base.resv);
+
 		if (WARN_ON_ONCE(!shmem->vmap_use_count))
 			return;
 
@@ -389,26 +364,6 @@ static void drm_gem_shmem_vunmap_locked(struct drm_gem_shmem_object *shmem,
 
 	shmem->vaddr = NULL;
 }
-
-/*
- * drm_gem_shmem_vunmap - Unmap a virtual mapping for a shmem GEM object
- * @shmem: shmem GEM object
- * @map: Kernel virtual address where the SHMEM GEM object was mapped
- *
- * This function cleans up a kernel virtual address mapping acquired by
- * drm_gem_shmem_vmap(). The mapping is only removed when the use count drops to
- * zero.
- *
- * This function hides the differences between dma-buf imported and natively
- * allocated objects.
- */
-void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
-			  struct iosys_map *map)
-{
-	mutex_lock(&shmem->vmap_lock);
-	drm_gem_shmem_vunmap_locked(shmem, map);
-	mutex_unlock(&shmem->vmap_lock);
-}
 EXPORT_SYMBOL(drm_gem_shmem_vunmap);
 
 static struct drm_gem_shmem_object *
@@ -441,24 +396,24 @@ drm_gem_shmem_create_with_handle(struct drm_file *file_priv,
  */
 int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
 {
-	mutex_lock(&shmem->pages_lock);
+	dma_resv_assert_held(shmem->base.resv);
 
 	if (shmem->madv >= 0)
 		shmem->madv = madv;
 
 	madv = shmem->madv;
 
-	mutex_unlock(&shmem->pages_lock);
-
 	return (madv >= 0);
 }
 EXPORT_SYMBOL(drm_gem_shmem_madvise);
 
-void drm_gem_shmem_purge_locked(struct drm_gem_shmem_object *shmem)
+void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
 {
 	struct drm_gem_object *obj = &shmem->base;
 	struct drm_device *dev = obj->dev;
 
+	dma_resv_assert_held(shmem->base.resv);
+
 	WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
 
 	dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
@@ -466,7 +421,7 @@ void drm_gem_shmem_purge_locked(struct drm_gem_shmem_object *shmem)
 	kfree(shmem->sgt);
 	shmem->sgt = NULL;
 
-	drm_gem_shmem_put_pages_locked(shmem);
+	drm_gem_shmem_put_pages(shmem);
 
 	shmem->madv = -1;
 
@@ -482,17 +437,6 @@ void drm_gem_shmem_purge_locked(struct drm_gem_shmem_object *shmem)
 
 	invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
 }
-EXPORT_SYMBOL(drm_gem_shmem_purge_locked);
-
-bool drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
-{
-	if (!mutex_trylock(&shmem->pages_lock))
-		return false;
-	drm_gem_shmem_purge_locked(shmem);
-	mutex_unlock(&shmem->pages_lock);
-
-	return true;
-}
 EXPORT_SYMBOL(drm_gem_shmem_purge);
 
 /**
@@ -548,7 +492,7 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
 	/* We don't use vmf->pgoff since that has the fake offset */
 	page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
 
-	mutex_lock(&shmem->pages_lock);
+	dma_resv_lock(shmem->base.resv, NULL);
 
 	if (page_offset >= num_pages ||
 	    WARN_ON_ONCE(!shmem->pages) ||
@@ -560,7 +504,7 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
 		ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
 	}
 
-	mutex_unlock(&shmem->pages_lock);
+	dma_resv_unlock(shmem->base.resv);
 
 	return ret;
 }
@@ -573,8 +517,10 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
 
 	WARN_ON(shmem->base.import_attach);
 
+	dma_resv_lock(shmem->base.resv, NULL);
 	ret = drm_gem_shmem_get_pages(shmem);
 	WARN_ON_ONCE(ret != 0);
+	dma_resv_unlock(shmem->base.resv);
 
 	drm_gem_vm_open(vma);
 }
@@ -584,7 +530,10 @@ static void drm_gem_shmem_vm_close(struct vm_area_struct *vma)
 	struct drm_gem_object *obj = vma->vm_private_data;
 	struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
 
+	dma_resv_lock(shmem->base.resv, NULL);
 	drm_gem_shmem_put_pages(shmem);
+	dma_resv_unlock(shmem->base.resv);
+
 	drm_gem_vm_close(vma);
 }
 
@@ -700,9 +649,11 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
 
 	WARN_ON(obj->import_attach);
 
+	dma_resv_lock(shmem->base.resv, NULL);
+
 	ret = drm_gem_shmem_get_pages(shmem);
 	if (ret)
-		return ERR_PTR(ret);
+		goto err_unlock;
 
 	sgt = drm_gem_shmem_get_sg_table(shmem);
 	if (IS_ERR(sgt)) {
@@ -716,6 +667,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
 
 	shmem->sgt = sgt;
 
+	dma_resv_unlock(shmem->base.resv);
+
 	return sgt;
 
 err_free_sgt:
@@ -723,6 +676,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
 	kfree(sgt);
 err_put_pages:
 	drm_gem_shmem_put_pages(shmem);
+err_unlock:
+	dma_resv_unlock(shmem->base.resv);
 	return ERR_PTR(ret);
 }
 EXPORT_SYMBOL_GPL(drm_gem_shmem_get_pages_sgt);
diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c
index 0f1ca0b0db49..5008f0c2428f 100644
--- a/drivers/gpu/drm/lima/lima_gem.c
+++ b/drivers/gpu/drm/lima/lima_gem.c
@@ -34,7 +34,7 @@ int lima_heap_alloc(struct lima_bo *bo, struct lima_vm *vm)
 
 	new_size = min(new_size, bo->base.base.size);
 
-	mutex_lock(&bo->base.pages_lock);
+	dma_resv_lock(bo->base.base.resv, NULL);
 
 	if (bo->base.pages) {
 		pages = bo->base.pages;
@@ -42,7 +42,7 @@ int lima_heap_alloc(struct lima_bo *bo, struct lima_vm *vm)
 		pages = kvmalloc_array(bo->base.base.size >> PAGE_SHIFT,
 				       sizeof(*pages), GFP_KERNEL | __GFP_ZERO);
 		if (!pages) {
-			mutex_unlock(&bo->base.pages_lock);
+			dma_resv_unlock(bo->base.base.resv);
 			return -ENOMEM;
 		}
 
@@ -56,13 +56,13 @@ int lima_heap_alloc(struct lima_bo *bo, struct lima_vm *vm)
 		struct page *page = shmem_read_mapping_page(mapping, i);
 
 		if (IS_ERR(page)) {
-			mutex_unlock(&bo->base.pages_lock);
+			dma_resv_unlock(bo->base.base.resv);
 			return PTR_ERR(page);
 		}
 		pages[i] = page;
 	}
 
-	mutex_unlock(&bo->base.pages_lock);
+	dma_resv_unlock(bo->base.base.resv);
 
 	ret = sg_alloc_table_from_pages(&sgt, pages, i, 0,
 					new_size, GFP_KERNEL);
diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
index e82931712d8a..ff003403fbbc 100644
--- a/drivers/gpu/drm/lima/lima_sched.c
+++ b/drivers/gpu/drm/lima/lima_sched.c
@@ -371,7 +371,7 @@ static void lima_sched_build_error_task_list(struct lima_sched_task *task)
 		} else {
 			buffer_chunk->size = lima_bo_size(bo);
 
-			ret = drm_gem_shmem_vmap(&bo->base, &map);
+			ret = drm_gem_vmap_unlocked(&bo->base.base, &map);
 			if (ret) {
 				kvfree(et);
 				goto out;
@@ -379,7 +379,7 @@ static void lima_sched_build_error_task_list(struct lima_sched_task *task)
 
 			memcpy(buffer_chunk + 1, map.vaddr, buffer_chunk->size);
 
-			drm_gem_shmem_vunmap(&bo->base, &map);
+			drm_gem_vunmap_unlocked(&bo->base.base, &map);
 		}
 
 		buffer_chunk = (void *)(buffer_chunk + 1) + buffer_chunk->size;
diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
index b1e6d238674f..859e240161d1 100644
--- a/drivers/gpu/drm/panfrost/panfrost_drv.c
+++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
@@ -405,6 +405,10 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
 
 	bo = to_panfrost_bo(gem_obj);
 
+	ret = dma_resv_lock_interruptible(bo->base.base.resv, NULL);
+	if (ret)
+		goto out_put_object;
+
 	mutex_lock(&pfdev->shrinker_lock);
 	mutex_lock(&bo->mappings.lock);
 	if (args->madv == PANFROST_MADV_DONTNEED) {
@@ -442,7 +446,8 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
 out_unlock_mappings:
 	mutex_unlock(&bo->mappings.lock);
 	mutex_unlock(&pfdev->shrinker_lock);
-
+	dma_resv_unlock(bo->base.base.resv);
+out_put_object:
 	drm_gem_object_put(gem_obj);
 	return ret;
 }
diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
index 77e7cb6d1ae3..a4bedfeb2ec4 100644
--- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
+++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
@@ -48,14 +48,14 @@ static bool panfrost_gem_purge(struct drm_gem_object *obj)
 	if (!mutex_trylock(&bo->mappings.lock))
 		return false;
 
-	if (!mutex_trylock(&shmem->pages_lock))
+	if (!dma_resv_trylock(shmem->base.resv))
 		goto unlock_mappings;
 
 	panfrost_gem_teardown_mappings_locked(bo);
-	drm_gem_shmem_purge_locked(&bo->base);
+	drm_gem_shmem_purge(&bo->base);
 	ret = true;
 
-	mutex_unlock(&shmem->pages_lock);
+	dma_resv_unlock(shmem->base.resv);
 
 unlock_mappings:
 	mutex_unlock(&bo->mappings.lock);
diff --git a/drivers/gpu/drm/panfrost/panfrost_mmu.c b/drivers/gpu/drm/panfrost/panfrost_mmu.c
index b285a8001b1d..e164017e84cd 100644
--- a/drivers/gpu/drm/panfrost/panfrost_mmu.c
+++ b/drivers/gpu/drm/panfrost/panfrost_mmu.c
@@ -424,6 +424,7 @@ static int panfrost_mmu_map_fault_addr(struct panfrost_device *pfdev, int as,
 	struct panfrost_gem_mapping *bomapping;
 	struct panfrost_gem_object *bo;
 	struct address_space *mapping;
+	struct drm_gem_object *obj;
 	pgoff_t page_offset;
 	struct sg_table *sgt;
 	struct page **pages;
@@ -446,15 +447,16 @@ static int panfrost_mmu_map_fault_addr(struct panfrost_device *pfdev, int as,
 	page_offset = addr >> PAGE_SHIFT;
 	page_offset -= bomapping->mmnode.start;
 
-	mutex_lock(&bo->base.pages_lock);
+	obj = &bo->base.base;
+
+	dma_resv_lock(obj->resv, NULL);
 
 	if (!bo->base.pages) {
 		bo->sgts = kvmalloc_array(bo->base.base.size / SZ_2M,
 				     sizeof(struct sg_table), GFP_KERNEL | __GFP_ZERO);
 		if (!bo->sgts) {
-			mutex_unlock(&bo->base.pages_lock);
 			ret = -ENOMEM;
-			goto err_bo;
+			goto err_unlock;
 		}
 
 		pages = kvmalloc_array(bo->base.base.size >> PAGE_SHIFT,
@@ -462,9 +464,8 @@ static int panfrost_mmu_map_fault_addr(struct panfrost_device *pfdev, int as,
 		if (!pages) {
 			kvfree(bo->sgts);
 			bo->sgts = NULL;
-			mutex_unlock(&bo->base.pages_lock);
 			ret = -ENOMEM;
-			goto err_bo;
+			goto err_unlock;
 		}
 		bo->base.pages = pages;
 		bo->base.pages_use_count = 1;
@@ -472,7 +473,6 @@ static int panfrost_mmu_map_fault_addr(struct panfrost_device *pfdev, int as,
 		pages = bo->base.pages;
 		if (pages[page_offset]) {
 			/* Pages are already mapped, bail out. */
-			mutex_unlock(&bo->base.pages_lock);
 			goto out;
 		}
 	}
@@ -483,14 +483,11 @@ static int panfrost_mmu_map_fault_addr(struct panfrost_device *pfdev, int as,
 	for (i = page_offset; i < page_offset + NUM_FAULT_PAGES; i++) {
 		pages[i] = shmem_read_mapping_page(mapping, i);
 		if (IS_ERR(pages[i])) {
-			mutex_unlock(&bo->base.pages_lock);
 			ret = PTR_ERR(pages[i]);
 			goto err_pages;
 		}
 	}
 
-	mutex_unlock(&bo->base.pages_lock);
-
 	sgt = &bo->sgts[page_offset / (SZ_2M / PAGE_SIZE)];
 	ret = sg_alloc_table_from_pages(sgt, pages + page_offset,
 					NUM_FAULT_PAGES, 0, SZ_2M, GFP_KERNEL);
@@ -509,6 +506,8 @@ static int panfrost_mmu_map_fault_addr(struct panfrost_device *pfdev, int as,
 	dev_dbg(pfdev->dev, "mapped page fault @ AS%d %llx", as, addr);
 
 out:
+	dma_resv_unlock(obj->resv);
+
 	panfrost_gem_mapping_put(bomapping);
 
 	return 0;
@@ -517,6 +516,8 @@ static int panfrost_mmu_map_fault_addr(struct panfrost_device *pfdev, int as,
 	sg_free_table(sgt);
 err_pages:
 	drm_gem_shmem_put_pages(&bo->base);
+err_unlock:
+	dma_resv_unlock(obj->resv);
 err_bo:
 	panfrost_gem_mapping_put(bomapping);
 	return ret;
diff --git a/drivers/gpu/drm/panfrost/panfrost_perfcnt.c b/drivers/gpu/drm/panfrost/panfrost_perfcnt.c
index bc0df93f7f21..ba9b6e2b2636 100644
--- a/drivers/gpu/drm/panfrost/panfrost_perfcnt.c
+++ b/drivers/gpu/drm/panfrost/panfrost_perfcnt.c
@@ -106,7 +106,7 @@ static int panfrost_perfcnt_enable_locked(struct panfrost_device *pfdev,
 		goto err_close_bo;
 	}
 
-	ret = drm_gem_shmem_vmap(bo, &map);
+	ret = drm_gem_vmap_unlocked(&bo->base, &map);
 	if (ret)
 		goto err_put_mapping;
 	perfcnt->buf = map.vaddr;
@@ -165,7 +165,7 @@ static int panfrost_perfcnt_enable_locked(struct panfrost_device *pfdev,
 	return 0;
 
 err_vunmap:
-	drm_gem_shmem_vunmap(bo, &map);
+	drm_gem_vunmap_unlocked(&bo->base, &map);
 err_put_mapping:
 	panfrost_gem_mapping_put(perfcnt->mapping);
 err_close_bo:
@@ -195,7 +195,7 @@ static int panfrost_perfcnt_disable_locked(struct panfrost_device *pfdev,
 		  GPU_PERFCNT_CFG_MODE(GPU_PERFCNT_CFG_MODE_OFF));
 
 	perfcnt->user = NULL;
-	drm_gem_shmem_vunmap(&perfcnt->mapping->obj->base, &map);
+	drm_gem_vunmap_unlocked(&perfcnt->mapping->obj->base.base, &map);
 	perfcnt->buf = NULL;
 	panfrost_gem_close(&perfcnt->mapping->obj->base.base, file_priv);
 	panfrost_mmu_as_put(pfdev, perfcnt->mapping->mmu);
diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
index d0a57853c188..9a8983ee8abe 100644
--- a/include/drm/drm_gem_shmem_helper.h
+++ b/include/drm/drm_gem_shmem_helper.h
@@ -26,11 +26,6 @@ struct drm_gem_shmem_object {
 	 */
 	struct drm_gem_object base;
 
-	/**
-	 * @pages_lock: Protects the page table and use count
-	 */
-	struct mutex pages_lock;
-
 	/**
 	 * @pages: Page table
 	 */
@@ -79,11 +74,6 @@ struct drm_gem_shmem_object {
 	 */
 	struct sg_table *sgt;
 
-	/**
-	 * @vmap_lock: Protects the vmap address and use count
-	 */
-	struct mutex vmap_lock;
-
 	/**
 	 * @vaddr: Kernel virtual address of the backing memory
 	 */
@@ -109,7 +99,6 @@ struct drm_gem_shmem_object {
 struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t size);
 void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem);
 
-int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem);
 void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem);
 int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem);
 void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem);
@@ -128,8 +117,7 @@ static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem
 		!shmem->base.dma_buf && !shmem->base.import_attach;
 }
 
-void drm_gem_shmem_purge_locked(struct drm_gem_shmem_object *shmem);
-bool drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
+void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
 
 struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
 struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
-- 
2.35.3

[PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Dmitry Osipenko]

Introduce a common DRM SHMEM shrinker framework that allows to reduce
code duplication among DRM drivers by replacing theirs custom shrinker
implementations with the generic shrinker.

In order to start using DRM SHMEM shrinker drivers should:

1. Implement new evict() shmem object callback.
2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
   activate shrinking of shmem GEMs.

This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.

Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
 .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
 drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
 include/drm/drm_device.h                      |   4 +
 include/drm/drm_gem_shmem_helper.h            |  87 ++-
 5 files changed, 594 insertions(+), 49 deletions(-)

diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 555fe212bd98..4cd0b5913492 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
 }
 EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
 
+static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
+{
+	return (shmem->madv >= 0) && shmem->evict &&
+		shmem->eviction_enabled && shmem->pages_use_count &&
+		!shmem->pages_pin_count && !shmem->base.dma_buf &&
+		!shmem->base.import_attach && shmem->sgt && !shmem->evicted;
+}
+
+static void
+drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
+{
+	struct drm_gem_object *obj = &shmem->base;
+	struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
+
+	dma_resv_assert_held(shmem->base.resv);
+
+	if (!gem_shrinker || obj->import_attach)
+		return;
+
+	mutex_lock(&gem_shrinker->lock);
+
+	if (drm_gem_shmem_is_evictable(shmem) ||
+	    drm_gem_shmem_is_purgeable(shmem))
+		list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
+	else if (shmem->madv < 0)
+		list_del_init(&shmem->madv_list);
+	else if (shmem->evicted)
+		list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
+	else if (!shmem->pages)
+		list_del_init(&shmem->madv_list);
+	else
+		list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
+
+	mutex_unlock(&gem_shrinker->lock);
+}
+
 /**
  * drm_gem_shmem_free - Free resources associated with a shmem GEM object
  * @shmem: shmem GEM object to free
@@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
 	} else {
 		dma_resv_lock(shmem->base.resv, NULL);
 
+		/* take out shmem GEM object from the memory shrinker */
+		drm_gem_shmem_madvise(shmem, -1);
+
 		WARN_ON(shmem->vmap_use_count);
 
 		if (shmem->sgt) {
@@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
 			sg_free_table(shmem->sgt);
 			kfree(shmem->sgt);
 		}
-		if (shmem->pages)
+		if (shmem->pages_use_count)
 			drm_gem_shmem_put_pages(shmem);
 
 		WARN_ON(shmem->pages_use_count);
@@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
 }
 EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
 
-static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
+/**
+ * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
+ * @shmem: shmem GEM object
+ *
+ * Tell memory shrinker that this GEM can be evicted. Initially eviction is
+ * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
+ *
+ * Returns:
+ * 0 on success or a negative error code on failure.
+ */
+int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
+{
+	dma_resv_lock(shmem->base.resv, NULL);
+
+	if (shmem->madv < 0)
+		return -ENOMEM;
+
+	shmem->eviction_enabled = true;
+
+	dma_resv_unlock(shmem->base.resv);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
+
+/**
+ * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
+ * @shmem: shmem GEM object
+ *
+ * Tell memory shrinker that this GEM can be purged. Initially purging is
+ * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
+ *
+ * Returns:
+ * 0 on success or a negative error code on failure.
+ */
+int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
+{
+	dma_resv_lock(shmem->base.resv, NULL);
+
+	if (shmem->madv < 0)
+		return -ENOMEM;
+
+	shmem->purge_enabled = true;
+
+	drm_gem_shmem_update_pages_state(shmem);
+
+	dma_resv_unlock(shmem->base.resv);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
+
+static int
+drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
 {
 	struct drm_gem_object *obj = &shmem->base;
 	struct page **pages;
 
-	if (shmem->pages_use_count++ > 0)
+	dma_resv_assert_held(shmem->base.resv);
+
+	if (shmem->madv < 0) {
+		WARN_ON(shmem->pages);
+		return -ENOMEM;
+	}
+
+	if (shmem->pages) {
+		WARN_ON(!shmem->evicted);
 		return 0;
+	}
+
+	if (WARN_ON(!shmem->pages_use_count))
+		return -EINVAL;
 
 	pages = drm_gem_get_pages(obj);
 	if (IS_ERR(pages)) {
 		DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
-		shmem->pages_use_count = 0;
 		return PTR_ERR(pages);
 	}
 
@@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
 	return 0;
 }
 
+static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
+{
+	int err;
+
+	dma_resv_assert_held(shmem->base.resv);
+
+	if (shmem->madv < 0)
+		return -ENOMEM;
+
+	if (shmem->pages_use_count++ > 0) {
+		err = drm_gem_shmem_swap_in(shmem);
+		if (err)
+			goto err_zero_use;
+
+		return 0;
+	}
+
+	err = drm_gem_shmem_acquire_pages(shmem);
+	if (err)
+		goto err_zero_use;
+
+	drm_gem_shmem_update_pages_state(shmem);
+
+	return 0;
+
+err_zero_use:
+	shmem->pages_use_count = 0;
+
+	return err;
+}
+
+static void
+drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
+{
+	struct drm_gem_object *obj = &shmem->base;
+
+	if (!shmem->pages) {
+		WARN_ON(!shmem->evicted && shmem->madv >= 0);
+		return;
+	}
+
+#ifdef CONFIG_X86
+	if (shmem->map_wc)
+		set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
+#endif
+
+	drm_gem_put_pages(obj, shmem->pages,
+			  shmem->pages_mark_dirty_on_put,
+			  shmem->pages_mark_accessed_on_put);
+	shmem->pages = NULL;
+}
+
 /*
  * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
  * @shmem: shmem GEM object
@@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
  */
 void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
 {
-	struct drm_gem_object *obj = &shmem->base;
-
 	dma_resv_assert_held(shmem->base.resv);
 
 	if (WARN_ON_ONCE(!shmem->pages_use_count))
@@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
 	if (--shmem->pages_use_count > 0)
 		return;
 
-#ifdef CONFIG_X86
-	if (shmem->map_wc)
-		set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
-#endif
+	drm_gem_shmem_release_pages(shmem);
 
-	drm_gem_put_pages(obj, shmem->pages,
-			  shmem->pages_mark_dirty_on_put,
-			  shmem->pages_mark_accessed_on_put);
-	shmem->pages = NULL;
+	drm_gem_shmem_update_pages_state(shmem);
 }
 EXPORT_SYMBOL(drm_gem_shmem_put_pages);
 
@@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
  */
 int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
 {
+	int ret;
+
 	dma_resv_assert_held(shmem->base.resv);
 
 	WARN_ON(shmem->base.import_attach);
 
-	return drm_gem_shmem_get_pages(shmem);
+	ret = drm_gem_shmem_get_pages(shmem);
+	if (!ret)
+		shmem->pages_pin_count++;
+
+	return ret;
 }
 EXPORT_SYMBOL(drm_gem_shmem_pin);
 
@@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
 	WARN_ON(shmem->base.import_attach);
 
 	drm_gem_shmem_put_pages(shmem);
+
+	shmem->pages_pin_count--;
 }
 EXPORT_SYMBOL(drm_gem_shmem_unpin);
 
@@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
 			return 0;
 		}
 
-		ret = drm_gem_shmem_get_pages(shmem);
+		ret = drm_gem_shmem_pin(shmem);
 		if (ret)
 			goto err_zero_use;
 
@@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
 
 err_put_pages:
 	if (!obj->import_attach)
-		drm_gem_shmem_put_pages(shmem);
+		drm_gem_shmem_unpin(shmem);
 err_zero_use:
 	shmem->vmap_use_count = 0;
 
@@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
 			return;
 
 		vunmap(shmem->vaddr);
-		drm_gem_shmem_put_pages(shmem);
+		drm_gem_shmem_unpin(shmem);
 	}
 
 	shmem->vaddr = NULL;
@@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
 
 	madv = shmem->madv;
 
+	drm_gem_shmem_update_pages_state(shmem);
+
 	return (madv >= 0);
 }
 EXPORT_SYMBOL(drm_gem_shmem_madvise);
 
-void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
+/**
+ * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
+ *                           hardware access to the memory.
+ * @shmem: shmem GEM object
+ *
+ * This function moves shmem GEM back to memory if it was previously evicted
+ * by the memory shrinker. The GEM is ready to use on success.
+ *
+ * Returns:
+ * 0 on success or a negative error code on failure.
+ */
+int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
 {
 	struct drm_gem_object *obj = &shmem->base;
-	struct drm_device *dev = obj->dev;
+	struct sg_table *sgt;
+	int err;
 
 	dma_resv_assert_held(shmem->base.resv);
 
-	WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
+	if (shmem->evicted) {
+		err = drm_gem_shmem_acquire_pages(shmem);
+		if (err)
+			return err;
+
+		sgt = drm_gem_shmem_get_sg_table(shmem);
+		if (IS_ERR(sgt))
+			return PTR_ERR(sgt);
+
+		err = dma_map_sgtable(obj->dev->dev, sgt,
+				      DMA_BIDIRECTIONAL, 0);
+		if (err) {
+			sg_free_table(sgt);
+			kfree(sgt);
+			return err;
+		}
 
-	dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
-	sg_free_table(shmem->sgt);
-	kfree(shmem->sgt);
-	shmem->sgt = NULL;
+		shmem->sgt = sgt;
+		shmem->evicted = false;
 
-	drm_gem_shmem_put_pages(shmem);
+		drm_gem_shmem_update_pages_state(shmem);
+	}
 
-	shmem->madv = -1;
+	if (!shmem->pages)
+		return -ENOMEM;
 
-	drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
-	drm_gem_free_mmap_offset(obj);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
 
-	/* Our goal here is to return as much of the memory as
-	 * is possible back to the system as we are called from OOM.
-	 * To do this we must instruct the shmfs to drop all of its
-	 * backing pages, *now*.
-	 */
-	shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
+static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
+{
+	struct drm_gem_object *obj = &shmem->base;
+	struct drm_device *dev = obj->dev;
 
-	invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
+	if (shmem->evicted)
+		return;
+
+	dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
+	drm_gem_shmem_release_pages(shmem);
+	drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
+
+	sg_free_table(shmem->sgt);
+	kfree(shmem->sgt);
+	shmem->sgt = NULL;
 }
-EXPORT_SYMBOL(drm_gem_shmem_purge);
 
 /**
  * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
@@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
 	vm_fault_t ret;
 	struct page *page;
 	pgoff_t page_offset;
+	bool pages_unpinned;
+	int err;
 
 	/* We don't use vmf->pgoff since that has the fake offset */
 	page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
 
 	dma_resv_lock(shmem->base.resv, NULL);
 
-	if (page_offset >= num_pages ||
-	    WARN_ON_ONCE(!shmem->pages) ||
-	    shmem->madv < 0) {
+	/* Sanity-check that we have the pages pointer when it should present */
+	pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
+	WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
+
+	if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
 		ret = VM_FAULT_SIGBUS;
 	} else {
+		err = drm_gem_shmem_swap_in(shmem);
+		if (err) {
+			ret = VM_FAULT_OOM;
+			goto unlock;
+		}
+
 		page = shmem->pages[page_offset];
 
 		ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
 	}
 
+unlock:
 	dma_resv_unlock(shmem->base.resv);
 
 	return ret;
@@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
 {
 	struct drm_gem_object *obj = vma->vm_private_data;
 	struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
-	int ret;
 
 	WARN_ON(shmem->base.import_attach);
 
 	dma_resv_lock(shmem->base.resv, NULL);
-	ret = drm_gem_shmem_get_pages(shmem);
-	WARN_ON_ONCE(ret != 0);
+
+	if (drm_gem_shmem_get_pages(shmem))
+		shmem->pages_use_count++;
+
+	drm_gem_shmem_update_pages_state(shmem);
 	dma_resv_unlock(shmem->base.resv);
 
 	drm_gem_vm_open(vma);
@@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
 void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
 			      struct drm_printer *p, unsigned int indent)
 {
+	drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
+	drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
 	drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
 
 	if (shmem->base.import_attach)
@@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
 		drm_printf_indent(p, indent, "vmap_use_count=%u\n",
 				  shmem->vmap_use_count);
 
+	drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
 	drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
+	drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
 }
 EXPORT_SYMBOL(drm_gem_shmem_print_info);
 
@@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
 
 	shmem->sgt = sgt;
 
+	drm_gem_shmem_update_pages_state(shmem);
+
 	dma_resv_unlock(shmem->base.resv);
 
 	return sgt;
@@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
 }
 EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
 
+static struct drm_gem_shmem_shrinker *
+to_drm_shrinker(struct shrinker *shrinker)
+{
+	return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
+}
+
+static unsigned long
+drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
+				     struct shrink_control *sc)
+{
+	struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
+	struct drm_gem_shmem_object *shmem;
+	unsigned long count = 0;
+
+	if (!mutex_trylock(&gem_shrinker->lock))
+		return 0;
+
+	list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
+		count += shmem->base.size;
+
+		if (count >= SHRINK_EMPTY)
+			break;
+	}
+
+	mutex_unlock(&gem_shrinker->lock);
+
+	if (count >= SHRINK_EMPTY)
+		return SHRINK_EMPTY - 1;
+
+	return count ?: SHRINK_EMPTY;
+}
+
+int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
+{
+	WARN_ON(!drm_gem_shmem_is_evictable(shmem));
+	WARN_ON(shmem->evicted);
+
+	drm_gem_shmem_unpin_pages(shmem);
+
+	shmem->evicted = true;
+	drm_gem_shmem_update_pages_state(shmem);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
+
+int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
+{
+	struct drm_gem_object *obj = &shmem->base;
+
+	WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
+
+	drm_gem_shmem_unpin_pages(shmem);
+	drm_gem_free_mmap_offset(obj);
+
+	/* Our goal here is to return as much of the memory as
+	 * is possible back to the system as we are called from OOM.
+	 * To do this we must instruct the shmfs to drop all of its
+	 * backing pages, *now*.
+	 */
+	shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
+
+	invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
+
+	shmem->madv = -1;
+	shmem->evicted = false;
+	drm_gem_shmem_update_pages_state(shmem);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
+
+static unsigned long
+drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
+					unsigned long nr_to_scan,
+					bool *lock_contention,
+					bool evict)
+{
+	struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
+	struct drm_gem_shmem_object *shmem;
+	struct list_head still_in_list;
+	struct drm_gem_object *obj;
+	unsigned long freed = 0;
+	size_t page_count;
+	int err;
+
+	INIT_LIST_HEAD(&still_in_list);
+
+	mutex_lock(&gem_shrinker->lock);
+
+	while (freed < nr_to_scan) {
+		shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
+						 typeof(*shmem), madv_list);
+		if (!shmem)
+			break;
+
+		obj = &shmem->base;
+		page_count = obj->size >> PAGE_SHIFT;
+		list_move_tail(&shmem->madv_list, &still_in_list);
+
+		if (evict) {
+			if (!drm_gem_shmem_is_evictable(shmem) ||
+			    get_nr_swap_pages() < page_count)
+				continue;
+		} else {
+			if (!drm_gem_shmem_is_purgeable(shmem))
+				continue;
+		}
+
+		/*
+		 * If it's in the process of being freed, gem_object->free()
+		 * may be blocked on lock waiting to remove it.  So just
+		 * skip it.
+		 */
+		if (!kref_get_unless_zero(&obj->refcount))
+			continue;
+
+		mutex_unlock(&gem_shrinker->lock);
+
+		/* prevent racing with job-submission code paths */
+		if (!dma_resv_trylock(obj->resv)) {
+			*lock_contention |= true;
+			goto shrinker_lock;
+		}
+
+		/* prevent racing with the dma-buf importing/exporting */
+		if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
+			*lock_contention |= true;
+			goto resv_unlock;
+		}
+
+		/* check whether h/w uses this object */
+		if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
+			goto object_name_unlock;
+
+		/* re-check whether eviction status hasn't changed */
+		if (!drm_gem_shmem_is_evictable(shmem) &&
+		    !drm_gem_shmem_is_purgeable(shmem))
+			goto object_name_unlock;
+
+		err = shmem->evict(shmem);
+		if (!err)
+			freed += obj->size >> PAGE_SHIFT;
+
+object_name_unlock:
+		mutex_unlock(&gem_shrinker->dev->object_name_lock);
+resv_unlock:
+		dma_resv_unlock(obj->resv);
+shrinker_lock:
+		drm_gem_object_put(&shmem->base);
+		mutex_lock(&gem_shrinker->lock);
+	}
+
+	list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
+
+	mutex_unlock(&gem_shrinker->lock);
+
+	return freed;
+}
+
+static unsigned long
+drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
+				    struct shrink_control *sc)
+{
+	unsigned long nr_to_scan = sc->nr_to_scan;
+	bool lock_contention = false;
+	unsigned long freed;
+
+	/* purge as many objects as we can */
+	freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
+							&lock_contention, false);
+
+	/* evict as many objects as we can */
+	if (freed < nr_to_scan)
+		freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
+								 nr_to_scan - freed,
+								 &lock_contention,
+								 true);
+
+	return (!freed && !lock_contention) ? SHRINK_STOP : freed;
+}
+
+/**
+ * drm_gem_shmem_shrinker_register() - Register shmem shrinker
+ * @dev: DRM device
+ *
+ * Returns:
+ * 0 on success or a negative error code on failure.
+ */
+int drm_gem_shmem_shrinker_register(struct drm_device *dev)
+{
+	struct drm_gem_shmem_shrinker *gem_shrinker;
+	int err;
+
+	if (WARN_ON(dev->shmem_shrinker))
+		return -EBUSY;
+
+	gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
+	if (!gem_shrinker)
+		return -ENOMEM;
+
+	gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
+	gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
+	gem_shrinker->base.seeks = DEFAULT_SEEKS;
+	gem_shrinker->dev = dev;
+
+	INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
+	INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
+	INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
+	mutex_init(&gem_shrinker->lock);
+
+	dev->shmem_shrinker = gem_shrinker;
+
+	err = register_shrinker(&gem_shrinker->base);
+	if (err) {
+		dev->shmem_shrinker = NULL;
+		kfree(gem_shrinker);
+		return err;
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
+
+/**
+ * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
+ * @dev: DRM device
+ */
+void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
+{
+	struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
+
+	if (gem_shrinker) {
+		unregister_shrinker(&gem_shrinker->base);
+		WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
+		WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
+		WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
+		mutex_destroy(&gem_shrinker->lock);
+		dev->shmem_shrinker = NULL;
+		kfree(gem_shrinker);
+	}
+}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
+
 MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
 MODULE_IMPORT_NS(DMA_BUF);
 MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
index a4bedfeb2ec4..7cc32556f908 100644
--- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
+++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
@@ -15,6 +15,13 @@
 #include "panfrost_gem.h"
 #include "panfrost_mmu.h"
 
+static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
+{
+	return (shmem->madv > 0) &&
+		!shmem->pages_pin_count && shmem->sgt &&
+		!shmem->base.dma_buf && !shmem->base.import_attach;
+}
+
 static unsigned long
 panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
 {
@@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
 		return 0;
 
 	list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
-		if (drm_gem_shmem_is_purgeable(shmem))
+		if (panfrost_gem_shmem_is_purgeable(shmem))
 			count += shmem->base.size >> PAGE_SHIFT;
 	}
 
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
index b2d93cb12ebf..81bacc7e1873 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -89,6 +89,7 @@ struct virtio_gpu_object {
 	uint32_t hw_res_handle;
 	bool dumb;
 	bool created;
+	bool detached;
 	bool host3d_blob, guest_blob;
 	uint32_t blob_mem, blob_flags;
 
@@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
 
 bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
 
+int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
+
 int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
 			       uint32_t *resid);
 /* virtgpu_prime.c */
diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
index 9923c7a6885e..929546cad894 100644
--- a/include/drm/drm_device.h
+++ b/include/drm/drm_device.h
@@ -16,6 +16,7 @@ struct drm_vblank_crtc;
 struct drm_vma_offset_manager;
 struct drm_vram_mm;
 struct drm_fb_helper;
+struct drm_gem_shmem_shrinker;
 
 struct inode;
 
@@ -277,6 +278,9 @@ struct drm_device {
 	/** @vram_mm: VRAM MM memory manager */
 	struct drm_vram_mm *vram_mm;
 
+	/** @shmem_shrinker: SHMEM GEM memory shrinker */
+	struct drm_gem_shmem_shrinker *shmem_shrinker;
+
 	/**
 	 * @switch_power_state:
 	 *
diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
index 9a8983ee8abe..62c640678a91 100644
--- a/include/drm/drm_gem_shmem_helper.h
+++ b/include/drm/drm_gem_shmem_helper.h
@@ -6,6 +6,7 @@
 #include <linux/fs.h>
 #include <linux/mm.h>
 #include <linux/mutex.h>
+#include <linux/shrinker.h>
 
 #include <drm/drm_file.h>
 #include <drm/drm_gem.h>
@@ -15,6 +16,7 @@
 struct dma_buf_attachment;
 struct drm_mode_create_dumb;
 struct drm_printer;
+struct drm_device;
 struct sg_table;
 
 /**
@@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
 	 */
 	unsigned int pages_use_count;
 
+	/**
+	 * @pages_pin_count:
+	 *
+	 * Reference count on the pinned pages table.
+	 * The pages can be evicted by memory shrinker
+	 * when the count reaches zero.
+	 */
+	unsigned int pages_pin_count;
+
 	/**
 	 * @madv: State for madvise
 	 *
 	 * 0 is active/inuse.
+	 * 1 is not-needed/can-be-purged
 	 * A negative value is the object is purged.
-	 * Positive values are driver specific and not used by the helpers.
 	 */
 	int madv;
 
@@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
 	 * @map_wc: map object write-combined (instead of using shmem defaults).
 	 */
 	bool map_wc;
+
+	/**
+	 * @eviction_enabled:
+	 *
+	 * The shmem pages can be evicted only if @eviction_enabled is set to true.
+	 * Used internally by memory shrinker.
+	 */
+	bool eviction_enabled;
+
+	/**
+	 * @purge_enabled:
+	 *
+	 * The shmem pages can be purged only if @purge_enabled is set to true.
+	 * Used internally by memory shrinker.
+	 */
+	bool purge_enabled;
+
+	/**
+	 * @evicted: True if shmem pages are evicted by the memory shrinker.
+	 * Used internally by memory shrinker.
+	 */
+	bool evicted;
+
+	/**
+	 * @evict:
+	 *
+	 * Invoked by shmem shrinker before evicting shmem GEM from memory.
+	 * GEM's DMA reservation is kept locked by the shrinker. This is
+	 * optional callback that should be specified by drivers.
+	 *
+	 * Returns 0 on success, or -errno on error.
+	 */
+	int (*evict)(struct drm_gem_shmem_object *shmem);
 };
 
 #define to_drm_gem_shmem_obj(obj) \
@@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
 
 int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
 
+int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
+int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
+
 static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
 {
-	return (shmem->madv > 0) &&
-		!shmem->vmap_use_count && shmem->sgt &&
-		!shmem->base.dma_buf && !shmem->base.import_attach;
+	return (shmem->madv > 0) && shmem->evict &&
+		shmem->purge_enabled && shmem->pages_use_count &&
+		!shmem->pages_pin_count && !shmem->base.dma_buf &&
+		!shmem->base.import_attach && (shmem->sgt || shmem->evicted);
 }
 
-void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
+int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
+
+int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
+int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
 
 struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
 struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
@@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
 	return drm_gem_shmem_mmap(shmem, vma);
 }
 
+/**
+ * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
+ */
+struct drm_gem_shmem_shrinker {
+	/** @base: Shrinker for purging shmem GEM objects */
+	struct shrinker base;
+
+	/** @lock: Protects @lru_* */
+	struct mutex lock;
+
+	/** @lru_pinned: List of pinned shmem GEM objects */
+	struct list_head lru_pinned;
+
+	/** @lru_evictable: List of shmem GEM objects to be evicted */
+	struct list_head lru_evictable;
+
+	/** @lru_evicted: List of evicted shmem GEM objects */
+	struct list_head lru_evicted;
+
+	/** @dev: DRM device that uses this shrinker */
+	struct drm_device *dev;
+};
+
+int drm_gem_shmem_shrinker_register(struct drm_device *dev);
+void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
+
 /*
  * Driver ops
  */
-- 
2.35.3

[PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Dmitry Osipenko]

Introduce a common DRM SHMEM shrinker framework that allows to reduce
code duplication among DRM drivers by replacing theirs custom shrinker
implementations with the generic shrinker.

In order to start using DRM SHMEM shrinker drivers should:

1. Implement new evict() shmem object callback.
2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
   activate shrinking of shmem GEMs.

This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.

Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
 .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
 drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
 include/drm/drm_device.h                      |   4 +
 include/drm/drm_gem_shmem_helper.h            |  87 ++-
 5 files changed, 594 insertions(+), 49 deletions(-)

diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
index 555fe212bd98..4cd0b5913492 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
 }
 EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
 
+static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
+{
+	return (shmem->madv >= 0) && shmem->evict &&
+		shmem->eviction_enabled && shmem->pages_use_count &&
+		!shmem->pages_pin_count && !shmem->base.dma_buf &&
+		!shmem->base.import_attach && shmem->sgt && !shmem->evicted;
+}
+
+static void
+drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
+{
+	struct drm_gem_object *obj = &shmem->base;
+	struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
+
+	dma_resv_assert_held(shmem->base.resv);
+
+	if (!gem_shrinker || obj->import_attach)
+		return;
+
+	mutex_lock(&gem_shrinker->lock);
+
+	if (drm_gem_shmem_is_evictable(shmem) ||
+	    drm_gem_shmem_is_purgeable(shmem))
+		list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
+	else if (shmem->madv < 0)
+		list_del_init(&shmem->madv_list);
+	else if (shmem->evicted)
+		list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
+	else if (!shmem->pages)
+		list_del_init(&shmem->madv_list);
+	else
+		list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
+
+	mutex_unlock(&gem_shrinker->lock);
+}
+
 /**
  * drm_gem_shmem_free - Free resources associated with a shmem GEM object
  * @shmem: shmem GEM object to free
@@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
 	} else {
 		dma_resv_lock(shmem->base.resv, NULL);
 
+		/* take out shmem GEM object from the memory shrinker */
+		drm_gem_shmem_madvise(shmem, -1);
+
 		WARN_ON(shmem->vmap_use_count);
 
 		if (shmem->sgt) {
@@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
 			sg_free_table(shmem->sgt);
 			kfree(shmem->sgt);
 		}
-		if (shmem->pages)
+		if (shmem->pages_use_count)
 			drm_gem_shmem_put_pages(shmem);
 
 		WARN_ON(shmem->pages_use_count);
@@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
 }
 EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
 
-static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
+/**
+ * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
+ * @shmem: shmem GEM object
+ *
+ * Tell memory shrinker that this GEM can be evicted. Initially eviction is
+ * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
+ *
+ * Returns:
+ * 0 on success or a negative error code on failure.
+ */
+int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
+{
+	dma_resv_lock(shmem->base.resv, NULL);
+
+	if (shmem->madv < 0)
+		return -ENOMEM;
+
+	shmem->eviction_enabled = true;
+
+	dma_resv_unlock(shmem->base.resv);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
+
+/**
+ * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
+ * @shmem: shmem GEM object
+ *
+ * Tell memory shrinker that this GEM can be purged. Initially purging is
+ * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
+ *
+ * Returns:
+ * 0 on success or a negative error code on failure.
+ */
+int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
+{
+	dma_resv_lock(shmem->base.resv, NULL);
+
+	if (shmem->madv < 0)
+		return -ENOMEM;
+
+	shmem->purge_enabled = true;
+
+	drm_gem_shmem_update_pages_state(shmem);
+
+	dma_resv_unlock(shmem->base.resv);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
+
+static int
+drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
 {
 	struct drm_gem_object *obj = &shmem->base;
 	struct page **pages;
 
-	if (shmem->pages_use_count++ > 0)
+	dma_resv_assert_held(shmem->base.resv);
+
+	if (shmem->madv < 0) {
+		WARN_ON(shmem->pages);
+		return -ENOMEM;
+	}
+
+	if (shmem->pages) {
+		WARN_ON(!shmem->evicted);
 		return 0;
+	}
+
+	if (WARN_ON(!shmem->pages_use_count))
+		return -EINVAL;
 
 	pages = drm_gem_get_pages(obj);
 	if (IS_ERR(pages)) {
 		DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
-		shmem->pages_use_count = 0;
 		return PTR_ERR(pages);
 	}
 
@@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
 	return 0;
 }
 
+static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
+{
+	int err;
+
+	dma_resv_assert_held(shmem->base.resv);
+
+	if (shmem->madv < 0)
+		return -ENOMEM;
+
+	if (shmem->pages_use_count++ > 0) {
+		err = drm_gem_shmem_swap_in(shmem);
+		if (err)
+			goto err_zero_use;
+
+		return 0;
+	}
+
+	err = drm_gem_shmem_acquire_pages(shmem);
+	if (err)
+		goto err_zero_use;
+
+	drm_gem_shmem_update_pages_state(shmem);
+
+	return 0;
+
+err_zero_use:
+	shmem->pages_use_count = 0;
+
+	return err;
+}
+
+static void
+drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
+{
+	struct drm_gem_object *obj = &shmem->base;
+
+	if (!shmem->pages) {
+		WARN_ON(!shmem->evicted && shmem->madv >= 0);
+		return;
+	}
+
+#ifdef CONFIG_X86
+	if (shmem->map_wc)
+		set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
+#endif
+
+	drm_gem_put_pages(obj, shmem->pages,
+			  shmem->pages_mark_dirty_on_put,
+			  shmem->pages_mark_accessed_on_put);
+	shmem->pages = NULL;
+}
+
 /*
  * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
  * @shmem: shmem GEM object
@@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
  */
 void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
 {
-	struct drm_gem_object *obj = &shmem->base;
-
 	dma_resv_assert_held(shmem->base.resv);
 
 	if (WARN_ON_ONCE(!shmem->pages_use_count))
@@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
 	if (--shmem->pages_use_count > 0)
 		return;
 
-#ifdef CONFIG_X86
-	if (shmem->map_wc)
-		set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
-#endif
+	drm_gem_shmem_release_pages(shmem);
 
-	drm_gem_put_pages(obj, shmem->pages,
-			  shmem->pages_mark_dirty_on_put,
-			  shmem->pages_mark_accessed_on_put);
-	shmem->pages = NULL;
+	drm_gem_shmem_update_pages_state(shmem);
 }
 EXPORT_SYMBOL(drm_gem_shmem_put_pages);
 
@@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
  */
 int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
 {
+	int ret;
+
 	dma_resv_assert_held(shmem->base.resv);
 
 	WARN_ON(shmem->base.import_attach);
 
-	return drm_gem_shmem_get_pages(shmem);
+	ret = drm_gem_shmem_get_pages(shmem);
+	if (!ret)
+		shmem->pages_pin_count++;
+
+	return ret;
 }
 EXPORT_SYMBOL(drm_gem_shmem_pin);
 
@@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
 	WARN_ON(shmem->base.import_attach);
 
 	drm_gem_shmem_put_pages(shmem);
+
+	shmem->pages_pin_count--;
 }
 EXPORT_SYMBOL(drm_gem_shmem_unpin);
 
@@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
 			return 0;
 		}
 
-		ret = drm_gem_shmem_get_pages(shmem);
+		ret = drm_gem_shmem_pin(shmem);
 		if (ret)
 			goto err_zero_use;
 
@@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
 
 err_put_pages:
 	if (!obj->import_attach)
-		drm_gem_shmem_put_pages(shmem);
+		drm_gem_shmem_unpin(shmem);
 err_zero_use:
 	shmem->vmap_use_count = 0;
 
@@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
 			return;
 
 		vunmap(shmem->vaddr);
-		drm_gem_shmem_put_pages(shmem);
+		drm_gem_shmem_unpin(shmem);
 	}
 
 	shmem->vaddr = NULL;
@@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
 
 	madv = shmem->madv;
 
+	drm_gem_shmem_update_pages_state(shmem);
+
 	return (madv >= 0);
 }
 EXPORT_SYMBOL(drm_gem_shmem_madvise);
 
-void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
+/**
+ * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
+ *                           hardware access to the memory.
+ * @shmem: shmem GEM object
+ *
+ * This function moves shmem GEM back to memory if it was previously evicted
+ * by the memory shrinker. The GEM is ready to use on success.
+ *
+ * Returns:
+ * 0 on success or a negative error code on failure.
+ */
+int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
 {
 	struct drm_gem_object *obj = &shmem->base;
-	struct drm_device *dev = obj->dev;
+	struct sg_table *sgt;
+	int err;
 
 	dma_resv_assert_held(shmem->base.resv);
 
-	WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
+	if (shmem->evicted) {
+		err = drm_gem_shmem_acquire_pages(shmem);
+		if (err)
+			return err;
+
+		sgt = drm_gem_shmem_get_sg_table(shmem);
+		if (IS_ERR(sgt))
+			return PTR_ERR(sgt);
+
+		err = dma_map_sgtable(obj->dev->dev, sgt,
+				      DMA_BIDIRECTIONAL, 0);
+		if (err) {
+			sg_free_table(sgt);
+			kfree(sgt);
+			return err;
+		}
 
-	dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
-	sg_free_table(shmem->sgt);
-	kfree(shmem->sgt);
-	shmem->sgt = NULL;
+		shmem->sgt = sgt;
+		shmem->evicted = false;
 
-	drm_gem_shmem_put_pages(shmem);
+		drm_gem_shmem_update_pages_state(shmem);
+	}
 
-	shmem->madv = -1;
+	if (!shmem->pages)
+		return -ENOMEM;
 
-	drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
-	drm_gem_free_mmap_offset(obj);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
 
-	/* Our goal here is to return as much of the memory as
-	 * is possible back to the system as we are called from OOM.
-	 * To do this we must instruct the shmfs to drop all of its
-	 * backing pages, *now*.
-	 */
-	shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
+static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
+{
+	struct drm_gem_object *obj = &shmem->base;
+	struct drm_device *dev = obj->dev;
 
-	invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
+	if (shmem->evicted)
+		return;
+
+	dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
+	drm_gem_shmem_release_pages(shmem);
+	drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
+
+	sg_free_table(shmem->sgt);
+	kfree(shmem->sgt);
+	shmem->sgt = NULL;
 }
-EXPORT_SYMBOL(drm_gem_shmem_purge);
 
 /**
  * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
@@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
 	vm_fault_t ret;
 	struct page *page;
 	pgoff_t page_offset;
+	bool pages_unpinned;
+	int err;
 
 	/* We don't use vmf->pgoff since that has the fake offset */
 	page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
 
 	dma_resv_lock(shmem->base.resv, NULL);
 
-	if (page_offset >= num_pages ||
-	    WARN_ON_ONCE(!shmem->pages) ||
-	    shmem->madv < 0) {
+	/* Sanity-check that we have the pages pointer when it should present */
+	pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
+	WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
+
+	if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
 		ret = VM_FAULT_SIGBUS;
 	} else {
+		err = drm_gem_shmem_swap_in(shmem);
+		if (err) {
+			ret = VM_FAULT_OOM;
+			goto unlock;
+		}
+
 		page = shmem->pages[page_offset];
 
 		ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
 	}
 
+unlock:
 	dma_resv_unlock(shmem->base.resv);
 
 	return ret;
@@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
 {
 	struct drm_gem_object *obj = vma->vm_private_data;
 	struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
-	int ret;
 
 	WARN_ON(shmem->base.import_attach);
 
 	dma_resv_lock(shmem->base.resv, NULL);
-	ret = drm_gem_shmem_get_pages(shmem);
-	WARN_ON_ONCE(ret != 0);
+
+	if (drm_gem_shmem_get_pages(shmem))
+		shmem->pages_use_count++;
+
+	drm_gem_shmem_update_pages_state(shmem);
 	dma_resv_unlock(shmem->base.resv);
 
 	drm_gem_vm_open(vma);
@@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
 void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
 			      struct drm_printer *p, unsigned int indent)
 {
+	drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
+	drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
 	drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
 
 	if (shmem->base.import_attach)
@@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
 		drm_printf_indent(p, indent, "vmap_use_count=%u\n",
 				  shmem->vmap_use_count);
 
+	drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
 	drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
+	drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
 }
 EXPORT_SYMBOL(drm_gem_shmem_print_info);
 
@@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
 
 	shmem->sgt = sgt;
 
+	drm_gem_shmem_update_pages_state(shmem);
+
 	dma_resv_unlock(shmem->base.resv);
 
 	return sgt;
@@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
 }
 EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
 
+static struct drm_gem_shmem_shrinker *
+to_drm_shrinker(struct shrinker *shrinker)
+{
+	return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
+}
+
+static unsigned long
+drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
+				     struct shrink_control *sc)
+{
+	struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
+	struct drm_gem_shmem_object *shmem;
+	unsigned long count = 0;
+
+	if (!mutex_trylock(&gem_shrinker->lock))
+		return 0;
+
+	list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
+		count += shmem->base.size;
+
+		if (count >= SHRINK_EMPTY)
+			break;
+	}
+
+	mutex_unlock(&gem_shrinker->lock);
+
+	if (count >= SHRINK_EMPTY)
+		return SHRINK_EMPTY - 1;
+
+	return count ?: SHRINK_EMPTY;
+}
+
+int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
+{
+	WARN_ON(!drm_gem_shmem_is_evictable(shmem));
+	WARN_ON(shmem->evicted);
+
+	drm_gem_shmem_unpin_pages(shmem);
+
+	shmem->evicted = true;
+	drm_gem_shmem_update_pages_state(shmem);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
+
+int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
+{
+	struct drm_gem_object *obj = &shmem->base;
+
+	WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
+
+	drm_gem_shmem_unpin_pages(shmem);
+	drm_gem_free_mmap_offset(obj);
+
+	/* Our goal here is to return as much of the memory as
+	 * is possible back to the system as we are called from OOM.
+	 * To do this we must instruct the shmfs to drop all of its
+	 * backing pages, *now*.
+	 */
+	shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
+
+	invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
+
+	shmem->madv = -1;
+	shmem->evicted = false;
+	drm_gem_shmem_update_pages_state(shmem);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
+
+static unsigned long
+drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
+					unsigned long nr_to_scan,
+					bool *lock_contention,
+					bool evict)
+{
+	struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
+	struct drm_gem_shmem_object *shmem;
+	struct list_head still_in_list;
+	struct drm_gem_object *obj;
+	unsigned long freed = 0;
+	size_t page_count;
+	int err;
+
+	INIT_LIST_HEAD(&still_in_list);
+
+	mutex_lock(&gem_shrinker->lock);
+
+	while (freed < nr_to_scan) {
+		shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
+						 typeof(*shmem), madv_list);
+		if (!shmem)
+			break;
+
+		obj = &shmem->base;
+		page_count = obj->size >> PAGE_SHIFT;
+		list_move_tail(&shmem->madv_list, &still_in_list);
+
+		if (evict) {
+			if (!drm_gem_shmem_is_evictable(shmem) ||
+			    get_nr_swap_pages() < page_count)
+				continue;
+		} else {
+			if (!drm_gem_shmem_is_purgeable(shmem))
+				continue;
+		}
+
+		/*
+		 * If it's in the process of being freed, gem_object->free()
+		 * may be blocked on lock waiting to remove it.  So just
+		 * skip it.
+		 */
+		if (!kref_get_unless_zero(&obj->refcount))
+			continue;
+
+		mutex_unlock(&gem_shrinker->lock);
+
+		/* prevent racing with job-submission code paths */
+		if (!dma_resv_trylock(obj->resv)) {
+			*lock_contention |= true;
+			goto shrinker_lock;
+		}
+
+		/* prevent racing with the dma-buf importing/exporting */
+		if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
+			*lock_contention |= true;
+			goto resv_unlock;
+		}
+
+		/* check whether h/w uses this object */
+		if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
+			goto object_name_unlock;
+
+		/* re-check whether eviction status hasn't changed */
+		if (!drm_gem_shmem_is_evictable(shmem) &&
+		    !drm_gem_shmem_is_purgeable(shmem))
+			goto object_name_unlock;
+
+		err = shmem->evict(shmem);
+		if (!err)
+			freed += obj->size >> PAGE_SHIFT;
+
+object_name_unlock:
+		mutex_unlock(&gem_shrinker->dev->object_name_lock);
+resv_unlock:
+		dma_resv_unlock(obj->resv);
+shrinker_lock:
+		drm_gem_object_put(&shmem->base);
+		mutex_lock(&gem_shrinker->lock);
+	}
+
+	list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
+
+	mutex_unlock(&gem_shrinker->lock);
+
+	return freed;
+}
+
+static unsigned long
+drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
+				    struct shrink_control *sc)
+{
+	unsigned long nr_to_scan = sc->nr_to_scan;
+	bool lock_contention = false;
+	unsigned long freed;
+
+	/* purge as many objects as we can */
+	freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
+							&lock_contention, false);
+
+	/* evict as many objects as we can */
+	if (freed < nr_to_scan)
+		freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
+								 nr_to_scan - freed,
+								 &lock_contention,
+								 true);
+
+	return (!freed && !lock_contention) ? SHRINK_STOP : freed;
+}
+
+/**
+ * drm_gem_shmem_shrinker_register() - Register shmem shrinker
+ * @dev: DRM device
+ *
+ * Returns:
+ * 0 on success or a negative error code on failure.
+ */
+int drm_gem_shmem_shrinker_register(struct drm_device *dev)
+{
+	struct drm_gem_shmem_shrinker *gem_shrinker;
+	int err;
+
+	if (WARN_ON(dev->shmem_shrinker))
+		return -EBUSY;
+
+	gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
+	if (!gem_shrinker)
+		return -ENOMEM;
+
+	gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
+	gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
+	gem_shrinker->base.seeks = DEFAULT_SEEKS;
+	gem_shrinker->dev = dev;
+
+	INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
+	INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
+	INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
+	mutex_init(&gem_shrinker->lock);
+
+	dev->shmem_shrinker = gem_shrinker;
+
+	err = register_shrinker(&gem_shrinker->base);
+	if (err) {
+		dev->shmem_shrinker = NULL;
+		kfree(gem_shrinker);
+		return err;
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
+
+/**
+ * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
+ * @dev: DRM device
+ */
+void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
+{
+	struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
+
+	if (gem_shrinker) {
+		unregister_shrinker(&gem_shrinker->base);
+		WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
+		WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
+		WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
+		mutex_destroy(&gem_shrinker->lock);
+		dev->shmem_shrinker = NULL;
+		kfree(gem_shrinker);
+	}
+}
+EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
+
 MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
 MODULE_IMPORT_NS(DMA_BUF);
 MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
index a4bedfeb2ec4..7cc32556f908 100644
--- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
+++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
@@ -15,6 +15,13 @@
 #include "panfrost_gem.h"
 #include "panfrost_mmu.h"
 
+static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
+{
+	return (shmem->madv > 0) &&
+		!shmem->pages_pin_count && shmem->sgt &&
+		!shmem->base.dma_buf && !shmem->base.import_attach;
+}
+
 static unsigned long
 panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
 {
@@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
 		return 0;
 
 	list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
-		if (drm_gem_shmem_is_purgeable(shmem))
+		if (panfrost_gem_shmem_is_purgeable(shmem))
 			count += shmem->base.size >> PAGE_SHIFT;
 	}
 
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
index b2d93cb12ebf..81bacc7e1873 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -89,6 +89,7 @@ struct virtio_gpu_object {
 	uint32_t hw_res_handle;
 	bool dumb;
 	bool created;
+	bool detached;
 	bool host3d_blob, guest_blob;
 	uint32_t blob_mem, blob_flags;
 
@@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
 
 bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
 
+int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
+
 int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
 			       uint32_t *resid);
 /* virtgpu_prime.c */
diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
index 9923c7a6885e..929546cad894 100644
--- a/include/drm/drm_device.h
+++ b/include/drm/drm_device.h
@@ -16,6 +16,7 @@ struct drm_vblank_crtc;
 struct drm_vma_offset_manager;
 struct drm_vram_mm;
 struct drm_fb_helper;
+struct drm_gem_shmem_shrinker;
 
 struct inode;
 
@@ -277,6 +278,9 @@ struct drm_device {
 	/** @vram_mm: VRAM MM memory manager */
 	struct drm_vram_mm *vram_mm;
 
+	/** @shmem_shrinker: SHMEM GEM memory shrinker */
+	struct drm_gem_shmem_shrinker *shmem_shrinker;
+
 	/**
 	 * @switch_power_state:
 	 *
diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
index 9a8983ee8abe..62c640678a91 100644
--- a/include/drm/drm_gem_shmem_helper.h
+++ b/include/drm/drm_gem_shmem_helper.h
@@ -6,6 +6,7 @@
 #include <linux/fs.h>
 #include <linux/mm.h>
 #include <linux/mutex.h>
+#include <linux/shrinker.h>
 
 #include <drm/drm_file.h>
 #include <drm/drm_gem.h>
@@ -15,6 +16,7 @@
 struct dma_buf_attachment;
 struct drm_mode_create_dumb;
 struct drm_printer;
+struct drm_device;
 struct sg_table;
 
 /**
@@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
 	 */
 	unsigned int pages_use_count;
 
+	/**
+	 * @pages_pin_count:
+	 *
+	 * Reference count on the pinned pages table.
+	 * The pages can be evicted by memory shrinker
+	 * when the count reaches zero.
+	 */
+	unsigned int pages_pin_count;
+
 	/**
 	 * @madv: State for madvise
 	 *
 	 * 0 is active/inuse.
+	 * 1 is not-needed/can-be-purged
 	 * A negative value is the object is purged.
-	 * Positive values are driver specific and not used by the helpers.
 	 */
 	int madv;
 
@@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
 	 * @map_wc: map object write-combined (instead of using shmem defaults).
 	 */
 	bool map_wc;
+
+	/**
+	 * @eviction_enabled:
+	 *
+	 * The shmem pages can be evicted only if @eviction_enabled is set to true.
+	 * Used internally by memory shrinker.
+	 */
+	bool eviction_enabled;
+
+	/**
+	 * @purge_enabled:
+	 *
+	 * The shmem pages can be purged only if @purge_enabled is set to true.
+	 * Used internally by memory shrinker.
+	 */
+	bool purge_enabled;
+
+	/**
+	 * @evicted: True if shmem pages are evicted by the memory shrinker.
+	 * Used internally by memory shrinker.
+	 */
+	bool evicted;
+
+	/**
+	 * @evict:
+	 *
+	 * Invoked by shmem shrinker before evicting shmem GEM from memory.
+	 * GEM's DMA reservation is kept locked by the shrinker. This is
+	 * optional callback that should be specified by drivers.
+	 *
+	 * Returns 0 on success, or -errno on error.
+	 */
+	int (*evict)(struct drm_gem_shmem_object *shmem);
 };
 
 #define to_drm_gem_shmem_obj(obj) \
@@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
 
 int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
 
+int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
+int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
+
 static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
 {
-	return (shmem->madv > 0) &&
-		!shmem->vmap_use_count && shmem->sgt &&
-		!shmem->base.dma_buf && !shmem->base.import_attach;
+	return (shmem->madv > 0) && shmem->evict &&
+		shmem->purge_enabled && shmem->pages_use_count &&
+		!shmem->pages_pin_count && !shmem->base.dma_buf &&
+		!shmem->base.import_attach && (shmem->sgt || shmem->evicted);
 }
 
-void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
+int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
+
+int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
+int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
 
 struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
 struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
@@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
 	return drm_gem_shmem_mmap(shmem, vma);
 }
 
+/**
+ * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
+ */
+struct drm_gem_shmem_shrinker {
+	/** @base: Shrinker for purging shmem GEM objects */
+	struct shrinker base;
+
+	/** @lock: Protects @lru_* */
+	struct mutex lock;
+
+	/** @lru_pinned: List of pinned shmem GEM objects */
+	struct list_head lru_pinned;
+
+	/** @lru_evictable: List of shmem GEM objects to be evicted */
+	struct list_head lru_evictable;
+
+	/** @lru_evicted: List of evicted shmem GEM objects */
+	struct list_head lru_evicted;
+
+	/** @dev: DRM device that uses this shrinker */
+	struct drm_device *dev;
+};
+
+int drm_gem_shmem_shrinker_register(struct drm_device *dev);
+void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
+
 /*
  * Driver ops
  */
-- 
2.35.3

[PATCH v6 18/22] drm/gem: Add drm_gem_pin_unlocked()

[Dmitry Osipenko]

Add unlocked variants of drm_gem_un/pin() functions and make them public.
These new helpers will take care of GEM dma-reservation locking for DRM
drivers.

We are going to add memory shrinking support to the VirtIO-GPU driver
that will need to pin framebuffers explicitly to prevent eviction of the
actively used buffers by the shrinker. VirtIO-GPU driver will use these
new generic helpers to pin shmem framebuffers.

Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/drm_gem.c | 29 +++++++++++++++++++++++++++++
 include/drm/drm_gem.h     |  3 +++
 2 files changed, 32 insertions(+)

diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index c61674887582..c909c935cfda 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -1179,6 +1179,35 @@ void drm_gem_unpin(struct drm_gem_object *obj)
 		obj->funcs->unpin(obj);
 }
 
+int drm_gem_pin_unlocked(struct drm_gem_object *obj)
+{
+	int ret;
+
+	if (!obj->funcs->pin)
+		return 0;
+
+	ret = dma_resv_lock_interruptible(obj->resv, NULL);
+	if (ret)
+		return ret;
+
+	ret = obj->funcs->pin(obj);
+	dma_resv_unlock(obj->resv);
+
+	return ret;
+}
+EXPORT_SYMBOL(drm_gem_pin_unlocked);
+
+void drm_gem_unpin_unlocked(struct drm_gem_object *obj)
+{
+	if (!obj->funcs->unpin)
+		return;
+
+	dma_resv_lock(obj->resv, NULL);
+	obj->funcs->unpin(obj);
+	dma_resv_unlock(obj->resv);
+}
+EXPORT_SYMBOL(drm_gem_unpin_unlocked);
+
 int drm_gem_vmap(struct drm_gem_object *obj, struct iosys_map *map)
 {
 	int ret;
diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h
index 0b427939f466..870d81e7a104 100644
--- a/include/drm/drm_gem.h
+++ b/include/drm/drm_gem.h
@@ -413,4 +413,7 @@ int drm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev,
 int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
 void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
 
+int drm_gem_pin_unlocked(struct drm_gem_object *obj);
+void drm_gem_unpin_unlocked(struct drm_gem_object *obj);
+
 #endif /* __DRM_GEM_H__ */
-- 
2.35.3

[PATCH v6 18/22] drm/gem: Add drm_gem_pin_unlocked()

[Dmitry Osipenko]

Add unlocked variants of drm_gem_un/pin() functions and make them public.
These new helpers will take care of GEM dma-reservation locking for DRM
drivers.

We are going to add memory shrinking support to the VirtIO-GPU driver
that will need to pin framebuffers explicitly to prevent eviction of the
actively used buffers by the shrinker. VirtIO-GPU driver will use these
new generic helpers to pin shmem framebuffers.

Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/drm_gem.c | 29 +++++++++++++++++++++++++++++
 include/drm/drm_gem.h     |  3 +++
 2 files changed, 32 insertions(+)

diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index c61674887582..c909c935cfda 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -1179,6 +1179,35 @@ void drm_gem_unpin(struct drm_gem_object *obj)
 		obj->funcs->unpin(obj);
 }
 
+int drm_gem_pin_unlocked(struct drm_gem_object *obj)
+{
+	int ret;
+
+	if (!obj->funcs->pin)
+		return 0;
+
+	ret = dma_resv_lock_interruptible(obj->resv, NULL);
+	if (ret)
+		return ret;
+
+	ret = obj->funcs->pin(obj);
+	dma_resv_unlock(obj->resv);
+
+	return ret;
+}
+EXPORT_SYMBOL(drm_gem_pin_unlocked);
+
+void drm_gem_unpin_unlocked(struct drm_gem_object *obj)
+{
+	if (!obj->funcs->unpin)
+		return;
+
+	dma_resv_lock(obj->resv, NULL);
+	obj->funcs->unpin(obj);
+	dma_resv_unlock(obj->resv);
+}
+EXPORT_SYMBOL(drm_gem_unpin_unlocked);
+
 int drm_gem_vmap(struct drm_gem_object *obj, struct iosys_map *map)
 {
 	int ret;
diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h
index 0b427939f466..870d81e7a104 100644
--- a/include/drm/drm_gem.h
+++ b/include/drm/drm_gem.h
@@ -413,4 +413,7 @@ int drm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev,
 int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
 void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
 
+int drm_gem_pin_unlocked(struct drm_gem_object *obj);
+void drm_gem_unpin_unlocked(struct drm_gem_object *obj);
+
 #endif /* __DRM_GEM_H__ */
-- 
2.35.3

[PATCH v6 19/22] drm/virtio: Support memory shrinking

[Dmitry Osipenko]

Support generic drm-shmem memory shrinker and add new madvise IOCTL to
the VirtIO-GPU driver. BO cache manager of Mesa driver will mark BOs as
"don't need" using the new IOCTL to let shrinker purge the marked BOs on
OOM, the shrinker will also evict unpurgeable shmem BOs from memory if
guest supports SWAP file or partition. Altogether this allows to prevent
OOM kills of guest applications that use VirGL by lowering memory pressure.

Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_drv.h    |  15 ++-
 drivers/gpu/drm/virtio/virtgpu_gem.c    |  55 ++++++++++
 drivers/gpu/drm/virtio/virtgpu_ioctl.c  |  37 +++++++
 drivers/gpu/drm/virtio/virtgpu_kms.c    |   9 ++
 drivers/gpu/drm/virtio/virtgpu_object.c | 138 +++++++++++++++++++-----
 drivers/gpu/drm/virtio/virtgpu_plane.c  |  22 +++-
 drivers/gpu/drm/virtio/virtgpu_vq.c     |  40 +++++++
 include/uapi/drm/virtgpu_drm.h          |  14 +++
 8 files changed, 300 insertions(+), 30 deletions(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
index 81bacc7e1873..26b570029940 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -275,7 +275,7 @@ struct virtio_gpu_fpriv {
 };
 
 /* virtgpu_ioctl.c */
-#define DRM_VIRTIO_NUM_IOCTLS 12
+#define DRM_VIRTIO_NUM_IOCTLS 13
 extern struct drm_ioctl_desc virtio_gpu_ioctls[DRM_VIRTIO_NUM_IOCTLS];
 void virtio_gpu_create_context(struct drm_device *dev, struct drm_file *file);
 
@@ -311,6 +311,10 @@ void virtio_gpu_array_put_free(struct virtio_gpu_object_array *objs);
 void virtio_gpu_array_put_free_delayed(struct virtio_gpu_device *vgdev,
 				       struct virtio_gpu_object_array *objs);
 void virtio_gpu_array_put_free_work(struct work_struct *work);
+int virtio_gpu_array_prepare(struct virtio_gpu_device *vgdev,
+			     struct virtio_gpu_object_array *objs);
+int virtio_gpu_gem_host_mem_release(struct virtio_gpu_object *bo);
+int virtio_gpu_gem_madvise(struct virtio_gpu_object *obj, int madv);
 
 /* virtgpu_vq.c */
 int virtio_gpu_alloc_vbufs(struct virtio_gpu_device *vgdev);
@@ -322,6 +326,8 @@ void virtio_gpu_cmd_create_resource(struct virtio_gpu_device *vgdev,
 				    struct virtio_gpu_fence *fence);
 void virtio_gpu_cmd_unref_resource(struct virtio_gpu_device *vgdev,
 				   struct virtio_gpu_object *bo);
+int virtio_gpu_cmd_release_resource(struct virtio_gpu_device *vgdev,
+				    struct virtio_gpu_object *bo);
 void virtio_gpu_cmd_transfer_to_host_2d(struct virtio_gpu_device *vgdev,
 					uint64_t offset,
 					uint32_t width, uint32_t height,
@@ -342,6 +348,9 @@ void virtio_gpu_object_attach(struct virtio_gpu_device *vgdev,
 			      struct virtio_gpu_object *obj,
 			      struct virtio_gpu_mem_entry *ents,
 			      unsigned int nents);
+void virtio_gpu_object_detach(struct virtio_gpu_device *vgdev,
+			      struct virtio_gpu_object *obj,
+			      struct virtio_gpu_fence *fence);
 int virtio_gpu_attach_status_page(struct virtio_gpu_device *vgdev);
 int virtio_gpu_detach_status_page(struct virtio_gpu_device *vgdev);
 void virtio_gpu_cursor_ping(struct virtio_gpu_device *vgdev,
@@ -486,4 +495,8 @@ void virtio_gpu_vram_unmap_dma_buf(struct device *dev,
 				   struct sg_table *sgt,
 				   enum dma_data_direction dir);
 
+/* virtgpu_gem_shrinker.c */
+int virtio_gpu_gem_shrinker_init(struct virtio_gpu_device *vgdev);
+void virtio_gpu_gem_shrinker_fini(struct virtio_gpu_device *vgdev);
+
 #endif
diff --git a/drivers/gpu/drm/virtio/virtgpu_gem.c b/drivers/gpu/drm/virtio/virtgpu_gem.c
index 7db48d17ee3a..6c5d98e0f071 100644
--- a/drivers/gpu/drm/virtio/virtgpu_gem.c
+++ b/drivers/gpu/drm/virtio/virtgpu_gem.c
@@ -294,3 +294,58 @@ void virtio_gpu_array_put_free_work(struct work_struct *work)
 	}
 	spin_unlock(&vgdev->obj_free_lock);
 }
+
+int virtio_gpu_array_prepare(struct virtio_gpu_device *vgdev,
+			     struct virtio_gpu_object_array *objs)
+{
+	struct virtio_gpu_object *bo;
+	int ret = 0;
+	u32 i;
+
+	for (i = 0; i < objs->nents; i++) {
+		bo = gem_to_virtio_gpu_obj(objs->objs[i]);
+
+		if (virtio_gpu_is_shmem(bo) && bo->detached) {
+			ret = virtio_gpu_reattach_shmem_object(bo);
+			if (ret)
+				break;
+		}
+	}
+
+	return ret;
+}
+
+int virtio_gpu_gem_madvise(struct virtio_gpu_object *bo, int madv)
+{
+	int ret;
+
+	/*
+	 * For now we support only purging BOs that are backed by guest's
+	 * memory.
+	 */
+	if (!virtio_gpu_is_shmem(bo))
+		return true;
+
+	dma_resv_lock(bo->base.base.resv, NULL);
+	ret = drm_gem_shmem_madvise(&bo->base, madv);
+	dma_resv_unlock(bo->base.base.resv);
+
+	return ret;
+}
+
+int virtio_gpu_gem_host_mem_release(struct virtio_gpu_object *bo)
+{
+	struct virtio_gpu_device *vgdev = bo->base.base.dev->dev_private;
+	int err;
+
+	if (bo->created) {
+		err = virtio_gpu_cmd_release_resource(vgdev, bo);
+		if (err)
+			return err;
+
+		virtio_gpu_notify(vgdev);
+		bo->created = false;
+	}
+
+	return 0;
+}
diff --git a/drivers/gpu/drm/virtio/virtgpu_ioctl.c b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
index f8d83358d2a0..55ee9bd2098e 100644
--- a/drivers/gpu/drm/virtio/virtgpu_ioctl.c
+++ b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
@@ -217,6 +217,10 @@ static int virtio_gpu_execbuffer_ioctl(struct drm_device *dev, void *data,
 		ret = virtio_gpu_array_lock_resv(buflist);
 		if (ret)
 			goto out_memdup;
+
+		ret = virtio_gpu_array_prepare(vgdev, buflist);
+		if (ret)
+			goto out_unresv;
 	}
 
 	out_fence = virtio_gpu_fence_alloc(vgdev, fence_ctx, ring_idx);
@@ -423,6 +427,10 @@ static int virtio_gpu_transfer_from_host_ioctl(struct drm_device *dev,
 	if (ret != 0)
 		goto err_put_free;
 
+	ret = virtio_gpu_array_prepare(vgdev, objs);
+	if (ret)
+		goto err_unlock;
+
 	fence = virtio_gpu_fence_alloc(vgdev, vgdev->fence_drv.context, 0);
 	if (!fence) {
 		ret = -ENOMEM;
@@ -482,6 +490,10 @@ static int virtio_gpu_transfer_to_host_ioctl(struct drm_device *dev, void *data,
 		if (ret != 0)
 			goto err_put_free;
 
+		ret = virtio_gpu_array_prepare(vgdev, objs);
+		if (ret)
+			goto err_unlock;
+
 		ret = -ENOMEM;
 		fence = virtio_gpu_fence_alloc(vgdev, vgdev->fence_drv.context,
 					       0);
@@ -836,6 +848,28 @@ static int virtio_gpu_context_init_ioctl(struct drm_device *dev,
 	return ret;
 }
 
+static int virtio_gpu_madvise_ioctl(struct drm_device *dev,
+				    void *data,
+				    struct drm_file *file)
+{
+	struct drm_virtgpu_madvise *args = data;
+	struct virtio_gpu_object *bo;
+	struct drm_gem_object *obj;
+
+	if (args->madv > VIRTGPU_MADV_DONTNEED)
+		return -EOPNOTSUPP;
+
+	obj = drm_gem_object_lookup(file, args->bo_handle);
+	if (!obj)
+		return -ENOENT;
+
+	bo = gem_to_virtio_gpu_obj(obj);
+	args->retained = virtio_gpu_gem_madvise(bo, args->madv);
+	drm_gem_object_put(obj);
+
+	return 0;
+}
+
 struct drm_ioctl_desc virtio_gpu_ioctls[DRM_VIRTIO_NUM_IOCTLS] = {
 	DRM_IOCTL_DEF_DRV(VIRTGPU_MAP, virtio_gpu_map_ioctl,
 			  DRM_RENDER_ALLOW),
@@ -875,4 +909,7 @@ struct drm_ioctl_desc virtio_gpu_ioctls[DRM_VIRTIO_NUM_IOCTLS] = {
 
 	DRM_IOCTL_DEF_DRV(VIRTGPU_CONTEXT_INIT, virtio_gpu_context_init_ioctl,
 			  DRM_RENDER_ALLOW),
+
+	DRM_IOCTL_DEF_DRV(VIRTGPU_MADVISE, virtio_gpu_madvise_ioctl,
+			  DRM_RENDER_ALLOW),
 };
diff --git a/drivers/gpu/drm/virtio/virtgpu_kms.c b/drivers/gpu/drm/virtio/virtgpu_kms.c
index 0d1e3eb61bee..1175999acea1 100644
--- a/drivers/gpu/drm/virtio/virtgpu_kms.c
+++ b/drivers/gpu/drm/virtio/virtgpu_kms.c
@@ -238,6 +238,12 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)
 		goto err_scanouts;
 	}
 
+	ret = drm_gem_shmem_shrinker_register(dev);
+	if (ret) {
+		DRM_ERROR("shrinker init failed\n");
+		goto err_modeset;
+	}
+
 	virtio_device_ready(vgdev->vdev);
 
 	if (num_capsets)
@@ -250,6 +256,8 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)
 			   5 * HZ);
 	return 0;
 
+err_modeset:
+	virtio_gpu_modeset_fini(vgdev);
 err_scanouts:
 	virtio_gpu_free_vbufs(vgdev);
 err_vbufs:
@@ -289,6 +297,7 @@ void virtio_gpu_release(struct drm_device *dev)
 	if (!vgdev)
 		return;
 
+	drm_gem_shmem_shrinker_unregister(dev);
 	virtio_gpu_modeset_fini(vgdev);
 	virtio_gpu_free_vbufs(vgdev);
 	virtio_gpu_cleanup_cap_cache(vgdev);
diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c b/drivers/gpu/drm/virtio/virtgpu_object.c
index 8d7728181de0..ddae4f9402f7 100644
--- a/drivers/gpu/drm/virtio/virtgpu_object.c
+++ b/drivers/gpu/drm/virtio/virtgpu_object.c
@@ -97,39 +97,54 @@ static void virtio_gpu_free_object(struct drm_gem_object *obj)
 	virtio_gpu_cleanup_object(bo);
 }
 
-static const struct drm_gem_object_funcs virtio_gpu_shmem_funcs = {
-	.free = virtio_gpu_free_object,
-	.open = virtio_gpu_gem_object_open,
-	.close = virtio_gpu_gem_object_close,
-	.print_info = drm_gem_shmem_object_print_info,
-	.export = virtgpu_gem_prime_export,
-	.pin = drm_gem_shmem_object_pin,
-	.unpin = drm_gem_shmem_object_unpin,
-	.get_sg_table = drm_gem_shmem_object_get_sg_table,
-	.vmap = drm_gem_shmem_object_vmap,
-	.vunmap = drm_gem_shmem_object_vunmap,
-	.mmap = drm_gem_shmem_object_mmap,
-	.vm_ops = &drm_gem_shmem_vm_ops,
-};
-
-bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo)
+static int virtio_gpu_detach_object_fenced(struct virtio_gpu_object *bo)
 {
-	return bo->base.base.funcs == &virtio_gpu_shmem_funcs;
+	struct virtio_gpu_device *vgdev = bo->base.base.dev->dev_private;
+	struct virtio_gpu_fence *fence;
+
+	fence = virtio_gpu_fence_alloc(vgdev, vgdev->fence_drv.context, 0);
+	if (!fence)
+		return -ENOMEM;
+
+	virtio_gpu_object_detach(vgdev, bo, fence);
+	virtio_gpu_notify(vgdev);
+
+	dma_fence_wait(&fence->f, false);
+	dma_fence_put(&fence->f);
+
+	bo->detached = true;
+
+	return 0;
 }
 
-struct drm_gem_object *virtio_gpu_create_object(struct drm_device *dev,
-						size_t size)
+static int virtio_gpu_shmem_evict(struct drm_gem_shmem_object *shmem)
 {
-	struct virtio_gpu_object_shmem *shmem;
-	struct drm_gem_shmem_object *dshmem;
+	struct virtio_gpu_object *bo = gem_to_virtio_gpu_obj(&shmem->base);
+	int err;
+
+	/*
+	 * At first tell host to stop using guest's memory to ensure that
+	 * host won't touch the released guest's memory once it's gone.
+	 */
+	if (!shmem->evicted) {
+		err = virtio_gpu_detach_object_fenced(bo);
+		if (err)
+			return err;
+	}
 
-	shmem = kzalloc(sizeof(*shmem), GFP_KERNEL);
-	if (!shmem)
-		return ERR_PTR(-ENOMEM);
+	if (drm_gem_shmem_is_purgeable(shmem)) {
+		err = virtio_gpu_gem_host_mem_release(bo);
+		if (err) {
+			virtio_gpu_reattach_shmem_object(bo);
+			return err;
+		}
 
-	dshmem = &shmem->base.base;
-	dshmem->base.funcs = &virtio_gpu_shmem_funcs;
-	return &dshmem->base;
+		drm_gem_shmem_purge(shmem);
+	} else {
+		drm_gem_shmem_evict(shmem);
+	}
+
+	return 0;
 }
 
 static int virtio_gpu_object_shmem_init(struct virtio_gpu_device *vgdev,
@@ -176,6 +191,64 @@ static int virtio_gpu_object_shmem_init(struct virtio_gpu_device *vgdev,
 	return 0;
 }
 
+int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo)
+{
+	struct virtio_gpu_device *vgdev = bo->base.base.dev->dev_private;
+	struct virtio_gpu_mem_entry *ents;
+	unsigned int nents;
+	int err;
+
+	err = drm_gem_shmem_swap_in(&bo->base);
+	if (err)
+		return err;
+
+	err = virtio_gpu_object_shmem_init(vgdev, bo, &ents, &nents);
+	if (err)
+		return err;
+
+	virtio_gpu_object_attach(vgdev, bo, ents, nents);
+	virtio_gpu_notify(vgdev);
+
+	bo->detached = false;
+
+	return 0;
+}
+
+static const struct drm_gem_object_funcs virtio_gpu_shmem_funcs = {
+	.free = virtio_gpu_free_object,
+	.open = virtio_gpu_gem_object_open,
+	.close = virtio_gpu_gem_object_close,
+	.print_info = drm_gem_shmem_object_print_info,
+	.export = virtgpu_gem_prime_export,
+	.pin = drm_gem_shmem_object_pin,
+	.unpin = drm_gem_shmem_object_unpin,
+	.get_sg_table = drm_gem_shmem_object_get_sg_table,
+	.vmap = drm_gem_shmem_object_vmap,
+	.vunmap = drm_gem_shmem_object_vunmap,
+	.mmap = drm_gem_shmem_object_mmap,
+	.vm_ops = &drm_gem_shmem_vm_ops,
+};
+
+bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo)
+{
+	return bo->base.base.funcs == &virtio_gpu_shmem_funcs;
+}
+
+struct drm_gem_object *virtio_gpu_create_object(struct drm_device *dev,
+						size_t size)
+{
+	struct virtio_gpu_object_shmem *shmem;
+	struct drm_gem_shmem_object *dshmem;
+
+	shmem = kzalloc(sizeof(*shmem), GFP_KERNEL);
+	if (!shmem)
+		return ERR_PTR(-ENOMEM);
+
+	dshmem = &shmem->base.base;
+	dshmem->base.funcs = &virtio_gpu_shmem_funcs;
+	return &dshmem->base;
+}
+
 int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
 			     struct virtio_gpu_object_params *params,
 			     struct virtio_gpu_object **bo_ptr,
@@ -201,6 +274,7 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
 		goto err_free_gem;
 
 	bo->dumb = params->dumb;
+	bo->base.evict = virtio_gpu_shmem_evict;
 
 	ret = virtio_gpu_object_shmem_init(vgdev, bo, &ents, &nents);
 	if (ret != 0)
@@ -228,10 +302,20 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
 		virtio_gpu_cmd_resource_create_3d(vgdev, bo, params,
 						  objs, fence);
 		virtio_gpu_object_attach(vgdev, bo, ents, nents);
+
+		shmem_obj->pages_mark_dirty_on_put = 1;
+
+		drm_gem_shmem_set_evictable(shmem_obj);
+		drm_gem_shmem_set_purgeable(shmem_obj);
 	} else {
 		virtio_gpu_cmd_create_resource(vgdev, bo, params,
 					       objs, fence);
 		virtio_gpu_object_attach(vgdev, bo, ents, nents);
+
+		shmem_obj->pages_mark_dirty_on_put = 1;
+
+		drm_gem_shmem_set_evictable(shmem_obj);
+		drm_gem_shmem_set_purgeable(shmem_obj);
 	}
 
 	*bo_ptr = bo;
diff --git a/drivers/gpu/drm/virtio/virtgpu_plane.c b/drivers/gpu/drm/virtio/virtgpu_plane.c
index 7148f3813d8b..246bf0c54996 100644
--- a/drivers/gpu/drm/virtio/virtgpu_plane.c
+++ b/drivers/gpu/drm/virtio/virtgpu_plane.c
@@ -246,20 +246,32 @@ static int virtio_gpu_plane_prepare_fb(struct drm_plane *plane,
 	struct virtio_gpu_device *vgdev = dev->dev_private;
 	struct virtio_gpu_framebuffer *vgfb;
 	struct virtio_gpu_object *bo;
+	int err;
 
 	if (!new_state->fb)
 		return 0;
 
 	vgfb = to_virtio_gpu_framebuffer(new_state->fb);
 	bo = gem_to_virtio_gpu_obj(vgfb->base.obj[0]);
-	if (!bo || (plane->type == DRM_PLANE_TYPE_PRIMARY && !bo->guest_blob))
+
+	if (virtio_gpu_is_shmem(bo)) {
+		err = drm_gem_pin_unlocked(&bo->base.base);
+		if (err)
+			return err;
+	}
+
+	if (plane->type == DRM_PLANE_TYPE_PRIMARY && !bo->guest_blob)
 		return 0;
 
 	if (bo->dumb && (plane->state->fb != new_state->fb)) {
 		vgfb->fence = virtio_gpu_fence_alloc(vgdev, vgdev->fence_drv.context,
 						     0);
-		if (!vgfb->fence)
+		if (!vgfb->fence) {
+			if (virtio_gpu_is_shmem(bo))
+				drm_gem_unpin_unlocked(&bo->base.base);
+
 			return -ENOMEM;
+		}
 	}
 
 	return 0;
@@ -269,15 +281,21 @@ static void virtio_gpu_plane_cleanup_fb(struct drm_plane *plane,
 					struct drm_plane_state *state)
 {
 	struct virtio_gpu_framebuffer *vgfb;
+	struct virtio_gpu_object *bo;
 
 	if (!state->fb)
 		return;
 
 	vgfb = to_virtio_gpu_framebuffer(state->fb);
+	bo = gem_to_virtio_gpu_obj(vgfb->base.obj[0]);
+
 	if (vgfb->fence) {
 		dma_fence_put(&vgfb->fence->f);
 		vgfb->fence = NULL;
 	}
+
+	if (virtio_gpu_is_shmem(bo))
+		drm_gem_unpin_unlocked(&bo->base.base);
 }
 
 static void virtio_gpu_cursor_plane_update(struct drm_plane *plane,
diff --git a/drivers/gpu/drm/virtio/virtgpu_vq.c b/drivers/gpu/drm/virtio/virtgpu_vq.c
index 06566e44307d..2a04dad1ae89 100644
--- a/drivers/gpu/drm/virtio/virtgpu_vq.c
+++ b/drivers/gpu/drm/virtio/virtgpu_vq.c
@@ -536,6 +536,21 @@ void virtio_gpu_cmd_unref_resource(struct virtio_gpu_device *vgdev,
 		virtio_gpu_cleanup_object(bo);
 }
 
+int virtio_gpu_cmd_release_resource(struct virtio_gpu_device *vgdev,
+				    struct virtio_gpu_object *bo)
+{
+	struct virtio_gpu_resource_unref *cmd_p;
+	struct virtio_gpu_vbuffer *vbuf;
+
+	cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
+	memset(cmd_p, 0, sizeof(*cmd_p));
+
+	cmd_p->hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_RESOURCE_UNREF);
+	cmd_p->resource_id = cpu_to_le32(bo->hw_res_handle);
+
+	return virtio_gpu_queue_ctrl_buffer(vgdev, vbuf);
+}
+
 void virtio_gpu_cmd_set_scanout(struct virtio_gpu_device *vgdev,
 				uint32_t scanout_id, uint32_t resource_id,
 				uint32_t width, uint32_t height,
@@ -636,6 +651,23 @@ virtio_gpu_cmd_resource_attach_backing(struct virtio_gpu_device *vgdev,
 	virtio_gpu_queue_fenced_ctrl_buffer(vgdev, vbuf, fence);
 }
 
+static void
+virtio_gpu_cmd_resource_detach_backing(struct virtio_gpu_device *vgdev,
+				       u32 resource_id,
+				       struct virtio_gpu_fence *fence)
+{
+	struct virtio_gpu_resource_attach_backing *cmd_p;
+	struct virtio_gpu_vbuffer *vbuf;
+
+	cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
+	memset(cmd_p, 0, sizeof(*cmd_p));
+
+	cmd_p->hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_RESOURCE_DETACH_BACKING);
+	cmd_p->resource_id = cpu_to_le32(resource_id);
+
+	virtio_gpu_queue_fenced_ctrl_buffer(vgdev, vbuf, fence);
+}
+
 static void virtio_gpu_cmd_get_display_info_cb(struct virtio_gpu_device *vgdev,
 					       struct virtio_gpu_vbuffer *vbuf)
 {
@@ -1099,6 +1131,14 @@ void virtio_gpu_object_attach(struct virtio_gpu_device *vgdev,
 					       ents, nents, NULL);
 }
 
+void virtio_gpu_object_detach(struct virtio_gpu_device *vgdev,
+			      struct virtio_gpu_object *obj,
+			      struct virtio_gpu_fence *fence)
+{
+	virtio_gpu_cmd_resource_detach_backing(vgdev, obj->hw_res_handle,
+					       fence);
+}
+
 void virtio_gpu_cursor_ping(struct virtio_gpu_device *vgdev,
 			    struct virtio_gpu_output *output)
 {
diff --git a/include/uapi/drm/virtgpu_drm.h b/include/uapi/drm/virtgpu_drm.h
index 0512fde5e697..12197d8e9759 100644
--- a/include/uapi/drm/virtgpu_drm.h
+++ b/include/uapi/drm/virtgpu_drm.h
@@ -48,6 +48,7 @@ extern "C" {
 #define DRM_VIRTGPU_GET_CAPS  0x09
 #define DRM_VIRTGPU_RESOURCE_CREATE_BLOB 0x0a
 #define DRM_VIRTGPU_CONTEXT_INIT 0x0b
+#define DRM_VIRTGPU_MADVISE 0x0c
 
 #define VIRTGPU_EXECBUF_FENCE_FD_IN	0x01
 #define VIRTGPU_EXECBUF_FENCE_FD_OUT	0x02
@@ -196,6 +197,15 @@ struct drm_virtgpu_context_init {
 	__u64 ctx_set_params;
 };
 
+#define VIRTGPU_MADV_WILLNEED 0
+#define VIRTGPU_MADV_DONTNEED 1
+struct drm_virtgpu_madvise {
+	__u32 bo_handle;
+	__u32 retained; /* out, non-zero if BO can be used */
+	__u32 madv;
+	__u32 pad;
+};
+
 /*
  * Event code that's given when VIRTGPU_CONTEXT_PARAM_POLL_RINGS_MASK is in
  * effect.  The event size is sizeof(drm_event), since there is no additional
@@ -246,6 +256,10 @@ struct drm_virtgpu_context_init {
 	DRM_IOWR(DRM_COMMAND_BASE + DRM_VIRTGPU_CONTEXT_INIT,		\
 		struct drm_virtgpu_context_init)
 
+#define DRM_IOCTL_VIRTGPU_MADVISE \
+	DRM_IOWR(DRM_COMMAND_BASE + DRM_VIRTGPU_MADVISE, \
+		 struct drm_virtgpu_madvise)
+
 #if defined(__cplusplus)
 }
 #endif
-- 
2.35.3

[PATCH v6 19/22] drm/virtio: Support memory shrinking

[Dmitry Osipenko]

Support generic drm-shmem memory shrinker and add new madvise IOCTL to
the VirtIO-GPU driver. BO cache manager of Mesa driver will mark BOs as
"don't need" using the new IOCTL to let shrinker purge the marked BOs on
OOM, the shrinker will also evict unpurgeable shmem BOs from memory if
guest supports SWAP file or partition. Altogether this allows to prevent
OOM kills of guest applications that use VirGL by lowering memory pressure.

Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_drv.h    |  15 ++-
 drivers/gpu/drm/virtio/virtgpu_gem.c    |  55 ++++++++++
 drivers/gpu/drm/virtio/virtgpu_ioctl.c  |  37 +++++++
 drivers/gpu/drm/virtio/virtgpu_kms.c    |   9 ++
 drivers/gpu/drm/virtio/virtgpu_object.c | 138 +++++++++++++++++++-----
 drivers/gpu/drm/virtio/virtgpu_plane.c  |  22 +++-
 drivers/gpu/drm/virtio/virtgpu_vq.c     |  40 +++++++
 include/uapi/drm/virtgpu_drm.h          |  14 +++
 8 files changed, 300 insertions(+), 30 deletions(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
index 81bacc7e1873..26b570029940 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -275,7 +275,7 @@ struct virtio_gpu_fpriv {
 };
 
 /* virtgpu_ioctl.c */
-#define DRM_VIRTIO_NUM_IOCTLS 12
+#define DRM_VIRTIO_NUM_IOCTLS 13
 extern struct drm_ioctl_desc virtio_gpu_ioctls[DRM_VIRTIO_NUM_IOCTLS];
 void virtio_gpu_create_context(struct drm_device *dev, struct drm_file *file);
 
@@ -311,6 +311,10 @@ void virtio_gpu_array_put_free(struct virtio_gpu_object_array *objs);
 void virtio_gpu_array_put_free_delayed(struct virtio_gpu_device *vgdev,
 				       struct virtio_gpu_object_array *objs);
 void virtio_gpu_array_put_free_work(struct work_struct *work);
+int virtio_gpu_array_prepare(struct virtio_gpu_device *vgdev,
+			     struct virtio_gpu_object_array *objs);
+int virtio_gpu_gem_host_mem_release(struct virtio_gpu_object *bo);
+int virtio_gpu_gem_madvise(struct virtio_gpu_object *obj, int madv);
 
 /* virtgpu_vq.c */
 int virtio_gpu_alloc_vbufs(struct virtio_gpu_device *vgdev);
@@ -322,6 +326,8 @@ void virtio_gpu_cmd_create_resource(struct virtio_gpu_device *vgdev,
 				    struct virtio_gpu_fence *fence);
 void virtio_gpu_cmd_unref_resource(struct virtio_gpu_device *vgdev,
 				   struct virtio_gpu_object *bo);
+int virtio_gpu_cmd_release_resource(struct virtio_gpu_device *vgdev,
+				    struct virtio_gpu_object *bo);
 void virtio_gpu_cmd_transfer_to_host_2d(struct virtio_gpu_device *vgdev,
 					uint64_t offset,
 					uint32_t width, uint32_t height,
@@ -342,6 +348,9 @@ void virtio_gpu_object_attach(struct virtio_gpu_device *vgdev,
 			      struct virtio_gpu_object *obj,
 			      struct virtio_gpu_mem_entry *ents,
 			      unsigned int nents);
+void virtio_gpu_object_detach(struct virtio_gpu_device *vgdev,
+			      struct virtio_gpu_object *obj,
+			      struct virtio_gpu_fence *fence);
 int virtio_gpu_attach_status_page(struct virtio_gpu_device *vgdev);
 int virtio_gpu_detach_status_page(struct virtio_gpu_device *vgdev);
 void virtio_gpu_cursor_ping(struct virtio_gpu_device *vgdev,
@@ -486,4 +495,8 @@ void virtio_gpu_vram_unmap_dma_buf(struct device *dev,
 				   struct sg_table *sgt,
 				   enum dma_data_direction dir);
 
+/* virtgpu_gem_shrinker.c */
+int virtio_gpu_gem_shrinker_init(struct virtio_gpu_device *vgdev);
+void virtio_gpu_gem_shrinker_fini(struct virtio_gpu_device *vgdev);
+
 #endif
diff --git a/drivers/gpu/drm/virtio/virtgpu_gem.c b/drivers/gpu/drm/virtio/virtgpu_gem.c
index 7db48d17ee3a..6c5d98e0f071 100644
--- a/drivers/gpu/drm/virtio/virtgpu_gem.c
+++ b/drivers/gpu/drm/virtio/virtgpu_gem.c
@@ -294,3 +294,58 @@ void virtio_gpu_array_put_free_work(struct work_struct *work)
 	}
 	spin_unlock(&vgdev->obj_free_lock);
 }
+
+int virtio_gpu_array_prepare(struct virtio_gpu_device *vgdev,
+			     struct virtio_gpu_object_array *objs)
+{
+	struct virtio_gpu_object *bo;
+	int ret = 0;
+	u32 i;
+
+	for (i = 0; i < objs->nents; i++) {
+		bo = gem_to_virtio_gpu_obj(objs->objs[i]);
+
+		if (virtio_gpu_is_shmem(bo) && bo->detached) {
+			ret = virtio_gpu_reattach_shmem_object(bo);
+			if (ret)
+				break;
+		}
+	}
+
+	return ret;
+}
+
+int virtio_gpu_gem_madvise(struct virtio_gpu_object *bo, int madv)
+{
+	int ret;
+
+	/*
+	 * For now we support only purging BOs that are backed by guest's
+	 * memory.
+	 */
+	if (!virtio_gpu_is_shmem(bo))
+		return true;
+
+	dma_resv_lock(bo->base.base.resv, NULL);
+	ret = drm_gem_shmem_madvise(&bo->base, madv);
+	dma_resv_unlock(bo->base.base.resv);
+
+	return ret;
+}
+
+int virtio_gpu_gem_host_mem_release(struct virtio_gpu_object *bo)
+{
+	struct virtio_gpu_device *vgdev = bo->base.base.dev->dev_private;
+	int err;
+
+	if (bo->created) {
+		err = virtio_gpu_cmd_release_resource(vgdev, bo);
+		if (err)
+			return err;
+
+		virtio_gpu_notify(vgdev);
+		bo->created = false;
+	}
+
+	return 0;
+}
diff --git a/drivers/gpu/drm/virtio/virtgpu_ioctl.c b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
index f8d83358d2a0..55ee9bd2098e 100644
--- a/drivers/gpu/drm/virtio/virtgpu_ioctl.c
+++ b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
@@ -217,6 +217,10 @@ static int virtio_gpu_execbuffer_ioctl(struct drm_device *dev, void *data,
 		ret = virtio_gpu_array_lock_resv(buflist);
 		if (ret)
 			goto out_memdup;
+
+		ret = virtio_gpu_array_prepare(vgdev, buflist);
+		if (ret)
+			goto out_unresv;
 	}
 
 	out_fence = virtio_gpu_fence_alloc(vgdev, fence_ctx, ring_idx);
@@ -423,6 +427,10 @@ static int virtio_gpu_transfer_from_host_ioctl(struct drm_device *dev,
 	if (ret != 0)
 		goto err_put_free;
 
+	ret = virtio_gpu_array_prepare(vgdev, objs);
+	if (ret)
+		goto err_unlock;
+
 	fence = virtio_gpu_fence_alloc(vgdev, vgdev->fence_drv.context, 0);
 	if (!fence) {
 		ret = -ENOMEM;
@@ -482,6 +490,10 @@ static int virtio_gpu_transfer_to_host_ioctl(struct drm_device *dev, void *data,
 		if (ret != 0)
 			goto err_put_free;
 
+		ret = virtio_gpu_array_prepare(vgdev, objs);
+		if (ret)
+			goto err_unlock;
+
 		ret = -ENOMEM;
 		fence = virtio_gpu_fence_alloc(vgdev, vgdev->fence_drv.context,
 					       0);
@@ -836,6 +848,28 @@ static int virtio_gpu_context_init_ioctl(struct drm_device *dev,
 	return ret;
 }
 
+static int virtio_gpu_madvise_ioctl(struct drm_device *dev,
+				    void *data,
+				    struct drm_file *file)
+{
+	struct drm_virtgpu_madvise *args = data;
+	struct virtio_gpu_object *bo;
+	struct drm_gem_object *obj;
+
+	if (args->madv > VIRTGPU_MADV_DONTNEED)
+		return -EOPNOTSUPP;
+
+	obj = drm_gem_object_lookup(file, args->bo_handle);
+	if (!obj)
+		return -ENOENT;
+
+	bo = gem_to_virtio_gpu_obj(obj);
+	args->retained = virtio_gpu_gem_madvise(bo, args->madv);
+	drm_gem_object_put(obj);
+
+	return 0;
+}
+
 struct drm_ioctl_desc virtio_gpu_ioctls[DRM_VIRTIO_NUM_IOCTLS] = {
 	DRM_IOCTL_DEF_DRV(VIRTGPU_MAP, virtio_gpu_map_ioctl,
 			  DRM_RENDER_ALLOW),
@@ -875,4 +909,7 @@ struct drm_ioctl_desc virtio_gpu_ioctls[DRM_VIRTIO_NUM_IOCTLS] = {
 
 	DRM_IOCTL_DEF_DRV(VIRTGPU_CONTEXT_INIT, virtio_gpu_context_init_ioctl,
 			  DRM_RENDER_ALLOW),
+
+	DRM_IOCTL_DEF_DRV(VIRTGPU_MADVISE, virtio_gpu_madvise_ioctl,
+			  DRM_RENDER_ALLOW),
 };
diff --git a/drivers/gpu/drm/virtio/virtgpu_kms.c b/drivers/gpu/drm/virtio/virtgpu_kms.c
index 0d1e3eb61bee..1175999acea1 100644
--- a/drivers/gpu/drm/virtio/virtgpu_kms.c
+++ b/drivers/gpu/drm/virtio/virtgpu_kms.c
@@ -238,6 +238,12 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)
 		goto err_scanouts;
 	}
 
+	ret = drm_gem_shmem_shrinker_register(dev);
+	if (ret) {
+		DRM_ERROR("shrinker init failed\n");
+		goto err_modeset;
+	}
+
 	virtio_device_ready(vgdev->vdev);
 
 	if (num_capsets)
@@ -250,6 +256,8 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)
 			   5 * HZ);
 	return 0;
 
+err_modeset:
+	virtio_gpu_modeset_fini(vgdev);
 err_scanouts:
 	virtio_gpu_free_vbufs(vgdev);
 err_vbufs:
@@ -289,6 +297,7 @@ void virtio_gpu_release(struct drm_device *dev)
 	if (!vgdev)
 		return;
 
+	drm_gem_shmem_shrinker_unregister(dev);
 	virtio_gpu_modeset_fini(vgdev);
 	virtio_gpu_free_vbufs(vgdev);
 	virtio_gpu_cleanup_cap_cache(vgdev);
diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c b/drivers/gpu/drm/virtio/virtgpu_object.c
index 8d7728181de0..ddae4f9402f7 100644
--- a/drivers/gpu/drm/virtio/virtgpu_object.c
+++ b/drivers/gpu/drm/virtio/virtgpu_object.c
@@ -97,39 +97,54 @@ static void virtio_gpu_free_object(struct drm_gem_object *obj)
 	virtio_gpu_cleanup_object(bo);
 }
 
-static const struct drm_gem_object_funcs virtio_gpu_shmem_funcs = {
-	.free = virtio_gpu_free_object,
-	.open = virtio_gpu_gem_object_open,
-	.close = virtio_gpu_gem_object_close,
-	.print_info = drm_gem_shmem_object_print_info,
-	.export = virtgpu_gem_prime_export,
-	.pin = drm_gem_shmem_object_pin,
-	.unpin = drm_gem_shmem_object_unpin,
-	.get_sg_table = drm_gem_shmem_object_get_sg_table,
-	.vmap = drm_gem_shmem_object_vmap,
-	.vunmap = drm_gem_shmem_object_vunmap,
-	.mmap = drm_gem_shmem_object_mmap,
-	.vm_ops = &drm_gem_shmem_vm_ops,
-};
-
-bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo)
+static int virtio_gpu_detach_object_fenced(struct virtio_gpu_object *bo)
 {
-	return bo->base.base.funcs == &virtio_gpu_shmem_funcs;
+	struct virtio_gpu_device *vgdev = bo->base.base.dev->dev_private;
+	struct virtio_gpu_fence *fence;
+
+	fence = virtio_gpu_fence_alloc(vgdev, vgdev->fence_drv.context, 0);
+	if (!fence)
+		return -ENOMEM;
+
+	virtio_gpu_object_detach(vgdev, bo, fence);
+	virtio_gpu_notify(vgdev);
+
+	dma_fence_wait(&fence->f, false);
+	dma_fence_put(&fence->f);
+
+	bo->detached = true;
+
+	return 0;
 }
 
-struct drm_gem_object *virtio_gpu_create_object(struct drm_device *dev,
-						size_t size)
+static int virtio_gpu_shmem_evict(struct drm_gem_shmem_object *shmem)
 {
-	struct virtio_gpu_object_shmem *shmem;
-	struct drm_gem_shmem_object *dshmem;
+	struct virtio_gpu_object *bo = gem_to_virtio_gpu_obj(&shmem->base);
+	int err;
+
+	/*
+	 * At first tell host to stop using guest's memory to ensure that
+	 * host won't touch the released guest's memory once it's gone.
+	 */
+	if (!shmem->evicted) {
+		err = virtio_gpu_detach_object_fenced(bo);
+		if (err)
+			return err;
+	}
 
-	shmem = kzalloc(sizeof(*shmem), GFP_KERNEL);
-	if (!shmem)
-		return ERR_PTR(-ENOMEM);
+	if (drm_gem_shmem_is_purgeable(shmem)) {
+		err = virtio_gpu_gem_host_mem_release(bo);
+		if (err) {
+			virtio_gpu_reattach_shmem_object(bo);
+			return err;
+		}
 
-	dshmem = &shmem->base.base;
-	dshmem->base.funcs = &virtio_gpu_shmem_funcs;
-	return &dshmem->base;
+		drm_gem_shmem_purge(shmem);
+	} else {
+		drm_gem_shmem_evict(shmem);
+	}
+
+	return 0;
 }
 
 static int virtio_gpu_object_shmem_init(struct virtio_gpu_device *vgdev,
@@ -176,6 +191,64 @@ static int virtio_gpu_object_shmem_init(struct virtio_gpu_device *vgdev,
 	return 0;
 }
 
+int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo)
+{
+	struct virtio_gpu_device *vgdev = bo->base.base.dev->dev_private;
+	struct virtio_gpu_mem_entry *ents;
+	unsigned int nents;
+	int err;
+
+	err = drm_gem_shmem_swap_in(&bo->base);
+	if (err)
+		return err;
+
+	err = virtio_gpu_object_shmem_init(vgdev, bo, &ents, &nents);
+	if (err)
+		return err;
+
+	virtio_gpu_object_attach(vgdev, bo, ents, nents);
+	virtio_gpu_notify(vgdev);
+
+	bo->detached = false;
+
+	return 0;
+}
+
+static const struct drm_gem_object_funcs virtio_gpu_shmem_funcs = {
+	.free = virtio_gpu_free_object,
+	.open = virtio_gpu_gem_object_open,
+	.close = virtio_gpu_gem_object_close,
+	.print_info = drm_gem_shmem_object_print_info,
+	.export = virtgpu_gem_prime_export,
+	.pin = drm_gem_shmem_object_pin,
+	.unpin = drm_gem_shmem_object_unpin,
+	.get_sg_table = drm_gem_shmem_object_get_sg_table,
+	.vmap = drm_gem_shmem_object_vmap,
+	.vunmap = drm_gem_shmem_object_vunmap,
+	.mmap = drm_gem_shmem_object_mmap,
+	.vm_ops = &drm_gem_shmem_vm_ops,
+};
+
+bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo)
+{
+	return bo->base.base.funcs == &virtio_gpu_shmem_funcs;
+}
+
+struct drm_gem_object *virtio_gpu_create_object(struct drm_device *dev,
+						size_t size)
+{
+	struct virtio_gpu_object_shmem *shmem;
+	struct drm_gem_shmem_object *dshmem;
+
+	shmem = kzalloc(sizeof(*shmem), GFP_KERNEL);
+	if (!shmem)
+		return ERR_PTR(-ENOMEM);
+
+	dshmem = &shmem->base.base;
+	dshmem->base.funcs = &virtio_gpu_shmem_funcs;
+	return &dshmem->base;
+}
+
 int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
 			     struct virtio_gpu_object_params *params,
 			     struct virtio_gpu_object **bo_ptr,
@@ -201,6 +274,7 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
 		goto err_free_gem;
 
 	bo->dumb = params->dumb;
+	bo->base.evict = virtio_gpu_shmem_evict;
 
 	ret = virtio_gpu_object_shmem_init(vgdev, bo, &ents, &nents);
 	if (ret != 0)
@@ -228,10 +302,20 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
 		virtio_gpu_cmd_resource_create_3d(vgdev, bo, params,
 						  objs, fence);
 		virtio_gpu_object_attach(vgdev, bo, ents, nents);
+
+		shmem_obj->pages_mark_dirty_on_put = 1;
+
+		drm_gem_shmem_set_evictable(shmem_obj);
+		drm_gem_shmem_set_purgeable(shmem_obj);
 	} else {
 		virtio_gpu_cmd_create_resource(vgdev, bo, params,
 					       objs, fence);
 		virtio_gpu_object_attach(vgdev, bo, ents, nents);
+
+		shmem_obj->pages_mark_dirty_on_put = 1;
+
+		drm_gem_shmem_set_evictable(shmem_obj);
+		drm_gem_shmem_set_purgeable(shmem_obj);
 	}
 
 	*bo_ptr = bo;
diff --git a/drivers/gpu/drm/virtio/virtgpu_plane.c b/drivers/gpu/drm/virtio/virtgpu_plane.c
index 7148f3813d8b..246bf0c54996 100644
--- a/drivers/gpu/drm/virtio/virtgpu_plane.c
+++ b/drivers/gpu/drm/virtio/virtgpu_plane.c
@@ -246,20 +246,32 @@ static int virtio_gpu_plane_prepare_fb(struct drm_plane *plane,
 	struct virtio_gpu_device *vgdev = dev->dev_private;
 	struct virtio_gpu_framebuffer *vgfb;
 	struct virtio_gpu_object *bo;
+	int err;
 
 	if (!new_state->fb)
 		return 0;
 
 	vgfb = to_virtio_gpu_framebuffer(new_state->fb);
 	bo = gem_to_virtio_gpu_obj(vgfb->base.obj[0]);
-	if (!bo || (plane->type == DRM_PLANE_TYPE_PRIMARY && !bo->guest_blob))
+
+	if (virtio_gpu_is_shmem(bo)) {
+		err = drm_gem_pin_unlocked(&bo->base.base);
+		if (err)
+			return err;
+	}
+
+	if (plane->type == DRM_PLANE_TYPE_PRIMARY && !bo->guest_blob)
 		return 0;
 
 	if (bo->dumb && (plane->state->fb != new_state->fb)) {
 		vgfb->fence = virtio_gpu_fence_alloc(vgdev, vgdev->fence_drv.context,
 						     0);
-		if (!vgfb->fence)
+		if (!vgfb->fence) {
+			if (virtio_gpu_is_shmem(bo))
+				drm_gem_unpin_unlocked(&bo->base.base);
+
 			return -ENOMEM;
+		}
 	}
 
 	return 0;
@@ -269,15 +281,21 @@ static void virtio_gpu_plane_cleanup_fb(struct drm_plane *plane,
 					struct drm_plane_state *state)
 {
 	struct virtio_gpu_framebuffer *vgfb;
+	struct virtio_gpu_object *bo;
 
 	if (!state->fb)
 		return;
 
 	vgfb = to_virtio_gpu_framebuffer(state->fb);
+	bo = gem_to_virtio_gpu_obj(vgfb->base.obj[0]);
+
 	if (vgfb->fence) {
 		dma_fence_put(&vgfb->fence->f);
 		vgfb->fence = NULL;
 	}
+
+	if (virtio_gpu_is_shmem(bo))
+		drm_gem_unpin_unlocked(&bo->base.base);
 }
 
 static void virtio_gpu_cursor_plane_update(struct drm_plane *plane,
diff --git a/drivers/gpu/drm/virtio/virtgpu_vq.c b/drivers/gpu/drm/virtio/virtgpu_vq.c
index 06566e44307d..2a04dad1ae89 100644
--- a/drivers/gpu/drm/virtio/virtgpu_vq.c
+++ b/drivers/gpu/drm/virtio/virtgpu_vq.c
@@ -536,6 +536,21 @@ void virtio_gpu_cmd_unref_resource(struct virtio_gpu_device *vgdev,
 		virtio_gpu_cleanup_object(bo);
 }
 
+int virtio_gpu_cmd_release_resource(struct virtio_gpu_device *vgdev,
+				    struct virtio_gpu_object *bo)
+{
+	struct virtio_gpu_resource_unref *cmd_p;
+	struct virtio_gpu_vbuffer *vbuf;
+
+	cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
+	memset(cmd_p, 0, sizeof(*cmd_p));
+
+	cmd_p->hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_RESOURCE_UNREF);
+	cmd_p->resource_id = cpu_to_le32(bo->hw_res_handle);
+
+	return virtio_gpu_queue_ctrl_buffer(vgdev, vbuf);
+}
+
 void virtio_gpu_cmd_set_scanout(struct virtio_gpu_device *vgdev,
 				uint32_t scanout_id, uint32_t resource_id,
 				uint32_t width, uint32_t height,
@@ -636,6 +651,23 @@ virtio_gpu_cmd_resource_attach_backing(struct virtio_gpu_device *vgdev,
 	virtio_gpu_queue_fenced_ctrl_buffer(vgdev, vbuf, fence);
 }
 
+static void
+virtio_gpu_cmd_resource_detach_backing(struct virtio_gpu_device *vgdev,
+				       u32 resource_id,
+				       struct virtio_gpu_fence *fence)
+{
+	struct virtio_gpu_resource_attach_backing *cmd_p;
+	struct virtio_gpu_vbuffer *vbuf;
+
+	cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
+	memset(cmd_p, 0, sizeof(*cmd_p));
+
+	cmd_p->hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_RESOURCE_DETACH_BACKING);
+	cmd_p->resource_id = cpu_to_le32(resource_id);
+
+	virtio_gpu_queue_fenced_ctrl_buffer(vgdev, vbuf, fence);
+}
+
 static void virtio_gpu_cmd_get_display_info_cb(struct virtio_gpu_device *vgdev,
 					       struct virtio_gpu_vbuffer *vbuf)
 {
@@ -1099,6 +1131,14 @@ void virtio_gpu_object_attach(struct virtio_gpu_device *vgdev,
 					       ents, nents, NULL);
 }
 
+void virtio_gpu_object_detach(struct virtio_gpu_device *vgdev,
+			      struct virtio_gpu_object *obj,
+			      struct virtio_gpu_fence *fence)
+{
+	virtio_gpu_cmd_resource_detach_backing(vgdev, obj->hw_res_handle,
+					       fence);
+}
+
 void virtio_gpu_cursor_ping(struct virtio_gpu_device *vgdev,
 			    struct virtio_gpu_output *output)
 {
diff --git a/include/uapi/drm/virtgpu_drm.h b/include/uapi/drm/virtgpu_drm.h
index 0512fde5e697..12197d8e9759 100644
--- a/include/uapi/drm/virtgpu_drm.h
+++ b/include/uapi/drm/virtgpu_drm.h
@@ -48,6 +48,7 @@ extern "C" {
 #define DRM_VIRTGPU_GET_CAPS  0x09
 #define DRM_VIRTGPU_RESOURCE_CREATE_BLOB 0x0a
 #define DRM_VIRTGPU_CONTEXT_INIT 0x0b
+#define DRM_VIRTGPU_MADVISE 0x0c
 
 #define VIRTGPU_EXECBUF_FENCE_FD_IN	0x01
 #define VIRTGPU_EXECBUF_FENCE_FD_OUT	0x02
@@ -196,6 +197,15 @@ struct drm_virtgpu_context_init {
 	__u64 ctx_set_params;
 };
 
+#define VIRTGPU_MADV_WILLNEED 0
+#define VIRTGPU_MADV_DONTNEED 1
+struct drm_virtgpu_madvise {
+	__u32 bo_handle;
+	__u32 retained; /* out, non-zero if BO can be used */
+	__u32 madv;
+	__u32 pad;
+};
+
 /*
  * Event code that's given when VIRTGPU_CONTEXT_PARAM_POLL_RINGS_MASK is in
  * effect.  The event size is sizeof(drm_event), since there is no additional
@@ -246,6 +256,10 @@ struct drm_virtgpu_context_init {
 	DRM_IOWR(DRM_COMMAND_BASE + DRM_VIRTGPU_CONTEXT_INIT,		\
 		struct drm_virtgpu_context_init)
 
+#define DRM_IOCTL_VIRTGPU_MADVISE \
+	DRM_IOWR(DRM_COMMAND_BASE + DRM_VIRTGPU_MADVISE, \
+		 struct drm_virtgpu_madvise)
+
 #if defined(__cplusplus)
 }
 #endif
-- 
2.35.3

[PATCH v6 20/22] drm/virtio: Use dev_is_pci()

[Dmitry Osipenko]

Use common dev_is_pci() helper to replace the strcmp("pci") used by driver.

Suggested-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_drv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.c b/drivers/gpu/drm/virtio/virtgpu_drv.c
index 0141b7df97ec..0035affc3e59 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.c
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.c
@@ -87,7 +87,7 @@ static int virtio_gpu_probe(struct virtio_device *vdev)
 		return PTR_ERR(dev);
 	vdev->priv = dev;
 
-	if (!strcmp(vdev->dev.parent->bus->name, "pci")) {
+	if (dev_is_pci(vdev->dev.parent)) {
 		ret = virtio_gpu_pci_quirk(dev);
 		if (ret)
 			goto err_free;
-- 
2.35.3

[PATCH v6 20/22] drm/virtio: Use dev_is_pci()

[Dmitry Osipenko]

Use common dev_is_pci() helper to replace the strcmp("pci") used by driver.

Suggested-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_drv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.c b/drivers/gpu/drm/virtio/virtgpu_drv.c
index 0141b7df97ec..0035affc3e59 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.c
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.c
@@ -87,7 +87,7 @@ static int virtio_gpu_probe(struct virtio_device *vdev)
 		return PTR_ERR(dev);
 	vdev->priv = dev;
 
-	if (!strcmp(vdev->dev.parent->bus->name, "pci")) {
+	if (dev_is_pci(vdev->dev.parent)) {
 		ret = virtio_gpu_pci_quirk(dev);
 		if (ret)
 			goto err_free;
-- 
2.35.3

[PATCH v6 21/22] drm/virtio: Return proper error codes instead of -1

[Dmitry Osipenko]

Don't return -1 in error cases, return proper error code. The returned
error codes propagate to error messages and to userspace and it's always
good to have a meaningful error number for debugging purposes.

Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_vq.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_vq.c b/drivers/gpu/drm/virtio/virtgpu_vq.c
index 2a04dad1ae89..40402367d593 100644
--- a/drivers/gpu/drm/virtio/virtgpu_vq.c
+++ b/drivers/gpu/drm/virtio/virtgpu_vq.c
@@ -320,7 +320,7 @@ static int virtio_gpu_queue_ctrl_sgs(struct virtio_gpu_device *vgdev,
 		if (fence && vbuf->objs)
 			virtio_gpu_array_unlock_resv(vbuf->objs);
 		free_vbuf(vgdev, vbuf);
-		return -1;
+		return -ENODEV;
 	}
 
 	if (vgdev->has_indirect)
@@ -384,7 +384,7 @@ static int virtio_gpu_queue_fenced_ctrl_buffer(struct virtio_gpu_device *vgdev,
 			if (!sgt) {
 				if (fence && vbuf->objs)
 					virtio_gpu_array_unlock_resv(vbuf->objs);
-				return -1;
+				return -ENOMEM;
 			}
 
 			elemcnt += sg_ents;
@@ -750,7 +750,7 @@ static int virtio_get_edid_block(void *data, u8 *buf,
 	size_t start = block * EDID_LENGTH;
 
 	if (start + len > le32_to_cpu(resp->size))
-		return -1;
+		return -EINVAL;
 	memcpy(buf, resp->edid + start, len);
 	return 0;
 }
-- 
2.35.3

[PATCH v6 21/22] drm/virtio: Return proper error codes instead of -1

[Dmitry Osipenko]

Don't return -1 in error cases, return proper error code. The returned
error codes propagate to error messages and to userspace and it's always
good to have a meaningful error number for debugging purposes.

Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/virtio/virtgpu_vq.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_vq.c b/drivers/gpu/drm/virtio/virtgpu_vq.c
index 2a04dad1ae89..40402367d593 100644
--- a/drivers/gpu/drm/virtio/virtgpu_vq.c
+++ b/drivers/gpu/drm/virtio/virtgpu_vq.c
@@ -320,7 +320,7 @@ static int virtio_gpu_queue_ctrl_sgs(struct virtio_gpu_device *vgdev,
 		if (fence && vbuf->objs)
 			virtio_gpu_array_unlock_resv(vbuf->objs);
 		free_vbuf(vgdev, vbuf);
-		return -1;
+		return -ENODEV;
 	}
 
 	if (vgdev->has_indirect)
@@ -384,7 +384,7 @@ static int virtio_gpu_queue_fenced_ctrl_buffer(struct virtio_gpu_device *vgdev,
 			if (!sgt) {
 				if (fence && vbuf->objs)
 					virtio_gpu_array_unlock_resv(vbuf->objs);
-				return -1;
+				return -ENOMEM;
 			}
 
 			elemcnt += sg_ents;
@@ -750,7 +750,7 @@ static int virtio_get_edid_block(void *data, u8 *buf,
 	size_t start = block * EDID_LENGTH;
 
 	if (start + len > le32_to_cpu(resp->size))
-		return -1;
+		return -EINVAL;
 	memcpy(buf, resp->edid + start, len);
 	return 0;
 }
-- 
2.35.3

[PATCH v6 22/22] drm/panfrost: Switch to generic memory shrinker

[Dmitry Osipenko]

Replace Panfrost's memory shrinker with a generic drm-shmem memory
shrinker.

Tested-by: Steven Price <steven.price@arm.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/panfrost/Makefile             |   1 -
 drivers/gpu/drm/panfrost/panfrost_device.h    |   4 -
 drivers/gpu/drm/panfrost/panfrost_drv.c       |  19 +--
 drivers/gpu/drm/panfrost/panfrost_gem.c       |  33 +++--
 drivers/gpu/drm/panfrost/panfrost_gem.h       |   9 --
 .../gpu/drm/panfrost/panfrost_gem_shrinker.c  | 129 ------------------
 drivers/gpu/drm/panfrost/panfrost_job.c       |  18 ++-
 7 files changed, 42 insertions(+), 171 deletions(-)
 delete mode 100644 drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c

diff --git a/drivers/gpu/drm/panfrost/Makefile b/drivers/gpu/drm/panfrost/Makefile
index b71935862417..ecf0864cb515 100644
--- a/drivers/gpu/drm/panfrost/Makefile
+++ b/drivers/gpu/drm/panfrost/Makefile
@@ -5,7 +5,6 @@ panfrost-y := \
 	panfrost_device.o \
 	panfrost_devfreq.o \
 	panfrost_gem.o \
-	panfrost_gem_shrinker.o \
 	panfrost_gpu.o \
 	panfrost_job.o \
 	panfrost_mmu.o \
diff --git a/drivers/gpu/drm/panfrost/panfrost_device.h b/drivers/gpu/drm/panfrost/panfrost_device.h
index 8b25278f34c8..fe04b21fc044 100644
--- a/drivers/gpu/drm/panfrost/panfrost_device.h
+++ b/drivers/gpu/drm/panfrost/panfrost_device.h
@@ -115,10 +115,6 @@ struct panfrost_device {
 		atomic_t pending;
 	} reset;
 
-	struct mutex shrinker_lock;
-	struct list_head shrinker_list;
-	struct shrinker shrinker;
-
 	struct panfrost_devfreq pfdevfreq;
 };
 
diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
index 859e240161d1..b77c99ba2475 100644
--- a/drivers/gpu/drm/panfrost/panfrost_drv.c
+++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
@@ -160,7 +160,6 @@ panfrost_lookup_bos(struct drm_device *dev,
 			break;
 		}
 
-		atomic_inc(&bo->gpu_usecount);
 		job->mappings[i] = mapping;
 	}
 
@@ -392,7 +391,6 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
 {
 	struct panfrost_file_priv *priv = file_priv->driver_priv;
 	struct drm_panfrost_madvise *args = data;
-	struct panfrost_device *pfdev = dev->dev_private;
 	struct drm_gem_object *gem_obj;
 	struct panfrost_gem_object *bo;
 	int ret = 0;
@@ -409,7 +407,6 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
 	if (ret)
 		goto out_put_object;
 
-	mutex_lock(&pfdev->shrinker_lock);
 	mutex_lock(&bo->mappings.lock);
 	if (args->madv == PANFROST_MADV_DONTNEED) {
 		struct panfrost_gem_mapping *first;
@@ -435,17 +432,8 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
 
 	args->retained = drm_gem_shmem_madvise(&bo->base, args->madv);
 
-	if (args->retained) {
-		if (args->madv == PANFROST_MADV_DONTNEED)
-			list_move_tail(&bo->base.madv_list,
-				       &pfdev->shrinker_list);
-		else if (args->madv == PANFROST_MADV_WILLNEED)
-			list_del_init(&bo->base.madv_list);
-	}
-
 out_unlock_mappings:
 	mutex_unlock(&bo->mappings.lock);
-	mutex_unlock(&pfdev->shrinker_lock);
 	dma_resv_unlock(bo->base.base.resv);
 out_put_object:
 	drm_gem_object_put(gem_obj);
@@ -577,9 +565,6 @@ static int panfrost_probe(struct platform_device *pdev)
 	ddev->dev_private = pfdev;
 	pfdev->ddev = ddev;
 
-	mutex_init(&pfdev->shrinker_lock);
-	INIT_LIST_HEAD(&pfdev->shrinker_list);
-
 	err = panfrost_device_init(pfdev);
 	if (err) {
 		if (err != -EPROBE_DEFER)
@@ -601,7 +586,7 @@ static int panfrost_probe(struct platform_device *pdev)
 	if (err < 0)
 		goto err_out1;
 
-	panfrost_gem_shrinker_init(ddev);
+	drm_gem_shmem_shrinker_register(ddev);
 
 	return 0;
 
@@ -619,8 +604,8 @@ static int panfrost_remove(struct platform_device *pdev)
 	struct panfrost_device *pfdev = platform_get_drvdata(pdev);
 	struct drm_device *ddev = pfdev->ddev;
 
+	drm_gem_shmem_shrinker_unregister(ddev);
 	drm_dev_unregister(ddev);
-	panfrost_gem_shrinker_cleanup(ddev);
 
 	pm_runtime_get_sync(pfdev->dev);
 	pm_runtime_disable(pfdev->dev);
diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.c b/drivers/gpu/drm/panfrost/panfrost_gem.c
index 293e799e2fe8..f1436405e3a0 100644
--- a/drivers/gpu/drm/panfrost/panfrost_gem.c
+++ b/drivers/gpu/drm/panfrost/panfrost_gem.c
@@ -19,16 +19,6 @@ static void panfrost_gem_free_object(struct drm_gem_object *obj)
 	struct panfrost_gem_object *bo = to_panfrost_bo(obj);
 	struct panfrost_device *pfdev = obj->dev->dev_private;
 
-	/*
-	 * Make sure the BO is no longer inserted in the shrinker list before
-	 * taking care of the destruction itself. If we don't do that we have a
-	 * race condition between this function and what's done in
-	 * panfrost_gem_shrinker_scan().
-	 */
-	mutex_lock(&pfdev->shrinker_lock);
-	list_del_init(&bo->base.madv_list);
-	mutex_unlock(&pfdev->shrinker_lock);
-
 	/*
 	 * If we still have mappings attached to the BO, there's a problem in
 	 * our refcounting.
@@ -209,6 +199,25 @@ static const struct drm_gem_object_funcs panfrost_gem_funcs = {
 	.vm_ops = &drm_gem_shmem_vm_ops,
 };
 
+static int panfrost_shmem_evict(struct drm_gem_shmem_object *shmem)
+{
+	struct panfrost_gem_object *bo = to_panfrost_bo(&shmem->base);
+
+	if (!drm_gem_shmem_is_purgeable(shmem))
+		return -EOPNOTSUPP;
+
+	if (!mutex_trylock(&bo->mappings.lock))
+		return -EBUSY;
+
+	panfrost_gem_teardown_mappings_locked(bo);
+
+	drm_gem_shmem_purge(shmem);
+
+	mutex_unlock(&bo->mappings.lock);
+
+	return 0;
+}
+
 /**
  * panfrost_gem_create_object - Implementation of driver->gem_create_object.
  * @dev: DRM device
@@ -230,6 +239,7 @@ struct drm_gem_object *panfrost_gem_create_object(struct drm_device *dev, size_t
 	mutex_init(&obj->mappings.lock);
 	obj->base.base.funcs = &panfrost_gem_funcs;
 	obj->base.map_wc = !pfdev->coherent;
+	obj->base.evict = panfrost_shmem_evict;
 
 	return &obj->base.base;
 }
@@ -266,6 +276,9 @@ panfrost_gem_create_with_handle(struct drm_file *file_priv,
 	if (ret)
 		return ERR_PTR(ret);
 
+	if (!bo->is_heap)
+		drm_gem_shmem_set_purgeable(shmem);
+
 	return bo;
 }
 
diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.h b/drivers/gpu/drm/panfrost/panfrost_gem.h
index 8088d5fd8480..09da064f1c07 100644
--- a/drivers/gpu/drm/panfrost/panfrost_gem.h
+++ b/drivers/gpu/drm/panfrost/panfrost_gem.h
@@ -30,12 +30,6 @@ struct panfrost_gem_object {
 		struct mutex lock;
 	} mappings;
 
-	/*
-	 * Count the number of jobs referencing this BO so we don't let the
-	 * shrinker reclaim this object prematurely.
-	 */
-	atomic_t gpu_usecount;
-
 	bool noexec		:1;
 	bool is_heap		:1;
 };
@@ -84,7 +78,4 @@ panfrost_gem_mapping_get(struct panfrost_gem_object *bo,
 void panfrost_gem_mapping_put(struct panfrost_gem_mapping *mapping);
 void panfrost_gem_teardown_mappings_locked(struct panfrost_gem_object *bo);
 
-void panfrost_gem_shrinker_init(struct drm_device *dev);
-void panfrost_gem_shrinker_cleanup(struct drm_device *dev);
-
 #endif /* __PANFROST_GEM_H__ */
diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
deleted file mode 100644
index 7cc32556f908..000000000000
--- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
+++ /dev/null
@@ -1,129 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/* Copyright (C) 2019 Arm Ltd.
- *
- * Based on msm_gem_freedreno.c:
- * Copyright (C) 2016 Red Hat
- * Author: Rob Clark <robdclark@gmail.com>
- */
-
-#include <linux/list.h>
-
-#include <drm/drm_device.h>
-#include <drm/drm_gem_shmem_helper.h>
-
-#include "panfrost_device.h"
-#include "panfrost_gem.h"
-#include "panfrost_mmu.h"
-
-static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
-{
-	return (shmem->madv > 0) &&
-		!shmem->pages_pin_count && shmem->sgt &&
-		!shmem->base.dma_buf && !shmem->base.import_attach;
-}
-
-static unsigned long
-panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
-{
-	struct panfrost_device *pfdev =
-		container_of(shrinker, struct panfrost_device, shrinker);
-	struct drm_gem_shmem_object *shmem;
-	unsigned long count = 0;
-
-	if (!mutex_trylock(&pfdev->shrinker_lock))
-		return 0;
-
-	list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
-		if (panfrost_gem_shmem_is_purgeable(shmem))
-			count += shmem->base.size >> PAGE_SHIFT;
-	}
-
-	mutex_unlock(&pfdev->shrinker_lock);
-
-	return count;
-}
-
-static bool panfrost_gem_purge(struct drm_gem_object *obj)
-{
-	struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
-	struct panfrost_gem_object *bo = to_panfrost_bo(obj);
-	bool ret = false;
-
-	if (atomic_read(&bo->gpu_usecount))
-		return false;
-
-	if (!mutex_trylock(&bo->mappings.lock))
-		return false;
-
-	if (!dma_resv_trylock(shmem->base.resv))
-		goto unlock_mappings;
-
-	panfrost_gem_teardown_mappings_locked(bo);
-	drm_gem_shmem_purge(&bo->base);
-	ret = true;
-
-	dma_resv_unlock(shmem->base.resv);
-
-unlock_mappings:
-	mutex_unlock(&bo->mappings.lock);
-	return ret;
-}
-
-static unsigned long
-panfrost_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc)
-{
-	struct panfrost_device *pfdev =
-		container_of(shrinker, struct panfrost_device, shrinker);
-	struct drm_gem_shmem_object *shmem, *tmp;
-	unsigned long freed = 0;
-
-	if (!mutex_trylock(&pfdev->shrinker_lock))
-		return SHRINK_STOP;
-
-	list_for_each_entry_safe(shmem, tmp, &pfdev->shrinker_list, madv_list) {
-		if (freed >= sc->nr_to_scan)
-			break;
-		if (drm_gem_shmem_is_purgeable(shmem) &&
-		    panfrost_gem_purge(&shmem->base)) {
-			freed += shmem->base.size >> PAGE_SHIFT;
-			list_del_init(&shmem->madv_list);
-		}
-	}
-
-	mutex_unlock(&pfdev->shrinker_lock);
-
-	if (freed > 0)
-		pr_info_ratelimited("Purging %lu bytes\n", freed << PAGE_SHIFT);
-
-	return freed;
-}
-
-/**
- * panfrost_gem_shrinker_init - Initialize panfrost shrinker
- * @dev: DRM device
- *
- * This function registers and sets up the panfrost shrinker.
- */
-void panfrost_gem_shrinker_init(struct drm_device *dev)
-{
-	struct panfrost_device *pfdev = dev->dev_private;
-	pfdev->shrinker.count_objects = panfrost_gem_shrinker_count;
-	pfdev->shrinker.scan_objects = panfrost_gem_shrinker_scan;
-	pfdev->shrinker.seeks = DEFAULT_SEEKS;
-	WARN_ON(register_shrinker(&pfdev->shrinker));
-}
-
-/**
- * panfrost_gem_shrinker_cleanup - Clean up panfrost shrinker
- * @dev: DRM device
- *
- * This function unregisters the panfrost shrinker.
- */
-void panfrost_gem_shrinker_cleanup(struct drm_device *dev)
-{
-	struct panfrost_device *pfdev = dev->dev_private;
-
-	if (pfdev->shrinker.nr_deferred) {
-		unregister_shrinker(&pfdev->shrinker);
-	}
-}
diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
index 7c4208476fbd..5c327a79455f 100644
--- a/drivers/gpu/drm/panfrost/panfrost_job.c
+++ b/drivers/gpu/drm/panfrost/panfrost_job.c
@@ -271,6 +271,19 @@ static void panfrost_attach_object_fences(struct drm_gem_object **bos,
 		dma_resv_add_fence(bos[i]->resv, fence, DMA_RESV_USAGE_WRITE);
 }
 
+static int panfrost_objects_prepare(struct drm_gem_object **bos, int bo_count)
+{
+	struct panfrost_gem_object *bo;
+	int ret = 0;
+
+	while (!ret && bo_count--) {
+		bo = to_panfrost_bo(bos[bo_count]);
+		ret = bo->base.madv ? -ENOMEM : 0;
+	}
+
+	return ret;
+}
+
 int panfrost_job_push(struct panfrost_job *job)
 {
 	struct panfrost_device *pfdev = job->pfdev;
@@ -282,6 +295,10 @@ int panfrost_job_push(struct panfrost_job *job)
 	if (ret)
 		return ret;
 
+	ret = panfrost_objects_prepare(job->bos, job->bo_count);
+	if (ret)
+		goto unlock;
+
 	mutex_lock(&pfdev->sched_lock);
 	drm_sched_job_arm(&job->base);
 
@@ -323,7 +340,6 @@ static void panfrost_job_cleanup(struct kref *ref)
 			if (!job->mappings[i])
 				break;
 
-			atomic_dec(&job->mappings[i]->obj->gpu_usecount);
 			panfrost_gem_mapping_put(job->mappings[i]);
 		}
 		kvfree(job->mappings);
-- 
2.35.3

[PATCH v6 22/22] drm/panfrost: Switch to generic memory shrinker

[Dmitry Osipenko]

Replace Panfrost's memory shrinker with a generic drm-shmem memory
shrinker.

Tested-by: Steven Price <steven.price@arm.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 drivers/gpu/drm/panfrost/Makefile             |   1 -
 drivers/gpu/drm/panfrost/panfrost_device.h    |   4 -
 drivers/gpu/drm/panfrost/panfrost_drv.c       |  19 +--
 drivers/gpu/drm/panfrost/panfrost_gem.c       |  33 +++--
 drivers/gpu/drm/panfrost/panfrost_gem.h       |   9 --
 .../gpu/drm/panfrost/panfrost_gem_shrinker.c  | 129 ------------------
 drivers/gpu/drm/panfrost/panfrost_job.c       |  18 ++-
 7 files changed, 42 insertions(+), 171 deletions(-)
 delete mode 100644 drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c

diff --git a/drivers/gpu/drm/panfrost/Makefile b/drivers/gpu/drm/panfrost/Makefile
index b71935862417..ecf0864cb515 100644
--- a/drivers/gpu/drm/panfrost/Makefile
+++ b/drivers/gpu/drm/panfrost/Makefile
@@ -5,7 +5,6 @@ panfrost-y := \
 	panfrost_device.o \
 	panfrost_devfreq.o \
 	panfrost_gem.o \
-	panfrost_gem_shrinker.o \
 	panfrost_gpu.o \
 	panfrost_job.o \
 	panfrost_mmu.o \
diff --git a/drivers/gpu/drm/panfrost/panfrost_device.h b/drivers/gpu/drm/panfrost/panfrost_device.h
index 8b25278f34c8..fe04b21fc044 100644
--- a/drivers/gpu/drm/panfrost/panfrost_device.h
+++ b/drivers/gpu/drm/panfrost/panfrost_device.h
@@ -115,10 +115,6 @@ struct panfrost_device {
 		atomic_t pending;
 	} reset;
 
-	struct mutex shrinker_lock;
-	struct list_head shrinker_list;
-	struct shrinker shrinker;
-
 	struct panfrost_devfreq pfdevfreq;
 };
 
diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
index 859e240161d1..b77c99ba2475 100644
--- a/drivers/gpu/drm/panfrost/panfrost_drv.c
+++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
@@ -160,7 +160,6 @@ panfrost_lookup_bos(struct drm_device *dev,
 			break;
 		}
 
-		atomic_inc(&bo->gpu_usecount);
 		job->mappings[i] = mapping;
 	}
 
@@ -392,7 +391,6 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
 {
 	struct panfrost_file_priv *priv = file_priv->driver_priv;
 	struct drm_panfrost_madvise *args = data;
-	struct panfrost_device *pfdev = dev->dev_private;
 	struct drm_gem_object *gem_obj;
 	struct panfrost_gem_object *bo;
 	int ret = 0;
@@ -409,7 +407,6 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
 	if (ret)
 		goto out_put_object;
 
-	mutex_lock(&pfdev->shrinker_lock);
 	mutex_lock(&bo->mappings.lock);
 	if (args->madv == PANFROST_MADV_DONTNEED) {
 		struct panfrost_gem_mapping *first;
@@ -435,17 +432,8 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
 
 	args->retained = drm_gem_shmem_madvise(&bo->base, args->madv);
 
-	if (args->retained) {
-		if (args->madv == PANFROST_MADV_DONTNEED)
-			list_move_tail(&bo->base.madv_list,
-				       &pfdev->shrinker_list);
-		else if (args->madv == PANFROST_MADV_WILLNEED)
-			list_del_init(&bo->base.madv_list);
-	}
-
 out_unlock_mappings:
 	mutex_unlock(&bo->mappings.lock);
-	mutex_unlock(&pfdev->shrinker_lock);
 	dma_resv_unlock(bo->base.base.resv);
 out_put_object:
 	drm_gem_object_put(gem_obj);
@@ -577,9 +565,6 @@ static int panfrost_probe(struct platform_device *pdev)
 	ddev->dev_private = pfdev;
 	pfdev->ddev = ddev;
 
-	mutex_init(&pfdev->shrinker_lock);
-	INIT_LIST_HEAD(&pfdev->shrinker_list);
-
 	err = panfrost_device_init(pfdev);
 	if (err) {
 		if (err != -EPROBE_DEFER)
@@ -601,7 +586,7 @@ static int panfrost_probe(struct platform_device *pdev)
 	if (err < 0)
 		goto err_out1;
 
-	panfrost_gem_shrinker_init(ddev);
+	drm_gem_shmem_shrinker_register(ddev);
 
 	return 0;
 
@@ -619,8 +604,8 @@ static int panfrost_remove(struct platform_device *pdev)
 	struct panfrost_device *pfdev = platform_get_drvdata(pdev);
 	struct drm_device *ddev = pfdev->ddev;
 
+	drm_gem_shmem_shrinker_unregister(ddev);
 	drm_dev_unregister(ddev);
-	panfrost_gem_shrinker_cleanup(ddev);
 
 	pm_runtime_get_sync(pfdev->dev);
 	pm_runtime_disable(pfdev->dev);
diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.c b/drivers/gpu/drm/panfrost/panfrost_gem.c
index 293e799e2fe8..f1436405e3a0 100644
--- a/drivers/gpu/drm/panfrost/panfrost_gem.c
+++ b/drivers/gpu/drm/panfrost/panfrost_gem.c
@@ -19,16 +19,6 @@ static void panfrost_gem_free_object(struct drm_gem_object *obj)
 	struct panfrost_gem_object *bo = to_panfrost_bo(obj);
 	struct panfrost_device *pfdev = obj->dev->dev_private;
 
-	/*
-	 * Make sure the BO is no longer inserted in the shrinker list before
-	 * taking care of the destruction itself. If we don't do that we have a
-	 * race condition between this function and what's done in
-	 * panfrost_gem_shrinker_scan().
-	 */
-	mutex_lock(&pfdev->shrinker_lock);
-	list_del_init(&bo->base.madv_list);
-	mutex_unlock(&pfdev->shrinker_lock);
-
 	/*
 	 * If we still have mappings attached to the BO, there's a problem in
 	 * our refcounting.
@@ -209,6 +199,25 @@ static const struct drm_gem_object_funcs panfrost_gem_funcs = {
 	.vm_ops = &drm_gem_shmem_vm_ops,
 };
 
+static int panfrost_shmem_evict(struct drm_gem_shmem_object *shmem)
+{
+	struct panfrost_gem_object *bo = to_panfrost_bo(&shmem->base);
+
+	if (!drm_gem_shmem_is_purgeable(shmem))
+		return -EOPNOTSUPP;
+
+	if (!mutex_trylock(&bo->mappings.lock))
+		return -EBUSY;
+
+	panfrost_gem_teardown_mappings_locked(bo);
+
+	drm_gem_shmem_purge(shmem);
+
+	mutex_unlock(&bo->mappings.lock);
+
+	return 0;
+}
+
 /**
  * panfrost_gem_create_object - Implementation of driver->gem_create_object.
  * @dev: DRM device
@@ -230,6 +239,7 @@ struct drm_gem_object *panfrost_gem_create_object(struct drm_device *dev, size_t
 	mutex_init(&obj->mappings.lock);
 	obj->base.base.funcs = &panfrost_gem_funcs;
 	obj->base.map_wc = !pfdev->coherent;
+	obj->base.evict = panfrost_shmem_evict;
 
 	return &obj->base.base;
 }
@@ -266,6 +276,9 @@ panfrost_gem_create_with_handle(struct drm_file *file_priv,
 	if (ret)
 		return ERR_PTR(ret);
 
+	if (!bo->is_heap)
+		drm_gem_shmem_set_purgeable(shmem);
+
 	return bo;
 }
 
diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.h b/drivers/gpu/drm/panfrost/panfrost_gem.h
index 8088d5fd8480..09da064f1c07 100644
--- a/drivers/gpu/drm/panfrost/panfrost_gem.h
+++ b/drivers/gpu/drm/panfrost/panfrost_gem.h
@@ -30,12 +30,6 @@ struct panfrost_gem_object {
 		struct mutex lock;
 	} mappings;
 
-	/*
-	 * Count the number of jobs referencing this BO so we don't let the
-	 * shrinker reclaim this object prematurely.
-	 */
-	atomic_t gpu_usecount;
-
 	bool noexec		:1;
 	bool is_heap		:1;
 };
@@ -84,7 +78,4 @@ panfrost_gem_mapping_get(struct panfrost_gem_object *bo,
 void panfrost_gem_mapping_put(struct panfrost_gem_mapping *mapping);
 void panfrost_gem_teardown_mappings_locked(struct panfrost_gem_object *bo);
 
-void panfrost_gem_shrinker_init(struct drm_device *dev);
-void panfrost_gem_shrinker_cleanup(struct drm_device *dev);
-
 #endif /* __PANFROST_GEM_H__ */
diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
deleted file mode 100644
index 7cc32556f908..000000000000
--- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
+++ /dev/null
@@ -1,129 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-/* Copyright (C) 2019 Arm Ltd.
- *
- * Based on msm_gem_freedreno.c:
- * Copyright (C) 2016 Red Hat
- * Author: Rob Clark <robdclark@gmail.com>
- */
-
-#include <linux/list.h>
-
-#include <drm/drm_device.h>
-#include <drm/drm_gem_shmem_helper.h>
-
-#include "panfrost_device.h"
-#include "panfrost_gem.h"
-#include "panfrost_mmu.h"
-
-static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
-{
-	return (shmem->madv > 0) &&
-		!shmem->pages_pin_count && shmem->sgt &&
-		!shmem->base.dma_buf && !shmem->base.import_attach;
-}
-
-static unsigned long
-panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
-{
-	struct panfrost_device *pfdev =
-		container_of(shrinker, struct panfrost_device, shrinker);
-	struct drm_gem_shmem_object *shmem;
-	unsigned long count = 0;
-
-	if (!mutex_trylock(&pfdev->shrinker_lock))
-		return 0;
-
-	list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
-		if (panfrost_gem_shmem_is_purgeable(shmem))
-			count += shmem->base.size >> PAGE_SHIFT;
-	}
-
-	mutex_unlock(&pfdev->shrinker_lock);
-
-	return count;
-}
-
-static bool panfrost_gem_purge(struct drm_gem_object *obj)
-{
-	struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
-	struct panfrost_gem_object *bo = to_panfrost_bo(obj);
-	bool ret = false;
-
-	if (atomic_read(&bo->gpu_usecount))
-		return false;
-
-	if (!mutex_trylock(&bo->mappings.lock))
-		return false;
-
-	if (!dma_resv_trylock(shmem->base.resv))
-		goto unlock_mappings;
-
-	panfrost_gem_teardown_mappings_locked(bo);
-	drm_gem_shmem_purge(&bo->base);
-	ret = true;
-
-	dma_resv_unlock(shmem->base.resv);
-
-unlock_mappings:
-	mutex_unlock(&bo->mappings.lock);
-	return ret;
-}
-
-static unsigned long
-panfrost_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc)
-{
-	struct panfrost_device *pfdev =
-		container_of(shrinker, struct panfrost_device, shrinker);
-	struct drm_gem_shmem_object *shmem, *tmp;
-	unsigned long freed = 0;
-
-	if (!mutex_trylock(&pfdev->shrinker_lock))
-		return SHRINK_STOP;
-
-	list_for_each_entry_safe(shmem, tmp, &pfdev->shrinker_list, madv_list) {
-		if (freed >= sc->nr_to_scan)
-			break;
-		if (drm_gem_shmem_is_purgeable(shmem) &&
-		    panfrost_gem_purge(&shmem->base)) {
-			freed += shmem->base.size >> PAGE_SHIFT;
-			list_del_init(&shmem->madv_list);
-		}
-	}
-
-	mutex_unlock(&pfdev->shrinker_lock);
-
-	if (freed > 0)
-		pr_info_ratelimited("Purging %lu bytes\n", freed << PAGE_SHIFT);
-
-	return freed;
-}
-
-/**
- * panfrost_gem_shrinker_init - Initialize panfrost shrinker
- * @dev: DRM device
- *
- * This function registers and sets up the panfrost shrinker.
- */
-void panfrost_gem_shrinker_init(struct drm_device *dev)
-{
-	struct panfrost_device *pfdev = dev->dev_private;
-	pfdev->shrinker.count_objects = panfrost_gem_shrinker_count;
-	pfdev->shrinker.scan_objects = panfrost_gem_shrinker_scan;
-	pfdev->shrinker.seeks = DEFAULT_SEEKS;
-	WARN_ON(register_shrinker(&pfdev->shrinker));
-}
-
-/**
- * panfrost_gem_shrinker_cleanup - Clean up panfrost shrinker
- * @dev: DRM device
- *
- * This function unregisters the panfrost shrinker.
- */
-void panfrost_gem_shrinker_cleanup(struct drm_device *dev)
-{
-	struct panfrost_device *pfdev = dev->dev_private;
-
-	if (pfdev->shrinker.nr_deferred) {
-		unregister_shrinker(&pfdev->shrinker);
-	}
-}
diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
index 7c4208476fbd..5c327a79455f 100644
--- a/drivers/gpu/drm/panfrost/panfrost_job.c
+++ b/drivers/gpu/drm/panfrost/panfrost_job.c
@@ -271,6 +271,19 @@ static void panfrost_attach_object_fences(struct drm_gem_object **bos,
 		dma_resv_add_fence(bos[i]->resv, fence, DMA_RESV_USAGE_WRITE);
 }
 
+static int panfrost_objects_prepare(struct drm_gem_object **bos, int bo_count)
+{
+	struct panfrost_gem_object *bo;
+	int ret = 0;
+
+	while (!ret && bo_count--) {
+		bo = to_panfrost_bo(bos[bo_count]);
+		ret = bo->base.madv ? -ENOMEM : 0;
+	}
+
+	return ret;
+}
+
 int panfrost_job_push(struct panfrost_job *job)
 {
 	struct panfrost_device *pfdev = job->pfdev;
@@ -282,6 +295,10 @@ int panfrost_job_push(struct panfrost_job *job)
 	if (ret)
 		return ret;
 
+	ret = panfrost_objects_prepare(job->bos, job->bo_count);
+	if (ret)
+		goto unlock;
+
 	mutex_lock(&pfdev->sched_lock);
 	drm_sched_job_arm(&job->base);
 
@@ -323,7 +340,6 @@ static void panfrost_job_cleanup(struct kref *ref)
 			if (!job->mappings[i])
 				break;
 
-			atomic_dec(&job->mappings[i]->obj->gpu_usecount);
 			panfrost_gem_mapping_put(job->mappings[i]);
 		}
 		kvfree(job->mappings);
-- 
2.35.3

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[kernel test robot]

Hi Dmitry,

I love your patch! Perhaps something to improve:

[auto build test WARNING on linus/master]
[cannot apply to drm/drm-next media-tree/master drm-intel/for-linux-next v5.18]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/intel-lab-lkp/linux/commits/Dmitry-Osipenko/Add-generic-memory-shrinker-to-VirtIO-GPU-and-Panfrost-DRM-drivers/20220527-075717
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git cdeffe87f790dfd1baa193020411ce9a538446d7
config: hexagon-randconfig-r045-20220524 (https://download.01.org/0day-ci/archive/20220527/202205271042.1WRbRP1r-lkp@intel.com/config)
compiler: clang version 15.0.0 (https://github.com/llvm/llvm-project 6f4644d194da594562027a5d458d9fb7a20ebc39)
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # https://github.com/intel-lab-lkp/linux/commit/97f090c47ec995a8cf3bced98526ee3eaa25f10f
        git remote add linux-review https://github.com/intel-lab-lkp/linux
        git fetch --no-tags linux-review Dmitry-Osipenko/Add-generic-memory-shrinker-to-VirtIO-GPU-and-Panfrost-DRM-drivers/20220527-075717
        git checkout 97f090c47ec995a8cf3bced98526ee3eaa25f10f
        # save the config file
        mkdir build_dir && cp config build_dir/.config
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=hexagon SHELL=/bin/bash drivers/dma-buf/

If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <lkp@intel.com>

All warnings (new ones prefixed by >>):

>> drivers/dma-buf/dma-buf.c:1339:10: warning: variable 'ret' is uninitialized when used here [-Wuninitialized]
                   return ret;
                          ^~~
   drivers/dma-buf/dma-buf.c:1331:9: note: initialize the variable 'ret' to silence this warning
           int ret;
                  ^
                   = 0
   1 warning generated.


vim +/ret +1339 drivers/dma-buf/dma-buf.c

  1327	
  1328	static int dma_buf_vmap_locked(struct dma_buf *dmabuf, struct iosys_map *map)
  1329	{
  1330		struct iosys_map ptr;
  1331		int ret;
  1332	
  1333		dma_resv_assert_held(dmabuf->resv);
  1334	
  1335		if (dmabuf->vmapping_counter) {
  1336			dmabuf->vmapping_counter++;
  1337			BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
  1338			*map = dmabuf->vmap_ptr;
> 1339			return ret;
  1340		}
  1341	
  1342		BUG_ON(iosys_map_is_set(&dmabuf->vmap_ptr));
  1343	
  1344		ret = dmabuf->ops->vmap(dmabuf, &ptr);
  1345		if (WARN_ON_ONCE(ret))
  1346			return ret;
  1347	
  1348		dmabuf->vmap_ptr = ptr;
  1349		dmabuf->vmapping_counter = 1;
  1350	
  1351		*map = dmabuf->vmap_ptr;
  1352	
  1353		return 0;
  1354	}
  1355	

-- 
0-DAY CI Kernel Test Service
https://01.org/lkp

Re: [PATCH v6 22/22] drm/panfrost: Switch to generic memory shrinker

[Alyssa Rosenzweig]

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

On Fri, May 27, 2022 at 02:50:40AM +0300, Dmitry Osipenko wrote:
> Replace Panfrost's memory shrinker with a generic drm-shmem memory
> shrinker.
> 
> Tested-by: Steven Price <steven.price@arm.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/panfrost/Makefile             |   1 -
>  drivers/gpu/drm/panfrost/panfrost_device.h    |   4 -
>  drivers/gpu/drm/panfrost/panfrost_drv.c       |  19 +--
>  drivers/gpu/drm/panfrost/panfrost_gem.c       |  33 +++--
>  drivers/gpu/drm/panfrost/panfrost_gem.h       |   9 --
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  | 129 ------------------
>  drivers/gpu/drm/panfrost/panfrost_job.c       |  18 ++-
>  7 files changed, 42 insertions(+), 171 deletions(-)
>  delete mode 100644 drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> 
> diff --git a/drivers/gpu/drm/panfrost/Makefile b/drivers/gpu/drm/panfrost/Makefile
> index b71935862417..ecf0864cb515 100644
> --- a/drivers/gpu/drm/panfrost/Makefile
> +++ b/drivers/gpu/drm/panfrost/Makefile
> @@ -5,7 +5,6 @@ panfrost-y := \
>  	panfrost_device.o \
>  	panfrost_devfreq.o \
>  	panfrost_gem.o \
> -	panfrost_gem_shrinker.o \
>  	panfrost_gpu.o \
>  	panfrost_job.o \
>  	panfrost_mmu.o \
> diff --git a/drivers/gpu/drm/panfrost/panfrost_device.h b/drivers/gpu/drm/panfrost/panfrost_device.h
> index 8b25278f34c8..fe04b21fc044 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_device.h
> +++ b/drivers/gpu/drm/panfrost/panfrost_device.h
> @@ -115,10 +115,6 @@ struct panfrost_device {
>  		atomic_t pending;
>  	} reset;
>  
> -	struct mutex shrinker_lock;
> -	struct list_head shrinker_list;
> -	struct shrinker shrinker;
> -
>  	struct panfrost_devfreq pfdevfreq;
>  };
>  
> diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
> index 859e240161d1..b77c99ba2475 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_drv.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
> @@ -160,7 +160,6 @@ panfrost_lookup_bos(struct drm_device *dev,
>  			break;
>  		}
>  
> -		atomic_inc(&bo->gpu_usecount);
>  		job->mappings[i] = mapping;
>  	}
>  
> @@ -392,7 +391,6 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
>  {
>  	struct panfrost_file_priv *priv = file_priv->driver_priv;
>  	struct drm_panfrost_madvise *args = data;
> -	struct panfrost_device *pfdev = dev->dev_private;
>  	struct drm_gem_object *gem_obj;
>  	struct panfrost_gem_object *bo;
>  	int ret = 0;
> @@ -409,7 +407,6 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
>  	if (ret)
>  		goto out_put_object;
>  
> -	mutex_lock(&pfdev->shrinker_lock);
>  	mutex_lock(&bo->mappings.lock);
>  	if (args->madv == PANFROST_MADV_DONTNEED) {
>  		struct panfrost_gem_mapping *first;
> @@ -435,17 +432,8 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
>  
>  	args->retained = drm_gem_shmem_madvise(&bo->base, args->madv);
>  
> -	if (args->retained) {
> -		if (args->madv == PANFROST_MADV_DONTNEED)
> -			list_move_tail(&bo->base.madv_list,
> -				       &pfdev->shrinker_list);
> -		else if (args->madv == PANFROST_MADV_WILLNEED)
> -			list_del_init(&bo->base.madv_list);
> -	}
> -
>  out_unlock_mappings:
>  	mutex_unlock(&bo->mappings.lock);
> -	mutex_unlock(&pfdev->shrinker_lock);
>  	dma_resv_unlock(bo->base.base.resv);
>  out_put_object:
>  	drm_gem_object_put(gem_obj);
> @@ -577,9 +565,6 @@ static int panfrost_probe(struct platform_device *pdev)
>  	ddev->dev_private = pfdev;
>  	pfdev->ddev = ddev;
>  
> -	mutex_init(&pfdev->shrinker_lock);
> -	INIT_LIST_HEAD(&pfdev->shrinker_list);
> -
>  	err = panfrost_device_init(pfdev);
>  	if (err) {
>  		if (err != -EPROBE_DEFER)
> @@ -601,7 +586,7 @@ static int panfrost_probe(struct platform_device *pdev)
>  	if (err < 0)
>  		goto err_out1;
>  
> -	panfrost_gem_shrinker_init(ddev);
> +	drm_gem_shmem_shrinker_register(ddev);
>  
>  	return 0;
>  
> @@ -619,8 +604,8 @@ static int panfrost_remove(struct platform_device *pdev)
>  	struct panfrost_device *pfdev = platform_get_drvdata(pdev);
>  	struct drm_device *ddev = pfdev->ddev;
>  
> +	drm_gem_shmem_shrinker_unregister(ddev);
>  	drm_dev_unregister(ddev);
> -	panfrost_gem_shrinker_cleanup(ddev);
>  
>  	pm_runtime_get_sync(pfdev->dev);
>  	pm_runtime_disable(pfdev->dev);
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.c b/drivers/gpu/drm/panfrost/panfrost_gem.c
> index 293e799e2fe8..f1436405e3a0 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem.c
> @@ -19,16 +19,6 @@ static void panfrost_gem_free_object(struct drm_gem_object *obj)
>  	struct panfrost_gem_object *bo = to_panfrost_bo(obj);
>  	struct panfrost_device *pfdev = obj->dev->dev_private;
>  
> -	/*
> -	 * Make sure the BO is no longer inserted in the shrinker list before
> -	 * taking care of the destruction itself. If we don't do that we have a
> -	 * race condition between this function and what's done in
> -	 * panfrost_gem_shrinker_scan().
> -	 */
> -	mutex_lock(&pfdev->shrinker_lock);
> -	list_del_init(&bo->base.madv_list);
> -	mutex_unlock(&pfdev->shrinker_lock);
> -
>  	/*
>  	 * If we still have mappings attached to the BO, there's a problem in
>  	 * our refcounting.
> @@ -209,6 +199,25 @@ static const struct drm_gem_object_funcs panfrost_gem_funcs = {
>  	.vm_ops = &drm_gem_shmem_vm_ops,
>  };
>  
> +static int panfrost_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +	struct panfrost_gem_object *bo = to_panfrost_bo(&shmem->base);
> +
> +	if (!drm_gem_shmem_is_purgeable(shmem))
> +		return -EOPNOTSUPP;
> +
> +	if (!mutex_trylock(&bo->mappings.lock))
> +		return -EBUSY;
> +
> +	panfrost_gem_teardown_mappings_locked(bo);
> +
> +	drm_gem_shmem_purge(shmem);
> +
> +	mutex_unlock(&bo->mappings.lock);
> +
> +	return 0;
> +}
> +
>  /**
>   * panfrost_gem_create_object - Implementation of driver->gem_create_object.
>   * @dev: DRM device
> @@ -230,6 +239,7 @@ struct drm_gem_object *panfrost_gem_create_object(struct drm_device *dev, size_t
>  	mutex_init(&obj->mappings.lock);
>  	obj->base.base.funcs = &panfrost_gem_funcs;
>  	obj->base.map_wc = !pfdev->coherent;
> +	obj->base.evict = panfrost_shmem_evict;
>  
>  	return &obj->base.base;
>  }
> @@ -266,6 +276,9 @@ panfrost_gem_create_with_handle(struct drm_file *file_priv,
>  	if (ret)
>  		return ERR_PTR(ret);
>  
> +	if (!bo->is_heap)
> +		drm_gem_shmem_set_purgeable(shmem);
> +
>  	return bo;
>  }
>  
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.h b/drivers/gpu/drm/panfrost/panfrost_gem.h
> index 8088d5fd8480..09da064f1c07 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem.h
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem.h
> @@ -30,12 +30,6 @@ struct panfrost_gem_object {
>  		struct mutex lock;
>  	} mappings;
>  
> -	/*
> -	 * Count the number of jobs referencing this BO so we don't let the
> -	 * shrinker reclaim this object prematurely.
> -	 */
> -	atomic_t gpu_usecount;
> -
>  	bool noexec		:1;
>  	bool is_heap		:1;
>  };
> @@ -84,7 +78,4 @@ panfrost_gem_mapping_get(struct panfrost_gem_object *bo,
>  void panfrost_gem_mapping_put(struct panfrost_gem_mapping *mapping);
>  void panfrost_gem_teardown_mappings_locked(struct panfrost_gem_object *bo);
>  
> -void panfrost_gem_shrinker_init(struct drm_device *dev);
> -void panfrost_gem_shrinker_cleanup(struct drm_device *dev);
> -
>  #endif /* __PANFROST_GEM_H__ */
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> deleted file mode 100644
> index 7cc32556f908..000000000000
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ /dev/null
> @@ -1,129 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/* Copyright (C) 2019 Arm Ltd.
> - *
> - * Based on msm_gem_freedreno.c:
> - * Copyright (C) 2016 Red Hat
> - * Author: Rob Clark <robdclark@gmail.com>
> - */
> -
> -#include <linux/list.h>
> -
> -#include <drm/drm_device.h>
> -#include <drm/drm_gem_shmem_helper.h>
> -
> -#include "panfrost_device.h"
> -#include "panfrost_gem.h"
> -#include "panfrost_mmu.h"
> -
> -static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> -{
> -	return (shmem->madv > 0) &&
> -		!shmem->pages_pin_count && shmem->sgt &&
> -		!shmem->base.dma_buf && !shmem->base.import_attach;
> -}
> -
> -static unsigned long
> -panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
> -{
> -	struct panfrost_device *pfdev =
> -		container_of(shrinker, struct panfrost_device, shrinker);
> -	struct drm_gem_shmem_object *shmem;
> -	unsigned long count = 0;
> -
> -	if (!mutex_trylock(&pfdev->shrinker_lock))
> -		return 0;
> -
> -	list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -		if (panfrost_gem_shmem_is_purgeable(shmem))
> -			count += shmem->base.size >> PAGE_SHIFT;
> -	}
> -
> -	mutex_unlock(&pfdev->shrinker_lock);
> -
> -	return count;
> -}
> -
> -static bool panfrost_gem_purge(struct drm_gem_object *obj)
> -{
> -	struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -	struct panfrost_gem_object *bo = to_panfrost_bo(obj);
> -	bool ret = false;
> -
> -	if (atomic_read(&bo->gpu_usecount))
> -		return false;
> -
> -	if (!mutex_trylock(&bo->mappings.lock))
> -		return false;
> -
> -	if (!dma_resv_trylock(shmem->base.resv))
> -		goto unlock_mappings;
> -
> -	panfrost_gem_teardown_mappings_locked(bo);
> -	drm_gem_shmem_purge(&bo->base);
> -	ret = true;
> -
> -	dma_resv_unlock(shmem->base.resv);
> -
> -unlock_mappings:
> -	mutex_unlock(&bo->mappings.lock);
> -	return ret;
> -}
> -
> -static unsigned long
> -panfrost_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc)
> -{
> -	struct panfrost_device *pfdev =
> -		container_of(shrinker, struct panfrost_device, shrinker);
> -	struct drm_gem_shmem_object *shmem, *tmp;
> -	unsigned long freed = 0;
> -
> -	if (!mutex_trylock(&pfdev->shrinker_lock))
> -		return SHRINK_STOP;
> -
> -	list_for_each_entry_safe(shmem, tmp, &pfdev->shrinker_list, madv_list) {
> -		if (freed >= sc->nr_to_scan)
> -			break;
> -		if (drm_gem_shmem_is_purgeable(shmem) &&
> -		    panfrost_gem_purge(&shmem->base)) {
> -			freed += shmem->base.size >> PAGE_SHIFT;
> -			list_del_init(&shmem->madv_list);
> -		}
> -	}
> -
> -	mutex_unlock(&pfdev->shrinker_lock);
> -
> -	if (freed > 0)
> -		pr_info_ratelimited("Purging %lu bytes\n", freed << PAGE_SHIFT);
> -
> -	return freed;
> -}
> -
> -/**
> - * panfrost_gem_shrinker_init - Initialize panfrost shrinker
> - * @dev: DRM device
> - *
> - * This function registers and sets up the panfrost shrinker.
> - */
> -void panfrost_gem_shrinker_init(struct drm_device *dev)
> -{
> -	struct panfrost_device *pfdev = dev->dev_private;
> -	pfdev->shrinker.count_objects = panfrost_gem_shrinker_count;
> -	pfdev->shrinker.scan_objects = panfrost_gem_shrinker_scan;
> -	pfdev->shrinker.seeks = DEFAULT_SEEKS;
> -	WARN_ON(register_shrinker(&pfdev->shrinker));
> -}
> -
> -/**
> - * panfrost_gem_shrinker_cleanup - Clean up panfrost shrinker
> - * @dev: DRM device
> - *
> - * This function unregisters the panfrost shrinker.
> - */
> -void panfrost_gem_shrinker_cleanup(struct drm_device *dev)
> -{
> -	struct panfrost_device *pfdev = dev->dev_private;
> -
> -	if (pfdev->shrinker.nr_deferred) {
> -		unregister_shrinker(&pfdev->shrinker);
> -	}
> -}
> diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
> index 7c4208476fbd..5c327a79455f 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_job.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_job.c
> @@ -271,6 +271,19 @@ static void panfrost_attach_object_fences(struct drm_gem_object **bos,
>  		dma_resv_add_fence(bos[i]->resv, fence, DMA_RESV_USAGE_WRITE);
>  }
>  
> +static int panfrost_objects_prepare(struct drm_gem_object **bos, int bo_count)
> +{
> +	struct panfrost_gem_object *bo;
> +	int ret = 0;
> +
> +	while (!ret && bo_count--) {
> +		bo = to_panfrost_bo(bos[bo_count]);
> +		ret = bo->base.madv ? -ENOMEM : 0;
> +	}
> +
> +	return ret;
> +}
> +
>  int panfrost_job_push(struct panfrost_job *job)
>  {
>  	struct panfrost_device *pfdev = job->pfdev;
> @@ -282,6 +295,10 @@ int panfrost_job_push(struct panfrost_job *job)
>  	if (ret)
>  		return ret;
>  
> +	ret = panfrost_objects_prepare(job->bos, job->bo_count);
> +	if (ret)
> +		goto unlock;
> +
>  	mutex_lock(&pfdev->sched_lock);
>  	drm_sched_job_arm(&job->base);
>  
> @@ -323,7 +340,6 @@ static void panfrost_job_cleanup(struct kref *ref)
>  			if (!job->mappings[i])
>  				break;
>  
> -			atomic_dec(&job->mappings[i]->obj->gpu_usecount);
>  			panfrost_gem_mapping_put(job->mappings[i]);
>  		}
>  		kvfree(job->mappings);
> -- 
> 2.35.3
> 

Re: [PATCH v6 22/22] drm/panfrost: Switch to generic memory shrinker

[Alyssa Rosenzweig]

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

On Fri, May 27, 2022 at 02:50:40AM +0300, Dmitry Osipenko wrote:
> Replace Panfrost's memory shrinker with a generic drm-shmem memory
> shrinker.
> 
> Tested-by: Steven Price <steven.price@arm.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/panfrost/Makefile             |   1 -
>  drivers/gpu/drm/panfrost/panfrost_device.h    |   4 -
>  drivers/gpu/drm/panfrost/panfrost_drv.c       |  19 +--
>  drivers/gpu/drm/panfrost/panfrost_gem.c       |  33 +++--
>  drivers/gpu/drm/panfrost/panfrost_gem.h       |   9 --
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  | 129 ------------------
>  drivers/gpu/drm/panfrost/panfrost_job.c       |  18 ++-
>  7 files changed, 42 insertions(+), 171 deletions(-)
>  delete mode 100644 drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> 
> diff --git a/drivers/gpu/drm/panfrost/Makefile b/drivers/gpu/drm/panfrost/Makefile
> index b71935862417..ecf0864cb515 100644
> --- a/drivers/gpu/drm/panfrost/Makefile
> +++ b/drivers/gpu/drm/panfrost/Makefile
> @@ -5,7 +5,6 @@ panfrost-y := \
>  	panfrost_device.o \
>  	panfrost_devfreq.o \
>  	panfrost_gem.o \
> -	panfrost_gem_shrinker.o \
>  	panfrost_gpu.o \
>  	panfrost_job.o \
>  	panfrost_mmu.o \
> diff --git a/drivers/gpu/drm/panfrost/panfrost_device.h b/drivers/gpu/drm/panfrost/panfrost_device.h
> index 8b25278f34c8..fe04b21fc044 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_device.h
> +++ b/drivers/gpu/drm/panfrost/panfrost_device.h
> @@ -115,10 +115,6 @@ struct panfrost_device {
>  		atomic_t pending;
>  	} reset;
>  
> -	struct mutex shrinker_lock;
> -	struct list_head shrinker_list;
> -	struct shrinker shrinker;
> -
>  	struct panfrost_devfreq pfdevfreq;
>  };
>  
> diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
> index 859e240161d1..b77c99ba2475 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_drv.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
> @@ -160,7 +160,6 @@ panfrost_lookup_bos(struct drm_device *dev,
>  			break;
>  		}
>  
> -		atomic_inc(&bo->gpu_usecount);
>  		job->mappings[i] = mapping;
>  	}
>  
> @@ -392,7 +391,6 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
>  {
>  	struct panfrost_file_priv *priv = file_priv->driver_priv;
>  	struct drm_panfrost_madvise *args = data;
> -	struct panfrost_device *pfdev = dev->dev_private;
>  	struct drm_gem_object *gem_obj;
>  	struct panfrost_gem_object *bo;
>  	int ret = 0;
> @@ -409,7 +407,6 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
>  	if (ret)
>  		goto out_put_object;
>  
> -	mutex_lock(&pfdev->shrinker_lock);
>  	mutex_lock(&bo->mappings.lock);
>  	if (args->madv == PANFROST_MADV_DONTNEED) {
>  		struct panfrost_gem_mapping *first;
> @@ -435,17 +432,8 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
>  
>  	args->retained = drm_gem_shmem_madvise(&bo->base, args->madv);
>  
> -	if (args->retained) {
> -		if (args->madv == PANFROST_MADV_DONTNEED)
> -			list_move_tail(&bo->base.madv_list,
> -				       &pfdev->shrinker_list);
> -		else if (args->madv == PANFROST_MADV_WILLNEED)
> -			list_del_init(&bo->base.madv_list);
> -	}
> -
>  out_unlock_mappings:
>  	mutex_unlock(&bo->mappings.lock);
> -	mutex_unlock(&pfdev->shrinker_lock);
>  	dma_resv_unlock(bo->base.base.resv);
>  out_put_object:
>  	drm_gem_object_put(gem_obj);
> @@ -577,9 +565,6 @@ static int panfrost_probe(struct platform_device *pdev)
>  	ddev->dev_private = pfdev;
>  	pfdev->ddev = ddev;
>  
> -	mutex_init(&pfdev->shrinker_lock);
> -	INIT_LIST_HEAD(&pfdev->shrinker_list);
> -
>  	err = panfrost_device_init(pfdev);
>  	if (err) {
>  		if (err != -EPROBE_DEFER)
> @@ -601,7 +586,7 @@ static int panfrost_probe(struct platform_device *pdev)
>  	if (err < 0)
>  		goto err_out1;
>  
> -	panfrost_gem_shrinker_init(ddev);
> +	drm_gem_shmem_shrinker_register(ddev);
>  
>  	return 0;
>  
> @@ -619,8 +604,8 @@ static int panfrost_remove(struct platform_device *pdev)
>  	struct panfrost_device *pfdev = platform_get_drvdata(pdev);
>  	struct drm_device *ddev = pfdev->ddev;
>  
> +	drm_gem_shmem_shrinker_unregister(ddev);
>  	drm_dev_unregister(ddev);
> -	panfrost_gem_shrinker_cleanup(ddev);
>  
>  	pm_runtime_get_sync(pfdev->dev);
>  	pm_runtime_disable(pfdev->dev);
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.c b/drivers/gpu/drm/panfrost/panfrost_gem.c
> index 293e799e2fe8..f1436405e3a0 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem.c
> @@ -19,16 +19,6 @@ static void panfrost_gem_free_object(struct drm_gem_object *obj)
>  	struct panfrost_gem_object *bo = to_panfrost_bo(obj);
>  	struct panfrost_device *pfdev = obj->dev->dev_private;
>  
> -	/*
> -	 * Make sure the BO is no longer inserted in the shrinker list before
> -	 * taking care of the destruction itself. If we don't do that we have a
> -	 * race condition between this function and what's done in
> -	 * panfrost_gem_shrinker_scan().
> -	 */
> -	mutex_lock(&pfdev->shrinker_lock);
> -	list_del_init(&bo->base.madv_list);
> -	mutex_unlock(&pfdev->shrinker_lock);
> -
>  	/*
>  	 * If we still have mappings attached to the BO, there's a problem in
>  	 * our refcounting.
> @@ -209,6 +199,25 @@ static const struct drm_gem_object_funcs panfrost_gem_funcs = {
>  	.vm_ops = &drm_gem_shmem_vm_ops,
>  };
>  
> +static int panfrost_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +	struct panfrost_gem_object *bo = to_panfrost_bo(&shmem->base);
> +
> +	if (!drm_gem_shmem_is_purgeable(shmem))
> +		return -EOPNOTSUPP;
> +
> +	if (!mutex_trylock(&bo->mappings.lock))
> +		return -EBUSY;
> +
> +	panfrost_gem_teardown_mappings_locked(bo);
> +
> +	drm_gem_shmem_purge(shmem);
> +
> +	mutex_unlock(&bo->mappings.lock);
> +
> +	return 0;
> +}
> +
>  /**
>   * panfrost_gem_create_object - Implementation of driver->gem_create_object.
>   * @dev: DRM device
> @@ -230,6 +239,7 @@ struct drm_gem_object *panfrost_gem_create_object(struct drm_device *dev, size_t
>  	mutex_init(&obj->mappings.lock);
>  	obj->base.base.funcs = &panfrost_gem_funcs;
>  	obj->base.map_wc = !pfdev->coherent;
> +	obj->base.evict = panfrost_shmem_evict;
>  
>  	return &obj->base.base;
>  }
> @@ -266,6 +276,9 @@ panfrost_gem_create_with_handle(struct drm_file *file_priv,
>  	if (ret)
>  		return ERR_PTR(ret);
>  
> +	if (!bo->is_heap)
> +		drm_gem_shmem_set_purgeable(shmem);
> +
>  	return bo;
>  }
>  
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.h b/drivers/gpu/drm/panfrost/panfrost_gem.h
> index 8088d5fd8480..09da064f1c07 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem.h
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem.h
> @@ -30,12 +30,6 @@ struct panfrost_gem_object {
>  		struct mutex lock;
>  	} mappings;
>  
> -	/*
> -	 * Count the number of jobs referencing this BO so we don't let the
> -	 * shrinker reclaim this object prematurely.
> -	 */
> -	atomic_t gpu_usecount;
> -
>  	bool noexec		:1;
>  	bool is_heap		:1;
>  };
> @@ -84,7 +78,4 @@ panfrost_gem_mapping_get(struct panfrost_gem_object *bo,
>  void panfrost_gem_mapping_put(struct panfrost_gem_mapping *mapping);
>  void panfrost_gem_teardown_mappings_locked(struct panfrost_gem_object *bo);
>  
> -void panfrost_gem_shrinker_init(struct drm_device *dev);
> -void panfrost_gem_shrinker_cleanup(struct drm_device *dev);
> -
>  #endif /* __PANFROST_GEM_H__ */
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> deleted file mode 100644
> index 7cc32556f908..000000000000
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ /dev/null
> @@ -1,129 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/* Copyright (C) 2019 Arm Ltd.
> - *
> - * Based on msm_gem_freedreno.c:
> - * Copyright (C) 2016 Red Hat
> - * Author: Rob Clark <robdclark@gmail.com>
> - */
> -
> -#include <linux/list.h>
> -
> -#include <drm/drm_device.h>
> -#include <drm/drm_gem_shmem_helper.h>
> -
> -#include "panfrost_device.h"
> -#include "panfrost_gem.h"
> -#include "panfrost_mmu.h"
> -
> -static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> -{
> -	return (shmem->madv > 0) &&
> -		!shmem->pages_pin_count && shmem->sgt &&
> -		!shmem->base.dma_buf && !shmem->base.import_attach;
> -}
> -
> -static unsigned long
> -panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
> -{
> -	struct panfrost_device *pfdev =
> -		container_of(shrinker, struct panfrost_device, shrinker);
> -	struct drm_gem_shmem_object *shmem;
> -	unsigned long count = 0;
> -
> -	if (!mutex_trylock(&pfdev->shrinker_lock))
> -		return 0;
> -
> -	list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -		if (panfrost_gem_shmem_is_purgeable(shmem))
> -			count += shmem->base.size >> PAGE_SHIFT;
> -	}
> -
> -	mutex_unlock(&pfdev->shrinker_lock);
> -
> -	return count;
> -}
> -
> -static bool panfrost_gem_purge(struct drm_gem_object *obj)
> -{
> -	struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -	struct panfrost_gem_object *bo = to_panfrost_bo(obj);
> -	bool ret = false;
> -
> -	if (atomic_read(&bo->gpu_usecount))
> -		return false;
> -
> -	if (!mutex_trylock(&bo->mappings.lock))
> -		return false;
> -
> -	if (!dma_resv_trylock(shmem->base.resv))
> -		goto unlock_mappings;
> -
> -	panfrost_gem_teardown_mappings_locked(bo);
> -	drm_gem_shmem_purge(&bo->base);
> -	ret = true;
> -
> -	dma_resv_unlock(shmem->base.resv);
> -
> -unlock_mappings:
> -	mutex_unlock(&bo->mappings.lock);
> -	return ret;
> -}
> -
> -static unsigned long
> -panfrost_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc)
> -{
> -	struct panfrost_device *pfdev =
> -		container_of(shrinker, struct panfrost_device, shrinker);
> -	struct drm_gem_shmem_object *shmem, *tmp;
> -	unsigned long freed = 0;
> -
> -	if (!mutex_trylock(&pfdev->shrinker_lock))
> -		return SHRINK_STOP;
> -
> -	list_for_each_entry_safe(shmem, tmp, &pfdev->shrinker_list, madv_list) {
> -		if (freed >= sc->nr_to_scan)
> -			break;
> -		if (drm_gem_shmem_is_purgeable(shmem) &&
> -		    panfrost_gem_purge(&shmem->base)) {
> -			freed += shmem->base.size >> PAGE_SHIFT;
> -			list_del_init(&shmem->madv_list);
> -		}
> -	}
> -
> -	mutex_unlock(&pfdev->shrinker_lock);
> -
> -	if (freed > 0)
> -		pr_info_ratelimited("Purging %lu bytes\n", freed << PAGE_SHIFT);
> -
> -	return freed;
> -}
> -
> -/**
> - * panfrost_gem_shrinker_init - Initialize panfrost shrinker
> - * @dev: DRM device
> - *
> - * This function registers and sets up the panfrost shrinker.
> - */
> -void panfrost_gem_shrinker_init(struct drm_device *dev)
> -{
> -	struct panfrost_device *pfdev = dev->dev_private;
> -	pfdev->shrinker.count_objects = panfrost_gem_shrinker_count;
> -	pfdev->shrinker.scan_objects = panfrost_gem_shrinker_scan;
> -	pfdev->shrinker.seeks = DEFAULT_SEEKS;
> -	WARN_ON(register_shrinker(&pfdev->shrinker));
> -}
> -
> -/**
> - * panfrost_gem_shrinker_cleanup - Clean up panfrost shrinker
> - * @dev: DRM device
> - *
> - * This function unregisters the panfrost shrinker.
> - */
> -void panfrost_gem_shrinker_cleanup(struct drm_device *dev)
> -{
> -	struct panfrost_device *pfdev = dev->dev_private;
> -
> -	if (pfdev->shrinker.nr_deferred) {
> -		unregister_shrinker(&pfdev->shrinker);
> -	}
> -}
> diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
> index 7c4208476fbd..5c327a79455f 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_job.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_job.c
> @@ -271,6 +271,19 @@ static void panfrost_attach_object_fences(struct drm_gem_object **bos,
>  		dma_resv_add_fence(bos[i]->resv, fence, DMA_RESV_USAGE_WRITE);
>  }
>  
> +static int panfrost_objects_prepare(struct drm_gem_object **bos, int bo_count)
> +{
> +	struct panfrost_gem_object *bo;
> +	int ret = 0;
> +
> +	while (!ret && bo_count--) {
> +		bo = to_panfrost_bo(bos[bo_count]);
> +		ret = bo->base.madv ? -ENOMEM : 0;
> +	}
> +
> +	return ret;
> +}
> +
>  int panfrost_job_push(struct panfrost_job *job)
>  {
>  	struct panfrost_device *pfdev = job->pfdev;
> @@ -282,6 +295,10 @@ int panfrost_job_push(struct panfrost_job *job)
>  	if (ret)
>  		return ret;
>  
> +	ret = panfrost_objects_prepare(job->bos, job->bo_count);
> +	if (ret)
> +		goto unlock;
> +
>  	mutex_lock(&pfdev->sched_lock);
>  	drm_sched_job_arm(&job->base);
>  
> @@ -323,7 +340,6 @@ static void panfrost_job_cleanup(struct kref *ref)
>  			if (!job->mappings[i])
>  				break;
>  
> -			atomic_dec(&job->mappings[i]->obj->gpu_usecount);
>  			panfrost_gem_mapping_put(job->mappings[i]);
>  		}
>  		kvfree(job->mappings);
> -- 
> 2.35.3
> 

Re: [PATCH v6 22/22] drm/panfrost: Switch to generic memory shrinker

[Alyssa Rosenzweig]

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

On Fri, May 27, 2022 at 02:50:40AM +0300, Dmitry Osipenko wrote:
> Replace Panfrost's memory shrinker with a generic drm-shmem memory
> shrinker.
> 
> Tested-by: Steven Price <steven.price@arm.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/panfrost/Makefile             |   1 -
>  drivers/gpu/drm/panfrost/panfrost_device.h    |   4 -
>  drivers/gpu/drm/panfrost/panfrost_drv.c       |  19 +--
>  drivers/gpu/drm/panfrost/panfrost_gem.c       |  33 +++--
>  drivers/gpu/drm/panfrost/panfrost_gem.h       |   9 --
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  | 129 ------------------
>  drivers/gpu/drm/panfrost/panfrost_job.c       |  18 ++-
>  7 files changed, 42 insertions(+), 171 deletions(-)
>  delete mode 100644 drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> 
> diff --git a/drivers/gpu/drm/panfrost/Makefile b/drivers/gpu/drm/panfrost/Makefile
> index b71935862417..ecf0864cb515 100644
> --- a/drivers/gpu/drm/panfrost/Makefile
> +++ b/drivers/gpu/drm/panfrost/Makefile
> @@ -5,7 +5,6 @@ panfrost-y := \
>  	panfrost_device.o \
>  	panfrost_devfreq.o \
>  	panfrost_gem.o \
> -	panfrost_gem_shrinker.o \
>  	panfrost_gpu.o \
>  	panfrost_job.o \
>  	panfrost_mmu.o \
> diff --git a/drivers/gpu/drm/panfrost/panfrost_device.h b/drivers/gpu/drm/panfrost/panfrost_device.h
> index 8b25278f34c8..fe04b21fc044 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_device.h
> +++ b/drivers/gpu/drm/panfrost/panfrost_device.h
> @@ -115,10 +115,6 @@ struct panfrost_device {
>  		atomic_t pending;
>  	} reset;
>  
> -	struct mutex shrinker_lock;
> -	struct list_head shrinker_list;
> -	struct shrinker shrinker;
> -
>  	struct panfrost_devfreq pfdevfreq;
>  };
>  
> diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
> index 859e240161d1..b77c99ba2475 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_drv.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
> @@ -160,7 +160,6 @@ panfrost_lookup_bos(struct drm_device *dev,
>  			break;
>  		}
>  
> -		atomic_inc(&bo->gpu_usecount);
>  		job->mappings[i] = mapping;
>  	}
>  
> @@ -392,7 +391,6 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
>  {
>  	struct panfrost_file_priv *priv = file_priv->driver_priv;
>  	struct drm_panfrost_madvise *args = data;
> -	struct panfrost_device *pfdev = dev->dev_private;
>  	struct drm_gem_object *gem_obj;
>  	struct panfrost_gem_object *bo;
>  	int ret = 0;
> @@ -409,7 +407,6 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
>  	if (ret)
>  		goto out_put_object;
>  
> -	mutex_lock(&pfdev->shrinker_lock);
>  	mutex_lock(&bo->mappings.lock);
>  	if (args->madv == PANFROST_MADV_DONTNEED) {
>  		struct panfrost_gem_mapping *first;
> @@ -435,17 +432,8 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
>  
>  	args->retained = drm_gem_shmem_madvise(&bo->base, args->madv);
>  
> -	if (args->retained) {
> -		if (args->madv == PANFROST_MADV_DONTNEED)
> -			list_move_tail(&bo->base.madv_list,
> -				       &pfdev->shrinker_list);
> -		else if (args->madv == PANFROST_MADV_WILLNEED)
> -			list_del_init(&bo->base.madv_list);
> -	}
> -
>  out_unlock_mappings:
>  	mutex_unlock(&bo->mappings.lock);
> -	mutex_unlock(&pfdev->shrinker_lock);
>  	dma_resv_unlock(bo->base.base.resv);
>  out_put_object:
>  	drm_gem_object_put(gem_obj);
> @@ -577,9 +565,6 @@ static int panfrost_probe(struct platform_device *pdev)
>  	ddev->dev_private = pfdev;
>  	pfdev->ddev = ddev;
>  
> -	mutex_init(&pfdev->shrinker_lock);
> -	INIT_LIST_HEAD(&pfdev->shrinker_list);
> -
>  	err = panfrost_device_init(pfdev);
>  	if (err) {
>  		if (err != -EPROBE_DEFER)
> @@ -601,7 +586,7 @@ static int panfrost_probe(struct platform_device *pdev)
>  	if (err < 0)
>  		goto err_out1;
>  
> -	panfrost_gem_shrinker_init(ddev);
> +	drm_gem_shmem_shrinker_register(ddev);
>  
>  	return 0;
>  
> @@ -619,8 +604,8 @@ static int panfrost_remove(struct platform_device *pdev)
>  	struct panfrost_device *pfdev = platform_get_drvdata(pdev);
>  	struct drm_device *ddev = pfdev->ddev;
>  
> +	drm_gem_shmem_shrinker_unregister(ddev);
>  	drm_dev_unregister(ddev);
> -	panfrost_gem_shrinker_cleanup(ddev);
>  
>  	pm_runtime_get_sync(pfdev->dev);
>  	pm_runtime_disable(pfdev->dev);
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.c b/drivers/gpu/drm/panfrost/panfrost_gem.c
> index 293e799e2fe8..f1436405e3a0 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem.c
> @@ -19,16 +19,6 @@ static void panfrost_gem_free_object(struct drm_gem_object *obj)
>  	struct panfrost_gem_object *bo = to_panfrost_bo(obj);
>  	struct panfrost_device *pfdev = obj->dev->dev_private;
>  
> -	/*
> -	 * Make sure the BO is no longer inserted in the shrinker list before
> -	 * taking care of the destruction itself. If we don't do that we have a
> -	 * race condition between this function and what's done in
> -	 * panfrost_gem_shrinker_scan().
> -	 */
> -	mutex_lock(&pfdev->shrinker_lock);
> -	list_del_init(&bo->base.madv_list);
> -	mutex_unlock(&pfdev->shrinker_lock);
> -
>  	/*
>  	 * If we still have mappings attached to the BO, there's a problem in
>  	 * our refcounting.
> @@ -209,6 +199,25 @@ static const struct drm_gem_object_funcs panfrost_gem_funcs = {
>  	.vm_ops = &drm_gem_shmem_vm_ops,
>  };
>  
> +static int panfrost_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +	struct panfrost_gem_object *bo = to_panfrost_bo(&shmem->base);
> +
> +	if (!drm_gem_shmem_is_purgeable(shmem))
> +		return -EOPNOTSUPP;
> +
> +	if (!mutex_trylock(&bo->mappings.lock))
> +		return -EBUSY;
> +
> +	panfrost_gem_teardown_mappings_locked(bo);
> +
> +	drm_gem_shmem_purge(shmem);
> +
> +	mutex_unlock(&bo->mappings.lock);
> +
> +	return 0;
> +}
> +
>  /**
>   * panfrost_gem_create_object - Implementation of driver->gem_create_object.
>   * @dev: DRM device
> @@ -230,6 +239,7 @@ struct drm_gem_object *panfrost_gem_create_object(struct drm_device *dev, size_t
>  	mutex_init(&obj->mappings.lock);
>  	obj->base.base.funcs = &panfrost_gem_funcs;
>  	obj->base.map_wc = !pfdev->coherent;
> +	obj->base.evict = panfrost_shmem_evict;
>  
>  	return &obj->base.base;
>  }
> @@ -266,6 +276,9 @@ panfrost_gem_create_with_handle(struct drm_file *file_priv,
>  	if (ret)
>  		return ERR_PTR(ret);
>  
> +	if (!bo->is_heap)
> +		drm_gem_shmem_set_purgeable(shmem);
> +
>  	return bo;
>  }
>  
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.h b/drivers/gpu/drm/panfrost/panfrost_gem.h
> index 8088d5fd8480..09da064f1c07 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem.h
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem.h
> @@ -30,12 +30,6 @@ struct panfrost_gem_object {
>  		struct mutex lock;
>  	} mappings;
>  
> -	/*
> -	 * Count the number of jobs referencing this BO so we don't let the
> -	 * shrinker reclaim this object prematurely.
> -	 */
> -	atomic_t gpu_usecount;
> -
>  	bool noexec		:1;
>  	bool is_heap		:1;
>  };
> @@ -84,7 +78,4 @@ panfrost_gem_mapping_get(struct panfrost_gem_object *bo,
>  void panfrost_gem_mapping_put(struct panfrost_gem_mapping *mapping);
>  void panfrost_gem_teardown_mappings_locked(struct panfrost_gem_object *bo);
>  
> -void panfrost_gem_shrinker_init(struct drm_device *dev);
> -void panfrost_gem_shrinker_cleanup(struct drm_device *dev);
> -
>  #endif /* __PANFROST_GEM_H__ */
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> deleted file mode 100644
> index 7cc32556f908..000000000000
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ /dev/null
> @@ -1,129 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/* Copyright (C) 2019 Arm Ltd.
> - *
> - * Based on msm_gem_freedreno.c:
> - * Copyright (C) 2016 Red Hat
> - * Author: Rob Clark <robdclark@gmail.com>
> - */
> -
> -#include <linux/list.h>
> -
> -#include <drm/drm_device.h>
> -#include <drm/drm_gem_shmem_helper.h>
> -
> -#include "panfrost_device.h"
> -#include "panfrost_gem.h"
> -#include "panfrost_mmu.h"
> -
> -static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> -{
> -	return (shmem->madv > 0) &&
> -		!shmem->pages_pin_count && shmem->sgt &&
> -		!shmem->base.dma_buf && !shmem->base.import_attach;
> -}
> -
> -static unsigned long
> -panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
> -{
> -	struct panfrost_device *pfdev =
> -		container_of(shrinker, struct panfrost_device, shrinker);
> -	struct drm_gem_shmem_object *shmem;
> -	unsigned long count = 0;
> -
> -	if (!mutex_trylock(&pfdev->shrinker_lock))
> -		return 0;
> -
> -	list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -		if (panfrost_gem_shmem_is_purgeable(shmem))
> -			count += shmem->base.size >> PAGE_SHIFT;
> -	}
> -
> -	mutex_unlock(&pfdev->shrinker_lock);
> -
> -	return count;
> -}
> -
> -static bool panfrost_gem_purge(struct drm_gem_object *obj)
> -{
> -	struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -	struct panfrost_gem_object *bo = to_panfrost_bo(obj);
> -	bool ret = false;
> -
> -	if (atomic_read(&bo->gpu_usecount))
> -		return false;
> -
> -	if (!mutex_trylock(&bo->mappings.lock))
> -		return false;
> -
> -	if (!dma_resv_trylock(shmem->base.resv))
> -		goto unlock_mappings;
> -
> -	panfrost_gem_teardown_mappings_locked(bo);
> -	drm_gem_shmem_purge(&bo->base);
> -	ret = true;
> -
> -	dma_resv_unlock(shmem->base.resv);
> -
> -unlock_mappings:
> -	mutex_unlock(&bo->mappings.lock);
> -	return ret;
> -}
> -
> -static unsigned long
> -panfrost_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc)
> -{
> -	struct panfrost_device *pfdev =
> -		container_of(shrinker, struct panfrost_device, shrinker);
> -	struct drm_gem_shmem_object *shmem, *tmp;
> -	unsigned long freed = 0;
> -
> -	if (!mutex_trylock(&pfdev->shrinker_lock))
> -		return SHRINK_STOP;
> -
> -	list_for_each_entry_safe(shmem, tmp, &pfdev->shrinker_list, madv_list) {
> -		if (freed >= sc->nr_to_scan)
> -			break;
> -		if (drm_gem_shmem_is_purgeable(shmem) &&
> -		    panfrost_gem_purge(&shmem->base)) {
> -			freed += shmem->base.size >> PAGE_SHIFT;
> -			list_del_init(&shmem->madv_list);
> -		}
> -	}
> -
> -	mutex_unlock(&pfdev->shrinker_lock);
> -
> -	if (freed > 0)
> -		pr_info_ratelimited("Purging %lu bytes\n", freed << PAGE_SHIFT);
> -
> -	return freed;
> -}
> -
> -/**
> - * panfrost_gem_shrinker_init - Initialize panfrost shrinker
> - * @dev: DRM device
> - *
> - * This function registers and sets up the panfrost shrinker.
> - */
> -void panfrost_gem_shrinker_init(struct drm_device *dev)
> -{
> -	struct panfrost_device *pfdev = dev->dev_private;
> -	pfdev->shrinker.count_objects = panfrost_gem_shrinker_count;
> -	pfdev->shrinker.scan_objects = panfrost_gem_shrinker_scan;
> -	pfdev->shrinker.seeks = DEFAULT_SEEKS;
> -	WARN_ON(register_shrinker(&pfdev->shrinker));
> -}
> -
> -/**
> - * panfrost_gem_shrinker_cleanup - Clean up panfrost shrinker
> - * @dev: DRM device
> - *
> - * This function unregisters the panfrost shrinker.
> - */
> -void panfrost_gem_shrinker_cleanup(struct drm_device *dev)
> -{
> -	struct panfrost_device *pfdev = dev->dev_private;
> -
> -	if (pfdev->shrinker.nr_deferred) {
> -		unregister_shrinker(&pfdev->shrinker);
> -	}
> -}
> diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
> index 7c4208476fbd..5c327a79455f 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_job.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_job.c
> @@ -271,6 +271,19 @@ static void panfrost_attach_object_fences(struct drm_gem_object **bos,
>  		dma_resv_add_fence(bos[i]->resv, fence, DMA_RESV_USAGE_WRITE);
>  }
>  
> +static int panfrost_objects_prepare(struct drm_gem_object **bos, int bo_count)
> +{
> +	struct panfrost_gem_object *bo;
> +	int ret = 0;
> +
> +	while (!ret && bo_count--) {
> +		bo = to_panfrost_bo(bos[bo_count]);
> +		ret = bo->base.madv ? -ENOMEM : 0;
> +	}
> +
> +	return ret;
> +}
> +
>  int panfrost_job_push(struct panfrost_job *job)
>  {
>  	struct panfrost_device *pfdev = job->pfdev;
> @@ -282,6 +295,10 @@ int panfrost_job_push(struct panfrost_job *job)
>  	if (ret)
>  		return ret;
>  
> +	ret = panfrost_objects_prepare(job->bos, job->bo_count);
> +	if (ret)
> +		goto unlock;
> +
>  	mutex_lock(&pfdev->sched_lock);
>  	drm_sched_job_arm(&job->base);
>  
> @@ -323,7 +340,6 @@ static void panfrost_job_cleanup(struct kref *ref)
>  			if (!job->mappings[i])
>  				break;
>  
> -			atomic_dec(&job->mappings[i]->obj->gpu_usecount);
>  			panfrost_gem_mapping_put(job->mappings[i]);
>  		}
>  		kvfree(job->mappings);
> -- 
> 2.35.3
> 

Re: [Intel-gfx] [PATCH v6 22/22] drm/panfrost: Switch to generic memory shrinker

[Alyssa Rosenzweig]

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

On Fri, May 27, 2022 at 02:50:40AM +0300, Dmitry Osipenko wrote:
> Replace Panfrost's memory shrinker with a generic drm-shmem memory
> shrinker.
> 
> Tested-by: Steven Price <steven.price@arm.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/panfrost/Makefile             |   1 -
>  drivers/gpu/drm/panfrost/panfrost_device.h    |   4 -
>  drivers/gpu/drm/panfrost/panfrost_drv.c       |  19 +--
>  drivers/gpu/drm/panfrost/panfrost_gem.c       |  33 +++--
>  drivers/gpu/drm/panfrost/panfrost_gem.h       |   9 --
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  | 129 ------------------
>  drivers/gpu/drm/panfrost/panfrost_job.c       |  18 ++-
>  7 files changed, 42 insertions(+), 171 deletions(-)
>  delete mode 100644 drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> 
> diff --git a/drivers/gpu/drm/panfrost/Makefile b/drivers/gpu/drm/panfrost/Makefile
> index b71935862417..ecf0864cb515 100644
> --- a/drivers/gpu/drm/panfrost/Makefile
> +++ b/drivers/gpu/drm/panfrost/Makefile
> @@ -5,7 +5,6 @@ panfrost-y := \
>  	panfrost_device.o \
>  	panfrost_devfreq.o \
>  	panfrost_gem.o \
> -	panfrost_gem_shrinker.o \
>  	panfrost_gpu.o \
>  	panfrost_job.o \
>  	panfrost_mmu.o \
> diff --git a/drivers/gpu/drm/panfrost/panfrost_device.h b/drivers/gpu/drm/panfrost/panfrost_device.h
> index 8b25278f34c8..fe04b21fc044 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_device.h
> +++ b/drivers/gpu/drm/panfrost/panfrost_device.h
> @@ -115,10 +115,6 @@ struct panfrost_device {
>  		atomic_t pending;
>  	} reset;
>  
> -	struct mutex shrinker_lock;
> -	struct list_head shrinker_list;
> -	struct shrinker shrinker;
> -
>  	struct panfrost_devfreq pfdevfreq;
>  };
>  
> diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
> index 859e240161d1..b77c99ba2475 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_drv.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
> @@ -160,7 +160,6 @@ panfrost_lookup_bos(struct drm_device *dev,
>  			break;
>  		}
>  
> -		atomic_inc(&bo->gpu_usecount);
>  		job->mappings[i] = mapping;
>  	}
>  
> @@ -392,7 +391,6 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
>  {
>  	struct panfrost_file_priv *priv = file_priv->driver_priv;
>  	struct drm_panfrost_madvise *args = data;
> -	struct panfrost_device *pfdev = dev->dev_private;
>  	struct drm_gem_object *gem_obj;
>  	struct panfrost_gem_object *bo;
>  	int ret = 0;
> @@ -409,7 +407,6 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
>  	if (ret)
>  		goto out_put_object;
>  
> -	mutex_lock(&pfdev->shrinker_lock);
>  	mutex_lock(&bo->mappings.lock);
>  	if (args->madv == PANFROST_MADV_DONTNEED) {
>  		struct panfrost_gem_mapping *first;
> @@ -435,17 +432,8 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
>  
>  	args->retained = drm_gem_shmem_madvise(&bo->base, args->madv);
>  
> -	if (args->retained) {
> -		if (args->madv == PANFROST_MADV_DONTNEED)
> -			list_move_tail(&bo->base.madv_list,
> -				       &pfdev->shrinker_list);
> -		else if (args->madv == PANFROST_MADV_WILLNEED)
> -			list_del_init(&bo->base.madv_list);
> -	}
> -
>  out_unlock_mappings:
>  	mutex_unlock(&bo->mappings.lock);
> -	mutex_unlock(&pfdev->shrinker_lock);
>  	dma_resv_unlock(bo->base.base.resv);
>  out_put_object:
>  	drm_gem_object_put(gem_obj);
> @@ -577,9 +565,6 @@ static int panfrost_probe(struct platform_device *pdev)
>  	ddev->dev_private = pfdev;
>  	pfdev->ddev = ddev;
>  
> -	mutex_init(&pfdev->shrinker_lock);
> -	INIT_LIST_HEAD(&pfdev->shrinker_list);
> -
>  	err = panfrost_device_init(pfdev);
>  	if (err) {
>  		if (err != -EPROBE_DEFER)
> @@ -601,7 +586,7 @@ static int panfrost_probe(struct platform_device *pdev)
>  	if (err < 0)
>  		goto err_out1;
>  
> -	panfrost_gem_shrinker_init(ddev);
> +	drm_gem_shmem_shrinker_register(ddev);
>  
>  	return 0;
>  
> @@ -619,8 +604,8 @@ static int panfrost_remove(struct platform_device *pdev)
>  	struct panfrost_device *pfdev = platform_get_drvdata(pdev);
>  	struct drm_device *ddev = pfdev->ddev;
>  
> +	drm_gem_shmem_shrinker_unregister(ddev);
>  	drm_dev_unregister(ddev);
> -	panfrost_gem_shrinker_cleanup(ddev);
>  
>  	pm_runtime_get_sync(pfdev->dev);
>  	pm_runtime_disable(pfdev->dev);
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.c b/drivers/gpu/drm/panfrost/panfrost_gem.c
> index 293e799e2fe8..f1436405e3a0 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem.c
> @@ -19,16 +19,6 @@ static void panfrost_gem_free_object(struct drm_gem_object *obj)
>  	struct panfrost_gem_object *bo = to_panfrost_bo(obj);
>  	struct panfrost_device *pfdev = obj->dev->dev_private;
>  
> -	/*
> -	 * Make sure the BO is no longer inserted in the shrinker list before
> -	 * taking care of the destruction itself. If we don't do that we have a
> -	 * race condition between this function and what's done in
> -	 * panfrost_gem_shrinker_scan().
> -	 */
> -	mutex_lock(&pfdev->shrinker_lock);
> -	list_del_init(&bo->base.madv_list);
> -	mutex_unlock(&pfdev->shrinker_lock);
> -
>  	/*
>  	 * If we still have mappings attached to the BO, there's a problem in
>  	 * our refcounting.
> @@ -209,6 +199,25 @@ static const struct drm_gem_object_funcs panfrost_gem_funcs = {
>  	.vm_ops = &drm_gem_shmem_vm_ops,
>  };
>  
> +static int panfrost_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +	struct panfrost_gem_object *bo = to_panfrost_bo(&shmem->base);
> +
> +	if (!drm_gem_shmem_is_purgeable(shmem))
> +		return -EOPNOTSUPP;
> +
> +	if (!mutex_trylock(&bo->mappings.lock))
> +		return -EBUSY;
> +
> +	panfrost_gem_teardown_mappings_locked(bo);
> +
> +	drm_gem_shmem_purge(shmem);
> +
> +	mutex_unlock(&bo->mappings.lock);
> +
> +	return 0;
> +}
> +
>  /**
>   * panfrost_gem_create_object - Implementation of driver->gem_create_object.
>   * @dev: DRM device
> @@ -230,6 +239,7 @@ struct drm_gem_object *panfrost_gem_create_object(struct drm_device *dev, size_t
>  	mutex_init(&obj->mappings.lock);
>  	obj->base.base.funcs = &panfrost_gem_funcs;
>  	obj->base.map_wc = !pfdev->coherent;
> +	obj->base.evict = panfrost_shmem_evict;
>  
>  	return &obj->base.base;
>  }
> @@ -266,6 +276,9 @@ panfrost_gem_create_with_handle(struct drm_file *file_priv,
>  	if (ret)
>  		return ERR_PTR(ret);
>  
> +	if (!bo->is_heap)
> +		drm_gem_shmem_set_purgeable(shmem);
> +
>  	return bo;
>  }
>  
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.h b/drivers/gpu/drm/panfrost/panfrost_gem.h
> index 8088d5fd8480..09da064f1c07 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem.h
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem.h
> @@ -30,12 +30,6 @@ struct panfrost_gem_object {
>  		struct mutex lock;
>  	} mappings;
>  
> -	/*
> -	 * Count the number of jobs referencing this BO so we don't let the
> -	 * shrinker reclaim this object prematurely.
> -	 */
> -	atomic_t gpu_usecount;
> -
>  	bool noexec		:1;
>  	bool is_heap		:1;
>  };
> @@ -84,7 +78,4 @@ panfrost_gem_mapping_get(struct panfrost_gem_object *bo,
>  void panfrost_gem_mapping_put(struct panfrost_gem_mapping *mapping);
>  void panfrost_gem_teardown_mappings_locked(struct panfrost_gem_object *bo);
>  
> -void panfrost_gem_shrinker_init(struct drm_device *dev);
> -void panfrost_gem_shrinker_cleanup(struct drm_device *dev);
> -
>  #endif /* __PANFROST_GEM_H__ */
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> deleted file mode 100644
> index 7cc32556f908..000000000000
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ /dev/null
> @@ -1,129 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0
> -/* Copyright (C) 2019 Arm Ltd.
> - *
> - * Based on msm_gem_freedreno.c:
> - * Copyright (C) 2016 Red Hat
> - * Author: Rob Clark <robdclark@gmail.com>
> - */
> -
> -#include <linux/list.h>
> -
> -#include <drm/drm_device.h>
> -#include <drm/drm_gem_shmem_helper.h>
> -
> -#include "panfrost_device.h"
> -#include "panfrost_gem.h"
> -#include "panfrost_mmu.h"
> -
> -static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> -{
> -	return (shmem->madv > 0) &&
> -		!shmem->pages_pin_count && shmem->sgt &&
> -		!shmem->base.dma_buf && !shmem->base.import_attach;
> -}
> -
> -static unsigned long
> -panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
> -{
> -	struct panfrost_device *pfdev =
> -		container_of(shrinker, struct panfrost_device, shrinker);
> -	struct drm_gem_shmem_object *shmem;
> -	unsigned long count = 0;
> -
> -	if (!mutex_trylock(&pfdev->shrinker_lock))
> -		return 0;
> -
> -	list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -		if (panfrost_gem_shmem_is_purgeable(shmem))
> -			count += shmem->base.size >> PAGE_SHIFT;
> -	}
> -
> -	mutex_unlock(&pfdev->shrinker_lock);
> -
> -	return count;
> -}
> -
> -static bool panfrost_gem_purge(struct drm_gem_object *obj)
> -{
> -	struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -	struct panfrost_gem_object *bo = to_panfrost_bo(obj);
> -	bool ret = false;
> -
> -	if (atomic_read(&bo->gpu_usecount))
> -		return false;
> -
> -	if (!mutex_trylock(&bo->mappings.lock))
> -		return false;
> -
> -	if (!dma_resv_trylock(shmem->base.resv))
> -		goto unlock_mappings;
> -
> -	panfrost_gem_teardown_mappings_locked(bo);
> -	drm_gem_shmem_purge(&bo->base);
> -	ret = true;
> -
> -	dma_resv_unlock(shmem->base.resv);
> -
> -unlock_mappings:
> -	mutex_unlock(&bo->mappings.lock);
> -	return ret;
> -}
> -
> -static unsigned long
> -panfrost_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc)
> -{
> -	struct panfrost_device *pfdev =
> -		container_of(shrinker, struct panfrost_device, shrinker);
> -	struct drm_gem_shmem_object *shmem, *tmp;
> -	unsigned long freed = 0;
> -
> -	if (!mutex_trylock(&pfdev->shrinker_lock))
> -		return SHRINK_STOP;
> -
> -	list_for_each_entry_safe(shmem, tmp, &pfdev->shrinker_list, madv_list) {
> -		if (freed >= sc->nr_to_scan)
> -			break;
> -		if (drm_gem_shmem_is_purgeable(shmem) &&
> -		    panfrost_gem_purge(&shmem->base)) {
> -			freed += shmem->base.size >> PAGE_SHIFT;
> -			list_del_init(&shmem->madv_list);
> -		}
> -	}
> -
> -	mutex_unlock(&pfdev->shrinker_lock);
> -
> -	if (freed > 0)
> -		pr_info_ratelimited("Purging %lu bytes\n", freed << PAGE_SHIFT);
> -
> -	return freed;
> -}
> -
> -/**
> - * panfrost_gem_shrinker_init - Initialize panfrost shrinker
> - * @dev: DRM device
> - *
> - * This function registers and sets up the panfrost shrinker.
> - */
> -void panfrost_gem_shrinker_init(struct drm_device *dev)
> -{
> -	struct panfrost_device *pfdev = dev->dev_private;
> -	pfdev->shrinker.count_objects = panfrost_gem_shrinker_count;
> -	pfdev->shrinker.scan_objects = panfrost_gem_shrinker_scan;
> -	pfdev->shrinker.seeks = DEFAULT_SEEKS;
> -	WARN_ON(register_shrinker(&pfdev->shrinker));
> -}
> -
> -/**
> - * panfrost_gem_shrinker_cleanup - Clean up panfrost shrinker
> - * @dev: DRM device
> - *
> - * This function unregisters the panfrost shrinker.
> - */
> -void panfrost_gem_shrinker_cleanup(struct drm_device *dev)
> -{
> -	struct panfrost_device *pfdev = dev->dev_private;
> -
> -	if (pfdev->shrinker.nr_deferred) {
> -		unregister_shrinker(&pfdev->shrinker);
> -	}
> -}
> diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
> index 7c4208476fbd..5c327a79455f 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_job.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_job.c
> @@ -271,6 +271,19 @@ static void panfrost_attach_object_fences(struct drm_gem_object **bos,
>  		dma_resv_add_fence(bos[i]->resv, fence, DMA_RESV_USAGE_WRITE);
>  }
>  
> +static int panfrost_objects_prepare(struct drm_gem_object **bos, int bo_count)
> +{
> +	struct panfrost_gem_object *bo;
> +	int ret = 0;
> +
> +	while (!ret && bo_count--) {
> +		bo = to_panfrost_bo(bos[bo_count]);
> +		ret = bo->base.madv ? -ENOMEM : 0;
> +	}
> +
> +	return ret;
> +}
> +
>  int panfrost_job_push(struct panfrost_job *job)
>  {
>  	struct panfrost_device *pfdev = job->pfdev;
> @@ -282,6 +295,10 @@ int panfrost_job_push(struct panfrost_job *job)
>  	if (ret)
>  		return ret;
>  
> +	ret = panfrost_objects_prepare(job->bos, job->bo_count);
> +	if (ret)
> +		goto unlock;
> +
>  	mutex_lock(&pfdev->sched_lock);
>  	drm_sched_job_arm(&job->base);
>  
> @@ -323,7 +340,6 @@ static void panfrost_job_cleanup(struct kref *ref)
>  			if (!job->mappings[i])
>  				break;
>  
> -			atomic_dec(&job->mappings[i]->obj->gpu_usecount);
>  			panfrost_gem_mapping_put(job->mappings[i]);
>  		}
>  		kvfree(job->mappings);
> -- 
> 2.35.3
> 

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[kernel test robot]

[-- Attachment #1: Type: text/plain, Size: 13966 bytes --]

CC: kbuild-all(a)lists.01.org
BCC: lkp(a)intel.com
In-Reply-To: <20220526235040.678984-15-dmitry.osipenko@collabora.com>
References: <20220526235040.678984-15-dmitry.osipenko@collabora.com>
TO: Dmitry Osipenko <dmitry.osipenko@collabora.com>
TO: David Airlie <airlied@linux.ie>
TO: Gerd Hoffmann <kraxel@redhat.com>
TO: Gurchetan Singh <gurchetansingh@chromium.org>
TO: "Chia-I Wu" <olvaffe@gmail.com>
TO: Daniel Vetter <daniel@ffwll.ch>
TO: Daniel Almeida <daniel.almeida@collabora.com>
TO: Gert Wollny <gert.wollny@collabora.com>
TO: Gustavo Padovan <gustavo.padovan@collabora.com>
TO: Daniel Stone <daniel@fooishbar.org>
TO: Tomeu Vizoso <tomeu.vizoso@collabora.com>
TO: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
TO: Maxime Ripard <mripard@kernel.org>
TO: Thomas Zimmermann <tzimmermann@suse.de>
TO: Rob Herring <robh@kernel.org>
TO: Steven Price <steven.price@arm.com>
TO: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
TO: Rob Clark <robdclark@gmail.com>
TO: Emil Velikov <emil.l.velikov@gmail.com>
TO: Robin Murphy <robin.murphy@arm.com>
TO: Qiang Yu <yuq825@gmail.com>
TO: Sumit Semwal <sumit.semwal@linaro.org>
TO: "Christian König" <christian.koenig@amd.com>
TO: "Pan, Xinhui" <Xinhui.Pan@amd.com>
TO: Thierry Reding <thierry.reding@gmail.com>
TO: Tomasz Figa <tfiga@chromium.org>
TO: Marek Szyprowski <m.szyprowski@samsung.com>
TO: Mauro Carvalho Chehab <mchehab@kernel.org>
CC: linux-media(a)vger.kernel.org
TO: Alex Deucher <alexander.deucher@amd.com>
TO: Jani Nikula <jani.nikula@linux.intel.com>

Hi Dmitry,

I love your patch! Perhaps something to improve:

[auto build test WARNING on linus/master]
[also build test WARNING on next-20220527]
[cannot apply to drm/drm-next media-tree/master drm-intel/for-linux-next v5.18]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/intel-lab-lkp/linux/commits/Dmitry-Osipenko/Add-generic-memory-shrinker-to-VirtIO-GPU-and-Panfrost-DRM-drivers/20220527-075717
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git cdeffe87f790dfd1baa193020411ce9a538446d7
:::::: branch date: 12 hours ago
:::::: commit date: 12 hours ago
config: i386-randconfig-m021 (https://download.01.org/0day-ci/archive/20220527/202205272006.EZ53cUSD-lkp(a)intel.com/config)
compiler: gcc-11 (Debian 11.3.0-1) 11.3.0

If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>

New smatch warnings:
drivers/dma-buf/dma-buf.c:791 dma_buf_dynamic_attach() warn: inconsistent returns 'dmabuf->resv'.
drivers/dma-buf/dma-buf.c:1339 dma_buf_vmap_locked() error: uninitialized symbol 'ret'.

Old smatch warnings:
drivers/dma-buf/dma-buf.c:576 dma_buf_export() warn: '&dmabuf->list_node' not removed from list

vim +791 drivers/dma-buf/dma-buf.c

84335675f2223c drivers/dma-buf/dma-buf.c Daniel Vetter      2021-01-15  691  
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  692  /**
85804b70cca68d drivers/dma-buf/dma-buf.c Daniel Vetter      2020-12-11  693   * dma_buf_dynamic_attach - Add the device to dma_buf's attachments list
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  694   * @dmabuf:		[in]	buffer to attach device to.
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  695   * @dev:		[in]	device to be attached.
6f49c2515e2258 drivers/dma-buf/dma-buf.c Randy Dunlap       2020-04-07  696   * @importer_ops:	[in]	importer operations for the attachment
6f49c2515e2258 drivers/dma-buf/dma-buf.c Randy Dunlap       2020-04-07  697   * @importer_priv:	[in]	importer private pointer for the attachment
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  698   *
2904a8c1311f02 drivers/dma-buf/dma-buf.c Daniel Vetter      2016-12-09  699   * Returns struct dma_buf_attachment pointer for this attachment. Attachments
2904a8c1311f02 drivers/dma-buf/dma-buf.c Daniel Vetter      2016-12-09  700   * must be cleaned up by calling dma_buf_detach().
2904a8c1311f02 drivers/dma-buf/dma-buf.c Daniel Vetter      2016-12-09  701   *
85804b70cca68d drivers/dma-buf/dma-buf.c Daniel Vetter      2020-12-11  702   * Optionally this calls &dma_buf_ops.attach to allow device-specific attach
85804b70cca68d drivers/dma-buf/dma-buf.c Daniel Vetter      2020-12-11  703   * functionality.
85804b70cca68d drivers/dma-buf/dma-buf.c Daniel Vetter      2020-12-11  704   *
2904a8c1311f02 drivers/dma-buf/dma-buf.c Daniel Vetter      2016-12-09  705   * Returns:
2904a8c1311f02 drivers/dma-buf/dma-buf.c Daniel Vetter      2016-12-09  706   *
2904a8c1311f02 drivers/dma-buf/dma-buf.c Daniel Vetter      2016-12-09  707   * A pointer to newly created &dma_buf_attachment on success, or a negative
2904a8c1311f02 drivers/dma-buf/dma-buf.c Daniel Vetter      2016-12-09  708   * error code wrapped into a pointer on failure.
2904a8c1311f02 drivers/dma-buf/dma-buf.c Daniel Vetter      2016-12-09  709   *
2904a8c1311f02 drivers/dma-buf/dma-buf.c Daniel Vetter      2016-12-09  710   * Note that this can fail if the backing storage of @dmabuf is in a place not
2904a8c1311f02 drivers/dma-buf/dma-buf.c Daniel Vetter      2016-12-09  711   * accessible to @dev, and cannot be moved to a more suitable place. This is
2904a8c1311f02 drivers/dma-buf/dma-buf.c Daniel Vetter      2016-12-09  712   * indicated with the error code -EBUSY.
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  713   */
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  714  struct dma_buf_attachment *
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  715  dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  716  		       const struct dma_buf_attach_ops *importer_ops,
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  717  		       void *importer_priv)
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  718  {
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  719  	struct dma_buf_attachment *attach;
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  720  	int ret;
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  721  
d1aa06a1eaf5f7 drivers/base/dma-buf.c    Laurent Pinchart   2012-01-26  722  	if (WARN_ON(!dmabuf || !dev))
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  723  		return ERR_PTR(-EINVAL);
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  724  
4981cdb063e3e9 drivers/dma-buf/dma-buf.c Christian König    2020-02-19  725  	if (WARN_ON(importer_ops && !importer_ops->move_notify))
4981cdb063e3e9 drivers/dma-buf/dma-buf.c Christian König    2020-02-19  726  		return ERR_PTR(-EINVAL);
4981cdb063e3e9 drivers/dma-buf/dma-buf.c Christian König    2020-02-19  727  
db7942b6292306 drivers/dma-buf/dma-buf.c Markus Elfring     2017-05-08  728  	attach = kzalloc(sizeof(*attach), GFP_KERNEL);
34d84ec4881d13 drivers/dma-buf/dma-buf.c Markus Elfring     2017-05-08  729  	if (!attach)
a9fbc3b73127ef drivers/base/dma-buf.c    Laurent Pinchart   2012-01-26  730  		return ERR_PTR(-ENOMEM);
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  731  
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  732  	attach->dev = dev;
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  733  	attach->dmabuf = dmabuf;
09606b5446c25b drivers/dma-buf/dma-buf.c Christian König    2018-03-22  734  	if (importer_ops)
09606b5446c25b drivers/dma-buf/dma-buf.c Christian König    2018-03-22  735  		attach->peer2peer = importer_ops->allow_peer2peer;
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  736  	attach->importer_ops = importer_ops;
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  737  	attach->importer_priv = importer_priv;
2ed9201bdd9a8e drivers/base/dma-buf.c    Laurent Pinchart   2012-01-26  738  
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko    2022-05-27  739  	dma_resv_lock(dmabuf->resv, NULL);
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko    2022-05-27  740  
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  741  	if (dmabuf->ops->attach) {
a19741e5e5a9f1 drivers/dma-buf/dma-buf.c Christian König    2018-05-28  742  		ret = dmabuf->ops->attach(dmabuf, attach);
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  743  		if (ret)
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  744  			goto err_attach;
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  745  	}
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  746  	list_add(&attach->node, &dmabuf->attachments);
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  747  
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  748  	/* When either the importer or the exporter can't handle dynamic
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  749  	 * mappings we cache the mapping here to avoid issues with the
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  750  	 * reservation object lock.
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  751  	 */
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  752  	if (dma_buf_attachment_is_dynamic(attach) !=
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  753  	    dma_buf_is_dynamic(dmabuf)) {
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  754  		struct sg_table *sgt;
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  755  
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  756  		if (dma_buf_is_dynamic(attach->dmabuf)) {
7e008b02557cce drivers/dma-buf/dma-buf.c Christian König    2021-05-17  757  			ret = dmabuf->ops->pin(attach);
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  758  			if (ret)
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  759  				goto err_unlock;
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  760  		}
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  761  
84335675f2223c drivers/dma-buf/dma-buf.c Daniel Vetter      2021-01-15  762  		sgt = __map_dma_buf(attach, DMA_BIDIRECTIONAL);
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  763  		if (!sgt)
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  764  			sgt = ERR_PTR(-ENOMEM);
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  765  		if (IS_ERR(sgt)) {
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  766  			ret = PTR_ERR(sgt);
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  767  			goto err_unpin;
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  768  		}
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  769  		attach->sgt = sgt;
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  770  		attach->dir = DMA_BIDIRECTIONAL;
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  771  	}
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  772  
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko    2022-05-27  773  	dma_resv_unlock(dmabuf->resv);
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko    2022-05-27  774  
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  775  	return attach;
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  776  
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  777  err_attach:
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko    2022-05-27  778  	dma_resv_unlock(attach->dmabuf->resv);
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  779  	kfree(attach);
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  780  	return ERR_PTR(ret);
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  781  
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  782  err_unpin:
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  783  	if (dma_buf_is_dynamic(attach->dmabuf))
7e008b02557cce drivers/dma-buf/dma-buf.c Christian König    2021-05-17  784  		dmabuf->ops->unpin(attach);
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  785  
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  786  err_unlock:
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko    2022-05-27  787  	dma_resv_unlock(dmabuf->resv);
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  788  
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  789  	dma_buf_detach(dmabuf, attach);
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko    2022-05-27  790  
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03 @791  	return ERR_PTR(ret);
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  792  }
16b0314aa746be drivers/dma-buf/dma-buf.c Greg Kroah-Hartman 2021-10-10  793  EXPORT_SYMBOL_NS_GPL(dma_buf_dynamic_attach, DMA_BUF);
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  794  

-- 
0-DAY CI Kernel Test Service
https://01.org/lkp

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dmitry Osipenko]

On 5/27/22 05:37, kernel test robot wrote:
>>> drivers/dma-buf/dma-buf.c:1339:10: warning: variable 'ret' is uninitialized when used here [-Wuninitialized]
>                    return ret;
>                           ^~~
>    drivers/dma-buf/dma-buf.c:1331:9: note: initialize the variable 'ret' to silence this warning
>            int ret;
>                   ^
>                    = 0
>    1 warning generated.

Interestingly, GCC doesn't detect this problem and it's a valid warning.
I'll correct it in v7.

-- 
Best regards,
Dmitry

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dmitry Osipenko]

[-- Attachment #1: Type: text/plain, Size: 552 bytes --]

On 5/27/22 05:37, kernel test robot wrote:
>>> drivers/dma-buf/dma-buf.c:1339:10: warning: variable 'ret' is uninitialized when used here [-Wuninitialized]
>                    return ret;
>                           ^~~
>    drivers/dma-buf/dma-buf.c:1331:9: note: initialize the variable 'ret' to silence this warning
>            int ret;
>                   ^
>                    = 0
>    1 warning generated.

Interestingly, GCC doesn't detect this problem and it's a valid warning.
I'll correct it in v7.

-- 
Best regards,
Dmitry

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[kernel test robot]

[-- Attachment #1: Type: text/plain, Size: 7186 bytes --]

CC: kbuild-all(a)lists.01.org
BCC: lkp(a)intel.com
In-Reply-To: <20220526235040.678984-18-dmitry.osipenko@collabora.com>
References: <20220526235040.678984-18-dmitry.osipenko@collabora.com>
TO: Dmitry Osipenko <dmitry.osipenko@collabora.com>
TO: David Airlie <airlied@linux.ie>
TO: Gerd Hoffmann <kraxel@redhat.com>
TO: Gurchetan Singh <gurchetansingh@chromium.org>
TO: "Chia-I Wu" <olvaffe@gmail.com>
TO: Daniel Vetter <daniel@ffwll.ch>
TO: Daniel Almeida <daniel.almeida@collabora.com>
TO: Gert Wollny <gert.wollny@collabora.com>
TO: Gustavo Padovan <gustavo.padovan@collabora.com>
TO: Daniel Stone <daniel@fooishbar.org>
TO: Tomeu Vizoso <tomeu.vizoso@collabora.com>
TO: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
TO: Maxime Ripard <mripard@kernel.org>
TO: Thomas Zimmermann <tzimmermann@suse.de>
TO: Rob Herring <robh@kernel.org>
TO: Steven Price <steven.price@arm.com>
TO: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
TO: Rob Clark <robdclark@gmail.com>
TO: Emil Velikov <emil.l.velikov@gmail.com>
TO: Robin Murphy <robin.murphy@arm.com>
TO: Qiang Yu <yuq825@gmail.com>
TO: Sumit Semwal <sumit.semwal@linaro.org>
TO: "Christian König" <christian.koenig@amd.com>
TO: "Pan, Xinhui" <Xinhui.Pan@amd.com>
TO: Thierry Reding <thierry.reding@gmail.com>
TO: Tomasz Figa <tfiga@chromium.org>
TO: Marek Szyprowski <m.szyprowski@samsung.com>
TO: Mauro Carvalho Chehab <mchehab@kernel.org>
CC: linux-media(a)vger.kernel.org
TO: Alex Deucher <alexander.deucher@amd.com>
TO: Jani Nikula <jani.nikula@linux.intel.com>

Hi Dmitry,

I love your patch! Perhaps something to improve:

[auto build test WARNING on linus/master]
[also build test WARNING on next-20220527]
[cannot apply to drm/drm-next media-tree/master drm-intel/for-linux-next v5.18]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/intel-lab-lkp/linux/commits/Dmitry-Osipenko/Add-generic-memory-shrinker-to-VirtIO-GPU-and-Panfrost-DRM-drivers/20220527-075717
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git cdeffe87f790dfd1baa193020411ce9a538446d7
:::::: branch date: 14 hours ago
:::::: commit date: 14 hours ago
config: i386-randconfig-m021 (https://download.01.org/0day-ci/archive/20220527/202205272155.MxDiru43-lkp(a)intel.com/config)
compiler: gcc-11 (Debian 11.3.0-1) 11.3.0

If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>

New smatch warnings:
drivers/gpu/drm/drm_gem_shmem_helper.c:226 drm_gem_shmem_set_evictable() warn: inconsistent returns 'shmem->base.resv'.
drivers/gpu/drm/drm_gem_shmem_helper.c:253 drm_gem_shmem_set_purgeable() warn: inconsistent returns 'shmem->base.resv'.
drivers/gpu/drm/drm_gem_shmem_helper.c:703 drm_gem_shmem_fault() error: we previously assumed 'shmem->pages' could be null (see line 694)

Old smatch warnings:
include/drm/drm_vma_manager.h:225 drm_vma_node_unmap() warn: should 'drm_vma_node_size(node) << 12' be a 64 bit type?

vim +226 drivers/gpu/drm/drm_gem_shmem_helper.c

2194a63a818db7 Noralf Trønnes  2019-03-12  204  
875fd932de880a Dmitry Osipenko 2022-05-27  205  /**
875fd932de880a Dmitry Osipenko 2022-05-27  206   * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
875fd932de880a Dmitry Osipenko 2022-05-27  207   * @shmem: shmem GEM object
875fd932de880a Dmitry Osipenko 2022-05-27  208   *
875fd932de880a Dmitry Osipenko 2022-05-27  209   * Tell memory shrinker that this GEM can be evicted. Initially eviction is
875fd932de880a Dmitry Osipenko 2022-05-27  210   * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
875fd932de880a Dmitry Osipenko 2022-05-27  211   *
875fd932de880a Dmitry Osipenko 2022-05-27  212   * Returns:
875fd932de880a Dmitry Osipenko 2022-05-27  213   * 0 on success or a negative error code on failure.
875fd932de880a Dmitry Osipenko 2022-05-27  214   */
875fd932de880a Dmitry Osipenko 2022-05-27  215  int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
875fd932de880a Dmitry Osipenko 2022-05-27  216  {
875fd932de880a Dmitry Osipenko 2022-05-27  217  	dma_resv_lock(shmem->base.resv, NULL);
875fd932de880a Dmitry Osipenko 2022-05-27  218  
875fd932de880a Dmitry Osipenko 2022-05-27  219  	if (shmem->madv < 0)
875fd932de880a Dmitry Osipenko 2022-05-27  220  		return -ENOMEM;
875fd932de880a Dmitry Osipenko 2022-05-27  221  
875fd932de880a Dmitry Osipenko 2022-05-27  222  	shmem->eviction_enabled = true;
875fd932de880a Dmitry Osipenko 2022-05-27  223  
875fd932de880a Dmitry Osipenko 2022-05-27  224  	dma_resv_unlock(shmem->base.resv);
875fd932de880a Dmitry Osipenko 2022-05-27  225  
875fd932de880a Dmitry Osipenko 2022-05-27 @226  	return 0;
875fd932de880a Dmitry Osipenko 2022-05-27  227  }
875fd932de880a Dmitry Osipenko 2022-05-27  228  EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
875fd932de880a Dmitry Osipenko 2022-05-27  229  
875fd932de880a Dmitry Osipenko 2022-05-27  230  /**
875fd932de880a Dmitry Osipenko 2022-05-27  231   * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
875fd932de880a Dmitry Osipenko 2022-05-27  232   * @shmem: shmem GEM object
875fd932de880a Dmitry Osipenko 2022-05-27  233   *
875fd932de880a Dmitry Osipenko 2022-05-27  234   * Tell memory shrinker that this GEM can be purged. Initially purging is
875fd932de880a Dmitry Osipenko 2022-05-27  235   * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
875fd932de880a Dmitry Osipenko 2022-05-27  236   *
875fd932de880a Dmitry Osipenko 2022-05-27  237   * Returns:
875fd932de880a Dmitry Osipenko 2022-05-27  238   * 0 on success or a negative error code on failure.
875fd932de880a Dmitry Osipenko 2022-05-27  239   */
875fd932de880a Dmitry Osipenko 2022-05-27  240  int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
875fd932de880a Dmitry Osipenko 2022-05-27  241  {
875fd932de880a Dmitry Osipenko 2022-05-27  242  	dma_resv_lock(shmem->base.resv, NULL);
875fd932de880a Dmitry Osipenko 2022-05-27  243  
875fd932de880a Dmitry Osipenko 2022-05-27  244  	if (shmem->madv < 0)
875fd932de880a Dmitry Osipenko 2022-05-27  245  		return -ENOMEM;
875fd932de880a Dmitry Osipenko 2022-05-27  246  
875fd932de880a Dmitry Osipenko 2022-05-27  247  	shmem->purge_enabled = true;
875fd932de880a Dmitry Osipenko 2022-05-27  248  
875fd932de880a Dmitry Osipenko 2022-05-27  249  	drm_gem_shmem_update_pages_state(shmem);
875fd932de880a Dmitry Osipenko 2022-05-27  250  
875fd932de880a Dmitry Osipenko 2022-05-27  251  	dma_resv_unlock(shmem->base.resv);
875fd932de880a Dmitry Osipenko 2022-05-27  252  
875fd932de880a Dmitry Osipenko 2022-05-27 @253  	return 0;
875fd932de880a Dmitry Osipenko 2022-05-27  254  }
875fd932de880a Dmitry Osipenko 2022-05-27  255  EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
875fd932de880a Dmitry Osipenko 2022-05-27  256  

-- 
0-DAY CI Kernel Test Service
https://01.org/lkp

Re: [PATCH v6 04/22] drm/panfrost: Fix shrinker list corruption by madvise IOCTL

[Alyssa Rosenzweig]

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

On Fri, May 27, 2022 at 02:50:22AM +0300, Dmitry Osipenko wrote:
> Calling madvise IOCTL twice on BO causes memory shrinker list corruption
> and crashes kernel because BO is already on the list and it's added to
> the list again, while BO should be removed from from the list before it's
> re-added. Fix it.
> 
> Cc: stable@vger.kernel.org
> Fixes: 013b65101315 ("drm/panfrost: Add madvise and shrinker support")
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/panfrost/panfrost_drv.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
> index 087e69b98d06..b1e6d238674f 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_drv.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
> @@ -433,8 +433,8 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
>  
>  	if (args->retained) {
>  		if (args->madv == PANFROST_MADV_DONTNEED)
> -			list_add_tail(&bo->base.madv_list,
> -				      &pfdev->shrinker_list);
> +			list_move_tail(&bo->base.madv_list,
> +				       &pfdev->shrinker_list);
>  		else if (args->madv == PANFROST_MADV_WILLNEED)
>  			list_del_init(&bo->base.madv_list);
>  	}
> -- 
> 2.35.3
> 

Re: [PATCH v6 04/22] drm/panfrost: Fix shrinker list corruption by madvise IOCTL

[Alyssa Rosenzweig]

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

On Fri, May 27, 2022 at 02:50:22AM +0300, Dmitry Osipenko wrote:
> Calling madvise IOCTL twice on BO causes memory shrinker list corruption
> and crashes kernel because BO is already on the list and it's added to
> the list again, while BO should be removed from from the list before it's
> re-added. Fix it.
> 
> Cc: stable@vger.kernel.org
> Fixes: 013b65101315 ("drm/panfrost: Add madvise and shrinker support")
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/panfrost/panfrost_drv.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
> index 087e69b98d06..b1e6d238674f 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_drv.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
> @@ -433,8 +433,8 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
>  
>  	if (args->retained) {
>  		if (args->madv == PANFROST_MADV_DONTNEED)
> -			list_add_tail(&bo->base.madv_list,
> -				      &pfdev->shrinker_list);
> +			list_move_tail(&bo->base.madv_list,
> +				       &pfdev->shrinker_list);
>  		else if (args->madv == PANFROST_MADV_WILLNEED)
>  			list_del_init(&bo->base.madv_list);
>  	}
> -- 
> 2.35.3
> 

Re: [PATCH v6 04/22] drm/panfrost: Fix shrinker list corruption by madvise IOCTL

[Alyssa Rosenzweig]

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

On Fri, May 27, 2022 at 02:50:22AM +0300, Dmitry Osipenko wrote:
> Calling madvise IOCTL twice on BO causes memory shrinker list corruption
> and crashes kernel because BO is already on the list and it's added to
> the list again, while BO should be removed from from the list before it's
> re-added. Fix it.
> 
> Cc: stable@vger.kernel.org
> Fixes: 013b65101315 ("drm/panfrost: Add madvise and shrinker support")
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/panfrost/panfrost_drv.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
> index 087e69b98d06..b1e6d238674f 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_drv.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
> @@ -433,8 +433,8 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
>  
>  	if (args->retained) {
>  		if (args->madv == PANFROST_MADV_DONTNEED)
> -			list_add_tail(&bo->base.madv_list,
> -				      &pfdev->shrinker_list);
> +			list_move_tail(&bo->base.madv_list,
> +				       &pfdev->shrinker_list);
>  		else if (args->madv == PANFROST_MADV_WILLNEED)
>  			list_del_init(&bo->base.madv_list);
>  	}
> -- 
> 2.35.3
> 

Re: [Intel-gfx] [PATCH v6 04/22] drm/panfrost: Fix shrinker list corruption by madvise IOCTL

[Alyssa Rosenzweig]

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>

On Fri, May 27, 2022 at 02:50:22AM +0300, Dmitry Osipenko wrote:
> Calling madvise IOCTL twice on BO causes memory shrinker list corruption
> and crashes kernel because BO is already on the list and it's added to
> the list again, while BO should be removed from from the list before it's
> re-added. Fix it.
> 
> Cc: stable@vger.kernel.org
> Fixes: 013b65101315 ("drm/panfrost: Add madvise and shrinker support")
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/panfrost/panfrost_drv.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
> index 087e69b98d06..b1e6d238674f 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_drv.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
> @@ -433,8 +433,8 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
>  
>  	if (args->retained) {
>  		if (args->madv == PANFROST_MADV_DONTNEED)
> -			list_add_tail(&bo->base.madv_list,
> -				      &pfdev->shrinker_list);
> +			list_move_tail(&bo->base.madv_list,
> +				       &pfdev->shrinker_list);
>  		else if (args->madv == PANFROST_MADV_WILLNEED)
>  			list_del_init(&bo->base.madv_list);
>  	}
> -- 
> 2.35.3
> 

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Christian König via Virtualization]

Hi Dmitry,

First of all please separate out this patch from the rest of the series, 
since this is a complex separate structural change.

Am 27.05.22 um 01:50 schrieb Dmitry Osipenko:
> All dma-bufs have dma-reservation lock that allows drivers to perform
> exclusive operations over shared dma-bufs. Today's dma-buf API has
> incomplete locking specification, which creates dead lock situation
> for dma-buf importers and exporters that don't coordinate theirs locks.

Well please drop that sentence. The locking specifications are actually 
very well defined, it's just that some drivers are a bit broken 
regarding them.

What you do here is rather moving all the non-dynamic drivers over to 
the dynamic locking specification (which is really nice to have).

I have tried this before and failed because catching all the locks in 
the right code paths are very tricky. So expect some fallout from this 
and make sure the kernel test robot and CI systems are clean.

> This patch introduces new locking convention for dma-buf users. From now
> on all dma-buf importers are responsible for holding dma-buf reservation
> lock around operations performed over dma-bufs.
>
> This patch implements the new dma-buf locking convention by:
>
>    1. Making dma-buf API functions to take the reservation lock.
>
>    2. Adding new locked variants of the dma-buf API functions for drivers
>       that need to manage imported dma-bufs under the held lock.

Instead of adding new locked variants please mark all variants which 
expect to be called without a lock with an _unlocked postfix.

This should make it easier to remove those in a follow up patch set and 
then fully move the locking into the importer.

>    3. Converting all drivers to the new locking scheme.

I have strong doubts that you got all of them. At least radeon and 
nouveau should grab the reservation lock in their ->attach callbacks 
somehow.

>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>   drivers/dma-buf/dma-buf.c                     | 270 +++++++++++-------
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
>   drivers/gpu/drm/drm_client.c                  |   4 +-
>   drivers/gpu/drm/drm_gem.c                     |  33 +++
>   drivers/gpu/drm/drm_gem_framebuffer_helper.c  |   6 +-
>   drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    |  10 +-
>   drivers/gpu/drm/qxl/qxl_object.c              |  17 +-
>   drivers/gpu/drm/qxl/qxl_prime.c               |   4 +-
>   .../common/videobuf2/videobuf2-dma-contig.c   |  11 +-
>   .../media/common/videobuf2/videobuf2-dma-sg.c |  11 +-
>   .../common/videobuf2/videobuf2-vmalloc.c      |  11 +-
>   include/drm/drm_gem.h                         |   3 +
>   include/linux/dma-buf.h                       |  14 +-
>   13 files changed, 241 insertions(+), 159 deletions(-)
>
> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
> index 32f55640890c..64a9909ccfa2 100644
> --- a/drivers/dma-buf/dma-buf.c
> +++ b/drivers/dma-buf/dma-buf.c
> @@ -552,7 +552,6 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)
>   	file->f_mode |= FMODE_LSEEK;
>   	dmabuf->file = file;
>   
> -	mutex_init(&dmabuf->lock);

Please make removing dmabuf->lock a separate change.

Regards,
Christian.

>   	INIT_LIST_HEAD(&dmabuf->attachments);
>   
>   	mutex_lock(&db_list.lock);
> @@ -737,14 +736,14 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   	attach->importer_ops = importer_ops;
>   	attach->importer_priv = importer_priv;  3. Converting all drivers to the new locking scheme.
>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>   drivers/dma-buf/dma-buf.c                     | 270 +++++++++++-------
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
>   drivers/gpu/drm/drm_client.c                  |   4 +-
>
>   
> +	dma_resv_lock(dmabuf->resv, NULL);
> +
>   	if (dmabuf->ops->attach) {
>   		ret = dmabuf->ops->attach(dmabuf, attach);
>   		if (ret)
>   			goto err_attach;
>   	}
> -	dma_resv_lock(dmabuf->resv, NULL);
>   	list_add(&attach->node, &dmabuf->attachments);
> -	dma_resv_unlock(dmabuf->resv);
>   
>   	/* When either the importer or the exporter can't handle dynamic
>   	 * mappings we cache the mapping here to avoid issues with the
> @@ -755,7 +754,6 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   		struct sg_table *sgt;
>   
>   		if (dma_buf_is_dynamic(attach->dmabuf)) {
> -			dma_resv_lock(attach->dmabuf->resv, NULL);
>   			ret = dmabuf->ops->pin(attach);
>   			if (ret)
>   				goto err_unlock;
> @@ -768,15 +766,16 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   			ret = PTR_ERR(sgt);
>   			goto err_unpin;
>   		}
> -		if (dma_buf_is_dynamic(attach->dmabuf))
> -			dma_resv_unlock(attach->dmabuf->resv);
>   		attach->sgt = sgt;
>   		attach->dir = DMA_BIDIRECTIONAL;
>   	}
>   
> +	dma_resv_unlock(dmabuf->resv);
> +
>   	return attach;
>   
>   err_attach:
> +	dma_resv_unlock(attach->dmabuf->resv);
>   	kfree(attach);
>   	return ERR_PTR(ret);
>   
> @@ -785,10 +784,10 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   		dmabuf->ops->unpin(attach);
>   
>   err_unlock:
> -	if (dma_buf_is_dynamic(attach->dmabuf))
> -		dma_resv_unlock(attach->dmabuf->resv);
> +	dma_resv_unlock(dmabuf->resv);
>   
>   	dma_buf_detach(dmabuf, attach);
> +
>   	return ERR_PTR(ret);
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_dynamic_attach, DMA_BUF);
> @@ -832,24 +831,23 @@ void dma_buf_detach(struct dma_buf *dmabuf, struct dma_buf_attachment *attach)
>   	if (WARN_ON(!dmabuf || !attach))
>   		return;
>   
> -	if (attach->sgt) {
> -		if (dma_buf_is_dynamic(attach->dmabuf))
> -			dma_resv_lock(attach->dmabuf->resv, NULL);
> +	if (WARN_ON(dmabuf != attach->dmabuf))
> +		return;
>   
> +	dma_resv_lock(dmabuf->resv, NULL);
> +
> +	if (attach->sgt) {
>   		__unmap_dma_buf(attach, attach->sgt, attach->dir);
>   
> -		if (dma_buf_is_dynamic(attach->dmabuf)) {
> +		if (dma_buf_is_dynamic(attach->dmabuf))
>   			dmabuf->ops->unpin(attach);
> -			dma_resv_unlock(attach->dmabuf->resv);
> -		}
>   	}
>   
> -	dma_resv_lock(dmabuf->resv, NULL);
>   	list_del(&attach->node);
> -	dma_resv_unlock(dmabuf->resv);
>   	if (dmabuf->ops->detach)
>   		dmabuf->ops->detach(dmabuf, attach);
>   
> +	dma_resv_unlock(dmabuf->resv);
>   	kfree(attach);
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_detach, DMA_BUF);
> @@ -906,28 +904,18 @@ void dma_buf_unpin(struct dma_buf_attachment *attach)
>   EXPORT_SYMBOL_NS_GPL(dma_buf_unpin, DMA_BUF);
>   
>   /**
> - * dma_buf_map_attachment - Returns the scatterlist table of the attachment;
> + * dma_buf_map_attachment_locked - Returns the scatterlist table of the attachment;
>    * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
>    * dma_buf_ops.
>    * @attach:	[in]	attachment whose scatterlist is to be returned
>    * @direction:	[in]	direction of DMA transfer
>    *
> - * Returns sg_table containing the scatterlist to be returned; returns ERR_PTR
> - * on error. May return -EINTR if it is interrupted by a signal.
> - *
> - * On success, the DMA addresses and lengths in the returned scatterlist are
> - * PAGE_SIZE aligned.
> - *
> - * A mapping must be unmapped by using dma_buf_unmap_attachment(). Note that
> - * the underlying backing storage is pinned for as long as a mapping exists,
> - * therefore users/importers should not hold onto a mapping for undue amounts of
> - * time.
> + * Locked variant of dma_buf_map_attachment().
>    *
> - * Important: Dynamic importers must wait for the exclusive fence of the struct
> - * dma_resv attached to the DMA-BUF first.
> + * Caller is responsible for holding dmabuf's reservation lock.
>    */
> -struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
> -					enum dma_data_direction direction)
> +struct sg_table *dma_buf_map_attachment_locked(struct dma_buf_attachment *attach,
> +					       enum dma_data_direction direction)
>   {
>   	struct sg_table *sg_table;
>   	int r;
> @@ -937,8 +925,7 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   	if (WARN_ON(!attach || !attach->dmabuf))
>   		return ERR_PTR(-EINVAL);
>   
> -	if (dma_buf_attachment_is_dynamic(attach))
> -		dma_resv_assert_held(attach->dmabuf->resv);
> +	dma_resv_assert_held(attach->dmabuf->resv);
>   
>   	if (attach->sgt) {
>   		/*
> @@ -953,7 +940,6 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   	}
>   
>   	if (dma_buf_is_dynamic(attach->dmabuf)) {
> -		dma_resv_assert_held(attach->dmabuf->resv);
>   		if (!IS_ENABLED(CONFIG_DMABUF_MOVE_NOTIFY)) {
>   			r = attach->dmabuf->ops->pin(attach);
>   			if (r)
> @@ -993,42 +979,101 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   #endif /* CONFIG_DMA_API_DEBUG */
>   	return sg_table;
>   }
> -EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment, DMA_BUF);
> +EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment_locked, DMA_BUF);
>   
>   /**
> - * dma_buf_unmap_attachment - unmaps and decreases usecount of the buffer;might
> - * deallocate the scatterlist associated. Is a wrapper for unmap_dma_buf() of
> + * dma_buf_map_attachment - Returns the scatterlist table of the attachment;
> + * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
>    * dma_buf_ops.
> - * @attach:	[in]	attachment to unmap buffer from
> - * @sg_table:	[in]	scatterlist info of the buffer to unmap
> - * @direction:  [in]    direction of DMA transfer
> + * @attach:	[in]	attachment whose scatterlist is to be returned
> + * @direction:	[in]	direction of DMA transfer
>    *
> - * This unmaps a DMA mapping for @attached obtained by dma_buf_map_attachment().
> + * Returns sg_table containing the scatterlist to be returned; returns ERR_PTR
> + * on error. May return -EINTR if it is interrupted by a signal.
> + *
> + * On success, the DMA addresses and lengths in the returned scatterlist are
> + * PAGE_SIZE aligned.
> + *
> + * A mapping must be unmapped by using dma_buf_unmap_attachment(). Note that
> + * the underlying backing storage is pinned for as long as a mapping exists,
> + * therefore users/importers should not hold onto a mapping for undue amounts of
> + * time.
> + *
> + * Important: Dynamic importers must wait for the exclusive fence of the struct
> + * dma_resv attached to the DMA-BUF first.
>    */
> -void dma_buf_unmap_attachment(struct dma_buf_attachment *attach,
> -				struct sg_table *sg_table,
> +struct sg_table *
> +dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   				enum dma_data_direction direction)
>   {
> +	struct sg_table *sg_table;
> +
>   	might_sleep();
>   
> -	if (WARN_ON(!attach || !attach->dmabuf || !sg_table))
> -		return;
> +	if (WARN_ON(!attach || !attach->dmabuf))
> +		return ERR_PTR(-EINVAL);
> +
> +	dma_resv_lock(attach->dmabuf->resv, NULL);
> +	sg_table = dma_buf_map_attachment_locked(attach, direction);
> +	dma_resv_unlock(attach->dmabuf->resv);
>   
> -	if (dma_buf_attachment_is_dynamic(attach))
> -		dma_resv_assert_held(attach->dmabuf->resv);
> +	return sg_table;
> +}
> +EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment, DMA_BUF);
> +
> +/**
> + * dma_buf_unmap_attachment_locked - Returns the scatterlist table of the attachment;
> + * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
> + * dma_buf_ops.
> + * @attach:	[in]	attachment whose scatterlist is to be returned
> + * @direction:	[in]	direction of DMA transfer
> + *
> + * Locked variant of dma_buf_unmap_attachment().
> + *
> + * Caller is responsible for holding dmabuf's reservation lock.
> + */
> +void dma_buf_unmap_attachment_locked(struct dma_buf_attachment *attach,
> +				     struct sg_table *sg_table,
> +				     enum dma_data_direction direction)
> +{
> +	might_sleep();
> +
> +	dma_resv_assert_held(attach->dmabuf->resv);
>   
>   	if (attach->sgt == sg_table)
>   		return;
>   
> -	if (dma_buf_is_dynamic(attach->dmabuf))
> -		dma_resv_assert_held(attach->dmabuf->resv);
> -
>   	__unmap_dma_buf(attach, sg_table, direction);
>   
>   	if (dma_buf_is_dynamic(attach->dmabuf) &&
>   	    !IS_ENABLED(CONFIG_DMABUF_MOVE_NOTIFY))
>   		dma_buf_unpin(attach);
>   }
> +EXPORT_SYMBOL_NS_GPL(dma_buf_unmap_attachment_locked, DMA_BUF);
> +
> +/**
> + * dma_buf_unmap_attachment - unmaps and decreases usecount of the buffer;might
> + * deallocate the scatterlist associated. Is a wrapper for unmap_dma_buf() of
> + * dma_buf_ops.
> + * @attach:	[in]	attachment to unmap buffer from
> + * @sg_table:	[in]	scatterlist info of the buffer to unmap
> + * @direction:  [in]    direction of DMA transfer
> + *
> + * This unmaps a DMA mapping for @attached obtained by dma_buf_map_attachment().
> + */
> +void dma_buf_unmap_attachment(struct dma_buf_attachment *attach,
> +			      struct sg_table *sg_table,
> +			      enum dma_data_direction direction)
> +{
> +	might_sleep();
> +
> +	if (WARN_ON(!attach || !attach->dmabuf || !sg_table))
> +		return;
> +
> +	dma_resv_lock(attach->dmabuf->resv, NULL);
> +	dma_buf_unmap_attachment_locked(attach, sg_table, direction);
> +	dma_resv_unlock(attach->dmabuf->resv);
> +}
>   EXPORT_SYMBOL_NS_GPL(dma_buf_unmap_attachment, DMA_BUF);
>   
>   /**
> @@ -1224,6 +1269,31 @@ int dma_buf_end_cpu_access(struct dma_buf *dmabuf,
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_end_cpu_access, DMA_BUF);
>   
> +static int dma_buf_mmap_locked(struct dma_buf *dmabuf,
> +			       struct vm_area_struct *vma,
> +			       unsigned long pgoff)
> +{
> +	dma_resv_assert_held(dmabuf->resv);
> +
> +	/* check if buffer supports mmap */
> +	if (!dmabuf->ops->mmap)
> +		return -EINVAL;
> +
> +	/* check for offset overflow */
> +	if (pgoff + vma_pages(vma) < pgoff)
> +		return -EOVERFLOW;
> +
> +	/* check for overflowing the buffer's size */
> +	if (pgoff + vma_pages(vma) >
> +	    dmabuf->size >> PAGE_SHIFT)
> +		return -EINVAL;
> +
> +	/* readjust the vma */
> +	vma_set_file(vma, dmabuf->file);
> +	vma->vm_pgoff = pgoff;
> +
> +	return dmabuf->ops->mmap(dmabuf, vma);
> +}
>   
>   /**
>    * dma_buf_mmap - Setup up a userspace mmap with the given vma
> @@ -1242,29 +1312,46 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_end_cpu_access, DMA_BUF);
>   int dma_buf_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma,
>   		 unsigned long pgoff)
>   {
> +	int ret;
> +
>   	if (WARN_ON(!dmabuf || !vma))
>   		return -EINVAL;
>   
> -	/* check if buffer supports mmap */
> -	if (!dmabuf->ops->mmap)
> -		return -EINVAL;
> +	dma_resv_lock(dmabuf->resv, NULL);
> +	ret = dma_buf_mmap_locked(dmabuf, vma, pgoff);
> +	dma_resv_unlock(dmabuf->resv);
>   
> -	/* check for offset overflow */
> -	if (pgoff + vma_pages(vma) < pgoff)
> -		return -EOVERFLOW;
> +	return ret;
> +}
> +EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
>   
> -	/* check for overflowing the buffer's size */
> -	if (pgoff + vma_pages(vma) >
> -	    dmabuf->size >> PAGE_SHIFT)
> -		return -EINVAL;
> +static int dma_buf_vmap_locked(struct dma_buf *dmabuf, struct iosys_map *map)
> +{
> +	struct iosys_map ptr;
> +	int ret;
>   
> -	/* readjust the vma */
> -	vma_set_file(vma, dmabuf->file);
> -	vma->vm_pgoff = pgoff;
> +	dma_resv_assert_held(dmabuf->resv);
>   
> -	return dmabuf->ops->mmap(dmabuf, vma);
> +	if (dmabuf->vmapping_counter) {
> +		dmabuf->vmapping_counter++;
> +		BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
> +		*map = dmabuf->vmap_ptr;
> +		return ret;
> +	}
> +
> +	BUG_ON(iosys_map_is_set(&dmabuf->vmap_ptr));
> +
> +	ret = dmabuf->ops->vmap(dmabuf, &ptr);
> +	if (WARN_ON_ONCE(ret))
> +		return ret;
> +
> +	dmabuf->vmap_ptr = ptr;
> +	dmabuf->vmapping_counter = 1;
> +
> +	*map = dmabuf->vmap_ptr;
> +
> +	return 0;
>   }
> -EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
>   
>   /**
>    * dma_buf_vmap - Create virtual mapping for the buffer object into kernel
> @@ -1284,8 +1371,7 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
>    */
>   int dma_buf_vmap(struct dma_buf *dmabuf, struct iosys_map *map)
>   {
> -	struct iosys_map ptr;
> -	int ret = 0;
> +	int ret;
>   
>   	iosys_map_clear(map);
>   
> @@ -1295,52 +1381,40 @@ int dma_buf_vmap(struct dma_buf *dmabuf, struct iosys_map *map)
>   	if (!dmabuf->ops->vmap)
>   		return -EINVAL;
>   
> -	mutex_lock(&dmabuf->lock);
> -	if (dmabuf->vmapping_counter) {
> -		dmabuf->vmapping_counter++;
> -		BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
> -		*map = dmabuf->vmap_ptr;
> -		goto out_unlock;
> -	}
> -
> -	BUG_ON(iosys_map_is_set(&dmabuf->vmap_ptr));
> -
> -	ret = dmabuf->ops->vmap(dmabuf, &ptr);
> -	if (WARN_ON_ONCE(ret))
> -		goto out_unlock;
> -
> -	dmabuf->vmap_ptr = ptr;
> -	dmabuf->vmapping_counter = 1;
> -
> -	*map = dmabuf->vmap_ptr;
> +	dma_resv_lock(dmabuf->resv, NULL);
> +	ret = dma_buf_vmap_locked(dmabuf, map);
> +	dma_resv_unlock(dmabuf->resv);
>   
> -out_unlock:
> -	mutex_unlock(&dmabuf->lock);
>   	return ret;
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_vmap, DMA_BUF);
>   
> -/**
> - * dma_buf_vunmap - Unmap a vmap obtained by dma_buf_vmap.
> - * @dmabuf:	[in]	buffer to vunmap
> - * @map:	[in]	vmap pointer to vunmap
> - */
> -void dma_buf_vunmap(struct dma_buf *dmabuf, struct iosys_map *map)
> +static void dma_buf_vunmap_locked(struct dma_buf *dmabuf, struct iosys_map *map)
>   {
> -	if (WARN_ON(!dmabuf))
> -		return;
> -
>   	BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
>   	BUG_ON(dmabuf->vmapping_counter == 0);
>   	BUG_ON(!iosys_map_is_equal(&dmabuf->vmap_ptr, map));
>   
> -	mutex_lock(&dmabuf->lock);
>   	if (--dmabuf->vmapping_counter == 0) {
>   		if (dmabuf->ops->vunmap)
>   			dmabuf->ops->vunmap(dmabuf, map);
>   		iosys_map_clear(&dmabuf->vmap_ptr);
>   	}
> -	mutex_unlock(&dmabuf->lock);
> +}
> +
> +/**
> + * dma_buf_vunmap - Unmap a vmap obtained by dma_buf_vmap.
> + * @dmabuf:	[in]	buffer to vunmap
> + * @map:	[in]	vmap pointer to vunmap
> + */
> +void dma_buf_vunmap(struct dma_buf *dmabuf, struct iosys_map *map)
> +{
> +	if (WARN_ON(!dmabuf))
> +		return;
> +
> +	dma_resv_lock(dmabuf->resv, NULL);
> +	dma_buf_vunmap_locked(dmabuf, map);
> +	dma_resv_unlock(dmabuf->resv);
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_vunmap, DMA_BUF);
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> index be6f76a30ac6..b704bdf5601a 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> @@ -882,7 +882,8 @@ static int amdgpu_ttm_backend_bind(struct ttm_device *bdev,
>   			struct sg_table *sgt;
>   
>   			attach = gtt->gobj->import_attach;
> -			sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL);
> +			sgt = dma_buf_map_attachment_locked(attach,
> +							    DMA_BIDIRECTIONAL);
>   			if (IS_ERR(sgt))
>   				return PTR_ERR(sgt);
>   
> @@ -1007,7 +1008,8 @@ static void amdgpu_ttm_backend_unbind(struct ttm_device *bdev,
>   		struct dma_buf_attachment *attach;
>   
>   		attach = gtt->gobj->import_attach;
> -		dma_buf_unmap_attachment(attach, ttm->sg, DMA_BIDIRECTIONAL);
> +		dma_buf_unmap_attachment_locked(attach, ttm->sg,
> +						DMA_BIDIRECTIONAL);
>   		ttm->sg = NULL;
>   	}
>   
> diff --git a/drivers/gpu/drm/drm_client.c b/drivers/gpu/drm/drm_client.c
> index af3b7395bf69..e9a1cd310352 100644
> --- a/drivers/gpu/drm/drm_client.c
> +++ b/drivers/gpu/drm/drm_client.c
> @@ -323,7 +323,7 @@ drm_client_buffer_vmap(struct drm_client_buffer *buffer,
>   	 * fd_install step out of the driver backend hooks, to make that
>   	 * final step optional for internal users.
>   	 */
> -	ret = drm_gem_vmap(buffer->gem, map);
> +	ret = drm_gem_vmap_unlocked(buffer->gem, map);
>   	if (ret)
>   		return ret;
>   
> @@ -345,7 +345,7 @@ void drm_client_buffer_vunmap(struct drm_client_buffer *buffer)
>   {
>   	struct iosys_map *map = &buffer->map;
>   
> -	drm_gem_vunmap(buffer->gem, map);
> +	drm_gem_vunmap_unlocked(buffer->gem, map);
>   }
>   EXPORT_SYMBOL(drm_client_buffer_vunmap);
>   
> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
> index 7c0b025508e4..c61674887582 100644
> --- a/drivers/gpu/drm/drm_gem.c
> +++ b/drivers/gpu/drm/drm_gem.c
> @@ -1053,7 +1053,12 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,
>   	vma->vm_ops = obj->funcs->vm_ops;
>   
>   	if (obj->funcs->mmap) {
> +		ret = dma_resv_lock_interruptible(obj->resv, NULL);
> +		if (ret)
> +			goto err_drm_gem_object_put;
> +
>   		ret = obj->funcs->mmap(obj, vma);
> +		dma_resv_unlock(obj->resv);
>   		if (ret)
>   			goto err_drm_gem_object_put;
>   		WARN_ON(!(vma->vm_flags & VM_DONTEXPAND));
> @@ -1158,6 +1163,8 @@ void drm_gem_print_info(struct drm_printer *p, unsigned int indent,
>   
>   int drm_gem_pin(struct drm_gem_object *obj)
>   {
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (obj->funcs->pin)
>   		return obj->funcs->pin(obj);
>   	else
> @@ -1166,6 +1173,8 @@ int drm_gem_pin(struct drm_gem_object *obj)
>   
>   void drm_gem_unpin(struct drm_gem_object *obj)
>   {
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (obj->funcs->unpin)
>   		obj->funcs->unpin(obj);
>   }
> @@ -1174,6 +1183,8 @@ int drm_gem_vmap(struct drm_gem_object *obj, struct iosys_map *map)
>   {
>   	int ret;
>   
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (!obj->funcs->vmap)
>   		return -EOPNOTSUPP;
>   
> @@ -1189,6 +1200,8 @@ EXPORT_SYMBOL(drm_gem_vmap);
>   
>   void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
>   {
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (iosys_map_is_null(map))
>   		return;
>   
> @@ -1200,6 +1213,26 @@ void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
>   }
>   EXPORT_SYMBOL(drm_gem_vunmap);
>   
> +int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
> +{
> +	int ret;
> +
> +	dma_resv_lock(obj->resv, NULL);
> +	ret = drm_gem_vmap(obj, map);
> +	dma_resv_unlock(obj->resv);
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL(drm_gem_vmap_unlocked);
> +
> +void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
> +{
> +	dma_resv_lock(obj->resv, NULL);
> +	drm_gem_vunmap(obj, map);
> +	dma_resv_unlock(obj->resv);
> +}
> +EXPORT_SYMBOL(drm_gem_vunmap_unlocked);
> +
>   /**
>    * drm_gem_lock_reservations - Sets up the ww context and acquires
>    * the lock on an array of GEM objects.
> diff --git a/drivers/gpu/drm/drm_gem_framebuffer_helper.c b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
> index f4619803acd0..a0bff53b158e 100644
> --- a/drivers/gpu/drm/drm_gem_framebuffer_helper.c
> +++ b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
> @@ -348,7 +348,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb,
>   			iosys_map_clear(&map[i]);
>   			continue;
>   		}
> -		ret = drm_gem_vmap(obj, &map[i]);
> +		ret = drm_gem_vmap_unlocked(obj, &map[i]);
>   		if (ret)
>   			goto err_drm_gem_vunmap;
>   	}
> @@ -370,7 +370,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb,
>   		obj = drm_gem_fb_get_obj(fb, i);
>   		if (!obj)
>   			continue;
> -		drm_gem_vunmap(obj, &map[i]);
> +		drm_gem_vunmap_unlocked(obj, &map[i]);
>   	}
>   	return ret;
>   }
> @@ -398,7 +398,7 @@ void drm_gem_fb_vunmap(struct drm_framebuffer *fb,
>   			continue;
>   		if (iosys_map_is_null(&map[i]))
>   			continue;
> -		drm_gem_vunmap(obj, &map[i]);
> +		drm_gem_vunmap_unlocked(obj, &map[i]);
>   	}
>   }
>   EXPORT_SYMBOL(drm_gem_fb_vunmap);
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
> index f5062d0c6333..09502d490da8 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
> @@ -72,7 +72,7 @@ static int i915_gem_dmabuf_vmap(struct dma_buf *dma_buf,
>   	struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);
>   	void *vaddr;
>   
> -	vaddr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
> +	vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
>   	if (IS_ERR(vaddr))
>   		return PTR_ERR(vaddr);
>   
> @@ -241,8 +241,8 @@ static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
>   
>   	assert_object_held(obj);
>   
> -	pages = dma_buf_map_attachment(obj->base.import_attach,
> -				       DMA_BIDIRECTIONAL);
> +	pages = dma_buf_map_attachment_locked(obj->base.import_attach,
> +					      DMA_BIDIRECTIONAL);
>   	if (IS_ERR(pages))
>   		return PTR_ERR(pages);
>   
> @@ -270,8 +270,8 @@ static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
>   static void i915_gem_object_put_pages_dmabuf(struct drm_i915_gem_object *obj,
>   					     struct sg_table *pages)
>   {
> -	dma_buf_unmap_attachment(obj->base.import_attach, pages,
> -				 DMA_BIDIRECTIONAL);
> +	dma_buf_unmap_attachment_locked(obj->base.import_attach, pages,
> +					DMA_BIDIRECTIONAL);
>   }
>   
>   static const struct drm_i915_gem_object_ops i915_gem_object_dmabuf_ops = {
> diff --git a/drivers/gpu/drm/qxl/qxl_object.c b/drivers/gpu/drm/qxl/qxl_object.c
> index b42a657e4c2f..a64cd635fbc0 100644
> --- a/drivers/gpu/drm/qxl/qxl_object.c
> +++ b/drivers/gpu/drm/qxl/qxl_object.c
> @@ -168,9 +168,16 @@ int qxl_bo_vmap_locked(struct qxl_bo *bo, struct iosys_map *map)
>   		bo->map_count++;
>   		goto out;
>   	}
> -	r = ttm_bo_vmap(&bo->tbo, &bo->map);
> +
> +	r = __qxl_bo_pin(bo);
>   	if (r)
>   		return r;
> +
> +	r = ttm_bo_vmap(&bo->tbo, &bo->map);
> +	if (r) {
> +		__qxl_bo_unpin(bo);
> +		return r;
> +	}
>   	bo->map_count = 1;
>   
>   	/* TODO: Remove kptr in favor of map everywhere. */
> @@ -192,12 +199,6 @@ int qxl_bo_vmap(struct qxl_bo *bo, struct iosys_map *map)
>   	if (r)
>   		return r;
>   
> -	r = __qxl_bo_pin(bo);
> -	if (r) {
> -		qxl_bo_unreserve(bo);
> -		return r;
> -	}
> -
>   	r = qxl_bo_vmap_locked(bo, map);
>   	qxl_bo_unreserve(bo);
>   	return r;
> @@ -247,6 +248,7 @@ void qxl_bo_vunmap_locked(struct qxl_bo *bo)
>   		return;
>   	bo->kptr = NULL;
>   	ttm_bo_vunmap(&bo->tbo, &bo->map);
> +	__qxl_bo_unpin(bo);
>   }
>   
>   int qxl_bo_vunmap(struct qxl_bo *bo)
> @@ -258,7 +260,6 @@ int qxl_bo_vunmap(struct qxl_bo *bo)
>   		return r;
>   
>   	qxl_bo_vunmap_locked(bo);
> -	__qxl_bo_unpin(bo);
>   	qxl_bo_unreserve(bo);
>   	return 0;
>   }
> diff --git a/drivers/gpu/drm/qxl/qxl_prime.c b/drivers/gpu/drm/qxl/qxl_prime.c
> index 142d01415acb..9169c26357d3 100644
> --- a/drivers/gpu/drm/qxl/qxl_prime.c
> +++ b/drivers/gpu/drm/qxl/qxl_prime.c
> @@ -59,7 +59,7 @@ int qxl_gem_prime_vmap(struct drm_gem_object *obj, struct iosys_map *map)
>   	struct qxl_bo *bo = gem_to_qxl_bo(obj);
>   	int ret;
>   
> -	ret = qxl_bo_vmap(bo, map);
> +	ret = qxl_bo_vmap_locked(bo, map);
>   	if (ret < 0)
>   		return ret;
>   
> @@ -71,5 +71,5 @@ void qxl_gem_prime_vunmap(struct drm_gem_object *obj,
>   {
>   	struct qxl_bo *bo = gem_to_qxl_bo(obj);
>   
> -	qxl_bo_vunmap(bo);
> +	qxl_bo_vunmap_locked(bo);
>   }
> diff --git a/drivers/media/common/videobuf2/videobuf2-dma-contig.c b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
> index 678b359717c4..617062076370 100644
> --- a/drivers/media/common/videobuf2/videobuf2-dma-contig.c
> +++ b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
> @@ -382,18 +382,12 @@ static struct sg_table *vb2_dc_dmabuf_ops_map(
>   	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
>   {
>   	struct vb2_dc_attachment *attach = db_attach->priv;
> -	/* stealing dmabuf mutex to serialize map/unmap operations */
> -	struct mutex *lock = &db_attach->dmabuf->lock;
>   	struct sg_table *sgt;
>   
> -	mutex_lock(lock);
> -
>   	sgt = &attach->sgt;
>   	/* return previously mapped sg table */
> -	if (attach->dma_dir == dma_dir) {
> -		mutex_unlock(lock);
> +	if (attach->dma_dir == dma_dir)
>   		return sgt;
> -	}
>   
>   	/* release any previous cache */
>   	if (attach->dma_dir != DMA_NONE) {
> @@ -409,14 +403,11 @@ static struct sg_table *vb2_dc_dmabuf_ops_map(
>   	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir,
>   			    DMA_ATTR_SKIP_CPU_SYNC)) {
>   		pr_err("failed to map scatterlist\n");
> -		mutex_unlock(lock);
>   		return ERR_PTR(-EIO);
>   	}
>   
>   	attach->dma_dir = dma_dir;
>   
> -	mutex_unlock(lock);
> -
>   	return sgt;
>   }
>   
> diff --git a/drivers/media/common/videobuf2/videobuf2-dma-sg.c b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
> index fa69158a65b1..d2075e7078cd 100644
> --- a/drivers/media/common/videobuf2/videobuf2-dma-sg.c
> +++ b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
> @@ -424,18 +424,12 @@ static struct sg_table *vb2_dma_sg_dmabuf_ops_map(
>   	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
>   {
>   	struct vb2_dma_sg_attachment *attach = db_attach->priv;
> -	/* stealing dmabuf mutex to serialize map/unmap operations */
> -	struct mutex *lock = &db_attach->dmabuf->lock;
>   	struct sg_table *sgt;
>   
> -	mutex_lock(lock);
> -
>   	sgt = &attach->sgt;
>   	/* return previously mapped sg table */
> -	if (attach->dma_dir == dma_dir) {
> -		mutex_unlock(lock);
> +	if (attach->dma_dir == dma_dir)
>   		return sgt;
> -	}
>   
>   	/* release any previous cache */
>   	if (attach->dma_dir != DMA_NONE) {
> @@ -446,14 +440,11 @@ static struct sg_table *vb2_dma_sg_dmabuf_ops_map(
>   	/* mapping to the client with new direction */
>   	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir, 0)) {
>   		pr_err("failed to map scatterlist\n");
> -		mutex_unlock(lock);
>   		return ERR_PTR(-EIO);
>   	}
>   
>   	attach->dma_dir = dma_dir;
>   
> -	mutex_unlock(lock);
> -
>   	return sgt;
>   }
>   
> diff --git a/drivers/media/common/videobuf2/videobuf2-vmalloc.c b/drivers/media/common/videobuf2/videobuf2-vmalloc.c
> index 948152f1596b..3d00a7f0aac1 100644
> --- a/drivers/media/common/videobuf2/videobuf2-vmalloc.c
> +++ b/drivers/media/common/videobuf2/videobuf2-vmalloc.c
> @@ -267,18 +267,12 @@ static struct sg_table *vb2_vmalloc_dmabuf_ops_map(
>   	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
>   {
>   	struct vb2_vmalloc_attachment *attach = db_attach->priv;
> -	/* stealing dmabuf mutex to serialize map/unmap operations */
> -	struct mutex *lock = &db_attach->dmabuf->lock;
>   	struct sg_table *sgt;
>   
> -	mutex_lock(lock);
> -
>   	sgt = &attach->sgt;
>   	/* return previously mapped sg table */
> -	if (attach->dma_dir == dma_dir) {
> -		mutex_unlock(lock);
> +	if (attach->dma_dir == dma_dir)
>   		return sgt;
> -	}
>   
>   	/* release any previous cache */
>   	if (attach->dma_dir != DMA_NONE) {
> @@ -289,14 +283,11 @@ static struct sg_table *vb2_vmalloc_dmabuf_ops_map(
>   	/* mapping to the client with new direction */
>   	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir, 0)) {
>   		pr_err("failed to map scatterlist\n");
> -		mutex_unlock(lock);
>   		return ERR_PTR(-EIO);
>   	}
>   
>   	attach->dma_dir = dma_dir;
>   
> -	mutex_unlock(lock);
> -
>   	return sgt;
>   }
>   
> diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h
> index 9d7c61a122dc..0b427939f466 100644
> --- a/include/drm/drm_gem.h
> +++ b/include/drm/drm_gem.h
> @@ -410,4 +410,7 @@ void drm_gem_unlock_reservations(struct drm_gem_object **objs, int count,
>   int drm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev,
>   			    u32 handle, u64 *offset);
>   
> +int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
> +void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
> +
>   #endif /* __DRM_GEM_H__ */
> diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
> index 71731796c8c3..23698c6b1d1e 100644
> --- a/include/linux/dma-buf.h
> +++ b/include/linux/dma-buf.h
> @@ -326,15 +326,6 @@ struct dma_buf {
>   	/** @ops: dma_buf_ops associated with this buffer object. */
>   	const struct dma_buf_ops *ops;
>   
> -	/**
> -	 * @lock:
> -	 *
> -	 * Used internally to serialize list manipulation, attach/detach and
> -	 * vmap/unmap. Note that in many cases this is superseeded by
> -	 * dma_resv_lock() on @resv.
> -	 */
> -	struct mutex lock;
> -
>   	/**
>   	 * @vmapping_counter:
>   	 *
> @@ -618,6 +609,11 @@ int dma_buf_fd(struct dma_buf *dmabuf, int flags);
>   struct dma_buf *dma_buf_get(int fd);
>   void dma_buf_put(struct dma_buf *dmabuf);
>   
> +struct sg_table *dma_buf_map_attachment_locked(struct dma_buf_attachment *,
> +					       enum dma_data_direction);
> +void dma_buf_unmap_attachment_locked(struct dma_buf_attachment *,
> +				     struct sg_table *,
> +				     enum dma_data_direction);
>   struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *,
>   					enum dma_data_direction);
>   void dma_buf_unmap_attachment(struct dma_buf_attachment *, struct sg_table *,


_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Christian König]

Hi Dmitry,

First of all please separate out this patch from the rest of the series, 
since this is a complex separate structural change.

Am 27.05.22 um 01:50 schrieb Dmitry Osipenko:
> All dma-bufs have dma-reservation lock that allows drivers to perform
> exclusive operations over shared dma-bufs. Today's dma-buf API has
> incomplete locking specification, which creates dead lock situation
> for dma-buf importers and exporters that don't coordinate theirs locks.

Well please drop that sentence. The locking specifications are actually 
very well defined, it's just that some drivers are a bit broken 
regarding them.

What you do here is rather moving all the non-dynamic drivers over to 
the dynamic locking specification (which is really nice to have).

I have tried this before and failed because catching all the locks in 
the right code paths are very tricky. So expect some fallout from this 
and make sure the kernel test robot and CI systems are clean.

> This patch introduces new locking convention for dma-buf users. From now
> on all dma-buf importers are responsible for holding dma-buf reservation
> lock around operations performed over dma-bufs.
>
> This patch implements the new dma-buf locking convention by:
>
>    1. Making dma-buf API functions to take the reservation lock.
>
>    2. Adding new locked variants of the dma-buf API functions for drivers
>       that need to manage imported dma-bufs under the held lock.

Instead of adding new locked variants please mark all variants which 
expect to be called without a lock with an _unlocked postfix.

This should make it easier to remove those in a follow up patch set and 
then fully move the locking into the importer.

>    3. Converting all drivers to the new locking scheme.

I have strong doubts that you got all of them. At least radeon and 
nouveau should grab the reservation lock in their ->attach callbacks 
somehow.

>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>   drivers/dma-buf/dma-buf.c                     | 270 +++++++++++-------
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
>   drivers/gpu/drm/drm_client.c                  |   4 +-
>   drivers/gpu/drm/drm_gem.c                     |  33 +++
>   drivers/gpu/drm/drm_gem_framebuffer_helper.c  |   6 +-
>   drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    |  10 +-
>   drivers/gpu/drm/qxl/qxl_object.c              |  17 +-
>   drivers/gpu/drm/qxl/qxl_prime.c               |   4 +-
>   .../common/videobuf2/videobuf2-dma-contig.c   |  11 +-
>   .../media/common/videobuf2/videobuf2-dma-sg.c |  11 +-
>   .../common/videobuf2/videobuf2-vmalloc.c      |  11 +-
>   include/drm/drm_gem.h                         |   3 +
>   include/linux/dma-buf.h                       |  14 +-
>   13 files changed, 241 insertions(+), 159 deletions(-)
>
> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
> index 32f55640890c..64a9909ccfa2 100644
> --- a/drivers/dma-buf/dma-buf.c
> +++ b/drivers/dma-buf/dma-buf.c
> @@ -552,7 +552,6 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)
>   	file->f_mode |= FMODE_LSEEK;
>   	dmabuf->file = file;
>   
> -	mutex_init(&dmabuf->lock);

Please make removing dmabuf->lock a separate change.

Regards,
Christian.

>   	INIT_LIST_HEAD(&dmabuf->attachments);
>   
>   	mutex_lock(&db_list.lock);
> @@ -737,14 +736,14 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   	attach->importer_ops = importer_ops;
>   	attach->importer_priv = importer_priv;  3. Converting all drivers to the new locking scheme.
>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>   drivers/dma-buf/dma-buf.c                     | 270 +++++++++++-------
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
>   drivers/gpu/drm/drm_client.c                  |   4 +-
>
>   
> +	dma_resv_lock(dmabuf->resv, NULL);
> +
>   	if (dmabuf->ops->attach) {
>   		ret = dmabuf->ops->attach(dmabuf, attach);
>   		if (ret)
>   			goto err_attach;
>   	}
> -	dma_resv_lock(dmabuf->resv, NULL);
>   	list_add(&attach->node, &dmabuf->attachments);
> -	dma_resv_unlock(dmabuf->resv);
>   
>   	/* When either the importer or the exporter can't handle dynamic
>   	 * mappings we cache the mapping here to avoid issues with the
> @@ -755,7 +754,6 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   		struct sg_table *sgt;
>   
>   		if (dma_buf_is_dynamic(attach->dmabuf)) {
> -			dma_resv_lock(attach->dmabuf->resv, NULL);
>   			ret = dmabuf->ops->pin(attach);
>   			if (ret)
>   				goto err_unlock;
> @@ -768,15 +766,16 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   			ret = PTR_ERR(sgt);
>   			goto err_unpin;
>   		}
> -		if (dma_buf_is_dynamic(attach->dmabuf))
> -			dma_resv_unlock(attach->dmabuf->resv);
>   		attach->sgt = sgt;
>   		attach->dir = DMA_BIDIRECTIONAL;
>   	}
>   
> +	dma_resv_unlock(dmabuf->resv);
> +
>   	return attach;
>   
>   err_attach:
> +	dma_resv_unlock(attach->dmabuf->resv);
>   	kfree(attach);
>   	return ERR_PTR(ret);
>   
> @@ -785,10 +784,10 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   		dmabuf->ops->unpin(attach);
>   
>   err_unlock:
> -	if (dma_buf_is_dynamic(attach->dmabuf))
> -		dma_resv_unlock(attach->dmabuf->resv);
> +	dma_resv_unlock(dmabuf->resv);
>   
>   	dma_buf_detach(dmabuf, attach);
> +
>   	return ERR_PTR(ret);
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_dynamic_attach, DMA_BUF);
> @@ -832,24 +831,23 @@ void dma_buf_detach(struct dma_buf *dmabuf, struct dma_buf_attachment *attach)
>   	if (WARN_ON(!dmabuf || !attach))
>   		return;
>   
> -	if (attach->sgt) {
> -		if (dma_buf_is_dynamic(attach->dmabuf))
> -			dma_resv_lock(attach->dmabuf->resv, NULL);
> +	if (WARN_ON(dmabuf != attach->dmabuf))
> +		return;
>   
> +	dma_resv_lock(dmabuf->resv, NULL);
> +
> +	if (attach->sgt) {
>   		__unmap_dma_buf(attach, attach->sgt, attach->dir);
>   
> -		if (dma_buf_is_dynamic(attach->dmabuf)) {
> +		if (dma_buf_is_dynamic(attach->dmabuf))
>   			dmabuf->ops->unpin(attach);
> -			dma_resv_unlock(attach->dmabuf->resv);
> -		}
>   	}
>   
> -	dma_resv_lock(dmabuf->resv, NULL);
>   	list_del(&attach->node);
> -	dma_resv_unlock(dmabuf->resv);
>   	if (dmabuf->ops->detach)
>   		dmabuf->ops->detach(dmabuf, attach);
>   
> +	dma_resv_unlock(dmabuf->resv);
>   	kfree(attach);
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_detach, DMA_BUF);
> @@ -906,28 +904,18 @@ void dma_buf_unpin(struct dma_buf_attachment *attach)
>   EXPORT_SYMBOL_NS_GPL(dma_buf_unpin, DMA_BUF);
>   
>   /**
> - * dma_buf_map_attachment - Returns the scatterlist table of the attachment;
> + * dma_buf_map_attachment_locked - Returns the scatterlist table of the attachment;
>    * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
>    * dma_buf_ops.
>    * @attach:	[in]	attachment whose scatterlist is to be returned
>    * @direction:	[in]	direction of DMA transfer
>    *
> - * Returns sg_table containing the scatterlist to be returned; returns ERR_PTR
> - * on error. May return -EINTR if it is interrupted by a signal.
> - *
> - * On success, the DMA addresses and lengths in the returned scatterlist are
> - * PAGE_SIZE aligned.
> - *
> - * A mapping must be unmapped by using dma_buf_unmap_attachment(). Note that
> - * the underlying backing storage is pinned for as long as a mapping exists,
> - * therefore users/importers should not hold onto a mapping for undue amounts of
> - * time.
> + * Locked variant of dma_buf_map_attachment().
>    *
> - * Important: Dynamic importers must wait for the exclusive fence of the struct
> - * dma_resv attached to the DMA-BUF first.
> + * Caller is responsible for holding dmabuf's reservation lock.
>    */
> -struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
> -					enum dma_data_direction direction)
> +struct sg_table *dma_buf_map_attachment_locked(struct dma_buf_attachment *attach,
> +					       enum dma_data_direction direction)
>   {
>   	struct sg_table *sg_table;
>   	int r;
> @@ -937,8 +925,7 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   	if (WARN_ON(!attach || !attach->dmabuf))
>   		return ERR_PTR(-EINVAL);
>   
> -	if (dma_buf_attachment_is_dynamic(attach))
> -		dma_resv_assert_held(attach->dmabuf->resv);
> +	dma_resv_assert_held(attach->dmabuf->resv);
>   
>   	if (attach->sgt) {
>   		/*
> @@ -953,7 +940,6 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   	}
>   
>   	if (dma_buf_is_dynamic(attach->dmabuf)) {
> -		dma_resv_assert_held(attach->dmabuf->resv);
>   		if (!IS_ENABLED(CONFIG_DMABUF_MOVE_NOTIFY)) {
>   			r = attach->dmabuf->ops->pin(attach);
>   			if (r)
> @@ -993,42 +979,101 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   #endif /* CONFIG_DMA_API_DEBUG */
>   	return sg_table;
>   }
> -EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment, DMA_BUF);
> +EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment_locked, DMA_BUF);
>   
>   /**
> - * dma_buf_unmap_attachment - unmaps and decreases usecount of the buffer;might
> - * deallocate the scatterlist associated. Is a wrapper for unmap_dma_buf() of
> + * dma_buf_map_attachment - Returns the scatterlist table of the attachment;
> + * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
>    * dma_buf_ops.
> - * @attach:	[in]	attachment to unmap buffer from
> - * @sg_table:	[in]	scatterlist info of the buffer to unmap
> - * @direction:  [in]    direction of DMA transfer
> + * @attach:	[in]	attachment whose scatterlist is to be returned
> + * @direction:	[in]	direction of DMA transfer
>    *
> - * This unmaps a DMA mapping for @attached obtained by dma_buf_map_attachment().
> + * Returns sg_table containing the scatterlist to be returned; returns ERR_PTR
> + * on error. May return -EINTR if it is interrupted by a signal.
> + *
> + * On success, the DMA addresses and lengths in the returned scatterlist are
> + * PAGE_SIZE aligned.
> + *
> + * A mapping must be unmapped by using dma_buf_unmap_attachment(). Note that
> + * the underlying backing storage is pinned for as long as a mapping exists,
> + * therefore users/importers should not hold onto a mapping for undue amounts of
> + * time.
> + *
> + * Important: Dynamic importers must wait for the exclusive fence of the struct
> + * dma_resv attached to the DMA-BUF first.
>    */
> -void dma_buf_unmap_attachment(struct dma_buf_attachment *attach,
> -				struct sg_table *sg_table,
> +struct sg_table *
> +dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   				enum dma_data_direction direction)
>   {
> +	struct sg_table *sg_table;
> +
>   	might_sleep();
>   
> -	if (WARN_ON(!attach || !attach->dmabuf || !sg_table))
> -		return;
> +	if (WARN_ON(!attach || !attach->dmabuf))
> +		return ERR_PTR(-EINVAL);
> +
> +	dma_resv_lock(attach->dmabuf->resv, NULL);
> +	sg_table = dma_buf_map_attachment_locked(attach, direction);
> +	dma_resv_unlock(attach->dmabuf->resv);
>   
> -	if (dma_buf_attachment_is_dynamic(attach))
> -		dma_resv_assert_held(attach->dmabuf->resv);
> +	return sg_table;
> +}
> +EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment, DMA_BUF);
> +
> +/**
> + * dma_buf_unmap_attachment_locked - Returns the scatterlist table of the attachment;
> + * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
> + * dma_buf_ops.
> + * @attach:	[in]	attachment whose scatterlist is to be returned
> + * @direction:	[in]	direction of DMA transfer
> + *
> + * Locked variant of dma_buf_unmap_attachment().
> + *
> + * Caller is responsible for holding dmabuf's reservation lock.
> + */
> +void dma_buf_unmap_attachment_locked(struct dma_buf_attachment *attach,
> +				     struct sg_table *sg_table,
> +				     enum dma_data_direction direction)
> +{
> +	might_sleep();
> +
> +	dma_resv_assert_held(attach->dmabuf->resv);
>   
>   	if (attach->sgt == sg_table)
>   		return;
>   
> -	if (dma_buf_is_dynamic(attach->dmabuf))
> -		dma_resv_assert_held(attach->dmabuf->resv);
> -
>   	__unmap_dma_buf(attach, sg_table, direction);
>   
>   	if (dma_buf_is_dynamic(attach->dmabuf) &&
>   	    !IS_ENABLED(CONFIG_DMABUF_MOVE_NOTIFY))
>   		dma_buf_unpin(attach);
>   }
> +EXPORT_SYMBOL_NS_GPL(dma_buf_unmap_attachment_locked, DMA_BUF);
> +
> +/**
> + * dma_buf_unmap_attachment - unmaps and decreases usecount of the buffer;might
> + * deallocate the scatterlist associated. Is a wrapper for unmap_dma_buf() of
> + * dma_buf_ops.
> + * @attach:	[in]	attachment to unmap buffer from
> + * @sg_table:	[in]	scatterlist info of the buffer to unmap
> + * @direction:  [in]    direction of DMA transfer
> + *
> + * This unmaps a DMA mapping for @attached obtained by dma_buf_map_attachment().
> + */
> +void dma_buf_unmap_attachment(struct dma_buf_attachment *attach,
> +			      struct sg_table *sg_table,
> +			      enum dma_data_direction direction)
> +{
> +	might_sleep();
> +
> +	if (WARN_ON(!attach || !attach->dmabuf || !sg_table))
> +		return;
> +
> +	dma_resv_lock(attach->dmabuf->resv, NULL);
> +	dma_buf_unmap_attachment_locked(attach, sg_table, direction);
> +	dma_resv_unlock(attach->dmabuf->resv);
> +}
>   EXPORT_SYMBOL_NS_GPL(dma_buf_unmap_attachment, DMA_BUF);
>   
>   /**
> @@ -1224,6 +1269,31 @@ int dma_buf_end_cpu_access(struct dma_buf *dmabuf,
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_end_cpu_access, DMA_BUF);
>   
> +static int dma_buf_mmap_locked(struct dma_buf *dmabuf,
> +			       struct vm_area_struct *vma,
> +			       unsigned long pgoff)
> +{
> +	dma_resv_assert_held(dmabuf->resv);
> +
> +	/* check if buffer supports mmap */
> +	if (!dmabuf->ops->mmap)
> +		return -EINVAL;
> +
> +	/* check for offset overflow */
> +	if (pgoff + vma_pages(vma) < pgoff)
> +		return -EOVERFLOW;
> +
> +	/* check for overflowing the buffer's size */
> +	if (pgoff + vma_pages(vma) >
> +	    dmabuf->size >> PAGE_SHIFT)
> +		return -EINVAL;
> +
> +	/* readjust the vma */
> +	vma_set_file(vma, dmabuf->file);
> +	vma->vm_pgoff = pgoff;
> +
> +	return dmabuf->ops->mmap(dmabuf, vma);
> +}
>   
>   /**
>    * dma_buf_mmap - Setup up a userspace mmap with the given vma
> @@ -1242,29 +1312,46 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_end_cpu_access, DMA_BUF);
>   int dma_buf_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma,
>   		 unsigned long pgoff)
>   {
> +	int ret;
> +
>   	if (WARN_ON(!dmabuf || !vma))
>   		return -EINVAL;
>   
> -	/* check if buffer supports mmap */
> -	if (!dmabuf->ops->mmap)
> -		return -EINVAL;
> +	dma_resv_lock(dmabuf->resv, NULL);
> +	ret = dma_buf_mmap_locked(dmabuf, vma, pgoff);
> +	dma_resv_unlock(dmabuf->resv);
>   
> -	/* check for offset overflow */
> -	if (pgoff + vma_pages(vma) < pgoff)
> -		return -EOVERFLOW;
> +	return ret;
> +}
> +EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
>   
> -	/* check for overflowing the buffer's size */
> -	if (pgoff + vma_pages(vma) >
> -	    dmabuf->size >> PAGE_SHIFT)
> -		return -EINVAL;
> +static int dma_buf_vmap_locked(struct dma_buf *dmabuf, struct iosys_map *map)
> +{
> +	struct iosys_map ptr;
> +	int ret;
>   
> -	/* readjust the vma */
> -	vma_set_file(vma, dmabuf->file);
> -	vma->vm_pgoff = pgoff;
> +	dma_resv_assert_held(dmabuf->resv);
>   
> -	return dmabuf->ops->mmap(dmabuf, vma);
> +	if (dmabuf->vmapping_counter) {
> +		dmabuf->vmapping_counter++;
> +		BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
> +		*map = dmabuf->vmap_ptr;
> +		return ret;
> +	}
> +
> +	BUG_ON(iosys_map_is_set(&dmabuf->vmap_ptr));
> +
> +	ret = dmabuf->ops->vmap(dmabuf, &ptr);
> +	if (WARN_ON_ONCE(ret))
> +		return ret;
> +
> +	dmabuf->vmap_ptr = ptr;
> +	dmabuf->vmapping_counter = 1;
> +
> +	*map = dmabuf->vmap_ptr;
> +
> +	return 0;
>   }
> -EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
>   
>   /**
>    * dma_buf_vmap - Create virtual mapping for the buffer object into kernel
> @@ -1284,8 +1371,7 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
>    */
>   int dma_buf_vmap(struct dma_buf *dmabuf, struct iosys_map *map)
>   {
> -	struct iosys_map ptr;
> -	int ret = 0;
> +	int ret;
>   
>   	iosys_map_clear(map);
>   
> @@ -1295,52 +1381,40 @@ int dma_buf_vmap(struct dma_buf *dmabuf, struct iosys_map *map)
>   	if (!dmabuf->ops->vmap)
>   		return -EINVAL;
>   
> -	mutex_lock(&dmabuf->lock);
> -	if (dmabuf->vmapping_counter) {
> -		dmabuf->vmapping_counter++;
> -		BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
> -		*map = dmabuf->vmap_ptr;
> -		goto out_unlock;
> -	}
> -
> -	BUG_ON(iosys_map_is_set(&dmabuf->vmap_ptr));
> -
> -	ret = dmabuf->ops->vmap(dmabuf, &ptr);
> -	if (WARN_ON_ONCE(ret))
> -		goto out_unlock;
> -
> -	dmabuf->vmap_ptr = ptr;
> -	dmabuf->vmapping_counter = 1;
> -
> -	*map = dmabuf->vmap_ptr;
> +	dma_resv_lock(dmabuf->resv, NULL);
> +	ret = dma_buf_vmap_locked(dmabuf, map);
> +	dma_resv_unlock(dmabuf->resv);
>   
> -out_unlock:
> -	mutex_unlock(&dmabuf->lock);
>   	return ret;
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_vmap, DMA_BUF);
>   
> -/**
> - * dma_buf_vunmap - Unmap a vmap obtained by dma_buf_vmap.
> - * @dmabuf:	[in]	buffer to vunmap
> - * @map:	[in]	vmap pointer to vunmap
> - */
> -void dma_buf_vunmap(struct dma_buf *dmabuf, struct iosys_map *map)
> +static void dma_buf_vunmap_locked(struct dma_buf *dmabuf, struct iosys_map *map)
>   {
> -	if (WARN_ON(!dmabuf))
> -		return;
> -
>   	BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
>   	BUG_ON(dmabuf->vmapping_counter == 0);
>   	BUG_ON(!iosys_map_is_equal(&dmabuf->vmap_ptr, map));
>   
> -	mutex_lock(&dmabuf->lock);
>   	if (--dmabuf->vmapping_counter == 0) {
>   		if (dmabuf->ops->vunmap)
>   			dmabuf->ops->vunmap(dmabuf, map);
>   		iosys_map_clear(&dmabuf->vmap_ptr);
>   	}
> -	mutex_unlock(&dmabuf->lock);
> +}
> +
> +/**
> + * dma_buf_vunmap - Unmap a vmap obtained by dma_buf_vmap.
> + * @dmabuf:	[in]	buffer to vunmap
> + * @map:	[in]	vmap pointer to vunmap
> + */
> +void dma_buf_vunmap(struct dma_buf *dmabuf, struct iosys_map *map)
> +{
> +	if (WARN_ON(!dmabuf))
> +		return;
> +
> +	dma_resv_lock(dmabuf->resv, NULL);
> +	dma_buf_vunmap_locked(dmabuf, map);
> +	dma_resv_unlock(dmabuf->resv);
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_vunmap, DMA_BUF);
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> index be6f76a30ac6..b704bdf5601a 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> @@ -882,7 +882,8 @@ static int amdgpu_ttm_backend_bind(struct ttm_device *bdev,
>   			struct sg_table *sgt;
>   
>   			attach = gtt->gobj->import_attach;
> -			sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL);
> +			sgt = dma_buf_map_attachment_locked(attach,
> +							    DMA_BIDIRECTIONAL);
>   			if (IS_ERR(sgt))
>   				return PTR_ERR(sgt);
>   
> @@ -1007,7 +1008,8 @@ static void amdgpu_ttm_backend_unbind(struct ttm_device *bdev,
>   		struct dma_buf_attachment *attach;
>   
>   		attach = gtt->gobj->import_attach;
> -		dma_buf_unmap_attachment(attach, ttm->sg, DMA_BIDIRECTIONAL);
> +		dma_buf_unmap_attachment_locked(attach, ttm->sg,
> +						DMA_BIDIRECTIONAL);
>   		ttm->sg = NULL;
>   	}
>   
> diff --git a/drivers/gpu/drm/drm_client.c b/drivers/gpu/drm/drm_client.c
> index af3b7395bf69..e9a1cd310352 100644
> --- a/drivers/gpu/drm/drm_client.c
> +++ b/drivers/gpu/drm/drm_client.c
> @@ -323,7 +323,7 @@ drm_client_buffer_vmap(struct drm_client_buffer *buffer,
>   	 * fd_install step out of the driver backend hooks, to make that
>   	 * final step optional for internal users.
>   	 */
> -	ret = drm_gem_vmap(buffer->gem, map);
> +	ret = drm_gem_vmap_unlocked(buffer->gem, map);
>   	if (ret)
>   		return ret;
>   
> @@ -345,7 +345,7 @@ void drm_client_buffer_vunmap(struct drm_client_buffer *buffer)
>   {
>   	struct iosys_map *map = &buffer->map;
>   
> -	drm_gem_vunmap(buffer->gem, map);
> +	drm_gem_vunmap_unlocked(buffer->gem, map);
>   }
>   EXPORT_SYMBOL(drm_client_buffer_vunmap);
>   
> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
> index 7c0b025508e4..c61674887582 100644
> --- a/drivers/gpu/drm/drm_gem.c
> +++ b/drivers/gpu/drm/drm_gem.c
> @@ -1053,7 +1053,12 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,
>   	vma->vm_ops = obj->funcs->vm_ops;
>   
>   	if (obj->funcs->mmap) {
> +		ret = dma_resv_lock_interruptible(obj->resv, NULL);
> +		if (ret)
> +			goto err_drm_gem_object_put;
> +
>   		ret = obj->funcs->mmap(obj, vma);
> +		dma_resv_unlock(obj->resv);
>   		if (ret)
>   			goto err_drm_gem_object_put;
>   		WARN_ON(!(vma->vm_flags & VM_DONTEXPAND));
> @@ -1158,6 +1163,8 @@ void drm_gem_print_info(struct drm_printer *p, unsigned int indent,
>   
>   int drm_gem_pin(struct drm_gem_object *obj)
>   {
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (obj->funcs->pin)
>   		return obj->funcs->pin(obj);
>   	else
> @@ -1166,6 +1173,8 @@ int drm_gem_pin(struct drm_gem_object *obj)
>   
>   void drm_gem_unpin(struct drm_gem_object *obj)
>   {
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (obj->funcs->unpin)
>   		obj->funcs->unpin(obj);
>   }
> @@ -1174,6 +1183,8 @@ int drm_gem_vmap(struct drm_gem_object *obj, struct iosys_map *map)
>   {
>   	int ret;
>   
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (!obj->funcs->vmap)
>   		return -EOPNOTSUPP;
>   
> @@ -1189,6 +1200,8 @@ EXPORT_SYMBOL(drm_gem_vmap);
>   
>   void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
>   {
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (iosys_map_is_null(map))
>   		return;
>   
> @@ -1200,6 +1213,26 @@ void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
>   }
>   EXPORT_SYMBOL(drm_gem_vunmap);
>   
> +int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
> +{
> +	int ret;
> +
> +	dma_resv_lock(obj->resv, NULL);
> +	ret = drm_gem_vmap(obj, map);
> +	dma_resv_unlock(obj->resv);
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL(drm_gem_vmap_unlocked);
> +
> +void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
> +{
> +	dma_resv_lock(obj->resv, NULL);
> +	drm_gem_vunmap(obj, map);
> +	dma_resv_unlock(obj->resv);
> +}
> +EXPORT_SYMBOL(drm_gem_vunmap_unlocked);
> +
>   /**
>    * drm_gem_lock_reservations - Sets up the ww context and acquires
>    * the lock on an array of GEM objects.
> diff --git a/drivers/gpu/drm/drm_gem_framebuffer_helper.c b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
> index f4619803acd0..a0bff53b158e 100644
> --- a/drivers/gpu/drm/drm_gem_framebuffer_helper.c
> +++ b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
> @@ -348,7 +348,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb,
>   			iosys_map_clear(&map[i]);
>   			continue;
>   		}
> -		ret = drm_gem_vmap(obj, &map[i]);
> +		ret = drm_gem_vmap_unlocked(obj, &map[i]);
>   		if (ret)
>   			goto err_drm_gem_vunmap;
>   	}
> @@ -370,7 +370,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb,
>   		obj = drm_gem_fb_get_obj(fb, i);
>   		if (!obj)
>   			continue;
> -		drm_gem_vunmap(obj, &map[i]);
> +		drm_gem_vunmap_unlocked(obj, &map[i]);
>   	}
>   	return ret;
>   }
> @@ -398,7 +398,7 @@ void drm_gem_fb_vunmap(struct drm_framebuffer *fb,
>   			continue;
>   		if (iosys_map_is_null(&map[i]))
>   			continue;
> -		drm_gem_vunmap(obj, &map[i]);
> +		drm_gem_vunmap_unlocked(obj, &map[i]);
>   	}
>   }
>   EXPORT_SYMBOL(drm_gem_fb_vunmap);
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
> index f5062d0c6333..09502d490da8 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
> @@ -72,7 +72,7 @@ static int i915_gem_dmabuf_vmap(struct dma_buf *dma_buf,
>   	struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);
>   	void *vaddr;
>   
> -	vaddr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
> +	vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
>   	if (IS_ERR(vaddr))
>   		return PTR_ERR(vaddr);
>   
> @@ -241,8 +241,8 @@ static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
>   
>   	assert_object_held(obj);
>   
> -	pages = dma_buf_map_attachment(obj->base.import_attach,
> -				       DMA_BIDIRECTIONAL);
> +	pages = dma_buf_map_attachment_locked(obj->base.import_attach,
> +					      DMA_BIDIRECTIONAL);
>   	if (IS_ERR(pages))
>   		return PTR_ERR(pages);
>   
> @@ -270,8 +270,8 @@ static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
>   static void i915_gem_object_put_pages_dmabuf(struct drm_i915_gem_object *obj,
>   					     struct sg_table *pages)
>   {
> -	dma_buf_unmap_attachment(obj->base.import_attach, pages,
> -				 DMA_BIDIRECTIONAL);
> +	dma_buf_unmap_attachment_locked(obj->base.import_attach, pages,
> +					DMA_BIDIRECTIONAL);
>   }
>   
>   static const struct drm_i915_gem_object_ops i915_gem_object_dmabuf_ops = {
> diff --git a/drivers/gpu/drm/qxl/qxl_object.c b/drivers/gpu/drm/qxl/qxl_object.c
> index b42a657e4c2f..a64cd635fbc0 100644
> --- a/drivers/gpu/drm/qxl/qxl_object.c
> +++ b/drivers/gpu/drm/qxl/qxl_object.c
> @@ -168,9 +168,16 @@ int qxl_bo_vmap_locked(struct qxl_bo *bo, struct iosys_map *map)
>   		bo->map_count++;
>   		goto out;
>   	}
> -	r = ttm_bo_vmap(&bo->tbo, &bo->map);
> +
> +	r = __qxl_bo_pin(bo);
>   	if (r)
>   		return r;
> +
> +	r = ttm_bo_vmap(&bo->tbo, &bo->map);
> +	if (r) {
> +		__qxl_bo_unpin(bo);
> +		return r;
> +	}
>   	bo->map_count = 1;
>   
>   	/* TODO: Remove kptr in favor of map everywhere. */
> @@ -192,12 +199,6 @@ int qxl_bo_vmap(struct qxl_bo *bo, struct iosys_map *map)
>   	if (r)
>   		return r;
>   
> -	r = __qxl_bo_pin(bo);
> -	if (r) {
> -		qxl_bo_unreserve(bo);
> -		return r;
> -	}
> -
>   	r = qxl_bo_vmap_locked(bo, map);
>   	qxl_bo_unreserve(bo);
>   	return r;
> @@ -247,6 +248,7 @@ void qxl_bo_vunmap_locked(struct qxl_bo *bo)
>   		return;
>   	bo->kptr = NULL;
>   	ttm_bo_vunmap(&bo->tbo, &bo->map);
> +	__qxl_bo_unpin(bo);
>   }
>   
>   int qxl_bo_vunmap(struct qxl_bo *bo)
> @@ -258,7 +260,6 @@ int qxl_bo_vunmap(struct qxl_bo *bo)
>   		return r;
>   
>   	qxl_bo_vunmap_locked(bo);
> -	__qxl_bo_unpin(bo);
>   	qxl_bo_unreserve(bo);
>   	return 0;
>   }
> diff --git a/drivers/gpu/drm/qxl/qxl_prime.c b/drivers/gpu/drm/qxl/qxl_prime.c
> index 142d01415acb..9169c26357d3 100644
> --- a/drivers/gpu/drm/qxl/qxl_prime.c
> +++ b/drivers/gpu/drm/qxl/qxl_prime.c
> @@ -59,7 +59,7 @@ int qxl_gem_prime_vmap(struct drm_gem_object *obj, struct iosys_map *map)
>   	struct qxl_bo *bo = gem_to_qxl_bo(obj);
>   	int ret;
>   
> -	ret = qxl_bo_vmap(bo, map);
> +	ret = qxl_bo_vmap_locked(bo, map);
>   	if (ret < 0)
>   		return ret;
>   
> @@ -71,5 +71,5 @@ void qxl_gem_prime_vunmap(struct drm_gem_object *obj,
>   {
>   	struct qxl_bo *bo = gem_to_qxl_bo(obj);
>   
> -	qxl_bo_vunmap(bo);
> +	qxl_bo_vunmap_locked(bo);
>   }
> diff --git a/drivers/media/common/videobuf2/videobuf2-dma-contig.c b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
> index 678b359717c4..617062076370 100644
> --- a/drivers/media/common/videobuf2/videobuf2-dma-contig.c
> +++ b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
> @@ -382,18 +382,12 @@ static struct sg_table *vb2_dc_dmabuf_ops_map(
>   	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
>   {
>   	struct vb2_dc_attachment *attach = db_attach->priv;
> -	/* stealing dmabuf mutex to serialize map/unmap operations */
> -	struct mutex *lock = &db_attach->dmabuf->lock;
>   	struct sg_table *sgt;
>   
> -	mutex_lock(lock);
> -
>   	sgt = &attach->sgt;
>   	/* return previously mapped sg table */
> -	if (attach->dma_dir == dma_dir) {
> -		mutex_unlock(lock);
> +	if (attach->dma_dir == dma_dir)
>   		return sgt;
> -	}
>   
>   	/* release any previous cache */
>   	if (attach->dma_dir != DMA_NONE) {
> @@ -409,14 +403,11 @@ static struct sg_table *vb2_dc_dmabuf_ops_map(
>   	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir,
>   			    DMA_ATTR_SKIP_CPU_SYNC)) {
>   		pr_err("failed to map scatterlist\n");
> -		mutex_unlock(lock);
>   		return ERR_PTR(-EIO);
>   	}
>   
>   	attach->dma_dir = dma_dir;
>   
> -	mutex_unlock(lock);
> -
>   	return sgt;
>   }
>   
> diff --git a/drivers/media/common/videobuf2/videobuf2-dma-sg.c b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
> index fa69158a65b1..d2075e7078cd 100644
> --- a/drivers/media/common/videobuf2/videobuf2-dma-sg.c
> +++ b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
> @@ -424,18 +424,12 @@ static struct sg_table *vb2_dma_sg_dmabuf_ops_map(
>   	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
>   {
>   	struct vb2_dma_sg_attachment *attach = db_attach->priv;
> -	/* stealing dmabuf mutex to serialize map/unmap operations */
> -	struct mutex *lock = &db_attach->dmabuf->lock;
>   	struct sg_table *sgt;
>   
> -	mutex_lock(lock);
> -
>   	sgt = &attach->sgt;
>   	/* return previously mapped sg table */
> -	if (attach->dma_dir == dma_dir) {
> -		mutex_unlock(lock);
> +	if (attach->dma_dir == dma_dir)
>   		return sgt;
> -	}
>   
>   	/* release any previous cache */
>   	if (attach->dma_dir != DMA_NONE) {
> @@ -446,14 +440,11 @@ static struct sg_table *vb2_dma_sg_dmabuf_ops_map(
>   	/* mapping to the client with new direction */
>   	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir, 0)) {
>   		pr_err("failed to map scatterlist\n");
> -		mutex_unlock(lock);
>   		return ERR_PTR(-EIO);
>   	}
>   
>   	attach->dma_dir = dma_dir;
>   
> -	mutex_unlock(lock);
> -
>   	return sgt;
>   }
>   
> diff --git a/drivers/media/common/videobuf2/videobuf2-vmalloc.c b/drivers/media/common/videobuf2/videobuf2-vmalloc.c
> index 948152f1596b..3d00a7f0aac1 100644
> --- a/drivers/media/common/videobuf2/videobuf2-vmalloc.c
> +++ b/drivers/media/common/videobuf2/videobuf2-vmalloc.c
> @@ -267,18 +267,12 @@ static struct sg_table *vb2_vmalloc_dmabuf_ops_map(
>   	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
>   {
>   	struct vb2_vmalloc_attachment *attach = db_attach->priv;
> -	/* stealing dmabuf mutex to serialize map/unmap operations */
> -	struct mutex *lock = &db_attach->dmabuf->lock;
>   	struct sg_table *sgt;
>   
> -	mutex_lock(lock);
> -
>   	sgt = &attach->sgt;
>   	/* return previously mapped sg table */
> -	if (attach->dma_dir == dma_dir) {
> -		mutex_unlock(lock);
> +	if (attach->dma_dir == dma_dir)
>   		return sgt;
> -	}
>   
>   	/* release any previous cache */
>   	if (attach->dma_dir != DMA_NONE) {
> @@ -289,14 +283,11 @@ static struct sg_table *vb2_vmalloc_dmabuf_ops_map(
>   	/* mapping to the client with new direction */
>   	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir, 0)) {
>   		pr_err("failed to map scatterlist\n");
> -		mutex_unlock(lock);
>   		return ERR_PTR(-EIO);
>   	}
>   
>   	attach->dma_dir = dma_dir;
>   
> -	mutex_unlock(lock);
> -
>   	return sgt;
>   }
>   
> diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h
> index 9d7c61a122dc..0b427939f466 100644
> --- a/include/drm/drm_gem.h
> +++ b/include/drm/drm_gem.h
> @@ -410,4 +410,7 @@ void drm_gem_unlock_reservations(struct drm_gem_object **objs, int count,
>   int drm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev,
>   			    u32 handle, u64 *offset);
>   
> +int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
> +void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
> +
>   #endif /* __DRM_GEM_H__ */
> diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
> index 71731796c8c3..23698c6b1d1e 100644
> --- a/include/linux/dma-buf.h
> +++ b/include/linux/dma-buf.h
> @@ -326,15 +326,6 @@ struct dma_buf {
>   	/** @ops: dma_buf_ops associated with this buffer object. */
>   	const struct dma_buf_ops *ops;
>   
> -	/**
> -	 * @lock:
> -	 *
> -	 * Used internally to serialize list manipulation, attach/detach and
> -	 * vmap/unmap. Note that in many cases this is superseeded by
> -	 * dma_resv_lock() on @resv.
> -	 */
> -	struct mutex lock;
> -
>   	/**
>   	 * @vmapping_counter:
>   	 *
> @@ -618,6 +609,11 @@ int dma_buf_fd(struct dma_buf *dmabuf, int flags);
>   struct dma_buf *dma_buf_get(int fd);
>   void dma_buf_put(struct dma_buf *dmabuf);
>   
> +struct sg_table *dma_buf_map_attachment_locked(struct dma_buf_attachment *,
> +					       enum dma_data_direction);
> +void dma_buf_unmap_attachment_locked(struct dma_buf_attachment *,
> +				     struct sg_table *,
> +				     enum dma_data_direction);
>   struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *,
>   					enum dma_data_direction);
>   void dma_buf_unmap_attachment(struct dma_buf_attachment *, struct sg_table *,

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Christian König]

Hi Dmitry,

First of all please separate out this patch from the rest of the series, 
since this is a complex separate structural change.

Am 27.05.22 um 01:50 schrieb Dmitry Osipenko:
> All dma-bufs have dma-reservation lock that allows drivers to perform
> exclusive operations over shared dma-bufs. Today's dma-buf API has
> incomplete locking specification, which creates dead lock situation
> for dma-buf importers and exporters that don't coordinate theirs locks.

Well please drop that sentence. The locking specifications are actually 
very well defined, it's just that some drivers are a bit broken 
regarding them.

What you do here is rather moving all the non-dynamic drivers over to 
the dynamic locking specification (which is really nice to have).

I have tried this before and failed because catching all the locks in 
the right code paths are very tricky. So expect some fallout from this 
and make sure the kernel test robot and CI systems are clean.

> This patch introduces new locking convention for dma-buf users. From now
> on all dma-buf importers are responsible for holding dma-buf reservation
> lock around operations performed over dma-bufs.
>
> This patch implements the new dma-buf locking convention by:
>
>    1. Making dma-buf API functions to take the reservation lock.
>
>    2. Adding new locked variants of the dma-buf API functions for drivers
>       that need to manage imported dma-bufs under the held lock.

Instead of adding new locked variants please mark all variants which 
expect to be called without a lock with an _unlocked postfix.

This should make it easier to remove those in a follow up patch set and 
then fully move the locking into the importer.

>    3. Converting all drivers to the new locking scheme.

I have strong doubts that you got all of them. At least radeon and 
nouveau should grab the reservation lock in their ->attach callbacks 
somehow.

>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>   drivers/dma-buf/dma-buf.c                     | 270 +++++++++++-------
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
>   drivers/gpu/drm/drm_client.c                  |   4 +-
>   drivers/gpu/drm/drm_gem.c                     |  33 +++
>   drivers/gpu/drm/drm_gem_framebuffer_helper.c  |   6 +-
>   drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    |  10 +-
>   drivers/gpu/drm/qxl/qxl_object.c              |  17 +-
>   drivers/gpu/drm/qxl/qxl_prime.c               |   4 +-
>   .../common/videobuf2/videobuf2-dma-contig.c   |  11 +-
>   .../media/common/videobuf2/videobuf2-dma-sg.c |  11 +-
>   .../common/videobuf2/videobuf2-vmalloc.c      |  11 +-
>   include/drm/drm_gem.h                         |   3 +
>   include/linux/dma-buf.h                       |  14 +-
>   13 files changed, 241 insertions(+), 159 deletions(-)
>
> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
> index 32f55640890c..64a9909ccfa2 100644
> --- a/drivers/dma-buf/dma-buf.c
> +++ b/drivers/dma-buf/dma-buf.c
> @@ -552,7 +552,6 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)
>   	file->f_mode |= FMODE_LSEEK;
>   	dmabuf->file = file;
>   
> -	mutex_init(&dmabuf->lock);

Please make removing dmabuf->lock a separate change.

Regards,
Christian.

>   	INIT_LIST_HEAD(&dmabuf->attachments);
>   
>   	mutex_lock(&db_list.lock);
> @@ -737,14 +736,14 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   	attach->importer_ops = importer_ops;
>   	attach->importer_priv = importer_priv;  3. Converting all drivers to the new locking scheme.
>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>   drivers/dma-buf/dma-buf.c                     | 270 +++++++++++-------
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
>   drivers/gpu/drm/drm_client.c                  |   4 +-
>
>   
> +	dma_resv_lock(dmabuf->resv, NULL);
> +
>   	if (dmabuf->ops->attach) {
>   		ret = dmabuf->ops->attach(dmabuf, attach);
>   		if (ret)
>   			goto err_attach;
>   	}
> -	dma_resv_lock(dmabuf->resv, NULL);
>   	list_add(&attach->node, &dmabuf->attachments);
> -	dma_resv_unlock(dmabuf->resv);
>   
>   	/* When either the importer or the exporter can't handle dynamic
>   	 * mappings we cache the mapping here to avoid issues with the
> @@ -755,7 +754,6 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   		struct sg_table *sgt;
>   
>   		if (dma_buf_is_dynamic(attach->dmabuf)) {
> -			dma_resv_lock(attach->dmabuf->resv, NULL);
>   			ret = dmabuf->ops->pin(attach);
>   			if (ret)
>   				goto err_unlock;
> @@ -768,15 +766,16 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   			ret = PTR_ERR(sgt);
>   			goto err_unpin;
>   		}
> -		if (dma_buf_is_dynamic(attach->dmabuf))
> -			dma_resv_unlock(attach->dmabuf->resv);
>   		attach->sgt = sgt;
>   		attach->dir = DMA_BIDIRECTIONAL;
>   	}
>   
> +	dma_resv_unlock(dmabuf->resv);
> +
>   	return attach;
>   
>   err_attach:
> +	dma_resv_unlock(attach->dmabuf->resv);
>   	kfree(attach);
>   	return ERR_PTR(ret);
>   
> @@ -785,10 +784,10 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   		dmabuf->ops->unpin(attach);
>   
>   err_unlock:
> -	if (dma_buf_is_dynamic(attach->dmabuf))
> -		dma_resv_unlock(attach->dmabuf->resv);
> +	dma_resv_unlock(dmabuf->resv);
>   
>   	dma_buf_detach(dmabuf, attach);
> +
>   	return ERR_PTR(ret);
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_dynamic_attach, DMA_BUF);
> @@ -832,24 +831,23 @@ void dma_buf_detach(struct dma_buf *dmabuf, struct dma_buf_attachment *attach)
>   	if (WARN_ON(!dmabuf || !attach))
>   		return;
>   
> -	if (attach->sgt) {
> -		if (dma_buf_is_dynamic(attach->dmabuf))
> -			dma_resv_lock(attach->dmabuf->resv, NULL);
> +	if (WARN_ON(dmabuf != attach->dmabuf))
> +		return;
>   
> +	dma_resv_lock(dmabuf->resv, NULL);
> +
> +	if (attach->sgt) {
>   		__unmap_dma_buf(attach, attach->sgt, attach->dir);
>   
> -		if (dma_buf_is_dynamic(attach->dmabuf)) {
> +		if (dma_buf_is_dynamic(attach->dmabuf))
>   			dmabuf->ops->unpin(attach);
> -			dma_resv_unlock(attach->dmabuf->resv);
> -		}
>   	}
>   
> -	dma_resv_lock(dmabuf->resv, NULL);
>   	list_del(&attach->node);
> -	dma_resv_unlock(dmabuf->resv);
>   	if (dmabuf->ops->detach)
>   		dmabuf->ops->detach(dmabuf, attach);
>   
> +	dma_resv_unlock(dmabuf->resv);
>   	kfree(attach);
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_detach, DMA_BUF);
> @@ -906,28 +904,18 @@ void dma_buf_unpin(struct dma_buf_attachment *attach)
>   EXPORT_SYMBOL_NS_GPL(dma_buf_unpin, DMA_BUF);
>   
>   /**
> - * dma_buf_map_attachment - Returns the scatterlist table of the attachment;
> + * dma_buf_map_attachment_locked - Returns the scatterlist table of the attachment;
>    * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
>    * dma_buf_ops.
>    * @attach:	[in]	attachment whose scatterlist is to be returned
>    * @direction:	[in]	direction of DMA transfer
>    *
> - * Returns sg_table containing the scatterlist to be returned; returns ERR_PTR
> - * on error. May return -EINTR if it is interrupted by a signal.
> - *
> - * On success, the DMA addresses and lengths in the returned scatterlist are
> - * PAGE_SIZE aligned.
> - *
> - * A mapping must be unmapped by using dma_buf_unmap_attachment(). Note that
> - * the underlying backing storage is pinned for as long as a mapping exists,
> - * therefore users/importers should not hold onto a mapping for undue amounts of
> - * time.
> + * Locked variant of dma_buf_map_attachment().
>    *
> - * Important: Dynamic importers must wait for the exclusive fence of the struct
> - * dma_resv attached to the DMA-BUF first.
> + * Caller is responsible for holding dmabuf's reservation lock.
>    */
> -struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
> -					enum dma_data_direction direction)
> +struct sg_table *dma_buf_map_attachment_locked(struct dma_buf_attachment *attach,
> +					       enum dma_data_direction direction)
>   {
>   	struct sg_table *sg_table;
>   	int r;
> @@ -937,8 +925,7 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   	if (WARN_ON(!attach || !attach->dmabuf))
>   		return ERR_PTR(-EINVAL);
>   
> -	if (dma_buf_attachment_is_dynamic(attach))
> -		dma_resv_assert_held(attach->dmabuf->resv);
> +	dma_resv_assert_held(attach->dmabuf->resv);
>   
>   	if (attach->sgt) {
>   		/*
> @@ -953,7 +940,6 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   	}
>   
>   	if (dma_buf_is_dynamic(attach->dmabuf)) {
> -		dma_resv_assert_held(attach->dmabuf->resv);
>   		if (!IS_ENABLED(CONFIG_DMABUF_MOVE_NOTIFY)) {
>   			r = attach->dmabuf->ops->pin(attach);
>   			if (r)
> @@ -993,42 +979,101 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   #endif /* CONFIG_DMA_API_DEBUG */
>   	return sg_table;
>   }
> -EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment, DMA_BUF);
> +EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment_locked, DMA_BUF);
>   
>   /**
> - * dma_buf_unmap_attachment - unmaps and decreases usecount of the buffer;might
> - * deallocate the scatterlist associated. Is a wrapper for unmap_dma_buf() of
> + * dma_buf_map_attachment - Returns the scatterlist table of the attachment;
> + * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
>    * dma_buf_ops.
> - * @attach:	[in]	attachment to unmap buffer from
> - * @sg_table:	[in]	scatterlist info of the buffer to unmap
> - * @direction:  [in]    direction of DMA transfer
> + * @attach:	[in]	attachment whose scatterlist is to be returned
> + * @direction:	[in]	direction of DMA transfer
>    *
> - * This unmaps a DMA mapping for @attached obtained by dma_buf_map_attachment().
> + * Returns sg_table containing the scatterlist to be returned; returns ERR_PTR
> + * on error. May return -EINTR if it is interrupted by a signal.
> + *
> + * On success, the DMA addresses and lengths in the returned scatterlist are
> + * PAGE_SIZE aligned.
> + *
> + * A mapping must be unmapped by using dma_buf_unmap_attachment(). Note that
> + * the underlying backing storage is pinned for as long as a mapping exists,
> + * therefore users/importers should not hold onto a mapping for undue amounts of
> + * time.
> + *
> + * Important: Dynamic importers must wait for the exclusive fence of the struct
> + * dma_resv attached to the DMA-BUF first.
>    */
> -void dma_buf_unmap_attachment(struct dma_buf_attachment *attach,
> -				struct sg_table *sg_table,
> +struct sg_table *
> +dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   				enum dma_data_direction direction)
>   {
> +	struct sg_table *sg_table;
> +
>   	might_sleep();
>   
> -	if (WARN_ON(!attach || !attach->dmabuf || !sg_table))
> -		return;
> +	if (WARN_ON(!attach || !attach->dmabuf))
> +		return ERR_PTR(-EINVAL);
> +
> +	dma_resv_lock(attach->dmabuf->resv, NULL);
> +	sg_table = dma_buf_map_attachment_locked(attach, direction);
> +	dma_resv_unlock(attach->dmabuf->resv);
>   
> -	if (dma_buf_attachment_is_dynamic(attach))
> -		dma_resv_assert_held(attach->dmabuf->resv);
> +	return sg_table;
> +}
> +EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment, DMA_BUF);
> +
> +/**
> + * dma_buf_unmap_attachment_locked - Returns the scatterlist table of the attachment;
> + * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
> + * dma_buf_ops.
> + * @attach:	[in]	attachment whose scatterlist is to be returned
> + * @direction:	[in]	direction of DMA transfer
> + *
> + * Locked variant of dma_buf_unmap_attachment().
> + *
> + * Caller is responsible for holding dmabuf's reservation lock.
> + */
> +void dma_buf_unmap_attachment_locked(struct dma_buf_attachment *attach,
> +				     struct sg_table *sg_table,
> +				     enum dma_data_direction direction)
> +{
> +	might_sleep();
> +
> +	dma_resv_assert_held(attach->dmabuf->resv);
>   
>   	if (attach->sgt == sg_table)
>   		return;
>   
> -	if (dma_buf_is_dynamic(attach->dmabuf))
> -		dma_resv_assert_held(attach->dmabuf->resv);
> -
>   	__unmap_dma_buf(attach, sg_table, direction);
>   
>   	if (dma_buf_is_dynamic(attach->dmabuf) &&
>   	    !IS_ENABLED(CONFIG_DMABUF_MOVE_NOTIFY))
>   		dma_buf_unpin(attach);
>   }
> +EXPORT_SYMBOL_NS_GPL(dma_buf_unmap_attachment_locked, DMA_BUF);
> +
> +/**
> + * dma_buf_unmap_attachment - unmaps and decreases usecount of the buffer;might
> + * deallocate the scatterlist associated. Is a wrapper for unmap_dma_buf() of
> + * dma_buf_ops.
> + * @attach:	[in]	attachment to unmap buffer from
> + * @sg_table:	[in]	scatterlist info of the buffer to unmap
> + * @direction:  [in]    direction of DMA transfer
> + *
> + * This unmaps a DMA mapping for @attached obtained by dma_buf_map_attachment().
> + */
> +void dma_buf_unmap_attachment(struct dma_buf_attachment *attach,
> +			      struct sg_table *sg_table,
> +			      enum dma_data_direction direction)
> +{
> +	might_sleep();
> +
> +	if (WARN_ON(!attach || !attach->dmabuf || !sg_table))
> +		return;
> +
> +	dma_resv_lock(attach->dmabuf->resv, NULL);
> +	dma_buf_unmap_attachment_locked(attach, sg_table, direction);
> +	dma_resv_unlock(attach->dmabuf->resv);
> +}
>   EXPORT_SYMBOL_NS_GPL(dma_buf_unmap_attachment, DMA_BUF);
>   
>   /**
> @@ -1224,6 +1269,31 @@ int dma_buf_end_cpu_access(struct dma_buf *dmabuf,
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_end_cpu_access, DMA_BUF);
>   
> +static int dma_buf_mmap_locked(struct dma_buf *dmabuf,
> +			       struct vm_area_struct *vma,
> +			       unsigned long pgoff)
> +{
> +	dma_resv_assert_held(dmabuf->resv);
> +
> +	/* check if buffer supports mmap */
> +	if (!dmabuf->ops->mmap)
> +		return -EINVAL;
> +
> +	/* check for offset overflow */
> +	if (pgoff + vma_pages(vma) < pgoff)
> +		return -EOVERFLOW;
> +
> +	/* check for overflowing the buffer's size */
> +	if (pgoff + vma_pages(vma) >
> +	    dmabuf->size >> PAGE_SHIFT)
> +		return -EINVAL;
> +
> +	/* readjust the vma */
> +	vma_set_file(vma, dmabuf->file);
> +	vma->vm_pgoff = pgoff;
> +
> +	return dmabuf->ops->mmap(dmabuf, vma);
> +}
>   
>   /**
>    * dma_buf_mmap - Setup up a userspace mmap with the given vma
> @@ -1242,29 +1312,46 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_end_cpu_access, DMA_BUF);
>   int dma_buf_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma,
>   		 unsigned long pgoff)
>   {
> +	int ret;
> +
>   	if (WARN_ON(!dmabuf || !vma))
>   		return -EINVAL;
>   
> -	/* check if buffer supports mmap */
> -	if (!dmabuf->ops->mmap)
> -		return -EINVAL;
> +	dma_resv_lock(dmabuf->resv, NULL);
> +	ret = dma_buf_mmap_locked(dmabuf, vma, pgoff);
> +	dma_resv_unlock(dmabuf->resv);
>   
> -	/* check for offset overflow */
> -	if (pgoff + vma_pages(vma) < pgoff)
> -		return -EOVERFLOW;
> +	return ret;
> +}
> +EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
>   
> -	/* check for overflowing the buffer's size */
> -	if (pgoff + vma_pages(vma) >
> -	    dmabuf->size >> PAGE_SHIFT)
> -		return -EINVAL;
> +static int dma_buf_vmap_locked(struct dma_buf *dmabuf, struct iosys_map *map)
> +{
> +	struct iosys_map ptr;
> +	int ret;
>   
> -	/* readjust the vma */
> -	vma_set_file(vma, dmabuf->file);
> -	vma->vm_pgoff = pgoff;
> +	dma_resv_assert_held(dmabuf->resv);
>   
> -	return dmabuf->ops->mmap(dmabuf, vma);
> +	if (dmabuf->vmapping_counter) {
> +		dmabuf->vmapping_counter++;
> +		BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
> +		*map = dmabuf->vmap_ptr;
> +		return ret;
> +	}
> +
> +	BUG_ON(iosys_map_is_set(&dmabuf->vmap_ptr));
> +
> +	ret = dmabuf->ops->vmap(dmabuf, &ptr);
> +	if (WARN_ON_ONCE(ret))
> +		return ret;
> +
> +	dmabuf->vmap_ptr = ptr;
> +	dmabuf->vmapping_counter = 1;
> +
> +	*map = dmabuf->vmap_ptr;
> +
> +	return 0;
>   }
> -EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
>   
>   /**
>    * dma_buf_vmap - Create virtual mapping for the buffer object into kernel
> @@ -1284,8 +1371,7 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
>    */
>   int dma_buf_vmap(struct dma_buf *dmabuf, struct iosys_map *map)
>   {
> -	struct iosys_map ptr;
> -	int ret = 0;
> +	int ret;
>   
>   	iosys_map_clear(map);
>   
> @@ -1295,52 +1381,40 @@ int dma_buf_vmap(struct dma_buf *dmabuf, struct iosys_map *map)
>   	if (!dmabuf->ops->vmap)
>   		return -EINVAL;
>   
> -	mutex_lock(&dmabuf->lock);
> -	if (dmabuf->vmapping_counter) {
> -		dmabuf->vmapping_counter++;
> -		BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
> -		*map = dmabuf->vmap_ptr;
> -		goto out_unlock;
> -	}
> -
> -	BUG_ON(iosys_map_is_set(&dmabuf->vmap_ptr));
> -
> -	ret = dmabuf->ops->vmap(dmabuf, &ptr);
> -	if (WARN_ON_ONCE(ret))
> -		goto out_unlock;
> -
> -	dmabuf->vmap_ptr = ptr;
> -	dmabuf->vmapping_counter = 1;
> -
> -	*map = dmabuf->vmap_ptr;
> +	dma_resv_lock(dmabuf->resv, NULL);
> +	ret = dma_buf_vmap_locked(dmabuf, map);
> +	dma_resv_unlock(dmabuf->resv);
>   
> -out_unlock:
> -	mutex_unlock(&dmabuf->lock);
>   	return ret;
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_vmap, DMA_BUF);
>   
> -/**
> - * dma_buf_vunmap - Unmap a vmap obtained by dma_buf_vmap.
> - * @dmabuf:	[in]	buffer to vunmap
> - * @map:	[in]	vmap pointer to vunmap
> - */
> -void dma_buf_vunmap(struct dma_buf *dmabuf, struct iosys_map *map)
> +static void dma_buf_vunmap_locked(struct dma_buf *dmabuf, struct iosys_map *map)
>   {
> -	if (WARN_ON(!dmabuf))
> -		return;
> -
>   	BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
>   	BUG_ON(dmabuf->vmapping_counter == 0);
>   	BUG_ON(!iosys_map_is_equal(&dmabuf->vmap_ptr, map));
>   
> -	mutex_lock(&dmabuf->lock);
>   	if (--dmabuf->vmapping_counter == 0) {
>   		if (dmabuf->ops->vunmap)
>   			dmabuf->ops->vunmap(dmabuf, map);
>   		iosys_map_clear(&dmabuf->vmap_ptr);
>   	}
> -	mutex_unlock(&dmabuf->lock);
> +}
> +
> +/**
> + * dma_buf_vunmap - Unmap a vmap obtained by dma_buf_vmap.
> + * @dmabuf:	[in]	buffer to vunmap
> + * @map:	[in]	vmap pointer to vunmap
> + */
> +void dma_buf_vunmap(struct dma_buf *dmabuf, struct iosys_map *map)
> +{
> +	if (WARN_ON(!dmabuf))
> +		return;
> +
> +	dma_resv_lock(dmabuf->resv, NULL);
> +	dma_buf_vunmap_locked(dmabuf, map);
> +	dma_resv_unlock(dmabuf->resv);
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_vunmap, DMA_BUF);
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> index be6f76a30ac6..b704bdf5601a 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> @@ -882,7 +882,8 @@ static int amdgpu_ttm_backend_bind(struct ttm_device *bdev,
>   			struct sg_table *sgt;
>   
>   			attach = gtt->gobj->import_attach;
> -			sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL);
> +			sgt = dma_buf_map_attachment_locked(attach,
> +							    DMA_BIDIRECTIONAL);
>   			if (IS_ERR(sgt))
>   				return PTR_ERR(sgt);
>   
> @@ -1007,7 +1008,8 @@ static void amdgpu_ttm_backend_unbind(struct ttm_device *bdev,
>   		struct dma_buf_attachment *attach;
>   
>   		attach = gtt->gobj->import_attach;
> -		dma_buf_unmap_attachment(attach, ttm->sg, DMA_BIDIRECTIONAL);
> +		dma_buf_unmap_attachment_locked(attach, ttm->sg,
> +						DMA_BIDIRECTIONAL);
>   		ttm->sg = NULL;
>   	}
>   
> diff --git a/drivers/gpu/drm/drm_client.c b/drivers/gpu/drm/drm_client.c
> index af3b7395bf69..e9a1cd310352 100644
> --- a/drivers/gpu/drm/drm_client.c
> +++ b/drivers/gpu/drm/drm_client.c
> @@ -323,7 +323,7 @@ drm_client_buffer_vmap(struct drm_client_buffer *buffer,
>   	 * fd_install step out of the driver backend hooks, to make that
>   	 * final step optional for internal users.
>   	 */
> -	ret = drm_gem_vmap(buffer->gem, map);
> +	ret = drm_gem_vmap_unlocked(buffer->gem, map);
>   	if (ret)
>   		return ret;
>   
> @@ -345,7 +345,7 @@ void drm_client_buffer_vunmap(struct drm_client_buffer *buffer)
>   {
>   	struct iosys_map *map = &buffer->map;
>   
> -	drm_gem_vunmap(buffer->gem, map);
> +	drm_gem_vunmap_unlocked(buffer->gem, map);
>   }
>   EXPORT_SYMBOL(drm_client_buffer_vunmap);
>   
> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
> index 7c0b025508e4..c61674887582 100644
> --- a/drivers/gpu/drm/drm_gem.c
> +++ b/drivers/gpu/drm/drm_gem.c
> @@ -1053,7 +1053,12 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,
>   	vma->vm_ops = obj->funcs->vm_ops;
>   
>   	if (obj->funcs->mmap) {
> +		ret = dma_resv_lock_interruptible(obj->resv, NULL);
> +		if (ret)
> +			goto err_drm_gem_object_put;
> +
>   		ret = obj->funcs->mmap(obj, vma);
> +		dma_resv_unlock(obj->resv);
>   		if (ret)
>   			goto err_drm_gem_object_put;
>   		WARN_ON(!(vma->vm_flags & VM_DONTEXPAND));
> @@ -1158,6 +1163,8 @@ void drm_gem_print_info(struct drm_printer *p, unsigned int indent,
>   
>   int drm_gem_pin(struct drm_gem_object *obj)
>   {
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (obj->funcs->pin)
>   		return obj->funcs->pin(obj);
>   	else
> @@ -1166,6 +1173,8 @@ int drm_gem_pin(struct drm_gem_object *obj)
>   
>   void drm_gem_unpin(struct drm_gem_object *obj)
>   {
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (obj->funcs->unpin)
>   		obj->funcs->unpin(obj);
>   }
> @@ -1174,6 +1183,8 @@ int drm_gem_vmap(struct drm_gem_object *obj, struct iosys_map *map)
>   {
>   	int ret;
>   
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (!obj->funcs->vmap)
>   		return -EOPNOTSUPP;
>   
> @@ -1189,6 +1200,8 @@ EXPORT_SYMBOL(drm_gem_vmap);
>   
>   void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
>   {
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (iosys_map_is_null(map))
>   		return;
>   
> @@ -1200,6 +1213,26 @@ void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
>   }
>   EXPORT_SYMBOL(drm_gem_vunmap);
>   
> +int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
> +{
> +	int ret;
> +
> +	dma_resv_lock(obj->resv, NULL);
> +	ret = drm_gem_vmap(obj, map);
> +	dma_resv_unlock(obj->resv);
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL(drm_gem_vmap_unlocked);
> +
> +void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
> +{
> +	dma_resv_lock(obj->resv, NULL);
> +	drm_gem_vunmap(obj, map);
> +	dma_resv_unlock(obj->resv);
> +}
> +EXPORT_SYMBOL(drm_gem_vunmap_unlocked);
> +
>   /**
>    * drm_gem_lock_reservations - Sets up the ww context and acquires
>    * the lock on an array of GEM objects.
> diff --git a/drivers/gpu/drm/drm_gem_framebuffer_helper.c b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
> index f4619803acd0..a0bff53b158e 100644
> --- a/drivers/gpu/drm/drm_gem_framebuffer_helper.c
> +++ b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
> @@ -348,7 +348,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb,
>   			iosys_map_clear(&map[i]);
>   			continue;
>   		}
> -		ret = drm_gem_vmap(obj, &map[i]);
> +		ret = drm_gem_vmap_unlocked(obj, &map[i]);
>   		if (ret)
>   			goto err_drm_gem_vunmap;
>   	}
> @@ -370,7 +370,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb,
>   		obj = drm_gem_fb_get_obj(fb, i);
>   		if (!obj)
>   			continue;
> -		drm_gem_vunmap(obj, &map[i]);
> +		drm_gem_vunmap_unlocked(obj, &map[i]);
>   	}
>   	return ret;
>   }
> @@ -398,7 +398,7 @@ void drm_gem_fb_vunmap(struct drm_framebuffer *fb,
>   			continue;
>   		if (iosys_map_is_null(&map[i]))
>   			continue;
> -		drm_gem_vunmap(obj, &map[i]);
> +		drm_gem_vunmap_unlocked(obj, &map[i]);
>   	}
>   }
>   EXPORT_SYMBOL(drm_gem_fb_vunmap);
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
> index f5062d0c6333..09502d490da8 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
> @@ -72,7 +72,7 @@ static int i915_gem_dmabuf_vmap(struct dma_buf *dma_buf,
>   	struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);
>   	void *vaddr;
>   
> -	vaddr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
> +	vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
>   	if (IS_ERR(vaddr))
>   		return PTR_ERR(vaddr);
>   
> @@ -241,8 +241,8 @@ static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
>   
>   	assert_object_held(obj);
>   
> -	pages = dma_buf_map_attachment(obj->base.import_attach,
> -				       DMA_BIDIRECTIONAL);
> +	pages = dma_buf_map_attachment_locked(obj->base.import_attach,
> +					      DMA_BIDIRECTIONAL);
>   	if (IS_ERR(pages))
>   		return PTR_ERR(pages);
>   
> @@ -270,8 +270,8 @@ static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
>   static void i915_gem_object_put_pages_dmabuf(struct drm_i915_gem_object *obj,
>   					     struct sg_table *pages)
>   {
> -	dma_buf_unmap_attachment(obj->base.import_attach, pages,
> -				 DMA_BIDIRECTIONAL);
> +	dma_buf_unmap_attachment_locked(obj->base.import_attach, pages,
> +					DMA_BIDIRECTIONAL);
>   }
>   
>   static const struct drm_i915_gem_object_ops i915_gem_object_dmabuf_ops = {
> diff --git a/drivers/gpu/drm/qxl/qxl_object.c b/drivers/gpu/drm/qxl/qxl_object.c
> index b42a657e4c2f..a64cd635fbc0 100644
> --- a/drivers/gpu/drm/qxl/qxl_object.c
> +++ b/drivers/gpu/drm/qxl/qxl_object.c
> @@ -168,9 +168,16 @@ int qxl_bo_vmap_locked(struct qxl_bo *bo, struct iosys_map *map)
>   		bo->map_count++;
>   		goto out;
>   	}
> -	r = ttm_bo_vmap(&bo->tbo, &bo->map);
> +
> +	r = __qxl_bo_pin(bo);
>   	if (r)
>   		return r;
> +
> +	r = ttm_bo_vmap(&bo->tbo, &bo->map);
> +	if (r) {
> +		__qxl_bo_unpin(bo);
> +		return r;
> +	}
>   	bo->map_count = 1;
>   
>   	/* TODO: Remove kptr in favor of map everywhere. */
> @@ -192,12 +199,6 @@ int qxl_bo_vmap(struct qxl_bo *bo, struct iosys_map *map)
>   	if (r)
>   		return r;
>   
> -	r = __qxl_bo_pin(bo);
> -	if (r) {
> -		qxl_bo_unreserve(bo);
> -		return r;
> -	}
> -
>   	r = qxl_bo_vmap_locked(bo, map);
>   	qxl_bo_unreserve(bo);
>   	return r;
> @@ -247,6 +248,7 @@ void qxl_bo_vunmap_locked(struct qxl_bo *bo)
>   		return;
>   	bo->kptr = NULL;
>   	ttm_bo_vunmap(&bo->tbo, &bo->map);
> +	__qxl_bo_unpin(bo);
>   }
>   
>   int qxl_bo_vunmap(struct qxl_bo *bo)
> @@ -258,7 +260,6 @@ int qxl_bo_vunmap(struct qxl_bo *bo)
>   		return r;
>   
>   	qxl_bo_vunmap_locked(bo);
> -	__qxl_bo_unpin(bo);
>   	qxl_bo_unreserve(bo);
>   	return 0;
>   }
> diff --git a/drivers/gpu/drm/qxl/qxl_prime.c b/drivers/gpu/drm/qxl/qxl_prime.c
> index 142d01415acb..9169c26357d3 100644
> --- a/drivers/gpu/drm/qxl/qxl_prime.c
> +++ b/drivers/gpu/drm/qxl/qxl_prime.c
> @@ -59,7 +59,7 @@ int qxl_gem_prime_vmap(struct drm_gem_object *obj, struct iosys_map *map)
>   	struct qxl_bo *bo = gem_to_qxl_bo(obj);
>   	int ret;
>   
> -	ret = qxl_bo_vmap(bo, map);
> +	ret = qxl_bo_vmap_locked(bo, map);
>   	if (ret < 0)
>   		return ret;
>   
> @@ -71,5 +71,5 @@ void qxl_gem_prime_vunmap(struct drm_gem_object *obj,
>   {
>   	struct qxl_bo *bo = gem_to_qxl_bo(obj);
>   
> -	qxl_bo_vunmap(bo);
> +	qxl_bo_vunmap_locked(bo);
>   }
> diff --git a/drivers/media/common/videobuf2/videobuf2-dma-contig.c b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
> index 678b359717c4..617062076370 100644
> --- a/drivers/media/common/videobuf2/videobuf2-dma-contig.c
> +++ b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
> @@ -382,18 +382,12 @@ static struct sg_table *vb2_dc_dmabuf_ops_map(
>   	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
>   {
>   	struct vb2_dc_attachment *attach = db_attach->priv;
> -	/* stealing dmabuf mutex to serialize map/unmap operations */
> -	struct mutex *lock = &db_attach->dmabuf->lock;
>   	struct sg_table *sgt;
>   
> -	mutex_lock(lock);
> -
>   	sgt = &attach->sgt;
>   	/* return previously mapped sg table */
> -	if (attach->dma_dir == dma_dir) {
> -		mutex_unlock(lock);
> +	if (attach->dma_dir == dma_dir)
>   		return sgt;
> -	}
>   
>   	/* release any previous cache */
>   	if (attach->dma_dir != DMA_NONE) {
> @@ -409,14 +403,11 @@ static struct sg_table *vb2_dc_dmabuf_ops_map(
>   	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir,
>   			    DMA_ATTR_SKIP_CPU_SYNC)) {
>   		pr_err("failed to map scatterlist\n");
> -		mutex_unlock(lock);
>   		return ERR_PTR(-EIO);
>   	}
>   
>   	attach->dma_dir = dma_dir;
>   
> -	mutex_unlock(lock);
> -
>   	return sgt;
>   }
>   
> diff --git a/drivers/media/common/videobuf2/videobuf2-dma-sg.c b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
> index fa69158a65b1..d2075e7078cd 100644
> --- a/drivers/media/common/videobuf2/videobuf2-dma-sg.c
> +++ b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
> @@ -424,18 +424,12 @@ static struct sg_table *vb2_dma_sg_dmabuf_ops_map(
>   	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
>   {
>   	struct vb2_dma_sg_attachment *attach = db_attach->priv;
> -	/* stealing dmabuf mutex to serialize map/unmap operations */
> -	struct mutex *lock = &db_attach->dmabuf->lock;
>   	struct sg_table *sgt;
>   
> -	mutex_lock(lock);
> -
>   	sgt = &attach->sgt;
>   	/* return previously mapped sg table */
> -	if (attach->dma_dir == dma_dir) {
> -		mutex_unlock(lock);
> +	if (attach->dma_dir == dma_dir)
>   		return sgt;
> -	}
>   
>   	/* release any previous cache */
>   	if (attach->dma_dir != DMA_NONE) {
> @@ -446,14 +440,11 @@ static struct sg_table *vb2_dma_sg_dmabuf_ops_map(
>   	/* mapping to the client with new direction */
>   	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir, 0)) {
>   		pr_err("failed to map scatterlist\n");
> -		mutex_unlock(lock);
>   		return ERR_PTR(-EIO);
>   	}
>   
>   	attach->dma_dir = dma_dir;
>   
> -	mutex_unlock(lock);
> -
>   	return sgt;
>   }
>   
> diff --git a/drivers/media/common/videobuf2/videobuf2-vmalloc.c b/drivers/media/common/videobuf2/videobuf2-vmalloc.c
> index 948152f1596b..3d00a7f0aac1 100644
> --- a/drivers/media/common/videobuf2/videobuf2-vmalloc.c
> +++ b/drivers/media/common/videobuf2/videobuf2-vmalloc.c
> @@ -267,18 +267,12 @@ static struct sg_table *vb2_vmalloc_dmabuf_ops_map(
>   	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
>   {
>   	struct vb2_vmalloc_attachment *attach = db_attach->priv;
> -	/* stealing dmabuf mutex to serialize map/unmap operations */
> -	struct mutex *lock = &db_attach->dmabuf->lock;
>   	struct sg_table *sgt;
>   
> -	mutex_lock(lock);
> -
>   	sgt = &attach->sgt;
>   	/* return previously mapped sg table */
> -	if (attach->dma_dir == dma_dir) {
> -		mutex_unlock(lock);
> +	if (attach->dma_dir == dma_dir)
>   		return sgt;
> -	}
>   
>   	/* release any previous cache */
>   	if (attach->dma_dir != DMA_NONE) {
> @@ -289,14 +283,11 @@ static struct sg_table *vb2_vmalloc_dmabuf_ops_map(
>   	/* mapping to the client with new direction */
>   	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir, 0)) {
>   		pr_err("failed to map scatterlist\n");
> -		mutex_unlock(lock);
>   		return ERR_PTR(-EIO);
>   	}
>   
>   	attach->dma_dir = dma_dir;
>   
> -	mutex_unlock(lock);
> -
>   	return sgt;
>   }
>   
> diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h
> index 9d7c61a122dc..0b427939f466 100644
> --- a/include/drm/drm_gem.h
> +++ b/include/drm/drm_gem.h
> @@ -410,4 +410,7 @@ void drm_gem_unlock_reservations(struct drm_gem_object **objs, int count,
>   int drm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev,
>   			    u32 handle, u64 *offset);
>   
> +int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
> +void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
> +
>   #endif /* __DRM_GEM_H__ */
> diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
> index 71731796c8c3..23698c6b1d1e 100644
> --- a/include/linux/dma-buf.h
> +++ b/include/linux/dma-buf.h
> @@ -326,15 +326,6 @@ struct dma_buf {
>   	/** @ops: dma_buf_ops associated with this buffer object. */
>   	const struct dma_buf_ops *ops;
>   
> -	/**
> -	 * @lock:
> -	 *
> -	 * Used internally to serialize list manipulation, attach/detach and
> -	 * vmap/unmap. Note that in many cases this is superseeded by
> -	 * dma_resv_lock() on @resv.
> -	 */
> -	struct mutex lock;
> -
>   	/**
>   	 * @vmapping_counter:
>   	 *
> @@ -618,6 +609,11 @@ int dma_buf_fd(struct dma_buf *dmabuf, int flags);
>   struct dma_buf *dma_buf_get(int fd);
>   void dma_buf_put(struct dma_buf *dmabuf);
>   
> +struct sg_table *dma_buf_map_attachment_locked(struct dma_buf_attachment *,
> +					       enum dma_data_direction);
> +void dma_buf_unmap_attachment_locked(struct dma_buf_attachment *,
> +				     struct sg_table *,
> +				     enum dma_data_direction);
>   struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *,
>   					enum dma_data_direction);
>   void dma_buf_unmap_attachment(struct dma_buf_attachment *, struct sg_table *,

Re: [Intel-gfx] [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Christian König]

Hi Dmitry,

First of all please separate out this patch from the rest of the series, 
since this is a complex separate structural change.

Am 27.05.22 um 01:50 schrieb Dmitry Osipenko:
> All dma-bufs have dma-reservation lock that allows drivers to perform
> exclusive operations over shared dma-bufs. Today's dma-buf API has
> incomplete locking specification, which creates dead lock situation
> for dma-buf importers and exporters that don't coordinate theirs locks.

Well please drop that sentence. The locking specifications are actually 
very well defined, it's just that some drivers are a bit broken 
regarding them.

What you do here is rather moving all the non-dynamic drivers over to 
the dynamic locking specification (which is really nice to have).

I have tried this before and failed because catching all the locks in 
the right code paths are very tricky. So expect some fallout from this 
and make sure the kernel test robot and CI systems are clean.

> This patch introduces new locking convention for dma-buf users. From now
> on all dma-buf importers are responsible for holding dma-buf reservation
> lock around operations performed over dma-bufs.
>
> This patch implements the new dma-buf locking convention by:
>
>    1. Making dma-buf API functions to take the reservation lock.
>
>    2. Adding new locked variants of the dma-buf API functions for drivers
>       that need to manage imported dma-bufs under the held lock.

Instead of adding new locked variants please mark all variants which 
expect to be called without a lock with an _unlocked postfix.

This should make it easier to remove those in a follow up patch set and 
then fully move the locking into the importer.

>    3. Converting all drivers to the new locking scheme.

I have strong doubts that you got all of them. At least radeon and 
nouveau should grab the reservation lock in their ->attach callbacks 
somehow.

>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>   drivers/dma-buf/dma-buf.c                     | 270 +++++++++++-------
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
>   drivers/gpu/drm/drm_client.c                  |   4 +-
>   drivers/gpu/drm/drm_gem.c                     |  33 +++
>   drivers/gpu/drm/drm_gem_framebuffer_helper.c  |   6 +-
>   drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    |  10 +-
>   drivers/gpu/drm/qxl/qxl_object.c              |  17 +-
>   drivers/gpu/drm/qxl/qxl_prime.c               |   4 +-
>   .../common/videobuf2/videobuf2-dma-contig.c   |  11 +-
>   .../media/common/videobuf2/videobuf2-dma-sg.c |  11 +-
>   .../common/videobuf2/videobuf2-vmalloc.c      |  11 +-
>   include/drm/drm_gem.h                         |   3 +
>   include/linux/dma-buf.h                       |  14 +-
>   13 files changed, 241 insertions(+), 159 deletions(-)
>
> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
> index 32f55640890c..64a9909ccfa2 100644
> --- a/drivers/dma-buf/dma-buf.c
> +++ b/drivers/dma-buf/dma-buf.c
> @@ -552,7 +552,6 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)
>   	file->f_mode |= FMODE_LSEEK;
>   	dmabuf->file = file;
>   
> -	mutex_init(&dmabuf->lock);

Please make removing dmabuf->lock a separate change.

Regards,
Christian.

>   	INIT_LIST_HEAD(&dmabuf->attachments);
>   
>   	mutex_lock(&db_list.lock);
> @@ -737,14 +736,14 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   	attach->importer_ops = importer_ops;
>   	attach->importer_priv = importer_priv;  3. Converting all drivers to the new locking scheme.
>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>   drivers/dma-buf/dma-buf.c                     | 270 +++++++++++-------
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
>   drivers/gpu/drm/drm_client.c                  |   4 +-
>
>   
> +	dma_resv_lock(dmabuf->resv, NULL);
> +
>   	if (dmabuf->ops->attach) {
>   		ret = dmabuf->ops->attach(dmabuf, attach);
>   		if (ret)
>   			goto err_attach;
>   	}
> -	dma_resv_lock(dmabuf->resv, NULL);
>   	list_add(&attach->node, &dmabuf->attachments);
> -	dma_resv_unlock(dmabuf->resv);
>   
>   	/* When either the importer or the exporter can't handle dynamic
>   	 * mappings we cache the mapping here to avoid issues with the
> @@ -755,7 +754,6 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   		struct sg_table *sgt;
>   
>   		if (dma_buf_is_dynamic(attach->dmabuf)) {
> -			dma_resv_lock(attach->dmabuf->resv, NULL);
>   			ret = dmabuf->ops->pin(attach);
>   			if (ret)
>   				goto err_unlock;
> @@ -768,15 +766,16 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   			ret = PTR_ERR(sgt);
>   			goto err_unpin;
>   		}
> -		if (dma_buf_is_dynamic(attach->dmabuf))
> -			dma_resv_unlock(attach->dmabuf->resv);
>   		attach->sgt = sgt;
>   		attach->dir = DMA_BIDIRECTIONAL;
>   	}
>   
> +	dma_resv_unlock(dmabuf->resv);
> +
>   	return attach;
>   
>   err_attach:
> +	dma_resv_unlock(attach->dmabuf->resv);
>   	kfree(attach);
>   	return ERR_PTR(ret);
>   
> @@ -785,10 +784,10 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
>   		dmabuf->ops->unpin(attach);
>   
>   err_unlock:
> -	if (dma_buf_is_dynamic(attach->dmabuf))
> -		dma_resv_unlock(attach->dmabuf->resv);
> +	dma_resv_unlock(dmabuf->resv);
>   
>   	dma_buf_detach(dmabuf, attach);
> +
>   	return ERR_PTR(ret);
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_dynamic_attach, DMA_BUF);
> @@ -832,24 +831,23 @@ void dma_buf_detach(struct dma_buf *dmabuf, struct dma_buf_attachment *attach)
>   	if (WARN_ON(!dmabuf || !attach))
>   		return;
>   
> -	if (attach->sgt) {
> -		if (dma_buf_is_dynamic(attach->dmabuf))
> -			dma_resv_lock(attach->dmabuf->resv, NULL);
> +	if (WARN_ON(dmabuf != attach->dmabuf))
> +		return;
>   
> +	dma_resv_lock(dmabuf->resv, NULL);
> +
> +	if (attach->sgt) {
>   		__unmap_dma_buf(attach, attach->sgt, attach->dir);
>   
> -		if (dma_buf_is_dynamic(attach->dmabuf)) {
> +		if (dma_buf_is_dynamic(attach->dmabuf))
>   			dmabuf->ops->unpin(attach);
> -			dma_resv_unlock(attach->dmabuf->resv);
> -		}
>   	}
>   
> -	dma_resv_lock(dmabuf->resv, NULL);
>   	list_del(&attach->node);
> -	dma_resv_unlock(dmabuf->resv);
>   	if (dmabuf->ops->detach)
>   		dmabuf->ops->detach(dmabuf, attach);
>   
> +	dma_resv_unlock(dmabuf->resv);
>   	kfree(attach);
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_detach, DMA_BUF);
> @@ -906,28 +904,18 @@ void dma_buf_unpin(struct dma_buf_attachment *attach)
>   EXPORT_SYMBOL_NS_GPL(dma_buf_unpin, DMA_BUF);
>   
>   /**
> - * dma_buf_map_attachment - Returns the scatterlist table of the attachment;
> + * dma_buf_map_attachment_locked - Returns the scatterlist table of the attachment;
>    * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
>    * dma_buf_ops.
>    * @attach:	[in]	attachment whose scatterlist is to be returned
>    * @direction:	[in]	direction of DMA transfer
>    *
> - * Returns sg_table containing the scatterlist to be returned; returns ERR_PTR
> - * on error. May return -EINTR if it is interrupted by a signal.
> - *
> - * On success, the DMA addresses and lengths in the returned scatterlist are
> - * PAGE_SIZE aligned.
> - *
> - * A mapping must be unmapped by using dma_buf_unmap_attachment(). Note that
> - * the underlying backing storage is pinned for as long as a mapping exists,
> - * therefore users/importers should not hold onto a mapping for undue amounts of
> - * time.
> + * Locked variant of dma_buf_map_attachment().
>    *
> - * Important: Dynamic importers must wait for the exclusive fence of the struct
> - * dma_resv attached to the DMA-BUF first.
> + * Caller is responsible for holding dmabuf's reservation lock.
>    */
> -struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
> -					enum dma_data_direction direction)
> +struct sg_table *dma_buf_map_attachment_locked(struct dma_buf_attachment *attach,
> +					       enum dma_data_direction direction)
>   {
>   	struct sg_table *sg_table;
>   	int r;
> @@ -937,8 +925,7 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   	if (WARN_ON(!attach || !attach->dmabuf))
>   		return ERR_PTR(-EINVAL);
>   
> -	if (dma_buf_attachment_is_dynamic(attach))
> -		dma_resv_assert_held(attach->dmabuf->resv);
> +	dma_resv_assert_held(attach->dmabuf->resv);
>   
>   	if (attach->sgt) {
>   		/*
> @@ -953,7 +940,6 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   	}
>   
>   	if (dma_buf_is_dynamic(attach->dmabuf)) {
> -		dma_resv_assert_held(attach->dmabuf->resv);
>   		if (!IS_ENABLED(CONFIG_DMABUF_MOVE_NOTIFY)) {
>   			r = attach->dmabuf->ops->pin(attach);
>   			if (r)
> @@ -993,42 +979,101 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   #endif /* CONFIG_DMA_API_DEBUG */
>   	return sg_table;
>   }
> -EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment, DMA_BUF);
> +EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment_locked, DMA_BUF);
>   
>   /**
> - * dma_buf_unmap_attachment - unmaps and decreases usecount of the buffer;might
> - * deallocate the scatterlist associated. Is a wrapper for unmap_dma_buf() of
> + * dma_buf_map_attachment - Returns the scatterlist table of the attachment;
> + * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
>    * dma_buf_ops.
> - * @attach:	[in]	attachment to unmap buffer from
> - * @sg_table:	[in]	scatterlist info of the buffer to unmap
> - * @direction:  [in]    direction of DMA transfer
> + * @attach:	[in]	attachment whose scatterlist is to be returned
> + * @direction:	[in]	direction of DMA transfer
>    *
> - * This unmaps a DMA mapping for @attached obtained by dma_buf_map_attachment().
> + * Returns sg_table containing the scatterlist to be returned; returns ERR_PTR
> + * on error. May return -EINTR if it is interrupted by a signal.
> + *
> + * On success, the DMA addresses and lengths in the returned scatterlist are
> + * PAGE_SIZE aligned.
> + *
> + * A mapping must be unmapped by using dma_buf_unmap_attachment(). Note that
> + * the underlying backing storage is pinned for as long as a mapping exists,
> + * therefore users/importers should not hold onto a mapping for undue amounts of
> + * time.
> + *
> + * Important: Dynamic importers must wait for the exclusive fence of the struct
> + * dma_resv attached to the DMA-BUF first.
>    */
> -void dma_buf_unmap_attachment(struct dma_buf_attachment *attach,
> -				struct sg_table *sg_table,
> +struct sg_table *
> +dma_buf_map_attachment(struct dma_buf_attachment *attach,
>   				enum dma_data_direction direction)
>   {
> +	struct sg_table *sg_table;
> +
>   	might_sleep();
>   
> -	if (WARN_ON(!attach || !attach->dmabuf || !sg_table))
> -		return;
> +	if (WARN_ON(!attach || !attach->dmabuf))
> +		return ERR_PTR(-EINVAL);
> +
> +	dma_resv_lock(attach->dmabuf->resv, NULL);
> +	sg_table = dma_buf_map_attachment_locked(attach, direction);
> +	dma_resv_unlock(attach->dmabuf->resv);
>   
> -	if (dma_buf_attachment_is_dynamic(attach))
> -		dma_resv_assert_held(attach->dmabuf->resv);
> +	return sg_table;
> +}
> +EXPORT_SYMBOL_NS_GPL(dma_buf_map_attachment, DMA_BUF);
> +
> +/**
> + * dma_buf_unmap_attachment_locked - Returns the scatterlist table of the attachment;
> + * mapped into _device_ address space. Is a wrapper for map_dma_buf() of the
> + * dma_buf_ops.
> + * @attach:	[in]	attachment whose scatterlist is to be returned
> + * @direction:	[in]	direction of DMA transfer
> + *
> + * Locked variant of dma_buf_unmap_attachment().
> + *
> + * Caller is responsible for holding dmabuf's reservation lock.
> + */
> +void dma_buf_unmap_attachment_locked(struct dma_buf_attachment *attach,
> +				     struct sg_table *sg_table,
> +				     enum dma_data_direction direction)
> +{
> +	might_sleep();
> +
> +	dma_resv_assert_held(attach->dmabuf->resv);
>   
>   	if (attach->sgt == sg_table)
>   		return;
>   
> -	if (dma_buf_is_dynamic(attach->dmabuf))
> -		dma_resv_assert_held(attach->dmabuf->resv);
> -
>   	__unmap_dma_buf(attach, sg_table, direction);
>   
>   	if (dma_buf_is_dynamic(attach->dmabuf) &&
>   	    !IS_ENABLED(CONFIG_DMABUF_MOVE_NOTIFY))
>   		dma_buf_unpin(attach);
>   }
> +EXPORT_SYMBOL_NS_GPL(dma_buf_unmap_attachment_locked, DMA_BUF);
> +
> +/**
> + * dma_buf_unmap_attachment - unmaps and decreases usecount of the buffer;might
> + * deallocate the scatterlist associated. Is a wrapper for unmap_dma_buf() of
> + * dma_buf_ops.
> + * @attach:	[in]	attachment to unmap buffer from
> + * @sg_table:	[in]	scatterlist info of the buffer to unmap
> + * @direction:  [in]    direction of DMA transfer
> + *
> + * This unmaps a DMA mapping for @attached obtained by dma_buf_map_attachment().
> + */
> +void dma_buf_unmap_attachment(struct dma_buf_attachment *attach,
> +			      struct sg_table *sg_table,
> +			      enum dma_data_direction direction)
> +{
> +	might_sleep();
> +
> +	if (WARN_ON(!attach || !attach->dmabuf || !sg_table))
> +		return;
> +
> +	dma_resv_lock(attach->dmabuf->resv, NULL);
> +	dma_buf_unmap_attachment_locked(attach, sg_table, direction);
> +	dma_resv_unlock(attach->dmabuf->resv);
> +}
>   EXPORT_SYMBOL_NS_GPL(dma_buf_unmap_attachment, DMA_BUF);
>   
>   /**
> @@ -1224,6 +1269,31 @@ int dma_buf_end_cpu_access(struct dma_buf *dmabuf,
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_end_cpu_access, DMA_BUF);
>   
> +static int dma_buf_mmap_locked(struct dma_buf *dmabuf,
> +			       struct vm_area_struct *vma,
> +			       unsigned long pgoff)
> +{
> +	dma_resv_assert_held(dmabuf->resv);
> +
> +	/* check if buffer supports mmap */
> +	if (!dmabuf->ops->mmap)
> +		return -EINVAL;
> +
> +	/* check for offset overflow */
> +	if (pgoff + vma_pages(vma) < pgoff)
> +		return -EOVERFLOW;
> +
> +	/* check for overflowing the buffer's size */
> +	if (pgoff + vma_pages(vma) >
> +	    dmabuf->size >> PAGE_SHIFT)
> +		return -EINVAL;
> +
> +	/* readjust the vma */
> +	vma_set_file(vma, dmabuf->file);
> +	vma->vm_pgoff = pgoff;
> +
> +	return dmabuf->ops->mmap(dmabuf, vma);
> +}
>   
>   /**
>    * dma_buf_mmap - Setup up a userspace mmap with the given vma
> @@ -1242,29 +1312,46 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_end_cpu_access, DMA_BUF);
>   int dma_buf_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma,
>   		 unsigned long pgoff)
>   {
> +	int ret;
> +
>   	if (WARN_ON(!dmabuf || !vma))
>   		return -EINVAL;
>   
> -	/* check if buffer supports mmap */
> -	if (!dmabuf->ops->mmap)
> -		return -EINVAL;
> +	dma_resv_lock(dmabuf->resv, NULL);
> +	ret = dma_buf_mmap_locked(dmabuf, vma, pgoff);
> +	dma_resv_unlock(dmabuf->resv);
>   
> -	/* check for offset overflow */
> -	if (pgoff + vma_pages(vma) < pgoff)
> -		return -EOVERFLOW;
> +	return ret;
> +}
> +EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
>   
> -	/* check for overflowing the buffer's size */
> -	if (pgoff + vma_pages(vma) >
> -	    dmabuf->size >> PAGE_SHIFT)
> -		return -EINVAL;
> +static int dma_buf_vmap_locked(struct dma_buf *dmabuf, struct iosys_map *map)
> +{
> +	struct iosys_map ptr;
> +	int ret;
>   
> -	/* readjust the vma */
> -	vma_set_file(vma, dmabuf->file);
> -	vma->vm_pgoff = pgoff;
> +	dma_resv_assert_held(dmabuf->resv);
>   
> -	return dmabuf->ops->mmap(dmabuf, vma);
> +	if (dmabuf->vmapping_counter) {
> +		dmabuf->vmapping_counter++;
> +		BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
> +		*map = dmabuf->vmap_ptr;
> +		return ret;
> +	}
> +
> +	BUG_ON(iosys_map_is_set(&dmabuf->vmap_ptr));
> +
> +	ret = dmabuf->ops->vmap(dmabuf, &ptr);
> +	if (WARN_ON_ONCE(ret))
> +		return ret;
> +
> +	dmabuf->vmap_ptr = ptr;
> +	dmabuf->vmapping_counter = 1;
> +
> +	*map = dmabuf->vmap_ptr;
> +
> +	return 0;
>   }
> -EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
>   
>   /**
>    * dma_buf_vmap - Create virtual mapping for the buffer object into kernel
> @@ -1284,8 +1371,7 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_mmap, DMA_BUF);
>    */
>   int dma_buf_vmap(struct dma_buf *dmabuf, struct iosys_map *map)
>   {
> -	struct iosys_map ptr;
> -	int ret = 0;
> +	int ret;
>   
>   	iosys_map_clear(map);
>   
> @@ -1295,52 +1381,40 @@ int dma_buf_vmap(struct dma_buf *dmabuf, struct iosys_map *map)
>   	if (!dmabuf->ops->vmap)
>   		return -EINVAL;
>   
> -	mutex_lock(&dmabuf->lock);
> -	if (dmabuf->vmapping_counter) {
> -		dmabuf->vmapping_counter++;
> -		BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
> -		*map = dmabuf->vmap_ptr;
> -		goto out_unlock;
> -	}
> -
> -	BUG_ON(iosys_map_is_set(&dmabuf->vmap_ptr));
> -
> -	ret = dmabuf->ops->vmap(dmabuf, &ptr);
> -	if (WARN_ON_ONCE(ret))
> -		goto out_unlock;
> -
> -	dmabuf->vmap_ptr = ptr;
> -	dmabuf->vmapping_counter = 1;
> -
> -	*map = dmabuf->vmap_ptr;
> +	dma_resv_lock(dmabuf->resv, NULL);
> +	ret = dma_buf_vmap_locked(dmabuf, map);
> +	dma_resv_unlock(dmabuf->resv);
>   
> -out_unlock:
> -	mutex_unlock(&dmabuf->lock);
>   	return ret;
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_vmap, DMA_BUF);
>   
> -/**
> - * dma_buf_vunmap - Unmap a vmap obtained by dma_buf_vmap.
> - * @dmabuf:	[in]	buffer to vunmap
> - * @map:	[in]	vmap pointer to vunmap
> - */
> -void dma_buf_vunmap(struct dma_buf *dmabuf, struct iosys_map *map)
> +static void dma_buf_vunmap_locked(struct dma_buf *dmabuf, struct iosys_map *map)
>   {
> -	if (WARN_ON(!dmabuf))
> -		return;
> -
>   	BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
>   	BUG_ON(dmabuf->vmapping_counter == 0);
>   	BUG_ON(!iosys_map_is_equal(&dmabuf->vmap_ptr, map));
>   
> -	mutex_lock(&dmabuf->lock);
>   	if (--dmabuf->vmapping_counter == 0) {
>   		if (dmabuf->ops->vunmap)
>   			dmabuf->ops->vunmap(dmabuf, map);
>   		iosys_map_clear(&dmabuf->vmap_ptr);
>   	}
> -	mutex_unlock(&dmabuf->lock);
> +}
> +
> +/**
> + * dma_buf_vunmap - Unmap a vmap obtained by dma_buf_vmap.
> + * @dmabuf:	[in]	buffer to vunmap
> + * @map:	[in]	vmap pointer to vunmap
> + */
> +void dma_buf_vunmap(struct dma_buf *dmabuf, struct iosys_map *map)
> +{
> +	if (WARN_ON(!dmabuf))
> +		return;
> +
> +	dma_resv_lock(dmabuf->resv, NULL);
> +	dma_buf_vunmap_locked(dmabuf, map);
> +	dma_resv_unlock(dmabuf->resv);
>   }
>   EXPORT_SYMBOL_NS_GPL(dma_buf_vunmap, DMA_BUF);
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> index be6f76a30ac6..b704bdf5601a 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> @@ -882,7 +882,8 @@ static int amdgpu_ttm_backend_bind(struct ttm_device *bdev,
>   			struct sg_table *sgt;
>   
>   			attach = gtt->gobj->import_attach;
> -			sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL);
> +			sgt = dma_buf_map_attachment_locked(attach,
> +							    DMA_BIDIRECTIONAL);
>   			if (IS_ERR(sgt))
>   				return PTR_ERR(sgt);
>   
> @@ -1007,7 +1008,8 @@ static void amdgpu_ttm_backend_unbind(struct ttm_device *bdev,
>   		struct dma_buf_attachment *attach;
>   
>   		attach = gtt->gobj->import_attach;
> -		dma_buf_unmap_attachment(attach, ttm->sg, DMA_BIDIRECTIONAL);
> +		dma_buf_unmap_attachment_locked(attach, ttm->sg,
> +						DMA_BIDIRECTIONAL);
>   		ttm->sg = NULL;
>   	}
>   
> diff --git a/drivers/gpu/drm/drm_client.c b/drivers/gpu/drm/drm_client.c
> index af3b7395bf69..e9a1cd310352 100644
> --- a/drivers/gpu/drm/drm_client.c
> +++ b/drivers/gpu/drm/drm_client.c
> @@ -323,7 +323,7 @@ drm_client_buffer_vmap(struct drm_client_buffer *buffer,
>   	 * fd_install step out of the driver backend hooks, to make that
>   	 * final step optional for internal users.
>   	 */
> -	ret = drm_gem_vmap(buffer->gem, map);
> +	ret = drm_gem_vmap_unlocked(buffer->gem, map);
>   	if (ret)
>   		return ret;
>   
> @@ -345,7 +345,7 @@ void drm_client_buffer_vunmap(struct drm_client_buffer *buffer)
>   {
>   	struct iosys_map *map = &buffer->map;
>   
> -	drm_gem_vunmap(buffer->gem, map);
> +	drm_gem_vunmap_unlocked(buffer->gem, map);
>   }
>   EXPORT_SYMBOL(drm_client_buffer_vunmap);
>   
> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
> index 7c0b025508e4..c61674887582 100644
> --- a/drivers/gpu/drm/drm_gem.c
> +++ b/drivers/gpu/drm/drm_gem.c
> @@ -1053,7 +1053,12 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,
>   	vma->vm_ops = obj->funcs->vm_ops;
>   
>   	if (obj->funcs->mmap) {
> +		ret = dma_resv_lock_interruptible(obj->resv, NULL);
> +		if (ret)
> +			goto err_drm_gem_object_put;
> +
>   		ret = obj->funcs->mmap(obj, vma);
> +		dma_resv_unlock(obj->resv);
>   		if (ret)
>   			goto err_drm_gem_object_put;
>   		WARN_ON(!(vma->vm_flags & VM_DONTEXPAND));
> @@ -1158,6 +1163,8 @@ void drm_gem_print_info(struct drm_printer *p, unsigned int indent,
>   
>   int drm_gem_pin(struct drm_gem_object *obj)
>   {
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (obj->funcs->pin)
>   		return obj->funcs->pin(obj);
>   	else
> @@ -1166,6 +1173,8 @@ int drm_gem_pin(struct drm_gem_object *obj)
>   
>   void drm_gem_unpin(struct drm_gem_object *obj)
>   {
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (obj->funcs->unpin)
>   		obj->funcs->unpin(obj);
>   }
> @@ -1174,6 +1183,8 @@ int drm_gem_vmap(struct drm_gem_object *obj, struct iosys_map *map)
>   {
>   	int ret;
>   
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (!obj->funcs->vmap)
>   		return -EOPNOTSUPP;
>   
> @@ -1189,6 +1200,8 @@ EXPORT_SYMBOL(drm_gem_vmap);
>   
>   void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
>   {
> +	dma_resv_assert_held(obj->resv);
> +
>   	if (iosys_map_is_null(map))
>   		return;
>   
> @@ -1200,6 +1213,26 @@ void drm_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
>   }
>   EXPORT_SYMBOL(drm_gem_vunmap);
>   
> +int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
> +{
> +	int ret;
> +
> +	dma_resv_lock(obj->resv, NULL);
> +	ret = drm_gem_vmap(obj, map);
> +	dma_resv_unlock(obj->resv);
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL(drm_gem_vmap_unlocked);
> +
> +void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map)
> +{
> +	dma_resv_lock(obj->resv, NULL);
> +	drm_gem_vunmap(obj, map);
> +	dma_resv_unlock(obj->resv);
> +}
> +EXPORT_SYMBOL(drm_gem_vunmap_unlocked);
> +
>   /**
>    * drm_gem_lock_reservations - Sets up the ww context and acquires
>    * the lock on an array of GEM objects.
> diff --git a/drivers/gpu/drm/drm_gem_framebuffer_helper.c b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
> index f4619803acd0..a0bff53b158e 100644
> --- a/drivers/gpu/drm/drm_gem_framebuffer_helper.c
> +++ b/drivers/gpu/drm/drm_gem_framebuffer_helper.c
> @@ -348,7 +348,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb,
>   			iosys_map_clear(&map[i]);
>   			continue;
>   		}
> -		ret = drm_gem_vmap(obj, &map[i]);
> +		ret = drm_gem_vmap_unlocked(obj, &map[i]);
>   		if (ret)
>   			goto err_drm_gem_vunmap;
>   	}
> @@ -370,7 +370,7 @@ int drm_gem_fb_vmap(struct drm_framebuffer *fb,
>   		obj = drm_gem_fb_get_obj(fb, i);
>   		if (!obj)
>   			continue;
> -		drm_gem_vunmap(obj, &map[i]);
> +		drm_gem_vunmap_unlocked(obj, &map[i]);
>   	}
>   	return ret;
>   }
> @@ -398,7 +398,7 @@ void drm_gem_fb_vunmap(struct drm_framebuffer *fb,
>   			continue;
>   		if (iosys_map_is_null(&map[i]))
>   			continue;
> -		drm_gem_vunmap(obj, &map[i]);
> +		drm_gem_vunmap_unlocked(obj, &map[i]);
>   	}
>   }
>   EXPORT_SYMBOL(drm_gem_fb_vunmap);
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
> index f5062d0c6333..09502d490da8 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
> @@ -72,7 +72,7 @@ static int i915_gem_dmabuf_vmap(struct dma_buf *dma_buf,
>   	struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);
>   	void *vaddr;
>   
> -	vaddr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
> +	vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
>   	if (IS_ERR(vaddr))
>   		return PTR_ERR(vaddr);
>   
> @@ -241,8 +241,8 @@ static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
>   
>   	assert_object_held(obj);
>   
> -	pages = dma_buf_map_attachment(obj->base.import_attach,
> -				       DMA_BIDIRECTIONAL);
> +	pages = dma_buf_map_attachment_locked(obj->base.import_attach,
> +					      DMA_BIDIRECTIONAL);
>   	if (IS_ERR(pages))
>   		return PTR_ERR(pages);
>   
> @@ -270,8 +270,8 @@ static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
>   static void i915_gem_object_put_pages_dmabuf(struct drm_i915_gem_object *obj,
>   					     struct sg_table *pages)
>   {
> -	dma_buf_unmap_attachment(obj->base.import_attach, pages,
> -				 DMA_BIDIRECTIONAL);
> +	dma_buf_unmap_attachment_locked(obj->base.import_attach, pages,
> +					DMA_BIDIRECTIONAL);
>   }
>   
>   static const struct drm_i915_gem_object_ops i915_gem_object_dmabuf_ops = {
> diff --git a/drivers/gpu/drm/qxl/qxl_object.c b/drivers/gpu/drm/qxl/qxl_object.c
> index b42a657e4c2f..a64cd635fbc0 100644
> --- a/drivers/gpu/drm/qxl/qxl_object.c
> +++ b/drivers/gpu/drm/qxl/qxl_object.c
> @@ -168,9 +168,16 @@ int qxl_bo_vmap_locked(struct qxl_bo *bo, struct iosys_map *map)
>   		bo->map_count++;
>   		goto out;
>   	}
> -	r = ttm_bo_vmap(&bo->tbo, &bo->map);
> +
> +	r = __qxl_bo_pin(bo);
>   	if (r)
>   		return r;
> +
> +	r = ttm_bo_vmap(&bo->tbo, &bo->map);
> +	if (r) {
> +		__qxl_bo_unpin(bo);
> +		return r;
> +	}
>   	bo->map_count = 1;
>   
>   	/* TODO: Remove kptr in favor of map everywhere. */
> @@ -192,12 +199,6 @@ int qxl_bo_vmap(struct qxl_bo *bo, struct iosys_map *map)
>   	if (r)
>   		return r;
>   
> -	r = __qxl_bo_pin(bo);
> -	if (r) {
> -		qxl_bo_unreserve(bo);
> -		return r;
> -	}
> -
>   	r = qxl_bo_vmap_locked(bo, map);
>   	qxl_bo_unreserve(bo);
>   	return r;
> @@ -247,6 +248,7 @@ void qxl_bo_vunmap_locked(struct qxl_bo *bo)
>   		return;
>   	bo->kptr = NULL;
>   	ttm_bo_vunmap(&bo->tbo, &bo->map);
> +	__qxl_bo_unpin(bo);
>   }
>   
>   int qxl_bo_vunmap(struct qxl_bo *bo)
> @@ -258,7 +260,6 @@ int qxl_bo_vunmap(struct qxl_bo *bo)
>   		return r;
>   
>   	qxl_bo_vunmap_locked(bo);
> -	__qxl_bo_unpin(bo);
>   	qxl_bo_unreserve(bo);
>   	return 0;
>   }
> diff --git a/drivers/gpu/drm/qxl/qxl_prime.c b/drivers/gpu/drm/qxl/qxl_prime.c
> index 142d01415acb..9169c26357d3 100644
> --- a/drivers/gpu/drm/qxl/qxl_prime.c
> +++ b/drivers/gpu/drm/qxl/qxl_prime.c
> @@ -59,7 +59,7 @@ int qxl_gem_prime_vmap(struct drm_gem_object *obj, struct iosys_map *map)
>   	struct qxl_bo *bo = gem_to_qxl_bo(obj);
>   	int ret;
>   
> -	ret = qxl_bo_vmap(bo, map);
> +	ret = qxl_bo_vmap_locked(bo, map);
>   	if (ret < 0)
>   		return ret;
>   
> @@ -71,5 +71,5 @@ void qxl_gem_prime_vunmap(struct drm_gem_object *obj,
>   {
>   	struct qxl_bo *bo = gem_to_qxl_bo(obj);
>   
> -	qxl_bo_vunmap(bo);
> +	qxl_bo_vunmap_locked(bo);
>   }
> diff --git a/drivers/media/common/videobuf2/videobuf2-dma-contig.c b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
> index 678b359717c4..617062076370 100644
> --- a/drivers/media/common/videobuf2/videobuf2-dma-contig.c
> +++ b/drivers/media/common/videobuf2/videobuf2-dma-contig.c
> @@ -382,18 +382,12 @@ static struct sg_table *vb2_dc_dmabuf_ops_map(
>   	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
>   {
>   	struct vb2_dc_attachment *attach = db_attach->priv;
> -	/* stealing dmabuf mutex to serialize map/unmap operations */
> -	struct mutex *lock = &db_attach->dmabuf->lock;
>   	struct sg_table *sgt;
>   
> -	mutex_lock(lock);
> -
>   	sgt = &attach->sgt;
>   	/* return previously mapped sg table */
> -	if (attach->dma_dir == dma_dir) {
> -		mutex_unlock(lock);
> +	if (attach->dma_dir == dma_dir)
>   		return sgt;
> -	}
>   
>   	/* release any previous cache */
>   	if (attach->dma_dir != DMA_NONE) {
> @@ -409,14 +403,11 @@ static struct sg_table *vb2_dc_dmabuf_ops_map(
>   	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir,
>   			    DMA_ATTR_SKIP_CPU_SYNC)) {
>   		pr_err("failed to map scatterlist\n");
> -		mutex_unlock(lock);
>   		return ERR_PTR(-EIO);
>   	}
>   
>   	attach->dma_dir = dma_dir;
>   
> -	mutex_unlock(lock);
> -
>   	return sgt;
>   }
>   
> diff --git a/drivers/media/common/videobuf2/videobuf2-dma-sg.c b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
> index fa69158a65b1..d2075e7078cd 100644
> --- a/drivers/media/common/videobuf2/videobuf2-dma-sg.c
> +++ b/drivers/media/common/videobuf2/videobuf2-dma-sg.c
> @@ -424,18 +424,12 @@ static struct sg_table *vb2_dma_sg_dmabuf_ops_map(
>   	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
>   {
>   	struct vb2_dma_sg_attachment *attach = db_attach->priv;
> -	/* stealing dmabuf mutex to serialize map/unmap operations */
> -	struct mutex *lock = &db_attach->dmabuf->lock;
>   	struct sg_table *sgt;
>   
> -	mutex_lock(lock);
> -
>   	sgt = &attach->sgt;
>   	/* return previously mapped sg table */
> -	if (attach->dma_dir == dma_dir) {
> -		mutex_unlock(lock);
> +	if (attach->dma_dir == dma_dir)
>   		return sgt;
> -	}
>   
>   	/* release any previous cache */
>   	if (attach->dma_dir != DMA_NONE) {
> @@ -446,14 +440,11 @@ static struct sg_table *vb2_dma_sg_dmabuf_ops_map(
>   	/* mapping to the client with new direction */
>   	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir, 0)) {
>   		pr_err("failed to map scatterlist\n");
> -		mutex_unlock(lock);
>   		return ERR_PTR(-EIO);
>   	}
>   
>   	attach->dma_dir = dma_dir;
>   
> -	mutex_unlock(lock);
> -
>   	return sgt;
>   }
>   
> diff --git a/drivers/media/common/videobuf2/videobuf2-vmalloc.c b/drivers/media/common/videobuf2/videobuf2-vmalloc.c
> index 948152f1596b..3d00a7f0aac1 100644
> --- a/drivers/media/common/videobuf2/videobuf2-vmalloc.c
> +++ b/drivers/media/common/videobuf2/videobuf2-vmalloc.c
> @@ -267,18 +267,12 @@ static struct sg_table *vb2_vmalloc_dmabuf_ops_map(
>   	struct dma_buf_attachment *db_attach, enum dma_data_direction dma_dir)
>   {
>   	struct vb2_vmalloc_attachment *attach = db_attach->priv;
> -	/* stealing dmabuf mutex to serialize map/unmap operations */
> -	struct mutex *lock = &db_attach->dmabuf->lock;
>   	struct sg_table *sgt;
>   
> -	mutex_lock(lock);
> -
>   	sgt = &attach->sgt;
>   	/* return previously mapped sg table */
> -	if (attach->dma_dir == dma_dir) {
> -		mutex_unlock(lock);
> +	if (attach->dma_dir == dma_dir)
>   		return sgt;
> -	}
>   
>   	/* release any previous cache */
>   	if (attach->dma_dir != DMA_NONE) {
> @@ -289,14 +283,11 @@ static struct sg_table *vb2_vmalloc_dmabuf_ops_map(
>   	/* mapping to the client with new direction */
>   	if (dma_map_sgtable(db_attach->dev, sgt, dma_dir, 0)) {
>   		pr_err("failed to map scatterlist\n");
> -		mutex_unlock(lock);
>   		return ERR_PTR(-EIO);
>   	}
>   
>   	attach->dma_dir = dma_dir;
>   
> -	mutex_unlock(lock);
> -
>   	return sgt;
>   }
>   
> diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h
> index 9d7c61a122dc..0b427939f466 100644
> --- a/include/drm/drm_gem.h
> +++ b/include/drm/drm_gem.h
> @@ -410,4 +410,7 @@ void drm_gem_unlock_reservations(struct drm_gem_object **objs, int count,
>   int drm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev,
>   			    u32 handle, u64 *offset);
>   
> +int drm_gem_vmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
> +void drm_gem_vunmap_unlocked(struct drm_gem_object *obj, struct iosys_map *map);
> +
>   #endif /* __DRM_GEM_H__ */
> diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
> index 71731796c8c3..23698c6b1d1e 100644
> --- a/include/linux/dma-buf.h
> +++ b/include/linux/dma-buf.h
> @@ -326,15 +326,6 @@ struct dma_buf {
>   	/** @ops: dma_buf_ops associated with this buffer object. */
>   	const struct dma_buf_ops *ops;
>   
> -	/**
> -	 * @lock:
> -	 *
> -	 * Used internally to serialize list manipulation, attach/detach and
> -	 * vmap/unmap. Note that in many cases this is superseeded by
> -	 * dma_resv_lock() on @resv.
> -	 */
> -	struct mutex lock;
> -
>   	/**
>   	 * @vmapping_counter:
>   	 *
> @@ -618,6 +609,11 @@ int dma_buf_fd(struct dma_buf *dmabuf, int flags);
>   struct dma_buf *dma_buf_get(int fd);
>   void dma_buf_put(struct dma_buf *dmabuf);
>   
> +struct sg_table *dma_buf_map_attachment_locked(struct dma_buf_attachment *,
> +					       enum dma_data_direction);
> +void dma_buf_unmap_attachment_locked(struct dma_buf_attachment *,
> +				     struct sg_table *,
> +				     enum dma_data_direction);
>   struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *,
>   					enum dma_data_direction);
>   void dma_buf_unmap_attachment(struct dma_buf_attachment *, struct sg_table *,

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dan Carpenter]

[-- Attachment #1: Type: text/plain, Size: 9217 bytes --]

[ I trimmed the CC list -dan ]

Hi Dmitry,

url:    https://github.com/intel-lab-lkp/linux/commits/Dmitry-Osipenko/Add-generic-memory-shrinker-to-VirtIO-GPU-and-Panfrost-DRM-drivers/20220527-075717
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git cdeffe87f790dfd1baa193020411ce9a538446d7
config: i386-randconfig-m021 (https://download.01.org/0day-ci/archive/20220527/202205272006.EZ53cUSD-lkp(a)intel.com/config)
compiler: gcc-11 (Debian 11.3.0-1) 11.3.0

If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>

New smatch warnings:
drivers/dma-buf/dma-buf.c:791 dma_buf_dynamic_attach() warn: inconsistent returns 'dmabuf->resv'.
drivers/dma-buf/dma-buf.c:1339 dma_buf_vmap_locked() error: uninitialized symbol 'ret'.

Old smatch warnings:
drivers/dma-buf/dma-buf.c:576 dma_buf_export() warn: '&dmabuf->list_node' not removed from list

vim +791 drivers/dma-buf/dma-buf.c

15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  714  struct dma_buf_attachment *
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  715  dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  716  		       const struct dma_buf_attach_ops *importer_ops,
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  717  		       void *importer_priv)
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  718  {
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  719  	struct dma_buf_attachment *attach;
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  720  	int ret;
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  721  
d1aa06a1eaf5f7 drivers/base/dma-buf.c    Laurent Pinchart   2012-01-26  722  	if (WARN_ON(!dmabuf || !dev))
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  723  		return ERR_PTR(-EINVAL);
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  724  
4981cdb063e3e9 drivers/dma-buf/dma-buf.c Christian König    2020-02-19  725  	if (WARN_ON(importer_ops && !importer_ops->move_notify))
4981cdb063e3e9 drivers/dma-buf/dma-buf.c Christian König    2020-02-19  726  		return ERR_PTR(-EINVAL);
4981cdb063e3e9 drivers/dma-buf/dma-buf.c Christian König    2020-02-19  727  
db7942b6292306 drivers/dma-buf/dma-buf.c Markus Elfring     2017-05-08  728  	attach = kzalloc(sizeof(*attach), GFP_KERNEL);
34d84ec4881d13 drivers/dma-buf/dma-buf.c Markus Elfring     2017-05-08  729  	if (!attach)
a9fbc3b73127ef drivers/base/dma-buf.c    Laurent Pinchart   2012-01-26  730  		return ERR_PTR(-ENOMEM);
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  731  
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  732  	attach->dev = dev;
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  733  	attach->dmabuf = dmabuf;
09606b5446c25b drivers/dma-buf/dma-buf.c Christian König    2018-03-22  734  	if (importer_ops)
09606b5446c25b drivers/dma-buf/dma-buf.c Christian König    2018-03-22  735  		attach->peer2peer = importer_ops->allow_peer2peer;
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  736  	attach->importer_ops = importer_ops;
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  737  	attach->importer_priv = importer_priv;
2ed9201bdd9a8e drivers/base/dma-buf.c    Laurent Pinchart   2012-01-26  738  
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko    2022-05-27  739  	dma_resv_lock(dmabuf->resv, NULL);
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko    2022-05-27  740  
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  741  	if (dmabuf->ops->attach) {
a19741e5e5a9f1 drivers/dma-buf/dma-buf.c Christian König    2018-05-28  742  		ret = dmabuf->ops->attach(dmabuf, attach);
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  743  		if (ret)
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  744  			goto err_attach;
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  745  	}
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  746  	list_add(&attach->node, &dmabuf->attachments);
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  747  
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  748  	/* When either the importer or the exporter can't handle dynamic
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  749  	 * mappings we cache the mapping here to avoid issues with the
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  750  	 * reservation object lock.
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  751  	 */
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  752  	if (dma_buf_attachment_is_dynamic(attach) !=
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  753  	    dma_buf_is_dynamic(dmabuf)) {
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  754  		struct sg_table *sgt;
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  755  
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  756  		if (dma_buf_is_dynamic(attach->dmabuf)) {
7e008b02557cce drivers/dma-buf/dma-buf.c Christian König    2021-05-17  757  			ret = dmabuf->ops->pin(attach);
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  758  			if (ret)
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  759  				goto err_unlock;
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  760  		}
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  761  
84335675f2223c drivers/dma-buf/dma-buf.c Daniel Vetter      2021-01-15  762  		sgt = __map_dma_buf(attach, DMA_BIDIRECTIONAL);
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  763  		if (!sgt)
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  764  			sgt = ERR_PTR(-ENOMEM);
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  765  		if (IS_ERR(sgt)) {
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  766  			ret = PTR_ERR(sgt);
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  767  			goto err_unpin;
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  768  		}
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  769  		attach->sgt = sgt;
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  770  		attach->dir = DMA_BIDIRECTIONAL;
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  771  	}
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  772  
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko    2022-05-27  773  	dma_resv_unlock(dmabuf->resv);
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko    2022-05-27  774  
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  775  	return attach;
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  776  
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  777  err_attach:
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko    2022-05-27  778  	dma_resv_unlock(attach->dmabuf->resv);

This is a false positive, but Smatch would prefer if the lock names
were the same everywhere:

	dma_resv_unlock(dmabuf->resv);

d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  779  	kfree(attach);
d15bd7ee445d07 drivers/base/dma-buf.c    Sumit Semwal       2011-12-26  780  	return ERR_PTR(ret);
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  781  
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  782  err_unpin:
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  783  	if (dma_buf_is_dynamic(attach->dmabuf))
7e008b02557cce drivers/dma-buf/dma-buf.c Christian König    2021-05-17  784  		dmabuf->ops->unpin(attach);
bb42df4662a447 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  785  
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  786  err_unlock:
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko    2022-05-27  787  	dma_resv_unlock(dmabuf->resv);
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  788  
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  789  	dma_buf_detach(dmabuf, attach);
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko    2022-05-27  790  
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03 @791  	return ERR_PTR(ret);
15fd552d186cb0 drivers/dma-buf/dma-buf.c Christian König    2018-07-03  792  }

-- 
0-DAY CI Kernel Test Service
https://01.org/lkp

[kbuild] Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Dan Carpenter]

Hi Dmitry,

url:    https://github.com/intel-lab-lkp/linux/commits/Dmitry-Osipenko/Add-generic-memory-shrinker-to-VirtIO-GPU-and-Panfrost-DRM-drivers/20220527-075717 
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git  cdeffe87f790dfd1baa193020411ce9a538446d7
config: i386-randconfig-m021 (https://download.01.org/0day-ci/archive/20220527/202205272155.MxDiru43-lkp@intel.com/config )
compiler: gcc-11 (Debian 11.3.0-1) 11.3.0

If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>

New smatch warnings:
drivers/gpu/drm/drm_gem_shmem_helper.c:226 drm_gem_shmem_set_evictable() warn: inconsistent returns 'shmem->base.resv'.
drivers/gpu/drm/drm_gem_shmem_helper.c:253 drm_gem_shmem_set_purgeable() warn: inconsistent returns 'shmem->base.resv'.
drivers/gpu/drm/drm_gem_shmem_helper.c:703 drm_gem_shmem_fault() error: we previously assumed 'shmem->pages' could be null (see line 694)

vim +226 drivers/gpu/drm/drm_gem_shmem_helper.c

875fd932de880a Dmitry Osipenko 2022-05-27  215  int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
875fd932de880a Dmitry Osipenko 2022-05-27  216  {
875fd932de880a Dmitry Osipenko 2022-05-27  217  	dma_resv_lock(shmem->base.resv, NULL);
875fd932de880a Dmitry Osipenko 2022-05-27  218  
875fd932de880a Dmitry Osipenko 2022-05-27  219  	if (shmem->madv < 0)
875fd932de880a Dmitry Osipenko 2022-05-27  220  		return -ENOMEM;

Unlock before returning?

875fd932de880a Dmitry Osipenko 2022-05-27  221  
875fd932de880a Dmitry Osipenko 2022-05-27  222  	shmem->eviction_enabled = true;
875fd932de880a Dmitry Osipenko 2022-05-27  223  
875fd932de880a Dmitry Osipenko 2022-05-27  224  	dma_resv_unlock(shmem->base.resv);
875fd932de880a Dmitry Osipenko 2022-05-27  225  
875fd932de880a Dmitry Osipenko 2022-05-27 @226  	return 0;
875fd932de880a Dmitry Osipenko 2022-05-27  227  }
875fd932de880a Dmitry Osipenko 2022-05-27  228  EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
875fd932de880a Dmitry Osipenko 2022-05-27  229  
875fd932de880a Dmitry Osipenko 2022-05-27  230  /**
875fd932de880a Dmitry Osipenko 2022-05-27  231   * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
875fd932de880a Dmitry Osipenko 2022-05-27  232   * @shmem: shmem GEM object
875fd932de880a Dmitry Osipenko 2022-05-27  233   *
875fd932de880a Dmitry Osipenko 2022-05-27  234   * Tell memory shrinker that this GEM can be purged. Initially purging is
875fd932de880a Dmitry Osipenko 2022-05-27  235   * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
875fd932de880a Dmitry Osipenko 2022-05-27  236   *
875fd932de880a Dmitry Osipenko 2022-05-27  237   * Returns:
875fd932de880a Dmitry Osipenko 2022-05-27  238   * 0 on success or a negative error code on failure.
875fd932de880a Dmitry Osipenko 2022-05-27  239   */
875fd932de880a Dmitry Osipenko 2022-05-27  240  int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
875fd932de880a Dmitry Osipenko 2022-05-27  241  {
875fd932de880a Dmitry Osipenko 2022-05-27  242  	dma_resv_lock(shmem->base.resv, NULL);
875fd932de880a Dmitry Osipenko 2022-05-27  243  
875fd932de880a Dmitry Osipenko 2022-05-27  244  	if (shmem->madv < 0)
875fd932de880a Dmitry Osipenko 2022-05-27  245  		return -ENOMEM;

Same.

875fd932de880a Dmitry Osipenko 2022-05-27  246  
875fd932de880a Dmitry Osipenko 2022-05-27  247  	shmem->purge_enabled = true;
875fd932de880a Dmitry Osipenko 2022-05-27  248  
875fd932de880a Dmitry Osipenko 2022-05-27  249  	drm_gem_shmem_update_pages_state(shmem);
875fd932de880a Dmitry Osipenko 2022-05-27  250  
875fd932de880a Dmitry Osipenko 2022-05-27  251  	dma_resv_unlock(shmem->base.resv);
875fd932de880a Dmitry Osipenko 2022-05-27  252  
875fd932de880a Dmitry Osipenko 2022-05-27 @253  	return 0;
875fd932de880a Dmitry Osipenko 2022-05-27  254  }

-- 
0-DAY CI Kernel Test Service
https://01.org/lkp 
_______________________________________________
kbuild mailing list -- kbuild@lists.01.org
To unsubscribe send an email to kbuild-leave@lists.01.org

[kbuild] Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Dan Carpenter]

[-- Attachment #1: Type: text/plain, Size: 4187 bytes --]

Hi Dmitry,

url:    https://github.com/intel-lab-lkp/linux/commits/Dmitry-Osipenko/Add-generic-memory-shrinker-to-VirtIO-GPU-and-Panfrost-DRM-drivers/20220527-075717 
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git  cdeffe87f790dfd1baa193020411ce9a538446d7
config: i386-randconfig-m021 (https://download.01.org/0day-ci/archive/20220527/202205272155.MxDiru43-lkp(a)intel.com/config )
compiler: gcc-11 (Debian 11.3.0-1) 11.3.0

If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>

New smatch warnings:
drivers/gpu/drm/drm_gem_shmem_helper.c:226 drm_gem_shmem_set_evictable() warn: inconsistent returns 'shmem->base.resv'.
drivers/gpu/drm/drm_gem_shmem_helper.c:253 drm_gem_shmem_set_purgeable() warn: inconsistent returns 'shmem->base.resv'.
drivers/gpu/drm/drm_gem_shmem_helper.c:703 drm_gem_shmem_fault() error: we previously assumed 'shmem->pages' could be null (see line 694)

vim +226 drivers/gpu/drm/drm_gem_shmem_helper.c

875fd932de880a Dmitry Osipenko 2022-05-27  215  int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
875fd932de880a Dmitry Osipenko 2022-05-27  216  {
875fd932de880a Dmitry Osipenko 2022-05-27  217  	dma_resv_lock(shmem->base.resv, NULL);
875fd932de880a Dmitry Osipenko 2022-05-27  218  
875fd932de880a Dmitry Osipenko 2022-05-27  219  	if (shmem->madv < 0)
875fd932de880a Dmitry Osipenko 2022-05-27  220  		return -ENOMEM;

Unlock before returning?

875fd932de880a Dmitry Osipenko 2022-05-27  221  
875fd932de880a Dmitry Osipenko 2022-05-27  222  	shmem->eviction_enabled = true;
875fd932de880a Dmitry Osipenko 2022-05-27  223  
875fd932de880a Dmitry Osipenko 2022-05-27  224  	dma_resv_unlock(shmem->base.resv);
875fd932de880a Dmitry Osipenko 2022-05-27  225  
875fd932de880a Dmitry Osipenko 2022-05-27 @226  	return 0;
875fd932de880a Dmitry Osipenko 2022-05-27  227  }
875fd932de880a Dmitry Osipenko 2022-05-27  228  EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
875fd932de880a Dmitry Osipenko 2022-05-27  229  
875fd932de880a Dmitry Osipenko 2022-05-27  230  /**
875fd932de880a Dmitry Osipenko 2022-05-27  231   * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
875fd932de880a Dmitry Osipenko 2022-05-27  232   * @shmem: shmem GEM object
875fd932de880a Dmitry Osipenko 2022-05-27  233   *
875fd932de880a Dmitry Osipenko 2022-05-27  234   * Tell memory shrinker that this GEM can be purged. Initially purging is
875fd932de880a Dmitry Osipenko 2022-05-27  235   * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
875fd932de880a Dmitry Osipenko 2022-05-27  236   *
875fd932de880a Dmitry Osipenko 2022-05-27  237   * Returns:
875fd932de880a Dmitry Osipenko 2022-05-27  238   * 0 on success or a negative error code on failure.
875fd932de880a Dmitry Osipenko 2022-05-27  239   */
875fd932de880a Dmitry Osipenko 2022-05-27  240  int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
875fd932de880a Dmitry Osipenko 2022-05-27  241  {
875fd932de880a Dmitry Osipenko 2022-05-27  242  	dma_resv_lock(shmem->base.resv, NULL);
875fd932de880a Dmitry Osipenko 2022-05-27  243  
875fd932de880a Dmitry Osipenko 2022-05-27  244  	if (shmem->madv < 0)
875fd932de880a Dmitry Osipenko 2022-05-27  245  		return -ENOMEM;

Same.

875fd932de880a Dmitry Osipenko 2022-05-27  246  
875fd932de880a Dmitry Osipenko 2022-05-27  247  	shmem->purge_enabled = true;
875fd932de880a Dmitry Osipenko 2022-05-27  248  
875fd932de880a Dmitry Osipenko 2022-05-27  249  	drm_gem_shmem_update_pages_state(shmem);
875fd932de880a Dmitry Osipenko 2022-05-27  250  
875fd932de880a Dmitry Osipenko 2022-05-27  251  	dma_resv_unlock(shmem->base.resv);
875fd932de880a Dmitry Osipenko 2022-05-27  252  
875fd932de880a Dmitry Osipenko 2022-05-27 @253  	return 0;
875fd932de880a Dmitry Osipenko 2022-05-27  254  }

-- 
0-DAY CI Kernel Test Service
https://01.org/lkp 
_______________________________________________
kbuild mailing list -- kbuild(a)lists.01.org
To unsubscribe send an email to kbuild-leave(a)lists.01.org

Re: [PATCH v6 04/22] drm/panfrost: Fix shrinker list corruption by madvise IOCTL

[Steven Price]

On 27/05/2022 00:50, Dmitry Osipenko wrote:
> Calling madvise IOCTL twice on BO causes memory shrinker list corruption
> and crashes kernel because BO is already on the list and it's added to
> the list again, while BO should be removed from from the list before it's
> re-added. Fix it.
> 
> Cc: stable@vger.kernel.org
> Fixes: 013b65101315 ("drm/panfrost: Add madvise and shrinker support")
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>

Reviewed-by: Steven Price <steven.price@arm.com>

> ---
>  drivers/gpu/drm/panfrost/panfrost_drv.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
> index 087e69b98d06..b1e6d238674f 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_drv.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
> @@ -433,8 +433,8 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
>  
>  	if (args->retained) {
>  		if (args->madv == PANFROST_MADV_DONTNEED)
> -			list_add_tail(&bo->base.madv_list,
> -				      &pfdev->shrinker_list);
> +			list_move_tail(&bo->base.madv_list,
> +				       &pfdev->shrinker_list);
>  		else if (args->madv == PANFROST_MADV_WILLNEED)
>  			list_del_init(&bo->base.madv_list);
>  	}

Re: [PATCH v6 04/22] drm/panfrost: Fix shrinker list corruption by madvise IOCTL

[Steven Price]

On 27/05/2022 00:50, Dmitry Osipenko wrote:
> Calling madvise IOCTL twice on BO causes memory shrinker list corruption
> and crashes kernel because BO is already on the list and it's added to
> the list again, while BO should be removed from from the list before it's
> re-added. Fix it.
> 
> Cc: stable@vger.kernel.org
> Fixes: 013b65101315 ("drm/panfrost: Add madvise and shrinker support")
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>

Reviewed-by: Steven Price <steven.price@arm.com>

> ---
>  drivers/gpu/drm/panfrost/panfrost_drv.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
> index 087e69b98d06..b1e6d238674f 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_drv.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
> @@ -433,8 +433,8 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
>  
>  	if (args->retained) {
>  		if (args->madv == PANFROST_MADV_DONTNEED)
> -			list_add_tail(&bo->base.madv_list,
> -				      &pfdev->shrinker_list);
> +			list_move_tail(&bo->base.madv_list,
> +				       &pfdev->shrinker_list);
>  		else if (args->madv == PANFROST_MADV_WILLNEED)
>  			list_del_init(&bo->base.madv_list);
>  	}

Re: [Intel-gfx] [PATCH v6 04/22] drm/panfrost: Fix shrinker list corruption by madvise IOCTL

[Steven Price]

On 27/05/2022 00:50, Dmitry Osipenko wrote:
> Calling madvise IOCTL twice on BO causes memory shrinker list corruption
> and crashes kernel because BO is already on the list and it's added to
> the list again, while BO should be removed from from the list before it's
> re-added. Fix it.
> 
> Cc: stable@vger.kernel.org
> Fixes: 013b65101315 ("drm/panfrost: Add madvise and shrinker support")
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>

Reviewed-by: Steven Price <steven.price@arm.com>

> ---
>  drivers/gpu/drm/panfrost/panfrost_drv.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
> index 087e69b98d06..b1e6d238674f 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_drv.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
> @@ -433,8 +433,8 @@ static int panfrost_ioctl_madvise(struct drm_device *dev, void *data,
>  
>  	if (args->retained) {
>  		if (args->madv == PANFROST_MADV_DONTNEED)
> -			list_add_tail(&bo->base.madv_list,
> -				      &pfdev->shrinker_list);
> +			list_move_tail(&bo->base.madv_list,
> +				       &pfdev->shrinker_list);
>  		else if (args->madv == PANFROST_MADV_WILLNEED)
>  			list_del_init(&bo->base.madv_list);
>  	}

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dmitry Osipenko]

Hello Christian,

On 5/30/22 09:50, Christian König wrote:
> Hi Dmitry,
> 
> First of all please separate out this patch from the rest of the series,
> since this is a complex separate structural change.

I assume all the patches will go via the DRM tree in the end since the
rest of the DRM patches in this series depend on this dma-buf change.
But I see that separation may ease reviewing of the dma-buf changes, so
let's try it.

> Am 27.05.22 um 01:50 schrieb Dmitry Osipenko:
>> All dma-bufs have dma-reservation lock that allows drivers to perform
>> exclusive operations over shared dma-bufs. Today's dma-buf API has
>> incomplete locking specification, which creates dead lock situation
>> for dma-buf importers and exporters that don't coordinate theirs locks.
> 
> Well please drop that sentence. The locking specifications are actually
> very well defined, it's just that some drivers are a bit broken
> regarding them.
> 
> What you do here is rather moving all the non-dynamic drivers over to
> the dynamic locking specification (which is really nice to have).

Indeed, this will be a better description, thank you! I'll update it.

> I have tried this before and failed because catching all the locks in
> the right code paths are very tricky. So expect some fallout from this
> and make sure the kernel test robot and CI systems are clean.

Sure, I'll fix up all the reported things in the next iteration.

BTW, have you ever posted yours version of the patch? Will be great if
we could compare the changed code paths.

>> This patch introduces new locking convention for dma-buf users. From now
>> on all dma-buf importers are responsible for holding dma-buf reservation
>> lock around operations performed over dma-bufs.
>>
>> This patch implements the new dma-buf locking convention by:
>>
>>    1. Making dma-buf API functions to take the reservation lock.
>>
>>    2. Adding new locked variants of the dma-buf API functions for drivers
>>       that need to manage imported dma-bufs under the held lock.
> 
> Instead of adding new locked variants please mark all variants which
> expect to be called without a lock with an _unlocked postfix.
> 
> This should make it easier to remove those in a follow up patch set and
> then fully move the locking into the importer.

Do we really want to move all the locks to the importers? Seems the
majority of drivers should be happy with the dma-buf helpers handling
the locking for them.

>>    3. Converting all drivers to the new locking scheme.
> 
> I have strong doubts that you got all of them. At least radeon and
> nouveau should grab the reservation lock in their ->attach callbacks
> somehow.

Radeon and Nouveau use gem_prime_import_sg_table() and they take resv
lock already, seems they should be okay (?)

I assume all the basics should covered in this v6. At minimum Intel,
Tegra, Panfrost, Lima and Rockchip drivers should be good. If I missed
something, then please let me know and I'll correct it.

>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>> ---
>>   drivers/dma-buf/dma-buf.c                     | 270 +++++++++++-------
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
>>   drivers/gpu/drm/drm_client.c                  |   4 +-
>>   drivers/gpu/drm/drm_gem.c                     |  33 +++
>>   drivers/gpu/drm/drm_gem_framebuffer_helper.c  |   6 +-
>>   drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    |  10 +-
>>   drivers/gpu/drm/qxl/qxl_object.c              |  17 +-
>>   drivers/gpu/drm/qxl/qxl_prime.c               |   4 +-
>>   .../common/videobuf2/videobuf2-dma-contig.c   |  11 +-
>>   .../media/common/videobuf2/videobuf2-dma-sg.c |  11 +-
>>   .../common/videobuf2/videobuf2-vmalloc.c      |  11 +-
>>   include/drm/drm_gem.h                         |   3 +
>>   include/linux/dma-buf.h                       |  14 +-
>>   13 files changed, 241 insertions(+), 159 deletions(-)
>>
>> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
>> index 32f55640890c..64a9909ccfa2 100644
>> --- a/drivers/dma-buf/dma-buf.c
>> +++ b/drivers/dma-buf/dma-buf.c
>> @@ -552,7 +552,6 @@ struct dma_buf *dma_buf_export(const struct
>> dma_buf_export_info *exp_info)
>>       file->f_mode |= FMODE_LSEEK;
>>       dmabuf->file = file;
>>   -    mutex_init(&dmabuf->lock);
> 
> Please make removing dmabuf->lock a separate change.

Alright

-- 
Best regards,
Dmitry

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dmitry Osipenko]

Hello Christian,

On 5/30/22 09:50, Christian König wrote:
> Hi Dmitry,
> 
> First of all please separate out this patch from the rest of the series,
> since this is a complex separate structural change.

I assume all the patches will go via the DRM tree in the end since the
rest of the DRM patches in this series depend on this dma-buf change.
But I see that separation may ease reviewing of the dma-buf changes, so
let's try it.

> Am 27.05.22 um 01:50 schrieb Dmitry Osipenko:
>> All dma-bufs have dma-reservation lock that allows drivers to perform
>> exclusive operations over shared dma-bufs. Today's dma-buf API has
>> incomplete locking specification, which creates dead lock situation
>> for dma-buf importers and exporters that don't coordinate theirs locks.
> 
> Well please drop that sentence. The locking specifications are actually
> very well defined, it's just that some drivers are a bit broken
> regarding them.
> 
> What you do here is rather moving all the non-dynamic drivers over to
> the dynamic locking specification (which is really nice to have).

Indeed, this will be a better description, thank you! I'll update it.

> I have tried this before and failed because catching all the locks in
> the right code paths are very tricky. So expect some fallout from this
> and make sure the kernel test robot and CI systems are clean.

Sure, I'll fix up all the reported things in the next iteration.

BTW, have you ever posted yours version of the patch? Will be great if
we could compare the changed code paths.

>> This patch introduces new locking convention for dma-buf users. From now
>> on all dma-buf importers are responsible for holding dma-buf reservation
>> lock around operations performed over dma-bufs.
>>
>> This patch implements the new dma-buf locking convention by:
>>
>>    1. Making dma-buf API functions to take the reservation lock.
>>
>>    2. Adding new locked variants of the dma-buf API functions for drivers
>>       that need to manage imported dma-bufs under the held lock.
> 
> Instead of adding new locked variants please mark all variants which
> expect to be called without a lock with an _unlocked postfix.
> 
> This should make it easier to remove those in a follow up patch set and
> then fully move the locking into the importer.

Do we really want to move all the locks to the importers? Seems the
majority of drivers should be happy with the dma-buf helpers handling
the locking for them.

>>    3. Converting all drivers to the new locking scheme.
> 
> I have strong doubts that you got all of them. At least radeon and
> nouveau should grab the reservation lock in their ->attach callbacks
> somehow.

Radeon and Nouveau use gem_prime_import_sg_table() and they take resv
lock already, seems they should be okay (?)

I assume all the basics should covered in this v6. At minimum Intel,
Tegra, Panfrost, Lima and Rockchip drivers should be good. If I missed
something, then please let me know and I'll correct it.

>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>> ---
>>   drivers/dma-buf/dma-buf.c                     | 270 +++++++++++-------
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
>>   drivers/gpu/drm/drm_client.c                  |   4 +-
>>   drivers/gpu/drm/drm_gem.c                     |  33 +++
>>   drivers/gpu/drm/drm_gem_framebuffer_helper.c  |   6 +-
>>   drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    |  10 +-
>>   drivers/gpu/drm/qxl/qxl_object.c              |  17 +-
>>   drivers/gpu/drm/qxl/qxl_prime.c               |   4 +-
>>   .../common/videobuf2/videobuf2-dma-contig.c   |  11 +-
>>   .../media/common/videobuf2/videobuf2-dma-sg.c |  11 +-
>>   .../common/videobuf2/videobuf2-vmalloc.c      |  11 +-
>>   include/drm/drm_gem.h                         |   3 +
>>   include/linux/dma-buf.h                       |  14 +-
>>   13 files changed, 241 insertions(+), 159 deletions(-)
>>
>> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
>> index 32f55640890c..64a9909ccfa2 100644
>> --- a/drivers/dma-buf/dma-buf.c
>> +++ b/drivers/dma-buf/dma-buf.c
>> @@ -552,7 +552,6 @@ struct dma_buf *dma_buf_export(const struct
>> dma_buf_export_info *exp_info)
>>       file->f_mode |= FMODE_LSEEK;
>>       dmabuf->file = file;
>>   -    mutex_init(&dmabuf->lock);
> 
> Please make removing dmabuf->lock a separate change.

Alright

-- 
Best regards,
Dmitry

Re: [kbuild] Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Dmitry Osipenko]

On 5/30/22 10:09, Dan Carpenter wrote:
> Hi Dmitry,
> 
> url:    https://github.com/intel-lab-lkp/linux/commits/Dmitry-Osipenko/Add-generic-memory-shrinker-to-VirtIO-GPU-and-Panfrost-DRM-drivers/20220527-075717 
> base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git  cdeffe87f790dfd1baa193020411ce9a538446d7
> config: i386-randconfig-m021 (https://download.01.org/0day-ci/archive/20220527/202205272155.MxDiru43-lkp@intel.com/config )
> compiler: gcc-11 (Debian 11.3.0-1) 11.3.0
> 
> If you fix the issue, kindly add following tag where applicable
> Reported-by: kernel test robot <lkp@intel.com>
> Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
> 
> New smatch warnings:
> drivers/gpu/drm/drm_gem_shmem_helper.c:226 drm_gem_shmem_set_evictable() warn: inconsistent returns 'shmem->base.resv'.
> drivers/gpu/drm/drm_gem_shmem_helper.c:253 drm_gem_shmem_set_purgeable() warn: inconsistent returns 'shmem->base.resv'.
> drivers/gpu/drm/drm_gem_shmem_helper.c:703 drm_gem_shmem_fault() error: we previously assumed 'shmem->pages' could be null (see line 694)
> 
> vim +226 drivers/gpu/drm/drm_gem_shmem_helper.c
> 
> 875fd932de880a Dmitry Osipenko 2022-05-27  215  int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> 875fd932de880a Dmitry Osipenko 2022-05-27  216  {
> 875fd932de880a Dmitry Osipenko 2022-05-27  217  	dma_resv_lock(shmem->base.resv, NULL);
> 875fd932de880a Dmitry Osipenko 2022-05-27  218  
> 875fd932de880a Dmitry Osipenko 2022-05-27  219  	if (shmem->madv < 0)
> 875fd932de880a Dmitry Osipenko 2022-05-27  220  		return -ENOMEM;
> 
> Unlock before returning?
> 
> 875fd932de880a Dmitry Osipenko 2022-05-27  221  
> 875fd932de880a Dmitry Osipenko 2022-05-27  222  	shmem->eviction_enabled = true;
> 875fd932de880a Dmitry Osipenko 2022-05-27  223  
> 875fd932de880a Dmitry Osipenko 2022-05-27  224  	dma_resv_unlock(shmem->base.resv);
> 875fd932de880a Dmitry Osipenko 2022-05-27  225  
> 875fd932de880a Dmitry Osipenko 2022-05-27 @226  	return 0;
> 875fd932de880a Dmitry Osipenko 2022-05-27  227  }
> 875fd932de880a Dmitry Osipenko 2022-05-27  228  EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> 875fd932de880a Dmitry Osipenko 2022-05-27  229  
> 875fd932de880a Dmitry Osipenko 2022-05-27  230  /**
> 875fd932de880a Dmitry Osipenko 2022-05-27  231   * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> 875fd932de880a Dmitry Osipenko 2022-05-27  232   * @shmem: shmem GEM object
> 875fd932de880a Dmitry Osipenko 2022-05-27  233   *
> 875fd932de880a Dmitry Osipenko 2022-05-27  234   * Tell memory shrinker that this GEM can be purged. Initially purging is
> 875fd932de880a Dmitry Osipenko 2022-05-27  235   * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> 875fd932de880a Dmitry Osipenko 2022-05-27  236   *
> 875fd932de880a Dmitry Osipenko 2022-05-27  237   * Returns:
> 875fd932de880a Dmitry Osipenko 2022-05-27  238   * 0 on success or a negative error code on failure.
> 875fd932de880a Dmitry Osipenko 2022-05-27  239   */
> 875fd932de880a Dmitry Osipenko 2022-05-27  240  int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> 875fd932de880a Dmitry Osipenko 2022-05-27  241  {
> 875fd932de880a Dmitry Osipenko 2022-05-27  242  	dma_resv_lock(shmem->base.resv, NULL);
> 875fd932de880a Dmitry Osipenko 2022-05-27  243  
> 875fd932de880a Dmitry Osipenko 2022-05-27  244  	if (shmem->madv < 0)
> 875fd932de880a Dmitry Osipenko 2022-05-27  245  		return -ENOMEM;
> 
> Same.

Indeed, thanks!

-- 
Best regards,
Dmitry

Re: [kbuild] Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Dmitry Osipenko]

[-- Attachment #1: Type: text/plain, Size: 3600 bytes --]

On 5/30/22 10:09, Dan Carpenter wrote:
> Hi Dmitry,
> 
> url:    https://github.com/intel-lab-lkp/linux/commits/Dmitry-Osipenko/Add-generic-memory-shrinker-to-VirtIO-GPU-and-Panfrost-DRM-drivers/20220527-075717 
> base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git  cdeffe87f790dfd1baa193020411ce9a538446d7
> config: i386-randconfig-m021 (https://download.01.org/0day-ci/archive/20220527/202205272155.MxDiru43-lkp(a)intel.com/config )
> compiler: gcc-11 (Debian 11.3.0-1) 11.3.0
> 
> If you fix the issue, kindly add following tag where applicable
> Reported-by: kernel test robot <lkp@intel.com>
> Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
> 
> New smatch warnings:
> drivers/gpu/drm/drm_gem_shmem_helper.c:226 drm_gem_shmem_set_evictable() warn: inconsistent returns 'shmem->base.resv'.
> drivers/gpu/drm/drm_gem_shmem_helper.c:253 drm_gem_shmem_set_purgeable() warn: inconsistent returns 'shmem->base.resv'.
> drivers/gpu/drm/drm_gem_shmem_helper.c:703 drm_gem_shmem_fault() error: we previously assumed 'shmem->pages' could be null (see line 694)
> 
> vim +226 drivers/gpu/drm/drm_gem_shmem_helper.c
> 
> 875fd932de880a Dmitry Osipenko 2022-05-27  215  int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> 875fd932de880a Dmitry Osipenko 2022-05-27  216  {
> 875fd932de880a Dmitry Osipenko 2022-05-27  217  	dma_resv_lock(shmem->base.resv, NULL);
> 875fd932de880a Dmitry Osipenko 2022-05-27  218  
> 875fd932de880a Dmitry Osipenko 2022-05-27  219  	if (shmem->madv < 0)
> 875fd932de880a Dmitry Osipenko 2022-05-27  220  		return -ENOMEM;
> 
> Unlock before returning?
> 
> 875fd932de880a Dmitry Osipenko 2022-05-27  221  
> 875fd932de880a Dmitry Osipenko 2022-05-27  222  	shmem->eviction_enabled = true;
> 875fd932de880a Dmitry Osipenko 2022-05-27  223  
> 875fd932de880a Dmitry Osipenko 2022-05-27  224  	dma_resv_unlock(shmem->base.resv);
> 875fd932de880a Dmitry Osipenko 2022-05-27  225  
> 875fd932de880a Dmitry Osipenko 2022-05-27 @226  	return 0;
> 875fd932de880a Dmitry Osipenko 2022-05-27  227  }
> 875fd932de880a Dmitry Osipenko 2022-05-27  228  EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> 875fd932de880a Dmitry Osipenko 2022-05-27  229  
> 875fd932de880a Dmitry Osipenko 2022-05-27  230  /**
> 875fd932de880a Dmitry Osipenko 2022-05-27  231   * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> 875fd932de880a Dmitry Osipenko 2022-05-27  232   * @shmem: shmem GEM object
> 875fd932de880a Dmitry Osipenko 2022-05-27  233   *
> 875fd932de880a Dmitry Osipenko 2022-05-27  234   * Tell memory shrinker that this GEM can be purged. Initially purging is
> 875fd932de880a Dmitry Osipenko 2022-05-27  235   * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> 875fd932de880a Dmitry Osipenko 2022-05-27  236   *
> 875fd932de880a Dmitry Osipenko 2022-05-27  237   * Returns:
> 875fd932de880a Dmitry Osipenko 2022-05-27  238   * 0 on success or a negative error code on failure.
> 875fd932de880a Dmitry Osipenko 2022-05-27  239   */
> 875fd932de880a Dmitry Osipenko 2022-05-27  240  int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> 875fd932de880a Dmitry Osipenko 2022-05-27  241  {
> 875fd932de880a Dmitry Osipenko 2022-05-27  242  	dma_resv_lock(shmem->base.resv, NULL);
> 875fd932de880a Dmitry Osipenko 2022-05-27  243  
> 875fd932de880a Dmitry Osipenko 2022-05-27  244  	if (shmem->madv < 0)
> 875fd932de880a Dmitry Osipenko 2022-05-27  245  		return -ENOMEM;
> 
> Same.

Indeed, thanks!

-- 
Best regards,
Dmitry

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Christian König via Virtualization]

Hi Dmitry,

Am 30.05.22 um 15:26 schrieb Dmitry Osipenko:
> Hello Christian,
>
> On 5/30/22 09:50, Christian König wrote:
>> Hi Dmitry,
>>
>> First of all please separate out this patch from the rest of the series,
>> since this is a complex separate structural change.
> I assume all the patches will go via the DRM tree in the end since the
> rest of the DRM patches in this series depend on this dma-buf change.
> But I see that separation may ease reviewing of the dma-buf changes, so
> let's try it.

That sounds like you are underestimating a bit how much trouble this 
will be.

>> I have tried this before and failed because catching all the locks in
>> the right code paths are very tricky. So expect some fallout from this
>> and make sure the kernel test robot and CI systems are clean.
> Sure, I'll fix up all the reported things in the next iteration.
>
> BTW, have you ever posted yours version of the patch? Will be great if
> we could compare the changed code paths.

No, I never even finished creating it after realizing how much work it 
would be.

>>> This patch introduces new locking convention for dma-buf users. From now
>>> on all dma-buf importers are responsible for holding dma-buf reservation
>>> lock around operations performed over dma-bufs.
>>>
>>> This patch implements the new dma-buf locking convention by:
>>>
>>>     1. Making dma-buf API functions to take the reservation lock.
>>>
>>>     2. Adding new locked variants of the dma-buf API functions for drivers
>>>        that need to manage imported dma-bufs under the held lock.
>> Instead of adding new locked variants please mark all variants which
>> expect to be called without a lock with an _unlocked postfix.
>>
>> This should make it easier to remove those in a follow up patch set and
>> then fully move the locking into the importer.
> Do we really want to move all the locks to the importers? Seems the
> majority of drivers should be happy with the dma-buf helpers handling
> the locking for them.

Yes, I clearly think so.

>
>>>     3. Converting all drivers to the new locking scheme.
>> I have strong doubts that you got all of them. At least radeon and
>> nouveau should grab the reservation lock in their ->attach callbacks
>> somehow.
> Radeon and Nouveau use gem_prime_import_sg_table() and they take resv
> lock already, seems they should be okay (?)

You are looking at the wrong side. You need to fix the export code path, 
not the import ones.

See for example attach on radeon works like this 
drm_gem_map_attach->drm_gem_pin->radeon_gem_prime_pin->radeon_bo_reserve->ttm_bo_reserve->dma_resv_lock.

Same for nouveau and probably a few other exporters as well. That will 
certainly cause a deadlock if you don't fix it.

I strongly suggest to do this step by step, first attach/detach and then 
the rest.

Regards,
Christian.

>
> I assume all the basics should covered in this v6. At minimum Intel,
> Tegra, Panfrost, Lima and Rockchip drivers should be good. If I missed
> something, then please let me know and I'll correct it.
>
>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>> ---
>>>    drivers/dma-buf/dma-buf.c                     | 270 +++++++++++-------
>>>    drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
>>>    drivers/gpu/drm/drm_client.c                  |   4 +-
>>>    drivers/gpu/drm/drm_gem.c                     |  33 +++
>>>    drivers/gpu/drm/drm_gem_framebuffer_helper.c  |   6 +-
>>>    drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    |  10 +-
>>>    drivers/gpu/drm/qxl/qxl_object.c              |  17 +-
>>>    drivers/gpu/drm/qxl/qxl_prime.c               |   4 +-
>>>    .../common/videobuf2/videobuf2-dma-contig.c   |  11 +-
>>>    .../media/common/videobuf2/videobuf2-dma-sg.c |  11 +-
>>>    .../common/videobuf2/videobuf2-vmalloc.c      |  11 +-
>>>    include/drm/drm_gem.h                         |   3 +
>>>    include/linux/dma-buf.h                       |  14 +-
>>>    13 files changed, 241 insertions(+), 159 deletions(-)
>>>
>>> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
>>> index 32f55640890c..64a9909ccfa2 100644
>>> --- a/drivers/dma-buf/dma-buf.c
>>> +++ b/drivers/dma-buf/dma-buf.c
>>> @@ -552,7 +552,6 @@ struct dma_buf *dma_buf_export(const struct
>>> dma_buf_export_info *exp_info)
>>>        file->f_mode |= FMODE_LSEEK;
>>>        dmabuf->file = file;
>>>    -    mutex_init(&dmabuf->lock);
>> Please make removing dmabuf->lock a separate change.
> Alright
>

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Christian König]

Hi Dmitry,

Am 30.05.22 um 15:26 schrieb Dmitry Osipenko:
> Hello Christian,
>
> On 5/30/22 09:50, Christian König wrote:
>> Hi Dmitry,
>>
>> First of all please separate out this patch from the rest of the series,
>> since this is a complex separate structural change.
> I assume all the patches will go via the DRM tree in the end since the
> rest of the DRM patches in this series depend on this dma-buf change.
> But I see that separation may ease reviewing of the dma-buf changes, so
> let's try it.

That sounds like you are underestimating a bit how much trouble this 
will be.

>> I have tried this before and failed because catching all the locks in
>> the right code paths are very tricky. So expect some fallout from this
>> and make sure the kernel test robot and CI systems are clean.
> Sure, I'll fix up all the reported things in the next iteration.
>
> BTW, have you ever posted yours version of the patch? Will be great if
> we could compare the changed code paths.

No, I never even finished creating it after realizing how much work it 
would be.

>>> This patch introduces new locking convention for dma-buf users. From now
>>> on all dma-buf importers are responsible for holding dma-buf reservation
>>> lock around operations performed over dma-bufs.
>>>
>>> This patch implements the new dma-buf locking convention by:
>>>
>>>     1. Making dma-buf API functions to take the reservation lock.
>>>
>>>     2. Adding new locked variants of the dma-buf API functions for drivers
>>>        that need to manage imported dma-bufs under the held lock.
>> Instead of adding new locked variants please mark all variants which
>> expect to be called without a lock with an _unlocked postfix.
>>
>> This should make it easier to remove those in a follow up patch set and
>> then fully move the locking into the importer.
> Do we really want to move all the locks to the importers? Seems the
> majority of drivers should be happy with the dma-buf helpers handling
> the locking for them.

Yes, I clearly think so.

>
>>>     3. Converting all drivers to the new locking scheme.
>> I have strong doubts that you got all of them. At least radeon and
>> nouveau should grab the reservation lock in their ->attach callbacks
>> somehow.
> Radeon and Nouveau use gem_prime_import_sg_table() and they take resv
> lock already, seems they should be okay (?)

You are looking at the wrong side. You need to fix the export code path, 
not the import ones.

See for example attach on radeon works like this 
drm_gem_map_attach->drm_gem_pin->radeon_gem_prime_pin->radeon_bo_reserve->ttm_bo_reserve->dma_resv_lock.

Same for nouveau and probably a few other exporters as well. That will 
certainly cause a deadlock if you don't fix it.

I strongly suggest to do this step by step, first attach/detach and then 
the rest.

Regards,
Christian.

>
> I assume all the basics should covered in this v6. At minimum Intel,
> Tegra, Panfrost, Lima and Rockchip drivers should be good. If I missed
> something, then please let me know and I'll correct it.
>
>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>> ---
>>>    drivers/dma-buf/dma-buf.c                     | 270 +++++++++++-------
>>>    drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
>>>    drivers/gpu/drm/drm_client.c                  |   4 +-
>>>    drivers/gpu/drm/drm_gem.c                     |  33 +++
>>>    drivers/gpu/drm/drm_gem_framebuffer_helper.c  |   6 +-
>>>    drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    |  10 +-
>>>    drivers/gpu/drm/qxl/qxl_object.c              |  17 +-
>>>    drivers/gpu/drm/qxl/qxl_prime.c               |   4 +-
>>>    .../common/videobuf2/videobuf2-dma-contig.c   |  11 +-
>>>    .../media/common/videobuf2/videobuf2-dma-sg.c |  11 +-
>>>    .../common/videobuf2/videobuf2-vmalloc.c      |  11 +-
>>>    include/drm/drm_gem.h                         |   3 +
>>>    include/linux/dma-buf.h                       |  14 +-
>>>    13 files changed, 241 insertions(+), 159 deletions(-)
>>>
>>> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
>>> index 32f55640890c..64a9909ccfa2 100644
>>> --- a/drivers/dma-buf/dma-buf.c
>>> +++ b/drivers/dma-buf/dma-buf.c
>>> @@ -552,7 +552,6 @@ struct dma_buf *dma_buf_export(const struct
>>> dma_buf_export_info *exp_info)
>>>        file->f_mode |= FMODE_LSEEK;
>>>        dmabuf->file = file;
>>>    -    mutex_init(&dmabuf->lock);
>> Please make removing dmabuf->lock a separate change.
> Alright
>

Re: [Intel-gfx] [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Christian König]

Hi Dmitry,

Am 30.05.22 um 15:26 schrieb Dmitry Osipenko:
> Hello Christian,
>
> On 5/30/22 09:50, Christian König wrote:
>> Hi Dmitry,
>>
>> First of all please separate out this patch from the rest of the series,
>> since this is a complex separate structural change.
> I assume all the patches will go via the DRM tree in the end since the
> rest of the DRM patches in this series depend on this dma-buf change.
> But I see that separation may ease reviewing of the dma-buf changes, so
> let's try it.

That sounds like you are underestimating a bit how much trouble this 
will be.

>> I have tried this before and failed because catching all the locks in
>> the right code paths are very tricky. So expect some fallout from this
>> and make sure the kernel test robot and CI systems are clean.
> Sure, I'll fix up all the reported things in the next iteration.
>
> BTW, have you ever posted yours version of the patch? Will be great if
> we could compare the changed code paths.

No, I never even finished creating it after realizing how much work it 
would be.

>>> This patch introduces new locking convention for dma-buf users. From now
>>> on all dma-buf importers are responsible for holding dma-buf reservation
>>> lock around operations performed over dma-bufs.
>>>
>>> This patch implements the new dma-buf locking convention by:
>>>
>>>     1. Making dma-buf API functions to take the reservation lock.
>>>
>>>     2. Adding new locked variants of the dma-buf API functions for drivers
>>>        that need to manage imported dma-bufs under the held lock.
>> Instead of adding new locked variants please mark all variants which
>> expect to be called without a lock with an _unlocked postfix.
>>
>> This should make it easier to remove those in a follow up patch set and
>> then fully move the locking into the importer.
> Do we really want to move all the locks to the importers? Seems the
> majority of drivers should be happy with the dma-buf helpers handling
> the locking for them.

Yes, I clearly think so.

>
>>>     3. Converting all drivers to the new locking scheme.
>> I have strong doubts that you got all of them. At least radeon and
>> nouveau should grab the reservation lock in their ->attach callbacks
>> somehow.
> Radeon and Nouveau use gem_prime_import_sg_table() and they take resv
> lock already, seems they should be okay (?)

You are looking at the wrong side. You need to fix the export code path, 
not the import ones.

See for example attach on radeon works like this 
drm_gem_map_attach->drm_gem_pin->radeon_gem_prime_pin->radeon_bo_reserve->ttm_bo_reserve->dma_resv_lock.

Same for nouveau and probably a few other exporters as well. That will 
certainly cause a deadlock if you don't fix it.

I strongly suggest to do this step by step, first attach/detach and then 
the rest.

Regards,
Christian.

>
> I assume all the basics should covered in this v6. At minimum Intel,
> Tegra, Panfrost, Lima and Rockchip drivers should be good. If I missed
> something, then please let me know and I'll correct it.
>
>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>> ---
>>>    drivers/dma-buf/dma-buf.c                     | 270 +++++++++++-------
>>>    drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
>>>    drivers/gpu/drm/drm_client.c                  |   4 +-
>>>    drivers/gpu/drm/drm_gem.c                     |  33 +++
>>>    drivers/gpu/drm/drm_gem_framebuffer_helper.c  |   6 +-
>>>    drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    |  10 +-
>>>    drivers/gpu/drm/qxl/qxl_object.c              |  17 +-
>>>    drivers/gpu/drm/qxl/qxl_prime.c               |   4 +-
>>>    .../common/videobuf2/videobuf2-dma-contig.c   |  11 +-
>>>    .../media/common/videobuf2/videobuf2-dma-sg.c |  11 +-
>>>    .../common/videobuf2/videobuf2-vmalloc.c      |  11 +-
>>>    include/drm/drm_gem.h                         |   3 +
>>>    include/linux/dma-buf.h                       |  14 +-
>>>    13 files changed, 241 insertions(+), 159 deletions(-)
>>>
>>> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
>>> index 32f55640890c..64a9909ccfa2 100644
>>> --- a/drivers/dma-buf/dma-buf.c
>>> +++ b/drivers/dma-buf/dma-buf.c
>>> @@ -552,7 +552,6 @@ struct dma_buf *dma_buf_export(const struct
>>> dma_buf_export_info *exp_info)
>>>        file->f_mode |= FMODE_LSEEK;
>>>        dmabuf->file = file;
>>>    -    mutex_init(&dmabuf->lock);
>> Please make removing dmabuf->lock a separate change.
> Alright
>

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Christian König]

Hi Dmitry,

Am 30.05.22 um 15:26 schrieb Dmitry Osipenko:
> Hello Christian,
>
> On 5/30/22 09:50, Christian König wrote:
>> Hi Dmitry,
>>
>> First of all please separate out this patch from the rest of the series,
>> since this is a complex separate structural change.
> I assume all the patches will go via the DRM tree in the end since the
> rest of the DRM patches in this series depend on this dma-buf change.
> But I see that separation may ease reviewing of the dma-buf changes, so
> let's try it.

That sounds like you are underestimating a bit how much trouble this 
will be.

>> I have tried this before and failed because catching all the locks in
>> the right code paths are very tricky. So expect some fallout from this
>> and make sure the kernel test robot and CI systems are clean.
> Sure, I'll fix up all the reported things in the next iteration.
>
> BTW, have you ever posted yours version of the patch? Will be great if
> we could compare the changed code paths.

No, I never even finished creating it after realizing how much work it 
would be.

>>> This patch introduces new locking convention for dma-buf users. From now
>>> on all dma-buf importers are responsible for holding dma-buf reservation
>>> lock around operations performed over dma-bufs.
>>>
>>> This patch implements the new dma-buf locking convention by:
>>>
>>>     1. Making dma-buf API functions to take the reservation lock.
>>>
>>>     2. Adding new locked variants of the dma-buf API functions for drivers
>>>        that need to manage imported dma-bufs under the held lock.
>> Instead of adding new locked variants please mark all variants which
>> expect to be called without a lock with an _unlocked postfix.
>>
>> This should make it easier to remove those in a follow up patch set and
>> then fully move the locking into the importer.
> Do we really want to move all the locks to the importers? Seems the
> majority of drivers should be happy with the dma-buf helpers handling
> the locking for them.

Yes, I clearly think so.

>
>>>     3. Converting all drivers to the new locking scheme.
>> I have strong doubts that you got all of them. At least radeon and
>> nouveau should grab the reservation lock in their ->attach callbacks
>> somehow.
> Radeon and Nouveau use gem_prime_import_sg_table() and they take resv
> lock already, seems they should be okay (?)

You are looking at the wrong side. You need to fix the export code path, 
not the import ones.

See for example attach on radeon works like this 
drm_gem_map_attach->drm_gem_pin->radeon_gem_prime_pin->radeon_bo_reserve->ttm_bo_reserve->dma_resv_lock.

Same for nouveau and probably a few other exporters as well. That will 
certainly cause a deadlock if you don't fix it.

I strongly suggest to do this step by step, first attach/detach and then 
the rest.

Regards,
Christian.

>
> I assume all the basics should covered in this v6. At minimum Intel,
> Tegra, Panfrost, Lima and Rockchip drivers should be good. If I missed
> something, then please let me know and I'll correct it.
>
>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>> ---
>>>    drivers/dma-buf/dma-buf.c                     | 270 +++++++++++-------
>>>    drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |   6 +-
>>>    drivers/gpu/drm/drm_client.c                  |   4 +-
>>>    drivers/gpu/drm/drm_gem.c                     |  33 +++
>>>    drivers/gpu/drm/drm_gem_framebuffer_helper.c  |   6 +-
>>>    drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    |  10 +-
>>>    drivers/gpu/drm/qxl/qxl_object.c              |  17 +-
>>>    drivers/gpu/drm/qxl/qxl_prime.c               |   4 +-
>>>    .../common/videobuf2/videobuf2-dma-contig.c   |  11 +-
>>>    .../media/common/videobuf2/videobuf2-dma-sg.c |  11 +-
>>>    .../common/videobuf2/videobuf2-vmalloc.c      |  11 +-
>>>    include/drm/drm_gem.h                         |   3 +
>>>    include/linux/dma-buf.h                       |  14 +-
>>>    13 files changed, 241 insertions(+), 159 deletions(-)
>>>
>>> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
>>> index 32f55640890c..64a9909ccfa2 100644
>>> --- a/drivers/dma-buf/dma-buf.c
>>> +++ b/drivers/dma-buf/dma-buf.c
>>> @@ -552,7 +552,6 @@ struct dma_buf *dma_buf_export(const struct
>>> dma_buf_export_info *exp_info)
>>>        file->f_mode |= FMODE_LSEEK;
>>>        dmabuf->file = file;
>>>    -    mutex_init(&dmabuf->lock);
>> Please make removing dmabuf->lock a separate change.
> Alright
>

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dmitry Osipenko]

On 5/30/22 16:41, Christian König wrote:
> Hi Dmitry,
> 
> Am 30.05.22 um 15:26 schrieb Dmitry Osipenko:
>> Hello Christian,
>>
>> On 5/30/22 09:50, Christian König wrote:
>>> Hi Dmitry,
>>>
>>> First of all please separate out this patch from the rest of the series,
>>> since this is a complex separate structural change.
>> I assume all the patches will go via the DRM tree in the end since the
>> rest of the DRM patches in this series depend on this dma-buf change.
>> But I see that separation may ease reviewing of the dma-buf changes, so
>> let's try it.
> 
> That sounds like you are underestimating a bit how much trouble this
> will be.
> 
>>> I have tried this before and failed because catching all the locks in
>>> the right code paths are very tricky. So expect some fallout from this
>>> and make sure the kernel test robot and CI systems are clean.
>> Sure, I'll fix up all the reported things in the next iteration.
>>
>> BTW, have you ever posted yours version of the patch? Will be great if
>> we could compare the changed code paths.
> 
> No, I never even finished creating it after realizing how much work it
> would be.
> 
>>>> This patch introduces new locking convention for dma-buf users. From
>>>> now
>>>> on all dma-buf importers are responsible for holding dma-buf
>>>> reservation
>>>> lock around operations performed over dma-bufs.
>>>>
>>>> This patch implements the new dma-buf locking convention by:
>>>>
>>>>     1. Making dma-buf API functions to take the reservation lock.
>>>>
>>>>     2. Adding new locked variants of the dma-buf API functions for
>>>> drivers
>>>>        that need to manage imported dma-bufs under the held lock.
>>> Instead of adding new locked variants please mark all variants which
>>> expect to be called without a lock with an _unlocked postfix.
>>>
>>> This should make it easier to remove those in a follow up patch set and
>>> then fully move the locking into the importer.
>> Do we really want to move all the locks to the importers? Seems the
>> majority of drivers should be happy with the dma-buf helpers handling
>> the locking for them.
> 
> Yes, I clearly think so.
> 
>>
>>>>     3. Converting all drivers to the new locking scheme.
>>> I have strong doubts that you got all of them. At least radeon and
>>> nouveau should grab the reservation lock in their ->attach callbacks
>>> somehow.
>> Radeon and Nouveau use gem_prime_import_sg_table() and they take resv
>> lock already, seems they should be okay (?)
> 
> You are looking at the wrong side. You need to fix the export code path,
> not the import ones.
> 
> See for example attach on radeon works like this
> drm_gem_map_attach->drm_gem_pin->radeon_gem_prime_pin->radeon_bo_reserve->ttm_bo_reserve->dma_resv_lock.

Yeah, I was looking at the both sides, but missed this one.

> Same for nouveau and probably a few other exporters as well. That will
> certainly cause a deadlock if you don't fix it.
> 
> I strongly suggest to do this step by step, first attach/detach and then
> the rest.

Thank you very much for the suggestions. I'll implement them in the next
version.

-- 
Best regards,
Dmitry

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dmitry Osipenko]

On 5/30/22 16:41, Christian König wrote:
> Hi Dmitry,
> 
> Am 30.05.22 um 15:26 schrieb Dmitry Osipenko:
>> Hello Christian,
>>
>> On 5/30/22 09:50, Christian König wrote:
>>> Hi Dmitry,
>>>
>>> First of all please separate out this patch from the rest of the series,
>>> since this is a complex separate structural change.
>> I assume all the patches will go via the DRM tree in the end since the
>> rest of the DRM patches in this series depend on this dma-buf change.
>> But I see that separation may ease reviewing of the dma-buf changes, so
>> let's try it.
> 
> That sounds like you are underestimating a bit how much trouble this
> will be.
> 
>>> I have tried this before and failed because catching all the locks in
>>> the right code paths are very tricky. So expect some fallout from this
>>> and make sure the kernel test robot and CI systems are clean.
>> Sure, I'll fix up all the reported things in the next iteration.
>>
>> BTW, have you ever posted yours version of the patch? Will be great if
>> we could compare the changed code paths.
> 
> No, I never even finished creating it after realizing how much work it
> would be.
> 
>>>> This patch introduces new locking convention for dma-buf users. From
>>>> now
>>>> on all dma-buf importers are responsible for holding dma-buf
>>>> reservation
>>>> lock around operations performed over dma-bufs.
>>>>
>>>> This patch implements the new dma-buf locking convention by:
>>>>
>>>>     1. Making dma-buf API functions to take the reservation lock.
>>>>
>>>>     2. Adding new locked variants of the dma-buf API functions for
>>>> drivers
>>>>        that need to manage imported dma-bufs under the held lock.
>>> Instead of adding new locked variants please mark all variants which
>>> expect to be called without a lock with an _unlocked postfix.
>>>
>>> This should make it easier to remove those in a follow up patch set and
>>> then fully move the locking into the importer.
>> Do we really want to move all the locks to the importers? Seems the
>> majority of drivers should be happy with the dma-buf helpers handling
>> the locking for them.
> 
> Yes, I clearly think so.
> 
>>
>>>>     3. Converting all drivers to the new locking scheme.
>>> I have strong doubts that you got all of them. At least radeon and
>>> nouveau should grab the reservation lock in their ->attach callbacks
>>> somehow.
>> Radeon and Nouveau use gem_prime_import_sg_table() and they take resv
>> lock already, seems they should be okay (?)
> 
> You are looking at the wrong side. You need to fix the export code path,
> not the import ones.
> 
> See for example attach on radeon works like this
> drm_gem_map_attach->drm_gem_pin->radeon_gem_prime_pin->radeon_bo_reserve->ttm_bo_reserve->dma_resv_lock.

Yeah, I was looking at the both sides, but missed this one.

> Same for nouveau and probably a few other exporters as well. That will
> certainly cause a deadlock if you don't fix it.
> 
> I strongly suggest to do this step by step, first attach/detach and then
> the rest.

Thank you very much for the suggestions. I'll implement them in the next
version.

-- 
Best regards,
Dmitry

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> Introduce a common DRM SHMEM shrinker framework that allows to reduce
> code duplication among DRM drivers by replacing theirs custom shrinker
> implementations with the generic shrinker.
>
> In order to start using DRM SHMEM shrinker drivers should:
>
> 1. Implement new evict() shmem object callback.
> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>    activate shrinking of shmem GEMs.
>
> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>
> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>

So I guess I get a price for being blind since forever, because this
thing existed since at least 2013. I just stumbled over
llist_lru.[hc], a purpose built list helper for shrinkers. I think we
should try to adopt that so that our gpu shrinkers look more like
shrinkers for everything else.

Apologies for this, since I fear this might cause a bit of churn.
Hopefully it's all contained to the list manipulation code in shmem
helpers, I don't think this should leak any further.
-Daniel

> ---
>  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>  include/drm/drm_device.h                      |   4 +
>  include/drm/drm_gem_shmem_helper.h            |  87 ++-
>  5 files changed, 594 insertions(+), 49 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 555fe212bd98..4cd0b5913492 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>
> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv >= 0) && shmem->evict &&
> +               shmem->eviction_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> +}
> +
> +static void
> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (!gem_shrinker || obj->import_attach)
> +               return;
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       if (drm_gem_shmem_is_evictable(shmem) ||
> +           drm_gem_shmem_is_purgeable(shmem))
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> +       else if (shmem->madv < 0)
> +               list_del_init(&shmem->madv_list);
> +       else if (shmem->evicted)
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> +       else if (!shmem->pages)
> +               list_del_init(&shmem->madv_list);
> +       else
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +}
> +
>  /**
>   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>   * @shmem: shmem GEM object to free
> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>         } else {
>                 dma_resv_lock(shmem->base.resv, NULL);
>
> +               /* take out shmem GEM object from the memory shrinker */
> +               drm_gem_shmem_madvise(shmem, -1);
> +
>                 WARN_ON(shmem->vmap_use_count);
>
>                 if (shmem->sgt) {
> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>                         sg_free_table(shmem->sgt);
>                         kfree(shmem->sgt);
>                 }
> -               if (shmem->pages)
> +               if (shmem->pages_use_count)
>                         drm_gem_shmem_put_pages(shmem);
>
>                 WARN_ON(shmem->pages_use_count);
> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>
> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->eviction_enabled = true;
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> +
> +/**
> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be purged. Initially purging is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->purge_enabled = true;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> +
> +static int
> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
>         struct page **pages;
>
> -       if (shmem->pages_use_count++ > 0)
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0) {
> +               WARN_ON(shmem->pages);
> +               return -ENOMEM;
> +       }
> +
> +       if (shmem->pages) {
> +               WARN_ON(!shmem->evicted);
>                 return 0;
> +       }
> +
> +       if (WARN_ON(!shmem->pages_use_count))
> +               return -EINVAL;
>
>         pages = drm_gem_get_pages(obj);
>         if (IS_ERR(pages)) {
>                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> -               shmem->pages_use_count = 0;
>                 return PTR_ERR(pages);
>         }
>
> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>         return 0;
>  }
>
> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       int err;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       if (shmem->pages_use_count++ > 0) {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err)
> +                       goto err_zero_use;
> +
> +               return 0;
> +       }
> +
> +       err = drm_gem_shmem_acquire_pages(shmem);
> +       if (err)
> +               goto err_zero_use;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +
> +err_zero_use:
> +       shmem->pages_use_count = 0;
> +
> +       return err;
> +}
> +
> +static void
> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       if (!shmem->pages) {
> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> +               return;
> +       }
> +
> +#ifdef CONFIG_X86
> +       if (shmem->map_wc)
> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> +#endif
> +
> +       drm_gem_put_pages(obj, shmem->pages,
> +                         shmem->pages_mark_dirty_on_put,
> +                         shmem->pages_mark_accessed_on_put);
> +       shmem->pages = NULL;
> +}
> +
>  /*
>   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>   * @shmem: shmem GEM object
> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>   */
>  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>  {
> -       struct drm_gem_object *obj = &shmem->base;
> -
>         dma_resv_assert_held(shmem->base.resv);
>
>         if (WARN_ON_ONCE(!shmem->pages_use_count))
> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>         if (--shmem->pages_use_count > 0)
>                 return;
>
> -#ifdef CONFIG_X86
> -       if (shmem->map_wc)
> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> -#endif
> +       drm_gem_shmem_release_pages(shmem);
>
> -       drm_gem_put_pages(obj, shmem->pages,
> -                         shmem->pages_mark_dirty_on_put,
> -                         shmem->pages_mark_accessed_on_put);
> -       shmem->pages = NULL;
> +       drm_gem_shmem_update_pages_state(shmem);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>
> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>   */
>  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>  {
> +       int ret;
> +
>         dma_resv_assert_held(shmem->base.resv);
>
>         WARN_ON(shmem->base.import_attach);
>
> -       return drm_gem_shmem_get_pages(shmem);
> +       ret = drm_gem_shmem_get_pages(shmem);
> +       if (!ret)
> +               shmem->pages_pin_count++;
> +
> +       return ret;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_pin);
>
> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>         WARN_ON(shmem->base.import_attach);
>
>         drm_gem_shmem_put_pages(shmem);
> +
> +       shmem->pages_pin_count--;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_unpin);
>
> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>                         return 0;
>                 }
>
> -               ret = drm_gem_shmem_get_pages(shmem);
> +               ret = drm_gem_shmem_pin(shmem);
>                 if (ret)
>                         goto err_zero_use;
>
> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>
>  err_put_pages:
>         if (!obj->import_attach)
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>  err_zero_use:
>         shmem->vmap_use_count = 0;
>
> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>                         return;
>
>                 vunmap(shmem->vaddr);
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>         }
>
>         shmem->vaddr = NULL;
> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>
>         madv = shmem->madv;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         return (madv >= 0);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_madvise);
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> + *                           hardware access to the memory.
> + * @shmem: shmem GEM object
> + *
> + * This function moves shmem GEM back to memory if it was previously evicted
> + * by the memory shrinker. The GEM is ready to use on success.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
> -       struct drm_device *dev = obj->dev;
> +       struct sg_table *sgt;
> +       int err;
>
>         dma_resv_assert_held(shmem->base.resv);
>
> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +       if (shmem->evicted) {
> +               err = drm_gem_shmem_acquire_pages(shmem);
> +               if (err)
> +                       return err;
> +
> +               sgt = drm_gem_shmem_get_sg_table(shmem);
> +               if (IS_ERR(sgt))
> +                       return PTR_ERR(sgt);
> +
> +               err = dma_map_sgtable(obj->dev->dev, sgt,
> +                                     DMA_BIDIRECTIONAL, 0);
> +               if (err) {
> +                       sg_free_table(sgt);
> +                       kfree(sgt);
> +                       return err;
> +               }
>
> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> -       sg_free_table(shmem->sgt);
> -       kfree(shmem->sgt);
> -       shmem->sgt = NULL;
> +               shmem->sgt = sgt;
> +               shmem->evicted = false;
>
> -       drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_update_pages_state(shmem);
> +       }
>
> -       shmem->madv = -1;
> +       if (!shmem->pages)
> +               return -ENOMEM;
>
> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> -       drm_gem_free_mmap_offset(obj);
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>
> -       /* Our goal here is to return as much of the memory as
> -        * is possible back to the system as we are called from OOM.
> -        * To do this we must instruct the shmfs to drop all of its
> -        * backing pages, *now*.
> -        */
> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_device *dev = obj->dev;
>
> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +       if (shmem->evicted)
> +               return;
> +
> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> +       drm_gem_shmem_release_pages(shmem);
> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> +
> +       sg_free_table(shmem->sgt);
> +       kfree(shmem->sgt);
> +       shmem->sgt = NULL;
>  }
> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>
>  /**
>   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>         vm_fault_t ret;
>         struct page *page;
>         pgoff_t page_offset;
> +       bool pages_unpinned;
> +       int err;
>
>         /* We don't use vmf->pgoff since that has the fake offset */
>         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>
>         dma_resv_lock(shmem->base.resv, NULL);
>
> -       if (page_offset >= num_pages ||
> -           WARN_ON_ONCE(!shmem->pages) ||
> -           shmem->madv < 0) {
> +       /* Sanity-check that we have the pages pointer when it should present */
> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> +
> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>                 ret = VM_FAULT_SIGBUS;
>         } else {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err) {
> +                       ret = VM_FAULT_OOM;
> +                       goto unlock;
> +               }
> +
>                 page = shmem->pages[page_offset];
>
>                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>         }
>
> +unlock:
>         dma_resv_unlock(shmem->base.resv);
>
>         return ret;
> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>  {
>         struct drm_gem_object *obj = vma->vm_private_data;
>         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -       int ret;
>
>         WARN_ON(shmem->base.import_attach);
>
>         dma_resv_lock(shmem->base.resv, NULL);
> -       ret = drm_gem_shmem_get_pages(shmem);
> -       WARN_ON_ONCE(ret != 0);
> +
> +       if (drm_gem_shmem_get_pages(shmem))
> +               shmem->pages_use_count++;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
>         dma_resv_unlock(shmem->base.resv);
>
>         drm_gem_vm_open(vma);
> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                               struct drm_printer *p, unsigned int indent)
>  {
> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>
>         if (shmem->base.import_attach)
> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>                                   shmem->vmap_use_count);
>
> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_print_info);
>
> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>
>         shmem->sgt = sgt;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         dma_resv_unlock(shmem->base.resv);
>
>         return sgt;
> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>
> +static struct drm_gem_shmem_shrinker *
> +to_drm_shrinker(struct shrinker *shrinker)
> +{
> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> +                                    struct shrink_control *sc)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       unsigned long count = 0;
> +
> +       if (!mutex_trylock(&gem_shrinker->lock))
> +               return 0;
> +
> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> +               count += shmem->base.size;
> +
> +               if (count >= SHRINK_EMPTY)
> +                       break;
> +       }
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       if (count >= SHRINK_EMPTY)
> +               return SHRINK_EMPTY - 1;
> +
> +       return count ?: SHRINK_EMPTY;
> +}
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> +       WARN_ON(shmem->evicted);
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +
> +       shmem->evicted = true;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> +
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +       drm_gem_free_mmap_offset(obj);
> +
> +       /* Our goal here is to return as much of the memory as
> +        * is possible back to the system as we are called from OOM.
> +        * To do this we must instruct the shmfs to drop all of its
> +        * backing pages, *now*.
> +        */
> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +
> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +
> +       shmem->madv = -1;
> +       shmem->evicted = false;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> +                                       unsigned long nr_to_scan,
> +                                       bool *lock_contention,
> +                                       bool evict)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       struct list_head still_in_list;
> +       struct drm_gem_object *obj;
> +       unsigned long freed = 0;
> +       size_t page_count;
> +       int err;
> +
> +       INIT_LIST_HEAD(&still_in_list);
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       while (freed < nr_to_scan) {
> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> +                                                typeof(*shmem), madv_list);
> +               if (!shmem)
> +                       break;
> +
> +               obj = &shmem->base;
> +               page_count = obj->size >> PAGE_SHIFT;
> +               list_move_tail(&shmem->madv_list, &still_in_list);
> +
> +               if (evict) {
> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> +                           get_nr_swap_pages() < page_count)
> +                               continue;
> +               } else {
> +                       if (!drm_gem_shmem_is_purgeable(shmem))
> +                               continue;
> +               }
> +
> +               /*
> +                * If it's in the process of being freed, gem_object->free()
> +                * may be blocked on lock waiting to remove it.  So just
> +                * skip it.
> +                */
> +               if (!kref_get_unless_zero(&obj->refcount))
> +                       continue;
> +
> +               mutex_unlock(&gem_shrinker->lock);
> +
> +               /* prevent racing with job-submission code paths */
> +               if (!dma_resv_trylock(obj->resv)) {
> +                       *lock_contention |= true;
> +                       goto shrinker_lock;
> +               }
> +
> +               /* prevent racing with the dma-buf importing/exporting */
> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> +                       *lock_contention |= true;
> +                       goto resv_unlock;
> +               }
> +
> +               /* check whether h/w uses this object */
> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> +                       goto object_name_unlock;
> +
> +               /* re-check whether eviction status hasn't changed */
> +               if (!drm_gem_shmem_is_evictable(shmem) &&
> +                   !drm_gem_shmem_is_purgeable(shmem))
> +                       goto object_name_unlock;
> +
> +               err = shmem->evict(shmem);
> +               if (!err)
> +                       freed += obj->size >> PAGE_SHIFT;
> +
> +object_name_unlock:
> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> +resv_unlock:
> +               dma_resv_unlock(obj->resv);
> +shrinker_lock:
> +               drm_gem_object_put(&shmem->base);
> +               mutex_lock(&gem_shrinker->lock);
> +       }
> +
> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       return freed;
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> +                                   struct shrink_control *sc)
> +{
> +       unsigned long nr_to_scan = sc->nr_to_scan;
> +       bool lock_contention = false;
> +       unsigned long freed;
> +
> +       /* purge as many objects as we can */
> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> +                                                       &lock_contention, false);
> +
> +       /* evict as many objects as we can */
> +       if (freed < nr_to_scan)
> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> +                                                                nr_to_scan - freed,
> +                                                                &lock_contention,
> +                                                                true);
> +
> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> +}
> +
> +/**
> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> + * @dev: DRM device
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker;
> +       int err;
> +
> +       if (WARN_ON(dev->shmem_shrinker))
> +               return -EBUSY;
> +
> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> +       if (!gem_shrinker)
> +               return -ENOMEM;
> +
> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> +       gem_shrinker->dev = dev;
> +
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> +       mutex_init(&gem_shrinker->lock);
> +
> +       dev->shmem_shrinker = gem_shrinker;
> +
> +       err = register_shrinker(&gem_shrinker->base);
> +       if (err) {
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +               return err;
> +       }
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> +
> +/**
> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> + * @dev: DRM device
> + */
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> +
> +       if (gem_shrinker) {
> +               unregister_shrinker(&gem_shrinker->base);
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> +               mutex_destroy(&gem_shrinker->lock);
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +       }
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> +
>  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>  MODULE_IMPORT_NS(DMA_BUF);
>  MODULE_LICENSE("GPL v2");
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> index a4bedfeb2ec4..7cc32556f908 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> @@ -15,6 +15,13 @@
>  #include "panfrost_gem.h"
>  #include "panfrost_mmu.h"
>
> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv > 0) &&
> +               !shmem->pages_pin_count && shmem->sgt &&
> +               !shmem->base.dma_buf && !shmem->base.import_attach;
> +}
> +
>  static unsigned long
>  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>  {
> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>                 return 0;
>
>         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -               if (drm_gem_shmem_is_purgeable(shmem))
> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>                         count += shmem->base.size >> PAGE_SHIFT;
>         }
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> index b2d93cb12ebf..81bacc7e1873 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>         uint32_t hw_res_handle;
>         bool dumb;
>         bool created;
> +       bool detached;
>         bool host3d_blob, guest_blob;
>         uint32_t blob_mem, blob_flags;
>
> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>
>  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>
> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> +
>  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>                                uint32_t *resid);
>  /* virtgpu_prime.c */
> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> index 9923c7a6885e..929546cad894 100644
> --- a/include/drm/drm_device.h
> +++ b/include/drm/drm_device.h
> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>  struct drm_vma_offset_manager;
>  struct drm_vram_mm;
>  struct drm_fb_helper;
> +struct drm_gem_shmem_shrinker;
>
>  struct inode;
>
> @@ -277,6 +278,9 @@ struct drm_device {
>         /** @vram_mm: VRAM MM memory manager */
>         struct drm_vram_mm *vram_mm;
>
> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> +
>         /**
>          * @switch_power_state:
>          *
> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> index 9a8983ee8abe..62c640678a91 100644
> --- a/include/drm/drm_gem_shmem_helper.h
> +++ b/include/drm/drm_gem_shmem_helper.h
> @@ -6,6 +6,7 @@
>  #include <linux/fs.h>
>  #include <linux/mm.h>
>  #include <linux/mutex.h>
> +#include <linux/shrinker.h>
>
>  #include <drm/drm_file.h>
>  #include <drm/drm_gem.h>
> @@ -15,6 +16,7 @@
>  struct dma_buf_attachment;
>  struct drm_mode_create_dumb;
>  struct drm_printer;
> +struct drm_device;
>  struct sg_table;
>
>  /**
> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>          */
>         unsigned int pages_use_count;
>
> +       /**
> +        * @pages_pin_count:
> +        *
> +        * Reference count on the pinned pages table.
> +        * The pages can be evicted by memory shrinker
> +        * when the count reaches zero.
> +        */
> +       unsigned int pages_pin_count;
> +
>         /**
>          * @madv: State for madvise
>          *
>          * 0 is active/inuse.
> +        * 1 is not-needed/can-be-purged
>          * A negative value is the object is purged.
> -        * Positive values are driver specific and not used by the helpers.
>          */
>         int madv;
>
> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>          * @map_wc: map object write-combined (instead of using shmem defaults).
>          */
>         bool map_wc;
> +
> +       /**
> +        * @eviction_enabled:
> +        *
> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool eviction_enabled;
> +
> +       /**
> +        * @purge_enabled:
> +        *
> +        * The shmem pages can be purged only if @purge_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool purge_enabled;
> +
> +       /**
> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> +        * Used internally by memory shrinker.
> +        */
> +       bool evicted;
> +
> +       /**
> +        * @evict:
> +        *
> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> +        * GEM's DMA reservation is kept locked by the shrinker. This is
> +        * optional callback that should be specified by drivers.
> +        *
> +        * Returns 0 on success, or -errno on error.
> +        */
> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>  };
>
>  #define to_drm_gem_shmem_obj(obj) \
> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>
>  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> +
>  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>  {
> -       return (shmem->madv > 0) &&
> -               !shmem->vmap_use_count && shmem->sgt &&
> -               !shmem->base.dma_buf && !shmem->base.import_attach;
> +       return (shmem->madv > 0) && shmem->evict &&
> +               shmem->purge_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>  }
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>
>  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>         return drm_gem_shmem_mmap(shmem, vma);
>  }
>
> +/**
> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> + */
> +struct drm_gem_shmem_shrinker {
> +       /** @base: Shrinker for purging shmem GEM objects */
> +       struct shrinker base;
> +
> +       /** @lock: Protects @lru_* */
> +       struct mutex lock;
> +
> +       /** @lru_pinned: List of pinned shmem GEM objects */
> +       struct list_head lru_pinned;
> +
> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> +       struct list_head lru_evictable;
> +
> +       /** @lru_evicted: List of evicted shmem GEM objects */
> +       struct list_head lru_evicted;
> +
> +       /** @dev: DRM device that uses this shrinker */
> +       struct drm_device *dev;
> +};
> +
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> +
>  /*
>   * Driver ops
>   */
> --
> 2.35.3
>


-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> Introduce a common DRM SHMEM shrinker framework that allows to reduce
> code duplication among DRM drivers by replacing theirs custom shrinker
> implementations with the generic shrinker.
>
> In order to start using DRM SHMEM shrinker drivers should:
>
> 1. Implement new evict() shmem object callback.
> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>    activate shrinking of shmem GEMs.
>
> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>
> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>

So I guess I get a price for being blind since forever, because this
thing existed since at least 2013. I just stumbled over
llist_lru.[hc], a purpose built list helper for shrinkers. I think we
should try to adopt that so that our gpu shrinkers look more like
shrinkers for everything else.

Apologies for this, since I fear this might cause a bit of churn.
Hopefully it's all contained to the list manipulation code in shmem
helpers, I don't think this should leak any further.
-Daniel

> ---
>  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>  include/drm/drm_device.h                      |   4 +
>  include/drm/drm_gem_shmem_helper.h            |  87 ++-
>  5 files changed, 594 insertions(+), 49 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 555fe212bd98..4cd0b5913492 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>
> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv >= 0) && shmem->evict &&
> +               shmem->eviction_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> +}
> +
> +static void
> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (!gem_shrinker || obj->import_attach)
> +               return;
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       if (drm_gem_shmem_is_evictable(shmem) ||
> +           drm_gem_shmem_is_purgeable(shmem))
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> +       else if (shmem->madv < 0)
> +               list_del_init(&shmem->madv_list);
> +       else if (shmem->evicted)
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> +       else if (!shmem->pages)
> +               list_del_init(&shmem->madv_list);
> +       else
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +}
> +
>  /**
>   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>   * @shmem: shmem GEM object to free
> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>         } else {
>                 dma_resv_lock(shmem->base.resv, NULL);
>
> +               /* take out shmem GEM object from the memory shrinker */
> +               drm_gem_shmem_madvise(shmem, -1);
> +
>                 WARN_ON(shmem->vmap_use_count);
>
>                 if (shmem->sgt) {
> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>                         sg_free_table(shmem->sgt);
>                         kfree(shmem->sgt);
>                 }
> -               if (shmem->pages)
> +               if (shmem->pages_use_count)
>                         drm_gem_shmem_put_pages(shmem);
>
>                 WARN_ON(shmem->pages_use_count);
> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>
> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->eviction_enabled = true;
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> +
> +/**
> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be purged. Initially purging is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->purge_enabled = true;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> +
> +static int
> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
>         struct page **pages;
>
> -       if (shmem->pages_use_count++ > 0)
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0) {
> +               WARN_ON(shmem->pages);
> +               return -ENOMEM;
> +       }
> +
> +       if (shmem->pages) {
> +               WARN_ON(!shmem->evicted);
>                 return 0;
> +       }
> +
> +       if (WARN_ON(!shmem->pages_use_count))
> +               return -EINVAL;
>
>         pages = drm_gem_get_pages(obj);
>         if (IS_ERR(pages)) {
>                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> -               shmem->pages_use_count = 0;
>                 return PTR_ERR(pages);
>         }
>
> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>         return 0;
>  }
>
> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       int err;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       if (shmem->pages_use_count++ > 0) {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err)
> +                       goto err_zero_use;
> +
> +               return 0;
> +       }
> +
> +       err = drm_gem_shmem_acquire_pages(shmem);
> +       if (err)
> +               goto err_zero_use;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +
> +err_zero_use:
> +       shmem->pages_use_count = 0;
> +
> +       return err;
> +}
> +
> +static void
> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       if (!shmem->pages) {
> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> +               return;
> +       }
> +
> +#ifdef CONFIG_X86
> +       if (shmem->map_wc)
> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> +#endif
> +
> +       drm_gem_put_pages(obj, shmem->pages,
> +                         shmem->pages_mark_dirty_on_put,
> +                         shmem->pages_mark_accessed_on_put);
> +       shmem->pages = NULL;
> +}
> +
>  /*
>   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>   * @shmem: shmem GEM object
> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>   */
>  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>  {
> -       struct drm_gem_object *obj = &shmem->base;
> -
>         dma_resv_assert_held(shmem->base.resv);
>
>         if (WARN_ON_ONCE(!shmem->pages_use_count))
> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>         if (--shmem->pages_use_count > 0)
>                 return;
>
> -#ifdef CONFIG_X86
> -       if (shmem->map_wc)
> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> -#endif
> +       drm_gem_shmem_release_pages(shmem);
>
> -       drm_gem_put_pages(obj, shmem->pages,
> -                         shmem->pages_mark_dirty_on_put,
> -                         shmem->pages_mark_accessed_on_put);
> -       shmem->pages = NULL;
> +       drm_gem_shmem_update_pages_state(shmem);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>
> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>   */
>  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>  {
> +       int ret;
> +
>         dma_resv_assert_held(shmem->base.resv);
>
>         WARN_ON(shmem->base.import_attach);
>
> -       return drm_gem_shmem_get_pages(shmem);
> +       ret = drm_gem_shmem_get_pages(shmem);
> +       if (!ret)
> +               shmem->pages_pin_count++;
> +
> +       return ret;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_pin);
>
> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>         WARN_ON(shmem->base.import_attach);
>
>         drm_gem_shmem_put_pages(shmem);
> +
> +       shmem->pages_pin_count--;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_unpin);
>
> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>                         return 0;
>                 }
>
> -               ret = drm_gem_shmem_get_pages(shmem);
> +               ret = drm_gem_shmem_pin(shmem);
>                 if (ret)
>                         goto err_zero_use;
>
> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>
>  err_put_pages:
>         if (!obj->import_attach)
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>  err_zero_use:
>         shmem->vmap_use_count = 0;
>
> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>                         return;
>
>                 vunmap(shmem->vaddr);
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>         }
>
>         shmem->vaddr = NULL;
> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>
>         madv = shmem->madv;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         return (madv >= 0);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_madvise);
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> + *                           hardware access to the memory.
> + * @shmem: shmem GEM object
> + *
> + * This function moves shmem GEM back to memory if it was previously evicted
> + * by the memory shrinker. The GEM is ready to use on success.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
> -       struct drm_device *dev = obj->dev;
> +       struct sg_table *sgt;
> +       int err;
>
>         dma_resv_assert_held(shmem->base.resv);
>
> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +       if (shmem->evicted) {
> +               err = drm_gem_shmem_acquire_pages(shmem);
> +               if (err)
> +                       return err;
> +
> +               sgt = drm_gem_shmem_get_sg_table(shmem);
> +               if (IS_ERR(sgt))
> +                       return PTR_ERR(sgt);
> +
> +               err = dma_map_sgtable(obj->dev->dev, sgt,
> +                                     DMA_BIDIRECTIONAL, 0);
> +               if (err) {
> +                       sg_free_table(sgt);
> +                       kfree(sgt);
> +                       return err;
> +               }
>
> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> -       sg_free_table(shmem->sgt);
> -       kfree(shmem->sgt);
> -       shmem->sgt = NULL;
> +               shmem->sgt = sgt;
> +               shmem->evicted = false;
>
> -       drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_update_pages_state(shmem);
> +       }
>
> -       shmem->madv = -1;
> +       if (!shmem->pages)
> +               return -ENOMEM;
>
> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> -       drm_gem_free_mmap_offset(obj);
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>
> -       /* Our goal here is to return as much of the memory as
> -        * is possible back to the system as we are called from OOM.
> -        * To do this we must instruct the shmfs to drop all of its
> -        * backing pages, *now*.
> -        */
> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_device *dev = obj->dev;
>
> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +       if (shmem->evicted)
> +               return;
> +
> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> +       drm_gem_shmem_release_pages(shmem);
> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> +
> +       sg_free_table(shmem->sgt);
> +       kfree(shmem->sgt);
> +       shmem->sgt = NULL;
>  }
> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>
>  /**
>   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>         vm_fault_t ret;
>         struct page *page;
>         pgoff_t page_offset;
> +       bool pages_unpinned;
> +       int err;
>
>         /* We don't use vmf->pgoff since that has the fake offset */
>         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>
>         dma_resv_lock(shmem->base.resv, NULL);
>
> -       if (page_offset >= num_pages ||
> -           WARN_ON_ONCE(!shmem->pages) ||
> -           shmem->madv < 0) {
> +       /* Sanity-check that we have the pages pointer when it should present */
> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> +
> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>                 ret = VM_FAULT_SIGBUS;
>         } else {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err) {
> +                       ret = VM_FAULT_OOM;
> +                       goto unlock;
> +               }
> +
>                 page = shmem->pages[page_offset];
>
>                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>         }
>
> +unlock:
>         dma_resv_unlock(shmem->base.resv);
>
>         return ret;
> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>  {
>         struct drm_gem_object *obj = vma->vm_private_data;
>         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -       int ret;
>
>         WARN_ON(shmem->base.import_attach);
>
>         dma_resv_lock(shmem->base.resv, NULL);
> -       ret = drm_gem_shmem_get_pages(shmem);
> -       WARN_ON_ONCE(ret != 0);
> +
> +       if (drm_gem_shmem_get_pages(shmem))
> +               shmem->pages_use_count++;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
>         dma_resv_unlock(shmem->base.resv);
>
>         drm_gem_vm_open(vma);
> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                               struct drm_printer *p, unsigned int indent)
>  {
> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>
>         if (shmem->base.import_attach)
> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>                                   shmem->vmap_use_count);
>
> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_print_info);
>
> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>
>         shmem->sgt = sgt;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         dma_resv_unlock(shmem->base.resv);
>
>         return sgt;
> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>
> +static struct drm_gem_shmem_shrinker *
> +to_drm_shrinker(struct shrinker *shrinker)
> +{
> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> +                                    struct shrink_control *sc)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       unsigned long count = 0;
> +
> +       if (!mutex_trylock(&gem_shrinker->lock))
> +               return 0;
> +
> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> +               count += shmem->base.size;
> +
> +               if (count >= SHRINK_EMPTY)
> +                       break;
> +       }
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       if (count >= SHRINK_EMPTY)
> +               return SHRINK_EMPTY - 1;
> +
> +       return count ?: SHRINK_EMPTY;
> +}
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> +       WARN_ON(shmem->evicted);
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +
> +       shmem->evicted = true;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> +
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +       drm_gem_free_mmap_offset(obj);
> +
> +       /* Our goal here is to return as much of the memory as
> +        * is possible back to the system as we are called from OOM.
> +        * To do this we must instruct the shmfs to drop all of its
> +        * backing pages, *now*.
> +        */
> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +
> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +
> +       shmem->madv = -1;
> +       shmem->evicted = false;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> +                                       unsigned long nr_to_scan,
> +                                       bool *lock_contention,
> +                                       bool evict)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       struct list_head still_in_list;
> +       struct drm_gem_object *obj;
> +       unsigned long freed = 0;
> +       size_t page_count;
> +       int err;
> +
> +       INIT_LIST_HEAD(&still_in_list);
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       while (freed < nr_to_scan) {
> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> +                                                typeof(*shmem), madv_list);
> +               if (!shmem)
> +                       break;
> +
> +               obj = &shmem->base;
> +               page_count = obj->size >> PAGE_SHIFT;
> +               list_move_tail(&shmem->madv_list, &still_in_list);
> +
> +               if (evict) {
> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> +                           get_nr_swap_pages() < page_count)
> +                               continue;
> +               } else {
> +                       if (!drm_gem_shmem_is_purgeable(shmem))
> +                               continue;
> +               }
> +
> +               /*
> +                * If it's in the process of being freed, gem_object->free()
> +                * may be blocked on lock waiting to remove it.  So just
> +                * skip it.
> +                */
> +               if (!kref_get_unless_zero(&obj->refcount))
> +                       continue;
> +
> +               mutex_unlock(&gem_shrinker->lock);
> +
> +               /* prevent racing with job-submission code paths */
> +               if (!dma_resv_trylock(obj->resv)) {
> +                       *lock_contention |= true;
> +                       goto shrinker_lock;
> +               }
> +
> +               /* prevent racing with the dma-buf importing/exporting */
> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> +                       *lock_contention |= true;
> +                       goto resv_unlock;
> +               }
> +
> +               /* check whether h/w uses this object */
> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> +                       goto object_name_unlock;
> +
> +               /* re-check whether eviction status hasn't changed */
> +               if (!drm_gem_shmem_is_evictable(shmem) &&
> +                   !drm_gem_shmem_is_purgeable(shmem))
> +                       goto object_name_unlock;
> +
> +               err = shmem->evict(shmem);
> +               if (!err)
> +                       freed += obj->size >> PAGE_SHIFT;
> +
> +object_name_unlock:
> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> +resv_unlock:
> +               dma_resv_unlock(obj->resv);
> +shrinker_lock:
> +               drm_gem_object_put(&shmem->base);
> +               mutex_lock(&gem_shrinker->lock);
> +       }
> +
> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       return freed;
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> +                                   struct shrink_control *sc)
> +{
> +       unsigned long nr_to_scan = sc->nr_to_scan;
> +       bool lock_contention = false;
> +       unsigned long freed;
> +
> +       /* purge as many objects as we can */
> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> +                                                       &lock_contention, false);
> +
> +       /* evict as many objects as we can */
> +       if (freed < nr_to_scan)
> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> +                                                                nr_to_scan - freed,
> +                                                                &lock_contention,
> +                                                                true);
> +
> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> +}
> +
> +/**
> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> + * @dev: DRM device
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker;
> +       int err;
> +
> +       if (WARN_ON(dev->shmem_shrinker))
> +               return -EBUSY;
> +
> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> +       if (!gem_shrinker)
> +               return -ENOMEM;
> +
> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> +       gem_shrinker->dev = dev;
> +
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> +       mutex_init(&gem_shrinker->lock);
> +
> +       dev->shmem_shrinker = gem_shrinker;
> +
> +       err = register_shrinker(&gem_shrinker->base);
> +       if (err) {
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +               return err;
> +       }
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> +
> +/**
> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> + * @dev: DRM device
> + */
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> +
> +       if (gem_shrinker) {
> +               unregister_shrinker(&gem_shrinker->base);
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> +               mutex_destroy(&gem_shrinker->lock);
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +       }
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> +
>  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>  MODULE_IMPORT_NS(DMA_BUF);
>  MODULE_LICENSE("GPL v2");
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> index a4bedfeb2ec4..7cc32556f908 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> @@ -15,6 +15,13 @@
>  #include "panfrost_gem.h"
>  #include "panfrost_mmu.h"
>
> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv > 0) &&
> +               !shmem->pages_pin_count && shmem->sgt &&
> +               !shmem->base.dma_buf && !shmem->base.import_attach;
> +}
> +
>  static unsigned long
>  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>  {
> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>                 return 0;
>
>         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -               if (drm_gem_shmem_is_purgeable(shmem))
> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>                         count += shmem->base.size >> PAGE_SHIFT;
>         }
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> index b2d93cb12ebf..81bacc7e1873 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>         uint32_t hw_res_handle;
>         bool dumb;
>         bool created;
> +       bool detached;
>         bool host3d_blob, guest_blob;
>         uint32_t blob_mem, blob_flags;
>
> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>
>  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>
> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> +
>  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>                                uint32_t *resid);
>  /* virtgpu_prime.c */
> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> index 9923c7a6885e..929546cad894 100644
> --- a/include/drm/drm_device.h
> +++ b/include/drm/drm_device.h
> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>  struct drm_vma_offset_manager;
>  struct drm_vram_mm;
>  struct drm_fb_helper;
> +struct drm_gem_shmem_shrinker;
>
>  struct inode;
>
> @@ -277,6 +278,9 @@ struct drm_device {
>         /** @vram_mm: VRAM MM memory manager */
>         struct drm_vram_mm *vram_mm;
>
> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> +
>         /**
>          * @switch_power_state:
>          *
> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> index 9a8983ee8abe..62c640678a91 100644
> --- a/include/drm/drm_gem_shmem_helper.h
> +++ b/include/drm/drm_gem_shmem_helper.h
> @@ -6,6 +6,7 @@
>  #include <linux/fs.h>
>  #include <linux/mm.h>
>  #include <linux/mutex.h>
> +#include <linux/shrinker.h>
>
>  #include <drm/drm_file.h>
>  #include <drm/drm_gem.h>
> @@ -15,6 +16,7 @@
>  struct dma_buf_attachment;
>  struct drm_mode_create_dumb;
>  struct drm_printer;
> +struct drm_device;
>  struct sg_table;
>
>  /**
> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>          */
>         unsigned int pages_use_count;
>
> +       /**
> +        * @pages_pin_count:
> +        *
> +        * Reference count on the pinned pages table.
> +        * The pages can be evicted by memory shrinker
> +        * when the count reaches zero.
> +        */
> +       unsigned int pages_pin_count;
> +
>         /**
>          * @madv: State for madvise
>          *
>          * 0 is active/inuse.
> +        * 1 is not-needed/can-be-purged
>          * A negative value is the object is purged.
> -        * Positive values are driver specific and not used by the helpers.
>          */
>         int madv;
>
> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>          * @map_wc: map object write-combined (instead of using shmem defaults).
>          */
>         bool map_wc;
> +
> +       /**
> +        * @eviction_enabled:
> +        *
> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool eviction_enabled;
> +
> +       /**
> +        * @purge_enabled:
> +        *
> +        * The shmem pages can be purged only if @purge_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool purge_enabled;
> +
> +       /**
> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> +        * Used internally by memory shrinker.
> +        */
> +       bool evicted;
> +
> +       /**
> +        * @evict:
> +        *
> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> +        * GEM's DMA reservation is kept locked by the shrinker. This is
> +        * optional callback that should be specified by drivers.
> +        *
> +        * Returns 0 on success, or -errno on error.
> +        */
> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>  };
>
>  #define to_drm_gem_shmem_obj(obj) \
> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>
>  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> +
>  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>  {
> -       return (shmem->madv > 0) &&
> -               !shmem->vmap_use_count && shmem->sgt &&
> -               !shmem->base.dma_buf && !shmem->base.import_attach;
> +       return (shmem->madv > 0) && shmem->evict &&
> +               shmem->purge_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>  }
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>
>  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>         return drm_gem_shmem_mmap(shmem, vma);
>  }
>
> +/**
> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> + */
> +struct drm_gem_shmem_shrinker {
> +       /** @base: Shrinker for purging shmem GEM objects */
> +       struct shrinker base;
> +
> +       /** @lock: Protects @lru_* */
> +       struct mutex lock;
> +
> +       /** @lru_pinned: List of pinned shmem GEM objects */
> +       struct list_head lru_pinned;
> +
> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> +       struct list_head lru_evictable;
> +
> +       /** @lru_evicted: List of evicted shmem GEM objects */
> +       struct list_head lru_evicted;
> +
> +       /** @dev: DRM device that uses this shrinker */
> +       struct drm_device *dev;
> +};
> +
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> +
>  /*
>   * Driver ops
>   */
> --
> 2.35.3
>


-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> Introduce a common DRM SHMEM shrinker framework that allows to reduce
> code duplication among DRM drivers by replacing theirs custom shrinker
> implementations with the generic shrinker.
>
> In order to start using DRM SHMEM shrinker drivers should:
>
> 1. Implement new evict() shmem object callback.
> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>    activate shrinking of shmem GEMs.
>
> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>
> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>

So I guess I get a price for being blind since forever, because this
thing existed since at least 2013. I just stumbled over
llist_lru.[hc], a purpose built list helper for shrinkers. I think we
should try to adopt that so that our gpu shrinkers look more like
shrinkers for everything else.

Apologies for this, since I fear this might cause a bit of churn.
Hopefully it's all contained to the list manipulation code in shmem
helpers, I don't think this should leak any further.
-Daniel

> ---
>  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>  include/drm/drm_device.h                      |   4 +
>  include/drm/drm_gem_shmem_helper.h            |  87 ++-
>  5 files changed, 594 insertions(+), 49 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 555fe212bd98..4cd0b5913492 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>
> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv >= 0) && shmem->evict &&
> +               shmem->eviction_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> +}
> +
> +static void
> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (!gem_shrinker || obj->import_attach)
> +               return;
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       if (drm_gem_shmem_is_evictable(shmem) ||
> +           drm_gem_shmem_is_purgeable(shmem))
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> +       else if (shmem->madv < 0)
> +               list_del_init(&shmem->madv_list);
> +       else if (shmem->evicted)
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> +       else if (!shmem->pages)
> +               list_del_init(&shmem->madv_list);
> +       else
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +}
> +
>  /**
>   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>   * @shmem: shmem GEM object to free
> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>         } else {
>                 dma_resv_lock(shmem->base.resv, NULL);
>
> +               /* take out shmem GEM object from the memory shrinker */
> +               drm_gem_shmem_madvise(shmem, -1);
> +
>                 WARN_ON(shmem->vmap_use_count);
>
>                 if (shmem->sgt) {
> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>                         sg_free_table(shmem->sgt);
>                         kfree(shmem->sgt);
>                 }
> -               if (shmem->pages)
> +               if (shmem->pages_use_count)
>                         drm_gem_shmem_put_pages(shmem);
>
>                 WARN_ON(shmem->pages_use_count);
> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>
> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->eviction_enabled = true;
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> +
> +/**
> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be purged. Initially purging is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->purge_enabled = true;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> +
> +static int
> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
>         struct page **pages;
>
> -       if (shmem->pages_use_count++ > 0)
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0) {
> +               WARN_ON(shmem->pages);
> +               return -ENOMEM;
> +       }
> +
> +       if (shmem->pages) {
> +               WARN_ON(!shmem->evicted);
>                 return 0;
> +       }
> +
> +       if (WARN_ON(!shmem->pages_use_count))
> +               return -EINVAL;
>
>         pages = drm_gem_get_pages(obj);
>         if (IS_ERR(pages)) {
>                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> -               shmem->pages_use_count = 0;
>                 return PTR_ERR(pages);
>         }
>
> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>         return 0;
>  }
>
> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       int err;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       if (shmem->pages_use_count++ > 0) {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err)
> +                       goto err_zero_use;
> +
> +               return 0;
> +       }
> +
> +       err = drm_gem_shmem_acquire_pages(shmem);
> +       if (err)
> +               goto err_zero_use;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +
> +err_zero_use:
> +       shmem->pages_use_count = 0;
> +
> +       return err;
> +}
> +
> +static void
> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       if (!shmem->pages) {
> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> +               return;
> +       }
> +
> +#ifdef CONFIG_X86
> +       if (shmem->map_wc)
> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> +#endif
> +
> +       drm_gem_put_pages(obj, shmem->pages,
> +                         shmem->pages_mark_dirty_on_put,
> +                         shmem->pages_mark_accessed_on_put);
> +       shmem->pages = NULL;
> +}
> +
>  /*
>   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>   * @shmem: shmem GEM object
> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>   */
>  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>  {
> -       struct drm_gem_object *obj = &shmem->base;
> -
>         dma_resv_assert_held(shmem->base.resv);
>
>         if (WARN_ON_ONCE(!shmem->pages_use_count))
> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>         if (--shmem->pages_use_count > 0)
>                 return;
>
> -#ifdef CONFIG_X86
> -       if (shmem->map_wc)
> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> -#endif
> +       drm_gem_shmem_release_pages(shmem);
>
> -       drm_gem_put_pages(obj, shmem->pages,
> -                         shmem->pages_mark_dirty_on_put,
> -                         shmem->pages_mark_accessed_on_put);
> -       shmem->pages = NULL;
> +       drm_gem_shmem_update_pages_state(shmem);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>
> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>   */
>  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>  {
> +       int ret;
> +
>         dma_resv_assert_held(shmem->base.resv);
>
>         WARN_ON(shmem->base.import_attach);
>
> -       return drm_gem_shmem_get_pages(shmem);
> +       ret = drm_gem_shmem_get_pages(shmem);
> +       if (!ret)
> +               shmem->pages_pin_count++;
> +
> +       return ret;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_pin);
>
> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>         WARN_ON(shmem->base.import_attach);
>
>         drm_gem_shmem_put_pages(shmem);
> +
> +       shmem->pages_pin_count--;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_unpin);
>
> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>                         return 0;
>                 }
>
> -               ret = drm_gem_shmem_get_pages(shmem);
> +               ret = drm_gem_shmem_pin(shmem);
>                 if (ret)
>                         goto err_zero_use;
>
> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>
>  err_put_pages:
>         if (!obj->import_attach)
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>  err_zero_use:
>         shmem->vmap_use_count = 0;
>
> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>                         return;
>
>                 vunmap(shmem->vaddr);
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>         }
>
>         shmem->vaddr = NULL;
> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>
>         madv = shmem->madv;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         return (madv >= 0);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_madvise);
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> + *                           hardware access to the memory.
> + * @shmem: shmem GEM object
> + *
> + * This function moves shmem GEM back to memory if it was previously evicted
> + * by the memory shrinker. The GEM is ready to use on success.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
> -       struct drm_device *dev = obj->dev;
> +       struct sg_table *sgt;
> +       int err;
>
>         dma_resv_assert_held(shmem->base.resv);
>
> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +       if (shmem->evicted) {
> +               err = drm_gem_shmem_acquire_pages(shmem);
> +               if (err)
> +                       return err;
> +
> +               sgt = drm_gem_shmem_get_sg_table(shmem);
> +               if (IS_ERR(sgt))
> +                       return PTR_ERR(sgt);
> +
> +               err = dma_map_sgtable(obj->dev->dev, sgt,
> +                                     DMA_BIDIRECTIONAL, 0);
> +               if (err) {
> +                       sg_free_table(sgt);
> +                       kfree(sgt);
> +                       return err;
> +               }
>
> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> -       sg_free_table(shmem->sgt);
> -       kfree(shmem->sgt);
> -       shmem->sgt = NULL;
> +               shmem->sgt = sgt;
> +               shmem->evicted = false;
>
> -       drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_update_pages_state(shmem);
> +       }
>
> -       shmem->madv = -1;
> +       if (!shmem->pages)
> +               return -ENOMEM;
>
> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> -       drm_gem_free_mmap_offset(obj);
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>
> -       /* Our goal here is to return as much of the memory as
> -        * is possible back to the system as we are called from OOM.
> -        * To do this we must instruct the shmfs to drop all of its
> -        * backing pages, *now*.
> -        */
> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_device *dev = obj->dev;
>
> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +       if (shmem->evicted)
> +               return;
> +
> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> +       drm_gem_shmem_release_pages(shmem);
> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> +
> +       sg_free_table(shmem->sgt);
> +       kfree(shmem->sgt);
> +       shmem->sgt = NULL;
>  }
> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>
>  /**
>   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>         vm_fault_t ret;
>         struct page *page;
>         pgoff_t page_offset;
> +       bool pages_unpinned;
> +       int err;
>
>         /* We don't use vmf->pgoff since that has the fake offset */
>         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>
>         dma_resv_lock(shmem->base.resv, NULL);
>
> -       if (page_offset >= num_pages ||
> -           WARN_ON_ONCE(!shmem->pages) ||
> -           shmem->madv < 0) {
> +       /* Sanity-check that we have the pages pointer when it should present */
> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> +
> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>                 ret = VM_FAULT_SIGBUS;
>         } else {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err) {
> +                       ret = VM_FAULT_OOM;
> +                       goto unlock;
> +               }
> +
>                 page = shmem->pages[page_offset];
>
>                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>         }
>
> +unlock:
>         dma_resv_unlock(shmem->base.resv);
>
>         return ret;
> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>  {
>         struct drm_gem_object *obj = vma->vm_private_data;
>         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -       int ret;
>
>         WARN_ON(shmem->base.import_attach);
>
>         dma_resv_lock(shmem->base.resv, NULL);
> -       ret = drm_gem_shmem_get_pages(shmem);
> -       WARN_ON_ONCE(ret != 0);
> +
> +       if (drm_gem_shmem_get_pages(shmem))
> +               shmem->pages_use_count++;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
>         dma_resv_unlock(shmem->base.resv);
>
>         drm_gem_vm_open(vma);
> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                               struct drm_printer *p, unsigned int indent)
>  {
> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>
>         if (shmem->base.import_attach)
> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>                                   shmem->vmap_use_count);
>
> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_print_info);
>
> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>
>         shmem->sgt = sgt;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         dma_resv_unlock(shmem->base.resv);
>
>         return sgt;
> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>
> +static struct drm_gem_shmem_shrinker *
> +to_drm_shrinker(struct shrinker *shrinker)
> +{
> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> +                                    struct shrink_control *sc)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       unsigned long count = 0;
> +
> +       if (!mutex_trylock(&gem_shrinker->lock))
> +               return 0;
> +
> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> +               count += shmem->base.size;
> +
> +               if (count >= SHRINK_EMPTY)
> +                       break;
> +       }
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       if (count >= SHRINK_EMPTY)
> +               return SHRINK_EMPTY - 1;
> +
> +       return count ?: SHRINK_EMPTY;
> +}
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> +       WARN_ON(shmem->evicted);
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +
> +       shmem->evicted = true;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> +
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +       drm_gem_free_mmap_offset(obj);
> +
> +       /* Our goal here is to return as much of the memory as
> +        * is possible back to the system as we are called from OOM.
> +        * To do this we must instruct the shmfs to drop all of its
> +        * backing pages, *now*.
> +        */
> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +
> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +
> +       shmem->madv = -1;
> +       shmem->evicted = false;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> +                                       unsigned long nr_to_scan,
> +                                       bool *lock_contention,
> +                                       bool evict)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       struct list_head still_in_list;
> +       struct drm_gem_object *obj;
> +       unsigned long freed = 0;
> +       size_t page_count;
> +       int err;
> +
> +       INIT_LIST_HEAD(&still_in_list);
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       while (freed < nr_to_scan) {
> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> +                                                typeof(*shmem), madv_list);
> +               if (!shmem)
> +                       break;
> +
> +               obj = &shmem->base;
> +               page_count = obj->size >> PAGE_SHIFT;
> +               list_move_tail(&shmem->madv_list, &still_in_list);
> +
> +               if (evict) {
> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> +                           get_nr_swap_pages() < page_count)
> +                               continue;
> +               } else {
> +                       if (!drm_gem_shmem_is_purgeable(shmem))
> +                               continue;
> +               }
> +
> +               /*
> +                * If it's in the process of being freed, gem_object->free()
> +                * may be blocked on lock waiting to remove it.  So just
> +                * skip it.
> +                */
> +               if (!kref_get_unless_zero(&obj->refcount))
> +                       continue;
> +
> +               mutex_unlock(&gem_shrinker->lock);
> +
> +               /* prevent racing with job-submission code paths */
> +               if (!dma_resv_trylock(obj->resv)) {
> +                       *lock_contention |= true;
> +                       goto shrinker_lock;
> +               }
> +
> +               /* prevent racing with the dma-buf importing/exporting */
> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> +                       *lock_contention |= true;
> +                       goto resv_unlock;
> +               }
> +
> +               /* check whether h/w uses this object */
> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> +                       goto object_name_unlock;
> +
> +               /* re-check whether eviction status hasn't changed */
> +               if (!drm_gem_shmem_is_evictable(shmem) &&
> +                   !drm_gem_shmem_is_purgeable(shmem))
> +                       goto object_name_unlock;
> +
> +               err = shmem->evict(shmem);
> +               if (!err)
> +                       freed += obj->size >> PAGE_SHIFT;
> +
> +object_name_unlock:
> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> +resv_unlock:
> +               dma_resv_unlock(obj->resv);
> +shrinker_lock:
> +               drm_gem_object_put(&shmem->base);
> +               mutex_lock(&gem_shrinker->lock);
> +       }
> +
> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       return freed;
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> +                                   struct shrink_control *sc)
> +{
> +       unsigned long nr_to_scan = sc->nr_to_scan;
> +       bool lock_contention = false;
> +       unsigned long freed;
> +
> +       /* purge as many objects as we can */
> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> +                                                       &lock_contention, false);
> +
> +       /* evict as many objects as we can */
> +       if (freed < nr_to_scan)
> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> +                                                                nr_to_scan - freed,
> +                                                                &lock_contention,
> +                                                                true);
> +
> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> +}
> +
> +/**
> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> + * @dev: DRM device
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker;
> +       int err;
> +
> +       if (WARN_ON(dev->shmem_shrinker))
> +               return -EBUSY;
> +
> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> +       if (!gem_shrinker)
> +               return -ENOMEM;
> +
> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> +       gem_shrinker->dev = dev;
> +
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> +       mutex_init(&gem_shrinker->lock);
> +
> +       dev->shmem_shrinker = gem_shrinker;
> +
> +       err = register_shrinker(&gem_shrinker->base);
> +       if (err) {
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +               return err;
> +       }
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> +
> +/**
> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> + * @dev: DRM device
> + */
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> +
> +       if (gem_shrinker) {
> +               unregister_shrinker(&gem_shrinker->base);
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> +               mutex_destroy(&gem_shrinker->lock);
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +       }
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> +
>  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>  MODULE_IMPORT_NS(DMA_BUF);
>  MODULE_LICENSE("GPL v2");
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> index a4bedfeb2ec4..7cc32556f908 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> @@ -15,6 +15,13 @@
>  #include "panfrost_gem.h"
>  #include "panfrost_mmu.h"
>
> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv > 0) &&
> +               !shmem->pages_pin_count && shmem->sgt &&
> +               !shmem->base.dma_buf && !shmem->base.import_attach;
> +}
> +
>  static unsigned long
>  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>  {
> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>                 return 0;
>
>         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -               if (drm_gem_shmem_is_purgeable(shmem))
> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>                         count += shmem->base.size >> PAGE_SHIFT;
>         }
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> index b2d93cb12ebf..81bacc7e1873 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>         uint32_t hw_res_handle;
>         bool dumb;
>         bool created;
> +       bool detached;
>         bool host3d_blob, guest_blob;
>         uint32_t blob_mem, blob_flags;
>
> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>
>  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>
> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> +
>  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>                                uint32_t *resid);
>  /* virtgpu_prime.c */
> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> index 9923c7a6885e..929546cad894 100644
> --- a/include/drm/drm_device.h
> +++ b/include/drm/drm_device.h
> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>  struct drm_vma_offset_manager;
>  struct drm_vram_mm;
>  struct drm_fb_helper;
> +struct drm_gem_shmem_shrinker;
>
>  struct inode;
>
> @@ -277,6 +278,9 @@ struct drm_device {
>         /** @vram_mm: VRAM MM memory manager */
>         struct drm_vram_mm *vram_mm;
>
> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> +
>         /**
>          * @switch_power_state:
>          *
> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> index 9a8983ee8abe..62c640678a91 100644
> --- a/include/drm/drm_gem_shmem_helper.h
> +++ b/include/drm/drm_gem_shmem_helper.h
> @@ -6,6 +6,7 @@
>  #include <linux/fs.h>
>  #include <linux/mm.h>
>  #include <linux/mutex.h>
> +#include <linux/shrinker.h>
>
>  #include <drm/drm_file.h>
>  #include <drm/drm_gem.h>
> @@ -15,6 +16,7 @@
>  struct dma_buf_attachment;
>  struct drm_mode_create_dumb;
>  struct drm_printer;
> +struct drm_device;
>  struct sg_table;
>
>  /**
> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>          */
>         unsigned int pages_use_count;
>
> +       /**
> +        * @pages_pin_count:
> +        *
> +        * Reference count on the pinned pages table.
> +        * The pages can be evicted by memory shrinker
> +        * when the count reaches zero.
> +        */
> +       unsigned int pages_pin_count;
> +
>         /**
>          * @madv: State for madvise
>          *
>          * 0 is active/inuse.
> +        * 1 is not-needed/can-be-purged
>          * A negative value is the object is purged.
> -        * Positive values are driver specific and not used by the helpers.
>          */
>         int madv;
>
> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>          * @map_wc: map object write-combined (instead of using shmem defaults).
>          */
>         bool map_wc;
> +
> +       /**
> +        * @eviction_enabled:
> +        *
> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool eviction_enabled;
> +
> +       /**
> +        * @purge_enabled:
> +        *
> +        * The shmem pages can be purged only if @purge_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool purge_enabled;
> +
> +       /**
> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> +        * Used internally by memory shrinker.
> +        */
> +       bool evicted;
> +
> +       /**
> +        * @evict:
> +        *
> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> +        * GEM's DMA reservation is kept locked by the shrinker. This is
> +        * optional callback that should be specified by drivers.
> +        *
> +        * Returns 0 on success, or -errno on error.
> +        */
> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>  };
>
>  #define to_drm_gem_shmem_obj(obj) \
> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>
>  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> +
>  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>  {
> -       return (shmem->madv > 0) &&
> -               !shmem->vmap_use_count && shmem->sgt &&
> -               !shmem->base.dma_buf && !shmem->base.import_attach;
> +       return (shmem->madv > 0) && shmem->evict &&
> +               shmem->purge_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>  }
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>
>  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>         return drm_gem_shmem_mmap(shmem, vma);
>  }
>
> +/**
> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> + */
> +struct drm_gem_shmem_shrinker {
> +       /** @base: Shrinker for purging shmem GEM objects */
> +       struct shrinker base;
> +
> +       /** @lock: Protects @lru_* */
> +       struct mutex lock;
> +
> +       /** @lru_pinned: List of pinned shmem GEM objects */
> +       struct list_head lru_pinned;
> +
> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> +       struct list_head lru_evictable;
> +
> +       /** @lru_evicted: List of evicted shmem GEM objects */
> +       struct list_head lru_evicted;
> +
> +       /** @dev: DRM device that uses this shrinker */
> +       struct drm_device *dev;
> +};
> +
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> +
>  /*
>   * Driver ops
>   */
> --
> 2.35.3
>


-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [Intel-gfx] [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> Introduce a common DRM SHMEM shrinker framework that allows to reduce
> code duplication among DRM drivers by replacing theirs custom shrinker
> implementations with the generic shrinker.
>
> In order to start using DRM SHMEM shrinker drivers should:
>
> 1. Implement new evict() shmem object callback.
> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>    activate shrinking of shmem GEMs.
>
> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>
> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>

So I guess I get a price for being blind since forever, because this
thing existed since at least 2013. I just stumbled over
llist_lru.[hc], a purpose built list helper for shrinkers. I think we
should try to adopt that so that our gpu shrinkers look more like
shrinkers for everything else.

Apologies for this, since I fear this might cause a bit of churn.
Hopefully it's all contained to the list manipulation code in shmem
helpers, I don't think this should leak any further.
-Daniel

> ---
>  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>  include/drm/drm_device.h                      |   4 +
>  include/drm/drm_gem_shmem_helper.h            |  87 ++-
>  5 files changed, 594 insertions(+), 49 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 555fe212bd98..4cd0b5913492 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>
> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv >= 0) && shmem->evict &&
> +               shmem->eviction_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> +}
> +
> +static void
> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (!gem_shrinker || obj->import_attach)
> +               return;
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       if (drm_gem_shmem_is_evictable(shmem) ||
> +           drm_gem_shmem_is_purgeable(shmem))
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> +       else if (shmem->madv < 0)
> +               list_del_init(&shmem->madv_list);
> +       else if (shmem->evicted)
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> +       else if (!shmem->pages)
> +               list_del_init(&shmem->madv_list);
> +       else
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +}
> +
>  /**
>   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>   * @shmem: shmem GEM object to free
> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>         } else {
>                 dma_resv_lock(shmem->base.resv, NULL);
>
> +               /* take out shmem GEM object from the memory shrinker */
> +               drm_gem_shmem_madvise(shmem, -1);
> +
>                 WARN_ON(shmem->vmap_use_count);
>
>                 if (shmem->sgt) {
> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>                         sg_free_table(shmem->sgt);
>                         kfree(shmem->sgt);
>                 }
> -               if (shmem->pages)
> +               if (shmem->pages_use_count)
>                         drm_gem_shmem_put_pages(shmem);
>
>                 WARN_ON(shmem->pages_use_count);
> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>
> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->eviction_enabled = true;
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> +
> +/**
> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be purged. Initially purging is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->purge_enabled = true;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> +
> +static int
> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
>         struct page **pages;
>
> -       if (shmem->pages_use_count++ > 0)
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0) {
> +               WARN_ON(shmem->pages);
> +               return -ENOMEM;
> +       }
> +
> +       if (shmem->pages) {
> +               WARN_ON(!shmem->evicted);
>                 return 0;
> +       }
> +
> +       if (WARN_ON(!shmem->pages_use_count))
> +               return -EINVAL;
>
>         pages = drm_gem_get_pages(obj);
>         if (IS_ERR(pages)) {
>                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> -               shmem->pages_use_count = 0;
>                 return PTR_ERR(pages);
>         }
>
> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>         return 0;
>  }
>
> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       int err;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       if (shmem->pages_use_count++ > 0) {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err)
> +                       goto err_zero_use;
> +
> +               return 0;
> +       }
> +
> +       err = drm_gem_shmem_acquire_pages(shmem);
> +       if (err)
> +               goto err_zero_use;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +
> +err_zero_use:
> +       shmem->pages_use_count = 0;
> +
> +       return err;
> +}
> +
> +static void
> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       if (!shmem->pages) {
> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> +               return;
> +       }
> +
> +#ifdef CONFIG_X86
> +       if (shmem->map_wc)
> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> +#endif
> +
> +       drm_gem_put_pages(obj, shmem->pages,
> +                         shmem->pages_mark_dirty_on_put,
> +                         shmem->pages_mark_accessed_on_put);
> +       shmem->pages = NULL;
> +}
> +
>  /*
>   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>   * @shmem: shmem GEM object
> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>   */
>  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>  {
> -       struct drm_gem_object *obj = &shmem->base;
> -
>         dma_resv_assert_held(shmem->base.resv);
>
>         if (WARN_ON_ONCE(!shmem->pages_use_count))
> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>         if (--shmem->pages_use_count > 0)
>                 return;
>
> -#ifdef CONFIG_X86
> -       if (shmem->map_wc)
> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> -#endif
> +       drm_gem_shmem_release_pages(shmem);
>
> -       drm_gem_put_pages(obj, shmem->pages,
> -                         shmem->pages_mark_dirty_on_put,
> -                         shmem->pages_mark_accessed_on_put);
> -       shmem->pages = NULL;
> +       drm_gem_shmem_update_pages_state(shmem);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>
> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>   */
>  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>  {
> +       int ret;
> +
>         dma_resv_assert_held(shmem->base.resv);
>
>         WARN_ON(shmem->base.import_attach);
>
> -       return drm_gem_shmem_get_pages(shmem);
> +       ret = drm_gem_shmem_get_pages(shmem);
> +       if (!ret)
> +               shmem->pages_pin_count++;
> +
> +       return ret;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_pin);
>
> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>         WARN_ON(shmem->base.import_attach);
>
>         drm_gem_shmem_put_pages(shmem);
> +
> +       shmem->pages_pin_count--;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_unpin);
>
> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>                         return 0;
>                 }
>
> -               ret = drm_gem_shmem_get_pages(shmem);
> +               ret = drm_gem_shmem_pin(shmem);
>                 if (ret)
>                         goto err_zero_use;
>
> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>
>  err_put_pages:
>         if (!obj->import_attach)
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>  err_zero_use:
>         shmem->vmap_use_count = 0;
>
> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>                         return;
>
>                 vunmap(shmem->vaddr);
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>         }
>
>         shmem->vaddr = NULL;
> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>
>         madv = shmem->madv;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         return (madv >= 0);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_madvise);
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> + *                           hardware access to the memory.
> + * @shmem: shmem GEM object
> + *
> + * This function moves shmem GEM back to memory if it was previously evicted
> + * by the memory shrinker. The GEM is ready to use on success.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
> -       struct drm_device *dev = obj->dev;
> +       struct sg_table *sgt;
> +       int err;
>
>         dma_resv_assert_held(shmem->base.resv);
>
> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +       if (shmem->evicted) {
> +               err = drm_gem_shmem_acquire_pages(shmem);
> +               if (err)
> +                       return err;
> +
> +               sgt = drm_gem_shmem_get_sg_table(shmem);
> +               if (IS_ERR(sgt))
> +                       return PTR_ERR(sgt);
> +
> +               err = dma_map_sgtable(obj->dev->dev, sgt,
> +                                     DMA_BIDIRECTIONAL, 0);
> +               if (err) {
> +                       sg_free_table(sgt);
> +                       kfree(sgt);
> +                       return err;
> +               }
>
> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> -       sg_free_table(shmem->sgt);
> -       kfree(shmem->sgt);
> -       shmem->sgt = NULL;
> +               shmem->sgt = sgt;
> +               shmem->evicted = false;
>
> -       drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_update_pages_state(shmem);
> +       }
>
> -       shmem->madv = -1;
> +       if (!shmem->pages)
> +               return -ENOMEM;
>
> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> -       drm_gem_free_mmap_offset(obj);
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>
> -       /* Our goal here is to return as much of the memory as
> -        * is possible back to the system as we are called from OOM.
> -        * To do this we must instruct the shmfs to drop all of its
> -        * backing pages, *now*.
> -        */
> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_device *dev = obj->dev;
>
> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +       if (shmem->evicted)
> +               return;
> +
> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> +       drm_gem_shmem_release_pages(shmem);
> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> +
> +       sg_free_table(shmem->sgt);
> +       kfree(shmem->sgt);
> +       shmem->sgt = NULL;
>  }
> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>
>  /**
>   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>         vm_fault_t ret;
>         struct page *page;
>         pgoff_t page_offset;
> +       bool pages_unpinned;
> +       int err;
>
>         /* We don't use vmf->pgoff since that has the fake offset */
>         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>
>         dma_resv_lock(shmem->base.resv, NULL);
>
> -       if (page_offset >= num_pages ||
> -           WARN_ON_ONCE(!shmem->pages) ||
> -           shmem->madv < 0) {
> +       /* Sanity-check that we have the pages pointer when it should present */
> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> +
> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>                 ret = VM_FAULT_SIGBUS;
>         } else {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err) {
> +                       ret = VM_FAULT_OOM;
> +                       goto unlock;
> +               }
> +
>                 page = shmem->pages[page_offset];
>
>                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>         }
>
> +unlock:
>         dma_resv_unlock(shmem->base.resv);
>
>         return ret;
> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>  {
>         struct drm_gem_object *obj = vma->vm_private_data;
>         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -       int ret;
>
>         WARN_ON(shmem->base.import_attach);
>
>         dma_resv_lock(shmem->base.resv, NULL);
> -       ret = drm_gem_shmem_get_pages(shmem);
> -       WARN_ON_ONCE(ret != 0);
> +
> +       if (drm_gem_shmem_get_pages(shmem))
> +               shmem->pages_use_count++;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
>         dma_resv_unlock(shmem->base.resv);
>
>         drm_gem_vm_open(vma);
> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                               struct drm_printer *p, unsigned int indent)
>  {
> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>
>         if (shmem->base.import_attach)
> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>                                   shmem->vmap_use_count);
>
> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_print_info);
>
> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>
>         shmem->sgt = sgt;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         dma_resv_unlock(shmem->base.resv);
>
>         return sgt;
> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>
> +static struct drm_gem_shmem_shrinker *
> +to_drm_shrinker(struct shrinker *shrinker)
> +{
> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> +                                    struct shrink_control *sc)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       unsigned long count = 0;
> +
> +       if (!mutex_trylock(&gem_shrinker->lock))
> +               return 0;
> +
> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> +               count += shmem->base.size;
> +
> +               if (count >= SHRINK_EMPTY)
> +                       break;
> +       }
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       if (count >= SHRINK_EMPTY)
> +               return SHRINK_EMPTY - 1;
> +
> +       return count ?: SHRINK_EMPTY;
> +}
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> +       WARN_ON(shmem->evicted);
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +
> +       shmem->evicted = true;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> +
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +       drm_gem_free_mmap_offset(obj);
> +
> +       /* Our goal here is to return as much of the memory as
> +        * is possible back to the system as we are called from OOM.
> +        * To do this we must instruct the shmfs to drop all of its
> +        * backing pages, *now*.
> +        */
> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +
> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +
> +       shmem->madv = -1;
> +       shmem->evicted = false;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> +                                       unsigned long nr_to_scan,
> +                                       bool *lock_contention,
> +                                       bool evict)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       struct list_head still_in_list;
> +       struct drm_gem_object *obj;
> +       unsigned long freed = 0;
> +       size_t page_count;
> +       int err;
> +
> +       INIT_LIST_HEAD(&still_in_list);
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       while (freed < nr_to_scan) {
> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> +                                                typeof(*shmem), madv_list);
> +               if (!shmem)
> +                       break;
> +
> +               obj = &shmem->base;
> +               page_count = obj->size >> PAGE_SHIFT;
> +               list_move_tail(&shmem->madv_list, &still_in_list);
> +
> +               if (evict) {
> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> +                           get_nr_swap_pages() < page_count)
> +                               continue;
> +               } else {
> +                       if (!drm_gem_shmem_is_purgeable(shmem))
> +                               continue;
> +               }
> +
> +               /*
> +                * If it's in the process of being freed, gem_object->free()
> +                * may be blocked on lock waiting to remove it.  So just
> +                * skip it.
> +                */
> +               if (!kref_get_unless_zero(&obj->refcount))
> +                       continue;
> +
> +               mutex_unlock(&gem_shrinker->lock);
> +
> +               /* prevent racing with job-submission code paths */
> +               if (!dma_resv_trylock(obj->resv)) {
> +                       *lock_contention |= true;
> +                       goto shrinker_lock;
> +               }
> +
> +               /* prevent racing with the dma-buf importing/exporting */
> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> +                       *lock_contention |= true;
> +                       goto resv_unlock;
> +               }
> +
> +               /* check whether h/w uses this object */
> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> +                       goto object_name_unlock;
> +
> +               /* re-check whether eviction status hasn't changed */
> +               if (!drm_gem_shmem_is_evictable(shmem) &&
> +                   !drm_gem_shmem_is_purgeable(shmem))
> +                       goto object_name_unlock;
> +
> +               err = shmem->evict(shmem);
> +               if (!err)
> +                       freed += obj->size >> PAGE_SHIFT;
> +
> +object_name_unlock:
> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> +resv_unlock:
> +               dma_resv_unlock(obj->resv);
> +shrinker_lock:
> +               drm_gem_object_put(&shmem->base);
> +               mutex_lock(&gem_shrinker->lock);
> +       }
> +
> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       return freed;
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> +                                   struct shrink_control *sc)
> +{
> +       unsigned long nr_to_scan = sc->nr_to_scan;
> +       bool lock_contention = false;
> +       unsigned long freed;
> +
> +       /* purge as many objects as we can */
> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> +                                                       &lock_contention, false);
> +
> +       /* evict as many objects as we can */
> +       if (freed < nr_to_scan)
> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> +                                                                nr_to_scan - freed,
> +                                                                &lock_contention,
> +                                                                true);
> +
> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> +}
> +
> +/**
> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> + * @dev: DRM device
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker;
> +       int err;
> +
> +       if (WARN_ON(dev->shmem_shrinker))
> +               return -EBUSY;
> +
> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> +       if (!gem_shrinker)
> +               return -ENOMEM;
> +
> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> +       gem_shrinker->dev = dev;
> +
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> +       mutex_init(&gem_shrinker->lock);
> +
> +       dev->shmem_shrinker = gem_shrinker;
> +
> +       err = register_shrinker(&gem_shrinker->base);
> +       if (err) {
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +               return err;
> +       }
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> +
> +/**
> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> + * @dev: DRM device
> + */
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> +
> +       if (gem_shrinker) {
> +               unregister_shrinker(&gem_shrinker->base);
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> +               mutex_destroy(&gem_shrinker->lock);
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +       }
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> +
>  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>  MODULE_IMPORT_NS(DMA_BUF);
>  MODULE_LICENSE("GPL v2");
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> index a4bedfeb2ec4..7cc32556f908 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> @@ -15,6 +15,13 @@
>  #include "panfrost_gem.h"
>  #include "panfrost_mmu.h"
>
> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv > 0) &&
> +               !shmem->pages_pin_count && shmem->sgt &&
> +               !shmem->base.dma_buf && !shmem->base.import_attach;
> +}
> +
>  static unsigned long
>  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>  {
> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>                 return 0;
>
>         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -               if (drm_gem_shmem_is_purgeable(shmem))
> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>                         count += shmem->base.size >> PAGE_SHIFT;
>         }
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> index b2d93cb12ebf..81bacc7e1873 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>         uint32_t hw_res_handle;
>         bool dumb;
>         bool created;
> +       bool detached;
>         bool host3d_blob, guest_blob;
>         uint32_t blob_mem, blob_flags;
>
> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>
>  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>
> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> +
>  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>                                uint32_t *resid);
>  /* virtgpu_prime.c */
> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> index 9923c7a6885e..929546cad894 100644
> --- a/include/drm/drm_device.h
> +++ b/include/drm/drm_device.h
> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>  struct drm_vma_offset_manager;
>  struct drm_vram_mm;
>  struct drm_fb_helper;
> +struct drm_gem_shmem_shrinker;
>
>  struct inode;
>
> @@ -277,6 +278,9 @@ struct drm_device {
>         /** @vram_mm: VRAM MM memory manager */
>         struct drm_vram_mm *vram_mm;
>
> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> +
>         /**
>          * @switch_power_state:
>          *
> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> index 9a8983ee8abe..62c640678a91 100644
> --- a/include/drm/drm_gem_shmem_helper.h
> +++ b/include/drm/drm_gem_shmem_helper.h
> @@ -6,6 +6,7 @@
>  #include <linux/fs.h>
>  #include <linux/mm.h>
>  #include <linux/mutex.h>
> +#include <linux/shrinker.h>
>
>  #include <drm/drm_file.h>
>  #include <drm/drm_gem.h>
> @@ -15,6 +16,7 @@
>  struct dma_buf_attachment;
>  struct drm_mode_create_dumb;
>  struct drm_printer;
> +struct drm_device;
>  struct sg_table;
>
>  /**
> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>          */
>         unsigned int pages_use_count;
>
> +       /**
> +        * @pages_pin_count:
> +        *
> +        * Reference count on the pinned pages table.
> +        * The pages can be evicted by memory shrinker
> +        * when the count reaches zero.
> +        */
> +       unsigned int pages_pin_count;
> +
>         /**
>          * @madv: State for madvise
>          *
>          * 0 is active/inuse.
> +        * 1 is not-needed/can-be-purged
>          * A negative value is the object is purged.
> -        * Positive values are driver specific and not used by the helpers.
>          */
>         int madv;
>
> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>          * @map_wc: map object write-combined (instead of using shmem defaults).
>          */
>         bool map_wc;
> +
> +       /**
> +        * @eviction_enabled:
> +        *
> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool eviction_enabled;
> +
> +       /**
> +        * @purge_enabled:
> +        *
> +        * The shmem pages can be purged only if @purge_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool purge_enabled;
> +
> +       /**
> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> +        * Used internally by memory shrinker.
> +        */
> +       bool evicted;
> +
> +       /**
> +        * @evict:
> +        *
> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> +        * GEM's DMA reservation is kept locked by the shrinker. This is
> +        * optional callback that should be specified by drivers.
> +        *
> +        * Returns 0 on success, or -errno on error.
> +        */
> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>  };
>
>  #define to_drm_gem_shmem_obj(obj) \
> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>
>  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> +
>  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>  {
> -       return (shmem->madv > 0) &&
> -               !shmem->vmap_use_count && shmem->sgt &&
> -               !shmem->base.dma_buf && !shmem->base.import_attach;
> +       return (shmem->madv > 0) && shmem->evict &&
> +               shmem->purge_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>  }
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>
>  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>         return drm_gem_shmem_mmap(shmem, vma);
>  }
>
> +/**
> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> + */
> +struct drm_gem_shmem_shrinker {
> +       /** @base: Shrinker for purging shmem GEM objects */
> +       struct shrinker base;
> +
> +       /** @lock: Protects @lru_* */
> +       struct mutex lock;
> +
> +       /** @lru_pinned: List of pinned shmem GEM objects */
> +       struct list_head lru_pinned;
> +
> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> +       struct list_head lru_evictable;
> +
> +       /** @lru_evicted: List of evicted shmem GEM objects */
> +       struct list_head lru_evicted;
> +
> +       /** @dev: DRM device that uses this shrinker */
> +       struct drm_device *dev;
> +};
> +
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> +
>  /*
>   * Driver ops
>   */
> --
> 2.35.3
>


-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> Introduce a common DRM SHMEM shrinker framework that allows to reduce
> code duplication among DRM drivers by replacing theirs custom shrinker
> implementations with the generic shrinker.
>
> In order to start using DRM SHMEM shrinker drivers should:
>
> 1. Implement new evict() shmem object callback.
> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>    activate shrinking of shmem GEMs.
>
> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>
> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>

So I guess I get a price for being blind since forever, because this
thing existed since at least 2013. I just stumbled over
llist_lru.[hc], a purpose built list helper for shrinkers. I think we
should try to adopt that so that our gpu shrinkers look more like
shrinkers for everything else.

Apologies for this, since I fear this might cause a bit of churn.
Hopefully it's all contained to the list manipulation code in shmem
helpers, I don't think this should leak any further.
-Daniel

> ---
>  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>  include/drm/drm_device.h                      |   4 +
>  include/drm/drm_gem_shmem_helper.h            |  87 ++-
>  5 files changed, 594 insertions(+), 49 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 555fe212bd98..4cd0b5913492 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>
> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv >= 0) && shmem->evict &&
> +               shmem->eviction_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> +}
> +
> +static void
> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (!gem_shrinker || obj->import_attach)
> +               return;
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       if (drm_gem_shmem_is_evictable(shmem) ||
> +           drm_gem_shmem_is_purgeable(shmem))
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> +       else if (shmem->madv < 0)
> +               list_del_init(&shmem->madv_list);
> +       else if (shmem->evicted)
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> +       else if (!shmem->pages)
> +               list_del_init(&shmem->madv_list);
> +       else
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +}
> +
>  /**
>   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>   * @shmem: shmem GEM object to free
> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>         } else {
>                 dma_resv_lock(shmem->base.resv, NULL);
>
> +               /* take out shmem GEM object from the memory shrinker */
> +               drm_gem_shmem_madvise(shmem, -1);
> +
>                 WARN_ON(shmem->vmap_use_count);
>
>                 if (shmem->sgt) {
> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>                         sg_free_table(shmem->sgt);
>                         kfree(shmem->sgt);
>                 }
> -               if (shmem->pages)
> +               if (shmem->pages_use_count)
>                         drm_gem_shmem_put_pages(shmem);
>
>                 WARN_ON(shmem->pages_use_count);
> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>
> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->eviction_enabled = true;
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> +
> +/**
> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be purged. Initially purging is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->purge_enabled = true;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> +
> +static int
> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
>         struct page **pages;
>
> -       if (shmem->pages_use_count++ > 0)
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0) {
> +               WARN_ON(shmem->pages);
> +               return -ENOMEM;
> +       }
> +
> +       if (shmem->pages) {
> +               WARN_ON(!shmem->evicted);
>                 return 0;
> +       }
> +
> +       if (WARN_ON(!shmem->pages_use_count))
> +               return -EINVAL;
>
>         pages = drm_gem_get_pages(obj);
>         if (IS_ERR(pages)) {
>                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> -               shmem->pages_use_count = 0;
>                 return PTR_ERR(pages);
>         }
>
> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>         return 0;
>  }
>
> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       int err;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       if (shmem->pages_use_count++ > 0) {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err)
> +                       goto err_zero_use;
> +
> +               return 0;
> +       }
> +
> +       err = drm_gem_shmem_acquire_pages(shmem);
> +       if (err)
> +               goto err_zero_use;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +
> +err_zero_use:
> +       shmem->pages_use_count = 0;
> +
> +       return err;
> +}
> +
> +static void
> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       if (!shmem->pages) {
> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> +               return;
> +       }
> +
> +#ifdef CONFIG_X86
> +       if (shmem->map_wc)
> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> +#endif
> +
> +       drm_gem_put_pages(obj, shmem->pages,
> +                         shmem->pages_mark_dirty_on_put,
> +                         shmem->pages_mark_accessed_on_put);
> +       shmem->pages = NULL;
> +}
> +
>  /*
>   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>   * @shmem: shmem GEM object
> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>   */
>  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>  {
> -       struct drm_gem_object *obj = &shmem->base;
> -
>         dma_resv_assert_held(shmem->base.resv);
>
>         if (WARN_ON_ONCE(!shmem->pages_use_count))
> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>         if (--shmem->pages_use_count > 0)
>                 return;
>
> -#ifdef CONFIG_X86
> -       if (shmem->map_wc)
> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> -#endif
> +       drm_gem_shmem_release_pages(shmem);
>
> -       drm_gem_put_pages(obj, shmem->pages,
> -                         shmem->pages_mark_dirty_on_put,
> -                         shmem->pages_mark_accessed_on_put);
> -       shmem->pages = NULL;
> +       drm_gem_shmem_update_pages_state(shmem);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>
> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>   */
>  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>  {
> +       int ret;
> +
>         dma_resv_assert_held(shmem->base.resv);
>
>         WARN_ON(shmem->base.import_attach);
>
> -       return drm_gem_shmem_get_pages(shmem);
> +       ret = drm_gem_shmem_get_pages(shmem);
> +       if (!ret)
> +               shmem->pages_pin_count++;
> +
> +       return ret;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_pin);
>
> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>         WARN_ON(shmem->base.import_attach);
>
>         drm_gem_shmem_put_pages(shmem);
> +
> +       shmem->pages_pin_count--;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_unpin);
>
> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>                         return 0;
>                 }
>
> -               ret = drm_gem_shmem_get_pages(shmem);
> +               ret = drm_gem_shmem_pin(shmem);
>                 if (ret)
>                         goto err_zero_use;
>
> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>
>  err_put_pages:
>         if (!obj->import_attach)
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>  err_zero_use:
>         shmem->vmap_use_count = 0;
>
> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>                         return;
>
>                 vunmap(shmem->vaddr);
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>         }
>
>         shmem->vaddr = NULL;
> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>
>         madv = shmem->madv;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         return (madv >= 0);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_madvise);
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> + *                           hardware access to the memory.
> + * @shmem: shmem GEM object
> + *
> + * This function moves shmem GEM back to memory if it was previously evicted
> + * by the memory shrinker. The GEM is ready to use on success.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
> -       struct drm_device *dev = obj->dev;
> +       struct sg_table *sgt;
> +       int err;
>
>         dma_resv_assert_held(shmem->base.resv);
>
> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +       if (shmem->evicted) {
> +               err = drm_gem_shmem_acquire_pages(shmem);
> +               if (err)
> +                       return err;
> +
> +               sgt = drm_gem_shmem_get_sg_table(shmem);
> +               if (IS_ERR(sgt))
> +                       return PTR_ERR(sgt);
> +
> +               err = dma_map_sgtable(obj->dev->dev, sgt,
> +                                     DMA_BIDIRECTIONAL, 0);
> +               if (err) {
> +                       sg_free_table(sgt);
> +                       kfree(sgt);
> +                       return err;
> +               }
>
> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> -       sg_free_table(shmem->sgt);
> -       kfree(shmem->sgt);
> -       shmem->sgt = NULL;
> +               shmem->sgt = sgt;
> +               shmem->evicted = false;
>
> -       drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_update_pages_state(shmem);
> +       }
>
> -       shmem->madv = -1;
> +       if (!shmem->pages)
> +               return -ENOMEM;
>
> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> -       drm_gem_free_mmap_offset(obj);
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>
> -       /* Our goal here is to return as much of the memory as
> -        * is possible back to the system as we are called from OOM.
> -        * To do this we must instruct the shmfs to drop all of its
> -        * backing pages, *now*.
> -        */
> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_device *dev = obj->dev;
>
> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +       if (shmem->evicted)
> +               return;
> +
> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> +       drm_gem_shmem_release_pages(shmem);
> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> +
> +       sg_free_table(shmem->sgt);
> +       kfree(shmem->sgt);
> +       shmem->sgt = NULL;
>  }
> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>
>  /**
>   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>         vm_fault_t ret;
>         struct page *page;
>         pgoff_t page_offset;
> +       bool pages_unpinned;
> +       int err;
>
>         /* We don't use vmf->pgoff since that has the fake offset */
>         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>
>         dma_resv_lock(shmem->base.resv, NULL);
>
> -       if (page_offset >= num_pages ||
> -           WARN_ON_ONCE(!shmem->pages) ||
> -           shmem->madv < 0) {
> +       /* Sanity-check that we have the pages pointer when it should present */
> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> +
> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>                 ret = VM_FAULT_SIGBUS;
>         } else {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err) {
> +                       ret = VM_FAULT_OOM;
> +                       goto unlock;
> +               }
> +
>                 page = shmem->pages[page_offset];
>
>                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>         }
>
> +unlock:
>         dma_resv_unlock(shmem->base.resv);
>
>         return ret;
> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>  {
>         struct drm_gem_object *obj = vma->vm_private_data;
>         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -       int ret;
>
>         WARN_ON(shmem->base.import_attach);
>
>         dma_resv_lock(shmem->base.resv, NULL);
> -       ret = drm_gem_shmem_get_pages(shmem);
> -       WARN_ON_ONCE(ret != 0);
> +
> +       if (drm_gem_shmem_get_pages(shmem))
> +               shmem->pages_use_count++;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
>         dma_resv_unlock(shmem->base.resv);
>
>         drm_gem_vm_open(vma);
> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                               struct drm_printer *p, unsigned int indent)
>  {
> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>
>         if (shmem->base.import_attach)
> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>                                   shmem->vmap_use_count);
>
> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_print_info);
>
> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>
>         shmem->sgt = sgt;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         dma_resv_unlock(shmem->base.resv);
>
>         return sgt;
> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>
> +static struct drm_gem_shmem_shrinker *
> +to_drm_shrinker(struct shrinker *shrinker)
> +{
> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> +                                    struct shrink_control *sc)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       unsigned long count = 0;
> +
> +       if (!mutex_trylock(&gem_shrinker->lock))
> +               return 0;
> +
> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> +               count += shmem->base.size;
> +
> +               if (count >= SHRINK_EMPTY)
> +                       break;
> +       }
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       if (count >= SHRINK_EMPTY)
> +               return SHRINK_EMPTY - 1;
> +
> +       return count ?: SHRINK_EMPTY;
> +}
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> +       WARN_ON(shmem->evicted);
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +
> +       shmem->evicted = true;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> +
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +       drm_gem_free_mmap_offset(obj);
> +
> +       /* Our goal here is to return as much of the memory as
> +        * is possible back to the system as we are called from OOM.
> +        * To do this we must instruct the shmfs to drop all of its
> +        * backing pages, *now*.
> +        */
> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +
> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +
> +       shmem->madv = -1;
> +       shmem->evicted = false;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> +                                       unsigned long nr_to_scan,
> +                                       bool *lock_contention,
> +                                       bool evict)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       struct list_head still_in_list;
> +       struct drm_gem_object *obj;
> +       unsigned long freed = 0;
> +       size_t page_count;
> +       int err;
> +
> +       INIT_LIST_HEAD(&still_in_list);
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       while (freed < nr_to_scan) {
> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> +                                                typeof(*shmem), madv_list);
> +               if (!shmem)
> +                       break;
> +
> +               obj = &shmem->base;
> +               page_count = obj->size >> PAGE_SHIFT;
> +               list_move_tail(&shmem->madv_list, &still_in_list);
> +
> +               if (evict) {
> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> +                           get_nr_swap_pages() < page_count)
> +                               continue;
> +               } else {
> +                       if (!drm_gem_shmem_is_purgeable(shmem))
> +                               continue;
> +               }
> +
> +               /*
> +                * If it's in the process of being freed, gem_object->free()
> +                * may be blocked on lock waiting to remove it.  So just
> +                * skip it.
> +                */
> +               if (!kref_get_unless_zero(&obj->refcount))
> +                       continue;
> +
> +               mutex_unlock(&gem_shrinker->lock);
> +
> +               /* prevent racing with job-submission code paths */
> +               if (!dma_resv_trylock(obj->resv)) {
> +                       *lock_contention |= true;
> +                       goto shrinker_lock;
> +               }
> +
> +               /* prevent racing with the dma-buf importing/exporting */
> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> +                       *lock_contention |= true;
> +                       goto resv_unlock;
> +               }
> +
> +               /* check whether h/w uses this object */
> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> +                       goto object_name_unlock;
> +
> +               /* re-check whether eviction status hasn't changed */
> +               if (!drm_gem_shmem_is_evictable(shmem) &&
> +                   !drm_gem_shmem_is_purgeable(shmem))
> +                       goto object_name_unlock;
> +
> +               err = shmem->evict(shmem);
> +               if (!err)
> +                       freed += obj->size >> PAGE_SHIFT;
> +
> +object_name_unlock:
> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> +resv_unlock:
> +               dma_resv_unlock(obj->resv);
> +shrinker_lock:
> +               drm_gem_object_put(&shmem->base);
> +               mutex_lock(&gem_shrinker->lock);
> +       }
> +
> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       return freed;
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> +                                   struct shrink_control *sc)
> +{
> +       unsigned long nr_to_scan = sc->nr_to_scan;
> +       bool lock_contention = false;
> +       unsigned long freed;
> +
> +       /* purge as many objects as we can */
> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> +                                                       &lock_contention, false);
> +
> +       /* evict as many objects as we can */
> +       if (freed < nr_to_scan)
> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> +                                                                nr_to_scan - freed,
> +                                                                &lock_contention,
> +                                                                true);
> +
> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> +}
> +
> +/**
> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> + * @dev: DRM device
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker;
> +       int err;
> +
> +       if (WARN_ON(dev->shmem_shrinker))
> +               return -EBUSY;
> +
> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> +       if (!gem_shrinker)
> +               return -ENOMEM;
> +
> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> +       gem_shrinker->dev = dev;
> +
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> +       mutex_init(&gem_shrinker->lock);
> +
> +       dev->shmem_shrinker = gem_shrinker;
> +
> +       err = register_shrinker(&gem_shrinker->base);
> +       if (err) {
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +               return err;
> +       }
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> +
> +/**
> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> + * @dev: DRM device
> + */
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> +
> +       if (gem_shrinker) {
> +               unregister_shrinker(&gem_shrinker->base);
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> +               mutex_destroy(&gem_shrinker->lock);
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +       }
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> +
>  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>  MODULE_IMPORT_NS(DMA_BUF);
>  MODULE_LICENSE("GPL v2");
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> index a4bedfeb2ec4..7cc32556f908 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> @@ -15,6 +15,13 @@
>  #include "panfrost_gem.h"
>  #include "panfrost_mmu.h"
>
> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv > 0) &&
> +               !shmem->pages_pin_count && shmem->sgt &&
> +               !shmem->base.dma_buf && !shmem->base.import_attach;
> +}
> +
>  static unsigned long
>  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>  {
> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>                 return 0;
>
>         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -               if (drm_gem_shmem_is_purgeable(shmem))
> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>                         count += shmem->base.size >> PAGE_SHIFT;
>         }
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> index b2d93cb12ebf..81bacc7e1873 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>         uint32_t hw_res_handle;
>         bool dumb;
>         bool created;
> +       bool detached;
>         bool host3d_blob, guest_blob;
>         uint32_t blob_mem, blob_flags;
>
> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>
>  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>
> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> +
>  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>                                uint32_t *resid);
>  /* virtgpu_prime.c */
> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> index 9923c7a6885e..929546cad894 100644
> --- a/include/drm/drm_device.h
> +++ b/include/drm/drm_device.h
> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>  struct drm_vma_offset_manager;
>  struct drm_vram_mm;
>  struct drm_fb_helper;
> +struct drm_gem_shmem_shrinker;
>
>  struct inode;
>
> @@ -277,6 +278,9 @@ struct drm_device {
>         /** @vram_mm: VRAM MM memory manager */
>         struct drm_vram_mm *vram_mm;
>
> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> +
>         /**
>          * @switch_power_state:
>          *
> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> index 9a8983ee8abe..62c640678a91 100644
> --- a/include/drm/drm_gem_shmem_helper.h
> +++ b/include/drm/drm_gem_shmem_helper.h
> @@ -6,6 +6,7 @@
>  #include <linux/fs.h>
>  #include <linux/mm.h>
>  #include <linux/mutex.h>
> +#include <linux/shrinker.h>
>
>  #include <drm/drm_file.h>
>  #include <drm/drm_gem.h>
> @@ -15,6 +16,7 @@
>  struct dma_buf_attachment;
>  struct drm_mode_create_dumb;
>  struct drm_printer;
> +struct drm_device;
>  struct sg_table;
>
>  /**
> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>          */
>         unsigned int pages_use_count;
>
> +       /**
> +        * @pages_pin_count:
> +        *
> +        * Reference count on the pinned pages table.
> +        * The pages can be evicted by memory shrinker
> +        * when the count reaches zero.
> +        */
> +       unsigned int pages_pin_count;
> +
>         /**
>          * @madv: State for madvise
>          *
>          * 0 is active/inuse.
> +        * 1 is not-needed/can-be-purged
>          * A negative value is the object is purged.
> -        * Positive values are driver specific and not used by the helpers.
>          */
>         int madv;
>
> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>          * @map_wc: map object write-combined (instead of using shmem defaults).
>          */
>         bool map_wc;
> +
> +       /**
> +        * @eviction_enabled:
> +        *
> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool eviction_enabled;
> +
> +       /**
> +        * @purge_enabled:
> +        *
> +        * The shmem pages can be purged only if @purge_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool purge_enabled;
> +
> +       /**
> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> +        * Used internally by memory shrinker.
> +        */
> +       bool evicted;
> +
> +       /**
> +        * @evict:
> +        *
> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> +        * GEM's DMA reservation is kept locked by the shrinker. This is
> +        * optional callback that should be specified by drivers.
> +        *
> +        * Returns 0 on success, or -errno on error.
> +        */
> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>  };
>
>  #define to_drm_gem_shmem_obj(obj) \
> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>
>  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> +
>  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>  {
> -       return (shmem->madv > 0) &&
> -               !shmem->vmap_use_count && shmem->sgt &&
> -               !shmem->base.dma_buf && !shmem->base.import_attach;
> +       return (shmem->madv > 0) && shmem->evict &&
> +               shmem->purge_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>  }
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>
>  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>         return drm_gem_shmem_mmap(shmem, vma);
>  }
>
> +/**
> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> + */
> +struct drm_gem_shmem_shrinker {
> +       /** @base: Shrinker for purging shmem GEM objects */
> +       struct shrinker base;
> +
> +       /** @lock: Protects @lru_* */
> +       struct mutex lock;
> +
> +       /** @lru_pinned: List of pinned shmem GEM objects */
> +       struct list_head lru_pinned;
> +
> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> +       struct list_head lru_evictable;
> +
> +       /** @lru_evicted: List of evicted shmem GEM objects */
> +       struct list_head lru_evicted;
> +
> +       /** @dev: DRM device that uses this shrinker */
> +       struct drm_device *dev;
> +};
> +
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> +
>  /*
>   * Driver ops
>   */
> --
> 2.35.3
>


-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [Intel-gfx] [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

On Sun, Jun 5, 2022 at 9:47 AM Daniel Vetter <daniel@ffwll.ch> wrote:
>
> On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
> >
> > Introduce a common DRM SHMEM shrinker framework that allows to reduce
> > code duplication among DRM drivers by replacing theirs custom shrinker
> > implementations with the generic shrinker.
> >
> > In order to start using DRM SHMEM shrinker drivers should:
> >
> > 1. Implement new evict() shmem object callback.
> > 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> > 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
> >    activate shrinking of shmem GEMs.
> >
> > This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> > Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
> >
> > Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> > Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>
> So I guess I get a price for being blind since forever, because this
> thing existed since at least 2013. I just stumbled over
> llist_lru.[hc], a purpose built list helper for shrinkers. I think we
> should try to adopt that so that our gpu shrinkers look more like
> shrinkers for everything else.

followup from a bit of irc discussion w/ danvet about list_lru:

* It seems to be missing a way to bail out of iteration before
  nr_to_scan is hit.. which is going to be inconvenient if you
  want to allow active bos on the LRU but bail scanning once
  you encounter the first one.

* Not sure if the numa node awareness is super useful for GEM
  bos

First issue is perhaps not too hard to fix.  But maybe a better
idea is a drm_gem_lru helper type thing which is more tailored
to GEM buffers?

BR,
-R

> Apologies for this, since I fear this might cause a bit of churn.
> Hopefully it's all contained to the list manipulation code in shmem
> helpers, I don't think this should leak any further.
> -Daniel
>
> > ---
> >  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
> >  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
> >  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
> >  include/drm/drm_device.h                      |   4 +
> >  include/drm/drm_gem_shmem_helper.h            |  87 ++-
> >  5 files changed, 594 insertions(+), 49 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > index 555fe212bd98..4cd0b5913492 100644
> > --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> > +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
> >  }
> >  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
> >
> > +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       return (shmem->madv >= 0) && shmem->evict &&
> > +               shmem->eviction_enabled && shmem->pages_use_count &&
> > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> > +}
> > +
> > +static void
> > +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> > +
> > +       dma_resv_assert_held(shmem->base.resv);
> > +
> > +       if (!gem_shrinker || obj->import_attach)
> > +               return;
> > +
> > +       mutex_lock(&gem_shrinker->lock);
> > +
> > +       if (drm_gem_shmem_is_evictable(shmem) ||
> > +           drm_gem_shmem_is_purgeable(shmem))
> > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> > +       else if (shmem->madv < 0)
> > +               list_del_init(&shmem->madv_list);
> > +       else if (shmem->evicted)
> > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> > +       else if (!shmem->pages)
> > +               list_del_init(&shmem->madv_list);
> > +       else
> > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> > +
> > +       mutex_unlock(&gem_shrinker->lock);
> > +}
> > +
> >  /**
> >   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
> >   * @shmem: shmem GEM object to free
> > @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> >         } else {
> >                 dma_resv_lock(shmem->base.resv, NULL);
> >
> > +               /* take out shmem GEM object from the memory shrinker */
> > +               drm_gem_shmem_madvise(shmem, -1);
> > +
> >                 WARN_ON(shmem->vmap_use_count);
> >
> >                 if (shmem->sgt) {
> > @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> >                         sg_free_table(shmem->sgt);
> >                         kfree(shmem->sgt);
> >                 }
> > -               if (shmem->pages)
> > +               if (shmem->pages_use_count)
> >                         drm_gem_shmem_put_pages(shmem);
> >
> >                 WARN_ON(shmem->pages_use_count);
> > @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> >  }
> >  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
> >
> > -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > +/**
> > + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> > + * @shmem: shmem GEM object
> > + *
> > + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       dma_resv_lock(shmem->base.resv, NULL);
> > +
> > +       if (shmem->madv < 0)
> > +               return -ENOMEM;
> > +
> > +       shmem->eviction_enabled = true;
> > +
> > +       dma_resv_unlock(shmem->base.resv);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> > +
> > +/**
> > + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> > + * @shmem: shmem GEM object
> > + *
> > + * Tell memory shrinker that this GEM can be purged. Initially purging is
> > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       dma_resv_lock(shmem->base.resv, NULL);
> > +
> > +       if (shmem->madv < 0)
> > +               return -ENOMEM;
> > +
> > +       shmem->purge_enabled = true;
> > +
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       dma_resv_unlock(shmem->base.resv);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> > +
> > +static int
> > +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
> >  {
> >         struct drm_gem_object *obj = &shmem->base;
> >         struct page **pages;
> >
> > -       if (shmem->pages_use_count++ > 0)
> > +       dma_resv_assert_held(shmem->base.resv);
> > +
> > +       if (shmem->madv < 0) {
> > +               WARN_ON(shmem->pages);
> > +               return -ENOMEM;
> > +       }
> > +
> > +       if (shmem->pages) {
> > +               WARN_ON(!shmem->evicted);
> >                 return 0;
> > +       }
> > +
> > +       if (WARN_ON(!shmem->pages_use_count))
> > +               return -EINVAL;
> >
> >         pages = drm_gem_get_pages(obj);
> >         if (IS_ERR(pages)) {
> >                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> > -               shmem->pages_use_count = 0;
> >                 return PTR_ERR(pages);
> >         }
> >
> > @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> >         return 0;
> >  }
> >
> > +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > +{
> > +       int err;
> > +
> > +       dma_resv_assert_held(shmem->base.resv);
> > +
> > +       if (shmem->madv < 0)
> > +               return -ENOMEM;
> > +
> > +       if (shmem->pages_use_count++ > 0) {
> > +               err = drm_gem_shmem_swap_in(shmem);
> > +               if (err)
> > +                       goto err_zero_use;
> > +
> > +               return 0;
> > +       }
> > +
> > +       err = drm_gem_shmem_acquire_pages(shmem);
> > +       if (err)
> > +               goto err_zero_use;
> > +
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       return 0;
> > +
> > +err_zero_use:
> > +       shmem->pages_use_count = 0;
> > +
> > +       return err;
> > +}
> > +
> > +static void
> > +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +
> > +       if (!shmem->pages) {
> > +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> > +               return;
> > +       }
> > +
> > +#ifdef CONFIG_X86
> > +       if (shmem->map_wc)
> > +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > +#endif
> > +
> > +       drm_gem_put_pages(obj, shmem->pages,
> > +                         shmem->pages_mark_dirty_on_put,
> > +                         shmem->pages_mark_accessed_on_put);
> > +       shmem->pages = NULL;
> > +}
> > +
> >  /*
> >   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
> >   * @shmem: shmem GEM object
> > @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> >   */
> >  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> >  {
> > -       struct drm_gem_object *obj = &shmem->base;
> > -
> >         dma_resv_assert_held(shmem->base.resv);
> >
> >         if (WARN_ON_ONCE(!shmem->pages_use_count))
> > @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> >         if (--shmem->pages_use_count > 0)
> >                 return;
> >
> > -#ifdef CONFIG_X86
> > -       if (shmem->map_wc)
> > -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > -#endif
> > +       drm_gem_shmem_release_pages(shmem);
> >
> > -       drm_gem_put_pages(obj, shmem->pages,
> > -                         shmem->pages_mark_dirty_on_put,
> > -                         shmem->pages_mark_accessed_on_put);
> > -       shmem->pages = NULL;
> > +       drm_gem_shmem_update_pages_state(shmem);
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> >
> > @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> >   */
> >  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
> >  {
> > +       int ret;
> > +
> >         dma_resv_assert_held(shmem->base.resv);
> >
> >         WARN_ON(shmem->base.import_attach);
> >
> > -       return drm_gem_shmem_get_pages(shmem);
> > +       ret = drm_gem_shmem_get_pages(shmem);
> > +       if (!ret)
> > +               shmem->pages_pin_count++;
> > +
> > +       return ret;
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_pin);
> >
> > @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
> >         WARN_ON(shmem->base.import_attach);
> >
> >         drm_gem_shmem_put_pages(shmem);
> > +
> > +       shmem->pages_pin_count--;
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_unpin);
> >
> > @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> >                         return 0;
> >                 }
> >
> > -               ret = drm_gem_shmem_get_pages(shmem);
> > +               ret = drm_gem_shmem_pin(shmem);
> >                 if (ret)
> >                         goto err_zero_use;
> >
> > @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> >
> >  err_put_pages:
> >         if (!obj->import_attach)
> > -               drm_gem_shmem_put_pages(shmem);
> > +               drm_gem_shmem_unpin(shmem);
> >  err_zero_use:
> >         shmem->vmap_use_count = 0;
> >
> > @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
> >                         return;
> >
> >                 vunmap(shmem->vaddr);
> > -               drm_gem_shmem_put_pages(shmem);
> > +               drm_gem_shmem_unpin(shmem);
> >         }
> >
> >         shmem->vaddr = NULL;
> > @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
> >
> >         madv = shmem->madv;
> >
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> >         return (madv >= 0);
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_madvise);
> >
> > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > +/**
> > + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> > + *                           hardware access to the memory.
> > + * @shmem: shmem GEM object
> > + *
> > + * This function moves shmem GEM back to memory if it was previously evicted
> > + * by the memory shrinker. The GEM is ready to use on success.
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
> >  {
> >         struct drm_gem_object *obj = &shmem->base;
> > -       struct drm_device *dev = obj->dev;
> > +       struct sg_table *sgt;
> > +       int err;
> >
> >         dma_resv_assert_held(shmem->base.resv);
> >
> > -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > +       if (shmem->evicted) {
> > +               err = drm_gem_shmem_acquire_pages(shmem);
> > +               if (err)
> > +                       return err;
> > +
> > +               sgt = drm_gem_shmem_get_sg_table(shmem);
> > +               if (IS_ERR(sgt))
> > +                       return PTR_ERR(sgt);
> > +
> > +               err = dma_map_sgtable(obj->dev->dev, sgt,
> > +                                     DMA_BIDIRECTIONAL, 0);
> > +               if (err) {
> > +                       sg_free_table(sgt);
> > +                       kfree(sgt);
> > +                       return err;
> > +               }
> >
> > -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > -       sg_free_table(shmem->sgt);
> > -       kfree(shmem->sgt);
> > -       shmem->sgt = NULL;
> > +               shmem->sgt = sgt;
> > +               shmem->evicted = false;
> >
> > -       drm_gem_shmem_put_pages(shmem);
> > +               drm_gem_shmem_update_pages_state(shmem);
> > +       }
> >
> > -       shmem->madv = -1;
> > +       if (!shmem->pages)
> > +               return -ENOMEM;
> >
> > -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > -       drm_gem_free_mmap_offset(obj);
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
> >
> > -       /* Our goal here is to return as much of the memory as
> > -        * is possible back to the system as we are called from OOM.
> > -        * To do this we must instruct the shmfs to drop all of its
> > -        * backing pages, *now*.
> > -        */
> > -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +       struct drm_device *dev = obj->dev;
> >
> > -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > +       if (shmem->evicted)
> > +               return;
> > +
> > +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > +       drm_gem_shmem_release_pages(shmem);
> > +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > +
> > +       sg_free_table(shmem->sgt);
> > +       kfree(shmem->sgt);
> > +       shmem->sgt = NULL;
> >  }
> > -EXPORT_SYMBOL(drm_gem_shmem_purge);
> >
> >  /**
> >   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> > @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
> >         vm_fault_t ret;
> >         struct page *page;
> >         pgoff_t page_offset;
> > +       bool pages_unpinned;
> > +       int err;
> >
> >         /* We don't use vmf->pgoff since that has the fake offset */
> >         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
> >
> >         dma_resv_lock(shmem->base.resv, NULL);
> >
> > -       if (page_offset >= num_pages ||
> > -           WARN_ON_ONCE(!shmem->pages) ||
> > -           shmem->madv < 0) {
> > +       /* Sanity-check that we have the pages pointer when it should present */
> > +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> > +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> > +
> > +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
> >                 ret = VM_FAULT_SIGBUS;
> >         } else {
> > +               err = drm_gem_shmem_swap_in(shmem);
> > +               if (err) {
> > +                       ret = VM_FAULT_OOM;
> > +                       goto unlock;
> > +               }
> > +
> >                 page = shmem->pages[page_offset];
> >
> >                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
> >         }
> >
> > +unlock:
> >         dma_resv_unlock(shmem->base.resv);
> >
> >         return ret;
> > @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
> >  {
> >         struct drm_gem_object *obj = vma->vm_private_data;
> >         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> > -       int ret;
> >
> >         WARN_ON(shmem->base.import_attach);
> >
> >         dma_resv_lock(shmem->base.resv, NULL);
> > -       ret = drm_gem_shmem_get_pages(shmem);
> > -       WARN_ON_ONCE(ret != 0);
> > +
> > +       if (drm_gem_shmem_get_pages(shmem))
> > +               shmem->pages_use_count++;
> > +
> > +       drm_gem_shmem_update_pages_state(shmem);
> >         dma_resv_unlock(shmem->base.resv);
> >
> >         drm_gem_vm_open(vma);
> > @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
> >  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> >                               struct drm_printer *p, unsigned int indent)
> >  {
> > +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> > +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
> >         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
> >
> >         if (shmem->base.import_attach)
> > @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> >                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
> >                                   shmem->vmap_use_count);
> >
> > +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
> >         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> > +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_print_info);
> >
> > @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
> >
> >         shmem->sgt = sgt;
> >
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> >         dma_resv_unlock(shmem->base.resv);
> >
> >         return sgt;
> > @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
> >  }
> >  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
> >
> > +static struct drm_gem_shmem_shrinker *
> > +to_drm_shrinker(struct shrinker *shrinker)
> > +{
> > +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> > +}
> > +
> > +static unsigned long
> > +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> > +                                    struct shrink_control *sc)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > +       struct drm_gem_shmem_object *shmem;
> > +       unsigned long count = 0;
> > +
> > +       if (!mutex_trylock(&gem_shrinker->lock))
> > +               return 0;
> > +
> > +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> > +               count += shmem->base.size;
> > +
> > +               if (count >= SHRINK_EMPTY)
> > +                       break;
> > +       }
> > +
> > +       mutex_unlock(&gem_shrinker->lock);
> > +
> > +       if (count >= SHRINK_EMPTY)
> > +               return SHRINK_EMPTY - 1;
> > +
> > +       return count ?: SHRINK_EMPTY;
> > +}
> > +
> > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> > +{
> > +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> > +       WARN_ON(shmem->evicted);
> > +
> > +       drm_gem_shmem_unpin_pages(shmem);
> > +
> > +       shmem->evicted = true;
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> > +
> > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +
> > +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > +
> > +       drm_gem_shmem_unpin_pages(shmem);
> > +       drm_gem_free_mmap_offset(obj);
> > +
> > +       /* Our goal here is to return as much of the memory as
> > +        * is possible back to the system as we are called from OOM.
> > +        * To do this we must instruct the shmfs to drop all of its
> > +        * backing pages, *now*.
> > +        */
> > +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > +
> > +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > +
> > +       shmem->madv = -1;
> > +       shmem->evicted = false;
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> > +
> > +static unsigned long
> > +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> > +                                       unsigned long nr_to_scan,
> > +                                       bool *lock_contention,
> > +                                       bool evict)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > +       struct drm_gem_shmem_object *shmem;
> > +       struct list_head still_in_list;
> > +       struct drm_gem_object *obj;
> > +       unsigned long freed = 0;
> > +       size_t page_count;
> > +       int err;
> > +
> > +       INIT_LIST_HEAD(&still_in_list);
> > +
> > +       mutex_lock(&gem_shrinker->lock);
> > +
> > +       while (freed < nr_to_scan) {
> > +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> > +                                                typeof(*shmem), madv_list);
> > +               if (!shmem)
> > +                       break;
> > +
> > +               obj = &shmem->base;
> > +               page_count = obj->size >> PAGE_SHIFT;
> > +               list_move_tail(&shmem->madv_list, &still_in_list);
> > +
> > +               if (evict) {
> > +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> > +                           get_nr_swap_pages() < page_count)
> > +                               continue;
> > +               } else {
> > +                       if (!drm_gem_shmem_is_purgeable(shmem))
> > +                               continue;
> > +               }
> > +
> > +               /*
> > +                * If it's in the process of being freed, gem_object->free()
> > +                * may be blocked on lock waiting to remove it.  So just
> > +                * skip it.
> > +                */
> > +               if (!kref_get_unless_zero(&obj->refcount))
> > +                       continue;
> > +
> > +               mutex_unlock(&gem_shrinker->lock);
> > +
> > +               /* prevent racing with job-submission code paths */
> > +               if (!dma_resv_trylock(obj->resv)) {
> > +                       *lock_contention |= true;
> > +                       goto shrinker_lock;
> > +               }
> > +
> > +               /* prevent racing with the dma-buf importing/exporting */
> > +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > +                       *lock_contention |= true;
> > +                       goto resv_unlock;
> > +               }
> > +
> > +               /* check whether h/w uses this object */
> > +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> > +                       goto object_name_unlock;
> > +
> > +               /* re-check whether eviction status hasn't changed */
> > +               if (!drm_gem_shmem_is_evictable(shmem) &&
> > +                   !drm_gem_shmem_is_purgeable(shmem))
> > +                       goto object_name_unlock;
> > +
> > +               err = shmem->evict(shmem);
> > +               if (!err)
> > +                       freed += obj->size >> PAGE_SHIFT;
> > +
> > +object_name_unlock:
> > +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> > +resv_unlock:
> > +               dma_resv_unlock(obj->resv);
> > +shrinker_lock:
> > +               drm_gem_object_put(&shmem->base);
> > +               mutex_lock(&gem_shrinker->lock);
> > +       }
> > +
> > +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> > +
> > +       mutex_unlock(&gem_shrinker->lock);
> > +
> > +       return freed;
> > +}
> > +
> > +static unsigned long
> > +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> > +                                   struct shrink_control *sc)
> > +{
> > +       unsigned long nr_to_scan = sc->nr_to_scan;
> > +       bool lock_contention = false;
> > +       unsigned long freed;
> > +
> > +       /* purge as many objects as we can */
> > +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> > +                                                       &lock_contention, false);
> > +
> > +       /* evict as many objects as we can */
> > +       if (freed < nr_to_scan)
> > +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> > +                                                                nr_to_scan - freed,
> > +                                                                &lock_contention,
> > +                                                                true);
> > +
> > +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> > +}
> > +
> > +/**
> > + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> > + * @dev: DRM device
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker;
> > +       int err;
> > +
> > +       if (WARN_ON(dev->shmem_shrinker))
> > +               return -EBUSY;
> > +
> > +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> > +       if (!gem_shrinker)
> > +               return -ENOMEM;
> > +
> > +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> > +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> > +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> > +       gem_shrinker->dev = dev;
> > +
> > +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> > +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> > +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> > +       mutex_init(&gem_shrinker->lock);
> > +
> > +       dev->shmem_shrinker = gem_shrinker;
> > +
> > +       err = register_shrinker(&gem_shrinker->base);
> > +       if (err) {
> > +               dev->shmem_shrinker = NULL;
> > +               kfree(gem_shrinker);
> > +               return err;
> > +       }
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> > +
> > +/**
> > + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> > + * @dev: DRM device
> > + */
> > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> > +
> > +       if (gem_shrinker) {
> > +               unregister_shrinker(&gem_shrinker->base);
> > +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> > +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> > +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> > +               mutex_destroy(&gem_shrinker->lock);
> > +               dev->shmem_shrinker = NULL;
> > +               kfree(gem_shrinker);
> > +       }
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> > +
> >  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
> >  MODULE_IMPORT_NS(DMA_BUF);
> >  MODULE_LICENSE("GPL v2");
> > diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > index a4bedfeb2ec4..7cc32556f908 100644
> > --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > @@ -15,6 +15,13 @@
> >  #include "panfrost_gem.h"
> >  #include "panfrost_mmu.h"
> >
> > +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       return (shmem->madv > 0) &&
> > +               !shmem->pages_pin_count && shmem->sgt &&
> > +               !shmem->base.dma_buf && !shmem->base.import_attach;
> > +}
> > +
> >  static unsigned long
> >  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
> >  {
> > @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
> >                 return 0;
> >
> >         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> > -               if (drm_gem_shmem_is_purgeable(shmem))
> > +               if (panfrost_gem_shmem_is_purgeable(shmem))
> >                         count += shmem->base.size >> PAGE_SHIFT;
> >         }
> >
> > diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > index b2d93cb12ebf..81bacc7e1873 100644
> > --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> > +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > @@ -89,6 +89,7 @@ struct virtio_gpu_object {
> >         uint32_t hw_res_handle;
> >         bool dumb;
> >         bool created;
> > +       bool detached;
> >         bool host3d_blob, guest_blob;
> >         uint32_t blob_mem, blob_flags;
> >
> > @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
> >
> >  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
> >
> > +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> > +
> >  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
> >                                uint32_t *resid);
> >  /* virtgpu_prime.c */
> > diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> > index 9923c7a6885e..929546cad894 100644
> > --- a/include/drm/drm_device.h
> > +++ b/include/drm/drm_device.h
> > @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
> >  struct drm_vma_offset_manager;
> >  struct drm_vram_mm;
> >  struct drm_fb_helper;
> > +struct drm_gem_shmem_shrinker;
> >
> >  struct inode;
> >
> > @@ -277,6 +278,9 @@ struct drm_device {
> >         /** @vram_mm: VRAM MM memory manager */
> >         struct drm_vram_mm *vram_mm;
> >
> > +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> > +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> > +
> >         /**
> >          * @switch_power_state:
> >          *
> > diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> > index 9a8983ee8abe..62c640678a91 100644
> > --- a/include/drm/drm_gem_shmem_helper.h
> > +++ b/include/drm/drm_gem_shmem_helper.h
> > @@ -6,6 +6,7 @@
> >  #include <linux/fs.h>
> >  #include <linux/mm.h>
> >  #include <linux/mutex.h>
> > +#include <linux/shrinker.h>
> >
> >  #include <drm/drm_file.h>
> >  #include <drm/drm_gem.h>
> > @@ -15,6 +16,7 @@
> >  struct dma_buf_attachment;
> >  struct drm_mode_create_dumb;
> >  struct drm_printer;
> > +struct drm_device;
> >  struct sg_table;
> >
> >  /**
> > @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
> >          */
> >         unsigned int pages_use_count;
> >
> > +       /**
> > +        * @pages_pin_count:
> > +        *
> > +        * Reference count on the pinned pages table.
> > +        * The pages can be evicted by memory shrinker
> > +        * when the count reaches zero.
> > +        */
> > +       unsigned int pages_pin_count;
> > +
> >         /**
> >          * @madv: State for madvise
> >          *
> >          * 0 is active/inuse.
> > +        * 1 is not-needed/can-be-purged
> >          * A negative value is the object is purged.
> > -        * Positive values are driver specific and not used by the helpers.
> >          */
> >         int madv;
> >
> > @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
> >          * @map_wc: map object write-combined (instead of using shmem defaults).
> >          */
> >         bool map_wc;
> > +
> > +       /**
> > +        * @eviction_enabled:
> > +        *
> > +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> > +        * Used internally by memory shrinker.
> > +        */
> > +       bool eviction_enabled;
> > +
> > +       /**
> > +        * @purge_enabled:
> > +        *
> > +        * The shmem pages can be purged only if @purge_enabled is set to true.
> > +        * Used internally by memory shrinker.
> > +        */
> > +       bool purge_enabled;
> > +
> > +       /**
> > +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> > +        * Used internally by memory shrinker.
> > +        */
> > +       bool evicted;
> > +
> > +       /**
> > +        * @evict:
> > +        *
> > +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> > +        * GEM's DMA reservation is kept locked by the shrinker. This is
> > +        * optional callback that should be specified by drivers.
> > +        *
> > +        * Returns 0 on success, or -errno on error.
> > +        */
> > +       int (*evict)(struct drm_gem_shmem_object *shmem);
> >  };
> >
> >  #define to_drm_gem_shmem_obj(obj) \
> > @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
> >
> >  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
> >
> > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> > +
> >  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> >  {
> > -       return (shmem->madv > 0) &&
> > -               !shmem->vmap_use_count && shmem->sgt &&
> > -               !shmem->base.dma_buf && !shmem->base.import_attach;
> > +       return (shmem->madv > 0) && shmem->evict &&
> > +               shmem->purge_enabled && shmem->pages_use_count &&
> > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
> >  }
> >
> > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> > +
> > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> >
> >  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
> >  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> > @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
> >         return drm_gem_shmem_mmap(shmem, vma);
> >  }
> >
> > +/**
> > + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> > + */
> > +struct drm_gem_shmem_shrinker {
> > +       /** @base: Shrinker for purging shmem GEM objects */
> > +       struct shrinker base;
> > +
> > +       /** @lock: Protects @lru_* */
> > +       struct mutex lock;
> > +
> > +       /** @lru_pinned: List of pinned shmem GEM objects */
> > +       struct list_head lru_pinned;
> > +
> > +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> > +       struct list_head lru_evictable;
> > +
> > +       /** @lru_evicted: List of evicted shmem GEM objects */
> > +       struct list_head lru_evicted;
> > +
> > +       /** @dev: DRM device that uses this shrinker */
> > +       struct drm_device *dev;
> > +};
> > +
> > +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> > +
> >  /*
> >   * Driver ops
> >   */
> > --
> > 2.35.3
> >
>
>
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

On Sun, Jun 5, 2022 at 9:47 AM Daniel Vetter <daniel@ffwll.ch> wrote:
>
> On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
> >
> > Introduce a common DRM SHMEM shrinker framework that allows to reduce
> > code duplication among DRM drivers by replacing theirs custom shrinker
> > implementations with the generic shrinker.
> >
> > In order to start using DRM SHMEM shrinker drivers should:
> >
> > 1. Implement new evict() shmem object callback.
> > 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> > 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
> >    activate shrinking of shmem GEMs.
> >
> > This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> > Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
> >
> > Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> > Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>
> So I guess I get a price for being blind since forever, because this
> thing existed since at least 2013. I just stumbled over
> llist_lru.[hc], a purpose built list helper for shrinkers. I think we
> should try to adopt that so that our gpu shrinkers look more like
> shrinkers for everything else.

followup from a bit of irc discussion w/ danvet about list_lru:

* It seems to be missing a way to bail out of iteration before
  nr_to_scan is hit.. which is going to be inconvenient if you
  want to allow active bos on the LRU but bail scanning once
  you encounter the first one.

* Not sure if the numa node awareness is super useful for GEM
  bos

First issue is perhaps not too hard to fix.  But maybe a better
idea is a drm_gem_lru helper type thing which is more tailored
to GEM buffers?

BR,
-R

> Apologies for this, since I fear this might cause a bit of churn.
> Hopefully it's all contained to the list manipulation code in shmem
> helpers, I don't think this should leak any further.
> -Daniel
>
> > ---
> >  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
> >  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
> >  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
> >  include/drm/drm_device.h                      |   4 +
> >  include/drm/drm_gem_shmem_helper.h            |  87 ++-
> >  5 files changed, 594 insertions(+), 49 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > index 555fe212bd98..4cd0b5913492 100644
> > --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> > +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
> >  }
> >  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
> >
> > +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       return (shmem->madv >= 0) && shmem->evict &&
> > +               shmem->eviction_enabled && shmem->pages_use_count &&
> > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> > +}
> > +
> > +static void
> > +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> > +
> > +       dma_resv_assert_held(shmem->base.resv);
> > +
> > +       if (!gem_shrinker || obj->import_attach)
> > +               return;
> > +
> > +       mutex_lock(&gem_shrinker->lock);
> > +
> > +       if (drm_gem_shmem_is_evictable(shmem) ||
> > +           drm_gem_shmem_is_purgeable(shmem))
> > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> > +       else if (shmem->madv < 0)
> > +               list_del_init(&shmem->madv_list);
> > +       else if (shmem->evicted)
> > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> > +       else if (!shmem->pages)
> > +               list_del_init(&shmem->madv_list);
> > +       else
> > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> > +
> > +       mutex_unlock(&gem_shrinker->lock);
> > +}
> > +
> >  /**
> >   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
> >   * @shmem: shmem GEM object to free
> > @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> >         } else {
> >                 dma_resv_lock(shmem->base.resv, NULL);
> >
> > +               /* take out shmem GEM object from the memory shrinker */
> > +               drm_gem_shmem_madvise(shmem, -1);
> > +
> >                 WARN_ON(shmem->vmap_use_count);
> >
> >                 if (shmem->sgt) {
> > @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> >                         sg_free_table(shmem->sgt);
> >                         kfree(shmem->sgt);
> >                 }
> > -               if (shmem->pages)
> > +               if (shmem->pages_use_count)
> >                         drm_gem_shmem_put_pages(shmem);
> >
> >                 WARN_ON(shmem->pages_use_count);
> > @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> >  }
> >  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
> >
> > -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > +/**
> > + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> > + * @shmem: shmem GEM object
> > + *
> > + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       dma_resv_lock(shmem->base.resv, NULL);
> > +
> > +       if (shmem->madv < 0)
> > +               return -ENOMEM;
> > +
> > +       shmem->eviction_enabled = true;
> > +
> > +       dma_resv_unlock(shmem->base.resv);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> > +
> > +/**
> > + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> > + * @shmem: shmem GEM object
> > + *
> > + * Tell memory shrinker that this GEM can be purged. Initially purging is
> > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       dma_resv_lock(shmem->base.resv, NULL);
> > +
> > +       if (shmem->madv < 0)
> > +               return -ENOMEM;
> > +
> > +       shmem->purge_enabled = true;
> > +
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       dma_resv_unlock(shmem->base.resv);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> > +
> > +static int
> > +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
> >  {
> >         struct drm_gem_object *obj = &shmem->base;
> >         struct page **pages;
> >
> > -       if (shmem->pages_use_count++ > 0)
> > +       dma_resv_assert_held(shmem->base.resv);
> > +
> > +       if (shmem->madv < 0) {
> > +               WARN_ON(shmem->pages);
> > +               return -ENOMEM;
> > +       }
> > +
> > +       if (shmem->pages) {
> > +               WARN_ON(!shmem->evicted);
> >                 return 0;
> > +       }
> > +
> > +       if (WARN_ON(!shmem->pages_use_count))
> > +               return -EINVAL;
> >
> >         pages = drm_gem_get_pages(obj);
> >         if (IS_ERR(pages)) {
> >                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> > -               shmem->pages_use_count = 0;
> >                 return PTR_ERR(pages);
> >         }
> >
> > @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> >         return 0;
> >  }
> >
> > +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > +{
> > +       int err;
> > +
> > +       dma_resv_assert_held(shmem->base.resv);
> > +
> > +       if (shmem->madv < 0)
> > +               return -ENOMEM;
> > +
> > +       if (shmem->pages_use_count++ > 0) {
> > +               err = drm_gem_shmem_swap_in(shmem);
> > +               if (err)
> > +                       goto err_zero_use;
> > +
> > +               return 0;
> > +       }
> > +
> > +       err = drm_gem_shmem_acquire_pages(shmem);
> > +       if (err)
> > +               goto err_zero_use;
> > +
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       return 0;
> > +
> > +err_zero_use:
> > +       shmem->pages_use_count = 0;
> > +
> > +       return err;
> > +}
> > +
> > +static void
> > +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +
> > +       if (!shmem->pages) {
> > +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> > +               return;
> > +       }
> > +
> > +#ifdef CONFIG_X86
> > +       if (shmem->map_wc)
> > +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > +#endif
> > +
> > +       drm_gem_put_pages(obj, shmem->pages,
> > +                         shmem->pages_mark_dirty_on_put,
> > +                         shmem->pages_mark_accessed_on_put);
> > +       shmem->pages = NULL;
> > +}
> > +
> >  /*
> >   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
> >   * @shmem: shmem GEM object
> > @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> >   */
> >  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> >  {
> > -       struct drm_gem_object *obj = &shmem->base;
> > -
> >         dma_resv_assert_held(shmem->base.resv);
> >
> >         if (WARN_ON_ONCE(!shmem->pages_use_count))
> > @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> >         if (--shmem->pages_use_count > 0)
> >                 return;
> >
> > -#ifdef CONFIG_X86
> > -       if (shmem->map_wc)
> > -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > -#endif
> > +       drm_gem_shmem_release_pages(shmem);
> >
> > -       drm_gem_put_pages(obj, shmem->pages,
> > -                         shmem->pages_mark_dirty_on_put,
> > -                         shmem->pages_mark_accessed_on_put);
> > -       shmem->pages = NULL;
> > +       drm_gem_shmem_update_pages_state(shmem);
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> >
> > @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> >   */
> >  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
> >  {
> > +       int ret;
> > +
> >         dma_resv_assert_held(shmem->base.resv);
> >
> >         WARN_ON(shmem->base.import_attach);
> >
> > -       return drm_gem_shmem_get_pages(shmem);
> > +       ret = drm_gem_shmem_get_pages(shmem);
> > +       if (!ret)
> > +               shmem->pages_pin_count++;
> > +
> > +       return ret;
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_pin);
> >
> > @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
> >         WARN_ON(shmem->base.import_attach);
> >
> >         drm_gem_shmem_put_pages(shmem);
> > +
> > +       shmem->pages_pin_count--;
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_unpin);
> >
> > @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> >                         return 0;
> >                 }
> >
> > -               ret = drm_gem_shmem_get_pages(shmem);
> > +               ret = drm_gem_shmem_pin(shmem);
> >                 if (ret)
> >                         goto err_zero_use;
> >
> > @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> >
> >  err_put_pages:
> >         if (!obj->import_attach)
> > -               drm_gem_shmem_put_pages(shmem);
> > +               drm_gem_shmem_unpin(shmem);
> >  err_zero_use:
> >         shmem->vmap_use_count = 0;
> >
> > @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
> >                         return;
> >
> >                 vunmap(shmem->vaddr);
> > -               drm_gem_shmem_put_pages(shmem);
> > +               drm_gem_shmem_unpin(shmem);
> >         }
> >
> >         shmem->vaddr = NULL;
> > @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
> >
> >         madv = shmem->madv;
> >
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> >         return (madv >= 0);
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_madvise);
> >
> > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > +/**
> > + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> > + *                           hardware access to the memory.
> > + * @shmem: shmem GEM object
> > + *
> > + * This function moves shmem GEM back to memory if it was previously evicted
> > + * by the memory shrinker. The GEM is ready to use on success.
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
> >  {
> >         struct drm_gem_object *obj = &shmem->base;
> > -       struct drm_device *dev = obj->dev;
> > +       struct sg_table *sgt;
> > +       int err;
> >
> >         dma_resv_assert_held(shmem->base.resv);
> >
> > -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > +       if (shmem->evicted) {
> > +               err = drm_gem_shmem_acquire_pages(shmem);
> > +               if (err)
> > +                       return err;
> > +
> > +               sgt = drm_gem_shmem_get_sg_table(shmem);
> > +               if (IS_ERR(sgt))
> > +                       return PTR_ERR(sgt);
> > +
> > +               err = dma_map_sgtable(obj->dev->dev, sgt,
> > +                                     DMA_BIDIRECTIONAL, 0);
> > +               if (err) {
> > +                       sg_free_table(sgt);
> > +                       kfree(sgt);
> > +                       return err;
> > +               }
> >
> > -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > -       sg_free_table(shmem->sgt);
> > -       kfree(shmem->sgt);
> > -       shmem->sgt = NULL;
> > +               shmem->sgt = sgt;
> > +               shmem->evicted = false;
> >
> > -       drm_gem_shmem_put_pages(shmem);
> > +               drm_gem_shmem_update_pages_state(shmem);
> > +       }
> >
> > -       shmem->madv = -1;
> > +       if (!shmem->pages)
> > +               return -ENOMEM;
> >
> > -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > -       drm_gem_free_mmap_offset(obj);
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
> >
> > -       /* Our goal here is to return as much of the memory as
> > -        * is possible back to the system as we are called from OOM.
> > -        * To do this we must instruct the shmfs to drop all of its
> > -        * backing pages, *now*.
> > -        */
> > -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +       struct drm_device *dev = obj->dev;
> >
> > -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > +       if (shmem->evicted)
> > +               return;
> > +
> > +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > +       drm_gem_shmem_release_pages(shmem);
> > +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > +
> > +       sg_free_table(shmem->sgt);
> > +       kfree(shmem->sgt);
> > +       shmem->sgt = NULL;
> >  }
> > -EXPORT_SYMBOL(drm_gem_shmem_purge);
> >
> >  /**
> >   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> > @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
> >         vm_fault_t ret;
> >         struct page *page;
> >         pgoff_t page_offset;
> > +       bool pages_unpinned;
> > +       int err;
> >
> >         /* We don't use vmf->pgoff since that has the fake offset */
> >         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
> >
> >         dma_resv_lock(shmem->base.resv, NULL);
> >
> > -       if (page_offset >= num_pages ||
> > -           WARN_ON_ONCE(!shmem->pages) ||
> > -           shmem->madv < 0) {
> > +       /* Sanity-check that we have the pages pointer when it should present */
> > +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> > +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> > +
> > +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
> >                 ret = VM_FAULT_SIGBUS;
> >         } else {
> > +               err = drm_gem_shmem_swap_in(shmem);
> > +               if (err) {
> > +                       ret = VM_FAULT_OOM;
> > +                       goto unlock;
> > +               }
> > +
> >                 page = shmem->pages[page_offset];
> >
> >                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
> >         }
> >
> > +unlock:
> >         dma_resv_unlock(shmem->base.resv);
> >
> >         return ret;
> > @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
> >  {
> >         struct drm_gem_object *obj = vma->vm_private_data;
> >         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> > -       int ret;
> >
> >         WARN_ON(shmem->base.import_attach);
> >
> >         dma_resv_lock(shmem->base.resv, NULL);
> > -       ret = drm_gem_shmem_get_pages(shmem);
> > -       WARN_ON_ONCE(ret != 0);
> > +
> > +       if (drm_gem_shmem_get_pages(shmem))
> > +               shmem->pages_use_count++;
> > +
> > +       drm_gem_shmem_update_pages_state(shmem);
> >         dma_resv_unlock(shmem->base.resv);
> >
> >         drm_gem_vm_open(vma);
> > @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
> >  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> >                               struct drm_printer *p, unsigned int indent)
> >  {
> > +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> > +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
> >         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
> >
> >         if (shmem->base.import_attach)
> > @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> >                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
> >                                   shmem->vmap_use_count);
> >
> > +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
> >         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> > +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_print_info);
> >
> > @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
> >
> >         shmem->sgt = sgt;
> >
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> >         dma_resv_unlock(shmem->base.resv);
> >
> >         return sgt;
> > @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
> >  }
> >  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
> >
> > +static struct drm_gem_shmem_shrinker *
> > +to_drm_shrinker(struct shrinker *shrinker)
> > +{
> > +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> > +}
> > +
> > +static unsigned long
> > +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> > +                                    struct shrink_control *sc)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > +       struct drm_gem_shmem_object *shmem;
> > +       unsigned long count = 0;
> > +
> > +       if (!mutex_trylock(&gem_shrinker->lock))
> > +               return 0;
> > +
> > +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> > +               count += shmem->base.size;
> > +
> > +               if (count >= SHRINK_EMPTY)
> > +                       break;
> > +       }
> > +
> > +       mutex_unlock(&gem_shrinker->lock);
> > +
> > +       if (count >= SHRINK_EMPTY)
> > +               return SHRINK_EMPTY - 1;
> > +
> > +       return count ?: SHRINK_EMPTY;
> > +}
> > +
> > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> > +{
> > +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> > +       WARN_ON(shmem->evicted);
> > +
> > +       drm_gem_shmem_unpin_pages(shmem);
> > +
> > +       shmem->evicted = true;
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> > +
> > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +
> > +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > +
> > +       drm_gem_shmem_unpin_pages(shmem);
> > +       drm_gem_free_mmap_offset(obj);
> > +
> > +       /* Our goal here is to return as much of the memory as
> > +        * is possible back to the system as we are called from OOM.
> > +        * To do this we must instruct the shmfs to drop all of its
> > +        * backing pages, *now*.
> > +        */
> > +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > +
> > +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > +
> > +       shmem->madv = -1;
> > +       shmem->evicted = false;
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> > +
> > +static unsigned long
> > +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> > +                                       unsigned long nr_to_scan,
> > +                                       bool *lock_contention,
> > +                                       bool evict)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > +       struct drm_gem_shmem_object *shmem;
> > +       struct list_head still_in_list;
> > +       struct drm_gem_object *obj;
> > +       unsigned long freed = 0;
> > +       size_t page_count;
> > +       int err;
> > +
> > +       INIT_LIST_HEAD(&still_in_list);
> > +
> > +       mutex_lock(&gem_shrinker->lock);
> > +
> > +       while (freed < nr_to_scan) {
> > +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> > +                                                typeof(*shmem), madv_list);
> > +               if (!shmem)
> > +                       break;
> > +
> > +               obj = &shmem->base;
> > +               page_count = obj->size >> PAGE_SHIFT;
> > +               list_move_tail(&shmem->madv_list, &still_in_list);
> > +
> > +               if (evict) {
> > +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> > +                           get_nr_swap_pages() < page_count)
> > +                               continue;
> > +               } else {
> > +                       if (!drm_gem_shmem_is_purgeable(shmem))
> > +                               continue;
> > +               }
> > +
> > +               /*
> > +                * If it's in the process of being freed, gem_object->free()
> > +                * may be blocked on lock waiting to remove it.  So just
> > +                * skip it.
> > +                */
> > +               if (!kref_get_unless_zero(&obj->refcount))
> > +                       continue;
> > +
> > +               mutex_unlock(&gem_shrinker->lock);
> > +
> > +               /* prevent racing with job-submission code paths */
> > +               if (!dma_resv_trylock(obj->resv)) {
> > +                       *lock_contention |= true;
> > +                       goto shrinker_lock;
> > +               }
> > +
> > +               /* prevent racing with the dma-buf importing/exporting */
> > +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > +                       *lock_contention |= true;
> > +                       goto resv_unlock;
> > +               }
> > +
> > +               /* check whether h/w uses this object */
> > +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> > +                       goto object_name_unlock;
> > +
> > +               /* re-check whether eviction status hasn't changed */
> > +               if (!drm_gem_shmem_is_evictable(shmem) &&
> > +                   !drm_gem_shmem_is_purgeable(shmem))
> > +                       goto object_name_unlock;
> > +
> > +               err = shmem->evict(shmem);
> > +               if (!err)
> > +                       freed += obj->size >> PAGE_SHIFT;
> > +
> > +object_name_unlock:
> > +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> > +resv_unlock:
> > +               dma_resv_unlock(obj->resv);
> > +shrinker_lock:
> > +               drm_gem_object_put(&shmem->base);
> > +               mutex_lock(&gem_shrinker->lock);
> > +       }
> > +
> > +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> > +
> > +       mutex_unlock(&gem_shrinker->lock);
> > +
> > +       return freed;
> > +}
> > +
> > +static unsigned long
> > +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> > +                                   struct shrink_control *sc)
> > +{
> > +       unsigned long nr_to_scan = sc->nr_to_scan;
> > +       bool lock_contention = false;
> > +       unsigned long freed;
> > +
> > +       /* purge as many objects as we can */
> > +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> > +                                                       &lock_contention, false);
> > +
> > +       /* evict as many objects as we can */
> > +       if (freed < nr_to_scan)
> > +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> > +                                                                nr_to_scan - freed,
> > +                                                                &lock_contention,
> > +                                                                true);
> > +
> > +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> > +}
> > +
> > +/**
> > + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> > + * @dev: DRM device
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker;
> > +       int err;
> > +
> > +       if (WARN_ON(dev->shmem_shrinker))
> > +               return -EBUSY;
> > +
> > +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> > +       if (!gem_shrinker)
> > +               return -ENOMEM;
> > +
> > +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> > +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> > +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> > +       gem_shrinker->dev = dev;
> > +
> > +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> > +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> > +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> > +       mutex_init(&gem_shrinker->lock);
> > +
> > +       dev->shmem_shrinker = gem_shrinker;
> > +
> > +       err = register_shrinker(&gem_shrinker->base);
> > +       if (err) {
> > +               dev->shmem_shrinker = NULL;
> > +               kfree(gem_shrinker);
> > +               return err;
> > +       }
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> > +
> > +/**
> > + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> > + * @dev: DRM device
> > + */
> > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> > +
> > +       if (gem_shrinker) {
> > +               unregister_shrinker(&gem_shrinker->base);
> > +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> > +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> > +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> > +               mutex_destroy(&gem_shrinker->lock);
> > +               dev->shmem_shrinker = NULL;
> > +               kfree(gem_shrinker);
> > +       }
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> > +
> >  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
> >  MODULE_IMPORT_NS(DMA_BUF);
> >  MODULE_LICENSE("GPL v2");
> > diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > index a4bedfeb2ec4..7cc32556f908 100644
> > --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > @@ -15,6 +15,13 @@
> >  #include "panfrost_gem.h"
> >  #include "panfrost_mmu.h"
> >
> > +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       return (shmem->madv > 0) &&
> > +               !shmem->pages_pin_count && shmem->sgt &&
> > +               !shmem->base.dma_buf && !shmem->base.import_attach;
> > +}
> > +
> >  static unsigned long
> >  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
> >  {
> > @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
> >                 return 0;
> >
> >         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> > -               if (drm_gem_shmem_is_purgeable(shmem))
> > +               if (panfrost_gem_shmem_is_purgeable(shmem))
> >                         count += shmem->base.size >> PAGE_SHIFT;
> >         }
> >
> > diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > index b2d93cb12ebf..81bacc7e1873 100644
> > --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> > +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > @@ -89,6 +89,7 @@ struct virtio_gpu_object {
> >         uint32_t hw_res_handle;
> >         bool dumb;
> >         bool created;
> > +       bool detached;
> >         bool host3d_blob, guest_blob;
> >         uint32_t blob_mem, blob_flags;
> >
> > @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
> >
> >  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
> >
> > +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> > +
> >  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
> >                                uint32_t *resid);
> >  /* virtgpu_prime.c */
> > diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> > index 9923c7a6885e..929546cad894 100644
> > --- a/include/drm/drm_device.h
> > +++ b/include/drm/drm_device.h
> > @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
> >  struct drm_vma_offset_manager;
> >  struct drm_vram_mm;
> >  struct drm_fb_helper;
> > +struct drm_gem_shmem_shrinker;
> >
> >  struct inode;
> >
> > @@ -277,6 +278,9 @@ struct drm_device {
> >         /** @vram_mm: VRAM MM memory manager */
> >         struct drm_vram_mm *vram_mm;
> >
> > +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> > +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> > +
> >         /**
> >          * @switch_power_state:
> >          *
> > diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> > index 9a8983ee8abe..62c640678a91 100644
> > --- a/include/drm/drm_gem_shmem_helper.h
> > +++ b/include/drm/drm_gem_shmem_helper.h
> > @@ -6,6 +6,7 @@
> >  #include <linux/fs.h>
> >  #include <linux/mm.h>
> >  #include <linux/mutex.h>
> > +#include <linux/shrinker.h>
> >
> >  #include <drm/drm_file.h>
> >  #include <drm/drm_gem.h>
> > @@ -15,6 +16,7 @@
> >  struct dma_buf_attachment;
> >  struct drm_mode_create_dumb;
> >  struct drm_printer;
> > +struct drm_device;
> >  struct sg_table;
> >
> >  /**
> > @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
> >          */
> >         unsigned int pages_use_count;
> >
> > +       /**
> > +        * @pages_pin_count:
> > +        *
> > +        * Reference count on the pinned pages table.
> > +        * The pages can be evicted by memory shrinker
> > +        * when the count reaches zero.
> > +        */
> > +       unsigned int pages_pin_count;
> > +
> >         /**
> >          * @madv: State for madvise
> >          *
> >          * 0 is active/inuse.
> > +        * 1 is not-needed/can-be-purged
> >          * A negative value is the object is purged.
> > -        * Positive values are driver specific and not used by the helpers.
> >          */
> >         int madv;
> >
> > @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
> >          * @map_wc: map object write-combined (instead of using shmem defaults).
> >          */
> >         bool map_wc;
> > +
> > +       /**
> > +        * @eviction_enabled:
> > +        *
> > +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> > +        * Used internally by memory shrinker.
> > +        */
> > +       bool eviction_enabled;
> > +
> > +       /**
> > +        * @purge_enabled:
> > +        *
> > +        * The shmem pages can be purged only if @purge_enabled is set to true.
> > +        * Used internally by memory shrinker.
> > +        */
> > +       bool purge_enabled;
> > +
> > +       /**
> > +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> > +        * Used internally by memory shrinker.
> > +        */
> > +       bool evicted;
> > +
> > +       /**
> > +        * @evict:
> > +        *
> > +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> > +        * GEM's DMA reservation is kept locked by the shrinker. This is
> > +        * optional callback that should be specified by drivers.
> > +        *
> > +        * Returns 0 on success, or -errno on error.
> > +        */
> > +       int (*evict)(struct drm_gem_shmem_object *shmem);
> >  };
> >
> >  #define to_drm_gem_shmem_obj(obj) \
> > @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
> >
> >  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
> >
> > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> > +
> >  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> >  {
> > -       return (shmem->madv > 0) &&
> > -               !shmem->vmap_use_count && shmem->sgt &&
> > -               !shmem->base.dma_buf && !shmem->base.import_attach;
> > +       return (shmem->madv > 0) && shmem->evict &&
> > +               shmem->purge_enabled && shmem->pages_use_count &&
> > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
> >  }
> >
> > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> > +
> > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> >
> >  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
> >  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> > @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
> >         return drm_gem_shmem_mmap(shmem, vma);
> >  }
> >
> > +/**
> > + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> > + */
> > +struct drm_gem_shmem_shrinker {
> > +       /** @base: Shrinker for purging shmem GEM objects */
> > +       struct shrinker base;
> > +
> > +       /** @lock: Protects @lru_* */
> > +       struct mutex lock;
> > +
> > +       /** @lru_pinned: List of pinned shmem GEM objects */
> > +       struct list_head lru_pinned;
> > +
> > +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> > +       struct list_head lru_evictable;
> > +
> > +       /** @lru_evicted: List of evicted shmem GEM objects */
> > +       struct list_head lru_evicted;
> > +
> > +       /** @dev: DRM device that uses this shrinker */
> > +       struct drm_device *dev;
> > +};
> > +
> > +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> > +
> >  /*
> >   * Driver ops
> >   */
> > --
> > 2.35.3
> >
>
>
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

On Sun, Jun 5, 2022 at 9:47 AM Daniel Vetter <daniel@ffwll.ch> wrote:
>
> On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
> >
> > Introduce a common DRM SHMEM shrinker framework that allows to reduce
> > code duplication among DRM drivers by replacing theirs custom shrinker
> > implementations with the generic shrinker.
> >
> > In order to start using DRM SHMEM shrinker drivers should:
> >
> > 1. Implement new evict() shmem object callback.
> > 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> > 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
> >    activate shrinking of shmem GEMs.
> >
> > This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> > Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
> >
> > Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> > Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>
> So I guess I get a price for being blind since forever, because this
> thing existed since at least 2013. I just stumbled over
> llist_lru.[hc], a purpose built list helper for shrinkers. I think we
> should try to adopt that so that our gpu shrinkers look more like
> shrinkers for everything else.

followup from a bit of irc discussion w/ danvet about list_lru:

* It seems to be missing a way to bail out of iteration before
  nr_to_scan is hit.. which is going to be inconvenient if you
  want to allow active bos on the LRU but bail scanning once
  you encounter the first one.

* Not sure if the numa node awareness is super useful for GEM
  bos

First issue is perhaps not too hard to fix.  But maybe a better
idea is a drm_gem_lru helper type thing which is more tailored
to GEM buffers?

BR,
-R

> Apologies for this, since I fear this might cause a bit of churn.
> Hopefully it's all contained to the list manipulation code in shmem
> helpers, I don't think this should leak any further.
> -Daniel
>
> > ---
> >  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
> >  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
> >  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
> >  include/drm/drm_device.h                      |   4 +
> >  include/drm/drm_gem_shmem_helper.h            |  87 ++-
> >  5 files changed, 594 insertions(+), 49 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > index 555fe212bd98..4cd0b5913492 100644
> > --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> > +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
> >  }
> >  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
> >
> > +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       return (shmem->madv >= 0) && shmem->evict &&
> > +               shmem->eviction_enabled && shmem->pages_use_count &&
> > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> > +}
> > +
> > +static void
> > +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> > +
> > +       dma_resv_assert_held(shmem->base.resv);
> > +
> > +       if (!gem_shrinker || obj->import_attach)
> > +               return;
> > +
> > +       mutex_lock(&gem_shrinker->lock);
> > +
> > +       if (drm_gem_shmem_is_evictable(shmem) ||
> > +           drm_gem_shmem_is_purgeable(shmem))
> > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> > +       else if (shmem->madv < 0)
> > +               list_del_init(&shmem->madv_list);
> > +       else if (shmem->evicted)
> > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> > +       else if (!shmem->pages)
> > +               list_del_init(&shmem->madv_list);
> > +       else
> > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> > +
> > +       mutex_unlock(&gem_shrinker->lock);
> > +}
> > +
> >  /**
> >   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
> >   * @shmem: shmem GEM object to free
> > @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> >         } else {
> >                 dma_resv_lock(shmem->base.resv, NULL);
> >
> > +               /* take out shmem GEM object from the memory shrinker */
> > +               drm_gem_shmem_madvise(shmem, -1);
> > +
> >                 WARN_ON(shmem->vmap_use_count);
> >
> >                 if (shmem->sgt) {
> > @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> >                         sg_free_table(shmem->sgt);
> >                         kfree(shmem->sgt);
> >                 }
> > -               if (shmem->pages)
> > +               if (shmem->pages_use_count)
> >                         drm_gem_shmem_put_pages(shmem);
> >
> >                 WARN_ON(shmem->pages_use_count);
> > @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> >  }
> >  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
> >
> > -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > +/**
> > + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> > + * @shmem: shmem GEM object
> > + *
> > + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       dma_resv_lock(shmem->base.resv, NULL);
> > +
> > +       if (shmem->madv < 0)
> > +               return -ENOMEM;
> > +
> > +       shmem->eviction_enabled = true;
> > +
> > +       dma_resv_unlock(shmem->base.resv);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> > +
> > +/**
> > + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> > + * @shmem: shmem GEM object
> > + *
> > + * Tell memory shrinker that this GEM can be purged. Initially purging is
> > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       dma_resv_lock(shmem->base.resv, NULL);
> > +
> > +       if (shmem->madv < 0)
> > +               return -ENOMEM;
> > +
> > +       shmem->purge_enabled = true;
> > +
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       dma_resv_unlock(shmem->base.resv);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> > +
> > +static int
> > +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
> >  {
> >         struct drm_gem_object *obj = &shmem->base;
> >         struct page **pages;
> >
> > -       if (shmem->pages_use_count++ > 0)
> > +       dma_resv_assert_held(shmem->base.resv);
> > +
> > +       if (shmem->madv < 0) {
> > +               WARN_ON(shmem->pages);
> > +               return -ENOMEM;
> > +       }
> > +
> > +       if (shmem->pages) {
> > +               WARN_ON(!shmem->evicted);
> >                 return 0;
> > +       }
> > +
> > +       if (WARN_ON(!shmem->pages_use_count))
> > +               return -EINVAL;
> >
> >         pages = drm_gem_get_pages(obj);
> >         if (IS_ERR(pages)) {
> >                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> > -               shmem->pages_use_count = 0;
> >                 return PTR_ERR(pages);
> >         }
> >
> > @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> >         return 0;
> >  }
> >
> > +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > +{
> > +       int err;
> > +
> > +       dma_resv_assert_held(shmem->base.resv);
> > +
> > +       if (shmem->madv < 0)
> > +               return -ENOMEM;
> > +
> > +       if (shmem->pages_use_count++ > 0) {
> > +               err = drm_gem_shmem_swap_in(shmem);
> > +               if (err)
> > +                       goto err_zero_use;
> > +
> > +               return 0;
> > +       }
> > +
> > +       err = drm_gem_shmem_acquire_pages(shmem);
> > +       if (err)
> > +               goto err_zero_use;
> > +
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       return 0;
> > +
> > +err_zero_use:
> > +       shmem->pages_use_count = 0;
> > +
> > +       return err;
> > +}
> > +
> > +static void
> > +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +
> > +       if (!shmem->pages) {
> > +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> > +               return;
> > +       }
> > +
> > +#ifdef CONFIG_X86
> > +       if (shmem->map_wc)
> > +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > +#endif
> > +
> > +       drm_gem_put_pages(obj, shmem->pages,
> > +                         shmem->pages_mark_dirty_on_put,
> > +                         shmem->pages_mark_accessed_on_put);
> > +       shmem->pages = NULL;
> > +}
> > +
> >  /*
> >   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
> >   * @shmem: shmem GEM object
> > @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> >   */
> >  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> >  {
> > -       struct drm_gem_object *obj = &shmem->base;
> > -
> >         dma_resv_assert_held(shmem->base.resv);
> >
> >         if (WARN_ON_ONCE(!shmem->pages_use_count))
> > @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> >         if (--shmem->pages_use_count > 0)
> >                 return;
> >
> > -#ifdef CONFIG_X86
> > -       if (shmem->map_wc)
> > -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > -#endif
> > +       drm_gem_shmem_release_pages(shmem);
> >
> > -       drm_gem_put_pages(obj, shmem->pages,
> > -                         shmem->pages_mark_dirty_on_put,
> > -                         shmem->pages_mark_accessed_on_put);
> > -       shmem->pages = NULL;
> > +       drm_gem_shmem_update_pages_state(shmem);
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> >
> > @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> >   */
> >  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
> >  {
> > +       int ret;
> > +
> >         dma_resv_assert_held(shmem->base.resv);
> >
> >         WARN_ON(shmem->base.import_attach);
> >
> > -       return drm_gem_shmem_get_pages(shmem);
> > +       ret = drm_gem_shmem_get_pages(shmem);
> > +       if (!ret)
> > +               shmem->pages_pin_count++;
> > +
> > +       return ret;
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_pin);
> >
> > @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
> >         WARN_ON(shmem->base.import_attach);
> >
> >         drm_gem_shmem_put_pages(shmem);
> > +
> > +       shmem->pages_pin_count--;
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_unpin);
> >
> > @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> >                         return 0;
> >                 }
> >
> > -               ret = drm_gem_shmem_get_pages(shmem);
> > +               ret = drm_gem_shmem_pin(shmem);
> >                 if (ret)
> >                         goto err_zero_use;
> >
> > @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> >
> >  err_put_pages:
> >         if (!obj->import_attach)
> > -               drm_gem_shmem_put_pages(shmem);
> > +               drm_gem_shmem_unpin(shmem);
> >  err_zero_use:
> >         shmem->vmap_use_count = 0;
> >
> > @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
> >                         return;
> >
> >                 vunmap(shmem->vaddr);
> > -               drm_gem_shmem_put_pages(shmem);
> > +               drm_gem_shmem_unpin(shmem);
> >         }
> >
> >         shmem->vaddr = NULL;
> > @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
> >
> >         madv = shmem->madv;
> >
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> >         return (madv >= 0);
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_madvise);
> >
> > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > +/**
> > + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> > + *                           hardware access to the memory.
> > + * @shmem: shmem GEM object
> > + *
> > + * This function moves shmem GEM back to memory if it was previously evicted
> > + * by the memory shrinker. The GEM is ready to use on success.
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
> >  {
> >         struct drm_gem_object *obj = &shmem->base;
> > -       struct drm_device *dev = obj->dev;
> > +       struct sg_table *sgt;
> > +       int err;
> >
> >         dma_resv_assert_held(shmem->base.resv);
> >
> > -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > +       if (shmem->evicted) {
> > +               err = drm_gem_shmem_acquire_pages(shmem);
> > +               if (err)
> > +                       return err;
> > +
> > +               sgt = drm_gem_shmem_get_sg_table(shmem);
> > +               if (IS_ERR(sgt))
> > +                       return PTR_ERR(sgt);
> > +
> > +               err = dma_map_sgtable(obj->dev->dev, sgt,
> > +                                     DMA_BIDIRECTIONAL, 0);
> > +               if (err) {
> > +                       sg_free_table(sgt);
> > +                       kfree(sgt);
> > +                       return err;
> > +               }
> >
> > -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > -       sg_free_table(shmem->sgt);
> > -       kfree(shmem->sgt);
> > -       shmem->sgt = NULL;
> > +               shmem->sgt = sgt;
> > +               shmem->evicted = false;
> >
> > -       drm_gem_shmem_put_pages(shmem);
> > +               drm_gem_shmem_update_pages_state(shmem);
> > +       }
> >
> > -       shmem->madv = -1;
> > +       if (!shmem->pages)
> > +               return -ENOMEM;
> >
> > -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > -       drm_gem_free_mmap_offset(obj);
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
> >
> > -       /* Our goal here is to return as much of the memory as
> > -        * is possible back to the system as we are called from OOM.
> > -        * To do this we must instruct the shmfs to drop all of its
> > -        * backing pages, *now*.
> > -        */
> > -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +       struct drm_device *dev = obj->dev;
> >
> > -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > +       if (shmem->evicted)
> > +               return;
> > +
> > +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > +       drm_gem_shmem_release_pages(shmem);
> > +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > +
> > +       sg_free_table(shmem->sgt);
> > +       kfree(shmem->sgt);
> > +       shmem->sgt = NULL;
> >  }
> > -EXPORT_SYMBOL(drm_gem_shmem_purge);
> >
> >  /**
> >   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> > @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
> >         vm_fault_t ret;
> >         struct page *page;
> >         pgoff_t page_offset;
> > +       bool pages_unpinned;
> > +       int err;
> >
> >         /* We don't use vmf->pgoff since that has the fake offset */
> >         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
> >
> >         dma_resv_lock(shmem->base.resv, NULL);
> >
> > -       if (page_offset >= num_pages ||
> > -           WARN_ON_ONCE(!shmem->pages) ||
> > -           shmem->madv < 0) {
> > +       /* Sanity-check that we have the pages pointer when it should present */
> > +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> > +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> > +
> > +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
> >                 ret = VM_FAULT_SIGBUS;
> >         } else {
> > +               err = drm_gem_shmem_swap_in(shmem);
> > +               if (err) {
> > +                       ret = VM_FAULT_OOM;
> > +                       goto unlock;
> > +               }
> > +
> >                 page = shmem->pages[page_offset];
> >
> >                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
> >         }
> >
> > +unlock:
> >         dma_resv_unlock(shmem->base.resv);
> >
> >         return ret;
> > @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
> >  {
> >         struct drm_gem_object *obj = vma->vm_private_data;
> >         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> > -       int ret;
> >
> >         WARN_ON(shmem->base.import_attach);
> >
> >         dma_resv_lock(shmem->base.resv, NULL);
> > -       ret = drm_gem_shmem_get_pages(shmem);
> > -       WARN_ON_ONCE(ret != 0);
> > +
> > +       if (drm_gem_shmem_get_pages(shmem))
> > +               shmem->pages_use_count++;
> > +
> > +       drm_gem_shmem_update_pages_state(shmem);
> >         dma_resv_unlock(shmem->base.resv);
> >
> >         drm_gem_vm_open(vma);
> > @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
> >  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> >                               struct drm_printer *p, unsigned int indent)
> >  {
> > +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> > +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
> >         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
> >
> >         if (shmem->base.import_attach)
> > @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> >                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
> >                                   shmem->vmap_use_count);
> >
> > +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
> >         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> > +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_print_info);
> >
> > @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
> >
> >         shmem->sgt = sgt;
> >
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> >         dma_resv_unlock(shmem->base.resv);
> >
> >         return sgt;
> > @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
> >  }
> >  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
> >
> > +static struct drm_gem_shmem_shrinker *
> > +to_drm_shrinker(struct shrinker *shrinker)
> > +{
> > +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> > +}
> > +
> > +static unsigned long
> > +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> > +                                    struct shrink_control *sc)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > +       struct drm_gem_shmem_object *shmem;
> > +       unsigned long count = 0;
> > +
> > +       if (!mutex_trylock(&gem_shrinker->lock))
> > +               return 0;
> > +
> > +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> > +               count += shmem->base.size;
> > +
> > +               if (count >= SHRINK_EMPTY)
> > +                       break;
> > +       }
> > +
> > +       mutex_unlock(&gem_shrinker->lock);
> > +
> > +       if (count >= SHRINK_EMPTY)
> > +               return SHRINK_EMPTY - 1;
> > +
> > +       return count ?: SHRINK_EMPTY;
> > +}
> > +
> > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> > +{
> > +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> > +       WARN_ON(shmem->evicted);
> > +
> > +       drm_gem_shmem_unpin_pages(shmem);
> > +
> > +       shmem->evicted = true;
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> > +
> > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +
> > +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > +
> > +       drm_gem_shmem_unpin_pages(shmem);
> > +       drm_gem_free_mmap_offset(obj);
> > +
> > +       /* Our goal here is to return as much of the memory as
> > +        * is possible back to the system as we are called from OOM.
> > +        * To do this we must instruct the shmfs to drop all of its
> > +        * backing pages, *now*.
> > +        */
> > +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > +
> > +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > +
> > +       shmem->madv = -1;
> > +       shmem->evicted = false;
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> > +
> > +static unsigned long
> > +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> > +                                       unsigned long nr_to_scan,
> > +                                       bool *lock_contention,
> > +                                       bool evict)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > +       struct drm_gem_shmem_object *shmem;
> > +       struct list_head still_in_list;
> > +       struct drm_gem_object *obj;
> > +       unsigned long freed = 0;
> > +       size_t page_count;
> > +       int err;
> > +
> > +       INIT_LIST_HEAD(&still_in_list);
> > +
> > +       mutex_lock(&gem_shrinker->lock);
> > +
> > +       while (freed < nr_to_scan) {
> > +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> > +                                                typeof(*shmem), madv_list);
> > +               if (!shmem)
> > +                       break;
> > +
> > +               obj = &shmem->base;
> > +               page_count = obj->size >> PAGE_SHIFT;
> > +               list_move_tail(&shmem->madv_list, &still_in_list);
> > +
> > +               if (evict) {
> > +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> > +                           get_nr_swap_pages() < page_count)
> > +                               continue;
> > +               } else {
> > +                       if (!drm_gem_shmem_is_purgeable(shmem))
> > +                               continue;
> > +               }
> > +
> > +               /*
> > +                * If it's in the process of being freed, gem_object->free()
> > +                * may be blocked on lock waiting to remove it.  So just
> > +                * skip it.
> > +                */
> > +               if (!kref_get_unless_zero(&obj->refcount))
> > +                       continue;
> > +
> > +               mutex_unlock(&gem_shrinker->lock);
> > +
> > +               /* prevent racing with job-submission code paths */
> > +               if (!dma_resv_trylock(obj->resv)) {
> > +                       *lock_contention |= true;
> > +                       goto shrinker_lock;
> > +               }
> > +
> > +               /* prevent racing with the dma-buf importing/exporting */
> > +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > +                       *lock_contention |= true;
> > +                       goto resv_unlock;
> > +               }
> > +
> > +               /* check whether h/w uses this object */
> > +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> > +                       goto object_name_unlock;
> > +
> > +               /* re-check whether eviction status hasn't changed */
> > +               if (!drm_gem_shmem_is_evictable(shmem) &&
> > +                   !drm_gem_shmem_is_purgeable(shmem))
> > +                       goto object_name_unlock;
> > +
> > +               err = shmem->evict(shmem);
> > +               if (!err)
> > +                       freed += obj->size >> PAGE_SHIFT;
> > +
> > +object_name_unlock:
> > +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> > +resv_unlock:
> > +               dma_resv_unlock(obj->resv);
> > +shrinker_lock:
> > +               drm_gem_object_put(&shmem->base);
> > +               mutex_lock(&gem_shrinker->lock);
> > +       }
> > +
> > +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> > +
> > +       mutex_unlock(&gem_shrinker->lock);
> > +
> > +       return freed;
> > +}
> > +
> > +static unsigned long
> > +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> > +                                   struct shrink_control *sc)
> > +{
> > +       unsigned long nr_to_scan = sc->nr_to_scan;
> > +       bool lock_contention = false;
> > +       unsigned long freed;
> > +
> > +       /* purge as many objects as we can */
> > +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> > +                                                       &lock_contention, false);
> > +
> > +       /* evict as many objects as we can */
> > +       if (freed < nr_to_scan)
> > +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> > +                                                                nr_to_scan - freed,
> > +                                                                &lock_contention,
> > +                                                                true);
> > +
> > +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> > +}
> > +
> > +/**
> > + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> > + * @dev: DRM device
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker;
> > +       int err;
> > +
> > +       if (WARN_ON(dev->shmem_shrinker))
> > +               return -EBUSY;
> > +
> > +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> > +       if (!gem_shrinker)
> > +               return -ENOMEM;
> > +
> > +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> > +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> > +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> > +       gem_shrinker->dev = dev;
> > +
> > +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> > +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> > +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> > +       mutex_init(&gem_shrinker->lock);
> > +
> > +       dev->shmem_shrinker = gem_shrinker;
> > +
> > +       err = register_shrinker(&gem_shrinker->base);
> > +       if (err) {
> > +               dev->shmem_shrinker = NULL;
> > +               kfree(gem_shrinker);
> > +               return err;
> > +       }
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> > +
> > +/**
> > + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> > + * @dev: DRM device
> > + */
> > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> > +
> > +       if (gem_shrinker) {
> > +               unregister_shrinker(&gem_shrinker->base);
> > +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> > +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> > +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> > +               mutex_destroy(&gem_shrinker->lock);
> > +               dev->shmem_shrinker = NULL;
> > +               kfree(gem_shrinker);
> > +       }
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> > +
> >  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
> >  MODULE_IMPORT_NS(DMA_BUF);
> >  MODULE_LICENSE("GPL v2");
> > diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > index a4bedfeb2ec4..7cc32556f908 100644
> > --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > @@ -15,6 +15,13 @@
> >  #include "panfrost_gem.h"
> >  #include "panfrost_mmu.h"
> >
> > +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       return (shmem->madv > 0) &&
> > +               !shmem->pages_pin_count && shmem->sgt &&
> > +               !shmem->base.dma_buf && !shmem->base.import_attach;
> > +}
> > +
> >  static unsigned long
> >  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
> >  {
> > @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
> >                 return 0;
> >
> >         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> > -               if (drm_gem_shmem_is_purgeable(shmem))
> > +               if (panfrost_gem_shmem_is_purgeable(shmem))
> >                         count += shmem->base.size >> PAGE_SHIFT;
> >         }
> >
> > diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > index b2d93cb12ebf..81bacc7e1873 100644
> > --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> > +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > @@ -89,6 +89,7 @@ struct virtio_gpu_object {
> >         uint32_t hw_res_handle;
> >         bool dumb;
> >         bool created;
> > +       bool detached;
> >         bool host3d_blob, guest_blob;
> >         uint32_t blob_mem, blob_flags;
> >
> > @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
> >
> >  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
> >
> > +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> > +
> >  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
> >                                uint32_t *resid);
> >  /* virtgpu_prime.c */
> > diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> > index 9923c7a6885e..929546cad894 100644
> > --- a/include/drm/drm_device.h
> > +++ b/include/drm/drm_device.h
> > @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
> >  struct drm_vma_offset_manager;
> >  struct drm_vram_mm;
> >  struct drm_fb_helper;
> > +struct drm_gem_shmem_shrinker;
> >
> >  struct inode;
> >
> > @@ -277,6 +278,9 @@ struct drm_device {
> >         /** @vram_mm: VRAM MM memory manager */
> >         struct drm_vram_mm *vram_mm;
> >
> > +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> > +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> > +
> >         /**
> >          * @switch_power_state:
> >          *
> > diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> > index 9a8983ee8abe..62c640678a91 100644
> > --- a/include/drm/drm_gem_shmem_helper.h
> > +++ b/include/drm/drm_gem_shmem_helper.h
> > @@ -6,6 +6,7 @@
> >  #include <linux/fs.h>
> >  #include <linux/mm.h>
> >  #include <linux/mutex.h>
> > +#include <linux/shrinker.h>
> >
> >  #include <drm/drm_file.h>
> >  #include <drm/drm_gem.h>
> > @@ -15,6 +16,7 @@
> >  struct dma_buf_attachment;
> >  struct drm_mode_create_dumb;
> >  struct drm_printer;
> > +struct drm_device;
> >  struct sg_table;
> >
> >  /**
> > @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
> >          */
> >         unsigned int pages_use_count;
> >
> > +       /**
> > +        * @pages_pin_count:
> > +        *
> > +        * Reference count on the pinned pages table.
> > +        * The pages can be evicted by memory shrinker
> > +        * when the count reaches zero.
> > +        */
> > +       unsigned int pages_pin_count;
> > +
> >         /**
> >          * @madv: State for madvise
> >          *
> >          * 0 is active/inuse.
> > +        * 1 is not-needed/can-be-purged
> >          * A negative value is the object is purged.
> > -        * Positive values are driver specific and not used by the helpers.
> >          */
> >         int madv;
> >
> > @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
> >          * @map_wc: map object write-combined (instead of using shmem defaults).
> >          */
> >         bool map_wc;
> > +
> > +       /**
> > +        * @eviction_enabled:
> > +        *
> > +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> > +        * Used internally by memory shrinker.
> > +        */
> > +       bool eviction_enabled;
> > +
> > +       /**
> > +        * @purge_enabled:
> > +        *
> > +        * The shmem pages can be purged only if @purge_enabled is set to true.
> > +        * Used internally by memory shrinker.
> > +        */
> > +       bool purge_enabled;
> > +
> > +       /**
> > +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> > +        * Used internally by memory shrinker.
> > +        */
> > +       bool evicted;
> > +
> > +       /**
> > +        * @evict:
> > +        *
> > +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> > +        * GEM's DMA reservation is kept locked by the shrinker. This is
> > +        * optional callback that should be specified by drivers.
> > +        *
> > +        * Returns 0 on success, or -errno on error.
> > +        */
> > +       int (*evict)(struct drm_gem_shmem_object *shmem);
> >  };
> >
> >  #define to_drm_gem_shmem_obj(obj) \
> > @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
> >
> >  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
> >
> > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> > +
> >  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> >  {
> > -       return (shmem->madv > 0) &&
> > -               !shmem->vmap_use_count && shmem->sgt &&
> > -               !shmem->base.dma_buf && !shmem->base.import_attach;
> > +       return (shmem->madv > 0) && shmem->evict &&
> > +               shmem->purge_enabled && shmem->pages_use_count &&
> > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
> >  }
> >
> > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> > +
> > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> >
> >  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
> >  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> > @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
> >         return drm_gem_shmem_mmap(shmem, vma);
> >  }
> >
> > +/**
> > + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> > + */
> > +struct drm_gem_shmem_shrinker {
> > +       /** @base: Shrinker for purging shmem GEM objects */
> > +       struct shrinker base;
> > +
> > +       /** @lock: Protects @lru_* */
> > +       struct mutex lock;
> > +
> > +       /** @lru_pinned: List of pinned shmem GEM objects */
> > +       struct list_head lru_pinned;
> > +
> > +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> > +       struct list_head lru_evictable;
> > +
> > +       /** @lru_evicted: List of evicted shmem GEM objects */
> > +       struct list_head lru_evicted;
> > +
> > +       /** @dev: DRM device that uses this shrinker */
> > +       struct drm_device *dev;
> > +};
> > +
> > +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> > +
> >  /*
> >   * Driver ops
> >   */
> > --
> > 2.35.3
> >
>
>
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

On Sun, Jun 5, 2022 at 9:47 AM Daniel Vetter <daniel@ffwll.ch> wrote:
>
> On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
> >
> > Introduce a common DRM SHMEM shrinker framework that allows to reduce
> > code duplication among DRM drivers by replacing theirs custom shrinker
> > implementations with the generic shrinker.
> >
> > In order to start using DRM SHMEM shrinker drivers should:
> >
> > 1. Implement new evict() shmem object callback.
> > 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> > 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
> >    activate shrinking of shmem GEMs.
> >
> > This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> > Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
> >
> > Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> > Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>
> So I guess I get a price for being blind since forever, because this
> thing existed since at least 2013. I just stumbled over
> llist_lru.[hc], a purpose built list helper for shrinkers. I think we
> should try to adopt that so that our gpu shrinkers look more like
> shrinkers for everything else.

followup from a bit of irc discussion w/ danvet about list_lru:

* It seems to be missing a way to bail out of iteration before
  nr_to_scan is hit.. which is going to be inconvenient if you
  want to allow active bos on the LRU but bail scanning once
  you encounter the first one.

* Not sure if the numa node awareness is super useful for GEM
  bos

First issue is perhaps not too hard to fix.  But maybe a better
idea is a drm_gem_lru helper type thing which is more tailored
to GEM buffers?

BR,
-R

> Apologies for this, since I fear this might cause a bit of churn.
> Hopefully it's all contained to the list manipulation code in shmem
> helpers, I don't think this should leak any further.
> -Daniel
>
> > ---
> >  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
> >  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
> >  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
> >  include/drm/drm_device.h                      |   4 +
> >  include/drm/drm_gem_shmem_helper.h            |  87 ++-
> >  5 files changed, 594 insertions(+), 49 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > index 555fe212bd98..4cd0b5913492 100644
> > --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> > +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
> >  }
> >  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
> >
> > +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       return (shmem->madv >= 0) && shmem->evict &&
> > +               shmem->eviction_enabled && shmem->pages_use_count &&
> > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> > +}
> > +
> > +static void
> > +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> > +
> > +       dma_resv_assert_held(shmem->base.resv);
> > +
> > +       if (!gem_shrinker || obj->import_attach)
> > +               return;
> > +
> > +       mutex_lock(&gem_shrinker->lock);
> > +
> > +       if (drm_gem_shmem_is_evictable(shmem) ||
> > +           drm_gem_shmem_is_purgeable(shmem))
> > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> > +       else if (shmem->madv < 0)
> > +               list_del_init(&shmem->madv_list);
> > +       else if (shmem->evicted)
> > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> > +       else if (!shmem->pages)
> > +               list_del_init(&shmem->madv_list);
> > +       else
> > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> > +
> > +       mutex_unlock(&gem_shrinker->lock);
> > +}
> > +
> >  /**
> >   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
> >   * @shmem: shmem GEM object to free
> > @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> >         } else {
> >                 dma_resv_lock(shmem->base.resv, NULL);
> >
> > +               /* take out shmem GEM object from the memory shrinker */
> > +               drm_gem_shmem_madvise(shmem, -1);
> > +
> >                 WARN_ON(shmem->vmap_use_count);
> >
> >                 if (shmem->sgt) {
> > @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> >                         sg_free_table(shmem->sgt);
> >                         kfree(shmem->sgt);
> >                 }
> > -               if (shmem->pages)
> > +               if (shmem->pages_use_count)
> >                         drm_gem_shmem_put_pages(shmem);
> >
> >                 WARN_ON(shmem->pages_use_count);
> > @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> >  }
> >  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
> >
> > -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > +/**
> > + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> > + * @shmem: shmem GEM object
> > + *
> > + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       dma_resv_lock(shmem->base.resv, NULL);
> > +
> > +       if (shmem->madv < 0)
> > +               return -ENOMEM;
> > +
> > +       shmem->eviction_enabled = true;
> > +
> > +       dma_resv_unlock(shmem->base.resv);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> > +
> > +/**
> > + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> > + * @shmem: shmem GEM object
> > + *
> > + * Tell memory shrinker that this GEM can be purged. Initially purging is
> > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       dma_resv_lock(shmem->base.resv, NULL);
> > +
> > +       if (shmem->madv < 0)
> > +               return -ENOMEM;
> > +
> > +       shmem->purge_enabled = true;
> > +
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       dma_resv_unlock(shmem->base.resv);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> > +
> > +static int
> > +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
> >  {
> >         struct drm_gem_object *obj = &shmem->base;
> >         struct page **pages;
> >
> > -       if (shmem->pages_use_count++ > 0)
> > +       dma_resv_assert_held(shmem->base.resv);
> > +
> > +       if (shmem->madv < 0) {
> > +               WARN_ON(shmem->pages);
> > +               return -ENOMEM;
> > +       }
> > +
> > +       if (shmem->pages) {
> > +               WARN_ON(!shmem->evicted);
> >                 return 0;
> > +       }
> > +
> > +       if (WARN_ON(!shmem->pages_use_count))
> > +               return -EINVAL;
> >
> >         pages = drm_gem_get_pages(obj);
> >         if (IS_ERR(pages)) {
> >                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> > -               shmem->pages_use_count = 0;
> >                 return PTR_ERR(pages);
> >         }
> >
> > @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> >         return 0;
> >  }
> >
> > +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > +{
> > +       int err;
> > +
> > +       dma_resv_assert_held(shmem->base.resv);
> > +
> > +       if (shmem->madv < 0)
> > +               return -ENOMEM;
> > +
> > +       if (shmem->pages_use_count++ > 0) {
> > +               err = drm_gem_shmem_swap_in(shmem);
> > +               if (err)
> > +                       goto err_zero_use;
> > +
> > +               return 0;
> > +       }
> > +
> > +       err = drm_gem_shmem_acquire_pages(shmem);
> > +       if (err)
> > +               goto err_zero_use;
> > +
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       return 0;
> > +
> > +err_zero_use:
> > +       shmem->pages_use_count = 0;
> > +
> > +       return err;
> > +}
> > +
> > +static void
> > +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +
> > +       if (!shmem->pages) {
> > +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> > +               return;
> > +       }
> > +
> > +#ifdef CONFIG_X86
> > +       if (shmem->map_wc)
> > +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > +#endif
> > +
> > +       drm_gem_put_pages(obj, shmem->pages,
> > +                         shmem->pages_mark_dirty_on_put,
> > +                         shmem->pages_mark_accessed_on_put);
> > +       shmem->pages = NULL;
> > +}
> > +
> >  /*
> >   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
> >   * @shmem: shmem GEM object
> > @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> >   */
> >  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> >  {
> > -       struct drm_gem_object *obj = &shmem->base;
> > -
> >         dma_resv_assert_held(shmem->base.resv);
> >
> >         if (WARN_ON_ONCE(!shmem->pages_use_count))
> > @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> >         if (--shmem->pages_use_count > 0)
> >                 return;
> >
> > -#ifdef CONFIG_X86
> > -       if (shmem->map_wc)
> > -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > -#endif
> > +       drm_gem_shmem_release_pages(shmem);
> >
> > -       drm_gem_put_pages(obj, shmem->pages,
> > -                         shmem->pages_mark_dirty_on_put,
> > -                         shmem->pages_mark_accessed_on_put);
> > -       shmem->pages = NULL;
> > +       drm_gem_shmem_update_pages_state(shmem);
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> >
> > @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> >   */
> >  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
> >  {
> > +       int ret;
> > +
> >         dma_resv_assert_held(shmem->base.resv);
> >
> >         WARN_ON(shmem->base.import_attach);
> >
> > -       return drm_gem_shmem_get_pages(shmem);
> > +       ret = drm_gem_shmem_get_pages(shmem);
> > +       if (!ret)
> > +               shmem->pages_pin_count++;
> > +
> > +       return ret;
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_pin);
> >
> > @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
> >         WARN_ON(shmem->base.import_attach);
> >
> >         drm_gem_shmem_put_pages(shmem);
> > +
> > +       shmem->pages_pin_count--;
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_unpin);
> >
> > @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> >                         return 0;
> >                 }
> >
> > -               ret = drm_gem_shmem_get_pages(shmem);
> > +               ret = drm_gem_shmem_pin(shmem);
> >                 if (ret)
> >                         goto err_zero_use;
> >
> > @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> >
> >  err_put_pages:
> >         if (!obj->import_attach)
> > -               drm_gem_shmem_put_pages(shmem);
> > +               drm_gem_shmem_unpin(shmem);
> >  err_zero_use:
> >         shmem->vmap_use_count = 0;
> >
> > @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
> >                         return;
> >
> >                 vunmap(shmem->vaddr);
> > -               drm_gem_shmem_put_pages(shmem);
> > +               drm_gem_shmem_unpin(shmem);
> >         }
> >
> >         shmem->vaddr = NULL;
> > @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
> >
> >         madv = shmem->madv;
> >
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> >         return (madv >= 0);
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_madvise);
> >
> > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > +/**
> > + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> > + *                           hardware access to the memory.
> > + * @shmem: shmem GEM object
> > + *
> > + * This function moves shmem GEM back to memory if it was previously evicted
> > + * by the memory shrinker. The GEM is ready to use on success.
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
> >  {
> >         struct drm_gem_object *obj = &shmem->base;
> > -       struct drm_device *dev = obj->dev;
> > +       struct sg_table *sgt;
> > +       int err;
> >
> >         dma_resv_assert_held(shmem->base.resv);
> >
> > -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > +       if (shmem->evicted) {
> > +               err = drm_gem_shmem_acquire_pages(shmem);
> > +               if (err)
> > +                       return err;
> > +
> > +               sgt = drm_gem_shmem_get_sg_table(shmem);
> > +               if (IS_ERR(sgt))
> > +                       return PTR_ERR(sgt);
> > +
> > +               err = dma_map_sgtable(obj->dev->dev, sgt,
> > +                                     DMA_BIDIRECTIONAL, 0);
> > +               if (err) {
> > +                       sg_free_table(sgt);
> > +                       kfree(sgt);
> > +                       return err;
> > +               }
> >
> > -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > -       sg_free_table(shmem->sgt);
> > -       kfree(shmem->sgt);
> > -       shmem->sgt = NULL;
> > +               shmem->sgt = sgt;
> > +               shmem->evicted = false;
> >
> > -       drm_gem_shmem_put_pages(shmem);
> > +               drm_gem_shmem_update_pages_state(shmem);
> > +       }
> >
> > -       shmem->madv = -1;
> > +       if (!shmem->pages)
> > +               return -ENOMEM;
> >
> > -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > -       drm_gem_free_mmap_offset(obj);
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
> >
> > -       /* Our goal here is to return as much of the memory as
> > -        * is possible back to the system as we are called from OOM.
> > -        * To do this we must instruct the shmfs to drop all of its
> > -        * backing pages, *now*.
> > -        */
> > -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +       struct drm_device *dev = obj->dev;
> >
> > -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > +       if (shmem->evicted)
> > +               return;
> > +
> > +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > +       drm_gem_shmem_release_pages(shmem);
> > +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > +
> > +       sg_free_table(shmem->sgt);
> > +       kfree(shmem->sgt);
> > +       shmem->sgt = NULL;
> >  }
> > -EXPORT_SYMBOL(drm_gem_shmem_purge);
> >
> >  /**
> >   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> > @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
> >         vm_fault_t ret;
> >         struct page *page;
> >         pgoff_t page_offset;
> > +       bool pages_unpinned;
> > +       int err;
> >
> >         /* We don't use vmf->pgoff since that has the fake offset */
> >         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
> >
> >         dma_resv_lock(shmem->base.resv, NULL);
> >
> > -       if (page_offset >= num_pages ||
> > -           WARN_ON_ONCE(!shmem->pages) ||
> > -           shmem->madv < 0) {
> > +       /* Sanity-check that we have the pages pointer when it should present */
> > +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> > +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> > +
> > +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
> >                 ret = VM_FAULT_SIGBUS;
> >         } else {
> > +               err = drm_gem_shmem_swap_in(shmem);
> > +               if (err) {
> > +                       ret = VM_FAULT_OOM;
> > +                       goto unlock;
> > +               }
> > +
> >                 page = shmem->pages[page_offset];
> >
> >                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
> >         }
> >
> > +unlock:
> >         dma_resv_unlock(shmem->base.resv);
> >
> >         return ret;
> > @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
> >  {
> >         struct drm_gem_object *obj = vma->vm_private_data;
> >         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> > -       int ret;
> >
> >         WARN_ON(shmem->base.import_attach);
> >
> >         dma_resv_lock(shmem->base.resv, NULL);
> > -       ret = drm_gem_shmem_get_pages(shmem);
> > -       WARN_ON_ONCE(ret != 0);
> > +
> > +       if (drm_gem_shmem_get_pages(shmem))
> > +               shmem->pages_use_count++;
> > +
> > +       drm_gem_shmem_update_pages_state(shmem);
> >         dma_resv_unlock(shmem->base.resv);
> >
> >         drm_gem_vm_open(vma);
> > @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
> >  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> >                               struct drm_printer *p, unsigned int indent)
> >  {
> > +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> > +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
> >         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
> >
> >         if (shmem->base.import_attach)
> > @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> >                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
> >                                   shmem->vmap_use_count);
> >
> > +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
> >         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> > +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_print_info);
> >
> > @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
> >
> >         shmem->sgt = sgt;
> >
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> >         dma_resv_unlock(shmem->base.resv);
> >
> >         return sgt;
> > @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
> >  }
> >  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
> >
> > +static struct drm_gem_shmem_shrinker *
> > +to_drm_shrinker(struct shrinker *shrinker)
> > +{
> > +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> > +}
> > +
> > +static unsigned long
> > +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> > +                                    struct shrink_control *sc)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > +       struct drm_gem_shmem_object *shmem;
> > +       unsigned long count = 0;
> > +
> > +       if (!mutex_trylock(&gem_shrinker->lock))
> > +               return 0;
> > +
> > +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> > +               count += shmem->base.size;
> > +
> > +               if (count >= SHRINK_EMPTY)
> > +                       break;
> > +       }
> > +
> > +       mutex_unlock(&gem_shrinker->lock);
> > +
> > +       if (count >= SHRINK_EMPTY)
> > +               return SHRINK_EMPTY - 1;
> > +
> > +       return count ?: SHRINK_EMPTY;
> > +}
> > +
> > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> > +{
> > +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> > +       WARN_ON(shmem->evicted);
> > +
> > +       drm_gem_shmem_unpin_pages(shmem);
> > +
> > +       shmem->evicted = true;
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> > +
> > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +
> > +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > +
> > +       drm_gem_shmem_unpin_pages(shmem);
> > +       drm_gem_free_mmap_offset(obj);
> > +
> > +       /* Our goal here is to return as much of the memory as
> > +        * is possible back to the system as we are called from OOM.
> > +        * To do this we must instruct the shmfs to drop all of its
> > +        * backing pages, *now*.
> > +        */
> > +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > +
> > +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > +
> > +       shmem->madv = -1;
> > +       shmem->evicted = false;
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> > +
> > +static unsigned long
> > +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> > +                                       unsigned long nr_to_scan,
> > +                                       bool *lock_contention,
> > +                                       bool evict)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > +       struct drm_gem_shmem_object *shmem;
> > +       struct list_head still_in_list;
> > +       struct drm_gem_object *obj;
> > +       unsigned long freed = 0;
> > +       size_t page_count;
> > +       int err;
> > +
> > +       INIT_LIST_HEAD(&still_in_list);
> > +
> > +       mutex_lock(&gem_shrinker->lock);
> > +
> > +       while (freed < nr_to_scan) {
> > +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> > +                                                typeof(*shmem), madv_list);
> > +               if (!shmem)
> > +                       break;
> > +
> > +               obj = &shmem->base;
> > +               page_count = obj->size >> PAGE_SHIFT;
> > +               list_move_tail(&shmem->madv_list, &still_in_list);
> > +
> > +               if (evict) {
> > +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> > +                           get_nr_swap_pages() < page_count)
> > +                               continue;
> > +               } else {
> > +                       if (!drm_gem_shmem_is_purgeable(shmem))
> > +                               continue;
> > +               }
> > +
> > +               /*
> > +                * If it's in the process of being freed, gem_object->free()
> > +                * may be blocked on lock waiting to remove it.  So just
> > +                * skip it.
> > +                */
> > +               if (!kref_get_unless_zero(&obj->refcount))
> > +                       continue;
> > +
> > +               mutex_unlock(&gem_shrinker->lock);
> > +
> > +               /* prevent racing with job-submission code paths */
> > +               if (!dma_resv_trylock(obj->resv)) {
> > +                       *lock_contention |= true;
> > +                       goto shrinker_lock;
> > +               }
> > +
> > +               /* prevent racing with the dma-buf importing/exporting */
> > +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > +                       *lock_contention |= true;
> > +                       goto resv_unlock;
> > +               }
> > +
> > +               /* check whether h/w uses this object */
> > +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> > +                       goto object_name_unlock;
> > +
> > +               /* re-check whether eviction status hasn't changed */
> > +               if (!drm_gem_shmem_is_evictable(shmem) &&
> > +                   !drm_gem_shmem_is_purgeable(shmem))
> > +                       goto object_name_unlock;
> > +
> > +               err = shmem->evict(shmem);
> > +               if (!err)
> > +                       freed += obj->size >> PAGE_SHIFT;
> > +
> > +object_name_unlock:
> > +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> > +resv_unlock:
> > +               dma_resv_unlock(obj->resv);
> > +shrinker_lock:
> > +               drm_gem_object_put(&shmem->base);
> > +               mutex_lock(&gem_shrinker->lock);
> > +       }
> > +
> > +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> > +
> > +       mutex_unlock(&gem_shrinker->lock);
> > +
> > +       return freed;
> > +}
> > +
> > +static unsigned long
> > +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> > +                                   struct shrink_control *sc)
> > +{
> > +       unsigned long nr_to_scan = sc->nr_to_scan;
> > +       bool lock_contention = false;
> > +       unsigned long freed;
> > +
> > +       /* purge as many objects as we can */
> > +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> > +                                                       &lock_contention, false);
> > +
> > +       /* evict as many objects as we can */
> > +       if (freed < nr_to_scan)
> > +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> > +                                                                nr_to_scan - freed,
> > +                                                                &lock_contention,
> > +                                                                true);
> > +
> > +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> > +}
> > +
> > +/**
> > + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> > + * @dev: DRM device
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker;
> > +       int err;
> > +
> > +       if (WARN_ON(dev->shmem_shrinker))
> > +               return -EBUSY;
> > +
> > +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> > +       if (!gem_shrinker)
> > +               return -ENOMEM;
> > +
> > +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> > +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> > +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> > +       gem_shrinker->dev = dev;
> > +
> > +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> > +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> > +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> > +       mutex_init(&gem_shrinker->lock);
> > +
> > +       dev->shmem_shrinker = gem_shrinker;
> > +
> > +       err = register_shrinker(&gem_shrinker->base);
> > +       if (err) {
> > +               dev->shmem_shrinker = NULL;
> > +               kfree(gem_shrinker);
> > +               return err;
> > +       }
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> > +
> > +/**
> > + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> > + * @dev: DRM device
> > + */
> > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> > +
> > +       if (gem_shrinker) {
> > +               unregister_shrinker(&gem_shrinker->base);
> > +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> > +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> > +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> > +               mutex_destroy(&gem_shrinker->lock);
> > +               dev->shmem_shrinker = NULL;
> > +               kfree(gem_shrinker);
> > +       }
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> > +
> >  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
> >  MODULE_IMPORT_NS(DMA_BUF);
> >  MODULE_LICENSE("GPL v2");
> > diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > index a4bedfeb2ec4..7cc32556f908 100644
> > --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > @@ -15,6 +15,13 @@
> >  #include "panfrost_gem.h"
> >  #include "panfrost_mmu.h"
> >
> > +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       return (shmem->madv > 0) &&
> > +               !shmem->pages_pin_count && shmem->sgt &&
> > +               !shmem->base.dma_buf && !shmem->base.import_attach;
> > +}
> > +
> >  static unsigned long
> >  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
> >  {
> > @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
> >                 return 0;
> >
> >         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> > -               if (drm_gem_shmem_is_purgeable(shmem))
> > +               if (panfrost_gem_shmem_is_purgeable(shmem))
> >                         count += shmem->base.size >> PAGE_SHIFT;
> >         }
> >
> > diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > index b2d93cb12ebf..81bacc7e1873 100644
> > --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> > +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > @@ -89,6 +89,7 @@ struct virtio_gpu_object {
> >         uint32_t hw_res_handle;
> >         bool dumb;
> >         bool created;
> > +       bool detached;
> >         bool host3d_blob, guest_blob;
> >         uint32_t blob_mem, blob_flags;
> >
> > @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
> >
> >  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
> >
> > +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> > +
> >  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
> >                                uint32_t *resid);
> >  /* virtgpu_prime.c */
> > diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> > index 9923c7a6885e..929546cad894 100644
> > --- a/include/drm/drm_device.h
> > +++ b/include/drm/drm_device.h
> > @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
> >  struct drm_vma_offset_manager;
> >  struct drm_vram_mm;
> >  struct drm_fb_helper;
> > +struct drm_gem_shmem_shrinker;
> >
> >  struct inode;
> >
> > @@ -277,6 +278,9 @@ struct drm_device {
> >         /** @vram_mm: VRAM MM memory manager */
> >         struct drm_vram_mm *vram_mm;
> >
> > +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> > +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> > +
> >         /**
> >          * @switch_power_state:
> >          *
> > diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> > index 9a8983ee8abe..62c640678a91 100644
> > --- a/include/drm/drm_gem_shmem_helper.h
> > +++ b/include/drm/drm_gem_shmem_helper.h
> > @@ -6,6 +6,7 @@
> >  #include <linux/fs.h>
> >  #include <linux/mm.h>
> >  #include <linux/mutex.h>
> > +#include <linux/shrinker.h>
> >
> >  #include <drm/drm_file.h>
> >  #include <drm/drm_gem.h>
> > @@ -15,6 +16,7 @@
> >  struct dma_buf_attachment;
> >  struct drm_mode_create_dumb;
> >  struct drm_printer;
> > +struct drm_device;
> >  struct sg_table;
> >
> >  /**
> > @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
> >          */
> >         unsigned int pages_use_count;
> >
> > +       /**
> > +        * @pages_pin_count:
> > +        *
> > +        * Reference count on the pinned pages table.
> > +        * The pages can be evicted by memory shrinker
> > +        * when the count reaches zero.
> > +        */
> > +       unsigned int pages_pin_count;
> > +
> >         /**
> >          * @madv: State for madvise
> >          *
> >          * 0 is active/inuse.
> > +        * 1 is not-needed/can-be-purged
> >          * A negative value is the object is purged.
> > -        * Positive values are driver specific and not used by the helpers.
> >          */
> >         int madv;
> >
> > @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
> >          * @map_wc: map object write-combined (instead of using shmem defaults).
> >          */
> >         bool map_wc;
> > +
> > +       /**
> > +        * @eviction_enabled:
> > +        *
> > +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> > +        * Used internally by memory shrinker.
> > +        */
> > +       bool eviction_enabled;
> > +
> > +       /**
> > +        * @purge_enabled:
> > +        *
> > +        * The shmem pages can be purged only if @purge_enabled is set to true.
> > +        * Used internally by memory shrinker.
> > +        */
> > +       bool purge_enabled;
> > +
> > +       /**
> > +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> > +        * Used internally by memory shrinker.
> > +        */
> > +       bool evicted;
> > +
> > +       /**
> > +        * @evict:
> > +        *
> > +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> > +        * GEM's DMA reservation is kept locked by the shrinker. This is
> > +        * optional callback that should be specified by drivers.
> > +        *
> > +        * Returns 0 on success, or -errno on error.
> > +        */
> > +       int (*evict)(struct drm_gem_shmem_object *shmem);
> >  };
> >
> >  #define to_drm_gem_shmem_obj(obj) \
> > @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
> >
> >  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
> >
> > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> > +
> >  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> >  {
> > -       return (shmem->madv > 0) &&
> > -               !shmem->vmap_use_count && shmem->sgt &&
> > -               !shmem->base.dma_buf && !shmem->base.import_attach;
> > +       return (shmem->madv > 0) && shmem->evict &&
> > +               shmem->purge_enabled && shmem->pages_use_count &&
> > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
> >  }
> >
> > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> > +
> > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> >
> >  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
> >  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> > @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
> >         return drm_gem_shmem_mmap(shmem, vma);
> >  }
> >
> > +/**
> > + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> > + */
> > +struct drm_gem_shmem_shrinker {
> > +       /** @base: Shrinker for purging shmem GEM objects */
> > +       struct shrinker base;
> > +
> > +       /** @lock: Protects @lru_* */
> > +       struct mutex lock;
> > +
> > +       /** @lru_pinned: List of pinned shmem GEM objects */
> > +       struct list_head lru_pinned;
> > +
> > +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> > +       struct list_head lru_evictable;
> > +
> > +       /** @lru_evicted: List of evicted shmem GEM objects */
> > +       struct list_head lru_evicted;
> > +
> > +       /** @dev: DRM device that uses this shrinker */
> > +       struct drm_device *dev;
> > +};
> > +
> > +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> > +
> >  /*
> >   * Driver ops
> >   */
> > --
> > 2.35.3
> >
>
>
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

On Sun, Jun 5, 2022 at 9:47 AM Daniel Vetter <daniel@ffwll.ch> wrote:
>
> On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
> >
> > Introduce a common DRM SHMEM shrinker framework that allows to reduce
> > code duplication among DRM drivers by replacing theirs custom shrinker
> > implementations with the generic shrinker.
> >
> > In order to start using DRM SHMEM shrinker drivers should:
> >
> > 1. Implement new evict() shmem object callback.
> > 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> > 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
> >    activate shrinking of shmem GEMs.
> >
> > This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> > Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
> >
> > Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> > Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>
> So I guess I get a price for being blind since forever, because this
> thing existed since at least 2013. I just stumbled over
> llist_lru.[hc], a purpose built list helper for shrinkers. I think we
> should try to adopt that so that our gpu shrinkers look more like
> shrinkers for everything else.

followup from a bit of irc discussion w/ danvet about list_lru:

* It seems to be missing a way to bail out of iteration before
  nr_to_scan is hit.. which is going to be inconvenient if you
  want to allow active bos on the LRU but bail scanning once
  you encounter the first one.

* Not sure if the numa node awareness is super useful for GEM
  bos

First issue is perhaps not too hard to fix.  But maybe a better
idea is a drm_gem_lru helper type thing which is more tailored
to GEM buffers?

BR,
-R

> Apologies for this, since I fear this might cause a bit of churn.
> Hopefully it's all contained to the list manipulation code in shmem
> helpers, I don't think this should leak any further.
> -Daniel
>
> > ---
> >  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
> >  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
> >  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
> >  include/drm/drm_device.h                      |   4 +
> >  include/drm/drm_gem_shmem_helper.h            |  87 ++-
> >  5 files changed, 594 insertions(+), 49 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > index 555fe212bd98..4cd0b5913492 100644
> > --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> > +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
> >  }
> >  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
> >
> > +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       return (shmem->madv >= 0) && shmem->evict &&
> > +               shmem->eviction_enabled && shmem->pages_use_count &&
> > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> > +}
> > +
> > +static void
> > +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> > +
> > +       dma_resv_assert_held(shmem->base.resv);
> > +
> > +       if (!gem_shrinker || obj->import_attach)
> > +               return;
> > +
> > +       mutex_lock(&gem_shrinker->lock);
> > +
> > +       if (drm_gem_shmem_is_evictable(shmem) ||
> > +           drm_gem_shmem_is_purgeable(shmem))
> > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> > +       else if (shmem->madv < 0)
> > +               list_del_init(&shmem->madv_list);
> > +       else if (shmem->evicted)
> > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> > +       else if (!shmem->pages)
> > +               list_del_init(&shmem->madv_list);
> > +       else
> > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> > +
> > +       mutex_unlock(&gem_shrinker->lock);
> > +}
> > +
> >  /**
> >   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
> >   * @shmem: shmem GEM object to free
> > @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> >         } else {
> >                 dma_resv_lock(shmem->base.resv, NULL);
> >
> > +               /* take out shmem GEM object from the memory shrinker */
> > +               drm_gem_shmem_madvise(shmem, -1);
> > +
> >                 WARN_ON(shmem->vmap_use_count);
> >
> >                 if (shmem->sgt) {
> > @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> >                         sg_free_table(shmem->sgt);
> >                         kfree(shmem->sgt);
> >                 }
> > -               if (shmem->pages)
> > +               if (shmem->pages_use_count)
> >                         drm_gem_shmem_put_pages(shmem);
> >
> >                 WARN_ON(shmem->pages_use_count);
> > @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> >  }
> >  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
> >
> > -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > +/**
> > + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> > + * @shmem: shmem GEM object
> > + *
> > + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       dma_resv_lock(shmem->base.resv, NULL);
> > +
> > +       if (shmem->madv < 0)
> > +               return -ENOMEM;
> > +
> > +       shmem->eviction_enabled = true;
> > +
> > +       dma_resv_unlock(shmem->base.resv);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> > +
> > +/**
> > + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> > + * @shmem: shmem GEM object
> > + *
> > + * Tell memory shrinker that this GEM can be purged. Initially purging is
> > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       dma_resv_lock(shmem->base.resv, NULL);
> > +
> > +       if (shmem->madv < 0)
> > +               return -ENOMEM;
> > +
> > +       shmem->purge_enabled = true;
> > +
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       dma_resv_unlock(shmem->base.resv);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> > +
> > +static int
> > +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
> >  {
> >         struct drm_gem_object *obj = &shmem->base;
> >         struct page **pages;
> >
> > -       if (shmem->pages_use_count++ > 0)
> > +       dma_resv_assert_held(shmem->base.resv);
> > +
> > +       if (shmem->madv < 0) {
> > +               WARN_ON(shmem->pages);
> > +               return -ENOMEM;
> > +       }
> > +
> > +       if (shmem->pages) {
> > +               WARN_ON(!shmem->evicted);
> >                 return 0;
> > +       }
> > +
> > +       if (WARN_ON(!shmem->pages_use_count))
> > +               return -EINVAL;
> >
> >         pages = drm_gem_get_pages(obj);
> >         if (IS_ERR(pages)) {
> >                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> > -               shmem->pages_use_count = 0;
> >                 return PTR_ERR(pages);
> >         }
> >
> > @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> >         return 0;
> >  }
> >
> > +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > +{
> > +       int err;
> > +
> > +       dma_resv_assert_held(shmem->base.resv);
> > +
> > +       if (shmem->madv < 0)
> > +               return -ENOMEM;
> > +
> > +       if (shmem->pages_use_count++ > 0) {
> > +               err = drm_gem_shmem_swap_in(shmem);
> > +               if (err)
> > +                       goto err_zero_use;
> > +
> > +               return 0;
> > +       }
> > +
> > +       err = drm_gem_shmem_acquire_pages(shmem);
> > +       if (err)
> > +               goto err_zero_use;
> > +
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       return 0;
> > +
> > +err_zero_use:
> > +       shmem->pages_use_count = 0;
> > +
> > +       return err;
> > +}
> > +
> > +static void
> > +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +
> > +       if (!shmem->pages) {
> > +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> > +               return;
> > +       }
> > +
> > +#ifdef CONFIG_X86
> > +       if (shmem->map_wc)
> > +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > +#endif
> > +
> > +       drm_gem_put_pages(obj, shmem->pages,
> > +                         shmem->pages_mark_dirty_on_put,
> > +                         shmem->pages_mark_accessed_on_put);
> > +       shmem->pages = NULL;
> > +}
> > +
> >  /*
> >   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
> >   * @shmem: shmem GEM object
> > @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> >   */
> >  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> >  {
> > -       struct drm_gem_object *obj = &shmem->base;
> > -
> >         dma_resv_assert_held(shmem->base.resv);
> >
> >         if (WARN_ON_ONCE(!shmem->pages_use_count))
> > @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> >         if (--shmem->pages_use_count > 0)
> >                 return;
> >
> > -#ifdef CONFIG_X86
> > -       if (shmem->map_wc)
> > -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > -#endif
> > +       drm_gem_shmem_release_pages(shmem);
> >
> > -       drm_gem_put_pages(obj, shmem->pages,
> > -                         shmem->pages_mark_dirty_on_put,
> > -                         shmem->pages_mark_accessed_on_put);
> > -       shmem->pages = NULL;
> > +       drm_gem_shmem_update_pages_state(shmem);
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> >
> > @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> >   */
> >  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
> >  {
> > +       int ret;
> > +
> >         dma_resv_assert_held(shmem->base.resv);
> >
> >         WARN_ON(shmem->base.import_attach);
> >
> > -       return drm_gem_shmem_get_pages(shmem);
> > +       ret = drm_gem_shmem_get_pages(shmem);
> > +       if (!ret)
> > +               shmem->pages_pin_count++;
> > +
> > +       return ret;
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_pin);
> >
> > @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
> >         WARN_ON(shmem->base.import_attach);
> >
> >         drm_gem_shmem_put_pages(shmem);
> > +
> > +       shmem->pages_pin_count--;
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_unpin);
> >
> > @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> >                         return 0;
> >                 }
> >
> > -               ret = drm_gem_shmem_get_pages(shmem);
> > +               ret = drm_gem_shmem_pin(shmem);
> >                 if (ret)
> >                         goto err_zero_use;
> >
> > @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> >
> >  err_put_pages:
> >         if (!obj->import_attach)
> > -               drm_gem_shmem_put_pages(shmem);
> > +               drm_gem_shmem_unpin(shmem);
> >  err_zero_use:
> >         shmem->vmap_use_count = 0;
> >
> > @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
> >                         return;
> >
> >                 vunmap(shmem->vaddr);
> > -               drm_gem_shmem_put_pages(shmem);
> > +               drm_gem_shmem_unpin(shmem);
> >         }
> >
> >         shmem->vaddr = NULL;
> > @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
> >
> >         madv = shmem->madv;
> >
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> >         return (madv >= 0);
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_madvise);
> >
> > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > +/**
> > + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> > + *                           hardware access to the memory.
> > + * @shmem: shmem GEM object
> > + *
> > + * This function moves shmem GEM back to memory if it was previously evicted
> > + * by the memory shrinker. The GEM is ready to use on success.
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
> >  {
> >         struct drm_gem_object *obj = &shmem->base;
> > -       struct drm_device *dev = obj->dev;
> > +       struct sg_table *sgt;
> > +       int err;
> >
> >         dma_resv_assert_held(shmem->base.resv);
> >
> > -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > +       if (shmem->evicted) {
> > +               err = drm_gem_shmem_acquire_pages(shmem);
> > +               if (err)
> > +                       return err;
> > +
> > +               sgt = drm_gem_shmem_get_sg_table(shmem);
> > +               if (IS_ERR(sgt))
> > +                       return PTR_ERR(sgt);
> > +
> > +               err = dma_map_sgtable(obj->dev->dev, sgt,
> > +                                     DMA_BIDIRECTIONAL, 0);
> > +               if (err) {
> > +                       sg_free_table(sgt);
> > +                       kfree(sgt);
> > +                       return err;
> > +               }
> >
> > -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > -       sg_free_table(shmem->sgt);
> > -       kfree(shmem->sgt);
> > -       shmem->sgt = NULL;
> > +               shmem->sgt = sgt;
> > +               shmem->evicted = false;
> >
> > -       drm_gem_shmem_put_pages(shmem);
> > +               drm_gem_shmem_update_pages_state(shmem);
> > +       }
> >
> > -       shmem->madv = -1;
> > +       if (!shmem->pages)
> > +               return -ENOMEM;
> >
> > -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > -       drm_gem_free_mmap_offset(obj);
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
> >
> > -       /* Our goal here is to return as much of the memory as
> > -        * is possible back to the system as we are called from OOM.
> > -        * To do this we must instruct the shmfs to drop all of its
> > -        * backing pages, *now*.
> > -        */
> > -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +       struct drm_device *dev = obj->dev;
> >
> > -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > +       if (shmem->evicted)
> > +               return;
> > +
> > +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > +       drm_gem_shmem_release_pages(shmem);
> > +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > +
> > +       sg_free_table(shmem->sgt);
> > +       kfree(shmem->sgt);
> > +       shmem->sgt = NULL;
> >  }
> > -EXPORT_SYMBOL(drm_gem_shmem_purge);
> >
> >  /**
> >   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> > @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
> >         vm_fault_t ret;
> >         struct page *page;
> >         pgoff_t page_offset;
> > +       bool pages_unpinned;
> > +       int err;
> >
> >         /* We don't use vmf->pgoff since that has the fake offset */
> >         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
> >
> >         dma_resv_lock(shmem->base.resv, NULL);
> >
> > -       if (page_offset >= num_pages ||
> > -           WARN_ON_ONCE(!shmem->pages) ||
> > -           shmem->madv < 0) {
> > +       /* Sanity-check that we have the pages pointer when it should present */
> > +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> > +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> > +
> > +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
> >                 ret = VM_FAULT_SIGBUS;
> >         } else {
> > +               err = drm_gem_shmem_swap_in(shmem);
> > +               if (err) {
> > +                       ret = VM_FAULT_OOM;
> > +                       goto unlock;
> > +               }
> > +
> >                 page = shmem->pages[page_offset];
> >
> >                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
> >         }
> >
> > +unlock:
> >         dma_resv_unlock(shmem->base.resv);
> >
> >         return ret;
> > @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
> >  {
> >         struct drm_gem_object *obj = vma->vm_private_data;
> >         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> > -       int ret;
> >
> >         WARN_ON(shmem->base.import_attach);
> >
> >         dma_resv_lock(shmem->base.resv, NULL);
> > -       ret = drm_gem_shmem_get_pages(shmem);
> > -       WARN_ON_ONCE(ret != 0);
> > +
> > +       if (drm_gem_shmem_get_pages(shmem))
> > +               shmem->pages_use_count++;
> > +
> > +       drm_gem_shmem_update_pages_state(shmem);
> >         dma_resv_unlock(shmem->base.resv);
> >
> >         drm_gem_vm_open(vma);
> > @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
> >  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> >                               struct drm_printer *p, unsigned int indent)
> >  {
> > +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> > +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
> >         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
> >
> >         if (shmem->base.import_attach)
> > @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> >                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
> >                                   shmem->vmap_use_count);
> >
> > +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
> >         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> > +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
> >  }
> >  EXPORT_SYMBOL(drm_gem_shmem_print_info);
> >
> > @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
> >
> >         shmem->sgt = sgt;
> >
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> >         dma_resv_unlock(shmem->base.resv);
> >
> >         return sgt;
> > @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
> >  }
> >  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
> >
> > +static struct drm_gem_shmem_shrinker *
> > +to_drm_shrinker(struct shrinker *shrinker)
> > +{
> > +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> > +}
> > +
> > +static unsigned long
> > +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> > +                                    struct shrink_control *sc)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > +       struct drm_gem_shmem_object *shmem;
> > +       unsigned long count = 0;
> > +
> > +       if (!mutex_trylock(&gem_shrinker->lock))
> > +               return 0;
> > +
> > +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> > +               count += shmem->base.size;
> > +
> > +               if (count >= SHRINK_EMPTY)
> > +                       break;
> > +       }
> > +
> > +       mutex_unlock(&gem_shrinker->lock);
> > +
> > +       if (count >= SHRINK_EMPTY)
> > +               return SHRINK_EMPTY - 1;
> > +
> > +       return count ?: SHRINK_EMPTY;
> > +}
> > +
> > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> > +{
> > +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> > +       WARN_ON(shmem->evicted);
> > +
> > +       drm_gem_shmem_unpin_pages(shmem);
> > +
> > +       shmem->evicted = true;
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> > +
> > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > +{
> > +       struct drm_gem_object *obj = &shmem->base;
> > +
> > +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > +
> > +       drm_gem_shmem_unpin_pages(shmem);
> > +       drm_gem_free_mmap_offset(obj);
> > +
> > +       /* Our goal here is to return as much of the memory as
> > +        * is possible back to the system as we are called from OOM.
> > +        * To do this we must instruct the shmfs to drop all of its
> > +        * backing pages, *now*.
> > +        */
> > +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > +
> > +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > +
> > +       shmem->madv = -1;
> > +       shmem->evicted = false;
> > +       drm_gem_shmem_update_pages_state(shmem);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> > +
> > +static unsigned long
> > +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> > +                                       unsigned long nr_to_scan,
> > +                                       bool *lock_contention,
> > +                                       bool evict)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > +       struct drm_gem_shmem_object *shmem;
> > +       struct list_head still_in_list;
> > +       struct drm_gem_object *obj;
> > +       unsigned long freed = 0;
> > +       size_t page_count;
> > +       int err;
> > +
> > +       INIT_LIST_HEAD(&still_in_list);
> > +
> > +       mutex_lock(&gem_shrinker->lock);
> > +
> > +       while (freed < nr_to_scan) {
> > +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> > +                                                typeof(*shmem), madv_list);
> > +               if (!shmem)
> > +                       break;
> > +
> > +               obj = &shmem->base;
> > +               page_count = obj->size >> PAGE_SHIFT;
> > +               list_move_tail(&shmem->madv_list, &still_in_list);
> > +
> > +               if (evict) {
> > +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> > +                           get_nr_swap_pages() < page_count)
> > +                               continue;
> > +               } else {
> > +                       if (!drm_gem_shmem_is_purgeable(shmem))
> > +                               continue;
> > +               }
> > +
> > +               /*
> > +                * If it's in the process of being freed, gem_object->free()
> > +                * may be blocked on lock waiting to remove it.  So just
> > +                * skip it.
> > +                */
> > +               if (!kref_get_unless_zero(&obj->refcount))
> > +                       continue;
> > +
> > +               mutex_unlock(&gem_shrinker->lock);
> > +
> > +               /* prevent racing with job-submission code paths */
> > +               if (!dma_resv_trylock(obj->resv)) {
> > +                       *lock_contention |= true;
> > +                       goto shrinker_lock;
> > +               }
> > +
> > +               /* prevent racing with the dma-buf importing/exporting */
> > +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > +                       *lock_contention |= true;
> > +                       goto resv_unlock;
> > +               }
> > +
> > +               /* check whether h/w uses this object */
> > +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> > +                       goto object_name_unlock;
> > +
> > +               /* re-check whether eviction status hasn't changed */
> > +               if (!drm_gem_shmem_is_evictable(shmem) &&
> > +                   !drm_gem_shmem_is_purgeable(shmem))
> > +                       goto object_name_unlock;
> > +
> > +               err = shmem->evict(shmem);
> > +               if (!err)
> > +                       freed += obj->size >> PAGE_SHIFT;
> > +
> > +object_name_unlock:
> > +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> > +resv_unlock:
> > +               dma_resv_unlock(obj->resv);
> > +shrinker_lock:
> > +               drm_gem_object_put(&shmem->base);
> > +               mutex_lock(&gem_shrinker->lock);
> > +       }
> > +
> > +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> > +
> > +       mutex_unlock(&gem_shrinker->lock);
> > +
> > +       return freed;
> > +}
> > +
> > +static unsigned long
> > +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> > +                                   struct shrink_control *sc)
> > +{
> > +       unsigned long nr_to_scan = sc->nr_to_scan;
> > +       bool lock_contention = false;
> > +       unsigned long freed;
> > +
> > +       /* purge as many objects as we can */
> > +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> > +                                                       &lock_contention, false);
> > +
> > +       /* evict as many objects as we can */
> > +       if (freed < nr_to_scan)
> > +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> > +                                                                nr_to_scan - freed,
> > +                                                                &lock_contention,
> > +                                                                true);
> > +
> > +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> > +}
> > +
> > +/**
> > + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> > + * @dev: DRM device
> > + *
> > + * Returns:
> > + * 0 on success or a negative error code on failure.
> > + */
> > +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker;
> > +       int err;
> > +
> > +       if (WARN_ON(dev->shmem_shrinker))
> > +               return -EBUSY;
> > +
> > +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> > +       if (!gem_shrinker)
> > +               return -ENOMEM;
> > +
> > +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> > +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> > +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> > +       gem_shrinker->dev = dev;
> > +
> > +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> > +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> > +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> > +       mutex_init(&gem_shrinker->lock);
> > +
> > +       dev->shmem_shrinker = gem_shrinker;
> > +
> > +       err = register_shrinker(&gem_shrinker->base);
> > +       if (err) {
> > +               dev->shmem_shrinker = NULL;
> > +               kfree(gem_shrinker);
> > +               return err;
> > +       }
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> > +
> > +/**
> > + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> > + * @dev: DRM device
> > + */
> > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> > +{
> > +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> > +
> > +       if (gem_shrinker) {
> > +               unregister_shrinker(&gem_shrinker->base);
> > +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> > +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> > +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> > +               mutex_destroy(&gem_shrinker->lock);
> > +               dev->shmem_shrinker = NULL;
> > +               kfree(gem_shrinker);
> > +       }
> > +}
> > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> > +
> >  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
> >  MODULE_IMPORT_NS(DMA_BUF);
> >  MODULE_LICENSE("GPL v2");
> > diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > index a4bedfeb2ec4..7cc32556f908 100644
> > --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > @@ -15,6 +15,13 @@
> >  #include "panfrost_gem.h"
> >  #include "panfrost_mmu.h"
> >
> > +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> > +{
> > +       return (shmem->madv > 0) &&
> > +               !shmem->pages_pin_count && shmem->sgt &&
> > +               !shmem->base.dma_buf && !shmem->base.import_attach;
> > +}
> > +
> >  static unsigned long
> >  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
> >  {
> > @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
> >                 return 0;
> >
> >         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> > -               if (drm_gem_shmem_is_purgeable(shmem))
> > +               if (panfrost_gem_shmem_is_purgeable(shmem))
> >                         count += shmem->base.size >> PAGE_SHIFT;
> >         }
> >
> > diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > index b2d93cb12ebf..81bacc7e1873 100644
> > --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> > +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > @@ -89,6 +89,7 @@ struct virtio_gpu_object {
> >         uint32_t hw_res_handle;
> >         bool dumb;
> >         bool created;
> > +       bool detached;
> >         bool host3d_blob, guest_blob;
> >         uint32_t blob_mem, blob_flags;
> >
> > @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
> >
> >  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
> >
> > +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> > +
> >  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
> >                                uint32_t *resid);
> >  /* virtgpu_prime.c */
> > diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> > index 9923c7a6885e..929546cad894 100644
> > --- a/include/drm/drm_device.h
> > +++ b/include/drm/drm_device.h
> > @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
> >  struct drm_vma_offset_manager;
> >  struct drm_vram_mm;
> >  struct drm_fb_helper;
> > +struct drm_gem_shmem_shrinker;
> >
> >  struct inode;
> >
> > @@ -277,6 +278,9 @@ struct drm_device {
> >         /** @vram_mm: VRAM MM memory manager */
> >         struct drm_vram_mm *vram_mm;
> >
> > +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> > +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> > +
> >         /**
> >          * @switch_power_state:
> >          *
> > diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> > index 9a8983ee8abe..62c640678a91 100644
> > --- a/include/drm/drm_gem_shmem_helper.h
> > +++ b/include/drm/drm_gem_shmem_helper.h
> > @@ -6,6 +6,7 @@
> >  #include <linux/fs.h>
> >  #include <linux/mm.h>
> >  #include <linux/mutex.h>
> > +#include <linux/shrinker.h>
> >
> >  #include <drm/drm_file.h>
> >  #include <drm/drm_gem.h>
> > @@ -15,6 +16,7 @@
> >  struct dma_buf_attachment;
> >  struct drm_mode_create_dumb;
> >  struct drm_printer;
> > +struct drm_device;
> >  struct sg_table;
> >
> >  /**
> > @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
> >          */
> >         unsigned int pages_use_count;
> >
> > +       /**
> > +        * @pages_pin_count:
> > +        *
> > +        * Reference count on the pinned pages table.
> > +        * The pages can be evicted by memory shrinker
> > +        * when the count reaches zero.
> > +        */
> > +       unsigned int pages_pin_count;
> > +
> >         /**
> >          * @madv: State for madvise
> >          *
> >          * 0 is active/inuse.
> > +        * 1 is not-needed/can-be-purged
> >          * A negative value is the object is purged.
> > -        * Positive values are driver specific and not used by the helpers.
> >          */
> >         int madv;
> >
> > @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
> >          * @map_wc: map object write-combined (instead of using shmem defaults).
> >          */
> >         bool map_wc;
> > +
> > +       /**
> > +        * @eviction_enabled:
> > +        *
> > +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> > +        * Used internally by memory shrinker.
> > +        */
> > +       bool eviction_enabled;
> > +
> > +       /**
> > +        * @purge_enabled:
> > +        *
> > +        * The shmem pages can be purged only if @purge_enabled is set to true.
> > +        * Used internally by memory shrinker.
> > +        */
> > +       bool purge_enabled;
> > +
> > +       /**
> > +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> > +        * Used internally by memory shrinker.
> > +        */
> > +       bool evicted;
> > +
> > +       /**
> > +        * @evict:
> > +        *
> > +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> > +        * GEM's DMA reservation is kept locked by the shrinker. This is
> > +        * optional callback that should be specified by drivers.
> > +        *
> > +        * Returns 0 on success, or -errno on error.
> > +        */
> > +       int (*evict)(struct drm_gem_shmem_object *shmem);
> >  };
> >
> >  #define to_drm_gem_shmem_obj(obj) \
> > @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
> >
> >  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
> >
> > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> > +
> >  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> >  {
> > -       return (shmem->madv > 0) &&
> > -               !shmem->vmap_use_count && shmem->sgt &&
> > -               !shmem->base.dma_buf && !shmem->base.import_attach;
> > +       return (shmem->madv > 0) && shmem->evict &&
> > +               shmem->purge_enabled && shmem->pages_use_count &&
> > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
> >  }
> >
> > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> > +
> > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> >
> >  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
> >  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> > @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
> >         return drm_gem_shmem_mmap(shmem, vma);
> >  }
> >
> > +/**
> > + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> > + */
> > +struct drm_gem_shmem_shrinker {
> > +       /** @base: Shrinker for purging shmem GEM objects */
> > +       struct shrinker base;
> > +
> > +       /** @lock: Protects @lru_* */
> > +       struct mutex lock;
> > +
> > +       /** @lru_pinned: List of pinned shmem GEM objects */
> > +       struct list_head lru_pinned;
> > +
> > +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> > +       struct list_head lru_evictable;
> > +
> > +       /** @lru_evicted: List of evicted shmem GEM objects */
> > +       struct list_head lru_evicted;
> > +
> > +       /** @dev: DRM device that uses this shrinker */
> > +       struct drm_device *dev;
> > +};
> > +
> > +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> > +
> >  /*
> >   * Driver ops
> >   */
> > --
> > 2.35.3
> >
>
>
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Sun, 5 Jun 2022 at 20:32, Rob Clark <robdclark@gmail.com> wrote:
>
> On Sun, Jun 5, 2022 at 9:47 AM Daniel Vetter <daniel@ffwll.ch> wrote:
> >
> > On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
> > <dmitry.osipenko@collabora.com> wrote:
> > >
> > > Introduce a common DRM SHMEM shrinker framework that allows to reduce
> > > code duplication among DRM drivers by replacing theirs custom shrinker
> > > implementations with the generic shrinker.
> > >
> > > In order to start using DRM SHMEM shrinker drivers should:
> > >
> > > 1. Implement new evict() shmem object callback.
> > > 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> > > 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
> > >    activate shrinking of shmem GEMs.
> > >
> > > This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> > > Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
> > >
> > > Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> > > Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> >
> > So I guess I get a price for being blind since forever, because this
> > thing existed since at least 2013. I just stumbled over
> > llist_lru.[hc], a purpose built list helper for shrinkers. I think we
> > should try to adopt that so that our gpu shrinkers look more like
> > shrinkers for everything else.
>
> followup from a bit of irc discussion w/ danvet about list_lru:
>
> * It seems to be missing a way to bail out of iteration before
>   nr_to_scan is hit.. which is going to be inconvenient if you
>   want to allow active bos on the LRU but bail scanning once
>   you encounter the first one.
>
> * Not sure if the numa node awareness is super useful for GEM
>   bos
>
> First issue is perhaps not too hard to fix.  But maybe a better
> idea is a drm_gem_lru helper type thing which is more tailored
> to GEM buffers?

Yeah I guess reusing list_lru isn't that good idea. So just
open-coding it for now, and then drm_gem_bo_lru or so if we need to
share it separately from shmem helpers with other drivers. Maybe will
be needed for ttm or so.
-Daniel

>
> BR,
> -R
>
> > Apologies for this, since I fear this might cause a bit of churn.
> > Hopefully it's all contained to the list manipulation code in shmem
> > helpers, I don't think this should leak any further.
> > -Daniel
> >
> > > ---
> > >  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
> > >  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
> > >  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
> > >  include/drm/drm_device.h                      |   4 +
> > >  include/drm/drm_gem_shmem_helper.h            |  87 ++-
> > >  5 files changed, 594 insertions(+), 49 deletions(-)
> > >
> > > diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > > index 555fe212bd98..4cd0b5913492 100644
> > > --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> > > +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > > @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
> > >  }
> > >  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
> > >
> > > +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       return (shmem->madv >= 0) && shmem->evict &&
> > > +               shmem->eviction_enabled && shmem->pages_use_count &&
> > > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > > +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> > > +}
> > > +
> > > +static void
> > > +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> > > +
> > > +       dma_resv_assert_held(shmem->base.resv);
> > > +
> > > +       if (!gem_shrinker || obj->import_attach)
> > > +               return;
> > > +
> > > +       mutex_lock(&gem_shrinker->lock);
> > > +
> > > +       if (drm_gem_shmem_is_evictable(shmem) ||
> > > +           drm_gem_shmem_is_purgeable(shmem))
> > > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> > > +       else if (shmem->madv < 0)
> > > +               list_del_init(&shmem->madv_list);
> > > +       else if (shmem->evicted)
> > > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> > > +       else if (!shmem->pages)
> > > +               list_del_init(&shmem->madv_list);
> > > +       else
> > > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> > > +
> > > +       mutex_unlock(&gem_shrinker->lock);
> > > +}
> > > +
> > >  /**
> > >   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
> > >   * @shmem: shmem GEM object to free
> > > @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> > >         } else {
> > >                 dma_resv_lock(shmem->base.resv, NULL);
> > >
> > > +               /* take out shmem GEM object from the memory shrinker */
> > > +               drm_gem_shmem_madvise(shmem, -1);
> > > +
> > >                 WARN_ON(shmem->vmap_use_count);
> > >
> > >                 if (shmem->sgt) {
> > > @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> > >                         sg_free_table(shmem->sgt);
> > >                         kfree(shmem->sgt);
> > >                 }
> > > -               if (shmem->pages)
> > > +               if (shmem->pages_use_count)
> > >                         drm_gem_shmem_put_pages(shmem);
> > >
> > >                 WARN_ON(shmem->pages_use_count);
> > > @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> > >  }
> > >  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
> > >
> > > -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > > +/**
> > > + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> > > + * @shmem: shmem GEM object
> > > + *
> > > + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> > > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       dma_resv_lock(shmem->base.resv, NULL);
> > > +
> > > +       if (shmem->madv < 0)
> > > +               return -ENOMEM;
> > > +
> > > +       shmem->eviction_enabled = true;
> > > +
> > > +       dma_resv_unlock(shmem->base.resv);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> > > +
> > > +/**
> > > + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> > > + * @shmem: shmem GEM object
> > > + *
> > > + * Tell memory shrinker that this GEM can be purged. Initially purging is
> > > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       dma_resv_lock(shmem->base.resv, NULL);
> > > +
> > > +       if (shmem->madv < 0)
> > > +               return -ENOMEM;
> > > +
> > > +       shmem->purge_enabled = true;
> > > +
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       dma_resv_unlock(shmem->base.resv);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> > > +
> > > +static int
> > > +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
> > >  {
> > >         struct drm_gem_object *obj = &shmem->base;
> > >         struct page **pages;
> > >
> > > -       if (shmem->pages_use_count++ > 0)
> > > +       dma_resv_assert_held(shmem->base.resv);
> > > +
> > > +       if (shmem->madv < 0) {
> > > +               WARN_ON(shmem->pages);
> > > +               return -ENOMEM;
> > > +       }
> > > +
> > > +       if (shmem->pages) {
> > > +               WARN_ON(!shmem->evicted);
> > >                 return 0;
> > > +       }
> > > +
> > > +       if (WARN_ON(!shmem->pages_use_count))
> > > +               return -EINVAL;
> > >
> > >         pages = drm_gem_get_pages(obj);
> > >         if (IS_ERR(pages)) {
> > >                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> > > -               shmem->pages_use_count = 0;
> > >                 return PTR_ERR(pages);
> > >         }
> > >
> > > @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > >         return 0;
> > >  }
> > >
> > > +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       int err;
> > > +
> > > +       dma_resv_assert_held(shmem->base.resv);
> > > +
> > > +       if (shmem->madv < 0)
> > > +               return -ENOMEM;
> > > +
> > > +       if (shmem->pages_use_count++ > 0) {
> > > +               err = drm_gem_shmem_swap_in(shmem);
> > > +               if (err)
> > > +                       goto err_zero_use;
> > > +
> > > +               return 0;
> > > +       }
> > > +
> > > +       err = drm_gem_shmem_acquire_pages(shmem);
> > > +       if (err)
> > > +               goto err_zero_use;
> > > +
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       return 0;
> > > +
> > > +err_zero_use:
> > > +       shmem->pages_use_count = 0;
> > > +
> > > +       return err;
> > > +}
> > > +
> > > +static void
> > > +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +
> > > +       if (!shmem->pages) {
> > > +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> > > +               return;
> > > +       }
> > > +
> > > +#ifdef CONFIG_X86
> > > +       if (shmem->map_wc)
> > > +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > > +#endif
> > > +
> > > +       drm_gem_put_pages(obj, shmem->pages,
> > > +                         shmem->pages_mark_dirty_on_put,
> > > +                         shmem->pages_mark_accessed_on_put);
> > > +       shmem->pages = NULL;
> > > +}
> > > +
> > >  /*
> > >   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
> > >   * @shmem: shmem GEM object
> > > @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > >   */
> > >  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> > >  {
> > > -       struct drm_gem_object *obj = &shmem->base;
> > > -
> > >         dma_resv_assert_held(shmem->base.resv);
> > >
> > >         if (WARN_ON_ONCE(!shmem->pages_use_count))
> > > @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> > >         if (--shmem->pages_use_count > 0)
> > >                 return;
> > >
> > > -#ifdef CONFIG_X86
> > > -       if (shmem->map_wc)
> > > -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > > -#endif
> > > +       drm_gem_shmem_release_pages(shmem);
> > >
> > > -       drm_gem_put_pages(obj, shmem->pages,
> > > -                         shmem->pages_mark_dirty_on_put,
> > > -                         shmem->pages_mark_accessed_on_put);
> > > -       shmem->pages = NULL;
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> > >
> > > @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> > >   */
> > >  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
> > >  {
> > > +       int ret;
> > > +
> > >         dma_resv_assert_held(shmem->base.resv);
> > >
> > >         WARN_ON(shmem->base.import_attach);
> > >
> > > -       return drm_gem_shmem_get_pages(shmem);
> > > +       ret = drm_gem_shmem_get_pages(shmem);
> > > +       if (!ret)
> > > +               shmem->pages_pin_count++;
> > > +
> > > +       return ret;
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_pin);
> > >
> > > @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
> > >         WARN_ON(shmem->base.import_attach);
> > >
> > >         drm_gem_shmem_put_pages(shmem);
> > > +
> > > +       shmem->pages_pin_count--;
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_unpin);
> > >
> > > @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> > >                         return 0;
> > >                 }
> > >
> > > -               ret = drm_gem_shmem_get_pages(shmem);
> > > +               ret = drm_gem_shmem_pin(shmem);
> > >                 if (ret)
> > >                         goto err_zero_use;
> > >
> > > @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> > >
> > >  err_put_pages:
> > >         if (!obj->import_attach)
> > > -               drm_gem_shmem_put_pages(shmem);
> > > +               drm_gem_shmem_unpin(shmem);
> > >  err_zero_use:
> > >         shmem->vmap_use_count = 0;
> > >
> > > @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
> > >                         return;
> > >
> > >                 vunmap(shmem->vaddr);
> > > -               drm_gem_shmem_put_pages(shmem);
> > > +               drm_gem_shmem_unpin(shmem);
> > >         }
> > >
> > >         shmem->vaddr = NULL;
> > > @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
> > >
> > >         madv = shmem->madv;
> > >
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > >         return (madv >= 0);
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_madvise);
> > >
> > > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > > +/**
> > > + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> > > + *                           hardware access to the memory.
> > > + * @shmem: shmem GEM object
> > > + *
> > > + * This function moves shmem GEM back to memory if it was previously evicted
> > > + * by the memory shrinker. The GEM is ready to use on success.
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
> > >  {
> > >         struct drm_gem_object *obj = &shmem->base;
> > > -       struct drm_device *dev = obj->dev;
> > > +       struct sg_table *sgt;
> > > +       int err;
> > >
> > >         dma_resv_assert_held(shmem->base.resv);
> > >
> > > -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > > +       if (shmem->evicted) {
> > > +               err = drm_gem_shmem_acquire_pages(shmem);
> > > +               if (err)
> > > +                       return err;
> > > +
> > > +               sgt = drm_gem_shmem_get_sg_table(shmem);
> > > +               if (IS_ERR(sgt))
> > > +                       return PTR_ERR(sgt);
> > > +
> > > +               err = dma_map_sgtable(obj->dev->dev, sgt,
> > > +                                     DMA_BIDIRECTIONAL, 0);
> > > +               if (err) {
> > > +                       sg_free_table(sgt);
> > > +                       kfree(sgt);
> > > +                       return err;
> > > +               }
> > >
> > > -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > > -       sg_free_table(shmem->sgt);
> > > -       kfree(shmem->sgt);
> > > -       shmem->sgt = NULL;
> > > +               shmem->sgt = sgt;
> > > +               shmem->evicted = false;
> > >
> > > -       drm_gem_shmem_put_pages(shmem);
> > > +               drm_gem_shmem_update_pages_state(shmem);
> > > +       }
> > >
> > > -       shmem->madv = -1;
> > > +       if (!shmem->pages)
> > > +               return -ENOMEM;
> > >
> > > -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > > -       drm_gem_free_mmap_offset(obj);
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
> > >
> > > -       /* Our goal here is to return as much of the memory as
> > > -        * is possible back to the system as we are called from OOM.
> > > -        * To do this we must instruct the shmfs to drop all of its
> > > -        * backing pages, *now*.
> > > -        */
> > > -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > > +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +       struct drm_device *dev = obj->dev;
> > >
> > > -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > > +       if (shmem->evicted)
> > > +               return;
> > > +
> > > +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > > +       drm_gem_shmem_release_pages(shmem);
> > > +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > > +
> > > +       sg_free_table(shmem->sgt);
> > > +       kfree(shmem->sgt);
> > > +       shmem->sgt = NULL;
> > >  }
> > > -EXPORT_SYMBOL(drm_gem_shmem_purge);
> > >
> > >  /**
> > >   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> > > @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
> > >         vm_fault_t ret;
> > >         struct page *page;
> > >         pgoff_t page_offset;
> > > +       bool pages_unpinned;
> > > +       int err;
> > >
> > >         /* We don't use vmf->pgoff since that has the fake offset */
> > >         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
> > >
> > >         dma_resv_lock(shmem->base.resv, NULL);
> > >
> > > -       if (page_offset >= num_pages ||
> > > -           WARN_ON_ONCE(!shmem->pages) ||
> > > -           shmem->madv < 0) {
> > > +       /* Sanity-check that we have the pages pointer when it should present */
> > > +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> > > +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> > > +
> > > +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
> > >                 ret = VM_FAULT_SIGBUS;
> > >         } else {
> > > +               err = drm_gem_shmem_swap_in(shmem);
> > > +               if (err) {
> > > +                       ret = VM_FAULT_OOM;
> > > +                       goto unlock;
> > > +               }
> > > +
> > >                 page = shmem->pages[page_offset];
> > >
> > >                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
> > >         }
> > >
> > > +unlock:
> > >         dma_resv_unlock(shmem->base.resv);
> > >
> > >         return ret;
> > > @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
> > >  {
> > >         struct drm_gem_object *obj = vma->vm_private_data;
> > >         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> > > -       int ret;
> > >
> > >         WARN_ON(shmem->base.import_attach);
> > >
> > >         dma_resv_lock(shmem->base.resv, NULL);
> > > -       ret = drm_gem_shmem_get_pages(shmem);
> > > -       WARN_ON_ONCE(ret != 0);
> > > +
> > > +       if (drm_gem_shmem_get_pages(shmem))
> > > +               shmem->pages_use_count++;
> > > +
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > >         dma_resv_unlock(shmem->base.resv);
> > >
> > >         drm_gem_vm_open(vma);
> > > @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
> > >  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> > >                               struct drm_printer *p, unsigned int indent)
> > >  {
> > > +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> > > +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
> > >         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
> > >
> > >         if (shmem->base.import_attach)
> > > @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> > >                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
> > >                                   shmem->vmap_use_count);
> > >
> > > +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
> > >         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> > > +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_print_info);
> > >
> > > @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
> > >
> > >         shmem->sgt = sgt;
> > >
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > >         dma_resv_unlock(shmem->base.resv);
> > >
> > >         return sgt;
> > > @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
> > >  }
> > >  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
> > >
> > > +static struct drm_gem_shmem_shrinker *
> > > +to_drm_shrinker(struct shrinker *shrinker)
> > > +{
> > > +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> > > +}
> > > +
> > > +static unsigned long
> > > +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> > > +                                    struct shrink_control *sc)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > > +       struct drm_gem_shmem_object *shmem;
> > > +       unsigned long count = 0;
> > > +
> > > +       if (!mutex_trylock(&gem_shrinker->lock))
> > > +               return 0;
> > > +
> > > +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> > > +               count += shmem->base.size;
> > > +
> > > +               if (count >= SHRINK_EMPTY)
> > > +                       break;
> > > +       }
> > > +
> > > +       mutex_unlock(&gem_shrinker->lock);
> > > +
> > > +       if (count >= SHRINK_EMPTY)
> > > +               return SHRINK_EMPTY - 1;
> > > +
> > > +       return count ?: SHRINK_EMPTY;
> > > +}
> > > +
> > > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> > > +       WARN_ON(shmem->evicted);
> > > +
> > > +       drm_gem_shmem_unpin_pages(shmem);
> > > +
> > > +       shmem->evicted = true;
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> > > +
> > > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +
> > > +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > > +
> > > +       drm_gem_shmem_unpin_pages(shmem);
> > > +       drm_gem_free_mmap_offset(obj);
> > > +
> > > +       /* Our goal here is to return as much of the memory as
> > > +        * is possible back to the system as we are called from OOM.
> > > +        * To do this we must instruct the shmfs to drop all of its
> > > +        * backing pages, *now*.
> > > +        */
> > > +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > > +
> > > +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > > +
> > > +       shmem->madv = -1;
> > > +       shmem->evicted = false;
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> > > +
> > > +static unsigned long
> > > +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> > > +                                       unsigned long nr_to_scan,
> > > +                                       bool *lock_contention,
> > > +                                       bool evict)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > > +       struct drm_gem_shmem_object *shmem;
> > > +       struct list_head still_in_list;
> > > +       struct drm_gem_object *obj;
> > > +       unsigned long freed = 0;
> > > +       size_t page_count;
> > > +       int err;
> > > +
> > > +       INIT_LIST_HEAD(&still_in_list);
> > > +
> > > +       mutex_lock(&gem_shrinker->lock);
> > > +
> > > +       while (freed < nr_to_scan) {
> > > +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> > > +                                                typeof(*shmem), madv_list);
> > > +               if (!shmem)
> > > +                       break;
> > > +
> > > +               obj = &shmem->base;
> > > +               page_count = obj->size >> PAGE_SHIFT;
> > > +               list_move_tail(&shmem->madv_list, &still_in_list);
> > > +
> > > +               if (evict) {
> > > +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> > > +                           get_nr_swap_pages() < page_count)
> > > +                               continue;
> > > +               } else {
> > > +                       if (!drm_gem_shmem_is_purgeable(shmem))
> > > +                               continue;
> > > +               }
> > > +
> > > +               /*
> > > +                * If it's in the process of being freed, gem_object->free()
> > > +                * may be blocked on lock waiting to remove it.  So just
> > > +                * skip it.
> > > +                */
> > > +               if (!kref_get_unless_zero(&obj->refcount))
> > > +                       continue;
> > > +
> > > +               mutex_unlock(&gem_shrinker->lock);
> > > +
> > > +               /* prevent racing with job-submission code paths */
> > > +               if (!dma_resv_trylock(obj->resv)) {
> > > +                       *lock_contention |= true;
> > > +                       goto shrinker_lock;
> > > +               }
> > > +
> > > +               /* prevent racing with the dma-buf importing/exporting */
> > > +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > > +                       *lock_contention |= true;
> > > +                       goto resv_unlock;
> > > +               }
> > > +
> > > +               /* check whether h/w uses this object */
> > > +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> > > +                       goto object_name_unlock;
> > > +
> > > +               /* re-check whether eviction status hasn't changed */
> > > +               if (!drm_gem_shmem_is_evictable(shmem) &&
> > > +                   !drm_gem_shmem_is_purgeable(shmem))
> > > +                       goto object_name_unlock;
> > > +
> > > +               err = shmem->evict(shmem);
> > > +               if (!err)
> > > +                       freed += obj->size >> PAGE_SHIFT;
> > > +
> > > +object_name_unlock:
> > > +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> > > +resv_unlock:
> > > +               dma_resv_unlock(obj->resv);
> > > +shrinker_lock:
> > > +               drm_gem_object_put(&shmem->base);
> > > +               mutex_lock(&gem_shrinker->lock);
> > > +       }
> > > +
> > > +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> > > +
> > > +       mutex_unlock(&gem_shrinker->lock);
> > > +
> > > +       return freed;
> > > +}
> > > +
> > > +static unsigned long
> > > +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> > > +                                   struct shrink_control *sc)
> > > +{
> > > +       unsigned long nr_to_scan = sc->nr_to_scan;
> > > +       bool lock_contention = false;
> > > +       unsigned long freed;
> > > +
> > > +       /* purge as many objects as we can */
> > > +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> > > +                                                       &lock_contention, false);
> > > +
> > > +       /* evict as many objects as we can */
> > > +       if (freed < nr_to_scan)
> > > +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> > > +                                                                nr_to_scan - freed,
> > > +                                                                &lock_contention,
> > > +                                                                true);
> > > +
> > > +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> > > +}
> > > +
> > > +/**
> > > + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> > > + * @dev: DRM device
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker;
> > > +       int err;
> > > +
> > > +       if (WARN_ON(dev->shmem_shrinker))
> > > +               return -EBUSY;
> > > +
> > > +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> > > +       if (!gem_shrinker)
> > > +               return -ENOMEM;
> > > +
> > > +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> > > +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> > > +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> > > +       gem_shrinker->dev = dev;
> > > +
> > > +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> > > +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> > > +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> > > +       mutex_init(&gem_shrinker->lock);
> > > +
> > > +       dev->shmem_shrinker = gem_shrinker;
> > > +
> > > +       err = register_shrinker(&gem_shrinker->base);
> > > +       if (err) {
> > > +               dev->shmem_shrinker = NULL;
> > > +               kfree(gem_shrinker);
> > > +               return err;
> > > +       }
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> > > +
> > > +/**
> > > + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> > > + * @dev: DRM device
> > > + */
> > > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> > > +
> > > +       if (gem_shrinker) {
> > > +               unregister_shrinker(&gem_shrinker->base);
> > > +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> > > +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> > > +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> > > +               mutex_destroy(&gem_shrinker->lock);
> > > +               dev->shmem_shrinker = NULL;
> > > +               kfree(gem_shrinker);
> > > +       }
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> > > +
> > >  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
> > >  MODULE_IMPORT_NS(DMA_BUF);
> > >  MODULE_LICENSE("GPL v2");
> > > diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > > index a4bedfeb2ec4..7cc32556f908 100644
> > > --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > > +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > > @@ -15,6 +15,13 @@
> > >  #include "panfrost_gem.h"
> > >  #include "panfrost_mmu.h"
> > >
> > > +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       return (shmem->madv > 0) &&
> > > +               !shmem->pages_pin_count && shmem->sgt &&
> > > +               !shmem->base.dma_buf && !shmem->base.import_attach;
> > > +}
> > > +
> > >  static unsigned long
> > >  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
> > >  {
> > > @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
> > >                 return 0;
> > >
> > >         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> > > -               if (drm_gem_shmem_is_purgeable(shmem))
> > > +               if (panfrost_gem_shmem_is_purgeable(shmem))
> > >                         count += shmem->base.size >> PAGE_SHIFT;
> > >         }
> > >
> > > diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > > index b2d93cb12ebf..81bacc7e1873 100644
> > > --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> > > +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > > @@ -89,6 +89,7 @@ struct virtio_gpu_object {
> > >         uint32_t hw_res_handle;
> > >         bool dumb;
> > >         bool created;
> > > +       bool detached;
> > >         bool host3d_blob, guest_blob;
> > >         uint32_t blob_mem, blob_flags;
> > >
> > > @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
> > >
> > >  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
> > >
> > > +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> > > +
> > >  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
> > >                                uint32_t *resid);
> > >  /* virtgpu_prime.c */
> > > diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> > > index 9923c7a6885e..929546cad894 100644
> > > --- a/include/drm/drm_device.h
> > > +++ b/include/drm/drm_device.h
> > > @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
> > >  struct drm_vma_offset_manager;
> > >  struct drm_vram_mm;
> > >  struct drm_fb_helper;
> > > +struct drm_gem_shmem_shrinker;
> > >
> > >  struct inode;
> > >
> > > @@ -277,6 +278,9 @@ struct drm_device {
> > >         /** @vram_mm: VRAM MM memory manager */
> > >         struct drm_vram_mm *vram_mm;
> > >
> > > +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> > > +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> > > +
> > >         /**
> > >          * @switch_power_state:
> > >          *
> > > diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> > > index 9a8983ee8abe..62c640678a91 100644
> > > --- a/include/drm/drm_gem_shmem_helper.h
> > > +++ b/include/drm/drm_gem_shmem_helper.h
> > > @@ -6,6 +6,7 @@
> > >  #include <linux/fs.h>
> > >  #include <linux/mm.h>
> > >  #include <linux/mutex.h>
> > > +#include <linux/shrinker.h>
> > >
> > >  #include <drm/drm_file.h>
> > >  #include <drm/drm_gem.h>
> > > @@ -15,6 +16,7 @@
> > >  struct dma_buf_attachment;
> > >  struct drm_mode_create_dumb;
> > >  struct drm_printer;
> > > +struct drm_device;
> > >  struct sg_table;
> > >
> > >  /**
> > > @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
> > >          */
> > >         unsigned int pages_use_count;
> > >
> > > +       /**
> > > +        * @pages_pin_count:
> > > +        *
> > > +        * Reference count on the pinned pages table.
> > > +        * The pages can be evicted by memory shrinker
> > > +        * when the count reaches zero.
> > > +        */
> > > +       unsigned int pages_pin_count;
> > > +
> > >         /**
> > >          * @madv: State for madvise
> > >          *
> > >          * 0 is active/inuse.
> > > +        * 1 is not-needed/can-be-purged
> > >          * A negative value is the object is purged.
> > > -        * Positive values are driver specific and not used by the helpers.
> > >          */
> > >         int madv;
> > >
> > > @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
> > >          * @map_wc: map object write-combined (instead of using shmem defaults).
> > >          */
> > >         bool map_wc;
> > > +
> > > +       /**
> > > +        * @eviction_enabled:
> > > +        *
> > > +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> > > +        * Used internally by memory shrinker.
> > > +        */
> > > +       bool eviction_enabled;
> > > +
> > > +       /**
> > > +        * @purge_enabled:
> > > +        *
> > > +        * The shmem pages can be purged only if @purge_enabled is set to true.
> > > +        * Used internally by memory shrinker.
> > > +        */
> > > +       bool purge_enabled;
> > > +
> > > +       /**
> > > +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> > > +        * Used internally by memory shrinker.
> > > +        */
> > > +       bool evicted;
> > > +
> > > +       /**
> > > +        * @evict:
> > > +        *
> > > +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> > > +        * GEM's DMA reservation is kept locked by the shrinker. This is
> > > +        * optional callback that should be specified by drivers.
> > > +        *
> > > +        * Returns 0 on success, or -errno on error.
> > > +        */
> > > +       int (*evict)(struct drm_gem_shmem_object *shmem);
> > >  };
> > >
> > >  #define to_drm_gem_shmem_obj(obj) \
> > > @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
> > >
> > >  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
> > >
> > > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> > > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> > > +
> > >  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> > >  {
> > > -       return (shmem->madv > 0) &&
> > > -               !shmem->vmap_use_count && shmem->sgt &&
> > > -               !shmem->base.dma_buf && !shmem->base.import_attach;
> > > +       return (shmem->madv > 0) && shmem->evict &&
> > > +               shmem->purge_enabled && shmem->pages_use_count &&
> > > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > > +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
> > >  }
> > >
> > > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> > > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> > > +
> > > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> > > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> > >
> > >  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
> > >  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> > > @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
> > >         return drm_gem_shmem_mmap(shmem, vma);
> > >  }
> > >
> > > +/**
> > > + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> > > + */
> > > +struct drm_gem_shmem_shrinker {
> > > +       /** @base: Shrinker for purging shmem GEM objects */
> > > +       struct shrinker base;
> > > +
> > > +       /** @lock: Protects @lru_* */
> > > +       struct mutex lock;
> > > +
> > > +       /** @lru_pinned: List of pinned shmem GEM objects */
> > > +       struct list_head lru_pinned;
> > > +
> > > +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> > > +       struct list_head lru_evictable;
> > > +
> > > +       /** @lru_evicted: List of evicted shmem GEM objects */
> > > +       struct list_head lru_evicted;
> > > +
> > > +       /** @dev: DRM device that uses this shrinker */
> > > +       struct drm_device *dev;
> > > +};
> > > +
> > > +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> > > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> > > +
> > >  /*
> > >   * Driver ops
> > >   */
> > > --
> > > 2.35.3
> > >
> >
> >
> > --
> > Daniel Vetter
> > Software Engineer, Intel Corporation
> > http://blog.ffwll.ch



-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Sun, 5 Jun 2022 at 20:32, Rob Clark <robdclark@gmail.com> wrote:
>
> On Sun, Jun 5, 2022 at 9:47 AM Daniel Vetter <daniel@ffwll.ch> wrote:
> >
> > On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
> > <dmitry.osipenko@collabora.com> wrote:
> > >
> > > Introduce a common DRM SHMEM shrinker framework that allows to reduce
> > > code duplication among DRM drivers by replacing theirs custom shrinker
> > > implementations with the generic shrinker.
> > >
> > > In order to start using DRM SHMEM shrinker drivers should:
> > >
> > > 1. Implement new evict() shmem object callback.
> > > 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> > > 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
> > >    activate shrinking of shmem GEMs.
> > >
> > > This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> > > Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
> > >
> > > Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> > > Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> >
> > So I guess I get a price for being blind since forever, because this
> > thing existed since at least 2013. I just stumbled over
> > llist_lru.[hc], a purpose built list helper for shrinkers. I think we
> > should try to adopt that so that our gpu shrinkers look more like
> > shrinkers for everything else.
>
> followup from a bit of irc discussion w/ danvet about list_lru:
>
> * It seems to be missing a way to bail out of iteration before
>   nr_to_scan is hit.. which is going to be inconvenient if you
>   want to allow active bos on the LRU but bail scanning once
>   you encounter the first one.
>
> * Not sure if the numa node awareness is super useful for GEM
>   bos
>
> First issue is perhaps not too hard to fix.  But maybe a better
> idea is a drm_gem_lru helper type thing which is more tailored
> to GEM buffers?

Yeah I guess reusing list_lru isn't that good idea. So just
open-coding it for now, and then drm_gem_bo_lru or so if we need to
share it separately from shmem helpers with other drivers. Maybe will
be needed for ttm or so.
-Daniel

>
> BR,
> -R
>
> > Apologies for this, since I fear this might cause a bit of churn.
> > Hopefully it's all contained to the list manipulation code in shmem
> > helpers, I don't think this should leak any further.
> > -Daniel
> >
> > > ---
> > >  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
> > >  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
> > >  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
> > >  include/drm/drm_device.h                      |   4 +
> > >  include/drm/drm_gem_shmem_helper.h            |  87 ++-
> > >  5 files changed, 594 insertions(+), 49 deletions(-)
> > >
> > > diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > > index 555fe212bd98..4cd0b5913492 100644
> > > --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> > > +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > > @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
> > >  }
> > >  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
> > >
> > > +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       return (shmem->madv >= 0) && shmem->evict &&
> > > +               shmem->eviction_enabled && shmem->pages_use_count &&
> > > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > > +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> > > +}
> > > +
> > > +static void
> > > +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> > > +
> > > +       dma_resv_assert_held(shmem->base.resv);
> > > +
> > > +       if (!gem_shrinker || obj->import_attach)
> > > +               return;
> > > +
> > > +       mutex_lock(&gem_shrinker->lock);
> > > +
> > > +       if (drm_gem_shmem_is_evictable(shmem) ||
> > > +           drm_gem_shmem_is_purgeable(shmem))
> > > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> > > +       else if (shmem->madv < 0)
> > > +               list_del_init(&shmem->madv_list);
> > > +       else if (shmem->evicted)
> > > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> > > +       else if (!shmem->pages)
> > > +               list_del_init(&shmem->madv_list);
> > > +       else
> > > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> > > +
> > > +       mutex_unlock(&gem_shrinker->lock);
> > > +}
> > > +
> > >  /**
> > >   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
> > >   * @shmem: shmem GEM object to free
> > > @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> > >         } else {
> > >                 dma_resv_lock(shmem->base.resv, NULL);
> > >
> > > +               /* take out shmem GEM object from the memory shrinker */
> > > +               drm_gem_shmem_madvise(shmem, -1);
> > > +
> > >                 WARN_ON(shmem->vmap_use_count);
> > >
> > >                 if (shmem->sgt) {
> > > @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> > >                         sg_free_table(shmem->sgt);
> > >                         kfree(shmem->sgt);
> > >                 }
> > > -               if (shmem->pages)
> > > +               if (shmem->pages_use_count)
> > >                         drm_gem_shmem_put_pages(shmem);
> > >
> > >                 WARN_ON(shmem->pages_use_count);
> > > @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> > >  }
> > >  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
> > >
> > > -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > > +/**
> > > + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> > > + * @shmem: shmem GEM object
> > > + *
> > > + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> > > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       dma_resv_lock(shmem->base.resv, NULL);
> > > +
> > > +       if (shmem->madv < 0)
> > > +               return -ENOMEM;
> > > +
> > > +       shmem->eviction_enabled = true;
> > > +
> > > +       dma_resv_unlock(shmem->base.resv);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> > > +
> > > +/**
> > > + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> > > + * @shmem: shmem GEM object
> > > + *
> > > + * Tell memory shrinker that this GEM can be purged. Initially purging is
> > > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       dma_resv_lock(shmem->base.resv, NULL);
> > > +
> > > +       if (shmem->madv < 0)
> > > +               return -ENOMEM;
> > > +
> > > +       shmem->purge_enabled = true;
> > > +
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       dma_resv_unlock(shmem->base.resv);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> > > +
> > > +static int
> > > +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
> > >  {
> > >         struct drm_gem_object *obj = &shmem->base;
> > >         struct page **pages;
> > >
> > > -       if (shmem->pages_use_count++ > 0)
> > > +       dma_resv_assert_held(shmem->base.resv);
> > > +
> > > +       if (shmem->madv < 0) {
> > > +               WARN_ON(shmem->pages);
> > > +               return -ENOMEM;
> > > +       }
> > > +
> > > +       if (shmem->pages) {
> > > +               WARN_ON(!shmem->evicted);
> > >                 return 0;
> > > +       }
> > > +
> > > +       if (WARN_ON(!shmem->pages_use_count))
> > > +               return -EINVAL;
> > >
> > >         pages = drm_gem_get_pages(obj);
> > >         if (IS_ERR(pages)) {
> > >                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> > > -               shmem->pages_use_count = 0;
> > >                 return PTR_ERR(pages);
> > >         }
> > >
> > > @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > >         return 0;
> > >  }
> > >
> > > +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       int err;
> > > +
> > > +       dma_resv_assert_held(shmem->base.resv);
> > > +
> > > +       if (shmem->madv < 0)
> > > +               return -ENOMEM;
> > > +
> > > +       if (shmem->pages_use_count++ > 0) {
> > > +               err = drm_gem_shmem_swap_in(shmem);
> > > +               if (err)
> > > +                       goto err_zero_use;
> > > +
> > > +               return 0;
> > > +       }
> > > +
> > > +       err = drm_gem_shmem_acquire_pages(shmem);
> > > +       if (err)
> > > +               goto err_zero_use;
> > > +
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       return 0;
> > > +
> > > +err_zero_use:
> > > +       shmem->pages_use_count = 0;
> > > +
> > > +       return err;
> > > +}
> > > +
> > > +static void
> > > +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +
> > > +       if (!shmem->pages) {
> > > +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> > > +               return;
> > > +       }
> > > +
> > > +#ifdef CONFIG_X86
> > > +       if (shmem->map_wc)
> > > +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > > +#endif
> > > +
> > > +       drm_gem_put_pages(obj, shmem->pages,
> > > +                         shmem->pages_mark_dirty_on_put,
> > > +                         shmem->pages_mark_accessed_on_put);
> > > +       shmem->pages = NULL;
> > > +}
> > > +
> > >  /*
> > >   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
> > >   * @shmem: shmem GEM object
> > > @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > >   */
> > >  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> > >  {
> > > -       struct drm_gem_object *obj = &shmem->base;
> > > -
> > >         dma_resv_assert_held(shmem->base.resv);
> > >
> > >         if (WARN_ON_ONCE(!shmem->pages_use_count))
> > > @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> > >         if (--shmem->pages_use_count > 0)
> > >                 return;
> > >
> > > -#ifdef CONFIG_X86
> > > -       if (shmem->map_wc)
> > > -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > > -#endif
> > > +       drm_gem_shmem_release_pages(shmem);
> > >
> > > -       drm_gem_put_pages(obj, shmem->pages,
> > > -                         shmem->pages_mark_dirty_on_put,
> > > -                         shmem->pages_mark_accessed_on_put);
> > > -       shmem->pages = NULL;
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> > >
> > > @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> > >   */
> > >  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
> > >  {
> > > +       int ret;
> > > +
> > >         dma_resv_assert_held(shmem->base.resv);
> > >
> > >         WARN_ON(shmem->base.import_attach);
> > >
> > > -       return drm_gem_shmem_get_pages(shmem);
> > > +       ret = drm_gem_shmem_get_pages(shmem);
> > > +       if (!ret)
> > > +               shmem->pages_pin_count++;
> > > +
> > > +       return ret;
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_pin);
> > >
> > > @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
> > >         WARN_ON(shmem->base.import_attach);
> > >
> > >         drm_gem_shmem_put_pages(shmem);
> > > +
> > > +       shmem->pages_pin_count--;
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_unpin);
> > >
> > > @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> > >                         return 0;
> > >                 }
> > >
> > > -               ret = drm_gem_shmem_get_pages(shmem);
> > > +               ret = drm_gem_shmem_pin(shmem);
> > >                 if (ret)
> > >                         goto err_zero_use;
> > >
> > > @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> > >
> > >  err_put_pages:
> > >         if (!obj->import_attach)
> > > -               drm_gem_shmem_put_pages(shmem);
> > > +               drm_gem_shmem_unpin(shmem);
> > >  err_zero_use:
> > >         shmem->vmap_use_count = 0;
> > >
> > > @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
> > >                         return;
> > >
> > >                 vunmap(shmem->vaddr);
> > > -               drm_gem_shmem_put_pages(shmem);
> > > +               drm_gem_shmem_unpin(shmem);
> > >         }
> > >
> > >         shmem->vaddr = NULL;
> > > @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
> > >
> > >         madv = shmem->madv;
> > >
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > >         return (madv >= 0);
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_madvise);
> > >
> > > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > > +/**
> > > + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> > > + *                           hardware access to the memory.
> > > + * @shmem: shmem GEM object
> > > + *
> > > + * This function moves shmem GEM back to memory if it was previously evicted
> > > + * by the memory shrinker. The GEM is ready to use on success.
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
> > >  {
> > >         struct drm_gem_object *obj = &shmem->base;
> > > -       struct drm_device *dev = obj->dev;
> > > +       struct sg_table *sgt;
> > > +       int err;
> > >
> > >         dma_resv_assert_held(shmem->base.resv);
> > >
> > > -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > > +       if (shmem->evicted) {
> > > +               err = drm_gem_shmem_acquire_pages(shmem);
> > > +               if (err)
> > > +                       return err;
> > > +
> > > +               sgt = drm_gem_shmem_get_sg_table(shmem);
> > > +               if (IS_ERR(sgt))
> > > +                       return PTR_ERR(sgt);
> > > +
> > > +               err = dma_map_sgtable(obj->dev->dev, sgt,
> > > +                                     DMA_BIDIRECTIONAL, 0);
> > > +               if (err) {
> > > +                       sg_free_table(sgt);
> > > +                       kfree(sgt);
> > > +                       return err;
> > > +               }
> > >
> > > -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > > -       sg_free_table(shmem->sgt);
> > > -       kfree(shmem->sgt);
> > > -       shmem->sgt = NULL;
> > > +               shmem->sgt = sgt;
> > > +               shmem->evicted = false;
> > >
> > > -       drm_gem_shmem_put_pages(shmem);
> > > +               drm_gem_shmem_update_pages_state(shmem);
> > > +       }
> > >
> > > -       shmem->madv = -1;
> > > +       if (!shmem->pages)
> > > +               return -ENOMEM;
> > >
> > > -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > > -       drm_gem_free_mmap_offset(obj);
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
> > >
> > > -       /* Our goal here is to return as much of the memory as
> > > -        * is possible back to the system as we are called from OOM.
> > > -        * To do this we must instruct the shmfs to drop all of its
> > > -        * backing pages, *now*.
> > > -        */
> > > -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > > +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +       struct drm_device *dev = obj->dev;
> > >
> > > -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > > +       if (shmem->evicted)
> > > +               return;
> > > +
> > > +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > > +       drm_gem_shmem_release_pages(shmem);
> > > +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > > +
> > > +       sg_free_table(shmem->sgt);
> > > +       kfree(shmem->sgt);
> > > +       shmem->sgt = NULL;
> > >  }
> > > -EXPORT_SYMBOL(drm_gem_shmem_purge);
> > >
> > >  /**
> > >   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> > > @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
> > >         vm_fault_t ret;
> > >         struct page *page;
> > >         pgoff_t page_offset;
> > > +       bool pages_unpinned;
> > > +       int err;
> > >
> > >         /* We don't use vmf->pgoff since that has the fake offset */
> > >         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
> > >
> > >         dma_resv_lock(shmem->base.resv, NULL);
> > >
> > > -       if (page_offset >= num_pages ||
> > > -           WARN_ON_ONCE(!shmem->pages) ||
> > > -           shmem->madv < 0) {
> > > +       /* Sanity-check that we have the pages pointer when it should present */
> > > +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> > > +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> > > +
> > > +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
> > >                 ret = VM_FAULT_SIGBUS;
> > >         } else {
> > > +               err = drm_gem_shmem_swap_in(shmem);
> > > +               if (err) {
> > > +                       ret = VM_FAULT_OOM;
> > > +                       goto unlock;
> > > +               }
> > > +
> > >                 page = shmem->pages[page_offset];
> > >
> > >                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
> > >         }
> > >
> > > +unlock:
> > >         dma_resv_unlock(shmem->base.resv);
> > >
> > >         return ret;
> > > @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
> > >  {
> > >         struct drm_gem_object *obj = vma->vm_private_data;
> > >         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> > > -       int ret;
> > >
> > >         WARN_ON(shmem->base.import_attach);
> > >
> > >         dma_resv_lock(shmem->base.resv, NULL);
> > > -       ret = drm_gem_shmem_get_pages(shmem);
> > > -       WARN_ON_ONCE(ret != 0);
> > > +
> > > +       if (drm_gem_shmem_get_pages(shmem))
> > > +               shmem->pages_use_count++;
> > > +
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > >         dma_resv_unlock(shmem->base.resv);
> > >
> > >         drm_gem_vm_open(vma);
> > > @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
> > >  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> > >                               struct drm_printer *p, unsigned int indent)
> > >  {
> > > +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> > > +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
> > >         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
> > >
> > >         if (shmem->base.import_attach)
> > > @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> > >                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
> > >                                   shmem->vmap_use_count);
> > >
> > > +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
> > >         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> > > +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_print_info);
> > >
> > > @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
> > >
> > >         shmem->sgt = sgt;
> > >
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > >         dma_resv_unlock(shmem->base.resv);
> > >
> > >         return sgt;
> > > @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
> > >  }
> > >  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
> > >
> > > +static struct drm_gem_shmem_shrinker *
> > > +to_drm_shrinker(struct shrinker *shrinker)
> > > +{
> > > +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> > > +}
> > > +
> > > +static unsigned long
> > > +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> > > +                                    struct shrink_control *sc)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > > +       struct drm_gem_shmem_object *shmem;
> > > +       unsigned long count = 0;
> > > +
> > > +       if (!mutex_trylock(&gem_shrinker->lock))
> > > +               return 0;
> > > +
> > > +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> > > +               count += shmem->base.size;
> > > +
> > > +               if (count >= SHRINK_EMPTY)
> > > +                       break;
> > > +       }
> > > +
> > > +       mutex_unlock(&gem_shrinker->lock);
> > > +
> > > +       if (count >= SHRINK_EMPTY)
> > > +               return SHRINK_EMPTY - 1;
> > > +
> > > +       return count ?: SHRINK_EMPTY;
> > > +}
> > > +
> > > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> > > +       WARN_ON(shmem->evicted);
> > > +
> > > +       drm_gem_shmem_unpin_pages(shmem);
> > > +
> > > +       shmem->evicted = true;
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> > > +
> > > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +
> > > +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > > +
> > > +       drm_gem_shmem_unpin_pages(shmem);
> > > +       drm_gem_free_mmap_offset(obj);
> > > +
> > > +       /* Our goal here is to return as much of the memory as
> > > +        * is possible back to the system as we are called from OOM.
> > > +        * To do this we must instruct the shmfs to drop all of its
> > > +        * backing pages, *now*.
> > > +        */
> > > +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > > +
> > > +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > > +
> > > +       shmem->madv = -1;
> > > +       shmem->evicted = false;
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> > > +
> > > +static unsigned long
> > > +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> > > +                                       unsigned long nr_to_scan,
> > > +                                       bool *lock_contention,
> > > +                                       bool evict)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > > +       struct drm_gem_shmem_object *shmem;
> > > +       struct list_head still_in_list;
> > > +       struct drm_gem_object *obj;
> > > +       unsigned long freed = 0;
> > > +       size_t page_count;
> > > +       int err;
> > > +
> > > +       INIT_LIST_HEAD(&still_in_list);
> > > +
> > > +       mutex_lock(&gem_shrinker->lock);
> > > +
> > > +       while (freed < nr_to_scan) {
> > > +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> > > +                                                typeof(*shmem), madv_list);
> > > +               if (!shmem)
> > > +                       break;
> > > +
> > > +               obj = &shmem->base;
> > > +               page_count = obj->size >> PAGE_SHIFT;
> > > +               list_move_tail(&shmem->madv_list, &still_in_list);
> > > +
> > > +               if (evict) {
> > > +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> > > +                           get_nr_swap_pages() < page_count)
> > > +                               continue;
> > > +               } else {
> > > +                       if (!drm_gem_shmem_is_purgeable(shmem))
> > > +                               continue;
> > > +               }
> > > +
> > > +               /*
> > > +                * If it's in the process of being freed, gem_object->free()
> > > +                * may be blocked on lock waiting to remove it.  So just
> > > +                * skip it.
> > > +                */
> > > +               if (!kref_get_unless_zero(&obj->refcount))
> > > +                       continue;
> > > +
> > > +               mutex_unlock(&gem_shrinker->lock);
> > > +
> > > +               /* prevent racing with job-submission code paths */
> > > +               if (!dma_resv_trylock(obj->resv)) {
> > > +                       *lock_contention |= true;
> > > +                       goto shrinker_lock;
> > > +               }
> > > +
> > > +               /* prevent racing with the dma-buf importing/exporting */
> > > +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > > +                       *lock_contention |= true;
> > > +                       goto resv_unlock;
> > > +               }
> > > +
> > > +               /* check whether h/w uses this object */
> > > +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> > > +                       goto object_name_unlock;
> > > +
> > > +               /* re-check whether eviction status hasn't changed */
> > > +               if (!drm_gem_shmem_is_evictable(shmem) &&
> > > +                   !drm_gem_shmem_is_purgeable(shmem))
> > > +                       goto object_name_unlock;
> > > +
> > > +               err = shmem->evict(shmem);
> > > +               if (!err)
> > > +                       freed += obj->size >> PAGE_SHIFT;
> > > +
> > > +object_name_unlock:
> > > +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> > > +resv_unlock:
> > > +               dma_resv_unlock(obj->resv);
> > > +shrinker_lock:
> > > +               drm_gem_object_put(&shmem->base);
> > > +               mutex_lock(&gem_shrinker->lock);
> > > +       }
> > > +
> > > +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> > > +
> > > +       mutex_unlock(&gem_shrinker->lock);
> > > +
> > > +       return freed;
> > > +}
> > > +
> > > +static unsigned long
> > > +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> > > +                                   struct shrink_control *sc)
> > > +{
> > > +       unsigned long nr_to_scan = sc->nr_to_scan;
> > > +       bool lock_contention = false;
> > > +       unsigned long freed;
> > > +
> > > +       /* purge as many objects as we can */
> > > +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> > > +                                                       &lock_contention, false);
> > > +
> > > +       /* evict as many objects as we can */
> > > +       if (freed < nr_to_scan)
> > > +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> > > +                                                                nr_to_scan - freed,
> > > +                                                                &lock_contention,
> > > +                                                                true);
> > > +
> > > +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> > > +}
> > > +
> > > +/**
> > > + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> > > + * @dev: DRM device
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker;
> > > +       int err;
> > > +
> > > +       if (WARN_ON(dev->shmem_shrinker))
> > > +               return -EBUSY;
> > > +
> > > +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> > > +       if (!gem_shrinker)
> > > +               return -ENOMEM;
> > > +
> > > +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> > > +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> > > +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> > > +       gem_shrinker->dev = dev;
> > > +
> > > +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> > > +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> > > +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> > > +       mutex_init(&gem_shrinker->lock);
> > > +
> > > +       dev->shmem_shrinker = gem_shrinker;
> > > +
> > > +       err = register_shrinker(&gem_shrinker->base);
> > > +       if (err) {
> > > +               dev->shmem_shrinker = NULL;
> > > +               kfree(gem_shrinker);
> > > +               return err;
> > > +       }
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> > > +
> > > +/**
> > > + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> > > + * @dev: DRM device
> > > + */
> > > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> > > +
> > > +       if (gem_shrinker) {
> > > +               unregister_shrinker(&gem_shrinker->base);
> > > +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> > > +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> > > +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> > > +               mutex_destroy(&gem_shrinker->lock);
> > > +               dev->shmem_shrinker = NULL;
> > > +               kfree(gem_shrinker);
> > > +       }
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> > > +
> > >  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
> > >  MODULE_IMPORT_NS(DMA_BUF);
> > >  MODULE_LICENSE("GPL v2");
> > > diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > > index a4bedfeb2ec4..7cc32556f908 100644
> > > --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > > +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > > @@ -15,6 +15,13 @@
> > >  #include "panfrost_gem.h"
> > >  #include "panfrost_mmu.h"
> > >
> > > +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       return (shmem->madv > 0) &&
> > > +               !shmem->pages_pin_count && shmem->sgt &&
> > > +               !shmem->base.dma_buf && !shmem->base.import_attach;
> > > +}
> > > +
> > >  static unsigned long
> > >  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
> > >  {
> > > @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
> > >                 return 0;
> > >
> > >         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> > > -               if (drm_gem_shmem_is_purgeable(shmem))
> > > +               if (panfrost_gem_shmem_is_purgeable(shmem))
> > >                         count += shmem->base.size >> PAGE_SHIFT;
> > >         }
> > >
> > > diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > > index b2d93cb12ebf..81bacc7e1873 100644
> > > --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> > > +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > > @@ -89,6 +89,7 @@ struct virtio_gpu_object {
> > >         uint32_t hw_res_handle;
> > >         bool dumb;
> > >         bool created;
> > > +       bool detached;
> > >         bool host3d_blob, guest_blob;
> > >         uint32_t blob_mem, blob_flags;
> > >
> > > @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
> > >
> > >  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
> > >
> > > +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> > > +
> > >  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
> > >                                uint32_t *resid);
> > >  /* virtgpu_prime.c */
> > > diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> > > index 9923c7a6885e..929546cad894 100644
> > > --- a/include/drm/drm_device.h
> > > +++ b/include/drm/drm_device.h
> > > @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
> > >  struct drm_vma_offset_manager;
> > >  struct drm_vram_mm;
> > >  struct drm_fb_helper;
> > > +struct drm_gem_shmem_shrinker;
> > >
> > >  struct inode;
> > >
> > > @@ -277,6 +278,9 @@ struct drm_device {
> > >         /** @vram_mm: VRAM MM memory manager */
> > >         struct drm_vram_mm *vram_mm;
> > >
> > > +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> > > +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> > > +
> > >         /**
> > >          * @switch_power_state:
> > >          *
> > > diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> > > index 9a8983ee8abe..62c640678a91 100644
> > > --- a/include/drm/drm_gem_shmem_helper.h
> > > +++ b/include/drm/drm_gem_shmem_helper.h
> > > @@ -6,6 +6,7 @@
> > >  #include <linux/fs.h>
> > >  #include <linux/mm.h>
> > >  #include <linux/mutex.h>
> > > +#include <linux/shrinker.h>
> > >
> > >  #include <drm/drm_file.h>
> > >  #include <drm/drm_gem.h>
> > > @@ -15,6 +16,7 @@
> > >  struct dma_buf_attachment;
> > >  struct drm_mode_create_dumb;
> > >  struct drm_printer;
> > > +struct drm_device;
> > >  struct sg_table;
> > >
> > >  /**
> > > @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
> > >          */
> > >         unsigned int pages_use_count;
> > >
> > > +       /**
> > > +        * @pages_pin_count:
> > > +        *
> > > +        * Reference count on the pinned pages table.
> > > +        * The pages can be evicted by memory shrinker
> > > +        * when the count reaches zero.
> > > +        */
> > > +       unsigned int pages_pin_count;
> > > +
> > >         /**
> > >          * @madv: State for madvise
> > >          *
> > >          * 0 is active/inuse.
> > > +        * 1 is not-needed/can-be-purged
> > >          * A negative value is the object is purged.
> > > -        * Positive values are driver specific and not used by the helpers.
> > >          */
> > >         int madv;
> > >
> > > @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
> > >          * @map_wc: map object write-combined (instead of using shmem defaults).
> > >          */
> > >         bool map_wc;
> > > +
> > > +       /**
> > > +        * @eviction_enabled:
> > > +        *
> > > +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> > > +        * Used internally by memory shrinker.
> > > +        */
> > > +       bool eviction_enabled;
> > > +
> > > +       /**
> > > +        * @purge_enabled:
> > > +        *
> > > +        * The shmem pages can be purged only if @purge_enabled is set to true.
> > > +        * Used internally by memory shrinker.
> > > +        */
> > > +       bool purge_enabled;
> > > +
> > > +       /**
> > > +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> > > +        * Used internally by memory shrinker.
> > > +        */
> > > +       bool evicted;
> > > +
> > > +       /**
> > > +        * @evict:
> > > +        *
> > > +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> > > +        * GEM's DMA reservation is kept locked by the shrinker. This is
> > > +        * optional callback that should be specified by drivers.
> > > +        *
> > > +        * Returns 0 on success, or -errno on error.
> > > +        */
> > > +       int (*evict)(struct drm_gem_shmem_object *shmem);
> > >  };
> > >
> > >  #define to_drm_gem_shmem_obj(obj) \
> > > @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
> > >
> > >  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
> > >
> > > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> > > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> > > +
> > >  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> > >  {
> > > -       return (shmem->madv > 0) &&
> > > -               !shmem->vmap_use_count && shmem->sgt &&
> > > -               !shmem->base.dma_buf && !shmem->base.import_attach;
> > > +       return (shmem->madv > 0) && shmem->evict &&
> > > +               shmem->purge_enabled && shmem->pages_use_count &&
> > > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > > +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
> > >  }
> > >
> > > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> > > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> > > +
> > > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> > > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> > >
> > >  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
> > >  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> > > @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
> > >         return drm_gem_shmem_mmap(shmem, vma);
> > >  }
> > >
> > > +/**
> > > + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> > > + */
> > > +struct drm_gem_shmem_shrinker {
> > > +       /** @base: Shrinker for purging shmem GEM objects */
> > > +       struct shrinker base;
> > > +
> > > +       /** @lock: Protects @lru_* */
> > > +       struct mutex lock;
> > > +
> > > +       /** @lru_pinned: List of pinned shmem GEM objects */
> > > +       struct list_head lru_pinned;
> > > +
> > > +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> > > +       struct list_head lru_evictable;
> > > +
> > > +       /** @lru_evicted: List of evicted shmem GEM objects */
> > > +       struct list_head lru_evicted;
> > > +
> > > +       /** @dev: DRM device that uses this shrinker */
> > > +       struct drm_device *dev;
> > > +};
> > > +
> > > +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> > > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> > > +
> > >  /*
> > >   * Driver ops
> > >   */
> > > --
> > > 2.35.3
> > >
> >
> >
> > --
> > Daniel Vetter
> > Software Engineer, Intel Corporation
> > http://blog.ffwll.ch



-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Sun, 5 Jun 2022 at 20:32, Rob Clark <robdclark@gmail.com> wrote:
>
> On Sun, Jun 5, 2022 at 9:47 AM Daniel Vetter <daniel@ffwll.ch> wrote:
> >
> > On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
> > <dmitry.osipenko@collabora.com> wrote:
> > >
> > > Introduce a common DRM SHMEM shrinker framework that allows to reduce
> > > code duplication among DRM drivers by replacing theirs custom shrinker
> > > implementations with the generic shrinker.
> > >
> > > In order to start using DRM SHMEM shrinker drivers should:
> > >
> > > 1. Implement new evict() shmem object callback.
> > > 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> > > 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
> > >    activate shrinking of shmem GEMs.
> > >
> > > This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> > > Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
> > >
> > > Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> > > Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> >
> > So I guess I get a price for being blind since forever, because this
> > thing existed since at least 2013. I just stumbled over
> > llist_lru.[hc], a purpose built list helper for shrinkers. I think we
> > should try to adopt that so that our gpu shrinkers look more like
> > shrinkers for everything else.
>
> followup from a bit of irc discussion w/ danvet about list_lru:
>
> * It seems to be missing a way to bail out of iteration before
>   nr_to_scan is hit.. which is going to be inconvenient if you
>   want to allow active bos on the LRU but bail scanning once
>   you encounter the first one.
>
> * Not sure if the numa node awareness is super useful for GEM
>   bos
>
> First issue is perhaps not too hard to fix.  But maybe a better
> idea is a drm_gem_lru helper type thing which is more tailored
> to GEM buffers?

Yeah I guess reusing list_lru isn't that good idea. So just
open-coding it for now, and then drm_gem_bo_lru or so if we need to
share it separately from shmem helpers with other drivers. Maybe will
be needed for ttm or so.
-Daniel

>
> BR,
> -R
>
> > Apologies for this, since I fear this might cause a bit of churn.
> > Hopefully it's all contained to the list manipulation code in shmem
> > helpers, I don't think this should leak any further.
> > -Daniel
> >
> > > ---
> > >  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
> > >  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
> > >  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
> > >  include/drm/drm_device.h                      |   4 +
> > >  include/drm/drm_gem_shmem_helper.h            |  87 ++-
> > >  5 files changed, 594 insertions(+), 49 deletions(-)
> > >
> > > diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > > index 555fe212bd98..4cd0b5913492 100644
> > > --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> > > +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > > @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
> > >  }
> > >  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
> > >
> > > +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       return (shmem->madv >= 0) && shmem->evict &&
> > > +               shmem->eviction_enabled && shmem->pages_use_count &&
> > > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > > +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> > > +}
> > > +
> > > +static void
> > > +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> > > +
> > > +       dma_resv_assert_held(shmem->base.resv);
> > > +
> > > +       if (!gem_shrinker || obj->import_attach)
> > > +               return;
> > > +
> > > +       mutex_lock(&gem_shrinker->lock);
> > > +
> > > +       if (drm_gem_shmem_is_evictable(shmem) ||
> > > +           drm_gem_shmem_is_purgeable(shmem))
> > > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> > > +       else if (shmem->madv < 0)
> > > +               list_del_init(&shmem->madv_list);
> > > +       else if (shmem->evicted)
> > > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> > > +       else if (!shmem->pages)
> > > +               list_del_init(&shmem->madv_list);
> > > +       else
> > > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> > > +
> > > +       mutex_unlock(&gem_shrinker->lock);
> > > +}
> > > +
> > >  /**
> > >   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
> > >   * @shmem: shmem GEM object to free
> > > @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> > >         } else {
> > >                 dma_resv_lock(shmem->base.resv, NULL);
> > >
> > > +               /* take out shmem GEM object from the memory shrinker */
> > > +               drm_gem_shmem_madvise(shmem, -1);
> > > +
> > >                 WARN_ON(shmem->vmap_use_count);
> > >
> > >                 if (shmem->sgt) {
> > > @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> > >                         sg_free_table(shmem->sgt);
> > >                         kfree(shmem->sgt);
> > >                 }
> > > -               if (shmem->pages)
> > > +               if (shmem->pages_use_count)
> > >                         drm_gem_shmem_put_pages(shmem);
> > >
> > >                 WARN_ON(shmem->pages_use_count);
> > > @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> > >  }
> > >  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
> > >
> > > -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > > +/**
> > > + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> > > + * @shmem: shmem GEM object
> > > + *
> > > + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> > > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       dma_resv_lock(shmem->base.resv, NULL);
> > > +
> > > +       if (shmem->madv < 0)
> > > +               return -ENOMEM;
> > > +
> > > +       shmem->eviction_enabled = true;
> > > +
> > > +       dma_resv_unlock(shmem->base.resv);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> > > +
> > > +/**
> > > + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> > > + * @shmem: shmem GEM object
> > > + *
> > > + * Tell memory shrinker that this GEM can be purged. Initially purging is
> > > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       dma_resv_lock(shmem->base.resv, NULL);
> > > +
> > > +       if (shmem->madv < 0)
> > > +               return -ENOMEM;
> > > +
> > > +       shmem->purge_enabled = true;
> > > +
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       dma_resv_unlock(shmem->base.resv);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> > > +
> > > +static int
> > > +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
> > >  {
> > >         struct drm_gem_object *obj = &shmem->base;
> > >         struct page **pages;
> > >
> > > -       if (shmem->pages_use_count++ > 0)
> > > +       dma_resv_assert_held(shmem->base.resv);
> > > +
> > > +       if (shmem->madv < 0) {
> > > +               WARN_ON(shmem->pages);
> > > +               return -ENOMEM;
> > > +       }
> > > +
> > > +       if (shmem->pages) {
> > > +               WARN_ON(!shmem->evicted);
> > >                 return 0;
> > > +       }
> > > +
> > > +       if (WARN_ON(!shmem->pages_use_count))
> > > +               return -EINVAL;
> > >
> > >         pages = drm_gem_get_pages(obj);
> > >         if (IS_ERR(pages)) {
> > >                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> > > -               shmem->pages_use_count = 0;
> > >                 return PTR_ERR(pages);
> > >         }
> > >
> > > @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > >         return 0;
> > >  }
> > >
> > > +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       int err;
> > > +
> > > +       dma_resv_assert_held(shmem->base.resv);
> > > +
> > > +       if (shmem->madv < 0)
> > > +               return -ENOMEM;
> > > +
> > > +       if (shmem->pages_use_count++ > 0) {
> > > +               err = drm_gem_shmem_swap_in(shmem);
> > > +               if (err)
> > > +                       goto err_zero_use;
> > > +
> > > +               return 0;
> > > +       }
> > > +
> > > +       err = drm_gem_shmem_acquire_pages(shmem);
> > > +       if (err)
> > > +               goto err_zero_use;
> > > +
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       return 0;
> > > +
> > > +err_zero_use:
> > > +       shmem->pages_use_count = 0;
> > > +
> > > +       return err;
> > > +}
> > > +
> > > +static void
> > > +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +
> > > +       if (!shmem->pages) {
> > > +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> > > +               return;
> > > +       }
> > > +
> > > +#ifdef CONFIG_X86
> > > +       if (shmem->map_wc)
> > > +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > > +#endif
> > > +
> > > +       drm_gem_put_pages(obj, shmem->pages,
> > > +                         shmem->pages_mark_dirty_on_put,
> > > +                         shmem->pages_mark_accessed_on_put);
> > > +       shmem->pages = NULL;
> > > +}
> > > +
> > >  /*
> > >   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
> > >   * @shmem: shmem GEM object
> > > @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > >   */
> > >  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> > >  {
> > > -       struct drm_gem_object *obj = &shmem->base;
> > > -
> > >         dma_resv_assert_held(shmem->base.resv);
> > >
> > >         if (WARN_ON_ONCE(!shmem->pages_use_count))
> > > @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> > >         if (--shmem->pages_use_count > 0)
> > >                 return;
> > >
> > > -#ifdef CONFIG_X86
> > > -       if (shmem->map_wc)
> > > -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > > -#endif
> > > +       drm_gem_shmem_release_pages(shmem);
> > >
> > > -       drm_gem_put_pages(obj, shmem->pages,
> > > -                         shmem->pages_mark_dirty_on_put,
> > > -                         shmem->pages_mark_accessed_on_put);
> > > -       shmem->pages = NULL;
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> > >
> > > @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> > >   */
> > >  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
> > >  {
> > > +       int ret;
> > > +
> > >         dma_resv_assert_held(shmem->base.resv);
> > >
> > >         WARN_ON(shmem->base.import_attach);
> > >
> > > -       return drm_gem_shmem_get_pages(shmem);
> > > +       ret = drm_gem_shmem_get_pages(shmem);
> > > +       if (!ret)
> > > +               shmem->pages_pin_count++;
> > > +
> > > +       return ret;
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_pin);
> > >
> > > @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
> > >         WARN_ON(shmem->base.import_attach);
> > >
> > >         drm_gem_shmem_put_pages(shmem);
> > > +
> > > +       shmem->pages_pin_count--;
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_unpin);
> > >
> > > @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> > >                         return 0;
> > >                 }
> > >
> > > -               ret = drm_gem_shmem_get_pages(shmem);
> > > +               ret = drm_gem_shmem_pin(shmem);
> > >                 if (ret)
> > >                         goto err_zero_use;
> > >
> > > @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> > >
> > >  err_put_pages:
> > >         if (!obj->import_attach)
> > > -               drm_gem_shmem_put_pages(shmem);
> > > +               drm_gem_shmem_unpin(shmem);
> > >  err_zero_use:
> > >         shmem->vmap_use_count = 0;
> > >
> > > @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
> > >                         return;
> > >
> > >                 vunmap(shmem->vaddr);
> > > -               drm_gem_shmem_put_pages(shmem);
> > > +               drm_gem_shmem_unpin(shmem);
> > >         }
> > >
> > >         shmem->vaddr = NULL;
> > > @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
> > >
> > >         madv = shmem->madv;
> > >
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > >         return (madv >= 0);
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_madvise);
> > >
> > > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > > +/**
> > > + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> > > + *                           hardware access to the memory.
> > > + * @shmem: shmem GEM object
> > > + *
> > > + * This function moves shmem GEM back to memory if it was previously evicted
> > > + * by the memory shrinker. The GEM is ready to use on success.
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
> > >  {
> > >         struct drm_gem_object *obj = &shmem->base;
> > > -       struct drm_device *dev = obj->dev;
> > > +       struct sg_table *sgt;
> > > +       int err;
> > >
> > >         dma_resv_assert_held(shmem->base.resv);
> > >
> > > -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > > +       if (shmem->evicted) {
> > > +               err = drm_gem_shmem_acquire_pages(shmem);
> > > +               if (err)
> > > +                       return err;
> > > +
> > > +               sgt = drm_gem_shmem_get_sg_table(shmem);
> > > +               if (IS_ERR(sgt))
> > > +                       return PTR_ERR(sgt);
> > > +
> > > +               err = dma_map_sgtable(obj->dev->dev, sgt,
> > > +                                     DMA_BIDIRECTIONAL, 0);
> > > +               if (err) {
> > > +                       sg_free_table(sgt);
> > > +                       kfree(sgt);
> > > +                       return err;
> > > +               }
> > >
> > > -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > > -       sg_free_table(shmem->sgt);
> > > -       kfree(shmem->sgt);
> > > -       shmem->sgt = NULL;
> > > +               shmem->sgt = sgt;
> > > +               shmem->evicted = false;
> > >
> > > -       drm_gem_shmem_put_pages(shmem);
> > > +               drm_gem_shmem_update_pages_state(shmem);
> > > +       }
> > >
> > > -       shmem->madv = -1;
> > > +       if (!shmem->pages)
> > > +               return -ENOMEM;
> > >
> > > -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > > -       drm_gem_free_mmap_offset(obj);
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
> > >
> > > -       /* Our goal here is to return as much of the memory as
> > > -        * is possible back to the system as we are called from OOM.
> > > -        * To do this we must instruct the shmfs to drop all of its
> > > -        * backing pages, *now*.
> > > -        */
> > > -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > > +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +       struct drm_device *dev = obj->dev;
> > >
> > > -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > > +       if (shmem->evicted)
> > > +               return;
> > > +
> > > +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > > +       drm_gem_shmem_release_pages(shmem);
> > > +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > > +
> > > +       sg_free_table(shmem->sgt);
> > > +       kfree(shmem->sgt);
> > > +       shmem->sgt = NULL;
> > >  }
> > > -EXPORT_SYMBOL(drm_gem_shmem_purge);
> > >
> > >  /**
> > >   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> > > @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
> > >         vm_fault_t ret;
> > >         struct page *page;
> > >         pgoff_t page_offset;
> > > +       bool pages_unpinned;
> > > +       int err;
> > >
> > >         /* We don't use vmf->pgoff since that has the fake offset */
> > >         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
> > >
> > >         dma_resv_lock(shmem->base.resv, NULL);
> > >
> > > -       if (page_offset >= num_pages ||
> > > -           WARN_ON_ONCE(!shmem->pages) ||
> > > -           shmem->madv < 0) {
> > > +       /* Sanity-check that we have the pages pointer when it should present */
> > > +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> > > +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> > > +
> > > +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
> > >                 ret = VM_FAULT_SIGBUS;
> > >         } else {
> > > +               err = drm_gem_shmem_swap_in(shmem);
> > > +               if (err) {
> > > +                       ret = VM_FAULT_OOM;
> > > +                       goto unlock;
> > > +               }
> > > +
> > >                 page = shmem->pages[page_offset];
> > >
> > >                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
> > >         }
> > >
> > > +unlock:
> > >         dma_resv_unlock(shmem->base.resv);
> > >
> > >         return ret;
> > > @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
> > >  {
> > >         struct drm_gem_object *obj = vma->vm_private_data;
> > >         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> > > -       int ret;
> > >
> > >         WARN_ON(shmem->base.import_attach);
> > >
> > >         dma_resv_lock(shmem->base.resv, NULL);
> > > -       ret = drm_gem_shmem_get_pages(shmem);
> > > -       WARN_ON_ONCE(ret != 0);
> > > +
> > > +       if (drm_gem_shmem_get_pages(shmem))
> > > +               shmem->pages_use_count++;
> > > +
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > >         dma_resv_unlock(shmem->base.resv);
> > >
> > >         drm_gem_vm_open(vma);
> > > @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
> > >  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> > >                               struct drm_printer *p, unsigned int indent)
> > >  {
> > > +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> > > +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
> > >         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
> > >
> > >         if (shmem->base.import_attach)
> > > @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> > >                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
> > >                                   shmem->vmap_use_count);
> > >
> > > +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
> > >         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> > > +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_print_info);
> > >
> > > @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
> > >
> > >         shmem->sgt = sgt;
> > >
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > >         dma_resv_unlock(shmem->base.resv);
> > >
> > >         return sgt;
> > > @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
> > >  }
> > >  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
> > >
> > > +static struct drm_gem_shmem_shrinker *
> > > +to_drm_shrinker(struct shrinker *shrinker)
> > > +{
> > > +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> > > +}
> > > +
> > > +static unsigned long
> > > +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> > > +                                    struct shrink_control *sc)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > > +       struct drm_gem_shmem_object *shmem;
> > > +       unsigned long count = 0;
> > > +
> > > +       if (!mutex_trylock(&gem_shrinker->lock))
> > > +               return 0;
> > > +
> > > +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> > > +               count += shmem->base.size;
> > > +
> > > +               if (count >= SHRINK_EMPTY)
> > > +                       break;
> > > +       }
> > > +
> > > +       mutex_unlock(&gem_shrinker->lock);
> > > +
> > > +       if (count >= SHRINK_EMPTY)
> > > +               return SHRINK_EMPTY - 1;
> > > +
> > > +       return count ?: SHRINK_EMPTY;
> > > +}
> > > +
> > > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> > > +       WARN_ON(shmem->evicted);
> > > +
> > > +       drm_gem_shmem_unpin_pages(shmem);
> > > +
> > > +       shmem->evicted = true;
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> > > +
> > > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +
> > > +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > > +
> > > +       drm_gem_shmem_unpin_pages(shmem);
> > > +       drm_gem_free_mmap_offset(obj);
> > > +
> > > +       /* Our goal here is to return as much of the memory as
> > > +        * is possible back to the system as we are called from OOM.
> > > +        * To do this we must instruct the shmfs to drop all of its
> > > +        * backing pages, *now*.
> > > +        */
> > > +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > > +
> > > +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > > +
> > > +       shmem->madv = -1;
> > > +       shmem->evicted = false;
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> > > +
> > > +static unsigned long
> > > +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> > > +                                       unsigned long nr_to_scan,
> > > +                                       bool *lock_contention,
> > > +                                       bool evict)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > > +       struct drm_gem_shmem_object *shmem;
> > > +       struct list_head still_in_list;
> > > +       struct drm_gem_object *obj;
> > > +       unsigned long freed = 0;
> > > +       size_t page_count;
> > > +       int err;
> > > +
> > > +       INIT_LIST_HEAD(&still_in_list);
> > > +
> > > +       mutex_lock(&gem_shrinker->lock);
> > > +
> > > +       while (freed < nr_to_scan) {
> > > +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> > > +                                                typeof(*shmem), madv_list);
> > > +               if (!shmem)
> > > +                       break;
> > > +
> > > +               obj = &shmem->base;
> > > +               page_count = obj->size >> PAGE_SHIFT;
> > > +               list_move_tail(&shmem->madv_list, &still_in_list);
> > > +
> > > +               if (evict) {
> > > +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> > > +                           get_nr_swap_pages() < page_count)
> > > +                               continue;
> > > +               } else {
> > > +                       if (!drm_gem_shmem_is_purgeable(shmem))
> > > +                               continue;
> > > +               }
> > > +
> > > +               /*
> > > +                * If it's in the process of being freed, gem_object->free()
> > > +                * may be blocked on lock waiting to remove it.  So just
> > > +                * skip it.
> > > +                */
> > > +               if (!kref_get_unless_zero(&obj->refcount))
> > > +                       continue;
> > > +
> > > +               mutex_unlock(&gem_shrinker->lock);
> > > +
> > > +               /* prevent racing with job-submission code paths */
> > > +               if (!dma_resv_trylock(obj->resv)) {
> > > +                       *lock_contention |= true;
> > > +                       goto shrinker_lock;
> > > +               }
> > > +
> > > +               /* prevent racing with the dma-buf importing/exporting */
> > > +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > > +                       *lock_contention |= true;
> > > +                       goto resv_unlock;
> > > +               }
> > > +
> > > +               /* check whether h/w uses this object */
> > > +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> > > +                       goto object_name_unlock;
> > > +
> > > +               /* re-check whether eviction status hasn't changed */
> > > +               if (!drm_gem_shmem_is_evictable(shmem) &&
> > > +                   !drm_gem_shmem_is_purgeable(shmem))
> > > +                       goto object_name_unlock;
> > > +
> > > +               err = shmem->evict(shmem);
> > > +               if (!err)
> > > +                       freed += obj->size >> PAGE_SHIFT;
> > > +
> > > +object_name_unlock:
> > > +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> > > +resv_unlock:
> > > +               dma_resv_unlock(obj->resv);
> > > +shrinker_lock:
> > > +               drm_gem_object_put(&shmem->base);
> > > +               mutex_lock(&gem_shrinker->lock);
> > > +       }
> > > +
> > > +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> > > +
> > > +       mutex_unlock(&gem_shrinker->lock);
> > > +
> > > +       return freed;
> > > +}
> > > +
> > > +static unsigned long
> > > +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> > > +                                   struct shrink_control *sc)
> > > +{
> > > +       unsigned long nr_to_scan = sc->nr_to_scan;
> > > +       bool lock_contention = false;
> > > +       unsigned long freed;
> > > +
> > > +       /* purge as many objects as we can */
> > > +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> > > +                                                       &lock_contention, false);
> > > +
> > > +       /* evict as many objects as we can */
> > > +       if (freed < nr_to_scan)
> > > +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> > > +                                                                nr_to_scan - freed,
> > > +                                                                &lock_contention,
> > > +                                                                true);
> > > +
> > > +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> > > +}
> > > +
> > > +/**
> > > + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> > > + * @dev: DRM device
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker;
> > > +       int err;
> > > +
> > > +       if (WARN_ON(dev->shmem_shrinker))
> > > +               return -EBUSY;
> > > +
> > > +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> > > +       if (!gem_shrinker)
> > > +               return -ENOMEM;
> > > +
> > > +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> > > +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> > > +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> > > +       gem_shrinker->dev = dev;
> > > +
> > > +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> > > +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> > > +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> > > +       mutex_init(&gem_shrinker->lock);
> > > +
> > > +       dev->shmem_shrinker = gem_shrinker;
> > > +
> > > +       err = register_shrinker(&gem_shrinker->base);
> > > +       if (err) {
> > > +               dev->shmem_shrinker = NULL;
> > > +               kfree(gem_shrinker);
> > > +               return err;
> > > +       }
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> > > +
> > > +/**
> > > + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> > > + * @dev: DRM device
> > > + */
> > > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> > > +
> > > +       if (gem_shrinker) {
> > > +               unregister_shrinker(&gem_shrinker->base);
> > > +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> > > +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> > > +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> > > +               mutex_destroy(&gem_shrinker->lock);
> > > +               dev->shmem_shrinker = NULL;
> > > +               kfree(gem_shrinker);
> > > +       }
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> > > +
> > >  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
> > >  MODULE_IMPORT_NS(DMA_BUF);
> > >  MODULE_LICENSE("GPL v2");
> > > diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > > index a4bedfeb2ec4..7cc32556f908 100644
> > > --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > > +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > > @@ -15,6 +15,13 @@
> > >  #include "panfrost_gem.h"
> > >  #include "panfrost_mmu.h"
> > >
> > > +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       return (shmem->madv > 0) &&
> > > +               !shmem->pages_pin_count && shmem->sgt &&
> > > +               !shmem->base.dma_buf && !shmem->base.import_attach;
> > > +}
> > > +
> > >  static unsigned long
> > >  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
> > >  {
> > > @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
> > >                 return 0;
> > >
> > >         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> > > -               if (drm_gem_shmem_is_purgeable(shmem))
> > > +               if (panfrost_gem_shmem_is_purgeable(shmem))
> > >                         count += shmem->base.size >> PAGE_SHIFT;
> > >         }
> > >
> > > diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > > index b2d93cb12ebf..81bacc7e1873 100644
> > > --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> > > +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > > @@ -89,6 +89,7 @@ struct virtio_gpu_object {
> > >         uint32_t hw_res_handle;
> > >         bool dumb;
> > >         bool created;
> > > +       bool detached;
> > >         bool host3d_blob, guest_blob;
> > >         uint32_t blob_mem, blob_flags;
> > >
> > > @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
> > >
> > >  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
> > >
> > > +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> > > +
> > >  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
> > >                                uint32_t *resid);
> > >  /* virtgpu_prime.c */
> > > diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> > > index 9923c7a6885e..929546cad894 100644
> > > --- a/include/drm/drm_device.h
> > > +++ b/include/drm/drm_device.h
> > > @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
> > >  struct drm_vma_offset_manager;
> > >  struct drm_vram_mm;
> > >  struct drm_fb_helper;
> > > +struct drm_gem_shmem_shrinker;
> > >
> > >  struct inode;
> > >
> > > @@ -277,6 +278,9 @@ struct drm_device {
> > >         /** @vram_mm: VRAM MM memory manager */
> > >         struct drm_vram_mm *vram_mm;
> > >
> > > +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> > > +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> > > +
> > >         /**
> > >          * @switch_power_state:
> > >          *
> > > diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> > > index 9a8983ee8abe..62c640678a91 100644
> > > --- a/include/drm/drm_gem_shmem_helper.h
> > > +++ b/include/drm/drm_gem_shmem_helper.h
> > > @@ -6,6 +6,7 @@
> > >  #include <linux/fs.h>
> > >  #include <linux/mm.h>
> > >  #include <linux/mutex.h>
> > > +#include <linux/shrinker.h>
> > >
> > >  #include <drm/drm_file.h>
> > >  #include <drm/drm_gem.h>
> > > @@ -15,6 +16,7 @@
> > >  struct dma_buf_attachment;
> > >  struct drm_mode_create_dumb;
> > >  struct drm_printer;
> > > +struct drm_device;
> > >  struct sg_table;
> > >
> > >  /**
> > > @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
> > >          */
> > >         unsigned int pages_use_count;
> > >
> > > +       /**
> > > +        * @pages_pin_count:
> > > +        *
> > > +        * Reference count on the pinned pages table.
> > > +        * The pages can be evicted by memory shrinker
> > > +        * when the count reaches zero.
> > > +        */
> > > +       unsigned int pages_pin_count;
> > > +
> > >         /**
> > >          * @madv: State for madvise
> > >          *
> > >          * 0 is active/inuse.
> > > +        * 1 is not-needed/can-be-purged
> > >          * A negative value is the object is purged.
> > > -        * Positive values are driver specific and not used by the helpers.
> > >          */
> > >         int madv;
> > >
> > > @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
> > >          * @map_wc: map object write-combined (instead of using shmem defaults).
> > >          */
> > >         bool map_wc;
> > > +
> > > +       /**
> > > +        * @eviction_enabled:
> > > +        *
> > > +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> > > +        * Used internally by memory shrinker.
> > > +        */
> > > +       bool eviction_enabled;
> > > +
> > > +       /**
> > > +        * @purge_enabled:
> > > +        *
> > > +        * The shmem pages can be purged only if @purge_enabled is set to true.
> > > +        * Used internally by memory shrinker.
> > > +        */
> > > +       bool purge_enabled;
> > > +
> > > +       /**
> > > +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> > > +        * Used internally by memory shrinker.
> > > +        */
> > > +       bool evicted;
> > > +
> > > +       /**
> > > +        * @evict:
> > > +        *
> > > +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> > > +        * GEM's DMA reservation is kept locked by the shrinker. This is
> > > +        * optional callback that should be specified by drivers.
> > > +        *
> > > +        * Returns 0 on success, or -errno on error.
> > > +        */
> > > +       int (*evict)(struct drm_gem_shmem_object *shmem);
> > >  };
> > >
> > >  #define to_drm_gem_shmem_obj(obj) \
> > > @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
> > >
> > >  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
> > >
> > > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> > > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> > > +
> > >  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> > >  {
> > > -       return (shmem->madv > 0) &&
> > > -               !shmem->vmap_use_count && shmem->sgt &&
> > > -               !shmem->base.dma_buf && !shmem->base.import_attach;
> > > +       return (shmem->madv > 0) && shmem->evict &&
> > > +               shmem->purge_enabled && shmem->pages_use_count &&
> > > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > > +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
> > >  }
> > >
> > > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> > > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> > > +
> > > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> > > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> > >
> > >  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
> > >  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> > > @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
> > >         return drm_gem_shmem_mmap(shmem, vma);
> > >  }
> > >
> > > +/**
> > > + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> > > + */
> > > +struct drm_gem_shmem_shrinker {
> > > +       /** @base: Shrinker for purging shmem GEM objects */
> > > +       struct shrinker base;
> > > +
> > > +       /** @lock: Protects @lru_* */
> > > +       struct mutex lock;
> > > +
> > > +       /** @lru_pinned: List of pinned shmem GEM objects */
> > > +       struct list_head lru_pinned;
> > > +
> > > +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> > > +       struct list_head lru_evictable;
> > > +
> > > +       /** @lru_evicted: List of evicted shmem GEM objects */
> > > +       struct list_head lru_evicted;
> > > +
> > > +       /** @dev: DRM device that uses this shrinker */
> > > +       struct drm_device *dev;
> > > +};
> > > +
> > > +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> > > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> > > +
> > >  /*
> > >   * Driver ops
> > >   */
> > > --
> > > 2.35.3
> > >
> >
> >
> > --
> > Daniel Vetter
> > Software Engineer, Intel Corporation
> > http://blog.ffwll.ch



-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [Intel-gfx] [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Sun, 5 Jun 2022 at 20:32, Rob Clark <robdclark@gmail.com> wrote:
>
> On Sun, Jun 5, 2022 at 9:47 AM Daniel Vetter <daniel@ffwll.ch> wrote:
> >
> > On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
> > <dmitry.osipenko@collabora.com> wrote:
> > >
> > > Introduce a common DRM SHMEM shrinker framework that allows to reduce
> > > code duplication among DRM drivers by replacing theirs custom shrinker
> > > implementations with the generic shrinker.
> > >
> > > In order to start using DRM SHMEM shrinker drivers should:
> > >
> > > 1. Implement new evict() shmem object callback.
> > > 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> > > 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
> > >    activate shrinking of shmem GEMs.
> > >
> > > This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> > > Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
> > >
> > > Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> > > Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> >
> > So I guess I get a price for being blind since forever, because this
> > thing existed since at least 2013. I just stumbled over
> > llist_lru.[hc], a purpose built list helper for shrinkers. I think we
> > should try to adopt that so that our gpu shrinkers look more like
> > shrinkers for everything else.
>
> followup from a bit of irc discussion w/ danvet about list_lru:
>
> * It seems to be missing a way to bail out of iteration before
>   nr_to_scan is hit.. which is going to be inconvenient if you
>   want to allow active bos on the LRU but bail scanning once
>   you encounter the first one.
>
> * Not sure if the numa node awareness is super useful for GEM
>   bos
>
> First issue is perhaps not too hard to fix.  But maybe a better
> idea is a drm_gem_lru helper type thing which is more tailored
> to GEM buffers?

Yeah I guess reusing list_lru isn't that good idea. So just
open-coding it for now, and then drm_gem_bo_lru or so if we need to
share it separately from shmem helpers with other drivers. Maybe will
be needed for ttm or so.
-Daniel

>
> BR,
> -R
>
> > Apologies for this, since I fear this might cause a bit of churn.
> > Hopefully it's all contained to the list manipulation code in shmem
> > helpers, I don't think this should leak any further.
> > -Daniel
> >
> > > ---
> > >  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
> > >  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
> > >  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
> > >  include/drm/drm_device.h                      |   4 +
> > >  include/drm/drm_gem_shmem_helper.h            |  87 ++-
> > >  5 files changed, 594 insertions(+), 49 deletions(-)
> > >
> > > diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > > index 555fe212bd98..4cd0b5913492 100644
> > > --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> > > +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > > @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
> > >  }
> > >  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
> > >
> > > +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       return (shmem->madv >= 0) && shmem->evict &&
> > > +               shmem->eviction_enabled && shmem->pages_use_count &&
> > > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > > +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> > > +}
> > > +
> > > +static void
> > > +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> > > +
> > > +       dma_resv_assert_held(shmem->base.resv);
> > > +
> > > +       if (!gem_shrinker || obj->import_attach)
> > > +               return;
> > > +
> > > +       mutex_lock(&gem_shrinker->lock);
> > > +
> > > +       if (drm_gem_shmem_is_evictable(shmem) ||
> > > +           drm_gem_shmem_is_purgeable(shmem))
> > > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> > > +       else if (shmem->madv < 0)
> > > +               list_del_init(&shmem->madv_list);
> > > +       else if (shmem->evicted)
> > > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> > > +       else if (!shmem->pages)
> > > +               list_del_init(&shmem->madv_list);
> > > +       else
> > > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> > > +
> > > +       mutex_unlock(&gem_shrinker->lock);
> > > +}
> > > +
> > >  /**
> > >   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
> > >   * @shmem: shmem GEM object to free
> > > @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> > >         } else {
> > >                 dma_resv_lock(shmem->base.resv, NULL);
> > >
> > > +               /* take out shmem GEM object from the memory shrinker */
> > > +               drm_gem_shmem_madvise(shmem, -1);
> > > +
> > >                 WARN_ON(shmem->vmap_use_count);
> > >
> > >                 if (shmem->sgt) {
> > > @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> > >                         sg_free_table(shmem->sgt);
> > >                         kfree(shmem->sgt);
> > >                 }
> > > -               if (shmem->pages)
> > > +               if (shmem->pages_use_count)
> > >                         drm_gem_shmem_put_pages(shmem);
> > >
> > >                 WARN_ON(shmem->pages_use_count);
> > > @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> > >  }
> > >  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
> > >
> > > -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > > +/**
> > > + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> > > + * @shmem: shmem GEM object
> > > + *
> > > + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> > > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       dma_resv_lock(shmem->base.resv, NULL);
> > > +
> > > +       if (shmem->madv < 0)
> > > +               return -ENOMEM;
> > > +
> > > +       shmem->eviction_enabled = true;
> > > +
> > > +       dma_resv_unlock(shmem->base.resv);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> > > +
> > > +/**
> > > + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> > > + * @shmem: shmem GEM object
> > > + *
> > > + * Tell memory shrinker that this GEM can be purged. Initially purging is
> > > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       dma_resv_lock(shmem->base.resv, NULL);
> > > +
> > > +       if (shmem->madv < 0)
> > > +               return -ENOMEM;
> > > +
> > > +       shmem->purge_enabled = true;
> > > +
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       dma_resv_unlock(shmem->base.resv);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> > > +
> > > +static int
> > > +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
> > >  {
> > >         struct drm_gem_object *obj = &shmem->base;
> > >         struct page **pages;
> > >
> > > -       if (shmem->pages_use_count++ > 0)
> > > +       dma_resv_assert_held(shmem->base.resv);
> > > +
> > > +       if (shmem->madv < 0) {
> > > +               WARN_ON(shmem->pages);
> > > +               return -ENOMEM;
> > > +       }
> > > +
> > > +       if (shmem->pages) {
> > > +               WARN_ON(!shmem->evicted);
> > >                 return 0;
> > > +       }
> > > +
> > > +       if (WARN_ON(!shmem->pages_use_count))
> > > +               return -EINVAL;
> > >
> > >         pages = drm_gem_get_pages(obj);
> > >         if (IS_ERR(pages)) {
> > >                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> > > -               shmem->pages_use_count = 0;
> > >                 return PTR_ERR(pages);
> > >         }
> > >
> > > @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > >         return 0;
> > >  }
> > >
> > > +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       int err;
> > > +
> > > +       dma_resv_assert_held(shmem->base.resv);
> > > +
> > > +       if (shmem->madv < 0)
> > > +               return -ENOMEM;
> > > +
> > > +       if (shmem->pages_use_count++ > 0) {
> > > +               err = drm_gem_shmem_swap_in(shmem);
> > > +               if (err)
> > > +                       goto err_zero_use;
> > > +
> > > +               return 0;
> > > +       }
> > > +
> > > +       err = drm_gem_shmem_acquire_pages(shmem);
> > > +       if (err)
> > > +               goto err_zero_use;
> > > +
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       return 0;
> > > +
> > > +err_zero_use:
> > > +       shmem->pages_use_count = 0;
> > > +
> > > +       return err;
> > > +}
> > > +
> > > +static void
> > > +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +
> > > +       if (!shmem->pages) {
> > > +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> > > +               return;
> > > +       }
> > > +
> > > +#ifdef CONFIG_X86
> > > +       if (shmem->map_wc)
> > > +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > > +#endif
> > > +
> > > +       drm_gem_put_pages(obj, shmem->pages,
> > > +                         shmem->pages_mark_dirty_on_put,
> > > +                         shmem->pages_mark_accessed_on_put);
> > > +       shmem->pages = NULL;
> > > +}
> > > +
> > >  /*
> > >   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
> > >   * @shmem: shmem GEM object
> > > @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > >   */
> > >  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> > >  {
> > > -       struct drm_gem_object *obj = &shmem->base;
> > > -
> > >         dma_resv_assert_held(shmem->base.resv);
> > >
> > >         if (WARN_ON_ONCE(!shmem->pages_use_count))
> > > @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> > >         if (--shmem->pages_use_count > 0)
> > >                 return;
> > >
> > > -#ifdef CONFIG_X86
> > > -       if (shmem->map_wc)
> > > -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > > -#endif
> > > +       drm_gem_shmem_release_pages(shmem);
> > >
> > > -       drm_gem_put_pages(obj, shmem->pages,
> > > -                         shmem->pages_mark_dirty_on_put,
> > > -                         shmem->pages_mark_accessed_on_put);
> > > -       shmem->pages = NULL;
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> > >
> > > @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> > >   */
> > >  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
> > >  {
> > > +       int ret;
> > > +
> > >         dma_resv_assert_held(shmem->base.resv);
> > >
> > >         WARN_ON(shmem->base.import_attach);
> > >
> > > -       return drm_gem_shmem_get_pages(shmem);
> > > +       ret = drm_gem_shmem_get_pages(shmem);
> > > +       if (!ret)
> > > +               shmem->pages_pin_count++;
> > > +
> > > +       return ret;
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_pin);
> > >
> > > @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
> > >         WARN_ON(shmem->base.import_attach);
> > >
> > >         drm_gem_shmem_put_pages(shmem);
> > > +
> > > +       shmem->pages_pin_count--;
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_unpin);
> > >
> > > @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> > >                         return 0;
> > >                 }
> > >
> > > -               ret = drm_gem_shmem_get_pages(shmem);
> > > +               ret = drm_gem_shmem_pin(shmem);
> > >                 if (ret)
> > >                         goto err_zero_use;
> > >
> > > @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> > >
> > >  err_put_pages:
> > >         if (!obj->import_attach)
> > > -               drm_gem_shmem_put_pages(shmem);
> > > +               drm_gem_shmem_unpin(shmem);
> > >  err_zero_use:
> > >         shmem->vmap_use_count = 0;
> > >
> > > @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
> > >                         return;
> > >
> > >                 vunmap(shmem->vaddr);
> > > -               drm_gem_shmem_put_pages(shmem);
> > > +               drm_gem_shmem_unpin(shmem);
> > >         }
> > >
> > >         shmem->vaddr = NULL;
> > > @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
> > >
> > >         madv = shmem->madv;
> > >
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > >         return (madv >= 0);
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_madvise);
> > >
> > > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > > +/**
> > > + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> > > + *                           hardware access to the memory.
> > > + * @shmem: shmem GEM object
> > > + *
> > > + * This function moves shmem GEM back to memory if it was previously evicted
> > > + * by the memory shrinker. The GEM is ready to use on success.
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
> > >  {
> > >         struct drm_gem_object *obj = &shmem->base;
> > > -       struct drm_device *dev = obj->dev;
> > > +       struct sg_table *sgt;
> > > +       int err;
> > >
> > >         dma_resv_assert_held(shmem->base.resv);
> > >
> > > -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > > +       if (shmem->evicted) {
> > > +               err = drm_gem_shmem_acquire_pages(shmem);
> > > +               if (err)
> > > +                       return err;
> > > +
> > > +               sgt = drm_gem_shmem_get_sg_table(shmem);
> > > +               if (IS_ERR(sgt))
> > > +                       return PTR_ERR(sgt);
> > > +
> > > +               err = dma_map_sgtable(obj->dev->dev, sgt,
> > > +                                     DMA_BIDIRECTIONAL, 0);
> > > +               if (err) {
> > > +                       sg_free_table(sgt);
> > > +                       kfree(sgt);
> > > +                       return err;
> > > +               }
> > >
> > > -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > > -       sg_free_table(shmem->sgt);
> > > -       kfree(shmem->sgt);
> > > -       shmem->sgt = NULL;
> > > +               shmem->sgt = sgt;
> > > +               shmem->evicted = false;
> > >
> > > -       drm_gem_shmem_put_pages(shmem);
> > > +               drm_gem_shmem_update_pages_state(shmem);
> > > +       }
> > >
> > > -       shmem->madv = -1;
> > > +       if (!shmem->pages)
> > > +               return -ENOMEM;
> > >
> > > -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > > -       drm_gem_free_mmap_offset(obj);
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
> > >
> > > -       /* Our goal here is to return as much of the memory as
> > > -        * is possible back to the system as we are called from OOM.
> > > -        * To do this we must instruct the shmfs to drop all of its
> > > -        * backing pages, *now*.
> > > -        */
> > > -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > > +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +       struct drm_device *dev = obj->dev;
> > >
> > > -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > > +       if (shmem->evicted)
> > > +               return;
> > > +
> > > +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > > +       drm_gem_shmem_release_pages(shmem);
> > > +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > > +
> > > +       sg_free_table(shmem->sgt);
> > > +       kfree(shmem->sgt);
> > > +       shmem->sgt = NULL;
> > >  }
> > > -EXPORT_SYMBOL(drm_gem_shmem_purge);
> > >
> > >  /**
> > >   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> > > @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
> > >         vm_fault_t ret;
> > >         struct page *page;
> > >         pgoff_t page_offset;
> > > +       bool pages_unpinned;
> > > +       int err;
> > >
> > >         /* We don't use vmf->pgoff since that has the fake offset */
> > >         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
> > >
> > >         dma_resv_lock(shmem->base.resv, NULL);
> > >
> > > -       if (page_offset >= num_pages ||
> > > -           WARN_ON_ONCE(!shmem->pages) ||
> > > -           shmem->madv < 0) {
> > > +       /* Sanity-check that we have the pages pointer when it should present */
> > > +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> > > +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> > > +
> > > +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
> > >                 ret = VM_FAULT_SIGBUS;
> > >         } else {
> > > +               err = drm_gem_shmem_swap_in(shmem);
> > > +               if (err) {
> > > +                       ret = VM_FAULT_OOM;
> > > +                       goto unlock;
> > > +               }
> > > +
> > >                 page = shmem->pages[page_offset];
> > >
> > >                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
> > >         }
> > >
> > > +unlock:
> > >         dma_resv_unlock(shmem->base.resv);
> > >
> > >         return ret;
> > > @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
> > >  {
> > >         struct drm_gem_object *obj = vma->vm_private_data;
> > >         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> > > -       int ret;
> > >
> > >         WARN_ON(shmem->base.import_attach);
> > >
> > >         dma_resv_lock(shmem->base.resv, NULL);
> > > -       ret = drm_gem_shmem_get_pages(shmem);
> > > -       WARN_ON_ONCE(ret != 0);
> > > +
> > > +       if (drm_gem_shmem_get_pages(shmem))
> > > +               shmem->pages_use_count++;
> > > +
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > >         dma_resv_unlock(shmem->base.resv);
> > >
> > >         drm_gem_vm_open(vma);
> > > @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
> > >  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> > >                               struct drm_printer *p, unsigned int indent)
> > >  {
> > > +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> > > +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
> > >         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
> > >
> > >         if (shmem->base.import_attach)
> > > @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> > >                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
> > >                                   shmem->vmap_use_count);
> > >
> > > +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
> > >         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> > > +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_print_info);
> > >
> > > @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
> > >
> > >         shmem->sgt = sgt;
> > >
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > >         dma_resv_unlock(shmem->base.resv);
> > >
> > >         return sgt;
> > > @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
> > >  }
> > >  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
> > >
> > > +static struct drm_gem_shmem_shrinker *
> > > +to_drm_shrinker(struct shrinker *shrinker)
> > > +{
> > > +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> > > +}
> > > +
> > > +static unsigned long
> > > +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> > > +                                    struct shrink_control *sc)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > > +       struct drm_gem_shmem_object *shmem;
> > > +       unsigned long count = 0;
> > > +
> > > +       if (!mutex_trylock(&gem_shrinker->lock))
> > > +               return 0;
> > > +
> > > +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> > > +               count += shmem->base.size;
> > > +
> > > +               if (count >= SHRINK_EMPTY)
> > > +                       break;
> > > +       }
> > > +
> > > +       mutex_unlock(&gem_shrinker->lock);
> > > +
> > > +       if (count >= SHRINK_EMPTY)
> > > +               return SHRINK_EMPTY - 1;
> > > +
> > > +       return count ?: SHRINK_EMPTY;
> > > +}
> > > +
> > > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> > > +       WARN_ON(shmem->evicted);
> > > +
> > > +       drm_gem_shmem_unpin_pages(shmem);
> > > +
> > > +       shmem->evicted = true;
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> > > +
> > > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +
> > > +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > > +
> > > +       drm_gem_shmem_unpin_pages(shmem);
> > > +       drm_gem_free_mmap_offset(obj);
> > > +
> > > +       /* Our goal here is to return as much of the memory as
> > > +        * is possible back to the system as we are called from OOM.
> > > +        * To do this we must instruct the shmfs to drop all of its
> > > +        * backing pages, *now*.
> > > +        */
> > > +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > > +
> > > +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > > +
> > > +       shmem->madv = -1;
> > > +       shmem->evicted = false;
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> > > +
> > > +static unsigned long
> > > +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> > > +                                       unsigned long nr_to_scan,
> > > +                                       bool *lock_contention,
> > > +                                       bool evict)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > > +       struct drm_gem_shmem_object *shmem;
> > > +       struct list_head still_in_list;
> > > +       struct drm_gem_object *obj;
> > > +       unsigned long freed = 0;
> > > +       size_t page_count;
> > > +       int err;
> > > +
> > > +       INIT_LIST_HEAD(&still_in_list);
> > > +
> > > +       mutex_lock(&gem_shrinker->lock);
> > > +
> > > +       while (freed < nr_to_scan) {
> > > +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> > > +                                                typeof(*shmem), madv_list);
> > > +               if (!shmem)
> > > +                       break;
> > > +
> > > +               obj = &shmem->base;
> > > +               page_count = obj->size >> PAGE_SHIFT;
> > > +               list_move_tail(&shmem->madv_list, &still_in_list);
> > > +
> > > +               if (evict) {
> > > +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> > > +                           get_nr_swap_pages() < page_count)
> > > +                               continue;
> > > +               } else {
> > > +                       if (!drm_gem_shmem_is_purgeable(shmem))
> > > +                               continue;
> > > +               }
> > > +
> > > +               /*
> > > +                * If it's in the process of being freed, gem_object->free()
> > > +                * may be blocked on lock waiting to remove it.  So just
> > > +                * skip it.
> > > +                */
> > > +               if (!kref_get_unless_zero(&obj->refcount))
> > > +                       continue;
> > > +
> > > +               mutex_unlock(&gem_shrinker->lock);
> > > +
> > > +               /* prevent racing with job-submission code paths */
> > > +               if (!dma_resv_trylock(obj->resv)) {
> > > +                       *lock_contention |= true;
> > > +                       goto shrinker_lock;
> > > +               }
> > > +
> > > +               /* prevent racing with the dma-buf importing/exporting */
> > > +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > > +                       *lock_contention |= true;
> > > +                       goto resv_unlock;
> > > +               }
> > > +
> > > +               /* check whether h/w uses this object */
> > > +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> > > +                       goto object_name_unlock;
> > > +
> > > +               /* re-check whether eviction status hasn't changed */
> > > +               if (!drm_gem_shmem_is_evictable(shmem) &&
> > > +                   !drm_gem_shmem_is_purgeable(shmem))
> > > +                       goto object_name_unlock;
> > > +
> > > +               err = shmem->evict(shmem);
> > > +               if (!err)
> > > +                       freed += obj->size >> PAGE_SHIFT;
> > > +
> > > +object_name_unlock:
> > > +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> > > +resv_unlock:
> > > +               dma_resv_unlock(obj->resv);
> > > +shrinker_lock:
> > > +               drm_gem_object_put(&shmem->base);
> > > +               mutex_lock(&gem_shrinker->lock);
> > > +       }
> > > +
> > > +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> > > +
> > > +       mutex_unlock(&gem_shrinker->lock);
> > > +
> > > +       return freed;
> > > +}
> > > +
> > > +static unsigned long
> > > +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> > > +                                   struct shrink_control *sc)
> > > +{
> > > +       unsigned long nr_to_scan = sc->nr_to_scan;
> > > +       bool lock_contention = false;
> > > +       unsigned long freed;
> > > +
> > > +       /* purge as many objects as we can */
> > > +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> > > +                                                       &lock_contention, false);
> > > +
> > > +       /* evict as many objects as we can */
> > > +       if (freed < nr_to_scan)
> > > +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> > > +                                                                nr_to_scan - freed,
> > > +                                                                &lock_contention,
> > > +                                                                true);
> > > +
> > > +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> > > +}
> > > +
> > > +/**
> > > + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> > > + * @dev: DRM device
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker;
> > > +       int err;
> > > +
> > > +       if (WARN_ON(dev->shmem_shrinker))
> > > +               return -EBUSY;
> > > +
> > > +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> > > +       if (!gem_shrinker)
> > > +               return -ENOMEM;
> > > +
> > > +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> > > +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> > > +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> > > +       gem_shrinker->dev = dev;
> > > +
> > > +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> > > +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> > > +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> > > +       mutex_init(&gem_shrinker->lock);
> > > +
> > > +       dev->shmem_shrinker = gem_shrinker;
> > > +
> > > +       err = register_shrinker(&gem_shrinker->base);
> > > +       if (err) {
> > > +               dev->shmem_shrinker = NULL;
> > > +               kfree(gem_shrinker);
> > > +               return err;
> > > +       }
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> > > +
> > > +/**
> > > + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> > > + * @dev: DRM device
> > > + */
> > > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> > > +
> > > +       if (gem_shrinker) {
> > > +               unregister_shrinker(&gem_shrinker->base);
> > > +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> > > +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> > > +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> > > +               mutex_destroy(&gem_shrinker->lock);
> > > +               dev->shmem_shrinker = NULL;
> > > +               kfree(gem_shrinker);
> > > +       }
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> > > +
> > >  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
> > >  MODULE_IMPORT_NS(DMA_BUF);
> > >  MODULE_LICENSE("GPL v2");
> > > diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > > index a4bedfeb2ec4..7cc32556f908 100644
> > > --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > > +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > > @@ -15,6 +15,13 @@
> > >  #include "panfrost_gem.h"
> > >  #include "panfrost_mmu.h"
> > >
> > > +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       return (shmem->madv > 0) &&
> > > +               !shmem->pages_pin_count && shmem->sgt &&
> > > +               !shmem->base.dma_buf && !shmem->base.import_attach;
> > > +}
> > > +
> > >  static unsigned long
> > >  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
> > >  {
> > > @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
> > >                 return 0;
> > >
> > >         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> > > -               if (drm_gem_shmem_is_purgeable(shmem))
> > > +               if (panfrost_gem_shmem_is_purgeable(shmem))
> > >                         count += shmem->base.size >> PAGE_SHIFT;
> > >         }
> > >
> > > diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > > index b2d93cb12ebf..81bacc7e1873 100644
> > > --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> > > +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > > @@ -89,6 +89,7 @@ struct virtio_gpu_object {
> > >         uint32_t hw_res_handle;
> > >         bool dumb;
> > >         bool created;
> > > +       bool detached;
> > >         bool host3d_blob, guest_blob;
> > >         uint32_t blob_mem, blob_flags;
> > >
> > > @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
> > >
> > >  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
> > >
> > > +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> > > +
> > >  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
> > >                                uint32_t *resid);
> > >  /* virtgpu_prime.c */
> > > diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> > > index 9923c7a6885e..929546cad894 100644
> > > --- a/include/drm/drm_device.h
> > > +++ b/include/drm/drm_device.h
> > > @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
> > >  struct drm_vma_offset_manager;
> > >  struct drm_vram_mm;
> > >  struct drm_fb_helper;
> > > +struct drm_gem_shmem_shrinker;
> > >
> > >  struct inode;
> > >
> > > @@ -277,6 +278,9 @@ struct drm_device {
> > >         /** @vram_mm: VRAM MM memory manager */
> > >         struct drm_vram_mm *vram_mm;
> > >
> > > +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> > > +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> > > +
> > >         /**
> > >          * @switch_power_state:
> > >          *
> > > diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> > > index 9a8983ee8abe..62c640678a91 100644
> > > --- a/include/drm/drm_gem_shmem_helper.h
> > > +++ b/include/drm/drm_gem_shmem_helper.h
> > > @@ -6,6 +6,7 @@
> > >  #include <linux/fs.h>
> > >  #include <linux/mm.h>
> > >  #include <linux/mutex.h>
> > > +#include <linux/shrinker.h>
> > >
> > >  #include <drm/drm_file.h>
> > >  #include <drm/drm_gem.h>
> > > @@ -15,6 +16,7 @@
> > >  struct dma_buf_attachment;
> > >  struct drm_mode_create_dumb;
> > >  struct drm_printer;
> > > +struct drm_device;
> > >  struct sg_table;
> > >
> > >  /**
> > > @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
> > >          */
> > >         unsigned int pages_use_count;
> > >
> > > +       /**
> > > +        * @pages_pin_count:
> > > +        *
> > > +        * Reference count on the pinned pages table.
> > > +        * The pages can be evicted by memory shrinker
> > > +        * when the count reaches zero.
> > > +        */
> > > +       unsigned int pages_pin_count;
> > > +
> > >         /**
> > >          * @madv: State for madvise
> > >          *
> > >          * 0 is active/inuse.
> > > +        * 1 is not-needed/can-be-purged
> > >          * A negative value is the object is purged.
> > > -        * Positive values are driver specific and not used by the helpers.
> > >          */
> > >         int madv;
> > >
> > > @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
> > >          * @map_wc: map object write-combined (instead of using shmem defaults).
> > >          */
> > >         bool map_wc;
> > > +
> > > +       /**
> > > +        * @eviction_enabled:
> > > +        *
> > > +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> > > +        * Used internally by memory shrinker.
> > > +        */
> > > +       bool eviction_enabled;
> > > +
> > > +       /**
> > > +        * @purge_enabled:
> > > +        *
> > > +        * The shmem pages can be purged only if @purge_enabled is set to true.
> > > +        * Used internally by memory shrinker.
> > > +        */
> > > +       bool purge_enabled;
> > > +
> > > +       /**
> > > +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> > > +        * Used internally by memory shrinker.
> > > +        */
> > > +       bool evicted;
> > > +
> > > +       /**
> > > +        * @evict:
> > > +        *
> > > +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> > > +        * GEM's DMA reservation is kept locked by the shrinker. This is
> > > +        * optional callback that should be specified by drivers.
> > > +        *
> > > +        * Returns 0 on success, or -errno on error.
> > > +        */
> > > +       int (*evict)(struct drm_gem_shmem_object *shmem);
> > >  };
> > >
> > >  #define to_drm_gem_shmem_obj(obj) \
> > > @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
> > >
> > >  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
> > >
> > > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> > > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> > > +
> > >  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> > >  {
> > > -       return (shmem->madv > 0) &&
> > > -               !shmem->vmap_use_count && shmem->sgt &&
> > > -               !shmem->base.dma_buf && !shmem->base.import_attach;
> > > +       return (shmem->madv > 0) && shmem->evict &&
> > > +               shmem->purge_enabled && shmem->pages_use_count &&
> > > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > > +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
> > >  }
> > >
> > > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> > > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> > > +
> > > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> > > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> > >
> > >  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
> > >  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> > > @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
> > >         return drm_gem_shmem_mmap(shmem, vma);
> > >  }
> > >
> > > +/**
> > > + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> > > + */
> > > +struct drm_gem_shmem_shrinker {
> > > +       /** @base: Shrinker for purging shmem GEM objects */
> > > +       struct shrinker base;
> > > +
> > > +       /** @lock: Protects @lru_* */
> > > +       struct mutex lock;
> > > +
> > > +       /** @lru_pinned: List of pinned shmem GEM objects */
> > > +       struct list_head lru_pinned;
> > > +
> > > +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> > > +       struct list_head lru_evictable;
> > > +
> > > +       /** @lru_evicted: List of evicted shmem GEM objects */
> > > +       struct list_head lru_evicted;
> > > +
> > > +       /** @dev: DRM device that uses this shrinker */
> > > +       struct drm_device *dev;
> > > +};
> > > +
> > > +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> > > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> > > +
> > >  /*
> > >   * Driver ops
> > >   */
> > > --
> > > 2.35.3
> > >
> >
> >
> > --
> > Daniel Vetter
> > Software Engineer, Intel Corporation
> > http://blog.ffwll.ch



-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Sun, 5 Jun 2022 at 20:32, Rob Clark <robdclark@gmail.com> wrote:
>
> On Sun, Jun 5, 2022 at 9:47 AM Daniel Vetter <daniel@ffwll.ch> wrote:
> >
> > On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
> > <dmitry.osipenko@collabora.com> wrote:
> > >
> > > Introduce a common DRM SHMEM shrinker framework that allows to reduce
> > > code duplication among DRM drivers by replacing theirs custom shrinker
> > > implementations with the generic shrinker.
> > >
> > > In order to start using DRM SHMEM shrinker drivers should:
> > >
> > > 1. Implement new evict() shmem object callback.
> > > 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> > > 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
> > >    activate shrinking of shmem GEMs.
> > >
> > > This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> > > Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
> > >
> > > Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> > > Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> >
> > So I guess I get a price for being blind since forever, because this
> > thing existed since at least 2013. I just stumbled over
> > llist_lru.[hc], a purpose built list helper for shrinkers. I think we
> > should try to adopt that so that our gpu shrinkers look more like
> > shrinkers for everything else.
>
> followup from a bit of irc discussion w/ danvet about list_lru:
>
> * It seems to be missing a way to bail out of iteration before
>   nr_to_scan is hit.. which is going to be inconvenient if you
>   want to allow active bos on the LRU but bail scanning once
>   you encounter the first one.
>
> * Not sure if the numa node awareness is super useful for GEM
>   bos
>
> First issue is perhaps not too hard to fix.  But maybe a better
> idea is a drm_gem_lru helper type thing which is more tailored
> to GEM buffers?

Yeah I guess reusing list_lru isn't that good idea. So just
open-coding it for now, and then drm_gem_bo_lru or so if we need to
share it separately from shmem helpers with other drivers. Maybe will
be needed for ttm or so.
-Daniel

>
> BR,
> -R
>
> > Apologies for this, since I fear this might cause a bit of churn.
> > Hopefully it's all contained to the list manipulation code in shmem
> > helpers, I don't think this should leak any further.
> > -Daniel
> >
> > > ---
> > >  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
> > >  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
> > >  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
> > >  include/drm/drm_device.h                      |   4 +
> > >  include/drm/drm_gem_shmem_helper.h            |  87 ++-
> > >  5 files changed, 594 insertions(+), 49 deletions(-)
> > >
> > > diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > > index 555fe212bd98..4cd0b5913492 100644
> > > --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> > > +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> > > @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
> > >  }
> > >  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
> > >
> > > +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       return (shmem->madv >= 0) && shmem->evict &&
> > > +               shmem->eviction_enabled && shmem->pages_use_count &&
> > > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > > +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> > > +}
> > > +
> > > +static void
> > > +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> > > +
> > > +       dma_resv_assert_held(shmem->base.resv);
> > > +
> > > +       if (!gem_shrinker || obj->import_attach)
> > > +               return;
> > > +
> > > +       mutex_lock(&gem_shrinker->lock);
> > > +
> > > +       if (drm_gem_shmem_is_evictable(shmem) ||
> > > +           drm_gem_shmem_is_purgeable(shmem))
> > > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> > > +       else if (shmem->madv < 0)
> > > +               list_del_init(&shmem->madv_list);
> > > +       else if (shmem->evicted)
> > > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> > > +       else if (!shmem->pages)
> > > +               list_del_init(&shmem->madv_list);
> > > +       else
> > > +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> > > +
> > > +       mutex_unlock(&gem_shrinker->lock);
> > > +}
> > > +
> > >  /**
> > >   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
> > >   * @shmem: shmem GEM object to free
> > > @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> > >         } else {
> > >                 dma_resv_lock(shmem->base.resv, NULL);
> > >
> > > +               /* take out shmem GEM object from the memory shrinker */
> > > +               drm_gem_shmem_madvise(shmem, -1);
> > > +
> > >                 WARN_ON(shmem->vmap_use_count);
> > >
> > >                 if (shmem->sgt) {
> > > @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> > >                         sg_free_table(shmem->sgt);
> > >                         kfree(shmem->sgt);
> > >                 }
> > > -               if (shmem->pages)
> > > +               if (shmem->pages_use_count)
> > >                         drm_gem_shmem_put_pages(shmem);
> > >
> > >                 WARN_ON(shmem->pages_use_count);
> > > @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
> > >  }
> > >  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
> > >
> > > -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > > +/**
> > > + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> > > + * @shmem: shmem GEM object
> > > + *
> > > + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> > > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       dma_resv_lock(shmem->base.resv, NULL);
> > > +
> > > +       if (shmem->madv < 0)
> > > +               return -ENOMEM;
> > > +
> > > +       shmem->eviction_enabled = true;
> > > +
> > > +       dma_resv_unlock(shmem->base.resv);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> > > +
> > > +/**
> > > + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> > > + * @shmem: shmem GEM object
> > > + *
> > > + * Tell memory shrinker that this GEM can be purged. Initially purging is
> > > + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       dma_resv_lock(shmem->base.resv, NULL);
> > > +
> > > +       if (shmem->madv < 0)
> > > +               return -ENOMEM;
> > > +
> > > +       shmem->purge_enabled = true;
> > > +
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       dma_resv_unlock(shmem->base.resv);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> > > +
> > > +static int
> > > +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
> > >  {
> > >         struct drm_gem_object *obj = &shmem->base;
> > >         struct page **pages;
> > >
> > > -       if (shmem->pages_use_count++ > 0)
> > > +       dma_resv_assert_held(shmem->base.resv);
> > > +
> > > +       if (shmem->madv < 0) {
> > > +               WARN_ON(shmem->pages);
> > > +               return -ENOMEM;
> > > +       }
> > > +
> > > +       if (shmem->pages) {
> > > +               WARN_ON(!shmem->evicted);
> > >                 return 0;
> > > +       }
> > > +
> > > +       if (WARN_ON(!shmem->pages_use_count))
> > > +               return -EINVAL;
> > >
> > >         pages = drm_gem_get_pages(obj);
> > >         if (IS_ERR(pages)) {
> > >                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> > > -               shmem->pages_use_count = 0;
> > >                 return PTR_ERR(pages);
> > >         }
> > >
> > > @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > >         return 0;
> > >  }
> > >
> > > +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       int err;
> > > +
> > > +       dma_resv_assert_held(shmem->base.resv);
> > > +
> > > +       if (shmem->madv < 0)
> > > +               return -ENOMEM;
> > > +
> > > +       if (shmem->pages_use_count++ > 0) {
> > > +               err = drm_gem_shmem_swap_in(shmem);
> > > +               if (err)
> > > +                       goto err_zero_use;
> > > +
> > > +               return 0;
> > > +       }
> > > +
> > > +       err = drm_gem_shmem_acquire_pages(shmem);
> > > +       if (err)
> > > +               goto err_zero_use;
> > > +
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       return 0;
> > > +
> > > +err_zero_use:
> > > +       shmem->pages_use_count = 0;
> > > +
> > > +       return err;
> > > +}
> > > +
> > > +static void
> > > +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +
> > > +       if (!shmem->pages) {
> > > +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> > > +               return;
> > > +       }
> > > +
> > > +#ifdef CONFIG_X86
> > > +       if (shmem->map_wc)
> > > +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > > +#endif
> > > +
> > > +       drm_gem_put_pages(obj, shmem->pages,
> > > +                         shmem->pages_mark_dirty_on_put,
> > > +                         shmem->pages_mark_accessed_on_put);
> > > +       shmem->pages = NULL;
> > > +}
> > > +
> > >  /*
> > >   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
> > >   * @shmem: shmem GEM object
> > > @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> > >   */
> > >  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> > >  {
> > > -       struct drm_gem_object *obj = &shmem->base;
> > > -
> > >         dma_resv_assert_held(shmem->base.resv);
> > >
> > >         if (WARN_ON_ONCE(!shmem->pages_use_count))
> > > @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
> > >         if (--shmem->pages_use_count > 0)
> > >                 return;
> > >
> > > -#ifdef CONFIG_X86
> > > -       if (shmem->map_wc)
> > > -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> > > -#endif
> > > +       drm_gem_shmem_release_pages(shmem);
> > >
> > > -       drm_gem_put_pages(obj, shmem->pages,
> > > -                         shmem->pages_mark_dirty_on_put,
> > > -                         shmem->pages_mark_accessed_on_put);
> > > -       shmem->pages = NULL;
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> > >
> > > @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
> > >   */
> > >  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
> > >  {
> > > +       int ret;
> > > +
> > >         dma_resv_assert_held(shmem->base.resv);
> > >
> > >         WARN_ON(shmem->base.import_attach);
> > >
> > > -       return drm_gem_shmem_get_pages(shmem);
> > > +       ret = drm_gem_shmem_get_pages(shmem);
> > > +       if (!ret)
> > > +               shmem->pages_pin_count++;
> > > +
> > > +       return ret;
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_pin);
> > >
> > > @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
> > >         WARN_ON(shmem->base.import_attach);
> > >
> > >         drm_gem_shmem_put_pages(shmem);
> > > +
> > > +       shmem->pages_pin_count--;
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_unpin);
> > >
> > > @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> > >                         return 0;
> > >                 }
> > >
> > > -               ret = drm_gem_shmem_get_pages(shmem);
> > > +               ret = drm_gem_shmem_pin(shmem);
> > >                 if (ret)
> > >                         goto err_zero_use;
> > >
> > > @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
> > >
> > >  err_put_pages:
> > >         if (!obj->import_attach)
> > > -               drm_gem_shmem_put_pages(shmem);
> > > +               drm_gem_shmem_unpin(shmem);
> > >  err_zero_use:
> > >         shmem->vmap_use_count = 0;
> > >
> > > @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
> > >                         return;
> > >
> > >                 vunmap(shmem->vaddr);
> > > -               drm_gem_shmem_put_pages(shmem);
> > > +               drm_gem_shmem_unpin(shmem);
> > >         }
> > >
> > >         shmem->vaddr = NULL;
> > > @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
> > >
> > >         madv = shmem->madv;
> > >
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > >         return (madv >= 0);
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_madvise);
> > >
> > > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > > +/**
> > > + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> > > + *                           hardware access to the memory.
> > > + * @shmem: shmem GEM object
> > > + *
> > > + * This function moves shmem GEM back to memory if it was previously evicted
> > > + * by the memory shrinker. The GEM is ready to use on success.
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
> > >  {
> > >         struct drm_gem_object *obj = &shmem->base;
> > > -       struct drm_device *dev = obj->dev;
> > > +       struct sg_table *sgt;
> > > +       int err;
> > >
> > >         dma_resv_assert_held(shmem->base.resv);
> > >
> > > -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > > +       if (shmem->evicted) {
> > > +               err = drm_gem_shmem_acquire_pages(shmem);
> > > +               if (err)
> > > +                       return err;
> > > +
> > > +               sgt = drm_gem_shmem_get_sg_table(shmem);
> > > +               if (IS_ERR(sgt))
> > > +                       return PTR_ERR(sgt);
> > > +
> > > +               err = dma_map_sgtable(obj->dev->dev, sgt,
> > > +                                     DMA_BIDIRECTIONAL, 0);
> > > +               if (err) {
> > > +                       sg_free_table(sgt);
> > > +                       kfree(sgt);
> > > +                       return err;
> > > +               }
> > >
> > > -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > > -       sg_free_table(shmem->sgt);
> > > -       kfree(shmem->sgt);
> > > -       shmem->sgt = NULL;
> > > +               shmem->sgt = sgt;
> > > +               shmem->evicted = false;
> > >
> > > -       drm_gem_shmem_put_pages(shmem);
> > > +               drm_gem_shmem_update_pages_state(shmem);
> > > +       }
> > >
> > > -       shmem->madv = -1;
> > > +       if (!shmem->pages)
> > > +               return -ENOMEM;
> > >
> > > -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > > -       drm_gem_free_mmap_offset(obj);
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
> > >
> > > -       /* Our goal here is to return as much of the memory as
> > > -        * is possible back to the system as we are called from OOM.
> > > -        * To do this we must instruct the shmfs to drop all of its
> > > -        * backing pages, *now*.
> > > -        */
> > > -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > > +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +       struct drm_device *dev = obj->dev;
> > >
> > > -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > > +       if (shmem->evicted)
> > > +               return;
> > > +
> > > +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> > > +       drm_gem_shmem_release_pages(shmem);
> > > +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> > > +
> > > +       sg_free_table(shmem->sgt);
> > > +       kfree(shmem->sgt);
> > > +       shmem->sgt = NULL;
> > >  }
> > > -EXPORT_SYMBOL(drm_gem_shmem_purge);
> > >
> > >  /**
> > >   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> > > @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
> > >         vm_fault_t ret;
> > >         struct page *page;
> > >         pgoff_t page_offset;
> > > +       bool pages_unpinned;
> > > +       int err;
> > >
> > >         /* We don't use vmf->pgoff since that has the fake offset */
> > >         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
> > >
> > >         dma_resv_lock(shmem->base.resv, NULL);
> > >
> > > -       if (page_offset >= num_pages ||
> > > -           WARN_ON_ONCE(!shmem->pages) ||
> > > -           shmem->madv < 0) {
> > > +       /* Sanity-check that we have the pages pointer when it should present */
> > > +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> > > +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> > > +
> > > +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
> > >                 ret = VM_FAULT_SIGBUS;
> > >         } else {
> > > +               err = drm_gem_shmem_swap_in(shmem);
> > > +               if (err) {
> > > +                       ret = VM_FAULT_OOM;
> > > +                       goto unlock;
> > > +               }
> > > +
> > >                 page = shmem->pages[page_offset];
> > >
> > >                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
> > >         }
> > >
> > > +unlock:
> > >         dma_resv_unlock(shmem->base.resv);
> > >
> > >         return ret;
> > > @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
> > >  {
> > >         struct drm_gem_object *obj = vma->vm_private_data;
> > >         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> > > -       int ret;
> > >
> > >         WARN_ON(shmem->base.import_attach);
> > >
> > >         dma_resv_lock(shmem->base.resv, NULL);
> > > -       ret = drm_gem_shmem_get_pages(shmem);
> > > -       WARN_ON_ONCE(ret != 0);
> > > +
> > > +       if (drm_gem_shmem_get_pages(shmem))
> > > +               shmem->pages_use_count++;
> > > +
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > >         dma_resv_unlock(shmem->base.resv);
> > >
> > >         drm_gem_vm_open(vma);
> > > @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
> > >  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> > >                               struct drm_printer *p, unsigned int indent)
> > >  {
> > > +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> > > +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
> > >         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
> > >
> > >         if (shmem->base.import_attach)
> > > @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
> > >                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
> > >                                   shmem->vmap_use_count);
> > >
> > > +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
> > >         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> > > +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
> > >  }
> > >  EXPORT_SYMBOL(drm_gem_shmem_print_info);
> > >
> > > @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
> > >
> > >         shmem->sgt = sgt;
> > >
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > >         dma_resv_unlock(shmem->base.resv);
> > >
> > >         return sgt;
> > > @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
> > >  }
> > >  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
> > >
> > > +static struct drm_gem_shmem_shrinker *
> > > +to_drm_shrinker(struct shrinker *shrinker)
> > > +{
> > > +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> > > +}
> > > +
> > > +static unsigned long
> > > +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> > > +                                    struct shrink_control *sc)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > > +       struct drm_gem_shmem_object *shmem;
> > > +       unsigned long count = 0;
> > > +
> > > +       if (!mutex_trylock(&gem_shrinker->lock))
> > > +               return 0;
> > > +
> > > +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> > > +               count += shmem->base.size;
> > > +
> > > +               if (count >= SHRINK_EMPTY)
> > > +                       break;
> > > +       }
> > > +
> > > +       mutex_unlock(&gem_shrinker->lock);
> > > +
> > > +       if (count >= SHRINK_EMPTY)
> > > +               return SHRINK_EMPTY - 1;
> > > +
> > > +       return count ?: SHRINK_EMPTY;
> > > +}
> > > +
> > > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> > > +       WARN_ON(shmem->evicted);
> > > +
> > > +       drm_gem_shmem_unpin_pages(shmem);
> > > +
> > > +       shmem->evicted = true;
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> > > +
> > > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       struct drm_gem_object *obj = &shmem->base;
> > > +
> > > +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> > > +
> > > +       drm_gem_shmem_unpin_pages(shmem);
> > > +       drm_gem_free_mmap_offset(obj);
> > > +
> > > +       /* Our goal here is to return as much of the memory as
> > > +        * is possible back to the system as we are called from OOM.
> > > +        * To do this we must instruct the shmfs to drop all of its
> > > +        * backing pages, *now*.
> > > +        */
> > > +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> > > +
> > > +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> > > +
> > > +       shmem->madv = -1;
> > > +       shmem->evicted = false;
> > > +       drm_gem_shmem_update_pages_state(shmem);
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> > > +
> > > +static unsigned long
> > > +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> > > +                                       unsigned long nr_to_scan,
> > > +                                       bool *lock_contention,
> > > +                                       bool evict)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > > +       struct drm_gem_shmem_object *shmem;
> > > +       struct list_head still_in_list;
> > > +       struct drm_gem_object *obj;
> > > +       unsigned long freed = 0;
> > > +       size_t page_count;
> > > +       int err;
> > > +
> > > +       INIT_LIST_HEAD(&still_in_list);
> > > +
> > > +       mutex_lock(&gem_shrinker->lock);
> > > +
> > > +       while (freed < nr_to_scan) {
> > > +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> > > +                                                typeof(*shmem), madv_list);
> > > +               if (!shmem)
> > > +                       break;
> > > +
> > > +               obj = &shmem->base;
> > > +               page_count = obj->size >> PAGE_SHIFT;
> > > +               list_move_tail(&shmem->madv_list, &still_in_list);
> > > +
> > > +               if (evict) {
> > > +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> > > +                           get_nr_swap_pages() < page_count)
> > > +                               continue;
> > > +               } else {
> > > +                       if (!drm_gem_shmem_is_purgeable(shmem))
> > > +                               continue;
> > > +               }
> > > +
> > > +               /*
> > > +                * If it's in the process of being freed, gem_object->free()
> > > +                * may be blocked on lock waiting to remove it.  So just
> > > +                * skip it.
> > > +                */
> > > +               if (!kref_get_unless_zero(&obj->refcount))
> > > +                       continue;
> > > +
> > > +               mutex_unlock(&gem_shrinker->lock);
> > > +
> > > +               /* prevent racing with job-submission code paths */
> > > +               if (!dma_resv_trylock(obj->resv)) {
> > > +                       *lock_contention |= true;
> > > +                       goto shrinker_lock;
> > > +               }
> > > +
> > > +               /* prevent racing with the dma-buf importing/exporting */
> > > +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > > +                       *lock_contention |= true;
> > > +                       goto resv_unlock;
> > > +               }
> > > +
> > > +               /* check whether h/w uses this object */
> > > +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> > > +                       goto object_name_unlock;
> > > +
> > > +               /* re-check whether eviction status hasn't changed */
> > > +               if (!drm_gem_shmem_is_evictable(shmem) &&
> > > +                   !drm_gem_shmem_is_purgeable(shmem))
> > > +                       goto object_name_unlock;
> > > +
> > > +               err = shmem->evict(shmem);
> > > +               if (!err)
> > > +                       freed += obj->size >> PAGE_SHIFT;
> > > +
> > > +object_name_unlock:
> > > +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> > > +resv_unlock:
> > > +               dma_resv_unlock(obj->resv);
> > > +shrinker_lock:
> > > +               drm_gem_object_put(&shmem->base);
> > > +               mutex_lock(&gem_shrinker->lock);
> > > +       }
> > > +
> > > +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> > > +
> > > +       mutex_unlock(&gem_shrinker->lock);
> > > +
> > > +       return freed;
> > > +}
> > > +
> > > +static unsigned long
> > > +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> > > +                                   struct shrink_control *sc)
> > > +{
> > > +       unsigned long nr_to_scan = sc->nr_to_scan;
> > > +       bool lock_contention = false;
> > > +       unsigned long freed;
> > > +
> > > +       /* purge as many objects as we can */
> > > +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> > > +                                                       &lock_contention, false);
> > > +
> > > +       /* evict as many objects as we can */
> > > +       if (freed < nr_to_scan)
> > > +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> > > +                                                                nr_to_scan - freed,
> > > +                                                                &lock_contention,
> > > +                                                                true);
> > > +
> > > +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> > > +}
> > > +
> > > +/**
> > > + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> > > + * @dev: DRM device
> > > + *
> > > + * Returns:
> > > + * 0 on success or a negative error code on failure.
> > > + */
> > > +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker;
> > > +       int err;
> > > +
> > > +       if (WARN_ON(dev->shmem_shrinker))
> > > +               return -EBUSY;
> > > +
> > > +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> > > +       if (!gem_shrinker)
> > > +               return -ENOMEM;
> > > +
> > > +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> > > +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> > > +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> > > +       gem_shrinker->dev = dev;
> > > +
> > > +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> > > +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> > > +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> > > +       mutex_init(&gem_shrinker->lock);
> > > +
> > > +       dev->shmem_shrinker = gem_shrinker;
> > > +
> > > +       err = register_shrinker(&gem_shrinker->base);
> > > +       if (err) {
> > > +               dev->shmem_shrinker = NULL;
> > > +               kfree(gem_shrinker);
> > > +               return err;
> > > +       }
> > > +
> > > +       return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> > > +
> > > +/**
> > > + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> > > + * @dev: DRM device
> > > + */
> > > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> > > +{
> > > +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> > > +
> > > +       if (gem_shrinker) {
> > > +               unregister_shrinker(&gem_shrinker->base);
> > > +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> > > +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> > > +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> > > +               mutex_destroy(&gem_shrinker->lock);
> > > +               dev->shmem_shrinker = NULL;
> > > +               kfree(gem_shrinker);
> > > +       }
> > > +}
> > > +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> > > +
> > >  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
> > >  MODULE_IMPORT_NS(DMA_BUF);
> > >  MODULE_LICENSE("GPL v2");
> > > diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > > index a4bedfeb2ec4..7cc32556f908 100644
> > > --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > > +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> > > @@ -15,6 +15,13 @@
> > >  #include "panfrost_gem.h"
> > >  #include "panfrost_mmu.h"
> > >
> > > +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> > > +{
> > > +       return (shmem->madv > 0) &&
> > > +               !shmem->pages_pin_count && shmem->sgt &&
> > > +               !shmem->base.dma_buf && !shmem->base.import_attach;
> > > +}
> > > +
> > >  static unsigned long
> > >  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
> > >  {
> > > @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
> > >                 return 0;
> > >
> > >         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> > > -               if (drm_gem_shmem_is_purgeable(shmem))
> > > +               if (panfrost_gem_shmem_is_purgeable(shmem))
> > >                         count += shmem->base.size >> PAGE_SHIFT;
> > >         }
> > >
> > > diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > > index b2d93cb12ebf..81bacc7e1873 100644
> > > --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> > > +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> > > @@ -89,6 +89,7 @@ struct virtio_gpu_object {
> > >         uint32_t hw_res_handle;
> > >         bool dumb;
> > >         bool created;
> > > +       bool detached;
> > >         bool host3d_blob, guest_blob;
> > >         uint32_t blob_mem, blob_flags;
> > >
> > > @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
> > >
> > >  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
> > >
> > > +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> > > +
> > >  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
> > >                                uint32_t *resid);
> > >  /* virtgpu_prime.c */
> > > diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> > > index 9923c7a6885e..929546cad894 100644
> > > --- a/include/drm/drm_device.h
> > > +++ b/include/drm/drm_device.h
> > > @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
> > >  struct drm_vma_offset_manager;
> > >  struct drm_vram_mm;
> > >  struct drm_fb_helper;
> > > +struct drm_gem_shmem_shrinker;
> > >
> > >  struct inode;
> > >
> > > @@ -277,6 +278,9 @@ struct drm_device {
> > >         /** @vram_mm: VRAM MM memory manager */
> > >         struct drm_vram_mm *vram_mm;
> > >
> > > +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> > > +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> > > +
> > >         /**
> > >          * @switch_power_state:
> > >          *
> > > diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> > > index 9a8983ee8abe..62c640678a91 100644
> > > --- a/include/drm/drm_gem_shmem_helper.h
> > > +++ b/include/drm/drm_gem_shmem_helper.h
> > > @@ -6,6 +6,7 @@
> > >  #include <linux/fs.h>
> > >  #include <linux/mm.h>
> > >  #include <linux/mutex.h>
> > > +#include <linux/shrinker.h>
> > >
> > >  #include <drm/drm_file.h>
> > >  #include <drm/drm_gem.h>
> > > @@ -15,6 +16,7 @@
> > >  struct dma_buf_attachment;
> > >  struct drm_mode_create_dumb;
> > >  struct drm_printer;
> > > +struct drm_device;
> > >  struct sg_table;
> > >
> > >  /**
> > > @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
> > >          */
> > >         unsigned int pages_use_count;
> > >
> > > +       /**
> > > +        * @pages_pin_count:
> > > +        *
> > > +        * Reference count on the pinned pages table.
> > > +        * The pages can be evicted by memory shrinker
> > > +        * when the count reaches zero.
> > > +        */
> > > +       unsigned int pages_pin_count;
> > > +
> > >         /**
> > >          * @madv: State for madvise
> > >          *
> > >          * 0 is active/inuse.
> > > +        * 1 is not-needed/can-be-purged
> > >          * A negative value is the object is purged.
> > > -        * Positive values are driver specific and not used by the helpers.
> > >          */
> > >         int madv;
> > >
> > > @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
> > >          * @map_wc: map object write-combined (instead of using shmem defaults).
> > >          */
> > >         bool map_wc;
> > > +
> > > +       /**
> > > +        * @eviction_enabled:
> > > +        *
> > > +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> > > +        * Used internally by memory shrinker.
> > > +        */
> > > +       bool eviction_enabled;
> > > +
> > > +       /**
> > > +        * @purge_enabled:
> > > +        *
> > > +        * The shmem pages can be purged only if @purge_enabled is set to true.
> > > +        * Used internally by memory shrinker.
> > > +        */
> > > +       bool purge_enabled;
> > > +
> > > +       /**
> > > +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> > > +        * Used internally by memory shrinker.
> > > +        */
> > > +       bool evicted;
> > > +
> > > +       /**
> > > +        * @evict:
> > > +        *
> > > +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> > > +        * GEM's DMA reservation is kept locked by the shrinker. This is
> > > +        * optional callback that should be specified by drivers.
> > > +        *
> > > +        * Returns 0 on success, or -errno on error.
> > > +        */
> > > +       int (*evict)(struct drm_gem_shmem_object *shmem);
> > >  };
> > >
> > >  #define to_drm_gem_shmem_obj(obj) \
> > > @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
> > >
> > >  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
> > >
> > > +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> > > +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> > > +
> > >  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> > >  {
> > > -       return (shmem->madv > 0) &&
> > > -               !shmem->vmap_use_count && shmem->sgt &&
> > > -               !shmem->base.dma_buf && !shmem->base.import_attach;
> > > +       return (shmem->madv > 0) && shmem->evict &&
> > > +               shmem->purge_enabled && shmem->pages_use_count &&
> > > +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> > > +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
> > >  }
> > >
> > > -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> > > +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> > > +
> > > +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> > > +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> > >
> > >  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
> > >  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> > > @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
> > >         return drm_gem_shmem_mmap(shmem, vma);
> > >  }
> > >
> > > +/**
> > > + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> > > + */
> > > +struct drm_gem_shmem_shrinker {
> > > +       /** @base: Shrinker for purging shmem GEM objects */
> > > +       struct shrinker base;
> > > +
> > > +       /** @lock: Protects @lru_* */
> > > +       struct mutex lock;
> > > +
> > > +       /** @lru_pinned: List of pinned shmem GEM objects */
> > > +       struct list_head lru_pinned;
> > > +
> > > +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> > > +       struct list_head lru_evictable;
> > > +
> > > +       /** @lru_evicted: List of evicted shmem GEM objects */
> > > +       struct list_head lru_evicted;
> > > +
> > > +       /** @dev: DRM device that uses this shrinker */
> > > +       struct drm_device *dev;
> > > +};
> > > +
> > > +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> > > +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> > > +
> > >  /*
> > >   * Driver ops
> > >   */
> > > --
> > > 2.35.3
> > >
> >
> >
> > --
> > Daniel Vetter
> > Software Engineer, Intel Corporation
> > http://blog.ffwll.ch



-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Christian König]

Am 05.06.22 um 18:47 schrieb Daniel Vetter:
> On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
>> Introduce a common DRM SHMEM shrinker framework that allows to reduce
>> code duplication among DRM drivers by replacing theirs custom shrinker
>> implementations with the generic shrinker.
>>
>> In order to start using DRM SHMEM shrinker drivers should:
>>
>> 1. Implement new evict() shmem object callback.
>> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
>> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>>     activate shrinking of shmem GEMs.
>>
>> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
>> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>>
>> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> So I guess I get a price for being blind since forever, because this
> thing existed since at least 2013. I just stumbled over
> llist_lru.[hc], a purpose built list helper for shrinkers. I think we
> should try to adopt that so that our gpu shrinkers look more like
> shrinkers for everything else.

What the heck are you talking about?

I can't find any llist_lru.[hc] in the linux kernel sources.

Christian.

>
> Apologies for this, since I fear this might cause a bit of churn.
> Hopefully it's all contained to the list manipulation code in shmem
> helpers, I don't think this should leak any further.
> -Daniel
>
>> ---
>>   drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>>   .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>>   drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>>   include/drm/drm_device.h                      |   4 +
>>   include/drm/drm_gem_shmem_helper.h            |  87 ++-
>>   5 files changed, 594 insertions(+), 49 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
>> index 555fe212bd98..4cd0b5913492 100644
>> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
>> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
>> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>>   }
>>   EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>>
>> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       return (shmem->madv >= 0) && shmem->evict &&
>> +               shmem->eviction_enabled && shmem->pages_use_count &&
>> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
>> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
>> +}
>> +
>> +static void
>> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
>> +
>> +       dma_resv_assert_held(shmem->base.resv);
>> +
>> +       if (!gem_shrinker || obj->import_attach)
>> +               return;
>> +
>> +       mutex_lock(&gem_shrinker->lock);
>> +
>> +       if (drm_gem_shmem_is_evictable(shmem) ||
>> +           drm_gem_shmem_is_purgeable(shmem))
>> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
>> +       else if (shmem->madv < 0)
>> +               list_del_init(&shmem->madv_list);
>> +       else if (shmem->evicted)
>> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
>> +       else if (!shmem->pages)
>> +               list_del_init(&shmem->madv_list);
>> +       else
>> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
>> +
>> +       mutex_unlock(&gem_shrinker->lock);
>> +}
>> +
>>   /**
>>    * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>>    * @shmem: shmem GEM object to free
>> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>>          } else {
>>                  dma_resv_lock(shmem->base.resv, NULL);
>>
>> +               /* take out shmem GEM object from the memory shrinker */
>> +               drm_gem_shmem_madvise(shmem, -1);
>> +
>>                  WARN_ON(shmem->vmap_use_count);
>>
>>                  if (shmem->sgt) {
>> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>>                          sg_free_table(shmem->sgt);
>>                          kfree(shmem->sgt);
>>                  }
>> -               if (shmem->pages)
>> +               if (shmem->pages_use_count)
>>                          drm_gem_shmem_put_pages(shmem);
>>
>>                  WARN_ON(shmem->pages_use_count);
>> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>>   }
>>   EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>>
>> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>> +/**
>> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
>> + * @shmem: shmem GEM object
>> + *
>> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
>> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       dma_resv_lock(shmem->base.resv, NULL);
>> +
>> +       if (shmem->madv < 0)
>> +               return -ENOMEM;
>> +
>> +       shmem->eviction_enabled = true;
>> +
>> +       dma_resv_unlock(shmem->base.resv);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
>> +
>> +/**
>> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
>> + * @shmem: shmem GEM object
>> + *
>> + * Tell memory shrinker that this GEM can be purged. Initially purging is
>> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       dma_resv_lock(shmem->base.resv, NULL);
>> +
>> +       if (shmem->madv < 0)
>> +               return -ENOMEM;
>> +
>> +       shmem->purge_enabled = true;
>> +
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       dma_resv_unlock(shmem->base.resv);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
>> +
>> +static int
>> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>>   {
>>          struct drm_gem_object *obj = &shmem->base;
>>          struct page **pages;
>>
>> -       if (shmem->pages_use_count++ > 0)
>> +       dma_resv_assert_held(shmem->base.resv);
>> +
>> +       if (shmem->madv < 0) {
>> +               WARN_ON(shmem->pages);
>> +               return -ENOMEM;
>> +       }
>> +
>> +       if (shmem->pages) {
>> +               WARN_ON(!shmem->evicted);
>>                  return 0;
>> +       }
>> +
>> +       if (WARN_ON(!shmem->pages_use_count))
>> +               return -EINVAL;
>>
>>          pages = drm_gem_get_pages(obj);
>>          if (IS_ERR(pages)) {
>>                  DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
>> -               shmem->pages_use_count = 0;
>>                  return PTR_ERR(pages);
>>          }
>>
>> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>>          return 0;
>>   }
>>
>> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>> +{
>> +       int err;
>> +
>> +       dma_resv_assert_held(shmem->base.resv);
>> +
>> +       if (shmem->madv < 0)
>> +               return -ENOMEM;
>> +
>> +       if (shmem->pages_use_count++ > 0) {
>> +               err = drm_gem_shmem_swap_in(shmem);
>> +               if (err)
>> +                       goto err_zero_use;
>> +
>> +               return 0;
>> +       }
>> +
>> +       err = drm_gem_shmem_acquire_pages(shmem);
>> +       if (err)
>> +               goto err_zero_use;
>> +
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       return 0;
>> +
>> +err_zero_use:
>> +       shmem->pages_use_count = 0;
>> +
>> +       return err;
>> +}
>> +
>> +static void
>> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +
>> +       if (!shmem->pages) {
>> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
>> +               return;
>> +       }
>> +
>> +#ifdef CONFIG_X86
>> +       if (shmem->map_wc)
>> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
>> +#endif
>> +
>> +       drm_gem_put_pages(obj, shmem->pages,
>> +                         shmem->pages_mark_dirty_on_put,
>> +                         shmem->pages_mark_accessed_on_put);
>> +       shmem->pages = NULL;
>> +}
>> +
>>   /*
>>    * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>>    * @shmem: shmem GEM object
>> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>>    */
>>   void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>>   {
>> -       struct drm_gem_object *obj = &shmem->base;
>> -
>>          dma_resv_assert_held(shmem->base.resv);
>>
>>          if (WARN_ON_ONCE(!shmem->pages_use_count))
>> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>>          if (--shmem->pages_use_count > 0)
>>                  return;
>>
>> -#ifdef CONFIG_X86
>> -       if (shmem->map_wc)
>> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
>> -#endif
>> +       drm_gem_shmem_release_pages(shmem);
>>
>> -       drm_gem_put_pages(obj, shmem->pages,
>> -                         shmem->pages_mark_dirty_on_put,
>> -                         shmem->pages_mark_accessed_on_put);
>> -       shmem->pages = NULL;
>> +       drm_gem_shmem_update_pages_state(shmem);
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>>
>> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>>    */
>>   int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>>   {
>> +       int ret;
>> +
>>          dma_resv_assert_held(shmem->base.resv);
>>
>>          WARN_ON(shmem->base.import_attach);
>>
>> -       return drm_gem_shmem_get_pages(shmem);
>> +       ret = drm_gem_shmem_get_pages(shmem);
>> +       if (!ret)
>> +               shmem->pages_pin_count++;
>> +
>> +       return ret;
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_pin);
>>
>> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>>          WARN_ON(shmem->base.import_attach);
>>
>>          drm_gem_shmem_put_pages(shmem);
>> +
>> +       shmem->pages_pin_count--;
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_unpin);
>>
>> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>>                          return 0;
>>                  }
>>
>> -               ret = drm_gem_shmem_get_pages(shmem);
>> +               ret = drm_gem_shmem_pin(shmem);
>>                  if (ret)
>>                          goto err_zero_use;
>>
>> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>>
>>   err_put_pages:
>>          if (!obj->import_attach)
>> -               drm_gem_shmem_put_pages(shmem);
>> +               drm_gem_shmem_unpin(shmem);
>>   err_zero_use:
>>          shmem->vmap_use_count = 0;
>>
>> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>>                          return;
>>
>>                  vunmap(shmem->vaddr);
>> -               drm_gem_shmem_put_pages(shmem);
>> +               drm_gem_shmem_unpin(shmem);
>>          }
>>
>>          shmem->vaddr = NULL;
>> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>>
>>          madv = shmem->madv;
>>
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>>          return (madv >= 0);
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_madvise);
>>
>> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
>> +/**
>> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
>> + *                           hardware access to the memory.
>> + * @shmem: shmem GEM object
>> + *
>> + * This function moves shmem GEM back to memory if it was previously evicted
>> + * by the memory shrinker. The GEM is ready to use on success.
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>>   {
>>          struct drm_gem_object *obj = &shmem->base;
>> -       struct drm_device *dev = obj->dev;
>> +       struct sg_table *sgt;
>> +       int err;
>>
>>          dma_resv_assert_held(shmem->base.resv);
>>
>> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
>> +       if (shmem->evicted) {
>> +               err = drm_gem_shmem_acquire_pages(shmem);
>> +               if (err)
>> +                       return err;
>> +
>> +               sgt = drm_gem_shmem_get_sg_table(shmem);
>> +               if (IS_ERR(sgt))
>> +                       return PTR_ERR(sgt);
>> +
>> +               err = dma_map_sgtable(obj->dev->dev, sgt,
>> +                                     DMA_BIDIRECTIONAL, 0);
>> +               if (err) {
>> +                       sg_free_table(sgt);
>> +                       kfree(sgt);
>> +                       return err;
>> +               }
>>
>> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
>> -       sg_free_table(shmem->sgt);
>> -       kfree(shmem->sgt);
>> -       shmem->sgt = NULL;
>> +               shmem->sgt = sgt;
>> +               shmem->evicted = false;
>>
>> -       drm_gem_shmem_put_pages(shmem);
>> +               drm_gem_shmem_update_pages_state(shmem);
>> +       }
>>
>> -       shmem->madv = -1;
>> +       if (!shmem->pages)
>> +               return -ENOMEM;
>>
>> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
>> -       drm_gem_free_mmap_offset(obj);
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>>
>> -       /* Our goal here is to return as much of the memory as
>> -        * is possible back to the system as we are called from OOM.
>> -        * To do this we must instruct the shmfs to drop all of its
>> -        * backing pages, *now*.
>> -        */
>> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
>> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +       struct drm_device *dev = obj->dev;
>>
>> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
>> +       if (shmem->evicted)
>> +               return;
>> +
>> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
>> +       drm_gem_shmem_release_pages(shmem);
>> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
>> +
>> +       sg_free_table(shmem->sgt);
>> +       kfree(shmem->sgt);
>> +       shmem->sgt = NULL;
>>   }
>> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>>
>>   /**
>>    * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
>> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>>          vm_fault_t ret;
>>          struct page *page;
>>          pgoff_t page_offset;
>> +       bool pages_unpinned;
>> +       int err;
>>
>>          /* We don't use vmf->pgoff since that has the fake offset */
>>          page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>>
>>          dma_resv_lock(shmem->base.resv, NULL);
>>
>> -       if (page_offset >= num_pages ||
>> -           WARN_ON_ONCE(!shmem->pages) ||
>> -           shmem->madv < 0) {
>> +       /* Sanity-check that we have the pages pointer when it should present */
>> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
>> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
>> +
>> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>>                  ret = VM_FAULT_SIGBUS;
>>          } else {
>> +               err = drm_gem_shmem_swap_in(shmem);
>> +               if (err) {
>> +                       ret = VM_FAULT_OOM;
>> +                       goto unlock;
>> +               }
>> +
>>                  page = shmem->pages[page_offset];
>>
>>                  ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>>          }
>>
>> +unlock:
>>          dma_resv_unlock(shmem->base.resv);
>>
>>          return ret;
>> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>>   {
>>          struct drm_gem_object *obj = vma->vm_private_data;
>>          struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
>> -       int ret;
>>
>>          WARN_ON(shmem->base.import_attach);
>>
>>          dma_resv_lock(shmem->base.resv, NULL);
>> -       ret = drm_gem_shmem_get_pages(shmem);
>> -       WARN_ON_ONCE(ret != 0);
>> +
>> +       if (drm_gem_shmem_get_pages(shmem))
>> +               shmem->pages_use_count++;
>> +
>> +       drm_gem_shmem_update_pages_state(shmem);
>>          dma_resv_unlock(shmem->base.resv);
>>
>>          drm_gem_vm_open(vma);
>> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>>   void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>>                                struct drm_printer *p, unsigned int indent)
>>   {
>> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
>> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>>          drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>>
>>          if (shmem->base.import_attach)
>> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>>                  drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>>                                    shmem->vmap_use_count);
>>
>> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>>          drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
>> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_print_info);
>>
>> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>>
>>          shmem->sgt = sgt;
>>
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>>          dma_resv_unlock(shmem->base.resv);
>>
>>          return sgt;
>> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>>   }
>>   EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>>
>> +static struct drm_gem_shmem_shrinker *
>> +to_drm_shrinker(struct shrinker *shrinker)
>> +{
>> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
>> +}
>> +
>> +static unsigned long
>> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
>> +                                    struct shrink_control *sc)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
>> +       struct drm_gem_shmem_object *shmem;
>> +       unsigned long count = 0;
>> +
>> +       if (!mutex_trylock(&gem_shrinker->lock))
>> +               return 0;
>> +
>> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
>> +               count += shmem->base.size;
>> +
>> +               if (count >= SHRINK_EMPTY)
>> +                       break;
>> +       }
>> +
>> +       mutex_unlock(&gem_shrinker->lock);
>> +
>> +       if (count >= SHRINK_EMPTY)
>> +               return SHRINK_EMPTY - 1;
>> +
>> +       return count ?: SHRINK_EMPTY;
>> +}
>> +
>> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
>> +{
>> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
>> +       WARN_ON(shmem->evicted);
>> +
>> +       drm_gem_shmem_unpin_pages(shmem);
>> +
>> +       shmem->evicted = true;
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
>> +
>> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +
>> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
>> +
>> +       drm_gem_shmem_unpin_pages(shmem);
>> +       drm_gem_free_mmap_offset(obj);
>> +
>> +       /* Our goal here is to return as much of the memory as
>> +        * is possible back to the system as we are called from OOM.
>> +        * To do this we must instruct the shmfs to drop all of its
>> +        * backing pages, *now*.
>> +        */
>> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
>> +
>> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
>> +
>> +       shmem->madv = -1;
>> +       shmem->evicted = false;
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
>> +
>> +static unsigned long
>> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
>> +                                       unsigned long nr_to_scan,
>> +                                       bool *lock_contention,
>> +                                       bool evict)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
>> +       struct drm_gem_shmem_object *shmem;
>> +       struct list_head still_in_list;
>> +       struct drm_gem_object *obj;
>> +       unsigned long freed = 0;
>> +       size_t page_count;
>> +       int err;
>> +
>> +       INIT_LIST_HEAD(&still_in_list);
>> +
>> +       mutex_lock(&gem_shrinker->lock);
>> +
>> +       while (freed < nr_to_scan) {
>> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
>> +                                                typeof(*shmem), madv_list);
>> +               if (!shmem)
>> +                       break;
>> +
>> +               obj = &shmem->base;
>> +               page_count = obj->size >> PAGE_SHIFT;
>> +               list_move_tail(&shmem->madv_list, &still_in_list);
>> +
>> +               if (evict) {
>> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
>> +                           get_nr_swap_pages() < page_count)
>> +                               continue;
>> +               } else {
>> +                       if (!drm_gem_shmem_is_purgeable(shmem))
>> +                               continue;
>> +               }
>> +
>> +               /*
>> +                * If it's in the process of being freed, gem_object->free()
>> +                * may be blocked on lock waiting to remove it.  So just
>> +                * skip it.
>> +                */
>> +               if (!kref_get_unless_zero(&obj->refcount))
>> +                       continue;
>> +
>> +               mutex_unlock(&gem_shrinker->lock);
>> +
>> +               /* prevent racing with job-submission code paths */
>> +               if (!dma_resv_trylock(obj->resv)) {
>> +                       *lock_contention |= true;
>> +                       goto shrinker_lock;
>> +               }
>> +
>> +               /* prevent racing with the dma-buf importing/exporting */
>> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
>> +                       *lock_contention |= true;
>> +                       goto resv_unlock;
>> +               }
>> +
>> +               /* check whether h/w uses this object */
>> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
>> +                       goto object_name_unlock;
>> +
>> +               /* re-check whether eviction status hasn't changed */
>> +               if (!drm_gem_shmem_is_evictable(shmem) &&
>> +                   !drm_gem_shmem_is_purgeable(shmem))
>> +                       goto object_name_unlock;
>> +
>> +               err = shmem->evict(shmem);
>> +               if (!err)
>> +                       freed += obj->size >> PAGE_SHIFT;
>> +
>> +object_name_unlock:
>> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
>> +resv_unlock:
>> +               dma_resv_unlock(obj->resv);
>> +shrinker_lock:
>> +               drm_gem_object_put(&shmem->base);
>> +               mutex_lock(&gem_shrinker->lock);
>> +       }
>> +
>> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
>> +
>> +       mutex_unlock(&gem_shrinker->lock);
>> +
>> +       return freed;
>> +}
>> +
>> +static unsigned long
>> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
>> +                                   struct shrink_control *sc)
>> +{
>> +       unsigned long nr_to_scan = sc->nr_to_scan;
>> +       bool lock_contention = false;
>> +       unsigned long freed;
>> +
>> +       /* purge as many objects as we can */
>> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
>> +                                                       &lock_contention, false);
>> +
>> +       /* evict as many objects as we can */
>> +       if (freed < nr_to_scan)
>> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
>> +                                                                nr_to_scan - freed,
>> +                                                                &lock_contention,
>> +                                                                true);
>> +
>> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
>> +}
>> +
>> +/**
>> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
>> + * @dev: DRM device
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker;
>> +       int err;
>> +
>> +       if (WARN_ON(dev->shmem_shrinker))
>> +               return -EBUSY;
>> +
>> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
>> +       if (!gem_shrinker)
>> +               return -ENOMEM;
>> +
>> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
>> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
>> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
>> +       gem_shrinker->dev = dev;
>> +
>> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
>> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
>> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
>> +       mutex_init(&gem_shrinker->lock);
>> +
>> +       dev->shmem_shrinker = gem_shrinker;
>> +
>> +       err = register_shrinker(&gem_shrinker->base);
>> +       if (err) {
>> +               dev->shmem_shrinker = NULL;
>> +               kfree(gem_shrinker);
>> +               return err;
>> +       }
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
>> +
>> +/**
>> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
>> + * @dev: DRM device
>> + */
>> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
>> +
>> +       if (gem_shrinker) {
>> +               unregister_shrinker(&gem_shrinker->base);
>> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
>> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
>> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
>> +               mutex_destroy(&gem_shrinker->lock);
>> +               dev->shmem_shrinker = NULL;
>> +               kfree(gem_shrinker);
>> +       }
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
>> +
>>   MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>>   MODULE_IMPORT_NS(DMA_BUF);
>>   MODULE_LICENSE("GPL v2");
>> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
>> index a4bedfeb2ec4..7cc32556f908 100644
>> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
>> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
>> @@ -15,6 +15,13 @@
>>   #include "panfrost_gem.h"
>>   #include "panfrost_mmu.h"
>>
>> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       return (shmem->madv > 0) &&
>> +               !shmem->pages_pin_count && shmem->sgt &&
>> +               !shmem->base.dma_buf && !shmem->base.import_attach;
>> +}
>> +
>>   static unsigned long
>>   panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>>   {
>> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>>                  return 0;
>>
>>          list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
>> -               if (drm_gem_shmem_is_purgeable(shmem))
>> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>>                          count += shmem->base.size >> PAGE_SHIFT;
>>          }
>>
>> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
>> index b2d93cb12ebf..81bacc7e1873 100644
>> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
>> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
>> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>>          uint32_t hw_res_handle;
>>          bool dumb;
>>          bool created;
>> +       bool detached;
>>          bool host3d_blob, guest_blob;
>>          uint32_t blob_mem, blob_flags;
>>
>> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>>
>>   bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>>
>> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
>> +
>>   int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>>                                 uint32_t *resid);
>>   /* virtgpu_prime.c */
>> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
>> index 9923c7a6885e..929546cad894 100644
>> --- a/include/drm/drm_device.h
>> +++ b/include/drm/drm_device.h
>> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>>   struct drm_vma_offset_manager;
>>   struct drm_vram_mm;
>>   struct drm_fb_helper;
>> +struct drm_gem_shmem_shrinker;
>>
>>   struct inode;
>>
>> @@ -277,6 +278,9 @@ struct drm_device {
>>          /** @vram_mm: VRAM MM memory manager */
>>          struct drm_vram_mm *vram_mm;
>>
>> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
>> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
>> +
>>          /**
>>           * @switch_power_state:
>>           *
>> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
>> index 9a8983ee8abe..62c640678a91 100644
>> --- a/include/drm/drm_gem_shmem_helper.h
>> +++ b/include/drm/drm_gem_shmem_helper.h
>> @@ -6,6 +6,7 @@
>>   #include <linux/fs.h>
>>   #include <linux/mm.h>
>>   #include <linux/mutex.h>
>> +#include <linux/shrinker.h>
>>
>>   #include <drm/drm_file.h>
>>   #include <drm/drm_gem.h>
>> @@ -15,6 +16,7 @@
>>   struct dma_buf_attachment;
>>   struct drm_mode_create_dumb;
>>   struct drm_printer;
>> +struct drm_device;
>>   struct sg_table;
>>
>>   /**
>> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>>           */
>>          unsigned int pages_use_count;
>>
>> +       /**
>> +        * @pages_pin_count:
>> +        *
>> +        * Reference count on the pinned pages table.
>> +        * The pages can be evicted by memory shrinker
>> +        * when the count reaches zero.
>> +        */
>> +       unsigned int pages_pin_count;
>> +
>>          /**
>>           * @madv: State for madvise
>>           *
>>           * 0 is active/inuse.
>> +        * 1 is not-needed/can-be-purged
>>           * A negative value is the object is purged.
>> -        * Positive values are driver specific and not used by the helpers.
>>           */
>>          int madv;
>>
>> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>>           * @map_wc: map object write-combined (instead of using shmem defaults).
>>           */
>>          bool map_wc;
>> +
>> +       /**
>> +        * @eviction_enabled:
>> +        *
>> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
>> +        * Used internally by memory shrinker.
>> +        */
>> +       bool eviction_enabled;
>> +
>> +       /**
>> +        * @purge_enabled:
>> +        *
>> +        * The shmem pages can be purged only if @purge_enabled is set to true.
>> +        * Used internally by memory shrinker.
>> +        */
>> +       bool purge_enabled;
>> +
>> +       /**
>> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
>> +        * Used internally by memory shrinker.
>> +        */
>> +       bool evicted;
>> +
>> +       /**
>> +        * @evict:
>> +        *
>> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
>> +        * GEM's DMA reservation is kept locked by the shrinker. This is
>> +        * optional callback that should be specified by drivers.
>> +        *
>> +        * Returns 0 on success, or -errno on error.
>> +        */
>> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>>   };
>>
>>   #define to_drm_gem_shmem_obj(obj) \
>> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>>
>>   int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>>
>> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
>> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
>> +
>>   static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>>   {
>> -       return (shmem->madv > 0) &&
>> -               !shmem->vmap_use_count && shmem->sgt &&
>> -               !shmem->base.dma_buf && !shmem->base.import_attach;
>> +       return (shmem->madv > 0) && shmem->evict &&
>> +               shmem->purge_enabled && shmem->pages_use_count &&
>> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
>> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>>   }
>>
>> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
>> +
>> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
>> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>>
>>   struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>>   struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
>> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>>          return drm_gem_shmem_mmap(shmem, vma);
>>   }
>>
>> +/**
>> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
>> + */
>> +struct drm_gem_shmem_shrinker {
>> +       /** @base: Shrinker for purging shmem GEM objects */
>> +       struct shrinker base;
>> +
>> +       /** @lock: Protects @lru_* */
>> +       struct mutex lock;
>> +
>> +       /** @lru_pinned: List of pinned shmem GEM objects */
>> +       struct list_head lru_pinned;
>> +
>> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
>> +       struct list_head lru_evictable;
>> +
>> +       /** @lru_evicted: List of evicted shmem GEM objects */
>> +       struct list_head lru_evicted;
>> +
>> +       /** @dev: DRM device that uses this shrinker */
>> +       struct drm_device *dev;
>> +};
>> +
>> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
>> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
>> +
>>   /*
>>    * Driver ops
>>    */
>> --
>> 2.35.3
>>
>

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Christian König]

Am 05.06.22 um 18:47 schrieb Daniel Vetter:
> On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
>> Introduce a common DRM SHMEM shrinker framework that allows to reduce
>> code duplication among DRM drivers by replacing theirs custom shrinker
>> implementations with the generic shrinker.
>>
>> In order to start using DRM SHMEM shrinker drivers should:
>>
>> 1. Implement new evict() shmem object callback.
>> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
>> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>>     activate shrinking of shmem GEMs.
>>
>> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
>> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>>
>> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> So I guess I get a price for being blind since forever, because this
> thing existed since at least 2013. I just stumbled over
> llist_lru.[hc], a purpose built list helper for shrinkers. I think we
> should try to adopt that so that our gpu shrinkers look more like
> shrinkers for everything else.

What the heck are you talking about?

I can't find any llist_lru.[hc] in the linux kernel sources.

Christian.

>
> Apologies for this, since I fear this might cause a bit of churn.
> Hopefully it's all contained to the list manipulation code in shmem
> helpers, I don't think this should leak any further.
> -Daniel
>
>> ---
>>   drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>>   .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>>   drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>>   include/drm/drm_device.h                      |   4 +
>>   include/drm/drm_gem_shmem_helper.h            |  87 ++-
>>   5 files changed, 594 insertions(+), 49 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
>> index 555fe212bd98..4cd0b5913492 100644
>> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
>> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
>> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>>   }
>>   EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>>
>> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       return (shmem->madv >= 0) && shmem->evict &&
>> +               shmem->eviction_enabled && shmem->pages_use_count &&
>> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
>> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
>> +}
>> +
>> +static void
>> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
>> +
>> +       dma_resv_assert_held(shmem->base.resv);
>> +
>> +       if (!gem_shrinker || obj->import_attach)
>> +               return;
>> +
>> +       mutex_lock(&gem_shrinker->lock);
>> +
>> +       if (drm_gem_shmem_is_evictable(shmem) ||
>> +           drm_gem_shmem_is_purgeable(shmem))
>> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
>> +       else if (shmem->madv < 0)
>> +               list_del_init(&shmem->madv_list);
>> +       else if (shmem->evicted)
>> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
>> +       else if (!shmem->pages)
>> +               list_del_init(&shmem->madv_list);
>> +       else
>> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
>> +
>> +       mutex_unlock(&gem_shrinker->lock);
>> +}
>> +
>>   /**
>>    * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>>    * @shmem: shmem GEM object to free
>> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>>          } else {
>>                  dma_resv_lock(shmem->base.resv, NULL);
>>
>> +               /* take out shmem GEM object from the memory shrinker */
>> +               drm_gem_shmem_madvise(shmem, -1);
>> +
>>                  WARN_ON(shmem->vmap_use_count);
>>
>>                  if (shmem->sgt) {
>> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>>                          sg_free_table(shmem->sgt);
>>                          kfree(shmem->sgt);
>>                  }
>> -               if (shmem->pages)
>> +               if (shmem->pages_use_count)
>>                          drm_gem_shmem_put_pages(shmem);
>>
>>                  WARN_ON(shmem->pages_use_count);
>> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>>   }
>>   EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>>
>> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>> +/**
>> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
>> + * @shmem: shmem GEM object
>> + *
>> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
>> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       dma_resv_lock(shmem->base.resv, NULL);
>> +
>> +       if (shmem->madv < 0)
>> +               return -ENOMEM;
>> +
>> +       shmem->eviction_enabled = true;
>> +
>> +       dma_resv_unlock(shmem->base.resv);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
>> +
>> +/**
>> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
>> + * @shmem: shmem GEM object
>> + *
>> + * Tell memory shrinker that this GEM can be purged. Initially purging is
>> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       dma_resv_lock(shmem->base.resv, NULL);
>> +
>> +       if (shmem->madv < 0)
>> +               return -ENOMEM;
>> +
>> +       shmem->purge_enabled = true;
>> +
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       dma_resv_unlock(shmem->base.resv);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
>> +
>> +static int
>> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>>   {
>>          struct drm_gem_object *obj = &shmem->base;
>>          struct page **pages;
>>
>> -       if (shmem->pages_use_count++ > 0)
>> +       dma_resv_assert_held(shmem->base.resv);
>> +
>> +       if (shmem->madv < 0) {
>> +               WARN_ON(shmem->pages);
>> +               return -ENOMEM;
>> +       }
>> +
>> +       if (shmem->pages) {
>> +               WARN_ON(!shmem->evicted);
>>                  return 0;
>> +       }
>> +
>> +       if (WARN_ON(!shmem->pages_use_count))
>> +               return -EINVAL;
>>
>>          pages = drm_gem_get_pages(obj);
>>          if (IS_ERR(pages)) {
>>                  DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
>> -               shmem->pages_use_count = 0;
>>                  return PTR_ERR(pages);
>>          }
>>
>> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>>          return 0;
>>   }
>>
>> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>> +{
>> +       int err;
>> +
>> +       dma_resv_assert_held(shmem->base.resv);
>> +
>> +       if (shmem->madv < 0)
>> +               return -ENOMEM;
>> +
>> +       if (shmem->pages_use_count++ > 0) {
>> +               err = drm_gem_shmem_swap_in(shmem);
>> +               if (err)
>> +                       goto err_zero_use;
>> +
>> +               return 0;
>> +       }
>> +
>> +       err = drm_gem_shmem_acquire_pages(shmem);
>> +       if (err)
>> +               goto err_zero_use;
>> +
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       return 0;
>> +
>> +err_zero_use:
>> +       shmem->pages_use_count = 0;
>> +
>> +       return err;
>> +}
>> +
>> +static void
>> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +
>> +       if (!shmem->pages) {
>> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
>> +               return;
>> +       }
>> +
>> +#ifdef CONFIG_X86
>> +       if (shmem->map_wc)
>> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
>> +#endif
>> +
>> +       drm_gem_put_pages(obj, shmem->pages,
>> +                         shmem->pages_mark_dirty_on_put,
>> +                         shmem->pages_mark_accessed_on_put);
>> +       shmem->pages = NULL;
>> +}
>> +
>>   /*
>>    * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>>    * @shmem: shmem GEM object
>> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>>    */
>>   void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>>   {
>> -       struct drm_gem_object *obj = &shmem->base;
>> -
>>          dma_resv_assert_held(shmem->base.resv);
>>
>>          if (WARN_ON_ONCE(!shmem->pages_use_count))
>> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>>          if (--shmem->pages_use_count > 0)
>>                  return;
>>
>> -#ifdef CONFIG_X86
>> -       if (shmem->map_wc)
>> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
>> -#endif
>> +       drm_gem_shmem_release_pages(shmem);
>>
>> -       drm_gem_put_pages(obj, shmem->pages,
>> -                         shmem->pages_mark_dirty_on_put,
>> -                         shmem->pages_mark_accessed_on_put);
>> -       shmem->pages = NULL;
>> +       drm_gem_shmem_update_pages_state(shmem);
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>>
>> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>>    */
>>   int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>>   {
>> +       int ret;
>> +
>>          dma_resv_assert_held(shmem->base.resv);
>>
>>          WARN_ON(shmem->base.import_attach);
>>
>> -       return drm_gem_shmem_get_pages(shmem);
>> +       ret = drm_gem_shmem_get_pages(shmem);
>> +       if (!ret)
>> +               shmem->pages_pin_count++;
>> +
>> +       return ret;
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_pin);
>>
>> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>>          WARN_ON(shmem->base.import_attach);
>>
>>          drm_gem_shmem_put_pages(shmem);
>> +
>> +       shmem->pages_pin_count--;
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_unpin);
>>
>> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>>                          return 0;
>>                  }
>>
>> -               ret = drm_gem_shmem_get_pages(shmem);
>> +               ret = drm_gem_shmem_pin(shmem);
>>                  if (ret)
>>                          goto err_zero_use;
>>
>> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>>
>>   err_put_pages:
>>          if (!obj->import_attach)
>> -               drm_gem_shmem_put_pages(shmem);
>> +               drm_gem_shmem_unpin(shmem);
>>   err_zero_use:
>>          shmem->vmap_use_count = 0;
>>
>> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>>                          return;
>>
>>                  vunmap(shmem->vaddr);
>> -               drm_gem_shmem_put_pages(shmem);
>> +               drm_gem_shmem_unpin(shmem);
>>          }
>>
>>          shmem->vaddr = NULL;
>> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>>
>>          madv = shmem->madv;
>>
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>>          return (madv >= 0);
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_madvise);
>>
>> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
>> +/**
>> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
>> + *                           hardware access to the memory.
>> + * @shmem: shmem GEM object
>> + *
>> + * This function moves shmem GEM back to memory if it was previously evicted
>> + * by the memory shrinker. The GEM is ready to use on success.
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>>   {
>>          struct drm_gem_object *obj = &shmem->base;
>> -       struct drm_device *dev = obj->dev;
>> +       struct sg_table *sgt;
>> +       int err;
>>
>>          dma_resv_assert_held(shmem->base.resv);
>>
>> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
>> +       if (shmem->evicted) {
>> +               err = drm_gem_shmem_acquire_pages(shmem);
>> +               if (err)
>> +                       return err;
>> +
>> +               sgt = drm_gem_shmem_get_sg_table(shmem);
>> +               if (IS_ERR(sgt))
>> +                       return PTR_ERR(sgt);
>> +
>> +               err = dma_map_sgtable(obj->dev->dev, sgt,
>> +                                     DMA_BIDIRECTIONAL, 0);
>> +               if (err) {
>> +                       sg_free_table(sgt);
>> +                       kfree(sgt);
>> +                       return err;
>> +               }
>>
>> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
>> -       sg_free_table(shmem->sgt);
>> -       kfree(shmem->sgt);
>> -       shmem->sgt = NULL;
>> +               shmem->sgt = sgt;
>> +               shmem->evicted = false;
>>
>> -       drm_gem_shmem_put_pages(shmem);
>> +               drm_gem_shmem_update_pages_state(shmem);
>> +       }
>>
>> -       shmem->madv = -1;
>> +       if (!shmem->pages)
>> +               return -ENOMEM;
>>
>> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
>> -       drm_gem_free_mmap_offset(obj);
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>>
>> -       /* Our goal here is to return as much of the memory as
>> -        * is possible back to the system as we are called from OOM.
>> -        * To do this we must instruct the shmfs to drop all of its
>> -        * backing pages, *now*.
>> -        */
>> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
>> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +       struct drm_device *dev = obj->dev;
>>
>> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
>> +       if (shmem->evicted)
>> +               return;
>> +
>> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
>> +       drm_gem_shmem_release_pages(shmem);
>> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
>> +
>> +       sg_free_table(shmem->sgt);
>> +       kfree(shmem->sgt);
>> +       shmem->sgt = NULL;
>>   }
>> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>>
>>   /**
>>    * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
>> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>>          vm_fault_t ret;
>>          struct page *page;
>>          pgoff_t page_offset;
>> +       bool pages_unpinned;
>> +       int err;
>>
>>          /* We don't use vmf->pgoff since that has the fake offset */
>>          page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>>
>>          dma_resv_lock(shmem->base.resv, NULL);
>>
>> -       if (page_offset >= num_pages ||
>> -           WARN_ON_ONCE(!shmem->pages) ||
>> -           shmem->madv < 0) {
>> +       /* Sanity-check that we have the pages pointer when it should present */
>> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
>> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
>> +
>> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>>                  ret = VM_FAULT_SIGBUS;
>>          } else {
>> +               err = drm_gem_shmem_swap_in(shmem);
>> +               if (err) {
>> +                       ret = VM_FAULT_OOM;
>> +                       goto unlock;
>> +               }
>> +
>>                  page = shmem->pages[page_offset];
>>
>>                  ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>>          }
>>
>> +unlock:
>>          dma_resv_unlock(shmem->base.resv);
>>
>>          return ret;
>> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>>   {
>>          struct drm_gem_object *obj = vma->vm_private_data;
>>          struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
>> -       int ret;
>>
>>          WARN_ON(shmem->base.import_attach);
>>
>>          dma_resv_lock(shmem->base.resv, NULL);
>> -       ret = drm_gem_shmem_get_pages(shmem);
>> -       WARN_ON_ONCE(ret != 0);
>> +
>> +       if (drm_gem_shmem_get_pages(shmem))
>> +               shmem->pages_use_count++;
>> +
>> +       drm_gem_shmem_update_pages_state(shmem);
>>          dma_resv_unlock(shmem->base.resv);
>>
>>          drm_gem_vm_open(vma);
>> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>>   void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>>                                struct drm_printer *p, unsigned int indent)
>>   {
>> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
>> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>>          drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>>
>>          if (shmem->base.import_attach)
>> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>>                  drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>>                                    shmem->vmap_use_count);
>>
>> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>>          drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
>> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_print_info);
>>
>> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>>
>>          shmem->sgt = sgt;
>>
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>>          dma_resv_unlock(shmem->base.resv);
>>
>>          return sgt;
>> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>>   }
>>   EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>>
>> +static struct drm_gem_shmem_shrinker *
>> +to_drm_shrinker(struct shrinker *shrinker)
>> +{
>> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
>> +}
>> +
>> +static unsigned long
>> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
>> +                                    struct shrink_control *sc)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
>> +       struct drm_gem_shmem_object *shmem;
>> +       unsigned long count = 0;
>> +
>> +       if (!mutex_trylock(&gem_shrinker->lock))
>> +               return 0;
>> +
>> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
>> +               count += shmem->base.size;
>> +
>> +               if (count >= SHRINK_EMPTY)
>> +                       break;
>> +       }
>> +
>> +       mutex_unlock(&gem_shrinker->lock);
>> +
>> +       if (count >= SHRINK_EMPTY)
>> +               return SHRINK_EMPTY - 1;
>> +
>> +       return count ?: SHRINK_EMPTY;
>> +}
>> +
>> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
>> +{
>> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
>> +       WARN_ON(shmem->evicted);
>> +
>> +       drm_gem_shmem_unpin_pages(shmem);
>> +
>> +       shmem->evicted = true;
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
>> +
>> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +
>> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
>> +
>> +       drm_gem_shmem_unpin_pages(shmem);
>> +       drm_gem_free_mmap_offset(obj);
>> +
>> +       /* Our goal here is to return as much of the memory as
>> +        * is possible back to the system as we are called from OOM.
>> +        * To do this we must instruct the shmfs to drop all of its
>> +        * backing pages, *now*.
>> +        */
>> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
>> +
>> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
>> +
>> +       shmem->madv = -1;
>> +       shmem->evicted = false;
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
>> +
>> +static unsigned long
>> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
>> +                                       unsigned long nr_to_scan,
>> +                                       bool *lock_contention,
>> +                                       bool evict)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
>> +       struct drm_gem_shmem_object *shmem;
>> +       struct list_head still_in_list;
>> +       struct drm_gem_object *obj;
>> +       unsigned long freed = 0;
>> +       size_t page_count;
>> +       int err;
>> +
>> +       INIT_LIST_HEAD(&still_in_list);
>> +
>> +       mutex_lock(&gem_shrinker->lock);
>> +
>> +       while (freed < nr_to_scan) {
>> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
>> +                                                typeof(*shmem), madv_list);
>> +               if (!shmem)
>> +                       break;
>> +
>> +               obj = &shmem->base;
>> +               page_count = obj->size >> PAGE_SHIFT;
>> +               list_move_tail(&shmem->madv_list, &still_in_list);
>> +
>> +               if (evict) {
>> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
>> +                           get_nr_swap_pages() < page_count)
>> +                               continue;
>> +               } else {
>> +                       if (!drm_gem_shmem_is_purgeable(shmem))
>> +                               continue;
>> +               }
>> +
>> +               /*
>> +                * If it's in the process of being freed, gem_object->free()
>> +                * may be blocked on lock waiting to remove it.  So just
>> +                * skip it.
>> +                */
>> +               if (!kref_get_unless_zero(&obj->refcount))
>> +                       continue;
>> +
>> +               mutex_unlock(&gem_shrinker->lock);
>> +
>> +               /* prevent racing with job-submission code paths */
>> +               if (!dma_resv_trylock(obj->resv)) {
>> +                       *lock_contention |= true;
>> +                       goto shrinker_lock;
>> +               }
>> +
>> +               /* prevent racing with the dma-buf importing/exporting */
>> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
>> +                       *lock_contention |= true;
>> +                       goto resv_unlock;
>> +               }
>> +
>> +               /* check whether h/w uses this object */
>> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
>> +                       goto object_name_unlock;
>> +
>> +               /* re-check whether eviction status hasn't changed */
>> +               if (!drm_gem_shmem_is_evictable(shmem) &&
>> +                   !drm_gem_shmem_is_purgeable(shmem))
>> +                       goto object_name_unlock;
>> +
>> +               err = shmem->evict(shmem);
>> +               if (!err)
>> +                       freed += obj->size >> PAGE_SHIFT;
>> +
>> +object_name_unlock:
>> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
>> +resv_unlock:
>> +               dma_resv_unlock(obj->resv);
>> +shrinker_lock:
>> +               drm_gem_object_put(&shmem->base);
>> +               mutex_lock(&gem_shrinker->lock);
>> +       }
>> +
>> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
>> +
>> +       mutex_unlock(&gem_shrinker->lock);
>> +
>> +       return freed;
>> +}
>> +
>> +static unsigned long
>> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
>> +                                   struct shrink_control *sc)
>> +{
>> +       unsigned long nr_to_scan = sc->nr_to_scan;
>> +       bool lock_contention = false;
>> +       unsigned long freed;
>> +
>> +       /* purge as many objects as we can */
>> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
>> +                                                       &lock_contention, false);
>> +
>> +       /* evict as many objects as we can */
>> +       if (freed < nr_to_scan)
>> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
>> +                                                                nr_to_scan - freed,
>> +                                                                &lock_contention,
>> +                                                                true);
>> +
>> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
>> +}
>> +
>> +/**
>> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
>> + * @dev: DRM device
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker;
>> +       int err;
>> +
>> +       if (WARN_ON(dev->shmem_shrinker))
>> +               return -EBUSY;
>> +
>> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
>> +       if (!gem_shrinker)
>> +               return -ENOMEM;
>> +
>> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
>> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
>> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
>> +       gem_shrinker->dev = dev;
>> +
>> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
>> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
>> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
>> +       mutex_init(&gem_shrinker->lock);
>> +
>> +       dev->shmem_shrinker = gem_shrinker;
>> +
>> +       err = register_shrinker(&gem_shrinker->base);
>> +       if (err) {
>> +               dev->shmem_shrinker = NULL;
>> +               kfree(gem_shrinker);
>> +               return err;
>> +       }
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
>> +
>> +/**
>> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
>> + * @dev: DRM device
>> + */
>> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
>> +
>> +       if (gem_shrinker) {
>> +               unregister_shrinker(&gem_shrinker->base);
>> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
>> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
>> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
>> +               mutex_destroy(&gem_shrinker->lock);
>> +               dev->shmem_shrinker = NULL;
>> +               kfree(gem_shrinker);
>> +       }
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
>> +
>>   MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>>   MODULE_IMPORT_NS(DMA_BUF);
>>   MODULE_LICENSE("GPL v2");
>> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
>> index a4bedfeb2ec4..7cc32556f908 100644
>> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
>> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
>> @@ -15,6 +15,13 @@
>>   #include "panfrost_gem.h"
>>   #include "panfrost_mmu.h"
>>
>> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       return (shmem->madv > 0) &&
>> +               !shmem->pages_pin_count && shmem->sgt &&
>> +               !shmem->base.dma_buf && !shmem->base.import_attach;
>> +}
>> +
>>   static unsigned long
>>   panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>>   {
>> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>>                  return 0;
>>
>>          list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
>> -               if (drm_gem_shmem_is_purgeable(shmem))
>> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>>                          count += shmem->base.size >> PAGE_SHIFT;
>>          }
>>
>> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
>> index b2d93cb12ebf..81bacc7e1873 100644
>> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
>> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
>> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>>          uint32_t hw_res_handle;
>>          bool dumb;
>>          bool created;
>> +       bool detached;
>>          bool host3d_blob, guest_blob;
>>          uint32_t blob_mem, blob_flags;
>>
>> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>>
>>   bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>>
>> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
>> +
>>   int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>>                                 uint32_t *resid);
>>   /* virtgpu_prime.c */
>> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
>> index 9923c7a6885e..929546cad894 100644
>> --- a/include/drm/drm_device.h
>> +++ b/include/drm/drm_device.h
>> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>>   struct drm_vma_offset_manager;
>>   struct drm_vram_mm;
>>   struct drm_fb_helper;
>> +struct drm_gem_shmem_shrinker;
>>
>>   struct inode;
>>
>> @@ -277,6 +278,9 @@ struct drm_device {
>>          /** @vram_mm: VRAM MM memory manager */
>>          struct drm_vram_mm *vram_mm;
>>
>> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
>> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
>> +
>>          /**
>>           * @switch_power_state:
>>           *
>> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
>> index 9a8983ee8abe..62c640678a91 100644
>> --- a/include/drm/drm_gem_shmem_helper.h
>> +++ b/include/drm/drm_gem_shmem_helper.h
>> @@ -6,6 +6,7 @@
>>   #include <linux/fs.h>
>>   #include <linux/mm.h>
>>   #include <linux/mutex.h>
>> +#include <linux/shrinker.h>
>>
>>   #include <drm/drm_file.h>
>>   #include <drm/drm_gem.h>
>> @@ -15,6 +16,7 @@
>>   struct dma_buf_attachment;
>>   struct drm_mode_create_dumb;
>>   struct drm_printer;
>> +struct drm_device;
>>   struct sg_table;
>>
>>   /**
>> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>>           */
>>          unsigned int pages_use_count;
>>
>> +       /**
>> +        * @pages_pin_count:
>> +        *
>> +        * Reference count on the pinned pages table.
>> +        * The pages can be evicted by memory shrinker
>> +        * when the count reaches zero.
>> +        */
>> +       unsigned int pages_pin_count;
>> +
>>          /**
>>           * @madv: State for madvise
>>           *
>>           * 0 is active/inuse.
>> +        * 1 is not-needed/can-be-purged
>>           * A negative value is the object is purged.
>> -        * Positive values are driver specific and not used by the helpers.
>>           */
>>          int madv;
>>
>> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>>           * @map_wc: map object write-combined (instead of using shmem defaults).
>>           */
>>          bool map_wc;
>> +
>> +       /**
>> +        * @eviction_enabled:
>> +        *
>> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
>> +        * Used internally by memory shrinker.
>> +        */
>> +       bool eviction_enabled;
>> +
>> +       /**
>> +        * @purge_enabled:
>> +        *
>> +        * The shmem pages can be purged only if @purge_enabled is set to true.
>> +        * Used internally by memory shrinker.
>> +        */
>> +       bool purge_enabled;
>> +
>> +       /**
>> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
>> +        * Used internally by memory shrinker.
>> +        */
>> +       bool evicted;
>> +
>> +       /**
>> +        * @evict:
>> +        *
>> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
>> +        * GEM's DMA reservation is kept locked by the shrinker. This is
>> +        * optional callback that should be specified by drivers.
>> +        *
>> +        * Returns 0 on success, or -errno on error.
>> +        */
>> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>>   };
>>
>>   #define to_drm_gem_shmem_obj(obj) \
>> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>>
>>   int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>>
>> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
>> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
>> +
>>   static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>>   {
>> -       return (shmem->madv > 0) &&
>> -               !shmem->vmap_use_count && shmem->sgt &&
>> -               !shmem->base.dma_buf && !shmem->base.import_attach;
>> +       return (shmem->madv > 0) && shmem->evict &&
>> +               shmem->purge_enabled && shmem->pages_use_count &&
>> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
>> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>>   }
>>
>> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
>> +
>> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
>> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>>
>>   struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>>   struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
>> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>>          return drm_gem_shmem_mmap(shmem, vma);
>>   }
>>
>> +/**
>> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
>> + */
>> +struct drm_gem_shmem_shrinker {
>> +       /** @base: Shrinker for purging shmem GEM objects */
>> +       struct shrinker base;
>> +
>> +       /** @lock: Protects @lru_* */
>> +       struct mutex lock;
>> +
>> +       /** @lru_pinned: List of pinned shmem GEM objects */
>> +       struct list_head lru_pinned;
>> +
>> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
>> +       struct list_head lru_evictable;
>> +
>> +       /** @lru_evicted: List of evicted shmem GEM objects */
>> +       struct list_head lru_evicted;
>> +
>> +       /** @dev: DRM device that uses this shrinker */
>> +       struct drm_device *dev;
>> +};
>> +
>> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
>> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
>> +
>>   /*
>>    * Driver ops
>>    */
>> --
>> 2.35.3
>>
>


_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Christian König]

Am 05.06.22 um 18:47 schrieb Daniel Vetter:
> On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
>> Introduce a common DRM SHMEM shrinker framework that allows to reduce
>> code duplication among DRM drivers by replacing theirs custom shrinker
>> implementations with the generic shrinker.
>>
>> In order to start using DRM SHMEM shrinker drivers should:
>>
>> 1. Implement new evict() shmem object callback.
>> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
>> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>>     activate shrinking of shmem GEMs.
>>
>> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
>> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>>
>> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> So I guess I get a price for being blind since forever, because this
> thing existed since at least 2013. I just stumbled over
> llist_lru.[hc], a purpose built list helper for shrinkers. I think we
> should try to adopt that so that our gpu shrinkers look more like
> shrinkers for everything else.

What the heck are you talking about?

I can't find any llist_lru.[hc] in the linux kernel sources.

Christian.

>
> Apologies for this, since I fear this might cause a bit of churn.
> Hopefully it's all contained to the list manipulation code in shmem
> helpers, I don't think this should leak any further.
> -Daniel
>
>> ---
>>   drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>>   .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>>   drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>>   include/drm/drm_device.h                      |   4 +
>>   include/drm/drm_gem_shmem_helper.h            |  87 ++-
>>   5 files changed, 594 insertions(+), 49 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
>> index 555fe212bd98..4cd0b5913492 100644
>> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
>> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
>> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>>   }
>>   EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>>
>> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       return (shmem->madv >= 0) && shmem->evict &&
>> +               shmem->eviction_enabled && shmem->pages_use_count &&
>> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
>> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
>> +}
>> +
>> +static void
>> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
>> +
>> +       dma_resv_assert_held(shmem->base.resv);
>> +
>> +       if (!gem_shrinker || obj->import_attach)
>> +               return;
>> +
>> +       mutex_lock(&gem_shrinker->lock);
>> +
>> +       if (drm_gem_shmem_is_evictable(shmem) ||
>> +           drm_gem_shmem_is_purgeable(shmem))
>> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
>> +       else if (shmem->madv < 0)
>> +               list_del_init(&shmem->madv_list);
>> +       else if (shmem->evicted)
>> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
>> +       else if (!shmem->pages)
>> +               list_del_init(&shmem->madv_list);
>> +       else
>> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
>> +
>> +       mutex_unlock(&gem_shrinker->lock);
>> +}
>> +
>>   /**
>>    * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>>    * @shmem: shmem GEM object to free
>> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>>          } else {
>>                  dma_resv_lock(shmem->base.resv, NULL);
>>
>> +               /* take out shmem GEM object from the memory shrinker */
>> +               drm_gem_shmem_madvise(shmem, -1);
>> +
>>                  WARN_ON(shmem->vmap_use_count);
>>
>>                  if (shmem->sgt) {
>> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>>                          sg_free_table(shmem->sgt);
>>                          kfree(shmem->sgt);
>>                  }
>> -               if (shmem->pages)
>> +               if (shmem->pages_use_count)
>>                          drm_gem_shmem_put_pages(shmem);
>>
>>                  WARN_ON(shmem->pages_use_count);
>> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>>   }
>>   EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>>
>> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>> +/**
>> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
>> + * @shmem: shmem GEM object
>> + *
>> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
>> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       dma_resv_lock(shmem->base.resv, NULL);
>> +
>> +       if (shmem->madv < 0)
>> +               return -ENOMEM;
>> +
>> +       shmem->eviction_enabled = true;
>> +
>> +       dma_resv_unlock(shmem->base.resv);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
>> +
>> +/**
>> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
>> + * @shmem: shmem GEM object
>> + *
>> + * Tell memory shrinker that this GEM can be purged. Initially purging is
>> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       dma_resv_lock(shmem->base.resv, NULL);
>> +
>> +       if (shmem->madv < 0)
>> +               return -ENOMEM;
>> +
>> +       shmem->purge_enabled = true;
>> +
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       dma_resv_unlock(shmem->base.resv);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
>> +
>> +static int
>> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>>   {
>>          struct drm_gem_object *obj = &shmem->base;
>>          struct page **pages;
>>
>> -       if (shmem->pages_use_count++ > 0)
>> +       dma_resv_assert_held(shmem->base.resv);
>> +
>> +       if (shmem->madv < 0) {
>> +               WARN_ON(shmem->pages);
>> +               return -ENOMEM;
>> +       }
>> +
>> +       if (shmem->pages) {
>> +               WARN_ON(!shmem->evicted);
>>                  return 0;
>> +       }
>> +
>> +       if (WARN_ON(!shmem->pages_use_count))
>> +               return -EINVAL;
>>
>>          pages = drm_gem_get_pages(obj);
>>          if (IS_ERR(pages)) {
>>                  DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
>> -               shmem->pages_use_count = 0;
>>                  return PTR_ERR(pages);
>>          }
>>
>> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>>          return 0;
>>   }
>>
>> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>> +{
>> +       int err;
>> +
>> +       dma_resv_assert_held(shmem->base.resv);
>> +
>> +       if (shmem->madv < 0)
>> +               return -ENOMEM;
>> +
>> +       if (shmem->pages_use_count++ > 0) {
>> +               err = drm_gem_shmem_swap_in(shmem);
>> +               if (err)
>> +                       goto err_zero_use;
>> +
>> +               return 0;
>> +       }
>> +
>> +       err = drm_gem_shmem_acquire_pages(shmem);
>> +       if (err)
>> +               goto err_zero_use;
>> +
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       return 0;
>> +
>> +err_zero_use:
>> +       shmem->pages_use_count = 0;
>> +
>> +       return err;
>> +}
>> +
>> +static void
>> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +
>> +       if (!shmem->pages) {
>> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
>> +               return;
>> +       }
>> +
>> +#ifdef CONFIG_X86
>> +       if (shmem->map_wc)
>> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
>> +#endif
>> +
>> +       drm_gem_put_pages(obj, shmem->pages,
>> +                         shmem->pages_mark_dirty_on_put,
>> +                         shmem->pages_mark_accessed_on_put);
>> +       shmem->pages = NULL;
>> +}
>> +
>>   /*
>>    * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>>    * @shmem: shmem GEM object
>> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>>    */
>>   void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>>   {
>> -       struct drm_gem_object *obj = &shmem->base;
>> -
>>          dma_resv_assert_held(shmem->base.resv);
>>
>>          if (WARN_ON_ONCE(!shmem->pages_use_count))
>> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>>          if (--shmem->pages_use_count > 0)
>>                  return;
>>
>> -#ifdef CONFIG_X86
>> -       if (shmem->map_wc)
>> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
>> -#endif
>> +       drm_gem_shmem_release_pages(shmem);
>>
>> -       drm_gem_put_pages(obj, shmem->pages,
>> -                         shmem->pages_mark_dirty_on_put,
>> -                         shmem->pages_mark_accessed_on_put);
>> -       shmem->pages = NULL;
>> +       drm_gem_shmem_update_pages_state(shmem);
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>>
>> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>>    */
>>   int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>>   {
>> +       int ret;
>> +
>>          dma_resv_assert_held(shmem->base.resv);
>>
>>          WARN_ON(shmem->base.import_attach);
>>
>> -       return drm_gem_shmem_get_pages(shmem);
>> +       ret = drm_gem_shmem_get_pages(shmem);
>> +       if (!ret)
>> +               shmem->pages_pin_count++;
>> +
>> +       return ret;
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_pin);
>>
>> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>>          WARN_ON(shmem->base.import_attach);
>>
>>          drm_gem_shmem_put_pages(shmem);
>> +
>> +       shmem->pages_pin_count--;
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_unpin);
>>
>> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>>                          return 0;
>>                  }
>>
>> -               ret = drm_gem_shmem_get_pages(shmem);
>> +               ret = drm_gem_shmem_pin(shmem);
>>                  if (ret)
>>                          goto err_zero_use;
>>
>> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>>
>>   err_put_pages:
>>          if (!obj->import_attach)
>> -               drm_gem_shmem_put_pages(shmem);
>> +               drm_gem_shmem_unpin(shmem);
>>   err_zero_use:
>>          shmem->vmap_use_count = 0;
>>
>> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>>                          return;
>>
>>                  vunmap(shmem->vaddr);
>> -               drm_gem_shmem_put_pages(shmem);
>> +               drm_gem_shmem_unpin(shmem);
>>          }
>>
>>          shmem->vaddr = NULL;
>> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>>
>>          madv = shmem->madv;
>>
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>>          return (madv >= 0);
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_madvise);
>>
>> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
>> +/**
>> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
>> + *                           hardware access to the memory.
>> + * @shmem: shmem GEM object
>> + *
>> + * This function moves shmem GEM back to memory if it was previously evicted
>> + * by the memory shrinker. The GEM is ready to use on success.
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>>   {
>>          struct drm_gem_object *obj = &shmem->base;
>> -       struct drm_device *dev = obj->dev;
>> +       struct sg_table *sgt;
>> +       int err;
>>
>>          dma_resv_assert_held(shmem->base.resv);
>>
>> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
>> +       if (shmem->evicted) {
>> +               err = drm_gem_shmem_acquire_pages(shmem);
>> +               if (err)
>> +                       return err;
>> +
>> +               sgt = drm_gem_shmem_get_sg_table(shmem);
>> +               if (IS_ERR(sgt))
>> +                       return PTR_ERR(sgt);
>> +
>> +               err = dma_map_sgtable(obj->dev->dev, sgt,
>> +                                     DMA_BIDIRECTIONAL, 0);
>> +               if (err) {
>> +                       sg_free_table(sgt);
>> +                       kfree(sgt);
>> +                       return err;
>> +               }
>>
>> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
>> -       sg_free_table(shmem->sgt);
>> -       kfree(shmem->sgt);
>> -       shmem->sgt = NULL;
>> +               shmem->sgt = sgt;
>> +               shmem->evicted = false;
>>
>> -       drm_gem_shmem_put_pages(shmem);
>> +               drm_gem_shmem_update_pages_state(shmem);
>> +       }
>>
>> -       shmem->madv = -1;
>> +       if (!shmem->pages)
>> +               return -ENOMEM;
>>
>> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
>> -       drm_gem_free_mmap_offset(obj);
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>>
>> -       /* Our goal here is to return as much of the memory as
>> -        * is possible back to the system as we are called from OOM.
>> -        * To do this we must instruct the shmfs to drop all of its
>> -        * backing pages, *now*.
>> -        */
>> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
>> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +       struct drm_device *dev = obj->dev;
>>
>> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
>> +       if (shmem->evicted)
>> +               return;
>> +
>> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
>> +       drm_gem_shmem_release_pages(shmem);
>> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
>> +
>> +       sg_free_table(shmem->sgt);
>> +       kfree(shmem->sgt);
>> +       shmem->sgt = NULL;
>>   }
>> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>>
>>   /**
>>    * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
>> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>>          vm_fault_t ret;
>>          struct page *page;
>>          pgoff_t page_offset;
>> +       bool pages_unpinned;
>> +       int err;
>>
>>          /* We don't use vmf->pgoff since that has the fake offset */
>>          page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>>
>>          dma_resv_lock(shmem->base.resv, NULL);
>>
>> -       if (page_offset >= num_pages ||
>> -           WARN_ON_ONCE(!shmem->pages) ||
>> -           shmem->madv < 0) {
>> +       /* Sanity-check that we have the pages pointer when it should present */
>> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
>> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
>> +
>> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>>                  ret = VM_FAULT_SIGBUS;
>>          } else {
>> +               err = drm_gem_shmem_swap_in(shmem);
>> +               if (err) {
>> +                       ret = VM_FAULT_OOM;
>> +                       goto unlock;
>> +               }
>> +
>>                  page = shmem->pages[page_offset];
>>
>>                  ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>>          }
>>
>> +unlock:
>>          dma_resv_unlock(shmem->base.resv);
>>
>>          return ret;
>> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>>   {
>>          struct drm_gem_object *obj = vma->vm_private_data;
>>          struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
>> -       int ret;
>>
>>          WARN_ON(shmem->base.import_attach);
>>
>>          dma_resv_lock(shmem->base.resv, NULL);
>> -       ret = drm_gem_shmem_get_pages(shmem);
>> -       WARN_ON_ONCE(ret != 0);
>> +
>> +       if (drm_gem_shmem_get_pages(shmem))
>> +               shmem->pages_use_count++;
>> +
>> +       drm_gem_shmem_update_pages_state(shmem);
>>          dma_resv_unlock(shmem->base.resv);
>>
>>          drm_gem_vm_open(vma);
>> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>>   void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>>                                struct drm_printer *p, unsigned int indent)
>>   {
>> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
>> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>>          drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>>
>>          if (shmem->base.import_attach)
>> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>>                  drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>>                                    shmem->vmap_use_count);
>>
>> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>>          drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
>> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_print_info);
>>
>> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>>
>>          shmem->sgt = sgt;
>>
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>>          dma_resv_unlock(shmem->base.resv);
>>
>>          return sgt;
>> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>>   }
>>   EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>>
>> +static struct drm_gem_shmem_shrinker *
>> +to_drm_shrinker(struct shrinker *shrinker)
>> +{
>> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
>> +}
>> +
>> +static unsigned long
>> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
>> +                                    struct shrink_control *sc)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
>> +       struct drm_gem_shmem_object *shmem;
>> +       unsigned long count = 0;
>> +
>> +       if (!mutex_trylock(&gem_shrinker->lock))
>> +               return 0;
>> +
>> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
>> +               count += shmem->base.size;
>> +
>> +               if (count >= SHRINK_EMPTY)
>> +                       break;
>> +       }
>> +
>> +       mutex_unlock(&gem_shrinker->lock);
>> +
>> +       if (count >= SHRINK_EMPTY)
>> +               return SHRINK_EMPTY - 1;
>> +
>> +       return count ?: SHRINK_EMPTY;
>> +}
>> +
>> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
>> +{
>> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
>> +       WARN_ON(shmem->evicted);
>> +
>> +       drm_gem_shmem_unpin_pages(shmem);
>> +
>> +       shmem->evicted = true;
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
>> +
>> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +
>> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
>> +
>> +       drm_gem_shmem_unpin_pages(shmem);
>> +       drm_gem_free_mmap_offset(obj);
>> +
>> +       /* Our goal here is to return as much of the memory as
>> +        * is possible back to the system as we are called from OOM.
>> +        * To do this we must instruct the shmfs to drop all of its
>> +        * backing pages, *now*.
>> +        */
>> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
>> +
>> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
>> +
>> +       shmem->madv = -1;
>> +       shmem->evicted = false;
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
>> +
>> +static unsigned long
>> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
>> +                                       unsigned long nr_to_scan,
>> +                                       bool *lock_contention,
>> +                                       bool evict)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
>> +       struct drm_gem_shmem_object *shmem;
>> +       struct list_head still_in_list;
>> +       struct drm_gem_object *obj;
>> +       unsigned long freed = 0;
>> +       size_t page_count;
>> +       int err;
>> +
>> +       INIT_LIST_HEAD(&still_in_list);
>> +
>> +       mutex_lock(&gem_shrinker->lock);
>> +
>> +       while (freed < nr_to_scan) {
>> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
>> +                                                typeof(*shmem), madv_list);
>> +               if (!shmem)
>> +                       break;
>> +
>> +               obj = &shmem->base;
>> +               page_count = obj->size >> PAGE_SHIFT;
>> +               list_move_tail(&shmem->madv_list, &still_in_list);
>> +
>> +               if (evict) {
>> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
>> +                           get_nr_swap_pages() < page_count)
>> +                               continue;
>> +               } else {
>> +                       if (!drm_gem_shmem_is_purgeable(shmem))
>> +                               continue;
>> +               }
>> +
>> +               /*
>> +                * If it's in the process of being freed, gem_object->free()
>> +                * may be blocked on lock waiting to remove it.  So just
>> +                * skip it.
>> +                */
>> +               if (!kref_get_unless_zero(&obj->refcount))
>> +                       continue;
>> +
>> +               mutex_unlock(&gem_shrinker->lock);
>> +
>> +               /* prevent racing with job-submission code paths */
>> +               if (!dma_resv_trylock(obj->resv)) {
>> +                       *lock_contention |= true;
>> +                       goto shrinker_lock;
>> +               }
>> +
>> +               /* prevent racing with the dma-buf importing/exporting */
>> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
>> +                       *lock_contention |= true;
>> +                       goto resv_unlock;
>> +               }
>> +
>> +               /* check whether h/w uses this object */
>> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
>> +                       goto object_name_unlock;
>> +
>> +               /* re-check whether eviction status hasn't changed */
>> +               if (!drm_gem_shmem_is_evictable(shmem) &&
>> +                   !drm_gem_shmem_is_purgeable(shmem))
>> +                       goto object_name_unlock;
>> +
>> +               err = shmem->evict(shmem);
>> +               if (!err)
>> +                       freed += obj->size >> PAGE_SHIFT;
>> +
>> +object_name_unlock:
>> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
>> +resv_unlock:
>> +               dma_resv_unlock(obj->resv);
>> +shrinker_lock:
>> +               drm_gem_object_put(&shmem->base);
>> +               mutex_lock(&gem_shrinker->lock);
>> +       }
>> +
>> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
>> +
>> +       mutex_unlock(&gem_shrinker->lock);
>> +
>> +       return freed;
>> +}
>> +
>> +static unsigned long
>> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
>> +                                   struct shrink_control *sc)
>> +{
>> +       unsigned long nr_to_scan = sc->nr_to_scan;
>> +       bool lock_contention = false;
>> +       unsigned long freed;
>> +
>> +       /* purge as many objects as we can */
>> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
>> +                                                       &lock_contention, false);
>> +
>> +       /* evict as many objects as we can */
>> +       if (freed < nr_to_scan)
>> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
>> +                                                                nr_to_scan - freed,
>> +                                                                &lock_contention,
>> +                                                                true);
>> +
>> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
>> +}
>> +
>> +/**
>> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
>> + * @dev: DRM device
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker;
>> +       int err;
>> +
>> +       if (WARN_ON(dev->shmem_shrinker))
>> +               return -EBUSY;
>> +
>> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
>> +       if (!gem_shrinker)
>> +               return -ENOMEM;
>> +
>> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
>> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
>> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
>> +       gem_shrinker->dev = dev;
>> +
>> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
>> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
>> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
>> +       mutex_init(&gem_shrinker->lock);
>> +
>> +       dev->shmem_shrinker = gem_shrinker;
>> +
>> +       err = register_shrinker(&gem_shrinker->base);
>> +       if (err) {
>> +               dev->shmem_shrinker = NULL;
>> +               kfree(gem_shrinker);
>> +               return err;
>> +       }
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
>> +
>> +/**
>> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
>> + * @dev: DRM device
>> + */
>> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
>> +
>> +       if (gem_shrinker) {
>> +               unregister_shrinker(&gem_shrinker->base);
>> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
>> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
>> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
>> +               mutex_destroy(&gem_shrinker->lock);
>> +               dev->shmem_shrinker = NULL;
>> +               kfree(gem_shrinker);
>> +       }
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
>> +
>>   MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>>   MODULE_IMPORT_NS(DMA_BUF);
>>   MODULE_LICENSE("GPL v2");
>> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
>> index a4bedfeb2ec4..7cc32556f908 100644
>> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
>> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
>> @@ -15,6 +15,13 @@
>>   #include "panfrost_gem.h"
>>   #include "panfrost_mmu.h"
>>
>> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       return (shmem->madv > 0) &&
>> +               !shmem->pages_pin_count && shmem->sgt &&
>> +               !shmem->base.dma_buf && !shmem->base.import_attach;
>> +}
>> +
>>   static unsigned long
>>   panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>>   {
>> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>>                  return 0;
>>
>>          list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
>> -               if (drm_gem_shmem_is_purgeable(shmem))
>> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>>                          count += shmem->base.size >> PAGE_SHIFT;
>>          }
>>
>> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
>> index b2d93cb12ebf..81bacc7e1873 100644
>> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
>> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
>> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>>          uint32_t hw_res_handle;
>>          bool dumb;
>>          bool created;
>> +       bool detached;
>>          bool host3d_blob, guest_blob;
>>          uint32_t blob_mem, blob_flags;
>>
>> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>>
>>   bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>>
>> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
>> +
>>   int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>>                                 uint32_t *resid);
>>   /* virtgpu_prime.c */
>> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
>> index 9923c7a6885e..929546cad894 100644
>> --- a/include/drm/drm_device.h
>> +++ b/include/drm/drm_device.h
>> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>>   struct drm_vma_offset_manager;
>>   struct drm_vram_mm;
>>   struct drm_fb_helper;
>> +struct drm_gem_shmem_shrinker;
>>
>>   struct inode;
>>
>> @@ -277,6 +278,9 @@ struct drm_device {
>>          /** @vram_mm: VRAM MM memory manager */
>>          struct drm_vram_mm *vram_mm;
>>
>> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
>> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
>> +
>>          /**
>>           * @switch_power_state:
>>           *
>> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
>> index 9a8983ee8abe..62c640678a91 100644
>> --- a/include/drm/drm_gem_shmem_helper.h
>> +++ b/include/drm/drm_gem_shmem_helper.h
>> @@ -6,6 +6,7 @@
>>   #include <linux/fs.h>
>>   #include <linux/mm.h>
>>   #include <linux/mutex.h>
>> +#include <linux/shrinker.h>
>>
>>   #include <drm/drm_file.h>
>>   #include <drm/drm_gem.h>
>> @@ -15,6 +16,7 @@
>>   struct dma_buf_attachment;
>>   struct drm_mode_create_dumb;
>>   struct drm_printer;
>> +struct drm_device;
>>   struct sg_table;
>>
>>   /**
>> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>>           */
>>          unsigned int pages_use_count;
>>
>> +       /**
>> +        * @pages_pin_count:
>> +        *
>> +        * Reference count on the pinned pages table.
>> +        * The pages can be evicted by memory shrinker
>> +        * when the count reaches zero.
>> +        */
>> +       unsigned int pages_pin_count;
>> +
>>          /**
>>           * @madv: State for madvise
>>           *
>>           * 0 is active/inuse.
>> +        * 1 is not-needed/can-be-purged
>>           * A negative value is the object is purged.
>> -        * Positive values are driver specific and not used by the helpers.
>>           */
>>          int madv;
>>
>> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>>           * @map_wc: map object write-combined (instead of using shmem defaults).
>>           */
>>          bool map_wc;
>> +
>> +       /**
>> +        * @eviction_enabled:
>> +        *
>> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
>> +        * Used internally by memory shrinker.
>> +        */
>> +       bool eviction_enabled;
>> +
>> +       /**
>> +        * @purge_enabled:
>> +        *
>> +        * The shmem pages can be purged only if @purge_enabled is set to true.
>> +        * Used internally by memory shrinker.
>> +        */
>> +       bool purge_enabled;
>> +
>> +       /**
>> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
>> +        * Used internally by memory shrinker.
>> +        */
>> +       bool evicted;
>> +
>> +       /**
>> +        * @evict:
>> +        *
>> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
>> +        * GEM's DMA reservation is kept locked by the shrinker. This is
>> +        * optional callback that should be specified by drivers.
>> +        *
>> +        * Returns 0 on success, or -errno on error.
>> +        */
>> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>>   };
>>
>>   #define to_drm_gem_shmem_obj(obj) \
>> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>>
>>   int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>>
>> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
>> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
>> +
>>   static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>>   {
>> -       return (shmem->madv > 0) &&
>> -               !shmem->vmap_use_count && shmem->sgt &&
>> -               !shmem->base.dma_buf && !shmem->base.import_attach;
>> +       return (shmem->madv > 0) && shmem->evict &&
>> +               shmem->purge_enabled && shmem->pages_use_count &&
>> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
>> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>>   }
>>
>> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
>> +
>> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
>> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>>
>>   struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>>   struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
>> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>>          return drm_gem_shmem_mmap(shmem, vma);
>>   }
>>
>> +/**
>> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
>> + */
>> +struct drm_gem_shmem_shrinker {
>> +       /** @base: Shrinker for purging shmem GEM objects */
>> +       struct shrinker base;
>> +
>> +       /** @lock: Protects @lru_* */
>> +       struct mutex lock;
>> +
>> +       /** @lru_pinned: List of pinned shmem GEM objects */
>> +       struct list_head lru_pinned;
>> +
>> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
>> +       struct list_head lru_evictable;
>> +
>> +       /** @lru_evicted: List of evicted shmem GEM objects */
>> +       struct list_head lru_evicted;
>> +
>> +       /** @dev: DRM device that uses this shrinker */
>> +       struct drm_device *dev;
>> +};
>> +
>> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
>> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
>> +
>>   /*
>>    * Driver ops
>>    */
>> --
>> 2.35.3
>>
>

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Christian König]

Am 05.06.22 um 18:47 schrieb Daniel Vetter:
> On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
>> Introduce a common DRM SHMEM shrinker framework that allows to reduce
>> code duplication among DRM drivers by replacing theirs custom shrinker
>> implementations with the generic shrinker.
>>
>> In order to start using DRM SHMEM shrinker drivers should:
>>
>> 1. Implement new evict() shmem object callback.
>> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
>> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>>     activate shrinking of shmem GEMs.
>>
>> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
>> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>>
>> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> So I guess I get a price for being blind since forever, because this
> thing existed since at least 2013. I just stumbled over
> llist_lru.[hc], a purpose built list helper for shrinkers. I think we
> should try to adopt that so that our gpu shrinkers look more like
> shrinkers for everything else.

What the heck are you talking about?

I can't find any llist_lru.[hc] in the linux kernel sources.

Christian.

>
> Apologies for this, since I fear this might cause a bit of churn.
> Hopefully it's all contained to the list manipulation code in shmem
> helpers, I don't think this should leak any further.
> -Daniel
>
>> ---
>>   drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>>   .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>>   drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>>   include/drm/drm_device.h                      |   4 +
>>   include/drm/drm_gem_shmem_helper.h            |  87 ++-
>>   5 files changed, 594 insertions(+), 49 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
>> index 555fe212bd98..4cd0b5913492 100644
>> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
>> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
>> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>>   }
>>   EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>>
>> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       return (shmem->madv >= 0) && shmem->evict &&
>> +               shmem->eviction_enabled && shmem->pages_use_count &&
>> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
>> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
>> +}
>> +
>> +static void
>> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
>> +
>> +       dma_resv_assert_held(shmem->base.resv);
>> +
>> +       if (!gem_shrinker || obj->import_attach)
>> +               return;
>> +
>> +       mutex_lock(&gem_shrinker->lock);
>> +
>> +       if (drm_gem_shmem_is_evictable(shmem) ||
>> +           drm_gem_shmem_is_purgeable(shmem))
>> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
>> +       else if (shmem->madv < 0)
>> +               list_del_init(&shmem->madv_list);
>> +       else if (shmem->evicted)
>> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
>> +       else if (!shmem->pages)
>> +               list_del_init(&shmem->madv_list);
>> +       else
>> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
>> +
>> +       mutex_unlock(&gem_shrinker->lock);
>> +}
>> +
>>   /**
>>    * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>>    * @shmem: shmem GEM object to free
>> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>>          } else {
>>                  dma_resv_lock(shmem->base.resv, NULL);
>>
>> +               /* take out shmem GEM object from the memory shrinker */
>> +               drm_gem_shmem_madvise(shmem, -1);
>> +
>>                  WARN_ON(shmem->vmap_use_count);
>>
>>                  if (shmem->sgt) {
>> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>>                          sg_free_table(shmem->sgt);
>>                          kfree(shmem->sgt);
>>                  }
>> -               if (shmem->pages)
>> +               if (shmem->pages_use_count)
>>                          drm_gem_shmem_put_pages(shmem);
>>
>>                  WARN_ON(shmem->pages_use_count);
>> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>>   }
>>   EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>>
>> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>> +/**
>> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
>> + * @shmem: shmem GEM object
>> + *
>> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
>> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       dma_resv_lock(shmem->base.resv, NULL);
>> +
>> +       if (shmem->madv < 0)
>> +               return -ENOMEM;
>> +
>> +       shmem->eviction_enabled = true;
>> +
>> +       dma_resv_unlock(shmem->base.resv);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
>> +
>> +/**
>> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
>> + * @shmem: shmem GEM object
>> + *
>> + * Tell memory shrinker that this GEM can be purged. Initially purging is
>> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       dma_resv_lock(shmem->base.resv, NULL);
>> +
>> +       if (shmem->madv < 0)
>> +               return -ENOMEM;
>> +
>> +       shmem->purge_enabled = true;
>> +
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       dma_resv_unlock(shmem->base.resv);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
>> +
>> +static int
>> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>>   {
>>          struct drm_gem_object *obj = &shmem->base;
>>          struct page **pages;
>>
>> -       if (shmem->pages_use_count++ > 0)
>> +       dma_resv_assert_held(shmem->base.resv);
>> +
>> +       if (shmem->madv < 0) {
>> +               WARN_ON(shmem->pages);
>> +               return -ENOMEM;
>> +       }
>> +
>> +       if (shmem->pages) {
>> +               WARN_ON(!shmem->evicted);
>>                  return 0;
>> +       }
>> +
>> +       if (WARN_ON(!shmem->pages_use_count))
>> +               return -EINVAL;
>>
>>          pages = drm_gem_get_pages(obj);
>>          if (IS_ERR(pages)) {
>>                  DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
>> -               shmem->pages_use_count = 0;
>>                  return PTR_ERR(pages);
>>          }
>>
>> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>>          return 0;
>>   }
>>
>> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>> +{
>> +       int err;
>> +
>> +       dma_resv_assert_held(shmem->base.resv);
>> +
>> +       if (shmem->madv < 0)
>> +               return -ENOMEM;
>> +
>> +       if (shmem->pages_use_count++ > 0) {
>> +               err = drm_gem_shmem_swap_in(shmem);
>> +               if (err)
>> +                       goto err_zero_use;
>> +
>> +               return 0;
>> +       }
>> +
>> +       err = drm_gem_shmem_acquire_pages(shmem);
>> +       if (err)
>> +               goto err_zero_use;
>> +
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       return 0;
>> +
>> +err_zero_use:
>> +       shmem->pages_use_count = 0;
>> +
>> +       return err;
>> +}
>> +
>> +static void
>> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +
>> +       if (!shmem->pages) {
>> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
>> +               return;
>> +       }
>> +
>> +#ifdef CONFIG_X86
>> +       if (shmem->map_wc)
>> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
>> +#endif
>> +
>> +       drm_gem_put_pages(obj, shmem->pages,
>> +                         shmem->pages_mark_dirty_on_put,
>> +                         shmem->pages_mark_accessed_on_put);
>> +       shmem->pages = NULL;
>> +}
>> +
>>   /*
>>    * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>>    * @shmem: shmem GEM object
>> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>>    */
>>   void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>>   {
>> -       struct drm_gem_object *obj = &shmem->base;
>> -
>>          dma_resv_assert_held(shmem->base.resv);
>>
>>          if (WARN_ON_ONCE(!shmem->pages_use_count))
>> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>>          if (--shmem->pages_use_count > 0)
>>                  return;
>>
>> -#ifdef CONFIG_X86
>> -       if (shmem->map_wc)
>> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
>> -#endif
>> +       drm_gem_shmem_release_pages(shmem);
>>
>> -       drm_gem_put_pages(obj, shmem->pages,
>> -                         shmem->pages_mark_dirty_on_put,
>> -                         shmem->pages_mark_accessed_on_put);
>> -       shmem->pages = NULL;
>> +       drm_gem_shmem_update_pages_state(shmem);
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>>
>> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>>    */
>>   int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>>   {
>> +       int ret;
>> +
>>          dma_resv_assert_held(shmem->base.resv);
>>
>>          WARN_ON(shmem->base.import_attach);
>>
>> -       return drm_gem_shmem_get_pages(shmem);
>> +       ret = drm_gem_shmem_get_pages(shmem);
>> +       if (!ret)
>> +               shmem->pages_pin_count++;
>> +
>> +       return ret;
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_pin);
>>
>> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>>          WARN_ON(shmem->base.import_attach);
>>
>>          drm_gem_shmem_put_pages(shmem);
>> +
>> +       shmem->pages_pin_count--;
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_unpin);
>>
>> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>>                          return 0;
>>                  }
>>
>> -               ret = drm_gem_shmem_get_pages(shmem);
>> +               ret = drm_gem_shmem_pin(shmem);
>>                  if (ret)
>>                          goto err_zero_use;
>>
>> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>>
>>   err_put_pages:
>>          if (!obj->import_attach)
>> -               drm_gem_shmem_put_pages(shmem);
>> +               drm_gem_shmem_unpin(shmem);
>>   err_zero_use:
>>          shmem->vmap_use_count = 0;
>>
>> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>>                          return;
>>
>>                  vunmap(shmem->vaddr);
>> -               drm_gem_shmem_put_pages(shmem);
>> +               drm_gem_shmem_unpin(shmem);
>>          }
>>
>>          shmem->vaddr = NULL;
>> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>>
>>          madv = shmem->madv;
>>
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>>          return (madv >= 0);
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_madvise);
>>
>> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
>> +/**
>> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
>> + *                           hardware access to the memory.
>> + * @shmem: shmem GEM object
>> + *
>> + * This function moves shmem GEM back to memory if it was previously evicted
>> + * by the memory shrinker. The GEM is ready to use on success.
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>>   {
>>          struct drm_gem_object *obj = &shmem->base;
>> -       struct drm_device *dev = obj->dev;
>> +       struct sg_table *sgt;
>> +       int err;
>>
>>          dma_resv_assert_held(shmem->base.resv);
>>
>> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
>> +       if (shmem->evicted) {
>> +               err = drm_gem_shmem_acquire_pages(shmem);
>> +               if (err)
>> +                       return err;
>> +
>> +               sgt = drm_gem_shmem_get_sg_table(shmem);
>> +               if (IS_ERR(sgt))
>> +                       return PTR_ERR(sgt);
>> +
>> +               err = dma_map_sgtable(obj->dev->dev, sgt,
>> +                                     DMA_BIDIRECTIONAL, 0);
>> +               if (err) {
>> +                       sg_free_table(sgt);
>> +                       kfree(sgt);
>> +                       return err;
>> +               }
>>
>> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
>> -       sg_free_table(shmem->sgt);
>> -       kfree(shmem->sgt);
>> -       shmem->sgt = NULL;
>> +               shmem->sgt = sgt;
>> +               shmem->evicted = false;
>>
>> -       drm_gem_shmem_put_pages(shmem);
>> +               drm_gem_shmem_update_pages_state(shmem);
>> +       }
>>
>> -       shmem->madv = -1;
>> +       if (!shmem->pages)
>> +               return -ENOMEM;
>>
>> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
>> -       drm_gem_free_mmap_offset(obj);
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>>
>> -       /* Our goal here is to return as much of the memory as
>> -        * is possible back to the system as we are called from OOM.
>> -        * To do this we must instruct the shmfs to drop all of its
>> -        * backing pages, *now*.
>> -        */
>> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
>> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +       struct drm_device *dev = obj->dev;
>>
>> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
>> +       if (shmem->evicted)
>> +               return;
>> +
>> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
>> +       drm_gem_shmem_release_pages(shmem);
>> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
>> +
>> +       sg_free_table(shmem->sgt);
>> +       kfree(shmem->sgt);
>> +       shmem->sgt = NULL;
>>   }
>> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>>
>>   /**
>>    * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
>> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>>          vm_fault_t ret;
>>          struct page *page;
>>          pgoff_t page_offset;
>> +       bool pages_unpinned;
>> +       int err;
>>
>>          /* We don't use vmf->pgoff since that has the fake offset */
>>          page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>>
>>          dma_resv_lock(shmem->base.resv, NULL);
>>
>> -       if (page_offset >= num_pages ||
>> -           WARN_ON_ONCE(!shmem->pages) ||
>> -           shmem->madv < 0) {
>> +       /* Sanity-check that we have the pages pointer when it should present */
>> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
>> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
>> +
>> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>>                  ret = VM_FAULT_SIGBUS;
>>          } else {
>> +               err = drm_gem_shmem_swap_in(shmem);
>> +               if (err) {
>> +                       ret = VM_FAULT_OOM;
>> +                       goto unlock;
>> +               }
>> +
>>                  page = shmem->pages[page_offset];
>>
>>                  ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>>          }
>>
>> +unlock:
>>          dma_resv_unlock(shmem->base.resv);
>>
>>          return ret;
>> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>>   {
>>          struct drm_gem_object *obj = vma->vm_private_data;
>>          struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
>> -       int ret;
>>
>>          WARN_ON(shmem->base.import_attach);
>>
>>          dma_resv_lock(shmem->base.resv, NULL);
>> -       ret = drm_gem_shmem_get_pages(shmem);
>> -       WARN_ON_ONCE(ret != 0);
>> +
>> +       if (drm_gem_shmem_get_pages(shmem))
>> +               shmem->pages_use_count++;
>> +
>> +       drm_gem_shmem_update_pages_state(shmem);
>>          dma_resv_unlock(shmem->base.resv);
>>
>>          drm_gem_vm_open(vma);
>> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>>   void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>>                                struct drm_printer *p, unsigned int indent)
>>   {
>> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
>> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>>          drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>>
>>          if (shmem->base.import_attach)
>> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>>                  drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>>                                    shmem->vmap_use_count);
>>
>> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>>          drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
>> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>>   }
>>   EXPORT_SYMBOL(drm_gem_shmem_print_info);
>>
>> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>>
>>          shmem->sgt = sgt;
>>
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>>          dma_resv_unlock(shmem->base.resv);
>>
>>          return sgt;
>> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>>   }
>>   EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>>
>> +static struct drm_gem_shmem_shrinker *
>> +to_drm_shrinker(struct shrinker *shrinker)
>> +{
>> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
>> +}
>> +
>> +static unsigned long
>> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
>> +                                    struct shrink_control *sc)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
>> +       struct drm_gem_shmem_object *shmem;
>> +       unsigned long count = 0;
>> +
>> +       if (!mutex_trylock(&gem_shrinker->lock))
>> +               return 0;
>> +
>> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
>> +               count += shmem->base.size;
>> +
>> +               if (count >= SHRINK_EMPTY)
>> +                       break;
>> +       }
>> +
>> +       mutex_unlock(&gem_shrinker->lock);
>> +
>> +       if (count >= SHRINK_EMPTY)
>> +               return SHRINK_EMPTY - 1;
>> +
>> +       return count ?: SHRINK_EMPTY;
>> +}
>> +
>> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
>> +{
>> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
>> +       WARN_ON(shmem->evicted);
>> +
>> +       drm_gem_shmem_unpin_pages(shmem);
>> +
>> +       shmem->evicted = true;
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
>> +
>> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
>> +{
>> +       struct drm_gem_object *obj = &shmem->base;
>> +
>> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
>> +
>> +       drm_gem_shmem_unpin_pages(shmem);
>> +       drm_gem_free_mmap_offset(obj);
>> +
>> +       /* Our goal here is to return as much of the memory as
>> +        * is possible back to the system as we are called from OOM.
>> +        * To do this we must instruct the shmfs to drop all of its
>> +        * backing pages, *now*.
>> +        */
>> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
>> +
>> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
>> +
>> +       shmem->madv = -1;
>> +       shmem->evicted = false;
>> +       drm_gem_shmem_update_pages_state(shmem);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
>> +
>> +static unsigned long
>> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
>> +                                       unsigned long nr_to_scan,
>> +                                       bool *lock_contention,
>> +                                       bool evict)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
>> +       struct drm_gem_shmem_object *shmem;
>> +       struct list_head still_in_list;
>> +       struct drm_gem_object *obj;
>> +       unsigned long freed = 0;
>> +       size_t page_count;
>> +       int err;
>> +
>> +       INIT_LIST_HEAD(&still_in_list);
>> +
>> +       mutex_lock(&gem_shrinker->lock);
>> +
>> +       while (freed < nr_to_scan) {
>> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
>> +                                                typeof(*shmem), madv_list);
>> +               if (!shmem)
>> +                       break;
>> +
>> +               obj = &shmem->base;
>> +               page_count = obj->size >> PAGE_SHIFT;
>> +               list_move_tail(&shmem->madv_list, &still_in_list);
>> +
>> +               if (evict) {
>> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
>> +                           get_nr_swap_pages() < page_count)
>> +                               continue;
>> +               } else {
>> +                       if (!drm_gem_shmem_is_purgeable(shmem))
>> +                               continue;
>> +               }
>> +
>> +               /*
>> +                * If it's in the process of being freed, gem_object->free()
>> +                * may be blocked on lock waiting to remove it.  So just
>> +                * skip it.
>> +                */
>> +               if (!kref_get_unless_zero(&obj->refcount))
>> +                       continue;
>> +
>> +               mutex_unlock(&gem_shrinker->lock);
>> +
>> +               /* prevent racing with job-submission code paths */
>> +               if (!dma_resv_trylock(obj->resv)) {
>> +                       *lock_contention |= true;
>> +                       goto shrinker_lock;
>> +               }
>> +
>> +               /* prevent racing with the dma-buf importing/exporting */
>> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
>> +                       *lock_contention |= true;
>> +                       goto resv_unlock;
>> +               }
>> +
>> +               /* check whether h/w uses this object */
>> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
>> +                       goto object_name_unlock;
>> +
>> +               /* re-check whether eviction status hasn't changed */
>> +               if (!drm_gem_shmem_is_evictable(shmem) &&
>> +                   !drm_gem_shmem_is_purgeable(shmem))
>> +                       goto object_name_unlock;
>> +
>> +               err = shmem->evict(shmem);
>> +               if (!err)
>> +                       freed += obj->size >> PAGE_SHIFT;
>> +
>> +object_name_unlock:
>> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
>> +resv_unlock:
>> +               dma_resv_unlock(obj->resv);
>> +shrinker_lock:
>> +               drm_gem_object_put(&shmem->base);
>> +               mutex_lock(&gem_shrinker->lock);
>> +       }
>> +
>> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
>> +
>> +       mutex_unlock(&gem_shrinker->lock);
>> +
>> +       return freed;
>> +}
>> +
>> +static unsigned long
>> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
>> +                                   struct shrink_control *sc)
>> +{
>> +       unsigned long nr_to_scan = sc->nr_to_scan;
>> +       bool lock_contention = false;
>> +       unsigned long freed;
>> +
>> +       /* purge as many objects as we can */
>> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
>> +                                                       &lock_contention, false);
>> +
>> +       /* evict as many objects as we can */
>> +       if (freed < nr_to_scan)
>> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
>> +                                                                nr_to_scan - freed,
>> +                                                                &lock_contention,
>> +                                                                true);
>> +
>> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
>> +}
>> +
>> +/**
>> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
>> + * @dev: DRM device
>> + *
>> + * Returns:
>> + * 0 on success or a negative error code on failure.
>> + */
>> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker;
>> +       int err;
>> +
>> +       if (WARN_ON(dev->shmem_shrinker))
>> +               return -EBUSY;
>> +
>> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
>> +       if (!gem_shrinker)
>> +               return -ENOMEM;
>> +
>> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
>> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
>> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
>> +       gem_shrinker->dev = dev;
>> +
>> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
>> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
>> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
>> +       mutex_init(&gem_shrinker->lock);
>> +
>> +       dev->shmem_shrinker = gem_shrinker;
>> +
>> +       err = register_shrinker(&gem_shrinker->base);
>> +       if (err) {
>> +               dev->shmem_shrinker = NULL;
>> +               kfree(gem_shrinker);
>> +               return err;
>> +       }
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
>> +
>> +/**
>> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
>> + * @dev: DRM device
>> + */
>> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
>> +
>> +       if (gem_shrinker) {
>> +               unregister_shrinker(&gem_shrinker->base);
>> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
>> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
>> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
>> +               mutex_destroy(&gem_shrinker->lock);
>> +               dev->shmem_shrinker = NULL;
>> +               kfree(gem_shrinker);
>> +       }
>> +}
>> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
>> +
>>   MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>>   MODULE_IMPORT_NS(DMA_BUF);
>>   MODULE_LICENSE("GPL v2");
>> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
>> index a4bedfeb2ec4..7cc32556f908 100644
>> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
>> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
>> @@ -15,6 +15,13 @@
>>   #include "panfrost_gem.h"
>>   #include "panfrost_mmu.h"
>>
>> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>> +{
>> +       return (shmem->madv > 0) &&
>> +               !shmem->pages_pin_count && shmem->sgt &&
>> +               !shmem->base.dma_buf && !shmem->base.import_attach;
>> +}
>> +
>>   static unsigned long
>>   panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>>   {
>> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>>                  return 0;
>>
>>          list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
>> -               if (drm_gem_shmem_is_purgeable(shmem))
>> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>>                          count += shmem->base.size >> PAGE_SHIFT;
>>          }
>>
>> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
>> index b2d93cb12ebf..81bacc7e1873 100644
>> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
>> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
>> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>>          uint32_t hw_res_handle;
>>          bool dumb;
>>          bool created;
>> +       bool detached;
>>          bool host3d_blob, guest_blob;
>>          uint32_t blob_mem, blob_flags;
>>
>> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>>
>>   bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>>
>> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
>> +
>>   int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>>                                 uint32_t *resid);
>>   /* virtgpu_prime.c */
>> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
>> index 9923c7a6885e..929546cad894 100644
>> --- a/include/drm/drm_device.h
>> +++ b/include/drm/drm_device.h
>> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>>   struct drm_vma_offset_manager;
>>   struct drm_vram_mm;
>>   struct drm_fb_helper;
>> +struct drm_gem_shmem_shrinker;
>>
>>   struct inode;
>>
>> @@ -277,6 +278,9 @@ struct drm_device {
>>          /** @vram_mm: VRAM MM memory manager */
>>          struct drm_vram_mm *vram_mm;
>>
>> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
>> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
>> +
>>          /**
>>           * @switch_power_state:
>>           *
>> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
>> index 9a8983ee8abe..62c640678a91 100644
>> --- a/include/drm/drm_gem_shmem_helper.h
>> +++ b/include/drm/drm_gem_shmem_helper.h
>> @@ -6,6 +6,7 @@
>>   #include <linux/fs.h>
>>   #include <linux/mm.h>
>>   #include <linux/mutex.h>
>> +#include <linux/shrinker.h>
>>
>>   #include <drm/drm_file.h>
>>   #include <drm/drm_gem.h>
>> @@ -15,6 +16,7 @@
>>   struct dma_buf_attachment;
>>   struct drm_mode_create_dumb;
>>   struct drm_printer;
>> +struct drm_device;
>>   struct sg_table;
>>
>>   /**
>> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>>           */
>>          unsigned int pages_use_count;
>>
>> +       /**
>> +        * @pages_pin_count:
>> +        *
>> +        * Reference count on the pinned pages table.
>> +        * The pages can be evicted by memory shrinker
>> +        * when the count reaches zero.
>> +        */
>> +       unsigned int pages_pin_count;
>> +
>>          /**
>>           * @madv: State for madvise
>>           *
>>           * 0 is active/inuse.
>> +        * 1 is not-needed/can-be-purged
>>           * A negative value is the object is purged.
>> -        * Positive values are driver specific and not used by the helpers.
>>           */
>>          int madv;
>>
>> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>>           * @map_wc: map object write-combined (instead of using shmem defaults).
>>           */
>>          bool map_wc;
>> +
>> +       /**
>> +        * @eviction_enabled:
>> +        *
>> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
>> +        * Used internally by memory shrinker.
>> +        */
>> +       bool eviction_enabled;
>> +
>> +       /**
>> +        * @purge_enabled:
>> +        *
>> +        * The shmem pages can be purged only if @purge_enabled is set to true.
>> +        * Used internally by memory shrinker.
>> +        */
>> +       bool purge_enabled;
>> +
>> +       /**
>> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
>> +        * Used internally by memory shrinker.
>> +        */
>> +       bool evicted;
>> +
>> +       /**
>> +        * @evict:
>> +        *
>> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
>> +        * GEM's DMA reservation is kept locked by the shrinker. This is
>> +        * optional callback that should be specified by drivers.
>> +        *
>> +        * Returns 0 on success, or -errno on error.
>> +        */
>> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>>   };
>>
>>   #define to_drm_gem_shmem_obj(obj) \
>> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>>
>>   int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>>
>> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
>> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
>> +
>>   static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>>   {
>> -       return (shmem->madv > 0) &&
>> -               !shmem->vmap_use_count && shmem->sgt &&
>> -               !shmem->base.dma_buf && !shmem->base.import_attach;
>> +       return (shmem->madv > 0) && shmem->evict &&
>> +               shmem->purge_enabled && shmem->pages_use_count &&
>> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
>> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>>   }
>>
>> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
>> +
>> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
>> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>>
>>   struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>>   struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
>> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>>          return drm_gem_shmem_mmap(shmem, vma);
>>   }
>>
>> +/**
>> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
>> + */
>> +struct drm_gem_shmem_shrinker {
>> +       /** @base: Shrinker for purging shmem GEM objects */
>> +       struct shrinker base;
>> +
>> +       /** @lock: Protects @lru_* */
>> +       struct mutex lock;
>> +
>> +       /** @lru_pinned: List of pinned shmem GEM objects */
>> +       struct list_head lru_pinned;
>> +
>> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
>> +       struct list_head lru_evictable;
>> +
>> +       /** @lru_evicted: List of evicted shmem GEM objects */
>> +       struct list_head lru_evicted;
>> +
>> +       /** @dev: DRM device that uses this shrinker */
>> +       struct drm_device *dev;
>> +};
>> +
>> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
>> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
>> +
>>   /*
>>    * Driver ops
>>    */
>> --
>> 2.35.3
>>
>

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Dmitry Osipenko]

On 6/6/22 13:57, Christian König wrote:
> Am 05.06.22 um 18:47 schrieb Daniel Vetter:
>> On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
>> <dmitry.osipenko@collabora.com> wrote:
>>> Introduce a common DRM SHMEM shrinker framework that allows to reduce
>>> code duplication among DRM drivers by replacing theirs custom shrinker
>>> implementations with the generic shrinker.
>>>
>>> In order to start using DRM SHMEM shrinker drivers should:
>>>
>>> 1. Implement new evict() shmem object callback.
>>> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
>>> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>>>     activate shrinking of shmem GEMs.
>>>
>>> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
>>> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>>>
>>> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>> So I guess I get a price for being blind since forever, because this
>> thing existed since at least 2013. I just stumbled over
>> llist_lru.[hc], a purpose built list helper for shrinkers. I think we
>> should try to adopt that so that our gpu shrinkers look more like
>> shrinkers for everything else.
> 
> What the heck are you talking about?
> 
> I can't find any llist_lru.[hc] in the linux kernel sources.

I think Daniel meant this:

https://elixir.bootlin.com/linux/v5.19-rc1/source/include/linux/list_lru.h

https://elixir.bootlin.com/linux/v5.19-rc1/source/mm/list_lru.c


-- 
Best regards,
Dmitry

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Dmitry Osipenko]

On 6/6/22 13:57, Christian König wrote:
> Am 05.06.22 um 18:47 schrieb Daniel Vetter:
>> On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
>> <dmitry.osipenko@collabora.com> wrote:
>>> Introduce a common DRM SHMEM shrinker framework that allows to reduce
>>> code duplication among DRM drivers by replacing theirs custom shrinker
>>> implementations with the generic shrinker.
>>>
>>> In order to start using DRM SHMEM shrinker drivers should:
>>>
>>> 1. Implement new evict() shmem object callback.
>>> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
>>> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>>>     activate shrinking of shmem GEMs.
>>>
>>> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
>>> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>>>
>>> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>> So I guess I get a price for being blind since forever, because this
>> thing existed since at least 2013. I just stumbled over
>> llist_lru.[hc], a purpose built list helper for shrinkers. I think we
>> should try to adopt that so that our gpu shrinkers look more like
>> shrinkers for everything else.
> 
> What the heck are you talking about?
> 
> I can't find any llist_lru.[hc] in the linux kernel sources.

I think Daniel meant this:

https://elixir.bootlin.com/linux/v5.19-rc1/source/include/linux/list_lru.h

https://elixir.bootlin.com/linux/v5.19-rc1/source/mm/list_lru.c


-- 
Best regards,
Dmitry

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Dmitry Osipenko]

On 6/6/22 13:57, Christian König wrote:
> Am 05.06.22 um 18:47 schrieb Daniel Vetter:
>> On Fri, 27 May 2022 at 01:55, Dmitry Osipenko
>> <dmitry.osipenko@collabora.com> wrote:
>>> Introduce a common DRM SHMEM shrinker framework that allows to reduce
>>> code duplication among DRM drivers by replacing theirs custom shrinker
>>> implementations with the generic shrinker.
>>>
>>> In order to start using DRM SHMEM shrinker drivers should:
>>>
>>> 1. Implement new evict() shmem object callback.
>>> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
>>> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>>>     activate shrinking of shmem GEMs.
>>>
>>> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
>>> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>>>
>>> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>> So I guess I get a price for being blind since forever, because this
>> thing existed since at least 2013. I just stumbled over
>> llist_lru.[hc], a purpose built list helper for shrinkers. I think we
>> should try to adopt that so that our gpu shrinkers look more like
>> shrinkers for everything else.
> 
> What the heck are you talking about?
> 
> I can't find any llist_lru.[hc] in the linux kernel sources.

I think Daniel meant this:

https://elixir.bootlin.com/linux/v5.19-rc1/source/include/linux/list_lru.h

https://elixir.bootlin.com/linux/v5.19-rc1/source/mm/list_lru.c


-- 
Best regards,
Dmitry

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Dmitry Osipenko]

On 5/27/22 02:50, Dmitry Osipenko wrote:
> Hello,
> 
> This patchset introduces memory shrinker for the VirtIO-GPU DRM driver
> and adds memory purging and eviction support to VirtIO-GPU driver.
> 
> The new dma-buf locking convention is introduced here as well.
> 
> During OOM, the shrinker will release BOs that are marked as "not needed"
> by userspace using the new madvise IOCTL, it will also evict idling BOs
> to SWAP. The userspace in this case is the Mesa VirGL driver, it will mark
> the cached BOs as "not needed", allowing kernel driver to release memory
> of the cached shmem BOs on lowmem situations, preventing OOM kills.
> 
> The Panfrost driver is switched to use generic memory shrinker.
> 
> This patchset includes improvements and fixes for various things that
> I found while was working on the shrinker.
> 
> The Mesa and IGT patches will be kept on hold until this kernel series
> will be approved and merged.
> 
> This patchset was tested using Qemu and crosvm, including both cases of
> IOMMU off/on.
> 
> Mesa: https://gitlab.freedesktop.org/digetx/mesa/-/commits/virgl-madvise
> IGT:  https://gitlab.freedesktop.org/digetx/igt-gpu-tools/-/commits/virtio-madvise
>       https://gitlab.freedesktop.org/digetx/igt-gpu-tools/-/commits/panfrost-madvise
> 
> Changelog:
> 
> v6: - Added new VirtIO-related fix patch that previously was sent separately
>       and didn't get much attention:
> 
>         drm/gem: Properly annotate WW context on drm_gem_lock_reservations() error
> 
>     - Added new patch that fixes mapping of imported dma-bufs for
>       Tegra DRM and other affected drivers. It's also handy to have it
>       for switching to the new dma-buf locking convention scheme:
> 
>         drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()
> 
>     - Added new patch that fixes shrinker list corruption for stable Panfrost
>       driver:
> 
>         drm/panfrost: Fix shrinker list corruption by madvise IOCTL
> 
>     - Added new minor patch-fix for drm-shmem:
> 
>         drm/shmem-helper: Add missing vunmap on error
> 
>     - Added fixes tag to the "Put mapping ..." patch like was suggested by
>       Steven Price.
> 
>     - Added new VirtIO-GPU driver improvement patch:
> 
>         drm/virtio: Return proper error codes instead of -1
> 
>     - Reworked shrinker patches like was suggested by Daniel Vetter:
> 
>         - Introduced the new locking convention for dma-bufs. Tested on
>           VirtIO-GPU, Panfrost, Lima, Tegra and Intel selftests.
> 
>         - Dropped separate purge() callback. Now single evict() does
>           everything.
> 
>         - Dropped swap_in() callback from drm-shmem objects. DRM drivers
>           now could and should restore only the required mappings.
> 
>         - Dropped dynamic counting of evictable pages. This simplifies
>           code in exchange to *potentially* burning more CPU time on OOM.
> 
> v5: - Added new for-stable patch "drm/panfrost: Put mapping instead of
>       shmem obj on panfrost_mmu_map_fault_addr() error" that corrects GEM's
>       refcounting in case of error.
> 
>     - The drm_gem_shmem_v[un]map() now takes a separate vmap_lock for
>       imported GEMs to avoid recursive locking of DMA reservations.
>       This addresses v4 comment from Thomas Zimmermann about the potential
>       deadlocking of vmapping.
> 
>     - Added ack from Thomas Zimmermann to "drm/shmem-helper: Correct
>       doc-comment of drm_gem_shmem_get_sg_table()" patch.
> 
>     - Dropped explicit shmem states from the generic shrinker patch as
>       was requested by Thomas Zimmermann.
> 
>     - Improved variable names and comments of the generic shrinker code.
> 
>     - Extended drm_gem_shmem_print_info() with the shrinker-state info in
>       the "drm/virtio: Support memory shrinking" patch.
> 
>     - Moved evict()/swap_in()/purge() callbacks from drm_gem_object_funcs
>       to drm_gem_shmem_object in the generic shrinker patch, for more
>       consistency.
> 
>     - Corrected bisectability of the patches that was broken in v4
>       by accident.
> 
>     - The virtio_gpu_plane_prepare_fb() now uses drm_gem_shmem_pin() instead
>       of drm_gem_shmem_set_unpurgeable_and_unevictable() and does it only for
>       shmem BOs in the "drm/virtio: Support memory shrinking" patch.
> 
>     - Made more functions private to drm_gem_shmem_helper.c as was requested
>       by Thomas Zimmermann. This minimizes number of the public shmem helpers.
> 
> v4: - Corrected minor W=1 warnings reported by kernel test robot for v3.
> 
>     - Renamed DRM_GEM_SHMEM_PAGES_STATE_ACTIVE/INACTIVE to PINNED/UNPINNED,
>       for more clarity.
> 
> v3: - Hardened shrinker's count() with usage of READ_ONCE() since we don't
>       use atomic type for counting and technically compiler is free to
>       re-fetch counter's variable.
> 
>     - "Correct drm_gem_shmem_get_sg_table() error handling" now uses
>       PTR_ERR_OR_ZERO(), fixing typo that was made in v2.
> 
>     - Removed obsoleted shrinker from the Panfrost driver, which I missed to
>       do in v2 by accident and Alyssa Rosenzweig managed to notice it.
> 
>     - CCed stable kernels in all patches that make fixes, even the minor ones,
>       like was suggested by Emil Velikov and added his r-b to the patches.
> 
>     - Added t-b from Steven Price to the Panfrost's shrinker patch.
> 
>     - Corrected doc-comment of drm_gem_shmem_object.madv, like was suggested
>       by Steven Price. Comment now says that madv=1 means "object is purged"
>       instead of saying that value is unused.
> 
>     - Added more doc-comments to the new shmem shrinker API.
> 
>     - The "Improve DMA API usage for shmem BOs" patch got more improvements
>       by removing the obsoleted drm_dev_set_unique() quirk and its comment.
> 
>     - Added patch that makes Virtio-GPU driver to use common dev_is_pci()
>       helper, which was suggested by Robin Murphy.
> 
>     - Added new "drm/shmem-helper: Take GEM reservation lock instead of
>       drm_gem_shmem locks" patch, which was suggested by Daniel Vetter.
> 
>     - Added new "drm/virtio: Simplify error handling of
>       virtio_gpu_object_create()" patch.
> 
>     - Improved "Correct doc-comment of drm_gem_shmem_get_sg_table()" patch,
>       like was suggested by Daniel Vetter, by saying that function returns
>       ERR_PTR() and not errno.
> 
>     - virtio_gpu_purge_object() is fenced properly now, turned out
>       virtio_gpu_notify() doesn't do fencing as I was supposing before.
>       Stress testing of memory eviction revealed that.
> 
>     - Added new patch that corrects virtio_gpu_plane_cleanup_fb() to use
>       appropriate atomic plane state.
> 
>     - SHMEM shrinker got eviction support.
> 
>     - VirtIO-GPU driver now supports memory eviction. It's enabled for a
>       non-blob GEMs only, i.e. for VirGL. The blobs don't support dynamic
>       attaching/detaching of guest's memory, so it's not trivial to enable
>       them.
> 
>     - Added patch that removes obsoleted drm_gem_shmem_purge()
> 
>     - Added patch that makes drm_gem_shmem_get_pages() private.
> 
>     - Added patch that fixes lockup on dma_resv_reserve_fences() error.
> 
> v2: - Improved shrinker by using a more fine-grained locking to reduce
>       contention during scan of objects and dropped locking from the
>       'counting' callback by tracking count of shrinkable pages. This
>       was suggested by Rob Clark in the comment to v1.
> 
>     - Factored out common shrinker code into drm_gem_shmem_helper.c
>       and switched Panfrost driver to use the new common memory shrinker.
>       This was proposed by Thomas Zimmermann in his prototype series that
>       he shared with us in the comment to v1. Note that I only compile-tested
>       the Panfrost driver.
> 
>     - Shrinker now takes object_name_lock during scan to prevent racing
>       with dma-buf exporting.
> 
>     - Shrinker now takes vmap_lock during scan to prevent racing with shmem
>       vmap/unmap code.
> 
>     - Added "Correct doc-comment of drm_gem_shmem_get_sg_table()" patch,
>       which I sent out previously as a standalone change, since the
>       drm_gem_shmem_helper.c is now touched by this patchset anyways and
>       it doesn't hurt to group all the patches together.
> 
> Dmitry Osipenko (22):
>   drm/gem: Properly annotate WW context on drm_gem_lock_reservations()
>     error
>   drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()
>   drm/panfrost: Put mapping instead of shmem obj on
>     panfrost_mmu_map_fault_addr() error
>   drm/panfrost: Fix shrinker list corruption by madvise IOCTL
>   drm/virtio: Correct drm_gem_shmem_get_sg_table() error handling
>   drm/virtio: Check whether transferred 2D BO is shmem
>   drm/virtio: Unlock reservations on virtio_gpu_object_shmem_init()
>     error
>   drm/virtio: Unlock reservations on dma_resv_reserve_fences() error
>   drm/virtio: Use appropriate atomic state in
>     virtio_gpu_plane_cleanup_fb()
>   drm/shmem-helper: Add missing vunmap on error
>   drm/shmem-helper: Correct doc-comment of drm_gem_shmem_get_sg_table()
...

Thomas, do you think it will be possible for you to take the fix patches
1-11 into the drm-fixes or would you prefer me to re-send them separately?

The VirtIO patches 12-13 also are good to go into drm-next, IMO.

I'm going to factor out the new dma-buf convention into a separate
patchset, like was suggested by Christian. But it will take me some time
to get the dma-buf patches ready and I also will be on a vacation soon.

At minimum nothing should hold the fixes, so will be great if they could
land sooner.

Thank you!

-- 
Best regards,
Dmitry

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Dmitry Osipenko]

On 5/27/22 02:50, Dmitry Osipenko wrote:
> Hello,
> 
> This patchset introduces memory shrinker for the VirtIO-GPU DRM driver
> and adds memory purging and eviction support to VirtIO-GPU driver.
> 
> The new dma-buf locking convention is introduced here as well.
> 
> During OOM, the shrinker will release BOs that are marked as "not needed"
> by userspace using the new madvise IOCTL, it will also evict idling BOs
> to SWAP. The userspace in this case is the Mesa VirGL driver, it will mark
> the cached BOs as "not needed", allowing kernel driver to release memory
> of the cached shmem BOs on lowmem situations, preventing OOM kills.
> 
> The Panfrost driver is switched to use generic memory shrinker.
> 
> This patchset includes improvements and fixes for various things that
> I found while was working on the shrinker.
> 
> The Mesa and IGT patches will be kept on hold until this kernel series
> will be approved and merged.
> 
> This patchset was tested using Qemu and crosvm, including both cases of
> IOMMU off/on.
> 
> Mesa: https://gitlab.freedesktop.org/digetx/mesa/-/commits/virgl-madvise
> IGT:  https://gitlab.freedesktop.org/digetx/igt-gpu-tools/-/commits/virtio-madvise
>       https://gitlab.freedesktop.org/digetx/igt-gpu-tools/-/commits/panfrost-madvise
> 
> Changelog:
> 
> v6: - Added new VirtIO-related fix patch that previously was sent separately
>       and didn't get much attention:
> 
>         drm/gem: Properly annotate WW context on drm_gem_lock_reservations() error
> 
>     - Added new patch that fixes mapping of imported dma-bufs for
>       Tegra DRM and other affected drivers. It's also handy to have it
>       for switching to the new dma-buf locking convention scheme:
> 
>         drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()
> 
>     - Added new patch that fixes shrinker list corruption for stable Panfrost
>       driver:
> 
>         drm/panfrost: Fix shrinker list corruption by madvise IOCTL
> 
>     - Added new minor patch-fix for drm-shmem:
> 
>         drm/shmem-helper: Add missing vunmap on error
> 
>     - Added fixes tag to the "Put mapping ..." patch like was suggested by
>       Steven Price.
> 
>     - Added new VirtIO-GPU driver improvement patch:
> 
>         drm/virtio: Return proper error codes instead of -1
> 
>     - Reworked shrinker patches like was suggested by Daniel Vetter:
> 
>         - Introduced the new locking convention for dma-bufs. Tested on
>           VirtIO-GPU, Panfrost, Lima, Tegra and Intel selftests.
> 
>         - Dropped separate purge() callback. Now single evict() does
>           everything.
> 
>         - Dropped swap_in() callback from drm-shmem objects. DRM drivers
>           now could and should restore only the required mappings.
> 
>         - Dropped dynamic counting of evictable pages. This simplifies
>           code in exchange to *potentially* burning more CPU time on OOM.
> 
> v5: - Added new for-stable patch "drm/panfrost: Put mapping instead of
>       shmem obj on panfrost_mmu_map_fault_addr() error" that corrects GEM's
>       refcounting in case of error.
> 
>     - The drm_gem_shmem_v[un]map() now takes a separate vmap_lock for
>       imported GEMs to avoid recursive locking of DMA reservations.
>       This addresses v4 comment from Thomas Zimmermann about the potential
>       deadlocking of vmapping.
> 
>     - Added ack from Thomas Zimmermann to "drm/shmem-helper: Correct
>       doc-comment of drm_gem_shmem_get_sg_table()" patch.
> 
>     - Dropped explicit shmem states from the generic shrinker patch as
>       was requested by Thomas Zimmermann.
> 
>     - Improved variable names and comments of the generic shrinker code.
> 
>     - Extended drm_gem_shmem_print_info() with the shrinker-state info in
>       the "drm/virtio: Support memory shrinking" patch.
> 
>     - Moved evict()/swap_in()/purge() callbacks from drm_gem_object_funcs
>       to drm_gem_shmem_object in the generic shrinker patch, for more
>       consistency.
> 
>     - Corrected bisectability of the patches that was broken in v4
>       by accident.
> 
>     - The virtio_gpu_plane_prepare_fb() now uses drm_gem_shmem_pin() instead
>       of drm_gem_shmem_set_unpurgeable_and_unevictable() and does it only for
>       shmem BOs in the "drm/virtio: Support memory shrinking" patch.
> 
>     - Made more functions private to drm_gem_shmem_helper.c as was requested
>       by Thomas Zimmermann. This minimizes number of the public shmem helpers.
> 
> v4: - Corrected minor W=1 warnings reported by kernel test robot for v3.
> 
>     - Renamed DRM_GEM_SHMEM_PAGES_STATE_ACTIVE/INACTIVE to PINNED/UNPINNED,
>       for more clarity.
> 
> v3: - Hardened shrinker's count() with usage of READ_ONCE() since we don't
>       use atomic type for counting and technically compiler is free to
>       re-fetch counter's variable.
> 
>     - "Correct drm_gem_shmem_get_sg_table() error handling" now uses
>       PTR_ERR_OR_ZERO(), fixing typo that was made in v2.
> 
>     - Removed obsoleted shrinker from the Panfrost driver, which I missed to
>       do in v2 by accident and Alyssa Rosenzweig managed to notice it.
> 
>     - CCed stable kernels in all patches that make fixes, even the minor ones,
>       like was suggested by Emil Velikov and added his r-b to the patches.
> 
>     - Added t-b from Steven Price to the Panfrost's shrinker patch.
> 
>     - Corrected doc-comment of drm_gem_shmem_object.madv, like was suggested
>       by Steven Price. Comment now says that madv=1 means "object is purged"
>       instead of saying that value is unused.
> 
>     - Added more doc-comments to the new shmem shrinker API.
> 
>     - The "Improve DMA API usage for shmem BOs" patch got more improvements
>       by removing the obsoleted drm_dev_set_unique() quirk and its comment.
> 
>     - Added patch that makes Virtio-GPU driver to use common dev_is_pci()
>       helper, which was suggested by Robin Murphy.
> 
>     - Added new "drm/shmem-helper: Take GEM reservation lock instead of
>       drm_gem_shmem locks" patch, which was suggested by Daniel Vetter.
> 
>     - Added new "drm/virtio: Simplify error handling of
>       virtio_gpu_object_create()" patch.
> 
>     - Improved "Correct doc-comment of drm_gem_shmem_get_sg_table()" patch,
>       like was suggested by Daniel Vetter, by saying that function returns
>       ERR_PTR() and not errno.
> 
>     - virtio_gpu_purge_object() is fenced properly now, turned out
>       virtio_gpu_notify() doesn't do fencing as I was supposing before.
>       Stress testing of memory eviction revealed that.
> 
>     - Added new patch that corrects virtio_gpu_plane_cleanup_fb() to use
>       appropriate atomic plane state.
> 
>     - SHMEM shrinker got eviction support.
> 
>     - VirtIO-GPU driver now supports memory eviction. It's enabled for a
>       non-blob GEMs only, i.e. for VirGL. The blobs don't support dynamic
>       attaching/detaching of guest's memory, so it's not trivial to enable
>       them.
> 
>     - Added patch that removes obsoleted drm_gem_shmem_purge()
> 
>     - Added patch that makes drm_gem_shmem_get_pages() private.
> 
>     - Added patch that fixes lockup on dma_resv_reserve_fences() error.
> 
> v2: - Improved shrinker by using a more fine-grained locking to reduce
>       contention during scan of objects and dropped locking from the
>       'counting' callback by tracking count of shrinkable pages. This
>       was suggested by Rob Clark in the comment to v1.
> 
>     - Factored out common shrinker code into drm_gem_shmem_helper.c
>       and switched Panfrost driver to use the new common memory shrinker.
>       This was proposed by Thomas Zimmermann in his prototype series that
>       he shared with us in the comment to v1. Note that I only compile-tested
>       the Panfrost driver.
> 
>     - Shrinker now takes object_name_lock during scan to prevent racing
>       with dma-buf exporting.
> 
>     - Shrinker now takes vmap_lock during scan to prevent racing with shmem
>       vmap/unmap code.
> 
>     - Added "Correct doc-comment of drm_gem_shmem_get_sg_table()" patch,
>       which I sent out previously as a standalone change, since the
>       drm_gem_shmem_helper.c is now touched by this patchset anyways and
>       it doesn't hurt to group all the patches together.
> 
> Dmitry Osipenko (22):
>   drm/gem: Properly annotate WW context on drm_gem_lock_reservations()
>     error
>   drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()
>   drm/panfrost: Put mapping instead of shmem obj on
>     panfrost_mmu_map_fault_addr() error
>   drm/panfrost: Fix shrinker list corruption by madvise IOCTL
>   drm/virtio: Correct drm_gem_shmem_get_sg_table() error handling
>   drm/virtio: Check whether transferred 2D BO is shmem
>   drm/virtio: Unlock reservations on virtio_gpu_object_shmem_init()
>     error
>   drm/virtio: Unlock reservations on dma_resv_reserve_fences() error
>   drm/virtio: Use appropriate atomic state in
>     virtio_gpu_plane_cleanup_fb()
>   drm/shmem-helper: Add missing vunmap on error
>   drm/shmem-helper: Correct doc-comment of drm_gem_shmem_get_sg_table()
...

Thomas, do you think it will be possible for you to take the fix patches
1-11 into the drm-fixes or would you prefer me to re-send them separately?

The VirtIO patches 12-13 also are good to go into drm-next, IMO.

I'm going to factor out the new dma-buf convention into a separate
patchset, like was suggested by Christian. But it will take me some time
to get the dma-buf patches ready and I also will be on a vacation soon.

At minimum nothing should hold the fixes, so will be great if they could
land sooner.

Thank you!

-- 
Best regards,
Dmitry

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Dmitry Osipenko]

On 5/27/22 02:50, Dmitry Osipenko wrote:
> Hello,
> 
> This patchset introduces memory shrinker for the VirtIO-GPU DRM driver
> and adds memory purging and eviction support to VirtIO-GPU driver.
> 
> The new dma-buf locking convention is introduced here as well.
> 
> During OOM, the shrinker will release BOs that are marked as "not needed"
> by userspace using the new madvise IOCTL, it will also evict idling BOs
> to SWAP. The userspace in this case is the Mesa VirGL driver, it will mark
> the cached BOs as "not needed", allowing kernel driver to release memory
> of the cached shmem BOs on lowmem situations, preventing OOM kills.
> 
> The Panfrost driver is switched to use generic memory shrinker.
> 
> This patchset includes improvements and fixes for various things that
> I found while was working on the shrinker.
> 
> The Mesa and IGT patches will be kept on hold until this kernel series
> will be approved and merged.
> 
> This patchset was tested using Qemu and crosvm, including both cases of
> IOMMU off/on.
> 
> Mesa: https://gitlab.freedesktop.org/digetx/mesa/-/commits/virgl-madvise
> IGT:  https://gitlab.freedesktop.org/digetx/igt-gpu-tools/-/commits/virtio-madvise
>       https://gitlab.freedesktop.org/digetx/igt-gpu-tools/-/commits/panfrost-madvise
> 
> Changelog:
> 
> v6: - Added new VirtIO-related fix patch that previously was sent separately
>       and didn't get much attention:
> 
>         drm/gem: Properly annotate WW context on drm_gem_lock_reservations() error
> 
>     - Added new patch that fixes mapping of imported dma-bufs for
>       Tegra DRM and other affected drivers. It's also handy to have it
>       for switching to the new dma-buf locking convention scheme:
> 
>         drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()
> 
>     - Added new patch that fixes shrinker list corruption for stable Panfrost
>       driver:
> 
>         drm/panfrost: Fix shrinker list corruption by madvise IOCTL
> 
>     - Added new minor patch-fix for drm-shmem:
> 
>         drm/shmem-helper: Add missing vunmap on error
> 
>     - Added fixes tag to the "Put mapping ..." patch like was suggested by
>       Steven Price.
> 
>     - Added new VirtIO-GPU driver improvement patch:
> 
>         drm/virtio: Return proper error codes instead of -1
> 
>     - Reworked shrinker patches like was suggested by Daniel Vetter:
> 
>         - Introduced the new locking convention for dma-bufs. Tested on
>           VirtIO-GPU, Panfrost, Lima, Tegra and Intel selftests.
> 
>         - Dropped separate purge() callback. Now single evict() does
>           everything.
> 
>         - Dropped swap_in() callback from drm-shmem objects. DRM drivers
>           now could and should restore only the required mappings.
> 
>         - Dropped dynamic counting of evictable pages. This simplifies
>           code in exchange to *potentially* burning more CPU time on OOM.
> 
> v5: - Added new for-stable patch "drm/panfrost: Put mapping instead of
>       shmem obj on panfrost_mmu_map_fault_addr() error" that corrects GEM's
>       refcounting in case of error.
> 
>     - The drm_gem_shmem_v[un]map() now takes a separate vmap_lock for
>       imported GEMs to avoid recursive locking of DMA reservations.
>       This addresses v4 comment from Thomas Zimmermann about the potential
>       deadlocking of vmapping.
> 
>     - Added ack from Thomas Zimmermann to "drm/shmem-helper: Correct
>       doc-comment of drm_gem_shmem_get_sg_table()" patch.
> 
>     - Dropped explicit shmem states from the generic shrinker patch as
>       was requested by Thomas Zimmermann.
> 
>     - Improved variable names and comments of the generic shrinker code.
> 
>     - Extended drm_gem_shmem_print_info() with the shrinker-state info in
>       the "drm/virtio: Support memory shrinking" patch.
> 
>     - Moved evict()/swap_in()/purge() callbacks from drm_gem_object_funcs
>       to drm_gem_shmem_object in the generic shrinker patch, for more
>       consistency.
> 
>     - Corrected bisectability of the patches that was broken in v4
>       by accident.
> 
>     - The virtio_gpu_plane_prepare_fb() now uses drm_gem_shmem_pin() instead
>       of drm_gem_shmem_set_unpurgeable_and_unevictable() and does it only for
>       shmem BOs in the "drm/virtio: Support memory shrinking" patch.
> 
>     - Made more functions private to drm_gem_shmem_helper.c as was requested
>       by Thomas Zimmermann. This minimizes number of the public shmem helpers.
> 
> v4: - Corrected minor W=1 warnings reported by kernel test robot for v3.
> 
>     - Renamed DRM_GEM_SHMEM_PAGES_STATE_ACTIVE/INACTIVE to PINNED/UNPINNED,
>       for more clarity.
> 
> v3: - Hardened shrinker's count() with usage of READ_ONCE() since we don't
>       use atomic type for counting and technically compiler is free to
>       re-fetch counter's variable.
> 
>     - "Correct drm_gem_shmem_get_sg_table() error handling" now uses
>       PTR_ERR_OR_ZERO(), fixing typo that was made in v2.
> 
>     - Removed obsoleted shrinker from the Panfrost driver, which I missed to
>       do in v2 by accident and Alyssa Rosenzweig managed to notice it.
> 
>     - CCed stable kernels in all patches that make fixes, even the minor ones,
>       like was suggested by Emil Velikov and added his r-b to the patches.
> 
>     - Added t-b from Steven Price to the Panfrost's shrinker patch.
> 
>     - Corrected doc-comment of drm_gem_shmem_object.madv, like was suggested
>       by Steven Price. Comment now says that madv=1 means "object is purged"
>       instead of saying that value is unused.
> 
>     - Added more doc-comments to the new shmem shrinker API.
> 
>     - The "Improve DMA API usage for shmem BOs" patch got more improvements
>       by removing the obsoleted drm_dev_set_unique() quirk and its comment.
> 
>     - Added patch that makes Virtio-GPU driver to use common dev_is_pci()
>       helper, which was suggested by Robin Murphy.
> 
>     - Added new "drm/shmem-helper: Take GEM reservation lock instead of
>       drm_gem_shmem locks" patch, which was suggested by Daniel Vetter.
> 
>     - Added new "drm/virtio: Simplify error handling of
>       virtio_gpu_object_create()" patch.
> 
>     - Improved "Correct doc-comment of drm_gem_shmem_get_sg_table()" patch,
>       like was suggested by Daniel Vetter, by saying that function returns
>       ERR_PTR() and not errno.
> 
>     - virtio_gpu_purge_object() is fenced properly now, turned out
>       virtio_gpu_notify() doesn't do fencing as I was supposing before.
>       Stress testing of memory eviction revealed that.
> 
>     - Added new patch that corrects virtio_gpu_plane_cleanup_fb() to use
>       appropriate atomic plane state.
> 
>     - SHMEM shrinker got eviction support.
> 
>     - VirtIO-GPU driver now supports memory eviction. It's enabled for a
>       non-blob GEMs only, i.e. for VirGL. The blobs don't support dynamic
>       attaching/detaching of guest's memory, so it's not trivial to enable
>       them.
> 
>     - Added patch that removes obsoleted drm_gem_shmem_purge()
> 
>     - Added patch that makes drm_gem_shmem_get_pages() private.
> 
>     - Added patch that fixes lockup on dma_resv_reserve_fences() error.
> 
> v2: - Improved shrinker by using a more fine-grained locking to reduce
>       contention during scan of objects and dropped locking from the
>       'counting' callback by tracking count of shrinkable pages. This
>       was suggested by Rob Clark in the comment to v1.
> 
>     - Factored out common shrinker code into drm_gem_shmem_helper.c
>       and switched Panfrost driver to use the new common memory shrinker.
>       This was proposed by Thomas Zimmermann in his prototype series that
>       he shared with us in the comment to v1. Note that I only compile-tested
>       the Panfrost driver.
> 
>     - Shrinker now takes object_name_lock during scan to prevent racing
>       with dma-buf exporting.
> 
>     - Shrinker now takes vmap_lock during scan to prevent racing with shmem
>       vmap/unmap code.
> 
>     - Added "Correct doc-comment of drm_gem_shmem_get_sg_table()" patch,
>       which I sent out previously as a standalone change, since the
>       drm_gem_shmem_helper.c is now touched by this patchset anyways and
>       it doesn't hurt to group all the patches together.
> 
> Dmitry Osipenko (22):
>   drm/gem: Properly annotate WW context on drm_gem_lock_reservations()
>     error
>   drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()
>   drm/panfrost: Put mapping instead of shmem obj on
>     panfrost_mmu_map_fault_addr() error
>   drm/panfrost: Fix shrinker list corruption by madvise IOCTL
>   drm/virtio: Correct drm_gem_shmem_get_sg_table() error handling
>   drm/virtio: Check whether transferred 2D BO is shmem
>   drm/virtio: Unlock reservations on virtio_gpu_object_shmem_init()
>     error
>   drm/virtio: Unlock reservations on dma_resv_reserve_fences() error
>   drm/virtio: Use appropriate atomic state in
>     virtio_gpu_plane_cleanup_fb()
>   drm/shmem-helper: Add missing vunmap on error
>   drm/shmem-helper: Correct doc-comment of drm_gem_shmem_get_sg_table()
...

Thomas, do you think it will be possible for you to take the fix patches
1-11 into the drm-fixes or would you prefer me to re-send them separately?

The VirtIO patches 12-13 also are good to go into drm-next, IMO.

I'm going to factor out the new dma-buf convention into a separate
patchset, like was suggested by Christian. But it will take me some time
to get the dma-buf patches ready and I also will be on a vacation soon.

At minimum nothing should hold the fixes, so will be great if they could
land sooner.

Thank you!

-- 
Best regards,
Dmitry

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

On Thu, May 26, 2022 at 4:55 PM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> Introduce a common DRM SHMEM shrinker framework that allows to reduce
> code duplication among DRM drivers by replacing theirs custom shrinker
> implementations with the generic shrinker.
>
> In order to start using DRM SHMEM shrinker drivers should:
>
> 1. Implement new evict() shmem object callback.
> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>    activate shrinking of shmem GEMs.
>
> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>
> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>  include/drm/drm_device.h                      |   4 +
>  include/drm/drm_gem_shmem_helper.h            |  87 ++-
>  5 files changed, 594 insertions(+), 49 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 555fe212bd98..4cd0b5913492 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>
> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv >= 0) && shmem->evict &&
> +               shmem->eviction_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> +}
> +
> +static void
> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (!gem_shrinker || obj->import_attach)
> +               return;
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       if (drm_gem_shmem_is_evictable(shmem) ||
> +           drm_gem_shmem_is_purgeable(shmem))
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> +       else if (shmem->madv < 0)
> +               list_del_init(&shmem->madv_list);
> +       else if (shmem->evicted)
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> +       else if (!shmem->pages)
> +               list_del_init(&shmem->madv_list);
> +       else
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +}
> +
>  /**
>   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>   * @shmem: shmem GEM object to free
> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>         } else {
>                 dma_resv_lock(shmem->base.resv, NULL);
>
> +               /* take out shmem GEM object from the memory shrinker */
> +               drm_gem_shmem_madvise(shmem, -1);
> +
>                 WARN_ON(shmem->vmap_use_count);
>
>                 if (shmem->sgt) {
> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>                         sg_free_table(shmem->sgt);
>                         kfree(shmem->sgt);
>                 }
> -               if (shmem->pages)
> +               if (shmem->pages_use_count)
>                         drm_gem_shmem_put_pages(shmem);
>
>                 WARN_ON(shmem->pages_use_count);
> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>
> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->eviction_enabled = true;
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> +
> +/**
> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be purged. Initially purging is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->purge_enabled = true;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> +
> +static int
> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
>         struct page **pages;
>
> -       if (shmem->pages_use_count++ > 0)
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0) {
> +               WARN_ON(shmem->pages);
> +               return -ENOMEM;
> +       }
> +
> +       if (shmem->pages) {
> +               WARN_ON(!shmem->evicted);
>                 return 0;
> +       }
> +
> +       if (WARN_ON(!shmem->pages_use_count))
> +               return -EINVAL;
>
>         pages = drm_gem_get_pages(obj);
>         if (IS_ERR(pages)) {
>                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> -               shmem->pages_use_count = 0;
>                 return PTR_ERR(pages);
>         }
>
> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>         return 0;
>  }
>
> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       int err;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       if (shmem->pages_use_count++ > 0) {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err)
> +                       goto err_zero_use;
> +
> +               return 0;
> +       }
> +
> +       err = drm_gem_shmem_acquire_pages(shmem);
> +       if (err)
> +               goto err_zero_use;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +
> +err_zero_use:
> +       shmem->pages_use_count = 0;
> +
> +       return err;
> +}
> +
> +static void
> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       if (!shmem->pages) {
> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> +               return;
> +       }
> +
> +#ifdef CONFIG_X86
> +       if (shmem->map_wc)
> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> +#endif
> +
> +       drm_gem_put_pages(obj, shmem->pages,
> +                         shmem->pages_mark_dirty_on_put,
> +                         shmem->pages_mark_accessed_on_put);
> +       shmem->pages = NULL;
> +}
> +
>  /*
>   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>   * @shmem: shmem GEM object
> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>   */
>  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>  {
> -       struct drm_gem_object *obj = &shmem->base;
> -
>         dma_resv_assert_held(shmem->base.resv);
>
>         if (WARN_ON_ONCE(!shmem->pages_use_count))
> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>         if (--shmem->pages_use_count > 0)
>                 return;
>
> -#ifdef CONFIG_X86
> -       if (shmem->map_wc)
> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> -#endif
> +       drm_gem_shmem_release_pages(shmem);
>
> -       drm_gem_put_pages(obj, shmem->pages,
> -                         shmem->pages_mark_dirty_on_put,
> -                         shmem->pages_mark_accessed_on_put);
> -       shmem->pages = NULL;
> +       drm_gem_shmem_update_pages_state(shmem);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>
> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>   */
>  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>  {
> +       int ret;
> +
>         dma_resv_assert_held(shmem->base.resv);
>
>         WARN_ON(shmem->base.import_attach);
>
> -       return drm_gem_shmem_get_pages(shmem);
> +       ret = drm_gem_shmem_get_pages(shmem);
> +       if (!ret)
> +               shmem->pages_pin_count++;
> +
> +       return ret;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_pin);
>
> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>         WARN_ON(shmem->base.import_attach);
>
>         drm_gem_shmem_put_pages(shmem);
> +
> +       shmem->pages_pin_count--;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_unpin);
>
> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>                         return 0;
>                 }
>
> -               ret = drm_gem_shmem_get_pages(shmem);
> +               ret = drm_gem_shmem_pin(shmem);
>                 if (ret)
>                         goto err_zero_use;
>
> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>
>  err_put_pages:
>         if (!obj->import_attach)
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>  err_zero_use:
>         shmem->vmap_use_count = 0;
>
> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>                         return;
>
>                 vunmap(shmem->vaddr);
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>         }
>
>         shmem->vaddr = NULL;
> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>
>         madv = shmem->madv;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         return (madv >= 0);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_madvise);
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> + *                           hardware access to the memory.
> + * @shmem: shmem GEM object
> + *
> + * This function moves shmem GEM back to memory if it was previously evicted
> + * by the memory shrinker. The GEM is ready to use on success.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
> -       struct drm_device *dev = obj->dev;
> +       struct sg_table *sgt;
> +       int err;
>
>         dma_resv_assert_held(shmem->base.resv);
>
> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +       if (shmem->evicted) {
> +               err = drm_gem_shmem_acquire_pages(shmem);
> +               if (err)
> +                       return err;
> +
> +               sgt = drm_gem_shmem_get_sg_table(shmem);
> +               if (IS_ERR(sgt))
> +                       return PTR_ERR(sgt);
> +
> +               err = dma_map_sgtable(obj->dev->dev, sgt,
> +                                     DMA_BIDIRECTIONAL, 0);
> +               if (err) {
> +                       sg_free_table(sgt);
> +                       kfree(sgt);
> +                       return err;
> +               }
>
> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> -       sg_free_table(shmem->sgt);
> -       kfree(shmem->sgt);
> -       shmem->sgt = NULL;
> +               shmem->sgt = sgt;
> +               shmem->evicted = false;
>
> -       drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_update_pages_state(shmem);
> +       }
>
> -       shmem->madv = -1;
> +       if (!shmem->pages)
> +               return -ENOMEM;
>
> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> -       drm_gem_free_mmap_offset(obj);
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>
> -       /* Our goal here is to return as much of the memory as
> -        * is possible back to the system as we are called from OOM.
> -        * To do this we must instruct the shmfs to drop all of its
> -        * backing pages, *now*.
> -        */
> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_device *dev = obj->dev;
>
> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +       if (shmem->evicted)
> +               return;
> +
> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> +       drm_gem_shmem_release_pages(shmem);
> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> +
> +       sg_free_table(shmem->sgt);
> +       kfree(shmem->sgt);
> +       shmem->sgt = NULL;
>  }
> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>
>  /**
>   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>         vm_fault_t ret;
>         struct page *page;
>         pgoff_t page_offset;
> +       bool pages_unpinned;
> +       int err;
>
>         /* We don't use vmf->pgoff since that has the fake offset */
>         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>
>         dma_resv_lock(shmem->base.resv, NULL);
>
> -       if (page_offset >= num_pages ||
> -           WARN_ON_ONCE(!shmem->pages) ||
> -           shmem->madv < 0) {
> +       /* Sanity-check that we have the pages pointer when it should present */
> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> +
> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>                 ret = VM_FAULT_SIGBUS;
>         } else {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err) {
> +                       ret = VM_FAULT_OOM;
> +                       goto unlock;
> +               }
> +
>                 page = shmem->pages[page_offset];
>
>                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>         }
>
> +unlock:
>         dma_resv_unlock(shmem->base.resv);
>
>         return ret;
> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>  {
>         struct drm_gem_object *obj = vma->vm_private_data;
>         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -       int ret;
>
>         WARN_ON(shmem->base.import_attach);
>
>         dma_resv_lock(shmem->base.resv, NULL);
> -       ret = drm_gem_shmem_get_pages(shmem);
> -       WARN_ON_ONCE(ret != 0);
> +
> +       if (drm_gem_shmem_get_pages(shmem))
> +               shmem->pages_use_count++;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
>         dma_resv_unlock(shmem->base.resv);
>
>         drm_gem_vm_open(vma);
> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                               struct drm_printer *p, unsigned int indent)
>  {
> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>
>         if (shmem->base.import_attach)
> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>                                   shmem->vmap_use_count);
>
> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_print_info);
>
> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>
>         shmem->sgt = sgt;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         dma_resv_unlock(shmem->base.resv);
>
>         return sgt;
> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>
> +static struct drm_gem_shmem_shrinker *
> +to_drm_shrinker(struct shrinker *shrinker)
> +{
> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> +                                    struct shrink_control *sc)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       unsigned long count = 0;
> +
> +       if (!mutex_trylock(&gem_shrinker->lock))
> +               return 0;
> +
> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> +               count += shmem->base.size;
> +
> +               if (count >= SHRINK_EMPTY)
> +                       break;
> +       }
> +
> +       mutex_unlock(&gem_shrinker->lock);

As I mentioned on other thread, count_objects, being approximate but
lockless and fast is the important thing.  Otherwise when you start
hitting the shrinker on many threads, you end up serializing them all,
even if you have no pages to return to the system at that point.

> +
> +       if (count >= SHRINK_EMPTY)
> +               return SHRINK_EMPTY - 1;
> +
> +       return count ?: SHRINK_EMPTY;
> +}
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> +       WARN_ON(shmem->evicted);
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +
> +       shmem->evicted = true;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> +
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +       drm_gem_free_mmap_offset(obj);
> +
> +       /* Our goal here is to return as much of the memory as
> +        * is possible back to the system as we are called from OOM.
> +        * To do this we must instruct the shmfs to drop all of its
> +        * backing pages, *now*.
> +        */
> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +
> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +
> +       shmem->madv = -1;
> +       shmem->evicted = false;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> +                                       unsigned long nr_to_scan,
> +                                       bool *lock_contention,
> +                                       bool evict)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       struct list_head still_in_list;
> +       struct drm_gem_object *obj;
> +       unsigned long freed = 0;
> +       size_t page_count;
> +       int err;
> +
> +       INIT_LIST_HEAD(&still_in_list);
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       while (freed < nr_to_scan) {
> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> +                                                typeof(*shmem), madv_list);
> +               if (!shmem)
> +                       break;
> +
> +               obj = &shmem->base;
> +               page_count = obj->size >> PAGE_SHIFT;
> +               list_move_tail(&shmem->madv_list, &still_in_list);
> +
> +               if (evict) {
> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> +                           get_nr_swap_pages() < page_count)
> +                               continue;
> +               } else {
> +                       if (!drm_gem_shmem_is_purgeable(shmem))
> +                               continue;
> +               }
> +
> +               /*
> +                * If it's in the process of being freed, gem_object->free()
> +                * may be blocked on lock waiting to remove it.  So just
> +                * skip it.
> +                */
> +               if (!kref_get_unless_zero(&obj->refcount))
> +                       continue;
> +
> +               mutex_unlock(&gem_shrinker->lock);
> +
> +               /* prevent racing with job-submission code paths */
> +               if (!dma_resv_trylock(obj->resv)) {
> +                       *lock_contention |= true;
> +                       goto shrinker_lock;
> +               }
> +
> +               /* prevent racing with the dma-buf importing/exporting */
> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> +                       *lock_contention |= true;
> +                       goto resv_unlock;
> +               }

I'm not sure this is a good idea to serialize on object_name_lock.
Purgeable buffers should never be shared (imported or exported).  So
at best you are avoiding evicting and immediately swapping back in, in
a rare case, at the cost of serializing multiple threads trying to
reclaim pages in parallel.

BR,
-R

> +
> +               /* check whether h/w uses this object */
> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> +                       goto object_name_unlock;
> +
> +               /* re-check whether eviction status hasn't changed */
> +               if (!drm_gem_shmem_is_evictable(shmem) &&
> +                   !drm_gem_shmem_is_purgeable(shmem))
> +                       goto object_name_unlock;
> +
> +               err = shmem->evict(shmem);
> +               if (!err)
> +                       freed += obj->size >> PAGE_SHIFT;
> +
> +object_name_unlock:
> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> +resv_unlock:
> +               dma_resv_unlock(obj->resv);
> +shrinker_lock:
> +               drm_gem_object_put(&shmem->base);
> +               mutex_lock(&gem_shrinker->lock);
> +       }
> +
> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       return freed;
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> +                                   struct shrink_control *sc)
> +{
> +       unsigned long nr_to_scan = sc->nr_to_scan;
> +       bool lock_contention = false;
> +       unsigned long freed;
> +
> +       /* purge as many objects as we can */
> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> +                                                       &lock_contention, false);
> +
> +       /* evict as many objects as we can */
> +       if (freed < nr_to_scan)
> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> +                                                                nr_to_scan - freed,
> +                                                                &lock_contention,
> +                                                                true);
> +
> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> +}
> +
> +/**
> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> + * @dev: DRM device
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker;
> +       int err;
> +
> +       if (WARN_ON(dev->shmem_shrinker))
> +               return -EBUSY;
> +
> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> +       if (!gem_shrinker)
> +               return -ENOMEM;
> +
> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> +       gem_shrinker->dev = dev;
> +
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> +       mutex_init(&gem_shrinker->lock);
> +
> +       dev->shmem_shrinker = gem_shrinker;
> +
> +       err = register_shrinker(&gem_shrinker->base);
> +       if (err) {
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +               return err;
> +       }
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> +
> +/**
> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> + * @dev: DRM device
> + */
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> +
> +       if (gem_shrinker) {
> +               unregister_shrinker(&gem_shrinker->base);
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> +               mutex_destroy(&gem_shrinker->lock);
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +       }
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> +
>  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>  MODULE_IMPORT_NS(DMA_BUF);
>  MODULE_LICENSE("GPL v2");
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> index a4bedfeb2ec4..7cc32556f908 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> @@ -15,6 +15,13 @@
>  #include "panfrost_gem.h"
>  #include "panfrost_mmu.h"
>
> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv > 0) &&
> +               !shmem->pages_pin_count && shmem->sgt &&
> +               !shmem->base.dma_buf && !shmem->base.import_attach;
> +}
> +
>  static unsigned long
>  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>  {
> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>                 return 0;
>
>         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -               if (drm_gem_shmem_is_purgeable(shmem))
> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>                         count += shmem->base.size >> PAGE_SHIFT;
>         }
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> index b2d93cb12ebf..81bacc7e1873 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>         uint32_t hw_res_handle;
>         bool dumb;
>         bool created;
> +       bool detached;
>         bool host3d_blob, guest_blob;
>         uint32_t blob_mem, blob_flags;
>
> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>
>  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>
> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> +
>  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>                                uint32_t *resid);
>  /* virtgpu_prime.c */
> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> index 9923c7a6885e..929546cad894 100644
> --- a/include/drm/drm_device.h
> +++ b/include/drm/drm_device.h
> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>  struct drm_vma_offset_manager;
>  struct drm_vram_mm;
>  struct drm_fb_helper;
> +struct drm_gem_shmem_shrinker;
>
>  struct inode;
>
> @@ -277,6 +278,9 @@ struct drm_device {
>         /** @vram_mm: VRAM MM memory manager */
>         struct drm_vram_mm *vram_mm;
>
> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> +
>         /**
>          * @switch_power_state:
>          *
> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> index 9a8983ee8abe..62c640678a91 100644
> --- a/include/drm/drm_gem_shmem_helper.h
> +++ b/include/drm/drm_gem_shmem_helper.h
> @@ -6,6 +6,7 @@
>  #include <linux/fs.h>
>  #include <linux/mm.h>
>  #include <linux/mutex.h>
> +#include <linux/shrinker.h>
>
>  #include <drm/drm_file.h>
>  #include <drm/drm_gem.h>
> @@ -15,6 +16,7 @@
>  struct dma_buf_attachment;
>  struct drm_mode_create_dumb;
>  struct drm_printer;
> +struct drm_device;
>  struct sg_table;
>
>  /**
> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>          */
>         unsigned int pages_use_count;
>
> +       /**
> +        * @pages_pin_count:
> +        *
> +        * Reference count on the pinned pages table.
> +        * The pages can be evicted by memory shrinker
> +        * when the count reaches zero.
> +        */
> +       unsigned int pages_pin_count;
> +
>         /**
>          * @madv: State for madvise
>          *
>          * 0 is active/inuse.
> +        * 1 is not-needed/can-be-purged
>          * A negative value is the object is purged.
> -        * Positive values are driver specific and not used by the helpers.
>          */
>         int madv;
>
> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>          * @map_wc: map object write-combined (instead of using shmem defaults).
>          */
>         bool map_wc;
> +
> +       /**
> +        * @eviction_enabled:
> +        *
> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool eviction_enabled;
> +
> +       /**
> +        * @purge_enabled:
> +        *
> +        * The shmem pages can be purged only if @purge_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool purge_enabled;
> +
> +       /**
> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> +        * Used internally by memory shrinker.
> +        */
> +       bool evicted;
> +
> +       /**
> +        * @evict:
> +        *
> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> +        * GEM's DMA reservation is kept locked by the shrinker. This is
> +        * optional callback that should be specified by drivers.
> +        *
> +        * Returns 0 on success, or -errno on error.
> +        */
> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>  };
>
>  #define to_drm_gem_shmem_obj(obj) \
> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>
>  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> +
>  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>  {
> -       return (shmem->madv > 0) &&
> -               !shmem->vmap_use_count && shmem->sgt &&
> -               !shmem->base.dma_buf && !shmem->base.import_attach;
> +       return (shmem->madv > 0) && shmem->evict &&
> +               shmem->purge_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>  }
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>
>  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>         return drm_gem_shmem_mmap(shmem, vma);
>  }
>
> +/**
> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> + */
> +struct drm_gem_shmem_shrinker {
> +       /** @base: Shrinker for purging shmem GEM objects */
> +       struct shrinker base;
> +
> +       /** @lock: Protects @lru_* */
> +       struct mutex lock;
> +
> +       /** @lru_pinned: List of pinned shmem GEM objects */
> +       struct list_head lru_pinned;
> +
> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> +       struct list_head lru_evictable;
> +
> +       /** @lru_evicted: List of evicted shmem GEM objects */
> +       struct list_head lru_evicted;
> +
> +       /** @dev: DRM device that uses this shrinker */
> +       struct drm_device *dev;
> +};
> +
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> +
>  /*
>   * Driver ops
>   */
> --
> 2.35.3
>

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

On Thu, May 26, 2022 at 4:55 PM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> Introduce a common DRM SHMEM shrinker framework that allows to reduce
> code duplication among DRM drivers by replacing theirs custom shrinker
> implementations with the generic shrinker.
>
> In order to start using DRM SHMEM shrinker drivers should:
>
> 1. Implement new evict() shmem object callback.
> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>    activate shrinking of shmem GEMs.
>
> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>
> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>  include/drm/drm_device.h                      |   4 +
>  include/drm/drm_gem_shmem_helper.h            |  87 ++-
>  5 files changed, 594 insertions(+), 49 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 555fe212bd98..4cd0b5913492 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>
> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv >= 0) && shmem->evict &&
> +               shmem->eviction_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> +}
> +
> +static void
> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (!gem_shrinker || obj->import_attach)
> +               return;
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       if (drm_gem_shmem_is_evictable(shmem) ||
> +           drm_gem_shmem_is_purgeable(shmem))
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> +       else if (shmem->madv < 0)
> +               list_del_init(&shmem->madv_list);
> +       else if (shmem->evicted)
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> +       else if (!shmem->pages)
> +               list_del_init(&shmem->madv_list);
> +       else
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +}
> +
>  /**
>   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>   * @shmem: shmem GEM object to free
> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>         } else {
>                 dma_resv_lock(shmem->base.resv, NULL);
>
> +               /* take out shmem GEM object from the memory shrinker */
> +               drm_gem_shmem_madvise(shmem, -1);
> +
>                 WARN_ON(shmem->vmap_use_count);
>
>                 if (shmem->sgt) {
> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>                         sg_free_table(shmem->sgt);
>                         kfree(shmem->sgt);
>                 }
> -               if (shmem->pages)
> +               if (shmem->pages_use_count)
>                         drm_gem_shmem_put_pages(shmem);
>
>                 WARN_ON(shmem->pages_use_count);
> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>
> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->eviction_enabled = true;
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> +
> +/**
> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be purged. Initially purging is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->purge_enabled = true;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> +
> +static int
> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
>         struct page **pages;
>
> -       if (shmem->pages_use_count++ > 0)
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0) {
> +               WARN_ON(shmem->pages);
> +               return -ENOMEM;
> +       }
> +
> +       if (shmem->pages) {
> +               WARN_ON(!shmem->evicted);
>                 return 0;
> +       }
> +
> +       if (WARN_ON(!shmem->pages_use_count))
> +               return -EINVAL;
>
>         pages = drm_gem_get_pages(obj);
>         if (IS_ERR(pages)) {
>                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> -               shmem->pages_use_count = 0;
>                 return PTR_ERR(pages);
>         }
>
> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>         return 0;
>  }
>
> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       int err;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       if (shmem->pages_use_count++ > 0) {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err)
> +                       goto err_zero_use;
> +
> +               return 0;
> +       }
> +
> +       err = drm_gem_shmem_acquire_pages(shmem);
> +       if (err)
> +               goto err_zero_use;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +
> +err_zero_use:
> +       shmem->pages_use_count = 0;
> +
> +       return err;
> +}
> +
> +static void
> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       if (!shmem->pages) {
> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> +               return;
> +       }
> +
> +#ifdef CONFIG_X86
> +       if (shmem->map_wc)
> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> +#endif
> +
> +       drm_gem_put_pages(obj, shmem->pages,
> +                         shmem->pages_mark_dirty_on_put,
> +                         shmem->pages_mark_accessed_on_put);
> +       shmem->pages = NULL;
> +}
> +
>  /*
>   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>   * @shmem: shmem GEM object
> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>   */
>  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>  {
> -       struct drm_gem_object *obj = &shmem->base;
> -
>         dma_resv_assert_held(shmem->base.resv);
>
>         if (WARN_ON_ONCE(!shmem->pages_use_count))
> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>         if (--shmem->pages_use_count > 0)
>                 return;
>
> -#ifdef CONFIG_X86
> -       if (shmem->map_wc)
> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> -#endif
> +       drm_gem_shmem_release_pages(shmem);
>
> -       drm_gem_put_pages(obj, shmem->pages,
> -                         shmem->pages_mark_dirty_on_put,
> -                         shmem->pages_mark_accessed_on_put);
> -       shmem->pages = NULL;
> +       drm_gem_shmem_update_pages_state(shmem);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>
> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>   */
>  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>  {
> +       int ret;
> +
>         dma_resv_assert_held(shmem->base.resv);
>
>         WARN_ON(shmem->base.import_attach);
>
> -       return drm_gem_shmem_get_pages(shmem);
> +       ret = drm_gem_shmem_get_pages(shmem);
> +       if (!ret)
> +               shmem->pages_pin_count++;
> +
> +       return ret;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_pin);
>
> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>         WARN_ON(shmem->base.import_attach);
>
>         drm_gem_shmem_put_pages(shmem);
> +
> +       shmem->pages_pin_count--;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_unpin);
>
> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>                         return 0;
>                 }
>
> -               ret = drm_gem_shmem_get_pages(shmem);
> +               ret = drm_gem_shmem_pin(shmem);
>                 if (ret)
>                         goto err_zero_use;
>
> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>
>  err_put_pages:
>         if (!obj->import_attach)
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>  err_zero_use:
>         shmem->vmap_use_count = 0;
>
> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>                         return;
>
>                 vunmap(shmem->vaddr);
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>         }
>
>         shmem->vaddr = NULL;
> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>
>         madv = shmem->madv;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         return (madv >= 0);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_madvise);
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> + *                           hardware access to the memory.
> + * @shmem: shmem GEM object
> + *
> + * This function moves shmem GEM back to memory if it was previously evicted
> + * by the memory shrinker. The GEM is ready to use on success.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
> -       struct drm_device *dev = obj->dev;
> +       struct sg_table *sgt;
> +       int err;
>
>         dma_resv_assert_held(shmem->base.resv);
>
> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +       if (shmem->evicted) {
> +               err = drm_gem_shmem_acquire_pages(shmem);
> +               if (err)
> +                       return err;
> +
> +               sgt = drm_gem_shmem_get_sg_table(shmem);
> +               if (IS_ERR(sgt))
> +                       return PTR_ERR(sgt);
> +
> +               err = dma_map_sgtable(obj->dev->dev, sgt,
> +                                     DMA_BIDIRECTIONAL, 0);
> +               if (err) {
> +                       sg_free_table(sgt);
> +                       kfree(sgt);
> +                       return err;
> +               }
>
> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> -       sg_free_table(shmem->sgt);
> -       kfree(shmem->sgt);
> -       shmem->sgt = NULL;
> +               shmem->sgt = sgt;
> +               shmem->evicted = false;
>
> -       drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_update_pages_state(shmem);
> +       }
>
> -       shmem->madv = -1;
> +       if (!shmem->pages)
> +               return -ENOMEM;
>
> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> -       drm_gem_free_mmap_offset(obj);
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>
> -       /* Our goal here is to return as much of the memory as
> -        * is possible back to the system as we are called from OOM.
> -        * To do this we must instruct the shmfs to drop all of its
> -        * backing pages, *now*.
> -        */
> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_device *dev = obj->dev;
>
> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +       if (shmem->evicted)
> +               return;
> +
> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> +       drm_gem_shmem_release_pages(shmem);
> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> +
> +       sg_free_table(shmem->sgt);
> +       kfree(shmem->sgt);
> +       shmem->sgt = NULL;
>  }
> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>
>  /**
>   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>         vm_fault_t ret;
>         struct page *page;
>         pgoff_t page_offset;
> +       bool pages_unpinned;
> +       int err;
>
>         /* We don't use vmf->pgoff since that has the fake offset */
>         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>
>         dma_resv_lock(shmem->base.resv, NULL);
>
> -       if (page_offset >= num_pages ||
> -           WARN_ON_ONCE(!shmem->pages) ||
> -           shmem->madv < 0) {
> +       /* Sanity-check that we have the pages pointer when it should present */
> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> +
> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>                 ret = VM_FAULT_SIGBUS;
>         } else {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err) {
> +                       ret = VM_FAULT_OOM;
> +                       goto unlock;
> +               }
> +
>                 page = shmem->pages[page_offset];
>
>                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>         }
>
> +unlock:
>         dma_resv_unlock(shmem->base.resv);
>
>         return ret;
> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>  {
>         struct drm_gem_object *obj = vma->vm_private_data;
>         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -       int ret;
>
>         WARN_ON(shmem->base.import_attach);
>
>         dma_resv_lock(shmem->base.resv, NULL);
> -       ret = drm_gem_shmem_get_pages(shmem);
> -       WARN_ON_ONCE(ret != 0);
> +
> +       if (drm_gem_shmem_get_pages(shmem))
> +               shmem->pages_use_count++;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
>         dma_resv_unlock(shmem->base.resv);
>
>         drm_gem_vm_open(vma);
> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                               struct drm_printer *p, unsigned int indent)
>  {
> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>
>         if (shmem->base.import_attach)
> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>                                   shmem->vmap_use_count);
>
> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_print_info);
>
> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>
>         shmem->sgt = sgt;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         dma_resv_unlock(shmem->base.resv);
>
>         return sgt;
> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>
> +static struct drm_gem_shmem_shrinker *
> +to_drm_shrinker(struct shrinker *shrinker)
> +{
> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> +                                    struct shrink_control *sc)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       unsigned long count = 0;
> +
> +       if (!mutex_trylock(&gem_shrinker->lock))
> +               return 0;
> +
> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> +               count += shmem->base.size;
> +
> +               if (count >= SHRINK_EMPTY)
> +                       break;
> +       }
> +
> +       mutex_unlock(&gem_shrinker->lock);

As I mentioned on other thread, count_objects, being approximate but
lockless and fast is the important thing.  Otherwise when you start
hitting the shrinker on many threads, you end up serializing them all,
even if you have no pages to return to the system at that point.

> +
> +       if (count >= SHRINK_EMPTY)
> +               return SHRINK_EMPTY - 1;
> +
> +       return count ?: SHRINK_EMPTY;
> +}
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> +       WARN_ON(shmem->evicted);
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +
> +       shmem->evicted = true;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> +
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +       drm_gem_free_mmap_offset(obj);
> +
> +       /* Our goal here is to return as much of the memory as
> +        * is possible back to the system as we are called from OOM.
> +        * To do this we must instruct the shmfs to drop all of its
> +        * backing pages, *now*.
> +        */
> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +
> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +
> +       shmem->madv = -1;
> +       shmem->evicted = false;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> +                                       unsigned long nr_to_scan,
> +                                       bool *lock_contention,
> +                                       bool evict)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       struct list_head still_in_list;
> +       struct drm_gem_object *obj;
> +       unsigned long freed = 0;
> +       size_t page_count;
> +       int err;
> +
> +       INIT_LIST_HEAD(&still_in_list);
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       while (freed < nr_to_scan) {
> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> +                                                typeof(*shmem), madv_list);
> +               if (!shmem)
> +                       break;
> +
> +               obj = &shmem->base;
> +               page_count = obj->size >> PAGE_SHIFT;
> +               list_move_tail(&shmem->madv_list, &still_in_list);
> +
> +               if (evict) {
> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> +                           get_nr_swap_pages() < page_count)
> +                               continue;
> +               } else {
> +                       if (!drm_gem_shmem_is_purgeable(shmem))
> +                               continue;
> +               }
> +
> +               /*
> +                * If it's in the process of being freed, gem_object->free()
> +                * may be blocked on lock waiting to remove it.  So just
> +                * skip it.
> +                */
> +               if (!kref_get_unless_zero(&obj->refcount))
> +                       continue;
> +
> +               mutex_unlock(&gem_shrinker->lock);
> +
> +               /* prevent racing with job-submission code paths */
> +               if (!dma_resv_trylock(obj->resv)) {
> +                       *lock_contention |= true;
> +                       goto shrinker_lock;
> +               }
> +
> +               /* prevent racing with the dma-buf importing/exporting */
> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> +                       *lock_contention |= true;
> +                       goto resv_unlock;
> +               }

I'm not sure this is a good idea to serialize on object_name_lock.
Purgeable buffers should never be shared (imported or exported).  So
at best you are avoiding evicting and immediately swapping back in, in
a rare case, at the cost of serializing multiple threads trying to
reclaim pages in parallel.

BR,
-R

> +
> +               /* check whether h/w uses this object */
> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> +                       goto object_name_unlock;
> +
> +               /* re-check whether eviction status hasn't changed */
> +               if (!drm_gem_shmem_is_evictable(shmem) &&
> +                   !drm_gem_shmem_is_purgeable(shmem))
> +                       goto object_name_unlock;
> +
> +               err = shmem->evict(shmem);
> +               if (!err)
> +                       freed += obj->size >> PAGE_SHIFT;
> +
> +object_name_unlock:
> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> +resv_unlock:
> +               dma_resv_unlock(obj->resv);
> +shrinker_lock:
> +               drm_gem_object_put(&shmem->base);
> +               mutex_lock(&gem_shrinker->lock);
> +       }
> +
> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       return freed;
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> +                                   struct shrink_control *sc)
> +{
> +       unsigned long nr_to_scan = sc->nr_to_scan;
> +       bool lock_contention = false;
> +       unsigned long freed;
> +
> +       /* purge as many objects as we can */
> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> +                                                       &lock_contention, false);
> +
> +       /* evict as many objects as we can */
> +       if (freed < nr_to_scan)
> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> +                                                                nr_to_scan - freed,
> +                                                                &lock_contention,
> +                                                                true);
> +
> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> +}
> +
> +/**
> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> + * @dev: DRM device
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker;
> +       int err;
> +
> +       if (WARN_ON(dev->shmem_shrinker))
> +               return -EBUSY;
> +
> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> +       if (!gem_shrinker)
> +               return -ENOMEM;
> +
> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> +       gem_shrinker->dev = dev;
> +
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> +       mutex_init(&gem_shrinker->lock);
> +
> +       dev->shmem_shrinker = gem_shrinker;
> +
> +       err = register_shrinker(&gem_shrinker->base);
> +       if (err) {
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +               return err;
> +       }
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> +
> +/**
> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> + * @dev: DRM device
> + */
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> +
> +       if (gem_shrinker) {
> +               unregister_shrinker(&gem_shrinker->base);
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> +               mutex_destroy(&gem_shrinker->lock);
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +       }
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> +
>  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>  MODULE_IMPORT_NS(DMA_BUF);
>  MODULE_LICENSE("GPL v2");
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> index a4bedfeb2ec4..7cc32556f908 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> @@ -15,6 +15,13 @@
>  #include "panfrost_gem.h"
>  #include "panfrost_mmu.h"
>
> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv > 0) &&
> +               !shmem->pages_pin_count && shmem->sgt &&
> +               !shmem->base.dma_buf && !shmem->base.import_attach;
> +}
> +
>  static unsigned long
>  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>  {
> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>                 return 0;
>
>         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -               if (drm_gem_shmem_is_purgeable(shmem))
> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>                         count += shmem->base.size >> PAGE_SHIFT;
>         }
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> index b2d93cb12ebf..81bacc7e1873 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>         uint32_t hw_res_handle;
>         bool dumb;
>         bool created;
> +       bool detached;
>         bool host3d_blob, guest_blob;
>         uint32_t blob_mem, blob_flags;
>
> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>
>  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>
> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> +
>  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>                                uint32_t *resid);
>  /* virtgpu_prime.c */
> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> index 9923c7a6885e..929546cad894 100644
> --- a/include/drm/drm_device.h
> +++ b/include/drm/drm_device.h
> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>  struct drm_vma_offset_manager;
>  struct drm_vram_mm;
>  struct drm_fb_helper;
> +struct drm_gem_shmem_shrinker;
>
>  struct inode;
>
> @@ -277,6 +278,9 @@ struct drm_device {
>         /** @vram_mm: VRAM MM memory manager */
>         struct drm_vram_mm *vram_mm;
>
> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> +
>         /**
>          * @switch_power_state:
>          *
> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> index 9a8983ee8abe..62c640678a91 100644
> --- a/include/drm/drm_gem_shmem_helper.h
> +++ b/include/drm/drm_gem_shmem_helper.h
> @@ -6,6 +6,7 @@
>  #include <linux/fs.h>
>  #include <linux/mm.h>
>  #include <linux/mutex.h>
> +#include <linux/shrinker.h>
>
>  #include <drm/drm_file.h>
>  #include <drm/drm_gem.h>
> @@ -15,6 +16,7 @@
>  struct dma_buf_attachment;
>  struct drm_mode_create_dumb;
>  struct drm_printer;
> +struct drm_device;
>  struct sg_table;
>
>  /**
> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>          */
>         unsigned int pages_use_count;
>
> +       /**
> +        * @pages_pin_count:
> +        *
> +        * Reference count on the pinned pages table.
> +        * The pages can be evicted by memory shrinker
> +        * when the count reaches zero.
> +        */
> +       unsigned int pages_pin_count;
> +
>         /**
>          * @madv: State for madvise
>          *
>          * 0 is active/inuse.
> +        * 1 is not-needed/can-be-purged
>          * A negative value is the object is purged.
> -        * Positive values are driver specific and not used by the helpers.
>          */
>         int madv;
>
> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>          * @map_wc: map object write-combined (instead of using shmem defaults).
>          */
>         bool map_wc;
> +
> +       /**
> +        * @eviction_enabled:
> +        *
> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool eviction_enabled;
> +
> +       /**
> +        * @purge_enabled:
> +        *
> +        * The shmem pages can be purged only if @purge_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool purge_enabled;
> +
> +       /**
> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> +        * Used internally by memory shrinker.
> +        */
> +       bool evicted;
> +
> +       /**
> +        * @evict:
> +        *
> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> +        * GEM's DMA reservation is kept locked by the shrinker. This is
> +        * optional callback that should be specified by drivers.
> +        *
> +        * Returns 0 on success, or -errno on error.
> +        */
> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>  };
>
>  #define to_drm_gem_shmem_obj(obj) \
> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>
>  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> +
>  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>  {
> -       return (shmem->madv > 0) &&
> -               !shmem->vmap_use_count && shmem->sgt &&
> -               !shmem->base.dma_buf && !shmem->base.import_attach;
> +       return (shmem->madv > 0) && shmem->evict &&
> +               shmem->purge_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>  }
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>
>  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>         return drm_gem_shmem_mmap(shmem, vma);
>  }
>
> +/**
> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> + */
> +struct drm_gem_shmem_shrinker {
> +       /** @base: Shrinker for purging shmem GEM objects */
> +       struct shrinker base;
> +
> +       /** @lock: Protects @lru_* */
> +       struct mutex lock;
> +
> +       /** @lru_pinned: List of pinned shmem GEM objects */
> +       struct list_head lru_pinned;
> +
> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> +       struct list_head lru_evictable;
> +
> +       /** @lru_evicted: List of evicted shmem GEM objects */
> +       struct list_head lru_evicted;
> +
> +       /** @dev: DRM device that uses this shrinker */
> +       struct drm_device *dev;
> +};
> +
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> +
>  /*
>   * Driver ops
>   */
> --
> 2.35.3
>

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

On Thu, May 26, 2022 at 4:55 PM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> Introduce a common DRM SHMEM shrinker framework that allows to reduce
> code duplication among DRM drivers by replacing theirs custom shrinker
> implementations with the generic shrinker.
>
> In order to start using DRM SHMEM shrinker drivers should:
>
> 1. Implement new evict() shmem object callback.
> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>    activate shrinking of shmem GEMs.
>
> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>
> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>  include/drm/drm_device.h                      |   4 +
>  include/drm/drm_gem_shmem_helper.h            |  87 ++-
>  5 files changed, 594 insertions(+), 49 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 555fe212bd98..4cd0b5913492 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>
> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv >= 0) && shmem->evict &&
> +               shmem->eviction_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> +}
> +
> +static void
> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (!gem_shrinker || obj->import_attach)
> +               return;
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       if (drm_gem_shmem_is_evictable(shmem) ||
> +           drm_gem_shmem_is_purgeable(shmem))
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> +       else if (shmem->madv < 0)
> +               list_del_init(&shmem->madv_list);
> +       else if (shmem->evicted)
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> +       else if (!shmem->pages)
> +               list_del_init(&shmem->madv_list);
> +       else
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +}
> +
>  /**
>   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>   * @shmem: shmem GEM object to free
> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>         } else {
>                 dma_resv_lock(shmem->base.resv, NULL);
>
> +               /* take out shmem GEM object from the memory shrinker */
> +               drm_gem_shmem_madvise(shmem, -1);
> +
>                 WARN_ON(shmem->vmap_use_count);
>
>                 if (shmem->sgt) {
> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>                         sg_free_table(shmem->sgt);
>                         kfree(shmem->sgt);
>                 }
> -               if (shmem->pages)
> +               if (shmem->pages_use_count)
>                         drm_gem_shmem_put_pages(shmem);
>
>                 WARN_ON(shmem->pages_use_count);
> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>
> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->eviction_enabled = true;
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> +
> +/**
> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be purged. Initially purging is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->purge_enabled = true;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> +
> +static int
> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
>         struct page **pages;
>
> -       if (shmem->pages_use_count++ > 0)
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0) {
> +               WARN_ON(shmem->pages);
> +               return -ENOMEM;
> +       }
> +
> +       if (shmem->pages) {
> +               WARN_ON(!shmem->evicted);
>                 return 0;
> +       }
> +
> +       if (WARN_ON(!shmem->pages_use_count))
> +               return -EINVAL;
>
>         pages = drm_gem_get_pages(obj);
>         if (IS_ERR(pages)) {
>                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> -               shmem->pages_use_count = 0;
>                 return PTR_ERR(pages);
>         }
>
> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>         return 0;
>  }
>
> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       int err;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       if (shmem->pages_use_count++ > 0) {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err)
> +                       goto err_zero_use;
> +
> +               return 0;
> +       }
> +
> +       err = drm_gem_shmem_acquire_pages(shmem);
> +       if (err)
> +               goto err_zero_use;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +
> +err_zero_use:
> +       shmem->pages_use_count = 0;
> +
> +       return err;
> +}
> +
> +static void
> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       if (!shmem->pages) {
> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> +               return;
> +       }
> +
> +#ifdef CONFIG_X86
> +       if (shmem->map_wc)
> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> +#endif
> +
> +       drm_gem_put_pages(obj, shmem->pages,
> +                         shmem->pages_mark_dirty_on_put,
> +                         shmem->pages_mark_accessed_on_put);
> +       shmem->pages = NULL;
> +}
> +
>  /*
>   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>   * @shmem: shmem GEM object
> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>   */
>  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>  {
> -       struct drm_gem_object *obj = &shmem->base;
> -
>         dma_resv_assert_held(shmem->base.resv);
>
>         if (WARN_ON_ONCE(!shmem->pages_use_count))
> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>         if (--shmem->pages_use_count > 0)
>                 return;
>
> -#ifdef CONFIG_X86
> -       if (shmem->map_wc)
> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> -#endif
> +       drm_gem_shmem_release_pages(shmem);
>
> -       drm_gem_put_pages(obj, shmem->pages,
> -                         shmem->pages_mark_dirty_on_put,
> -                         shmem->pages_mark_accessed_on_put);
> -       shmem->pages = NULL;
> +       drm_gem_shmem_update_pages_state(shmem);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>
> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>   */
>  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>  {
> +       int ret;
> +
>         dma_resv_assert_held(shmem->base.resv);
>
>         WARN_ON(shmem->base.import_attach);
>
> -       return drm_gem_shmem_get_pages(shmem);
> +       ret = drm_gem_shmem_get_pages(shmem);
> +       if (!ret)
> +               shmem->pages_pin_count++;
> +
> +       return ret;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_pin);
>
> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>         WARN_ON(shmem->base.import_attach);
>
>         drm_gem_shmem_put_pages(shmem);
> +
> +       shmem->pages_pin_count--;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_unpin);
>
> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>                         return 0;
>                 }
>
> -               ret = drm_gem_shmem_get_pages(shmem);
> +               ret = drm_gem_shmem_pin(shmem);
>                 if (ret)
>                         goto err_zero_use;
>
> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>
>  err_put_pages:
>         if (!obj->import_attach)
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>  err_zero_use:
>         shmem->vmap_use_count = 0;
>
> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>                         return;
>
>                 vunmap(shmem->vaddr);
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>         }
>
>         shmem->vaddr = NULL;
> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>
>         madv = shmem->madv;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         return (madv >= 0);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_madvise);
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> + *                           hardware access to the memory.
> + * @shmem: shmem GEM object
> + *
> + * This function moves shmem GEM back to memory if it was previously evicted
> + * by the memory shrinker. The GEM is ready to use on success.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
> -       struct drm_device *dev = obj->dev;
> +       struct sg_table *sgt;
> +       int err;
>
>         dma_resv_assert_held(shmem->base.resv);
>
> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +       if (shmem->evicted) {
> +               err = drm_gem_shmem_acquire_pages(shmem);
> +               if (err)
> +                       return err;
> +
> +               sgt = drm_gem_shmem_get_sg_table(shmem);
> +               if (IS_ERR(sgt))
> +                       return PTR_ERR(sgt);
> +
> +               err = dma_map_sgtable(obj->dev->dev, sgt,
> +                                     DMA_BIDIRECTIONAL, 0);
> +               if (err) {
> +                       sg_free_table(sgt);
> +                       kfree(sgt);
> +                       return err;
> +               }
>
> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> -       sg_free_table(shmem->sgt);
> -       kfree(shmem->sgt);
> -       shmem->sgt = NULL;
> +               shmem->sgt = sgt;
> +               shmem->evicted = false;
>
> -       drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_update_pages_state(shmem);
> +       }
>
> -       shmem->madv = -1;
> +       if (!shmem->pages)
> +               return -ENOMEM;
>
> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> -       drm_gem_free_mmap_offset(obj);
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>
> -       /* Our goal here is to return as much of the memory as
> -        * is possible back to the system as we are called from OOM.
> -        * To do this we must instruct the shmfs to drop all of its
> -        * backing pages, *now*.
> -        */
> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_device *dev = obj->dev;
>
> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +       if (shmem->evicted)
> +               return;
> +
> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> +       drm_gem_shmem_release_pages(shmem);
> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> +
> +       sg_free_table(shmem->sgt);
> +       kfree(shmem->sgt);
> +       shmem->sgt = NULL;
>  }
> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>
>  /**
>   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>         vm_fault_t ret;
>         struct page *page;
>         pgoff_t page_offset;
> +       bool pages_unpinned;
> +       int err;
>
>         /* We don't use vmf->pgoff since that has the fake offset */
>         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>
>         dma_resv_lock(shmem->base.resv, NULL);
>
> -       if (page_offset >= num_pages ||
> -           WARN_ON_ONCE(!shmem->pages) ||
> -           shmem->madv < 0) {
> +       /* Sanity-check that we have the pages pointer when it should present */
> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> +
> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>                 ret = VM_FAULT_SIGBUS;
>         } else {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err) {
> +                       ret = VM_FAULT_OOM;
> +                       goto unlock;
> +               }
> +
>                 page = shmem->pages[page_offset];
>
>                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>         }
>
> +unlock:
>         dma_resv_unlock(shmem->base.resv);
>
>         return ret;
> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>  {
>         struct drm_gem_object *obj = vma->vm_private_data;
>         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -       int ret;
>
>         WARN_ON(shmem->base.import_attach);
>
>         dma_resv_lock(shmem->base.resv, NULL);
> -       ret = drm_gem_shmem_get_pages(shmem);
> -       WARN_ON_ONCE(ret != 0);
> +
> +       if (drm_gem_shmem_get_pages(shmem))
> +               shmem->pages_use_count++;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
>         dma_resv_unlock(shmem->base.resv);
>
>         drm_gem_vm_open(vma);
> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                               struct drm_printer *p, unsigned int indent)
>  {
> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>
>         if (shmem->base.import_attach)
> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>                                   shmem->vmap_use_count);
>
> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_print_info);
>
> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>
>         shmem->sgt = sgt;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         dma_resv_unlock(shmem->base.resv);
>
>         return sgt;
> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>
> +static struct drm_gem_shmem_shrinker *
> +to_drm_shrinker(struct shrinker *shrinker)
> +{
> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> +                                    struct shrink_control *sc)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       unsigned long count = 0;
> +
> +       if (!mutex_trylock(&gem_shrinker->lock))
> +               return 0;
> +
> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> +               count += shmem->base.size;
> +
> +               if (count >= SHRINK_EMPTY)
> +                       break;
> +       }
> +
> +       mutex_unlock(&gem_shrinker->lock);

As I mentioned on other thread, count_objects, being approximate but
lockless and fast is the important thing.  Otherwise when you start
hitting the shrinker on many threads, you end up serializing them all,
even if you have no pages to return to the system at that point.

> +
> +       if (count >= SHRINK_EMPTY)
> +               return SHRINK_EMPTY - 1;
> +
> +       return count ?: SHRINK_EMPTY;
> +}
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> +       WARN_ON(shmem->evicted);
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +
> +       shmem->evicted = true;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> +
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +       drm_gem_free_mmap_offset(obj);
> +
> +       /* Our goal here is to return as much of the memory as
> +        * is possible back to the system as we are called from OOM.
> +        * To do this we must instruct the shmfs to drop all of its
> +        * backing pages, *now*.
> +        */
> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +
> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +
> +       shmem->madv = -1;
> +       shmem->evicted = false;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> +                                       unsigned long nr_to_scan,
> +                                       bool *lock_contention,
> +                                       bool evict)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       struct list_head still_in_list;
> +       struct drm_gem_object *obj;
> +       unsigned long freed = 0;
> +       size_t page_count;
> +       int err;
> +
> +       INIT_LIST_HEAD(&still_in_list);
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       while (freed < nr_to_scan) {
> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> +                                                typeof(*shmem), madv_list);
> +               if (!shmem)
> +                       break;
> +
> +               obj = &shmem->base;
> +               page_count = obj->size >> PAGE_SHIFT;
> +               list_move_tail(&shmem->madv_list, &still_in_list);
> +
> +               if (evict) {
> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> +                           get_nr_swap_pages() < page_count)
> +                               continue;
> +               } else {
> +                       if (!drm_gem_shmem_is_purgeable(shmem))
> +                               continue;
> +               }
> +
> +               /*
> +                * If it's in the process of being freed, gem_object->free()
> +                * may be blocked on lock waiting to remove it.  So just
> +                * skip it.
> +                */
> +               if (!kref_get_unless_zero(&obj->refcount))
> +                       continue;
> +
> +               mutex_unlock(&gem_shrinker->lock);
> +
> +               /* prevent racing with job-submission code paths */
> +               if (!dma_resv_trylock(obj->resv)) {
> +                       *lock_contention |= true;
> +                       goto shrinker_lock;
> +               }
> +
> +               /* prevent racing with the dma-buf importing/exporting */
> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> +                       *lock_contention |= true;
> +                       goto resv_unlock;
> +               }

I'm not sure this is a good idea to serialize on object_name_lock.
Purgeable buffers should never be shared (imported or exported).  So
at best you are avoiding evicting and immediately swapping back in, in
a rare case, at the cost of serializing multiple threads trying to
reclaim pages in parallel.

BR,
-R

> +
> +               /* check whether h/w uses this object */
> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> +                       goto object_name_unlock;
> +
> +               /* re-check whether eviction status hasn't changed */
> +               if (!drm_gem_shmem_is_evictable(shmem) &&
> +                   !drm_gem_shmem_is_purgeable(shmem))
> +                       goto object_name_unlock;
> +
> +               err = shmem->evict(shmem);
> +               if (!err)
> +                       freed += obj->size >> PAGE_SHIFT;
> +
> +object_name_unlock:
> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> +resv_unlock:
> +               dma_resv_unlock(obj->resv);
> +shrinker_lock:
> +               drm_gem_object_put(&shmem->base);
> +               mutex_lock(&gem_shrinker->lock);
> +       }
> +
> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       return freed;
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> +                                   struct shrink_control *sc)
> +{
> +       unsigned long nr_to_scan = sc->nr_to_scan;
> +       bool lock_contention = false;
> +       unsigned long freed;
> +
> +       /* purge as many objects as we can */
> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> +                                                       &lock_contention, false);
> +
> +       /* evict as many objects as we can */
> +       if (freed < nr_to_scan)
> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> +                                                                nr_to_scan - freed,
> +                                                                &lock_contention,
> +                                                                true);
> +
> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> +}
> +
> +/**
> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> + * @dev: DRM device
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker;
> +       int err;
> +
> +       if (WARN_ON(dev->shmem_shrinker))
> +               return -EBUSY;
> +
> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> +       if (!gem_shrinker)
> +               return -ENOMEM;
> +
> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> +       gem_shrinker->dev = dev;
> +
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> +       mutex_init(&gem_shrinker->lock);
> +
> +       dev->shmem_shrinker = gem_shrinker;
> +
> +       err = register_shrinker(&gem_shrinker->base);
> +       if (err) {
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +               return err;
> +       }
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> +
> +/**
> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> + * @dev: DRM device
> + */
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> +
> +       if (gem_shrinker) {
> +               unregister_shrinker(&gem_shrinker->base);
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> +               mutex_destroy(&gem_shrinker->lock);
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +       }
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> +
>  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>  MODULE_IMPORT_NS(DMA_BUF);
>  MODULE_LICENSE("GPL v2");
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> index a4bedfeb2ec4..7cc32556f908 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> @@ -15,6 +15,13 @@
>  #include "panfrost_gem.h"
>  #include "panfrost_mmu.h"
>
> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv > 0) &&
> +               !shmem->pages_pin_count && shmem->sgt &&
> +               !shmem->base.dma_buf && !shmem->base.import_attach;
> +}
> +
>  static unsigned long
>  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>  {
> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>                 return 0;
>
>         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -               if (drm_gem_shmem_is_purgeable(shmem))
> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>                         count += shmem->base.size >> PAGE_SHIFT;
>         }
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> index b2d93cb12ebf..81bacc7e1873 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>         uint32_t hw_res_handle;
>         bool dumb;
>         bool created;
> +       bool detached;
>         bool host3d_blob, guest_blob;
>         uint32_t blob_mem, blob_flags;
>
> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>
>  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>
> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> +
>  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>                                uint32_t *resid);
>  /* virtgpu_prime.c */
> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> index 9923c7a6885e..929546cad894 100644
> --- a/include/drm/drm_device.h
> +++ b/include/drm/drm_device.h
> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>  struct drm_vma_offset_manager;
>  struct drm_vram_mm;
>  struct drm_fb_helper;
> +struct drm_gem_shmem_shrinker;
>
>  struct inode;
>
> @@ -277,6 +278,9 @@ struct drm_device {
>         /** @vram_mm: VRAM MM memory manager */
>         struct drm_vram_mm *vram_mm;
>
> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> +
>         /**
>          * @switch_power_state:
>          *
> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> index 9a8983ee8abe..62c640678a91 100644
> --- a/include/drm/drm_gem_shmem_helper.h
> +++ b/include/drm/drm_gem_shmem_helper.h
> @@ -6,6 +6,7 @@
>  #include <linux/fs.h>
>  #include <linux/mm.h>
>  #include <linux/mutex.h>
> +#include <linux/shrinker.h>
>
>  #include <drm/drm_file.h>
>  #include <drm/drm_gem.h>
> @@ -15,6 +16,7 @@
>  struct dma_buf_attachment;
>  struct drm_mode_create_dumb;
>  struct drm_printer;
> +struct drm_device;
>  struct sg_table;
>
>  /**
> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>          */
>         unsigned int pages_use_count;
>
> +       /**
> +        * @pages_pin_count:
> +        *
> +        * Reference count on the pinned pages table.
> +        * The pages can be evicted by memory shrinker
> +        * when the count reaches zero.
> +        */
> +       unsigned int pages_pin_count;
> +
>         /**
>          * @madv: State for madvise
>          *
>          * 0 is active/inuse.
> +        * 1 is not-needed/can-be-purged
>          * A negative value is the object is purged.
> -        * Positive values are driver specific and not used by the helpers.
>          */
>         int madv;
>
> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>          * @map_wc: map object write-combined (instead of using shmem defaults).
>          */
>         bool map_wc;
> +
> +       /**
> +        * @eviction_enabled:
> +        *
> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool eviction_enabled;
> +
> +       /**
> +        * @purge_enabled:
> +        *
> +        * The shmem pages can be purged only if @purge_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool purge_enabled;
> +
> +       /**
> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> +        * Used internally by memory shrinker.
> +        */
> +       bool evicted;
> +
> +       /**
> +        * @evict:
> +        *
> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> +        * GEM's DMA reservation is kept locked by the shrinker. This is
> +        * optional callback that should be specified by drivers.
> +        *
> +        * Returns 0 on success, or -errno on error.
> +        */
> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>  };
>
>  #define to_drm_gem_shmem_obj(obj) \
> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>
>  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> +
>  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>  {
> -       return (shmem->madv > 0) &&
> -               !shmem->vmap_use_count && shmem->sgt &&
> -               !shmem->base.dma_buf && !shmem->base.import_attach;
> +       return (shmem->madv > 0) && shmem->evict &&
> +               shmem->purge_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>  }
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>
>  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>         return drm_gem_shmem_mmap(shmem, vma);
>  }
>
> +/**
> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> + */
> +struct drm_gem_shmem_shrinker {
> +       /** @base: Shrinker for purging shmem GEM objects */
> +       struct shrinker base;
> +
> +       /** @lock: Protects @lru_* */
> +       struct mutex lock;
> +
> +       /** @lru_pinned: List of pinned shmem GEM objects */
> +       struct list_head lru_pinned;
> +
> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> +       struct list_head lru_evictable;
> +
> +       /** @lru_evicted: List of evicted shmem GEM objects */
> +       struct list_head lru_evicted;
> +
> +       /** @dev: DRM device that uses this shrinker */
> +       struct drm_device *dev;
> +};
> +
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> +
>  /*
>   * Driver ops
>   */
> --
> 2.35.3
>

Re: [Intel-gfx] [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

On Thu, May 26, 2022 at 4:55 PM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> Introduce a common DRM SHMEM shrinker framework that allows to reduce
> code duplication among DRM drivers by replacing theirs custom shrinker
> implementations with the generic shrinker.
>
> In order to start using DRM SHMEM shrinker drivers should:
>
> 1. Implement new evict() shmem object callback.
> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>    activate shrinking of shmem GEMs.
>
> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>
> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>  include/drm/drm_device.h                      |   4 +
>  include/drm/drm_gem_shmem_helper.h            |  87 ++-
>  5 files changed, 594 insertions(+), 49 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 555fe212bd98..4cd0b5913492 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>
> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv >= 0) && shmem->evict &&
> +               shmem->eviction_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> +}
> +
> +static void
> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (!gem_shrinker || obj->import_attach)
> +               return;
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       if (drm_gem_shmem_is_evictable(shmem) ||
> +           drm_gem_shmem_is_purgeable(shmem))
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> +       else if (shmem->madv < 0)
> +               list_del_init(&shmem->madv_list);
> +       else if (shmem->evicted)
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> +       else if (!shmem->pages)
> +               list_del_init(&shmem->madv_list);
> +       else
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +}
> +
>  /**
>   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>   * @shmem: shmem GEM object to free
> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>         } else {
>                 dma_resv_lock(shmem->base.resv, NULL);
>
> +               /* take out shmem GEM object from the memory shrinker */
> +               drm_gem_shmem_madvise(shmem, -1);
> +
>                 WARN_ON(shmem->vmap_use_count);
>
>                 if (shmem->sgt) {
> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>                         sg_free_table(shmem->sgt);
>                         kfree(shmem->sgt);
>                 }
> -               if (shmem->pages)
> +               if (shmem->pages_use_count)
>                         drm_gem_shmem_put_pages(shmem);
>
>                 WARN_ON(shmem->pages_use_count);
> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>
> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->eviction_enabled = true;
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> +
> +/**
> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be purged. Initially purging is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->purge_enabled = true;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> +
> +static int
> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
>         struct page **pages;
>
> -       if (shmem->pages_use_count++ > 0)
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0) {
> +               WARN_ON(shmem->pages);
> +               return -ENOMEM;
> +       }
> +
> +       if (shmem->pages) {
> +               WARN_ON(!shmem->evicted);
>                 return 0;
> +       }
> +
> +       if (WARN_ON(!shmem->pages_use_count))
> +               return -EINVAL;
>
>         pages = drm_gem_get_pages(obj);
>         if (IS_ERR(pages)) {
>                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> -               shmem->pages_use_count = 0;
>                 return PTR_ERR(pages);
>         }
>
> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>         return 0;
>  }
>
> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       int err;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       if (shmem->pages_use_count++ > 0) {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err)
> +                       goto err_zero_use;
> +
> +               return 0;
> +       }
> +
> +       err = drm_gem_shmem_acquire_pages(shmem);
> +       if (err)
> +               goto err_zero_use;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +
> +err_zero_use:
> +       shmem->pages_use_count = 0;
> +
> +       return err;
> +}
> +
> +static void
> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       if (!shmem->pages) {
> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> +               return;
> +       }
> +
> +#ifdef CONFIG_X86
> +       if (shmem->map_wc)
> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> +#endif
> +
> +       drm_gem_put_pages(obj, shmem->pages,
> +                         shmem->pages_mark_dirty_on_put,
> +                         shmem->pages_mark_accessed_on_put);
> +       shmem->pages = NULL;
> +}
> +
>  /*
>   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>   * @shmem: shmem GEM object
> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>   */
>  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>  {
> -       struct drm_gem_object *obj = &shmem->base;
> -
>         dma_resv_assert_held(shmem->base.resv);
>
>         if (WARN_ON_ONCE(!shmem->pages_use_count))
> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>         if (--shmem->pages_use_count > 0)
>                 return;
>
> -#ifdef CONFIG_X86
> -       if (shmem->map_wc)
> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> -#endif
> +       drm_gem_shmem_release_pages(shmem);
>
> -       drm_gem_put_pages(obj, shmem->pages,
> -                         shmem->pages_mark_dirty_on_put,
> -                         shmem->pages_mark_accessed_on_put);
> -       shmem->pages = NULL;
> +       drm_gem_shmem_update_pages_state(shmem);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>
> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>   */
>  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>  {
> +       int ret;
> +
>         dma_resv_assert_held(shmem->base.resv);
>
>         WARN_ON(shmem->base.import_attach);
>
> -       return drm_gem_shmem_get_pages(shmem);
> +       ret = drm_gem_shmem_get_pages(shmem);
> +       if (!ret)
> +               shmem->pages_pin_count++;
> +
> +       return ret;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_pin);
>
> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>         WARN_ON(shmem->base.import_attach);
>
>         drm_gem_shmem_put_pages(shmem);
> +
> +       shmem->pages_pin_count--;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_unpin);
>
> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>                         return 0;
>                 }
>
> -               ret = drm_gem_shmem_get_pages(shmem);
> +               ret = drm_gem_shmem_pin(shmem);
>                 if (ret)
>                         goto err_zero_use;
>
> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>
>  err_put_pages:
>         if (!obj->import_attach)
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>  err_zero_use:
>         shmem->vmap_use_count = 0;
>
> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>                         return;
>
>                 vunmap(shmem->vaddr);
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>         }
>
>         shmem->vaddr = NULL;
> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>
>         madv = shmem->madv;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         return (madv >= 0);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_madvise);
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> + *                           hardware access to the memory.
> + * @shmem: shmem GEM object
> + *
> + * This function moves shmem GEM back to memory if it was previously evicted
> + * by the memory shrinker. The GEM is ready to use on success.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
> -       struct drm_device *dev = obj->dev;
> +       struct sg_table *sgt;
> +       int err;
>
>         dma_resv_assert_held(shmem->base.resv);
>
> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +       if (shmem->evicted) {
> +               err = drm_gem_shmem_acquire_pages(shmem);
> +               if (err)
> +                       return err;
> +
> +               sgt = drm_gem_shmem_get_sg_table(shmem);
> +               if (IS_ERR(sgt))
> +                       return PTR_ERR(sgt);
> +
> +               err = dma_map_sgtable(obj->dev->dev, sgt,
> +                                     DMA_BIDIRECTIONAL, 0);
> +               if (err) {
> +                       sg_free_table(sgt);
> +                       kfree(sgt);
> +                       return err;
> +               }
>
> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> -       sg_free_table(shmem->sgt);
> -       kfree(shmem->sgt);
> -       shmem->sgt = NULL;
> +               shmem->sgt = sgt;
> +               shmem->evicted = false;
>
> -       drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_update_pages_state(shmem);
> +       }
>
> -       shmem->madv = -1;
> +       if (!shmem->pages)
> +               return -ENOMEM;
>
> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> -       drm_gem_free_mmap_offset(obj);
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>
> -       /* Our goal here is to return as much of the memory as
> -        * is possible back to the system as we are called from OOM.
> -        * To do this we must instruct the shmfs to drop all of its
> -        * backing pages, *now*.
> -        */
> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_device *dev = obj->dev;
>
> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +       if (shmem->evicted)
> +               return;
> +
> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> +       drm_gem_shmem_release_pages(shmem);
> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> +
> +       sg_free_table(shmem->sgt);
> +       kfree(shmem->sgt);
> +       shmem->sgt = NULL;
>  }
> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>
>  /**
>   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>         vm_fault_t ret;
>         struct page *page;
>         pgoff_t page_offset;
> +       bool pages_unpinned;
> +       int err;
>
>         /* We don't use vmf->pgoff since that has the fake offset */
>         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>
>         dma_resv_lock(shmem->base.resv, NULL);
>
> -       if (page_offset >= num_pages ||
> -           WARN_ON_ONCE(!shmem->pages) ||
> -           shmem->madv < 0) {
> +       /* Sanity-check that we have the pages pointer when it should present */
> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> +
> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>                 ret = VM_FAULT_SIGBUS;
>         } else {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err) {
> +                       ret = VM_FAULT_OOM;
> +                       goto unlock;
> +               }
> +
>                 page = shmem->pages[page_offset];
>
>                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>         }
>
> +unlock:
>         dma_resv_unlock(shmem->base.resv);
>
>         return ret;
> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>  {
>         struct drm_gem_object *obj = vma->vm_private_data;
>         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -       int ret;
>
>         WARN_ON(shmem->base.import_attach);
>
>         dma_resv_lock(shmem->base.resv, NULL);
> -       ret = drm_gem_shmem_get_pages(shmem);
> -       WARN_ON_ONCE(ret != 0);
> +
> +       if (drm_gem_shmem_get_pages(shmem))
> +               shmem->pages_use_count++;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
>         dma_resv_unlock(shmem->base.resv);
>
>         drm_gem_vm_open(vma);
> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                               struct drm_printer *p, unsigned int indent)
>  {
> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>
>         if (shmem->base.import_attach)
> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>                                   shmem->vmap_use_count);
>
> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_print_info);
>
> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>
>         shmem->sgt = sgt;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         dma_resv_unlock(shmem->base.resv);
>
>         return sgt;
> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>
> +static struct drm_gem_shmem_shrinker *
> +to_drm_shrinker(struct shrinker *shrinker)
> +{
> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> +                                    struct shrink_control *sc)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       unsigned long count = 0;
> +
> +       if (!mutex_trylock(&gem_shrinker->lock))
> +               return 0;
> +
> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> +               count += shmem->base.size;
> +
> +               if (count >= SHRINK_EMPTY)
> +                       break;
> +       }
> +
> +       mutex_unlock(&gem_shrinker->lock);

As I mentioned on other thread, count_objects, being approximate but
lockless and fast is the important thing.  Otherwise when you start
hitting the shrinker on many threads, you end up serializing them all,
even if you have no pages to return to the system at that point.

> +
> +       if (count >= SHRINK_EMPTY)
> +               return SHRINK_EMPTY - 1;
> +
> +       return count ?: SHRINK_EMPTY;
> +}
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> +       WARN_ON(shmem->evicted);
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +
> +       shmem->evicted = true;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> +
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +       drm_gem_free_mmap_offset(obj);
> +
> +       /* Our goal here is to return as much of the memory as
> +        * is possible back to the system as we are called from OOM.
> +        * To do this we must instruct the shmfs to drop all of its
> +        * backing pages, *now*.
> +        */
> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +
> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +
> +       shmem->madv = -1;
> +       shmem->evicted = false;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> +                                       unsigned long nr_to_scan,
> +                                       bool *lock_contention,
> +                                       bool evict)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       struct list_head still_in_list;
> +       struct drm_gem_object *obj;
> +       unsigned long freed = 0;
> +       size_t page_count;
> +       int err;
> +
> +       INIT_LIST_HEAD(&still_in_list);
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       while (freed < nr_to_scan) {
> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> +                                                typeof(*shmem), madv_list);
> +               if (!shmem)
> +                       break;
> +
> +               obj = &shmem->base;
> +               page_count = obj->size >> PAGE_SHIFT;
> +               list_move_tail(&shmem->madv_list, &still_in_list);
> +
> +               if (evict) {
> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> +                           get_nr_swap_pages() < page_count)
> +                               continue;
> +               } else {
> +                       if (!drm_gem_shmem_is_purgeable(shmem))
> +                               continue;
> +               }
> +
> +               /*
> +                * If it's in the process of being freed, gem_object->free()
> +                * may be blocked on lock waiting to remove it.  So just
> +                * skip it.
> +                */
> +               if (!kref_get_unless_zero(&obj->refcount))
> +                       continue;
> +
> +               mutex_unlock(&gem_shrinker->lock);
> +
> +               /* prevent racing with job-submission code paths */
> +               if (!dma_resv_trylock(obj->resv)) {
> +                       *lock_contention |= true;
> +                       goto shrinker_lock;
> +               }
> +
> +               /* prevent racing with the dma-buf importing/exporting */
> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> +                       *lock_contention |= true;
> +                       goto resv_unlock;
> +               }

I'm not sure this is a good idea to serialize on object_name_lock.
Purgeable buffers should never be shared (imported or exported).  So
at best you are avoiding evicting and immediately swapping back in, in
a rare case, at the cost of serializing multiple threads trying to
reclaim pages in parallel.

BR,
-R

> +
> +               /* check whether h/w uses this object */
> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> +                       goto object_name_unlock;
> +
> +               /* re-check whether eviction status hasn't changed */
> +               if (!drm_gem_shmem_is_evictable(shmem) &&
> +                   !drm_gem_shmem_is_purgeable(shmem))
> +                       goto object_name_unlock;
> +
> +               err = shmem->evict(shmem);
> +               if (!err)
> +                       freed += obj->size >> PAGE_SHIFT;
> +
> +object_name_unlock:
> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> +resv_unlock:
> +               dma_resv_unlock(obj->resv);
> +shrinker_lock:
> +               drm_gem_object_put(&shmem->base);
> +               mutex_lock(&gem_shrinker->lock);
> +       }
> +
> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       return freed;
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> +                                   struct shrink_control *sc)
> +{
> +       unsigned long nr_to_scan = sc->nr_to_scan;
> +       bool lock_contention = false;
> +       unsigned long freed;
> +
> +       /* purge as many objects as we can */
> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> +                                                       &lock_contention, false);
> +
> +       /* evict as many objects as we can */
> +       if (freed < nr_to_scan)
> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> +                                                                nr_to_scan - freed,
> +                                                                &lock_contention,
> +                                                                true);
> +
> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> +}
> +
> +/**
> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> + * @dev: DRM device
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker;
> +       int err;
> +
> +       if (WARN_ON(dev->shmem_shrinker))
> +               return -EBUSY;
> +
> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> +       if (!gem_shrinker)
> +               return -ENOMEM;
> +
> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> +       gem_shrinker->dev = dev;
> +
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> +       mutex_init(&gem_shrinker->lock);
> +
> +       dev->shmem_shrinker = gem_shrinker;
> +
> +       err = register_shrinker(&gem_shrinker->base);
> +       if (err) {
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +               return err;
> +       }
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> +
> +/**
> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> + * @dev: DRM device
> + */
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> +
> +       if (gem_shrinker) {
> +               unregister_shrinker(&gem_shrinker->base);
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> +               mutex_destroy(&gem_shrinker->lock);
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +       }
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> +
>  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>  MODULE_IMPORT_NS(DMA_BUF);
>  MODULE_LICENSE("GPL v2");
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> index a4bedfeb2ec4..7cc32556f908 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> @@ -15,6 +15,13 @@
>  #include "panfrost_gem.h"
>  #include "panfrost_mmu.h"
>
> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv > 0) &&
> +               !shmem->pages_pin_count && shmem->sgt &&
> +               !shmem->base.dma_buf && !shmem->base.import_attach;
> +}
> +
>  static unsigned long
>  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>  {
> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>                 return 0;
>
>         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -               if (drm_gem_shmem_is_purgeable(shmem))
> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>                         count += shmem->base.size >> PAGE_SHIFT;
>         }
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> index b2d93cb12ebf..81bacc7e1873 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>         uint32_t hw_res_handle;
>         bool dumb;
>         bool created;
> +       bool detached;
>         bool host3d_blob, guest_blob;
>         uint32_t blob_mem, blob_flags;
>
> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>
>  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>
> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> +
>  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>                                uint32_t *resid);
>  /* virtgpu_prime.c */
> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> index 9923c7a6885e..929546cad894 100644
> --- a/include/drm/drm_device.h
> +++ b/include/drm/drm_device.h
> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>  struct drm_vma_offset_manager;
>  struct drm_vram_mm;
>  struct drm_fb_helper;
> +struct drm_gem_shmem_shrinker;
>
>  struct inode;
>
> @@ -277,6 +278,9 @@ struct drm_device {
>         /** @vram_mm: VRAM MM memory manager */
>         struct drm_vram_mm *vram_mm;
>
> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> +
>         /**
>          * @switch_power_state:
>          *
> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> index 9a8983ee8abe..62c640678a91 100644
> --- a/include/drm/drm_gem_shmem_helper.h
> +++ b/include/drm/drm_gem_shmem_helper.h
> @@ -6,6 +6,7 @@
>  #include <linux/fs.h>
>  #include <linux/mm.h>
>  #include <linux/mutex.h>
> +#include <linux/shrinker.h>
>
>  #include <drm/drm_file.h>
>  #include <drm/drm_gem.h>
> @@ -15,6 +16,7 @@
>  struct dma_buf_attachment;
>  struct drm_mode_create_dumb;
>  struct drm_printer;
> +struct drm_device;
>  struct sg_table;
>
>  /**
> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>          */
>         unsigned int pages_use_count;
>
> +       /**
> +        * @pages_pin_count:
> +        *
> +        * Reference count on the pinned pages table.
> +        * The pages can be evicted by memory shrinker
> +        * when the count reaches zero.
> +        */
> +       unsigned int pages_pin_count;
> +
>         /**
>          * @madv: State for madvise
>          *
>          * 0 is active/inuse.
> +        * 1 is not-needed/can-be-purged
>          * A negative value is the object is purged.
> -        * Positive values are driver specific and not used by the helpers.
>          */
>         int madv;
>
> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>          * @map_wc: map object write-combined (instead of using shmem defaults).
>          */
>         bool map_wc;
> +
> +       /**
> +        * @eviction_enabled:
> +        *
> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool eviction_enabled;
> +
> +       /**
> +        * @purge_enabled:
> +        *
> +        * The shmem pages can be purged only if @purge_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool purge_enabled;
> +
> +       /**
> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> +        * Used internally by memory shrinker.
> +        */
> +       bool evicted;
> +
> +       /**
> +        * @evict:
> +        *
> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> +        * GEM's DMA reservation is kept locked by the shrinker. This is
> +        * optional callback that should be specified by drivers.
> +        *
> +        * Returns 0 on success, or -errno on error.
> +        */
> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>  };
>
>  #define to_drm_gem_shmem_obj(obj) \
> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>
>  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> +
>  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>  {
> -       return (shmem->madv > 0) &&
> -               !shmem->vmap_use_count && shmem->sgt &&
> -               !shmem->base.dma_buf && !shmem->base.import_attach;
> +       return (shmem->madv > 0) && shmem->evict &&
> +               shmem->purge_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>  }
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>
>  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>         return drm_gem_shmem_mmap(shmem, vma);
>  }
>
> +/**
> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> + */
> +struct drm_gem_shmem_shrinker {
> +       /** @base: Shrinker for purging shmem GEM objects */
> +       struct shrinker base;
> +
> +       /** @lock: Protects @lru_* */
> +       struct mutex lock;
> +
> +       /** @lru_pinned: List of pinned shmem GEM objects */
> +       struct list_head lru_pinned;
> +
> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> +       struct list_head lru_evictable;
> +
> +       /** @lru_evicted: List of evicted shmem GEM objects */
> +       struct list_head lru_evicted;
> +
> +       /** @dev: DRM device that uses this shrinker */
> +       struct drm_device *dev;
> +};
> +
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> +
>  /*
>   * Driver ops
>   */
> --
> 2.35.3
>

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

On Thu, May 26, 2022 at 4:55 PM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> Introduce a common DRM SHMEM shrinker framework that allows to reduce
> code duplication among DRM drivers by replacing theirs custom shrinker
> implementations with the generic shrinker.
>
> In order to start using DRM SHMEM shrinker drivers should:
>
> 1. Implement new evict() shmem object callback.
> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>    activate shrinking of shmem GEMs.
>
> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>
> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>  include/drm/drm_device.h                      |   4 +
>  include/drm/drm_gem_shmem_helper.h            |  87 ++-
>  5 files changed, 594 insertions(+), 49 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 555fe212bd98..4cd0b5913492 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>
> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv >= 0) && shmem->evict &&
> +               shmem->eviction_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> +}
> +
> +static void
> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (!gem_shrinker || obj->import_attach)
> +               return;
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       if (drm_gem_shmem_is_evictable(shmem) ||
> +           drm_gem_shmem_is_purgeable(shmem))
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> +       else if (shmem->madv < 0)
> +               list_del_init(&shmem->madv_list);
> +       else if (shmem->evicted)
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> +       else if (!shmem->pages)
> +               list_del_init(&shmem->madv_list);
> +       else
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +}
> +
>  /**
>   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>   * @shmem: shmem GEM object to free
> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>         } else {
>                 dma_resv_lock(shmem->base.resv, NULL);
>
> +               /* take out shmem GEM object from the memory shrinker */
> +               drm_gem_shmem_madvise(shmem, -1);
> +
>                 WARN_ON(shmem->vmap_use_count);
>
>                 if (shmem->sgt) {
> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>                         sg_free_table(shmem->sgt);
>                         kfree(shmem->sgt);
>                 }
> -               if (shmem->pages)
> +               if (shmem->pages_use_count)
>                         drm_gem_shmem_put_pages(shmem);
>
>                 WARN_ON(shmem->pages_use_count);
> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>
> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->eviction_enabled = true;
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> +
> +/**
> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be purged. Initially purging is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->purge_enabled = true;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> +
> +static int
> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
>         struct page **pages;
>
> -       if (shmem->pages_use_count++ > 0)
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0) {
> +               WARN_ON(shmem->pages);
> +               return -ENOMEM;
> +       }
> +
> +       if (shmem->pages) {
> +               WARN_ON(!shmem->evicted);
>                 return 0;
> +       }
> +
> +       if (WARN_ON(!shmem->pages_use_count))
> +               return -EINVAL;
>
>         pages = drm_gem_get_pages(obj);
>         if (IS_ERR(pages)) {
>                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> -               shmem->pages_use_count = 0;
>                 return PTR_ERR(pages);
>         }
>
> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>         return 0;
>  }
>
> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       int err;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       if (shmem->pages_use_count++ > 0) {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err)
> +                       goto err_zero_use;
> +
> +               return 0;
> +       }
> +
> +       err = drm_gem_shmem_acquire_pages(shmem);
> +       if (err)
> +               goto err_zero_use;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +
> +err_zero_use:
> +       shmem->pages_use_count = 0;
> +
> +       return err;
> +}
> +
> +static void
> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       if (!shmem->pages) {
> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> +               return;
> +       }
> +
> +#ifdef CONFIG_X86
> +       if (shmem->map_wc)
> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> +#endif
> +
> +       drm_gem_put_pages(obj, shmem->pages,
> +                         shmem->pages_mark_dirty_on_put,
> +                         shmem->pages_mark_accessed_on_put);
> +       shmem->pages = NULL;
> +}
> +
>  /*
>   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>   * @shmem: shmem GEM object
> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>   */
>  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>  {
> -       struct drm_gem_object *obj = &shmem->base;
> -
>         dma_resv_assert_held(shmem->base.resv);
>
>         if (WARN_ON_ONCE(!shmem->pages_use_count))
> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>         if (--shmem->pages_use_count > 0)
>                 return;
>
> -#ifdef CONFIG_X86
> -       if (shmem->map_wc)
> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> -#endif
> +       drm_gem_shmem_release_pages(shmem);
>
> -       drm_gem_put_pages(obj, shmem->pages,
> -                         shmem->pages_mark_dirty_on_put,
> -                         shmem->pages_mark_accessed_on_put);
> -       shmem->pages = NULL;
> +       drm_gem_shmem_update_pages_state(shmem);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>
> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>   */
>  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>  {
> +       int ret;
> +
>         dma_resv_assert_held(shmem->base.resv);
>
>         WARN_ON(shmem->base.import_attach);
>
> -       return drm_gem_shmem_get_pages(shmem);
> +       ret = drm_gem_shmem_get_pages(shmem);
> +       if (!ret)
> +               shmem->pages_pin_count++;
> +
> +       return ret;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_pin);
>
> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>         WARN_ON(shmem->base.import_attach);
>
>         drm_gem_shmem_put_pages(shmem);
> +
> +       shmem->pages_pin_count--;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_unpin);
>
> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>                         return 0;
>                 }
>
> -               ret = drm_gem_shmem_get_pages(shmem);
> +               ret = drm_gem_shmem_pin(shmem);
>                 if (ret)
>                         goto err_zero_use;
>
> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>
>  err_put_pages:
>         if (!obj->import_attach)
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>  err_zero_use:
>         shmem->vmap_use_count = 0;
>
> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>                         return;
>
>                 vunmap(shmem->vaddr);
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>         }
>
>         shmem->vaddr = NULL;
> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>
>         madv = shmem->madv;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         return (madv >= 0);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_madvise);
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> + *                           hardware access to the memory.
> + * @shmem: shmem GEM object
> + *
> + * This function moves shmem GEM back to memory if it was previously evicted
> + * by the memory shrinker. The GEM is ready to use on success.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
> -       struct drm_device *dev = obj->dev;
> +       struct sg_table *sgt;
> +       int err;
>
>         dma_resv_assert_held(shmem->base.resv);
>
> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +       if (shmem->evicted) {
> +               err = drm_gem_shmem_acquire_pages(shmem);
> +               if (err)
> +                       return err;
> +
> +               sgt = drm_gem_shmem_get_sg_table(shmem);
> +               if (IS_ERR(sgt))
> +                       return PTR_ERR(sgt);
> +
> +               err = dma_map_sgtable(obj->dev->dev, sgt,
> +                                     DMA_BIDIRECTIONAL, 0);
> +               if (err) {
> +                       sg_free_table(sgt);
> +                       kfree(sgt);
> +                       return err;
> +               }
>
> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> -       sg_free_table(shmem->sgt);
> -       kfree(shmem->sgt);
> -       shmem->sgt = NULL;
> +               shmem->sgt = sgt;
> +               shmem->evicted = false;
>
> -       drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_update_pages_state(shmem);
> +       }
>
> -       shmem->madv = -1;
> +       if (!shmem->pages)
> +               return -ENOMEM;
>
> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> -       drm_gem_free_mmap_offset(obj);
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>
> -       /* Our goal here is to return as much of the memory as
> -        * is possible back to the system as we are called from OOM.
> -        * To do this we must instruct the shmfs to drop all of its
> -        * backing pages, *now*.
> -        */
> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_device *dev = obj->dev;
>
> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +       if (shmem->evicted)
> +               return;
> +
> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> +       drm_gem_shmem_release_pages(shmem);
> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> +
> +       sg_free_table(shmem->sgt);
> +       kfree(shmem->sgt);
> +       shmem->sgt = NULL;
>  }
> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>
>  /**
>   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>         vm_fault_t ret;
>         struct page *page;
>         pgoff_t page_offset;
> +       bool pages_unpinned;
> +       int err;
>
>         /* We don't use vmf->pgoff since that has the fake offset */
>         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>
>         dma_resv_lock(shmem->base.resv, NULL);
>
> -       if (page_offset >= num_pages ||
> -           WARN_ON_ONCE(!shmem->pages) ||
> -           shmem->madv < 0) {
> +       /* Sanity-check that we have the pages pointer when it should present */
> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> +
> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>                 ret = VM_FAULT_SIGBUS;
>         } else {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err) {
> +                       ret = VM_FAULT_OOM;
> +                       goto unlock;
> +               }
> +
>                 page = shmem->pages[page_offset];
>
>                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>         }
>
> +unlock:
>         dma_resv_unlock(shmem->base.resv);
>
>         return ret;
> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>  {
>         struct drm_gem_object *obj = vma->vm_private_data;
>         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -       int ret;
>
>         WARN_ON(shmem->base.import_attach);
>
>         dma_resv_lock(shmem->base.resv, NULL);
> -       ret = drm_gem_shmem_get_pages(shmem);
> -       WARN_ON_ONCE(ret != 0);
> +
> +       if (drm_gem_shmem_get_pages(shmem))
> +               shmem->pages_use_count++;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
>         dma_resv_unlock(shmem->base.resv);
>
>         drm_gem_vm_open(vma);
> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                               struct drm_printer *p, unsigned int indent)
>  {
> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>
>         if (shmem->base.import_attach)
> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>                                   shmem->vmap_use_count);
>
> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_print_info);
>
> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>
>         shmem->sgt = sgt;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         dma_resv_unlock(shmem->base.resv);
>
>         return sgt;
> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>
> +static struct drm_gem_shmem_shrinker *
> +to_drm_shrinker(struct shrinker *shrinker)
> +{
> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> +                                    struct shrink_control *sc)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       unsigned long count = 0;
> +
> +       if (!mutex_trylock(&gem_shrinker->lock))
> +               return 0;
> +
> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> +               count += shmem->base.size;
> +
> +               if (count >= SHRINK_EMPTY)
> +                       break;
> +       }
> +
> +       mutex_unlock(&gem_shrinker->lock);

As I mentioned on other thread, count_objects, being approximate but
lockless and fast is the important thing.  Otherwise when you start
hitting the shrinker on many threads, you end up serializing them all,
even if you have no pages to return to the system at that point.

> +
> +       if (count >= SHRINK_EMPTY)
> +               return SHRINK_EMPTY - 1;
> +
> +       return count ?: SHRINK_EMPTY;
> +}
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> +       WARN_ON(shmem->evicted);
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +
> +       shmem->evicted = true;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> +
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +       drm_gem_free_mmap_offset(obj);
> +
> +       /* Our goal here is to return as much of the memory as
> +        * is possible back to the system as we are called from OOM.
> +        * To do this we must instruct the shmfs to drop all of its
> +        * backing pages, *now*.
> +        */
> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +
> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +
> +       shmem->madv = -1;
> +       shmem->evicted = false;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> +                                       unsigned long nr_to_scan,
> +                                       bool *lock_contention,
> +                                       bool evict)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       struct list_head still_in_list;
> +       struct drm_gem_object *obj;
> +       unsigned long freed = 0;
> +       size_t page_count;
> +       int err;
> +
> +       INIT_LIST_HEAD(&still_in_list);
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       while (freed < nr_to_scan) {
> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> +                                                typeof(*shmem), madv_list);
> +               if (!shmem)
> +                       break;
> +
> +               obj = &shmem->base;
> +               page_count = obj->size >> PAGE_SHIFT;
> +               list_move_tail(&shmem->madv_list, &still_in_list);
> +
> +               if (evict) {
> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> +                           get_nr_swap_pages() < page_count)
> +                               continue;
> +               } else {
> +                       if (!drm_gem_shmem_is_purgeable(shmem))
> +                               continue;
> +               }
> +
> +               /*
> +                * If it's in the process of being freed, gem_object->free()
> +                * may be blocked on lock waiting to remove it.  So just
> +                * skip it.
> +                */
> +               if (!kref_get_unless_zero(&obj->refcount))
> +                       continue;
> +
> +               mutex_unlock(&gem_shrinker->lock);
> +
> +               /* prevent racing with job-submission code paths */
> +               if (!dma_resv_trylock(obj->resv)) {
> +                       *lock_contention |= true;
> +                       goto shrinker_lock;
> +               }
> +
> +               /* prevent racing with the dma-buf importing/exporting */
> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> +                       *lock_contention |= true;
> +                       goto resv_unlock;
> +               }

I'm not sure this is a good idea to serialize on object_name_lock.
Purgeable buffers should never be shared (imported or exported).  So
at best you are avoiding evicting and immediately swapping back in, in
a rare case, at the cost of serializing multiple threads trying to
reclaim pages in parallel.

BR,
-R

> +
> +               /* check whether h/w uses this object */
> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> +                       goto object_name_unlock;
> +
> +               /* re-check whether eviction status hasn't changed */
> +               if (!drm_gem_shmem_is_evictable(shmem) &&
> +                   !drm_gem_shmem_is_purgeable(shmem))
> +                       goto object_name_unlock;
> +
> +               err = shmem->evict(shmem);
> +               if (!err)
> +                       freed += obj->size >> PAGE_SHIFT;
> +
> +object_name_unlock:
> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> +resv_unlock:
> +               dma_resv_unlock(obj->resv);
> +shrinker_lock:
> +               drm_gem_object_put(&shmem->base);
> +               mutex_lock(&gem_shrinker->lock);
> +       }
> +
> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       return freed;
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> +                                   struct shrink_control *sc)
> +{
> +       unsigned long nr_to_scan = sc->nr_to_scan;
> +       bool lock_contention = false;
> +       unsigned long freed;
> +
> +       /* purge as many objects as we can */
> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> +                                                       &lock_contention, false);
> +
> +       /* evict as many objects as we can */
> +       if (freed < nr_to_scan)
> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> +                                                                nr_to_scan - freed,
> +                                                                &lock_contention,
> +                                                                true);
> +
> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> +}
> +
> +/**
> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> + * @dev: DRM device
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker;
> +       int err;
> +
> +       if (WARN_ON(dev->shmem_shrinker))
> +               return -EBUSY;
> +
> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> +       if (!gem_shrinker)
> +               return -ENOMEM;
> +
> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> +       gem_shrinker->dev = dev;
> +
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> +       mutex_init(&gem_shrinker->lock);
> +
> +       dev->shmem_shrinker = gem_shrinker;
> +
> +       err = register_shrinker(&gem_shrinker->base);
> +       if (err) {
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +               return err;
> +       }
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> +
> +/**
> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> + * @dev: DRM device
> + */
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> +
> +       if (gem_shrinker) {
> +               unregister_shrinker(&gem_shrinker->base);
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> +               mutex_destroy(&gem_shrinker->lock);
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +       }
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> +
>  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>  MODULE_IMPORT_NS(DMA_BUF);
>  MODULE_LICENSE("GPL v2");
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> index a4bedfeb2ec4..7cc32556f908 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> @@ -15,6 +15,13 @@
>  #include "panfrost_gem.h"
>  #include "panfrost_mmu.h"
>
> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv > 0) &&
> +               !shmem->pages_pin_count && shmem->sgt &&
> +               !shmem->base.dma_buf && !shmem->base.import_attach;
> +}
> +
>  static unsigned long
>  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>  {
> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>                 return 0;
>
>         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -               if (drm_gem_shmem_is_purgeable(shmem))
> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>                         count += shmem->base.size >> PAGE_SHIFT;
>         }
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> index b2d93cb12ebf..81bacc7e1873 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>         uint32_t hw_res_handle;
>         bool dumb;
>         bool created;
> +       bool detached;
>         bool host3d_blob, guest_blob;
>         uint32_t blob_mem, blob_flags;
>
> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>
>  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>
> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> +
>  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>                                uint32_t *resid);
>  /* virtgpu_prime.c */
> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> index 9923c7a6885e..929546cad894 100644
> --- a/include/drm/drm_device.h
> +++ b/include/drm/drm_device.h
> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>  struct drm_vma_offset_manager;
>  struct drm_vram_mm;
>  struct drm_fb_helper;
> +struct drm_gem_shmem_shrinker;
>
>  struct inode;
>
> @@ -277,6 +278,9 @@ struct drm_device {
>         /** @vram_mm: VRAM MM memory manager */
>         struct drm_vram_mm *vram_mm;
>
> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> +
>         /**
>          * @switch_power_state:
>          *
> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> index 9a8983ee8abe..62c640678a91 100644
> --- a/include/drm/drm_gem_shmem_helper.h
> +++ b/include/drm/drm_gem_shmem_helper.h
> @@ -6,6 +6,7 @@
>  #include <linux/fs.h>
>  #include <linux/mm.h>
>  #include <linux/mutex.h>
> +#include <linux/shrinker.h>
>
>  #include <drm/drm_file.h>
>  #include <drm/drm_gem.h>
> @@ -15,6 +16,7 @@
>  struct dma_buf_attachment;
>  struct drm_mode_create_dumb;
>  struct drm_printer;
> +struct drm_device;
>  struct sg_table;
>
>  /**
> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>          */
>         unsigned int pages_use_count;
>
> +       /**
> +        * @pages_pin_count:
> +        *
> +        * Reference count on the pinned pages table.
> +        * The pages can be evicted by memory shrinker
> +        * when the count reaches zero.
> +        */
> +       unsigned int pages_pin_count;
> +
>         /**
>          * @madv: State for madvise
>          *
>          * 0 is active/inuse.
> +        * 1 is not-needed/can-be-purged
>          * A negative value is the object is purged.
> -        * Positive values are driver specific and not used by the helpers.
>          */
>         int madv;
>
> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>          * @map_wc: map object write-combined (instead of using shmem defaults).
>          */
>         bool map_wc;
> +
> +       /**
> +        * @eviction_enabled:
> +        *
> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool eviction_enabled;
> +
> +       /**
> +        * @purge_enabled:
> +        *
> +        * The shmem pages can be purged only if @purge_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool purge_enabled;
> +
> +       /**
> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> +        * Used internally by memory shrinker.
> +        */
> +       bool evicted;
> +
> +       /**
> +        * @evict:
> +        *
> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> +        * GEM's DMA reservation is kept locked by the shrinker. This is
> +        * optional callback that should be specified by drivers.
> +        *
> +        * Returns 0 on success, or -errno on error.
> +        */
> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>  };
>
>  #define to_drm_gem_shmem_obj(obj) \
> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>
>  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> +
>  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>  {
> -       return (shmem->madv > 0) &&
> -               !shmem->vmap_use_count && shmem->sgt &&
> -               !shmem->base.dma_buf && !shmem->base.import_attach;
> +       return (shmem->madv > 0) && shmem->evict &&
> +               shmem->purge_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>  }
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>
>  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>         return drm_gem_shmem_mmap(shmem, vma);
>  }
>
> +/**
> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> + */
> +struct drm_gem_shmem_shrinker {
> +       /** @base: Shrinker for purging shmem GEM objects */
> +       struct shrinker base;
> +
> +       /** @lock: Protects @lru_* */
> +       struct mutex lock;
> +
> +       /** @lru_pinned: List of pinned shmem GEM objects */
> +       struct list_head lru_pinned;
> +
> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> +       struct list_head lru_evictable;
> +
> +       /** @lru_evicted: List of evicted shmem GEM objects */
> +       struct list_head lru_evicted;
> +
> +       /** @dev: DRM device that uses this shrinker */
> +       struct drm_device *dev;
> +};
> +
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> +
>  /*
>   * Driver ops
>   */
> --
> 2.35.3
>

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Dmitry Osipenko]

On 6/19/22 20:53, Rob Clark wrote:
...
>> +static unsigned long
>> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
>> +                                    struct shrink_control *sc)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
>> +       struct drm_gem_shmem_object *shmem;
>> +       unsigned long count = 0;
>> +
>> +       if (!mutex_trylock(&gem_shrinker->lock))
>> +               return 0;
>> +
>> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
>> +               count += shmem->base.size;
>> +
>> +               if (count >= SHRINK_EMPTY)
>> +                       break;
>> +       }
>> +
>> +       mutex_unlock(&gem_shrinker->lock);
> 
> As I mentioned on other thread, count_objects, being approximate but
> lockless and fast is the important thing.  Otherwise when you start
> hitting the shrinker on many threads, you end up serializing them all,
> even if you have no pages to return to the system at that point.

Daniel's point for dropping the lockless variant was that we're already
in trouble if we're hitting shrinker too often and extra optimizations
won't bring much benefits to us.

Alright, I'll add back the lockless variant (or will use yours
drm_gem_lru) in the next revision. The code difference is very small
after all.

...
>> +               /* prevent racing with the dma-buf importing/exporting */
>> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
>> +                       *lock_contention |= true;
>> +                       goto resv_unlock;
>> +               }
> 
> I'm not sure this is a good idea to serialize on object_name_lock.
> Purgeable buffers should never be shared (imported or exported).  So
> at best you are avoiding evicting and immediately swapping back in, in
> a rare case, at the cost of serializing multiple threads trying to
> reclaim pages in parallel.

The object_name_lock shouldn't cause contention in practice. But objects
are also pinned on attachment, hence maybe this lock is indeed
unnecessary.. I'll re-check it.

-- 
Best regards,
Dmitry

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Dmitry Osipenko]

On 6/19/22 20:53, Rob Clark wrote:
...
>> +static unsigned long
>> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
>> +                                    struct shrink_control *sc)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
>> +       struct drm_gem_shmem_object *shmem;
>> +       unsigned long count = 0;
>> +
>> +       if (!mutex_trylock(&gem_shrinker->lock))
>> +               return 0;
>> +
>> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
>> +               count += shmem->base.size;
>> +
>> +               if (count >= SHRINK_EMPTY)
>> +                       break;
>> +       }
>> +
>> +       mutex_unlock(&gem_shrinker->lock);
> 
> As I mentioned on other thread, count_objects, being approximate but
> lockless and fast is the important thing.  Otherwise when you start
> hitting the shrinker on many threads, you end up serializing them all,
> even if you have no pages to return to the system at that point.

Daniel's point for dropping the lockless variant was that we're already
in trouble if we're hitting shrinker too often and extra optimizations
won't bring much benefits to us.

Alright, I'll add back the lockless variant (or will use yours
drm_gem_lru) in the next revision. The code difference is very small
after all.

...
>> +               /* prevent racing with the dma-buf importing/exporting */
>> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
>> +                       *lock_contention |= true;
>> +                       goto resv_unlock;
>> +               }
> 
> I'm not sure this is a good idea to serialize on object_name_lock.
> Purgeable buffers should never be shared (imported or exported).  So
> at best you are avoiding evicting and immediately swapping back in, in
> a rare case, at the cost of serializing multiple threads trying to
> reclaim pages in parallel.

The object_name_lock shouldn't cause contention in practice. But objects
are also pinned on attachment, hence maybe this lock is indeed
unnecessary.. I'll re-check it.

-- 
Best regards,
Dmitry

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Dmitry Osipenko]

On 6/19/22 20:53, Rob Clark wrote:
...
>> +static unsigned long
>> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
>> +                                    struct shrink_control *sc)
>> +{
>> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
>> +       struct drm_gem_shmem_object *shmem;
>> +       unsigned long count = 0;
>> +
>> +       if (!mutex_trylock(&gem_shrinker->lock))
>> +               return 0;
>> +
>> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
>> +               count += shmem->base.size;
>> +
>> +               if (count >= SHRINK_EMPTY)
>> +                       break;
>> +       }
>> +
>> +       mutex_unlock(&gem_shrinker->lock);
> 
> As I mentioned on other thread, count_objects, being approximate but
> lockless and fast is the important thing.  Otherwise when you start
> hitting the shrinker on many threads, you end up serializing them all,
> even if you have no pages to return to the system at that point.

Daniel's point for dropping the lockless variant was that we're already
in trouble if we're hitting shrinker too often and extra optimizations
won't bring much benefits to us.

Alright, I'll add back the lockless variant (or will use yours
drm_gem_lru) in the next revision. The code difference is very small
after all.

...
>> +               /* prevent racing with the dma-buf importing/exporting */
>> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
>> +                       *lock_contention |= true;
>> +                       goto resv_unlock;
>> +               }
> 
> I'm not sure this is a good idea to serialize on object_name_lock.
> Purgeable buffers should never be shared (imported or exported).  So
> at best you are avoiding evicting and immediately swapping back in, in
> a rare case, at the cost of serializing multiple threads trying to
> reclaim pages in parallel.

The object_name_lock shouldn't cause contention in practice. But objects
are also pinned on attachment, hence maybe this lock is indeed
unnecessary.. I'll re-check it.

-- 
Best regards,
Dmitry

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

On Mon, Jun 20, 2022 at 7:09 AM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> On 6/19/22 20:53, Rob Clark wrote:
> ...
> >> +static unsigned long
> >> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> >> +                                    struct shrink_control *sc)
> >> +{
> >> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> >> +       struct drm_gem_shmem_object *shmem;
> >> +       unsigned long count = 0;
> >> +
> >> +       if (!mutex_trylock(&gem_shrinker->lock))
> >> +               return 0;
> >> +
> >> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> >> +               count += shmem->base.size;
> >> +
> >> +               if (count >= SHRINK_EMPTY)
> >> +                       break;
> >> +       }
> >> +
> >> +       mutex_unlock(&gem_shrinker->lock);
> >
> > As I mentioned on other thread, count_objects, being approximate but
> > lockless and fast is the important thing.  Otherwise when you start
> > hitting the shrinker on many threads, you end up serializing them all,
> > even if you have no pages to return to the system at that point.
>
> Daniel's point for dropping the lockless variant was that we're already
> in trouble if we're hitting shrinker too often and extra optimizations
> won't bring much benefits to us.

At least with zram swap (which I highly recommend using even if you
are not using a physical swap file/partition), swapin/out is actually
quite fast.  And if you are leaning on zram swap to fit 8GB of chrome
browser on a 4GB device, the shrinker gets hit quite a lot.  Lower
spec (4GB RAM) chromebooks can be under constant memory pressure and
can quite easily get into a situation where you are hitting the
shrinker on many threads simultaneously.  So it is pretty important
for all shrinkers in the system (not just drm driver) to be as
concurrent as possible.  As long as you avoid serializing reclaim on
all the threads, performance can still be quite good, but if you don't
performance will fall off a cliff.

jfwiw, we are seeing pretty good results (iirc 40-70% increase in open
tab counts) with the combination of eviction + multigen LRU[1] +
sizing zram swap to be 2x physical RAM

[1] https://lwn.net/Articles/856931/

> Alright, I'll add back the lockless variant (or will use yours
> drm_gem_lru) in the next revision. The code difference is very small
> after all.
>
> ...
> >> +               /* prevent racing with the dma-buf importing/exporting */
> >> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> >> +                       *lock_contention |= true;
> >> +                       goto resv_unlock;
> >> +               }
> >
> > I'm not sure this is a good idea to serialize on object_name_lock.
> > Purgeable buffers should never be shared (imported or exported).  So
> > at best you are avoiding evicting and immediately swapping back in, in
> > a rare case, at the cost of serializing multiple threads trying to
> > reclaim pages in parallel.
>
> The object_name_lock shouldn't cause contention in practice. But objects
> are also pinned on attachment, hence maybe this lock is indeed
> unnecessary.. I'll re-check it.

I'm not worried about contention with export/import/etc, but
contention between multiple threads hitting the shrinker in parallel.
I guess since you are using trylock, it won't *block* the other
threads hitting shrinker, but they'll just end up looping in
do_shrink_slab() because they are hitting contention.

I'd have to do some experiments to see how it works out in practice,
but my gut feel is that it isn't a good idea

BR,
-R

> --
> Best regards,
> Dmitry

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

On Mon, Jun 20, 2022 at 7:09 AM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> On 6/19/22 20:53, Rob Clark wrote:
> ...
> >> +static unsigned long
> >> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> >> +                                    struct shrink_control *sc)
> >> +{
> >> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> >> +       struct drm_gem_shmem_object *shmem;
> >> +       unsigned long count = 0;
> >> +
> >> +       if (!mutex_trylock(&gem_shrinker->lock))
> >> +               return 0;
> >> +
> >> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> >> +               count += shmem->base.size;
> >> +
> >> +               if (count >= SHRINK_EMPTY)
> >> +                       break;
> >> +       }
> >> +
> >> +       mutex_unlock(&gem_shrinker->lock);
> >
> > As I mentioned on other thread, count_objects, being approximate but
> > lockless and fast is the important thing.  Otherwise when you start
> > hitting the shrinker on many threads, you end up serializing them all,
> > even if you have no pages to return to the system at that point.
>
> Daniel's point for dropping the lockless variant was that we're already
> in trouble if we're hitting shrinker too often and extra optimizations
> won't bring much benefits to us.

At least with zram swap (which I highly recommend using even if you
are not using a physical swap file/partition), swapin/out is actually
quite fast.  And if you are leaning on zram swap to fit 8GB of chrome
browser on a 4GB device, the shrinker gets hit quite a lot.  Lower
spec (4GB RAM) chromebooks can be under constant memory pressure and
can quite easily get into a situation where you are hitting the
shrinker on many threads simultaneously.  So it is pretty important
for all shrinkers in the system (not just drm driver) to be as
concurrent as possible.  As long as you avoid serializing reclaim on
all the threads, performance can still be quite good, but if you don't
performance will fall off a cliff.

jfwiw, we are seeing pretty good results (iirc 40-70% increase in open
tab counts) with the combination of eviction + multigen LRU[1] +
sizing zram swap to be 2x physical RAM

[1] https://lwn.net/Articles/856931/

> Alright, I'll add back the lockless variant (or will use yours
> drm_gem_lru) in the next revision. The code difference is very small
> after all.
>
> ...
> >> +               /* prevent racing with the dma-buf importing/exporting */
> >> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> >> +                       *lock_contention |= true;
> >> +                       goto resv_unlock;
> >> +               }
> >
> > I'm not sure this is a good idea to serialize on object_name_lock.
> > Purgeable buffers should never be shared (imported or exported).  So
> > at best you are avoiding evicting and immediately swapping back in, in
> > a rare case, at the cost of serializing multiple threads trying to
> > reclaim pages in parallel.
>
> The object_name_lock shouldn't cause contention in practice. But objects
> are also pinned on attachment, hence maybe this lock is indeed
> unnecessary.. I'll re-check it.

I'm not worried about contention with export/import/etc, but
contention between multiple threads hitting the shrinker in parallel.
I guess since you are using trylock, it won't *block* the other
threads hitting shrinker, but they'll just end up looping in
do_shrink_slab() because they are hitting contention.

I'd have to do some experiments to see how it works out in practice,
but my gut feel is that it isn't a good idea

BR,
-R

> --
> Best regards,
> Dmitry
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [Intel-gfx] [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

On Mon, Jun 20, 2022 at 7:09 AM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> On 6/19/22 20:53, Rob Clark wrote:
> ...
> >> +static unsigned long
> >> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> >> +                                    struct shrink_control *sc)
> >> +{
> >> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> >> +       struct drm_gem_shmem_object *shmem;
> >> +       unsigned long count = 0;
> >> +
> >> +       if (!mutex_trylock(&gem_shrinker->lock))
> >> +               return 0;
> >> +
> >> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> >> +               count += shmem->base.size;
> >> +
> >> +               if (count >= SHRINK_EMPTY)
> >> +                       break;
> >> +       }
> >> +
> >> +       mutex_unlock(&gem_shrinker->lock);
> >
> > As I mentioned on other thread, count_objects, being approximate but
> > lockless and fast is the important thing.  Otherwise when you start
> > hitting the shrinker on many threads, you end up serializing them all,
> > even if you have no pages to return to the system at that point.
>
> Daniel's point for dropping the lockless variant was that we're already
> in trouble if we're hitting shrinker too often and extra optimizations
> won't bring much benefits to us.

At least with zram swap (which I highly recommend using even if you
are not using a physical swap file/partition), swapin/out is actually
quite fast.  And if you are leaning on zram swap to fit 8GB of chrome
browser on a 4GB device, the shrinker gets hit quite a lot.  Lower
spec (4GB RAM) chromebooks can be under constant memory pressure and
can quite easily get into a situation where you are hitting the
shrinker on many threads simultaneously.  So it is pretty important
for all shrinkers in the system (not just drm driver) to be as
concurrent as possible.  As long as you avoid serializing reclaim on
all the threads, performance can still be quite good, but if you don't
performance will fall off a cliff.

jfwiw, we are seeing pretty good results (iirc 40-70% increase in open
tab counts) with the combination of eviction + multigen LRU[1] +
sizing zram swap to be 2x physical RAM

[1] https://lwn.net/Articles/856931/

> Alright, I'll add back the lockless variant (or will use yours
> drm_gem_lru) in the next revision. The code difference is very small
> after all.
>
> ...
> >> +               /* prevent racing with the dma-buf importing/exporting */
> >> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> >> +                       *lock_contention |= true;
> >> +                       goto resv_unlock;
> >> +               }
> >
> > I'm not sure this is a good idea to serialize on object_name_lock.
> > Purgeable buffers should never be shared (imported or exported).  So
> > at best you are avoiding evicting and immediately swapping back in, in
> > a rare case, at the cost of serializing multiple threads trying to
> > reclaim pages in parallel.
>
> The object_name_lock shouldn't cause contention in practice. But objects
> are also pinned on attachment, hence maybe this lock is indeed
> unnecessary.. I'll re-check it.

I'm not worried about contention with export/import/etc, but
contention between multiple threads hitting the shrinker in parallel.
I guess since you are using trylock, it won't *block* the other
threads hitting shrinker, but they'll just end up looping in
do_shrink_slab() because they are hitting contention.

I'd have to do some experiments to see how it works out in practice,
but my gut feel is that it isn't a good idea

BR,
-R

> --
> Best regards,
> Dmitry

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

On Mon, Jun 20, 2022 at 7:09 AM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> On 6/19/22 20:53, Rob Clark wrote:
> ...
> >> +static unsigned long
> >> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> >> +                                    struct shrink_control *sc)
> >> +{
> >> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> >> +       struct drm_gem_shmem_object *shmem;
> >> +       unsigned long count = 0;
> >> +
> >> +       if (!mutex_trylock(&gem_shrinker->lock))
> >> +               return 0;
> >> +
> >> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> >> +               count += shmem->base.size;
> >> +
> >> +               if (count >= SHRINK_EMPTY)
> >> +                       break;
> >> +       }
> >> +
> >> +       mutex_unlock(&gem_shrinker->lock);
> >
> > As I mentioned on other thread, count_objects, being approximate but
> > lockless and fast is the important thing.  Otherwise when you start
> > hitting the shrinker on many threads, you end up serializing them all,
> > even if you have no pages to return to the system at that point.
>
> Daniel's point for dropping the lockless variant was that we're already
> in trouble if we're hitting shrinker too often and extra optimizations
> won't bring much benefits to us.

At least with zram swap (which I highly recommend using even if you
are not using a physical swap file/partition), swapin/out is actually
quite fast.  And if you are leaning on zram swap to fit 8GB of chrome
browser on a 4GB device, the shrinker gets hit quite a lot.  Lower
spec (4GB RAM) chromebooks can be under constant memory pressure and
can quite easily get into a situation where you are hitting the
shrinker on many threads simultaneously.  So it is pretty important
for all shrinkers in the system (not just drm driver) to be as
concurrent as possible.  As long as you avoid serializing reclaim on
all the threads, performance can still be quite good, but if you don't
performance will fall off a cliff.

jfwiw, we are seeing pretty good results (iirc 40-70% increase in open
tab counts) with the combination of eviction + multigen LRU[1] +
sizing zram swap to be 2x physical RAM

[1] https://lwn.net/Articles/856931/

> Alright, I'll add back the lockless variant (or will use yours
> drm_gem_lru) in the next revision. The code difference is very small
> after all.
>
> ...
> >> +               /* prevent racing with the dma-buf importing/exporting */
> >> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> >> +                       *lock_contention |= true;
> >> +                       goto resv_unlock;
> >> +               }
> >
> > I'm not sure this is a good idea to serialize on object_name_lock.
> > Purgeable buffers should never be shared (imported or exported).  So
> > at best you are avoiding evicting and immediately swapping back in, in
> > a rare case, at the cost of serializing multiple threads trying to
> > reclaim pages in parallel.
>
> The object_name_lock shouldn't cause contention in practice. But objects
> are also pinned on attachment, hence maybe this lock is indeed
> unnecessary.. I'll re-check it.

I'm not worried about contention with export/import/etc, but
contention between multiple threads hitting the shrinker in parallel.
I guess since you are using trylock, it won't *block* the other
threads hitting shrinker, but they'll just end up looping in
do_shrink_slab() because they are hitting contention.

I'd have to do some experiments to see how it works out in practice,
but my gut feel is that it isn't a good idea

BR,
-R

> --
> Best regards,
> Dmitry

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

On Mon, Jun 20, 2022 at 7:09 AM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> On 6/19/22 20:53, Rob Clark wrote:
> ...
> >> +static unsigned long
> >> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> >> +                                    struct shrink_control *sc)
> >> +{
> >> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> >> +       struct drm_gem_shmem_object *shmem;
> >> +       unsigned long count = 0;
> >> +
> >> +       if (!mutex_trylock(&gem_shrinker->lock))
> >> +               return 0;
> >> +
> >> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> >> +               count += shmem->base.size;
> >> +
> >> +               if (count >= SHRINK_EMPTY)
> >> +                       break;
> >> +       }
> >> +
> >> +       mutex_unlock(&gem_shrinker->lock);
> >
> > As I mentioned on other thread, count_objects, being approximate but
> > lockless and fast is the important thing.  Otherwise when you start
> > hitting the shrinker on many threads, you end up serializing them all,
> > even if you have no pages to return to the system at that point.
>
> Daniel's point for dropping the lockless variant was that we're already
> in trouble if we're hitting shrinker too often and extra optimizations
> won't bring much benefits to us.

At least with zram swap (which I highly recommend using even if you
are not using a physical swap file/partition), swapin/out is actually
quite fast.  And if you are leaning on zram swap to fit 8GB of chrome
browser on a 4GB device, the shrinker gets hit quite a lot.  Lower
spec (4GB RAM) chromebooks can be under constant memory pressure and
can quite easily get into a situation where you are hitting the
shrinker on many threads simultaneously.  So it is pretty important
for all shrinkers in the system (not just drm driver) to be as
concurrent as possible.  As long as you avoid serializing reclaim on
all the threads, performance can still be quite good, but if you don't
performance will fall off a cliff.

jfwiw, we are seeing pretty good results (iirc 40-70% increase in open
tab counts) with the combination of eviction + multigen LRU[1] +
sizing zram swap to be 2x physical RAM

[1] https://lwn.net/Articles/856931/

> Alright, I'll add back the lockless variant (or will use yours
> drm_gem_lru) in the next revision. The code difference is very small
> after all.
>
> ...
> >> +               /* prevent racing with the dma-buf importing/exporting */
> >> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> >> +                       *lock_contention |= true;
> >> +                       goto resv_unlock;
> >> +               }
> >
> > I'm not sure this is a good idea to serialize on object_name_lock.
> > Purgeable buffers should never be shared (imported or exported).  So
> > at best you are avoiding evicting and immediately swapping back in, in
> > a rare case, at the cost of serializing multiple threads trying to
> > reclaim pages in parallel.
>
> The object_name_lock shouldn't cause contention in practice. But objects
> are also pinned on attachment, hence maybe this lock is indeed
> unnecessary.. I'll re-check it.

I'm not worried about contention with export/import/etc, but
contention between multiple threads hitting the shrinker in parallel.
I guess since you are using trylock, it won't *block* the other
threads hitting shrinker, but they'll just end up looping in
do_shrink_slab() because they are hitting contention.

I'd have to do some experiments to see how it works out in practice,
but my gut feel is that it isn't a good idea

BR,
-R

> --
> Best regards,
> Dmitry

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

()

On Thu, May 26, 2022 at 4:55 PM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> Introduce a common DRM SHMEM shrinker framework that allows to reduce
> code duplication among DRM drivers by replacing theirs custom shrinker
> implementations with the generic shrinker.
>
> In order to start using DRM SHMEM shrinker drivers should:
>
> 1. Implement new evict() shmem object callback.
> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>    activate shrinking of shmem GEMs.
>
> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>
> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>  include/drm/drm_device.h                      |   4 +
>  include/drm/drm_gem_shmem_helper.h            |  87 ++-
>  5 files changed, 594 insertions(+), 49 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 555fe212bd98..4cd0b5913492 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>
> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv >= 0) && shmem->evict &&
> +               shmem->eviction_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> +}
> +
> +static void
> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (!gem_shrinker || obj->import_attach)
> +               return;
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       if (drm_gem_shmem_is_evictable(shmem) ||
> +           drm_gem_shmem_is_purgeable(shmem))
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> +       else if (shmem->madv < 0)
> +               list_del_init(&shmem->madv_list);
> +       else if (shmem->evicted)
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> +       else if (!shmem->pages)
> +               list_del_init(&shmem->madv_list);
> +       else
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +}
> +
>  /**
>   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>   * @shmem: shmem GEM object to free
> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>         } else {
>                 dma_resv_lock(shmem->base.resv, NULL);
>
> +               /* take out shmem GEM object from the memory shrinker */
> +               drm_gem_shmem_madvise(shmem, -1);
> +
>                 WARN_ON(shmem->vmap_use_count);
>
>                 if (shmem->sgt) {
> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>                         sg_free_table(shmem->sgt);
>                         kfree(shmem->sgt);
>                 }
> -               if (shmem->pages)
> +               if (shmem->pages_use_count)
>                         drm_gem_shmem_put_pages(shmem);
>
>                 WARN_ON(shmem->pages_use_count);
> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>
> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->eviction_enabled = true;
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> +
> +/**
> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be purged. Initially purging is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->purge_enabled = true;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> +
> +static int
> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
>         struct page **pages;
>
> -       if (shmem->pages_use_count++ > 0)
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0) {
> +               WARN_ON(shmem->pages);
> +               return -ENOMEM;
> +       }
> +
> +       if (shmem->pages) {
> +               WARN_ON(!shmem->evicted);
>                 return 0;
> +       }
> +
> +       if (WARN_ON(!shmem->pages_use_count))
> +               return -EINVAL;
>
>         pages = drm_gem_get_pages(obj);
>         if (IS_ERR(pages)) {
>                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> -               shmem->pages_use_count = 0;
>                 return PTR_ERR(pages);
>         }
>
> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>         return 0;
>  }
>
> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       int err;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       if (shmem->pages_use_count++ > 0) {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err)
> +                       goto err_zero_use;
> +
> +               return 0;
> +       }
> +
> +       err = drm_gem_shmem_acquire_pages(shmem);
> +       if (err)
> +               goto err_zero_use;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +
> +err_zero_use:
> +       shmem->pages_use_count = 0;
> +
> +       return err;
> +}
> +
> +static void
> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       if (!shmem->pages) {
> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> +               return;
> +       }
> +
> +#ifdef CONFIG_X86
> +       if (shmem->map_wc)
> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> +#endif
> +
> +       drm_gem_put_pages(obj, shmem->pages,
> +                         shmem->pages_mark_dirty_on_put,
> +                         shmem->pages_mark_accessed_on_put);
> +       shmem->pages = NULL;
> +}
> +
>  /*
>   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>   * @shmem: shmem GEM object
> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>   */
>  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>  {
> -       struct drm_gem_object *obj = &shmem->base;
> -
>         dma_resv_assert_held(shmem->base.resv);
>
>         if (WARN_ON_ONCE(!shmem->pages_use_count))
> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>         if (--shmem->pages_use_count > 0)
>                 return;
>
> -#ifdef CONFIG_X86
> -       if (shmem->map_wc)
> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> -#endif
> +       drm_gem_shmem_release_pages(shmem);
>
> -       drm_gem_put_pages(obj, shmem->pages,
> -                         shmem->pages_mark_dirty_on_put,
> -                         shmem->pages_mark_accessed_on_put);
> -       shmem->pages = NULL;
> +       drm_gem_shmem_update_pages_state(shmem);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>
> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>   */
>  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>  {
> +       int ret;
> +
>         dma_resv_assert_held(shmem->base.resv);
>
>         WARN_ON(shmem->base.import_attach);
>
> -       return drm_gem_shmem_get_pages(shmem);
> +       ret = drm_gem_shmem_get_pages(shmem);
> +       if (!ret)
> +               shmem->pages_pin_count++;
> +
> +       return ret;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_pin);
>
> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>         WARN_ON(shmem->base.import_attach);
>
>         drm_gem_shmem_put_pages(shmem);
> +
> +       shmem->pages_pin_count--;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_unpin);
>
> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>                         return 0;
>                 }
>
> -               ret = drm_gem_shmem_get_pages(shmem);
> +               ret = drm_gem_shmem_pin(shmem);
>                 if (ret)
>                         goto err_zero_use;
>
> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>
>  err_put_pages:
>         if (!obj->import_attach)
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>  err_zero_use:
>         shmem->vmap_use_count = 0;
>
> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>                         return;
>
>                 vunmap(shmem->vaddr);
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>         }
>
>         shmem->vaddr = NULL;
> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>
>         madv = shmem->madv;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         return (madv >= 0);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_madvise);
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> + *                           hardware access to the memory.
> + * @shmem: shmem GEM object
> + *
> + * This function moves shmem GEM back to memory if it was previously evicted
> + * by the memory shrinker. The GEM is ready to use on success.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
> -       struct drm_device *dev = obj->dev;
> +       struct sg_table *sgt;
> +       int err;
>
>         dma_resv_assert_held(shmem->base.resv);
>
> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +       if (shmem->evicted) {
> +               err = drm_gem_shmem_acquire_pages(shmem);
> +               if (err)
> +                       return err;
> +
> +               sgt = drm_gem_shmem_get_sg_table(shmem);
> +               if (IS_ERR(sgt))
> +                       return PTR_ERR(sgt);
> +
> +               err = dma_map_sgtable(obj->dev->dev, sgt,
> +                                     DMA_BIDIRECTIONAL, 0);
> +               if (err) {
> +                       sg_free_table(sgt);
> +                       kfree(sgt);
> +                       return err;
> +               }
>
> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> -       sg_free_table(shmem->sgt);
> -       kfree(shmem->sgt);
> -       shmem->sgt = NULL;
> +               shmem->sgt = sgt;
> +               shmem->evicted = false;
>
> -       drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_update_pages_state(shmem);
> +       }
>
> -       shmem->madv = -1;
> +       if (!shmem->pages)
> +               return -ENOMEM;
>
> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> -       drm_gem_free_mmap_offset(obj);
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>
> -       /* Our goal here is to return as much of the memory as
> -        * is possible back to the system as we are called from OOM.
> -        * To do this we must instruct the shmfs to drop all of its
> -        * backing pages, *now*.
> -        */
> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_device *dev = obj->dev;
>
> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +       if (shmem->evicted)
> +               return;
> +
> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> +       drm_gem_shmem_release_pages(shmem);
> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> +
> +       sg_free_table(shmem->sgt);
> +       kfree(shmem->sgt);
> +       shmem->sgt = NULL;
>  }
> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>
>  /**
>   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>         vm_fault_t ret;
>         struct page *page;
>         pgoff_t page_offset;
> +       bool pages_unpinned;
> +       int err;
>
>         /* We don't use vmf->pgoff since that has the fake offset */
>         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>
>         dma_resv_lock(shmem->base.resv, NULL);
>
> -       if (page_offset >= num_pages ||
> -           WARN_ON_ONCE(!shmem->pages) ||
> -           shmem->madv < 0) {
> +       /* Sanity-check that we have the pages pointer when it should present */
> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> +
> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>                 ret = VM_FAULT_SIGBUS;
>         } else {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err) {
> +                       ret = VM_FAULT_OOM;
> +                       goto unlock;
> +               }
> +
>                 page = shmem->pages[page_offset];
>
>                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>         }
>
> +unlock:
>         dma_resv_unlock(shmem->base.resv);
>
>         return ret;
> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>  {
>         struct drm_gem_object *obj = vma->vm_private_data;
>         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -       int ret;
>
>         WARN_ON(shmem->base.import_attach);
>
>         dma_resv_lock(shmem->base.resv, NULL);
> -       ret = drm_gem_shmem_get_pages(shmem);
> -       WARN_ON_ONCE(ret != 0);
> +
> +       if (drm_gem_shmem_get_pages(shmem))
> +               shmem->pages_use_count++;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
>         dma_resv_unlock(shmem->base.resv);
>
>         drm_gem_vm_open(vma);
> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                               struct drm_printer *p, unsigned int indent)
>  {
> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>
>         if (shmem->base.import_attach)
> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>                                   shmem->vmap_use_count);
>
> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_print_info);
>
> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>
>         shmem->sgt = sgt;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         dma_resv_unlock(shmem->base.resv);
>
>         return sgt;
> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>
> +static struct drm_gem_shmem_shrinker *
> +to_drm_shrinker(struct shrinker *shrinker)
> +{
> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> +                                    struct shrink_control *sc)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       unsigned long count = 0;
> +
> +       if (!mutex_trylock(&gem_shrinker->lock))
> +               return 0;
> +
> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> +               count += shmem->base.size;
> +
> +               if (count >= SHRINK_EMPTY)
> +                       break;
> +       }
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       if (count >= SHRINK_EMPTY)
> +               return SHRINK_EMPTY - 1;
> +
> +       return count ?: SHRINK_EMPTY;
> +}
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> +       WARN_ON(shmem->evicted);
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +
> +       shmem->evicted = true;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> +
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +       drm_gem_free_mmap_offset(obj);
> +
> +       /* Our goal here is to return as much of the memory as
> +        * is possible back to the system as we are called from OOM.
> +        * To do this we must instruct the shmfs to drop all of its
> +        * backing pages, *now*.
> +        */
> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +
> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +
> +       shmem->madv = -1;
> +       shmem->evicted = false;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> +                                       unsigned long nr_to_scan,
> +                                       bool *lock_contention,
> +                                       bool evict)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       struct list_head still_in_list;
> +       struct drm_gem_object *obj;
> +       unsigned long freed = 0;
> +       size_t page_count;
> +       int err;
> +
> +       INIT_LIST_HEAD(&still_in_list);
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       while (freed < nr_to_scan) {
> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> +                                                typeof(*shmem), madv_list);
> +               if (!shmem)
> +                       break;
> +
> +               obj = &shmem->base;
> +               page_count = obj->size >> PAGE_SHIFT;
> +               list_move_tail(&shmem->madv_list, &still_in_list);
> +
> +               if (evict) {
> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> +                           get_nr_swap_pages() < page_count)
> +                               continue;
> +               } else {
> +                       if (!drm_gem_shmem_is_purgeable(shmem))
> +                               continue;
> +               }
> +
> +               /*
> +                * If it's in the process of being freed, gem_object->free()
> +                * may be blocked on lock waiting to remove it.  So just
> +                * skip it.
> +                */
> +               if (!kref_get_unless_zero(&obj->refcount))
> +                       continue;
> +
> +               mutex_unlock(&gem_shrinker->lock);
> +
> +               /* prevent racing with job-submission code paths */
> +               if (!dma_resv_trylock(obj->resv)) {
> +                       *lock_contention |= true;
> +                       goto shrinker_lock;
> +               }
> +
> +               /* prevent racing with the dma-buf importing/exporting */
> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> +                       *lock_contention |= true;
> +                       goto resv_unlock;
> +               }
> +
> +               /* check whether h/w uses this object */
> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> +                       goto object_name_unlock;
> +
> +               /* re-check whether eviction status hasn't changed */
> +               if (!drm_gem_shmem_is_evictable(shmem) &&
> +                   !drm_gem_shmem_is_purgeable(shmem))
> +                       goto object_name_unlock;
> +
> +               err = shmem->evict(shmem);
> +               if (!err)
> +                       freed += obj->size >> PAGE_SHIFT;
> +
> +object_name_unlock:
> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> +resv_unlock:
> +               dma_resv_unlock(obj->resv);
> +shrinker_lock:
> +               drm_gem_object_put(&shmem->base);
> +               mutex_lock(&gem_shrinker->lock);
> +       }
> +
> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       return freed;
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> +                                   struct shrink_control *sc)
> +{
> +       unsigned long nr_to_scan = sc->nr_to_scan;
> +       bool lock_contention = false;
> +       unsigned long freed;
> +
> +       /* purge as many objects as we can */
> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> +                                                       &lock_contention, false);
> +
> +       /* evict as many objects as we can */
> +       if (freed < nr_to_scan)

oh, one other small note, both in scan_objects() and count_objects(),
you should check that get_nr_swap_pages()>0 before counting
evictable/willneed objects.  (And you probably want to keep separate
LRUs for dontneed vs willneed to accomplish that.)  At least for CrOS,
inside the VM there is no swap enabled (but instead we rely on zram
swap in the host.. plus vm-balloon to balance memory pressure between
host and guest)

BR,
-R


> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> +                                                                nr_to_scan - freed,
> +                                                                &lock_contention,
> +                                                                true);
> +
> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> +}
> +
> +/**
> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> + * @dev: DRM device
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker;
> +       int err;
> +
> +       if (WARN_ON(dev->shmem_shrinker))
> +               return -EBUSY;
> +
> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> +       if (!gem_shrinker)
> +               return -ENOMEM;
> +
> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> +       gem_shrinker->dev = dev;
> +
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> +       mutex_init(&gem_shrinker->lock);
> +
> +       dev->shmem_shrinker = gem_shrinker;
> +
> +       err = register_shrinker(&gem_shrinker->base);
> +       if (err) {
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +               return err;
> +       }
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> +
> +/**
> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> + * @dev: DRM device
> + */
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> +
> +       if (gem_shrinker) {
> +               unregister_shrinker(&gem_shrinker->base);
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> +               mutex_destroy(&gem_shrinker->lock);
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +       }
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> +
>  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>  MODULE_IMPORT_NS(DMA_BUF);
>  MODULE_LICENSE("GPL v2");
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> index a4bedfeb2ec4..7cc32556f908 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> @@ -15,6 +15,13 @@
>  #include "panfrost_gem.h"
>  #include "panfrost_mmu.h"
>
> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv > 0) &&
> +               !shmem->pages_pin_count && shmem->sgt &&
> +               !shmem->base.dma_buf && !shmem->base.import_attach;
> +}
> +
>  static unsigned long
>  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>  {
> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>                 return 0;
>
>         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -               if (drm_gem_shmem_is_purgeable(shmem))
> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>                         count += shmem->base.size >> PAGE_SHIFT;
>         }
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> index b2d93cb12ebf..81bacc7e1873 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>         uint32_t hw_res_handle;
>         bool dumb;
>         bool created;
> +       bool detached;
>         bool host3d_blob, guest_blob;
>         uint32_t blob_mem, blob_flags;
>
> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>
>  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>
> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> +
>  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>                                uint32_t *resid);
>  /* virtgpu_prime.c */
> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> index 9923c7a6885e..929546cad894 100644
> --- a/include/drm/drm_device.h
> +++ b/include/drm/drm_device.h
> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>  struct drm_vma_offset_manager;
>  struct drm_vram_mm;
>  struct drm_fb_helper;
> +struct drm_gem_shmem_shrinker;
>
>  struct inode;
>
> @@ -277,6 +278,9 @@ struct drm_device {
>         /** @vram_mm: VRAM MM memory manager */
>         struct drm_vram_mm *vram_mm;
>
> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> +
>         /**
>          * @switch_power_state:
>          *
> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> index 9a8983ee8abe..62c640678a91 100644
> --- a/include/drm/drm_gem_shmem_helper.h
> +++ b/include/drm/drm_gem_shmem_helper.h
> @@ -6,6 +6,7 @@
>  #include <linux/fs.h>
>  #include <linux/mm.h>
>  #include <linux/mutex.h>
> +#include <linux/shrinker.h>
>
>  #include <drm/drm_file.h>
>  #include <drm/drm_gem.h>
> @@ -15,6 +16,7 @@
>  struct dma_buf_attachment;
>  struct drm_mode_create_dumb;
>  struct drm_printer;
> +struct drm_device;
>  struct sg_table;
>
>  /**
> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>          */
>         unsigned int pages_use_count;
>
> +       /**
> +        * @pages_pin_count:
> +        *
> +        * Reference count on the pinned pages table.
> +        * The pages can be evicted by memory shrinker
> +        * when the count reaches zero.
> +        */
> +       unsigned int pages_pin_count;
> +
>         /**
>          * @madv: State for madvise
>          *
>          * 0 is active/inuse.
> +        * 1 is not-needed/can-be-purged
>          * A negative value is the object is purged.
> -        * Positive values are driver specific and not used by the helpers.
>          */
>         int madv;
>
> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>          * @map_wc: map object write-combined (instead of using shmem defaults).
>          */
>         bool map_wc;
> +
> +       /**
> +        * @eviction_enabled:
> +        *
> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool eviction_enabled;
> +
> +       /**
> +        * @purge_enabled:
> +        *
> +        * The shmem pages can be purged only if @purge_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool purge_enabled;
> +
> +       /**
> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> +        * Used internally by memory shrinker.
> +        */
> +       bool evicted;
> +
> +       /**
> +        * @evict:
> +        *
> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> +        * GEM's DMA reservation is kept locked by the shrinker. This is
> +        * optional callback that should be specified by drivers.
> +        *
> +        * Returns 0 on success, or -errno on error.
> +        */
> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>  };
>
>  #define to_drm_gem_shmem_obj(obj) \
> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>
>  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> +
>  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>  {
> -       return (shmem->madv > 0) &&
> -               !shmem->vmap_use_count && shmem->sgt &&
> -               !shmem->base.dma_buf && !shmem->base.import_attach;
> +       return (shmem->madv > 0) && shmem->evict &&
> +               shmem->purge_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>  }
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>
>  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>         return drm_gem_shmem_mmap(shmem, vma);
>  }
>
> +/**
> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> + */
> +struct drm_gem_shmem_shrinker {
> +       /** @base: Shrinker for purging shmem GEM objects */
> +       struct shrinker base;
> +
> +       /** @lock: Protects @lru_* */
> +       struct mutex lock;
> +
> +       /** @lru_pinned: List of pinned shmem GEM objects */
> +       struct list_head lru_pinned;
> +
> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> +       struct list_head lru_evictable;
> +
> +       /** @lru_evicted: List of evicted shmem GEM objects */
> +       struct list_head lru_evicted;
> +
> +       /** @dev: DRM device that uses this shrinker */
> +       struct drm_device *dev;
> +};
> +
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> +
>  /*
>   * Driver ops
>   */
> --
> 2.35.3
>

Re: [Intel-gfx] [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

()

On Thu, May 26, 2022 at 4:55 PM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> Introduce a common DRM SHMEM shrinker framework that allows to reduce
> code duplication among DRM drivers by replacing theirs custom shrinker
> implementations with the generic shrinker.
>
> In order to start using DRM SHMEM shrinker drivers should:
>
> 1. Implement new evict() shmem object callback.
> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>    activate shrinking of shmem GEMs.
>
> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>
> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>  include/drm/drm_device.h                      |   4 +
>  include/drm/drm_gem_shmem_helper.h            |  87 ++-
>  5 files changed, 594 insertions(+), 49 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 555fe212bd98..4cd0b5913492 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>
> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv >= 0) && shmem->evict &&
> +               shmem->eviction_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> +}
> +
> +static void
> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (!gem_shrinker || obj->import_attach)
> +               return;
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       if (drm_gem_shmem_is_evictable(shmem) ||
> +           drm_gem_shmem_is_purgeable(shmem))
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> +       else if (shmem->madv < 0)
> +               list_del_init(&shmem->madv_list);
> +       else if (shmem->evicted)
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> +       else if (!shmem->pages)
> +               list_del_init(&shmem->madv_list);
> +       else
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +}
> +
>  /**
>   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>   * @shmem: shmem GEM object to free
> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>         } else {
>                 dma_resv_lock(shmem->base.resv, NULL);
>
> +               /* take out shmem GEM object from the memory shrinker */
> +               drm_gem_shmem_madvise(shmem, -1);
> +
>                 WARN_ON(shmem->vmap_use_count);
>
>                 if (shmem->sgt) {
> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>                         sg_free_table(shmem->sgt);
>                         kfree(shmem->sgt);
>                 }
> -               if (shmem->pages)
> +               if (shmem->pages_use_count)
>                         drm_gem_shmem_put_pages(shmem);
>
>                 WARN_ON(shmem->pages_use_count);
> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>
> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->eviction_enabled = true;
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> +
> +/**
> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be purged. Initially purging is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->purge_enabled = true;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> +
> +static int
> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
>         struct page **pages;
>
> -       if (shmem->pages_use_count++ > 0)
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0) {
> +               WARN_ON(shmem->pages);
> +               return -ENOMEM;
> +       }
> +
> +       if (shmem->pages) {
> +               WARN_ON(!shmem->evicted);
>                 return 0;
> +       }
> +
> +       if (WARN_ON(!shmem->pages_use_count))
> +               return -EINVAL;
>
>         pages = drm_gem_get_pages(obj);
>         if (IS_ERR(pages)) {
>                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> -               shmem->pages_use_count = 0;
>                 return PTR_ERR(pages);
>         }
>
> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>         return 0;
>  }
>
> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       int err;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       if (shmem->pages_use_count++ > 0) {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err)
> +                       goto err_zero_use;
> +
> +               return 0;
> +       }
> +
> +       err = drm_gem_shmem_acquire_pages(shmem);
> +       if (err)
> +               goto err_zero_use;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +
> +err_zero_use:
> +       shmem->pages_use_count = 0;
> +
> +       return err;
> +}
> +
> +static void
> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       if (!shmem->pages) {
> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> +               return;
> +       }
> +
> +#ifdef CONFIG_X86
> +       if (shmem->map_wc)
> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> +#endif
> +
> +       drm_gem_put_pages(obj, shmem->pages,
> +                         shmem->pages_mark_dirty_on_put,
> +                         shmem->pages_mark_accessed_on_put);
> +       shmem->pages = NULL;
> +}
> +
>  /*
>   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>   * @shmem: shmem GEM object
> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>   */
>  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>  {
> -       struct drm_gem_object *obj = &shmem->base;
> -
>         dma_resv_assert_held(shmem->base.resv);
>
>         if (WARN_ON_ONCE(!shmem->pages_use_count))
> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>         if (--shmem->pages_use_count > 0)
>                 return;
>
> -#ifdef CONFIG_X86
> -       if (shmem->map_wc)
> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> -#endif
> +       drm_gem_shmem_release_pages(shmem);
>
> -       drm_gem_put_pages(obj, shmem->pages,
> -                         shmem->pages_mark_dirty_on_put,
> -                         shmem->pages_mark_accessed_on_put);
> -       shmem->pages = NULL;
> +       drm_gem_shmem_update_pages_state(shmem);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>
> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>   */
>  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>  {
> +       int ret;
> +
>         dma_resv_assert_held(shmem->base.resv);
>
>         WARN_ON(shmem->base.import_attach);
>
> -       return drm_gem_shmem_get_pages(shmem);
> +       ret = drm_gem_shmem_get_pages(shmem);
> +       if (!ret)
> +               shmem->pages_pin_count++;
> +
> +       return ret;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_pin);
>
> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>         WARN_ON(shmem->base.import_attach);
>
>         drm_gem_shmem_put_pages(shmem);
> +
> +       shmem->pages_pin_count--;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_unpin);
>
> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>                         return 0;
>                 }
>
> -               ret = drm_gem_shmem_get_pages(shmem);
> +               ret = drm_gem_shmem_pin(shmem);
>                 if (ret)
>                         goto err_zero_use;
>
> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>
>  err_put_pages:
>         if (!obj->import_attach)
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>  err_zero_use:
>         shmem->vmap_use_count = 0;
>
> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>                         return;
>
>                 vunmap(shmem->vaddr);
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>         }
>
>         shmem->vaddr = NULL;
> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>
>         madv = shmem->madv;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         return (madv >= 0);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_madvise);
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> + *                           hardware access to the memory.
> + * @shmem: shmem GEM object
> + *
> + * This function moves shmem GEM back to memory if it was previously evicted
> + * by the memory shrinker. The GEM is ready to use on success.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
> -       struct drm_device *dev = obj->dev;
> +       struct sg_table *sgt;
> +       int err;
>
>         dma_resv_assert_held(shmem->base.resv);
>
> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +       if (shmem->evicted) {
> +               err = drm_gem_shmem_acquire_pages(shmem);
> +               if (err)
> +                       return err;
> +
> +               sgt = drm_gem_shmem_get_sg_table(shmem);
> +               if (IS_ERR(sgt))
> +                       return PTR_ERR(sgt);
> +
> +               err = dma_map_sgtable(obj->dev->dev, sgt,
> +                                     DMA_BIDIRECTIONAL, 0);
> +               if (err) {
> +                       sg_free_table(sgt);
> +                       kfree(sgt);
> +                       return err;
> +               }
>
> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> -       sg_free_table(shmem->sgt);
> -       kfree(shmem->sgt);
> -       shmem->sgt = NULL;
> +               shmem->sgt = sgt;
> +               shmem->evicted = false;
>
> -       drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_update_pages_state(shmem);
> +       }
>
> -       shmem->madv = -1;
> +       if (!shmem->pages)
> +               return -ENOMEM;
>
> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> -       drm_gem_free_mmap_offset(obj);
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>
> -       /* Our goal here is to return as much of the memory as
> -        * is possible back to the system as we are called from OOM.
> -        * To do this we must instruct the shmfs to drop all of its
> -        * backing pages, *now*.
> -        */
> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_device *dev = obj->dev;
>
> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +       if (shmem->evicted)
> +               return;
> +
> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> +       drm_gem_shmem_release_pages(shmem);
> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> +
> +       sg_free_table(shmem->sgt);
> +       kfree(shmem->sgt);
> +       shmem->sgt = NULL;
>  }
> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>
>  /**
>   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>         vm_fault_t ret;
>         struct page *page;
>         pgoff_t page_offset;
> +       bool pages_unpinned;
> +       int err;
>
>         /* We don't use vmf->pgoff since that has the fake offset */
>         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>
>         dma_resv_lock(shmem->base.resv, NULL);
>
> -       if (page_offset >= num_pages ||
> -           WARN_ON_ONCE(!shmem->pages) ||
> -           shmem->madv < 0) {
> +       /* Sanity-check that we have the pages pointer when it should present */
> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> +
> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>                 ret = VM_FAULT_SIGBUS;
>         } else {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err) {
> +                       ret = VM_FAULT_OOM;
> +                       goto unlock;
> +               }
> +
>                 page = shmem->pages[page_offset];
>
>                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>         }
>
> +unlock:
>         dma_resv_unlock(shmem->base.resv);
>
>         return ret;
> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>  {
>         struct drm_gem_object *obj = vma->vm_private_data;
>         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -       int ret;
>
>         WARN_ON(shmem->base.import_attach);
>
>         dma_resv_lock(shmem->base.resv, NULL);
> -       ret = drm_gem_shmem_get_pages(shmem);
> -       WARN_ON_ONCE(ret != 0);
> +
> +       if (drm_gem_shmem_get_pages(shmem))
> +               shmem->pages_use_count++;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
>         dma_resv_unlock(shmem->base.resv);
>
>         drm_gem_vm_open(vma);
> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                               struct drm_printer *p, unsigned int indent)
>  {
> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>
>         if (shmem->base.import_attach)
> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>                                   shmem->vmap_use_count);
>
> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_print_info);
>
> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>
>         shmem->sgt = sgt;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         dma_resv_unlock(shmem->base.resv);
>
>         return sgt;
> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>
> +static struct drm_gem_shmem_shrinker *
> +to_drm_shrinker(struct shrinker *shrinker)
> +{
> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> +                                    struct shrink_control *sc)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       unsigned long count = 0;
> +
> +       if (!mutex_trylock(&gem_shrinker->lock))
> +               return 0;
> +
> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> +               count += shmem->base.size;
> +
> +               if (count >= SHRINK_EMPTY)
> +                       break;
> +       }
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       if (count >= SHRINK_EMPTY)
> +               return SHRINK_EMPTY - 1;
> +
> +       return count ?: SHRINK_EMPTY;
> +}
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> +       WARN_ON(shmem->evicted);
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +
> +       shmem->evicted = true;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> +
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +       drm_gem_free_mmap_offset(obj);
> +
> +       /* Our goal here is to return as much of the memory as
> +        * is possible back to the system as we are called from OOM.
> +        * To do this we must instruct the shmfs to drop all of its
> +        * backing pages, *now*.
> +        */
> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +
> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +
> +       shmem->madv = -1;
> +       shmem->evicted = false;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> +                                       unsigned long nr_to_scan,
> +                                       bool *lock_contention,
> +                                       bool evict)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       struct list_head still_in_list;
> +       struct drm_gem_object *obj;
> +       unsigned long freed = 0;
> +       size_t page_count;
> +       int err;
> +
> +       INIT_LIST_HEAD(&still_in_list);
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       while (freed < nr_to_scan) {
> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> +                                                typeof(*shmem), madv_list);
> +               if (!shmem)
> +                       break;
> +
> +               obj = &shmem->base;
> +               page_count = obj->size >> PAGE_SHIFT;
> +               list_move_tail(&shmem->madv_list, &still_in_list);
> +
> +               if (evict) {
> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> +                           get_nr_swap_pages() < page_count)
> +                               continue;
> +               } else {
> +                       if (!drm_gem_shmem_is_purgeable(shmem))
> +                               continue;
> +               }
> +
> +               /*
> +                * If it's in the process of being freed, gem_object->free()
> +                * may be blocked on lock waiting to remove it.  So just
> +                * skip it.
> +                */
> +               if (!kref_get_unless_zero(&obj->refcount))
> +                       continue;
> +
> +               mutex_unlock(&gem_shrinker->lock);
> +
> +               /* prevent racing with job-submission code paths */
> +               if (!dma_resv_trylock(obj->resv)) {
> +                       *lock_contention |= true;
> +                       goto shrinker_lock;
> +               }
> +
> +               /* prevent racing with the dma-buf importing/exporting */
> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> +                       *lock_contention |= true;
> +                       goto resv_unlock;
> +               }
> +
> +               /* check whether h/w uses this object */
> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> +                       goto object_name_unlock;
> +
> +               /* re-check whether eviction status hasn't changed */
> +               if (!drm_gem_shmem_is_evictable(shmem) &&
> +                   !drm_gem_shmem_is_purgeable(shmem))
> +                       goto object_name_unlock;
> +
> +               err = shmem->evict(shmem);
> +               if (!err)
> +                       freed += obj->size >> PAGE_SHIFT;
> +
> +object_name_unlock:
> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> +resv_unlock:
> +               dma_resv_unlock(obj->resv);
> +shrinker_lock:
> +               drm_gem_object_put(&shmem->base);
> +               mutex_lock(&gem_shrinker->lock);
> +       }
> +
> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       return freed;
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> +                                   struct shrink_control *sc)
> +{
> +       unsigned long nr_to_scan = sc->nr_to_scan;
> +       bool lock_contention = false;
> +       unsigned long freed;
> +
> +       /* purge as many objects as we can */
> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> +                                                       &lock_contention, false);
> +
> +       /* evict as many objects as we can */
> +       if (freed < nr_to_scan)

oh, one other small note, both in scan_objects() and count_objects(),
you should check that get_nr_swap_pages()>0 before counting
evictable/willneed objects.  (And you probably want to keep separate
LRUs for dontneed vs willneed to accomplish that.)  At least for CrOS,
inside the VM there is no swap enabled (but instead we rely on zram
swap in the host.. plus vm-balloon to balance memory pressure between
host and guest)

BR,
-R


> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> +                                                                nr_to_scan - freed,
> +                                                                &lock_contention,
> +                                                                true);
> +
> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> +}
> +
> +/**
> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> + * @dev: DRM device
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker;
> +       int err;
> +
> +       if (WARN_ON(dev->shmem_shrinker))
> +               return -EBUSY;
> +
> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> +       if (!gem_shrinker)
> +               return -ENOMEM;
> +
> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> +       gem_shrinker->dev = dev;
> +
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> +       mutex_init(&gem_shrinker->lock);
> +
> +       dev->shmem_shrinker = gem_shrinker;
> +
> +       err = register_shrinker(&gem_shrinker->base);
> +       if (err) {
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +               return err;
> +       }
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> +
> +/**
> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> + * @dev: DRM device
> + */
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> +
> +       if (gem_shrinker) {
> +               unregister_shrinker(&gem_shrinker->base);
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> +               mutex_destroy(&gem_shrinker->lock);
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +       }
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> +
>  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>  MODULE_IMPORT_NS(DMA_BUF);
>  MODULE_LICENSE("GPL v2");
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> index a4bedfeb2ec4..7cc32556f908 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> @@ -15,6 +15,13 @@
>  #include "panfrost_gem.h"
>  #include "panfrost_mmu.h"
>
> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv > 0) &&
> +               !shmem->pages_pin_count && shmem->sgt &&
> +               !shmem->base.dma_buf && !shmem->base.import_attach;
> +}
> +
>  static unsigned long
>  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>  {
> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>                 return 0;
>
>         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -               if (drm_gem_shmem_is_purgeable(shmem))
> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>                         count += shmem->base.size >> PAGE_SHIFT;
>         }
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> index b2d93cb12ebf..81bacc7e1873 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>         uint32_t hw_res_handle;
>         bool dumb;
>         bool created;
> +       bool detached;
>         bool host3d_blob, guest_blob;
>         uint32_t blob_mem, blob_flags;
>
> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>
>  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>
> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> +
>  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>                                uint32_t *resid);
>  /* virtgpu_prime.c */
> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> index 9923c7a6885e..929546cad894 100644
> --- a/include/drm/drm_device.h
> +++ b/include/drm/drm_device.h
> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>  struct drm_vma_offset_manager;
>  struct drm_vram_mm;
>  struct drm_fb_helper;
> +struct drm_gem_shmem_shrinker;
>
>  struct inode;
>
> @@ -277,6 +278,9 @@ struct drm_device {
>         /** @vram_mm: VRAM MM memory manager */
>         struct drm_vram_mm *vram_mm;
>
> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> +
>         /**
>          * @switch_power_state:
>          *
> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> index 9a8983ee8abe..62c640678a91 100644
> --- a/include/drm/drm_gem_shmem_helper.h
> +++ b/include/drm/drm_gem_shmem_helper.h
> @@ -6,6 +6,7 @@
>  #include <linux/fs.h>
>  #include <linux/mm.h>
>  #include <linux/mutex.h>
> +#include <linux/shrinker.h>
>
>  #include <drm/drm_file.h>
>  #include <drm/drm_gem.h>
> @@ -15,6 +16,7 @@
>  struct dma_buf_attachment;
>  struct drm_mode_create_dumb;
>  struct drm_printer;
> +struct drm_device;
>  struct sg_table;
>
>  /**
> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>          */
>         unsigned int pages_use_count;
>
> +       /**
> +        * @pages_pin_count:
> +        *
> +        * Reference count on the pinned pages table.
> +        * The pages can be evicted by memory shrinker
> +        * when the count reaches zero.
> +        */
> +       unsigned int pages_pin_count;
> +
>         /**
>          * @madv: State for madvise
>          *
>          * 0 is active/inuse.
> +        * 1 is not-needed/can-be-purged
>          * A negative value is the object is purged.
> -        * Positive values are driver specific and not used by the helpers.
>          */
>         int madv;
>
> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>          * @map_wc: map object write-combined (instead of using shmem defaults).
>          */
>         bool map_wc;
> +
> +       /**
> +        * @eviction_enabled:
> +        *
> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool eviction_enabled;
> +
> +       /**
> +        * @purge_enabled:
> +        *
> +        * The shmem pages can be purged only if @purge_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool purge_enabled;
> +
> +       /**
> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> +        * Used internally by memory shrinker.
> +        */
> +       bool evicted;
> +
> +       /**
> +        * @evict:
> +        *
> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> +        * GEM's DMA reservation is kept locked by the shrinker. This is
> +        * optional callback that should be specified by drivers.
> +        *
> +        * Returns 0 on success, or -errno on error.
> +        */
> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>  };
>
>  #define to_drm_gem_shmem_obj(obj) \
> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>
>  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> +
>  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>  {
> -       return (shmem->madv > 0) &&
> -               !shmem->vmap_use_count && shmem->sgt &&
> -               !shmem->base.dma_buf && !shmem->base.import_attach;
> +       return (shmem->madv > 0) && shmem->evict &&
> +               shmem->purge_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>  }
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>
>  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>         return drm_gem_shmem_mmap(shmem, vma);
>  }
>
> +/**
> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> + */
> +struct drm_gem_shmem_shrinker {
> +       /** @base: Shrinker for purging shmem GEM objects */
> +       struct shrinker base;
> +
> +       /** @lock: Protects @lru_* */
> +       struct mutex lock;
> +
> +       /** @lru_pinned: List of pinned shmem GEM objects */
> +       struct list_head lru_pinned;
> +
> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> +       struct list_head lru_evictable;
> +
> +       /** @lru_evicted: List of evicted shmem GEM objects */
> +       struct list_head lru_evicted;
> +
> +       /** @dev: DRM device that uses this shrinker */
> +       struct drm_device *dev;
> +};
> +
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> +
>  /*
>   * Driver ops
>   */
> --
> 2.35.3
>

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

()

On Thu, May 26, 2022 at 4:55 PM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> Introduce a common DRM SHMEM shrinker framework that allows to reduce
> code duplication among DRM drivers by replacing theirs custom shrinker
> implementations with the generic shrinker.
>
> In order to start using DRM SHMEM shrinker drivers should:
>
> 1. Implement new evict() shmem object callback.
> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>    activate shrinking of shmem GEMs.
>
> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>
> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>  include/drm/drm_device.h                      |   4 +
>  include/drm/drm_gem_shmem_helper.h            |  87 ++-
>  5 files changed, 594 insertions(+), 49 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 555fe212bd98..4cd0b5913492 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>
> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv >= 0) && shmem->evict &&
> +               shmem->eviction_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> +}
> +
> +static void
> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (!gem_shrinker || obj->import_attach)
> +               return;
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       if (drm_gem_shmem_is_evictable(shmem) ||
> +           drm_gem_shmem_is_purgeable(shmem))
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> +       else if (shmem->madv < 0)
> +               list_del_init(&shmem->madv_list);
> +       else if (shmem->evicted)
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> +       else if (!shmem->pages)
> +               list_del_init(&shmem->madv_list);
> +       else
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +}
> +
>  /**
>   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>   * @shmem: shmem GEM object to free
> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>         } else {
>                 dma_resv_lock(shmem->base.resv, NULL);
>
> +               /* take out shmem GEM object from the memory shrinker */
> +               drm_gem_shmem_madvise(shmem, -1);
> +
>                 WARN_ON(shmem->vmap_use_count);
>
>                 if (shmem->sgt) {
> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>                         sg_free_table(shmem->sgt);
>                         kfree(shmem->sgt);
>                 }
> -               if (shmem->pages)
> +               if (shmem->pages_use_count)
>                         drm_gem_shmem_put_pages(shmem);
>
>                 WARN_ON(shmem->pages_use_count);
> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>
> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->eviction_enabled = true;
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> +
> +/**
> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be purged. Initially purging is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->purge_enabled = true;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> +
> +static int
> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
>         struct page **pages;
>
> -       if (shmem->pages_use_count++ > 0)
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0) {
> +               WARN_ON(shmem->pages);
> +               return -ENOMEM;
> +       }
> +
> +       if (shmem->pages) {
> +               WARN_ON(!shmem->evicted);
>                 return 0;
> +       }
> +
> +       if (WARN_ON(!shmem->pages_use_count))
> +               return -EINVAL;
>
>         pages = drm_gem_get_pages(obj);
>         if (IS_ERR(pages)) {
>                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> -               shmem->pages_use_count = 0;
>                 return PTR_ERR(pages);
>         }
>
> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>         return 0;
>  }
>
> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       int err;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       if (shmem->pages_use_count++ > 0) {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err)
> +                       goto err_zero_use;
> +
> +               return 0;
> +       }
> +
> +       err = drm_gem_shmem_acquire_pages(shmem);
> +       if (err)
> +               goto err_zero_use;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +
> +err_zero_use:
> +       shmem->pages_use_count = 0;
> +
> +       return err;
> +}
> +
> +static void
> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       if (!shmem->pages) {
> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> +               return;
> +       }
> +
> +#ifdef CONFIG_X86
> +       if (shmem->map_wc)
> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> +#endif
> +
> +       drm_gem_put_pages(obj, shmem->pages,
> +                         shmem->pages_mark_dirty_on_put,
> +                         shmem->pages_mark_accessed_on_put);
> +       shmem->pages = NULL;
> +}
> +
>  /*
>   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>   * @shmem: shmem GEM object
> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>   */
>  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>  {
> -       struct drm_gem_object *obj = &shmem->base;
> -
>         dma_resv_assert_held(shmem->base.resv);
>
>         if (WARN_ON_ONCE(!shmem->pages_use_count))
> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>         if (--shmem->pages_use_count > 0)
>                 return;
>
> -#ifdef CONFIG_X86
> -       if (shmem->map_wc)
> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> -#endif
> +       drm_gem_shmem_release_pages(shmem);
>
> -       drm_gem_put_pages(obj, shmem->pages,
> -                         shmem->pages_mark_dirty_on_put,
> -                         shmem->pages_mark_accessed_on_put);
> -       shmem->pages = NULL;
> +       drm_gem_shmem_update_pages_state(shmem);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>
> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>   */
>  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>  {
> +       int ret;
> +
>         dma_resv_assert_held(shmem->base.resv);
>
>         WARN_ON(shmem->base.import_attach);
>
> -       return drm_gem_shmem_get_pages(shmem);
> +       ret = drm_gem_shmem_get_pages(shmem);
> +       if (!ret)
> +               shmem->pages_pin_count++;
> +
> +       return ret;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_pin);
>
> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>         WARN_ON(shmem->base.import_attach);
>
>         drm_gem_shmem_put_pages(shmem);
> +
> +       shmem->pages_pin_count--;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_unpin);
>
> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>                         return 0;
>                 }
>
> -               ret = drm_gem_shmem_get_pages(shmem);
> +               ret = drm_gem_shmem_pin(shmem);
>                 if (ret)
>                         goto err_zero_use;
>
> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>
>  err_put_pages:
>         if (!obj->import_attach)
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>  err_zero_use:
>         shmem->vmap_use_count = 0;
>
> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>                         return;
>
>                 vunmap(shmem->vaddr);
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>         }
>
>         shmem->vaddr = NULL;
> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>
>         madv = shmem->madv;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         return (madv >= 0);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_madvise);
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> + *                           hardware access to the memory.
> + * @shmem: shmem GEM object
> + *
> + * This function moves shmem GEM back to memory if it was previously evicted
> + * by the memory shrinker. The GEM is ready to use on success.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
> -       struct drm_device *dev = obj->dev;
> +       struct sg_table *sgt;
> +       int err;
>
>         dma_resv_assert_held(shmem->base.resv);
>
> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +       if (shmem->evicted) {
> +               err = drm_gem_shmem_acquire_pages(shmem);
> +               if (err)
> +                       return err;
> +
> +               sgt = drm_gem_shmem_get_sg_table(shmem);
> +               if (IS_ERR(sgt))
> +                       return PTR_ERR(sgt);
> +
> +               err = dma_map_sgtable(obj->dev->dev, sgt,
> +                                     DMA_BIDIRECTIONAL, 0);
> +               if (err) {
> +                       sg_free_table(sgt);
> +                       kfree(sgt);
> +                       return err;
> +               }
>
> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> -       sg_free_table(shmem->sgt);
> -       kfree(shmem->sgt);
> -       shmem->sgt = NULL;
> +               shmem->sgt = sgt;
> +               shmem->evicted = false;
>
> -       drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_update_pages_state(shmem);
> +       }
>
> -       shmem->madv = -1;
> +       if (!shmem->pages)
> +               return -ENOMEM;
>
> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> -       drm_gem_free_mmap_offset(obj);
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>
> -       /* Our goal here is to return as much of the memory as
> -        * is possible back to the system as we are called from OOM.
> -        * To do this we must instruct the shmfs to drop all of its
> -        * backing pages, *now*.
> -        */
> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_device *dev = obj->dev;
>
> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +       if (shmem->evicted)
> +               return;
> +
> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> +       drm_gem_shmem_release_pages(shmem);
> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> +
> +       sg_free_table(shmem->sgt);
> +       kfree(shmem->sgt);
> +       shmem->sgt = NULL;
>  }
> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>
>  /**
>   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>         vm_fault_t ret;
>         struct page *page;
>         pgoff_t page_offset;
> +       bool pages_unpinned;
> +       int err;
>
>         /* We don't use vmf->pgoff since that has the fake offset */
>         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>
>         dma_resv_lock(shmem->base.resv, NULL);
>
> -       if (page_offset >= num_pages ||
> -           WARN_ON_ONCE(!shmem->pages) ||
> -           shmem->madv < 0) {
> +       /* Sanity-check that we have the pages pointer when it should present */
> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> +
> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>                 ret = VM_FAULT_SIGBUS;
>         } else {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err) {
> +                       ret = VM_FAULT_OOM;
> +                       goto unlock;
> +               }
> +
>                 page = shmem->pages[page_offset];
>
>                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>         }
>
> +unlock:
>         dma_resv_unlock(shmem->base.resv);
>
>         return ret;
> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>  {
>         struct drm_gem_object *obj = vma->vm_private_data;
>         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -       int ret;
>
>         WARN_ON(shmem->base.import_attach);
>
>         dma_resv_lock(shmem->base.resv, NULL);
> -       ret = drm_gem_shmem_get_pages(shmem);
> -       WARN_ON_ONCE(ret != 0);
> +
> +       if (drm_gem_shmem_get_pages(shmem))
> +               shmem->pages_use_count++;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
>         dma_resv_unlock(shmem->base.resv);
>
>         drm_gem_vm_open(vma);
> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                               struct drm_printer *p, unsigned int indent)
>  {
> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>
>         if (shmem->base.import_attach)
> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>                                   shmem->vmap_use_count);
>
> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_print_info);
>
> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>
>         shmem->sgt = sgt;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         dma_resv_unlock(shmem->base.resv);
>
>         return sgt;
> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>
> +static struct drm_gem_shmem_shrinker *
> +to_drm_shrinker(struct shrinker *shrinker)
> +{
> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> +                                    struct shrink_control *sc)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       unsigned long count = 0;
> +
> +       if (!mutex_trylock(&gem_shrinker->lock))
> +               return 0;
> +
> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> +               count += shmem->base.size;
> +
> +               if (count >= SHRINK_EMPTY)
> +                       break;
> +       }
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       if (count >= SHRINK_EMPTY)
> +               return SHRINK_EMPTY - 1;
> +
> +       return count ?: SHRINK_EMPTY;
> +}
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> +       WARN_ON(shmem->evicted);
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +
> +       shmem->evicted = true;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> +
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +       drm_gem_free_mmap_offset(obj);
> +
> +       /* Our goal here is to return as much of the memory as
> +        * is possible back to the system as we are called from OOM.
> +        * To do this we must instruct the shmfs to drop all of its
> +        * backing pages, *now*.
> +        */
> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +
> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +
> +       shmem->madv = -1;
> +       shmem->evicted = false;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> +                                       unsigned long nr_to_scan,
> +                                       bool *lock_contention,
> +                                       bool evict)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       struct list_head still_in_list;
> +       struct drm_gem_object *obj;
> +       unsigned long freed = 0;
> +       size_t page_count;
> +       int err;
> +
> +       INIT_LIST_HEAD(&still_in_list);
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       while (freed < nr_to_scan) {
> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> +                                                typeof(*shmem), madv_list);
> +               if (!shmem)
> +                       break;
> +
> +               obj = &shmem->base;
> +               page_count = obj->size >> PAGE_SHIFT;
> +               list_move_tail(&shmem->madv_list, &still_in_list);
> +
> +               if (evict) {
> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> +                           get_nr_swap_pages() < page_count)
> +                               continue;
> +               } else {
> +                       if (!drm_gem_shmem_is_purgeable(shmem))
> +                               continue;
> +               }
> +
> +               /*
> +                * If it's in the process of being freed, gem_object->free()
> +                * may be blocked on lock waiting to remove it.  So just
> +                * skip it.
> +                */
> +               if (!kref_get_unless_zero(&obj->refcount))
> +                       continue;
> +
> +               mutex_unlock(&gem_shrinker->lock);
> +
> +               /* prevent racing with job-submission code paths */
> +               if (!dma_resv_trylock(obj->resv)) {
> +                       *lock_contention |= true;
> +                       goto shrinker_lock;
> +               }
> +
> +               /* prevent racing with the dma-buf importing/exporting */
> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> +                       *lock_contention |= true;
> +                       goto resv_unlock;
> +               }
> +
> +               /* check whether h/w uses this object */
> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> +                       goto object_name_unlock;
> +
> +               /* re-check whether eviction status hasn't changed */
> +               if (!drm_gem_shmem_is_evictable(shmem) &&
> +                   !drm_gem_shmem_is_purgeable(shmem))
> +                       goto object_name_unlock;
> +
> +               err = shmem->evict(shmem);
> +               if (!err)
> +                       freed += obj->size >> PAGE_SHIFT;
> +
> +object_name_unlock:
> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> +resv_unlock:
> +               dma_resv_unlock(obj->resv);
> +shrinker_lock:
> +               drm_gem_object_put(&shmem->base);
> +               mutex_lock(&gem_shrinker->lock);
> +       }
> +
> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       return freed;
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> +                                   struct shrink_control *sc)
> +{
> +       unsigned long nr_to_scan = sc->nr_to_scan;
> +       bool lock_contention = false;
> +       unsigned long freed;
> +
> +       /* purge as many objects as we can */
> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> +                                                       &lock_contention, false);
> +
> +       /* evict as many objects as we can */
> +       if (freed < nr_to_scan)

oh, one other small note, both in scan_objects() and count_objects(),
you should check that get_nr_swap_pages()>0 before counting
evictable/willneed objects.  (And you probably want to keep separate
LRUs for dontneed vs willneed to accomplish that.)  At least for CrOS,
inside the VM there is no swap enabled (but instead we rely on zram
swap in the host.. plus vm-balloon to balance memory pressure between
host and guest)

BR,
-R


> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> +                                                                nr_to_scan - freed,
> +                                                                &lock_contention,
> +                                                                true);
> +
> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> +}
> +
> +/**
> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> + * @dev: DRM device
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker;
> +       int err;
> +
> +       if (WARN_ON(dev->shmem_shrinker))
> +               return -EBUSY;
> +
> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> +       if (!gem_shrinker)
> +               return -ENOMEM;
> +
> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> +       gem_shrinker->dev = dev;
> +
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> +       mutex_init(&gem_shrinker->lock);
> +
> +       dev->shmem_shrinker = gem_shrinker;
> +
> +       err = register_shrinker(&gem_shrinker->base);
> +       if (err) {
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +               return err;
> +       }
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> +
> +/**
> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> + * @dev: DRM device
> + */
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> +
> +       if (gem_shrinker) {
> +               unregister_shrinker(&gem_shrinker->base);
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> +               mutex_destroy(&gem_shrinker->lock);
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +       }
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> +
>  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>  MODULE_IMPORT_NS(DMA_BUF);
>  MODULE_LICENSE("GPL v2");
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> index a4bedfeb2ec4..7cc32556f908 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> @@ -15,6 +15,13 @@
>  #include "panfrost_gem.h"
>  #include "panfrost_mmu.h"
>
> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv > 0) &&
> +               !shmem->pages_pin_count && shmem->sgt &&
> +               !shmem->base.dma_buf && !shmem->base.import_attach;
> +}
> +
>  static unsigned long
>  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>  {
> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>                 return 0;
>
>         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -               if (drm_gem_shmem_is_purgeable(shmem))
> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>                         count += shmem->base.size >> PAGE_SHIFT;
>         }
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> index b2d93cb12ebf..81bacc7e1873 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>         uint32_t hw_res_handle;
>         bool dumb;
>         bool created;
> +       bool detached;
>         bool host3d_blob, guest_blob;
>         uint32_t blob_mem, blob_flags;
>
> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>
>  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>
> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> +
>  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>                                uint32_t *resid);
>  /* virtgpu_prime.c */
> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> index 9923c7a6885e..929546cad894 100644
> --- a/include/drm/drm_device.h
> +++ b/include/drm/drm_device.h
> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>  struct drm_vma_offset_manager;
>  struct drm_vram_mm;
>  struct drm_fb_helper;
> +struct drm_gem_shmem_shrinker;
>
>  struct inode;
>
> @@ -277,6 +278,9 @@ struct drm_device {
>         /** @vram_mm: VRAM MM memory manager */
>         struct drm_vram_mm *vram_mm;
>
> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> +
>         /**
>          * @switch_power_state:
>          *
> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> index 9a8983ee8abe..62c640678a91 100644
> --- a/include/drm/drm_gem_shmem_helper.h
> +++ b/include/drm/drm_gem_shmem_helper.h
> @@ -6,6 +6,7 @@
>  #include <linux/fs.h>
>  #include <linux/mm.h>
>  #include <linux/mutex.h>
> +#include <linux/shrinker.h>
>
>  #include <drm/drm_file.h>
>  #include <drm/drm_gem.h>
> @@ -15,6 +16,7 @@
>  struct dma_buf_attachment;
>  struct drm_mode_create_dumb;
>  struct drm_printer;
> +struct drm_device;
>  struct sg_table;
>
>  /**
> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>          */
>         unsigned int pages_use_count;
>
> +       /**
> +        * @pages_pin_count:
> +        *
> +        * Reference count on the pinned pages table.
> +        * The pages can be evicted by memory shrinker
> +        * when the count reaches zero.
> +        */
> +       unsigned int pages_pin_count;
> +
>         /**
>          * @madv: State for madvise
>          *
>          * 0 is active/inuse.
> +        * 1 is not-needed/can-be-purged
>          * A negative value is the object is purged.
> -        * Positive values are driver specific and not used by the helpers.
>          */
>         int madv;
>
> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>          * @map_wc: map object write-combined (instead of using shmem defaults).
>          */
>         bool map_wc;
> +
> +       /**
> +        * @eviction_enabled:
> +        *
> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool eviction_enabled;
> +
> +       /**
> +        * @purge_enabled:
> +        *
> +        * The shmem pages can be purged only if @purge_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool purge_enabled;
> +
> +       /**
> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> +        * Used internally by memory shrinker.
> +        */
> +       bool evicted;
> +
> +       /**
> +        * @evict:
> +        *
> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> +        * GEM's DMA reservation is kept locked by the shrinker. This is
> +        * optional callback that should be specified by drivers.
> +        *
> +        * Returns 0 on success, or -errno on error.
> +        */
> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>  };
>
>  #define to_drm_gem_shmem_obj(obj) \
> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>
>  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> +
>  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>  {
> -       return (shmem->madv > 0) &&
> -               !shmem->vmap_use_count && shmem->sgt &&
> -               !shmem->base.dma_buf && !shmem->base.import_attach;
> +       return (shmem->madv > 0) && shmem->evict &&
> +               shmem->purge_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>  }
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>
>  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>         return drm_gem_shmem_mmap(shmem, vma);
>  }
>
> +/**
> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> + */
> +struct drm_gem_shmem_shrinker {
> +       /** @base: Shrinker for purging shmem GEM objects */
> +       struct shrinker base;
> +
> +       /** @lock: Protects @lru_* */
> +       struct mutex lock;
> +
> +       /** @lru_pinned: List of pinned shmem GEM objects */
> +       struct list_head lru_pinned;
> +
> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> +       struct list_head lru_evictable;
> +
> +       /** @lru_evicted: List of evicted shmem GEM objects */
> +       struct list_head lru_evicted;
> +
> +       /** @dev: DRM device that uses this shrinker */
> +       struct drm_device *dev;
> +};
> +
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> +
>  /*
>   * Driver ops
>   */
> --
> 2.35.3
>

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

()

On Thu, May 26, 2022 at 4:55 PM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> Introduce a common DRM SHMEM shrinker framework that allows to reduce
> code duplication among DRM drivers by replacing theirs custom shrinker
> implementations with the generic shrinker.
>
> In order to start using DRM SHMEM shrinker drivers should:
>
> 1. Implement new evict() shmem object callback.
> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>    activate shrinking of shmem GEMs.
>
> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>
> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>  include/drm/drm_device.h                      |   4 +
>  include/drm/drm_gem_shmem_helper.h            |  87 ++-
>  5 files changed, 594 insertions(+), 49 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 555fe212bd98..4cd0b5913492 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>
> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv >= 0) && shmem->evict &&
> +               shmem->eviction_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> +}
> +
> +static void
> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (!gem_shrinker || obj->import_attach)
> +               return;
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       if (drm_gem_shmem_is_evictable(shmem) ||
> +           drm_gem_shmem_is_purgeable(shmem))
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> +       else if (shmem->madv < 0)
> +               list_del_init(&shmem->madv_list);
> +       else if (shmem->evicted)
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> +       else if (!shmem->pages)
> +               list_del_init(&shmem->madv_list);
> +       else
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +}
> +
>  /**
>   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>   * @shmem: shmem GEM object to free
> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>         } else {
>                 dma_resv_lock(shmem->base.resv, NULL);
>
> +               /* take out shmem GEM object from the memory shrinker */
> +               drm_gem_shmem_madvise(shmem, -1);
> +
>                 WARN_ON(shmem->vmap_use_count);
>
>                 if (shmem->sgt) {
> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>                         sg_free_table(shmem->sgt);
>                         kfree(shmem->sgt);
>                 }
> -               if (shmem->pages)
> +               if (shmem->pages_use_count)
>                         drm_gem_shmem_put_pages(shmem);
>
>                 WARN_ON(shmem->pages_use_count);
> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>
> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->eviction_enabled = true;
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> +
> +/**
> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be purged. Initially purging is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->purge_enabled = true;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> +
> +static int
> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
>         struct page **pages;
>
> -       if (shmem->pages_use_count++ > 0)
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0) {
> +               WARN_ON(shmem->pages);
> +               return -ENOMEM;
> +       }
> +
> +       if (shmem->pages) {
> +               WARN_ON(!shmem->evicted);
>                 return 0;
> +       }
> +
> +       if (WARN_ON(!shmem->pages_use_count))
> +               return -EINVAL;
>
>         pages = drm_gem_get_pages(obj);
>         if (IS_ERR(pages)) {
>                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> -               shmem->pages_use_count = 0;
>                 return PTR_ERR(pages);
>         }
>
> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>         return 0;
>  }
>
> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       int err;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       if (shmem->pages_use_count++ > 0) {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err)
> +                       goto err_zero_use;
> +
> +               return 0;
> +       }
> +
> +       err = drm_gem_shmem_acquire_pages(shmem);
> +       if (err)
> +               goto err_zero_use;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +
> +err_zero_use:
> +       shmem->pages_use_count = 0;
> +
> +       return err;
> +}
> +
> +static void
> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       if (!shmem->pages) {
> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> +               return;
> +       }
> +
> +#ifdef CONFIG_X86
> +       if (shmem->map_wc)
> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> +#endif
> +
> +       drm_gem_put_pages(obj, shmem->pages,
> +                         shmem->pages_mark_dirty_on_put,
> +                         shmem->pages_mark_accessed_on_put);
> +       shmem->pages = NULL;
> +}
> +
>  /*
>   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>   * @shmem: shmem GEM object
> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>   */
>  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>  {
> -       struct drm_gem_object *obj = &shmem->base;
> -
>         dma_resv_assert_held(shmem->base.resv);
>
>         if (WARN_ON_ONCE(!shmem->pages_use_count))
> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>         if (--shmem->pages_use_count > 0)
>                 return;
>
> -#ifdef CONFIG_X86
> -       if (shmem->map_wc)
> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> -#endif
> +       drm_gem_shmem_release_pages(shmem);
>
> -       drm_gem_put_pages(obj, shmem->pages,
> -                         shmem->pages_mark_dirty_on_put,
> -                         shmem->pages_mark_accessed_on_put);
> -       shmem->pages = NULL;
> +       drm_gem_shmem_update_pages_state(shmem);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>
> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>   */
>  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>  {
> +       int ret;
> +
>         dma_resv_assert_held(shmem->base.resv);
>
>         WARN_ON(shmem->base.import_attach);
>
> -       return drm_gem_shmem_get_pages(shmem);
> +       ret = drm_gem_shmem_get_pages(shmem);
> +       if (!ret)
> +               shmem->pages_pin_count++;
> +
> +       return ret;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_pin);
>
> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>         WARN_ON(shmem->base.import_attach);
>
>         drm_gem_shmem_put_pages(shmem);
> +
> +       shmem->pages_pin_count--;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_unpin);
>
> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>                         return 0;
>                 }
>
> -               ret = drm_gem_shmem_get_pages(shmem);
> +               ret = drm_gem_shmem_pin(shmem);
>                 if (ret)
>                         goto err_zero_use;
>
> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>
>  err_put_pages:
>         if (!obj->import_attach)
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>  err_zero_use:
>         shmem->vmap_use_count = 0;
>
> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>                         return;
>
>                 vunmap(shmem->vaddr);
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>         }
>
>         shmem->vaddr = NULL;
> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>
>         madv = shmem->madv;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         return (madv >= 0);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_madvise);
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> + *                           hardware access to the memory.
> + * @shmem: shmem GEM object
> + *
> + * This function moves shmem GEM back to memory if it was previously evicted
> + * by the memory shrinker. The GEM is ready to use on success.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
> -       struct drm_device *dev = obj->dev;
> +       struct sg_table *sgt;
> +       int err;
>
>         dma_resv_assert_held(shmem->base.resv);
>
> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +       if (shmem->evicted) {
> +               err = drm_gem_shmem_acquire_pages(shmem);
> +               if (err)
> +                       return err;
> +
> +               sgt = drm_gem_shmem_get_sg_table(shmem);
> +               if (IS_ERR(sgt))
> +                       return PTR_ERR(sgt);
> +
> +               err = dma_map_sgtable(obj->dev->dev, sgt,
> +                                     DMA_BIDIRECTIONAL, 0);
> +               if (err) {
> +                       sg_free_table(sgt);
> +                       kfree(sgt);
> +                       return err;
> +               }
>
> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> -       sg_free_table(shmem->sgt);
> -       kfree(shmem->sgt);
> -       shmem->sgt = NULL;
> +               shmem->sgt = sgt;
> +               shmem->evicted = false;
>
> -       drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_update_pages_state(shmem);
> +       }
>
> -       shmem->madv = -1;
> +       if (!shmem->pages)
> +               return -ENOMEM;
>
> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> -       drm_gem_free_mmap_offset(obj);
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>
> -       /* Our goal here is to return as much of the memory as
> -        * is possible back to the system as we are called from OOM.
> -        * To do this we must instruct the shmfs to drop all of its
> -        * backing pages, *now*.
> -        */
> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_device *dev = obj->dev;
>
> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +       if (shmem->evicted)
> +               return;
> +
> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> +       drm_gem_shmem_release_pages(shmem);
> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> +
> +       sg_free_table(shmem->sgt);
> +       kfree(shmem->sgt);
> +       shmem->sgt = NULL;
>  }
> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>
>  /**
>   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>         vm_fault_t ret;
>         struct page *page;
>         pgoff_t page_offset;
> +       bool pages_unpinned;
> +       int err;
>
>         /* We don't use vmf->pgoff since that has the fake offset */
>         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>
>         dma_resv_lock(shmem->base.resv, NULL);
>
> -       if (page_offset >= num_pages ||
> -           WARN_ON_ONCE(!shmem->pages) ||
> -           shmem->madv < 0) {
> +       /* Sanity-check that we have the pages pointer when it should present */
> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> +
> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>                 ret = VM_FAULT_SIGBUS;
>         } else {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err) {
> +                       ret = VM_FAULT_OOM;
> +                       goto unlock;
> +               }
> +
>                 page = shmem->pages[page_offset];
>
>                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>         }
>
> +unlock:
>         dma_resv_unlock(shmem->base.resv);
>
>         return ret;
> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>  {
>         struct drm_gem_object *obj = vma->vm_private_data;
>         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -       int ret;
>
>         WARN_ON(shmem->base.import_attach);
>
>         dma_resv_lock(shmem->base.resv, NULL);
> -       ret = drm_gem_shmem_get_pages(shmem);
> -       WARN_ON_ONCE(ret != 0);
> +
> +       if (drm_gem_shmem_get_pages(shmem))
> +               shmem->pages_use_count++;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
>         dma_resv_unlock(shmem->base.resv);
>
>         drm_gem_vm_open(vma);
> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                               struct drm_printer *p, unsigned int indent)
>  {
> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>
>         if (shmem->base.import_attach)
> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>                                   shmem->vmap_use_count);
>
> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_print_info);
>
> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>
>         shmem->sgt = sgt;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         dma_resv_unlock(shmem->base.resv);
>
>         return sgt;
> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>
> +static struct drm_gem_shmem_shrinker *
> +to_drm_shrinker(struct shrinker *shrinker)
> +{
> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> +                                    struct shrink_control *sc)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       unsigned long count = 0;
> +
> +       if (!mutex_trylock(&gem_shrinker->lock))
> +               return 0;
> +
> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> +               count += shmem->base.size;
> +
> +               if (count >= SHRINK_EMPTY)
> +                       break;
> +       }
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       if (count >= SHRINK_EMPTY)
> +               return SHRINK_EMPTY - 1;
> +
> +       return count ?: SHRINK_EMPTY;
> +}
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> +       WARN_ON(shmem->evicted);
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +
> +       shmem->evicted = true;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> +
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +       drm_gem_free_mmap_offset(obj);
> +
> +       /* Our goal here is to return as much of the memory as
> +        * is possible back to the system as we are called from OOM.
> +        * To do this we must instruct the shmfs to drop all of its
> +        * backing pages, *now*.
> +        */
> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +
> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +
> +       shmem->madv = -1;
> +       shmem->evicted = false;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> +                                       unsigned long nr_to_scan,
> +                                       bool *lock_contention,
> +                                       bool evict)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       struct list_head still_in_list;
> +       struct drm_gem_object *obj;
> +       unsigned long freed = 0;
> +       size_t page_count;
> +       int err;
> +
> +       INIT_LIST_HEAD(&still_in_list);
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       while (freed < nr_to_scan) {
> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> +                                                typeof(*shmem), madv_list);
> +               if (!shmem)
> +                       break;
> +
> +               obj = &shmem->base;
> +               page_count = obj->size >> PAGE_SHIFT;
> +               list_move_tail(&shmem->madv_list, &still_in_list);
> +
> +               if (evict) {
> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> +                           get_nr_swap_pages() < page_count)
> +                               continue;
> +               } else {
> +                       if (!drm_gem_shmem_is_purgeable(shmem))
> +                               continue;
> +               }
> +
> +               /*
> +                * If it's in the process of being freed, gem_object->free()
> +                * may be blocked on lock waiting to remove it.  So just
> +                * skip it.
> +                */
> +               if (!kref_get_unless_zero(&obj->refcount))
> +                       continue;
> +
> +               mutex_unlock(&gem_shrinker->lock);
> +
> +               /* prevent racing with job-submission code paths */
> +               if (!dma_resv_trylock(obj->resv)) {
> +                       *lock_contention |= true;
> +                       goto shrinker_lock;
> +               }
> +
> +               /* prevent racing with the dma-buf importing/exporting */
> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> +                       *lock_contention |= true;
> +                       goto resv_unlock;
> +               }
> +
> +               /* check whether h/w uses this object */
> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> +                       goto object_name_unlock;
> +
> +               /* re-check whether eviction status hasn't changed */
> +               if (!drm_gem_shmem_is_evictable(shmem) &&
> +                   !drm_gem_shmem_is_purgeable(shmem))
> +                       goto object_name_unlock;
> +
> +               err = shmem->evict(shmem);
> +               if (!err)
> +                       freed += obj->size >> PAGE_SHIFT;
> +
> +object_name_unlock:
> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> +resv_unlock:
> +               dma_resv_unlock(obj->resv);
> +shrinker_lock:
> +               drm_gem_object_put(&shmem->base);
> +               mutex_lock(&gem_shrinker->lock);
> +       }
> +
> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       return freed;
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> +                                   struct shrink_control *sc)
> +{
> +       unsigned long nr_to_scan = sc->nr_to_scan;
> +       bool lock_contention = false;
> +       unsigned long freed;
> +
> +       /* purge as many objects as we can */
> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> +                                                       &lock_contention, false);
> +
> +       /* evict as many objects as we can */
> +       if (freed < nr_to_scan)

oh, one other small note, both in scan_objects() and count_objects(),
you should check that get_nr_swap_pages()>0 before counting
evictable/willneed objects.  (And you probably want to keep separate
LRUs for dontneed vs willneed to accomplish that.)  At least for CrOS,
inside the VM there is no swap enabled (but instead we rely on zram
swap in the host.. plus vm-balloon to balance memory pressure between
host and guest)

BR,
-R


> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> +                                                                nr_to_scan - freed,
> +                                                                &lock_contention,
> +                                                                true);
> +
> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> +}
> +
> +/**
> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> + * @dev: DRM device
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker;
> +       int err;
> +
> +       if (WARN_ON(dev->shmem_shrinker))
> +               return -EBUSY;
> +
> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> +       if (!gem_shrinker)
> +               return -ENOMEM;
> +
> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> +       gem_shrinker->dev = dev;
> +
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> +       mutex_init(&gem_shrinker->lock);
> +
> +       dev->shmem_shrinker = gem_shrinker;
> +
> +       err = register_shrinker(&gem_shrinker->base);
> +       if (err) {
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +               return err;
> +       }
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> +
> +/**
> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> + * @dev: DRM device
> + */
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> +
> +       if (gem_shrinker) {
> +               unregister_shrinker(&gem_shrinker->base);
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> +               mutex_destroy(&gem_shrinker->lock);
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +       }
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> +
>  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>  MODULE_IMPORT_NS(DMA_BUF);
>  MODULE_LICENSE("GPL v2");
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> index a4bedfeb2ec4..7cc32556f908 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> @@ -15,6 +15,13 @@
>  #include "panfrost_gem.h"
>  #include "panfrost_mmu.h"
>
> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv > 0) &&
> +               !shmem->pages_pin_count && shmem->sgt &&
> +               !shmem->base.dma_buf && !shmem->base.import_attach;
> +}
> +
>  static unsigned long
>  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>  {
> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>                 return 0;
>
>         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -               if (drm_gem_shmem_is_purgeable(shmem))
> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>                         count += shmem->base.size >> PAGE_SHIFT;
>         }
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> index b2d93cb12ebf..81bacc7e1873 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>         uint32_t hw_res_handle;
>         bool dumb;
>         bool created;
> +       bool detached;
>         bool host3d_blob, guest_blob;
>         uint32_t blob_mem, blob_flags;
>
> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>
>  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>
> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> +
>  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>                                uint32_t *resid);
>  /* virtgpu_prime.c */
> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> index 9923c7a6885e..929546cad894 100644
> --- a/include/drm/drm_device.h
> +++ b/include/drm/drm_device.h
> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>  struct drm_vma_offset_manager;
>  struct drm_vram_mm;
>  struct drm_fb_helper;
> +struct drm_gem_shmem_shrinker;
>
>  struct inode;
>
> @@ -277,6 +278,9 @@ struct drm_device {
>         /** @vram_mm: VRAM MM memory manager */
>         struct drm_vram_mm *vram_mm;
>
> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> +
>         /**
>          * @switch_power_state:
>          *
> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> index 9a8983ee8abe..62c640678a91 100644
> --- a/include/drm/drm_gem_shmem_helper.h
> +++ b/include/drm/drm_gem_shmem_helper.h
> @@ -6,6 +6,7 @@
>  #include <linux/fs.h>
>  #include <linux/mm.h>
>  #include <linux/mutex.h>
> +#include <linux/shrinker.h>
>
>  #include <drm/drm_file.h>
>  #include <drm/drm_gem.h>
> @@ -15,6 +16,7 @@
>  struct dma_buf_attachment;
>  struct drm_mode_create_dumb;
>  struct drm_printer;
> +struct drm_device;
>  struct sg_table;
>
>  /**
> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>          */
>         unsigned int pages_use_count;
>
> +       /**
> +        * @pages_pin_count:
> +        *
> +        * Reference count on the pinned pages table.
> +        * The pages can be evicted by memory shrinker
> +        * when the count reaches zero.
> +        */
> +       unsigned int pages_pin_count;
> +
>         /**
>          * @madv: State for madvise
>          *
>          * 0 is active/inuse.
> +        * 1 is not-needed/can-be-purged
>          * A negative value is the object is purged.
> -        * Positive values are driver specific and not used by the helpers.
>          */
>         int madv;
>
> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>          * @map_wc: map object write-combined (instead of using shmem defaults).
>          */
>         bool map_wc;
> +
> +       /**
> +        * @eviction_enabled:
> +        *
> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool eviction_enabled;
> +
> +       /**
> +        * @purge_enabled:
> +        *
> +        * The shmem pages can be purged only if @purge_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool purge_enabled;
> +
> +       /**
> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> +        * Used internally by memory shrinker.
> +        */
> +       bool evicted;
> +
> +       /**
> +        * @evict:
> +        *
> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> +        * GEM's DMA reservation is kept locked by the shrinker. This is
> +        * optional callback that should be specified by drivers.
> +        *
> +        * Returns 0 on success, or -errno on error.
> +        */
> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>  };
>
>  #define to_drm_gem_shmem_obj(obj) \
> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>
>  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> +
>  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>  {
> -       return (shmem->madv > 0) &&
> -               !shmem->vmap_use_count && shmem->sgt &&
> -               !shmem->base.dma_buf && !shmem->base.import_attach;
> +       return (shmem->madv > 0) && shmem->evict &&
> +               shmem->purge_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>  }
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>
>  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>         return drm_gem_shmem_mmap(shmem, vma);
>  }
>
> +/**
> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> + */
> +struct drm_gem_shmem_shrinker {
> +       /** @base: Shrinker for purging shmem GEM objects */
> +       struct shrinker base;
> +
> +       /** @lock: Protects @lru_* */
> +       struct mutex lock;
> +
> +       /** @lru_pinned: List of pinned shmem GEM objects */
> +       struct list_head lru_pinned;
> +
> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> +       struct list_head lru_evictable;
> +
> +       /** @lru_evicted: List of evicted shmem GEM objects */
> +       struct list_head lru_evicted;
> +
> +       /** @dev: DRM device that uses this shrinker */
> +       struct drm_device *dev;
> +};
> +
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> +
>  /*
>   * Driver ops
>   */
> --
> 2.35.3
>

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Rob Clark]

()

On Thu, May 26, 2022 at 4:55 PM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> Introduce a common DRM SHMEM shrinker framework that allows to reduce
> code duplication among DRM drivers by replacing theirs custom shrinker
> implementations with the generic shrinker.
>
> In order to start using DRM SHMEM shrinker drivers should:
>
> 1. Implement new evict() shmem object callback.
> 2. Register shrinker using drm_gem_shmem_shrinker_register(drm_device).
> 3. Use drm_gem_shmem_set_purgeable(shmem) and alike API functions to
>    activate shrinking of shmem GEMs.
>
> This patch is based on a ideas borrowed from Rob's Clark MSM shrinker,
> Thomas' Zimmermann variant of SHMEM shrinker and Intel's i915 shrinker.
>
> Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>  drivers/gpu/drm/drm_gem_shmem_helper.c        | 540 ++++++++++++++++--
>  .../gpu/drm/panfrost/panfrost_gem_shrinker.c  |   9 +-
>  drivers/gpu/drm/virtio/virtgpu_drv.h          |   3 +
>  include/drm/drm_device.h                      |   4 +
>  include/drm/drm_gem_shmem_helper.h            |  87 ++-
>  5 files changed, 594 insertions(+), 49 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 555fe212bd98..4cd0b5913492 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -126,6 +126,42 @@ struct drm_gem_shmem_object *drm_gem_shmem_create(struct drm_device *dev, size_t
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_create);
>
> +static bool drm_gem_shmem_is_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv >= 0) && shmem->evict &&
> +               shmem->eviction_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && shmem->sgt && !shmem->evicted;
> +}
> +
> +static void
> +drm_gem_shmem_update_pages_state(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_gem_shmem_shrinker *gem_shrinker = obj->dev->shmem_shrinker;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (!gem_shrinker || obj->import_attach)
> +               return;
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       if (drm_gem_shmem_is_evictable(shmem) ||
> +           drm_gem_shmem_is_purgeable(shmem))
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evictable);
> +       else if (shmem->madv < 0)
> +               list_del_init(&shmem->madv_list);
> +       else if (shmem->evicted)
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_evicted);
> +       else if (!shmem->pages)
> +               list_del_init(&shmem->madv_list);
> +       else
> +               list_move_tail(&shmem->madv_list, &gem_shrinker->lru_pinned);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +}
> +
>  /**
>   * drm_gem_shmem_free - Free resources associated with a shmem GEM object
>   * @shmem: shmem GEM object to free
> @@ -142,6 +178,9 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>         } else {
>                 dma_resv_lock(shmem->base.resv, NULL);
>
> +               /* take out shmem GEM object from the memory shrinker */
> +               drm_gem_shmem_madvise(shmem, -1);
> +
>                 WARN_ON(shmem->vmap_use_count);
>
>                 if (shmem->sgt) {
> @@ -150,7 +189,7 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>                         sg_free_table(shmem->sgt);
>                         kfree(shmem->sgt);
>                 }
> -               if (shmem->pages)
> +               if (shmem->pages_use_count)
>                         drm_gem_shmem_put_pages(shmem);
>
>                 WARN_ON(shmem->pages_use_count);
> @@ -163,18 +202,82 @@ void drm_gem_shmem_free(struct drm_gem_shmem_object *shmem)
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_free);
>
> -static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_set_evictable() - Make GEM evictable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be evicted. Initially eviction is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->eviction_enabled = true;
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_evictable);
> +
> +/**
> + * drm_gem_shmem_set_purgeable() - Make GEM purgeable by memory shrinker
> + * @shmem: shmem GEM object
> + *
> + * Tell memory shrinker that this GEM can be purged. Initially purging is
> + * disabled for all GEMs. If GEM was purged, then -ENOMEM is returned.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       dma_resv_lock(shmem->base.resv, NULL);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       shmem->purge_enabled = true;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       dma_resv_unlock(shmem->base.resv);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_set_purgeable);
> +
> +static int
> +drm_gem_shmem_acquire_pages(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
>         struct page **pages;
>
> -       if (shmem->pages_use_count++ > 0)
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0) {
> +               WARN_ON(shmem->pages);
> +               return -ENOMEM;
> +       }
> +
> +       if (shmem->pages) {
> +               WARN_ON(!shmem->evicted);
>                 return 0;
> +       }
> +
> +       if (WARN_ON(!shmem->pages_use_count))
> +               return -EINVAL;
>
>         pages = drm_gem_get_pages(obj);
>         if (IS_ERR(pages)) {
>                 DRM_DEBUG_KMS("Failed to get pages (%ld)\n", PTR_ERR(pages));
> -               shmem->pages_use_count = 0;
>                 return PTR_ERR(pages);
>         }
>
> @@ -193,6 +296,58 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>         return 0;
>  }
>
> +static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       int err;
> +
> +       dma_resv_assert_held(shmem->base.resv);
> +
> +       if (shmem->madv < 0)
> +               return -ENOMEM;
> +
> +       if (shmem->pages_use_count++ > 0) {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err)
> +                       goto err_zero_use;
> +
> +               return 0;
> +       }
> +
> +       err = drm_gem_shmem_acquire_pages(shmem);
> +       if (err)
> +               goto err_zero_use;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +
> +err_zero_use:
> +       shmem->pages_use_count = 0;
> +
> +       return err;
> +}
> +
> +static void
> +drm_gem_shmem_release_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       if (!shmem->pages) {
> +               WARN_ON(!shmem->evicted && shmem->madv >= 0);
> +               return;
> +       }
> +
> +#ifdef CONFIG_X86
> +       if (shmem->map_wc)
> +               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> +#endif
> +
> +       drm_gem_put_pages(obj, shmem->pages,
> +                         shmem->pages_mark_dirty_on_put,
> +                         shmem->pages_mark_accessed_on_put);
> +       shmem->pages = NULL;
> +}
> +
>  /*
>   * drm_gem_shmem_put_pages - Decrease use count on the backing pages for a shmem GEM object
>   * @shmem: shmem GEM object
> @@ -201,8 +356,6 @@ static int drm_gem_shmem_get_pages(struct drm_gem_shmem_object *shmem)
>   */
>  void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>  {
> -       struct drm_gem_object *obj = &shmem->base;
> -
>         dma_resv_assert_held(shmem->base.resv);
>
>         if (WARN_ON_ONCE(!shmem->pages_use_count))
> @@ -211,15 +364,9 @@ void drm_gem_shmem_put_pages(struct drm_gem_shmem_object *shmem)
>         if (--shmem->pages_use_count > 0)
>                 return;
>
> -#ifdef CONFIG_X86
> -       if (shmem->map_wc)
> -               set_pages_array_wb(shmem->pages, obj->size >> PAGE_SHIFT);
> -#endif
> +       drm_gem_shmem_release_pages(shmem);
>
> -       drm_gem_put_pages(obj, shmem->pages,
> -                         shmem->pages_mark_dirty_on_put,
> -                         shmem->pages_mark_accessed_on_put);
> -       shmem->pages = NULL;
> +       drm_gem_shmem_update_pages_state(shmem);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>
> @@ -235,11 +382,17 @@ EXPORT_SYMBOL(drm_gem_shmem_put_pages);
>   */
>  int drm_gem_shmem_pin(struct drm_gem_shmem_object *shmem)
>  {
> +       int ret;
> +
>         dma_resv_assert_held(shmem->base.resv);
>
>         WARN_ON(shmem->base.import_attach);
>
> -       return drm_gem_shmem_get_pages(shmem);
> +       ret = drm_gem_shmem_get_pages(shmem);
> +       if (!ret)
> +               shmem->pages_pin_count++;
> +
> +       return ret;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_pin);
>
> @@ -257,6 +410,8 @@ void drm_gem_shmem_unpin(struct drm_gem_shmem_object *shmem)
>         WARN_ON(shmem->base.import_attach);
>
>         drm_gem_shmem_put_pages(shmem);
> +
> +       shmem->pages_pin_count--;
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_unpin);
>
> @@ -299,7 +454,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>                         return 0;
>                 }
>
> -               ret = drm_gem_shmem_get_pages(shmem);
> +               ret = drm_gem_shmem_pin(shmem);
>                 if (ret)
>                         goto err_zero_use;
>
> @@ -322,7 +477,7 @@ int drm_gem_shmem_vmap(struct drm_gem_shmem_object *shmem,
>
>  err_put_pages:
>         if (!obj->import_attach)
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>  err_zero_use:
>         shmem->vmap_use_count = 0;
>
> @@ -359,7 +514,7 @@ void drm_gem_shmem_vunmap(struct drm_gem_shmem_object *shmem,
>                         return;
>
>                 vunmap(shmem->vaddr);
> -               drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_unpin(shmem);
>         }
>
>         shmem->vaddr = NULL;
> @@ -403,41 +558,77 @@ int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv)
>
>         madv = shmem->madv;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         return (madv >= 0);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_madvise);
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +/**
> + * drm_gem_shmem_swap_in() - Moves shmem GEM back to memory and enables
> + *                           hardware access to the memory.
> + * @shmem: shmem GEM object
> + *
> + * This function moves shmem GEM back to memory if it was previously evicted
> + * by the memory shrinker. The GEM is ready to use on success.
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem)
>  {
>         struct drm_gem_object *obj = &shmem->base;
> -       struct drm_device *dev = obj->dev;
> +       struct sg_table *sgt;
> +       int err;
>
>         dma_resv_assert_held(shmem->base.resv);
>
> -       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +       if (shmem->evicted) {
> +               err = drm_gem_shmem_acquire_pages(shmem);
> +               if (err)
> +                       return err;
> +
> +               sgt = drm_gem_shmem_get_sg_table(shmem);
> +               if (IS_ERR(sgt))
> +                       return PTR_ERR(sgt);
> +
> +               err = dma_map_sgtable(obj->dev->dev, sgt,
> +                                     DMA_BIDIRECTIONAL, 0);
> +               if (err) {
> +                       sg_free_table(sgt);
> +                       kfree(sgt);
> +                       return err;
> +               }
>
> -       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> -       sg_free_table(shmem->sgt);
> -       kfree(shmem->sgt);
> -       shmem->sgt = NULL;
> +               shmem->sgt = sgt;
> +               shmem->evicted = false;
>
> -       drm_gem_shmem_put_pages(shmem);
> +               drm_gem_shmem_update_pages_state(shmem);
> +       }
>
> -       shmem->madv = -1;
> +       if (!shmem->pages)
> +               return -ENOMEM;
>
> -       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> -       drm_gem_free_mmap_offset(obj);
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_swap_in);
>
> -       /* Our goal here is to return as much of the memory as
> -        * is possible back to the system as we are called from OOM.
> -        * To do this we must instruct the shmfs to drop all of its
> -        * backing pages, *now*.
> -        */
> -       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +static void drm_gem_shmem_unpin_pages(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +       struct drm_device *dev = obj->dev;
>
> -       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +       if (shmem->evicted)
> +               return;
> +
> +       dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
> +       drm_gem_shmem_release_pages(shmem);
> +       drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping);
> +
> +       sg_free_table(shmem->sgt);
> +       kfree(shmem->sgt);
> +       shmem->sgt = NULL;
>  }
> -EXPORT_SYMBOL(drm_gem_shmem_purge);
>
>  /**
>   * drm_gem_shmem_dumb_create - Create a dumb shmem buffer object
> @@ -488,22 +679,33 @@ static vm_fault_t drm_gem_shmem_fault(struct vm_fault *vmf)
>         vm_fault_t ret;
>         struct page *page;
>         pgoff_t page_offset;
> +       bool pages_unpinned;
> +       int err;
>
>         /* We don't use vmf->pgoff since that has the fake offset */
>         page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT;
>
>         dma_resv_lock(shmem->base.resv, NULL);
>
> -       if (page_offset >= num_pages ||
> -           WARN_ON_ONCE(!shmem->pages) ||
> -           shmem->madv < 0) {
> +       /* Sanity-check that we have the pages pointer when it should present */
> +       pages_unpinned = (shmem->evicted || shmem->madv < 0 || !shmem->pages_use_count);
> +       WARN_ON_ONCE(!shmem->pages ^ pages_unpinned);
> +
> +       if (page_offset >= num_pages || (!shmem->pages && !shmem->evicted)) {
>                 ret = VM_FAULT_SIGBUS;
>         } else {
> +               err = drm_gem_shmem_swap_in(shmem);
> +               if (err) {
> +                       ret = VM_FAULT_OOM;
> +                       goto unlock;
> +               }
> +
>                 page = shmem->pages[page_offset];
>
>                 ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page));
>         }
>
> +unlock:
>         dma_resv_unlock(shmem->base.resv);
>
>         return ret;
> @@ -513,13 +715,15 @@ static void drm_gem_shmem_vm_open(struct vm_area_struct *vma)
>  {
>         struct drm_gem_object *obj = vma->vm_private_data;
>         struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
> -       int ret;
>
>         WARN_ON(shmem->base.import_attach);
>
>         dma_resv_lock(shmem->base.resv, NULL);
> -       ret = drm_gem_shmem_get_pages(shmem);
> -       WARN_ON_ONCE(ret != 0);
> +
> +       if (drm_gem_shmem_get_pages(shmem))
> +               shmem->pages_use_count++;
> +
> +       drm_gem_shmem_update_pages_state(shmem);
>         dma_resv_unlock(shmem->base.resv);
>
>         drm_gem_vm_open(vma);
> @@ -583,6 +787,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_mmap);
>  void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                               struct drm_printer *p, unsigned int indent)
>  {
> +       drm_printf_indent(p, indent, "eviction_enabled=%d\n", shmem->eviction_enabled);
> +       drm_printf_indent(p, indent, "purge_enabled=%d\n", shmem->purge_enabled);
>         drm_printf_indent(p, indent, "pages_use_count=%u\n", shmem->pages_use_count);
>
>         if (shmem->base.import_attach)
> @@ -592,7 +798,9 @@ void drm_gem_shmem_print_info(const struct drm_gem_shmem_object *shmem,
>                 drm_printf_indent(p, indent, "vmap_use_count=%u\n",
>                                   shmem->vmap_use_count);
>
> +       drm_printf_indent(p, indent, "evicted=%d\n", shmem->evicted);
>         drm_printf_indent(p, indent, "vaddr=%p\n", shmem->vaddr);
> +       drm_printf_indent(p, indent, "madv=%d\n", shmem->madv);
>  }
>  EXPORT_SYMBOL(drm_gem_shmem_print_info);
>
> @@ -667,6 +875,8 @@ struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem)
>
>         shmem->sgt = sgt;
>
> +       drm_gem_shmem_update_pages_state(shmem);
> +
>         dma_resv_unlock(shmem->base.resv);
>
>         return sgt;
> @@ -717,6 +927,250 @@ drm_gem_shmem_prime_import_sg_table(struct drm_device *dev,
>  }
>  EXPORT_SYMBOL_GPL(drm_gem_shmem_prime_import_sg_table);
>
> +static struct drm_gem_shmem_shrinker *
> +to_drm_shrinker(struct shrinker *shrinker)
> +{
> +       return container_of(shrinker, struct drm_gem_shmem_shrinker, base);
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> +                                    struct shrink_control *sc)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       unsigned long count = 0;
> +
> +       if (!mutex_trylock(&gem_shrinker->lock))
> +               return 0;
> +
> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> +               count += shmem->base.size;
> +
> +               if (count >= SHRINK_EMPTY)
> +                       break;
> +       }
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       if (count >= SHRINK_EMPTY)
> +               return SHRINK_EMPTY - 1;
> +
> +       return count ?: SHRINK_EMPTY;
> +}
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem)
> +{
> +       WARN_ON(!drm_gem_shmem_is_evictable(shmem));
> +       WARN_ON(shmem->evicted);
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +
> +       shmem->evicted = true;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_evict);
> +
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem)
> +{
> +       struct drm_gem_object *obj = &shmem->base;
> +
> +       WARN_ON(!drm_gem_shmem_is_purgeable(shmem));
> +
> +       drm_gem_shmem_unpin_pages(shmem);
> +       drm_gem_free_mmap_offset(obj);
> +
> +       /* Our goal here is to return as much of the memory as
> +        * is possible back to the system as we are called from OOM.
> +        * To do this we must instruct the shmfs to drop all of its
> +        * backing pages, *now*.
> +        */
> +       shmem_truncate_range(file_inode(obj->filp), 0, (loff_t)-1);
> +
> +       invalidate_mapping_pages(file_inode(obj->filp)->i_mapping, 0, (loff_t)-1);
> +
> +       shmem->madv = -1;
> +       shmem->evicted = false;
> +       drm_gem_shmem_update_pages_state(shmem);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_purge);
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_run_objects_scan(struct shrinker *shrinker,
> +                                       unsigned long nr_to_scan,
> +                                       bool *lock_contention,
> +                                       bool evict)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> +       struct drm_gem_shmem_object *shmem;
> +       struct list_head still_in_list;
> +       struct drm_gem_object *obj;
> +       unsigned long freed = 0;
> +       size_t page_count;
> +       int err;
> +
> +       INIT_LIST_HEAD(&still_in_list);
> +
> +       mutex_lock(&gem_shrinker->lock);
> +
> +       while (freed < nr_to_scan) {
> +               shmem = list_first_entry_or_null(&gem_shrinker->lru_evictable,
> +                                                typeof(*shmem), madv_list);
> +               if (!shmem)
> +                       break;
> +
> +               obj = &shmem->base;
> +               page_count = obj->size >> PAGE_SHIFT;
> +               list_move_tail(&shmem->madv_list, &still_in_list);
> +
> +               if (evict) {
> +                       if (!drm_gem_shmem_is_evictable(shmem) ||
> +                           get_nr_swap_pages() < page_count)
> +                               continue;
> +               } else {
> +                       if (!drm_gem_shmem_is_purgeable(shmem))
> +                               continue;
> +               }
> +
> +               /*
> +                * If it's in the process of being freed, gem_object->free()
> +                * may be blocked on lock waiting to remove it.  So just
> +                * skip it.
> +                */
> +               if (!kref_get_unless_zero(&obj->refcount))
> +                       continue;
> +
> +               mutex_unlock(&gem_shrinker->lock);
> +
> +               /* prevent racing with job-submission code paths */
> +               if (!dma_resv_trylock(obj->resv)) {
> +                       *lock_contention |= true;
> +                       goto shrinker_lock;
> +               }
> +
> +               /* prevent racing with the dma-buf importing/exporting */
> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> +                       *lock_contention |= true;
> +                       goto resv_unlock;
> +               }
> +
> +               /* check whether h/w uses this object */
> +               if (!dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_WRITE))
> +                       goto object_name_unlock;
> +
> +               /* re-check whether eviction status hasn't changed */
> +               if (!drm_gem_shmem_is_evictable(shmem) &&
> +                   !drm_gem_shmem_is_purgeable(shmem))
> +                       goto object_name_unlock;
> +
> +               err = shmem->evict(shmem);
> +               if (!err)
> +                       freed += obj->size >> PAGE_SHIFT;
> +
> +object_name_unlock:
> +               mutex_unlock(&gem_shrinker->dev->object_name_lock);
> +resv_unlock:
> +               dma_resv_unlock(obj->resv);
> +shrinker_lock:
> +               drm_gem_object_put(&shmem->base);
> +               mutex_lock(&gem_shrinker->lock);
> +       }
> +
> +       list_splice_tail(&still_in_list, &gem_shrinker->lru_evictable);
> +
> +       mutex_unlock(&gem_shrinker->lock);
> +
> +       return freed;
> +}
> +
> +static unsigned long
> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
> +                                   struct shrink_control *sc)
> +{
> +       unsigned long nr_to_scan = sc->nr_to_scan;
> +       bool lock_contention = false;
> +       unsigned long freed;
> +
> +       /* purge as many objects as we can */
> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
> +                                                       &lock_contention, false);
> +
> +       /* evict as many objects as we can */
> +       if (freed < nr_to_scan)

oh, one other small note, both in scan_objects() and count_objects(),
you should check that get_nr_swap_pages()>0 before counting
evictable/willneed objects.  (And you probably want to keep separate
LRUs for dontneed vs willneed to accomplish that.)  At least for CrOS,
inside the VM there is no swap enabled (but instead we rely on zram
swap in the host.. plus vm-balloon to balance memory pressure between
host and guest)

BR,
-R


> +               freed += drm_gem_shmem_shrinker_run_objects_scan(shrinker,
> +                                                                nr_to_scan - freed,
> +                                                                &lock_contention,
> +                                                                true);
> +
> +       return (!freed && !lock_contention) ? SHRINK_STOP : freed;
> +}
> +
> +/**
> + * drm_gem_shmem_shrinker_register() - Register shmem shrinker
> + * @dev: DRM device
> + *
> + * Returns:
> + * 0 on success or a negative error code on failure.
> + */
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker;
> +       int err;
> +
> +       if (WARN_ON(dev->shmem_shrinker))
> +               return -EBUSY;
> +
> +       gem_shrinker = kzalloc(sizeof(*gem_shrinker), GFP_KERNEL);
> +       if (!gem_shrinker)
> +               return -ENOMEM;
> +
> +       gem_shrinker->base.count_objects = drm_gem_shmem_shrinker_count_objects;
> +       gem_shrinker->base.scan_objects = drm_gem_shmem_shrinker_scan_objects;
> +       gem_shrinker->base.seeks = DEFAULT_SEEKS;
> +       gem_shrinker->dev = dev;
> +
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evictable);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_evicted);
> +       INIT_LIST_HEAD(&gem_shrinker->lru_pinned);
> +       mutex_init(&gem_shrinker->lock);
> +
> +       dev->shmem_shrinker = gem_shrinker;
> +
> +       err = register_shrinker(&gem_shrinker->base);
> +       if (err) {
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +               return err;
> +       }
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_register);
> +
> +/**
> + * drm_gem_shmem_shrinker_unregister() - Unregister shmem shrinker
> + * @dev: DRM device
> + */
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev)
> +{
> +       struct drm_gem_shmem_shrinker *gem_shrinker = dev->shmem_shrinker;
> +
> +       if (gem_shrinker) {
> +               unregister_shrinker(&gem_shrinker->base);
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evictable));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_evicted));
> +               WARN_ON(!list_empty(&gem_shrinker->lru_pinned));
> +               mutex_destroy(&gem_shrinker->lock);
> +               dev->shmem_shrinker = NULL;
> +               kfree(gem_shrinker);
> +       }
> +}
> +EXPORT_SYMBOL_GPL(drm_gem_shmem_shrinker_unregister);
> +
>  MODULE_DESCRIPTION("DRM SHMEM memory-management helpers");
>  MODULE_IMPORT_NS(DMA_BUF);
>  MODULE_LICENSE("GPL v2");
> diff --git a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> index a4bedfeb2ec4..7cc32556f908 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_gem_shrinker.c
> @@ -15,6 +15,13 @@
>  #include "panfrost_gem.h"
>  #include "panfrost_mmu.h"
>
> +static bool panfrost_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
> +{
> +       return (shmem->madv > 0) &&
> +               !shmem->pages_pin_count && shmem->sgt &&
> +               !shmem->base.dma_buf && !shmem->base.import_attach;
> +}
> +
>  static unsigned long
>  panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
>  {
> @@ -27,7 +34,7 @@ panfrost_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc
>                 return 0;
>
>         list_for_each_entry(shmem, &pfdev->shrinker_list, madv_list) {
> -               if (drm_gem_shmem_is_purgeable(shmem))
> +               if (panfrost_gem_shmem_is_purgeable(shmem))
>                         count += shmem->base.size >> PAGE_SHIFT;
>         }
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> index b2d93cb12ebf..81bacc7e1873 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_drv.h
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -89,6 +89,7 @@ struct virtio_gpu_object {
>         uint32_t hw_res_handle;
>         bool dumb;
>         bool created;
> +       bool detached;
>         bool host3d_blob, guest_blob;
>         uint32_t blob_mem, blob_flags;
>
> @@ -453,6 +454,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
>
>  bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
>
> +int virtio_gpu_reattach_shmem_object(struct virtio_gpu_object *bo);
> +
>  int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
>                                uint32_t *resid);
>  /* virtgpu_prime.c */
> diff --git a/include/drm/drm_device.h b/include/drm/drm_device.h
> index 9923c7a6885e..929546cad894 100644
> --- a/include/drm/drm_device.h
> +++ b/include/drm/drm_device.h
> @@ -16,6 +16,7 @@ struct drm_vblank_crtc;
>  struct drm_vma_offset_manager;
>  struct drm_vram_mm;
>  struct drm_fb_helper;
> +struct drm_gem_shmem_shrinker;
>
>  struct inode;
>
> @@ -277,6 +278,9 @@ struct drm_device {
>         /** @vram_mm: VRAM MM memory manager */
>         struct drm_vram_mm *vram_mm;
>
> +       /** @shmem_shrinker: SHMEM GEM memory shrinker */
> +       struct drm_gem_shmem_shrinker *shmem_shrinker;
> +
>         /**
>          * @switch_power_state:
>          *
> diff --git a/include/drm/drm_gem_shmem_helper.h b/include/drm/drm_gem_shmem_helper.h
> index 9a8983ee8abe..62c640678a91 100644
> --- a/include/drm/drm_gem_shmem_helper.h
> +++ b/include/drm/drm_gem_shmem_helper.h
> @@ -6,6 +6,7 @@
>  #include <linux/fs.h>
>  #include <linux/mm.h>
>  #include <linux/mutex.h>
> +#include <linux/shrinker.h>
>
>  #include <drm/drm_file.h>
>  #include <drm/drm_gem.h>
> @@ -15,6 +16,7 @@
>  struct dma_buf_attachment;
>  struct drm_mode_create_dumb;
>  struct drm_printer;
> +struct drm_device;
>  struct sg_table;
>
>  /**
> @@ -39,12 +41,21 @@ struct drm_gem_shmem_object {
>          */
>         unsigned int pages_use_count;
>
> +       /**
> +        * @pages_pin_count:
> +        *
> +        * Reference count on the pinned pages table.
> +        * The pages can be evicted by memory shrinker
> +        * when the count reaches zero.
> +        */
> +       unsigned int pages_pin_count;
> +
>         /**
>          * @madv: State for madvise
>          *
>          * 0 is active/inuse.
> +        * 1 is not-needed/can-be-purged
>          * A negative value is the object is purged.
> -        * Positive values are driver specific and not used by the helpers.
>          */
>         int madv;
>
> @@ -91,6 +102,39 @@ struct drm_gem_shmem_object {
>          * @map_wc: map object write-combined (instead of using shmem defaults).
>          */
>         bool map_wc;
> +
> +       /**
> +        * @eviction_enabled:
> +        *
> +        * The shmem pages can be evicted only if @eviction_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool eviction_enabled;
> +
> +       /**
> +        * @purge_enabled:
> +        *
> +        * The shmem pages can be purged only if @purge_enabled is set to true.
> +        * Used internally by memory shrinker.
> +        */
> +       bool purge_enabled;
> +
> +       /**
> +        * @evicted: True if shmem pages are evicted by the memory shrinker.
> +        * Used internally by memory shrinker.
> +        */
> +       bool evicted;
> +
> +       /**
> +        * @evict:
> +        *
> +        * Invoked by shmem shrinker before evicting shmem GEM from memory.
> +        * GEM's DMA reservation is kept locked by the shrinker. This is
> +        * optional callback that should be specified by drivers.
> +        *
> +        * Returns 0 on success, or -errno on error.
> +        */
> +       int (*evict)(struct drm_gem_shmem_object *shmem);
>  };
>
>  #define to_drm_gem_shmem_obj(obj) \
> @@ -110,14 +154,21 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct
>
>  int drm_gem_shmem_madvise(struct drm_gem_shmem_object *shmem, int madv);
>
> +int drm_gem_shmem_set_purgeable(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_set_evictable(struct drm_gem_shmem_object *shmem);
> +
>  static inline bool drm_gem_shmem_is_purgeable(struct drm_gem_shmem_object *shmem)
>  {
> -       return (shmem->madv > 0) &&
> -               !shmem->vmap_use_count && shmem->sgt &&
> -               !shmem->base.dma_buf && !shmem->base.import_attach;
> +       return (shmem->madv > 0) && shmem->evict &&
> +               shmem->purge_enabled && shmem->pages_use_count &&
> +               !shmem->pages_pin_count && !shmem->base.dma_buf &&
> +               !shmem->base.import_attach && (shmem->sgt || shmem->evicted);
>  }
>
> -void drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_swap_in(struct drm_gem_shmem_object *shmem);
> +
> +int drm_gem_shmem_evict(struct drm_gem_shmem_object *shmem);
> +int drm_gem_shmem_purge(struct drm_gem_shmem_object *shmem);
>
>  struct sg_table *drm_gem_shmem_get_sg_table(struct drm_gem_shmem_object *shmem);
>  struct sg_table *drm_gem_shmem_get_pages_sgt(struct drm_gem_shmem_object *shmem);
> @@ -260,6 +311,32 @@ static inline int drm_gem_shmem_object_mmap(struct drm_gem_object *obj, struct v
>         return drm_gem_shmem_mmap(shmem, vma);
>  }
>
> +/**
> + * struct drm_gem_shmem_shrinker - Generic memory shrinker for shmem GEMs
> + */
> +struct drm_gem_shmem_shrinker {
> +       /** @base: Shrinker for purging shmem GEM objects */
> +       struct shrinker base;
> +
> +       /** @lock: Protects @lru_* */
> +       struct mutex lock;
> +
> +       /** @lru_pinned: List of pinned shmem GEM objects */
> +       struct list_head lru_pinned;
> +
> +       /** @lru_evictable: List of shmem GEM objects to be evicted */
> +       struct list_head lru_evictable;
> +
> +       /** @lru_evicted: List of evicted shmem GEM objects */
> +       struct list_head lru_evicted;
> +
> +       /** @dev: DRM device that uses this shrinker */
> +       struct drm_device *dev;
> +};
> +
> +int drm_gem_shmem_shrinker_register(struct drm_device *dev);
> +void drm_gem_shmem_shrinker_unregister(struct drm_device *dev);
> +
>  /*
>   * Driver ops
>   */
> --
> 2.35.3
>

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Dmitry Osipenko]

20.06.2022 18:37, Rob Clark пишет:
>> +static unsigned long
>> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
>> +                                   struct shrink_control *sc)
>> +{
>> +       unsigned long nr_to_scan = sc->nr_to_scan;
>> +       bool lock_contention = false;
>> +       unsigned long freed;
>> +
>> +       /* purge as many objects as we can */
>> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
>> +                                                       &lock_contention, false);
>> +
>> +       /* evict as many objects as we can */
>> +       if (freed < nr_to_scan)
> oh, one other small note, both in scan_objects() and count_objects(),
> you should check that get_nr_swap_pages()>0 before counting
> evictable/willneed objects.  (And you probably want to keep separate
> LRUs for dontneed vs willneed to accomplish that.)  At least for CrOS,
> inside the VM there is no swap enabled (but instead we rely on zram
> swap in the host.. plus vm-balloon to balance memory pressure between
> host and guest)

Noted, thank you!

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Dmitry Osipenko]

20.06.2022 18:37, Rob Clark пишет:
>> +static unsigned long
>> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
>> +                                   struct shrink_control *sc)
>> +{
>> +       unsigned long nr_to_scan = sc->nr_to_scan;
>> +       bool lock_contention = false;
>> +       unsigned long freed;
>> +
>> +       /* purge as many objects as we can */
>> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
>> +                                                       &lock_contention, false);
>> +
>> +       /* evict as many objects as we can */
>> +       if (freed < nr_to_scan)
> oh, one other small note, both in scan_objects() and count_objects(),
> you should check that get_nr_swap_pages()>0 before counting
> evictable/willneed objects.  (And you probably want to keep separate
> LRUs for dontneed vs willneed to accomplish that.)  At least for CrOS,
> inside the VM there is no swap enabled (but instead we rely on zram
> swap in the host.. plus vm-balloon to balance memory pressure between
> host and guest)

Noted, thank you!

Re: [Intel-gfx] [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Dmitry Osipenko]

20.06.2022 18:37, Rob Clark пишет:
>> +static unsigned long
>> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
>> +                                   struct shrink_control *sc)
>> +{
>> +       unsigned long nr_to_scan = sc->nr_to_scan;
>> +       bool lock_contention = false;
>> +       unsigned long freed;
>> +
>> +       /* purge as many objects as we can */
>> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
>> +                                                       &lock_contention, false);
>> +
>> +       /* evict as many objects as we can */
>> +       if (freed < nr_to_scan)
> oh, one other small note, both in scan_objects() and count_objects(),
> you should check that get_nr_swap_pages()>0 before counting
> evictable/willneed objects.  (And you probably want to keep separate
> LRUs for dontneed vs willneed to accomplish that.)  At least for CrOS,
> inside the VM there is no swap enabled (but instead we rely on zram
> swap in the host.. plus vm-balloon to balance memory pressure between
> host and guest)

Noted, thank you!

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Dmitry Osipenko]

20.06.2022 18:37, Rob Clark пишет:
>> +static unsigned long
>> +drm_gem_shmem_shrinker_scan_objects(struct shrinker *shrinker,
>> +                                   struct shrink_control *sc)
>> +{
>> +       unsigned long nr_to_scan = sc->nr_to_scan;
>> +       bool lock_contention = false;
>> +       unsigned long freed;
>> +
>> +       /* purge as many objects as we can */
>> +       freed = drm_gem_shmem_shrinker_run_objects_scan(shrinker, nr_to_scan,
>> +                                                       &lock_contention, false);
>> +
>> +       /* evict as many objects as we can */
>> +       if (freed < nr_to_scan)
> oh, one other small note, both in scan_objects() and count_objects(),
> you should check that get_nr_swap_pages()>0 before counting
> evictable/willneed objects.  (And you probably want to keep separate
> LRUs for dontneed vs willneed to accomplish that.)  At least for CrOS,
> inside the VM there is no swap enabled (but instead we rely on zram
> swap in the host.. plus vm-balloon to balance memory pressure between
> host and guest)

Noted, thank you!

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Sun, Jun 19, 2022 at 10:53:03AM -0700, Rob Clark wrote:
> On Thu, May 26, 2022 at 4:55 PM Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
> > +       mutex_unlock(&gem_shrinker->lock);
> 
> As I mentioned on other thread, count_objects, being approximate but
> lockless and fast is the important thing.  Otherwise when you start
> hitting the shrinker on many threads, you end up serializing them all,
> even if you have no pages to return to the system at that point.

Yeah agreed, seems like I was wrong here :-) Atomic counter or something
would also be in link the the lru_list stuff.

It would be to record this in the kerneldoc for the shrinker structure
though, to make sure this is all understood.

> > +               /* prevent racing with the dma-buf importing/exporting */
> > +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > +                       *lock_contention |= true;
> > +                       goto resv_unlock;
> > +               }
> 
> I'm not sure this is a good idea to serialize on object_name_lock.
> Purgeable buffers should never be shared (imported or exported).  So
> at best you are avoiding evicting and immediately swapping back in, in
> a rare case, at the cost of serializing multiple threads trying to
> reclaim pages in parallel.

Yeah this sounds really bad. Plus this is a per-device lock, and doing
those with trylock means the shrinker will fail to find shrinkable memory
way too often. We need to engineer this out somehow.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Sun, Jun 19, 2022 at 10:53:03AM -0700, Rob Clark wrote:
> On Thu, May 26, 2022 at 4:55 PM Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
> > +       mutex_unlock(&gem_shrinker->lock);
> 
> As I mentioned on other thread, count_objects, being approximate but
> lockless and fast is the important thing.  Otherwise when you start
> hitting the shrinker on many threads, you end up serializing them all,
> even if you have no pages to return to the system at that point.

Yeah agreed, seems like I was wrong here :-) Atomic counter or something
would also be in link the the lru_list stuff.

It would be to record this in the kerneldoc for the shrinker structure
though, to make sure this is all understood.

> > +               /* prevent racing with the dma-buf importing/exporting */
> > +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > +                       *lock_contention |= true;
> > +                       goto resv_unlock;
> > +               }
> 
> I'm not sure this is a good idea to serialize on object_name_lock.
> Purgeable buffers should never be shared (imported or exported).  So
> at best you are avoiding evicting and immediately swapping back in, in
> a rare case, at the cost of serializing multiple threads trying to
> reclaim pages in parallel.

Yeah this sounds really bad. Plus this is a per-device lock, and doing
those with trylock means the shrinker will fail to find shrinkable memory
way too often. We need to engineer this out somehow.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Sun, Jun 19, 2022 at 10:53:03AM -0700, Rob Clark wrote:
> On Thu, May 26, 2022 at 4:55 PM Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
> > +       mutex_unlock(&gem_shrinker->lock);
> 
> As I mentioned on other thread, count_objects, being approximate but
> lockless and fast is the important thing.  Otherwise when you start
> hitting the shrinker on many threads, you end up serializing them all,
> even if you have no pages to return to the system at that point.

Yeah agreed, seems like I was wrong here :-) Atomic counter or something
would also be in link the the lru_list stuff.

It would be to record this in the kerneldoc for the shrinker structure
though, to make sure this is all understood.

> > +               /* prevent racing with the dma-buf importing/exporting */
> > +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > +                       *lock_contention |= true;
> > +                       goto resv_unlock;
> > +               }
> 
> I'm not sure this is a good idea to serialize on object_name_lock.
> Purgeable buffers should never be shared (imported or exported).  So
> at best you are avoiding evicting and immediately swapping back in, in
> a rare case, at the cost of serializing multiple threads trying to
> reclaim pages in parallel.

Yeah this sounds really bad. Plus this is a per-device lock, and doing
those with trylock means the shrinker will fail to find shrinkable memory
way too often. We need to engineer this out somehow.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [Intel-gfx] [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Sun, Jun 19, 2022 at 10:53:03AM -0700, Rob Clark wrote:
> On Thu, May 26, 2022 at 4:55 PM Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
> > +       mutex_unlock(&gem_shrinker->lock);
> 
> As I mentioned on other thread, count_objects, being approximate but
> lockless and fast is the important thing.  Otherwise when you start
> hitting the shrinker on many threads, you end up serializing them all,
> even if you have no pages to return to the system at that point.

Yeah agreed, seems like I was wrong here :-) Atomic counter or something
would also be in link the the lru_list stuff.

It would be to record this in the kerneldoc for the shrinker structure
though, to make sure this is all understood.

> > +               /* prevent racing with the dma-buf importing/exporting */
> > +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > +                       *lock_contention |= true;
> > +                       goto resv_unlock;
> > +               }
> 
> I'm not sure this is a good idea to serialize on object_name_lock.
> Purgeable buffers should never be shared (imported or exported).  So
> at best you are avoiding evicting and immediately swapping back in, in
> a rare case, at the cost of serializing multiple threads trying to
> reclaim pages in parallel.

Yeah this sounds really bad. Plus this is a per-device lock, and doing
those with trylock means the shrinker will fail to find shrinkable memory
way too often. We need to engineer this out somehow.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Sun, Jun 19, 2022 at 10:53:03AM -0700, Rob Clark wrote:
> On Thu, May 26, 2022 at 4:55 PM Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
> > +       mutex_unlock(&gem_shrinker->lock);
> 
> As I mentioned on other thread, count_objects, being approximate but
> lockless and fast is the important thing.  Otherwise when you start
> hitting the shrinker on many threads, you end up serializing them all,
> even if you have no pages to return to the system at that point.

Yeah agreed, seems like I was wrong here :-) Atomic counter or something
would also be in link the the lru_list stuff.

It would be to record this in the kerneldoc for the shrinker structure
though, to make sure this is all understood.

> > +               /* prevent racing with the dma-buf importing/exporting */
> > +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > +                       *lock_contention |= true;
> > +                       goto resv_unlock;
> > +               }
> 
> I'm not sure this is a good idea to serialize on object_name_lock.
> Purgeable buffers should never be shared (imported or exported).  So
> at best you are avoiding evicting and immediately swapping back in, in
> a rare case, at the cost of serializing multiple threads trying to
> reclaim pages in parallel.

Yeah this sounds really bad. Plus this is a per-device lock, and doing
those with trylock means the shrinker will fail to find shrinkable memory
way too often. We need to engineer this out somehow.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Mon, Jun 20, 2022 at 08:18:04AM -0700, Rob Clark wrote:
> On Mon, Jun 20, 2022 at 7:09 AM Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
> >
> > On 6/19/22 20:53, Rob Clark wrote:
> > ...
> > >> +static unsigned long
> > >> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> > >> +                                    struct shrink_control *sc)
> > >> +{
> > >> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > >> +       struct drm_gem_shmem_object *shmem;
> > >> +       unsigned long count = 0;
> > >> +
> > >> +       if (!mutex_trylock(&gem_shrinker->lock))
> > >> +               return 0;
> > >> +
> > >> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> > >> +               count += shmem->base.size;
> > >> +
> > >> +               if (count >= SHRINK_EMPTY)
> > >> +                       break;
> > >> +       }
> > >> +
> > >> +       mutex_unlock(&gem_shrinker->lock);
> > >
> > > As I mentioned on other thread, count_objects, being approximate but
> > > lockless and fast is the important thing.  Otherwise when you start
> > > hitting the shrinker on many threads, you end up serializing them all,
> > > even if you have no pages to return to the system at that point.
> >
> > Daniel's point for dropping the lockless variant was that we're already
> > in trouble if we're hitting shrinker too often and extra optimizations
> > won't bring much benefits to us.
> 
> At least with zram swap (which I highly recommend using even if you
> are not using a physical swap file/partition), swapin/out is actually
> quite fast.  And if you are leaning on zram swap to fit 8GB of chrome
> browser on a 4GB device, the shrinker gets hit quite a lot.  Lower
> spec (4GB RAM) chromebooks can be under constant memory pressure and
> can quite easily get into a situation where you are hitting the
> shrinker on many threads simultaneously.  So it is pretty important
> for all shrinkers in the system (not just drm driver) to be as
> concurrent as possible.  As long as you avoid serializing reclaim on
> all the threads, performance can still be quite good, but if you don't
> performance will fall off a cliff.
> 
> jfwiw, we are seeing pretty good results (iirc 40-70% increase in open
> tab counts) with the combination of eviction + multigen LRU[1] +
> sizing zram swap to be 2x physical RAM
> 
> [1] https://lwn.net/Articles/856931/
> 
> > Alright, I'll add back the lockless variant (or will use yours
> > drm_gem_lru) in the next revision. The code difference is very small
> > after all.
> >
> > ...
> > >> +               /* prevent racing with the dma-buf importing/exporting */
> > >> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > >> +                       *lock_contention |= true;
> > >> +                       goto resv_unlock;
> > >> +               }
> > >
> > > I'm not sure this is a good idea to serialize on object_name_lock.
> > > Purgeable buffers should never be shared (imported or exported).  So
> > > at best you are avoiding evicting and immediately swapping back in, in
> > > a rare case, at the cost of serializing multiple threads trying to
> > > reclaim pages in parallel.
> >
> > The object_name_lock shouldn't cause contention in practice. But objects
> > are also pinned on attachment, hence maybe this lock is indeed
> > unnecessary.. I'll re-check it.
> 
> I'm not worried about contention with export/import/etc, but
> contention between multiple threads hitting the shrinker in parallel.
> I guess since you are using trylock, it won't *block* the other
> threads hitting shrinker, but they'll just end up looping in
> do_shrink_slab() because they are hitting contention.
> 
> I'd have to do some experiments to see how it works out in practice,
> but my gut feel is that it isn't a good idea

Yeah trylock on anything else than the object lock is No Good in the
shrinker. And it really shouldn't be needed, since import/export should
pin stuff as needed. Which should be protected by the dma_resv object
lock. If not, we need to fix that.

Picking a random drm-internal lock like this is definitely no good design.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Mon, Jun 20, 2022 at 08:18:04AM -0700, Rob Clark wrote:
> On Mon, Jun 20, 2022 at 7:09 AM Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
> >
> > On 6/19/22 20:53, Rob Clark wrote:
> > ...
> > >> +static unsigned long
> > >> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> > >> +                                    struct shrink_control *sc)
> > >> +{
> > >> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > >> +       struct drm_gem_shmem_object *shmem;
> > >> +       unsigned long count = 0;
> > >> +
> > >> +       if (!mutex_trylock(&gem_shrinker->lock))
> > >> +               return 0;
> > >> +
> > >> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> > >> +               count += shmem->base.size;
> > >> +
> > >> +               if (count >= SHRINK_EMPTY)
> > >> +                       break;
> > >> +       }
> > >> +
> > >> +       mutex_unlock(&gem_shrinker->lock);
> > >
> > > As I mentioned on other thread, count_objects, being approximate but
> > > lockless and fast is the important thing.  Otherwise when you start
> > > hitting the shrinker on many threads, you end up serializing them all,
> > > even if you have no pages to return to the system at that point.
> >
> > Daniel's point for dropping the lockless variant was that we're already
> > in trouble if we're hitting shrinker too often and extra optimizations
> > won't bring much benefits to us.
> 
> At least with zram swap (which I highly recommend using even if you
> are not using a physical swap file/partition), swapin/out is actually
> quite fast.  And if you are leaning on zram swap to fit 8GB of chrome
> browser on a 4GB device, the shrinker gets hit quite a lot.  Lower
> spec (4GB RAM) chromebooks can be under constant memory pressure and
> can quite easily get into a situation where you are hitting the
> shrinker on many threads simultaneously.  So it is pretty important
> for all shrinkers in the system (not just drm driver) to be as
> concurrent as possible.  As long as you avoid serializing reclaim on
> all the threads, performance can still be quite good, but if you don't
> performance will fall off a cliff.
> 
> jfwiw, we are seeing pretty good results (iirc 40-70% increase in open
> tab counts) with the combination of eviction + multigen LRU[1] +
> sizing zram swap to be 2x physical RAM
> 
> [1] https://lwn.net/Articles/856931/
> 
> > Alright, I'll add back the lockless variant (or will use yours
> > drm_gem_lru) in the next revision. The code difference is very small
> > after all.
> >
> > ...
> > >> +               /* prevent racing with the dma-buf importing/exporting */
> > >> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > >> +                       *lock_contention |= true;
> > >> +                       goto resv_unlock;
> > >> +               }
> > >
> > > I'm not sure this is a good idea to serialize on object_name_lock.
> > > Purgeable buffers should never be shared (imported or exported).  So
> > > at best you are avoiding evicting and immediately swapping back in, in
> > > a rare case, at the cost of serializing multiple threads trying to
> > > reclaim pages in parallel.
> >
> > The object_name_lock shouldn't cause contention in practice. But objects
> > are also pinned on attachment, hence maybe this lock is indeed
> > unnecessary.. I'll re-check it.
> 
> I'm not worried about contention with export/import/etc, but
> contention between multiple threads hitting the shrinker in parallel.
> I guess since you are using trylock, it won't *block* the other
> threads hitting shrinker, but they'll just end up looping in
> do_shrink_slab() because they are hitting contention.
> 
> I'd have to do some experiments to see how it works out in practice,
> but my gut feel is that it isn't a good idea

Yeah trylock on anything else than the object lock is No Good in the
shrinker. And it really shouldn't be needed, since import/export should
pin stuff as needed. Which should be protected by the dma_resv object
lock. If not, we need to fix that.

Picking a random drm-internal lock like this is definitely no good design.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Mon, Jun 20, 2022 at 08:18:04AM -0700, Rob Clark wrote:
> On Mon, Jun 20, 2022 at 7:09 AM Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
> >
> > On 6/19/22 20:53, Rob Clark wrote:
> > ...
> > >> +static unsigned long
> > >> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> > >> +                                    struct shrink_control *sc)
> > >> +{
> > >> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > >> +       struct drm_gem_shmem_object *shmem;
> > >> +       unsigned long count = 0;
> > >> +
> > >> +       if (!mutex_trylock(&gem_shrinker->lock))
> > >> +               return 0;
> > >> +
> > >> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> > >> +               count += shmem->base.size;
> > >> +
> > >> +               if (count >= SHRINK_EMPTY)
> > >> +                       break;
> > >> +       }
> > >> +
> > >> +       mutex_unlock(&gem_shrinker->lock);
> > >
> > > As I mentioned on other thread, count_objects, being approximate but
> > > lockless and fast is the important thing.  Otherwise when you start
> > > hitting the shrinker on many threads, you end up serializing them all,
> > > even if you have no pages to return to the system at that point.
> >
> > Daniel's point for dropping the lockless variant was that we're already
> > in trouble if we're hitting shrinker too often and extra optimizations
> > won't bring much benefits to us.
> 
> At least with zram swap (which I highly recommend using even if you
> are not using a physical swap file/partition), swapin/out is actually
> quite fast.  And if you are leaning on zram swap to fit 8GB of chrome
> browser on a 4GB device, the shrinker gets hit quite a lot.  Lower
> spec (4GB RAM) chromebooks can be under constant memory pressure and
> can quite easily get into a situation where you are hitting the
> shrinker on many threads simultaneously.  So it is pretty important
> for all shrinkers in the system (not just drm driver) to be as
> concurrent as possible.  As long as you avoid serializing reclaim on
> all the threads, performance can still be quite good, but if you don't
> performance will fall off a cliff.
> 
> jfwiw, we are seeing pretty good results (iirc 40-70% increase in open
> tab counts) with the combination of eviction + multigen LRU[1] +
> sizing zram swap to be 2x physical RAM
> 
> [1] https://lwn.net/Articles/856931/
> 
> > Alright, I'll add back the lockless variant (or will use yours
> > drm_gem_lru) in the next revision. The code difference is very small
> > after all.
> >
> > ...
> > >> +               /* prevent racing with the dma-buf importing/exporting */
> > >> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > >> +                       *lock_contention |= true;
> > >> +                       goto resv_unlock;
> > >> +               }
> > >
> > > I'm not sure this is a good idea to serialize on object_name_lock.
> > > Purgeable buffers should never be shared (imported or exported).  So
> > > at best you are avoiding evicting and immediately swapping back in, in
> > > a rare case, at the cost of serializing multiple threads trying to
> > > reclaim pages in parallel.
> >
> > The object_name_lock shouldn't cause contention in practice. But objects
> > are also pinned on attachment, hence maybe this lock is indeed
> > unnecessary.. I'll re-check it.
> 
> I'm not worried about contention with export/import/etc, but
> contention between multiple threads hitting the shrinker in parallel.
> I guess since you are using trylock, it won't *block* the other
> threads hitting shrinker, but they'll just end up looping in
> do_shrink_slab() because they are hitting contention.
> 
> I'd have to do some experiments to see how it works out in practice,
> but my gut feel is that it isn't a good idea

Yeah trylock on anything else than the object lock is No Good in the
shrinker. And it really shouldn't be needed, since import/export should
pin stuff as needed. Which should be protected by the dma_resv object
lock. If not, we need to fix that.

Picking a random drm-internal lock like this is definitely no good design.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [Intel-gfx] [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Mon, Jun 20, 2022 at 08:18:04AM -0700, Rob Clark wrote:
> On Mon, Jun 20, 2022 at 7:09 AM Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
> >
> > On 6/19/22 20:53, Rob Clark wrote:
> > ...
> > >> +static unsigned long
> > >> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> > >> +                                    struct shrink_control *sc)
> > >> +{
> > >> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > >> +       struct drm_gem_shmem_object *shmem;
> > >> +       unsigned long count = 0;
> > >> +
> > >> +       if (!mutex_trylock(&gem_shrinker->lock))
> > >> +               return 0;
> > >> +
> > >> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> > >> +               count += shmem->base.size;
> > >> +
> > >> +               if (count >= SHRINK_EMPTY)
> > >> +                       break;
> > >> +       }
> > >> +
> > >> +       mutex_unlock(&gem_shrinker->lock);
> > >
> > > As I mentioned on other thread, count_objects, being approximate but
> > > lockless and fast is the important thing.  Otherwise when you start
> > > hitting the shrinker on many threads, you end up serializing them all,
> > > even if you have no pages to return to the system at that point.
> >
> > Daniel's point for dropping the lockless variant was that we're already
> > in trouble if we're hitting shrinker too often and extra optimizations
> > won't bring much benefits to us.
> 
> At least with zram swap (which I highly recommend using even if you
> are not using a physical swap file/partition), swapin/out is actually
> quite fast.  And if you are leaning on zram swap to fit 8GB of chrome
> browser on a 4GB device, the shrinker gets hit quite a lot.  Lower
> spec (4GB RAM) chromebooks can be under constant memory pressure and
> can quite easily get into a situation where you are hitting the
> shrinker on many threads simultaneously.  So it is pretty important
> for all shrinkers in the system (not just drm driver) to be as
> concurrent as possible.  As long as you avoid serializing reclaim on
> all the threads, performance can still be quite good, but if you don't
> performance will fall off a cliff.
> 
> jfwiw, we are seeing pretty good results (iirc 40-70% increase in open
> tab counts) with the combination of eviction + multigen LRU[1] +
> sizing zram swap to be 2x physical RAM
> 
> [1] https://lwn.net/Articles/856931/
> 
> > Alright, I'll add back the lockless variant (or will use yours
> > drm_gem_lru) in the next revision. The code difference is very small
> > after all.
> >
> > ...
> > >> +               /* prevent racing with the dma-buf importing/exporting */
> > >> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > >> +                       *lock_contention |= true;
> > >> +                       goto resv_unlock;
> > >> +               }
> > >
> > > I'm not sure this is a good idea to serialize on object_name_lock.
> > > Purgeable buffers should never be shared (imported or exported).  So
> > > at best you are avoiding evicting and immediately swapping back in, in
> > > a rare case, at the cost of serializing multiple threads trying to
> > > reclaim pages in parallel.
> >
> > The object_name_lock shouldn't cause contention in practice. But objects
> > are also pinned on attachment, hence maybe this lock is indeed
> > unnecessary.. I'll re-check it.
> 
> I'm not worried about contention with export/import/etc, but
> contention between multiple threads hitting the shrinker in parallel.
> I guess since you are using trylock, it won't *block* the other
> threads hitting shrinker, but they'll just end up looping in
> do_shrink_slab() because they are hitting contention.
> 
> I'd have to do some experiments to see how it works out in practice,
> but my gut feel is that it isn't a good idea

Yeah trylock on anything else than the object lock is No Good in the
shrinker. And it really shouldn't be needed, since import/export should
pin stuff as needed. Which should be protected by the dma_resv object
lock. If not, we need to fix that.

Picking a random drm-internal lock like this is definitely no good design.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v6 17/22] drm/shmem-helper: Add generic memory shrinker

[Daniel Vetter]

On Mon, Jun 20, 2022 at 08:18:04AM -0700, Rob Clark wrote:
> On Mon, Jun 20, 2022 at 7:09 AM Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
> >
> > On 6/19/22 20:53, Rob Clark wrote:
> > ...
> > >> +static unsigned long
> > >> +drm_gem_shmem_shrinker_count_objects(struct shrinker *shrinker,
> > >> +                                    struct shrink_control *sc)
> > >> +{
> > >> +       struct drm_gem_shmem_shrinker *gem_shrinker = to_drm_shrinker(shrinker);
> > >> +       struct drm_gem_shmem_object *shmem;
> > >> +       unsigned long count = 0;
> > >> +
> > >> +       if (!mutex_trylock(&gem_shrinker->lock))
> > >> +               return 0;
> > >> +
> > >> +       list_for_each_entry(shmem, &gem_shrinker->lru_evictable, madv_list) {
> > >> +               count += shmem->base.size;
> > >> +
> > >> +               if (count >= SHRINK_EMPTY)
> > >> +                       break;
> > >> +       }
> > >> +
> > >> +       mutex_unlock(&gem_shrinker->lock);
> > >
> > > As I mentioned on other thread, count_objects, being approximate but
> > > lockless and fast is the important thing.  Otherwise when you start
> > > hitting the shrinker on many threads, you end up serializing them all,
> > > even if you have no pages to return to the system at that point.
> >
> > Daniel's point for dropping the lockless variant was that we're already
> > in trouble if we're hitting shrinker too often and extra optimizations
> > won't bring much benefits to us.
> 
> At least with zram swap (which I highly recommend using even if you
> are not using a physical swap file/partition), swapin/out is actually
> quite fast.  And if you are leaning on zram swap to fit 8GB of chrome
> browser on a 4GB device, the shrinker gets hit quite a lot.  Lower
> spec (4GB RAM) chromebooks can be under constant memory pressure and
> can quite easily get into a situation where you are hitting the
> shrinker on many threads simultaneously.  So it is pretty important
> for all shrinkers in the system (not just drm driver) to be as
> concurrent as possible.  As long as you avoid serializing reclaim on
> all the threads, performance can still be quite good, but if you don't
> performance will fall off a cliff.
> 
> jfwiw, we are seeing pretty good results (iirc 40-70% increase in open
> tab counts) with the combination of eviction + multigen LRU[1] +
> sizing zram swap to be 2x physical RAM
> 
> [1] https://lwn.net/Articles/856931/
> 
> > Alright, I'll add back the lockless variant (or will use yours
> > drm_gem_lru) in the next revision. The code difference is very small
> > after all.
> >
> > ...
> > >> +               /* prevent racing with the dma-buf importing/exporting */
> > >> +               if (!mutex_trylock(&gem_shrinker->dev->object_name_lock)) {
> > >> +                       *lock_contention |= true;
> > >> +                       goto resv_unlock;
> > >> +               }
> > >
> > > I'm not sure this is a good idea to serialize on object_name_lock.
> > > Purgeable buffers should never be shared (imported or exported).  So
> > > at best you are avoiding evicting and immediately swapping back in, in
> > > a rare case, at the cost of serializing multiple threads trying to
> > > reclaim pages in parallel.
> >
> > The object_name_lock shouldn't cause contention in practice. But objects
> > are also pinned on attachment, hence maybe this lock is indeed
> > unnecessary.. I'll re-check it.
> 
> I'm not worried about contention with export/import/etc, but
> contention between multiple threads hitting the shrinker in parallel.
> I guess since you are using trylock, it won't *block* the other
> threads hitting shrinker, but they'll just end up looping in
> do_shrink_slab() because they are hitting contention.
> 
> I'd have to do some experiments to see how it works out in practice,
> but my gut feel is that it isn't a good idea

Yeah trylock on anything else than the object lock is No Good in the
shrinker. And it really shouldn't be needed, since import/export should
pin stuff as needed. Which should be protected by the dma_resv object
lock. If not, we need to fix that.

Picking a random drm-internal lock like this is definitely no good design.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Robin Murphy]

On 2022-05-27 00:50, Dmitry Osipenko wrote:
> Hello,
> 
> This patchset introduces memory shrinker for the VirtIO-GPU DRM driver
> and adds memory purging and eviction support to VirtIO-GPU driver.
> 
> The new dma-buf locking convention is introduced here as well.
> 
> During OOM, the shrinker will release BOs that are marked as "not needed"
> by userspace using the new madvise IOCTL, it will also evict idling BOs
> to SWAP. The userspace in this case is the Mesa VirGL driver, it will mark
> the cached BOs as "not needed", allowing kernel driver to release memory
> of the cached shmem BOs on lowmem situations, preventing OOM kills.
> 
> The Panfrost driver is switched to use generic memory shrinker.

I think we still have some outstanding issues here - Alyssa reported 
some weirdness yesterday, so I just tried provoking a low-memory 
condition locally with this series applied and a few debug options 
enabled, and the results as below were... interesting.

Thanks,
Robin.

----->8-----
[   68.295951] ======================================================
[   68.295956] WARNING: possible circular locking dependency detected
[   68.295963] 5.19.0-rc3+ #400 Not tainted
[   68.295972] ------------------------------------------------------
[   68.295977] cc1/295 is trying to acquire lock:
[   68.295986] ffff000008d7f1a0 
(reservation_ww_class_mutex){+.+.}-{3:3}, at: drm_gem_shmem_free+0x7c/0x198
[   68.296036]
[   68.296036] but task is already holding lock:
[   68.296041] ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at: 
__alloc_pages_slowpath.constprop.0+0x4d8/0x1470
[   68.296080]
[   68.296080] which lock already depends on the new lock.
[   68.296080]
[   68.296085]
[   68.296085] the existing dependency chain (in reverse order) is:
[   68.296090]
[   68.296090] -> #1 (fs_reclaim){+.+.}-{0:0}:
[   68.296111]        fs_reclaim_acquire+0xb8/0x150
[   68.296130]        dma_resv_lockdep+0x298/0x3fc
[   68.296148]        do_one_initcall+0xe4/0x5f8
[   68.296163]        kernel_init_freeable+0x414/0x49c
[   68.296180]        kernel_init+0x2c/0x148
[   68.296195]        ret_from_fork+0x10/0x20
[   68.296207]
[   68.296207] -> #0 (reservation_ww_class_mutex){+.+.}-{3:3}:
[   68.296229]        __lock_acquire+0x1724/0x2398
[   68.296246]        lock_acquire+0x218/0x5b0
[   68.296260]        __ww_mutex_lock.constprop.0+0x158/0x2378
[   68.296277]        ww_mutex_lock+0x7c/0x4d8
[   68.296291]        drm_gem_shmem_free+0x7c/0x198
[   68.296304]        panfrost_gem_free_object+0x118/0x138
[   68.296318]        drm_gem_object_free+0x40/0x68
[   68.296334]        drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
[   68.296352]        drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
[   68.296368]        do_shrink_slab+0x220/0x808
[   68.296381]        shrink_slab+0x11c/0x408
[   68.296392]        shrink_node+0x6ac/0xb90
[   68.296403]        do_try_to_free_pages+0x1dc/0x8d0
[   68.296416]        try_to_free_pages+0x1ec/0x5b0
[   68.296429]        __alloc_pages_slowpath.constprop.0+0x528/0x1470
[   68.296444]        __alloc_pages+0x4e0/0x5b8
[   68.296455]        __folio_alloc+0x24/0x60
[   68.296467]        vma_alloc_folio+0xb8/0x2f8
[   68.296483]        alloc_zeroed_user_highpage_movable+0x58/0x68
[   68.296498]        __handle_mm_fault+0x918/0x12a8
[   68.296513]        handle_mm_fault+0x130/0x300
[   68.296527]        do_page_fault+0x1d0/0x568
[   68.296539]        do_translation_fault+0xa0/0xb8
[   68.296551]        do_mem_abort+0x68/0xf8
[   68.296562]        el0_da+0x74/0x100
[   68.296572]        el0t_64_sync_handler+0x68/0xc0
[   68.296585]        el0t_64_sync+0x18c/0x190
[   68.296596]
[   68.296596] other info that might help us debug this:
[   68.296596]
[   68.296601]  Possible unsafe locking scenario:
[   68.296601]
[   68.296604]        CPU0                    CPU1
[   68.296608]        ----                    ----
[   68.296612]   lock(fs_reclaim);
[   68.296622] 
lock(reservation_ww_class_mutex);
[   68.296633]                                lock(fs_reclaim);
[   68.296644]   lock(reservation_ww_class_mutex);
[   68.296654]
[   68.296654]  *** DEADLOCK ***
[   68.296654]
[   68.296658] 3 locks held by cc1/295:
[   68.296666]  #0: ffff00000616e898 (&mm->mmap_lock){++++}-{3:3}, at: 
do_page_fault+0x144/0x568
[   68.296702]  #1: ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at: 
__alloc_pages_slowpath.constprop.0+0x4d8/0x1470
[   68.296740]  #2: ffff80000c1215b0 (shrinker_rwsem){++++}-{3:3}, at: 
shrink_slab+0xc0/0x408
[   68.296774]
[   68.296774] stack backtrace:
[   68.296780] CPU: 2 PID: 295 Comm: cc1 Not tainted 5.19.0-rc3+ #400
[   68.296794] Hardware name: ARM LTD ARM Juno Development Platform/ARM 
Juno Development Platform, BIOS EDK II Sep  3 2019
[   68.296803] Call trace:
[   68.296808]  dump_backtrace+0x1e4/0x1f0
[   68.296821]  show_stack+0x20/0x70
[   68.296832]  dump_stack_lvl+0x8c/0xb8
[   68.296849]  dump_stack+0x1c/0x38
[   68.296864]  print_circular_bug.isra.0+0x284/0x378
[   68.296881]  check_noncircular+0x1d8/0x1f8
[   68.296896]  __lock_acquire+0x1724/0x2398
[   68.296911]  lock_acquire+0x218/0x5b0
[   68.296926]  __ww_mutex_lock.constprop.0+0x158/0x2378
[   68.296942]  ww_mutex_lock+0x7c/0x4d8
[   68.296956]  drm_gem_shmem_free+0x7c/0x198
[   68.296970]  panfrost_gem_free_object+0x118/0x138
[   68.296984]  drm_gem_object_free+0x40/0x68
[   68.296999]  drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
[   68.297017]  drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
[   68.297033]  do_shrink_slab+0x220/0x808
[   68.297045]  shrink_slab+0x11c/0x408
[   68.297056]  shrink_node+0x6ac/0xb90
[   68.297068]  do_try_to_free_pages+0x1dc/0x8d0
[   68.297081]  try_to_free_pages+0x1ec/0x5b0
[   68.297094]  __alloc_pages_slowpath.constprop.0+0x528/0x1470
[   68.297110]  __alloc_pages+0x4e0/0x5b8
[   68.297122]  __folio_alloc+0x24/0x60
[   68.297134]  vma_alloc_folio+0xb8/0x2f8
[   68.297148]  alloc_zeroed_user_highpage_movable+0x58/0x68
[   68.297163]  __handle_mm_fault+0x918/0x12a8
[   68.297178]  handle_mm_fault+0x130/0x300
[   68.297193]  do_page_fault+0x1d0/0x568
[   68.297205]  do_translation_fault+0xa0/0xb8
[   68.297218]  do_mem_abort+0x68/0xf8
[   68.297229]  el0_da+0x74/0x100
[   68.297239]  el0t_64_sync_handler+0x68/0xc0
[   68.297252]  el0t_64_sync+0x18c/0x190
[   68.471812] arm-scmi firmware:scmi: timed out in resp(caller: 
scmi_power_state_set+0x11c/0x190)
[   68.501947] arm-scmi firmware:scmi: Message for 119 type 0 is not 
expected!
[   68.939686] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000915e2d34
[   69.739386] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ac77ac55
[   70.415329] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ee980c7e
[   70.987166] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ffb7ff37
[   71.914939] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000000e92b26e
[   72.426987] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000c036a911
[   73.578683] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000001c6fc094
[   74.090555] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000075d00f9
[   74.922709] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000005add546
[   75.434401] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000000154189b
[   76.394300] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ac77ac55
[   76.906236] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ee980c7e
[   79.657234] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000f6d059fb
[   80.168831] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000061a0f6bf
[   80.808354] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000071ade02
[   81.319967] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000b0afea73
[   81.831574] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000d78f36c2
[   82.343160] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000000f689397
[   83.046689] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000412c2a2f
[   83.558352] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000020e551b3
[   84.261913] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000009437aace
[   84.773576] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000001c6fc094
[   85.317275] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000c036a911
[   85.829035] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000000e92b26e
[   86.660555] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ac77ac55
[   87.172126] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000b940e406
[   87.875846] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000001c6fc094
[   88.387443] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000009437aace
[   89.059175] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000075dadb7f
[   89.570960] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000005add546
[   90.146687] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000cba2873c
[   90.662497] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000a4beb490
[   95.392748] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000005b5fc4ec
[   95.904179] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000a17436ee
[   96.416085] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000003888d2a7
[   96.927874] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000093e04a98
[   97.439742] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000c036a911
[   97.954109] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000084c51113
[   98.467374] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000664663ce
[   98.975192] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000060f2d45c
[   99.487231] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000b29288f8
[   99.998833] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000002f07ab24
[  100.510744] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000008c15c751
[  100.511411] 
==================================================================
[  100.511419] BUG: KASAN: use-after-free in irq_work_single+0xa4/0x110
[  100.511445] Write of size 4 at addr ffff0000107f5830 by task 
glmark2-es2-drm/280
[  100.511458]
[  100.511464] CPU: 1 PID: 280 Comm: glmark2-es2-drm Not tainted 
5.19.0-rc3+ #400
[  100.511479] Hardware name: ARM LTD ARM Juno Development Platform/ARM 
Juno Development Platform, BIOS EDK II Sep  3 2019
[  100.511489] Call trace:
[  100.511494]  dump_backtrace+0x1e4/0x1f0
[  100.511512]  show_stack+0x20/0x70
[  100.511523]  dump_stack_lvl+0x8c/0xb8
[  100.511543]  print_report+0x16c/0x668
[  100.511559]  kasan_report+0x80/0x208
[  100.511574]  kasan_check_range+0x100/0x1b8
[  100.511590]  __kasan_check_write+0x34/0x60
[  100.511607]  irq_work_single+0xa4/0x110
[  100.511619]  irq_work_run_list+0x6c/0x88
[  100.511632]  irq_work_run+0x28/0x48
[  100.511644]  ipi_handler+0x254/0x468
[  100.511664]  handle_percpu_devid_irq+0x11c/0x518
[  100.511681]  generic_handle_domain_irq+0x50/0x70
[  100.511699]  gic_handle_irq+0xd4/0x118
[  100.511711]  call_on_irq_stack+0x2c/0x58
[  100.511725]  do_interrupt_handler+0xc0/0xc8
[  100.511741]  el1_interrupt+0x40/0x68
[  100.511754]  el1h_64_irq_handler+0x18/0x28
[  100.511767]  el1h_64_irq+0x64/0x68
[  100.511778]  irq_work_queue+0xc0/0xd8
[  100.511790]  drm_sched_entity_fini+0x2c4/0x3b0
[  100.511805]  drm_sched_entity_destroy+0x2c/0x40
[  100.511818]  panfrost_job_close+0x44/0x1c0
[  100.511833]  panfrost_postclose+0x38/0x60
[  100.511845]  drm_file_free.part.0+0x33c/0x4b8
[  100.511862]  drm_close_helper.isra.0+0xc0/0xd8
[  100.511877]  drm_release+0xe4/0x1e0
[  100.511891]  __fput+0xf8/0x390
[  100.511904]  ____fput+0x18/0x28
[  100.511917]  task_work_run+0xc4/0x1e0
[  100.511929]  do_exit+0x554/0x1168
[  100.511945]  do_group_exit+0x60/0x108
[  100.511960]  __arm64_sys_exit_group+0x34/0x38
[  100.511977]  invoke_syscall+0x64/0x180
[  100.511993]  el0_svc_common.constprop.0+0x13c/0x170
[  100.512012]  do_el0_svc+0x48/0xe8
[  100.512028]  el0_svc+0x5c/0xe0
[  100.512038]  el0t_64_sync_handler+0xb8/0xc0
[  100.512051]  el0t_64_sync+0x18c/0x190
[  100.512064]
[  100.512068] Allocated by task 280:
[  100.512075]  kasan_save_stack+0x2c/0x58
[  100.512091]  __kasan_kmalloc+0x90/0xb8
[  100.512105]  kmem_cache_alloc_trace+0x1d4/0x330
[  100.512118]  panfrost_ioctl_submit+0x100/0x630
[  100.512131]  drm_ioctl_kernel+0x160/0x250
[  100.512147]  drm_ioctl+0x36c/0x628
[  100.512161]  __arm64_sys_ioctl+0xd8/0x120
[  100.512178]  invoke_syscall+0x64/0x180
[  100.512194]  el0_svc_common.constprop.0+0x13c/0x170
[  100.512211]  do_el0_svc+0x48/0xe8
[  100.512226]  el0_svc+0x5c/0xe0
[  100.512236]  el0t_64_sync_handler+0xb8/0xc0
[  100.512248]  el0t_64_sync+0x18c/0x190
[  100.512259]
[  100.512262] Freed by task 280:
[  100.512268]  kasan_save_stack+0x2c/0x58
[  100.512283]  kasan_set_track+0x2c/0x40
[  100.512296]  kasan_set_free_info+0x28/0x50
[  100.512312]  __kasan_slab_free+0xf0/0x170
[  100.512326]  kfree+0x124/0x418
[  100.512337]  panfrost_job_cleanup+0x1f0/0x298
[  100.512350]  panfrost_job_free+0x80/0xb0
[  100.512363]  drm_sched_entity_kill_jobs_irq_work+0x80/0xa0
[  100.512377]  irq_work_single+0x88/0x110
[  100.512389]  irq_work_run_list+0x6c/0x88
[  100.512401]  irq_work_run+0x28/0x48
[  100.512413]  ipi_handler+0x254/0x468
[  100.512427]  handle_percpu_devid_irq+0x11c/0x518
[  100.512443]  generic_handle_domain_irq+0x50/0x70
[  100.512460]  gic_handle_irq+0xd4/0x118
[  100.512471]
[  100.512474] The buggy address belongs to the object at ffff0000107f5800
[  100.512474]  which belongs to the cache kmalloc-512 of size 512
[  100.512484] The buggy address is located 48 bytes inside of
[  100.512484]  512-byte region [ffff0000107f5800, ffff0000107f5a00)
[  100.512497]
[  100.512500] The buggy address belongs to the physical page:
[  100.512506] page:000000000a626feb refcount:1 mapcount:0 
mapping:0000000000000000 index:0x0 pfn:0x907f4
[  100.512520] head:000000000a626feb order:2 compound_mapcount:0 
compound_pincount:0
[  100.512530] flags: 
0xffff00000010200(slab|head|node=0|zone=0|lastcpupid=0xffff)
[  100.512556] raw: 0ffff00000010200 fffffc0000076400 dead000000000002 
ffff000000002600
[  100.512569] raw: 0000000000000000 0000000080100010 00000001ffffffff 
0000000000000000
[  100.512577] page dumped because: kasan: bad access detected
[  100.512582]
[  100.512585] Memory state around the buggy address:
[  100.512592]  ffff0000107f5700: fc fc fc fc fc fc fc fc fc fc fc fc fc 
fc fc fc
[  100.512602]  ffff0000107f5780: fc fc fc fc fc fc fc fc fc fc fc fc fc 
fc fc fc
[  100.512612] >ffff0000107f5800: fa fb fb fb fb fb fb fb fb fb fb fb fb 
fb fb fb
[  100.512619]                                      ^
[  100.512627]  ffff0000107f5880: fb fb fb fb fb fb fb fb fb fb fb fb fb 
fb fb fb
[  100.512636]  ffff0000107f5900: fb fb fb fb fb fb fb fb fb fb fb fb fb 
fb fb fb
[  100.512643] 
==================================================================
[  101.022573] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000be4b1b31
[  101.534469] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000a8ff2c8a
[  101.535981] BUG: sleeping function called from invalid context at 
kernel/locking/mutex.c:870
[  101.535994] in_atomic(): 1, irqs_disabled(): 128, non_block: 0, pid: 
280, name: glmark2-es2-drm
[  101.536006] preempt_count: 10000, expected: 0
[  101.536012] RCU nest depth: 0, expected: 0
[  101.536019] INFO: lockdep is turned off.
[  101.536023] irq event stamp: 1666508
[  101.536029] hardirqs last  enabled at (1666507): [<ffff80000997ed70>] 
exit_to_kernel_mode.isra.0+0x40/0x140
[  101.536056] hardirqs last disabled at (1666508): [<ffff800009985030>] 
__schedule+0xb38/0xea8
[  101.536076] softirqs last  enabled at (1664950): [<ffff800008010ac8>] 
__do_softirq+0x6b8/0x89c
[  101.536092] softirqs last disabled at (1664941): [<ffff8000080e4fdc>] 
irq_exit_rcu+0x27c/0x2b0
[  101.536118] CPU: 1 PID: 280 Comm: glmark2-es2-drm Tainted: G    B 
          5.19.0-rc3+ #400
[  101.536134] Hardware name: ARM LTD ARM Juno Development Platform/ARM 
Juno Development Platform, BIOS EDK II Sep  3 2019
[  101.536143] Call trace:
[  101.536147]  dump_backtrace+0x1e4/0x1f0
[  101.536161]  show_stack+0x20/0x70
[  101.536171]  dump_stack_lvl+0x8c/0xb8
[  101.536189]  dump_stack+0x1c/0x38
[  101.536204]  __might_resched+0x1f0/0x2b0
[  101.536220]  __might_sleep+0x74/0xd0
[  101.536234]  ww_mutex_lock+0x40/0x4d8
[  101.536249]  drm_gem_shmem_free+0x7c/0x198
[  101.536264]  panfrost_gem_free_object+0x118/0x138
[  101.536278]  drm_gem_object_free+0x40/0x68
[  101.536295]  panfrost_job_cleanup+0x1bc/0x298
[  101.536309]  panfrost_job_free+0x80/0xb0
[  101.536322]  drm_sched_entity_kill_jobs_irq_work+0x80/0xa0
[  101.536337]  irq_work_single+0x88/0x110
[  101.536351]  irq_work_run_list+0x6c/0x88
[  101.536364]  irq_work_run+0x28/0x48
[  101.536375]  ipi_handler+0x254/0x468
[  101.536392]  handle_percpu_devid_irq+0x11c/0x518
[  101.536409]  generic_handle_domain_irq+0x50/0x70
[  101.536428]  gic_handle_irq+0xd4/0x118
[  101.536439]  call_on_irq_stack+0x2c/0x58
[  101.536453]  do_interrupt_handler+0xc0/0xc8
[  101.536468]  el1_interrupt+0x40/0x68
[  101.536479]  el1h_64_irq_handler+0x18/0x28
[  101.536492]  el1h_64_irq+0x64/0x68
[  101.536503]  __asan_load8+0x30/0xd0
[  101.536519]  drm_sched_entity_fini+0x1e8/0x3b0
[  101.536532]  drm_sched_entity_destroy+0x2c/0x40
[  101.536545]  panfrost_job_close+0x44/0x1c0
[  101.536559]  panfrost_postclose+0x38/0x60
[  101.536571]  drm_file_free.part.0+0x33c/0x4b8
[  101.536586]  drm_close_helper.isra.0+0xc0/0xd8
[  101.536601]  drm_release+0xe4/0x1e0
[  101.536615]  __fput+0xf8/0x390
[  101.536628]  ____fput+0x18/0x28
[  101.536640]  task_work_run+0xc4/0x1e0
[  101.536652]  do_exit+0x554/0x1168
[  101.536667]  do_group_exit+0x60/0x108
[  101.536682]  __arm64_sys_exit_group+0x34/0x38
[  101.536698]  invoke_syscall+0x64/0x180
[  101.536714]  el0_svc_common.constprop.0+0x13c/0x170
[  101.536733]  do_el0_svc+0x48/0xe8
[  101.536748]  el0_svc+0x5c/0xe0
[  101.536759]  el0t_64_sync_handler+0xb8/0xc0
[  101.536771]  el0t_64_sync+0x18c/0x190
[  101.541928] ------------[ cut here ]------------
[  101.541934] kernel BUG at kernel/irq_work.c:235!
[  101.541944] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
[  101.541961] Modules linked in:
[  101.541978] CPU: 1 PID: 280 Comm: glmark2-es2-drm Tainted: G    B   W 
         5.19.0-rc3+ #400
[  101.541997] Hardware name: ARM LTD ARM Juno Development Platform/ARM 
Juno Development Platform, BIOS EDK II Sep  3 2019
[  101.542009] pstate: 40000005 (nZcv daif -PAN -UAO -TCO -DIT -SSBS 
BTYPE=--)
[  101.542027] pc : irq_work_run_list+0x80/0x88
[  101.542044] lr : irq_work_run+0x34/0x48
[  101.542060] sp : ffff80000da37eb0
[  101.542069] x29: ffff80000da37eb0 x28: ffff000006bb0000 x27: 
ffff000006bb0008
[  101.542107] x26: ffff80000da37f20 x25: ffff8000080304d8 x24: 
0000000000000001
[  101.542142] x23: ffff80000abcd008 x22: ffff80000da37ed0 x21: 
ffff80001c0de000
[  101.542177] x20: ffff80000abcd008 x19: ffff80000abdbad0 x18: 
0000000000000000
[  101.542212] x17: 616e202c30383220 x16: 3a646970202c3020 x15: 
ffff8000082df9d0
[  101.542246] x14: ffff800008dfada8 x13: 0000000000000003 x12: 
1fffe000018b2a06
[  101.542280] x11: ffff6000018b2a06 x10: dfff800000000000 x9 : 
ffff00000c595033
[  101.542315] x8 : ffff6000018b2a07 x7 : 0000000000000001 x6 : 
00000000000000fb
[  101.542349] x5 : ffff00000c595030 x4 : 0000000000000000 x3 : 
ffff00000c595030
[  101.542382] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 
ffff000026cb9ad0
[  101.542416] Call trace:
[  101.542424]  irq_work_run_list+0x80/0x88
[  101.542441]  ipi_handler+0x254/0x468
[  101.542460]  handle_percpu_devid_irq+0x11c/0x518
[  101.542480]  generic_handle_domain_irq+0x50/0x70
[  101.542501]  gic_handle_irq+0xd4/0x118
[  101.542516]  call_on_irq_stack+0x2c/0x58
[  101.542534]  do_interrupt_handler+0xc0/0xc8
[  101.542553]  el1_interrupt+0x40/0x68
[  101.542568]  el1h_64_irq_handler+0x18/0x28
[  101.542584]  el1h_64_irq+0x64/0x68
[  101.542599]  __asan_load8+0x30/0xd0
[  101.542617]  drm_sched_entity_fini+0x1e8/0x3b0
[  101.542634]  drm_sched_entity_destroy+0x2c/0x40
[  101.542651]  panfrost_job_close+0x44/0x1c0
[  101.542669]  panfrost_postclose+0x38/0x60
[  101.542685]  drm_file_free.part.0+0x33c/0x4b8
[  101.542704]  drm_close_helper.isra.0+0xc0/0xd8
[  101.542723]  drm_release+0xe4/0x1e0
[  101.542740]  __fput+0xf8/0x390
[  101.542756]  ____fput+0x18/0x28
[  101.542773]  task_work_run+0xc4/0x1e0
[  101.542788]  do_exit+0x554/0x1168
[  101.542806]  do_group_exit+0x60/0x108
[  101.542825]  __arm64_sys_exit_group+0x34/0x38
[  101.542845]  invoke_syscall+0x64/0x180
[  101.542865]  el0_svc_common.constprop.0+0x13c/0x170
[  101.542887]  do_el0_svc+0x48/0xe8
[  101.542906]  el0_svc+0x5c/0xe0
[  101.542921]  el0t_64_sync_handler+0xb8/0xc0
[  101.542938]  el0t_64_sync+0x18c/0x190
[  101.542960] Code: a94153f3 a8c27bfd d50323bf d65f03c0 (d4210000)
[  101.542979] ---[ end trace 0000000000000000 ]---
[  101.678650] Kernel panic - not syncing: Oops - BUG: Fatal exception 
in interrupt
[  102.046301] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000001da14c98
[  103.227334] SMP: stopping secondary CPUs
[  103.241055] Kernel Offset: disabled
[  103.254316] CPU features: 0x800,00184810,00001086
[  103.268904] Memory Limit: 800 MB
[  103.411625] ---[ end Kernel panic - not syncing: Oops - BUG: Fatal 
exception in interrupt ]---

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Robin Murphy]

On 2022-05-27 00:50, Dmitry Osipenko wrote:
> Hello,
> 
> This patchset introduces memory shrinker for the VirtIO-GPU DRM driver
> and adds memory purging and eviction support to VirtIO-GPU driver.
> 
> The new dma-buf locking convention is introduced here as well.
> 
> During OOM, the shrinker will release BOs that are marked as "not needed"
> by userspace using the new madvise IOCTL, it will also evict idling BOs
> to SWAP. The userspace in this case is the Mesa VirGL driver, it will mark
> the cached BOs as "not needed", allowing kernel driver to release memory
> of the cached shmem BOs on lowmem situations, preventing OOM kills.
> 
> The Panfrost driver is switched to use generic memory shrinker.

I think we still have some outstanding issues here - Alyssa reported 
some weirdness yesterday, so I just tried provoking a low-memory 
condition locally with this series applied and a few debug options 
enabled, and the results as below were... interesting.

Thanks,
Robin.

----->8-----
[   68.295951] ======================================================
[   68.295956] WARNING: possible circular locking dependency detected
[   68.295963] 5.19.0-rc3+ #400 Not tainted
[   68.295972] ------------------------------------------------------
[   68.295977] cc1/295 is trying to acquire lock:
[   68.295986] ffff000008d7f1a0 
(reservation_ww_class_mutex){+.+.}-{3:3}, at: drm_gem_shmem_free+0x7c/0x198
[   68.296036]
[   68.296036] but task is already holding lock:
[   68.296041] ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at: 
__alloc_pages_slowpath.constprop.0+0x4d8/0x1470
[   68.296080]
[   68.296080] which lock already depends on the new lock.
[   68.296080]
[   68.296085]
[   68.296085] the existing dependency chain (in reverse order) is:
[   68.296090]
[   68.296090] -> #1 (fs_reclaim){+.+.}-{0:0}:
[   68.296111]        fs_reclaim_acquire+0xb8/0x150
[   68.296130]        dma_resv_lockdep+0x298/0x3fc
[   68.296148]        do_one_initcall+0xe4/0x5f8
[   68.296163]        kernel_init_freeable+0x414/0x49c
[   68.296180]        kernel_init+0x2c/0x148
[   68.296195]        ret_from_fork+0x10/0x20
[   68.296207]
[   68.296207] -> #0 (reservation_ww_class_mutex){+.+.}-{3:3}:
[   68.296229]        __lock_acquire+0x1724/0x2398
[   68.296246]        lock_acquire+0x218/0x5b0
[   68.296260]        __ww_mutex_lock.constprop.0+0x158/0x2378
[   68.296277]        ww_mutex_lock+0x7c/0x4d8
[   68.296291]        drm_gem_shmem_free+0x7c/0x198
[   68.296304]        panfrost_gem_free_object+0x118/0x138
[   68.296318]        drm_gem_object_free+0x40/0x68
[   68.296334]        drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
[   68.296352]        drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
[   68.296368]        do_shrink_slab+0x220/0x808
[   68.296381]        shrink_slab+0x11c/0x408
[   68.296392]        shrink_node+0x6ac/0xb90
[   68.296403]        do_try_to_free_pages+0x1dc/0x8d0
[   68.296416]        try_to_free_pages+0x1ec/0x5b0
[   68.296429]        __alloc_pages_slowpath.constprop.0+0x528/0x1470
[   68.296444]        __alloc_pages+0x4e0/0x5b8
[   68.296455]        __folio_alloc+0x24/0x60
[   68.296467]        vma_alloc_folio+0xb8/0x2f8
[   68.296483]        alloc_zeroed_user_highpage_movable+0x58/0x68
[   68.296498]        __handle_mm_fault+0x918/0x12a8
[   68.296513]        handle_mm_fault+0x130/0x300
[   68.296527]        do_page_fault+0x1d0/0x568
[   68.296539]        do_translation_fault+0xa0/0xb8
[   68.296551]        do_mem_abort+0x68/0xf8
[   68.296562]        el0_da+0x74/0x100
[   68.296572]        el0t_64_sync_handler+0x68/0xc0
[   68.296585]        el0t_64_sync+0x18c/0x190
[   68.296596]
[   68.296596] other info that might help us debug this:
[   68.296596]
[   68.296601]  Possible unsafe locking scenario:
[   68.296601]
[   68.296604]        CPU0                    CPU1
[   68.296608]        ----                    ----
[   68.296612]   lock(fs_reclaim);
[   68.296622] 
lock(reservation_ww_class_mutex);
[   68.296633]                                lock(fs_reclaim);
[   68.296644]   lock(reservation_ww_class_mutex);
[   68.296654]
[   68.296654]  *** DEADLOCK ***
[   68.296654]
[   68.296658] 3 locks held by cc1/295:
[   68.296666]  #0: ffff00000616e898 (&mm->mmap_lock){++++}-{3:3}, at: 
do_page_fault+0x144/0x568
[   68.296702]  #1: ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at: 
__alloc_pages_slowpath.constprop.0+0x4d8/0x1470
[   68.296740]  #2: ffff80000c1215b0 (shrinker_rwsem){++++}-{3:3}, at: 
shrink_slab+0xc0/0x408
[   68.296774]
[   68.296774] stack backtrace:
[   68.296780] CPU: 2 PID: 295 Comm: cc1 Not tainted 5.19.0-rc3+ #400
[   68.296794] Hardware name: ARM LTD ARM Juno Development Platform/ARM 
Juno Development Platform, BIOS EDK II Sep  3 2019
[   68.296803] Call trace:
[   68.296808]  dump_backtrace+0x1e4/0x1f0
[   68.296821]  show_stack+0x20/0x70
[   68.296832]  dump_stack_lvl+0x8c/0xb8
[   68.296849]  dump_stack+0x1c/0x38
[   68.296864]  print_circular_bug.isra.0+0x284/0x378
[   68.296881]  check_noncircular+0x1d8/0x1f8
[   68.296896]  __lock_acquire+0x1724/0x2398
[   68.296911]  lock_acquire+0x218/0x5b0
[   68.296926]  __ww_mutex_lock.constprop.0+0x158/0x2378
[   68.296942]  ww_mutex_lock+0x7c/0x4d8
[   68.296956]  drm_gem_shmem_free+0x7c/0x198
[   68.296970]  panfrost_gem_free_object+0x118/0x138
[   68.296984]  drm_gem_object_free+0x40/0x68
[   68.296999]  drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
[   68.297017]  drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
[   68.297033]  do_shrink_slab+0x220/0x808
[   68.297045]  shrink_slab+0x11c/0x408
[   68.297056]  shrink_node+0x6ac/0xb90
[   68.297068]  do_try_to_free_pages+0x1dc/0x8d0
[   68.297081]  try_to_free_pages+0x1ec/0x5b0
[   68.297094]  __alloc_pages_slowpath.constprop.0+0x528/0x1470
[   68.297110]  __alloc_pages+0x4e0/0x5b8
[   68.297122]  __folio_alloc+0x24/0x60
[   68.297134]  vma_alloc_folio+0xb8/0x2f8
[   68.297148]  alloc_zeroed_user_highpage_movable+0x58/0x68
[   68.297163]  __handle_mm_fault+0x918/0x12a8
[   68.297178]  handle_mm_fault+0x130/0x300
[   68.297193]  do_page_fault+0x1d0/0x568
[   68.297205]  do_translation_fault+0xa0/0xb8
[   68.297218]  do_mem_abort+0x68/0xf8
[   68.297229]  el0_da+0x74/0x100
[   68.297239]  el0t_64_sync_handler+0x68/0xc0
[   68.297252]  el0t_64_sync+0x18c/0x190
[   68.471812] arm-scmi firmware:scmi: timed out in resp(caller: 
scmi_power_state_set+0x11c/0x190)
[   68.501947] arm-scmi firmware:scmi: Message for 119 type 0 is not 
expected!
[   68.939686] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000915e2d34
[   69.739386] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ac77ac55
[   70.415329] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ee980c7e
[   70.987166] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ffb7ff37
[   71.914939] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000000e92b26e
[   72.426987] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000c036a911
[   73.578683] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000001c6fc094
[   74.090555] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000075d00f9
[   74.922709] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000005add546
[   75.434401] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000000154189b
[   76.394300] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ac77ac55
[   76.906236] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ee980c7e
[   79.657234] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000f6d059fb
[   80.168831] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000061a0f6bf
[   80.808354] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000071ade02
[   81.319967] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000b0afea73
[   81.831574] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000d78f36c2
[   82.343160] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000000f689397
[   83.046689] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000412c2a2f
[   83.558352] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000020e551b3
[   84.261913] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000009437aace
[   84.773576] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000001c6fc094
[   85.317275] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000c036a911
[   85.829035] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000000e92b26e
[   86.660555] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ac77ac55
[   87.172126] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000b940e406
[   87.875846] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000001c6fc094
[   88.387443] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000009437aace
[   89.059175] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000075dadb7f
[   89.570960] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000005add546
[   90.146687] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000cba2873c
[   90.662497] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000a4beb490
[   95.392748] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000005b5fc4ec
[   95.904179] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000a17436ee
[   96.416085] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000003888d2a7
[   96.927874] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000093e04a98
[   97.439742] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000c036a911
[   97.954109] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000084c51113
[   98.467374] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000664663ce
[   98.975192] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000060f2d45c
[   99.487231] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000b29288f8
[   99.998833] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000002f07ab24
[  100.510744] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000008c15c751
[  100.511411] 
==================================================================
[  100.511419] BUG: KASAN: use-after-free in irq_work_single+0xa4/0x110
[  100.511445] Write of size 4 at addr ffff0000107f5830 by task 
glmark2-es2-drm/280
[  100.511458]
[  100.511464] CPU: 1 PID: 280 Comm: glmark2-es2-drm Not tainted 
5.19.0-rc3+ #400
[  100.511479] Hardware name: ARM LTD ARM Juno Development Platform/ARM 
Juno Development Platform, BIOS EDK II Sep  3 2019
[  100.511489] Call trace:
[  100.511494]  dump_backtrace+0x1e4/0x1f0
[  100.511512]  show_stack+0x20/0x70
[  100.511523]  dump_stack_lvl+0x8c/0xb8
[  100.511543]  print_report+0x16c/0x668
[  100.511559]  kasan_report+0x80/0x208
[  100.511574]  kasan_check_range+0x100/0x1b8
[  100.511590]  __kasan_check_write+0x34/0x60
[  100.511607]  irq_work_single+0xa4/0x110
[  100.511619]  irq_work_run_list+0x6c/0x88
[  100.511632]  irq_work_run+0x28/0x48
[  100.511644]  ipi_handler+0x254/0x468
[  100.511664]  handle_percpu_devid_irq+0x11c/0x518
[  100.511681]  generic_handle_domain_irq+0x50/0x70
[  100.511699]  gic_handle_irq+0xd4/0x118
[  100.511711]  call_on_irq_stack+0x2c/0x58
[  100.511725]  do_interrupt_handler+0xc0/0xc8
[  100.511741]  el1_interrupt+0x40/0x68
[  100.511754]  el1h_64_irq_handler+0x18/0x28
[  100.511767]  el1h_64_irq+0x64/0x68
[  100.511778]  irq_work_queue+0xc0/0xd8
[  100.511790]  drm_sched_entity_fini+0x2c4/0x3b0
[  100.511805]  drm_sched_entity_destroy+0x2c/0x40
[  100.511818]  panfrost_job_close+0x44/0x1c0
[  100.511833]  panfrost_postclose+0x38/0x60
[  100.511845]  drm_file_free.part.0+0x33c/0x4b8
[  100.511862]  drm_close_helper.isra.0+0xc0/0xd8
[  100.511877]  drm_release+0xe4/0x1e0
[  100.511891]  __fput+0xf8/0x390
[  100.511904]  ____fput+0x18/0x28
[  100.511917]  task_work_run+0xc4/0x1e0
[  100.511929]  do_exit+0x554/0x1168
[  100.511945]  do_group_exit+0x60/0x108
[  100.511960]  __arm64_sys_exit_group+0x34/0x38
[  100.511977]  invoke_syscall+0x64/0x180
[  100.511993]  el0_svc_common.constprop.0+0x13c/0x170
[  100.512012]  do_el0_svc+0x48/0xe8
[  100.512028]  el0_svc+0x5c/0xe0
[  100.512038]  el0t_64_sync_handler+0xb8/0xc0
[  100.512051]  el0t_64_sync+0x18c/0x190
[  100.512064]
[  100.512068] Allocated by task 280:
[  100.512075]  kasan_save_stack+0x2c/0x58
[  100.512091]  __kasan_kmalloc+0x90/0xb8
[  100.512105]  kmem_cache_alloc_trace+0x1d4/0x330
[  100.512118]  panfrost_ioctl_submit+0x100/0x630
[  100.512131]  drm_ioctl_kernel+0x160/0x250
[  100.512147]  drm_ioctl+0x36c/0x628
[  100.512161]  __arm64_sys_ioctl+0xd8/0x120
[  100.512178]  invoke_syscall+0x64/0x180
[  100.512194]  el0_svc_common.constprop.0+0x13c/0x170
[  100.512211]  do_el0_svc+0x48/0xe8
[  100.512226]  el0_svc+0x5c/0xe0
[  100.512236]  el0t_64_sync_handler+0xb8/0xc0
[  100.512248]  el0t_64_sync+0x18c/0x190
[  100.512259]
[  100.512262] Freed by task 280:
[  100.512268]  kasan_save_stack+0x2c/0x58
[  100.512283]  kasan_set_track+0x2c/0x40
[  100.512296]  kasan_set_free_info+0x28/0x50
[  100.512312]  __kasan_slab_free+0xf0/0x170
[  100.512326]  kfree+0x124/0x418
[  100.512337]  panfrost_job_cleanup+0x1f0/0x298
[  100.512350]  panfrost_job_free+0x80/0xb0
[  100.512363]  drm_sched_entity_kill_jobs_irq_work+0x80/0xa0
[  100.512377]  irq_work_single+0x88/0x110
[  100.512389]  irq_work_run_list+0x6c/0x88
[  100.512401]  irq_work_run+0x28/0x48
[  100.512413]  ipi_handler+0x254/0x468
[  100.512427]  handle_percpu_devid_irq+0x11c/0x518
[  100.512443]  generic_handle_domain_irq+0x50/0x70
[  100.512460]  gic_handle_irq+0xd4/0x118
[  100.512471]
[  100.512474] The buggy address belongs to the object at ffff0000107f5800
[  100.512474]  which belongs to the cache kmalloc-512 of size 512
[  100.512484] The buggy address is located 48 bytes inside of
[  100.512484]  512-byte region [ffff0000107f5800, ffff0000107f5a00)
[  100.512497]
[  100.512500] The buggy address belongs to the physical page:
[  100.512506] page:000000000a626feb refcount:1 mapcount:0 
mapping:0000000000000000 index:0x0 pfn:0x907f4
[  100.512520] head:000000000a626feb order:2 compound_mapcount:0 
compound_pincount:0
[  100.512530] flags: 
0xffff00000010200(slab|head|node=0|zone=0|lastcpupid=0xffff)
[  100.512556] raw: 0ffff00000010200 fffffc0000076400 dead000000000002 
ffff000000002600
[  100.512569] raw: 0000000000000000 0000000080100010 00000001ffffffff 
0000000000000000
[  100.512577] page dumped because: kasan: bad access detected
[  100.512582]
[  100.512585] Memory state around the buggy address:
[  100.512592]  ffff0000107f5700: fc fc fc fc fc fc fc fc fc fc fc fc fc 
fc fc fc
[  100.512602]  ffff0000107f5780: fc fc fc fc fc fc fc fc fc fc fc fc fc 
fc fc fc
[  100.512612] >ffff0000107f5800: fa fb fb fb fb fb fb fb fb fb fb fb fb 
fb fb fb
[  100.512619]                                      ^
[  100.512627]  ffff0000107f5880: fb fb fb fb fb fb fb fb fb fb fb fb fb 
fb fb fb
[  100.512636]  ffff0000107f5900: fb fb fb fb fb fb fb fb fb fb fb fb fb 
fb fb fb
[  100.512643] 
==================================================================
[  101.022573] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000be4b1b31
[  101.534469] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000a8ff2c8a
[  101.535981] BUG: sleeping function called from invalid context at 
kernel/locking/mutex.c:870
[  101.535994] in_atomic(): 1, irqs_disabled(): 128, non_block: 0, pid: 
280, name: glmark2-es2-drm
[  101.536006] preempt_count: 10000, expected: 0
[  101.536012] RCU nest depth: 0, expected: 0
[  101.536019] INFO: lockdep is turned off.
[  101.536023] irq event stamp: 1666508
[  101.536029] hardirqs last  enabled at (1666507): [<ffff80000997ed70>] 
exit_to_kernel_mode.isra.0+0x40/0x140
[  101.536056] hardirqs last disabled at (1666508): [<ffff800009985030>] 
__schedule+0xb38/0xea8
[  101.536076] softirqs last  enabled at (1664950): [<ffff800008010ac8>] 
__do_softirq+0x6b8/0x89c
[  101.536092] softirqs last disabled at (1664941): [<ffff8000080e4fdc>] 
irq_exit_rcu+0x27c/0x2b0
[  101.536118] CPU: 1 PID: 280 Comm: glmark2-es2-drm Tainted: G    B 
          5.19.0-rc3+ #400
[  101.536134] Hardware name: ARM LTD ARM Juno Development Platform/ARM 
Juno Development Platform, BIOS EDK II Sep  3 2019
[  101.536143] Call trace:
[  101.536147]  dump_backtrace+0x1e4/0x1f0
[  101.536161]  show_stack+0x20/0x70
[  101.536171]  dump_stack_lvl+0x8c/0xb8
[  101.536189]  dump_stack+0x1c/0x38
[  101.536204]  __might_resched+0x1f0/0x2b0
[  101.536220]  __might_sleep+0x74/0xd0
[  101.536234]  ww_mutex_lock+0x40/0x4d8
[  101.536249]  drm_gem_shmem_free+0x7c/0x198
[  101.536264]  panfrost_gem_free_object+0x118/0x138
[  101.536278]  drm_gem_object_free+0x40/0x68
[  101.536295]  panfrost_job_cleanup+0x1bc/0x298
[  101.536309]  panfrost_job_free+0x80/0xb0
[  101.536322]  drm_sched_entity_kill_jobs_irq_work+0x80/0xa0
[  101.536337]  irq_work_single+0x88/0x110
[  101.536351]  irq_work_run_list+0x6c/0x88
[  101.536364]  irq_work_run+0x28/0x48
[  101.536375]  ipi_handler+0x254/0x468
[  101.536392]  handle_percpu_devid_irq+0x11c/0x518
[  101.536409]  generic_handle_domain_irq+0x50/0x70
[  101.536428]  gic_handle_irq+0xd4/0x118
[  101.536439]  call_on_irq_stack+0x2c/0x58
[  101.536453]  do_interrupt_handler+0xc0/0xc8
[  101.536468]  el1_interrupt+0x40/0x68
[  101.536479]  el1h_64_irq_handler+0x18/0x28
[  101.536492]  el1h_64_irq+0x64/0x68
[  101.536503]  __asan_load8+0x30/0xd0
[  101.536519]  drm_sched_entity_fini+0x1e8/0x3b0
[  101.536532]  drm_sched_entity_destroy+0x2c/0x40
[  101.536545]  panfrost_job_close+0x44/0x1c0
[  101.536559]  panfrost_postclose+0x38/0x60
[  101.536571]  drm_file_free.part.0+0x33c/0x4b8
[  101.536586]  drm_close_helper.isra.0+0xc0/0xd8
[  101.536601]  drm_release+0xe4/0x1e0
[  101.536615]  __fput+0xf8/0x390
[  101.536628]  ____fput+0x18/0x28
[  101.536640]  task_work_run+0xc4/0x1e0
[  101.536652]  do_exit+0x554/0x1168
[  101.536667]  do_group_exit+0x60/0x108
[  101.536682]  __arm64_sys_exit_group+0x34/0x38
[  101.536698]  invoke_syscall+0x64/0x180
[  101.536714]  el0_svc_common.constprop.0+0x13c/0x170
[  101.536733]  do_el0_svc+0x48/0xe8
[  101.536748]  el0_svc+0x5c/0xe0
[  101.536759]  el0t_64_sync_handler+0xb8/0xc0
[  101.536771]  el0t_64_sync+0x18c/0x190
[  101.541928] ------------[ cut here ]------------
[  101.541934] kernel BUG at kernel/irq_work.c:235!
[  101.541944] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
[  101.541961] Modules linked in:
[  101.541978] CPU: 1 PID: 280 Comm: glmark2-es2-drm Tainted: G    B   W 
         5.19.0-rc3+ #400
[  101.541997] Hardware name: ARM LTD ARM Juno Development Platform/ARM 
Juno Development Platform, BIOS EDK II Sep  3 2019
[  101.542009] pstate: 40000005 (nZcv daif -PAN -UAO -TCO -DIT -SSBS 
BTYPE=--)
[  101.542027] pc : irq_work_run_list+0x80/0x88
[  101.542044] lr : irq_work_run+0x34/0x48
[  101.542060] sp : ffff80000da37eb0
[  101.542069] x29: ffff80000da37eb0 x28: ffff000006bb0000 x27: 
ffff000006bb0008
[  101.542107] x26: ffff80000da37f20 x25: ffff8000080304d8 x24: 
0000000000000001
[  101.542142] x23: ffff80000abcd008 x22: ffff80000da37ed0 x21: 
ffff80001c0de000
[  101.542177] x20: ffff80000abcd008 x19: ffff80000abdbad0 x18: 
0000000000000000
[  101.542212] x17: 616e202c30383220 x16: 3a646970202c3020 x15: 
ffff8000082df9d0
[  101.542246] x14: ffff800008dfada8 x13: 0000000000000003 x12: 
1fffe000018b2a06
[  101.542280] x11: ffff6000018b2a06 x10: dfff800000000000 x9 : 
ffff00000c595033
[  101.542315] x8 : ffff6000018b2a07 x7 : 0000000000000001 x6 : 
00000000000000fb
[  101.542349] x5 : ffff00000c595030 x4 : 0000000000000000 x3 : 
ffff00000c595030
[  101.542382] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 
ffff000026cb9ad0
[  101.542416] Call trace:
[  101.542424]  irq_work_run_list+0x80/0x88
[  101.542441]  ipi_handler+0x254/0x468
[  101.542460]  handle_percpu_devid_irq+0x11c/0x518
[  101.542480]  generic_handle_domain_irq+0x50/0x70
[  101.542501]  gic_handle_irq+0xd4/0x118
[  101.542516]  call_on_irq_stack+0x2c/0x58
[  101.542534]  do_interrupt_handler+0xc0/0xc8
[  101.542553]  el1_interrupt+0x40/0x68
[  101.542568]  el1h_64_irq_handler+0x18/0x28
[  101.542584]  el1h_64_irq+0x64/0x68
[  101.542599]  __asan_load8+0x30/0xd0
[  101.542617]  drm_sched_entity_fini+0x1e8/0x3b0
[  101.542634]  drm_sched_entity_destroy+0x2c/0x40
[  101.542651]  panfrost_job_close+0x44/0x1c0
[  101.542669]  panfrost_postclose+0x38/0x60
[  101.542685]  drm_file_free.part.0+0x33c/0x4b8
[  101.542704]  drm_close_helper.isra.0+0xc0/0xd8
[  101.542723]  drm_release+0xe4/0x1e0
[  101.542740]  __fput+0xf8/0x390
[  101.542756]  ____fput+0x18/0x28
[  101.542773]  task_work_run+0xc4/0x1e0
[  101.542788]  do_exit+0x554/0x1168
[  101.542806]  do_group_exit+0x60/0x108
[  101.542825]  __arm64_sys_exit_group+0x34/0x38
[  101.542845]  invoke_syscall+0x64/0x180
[  101.542865]  el0_svc_common.constprop.0+0x13c/0x170
[  101.542887]  do_el0_svc+0x48/0xe8
[  101.542906]  el0_svc+0x5c/0xe0
[  101.542921]  el0t_64_sync_handler+0xb8/0xc0
[  101.542938]  el0t_64_sync+0x18c/0x190
[  101.542960] Code: a94153f3 a8c27bfd d50323bf d65f03c0 (d4210000)
[  101.542979] ---[ end trace 0000000000000000 ]---
[  101.678650] Kernel panic - not syncing: Oops - BUG: Fatal exception 
in interrupt
[  102.046301] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000001da14c98
[  103.227334] SMP: stopping secondary CPUs
[  103.241055] Kernel Offset: disabled
[  103.254316] CPU features: 0x800,00184810,00001086
[  103.268904] Memory Limit: 800 MB
[  103.411625] ---[ end Kernel panic - not syncing: Oops - BUG: Fatal 
exception in interrupt ]---
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Robin Murphy]

On 2022-05-27 00:50, Dmitry Osipenko wrote:
> Hello,
> 
> This patchset introduces memory shrinker for the VirtIO-GPU DRM driver
> and adds memory purging and eviction support to VirtIO-GPU driver.
> 
> The new dma-buf locking convention is introduced here as well.
> 
> During OOM, the shrinker will release BOs that are marked as "not needed"
> by userspace using the new madvise IOCTL, it will also evict idling BOs
> to SWAP. The userspace in this case is the Mesa VirGL driver, it will mark
> the cached BOs as "not needed", allowing kernel driver to release memory
> of the cached shmem BOs on lowmem situations, preventing OOM kills.
> 
> The Panfrost driver is switched to use generic memory shrinker.

I think we still have some outstanding issues here - Alyssa reported 
some weirdness yesterday, so I just tried provoking a low-memory 
condition locally with this series applied and a few debug options 
enabled, and the results as below were... interesting.

Thanks,
Robin.

----->8-----
[   68.295951] ======================================================
[   68.295956] WARNING: possible circular locking dependency detected
[   68.295963] 5.19.0-rc3+ #400 Not tainted
[   68.295972] ------------------------------------------------------
[   68.295977] cc1/295 is trying to acquire lock:
[   68.295986] ffff000008d7f1a0 
(reservation_ww_class_mutex){+.+.}-{3:3}, at: drm_gem_shmem_free+0x7c/0x198
[   68.296036]
[   68.296036] but task is already holding lock:
[   68.296041] ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at: 
__alloc_pages_slowpath.constprop.0+0x4d8/0x1470
[   68.296080]
[   68.296080] which lock already depends on the new lock.
[   68.296080]
[   68.296085]
[   68.296085] the existing dependency chain (in reverse order) is:
[   68.296090]
[   68.296090] -> #1 (fs_reclaim){+.+.}-{0:0}:
[   68.296111]        fs_reclaim_acquire+0xb8/0x150
[   68.296130]        dma_resv_lockdep+0x298/0x3fc
[   68.296148]        do_one_initcall+0xe4/0x5f8
[   68.296163]        kernel_init_freeable+0x414/0x49c
[   68.296180]        kernel_init+0x2c/0x148
[   68.296195]        ret_from_fork+0x10/0x20
[   68.296207]
[   68.296207] -> #0 (reservation_ww_class_mutex){+.+.}-{3:3}:
[   68.296229]        __lock_acquire+0x1724/0x2398
[   68.296246]        lock_acquire+0x218/0x5b0
[   68.296260]        __ww_mutex_lock.constprop.0+0x158/0x2378
[   68.296277]        ww_mutex_lock+0x7c/0x4d8
[   68.296291]        drm_gem_shmem_free+0x7c/0x198
[   68.296304]        panfrost_gem_free_object+0x118/0x138
[   68.296318]        drm_gem_object_free+0x40/0x68
[   68.296334]        drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
[   68.296352]        drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
[   68.296368]        do_shrink_slab+0x220/0x808
[   68.296381]        shrink_slab+0x11c/0x408
[   68.296392]        shrink_node+0x6ac/0xb90
[   68.296403]        do_try_to_free_pages+0x1dc/0x8d0
[   68.296416]        try_to_free_pages+0x1ec/0x5b0
[   68.296429]        __alloc_pages_slowpath.constprop.0+0x528/0x1470
[   68.296444]        __alloc_pages+0x4e0/0x5b8
[   68.296455]        __folio_alloc+0x24/0x60
[   68.296467]        vma_alloc_folio+0xb8/0x2f8
[   68.296483]        alloc_zeroed_user_highpage_movable+0x58/0x68
[   68.296498]        __handle_mm_fault+0x918/0x12a8
[   68.296513]        handle_mm_fault+0x130/0x300
[   68.296527]        do_page_fault+0x1d0/0x568
[   68.296539]        do_translation_fault+0xa0/0xb8
[   68.296551]        do_mem_abort+0x68/0xf8
[   68.296562]        el0_da+0x74/0x100
[   68.296572]        el0t_64_sync_handler+0x68/0xc0
[   68.296585]        el0t_64_sync+0x18c/0x190
[   68.296596]
[   68.296596] other info that might help us debug this:
[   68.296596]
[   68.296601]  Possible unsafe locking scenario:
[   68.296601]
[   68.296604]        CPU0                    CPU1
[   68.296608]        ----                    ----
[   68.296612]   lock(fs_reclaim);
[   68.296622] 
lock(reservation_ww_class_mutex);
[   68.296633]                                lock(fs_reclaim);
[   68.296644]   lock(reservation_ww_class_mutex);
[   68.296654]
[   68.296654]  *** DEADLOCK ***
[   68.296654]
[   68.296658] 3 locks held by cc1/295:
[   68.296666]  #0: ffff00000616e898 (&mm->mmap_lock){++++}-{3:3}, at: 
do_page_fault+0x144/0x568
[   68.296702]  #1: ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at: 
__alloc_pages_slowpath.constprop.0+0x4d8/0x1470
[   68.296740]  #2: ffff80000c1215b0 (shrinker_rwsem){++++}-{3:3}, at: 
shrink_slab+0xc0/0x408
[   68.296774]
[   68.296774] stack backtrace:
[   68.296780] CPU: 2 PID: 295 Comm: cc1 Not tainted 5.19.0-rc3+ #400
[   68.296794] Hardware name: ARM LTD ARM Juno Development Platform/ARM 
Juno Development Platform, BIOS EDK II Sep  3 2019
[   68.296803] Call trace:
[   68.296808]  dump_backtrace+0x1e4/0x1f0
[   68.296821]  show_stack+0x20/0x70
[   68.296832]  dump_stack_lvl+0x8c/0xb8
[   68.296849]  dump_stack+0x1c/0x38
[   68.296864]  print_circular_bug.isra.0+0x284/0x378
[   68.296881]  check_noncircular+0x1d8/0x1f8
[   68.296896]  __lock_acquire+0x1724/0x2398
[   68.296911]  lock_acquire+0x218/0x5b0
[   68.296926]  __ww_mutex_lock.constprop.0+0x158/0x2378
[   68.296942]  ww_mutex_lock+0x7c/0x4d8
[   68.296956]  drm_gem_shmem_free+0x7c/0x198
[   68.296970]  panfrost_gem_free_object+0x118/0x138
[   68.296984]  drm_gem_object_free+0x40/0x68
[   68.296999]  drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
[   68.297017]  drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
[   68.297033]  do_shrink_slab+0x220/0x808
[   68.297045]  shrink_slab+0x11c/0x408
[   68.297056]  shrink_node+0x6ac/0xb90
[   68.297068]  do_try_to_free_pages+0x1dc/0x8d0
[   68.297081]  try_to_free_pages+0x1ec/0x5b0
[   68.297094]  __alloc_pages_slowpath.constprop.0+0x528/0x1470
[   68.297110]  __alloc_pages+0x4e0/0x5b8
[   68.297122]  __folio_alloc+0x24/0x60
[   68.297134]  vma_alloc_folio+0xb8/0x2f8
[   68.297148]  alloc_zeroed_user_highpage_movable+0x58/0x68
[   68.297163]  __handle_mm_fault+0x918/0x12a8
[   68.297178]  handle_mm_fault+0x130/0x300
[   68.297193]  do_page_fault+0x1d0/0x568
[   68.297205]  do_translation_fault+0xa0/0xb8
[   68.297218]  do_mem_abort+0x68/0xf8
[   68.297229]  el0_da+0x74/0x100
[   68.297239]  el0t_64_sync_handler+0x68/0xc0
[   68.297252]  el0t_64_sync+0x18c/0x190
[   68.471812] arm-scmi firmware:scmi: timed out in resp(caller: 
scmi_power_state_set+0x11c/0x190)
[   68.501947] arm-scmi firmware:scmi: Message for 119 type 0 is not 
expected!
[   68.939686] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000915e2d34
[   69.739386] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ac77ac55
[   70.415329] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ee980c7e
[   70.987166] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ffb7ff37
[   71.914939] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000000e92b26e
[   72.426987] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000c036a911
[   73.578683] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000001c6fc094
[   74.090555] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000075d00f9
[   74.922709] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000005add546
[   75.434401] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000000154189b
[   76.394300] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ac77ac55
[   76.906236] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ee980c7e
[   79.657234] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000f6d059fb
[   80.168831] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000061a0f6bf
[   80.808354] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000071ade02
[   81.319967] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000b0afea73
[   81.831574] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000d78f36c2
[   82.343160] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000000f689397
[   83.046689] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000412c2a2f
[   83.558352] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000020e551b3
[   84.261913] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000009437aace
[   84.773576] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000001c6fc094
[   85.317275] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000c036a911
[   85.829035] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000000e92b26e
[   86.660555] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000ac77ac55
[   87.172126] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000b940e406
[   87.875846] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000001c6fc094
[   88.387443] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000009437aace
[   89.059175] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000075dadb7f
[   89.570960] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000005add546
[   90.146687] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000cba2873c
[   90.662497] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000a4beb490
[   95.392748] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000005b5fc4ec
[   95.904179] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000a17436ee
[   96.416085] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000003888d2a7
[   96.927874] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000093e04a98
[   97.439742] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000c036a911
[   97.954109] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000084c51113
[   98.467374] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000664663ce
[   98.975192] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=0000000060f2d45c
[   99.487231] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000b29288f8
[   99.998833] panfrost 2d000000.gpu: gpu sched timeout, js=0, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000002f07ab24
[  100.510744] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000008c15c751
[  100.511411] 
==================================================================
[  100.511419] BUG: KASAN: use-after-free in irq_work_single+0xa4/0x110
[  100.511445] Write of size 4 at addr ffff0000107f5830 by task 
glmark2-es2-drm/280
[  100.511458]
[  100.511464] CPU: 1 PID: 280 Comm: glmark2-es2-drm Not tainted 
5.19.0-rc3+ #400
[  100.511479] Hardware name: ARM LTD ARM Juno Development Platform/ARM 
Juno Development Platform, BIOS EDK II Sep  3 2019
[  100.511489] Call trace:
[  100.511494]  dump_backtrace+0x1e4/0x1f0
[  100.511512]  show_stack+0x20/0x70
[  100.511523]  dump_stack_lvl+0x8c/0xb8
[  100.511543]  print_report+0x16c/0x668
[  100.511559]  kasan_report+0x80/0x208
[  100.511574]  kasan_check_range+0x100/0x1b8
[  100.511590]  __kasan_check_write+0x34/0x60
[  100.511607]  irq_work_single+0xa4/0x110
[  100.511619]  irq_work_run_list+0x6c/0x88
[  100.511632]  irq_work_run+0x28/0x48
[  100.511644]  ipi_handler+0x254/0x468
[  100.511664]  handle_percpu_devid_irq+0x11c/0x518
[  100.511681]  generic_handle_domain_irq+0x50/0x70
[  100.511699]  gic_handle_irq+0xd4/0x118
[  100.511711]  call_on_irq_stack+0x2c/0x58
[  100.511725]  do_interrupt_handler+0xc0/0xc8
[  100.511741]  el1_interrupt+0x40/0x68
[  100.511754]  el1h_64_irq_handler+0x18/0x28
[  100.511767]  el1h_64_irq+0x64/0x68
[  100.511778]  irq_work_queue+0xc0/0xd8
[  100.511790]  drm_sched_entity_fini+0x2c4/0x3b0
[  100.511805]  drm_sched_entity_destroy+0x2c/0x40
[  100.511818]  panfrost_job_close+0x44/0x1c0
[  100.511833]  panfrost_postclose+0x38/0x60
[  100.511845]  drm_file_free.part.0+0x33c/0x4b8
[  100.511862]  drm_close_helper.isra.0+0xc0/0xd8
[  100.511877]  drm_release+0xe4/0x1e0
[  100.511891]  __fput+0xf8/0x390
[  100.511904]  ____fput+0x18/0x28
[  100.511917]  task_work_run+0xc4/0x1e0
[  100.511929]  do_exit+0x554/0x1168
[  100.511945]  do_group_exit+0x60/0x108
[  100.511960]  __arm64_sys_exit_group+0x34/0x38
[  100.511977]  invoke_syscall+0x64/0x180
[  100.511993]  el0_svc_common.constprop.0+0x13c/0x170
[  100.512012]  do_el0_svc+0x48/0xe8
[  100.512028]  el0_svc+0x5c/0xe0
[  100.512038]  el0t_64_sync_handler+0xb8/0xc0
[  100.512051]  el0t_64_sync+0x18c/0x190
[  100.512064]
[  100.512068] Allocated by task 280:
[  100.512075]  kasan_save_stack+0x2c/0x58
[  100.512091]  __kasan_kmalloc+0x90/0xb8
[  100.512105]  kmem_cache_alloc_trace+0x1d4/0x330
[  100.512118]  panfrost_ioctl_submit+0x100/0x630
[  100.512131]  drm_ioctl_kernel+0x160/0x250
[  100.512147]  drm_ioctl+0x36c/0x628
[  100.512161]  __arm64_sys_ioctl+0xd8/0x120
[  100.512178]  invoke_syscall+0x64/0x180
[  100.512194]  el0_svc_common.constprop.0+0x13c/0x170
[  100.512211]  do_el0_svc+0x48/0xe8
[  100.512226]  el0_svc+0x5c/0xe0
[  100.512236]  el0t_64_sync_handler+0xb8/0xc0
[  100.512248]  el0t_64_sync+0x18c/0x190
[  100.512259]
[  100.512262] Freed by task 280:
[  100.512268]  kasan_save_stack+0x2c/0x58
[  100.512283]  kasan_set_track+0x2c/0x40
[  100.512296]  kasan_set_free_info+0x28/0x50
[  100.512312]  __kasan_slab_free+0xf0/0x170
[  100.512326]  kfree+0x124/0x418
[  100.512337]  panfrost_job_cleanup+0x1f0/0x298
[  100.512350]  panfrost_job_free+0x80/0xb0
[  100.512363]  drm_sched_entity_kill_jobs_irq_work+0x80/0xa0
[  100.512377]  irq_work_single+0x88/0x110
[  100.512389]  irq_work_run_list+0x6c/0x88
[  100.512401]  irq_work_run+0x28/0x48
[  100.512413]  ipi_handler+0x254/0x468
[  100.512427]  handle_percpu_devid_irq+0x11c/0x518
[  100.512443]  generic_handle_domain_irq+0x50/0x70
[  100.512460]  gic_handle_irq+0xd4/0x118
[  100.512471]
[  100.512474] The buggy address belongs to the object at ffff0000107f5800
[  100.512474]  which belongs to the cache kmalloc-512 of size 512
[  100.512484] The buggy address is located 48 bytes inside of
[  100.512484]  512-byte region [ffff0000107f5800, ffff0000107f5a00)
[  100.512497]
[  100.512500] The buggy address belongs to the physical page:
[  100.512506] page:000000000a626feb refcount:1 mapcount:0 
mapping:0000000000000000 index:0x0 pfn:0x907f4
[  100.512520] head:000000000a626feb order:2 compound_mapcount:0 
compound_pincount:0
[  100.512530] flags: 
0xffff00000010200(slab|head|node=0|zone=0|lastcpupid=0xffff)
[  100.512556] raw: 0ffff00000010200 fffffc0000076400 dead000000000002 
ffff000000002600
[  100.512569] raw: 0000000000000000 0000000080100010 00000001ffffffff 
0000000000000000
[  100.512577] page dumped because: kasan: bad access detected
[  100.512582]
[  100.512585] Memory state around the buggy address:
[  100.512592]  ffff0000107f5700: fc fc fc fc fc fc fc fc fc fc fc fc fc 
fc fc fc
[  100.512602]  ffff0000107f5780: fc fc fc fc fc fc fc fc fc fc fc fc fc 
fc fc fc
[  100.512612] >ffff0000107f5800: fa fb fb fb fb fb fb fb fb fb fb fb fb 
fb fb fb
[  100.512619]                                      ^
[  100.512627]  ffff0000107f5880: fb fb fb fb fb fb fb fb fb fb fb fb fb 
fb fb fb
[  100.512636]  ffff0000107f5900: fb fb fb fb fb fb fb fb fb fb fb fb fb 
fb fb fb
[  100.512643] 
==================================================================
[  101.022573] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000be4b1b31
[  101.534469] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=00000000a8ff2c8a
[  101.535981] BUG: sleeping function called from invalid context at 
kernel/locking/mutex.c:870
[  101.535994] in_atomic(): 1, irqs_disabled(): 128, non_block: 0, pid: 
280, name: glmark2-es2-drm
[  101.536006] preempt_count: 10000, expected: 0
[  101.536012] RCU nest depth: 0, expected: 0
[  101.536019] INFO: lockdep is turned off.
[  101.536023] irq event stamp: 1666508
[  101.536029] hardirqs last  enabled at (1666507): [<ffff80000997ed70>] 
exit_to_kernel_mode.isra.0+0x40/0x140
[  101.536056] hardirqs last disabled at (1666508): [<ffff800009985030>] 
__schedule+0xb38/0xea8
[  101.536076] softirqs last  enabled at (1664950): [<ffff800008010ac8>] 
__do_softirq+0x6b8/0x89c
[  101.536092] softirqs last disabled at (1664941): [<ffff8000080e4fdc>] 
irq_exit_rcu+0x27c/0x2b0
[  101.536118] CPU: 1 PID: 280 Comm: glmark2-es2-drm Tainted: G    B 
          5.19.0-rc3+ #400
[  101.536134] Hardware name: ARM LTD ARM Juno Development Platform/ARM 
Juno Development Platform, BIOS EDK II Sep  3 2019
[  101.536143] Call trace:
[  101.536147]  dump_backtrace+0x1e4/0x1f0
[  101.536161]  show_stack+0x20/0x70
[  101.536171]  dump_stack_lvl+0x8c/0xb8
[  101.536189]  dump_stack+0x1c/0x38
[  101.536204]  __might_resched+0x1f0/0x2b0
[  101.536220]  __might_sleep+0x74/0xd0
[  101.536234]  ww_mutex_lock+0x40/0x4d8
[  101.536249]  drm_gem_shmem_free+0x7c/0x198
[  101.536264]  panfrost_gem_free_object+0x118/0x138
[  101.536278]  drm_gem_object_free+0x40/0x68
[  101.536295]  panfrost_job_cleanup+0x1bc/0x298
[  101.536309]  panfrost_job_free+0x80/0xb0
[  101.536322]  drm_sched_entity_kill_jobs_irq_work+0x80/0xa0
[  101.536337]  irq_work_single+0x88/0x110
[  101.536351]  irq_work_run_list+0x6c/0x88
[  101.536364]  irq_work_run+0x28/0x48
[  101.536375]  ipi_handler+0x254/0x468
[  101.536392]  handle_percpu_devid_irq+0x11c/0x518
[  101.536409]  generic_handle_domain_irq+0x50/0x70
[  101.536428]  gic_handle_irq+0xd4/0x118
[  101.536439]  call_on_irq_stack+0x2c/0x58
[  101.536453]  do_interrupt_handler+0xc0/0xc8
[  101.536468]  el1_interrupt+0x40/0x68
[  101.536479]  el1h_64_irq_handler+0x18/0x28
[  101.536492]  el1h_64_irq+0x64/0x68
[  101.536503]  __asan_load8+0x30/0xd0
[  101.536519]  drm_sched_entity_fini+0x1e8/0x3b0
[  101.536532]  drm_sched_entity_destroy+0x2c/0x40
[  101.536545]  panfrost_job_close+0x44/0x1c0
[  101.536559]  panfrost_postclose+0x38/0x60
[  101.536571]  drm_file_free.part.0+0x33c/0x4b8
[  101.536586]  drm_close_helper.isra.0+0xc0/0xd8
[  101.536601]  drm_release+0xe4/0x1e0
[  101.536615]  __fput+0xf8/0x390
[  101.536628]  ____fput+0x18/0x28
[  101.536640]  task_work_run+0xc4/0x1e0
[  101.536652]  do_exit+0x554/0x1168
[  101.536667]  do_group_exit+0x60/0x108
[  101.536682]  __arm64_sys_exit_group+0x34/0x38
[  101.536698]  invoke_syscall+0x64/0x180
[  101.536714]  el0_svc_common.constprop.0+0x13c/0x170
[  101.536733]  do_el0_svc+0x48/0xe8
[  101.536748]  el0_svc+0x5c/0xe0
[  101.536759]  el0t_64_sync_handler+0xb8/0xc0
[  101.536771]  el0t_64_sync+0x18c/0x190
[  101.541928] ------------[ cut here ]------------
[  101.541934] kernel BUG at kernel/irq_work.c:235!
[  101.541944] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
[  101.541961] Modules linked in:
[  101.541978] CPU: 1 PID: 280 Comm: glmark2-es2-drm Tainted: G    B   W 
         5.19.0-rc3+ #400
[  101.541997] Hardware name: ARM LTD ARM Juno Development Platform/ARM 
Juno Development Platform, BIOS EDK II Sep  3 2019
[  101.542009] pstate: 40000005 (nZcv daif -PAN -UAO -TCO -DIT -SSBS 
BTYPE=--)
[  101.542027] pc : irq_work_run_list+0x80/0x88
[  101.542044] lr : irq_work_run+0x34/0x48
[  101.542060] sp : ffff80000da37eb0
[  101.542069] x29: ffff80000da37eb0 x28: ffff000006bb0000 x27: 
ffff000006bb0008
[  101.542107] x26: ffff80000da37f20 x25: ffff8000080304d8 x24: 
0000000000000001
[  101.542142] x23: ffff80000abcd008 x22: ffff80000da37ed0 x21: 
ffff80001c0de000
[  101.542177] x20: ffff80000abcd008 x19: ffff80000abdbad0 x18: 
0000000000000000
[  101.542212] x17: 616e202c30383220 x16: 3a646970202c3020 x15: 
ffff8000082df9d0
[  101.542246] x14: ffff800008dfada8 x13: 0000000000000003 x12: 
1fffe000018b2a06
[  101.542280] x11: ffff6000018b2a06 x10: dfff800000000000 x9 : 
ffff00000c595033
[  101.542315] x8 : ffff6000018b2a07 x7 : 0000000000000001 x6 : 
00000000000000fb
[  101.542349] x5 : ffff00000c595030 x4 : 0000000000000000 x3 : 
ffff00000c595030
[  101.542382] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 
ffff000026cb9ad0
[  101.542416] Call trace:
[  101.542424]  irq_work_run_list+0x80/0x88
[  101.542441]  ipi_handler+0x254/0x468
[  101.542460]  handle_percpu_devid_irq+0x11c/0x518
[  101.542480]  generic_handle_domain_irq+0x50/0x70
[  101.542501]  gic_handle_irq+0xd4/0x118
[  101.542516]  call_on_irq_stack+0x2c/0x58
[  101.542534]  do_interrupt_handler+0xc0/0xc8
[  101.542553]  el1_interrupt+0x40/0x68
[  101.542568]  el1h_64_irq_handler+0x18/0x28
[  101.542584]  el1h_64_irq+0x64/0x68
[  101.542599]  __asan_load8+0x30/0xd0
[  101.542617]  drm_sched_entity_fini+0x1e8/0x3b0
[  101.542634]  drm_sched_entity_destroy+0x2c/0x40
[  101.542651]  panfrost_job_close+0x44/0x1c0
[  101.542669]  panfrost_postclose+0x38/0x60
[  101.542685]  drm_file_free.part.0+0x33c/0x4b8
[  101.542704]  drm_close_helper.isra.0+0xc0/0xd8
[  101.542723]  drm_release+0xe4/0x1e0
[  101.542740]  __fput+0xf8/0x390
[  101.542756]  ____fput+0x18/0x28
[  101.542773]  task_work_run+0xc4/0x1e0
[  101.542788]  do_exit+0x554/0x1168
[  101.542806]  do_group_exit+0x60/0x108
[  101.542825]  __arm64_sys_exit_group+0x34/0x38
[  101.542845]  invoke_syscall+0x64/0x180
[  101.542865]  el0_svc_common.constprop.0+0x13c/0x170
[  101.542887]  do_el0_svc+0x48/0xe8
[  101.542906]  el0_svc+0x5c/0xe0
[  101.542921]  el0t_64_sync_handler+0xb8/0xc0
[  101.542938]  el0t_64_sync+0x18c/0x190
[  101.542960] Code: a94153f3 a8c27bfd d50323bf d65f03c0 (d4210000)
[  101.542979] ---[ end trace 0000000000000000 ]---
[  101.678650] Kernel panic - not syncing: Oops - BUG: Fatal exception 
in interrupt
[  102.046301] panfrost 2d000000.gpu: gpu sched timeout, js=1, 
config=0x0, status=0x0, head=0x0, tail=0x0, sched_job=000000001da14c98
[  103.227334] SMP: stopping secondary CPUs
[  103.241055] Kernel Offset: disabled
[  103.254316] CPU features: 0x800,00184810,00001086
[  103.268904] Memory Limit: 800 MB
[  103.411625] ---[ end Kernel panic - not syncing: Oops - BUG: Fatal 
exception in interrupt ]---

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Dmitry Osipenko]

On 6/28/22 15:31, Robin Murphy wrote:
> ----->8-----
> [   68.295951] ======================================================
> [   68.295956] WARNING: possible circular locking dependency detected
> [   68.295963] 5.19.0-rc3+ #400 Not tainted
> [   68.295972] ------------------------------------------------------
> [   68.295977] cc1/295 is trying to acquire lock:
> [   68.295986] ffff000008d7f1a0
> (reservation_ww_class_mutex){+.+.}-{3:3}, at: drm_gem_shmem_free+0x7c/0x198
> [   68.296036]
> [   68.296036] but task is already holding lock:
> [   68.296041] ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at:
> __alloc_pages_slowpath.constprop.0+0x4d8/0x1470
> [   68.296080]
> [   68.296080] which lock already depends on the new lock.
> [   68.296080]
> [   68.296085]
> [   68.296085] the existing dependency chain (in reverse order) is:
> [   68.296090]
> [   68.296090] -> #1 (fs_reclaim){+.+.}-{0:0}:
> [   68.296111]        fs_reclaim_acquire+0xb8/0x150
> [   68.296130]        dma_resv_lockdep+0x298/0x3fc
> [   68.296148]        do_one_initcall+0xe4/0x5f8
> [   68.296163]        kernel_init_freeable+0x414/0x49c
> [   68.296180]        kernel_init+0x2c/0x148
> [   68.296195]        ret_from_fork+0x10/0x20
> [   68.296207]
> [   68.296207] -> #0 (reservation_ww_class_mutex){+.+.}-{3:3}:
> [   68.296229]        __lock_acquire+0x1724/0x2398
> [   68.296246]        lock_acquire+0x218/0x5b0
> [   68.296260]        __ww_mutex_lock.constprop.0+0x158/0x2378
> [   68.296277]        ww_mutex_lock+0x7c/0x4d8
> [   68.296291]        drm_gem_shmem_free+0x7c/0x198
> [   68.296304]        panfrost_gem_free_object+0x118/0x138
> [   68.296318]        drm_gem_object_free+0x40/0x68
> [   68.296334]        drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
> [   68.296352]        drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
> [   68.296368]        do_shrink_slab+0x220/0x808
> [   68.296381]        shrink_slab+0x11c/0x408
> [   68.296392]        shrink_node+0x6ac/0xb90
> [   68.296403]        do_try_to_free_pages+0x1dc/0x8d0
> [   68.296416]        try_to_free_pages+0x1ec/0x5b0
> [   68.296429]        __alloc_pages_slowpath.constprop.0+0x528/0x1470
> [   68.296444]        __alloc_pages+0x4e0/0x5b8
> [   68.296455]        __folio_alloc+0x24/0x60
> [   68.296467]        vma_alloc_folio+0xb8/0x2f8
> [   68.296483]        alloc_zeroed_user_highpage_movable+0x58/0x68
> [   68.296498]        __handle_mm_fault+0x918/0x12a8
> [   68.296513]        handle_mm_fault+0x130/0x300
> [   68.296527]        do_page_fault+0x1d0/0x568
> [   68.296539]        do_translation_fault+0xa0/0xb8
> [   68.296551]        do_mem_abort+0x68/0xf8
> [   68.296562]        el0_da+0x74/0x100
> [   68.296572]        el0t_64_sync_handler+0x68/0xc0
> [   68.296585]        el0t_64_sync+0x18c/0x190
> [   68.296596]
> [   68.296596] other info that might help us debug this:
> [   68.296596]
> [   68.296601]  Possible unsafe locking scenario:
> [   68.296601]
> [   68.296604]        CPU0                    CPU1
> [   68.296608]        ----                    ----
> [   68.296612]   lock(fs_reclaim);
> [   68.296622] lock(reservation_ww_class_mutex);
> [   68.296633]                                lock(fs_reclaim);
> [   68.296644]   lock(reservation_ww_class_mutex);
> [   68.296654]
> [   68.296654]  *** DEADLOCK ***

This splat could be ignored for now. I'm aware about it, although
haven't looked closely at how to fix it since it's a kind of a lockdep
misreporting.

-- 
Best regards,
Dmitry

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Dmitry Osipenko]

On 6/28/22 15:31, Robin Murphy wrote:
> ----->8-----
> [   68.295951] ======================================================
> [   68.295956] WARNING: possible circular locking dependency detected
> [   68.295963] 5.19.0-rc3+ #400 Not tainted
> [   68.295972] ------------------------------------------------------
> [   68.295977] cc1/295 is trying to acquire lock:
> [   68.295986] ffff000008d7f1a0
> (reservation_ww_class_mutex){+.+.}-{3:3}, at: drm_gem_shmem_free+0x7c/0x198
> [   68.296036]
> [   68.296036] but task is already holding lock:
> [   68.296041] ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at:
> __alloc_pages_slowpath.constprop.0+0x4d8/0x1470
> [   68.296080]
> [   68.296080] which lock already depends on the new lock.
> [   68.296080]
> [   68.296085]
> [   68.296085] the existing dependency chain (in reverse order) is:
> [   68.296090]
> [   68.296090] -> #1 (fs_reclaim){+.+.}-{0:0}:
> [   68.296111]        fs_reclaim_acquire+0xb8/0x150
> [   68.296130]        dma_resv_lockdep+0x298/0x3fc
> [   68.296148]        do_one_initcall+0xe4/0x5f8
> [   68.296163]        kernel_init_freeable+0x414/0x49c
> [   68.296180]        kernel_init+0x2c/0x148
> [   68.296195]        ret_from_fork+0x10/0x20
> [   68.296207]
> [   68.296207] -> #0 (reservation_ww_class_mutex){+.+.}-{3:3}:
> [   68.296229]        __lock_acquire+0x1724/0x2398
> [   68.296246]        lock_acquire+0x218/0x5b0
> [   68.296260]        __ww_mutex_lock.constprop.0+0x158/0x2378
> [   68.296277]        ww_mutex_lock+0x7c/0x4d8
> [   68.296291]        drm_gem_shmem_free+0x7c/0x198
> [   68.296304]        panfrost_gem_free_object+0x118/0x138
> [   68.296318]        drm_gem_object_free+0x40/0x68
> [   68.296334]        drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
> [   68.296352]        drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
> [   68.296368]        do_shrink_slab+0x220/0x808
> [   68.296381]        shrink_slab+0x11c/0x408
> [   68.296392]        shrink_node+0x6ac/0xb90
> [   68.296403]        do_try_to_free_pages+0x1dc/0x8d0
> [   68.296416]        try_to_free_pages+0x1ec/0x5b0
> [   68.296429]        __alloc_pages_slowpath.constprop.0+0x528/0x1470
> [   68.296444]        __alloc_pages+0x4e0/0x5b8
> [   68.296455]        __folio_alloc+0x24/0x60
> [   68.296467]        vma_alloc_folio+0xb8/0x2f8
> [   68.296483]        alloc_zeroed_user_highpage_movable+0x58/0x68
> [   68.296498]        __handle_mm_fault+0x918/0x12a8
> [   68.296513]        handle_mm_fault+0x130/0x300
> [   68.296527]        do_page_fault+0x1d0/0x568
> [   68.296539]        do_translation_fault+0xa0/0xb8
> [   68.296551]        do_mem_abort+0x68/0xf8
> [   68.296562]        el0_da+0x74/0x100
> [   68.296572]        el0t_64_sync_handler+0x68/0xc0
> [   68.296585]        el0t_64_sync+0x18c/0x190
> [   68.296596]
> [   68.296596] other info that might help us debug this:
> [   68.296596]
> [   68.296601]  Possible unsafe locking scenario:
> [   68.296601]
> [   68.296604]        CPU0                    CPU1
> [   68.296608]        ----                    ----
> [   68.296612]   lock(fs_reclaim);
> [   68.296622] lock(reservation_ww_class_mutex);
> [   68.296633]                                lock(fs_reclaim);
> [   68.296644]   lock(reservation_ww_class_mutex);
> [   68.296654]
> [   68.296654]  *** DEADLOCK ***

This splat could be ignored for now. I'm aware about it, although
haven't looked closely at how to fix it since it's a kind of a lockdep
misreporting.

-- 
Best regards,
Dmitry

Re: [Intel-gfx] [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Dmitry Osipenko]

On 6/28/22 15:31, Robin Murphy wrote:
> ----->8-----
> [   68.295951] ======================================================
> [   68.295956] WARNING: possible circular locking dependency detected
> [   68.295963] 5.19.0-rc3+ #400 Not tainted
> [   68.295972] ------------------------------------------------------
> [   68.295977] cc1/295 is trying to acquire lock:
> [   68.295986] ffff000008d7f1a0
> (reservation_ww_class_mutex){+.+.}-{3:3}, at: drm_gem_shmem_free+0x7c/0x198
> [   68.296036]
> [   68.296036] but task is already holding lock:
> [   68.296041] ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at:
> __alloc_pages_slowpath.constprop.0+0x4d8/0x1470
> [   68.296080]
> [   68.296080] which lock already depends on the new lock.
> [   68.296080]
> [   68.296085]
> [   68.296085] the existing dependency chain (in reverse order) is:
> [   68.296090]
> [   68.296090] -> #1 (fs_reclaim){+.+.}-{0:0}:
> [   68.296111]        fs_reclaim_acquire+0xb8/0x150
> [   68.296130]        dma_resv_lockdep+0x298/0x3fc
> [   68.296148]        do_one_initcall+0xe4/0x5f8
> [   68.296163]        kernel_init_freeable+0x414/0x49c
> [   68.296180]        kernel_init+0x2c/0x148
> [   68.296195]        ret_from_fork+0x10/0x20
> [   68.296207]
> [   68.296207] -> #0 (reservation_ww_class_mutex){+.+.}-{3:3}:
> [   68.296229]        __lock_acquire+0x1724/0x2398
> [   68.296246]        lock_acquire+0x218/0x5b0
> [   68.296260]        __ww_mutex_lock.constprop.0+0x158/0x2378
> [   68.296277]        ww_mutex_lock+0x7c/0x4d8
> [   68.296291]        drm_gem_shmem_free+0x7c/0x198
> [   68.296304]        panfrost_gem_free_object+0x118/0x138
> [   68.296318]        drm_gem_object_free+0x40/0x68
> [   68.296334]        drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
> [   68.296352]        drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
> [   68.296368]        do_shrink_slab+0x220/0x808
> [   68.296381]        shrink_slab+0x11c/0x408
> [   68.296392]        shrink_node+0x6ac/0xb90
> [   68.296403]        do_try_to_free_pages+0x1dc/0x8d0
> [   68.296416]        try_to_free_pages+0x1ec/0x5b0
> [   68.296429]        __alloc_pages_slowpath.constprop.0+0x528/0x1470
> [   68.296444]        __alloc_pages+0x4e0/0x5b8
> [   68.296455]        __folio_alloc+0x24/0x60
> [   68.296467]        vma_alloc_folio+0xb8/0x2f8
> [   68.296483]        alloc_zeroed_user_highpage_movable+0x58/0x68
> [   68.296498]        __handle_mm_fault+0x918/0x12a8
> [   68.296513]        handle_mm_fault+0x130/0x300
> [   68.296527]        do_page_fault+0x1d0/0x568
> [   68.296539]        do_translation_fault+0xa0/0xb8
> [   68.296551]        do_mem_abort+0x68/0xf8
> [   68.296562]        el0_da+0x74/0x100
> [   68.296572]        el0t_64_sync_handler+0x68/0xc0
> [   68.296585]        el0t_64_sync+0x18c/0x190
> [   68.296596]
> [   68.296596] other info that might help us debug this:
> [   68.296596]
> [   68.296601]  Possible unsafe locking scenario:
> [   68.296601]
> [   68.296604]        CPU0                    CPU1
> [   68.296608]        ----                    ----
> [   68.296612]   lock(fs_reclaim);
> [   68.296622] lock(reservation_ww_class_mutex);
> [   68.296633]                                lock(fs_reclaim);
> [   68.296644]   lock(reservation_ww_class_mutex);
> [   68.296654]
> [   68.296654]  *** DEADLOCK ***

This splat could be ignored for now. I'm aware about it, although
haven't looked closely at how to fix it since it's a kind of a lockdep
misreporting.

-- 
Best regards,
Dmitry

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Dmitry Osipenko]

On 6/28/22 15:31, Robin Murphy wrote:
> [  100.511411]
> ==================================================================
> [  100.511419] BUG: KASAN: use-after-free in irq_work_single+0xa4/0x110
> [  100.511445] Write of size 4 at addr ffff0000107f5830 by task
> glmark2-es2-drm/280
> [  100.511458]
> [  100.511464] CPU: 1 PID: 280 Comm: glmark2-es2-drm Not tainted
> 5.19.0-rc3+ #400
> [  100.511479] Hardware name: ARM LTD ARM Juno Development Platform/ARM
> Juno Development Platform, BIOS EDK II Sep  3 2019
> [  100.511489] Call trace:
> [  100.511494]  dump_backtrace+0x1e4/0x1f0
> [  100.511512]  show_stack+0x20/0x70
> [  100.511523]  dump_stack_lvl+0x8c/0xb8
> [  100.511543]  print_report+0x16c/0x668
> [  100.511559]  kasan_report+0x80/0x208
> [  100.511574]  kasan_check_range+0x100/0x1b8
> [  100.511590]  __kasan_check_write+0x34/0x60
> [  100.511607]  irq_work_single+0xa4/0x110
> [  100.511619]  irq_work_run_list+0x6c/0x88
> [  100.511632]  irq_work_run+0x28/0x48
> [  100.511644]  ipi_handler+0x254/0x468
> [  100.511664]  handle_percpu_devid_irq+0x11c/0x518
> [  100.511681]  generic_handle_domain_irq+0x50/0x70
> [  100.511699]  gic_handle_irq+0xd4/0x118
> [  100.511711]  call_on_irq_stack+0x2c/0x58
> [  100.511725]  do_interrupt_handler+0xc0/0xc8
> [  100.511741]  el1_interrupt+0x40/0x68
> [  100.511754]  el1h_64_irq_handler+0x18/0x28
> [  100.511767]  el1h_64_irq+0x64/0x68
> [  100.511778]  irq_work_queue+0xc0/0xd8
> [  100.511790]  drm_sched_entity_fini+0x2c4/0x3b0
> [  100.511805]  drm_sched_entity_destroy+0x2c/0x40
> [  100.511818]  panfrost_job_close+0x44/0x1c0
> [  100.511833]  panfrost_postclose+0x38/0x60
> [  100.511845]  drm_file_free.part.0+0x33c/0x4b8
> [  100.511862]  drm_close_helper.isra.0+0xc0/0xd8
> [  100.511877]  drm_release+0xe4/0x1e0
> [  100.511891]  __fput+0xf8/0x390
> [  100.511904]  ____fput+0x18/0x28
> [  100.511917]  task_work_run+0xc4/0x1e0
> [  100.511929]  do_exit+0x554/0x1168
> [  100.511945]  do_group_exit+0x60/0x108
> [  100.511960]  __arm64_sys_exit_group+0x34/0x38
> [  100.511977]  invoke_syscall+0x64/0x180
> [  100.511993]  el0_svc_common.constprop.0+0x13c/0x170
> [  100.512012]  do_el0_svc+0x48/0xe8
> [  100.512028]  el0_svc+0x5c/0xe0
> [  100.512038]  el0t_64_sync_handler+0xb8/0xc0
> [  100.512051]  el0t_64_sync+0x18c/0x190
> [  100.512064]

This one shall be fixed by [1] that is not in the RC kernel yet, please
use linux-next.

[1]
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20220628&id=7d64c40a7d96190d9d06e240305389e025295916

-- 
Best regards,
Dmitry

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Dmitry Osipenko]

On 6/28/22 15:31, Robin Murphy wrote:
> [  100.511411]
> ==================================================================
> [  100.511419] BUG: KASAN: use-after-free in irq_work_single+0xa4/0x110
> [  100.511445] Write of size 4 at addr ffff0000107f5830 by task
> glmark2-es2-drm/280
> [  100.511458]
> [  100.511464] CPU: 1 PID: 280 Comm: glmark2-es2-drm Not tainted
> 5.19.0-rc3+ #400
> [  100.511479] Hardware name: ARM LTD ARM Juno Development Platform/ARM
> Juno Development Platform, BIOS EDK II Sep  3 2019
> [  100.511489] Call trace:
> [  100.511494]  dump_backtrace+0x1e4/0x1f0
> [  100.511512]  show_stack+0x20/0x70
> [  100.511523]  dump_stack_lvl+0x8c/0xb8
> [  100.511543]  print_report+0x16c/0x668
> [  100.511559]  kasan_report+0x80/0x208
> [  100.511574]  kasan_check_range+0x100/0x1b8
> [  100.511590]  __kasan_check_write+0x34/0x60
> [  100.511607]  irq_work_single+0xa4/0x110
> [  100.511619]  irq_work_run_list+0x6c/0x88
> [  100.511632]  irq_work_run+0x28/0x48
> [  100.511644]  ipi_handler+0x254/0x468
> [  100.511664]  handle_percpu_devid_irq+0x11c/0x518
> [  100.511681]  generic_handle_domain_irq+0x50/0x70
> [  100.511699]  gic_handle_irq+0xd4/0x118
> [  100.511711]  call_on_irq_stack+0x2c/0x58
> [  100.511725]  do_interrupt_handler+0xc0/0xc8
> [  100.511741]  el1_interrupt+0x40/0x68
> [  100.511754]  el1h_64_irq_handler+0x18/0x28
> [  100.511767]  el1h_64_irq+0x64/0x68
> [  100.511778]  irq_work_queue+0xc0/0xd8
> [  100.511790]  drm_sched_entity_fini+0x2c4/0x3b0
> [  100.511805]  drm_sched_entity_destroy+0x2c/0x40
> [  100.511818]  panfrost_job_close+0x44/0x1c0
> [  100.511833]  panfrost_postclose+0x38/0x60
> [  100.511845]  drm_file_free.part.0+0x33c/0x4b8
> [  100.511862]  drm_close_helper.isra.0+0xc0/0xd8
> [  100.511877]  drm_release+0xe4/0x1e0
> [  100.511891]  __fput+0xf8/0x390
> [  100.511904]  ____fput+0x18/0x28
> [  100.511917]  task_work_run+0xc4/0x1e0
> [  100.511929]  do_exit+0x554/0x1168
> [  100.511945]  do_group_exit+0x60/0x108
> [  100.511960]  __arm64_sys_exit_group+0x34/0x38
> [  100.511977]  invoke_syscall+0x64/0x180
> [  100.511993]  el0_svc_common.constprop.0+0x13c/0x170
> [  100.512012]  do_el0_svc+0x48/0xe8
> [  100.512028]  el0_svc+0x5c/0xe0
> [  100.512038]  el0t_64_sync_handler+0xb8/0xc0
> [  100.512051]  el0t_64_sync+0x18c/0x190
> [  100.512064]

This one shall be fixed by [1] that is not in the RC kernel yet, please
use linux-next.

[1]
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20220628&id=7d64c40a7d96190d9d06e240305389e025295916

-- 
Best regards,
Dmitry

Re: [Intel-gfx] [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Dmitry Osipenko]

On 6/28/22 15:31, Robin Murphy wrote:
> [  100.511411]
> ==================================================================
> [  100.511419] BUG: KASAN: use-after-free in irq_work_single+0xa4/0x110
> [  100.511445] Write of size 4 at addr ffff0000107f5830 by task
> glmark2-es2-drm/280
> [  100.511458]
> [  100.511464] CPU: 1 PID: 280 Comm: glmark2-es2-drm Not tainted
> 5.19.0-rc3+ #400
> [  100.511479] Hardware name: ARM LTD ARM Juno Development Platform/ARM
> Juno Development Platform, BIOS EDK II Sep  3 2019
> [  100.511489] Call trace:
> [  100.511494]  dump_backtrace+0x1e4/0x1f0
> [  100.511512]  show_stack+0x20/0x70
> [  100.511523]  dump_stack_lvl+0x8c/0xb8
> [  100.511543]  print_report+0x16c/0x668
> [  100.511559]  kasan_report+0x80/0x208
> [  100.511574]  kasan_check_range+0x100/0x1b8
> [  100.511590]  __kasan_check_write+0x34/0x60
> [  100.511607]  irq_work_single+0xa4/0x110
> [  100.511619]  irq_work_run_list+0x6c/0x88
> [  100.511632]  irq_work_run+0x28/0x48
> [  100.511644]  ipi_handler+0x254/0x468
> [  100.511664]  handle_percpu_devid_irq+0x11c/0x518
> [  100.511681]  generic_handle_domain_irq+0x50/0x70
> [  100.511699]  gic_handle_irq+0xd4/0x118
> [  100.511711]  call_on_irq_stack+0x2c/0x58
> [  100.511725]  do_interrupt_handler+0xc0/0xc8
> [  100.511741]  el1_interrupt+0x40/0x68
> [  100.511754]  el1h_64_irq_handler+0x18/0x28
> [  100.511767]  el1h_64_irq+0x64/0x68
> [  100.511778]  irq_work_queue+0xc0/0xd8
> [  100.511790]  drm_sched_entity_fini+0x2c4/0x3b0
> [  100.511805]  drm_sched_entity_destroy+0x2c/0x40
> [  100.511818]  panfrost_job_close+0x44/0x1c0
> [  100.511833]  panfrost_postclose+0x38/0x60
> [  100.511845]  drm_file_free.part.0+0x33c/0x4b8
> [  100.511862]  drm_close_helper.isra.0+0xc0/0xd8
> [  100.511877]  drm_release+0xe4/0x1e0
> [  100.511891]  __fput+0xf8/0x390
> [  100.511904]  ____fput+0x18/0x28
> [  100.511917]  task_work_run+0xc4/0x1e0
> [  100.511929]  do_exit+0x554/0x1168
> [  100.511945]  do_group_exit+0x60/0x108
> [  100.511960]  __arm64_sys_exit_group+0x34/0x38
> [  100.511977]  invoke_syscall+0x64/0x180
> [  100.511993]  el0_svc_common.constprop.0+0x13c/0x170
> [  100.512012]  do_el0_svc+0x48/0xe8
> [  100.512028]  el0_svc+0x5c/0xe0
> [  100.512038]  el0t_64_sync_handler+0xb8/0xc0
> [  100.512051]  el0t_64_sync+0x18c/0x190
> [  100.512064]

This one shall be fixed by [1] that is not in the RC kernel yet, please
use linux-next.

[1]
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20220628&id=7d64c40a7d96190d9d06e240305389e025295916

-- 
Best regards,
Dmitry

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Dmitry Osipenko]

Hello Robin,

On 6/28/22 15:31, Robin Murphy wrote:
>> Hello,
>>
>> This patchset introduces memory shrinker for the VirtIO-GPU DRM driver
>> and adds memory purging and eviction support to VirtIO-GPU driver.
>>
>> The new dma-buf locking convention is introduced here as well.
>>
>> During OOM, the shrinker will release BOs that are marked as "not needed"
>> by userspace using the new madvise IOCTL, it will also evict idling BOs
>> to SWAP. The userspace in this case is the Mesa VirGL driver, it will
>> mark
>> the cached BOs as "not needed", allowing kernel driver to release memory
>> of the cached shmem BOs on lowmem situations, preventing OOM kills.
>>
>> The Panfrost driver is switched to use generic memory shrinker.
> 
> I think we still have some outstanding issues here - Alyssa reported
> some weirdness yesterday, so I just tried provoking a low-memory
> condition locally with this series applied and a few debug options
> enabled, and the results as below were... interesting.

The warning and crash that you got actually are the minor issues.

Alyssa caught an interesting PREEMPT_DEBUG issue in the shrinker that I
haven't seen before.

She is also experiencing another problem in the Panfrost driver with a
bad shmem pages (I think). It is unrelated to this patchset and
apparently require an extra setup for the reproduction.

-- 
Best regards,
Dmitry

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Dmitry Osipenko]

Hello Robin,

On 6/28/22 15:31, Robin Murphy wrote:
>> Hello,
>>
>> This patchset introduces memory shrinker for the VirtIO-GPU DRM driver
>> and adds memory purging and eviction support to VirtIO-GPU driver.
>>
>> The new dma-buf locking convention is introduced here as well.
>>
>> During OOM, the shrinker will release BOs that are marked as "not needed"
>> by userspace using the new madvise IOCTL, it will also evict idling BOs
>> to SWAP. The userspace in this case is the Mesa VirGL driver, it will
>> mark
>> the cached BOs as "not needed", allowing kernel driver to release memory
>> of the cached shmem BOs on lowmem situations, preventing OOM kills.
>>
>> The Panfrost driver is switched to use generic memory shrinker.
> 
> I think we still have some outstanding issues here - Alyssa reported
> some weirdness yesterday, so I just tried provoking a low-memory
> condition locally with this series applied and a few debug options
> enabled, and the results as below were... interesting.

The warning and crash that you got actually are the minor issues.

Alyssa caught an interesting PREEMPT_DEBUG issue in the shrinker that I
haven't seen before.

She is also experiencing another problem in the Panfrost driver with a
bad shmem pages (I think). It is unrelated to this patchset and
apparently require an extra setup for the reproduction.

-- 
Best regards,
Dmitry

Re: [Intel-gfx] [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Dmitry Osipenko]

Hello Robin,

On 6/28/22 15:31, Robin Murphy wrote:
>> Hello,
>>
>> This patchset introduces memory shrinker for the VirtIO-GPU DRM driver
>> and adds memory purging and eviction support to VirtIO-GPU driver.
>>
>> The new dma-buf locking convention is introduced here as well.
>>
>> During OOM, the shrinker will release BOs that are marked as "not needed"
>> by userspace using the new madvise IOCTL, it will also evict idling BOs
>> to SWAP. The userspace in this case is the Mesa VirGL driver, it will
>> mark
>> the cached BOs as "not needed", allowing kernel driver to release memory
>> of the cached shmem BOs on lowmem situations, preventing OOM kills.
>>
>> The Panfrost driver is switched to use generic memory shrinker.
> 
> I think we still have some outstanding issues here - Alyssa reported
> some weirdness yesterday, so I just tried provoking a low-memory
> condition locally with this series applied and a few debug options
> enabled, and the results as below were... interesting.

The warning and crash that you got actually are the minor issues.

Alyssa caught an interesting PREEMPT_DEBUG issue in the shrinker that I
haven't seen before.

She is also experiencing another problem in the Panfrost driver with a
bad shmem pages (I think). It is unrelated to this patchset and
apparently require an extra setup for the reproduction.

-- 
Best regards,
Dmitry

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Rob Clark]

On Tue, Jun 28, 2022 at 5:51 AM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> On 6/28/22 15:31, Robin Murphy wrote:
> > ----->8-----
> > [   68.295951] ======================================================
> > [   68.295956] WARNING: possible circular locking dependency detected
> > [   68.295963] 5.19.0-rc3+ #400 Not tainted
> > [   68.295972] ------------------------------------------------------
> > [   68.295977] cc1/295 is trying to acquire lock:
> > [   68.295986] ffff000008d7f1a0
> > (reservation_ww_class_mutex){+.+.}-{3:3}, at: drm_gem_shmem_free+0x7c/0x198
> > [   68.296036]
> > [   68.296036] but task is already holding lock:
> > [   68.296041] ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at:
> > __alloc_pages_slowpath.constprop.0+0x4d8/0x1470
> > [   68.296080]
> > [   68.296080] which lock already depends on the new lock.
> > [   68.296080]
> > [   68.296085]
> > [   68.296085] the existing dependency chain (in reverse order) is:
> > [   68.296090]
> > [   68.296090] -> #1 (fs_reclaim){+.+.}-{0:0}:
> > [   68.296111]        fs_reclaim_acquire+0xb8/0x150
> > [   68.296130]        dma_resv_lockdep+0x298/0x3fc
> > [   68.296148]        do_one_initcall+0xe4/0x5f8
> > [   68.296163]        kernel_init_freeable+0x414/0x49c
> > [   68.296180]        kernel_init+0x2c/0x148
> > [   68.296195]        ret_from_fork+0x10/0x20
> > [   68.296207]
> > [   68.296207] -> #0 (reservation_ww_class_mutex){+.+.}-{3:3}:
> > [   68.296229]        __lock_acquire+0x1724/0x2398
> > [   68.296246]        lock_acquire+0x218/0x5b0
> > [   68.296260]        __ww_mutex_lock.constprop.0+0x158/0x2378
> > [   68.296277]        ww_mutex_lock+0x7c/0x4d8
> > [   68.296291]        drm_gem_shmem_free+0x7c/0x198
> > [   68.296304]        panfrost_gem_free_object+0x118/0x138
> > [   68.296318]        drm_gem_object_free+0x40/0x68
> > [   68.296334]        drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
> > [   68.296352]        drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
> > [   68.296368]        do_shrink_slab+0x220/0x808
> > [   68.296381]        shrink_slab+0x11c/0x408
> > [   68.296392]        shrink_node+0x6ac/0xb90
> > [   68.296403]        do_try_to_free_pages+0x1dc/0x8d0
> > [   68.296416]        try_to_free_pages+0x1ec/0x5b0
> > [   68.296429]        __alloc_pages_slowpath.constprop.0+0x528/0x1470
> > [   68.296444]        __alloc_pages+0x4e0/0x5b8
> > [   68.296455]        __folio_alloc+0x24/0x60
> > [   68.296467]        vma_alloc_folio+0xb8/0x2f8
> > [   68.296483]        alloc_zeroed_user_highpage_movable+0x58/0x68
> > [   68.296498]        __handle_mm_fault+0x918/0x12a8
> > [   68.296513]        handle_mm_fault+0x130/0x300
> > [   68.296527]        do_page_fault+0x1d0/0x568
> > [   68.296539]        do_translation_fault+0xa0/0xb8
> > [   68.296551]        do_mem_abort+0x68/0xf8
> > [   68.296562]        el0_da+0x74/0x100
> > [   68.296572]        el0t_64_sync_handler+0x68/0xc0
> > [   68.296585]        el0t_64_sync+0x18c/0x190
> > [   68.296596]
> > [   68.296596] other info that might help us debug this:
> > [   68.296596]
> > [   68.296601]  Possible unsafe locking scenario:
> > [   68.296601]
> > [   68.296604]        CPU0                    CPU1
> > [   68.296608]        ----                    ----
> > [   68.296612]   lock(fs_reclaim);
> > [   68.296622] lock(reservation_ww_class_mutex);
> > [   68.296633]                                lock(fs_reclaim);
> > [   68.296644]   lock(reservation_ww_class_mutex);
> > [   68.296654]
> > [   68.296654]  *** DEADLOCK ***
>
> This splat could be ignored for now. I'm aware about it, although
> haven't looked closely at how to fix it since it's a kind of a lockdep
> misreporting.

The lockdep splat could be fixed with something similar to what I've
done in msm, ie. basically just not acquire the lock in the finalizer:

https://patchwork.freedesktop.org/patch/489364/

There is one gotcha to watch for, as danvet pointed out
(scan_objects() could still see the obj in the LRU before the
finalizer removes it), but if scan_objects() does the
kref_get_unless_zero() trick, it is safe.

BR,
-R
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Rob Clark]

On Tue, Jun 28, 2022 at 5:51 AM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> On 6/28/22 15:31, Robin Murphy wrote:
> > ----->8-----
> > [   68.295951] ======================================================
> > [   68.295956] WARNING: possible circular locking dependency detected
> > [   68.295963] 5.19.0-rc3+ #400 Not tainted
> > [   68.295972] ------------------------------------------------------
> > [   68.295977] cc1/295 is trying to acquire lock:
> > [   68.295986] ffff000008d7f1a0
> > (reservation_ww_class_mutex){+.+.}-{3:3}, at: drm_gem_shmem_free+0x7c/0x198
> > [   68.296036]
> > [   68.296036] but task is already holding lock:
> > [   68.296041] ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at:
> > __alloc_pages_slowpath.constprop.0+0x4d8/0x1470
> > [   68.296080]
> > [   68.296080] which lock already depends on the new lock.
> > [   68.296080]
> > [   68.296085]
> > [   68.296085] the existing dependency chain (in reverse order) is:
> > [   68.296090]
> > [   68.296090] -> #1 (fs_reclaim){+.+.}-{0:0}:
> > [   68.296111]        fs_reclaim_acquire+0xb8/0x150
> > [   68.296130]        dma_resv_lockdep+0x298/0x3fc
> > [   68.296148]        do_one_initcall+0xe4/0x5f8
> > [   68.296163]        kernel_init_freeable+0x414/0x49c
> > [   68.296180]        kernel_init+0x2c/0x148
> > [   68.296195]        ret_from_fork+0x10/0x20
> > [   68.296207]
> > [   68.296207] -> #0 (reservation_ww_class_mutex){+.+.}-{3:3}:
> > [   68.296229]        __lock_acquire+0x1724/0x2398
> > [   68.296246]        lock_acquire+0x218/0x5b0
> > [   68.296260]        __ww_mutex_lock.constprop.0+0x158/0x2378
> > [   68.296277]        ww_mutex_lock+0x7c/0x4d8
> > [   68.296291]        drm_gem_shmem_free+0x7c/0x198
> > [   68.296304]        panfrost_gem_free_object+0x118/0x138
> > [   68.296318]        drm_gem_object_free+0x40/0x68
> > [   68.296334]        drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
> > [   68.296352]        drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
> > [   68.296368]        do_shrink_slab+0x220/0x808
> > [   68.296381]        shrink_slab+0x11c/0x408
> > [   68.296392]        shrink_node+0x6ac/0xb90
> > [   68.296403]        do_try_to_free_pages+0x1dc/0x8d0
> > [   68.296416]        try_to_free_pages+0x1ec/0x5b0
> > [   68.296429]        __alloc_pages_slowpath.constprop.0+0x528/0x1470
> > [   68.296444]        __alloc_pages+0x4e0/0x5b8
> > [   68.296455]        __folio_alloc+0x24/0x60
> > [   68.296467]        vma_alloc_folio+0xb8/0x2f8
> > [   68.296483]        alloc_zeroed_user_highpage_movable+0x58/0x68
> > [   68.296498]        __handle_mm_fault+0x918/0x12a8
> > [   68.296513]        handle_mm_fault+0x130/0x300
> > [   68.296527]        do_page_fault+0x1d0/0x568
> > [   68.296539]        do_translation_fault+0xa0/0xb8
> > [   68.296551]        do_mem_abort+0x68/0xf8
> > [   68.296562]        el0_da+0x74/0x100
> > [   68.296572]        el0t_64_sync_handler+0x68/0xc0
> > [   68.296585]        el0t_64_sync+0x18c/0x190
> > [   68.296596]
> > [   68.296596] other info that might help us debug this:
> > [   68.296596]
> > [   68.296601]  Possible unsafe locking scenario:
> > [   68.296601]
> > [   68.296604]        CPU0                    CPU1
> > [   68.296608]        ----                    ----
> > [   68.296612]   lock(fs_reclaim);
> > [   68.296622] lock(reservation_ww_class_mutex);
> > [   68.296633]                                lock(fs_reclaim);
> > [   68.296644]   lock(reservation_ww_class_mutex);
> > [   68.296654]
> > [   68.296654]  *** DEADLOCK ***
>
> This splat could be ignored for now. I'm aware about it, although
> haven't looked closely at how to fix it since it's a kind of a lockdep
> misreporting.

The lockdep splat could be fixed with something similar to what I've
done in msm, ie. basically just not acquire the lock in the finalizer:

https://patchwork.freedesktop.org/patch/489364/

There is one gotcha to watch for, as danvet pointed out
(scan_objects() could still see the obj in the LRU before the
finalizer removes it), but if scan_objects() does the
kref_get_unless_zero() trick, it is safe.

BR,
-R

Re: [Intel-gfx] [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Rob Clark]

On Tue, Jun 28, 2022 at 5:51 AM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> On 6/28/22 15:31, Robin Murphy wrote:
> > ----->8-----
> > [   68.295951] ======================================================
> > [   68.295956] WARNING: possible circular locking dependency detected
> > [   68.295963] 5.19.0-rc3+ #400 Not tainted
> > [   68.295972] ------------------------------------------------------
> > [   68.295977] cc1/295 is trying to acquire lock:
> > [   68.295986] ffff000008d7f1a0
> > (reservation_ww_class_mutex){+.+.}-{3:3}, at: drm_gem_shmem_free+0x7c/0x198
> > [   68.296036]
> > [   68.296036] but task is already holding lock:
> > [   68.296041] ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at:
> > __alloc_pages_slowpath.constprop.0+0x4d8/0x1470
> > [   68.296080]
> > [   68.296080] which lock already depends on the new lock.
> > [   68.296080]
> > [   68.296085]
> > [   68.296085] the existing dependency chain (in reverse order) is:
> > [   68.296090]
> > [   68.296090] -> #1 (fs_reclaim){+.+.}-{0:0}:
> > [   68.296111]        fs_reclaim_acquire+0xb8/0x150
> > [   68.296130]        dma_resv_lockdep+0x298/0x3fc
> > [   68.296148]        do_one_initcall+0xe4/0x5f8
> > [   68.296163]        kernel_init_freeable+0x414/0x49c
> > [   68.296180]        kernel_init+0x2c/0x148
> > [   68.296195]        ret_from_fork+0x10/0x20
> > [   68.296207]
> > [   68.296207] -> #0 (reservation_ww_class_mutex){+.+.}-{3:3}:
> > [   68.296229]        __lock_acquire+0x1724/0x2398
> > [   68.296246]        lock_acquire+0x218/0x5b0
> > [   68.296260]        __ww_mutex_lock.constprop.0+0x158/0x2378
> > [   68.296277]        ww_mutex_lock+0x7c/0x4d8
> > [   68.296291]        drm_gem_shmem_free+0x7c/0x198
> > [   68.296304]        panfrost_gem_free_object+0x118/0x138
> > [   68.296318]        drm_gem_object_free+0x40/0x68
> > [   68.296334]        drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
> > [   68.296352]        drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
> > [   68.296368]        do_shrink_slab+0x220/0x808
> > [   68.296381]        shrink_slab+0x11c/0x408
> > [   68.296392]        shrink_node+0x6ac/0xb90
> > [   68.296403]        do_try_to_free_pages+0x1dc/0x8d0
> > [   68.296416]        try_to_free_pages+0x1ec/0x5b0
> > [   68.296429]        __alloc_pages_slowpath.constprop.0+0x528/0x1470
> > [   68.296444]        __alloc_pages+0x4e0/0x5b8
> > [   68.296455]        __folio_alloc+0x24/0x60
> > [   68.296467]        vma_alloc_folio+0xb8/0x2f8
> > [   68.296483]        alloc_zeroed_user_highpage_movable+0x58/0x68
> > [   68.296498]        __handle_mm_fault+0x918/0x12a8
> > [   68.296513]        handle_mm_fault+0x130/0x300
> > [   68.296527]        do_page_fault+0x1d0/0x568
> > [   68.296539]        do_translation_fault+0xa0/0xb8
> > [   68.296551]        do_mem_abort+0x68/0xf8
> > [   68.296562]        el0_da+0x74/0x100
> > [   68.296572]        el0t_64_sync_handler+0x68/0xc0
> > [   68.296585]        el0t_64_sync+0x18c/0x190
> > [   68.296596]
> > [   68.296596] other info that might help us debug this:
> > [   68.296596]
> > [   68.296601]  Possible unsafe locking scenario:
> > [   68.296601]
> > [   68.296604]        CPU0                    CPU1
> > [   68.296608]        ----                    ----
> > [   68.296612]   lock(fs_reclaim);
> > [   68.296622] lock(reservation_ww_class_mutex);
> > [   68.296633]                                lock(fs_reclaim);
> > [   68.296644]   lock(reservation_ww_class_mutex);
> > [   68.296654]
> > [   68.296654]  *** DEADLOCK ***
>
> This splat could be ignored for now. I'm aware about it, although
> haven't looked closely at how to fix it since it's a kind of a lockdep
> misreporting.

The lockdep splat could be fixed with something similar to what I've
done in msm, ie. basically just not acquire the lock in the finalizer:

https://patchwork.freedesktop.org/patch/489364/

There is one gotcha to watch for, as danvet pointed out
(scan_objects() could still see the obj in the LRU before the
finalizer removes it), but if scan_objects() does the
kref_get_unless_zero() trick, it is safe.

BR,
-R

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Rob Clark]

On Tue, Jun 28, 2022 at 5:51 AM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> On 6/28/22 15:31, Robin Murphy wrote:
> > ----->8-----
> > [   68.295951] ======================================================
> > [   68.295956] WARNING: possible circular locking dependency detected
> > [   68.295963] 5.19.0-rc3+ #400 Not tainted
> > [   68.295972] ------------------------------------------------------
> > [   68.295977] cc1/295 is trying to acquire lock:
> > [   68.295986] ffff000008d7f1a0
> > (reservation_ww_class_mutex){+.+.}-{3:3}, at: drm_gem_shmem_free+0x7c/0x198
> > [   68.296036]
> > [   68.296036] but task is already holding lock:
> > [   68.296041] ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at:
> > __alloc_pages_slowpath.constprop.0+0x4d8/0x1470
> > [   68.296080]
> > [   68.296080] which lock already depends on the new lock.
> > [   68.296080]
> > [   68.296085]
> > [   68.296085] the existing dependency chain (in reverse order) is:
> > [   68.296090]
> > [   68.296090] -> #1 (fs_reclaim){+.+.}-{0:0}:
> > [   68.296111]        fs_reclaim_acquire+0xb8/0x150
> > [   68.296130]        dma_resv_lockdep+0x298/0x3fc
> > [   68.296148]        do_one_initcall+0xe4/0x5f8
> > [   68.296163]        kernel_init_freeable+0x414/0x49c
> > [   68.296180]        kernel_init+0x2c/0x148
> > [   68.296195]        ret_from_fork+0x10/0x20
> > [   68.296207]
> > [   68.296207] -> #0 (reservation_ww_class_mutex){+.+.}-{3:3}:
> > [   68.296229]        __lock_acquire+0x1724/0x2398
> > [   68.296246]        lock_acquire+0x218/0x5b0
> > [   68.296260]        __ww_mutex_lock.constprop.0+0x158/0x2378
> > [   68.296277]        ww_mutex_lock+0x7c/0x4d8
> > [   68.296291]        drm_gem_shmem_free+0x7c/0x198
> > [   68.296304]        panfrost_gem_free_object+0x118/0x138
> > [   68.296318]        drm_gem_object_free+0x40/0x68
> > [   68.296334]        drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
> > [   68.296352]        drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
> > [   68.296368]        do_shrink_slab+0x220/0x808
> > [   68.296381]        shrink_slab+0x11c/0x408
> > [   68.296392]        shrink_node+0x6ac/0xb90
> > [   68.296403]        do_try_to_free_pages+0x1dc/0x8d0
> > [   68.296416]        try_to_free_pages+0x1ec/0x5b0
> > [   68.296429]        __alloc_pages_slowpath.constprop.0+0x528/0x1470
> > [   68.296444]        __alloc_pages+0x4e0/0x5b8
> > [   68.296455]        __folio_alloc+0x24/0x60
> > [   68.296467]        vma_alloc_folio+0xb8/0x2f8
> > [   68.296483]        alloc_zeroed_user_highpage_movable+0x58/0x68
> > [   68.296498]        __handle_mm_fault+0x918/0x12a8
> > [   68.296513]        handle_mm_fault+0x130/0x300
> > [   68.296527]        do_page_fault+0x1d0/0x568
> > [   68.296539]        do_translation_fault+0xa0/0xb8
> > [   68.296551]        do_mem_abort+0x68/0xf8
> > [   68.296562]        el0_da+0x74/0x100
> > [   68.296572]        el0t_64_sync_handler+0x68/0xc0
> > [   68.296585]        el0t_64_sync+0x18c/0x190
> > [   68.296596]
> > [   68.296596] other info that might help us debug this:
> > [   68.296596]
> > [   68.296601]  Possible unsafe locking scenario:
> > [   68.296601]
> > [   68.296604]        CPU0                    CPU1
> > [   68.296608]        ----                    ----
> > [   68.296612]   lock(fs_reclaim);
> > [   68.296622] lock(reservation_ww_class_mutex);
> > [   68.296633]                                lock(fs_reclaim);
> > [   68.296644]   lock(reservation_ww_class_mutex);
> > [   68.296654]
> > [   68.296654]  *** DEADLOCK ***
>
> This splat could be ignored for now. I'm aware about it, although
> haven't looked closely at how to fix it since it's a kind of a lockdep
> misreporting.

The lockdep splat could be fixed with something similar to what I've
done in msm, ie. basically just not acquire the lock in the finalizer:

https://patchwork.freedesktop.org/patch/489364/

There is one gotcha to watch for, as danvet pointed out
(scan_objects() could still see the obj in the LRU before the
finalizer removes it), but if scan_objects() does the
kref_get_unless_zero() trick, it is safe.

BR,
-R

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Rob Clark]

On Tue, Jun 28, 2022 at 5:51 AM Dmitry Osipenko
<dmitry.osipenko@collabora.com> wrote:
>
> On 6/28/22 15:31, Robin Murphy wrote:
> > ----->8-----
> > [   68.295951] ======================================================
> > [   68.295956] WARNING: possible circular locking dependency detected
> > [   68.295963] 5.19.0-rc3+ #400 Not tainted
> > [   68.295972] ------------------------------------------------------
> > [   68.295977] cc1/295 is trying to acquire lock:
> > [   68.295986] ffff000008d7f1a0
> > (reservation_ww_class_mutex){+.+.}-{3:3}, at: drm_gem_shmem_free+0x7c/0x198
> > [   68.296036]
> > [   68.296036] but task is already holding lock:
> > [   68.296041] ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at:
> > __alloc_pages_slowpath.constprop.0+0x4d8/0x1470
> > [   68.296080]
> > [   68.296080] which lock already depends on the new lock.
> > [   68.296080]
> > [   68.296085]
> > [   68.296085] the existing dependency chain (in reverse order) is:
> > [   68.296090]
> > [   68.296090] -> #1 (fs_reclaim){+.+.}-{0:0}:
> > [   68.296111]        fs_reclaim_acquire+0xb8/0x150
> > [   68.296130]        dma_resv_lockdep+0x298/0x3fc
> > [   68.296148]        do_one_initcall+0xe4/0x5f8
> > [   68.296163]        kernel_init_freeable+0x414/0x49c
> > [   68.296180]        kernel_init+0x2c/0x148
> > [   68.296195]        ret_from_fork+0x10/0x20
> > [   68.296207]
> > [   68.296207] -> #0 (reservation_ww_class_mutex){+.+.}-{3:3}:
> > [   68.296229]        __lock_acquire+0x1724/0x2398
> > [   68.296246]        lock_acquire+0x218/0x5b0
> > [   68.296260]        __ww_mutex_lock.constprop.0+0x158/0x2378
> > [   68.296277]        ww_mutex_lock+0x7c/0x4d8
> > [   68.296291]        drm_gem_shmem_free+0x7c/0x198
> > [   68.296304]        panfrost_gem_free_object+0x118/0x138
> > [   68.296318]        drm_gem_object_free+0x40/0x68
> > [   68.296334]        drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
> > [   68.296352]        drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
> > [   68.296368]        do_shrink_slab+0x220/0x808
> > [   68.296381]        shrink_slab+0x11c/0x408
> > [   68.296392]        shrink_node+0x6ac/0xb90
> > [   68.296403]        do_try_to_free_pages+0x1dc/0x8d0
> > [   68.296416]        try_to_free_pages+0x1ec/0x5b0
> > [   68.296429]        __alloc_pages_slowpath.constprop.0+0x528/0x1470
> > [   68.296444]        __alloc_pages+0x4e0/0x5b8
> > [   68.296455]        __folio_alloc+0x24/0x60
> > [   68.296467]        vma_alloc_folio+0xb8/0x2f8
> > [   68.296483]        alloc_zeroed_user_highpage_movable+0x58/0x68
> > [   68.296498]        __handle_mm_fault+0x918/0x12a8
> > [   68.296513]        handle_mm_fault+0x130/0x300
> > [   68.296527]        do_page_fault+0x1d0/0x568
> > [   68.296539]        do_translation_fault+0xa0/0xb8
> > [   68.296551]        do_mem_abort+0x68/0xf8
> > [   68.296562]        el0_da+0x74/0x100
> > [   68.296572]        el0t_64_sync_handler+0x68/0xc0
> > [   68.296585]        el0t_64_sync+0x18c/0x190
> > [   68.296596]
> > [   68.296596] other info that might help us debug this:
> > [   68.296596]
> > [   68.296601]  Possible unsafe locking scenario:
> > [   68.296601]
> > [   68.296604]        CPU0                    CPU1
> > [   68.296608]        ----                    ----
> > [   68.296612]   lock(fs_reclaim);
> > [   68.296622] lock(reservation_ww_class_mutex);
> > [   68.296633]                                lock(fs_reclaim);
> > [   68.296644]   lock(reservation_ww_class_mutex);
> > [   68.296654]
> > [   68.296654]  *** DEADLOCK ***
>
> This splat could be ignored for now. I'm aware about it, although
> haven't looked closely at how to fix it since it's a kind of a lockdep
> misreporting.

The lockdep splat could be fixed with something similar to what I've
done in msm, ie. basically just not acquire the lock in the finalizer:

https://patchwork.freedesktop.org/patch/489364/

There is one gotcha to watch for, as danvet pointed out
(scan_objects() could still see the obj in the LRU before the
finalizer removes it), but if scan_objects() does the
kref_get_unless_zero() trick, it is safe.

BR,
-R

Re: [PATCH v6 01/22] drm/gem: Properly annotate WW context on drm_gem_lock_reservations() error

[Thomas Hellström (Intel)]

Hi,

On 5/27/22 01:50, Dmitry Osipenko wrote:
> Use ww_acquire_fini() in the error code paths. Otherwise lockdep
> thinks that lock is held when lock's memory is freed after the
> drm_gem_lock_reservations() error. The WW needs to be annotated
> as "freed"

s /WW/ww_acquire_context/ ?
s /"freed"/"released"/ ?


> , which fixes the noisy "WARNING: held lock freed!" splat
> of VirtIO-GPU driver with CONFIG_DEBUG_MUTEXES=y and enabled lockdep.
>
> Cc: stable@vger.kernel.org

Can you dig up the commit in error and add a Fixes: Tag?

Using that and "dim fixes" will also make the Cc: stable tag a bit more 
verbose.

With that fixed,

Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>


> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>   drivers/gpu/drm/drm_gem.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
> index eb0c2d041f13..86d670c71286 100644
> --- a/drivers/gpu/drm/drm_gem.c
> +++ b/drivers/gpu/drm/drm_gem.c
> @@ -1226,7 +1226,7 @@ drm_gem_lock_reservations(struct drm_gem_object **objs, int count,
>   		ret = dma_resv_lock_slow_interruptible(obj->resv,
>   								 acquire_ctx);
>   		if (ret) {
> -			ww_acquire_done(acquire_ctx);
> +			ww_acquire_fini(acquire_ctx);
>   			return ret;
>   		}
>   	}
> @@ -1251,7 +1251,7 @@ drm_gem_lock_reservations(struct drm_gem_object **objs, int count,
>   				goto retry;
>   			}
>   
> -			ww_acquire_done(acquire_ctx);
> +			ww_acquire_fini(acquire_ctx);
>   			return ret;
>   		}
>   	}

Re: [Intel-gfx] [PATCH v6 01/22] drm/gem: Properly annotate WW context on drm_gem_lock_reservations() error

[Thomas Hellström (Intel)]

Hi,

On 5/27/22 01:50, Dmitry Osipenko wrote:
> Use ww_acquire_fini() in the error code paths. Otherwise lockdep
> thinks that lock is held when lock's memory is freed after the
> drm_gem_lock_reservations() error. The WW needs to be annotated
> as "freed"

s /WW/ww_acquire_context/ ?
s /"freed"/"released"/ ?


> , which fixes the noisy "WARNING: held lock freed!" splat
> of VirtIO-GPU driver with CONFIG_DEBUG_MUTEXES=y and enabled lockdep.
>
> Cc: stable@vger.kernel.org

Can you dig up the commit in error and add a Fixes: Tag?

Using that and "dim fixes" will also make the Cc: stable tag a bit more 
verbose.

With that fixed,

Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>


> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>   drivers/gpu/drm/drm_gem.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
> index eb0c2d041f13..86d670c71286 100644
> --- a/drivers/gpu/drm/drm_gem.c
> +++ b/drivers/gpu/drm/drm_gem.c
> @@ -1226,7 +1226,7 @@ drm_gem_lock_reservations(struct drm_gem_object **objs, int count,
>   		ret = dma_resv_lock_slow_interruptible(obj->resv,
>   								 acquire_ctx);
>   		if (ret) {
> -			ww_acquire_done(acquire_ctx);
> +			ww_acquire_fini(acquire_ctx);
>   			return ret;
>   		}
>   	}
> @@ -1251,7 +1251,7 @@ drm_gem_lock_reservations(struct drm_gem_object **objs, int count,
>   				goto retry;
>   			}
>   
> -			ww_acquire_done(acquire_ctx);
> +			ww_acquire_fini(acquire_ctx);
>   			return ret;
>   		}
>   	}

Re: [PATCH v6 01/22] drm/gem: Properly annotate WW context on drm_gem_lock_reservations() error

[Thomas Hellström (Intel)]

Hi,

On 5/27/22 01:50, Dmitry Osipenko wrote:
> Use ww_acquire_fini() in the error code paths. Otherwise lockdep
> thinks that lock is held when lock's memory is freed after the
> drm_gem_lock_reservations() error. The WW needs to be annotated
> as "freed"

s /WW/ww_acquire_context/ ?
s /"freed"/"released"/ ?


> , which fixes the noisy "WARNING: held lock freed!" splat
> of VirtIO-GPU driver with CONFIG_DEBUG_MUTEXES=y and enabled lockdep.
>
> Cc: stable@vger.kernel.org

Can you dig up the commit in error and add a Fixes: Tag?

Using that and "dim fixes" will also make the Cc: stable tag a bit more 
verbose.

With that fixed,

Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>


> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>   drivers/gpu/drm/drm_gem.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
> index eb0c2d041f13..86d670c71286 100644
> --- a/drivers/gpu/drm/drm_gem.c
> +++ b/drivers/gpu/drm/drm_gem.c
> @@ -1226,7 +1226,7 @@ drm_gem_lock_reservations(struct drm_gem_object **objs, int count,
>   		ret = dma_resv_lock_slow_interruptible(obj->resv,
>   								 acquire_ctx);
>   		if (ret) {
> -			ww_acquire_done(acquire_ctx);
> +			ww_acquire_fini(acquire_ctx);
>   			return ret;
>   		}
>   	}
> @@ -1251,7 +1251,7 @@ drm_gem_lock_reservations(struct drm_gem_object **objs, int count,
>   				goto retry;
>   			}
>   
> -			ww_acquire_done(acquire_ctx);
> +			ww_acquire_fini(acquire_ctx);
>   			return ret;
>   		}
>   	}

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Thomas Hellström (Intel)]

On 5/30/22 15:57, Dmitry Osipenko wrote:
> On 5/30/22 16:41, Christian König wrote:
>> Hi Dmitry,
>>
>> Am 30.05.22 um 15:26 schrieb Dmitry Osipenko:
>>> Hello Christian,
>>>
>>> On 5/30/22 09:50, Christian König wrote:
>>>> Hi Dmitry,
>>>>
>>>> First of all please separate out this patch from the rest of the series,
>>>> since this is a complex separate structural change.
>>> I assume all the patches will go via the DRM tree in the end since the
>>> rest of the DRM patches in this series depend on this dma-buf change.
>>> But I see that separation may ease reviewing of the dma-buf changes, so
>>> let's try it.
>> That sounds like you are underestimating a bit how much trouble this
>> will be.
>>
>>>> I have tried this before and failed because catching all the locks in
>>>> the right code paths are very tricky. So expect some fallout from this
>>>> and make sure the kernel test robot and CI systems are clean.
>>> Sure, I'll fix up all the reported things in the next iteration.
>>>
>>> BTW, have you ever posted yours version of the patch? Will be great if
>>> we could compare the changed code paths.
>> No, I never even finished creating it after realizing how much work it
>> would be.
>>
>>>>> This patch introduces new locking convention for dma-buf users. From
>>>>> now
>>>>> on all dma-buf importers are responsible for holding dma-buf
>>>>> reservation
>>>>> lock around operations performed over dma-bufs.
>>>>>
>>>>> This patch implements the new dma-buf locking convention by:
>>>>>
>>>>>      1. Making dma-buf API functions to take the reservation lock.
>>>>>
>>>>>      2. Adding new locked variants of the dma-buf API functions for
>>>>> drivers
>>>>>         that need to manage imported dma-bufs under the held lock.
>>>> Instead of adding new locked variants please mark all variants which
>>>> expect to be called without a lock with an _unlocked postfix.
>>>>
>>>> This should make it easier to remove those in a follow up patch set and
>>>> then fully move the locking into the importer.
>>> Do we really want to move all the locks to the importers? Seems the
>>> majority of drivers should be happy with the dma-buf helpers handling
>>> the locking for them.
>> Yes, I clearly think so.
>>
>>>>>      3. Converting all drivers to the new locking scheme.
>>>> I have strong doubts that you got all of them. At least radeon and
>>>> nouveau should grab the reservation lock in their ->attach callbacks
>>>> somehow.
>>> Radeon and Nouveau use gem_prime_import_sg_table() and they take resv
>>> lock already, seems they should be okay (?)
>> You are looking at the wrong side. You need to fix the export code path,
>> not the import ones.
>>
>> See for example attach on radeon works like this
>> drm_gem_map_attach->drm_gem_pin->radeon_gem_prime_pin->radeon_bo_reserve->ttm_bo_reserve->dma_resv_lock.
> Yeah, I was looking at the both sides, but missed this one.

Also i915 will run into trouble with attach. In particular since i915 
starts a full ww transaction in its attach callback to be able to lock 
other objects if migration is needed. I think i915 CI would catch this 
in a selftest.

Perhaps it's worthwile to take a step back and figure out, if the 
importer is required to lock, which callbacks might need a ww acquire 
context?

(And off-topic, Since we do a lot of fancy stuff under dma-resv locks 
including waiting for fences and other locks, IMO taking these locks 
uninterruptible should ring a warning bell)

/Thomas

>
>> Same for nouveau and probably a few other exporters as well. That will
>> certainly cause a deadlock if you don't fix it.
>>
>> I strongly suggest to do this step by step, first attach/detach and then
>> the rest.
> Thank you very much for the suggestions. I'll implement them in the next
> version.
>

Re: [Intel-gfx] [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Thomas Hellström (Intel)]

On 5/30/22 15:57, Dmitry Osipenko wrote:
> On 5/30/22 16:41, Christian König wrote:
>> Hi Dmitry,
>>
>> Am 30.05.22 um 15:26 schrieb Dmitry Osipenko:
>>> Hello Christian,
>>>
>>> On 5/30/22 09:50, Christian König wrote:
>>>> Hi Dmitry,
>>>>
>>>> First of all please separate out this patch from the rest of the series,
>>>> since this is a complex separate structural change.
>>> I assume all the patches will go via the DRM tree in the end since the
>>> rest of the DRM patches in this series depend on this dma-buf change.
>>> But I see that separation may ease reviewing of the dma-buf changes, so
>>> let's try it.
>> That sounds like you are underestimating a bit how much trouble this
>> will be.
>>
>>>> I have tried this before and failed because catching all the locks in
>>>> the right code paths are very tricky. So expect some fallout from this
>>>> and make sure the kernel test robot and CI systems are clean.
>>> Sure, I'll fix up all the reported things in the next iteration.
>>>
>>> BTW, have you ever posted yours version of the patch? Will be great if
>>> we could compare the changed code paths.
>> No, I never even finished creating it after realizing how much work it
>> would be.
>>
>>>>> This patch introduces new locking convention for dma-buf users. From
>>>>> now
>>>>> on all dma-buf importers are responsible for holding dma-buf
>>>>> reservation
>>>>> lock around operations performed over dma-bufs.
>>>>>
>>>>> This patch implements the new dma-buf locking convention by:
>>>>>
>>>>>      1. Making dma-buf API functions to take the reservation lock.
>>>>>
>>>>>      2. Adding new locked variants of the dma-buf API functions for
>>>>> drivers
>>>>>         that need to manage imported dma-bufs under the held lock.
>>>> Instead of adding new locked variants please mark all variants which
>>>> expect to be called without a lock with an _unlocked postfix.
>>>>
>>>> This should make it easier to remove those in a follow up patch set and
>>>> then fully move the locking into the importer.
>>> Do we really want to move all the locks to the importers? Seems the
>>> majority of drivers should be happy with the dma-buf helpers handling
>>> the locking for them.
>> Yes, I clearly think so.
>>
>>>>>      3. Converting all drivers to the new locking scheme.
>>>> I have strong doubts that you got all of them. At least radeon and
>>>> nouveau should grab the reservation lock in their ->attach callbacks
>>>> somehow.
>>> Radeon and Nouveau use gem_prime_import_sg_table() and they take resv
>>> lock already, seems they should be okay (?)
>> You are looking at the wrong side. You need to fix the export code path,
>> not the import ones.
>>
>> See for example attach on radeon works like this
>> drm_gem_map_attach->drm_gem_pin->radeon_gem_prime_pin->radeon_bo_reserve->ttm_bo_reserve->dma_resv_lock.
> Yeah, I was looking at the both sides, but missed this one.

Also i915 will run into trouble with attach. In particular since i915 
starts a full ww transaction in its attach callback to be able to lock 
other objects if migration is needed. I think i915 CI would catch this 
in a selftest.

Perhaps it's worthwile to take a step back and figure out, if the 
importer is required to lock, which callbacks might need a ww acquire 
context?

(And off-topic, Since we do a lot of fancy stuff under dma-resv locks 
including waiting for fences and other locks, IMO taking these locks 
uninterruptible should ring a warning bell)

/Thomas

>
>> Same for nouveau and probably a few other exporters as well. That will
>> certainly cause a deadlock if you don't fix it.
>>
>> I strongly suggest to do this step by step, first attach/detach and then
>> the rest.
> Thank you very much for the suggestions. I'll implement them in the next
> version.
>

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Thomas Hellström (Intel)]

On 5/30/22 15:57, Dmitry Osipenko wrote:
> On 5/30/22 16:41, Christian König wrote:
>> Hi Dmitry,
>>
>> Am 30.05.22 um 15:26 schrieb Dmitry Osipenko:
>>> Hello Christian,
>>>
>>> On 5/30/22 09:50, Christian König wrote:
>>>> Hi Dmitry,
>>>>
>>>> First of all please separate out this patch from the rest of the series,
>>>> since this is a complex separate structural change.
>>> I assume all the patches will go via the DRM tree in the end since the
>>> rest of the DRM patches in this series depend on this dma-buf change.
>>> But I see that separation may ease reviewing of the dma-buf changes, so
>>> let's try it.
>> That sounds like you are underestimating a bit how much trouble this
>> will be.
>>
>>>> I have tried this before and failed because catching all the locks in
>>>> the right code paths are very tricky. So expect some fallout from this
>>>> and make sure the kernel test robot and CI systems are clean.
>>> Sure, I'll fix up all the reported things in the next iteration.
>>>
>>> BTW, have you ever posted yours version of the patch? Will be great if
>>> we could compare the changed code paths.
>> No, I never even finished creating it after realizing how much work it
>> would be.
>>
>>>>> This patch introduces new locking convention for dma-buf users. From
>>>>> now
>>>>> on all dma-buf importers are responsible for holding dma-buf
>>>>> reservation
>>>>> lock around operations performed over dma-bufs.
>>>>>
>>>>> This patch implements the new dma-buf locking convention by:
>>>>>
>>>>>      1. Making dma-buf API functions to take the reservation lock.
>>>>>
>>>>>      2. Adding new locked variants of the dma-buf API functions for
>>>>> drivers
>>>>>         that need to manage imported dma-bufs under the held lock.
>>>> Instead of adding new locked variants please mark all variants which
>>>> expect to be called without a lock with an _unlocked postfix.
>>>>
>>>> This should make it easier to remove those in a follow up patch set and
>>>> then fully move the locking into the importer.
>>> Do we really want to move all the locks to the importers? Seems the
>>> majority of drivers should be happy with the dma-buf helpers handling
>>> the locking for them.
>> Yes, I clearly think so.
>>
>>>>>      3. Converting all drivers to the new locking scheme.
>>>> I have strong doubts that you got all of them. At least radeon and
>>>> nouveau should grab the reservation lock in their ->attach callbacks
>>>> somehow.
>>> Radeon and Nouveau use gem_prime_import_sg_table() and they take resv
>>> lock already, seems they should be okay (?)
>> You are looking at the wrong side. You need to fix the export code path,
>> not the import ones.
>>
>> See for example attach on radeon works like this
>> drm_gem_map_attach->drm_gem_pin->radeon_gem_prime_pin->radeon_bo_reserve->ttm_bo_reserve->dma_resv_lock.
> Yeah, I was looking at the both sides, but missed this one.

Also i915 will run into trouble with attach. In particular since i915 
starts a full ww transaction in its attach callback to be able to lock 
other objects if migration is needed. I think i915 CI would catch this 
in a selftest.

Perhaps it's worthwile to take a step back and figure out, if the 
importer is required to lock, which callbacks might need a ww acquire 
context?

(And off-topic, Since we do a lot of fancy stuff under dma-resv locks 
including waiting for fences and other locks, IMO taking these locks 
uninterruptible should ring a warning bell)

/Thomas

>
>> Same for nouveau and probably a few other exporters as well. That will
>> certainly cause a deadlock if you don't fix it.
>>
>> I strongly suggest to do this step by step, first attach/detach and then
>> the rest.
> Thank you very much for the suggestions. I'll implement them in the next
> version.
>

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Dmitry Osipenko]

On 6/28/22 19:48, Rob Clark wrote:
> On Tue, Jun 28, 2022 at 5:51 AM Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
>>
>> On 6/28/22 15:31, Robin Murphy wrote:
>>> ----->8-----
>>> [   68.295951] ======================================================
>>> [   68.295956] WARNING: possible circular locking dependency detected
>>> [   68.295963] 5.19.0-rc3+ #400 Not tainted
>>> [   68.295972] ------------------------------------------------------
>>> [   68.295977] cc1/295 is trying to acquire lock:
>>> [   68.295986] ffff000008d7f1a0
>>> (reservation_ww_class_mutex){+.+.}-{3:3}, at: drm_gem_shmem_free+0x7c/0x198
>>> [   68.296036]
>>> [   68.296036] but task is already holding lock:
>>> [   68.296041] ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at:
>>> __alloc_pages_slowpath.constprop.0+0x4d8/0x1470
>>> [   68.296080]
>>> [   68.296080] which lock already depends on the new lock.
>>> [   68.296080]
>>> [   68.296085]
>>> [   68.296085] the existing dependency chain (in reverse order) is:
>>> [   68.296090]
>>> [   68.296090] -> #1 (fs_reclaim){+.+.}-{0:0}:
>>> [   68.296111]        fs_reclaim_acquire+0xb8/0x150
>>> [   68.296130]        dma_resv_lockdep+0x298/0x3fc
>>> [   68.296148]        do_one_initcall+0xe4/0x5f8
>>> [   68.296163]        kernel_init_freeable+0x414/0x49c
>>> [   68.296180]        kernel_init+0x2c/0x148
>>> [   68.296195]        ret_from_fork+0x10/0x20
>>> [   68.296207]
>>> [   68.296207] -> #0 (reservation_ww_class_mutex){+.+.}-{3:3}:
>>> [   68.296229]        __lock_acquire+0x1724/0x2398
>>> [   68.296246]        lock_acquire+0x218/0x5b0
>>> [   68.296260]        __ww_mutex_lock.constprop.0+0x158/0x2378
>>> [   68.296277]        ww_mutex_lock+0x7c/0x4d8
>>> [   68.296291]        drm_gem_shmem_free+0x7c/0x198
>>> [   68.296304]        panfrost_gem_free_object+0x118/0x138
>>> [   68.296318]        drm_gem_object_free+0x40/0x68
>>> [   68.296334]        drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
>>> [   68.296352]        drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
>>> [   68.296368]        do_shrink_slab+0x220/0x808
>>> [   68.296381]        shrink_slab+0x11c/0x408
>>> [   68.296392]        shrink_node+0x6ac/0xb90
>>> [   68.296403]        do_try_to_free_pages+0x1dc/0x8d0
>>> [   68.296416]        try_to_free_pages+0x1ec/0x5b0
>>> [   68.296429]        __alloc_pages_slowpath.constprop.0+0x528/0x1470
>>> [   68.296444]        __alloc_pages+0x4e0/0x5b8
>>> [   68.296455]        __folio_alloc+0x24/0x60
>>> [   68.296467]        vma_alloc_folio+0xb8/0x2f8
>>> [   68.296483]        alloc_zeroed_user_highpage_movable+0x58/0x68
>>> [   68.296498]        __handle_mm_fault+0x918/0x12a8
>>> [   68.296513]        handle_mm_fault+0x130/0x300
>>> [   68.296527]        do_page_fault+0x1d0/0x568
>>> [   68.296539]        do_translation_fault+0xa0/0xb8
>>> [   68.296551]        do_mem_abort+0x68/0xf8
>>> [   68.296562]        el0_da+0x74/0x100
>>> [   68.296572]        el0t_64_sync_handler+0x68/0xc0
>>> [   68.296585]        el0t_64_sync+0x18c/0x190
>>> [   68.296596]
>>> [   68.296596] other info that might help us debug this:
>>> [   68.296596]
>>> [   68.296601]  Possible unsafe locking scenario:
>>> [   68.296601]
>>> [   68.296604]        CPU0                    CPU1
>>> [   68.296608]        ----                    ----
>>> [   68.296612]   lock(fs_reclaim);
>>> [   68.296622] lock(reservation_ww_class_mutex);
>>> [   68.296633]                                lock(fs_reclaim);
>>> [   68.296644]   lock(reservation_ww_class_mutex);
>>> [   68.296654]
>>> [   68.296654]  *** DEADLOCK ***
>>
>> This splat could be ignored for now. I'm aware about it, although
>> haven't looked closely at how to fix it since it's a kind of a lockdep
>> misreporting.
> 
> The lockdep splat could be fixed with something similar to what I've
> done in msm, ie. basically just not acquire the lock in the finalizer:
> 
> https://patchwork.freedesktop.org/patch/489364/
> 
> There is one gotcha to watch for, as danvet pointed out
> (scan_objects() could still see the obj in the LRU before the
> finalizer removes it), but if scan_objects() does the
> kref_get_unless_zero() trick, it is safe.

Nice, thank you!

-- 
Best regards,
Dmitry

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Dmitry Osipenko]

On 6/28/22 19:48, Rob Clark wrote:
> On Tue, Jun 28, 2022 at 5:51 AM Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
>>
>> On 6/28/22 15:31, Robin Murphy wrote:
>>> ----->8-----
>>> [   68.295951] ======================================================
>>> [   68.295956] WARNING: possible circular locking dependency detected
>>> [   68.295963] 5.19.0-rc3+ #400 Not tainted
>>> [   68.295972] ------------------------------------------------------
>>> [   68.295977] cc1/295 is trying to acquire lock:
>>> [   68.295986] ffff000008d7f1a0
>>> (reservation_ww_class_mutex){+.+.}-{3:3}, at: drm_gem_shmem_free+0x7c/0x198
>>> [   68.296036]
>>> [   68.296036] but task is already holding lock:
>>> [   68.296041] ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at:
>>> __alloc_pages_slowpath.constprop.0+0x4d8/0x1470
>>> [   68.296080]
>>> [   68.296080] which lock already depends on the new lock.
>>> [   68.296080]
>>> [   68.296085]
>>> [   68.296085] the existing dependency chain (in reverse order) is:
>>> [   68.296090]
>>> [   68.296090] -> #1 (fs_reclaim){+.+.}-{0:0}:
>>> [   68.296111]        fs_reclaim_acquire+0xb8/0x150
>>> [   68.296130]        dma_resv_lockdep+0x298/0x3fc
>>> [   68.296148]        do_one_initcall+0xe4/0x5f8
>>> [   68.296163]        kernel_init_freeable+0x414/0x49c
>>> [   68.296180]        kernel_init+0x2c/0x148
>>> [   68.296195]        ret_from_fork+0x10/0x20
>>> [   68.296207]
>>> [   68.296207] -> #0 (reservation_ww_class_mutex){+.+.}-{3:3}:
>>> [   68.296229]        __lock_acquire+0x1724/0x2398
>>> [   68.296246]        lock_acquire+0x218/0x5b0
>>> [   68.296260]        __ww_mutex_lock.constprop.0+0x158/0x2378
>>> [   68.296277]        ww_mutex_lock+0x7c/0x4d8
>>> [   68.296291]        drm_gem_shmem_free+0x7c/0x198
>>> [   68.296304]        panfrost_gem_free_object+0x118/0x138
>>> [   68.296318]        drm_gem_object_free+0x40/0x68
>>> [   68.296334]        drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
>>> [   68.296352]        drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
>>> [   68.296368]        do_shrink_slab+0x220/0x808
>>> [   68.296381]        shrink_slab+0x11c/0x408
>>> [   68.296392]        shrink_node+0x6ac/0xb90
>>> [   68.296403]        do_try_to_free_pages+0x1dc/0x8d0
>>> [   68.296416]        try_to_free_pages+0x1ec/0x5b0
>>> [   68.296429]        __alloc_pages_slowpath.constprop.0+0x528/0x1470
>>> [   68.296444]        __alloc_pages+0x4e0/0x5b8
>>> [   68.296455]        __folio_alloc+0x24/0x60
>>> [   68.296467]        vma_alloc_folio+0xb8/0x2f8
>>> [   68.296483]        alloc_zeroed_user_highpage_movable+0x58/0x68
>>> [   68.296498]        __handle_mm_fault+0x918/0x12a8
>>> [   68.296513]        handle_mm_fault+0x130/0x300
>>> [   68.296527]        do_page_fault+0x1d0/0x568
>>> [   68.296539]        do_translation_fault+0xa0/0xb8
>>> [   68.296551]        do_mem_abort+0x68/0xf8
>>> [   68.296562]        el0_da+0x74/0x100
>>> [   68.296572]        el0t_64_sync_handler+0x68/0xc0
>>> [   68.296585]        el0t_64_sync+0x18c/0x190
>>> [   68.296596]
>>> [   68.296596] other info that might help us debug this:
>>> [   68.296596]
>>> [   68.296601]  Possible unsafe locking scenario:
>>> [   68.296601]
>>> [   68.296604]        CPU0                    CPU1
>>> [   68.296608]        ----                    ----
>>> [   68.296612]   lock(fs_reclaim);
>>> [   68.296622] lock(reservation_ww_class_mutex);
>>> [   68.296633]                                lock(fs_reclaim);
>>> [   68.296644]   lock(reservation_ww_class_mutex);
>>> [   68.296654]
>>> [   68.296654]  *** DEADLOCK ***
>>
>> This splat could be ignored for now. I'm aware about it, although
>> haven't looked closely at how to fix it since it's a kind of a lockdep
>> misreporting.
> 
> The lockdep splat could be fixed with something similar to what I've
> done in msm, ie. basically just not acquire the lock in the finalizer:
> 
> https://patchwork.freedesktop.org/patch/489364/
> 
> There is one gotcha to watch for, as danvet pointed out
> (scan_objects() could still see the obj in the LRU before the
> finalizer removes it), but if scan_objects() does the
> kref_get_unless_zero() trick, it is safe.

Nice, thank you!

-- 
Best regards,
Dmitry

Re: [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Dmitry Osipenko]

On 6/28/22 19:48, Rob Clark wrote:
> On Tue, Jun 28, 2022 at 5:51 AM Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
>>
>> On 6/28/22 15:31, Robin Murphy wrote:
>>> ----->8-----
>>> [   68.295951] ======================================================
>>> [   68.295956] WARNING: possible circular locking dependency detected
>>> [   68.295963] 5.19.0-rc3+ #400 Not tainted
>>> [   68.295972] ------------------------------------------------------
>>> [   68.295977] cc1/295 is trying to acquire lock:
>>> [   68.295986] ffff000008d7f1a0
>>> (reservation_ww_class_mutex){+.+.}-{3:3}, at: drm_gem_shmem_free+0x7c/0x198
>>> [   68.296036]
>>> [   68.296036] but task is already holding lock:
>>> [   68.296041] ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at:
>>> __alloc_pages_slowpath.constprop.0+0x4d8/0x1470
>>> [   68.296080]
>>> [   68.296080] which lock already depends on the new lock.
>>> [   68.296080]
>>> [   68.296085]
>>> [   68.296085] the existing dependency chain (in reverse order) is:
>>> [   68.296090]
>>> [   68.296090] -> #1 (fs_reclaim){+.+.}-{0:0}:
>>> [   68.296111]        fs_reclaim_acquire+0xb8/0x150
>>> [   68.296130]        dma_resv_lockdep+0x298/0x3fc
>>> [   68.296148]        do_one_initcall+0xe4/0x5f8
>>> [   68.296163]        kernel_init_freeable+0x414/0x49c
>>> [   68.296180]        kernel_init+0x2c/0x148
>>> [   68.296195]        ret_from_fork+0x10/0x20
>>> [   68.296207]
>>> [   68.296207] -> #0 (reservation_ww_class_mutex){+.+.}-{3:3}:
>>> [   68.296229]        __lock_acquire+0x1724/0x2398
>>> [   68.296246]        lock_acquire+0x218/0x5b0
>>> [   68.296260]        __ww_mutex_lock.constprop.0+0x158/0x2378
>>> [   68.296277]        ww_mutex_lock+0x7c/0x4d8
>>> [   68.296291]        drm_gem_shmem_free+0x7c/0x198
>>> [   68.296304]        panfrost_gem_free_object+0x118/0x138
>>> [   68.296318]        drm_gem_object_free+0x40/0x68
>>> [   68.296334]        drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
>>> [   68.296352]        drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
>>> [   68.296368]        do_shrink_slab+0x220/0x808
>>> [   68.296381]        shrink_slab+0x11c/0x408
>>> [   68.296392]        shrink_node+0x6ac/0xb90
>>> [   68.296403]        do_try_to_free_pages+0x1dc/0x8d0
>>> [   68.296416]        try_to_free_pages+0x1ec/0x5b0
>>> [   68.296429]        __alloc_pages_slowpath.constprop.0+0x528/0x1470
>>> [   68.296444]        __alloc_pages+0x4e0/0x5b8
>>> [   68.296455]        __folio_alloc+0x24/0x60
>>> [   68.296467]        vma_alloc_folio+0xb8/0x2f8
>>> [   68.296483]        alloc_zeroed_user_highpage_movable+0x58/0x68
>>> [   68.296498]        __handle_mm_fault+0x918/0x12a8
>>> [   68.296513]        handle_mm_fault+0x130/0x300
>>> [   68.296527]        do_page_fault+0x1d0/0x568
>>> [   68.296539]        do_translation_fault+0xa0/0xb8
>>> [   68.296551]        do_mem_abort+0x68/0xf8
>>> [   68.296562]        el0_da+0x74/0x100
>>> [   68.296572]        el0t_64_sync_handler+0x68/0xc0
>>> [   68.296585]        el0t_64_sync+0x18c/0x190
>>> [   68.296596]
>>> [   68.296596] other info that might help us debug this:
>>> [   68.296596]
>>> [   68.296601]  Possible unsafe locking scenario:
>>> [   68.296601]
>>> [   68.296604]        CPU0                    CPU1
>>> [   68.296608]        ----                    ----
>>> [   68.296612]   lock(fs_reclaim);
>>> [   68.296622] lock(reservation_ww_class_mutex);
>>> [   68.296633]                                lock(fs_reclaim);
>>> [   68.296644]   lock(reservation_ww_class_mutex);
>>> [   68.296654]
>>> [   68.296654]  *** DEADLOCK ***
>>
>> This splat could be ignored for now. I'm aware about it, although
>> haven't looked closely at how to fix it since it's a kind of a lockdep
>> misreporting.
> 
> The lockdep splat could be fixed with something similar to what I've
> done in msm, ie. basically just not acquire the lock in the finalizer:
> 
> https://patchwork.freedesktop.org/patch/489364/
> 
> There is one gotcha to watch for, as danvet pointed out
> (scan_objects() could still see the obj in the LRU before the
> finalizer removes it), but if scan_objects() does the
> kref_get_unless_zero() trick, it is safe.

Nice, thank you!

-- 
Best regards,
Dmitry

Re: [Intel-gfx] [PATCH v6 00/22] Add generic memory shrinker to VirtIO-GPU and Panfrost DRM drivers

[Dmitry Osipenko]

On 6/28/22 19:48, Rob Clark wrote:
> On Tue, Jun 28, 2022 at 5:51 AM Dmitry Osipenko
> <dmitry.osipenko@collabora.com> wrote:
>>
>> On 6/28/22 15:31, Robin Murphy wrote:
>>> ----->8-----
>>> [   68.295951] ======================================================
>>> [   68.295956] WARNING: possible circular locking dependency detected
>>> [   68.295963] 5.19.0-rc3+ #400 Not tainted
>>> [   68.295972] ------------------------------------------------------
>>> [   68.295977] cc1/295 is trying to acquire lock:
>>> [   68.295986] ffff000008d7f1a0
>>> (reservation_ww_class_mutex){+.+.}-{3:3}, at: drm_gem_shmem_free+0x7c/0x198
>>> [   68.296036]
>>> [   68.296036] but task is already holding lock:
>>> [   68.296041] ffff80000c14b820 (fs_reclaim){+.+.}-{0:0}, at:
>>> __alloc_pages_slowpath.constprop.0+0x4d8/0x1470
>>> [   68.296080]
>>> [   68.296080] which lock already depends on the new lock.
>>> [   68.296080]
>>> [   68.296085]
>>> [   68.296085] the existing dependency chain (in reverse order) is:
>>> [   68.296090]
>>> [   68.296090] -> #1 (fs_reclaim){+.+.}-{0:0}:
>>> [   68.296111]        fs_reclaim_acquire+0xb8/0x150
>>> [   68.296130]        dma_resv_lockdep+0x298/0x3fc
>>> [   68.296148]        do_one_initcall+0xe4/0x5f8
>>> [   68.296163]        kernel_init_freeable+0x414/0x49c
>>> [   68.296180]        kernel_init+0x2c/0x148
>>> [   68.296195]        ret_from_fork+0x10/0x20
>>> [   68.296207]
>>> [   68.296207] -> #0 (reservation_ww_class_mutex){+.+.}-{3:3}:
>>> [   68.296229]        __lock_acquire+0x1724/0x2398
>>> [   68.296246]        lock_acquire+0x218/0x5b0
>>> [   68.296260]        __ww_mutex_lock.constprop.0+0x158/0x2378
>>> [   68.296277]        ww_mutex_lock+0x7c/0x4d8
>>> [   68.296291]        drm_gem_shmem_free+0x7c/0x198
>>> [   68.296304]        panfrost_gem_free_object+0x118/0x138
>>> [   68.296318]        drm_gem_object_free+0x40/0x68
>>> [   68.296334]        drm_gem_shmem_shrinker_run_objects_scan+0x42c/0x5b8
>>> [   68.296352]        drm_gem_shmem_shrinker_scan_objects+0xa4/0x170
>>> [   68.296368]        do_shrink_slab+0x220/0x808
>>> [   68.296381]        shrink_slab+0x11c/0x408
>>> [   68.296392]        shrink_node+0x6ac/0xb90
>>> [   68.296403]        do_try_to_free_pages+0x1dc/0x8d0
>>> [   68.296416]        try_to_free_pages+0x1ec/0x5b0
>>> [   68.296429]        __alloc_pages_slowpath.constprop.0+0x528/0x1470
>>> [   68.296444]        __alloc_pages+0x4e0/0x5b8
>>> [   68.296455]        __folio_alloc+0x24/0x60
>>> [   68.296467]        vma_alloc_folio+0xb8/0x2f8
>>> [   68.296483]        alloc_zeroed_user_highpage_movable+0x58/0x68
>>> [   68.296498]        __handle_mm_fault+0x918/0x12a8
>>> [   68.296513]        handle_mm_fault+0x130/0x300
>>> [   68.296527]        do_page_fault+0x1d0/0x568
>>> [   68.296539]        do_translation_fault+0xa0/0xb8
>>> [   68.296551]        do_mem_abort+0x68/0xf8
>>> [   68.296562]        el0_da+0x74/0x100
>>> [   68.296572]        el0t_64_sync_handler+0x68/0xc0
>>> [   68.296585]        el0t_64_sync+0x18c/0x190
>>> [   68.296596]
>>> [   68.296596] other info that might help us debug this:
>>> [   68.296596]
>>> [   68.296601]  Possible unsafe locking scenario:
>>> [   68.296601]
>>> [   68.296604]        CPU0                    CPU1
>>> [   68.296608]        ----                    ----
>>> [   68.296612]   lock(fs_reclaim);
>>> [   68.296622] lock(reservation_ww_class_mutex);
>>> [   68.296633]                                lock(fs_reclaim);
>>> [   68.296644]   lock(reservation_ww_class_mutex);
>>> [   68.296654]
>>> [   68.296654]  *** DEADLOCK ***
>>
>> This splat could be ignored for now. I'm aware about it, although
>> haven't looked closely at how to fix it since it's a kind of a lockdep
>> misreporting.
> 
> The lockdep splat could be fixed with something similar to what I've
> done in msm, ie. basically just not acquire the lock in the finalizer:
> 
> https://patchwork.freedesktop.org/patch/489364/
> 
> There is one gotcha to watch for, as danvet pointed out
> (scan_objects() could still see the obj in the LRU before the
> finalizer removes it), but if scan_objects() does the
> kref_get_unless_zero() trick, it is safe.

Nice, thank you!

-- 
Best regards,
Dmitry

Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Thomas Hellström (Intel)]

On 5/27/22 01:50, Dmitry Osipenko wrote:
> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
> handle imported dma-bufs properly, which results in mapping of something
> else than the imported dma-buf. For example, on NVIDIA Tegra we get a hard
> lockup when userspace writes to the memory mapping of a dma-buf that was
> imported into Tegra's DRM GEM.
>
> To fix this bug, move mapping of imported dma-bufs to drm_gem_mmap_obj().
> Now mmaping of imported dma-bufs works properly for all DRM drivers.
Same comment about Fixes: as in patch 1,
>
> Cc: stable@vger.kernel.org
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>   drivers/gpu/drm/drm_gem.c              | 3 +++
>   drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>   drivers/gpu/drm/tegra/gem.c            | 4 ++++
>   3 files changed, 7 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
> index 86d670c71286..7c0b025508e4 100644
> --- a/drivers/gpu/drm/drm_gem.c
> +++ b/drivers/gpu/drm/drm_gem.c
> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,
>   	if (obj_size < vma->vm_end - vma->vm_start)
>   		return -EINVAL;
>   
> +	if (obj->import_attach)
> +		return dma_buf_mmap(obj->dma_buf, vma, 0);

If we start enabling mmaping of imported dma-bufs on a majority of 
drivers in this way, how do we ensure that user-space is not blindly 
using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC 
which is needed before and after cpu access of mmap'ed dma-bufs?

I was under the impression (admittedly without looking) that the few 
drivers that actually called into dma_buf_mmap() had some private 
user-mode driver code in place that ensured this happened.

/Thomas


> +
>   	/* Take a ref for this mapping of the object, so that the fault
>   	 * handler can dereference the mmap offset's pointer to the object.
>   	 * This reference is cleaned up by the corresponding vm_close
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 8ad0e02991ca..6190f5018986 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -609,17 +609,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_vm_ops);
>    */
>   int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct *vma)
>   {
> -	struct drm_gem_object *obj = &shmem->base;
>   	int ret;
>   
> -	if (obj->import_attach) {
> -		/* Drop the reference drm_gem_mmap_obj() acquired.*/
> -		drm_gem_object_put(obj);
> -		vma->vm_private_data = NULL;
> -
> -		return dma_buf_mmap(obj->dma_buf, vma, 0);
> -	}
> -
>   	ret = drm_gem_shmem_get_pages(shmem);
>   	if (ret) {
>   		drm_gem_vm_close(vma);
> diff --git a/drivers/gpu/drm/tegra/gem.c b/drivers/gpu/drm/tegra/gem.c
> index 7c7dd84e6db8..f92aa20d63bb 100644
> --- a/drivers/gpu/drm/tegra/gem.c
> +++ b/drivers/gpu/drm/tegra/gem.c
> @@ -564,6 +564,10 @@ int __tegra_gem_mmap(struct drm_gem_object *gem, struct vm_area_struct *vma)
>   {
>   	struct tegra_bo *bo = to_tegra_bo(gem);
>   
> +	/* imported dmu-buf is mapped by drm_gem_mmap_obj()  */
> +	if (gem->import_attach)
> +		return 0;
> +
>   	if (!bo->pages) {
>   		unsigned long vm_pgoff = vma->vm_pgoff;
>   		int err;

Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Thomas Hellström (Intel)]

On 5/27/22 01:50, Dmitry Osipenko wrote:
> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
> handle imported dma-bufs properly, which results in mapping of something
> else than the imported dma-buf. For example, on NVIDIA Tegra we get a hard
> lockup when userspace writes to the memory mapping of a dma-buf that was
> imported into Tegra's DRM GEM.
>
> To fix this bug, move mapping of imported dma-bufs to drm_gem_mmap_obj().
> Now mmaping of imported dma-bufs works properly for all DRM drivers.
Same comment about Fixes: as in patch 1,
>
> Cc: stable@vger.kernel.org
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>   drivers/gpu/drm/drm_gem.c              | 3 +++
>   drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>   drivers/gpu/drm/tegra/gem.c            | 4 ++++
>   3 files changed, 7 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
> index 86d670c71286..7c0b025508e4 100644
> --- a/drivers/gpu/drm/drm_gem.c
> +++ b/drivers/gpu/drm/drm_gem.c
> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,
>   	if (obj_size < vma->vm_end - vma->vm_start)
>   		return -EINVAL;
>   
> +	if (obj->import_attach)
> +		return dma_buf_mmap(obj->dma_buf, vma, 0);

If we start enabling mmaping of imported dma-bufs on a majority of 
drivers in this way, how do we ensure that user-space is not blindly 
using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC 
which is needed before and after cpu access of mmap'ed dma-bufs?

I was under the impression (admittedly without looking) that the few 
drivers that actually called into dma_buf_mmap() had some private 
user-mode driver code in place that ensured this happened.

/Thomas


> +
>   	/* Take a ref for this mapping of the object, so that the fault
>   	 * handler can dereference the mmap offset's pointer to the object.
>   	 * This reference is cleaned up by the corresponding vm_close
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 8ad0e02991ca..6190f5018986 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -609,17 +609,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_vm_ops);
>    */
>   int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct *vma)
>   {
> -	struct drm_gem_object *obj = &shmem->base;
>   	int ret;
>   
> -	if (obj->import_attach) {
> -		/* Drop the reference drm_gem_mmap_obj() acquired.*/
> -		drm_gem_object_put(obj);
> -		vma->vm_private_data = NULL;
> -
> -		return dma_buf_mmap(obj->dma_buf, vma, 0);
> -	}
> -
>   	ret = drm_gem_shmem_get_pages(shmem);
>   	if (ret) {
>   		drm_gem_vm_close(vma);
> diff --git a/drivers/gpu/drm/tegra/gem.c b/drivers/gpu/drm/tegra/gem.c
> index 7c7dd84e6db8..f92aa20d63bb 100644
> --- a/drivers/gpu/drm/tegra/gem.c
> +++ b/drivers/gpu/drm/tegra/gem.c
> @@ -564,6 +564,10 @@ int __tegra_gem_mmap(struct drm_gem_object *gem, struct vm_area_struct *vma)
>   {
>   	struct tegra_bo *bo = to_tegra_bo(gem);
>   
> +	/* imported dmu-buf is mapped by drm_gem_mmap_obj()  */
> +	if (gem->import_attach)
> +		return 0;
> +
>   	if (!bo->pages) {
>   		unsigned long vm_pgoff = vma->vm_pgoff;
>   		int err;

Re: [Intel-gfx] [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Thomas Hellström (Intel)]

On 5/27/22 01:50, Dmitry Osipenko wrote:
> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
> handle imported dma-bufs properly, which results in mapping of something
> else than the imported dma-buf. For example, on NVIDIA Tegra we get a hard
> lockup when userspace writes to the memory mapping of a dma-buf that was
> imported into Tegra's DRM GEM.
>
> To fix this bug, move mapping of imported dma-bufs to drm_gem_mmap_obj().
> Now mmaping of imported dma-bufs works properly for all DRM drivers.
Same comment about Fixes: as in patch 1,
>
> Cc: stable@vger.kernel.org
> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>   drivers/gpu/drm/drm_gem.c              | 3 +++
>   drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>   drivers/gpu/drm/tegra/gem.c            | 4 ++++
>   3 files changed, 7 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
> index 86d670c71286..7c0b025508e4 100644
> --- a/drivers/gpu/drm/drm_gem.c
> +++ b/drivers/gpu/drm/drm_gem.c
> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,
>   	if (obj_size < vma->vm_end - vma->vm_start)
>   		return -EINVAL;
>   
> +	if (obj->import_attach)
> +		return dma_buf_mmap(obj->dma_buf, vma, 0);

If we start enabling mmaping of imported dma-bufs on a majority of 
drivers in this way, how do we ensure that user-space is not blindly 
using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC 
which is needed before and after cpu access of mmap'ed dma-bufs?

I was under the impression (admittedly without looking) that the few 
drivers that actually called into dma_buf_mmap() had some private 
user-mode driver code in place that ensured this happened.

/Thomas


> +
>   	/* Take a ref for this mapping of the object, so that the fault
>   	 * handler can dereference the mmap offset's pointer to the object.
>   	 * This reference is cleaned up by the corresponding vm_close
> diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
> index 8ad0e02991ca..6190f5018986 100644
> --- a/drivers/gpu/drm/drm_gem_shmem_helper.c
> +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
> @@ -609,17 +609,8 @@ EXPORT_SYMBOL_GPL(drm_gem_shmem_vm_ops);
>    */
>   int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct *vma)
>   {
> -	struct drm_gem_object *obj = &shmem->base;
>   	int ret;
>   
> -	if (obj->import_attach) {
> -		/* Drop the reference drm_gem_mmap_obj() acquired.*/
> -		drm_gem_object_put(obj);
> -		vma->vm_private_data = NULL;
> -
> -		return dma_buf_mmap(obj->dma_buf, vma, 0);
> -	}
> -
>   	ret = drm_gem_shmem_get_pages(shmem);
>   	if (ret) {
>   		drm_gem_vm_close(vma);
> diff --git a/drivers/gpu/drm/tegra/gem.c b/drivers/gpu/drm/tegra/gem.c
> index 7c7dd84e6db8..f92aa20d63bb 100644
> --- a/drivers/gpu/drm/tegra/gem.c
> +++ b/drivers/gpu/drm/tegra/gem.c
> @@ -564,6 +564,10 @@ int __tegra_gem_mmap(struct drm_gem_object *gem, struct vm_area_struct *vma)
>   {
>   	struct tegra_bo *bo = to_tegra_bo(gem);
>   
> +	/* imported dmu-buf is mapped by drm_gem_mmap_obj()  */
> +	if (gem->import_attach)
> +		return 0;
> +
>   	if (!bo->pages) {
>   		unsigned long vm_pgoff = vma->vm_pgoff;
>   		int err;

Re: [PATCH v6 08/22] drm/virtio: Unlock reservations on dma_resv_reserve_fences() error

[Thomas Hellström (Intel)]

On 5/27/22 01:50, Dmitry Osipenko wrote:
> Unlock reservations on dma_resv_reserve_fences() error to fix recursive
> locking of the reservations when this error happens.
Fixes:
> Cc: stable@vger.kernel.org

With that fixed,

Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>


> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>   drivers/gpu/drm/virtio/virtgpu_gem.c | 4 +++-
>   1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_gem.c b/drivers/gpu/drm/virtio/virtgpu_gem.c
> index 580a78809836..7db48d17ee3a 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_gem.c
> +++ b/drivers/gpu/drm/virtio/virtgpu_gem.c
> @@ -228,8 +228,10 @@ int virtio_gpu_array_lock_resv(struct virtio_gpu_object_array *objs)
>   
>   	for (i = 0; i < objs->nents; ++i) {
>   		ret = dma_resv_reserve_fences(objs->objs[i]->resv, 1);
> -		if (ret)
> +		if (ret) {
> +			virtio_gpu_array_unlock_resv(objs);
>   			return ret;
> +		}
>   	}
>   	return ret;
>   }

Re: [PATCH v6 08/22] drm/virtio: Unlock reservations on dma_resv_reserve_fences() error

[Thomas Hellström (Intel)]

On 5/27/22 01:50, Dmitry Osipenko wrote:
> Unlock reservations on dma_resv_reserve_fences() error to fix recursive
> locking of the reservations when this error happens.
Fixes:
> Cc: stable@vger.kernel.org

With that fixed,

Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>


> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>   drivers/gpu/drm/virtio/virtgpu_gem.c | 4 +++-
>   1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_gem.c b/drivers/gpu/drm/virtio/virtgpu_gem.c
> index 580a78809836..7db48d17ee3a 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_gem.c
> +++ b/drivers/gpu/drm/virtio/virtgpu_gem.c
> @@ -228,8 +228,10 @@ int virtio_gpu_array_lock_resv(struct virtio_gpu_object_array *objs)
>   
>   	for (i = 0; i < objs->nents; ++i) {
>   		ret = dma_resv_reserve_fences(objs->objs[i]->resv, 1);
> -		if (ret)
> +		if (ret) {
> +			virtio_gpu_array_unlock_resv(objs);
>   			return ret;
> +		}
>   	}
>   	return ret;
>   }

Re: [Intel-gfx] [PATCH v6 08/22] drm/virtio: Unlock reservations on dma_resv_reserve_fences() error

[Thomas Hellström (Intel)]

On 5/27/22 01:50, Dmitry Osipenko wrote:
> Unlock reservations on dma_resv_reserve_fences() error to fix recursive
> locking of the reservations when this error happens.
Fixes:
> Cc: stable@vger.kernel.org

With that fixed,

Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>


> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> ---
>   drivers/gpu/drm/virtio/virtgpu_gem.c | 4 +++-
>   1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/virtio/virtgpu_gem.c b/drivers/gpu/drm/virtio/virtgpu_gem.c
> index 580a78809836..7db48d17ee3a 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_gem.c
> +++ b/drivers/gpu/drm/virtio/virtgpu_gem.c
> @@ -228,8 +228,10 @@ int virtio_gpu_array_lock_resv(struct virtio_gpu_object_array *objs)
>   
>   	for (i = 0; i < objs->nents; ++i) {
>   		ret = dma_resv_reserve_fences(objs->objs[i]->resv, 1);
> -		if (ret)
> +		if (ret) {
> +			virtio_gpu_array_unlock_resv(objs);
>   			return ret;
> +		}
>   	}
>   	return ret;
>   }

Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Dmitry Osipenko]

On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
> 
> On 5/27/22 01:50, Dmitry Osipenko wrote:
>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>> handle imported dma-bufs properly, which results in mapping of something
>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>> hard
>> lockup when userspace writes to the memory mapping of a dma-buf that was
>> imported into Tegra's DRM GEM.
>>
>> To fix this bug, move mapping of imported dma-bufs to drm_gem_mmap_obj().
>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
> Same comment about Fixes: as in patch 1,
>>
>> Cc: stable@vger.kernel.org
>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>> ---
>>   drivers/gpu/drm/drm_gem.c              | 3 +++
>>   drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>   drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>   3 files changed, 7 insertions(+), 9 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>> index 86d670c71286..7c0b025508e4 100644
>> --- a/drivers/gpu/drm/drm_gem.c
>> +++ b/drivers/gpu/drm/drm_gem.c
>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj,
>> unsigned long obj_size,
>>       if (obj_size < vma->vm_end - vma->vm_start)
>>           return -EINVAL;
>>   +    if (obj->import_attach)
>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
> 
> If we start enabling mmaping of imported dma-bufs on a majority of
> drivers in this way, how do we ensure that user-space is not blindly
> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
> which is needed before and after cpu access of mmap'ed dma-bufs?
> 
> I was under the impression (admittedly without looking) that the few
> drivers that actually called into dma_buf_mmap() had some private
> user-mode driver code in place that ensured this happened.

Since it's a userspace who does the mapping, then it should be a
responsibility of userspace to do all the necessary syncing. I'm not
sure whether anyone in userspace really needs to map imported dma-bufs
in practice. Nevertheless, this use-case is broken and should be fixed
by either allowing to do the mapping or prohibiting it.

-- 
Best regards,
Dmitry

Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Dmitry Osipenko]

On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
> 
> On 5/27/22 01:50, Dmitry Osipenko wrote:
>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>> handle imported dma-bufs properly, which results in mapping of something
>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>> hard
>> lockup when userspace writes to the memory mapping of a dma-buf that was
>> imported into Tegra's DRM GEM.
>>
>> To fix this bug, move mapping of imported dma-bufs to drm_gem_mmap_obj().
>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
> Same comment about Fixes: as in patch 1,
>>
>> Cc: stable@vger.kernel.org
>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>> ---
>>   drivers/gpu/drm/drm_gem.c              | 3 +++
>>   drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>   drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>   3 files changed, 7 insertions(+), 9 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>> index 86d670c71286..7c0b025508e4 100644
>> --- a/drivers/gpu/drm/drm_gem.c
>> +++ b/drivers/gpu/drm/drm_gem.c
>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj,
>> unsigned long obj_size,
>>       if (obj_size < vma->vm_end - vma->vm_start)
>>           return -EINVAL;
>>   +    if (obj->import_attach)
>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
> 
> If we start enabling mmaping of imported dma-bufs on a majority of
> drivers in this way, how do we ensure that user-space is not blindly
> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
> which is needed before and after cpu access of mmap'ed dma-bufs?
> 
> I was under the impression (admittedly without looking) that the few
> drivers that actually called into dma_buf_mmap() had some private
> user-mode driver code in place that ensured this happened.

Since it's a userspace who does the mapping, then it should be a
responsibility of userspace to do all the necessary syncing. I'm not
sure whether anyone in userspace really needs to map imported dma-bufs
in practice. Nevertheless, this use-case is broken and should be fixed
by either allowing to do the mapping or prohibiting it.

-- 
Best regards,
Dmitry

Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Dmitry Osipenko]

On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
> 
> On 5/27/22 01:50, Dmitry Osipenko wrote:
>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>> handle imported dma-bufs properly, which results in mapping of something
>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>> hard
>> lockup when userspace writes to the memory mapping of a dma-buf that was
>> imported into Tegra's DRM GEM.
>>
>> To fix this bug, move mapping of imported dma-bufs to drm_gem_mmap_obj().
>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
> Same comment about Fixes: as in patch 1,
>>
>> Cc: stable@vger.kernel.org
>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>> ---
>>   drivers/gpu/drm/drm_gem.c              | 3 +++
>>   drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>   drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>   3 files changed, 7 insertions(+), 9 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>> index 86d670c71286..7c0b025508e4 100644
>> --- a/drivers/gpu/drm/drm_gem.c
>> +++ b/drivers/gpu/drm/drm_gem.c
>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj,
>> unsigned long obj_size,
>>       if (obj_size < vma->vm_end - vma->vm_start)
>>           return -EINVAL;
>>   +    if (obj->import_attach)
>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
> 
> If we start enabling mmaping of imported dma-bufs on a majority of
> drivers in this way, how do we ensure that user-space is not blindly
> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
> which is needed before and after cpu access of mmap'ed dma-bufs?
> 
> I was under the impression (admittedly without looking) that the few
> drivers that actually called into dma_buf_mmap() had some private
> user-mode driver code in place that ensured this happened.

Since it's a userspace who does the mapping, then it should be a
responsibility of userspace to do all the necessary syncing. I'm not
sure whether anyone in userspace really needs to map imported dma-bufs
in practice. Nevertheless, this use-case is broken and should be fixed
by either allowing to do the mapping or prohibiting it.

-- 
Best regards,
Dmitry

Re: [Intel-gfx] [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Dmitry Osipenko]

On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
> 
> On 5/27/22 01:50, Dmitry Osipenko wrote:
>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>> handle imported dma-bufs properly, which results in mapping of something
>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>> hard
>> lockup when userspace writes to the memory mapping of a dma-buf that was
>> imported into Tegra's DRM GEM.
>>
>> To fix this bug, move mapping of imported dma-bufs to drm_gem_mmap_obj().
>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
> Same comment about Fixes: as in patch 1,
>>
>> Cc: stable@vger.kernel.org
>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>> ---
>>   drivers/gpu/drm/drm_gem.c              | 3 +++
>>   drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>   drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>   3 files changed, 7 insertions(+), 9 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>> index 86d670c71286..7c0b025508e4 100644
>> --- a/drivers/gpu/drm/drm_gem.c
>> +++ b/drivers/gpu/drm/drm_gem.c
>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj,
>> unsigned long obj_size,
>>       if (obj_size < vma->vm_end - vma->vm_start)
>>           return -EINVAL;
>>   +    if (obj->import_attach)
>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
> 
> If we start enabling mmaping of imported dma-bufs on a majority of
> drivers in this way, how do we ensure that user-space is not blindly
> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
> which is needed before and after cpu access of mmap'ed dma-bufs?
> 
> I was under the impression (admittedly without looking) that the few
> drivers that actually called into dma_buf_mmap() had some private
> user-mode driver code in place that ensured this happened.

Since it's a userspace who does the mapping, then it should be a
responsibility of userspace to do all the necessary syncing. I'm not
sure whether anyone in userspace really needs to map imported dma-bufs
in practice. Nevertheless, this use-case is broken and should be fixed
by either allowing to do the mapping or prohibiting it.

-- 
Best regards,
Dmitry

Re: [PATCH v6 01/22] drm/gem: Properly annotate WW context on drm_gem_lock_reservations() error

[Dmitry Osipenko]

On 6/28/22 23:12, Thomas Hellström (Intel) wrote:
> Hi,
> 
> On 5/27/22 01:50, Dmitry Osipenko wrote:
>> Use ww_acquire_fini() in the error code paths. Otherwise lockdep
>> thinks that lock is held when lock's memory is freed after the
>> drm_gem_lock_reservations() error. The WW needs to be annotated
>> as "freed"
> 
> s /WW/ww_acquire_context/ ?
> s /"freed"/"released"/ ?
> 
> 
>> , which fixes the noisy "WARNING: held lock freed!" splat
>> of VirtIO-GPU driver with CONFIG_DEBUG_MUTEXES=y and enabled lockdep.
>>
>> Cc: stable@vger.kernel.org
> 
> Can you dig up the commit in error and add a Fixes: Tag?
> 
> Using that and "dim fixes" will also make the Cc: stable tag a bit more
> verbose.
> 
> With that fixed,
> 
> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>

I'll update this patch, thank you for taking a look.

-- 
Best regards,
Dmitry

Re: [PATCH v6 01/22] drm/gem: Properly annotate WW context on drm_gem_lock_reservations() error

[Dmitry Osipenko]

On 6/28/22 23:12, Thomas Hellström (Intel) wrote:
> Hi,
> 
> On 5/27/22 01:50, Dmitry Osipenko wrote:
>> Use ww_acquire_fini() in the error code paths. Otherwise lockdep
>> thinks that lock is held when lock's memory is freed after the
>> drm_gem_lock_reservations() error. The WW needs to be annotated
>> as "freed"
> 
> s /WW/ww_acquire_context/ ?
> s /"freed"/"released"/ ?
> 
> 
>> , which fixes the noisy "WARNING: held lock freed!" splat
>> of VirtIO-GPU driver with CONFIG_DEBUG_MUTEXES=y and enabled lockdep.
>>
>> Cc: stable@vger.kernel.org
> 
> Can you dig up the commit in error and add a Fixes: Tag?
> 
> Using that and "dim fixes" will also make the Cc: stable tag a bit more
> verbose.
> 
> With that fixed,
> 
> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>

I'll update this patch, thank you for taking a look.

-- 
Best regards,
Dmitry

Re: [Intel-gfx] [PATCH v6 01/22] drm/gem: Properly annotate WW context on drm_gem_lock_reservations() error

[Dmitry Osipenko]

On 6/28/22 23:12, Thomas Hellström (Intel) wrote:
> Hi,
> 
> On 5/27/22 01:50, Dmitry Osipenko wrote:
>> Use ww_acquire_fini() in the error code paths. Otherwise lockdep
>> thinks that lock is held when lock's memory is freed after the
>> drm_gem_lock_reservations() error. The WW needs to be annotated
>> as "freed"
> 
> s /WW/ww_acquire_context/ ?
> s /"freed"/"released"/ ?
> 
> 
>> , which fixes the noisy "WARNING: held lock freed!" splat
>> of VirtIO-GPU driver with CONFIG_DEBUG_MUTEXES=y and enabled lockdep.
>>
>> Cc: stable@vger.kernel.org
> 
> Can you dig up the commit in error and add a Fixes: Tag?
> 
> Using that and "dim fixes" will also make the Cc: stable tag a bit more
> verbose.
> 
> With that fixed,
> 
> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>

I'll update this patch, thank you for taking a look.

-- 
Best regards,
Dmitry

Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Thomas Hellström (Intel)]

On 6/29/22 10:22, Dmitry Osipenko wrote:
> On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
>> On 5/27/22 01:50, Dmitry Osipenko wrote:
>>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>>> handle imported dma-bufs properly, which results in mapping of something
>>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>>> hard
>>> lockup when userspace writes to the memory mapping of a dma-buf that was
>>> imported into Tegra's DRM GEM.
>>>
>>> To fix this bug, move mapping of imported dma-bufs to drm_gem_mmap_obj().
>>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
>> Same comment about Fixes: as in patch 1,
>>> Cc: stable@vger.kernel.org
>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>> ---
>>>    drivers/gpu/drm/drm_gem.c              | 3 +++
>>>    drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>>    drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>>    3 files changed, 7 insertions(+), 9 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>>> index 86d670c71286..7c0b025508e4 100644
>>> --- a/drivers/gpu/drm/drm_gem.c
>>> +++ b/drivers/gpu/drm/drm_gem.c
>>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj,
>>> unsigned long obj_size,
>>>        if (obj_size < vma->vm_end - vma->vm_start)
>>>            return -EINVAL;
>>>    +    if (obj->import_attach)
>>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
>> If we start enabling mmaping of imported dma-bufs on a majority of
>> drivers in this way, how do we ensure that user-space is not blindly
>> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
>> which is needed before and after cpu access of mmap'ed dma-bufs?
>>
>> I was under the impression (admittedly without looking) that the few
>> drivers that actually called into dma_buf_mmap() had some private
>> user-mode driver code in place that ensured this happened.
> Since it's a userspace who does the mapping, then it should be a
> responsibility of userspace to do all the necessary syncing.

Sure, but nothing prohibits user-space to ignore the syncing thinking 
"It works anyway", testing those drivers where the syncing is a NOP. And 
when a driver that finally needs syncing is tested it's too late to fix 
all broken user-space.

>   I'm not
> sure whether anyone in userspace really needs to map imported dma-bufs
> in practice. Nevertheless, this use-case is broken and should be fixed
> by either allowing to do the mapping or prohibiting it.
>
Then I'd vote for prohibiting it, at least for now. And for the future 
moving forward we could perhaps revisit the dma-buf need for syncing, 
requiring those drivers that actually need it to implement emulated 
coherent memory which can be done not too inefficiently (vmwgfx being 
one example).

/Thomas

Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Thomas Hellström (Intel)]

On 6/29/22 10:22, Dmitry Osipenko wrote:
> On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
>> On 5/27/22 01:50, Dmitry Osipenko wrote:
>>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>>> handle imported dma-bufs properly, which results in mapping of something
>>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>>> hard
>>> lockup when userspace writes to the memory mapping of a dma-buf that was
>>> imported into Tegra's DRM GEM.
>>>
>>> To fix this bug, move mapping of imported dma-bufs to drm_gem_mmap_obj().
>>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
>> Same comment about Fixes: as in patch 1,
>>> Cc: stable@vger.kernel.org
>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>> ---
>>>    drivers/gpu/drm/drm_gem.c              | 3 +++
>>>    drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>>    drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>>    3 files changed, 7 insertions(+), 9 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>>> index 86d670c71286..7c0b025508e4 100644
>>> --- a/drivers/gpu/drm/drm_gem.c
>>> +++ b/drivers/gpu/drm/drm_gem.c
>>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj,
>>> unsigned long obj_size,
>>>        if (obj_size < vma->vm_end - vma->vm_start)
>>>            return -EINVAL;
>>>    +    if (obj->import_attach)
>>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
>> If we start enabling mmaping of imported dma-bufs on a majority of
>> drivers in this way, how do we ensure that user-space is not blindly
>> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
>> which is needed before and after cpu access of mmap'ed dma-bufs?
>>
>> I was under the impression (admittedly without looking) that the few
>> drivers that actually called into dma_buf_mmap() had some private
>> user-mode driver code in place that ensured this happened.
> Since it's a userspace who does the mapping, then it should be a
> responsibility of userspace to do all the necessary syncing.

Sure, but nothing prohibits user-space to ignore the syncing thinking 
"It works anyway", testing those drivers where the syncing is a NOP. And 
when a driver that finally needs syncing is tested it's too late to fix 
all broken user-space.

>   I'm not
> sure whether anyone in userspace really needs to map imported dma-bufs
> in practice. Nevertheless, this use-case is broken and should be fixed
> by either allowing to do the mapping or prohibiting it.
>
Then I'd vote for prohibiting it, at least for now. And for the future 
moving forward we could perhaps revisit the dma-buf need for syncing, 
requiring those drivers that actually need it to implement emulated 
coherent memory which can be done not too inefficiently (vmwgfx being 
one example).

/Thomas

Re: [Intel-gfx] [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Thomas Hellström (Intel)]

On 6/29/22 10:22, Dmitry Osipenko wrote:
> On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
>> On 5/27/22 01:50, Dmitry Osipenko wrote:
>>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>>> handle imported dma-bufs properly, which results in mapping of something
>>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>>> hard
>>> lockup when userspace writes to the memory mapping of a dma-buf that was
>>> imported into Tegra's DRM GEM.
>>>
>>> To fix this bug, move mapping of imported dma-bufs to drm_gem_mmap_obj().
>>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
>> Same comment about Fixes: as in patch 1,
>>> Cc: stable@vger.kernel.org
>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>> ---
>>>    drivers/gpu/drm/drm_gem.c              | 3 +++
>>>    drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>>    drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>>    3 files changed, 7 insertions(+), 9 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>>> index 86d670c71286..7c0b025508e4 100644
>>> --- a/drivers/gpu/drm/drm_gem.c
>>> +++ b/drivers/gpu/drm/drm_gem.c
>>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj,
>>> unsigned long obj_size,
>>>        if (obj_size < vma->vm_end - vma->vm_start)
>>>            return -EINVAL;
>>>    +    if (obj->import_attach)
>>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
>> If we start enabling mmaping of imported dma-bufs on a majority of
>> drivers in this way, how do we ensure that user-space is not blindly
>> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
>> which is needed before and after cpu access of mmap'ed dma-bufs?
>>
>> I was under the impression (admittedly without looking) that the few
>> drivers that actually called into dma_buf_mmap() had some private
>> user-mode driver code in place that ensured this happened.
> Since it's a userspace who does the mapping, then it should be a
> responsibility of userspace to do all the necessary syncing.

Sure, but nothing prohibits user-space to ignore the syncing thinking 
"It works anyway", testing those drivers where the syncing is a NOP. And 
when a driver that finally needs syncing is tested it's too late to fix 
all broken user-space.

>   I'm not
> sure whether anyone in userspace really needs to map imported dma-bufs
> in practice. Nevertheless, this use-case is broken and should be fixed
> by either allowing to do the mapping or prohibiting it.
>
Then I'd vote for prohibiting it, at least for now. And for the future 
moving forward we could perhaps revisit the dma-buf need for syncing, 
requiring those drivers that actually need it to implement emulated 
coherent memory which can be done not too inefficiently (vmwgfx being 
one example).

/Thomas

Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Thomas Hellström (Intel)]

On 6/29/22 10:22, Dmitry Osipenko wrote:
> On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
>> On 5/27/22 01:50, Dmitry Osipenko wrote:
>>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>>> handle imported dma-bufs properly, which results in mapping of something
>>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>>> hard
>>> lockup when userspace writes to the memory mapping of a dma-buf that was
>>> imported into Tegra's DRM GEM.
>>>
>>> To fix this bug, move mapping of imported dma-bufs to drm_gem_mmap_obj().
>>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
>> Same comment about Fixes: as in patch 1,
>>> Cc: stable@vger.kernel.org
>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>> ---
>>>    drivers/gpu/drm/drm_gem.c              | 3 +++
>>>    drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>>    drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>>    3 files changed, 7 insertions(+), 9 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>>> index 86d670c71286..7c0b025508e4 100644
>>> --- a/drivers/gpu/drm/drm_gem.c
>>> +++ b/drivers/gpu/drm/drm_gem.c
>>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj,
>>> unsigned long obj_size,
>>>        if (obj_size < vma->vm_end - vma->vm_start)
>>>            return -EINVAL;
>>>    +    if (obj->import_attach)
>>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
>> If we start enabling mmaping of imported dma-bufs on a majority of
>> drivers in this way, how do we ensure that user-space is not blindly
>> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
>> which is needed before and after cpu access of mmap'ed dma-bufs?
>>
>> I was under the impression (admittedly without looking) that the few
>> drivers that actually called into dma_buf_mmap() had some private
>> user-mode driver code in place that ensured this happened.
> Since it's a userspace who does the mapping, then it should be a
> responsibility of userspace to do all the necessary syncing.

Sure, but nothing prohibits user-space to ignore the syncing thinking 
"It works anyway", testing those drivers where the syncing is a NOP. And 
when a driver that finally needs syncing is tested it's too late to fix 
all broken user-space.

>   I'm not
> sure whether anyone in userspace really needs to map imported dma-bufs
> in practice. Nevertheless, this use-case is broken and should be fixed
> by either allowing to do the mapping or prohibiting it.
>
Then I'd vote for prohibiting it, at least for now. And for the future 
moving forward we could perhaps revisit the dma-buf need for syncing, 
requiring those drivers that actually need it to implement emulated 
coherent memory which can be done not too inefficiently (vmwgfx being 
one example).

/Thomas

Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Dmitry Osipenko]

On 6/29/22 11:43, Thomas Hellström (Intel) wrote:
> 
> On 6/29/22 10:22, Dmitry Osipenko wrote:
>> On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
>>> On 5/27/22 01:50, Dmitry Osipenko wrote:
>>>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>>>> handle imported dma-bufs properly, which results in mapping of
>>>> something
>>>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>>>> hard
>>>> lockup when userspace writes to the memory mapping of a dma-buf that
>>>> was
>>>> imported into Tegra's DRM GEM.
>>>>
>>>> To fix this bug, move mapping of imported dma-bufs to
>>>> drm_gem_mmap_obj().
>>>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
>>> Same comment about Fixes: as in patch 1,
>>>> Cc: stable@vger.kernel.org
>>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>>> ---
>>>>    drivers/gpu/drm/drm_gem.c              | 3 +++
>>>>    drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>>>    drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>>>    3 files changed, 7 insertions(+), 9 deletions(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>>>> index 86d670c71286..7c0b025508e4 100644
>>>> --- a/drivers/gpu/drm/drm_gem.c
>>>> +++ b/drivers/gpu/drm/drm_gem.c
>>>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj,
>>>> unsigned long obj_size,
>>>>        if (obj_size < vma->vm_end - vma->vm_start)
>>>>            return -EINVAL;
>>>>    +    if (obj->import_attach)
>>>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
>>> If we start enabling mmaping of imported dma-bufs on a majority of
>>> drivers in this way, how do we ensure that user-space is not blindly
>>> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
>>> which is needed before and after cpu access of mmap'ed dma-bufs?
>>>
>>> I was under the impression (admittedly without looking) that the few
>>> drivers that actually called into dma_buf_mmap() had some private
>>> user-mode driver code in place that ensured this happened.
>> Since it's a userspace who does the mapping, then it should be a
>> responsibility of userspace to do all the necessary syncing.
> 
> Sure, but nothing prohibits user-space to ignore the syncing thinking
> "It works anyway", testing those drivers where the syncing is a NOP. And
> when a driver that finally needs syncing is tested it's too late to fix
> all broken user-space.
> 
>>   I'm not
>> sure whether anyone in userspace really needs to map imported dma-bufs
>> in practice. Nevertheless, this use-case is broken and should be fixed
>> by either allowing to do the mapping or prohibiting it.
>>
> Then I'd vote for prohibiting it, at least for now. And for the future
> moving forward we could perhaps revisit the dma-buf need for syncing,
> requiring those drivers that actually need it to implement emulated
> coherent memory which can be done not too inefficiently (vmwgfx being
> one example).

Alright, I'll change it to prohibit the mapping. This indeed should be a
better option.

-- 
Best regards,
Dmitry

Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Dmitry Osipenko]

On 6/29/22 11:43, Thomas Hellström (Intel) wrote:
> 
> On 6/29/22 10:22, Dmitry Osipenko wrote:
>> On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
>>> On 5/27/22 01:50, Dmitry Osipenko wrote:
>>>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>>>> handle imported dma-bufs properly, which results in mapping of
>>>> something
>>>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>>>> hard
>>>> lockup when userspace writes to the memory mapping of a dma-buf that
>>>> was
>>>> imported into Tegra's DRM GEM.
>>>>
>>>> To fix this bug, move mapping of imported dma-bufs to
>>>> drm_gem_mmap_obj().
>>>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
>>> Same comment about Fixes: as in patch 1,
>>>> Cc: stable@vger.kernel.org
>>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>>> ---
>>>>    drivers/gpu/drm/drm_gem.c              | 3 +++
>>>>    drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>>>    drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>>>    3 files changed, 7 insertions(+), 9 deletions(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>>>> index 86d670c71286..7c0b025508e4 100644
>>>> --- a/drivers/gpu/drm/drm_gem.c
>>>> +++ b/drivers/gpu/drm/drm_gem.c
>>>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj,
>>>> unsigned long obj_size,
>>>>        if (obj_size < vma->vm_end - vma->vm_start)
>>>>            return -EINVAL;
>>>>    +    if (obj->import_attach)
>>>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
>>> If we start enabling mmaping of imported dma-bufs on a majority of
>>> drivers in this way, how do we ensure that user-space is not blindly
>>> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
>>> which is needed before and after cpu access of mmap'ed dma-bufs?
>>>
>>> I was under the impression (admittedly without looking) that the few
>>> drivers that actually called into dma_buf_mmap() had some private
>>> user-mode driver code in place that ensured this happened.
>> Since it's a userspace who does the mapping, then it should be a
>> responsibility of userspace to do all the necessary syncing.
> 
> Sure, but nothing prohibits user-space to ignore the syncing thinking
> "It works anyway", testing those drivers where the syncing is a NOP. And
> when a driver that finally needs syncing is tested it's too late to fix
> all broken user-space.
> 
>>   I'm not
>> sure whether anyone in userspace really needs to map imported dma-bufs
>> in practice. Nevertheless, this use-case is broken and should be fixed
>> by either allowing to do the mapping or prohibiting it.
>>
> Then I'd vote for prohibiting it, at least for now. And for the future
> moving forward we could perhaps revisit the dma-buf need for syncing,
> requiring those drivers that actually need it to implement emulated
> coherent memory which can be done not too inefficiently (vmwgfx being
> one example).

Alright, I'll change it to prohibit the mapping. This indeed should be a
better option.

-- 
Best regards,
Dmitry

Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Dmitry Osipenko]

On 6/29/22 11:43, Thomas Hellström (Intel) wrote:
> 
> On 6/29/22 10:22, Dmitry Osipenko wrote:
>> On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
>>> On 5/27/22 01:50, Dmitry Osipenko wrote:
>>>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>>>> handle imported dma-bufs properly, which results in mapping of
>>>> something
>>>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>>>> hard
>>>> lockup when userspace writes to the memory mapping of a dma-buf that
>>>> was
>>>> imported into Tegra's DRM GEM.
>>>>
>>>> To fix this bug, move mapping of imported dma-bufs to
>>>> drm_gem_mmap_obj().
>>>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
>>> Same comment about Fixes: as in patch 1,
>>>> Cc: stable@vger.kernel.org
>>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>>> ---
>>>>    drivers/gpu/drm/drm_gem.c              | 3 +++
>>>>    drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>>>    drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>>>    3 files changed, 7 insertions(+), 9 deletions(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>>>> index 86d670c71286..7c0b025508e4 100644
>>>> --- a/drivers/gpu/drm/drm_gem.c
>>>> +++ b/drivers/gpu/drm/drm_gem.c
>>>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj,
>>>> unsigned long obj_size,
>>>>        if (obj_size < vma->vm_end - vma->vm_start)
>>>>            return -EINVAL;
>>>>    +    if (obj->import_attach)
>>>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
>>> If we start enabling mmaping of imported dma-bufs on a majority of
>>> drivers in this way, how do we ensure that user-space is not blindly
>>> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
>>> which is needed before and after cpu access of mmap'ed dma-bufs?
>>>
>>> I was under the impression (admittedly without looking) that the few
>>> drivers that actually called into dma_buf_mmap() had some private
>>> user-mode driver code in place that ensured this happened.
>> Since it's a userspace who does the mapping, then it should be a
>> responsibility of userspace to do all the necessary syncing.
> 
> Sure, but nothing prohibits user-space to ignore the syncing thinking
> "It works anyway", testing those drivers where the syncing is a NOP. And
> when a driver that finally needs syncing is tested it's too late to fix
> all broken user-space.
> 
>>   I'm not
>> sure whether anyone in userspace really needs to map imported dma-bufs
>> in practice. Nevertheless, this use-case is broken and should be fixed
>> by either allowing to do the mapping or prohibiting it.
>>
> Then I'd vote for prohibiting it, at least for now. And for the future
> moving forward we could perhaps revisit the dma-buf need for syncing,
> requiring those drivers that actually need it to implement emulated
> coherent memory which can be done not too inefficiently (vmwgfx being
> one example).

Alright, I'll change it to prohibit the mapping. This indeed should be a
better option.

-- 
Best regards,
Dmitry

Re: [Intel-gfx] [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Dmitry Osipenko]

On 6/29/22 11:43, Thomas Hellström (Intel) wrote:
> 
> On 6/29/22 10:22, Dmitry Osipenko wrote:
>> On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
>>> On 5/27/22 01:50, Dmitry Osipenko wrote:
>>>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>>>> handle imported dma-bufs properly, which results in mapping of
>>>> something
>>>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>>>> hard
>>>> lockup when userspace writes to the memory mapping of a dma-buf that
>>>> was
>>>> imported into Tegra's DRM GEM.
>>>>
>>>> To fix this bug, move mapping of imported dma-bufs to
>>>> drm_gem_mmap_obj().
>>>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
>>> Same comment about Fixes: as in patch 1,
>>>> Cc: stable@vger.kernel.org
>>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>>> ---
>>>>    drivers/gpu/drm/drm_gem.c              | 3 +++
>>>>    drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>>>    drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>>>    3 files changed, 7 insertions(+), 9 deletions(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>>>> index 86d670c71286..7c0b025508e4 100644
>>>> --- a/drivers/gpu/drm/drm_gem.c
>>>> +++ b/drivers/gpu/drm/drm_gem.c
>>>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj,
>>>> unsigned long obj_size,
>>>>        if (obj_size < vma->vm_end - vma->vm_start)
>>>>            return -EINVAL;
>>>>    +    if (obj->import_attach)
>>>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
>>> If we start enabling mmaping of imported dma-bufs on a majority of
>>> drivers in this way, how do we ensure that user-space is not blindly
>>> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
>>> which is needed before and after cpu access of mmap'ed dma-bufs?
>>>
>>> I was under the impression (admittedly without looking) that the few
>>> drivers that actually called into dma_buf_mmap() had some private
>>> user-mode driver code in place that ensured this happened.
>> Since it's a userspace who does the mapping, then it should be a
>> responsibility of userspace to do all the necessary syncing.
> 
> Sure, but nothing prohibits user-space to ignore the syncing thinking
> "It works anyway", testing those drivers where the syncing is a NOP. And
> when a driver that finally needs syncing is tested it's too late to fix
> all broken user-space.
> 
>>   I'm not
>> sure whether anyone in userspace really needs to map imported dma-bufs
>> in practice. Nevertheless, this use-case is broken and should be fixed
>> by either allowing to do the mapping or prohibiting it.
>>
> Then I'd vote for prohibiting it, at least for now. And for the future
> moving forward we could perhaps revisit the dma-buf need for syncing,
> requiring those drivers that actually need it to implement emulated
> coherent memory which can be done not too inefficiently (vmwgfx being
> one example).

Alright, I'll change it to prohibit the mapping. This indeed should be a
better option.

-- 
Best regards,
Dmitry

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dmitry Osipenko]

On 6/29/22 00:26, Thomas Hellström (Intel) wrote:
> 
> On 5/30/22 15:57, Dmitry Osipenko wrote:
>> On 5/30/22 16:41, Christian König wrote:
>>> Hi Dmitry,
>>>
>>> Am 30.05.22 um 15:26 schrieb Dmitry Osipenko:
>>>> Hello Christian,
>>>>
>>>> On 5/30/22 09:50, Christian König wrote:
>>>>> Hi Dmitry,
>>>>>
>>>>> First of all please separate out this patch from the rest of the
>>>>> series,
>>>>> since this is a complex separate structural change.
>>>> I assume all the patches will go via the DRM tree in the end since the
>>>> rest of the DRM patches in this series depend on this dma-buf change.
>>>> But I see that separation may ease reviewing of the dma-buf changes, so
>>>> let's try it.
>>> That sounds like you are underestimating a bit how much trouble this
>>> will be.
>>>
>>>>> I have tried this before and failed because catching all the locks in
>>>>> the right code paths are very tricky. So expect some fallout from this
>>>>> and make sure the kernel test robot and CI systems are clean.
>>>> Sure, I'll fix up all the reported things in the next iteration.
>>>>
>>>> BTW, have you ever posted yours version of the patch? Will be great if
>>>> we could compare the changed code paths.
>>> No, I never even finished creating it after realizing how much work it
>>> would be.
>>>
>>>>>> This patch introduces new locking convention for dma-buf users. From
>>>>>> now
>>>>>> on all dma-buf importers are responsible for holding dma-buf
>>>>>> reservation
>>>>>> lock around operations performed over dma-bufs.
>>>>>>
>>>>>> This patch implements the new dma-buf locking convention by:
>>>>>>
>>>>>>      1. Making dma-buf API functions to take the reservation lock.
>>>>>>
>>>>>>      2. Adding new locked variants of the dma-buf API functions for
>>>>>> drivers
>>>>>>         that need to manage imported dma-bufs under the held lock.
>>>>> Instead of adding new locked variants please mark all variants which
>>>>> expect to be called without a lock with an _unlocked postfix.
>>>>>
>>>>> This should make it easier to remove those in a follow up patch set
>>>>> and
>>>>> then fully move the locking into the importer.
>>>> Do we really want to move all the locks to the importers? Seems the
>>>> majority of drivers should be happy with the dma-buf helpers handling
>>>> the locking for them.
>>> Yes, I clearly think so.
>>>
>>>>>>      3. Converting all drivers to the new locking scheme.
>>>>> I have strong doubts that you got all of them. At least radeon and
>>>>> nouveau should grab the reservation lock in their ->attach callbacks
>>>>> somehow.
>>>> Radeon and Nouveau use gem_prime_import_sg_table() and they take resv
>>>> lock already, seems they should be okay (?)
>>> You are looking at the wrong side. You need to fix the export code path,
>>> not the import ones.
>>>
>>> See for example attach on radeon works like this
>>> drm_gem_map_attach->drm_gem_pin->radeon_gem_prime_pin->radeon_bo_reserve->ttm_bo_reserve->dma_resv_lock.
>>>
>> Yeah, I was looking at the both sides, but missed this one.
> 
> Also i915 will run into trouble with attach. In particular since i915
> starts a full ww transaction in its attach callback to be able to lock
> other objects if migration is needed. I think i915 CI would catch this
> in a selftest.

Seems it indeed it should deadlock. But i915 selftests apparently
should've caught it and they didn't, I'll re-check what happened.

> Perhaps it's worthwile to take a step back and figure out, if the
> importer is required to lock, which callbacks might need a ww acquire
> context?

I'll take this into account, thanks.

> (And off-topic, Since we do a lot of fancy stuff under dma-resv locks
> including waiting for fences and other locks, IMO taking these locks
> uninterruptible should ring a warning bell)

I had the same thought and had a version that used the interruptible
locking variant, but then decided to fall back to the uninterruptible,
don't remember why. I'll revisit this.

-- 
Best regards,
Dmitry

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dmitry Osipenko]

On 6/29/22 00:26, Thomas Hellström (Intel) wrote:
> 
> On 5/30/22 15:57, Dmitry Osipenko wrote:
>> On 5/30/22 16:41, Christian König wrote:
>>> Hi Dmitry,
>>>
>>> Am 30.05.22 um 15:26 schrieb Dmitry Osipenko:
>>>> Hello Christian,
>>>>
>>>> On 5/30/22 09:50, Christian König wrote:
>>>>> Hi Dmitry,
>>>>>
>>>>> First of all please separate out this patch from the rest of the
>>>>> series,
>>>>> since this is a complex separate structural change.
>>>> I assume all the patches will go via the DRM tree in the end since the
>>>> rest of the DRM patches in this series depend on this dma-buf change.
>>>> But I see that separation may ease reviewing of the dma-buf changes, so
>>>> let's try it.
>>> That sounds like you are underestimating a bit how much trouble this
>>> will be.
>>>
>>>>> I have tried this before and failed because catching all the locks in
>>>>> the right code paths are very tricky. So expect some fallout from this
>>>>> and make sure the kernel test robot and CI systems are clean.
>>>> Sure, I'll fix up all the reported things in the next iteration.
>>>>
>>>> BTW, have you ever posted yours version of the patch? Will be great if
>>>> we could compare the changed code paths.
>>> No, I never even finished creating it after realizing how much work it
>>> would be.
>>>
>>>>>> This patch introduces new locking convention for dma-buf users. From
>>>>>> now
>>>>>> on all dma-buf importers are responsible for holding dma-buf
>>>>>> reservation
>>>>>> lock around operations performed over dma-bufs.
>>>>>>
>>>>>> This patch implements the new dma-buf locking convention by:
>>>>>>
>>>>>>      1. Making dma-buf API functions to take the reservation lock.
>>>>>>
>>>>>>      2. Adding new locked variants of the dma-buf API functions for
>>>>>> drivers
>>>>>>         that need to manage imported dma-bufs under the held lock.
>>>>> Instead of adding new locked variants please mark all variants which
>>>>> expect to be called without a lock with an _unlocked postfix.
>>>>>
>>>>> This should make it easier to remove those in a follow up patch set
>>>>> and
>>>>> then fully move the locking into the importer.
>>>> Do we really want to move all the locks to the importers? Seems the
>>>> majority of drivers should be happy with the dma-buf helpers handling
>>>> the locking for them.
>>> Yes, I clearly think so.
>>>
>>>>>>      3. Converting all drivers to the new locking scheme.
>>>>> I have strong doubts that you got all of them. At least radeon and
>>>>> nouveau should grab the reservation lock in their ->attach callbacks
>>>>> somehow.
>>>> Radeon and Nouveau use gem_prime_import_sg_table() and they take resv
>>>> lock already, seems they should be okay (?)
>>> You are looking at the wrong side. You need to fix the export code path,
>>> not the import ones.
>>>
>>> See for example attach on radeon works like this
>>> drm_gem_map_attach->drm_gem_pin->radeon_gem_prime_pin->radeon_bo_reserve->ttm_bo_reserve->dma_resv_lock.
>>>
>> Yeah, I was looking at the both sides, but missed this one.
> 
> Also i915 will run into trouble with attach. In particular since i915
> starts a full ww transaction in its attach callback to be able to lock
> other objects if migration is needed. I think i915 CI would catch this
> in a selftest.

Seems it indeed it should deadlock. But i915 selftests apparently
should've caught it and they didn't, I'll re-check what happened.

> Perhaps it's worthwile to take a step back and figure out, if the
> importer is required to lock, which callbacks might need a ww acquire
> context?

I'll take this into account, thanks.

> (And off-topic, Since we do a lot of fancy stuff under dma-resv locks
> including waiting for fences and other locks, IMO taking these locks
> uninterruptible should ring a warning bell)

I had the same thought and had a version that used the interruptible
locking variant, but then decided to fall back to the uninterruptible,
don't remember why. I'll revisit this.

-- 
Best regards,
Dmitry

Re: [Intel-gfx] [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dmitry Osipenko]

On 6/29/22 00:26, Thomas Hellström (Intel) wrote:
> 
> On 5/30/22 15:57, Dmitry Osipenko wrote:
>> On 5/30/22 16:41, Christian König wrote:
>>> Hi Dmitry,
>>>
>>> Am 30.05.22 um 15:26 schrieb Dmitry Osipenko:
>>>> Hello Christian,
>>>>
>>>> On 5/30/22 09:50, Christian König wrote:
>>>>> Hi Dmitry,
>>>>>
>>>>> First of all please separate out this patch from the rest of the
>>>>> series,
>>>>> since this is a complex separate structural change.
>>>> I assume all the patches will go via the DRM tree in the end since the
>>>> rest of the DRM patches in this series depend on this dma-buf change.
>>>> But I see that separation may ease reviewing of the dma-buf changes, so
>>>> let's try it.
>>> That sounds like you are underestimating a bit how much trouble this
>>> will be.
>>>
>>>>> I have tried this before and failed because catching all the locks in
>>>>> the right code paths are very tricky. So expect some fallout from this
>>>>> and make sure the kernel test robot and CI systems are clean.
>>>> Sure, I'll fix up all the reported things in the next iteration.
>>>>
>>>> BTW, have you ever posted yours version of the patch? Will be great if
>>>> we could compare the changed code paths.
>>> No, I never even finished creating it after realizing how much work it
>>> would be.
>>>
>>>>>> This patch introduces new locking convention for dma-buf users. From
>>>>>> now
>>>>>> on all dma-buf importers are responsible for holding dma-buf
>>>>>> reservation
>>>>>> lock around operations performed over dma-bufs.
>>>>>>
>>>>>> This patch implements the new dma-buf locking convention by:
>>>>>>
>>>>>>      1. Making dma-buf API functions to take the reservation lock.
>>>>>>
>>>>>>      2. Adding new locked variants of the dma-buf API functions for
>>>>>> drivers
>>>>>>         that need to manage imported dma-bufs under the held lock.
>>>>> Instead of adding new locked variants please mark all variants which
>>>>> expect to be called without a lock with an _unlocked postfix.
>>>>>
>>>>> This should make it easier to remove those in a follow up patch set
>>>>> and
>>>>> then fully move the locking into the importer.
>>>> Do we really want to move all the locks to the importers? Seems the
>>>> majority of drivers should be happy with the dma-buf helpers handling
>>>> the locking for them.
>>> Yes, I clearly think so.
>>>
>>>>>>      3. Converting all drivers to the new locking scheme.
>>>>> I have strong doubts that you got all of them. At least radeon and
>>>>> nouveau should grab the reservation lock in their ->attach callbacks
>>>>> somehow.
>>>> Radeon and Nouveau use gem_prime_import_sg_table() and they take resv
>>>> lock already, seems they should be okay (?)
>>> You are looking at the wrong side. You need to fix the export code path,
>>> not the import ones.
>>>
>>> See for example attach on radeon works like this
>>> drm_gem_map_attach->drm_gem_pin->radeon_gem_prime_pin->radeon_bo_reserve->ttm_bo_reserve->dma_resv_lock.
>>>
>> Yeah, I was looking at the both sides, but missed this one.
> 
> Also i915 will run into trouble with attach. In particular since i915
> starts a full ww transaction in its attach callback to be able to lock
> other objects if migration is needed. I think i915 CI would catch this
> in a selftest.

Seems it indeed it should deadlock. But i915 selftests apparently
should've caught it and they didn't, I'll re-check what happened.

> Perhaps it's worthwile to take a step back and figure out, if the
> importer is required to lock, which callbacks might need a ww acquire
> context?

I'll take this into account, thanks.

> (And off-topic, Since we do a lot of fancy stuff under dma-resv locks
> including waiting for fences and other locks, IMO taking these locks
> uninterruptible should ring a warning bell)

I had the same thought and had a version that used the interruptible
locking variant, but then decided to fall back to the uninterruptible,
don't remember why. I'll revisit this.

-- 
Best regards,
Dmitry

Re: [Linaro-mm-sig] Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Christian König]

Am 30.06.22 um 01:06 schrieb Dmitry Osipenko:
> On 6/29/22 11:43, Thomas Hellström (Intel) wrote:
>> On 6/29/22 10:22, Dmitry Osipenko wrote:
>>> On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
>>>> On 5/27/22 01:50, Dmitry Osipenko wrote:
>>>>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>>>>> handle imported dma-bufs properly, which results in mapping of
>>>>> something
>>>>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>>>>> hard
>>>>> lockup when userspace writes to the memory mapping of a dma-buf that
>>>>> was
>>>>> imported into Tegra's DRM GEM.
>>>>>
>>>>> To fix this bug, move mapping of imported dma-bufs to
>>>>> drm_gem_mmap_obj().
>>>>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
>>>> Same comment about Fixes: as in patch 1,
>>>>> Cc: stable@vger.kernel.org
>>>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>>>> ---
>>>>>     drivers/gpu/drm/drm_gem.c              | 3 +++
>>>>>     drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>>>>     drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>>>>     3 files changed, 7 insertions(+), 9 deletions(-)
>>>>>
>>>>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>>>>> index 86d670c71286..7c0b025508e4 100644
>>>>> --- a/drivers/gpu/drm/drm_gem.c
>>>>> +++ b/drivers/gpu/drm/drm_gem.c
>>>>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj,
>>>>> unsigned long obj_size,
>>>>>         if (obj_size < vma->vm_end - vma->vm_start)
>>>>>             return -EINVAL;
>>>>>     +    if (obj->import_attach)
>>>>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
>>>> If we start enabling mmaping of imported dma-bufs on a majority of
>>>> drivers in this way, how do we ensure that user-space is not blindly
>>>> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
>>>> which is needed before and after cpu access of mmap'ed dma-bufs?
>>>>
>>>> I was under the impression (admittedly without looking) that the few
>>>> drivers that actually called into dma_buf_mmap() had some private
>>>> user-mode driver code in place that ensured this happened.
>>> Since it's a userspace who does the mapping, then it should be a
>>> responsibility of userspace to do all the necessary syncing.
>> Sure, but nothing prohibits user-space to ignore the syncing thinking
>> "It works anyway", testing those drivers where the syncing is a NOP. And
>> when a driver that finally needs syncing is tested it's too late to fix
>> all broken user-space.
>>
>>>    I'm not
>>> sure whether anyone in userspace really needs to map imported dma-bufs
>>> in practice. Nevertheless, this use-case is broken and should be fixed
>>> by either allowing to do the mapping or prohibiting it.
>>>
>> Then I'd vote for prohibiting it, at least for now. And for the future
>> moving forward we could perhaps revisit the dma-buf need for syncing,
>> requiring those drivers that actually need it to implement emulated
>> coherent memory which can be done not too inefficiently (vmwgfx being
>> one example).
> Alright, I'll change it to prohibit the mapping. This indeed should be a
> better option.

Oh, yes please. But I would expect that some people start screaming.

Over time I've got tons of TTM patches because people illegally tried to 
mmap() imported DMA-bufs in their driver.

Anyway this is probably the right thing to do and we can work on fixing 
the fallout later on.

Regards,
Christian.

Re: [Linaro-mm-sig] Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Christian König via Virtualization]

Am 30.06.22 um 01:06 schrieb Dmitry Osipenko:
> On 6/29/22 11:43, Thomas Hellström (Intel) wrote:
>> On 6/29/22 10:22, Dmitry Osipenko wrote:
>>> On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
>>>> On 5/27/22 01:50, Dmitry Osipenko wrote:
>>>>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>>>>> handle imported dma-bufs properly, which results in mapping of
>>>>> something
>>>>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>>>>> hard
>>>>> lockup when userspace writes to the memory mapping of a dma-buf that
>>>>> was
>>>>> imported into Tegra's DRM GEM.
>>>>>
>>>>> To fix this bug, move mapping of imported dma-bufs to
>>>>> drm_gem_mmap_obj().
>>>>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
>>>> Same comment about Fixes: as in patch 1,
>>>>> Cc: stable@vger.kernel.org
>>>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>>>> ---
>>>>>     drivers/gpu/drm/drm_gem.c              | 3 +++
>>>>>     drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>>>>     drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>>>>     3 files changed, 7 insertions(+), 9 deletions(-)
>>>>>
>>>>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>>>>> index 86d670c71286..7c0b025508e4 100644
>>>>> --- a/drivers/gpu/drm/drm_gem.c
>>>>> +++ b/drivers/gpu/drm/drm_gem.c
>>>>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj,
>>>>> unsigned long obj_size,
>>>>>         if (obj_size < vma->vm_end - vma->vm_start)
>>>>>             return -EINVAL;
>>>>>     +    if (obj->import_attach)
>>>>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
>>>> If we start enabling mmaping of imported dma-bufs on a majority of
>>>> drivers in this way, how do we ensure that user-space is not blindly
>>>> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
>>>> which is needed before and after cpu access of mmap'ed dma-bufs?
>>>>
>>>> I was under the impression (admittedly without looking) that the few
>>>> drivers that actually called into dma_buf_mmap() had some private
>>>> user-mode driver code in place that ensured this happened.
>>> Since it's a userspace who does the mapping, then it should be a
>>> responsibility of userspace to do all the necessary syncing.
>> Sure, but nothing prohibits user-space to ignore the syncing thinking
>> "It works anyway", testing those drivers where the syncing is a NOP. And
>> when a driver that finally needs syncing is tested it's too late to fix
>> all broken user-space.
>>
>>>    I'm not
>>> sure whether anyone in userspace really needs to map imported dma-bufs
>>> in practice. Nevertheless, this use-case is broken and should be fixed
>>> by either allowing to do the mapping or prohibiting it.
>>>
>> Then I'd vote for prohibiting it, at least for now. And for the future
>> moving forward we could perhaps revisit the dma-buf need for syncing,
>> requiring those drivers that actually need it to implement emulated
>> coherent memory which can be done not too inefficiently (vmwgfx being
>> one example).
> Alright, I'll change it to prohibit the mapping. This indeed should be a
> better option.

Oh, yes please. But I would expect that some people start screaming.

Over time I've got tons of TTM patches because people illegally tried to 
mmap() imported DMA-bufs in their driver.

Anyway this is probably the right thing to do and we can work on fixing 
the fallout later on.

Regards,
Christian.


_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [Linaro-mm-sig] Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Christian König]

Am 30.06.22 um 01:06 schrieb Dmitry Osipenko:
> On 6/29/22 11:43, Thomas Hellström (Intel) wrote:
>> On 6/29/22 10:22, Dmitry Osipenko wrote:
>>> On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
>>>> On 5/27/22 01:50, Dmitry Osipenko wrote:
>>>>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>>>>> handle imported dma-bufs properly, which results in mapping of
>>>>> something
>>>>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>>>>> hard
>>>>> lockup when userspace writes to the memory mapping of a dma-buf that
>>>>> was
>>>>> imported into Tegra's DRM GEM.
>>>>>
>>>>> To fix this bug, move mapping of imported dma-bufs to
>>>>> drm_gem_mmap_obj().
>>>>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
>>>> Same comment about Fixes: as in patch 1,
>>>>> Cc: stable@vger.kernel.org
>>>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>>>> ---
>>>>>     drivers/gpu/drm/drm_gem.c              | 3 +++
>>>>>     drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>>>>     drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>>>>     3 files changed, 7 insertions(+), 9 deletions(-)
>>>>>
>>>>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>>>>> index 86d670c71286..7c0b025508e4 100644
>>>>> --- a/drivers/gpu/drm/drm_gem.c
>>>>> +++ b/drivers/gpu/drm/drm_gem.c
>>>>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj,
>>>>> unsigned long obj_size,
>>>>>         if (obj_size < vma->vm_end - vma->vm_start)
>>>>>             return -EINVAL;
>>>>>     +    if (obj->import_attach)
>>>>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
>>>> If we start enabling mmaping of imported dma-bufs on a majority of
>>>> drivers in this way, how do we ensure that user-space is not blindly
>>>> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
>>>> which is needed before and after cpu access of mmap'ed dma-bufs?
>>>>
>>>> I was under the impression (admittedly without looking) that the few
>>>> drivers that actually called into dma_buf_mmap() had some private
>>>> user-mode driver code in place that ensured this happened.
>>> Since it's a userspace who does the mapping, then it should be a
>>> responsibility of userspace to do all the necessary syncing.
>> Sure, but nothing prohibits user-space to ignore the syncing thinking
>> "It works anyway", testing those drivers where the syncing is a NOP. And
>> when a driver that finally needs syncing is tested it's too late to fix
>> all broken user-space.
>>
>>>    I'm not
>>> sure whether anyone in userspace really needs to map imported dma-bufs
>>> in practice. Nevertheless, this use-case is broken and should be fixed
>>> by either allowing to do the mapping or prohibiting it.
>>>
>> Then I'd vote for prohibiting it, at least for now. And for the future
>> moving forward we could perhaps revisit the dma-buf need for syncing,
>> requiring those drivers that actually need it to implement emulated
>> coherent memory which can be done not too inefficiently (vmwgfx being
>> one example).
> Alright, I'll change it to prohibit the mapping. This indeed should be a
> better option.

Oh, yes please. But I would expect that some people start screaming.

Over time I've got tons of TTM patches because people illegally tried to 
mmap() imported DMA-bufs in their driver.

Anyway this is probably the right thing to do and we can work on fixing 
the fallout later on.

Regards,
Christian.

Re: [Linaro-mm-sig] Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Christian König]

Am 30.06.22 um 01:06 schrieb Dmitry Osipenko:
> On 6/29/22 11:43, Thomas Hellström (Intel) wrote:
>> On 6/29/22 10:22, Dmitry Osipenko wrote:
>>> On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
>>>> On 5/27/22 01:50, Dmitry Osipenko wrote:
>>>>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>>>>> handle imported dma-bufs properly, which results in mapping of
>>>>> something
>>>>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>>>>> hard
>>>>> lockup when userspace writes to the memory mapping of a dma-buf that
>>>>> was
>>>>> imported into Tegra's DRM GEM.
>>>>>
>>>>> To fix this bug, move mapping of imported dma-bufs to
>>>>> drm_gem_mmap_obj().
>>>>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
>>>> Same comment about Fixes: as in patch 1,
>>>>> Cc: stable@vger.kernel.org
>>>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>>>> ---
>>>>>     drivers/gpu/drm/drm_gem.c              | 3 +++
>>>>>     drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>>>>     drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>>>>     3 files changed, 7 insertions(+), 9 deletions(-)
>>>>>
>>>>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>>>>> index 86d670c71286..7c0b025508e4 100644
>>>>> --- a/drivers/gpu/drm/drm_gem.c
>>>>> +++ b/drivers/gpu/drm/drm_gem.c
>>>>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj,
>>>>> unsigned long obj_size,
>>>>>         if (obj_size < vma->vm_end - vma->vm_start)
>>>>>             return -EINVAL;
>>>>>     +    if (obj->import_attach)
>>>>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
>>>> If we start enabling mmaping of imported dma-bufs on a majority of
>>>> drivers in this way, how do we ensure that user-space is not blindly
>>>> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
>>>> which is needed before and after cpu access of mmap'ed dma-bufs?
>>>>
>>>> I was under the impression (admittedly without looking) that the few
>>>> drivers that actually called into dma_buf_mmap() had some private
>>>> user-mode driver code in place that ensured this happened.
>>> Since it's a userspace who does the mapping, then it should be a
>>> responsibility of userspace to do all the necessary syncing.
>> Sure, but nothing prohibits user-space to ignore the syncing thinking
>> "It works anyway", testing those drivers where the syncing is a NOP. And
>> when a driver that finally needs syncing is tested it's too late to fix
>> all broken user-space.
>>
>>>    I'm not
>>> sure whether anyone in userspace really needs to map imported dma-bufs
>>> in practice. Nevertheless, this use-case is broken and should be fixed
>>> by either allowing to do the mapping or prohibiting it.
>>>
>> Then I'd vote for prohibiting it, at least for now. And for the future
>> moving forward we could perhaps revisit the dma-buf need for syncing,
>> requiring those drivers that actually need it to implement emulated
>> coherent memory which can be done not too inefficiently (vmwgfx being
>> one example).
> Alright, I'll change it to prohibit the mapping. This indeed should be a
> better option.

Oh, yes please. But I would expect that some people start screaming.

Over time I've got tons of TTM patches because people illegally tried to 
mmap() imported DMA-bufs in their driver.

Anyway this is probably the right thing to do and we can work on fixing 
the fallout later on.

Regards,
Christian.

Re: [Intel-gfx] [Linaro-mm-sig] Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Christian König]

Am 30.06.22 um 01:06 schrieb Dmitry Osipenko:
> On 6/29/22 11:43, Thomas Hellström (Intel) wrote:
>> On 6/29/22 10:22, Dmitry Osipenko wrote:
>>> On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
>>>> On 5/27/22 01:50, Dmitry Osipenko wrote:
>>>>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>>>>> handle imported dma-bufs properly, which results in mapping of
>>>>> something
>>>>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>>>>> hard
>>>>> lockup when userspace writes to the memory mapping of a dma-buf that
>>>>> was
>>>>> imported into Tegra's DRM GEM.
>>>>>
>>>>> To fix this bug, move mapping of imported dma-bufs to
>>>>> drm_gem_mmap_obj().
>>>>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
>>>> Same comment about Fixes: as in patch 1,
>>>>> Cc: stable@vger.kernel.org
>>>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>>>> ---
>>>>>     drivers/gpu/drm/drm_gem.c              | 3 +++
>>>>>     drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>>>>     drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>>>>     3 files changed, 7 insertions(+), 9 deletions(-)
>>>>>
>>>>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>>>>> index 86d670c71286..7c0b025508e4 100644
>>>>> --- a/drivers/gpu/drm/drm_gem.c
>>>>> +++ b/drivers/gpu/drm/drm_gem.c
>>>>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj,
>>>>> unsigned long obj_size,
>>>>>         if (obj_size < vma->vm_end - vma->vm_start)
>>>>>             return -EINVAL;
>>>>>     +    if (obj->import_attach)
>>>>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
>>>> If we start enabling mmaping of imported dma-bufs on a majority of
>>>> drivers in this way, how do we ensure that user-space is not blindly
>>>> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
>>>> which is needed before and after cpu access of mmap'ed dma-bufs?
>>>>
>>>> I was under the impression (admittedly without looking) that the few
>>>> drivers that actually called into dma_buf_mmap() had some private
>>>> user-mode driver code in place that ensured this happened.
>>> Since it's a userspace who does the mapping, then it should be a
>>> responsibility of userspace to do all the necessary syncing.
>> Sure, but nothing prohibits user-space to ignore the syncing thinking
>> "It works anyway", testing those drivers where the syncing is a NOP. And
>> when a driver that finally needs syncing is tested it's too late to fix
>> all broken user-space.
>>
>>>    I'm not
>>> sure whether anyone in userspace really needs to map imported dma-bufs
>>> in practice. Nevertheless, this use-case is broken and should be fixed
>>> by either allowing to do the mapping or prohibiting it.
>>>
>> Then I'd vote for prohibiting it, at least for now. And for the future
>> moving forward we could perhaps revisit the dma-buf need for syncing,
>> requiring those drivers that actually need it to implement emulated
>> coherent memory which can be done not too inefficiently (vmwgfx being
>> one example).
> Alright, I'll change it to prohibit the mapping. This indeed should be a
> better option.

Oh, yes please. But I would expect that some people start screaming.

Over time I've got tons of TTM patches because people illegally tried to 
mmap() imported DMA-bufs in their driver.

Anyway this is probably the right thing to do and we can work on fixing 
the fallout later on.

Regards,
Christian.

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dmitry Osipenko]

On 7/1/22 13:43, Dmitry Osipenko wrote:
> On 6/29/22 00:26, Thomas Hellström (Intel) wrote:
>> On 5/30/22 15:57, Dmitry Osipenko wrote:
>>> On 5/30/22 16:41, Christian König wrote:
>>>> Hi Dmitry,
>>>>
>>>> Am 30.05.22 um 15:26 schrieb Dmitry Osipenko:
>>>>> Hello Christian,
>>>>>
>>>>> On 5/30/22 09:50, Christian König wrote:
>>>>>> Hi Dmitry,
>>>>>>
>>>>>> First of all please separate out this patch from the rest of the
>>>>>> series,
>>>>>> since this is a complex separate structural change.
>>>>> I assume all the patches will go via the DRM tree in the end since the
>>>>> rest of the DRM patches in this series depend on this dma-buf change.
>>>>> But I see that separation may ease reviewing of the dma-buf changes, so
>>>>> let's try it.
>>>> That sounds like you are underestimating a bit how much trouble this
>>>> will be.
>>>>
>>>>>> I have tried this before and failed because catching all the locks in
>>>>>> the right code paths are very tricky. So expect some fallout from this
>>>>>> and make sure the kernel test robot and CI systems are clean.
>>>>> Sure, I'll fix up all the reported things in the next iteration.
>>>>>
>>>>> BTW, have you ever posted yours version of the patch? Will be great if
>>>>> we could compare the changed code paths.
>>>> No, I never even finished creating it after realizing how much work it
>>>> would be.
>>>>
>>>>>>> This patch introduces new locking convention for dma-buf users. From
>>>>>>> now
>>>>>>> on all dma-buf importers are responsible for holding dma-buf
>>>>>>> reservation
>>>>>>> lock around operations performed over dma-bufs.
>>>>>>>
>>>>>>> This patch implements the new dma-buf locking convention by:
>>>>>>>
>>>>>>>      1. Making dma-buf API functions to take the reservation lock.
>>>>>>>
>>>>>>>      2. Adding new locked variants of the dma-buf API functions for
>>>>>>> drivers
>>>>>>>         that need to manage imported dma-bufs under the held lock.
>>>>>> Instead of adding new locked variants please mark all variants which
>>>>>> expect to be called without a lock with an _unlocked postfix.
>>>>>>
>>>>>> This should make it easier to remove those in a follow up patch set
>>>>>> and
>>>>>> then fully move the locking into the importer.
>>>>> Do we really want to move all the locks to the importers? Seems the
>>>>> majority of drivers should be happy with the dma-buf helpers handling
>>>>> the locking for them.
>>>> Yes, I clearly think so.
>>>>
>>>>>>>      3. Converting all drivers to the new locking scheme.
>>>>>> I have strong doubts that you got all of them. At least radeon and
>>>>>> nouveau should grab the reservation lock in their ->attach callbacks
>>>>>> somehow.
>>>>> Radeon and Nouveau use gem_prime_import_sg_table() and they take resv
>>>>> lock already, seems they should be okay (?)
>>>> You are looking at the wrong side. You need to fix the export code path,
>>>> not the import ones.
>>>>
>>>> See for example attach on radeon works like this
>>>> drm_gem_map_attach->drm_gem_pin->radeon_gem_prime_pin->radeon_bo_reserve->ttm_bo_reserve->dma_resv_lock.
>>>>
>>> Yeah, I was looking at the both sides, but missed this one.
>> Also i915 will run into trouble with attach. In particular since i915
>> starts a full ww transaction in its attach callback to be able to lock
>> other objects if migration is needed. I think i915 CI would catch this
>> in a selftest.
> Seems it indeed it should deadlock. But i915 selftests apparently
> should've caught it and they didn't, I'll re-check what happened.
> 

The i915 selftests use a separate mock_dmabuf_ops. That's why it works
for the selftests, i.e. there is no deadlock.

Thomas, would i915 CI run a different set of tests or will it be the
default i915 selftests ran by IGT?

-- 
Best regards,
Dmitry

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dmitry Osipenko]

On 7/1/22 13:43, Dmitry Osipenko wrote:
> On 6/29/22 00:26, Thomas Hellström (Intel) wrote:
>> On 5/30/22 15:57, Dmitry Osipenko wrote:
>>> On 5/30/22 16:41, Christian König wrote:
>>>> Hi Dmitry,
>>>>
>>>> Am 30.05.22 um 15:26 schrieb Dmitry Osipenko:
>>>>> Hello Christian,
>>>>>
>>>>> On 5/30/22 09:50, Christian König wrote:
>>>>>> Hi Dmitry,
>>>>>>
>>>>>> First of all please separate out this patch from the rest of the
>>>>>> series,
>>>>>> since this is a complex separate structural change.
>>>>> I assume all the patches will go via the DRM tree in the end since the
>>>>> rest of the DRM patches in this series depend on this dma-buf change.
>>>>> But I see that separation may ease reviewing of the dma-buf changes, so
>>>>> let's try it.
>>>> That sounds like you are underestimating a bit how much trouble this
>>>> will be.
>>>>
>>>>>> I have tried this before and failed because catching all the locks in
>>>>>> the right code paths are very tricky. So expect some fallout from this
>>>>>> and make sure the kernel test robot and CI systems are clean.
>>>>> Sure, I'll fix up all the reported things in the next iteration.
>>>>>
>>>>> BTW, have you ever posted yours version of the patch? Will be great if
>>>>> we could compare the changed code paths.
>>>> No, I never even finished creating it after realizing how much work it
>>>> would be.
>>>>
>>>>>>> This patch introduces new locking convention for dma-buf users. From
>>>>>>> now
>>>>>>> on all dma-buf importers are responsible for holding dma-buf
>>>>>>> reservation
>>>>>>> lock around operations performed over dma-bufs.
>>>>>>>
>>>>>>> This patch implements the new dma-buf locking convention by:
>>>>>>>
>>>>>>>      1. Making dma-buf API functions to take the reservation lock.
>>>>>>>
>>>>>>>      2. Adding new locked variants of the dma-buf API functions for
>>>>>>> drivers
>>>>>>>         that need to manage imported dma-bufs under the held lock.
>>>>>> Instead of adding new locked variants please mark all variants which
>>>>>> expect to be called without a lock with an _unlocked postfix.
>>>>>>
>>>>>> This should make it easier to remove those in a follow up patch set
>>>>>> and
>>>>>> then fully move the locking into the importer.
>>>>> Do we really want to move all the locks to the importers? Seems the
>>>>> majority of drivers should be happy with the dma-buf helpers handling
>>>>> the locking for them.
>>>> Yes, I clearly think so.
>>>>
>>>>>>>      3. Converting all drivers to the new locking scheme.
>>>>>> I have strong doubts that you got all of them. At least radeon and
>>>>>> nouveau should grab the reservation lock in their ->attach callbacks
>>>>>> somehow.
>>>>> Radeon and Nouveau use gem_prime_import_sg_table() and they take resv
>>>>> lock already, seems they should be okay (?)
>>>> You are looking at the wrong side. You need to fix the export code path,
>>>> not the import ones.
>>>>
>>>> See for example attach on radeon works like this
>>>> drm_gem_map_attach->drm_gem_pin->radeon_gem_prime_pin->radeon_bo_reserve->ttm_bo_reserve->dma_resv_lock.
>>>>
>>> Yeah, I was looking at the both sides, but missed this one.
>> Also i915 will run into trouble with attach. In particular since i915
>> starts a full ww transaction in its attach callback to be able to lock
>> other objects if migration is needed. I think i915 CI would catch this
>> in a selftest.
> Seems it indeed it should deadlock. But i915 selftests apparently
> should've caught it and they didn't, I'll re-check what happened.
> 

The i915 selftests use a separate mock_dmabuf_ops. That's why it works
for the selftests, i.e. there is no deadlock.

Thomas, would i915 CI run a different set of tests or will it be the
default i915 selftests ran by IGT?

-- 
Best regards,
Dmitry

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dmitry Osipenko]

On 7/1/22 13:43, Dmitry Osipenko wrote:
> On 6/29/22 00:26, Thomas Hellström (Intel) wrote:
>> On 5/30/22 15:57, Dmitry Osipenko wrote:
>>> On 5/30/22 16:41, Christian König wrote:
>>>> Hi Dmitry,
>>>>
>>>> Am 30.05.22 um 15:26 schrieb Dmitry Osipenko:
>>>>> Hello Christian,
>>>>>
>>>>> On 5/30/22 09:50, Christian König wrote:
>>>>>> Hi Dmitry,
>>>>>>
>>>>>> First of all please separate out this patch from the rest of the
>>>>>> series,
>>>>>> since this is a complex separate structural change.
>>>>> I assume all the patches will go via the DRM tree in the end since the
>>>>> rest of the DRM patches in this series depend on this dma-buf change.
>>>>> But I see that separation may ease reviewing of the dma-buf changes, so
>>>>> let's try it.
>>>> That sounds like you are underestimating a bit how much trouble this
>>>> will be.
>>>>
>>>>>> I have tried this before and failed because catching all the locks in
>>>>>> the right code paths are very tricky. So expect some fallout from this
>>>>>> and make sure the kernel test robot and CI systems are clean.
>>>>> Sure, I'll fix up all the reported things in the next iteration.
>>>>>
>>>>> BTW, have you ever posted yours version of the patch? Will be great if
>>>>> we could compare the changed code paths.
>>>> No, I never even finished creating it after realizing how much work it
>>>> would be.
>>>>
>>>>>>> This patch introduces new locking convention for dma-buf users. From
>>>>>>> now
>>>>>>> on all dma-buf importers are responsible for holding dma-buf
>>>>>>> reservation
>>>>>>> lock around operations performed over dma-bufs.
>>>>>>>
>>>>>>> This patch implements the new dma-buf locking convention by:
>>>>>>>
>>>>>>>      1. Making dma-buf API functions to take the reservation lock.
>>>>>>>
>>>>>>>      2. Adding new locked variants of the dma-buf API functions for
>>>>>>> drivers
>>>>>>>         that need to manage imported dma-bufs under the held lock.
>>>>>> Instead of adding new locked variants please mark all variants which
>>>>>> expect to be called without a lock with an _unlocked postfix.
>>>>>>
>>>>>> This should make it easier to remove those in a follow up patch set
>>>>>> and
>>>>>> then fully move the locking into the importer.
>>>>> Do we really want to move all the locks to the importers? Seems the
>>>>> majority of drivers should be happy with the dma-buf helpers handling
>>>>> the locking for them.
>>>> Yes, I clearly think so.
>>>>
>>>>>>>      3. Converting all drivers to the new locking scheme.
>>>>>> I have strong doubts that you got all of them. At least radeon and
>>>>>> nouveau should grab the reservation lock in their ->attach callbacks
>>>>>> somehow.
>>>>> Radeon and Nouveau use gem_prime_import_sg_table() and they take resv
>>>>> lock already, seems they should be okay (?)
>>>> You are looking at the wrong side. You need to fix the export code path,
>>>> not the import ones.
>>>>
>>>> See for example attach on radeon works like this
>>>> drm_gem_map_attach->drm_gem_pin->radeon_gem_prime_pin->radeon_bo_reserve->ttm_bo_reserve->dma_resv_lock.
>>>>
>>> Yeah, I was looking at the both sides, but missed this one.
>> Also i915 will run into trouble with attach. In particular since i915
>> starts a full ww transaction in its attach callback to be able to lock
>> other objects if migration is needed. I think i915 CI would catch this
>> in a selftest.
> Seems it indeed it should deadlock. But i915 selftests apparently
> should've caught it and they didn't, I'll re-check what happened.
> 

The i915 selftests use a separate mock_dmabuf_ops. That's why it works
for the selftests, i.e. there is no deadlock.

Thomas, would i915 CI run a different set of tests or will it be the
default i915 selftests ran by IGT?

-- 
Best regards,
Dmitry

Re: [Intel-gfx] [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dmitry Osipenko]

On 7/1/22 13:43, Dmitry Osipenko wrote:
> On 6/29/22 00:26, Thomas Hellström (Intel) wrote:
>> On 5/30/22 15:57, Dmitry Osipenko wrote:
>>> On 5/30/22 16:41, Christian König wrote:
>>>> Hi Dmitry,
>>>>
>>>> Am 30.05.22 um 15:26 schrieb Dmitry Osipenko:
>>>>> Hello Christian,
>>>>>
>>>>> On 5/30/22 09:50, Christian König wrote:
>>>>>> Hi Dmitry,
>>>>>>
>>>>>> First of all please separate out this patch from the rest of the
>>>>>> series,
>>>>>> since this is a complex separate structural change.
>>>>> I assume all the patches will go via the DRM tree in the end since the
>>>>> rest of the DRM patches in this series depend on this dma-buf change.
>>>>> But I see that separation may ease reviewing of the dma-buf changes, so
>>>>> let's try it.
>>>> That sounds like you are underestimating a bit how much trouble this
>>>> will be.
>>>>
>>>>>> I have tried this before and failed because catching all the locks in
>>>>>> the right code paths are very tricky. So expect some fallout from this
>>>>>> and make sure the kernel test robot and CI systems are clean.
>>>>> Sure, I'll fix up all the reported things in the next iteration.
>>>>>
>>>>> BTW, have you ever posted yours version of the patch? Will be great if
>>>>> we could compare the changed code paths.
>>>> No, I never even finished creating it after realizing how much work it
>>>> would be.
>>>>
>>>>>>> This patch introduces new locking convention for dma-buf users. From
>>>>>>> now
>>>>>>> on all dma-buf importers are responsible for holding dma-buf
>>>>>>> reservation
>>>>>>> lock around operations performed over dma-bufs.
>>>>>>>
>>>>>>> This patch implements the new dma-buf locking convention by:
>>>>>>>
>>>>>>>      1. Making dma-buf API functions to take the reservation lock.
>>>>>>>
>>>>>>>      2. Adding new locked variants of the dma-buf API functions for
>>>>>>> drivers
>>>>>>>         that need to manage imported dma-bufs under the held lock.
>>>>>> Instead of adding new locked variants please mark all variants which
>>>>>> expect to be called without a lock with an _unlocked postfix.
>>>>>>
>>>>>> This should make it easier to remove those in a follow up patch set
>>>>>> and
>>>>>> then fully move the locking into the importer.
>>>>> Do we really want to move all the locks to the importers? Seems the
>>>>> majority of drivers should be happy with the dma-buf helpers handling
>>>>> the locking for them.
>>>> Yes, I clearly think so.
>>>>
>>>>>>>      3. Converting all drivers to the new locking scheme.
>>>>>> I have strong doubts that you got all of them. At least radeon and
>>>>>> nouveau should grab the reservation lock in their ->attach callbacks
>>>>>> somehow.
>>>>> Radeon and Nouveau use gem_prime_import_sg_table() and they take resv
>>>>> lock already, seems they should be okay (?)
>>>> You are looking at the wrong side. You need to fix the export code path,
>>>> not the import ones.
>>>>
>>>> See for example attach on radeon works like this
>>>> drm_gem_map_attach->drm_gem_pin->radeon_gem_prime_pin->radeon_bo_reserve->ttm_bo_reserve->dma_resv_lock.
>>>>
>>> Yeah, I was looking at the both sides, but missed this one.
>> Also i915 will run into trouble with attach. In particular since i915
>> starts a full ww transaction in its attach callback to be able to lock
>> other objects if migration is needed. I think i915 CI would catch this
>> in a selftest.
> Seems it indeed it should deadlock. But i915 selftests apparently
> should've caught it and they didn't, I'll re-check what happened.
> 

The i915 selftests use a separate mock_dmabuf_ops. That's why it works
for the selftests, i.e. there is no deadlock.

Thomas, would i915 CI run a different set of tests or will it be the
default i915 selftests ran by IGT?

-- 
Best regards,
Dmitry

Re: [Linaro-mm-sig] Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Dmitry Osipenko]

On 7/4/22 15:33, Christian König wrote:
> Am 30.06.22 um 01:06 schrieb Dmitry Osipenko:
>> On 6/29/22 11:43, Thomas Hellström (Intel) wrote:
>>> On 6/29/22 10:22, Dmitry Osipenko wrote:
>>>> On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
>>>>> On 5/27/22 01:50, Dmitry Osipenko wrote:
>>>>>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>>>>>> handle imported dma-bufs properly, which results in mapping of
>>>>>> something
>>>>>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>>>>>> hard
>>>>>> lockup when userspace writes to the memory mapping of a dma-buf that
>>>>>> was
>>>>>> imported into Tegra's DRM GEM.
>>>>>>
>>>>>> To fix this bug, move mapping of imported dma-bufs to
>>>>>> drm_gem_mmap_obj().
>>>>>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
>>>>> Same comment about Fixes: as in patch 1,
>>>>>> Cc: stable@vger.kernel.org
>>>>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>>>>> ---
>>>>>>     drivers/gpu/drm/drm_gem.c              | 3 +++
>>>>>>     drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>>>>>     drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>>>>>     3 files changed, 7 insertions(+), 9 deletions(-)
>>>>>>
>>>>>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>>>>>> index 86d670c71286..7c0b025508e4 100644
>>>>>> --- a/drivers/gpu/drm/drm_gem.c
>>>>>> +++ b/drivers/gpu/drm/drm_gem.c
>>>>>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object
>>>>>> *obj,
>>>>>> unsigned long obj_size,
>>>>>>         if (obj_size < vma->vm_end - vma->vm_start)
>>>>>>             return -EINVAL;
>>>>>>     +    if (obj->import_attach)
>>>>>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
>>>>> If we start enabling mmaping of imported dma-bufs on a majority of
>>>>> drivers in this way, how do we ensure that user-space is not blindly
>>>>> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
>>>>> which is needed before and after cpu access of mmap'ed dma-bufs?
>>>>>
>>>>> I was under the impression (admittedly without looking) that the few
>>>>> drivers that actually called into dma_buf_mmap() had some private
>>>>> user-mode driver code in place that ensured this happened.
>>>> Since it's a userspace who does the mapping, then it should be a
>>>> responsibility of userspace to do all the necessary syncing.
>>> Sure, but nothing prohibits user-space to ignore the syncing thinking
>>> "It works anyway", testing those drivers where the syncing is a NOP. And
>>> when a driver that finally needs syncing is tested it's too late to fix
>>> all broken user-space.
>>>
>>>>    I'm not
>>>> sure whether anyone in userspace really needs to map imported dma-bufs
>>>> in practice. Nevertheless, this use-case is broken and should be fixed
>>>> by either allowing to do the mapping or prohibiting it.
>>>>
>>> Then I'd vote for prohibiting it, at least for now. And for the future
>>> moving forward we could perhaps revisit the dma-buf need for syncing,
>>> requiring those drivers that actually need it to implement emulated
>>> coherent memory which can be done not too inefficiently (vmwgfx being
>>> one example).
>> Alright, I'll change it to prohibit the mapping. This indeed should be a
>> better option.
> 
> Oh, yes please. But I would expect that some people start screaming.
> 
> Over time I've got tons of TTM patches because people illegally tried to
> mmap() imported DMA-bufs in their driver.
> 
> Anyway this is probably the right thing to do and we can work on fixing
> the fallout later on.

I already sent out the patch [1] that prohibits the mapping. Would be
great if you all could take a look and give a r-b, thanks in advance.

[1] https://patchwork.freedesktop.org/patch/492148/

-- 
Best regards,
Dmitry

Re: [Linaro-mm-sig] Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Dmitry Osipenko]

On 7/4/22 15:33, Christian König wrote:
> Am 30.06.22 um 01:06 schrieb Dmitry Osipenko:
>> On 6/29/22 11:43, Thomas Hellström (Intel) wrote:
>>> On 6/29/22 10:22, Dmitry Osipenko wrote:
>>>> On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
>>>>> On 5/27/22 01:50, Dmitry Osipenko wrote:
>>>>>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>>>>>> handle imported dma-bufs properly, which results in mapping of
>>>>>> something
>>>>>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>>>>>> hard
>>>>>> lockup when userspace writes to the memory mapping of a dma-buf that
>>>>>> was
>>>>>> imported into Tegra's DRM GEM.
>>>>>>
>>>>>> To fix this bug, move mapping of imported dma-bufs to
>>>>>> drm_gem_mmap_obj().
>>>>>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
>>>>> Same comment about Fixes: as in patch 1,
>>>>>> Cc: stable@vger.kernel.org
>>>>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>>>>> ---
>>>>>>     drivers/gpu/drm/drm_gem.c              | 3 +++
>>>>>>     drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>>>>>     drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>>>>>     3 files changed, 7 insertions(+), 9 deletions(-)
>>>>>>
>>>>>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>>>>>> index 86d670c71286..7c0b025508e4 100644
>>>>>> --- a/drivers/gpu/drm/drm_gem.c
>>>>>> +++ b/drivers/gpu/drm/drm_gem.c
>>>>>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object
>>>>>> *obj,
>>>>>> unsigned long obj_size,
>>>>>>         if (obj_size < vma->vm_end - vma->vm_start)
>>>>>>             return -EINVAL;
>>>>>>     +    if (obj->import_attach)
>>>>>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
>>>>> If we start enabling mmaping of imported dma-bufs on a majority of
>>>>> drivers in this way, how do we ensure that user-space is not blindly
>>>>> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
>>>>> which is needed before and after cpu access of mmap'ed dma-bufs?
>>>>>
>>>>> I was under the impression (admittedly without looking) that the few
>>>>> drivers that actually called into dma_buf_mmap() had some private
>>>>> user-mode driver code in place that ensured this happened.
>>>> Since it's a userspace who does the mapping, then it should be a
>>>> responsibility of userspace to do all the necessary syncing.
>>> Sure, but nothing prohibits user-space to ignore the syncing thinking
>>> "It works anyway", testing those drivers where the syncing is a NOP. And
>>> when a driver that finally needs syncing is tested it's too late to fix
>>> all broken user-space.
>>>
>>>>    I'm not
>>>> sure whether anyone in userspace really needs to map imported dma-bufs
>>>> in practice. Nevertheless, this use-case is broken and should be fixed
>>>> by either allowing to do the mapping or prohibiting it.
>>>>
>>> Then I'd vote for prohibiting it, at least for now. And for the future
>>> moving forward we could perhaps revisit the dma-buf need for syncing,
>>> requiring those drivers that actually need it to implement emulated
>>> coherent memory which can be done not too inefficiently (vmwgfx being
>>> one example).
>> Alright, I'll change it to prohibit the mapping. This indeed should be a
>> better option.
> 
> Oh, yes please. But I would expect that some people start screaming.
> 
> Over time I've got tons of TTM patches because people illegally tried to
> mmap() imported DMA-bufs in their driver.
> 
> Anyway this is probably the right thing to do and we can work on fixing
> the fallout later on.

I already sent out the patch [1] that prohibits the mapping. Would be
great if you all could take a look and give a r-b, thanks in advance.

[1] https://patchwork.freedesktop.org/patch/492148/

-- 
Best regards,
Dmitry

Re: [Linaro-mm-sig] Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Dmitry Osipenko]

On 7/4/22 15:33, Christian König wrote:
> Am 30.06.22 um 01:06 schrieb Dmitry Osipenko:
>> On 6/29/22 11:43, Thomas Hellström (Intel) wrote:
>>> On 6/29/22 10:22, Dmitry Osipenko wrote:
>>>> On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
>>>>> On 5/27/22 01:50, Dmitry Osipenko wrote:
>>>>>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>>>>>> handle imported dma-bufs properly, which results in mapping of
>>>>>> something
>>>>>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>>>>>> hard
>>>>>> lockup when userspace writes to the memory mapping of a dma-buf that
>>>>>> was
>>>>>> imported into Tegra's DRM GEM.
>>>>>>
>>>>>> To fix this bug, move mapping of imported dma-bufs to
>>>>>> drm_gem_mmap_obj().
>>>>>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
>>>>> Same comment about Fixes: as in patch 1,
>>>>>> Cc: stable@vger.kernel.org
>>>>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>>>>> ---
>>>>>>     drivers/gpu/drm/drm_gem.c              | 3 +++
>>>>>>     drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>>>>>     drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>>>>>     3 files changed, 7 insertions(+), 9 deletions(-)
>>>>>>
>>>>>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>>>>>> index 86d670c71286..7c0b025508e4 100644
>>>>>> --- a/drivers/gpu/drm/drm_gem.c
>>>>>> +++ b/drivers/gpu/drm/drm_gem.c
>>>>>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object
>>>>>> *obj,
>>>>>> unsigned long obj_size,
>>>>>>         if (obj_size < vma->vm_end - vma->vm_start)
>>>>>>             return -EINVAL;
>>>>>>     +    if (obj->import_attach)
>>>>>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
>>>>> If we start enabling mmaping of imported dma-bufs on a majority of
>>>>> drivers in this way, how do we ensure that user-space is not blindly
>>>>> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
>>>>> which is needed before and after cpu access of mmap'ed dma-bufs?
>>>>>
>>>>> I was under the impression (admittedly without looking) that the few
>>>>> drivers that actually called into dma_buf_mmap() had some private
>>>>> user-mode driver code in place that ensured this happened.
>>>> Since it's a userspace who does the mapping, then it should be a
>>>> responsibility of userspace to do all the necessary syncing.
>>> Sure, but nothing prohibits user-space to ignore the syncing thinking
>>> "It works anyway", testing those drivers where the syncing is a NOP. And
>>> when a driver that finally needs syncing is tested it's too late to fix
>>> all broken user-space.
>>>
>>>>    I'm not
>>>> sure whether anyone in userspace really needs to map imported dma-bufs
>>>> in practice. Nevertheless, this use-case is broken and should be fixed
>>>> by either allowing to do the mapping or prohibiting it.
>>>>
>>> Then I'd vote for prohibiting it, at least for now. And for the future
>>> moving forward we could perhaps revisit the dma-buf need for syncing,
>>> requiring those drivers that actually need it to implement emulated
>>> coherent memory which can be done not too inefficiently (vmwgfx being
>>> one example).
>> Alright, I'll change it to prohibit the mapping. This indeed should be a
>> better option.
> 
> Oh, yes please. But I would expect that some people start screaming.
> 
> Over time I've got tons of TTM patches because people illegally tried to
> mmap() imported DMA-bufs in their driver.
> 
> Anyway this is probably the right thing to do and we can work on fixing
> the fallout later on.

I already sent out the patch [1] that prohibits the mapping. Would be
great if you all could take a look and give a r-b, thanks in advance.

[1] https://patchwork.freedesktop.org/patch/492148/

-- 
Best regards,
Dmitry

Re: [Intel-gfx] [Linaro-mm-sig] Re: [PATCH v6 02/22] drm/gem: Move mapping of imported dma-bufs to drm_gem_mmap_obj()

[Dmitry Osipenko]

On 7/4/22 15:33, Christian König wrote:
> Am 30.06.22 um 01:06 schrieb Dmitry Osipenko:
>> On 6/29/22 11:43, Thomas Hellström (Intel) wrote:
>>> On 6/29/22 10:22, Dmitry Osipenko wrote:
>>>> On 6/29/22 09:40, Thomas Hellström (Intel) wrote:
>>>>> On 5/27/22 01:50, Dmitry Osipenko wrote:
>>>>>> Drivers that use drm_gem_mmap() and drm_gem_mmap_obj() helpers don't
>>>>>> handle imported dma-bufs properly, which results in mapping of
>>>>>> something
>>>>>> else than the imported dma-buf. For example, on NVIDIA Tegra we get a
>>>>>> hard
>>>>>> lockup when userspace writes to the memory mapping of a dma-buf that
>>>>>> was
>>>>>> imported into Tegra's DRM GEM.
>>>>>>
>>>>>> To fix this bug, move mapping of imported dma-bufs to
>>>>>> drm_gem_mmap_obj().
>>>>>> Now mmaping of imported dma-bufs works properly for all DRM drivers.
>>>>> Same comment about Fixes: as in patch 1,
>>>>>> Cc: stable@vger.kernel.org
>>>>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>>>>> ---
>>>>>>     drivers/gpu/drm/drm_gem.c              | 3 +++
>>>>>>     drivers/gpu/drm/drm_gem_shmem_helper.c | 9 ---------
>>>>>>     drivers/gpu/drm/tegra/gem.c            | 4 ++++
>>>>>>     3 files changed, 7 insertions(+), 9 deletions(-)
>>>>>>
>>>>>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>>>>>> index 86d670c71286..7c0b025508e4 100644
>>>>>> --- a/drivers/gpu/drm/drm_gem.c
>>>>>> +++ b/drivers/gpu/drm/drm_gem.c
>>>>>> @@ -1038,6 +1038,9 @@ int drm_gem_mmap_obj(struct drm_gem_object
>>>>>> *obj,
>>>>>> unsigned long obj_size,
>>>>>>         if (obj_size < vma->vm_end - vma->vm_start)
>>>>>>             return -EINVAL;
>>>>>>     +    if (obj->import_attach)
>>>>>> +        return dma_buf_mmap(obj->dma_buf, vma, 0);
>>>>> If we start enabling mmaping of imported dma-bufs on a majority of
>>>>> drivers in this way, how do we ensure that user-space is not blindly
>>>>> using the object mmap without calling the needed DMA_BUF_IOCTL_SYNC
>>>>> which is needed before and after cpu access of mmap'ed dma-bufs?
>>>>>
>>>>> I was under the impression (admittedly without looking) that the few
>>>>> drivers that actually called into dma_buf_mmap() had some private
>>>>> user-mode driver code in place that ensured this happened.
>>>> Since it's a userspace who does the mapping, then it should be a
>>>> responsibility of userspace to do all the necessary syncing.
>>> Sure, but nothing prohibits user-space to ignore the syncing thinking
>>> "It works anyway", testing those drivers where the syncing is a NOP. And
>>> when a driver that finally needs syncing is tested it's too late to fix
>>> all broken user-space.
>>>
>>>>    I'm not
>>>> sure whether anyone in userspace really needs to map imported dma-bufs
>>>> in practice. Nevertheless, this use-case is broken and should be fixed
>>>> by either allowing to do the mapping or prohibiting it.
>>>>
>>> Then I'd vote for prohibiting it, at least for now. And for the future
>>> moving forward we could perhaps revisit the dma-buf need for syncing,
>>> requiring those drivers that actually need it to implement emulated
>>> coherent memory which can be done not too inefficiently (vmwgfx being
>>> one example).
>> Alright, I'll change it to prohibit the mapping. This indeed should be a
>> better option.
> 
> Oh, yes please. But I would expect that some people start screaming.
> 
> Over time I've got tons of TTM patches because people illegally tried to
> mmap() imported DMA-bufs in their driver.
> 
> Anyway this is probably the right thing to do and we can work on fixing
> the fallout later on.

I already sent out the patch [1] that prohibits the mapping. Would be
great if you all could take a look and give a r-b, thanks in advance.

[1] https://patchwork.freedesktop.org/patch/492148/

-- 
Best regards,
Dmitry

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dmitry Osipenko]

On 7/5/22 01:38, Dmitry Osipenko wrote:
...
>>> Also i915 will run into trouble with attach. In particular since i915
>>> starts a full ww transaction in its attach callback to be able to lock
>>> other objects if migration is needed. I think i915 CI would catch this
>>> in a selftest.
>> Seems it indeed it should deadlock. But i915 selftests apparently
>> should've caught it and they didn't, I'll re-check what happened.
>>
> 
> The i915 selftests use a separate mock_dmabuf_ops. That's why it works
> for the selftests, i.e. there is no deadlock.
> 
> Thomas, would i915 CI run a different set of tests or will it be the
> default i915 selftests ran by IGT?
> 

Nevermind, I had a local kernel change that was forgotten about.. it
prevented the i915 live tests from running.

-- 
Best regards,
Dmitry

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dmitry Osipenko]

On 7/5/22 01:38, Dmitry Osipenko wrote:
...
>>> Also i915 will run into trouble with attach. In particular since i915
>>> starts a full ww transaction in its attach callback to be able to lock
>>> other objects if migration is needed. I think i915 CI would catch this
>>> in a selftest.
>> Seems it indeed it should deadlock. But i915 selftests apparently
>> should've caught it and they didn't, I'll re-check what happened.
>>
> 
> The i915 selftests use a separate mock_dmabuf_ops. That's why it works
> for the selftests, i.e. there is no deadlock.
> 
> Thomas, would i915 CI run a different set of tests or will it be the
> default i915 selftests ran by IGT?
> 

Nevermind, I had a local kernel change that was forgotten about.. it
prevented the i915 live tests from running.

-- 
Best regards,
Dmitry

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dmitry Osipenko]

On 7/5/22 01:38, Dmitry Osipenko wrote:
...
>>> Also i915 will run into trouble with attach. In particular since i915
>>> starts a full ww transaction in its attach callback to be able to lock
>>> other objects if migration is needed. I think i915 CI would catch this
>>> in a selftest.
>> Seems it indeed it should deadlock. But i915 selftests apparently
>> should've caught it and they didn't, I'll re-check what happened.
>>
> 
> The i915 selftests use a separate mock_dmabuf_ops. That's why it works
> for the selftests, i.e. there is no deadlock.
> 
> Thomas, would i915 CI run a different set of tests or will it be the
> default i915 selftests ran by IGT?
> 

Nevermind, I had a local kernel change that was forgotten about.. it
prevented the i915 live tests from running.

-- 
Best regards,
Dmitry

Re: [Intel-gfx] [PATCH v6 14/22] dma-buf: Introduce new locking convention

[Dmitry Osipenko]

On 7/5/22 01:38, Dmitry Osipenko wrote:
...
>>> Also i915 will run into trouble with attach. In particular since i915
>>> starts a full ww transaction in its attach callback to be able to lock
>>> other objects if migration is needed. I think i915 CI would catch this
>>> in a selftest.
>> Seems it indeed it should deadlock. But i915 selftests apparently
>> should've caught it and they didn't, I'll re-check what happened.
>>
> 
> The i915 selftests use a separate mock_dmabuf_ops. That's why it works
> for the selftests, i.e. there is no deadlock.
> 
> Thomas, would i915 CI run a different set of tests or will it be the
> default i915 selftests ran by IGT?
> 

Nevermind, I had a local kernel change that was forgotten about.. it
prevented the i915 live tests from running.

-- 
Best regards,
Dmitry

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[kernel test robot]

Hi Dmitry,

I love your patch! Perhaps something to improve:

[auto build test WARNING on linus/master]
[also build test WARNING on next-20220527]
[cannot apply to drm/drm-next media-tree/master drm-intel/for-linux-next v5.18]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/intel-lab-lkp/linux/commits/Dmitry-Osipenko/Add-generic-memory-shrinker-to-VirtIO-GPU-and-Panfrost-DRM-drivers/20220527-075717
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git cdeffe87f790dfd1baa193020411ce9a538446d7
config: arm-randconfig-c002-20220524 (https://download.01.org/0day-ci/archive/20220528/202205280550.MWGs9cj4-lkp@intel.com/config)
compiler: clang version 15.0.0 (https://github.com/llvm/llvm-project 134d7f9a4b97e9035150d970bd9e376043c4577e)
reproduce (this is a W=1 build):
         wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
         chmod +x ~/bin/make.cross
         # install arm cross compiling tool for clang build
         # apt-get install binutils-arm-linux-gnueabi
         # https://github.com/intel-lab-lkp/linux/commit/97f090c47ec995a8cf3bced98526ee3eaa25f10f
         git remote add linux-review https://github.com/intel-lab-lkp/linux
         git fetch --no-tags linux-review Dmitry-Osipenko/Add-generic-memory-shrinker-to-VirtIO-GPU-and-Panfrost-DRM-drivers/20220527-075717
         git checkout 97f090c47ec995a8cf3bced98526ee3eaa25f10f
         # save the config file
         COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=arm clang-analyzer

If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <yujie.liu@intel.com>


clang-analyzer warnings: (new ones prefixed by >>)

 >> drivers/dma-buf/dma-buf.c:1339:3: warning: Undefined or garbage value returned to caller [clang-analyzer-core.uninitialized.UndefReturn]
                    return ret;
                    ^

vim +1339 drivers/dma-buf/dma-buf.c

4c78513e457f72 drivers/base/dma-buf.c    Daniel Vetter   2012-04-24  1327
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1328  static int dma_buf_vmap_locked(struct dma_buf *dmabuf, struct iosys_map *map)
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1329  {
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1330  	struct iosys_map ptr;
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27 @1331  	int ret;
4c78513e457f72 drivers/base/dma-buf.c    Daniel Vetter   2012-04-24  1332
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1333  	dma_resv_assert_held(dmabuf->resv);
4c78513e457f72 drivers/base/dma-buf.c    Daniel Vetter   2012-04-24  1334
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1335  	if (dmabuf->vmapping_counter) {
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1336  		dmabuf->vmapping_counter++;
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1337  		BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1338  		*map = dmabuf->vmap_ptr;
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27 @1339  		return ret;
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1340  	}
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1341
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1342  	BUG_ON(iosys_map_is_set(&dmabuf->vmap_ptr));
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1343
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1344  	ret = dmabuf->ops->vmap(dmabuf, &ptr);
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1345  	if (WARN_ON_ONCE(ret))
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1346  		return ret;
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1347
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1348  	dmabuf->vmap_ptr = ptr;
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1349  	dmabuf->vmapping_counter = 1;
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1350
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1351  	*map = dmabuf->vmap_ptr;
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1352
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1353  	return 0;
4c78513e457f72 drivers/base/dma-buf.c    Daniel Vetter   2012-04-24  1354  }
98f86c9e4ae320 drivers/base/dma-buf.c    Dave Airlie     2012-05-20  1355

-- 
0-DAY CI Kernel Test Service
https://01.org/lkp

Re: [PATCH v6 14/22] dma-buf: Introduce new locking convention

[kernel test robot]

[-- Attachment #1: Type: text/plain, Size: 27608 bytes --]

CC: llvm(a)lists.linux.dev
CC: kbuild-all(a)lists.01.org
BCC: lkp(a)intel.com
In-Reply-To: <20220526235040.678984-15-dmitry.osipenko@collabora.com>
References: <20220526235040.678984-15-dmitry.osipenko@collabora.com>
TO: Dmitry Osipenko <dmitry.osipenko@collabora.com>
TO: David Airlie <airlied@linux.ie>
TO: Gerd Hoffmann <kraxel@redhat.com>
TO: Gurchetan Singh <gurchetansingh@chromium.org>
TO: "Chia-I Wu" <olvaffe@gmail.com>
TO: Daniel Vetter <daniel@ffwll.ch>
TO: Daniel Almeida <daniel.almeida@collabora.com>
TO: Gert Wollny <gert.wollny@collabora.com>
TO: Gustavo Padovan <gustavo.padovan@collabora.com>
TO: Daniel Stone <daniel@fooishbar.org>
TO: Tomeu Vizoso <tomeu.vizoso@collabora.com>
TO: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
TO: Maxime Ripard <mripard@kernel.org>
TO: Thomas Zimmermann <tzimmermann@suse.de>
TO: Rob Herring <robh@kernel.org>
TO: Steven Price <steven.price@arm.com>
TO: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
TO: Rob Clark <robdclark@gmail.com>
TO: Emil Velikov <emil.l.velikov@gmail.com>
TO: Robin Murphy <robin.murphy@arm.com>
TO: Qiang Yu <yuq825@gmail.com>
TO: Sumit Semwal <sumit.semwal@linaro.org>
TO: "Christian König" <christian.koenig@amd.com>
TO: "Pan, Xinhui" <Xinhui.Pan@amd.com>
TO: Thierry Reding <thierry.reding@gmail.com>
TO: Tomasz Figa <tfiga@chromium.org>
TO: Marek Szyprowski <m.szyprowski@samsung.com>
TO: Mauro Carvalho Chehab <mchehab@kernel.org>
CC: linux-media(a)vger.kernel.org
TO: Alex Deucher <alexander.deucher@amd.com>
TO: Jani Nikula <jani.nikula@linux.intel.com>

Hi Dmitry,

I love your patch! Perhaps something to improve:

[auto build test WARNING on linus/master]
[also build test WARNING on next-20220527]
[cannot apply to drm/drm-next media-tree/master drm-intel/for-linux-next v5.18]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/intel-lab-lkp/linux/commits/Dmitry-Osipenko/Add-generic-memory-shrinker-to-VirtIO-GPU-and-Panfrost-DRM-drivers/20220527-075717
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git cdeffe87f790dfd1baa193020411ce9a538446d7
:::::: branch date: 22 hours ago
:::::: commit date: 22 hours ago
config: arm-randconfig-c002-20220524 (https://download.01.org/0day-ci/archive/20220528/202205280550.MWGs9cj4-lkp(a)intel.com/config)
compiler: clang version 15.0.0 (https://github.com/llvm/llvm-project 134d7f9a4b97e9035150d970bd9e376043c4577e)
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # install arm cross compiling tool for clang build
        # apt-get install binutils-arm-linux-gnueabi
        # https://github.com/intel-lab-lkp/linux/commit/97f090c47ec995a8cf3bced98526ee3eaa25f10f
        git remote add linux-review https://github.com/intel-lab-lkp/linux
        git fetch --no-tags linux-review Dmitry-Osipenko/Add-generic-memory-shrinker-to-VirtIO-GPU-and-Panfrost-DRM-drivers/20220527-075717
        git checkout 97f090c47ec995a8cf3bced98526ee3eaa25f10f
        # save the config file
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=arm clang-analyzer 

If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <lkp@intel.com>


clang-analyzer warnings: (new ones prefixed by >>)
                  ^~~~~~~
   drivers/thermal/thermal_sysfs.c:602:9: warning: Call to function 'sprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'sprintf_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
           return sprintf(buf, "%ld\n", state);
                  ^~~~~~~
   drivers/thermal/thermal_sysfs.c:602:9: note: Call to function 'sprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'sprintf_s' in case of C11
           return sprintf(buf, "%ld\n", state);
                  ^~~~~~~
   drivers/thermal/thermal_sysfs.c:613:6: warning: Call to function 'sscanf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'sscanf_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
           if (sscanf(buf, "%ld\n", &state) != 1)
               ^~~~~~
   drivers/thermal/thermal_sysfs.c:613:6: note: Call to function 'sscanf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'sscanf_s' in case of C11
           if (sscanf(buf, "%ld\n", &state) != 1)
               ^~~~~~
   drivers/thermal/thermal_sysfs.c:702:8: warning: Call to function 'sprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'sprintf_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
           ret = sprintf(buf, "%u\n", stats->total_trans);
                 ^~~~~~~
   drivers/thermal/thermal_sysfs.c:702:8: note: Call to function 'sprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'sprintf_s' in case of C11
           ret = sprintf(buf, "%u\n", stats->total_trans);
                 ^~~~~~~
   drivers/thermal/thermal_sysfs.c:721:10: warning: Call to function 'sprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'sprintf_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
                   len += sprintf(buf + len, "state%u\t%llu\n", i,
                          ^~~~~~~
   drivers/thermal/thermal_sysfs.c:721:10: note: Call to function 'sprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'sprintf_s' in case of C11
                   len += sprintf(buf + len, "state%u\t%llu\n", i,
                          ^~~~~~~
   drivers/thermal/thermal_sysfs.c:741:2: warning: Call to function 'memset' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'memset_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
           memset(stats->trans_table, 0,
           ^~~~~~
   drivers/thermal/thermal_sysfs.c:741:2: note: Call to function 'memset' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'memset_s' in case of C11
           memset(stats->trans_table, 0,
           ^~~~~~
   drivers/thermal/thermal_sysfs.c:760:9: warning: Call to function 'snprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'snprintf_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
           len += snprintf(buf + len, PAGE_SIZE - len, " From  :    To\n");
                  ^~~~~~~~
   drivers/thermal/thermal_sysfs.c:760:9: note: Call to function 'snprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'snprintf_s' in case of C11
           len += snprintf(buf + len, PAGE_SIZE - len, " From  :    To\n");
                  ^~~~~~~~
   drivers/thermal/thermal_sysfs.c:761:9: warning: Call to function 'snprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'snprintf_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
           len += snprintf(buf + len, PAGE_SIZE - len, "       : ");
                  ^~~~~~~~
   drivers/thermal/thermal_sysfs.c:761:9: note: Call to function 'snprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'snprintf_s' in case of C11
           len += snprintf(buf + len, PAGE_SIZE - len, "       : ");
                  ^~~~~~~~
   drivers/thermal/thermal_sysfs.c:765:10: warning: Call to function 'snprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'snprintf_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
                   len += snprintf(buf + len, PAGE_SIZE - len, "state%2u  ", i);
                          ^~~~~~~~
   drivers/thermal/thermal_sysfs.c:765:10: note: Call to function 'snprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'snprintf_s' in case of C11
                   len += snprintf(buf + len, PAGE_SIZE - len, "state%2u  ", i);
                          ^~~~~~~~
   drivers/thermal/thermal_sysfs.c:770:9: warning: Call to function 'snprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'snprintf_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
           len += snprintf(buf + len, PAGE_SIZE - len, "\n");
                  ^~~~~~~~
   drivers/thermal/thermal_sysfs.c:770:9: note: Call to function 'snprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'snprintf_s' in case of C11
           len += snprintf(buf + len, PAGE_SIZE - len, "\n");
                  ^~~~~~~~
   drivers/thermal/thermal_sysfs.c:776:10: warning: Call to function 'snprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'snprintf_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
                   len += snprintf(buf + len, PAGE_SIZE - len, "state%2u:", i);
                          ^~~~~~~~
   drivers/thermal/thermal_sysfs.c:776:10: note: Call to function 'snprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'snprintf_s' in case of C11
                   len += snprintf(buf + len, PAGE_SIZE - len, "state%2u:", i);
                          ^~~~~~~~
   drivers/thermal/thermal_sysfs.c:781:11: warning: Call to function 'snprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'snprintf_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
                           len += snprintf(buf + len, PAGE_SIZE - len, "%8u ",
                                  ^~~~~~~~
   drivers/thermal/thermal_sysfs.c:781:11: note: Call to function 'snprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'snprintf_s' in case of C11
                           len += snprintf(buf + len, PAGE_SIZE - len, "%8u ",
                                  ^~~~~~~~
   drivers/thermal/thermal_sysfs.c:786:10: warning: Call to function 'snprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'snprintf_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
                   len += snprintf(buf + len, PAGE_SIZE - len, "\n");
                          ^~~~~~~~
   drivers/thermal/thermal_sysfs.c:786:10: note: Call to function 'snprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'snprintf_s' in case of C11
                   len += snprintf(buf + len, PAGE_SIZE - len, "\n");
                          ^~~~~~~~
   drivers/thermal/thermal_sysfs.c:881:9: warning: Call to function 'sprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'sprintf_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
           return sprintf(buf, "%d\n", instance->trip);
                  ^~~~~~~
   drivers/thermal/thermal_sysfs.c:881:9: note: Call to function 'sprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'sprintf_s' in case of C11
           return sprintf(buf, "%d\n", instance->trip);
                  ^~~~~~~
   drivers/thermal/thermal_sysfs.c:891:9: warning: Call to function 'sprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'sprintf_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
           return sprintf(buf, "%d\n", instance->weight);
                  ^~~~~~~
   drivers/thermal/thermal_sysfs.c:891:9: note: Call to function 'sprintf' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'sprintf_s' in case of C11
           return sprintf(buf, "%d\n", instance->weight);
                  ^~~~~~~
   Suppressed 16 warnings (16 in non-user code).
   Use -header-filter=.* to display errors from all non-system headers. Use -system-headers to display errors from system headers as well.
   7 warnings generated.
   Suppressed 7 warnings (7 in non-user code).
   Use -header-filter=.* to display errors from all non-system headers. Use -system-headers to display errors from system headers as well.
   8 warnings generated.
   lib/xarray.c:2035:18: warning: Value stored to 'node' during its initialization is never read [clang-analyzer-deadcode.DeadStores]
           struct xa_node *node = xas->xa_node;
                           ^~~~   ~~~~~~~~~~~~
   lib/xarray.c:2035:18: note: Value stored to 'node' during its initialization is never read
           struct xa_node *node = xas->xa_node;
                           ^~~~   ~~~~~~~~~~~~
   Suppressed 7 warnings (7 in non-user code).
   Use -header-filter=.* to display errors from all non-system headers. Use -system-headers to display errors from system headers as well.
   36 warnings generated.
>> drivers/dma-buf/dma-buf.c:1339:3: warning: Undefined or garbage value returned to caller [clang-analyzer-core.uninitialized.UndefReturn]
                   return ret;
                   ^
   drivers/dma-buf/dma-buf.c:1378:14: note: Assuming 'dmabuf' is non-null
           if (WARN_ON(!dmabuf))
                       ^
   include/asm-generic/bug.h:122:25: note: expanded from macro 'WARN_ON'
           int __ret_warn_on = !!(condition);                              \
                                  ^~~~~~~~~
   drivers/dma-buf/dma-buf.c:1378:6: note: Taking false branch
           if (WARN_ON(!dmabuf))
               ^
   include/asm-generic/bug.h:123:2: note: expanded from macro 'WARN_ON'
           if (unlikely(__ret_warn_on))                                    \
           ^
   drivers/dma-buf/dma-buf.c:1378:2: note: Taking false branch
           if (WARN_ON(!dmabuf))
           ^
   drivers/dma-buf/dma-buf.c:1381:6: note: Assuming field 'vmap' is non-null
           if (!dmabuf->ops->vmap)
               ^~~~~~~~~~~~~~~~~~
   drivers/dma-buf/dma-buf.c:1381:2: note: Taking false branch
           if (!dmabuf->ops->vmap)
           ^
   drivers/dma-buf/dma-buf.c:1385:8: note: Calling 'dma_buf_vmap_locked'
           ret = dma_buf_vmap_locked(dmabuf, map);
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   drivers/dma-buf/dma-buf.c:1331:2: note: 'ret' declared without an initial value
           int ret;
           ^~~~~~~
   drivers/dma-buf/dma-buf.c:1333:2: note: Loop condition is false.  Exiting loop
           dma_resv_assert_held(dmabuf->resv);
           ^
   include/linux/dma-resv.h:302:35: note: expanded from macro 'dma_resv_assert_held'
   #define dma_resv_assert_held(obj) lockdep_assert_held(&(obj)->lock.base)
                                     ^
   include/linux/lockdep.h:411:34: note: expanded from macro 'lockdep_assert_held'
   #define lockdep_assert_held(l)                  do { (void)(l); } while (0)
                                                   ^
   drivers/dma-buf/dma-buf.c:1335:6: note: Assuming field 'vmapping_counter' is not equal to 0
           if (dmabuf->vmapping_counter) {
               ^~~~~~~~~~~~~~~~~~~~~~~~
   drivers/dma-buf/dma-buf.c:1335:2: note: Taking true branch
           if (dmabuf->vmapping_counter) {
           ^
   drivers/dma-buf/dma-buf.c:1337:3: note: Taking false branch
                   BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
                   ^
   include/asm-generic/bug.h:71:32: note: expanded from macro 'BUG_ON'
   #define BUG_ON(condition) do { if (unlikely(condition)) BUG(); } while (0)
                                  ^
   drivers/dma-buf/dma-buf.c:1337:3: note: Loop condition is false.  Exiting loop
                   BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
                   ^
   include/asm-generic/bug.h:71:27: note: expanded from macro 'BUG_ON'
   #define BUG_ON(condition) do { if (unlikely(condition)) BUG(); } while (0)
                             ^
   drivers/dma-buf/dma-buf.c:1339:3: note: Undefined or garbage value returned to caller
                   return ret;
                   ^      ~~~
   Suppressed 35 warnings (34 in non-user code, 1 with check filters).
   Use -header-filter=.* to display errors from all non-system headers. Use -system-headers to display errors from system headers as well.
   45 warnings generated.
   fs/xfs/libxfs/xfs_refcount_btree.c:66:2: warning: Call to function 'memset' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'memset_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
           memset(&args, 0, sizeof(args));
           ^~~~~~
   fs/xfs/libxfs/xfs_refcount_btree.c:66:2: note: Call to function 'memset' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'memset_s' in case of C11
           memset(&args, 0, sizeof(args));
           ^~~~~~
   Suppressed 44 warnings (44 in non-user code).
   Use -header-filter=.* to display errors from all non-system headers. Use -system-headers to display errors from system headers as well.
   52 warnings generated.
   fs/xfs/libxfs/xfs_sb.c:559:2: warning: Call to function 'memcpy' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'memcpy_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
           memcpy(&to->sb_uuid, &from->sb_uuid, sizeof(to->sb_uuid));
           ^~~~~~
   fs/xfs/libxfs/xfs_sb.c:559:2: note: Call to function 'memcpy' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'memcpy_s' in case of C11
           memcpy(&to->sb_uuid, &from->sb_uuid, sizeof(to->sb_uuid));
           ^~~~~~
   fs/xfs/libxfs/xfs_sb.c:573:2: warning: Call to function 'memcpy' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'memcpy_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
           memcpy(&to->sb_fname, &from->sb_fname, sizeof(to->sb_fname));
           ^~~~~~
   fs/xfs/libxfs/xfs_sb.c:573:2: note: Call to function 'memcpy' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'memcpy_s' in case of C11
           memcpy(&to->sb_fname, &from->sb_fname, sizeof(to->sb_fname));
           ^~~~~~
   fs/xfs/libxfs/xfs_sb.c:708:2: warning: Call to function 'memcpy' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'memcpy_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
           memcpy(&to->sb_uuid, &from->sb_uuid, sizeof(to->sb_uuid));
           ^~~~~~
   fs/xfs/libxfs/xfs_sb.c:708:2: note: Call to function 'memcpy' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'memcpy_s' in case of C11
           memcpy(&to->sb_uuid, &from->sb_uuid, sizeof(to->sb_uuid));
           ^~~~~~
   fs/xfs/libxfs/xfs_sb.c:722:2: warning: Call to function 'memcpy' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'memcpy_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
           memcpy(&to->sb_fname, &from->sb_fname, sizeof(to->sb_fname));
           ^~~~~~
   fs/xfs/libxfs/xfs_sb.c:722:2: note: Call to function 'memcpy' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'memcpy_s' in case of C11
           memcpy(&to->sb_fname, &from->sb_fname, sizeof(to->sb_fname));
           ^~~~~~
   fs/xfs/libxfs/xfs_sb.c:1128:2: warning: Call to function 'memset' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'memset_s' in case of C11 [clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling]
           memset(geo, 0, sizeof(struct xfs_fsop_geom));
           ^~~~~~
   fs/xfs/libxfs/xfs_sb.c:1128:2: note: Call to function 'memset' is insecure as it does not provide security checks introduced in the C11 standard. Replace with analogous functions that support length arguments or provides boundary checks such as 'memset_s' in case of C11
           memset(geo, 0, sizeof(struct xfs_fsop_geom));

vim +1339 drivers/dma-buf/dma-buf.c

4c78513e457f72 drivers/base/dma-buf.c    Daniel Vetter   2012-04-24  1327  
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1328  static int dma_buf_vmap_locked(struct dma_buf *dmabuf, struct iosys_map *map)
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1329  {
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1330  	struct iosys_map ptr;
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1331  	int ret;
4c78513e457f72 drivers/base/dma-buf.c    Daniel Vetter   2012-04-24  1332  
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1333  	dma_resv_assert_held(dmabuf->resv);
4c78513e457f72 drivers/base/dma-buf.c    Daniel Vetter   2012-04-24  1334  
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1335  	if (dmabuf->vmapping_counter) {
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1336  		dmabuf->vmapping_counter++;
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1337  		BUG_ON(iosys_map_is_null(&dmabuf->vmap_ptr));
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1338  		*map = dmabuf->vmap_ptr;
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27 @1339  		return ret;
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1340  	}
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1341  
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1342  	BUG_ON(iosys_map_is_set(&dmabuf->vmap_ptr));
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1343  
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1344  	ret = dmabuf->ops->vmap(dmabuf, &ptr);
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1345  	if (WARN_ON_ONCE(ret))
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1346  		return ret;
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1347  
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1348  	dmabuf->vmap_ptr = ptr;
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1349  	dmabuf->vmapping_counter = 1;
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1350  
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1351  	*map = dmabuf->vmap_ptr;
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1352  
97f090c47ec995 drivers/dma-buf/dma-buf.c Dmitry Osipenko 2022-05-27  1353  	return 0;
4c78513e457f72 drivers/base/dma-buf.c    Daniel Vetter   2012-04-24  1354  }
98f86c9e4ae320 drivers/base/dma-buf.c    Dave Airlie     2012-05-20  1355  

-- 
0-DAY CI Kernel Test Service
https://01.org/lkp