[PATCH v1 00/12] MEMORY_DEVICE_COHERENT for CPU-accessible coherent device memory

[Alex Sierra <alex.sierra@amd.com>]

This patch series introduces MEMORY_DEVICE_COHERENT, a type of memory
owned by a device that can be mapped into CPU page tables like
MEMORY_DEVICE_GENERIC and can also be migrated like MEMORY_DEVICE_PRIVATE.
With MEMORY_DEVICE_COHERENT, we isolate the new memory type from other
subsystems as far as possible, though there are some small changes to
other subsystems such as filesystem DAX, to handle the new memory type
appropriately.

We use ZONE_DEVICE for this instead of NUMA so that the amdgpu
allocator can manage it without conflicting with core mm for non-unified
memory use cases.

How it works: The system BIOS advertises the GPU device memory (aka VRAM)
as SPM (special purpose memory) in the UEFI system address map.
The amdgpu driver registers the memory with devmap as
MEMORY_DEVICE_COHERENT using devm_memremap_pages.

The initial user for this hardware page migration capability will be
the Frontier supercomputer project. Our nodes in the lab have .5 TB of
system memory plus 256 GB of device memory split across 4 GPUs, all in
the same coherent address space. Page migration is expected to improve
application efficiency significantly. We will report empirical results
as they become available.

This includes patches originally by Ralph Campbell to change ZONE_DEVICE
reference counting as requested in previous reviews of this patch series
(see https://patchwork.freedesktop.org/series/90706/). We extended
hmm_test to cover migration of MEMORY_DEVICE_COHERENT. This patch set
builds on HMM and our SVM memory manager already merged in 5.14.
We would like to complete review and merge this migration patchset for
5.16.

Alex Sierra (10):
  mm: add zone device coherent type memory support
  mm: add device coherent vma selection for memory migration
  drm/amdkfd: ref count init for device pages
  drm/amdkfd: add SPM support for SVM
  drm/amdkfd: coherent type as sys mem on migration to ram
  lib: test_hmm add ioctl to get zone device type
  lib: test_hmm add module param for zone device type
  lib: add support for device coherent type in test_hmm
  tools: update hmm-test to support device coherent type
  tools: update test_hmm script to support SP config

Ralph Campbell (2):
  ext4/xfs: add page refcount helper
  mm: remove extra ZONE_DEVICE struct page refcount

 arch/powerpc/kvm/book3s_hv_uvmem.c       |   2 +-
 drivers/gpu/drm/amd/amdkfd/kfd_migrate.c |  40 ++--
 drivers/gpu/drm/nouveau/nouveau_dmem.c   |   2 +-
 fs/dax.c                                 |   8 +-
 fs/ext4/inode.c                          |   5 +-
 fs/fuse/dax.c                            |   4 +-
 fs/xfs/xfs_file.c                        |   4 +-
 include/linux/dax.h                      |  10 +
 include/linux/memremap.h                 |  15 +-
 include/linux/migrate.h                  |   1 +
 include/linux/mm.h                       |  19 +-
 lib/test_hmm.c                           | 276 +++++++++++++++++------
 lib/test_hmm_uapi.h                      |  20 +-
 mm/internal.h                            |   8 +
 mm/memcontrol.c                          |  12 +-
 mm/memory-failure.c                      |   6 +-
 mm/memremap.c                            |  71 ++----
 mm/migrate.c                             |  33 +--
 mm/page_alloc.c                          |   3 +
 mm/swap.c                                |  45 +---
 tools/testing/selftests/vm/hmm-tests.c   | 137 +++++++++--
 tools/testing/selftests/vm/test_hmm.sh   |  20 +-
 22 files changed, 490 insertions(+), 251 deletions(-)

-- 
2.32.0