From: Felix Kuehling <Felix.Kuehling@amd.com> To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH 00/44] Add HMM-based SVM memory manager to KFD v2 Date: Mon, 22 Mar 2021 06:58:16 -0400 [thread overview] Message-ID: <20210322105900.14068-1-Felix.Kuehling@amd.com> (raw) Since the last patch series I sent on Jan 6 a lot has changed. Patches 1-33 are the cleaned up, rebased on amd-staging-drm-next 5.11 version from about a week ago. The remaining 11 patches are current work-in-progress with further cleanup and fixes. MMU notifiers and CPU page faults now can split ranges and update our range data structures without taking heavy locks by doing some of the critical work in a deferred work handler. This includes updating MMU notifiers and the SVM range interval tree. In the mean time, new ranges can live as children of their parent ranges until the deferred work handler consolidates them in the main interval tree. We also added proper DMA mapping of system memory pages. Current work in progress is cleaning up all the locking, simplifying our code and data structures and resolving a few known bugs. This series and the corresponding ROCm Thunk and KFDTest changes are also available on gitub: https://github.com/RadeonOpenCompute/ROCK-Kernel-Driver/tree/fxkamd/hmm-wip https://github.com/RadeonOpenCompute/ROCT-Thunk-Interface/tree/fxkamd/hmm-wip An updated Thunk Alex Sierra (10): drm/amdgpu: replace per_device_list by array drm/amdkfd: helper to convert gpu id and idx drm/amdkfd: add xnack enabled flag to kfd_process drm/amdkfd: add ioctl to configure and query xnack retries drm/amdgpu: enable 48-bit IH timestamp counter drm/amdkfd: SVM API call to restore page tables drm/amdkfd: add svm_bo reference for eviction fence drm/amdgpu: add param bit flag to create SVM BOs drm/amdgpu: svm bo enable_signal call condition drm/amdgpu: add svm_bo eviction to enable_signal cb Felix Kuehling (22): drm/amdkfd: map svm range to GPUs drm/amdkfd: svm range eviction and restore drm/amdkfd: validate vram svm range from TTM drm/amdkfd: HMM migrate ram to vram drm/amdkfd: HMM migrate vram to ram drm/amdkfd: invalidate tables on page retry fault drm/amdkfd: page table restore through svm API drm/amdkfd: add svm_bo eviction mechanism support drm/amdkfd: refine migration policy with xnack on drm/amdkfd: add svm range validate timestamp drm/amdkfd: multiple gpu migrate vram to vram drm/amdkfd: Fix dma unmapping drm/amdkfd: Call mutex_destroy drm/amdkfd: Fix spurious restore failures drm/amdkfd: Fix svm_bo_list locking in eviction worker drm/amdkfd: Simplify split_by_granularity drm/amdkfd: Point out several race conditions drm/amdkfd: Return pdd from kfd_process_device_from_gduid drm/amdkfd: Remove broken deferred mapping drm/amdkfd: Allow invalid pages in migration.src drm/amdkfd: Correct locking during migration and mapping drm/amdkfd: Nested locking and invalidation of child ranges Philip Yang (12): drm/amdkfd: add svm ioctl API drm/amdkfd: register svm range drm/amdkfd: add svm ioctl GET_ATTR op drm/amdgpu: add common HMM get pages function drm/amdkfd: validate svm range system memory drm/amdkfd: deregister svm range drm/amdgpu: export vm update mapping interface drm/amdkfd: register HMM device private zone drm/amdkfd: support xgmi same hive mapping drm/amdkfd: copy memory through gart table drm/amdgpu: reserve fence slot to update page table drm/amdkfd: Add SVM API support capability bits drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 4 + drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 4 +- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_fence.c | 16 +- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 13 +- drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 83 + drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h | 7 + drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 4 + drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 90 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 48 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 11 + drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 1 + drivers/gpu/drm/amd/amdkfd/Kconfig | 1 + drivers/gpu/drm/amd/amdkfd/Makefile | 4 +- drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 173 +- drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c | 4 + drivers/gpu/drm/amd/amdkfd/kfd_iommu.c | 8 +- drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 922 ++++++ drivers/gpu/drm/amd/amdkfd/kfd_migrate.h | 59 + drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 54 +- drivers/gpu/drm/amd/amdkfd/kfd_process.c | 191 +- .../amd/amdkfd/kfd_process_queue_manager.c | 6 +- drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 2865 +++++++++++++++++ drivers/gpu/drm/amd/amdkfd/kfd_svm.h | 175 + drivers/gpu/drm/amd/amdkfd/kfd_topology.c | 6 + drivers/gpu/drm/amd/amdkfd/kfd_topology.h | 10 +- include/uapi/linux/kfd_ioctl.h | 171 +- 26 files changed, 4681 insertions(+), 249 deletions(-) create mode 100644 drivers/gpu/drm/amd/amdkfd/kfd_migrate.c create mode 100644 drivers/gpu/drm/amd/amdkfd/kfd_migrate.h create mode 100644 drivers/gpu/drm/amd/amdkfd/kfd_svm.c create mode 100644 drivers/gpu/drm/amd/amdkfd/kfd_svm.h -- 2.31.0 _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
WARNING: multiple messages have this Message-ID (diff)
From: Felix Kuehling <Felix.Kuehling@amd.com> To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH 00/44] Add HMM-based SVM memory manager to KFD v2 Date: Mon, 22 Mar 2021 06:58:16 -0400 [thread overview] Message-ID: <20210322105900.14068-1-Felix.Kuehling@amd.com> (raw) Since the last patch series I sent on Jan 6 a lot has changed. Patches 1-33 are the cleaned up, rebased on amd-staging-drm-next 5.11 version from about a week ago. The remaining 11 patches are current work-in-progress with further cleanup and fixes. MMU notifiers and CPU page faults now can split ranges and update our range data structures without taking heavy locks by doing some of the critical work in a deferred work handler. This includes updating MMU notifiers and the SVM range interval tree. In the mean time, new ranges can live as children of their parent ranges until the deferred work handler consolidates them in the main interval tree. We also added proper DMA mapping of system memory pages. Current work in progress is cleaning up all the locking, simplifying our code and data structures and resolving a few known bugs. This series and the corresponding ROCm Thunk and KFDTest changes are also available on gitub: https://github.com/RadeonOpenCompute/ROCK-Kernel-Driver/tree/fxkamd/hmm-wip https://github.com/RadeonOpenCompute/ROCT-Thunk-Interface/tree/fxkamd/hmm-wip An updated Thunk Alex Sierra (10): drm/amdgpu: replace per_device_list by array drm/amdkfd: helper to convert gpu id and idx drm/amdkfd: add xnack enabled flag to kfd_process drm/amdkfd: add ioctl to configure and query xnack retries drm/amdgpu: enable 48-bit IH timestamp counter drm/amdkfd: SVM API call to restore page tables drm/amdkfd: add svm_bo reference for eviction fence drm/amdgpu: add param bit flag to create SVM BOs drm/amdgpu: svm bo enable_signal call condition drm/amdgpu: add svm_bo eviction to enable_signal cb Felix Kuehling (22): drm/amdkfd: map svm range to GPUs drm/amdkfd: svm range eviction and restore drm/amdkfd: validate vram svm range from TTM drm/amdkfd: HMM migrate ram to vram drm/amdkfd: HMM migrate vram to ram drm/amdkfd: invalidate tables on page retry fault drm/amdkfd: page table restore through svm API drm/amdkfd: add svm_bo eviction mechanism support drm/amdkfd: refine migration policy with xnack on drm/amdkfd: add svm range validate timestamp drm/amdkfd: multiple gpu migrate vram to vram drm/amdkfd: Fix dma unmapping drm/amdkfd: Call mutex_destroy drm/amdkfd: Fix spurious restore failures drm/amdkfd: Fix svm_bo_list locking in eviction worker drm/amdkfd: Simplify split_by_granularity drm/amdkfd: Point out several race conditions drm/amdkfd: Return pdd from kfd_process_device_from_gduid drm/amdkfd: Remove broken deferred mapping drm/amdkfd: Allow invalid pages in migration.src drm/amdkfd: Correct locking during migration and mapping drm/amdkfd: Nested locking and invalidation of child ranges Philip Yang (12): drm/amdkfd: add svm ioctl API drm/amdkfd: register svm range drm/amdkfd: add svm ioctl GET_ATTR op drm/amdgpu: add common HMM get pages function drm/amdkfd: validate svm range system memory drm/amdkfd: deregister svm range drm/amdgpu: export vm update mapping interface drm/amdkfd: register HMM device private zone drm/amdkfd: support xgmi same hive mapping drm/amdkfd: copy memory through gart table drm/amdgpu: reserve fence slot to update page table drm/amdkfd: Add SVM API support capability bits drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 4 + drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 4 +- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_fence.c | 16 +- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 13 +- drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 83 + drivers/gpu/drm/amd/amdgpu/amdgpu_mn.h | 7 + drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 4 + drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 90 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 48 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 11 + drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 1 + drivers/gpu/drm/amd/amdkfd/Kconfig | 1 + drivers/gpu/drm/amd/amdkfd/Makefile | 4 +- drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 173 +- drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c | 4 + drivers/gpu/drm/amd/amdkfd/kfd_iommu.c | 8 +- drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 922 ++++++ drivers/gpu/drm/amd/amdkfd/kfd_migrate.h | 59 + drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 54 +- drivers/gpu/drm/amd/amdkfd/kfd_process.c | 191 +- .../amd/amdkfd/kfd_process_queue_manager.c | 6 +- drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 2865 +++++++++++++++++ drivers/gpu/drm/amd/amdkfd/kfd_svm.h | 175 + drivers/gpu/drm/amd/amdkfd/kfd_topology.c | 6 + drivers/gpu/drm/amd/amdkfd/kfd_topology.h | 10 +- include/uapi/linux/kfd_ioctl.h | 171 +- 26 files changed, 4681 insertions(+), 249 deletions(-) create mode 100644 drivers/gpu/drm/amd/amdkfd/kfd_migrate.c create mode 100644 drivers/gpu/drm/amd/amdkfd/kfd_migrate.h create mode 100644 drivers/gpu/drm/amd/amdkfd/kfd_svm.c create mode 100644 drivers/gpu/drm/amd/amdkfd/kfd_svm.h -- 2.31.0 _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
next reply other threads:[~2021-03-22 11:07 UTC|newest] Thread overview: 100+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-03-22 10:58 Felix Kuehling [this message] 2021-03-22 10:58 ` [PATCH 00/44] Add HMM-based SVM memory manager to KFD v2 Felix Kuehling 2021-03-22 10:58 ` [PATCH 01/44] drm/amdgpu: replace per_device_list by array Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-23 14:30 ` Kim, Jonathan 2021-03-23 14:30 ` Kim, Jonathan 2021-03-22 10:58 ` [PATCH 02/44] drm/amdkfd: helper to convert gpu id and idx Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 03/44] drm/amdkfd: add svm ioctl API Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 04/44] drm/amdkfd: register svm range Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 05/44] drm/amdkfd: add svm ioctl GET_ATTR op Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 06/44] drm/amdgpu: add common HMM get pages function Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 07/44] drm/amdkfd: validate svm range system memory Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 08/44] drm/amdkfd: deregister svm range Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 09/44] drm/amdgpu: export vm update mapping interface Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 10/44] drm/amdkfd: map svm range to GPUs Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 11/44] drm/amdkfd: svm range eviction and restore Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 12/44] drm/amdkfd: add xnack enabled flag to kfd_process Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 13/44] drm/amdkfd: add ioctl to configure and query xnack retries Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 14/44] drm/amdkfd: register HMM device private zone Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 15/44] drm/amdkfd: validate vram svm range from TTM Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 16/44] drm/amdkfd: support xgmi same hive mapping Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 17/44] drm/amdkfd: copy memory through gart table Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 18/44] drm/amdkfd: HMM migrate ram to vram Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 19/44] drm/amdkfd: HMM migrate vram to ram Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 20/44] drm/amdkfd: invalidate tables on page retry fault Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 21/44] drm/amdgpu: enable 48-bit IH timestamp counter Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 22/44] drm/amdkfd: page table restore through svm API Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 23/44] drm/amdkfd: SVM API call to restore page tables Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 24/44] drm/amdkfd: add svm_bo reference for eviction fence Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 25/44] drm/amdgpu: add param bit flag to create SVM BOs Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 26/44] drm/amdkfd: add svm_bo eviction mechanism support Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 27/44] drm/amdgpu: svm bo enable_signal call condition Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 28/44] drm/amdgpu: add svm_bo eviction to enable_signal cb Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 29/44] drm/amdgpu: reserve fence slot to update page table Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 13:32 ` Christian König 2021-03-22 13:32 ` Christian König 2021-03-22 10:58 ` [PATCH 30/44] drm/amdkfd: refine migration policy with xnack on Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 31/44] drm/amdkfd: add svm range validate timestamp Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 32/44] drm/amdkfd: multiple gpu migrate vram to vram Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 33/44] drm/amdkfd: Add SVM API support capability bits Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 34/44] drm/amdkfd: Fix dma unmapping Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 35/44] drm/amdkfd: Call mutex_destroy Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 36/44] drm/amdkfd: Fix spurious restore failures Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 37/44] drm/amdkfd: Fix svm_bo_list locking in eviction worker Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 38/44] drm/amdkfd: Simplify split_by_granularity Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 39/44] drm/amdkfd: Point out several race conditions Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 40/44] drm/amdkfd: Return pdd from kfd_process_device_from_gduid Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 41/44] drm/amdkfd: Remove broken deferred mapping Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 42/44] drm/amdkfd: Allow invalid pages in migration.src Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 43/44] drm/amdkfd: Correct locking during migration and mapping Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:59 ` [PATCH 44/44] drm/amdkfd: Nested locking and invalidation of child ranges Felix Kuehling 2021-03-22 10:59 ` Felix Kuehling 2021-03-22 14:15 ` [PATCH 00/44] Add HMM-based SVM memory manager to KFD v2 Daniel Vetter 2021-03-22 14:15 ` Daniel Vetter 2021-03-22 16:06 ` Felix Kuehling 2021-03-22 16:06 ` Felix Kuehling 2021-03-22 17:04 ` Daniel Vetter 2021-03-22 17:04 ` Daniel Vetter
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20210322105900.14068-1-Felix.Kuehling@amd.com \ --to=felix.kuehling@amd.com \ --cc=amd-gfx@lists.freedesktop.org \ --cc=dri-devel@lists.freedesktop.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.