From: Christoph Hellwig <hch@lst.de>
To: Andrew Morton <akpm@linux-foundation.org>,
Dan Williams <dan.j.williams@intel.com>
Cc: "Felix Kuehling" <Felix.Kuehling@amd.com>,
"Alex Deucher" <alexander.deucher@amd.com>,
"Christian König" <christian.koenig@amd.com>,
"Pan, Xinhui" <Xinhui.Pan@amd.com>,
"Ben Skeggs" <bskeggs@redhat.com>,
"Karol Herbst" <kherbst@redhat.com>,
"Lyude Paul" <lyude@redhat.com>, "Jason Gunthorpe" <jgg@ziepe.ca>,
"Alistair Popple" <apopple@nvidia.com>,
"Logan Gunthorpe" <logang@deltatee.com>,
"Ralph Campbell" <rcampbell@nvidia.com>,
linux-kernel@vger.kernel.org, amd-gfx@lists.freedesktop.org,
dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org,
nvdimm@lists.linux.dev, linux-mm@kvack.org,
"Alex Sierra" <alex.sierra@amd.com>
Subject: [PATCH 18/27] drm/amdkfd: add SPM support for SVM
Date: Thu, 10 Feb 2022 08:28:19 +0100 [thread overview]
Message-ID: <20220210072828.2930359-19-hch@lst.de> (raw)
In-Reply-To: <20220210072828.2930359-1-hch@lst.de>
From: Alex Sierra <alex.sierra@amd.com>
When CPU is connected throug XGMI, it has coherent
access to VRAM resource. In this case that resource
is taken from a table in the device gmc aperture base.
This resource is used along with the device type, which could
be DEVICE_PRIVATE or DEVICE_COHERENT to create the device
page map region.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 28 ++++++++++++++----------
1 file changed, 17 insertions(+), 11 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c
index e27ca375876230..2c51f2ac3b46ac 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c
@@ -933,7 +933,7 @@ int svm_migrate_init(struct amdgpu_device *adev)
{
struct kfd_dev *kfddev = adev->kfd.dev;
struct dev_pagemap *pgmap;
- struct resource *res;
+ struct resource *res = NULL;
unsigned long size;
void *r;
@@ -948,28 +948,34 @@ int svm_migrate_init(struct amdgpu_device *adev)
* should remove reserved size
*/
size = ALIGN(adev->gmc.real_vram_size, 2ULL << 20);
- res = devm_request_free_mem_region(adev->dev, &iomem_resource, size);
- if (IS_ERR(res))
- return -ENOMEM;
+ if (adev->gmc.xgmi.connected_to_cpu) {
+ pgmap->range.start = adev->gmc.aper_base;
+ pgmap->range.end = adev->gmc.aper_base + adev->gmc.aper_size - 1;
+ pgmap->type = MEMORY_DEVICE_COHERENT;
+ } else {
+ res = devm_request_free_mem_region(adev->dev, &iomem_resource, size);
+ if (IS_ERR(res))
+ return -ENOMEM;
+ pgmap->range.start = res->start;
+ pgmap->range.end = res->end;
+ pgmap->type = MEMORY_DEVICE_PRIVATE;
+ }
- pgmap->type = MEMORY_DEVICE_PRIVATE;
pgmap->nr_range = 1;
- pgmap->range.start = res->start;
- pgmap->range.end = res->end;
pgmap->ops = &svm_migrate_pgmap_ops;
pgmap->owner = SVM_ADEV_PGMAP_OWNER(adev);
- pgmap->flags = MIGRATE_VMA_SELECT_DEVICE_PRIVATE;
-
+ pgmap->flags = 0;
/* Device manager releases device-specific resources, memory region and
* pgmap when driver disconnects from device.
*/
r = devm_memremap_pages(adev->dev, pgmap);
if (IS_ERR(r)) {
pr_err("failed to register HMM device memory\n");
-
/* Disable SVM support capability */
pgmap->type = 0;
- devm_release_mem_region(adev->dev, res->start, resource_size(res));
+ if (pgmap->type == MEMORY_DEVICE_PRIVATE)
+ devm_release_mem_region(adev->dev, res->start,
+ res->end - res->start + 1);
return PTR_ERR(r);
}
--
2.30.2
next prev parent reply other threads:[~2022-02-10 7:29 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-10 7:28 start sorting out the ZONE_DEVICE refcount mess v2 Christoph Hellwig
2022-02-10 7:28 ` [PATCH 01/27] mm: remove a pointless CONFIG_ZONE_DEVICE check in memremap_pages Christoph Hellwig
2022-02-10 11:37 ` Miaohe Lin
2022-02-10 7:28 ` [PATCH 02/27] mm: remove the __KERNEL__ guard from <linux/mm.h> Christoph Hellwig
2022-02-10 7:28 ` [PATCH 03/27] mm: remove pointless includes from <linux/hmm.h> Christoph Hellwig
2022-02-10 9:39 ` Muchun Song
2022-02-10 7:28 ` [PATCH 04/27] mm: move free_devmap_managed_page to memremap.c Christoph Hellwig
2022-02-10 7:28 ` [PATCH 05/27] mm: simplify freeing of devmap managed pages Christoph Hellwig
2022-02-10 7:28 ` [PATCH 06/27] mm: don't include <linux/memremap.h> in <linux/mm.h> Christoph Hellwig
2022-02-10 7:28 ` [PATCH 07/27] mm: remove the extra ZONE_DEVICE struct page refcount Christoph Hellwig
2022-02-10 7:28 ` [PATCH 08/27] fsdax: depend on ZONE_DEVICE || FS_DAX_LIMITED Christoph Hellwig
2022-02-10 7:28 ` [PATCH 09/27] mm: generalize the pgmap based page_free infrastructure Christoph Hellwig
2022-02-14 17:37 ` Logan Gunthorpe
2022-02-10 7:28 ` [PATCH 10/27] mm: refactor check_and_migrate_movable_pages Christoph Hellwig
2022-02-10 7:28 ` [PATCH 11/27] mm: refactor the ZONE_DEVICE handling in migrate_vma_insert_page Christoph Hellwig
2022-02-10 10:48 ` Alistair Popple
2022-02-10 7:28 ` [PATCH 12/27] mm: refactor the ZONE_DEVICE handling in migrate_vma_pages Christoph Hellwig
2022-02-10 10:46 ` Alistair Popple
2022-02-10 7:28 ` [PATCH 13/27] mm: move the migrate_vma_* device migration code into it's own file Christoph Hellwig
2022-02-10 10:35 ` Alistair Popple
2022-02-10 15:23 ` Christoph Hellwig
2022-02-10 7:28 ` [PATCH 14/27] mm: build migrate_vma_* for all configs with ZONE_DEVICE support Christoph Hellwig
2022-02-10 10:43 ` Alistair Popple
2022-02-10 7:28 ` [PATCH 15/27] mm: add zone device coherent type memory support Christoph Hellwig
2022-02-10 7:28 ` [PATCH 16/27] mm: add device coherent vma selection for memory migration Christoph Hellwig
2022-02-10 7:28 ` [PATCH 17/27] mm/gup: fail get_user_pages for LONGTERM dev coherent type Christoph Hellwig
2022-02-10 7:28 ` Christoph Hellwig [this message]
2022-02-10 7:28 ` [PATCH 19/27] drm/amdkfd: coherent type as sys mem on migration to ram Christoph Hellwig
2022-02-10 7:28 ` [PATCH 20/27] lib: test_hmm add ioctl to get zone device type Christoph Hellwig
2022-02-10 7:28 ` [PATCH 21/27] lib: test_hmm add module param for " Christoph Hellwig
2022-02-10 7:28 ` [PATCH 22/27] lib: add support for device coherent type in test_hmm Christoph Hellwig
2022-02-10 7:28 ` [PATCH 23/27] tools: update hmm-test to support device coherent type Christoph Hellwig
2022-02-10 7:28 ` [PATCH 24/27] tools: update test_hmm script to support SP config Christoph Hellwig
2022-02-10 7:28 ` [PATCH 25/27] mm: remove the vma check in migrate_vma_setup() Christoph Hellwig
2022-02-10 7:28 ` [PATCH 26/27] mm/gup: migrate device coherent pages when pinning instead of failing Christoph Hellwig
2022-02-10 7:28 ` [PATCH 27/27] tools: add hmm gup test for long term pinned device pages Christoph Hellwig
2022-02-10 10:56 ` start sorting out the ZONE_DEVICE refcount mess v2 Alistair Popple
2022-02-10 17:36 ` Sierra Guiza, Alejandro (Alex)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220210072828.2930359-19-hch@lst.de \
--to=hch@lst.de \
--cc=Felix.Kuehling@amd.com \
--cc=Xinhui.Pan@amd.com \
--cc=akpm@linux-foundation.org \
--cc=alex.sierra@amd.com \
--cc=alexander.deucher@amd.com \
--cc=amd-gfx@lists.freedesktop.org \
--cc=apopple@nvidia.com \
--cc=bskeggs@redhat.com \
--cc=christian.koenig@amd.com \
--cc=dan.j.williams@intel.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=jgg@ziepe.ca \
--cc=kherbst@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=logang@deltatee.com \
--cc=lyude@redhat.com \
--cc=nouveau@lists.freedesktop.org \
--cc=nvdimm@lists.linux.dev \
--cc=rcampbell@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).