From: Christoph Hellwig <hch@lst.de> To: Andrew Morton <akpm@linux-foundation.org>, Dan Williams <dan.j.williams@intel.com> Cc: "Felix Kuehling" <Felix.Kuehling@amd.com>, "Alex Deucher" <alexander.deucher@amd.com>, "Christian König" <christian.koenig@amd.com>, "Pan, Xinhui" <Xinhui.Pan@amd.com>, "Ben Skeggs" <bskeggs@redhat.com>, "Karol Herbst" <kherbst@redhat.com>, "Lyude Paul" <lyude@redhat.com>, "Jason Gunthorpe" <jgg@ziepe.ca>, "Alistair Popple" <apopple@nvidia.com>, "Logan Gunthorpe" <logang@deltatee.com>, "Ralph Campbell" <rcampbell@nvidia.com>, linux-kernel@vger.kernel.org, amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, nvdimm@lists.linux.dev, linux-mm@kvack.org, "Alex Sierra" <alex.sierra@amd.com> Subject: [PATCH 18/27] drm/amdkfd: add SPM support for SVM Date: Thu, 10 Feb 2022 08:28:19 +0100 [thread overview] Message-ID: <20220210072828.2930359-19-hch@lst.de> (raw) In-Reply-To: <20220210072828.2930359-1-hch@lst.de> From: Alex Sierra <alex.sierra@amd.com> When CPU is connected throug XGMI, it has coherent access to VRAM resource. In this case that resource is taken from a table in the device gmc aperture base. This resource is used along with the device type, which could be DEVICE_PRIVATE or DEVICE_COHERENT to create the device page map region. Signed-off-by: Alex Sierra <alex.sierra@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Christoph Hellwig <hch@lst.de> --- drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 28 ++++++++++++++---------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c index e27ca375876230..2c51f2ac3b46ac 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c @@ -933,7 +933,7 @@ int svm_migrate_init(struct amdgpu_device *adev) { struct kfd_dev *kfddev = adev->kfd.dev; struct dev_pagemap *pgmap; - struct resource *res; + struct resource *res = NULL; unsigned long size; void *r; @@ -948,28 +948,34 @@ int svm_migrate_init(struct amdgpu_device *adev) * should remove reserved size */ size = ALIGN(adev->gmc.real_vram_size, 2ULL << 20); - res = devm_request_free_mem_region(adev->dev, &iomem_resource, size); - if (IS_ERR(res)) - return -ENOMEM; + if (adev->gmc.xgmi.connected_to_cpu) { + pgmap->range.start = adev->gmc.aper_base; + pgmap->range.end = adev->gmc.aper_base + adev->gmc.aper_size - 1; + pgmap->type = MEMORY_DEVICE_COHERENT; + } else { + res = devm_request_free_mem_region(adev->dev, &iomem_resource, size); + if (IS_ERR(res)) + return -ENOMEM; + pgmap->range.start = res->start; + pgmap->range.end = res->end; + pgmap->type = MEMORY_DEVICE_PRIVATE; + } - pgmap->type = MEMORY_DEVICE_PRIVATE; pgmap->nr_range = 1; - pgmap->range.start = res->start; - pgmap->range.end = res->end; pgmap->ops = &svm_migrate_pgmap_ops; pgmap->owner = SVM_ADEV_PGMAP_OWNER(adev); - pgmap->flags = MIGRATE_VMA_SELECT_DEVICE_PRIVATE; - + pgmap->flags = 0; /* Device manager releases device-specific resources, memory region and * pgmap when driver disconnects from device. */ r = devm_memremap_pages(adev->dev, pgmap); if (IS_ERR(r)) { pr_err("failed to register HMM device memory\n"); - /* Disable SVM support capability */ pgmap->type = 0; - devm_release_mem_region(adev->dev, res->start, resource_size(res)); + if (pgmap->type == MEMORY_DEVICE_PRIVATE) + devm_release_mem_region(adev->dev, res->start, + res->end - res->start + 1); return PTR_ERR(r); } -- 2.30.2
WARNING: multiple messages have this Message-ID
From: Christoph Hellwig <hch@lst.de> To: Andrew Morton <akpm@linux-foundation.org>, Dan Williams <dan.j.williams@intel.com> Cc: "Alex Sierra" <alex.sierra@amd.com>, nvdimm@lists.linux.dev, "Ralph Campbell" <rcampbell@nvidia.com>, "Alistair Popple" <apopple@nvidia.com>, dri-devel@lists.freedesktop.org, linux-mm@kvack.org, nouveau@lists.freedesktop.org, "Felix Kuehling" <Felix.Kuehling@amd.com>, "Pan, Xinhui" <Xinhui.Pan@amd.com>, linux-kernel@vger.kernel.org, amd-gfx@lists.freedesktop.org, "Jason Gunthorpe" <jgg@ziepe.ca>, "Ben Skeggs" <bskeggs@redhat.com>, "Alex Deucher" <alexander.deucher@amd.com>, "Logan Gunthorpe" <logang@deltatee.com>, "Christian König" <christian.koenig@amd.com> Subject: [Nouveau] [PATCH 18/27] drm/amdkfd: add SPM support for SVM Date: Thu, 10 Feb 2022 08:28:19 +0100 [thread overview] Message-ID: <20220210072828.2930359-19-hch@lst.de> (raw) In-Reply-To: <20220210072828.2930359-1-hch@lst.de> From: Alex Sierra <alex.sierra@amd.com> When CPU is connected throug XGMI, it has coherent access to VRAM resource. In this case that resource is taken from a table in the device gmc aperture base. This resource is used along with the device type, which could be DEVICE_PRIVATE or DEVICE_COHERENT to create the device page map region. Signed-off-by: Alex Sierra <alex.sierra@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Christoph Hellwig <hch@lst.de> --- drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 28 ++++++++++++++---------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c index e27ca375876230..2c51f2ac3b46ac 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c @@ -933,7 +933,7 @@ int svm_migrate_init(struct amdgpu_device *adev) { struct kfd_dev *kfddev = adev->kfd.dev; struct dev_pagemap *pgmap; - struct resource *res; + struct resource *res = NULL; unsigned long size; void *r; @@ -948,28 +948,34 @@ int svm_migrate_init(struct amdgpu_device *adev) * should remove reserved size */ size = ALIGN(adev->gmc.real_vram_size, 2ULL << 20); - res = devm_request_free_mem_region(adev->dev, &iomem_resource, size); - if (IS_ERR(res)) - return -ENOMEM; + if (adev->gmc.xgmi.connected_to_cpu) { + pgmap->range.start = adev->gmc.aper_base; + pgmap->range.end = adev->gmc.aper_base + adev->gmc.aper_size - 1; + pgmap->type = MEMORY_DEVICE_COHERENT; + } else { + res = devm_request_free_mem_region(adev->dev, &iomem_resource, size); + if (IS_ERR(res)) + return -ENOMEM; + pgmap->range.start = res->start; + pgmap->range.end = res->end; + pgmap->type = MEMORY_DEVICE_PRIVATE; + } - pgmap->type = MEMORY_DEVICE_PRIVATE; pgmap->nr_range = 1; - pgmap->range.start = res->start; - pgmap->range.end = res->end; pgmap->ops = &svm_migrate_pgmap_ops; pgmap->owner = SVM_ADEV_PGMAP_OWNER(adev); - pgmap->flags = MIGRATE_VMA_SELECT_DEVICE_PRIVATE; - + pgmap->flags = 0; /* Device manager releases device-specific resources, memory region and * pgmap when driver disconnects from device. */ r = devm_memremap_pages(adev->dev, pgmap); if (IS_ERR(r)) { pr_err("failed to register HMM device memory\n"); - /* Disable SVM support capability */ pgmap->type = 0; - devm_release_mem_region(adev->dev, res->start, resource_size(res)); + if (pgmap->type == MEMORY_DEVICE_PRIVATE) + devm_release_mem_region(adev->dev, res->start, + res->end - res->start + 1); return PTR_ERR(r); } -- 2.30.2
WARNING: multiple messages have this Message-ID
From: Christoph Hellwig <hch@lst.de> To: Andrew Morton <akpm@linux-foundation.org>, Dan Williams <dan.j.williams@intel.com> Cc: "Alex Sierra" <alex.sierra@amd.com>, nvdimm@lists.linux.dev, "Ralph Campbell" <rcampbell@nvidia.com>, "Alistair Popple" <apopple@nvidia.com>, dri-devel@lists.freedesktop.org, "Karol Herbst" <kherbst@redhat.com>, linux-mm@kvack.org, nouveau@lists.freedesktop.org, "Felix Kuehling" <Felix.Kuehling@amd.com>, "Pan, Xinhui" <Xinhui.Pan@amd.com>, linux-kernel@vger.kernel.org, amd-gfx@lists.freedesktop.org, "Jason Gunthorpe" <jgg@ziepe.ca>, "Ben Skeggs" <bskeggs@redhat.com>, "Alex Deucher" <alexander.deucher@amd.com>, "Logan Gunthorpe" <logang@deltatee.com>, "Christian König" <christian.koenig@amd.com> Subject: [PATCH 18/27] drm/amdkfd: add SPM support for SVM Date: Thu, 10 Feb 2022 08:28:19 +0100 [thread overview] Message-ID: <20220210072828.2930359-19-hch@lst.de> (raw) In-Reply-To: <20220210072828.2930359-1-hch@lst.de> From: Alex Sierra <alex.sierra@amd.com> When CPU is connected throug XGMI, it has coherent access to VRAM resource. In this case that resource is taken from a table in the device gmc aperture base. This resource is used along with the device type, which could be DEVICE_PRIVATE or DEVICE_COHERENT to create the device page map region. Signed-off-by: Alex Sierra <alex.sierra@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Christoph Hellwig <hch@lst.de> --- drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 28 ++++++++++++++---------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c index e27ca375876230..2c51f2ac3b46ac 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c @@ -933,7 +933,7 @@ int svm_migrate_init(struct amdgpu_device *adev) { struct kfd_dev *kfddev = adev->kfd.dev; struct dev_pagemap *pgmap; - struct resource *res; + struct resource *res = NULL; unsigned long size; void *r; @@ -948,28 +948,34 @@ int svm_migrate_init(struct amdgpu_device *adev) * should remove reserved size */ size = ALIGN(adev->gmc.real_vram_size, 2ULL << 20); - res = devm_request_free_mem_region(adev->dev, &iomem_resource, size); - if (IS_ERR(res)) - return -ENOMEM; + if (adev->gmc.xgmi.connected_to_cpu) { + pgmap->range.start = adev->gmc.aper_base; + pgmap->range.end = adev->gmc.aper_base + adev->gmc.aper_size - 1; + pgmap->type = MEMORY_DEVICE_COHERENT; + } else { + res = devm_request_free_mem_region(adev->dev, &iomem_resource, size); + if (IS_ERR(res)) + return -ENOMEM; + pgmap->range.start = res->start; + pgmap->range.end = res->end; + pgmap->type = MEMORY_DEVICE_PRIVATE; + } - pgmap->type = MEMORY_DEVICE_PRIVATE; pgmap->nr_range = 1; - pgmap->range.start = res->start; - pgmap->range.end = res->end; pgmap->ops = &svm_migrate_pgmap_ops; pgmap->owner = SVM_ADEV_PGMAP_OWNER(adev); - pgmap->flags = MIGRATE_VMA_SELECT_DEVICE_PRIVATE; - + pgmap->flags = 0; /* Device manager releases device-specific resources, memory region and * pgmap when driver disconnects from device. */ r = devm_memremap_pages(adev->dev, pgmap); if (IS_ERR(r)) { pr_err("failed to register HMM device memory\n"); - /* Disable SVM support capability */ pgmap->type = 0; - devm_release_mem_region(adev->dev, res->start, resource_size(res)); + if (pgmap->type == MEMORY_DEVICE_PRIVATE) + devm_release_mem_region(adev->dev, res->start, + res->end - res->start + 1); return PTR_ERR(r); } -- 2.30.2
next prev parent reply other threads:[~2022-02-10 7:29 UTC|newest] Thread overview: 114+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-02-10 7:28 start sorting out the ZONE_DEVICE refcount mess v2 Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 01/27] mm: remove a pointless CONFIG_ZONE_DEVICE check in memremap_pages Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 11:37 ` Miaohe Lin 2022-02-10 11:37 ` [Nouveau] " Miaohe Lin 2022-02-10 11:37 ` Miaohe Lin 2022-02-10 7:28 ` [PATCH 02/27] mm: remove the __KERNEL__ guard from <linux/mm.h> Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 03/27] mm: remove pointless includes from <linux/hmm.h> Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 9:39 ` Muchun Song 2022-02-10 9:39 ` [Nouveau] " Muchun Song 2022-02-10 9:39 ` Muchun Song 2022-02-10 7:28 ` [PATCH 04/27] mm: move free_devmap_managed_page to memremap.c Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 05/27] mm: simplify freeing of devmap managed pages Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 06/27] mm: don't include <linux/memremap.h> in <linux/mm.h> Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 07/27] mm: remove the extra ZONE_DEVICE struct page refcount Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 08/27] fsdax: depend on ZONE_DEVICE || FS_DAX_LIMITED Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 09/27] mm: generalize the pgmap based page_free infrastructure Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-14 17:37 ` Logan Gunthorpe 2022-02-14 17:37 ` Logan Gunthorpe 2022-02-14 17:37 ` [Nouveau] " Logan Gunthorpe 2022-02-10 7:28 ` [PATCH 10/27] mm: refactor check_and_migrate_movable_pages Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 11/27] mm: refactor the ZONE_DEVICE handling in migrate_vma_insert_page Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 10:48 ` Alistair Popple 2022-02-10 10:48 ` Alistair Popple 2022-02-10 10:48 ` [Nouveau] " Alistair Popple 2022-02-10 7:28 ` [PATCH 12/27] mm: refactor the ZONE_DEVICE handling in migrate_vma_pages Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 10:46 ` Alistair Popple 2022-02-10 10:46 ` Alistair Popple 2022-02-10 10:46 ` [Nouveau] " Alistair Popple 2022-02-10 7:28 ` [PATCH 13/27] mm: move the migrate_vma_* device migration code into it's own file Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 10:35 ` Alistair Popple 2022-02-10 10:35 ` Alistair Popple 2022-02-10 10:35 ` [Nouveau] " Alistair Popple 2022-02-10 15:23 ` Christoph Hellwig 2022-02-10 15:23 ` Christoph Hellwig 2022-02-10 15:23 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 14/27] mm: build migrate_vma_* for all configs with ZONE_DEVICE support Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 10:43 ` Alistair Popple 2022-02-10 10:43 ` Alistair Popple 2022-02-10 10:43 ` [Nouveau] " Alistair Popple 2022-02-10 7:28 ` [PATCH 15/27] mm: add zone device coherent type memory support Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 16/27] mm: add device coherent vma selection for memory migration Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 17/27] mm/gup: fail get_user_pages for LONGTERM dev coherent type Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig [this message] 2022-02-10 7:28 ` [PATCH 18/27] drm/amdkfd: add SPM support for SVM Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 19/27] drm/amdkfd: coherent type as sys mem on migration to ram Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 20/27] lib: test_hmm add ioctl to get zone device type Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 21/27] lib: test_hmm add module param for " Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 22/27] lib: add support for device coherent type in test_hmm Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 23/27] tools: update hmm-test to support device coherent type Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 24/27] tools: update test_hmm script to support SP config Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 25/27] mm: remove the vma check in migrate_vma_setup() Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 26/27] mm/gup: migrate device coherent pages when pinning instead of failing Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 7:28 ` [PATCH 27/27] tools: add hmm gup test for long term pinned device pages Christoph Hellwig 2022-02-10 7:28 ` Christoph Hellwig 2022-02-10 7:28 ` [Nouveau] " Christoph Hellwig 2022-02-10 10:56 ` start sorting out the ZONE_DEVICE refcount mess v2 Alistair Popple 2022-02-10 10:56 ` Alistair Popple 2022-02-10 10:56 ` [Nouveau] " Alistair Popple 2022-02-10 17:36 ` Sierra Guiza, Alejandro (Alex) 2022-02-10 17:36 ` [Nouveau] " Sierra Guiza, Alejandro (Alex) 2022-02-10 17:36 ` Sierra Guiza, Alejandro (Alex)
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220210072828.2930359-19-hch@lst.de \ --to=hch@lst.de \ --cc=Felix.Kuehling@amd.com \ --cc=Xinhui.Pan@amd.com \ --cc=akpm@linux-foundation.org \ --cc=alex.sierra@amd.com \ --cc=alexander.deucher@amd.com \ --cc=amd-gfx@lists.freedesktop.org \ --cc=apopple@nvidia.com \ --cc=bskeggs@redhat.com \ --cc=christian.koenig@amd.com \ --cc=dan.j.williams@intel.com \ --cc=dri-devel@lists.freedesktop.org \ --cc=jgg@ziepe.ca \ --cc=kherbst@redhat.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=logang@deltatee.com \ --cc=lyude@redhat.com \ --cc=nouveau@lists.freedesktop.org \ --cc=nvdimm@lists.linux.dev \ --cc=rcampbell@nvidia.com \ --subject='Re: [PATCH 18/27] drm/amdkfd: add SPM support for SVM' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.