From: "Zeng, Oak" <Oak.Zeng@amd.com> To: "Kuehling, Felix" <Felix.Kuehling@amd.com>, "amd-gfx@lists.freedesktop.org" <amd-gfx@lists.freedesktop.org>, "dri-devel@lists.freedesktop.org" <dri-devel@lists.freedesktop.org> Subject: Re: [PATCH v2 00/10] Implement multi-GPU DMA mappings for KFD Date: Tue, 27 Apr 2021 15:16:12 +0000 [thread overview] Message-ID: <A4572F97-6ACD-4279-ABA6-37C7969DE0BD@amd.com> (raw) In-Reply-To: <20210422013058.6305-1-Felix.Kuehling@amd.com> This series is Acked-by: Oak Zeng <Oak.Zeng@amd.com> Regards, Oak On 2021-04-21, 9:31 PM, "dri-devel on behalf of Felix Kuehling" <dri-devel-bounces@lists.freedesktop.org on behalf of Felix.Kuehling@amd.com> wrote: This patch series fixes DMA-mappings of system memory (GTT and userptr) for KFD running on multi-GPU systems with IOMMU enabled. One SG-BO per GPU is needed to maintain the DMA mappings of each BO. Changes in v2: - Made the original BO parent of the SG BO to fix bo destruction order - Removed individualiation hack that is, not needed with parent BO - Removed resv locking hace in amdgpu_ttm_unpopulate, not needed without the individualization hack - Added a patch to enable the Intel IOMMU driver in rock-dbg_defconfig - Added a patch to move dmabuf attach/detach into backend_(un)bind I'm still seeing some IOMMU access faults in the eviction test. They seem to be related to userptr handling. They happen even without this patch series on a single-GPU system, where this patch series is not needed. I believe this is an old problem in KFD or amdgpu that is being exposed by device isolation from the IOMMU. I'm debugging it, but it should not hold up this patch series. "drm/ttm: Don't count pages in SG BOs against pages_limit" was already applied to drm-misc (I think). I'm still including it here because my patches depend on it. Without that, the SG BOs created for DMA mappings cause many tests fail because TTM incorrectly thinks it's out of memory. Felix Kuehling (10): rock-dbg_defconfig: Enable Intel IOMMU drm/amdgpu: Rename kfd_bo_va_list to kfd_mem_attachment drm/amdgpu: Keep a bo-reference per-attachment drm/amdgpu: Simplify AQL queue mapping drm/amdgpu: Add multi-GPU DMA mapping helpers drm/amdgpu: DMA map/unmap when updating GPU mappings drm/amdgpu: Move kfd_mem_attach outside reservation drm/amdgpu: Add DMA mapping of GTT BOs drm/ttm: Don't count pages in SG BOs against pages_limit drm/amdgpu: Move dmabuf attach/detach to backend_(un)bind arch/x86/configs/rock-dbg_defconfig | 11 +- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 18 +- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 530 ++++++++++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 51 +- drivers/gpu/drm/ttm/ttm_tt.c | 27 +- 5 files changed, 437 insertions(+), 200 deletions(-) -- 2.31.1 _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&data=04%7C01%7Coak.zeng%40amd.com%7Cfb31922bd50846641e9508d9052e635d%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637546519058204046%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=yxNesWxDmM5H8ObiNmeaa0DBIEyptiBpjUKSUqS%2B52M%3D&reserved=0 _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
WARNING: multiple messages have this Message-ID (diff)
From: "Zeng, Oak" <Oak.Zeng@amd.com> To: "Kuehling, Felix" <Felix.Kuehling@amd.com>, "amd-gfx@lists.freedesktop.org" <amd-gfx@lists.freedesktop.org>, "dri-devel@lists.freedesktop.org" <dri-devel@lists.freedesktop.org> Subject: Re: [PATCH v2 00/10] Implement multi-GPU DMA mappings for KFD Date: Tue, 27 Apr 2021 15:16:12 +0000 [thread overview] Message-ID: <A4572F97-6ACD-4279-ABA6-37C7969DE0BD@amd.com> (raw) In-Reply-To: <20210422013058.6305-1-Felix.Kuehling@amd.com> This series is Acked-by: Oak Zeng <Oak.Zeng@amd.com> Regards, Oak On 2021-04-21, 9:31 PM, "dri-devel on behalf of Felix Kuehling" <dri-devel-bounces@lists.freedesktop.org on behalf of Felix.Kuehling@amd.com> wrote: This patch series fixes DMA-mappings of system memory (GTT and userptr) for KFD running on multi-GPU systems with IOMMU enabled. One SG-BO per GPU is needed to maintain the DMA mappings of each BO. Changes in v2: - Made the original BO parent of the SG BO to fix bo destruction order - Removed individualiation hack that is, not needed with parent BO - Removed resv locking hace in amdgpu_ttm_unpopulate, not needed without the individualization hack - Added a patch to enable the Intel IOMMU driver in rock-dbg_defconfig - Added a patch to move dmabuf attach/detach into backend_(un)bind I'm still seeing some IOMMU access faults in the eviction test. They seem to be related to userptr handling. They happen even without this patch series on a single-GPU system, where this patch series is not needed. I believe this is an old problem in KFD or amdgpu that is being exposed by device isolation from the IOMMU. I'm debugging it, but it should not hold up this patch series. "drm/ttm: Don't count pages in SG BOs against pages_limit" was already applied to drm-misc (I think). I'm still including it here because my patches depend on it. Without that, the SG BOs created for DMA mappings cause many tests fail because TTM incorrectly thinks it's out of memory. Felix Kuehling (10): rock-dbg_defconfig: Enable Intel IOMMU drm/amdgpu: Rename kfd_bo_va_list to kfd_mem_attachment drm/amdgpu: Keep a bo-reference per-attachment drm/amdgpu: Simplify AQL queue mapping drm/amdgpu: Add multi-GPU DMA mapping helpers drm/amdgpu: DMA map/unmap when updating GPU mappings drm/amdgpu: Move kfd_mem_attach outside reservation drm/amdgpu: Add DMA mapping of GTT BOs drm/ttm: Don't count pages in SG BOs against pages_limit drm/amdgpu: Move dmabuf attach/detach to backend_(un)bind arch/x86/configs/rock-dbg_defconfig | 11 +- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 18 +- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 530 ++++++++++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 51 +- drivers/gpu/drm/ttm/ttm_tt.c | 27 +- 5 files changed, 437 insertions(+), 200 deletions(-) -- 2.31.1 _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&data=04%7C01%7Coak.zeng%40amd.com%7Cfb31922bd50846641e9508d9052e635d%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637546519058204046%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=yxNesWxDmM5H8ObiNmeaa0DBIEyptiBpjUKSUqS%2B52M%3D&reserved=0 _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
next prev parent reply other threads:[~2021-04-27 15:16 UTC|newest] Thread overview: 64+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-04-22 1:30 [PATCH v2 00/10] Implement multi-GPU DMA mappings for KFD Felix Kuehling 2021-04-22 1:30 ` Felix Kuehling 2021-04-22 1:30 ` [PATCH v2 01/10] rock-dbg_defconfig: Enable Intel IOMMU Felix Kuehling 2021-04-22 1:30 ` Felix Kuehling 2021-04-22 1:30 ` [PATCH v2 02/10] drm/amdgpu: Rename kfd_bo_va_list to kfd_mem_attachment Felix Kuehling 2021-04-22 1:30 ` Felix Kuehling 2021-05-10 22:00 ` Errabolu, Ramesh 2021-05-10 22:00 ` Errabolu, Ramesh 2021-04-22 1:30 ` [PATCH v2 03/10] drm/amdgpu: Keep a bo-reference per-attachment Felix Kuehling 2021-04-22 1:30 ` Felix Kuehling 2021-05-10 22:00 ` Errabolu, Ramesh 2021-05-10 22:00 ` Errabolu, Ramesh 2021-04-22 1:30 ` [PATCH v2 04/10] drm/amdgpu: Simplify AQL queue mapping Felix Kuehling 2021-04-22 1:30 ` Felix Kuehling 2021-04-23 1:33 ` Zeng, Oak 2021-04-23 1:33 ` Zeng, Oak 2021-04-23 7:23 ` Felix Kuehling 2021-04-23 7:23 ` Felix Kuehling 2021-05-10 22:03 ` Errabolu, Ramesh 2021-05-10 22:03 ` Errabolu, Ramesh 2021-04-22 1:30 ` [PATCH v2 05/10] drm/amdgpu: Add multi-GPU DMA mapping helpers Felix Kuehling 2021-04-22 1:30 ` Felix Kuehling 2021-04-27 0:09 ` Zeng, Oak 2021-04-27 0:09 ` Zeng, Oak 2021-04-27 3:41 ` Felix Kuehling 2021-04-27 3:41 ` Felix Kuehling 2021-05-10 22:05 ` Errabolu, Ramesh 2021-05-10 22:05 ` Errabolu, Ramesh 2021-04-22 1:30 ` [PATCH v2 06/10] drm/amdgpu: DMA map/unmap when updating GPU mappings Felix Kuehling 2021-04-22 1:30 ` Felix Kuehling 2021-04-27 0:23 ` Zeng, Oak 2021-04-27 0:23 ` Zeng, Oak 2021-04-27 3:47 ` Felix Kuehling 2021-04-27 3:47 ` Felix Kuehling 2021-05-10 22:06 ` Errabolu, Ramesh 2021-05-10 22:06 ` Errabolu, Ramesh 2021-04-22 1:30 ` [PATCH v2 07/10] drm/amdgpu: Move kfd_mem_attach outside reservation Felix Kuehling 2021-04-22 1:30 ` Felix Kuehling 2021-05-10 22:06 ` Errabolu, Ramesh 2021-05-10 22:06 ` Errabolu, Ramesh 2021-04-22 1:30 ` [PATCH v2 08/10] drm/amdgpu: Add DMA mapping of GTT BOs Felix Kuehling 2021-04-22 1:30 ` Felix Kuehling 2021-04-27 0:35 ` Zeng, Oak 2021-04-27 0:35 ` Zeng, Oak 2021-04-27 3:56 ` Felix Kuehling 2021-04-27 3:56 ` Felix Kuehling 2021-04-27 14:29 ` Zeng, Oak 2021-04-27 14:29 ` Zeng, Oak 2021-04-27 15:08 ` Felix Kuehling 2021-04-27 15:08 ` Felix Kuehling 2021-05-10 22:07 ` Errabolu, Ramesh 2021-05-10 22:07 ` Errabolu, Ramesh 2021-04-22 1:30 ` [PATCH v2 09/10] drm/ttm: Don't count pages in SG BOs against pages_limit Felix Kuehling 2021-04-22 1:30 ` Felix Kuehling 2021-05-10 22:08 ` Errabolu, Ramesh 2021-05-10 22:08 ` Errabolu, Ramesh 2021-04-22 1:30 ` [PATCH v2 10/10] drm/amdgpu: Move dmabuf attach/detach to backend_(un)bind Felix Kuehling 2021-04-22 1:30 ` Felix Kuehling 2021-04-22 11:20 ` Christian König 2021-04-22 11:20 ` Christian König 2021-05-10 22:09 ` Errabolu, Ramesh 2021-05-10 22:09 ` Errabolu, Ramesh 2021-04-27 15:16 ` Zeng, Oak [this message] 2021-04-27 15:16 ` [PATCH v2 00/10] Implement multi-GPU DMA mappings for KFD Zeng, Oak
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=A4572F97-6ACD-4279-ABA6-37C7969DE0BD@amd.com \ --to=oak.zeng@amd.com \ --cc=Felix.Kuehling@amd.com \ --cc=amd-gfx@lists.freedesktop.org \ --cc=dri-devel@lists.freedesktop.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.