From: "Thomas Hellström (VMware)" <thomas_os@shipmail.org> To: linux-mm@kvack.org, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org Cc: pv-drivers@vmware.com, linux-graphics-maintainer@vmware.com, "Thomas Hellstrom (VMware)" <thomas_os@shipmail.org>, "Andrew Morton" <akpm@linux-foundation.org>, "Michal Hocko" <mhocko@suse.com>, "Matthew Wilcox (Oracle)" <willy@infradead.org>, "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>, "Ralph Campbell" <rcampbell@nvidia.com>, "Jérôme Glisse" <jglisse@redhat.com>, "Christian König" <christian.koenig@amd.com>, "Dan Williams" <dan.j.williams@intel.com>, "Roland Scheidegger" <sroland@vmware.com> Subject: [PATCH v7 8/9] drm/vmwgfx: Introduce a huge page aligning TTM range manager Date: Tue, 24 Mar 2020 21:11:22 +0100 [thread overview] Message-ID: <20200324201123.3118-9-thomas_os@shipmail.org> (raw) In-Reply-To: <20200324201123.3118-1-thomas_os@shipmail.org> From: "Thomas Hellstrom (VMware)" <thomas_os@shipmail.org> Using huge page-table entries requires that the physical address of the start of a buffer object is huge page size aligned. Make a special version of the TTM range manager that accomplishes this, but falls back to a smaller page size alignment (PUD->PMD, PMD->NORMAL) to avoid eviction. If other drivers want to use it in the future, it can be made a TTM generic helper. Note that drivers can force eviction for a certain alignment by assigning the TTM GPU alignment correspondingly. Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Michal Hocko <mhocko@suse.com> Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org> Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> Cc: Ralph Campbell <rcampbell@nvidia.com> Cc: "Jérôme Glisse" <jglisse@redhat.com> Cc: "Christian König" <christian.koenig@amd.com> Cc: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Thomas Hellstrom (VMware) <thomas_os@shipmail.org> Reviewed-by: Roland Scheidegger <sroland@vmware.com> Acked-by: Christian König <christian.koenig@amd.com> --- drivers/gpu/drm/vmwgfx/Makefile | 1 + drivers/gpu/drm/vmwgfx/vmwgfx_drv.h | 7 ++ drivers/gpu/drm/vmwgfx/vmwgfx_thp.c | 166 ++++++++++++++++++++++++++++ 3 files changed, 174 insertions(+) create mode 100644 drivers/gpu/drm/vmwgfx/vmwgfx_thp.c diff --git a/drivers/gpu/drm/vmwgfx/Makefile b/drivers/gpu/drm/vmwgfx/Makefile index c877a21a0739..421dd2a497a5 100644 --- a/drivers/gpu/drm/vmwgfx/Makefile +++ b/drivers/gpu/drm/vmwgfx/Makefile @@ -11,4 +11,5 @@ vmwgfx-y := vmwgfx_execbuf.o vmwgfx_gmr.o vmwgfx_kms.o vmwgfx_drv.o \ vmwgfx_validation.o vmwgfx_page_dirty.o \ ttm_object.o ttm_lock.o +vmwgfx-$(CONFIG_TRANSPARENT_HUGEPAGE) += vmwgfx_thp.o obj-$(CONFIG_DRM_VMWGFX) := vmwgfx.o diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h index 6fc8d5c171c6..d19d28c13671 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h @@ -1407,6 +1407,13 @@ vm_fault_t vmw_bo_vm_huge_fault(struct vm_fault *vmf, enum page_entry_size pe_size); #endif +/* Transparent hugepage support - vmwgfx_thp.c */ +#ifdef CONFIG_TRANSPARENT_HUGEPAGE +extern const struct ttm_mem_type_manager_func vmw_thp_func; +#else +#define vmw_thp_func ttm_bo_manager_func +#endif + /** * VMW_DEBUG_KMS - Debug output for kernel mode-setting * diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_thp.c b/drivers/gpu/drm/vmwgfx/vmwgfx_thp.c new file mode 100644 index 000000000000..b7c816ba7166 --- /dev/null +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_thp.c @@ -0,0 +1,166 @@ +// SPDX-License-Identifier: GPL-2.0 OR MIT +/* + * Huge page-table-entry support for IO memory. + * + * Copyright (C) 2007-2019 Vmware, Inc. All rights reservedd. + */ +#include "vmwgfx_drv.h" +#include <drm/ttm/ttm_module.h> +#include <drm/ttm/ttm_bo_driver.h> +#include <drm/ttm/ttm_placement.h> + +/** + * struct vmw_thp_manager - Range manager implementing huge page alignment + * + * @mm: The underlying range manager. Protected by @lock. + * @lock: Manager lock. + */ +struct vmw_thp_manager { + struct drm_mm mm; + spinlock_t lock; +}; + +static int vmw_thp_insert_aligned(struct drm_mm *mm, struct drm_mm_node *node, + unsigned long align_pages, + const struct ttm_place *place, + struct ttm_mem_reg *mem, + unsigned long lpfn, + enum drm_mm_insert_mode mode) +{ + if (align_pages >= mem->page_alignment && + (!mem->page_alignment || align_pages % mem->page_alignment == 0)) { + return drm_mm_insert_node_in_range(mm, node, + mem->num_pages, + align_pages, 0, + place->fpfn, lpfn, mode); + } + + return -ENOSPC; +} + +static int vmw_thp_get_node(struct ttm_mem_type_manager *man, + struct ttm_buffer_object *bo, + const struct ttm_place *place, + struct ttm_mem_reg *mem) +{ + struct vmw_thp_manager *rman = (struct vmw_thp_manager *) man->priv; + struct drm_mm *mm = &rman->mm; + struct drm_mm_node *node; + unsigned long align_pages; + unsigned long lpfn; + enum drm_mm_insert_mode mode = DRM_MM_INSERT_BEST; + int ret; + + node = kzalloc(sizeof(*node), GFP_KERNEL); + if (!node) + return -ENOMEM; + + lpfn = place->lpfn; + if (!lpfn) + lpfn = man->size; + + mode = DRM_MM_INSERT_BEST; + if (place->flags & TTM_PL_FLAG_TOPDOWN) + mode = DRM_MM_INSERT_HIGH; + + spin_lock(&rman->lock); + if (IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD)) { + align_pages = (HPAGE_PUD_SIZE >> PAGE_SHIFT); + if (mem->num_pages >= align_pages) { + ret = vmw_thp_insert_aligned(mm, node, align_pages, + place, mem, lpfn, mode); + if (!ret) + goto found_unlock; + } + } + + align_pages = (HPAGE_PMD_SIZE >> PAGE_SHIFT); + if (mem->num_pages >= align_pages) { + ret = vmw_thp_insert_aligned(mm, node, align_pages, place, mem, + lpfn, mode); + if (!ret) + goto found_unlock; + } + + ret = drm_mm_insert_node_in_range(mm, node, mem->num_pages, + mem->page_alignment, 0, + place->fpfn, lpfn, mode); +found_unlock: + spin_unlock(&rman->lock); + + if (unlikely(ret)) { + kfree(node); + } else { + mem->mm_node = node; + mem->start = node->start; + } + + return 0; +} + + + +static void vmw_thp_put_node(struct ttm_mem_type_manager *man, + struct ttm_mem_reg *mem) +{ + struct vmw_thp_manager *rman = (struct vmw_thp_manager *) man->priv; + + if (mem->mm_node) { + spin_lock(&rman->lock); + drm_mm_remove_node(mem->mm_node); + spin_unlock(&rman->lock); + + kfree(mem->mm_node); + mem->mm_node = NULL; + } +} + +static int vmw_thp_init(struct ttm_mem_type_manager *man, + unsigned long p_size) +{ + struct vmw_thp_manager *rman; + + rman = kzalloc(sizeof(*rman), GFP_KERNEL); + if (!rman) + return -ENOMEM; + + drm_mm_init(&rman->mm, 0, p_size); + spin_lock_init(&rman->lock); + man->priv = rman; + return 0; +} + +static int vmw_thp_takedown(struct ttm_mem_type_manager *man) +{ + struct vmw_thp_manager *rman = (struct vmw_thp_manager *) man->priv; + struct drm_mm *mm = &rman->mm; + + spin_lock(&rman->lock); + if (drm_mm_clean(mm)) { + drm_mm_takedown(mm); + spin_unlock(&rman->lock); + kfree(rman); + man->priv = NULL; + return 0; + } + spin_unlock(&rman->lock); + return -EBUSY; +} + +static void vmw_thp_debug(struct ttm_mem_type_manager *man, + struct drm_printer *printer) +{ + struct vmw_thp_manager *rman = (struct vmw_thp_manager *) man->priv; + + spin_lock(&rman->lock); + drm_mm_print(&rman->mm, printer); + spin_unlock(&rman->lock); +} + +const struct ttm_mem_type_manager_func vmw_thp_func = { + .init = vmw_thp_init, + .takedown = vmw_thp_takedown, + .get_node = vmw_thp_get_node, + .put_node = vmw_thp_put_node, + .debug = vmw_thp_debug +}; -- 2.21.1
WARNING: multiple messages have this Message-ID (diff)
From: "Thomas Hellström (VMware)" <thomas_os@shipmail.org> To: linux-mm@kvack.org, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org Cc: "Ralph Campbell" <rcampbell@nvidia.com>, "Michal Hocko" <mhocko@suse.com>, pv-drivers@vmware.com, "Roland Scheidegger" <sroland@vmware.com>, "Thomas Hellstrom (VMware)" <thomas_os@shipmail.org>, "Dan Williams" <dan.j.williams@intel.com>, "Matthew Wilcox (Oracle)" <willy@infradead.org>, "Jérôme Glisse" <jglisse@redhat.com>, linux-graphics-maintainer@vmware.com, "Andrew Morton" <akpm@linux-foundation.org>, "Christian König" <christian.koenig@amd.com>, "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> Subject: [PATCH v7 8/9] drm/vmwgfx: Introduce a huge page aligning TTM range manager Date: Tue, 24 Mar 2020 21:11:22 +0100 [thread overview] Message-ID: <20200324201123.3118-9-thomas_os@shipmail.org> (raw) In-Reply-To: <20200324201123.3118-1-thomas_os@shipmail.org> From: "Thomas Hellstrom (VMware)" <thomas_os@shipmail.org> Using huge page-table entries requires that the physical address of the start of a buffer object is huge page size aligned. Make a special version of the TTM range manager that accomplishes this, but falls back to a smaller page size alignment (PUD->PMD, PMD->NORMAL) to avoid eviction. If other drivers want to use it in the future, it can be made a TTM generic helper. Note that drivers can force eviction for a certain alignment by assigning the TTM GPU alignment correspondingly. Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Michal Hocko <mhocko@suse.com> Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org> Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> Cc: Ralph Campbell <rcampbell@nvidia.com> Cc: "Jérôme Glisse" <jglisse@redhat.com> Cc: "Christian König" <christian.koenig@amd.com> Cc: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Thomas Hellstrom (VMware) <thomas_os@shipmail.org> Reviewed-by: Roland Scheidegger <sroland@vmware.com> Acked-by: Christian König <christian.koenig@amd.com> --- drivers/gpu/drm/vmwgfx/Makefile | 1 + drivers/gpu/drm/vmwgfx/vmwgfx_drv.h | 7 ++ drivers/gpu/drm/vmwgfx/vmwgfx_thp.c | 166 ++++++++++++++++++++++++++++ 3 files changed, 174 insertions(+) create mode 100644 drivers/gpu/drm/vmwgfx/vmwgfx_thp.c diff --git a/drivers/gpu/drm/vmwgfx/Makefile b/drivers/gpu/drm/vmwgfx/Makefile index c877a21a0739..421dd2a497a5 100644 --- a/drivers/gpu/drm/vmwgfx/Makefile +++ b/drivers/gpu/drm/vmwgfx/Makefile @@ -11,4 +11,5 @@ vmwgfx-y := vmwgfx_execbuf.o vmwgfx_gmr.o vmwgfx_kms.o vmwgfx_drv.o \ vmwgfx_validation.o vmwgfx_page_dirty.o \ ttm_object.o ttm_lock.o +vmwgfx-$(CONFIG_TRANSPARENT_HUGEPAGE) += vmwgfx_thp.o obj-$(CONFIG_DRM_VMWGFX) := vmwgfx.o diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h index 6fc8d5c171c6..d19d28c13671 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h @@ -1407,6 +1407,13 @@ vm_fault_t vmw_bo_vm_huge_fault(struct vm_fault *vmf, enum page_entry_size pe_size); #endif +/* Transparent hugepage support - vmwgfx_thp.c */ +#ifdef CONFIG_TRANSPARENT_HUGEPAGE +extern const struct ttm_mem_type_manager_func vmw_thp_func; +#else +#define vmw_thp_func ttm_bo_manager_func +#endif + /** * VMW_DEBUG_KMS - Debug output for kernel mode-setting * diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_thp.c b/drivers/gpu/drm/vmwgfx/vmwgfx_thp.c new file mode 100644 index 000000000000..b7c816ba7166 --- /dev/null +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_thp.c @@ -0,0 +1,166 @@ +// SPDX-License-Identifier: GPL-2.0 OR MIT +/* + * Huge page-table-entry support for IO memory. + * + * Copyright (C) 2007-2019 Vmware, Inc. All rights reservedd. + */ +#include "vmwgfx_drv.h" +#include <drm/ttm/ttm_module.h> +#include <drm/ttm/ttm_bo_driver.h> +#include <drm/ttm/ttm_placement.h> + +/** + * struct vmw_thp_manager - Range manager implementing huge page alignment + * + * @mm: The underlying range manager. Protected by @lock. + * @lock: Manager lock. + */ +struct vmw_thp_manager { + struct drm_mm mm; + spinlock_t lock; +}; + +static int vmw_thp_insert_aligned(struct drm_mm *mm, struct drm_mm_node *node, + unsigned long align_pages, + const struct ttm_place *place, + struct ttm_mem_reg *mem, + unsigned long lpfn, + enum drm_mm_insert_mode mode) +{ + if (align_pages >= mem->page_alignment && + (!mem->page_alignment || align_pages % mem->page_alignment == 0)) { + return drm_mm_insert_node_in_range(mm, node, + mem->num_pages, + align_pages, 0, + place->fpfn, lpfn, mode); + } + + return -ENOSPC; +} + +static int vmw_thp_get_node(struct ttm_mem_type_manager *man, + struct ttm_buffer_object *bo, + const struct ttm_place *place, + struct ttm_mem_reg *mem) +{ + struct vmw_thp_manager *rman = (struct vmw_thp_manager *) man->priv; + struct drm_mm *mm = &rman->mm; + struct drm_mm_node *node; + unsigned long align_pages; + unsigned long lpfn; + enum drm_mm_insert_mode mode = DRM_MM_INSERT_BEST; + int ret; + + node = kzalloc(sizeof(*node), GFP_KERNEL); + if (!node) + return -ENOMEM; + + lpfn = place->lpfn; + if (!lpfn) + lpfn = man->size; + + mode = DRM_MM_INSERT_BEST; + if (place->flags & TTM_PL_FLAG_TOPDOWN) + mode = DRM_MM_INSERT_HIGH; + + spin_lock(&rman->lock); + if (IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD)) { + align_pages = (HPAGE_PUD_SIZE >> PAGE_SHIFT); + if (mem->num_pages >= align_pages) { + ret = vmw_thp_insert_aligned(mm, node, align_pages, + place, mem, lpfn, mode); + if (!ret) + goto found_unlock; + } + } + + align_pages = (HPAGE_PMD_SIZE >> PAGE_SHIFT); + if (mem->num_pages >= align_pages) { + ret = vmw_thp_insert_aligned(mm, node, align_pages, place, mem, + lpfn, mode); + if (!ret) + goto found_unlock; + } + + ret = drm_mm_insert_node_in_range(mm, node, mem->num_pages, + mem->page_alignment, 0, + place->fpfn, lpfn, mode); +found_unlock: + spin_unlock(&rman->lock); + + if (unlikely(ret)) { + kfree(node); + } else { + mem->mm_node = node; + mem->start = node->start; + } + + return 0; +} + + + +static void vmw_thp_put_node(struct ttm_mem_type_manager *man, + struct ttm_mem_reg *mem) +{ + struct vmw_thp_manager *rman = (struct vmw_thp_manager *) man->priv; + + if (mem->mm_node) { + spin_lock(&rman->lock); + drm_mm_remove_node(mem->mm_node); + spin_unlock(&rman->lock); + + kfree(mem->mm_node); + mem->mm_node = NULL; + } +} + +static int vmw_thp_init(struct ttm_mem_type_manager *man, + unsigned long p_size) +{ + struct vmw_thp_manager *rman; + + rman = kzalloc(sizeof(*rman), GFP_KERNEL); + if (!rman) + return -ENOMEM; + + drm_mm_init(&rman->mm, 0, p_size); + spin_lock_init(&rman->lock); + man->priv = rman; + return 0; +} + +static int vmw_thp_takedown(struct ttm_mem_type_manager *man) +{ + struct vmw_thp_manager *rman = (struct vmw_thp_manager *) man->priv; + struct drm_mm *mm = &rman->mm; + + spin_lock(&rman->lock); + if (drm_mm_clean(mm)) { + drm_mm_takedown(mm); + spin_unlock(&rman->lock); + kfree(rman); + man->priv = NULL; + return 0; + } + spin_unlock(&rman->lock); + return -EBUSY; +} + +static void vmw_thp_debug(struct ttm_mem_type_manager *man, + struct drm_printer *printer) +{ + struct vmw_thp_manager *rman = (struct vmw_thp_manager *) man->priv; + + spin_lock(&rman->lock); + drm_mm_print(&rman->mm, printer); + spin_unlock(&rman->lock); +} + +const struct ttm_mem_type_manager_func vmw_thp_func = { + .init = vmw_thp_init, + .takedown = vmw_thp_takedown, + .get_node = vmw_thp_get_node, + .put_node = vmw_thp_put_node, + .debug = vmw_thp_debug +}; -- 2.21.1 _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
next prev parent reply other threads:[~2020-03-24 20:12 UTC|newest] Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-03-24 20:11 [PATCH v7 0/9] Huge page-table entries for TTM Thomas Hellström (VMware) 2020-03-24 20:11 ` Thomas Hellström (VMware) 2020-03-24 20:11 ` [PATCH v7 1/9] fs: Constify vma argument to vma_is_dax Thomas Hellström (VMware) 2020-03-24 20:11 ` Thomas Hellström (VMware) 2020-03-26 2:49 ` Matthew Wilcox 2020-03-26 2:49 ` Matthew Wilcox 2020-03-26 7:19 ` Pankaj Gupta 2020-03-26 7:19 ` Pankaj Gupta 2020-03-24 20:11 ` [PATCH v7 2/9] mm: Introduce vma_is_special_huge Thomas Hellström (VMware) 2020-03-24 20:11 ` Thomas Hellström (VMware) 2020-03-24 20:11 ` [PATCH v7 3/9] mm: Split huge pages on write-notify or COW Thomas Hellström (VMware) 2020-03-24 20:11 ` Thomas Hellström (VMware) 2020-03-24 20:11 ` [PATCH v7 4/9] mm: Add vmf_insert_pfn_xxx_prot() for huge page-table entries Thomas Hellström (VMware) 2020-03-24 20:11 ` Thomas Hellström (VMware) 2020-03-24 20:11 ` [PATCH v7 5/9] drm/ttm, drm/vmwgfx: Support huge TTM pagefaults Thomas Hellström (VMware) 2020-03-24 20:11 ` Thomas Hellström (VMware) 2020-03-24 20:11 ` [PATCH v7 6/9] drm/vmwgfx: Support huge page faults Thomas Hellström (VMware) 2020-03-24 20:11 ` Thomas Hellström (VMware) 2020-03-24 20:11 ` [PATCH v7 7/9] drm: Add a drm_get_unmapped_area() helper Thomas Hellström (VMware) 2020-03-24 20:11 ` Thomas Hellström (VMware) 2020-03-24 20:11 ` Thomas Hellström (VMware) [this message] 2020-03-24 20:11 ` [PATCH v7 8/9] drm/vmwgfx: Introduce a huge page aligning TTM range manager Thomas Hellström (VMware) 2020-03-24 20:11 ` [PATCH v7 9/9] drm/vmwgfx: Hook up the helpers to align buffer objects Thomas Hellström (VMware) 2020-03-24 20:11 ` Thomas Hellström (VMware)
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200324201123.3118-9-thomas_os@shipmail.org \ --to=thomas_os@shipmail.org \ --cc=akpm@linux-foundation.org \ --cc=christian.koenig@amd.com \ --cc=dan.j.williams@intel.com \ --cc=dri-devel@lists.freedesktop.org \ --cc=jglisse@redhat.com \ --cc=kirill.shutemov@linux.intel.com \ --cc=linux-graphics-maintainer@vmware.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=mhocko@suse.com \ --cc=pv-drivers@vmware.com \ --cc=rcampbell@nvidia.com \ --cc=sroland@vmware.com \ --cc=willy@infradead.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.