From: Felix Kuehling <Felix.Kuehling@amd.com> To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Cc: Alex Sierra <alex.sierra@amd.com> Subject: [PATCH 12/44] drm/amdkfd: add xnack enabled flag to kfd_process Date: Mon, 22 Mar 2021 06:58:28 -0400 [thread overview] Message-ID: <20210322105900.14068-13-Felix.Kuehling@amd.com> (raw) In-Reply-To: <20210322105900.14068-1-Felix.Kuehling@amd.com> From: Alex Sierra <alex.sierra@amd.com> This flag is useful at cpu invalidation page table decision. Between select queue eviction or page fault. Signed-off-by: Alex Sierra <alex.sierra@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> --- drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 4 +++ drivers/gpu/drm/amd/amdkfd/kfd_process.c | 36 ++++++++++++++++++++++++ 2 files changed, 40 insertions(+) diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h index 7d70af26b5c7..d66430740e52 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h @@ -824,6 +824,8 @@ struct kfd_process { /* shared virtual memory registered by this process */ struct svm_range_list svms; bool svm_disabled; + + bool xnack_enabled; }; #define KFD_PROCESS_TABLE_SIZE 5 /* bits: 32 entries */ @@ -877,6 +879,8 @@ struct kfd_process_device *kfd_get_process_device_data(struct kfd_dev *dev, struct kfd_process_device *kfd_create_process_device_data(struct kfd_dev *dev, struct kfd_process *p); +bool kfd_process_xnack_supported(struct kfd_process *p); + int kfd_reserved_mem_mmap(struct kfd_dev *dev, struct kfd_process *process, struct vm_area_struct *vma); diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c index 321895d7555a..c8479f6bd68c 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c @@ -1193,6 +1193,39 @@ void kfd_process_set_trap_handler(struct qcm_process_device *qpd, } } +bool kfd_process_xnack_supported(struct kfd_process *p) +{ + int i; + + /* On most GFXv9 GPUs, the retry mode in the SQ must match the + * boot time retry setting. Mixing processes with different + * XNACK/retry settings can hang the GPU. + * + * Different GPUs can have different noretry settings depending + * on HW bugs or limitations. We need to find at least one + * XNACK mode for this process that's compatible with all GPUs. + * Fortunately GPUs with retry enabled (noretry=0) can run code + * built for XNACK-off. On GFXv9 it may perform slower. + * + * Therefore applications built for XNACK-off can always be + * supported and will be our fallback if any GPU does not + * support retry. + */ + for (i = 0; i < p->n_pdds; i++) { + struct kfd_dev *dev = p->pdds[i]->dev; + + /* Only consider GFXv9 and higher GPUs. Older GPUs don't + * support the SVM APIs and don't need to be considered + * for the XNACK mode selection. + */ + if (dev->device_info->asic_family >= CHIP_VEGA10 && + dev->noretry) + return false; + } + + return true; +} + /* * On return the kfd_process is fully operational and will be freed when the * mm is released @@ -1232,6 +1265,9 @@ static struct kfd_process *create_process(const struct task_struct *thread) if (err != 0) goto err_init_apertures; + /* Check XNACK support after PDDs are created in kfd_init_apertures */ + process->xnack_enabled = kfd_process_xnack_supported(process); + err = svm_range_list_init(process); if (err) goto err_init_svm_range_list; -- 2.31.0 _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
WARNING: multiple messages have this Message-ID (diff)
From: Felix Kuehling <Felix.Kuehling@amd.com> To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Cc: Alex Sierra <alex.sierra@amd.com> Subject: [PATCH 12/44] drm/amdkfd: add xnack enabled flag to kfd_process Date: Mon, 22 Mar 2021 06:58:28 -0400 [thread overview] Message-ID: <20210322105900.14068-13-Felix.Kuehling@amd.com> (raw) In-Reply-To: <20210322105900.14068-1-Felix.Kuehling@amd.com> From: Alex Sierra <alex.sierra@amd.com> This flag is useful at cpu invalidation page table decision. Between select queue eviction or page fault. Signed-off-by: Alex Sierra <alex.sierra@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> --- drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 4 +++ drivers/gpu/drm/amd/amdkfd/kfd_process.c | 36 ++++++++++++++++++++++++ 2 files changed, 40 insertions(+) diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h index 7d70af26b5c7..d66430740e52 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h @@ -824,6 +824,8 @@ struct kfd_process { /* shared virtual memory registered by this process */ struct svm_range_list svms; bool svm_disabled; + + bool xnack_enabled; }; #define KFD_PROCESS_TABLE_SIZE 5 /* bits: 32 entries */ @@ -877,6 +879,8 @@ struct kfd_process_device *kfd_get_process_device_data(struct kfd_dev *dev, struct kfd_process_device *kfd_create_process_device_data(struct kfd_dev *dev, struct kfd_process *p); +bool kfd_process_xnack_supported(struct kfd_process *p); + int kfd_reserved_mem_mmap(struct kfd_dev *dev, struct kfd_process *process, struct vm_area_struct *vma); diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c index 321895d7555a..c8479f6bd68c 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c @@ -1193,6 +1193,39 @@ void kfd_process_set_trap_handler(struct qcm_process_device *qpd, } } +bool kfd_process_xnack_supported(struct kfd_process *p) +{ + int i; + + /* On most GFXv9 GPUs, the retry mode in the SQ must match the + * boot time retry setting. Mixing processes with different + * XNACK/retry settings can hang the GPU. + * + * Different GPUs can have different noretry settings depending + * on HW bugs or limitations. We need to find at least one + * XNACK mode for this process that's compatible with all GPUs. + * Fortunately GPUs with retry enabled (noretry=0) can run code + * built for XNACK-off. On GFXv9 it may perform slower. + * + * Therefore applications built for XNACK-off can always be + * supported and will be our fallback if any GPU does not + * support retry. + */ + for (i = 0; i < p->n_pdds; i++) { + struct kfd_dev *dev = p->pdds[i]->dev; + + /* Only consider GFXv9 and higher GPUs. Older GPUs don't + * support the SVM APIs and don't need to be considered + * for the XNACK mode selection. + */ + if (dev->device_info->asic_family >= CHIP_VEGA10 && + dev->noretry) + return false; + } + + return true; +} + /* * On return the kfd_process is fully operational and will be freed when the * mm is released @@ -1232,6 +1265,9 @@ static struct kfd_process *create_process(const struct task_struct *thread) if (err != 0) goto err_init_apertures; + /* Check XNACK support after PDDs are created in kfd_init_apertures */ + process->xnack_enabled = kfd_process_xnack_supported(process); + err = svm_range_list_init(process); if (err) goto err_init_svm_range_list; -- 2.31.0 _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
next prev parent reply other threads:[~2021-03-22 11:07 UTC|newest] Thread overview: 100+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-03-22 10:58 [PATCH 00/44] Add HMM-based SVM memory manager to KFD v2 Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 01/44] drm/amdgpu: replace per_device_list by array Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-23 14:30 ` Kim, Jonathan 2021-03-23 14:30 ` Kim, Jonathan 2021-03-22 10:58 ` [PATCH 02/44] drm/amdkfd: helper to convert gpu id and idx Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 03/44] drm/amdkfd: add svm ioctl API Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 04/44] drm/amdkfd: register svm range Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 05/44] drm/amdkfd: add svm ioctl GET_ATTR op Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 06/44] drm/amdgpu: add common HMM get pages function Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 07/44] drm/amdkfd: validate svm range system memory Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 08/44] drm/amdkfd: deregister svm range Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 09/44] drm/amdgpu: export vm update mapping interface Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 10/44] drm/amdkfd: map svm range to GPUs Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 11/44] drm/amdkfd: svm range eviction and restore Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling [this message] 2021-03-22 10:58 ` [PATCH 12/44] drm/amdkfd: add xnack enabled flag to kfd_process Felix Kuehling 2021-03-22 10:58 ` [PATCH 13/44] drm/amdkfd: add ioctl to configure and query xnack retries Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 14/44] drm/amdkfd: register HMM device private zone Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 15/44] drm/amdkfd: validate vram svm range from TTM Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 16/44] drm/amdkfd: support xgmi same hive mapping Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 17/44] drm/amdkfd: copy memory through gart table Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 18/44] drm/amdkfd: HMM migrate ram to vram Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 19/44] drm/amdkfd: HMM migrate vram to ram Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 20/44] drm/amdkfd: invalidate tables on page retry fault Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 21/44] drm/amdgpu: enable 48-bit IH timestamp counter Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 22/44] drm/amdkfd: page table restore through svm API Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 23/44] drm/amdkfd: SVM API call to restore page tables Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 24/44] drm/amdkfd: add svm_bo reference for eviction fence Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 25/44] drm/amdgpu: add param bit flag to create SVM BOs Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 26/44] drm/amdkfd: add svm_bo eviction mechanism support Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 27/44] drm/amdgpu: svm bo enable_signal call condition Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 28/44] drm/amdgpu: add svm_bo eviction to enable_signal cb Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 29/44] drm/amdgpu: reserve fence slot to update page table Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 13:32 ` Christian König 2021-03-22 13:32 ` Christian König 2021-03-22 10:58 ` [PATCH 30/44] drm/amdkfd: refine migration policy with xnack on Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 31/44] drm/amdkfd: add svm range validate timestamp Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 32/44] drm/amdkfd: multiple gpu migrate vram to vram Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 33/44] drm/amdkfd: Add SVM API support capability bits Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 34/44] drm/amdkfd: Fix dma unmapping Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 35/44] drm/amdkfd: Call mutex_destroy Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 36/44] drm/amdkfd: Fix spurious restore failures Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 37/44] drm/amdkfd: Fix svm_bo_list locking in eviction worker Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 38/44] drm/amdkfd: Simplify split_by_granularity Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 39/44] drm/amdkfd: Point out several race conditions Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 40/44] drm/amdkfd: Return pdd from kfd_process_device_from_gduid Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 41/44] drm/amdkfd: Remove broken deferred mapping Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 42/44] drm/amdkfd: Allow invalid pages in migration.src Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:58 ` [PATCH 43/44] drm/amdkfd: Correct locking during migration and mapping Felix Kuehling 2021-03-22 10:58 ` Felix Kuehling 2021-03-22 10:59 ` [PATCH 44/44] drm/amdkfd: Nested locking and invalidation of child ranges Felix Kuehling 2021-03-22 10:59 ` Felix Kuehling 2021-03-22 14:15 ` [PATCH 00/44] Add HMM-based SVM memory manager to KFD v2 Daniel Vetter 2021-03-22 14:15 ` Daniel Vetter 2021-03-22 16:06 ` Felix Kuehling 2021-03-22 16:06 ` Felix Kuehling 2021-03-22 17:04 ` Daniel Vetter 2021-03-22 17:04 ` Daniel Vetter
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20210322105900.14068-13-Felix.Kuehling@amd.com \ --to=felix.kuehling@amd.com \ --cc=alex.sierra@amd.com \ --cc=amd-gfx@lists.freedesktop.org \ --cc=dri-devel@lists.freedesktop.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.