From: "Zhou, Peng Ju" <PengJu.Zhou@amd.com>
To: "Kuehling, Felix" <Felix.Kuehling@amd.com>,
"amd-gfx@lists.freedesktop.org" <amd-gfx@lists.freedesktop.org>
Subject: RE: [PATCH 1/8] drm/amdgpu: change MMHUB register access from MMIO to RLCG
Date: Thu, 15 Apr 2021 07:25:38 +0000 [thread overview]
Message-ID: <DM6PR12MB4371D57D363B2E70042D62F2F84D9@DM6PR12MB4371.namprd12.prod.outlook.com> (raw)
In-Reply-To: <DM6PR12MB4371E4C410C5940878820261F8739@DM6PR12MB4371.namprd12.prod.outlook.com>
[AMD Official Use Only - Internal Distribution Only]
Hi Felix
Thanks for your proposal about " Given the number of call-sites being modified in this patch series"
We discussed internally, and have following concern:
1. expose our ranges to opensource.
2. lost the orthogonality in our software stack.
So we want to keep our original workaround.
Do you agree?
----------------------------------------------------------------------
BW
Pengju Zhou
-----Original Message-----
From: Zhou, Peng Ju
Sent: Friday, April 9, 2021 11:36 AM
To: Kuehling, Felix <Felix.Kuehling@amd.com>; amd-gfx@lists.freedesktop.org
Subject: RE: [PATCH 1/8] drm/amdgpu: change MMHUB register access from MMIO to RLCG
[AMD Official Use Only - Internal Distribution Only]
Hi Felix
That is a great idea, I will try it.
----------------------------------------------------------------------
BW
Pengju Zhou
-----Original Message-----
From: Kuehling, Felix <Felix.Kuehling@amd.com>
Sent: Thursday, April 8, 2021 10:58 PM
To: Zhou, Peng Ju <PengJu.Zhou@amd.com>; amd-gfx@lists.freedesktop.org
Subject: Re: [PATCH 1/8] drm/amdgpu: change MMHUB register access from MMIO to RLCG
Given the number of call-sites being modified in this patch series, would it be easier (and more maintainable) to change the behaviour or the regular register macros and add NO_RLC versions for the exceptions, similar to NO_KIQ?
Regards,
Felix
Am 2021-04-08 um 6:21 a.m. schrieb Peng Ju Zhou:
> From: pengzhou <PengJu.Zhou@amd.com>
>
> In SRIOV environment, KMD should access MMHUB registers with RLCG if
> MMHUB indirect access bit enabled.
>
> Change MMHUB register access from MMIO to RLCG.
>
> Signed-off-by: pengzhou <PengJu.Zhou@amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c | 12 ++++++--
> drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c | 39 +++++++++++++------------
> 2 files changed, 29 insertions(+), 22 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
> b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
> index 2bfd620576f2..42818c40d08c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
> @@ -38,6 +38,7 @@
> #include "soc15.h"
> #include "soc15d.h"
> #include "soc15_common.h"
> +#include "gc/gc_10_1_0_offset.h"
>
> #include "nbio_v2_3.h"
>
> @@ -253,7 +254,10 @@ static void gmc_v10_0_flush_vm_hub(struct amdgpu_device *adev, uint32_t vmid,
> DRM_ERROR("Timeout waiting for sem acquire in VM flush!\n");
> }
>
> - WREG32_NO_KIQ(hub->vm_inv_eng0_req + hub->eng_distance * eng, inv_req);
> + if (vmhub == AMDGPU_MMHUB_0)
> + WREG32_RLC_NO_KIQ((hub->vm_inv_eng0_req + eng), inv_req);
> + else
> + WREG32_NO_KIQ(hub->vm_inv_eng0_req + eng, inv_req);
>
> /*
> * Issue a dummy read to wait for the ACK register to be cleared @@
> -280,8 +284,10 @@ static void gmc_v10_0_flush_vm_hub(struct amdgpu_device *adev, uint32_t vmid,
> * add semaphore release after invalidation,
> * write with 0 means semaphore release
> */
> - WREG32_NO_KIQ(hub->vm_inv_eng0_sem +
> - hub->eng_distance * eng, 0);
> + if (vmhub == AMDGPU_MMHUB_0)
> + WREG32_RLC_NO_KIQ(hub->vm_inv_eng0_sem + eng, 0);
> + else
> + WREG32_NO_KIQ(hub->vm_inv_eng0_sem + eng, 0);
>
> spin_unlock(&adev->gmc.invalidate_lock);
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c
> b/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c
> index da7edd1ed6b2..e8ecdf383192 100644
> --- a/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c
> @@ -29,6 +29,7 @@
> #include "mmhub/mmhub_2_0_0_default.h"
> #include "navi10_enum.h"
>
> +#include "gc/gc_10_1_0_offset.h"
> #include "soc15_common.h"
>
> #define mmMM_ATC_L2_MISC_CG_Sienna_Cichlid 0x064d
> @@ -165,11 +166,11 @@ static void mmhub_v2_0_setup_vm_pt_regs(struct
> amdgpu_device *adev, uint32_t vmi {
> struct amdgpu_vmhub *hub = &adev->vmhub[AMDGPU_MMHUB_0];
>
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_CONTEXT0_PAGE_TABLE_BASE_ADDR_LO32,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0,
> +mmMMVM_CONTEXT0_PAGE_TABLE_BASE_ADDR_LO32,
> hub->ctx_addr_distance * vmid,
> lower_32_bits(page_table_base));
>
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_CONTEXT0_PAGE_TABLE_BASE_ADDR_HI32,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0,
> +mmMMVM_CONTEXT0_PAGE_TABLE_BASE_ADDR_HI32,
> hub->ctx_addr_distance * vmid,
> upper_32_bits(page_table_base)); } @@ -180,14 +181,14 @@
> static void mmhub_v2_0_init_gart_aperture_regs(struct amdgpu_device
> *adev)
>
> mmhub_v2_0_setup_vm_pt_regs(adev, 0, pt_base);
>
> - WREG32_SOC15(MMHUB, 0, mmMMVM_CONTEXT0_PAGE_TABLE_START_ADDR_LO32,
> + WREG32_SOC15_RLC(MMHUB, 0,
> +mmMMVM_CONTEXT0_PAGE_TABLE_START_ADDR_LO32,
> (u32)(adev->gmc.gart_start >> 12));
> - WREG32_SOC15(MMHUB, 0, mmMMVM_CONTEXT0_PAGE_TABLE_START_ADDR_HI32,
> + WREG32_SOC15_RLC(MMHUB, 0,
> +mmMMVM_CONTEXT0_PAGE_TABLE_START_ADDR_HI32,
> (u32)(adev->gmc.gart_start >> 44));
>
> - WREG32_SOC15(MMHUB, 0, mmMMVM_CONTEXT0_PAGE_TABLE_END_ADDR_LO32,
> + WREG32_SOC15_RLC(MMHUB, 0, mmMMVM_CONTEXT0_PAGE_TABLE_END_ADDR_LO32,
> (u32)(adev->gmc.gart_end >> 12));
> - WREG32_SOC15(MMHUB, 0, mmMMVM_CONTEXT0_PAGE_TABLE_END_ADDR_HI32,
> + WREG32_SOC15_RLC(MMHUB, 0, mmMMVM_CONTEXT0_PAGE_TABLE_END_ADDR_HI32,
> (u32)(adev->gmc.gart_end >> 44)); }
>
> @@ -197,9 +198,9 @@ static void mmhub_v2_0_init_system_aperture_regs(struct amdgpu_device *adev)
> uint32_t tmp;
>
> /* Program the AGP BAR */
> - WREG32_SOC15(MMHUB, 0, mmMMMC_VM_AGP_BASE, 0);
> - WREG32_SOC15(MMHUB, 0, mmMMMC_VM_AGP_BOT, adev->gmc.agp_start >> 24);
> - WREG32_SOC15(MMHUB, 0, mmMMMC_VM_AGP_TOP, adev->gmc.agp_end >> 24);
> + WREG32_SOC15_RLC(MMHUB, 0, mmMMMC_VM_AGP_BASE, 0);
> + WREG32_SOC15_RLC(MMHUB, 0, mmMMMC_VM_AGP_BOT, adev->gmc.agp_start >> 24);
> + WREG32_SOC15_RLC(MMHUB, 0, mmMMMC_VM_AGP_TOP, adev->gmc.agp_end >>
> +24);
>
> if (!amdgpu_sriov_vf(adev)) {
> /* Program the system aperture low logical page number. */ @@
> -304,12 +305,12 @@ static void mmhub_v2_0_enable_system_domain(struct
> amdgpu_device *adev) {
> uint32_t tmp;
>
> - tmp = RREG32_SOC15(MMHUB, 0, mmMMVM_CONTEXT0_CNTL);
> + tmp = RREG32_SOC15_RLC(MMHUB, 0, mmMMVM_CONTEXT0_CNTL);
> tmp = REG_SET_FIELD(tmp, MMVM_CONTEXT0_CNTL, ENABLE_CONTEXT, 1);
> tmp = REG_SET_FIELD(tmp, MMVM_CONTEXT0_CNTL, PAGE_TABLE_DEPTH, 0);
> tmp = REG_SET_FIELD(tmp, MMVM_CONTEXT0_CNTL,
> RETRY_PERMISSION_OR_INVALID_PAGE_FAULT, 0);
> - WREG32_SOC15(MMHUB, 0, mmMMVM_CONTEXT0_CNTL, tmp);
> + WREG32_SOC15_RLC(MMHUB, 0, mmMMVM_CONTEXT0_CNTL, tmp);
> }
>
> static void mmhub_v2_0_disable_identity_aperture(struct amdgpu_device
> *adev) @@ -371,16 +372,16 @@ static void mmhub_v2_0_setup_vmid_config(struct amdgpu_device *adev)
> tmp = REG_SET_FIELD(tmp, MMVM_CONTEXT1_CNTL,
> RETRY_PERMISSION_OR_INVALID_PAGE_FAULT,
> !adev->gmc.noretry);
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_CONTEXT1_CNTL,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0, mmMMVM_CONTEXT1_CNTL,
> i * hub->ctx_distance, tmp);
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_CONTEXT1_PAGE_TABLE_START_ADDR_LO32,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0,
> +mmMMVM_CONTEXT1_PAGE_TABLE_START_ADDR_LO32,
> i * hub->ctx_addr_distance, 0);
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_CONTEXT1_PAGE_TABLE_START_ADDR_HI32,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0,
> +mmMMVM_CONTEXT1_PAGE_TABLE_START_ADDR_HI32,
> i * hub->ctx_addr_distance, 0);
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_CONTEXT1_PAGE_TABLE_END_ADDR_LO32,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0,
> +mmMMVM_CONTEXT1_PAGE_TABLE_END_ADDR_LO32,
> i * hub->ctx_addr_distance,
> lower_32_bits(adev->vm_manager.max_pfn - 1));
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_CONTEXT1_PAGE_TABLE_END_ADDR_HI32,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0,
> +mmMMVM_CONTEXT1_PAGE_TABLE_END_ADDR_HI32,
> i * hub->ctx_addr_distance,
> upper_32_bits(adev->vm_manager.max_pfn - 1));
> }
> @@ -392,9 +393,9 @@ static void mmhub_v2_0_program_invalidation(struct amdgpu_device *adev)
> unsigned i;
>
> for (i = 0; i < 18; ++i) {
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_INVALIDATE_ENG0_ADDR_RANGE_LO32,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0,
> +mmMMVM_INVALIDATE_ENG0_ADDR_RANGE_LO32,
> i * hub->eng_addr_distance, 0xffffffff);
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_INVALIDATE_ENG0_ADDR_RANGE_HI32,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0,
> +mmMMVM_INVALIDATE_ENG0_ADDR_RANGE_HI32,
> i * hub->eng_addr_distance, 0x1f);
> }
> }
> @@ -423,7 +424,7 @@ static void mmhub_v2_0_gart_disable(struct
> amdgpu_device *adev)
>
> /* Disable all tables */
> for (i = 0; i < AMDGPU_NUM_VMID; i++)
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_CONTEXT0_CNTL,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0, mmMMVM_CONTEXT0_CNTL,
> i * hub->ctx_distance, 0);
>
> /* Setup TLB control */
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
next prev parent reply other threads:[~2021-04-15 7:25 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-08 10:21 [PATCH 1/8] drm/amdgpu: change MMHUB register access from MMIO to RLCG Peng Ju Zhou
2021-04-08 10:21 ` [PATCH 2/8] drm/amdgpu: Change GC(KFD/GFX) " Peng Ju Zhou
2021-04-08 10:22 ` [PATCH 3/8] drm/amdgpu: Change GC(SDMA) " Peng Ju Zhou
2021-04-08 10:22 ` [PATCH 4/8] drm/amdgpu: Change GC(GFX/GFXHUB) " Peng Ju Zhou
2021-04-08 10:22 ` [PATCH 5/8] drm/amdgpu: Change GC(GMC/GFXHUB/..) " Peng Ju Zhou
2021-04-08 10:22 ` [PATCH 6/8] drm/amdgpu: Change GC(GFXHUB) " Peng Ju Zhou
2021-04-08 10:22 ` [PATCH 7/8] drm/amdgpu: Change GC(GFX) " Peng Ju Zhou
2021-04-08 10:22 ` [PATCH 8/8] drm/amdgpu: Use PSP to program IH_RB_CNTL* registers Peng Ju Zhou
2021-04-08 14:58 ` [PATCH 1/8] drm/amdgpu: change MMHUB register access from MMIO to RLCG Felix Kuehling
2021-04-09 3:35 ` Zhou, Peng Ju
2021-04-15 7:25 ` Zhou, Peng Ju [this message]
2021-04-15 15:47 ` Felix Kuehling
2021-04-16 3:18 ` Zhou, Peng Ju
-- strict thread matches above, loose matches on Subject: below --
2021-04-08 5:32 Peng Ju Zhou
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DM6PR12MB4371D57D363B2E70042D62F2F84D9@DM6PR12MB4371.namprd12.prod.outlook.com \
--to=pengju.zhou@amd.com \
--cc=Felix.Kuehling@amd.com \
--cc=amd-gfx@lists.freedesktop.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.