[PATCH 1/2] drm/amdgpu: resize VRAM BAR for CPU access on gfx10

[PATCH 1/2] drm/amdgpu: resize VRAM BAR for CPU access on gfx10

[Alan Swanson]

Try to resize BAR0 to let CPU access all of VRAM on Navi. Syncs
code with previous gfx generations from commit d6895ad39f3b39
("drm/amdgpu: resize VRAM BAR for CPU access v6").

Signed-off-by: Alan Swanson <reiver@improbability.net>
---
Unfortunately cannot test this with my RX5700 on my AMD X470
motherboard as its BIOS "Above 4G decoding" option does not
remap the PCI root bus. Trying "pci=nocrs" also unsuccessful.
However, I assume it's just absent due to missed commit
overlap during gfx10 bring up and should work.

 drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
index edaa50d85..ba2b7ac0c 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
@@ -686,17 +686,23 @@ static void gmc_v10_0_vram_gtt_location(struct amdgpu_device *adev,
  */
 static int gmc_v10_0_mc_init(struct amdgpu_device *adev)
 {
-	/* Could aper size report 0 ? */
-	adev->gmc.aper_base = pci_resource_start(adev->pdev, 0);
-	adev->gmc.aper_size = pci_resource_len(adev->pdev, 0);
+	int r;
 
 	/* size in MB on si */
 	adev->gmc.mc_vram_size =
 		adev->nbio.funcs->get_memsize(adev) * 1024ULL * 1024ULL;
 	adev->gmc.real_vram_size = adev->gmc.mc_vram_size;
-	adev->gmc.visible_vram_size = adev->gmc.aper_size;
+
+	if (!(adev->flags & AMD_IS_APU)) {
+		r = amdgpu_device_resize_fb_bar(adev);
+		if (r)
+			return r;
+	}
+	adev->gmc.aper_base = pci_resource_start(adev->pdev, 0);
+	adev->gmc.aper_size = pci_resource_len(adev->pdev, 0);
 
 	/* In case the PCI BAR is larger than the actual amount of vram */
+	adev->gmc.visible_vram_size = adev->gmc.aper_size;
 	if (adev->gmc.visible_vram_size > adev->gmc.real_vram_size)
 		adev->gmc.visible_vram_size = adev->gmc.real_vram_size;
 
-- 
2.26.2

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 2/2] drm/amdgpu: Advise if unable to resize BAR

[Alan Swanson]

Even with the "Above 4G decoding" (or similar) BIOS option enabled,
many BIOS do not assign the PCI root bus a 64-bit address space.

If available, "MMIOH Base" and "MMIO High Size" (or similar) BIOS
options should allow mapping to the desired address spaces.

Signed-off-by: Alan Swanson <reiver@improbability.net>
---
Useful to know why bar resizing isn't happening.

 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 2f0e8da7b..39a7f7212 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -919,8 +919,10 @@ int amdgpu_device_resize_fb_bar(struct amdgpu_device *adev)
 	}
 
 	/* Trying to resize is pointless without a root hub window above 4GB */
-	if (!res)
+	if (!res) {
+		DRM_INFO("Unable to resize BAR as PCI bus address space below 4GB.");
 		return 0;
+	}
 
 	/* Disable memory decoding while we change the BAR addresses and size */
 	pci_read_config_word(adev->pdev, PCI_COMMAND, &cmd);
-- 
2.26.2

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 1/2] drm/amdgpu: resize VRAM BAR for CPU access on gfx10

[Alex Deucher]

On Thu, May 21, 2020 at 4:48 PM Alan Swanson <reiver@improbability.net> wrote:
>
> Try to resize BAR0 to let CPU access all of VRAM on Navi. Syncs
> code with previous gfx generations from commit d6895ad39f3b39
> ("drm/amdgpu: resize VRAM BAR for CPU access v6").
>
> Signed-off-by: Alan Swanson <reiver@improbability.net>

Tested and applied.  Thanks!

Alex

> ---
> Unfortunately cannot test this with my RX5700 on my AMD X470
> motherboard as its BIOS "Above 4G decoding" option does not
> remap the PCI root bus. Trying "pci=nocrs" also unsuccessful.
> However, I assume it's just absent due to missed commit
> overlap during gfx10 bring up and should work.
>
>  drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c | 14 ++++++++++----
>  1 file changed, 10 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
> index edaa50d85..ba2b7ac0c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
> @@ -686,17 +686,23 @@ static void gmc_v10_0_vram_gtt_location(struct amdgpu_device *adev,
>   */
>  static int gmc_v10_0_mc_init(struct amdgpu_device *adev)
>  {
> -       /* Could aper size report 0 ? */
> -       adev->gmc.aper_base = pci_resource_start(adev->pdev, 0);
> -       adev->gmc.aper_size = pci_resource_len(adev->pdev, 0);
> +       int r;
>
>         /* size in MB on si */
>         adev->gmc.mc_vram_size =
>                 adev->nbio.funcs->get_memsize(adev) * 1024ULL * 1024ULL;
>         adev->gmc.real_vram_size = adev->gmc.mc_vram_size;
> -       adev->gmc.visible_vram_size = adev->gmc.aper_size;
> +
> +       if (!(adev->flags & AMD_IS_APU)) {
> +               r = amdgpu_device_resize_fb_bar(adev);
> +               if (r)
> +                       return r;
> +       }
> +       adev->gmc.aper_base = pci_resource_start(adev->pdev, 0);
> +       adev->gmc.aper_size = pci_resource_len(adev->pdev, 0);
>
>         /* In case the PCI BAR is larger than the actual amount of vram */
> +       adev->gmc.visible_vram_size = adev->gmc.aper_size;
>         if (adev->gmc.visible_vram_size > adev->gmc.real_vram_size)
>                 adev->gmc.visible_vram_size = adev->gmc.real_vram_size;
>
> --
> 2.26.2
>
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 2/2] drm/amdgpu: Advise if unable to resize BAR

[Alex Deucher]

On Thu, May 21, 2020 at 4:45 PM Alan Swanson <reiver@improbability.net> wrote:
>
> Even with the "Above 4G decoding" (or similar) BIOS option enabled,
> many BIOS do not assign the PCI root bus a 64-bit address space.
>
> If available, "MMIOH Base" and "MMIO High Size" (or similar) BIOS
> options should allow mapping to the desired address spaces.
>
> Signed-off-by: Alan Swanson <reiver@improbability.net>
> ---
> Useful to know why bar resizing isn't happening.

This will spam a lot of people and probably cause confusion.  I'd
prefer to drop this one.

Alex


>
>  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index 2f0e8da7b..39a7f7212 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -919,8 +919,10 @@ int amdgpu_device_resize_fb_bar(struct amdgpu_device *adev)
>         }
>
>         /* Trying to resize is pointless without a root hub window above 4GB */
> -       if (!res)
> +       if (!res) {
> +               DRM_INFO("Unable to resize BAR as PCI bus address space below 4GB.");
>                 return 0;
> +       }
>
>         /* Disable memory decoding while we change the BAR addresses and size */
>         pci_read_config_word(adev->pdev, PCI_COMMAND, &cmd);
> --
> 2.26.2
>
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 2/2] drm/amdgpu: Advise if unable to resize BAR

[Christian König]

Am 21.05.20 um 23:32 schrieb Alex Deucher:
> On Thu, May 21, 2020 at 4:45 PM Alan Swanson <reiver@improbability.net> wrote:
>> Even with the "Above 4G decoding" (or similar) BIOS option enabled,
>> many BIOS do not assign the PCI root bus a 64-bit address space.
>>
>> If available, "MMIOH Base" and "MMIO High Size" (or similar) BIOS
>> options should allow mapping to the desired address spaces.
>>
>> Signed-off-by: Alan Swanson <reiver@improbability.net>
>> ---
>> Useful to know why bar resizing isn't happening.
> This will spam a lot of people and probably cause confusion.  I'd
> prefer to drop this one.

Agreed, you can just look at /proc/iomem to figure out if resources 
above 4GB are available or not.

Christian.

>
> Alex
>
>
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 +++-
>>   1 file changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> index 2f0e8da7b..39a7f7212 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> @@ -919,8 +919,10 @@ int amdgpu_device_resize_fb_bar(struct amdgpu_device *adev)
>>          }
>>
>>          /* Trying to resize is pointless without a root hub window above 4GB */
>> -       if (!res)
>> +       if (!res) {
>> +               DRM_INFO("Unable to resize BAR as PCI bus address space below 4GB.");
>>                  return 0;
>> +       }
>>
>>          /* Disable memory decoding while we change the BAR addresses and size */
>>          pci_read_config_word(adev->pdev, PCI_COMMAND, &cmd);
>> --
>> 2.26.2
>>
>> _______________________________________________
>> amd-gfx mailing list
>> amd-gfx@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx