[PATCH v2 1/2] drm/amd/debugfs: Expose GFXOFF state to userspace

[PATCH v2 1/2] drm/amd/debugfs: Expose GFXOFF state to userspace

[André Almeida]

GFXOFF has two different "state" values: one to define if the GPU is
allowed/disallowed to enter GFXOFF, usually called state; and another
one to define if currently GFXOFF is being used, usually called status.
Even when GFXOFF is allowed, GPU firmware can decide to not used it
accordingly to the GPU load.

Userspace can allow/disallow GPUs to enter into GFXOFF via debugfs. The
kernel maintains a counter of requests for GFXOFF (gfx_off_req_count)
that should be decreased to allow GFXOFF and increased to disallow.

The issue with this interface is that userspace can't be sure if GFXOFF
is currently allowed. Even by checking amdgpu_gfxoff file, one might get
an ambiguous 2, that means that GPU is currently out of GFXOFF, but that
can be either because it's currently disallowed or because it's allowed
but given the current GPU load it's enabled. Then, userspace needs to
rely on the fact that GFXOFF is enabled by default on boot and to track
this information.

To make userspace life easier and GFXOFF more reliable, return the
current state of GFXOFF to userspace when reading amdgpu_gfxoff with the
same semantics of writing: 0 means not allowed, not 0 means allowed.

Expose the current status of GFXOFF through a new file,
amdgpu_gfxoff_status.

Signed-off-by: André Almeida <andrealmeid@igalia.com>
---
Changes from v1: none

 drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c | 49 ++++++++++++++++++++-
 1 file changed, 47 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c
index f3b3c688e4e7..e2eec985adb3 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c
@@ -1117,13 +1117,50 @@ static ssize_t amdgpu_debugfs_gfxoff_read(struct file *f, char __user *buf,
 	}
 
 	while (size) {
-		uint32_t value;
+		u32 value = adev->gfx.gfx_off_state;
+
+		r = put_user(value, (u32 *)buf);
+		if (r)
+			goto out;
+
+		result += 4;
+		buf += 4;
+		*pos += 4;
+		size -= 4;
+	}
+
+	r = result;
+out:
+	pm_runtime_mark_last_busy(adev_to_drm(adev)->dev);
+	pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
+
+	return r;
+}
+
+static ssize_t amdgpu_debugfs_gfxoff_status_read(struct file *f, char __user *buf,
+						 size_t size, loff_t *pos)
+{
+	struct amdgpu_device *adev = file_inode(f)->i_private;
+	ssize_t result = 0;
+	int r;
+
+	if (size & 0x3 || *pos & 0x3)
+		return -EINVAL;
+
+	r = pm_runtime_get_sync(adev_to_drm(adev)->dev);
+	if (r < 0) {
+		pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
+		return r;
+	}
+
+	while (size) {
+		u32 value;
 
 		r = amdgpu_get_gfx_off_status(adev, &value);
 		if (r)
 			goto out;
 
-		r = put_user(value, (uint32_t *)buf);
+		r = put_user(value, (u32 *)buf);
 		if (r)
 			goto out;
 
@@ -1206,6 +1243,12 @@ static const struct file_operations amdgpu_debugfs_gfxoff_fops = {
 	.llseek = default_llseek
 };
 
+static const struct file_operations amdgpu_debugfs_gfxoff_status_fops = {
+	.owner = THIS_MODULE,
+	.read = amdgpu_debugfs_gfxoff_status_read,
+	.llseek = default_llseek
+};
+
 static const struct file_operations *debugfs_regs[] = {
 	&amdgpu_debugfs_regs_fops,
 	&amdgpu_debugfs_regs2_fops,
@@ -1217,6 +1260,7 @@ static const struct file_operations *debugfs_regs[] = {
 	&amdgpu_debugfs_wave_fops,
 	&amdgpu_debugfs_gpr_fops,
 	&amdgpu_debugfs_gfxoff_fops,
+	&amdgpu_debugfs_gfxoff_status_fops,
 };
 
 static const char *debugfs_regs_names[] = {
@@ -1230,6 +1274,7 @@ static const char *debugfs_regs_names[] = {
 	"amdgpu_wave",
 	"amdgpu_gpr",
 	"amdgpu_gfxoff",
+	"amdgpu_gfxoff_status",
 };
 
 /**
-- 
2.37.0

[PATCH v2 2/2] Documentation/gpu: Add GFXOFF section

[André Almeida]

Add a GFXOFF section at "GPU Power Controls" file, explaining what it is
and how userspace can interact with it.

Signed-off-by: André Almeida <andrealmeid@igalia.com>
---
Changes from v1: file created

 Documentation/gpu/amdgpu/thermal.rst | 41 ++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)

diff --git a/Documentation/gpu/amdgpu/thermal.rst b/Documentation/gpu/amdgpu/thermal.rst
index 8aeb0186c9ef..14c0fb874cf6 100644
--- a/Documentation/gpu/amdgpu/thermal.rst
+++ b/Documentation/gpu/amdgpu/thermal.rst
@@ -63,3 +63,44 @@ gpu_metrics
 
 .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
    :doc: gpu_metrics
+
+GFXOFF
+======
+
+GFXOFF is a feature found in some mobile GPUs that saves power consumption. The
+card's firmware uses RLC (RunList Controller) to power off the gfx engine
+dynamically when there is no workload on gfx pipe and puts gfx into "idle"
+state. GFXOFF is on by default on supported GPUs.
+
+Userspace can interact with GFXOFF through a debugfs interface:
+
+``amdgpu_gfxoff``
+-----------------
+
+Use it to enable/disable GFXOFF, and to check if it's current enabled/disabled::
+
+  $ xxd -l1 -p /sys/kernel/debug/dri/0/amdgpu_gfxoff
+  01
+
+- Write 0 to disable it, and 1 to enable it.
+- Read 0 means it's disabled, 1 it's enabled.
+
+If it's enabled, that means that the GPU is free to enter into GFXOFF mode as
+needed. Disabled means that it will never enter GFXOFF mode.
+
+``amdgpu_gfxoff_status``
+------------------------
+
+Read it to check current GFXOFF's status of a GPU::
+
+  $ xxd -l1 -p /sys/kernel/debug/dri/0/amdgpu_gfxoff_status
+  02
+
+- 0: GPU is in GFXOFF state, the gfx engine is powered down.
+- 1: Transition out of GFXOFF state
+- 2: Not in GFXOFF state
+- 3: Transition into GFXOFF state
+
+If GFXOFF is enabled, the value will be transitioning around [0, 3], always
+getting into 0 when possible. When it's disabled, it's always at 2. Returns
+``-EINVAL`` if it's not supported.
-- 
2.37.0

Re: [PATCH v2 2/2] Documentation/gpu: Add GFXOFF section

[Alex Deucher]

Applied the series with some minor tweaks to the documentation.

Thanks!

Alex

On Thu, Jul 14, 2022 at 3:18 PM André Almeida <andrealmeid@igalia.com> wrote:
>
> Add a GFXOFF section at "GPU Power Controls" file, explaining what it is
> and how userspace can interact with it.
>
> Signed-off-by: André Almeida <andrealmeid@igalia.com>
> ---
> Changes from v1: file created
>
>  Documentation/gpu/amdgpu/thermal.rst | 41 ++++++++++++++++++++++++++++
>  1 file changed, 41 insertions(+)
>
> diff --git a/Documentation/gpu/amdgpu/thermal.rst b/Documentation/gpu/amdgpu/thermal.rst
> index 8aeb0186c9ef..14c0fb874cf6 100644
> --- a/Documentation/gpu/amdgpu/thermal.rst
> +++ b/Documentation/gpu/amdgpu/thermal.rst
> @@ -63,3 +63,44 @@ gpu_metrics
>
>  .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c
>     :doc: gpu_metrics
> +
> +GFXOFF
> +======
> +
> +GFXOFF is a feature found in some mobile GPUs that saves power consumption. The
> +card's firmware uses RLC (RunList Controller) to power off the gfx engine
> +dynamically when there is no workload on gfx pipe and puts gfx into "idle"
> +state. GFXOFF is on by default on supported GPUs.
> +
> +Userspace can interact with GFXOFF through a debugfs interface:
> +
> +``amdgpu_gfxoff``
> +-----------------
> +
> +Use it to enable/disable GFXOFF, and to check if it's current enabled/disabled::
> +
> +  $ xxd -l1 -p /sys/kernel/debug/dri/0/amdgpu_gfxoff
> +  01
> +
> +- Write 0 to disable it, and 1 to enable it.
> +- Read 0 means it's disabled, 1 it's enabled.
> +
> +If it's enabled, that means that the GPU is free to enter into GFXOFF mode as
> +needed. Disabled means that it will never enter GFXOFF mode.
> +
> +``amdgpu_gfxoff_status``
> +------------------------
> +
> +Read it to check current GFXOFF's status of a GPU::
> +
> +  $ xxd -l1 -p /sys/kernel/debug/dri/0/amdgpu_gfxoff_status
> +  02
> +
> +- 0: GPU is in GFXOFF state, the gfx engine is powered down.
> +- 1: Transition out of GFXOFF state
> +- 2: Not in GFXOFF state
> +- 3: Transition into GFXOFF state
> +
> +If GFXOFF is enabled, the value will be transitioning around [0, 3], always
> +getting into 0 when possible. When it's disabled, it's always at 2. Returns
> +``-EINVAL`` if it's not supported.
> --
> 2.37.0
>