RE: [PATCH v3 00/21] cpufreq: introduce a new AMD CPU frequency control mechanism

From: "Du, Xiaojian" <Xiaojian.Du@amd.com>
To: Matt McDonald <gardotd426@gmail.com>
Cc: Giovanni Gherdovich <ggherdovich@suse.cz>,
	"Rafael J . Wysocki" <rafael.j.wysocki@intel.com>,
	Viresh Kumar <viresh.kumar@linaro.org>,
	Shuah Khan <skhan@linuxfoundation.org>,
	Borislav Petkov <bp@suse.de>,
	Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@kernel.org>,
	"linux-pm@vger.kernel.org" <linux-pm@vger.kernel.org>,
	"Sharma, Deepak" <Deepak.Sharma@amd.com>,
	"Deucher, Alexander" <Alexander.Deucher@amd.com>,
	"Limonciello, Mario" <Mario.Limonciello@amd.com>,
	Steven Noonan <steven@valvesoftware.com>,
	"Fontenot, Nathan" <Nathan.Fontenot@amd.com>,
	"Su, Jinzhou (Joe)" <Jinzhou.Su@amd.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"x86@kernel.org" <x86@kernel.org>,
	"Huang, Ray" <Ray.Huang@amd.com>
Subject: RE: [PATCH v3 00/21] cpufreq: introduce a new AMD CPU frequency control mechanism
Date: Fri, 12 Nov 2021 11:21:03 +0000	[thread overview]
Message-ID: <DM4PR12MB5136084E5DC1809FBB578075F1959@DM4PR12MB5136.namprd12.prod.outlook.com> (raw)
In-Reply-To: <YYjr2Oi3gqJPFm8x@hr-amd>

[AMD Official Use Only]

Hi Matt,

Thanks for you test, we are very happy to receive the feedback from you and community.
We try to reproduce the issue you reported in our local environment.

Hardware configuration:
CPU: 5900X 12core
MEM: DDR4 8*2GB @2667MHz@2channel
GPU: VEGA20, Radeon VII
Mainboard: B550
Kennel: 5.15-rc, custom kernel, with acpi-cpufreq and amd_pstate driver.

We build two sets of the same system and install the pure Ubuntu20.04.3 OS and Steam.
The software version of Steam is default.
And we use the *USB synchronizer* to control the two systems at the same time.

For "Control" game:
Graphics option: default setting, 1080P, to avoid GPU performance bottle.
GPU driver package is:
https://drivers.amd.com/drivers/linux/amdgpu-pro-21.20-1292797-rhel-8.4.tar.xz
(Installed with command: ./amdgpu-install  --no-dkms)

The only difference of the two systems is the different cpufreq driver: one is acpi-cpufreq, another is amd_pstate.

From our test result, we can't find one obvious performance gap between the two systems, they all run the "Control" at 100-120fps.
You can fetch the result capture from the following picture and videos, they will show the two screens at the same time:

One picture:
https://drive.google.com/file/d/1PvSduykJn9U5MMOhzFWycnbmGmznalM3/view?usp=sharing

Two videos:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdrive.google.com%2Ffile%2Fd%2F1nQQEteL-v_zQxnOJpyW8JqvRW2FFDN2Z%2Fview%3Fusp%3Dsharing&data=04%7C01%7Cray.huang%40amd.com%7C2103847cc456406b2d0508d9a5c6c3c0%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637723096252262986%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=NH21Xjhg8BWm17JJW%2F5hN8JIMkXYwjQCIrTxxjSjrIE%3D&reserved=0

https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdrive.google.com%2Ffile%2Fd%2F1heuPgFG71SQHvGb6wfedrQciBfE2rhnu%2Fview%3Fusp%3Dsharing&data=04%7C01%7Cray.huang%40amd.com%7C2103847cc456406b2d0508d9a5c6c3c0%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637723096252272980%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6%2BcvgbUSkk%2BaRThfID5wIbjjY6sHusJ90uygw6%2FO6m4%3D&reserved=0

We don't test with NV GPU cards, because we have no NV RTX cards so far.
But we can test again with navi21 GPU, named as RX 6900xt/6800xt/6800, if the issue is related to ray trace.
Would you have any chance to use one AMD GPU to re-test with your system?

Anyway, very appreciated for your feedback, we need more feedback to improve our AMD CPU driver.

Thanks,
Xiaojian

-----Original Message-----
From: Huang, Ray <Ray.Huang@amd.com>
Sent: 2021年11月8日 17:20
To: Matt McDonald <gardotd426@gmail.com>
Cc: Giovanni Gherdovich <ggherdovich@suse.cz>; Rafael J . Wysocki <rafael.j.wysocki@intel.com>; Viresh Kumar <viresh.kumar@linaro.org>; Shuah Khan <skhan@linuxfoundation.org>; Borislav Petkov <bp@suse.de>; Peter Zijlstra <peterz@infradead.org>; Ingo Molnar <mingo@kernel.org>; linux-pm@vger.kernel.org; Sharma, Deepak <Deepak.Sharma@amd.com>; Deucher, Alexander <Alexander.Deucher@amd.com>; Limonciello, Mario <Mario.Limonciello@amd.com>; Steven Noonan <steven@valvesoftware.com>; Fontenot, Nathan <Nathan.Fontenot@amd.com>; Su, Jinzhou (Joe) <Jinzhou.Su@amd.com>; Du, Xiaojian <Xiaojian.Du@amd.com>; linux-kernel@vger.kernel.org; x86@kernel.org
Subject: Re: [PATCH v3 00/21] cpufreq: introduce a new AMD CPU frequency control mechanism

On Sat, Nov 06, 2021 at 04:58:35PM +0800, Matt McDonald wrote:
> > > I've tested this driver and it seems the results are a little
> > > underwhelming.
> > > The test machine is a two sockets server with two AMD EPYC 7713,
> > > family:model:stepping 25:1:1, 128 cores/256 threads, 256G of
> > > memory and SSD storage. On this system, the amd-pstate driver
> > > works only in "shared memory support", not in "full MSR support",
> > > meaning that frequency switches are triggered from a workqueue
> > > instead of scheduler context (!fast_switch).
>
> Huang, I've also done some detailed testing, and while many synthetic
> benchmarks seem to show minimal differences between this new frequency
> control mechanism and acpi_cpufreq, the general user experience seems
> a bit degraded, but most of all, gaming performance in many instances
> (if not all) is cut in half. Fully half.
>
> I have an RTX 3090 and a Ryzen 9 5900X, with 32GB (4x8) DDR4 3600. In

May we know the family/model id of your processors?

> Control with DLSS and RT enabled, on 5.15.rc5 with acpi_cpufreq, I get
> 120-130 fps at 1440p. The same exact kernel with v3 of AMD_CPPC gives
> me 50 fps. GPU usage is still at 100, but the CPU frequency is being
> reported as like 5100Mhz*, and other assorted weirdness, but most
> importantly the fps is stuck at 50. This is regardless of performance
> scheduler (schedutil, ondemand, userspace or performance).

May we know your SMU version in your SBIOS?

Thanks,
Ray

>
> *My CPU can indeed boost over 5GHz on a single core here and there,
> but this was constant and on all cores, so clearly it wasn't accurate.
>
> Also, from the documentation it looks like there's supposed to be a
> way to fall back to acpi_cpufreq, but I found no such way to do that.
> If AMD_CPPC was built into the kernel, I had to use amd-pstate, there
> was no other option. Maybe I misinterpreted and acpi-cpufreq is only
> able to be used as a fallback for CPUs that don't support amd-pstate.
>
> I know that gaming on Linux hasn't historically been one of AMD's
> priorities with their CPUs, but with the Steam Deck upcoming I would
> imagine this is a pretty important use-case, and I've tested multiple
> games and they all lose a full 50% performance. I'm happy to test any
> revisions or even kernel parameters or whatever else to try and get
> this sorted.
>
>
>
> > Would you mind that we add a module param or filter the known good
> > processors (mobile parts) to load amd-pstate. And others can use the
> > param to switch between amd-pstate and acpi-cpufreq manually? After
> > we address the performance gap, then we can switch it back.
>
>
> This would be something I would be interested to try.
>
> >
> > It seems the issue mainly from the processors with big number of
> > cores and threads. Let's find the similiar family threadripper or
> > EYPC processors to duplicate the test results. Will contact at you
> > for details. :-)
>
> This may be an interesting route of investigation, I could potentially
> try running a game with `taskset -c 0-7` or something similar.
>
> >
>