Re: [PATCH v5 00/21] x86/resctrl: Make resctrl_arch_rmid_read() return values in bytes

From: Reinette Chatre <reinette.chatre@intel.com>
To: <xhao@linux.alibaba.com>, James Morse <james.morse@arm.com>,
	<x86@kernel.org>, <linux-kernel@vger.kernel.org>
Cc: Fenghua Yu <fenghua.yu@intel.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	H Peter Anvin <hpa@zytor.com>, Babu Moger <Babu.Moger@amd.com>,
	<shameerali.kolothum.thodi@huawei.com>,
	D Scott Phillips OS <scott@os.amperecomputing.com>,
	<lcherian@marvell.com>, <bobo.shaobowang@huawei.com>,
	<tan.shaopeng@fujitsu.com>, Jamie Iles <quic_jiles@quicinc.com>,
	Cristian Marussi <cristian.marussi@arm.com>,
	<xingxin.hx@openanolis.org>, <baolin.wang@linux.alibaba.com>
Subject: Re: [PATCH v5 00/21] x86/resctrl: Make resctrl_arch_rmid_read() return values in bytes
Date: Tue, 23 Aug 2022 10:09:53 -0700	[thread overview]
Message-ID: <5adf4968-b079-2fd3-dd61-09ed16f74080@intel.com> (raw)
In-Reply-To: <f5ab048c-4a3a-8292-8a08-5fdaff739381@linux.alibaba.com>

Hi,

On 7/3/2022 8:54 AM, Xin Hao wrote:
> Hi  james,
> 
> I have a review all of the patches, it looks goot to me, but i also test them once again, i have a little confusion with my test.
> 
> # mkdir p1
> 
> # echo "L3:0=001;1=001" > p1/schemata
> 
> # [root@iZbp1bu26qv0j3ddyusot3Z p1]# cat schemata
>     MB:0=100;1=100
>     L3:0=001;1=001
> 
> # memhog -r1000000 1000m > /mnt/log &
> 
> [1] 53023
> [root@iZbp1bu26qv0j3ddyusot3Z p1]# echo 53023 > tasks
> [
> [root@iZbp1bu26qv0j3ddyusot3Z p1]# cat mon_data/mon_L3_00/llc_occupancy
> 3833856
> [root@iZbp1bu26qv0j3ddyusot3Z p1]# cat mon_data/mon_L3_00/llc_occupancy
> 3620864
> [root@iZbp1bu26qv0j3ddyusot3Z p1]# cat mon_data/mon_L3_00/llc_occupancy
> 3727360
> [root@iZbp1bu26qv0j3ddyusot3Z p1]# cat size
>     MB:0=100;1=100
>     L3:0=3407872;1=3407872
> 
> Obviously, the value has been overflowed,  Can you explain why？

Are you seeing different behavior before and after you apply this
series?

I do not think the conclusion should immediately be that there is an
overflow issue. Have you perhaps run into the scenario "Notes on
cache occupancy monitoring and control" described in
Documentation/x86/resctrl.rst?

When "memhog" starts it can allocate to the entire L3 for a while
before it is moved to the constrained resource group. It's cache
lines are not evicted as part of this move so it is not unusual for
it to have more lines in L3 than it is allowed to allocate into.

Understanding the occupancy values require understanding of the workload
as well as the system environment.

Depending on the workload's data usage (for example if it keeps loading
new data - note that if the workload keeps loading the same data and the
data is already present in an area of cache that the workload cannot
allocate into then the data read would still result in a cache hit for the
workload, the data would not be moved to the area the
workload can allocate into) and other workloads on the system (there is
other load present also that evicts the lines owned by the workload) the
L3 occupancy rate should go down after a while to match the space it
can allocate into.

Reinette