From: Reinette Chatre <reinette.chatre@intel.com>
To: James Morse <james.morse@arm.com>, <x86@kernel.org>,
<linux-kernel@vger.kernel.org>
Cc: Fenghua Yu <fenghua.yu@intel.com>,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
H Peter Anvin <hpa@zytor.com>, Babu Moger <Babu.Moger@amd.com>,
<shameerali.kolothum.thodi@huawei.com>,
Jamie Iles <jamie@nuviainc.com>,
"D Scott Phillips OS" <scott@os.amperecomputing.com>,
<lcherian@marvell.com>, <bobo.shaobowang@huawei.com>,
<tan.shaopeng@fujitsu.com>
Subject: Re: [PATCH v3 12/21] x86/resctrl: Calculate bandwidth from the previous __mon_event_count() chunks
Date: Fri, 4 Mar 2022 16:27:00 -0800 [thread overview]
Message-ID: <c4088205-a344-5ef8-15e2-9ccc5d1691ea@intel.com> (raw)
In-Reply-To: <20220217182110.7176-13-james.morse@arm.com>
Hi James,
On 2/17/2022 10:21 AM, James Morse wrote:
> mbm_bw_count() is only called by the mbm_handle_overflow() worker once a
> second. It reads the hardware register, calculates the bandwidth and
> updates m->prev_bw_msr which is used to hold the previous hardware register
> value.
>
> Operating directly on hardware register values makes it difficult to make
> this code architecture independent, so that it can be moved to /fs/,
> making the mba_sc feature something resctrl supports with no additional
> support from the architecture.
> Prior to calling mbm_bw_count(), mbm_update() reads from the same hardware
> register using __mon_event_count().
>
> Change mbm_bw_count() to use the current chunks value most recently saved by
> __mon_event_count(). This removes an extra call to __rmid_read().
> Instead of using m->prev_msr to calculate the number of chunks seen,
> use the rr->val that was updated by __mon_event_count(). This removes a extra
> calls to mbm_overflow_count() and get_corrected_mbm_count().
"removes a extra calls" -> "removes an extra call" ?
__mon_event_count() ends with "rr->val += get_corrected_mbm_count()" and
it is called twice by mbm_update(). The intention in this change is for
mbm_bw_count() to benefit from the rmid read done just before ...
but would using rr->val within mbm_bw_count() not result in it getting
data from both rmid reads due to the increment?
mbm_update() {
rr.val = 0;
if (is_mbm_total_enabled()) {
...
__mon_event_count() /* rr->val += ... */
}
if (is_mbm_local_enabled()) {
...
__mon_event_count() /* rr->val += ... */
if (is_mba_sc(NULL))
mbm_bw_count(rmid, &rr); /* Use rr-> val */
}
Should rr.val perhaps be reset before each __mon_event_count() call instead of
just at the beginning of mbm_update()?
> Calculating bandwidth like this means mbm_bw_count() no longer operates
> on hardware register values directly.
>
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
> Changes since v2:
> * Expanded commit message
>
> Changes since v1:
> * This patch was rewritten
> ---
> arch/x86/kernel/cpu/resctrl/internal.h | 4 ++--
> arch/x86/kernel/cpu/resctrl/monitor.c | 24 +++++++++++++++---------
> 2 files changed, 17 insertions(+), 11 deletions(-)
>
> diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
> index 3b9e43ba7590..c50c8911ef59 100644
> --- a/arch/x86/kernel/cpu/resctrl/internal.h
> +++ b/arch/x86/kernel/cpu/resctrl/internal.h
> @@ -289,7 +289,7 @@ struct rftype {
> * struct mbm_state - status for each MBM counter in each domain
> * @chunks: Total data moved (multiply by rdt_group.mon_scale to get bytes)
> * @prev_msr: Value of IA32_QM_CTR for this RMID last time we read it
> - * @prev_bw_msr:Value of previous IA32_QM_CTR for bandwidth counting
> + * @prev_bw_chunks: Previous chunks value read when for bandwidth calculation
"value read when for" -> "value read for"?
> * @prev_bw: The most recent bandwidth in MBps
> * @delta_bw: Difference between the current and previous bandwidth
> * @delta_comp: Indicates whether to compute the delta_bw
> @@ -297,7 +297,7 @@ struct rftype {
> struct mbm_state {
> u64 chunks;
> u64 prev_msr;
> - u64 prev_bw_msr;
> + u64 prev_bw_chunks;
> u32 prev_bw;
> u32 delta_bw;
> bool delta_comp;
> diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
> index ac1a2e8998bb..8ae375e29256 100644
> --- a/arch/x86/kernel/cpu/resctrl/monitor.c
> +++ b/arch/x86/kernel/cpu/resctrl/monitor.c
> @@ -315,7 +315,7 @@ static u64 __mon_event_count(u32 rmid, struct rmid_read *rr)
>
> if (rr->first) {
> memset(m, 0, sizeof(struct mbm_state));
> - m->prev_bw_msr = m->prev_msr = tval;
> + m->prev_msr = tval;
> return 0;
> }
>
> @@ -329,27 +329,32 @@ static u64 __mon_event_count(u32 rmid, struct rmid_read *rr)
> }
>
> /*
> + * mbm_bw_count() - Update bw count from values previously read by
> + * __mon_event_count().
> + * @rmid: The rmid used to identify the cached mbm_state.
> + * @rr: The struct rmid_read populated by __mon_event_count().
> + *
> * Supporting function to calculate the memory bandwidth
> - * and delta bandwidth in MBps.
> + * and delta bandwidth in MBps. The chunks value previously read by
> + * __mon_event_count() is compared with the chunks value from the previous
> + * invocation. This must be called oncer per second to maintain values in MBps.
"must be called oncer" -> "must be called once"
> */
> static void mbm_bw_count(u32 rmid, struct rmid_read *rr)
> {
> struct rdt_hw_resource *hw_res = resctrl_to_arch_res(rr->r);
> struct mbm_state *m = &rr->d->mbm_local[rmid];
> - u64 tval, cur_bw, chunks;
> + u64 cur_bw, chunks, cur_chunks;
>
> - tval = __rmid_read(rmid, rr->evtid);
> - if (tval & (RMID_VAL_ERROR | RMID_VAL_UNAVAIL))
> - return;
> + cur_chunks = rr->val;
> + chunks = cur_chunks - m->prev_bw_chunks;
> + m->prev_bw_chunks = cur_chunks;
>
> - chunks = mbm_overflow_count(m->prev_bw_msr, tval, hw_res->mbm_width);
> - cur_bw = (get_corrected_mbm_count(rmid, chunks) * hw_res->mon_scale) >> 20;
> + cur_bw = (chunks * hw_res->mon_scale) >> 20;
>
> if (m->delta_comp)
> m->delta_bw = abs(cur_bw - m->prev_bw);
> m->delta_comp = false;
> m->prev_bw = cur_bw;
> - m->prev_bw_msr = tval;
> }
>
> /*
> @@ -509,6 +514,7 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, int rmid)
> rr.first = false;
> rr.r = r;
> rr.d = d;
> + rr.val = 0;
>
> /*
> * This is protected from concurrent reads from user
Reinette
next prev parent reply other threads:[~2022-03-05 0:27 UTC|newest]
Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-17 18:20 [PATCH v3 00/21] x86/resctrl: Make resctrl_arch_rmid_read() return values in bytes James Morse
2022-02-17 18:20 ` [PATCH v3 01/21] x86/resctrl: Kill off alloc_enabled James Morse
2022-03-16 21:48 ` Reinette Chatre
2022-02-17 18:20 ` [PATCH v3 02/21] x86/resctrl: Merge mon_capable and mon_enabled James Morse
2022-02-17 18:20 ` [PATCH v3 03/21] x86/resctrl: Add domain online callback for resctrl work James Morse
2022-02-17 18:20 ` [PATCH v3 04/21] x86/resctrl: Group struct rdt_hw_domain cleanup James Morse
2022-02-17 18:20 ` [PATCH v3 05/21] x86/resctrl: Add domain offline callback for resctrl work James Morse
2022-02-17 18:20 ` [PATCH v3 06/21] x86/resctrl: Remove set_mba_sc()s control array re-initialisation James Morse
2022-02-17 18:20 ` [PATCH v3 07/21] x86/resctrl: Create mba_sc configuration in the rdt_domain James Morse
2022-03-05 0:26 ` Reinette Chatre
2022-03-30 16:43 ` James Morse
2022-03-16 21:50 ` Reinette Chatre
2022-03-30 16:43 ` James Morse
2022-04-01 22:54 ` Reinette Chatre
2022-04-04 16:35 ` James Morse
2022-04-04 20:43 ` Reinette Chatre
2022-02-17 18:20 ` [PATCH v3 08/21] x86/resctrl: Switch over to the resctrl mbps_val list James Morse
2022-02-17 18:20 ` [PATCH v3 09/21] x86/resctrl: Remove architecture copy of mbps_val James Morse
2022-02-17 18:20 ` [PATCH v3 10/21] x86/resctrl: Abstract and use supports_mba_mbps() James Morse
2022-02-17 18:21 ` [PATCH v3 11/21] x86/resctrl: Allow update_mba_bw() to update controls directly James Morse
2022-02-17 18:21 ` [PATCH v3 12/21] x86/resctrl: Calculate bandwidth from the previous __mon_event_count() chunks James Morse
2022-03-05 0:27 ` Reinette Chatre [this message]
2022-03-30 16:44 ` James Morse
2022-02-17 18:21 ` [PATCH v3 13/21] x86/recstrl: Add per-rmid arch private storage for overflow and chunks James Morse
2022-03-16 21:50 ` Reinette Chatre
2022-02-17 18:21 ` [PATCH v3 14/21] x86/recstrl: Allow per-rmid arch private storage to be reset James Morse
2022-03-16 21:50 ` Reinette Chatre
2022-02-17 18:21 ` [PATCH v3 15/21] x86/resctrl: Abstract __rmid_read() James Morse
2022-03-16 21:52 ` Reinette Chatre
2022-03-30 16:44 ` James Morse
2022-02-17 18:21 ` [PATCH v3 16/21] x86/resctrl: Pass the required parameters into resctrl_arch_rmid_read() James Morse
2022-03-23 20:58 ` Rob Herring
2022-03-30 16:45 ` James Morse
2022-02-17 18:21 ` [PATCH v3 17/21] x86/resctrl: Move mbm_overflow_count() " James Morse
2022-02-17 18:21 ` [PATCH v3 18/21] x86/resctrl: Move get_corrected_mbm_count() " James Morse
2022-02-17 18:21 ` [PATCH v3 19/21] x86/resctrl: Rename and change the units of resctrl_cqm_threshold James Morse
2022-03-17 17:00 ` Reinette Chatre
2022-03-30 16:45 ` James Morse
2022-04-01 22:55 ` Reinette Chatre
2022-02-17 18:21 ` [PATCH v3 20/21] x86/resctrl: Add resctrl_rmid_realloc_limit to abstract x86's boot_cpu_data James Morse
2022-02-17 18:21 ` [PATCH v3 21/21] x86/resctrl: Make resctrl_arch_rmid_read() return values in bytes James Morse
2022-03-23 21:17 ` Rob Herring
2022-04-04 16:36 ` James Morse
2022-03-07 12:56 ` [PATCH v3 00/21] " Jamie Iles
2022-04-04 16:36 ` James Morse
2022-03-15 6:41 ` Xin Hao
2022-04-04 16:36 ` James Morse
2022-03-15 8:16 ` tan.shaopeng
2022-04-04 16:35 ` James Morse
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c4088205-a344-5ef8-15e2-9ccc5d1691ea@intel.com \
--to=reinette.chatre@intel.com \
--cc=Babu.Moger@amd.com \
--cc=bobo.shaobowang@huawei.com \
--cc=bp@alien8.de \
--cc=fenghua.yu@intel.com \
--cc=hpa@zytor.com \
--cc=james.morse@arm.com \
--cc=jamie@nuviainc.com \
--cc=lcherian@marvell.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=scott@os.amperecomputing.com \
--cc=shameerali.kolothum.thodi@huawei.com \
--cc=tan.shaopeng@fujitsu.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).