* [PATCH v5 01/24] docs: create L2 Cache Allocation Technology (CAT) feature document
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 9:11 ` Dario Faggioli
2017-01-18 2:02 ` [PATCH v5 02/24] x86: refactor psr: remove L3 CAT/CDP codes Yi Sun
` (22 subsequent siblings)
23 siblings, 1 reply; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
This patch creates L2 CAT feature document in doc/features/.
It describes details of L2 CAT.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
docs/features/intel_psr_l2_cat.pandoc | 347 ++++++++++++++++++++++++++++++++++
1 file changed, 347 insertions(+)
create mode 100644 docs/features/intel_psr_l2_cat.pandoc
diff --git a/docs/features/intel_psr_l2_cat.pandoc b/docs/features/intel_psr_l2_cat.pandoc
new file mode 100644
index 0000000..77bd61f
--- /dev/null
+++ b/docs/features/intel_psr_l2_cat.pandoc
@@ -0,0 +1,347 @@
+% Intel L2 Cache Allocation Technology (L2 CAT) Feature
+% Revision 1.0
+
+\clearpage
+
+# Basics
+
+---------------- ----------------------------------------------------
+ Status: **Tech Preview**
+
+Architecture(s): Intel x86
+
+ Component(s): Hypervisor, toolstack
+
+ Hardware: Atom codename Goldmont and beyond CPUs
+---------------- ----------------------------------------------------
+
+# Overview
+
+L2 CAT allows an OS or Hypervisor/VMM to control allocation of a
+CPU's shared L2 cache based on application priority or Class of Service
+(COS). Each CLOS is configured using capacity bitmasks (CBM) which
+represent cache capacity and indicate the degree of overlap and
+isolation between classes. Once L2 CAT is configured, the processor
+allows access to portions of L2 cache according to the established
+class of service.
+
+## Terminology
+
+* CAT Cache Allocation Technology
+* CBM Capacity BitMasks
+* CDP Code and Data Prioritization
+* COS/CLOS Class of Service
+* MSRs Machine Specific Registers
+* PSR Intel Platform Shared Resource
+* VMM Virtual Machine Monitor
+
+# User details
+
+* Feature Enabling:
+
+ Add "psr=cat" to boot line parameter to enable all supported level CAT
+ features.
+
+* xl interfaces:
+
+ 1. `psr-cat-show [OPTIONS] domain-id`:
+
+ Show domain L2 or L3 CAT CBM.
+
+ New option `-l` is added.
+ `-l2`: Show cbm for L2 cache.
+ `-l3`: Show cbm for L3 cache.
+
+ If neither `-l2` nor `-l3` is given, show both of them. If any one
+ is not supported, will print error info.
+
+ 2. `psr-cat-cbm-set [OPTIONS] domain-id cbm`:
+
+ Set domain L2 or L3 CBM.
+
+ New option `-l` is added.
+ `-l2`: Specify cbm for L2 cache.
+ `-l3`: Specify cbm for L3 cache.
+
+ If neither `-l2` nor `-l3` is given, level 3 is the default option.
+
+ 3. `psr-hwinfo [OPTIONS]`:
+
+ Show L2 & L3 CAT HW informations on every socket.
+
+# Technical details
+
+L2 CAT is a member of Intel PSR features and part of CAT, it shares
+some base PSR infrastructure in Xen.
+
+## Hardware perspective
+
+L2 CAT defines a new range MSRs to assign different L2 cache access
+patterns which are known as CBMs, each CBM is associated with a COS.
+
+```
+
+ +----------------------------+----------------+
+ IA32_PQR_ASSOC | MSR (per socket) | Address |
+ +----+---+-------+ +----------------------------+----------------+
+ | |COS| | | IA32_L2_QOS_MASK_0 | 0xD10 |
+ +----+---+-------+ +----------------------------+----------------+
+ └-------------> | ... | ... |
+ +----------------------------+----------------+
+ | IA32_L2_QOS_MASK_n | 0xD10+n (n<64) |
+ +----------------------------+----------------+
+```
+
+When context switch happens, the COS of VCPU is written to per-thread
+MSR `IA32_PQR_ASSOC`, and then hardware enforces L2 cache allocation
+according to the corresponding CBM.
+
+## The relationship between L2 CAT and L3 CAT/CDP
+
+L2 CAT is independent of L3 CAT/CDP, which means L2 CAT would be enabled
+while L3 CAT/CDP is disabled, or L2 CAT and L3 CAT/CDP are all enabled.
+
+L2 CAT uses a new range CBMs from 0xD10 ~ 0xD10+n (n<64), following by
+the L3 CAT/CDP CBMs, and supports setting different L2 cache accessing
+patterns from L3 cache. Like L3 CAT/CDP requirement, the bits of CBM of
+L2 CAT must be continuous too.
+
+N.B. L2 CAT and L3 CAT/CDP share the same COS field in the same
+associate register `IA32_PQR_ASSOC`, which means one COS associates to a
+pair of L2 CBM and L3 CBM.
+
+Besides, the max COS of L2 CAT may be different from L3 CAT/CDP (or
+other PSR features in future). In some cases, a VM is permitted to have a
+COS that is beyond one (or more) of PSR features but within the others.
+For instance, let's assume the max COS of L2 CAT is 8 but the max COS of
+L3 CAT is 16, when a VM is assigned 9 as COS, the L3 CBM associated to
+COS 9 would be enforced, but for L2 CAT, the behavior is fully open (no
+limit) since COS 9 is beyond the max COS (8) of L2 CAT.
+
+## Design Overview
+
+* Core COS/CBM association
+
+ When enforcing L2 CAT, all cores of domains have the same default
+ COS (COS0) which associated to the fully open CBM (all ones bitmask)
+ to access all L2 cache. The default COS is used only in hypervisor
+ and is transparent to tool stack and user.
+
+ System administrator can change PSR allocation policy at runtime by
+ tool stack. Since L2 CAT share COS with L3 CAT/CDP, a COS corresponds
+ to a 2-tuple, like [L2 CBM, L3 CBM] with only-CAT enabled, when CDP
+ is enabled, one COS corresponds to a 3-tuple, like [L2 CBM,
+ L3 Code_CBM, L3 Data_CBM]. If neither L3 CAT nor L3 CDP is enabled,
+ things would be easier, one COS corresponds to one L2 CBM.
+
+* VCPU schedule
+
+ This part reuses L3 CAT COS infrastructure.
+
+* Multi-sockets
+
+ Different sockets may have different L2 CAT capability (e.g. max COS)
+ although it is consistent on the same socket. So the capability of
+ per-socket L2 CAT is specified.
+
+## Implementation Description
+
+* Hypervisor interfaces:
+
+ 1. Boot line parameter "psr=cat" now will enable L2 CAT and L3
+ CAT if hardware supported.
+
+ 2. SYSCTL:
+ - XEN_SYSCTL_PSR_CAT_get_l2_info: Get L2 CAT information.
+
+ 3. DOMCTL:
+ - XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM: Get L2 CBM for a domain.
+ - XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM: Set L2 CBM for a domain.
+
+* xl interfaces:
+
+ 1. psr-cat-show -l2 domain-id
+ Show L2 cbm for a domain.
+ => XEN_SYSCTL_PSR_CAT_get_l2_info /
+ XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM
+
+ 2. psr-mba-set -l2 domain-id cbm
+ Set L2 cbm for a domain.
+ => XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM
+
+ 3. psr-hwinfo
+ Show PSR HW information, including L2 CAT
+ => XEN_SYSCTL_PSR_CAT_get_l2_info
+
+* Key data structure:
+
+ 1. Feature HW info
+
+ ```
+ struct psr_cat_hw_info {
+ unsigned int cbm_len;
+ unsigned int cos_max;
+ };
+ ```
+
+ - Member `cbm_len`
+
+ `cbm_len` is one of the hardware info of CAT.
+
+ - Member `cos_max`
+
+ `cos_max` is one of the hardware info of CAT.
+
+ 2. Feature list node
+
+ ```
+ struct feat_node {
+ enum psr_feat_type feature;
+ struct feat_ops ops;
+ struct psr_cat_hw_info info;
+ uint64_t cos_reg_val[MAX_COS_REG_NUM];
+ struct list_head list;
+ };
+ ```
+
+ When a PSR enforcement feature is enabled, it will be added into a
+ feature list. The head of the list is created in psr initialization.
+
+ - Member `feature`
+
+ `feature` is an integer number, to indicate which feature the list entry
+ corresponds to.
+
+ - Member `ops`
+
+ `ops` maintains a callback function list of the feature. It will be introduced
+ in details later.
+
+ - Member `info`
+
+ `info` maintains the feature HW information which can be got through
+ psr_hwinfo command.
+
+ - Member `cos_reg_val`
+
+ `cos_reg_val` is an array to maintain the value set in all COS registers of
+ the feature.
+
+ 3. Per-socket PSR features information structure
+
+ ```
+ struct psr_cat_socket_info {
+ unsigned int feat_mask;
+ unsigned int nr_feat;
+ struct list_head feat_list;
+ unsigned int cos_ref[MAX_COS_REG_NUM];
+ spinlock_t ref_lock;
+ };
+ ```
+
+ We collect all PSR allocation features information of a socket in
+ this `struct psr_cat_socket_info`.
+
+ - Member `feat_mask`
+
+ `feat_mask` is a bitmap, to indicate which feature is enabled on
+ current socket. We define `feat_mask` bitmap as:
+
+ bit 0~1: L3 CAT status, [01] stands for L3 CAT only and [10]
+ stands for L3 CDP is enalbed.
+
+ bit 2: L2 CAT status.
+
+ - Member `cos_ref`
+
+ `cos_ref` is an array which maintains the reference of one COS.
+ If the COS is used by one domain, the reference will increase one.
+ If a domain releases the COS, the reference will decrease one. The
+ array is indexed by COS.
+
+ 4. Feature operation functions structure
+
+ ```
+ struct feat_ops {
+ void (*init_feature)(unsigned int eax, unsigned int ebx,
+ unsigned int ecx, unsigned int edx,
+ struct feat_node *feat,
+ struct psr_cat_socket_info *info);
+ int (*get_feat_info)(const struct feat_node *feat, enum cbm_type type,
+ uint32_t dat[], uint32_t array_len);
+ int (*get_val)(const struct feat_node *feat, unsigned int cos,
+ enum cbm_type type, uint64_t *val);
+ unsigned int (*get_max_cos_max)(const struct feat_node *feat);
+ unsigned int (*get_cos_num)(const struct feat_node *feat);
+ int (*get_old_val)(uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int old_cos);
+ int (*set_new_val)(uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int old_cos,
+ enum cbm_type type,
+ uint64_t m);
+ int (*compare_val)(const uint64_t val[], const struct feat_node *feat,
+ unsigned int cos, bool *found);
+ unsigned int (*get_cos_max_from_type)(const struct feat_node *feat,
+ enum cbm_type type);
+ unsigned int (*exceeds_cos_max)(const uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int cos);
+ int (*write_msr)(unsigned int cos, const uint64_t val[],
+ struct feat_node *feat);
+ };
+ ```
+
+ We abstract above callback functions to encapsulate the feature specific
+ behaviors into them. Then, it is easy to add a new feature. We just need:
+ 1) Implement such ops and callback functions for every feature.
+ 2) Register the ops into `struct feat_node`.
+ 3) Add the feature into feature list during CPU initialization.
+
+# Limitations
+
+L2 CAT can only work on HW which enables it(check by CPUID). So far, there
+is no HW enables both L2 CAT and L3 CAT/CDP. But SW implementation has considered
+such scenario to enable both L2 CAT and L3 CAT/CDP.
+
+# Testing
+
+L2 CAT uses same xl interfaces as L3 CAT/CDP. So, we can execute these
+commands to verify L2 CAT and L3 CAT/CDP on different HWs support them.
+
+For example:
+ root@:~$ xl psr-hwinfo --cat
+ Cache Allocation Technology (CAT): L2
+ Socket ID : 0
+ Maximum COS : 3
+ CBM length : 8
+ Default CBM : 0xff
+
+ root@:~$ xl psr-cat-cbm-set -l2 1 0x7f
+
+ root@:~$ xl psr-cat-show -l2 1
+ Socket ID : 0
+ Default CBM : 0xff
+ ID NAME CBM
+ 1 ubuntu14 0x7f
+
+# Areas for improvement
+
+N/A
+
+# Known issues
+
+N/A
+
+# References
+
+"INTEL® RESOURCE DIRECTOR TECHNOLOGY (INTEL® RDT) ALLOCATION FEATURES" [Intel® 64 and IA-32 Architectures Software Developer Manuals, vol3](http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html)
+
+# History
+
+------------------------------------------------------------------------
+Date Revision Version Notes
+---------- -------- -------- -------------------------------------------
+2016-08-12 1.0 Xen 4.9 Design document written
+---------- -------- -------- -------------------------------------------
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* Re: [PATCH v5 01/24] docs: create L2 Cache Allocation Technology (CAT) feature document
2017-01-18 2:02 ` [PATCH v5 01/24] docs: create L2 Cache Allocation Technology (CAT) feature document Yi Sun
@ 2017-01-18 9:11 ` Dario Faggioli
2017-01-18 9:37 ` Yi Sun
2017-01-19 6:11 ` Yi Sun
0 siblings, 2 replies; 30+ messages in thread
From: Dario Faggioli @ 2017-01-18 9:11 UTC (permalink / raw)
To: Yi Sun, xen-devel
Cc: wei.liu2, he.chen, andrew.cooper3, konrad.wilk, ian.jackson,
mengxu, jbeulich, chao.p.peng
[-- Attachment #1.1: Type: text/plain, Size: 4315 bytes --]
On Wed, 2017-01-18 at 10:02 +0800, Yi Sun wrote:
> This patch creates L2 CAT feature document in doc/features/.
> It describes details of L2 CAT.
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> ---
>
Hey,
it is very very useful to put _RIGHT_HERE_ a summary of what changed,
within, this patch, wrt the previous version.
That helps reviewers quite a bit, especially in making sure that you
hav considered and addressed comments made during the previous
iterations.
> docs/features/intel_psr_l2_cat.pandoc | 347
> ++++++++++++++++++++++++++++++++++
> 1 file changed, 347 insertions(+)
> create mode 100644 docs/features/intel_psr_l2_cat.pandoc
> --- /dev/null
> +++ b/docs/features/intel_psr_l2_cat.pandoc
> @@ -0,0 +1,347 @@
> +% Intel L2 Cache Allocation Technology (L2 CAT) Feature
> +% Revision 1.0
> +
> +\clearpage
> +
> +# Basics
> +
> +---------------- -------------------------------------------------
> ---
> + Status: **Tech Preview**
> +
> +Architecture(s): Intel x86
> +
> + Component(s): Hypervisor, toolstack
> +
> + Hardware: Atom codename Goldmont and beyond CPUs
> +---------------- -------------------------------------------------
> ---
> +
> +# Overview
> +
> +L2 CAT allows an OS or Hypervisor/VMM to control allocation of a
> +CPU's shared L2 cache based on application priority or Class of
> Service
> +(COS). Each CLOS is configured using capacity bitmasks (CBM) which
> +represent cache capacity and indicate the degree of overlap and
> +isolation between classes. Once L2 CAT is configured, the processor
> +allows access to portions of L2 cache according to the established
> +class of service.
> +
Well, considering that even here in the overview, acronyms are used,
that are then defined in the section below, it would look better to me
to move the Terminology section up (and let it be the first one).
> +## Terminology
> +
> +* CAT Cache Allocation Technology
> +* CBM Capacity BitMasks
> +* CDP Code and Data Prioritization
> +* COS/CLOS Class of Service
> +* MSRs Machine Specific Registers
> +* PSR Intel Platform Shared Resource
> +* VMM Virtual Machine Monitor
>
So, the term 'hypervisor' is (must be!) quite a well known one, for
anyone reading docs in this directory. And, in the docs we already have
around in here, it is far more common, AFAICT, to call an hypervisor
'hypervisor', rather than 'Virtual Machine Monitor'.
Therefore, I don't think there is the need to define the term 'VMM' in
this section, and, at the same time, I'd replace occurrences of
'Hypervisor/VMM' to just 'hypervisor' (the only one of which appear to
be the one above, in the Overview).
> +# User details
> +
> +* Feature Enabling:
> +
> + Add "psr=cat" to boot line parameter to enable all supported level
> CAT
> + features.
> +
> +* xl interfaces:
> +
> + 1. `psr-cat-show [OPTIONS] domain-id`:
> +
> + Show domain L2 or L3 CAT CBM.
> +
> + New option `-l` is added.
> + `-l2`: Show cbm for L2 cache.
> + `-l3`: Show cbm for L3 cache.
> +
> + If neither `-l2` nor `-l3` is given, show both of them. If any
> one
> + is not supported, will print error info.
> +
I actually think the best behavior would be:
- if -lX is specified, and LX is not supported ==> print error;
- if no -l is specified ==> print info about the supported levels.
(See comment on patch 21.)
> +## The relationship between L2 CAT and L3 CAT/CDP
> +
> +L2 CAT is independent of L3 CAT/CDP, which means L2 CAT would be
> enabled
> +while L3 CAT/CDP is disabled, or L2 CAT and L3 CAT/CDP are all
> enabled.
> +
I find 'would be enabled' and 'are all enabled' a bit confusing.
Maybe: "which means L2 cat can be enabled while L3 CAT/CDP is disabled,
or L2 CAT and L3 CAT/CDP can be all enabled"
?
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
[-- Attachment #2: Type: text/plain, Size: 127 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH v5 01/24] docs: create L2 Cache Allocation Technology (CAT) feature document
2017-01-18 9:11 ` Dario Faggioli
@ 2017-01-18 9:37 ` Yi Sun
2017-01-19 6:11 ` Yi Sun
1 sibling, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 9:37 UTC (permalink / raw)
To: Dario Faggioli
Cc: wei.liu2, he.chen, andrew.cooper3, konrad.wilk, ian.jackson,
mengxu, jbeulich, chao.p.peng, xen-devel
On 17-01-18 10:11:15, Dario Faggioli wrote:
> On Wed, 2017-01-18 at 10:02 +0800, Yi Sun wrote:
> > This patch creates L2 CAT feature document in doc/features/.
> > It describes details of L2 CAT.
> >
> > Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> > ---
> >
> Hey,
>
> it is very very useful to put _RIGHT_HERE_ a summary of what changed,
> within, this patch, wrt the previous version.
>
> That helps reviewers quite a bit, especially in making sure that you
> hav considered and addressed comments made during the previous
> iterations.
>
Sorry, forget this. Can I use "RESEND" to send updated patches again?
Thanks,
Sun Yi
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH v5 01/24] docs: create L2 Cache Allocation Technology (CAT) feature document
2017-01-18 9:11 ` Dario Faggioli
2017-01-18 9:37 ` Yi Sun
@ 2017-01-19 6:11 ` Yi Sun
1 sibling, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-19 6:11 UTC (permalink / raw)
To: Dario Faggioli
Cc: wei.liu2, he.chen, andrew.cooper3, konrad.wilk, ian.jackson,
mengxu, jbeulich, chao.p.peng, xen-devel
On 17-01-18 10:11:15, Dario Faggioli wrote:
> On Wed, 2017-01-18 at 10:02 +0800, Yi Sun wrote:
> > This patch creates L2 CAT feature document in doc/features/.
> > It describes details of L2 CAT.
> >
> > Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> > ---
> >
> Hey,
>
> it is very very useful to put _RIGHT_HERE_ a summary of what changed,
> within, this patch, wrt the previous version.
>
> That helps reviewers quite a bit, especially in making sure that you
> hav considered and addressed comments made during the previous
> iterations.
>
I have resend the patches with summary of what changed. Please check.
Thanks!
Your comments for these patches will be recorded.
> > docs/features/intel_psr_l2_cat.pandoc | 347
> > ++++++++++++++++++++++++++++++++++
> > 1 file changed, 347 insertions(+)
> > create mode 100644 docs/features/intel_psr_l2_cat.pandoc
>
> > --- /dev/null
> > +++ b/docs/features/intel_psr_l2_cat.pandoc
> > +
> > +# User details
> > +
> > +* Feature Enabling:
> > +
> > + Add "psr=cat" to boot line parameter to enable all supported level
> > CAT
> > + features.
> > +
> > +* xl interfaces:
> > +
> > + 1. `psr-cat-show [OPTIONS] domain-id`:
> > +
> > + Show domain L2 or L3 CAT CBM.
> > +
> > + New option `-l` is added.
> > + `-l2`: Show cbm for L2 cache.
> > + `-l3`: Show cbm for L3 cache.
> > +
> > + If neither `-l2` nor `-l3` is given, show both of them. If any
> > one
> > + is not supported, will print error info.
> > +
> I actually think the best behavior would be:
> - if -lX is specified, and LX is not supported ==> print error;
Will return error for this case. But the error log is not accurate so far.
Will refine this. Thanks!
> - if no -l is specified ==> print info about the supported levels.
>
That is my bad. In fact, the l3 info will be shown if no -l is specified.
> (See comment on patch 21.)
>
> > +## The relationship between L2 CAT and L3 CAT/CDP
> > +
> > +L2 CAT is independent of L3 CAT/CDP, which means L2 CAT would be
> > enabled
> > +while L3 CAT/CDP is disabled, or L2 CAT and L3 CAT/CDP are all
> > enabled.
> > +
>
> I find 'would be enabled' and 'are all enabled' a bit confusing.
>
> Maybe: "which means L2 cat can be enabled while L3 CAT/CDP is disabled,
> or L2 CAT and L3 CAT/CDP can be all enabled"
>
> ?
Agree, thank you!
>
> Regards,
> Dario
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH v5 02/24] x86: refactor psr: remove L3 CAT/CDP codes.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
2017-01-18 2:02 ` [PATCH v5 01/24] docs: create L2 Cache Allocation Technology (CAT) feature document Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 03/24] x86: refactor psr: implement main data structures Yi Sun
` (21 subsequent siblings)
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
The current cache allocation codes in psr.c do not consider
future features addition and are not friendly to extend.
To make psr.c be more flexible to add new features and fulfill
the program principle, open for extension but closed for
modification, we have to refactor the psr.c:
1. Analyze cache allocation features and abstract general data
structures.
2. Analyze the init and all other functions flow, abstract all
steps that different features may have different implementations.
Make these steps be callback functions and register feature
specific fuctions. Then, the main processes will not be changed
when introducing a new feature.
Because the quantity of refactor codes is big and the logics are
changed a lot, it will cause reviewers confused if just change
old codes. Reviewers have to understand both old codes and new
implementations. After review iterations from V1 to V3, Jan has
proposed to remove all old cache allocation codes firstly, then
implement new codes step by step. This will help to make codes
be more easily reviewable.
There is no construction without destruction. So, this patch
removes all current L3 CAT/CDP codes in psr.c. The following
patches will introduce the new mechanism.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
xen/arch/x86/psr.c | 470 +----------------------------------------------------
1 file changed, 5 insertions(+), 465 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 0b5073c..96a8589 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -23,24 +23,6 @@
#define PSR_CAT (1<<1)
#define PSR_CDP (1<<2)
-struct psr_cat_cbm {
- union {
- uint64_t cbm;
- struct {
- uint64_t code;
- uint64_t data;
- };
- };
- unsigned int ref;
-};
-
-struct psr_cat_socket_info {
- unsigned int cbm_len;
- unsigned int cos_max;
- struct psr_cat_cbm *cos_to_cbm;
- spinlock_t cbm_lock;
-};
-
struct psr_assoc {
uint64_t val;
uint64_t cos_mask;
@@ -48,26 +30,11 @@ struct psr_assoc {
struct psr_cmt *__read_mostly psr_cmt;
-static unsigned long *__read_mostly cat_socket_enable;
-static struct psr_cat_socket_info *__read_mostly cat_socket_info;
-static unsigned long *__read_mostly cdp_socket_enable;
-
static unsigned int opt_psr;
static unsigned int __initdata opt_rmid_max = 255;
-static unsigned int __read_mostly opt_cos_max = 255;
static uint64_t rmid_mask;
static DEFINE_PER_CPU(struct psr_assoc, psr_assoc);
-static struct psr_cat_cbm *temp_cos_to_cbm;
-
-static unsigned int get_socket_cpu(unsigned int socket)
-{
- if ( likely(socket < nr_sockets) )
- return cpumask_any(socket_cpumask[socket]);
-
- return nr_cpu_ids;
-}
-
static void __init parse_psr_bool(char *s, char *value, char *feature,
unsigned int mask)
{
@@ -107,9 +74,6 @@ static void __init parse_psr_param(char *s)
if ( val_str && !strcmp(s, "rmid_max") )
opt_rmid_max = simple_strtoul(val_str, NULL, 0);
- if ( val_str && !strcmp(s, "cos_max") )
- opt_cos_max = simple_strtoul(val_str, NULL, 0);
-
s = ss + 1;
} while ( ss );
}
@@ -213,16 +177,7 @@ static inline void psr_assoc_init(void)
{
struct psr_assoc *psra = &this_cpu(psr_assoc);
- if ( cat_socket_info )
- {
- unsigned int socket = cpu_to_socket(smp_processor_id());
-
- if ( test_bit(socket, cat_socket_enable) )
- psra->cos_mask = ((1ull << get_count_order(
- cat_socket_info[socket].cos_max)) - 1) << 32;
- }
-
- if ( psr_cmt_enabled() || psra->cos_mask )
+ if ( psr_cmt_enabled() )
rdmsrl(MSR_IA32_PSR_ASSOC, psra->val);
}
@@ -231,12 +186,6 @@ static inline void psr_assoc_rmid(uint64_t *reg, unsigned int rmid)
*reg = (*reg & ~rmid_mask) | (rmid & rmid_mask);
}
-static inline void psr_assoc_cos(uint64_t *reg, unsigned int cos,
- uint64_t cos_mask)
-{
- *reg = (*reg & ~cos_mask) | (((uint64_t)cos << 32) & cos_mask);
-}
-
void psr_ctxt_switch_to(struct domain *d)
{
struct psr_assoc *psra = &this_cpu(psr_assoc);
@@ -245,459 +194,54 @@ void psr_ctxt_switch_to(struct domain *d)
if ( psr_cmt_enabled() )
psr_assoc_rmid(®, d->arch.psr_rmid);
- if ( psra->cos_mask )
- psr_assoc_cos(®, d->arch.psr_cos_ids ?
- d->arch.psr_cos_ids[cpu_to_socket(smp_processor_id())] :
- 0, psra->cos_mask);
-
if ( reg != psra->val )
{
wrmsrl(MSR_IA32_PSR_ASSOC, reg);
psra->val = reg;
}
}
-static struct psr_cat_socket_info *get_cat_socket_info(unsigned int socket)
-{
- if ( !cat_socket_info )
- return ERR_PTR(-ENODEV);
-
- if ( socket >= nr_sockets )
- return ERR_PTR(-ENOTSOCK);
-
- if ( !test_bit(socket, cat_socket_enable) )
- return ERR_PTR(-ENOENT);
-
- return cat_socket_info + socket;
-}
-
-static inline bool_t cdp_is_enabled(unsigned int socket)
-{
- return cdp_socket_enable && test_bit(socket, cdp_socket_enable);
-}
int psr_get_cat_l3_info(unsigned int socket, uint32_t *cbm_len,
uint32_t *cos_max, uint32_t *flags)
{
- struct psr_cat_socket_info *info = get_cat_socket_info(socket);
-
- if ( IS_ERR(info) )
- return PTR_ERR(info);
-
- *cbm_len = info->cbm_len;
- *cos_max = info->cos_max;
-
- *flags = 0;
- if ( cdp_is_enabled(socket) )
- *flags |= XEN_SYSCTL_PSR_CAT_L3_CDP;
-
return 0;
}
int psr_get_l3_cbm(struct domain *d, unsigned int socket,
uint64_t *cbm, enum cbm_type type)
{
- struct psr_cat_socket_info *info = get_cat_socket_info(socket);
- bool_t cdp_enabled = cdp_is_enabled(socket);
-
- if ( IS_ERR(info) )
- return PTR_ERR(info);
-
- switch ( type )
- {
- case PSR_CBM_TYPE_L3:
- if ( cdp_enabled )
- return -EXDEV;
- *cbm = info->cos_to_cbm[d->arch.psr_cos_ids[socket]].cbm;
- break;
-
- case PSR_CBM_TYPE_L3_CODE:
- if ( !cdp_enabled )
- *cbm = info->cos_to_cbm[d->arch.psr_cos_ids[socket]].cbm;
- else
- *cbm = info->cos_to_cbm[d->arch.psr_cos_ids[socket]].code;
- break;
-
- case PSR_CBM_TYPE_L3_DATA:
- if ( !cdp_enabled )
- *cbm = info->cos_to_cbm[d->arch.psr_cos_ids[socket]].cbm;
- else
- *cbm = info->cos_to_cbm[d->arch.psr_cos_ids[socket]].data;
- break;
-
- default:
- ASSERT_UNREACHABLE();
- }
-
- return 0;
-}
-
-static bool_t psr_check_cbm(unsigned int cbm_len, uint64_t cbm)
-{
- unsigned int first_bit, zero_bit;
-
- /* Set bits should only in the range of [0, cbm_len). */
- if ( cbm & (~0ull << cbm_len) )
- return 0;
-
- /* At least one bit need to be set. */
- if ( cbm == 0 )
- return 0;
-
- first_bit = find_first_bit(&cbm, cbm_len);
- zero_bit = find_next_zero_bit(&cbm, cbm_len, first_bit);
-
- /* Set bits should be contiguous. */
- if ( zero_bit < cbm_len &&
- find_next_bit(&cbm, cbm_len, zero_bit) < cbm_len )
- return 0;
-
- return 1;
-}
-
-struct cos_cbm_info
-{
- unsigned int cos;
- bool_t cdp;
- uint64_t cbm_code;
- uint64_t cbm_data;
-};
-
-static void do_write_l3_cbm(void *data)
-{
- struct cos_cbm_info *info = data;
-
- if ( info->cdp )
- {
- wrmsrl(MSR_IA32_PSR_L3_MASK_CODE(info->cos), info->cbm_code);
- wrmsrl(MSR_IA32_PSR_L3_MASK_DATA(info->cos), info->cbm_data);
- }
- else
- wrmsrl(MSR_IA32_PSR_L3_MASK(info->cos), info->cbm_code);
-}
-
-static int write_l3_cbm(unsigned int socket, unsigned int cos,
- uint64_t cbm_code, uint64_t cbm_data, bool_t cdp)
-{
- struct cos_cbm_info info =
- {
- .cos = cos,
- .cbm_code = cbm_code,
- .cbm_data = cbm_data,
- .cdp = cdp,
- };
-
- if ( socket == cpu_to_socket(smp_processor_id()) )
- do_write_l3_cbm(&info);
- else
- {
- unsigned int cpu = get_socket_cpu(socket);
-
- if ( cpu >= nr_cpu_ids )
- return -ENOTSOCK;
- on_selected_cpus(cpumask_of(cpu), do_write_l3_cbm, &info, 1);
- }
-
return 0;
}
-static int find_cos(struct psr_cat_cbm *map, unsigned int cos_max,
- uint64_t cbm_code, uint64_t cbm_data, bool_t cdp_enabled)
-{
- unsigned int cos;
-
- for ( cos = 0; cos <= cos_max; cos++ )
- {
- if ( (map[cos].ref || cos == 0) &&
- ((!cdp_enabled && map[cos].cbm == cbm_code) ||
- (cdp_enabled && map[cos].code == cbm_code &&
- map[cos].data == cbm_data)) )
- return cos;
- }
-
- return -ENOENT;
-}
-
-static int pick_avail_cos(struct psr_cat_cbm *map, unsigned int cos_max,
- unsigned int old_cos)
-{
- unsigned int cos;
-
- /* If old cos is referred only by the domain, then use it. */
- if ( map[old_cos].ref == 1 && old_cos != 0 )
- return old_cos;
-
- /* Find an unused one other than cos0. */
- for ( cos = 1; cos <= cos_max; cos++ )
- if ( map[cos].ref == 0 )
- return cos;
-
- return -ENOENT;
-}
-
int psr_set_l3_cbm(struct domain *d, unsigned int socket,
uint64_t cbm, enum cbm_type type)
{
- unsigned int old_cos, cos_max;
- int cos, ret;
- uint64_t cbm_data, cbm_code;
- bool_t cdp_enabled = cdp_is_enabled(socket);
- struct psr_cat_cbm *map;
- struct psr_cat_socket_info *info = get_cat_socket_info(socket);
-
- if ( IS_ERR(info) )
- return PTR_ERR(info);
-
- if ( !psr_check_cbm(info->cbm_len, cbm) )
- return -EINVAL;
-
- if ( !cdp_enabled && (type == PSR_CBM_TYPE_L3_CODE ||
- type == PSR_CBM_TYPE_L3_DATA) )
- return -ENXIO;
-
- cos_max = info->cos_max;
- old_cos = d->arch.psr_cos_ids[socket];
- map = info->cos_to_cbm;
-
- switch ( type )
- {
- case PSR_CBM_TYPE_L3:
- cbm_code = cbm;
- cbm_data = cbm;
- break;
-
- case PSR_CBM_TYPE_L3_CODE:
- cbm_code = cbm;
- cbm_data = map[old_cos].data;
- break;
-
- case PSR_CBM_TYPE_L3_DATA:
- cbm_code = map[old_cos].code;
- cbm_data = cbm;
- break;
-
- default:
- ASSERT_UNREACHABLE();
- return -EINVAL;
- }
-
- spin_lock(&info->cbm_lock);
- cos = find_cos(map, cos_max, cbm_code, cbm_data, cdp_enabled);
- if ( cos >= 0 )
- {
- if ( cos == old_cos )
- {
- spin_unlock(&info->cbm_lock);
- return 0;
- }
- }
- else
- {
- cos = pick_avail_cos(map, cos_max, old_cos);
- if ( cos < 0 )
- {
- spin_unlock(&info->cbm_lock);
- return cos;
- }
-
- /* We try to avoid writing MSR. */
- if ( (cdp_enabled &&
- (map[cos].code != cbm_code || map[cos].data != cbm_data)) ||
- (!cdp_enabled && map[cos].cbm != cbm_code) )
- {
- ret = write_l3_cbm(socket, cos, cbm_code, cbm_data, cdp_enabled);
- if ( ret )
- {
- spin_unlock(&info->cbm_lock);
- return ret;
- }
- map[cos].code = cbm_code;
- map[cos].data = cbm_data;
- }
- }
-
- map[cos].ref++;
- map[old_cos].ref--;
- spin_unlock(&info->cbm_lock);
-
- d->arch.psr_cos_ids[socket] = cos;
-
return 0;
}
-/* Called with domain lock held, no extra lock needed for 'psr_cos_ids' */
-static void psr_free_cos(struct domain *d)
-{
- unsigned int socket;
- unsigned int cos;
- struct psr_cat_socket_info *info;
-
- if( !d->arch.psr_cos_ids )
- return;
-
- for_each_set_bit(socket, cat_socket_enable, nr_sockets)
- {
- if ( (cos = d->arch.psr_cos_ids[socket]) == 0 )
- continue;
-
- info = cat_socket_info + socket;
- spin_lock(&info->cbm_lock);
- info->cos_to_cbm[cos].ref--;
- spin_unlock(&info->cbm_lock);
- }
-
- xfree(d->arch.psr_cos_ids);
- d->arch.psr_cos_ids = NULL;
-}
-
int psr_domain_init(struct domain *d)
{
- if ( cat_socket_info )
- {
- d->arch.psr_cos_ids = xzalloc_array(unsigned int, nr_sockets);
- if ( !d->arch.psr_cos_ids )
- return -ENOMEM;
- }
-
return 0;
}
void psr_domain_free(struct domain *d)
{
psr_free_rmid(d);
- psr_free_cos(d);
-}
-
-static int cat_cpu_prepare(unsigned int cpu)
-{
- if ( !cat_socket_info )
- return 0;
-
- if ( temp_cos_to_cbm == NULL &&
- (temp_cos_to_cbm = xzalloc_array(struct psr_cat_cbm,
- opt_cos_max + 1UL)) == NULL )
- return -ENOMEM;
-
- return 0;
-}
-
-static void cat_cpu_init(void)
-{
- unsigned int eax, ebx, ecx, edx;
- struct psr_cat_socket_info *info;
- unsigned int socket;
- unsigned int cpu = smp_processor_id();
- uint64_t val;
- const struct cpuinfo_x86 *c = cpu_data + cpu;
-
- if ( !cpu_has(c, X86_FEATURE_PQE) || c->cpuid_level < PSR_CPUID_LEVEL_CAT )
- return;
-
- socket = cpu_to_socket(cpu);
- if ( test_bit(socket, cat_socket_enable) )
- return;
-
- cpuid_count(PSR_CPUID_LEVEL_CAT, 0, &eax, &ebx, &ecx, &edx);
- if ( ebx & PSR_RESOURCE_TYPE_L3 )
- {
- cpuid_count(PSR_CPUID_LEVEL_CAT, 1, &eax, &ebx, &ecx, &edx);
- info = cat_socket_info + socket;
- info->cbm_len = (eax & 0x1f) + 1;
- info->cos_max = min(opt_cos_max, edx & 0xffff);
-
- info->cos_to_cbm = temp_cos_to_cbm;
- temp_cos_to_cbm = NULL;
- /* cos=0 is reserved as default cbm(all ones). */
- info->cos_to_cbm[0].cbm = (1ull << info->cbm_len) - 1;
-
- spin_lock_init(&info->cbm_lock);
-
- set_bit(socket, cat_socket_enable);
-
- if ( (ecx & PSR_CAT_CDP_CAPABILITY) && (opt_psr & PSR_CDP) &&
- cdp_socket_enable && !test_bit(socket, cdp_socket_enable) )
- {
- info->cos_to_cbm[0].code = (1ull << info->cbm_len) - 1;
- info->cos_to_cbm[0].data = (1ull << info->cbm_len) - 1;
-
- /* We only write mask1 since mask0 is always all ones by default. */
- wrmsrl(MSR_IA32_PSR_L3_MASK(1), (1ull << info->cbm_len) - 1);
-
- rdmsrl(MSR_IA32_PSR_L3_QOS_CFG, val);
- wrmsrl(MSR_IA32_PSR_L3_QOS_CFG, val | (1 << PSR_L3_QOS_CDP_ENABLE_BIT));
-
- /* Cut half of cos_max when CDP is enabled. */
- info->cos_max >>= 1;
-
- set_bit(socket, cdp_socket_enable);
- }
- printk(XENLOG_INFO "CAT: enabled on socket %u, cos_max:%u, cbm_len:%u, CDP:%s\n",
- socket, info->cos_max, info->cbm_len,
- cdp_is_enabled(socket) ? "on" : "off");
- }
-}
-
-static void cat_cpu_fini(unsigned int cpu)
-{
- unsigned int socket = cpu_to_socket(cpu);
-
- if ( !socket_cpumask[socket] || cpumask_empty(socket_cpumask[socket]) )
- {
- struct psr_cat_socket_info *info = cat_socket_info + socket;
-
- if ( info->cos_to_cbm )
- {
- xfree(info->cos_to_cbm);
- info->cos_to_cbm = NULL;
- }
-
- if ( cdp_is_enabled(socket) )
- clear_bit(socket, cdp_socket_enable);
-
- clear_bit(socket, cat_socket_enable);
- }
-}
-
-static void __init psr_cat_free(void)
-{
- xfree(cat_socket_enable);
- cat_socket_enable = NULL;
- xfree(cat_socket_info);
- cat_socket_info = NULL;
-}
-
-static void __init init_psr_cat(void)
-{
- if ( opt_cos_max < 1 )
- {
- printk(XENLOG_INFO "CAT: disabled, cos_max is too small\n");
- return;
- }
-
- cat_socket_enable = xzalloc_array(unsigned long, BITS_TO_LONGS(nr_sockets));
- cat_socket_info = xzalloc_array(struct psr_cat_socket_info, nr_sockets);
- cdp_socket_enable = xzalloc_array(unsigned long, BITS_TO_LONGS(nr_sockets));
-
- if ( !cat_socket_enable || !cat_socket_info )
- psr_cat_free();
}
static int psr_cpu_prepare(unsigned int cpu)
{
- return cat_cpu_prepare(cpu);
+ return 0;
}
static void psr_cpu_init(void)
{
- if ( cat_socket_info )
- cat_cpu_init();
-
psr_assoc_init();
}
static void psr_cpu_fini(unsigned int cpu)
{
- if ( cat_socket_info )
- cat_cpu_fini(cpu);
+ return;
}
static int cpu_callback(
@@ -738,14 +282,10 @@ static int __init psr_presmp_init(void)
if ( (opt_psr & PSR_CMT) && opt_rmid_max )
init_psr_cmt(opt_rmid_max);
- if ( opt_psr & PSR_CAT )
- init_psr_cat();
-
- if ( psr_cpu_prepare(0) )
- psr_cat_free();
+ psr_cpu_prepare(0);
psr_cpu_init();
- if ( psr_cmt_enabled() || cat_socket_info )
+ if ( psr_cmt_enabled() )
register_cpu_notifier(&cpu_nfb);
return 0;
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 03/24] x86: refactor psr: implement main data structures.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
2017-01-18 2:02 ` [PATCH v5 01/24] docs: create L2 Cache Allocation Technology (CAT) feature document Yi Sun
2017-01-18 2:02 ` [PATCH v5 02/24] x86: refactor psr: remove L3 CAT/CDP codes Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 04/24] x86: refactor psr: implement CPU init and free flow Yi Sun
` (20 subsequent siblings)
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
To construct an extendible framework, we need analyze PSR features
and abstract the common things and feature specific things. Then,
encapsulate them into different data structures.
By analyzing PSR features, we can get below map.
+------+------+------+
--------->| Dom0 | Dom1 | ... |
| +------+------+------+
| |
|Dom ID | cos_id of domain
| V
| +-----------------------------------------------------------------------------+
User --------->| PSR |
Socket ID | +--------------+---------------+---------------+ |
| | Socket0 Info | Socket 1 Info | ... | |
| +--------------+---------------+---------------+ |
| | cos_id=0 cos_id=1 ... |
| | +-----------------------+-----------------------+-----------+ |
| |->Ref : | ref 0 | ref 1 | ... | |
| | +-----------------------+-----------------------+-----------+ |
| | +-----------------------+-----------------------+-----------+ |
| |->L3 CAT: | cos 0 | cos 1 | ... | |
| | +-----------------------+-----------------------+-----------+ |
| | +-----------------------+-----------------------+-----------+ |
| |->L2 CAT: | cos 0 | cos 1 | ... | |
| | +-----------------------+-----------------------+-----------+ |
| | +-----------+-----------+-----------+-----------+-----------+ |
| |->CDP : | cos0 code | cos0 data | cos1 code | cos1 data | ... | |
| +-----------+-----------+-----------+-----------+-----------+ |
+-----------------------------------------------------------------------------+
So, we need define a socket info data structure, 'struct
psr_socket_info' to manage information per socket. It contains a
reference count array according to COS ID and a feature list to
manage all features enabled. Every entry of the reference count
array is used to record how many domains are using the COS registers
according to the COS ID. For example, L3 CAT and L2 CAT are enabled,
Dom1 uses COS_ID=1 registers of both features to save CBM values, like
below.
+-------+-------+-------+-----+
| COS 0 | COS 1 | COS 2 | ... |
+-------+-------+-------+-----+
L3 CAT | 0x7ff | 0x1ff | ... | ... |
+-------+-------+-------+-----+
L2 CAT | 0xff | 0xff | ... | ... |
+-------+-------+-------+-----+
If Dom2 has same CBM values, it can reuse these registers which COS_ID=1.
That means, both Dom1 and Dom2 use same COS registers(ID=1) to save same
L3/L2 values. So, the value ref[1] is 2 which means 2 domains are using
COS_ID 1.
To manage a feature, we need define a feature node data structure,
'struct feat_node', to manage feature's specific HW info, its callback
functions (all feature's specific behaviors are encapsulated into these
callback functions), and an array of all COS registers values of this
feature.
CDP is a special feature which uses two entries of the array
for one COS ID. So, the number of CDP COS registers is the half of L3
CAT. E.g. L3 CAT has 16 COS registers, then CDP has 8 COS registers if
it is enabled. CDP uses the COS registers array as below.
+-----------+-----------+-----------+-----------+-----------+
CDP cos_reg_val[] index: | 0 | 1 | 2 | 3 | ... |
+-----------+-----------+-----------+-----------+-----------+
value: | cos0 code | cos0 data | cos1 code | cos1 data | ... |
+-----------+-----------+-----------+-----------+-----------+
For more details, please refer spec and codes.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 104 +++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 104 insertions(+)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 96a8589..f7ff3fc 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -17,12 +17,116 @@
#include <xen/cpu.h>
#include <xen/err.h>
#include <xen/sched.h>
+#include <xen/list.h>
#include <asm/psr.h>
+/*
+ * Terminology:
+ * - CAT Cache Allocation Technology
+ * - CBM Capacity BitMasks
+ * - CDP Code and Data Prioritization
+ * - COS/CLOS Class of Service. Also mean COS registers.
+ * - COS_MAX Max number of COS for the feature (minus 1)
+ * - MSRs Machine Specific Registers
+ * - PSR Intel Platform Shared Resource
+ */
+
#define PSR_CMT (1<<0)
#define PSR_CAT (1<<1)
#define PSR_CDP (1<<2)
+/*
+ * Per SDM chapter 'Cache Allocation Technology: Cache Mask Configuration',
+ * the MSRs range from 0C90H through 0D0FH (inclusive), enables support for
+ * up to 128 L3 CAT Classes of Service. The COS_ID=[0,127].
+ *
+ * The MSRs range from 0D10H through 0D4FH (inclusive), enables support for
+ * up to 64 L2 CAT COS. The COS_ID=[0,63].
+ *
+ * So, the maximum COS register count of one feature is 128.
+ */
+#define MAX_COS_REG_CNT 128
+
+/*
+ * PSR features are managed per socket. Below structure defines the members
+ * used to manage these features.
+ * feat_mask - Mask used to record features enabled on socket. There may be
+ * some features enabled at same time.
+ * nr_feat - Record how many features enabled.
+ * feat_list - A list used to manage all features enabled.
+ * cos_ref - A reference count array to record how many domains are using the
+ * COS_ID.
+ * Every entry of cos_ref corresponds to one COS ID.
+ * ref_lock - A lock to protect cos_ref.
+ */
+struct psr_socket_info {
+ /*
+ * bit 0: L3 CAT
+ * bit 1: L3 CDP
+ * bit 2: L2 CAT
+ */
+ unsigned int feat_mask;
+ unsigned int nr_feat;
+ struct list_head feat_list;
+ unsigned int cos_ref[MAX_COS_REG_CNT];
+ spinlock_t ref_lock;
+};
+
+enum psr_feat_type {
+ PSR_SOCKET_L3_CAT = 0,
+ PSR_SOCKET_L3_CDP,
+ PSR_SOCKET_L2_CAT,
+};
+
+/* CAT/CDP HW info data structure. */
+struct psr_cat_hw_info {
+ unsigned int cbm_len;
+ unsigned int cos_max;
+};
+
+/* Encapsulate feature specific HW info here. */
+struct feat_hw_info {
+ union {
+ struct psr_cat_hw_info l3_cat_info;
+ };
+};
+
+struct feat_node;
+
+/*
+ * This structure defines feature operation callback functions. Every feature
+ * enabled MUST implement such callback functions and register them to ops.
+ *
+ * Feature specific behaviors will be encapsulated into these callback
+ * functions. Then, the main flows will not be changed when introducing a new
+ * feature.
+ */
+struct feat_ops {
+ /* get_cos_max is used to get feature's cos_max. */
+ unsigned int (*get_cos_max)(const struct feat_node *feat);
+};
+
+/*
+ * This structure represents one feature.
+ * feature - Which feature it is.
+ * feat_ops - Feature operation callback functions.
+ * info - Feature HW info.
+ * cos_reg_val - Array to store the values of COS registers. One entry stores
+ * the value of one COS register.
+ * For L3 CAT and L2 CAT, one entry corresponds to one COS_ID.
+ * For CDP, two entries correspond to one COS_ID. E.g.
+ * COS_ID=0 corresponds to cos_reg_val[0] (Data) and
+ * cos_reg_val[1] (Code).
+ * list - Feature list.
+ */
+struct feat_node {
+ enum psr_feat_type feature;
+ struct feat_ops ops;
+ struct feat_hw_info info;
+ uint64_t cos_reg_val[MAX_COS_REG_CNT];
+ struct list_head list;
+};
+
struct psr_assoc {
uint64_t val;
uint64_t cos_mask;
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 04/24] x86: refactor psr: implement CPU init and free flow.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (2 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 03/24] x86: refactor psr: implement main data structures Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 05/24] x86: refactor psr: implement Domain init/free and schedule flows Yi Sun
` (19 subsequent siblings)
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements the CPU init and free flow including L3 CAT
initialization and feature list free.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 176 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 174 insertions(+), 2 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index f7ff3fc..e9dc07a 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -35,6 +35,9 @@
#define PSR_CAT (1<<1)
#define PSR_CDP (1<<2)
+#define CAT_CBM_LEN_MASK 0x1f
+#define CAT_COS_MAX_MASK 0xffff
+
/*
* Per SDM chapter 'Cache Allocation Technology: Cache Mask Configuration',
* the MSRs range from 0C90H through 0D0FH (inclusive), enables support for
@@ -127,6 +130,13 @@ struct feat_node {
struct list_head list;
};
+struct cpuid_leaf_regs {
+ unsigned int eax;
+ unsigned int ebx;
+ unsigned int ecx;
+ unsigned int edx;
+};
+
struct psr_assoc {
uint64_t val;
uint64_t cos_mask;
@@ -134,11 +144,76 @@ struct psr_assoc {
struct psr_cmt *__read_mostly psr_cmt;
+static struct psr_socket_info *__read_mostly socket_info;
+
static unsigned int opt_psr;
static unsigned int __initdata opt_rmid_max = 255;
+static unsigned int __read_mostly opt_cos_max = MAX_COS_REG_CNT;
static uint64_t rmid_mask;
static DEFINE_PER_CPU(struct psr_assoc, psr_assoc);
+/*
+ * Declare global feature list entry for every feature to facilitate the
+ * feature list creation. It will be allocated in psr_cpu_prepare() and
+ * inserted into feature list in cpu_init_work().
+ */
+static struct feat_node *feat_l3_cat;
+
+/* Common functions. */
+static void free_feature(struct psr_socket_info *info)
+{
+ struct feat_node *feat, *next;
+
+ if ( !info )
+ return;
+
+ list_for_each_entry_safe(feat, next, &info->feat_list, list)
+ {
+ clear_bit(feat->feature, &info->feat_mask);
+ list_del(&feat->list);
+ xfree(feat);
+ }
+}
+
+/* L3 CAT functions implementation. */
+static void l3_cat_init_feature(struct cpuid_leaf_regs regs,
+ struct feat_node *feat,
+ struct psr_socket_info *info)
+{
+ struct psr_cat_hw_info l3_cat;
+ unsigned int socket;
+
+ /* No valid value so do not enable feature. */
+ if ( !regs.eax || !regs.edx )
+ return;
+
+ l3_cat.cbm_len = (regs.eax & CAT_CBM_LEN_MASK) + 1;
+ l3_cat.cos_max = min(opt_cos_max, regs.edx & CAT_COS_MAX_MASK);
+
+ /* cos=0 is reserved as default cbm(all bits within cbm_len are 1). */
+ feat->cos_reg_val[0] = (1ull << l3_cat.cbm_len) - 1;
+
+ feat->feature = PSR_SOCKET_L3_CAT;
+ __set_bit(PSR_SOCKET_L3_CAT, &info->feat_mask);
+
+ feat->info.l3_cat_info = l3_cat;
+
+ info->nr_feat++;
+
+ /* Add this feature into list. */
+ list_add_tail(&feat->list, &info->feat_list);
+
+ socket = cpu_to_socket(smp_processor_id());
+ if ( opt_cpu_info )
+ printk(XENLOG_INFO
+ "L3 CAT: enabled on socket %u, cos_max:%u, cbm_len:%u\n",
+ socket, feat->info.l3_cat_info.cos_max,
+ feat->info.l3_cat_info.cbm_len);
+}
+
+static const struct feat_ops l3_cat_ops = {
+};
+
static void __init parse_psr_bool(char *s, char *value, char *feature,
unsigned int mask)
{
@@ -178,6 +253,9 @@ static void __init parse_psr_param(char *s)
if ( val_str && !strcmp(s, "rmid_max") )
opt_rmid_max = simple_strtoul(val_str, NULL, 0);
+ if ( val_str && !strcmp(s, "cos_max") )
+ opt_cos_max = simple_strtoul(val_str, NULL, 0);
+
s = ss + 1;
} while ( ss );
}
@@ -333,18 +411,108 @@ void psr_domain_free(struct domain *d)
psr_free_rmid(d);
}
+static void cpu_init_work(void)
+{
+ struct psr_socket_info *info;
+ unsigned int socket;
+ unsigned int cpu = smp_processor_id();
+ struct feat_node *feat;
+ struct cpuid_leaf_regs regs;
+
+ if ( !cpu_has(¤t_cpu_data, X86_FEATURE_PQE) )
+ return;
+ else if ( current_cpu_data.cpuid_level < PSR_CPUID_LEVEL_CAT )
+ {
+ clear_bit(X86_FEATURE_PQE, current_cpu_data.x86_capability);
+ return;
+ }
+
+ socket = cpu_to_socket(cpu);
+ info = socket_info + socket;
+ if ( info->feat_mask )
+ return;
+
+ INIT_LIST_HEAD(&info->feat_list);
+ spin_lock_init(&info->ref_lock);
+
+ cpuid_count(PSR_CPUID_LEVEL_CAT, 0,
+ ®s.eax, ®s.ebx, ®s.ecx, ®s.edx);
+ if ( regs.ebx & PSR_RESOURCE_TYPE_L3 )
+ {
+ cpuid_count(PSR_CPUID_LEVEL_CAT, 1,
+ ®s.eax, ®s.ebx, ®s.ecx, ®s.edx);
+
+ feat = feat_l3_cat;
+ feat_l3_cat = NULL;
+ feat->ops = l3_cat_ops;
+
+ l3_cat_init_feature(regs, feat, info);
+ }
+}
+
+static void cpu_fini_work(unsigned int cpu)
+{
+ unsigned int socket = cpu_to_socket(cpu);
+
+ if ( !socket_cpumask[socket] || cpumask_empty(socket_cpumask[socket]) )
+ {
+ free_feature(socket_info + socket);
+ }
+}
+
+static void __init init_psr(void)
+{
+ if ( opt_cos_max < 1 )
+ {
+ printk(XENLOG_INFO "CAT: disabled, cos_max is too small\n");
+ return;
+ }
+
+ socket_info = xzalloc_array(struct psr_socket_info, nr_sockets);
+
+ if ( !socket_info )
+ {
+ printk(XENLOG_INFO "Fail to alloc socket_info!\n");
+ return;
+ }
+}
+
+static void __init psr_free(void)
+{
+ unsigned int i;
+
+ for ( i = 0; i < nr_sockets; i++ )
+ free_feature(&socket_info[i]);
+
+ xfree(socket_info);
+ socket_info = NULL;
+}
+
static int psr_cpu_prepare(unsigned int cpu)
{
+ if ( !socket_info )
+ return 0;
+
+ /* Malloc memory for the global feature head here. */
+ if ( feat_l3_cat == NULL &&
+ (feat_l3_cat = xzalloc(struct feat_node)) == NULL )
+ return -ENOMEM;
+
return 0;
}
static void psr_cpu_init(void)
{
+ if ( socket_info )
+ cpu_init_work();
+
psr_assoc_init();
}
static void psr_cpu_fini(unsigned int cpu)
{
+ if ( socket_info )
+ cpu_fini_work(cpu);
return;
}
@@ -386,10 +554,14 @@ static int __init psr_presmp_init(void)
if ( (opt_psr & PSR_CMT) && opt_rmid_max )
init_psr_cmt(opt_rmid_max);
- psr_cpu_prepare(0);
+ if ( opt_psr & PSR_CAT )
+ init_psr();
+
+ if ( psr_cpu_prepare(0) )
+ psr_free();
psr_cpu_init();
- if ( psr_cmt_enabled() )
+ if ( psr_cmt_enabled() || socket_info )
register_cpu_notifier(&cpu_nfb);
return 0;
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 05/24] x86: refactor psr: implement Domain init/free and schedule flows.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (3 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 04/24] x86: refactor psr: implement CPU init and free flow Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 06/24] x86: refactor psr: implement get hw info flow Yi Sun
` (18 subsequent siblings)
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements the Domain init/free and schedule flows.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 61 insertions(+), 1 deletion(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index e9dc07a..7f06235 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -50,6 +50,8 @@
*/
#define MAX_COS_REG_CNT 128
+#define PSR_ASSOC_REG_POS 32
+
/*
* PSR features are managed per socket. Below structure defines the members
* used to manage these features.
@@ -211,7 +213,13 @@ static void l3_cat_init_feature(struct cpuid_leaf_regs regs,
feat->info.l3_cat_info.cbm_len);
}
+static unsigned int l3_cat_get_cos_max(const struct feat_node *feat)
+{
+ return feat->info.l3_cat_info.cos_max;
+}
+
static const struct feat_ops l3_cat_ops = {
+ .get_cos_max = l3_cat_get_cos_max,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
@@ -355,11 +363,33 @@ void psr_free_rmid(struct domain *d)
d->arch.psr_rmid = 0;
}
+static inline unsigned int get_max_cos_max(const struct psr_socket_info *info)
+{
+ const struct feat_node *feat;
+ unsigned int cos_max = 0;
+
+ list_for_each_entry(feat, &info->feat_list, list)
+ cos_max = max(feat->ops.get_cos_max(feat), cos_max);
+
+ return cos_max;
+}
+
static inline void psr_assoc_init(void)
{
struct psr_assoc *psra = &this_cpu(psr_assoc);
- if ( psr_cmt_enabled() )
+ if ( socket_info )
+ {
+ unsigned int socket = cpu_to_socket(smp_processor_id());
+ const struct psr_socket_info *info = socket_info + socket;
+ unsigned int cos_max = get_max_cos_max(info);
+
+ if ( info->feat_mask )
+ psra->cos_mask = ((1ull << get_count_order(cos_max)) - 1) <<
+ PSR_ASSOC_REG_POS;
+ }
+
+ if ( psr_cmt_enabled() || psra->cos_mask )
rdmsrl(MSR_IA32_PSR_ASSOC, psra->val);
}
@@ -368,6 +398,13 @@ static inline void psr_assoc_rmid(uint64_t *reg, unsigned int rmid)
*reg = (*reg & ~rmid_mask) | (rmid & rmid_mask);
}
+static inline void psr_assoc_cos(uint64_t *reg, unsigned int cos,
+ uint64_t cos_mask)
+{
+ *reg = (*reg & ~cos_mask) |
+ (((uint64_t)cos << PSR_ASSOC_REG_POS) & cos_mask);
+}
+
void psr_ctxt_switch_to(struct domain *d)
{
struct psr_assoc *psra = &this_cpu(psr_assoc);
@@ -376,6 +413,11 @@ void psr_ctxt_switch_to(struct domain *d)
if ( psr_cmt_enabled() )
psr_assoc_rmid(®, d->arch.psr_rmid);
+ if ( psra->cos_mask )
+ psr_assoc_cos(®, d->arch.psr_cos_ids ?
+ d->arch.psr_cos_ids[cpu_to_socket(smp_processor_id())] :
+ 0, psra->cos_mask);
+
if ( reg != psra->val )
{
wrmsrl(MSR_IA32_PSR_ASSOC, reg);
@@ -401,14 +443,32 @@ int psr_set_l3_cbm(struct domain *d, unsigned int socket,
return 0;
}
+/* Called with domain lock held, no extra lock needed for 'psr_cos_ids' */
+static void psr_free_cos(struct domain *d)
+{
+ if( !d->arch.psr_cos_ids )
+ return;
+
+ xfree(d->arch.psr_cos_ids);
+ d->arch.psr_cos_ids = NULL;
+}
+
int psr_domain_init(struct domain *d)
{
+ if ( socket_info )
+ {
+ d->arch.psr_cos_ids = xzalloc_array(unsigned int, nr_sockets);
+ if ( !d->arch.psr_cos_ids )
+ return -ENOMEM;
+ }
+
return 0;
}
void psr_domain_free(struct domain *d)
{
psr_free_rmid(d);
+ psr_free_cos(d);
}
static void cpu_init_work(void)
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 06/24] x86: refactor psr: implement get hw info flow.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (4 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 05/24] x86: refactor psr: implement Domain init/free and schedule flows Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 07/24] x86: refactor psr: implement get value flow Yi Sun
` (17 subsequent siblings)
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements get HW info flow including L3 CAT callback
function.
It also changes sysctl interface to make it more general.
With this patch, 'psr-hwinfo' can work for L3 CAT.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 73 +++++++++++++++++++++++++++++++++++++++++++++--
xen/arch/x86/sysctl.c | 14 +++++----
xen/include/asm-x86/psr.h | 9 ++++--
3 files changed, 86 insertions(+), 10 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 7f06235..319bfcc 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -109,6 +109,9 @@ struct feat_node;
struct feat_ops {
/* get_cos_max is used to get feature's cos_max. */
unsigned int (*get_cos_max)(const struct feat_node *feat);
+ /* get_feat_info is used to get feature HW info. */
+ bool (*get_feat_info)(const struct feat_node *feat,
+ uint32_t data[], unsigned int array_len);
};
/*
@@ -177,6 +180,23 @@ static void free_feature(struct psr_socket_info *info)
}
}
+static enum psr_feat_type psr_cbm_type_to_feat_type(enum cbm_type type)
+{
+ enum psr_feat_type feat_type;
+
+ /* Judge if feature is enabled. */
+ switch ( type ) {
+ case PSR_CBM_TYPE_L3:
+ feat_type = PSR_SOCKET_L3_CAT;
+ break;
+ default:
+ feat_type = 0xFFFF;
+ break;
+ }
+
+ return feat_type;
+}
+
/* L3 CAT functions implementation. */
static void l3_cat_init_feature(struct cpuid_leaf_regs regs,
struct feat_node *feat,
@@ -218,8 +238,22 @@ static unsigned int l3_cat_get_cos_max(const struct feat_node *feat)
return feat->info.l3_cat_info.cos_max;
}
+static bool l3_cat_get_feat_info(const struct feat_node *feat,
+ uint32_t data[], unsigned int array_len)
+{
+ if ( !data || 3 > array_len )
+ return false;
+
+ data[CBM_LEN] = feat->info.l3_cat_info.cbm_len;
+ data[COS_MAX] = feat->info.l3_cat_info.cos_max;
+ data[PSR_FLAG] = 0;
+
+ return true;
+}
+
static const struct feat_ops l3_cat_ops = {
.get_cos_max = l3_cat_get_cos_max,
+ .get_feat_info = l3_cat_get_feat_info,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
@@ -425,10 +459,43 @@ void psr_ctxt_switch_to(struct domain *d)
}
}
-int psr_get_cat_l3_info(unsigned int socket, uint32_t *cbm_len,
- uint32_t *cos_max, uint32_t *flags)
+static struct psr_socket_info *get_socket_info(unsigned int socket)
{
- return 0;
+ if ( !socket_info )
+ return ERR_PTR(-ENODEV);
+
+ if ( socket >= nr_sockets )
+ return ERR_PTR(-ENOTSOCK);
+
+ if ( !socket_info[socket].feat_mask )
+ return ERR_PTR(-ENOENT);
+
+ return socket_info + socket;
+}
+
+int psr_get_info(unsigned int socket, enum cbm_type type,
+ uint32_t data[], unsigned int array_len)
+{
+ const struct psr_socket_info *info = get_socket_info(socket);
+ const struct feat_node *feat;
+ enum psr_feat_type feat_type;
+
+ if ( IS_ERR(info) )
+ return PTR_ERR(info);
+
+ feat_type = psr_cbm_type_to_feat_type(type);
+ list_for_each_entry(feat, &info->feat_list, list)
+ {
+ if ( feat->feature != feat_type )
+ continue;
+
+ if ( feat->ops.get_feat_info(feat, data, array_len) )
+ return 0;
+ else
+ return -EINVAL;
+ }
+
+ return -ENOENT;
}
int psr_get_l3_cbm(struct domain *d, unsigned int socket,
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index 14e7dc7..d90db78 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -176,15 +176,19 @@ long arch_do_sysctl(
switch ( sysctl->u.psr_cat_op.cmd )
{
case XEN_SYSCTL_PSR_CAT_get_l3_info:
- ret = psr_get_cat_l3_info(sysctl->u.psr_cat_op.target,
- &sysctl->u.psr_cat_op.u.l3_info.cbm_len,
- &sysctl->u.psr_cat_op.u.l3_info.cos_max,
- &sysctl->u.psr_cat_op.u.l3_info.flags);
+ {
+ uint32_t data[3];
+ ret = psr_get_info(sysctl->u.psr_cat_op.target,
+ PSR_CBM_TYPE_L3, data, 3);
+
+ sysctl->u.psr_cat_op.u.l3_info.cbm_len = data[CBM_LEN];
+ sysctl->u.psr_cat_op.u.l3_info.cos_max = data[COS_MAX];
+ sysctl->u.psr_cat_op.u.l3_info.flags = data[PSR_FLAG];
if ( !ret && __copy_field_to_guest(u_sysctl, sysctl, u.psr_cat_op) )
ret = -EFAULT;
break;
-
+ }
default:
ret = -EOPNOTSUPP;
break;
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index 57f47e9..e3b18bc 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -33,6 +33,11 @@
/* L3 CDP Enable bit*/
#define PSR_L3_QOS_CDP_ENABLE_BIT 0x0
+/* Used by psr_get_info() */
+#define CBM_LEN 0
+#define COS_MAX 1
+#define PSR_FLAG 2
+
struct psr_cmt_l3 {
unsigned int features;
unsigned int upscaling_factor;
@@ -63,8 +68,8 @@ int psr_alloc_rmid(struct domain *d);
void psr_free_rmid(struct domain *d);
void psr_ctxt_switch_to(struct domain *d);
-int psr_get_cat_l3_info(unsigned int socket, uint32_t *cbm_len,
- uint32_t *cos_max, uint32_t *flags);
+int psr_get_info(unsigned int socket, enum cbm_type type,
+ uint32_t data[], unsigned int array_len);
int psr_get_l3_cbm(struct domain *d, unsigned int socket,
uint64_t *cbm, enum cbm_type type);
int psr_set_l3_cbm(struct domain *d, unsigned int socket,
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 07/24] x86: refactor psr: implement get value flow.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (5 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 06/24] x86: refactor psr: implement get hw info flow Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 08/24] x86: refactor psr: set value: implement framework Yi Sun
` (16 subsequent siblings)
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements get value flow including L3 CAT callback
function.
It also changes domctl interface to make it more general.
With this patch, 'psr-cat-show' can work for L3 CAT.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/domctl.c | 18 +++++++++---------
xen/arch/x86/psr.c | 41 ++++++++++++++++++++++++++++++++++++++---
xen/include/asm-x86/psr.h | 4 ++--
3 files changed, 49 insertions(+), 14 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index ab141b1..11d2127 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1383,23 +1383,23 @@ long arch_do_domctl(
break;
case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM:
- ret = psr_get_l3_cbm(d, domctl->u.psr_cat_op.target,
- &domctl->u.psr_cat_op.data,
- PSR_CBM_TYPE_L3);
+ ret = psr_get_val(d, domctl->u.psr_cat_op.target,
+ &domctl->u.psr_cat_op.data,
+ PSR_CBM_TYPE_L3);
copyback = 1;
break;
case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CODE:
- ret = psr_get_l3_cbm(d, domctl->u.psr_cat_op.target,
- &domctl->u.psr_cat_op.data,
- PSR_CBM_TYPE_L3_CODE);
+ ret = psr_get_val(d, domctl->u.psr_cat_op.target,
+ &domctl->u.psr_cat_op.data,
+ PSR_CBM_TYPE_L3_CODE);
copyback = 1;
break;
case XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA:
- ret = psr_get_l3_cbm(d, domctl->u.psr_cat_op.target,
- &domctl->u.psr_cat_op.data,
- PSR_CBM_TYPE_L3_DATA);
+ ret = psr_get_val(d, domctl->u.psr_cat_op.target,
+ &domctl->u.psr_cat_op.data,
+ PSR_CBM_TYPE_L3_DATA);
copyback = 1;
break;
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 319bfcc..3cbb60c 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -112,6 +112,9 @@ struct feat_ops {
/* get_feat_info is used to get feature HW info. */
bool (*get_feat_info)(const struct feat_node *feat,
uint32_t data[], unsigned int array_len);
+ /* get_val is used to get feature COS register value. */
+ bool (*get_val)(const struct feat_node *feat, unsigned int cos,
+ enum cbm_type type, uint64_t *val);
};
/*
@@ -251,9 +254,22 @@ static bool l3_cat_get_feat_info(const struct feat_node *feat,
return true;
}
+static bool l3_cat_get_val(const struct feat_node *feat, unsigned int cos,
+ enum cbm_type type, uint64_t *val)
+{
+ if ( cos > feat->info.l3_cat_info.cos_max )
+ /* Use default value. */
+ cos = 0;
+
+ *val = feat->cos_reg_val[cos];
+
+ return true;
+}
+
static const struct feat_ops l3_cat_ops = {
.get_cos_max = l3_cat_get_cos_max,
.get_feat_info = l3_cat_get_feat_info,
+ .get_val = l3_cat_get_val,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
@@ -498,10 +514,29 @@ int psr_get_info(unsigned int socket, enum cbm_type type,
return -ENOENT;
}
-int psr_get_l3_cbm(struct domain *d, unsigned int socket,
- uint64_t *cbm, enum cbm_type type)
+int psr_get_val(struct domain *d, unsigned int socket,
+ uint64_t *val, enum cbm_type type)
{
- return 0;
+ const struct psr_socket_info *info = get_socket_info(socket);
+ unsigned int cos = d->arch.psr_cos_ids[socket];
+ const struct feat_node *feat;
+ enum psr_feat_type feat_type;
+
+ if ( IS_ERR(info) )
+ return PTR_ERR(info);
+
+ feat_type = psr_cbm_type_to_feat_type(type);
+ list_for_each_entry(feat, &info->feat_list, list)
+ {
+ if ( feat->feature != feat_type )
+ continue;
+
+ if ( feat->ops.get_val(feat, cos, type, val) )
+ /* Found */
+ return 0;
+ }
+
+ return -ENOENT;
}
int psr_set_l3_cbm(struct domain *d, unsigned int socket,
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index e3b18bc..d50e359 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -70,8 +70,8 @@ void psr_ctxt_switch_to(struct domain *d);
int psr_get_info(unsigned int socket, enum cbm_type type,
uint32_t data[], unsigned int array_len);
-int psr_get_l3_cbm(struct domain *d, unsigned int socket,
- uint64_t *cbm, enum cbm_type type);
+int psr_get_val(struct domain *d, unsigned int socket,
+ uint64_t *val, enum cbm_type type);
int psr_set_l3_cbm(struct domain *d, unsigned int socket,
uint64_t cbm, enum cbm_type type);
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 08/24] x86: refactor psr: set value: implement framework.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (6 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 07/24] x86: refactor psr: implement get value flow Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 09/24] x86: refactor psr: set value: assemble features value array Yi Sun
` (15 subsequent siblings)
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
As set value flow is the most complicated one in psr, it will be
divided to some patches to make things clearer. This patch
implements the set value framework to show a whole picture firstly.
It also changes domctl interface to make it more general.
To make the set value flow be general and can support multiple features
at same time, it includes below steps:
1. Get COS ID of current domain using.
2. Assemble a value array to store all features current value
in it and replace the current value of the feature which is
being set to the new input value.
3. Find if there is already a COS ID on which all features'
values are same as the array. Then, we can reuse this COS
ID.
4. If fail to find, we need pick an available COS ID. Only COS ID which ref
is 0 or 1 can be picked.
5. Write all features MSRs according to the COS ID.
6. Update ref according to COS ID.
7. Save the COS ID into current domain's psr_cos_ids[socket] so that we
can know which COS the domain is using on the socket.
So, some functions are abstracted and the callback functions will be
implemented in next patches.
Here is an example to understand the process. The CPU supports
two featuers, e.g. L3 CAT and L2 CAT. user wants to set L3 CAT
of Dom1 to 0x1ff.
1. Get the old_cos of Dom1 which is 0. L3 CAT is the first
element of feature list. The COS registers values are below at
this time.
-------------------------------
| COS 0 | COS 1 | COS 2 | ... |
-------------------------------
L3 CAT | 0x7ff | ... | ... | ... |
-------------------------------
L2 CAT | 0xff | ... | ... | ... |
-------------------------------
2. Assemble The value array to be:
val[0]: 0x1ff
val[1]: 0xff
3. It cannot find a matching COS.
4. Allocate COS 1 to store the value set.
5. Write the COS 1 registers. The COS registers values are
changed to below now.
-------------------------------
| COS 0 | COS 1 | COS 2 | ... |
-------------------------------
L3 CAT | 0x7ff | 0x1ff | ... | ... |
-------------------------------
L2 CAT | 0xff | 0xff | ... | ... |
-------------------------------
6. The ref[1] is increased to 1 because Dom1 is using it now.
7. Save 1 to Dom1's psr_cos_ids[socket].
Then, user wants to set L3 CAT of Dom2 to 0x1ff too. The old_cos
of Dom2 is 0 too. Repeat above flow.
The val array assembled is:
val[0]: 0x1ff
val[1]: 0xff
So, it can find a matching COS, COS 1. Then, it can reuse COS 1
for Dom2.
The ref[1] is increased to 2 now because both Dom1 and Dom2 are
using this COS ID. Set 1 to Dom2's psr_cos_ids[socket].
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/domctl.c | 18 ++---
xen/arch/x86/psr.c | 202 +++++++++++++++++++++++++++++++++++++++++++++-
xen/include/asm-x86/psr.h | 4 +-
3 files changed, 210 insertions(+), 14 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 11d2127..db56500 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1365,21 +1365,21 @@ long arch_do_domctl(
switch ( domctl->u.psr_cat_op.cmd )
{
case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM:
- ret = psr_set_l3_cbm(d, domctl->u.psr_cat_op.target,
- domctl->u.psr_cat_op.data,
- PSR_CBM_TYPE_L3);
+ ret = psr_set_val(d, domctl->u.psr_cat_op.target,
+ domctl->u.psr_cat_op.data,
+ PSR_CBM_TYPE_L3);
break;
case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CODE:
- ret = psr_set_l3_cbm(d, domctl->u.psr_cat_op.target,
- domctl->u.psr_cat_op.data,
- PSR_CBM_TYPE_L3_CODE);
+ ret = psr_set_val(d, domctl->u.psr_cat_op.target,
+ domctl->u.psr_cat_op.data,
+ PSR_CBM_TYPE_L3_CODE);
break;
case XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA:
- ret = psr_set_l3_cbm(d, domctl->u.psr_cat_op.target,
- domctl->u.psr_cat_op.data,
- PSR_CBM_TYPE_L3_DATA);
+ ret = psr_set_val(d, domctl->u.psr_cat_op.target,
+ domctl->u.psr_cat_op.data,
+ PSR_CBM_TYPE_L3_DATA);
break;
case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM:
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 3cbb60c..050b0df 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -539,18 +539,214 @@ int psr_get_val(struct domain *d, unsigned int socket,
return -ENOENT;
}
-int psr_set_l3_cbm(struct domain *d, unsigned int socket,
- uint64_t cbm, enum cbm_type type)
+/* Set value functions */
+static unsigned int get_cos_num(const struct psr_socket_info *info)
{
return 0;
}
+static int assemble_val_array(uint64_t *val,
+ uint32_t array_len,
+ const struct psr_socket_info *info,
+ unsigned int old_cos)
+{
+ return -EINVAL;
+}
+
+static int set_new_val_to_array(uint64_t *val,
+ uint32_t array_len,
+ const struct psr_socket_info *info,
+ enum psr_feat_type feat_type,
+ enum cbm_type type,
+ uint64_t m)
+{
+ return -EINVAL;
+}
+
+static int find_cos(const uint64_t *val, uint32_t array_len,
+ enum psr_feat_type feat_type,
+ const struct psr_socket_info *info)
+{
+ return -ENOENT;
+}
+
+static int pick_avail_cos(const struct psr_socket_info *info,
+ const uint64_t *val, uint32_t array_len,
+ unsigned int old_cos,
+ enum psr_feat_type feat_type)
+{
+ return -ENOENT;
+}
+
+static int write_psr_msr(unsigned int socket, unsigned int cos,
+ const uint64_t *val)
+{
+ return -ENOENT;
+}
+
+int psr_set_val(struct domain *d, unsigned int socket,
+ uint64_t val, enum cbm_type type)
+{
+ unsigned int old_cos;
+ int cos, ret;
+ unsigned int *ref;
+ uint64_t *val_array;
+ struct psr_socket_info *info = get_socket_info(socket);
+ uint32_t array_len;
+ enum psr_feat_type feat_type;
+
+ if ( IS_ERR(info) )
+ return PTR_ERR(info);
+
+ feat_type = psr_cbm_type_to_feat_type(type);
+ if ( !test_bit(feat_type, &info->feat_mask) )
+ return -ENOENT;
+
+ /*
+ * Step 0:
+ * old_cos means the COS ID current domain is using. By default, it is 0.
+ *
+ * For every COS ID, there is a reference count to record how many domains
+ * are using the COS register corresponding to this COS ID.
+ * - If ref[old_cos] is 0, that means this COS is not used by any domain.
+ * - If ref[old_cos] is 1, that means this COS is only used by current
+ * domain.
+ * - If ref[old_cos] is more than 1, that mean multiple domains are using
+ * this COS.
+ */
+ old_cos = d->arch.psr_cos_ids[socket];
+ if ( old_cos > MAX_COS_REG_CNT )
+ return -EOVERFLOW;
+
+ ref = info->cos_ref;
+
+ /*
+ * Step 1:
+ * Assemle a value array to store all featues cos_reg_val[old_cos].
+ * And, set the input val into array according to the feature's
+ * position in array.
+ */
+ array_len = get_cos_num(info);
+ val_array = xzalloc_array(uint64_t, array_len);
+ if ( !val_array )
+ return -ENOMEM;
+
+ if ( (ret = assemble_val_array(val_array, array_len, info, old_cos)) != 0 )
+ {
+ xfree(val_array);
+ return ret;
+ }
+
+ if ( (ret = set_new_val_to_array(val_array, array_len, info,
+ feat_type, type, val)) != 0 )
+ {
+ xfree(val_array);
+ return ret;
+ }
+
+ /*
+ * Lock here to make sure the ref is not changed during find and
+ * write process.
+ */
+ spin_lock(&info->ref_lock);
+
+ /*
+ * Step 2:
+ * Try to find if there is already a COS ID on which all features' values
+ * are same as the array. Then, we can reuse this COS ID.
+ */
+ cos = find_cos(val_array, array_len, feat_type, info);
+ if ( cos >= 0 )
+ {
+ if ( cos == old_cos )
+ {
+ spin_unlock(&info->ref_lock);
+ xfree(val_array);
+ return 0;
+ }
+ }
+ else
+ {
+ /*
+ * Step 3:
+ * If fail to find, we need allocate a new COS ID.
+ * If multiple domains are using same COS ID, its ref is more
+ * than 1. That means we cannot free this COS to make current domain
+ * use it. Because other domains are using the value saved in the COS.
+ * Unless the ref is changed to 1 (mean only current domain is using
+ * it), we cannot allocate the COS ID to current domain.
+ * So, only the COS ID which ref is 1 or 0 can be allocated.
+ */
+ cos = pick_avail_cos(info, val_array, array_len, old_cos, feat_type);
+ if ( cos < 0 )
+ {
+ spin_unlock(&info->ref_lock);
+ xfree(val_array);
+ return cos;
+ }
+
+ /*
+ * Step 4:
+ * Write all features MSRs according to the COS ID.
+ */
+ ret = write_psr_msr(socket, cos, val_array);
+ if ( ret )
+ {
+ spin_unlock(&info->ref_lock);
+ xfree(val_array);
+ return ret;
+ }
+ }
+
+ /*
+ * Step 5:
+ * Update ref according to COS ID.
+ */
+ ref[cos]++;
+ ASSERT(ref[cos] || cos == 0);
+ ref[old_cos]--;
+ spin_unlock(&info->ref_lock);
+
+ /*
+ * Step 6:
+ * Save the COS ID into current domain's psr_cos_ids[] so that we can know
+ * which COS the domain is using on the socket. One domain can only use
+ * one COS ID at same time on each socket.
+ */
+ d->arch.psr_cos_ids[socket] = cos;
+ xfree(val_array);
+
+ return 0;
+}
+
/* Called with domain lock held, no extra lock needed for 'psr_cos_ids' */
static void psr_free_cos(struct domain *d)
{
- if( !d->arch.psr_cos_ids )
+ unsigned int socket, cos;
+
+ if ( !d->arch.psr_cos_ids )
return;
+ /* Domain is free so its cos_ref should be decreased. */
+ for ( socket = 0; socket < nr_sockets; socket++ )
+ {
+ struct psr_socket_info *info;
+
+ /* cos 0 is default one which does not need be handled. */
+ if ( (cos = d->arch.psr_cos_ids[socket]) == 0 )
+ continue;
+
+ /*
+ * If domain uses other cos ids, all corresponding refs must have been
+ * increased 1 for this domain. So, we need decrease them.
+ */
+ info = socket_info + socket;
+ ASSERT(info->cos_ref[cos] || cos == 0);
+ spin_lock(&info->ref_lock);
+ info->cos_ref[cos]--;
+ spin_unlock(&info->ref_lock);
+ }
+
xfree(d->arch.psr_cos_ids);
d->arch.psr_cos_ids = NULL;
}
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index d50e359..97214fe 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -72,8 +72,8 @@ int psr_get_info(unsigned int socket, enum cbm_type type,
uint32_t data[], unsigned int array_len);
int psr_get_val(struct domain *d, unsigned int socket,
uint64_t *val, enum cbm_type type);
-int psr_set_l3_cbm(struct domain *d, unsigned int socket,
- uint64_t cbm, enum cbm_type type);
+int psr_set_val(struct domain *d, unsigned int socket,
+ uint64_t val, enum cbm_type type);
int psr_domain_init(struct domain *d);
void psr_domain_free(struct domain *d);
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 09/24] x86: refactor psr: set value: assemble features value array.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (7 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 08/24] x86: refactor psr: set value: implement framework Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 10/24] x86: refactor psr: set value: implement cos finding flow Yi Sun
` (14 subsequent siblings)
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
Only can one COS ID be used by one domain at one time. That means all enabled
features' COS registers at this COS ID are valid for this domain at that time.
When user updates a feature's value, we need make sure all other features'
values are not affected. So, we firstly need assemble an array which contains
all features current values and replace the setting feature's value in array
to new value.
Then, we can try to find if there is a COS ID on which all features' COS
registers values are same as the array. If we can find, we just use this COS
ID. If fail to find, we need allocate a new COS ID.
This patch implements value array assembling flow.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 145 +++++++++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 142 insertions(+), 3 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 050b0df..7c6f2bf 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -115,6 +115,32 @@ struct feat_ops {
/* get_val is used to get feature COS register value. */
bool (*get_val)(const struct feat_node *feat, unsigned int cos,
enum cbm_type type, uint64_t *val);
+ /*
+ * get_cos_num is used to get the COS registers amount used by the
+ * feature for one setting, e.g. CDP uses 2 COSs but CAT uses 1.
+ */
+ unsigned int (*get_cos_num)(const struct feat_node *feat);
+ /*
+ * get_old_val and set_new_val are a pair of functions called in order.
+ * The caller will traverse all features in the list and call both
+ * functions for every feature to do below two things:
+ * 1. get old_cos register value of all supported features and
+ * 2. set the new value for the feature.
+ *
+ * All the values are set into value array according the traversal order,
+ * meaning the same order of feature list members.
+ *
+ * The return value meaning:
+ * 0 - success.
+ * negative - error.
+ */
+ int (*get_old_val)(uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int old_cos);
+ int (*set_new_val)(uint64_t val[],
+ const struct feat_node *feat,
+ enum cbm_type type,
+ uint64_t m);
};
/*
@@ -200,6 +226,29 @@ static enum psr_feat_type psr_cbm_type_to_feat_type(enum cbm_type type)
return feat_type;
}
+static bool psr_check_cbm(unsigned int cbm_len, uint64_t cbm)
+{
+ unsigned int first_bit, zero_bit;
+
+ /* Set bits should only in the range of [0, cbm_len). */
+ if ( cbm & (~0ull << cbm_len) )
+ return false;
+
+ /* At least one bit need to be set. */
+ if ( cbm == 0 )
+ return false;
+
+ first_bit = find_first_bit(&cbm, cbm_len);
+ zero_bit = find_next_zero_bit(&cbm, cbm_len, first_bit);
+
+ /* Set bits should be contiguous. */
+ if ( zero_bit < cbm_len &&
+ find_next_bit(&cbm, cbm_len, zero_bit) < cbm_len )
+ return false;
+
+ return true;
+}
+
/* L3 CAT functions implementation. */
static void l3_cat_init_feature(struct cpuid_leaf_regs regs,
struct feat_node *feat,
@@ -266,10 +315,45 @@ static bool l3_cat_get_val(const struct feat_node *feat, unsigned int cos,
return true;
}
+static unsigned int l3_cat_get_cos_num(const struct feat_node *feat)
+{
+ return 1;
+}
+
+static int l3_cat_get_old_val(uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int old_cos)
+{
+ if ( old_cos > feat->info.l3_cat_info.cos_max )
+ /* Use default value. */
+ old_cos = 0;
+
+ /* CAT */
+ val[0] = feat->cos_reg_val[old_cos];
+
+ return 0;
+}
+
+static int l3_cat_set_new_val(uint64_t val[],
+ const struct feat_node *feat,
+ enum cbm_type type,
+ uint64_t m)
+{
+ if ( !psr_check_cbm(feat->info.l3_cat_info.cbm_len, m) )
+ return -EINVAL;
+
+ val[0] = m;
+
+ return 0;
+}
+
static const struct feat_ops l3_cat_ops = {
.get_cos_max = l3_cat_get_cos_max,
.get_feat_info = l3_cat_get_feat_info,
.get_val = l3_cat_get_val,
+ .get_cos_num = l3_cat_get_cos_num,
+ .get_old_val = l3_cat_get_old_val,
+ .set_new_val = l3_cat_set_new_val,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
@@ -542,7 +626,14 @@ int psr_get_val(struct domain *d, unsigned int socket,
/* Set value functions */
static unsigned int get_cos_num(const struct psr_socket_info *info)
{
- return 0;
+ const struct feat_node *feat_tmp;
+ unsigned int num = 0;
+
+ /* Get all features total amount. */
+ list_for_each_entry(feat_tmp, &info->feat_list, list)
+ num += feat_tmp->ops.get_cos_num(feat_tmp);
+
+ return num;
}
static int assemble_val_array(uint64_t *val,
@@ -550,7 +641,25 @@ static int assemble_val_array(uint64_t *val,
const struct psr_socket_info *info,
unsigned int old_cos)
{
- return -EINVAL;
+ const struct feat_node *feat;
+ int ret;
+ uint64_t *val_tmp = val;
+
+ if ( !val )
+ return -EINVAL;
+
+ /* Get all features current values according to old_cos. */
+ list_for_each_entry(feat, &info->feat_list, list)
+ {
+ /* value getting order is same as feature list */
+ ret = feat->ops.get_old_val(val_tmp, feat, old_cos);
+
+ val_tmp += feat->ops.get_cos_num(feat);
+ if ( val_tmp - val > array_len)
+ return -EINVAL;
+ }
+
+ return 0;
}
static int set_new_val_to_array(uint64_t *val,
@@ -560,7 +669,37 @@ static int set_new_val_to_array(uint64_t *val,
enum cbm_type type,
uint64_t m)
{
- return -EINVAL;
+ const struct feat_node *feat;
+ int ret;
+ uint64_t *val_tmp = val;
+
+ /* Set new value into array according to feature's position in array. */
+ list_for_each_entry(feat, &info->feat_list, list)
+ {
+ if ( feat->feature != feat_type )
+ {
+ val_tmp += feat->ops.get_cos_num(feat);
+ if ( val_tmp - val > array_len)
+ return -EINVAL;
+
+ continue;
+ }
+
+ /*
+ * Value setting position is same as feature list.
+ * Different features may have different setting behaviors, e.g. CDP
+ * has two values (DATA/CODE) which need us to save input value to
+ * different position in the array according to type, so we have to
+ * maintain a callback function.
+ */
+ ret = feat->ops.set_new_val(val_tmp, feat, type, m);
+ if ( ret )
+ return ret;
+ else
+ break;
+ }
+
+ return 0;
}
static int find_cos(const uint64_t *val, uint32_t array_len,
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 10/24] x86: refactor psr: set value: implement cos finding flow.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (8 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 09/24] x86: refactor psr: set value: assemble features value array Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 11/24] x86: refactor psr: set value: implement cos id picking flow Yi Sun
` (13 subsequent siblings)
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
Continue with patch:
'x86: refactor psr: set value: assemble features value array'
We can try to find if there is a COS ID on which all features' COS registers
values are same as the array assembled before.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 93 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 93 insertions(+)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 7c6f2bf..8832e08 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -141,6 +141,19 @@ struct feat_ops {
const struct feat_node *feat,
enum cbm_type type,
uint64_t m);
+ /*
+ * compare_val is used in set value process to compare if the
+ * input value array can match all the features' COS registers values
+ * according to input cos id.
+ *
+ * The return value is the amount of entries to skip in the value array
+ * or error.
+ * 1 - one entry in value array.
+ * 2 - two entries in value array, e.g. CDP uses two entries.
+ * negative - error.
+ */
+ int (*compare_val)(const uint64_t val[], const struct feat_node *feat,
+ unsigned int cos, bool *found);
};
/*
@@ -347,6 +360,34 @@ static int l3_cat_set_new_val(uint64_t val[],
return 0;
}
+static int l3_cat_compare_val(const uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int cos, bool *found)
+{
+ uint64_t l3_def_cbm;
+
+ l3_def_cbm = (1ull << feat->info.l3_cat_info.cbm_len) - 1;
+
+ /*
+ * Different features' cos_max are different. If cos id of the feature
+ * being set exceeds other feature's cos_max, the val of other feature
+ * must be default value. HW supports such case.
+ */
+ if ( cos > feat->info.l3_cat_info.cos_max )
+ {
+ if ( val[0] != l3_def_cbm )
+ {
+ *found = false;
+ return -ENOENT;
+ }
+ *found = true;
+ }
+ else
+ *found = (val[0] == feat->cos_reg_val[cos]);
+
+ return 0;
+}
+
static const struct feat_ops l3_cat_ops = {
.get_cos_max = l3_cat_get_cos_max,
.get_feat_info = l3_cat_get_feat_info,
@@ -354,6 +395,7 @@ static const struct feat_ops l3_cat_ops = {
.get_cos_num = l3_cat_get_cos_num,
.get_old_val = l3_cat_get_old_val,
.set_new_val = l3_cat_set_new_val,
+ .compare_val = l3_cat_compare_val,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
@@ -706,6 +748,57 @@ static int find_cos(const uint64_t *val, uint32_t array_len,
enum psr_feat_type feat_type,
const struct psr_socket_info *info)
{
+ unsigned int cos;
+ const unsigned int *ref = info->cos_ref;
+ const struct feat_node *feat;
+ const uint64_t *val_tmp = val;
+ int ret;
+ bool found = false;
+ unsigned int cos_max = 0;
+
+ /* cos_max is the one of the feature which is being set. */
+ list_for_each_entry(feat, &info->feat_list, list)
+ {
+ if ( feat->feature != feat_type )
+ continue;
+
+ cos_max = feat->ops.get_cos_max(feat);
+ if ( cos_max > 0 )
+ break;
+ }
+
+ for ( cos = 0; cos <= cos_max; cos++ )
+ {
+ if ( cos && !ref[cos] )
+ continue;
+
+ /* Not found, need find again from beginning. */
+ val_tmp = val;
+ list_for_each_entry(feat, &info->feat_list, list)
+ {
+ /*
+ * Compare value according to feature list order.
+ * We must follow this order because value array is assembled
+ * as this order in get_old_set_new().
+ */
+ ret = feat->ops.compare_val(val_tmp, feat, cos, &found);
+ if ( ret < 0 )
+ return ret;
+
+ /* If fail to match, go to next cos to compare. */
+ if ( !found )
+ break;
+
+ val_tmp += feat->ops.get_cos_num(feat);
+ if ( val_tmp - val > array_len )
+ return -EINVAL;
+ }
+
+ /* For this COS ID all entries in the values array did match. Use it. */
+ if ( found )
+ return cos;
+ }
+
return -ENOENT;
}
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 11/24] x86: refactor psr: set value: implement cos id picking flow.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (9 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 10/24] x86: refactor psr: set value: implement cos finding flow Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 12/24] x86: refactor psr: set value: implement write msr flow Yi Sun
` (12 subsequent siblings)
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
Continue with previous patch:
'x86: refactor psr: set value: implement cos finding flow.'
If fail to find a COS ID, we need pick a new COS ID for domain. Only COS ID
that ref[COS_ID] is 1 or 0 can be picked to input a new set feature values.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 99 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 99 insertions(+)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 8832e08..c3e25bf 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -154,6 +154,17 @@ struct feat_ops {
*/
int (*compare_val)(const uint64_t val[], const struct feat_node *feat,
unsigned int cos, bool *found);
+ /*
+ * fits_cos_max is used to check if the input cos id exceeds the
+ * feature's cos_max and if the input value is not the default one.
+ * Even if the associated cos exceeds the cos_max, HW can work with default
+ * value. That is the reason we need check if input value is default one.
+ * If both criteria are fulfilled, that means the input exceeds the range.
+ * If not, that means the input fits the requirements.
+ */
+ bool (*fits_cos_max)(const uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int cos);
};
/*
@@ -388,6 +399,25 @@ static int l3_cat_compare_val(const uint64_t val[],
return 0;
}
+static bool l3_cat_fits_cos_max(const uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int cos)
+{
+ uint64_t l3_def_cbm;
+
+ l3_def_cbm = (1ull << feat->info.l3_cat_info.cbm_len) - 1;
+
+ if ( cos > feat->info.l3_cat_info.cos_max &&
+ val[0] != l3_def_cbm )
+ /*
+ * Exceed cos_max and value to set is not default,
+ * return error.
+ */
+ return false;
+
+ return true;
+}
+
static const struct feat_ops l3_cat_ops = {
.get_cos_max = l3_cat_get_cos_max,
.get_feat_info = l3_cat_get_feat_info,
@@ -396,6 +426,7 @@ static const struct feat_ops l3_cat_ops = {
.get_old_val = l3_cat_get_old_val,
.set_new_val = l3_cat_set_new_val,
.compare_val = l3_cat_compare_val,
+ .fits_cos_max = l3_cat_fits_cos_max,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
@@ -802,11 +833,79 @@ static int find_cos(const uint64_t *val, uint32_t array_len,
return -ENOENT;
}
+static bool fits_cos_max(const uint64_t *val,
+ uint32_t array_len,
+ const struct psr_socket_info *info,
+ unsigned int cos)
+{
+ unsigned int ret;
+ const uint64_t *val_tmp = val;
+ const struct feat_node *feat;
+
+ list_for_each_entry(feat, &info->feat_list, list)
+ {
+ ret = feat->ops.fits_cos_max(val_tmp, feat, cos);
+ if ( !ret )
+ return false;
+
+ val_tmp += feat->ops.get_cos_num(feat);
+ if ( val_tmp - val > array_len )
+ return false;
+ }
+
+ return true;
+}
+
static int pick_avail_cos(const struct psr_socket_info *info,
const uint64_t *val, uint32_t array_len,
unsigned int old_cos,
enum psr_feat_type feat_type)
{
+ unsigned int cos;
+ unsigned int cos_max = 0;
+ const struct feat_node *feat;
+ const unsigned int *ref = info->cos_ref;
+
+ /*
+ * cos_max is the one of the feature which is being set.
+ */
+ list_for_each_entry(feat, &info->feat_list, list)
+ {
+ if ( feat->feature != feat_type )
+ continue;
+
+ cos_max = feat->ops.get_cos_max(feat);
+ if ( cos_max > 0 )
+ break;
+ }
+
+ if ( !cos_max )
+ return -ENOENT;
+
+ /*
+ * If old cos is referred only by the domain, then use it. And, we cannot
+ * use id 0 because it stores the default values.
+ */
+ if ( old_cos && ref[old_cos] == 1 &&
+ fits_cos_max(val, array_len, info, old_cos) )
+ return old_cos;
+
+ /* Find an unused one other than cos0. */
+ for ( cos = 1; cos <= cos_max; cos++ )
+ {
+ /*
+ * ref is 0 means this COS is not used by other domain and
+ * can be used for current setting.
+ */
+ if ( !ref[cos] )
+ {
+ if ( !fits_cos_max(val, array_len, info, cos) )
+ return -ENOENT;
+
+ return cos;
+ }
+ }
+
return -ENOENT;
}
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 12/24] x86: refactor psr: set value: implement write msr flow.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (10 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 11/24] x86: refactor psr: set value: implement cos id picking flow Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 13/24] x86: refactor psr: implement CPU init and free flow for CDP Yi Sun
` (11 subsequent siblings)
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
Continue with previous patch:
'x86: refactor psr: set value: implement cos id picking flow.'
We have got all features values and COS ID to set. Then, we write MSRs of all
features except the setting value is same as original value.
Till now, set value process is completed.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 78 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 77 insertions(+), 1 deletion(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index c3e25bf..b8d3c82 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -165,6 +165,9 @@ struct feat_ops {
bool (*fits_cos_max)(const uint64_t val[],
const struct feat_node *feat,
unsigned int cos);
+ /* write_msr is used to write out feature MSR register. */
+ int (*write_msr)(unsigned int cos, const uint64_t val[],
+ struct feat_node *feat);
};
/*
@@ -418,6 +421,21 @@ static bool l3_cat_fits_cos_max(const uint64_t val[],
return true;
}
+static int l3_cat_write_msr(unsigned int cos, const uint64_t val[],
+ struct feat_node *feat)
+{
+ if ( cos > feat->info.l3_cat_info.cos_max )
+ return -EINVAL;
+
+ if ( feat->cos_reg_val[cos] != val[0] )
+ {
+ feat->cos_reg_val[cos] = val[0];
+ wrmsrl(MSR_IA32_PSR_L3_MASK(cos), val[0]);
+ }
+
+ return 0;
+}
+
static const struct feat_ops l3_cat_ops = {
.get_cos_max = l3_cat_get_cos_max,
.get_feat_info = l3_cat_get_feat_info,
@@ -427,6 +445,7 @@ static const struct feat_ops l3_cat_ops = {
.set_new_val = l3_cat_set_new_val,
.compare_val = l3_cat_compare_val,
.fits_cos_max = l3_cat_fits_cos_max,
+ .write_msr = l3_cat_write_msr,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
@@ -909,10 +928,67 @@ static int pick_avail_cos(const struct psr_socket_info *info,
return -ENOENT;
}
+static unsigned int get_socket_cpu(unsigned int socket)
+{
+ if ( likely(socket < nr_sockets) )
+ return cpumask_any(socket_cpumask[socket]);
+
+ return nr_cpu_ids;
+}
+
+struct cos_write_info
+{
+ unsigned int cos;
+ struct list_head *feat_list;
+ const uint64_t *val;
+};
+
+static void do_write_psr_msr(void *data)
+{
+ struct cos_write_info *info = (struct cos_write_info *)data;
+ unsigned int cos = info->cos;
+ struct list_head *feat_list= info->feat_list;
+ const uint64_t *val = info->val;
+ struct feat_node *feat;
+ int ret;
+
+ if ( !feat_list )
+ return;
+
+ /* We need set all features values into MSRs. */
+ list_for_each_entry(feat, feat_list, list)
+ {
+ ret = feat->ops.write_msr(cos, val, feat);
+ if ( ret < 0 )
+ return;
+
+ val += feat->ops.get_cos_num(feat);
+ }
+}
+
static int write_psr_msr(unsigned int socket, unsigned int cos,
const uint64_t *val)
{
- return -ENOENT;
+ struct psr_socket_info *info = get_socket_info(socket);
+ struct cos_write_info data =
+ {
+ .cos = cos,
+ .feat_list = &info->feat_list,
+ .val = val,
+ };
+
+ if ( socket == cpu_to_socket(smp_processor_id()) )
+ do_write_psr_msr(&data);
+ else
+ {
+ unsigned int cpu = get_socket_cpu(socket);
+
+ if ( cpu >= nr_cpu_ids )
+ return -ENOTSOCK;
+ on_selected_cpus(cpumask_of(cpu), do_write_psr_msr, &data, 1);
+ }
+
+ return 0;
}
int psr_set_val(struct domain *d, unsigned int socket,
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 13/24] x86: refactor psr: implement CPU init and free flow for CDP.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (11 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 12/24] x86: refactor psr: set value: implement write msr flow Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 14/24] x86: refactor psr: implement get hw info " Yi Sun
` (10 subsequent siblings)
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements the CPU init and free flow for CDP including L3 CDP
initialization callback function.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 98 +++++++++++++++++++++++++++++++++++++++++++++++++++---
1 file changed, 93 insertions(+), 5 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index b8d3c82..a979128 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -93,6 +93,7 @@ struct psr_cat_hw_info {
struct feat_hw_info {
union {
struct psr_cat_hw_info l3_cat_info;
+ struct psr_cat_hw_info l3_cdp_info;
};
};
@@ -197,6 +198,21 @@ struct cpuid_leaf_regs {
unsigned int ecx;
unsigned int edx;
};
+/*
+ * get_data - get DATA COS register value from input COS ID.
+ * @feat: the feature list entry.
+ * @cos: the COS ID.
+ */
+#define get_cdp_data(feat, cos) \
+ ( feat->cos_reg_val[cos * 2] )
+
+/*
+ * get_cdp_code - get CODE COS register value from input COS ID.
+ * @feat: the feature list entry.
+ * @cos: the COS ID.
+ */
+#define get_cdp_code(feat, cos) \
+ ( feat->cos_reg_val[cos * 2 + 1] )
struct psr_assoc {
uint64_t val;
@@ -219,6 +235,7 @@ static DEFINE_PER_CPU(struct psr_assoc, psr_assoc);
* inserted into feature list in cpu_init_work().
*/
static struct feat_node *feat_l3_cat;
+static struct feat_node *feat_l3_cdp;
/* Common functions. */
static void free_feature(struct psr_socket_info *info)
@@ -448,6 +465,61 @@ static const struct feat_ops l3_cat_ops = {
.write_msr = l3_cat_write_msr,
};
+/* L3 CDP functions implementation. */
+static void l3_cdp_init_feature(struct cpuid_leaf_regs regs,
+ struct feat_node *feat,
+ struct psr_socket_info *info)
+{
+ struct psr_cat_hw_info l3_cdp;
+ unsigned int socket;
+ uint64_t val;
+
+ /* No valid value so do not enable feature. */
+ if ( !regs.eax || !regs.edx )
+ return;
+
+ l3_cdp.cbm_len = (regs.eax & CAT_CBM_LEN_MASK) + 1;
+ /* Cut half of cos_max when CDP is enabled. */
+ l3_cdp.cos_max = min(opt_cos_max, regs.edx & CAT_COS_MAX_MASK) >> 1;
+
+ /* cos=0 is reserved as default cbm(all ones). */
+ get_cdp_code(feat, 0) =
+ (1ull << l3_cdp.cbm_len) - 1;
+ get_cdp_data(feat, 0) =
+ (1ull << l3_cdp.cbm_len) - 1;
+
+ /* We only write mask1 since mask0 is always all ones by default. */
+ wrmsrl(MSR_IA32_PSR_L3_MASK(1), (1ull << l3_cdp.cbm_len) - 1);
+ rdmsrl(MSR_IA32_PSR_L3_QOS_CFG, val);
+ wrmsrl(MSR_IA32_PSR_L3_QOS_CFG, val | (1 << PSR_L3_QOS_CDP_ENABLE_BIT));
+
+ feat->feature = PSR_SOCKET_L3_CDP;
+ __set_bit(PSR_SOCKET_L3_CDP, &info->feat_mask);
+
+ feat->info.l3_cdp_info = l3_cdp;
+
+ info->nr_feat++;
+
+ /* Add this feature into list. */
+ list_add_tail(&feat->list, &info->feat_list);
+
+ socket = cpu_to_socket(smp_processor_id());
+ if ( opt_cpu_info )
+ printk(XENLOG_INFO
+ "L3 CDP: enabled on socket %u, cos_max:%u, cbm_len:%u\n",
+ socket, feat->info.l3_cdp_info.cos_max,
+ feat->info.l3_cdp_info.cbm_len);
+}
+
+static unsigned int l3_cdp_get_cos_max(const struct feat_node *feat)
+{
+ return feat->info.l3_cdp_info.cos_max;
+}
+
+struct feat_ops l3_cdp_ops = {
+ .get_cos_max = l3_cdp_get_cos_max,
+};
+
static void __init parse_psr_bool(char *s, char *value, char *feature,
unsigned int mask)
{
@@ -1207,11 +1279,19 @@ static void cpu_init_work(void)
cpuid_count(PSR_CPUID_LEVEL_CAT, 1,
®s.eax, ®s.ebx, ®s.ecx, ®s.edx);
- feat = feat_l3_cat;
- feat_l3_cat = NULL;
- feat->ops = l3_cat_ops;
-
- l3_cat_init_feature(regs, feat, info);
+ if ( (regs.ecx & PSR_CAT_CDP_CAPABILITY) && (opt_psr & PSR_CDP) &&
+ !test_bit(PSR_SOCKET_L3_CDP, &info->feat_mask) )
+ {
+ feat = feat_l3_cdp;
+ feat_l3_cdp = NULL;
+ feat->ops = l3_cdp_ops;
+ l3_cdp_init_feature(regs, feat, info);
+ } else {
+ feat = feat_l3_cat;
+ feat_l3_cat = NULL;
+ feat->ops = l3_cat_ops;
+ l3_cat_init_feature(regs, feat, info);
+ }
}
}
@@ -1263,6 +1343,14 @@ static int psr_cpu_prepare(unsigned int cpu)
(feat_l3_cat = xzalloc(struct feat_node)) == NULL )
return -ENOMEM;
+ if ( feat_l3_cdp == NULL &&
+ (feat_l3_cdp = xzalloc(struct feat_node)) == NULL )
+ {
+ xfree(feat_l3_cat);
+ feat_l3_cat = NULL;
+ return -ENOMEM;
+ }
+
return 0;
}
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 14/24] x86: refactor psr: implement get hw info flow for CDP.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (12 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 13/24] x86: refactor psr: implement CPU init and free flow for CDP Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 15/24] x86: refactor psr: implement get value " Yi Sun
` (9 subsequent siblings)
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements get HW info flow for CDP including L3 CDP callback
function.
It also changes sysctl function to make it work for CDP.
With this patch, 'psr-hwinfo' can work for L3 CDP.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 18 ++++++++++++++++++
xen/arch/x86/sysctl.c | 24 +++++++++++++++++++++---
2 files changed, 39 insertions(+), 3 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index a979128..b856761 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -262,6 +262,10 @@ static enum psr_feat_type psr_cbm_type_to_feat_type(enum cbm_type type)
case PSR_CBM_TYPE_L3:
feat_type = PSR_SOCKET_L3_CAT;
break;
+ case PSR_CBM_TYPE_L3_DATA:
+ case PSR_CBM_TYPE_L3_CODE:
+ feat_type = PSR_SOCKET_L3_CDP;
+ break;
default:
feat_type = 0xFFFF;
break;
@@ -516,8 +520,22 @@ static unsigned int l3_cdp_get_cos_max(const struct feat_node *feat)
return feat->info.l3_cdp_info.cos_max;
}
+static bool l3_cdp_get_feat_info(const struct feat_node *feat,
+ uint32_t data[], uint32_t array_len)
+{
+ if ( !data || 3 > array_len )
+ return false;
+
+ data[CBM_LEN] = feat->info.l3_cdp_info.cbm_len;
+ data[COS_MAX] = feat->info.l3_cdp_info.cos_max;
+ data[PSR_FLAG] |= XEN_SYSCTL_PSR_CAT_L3_CDP;
+
+ return true;
+}
+
struct feat_ops l3_cdp_ops = {
.get_cos_max = l3_cdp_get_cos_max,
+ .get_feat_info = l3_cdp_get_feat_info,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index d90db78..a4c8cfe 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -181,9 +181,27 @@ long arch_do_sysctl(
ret = psr_get_info(sysctl->u.psr_cat_op.target,
PSR_CBM_TYPE_L3, data, 3);
- sysctl->u.psr_cat_op.u.l3_info.cbm_len = data[CBM_LEN];
- sysctl->u.psr_cat_op.u.l3_info.cos_max = data[COS_MAX];
- sysctl->u.psr_cat_op.u.l3_info.flags = data[PSR_FLAG];
+ if ( !ret )
+ {
+ sysctl->u.psr_cat_op.u.l3_info.cbm_len = data[CBM_LEN];
+ sysctl->u.psr_cat_op.u.l3_info.cos_max = data[COS_MAX];
+ sysctl->u.psr_cat_op.u.l3_info.flags = data[PSR_FLAG];
+ } else {
+ /*
+ * Check if CDP is enabled.
+ *
+ * Per spec, L3 CAT and CDP cannot co-exist. So, we need replace
+ * output values to CDP's if it is enabled.
+ */
+ ret = psr_get_info(sysctl->u.psr_cat_op.target,
+ PSR_CBM_TYPE_L3_CODE, data, 3);
+ if ( !ret )
+ {
+ sysctl->u.psr_cat_op.u.l3_info.cbm_len = data[CBM_LEN];
+ sysctl->u.psr_cat_op.u.l3_info.cos_max = data[COS_MAX];
+ sysctl->u.psr_cat_op.u.l3_info.flags = data[PSR_FLAG];
+ }
+ }
if ( !ret && __copy_field_to_guest(u_sysctl, sysctl, u.psr_cat_op) )
ret = -EFAULT;
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 15/24] x86: refactor psr: implement get value flow for CDP.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (13 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 14/24] x86: refactor psr: implement get hw info " Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 16/24] x86: refactor psr: implement set value callback functions " Yi Sun
` (8 subsequent siblings)
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements L3 CDP get value callback function.
With this patch, 'psr-cat-show' can work for L3 CDP.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index b856761..dc062ff 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -533,9 +533,25 @@ static bool l3_cdp_get_feat_info(const struct feat_node *feat,
return true;
}
+static bool l3_cdp_get_val(const struct feat_node *feat, unsigned int cos,
+ enum cbm_type type, uint64_t *val)
+{
+ if ( cos > feat->info.l3_cdp_info.cos_max )
+ /* Use default value. */
+ cos = 0;
+
+ if ( type == PSR_CBM_TYPE_L3_DATA )
+ *val = get_cdp_data(feat, cos);
+ else
+ *val = get_cdp_code(feat, cos);
+
+ return true;
+}
+
struct feat_ops l3_cdp_ops = {
.get_cos_max = l3_cdp_get_cos_max,
.get_feat_info = l3_cdp_get_feat_info,
+ .get_val = l3_cdp_get_val,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 16/24] x86: refactor psr: implement set value callback functions for CDP.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (14 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 15/24] x86: refactor psr: implement get value " Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 17/24] x86: L2 CAT: implement CPU init and free flow Yi Sun
` (7 subsequent siblings)
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements L3 CDP set value related callback functions.
With this patch, 'psr-cat-cbm-set' command can work for L3 CDP.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 118 +++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 118 insertions(+)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index dc062ff..596e5b1 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -548,10 +548,128 @@ static bool l3_cdp_get_val(const struct feat_node *feat, unsigned int cos,
return true;
}
+static unsigned int l3_cdp_get_cos_num(const struct feat_node *feat)
+{
+ return 2;
+}
+
+static int l3_cdp_get_old_val(uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int old_cos)
+{
+ if ( old_cos > feat->info.l3_cdp_info.cos_max )
+ /* Use default value. */
+ old_cos = 0;
+
+ /* Data */
+ val[0] = get_cdp_data(feat, old_cos);
+ /* Code */
+ val[1] = get_cdp_code(feat, old_cos);
+
+ return 0;
+}
+
+static int l3_cdp_set_new_val(uint64_t val[],
+ const struct feat_node *feat,
+ enum cbm_type type,
+ uint64_t m)
+{
+ if ( !psr_check_cbm(feat->info.l3_cdp_info.cbm_len, m) )
+ return -EINVAL;
+
+ if ( type == PSR_CBM_TYPE_L3_DATA )
+ val[0] = m;
+ else
+ val[1] = m;
+
+ return 0;
+}
+
+static int l3_cdp_compare_val(const uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int cos, bool *found)
+{
+ uint64_t l3_def_cbm;
+
+ l3_def_cbm = (1ull << feat->info.l3_cdp_info.cbm_len) - 1;
+
+ /*
+ * Different features' cos_max are different. If cos id of the feature
+ * being set exceeds other feature's cos_max, the val of other feature
+ * must be default value. HW supports such case.
+ */
+ if ( cos > feat->info.l3_cdp_info.cos_max )
+ {
+ if ( val[0] != l3_def_cbm ||
+ val[1] != l3_def_cbm )
+ {
+ *found = false;
+ return -ENOENT;
+ }
+ *found = true;
+ }
+ else
+ *found = (val[0] == get_cdp_data(feat, cos) &&
+ val[1] == get_cdp_code(feat, cos));
+
+ return 0;
+}
+
+static bool l3_cdp_fits_cos_max(const uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int cos)
+{
+ uint64_t l3_def_cbm;
+
+ l3_def_cbm = (1ull << feat->info.l3_cdp_info.cbm_len) - 1;
+
+ if ( cos > feat->info.l3_cdp_info.cos_max &&
+ (val[0] != l3_def_cbm || val[1] != l3_def_cbm) )
+ /*
+ * Exceed cos_max and value to set is not default,
+ * return error.
+ */
+ return false;
+
+ return true;
+}
+
+static int l3_cdp_write_msr(unsigned int cos, const uint64_t val[],
+ struct feat_node *feat)
+{
+ /*
+ * If input cos is more than the cos_max of the feature, we should
+ * not set the value.
+ */
+ if ( cos > feat->info.l3_cdp_info.cos_max )
+ return -EINVAL;
+
+ /* Data */
+ if ( get_cdp_data(feat, cos) != val[0] )
+ {
+ get_cdp_data(feat, cos) = val[0];
+ wrmsrl(MSR_IA32_PSR_L3_MASK_DATA(cos), val[0]);
+ }
+ /* Code */
+ if ( get_cdp_code(feat, cos) != val[1] )
+ {
+ get_cdp_code(feat, cos) = val[1];
+ wrmsrl(MSR_IA32_PSR_L3_MASK_CODE(cos), val[1]);
+ }
+
+ return 0;
+}
+
struct feat_ops l3_cdp_ops = {
.get_cos_max = l3_cdp_get_cos_max,
.get_feat_info = l3_cdp_get_feat_info,
.get_val = l3_cdp_get_val,
+ .get_cos_num = l3_cdp_get_cos_num,
+ .get_old_val = l3_cdp_get_old_val,
+ .set_new_val = l3_cdp_set_new_val,
+ .compare_val = l3_cdp_compare_val,
+ .fits_cos_max = l3_cdp_fits_cos_max,
+ .write_msr = l3_cdp_write_msr,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 17/24] x86: L2 CAT: implement CPU init and free flow.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (15 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 16/24] x86: refactor psr: implement set value callback functions " Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 18/24] x86: L2 CAT: implement get hw info flow Yi Sun
` (6 subsequent siblings)
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements the CPU init and free flow for L2 CAT including
L2 CAT initialization callback function.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 72 +++++++++++++++++++++++++++++++++++++++++++++++
xen/include/asm-x86/psr.h | 1 +
2 files changed, 73 insertions(+)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 596e5b1..5320ae6 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -94,6 +94,7 @@ struct feat_hw_info {
union {
struct psr_cat_hw_info l3_cat_info;
struct psr_cat_hw_info l3_cdp_info;
+ struct psr_cat_hw_info l2_cat_info;
};
};
@@ -236,6 +237,7 @@ static DEFINE_PER_CPU(struct psr_assoc, psr_assoc);
*/
static struct feat_node *feat_l3_cat;
static struct feat_node *feat_l3_cdp;
+static struct feat_node *feat_l2_cat;
/* Common functions. */
static void free_feature(struct psr_socket_info *info)
@@ -672,6 +674,51 @@ struct feat_ops l3_cdp_ops = {
.write_msr = l3_cdp_write_msr,
};
+/* L2 CAT callback functions implementation. */
+static void l2_cat_init_feature(struct cpuid_leaf_regs regs,
+ struct feat_node *feat,
+ struct psr_socket_info *info)
+{
+ struct psr_cat_hw_info l2_cat;
+ unsigned int socket;
+
+ /* No valid values so do not enable the feature. */
+ if ( !regs.eax || !regs.edx )
+ return;
+
+ l2_cat.cbm_len = (regs.eax & CAT_CBM_LEN_MASK) + 1;
+ l2_cat.cos_max = min(opt_cos_max, regs.edx & CAT_COS_MAX_MASK);
+
+ /* cos=0 is reserved as default cbm(all ones). */
+ feat->cos_reg_val[0] = (1ull << l2_cat.cbm_len) - 1;
+
+ feat->feature = PSR_SOCKET_L2_CAT;
+ __set_bit(PSR_SOCKET_L2_CAT, &info->feat_mask);
+
+ feat->info.l2_cat_info = l2_cat;
+
+ info->nr_feat++;
+
+ /* Add this feature into list. */
+ list_add_tail(&feat->list, &info->feat_list);
+
+ socket = cpu_to_socket(smp_processor_id());
+ if ( opt_cpu_info )
+ printk(XENLOG_INFO
+ "L2 CAT: enabled on socket %u, cos_max:%u, cbm_len:%u.\n",
+ socket, feat->info.l2_cat_info.cos_max,
+ feat->info.l2_cat_info.cbm_len);
+}
+
+static unsigned int l2_cat_get_cos_max(const struct feat_node *feat)
+{
+ return feat->info.l2_cat_info.cos_max;
+}
+
+struct feat_ops l2_cat_ops = {
+ .get_cos_max = l2_cat_get_cos_max,
+};
+
static void __init parse_psr_bool(char *s, char *value, char *feature,
unsigned int mask)
{
@@ -1445,6 +1492,20 @@ static void cpu_init_work(void)
l3_cat_init_feature(regs, feat, info);
}
}
+
+ cpuid_count(PSR_CPUID_LEVEL_CAT, 0,
+ ®s.eax, ®s.ebx, ®s.ecx, ®s.edx);
+ if ( regs.ebx & PSR_RESOURCE_TYPE_L2 )
+ {
+ /* Initialize L2 CAT according to CPUID. */
+ cpuid_count(PSR_CPUID_LEVEL_CAT, 2,
+ ®s.eax, ®s.ebx, ®s.ecx, ®s.edx);
+
+ feat = feat_l2_cat;
+ feat_l2_cat = NULL;
+ feat->ops = l2_cat_ops;
+ l2_cat_init_feature(regs, feat, info);
+ }
}
static void cpu_fini_work(unsigned int cpu)
@@ -1503,6 +1564,17 @@ static int psr_cpu_prepare(unsigned int cpu)
return -ENOMEM;
}
+ if ( feat_l2_cat == NULL &&
+ (feat_l2_cat = xzalloc(struct feat_node)) == NULL )
+ {
+ xfree(feat_l3_cat);
+ feat_l3_cat = NULL;
+
+ xfree(feat_l3_cdp);
+ feat_l3_cdp = NULL;
+ return -ENOMEM;
+ }
+
return 0;
}
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index 97214fe..d2c7a13 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -23,6 +23,7 @@
/* Resource Type Enumeration */
#define PSR_RESOURCE_TYPE_L3 0x2
+#define PSR_RESOURCE_TYPE_L2 0x4
/* L3 Monitoring Features */
#define PSR_CMT_L3_OCCUPANCY 0x1
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 18/24] x86: L2 CAT: implement get hw info flow.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (16 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 17/24] x86: L2 CAT: implement CPU init and free flow Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 19/24] x86: L2 CAT: implement get value flow Yi Sun
` (5 subsequent siblings)
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements get HW info flow for L2 CAT including L2 CAT callback
function.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 16 ++++++++++++++++
xen/arch/x86/sysctl.c | 15 +++++++++++++++
xen/include/asm-x86/psr.h | 1 +
xen/include/public/sysctl.h | 6 ++++++
4 files changed, 38 insertions(+)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 5320ae6..b630c48 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -268,6 +268,9 @@ static enum psr_feat_type psr_cbm_type_to_feat_type(enum cbm_type type)
case PSR_CBM_TYPE_L3_CODE:
feat_type = PSR_SOCKET_L3_CDP;
break;
+ case PSR_CBM_TYPE_L2:
+ feat_type = PSR_SOCKET_L2_CAT;
+ break;
default:
feat_type = 0xFFFF;
break;
@@ -715,8 +718,21 @@ static unsigned int l2_cat_get_cos_max(const struct feat_node *feat)
return feat->info.l2_cat_info.cos_max;
}
+static bool l2_cat_get_feat_info(const struct feat_node *feat,
+ uint32_t data[], uint32_t array_len)
+{
+ if ( !data || 2 > array_len )
+ return false;
+
+ data[CBM_LEN] = feat->info.l2_cat_info.cbm_len;
+ data[COS_MAX] = feat->info.l2_cat_info.cos_max;
+
+ return true;
+}
+
struct feat_ops l2_cat_ops = {
.get_cos_max = l2_cat_get_cos_max,
+ .get_feat_info = l2_cat_get_feat_info,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index a4c8cfe..ae3600a 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -207,6 +207,21 @@ long arch_do_sysctl(
ret = -EFAULT;
break;
}
+ case XEN_SYSCTL_PSR_CAT_get_l2_info:
+ {
+ uint32_t dat[2];
+ ret = psr_get_info(sysctl->u.psr_cat_op.target,
+ PSR_CBM_TYPE_L2, dat, 2);
+ if ( ret )
+ break;
+
+ sysctl->u.psr_cat_op.u.l2_info.cbm_len = dat[CBM_LEN];
+ sysctl->u.psr_cat_op.u.l2_info.cos_max = dat[COS_MAX];
+
+ if ( !ret && __copy_field_to_guest(u_sysctl, sysctl, u.psr_cat_op) )
+ ret = -EFAULT;
+ break;
+ }
default:
ret = -EOPNOTSUPP;
break;
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index d2c7a13..31aa332 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -56,6 +56,7 @@ enum cbm_type {
PSR_CBM_TYPE_L3,
PSR_CBM_TYPE_L3_CODE,
PSR_CBM_TYPE_L3_DATA,
+ PSR_CBM_TYPE_L2,
};
extern struct psr_cmt *psr_cmt;
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index 00f5e77..cbf5372 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -744,6 +744,7 @@ typedef struct xen_sysctl_pcitopoinfo xen_sysctl_pcitopoinfo_t;
DEFINE_XEN_GUEST_HANDLE(xen_sysctl_pcitopoinfo_t);
#define XEN_SYSCTL_PSR_CAT_get_l3_info 0
+#define XEN_SYSCTL_PSR_CAT_get_l2_info 1
struct xen_sysctl_psr_cat_op {
uint32_t cmd; /* IN: XEN_SYSCTL_PSR_CAT_* */
uint32_t target; /* IN */
@@ -754,6 +755,11 @@ struct xen_sysctl_psr_cat_op {
#define XEN_SYSCTL_PSR_CAT_L3_CDP (1u << 0)
uint32_t flags; /* OUT: CAT flags */
} l3_info;
+
+ struct {
+ uint32_t cbm_len; /* OUT: CBM length */
+ uint32_t cos_max; /* OUT: Maximum COS */
+ } l2_info;
} u;
};
typedef struct xen_sysctl_psr_cat_op xen_sysctl_psr_cat_op_t;
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 19/24] x86: L2 CAT: implement get value flow.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (17 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 18/24] x86: L2 CAT: implement get hw info flow Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 20/24] x86: L2 CAT: implement set " Yi Sun
` (4 subsequent siblings)
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements L2 CAT get value callback function and
interface in domctl.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/domctl.c | 7 +++++++
xen/arch/x86/psr.c | 12 ++++++++++++
xen/include/public/domctl.h | 1 +
3 files changed, 20 insertions(+)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index db56500..af6153d 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1403,6 +1403,13 @@ long arch_do_domctl(
copyback = 1;
break;
+ case XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM:
+ ret = psr_get_val(d, domctl->u.psr_cat_op.target,
+ &domctl->u.psr_cat_op.data,
+ PSR_CBM_TYPE_L2);
+ copyback = 1;
+ break;
+
default:
ret = -EOPNOTSUPP;
break;
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index b630c48..1fad540 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -730,9 +730,21 @@ static bool l2_cat_get_feat_info(const struct feat_node *feat,
return true;
}
+static bool l2_cat_get_val(const struct feat_node *feat, unsigned int cos,
+ enum cbm_type type, uint64_t *val)
+{
+ if ( cos > feat->info.l2_cat_info.cos_max )
+ cos = 0;
+
+ *val = feat->cos_reg_val[cos];
+
+ return true;
+}
+
struct feat_ops l2_cat_ops = {
.get_cos_max = l2_cat_get_cos_max,
.get_feat_info = l2_cat_get_feat_info,
+ .get_val = l2_cat_get_val,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 85cbb7c..8c183ba 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1138,6 +1138,7 @@ struct xen_domctl_psr_cat_op {
#define XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA 3
#define XEN_DOMCTL_PSR_CAT_OP_GET_L3_CODE 4
#define XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA 5
+#define XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM 7
uint32_t cmd; /* IN: XEN_DOMCTL_PSR_CAT_OP_* */
uint32_t target; /* IN */
uint64_t data; /* IN/OUT */
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 20/24] x86: L2 CAT: implement set value flow.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (18 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 19/24] x86: L2 CAT: implement get value flow Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 21/24] tools: L2 CAT: support get HW info for L2 CAT Yi Sun
` (3 subsequent siblings)
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements L2 CAT set value related callback functions
and domctl interface.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/domctl.c | 6 +++
xen/arch/x86/psr.c | 92 +++++++++++++++++++++++++++++++++++++++++
xen/include/asm-x86/msr-index.h | 1 +
xen/include/public/domctl.h | 1 +
4 files changed, 100 insertions(+)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index af6153d..2767c6a 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1382,6 +1382,12 @@ long arch_do_domctl(
PSR_CBM_TYPE_L3_DATA);
break;
+ case XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM:
+ ret = psr_set_val(d, domctl->u.psr_cat_op.target,
+ domctl->u.psr_cat_op.data,
+ PSR_CBM_TYPE_L2);
+ break;
+
case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM:
ret = psr_get_val(d, domctl->u.psr_cat_op.target,
&domctl->u.psr_cat_op.data,
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 1fad540..13d85e0 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -741,10 +741,102 @@ static bool l2_cat_get_val(const struct feat_node *feat, unsigned int cos,
return true;
}
+static unsigned int l2_cat_get_cos_num(const struct feat_node *feat)
+{
+ /* L2 CAT uses one COS. */
+ return 1;
+}
+
+static int l2_cat_get_old_val(uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int old_cos)
+{
+ if ( old_cos > feat->info.l2_cat_info.cos_max )
+ /* Use default value. */
+ old_cos = 0;
+
+ val[0] = feat->cos_reg_val[old_cos];
+
+ return 0;
+}
+
+static int l2_cat_set_new_val(uint64_t val[],
+ const struct feat_node *feat,
+ enum cbm_type type,
+ uint64_t m)
+{
+ if ( !psr_check_cbm(feat->info.l2_cat_info.cbm_len, m) )
+ return -EINVAL;
+
+ val[0] = m;
+
+ return 0;
+}
+
+static int l2_cat_compare_val(const uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int cos, bool *found)
+{
+ uint64_t l2_def_cbm;
+
+ l2_def_cbm = (1ull << feat->info.l2_cat_info.cbm_len) - 1;
+
+ if ( cos > feat->info.l2_cat_info.cos_max )
+ {
+ if ( val[0] != l2_def_cbm )
+ {
+ *found = false;
+ return -ENOENT;
+ }
+ *found = true;
+ }
+ else
+ *found = (val[0] == feat->cos_reg_val[cos]);
+
+ return 0;
+}
+
+static bool l2_cat_fits_cos_max(const uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int cos)
+{
+ uint64_t l2_def_cbm;
+
+ l2_def_cbm = (1ull << feat->info.l2_cat_info.cbm_len) - 1;
+
+ if ( cos > feat->info.l2_cat_info.cos_max &&
+ val[0] != l2_def_cbm )
+ /*
+ * Exceed cos_max and value to set is not default,
+ * return error.
+ */
+ return false;
+
+ return true;
+}
+
+static int l2_cat_write_msr(unsigned int cos, const uint64_t val[],
+ struct feat_node *feat)
+{
+ if ( cos > feat->info.l2_cat_info.cos_max )
+ return -EINVAL;
+
+ feat->cos_reg_val[cos] = val[0];
+ wrmsrl(MSR_IA32_PSR_L2_MASK(cos), val[0]);
+
+ return 0;
+}
+
struct feat_ops l2_cat_ops = {
.get_cos_max = l2_cat_get_cos_max,
.get_feat_info = l2_cat_get_feat_info,
.get_val = l2_cat_get_val,
+ .get_cos_num = l2_cat_get_cos_num,
+ .get_old_val = l2_cat_get_old_val,
+ .set_new_val = l2_cat_set_new_val,
+ .compare_val = l2_cat_compare_val,
+ .fits_cos_max = l2_cat_fits_cos_max,
+ .write_msr = l2_cat_write_msr,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
diff --git a/xen/include/asm-x86/msr-index.h b/xen/include/asm-x86/msr-index.h
index 98dbff1..a41e63a 100644
--- a/xen/include/asm-x86/msr-index.h
+++ b/xen/include/asm-x86/msr-index.h
@@ -343,6 +343,7 @@
#define MSR_IA32_PSR_L3_MASK(n) (0x00000c90 + (n))
#define MSR_IA32_PSR_L3_MASK_CODE(n) (0x00000c90 + (n) * 2 + 1)
#define MSR_IA32_PSR_L3_MASK_DATA(n) (0x00000c90 + (n) * 2)
+#define MSR_IA32_PSR_L2_MASK(n) (0x00000d10 + (n))
/* Intel Model 6 */
#define MSR_P6_PERFCTR(n) (0x000000c1 + (n))
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 8c183ba..523a2cd 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1138,6 +1138,7 @@ struct xen_domctl_psr_cat_op {
#define XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA 3
#define XEN_DOMCTL_PSR_CAT_OP_GET_L3_CODE 4
#define XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA 5
+#define XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM 6
#define XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM 7
uint32_t cmd; /* IN: XEN_DOMCTL_PSR_CAT_OP_* */
uint32_t target; /* IN */
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 21/24] tools: L2 CAT: support get HW info for L2 CAT.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (19 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 20/24] x86: L2 CAT: implement set " Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 8:33 ` Dario Faggioli
2017-01-18 2:02 ` [PATCH v5 22/24] tools: L2 CAT: support show cbm " Yi Sun
` (2 subsequent siblings)
23 siblings, 1 reply; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements xl/xc changes to support get HW info
for L2 CAT.
'xl psr-hwinfo' is updated to show both L3 CAT and L2 CAT
info.
Example(on machine which only supports L2 CAT):
Cache Monitoring Technology (CMT):
Enabled : 0
Cache Allocation Technology (CAT): L3
libxl: error: libxl_psr.c:100:libxl__psr_cat_log_err_msg: CAT is not enabled on the socket: No such file or directory
Failed to get l3 cat info
Cache Allocation Technology (CAT): L2
Socket ID : 0
Maximum COS : 3
CBM length : 8
Default CBM : 0xff
Signed-off-by: He Chen <he.chen@linux.intel.com>
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
tools/libxc/include/xenctrl.h | 6 ++---
tools/libxc/xc_psr.c | 40 +++++++++++++++++++++++----------
tools/libxl/libxl.h | 9 ++++++++
tools/libxl/libxl_psr.c | 19 +++++++++++-----
tools/libxl/libxl_types.idl | 1 +
tools/libxl/xl_cmdimpl.c | 52 +++++++++++++++++++++++++++++++++----------
6 files changed, 95 insertions(+), 32 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 4ab0f57..7ea0c92 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2626,9 +2626,9 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_cat_type type, uint32_t target,
uint64_t *data);
-int xc_psr_cat_get_l3_info(xc_interface *xch, uint32_t socket,
- uint32_t *cos_max, uint32_t *cbm_len,
- bool *cdp_enabled);
+int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
+ uint32_t *cos_max, uint32_t *cbm_len,
+ bool *cdp_enabled);
int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 43b3286..6c61aa5 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -317,24 +317,40 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
return rc;
}
-int xc_psr_cat_get_l3_info(xc_interface *xch, uint32_t socket,
- uint32_t *cos_max, uint32_t *cbm_len,
- bool *cdp_enabled)
+int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
+ uint32_t *cos_max, uint32_t *cbm_len, bool *cdp_enabled)
{
- int rc;
+ int rc = -1;
DECLARE_SYSCTL;
sysctl.cmd = XEN_SYSCTL_psr_cat_op;
- sysctl.u.psr_cat_op.cmd = XEN_SYSCTL_PSR_CAT_get_l3_info;
sysctl.u.psr_cat_op.target = socket;
- rc = xc_sysctl(xch, &sysctl);
- if ( !rc )
- {
- *cos_max = sysctl.u.psr_cat_op.u.l3_info.cos_max;
- *cbm_len = sysctl.u.psr_cat_op.u.l3_info.cbm_len;
- *cdp_enabled = sysctl.u.psr_cat_op.u.l3_info.flags &
- XEN_SYSCTL_PSR_CAT_L3_CDP;
+ switch ( lvl ) {
+ case 2:
+ sysctl.u.psr_cat_op.cmd = XEN_SYSCTL_PSR_CAT_get_l2_info;
+ rc = xc_sysctl(xch, &sysctl);
+ if ( !rc )
+ {
+ *cos_max = sysctl.u.psr_cat_op.u.l2_info.cos_max;
+ *cbm_len = sysctl.u.psr_cat_op.u.l2_info.cbm_len;
+ *cdp_enabled = false;
+ }
+ break;
+ case 3:
+ sysctl.u.psr_cat_op.cmd = XEN_SYSCTL_PSR_CAT_get_l3_info;
+ rc = xc_sysctl(xch, &sysctl);
+ if ( !rc )
+ {
+ *cos_max = sysctl.u.psr_cat_op.u.l3_info.cos_max;
+ *cbm_len = sysctl.u.psr_cat_op.u.l3_info.cbm_len;
+ *cdp_enabled = sysctl.u.psr_cat_op.u.l3_info.flags &
+ XEN_SYSCTL_PSR_CAT_L3_CDP;
+ }
+ break;
+ default:
+ errno = EOPNOTSUPP;
+ break;
}
return rc;
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 3924464..c75a928 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -904,6 +904,13 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const libxl_mac *src);
* If this is defined, the Code and Data Prioritization feature is supported.
*/
#define LIBXL_HAVE_PSR_CDP 1
+
+/*
+ * LIBXL_HAVE_PSR_L2_CAT
+ *
+ * If this is defined, the L2 Cache Allocation Technology feature is supported.
+ */
+#define LIBXL_HAVE_PSR_L2_CAT 1
#endif
/*
@@ -2166,6 +2173,8 @@ int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
* On success, the function returns an array of elements in 'info',
* and the length in 'nr'.
*/
+int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
+ int *nr, unsigned int lvl);
int libxl_psr_cat_get_l3_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
int *nr);
void libxl_psr_cat_info_list_free(libxl_psr_cat_info *list, int nr);
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index ec5c79d..68c45da 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -352,8 +352,8 @@ int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
return rc;
}
-int libxl_psr_cat_get_l3_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
- int *nr)
+int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
+ int *nr, unsigned int lvl)
{
GC_INIT(ctx);
int rc;
@@ -380,9 +380,8 @@ int libxl_psr_cat_get_l3_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
libxl_for_each_set_bit(socketid, socketmap) {
ptr[i].id = socketid;
- if (xc_psr_cat_get_l3_info(ctx->xch, socketid, &ptr[i].cos_max,
- &ptr[i].cbm_len, &ptr[i].cdp_enabled)) {
- libxl__psr_cat_log_err_msg(gc, errno);
+ if (xc_psr_cat_get_info(ctx->xch, socketid, lvl, &ptr[i].cos_max,
+ &ptr[i].cbm_len, &ptr[i].cdp_enabled)) {
rc = ERROR_FAIL;
free(ptr);
goto out;
@@ -398,6 +397,16 @@ out:
return rc;
}
+int libxl_psr_cat_get_l3_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
+ int *nr)
+{
+ int rc;
+
+ rc = libxl_psr_cat_get_info(ctx, info, nr, 3);
+
+ return rc;
+}
+
void libxl_psr_cat_info_list_free(libxl_psr_cat_info *list, int nr)
{
int i;
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index a612d1f..5a401b8 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -899,6 +899,7 @@ libxl_psr_cbm_type = Enumeration("psr_cbm_type", [
(1, "L3_CBM"),
(2, "L3_CBM_CODE"),
(3, "L3_CBM_DATA"),
+ (4, "L2_CBM"),
])
libxl_psr_cat_info = Struct("psr_cat_info", [
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 7e8a8ae..42d6827 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -9331,21 +9331,19 @@ int main_psr_cmt_show(int argc, char **argv)
}
#endif
-#ifdef LIBXL_HAVE_PSR_CAT
-static int psr_cat_hwinfo(void)
+#if defined(LIBXL_HAVE_PSR_CAT) || defined(LIBXL_HAVE_PSR_L2_CAT)
+static int psr_l3_cat_hwinfo(void)
{
- int rc;
- int i, nr;
+ int rc, nr;
+ unsigned int i;
uint32_t l3_cache_size;
libxl_psr_cat_info *info;
- printf("Cache Allocation Technology (CAT):\n");
-
- rc = libxl_psr_cat_get_l3_info(ctx, &info, &nr);
- if (rc) {
- fprintf(stderr, "Failed to get cat info\n");
+ rc = libxl_psr_cat_get_info(ctx, &info, &nr, 3);
+ if (rc)
return rc;
- }
+
+ printf("Cache Allocation Technology (CAT): L3\n");
for (i = 0; i < nr; i++) {
rc = libxl_psr_cmt_get_l3_cache_size(ctx, info[i].id, &l3_cache_size);
@@ -9454,7 +9452,7 @@ static int psr_cat_show(uint32_t domid)
int rc;
libxl_psr_cat_info *info;
- rc = libxl_psr_cat_get_l3_info(ctx, &info, &nr);
+ rc = libxl_psr_cat_get_info(ctx, &info, &nr, 3);
if (rc) {
fprintf(stderr, "Failed to get cat info\n");
return rc;
@@ -9471,6 +9469,32 @@ out:
return rc;
}
+static int psr_l2_cat_hwinfo(void)
+{
+ int rc;
+ unsigned int i;
+ int nr;
+ libxl_psr_cat_info *info;
+
+ rc = libxl_psr_cat_get_info(ctx, &info, &nr, 2);
+ if (rc)
+ return rc;
+
+ printf("Cache Allocation Technology (CAT): L2\n");
+
+ for (i = 0; i < nr; i++) {
+ /* There is no CMT on L2 cache so far. */
+ printf("%-16s: %u\n", "Socket ID", info[i].id);
+ printf("%-16s: %u\n", "Maximum COS", info[i].cos_max);
+ printf("%-16s: %u\n", "CBM length", info[i].cbm_len);
+ printf("%-16s: %#llx\n", "Default CBM",
+ (1ull << info[i].cbm_len) - 1);
+ }
+
+ libxl_psr_cat_info_list_free(info, nr);
+ return rc;
+}
+
int main_psr_cat_cbm_set(int argc, char **argv)
{
uint32_t domid;
@@ -9588,7 +9612,11 @@ int main_psr_hwinfo(int argc, char **argv)
ret = psr_cmt_hwinfo();
if (!ret && (all || cat))
- ret = psr_cat_hwinfo();
+ ret = psr_l3_cat_hwinfo();
+
+ /* L2 CAT is independent of CMT and L3 CAT */
+ if (all || cat)
+ ret = psr_l2_cat_hwinfo();
return ret;
}
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* Re: [PATCH v5 21/24] tools: L2 CAT: support get HW info for L2 CAT.
2017-01-18 2:02 ` [PATCH v5 21/24] tools: L2 CAT: support get HW info for L2 CAT Yi Sun
@ 2017-01-18 8:33 ` Dario Faggioli
0 siblings, 0 replies; 30+ messages in thread
From: Dario Faggioli @ 2017-01-18 8:33 UTC (permalink / raw)
To: Yi Sun, xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, he.chen, ian.jackson,
mengxu, jbeulich, chao.p.peng
[-- Attachment #1.1: Type: text/plain, Size: 1442 bytes --]
On Wed, 2017-01-18 at 10:02 +0800, Yi Sun wrote:
> This patch implements xl/xc changes to support get HW info
> for L2 CAT.
>
> 'xl psr-hwinfo' is updated to show both L3 CAT and L2 CAT
> info.
>
> Example(on machine which only supports L2 CAT):
> Cache Monitoring Technology (CMT):
> Enabled : 0
> Cache Allocation Technology (CAT): L3
> libxl: error: libxl_psr.c:100:libxl__psr_cat_log_err_msg: CAT is not
> enabled on the socket: No such file or directory
> Failed to get l3 cat info
> Cache Allocation Technology (CAT): L2
> Socket ID : 0
> Maximum COS : 3
> CBM length : 8
> Default CBM : 0xff
>
I think we settled, in v4, for not having this awkward error message
showing itself in this way "only" because of (legitimate!) lack of
hardware support for some of the features?
https://lists.xenproject.org/archives/html/xen-devel/2017-01/msg00604.html
https://lists.xenproject.org/archives/html/xen-devel/2017-01/msg00792.html
https://lists.xenproject.org/archives/html/xen-devel/2017-01/msg00797.html
Nevertheless, I've had a look at the code, and it looks good to me.
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
[-- Attachment #2: Type: text/plain, Size: 127 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH v5 22/24] tools: L2 CAT: support show cbm for L2 CAT.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (20 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 21/24] tools: L2 CAT: support get HW info for L2 CAT Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 2:02 ` [PATCH v5 23/24] tools: L2 CAT: support set " Yi Sun
2017-01-18 2:02 ` [PATCH v5 24/24] docs: add L2 CAT description in docs Yi Sun
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements changes in xl/xc changes to support
showing CBM of L2 CAT.
The new level option is introduced to original CAT showing
command in order to show CBM for specified level CAT.
- 'xl psr-cat-show' is updated to show CBM of a domain
according to input cache level.
Examples:
root@:~$ xl psr-cat-show -l2 1
Socket ID : 0
Default CBM : 0xff
ID NAME CBM
1 ubuntu14 0x7f
Signed-off-by: He Chen <he.chen@linux.intel.com>
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
tools/libxc/include/xenctrl.h | 1 +
tools/libxc/xc_psr.c | 3 ++
tools/libxl/xl_cmdimpl.c | 81 ++++++++++++++++++++++++++++---------------
tools/libxl/xl_cmdtable.c | 3 +-
4 files changed, 59 insertions(+), 29 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 7ea0c92..a009625 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2602,6 +2602,7 @@ enum xc_psr_cat_type {
XC_PSR_CAT_L3_CBM = 1,
XC_PSR_CAT_L3_CBM_CODE = 2,
XC_PSR_CAT_L3_CBM_DATA = 3,
+ XC_PSR_CAT_L2_CBM = 4,
};
typedef enum xc_psr_cat_type xc_psr_cat_type;
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 6c61aa5..f0aed2d 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -299,6 +299,9 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
case XC_PSR_CAT_L3_CBM_DATA:
cmd = XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA;
break;
+ case XC_PSR_CAT_L2_CBM:
+ cmd = XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM;
+ break;
default:
errno = EINVAL;
return -1;
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 42d6827..cdcee5f 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -9379,7 +9379,7 @@ static void psr_cat_print_one_domain_cbm_type(uint32_t domid, uint32_t socketid,
}
static void psr_cat_print_one_domain_cbm(uint32_t domid, uint32_t socketid,
- bool cdp_enabled)
+ bool cdp_enabled, unsigned int lvl)
{
char *domain_name;
@@ -9387,27 +9387,38 @@ static void psr_cat_print_one_domain_cbm(uint32_t domid, uint32_t socketid,
printf("%5d%25s", domid, domain_name);
free(domain_name);
- if (!cdp_enabled) {
- psr_cat_print_one_domain_cbm_type(domid, socketid,
- LIBXL_PSR_CBM_TYPE_L3_CBM);
- } else {
- psr_cat_print_one_domain_cbm_type(domid, socketid,
- LIBXL_PSR_CBM_TYPE_L3_CBM_CODE);
+ switch (lvl) {
+ case 3:
+ if (!cdp_enabled) {
+ psr_cat_print_one_domain_cbm_type(domid, socketid,
+ LIBXL_PSR_CBM_TYPE_L3_CBM);
+ } else {
+ psr_cat_print_one_domain_cbm_type(domid, socketid,
+ LIBXL_PSR_CBM_TYPE_L3_CBM_CODE);
+ psr_cat_print_one_domain_cbm_type(domid, socketid,
+ LIBXL_PSR_CBM_TYPE_L3_CBM_DATA);
+ }
+ break;
+ case 2:
psr_cat_print_one_domain_cbm_type(domid, socketid,
- LIBXL_PSR_CBM_TYPE_L3_CBM_DATA);
+ LIBXL_PSR_CBM_TYPE_L2_CBM);
+ break;
+ default:
+ printf("Input lvl %d is wrong!", lvl);
+ break;
}
printf("\n");
}
static int psr_cat_print_domain_cbm(uint32_t domid, uint32_t socketid,
- bool cdp_enabled)
+ bool cdp_enabled, unsigned int lvl)
{
int i, nr_domains;
libxl_dominfo *list;
if (domid != INVALID_DOMID) {
- psr_cat_print_one_domain_cbm(domid, socketid, cdp_enabled);
+ psr_cat_print_one_domain_cbm(domid, socketid, cdp_enabled, lvl);
return 0;
}
@@ -9417,49 +9428,55 @@ static int psr_cat_print_domain_cbm(uint32_t domid, uint32_t socketid,
}
for (i = 0; i < nr_domains; i++)
- psr_cat_print_one_domain_cbm(list[i].domid, socketid, cdp_enabled);
+ psr_cat_print_one_domain_cbm(list[i].domid, socketid, cdp_enabled, lvl);
libxl_dominfo_list_free(list, nr_domains);
return 0;
}
-static int psr_cat_print_socket(uint32_t domid, libxl_psr_cat_info *info)
+static int psr_cat_print_socket(uint32_t domid, libxl_psr_cat_info *info,
+ unsigned int lvl)
{
int rc;
uint32_t l3_cache_size;
- rc = libxl_psr_cmt_get_l3_cache_size(ctx, info->id, &l3_cache_size);
- if (rc) {
- fprintf(stderr, "Failed to get l3 cache size for socket:%d\n",
- info->id);
- return -1;
+ printf("%-16s: %u\n", "Socket ID", info->id);
+
+ /* So far, CMT only supports L3 cache. */
+ if (lvl == 3)
+ {
+ rc = libxl_psr_cmt_get_l3_cache_size(ctx, info->id, &l3_cache_size);
+ if (rc) {
+ fprintf(stderr, "Failed to get l3 cache size for socket:%d\n",
+ info->id);
+ return -1;
+ }
+ printf("%-16s: %uKB\n", "L3 Cache", l3_cache_size);
}
- printf("%-16s: %u\n", "Socket ID", info->id);
- printf("%-16s: %uKB\n", "L3 Cache", l3_cache_size);
printf("%-16s: %#llx\n", "Default CBM", (1ull << info->cbm_len) - 1);
if (info->cdp_enabled)
printf("%5s%25s%16s%16s\n", "ID", "NAME", "CBM (code)", "CBM (data)");
else
printf("%5s%25s%16s\n", "ID", "NAME", "CBM");
- return psr_cat_print_domain_cbm(domid, info->id, info->cdp_enabled);
+ return psr_cat_print_domain_cbm(domid, info->id, info->cdp_enabled, lvl);
}
-static int psr_cat_show(uint32_t domid)
+static int psr_cat_show(uint32_t domid, unsigned int lvl)
{
int i, nr;
int rc;
libxl_psr_cat_info *info;
- rc = libxl_psr_cat_get_info(ctx, &info, &nr, 3);
+ rc = libxl_psr_cat_get_info(ctx, &info, &nr, lvl);
if (rc) {
- fprintf(stderr, "Failed to get cat info\n");
+ fprintf(stderr, "Failed to get %s cat info\n", (lvl == 3)?"L3":"L2");
return rc;
}
for (i = 0; i < nr; i++) {
- rc = psr_cat_print_socket(domid, info + i);
+ rc = psr_cat_print_socket(domid, info + i, lvl);
if (rc)
goto out;
}
@@ -9570,11 +9587,19 @@ int main_psr_cat_cbm_set(int argc, char **argv)
int main_psr_cat_show(int argc, char **argv)
{
- int opt;
+ int opt = 0;
uint32_t domid;
+ unsigned int lvl = 3;
- SWITCH_FOREACH_OPT(opt, "", NULL, "psr-cat-show", 0) {
- /* No options */
+ static struct option opts[] = {
+ {"level", 1, 0, 'l'},
+ COMMON_LONG_OPTS
+ };
+
+ SWITCH_FOREACH_OPT(opt, "l:", opts, "psr-cat-show", 0) {
+ case 'l':
+ lvl = atoi(optarg);
+ break;
}
if (optind >= argc)
@@ -9586,7 +9611,7 @@ int main_psr_cat_show(int argc, char **argv)
return 2;
}
- return psr_cat_show(domid);
+ return psr_cat_show(domid, lvl);
}
int main_psr_hwinfo(int argc, char **argv)
diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c
index 588d5d9..c5fbad4 100644
--- a/tools/libxl/xl_cmdtable.c
+++ b/tools/libxl/xl_cmdtable.c
@@ -556,7 +556,8 @@ struct cmd_spec cmd_table[] = {
{ "psr-cat-show",
&main_psr_cat_show, 0, 1,
"Show Cache Allocation Technology information",
- "<Domain>",
+ "[options] <Domain>",
+ "-l <level> Specify the cache level to process, otherwise L3 cache is processed\n"
},
#endif
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* [PATCH v5 23/24] tools: L2 CAT: support set cbm for L2 CAT.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (21 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 22/24] tools: L2 CAT: support show cbm " Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
2017-01-18 8:51 ` Dario Faggioli
2017-01-18 2:02 ` [PATCH v5 24/24] docs: add L2 CAT description in docs Yi Sun
23 siblings, 1 reply; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements the xl/xc changes to support set CBM
for L2 CAT.
The new level option is introduced to original CAT setting
command in order to set CBM for specified level CAT.
- 'xl psr-cat-cbm-set' is updated to set cache capacity
bitmasks(CBM) for a domain according to input cache level.
root@:~$ xl psr-cat-cbm-set -l2 1 0x7f
root@:~$ xl psr-cat-show -l2 1
Socket ID : 0
Default CBM : 0xff
ID NAME CBM
1 ubuntu14 0x7f
Signed-off-by: He Chen <he.chen@linux.intel.com>
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
tools/libxc/xc_psr.c | 3 +++
tools/libxl/xl_cmdimpl.c | 31 ++++++++++++++++++++-----------
tools/libxl/xl_cmdtable.c | 1 +
3 files changed, 24 insertions(+), 11 deletions(-)
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index f0aed2d..67556bb 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -266,6 +266,9 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
case XC_PSR_CAT_L3_CBM_DATA:
cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA;
break;
+ case XC_PSR_CAT_L2_CBM:
+ cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM;
+ break;
default:
errno = EINVAL;
return -1;
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index cdcee5f..a32438c 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -9523,19 +9523,21 @@ int main_psr_cat_cbm_set(int argc, char **argv)
char *value;
libxl_string_list socket_list;
unsigned long start, end;
- int i, j, len;
+ unsigned int i, j, len;
+ unsigned int lvl = 3;
static struct option opts[] = {
{"socket", 1, 0, 's'},
{"data", 0, 0, 'd'},
{"code", 0, 0, 'c'},
+ {"level", 1, 0, 'l'},
COMMON_LONG_OPTS
};
libxl_socket_bitmap_alloc(ctx, &target_map, 0);
libxl_bitmap_set_none(&target_map);
- SWITCH_FOREACH_OPT(opt, "s:cd", opts, "psr-cat-cbm-set", 2) {
+ SWITCH_FOREACH_OPT(opt, "s:l:cd", opts, "psr-cat-cbm-set", 2) {
case 's':
trim(isspace, optarg, &value);
split_string_into_string_list(value, ",", &socket_list);
@@ -9555,17 +9557,24 @@ int main_psr_cat_cbm_set(int argc, char **argv)
case 'c':
opt_code = 1;
break;
+ case 'l':
+ lvl = atoi(optarg);
+ break;
}
- if (opt_data && opt_code) {
- fprintf(stderr, "Cannot handle -c and -d at the same time\n");
- return -1;
- } else if (opt_data) {
- type = LIBXL_PSR_CBM_TYPE_L3_CBM_DATA;
- } else if (opt_code) {
- type = LIBXL_PSR_CBM_TYPE_L3_CBM_CODE;
- } else {
- type = LIBXL_PSR_CBM_TYPE_L3_CBM;
+ if (lvl == 2)
+ type = LIBXL_PSR_CBM_TYPE_L2_CBM;
+ else if (lvl == 3) {
+ if (opt_data && opt_code) {
+ fprintf(stderr, "Cannot handle -c and -d at the same time\n");
+ return ERROR_FAIL;
+ } else if (opt_data) {
+ type = LIBXL_PSR_CBM_TYPE_L3_CBM_DATA;
+ } else if (opt_code) {
+ type = LIBXL_PSR_CBM_TYPE_L3_CBM_CODE;
+ } else {
+ type = LIBXL_PSR_CBM_TYPE_L3_CBM;
+ }
}
if (libxl_bitmap_is_empty(&target_map))
diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c
index c5fbad4..32c3ee5 100644
--- a/tools/libxl/xl_cmdtable.c
+++ b/tools/libxl/xl_cmdtable.c
@@ -550,6 +550,7 @@ struct cmd_spec cmd_table[] = {
"Set cache capacity bitmasks(CBM) for a domain",
"[options] <Domain> <CBM>",
"-s <socket> Specify the socket to process, otherwise all sockets are processed\n"
+ "-l <level> Specify the cache level to process, otherwise L3 cache is processed\n"
"-c Set code CBM if CDP is supported\n"
"-d Set data CBM if CDP is supported\n"
},
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread* Re: [PATCH v5 23/24] tools: L2 CAT: support set cbm for L2 CAT.
2017-01-18 2:02 ` [PATCH v5 23/24] tools: L2 CAT: support set " Yi Sun
@ 2017-01-18 8:51 ` Dario Faggioli
0 siblings, 0 replies; 30+ messages in thread
From: Dario Faggioli @ 2017-01-18 8:51 UTC (permalink / raw)
To: Yi Sun, xen-devel
Cc: wei.liu2, he.chen, andrew.cooper3, konrad.wilk, ian.jackson,
mengxu, jbeulich, chao.p.peng
[-- Attachment #1.1: Type: text/plain, Size: 3545 bytes --]
On Wed, 2017-01-18 at 10:02 +0800, Yi Sun wrote:
> --- a/tools/libxl/xl_cmdimpl.c
> +++ b/tools/libxl/xl_cmdimpl.c
> @@ -9523,19 +9523,21 @@ int main_psr_cat_cbm_set(int argc, char
> **argv)
> char *value;
> libxl_string_list socket_list;
> unsigned long start, end;
> - int i, j, len;
> + unsigned int i, j, len;
>
This type change has nothing to do with this very patch, has it?
> + unsigned int lvl = 3;
>
Yeah, well, I understand that it's an improvement, and that, while
you're making changes in the area, you may want to do it... but at the
same time it is undeniably confusing to see the change here.
Personally, I don't like it to be part of this patch, but this is not
my call. :-)
> @@ -9555,17 +9557,24 @@ int main_psr_cat_cbm_set(int argc, char
> **argv)
> case 'c':
> opt_code = 1;
> break;
> + case 'l':
> + lvl = atoi(optarg);
> + break;
> }
>
> - if (opt_data && opt_code) {
> - fprintf(stderr, "Cannot handle -c and -d at the same
> time\n");
> - return -1;
> - } else if (opt_data) {
> - type = LIBXL_PSR_CBM_TYPE_L3_CBM_DATA;
> - } else if (opt_code) {
> - type = LIBXL_PSR_CBM_TYPE_L3_CBM_CODE;
> - } else {
> - type = LIBXL_PSR_CBM_TYPE_L3_CBM;
> + if (lvl == 2)
> + type = LIBXL_PSR_CBM_TYPE_L2_CBM;
> + else if (lvl == 3) {
> + if (opt_data && opt_code) {
> + fprintf(stderr, "Cannot handle -c and -d at the same
> time\n");
> + return ERROR_FAIL;
>
When the xl program is terminating --and in this case it is, as we are
returning from a main_foo_bar() function-- the exit code should be
either EXIT_SUCCESS (if everything went ok) or EXIT_FAILURE (if
something went wrong, like in this case).
I know this not yet happens everywhere in xl, but, please, let's cope
with that for new code.
Apart of these two things I've mentioned, the patch looks ok to me.
Regards,
Dario
> + } else if (opt_data) {
> + type = LIBXL_PSR_CBM_TYPE_L3_CBM_DATA;
> + } else if (opt_code) {
> + type = LIBXL_PSR_CBM_TYPE_L3_CBM_CODE;
> + } else {
> + type = LIBXL_PSR_CBM_TYPE_L3_CBM;
> + }
> }
>
> if (libxl_bitmap_is_empty(&target_map))
> diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c
> index c5fbad4..32c3ee5 100644
> --- a/tools/libxl/xl_cmdtable.c
> +++ b/tools/libxl/xl_cmdtable.c
> @@ -550,6 +550,7 @@ struct cmd_spec cmd_table[] = {
> "Set cache capacity bitmasks(CBM) for a domain",
> "[options] <Domain> <CBM>",
> "-s <socket> Specify the socket to process, otherwise
> all sockets are processed\n"
> + "-l <level> Specify the cache level to process,
> otherwise L3 cache is processed\n"
> "-c Set code CBM if CDP is supported\n"
> "-d Set data CBM if CDP is supported\n"
> },
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
[-- Attachment #2: Type: text/plain, Size: 127 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH v5 24/24] docs: add L2 CAT description in docs.
2017-01-18 2:02 [PATCH v5 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (22 preceding siblings ...)
2017-01-18 2:02 ` [PATCH v5 23/24] tools: L2 CAT: support set " Yi Sun
@ 2017-01-18 2:02 ` Yi Sun
23 siblings, 0 replies; 30+ messages in thread
From: Yi Sun @ 2017-01-18 2:02 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, konrad.wilk, andrew.cooper3, dario.faggioli, he.chen,
ian.jackson, Yi Sun, mengxu, jbeulich, chao.p.peng
This patch adds L2 CAT description in related documents.
Signed-off-by: He Chen <he.chen@linux.intel.com>
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
docs/man/xl.pod.1.in | 25 ++++++++++++++++++++++---
docs/misc/xl-psr.markdown | 10 ++++++++--
2 files changed, 30 insertions(+), 5 deletions(-)
diff --git a/docs/man/xl.pod.1.in b/docs/man/xl.pod.1.in
index 8e2aa5b..2c41ea7 100644
--- a/docs/man/xl.pod.1.in
+++ b/docs/man/xl.pod.1.in
@@ -1701,6 +1701,9 @@ occupancy monitoring share the same set of underlying monitoring service. Once
a domain is attached to the monitoring service, monitoring data can be shown
for any of these monitoring types.
+There is no cache monitoring and memory bandwidth monitoring on L2 cache so
+far.
+
=over 4
=item B<psr-cmt-attach> [I<domain-id>]
@@ -1725,7 +1728,7 @@ monitor types are:
Intel Broadwell and later server platforms offer capabilities to configure and
make use of the Cache Allocation Technology (CAT) mechanisms, which enable more
-cache resources (i.e. L3 cache) to be made available for high priority
+cache resources (i.e. L3/L2 cache) to be made available for high priority
applications. In the Xen implementation, CAT is used to control cache allocation
on VM basis. To enforce cache on a specific domain, just set capacity bitmasks
(CBM) for the domain.
@@ -1735,7 +1738,7 @@ Intel Broadwell and later server platforms also offer Code/Data Prioritization
applications. CDP is used on a per VM basis in the Xen implementation. To
specify code or data CBM for the domain, CDP feature must be enabled and CBM
type options need to be specified when setting CBM, and the type options (code
-and data) are mutually exclusive.
+and data) are mutually exclusive. There is no CDP support on L2 so far.
=over 4
@@ -1752,6 +1755,11 @@ B<OPTIONS>
Specify the socket to process, otherwise all sockets are processed.
+=item B<-l LEVEL>, B<--level=LEVEL>
+
+Specify the cache level to process, otherwise the last level cache (L3) is
+processed.
+
=item B<-c>, B<--code>
Set code CBM when CDP is enabled.
@@ -1762,10 +1770,21 @@ Set data CBM when CDP is enabled.
=back
-=item B<psr-cat-show> [I<domain-id>]
+=item B<psr-cat-show> [I<OPTIONS>] [I<domain-id>]
Show CAT settings for a certain domain or all domains.
+B<OPTIONS>
+
+=over 4
+
+=item B<-l LEVEL>, B<--level=LEVEL>
+
+Specify the cache level to process, otherwise the last level cache (L3) is
+processed.
+
+=back
+
=back
=head1 IGNORED FOR COMPATIBILITY WITH XM
diff --git a/docs/misc/xl-psr.markdown b/docs/misc/xl-psr.markdown
index c3c1e8e..bd2b6bd 100644
--- a/docs/misc/xl-psr.markdown
+++ b/docs/misc/xl-psr.markdown
@@ -70,7 +70,7 @@ total-mem-bandwidth instead of cache-occupancy). E.g. after a `xl psr-cmt-attach
Cache Allocation Technology (CAT) is a new feature available on Intel
Broadwell and later server platforms that allows an OS or Hypervisor/VMM to
-partition cache allocation (i.e. L3 cache) based on application priority or
+partition cache allocation (i.e. L3/L2 cache) based on application priority or
Class of Service (COS). Each COS is configured using capacity bitmasks (CBM)
which represent cache capacity and indicate the degree of overlap and
isolation between classes. System cache resource is divided into numbers of
@@ -119,13 +119,19 @@ A cbm is valid only when:
In a multi-socket system, the same cbm will be set on each socket by default.
Per socket cbm can be specified with the `--socket SOCKET` option.
+In different systems, the different cache level is supported, e.g. L3 cache or
+L2 cache. Per cache level cbm can be specified with the `--level LEVEL` option.
+
Setting the CBM may not be successful if insufficient COS is available. In
such case unused COS(es) may be freed by setting CBM of all related domains to
its default value(all-ones).
Per domain CBM settings can be shown by:
-`xl psr-cat-show`
+`xl psr-cat-show [OPTIONS] <domid>`
+
+In different systems, the different cache level is supported, e.g. L3 cache or
+L2 cache. Per cache level cbm can be specified with the `--level LEVEL` option.
## Code and Data Prioritization (CDP)
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 30+ messages in thread