* [PATCH v4 01/24] docs: create L2 Cache Allocation Technology (CAT) feature document
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
@ 2016-12-14 4:07 ` Yi Sun
2016-12-14 4:07 ` [PATCH v4 02/24] x86: refactor psr: remove L3 CAT/CDP codes Yi Sun
` (22 subsequent siblings)
23 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:07 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
This patch creates L2 CAT feature document in doc/features/.
It describes details of L2 CAT.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
docs/features/intel_psr_l2_cat.pandoc | 347 ++++++++++++++++++++++++++++++++++
1 file changed, 347 insertions(+)
create mode 100644 docs/features/intel_psr_l2_cat.pandoc
diff --git a/docs/features/intel_psr_l2_cat.pandoc b/docs/features/intel_psr_l2_cat.pandoc
new file mode 100644
index 0000000..77bd61f
--- /dev/null
+++ b/docs/features/intel_psr_l2_cat.pandoc
@@ -0,0 +1,347 @@
+% Intel L2 Cache Allocation Technology (L2 CAT) Feature
+% Revision 1.0
+
+\clearpage
+
+# Basics
+
+---------------- ----------------------------------------------------
+ Status: **Tech Preview**
+
+Architecture(s): Intel x86
+
+ Component(s): Hypervisor, toolstack
+
+ Hardware: Atom codename Goldmont and beyond CPUs
+---------------- ----------------------------------------------------
+
+# Overview
+
+L2 CAT allows an OS or Hypervisor/VMM to control allocation of a
+CPU's shared L2 cache based on application priority or Class of Service
+(COS). Each CLOS is configured using capacity bitmasks (CBM) which
+represent cache capacity and indicate the degree of overlap and
+isolation between classes. Once L2 CAT is configured, the processor
+allows access to portions of L2 cache according to the established
+class of service.
+
+## Terminology
+
+* CAT Cache Allocation Technology
+* CBM Capacity BitMasks
+* CDP Code and Data Prioritization
+* COS/CLOS Class of Service
+* MSRs Machine Specific Registers
+* PSR Intel Platform Shared Resource
+* VMM Virtual Machine Monitor
+
+# User details
+
+* Feature Enabling:
+
+ Add "psr=cat" to boot line parameter to enable all supported level CAT
+ features.
+
+* xl interfaces:
+
+ 1. `psr-cat-show [OPTIONS] domain-id`:
+
+ Show domain L2 or L3 CAT CBM.
+
+ New option `-l` is added.
+ `-l2`: Show cbm for L2 cache.
+ `-l3`: Show cbm for L3 cache.
+
+ If neither `-l2` nor `-l3` is given, show both of them. If any one
+ is not supported, will print error info.
+
+ 2. `psr-cat-cbm-set [OPTIONS] domain-id cbm`:
+
+ Set domain L2 or L3 CBM.
+
+ New option `-l` is added.
+ `-l2`: Specify cbm for L2 cache.
+ `-l3`: Specify cbm for L3 cache.
+
+ If neither `-l2` nor `-l3` is given, level 3 is the default option.
+
+ 3. `psr-hwinfo [OPTIONS]`:
+
+ Show L2 & L3 CAT HW informations on every socket.
+
+# Technical details
+
+L2 CAT is a member of Intel PSR features and part of CAT, it shares
+some base PSR infrastructure in Xen.
+
+## Hardware perspective
+
+L2 CAT defines a new range MSRs to assign different L2 cache access
+patterns which are known as CBMs, each CBM is associated with a COS.
+
+```
+
+ +----------------------------+----------------+
+ IA32_PQR_ASSOC | MSR (per socket) | Address |
+ +----+---+-------+ +----------------------------+----------------+
+ | |COS| | | IA32_L2_QOS_MASK_0 | 0xD10 |
+ +----+---+-------+ +----------------------------+----------------+
+ └-------------> | ... | ... |
+ +----------------------------+----------------+
+ | IA32_L2_QOS_MASK_n | 0xD10+n (n<64) |
+ +----------------------------+----------------+
+```
+
+When context switch happens, the COS of VCPU is written to per-thread
+MSR `IA32_PQR_ASSOC`, and then hardware enforces L2 cache allocation
+according to the corresponding CBM.
+
+## The relationship between L2 CAT and L3 CAT/CDP
+
+L2 CAT is independent of L3 CAT/CDP, which means L2 CAT would be enabled
+while L3 CAT/CDP is disabled, or L2 CAT and L3 CAT/CDP are all enabled.
+
+L2 CAT uses a new range CBMs from 0xD10 ~ 0xD10+n (n<64), following by
+the L3 CAT/CDP CBMs, and supports setting different L2 cache accessing
+patterns from L3 cache. Like L3 CAT/CDP requirement, the bits of CBM of
+L2 CAT must be continuous too.
+
+N.B. L2 CAT and L3 CAT/CDP share the same COS field in the same
+associate register `IA32_PQR_ASSOC`, which means one COS associates to a
+pair of L2 CBM and L3 CBM.
+
+Besides, the max COS of L2 CAT may be different from L3 CAT/CDP (or
+other PSR features in future). In some cases, a VM is permitted to have a
+COS that is beyond one (or more) of PSR features but within the others.
+For instance, let's assume the max COS of L2 CAT is 8 but the max COS of
+L3 CAT is 16, when a VM is assigned 9 as COS, the L3 CBM associated to
+COS 9 would be enforced, but for L2 CAT, the behavior is fully open (no
+limit) since COS 9 is beyond the max COS (8) of L2 CAT.
+
+## Design Overview
+
+* Core COS/CBM association
+
+ When enforcing L2 CAT, all cores of domains have the same default
+ COS (COS0) which associated to the fully open CBM (all ones bitmask)
+ to access all L2 cache. The default COS is used only in hypervisor
+ and is transparent to tool stack and user.
+
+ System administrator can change PSR allocation policy at runtime by
+ tool stack. Since L2 CAT share COS with L3 CAT/CDP, a COS corresponds
+ to a 2-tuple, like [L2 CBM, L3 CBM] with only-CAT enabled, when CDP
+ is enabled, one COS corresponds to a 3-tuple, like [L2 CBM,
+ L3 Code_CBM, L3 Data_CBM]. If neither L3 CAT nor L3 CDP is enabled,
+ things would be easier, one COS corresponds to one L2 CBM.
+
+* VCPU schedule
+
+ This part reuses L3 CAT COS infrastructure.
+
+* Multi-sockets
+
+ Different sockets may have different L2 CAT capability (e.g. max COS)
+ although it is consistent on the same socket. So the capability of
+ per-socket L2 CAT is specified.
+
+## Implementation Description
+
+* Hypervisor interfaces:
+
+ 1. Boot line parameter "psr=cat" now will enable L2 CAT and L3
+ CAT if hardware supported.
+
+ 2. SYSCTL:
+ - XEN_SYSCTL_PSR_CAT_get_l2_info: Get L2 CAT information.
+
+ 3. DOMCTL:
+ - XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM: Get L2 CBM for a domain.
+ - XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM: Set L2 CBM for a domain.
+
+* xl interfaces:
+
+ 1. psr-cat-show -l2 domain-id
+ Show L2 cbm for a domain.
+ => XEN_SYSCTL_PSR_CAT_get_l2_info /
+ XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM
+
+ 2. psr-mba-set -l2 domain-id cbm
+ Set L2 cbm for a domain.
+ => XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM
+
+ 3. psr-hwinfo
+ Show PSR HW information, including L2 CAT
+ => XEN_SYSCTL_PSR_CAT_get_l2_info
+
+* Key data structure:
+
+ 1. Feature HW info
+
+ ```
+ struct psr_cat_hw_info {
+ unsigned int cbm_len;
+ unsigned int cos_max;
+ };
+ ```
+
+ - Member `cbm_len`
+
+ `cbm_len` is one of the hardware info of CAT.
+
+ - Member `cos_max`
+
+ `cos_max` is one of the hardware info of CAT.
+
+ 2. Feature list node
+
+ ```
+ struct feat_node {
+ enum psr_feat_type feature;
+ struct feat_ops ops;
+ struct psr_cat_hw_info info;
+ uint64_t cos_reg_val[MAX_COS_REG_NUM];
+ struct list_head list;
+ };
+ ```
+
+ When a PSR enforcement feature is enabled, it will be added into a
+ feature list. The head of the list is created in psr initialization.
+
+ - Member `feature`
+
+ `feature` is an integer number, to indicate which feature the list entry
+ corresponds to.
+
+ - Member `ops`
+
+ `ops` maintains a callback function list of the feature. It will be introduced
+ in details later.
+
+ - Member `info`
+
+ `info` maintains the feature HW information which can be got through
+ psr_hwinfo command.
+
+ - Member `cos_reg_val`
+
+ `cos_reg_val` is an array to maintain the value set in all COS registers of
+ the feature.
+
+ 3. Per-socket PSR features information structure
+
+ ```
+ struct psr_cat_socket_info {
+ unsigned int feat_mask;
+ unsigned int nr_feat;
+ struct list_head feat_list;
+ unsigned int cos_ref[MAX_COS_REG_NUM];
+ spinlock_t ref_lock;
+ };
+ ```
+
+ We collect all PSR allocation features information of a socket in
+ this `struct psr_cat_socket_info`.
+
+ - Member `feat_mask`
+
+ `feat_mask` is a bitmap, to indicate which feature is enabled on
+ current socket. We define `feat_mask` bitmap as:
+
+ bit 0~1: L3 CAT status, [01] stands for L3 CAT only and [10]
+ stands for L3 CDP is enalbed.
+
+ bit 2: L2 CAT status.
+
+ - Member `cos_ref`
+
+ `cos_ref` is an array which maintains the reference of one COS.
+ If the COS is used by one domain, the reference will increase one.
+ If a domain releases the COS, the reference will decrease one. The
+ array is indexed by COS.
+
+ 4. Feature operation functions structure
+
+ ```
+ struct feat_ops {
+ void (*init_feature)(unsigned int eax, unsigned int ebx,
+ unsigned int ecx, unsigned int edx,
+ struct feat_node *feat,
+ struct psr_cat_socket_info *info);
+ int (*get_feat_info)(const struct feat_node *feat, enum cbm_type type,
+ uint32_t dat[], uint32_t array_len);
+ int (*get_val)(const struct feat_node *feat, unsigned int cos,
+ enum cbm_type type, uint64_t *val);
+ unsigned int (*get_max_cos_max)(const struct feat_node *feat);
+ unsigned int (*get_cos_num)(const struct feat_node *feat);
+ int (*get_old_val)(uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int old_cos);
+ int (*set_new_val)(uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int old_cos,
+ enum cbm_type type,
+ uint64_t m);
+ int (*compare_val)(const uint64_t val[], const struct feat_node *feat,
+ unsigned int cos, bool *found);
+ unsigned int (*get_cos_max_from_type)(const struct feat_node *feat,
+ enum cbm_type type);
+ unsigned int (*exceeds_cos_max)(const uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int cos);
+ int (*write_msr)(unsigned int cos, const uint64_t val[],
+ struct feat_node *feat);
+ };
+ ```
+
+ We abstract above callback functions to encapsulate the feature specific
+ behaviors into them. Then, it is easy to add a new feature. We just need:
+ 1) Implement such ops and callback functions for every feature.
+ 2) Register the ops into `struct feat_node`.
+ 3) Add the feature into feature list during CPU initialization.
+
+# Limitations
+
+L2 CAT can only work on HW which enables it(check by CPUID). So far, there
+is no HW enables both L2 CAT and L3 CAT/CDP. But SW implementation has considered
+such scenario to enable both L2 CAT and L3 CAT/CDP.
+
+# Testing
+
+L2 CAT uses same xl interfaces as L3 CAT/CDP. So, we can execute these
+commands to verify L2 CAT and L3 CAT/CDP on different HWs support them.
+
+For example:
+ root@:~$ xl psr-hwinfo --cat
+ Cache Allocation Technology (CAT): L2
+ Socket ID : 0
+ Maximum COS : 3
+ CBM length : 8
+ Default CBM : 0xff
+
+ root@:~$ xl psr-cat-cbm-set -l2 1 0x7f
+
+ root@:~$ xl psr-cat-show -l2 1
+ Socket ID : 0
+ Default CBM : 0xff
+ ID NAME CBM
+ 1 ubuntu14 0x7f
+
+# Areas for improvement
+
+N/A
+
+# Known issues
+
+N/A
+
+# References
+
+"INTEL® RESOURCE DIRECTOR TECHNOLOGY (INTEL® RDT) ALLOCATION FEATURES" [Intel® 64 and IA-32 Architectures Software Developer Manuals, vol3](http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html)
+
+# History
+
+------------------------------------------------------------------------
+Date Revision Version Notes
+---------- -------- -------- -------------------------------------------
+2016-08-12 1.0 Xen 4.9 Design document written
+---------- -------- -------- -------------------------------------------
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v4 02/24] x86: refactor psr: remove L3 CAT/CDP codes.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
2016-12-14 4:07 ` [PATCH v4 01/24] docs: create L2 Cache Allocation Technology (CAT) feature document Yi Sun
@ 2016-12-14 4:07 ` Yi Sun
2016-12-22 16:03 ` Jan Beulich
2016-12-14 4:07 ` [PATCH v4 03/24] x86: refactor psr: implement main data structures Yi Sun
` (21 subsequent siblings)
23 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:07 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
The current cache allocation codes in psr.c do not consider
future features addition and are not friendly to extend.
To make psr.c be more flexible to add new features and fulfill
the program principle, open for extension but closed for
modification, we have to refactor the psr.c:
1. Analyze cache allocation features and abstract general data
structures.
2. Analyze the init and all other functions flow, abstract all
steps that different features may have different implementations.
Make these steps be callback functions and register feature
specific fuctions. Then, the main processes will not be changed
when introducing a new feature.
Because the quantity of refactor codes is big and the logics are
changed a lot, it will cause reviewers confused if just change
old codes. Reviewers have to understand both old codes and new
implementations. After review iterations from V1 to V3, Jan has
proposed to remove all old cache allocation codes firstly, then
implement new codes step by step. This will help to make codes
be more easily reviewable.
There is no construction without destruction. So, this patch
removes all current L3 CAT/CDP codes in psr.c. The following
patches will introduce the new mechanism.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 470 +----------------------------------------------------
1 file changed, 5 insertions(+), 465 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 0b5073c..96a8589 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -23,24 +23,6 @@
#define PSR_CAT (1<<1)
#define PSR_CDP (1<<2)
-struct psr_cat_cbm {
- union {
- uint64_t cbm;
- struct {
- uint64_t code;
- uint64_t data;
- };
- };
- unsigned int ref;
-};
-
-struct psr_cat_socket_info {
- unsigned int cbm_len;
- unsigned int cos_max;
- struct psr_cat_cbm *cos_to_cbm;
- spinlock_t cbm_lock;
-};
-
struct psr_assoc {
uint64_t val;
uint64_t cos_mask;
@@ -48,26 +30,11 @@ struct psr_assoc {
struct psr_cmt *__read_mostly psr_cmt;
-static unsigned long *__read_mostly cat_socket_enable;
-static struct psr_cat_socket_info *__read_mostly cat_socket_info;
-static unsigned long *__read_mostly cdp_socket_enable;
-
static unsigned int opt_psr;
static unsigned int __initdata opt_rmid_max = 255;
-static unsigned int __read_mostly opt_cos_max = 255;
static uint64_t rmid_mask;
static DEFINE_PER_CPU(struct psr_assoc, psr_assoc);
-static struct psr_cat_cbm *temp_cos_to_cbm;
-
-static unsigned int get_socket_cpu(unsigned int socket)
-{
- if ( likely(socket < nr_sockets) )
- return cpumask_any(socket_cpumask[socket]);
-
- return nr_cpu_ids;
-}
-
static void __init parse_psr_bool(char *s, char *value, char *feature,
unsigned int mask)
{
@@ -107,9 +74,6 @@ static void __init parse_psr_param(char *s)
if ( val_str && !strcmp(s, "rmid_max") )
opt_rmid_max = simple_strtoul(val_str, NULL, 0);
- if ( val_str && !strcmp(s, "cos_max") )
- opt_cos_max = simple_strtoul(val_str, NULL, 0);
-
s = ss + 1;
} while ( ss );
}
@@ -213,16 +177,7 @@ static inline void psr_assoc_init(void)
{
struct psr_assoc *psra = &this_cpu(psr_assoc);
- if ( cat_socket_info )
- {
- unsigned int socket = cpu_to_socket(smp_processor_id());
-
- if ( test_bit(socket, cat_socket_enable) )
- psra->cos_mask = ((1ull << get_count_order(
- cat_socket_info[socket].cos_max)) - 1) << 32;
- }
-
- if ( psr_cmt_enabled() || psra->cos_mask )
+ if ( psr_cmt_enabled() )
rdmsrl(MSR_IA32_PSR_ASSOC, psra->val);
}
@@ -231,12 +186,6 @@ static inline void psr_assoc_rmid(uint64_t *reg, unsigned int rmid)
*reg = (*reg & ~rmid_mask) | (rmid & rmid_mask);
}
-static inline void psr_assoc_cos(uint64_t *reg, unsigned int cos,
- uint64_t cos_mask)
-{
- *reg = (*reg & ~cos_mask) | (((uint64_t)cos << 32) & cos_mask);
-}
-
void psr_ctxt_switch_to(struct domain *d)
{
struct psr_assoc *psra = &this_cpu(psr_assoc);
@@ -245,459 +194,54 @@ void psr_ctxt_switch_to(struct domain *d)
if ( psr_cmt_enabled() )
psr_assoc_rmid(®, d->arch.psr_rmid);
- if ( psra->cos_mask )
- psr_assoc_cos(®, d->arch.psr_cos_ids ?
- d->arch.psr_cos_ids[cpu_to_socket(smp_processor_id())] :
- 0, psra->cos_mask);
-
if ( reg != psra->val )
{
wrmsrl(MSR_IA32_PSR_ASSOC, reg);
psra->val = reg;
}
}
-static struct psr_cat_socket_info *get_cat_socket_info(unsigned int socket)
-{
- if ( !cat_socket_info )
- return ERR_PTR(-ENODEV);
-
- if ( socket >= nr_sockets )
- return ERR_PTR(-ENOTSOCK);
-
- if ( !test_bit(socket, cat_socket_enable) )
- return ERR_PTR(-ENOENT);
-
- return cat_socket_info + socket;
-}
-
-static inline bool_t cdp_is_enabled(unsigned int socket)
-{
- return cdp_socket_enable && test_bit(socket, cdp_socket_enable);
-}
int psr_get_cat_l3_info(unsigned int socket, uint32_t *cbm_len,
uint32_t *cos_max, uint32_t *flags)
{
- struct psr_cat_socket_info *info = get_cat_socket_info(socket);
-
- if ( IS_ERR(info) )
- return PTR_ERR(info);
-
- *cbm_len = info->cbm_len;
- *cos_max = info->cos_max;
-
- *flags = 0;
- if ( cdp_is_enabled(socket) )
- *flags |= XEN_SYSCTL_PSR_CAT_L3_CDP;
-
return 0;
}
int psr_get_l3_cbm(struct domain *d, unsigned int socket,
uint64_t *cbm, enum cbm_type type)
{
- struct psr_cat_socket_info *info = get_cat_socket_info(socket);
- bool_t cdp_enabled = cdp_is_enabled(socket);
-
- if ( IS_ERR(info) )
- return PTR_ERR(info);
-
- switch ( type )
- {
- case PSR_CBM_TYPE_L3:
- if ( cdp_enabled )
- return -EXDEV;
- *cbm = info->cos_to_cbm[d->arch.psr_cos_ids[socket]].cbm;
- break;
-
- case PSR_CBM_TYPE_L3_CODE:
- if ( !cdp_enabled )
- *cbm = info->cos_to_cbm[d->arch.psr_cos_ids[socket]].cbm;
- else
- *cbm = info->cos_to_cbm[d->arch.psr_cos_ids[socket]].code;
- break;
-
- case PSR_CBM_TYPE_L3_DATA:
- if ( !cdp_enabled )
- *cbm = info->cos_to_cbm[d->arch.psr_cos_ids[socket]].cbm;
- else
- *cbm = info->cos_to_cbm[d->arch.psr_cos_ids[socket]].data;
- break;
-
- default:
- ASSERT_UNREACHABLE();
- }
-
- return 0;
-}
-
-static bool_t psr_check_cbm(unsigned int cbm_len, uint64_t cbm)
-{
- unsigned int first_bit, zero_bit;
-
- /* Set bits should only in the range of [0, cbm_len). */
- if ( cbm & (~0ull << cbm_len) )
- return 0;
-
- /* At least one bit need to be set. */
- if ( cbm == 0 )
- return 0;
-
- first_bit = find_first_bit(&cbm, cbm_len);
- zero_bit = find_next_zero_bit(&cbm, cbm_len, first_bit);
-
- /* Set bits should be contiguous. */
- if ( zero_bit < cbm_len &&
- find_next_bit(&cbm, cbm_len, zero_bit) < cbm_len )
- return 0;
-
- return 1;
-}
-
-struct cos_cbm_info
-{
- unsigned int cos;
- bool_t cdp;
- uint64_t cbm_code;
- uint64_t cbm_data;
-};
-
-static void do_write_l3_cbm(void *data)
-{
- struct cos_cbm_info *info = data;
-
- if ( info->cdp )
- {
- wrmsrl(MSR_IA32_PSR_L3_MASK_CODE(info->cos), info->cbm_code);
- wrmsrl(MSR_IA32_PSR_L3_MASK_DATA(info->cos), info->cbm_data);
- }
- else
- wrmsrl(MSR_IA32_PSR_L3_MASK(info->cos), info->cbm_code);
-}
-
-static int write_l3_cbm(unsigned int socket, unsigned int cos,
- uint64_t cbm_code, uint64_t cbm_data, bool_t cdp)
-{
- struct cos_cbm_info info =
- {
- .cos = cos,
- .cbm_code = cbm_code,
- .cbm_data = cbm_data,
- .cdp = cdp,
- };
-
- if ( socket == cpu_to_socket(smp_processor_id()) )
- do_write_l3_cbm(&info);
- else
- {
- unsigned int cpu = get_socket_cpu(socket);
-
- if ( cpu >= nr_cpu_ids )
- return -ENOTSOCK;
- on_selected_cpus(cpumask_of(cpu), do_write_l3_cbm, &info, 1);
- }
-
return 0;
}
-static int find_cos(struct psr_cat_cbm *map, unsigned int cos_max,
- uint64_t cbm_code, uint64_t cbm_data, bool_t cdp_enabled)
-{
- unsigned int cos;
-
- for ( cos = 0; cos <= cos_max; cos++ )
- {
- if ( (map[cos].ref || cos == 0) &&
- ((!cdp_enabled && map[cos].cbm == cbm_code) ||
- (cdp_enabled && map[cos].code == cbm_code &&
- map[cos].data == cbm_data)) )
- return cos;
- }
-
- return -ENOENT;
-}
-
-static int pick_avail_cos(struct psr_cat_cbm *map, unsigned int cos_max,
- unsigned int old_cos)
-{
- unsigned int cos;
-
- /* If old cos is referred only by the domain, then use it. */
- if ( map[old_cos].ref == 1 && old_cos != 0 )
- return old_cos;
-
- /* Find an unused one other than cos0. */
- for ( cos = 1; cos <= cos_max; cos++ )
- if ( map[cos].ref == 0 )
- return cos;
-
- return -ENOENT;
-}
-
int psr_set_l3_cbm(struct domain *d, unsigned int socket,
uint64_t cbm, enum cbm_type type)
{
- unsigned int old_cos, cos_max;
- int cos, ret;
- uint64_t cbm_data, cbm_code;
- bool_t cdp_enabled = cdp_is_enabled(socket);
- struct psr_cat_cbm *map;
- struct psr_cat_socket_info *info = get_cat_socket_info(socket);
-
- if ( IS_ERR(info) )
- return PTR_ERR(info);
-
- if ( !psr_check_cbm(info->cbm_len, cbm) )
- return -EINVAL;
-
- if ( !cdp_enabled && (type == PSR_CBM_TYPE_L3_CODE ||
- type == PSR_CBM_TYPE_L3_DATA) )
- return -ENXIO;
-
- cos_max = info->cos_max;
- old_cos = d->arch.psr_cos_ids[socket];
- map = info->cos_to_cbm;
-
- switch ( type )
- {
- case PSR_CBM_TYPE_L3:
- cbm_code = cbm;
- cbm_data = cbm;
- break;
-
- case PSR_CBM_TYPE_L3_CODE:
- cbm_code = cbm;
- cbm_data = map[old_cos].data;
- break;
-
- case PSR_CBM_TYPE_L3_DATA:
- cbm_code = map[old_cos].code;
- cbm_data = cbm;
- break;
-
- default:
- ASSERT_UNREACHABLE();
- return -EINVAL;
- }
-
- spin_lock(&info->cbm_lock);
- cos = find_cos(map, cos_max, cbm_code, cbm_data, cdp_enabled);
- if ( cos >= 0 )
- {
- if ( cos == old_cos )
- {
- spin_unlock(&info->cbm_lock);
- return 0;
- }
- }
- else
- {
- cos = pick_avail_cos(map, cos_max, old_cos);
- if ( cos < 0 )
- {
- spin_unlock(&info->cbm_lock);
- return cos;
- }
-
- /* We try to avoid writing MSR. */
- if ( (cdp_enabled &&
- (map[cos].code != cbm_code || map[cos].data != cbm_data)) ||
- (!cdp_enabled && map[cos].cbm != cbm_code) )
- {
- ret = write_l3_cbm(socket, cos, cbm_code, cbm_data, cdp_enabled);
- if ( ret )
- {
- spin_unlock(&info->cbm_lock);
- return ret;
- }
- map[cos].code = cbm_code;
- map[cos].data = cbm_data;
- }
- }
-
- map[cos].ref++;
- map[old_cos].ref--;
- spin_unlock(&info->cbm_lock);
-
- d->arch.psr_cos_ids[socket] = cos;
-
return 0;
}
-/* Called with domain lock held, no extra lock needed for 'psr_cos_ids' */
-static void psr_free_cos(struct domain *d)
-{
- unsigned int socket;
- unsigned int cos;
- struct psr_cat_socket_info *info;
-
- if( !d->arch.psr_cos_ids )
- return;
-
- for_each_set_bit(socket, cat_socket_enable, nr_sockets)
- {
- if ( (cos = d->arch.psr_cos_ids[socket]) == 0 )
- continue;
-
- info = cat_socket_info + socket;
- spin_lock(&info->cbm_lock);
- info->cos_to_cbm[cos].ref--;
- spin_unlock(&info->cbm_lock);
- }
-
- xfree(d->arch.psr_cos_ids);
- d->arch.psr_cos_ids = NULL;
-}
-
int psr_domain_init(struct domain *d)
{
- if ( cat_socket_info )
- {
- d->arch.psr_cos_ids = xzalloc_array(unsigned int, nr_sockets);
- if ( !d->arch.psr_cos_ids )
- return -ENOMEM;
- }
-
return 0;
}
void psr_domain_free(struct domain *d)
{
psr_free_rmid(d);
- psr_free_cos(d);
-}
-
-static int cat_cpu_prepare(unsigned int cpu)
-{
- if ( !cat_socket_info )
- return 0;
-
- if ( temp_cos_to_cbm == NULL &&
- (temp_cos_to_cbm = xzalloc_array(struct psr_cat_cbm,
- opt_cos_max + 1UL)) == NULL )
- return -ENOMEM;
-
- return 0;
-}
-
-static void cat_cpu_init(void)
-{
- unsigned int eax, ebx, ecx, edx;
- struct psr_cat_socket_info *info;
- unsigned int socket;
- unsigned int cpu = smp_processor_id();
- uint64_t val;
- const struct cpuinfo_x86 *c = cpu_data + cpu;
-
- if ( !cpu_has(c, X86_FEATURE_PQE) || c->cpuid_level < PSR_CPUID_LEVEL_CAT )
- return;
-
- socket = cpu_to_socket(cpu);
- if ( test_bit(socket, cat_socket_enable) )
- return;
-
- cpuid_count(PSR_CPUID_LEVEL_CAT, 0, &eax, &ebx, &ecx, &edx);
- if ( ebx & PSR_RESOURCE_TYPE_L3 )
- {
- cpuid_count(PSR_CPUID_LEVEL_CAT, 1, &eax, &ebx, &ecx, &edx);
- info = cat_socket_info + socket;
- info->cbm_len = (eax & 0x1f) + 1;
- info->cos_max = min(opt_cos_max, edx & 0xffff);
-
- info->cos_to_cbm = temp_cos_to_cbm;
- temp_cos_to_cbm = NULL;
- /* cos=0 is reserved as default cbm(all ones). */
- info->cos_to_cbm[0].cbm = (1ull << info->cbm_len) - 1;
-
- spin_lock_init(&info->cbm_lock);
-
- set_bit(socket, cat_socket_enable);
-
- if ( (ecx & PSR_CAT_CDP_CAPABILITY) && (opt_psr & PSR_CDP) &&
- cdp_socket_enable && !test_bit(socket, cdp_socket_enable) )
- {
- info->cos_to_cbm[0].code = (1ull << info->cbm_len) - 1;
- info->cos_to_cbm[0].data = (1ull << info->cbm_len) - 1;
-
- /* We only write mask1 since mask0 is always all ones by default. */
- wrmsrl(MSR_IA32_PSR_L3_MASK(1), (1ull << info->cbm_len) - 1);
-
- rdmsrl(MSR_IA32_PSR_L3_QOS_CFG, val);
- wrmsrl(MSR_IA32_PSR_L3_QOS_CFG, val | (1 << PSR_L3_QOS_CDP_ENABLE_BIT));
-
- /* Cut half of cos_max when CDP is enabled. */
- info->cos_max >>= 1;
-
- set_bit(socket, cdp_socket_enable);
- }
- printk(XENLOG_INFO "CAT: enabled on socket %u, cos_max:%u, cbm_len:%u, CDP:%s\n",
- socket, info->cos_max, info->cbm_len,
- cdp_is_enabled(socket) ? "on" : "off");
- }
-}
-
-static void cat_cpu_fini(unsigned int cpu)
-{
- unsigned int socket = cpu_to_socket(cpu);
-
- if ( !socket_cpumask[socket] || cpumask_empty(socket_cpumask[socket]) )
- {
- struct psr_cat_socket_info *info = cat_socket_info + socket;
-
- if ( info->cos_to_cbm )
- {
- xfree(info->cos_to_cbm);
- info->cos_to_cbm = NULL;
- }
-
- if ( cdp_is_enabled(socket) )
- clear_bit(socket, cdp_socket_enable);
-
- clear_bit(socket, cat_socket_enable);
- }
-}
-
-static void __init psr_cat_free(void)
-{
- xfree(cat_socket_enable);
- cat_socket_enable = NULL;
- xfree(cat_socket_info);
- cat_socket_info = NULL;
-}
-
-static void __init init_psr_cat(void)
-{
- if ( opt_cos_max < 1 )
- {
- printk(XENLOG_INFO "CAT: disabled, cos_max is too small\n");
- return;
- }
-
- cat_socket_enable = xzalloc_array(unsigned long, BITS_TO_LONGS(nr_sockets));
- cat_socket_info = xzalloc_array(struct psr_cat_socket_info, nr_sockets);
- cdp_socket_enable = xzalloc_array(unsigned long, BITS_TO_LONGS(nr_sockets));
-
- if ( !cat_socket_enable || !cat_socket_info )
- psr_cat_free();
}
static int psr_cpu_prepare(unsigned int cpu)
{
- return cat_cpu_prepare(cpu);
+ return 0;
}
static void psr_cpu_init(void)
{
- if ( cat_socket_info )
- cat_cpu_init();
-
psr_assoc_init();
}
static void psr_cpu_fini(unsigned int cpu)
{
- if ( cat_socket_info )
- cat_cpu_fini(cpu);
+ return;
}
static int cpu_callback(
@@ -738,14 +282,10 @@ static int __init psr_presmp_init(void)
if ( (opt_psr & PSR_CMT) && opt_rmid_max )
init_psr_cmt(opt_rmid_max);
- if ( opt_psr & PSR_CAT )
- init_psr_cat();
-
- if ( psr_cpu_prepare(0) )
- psr_cat_free();
+ psr_cpu_prepare(0);
psr_cpu_init();
- if ( psr_cmt_enabled() || cat_socket_info )
+ if ( psr_cmt_enabled() )
register_cpu_notifier(&cpu_nfb);
return 0;
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* Re: [PATCH v4 02/24] x86: refactor psr: remove L3 CAT/CDP codes.
2016-12-14 4:07 ` [PATCH v4 02/24] x86: refactor psr: remove L3 CAT/CDP codes Yi Sun
@ 2016-12-22 16:03 ` Jan Beulich
2016-12-26 2:28 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2016-12-22 16:03 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> The current cache allocation codes in psr.c do not consider
> future features addition and are not friendly to extend.
>
> To make psr.c be more flexible to add new features and fulfill
> the program principle, open for extension but closed for
> modification, we have to refactor the psr.c:
> 1. Analyze cache allocation features and abstract general data
> structures.
> 2. Analyze the init and all other functions flow, abstract all
> steps that different features may have different implementations.
> Make these steps be callback functions and register feature
> specific fuctions. Then, the main processes will not be changed
> when introducing a new feature.
>
> Because the quantity of refactor codes is big and the logics are
> changed a lot, it will cause reviewers confused if just change
> old codes. Reviewers have to understand both old codes and new
> implementations. After review iterations from V1 to V3, Jan has
> proposed to remove all old cache allocation codes firstly, then
> implement new codes step by step. This will help to make codes
> be more easily reviewable.
>
> There is no construction without destruction. So, this patch
> removes all current L3 CAT/CDP codes in psr.c. The following
> patches will introduce the new mechanism.
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
with one question:
> @@ -738,14 +282,10 @@ static int __init psr_presmp_init(void)
> if ( (opt_psr & PSR_CMT) && opt_rmid_max )
> init_psr_cmt(opt_rmid_max);
>
> - if ( opt_psr & PSR_CAT )
> - init_psr_cat();
> -
> - if ( psr_cpu_prepare(0) )
> - psr_cat_free();
> + psr_cpu_prepare(0);
>
> psr_cpu_init();
> - if ( psr_cmt_enabled() || cat_socket_info )
> + if ( psr_cmt_enabled() )
> register_cpu_notifier(&cpu_nfb);
You retain this notifier - do you expect to need it going forward?
Till now it was used to allocate that strange temporary object,
which iirc we've discussed (during the earlier versions) would go
away.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* Re: [PATCH v4 02/24] x86: refactor psr: remove L3 CAT/CDP codes.
2016-12-22 16:03 ` Jan Beulich
@ 2016-12-26 2:28 ` Yi Sun
0 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2016-12-26 2:28 UTC (permalink / raw)
To: Jan Beulich
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
On 16-12-22 09:03:25, Jan Beulich wrote:
> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> > There is no construction without destruction. So, this patch
> > removes all current L3 CAT/CDP codes in psr.c. The following
> > patches will introduce the new mechanism.
> >
> > Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
>
> Acked-by: Jan Beulich <jbeulich@suse.com>
Thank you!
> with one question:
>
> > @@ -738,14 +282,10 @@ static int __init psr_presmp_init(void)
> > if ( (opt_psr & PSR_CMT) && opt_rmid_max )
> > init_psr_cmt(opt_rmid_max);
> >
> > - if ( opt_psr & PSR_CAT )
> > - init_psr_cat();
> > -
> > - if ( psr_cpu_prepare(0) )
> > - psr_cat_free();
> > + psr_cpu_prepare(0);
> >
> > psr_cpu_init();
> > - if ( psr_cmt_enabled() || cat_socket_info )
> > + if ( psr_cmt_enabled() )
> > register_cpu_notifier(&cpu_nfb);
>
> You retain this notifier - do you expect to need it going forward?
> Till now it was used to allocate that strange temporary object,
> which iirc we've discussed (during the earlier versions) would go
> away.
>
> Jan
Do you mean register_cpu_notifier() or psr_cpu_prepare()? For
register_cpu_notifier(), it is needed because we need handle CPU events
through it. For psr_cpu_prepare(), it is needed too. You can see it is
used to allocate memory for feature node in patch 04.
BRs,
Sun Yi
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v4 03/24] x86: refactor psr: implement main data structures.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
2016-12-14 4:07 ` [PATCH v4 01/24] docs: create L2 Cache Allocation Technology (CAT) feature document Yi Sun
2016-12-14 4:07 ` [PATCH v4 02/24] x86: refactor psr: remove L3 CAT/CDP codes Yi Sun
@ 2016-12-14 4:07 ` Yi Sun
2016-12-22 16:13 ` Jan Beulich
2016-12-14 4:07 ` [PATCH v4 04/24] x86: refactor psr: implement CPU init and free flow Yi Sun
` (20 subsequent siblings)
23 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:07 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
To construct an extendible framework, we need analyze PSR features
and abstract the common things and feature specific things. Then,
encapsulate them into different data structures.
By analyzing PSR features, we can get below map.
+------+------+------+
--------->| Dom0 | Dom1 | ... |
| +------+------+------+
| |
|Dom ID | cos_id of domain
| V
| +-----------------------------------------------------------------------------+
User --------->| PSR |
Socket ID | +--------------+---------------+---------------+ |
| | Socket0 Info | Socket 1 Info | ... | |
| +--------------+---------------+---------------+ |
| | cos_id=0 cos_id=1 ... |
| | +-----------------------+-----------------------+-----------+ |
| |->Ref : | ref 0 | ref 1 | ... | |
| | +-----------------------+-----------------------+-----------+ |
| | +-----------------------+-----------------------+-----------+ |
| |->L3 CAT: | cos 0 | cos 1 | ... | |
| | +-----------------------+-----------------------+-----------+ |
| | +-----------------------+-----------------------+-----------+ |
| |->L2 CAT: | cos 0 | cos 1 | ... | |
| | +-----------------------+-----------------------+-----------+ |
| | +-----------+-----------+-----------+-----------+-----------+ |
| |->CDP : | cos0 code | cos0 data | cos1 code | cos1 data | ... | |
| +-----------+-----------+-----------+-----------+-----------+ |
+-----------------------------------------------------------------------------+
So, we need define a socket info data structure, 'struct
psr_socket_info' to manage information per socket. It contains a
reference count array according to COS ID and a feature list to
manage all features enabled. Every entry of the reference count
array is used to record how many domains are using the COS registers
according to the COS ID. For example, L3 CAT and L2 CAT are enabled,
Dom1 uses COS_ID=1 registers of both features to save CBM values, like
below.
+-------+-------+-------+-----+
| COS 0 | COS 1 | COS 2 | ... |
+-------+-------+-------+-----+
L3 CAT | 0x7ff | 0x1ff | ... | ... |
+-------+-------+-------+-----+
L2 CAT | 0xff | 0xff | ... | ... |
+-------+-------+-------+-----+
If Dom2 has same CBM values, it can reuse these registers which COS_ID=1.
That means, both Dom1 and Dom2 use same COS registers(ID=1) to save same
L3/L2 values. So, the value ref[1] is 2 which means 2 domains are using
COS_ID 1.
To manage a feature, we need define a feature node data structure,
'struct feat_node', to manage feature's specific HW info, its callback
functions (all feature's specific behaviors are encapsulated into these
callback functions), and an array of all COS registers values of this
feature. CDP is a special feature which uses two entries of the array
for one COS ID. So, the number of CDP COS registers is the half of L3
CAT. E.g. L3 CAT has 16 COS registers, then CDP has 8 COS registers if
it is enabled.
For details, please refer spec and codes.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 111 +++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 111 insertions(+)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 96a8589..49a4598 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -17,12 +17,123 @@
#include <xen/cpu.h>
#include <xen/err.h>
#include <xen/sched.h>
+#include <xen/list.h>
#include <asm/psr.h>
+/*
+ * Terminology:
+ * - CAT Cache Allocation Technology
+ * - CBM Capacity BitMasks
+ * - CDP Code and Data Prioritization
+ * - COS/CLOS Class of Service. Also mean COS registers.
+ * - COS_MAX Max number of COS for the feature (minus 1)
+ * - MSRs Machine Specific Registers
+ * - PSR Intel Platform Shared Resource
+ */
+
#define PSR_CMT (1<<0)
#define PSR_CAT (1<<1)
#define PSR_CDP (1<<2)
+/*
+ * Per SDM 17.17.3.3 'Cache Allocation Technology: Cache Mask Configuration',
+ * the MSRs range from 0C90H through 0D0FH (inclusive), enables support for
+ * up to 128 L3 CAT Classes of Service. The COS_ID=[0,127].
+ *
+ * The MSRs range from 0D10H through 0D4FH (inclusive), enables support for
+ * up to 64 L2 CAT COS. The COS_ID=[0,63].
+ *
+ * So, the maximum COS register count of one feature is 128.
+ */
+#define MAX_COS_REG_CNT 128
+
+/*
+ * PSR features are managed per socket. Below structure defines the members
+ * used to manage these features.
+ * feat_mask - Mask used to record features enabled on socket. There may be
+ * some features enabled at same time.
+ * nr_feat - Record how many features enabled.
+ * feat_list - A list used to manage all features enabled.
+ * cos_ref - A reference count array to record how many domains are using the
+ * COS_ID.
+ * Every entry of cos_ref corresponds to one COS ID.
+ * ref_lock - A lock to protect cos_ref.
+ */
+struct psr_socket_info {
+ /*
+ * bit 0: L3 CAT
+ * bit 1: L3 CDP
+ * bit 2: L2 CAT
+ */
+ unsigned int feat_mask;
+ unsigned int nr_feat;
+ struct list_head feat_list;
+ unsigned int cos_ref[MAX_COS_REG_CNT];
+ spinlock_t ref_lock;
+};
+
+enum psr_feat_type {
+ PSR_SOCKET_L3_CAT = 0,
+ PSR_SOCKET_L3_CDP,
+ PSR_SOCKET_L2_CAT,
+};
+
+/* CAT/CDP HW info data structure. */
+struct psr_cat_hw_info {
+ unsigned int cbm_len;
+ unsigned int cos_max;
+};
+
+/* Encapsulate feature specific HW info here. */
+struct feat_hw_info {
+ union {
+ struct psr_cat_hw_info l3_cat_info;
+ };
+};
+
+struct feat_node;
+
+/*
+ * This structure defines feature operation callback functions. Every feature
+ * enabled MUST implement such callback functions and register them to ops.
+ *
+ * Feature specific behaviors will be encapsulated into these callback
+ * functions. Then, the main flows will not be changed when introducing a new
+ * feature.
+ */
+struct feat_ops {
+ /*
+ * init_feature is used in cpu initialization process to do feature
+ * specific initialization works.
+ */
+ void (*init_feature)(unsigned int eax, unsigned int ebx,
+ unsigned int ecx, unsigned int edx,
+ struct feat_node *feat,
+ struct psr_socket_info *info);
+};
+
+
+/*
+ * This structure represents one feature.
+ * feature - Which feature it is.
+ * feat_ops - Feature operation callback functions.
+ * info - Feature HW info.
+ * cos_reg_val - Array to store the values of COS registers. One entry stores
+ * the value of one COS register.
+ * For L3 CAT and L2 CAT, one entry corresponds to one COS_ID.
+ * For CDP, two entries correspond to one COS_ID. E.g.
+ * COS_ID=0 corresponds to cos_reg_val[0] (Data) and
+ * cos_reg_val[1] (Code).
+ * list - Feature list.
+ */
+struct feat_node {
+ enum psr_feat_type feature;
+ struct feat_ops ops;
+ struct feat_hw_info info;
+ uint64_t cos_reg_val[MAX_COS_REG_CNT];
+ struct list_head list;
+};
+
struct psr_assoc {
uint64_t val;
uint64_t cos_mask;
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* Re: [PATCH v4 03/24] x86: refactor psr: implement main data structures.
2016-12-14 4:07 ` [PATCH v4 03/24] x86: refactor psr: implement main data structures Yi Sun
@ 2016-12-22 16:13 ` Jan Beulich
2016-12-26 6:56 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2016-12-22 16:13 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> To construct an extendible framework, we need analyze PSR features
> and abstract the common things and feature specific things. Then,
> encapsulate them into different data structures.
>
> By analyzing PSR features, we can get below map.
> +------+------+------+
> --------->| Dom0 | Dom1 | ... |
> | +------+------+------+
> | |
> |Dom ID | cos_id of domain
> | V
> | +-----------------------------------------------------------------------------+
> User --------->| PSR
> |
> Socket ID | +--------------+---------------+---------------+ |
> | | Socket0 Info | Socket 1 Info | ... | |
> | +--------------+---------------+---------------+ |
> | | cos_id=0 cos_id=1 ... |
> | | +-----------------------+-----------------------+-----------+ |
> | |->Ref : | ref 0 | ref 1 | ... | |
> | | +-----------------------+-----------------------+-----------+ |
> | | +-----------------------+-----------------------+-----------+ |
> | |->L3 CAT: | cos 0 | cos 1 | ... | |
> | | +-----------------------+-----------------------+-----------+ |
> | | +-----------------------+-----------------------+-----------+ |
> | |->L2 CAT: | cos 0 | cos 1 | ... | |
> | | +-----------------------+-----------------------+-----------+ |
> | | +-----------+-----------+-----------+-----------+-----------+ |
> | |->CDP : | cos0 code | cos0 data | cos1 code | cos1 data | ... | |
> | +-----------+-----------+-----------+-----------+-----------+ |
> +-----------------------------------------------------------------------------+
>
> So, we need define a socket info data structure, 'struct
> psr_socket_info' to manage information per socket. It contains a
> reference count array according to COS ID and a feature list to
> manage all features enabled. Every entry of the reference count
> array is used to record how many domains are using the COS registers
> according to the COS ID. For example, L3 CAT and L2 CAT are enabled,
> Dom1 uses COS_ID=1 registers of both features to save CBM values, like
> below.
> +-------+-------+-------+-----+
> | COS 0 | COS 1 | COS 2 | ... |
> +-------+-------+-------+-----+
> L3 CAT | 0x7ff | 0x1ff | ... | ... |
> +-------+-------+-------+-----+
> L2 CAT | 0xff | 0xff | ... | ... |
> +-------+-------+-------+-----+
>
> If Dom2 has same CBM values, it can reuse these registers which COS_ID=1.
> That means, both Dom1 and Dom2 use same COS registers(ID=1) to save same
> L3/L2 values. So, the value ref[1] is 2 which means 2 domains are using
> COS_ID 1.
>
> To manage a feature, we need define a feature node data structure,
> 'struct feat_node', to manage feature's specific HW info, its callback
> functions (all feature's specific behaviors are encapsulated into these
> callback functions), and an array of all COS registers values of this
> feature. CDP is a special feature which uses two entries of the array
> for one COS ID. So, the number of CDP COS registers is the half of L3
> CAT. E.g. L3 CAT has 16 COS registers, then CDP has 8 COS registers if
> it is enabled.
The special nature of CDP will make some special handling necessary,
which may need reflection in data structure arrangement. Would you
mind spelling out here how CDP handling is intended to work?
> +/*
> + * Per SDM 17.17.3.3 'Cache Allocation Technology: Cache Mask Configuration',
I think I've asked before to omit section numbers, which tend to
change. Just the title will be enough.
> +struct feat_node;
> +
> +/*
> + * This structure defines feature operation callback functions. Every feature
> + * enabled MUST implement such callback functions and register them to ops.
> + *
> + * Feature specific behaviors will be encapsulated into these callback
> + * functions. Then, the main flows will not be changed when introducing a new
> + * feature.
> + */
> +struct feat_ops {
> + /*
> + * init_feature is used in cpu initialization process to do feature
> + * specific initialization works.
> + */
> + void (*init_feature)(unsigned int eax, unsigned int ebx,
> + unsigned int ecx, unsigned int edx,
> + struct feat_node *feat,
> + struct psr_socket_info *info);
> +};
What is the reason to have a separate structure for this, when you
don't store a pointer in struct feat_node? If this was inlined there,
the odd forward declaration of struct feat_node wouldn't be needed
either. (The same question may apply to struct feat_hw_info.)
> +
> +
Please avoid double blank lines.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 03/24] x86: refactor psr: implement main data structures.
2016-12-22 16:13 ` Jan Beulich
@ 2016-12-26 6:56 ` Yi Sun
2017-01-03 8:00 ` Jan Beulich
0 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2016-12-26 6:56 UTC (permalink / raw)
To: Jan Beulich
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
Hi,
I just realize it is Christmas holiday. Merry Christmas!
On 16-12-22 09:13:43, Jan Beulich wrote:
> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> > To construct an extendible framework, we need analyze PSR features
> > and abstract the common things and feature specific things. Then,
> > encapsulate them into different data structures.
> >
> > By analyzing PSR features, we can get below map.
> > +------+------+------+
> > --------->| Dom0 | Dom1 | ... |
> > | +------+------+------+
> > | |
> > |Dom ID | cos_id of domain
> > | V
> > | +-----------------------------------------------------------------------------+
> > User --------->| PSR
> > |
> > Socket ID | +--------------+---------------+---------------+ |
> > | | Socket0 Info | Socket 1 Info | ... | |
> > | +--------------+---------------+---------------+ |
> > | | cos_id=0 cos_id=1 ... |
> > | | +-----------------------+-----------------------+-----------+ |
> > | |->Ref : | ref 0 | ref 1 | ... | |
> > | | +-----------------------+-----------------------+-----------+ |
> > | | +-----------------------+-----------------------+-----------+ |
> > | |->L3 CAT: | cos 0 | cos 1 | ... | |
> > | | +-----------------------+-----------------------+-----------+ |
> > | | +-----------------------+-----------------------+-----------+ |
> > | |->L2 CAT: | cos 0 | cos 1 | ... | |
> > | | +-----------------------+-----------------------+-----------+ |
> > | | +-----------+-----------+-----------+-----------+-----------+ |
> > | |->CDP : | cos0 code | cos0 data | cos1 code | cos1 data | ... | |
> > | +-----------+-----------+-----------+-----------+-----------+ |
> > +-----------------------------------------------------------------------------+
> >
> > So, we need define a socket info data structure, 'struct
> > psr_socket_info' to manage information per socket. It contains a
> > reference count array according to COS ID and a feature list to
> > manage all features enabled. Every entry of the reference count
> > array is used to record how many domains are using the COS registers
> > according to the COS ID. For example, L3 CAT and L2 CAT are enabled,
> > Dom1 uses COS_ID=1 registers of both features to save CBM values, like
> > below.
> > +-------+-------+-------+-----+
> > | COS 0 | COS 1 | COS 2 | ... |
> > +-------+-------+-------+-----+
> > L3 CAT | 0x7ff | 0x1ff | ... | ... |
> > +-------+-------+-------+-----+
> > L2 CAT | 0xff | 0xff | ... | ... |
> > +-------+-------+-------+-----+
> >
> > If Dom2 has same CBM values, it can reuse these registers which COS_ID=1.
> > That means, both Dom1 and Dom2 use same COS registers(ID=1) to save same
> > L3/L2 values. So, the value ref[1] is 2 which means 2 domains are using
> > COS_ID 1.
> >
> > To manage a feature, we need define a feature node data structure,
> > 'struct feat_node', to manage feature's specific HW info, its callback
> > functions (all feature's specific behaviors are encapsulated into these
> > callback functions), and an array of all COS registers values of this
> > feature. CDP is a special feature which uses two entries of the array
> > for one COS ID. So, the number of CDP COS registers is the half of L3
> > CAT. E.g. L3 CAT has 16 COS registers, then CDP has 8 COS registers if
> > it is enabled.
>
> The special nature of CDP will make some special handling necessary,
> which may need reflection in data structure arrangement. Would you
> mind spelling out here how CDP handling is intended to work?
>
Yes, CDP has its special handling processes. The main difference has been
described above that CDP has half number of COS registers and uses two entries.
Because of these, I split CDP out from L3 CAT and implement CDP its own feature
callback functions from patch 13 to patch 16. You can check them for details.
Thanks!
> > +/*
> > + * Per SDM 17.17.3.3 'Cache Allocation Technology: Cache Mask Configuration',
>
> I think I've asked before to omit section numbers, which tend to
> change. Just the title will be enough.
>
Sorry for this, will mention title only.
> > +struct feat_node;
> > +
> > +/*
> > + * This structure defines feature operation callback functions. Every feature
> > + * enabled MUST implement such callback functions and register them to ops.
> > + *
> > + * Feature specific behaviors will be encapsulated into these callback
> > + * functions. Then, the main flows will not be changed when introducing a new
> > + * feature.
> > + */
> > +struct feat_ops {
> > + /*
> > + * init_feature is used in cpu initialization process to do feature
> > + * specific initialization works.
> > + */
> > + void (*init_feature)(unsigned int eax, unsigned int ebx,
> > + unsigned int ecx, unsigned int edx,
> > + struct feat_node *feat,
> > + struct psr_socket_info *info);
> > +};
>
> What is the reason to have a separate structure for this, when you
> don't store a pointer in struct feat_node? If this was inlined there,
> the odd forward declaration of struct feat_node wouldn't be needed
> either. (The same question may apply to struct feat_hw_info.)
>
I just want to make codes be clear. If you prefer inline declaration, I think I
should change it as below, right?
struct feat_node {
......
struct feat_ops {
......
} ops;
struct feat_hw_info {
......
} info;
......
};
> > +
> > +
>
> Please avoid double blank lines.
>
Sorry, will remove it. Thanks!
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 03/24] x86: refactor psr: implement main data structures.
2016-12-26 6:56 ` Yi Sun
@ 2017-01-03 8:00 ` Jan Beulich
2017-01-03 8:49 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2017-01-03 8:00 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 26.12.16 at 07:56, <yi.y.sun@linux.intel.com> wrote:
> On 16-12-22 09:13:43, Jan Beulich wrote:
>> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
>> > So, we need define a socket info data structure, 'struct
>> > psr_socket_info' to manage information per socket. It contains a
>> > reference count array according to COS ID and a feature list to
>> > manage all features enabled. Every entry of the reference count
>> > array is used to record how many domains are using the COS registers
>> > according to the COS ID. For example, L3 CAT and L2 CAT are enabled,
>> > Dom1 uses COS_ID=1 registers of both features to save CBM values, like
>> > below.
>> > +-------+-------+-------+-----+
>> > | COS 0 | COS 1 | COS 2 | ... |
>> > +-------+-------+-------+-----+
>> > L3 CAT | 0x7ff | 0x1ff | ... | ... |
>> > +-------+-------+-------+-----+
>> > L2 CAT | 0xff | 0xff | ... | ... |
>> > +-------+-------+-------+-----+
>> >
>> > If Dom2 has same CBM values, it can reuse these registers which COS_ID=1.
>> > That means, both Dom1 and Dom2 use same COS registers(ID=1) to save same
>> > L3/L2 values. So, the value ref[1] is 2 which means 2 domains are using
>> > COS_ID 1.
>> >
>> > To manage a feature, we need define a feature node data structure,
>> > 'struct feat_node', to manage feature's specific HW info, its callback
>> > functions (all feature's specific behaviors are encapsulated into these
>> > callback functions), and an array of all COS registers values of this
>> > feature. CDP is a special feature which uses two entries of the array
>> > for one COS ID. So, the number of CDP COS registers is the half of L3
>> > CAT. E.g. L3 CAT has 16 COS registers, then CDP has 8 COS registers if
>> > it is enabled.
>>
>> The special nature of CDP will make some special handling necessary,
>> which may need reflection in data structure arrangement. Would you
>> mind spelling out here how CDP handling is intended to work?
>>
> Yes, CDP has its special handling processes. The main difference has been
> described above that CDP has half number of COS registers and uses two entries.
> Because of these, I split CDP out from L3 CAT and implement CDP its own feature
> callback functions from patch 13 to patch 16. You can check them for details.
Well, my point was to at least sketch out your (data structure
related) intentions in the comment here, to help reviewers (and
future readers) understand how the data structures fit that
special case.
>> > +struct feat_node;
>> > +
>> > +/*
>> > + * This structure defines feature operation callback functions. Every feature
>> > + * enabled MUST implement such callback functions and register them to ops.
>> > + *
>> > + * Feature specific behaviors will be encapsulated into these callback
>> > + * functions. Then, the main flows will not be changed when introducing a new
>> > + * feature.
>> > + */
>> > +struct feat_ops {
>> > + /*
>> > + * init_feature is used in cpu initialization process to do feature
>> > + * specific initialization works.
>> > + */
>> > + void (*init_feature)(unsigned int eax, unsigned int ebx,
>> > + unsigned int ecx, unsigned int edx,
>> > + struct feat_node *feat,
>> > + struct psr_socket_info *info);
>> > +};
>>
>> What is the reason to have a separate structure for this, when you
>> don't store a pointer in struct feat_node? If this was inlined there,
>> the odd forward declaration of struct feat_node wouldn't be needed
>> either. (The same question may apply to struct feat_hw_info.)
>>
> I just want to make codes be clear. If you prefer inline declaration, I
> think I
> should change it as below, right?
>
> struct feat_node {
> ......
> struct feat_ops {
> ......
> } ops;
> struct feat_hw_info {
> ......
> } info;
> ......
> };
Well, not exactly: The struct <tag> { ... } <name>; wrappers
are unnecessary then too. With them kept there indeed would be
no big difference between both variants.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 03/24] x86: refactor psr: implement main data structures.
2017-01-03 8:00 ` Jan Beulich
@ 2017-01-03 8:49 ` Yi Sun
2017-01-03 9:12 ` Jan Beulich
0 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2017-01-03 8:49 UTC (permalink / raw)
To: Jan Beulich
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
On 17-01-03 01:00:37, Jan Beulich wrote:
> >>> On 26.12.16 at 07:56, <yi.y.sun@linux.intel.com> wrote:
> > On 16-12-22 09:13:43, Jan Beulich wrote:
> >> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> >> > If Dom2 has same CBM values, it can reuse these registers which COS_ID=1.
> >> > That means, both Dom1 and Dom2 use same COS registers(ID=1) to save same
> >> > L3/L2 values. So, the value ref[1] is 2 which means 2 domains are using
> >> > COS_ID 1.
> >> >
> >> > To manage a feature, we need define a feature node data structure,
> >> > 'struct feat_node', to manage feature's specific HW info, its callback
> >> > functions (all feature's specific behaviors are encapsulated into these
> >> > callback functions), and an array of all COS registers values of this
> >> > feature. CDP is a special feature which uses two entries of the array
> >> > for one COS ID. So, the number of CDP COS registers is the half of L3
> >> > CAT. E.g. L3 CAT has 16 COS registers, then CDP has 8 COS registers if
> >> > it is enabled.
> >>
> >> The special nature of CDP will make some special handling necessary,
> >> which may need reflection in data structure arrangement. Would you
> >> mind spelling out here how CDP handling is intended to work?
> >>
> > Yes, CDP has its special handling processes. The main difference has been
> > described above that CDP has half number of COS registers and uses two entries.
> > Because of these, I split CDP out from L3 CAT and implement CDP its own feature
> > callback functions from patch 13 to patch 16. You can check them for details.
>
> Well, my point was to at least sketch out your (data structure
> related) intentions in the comment here, to help reviewers (and
> future readers) understand how the data structures fit that
> special case.
>
Thanks! Will add more comments here to explain CDP special points.
> >> > +struct feat_node;
> >> > +
> >> > +/*
> >> > + * This structure defines feature operation callback functions. Every feature
> >> > + * enabled MUST implement such callback functions and register them to ops.
> >> > + *
> >> > + * Feature specific behaviors will be encapsulated into these callback
> >> > + * functions. Then, the main flows will not be changed when introducing a new
> >> > + * feature.
> >> > + */
> >> > +struct feat_ops {
> >> > + /*
> >> > + * init_feature is used in cpu initialization process to do feature
> >> > + * specific initialization works.
> >> > + */
> >> > + void (*init_feature)(unsigned int eax, unsigned int ebx,
> >> > + unsigned int ecx, unsigned int edx,
> >> > + struct feat_node *feat,
> >> > + struct psr_socket_info *info);
> >> > +};
> >>
> >> What is the reason to have a separate structure for this, when you
> >> don't store a pointer in struct feat_node? If this was inlined there,
> >> the odd forward declaration of struct feat_node wouldn't be needed
> >> either. (The same question may apply to struct feat_hw_info.)
> >>
> > I just want to make codes be clear. If you prefer inline declaration, I
> > think I
> > should change it as below, right?
> >
> > struct feat_node {
> > ......
> > struct feat_ops {
> > ......
> > } ops;
> > struct feat_hw_info {
> > ......
> > } info;
> > ......
> > };
>
> Well, not exactly: The struct <tag> { ... } <name>; wrappers
> are unnecessary then too. With them kept there indeed would be
> no big difference between both variants.
>
To facilitate the callback functions assignment for a feature, I defined
feature specific callback function ops like below.
struct feat_ops l3_cat_ops = {
.init_feature = l3_cat_init_feature,
.get_max_cos_max = l3_cat_get_max_cos_max,
......
};
And directly assign it to global feature node in initialization process like
below.
static void cpu_init_work(void)
{
......
feat_tmp = feat_l3_cat;
feat_l3_cat = NULL;
feat_tmp->ops = l3_cat_ops;
......
}
I think this can make codes be clear. How do you think? Is this way acceptable?
Thanks!
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 03/24] x86: refactor psr: implement main data structures.
2017-01-03 8:49 ` Yi Sun
@ 2017-01-03 9:12 ` Jan Beulich
2017-01-03 10:28 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2017-01-03 9:12 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 03.01.17 at 09:49, <yi.y.sun@linux.intel.com> wrote:
> On 17-01-03 01:00:37, Jan Beulich wrote:
>> >>> On 26.12.16 at 07:56, <yi.y.sun@linux.intel.com> wrote:
>> > On 16-12-22 09:13:43, Jan Beulich wrote:
>> >> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
>> >> > +struct feat_node;
>> >> > +
>> >> > +/*
>> >> > + * This structure defines feature operation callback functions. Every feature
>> >> > + * enabled MUST implement such callback functions and register them to ops.
>> >> > + *
>> >> > + * Feature specific behaviors will be encapsulated into these callback
>> >> > + * functions. Then, the main flows will not be changed when introducing a new
>> >> > + * feature.
>> >> > + */
>> >> > +struct feat_ops {
>> >> > + /*
>> >> > + * init_feature is used in cpu initialization process to do feature
>> >> > + * specific initialization works.
>> >> > + */
>> >> > + void (*init_feature)(unsigned int eax, unsigned int ebx,
>> >> > + unsigned int ecx, unsigned int edx,
>> >> > + struct feat_node *feat,
>> >> > + struct psr_socket_info *info);
>> >> > +};
>> >>
>> >> What is the reason to have a separate structure for this, when you
>> >> don't store a pointer in struct feat_node? If this was inlined there,
>> >> the odd forward declaration of struct feat_node wouldn't be needed
>> >> either. (The same question may apply to struct feat_hw_info.)
>> >>
>> > I just want to make codes be clear. If you prefer inline declaration, I
>> > think I
>> > should change it as below, right?
>> >
>> > struct feat_node {
>> > ......
>> > struct feat_ops {
>> > ......
>> > } ops;
>> > struct feat_hw_info {
>> > ......
>> > } info;
>> > ......
>> > };
>>
>> Well, not exactly: The struct <tag> { ... } <name>; wrappers
>> are unnecessary then too. With them kept there indeed would be
>> no big difference between both variants.
>>
> To facilitate the callback functions assignment for a feature, I defined
> feature specific callback function ops like below.
>
> struct feat_ops l3_cat_ops = {
> .init_feature = l3_cat_init_feature,
> .get_max_cos_max = l3_cat_get_max_cos_max,
> ......
> };
>
> And directly assign it to global feature node in initialization process like
> below.
>
> static void cpu_init_work(void)
> {
> ......
> feat_tmp = feat_l3_cat;
> feat_l3_cat = NULL;
> feat_tmp->ops = l3_cat_ops;
> ......
> }
>
> I think this can make codes be clear. How do you think? Is this way
> acceptable?
Yes.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 03/24] x86: refactor psr: implement main data structures.
2017-01-03 9:12 ` Jan Beulich
@ 2017-01-03 10:28 ` Yi Sun
2017-01-03 11:23 ` Jan Beulich
0 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2017-01-03 10:28 UTC (permalink / raw)
To: Jan Beulich
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
On 17-01-03 02:12:53, Jan Beulich wrote:
> >>> On 03.01.17 at 09:49, <yi.y.sun@linux.intel.com> wrote:
> > On 17-01-03 01:00:37, Jan Beulich wrote:
> >> >>> On 26.12.16 at 07:56, <yi.y.sun@linux.intel.com> wrote:
> >> > On 16-12-22 09:13:43, Jan Beulich wrote:
> >> >> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> >> >> > +struct feat_node;
> >> >> > +
> >> >> > +/*
> >> >> > + * This structure defines feature operation callback functions. Every feature
> >> >> > + * enabled MUST implement such callback functions and register them to ops.
> >> >> > + *
> >> >> > + * Feature specific behaviors will be encapsulated into these callback
> >> >> > + * functions. Then, the main flows will not be changed when introducing a new
> >> >> > + * feature.
> >> >> > + */
> >> >> > +struct feat_ops {
> >> >> > + /*
> >> >> > + * init_feature is used in cpu initialization process to do feature
> >> >> > + * specific initialization works.
> >> >> > + */
> >> >> > + void (*init_feature)(unsigned int eax, unsigned int ebx,
> >> >> > + unsigned int ecx, unsigned int edx,
> >> >> > + struct feat_node *feat,
> >> >> > + struct psr_socket_info *info);
> >> >> > +};
> >> >>
> >> >> What is the reason to have a separate structure for this, when you
> >> >> don't store a pointer in struct feat_node? If this was inlined there,
> >> >> the odd forward declaration of struct feat_node wouldn't be needed
> >> >> either. (The same question may apply to struct feat_hw_info.)
> >> >>
> >> > I just want to make codes be clear. If you prefer inline declaration, I
> >> > think I
> >> > should change it as below, right?
> >> >
> >> > struct feat_node {
> >> > ......
> >> > struct feat_ops {
> >> > ......
> >> > } ops;
> >> > struct feat_hw_info {
> >> > ......
> >> > } info;
> >> > ......
> >> > };
> >>
> >> Well, not exactly: The struct <tag> { ... } <name>; wrappers
> >> are unnecessary then too. With them kept there indeed would be
> >> no big difference between both variants.
> >>
> > To facilitate the callback functions assignment for a feature, I defined
> > feature specific callback function ops like below.
> >
> > struct feat_ops l3_cat_ops = {
> > .init_feature = l3_cat_init_feature,
> > .get_max_cos_max = l3_cat_get_max_cos_max,
> > ......
> > };
> >
> > And directly assign it to global feature node in initialization process like
> > below.
> >
> > static void cpu_init_work(void)
> > {
> > ......
> > feat_tmp = feat_l3_cat;
> > feat_l3_cat = NULL;
> > feat_tmp->ops = l3_cat_ops;
> > ......
> > }
> >
> > I think this can make codes be clear. How do you think? Is this way
> > acceptable?
>
> Yes.
>
Thanks a lot! Do you have any comment to other patches?
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 03/24] x86: refactor psr: implement main data structures.
2017-01-03 10:28 ` Yi Sun
@ 2017-01-03 11:23 ` Jan Beulich
0 siblings, 0 replies; 79+ messages in thread
From: Jan Beulich @ 2017-01-03 11:23 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 03.01.17 at 11:28, <yi.y.sun@linux.intel.com> wrote:
> Do you have any comment to other patches?
Well, I'll get to them as time permits.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v4 04/24] x86: refactor psr: implement CPU init and free flow.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (2 preceding siblings ...)
2016-12-14 4:07 ` [PATCH v4 03/24] x86: refactor psr: implement main data structures Yi Sun
@ 2016-12-14 4:07 ` Yi Sun
2017-01-10 11:45 ` Jan Beulich
2016-12-14 4:07 ` [PATCH v4 05/24] x86: refactor psr: implement Domain init/free and schedule flows Yi Sun
` (19 subsequent siblings)
23 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:07 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements the CPU init and free flow including L3 CAT
initialization and feature list free.
Per this patch, you can see how callback functions work and how the
feature list is handled.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 179 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 176 insertions(+), 3 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 49a4598..fa9bc32 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -35,6 +35,9 @@
#define PSR_CAT (1<<1)
#define PSR_CDP (1<<2)
+#define CAT_CBM_LEN_MASK 0x1f
+#define CAT_COS_MAX_MASK 0xffff
+
/*
* Per SDM 17.17.3.3 'Cache Allocation Technology: Cache Mask Configuration',
* the MSRs range from 0C90H through 0D0FH (inclusive), enables support for
@@ -141,11 +144,79 @@ struct psr_assoc {
struct psr_cmt *__read_mostly psr_cmt;
+static struct psr_socket_info *__read_mostly socket_info;
+
static unsigned int opt_psr;
static unsigned int __initdata opt_rmid_max = 255;
+static unsigned int __read_mostly opt_cos_max = MAX_COS_REG_CNT;
static uint64_t rmid_mask;
static DEFINE_PER_CPU(struct psr_assoc, psr_assoc);
+/* Declare feature list entry. */
+static struct feat_node *feat_l3_cat;
+
+/* Common functions. */
+static void free_feature(struct psr_socket_info *info)
+{
+ struct feat_node *feat_tmp;
+
+ if ( !info )
+ return;
+
+ list_for_each_entry(feat_tmp, &info->feat_list, list)
+ {
+ clear_bit(feat_tmp->feature, &info->feat_mask);
+ list_del(&feat_tmp->list);
+ xfree(feat_tmp);
+ }
+
+ /* Free feature which are not added into feat_list. */
+ if ( feat_l3_cat )
+ {
+ xfree(feat_l3_cat);
+ feat_l3_cat = NULL;
+ }
+}
+
+/* L3 CAT callback functions implementation. */
+static void l3_cat_init_feature(unsigned int eax, unsigned int ebx,
+ unsigned int ecx, unsigned int edx,
+ struct feat_node *feat,
+ struct psr_socket_info *info)
+{
+ struct psr_cat_hw_info l3_cat;
+ unsigned int socket;
+
+ /* No valid value so do not enable feature. */
+ if ( !eax || !edx )
+ return;
+
+ l3_cat.cbm_len = (eax & CAT_CBM_LEN_MASK) + 1;
+ l3_cat.cos_max = min(opt_cos_max, edx & CAT_COS_MAX_MASK);
+
+ /* cos=0 is reserved as default cbm(all ones). */
+ feat->cos_reg_val[0] = (1ull << l3_cat.cbm_len) - 1;
+
+ feat->feature = PSR_SOCKET_L3_CAT;
+ __set_bit(PSR_SOCKET_L3_CAT, &info->feat_mask);
+
+ feat->info.l3_cat_info = l3_cat;
+
+ info->nr_feat++;
+
+ /* Add this feature into list. */
+ list_add_tail(&feat->list, &info->feat_list);
+
+ socket = cpu_to_socket(smp_processor_id());
+ printk(XENLOG_INFO "L3 CAT: enabled on socket %u, cos_max:%u, cbm_len:%u\n",
+ socket, feat->info.l3_cat_info.cos_max,
+ feat->info.l3_cat_info.cbm_len);
+}
+
+struct feat_ops l3_cat_ops = {
+ .init_feature = l3_cat_init_feature,
+};
+
static void __init parse_psr_bool(char *s, char *value, char *feature,
unsigned int mask)
{
@@ -185,6 +256,9 @@ static void __init parse_psr_param(char *s)
if ( val_str && !strcmp(s, "rmid_max") )
opt_rmid_max = simple_strtoul(val_str, NULL, 0);
+ if ( val_str && !strcmp(s, "cos_max") )
+ opt_cos_max = simple_strtoul(val_str, NULL, 0);
+
s = ss + 1;
} while ( ss );
}
@@ -340,18 +414,113 @@ void psr_domain_free(struct domain *d)
psr_free_rmid(d);
}
-static int psr_cpu_prepare(unsigned int cpu)
+static int cpu_prepare_work(unsigned int cpu)
{
+ if ( !socket_info )
+ return 0;
+
+ /* Malloc memory for the global feature head here. */
+ if ( feat_l3_cat == NULL &&
+ (feat_l3_cat = xzalloc(struct feat_node)) == NULL )
+ return -ENOMEM;
+
return 0;
}
+static void cpu_init_work(void)
+{
+ unsigned int eax, ebx, ecx, edx;
+ struct psr_socket_info *info;
+ unsigned int socket;
+ unsigned int cpu = smp_processor_id();
+ const struct cpuinfo_x86 *c = cpu_data + cpu;
+ struct feat_node *feat_tmp;
+
+ if ( !cpu_has(c, X86_FEATURE_PQE) || c->cpuid_level < PSR_CPUID_LEVEL_CAT )
+ return;
+
+ socket = cpu_to_socket(cpu);
+ info = socket_info + socket;
+ if ( info->feat_mask )
+ return;
+
+ spin_lock_init(&info->ref_lock);
+
+ cpuid_count(PSR_CPUID_LEVEL_CAT, 0, &eax, &ebx, &ecx, &edx);
+ if ( ebx & PSR_RESOURCE_TYPE_L3 )
+ {
+ cpuid_count(PSR_CPUID_LEVEL_CAT, 1, &eax, &ebx, &ecx, &edx);
+
+ feat_tmp = feat_l3_cat;
+ feat_l3_cat = NULL;
+ feat_tmp->ops = l3_cat_ops;
+
+ feat_tmp->ops.init_feature(eax, ebx, ecx, edx, feat_tmp, info);
+ }
+}
+
+static void cpu_fini_work(unsigned int cpu)
+{
+ unsigned int socket = cpu_to_socket(cpu);
+
+ if ( !socket_cpumask[socket] || cpumask_empty(socket_cpumask[socket]) )
+ {
+ struct psr_socket_info *info = socket_info + socket;
+
+ free_feature(info);
+ }
+}
+
+static void __init init_psr(void)
+{
+ unsigned int i;
+
+ if ( opt_cos_max < 1 )
+ {
+ printk(XENLOG_INFO "CAT: disabled, cos_max is too small\n");
+ return;
+ }
+
+ socket_info = xzalloc_array(struct psr_socket_info, nr_sockets);
+
+ if ( !socket_info )
+ {
+ printk(XENLOG_INFO "Fail to alloc socket_info!\n");
+ return;
+ }
+
+ for ( i = 0; i < nr_sockets; i++ )
+ INIT_LIST_HEAD(&socket_info[i].feat_list);
+}
+
+static void __init psr_free(void)
+{
+ unsigned int i;
+
+ for ( i = 0; i < nr_sockets; i++ )
+ free_feature(&socket_info[i]);
+
+ xfree(socket_info);
+ socket_info = NULL;
+}
+
+static int psr_cpu_prepare(unsigned int cpu)
+{
+ return cpu_prepare_work(cpu);
+}
+
static void psr_cpu_init(void)
{
+ if ( socket_info )
+ cpu_init_work();
+
psr_assoc_init();
}
static void psr_cpu_fini(unsigned int cpu)
{
+ if ( socket_info )
+ cpu_fini_work(cpu);
return;
}
@@ -393,10 +562,14 @@ static int __init psr_presmp_init(void)
if ( (opt_psr & PSR_CMT) && opt_rmid_max )
init_psr_cmt(opt_rmid_max);
- psr_cpu_prepare(0);
+ if ( opt_psr & PSR_CAT )
+ init_psr();
+
+ if ( psr_cpu_prepare(0) )
+ psr_free();
psr_cpu_init();
- if ( psr_cmt_enabled() )
+ if ( psr_cmt_enabled() || socket_info )
register_cpu_notifier(&cpu_nfb);
return 0;
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* Re: [PATCH v4 04/24] x86: refactor psr: implement CPU init and free flow.
2016-12-14 4:07 ` [PATCH v4 04/24] x86: refactor psr: implement CPU init and free flow Yi Sun
@ 2017-01-10 11:45 ` Jan Beulich
2017-01-11 3:14 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2017-01-10 11:45 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> @@ -141,11 +144,79 @@ struct psr_assoc {
>
> struct psr_cmt *__read_mostly psr_cmt;
>
> +static struct psr_socket_info *__read_mostly socket_info;
> +
> static unsigned int opt_psr;
> static unsigned int __initdata opt_rmid_max = 255;
> +static unsigned int __read_mostly opt_cos_max = MAX_COS_REG_CNT;
> static uint64_t rmid_mask;
> static DEFINE_PER_CPU(struct psr_assoc, psr_assoc);
>
> +/* Declare feature list entry. */
> +static struct feat_node *feat_l3_cat;
Hmm, if you indeed (again) need such a helper object, then please
make the comment actually say so. As it is, the comment is mostly
meaningless.
> +/* Common functions. */
> +static void free_feature(struct psr_socket_info *info)
> +{
> + struct feat_node *feat_tmp;
> +
> + if ( !info )
> + return;
> +
> + list_for_each_entry(feat_tmp, &info->feat_list, list)
> + {
> + clear_bit(feat_tmp->feature, &info->feat_mask);
> + list_del(&feat_tmp->list);
> + xfree(feat_tmp);
> + }
This requires list_for_each_entry_safe() to be used, to avoid a
use-after-free issue (or alternatively a while(!list_empty()) loop).
> + /* Free feature which are not added into feat_list. */
> + if ( feat_l3_cat )
> + {
> + xfree(feat_l3_cat);
> + feat_l3_cat = NULL;
> + }
Why don't you leave this around, avoiding the need for an
allocation the next time a CPU comes online? Also note that xfree()
deals fine with a NULL input, so conditionals like this are pointless.
> +/* L3 CAT callback functions implementation. */
> +static void l3_cat_init_feature(unsigned int eax, unsigned int ebx,
> + unsigned int ecx, unsigned int edx,
This is rather unfortunate naming: How does the reader of this code
know what these values represent, without having to first go look in
the caller?
> + struct feat_node *feat,
> + struct psr_socket_info *info)
> +{
> + struct psr_cat_hw_info l3_cat;
> + unsigned int socket;
> +
> + /* No valid value so do not enable feature. */
> + if ( !eax || !edx )
> + return;
> +
> + l3_cat.cbm_len = (eax & CAT_CBM_LEN_MASK) + 1;
> + l3_cat.cos_max = min(opt_cos_max, edx & CAT_COS_MAX_MASK);
> +
> + /* cos=0 is reserved as default cbm(all ones). */
> + feat->cos_reg_val[0] = (1ull << l3_cat.cbm_len) - 1;
Considering how cbm_len gets calculated a few lines up, I can't see
how this can end up being all ones (as the comment says). At most
this can be 0xffffffff (as a 64-bit value) afaics.
> + feat->feature = PSR_SOCKET_L3_CAT;
> + __set_bit(PSR_SOCKET_L3_CAT, &info->feat_mask);
> +
> + feat->info.l3_cat_info = l3_cat;
> +
> + info->nr_feat++;
> +
> + /* Add this feature into list. */
> + list_add_tail(&feat->list, &info->feat_list);
> +
> + socket = cpu_to_socket(smp_processor_id());
> + printk(XENLOG_INFO "L3 CAT: enabled on socket %u, cos_max:%u, cbm_len:%u\n",
> + socket, feat->info.l3_cat_info.cos_max,
> + feat->info.l3_cat_info.cbm_len);
I don't think we want such printed for every socket, at least not by
default. Please, if you want to keep it, make it dependent upon e.g.
opt_cpu_info.
> +}
> +
> +struct feat_ops l3_cat_ops = {
static const
> @@ -340,18 +414,113 @@ void psr_domain_free(struct domain *d)
> psr_free_rmid(d);
> }
>
> -static int psr_cpu_prepare(unsigned int cpu)
> +static int cpu_prepare_work(unsigned int cpu)
> {
> + if ( !socket_info )
> + return 0;
> +
> + /* Malloc memory for the global feature head here. */
> + if ( feat_l3_cat == NULL &&
> + (feat_l3_cat = xzalloc(struct feat_node)) == NULL )
> + return -ENOMEM;
> +
> return 0;
> }
>
> +static void cpu_init_work(void)
> +{
> + unsigned int eax, ebx, ecx, edx;
> + struct psr_socket_info *info;
> + unsigned int socket;
> + unsigned int cpu = smp_processor_id();
> + const struct cpuinfo_x86 *c = cpu_data + cpu;
Please use current_cpu_data instead of open coding it.
> + struct feat_node *feat_tmp;
Looking at the uses, I don't think this is temporary in any way - why
not just "feat"?
> + if ( !cpu_has(c, X86_FEATURE_PQE) || c->cpuid_level < PSR_CPUID_LEVEL_CAT )
> + return;
Instead of such a double check, please consider clearing the PQE
feature bit when the maximum CPUID level is too low (which
shouldn't happen anyway).
> + socket = cpu_to_socket(cpu);
> + info = socket_info + socket;
> + if ( info->feat_mask )
> + return;
> +
> + spin_lock_init(&info->ref_lock);
> +
> + cpuid_count(PSR_CPUID_LEVEL_CAT, 0, &eax, &ebx, &ecx, &edx);
> + if ( ebx & PSR_RESOURCE_TYPE_L3 )
> + {
> + cpuid_count(PSR_CPUID_LEVEL_CAT, 1, &eax, &ebx, &ecx, &edx);
> +
> + feat_tmp = feat_l3_cat;
> + feat_l3_cat = NULL;
> + feat_tmp->ops = l3_cat_ops;
> +
> + feat_tmp->ops.init_feature(eax, ebx, ecx, edx, feat_tmp, info);
What's the point of the indirect call here, when you know the
function is l3_cat_init_feature()?
> +static void cpu_fini_work(unsigned int cpu)
> +{
> + unsigned int socket = cpu_to_socket(cpu);
> +
> + if ( !socket_cpumask[socket] || cpumask_empty(socket_cpumask[socket]) )
> + {
> + struct psr_socket_info *info = socket_info + socket;
> +
> + free_feature(info);
Pointless local variable "info", unless later patches add further uses.
> +static void __init init_psr(void)
> +{
> + unsigned int i;
> +
> + if ( opt_cos_max < 1 )
> + {
> + printk(XENLOG_INFO "CAT: disabled, cos_max is too small\n");
> + return;
> + }
> +
> + socket_info = xzalloc_array(struct psr_socket_info, nr_sockets);
> +
> + if ( !socket_info )
> + {
> + printk(XENLOG_INFO "Fail to alloc socket_info!\n");
> + return;
> + }
> +
> + for ( i = 0; i < nr_sockets; i++ )
> + INIT_LIST_HEAD(&socket_info[i].feat_list);
Please decide for one central place where to do such initialization:
This and spin_lock_init() really should live together (and I think
better there, not here).
> +static int psr_cpu_prepare(unsigned int cpu)
> +{
> + return cpu_prepare_work(cpu);
> +}
What is this wrapper good for?
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 04/24] x86: refactor psr: implement CPU init and free flow.
2017-01-10 11:45 ` Jan Beulich
@ 2017-01-11 3:14 ` Yi Sun
2017-01-11 13:48 ` Jan Beulich
0 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2017-01-11 3:14 UTC (permalink / raw)
To: Jan Beulich
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
On 17-01-10 04:45:05, Jan Beulich wrote:
> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> > @@ -141,11 +144,79 @@ struct psr_assoc {
> >
> > struct psr_cmt *__read_mostly psr_cmt;
> >
> > +static struct psr_socket_info *__read_mostly socket_info;
> > +
> > static unsigned int opt_psr;
> > static unsigned int __initdata opt_rmid_max = 255;
> > +static unsigned int __read_mostly opt_cos_max = MAX_COS_REG_CNT;
> > static uint64_t rmid_mask;
> > static DEFINE_PER_CPU(struct psr_assoc, psr_assoc);
> >
> > +/* Declare feature list entry. */
> > +static struct feat_node *feat_l3_cat;
>
> Hmm, if you indeed (again) need such a helper object, then please
> make the comment actually say so. As it is, the comment is mostly
> meaningless.
>
Thanks! Will add more comments to explain it.
> > +/* Common functions. */
> > +static void free_feature(struct psr_socket_info *info)
> > +{
> > + struct feat_node *feat_tmp;
> > +
> > + if ( !info )
> > + return;
> > +
> > + list_for_each_entry(feat_tmp, &info->feat_list, list)
> > + {
> > + clear_bit(feat_tmp->feature, &info->feat_mask);
> > + list_del(&feat_tmp->list);
> > + xfree(feat_tmp);
> > + }
>
> This requires list_for_each_entry_safe() to be used, to avoid a
> use-after-free issue (or alternatively a while(!list_empty()) loop).
>
Thanks for the suggestion!
> > + /* Free feature which are not added into feat_list. */
> > + if ( feat_l3_cat )
> > + {
> > + xfree(feat_l3_cat);
> > + feat_l3_cat = NULL;
> > + }
>
> Why don't you leave this around, avoiding the need for an
> allocation the next time a CPU comes online? Also note that xfree()
> deals fine with a NULL input, so conditionals like this are pointless.
>
Thanks! Will keep them.
> > +/* L3 CAT callback functions implementation. */
> > +static void l3_cat_init_feature(unsigned int eax, unsigned int ebx,
> > + unsigned int ecx, unsigned int edx,
>
> This is rather unfortunate naming: How does the reader of this code
> know what these values represent, without having to first go look in
> the caller?
>
Do you mean the 'eax'-'edx'? How about 'eax_register'?
> > + struct feat_node *feat,
> > + struct psr_socket_info *info)
> > +{
> > + struct psr_cat_hw_info l3_cat;
> > + unsigned int socket;
> > +
> > + /* No valid value so do not enable feature. */
> > + if ( !eax || !edx )
> > + return;
> > +
> > + l3_cat.cbm_len = (eax & CAT_CBM_LEN_MASK) + 1;
> > + l3_cat.cos_max = min(opt_cos_max, edx & CAT_COS_MAX_MASK);
> > +
> > + /* cos=0 is reserved as default cbm(all ones). */
> > + feat->cos_reg_val[0] = (1ull << l3_cat.cbm_len) - 1;
>
> Considering how cbm_len gets calculated a few lines up, I can't see
> how this can end up being all ones (as the comment says). At most
> this can be 0xffffffff (as a 64-bit value) afaics.
>
Sorry for the confusion. All one means all bits within cbm_len are 1. E.g.
the cbm_len is 11. Then, value of cos_reg_val[0] is '(1 << 11) - 1', equals
0x7ff.
Will correct the comment.
> > + feat->feature = PSR_SOCKET_L3_CAT;
> > + __set_bit(PSR_SOCKET_L3_CAT, &info->feat_mask);
> > +
> > + feat->info.l3_cat_info = l3_cat;
> > +
> > + info->nr_feat++;
> > +
> > + /* Add this feature into list. */
> > + list_add_tail(&feat->list, &info->feat_list);
> > +
> > + socket = cpu_to_socket(smp_processor_id());
> > + printk(XENLOG_INFO "L3 CAT: enabled on socket %u, cos_max:%u, cbm_len:%u\n",
> > + socket, feat->info.l3_cat_info.cos_max,
> > + feat->info.l3_cat_info.cbm_len);
>
> I don't think we want such printed for every socket, at least not by
> default. Please, if you want to keep it, make it dependent upon e.g.
> opt_cpu_info.
>
Thanks! Will limit the print by opt_cpu_info.
> > +}
> > +
> > +struct feat_ops l3_cat_ops = {
>
> static const
>
Ok, thanks!
> > @@ -340,18 +414,113 @@ void psr_domain_free(struct domain *d)
> > psr_free_rmid(d);
> > }
> >
> > -static int psr_cpu_prepare(unsigned int cpu)
> > +static int cpu_prepare_work(unsigned int cpu)
> > {
> > + if ( !socket_info )
> > + return 0;
> > +
> > + /* Malloc memory for the global feature head here. */
> > + if ( feat_l3_cat == NULL &&
> > + (feat_l3_cat = xzalloc(struct feat_node)) == NULL )
> > + return -ENOMEM;
> > +
> > return 0;
> > }
> >
> > +static void cpu_init_work(void)
> > +{
> > + unsigned int eax, ebx, ecx, edx;
> > + struct psr_socket_info *info;
> > + unsigned int socket;
> > + unsigned int cpu = smp_processor_id();
> > + const struct cpuinfo_x86 *c = cpu_data + cpu;
>
> Please use current_cpu_data instead of open coding it.
>
Thanks for the suggestion!
> > + struct feat_node *feat_tmp;
>
> Looking at the uses, I don't think this is temporary in any way - why
> not just "feat"?
>
No problem, thanks!
> > + if ( !cpu_has(c, X86_FEATURE_PQE) || c->cpuid_level < PSR_CPUID_LEVEL_CAT )
> > + return;
>
> Instead of such a double check, please consider clearing the PQE
> feature bit when the maximum CPUID level is too low (which
> shouldn't happen anyway).
>
Is this the responsibility of psr.c? X86_FEATURE_PQE bit is set by HW. Even the
bit is set but CPUID level is low, I think SW would be better to keep it but
not clear it. Because it indicates the HW capability. How do you think? Thanks!
> > + socket = cpu_to_socket(cpu);
> > + info = socket_info + socket;
> > + if ( info->feat_mask )
> > + return;
> > +
> > + spin_lock_init(&info->ref_lock);
> > +
> > + cpuid_count(PSR_CPUID_LEVEL_CAT, 0, &eax, &ebx, &ecx, &edx);
> > + if ( ebx & PSR_RESOURCE_TYPE_L3 )
> > + {
> > + cpuid_count(PSR_CPUID_LEVEL_CAT, 1, &eax, &ebx, &ecx, &edx);
> > +
> > + feat_tmp = feat_l3_cat;
> > + feat_l3_cat = NULL;
> > + feat_tmp->ops = l3_cat_ops;
> > +
> > + feat_tmp->ops.init_feature(eax, ebx, ecx, edx, feat_tmp, info);
>
> What's the point of the indirect call here, when you know the
> function is l3_cat_init_feature()?
>
Hmm, just want to keep the callback function calling style.
> > +static void cpu_fini_work(unsigned int cpu)
> > +{
> > + unsigned int socket = cpu_to_socket(cpu);
> > +
> > + if ( !socket_cpumask[socket] || cpumask_empty(socket_cpumask[socket]) )
> > + {
> > + struct psr_socket_info *info = socket_info + socket;
> > +
> > + free_feature(info);
>
> Pointless local variable "info", unless later patches add further uses.
>
Ok, will remove this variable. Thanks!
> > +static void __init init_psr(void)
> > +{
> > + unsigned int i;
> > +
> > + if ( opt_cos_max < 1 )
> > + {
> > + printk(XENLOG_INFO "CAT: disabled, cos_max is too small\n");
> > + return;
> > + }
> > +
> > + socket_info = xzalloc_array(struct psr_socket_info, nr_sockets);
> > +
> > + if ( !socket_info )
> > + {
> > + printk(XENLOG_INFO "Fail to alloc socket_info!\n");
> > + return;
> > + }
> > +
> > + for ( i = 0; i < nr_sockets; i++ )
> > + INIT_LIST_HEAD(&socket_info[i].feat_list);
>
> Please decide for one central place where to do such initialization:
> This and spin_lock_init() really should live together (and I think
> better there, not here).
>
Looks good, thanks!
> > +static int psr_cpu_prepare(unsigned int cpu)
> > +{
> > + return cpu_prepare_work(cpu);
> > +}
>
> What is this wrapper good for?
>
Just keep the old codes.
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 04/24] x86: refactor psr: implement CPU init and free flow.
2017-01-11 3:14 ` Yi Sun
@ 2017-01-11 13:48 ` Jan Beulich
2017-01-12 1:07 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2017-01-11 13:48 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 11.01.17 at 04:14, <yi.y.sun@linux.intel.com> wrote:
> On 17-01-10 04:45:05, Jan Beulich wrote:
>> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
>> > +/* L3 CAT callback functions implementation. */
>> > +static void l3_cat_init_feature(unsigned int eax, unsigned int ebx,
>> > + unsigned int ecx, unsigned int edx,
>>
>> This is rather unfortunate naming: How does the reader of this code
>> know what these values represent, without having to first go look in
>> the caller?
>>
> Do you mean the 'eax'-'edx'?
Yes.
> How about 'eax_register'?
How would that be any better? Perhaps the best way of making the
naming obvious would be to use the new cpuid_leaf structure here.
>> > + if ( !cpu_has(c, X86_FEATURE_PQE) || c->cpuid_level < PSR_CPUID_LEVEL_CAT )
>> > + return;
>>
>> Instead of such a double check, please consider clearing the PQE
>> feature bit when the maximum CPUID level is too low (which
>> shouldn't happen anyway).
>>
> Is this the responsibility of psr.c? X86_FEATURE_PQE bit is set by HW. Even the
> bit is set but CPUID level is low, I think SW would be better to keep it but
> not clear it. Because it indicates the HW capability. How do you think?
What use if keeping the flag if we can't use the feature? And to
answer your first question - whether that's being done in psr.c,
cpu/common.c, or cpu/intel.c I don't really care all that much; it
would certainly feel most natural to go here.
>> > + socket = cpu_to_socket(cpu);
>> > + info = socket_info + socket;
>> > + if ( info->feat_mask )
>> > + return;
>> > +
>> > + spin_lock_init(&info->ref_lock);
>> > +
>> > + cpuid_count(PSR_CPUID_LEVEL_CAT, 0, &eax, &ebx, &ecx, &edx);
>> > + if ( ebx & PSR_RESOURCE_TYPE_L3 )
>> > + {
>> > + cpuid_count(PSR_CPUID_LEVEL_CAT, 1, &eax, &ebx, &ecx, &edx);
>> > +
>> > + feat_tmp = feat_l3_cat;
>> > + feat_l3_cat = NULL;
>> > + feat_tmp->ops = l3_cat_ops;
>> > +
>> > + feat_tmp->ops.init_feature(eax, ebx, ecx, edx, feat_tmp, info);
>>
>> What's the point of the indirect call here, when you know the
>> function is l3_cat_init_feature()?
>>
> Hmm, just want to keep the callback function calling style.
Please don't use indirect calls when you don't need them.
>> > +static int psr_cpu_prepare(unsigned int cpu)
>> > +{
>> > + return cpu_prepare_work(cpu);
>> > +}
>>
>> What is this wrapper good for?
>>
> Just keep the old codes.
Well, you're overhauling the old code anyway (and you're actively
adding this function here), so - please don't introduce pointless
wrappers like this. They only complicate anyone following call flow,
even if just slightly.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 04/24] x86: refactor psr: implement CPU init and free flow.
2017-01-11 13:48 ` Jan Beulich
@ 2017-01-12 1:07 ` Yi Sun
0 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2017-01-12 1:07 UTC (permalink / raw)
To: Jan Beulich
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
On 17-01-11 06:48:27, Jan Beulich wrote:
> >>> On 11.01.17 at 04:14, <yi.y.sun@linux.intel.com> wrote:
> > On 17-01-10 04:45:05, Jan Beulich wrote:
> >> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> >> > +/* L3 CAT callback functions implementation. */
> >> > +static void l3_cat_init_feature(unsigned int eax, unsigned int ebx,
> >> > + unsigned int ecx, unsigned int edx,
> >>
> >> This is rather unfortunate naming: How does the reader of this code
> >> know what these values represent, without having to first go look in
> >> the caller?
> >>
> > Do you mean the 'eax'-'edx'?
>
> Yes.
>
> > How about 'eax_register'?
>
> How would that be any better? Perhaps the best way of making the
> naming obvious would be to use the new cpuid_leaf structure here.
>
Ok, will consider to assemble them into a structure.
> >> > + if ( !cpu_has(c, X86_FEATURE_PQE) || c->cpuid_level < PSR_CPUID_LEVEL_CAT )
> >> > + return;
> >>
> >> Instead of such a double check, please consider clearing the PQE
> >> feature bit when the maximum CPUID level is too low (which
> >> shouldn't happen anyway).
> >>
> > Is this the responsibility of psr.c? X86_FEATURE_PQE bit is set by HW. Even the
> > bit is set but CPUID level is low, I think SW would be better to keep it but
> > not clear it. Because it indicates the HW capability. How do you think?
>
> What use if keeping the flag if we can't use the feature? And to
> answer your first question - whether that's being done in psr.c,
> cpu/common.c, or cpu/intel.c I don't really care all that much; it
> would certainly feel most natural to go here.
>
Ok, will consider it.
> >> > + socket = cpu_to_socket(cpu);
> >> > + info = socket_info + socket;
> >> > + if ( info->feat_mask )
> >> > + return;
> >> > +
> >> > + spin_lock_init(&info->ref_lock);
> >> > +
> >> > + cpuid_count(PSR_CPUID_LEVEL_CAT, 0, &eax, &ebx, &ecx, &edx);
> >> > + if ( ebx & PSR_RESOURCE_TYPE_L3 )
> >> > + {
> >> > + cpuid_count(PSR_CPUID_LEVEL_CAT, 1, &eax, &ebx, &ecx, &edx);
> >> > +
> >> > + feat_tmp = feat_l3_cat;
> >> > + feat_l3_cat = NULL;
> >> > + feat_tmp->ops = l3_cat_ops;
> >> > +
> >> > + feat_tmp->ops.init_feature(eax, ebx, ecx, edx, feat_tmp, info);
> >>
> >> What's the point of the indirect call here, when you know the
> >> function is l3_cat_init_feature()?
> >>
> > Hmm, just want to keep the callback function calling style.
>
> Please don't use indirect calls when you don't need them.
>
Ok, thanks!
> >> > +static int psr_cpu_prepare(unsigned int cpu)
> >> > +{
> >> > + return cpu_prepare_work(cpu);
> >> > +}
> >>
> >> What is this wrapper good for?
> >>
> > Just keep the old codes.
>
> Well, you're overhauling the old code anyway (and you're actively
> adding this function here), so - please don't introduce pointless
> wrappers like this. They only complicate anyone following call flow,
> even if just slightly.
>
Sure, thanks!
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v4 05/24] x86: refactor psr: implement Domain init/free and schedule flows.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (3 preceding siblings ...)
2016-12-14 4:07 ` [PATCH v4 04/24] x86: refactor psr: implement CPU init and free flow Yi Sun
@ 2016-12-14 4:07 ` Yi Sun
2017-01-10 13:34 ` Jan Beulich
2016-12-14 4:07 ` [PATCH v4 06/24] x86: refactor psr: implement get hw info flow Yi Sun
` (18 subsequent siblings)
23 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:07 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements the Domain init/free and schedule flows.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 60 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 59 insertions(+), 1 deletion(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index fa9bc32..746c90e 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -113,6 +113,8 @@ struct feat_ops {
unsigned int ecx, unsigned int edx,
struct feat_node *feat,
struct psr_socket_info *info);
+ /* get_max_cos_max is used to get feature's cos_max. */
+ unsigned int (*get_max_cos_max)(const struct feat_node *feat);
};
@@ -213,8 +215,14 @@ static void l3_cat_init_feature(unsigned int eax, unsigned int ebx,
feat->info.l3_cat_info.cbm_len);
}
+static unsigned int l3_cat_get_max_cos_max(const struct feat_node *feat)
+{
+ return feat->info.l3_cat_info.cos_max;
+}
+
struct feat_ops l3_cat_ops = {
.init_feature = l3_cat_init_feature,
+ .get_max_cos_max = l3_cat_get_max_cos_max,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
@@ -358,11 +366,32 @@ void psr_free_rmid(struct domain *d)
d->arch.psr_rmid = 0;
}
+static inline unsigned int get_max_cos_max(const struct psr_socket_info *info)
+{
+ const struct feat_node *feat_tmp;
+ unsigned int cos_max = 0;
+
+ list_for_each_entry(feat_tmp, &info->feat_list, list)
+ cos_max = max(feat_tmp->ops.get_max_cos_max(feat_tmp), cos_max);
+
+ return cos_max;
+}
+
static inline void psr_assoc_init(void)
{
struct psr_assoc *psra = &this_cpu(psr_assoc);
- if ( psr_cmt_enabled() )
+ if ( socket_info )
+ {
+ unsigned int socket = cpu_to_socket(smp_processor_id());
+ const struct psr_socket_info *info = socket_info + socket;
+ unsigned int cos_max = get_max_cos_max(info);
+
+ if ( info->feat_mask )
+ psra->cos_mask = ((1ull << get_count_order(cos_max)) - 1) << 32;
+ }
+
+ if ( psr_cmt_enabled() || psra->cos_mask )
rdmsrl(MSR_IA32_PSR_ASSOC, psra->val);
}
@@ -371,6 +400,12 @@ static inline void psr_assoc_rmid(uint64_t *reg, unsigned int rmid)
*reg = (*reg & ~rmid_mask) | (rmid & rmid_mask);
}
+static inline void psr_assoc_cos(uint64_t *reg, unsigned int cos,
+ uint64_t cos_mask)
+{
+ *reg = (*reg & ~cos_mask) | (((uint64_t)cos << 32) & cos_mask);
+}
+
void psr_ctxt_switch_to(struct domain *d)
{
struct psr_assoc *psra = &this_cpu(psr_assoc);
@@ -379,6 +414,11 @@ void psr_ctxt_switch_to(struct domain *d)
if ( psr_cmt_enabled() )
psr_assoc_rmid(®, d->arch.psr_rmid);
+ if ( psra->cos_mask )
+ psr_assoc_cos(®, d->arch.psr_cos_ids ?
+ d->arch.psr_cos_ids[cpu_to_socket(smp_processor_id())] :
+ 0, psra->cos_mask);
+
if ( reg != psra->val )
{
wrmsrl(MSR_IA32_PSR_ASSOC, reg);
@@ -404,14 +444,32 @@ int psr_set_l3_cbm(struct domain *d, unsigned int socket,
return 0;
}
+/* Called with domain lock held, no extra lock needed for 'psr_cos_ids' */
+static void psr_free_cos(struct domain *d)
+{
+ if( !d->arch.psr_cos_ids )
+ return;
+
+ xfree(d->arch.psr_cos_ids);
+ d->arch.psr_cos_ids = NULL;
+}
+
int psr_domain_init(struct domain *d)
{
+ if ( socket_info )
+ {
+ d->arch.psr_cos_ids = xzalloc_array(unsigned int, nr_sockets);
+ if ( !d->arch.psr_cos_ids )
+ return -ENOMEM;
+ }
+
return 0;
}
void psr_domain_free(struct domain *d)
{
psr_free_rmid(d);
+ psr_free_cos(d);
}
static int cpu_prepare_work(unsigned int cpu)
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* Re: [PATCH v4 05/24] x86: refactor psr: implement Domain init/free and schedule flows.
2016-12-14 4:07 ` [PATCH v4 05/24] x86: refactor psr: implement Domain init/free and schedule flows Yi Sun
@ 2017-01-10 13:34 ` Jan Beulich
2017-01-11 3:17 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2017-01-10 13:34 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> @@ -358,11 +366,32 @@ void psr_free_rmid(struct domain *d)
> d->arch.psr_rmid = 0;
> }
>
> +static inline unsigned int get_max_cos_max(const struct psr_socket_info *info)
> +{
> + const struct feat_node *feat_tmp;
Same remark as on the earlier patch regarding the _tmp suffix.
Should this reoccur in later patches, I won't give the same
comment again.
> + unsigned int cos_max = 0;
> +
> + list_for_each_entry(feat_tmp, &info->feat_list, list)
> + cos_max = max(feat_tmp->ops.get_max_cos_max(feat_tmp), cos_max);
> +
> + return cos_max;
> +}
> +
> static inline void psr_assoc_init(void)
> {
> struct psr_assoc *psra = &this_cpu(psr_assoc);
>
> - if ( psr_cmt_enabled() )
> + if ( socket_info )
> + {
> + unsigned int socket = cpu_to_socket(smp_processor_id());
> + const struct psr_socket_info *info = socket_info + socket;
> + unsigned int cos_max = get_max_cos_max(info);
> +
> + if ( info->feat_mask )
> + psra->cos_mask = ((1ull << get_count_order(cos_max)) - 1) << 32;
> + }
> +
> + if ( psr_cmt_enabled() || psra->cos_mask )
> rdmsrl(MSR_IA32_PSR_ASSOC, psra->val);
> }
>
> @@ -371,6 +400,12 @@ static inline void psr_assoc_rmid(uint64_t *reg, unsigned int rmid)
> *reg = (*reg & ~rmid_mask) | (rmid & rmid_mask);
> }
>
> +static inline void psr_assoc_cos(uint64_t *reg, unsigned int cos,
> + uint64_t cos_mask)
> +{
> + *reg = (*reg & ~cos_mask) | (((uint64_t)cos << 32) & cos_mask);
This recurring 32 would perhaps better become a #define, so they
can be identified as the same entity (at least I think they are).
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 05/24] x86: refactor psr: implement Domain init/free and schedule flows.
2017-01-10 13:34 ` Jan Beulich
@ 2017-01-11 3:17 ` Yi Sun
0 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2017-01-11 3:17 UTC (permalink / raw)
To: Jan Beulich
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
On 17-01-10 06:34:50, Jan Beulich wrote:
> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> > @@ -358,11 +366,32 @@ void psr_free_rmid(struct domain *d)
> > d->arch.psr_rmid = 0;
> > }
> >
> > +static inline unsigned int get_max_cos_max(const struct psr_socket_info *info)
> > +{
> > + const struct feat_node *feat_tmp;
>
> Same remark as on the earlier patch regarding the _tmp suffix.
> Should this reoccur in later patches, I won't give the same
> comment again.
>
Thank you! I will take care all of them.
> > + unsigned int cos_max = 0;
> > +
> > + list_for_each_entry(feat_tmp, &info->feat_list, list)
> > + cos_max = max(feat_tmp->ops.get_max_cos_max(feat_tmp), cos_max);
> > +
> > + return cos_max;
> > +}
> > +
> > static inline void psr_assoc_init(void)
> > {
> > struct psr_assoc *psra = &this_cpu(psr_assoc);
> >
> > - if ( psr_cmt_enabled() )
> > + if ( socket_info )
> > + {
> > + unsigned int socket = cpu_to_socket(smp_processor_id());
> > + const struct psr_socket_info *info = socket_info + socket;
> > + unsigned int cos_max = get_max_cos_max(info);
> > +
> > + if ( info->feat_mask )
> > + psra->cos_mask = ((1ull << get_count_order(cos_max)) - 1) << 32;
> > + }
> > +
> > + if ( psr_cmt_enabled() || psra->cos_mask )
> > rdmsrl(MSR_IA32_PSR_ASSOC, psra->val);
> > }
> >
> > @@ -371,6 +400,12 @@ static inline void psr_assoc_rmid(uint64_t *reg, unsigned int rmid)
> > *reg = (*reg & ~rmid_mask) | (rmid & rmid_mask);
> > }
> >
> > +static inline void psr_assoc_cos(uint64_t *reg, unsigned int cos,
> > + uint64_t cos_mask)
> > +{
> > + *reg = (*reg & ~cos_mask) | (((uint64_t)cos << 32) & cos_mask);
>
> This recurring 32 would perhaps better become a #define, so they
> can be identified as the same entity (at least I think they are).
>
Good suggestion, thanks!
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v4 06/24] x86: refactor psr: implement get hw info flow.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (4 preceding siblings ...)
2016-12-14 4:07 ` [PATCH v4 05/24] x86: refactor psr: implement Domain init/free and schedule flows Yi Sun
@ 2016-12-14 4:07 ` Yi Sun
2017-01-10 13:46 ` Jan Beulich
2016-12-14 4:07 ` [PATCH v4 07/24] x86: refactor psr: implement get value flow Yi Sun
` (17 subsequent siblings)
23 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:07 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements get HW info flow including L3 CAT callback
function.
It also changes sysctl interface to make it more general.
With this patch, 'psr-hwinfo' can work for L3 CAT.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 48 ++++++++++++++++++++++++++++++++++++++++++++---
xen/arch/x86/sysctl.c | 14 +++++++++-----
xen/include/asm-x86/psr.h | 9 +++++++--
3 files changed, 61 insertions(+), 10 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 746c90e..095b3a7 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -115,6 +115,9 @@ struct feat_ops {
struct psr_socket_info *info);
/* get_max_cos_max is used to get feature's cos_max. */
unsigned int (*get_max_cos_max)(const struct feat_node *feat);
+ /* get_feat_info is used to get feature HW info. */
+ bool (*get_feat_info)(const struct feat_node *feat, enum cbm_type type,
+ uint32_t dat[], uint32_t array_len);
};
@@ -220,9 +223,24 @@ static unsigned int l3_cat_get_max_cos_max(const struct feat_node *feat)
return feat->info.l3_cat_info.cos_max;
}
+static bool l3_cat_get_feat_info(const struct feat_node *feat,
+ enum cbm_type type,
+ uint32_t dat[], uint32_t array_len)
+{
+ if ( !dat || 3 > array_len || type != PSR_CBM_TYPE_L3 )
+ return false;
+
+ dat[CBM_LEN] = feat->info.l3_cat_info.cbm_len;
+ dat[COS_MAX] = feat->info.l3_cat_info.cos_max;
+ dat[CDP_FLAG] = 0;
+
+ return true;
+}
+
struct feat_ops l3_cat_ops = {
.init_feature = l3_cat_init_feature,
.get_max_cos_max = l3_cat_get_max_cos_max,
+ .get_feat_info = l3_cat_get_feat_info,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
@@ -426,10 +444,34 @@ void psr_ctxt_switch_to(struct domain *d)
}
}
-int psr_get_cat_l3_info(unsigned int socket, uint32_t *cbm_len,
- uint32_t *cos_max, uint32_t *flags)
+static struct psr_socket_info *get_socket_info(unsigned int socket)
{
- return 0;
+ if ( !socket_info )
+ return ERR_PTR(-ENODEV);
+
+ if ( socket >= nr_sockets )
+ return ERR_PTR(-ENOTSOCK);
+
+ if ( !socket_info[socket].feat_mask )
+ return ERR_PTR(-ENOENT);
+
+ return socket_info + socket;
+}
+
+int psr_get_info(unsigned int socket, enum cbm_type type,
+ uint32_t dat[], uint32_t array_len)
+{
+ struct psr_socket_info *info = get_socket_info(socket);
+ struct feat_node *feat_tmp;
+
+ if ( IS_ERR(info) )
+ return PTR_ERR(info);
+
+ list_for_each_entry(feat_tmp, &info->feat_list, list)
+ if ( feat_tmp->ops.get_feat_info(feat_tmp, type, dat, array_len) )
+ return 0;
+
+ return -ENOENT;
}
int psr_get_l3_cbm(struct domain *d, unsigned int socket,
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index 14e7dc7..168ed45 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -176,15 +176,19 @@ long arch_do_sysctl(
switch ( sysctl->u.psr_cat_op.cmd )
{
case XEN_SYSCTL_PSR_CAT_get_l3_info:
- ret = psr_get_cat_l3_info(sysctl->u.psr_cat_op.target,
- &sysctl->u.psr_cat_op.u.l3_info.cbm_len,
- &sysctl->u.psr_cat_op.u.l3_info.cos_max,
- &sysctl->u.psr_cat_op.u.l3_info.flags);
+ {
+ uint32_t dat[3];
+ ret = psr_get_info(sysctl->u.psr_cat_op.target,
+ PSR_CBM_TYPE_L3, dat, 3);
+
+ sysctl->u.psr_cat_op.u.l3_info.cbm_len = dat[CBM_LEN];
+ sysctl->u.psr_cat_op.u.l3_info.cos_max = dat[COS_MAX];
+ sysctl->u.psr_cat_op.u.l3_info.flags = dat[CDP_FLAG];
if ( !ret && __copy_field_to_guest(u_sysctl, sysctl, u.psr_cat_op) )
ret = -EFAULT;
break;
-
+ }
default:
ret = -EOPNOTSUPP;
break;
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index 57f47e9..c872fff 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -33,6 +33,11 @@
/* L3 CDP Enable bit*/
#define PSR_L3_QOS_CDP_ENABLE_BIT 0x0
+/* Used by psr_get_info() */
+#define CBM_LEN 0
+#define COS_MAX 1
+#define CDP_FLAG 2
+
struct psr_cmt_l3 {
unsigned int features;
unsigned int upscaling_factor;
@@ -63,8 +68,8 @@ int psr_alloc_rmid(struct domain *d);
void psr_free_rmid(struct domain *d);
void psr_ctxt_switch_to(struct domain *d);
-int psr_get_cat_l3_info(unsigned int socket, uint32_t *cbm_len,
- uint32_t *cos_max, uint32_t *flags);
+int psr_get_info(unsigned int socket, enum cbm_type type,
+ uint32_t dat[], uint32_t array_len);
int psr_get_l3_cbm(struct domain *d, unsigned int socket,
uint64_t *cbm, enum cbm_type type);
int psr_set_l3_cbm(struct domain *d, unsigned int socket,
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* Re: [PATCH v4 06/24] x86: refactor psr: implement get hw info flow.
2016-12-14 4:07 ` [PATCH v4 06/24] x86: refactor psr: implement get hw info flow Yi Sun
@ 2017-01-10 13:46 ` Jan Beulich
2017-01-11 5:13 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2017-01-10 13:46 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> --- a/xen/arch/x86/psr.c
> +++ b/xen/arch/x86/psr.c
> @@ -115,6 +115,9 @@ struct feat_ops {
> struct psr_socket_info *info);
> /* get_max_cos_max is used to get feature's cos_max. */
> unsigned int (*get_max_cos_max)(const struct feat_node *feat);
> + /* get_feat_info is used to get feature HW info. */
> + bool (*get_feat_info)(const struct feat_node *feat, enum cbm_type type,
> + uint32_t dat[], uint32_t array_len);
data, value, or val would all seem okay, but dat suggests an acronym
of other than data (which I think it is meant to be).
> @@ -220,9 +223,24 @@ static unsigned int l3_cat_get_max_cos_max(const struct
> feat_node *feat)
> return feat->info.l3_cat_info.cos_max;
> }
>
> +static bool l3_cat_get_feat_info(const struct feat_node *feat,
> + enum cbm_type type,
> + uint32_t dat[], uint32_t array_len)
array_len wants to be size_t or unsigned int. And more generally,
please avoid fixed width types when you don't really mean such.
> +int psr_get_info(unsigned int socket, enum cbm_type type,
> + uint32_t dat[], uint32_t array_len)
> +{
> + struct psr_socket_info *info = get_socket_info(socket);
> + struct feat_node *feat_tmp;
With the hook function taking a pointer to const I don#t see why
this one can't be const, too.
> + if ( IS_ERR(info) )
> + return PTR_ERR(info);
> +
> + list_for_each_entry(feat_tmp, &info->feat_list, list)
> + if ( feat_tmp->ops.get_feat_info(feat_tmp, type, dat, array_len) )
Wouldn't the type check better be done here than inside each
function? That would then also allow for terminating the loop
earlier (when the type was found, instead of when a function
returns success).
> --- a/xen/include/asm-x86/psr.h
> +++ b/xen/include/asm-x86/psr.h
> @@ -33,6 +33,11 @@
> /* L3 CDP Enable bit*/
> #define PSR_L3_QOS_CDP_ENABLE_BIT 0x0
>
> +/* Used by psr_get_info() */
> +#define CBM_LEN 0
> +#define COS_MAX 1
> +#define CDP_FLAG 2
These needing putting in a header means that you want to prefix
them, e.g. by PSR_. Also with the next value used e.g. as array
dimension, I think you also want to name that value (currently 3)
and use it instead of a plain number which would need to be
adjusted everywhere once a value gets added here.
Also - is CDP_FLAG really a suitable name for flags? Wouldn't
PSR_FLAGS be better (as being more general)?
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 06/24] x86: refactor psr: implement get hw info flow.
2017-01-10 13:46 ` Jan Beulich
@ 2017-01-11 5:13 ` Yi Sun
2017-01-11 13:53 ` Jan Beulich
0 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2017-01-11 5:13 UTC (permalink / raw)
To: Jan Beulich
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
On 17-01-10 06:46:03, Jan Beulich wrote:
> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> > --- a/xen/arch/x86/psr.c
> > +++ b/xen/arch/x86/psr.c
> > @@ -115,6 +115,9 @@ struct feat_ops {
> > struct psr_socket_info *info);
> > /* get_max_cos_max is used to get feature's cos_max. */
> > unsigned int (*get_max_cos_max)(const struct feat_node *feat);
> > + /* get_feat_info is used to get feature HW info. */
> > + bool (*get_feat_info)(const struct feat_node *feat, enum cbm_type type,
> > + uint32_t dat[], uint32_t array_len);
>
> data, value, or val would all seem okay, but dat suggests an acronym
> of other than data (which I think it is meant to be).
>
Ok, will change it to data.
> > @@ -220,9 +223,24 @@ static unsigned int l3_cat_get_max_cos_max(const struct
> > feat_node *feat)
> > return feat->info.l3_cat_info.cos_max;
> > }
> >
> > +static bool l3_cat_get_feat_info(const struct feat_node *feat,
> > + enum cbm_type type,
> > + uint32_t dat[], uint32_t array_len)
>
> array_len wants to be size_t or unsigned int. And more generally,
> please avoid fixed width types when you don't really mean such.
>
Got it, thank you!
> > +int psr_get_info(unsigned int socket, enum cbm_type type,
> > + uint32_t dat[], uint32_t array_len)
> > +{
> > + struct psr_socket_info *info = get_socket_info(socket);
> > + struct feat_node *feat_tmp;
>
> With the hook function taking a pointer to const I don#t see why
> this one can't be const, too.
>
Do you mean feat_tmp? Thanks!
> > + if ( IS_ERR(info) )
> > + return PTR_ERR(info);
> > +
> > + list_for_each_entry(feat_tmp, &info->feat_list, list)
> > + if ( feat_tmp->ops.get_feat_info(feat_tmp, type, dat, array_len) )
>
> Wouldn't the type check better be done here than inside each
> function? That would then also allow for terminating the loop
> earlier (when the type was found, instead of when a function
> returns success).
>
Ok, thanks for the suggestion!
> > --- a/xen/include/asm-x86/psr.h
> > +++ b/xen/include/asm-x86/psr.h
> > @@ -33,6 +33,11 @@
> > /* L3 CDP Enable bit*/
> > #define PSR_L3_QOS_CDP_ENABLE_BIT 0x0
> >
> > +/* Used by psr_get_info() */
> > +#define CBM_LEN 0
> > +#define COS_MAX 1
> > +#define CDP_FLAG 2
>
> These needing putting in a header means that you want to prefix
> them, e.g. by PSR_. Also with the next value used e.g. as array
> dimension, I think you also want to name that value (currently 3)
> and use it instead of a plain number which would need to be
> adjusted everywhere once a value gets added here.
>
> Also - is CDP_FLAG really a suitable name for flags? Wouldn't
> PSR_FLAGS be better (as being more general)?
>
Thanks for the suggestions!
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 06/24] x86: refactor psr: implement get hw info flow.
2017-01-11 5:13 ` Yi Sun
@ 2017-01-11 13:53 ` Jan Beulich
2017-01-12 1:08 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2017-01-11 13:53 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 11.01.17 at 06:13, <yi.y.sun@linux.intel.com> wrote:
> On 17-01-10 06:46:03, Jan Beulich wrote:
>> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
>> > +int psr_get_info(unsigned int socket, enum cbm_type type,
>> > + uint32_t dat[], uint32_t array_len)
>> > +{
>> > + struct psr_socket_info *info = get_socket_info(socket);
>> > + struct feat_node *feat_tmp;
>>
>> With the hook function taking a pointer to const I don#t see why
>> this one can't be const, too.
>>
> Do you mean feat_tmp? Thanks!
Not sure why you ask - yes, that's why I've placed the comment at
precisely this line.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 06/24] x86: refactor psr: implement get hw info flow.
2017-01-11 13:53 ` Jan Beulich
@ 2017-01-12 1:08 ` Yi Sun
0 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2017-01-12 1:08 UTC (permalink / raw)
To: Jan Beulich
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
On 17-01-11 06:53:30, Jan Beulich wrote:
> >>> On 11.01.17 at 06:13, <yi.y.sun@linux.intel.com> wrote:
> > On 17-01-10 06:46:03, Jan Beulich wrote:
> >> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> >> > +int psr_get_info(unsigned int socket, enum cbm_type type,
> >> > + uint32_t dat[], uint32_t array_len)
> >> > +{
> >> > + struct psr_socket_info *info = get_socket_info(socket);
> >> > + struct feat_node *feat_tmp;
> >>
> >> With the hook function taking a pointer to const I don#t see why
> >> this one can't be const, too.
> >>
> > Do you mean feat_tmp? Thanks!
>
> Not sure why you ask - yes, that's why I've placed the comment at
> precisely this line.
>
Got it, thanks!
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v4 07/24] x86: refactor psr: implement get value flow.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (5 preceding siblings ...)
2016-12-14 4:07 ` [PATCH v4 06/24] x86: refactor psr: implement get hw info flow Yi Sun
@ 2016-12-14 4:07 ` Yi Sun
2017-01-10 13:50 ` Jan Beulich
2016-12-14 4:07 ` [PATCH v4 08/24] x86: refactor psr: set value: implement framework Yi Sun
` (16 subsequent siblings)
23 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:07 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements get value flow including L3 CAT callback
function.
It also changes domctl interface to make it more general.
With this patch, 'psr-cat-show' can work for L3 CAT.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/domctl.c | 18 +++++++++---------
xen/arch/x86/psr.c | 39 ++++++++++++++++++++++++++++++++++++---
xen/include/asm-x86/psr.h | 4 ++--
3 files changed, 47 insertions(+), 14 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 2a2fe04..3d7fc34 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1404,23 +1404,23 @@ long arch_do_domctl(
break;
case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM:
- ret = psr_get_l3_cbm(d, domctl->u.psr_cat_op.target,
- &domctl->u.psr_cat_op.data,
- PSR_CBM_TYPE_L3);
+ ret = psr_get_val(d, domctl->u.psr_cat_op.target,
+ &domctl->u.psr_cat_op.data,
+ PSR_CBM_TYPE_L3);
copyback = 1;
break;
case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CODE:
- ret = psr_get_l3_cbm(d, domctl->u.psr_cat_op.target,
- &domctl->u.psr_cat_op.data,
- PSR_CBM_TYPE_L3_CODE);
+ ret = psr_get_val(d, domctl->u.psr_cat_op.target,
+ &domctl->u.psr_cat_op.data,
+ PSR_CBM_TYPE_L3_CODE);
copyback = 1;
break;
case XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA:
- ret = psr_get_l3_cbm(d, domctl->u.psr_cat_op.target,
- &domctl->u.psr_cat_op.data,
- PSR_CBM_TYPE_L3_DATA);
+ ret = psr_get_val(d, domctl->u.psr_cat_op.target,
+ &domctl->u.psr_cat_op.data,
+ PSR_CBM_TYPE_L3_DATA);
copyback = 1;
break;
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 095b3a7..1ea694e 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -118,6 +118,9 @@ struct feat_ops {
/* get_feat_info is used to get feature HW info. */
bool (*get_feat_info)(const struct feat_node *feat, enum cbm_type type,
uint32_t dat[], uint32_t array_len);
+ /* get_val is used to get feature COS register value. */
+ bool (*get_val)(const struct feat_node *feat, unsigned int cos,
+ enum cbm_type type, uint64_t *val);
};
@@ -237,10 +240,26 @@ static bool l3_cat_get_feat_info(const struct feat_node *feat,
return true;
}
+static bool l3_cat_get_val(const struct feat_node *feat, unsigned int cos,
+ enum cbm_type type, uint64_t *val)
+{
+ if ( type != PSR_CBM_TYPE_L3 )
+ return false;
+
+ if ( cos > feat->info.l3_cat_info.cos_max )
+ /* Use default value. */
+ cos = 0;
+
+ *val = feat->cos_reg_val[cos];
+
+ return true;
+}
+
struct feat_ops l3_cat_ops = {
.init_feature = l3_cat_init_feature,
.get_max_cos_max = l3_cat_get_max_cos_max,
.get_feat_info = l3_cat_get_feat_info,
+ .get_val = l3_cat_get_val,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
@@ -474,10 +493,24 @@ int psr_get_info(unsigned int socket, enum cbm_type type,
return -ENOENT;
}
-int psr_get_l3_cbm(struct domain *d, unsigned int socket,
- uint64_t *cbm, enum cbm_type type)
+int psr_get_val(struct domain *d, unsigned int socket,
+ uint64_t *val, enum cbm_type type)
{
- return 0;
+ const struct psr_socket_info *info = get_socket_info(socket);
+ unsigned int cos = d->arch.psr_cos_ids[socket];
+ const struct feat_node *feat_tmp;
+
+ if ( IS_ERR(info) )
+ return PTR_ERR(info);
+
+ list_for_each_entry(feat_tmp, &info->feat_list, list)
+ {
+ if ( feat_tmp->ops.get_val(feat_tmp, cos, type, val) )
+ /* Found */
+ return 0;
+ }
+
+ return -ENOENT;
}
int psr_set_l3_cbm(struct domain *d, unsigned int socket,
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index c872fff..8e3c90e 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -70,8 +70,8 @@ void psr_ctxt_switch_to(struct domain *d);
int psr_get_info(unsigned int socket, enum cbm_type type,
uint32_t dat[], uint32_t array_len);
-int psr_get_l3_cbm(struct domain *d, unsigned int socket,
- uint64_t *cbm, enum cbm_type type);
+int psr_get_val(struct domain *d, unsigned int socket,
+ uint64_t *val, enum cbm_type type);
int psr_set_l3_cbm(struct domain *d, unsigned int socket,
uint64_t cbm, enum cbm_type type);
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* Re: [PATCH v4 07/24] x86: refactor psr: implement get value flow.
2016-12-14 4:07 ` [PATCH v4 07/24] x86: refactor psr: implement get value flow Yi Sun
@ 2017-01-10 13:50 ` Jan Beulich
2017-01-11 5:16 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2017-01-10 13:50 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> This patch implements get value flow including L3 CAT callback
> function.
>
> It also changes domctl interface to make it more general.
>
> With this patch, 'psr-cat-show' can work for L3 CAT.
How about CDP? You don't implement anything for it here, and looking
over the subjects of the remaining patches there also doesn't seem to
be anything to come.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* Re: [PATCH v4 07/24] x86: refactor psr: implement get value flow.
2017-01-10 13:50 ` Jan Beulich
@ 2017-01-11 5:16 ` Yi Sun
2017-01-11 13:54 ` Jan Beulich
0 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2017-01-11 5:16 UTC (permalink / raw)
To: Jan Beulich
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
On 17-01-10 06:50:36, Jan Beulich wrote:
> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> > This patch implements get value flow including L3 CAT callback
> > function.
> >
> > It also changes domctl interface to make it more general.
> >
> > With this patch, 'psr-cat-show' can work for L3 CAT.
>
> How about CDP? You don't implement anything for it here, and looking
> over the subjects of the remaining patches there also doesn't seem to
> be anything to come.
>
I split CDP codes out to a few patches from 13 to 16. CDP is a stand alone
feature now but not combined with L3 CAT.
[PATCH v4 13/24] x86: refactor psr: implement CPU init and free flow for CDP.
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* Re: [PATCH v4 07/24] x86: refactor psr: implement get value flow.
2017-01-11 5:16 ` Yi Sun
@ 2017-01-11 13:54 ` Jan Beulich
2017-01-12 1:09 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2017-01-11 13:54 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 11.01.17 at 06:16, <yi.y.sun@linux.intel.com> wrote:
> On 17-01-10 06:50:36, Jan Beulich wrote:
>> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
>> > This patch implements get value flow including L3 CAT callback
>> > function.
>> >
>> > It also changes domctl interface to make it more general.
>> >
>> > With this patch, 'psr-cat-show' can work for L3 CAT.
>>
>> How about CDP? You don't implement anything for it here, and looking
>> over the subjects of the remaining patches there also doesn't seem to
>> be anything to come.
>>
> I split CDP codes out to a few patches from 13 to 16. CDP is a stand alone
> feature now but not combined with L3 CAT.
>
> [PATCH v4 13/24] x86: refactor psr: implement CPU init and free flow for
> CDP.
So I guess the "get value" part is to be found there then, too?
Because that's what I was looking for before making the remark.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* Re: [PATCH v4 07/24] x86: refactor psr: implement get value flow.
2017-01-11 13:54 ` Jan Beulich
@ 2017-01-12 1:09 ` Yi Sun
0 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2017-01-12 1:09 UTC (permalink / raw)
To: Jan Beulich
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
On 17-01-11 06:54:30, Jan Beulich wrote:
> >>> On 11.01.17 at 06:16, <yi.y.sun@linux.intel.com> wrote:
> > On 17-01-10 06:50:36, Jan Beulich wrote:
> >> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> >> > This patch implements get value flow including L3 CAT callback
> >> > function.
> >> >
> >> > It also changes domctl interface to make it more general.
> >> >
> >> > With this patch, 'psr-cat-show' can work for L3 CAT.
> >>
> >> How about CDP? You don't implement anything for it here, and looking
> >> over the subjects of the remaining patches there also doesn't seem to
> >> be anything to come.
> >>
> > I split CDP codes out to a few patches from 13 to 16. CDP is a stand alone
> > feature now but not combined with L3 CAT.
> >
> > [PATCH v4 13/24] x86: refactor psr: implement CPU init and free flow for
> > CDP.
>
> So I guess the "get value" part is to be found there then, too?
> Because that's what I was looking for before making the remark.
>
Yes, you can check this one.
[PATCH v4 15/24] x86: refactor psr: implement get value flow for CDP.
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v4 08/24] x86: refactor psr: set value: implement framework.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (6 preceding siblings ...)
2016-12-14 4:07 ` [PATCH v4 07/24] x86: refactor psr: implement get value flow Yi Sun
@ 2016-12-14 4:07 ` Yi Sun
2017-01-10 14:17 ` Jan Beulich
2016-12-14 4:07 ` [PATCH v4 09/24] x86: refactor psr: set value: assemble features value array Yi Sun
` (15 subsequent siblings)
23 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:07 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
As set value flow is the most complicated one in psr, it will be
divided to some patches to make things clearer. This patch
implements the set value framework to show a whole picture firstly.
It also changes domctl interface to make it more general.
To make the set value flow be general and can support multiple features
at same time, it includes below steps:
1. Get COS ID of current domain using.
2. Assemble a value array to store all features current value
in it and replace the current value of the feature which is
being set to the new input value.
3. Find if there is already a COS ID on which all features'
values are same as the array. Then, we can reuse this COS
ID.
4. If fail to find, we need allocate a new COS ID. Only COS ID which ref
is 0 or 1 can be allocated.
5. Write all features MSRs according to the COS ID.
6. Update ref according to COS ID.
7. Save the COS ID into current domain's psr_cos_ids[socket] so that we
can know which COS the domain is using on the socket.
So, some functions are abstracted and the callback functions will be
implemented in next patches.
Here is an example to understand the process. The CPU supports
two featuers, e.g. L3 CAT and L2 CAT. user wants to set L3 CAT
of Dom1 to 0x1ff.
1. Get the old_cos of Dom1 which is 0. L3 CAT is the first
element of feature list. The COS registers values are below at
this time.
-------------------------------
| COS 0 | COS 1 | COS 2 | ... |
-------------------------------
L3 CAT | 0x7ff | ... | ... | ... |
-------------------------------
L2 CAT | 0xff | ... | ... | ... |
-------------------------------
2. Assemble The value array to be:
val[0]: 0x1ff
val[1]: 0xff
3. It cannot find a matching COS.
4. Allocate COS 1 to store the value set.
5. Write the COS 1 registers. The COS registers values are
changed to below now.
-------------------------------
| COS 0 | COS 1 | COS 2 | ... |
-------------------------------
L3 CAT | 0x7ff | 0x1ff | ... | ... |
-------------------------------
L2 CAT | 0xff | 0xff | ... | ... |
-------------------------------
6. The ref[1] is increased to 1 because Dom1 is using it now.
7. Save 1 to Dom1's psr_cos_ids[socket].
Then, user wants to set L3 CAT of Dom2 to 0x1ff too. The old_cos
of Dom2 is 0 too. Repeat above flow.
The val array assembled is:
val[0]: 0x1ff
val[1]: 0xff
So, it can find a matching COS, COS 1. Then, it can reuse COS 1
for Dom2.
The ref[1] is increased to 2 now because both Dom1 and Dom2 are
using this COS ID. Set 1 to Dom2's psr_cos_ids[socket].
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/domctl.c | 18 ++---
xen/arch/x86/psr.c | 183 +++++++++++++++++++++++++++++++++++++++++++++-
xen/include/asm-x86/psr.h | 4 +-
3 files changed, 192 insertions(+), 13 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 3d7fc34..8e5502a 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1386,21 +1386,21 @@ long arch_do_domctl(
switch ( domctl->u.psr_cat_op.cmd )
{
case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM:
- ret = psr_set_l3_cbm(d, domctl->u.psr_cat_op.target,
- domctl->u.psr_cat_op.data,
- PSR_CBM_TYPE_L3);
+ ret = psr_set_val(d, domctl->u.psr_cat_op.target,
+ domctl->u.psr_cat_op.data,
+ PSR_CBM_TYPE_L3);
break;
case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CODE:
- ret = psr_set_l3_cbm(d, domctl->u.psr_cat_op.target,
- domctl->u.psr_cat_op.data,
- PSR_CBM_TYPE_L3_CODE);
+ ret = psr_set_val(d, domctl->u.psr_cat_op.target,
+ domctl->u.psr_cat_op.data,
+ PSR_CBM_TYPE_L3_CODE);
break;
case XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA:
- ret = psr_set_l3_cbm(d, domctl->u.psr_cat_op.target,
- domctl->u.psr_cat_op.data,
- PSR_CBM_TYPE_L3_DATA);
+ ret = psr_set_val(d, domctl->u.psr_cat_op.target,
+ domctl->u.psr_cat_op.data,
+ PSR_CBM_TYPE_L3_DATA);
break;
case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM:
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 1ea694e..838f2d1 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -513,18 +513,197 @@ int psr_get_val(struct domain *d, unsigned int socket,
return -ENOENT;
}
-int psr_set_l3_cbm(struct domain *d, unsigned int socket,
- uint64_t cbm, enum cbm_type type)
+/* Set value functions */
+static unsigned int get_cos_num(const struct psr_socket_info *info)
{
return 0;
}
+static int get_old_set_new(uint64_t *val,
+ uint32_t array_len,
+ const struct psr_socket_info *info,
+ unsigned int old_cos,
+ enum cbm_type type,
+ uint64_t m)
+{
+ return 0;
+}
+
+static int find_cos(const uint64_t *val, uint32_t array_len,
+ enum cbm_type type,
+ const struct psr_socket_info *info)
+{
+ return 0;
+}
+
+static int alloc_new_cos(const struct psr_socket_info *info,
+ const uint64_t *val, uint32_t array_len,
+ unsigned int old_cos,
+ enum cbm_type type)
+{
+ return 0;
+}
+
+static int write_psr_msr(unsigned int socket, unsigned int cos,
+ const uint64_t *val)
+{
+ return 0;
+}
+
+int psr_set_val(struct domain *d, unsigned int socket,
+ uint64_t val, enum cbm_type type)
+{
+ unsigned int old_cos;
+ int cos, ret;
+ unsigned int *ref;
+ uint64_t *val_array;
+ struct psr_socket_info *info = get_socket_info(socket);
+ uint32_t array_len;
+
+ if ( IS_ERR(info) )
+ return PTR_ERR(info);
+
+ /*
+ * Step 0:
+ * old_cos means the COS ID current domain is using. By default, it is 0.
+ *
+ * For every COS ID, there is a reference count to record how many domains
+ * are using the COS register corresponding to this COS ID.
+ * - If ref[old_cos] is 0, that means this COS is not used by any domain.
+ * - If ref[old_cos] is 1, that means this COS is only used by current
+ * domain.
+ * - If ref[old_cos] is more than 1, that mean multiple domains are using
+ * this COS.
+ */
+ old_cos = d->arch.psr_cos_ids[socket];
+ if ( old_cos > MAX_COS_REG_CNT )
+ return -EOVERFLOW;
+
+ ref = info->cos_ref;
+
+ /*
+ * Step 1:
+ * Assemle a value array to store all featues cos_reg_val[old_cos].
+ * And, set the input val into array according to the feature's
+ * position in array.
+ */
+ array_len = get_cos_num((const struct psr_socket_info *)info);
+ val_array = xzalloc_array(uint64_t, array_len);
+ if ( !val_array )
+ return -ENOMEM;
+
+ if ( (ret = get_old_set_new(val_array, array_len,
+ (const struct psr_socket_info *)info,
+ old_cos, type, val)) != 0 )
+ {
+ xfree(val_array);
+ return ret;
+ }
+
+ /*
+ * Lock here to make sure the ref is not changed during find and
+ * write process.
+ */
+ spin_lock(&info->ref_lock);
+
+ /*
+ * Step 2:
+ * Try to find if there is already a COS ID on which all features' values
+ * are same as the array. Then, we can reuse this COS ID.
+ */
+ cos = find_cos((const uint64_t *)val_array, array_len, type,
+ (const struct psr_socket_info *)info);
+ if ( cos >= 0 )
+ {
+ if ( cos == old_cos )
+ {
+ spin_unlock(&info->ref_lock);
+ xfree(val_array);
+ return 0;
+ }
+ }
+ else
+ {
+ /*
+ * Step 3:
+ * If fail to find, we need allocate a new COS ID.
+ * If multiple domains are using same COS ID, its ref is more
+ * than 1. That means we cannot free this COS to make current domain
+ * use it. Because other domains are using the value saved in the COS.
+ * Unless the ref is changed to 1 (mean only current domain is using
+ * it), we cannot allocate the COS ID to current domain.
+ * So, only the COS ID which ref is 1 or 0 can be allocated.
+ */
+ cos = alloc_new_cos((const struct psr_socket_info *)info,
+ (const uint64_t *)val_array, array_len,
+ old_cos, type);
+ if ( cos < 0 )
+ {
+ spin_unlock(&info->ref_lock);
+ xfree(val_array);
+ return cos;
+ }
+
+ /*
+ * Step 4:
+ * Write all features MSRs according to the COS ID.
+ */
+ ret = write_psr_msr(socket, cos, (const uint64_t *)val_array);
+ if ( ret )
+ {
+ spin_unlock(&info->ref_lock);
+ xfree(val_array);
+ return ret;
+ }
+ }
+
+ /*
+ * Step 5:
+ * Update ref according to COS ID.
+ */
+ ref[cos]++;
+ ref[old_cos]--;
+ spin_unlock(&info->ref_lock);
+
+ /*
+ * Step 6:
+ * Save the COS ID into current domain's psr_cos_ids[] so that we can know
+ * which COS the domain is using on the socket. One domain can only use
+ * one COS ID at same time.
+ */
+ d->arch.psr_cos_ids[socket] = cos;
+ xfree(val_array);
+
+ return 0;
+}
+
/* Called with domain lock held, no extra lock needed for 'psr_cos_ids' */
static void psr_free_cos(struct domain *d)
{
+ unsigned int socket;
+ unsigned int cos;
+ struct psr_socket_info *info;
+
if( !d->arch.psr_cos_ids )
return;
+ /* Domain is free so its cos_ref should be decreased. */
+ for( socket = 0; socket < nr_sockets; socket++ )
+ {
+ /* cos 0 is default one which does not need be handled. */
+ if ( (cos = d->arch.psr_cos_ids[socket]) == 0 )
+ continue;
+
+ /*
+ * If domain uses other cos ids, all corresponding refs must have been
+ * increased 1 for this domain. So, we need decrease them.
+ */
+ info = socket_info + socket;
+ spin_lock(&info->ref_lock);
+ info->cos_ref[cos]--;
+ spin_unlock(&info->ref_lock);
+ }
+
xfree(d->arch.psr_cos_ids);
d->arch.psr_cos_ids = NULL;
}
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index 8e3c90e..d245de3 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -72,8 +72,8 @@ int psr_get_info(unsigned int socket, enum cbm_type type,
uint32_t dat[], uint32_t array_len);
int psr_get_val(struct domain *d, unsigned int socket,
uint64_t *val, enum cbm_type type);
-int psr_set_l3_cbm(struct domain *d, unsigned int socket,
- uint64_t cbm, enum cbm_type type);
+int psr_set_val(struct domain *d, unsigned int socket,
+ uint64_t val, enum cbm_type type);
int psr_domain_init(struct domain *d);
void psr_domain_free(struct domain *d);
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* Re: [PATCH v4 08/24] x86: refactor psr: set value: implement framework.
2016-12-14 4:07 ` [PATCH v4 08/24] x86: refactor psr: set value: implement framework Yi Sun
@ 2017-01-10 14:17 ` Jan Beulich
2017-01-11 5:57 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2017-01-10 14:17 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
> To make the set value flow be general and can support multiple features
> at same time, it includes below steps:
> 1. Get COS ID of current domain using.
> 2. Assemble a value array to store all features current value
> in it and replace the current value of the feature which is
> being set to the new input value.
> 3. Find if there is already a COS ID on which all features'
> values are same as the array. Then, we can reuse this COS
> ID.
> 4. If fail to find, we need allocate a new COS ID. Only COS ID which ref
> is 0 or 1 can be allocated.
Using "allocate" here in conjunction with ref count being 1 is a little
misleading here - allocation would mean a fresh ID, whereas in the
case of ref == 1 you mean to re-use the current one.
> --- a/xen/arch/x86/psr.c
> +++ b/xen/arch/x86/psr.c
> @@ -513,18 +513,197 @@ int psr_get_val(struct domain *d, unsigned int socket,
> return -ENOENT;
> }
>
> -int psr_set_l3_cbm(struct domain *d, unsigned int socket,
> - uint64_t cbm, enum cbm_type type)
> +/* Set value functions */
> +static unsigned int get_cos_num(const struct psr_socket_info *info)
> {
> return 0;
> }
>
> +static int get_old_set_new(uint64_t *val,
> + uint32_t array_len,
> + const struct psr_socket_info *info,
> + unsigned int old_cos,
> + enum cbm_type type,
> + uint64_t m)
> +{
> + return 0;
> +}
> +
> +static int find_cos(const uint64_t *val, uint32_t array_len,
> + enum cbm_type type,
> + const struct psr_socket_info *info)
> +{
> + return 0;
> +}
> +
> +static int alloc_new_cos(const struct psr_socket_info *info,
> + const uint64_t *val, uint32_t array_len,
> + unsigned int old_cos,
> + enum cbm_type type)
> +{
> + return 0;
> +}
> +
> +static int write_psr_msr(unsigned int socket, unsigned int cos,
> + const uint64_t *val)
> +{
> + return 0;
> +}
I think all of the above functions should return an error as long as
they're stubbed out, yet I don't think 0 means error in all cases (in
particular a return value of plain int suggests 0 to mean success).
> +int psr_set_val(struct domain *d, unsigned int socket,
> + uint64_t val, enum cbm_type type)
> +{
> + unsigned int old_cos;
> + int cos, ret;
> + unsigned int *ref;
> + uint64_t *val_array;
> + struct psr_socket_info *info = get_socket_info(socket);
> + uint32_t array_len;
> +
> + if ( IS_ERR(info) )
> + return PTR_ERR(info);
> +
> + /*
> + * Step 0:
> + * old_cos means the COS ID current domain is using. By default, it is 0.
> + *
> + * For every COS ID, there is a reference count to record how many domains
> + * are using the COS register corresponding to this COS ID.
> + * - If ref[old_cos] is 0, that means this COS is not used by any domain.
> + * - If ref[old_cos] is 1, that means this COS is only used by current
> + * domain.
> + * - If ref[old_cos] is more than 1, that mean multiple domains are using
> + * this COS.
> + */
> + old_cos = d->arch.psr_cos_ids[socket];
> + if ( old_cos > MAX_COS_REG_CNT )
> + return -EOVERFLOW;
> +
> + ref = info->cos_ref;
> +
> + /*
> + * Step 1:
> + * Assemle a value array to store all featues cos_reg_val[old_cos].
> + * And, set the input val into array according to the feature's
> + * position in array.
> + */
> + array_len = get_cos_num((const struct psr_socket_info *)info);
What is this cast doing here? (There are more of this kind below.)
> + val_array = xzalloc_array(uint64_t, array_len);
> + if ( !val_array )
> + return -ENOMEM;
> +
> + if ( (ret = get_old_set_new(val_array, array_len,
> + (const struct psr_socket_info *)info,
> + old_cos, type, val)) != 0 )
Just like for earlier versions I continue to be unconvinced that
the get-current-settings and the replace-target-value should be
a single operation. In particular I'd expect the function to be able
to store the target value, as long as the array entries are
ordered in a suitable way.
> + {
> + xfree(val_array);
> + return ret;
> + }
> +
> + /*
> + * Lock here to make sure the ref is not changed during find and
> + * write process.
> + */
> + spin_lock(&info->ref_lock);
> +
> + /*
> + * Step 2:
> + * Try to find if there is already a COS ID on which all features' values
> + * are same as the array. Then, we can reuse this COS ID.
> + */
> + cos = find_cos((const uint64_t *)val_array, array_len, type,
> + (const struct psr_socket_info *)info);
> + if ( cos >= 0 )
> + {
> + if ( cos == old_cos )
> + {
> + spin_unlock(&info->ref_lock);
> + xfree(val_array);
> + return 0;
> + }
> + }
> + else
> + {
> + /*
> + * Step 3:
> + * If fail to find, we need allocate a new COS ID.
> + * If multiple domains are using same COS ID, its ref is more
> + * than 1. That means we cannot free this COS to make current domain
> + * use it. Because other domains are using the value saved in the COS.
> + * Unless the ref is changed to 1 (mean only current domain is using
> + * it), we cannot allocate the COS ID to current domain.
> + * So, only the COS ID which ref is 1 or 0 can be allocated.
> + */
I suppose this comment will make more sense when further patches
get applied, as so far there was no ref count check at all anywhere.
> + cos = alloc_new_cos((const struct psr_socket_info *)info,
> + (const uint64_t *)val_array, array_len,
> + old_cos, type);
> + if ( cos < 0 )
> + {
> + spin_unlock(&info->ref_lock);
> + xfree(val_array);
> + return cos;
> + }
> +
> + /*
> + * Step 4:
> + * Write all features MSRs according to the COS ID.
> + */
> + ret = write_psr_msr(socket, cos, (const uint64_t *)val_array);
> + if ( ret )
> + {
> + spin_unlock(&info->ref_lock);
> + xfree(val_array);
> + return ret;
> + }
> + }
> +
> + /*
> + * Step 5:
> + * Update ref according to COS ID.
> + */
> + ref[cos]++;
> + ref[old_cos]--;
> + spin_unlock(&info->ref_lock);
> +
> + /*
> + * Step 6:
> + * Save the COS ID into current domain's psr_cos_ids[] so that we can know
> + * which COS the domain is using on the socket. One domain can only use
> + * one COS ID at same time.
To help readers, perhaps this last sentence should be completed with
"... on each socket"?
> + */
> + d->arch.psr_cos_ids[socket] = cos;
> + xfree(val_array);
> +
> + return 0;
> +}
> +
> /* Called with domain lock held, no extra lock needed for 'psr_cos_ids' */
> static void psr_free_cos(struct domain *d)
> {
> + unsigned int socket;
> + unsigned int cos;
These could (and imo should) be joined together.
> + struct psr_socket_info *info;
> +
> if( !d->arch.psr_cos_ids )
> return;
>
> + /* Domain is free so its cos_ref should be decreased. */
> + for( socket = 0; socket < nr_sockets; socket++ )
Missing blank (also in the if() above, not sure by which earlier patch
that got introduced).
> + {
> + /* cos 0 is default one which does not need be handled. */
> + if ( (cos = d->arch.psr_cos_ids[socket]) == 0 )
> + continue;
> +
> + /*
> + * If domain uses other cos ids, all corresponding refs must have been
> + * increased 1 for this domain. So, we need decrease them.
> + */
> + info = socket_info + socket;
> + spin_lock(&info->ref_lock);
> + info->cos_ref[cos]--;
While likely also relevant in other places, this one in particular
suggests that you should add ASSERT()s: Before decrements and
after increments the ref count should not be zero.
Also please move the declaration of at least "info" into the most
narrow scope possible.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 08/24] x86: refactor psr: set value: implement framework.
2017-01-10 14:17 ` Jan Beulich
@ 2017-01-11 5:57 ` Yi Sun
0 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2017-01-11 5:57 UTC (permalink / raw)
To: Jan Beulich
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
On 17-01-10 07:17:38, Jan Beulich wrote:
> > To make the set value flow be general and can support multiple features
> > at same time, it includes below steps:
> > 1. Get COS ID of current domain using.
> > 2. Assemble a value array to store all features current value
> > in it and replace the current value of the feature which is
> > being set to the new input value.
> > 3. Find if there is already a COS ID on which all features'
> > values are same as the array. Then, we can reuse this COS
> > ID.
> > 4. If fail to find, we need allocate a new COS ID. Only COS ID which ref
> > is 0 or 1 can be allocated.
>
> Using "allocate" here in conjunction with ref count being 1 is a little
> misleading here - allocation would mean a fresh ID, whereas in the
> case of ref == 1 you mean to re-use the current one.
>
Maybe 'pick an available COS ID' is more appropriate.
> > --- a/xen/arch/x86/psr.c
> > +++ b/xen/arch/x86/psr.c
> > @@ -513,18 +513,197 @@ int psr_get_val(struct domain *d, unsigned int socket,
> > return -ENOENT;
> > }
> >
> > -int psr_set_l3_cbm(struct domain *d, unsigned int socket,
> > - uint64_t cbm, enum cbm_type type)
> > +/* Set value functions */
> > +static unsigned int get_cos_num(const struct psr_socket_info *info)
> > {
> > return 0;
> > }
> >
> > +static int get_old_set_new(uint64_t *val,
> > + uint32_t array_len,
> > + const struct psr_socket_info *info,
> > + unsigned int old_cos,
> > + enum cbm_type type,
> > + uint64_t m)
> > +{
> > + return 0;
> > +}
> > +
> > +static int find_cos(const uint64_t *val, uint32_t array_len,
> > + enum cbm_type type,
> > + const struct psr_socket_info *info)
> > +{
> > + return 0;
> > +}
> > +
> > +static int alloc_new_cos(const struct psr_socket_info *info,
> > + const uint64_t *val, uint32_t array_len,
> > + unsigned int old_cos,
> > + enum cbm_type type)
> > +{
> > + return 0;
> > +}
> > +
> > +static int write_psr_msr(unsigned int socket, unsigned int cos,
> > + const uint64_t *val)
> > +{
> > + return 0;
> > +}
>
> I think all of the above functions should return an error as long as
> they're stubbed out, yet I don't think 0 means error in all cases (in
> particular a return value of plain int suggests 0 to mean success).
>
Thanks, will consider the return values carefully.
> > +int psr_set_val(struct domain *d, unsigned int socket,
> > + uint64_t val, enum cbm_type type)
> > +{
> > + unsigned int old_cos;
> > + int cos, ret;
> > + unsigned int *ref;
> > + uint64_t *val_array;
> > + struct psr_socket_info *info = get_socket_info(socket);
> > + uint32_t array_len;
> > +
> > + if ( IS_ERR(info) )
> > + return PTR_ERR(info);
> > +
> > + /*
> > + * Step 0:
> > + * old_cos means the COS ID current domain is using. By default, it is 0.
> > + *
> > + * For every COS ID, there is a reference count to record how many domains
> > + * are using the COS register corresponding to this COS ID.
> > + * - If ref[old_cos] is 0, that means this COS is not used by any domain.
> > + * - If ref[old_cos] is 1, that means this COS is only used by current
> > + * domain.
> > + * - If ref[old_cos] is more than 1, that mean multiple domains are using
> > + * this COS.
> > + */
> > + old_cos = d->arch.psr_cos_ids[socket];
> > + if ( old_cos > MAX_COS_REG_CNT )
> > + return -EOVERFLOW;
> > +
> > + ref = info->cos_ref;
> > +
> > + /*
> > + * Step 1:
> > + * Assemle a value array to store all featues cos_reg_val[old_cos].
> > + * And, set the input val into array according to the feature's
> > + * position in array.
> > + */
> > + array_len = get_cos_num((const struct psr_socket_info *)info);
>
> What is this cast doing here? (There are more of this kind below.)
>
Hmm, I remember there may be warning without cast. Let me confirm it. If no
warning, will remove them.
> > + val_array = xzalloc_array(uint64_t, array_len);
> > + if ( !val_array )
> > + return -ENOMEM;
> > +
> > + if ( (ret = get_old_set_new(val_array, array_len,
> > + (const struct psr_socket_info *)info,
> > + old_cos, type, val)) != 0 )
>
> Just like for earlier versions I continue to be unconvinced that
> the get-current-settings and the replace-target-value should be
> a single operation. In particular I'd expect the function to be able
> to store the target value, as long as the array entries are
> ordered in a suitable way.
>
Ok, will split to two functions. One for getting old values of all features.
The other stores the target value into array according to features position
in feature list.
> > + {
> > + xfree(val_array);
> > + return ret;
> > + }
> > +
> > + /*
> > + * Lock here to make sure the ref is not changed during find and
> > + * write process.
> > + */
> > + spin_lock(&info->ref_lock);
> > +
> > + /*
> > + * Step 2:
> > + * Try to find if there is already a COS ID on which all features' values
> > + * are same as the array. Then, we can reuse this COS ID.
> > + */
> > + cos = find_cos((const uint64_t *)val_array, array_len, type,
> > + (const struct psr_socket_info *)info);
> > + if ( cos >= 0 )
> > + {
> > + if ( cos == old_cos )
> > + {
> > + spin_unlock(&info->ref_lock);
> > + xfree(val_array);
> > + return 0;
> > + }
> > + }
> > + else
> > + {
> > + /*
> > + * Step 3:
> > + * If fail to find, we need allocate a new COS ID.
> > + * If multiple domains are using same COS ID, its ref is more
> > + * than 1. That means we cannot free this COS to make current domain
> > + * use it. Because other domains are using the value saved in the COS.
> > + * Unless the ref is changed to 1 (mean only current domain is using
> > + * it), we cannot allocate the COS ID to current domain.
> > + * So, only the COS ID which ref is 1 or 0 can be allocated.
> > + */
>
> I suppose this comment will make more sense when further patches
> get applied, as so far there was no ref count check at all anywhere.
>
Yes, the implementaion of alloc_new_cos() will show this. Comments here just
want to make readers be clear of the whole process.
> > + cos = alloc_new_cos((const struct psr_socket_info *)info,
> > + (const uint64_t *)val_array, array_len,
> > + old_cos, type);
> > + if ( cos < 0 )
> > + {
> > + spin_unlock(&info->ref_lock);
> > + xfree(val_array);
> > + return cos;
> > + }
> > +
> > + /*
> > + * Step 4:
> > + * Write all features MSRs according to the COS ID.
> > + */
> > + ret = write_psr_msr(socket, cos, (const uint64_t *)val_array);
> > + if ( ret )
> > + {
> > + spin_unlock(&info->ref_lock);
> > + xfree(val_array);
> > + return ret;
> > + }
> > + }
> > +
> > + /*
> > + * Step 5:
> > + * Update ref according to COS ID.
> > + */
> > + ref[cos]++;
> > + ref[old_cos]--;
> > + spin_unlock(&info->ref_lock);
> > +
> > + /*
> > + * Step 6:
> > + * Save the COS ID into current domain's psr_cos_ids[] so that we can know
> > + * which COS the domain is using on the socket. One domain can only use
> > + * one COS ID at same time.
>
> To help readers, perhaps this last sentence should be completed with
> "... on each socket"?
>
Thank you!
> > + */
> > + d->arch.psr_cos_ids[socket] = cos;
> > + xfree(val_array);
> > +
> > + return 0;
> > +}
> > +
> > /* Called with domain lock held, no extra lock needed for 'psr_cos_ids' */
> > static void psr_free_cos(struct domain *d)
> > {
> > + unsigned int socket;
> > + unsigned int cos;
>
> These could (and imo should) be joined together.
>
Ok, thanks!
> > + struct psr_socket_info *info;
> > +
> > if( !d->arch.psr_cos_ids )
> > return;
> >
> > + /* Domain is free so its cos_ref should be decreased. */
> > + for( socket = 0; socket < nr_sockets; socket++ )
>
> Missing blank (also in the if() above, not sure by which earlier patch
> that got introduced).
>
Oh, sorry!
> > + {
> > + /* cos 0 is default one which does not need be handled. */
> > + if ( (cos = d->arch.psr_cos_ids[socket]) == 0 )
> > + continue;
> > +
> > + /*
> > + * If domain uses other cos ids, all corresponding refs must have been
> > + * increased 1 for this domain. So, we need decrease them.
> > + */
> > + info = socket_info + socket;
> > + spin_lock(&info->ref_lock);
> > + info->cos_ref[cos]--;
>
> While likely also relevant in other places, this one in particular
> suggests that you should add ASSERT()s: Before decrements and
> after increments the ref count should not be zero.
>
Ok, thanks, will check it.
> Also please move the declaration of at least "info" into the most
> narrow scope possible.
>
Sure, will move it down.
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v4 09/24] x86: refactor psr: set value: assemble features value array.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (7 preceding siblings ...)
2016-12-14 4:07 ` [PATCH v4 08/24] x86: refactor psr: set value: implement framework Yi Sun
@ 2016-12-14 4:07 ` Yi Sun
2017-01-10 14:34 ` Jan Beulich
2016-12-14 4:07 ` [PATCH v4 10/24] x86: refactor psr: set value: implement cos finding flow Yi Sun
` (14 subsequent siblings)
23 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:07 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
Only can one COS ID be used by one domain at one time. That means all enabled
features' COS registers at this COS ID are valid for this domain at that time.
When user updates a feature's value, we need make sure all other features'
values are not affected. So, we firstly need assemble an array which contains
all features current values and replace the setting feature's value in array
to new value.
Then, we can try to find if there is a COS ID on which all features' COS
registers values are same as the array. If we can find, we just use this COS
ID. If fail to find, we need allocate a new COS ID.
This patch implements value array assembling flow.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 135 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 134 insertions(+), 1 deletion(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 838f2d1..8fcaa76 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -121,6 +121,35 @@ struct feat_ops {
/* get_val is used to get feature COS register value. */
bool (*get_val)(const struct feat_node *feat, unsigned int cos,
enum cbm_type type, uint64_t *val);
+ /*
+ * get_cos_num is used to get the COS registers amount used by the
+ * feature for one setting, e.g. CDP uses 2 COSs but CAT uses 1.
+ */
+ unsigned int (*get_cos_num)(const struct feat_node *feat);
+ /*
+ * get_old_val and set_new_val are a pair of functions called together.
+ * The caller will traverse all features in the list and call both
+ * functions for every feature to do below two things:
+ * 1. get old_cos register value of all supported features and
+ * 2. set the new value for the feature.
+ *
+ * All the values are set into value array according the traversal order,
+ * meaning the same order of feature list members.
+ *
+ * The return value is the amount of entries to skip in the value array
+ * or error.
+ * 1 - one entry in value array.
+ * 2 - two entries in value array, e.g. CDP uses two entries.
+ * negative - error.
+ */
+ int (*get_old_val)(uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int old_cos);
+ int (*set_new_val)(uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int old_cos,
+ enum cbm_type type,
+ uint64_t m);
};
@@ -186,6 +215,29 @@ static void free_feature(struct psr_socket_info *info)
}
}
+static bool_t psr_check_cbm(unsigned int cbm_len, uint64_t cbm)
+{
+ unsigned int first_bit, zero_bit;
+
+ /* Set bits should only in the range of [0, cbm_len). */
+ if ( cbm & (~0ull << cbm_len) )
+ return 0;
+
+ /* At least one bit need to be set. */
+ if ( cbm == 0 )
+ return 0;
+
+ first_bit = find_first_bit(&cbm, cbm_len);
+ zero_bit = find_next_zero_bit(&cbm, cbm_len, first_bit);
+
+ /* Set bits should be contiguous. */
+ if ( zero_bit < cbm_len &&
+ find_next_bit(&cbm, cbm_len, zero_bit) < cbm_len )
+ return 0;
+
+ return 1;
+}
+
/* L3 CAT callback functions implementation. */
static void l3_cat_init_feature(unsigned int eax, unsigned int ebx,
unsigned int ecx, unsigned int edx,
@@ -255,11 +307,53 @@ static bool l3_cat_get_val(const struct feat_node *feat, unsigned int cos,
return true;
}
+static unsigned int l3_cat_get_cos_num(const struct feat_node *feat)
+{
+ return 1;
+}
+
+static int l3_cat_get_old_val(uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int old_cos)
+{
+ if ( old_cos > feat->info.l3_cat_info.cos_max )
+ /* Use default value. */
+ old_cos = 0;
+
+ /* CAT */
+ val[0] = feat->cos_reg_val[old_cos];
+
+ /* L3 CAT uses one COS. */
+ return 1;
+}
+
+static int l3_cat_set_new_val(uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int old_cos,
+ enum cbm_type type,
+ uint64_t m)
+{
+ if ( type != PSR_CBM_TYPE_L3 )
+ /* L3 CAT uses one COS. Skip it. */
+ return 1;
+
+ if ( !psr_check_cbm(feat->info.l3_cat_info.cbm_len, m) )
+ return -EINVAL;
+
+ val[0] = m;
+
+ /* L3 CAT uses one COS. */
+ return 1;
+}
+
struct feat_ops l3_cat_ops = {
.init_feature = l3_cat_init_feature,
.get_max_cos_max = l3_cat_get_max_cos_max,
.get_feat_info = l3_cat_get_feat_info,
.get_val = l3_cat_get_val,
+ .get_cos_num = l3_cat_get_cos_num,
+ .get_old_val = l3_cat_get_old_val,
+ .set_new_val = l3_cat_set_new_val,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
@@ -516,7 +610,14 @@ int psr_get_val(struct domain *d, unsigned int socket,
/* Set value functions */
static unsigned int get_cos_num(const struct psr_socket_info *info)
{
- return 0;
+ const struct feat_node *feat_tmp;
+ unsigned int num = 0;
+
+ /* Get all features total amount. */
+ list_for_each_entry(feat_tmp, &info->feat_list, list)
+ num += feat_tmp->ops.get_cos_num(feat_tmp);
+
+ return num;
}
static int get_old_set_new(uint64_t *val,
@@ -526,6 +627,38 @@ static int get_old_set_new(uint64_t *val,
enum cbm_type type,
uint64_t m)
{
+ const struct feat_node *feat_tmp;
+ int ret;
+ uint64_t *val_tmp = val;
+
+ if ( !val )
+ return -EINVAL;
+
+ /* Get all features current values according to old_cos. */
+ list_for_each_entry(feat_tmp, &info->feat_list, list)
+ {
+ /* value getting order is same as feature list */
+ ret = feat_tmp->ops.get_old_val(val_tmp, feat_tmp, old_cos);
+
+ val_tmp += ret;
+ if ( val_tmp - val > array_len)
+ return -EINVAL;
+ }
+
+ /* Set new value into array according to feature's position in array. */
+ val_tmp = val;
+ list_for_each_entry(feat_tmp, &info->feat_list, list)
+ {
+ /* value setting order is same as feature list */
+ ret = feat_tmp->ops.set_new_val(val_tmp, feat_tmp, old_cos, type, m);
+ if ( ret < 0 )
+ return ret;
+
+ val_tmp += ret;
+ if ( val_tmp - val > array_len)
+ return -EINVAL;
+ }
+
return 0;
}
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* Re: [PATCH v4 09/24] x86: refactor psr: set value: assemble features value array.
2016-12-14 4:07 ` [PATCH v4 09/24] x86: refactor psr: set value: assemble features value array Yi Sun
@ 2017-01-10 14:34 ` Jan Beulich
2017-01-11 6:07 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2017-01-10 14:34 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> --- a/xen/arch/x86/psr.c
> +++ b/xen/arch/x86/psr.c
> @@ -121,6 +121,35 @@ struct feat_ops {
> /* get_val is used to get feature COS register value. */
> bool (*get_val)(const struct feat_node *feat, unsigned int cos,
> enum cbm_type type, uint64_t *val);
> + /*
> + * get_cos_num is used to get the COS registers amount used by the
> + * feature for one setting, e.g. CDP uses 2 COSs but CAT uses 1.
> + */
> + unsigned int (*get_cos_num)(const struct feat_node *feat);
> + /*
> + * get_old_val and set_new_val are a pair of functions called together.
> + * The caller will traverse all features in the list and call both
> + * functions for every feature to do below two things:
> + * 1. get old_cos register value of all supported features and
> + * 2. set the new value for the feature.
> + *
> + * All the values are set into value array according the traversal order,
> + * meaning the same order of feature list members.
> + *
> + * The return value is the amount of entries to skip in the value array
> + * or error.
> + * 1 - one entry in value array.
> + * 2 - two entries in value array, e.g. CDP uses two entries.
Doesn't this match the get_cos_num() return value? Would be nice to
be spelled out (and, where possible, ASSERT()ed).
> @@ -186,6 +215,29 @@ static void free_feature(struct psr_socket_info *info)
> }
> }
>
> +static bool_t psr_check_cbm(unsigned int cbm_len, uint64_t cbm)
bool (and then true/false in the function body)
> +static int l3_cat_set_new_val(uint64_t val[],
> + const struct feat_node *feat,
> + unsigned int old_cos,
> + enum cbm_type type,
> + uint64_t m)
> +{
> + if ( type != PSR_CBM_TYPE_L3 )
> + /* L3 CAT uses one COS. Skip it. */
> + return 1;
If this is the wrong type, how can you return 1 (or any positive
value) here? This basically suggests that, as recommended for an
earlier operation already, that you want the type check done in
the caller. Or wait - isn't type not matching an error here, as you
iterate the same list twice in the same order? Which then gets us
back to me mentioning that the set-new-value should really be
done in common code, not in the callbacks; it would really only be
the check (psr_check_cbm() in the L3 case above) which would
require a callback.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 09/24] x86: refactor psr: set value: assemble features value array.
2017-01-10 14:34 ` Jan Beulich
@ 2017-01-11 6:07 ` Yi Sun
2017-01-11 13:57 ` Jan Beulich
0 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2017-01-11 6:07 UTC (permalink / raw)
To: Jan Beulich
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
On 17-01-10 07:34:00, Jan Beulich wrote:
> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> > --- a/xen/arch/x86/psr.c
> > +++ b/xen/arch/x86/psr.c
> > @@ -121,6 +121,35 @@ struct feat_ops {
> > /* get_val is used to get feature COS register value. */
> > bool (*get_val)(const struct feat_node *feat, unsigned int cos,
> > enum cbm_type type, uint64_t *val);
> > + /*
> > + * get_cos_num is used to get the COS registers amount used by the
> > + * feature for one setting, e.g. CDP uses 2 COSs but CAT uses 1.
> > + */
> > + unsigned int (*get_cos_num)(const struct feat_node *feat);
> > + /*
> > + * get_old_val and set_new_val are a pair of functions called together.
> > + * The caller will traverse all features in the list and call both
> > + * functions for every feature to do below two things:
> > + * 1. get old_cos register value of all supported features and
> > + * 2. set the new value for the feature.
> > + *
> > + * All the values are set into value array according the traversal order,
> > + * meaning the same order of feature list members.
> > + *
> > + * The return value is the amount of entries to skip in the value array
> > + * or error.
> > + * 1 - one entry in value array.
> > + * 2 - two entries in value array, e.g. CDP uses two entries.
>
> Doesn't this match the get_cos_num() return value? Would be nice to
> be spelled out (and, where possible, ASSERT()ed).
>
Yes, thanks!
> > @@ -186,6 +215,29 @@ static void free_feature(struct psr_socket_info *info)
> > }
> > }
> >
> > +static bool_t psr_check_cbm(unsigned int cbm_len, uint64_t cbm)
>
> bool (and then true/false in the function body)
>
Ok, thanks!
> > +static int l3_cat_set_new_val(uint64_t val[],
> > + const struct feat_node *feat,
> > + unsigned int old_cos,
> > + enum cbm_type type,
> > + uint64_t m)
> > +{
> > + if ( type != PSR_CBM_TYPE_L3 )
> > + /* L3 CAT uses one COS. Skip it. */
> > + return 1;
>
> If this is the wrong type, how can you return 1 (or any positive
> value) here? This basically suggests that, as recommended for an
> earlier operation already, that you want the type check done in
> the caller. Or wait - isn't type not matching an error here, as you
> iterate the same list twice in the same order? Which then gets us
> back to me mentioning that the set-new-value should really be
> done in common code, not in the callbacks; it would really only be
> the check (psr_check_cbm() in the L3 case above) which would
> require a callback.
>
Your understanding is right. The value array will be iterated twice. First,
assemble all features old value into it. Second, set the target value into
array according to the target feature's position in the array. Because
different features may have different behaviors, e.g. CDP uses two entries
of the array, I think it would be better to make 'set_new_val' be a callback
function.
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 09/24] x86: refactor psr: set value: assemble features value array.
2017-01-11 6:07 ` Yi Sun
@ 2017-01-11 13:57 ` Jan Beulich
2017-01-12 1:17 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2017-01-11 13:57 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 11.01.17 at 07:07, <yi.y.sun@linux.intel.com> wrote:
> On 17-01-10 07:34:00, Jan Beulich wrote:
>> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
>> > +static int l3_cat_set_new_val(uint64_t val[],
>> > + const struct feat_node *feat,
>> > + unsigned int old_cos,
>> > + enum cbm_type type,
>> > + uint64_t m)
>> > +{
>> > + if ( type != PSR_CBM_TYPE_L3 )
>> > + /* L3 CAT uses one COS. Skip it. */
>> > + return 1;
>>
>> If this is the wrong type, how can you return 1 (or any positive
>> value) here? This basically suggests that, as recommended for an
>> earlier operation already, that you want the type check done in
>> the caller. Or wait - isn't type not matching an error here, as you
>> iterate the same list twice in the same order? Which then gets us
>> back to me mentioning that the set-new-value should really be
>> done in common code, not in the callbacks; it would really only be
>> the check (psr_check_cbm() in the L3 case above) which would
>> require a callback.
>>
> Your understanding is right. The value array will be iterated twice. First,
> assemble all features old value into it. Second, set the target value into
> array according to the target feature's position in the array. Because
> different features may have different behaviors, e.g. CDP uses two entries
> of the array, I think it would be better to make 'set_new_val' be a callback
> function.
Bot for common code to do that, all it takes is knowing the number
of registers, which you already have a callback for.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 09/24] x86: refactor psr: set value: assemble features value array.
2017-01-11 13:57 ` Jan Beulich
@ 2017-01-12 1:17 ` Yi Sun
0 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2017-01-12 1:17 UTC (permalink / raw)
To: Jan Beulich
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
On 17-01-11 06:57:30, Jan Beulich wrote:
> >>> On 11.01.17 at 07:07, <yi.y.sun@linux.intel.com> wrote:
> > On 17-01-10 07:34:00, Jan Beulich wrote:
> >> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> >> > +static int l3_cat_set_new_val(uint64_t val[],
> >> > + const struct feat_node *feat,
> >> > + unsigned int old_cos,
> >> > + enum cbm_type type,
> >> > + uint64_t m)
> >> > +{
> >> > + if ( type != PSR_CBM_TYPE_L3 )
> >> > + /* L3 CAT uses one COS. Skip it. */
> >> > + return 1;
> >>
> >> If this is the wrong type, how can you return 1 (or any positive
> >> value) here? This basically suggests that, as recommended for an
> >> earlier operation already, that you want the type check done in
> >> the caller. Or wait - isn't type not matching an error here, as you
> >> iterate the same list twice in the same order? Which then gets us
> >> back to me mentioning that the set-new-value should really be
> >> done in common code, not in the callbacks; it would really only be
> >> the check (psr_check_cbm() in the L3 case above) which would
> >> require a callback.
> >>
> > Your understanding is right. The value array will be iterated twice. First,
> > assemble all features old value into it. Second, set the target value into
> > array according to the target feature's position in the array. Because
> > different features may have different behaviors, e.g. CDP uses two entries
> > of the array, I think it would be better to make 'set_new_val' be a callback
> > function.
>
> Bot for common code to do that, all it takes is knowing the number
> of registers, which you already have a callback for.
>
Ok, will consider it. Thanks!
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v4 10/24] x86: refactor psr: set value: implement cos finding flow.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (8 preceding siblings ...)
2016-12-14 4:07 ` [PATCH v4 09/24] x86: refactor psr: set value: assemble features value array Yi Sun
@ 2016-12-14 4:07 ` Yi Sun
2017-01-10 14:53 ` Jan Beulich
2016-12-14 4:07 ` [PATCH v4 11/24] x86: refactor psr: set value: implement cos id allocation flow Yi Sun
` (13 subsequent siblings)
23 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:07 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
Continue with previous patch, we can try to find if there is a
COS ID on which all features' COS registers values are same as
the array assembled before.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 111 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 110 insertions(+), 1 deletion(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 8fcaa76..6b2b1e0 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -150,6 +150,25 @@ struct feat_ops {
unsigned int old_cos,
enum cbm_type type,
uint64_t m);
+ /*
+ * get_cos_max_from_type is used to get the cos_max value of the feature
+ * according to input type.
+ */
+ unsigned int (*get_cos_max_from_type)(const struct feat_node *feat,
+ enum cbm_type type);
+ /*
+ * compare_val is used in set value process to compare if the
+ * input value array can match all the features' COS registers values
+ * according to input cos id.
+ *
+ * The return value is the amount of entries to skip in the value array
+ * or error.
+ * 1 - one entry in value array.
+ * 2 - two entries in value array, e.g. CDP uses two entries.
+ * negative - error.
+ */
+ int (*compare_val)(const uint64_t val[], const struct feat_node *feat,
+ unsigned int cos, bool *found);
};
@@ -346,6 +365,43 @@ static int l3_cat_set_new_val(uint64_t val[],
return 1;
}
+static unsigned int l3_cat_get_cos_max_from_type(const struct feat_node *feat,
+ enum cbm_type type)
+{
+ if ( type != PSR_CBM_TYPE_L3 )
+ return 0;
+
+ return feat->info.l3_cat_info.cos_max;
+}
+
+static int l3_cat_compare_val(const uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int cos, bool *found)
+{
+ uint64_t l3_def_cbm;
+
+ l3_def_cbm = (1ull << feat->info.l3_cat_info.cbm_len) - 1;
+
+ /*
+ * Different features' cos_max are different. If cos id of the feature
+ * being set exceeds other feature's cos_max, the val of other feature
+ * must be default value. HW supports such case.
+ */
+ if ( cos > feat->info.l3_cat_info.cos_max )
+ {
+ if ( val[0] != l3_def_cbm )
+ {
+ *found = false;
+ return -ENOENT;
+ }
+ *found = true;
+ }
+ else
+ *found = (val[0] == feat->cos_reg_val[cos]);
+
+ /* L3 CAT uses one COS. */
+ return 1;
+}
struct feat_ops l3_cat_ops = {
.init_feature = l3_cat_init_feature,
.get_max_cos_max = l3_cat_get_max_cos_max,
@@ -354,6 +410,8 @@ struct feat_ops l3_cat_ops = {
.get_cos_num = l3_cat_get_cos_num,
.get_old_val = l3_cat_get_old_val,
.set_new_val = l3_cat_set_new_val,
+ .get_cos_max_from_type = l3_cat_get_cos_max_from_type,
+ .compare_val = l3_cat_compare_val,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
@@ -666,7 +724,58 @@ static int find_cos(const uint64_t *val, uint32_t array_len,
enum cbm_type type,
const struct psr_socket_info *info)
{
- return 0;
+ unsigned int cos;
+ const unsigned int *ref = info->cos_ref;
+ const struct feat_node *feat_tmp;
+ const uint64_t *val_tmp = val;
+ int ret;
+ bool found = false;
+ unsigned int cos_max = 0;
+
+ /* cos_max is the one of the feature which is being set. */
+ list_for_each_entry(feat_tmp, &info->feat_list, list)
+ {
+ cos_max = feat_tmp->ops.get_cos_max_from_type(feat_tmp, type);
+ if ( cos_max > 0 )
+ break;
+ }
+
+ for ( cos = 0; cos <= cos_max; cos++ )
+ {
+ if ( cos && !ref[cos] )
+ continue;
+
+ /* Not found, need find again from beginning. */
+ val_tmp = val;
+ list_for_each_entry(feat_tmp, &info->feat_list, list)
+ {
+ /*
+ * Compare value according to feature list order.
+ * We must follow this order because value array is assembled
+ * as this order in get_old_set_new().
+ */
+ ret = feat_tmp->ops.compare_val(val_tmp, feat_tmp, cos, &found);
+ if ( ret < 0 )
+ return ret;
+
+ /* If fail to match, go to next cos to compare. */
+ if ( !found )
+ break;
+
+ val_tmp += ret;
+ if ( val_tmp - val > array_len )
+ return -EINVAL;
+ }
+
+ /*
+ * With this cos id, every entry of value array can match. This cos
+ * is what we find.
+ */
+ if ( found )
+ return cos;
+ }
+
+ return -ENOENT;
}
static int alloc_new_cos(const struct psr_socket_info *info,
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* Re: [PATCH v4 10/24] x86: refactor psr: set value: implement cos finding flow.
2016-12-14 4:07 ` [PATCH v4 10/24] x86: refactor psr: set value: implement cos finding flow Yi Sun
@ 2017-01-10 14:53 ` Jan Beulich
2017-01-11 6:10 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2017-01-10 14:53 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> Continue with previous patch, we can try to find if there is a
Please take into consideration that a series may be applied in small
steps. References such as "previous patch" are thus possibly
meaningless. Please instead refer to the patch by title. Also I think
you mean "continue from ...".
> @@ -666,7 +724,58 @@ static int find_cos(const uint64_t *val, uint32_t array_len,
> enum cbm_type type,
> const struct psr_socket_info *info)
> {
> - return 0;
> + unsigned int cos;
> + const unsigned int *ref = info->cos_ref;
> + const struct feat_node *feat_tmp;
> + const uint64_t *val_tmp = val;
> + int ret;
> + bool found = false;
> + unsigned int cos_max = 0;
> +
> + /* cos_max is the one of the feature which is being set. */
> + list_for_each_entry(feat_tmp, &info->feat_list, list)
> + {
> + cos_max = feat_tmp->ops.get_cos_max_from_type(feat_tmp, type);
> + if ( cos_max > 0 )
> + break;
> + }
> +
> + for ( cos = 0; cos <= cos_max; cos++ )
> + {
> + if ( cos && !ref[cos] )
> + continue;
> +
> + /* Not found, need find again from beginning. */
> + val_tmp = val;
> + list_for_each_entry(feat_tmp, &info->feat_list, list)
> + {
> + /*
> + * Compare value according to feature list order.
> + * We must follow this order because value array is assembled
> + * as this order in get_old_set_new().
> + */
> + ret = feat_tmp->ops.compare_val(val_tmp, feat_tmp, cos, &found);
> + if ( ret < 0 )
> + return ret;
> +
> + /* If fail to match, go to next cos to compare. */
> + if ( !found )
> + break;
> +
> + val_tmp += ret;
> + if ( val_tmp - val > array_len )
> + return -EINVAL;
> + }
> +
> + /*
> + * With this cos id, every entry of value array can match. This cos
> + * is what we find.
> + */
"can match" seems rather misleading to me. I think you mean
something like "For this COS ID all entries in the values array did
match. Use it."
Other than that various of the comments given for earlier patches
apply here, in particular the fact that the type matching should
move out of the hook functions.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 10/24] x86: refactor psr: set value: implement cos finding flow.
2017-01-10 14:53 ` Jan Beulich
@ 2017-01-11 6:10 ` Yi Sun
0 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2017-01-11 6:10 UTC (permalink / raw)
To: Jan Beulich
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, chao.p.peng, xen-devel
On 17-01-10 07:53:07, Jan Beulich wrote:
> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> > Continue with previous patch, we can try to find if there is a
>
> Please take into consideration that a series may be applied in small
> steps. References such as "previous patch" are thus possibly
> meaningless. Please instead refer to the patch by title. Also I think
> you mean "continue from ...".
>
Thank you!
> > @@ -666,7 +724,58 @@ static int find_cos(const uint64_t *val, uint32_t array_len,
> > enum cbm_type type,
> > const struct psr_socket_info *info)
> > {
> > - return 0;
> > + unsigned int cos;
> > + const unsigned int *ref = info->cos_ref;
> > + const struct feat_node *feat_tmp;
> > + const uint64_t *val_tmp = val;
> > + int ret;
> > + bool found = false;
> > + unsigned int cos_max = 0;
> > +
> > + /* cos_max is the one of the feature which is being set. */
> > + list_for_each_entry(feat_tmp, &info->feat_list, list)
> > + {
> > + cos_max = feat_tmp->ops.get_cos_max_from_type(feat_tmp, type);
> > + if ( cos_max > 0 )
> > + break;
> > + }
> > +
> > + for ( cos = 0; cos <= cos_max; cos++ )
> > + {
> > + if ( cos && !ref[cos] )
> > + continue;
> > +
> > + /* Not found, need find again from beginning. */
> > + val_tmp = val;
> > + list_for_each_entry(feat_tmp, &info->feat_list, list)
> > + {
> > + /*
> > + * Compare value according to feature list order.
> > + * We must follow this order because value array is assembled
> > + * as this order in get_old_set_new().
> > + */
> > + ret = feat_tmp->ops.compare_val(val_tmp, feat_tmp, cos, &found);
> > + if ( ret < 0 )
> > + return ret;
> > +
> > + /* If fail to match, go to next cos to compare. */
> > + if ( !found )
> > + break;
> > +
> > + val_tmp += ret;
> > + if ( val_tmp - val > array_len )
> > + return -EINVAL;
> > + }
> > +
> > + /*
> > + * With this cos id, every entry of value array can match. This cos
> > + * is what we find.
> > + */
>
> "can match" seems rather misleading to me. I think you mean
> something like "For this COS ID all entries in the values array did
> match. Use it."
>
Yes, sorry for the wording.
> Other than that various of the comments given for earlier patches
> apply here, in particular the fact that the type matching should
> move out of the hook functions.
>
Will try this. Thanks!
> Jan
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v4 11/24] x86: refactor psr: set value: implement cos id allocation flow.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (9 preceding siblings ...)
2016-12-14 4:07 ` [PATCH v4 10/24] x86: refactor psr: set value: implement cos finding flow Yi Sun
@ 2016-12-14 4:07 ` Yi Sun
2017-01-10 15:08 ` Jan Beulich
2016-12-14 4:07 ` [PATCH v4 12/24] x86: refactor psr: set value: implement write msr flow Yi Sun
` (12 subsequent siblings)
23 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:07 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
Continue with previous patches, if fail to find a COS ID, we need allocate
a new COS ID for domain. Only COS ID that ref[COS_ID] is 1 or 0 can be
allocated to input a new set feature values.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 105 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 104 insertions(+), 1 deletion(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 6b2b1e0..ac98c39 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -169,6 +169,23 @@ struct feat_ops {
*/
int (*compare_val)(const uint64_t val[], const struct feat_node *feat,
unsigned int cos, bool *found);
+ /*
+ * exceeds_cos_max is used to check if the input cos id exceeds the
+ * feature's cos_max and if the input value is not the default one.
+ * Even if the associated cos exceeds the cos_max, HW can work with default
+ * value. That is the reason we need check if input value is default one.
+ * If both criteria are fulfilled, that means the input exceeds the
+ * range.
+ *
+ * The return value of the function means the number of the value array
+ * entries to skip or error.
+ * 1 - one entry in value array.
+ * 2 - two entries in value array, e.g. CDP uses two entries.
+ * 0 - error, exceed cos_max and the input value is not default.
+ */
+ unsigned int (*exceeds_cos_max)(const uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int cos);
};
@@ -402,6 +419,28 @@ static int l3_cat_compare_val(const uint64_t val[],
/* L3 CAT uses one COS. */
return 1;
}
+
+static unsigned int l3_cat_exceeds_cos_max(const uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int cos)
+{
+ uint64_t l3_def_cbm;
+
+ l3_def_cbm = (1ull << feat->info.l3_cat_info.cbm_len) - 1;
+
+ /* CAT */
+ if ( cos > feat->info.l3_cat_info.cos_max &&
+ val[0] != l3_def_cbm )
+ /*
+ * Exceed cos_max and value to set is not default,
+ * return error.
+ */
+ return 0;
+
+ /* L3 CAT uses one COS. */
+ return 1;
+}
+
struct feat_ops l3_cat_ops = {
.init_feature = l3_cat_init_feature,
.get_max_cos_max = l3_cat_get_max_cos_max,
@@ -412,6 +451,7 @@ struct feat_ops l3_cat_ops = {
.set_new_val = l3_cat_set_new_val,
.get_cos_max_from_type = l3_cat_get_cos_max_from_type,
.compare_val = l3_cat_compare_val,
+ .exceeds_cos_max = l3_cat_exceeds_cos_max,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
@@ -778,12 +818,75 @@ static int find_cos(const uint64_t *val, uint32_t array_len,
return -ENOENT;
}
+static bool exceeds_cos_max(const uint64_t *val,
+ uint32_t array_len,
+ const struct psr_socket_info *info,
+ unsigned int cos)
+{
+ unsigned int ret;
+ const uint64_t *val_tmp = val;
+ const struct feat_node *feat_tmp;
+
+ list_for_each_entry(feat_tmp, &info->feat_list, list)
+ {
+ ret = feat_tmp->ops.exceeds_cos_max(val_tmp, feat_tmp, cos);
+ if ( !ret )
+ return false;
+
+ val_tmp += ret;
+ if ( val_tmp - val > array_len )
+ return false;
+ }
+
+ return true;
+}
+
static int alloc_new_cos(const struct psr_socket_info *info,
const uint64_t *val, uint32_t array_len,
unsigned int old_cos,
enum cbm_type type)
{
- return 0;
+ unsigned int cos;
+ unsigned int cos_max = 0;
+ const struct feat_node *feat_tmp;
+ const unsigned int *ref = info->cos_ref;
+
+ /*
+ * cos_max is the one of the feature which is being set.
+ */
+ list_for_each_entry(feat_tmp, &info->feat_list, list)
+ {
+ cos_max = feat_tmp->ops.get_cos_max_from_type(feat_tmp, type);
+ if ( cos_max > 0 )
+ break;
+ }
+
+ if ( !cos_max )
+ return -ENOENT;
+
+ /*
+ * If old cos is referred only by the domain, then use it. And, we cannot
+ * use id 0 because it stores the default values.
+ */
+ if ( ref[old_cos] == 1 && old_cos )
+ if ( exceeds_cos_max(val, array_len, info, old_cos) )
+ return old_cos;
+
+ /* Find an unused one other than cos0. */
+ for ( cos = 1; cos <= cos_max; cos++ )
+ /*
+ * ref is 0 means this COS is not used by other domain and
+ * can be used for current setting.
+ */
+ if ( !ref[cos] )
+ {
+ if ( !exceeds_cos_max(val, array_len, info, cos) )
+ return -ENOENT;
+
+ return cos;
+ }
+
+ return -ENOENT;
}
static int write_psr_msr(unsigned int socket, unsigned int cos,
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* Re: [PATCH v4 11/24] x86: refactor psr: set value: implement cos id allocation flow.
2016-12-14 4:07 ` [PATCH v4 11/24] x86: refactor psr: set value: implement cos id allocation flow Yi Sun
@ 2017-01-10 15:08 ` Jan Beulich
2017-01-11 6:16 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2017-01-10 15:08 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> --- a/xen/arch/x86/psr.c
> +++ b/xen/arch/x86/psr.c
> @@ -169,6 +169,23 @@ struct feat_ops {
> */
> int (*compare_val)(const uint64_t val[], const struct feat_node *feat,
> unsigned int cos, bool *found);
> + /*
> + * exceeds_cos_max is used to check if the input cos id exceeds the
> + * feature's cos_max and if the input value is not the default one.
> + * Even if the associated cos exceeds the cos_max, HW can work with default
> + * value. That is the reason we need check if input value is default one.
> + * If both criteria are fulfilled, that means the input exceeds the
> + * range.
Isn't this last sentence the wrong way round?
> + * The return value of the function means the number of the value array
> + * entries to skip or error.
> + * 1 - one entry in value array.
> + * 2 - two entries in value array, e.g. CDP uses two entries.
> + * 0 - error, exceed cos_max and the input value is not default.
> + */
> + unsigned int (*exceeds_cos_max)(const uint64_t val[],
> + const struct feat_node *feat,
> + unsigned int cos);
IIRC I did recommend "exceeds" in the name during earlier review,
but also iirc the semantics of the call were different back then.
Please try to name functions such that they describe themselves
in at least a minimalistic ways. My main issue with this naming is
that the function returning non-zero (i.e. true in C meaning within
conditionals) means "no" here rather than "yes". fits_cos_max()
would therefore be a possibly better fit.
> +static bool exceeds_cos_max(const uint64_t *val,
> + uint32_t array_len,
> + const struct psr_socket_info *info,
> + unsigned int cos)
> +{
> + unsigned int ret;
> + const uint64_t *val_tmp = val;
> + const struct feat_node *feat_tmp;
> +
> + list_for_each_entry(feat_tmp, &info->feat_list, list)
> + {
> + ret = feat_tmp->ops.exceeds_cos_max(val_tmp, feat_tmp, cos);
> + if ( !ret )
> + return false;
> +
> + val_tmp += ret;
> + if ( val_tmp - val > array_len )
> + return false;
> + }
> +
> + return true;
> +}
Similarly here - name and return value don't fit together; I think
you want to invert the return values or (along the lines above)
rename the function.
> static int alloc_new_cos(const struct psr_socket_info *info,
> const uint64_t *val, uint32_t array_len,
> unsigned int old_cos,
> enum cbm_type type)
> {
> - return 0;
> + unsigned int cos;
> + unsigned int cos_max = 0;
> + const struct feat_node *feat_tmp;
> + const unsigned int *ref = info->cos_ref;
> +
> + /*
> + * cos_max is the one of the feature which is being set.
> + */
> + list_for_each_entry(feat_tmp, &info->feat_list, list)
> + {
> + cos_max = feat_tmp->ops.get_cos_max_from_type(feat_tmp, type);
> + if ( cos_max > 0 )
> + break;
> + }
> +
> + if ( !cos_max )
> + return -ENOENT;
> +
> + /*
> + * If old cos is referred only by the domain, then use it. And, we cannot
> + * use id 0 because it stores the default values.
> + */
> + if ( ref[old_cos] == 1 && old_cos )
Please swap both sides of && - cheaper checks should come first if
possible.
> + if ( exceeds_cos_max(val, array_len, info, old_cos) )
Also please fold the two if()-s.
> + return old_cos;
And then, as indicated before, this part is not really an allocation,
but a re-use, so would likely better move into the caller (or the
function's name should be adjusted).
> + /* Find an unused one other than cos0. */
> + for ( cos = 1; cos <= cos_max; cos++ )
> + /*
> + * ref is 0 means this COS is not used by other domain and
> + * can be used for current setting.
> + */
> + if ( !ref[cos] )
> + {
> + if ( !exceeds_cos_max(val, array_len, info, cos) )
> + return -ENOENT;
> +
> + return cos;
> + }
While a comment is just white space, this looks suspicious without
another pair of braces around the for() body.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 11/24] x86: refactor psr: set value: implement cos id allocation flow.
2017-01-10 15:08 ` Jan Beulich
@ 2017-01-11 6:16 ` Yi Sun
0 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2017-01-11 6:16 UTC (permalink / raw)
To: Jan Beulich
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
On 17-01-10 08:08:19, Jan Beulich wrote:
> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> > --- a/xen/arch/x86/psr.c
> > +++ b/xen/arch/x86/psr.c
> > @@ -169,6 +169,23 @@ struct feat_ops {
> > */
> > int (*compare_val)(const uint64_t val[], const struct feat_node *feat,
> > unsigned int cos, bool *found);
> > + /*
> > + * exceeds_cos_max is used to check if the input cos id exceeds the
> > + * feature's cos_max and if the input value is not the default one.
> > + * Even if the associated cos exceeds the cos_max, HW can work with default
> > + * value. That is the reason we need check if input value is default one.
> > + * If both criteria are fulfilled, that means the input exceeds the
> > + * range.
>
> Isn't this last sentence the wrong way round?
>
Sorry.
> > + * The return value of the function means the number of the value array
> > + * entries to skip or error.
> > + * 1 - one entry in value array.
> > + * 2 - two entries in value array, e.g. CDP uses two entries.
> > + * 0 - error, exceed cos_max and the input value is not default.
> > + */
> > + unsigned int (*exceeds_cos_max)(const uint64_t val[],
> > + const struct feat_node *feat,
> > + unsigned int cos);
>
> IIRC I did recommend "exceeds" in the name during earlier review,
> but also iirc the semantics of the call were different back then.
> Please try to name functions such that they describe themselves
> in at least a minimalistic ways. My main issue with this naming is
> that the function returning non-zero (i.e. true in C meaning within
> conditionals) means "no" here rather than "yes". fits_cos_max()
> would therefore be a possibly better fit.
>
Thanks for the suggestion!
> > +static bool exceeds_cos_max(const uint64_t *val,
> > + uint32_t array_len,
> > + const struct psr_socket_info *info,
> > + unsigned int cos)
> > +{
> > + unsigned int ret;
> > + const uint64_t *val_tmp = val;
> > + const struct feat_node *feat_tmp;
> > +
> > + list_for_each_entry(feat_tmp, &info->feat_list, list)
> > + {
> > + ret = feat_tmp->ops.exceeds_cos_max(val_tmp, feat_tmp, cos);
> > + if ( !ret )
> > + return false;
> > +
> > + val_tmp += ret;
> > + if ( val_tmp - val > array_len )
> > + return false;
> > + }
> > +
> > + return true;
> > +}
>
> Similarly here - name and return value don't fit together; I think
> you want to invert the return values or (along the lines above)
> rename the function.
>
Will rename the callback function to make it accurate. Thanks!
> > static int alloc_new_cos(const struct psr_socket_info *info,
> > const uint64_t *val, uint32_t array_len,
> > unsigned int old_cos,
> > enum cbm_type type)
> > {
> > - return 0;
> > + unsigned int cos;
> > + unsigned int cos_max = 0;
> > + const struct feat_node *feat_tmp;
> > + const unsigned int *ref = info->cos_ref;
> > +
> > + /*
> > + * cos_max is the one of the feature which is being set.
> > + */
> > + list_for_each_entry(feat_tmp, &info->feat_list, list)
> > + {
> > + cos_max = feat_tmp->ops.get_cos_max_from_type(feat_tmp, type);
> > + if ( cos_max > 0 )
> > + break;
> > + }
> > +
> > + if ( !cos_max )
> > + return -ENOENT;
> > +
> > + /*
> > + * If old cos is referred only by the domain, then use it. And, we cannot
> > + * use id 0 because it stores the default values.
> > + */
> > + if ( ref[old_cos] == 1 && old_cos )
>
> Please swap both sides of && - cheaper checks should come first if
> possible.
>
Sure, thanks!
> > + if ( exceeds_cos_max(val, array_len, info, old_cos) )
>
> Also please fold the two if()-s.
>
Ok, thanks!
> > + return old_cos;
>
> And then, as indicated before, this part is not really an allocation,
> but a re-use, so would likely better move into the caller (or the
> function's name should be adjusted).
>
Prefer to change function name to 'pick_avail_cos'.
> > + /* Find an unused one other than cos0. */
> > + for ( cos = 1; cos <= cos_max; cos++ )
> > + /*
> > + * ref is 0 means this COS is not used by other domain and
> > + * can be used for current setting.
> > + */
> > + if ( !ref[cos] )
> > + {
> > + if ( !exceeds_cos_max(val, array_len, info, cos) )
> > + return -ENOENT;
> > +
> > + return cos;
> > + }
>
> While a comment is just white space, this looks suspicious without
> another pair of braces around the for() body.
>
Sure.
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v4 12/24] x86: refactor psr: set value: implement write msr flow.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (10 preceding siblings ...)
2016-12-14 4:07 ` [PATCH v4 11/24] x86: refactor psr: set value: implement cos id allocation flow Yi Sun
@ 2016-12-14 4:07 ` Yi Sun
2017-01-10 15:15 ` Jan Beulich
2016-12-14 4:07 ` [PATCH v4 13/24] x86: refactor psr: implement CPU init and free flow for CDP Yi Sun
` (11 subsequent siblings)
23 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:07 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
Continue with previous patches, we have got all features values and
COS ID to set. Then, we write MSRs of all features except the setting
value is same as original value.
Till now, set value process is completed.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 78 insertions(+)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index ac98c39..ec757a2 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -186,6 +186,9 @@ struct feat_ops {
unsigned int (*exceeds_cos_max)(const uint64_t val[],
const struct feat_node *feat,
unsigned int cos);
+ /* write_msr is used to write out feature MSR register. */
+ int (*write_msr)(unsigned int cos, const uint64_t val[],
+ struct feat_node *feat);
};
@@ -441,6 +444,22 @@ static unsigned int l3_cat_exceeds_cos_max(const uint64_t val[],
return 1;
}
+static int l3_cat_write_msr(unsigned int cos, const uint64_t val[],
+ struct feat_node *feat)
+{
+ if ( cos > feat->info.l3_cat_info.cos_max )
+ /* L3 CAT uses one COS. */
+ return 1;
+
+ if ( feat->cos_reg_val[cos] != val[0] )
+ {
+ feat->cos_reg_val[cos] = val[0];
+ wrmsrl(MSR_IA32_PSR_L3_MASK(cos), val[0]);
+ }
+ /* L3 CAT uses one COS. */
+ return 1;
+}
+
struct feat_ops l3_cat_ops = {
.init_feature = l3_cat_init_feature,
.get_max_cos_max = l3_cat_get_max_cos_max,
@@ -452,6 +471,7 @@ struct feat_ops l3_cat_ops = {
.get_cos_max_from_type = l3_cat_get_cos_max_from_type,
.compare_val = l3_cat_compare_val,
.exceeds_cos_max = l3_cat_exceeds_cos_max,
+ .write_msr = l3_cat_write_msr,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
@@ -889,9 +909,67 @@ static int alloc_new_cos(const struct psr_socket_info *info,
return -ENOENT;
}
+static unsigned int get_socket_cpu(unsigned int socket)
+{
+ if ( likely(socket < nr_sockets) )
+ return cpumask_any(socket_cpumask[socket]);
+
+ return nr_cpu_ids;
+}
+
+struct cos_write_info
+{
+ unsigned int cos;
+ struct list_head *feat_list;
+ const uint64_t *val;
+};
+
+static void do_write_psr_msr(void *data)
+{
+ struct cos_write_info *info = (struct cos_write_info *)data;
+ unsigned int cos = info->cos;
+ struct list_head *feat_list= info->feat_list;
+ const uint64_t *val = info->val;
+ struct feat_node *feat_tmp;
+ int ret;
+
+ if ( !feat_list )
+ return;
+
+ /* We need set all features values into MSRs. */
+ list_for_each_entry(feat_tmp, feat_list, list)
+ {
+ ret = feat_tmp->ops.write_msr(cos, val, feat_tmp);
+ if ( ret <= 0)
+ return;
+
+ val += ret;
+ }
+}
+
static int write_psr_msr(unsigned int socket, unsigned int cos,
const uint64_t *val)
{
+ struct psr_socket_info *info = get_socket_info(socket);
+
+ struct cos_write_info data =
+ {
+ .cos = cos,
+ .feat_list = &info->feat_list,
+ .val = val,
+ };
+
+ if ( socket == cpu_to_socket(smp_processor_id()) )
+ do_write_psr_msr(&data);
+ else
+ {
+ unsigned int cpu = get_socket_cpu(socket);
+
+ if ( cpu >= nr_cpu_ids )
+ return -ENOTSOCK;
+ on_selected_cpus(cpumask_of(cpu), do_write_psr_msr, &data, 1);
+ }
+
return 0;
}
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* Re: [PATCH v4 12/24] x86: refactor psr: set value: implement write msr flow.
2016-12-14 4:07 ` [PATCH v4 12/24] x86: refactor psr: set value: implement write msr flow Yi Sun
@ 2017-01-10 15:15 ` Jan Beulich
2017-01-11 6:22 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2017-01-10 15:15 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> --- a/xen/arch/x86/psr.c
> +++ b/xen/arch/x86/psr.c
> @@ -186,6 +186,9 @@ struct feat_ops {
> unsigned int (*exceeds_cos_max)(const uint64_t val[],
> const struct feat_node *feat,
> unsigned int cos);
> + /* write_msr is used to write out feature MSR register. */
> + int (*write_msr)(unsigned int cos, const uint64_t val[],
> + struct feat_node *feat);
Looks like this function again returns number-of-values, yet this time
without a comment saying so. While you don't need to replicate
that description multiple time, please at least has a brief reference.
That said, with the type checks moved out I think this return value
model won't be needed anymore - the caller, having checked the
type, could then simply call the get-num-val (or however it was
named) hook to know how many array entries to skip.
> @@ -889,9 +909,67 @@ static int alloc_new_cos(const struct psr_socket_info *info,
> return -ENOENT;
> }
>
> +static unsigned int get_socket_cpu(unsigned int socket)
> +{
> + if ( likely(socket < nr_sockets) )
> + return cpumask_any(socket_cpumask[socket]);
> +
> + return nr_cpu_ids;
> +}
> +
> +struct cos_write_info
> +{
> + unsigned int cos;
> + struct list_head *feat_list;
> + const uint64_t *val;
> +};
> +
> +static void do_write_psr_msr(void *data)
> +{
> + struct cos_write_info *info = (struct cos_write_info *)data;
> + unsigned int cos = info->cos;
> + struct list_head *feat_list= info->feat_list;
> + const uint64_t *val = info->val;
> + struct feat_node *feat_tmp;
> + int ret;
> +
> + if ( !feat_list )
> + return;
> +
> + /* We need set all features values into MSRs. */
> + list_for_each_entry(feat_tmp, feat_list, list)
> + {
> + ret = feat_tmp->ops.write_msr(cos, val, feat_tmp);
> + if ( ret <= 0)
Missing blank.
> + return;
> +
> + val += ret;
> + }
> +}
> +
> static int write_psr_msr(unsigned int socket, unsigned int cos,
> const uint64_t *val)
> {
> + struct psr_socket_info *info = get_socket_info(socket);
> +
> + struct cos_write_info data =
No blank lines between declarations please (unless there are
extraordinarily many).
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 12/24] x86: refactor psr: set value: implement write msr flow.
2017-01-10 15:15 ` Jan Beulich
@ 2017-01-11 6:22 ` Yi Sun
2017-01-11 14:01 ` Jan Beulich
0 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2017-01-11 6:22 UTC (permalink / raw)
To: Jan Beulich
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
On 17-01-10 08:15:15, Jan Beulich wrote:
> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> > --- a/xen/arch/x86/psr.c
> > +++ b/xen/arch/x86/psr.c
> > @@ -186,6 +186,9 @@ struct feat_ops {
> > unsigned int (*exceeds_cos_max)(const uint64_t val[],
> > const struct feat_node *feat,
> > unsigned int cos);
> > + /* write_msr is used to write out feature MSR register. */
> > + int (*write_msr)(unsigned int cos, const uint64_t val[],
> > + struct feat_node *feat);
>
> Looks like this function again returns number-of-values, yet this time
> without a comment saying so. While you don't need to replicate
> that description multiple time, please at least has a brief reference.
> That said, with the type checks moved out I think this return value
> model won't be needed anymore - the caller, having checked the
> type, could then simply call the get-num-val (or however it was
> named) hook to know how many array entries to skip.
>
For write msr, we may need iterate the whole feature list to write values for
every feature if the input value is not same as old on the COS ID. So, I prefer
to keep current return value, the number-of-values handled. That would be clear
and easy to implement. Of course, we can call get_cos_num to get the returen
value or define a macro to replace the digit. How do you think?
> > @@ -889,9 +909,67 @@ static int alloc_new_cos(const struct psr_socket_info *info,
> > return -ENOENT;
> > }
> >
> > +static unsigned int get_socket_cpu(unsigned int socket)
> > +{
> > + if ( likely(socket < nr_sockets) )
> > + return cpumask_any(socket_cpumask[socket]);
> > +
> > + return nr_cpu_ids;
> > +}
> > +
> > +struct cos_write_info
> > +{
> > + unsigned int cos;
> > + struct list_head *feat_list;
> > + const uint64_t *val;
> > +};
> > +
> > +static void do_write_psr_msr(void *data)
> > +{
> > + struct cos_write_info *info = (struct cos_write_info *)data;
> > + unsigned int cos = info->cos;
> > + struct list_head *feat_list= info->feat_list;
> > + const uint64_t *val = info->val;
> > + struct feat_node *feat_tmp;
> > + int ret;
> > +
> > + if ( !feat_list )
> > + return;
> > +
> > + /* We need set all features values into MSRs. */
> > + list_for_each_entry(feat_tmp, feat_list, list)
> > + {
> > + ret = feat_tmp->ops.write_msr(cos, val, feat_tmp);
> > + if ( ret <= 0)
>
> Missing blank.
>
Sorry.
> > + return;
> > +
> > + val += ret;
> > + }
> > +}
> > +
> > static int write_psr_msr(unsigned int socket, unsigned int cos,
> > const uint64_t *val)
> > {
> > + struct psr_socket_info *info = get_socket_info(socket);
> > +
> > + struct cos_write_info data =
>
> No blank lines between declarations please (unless there are
> extraordinarily many).
>
Ok, thanks!
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 12/24] x86: refactor psr: set value: implement write msr flow.
2017-01-11 6:22 ` Yi Sun
@ 2017-01-11 14:01 ` Jan Beulich
2017-01-12 1:22 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2017-01-11 14:01 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 11.01.17 at 07:22, <yi.y.sun@linux.intel.com> wrote:
> On 17-01-10 08:15:15, Jan Beulich wrote:
>> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
>> > --- a/xen/arch/x86/psr.c
>> > +++ b/xen/arch/x86/psr.c
>> > @@ -186,6 +186,9 @@ struct feat_ops {
>> > unsigned int (*exceeds_cos_max)(const uint64_t val[],
>> > const struct feat_node *feat,
>> > unsigned int cos);
>> > + /* write_msr is used to write out feature MSR register. */
>> > + int (*write_msr)(unsigned int cos, const uint64_t val[],
>> > + struct feat_node *feat);
>>
>> Looks like this function again returns number-of-values, yet this time
>> without a comment saying so. While you don't need to replicate
>> that description multiple time, please at least has a brief reference.
>> That said, with the type checks moved out I think this return value
>> model won't be needed anymore - the caller, having checked the
>> type, could then simply call the get-num-val (or however it was
>> named) hook to know how many array entries to skip.
>>
> For write msr, we may need iterate the whole feature list to write values for
> every feature if the input value is not same as old on the COS ID. So, I prefer
> to keep current return value, the number-of-values handled. That would be clear
> and easy to implement. Of course, we can call get_cos_num to get the returen
> value or define a macro to replace the digit. How do you think?
Well, my general reservation here is that this way you require about
half a dozen functions to all return the same value. If the value
changes (or if somebody clones the set), there's the risk of one not
getting properly updated. Therefore I'd much prefer for just one
function to return the count. And I'm relatively certain that with the
type checks moved out, this will actually end up being the more
natural way.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 12/24] x86: refactor psr: set value: implement write msr flow.
2017-01-11 14:01 ` Jan Beulich
@ 2017-01-12 1:22 ` Yi Sun
2017-01-12 9:40 ` Jan Beulich
0 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2017-01-12 1:22 UTC (permalink / raw)
To: Jan Beulich
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
On 17-01-11 07:01:23, Jan Beulich wrote:
> >>> On 11.01.17 at 07:22, <yi.y.sun@linux.intel.com> wrote:
> > On 17-01-10 08:15:15, Jan Beulich wrote:
> >> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> >> > --- a/xen/arch/x86/psr.c
> >> > +++ b/xen/arch/x86/psr.c
> >> > @@ -186,6 +186,9 @@ struct feat_ops {
> >> > unsigned int (*exceeds_cos_max)(const uint64_t val[],
> >> > const struct feat_node *feat,
> >> > unsigned int cos);
> >> > + /* write_msr is used to write out feature MSR register. */
> >> > + int (*write_msr)(unsigned int cos, const uint64_t val[],
> >> > + struct feat_node *feat);
> >>
> >> Looks like this function again returns number-of-values, yet this time
> >> without a comment saying so. While you don't need to replicate
> >> that description multiple time, please at least has a brief reference.
> >> That said, with the type checks moved out I think this return value
> >> model won't be needed anymore - the caller, having checked the
> >> type, could then simply call the get-num-val (or however it was
> >> named) hook to know how many array entries to skip.
> >>
> > For write msr, we may need iterate the whole feature list to write values for
> > every feature if the input value is not same as old on the COS ID. So, I prefer
> > to keep current return value, the number-of-values handled. That would be clear
> > and easy to implement. Of course, we can call get_cos_num to get the returen
> > value or define a macro to replace the digit. How do you think?
>
> Well, my general reservation here is that this way you require about
> half a dozen functions to all return the same value. If the value
> changes (or if somebody clones the set), there's the risk of one not
> getting properly updated. Therefore I'd much prefer for just one
> function to return the count. And I'm relatively certain that with the
> type checks moved out, this will actually end up being the more
> natural way.
>
I imagine the way as your suggestion. It might be below flow for this write_msr.
list_for_each_entry(feat...) {
feat->write_msr(..., val_array);
val_array += feat->get_cos_num();
......
}
Is that what you think? Thanks!
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 12/24] x86: refactor psr: set value: implement write msr flow.
2017-01-12 1:22 ` Yi Sun
@ 2017-01-12 9:40 ` Jan Beulich
2017-01-12 10:22 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2017-01-12 9:40 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 12.01.17 at 02:22, <yi.y.sun@linux.intel.com> wrote:
> On 17-01-11 07:01:23, Jan Beulich wrote:
>> >>> On 11.01.17 at 07:22, <yi.y.sun@linux.intel.com> wrote:
>> > On 17-01-10 08:15:15, Jan Beulich wrote:
>> >> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
>> >> > --- a/xen/arch/x86/psr.c
>> >> > +++ b/xen/arch/x86/psr.c
>> >> > @@ -186,6 +186,9 @@ struct feat_ops {
>> >> > unsigned int (*exceeds_cos_max)(const uint64_t val[],
>> >> > const struct feat_node *feat,
>> >> > unsigned int cos);
>> >> > + /* write_msr is used to write out feature MSR register. */
>> >> > + int (*write_msr)(unsigned int cos, const uint64_t val[],
>> >> > + struct feat_node *feat);
>> >>
>> >> Looks like this function again returns number-of-values, yet this time
>> >> without a comment saying so. While you don't need to replicate
>> >> that description multiple time, please at least has a brief reference.
>> >> That said, with the type checks moved out I think this return value
>> >> model won't be needed anymore - the caller, having checked the
>> >> type, could then simply call the get-num-val (or however it was
>> >> named) hook to know how many array entries to skip.
>> >>
>> > For write msr, we may need iterate the whole feature list to write values for
>> > every feature if the input value is not same as old on the COS ID. So, I prefer
>> > to keep current return value, the number-of-values handled. That would be clear
>> > and easy to implement. Of course, we can call get_cos_num to get the returen
>> > value or define a macro to replace the digit. How do you think?
>>
>> Well, my general reservation here is that this way you require about
>> half a dozen functions to all return the same value. If the value
>> changes (or if somebody clones the set), there's the risk of one not
>> getting properly updated. Therefore I'd much prefer for just one
>> function to return the count. And I'm relatively certain that with the
>> type checks moved out, this will actually end up being the more
>> natural way.
>>
> I imagine the way as your suggestion. It might be below flow for this
> write_msr.
>
> list_for_each_entry(feat...) {
> feat->write_msr(..., val_array);
> val_array += feat->get_cos_num();
> ......
> }
>
> Is that what you think? Thanks!
Yes, something along these lines.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 12/24] x86: refactor psr: set value: implement write msr flow.
2017-01-12 9:40 ` Jan Beulich
@ 2017-01-12 10:22 ` Yi Sun
0 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2017-01-12 10:22 UTC (permalink / raw)
To: Jan Beulich
Cc: wei.liu2, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
On 17-01-12 02:40:41, Jan Beulich wrote:
> >>> On 12.01.17 at 02:22, <yi.y.sun@linux.intel.com> wrote:
> > On 17-01-11 07:01:23, Jan Beulich wrote:
> >> >>> On 11.01.17 at 07:22, <yi.y.sun@linux.intel.com> wrote:
> >> > On 17-01-10 08:15:15, Jan Beulich wrote:
> >> >> >>> On 14.12.16 at 05:07, <yi.y.sun@linux.intel.com> wrote:
> >> >> > --- a/xen/arch/x86/psr.c
> >> >> > +++ b/xen/arch/x86/psr.c
> >> >> > @@ -186,6 +186,9 @@ struct feat_ops {
> >> >> > unsigned int (*exceeds_cos_max)(const uint64_t val[],
> >> >> > const struct feat_node *feat,
> >> >> > unsigned int cos);
> >> >> > + /* write_msr is used to write out feature MSR register. */
> >> >> > + int (*write_msr)(unsigned int cos, const uint64_t val[],
> >> >> > + struct feat_node *feat);
> >> >>
> >> >> Looks like this function again returns number-of-values, yet this time
> >> >> without a comment saying so. While you don't need to replicate
> >> >> that description multiple time, please at least has a brief reference.
> >> >> That said, with the type checks moved out I think this return value
> >> >> model won't be needed anymore - the caller, having checked the
> >> >> type, could then simply call the get-num-val (or however it was
> >> >> named) hook to know how many array entries to skip.
> >> >>
> >> > For write msr, we may need iterate the whole feature list to write values for
> >> > every feature if the input value is not same as old on the COS ID. So, I prefer
> >> > to keep current return value, the number-of-values handled. That would be clear
> >> > and easy to implement. Of course, we can call get_cos_num to get the returen
> >> > value or define a macro to replace the digit. How do you think?
> >>
> >> Well, my general reservation here is that this way you require about
> >> half a dozen functions to all return the same value. If the value
> >> changes (or if somebody clones the set), there's the risk of one not
> >> getting properly updated. Therefore I'd much prefer for just one
> >> function to return the count. And I'm relatively certain that with the
> >> type checks moved out, this will actually end up being the more
> >> natural way.
> >>
> > I imagine the way as your suggestion. It might be below flow for this
> > write_msr.
> >
> > list_for_each_entry(feat...) {
> > feat->write_msr(..., val_array);
> > val_array += feat->get_cos_num();
> > ......
> > }
> >
> > Is that what you think? Thanks!
>
> Yes, something along these lines.
>
Got it, thanks!
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v4 13/24] x86: refactor psr: implement CPU init and free flow for CDP.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (11 preceding siblings ...)
2016-12-14 4:07 ` [PATCH v4 12/24] x86: refactor psr: set value: implement write msr flow Yi Sun
@ 2016-12-14 4:07 ` Yi Sun
2016-12-14 4:07 ` [PATCH v4 14/24] x86: refactor psr: implement get hw info " Yi Sun
` (10 subsequent siblings)
23 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:07 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements the CPU init and free flow for CDP including L3 CDP
initialization callback function.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 101 +++++++++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 98 insertions(+), 3 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index ec757a2..09da12c 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -91,6 +91,7 @@ struct psr_cat_hw_info {
struct feat_hw_info {
union {
struct psr_cat_hw_info l3_cat_info;
+ struct psr_cat_hw_info l3_cdp_info;
};
};
@@ -213,6 +214,22 @@ struct feat_node {
struct list_head list;
};
+/*
+ * get_data - get DATA COS register value from input COS ID.
+ * @feat: the feature list entry.
+ * @cos: the COS ID.
+ */
+#define get_cdp_data(feat, cos) \
+ ( feat->cos_reg_val[cos * 2] )
+
+/*
+ * get_cdp_code - get CODE COS register value from input COS ID.
+ * @feat: the feature list entry.
+ * @cos: the COS ID.
+ */
+#define get_cdp_code(feat, cos) \
+ ( feat->cos_reg_val[cos * 2 + 1] )
+
struct psr_assoc {
uint64_t val;
uint64_t cos_mask;
@@ -230,6 +247,7 @@ static DEFINE_PER_CPU(struct psr_assoc, psr_assoc);
/* Declare feature list entry. */
static struct feat_node *feat_l3_cat;
+static struct feat_node *feat_l3_cdp;
/* Common functions. */
static void free_feature(struct psr_socket_info *info)
@@ -252,6 +270,12 @@ static void free_feature(struct psr_socket_info *info)
xfree(feat_l3_cat);
feat_l3_cat = NULL;
}
+
+ if ( feat_l3_cdp )
+ {
+ xfree(feat_l3_cdp);
+ feat_l3_cdp = NULL;
+ }
}
static bool_t psr_check_cbm(unsigned int cbm_len, uint64_t cbm)
@@ -474,6 +498,61 @@ struct feat_ops l3_cat_ops = {
.write_msr = l3_cat_write_msr,
};
+/* L3 CDP callback functions implementation. */
+static void l3_cdp_init_feature(unsigned int eax, unsigned int ebx,
+ unsigned int ecx, unsigned int edx,
+ struct feat_node *feat,
+ struct psr_socket_info *info)
+{
+ struct psr_cat_hw_info l3_cdp;
+ unsigned int socket;
+ uint64_t val;
+
+ /* No valid value so do not enable feature. */
+ if ( !eax || !edx )
+ return;
+
+ l3_cdp.cbm_len = (eax & CAT_CBM_LEN_MASK) + 1;
+ /* Cut half of cos_max when CDP is enabled. */
+ l3_cdp.cos_max = min(opt_cos_max, edx & CAT_COS_MAX_MASK) >> 1;
+
+ /* cos=0 is reserved as default cbm(all ones). */
+ get_cdp_code(feat, 0) =
+ (1ull << l3_cdp.cbm_len) - 1;
+ get_cdp_data(feat, 0) =
+ (1ull << l3_cdp.cbm_len) - 1;
+
+ /* We only write mask1 since mask0 is always all ones by default. */
+ wrmsrl(MSR_IA32_PSR_L3_MASK(1), (1ull << l3_cdp.cbm_len) - 1);
+ rdmsrl(MSR_IA32_PSR_L3_QOS_CFG, val);
+ wrmsrl(MSR_IA32_PSR_L3_QOS_CFG, val | (1 << PSR_L3_QOS_CDP_ENABLE_BIT));
+
+ feat->feature = PSR_SOCKET_L3_CDP;
+ __set_bit(PSR_SOCKET_L3_CDP, &info->feat_mask);
+
+ feat->info.l3_cdp_info = l3_cdp;
+
+ info->nr_feat++;
+
+ /* Add this feature into list. */
+ list_add_tail(&feat->list, &info->feat_list);
+
+ socket = cpu_to_socket(smp_processor_id());
+ printk(XENLOG_INFO "L3 CDP: enabled on socket %u, cos_max:%u, cbm_len:%u\n",
+ socket, feat->info.l3_cdp_info.cos_max,
+ feat->info.l3_cdp_info.cbm_len);
+}
+
+static unsigned int l3_cdp_get_max_cos_max(const struct feat_node *feat)
+{
+ return feat->info.l3_cdp_info.cos_max;
+}
+
+struct feat_ops l3_cdp_ops = {
+ .init_feature = l3_cdp_init_feature,
+ .get_max_cos_max = l3_cdp_get_max_cos_max,
+};
+
static void __init parse_psr_bool(char *s, char *value, char *feature,
unsigned int mask)
{
@@ -1159,6 +1238,14 @@ static int cpu_prepare_work(unsigned int cpu)
(feat_l3_cat = xzalloc(struct feat_node)) == NULL )
return -ENOMEM;
+ if ( feat_l3_cdp == NULL &&
+ (feat_l3_cdp = xzalloc(struct feat_node)) == NULL )
+ {
+ xfree(feat_l3_cat);
+ feat_l3_cat = NULL;
+ return -ENOMEM;
+ }
+
return 0;
}
@@ -1186,9 +1273,17 @@ static void cpu_init_work(void)
{
cpuid_count(PSR_CPUID_LEVEL_CAT, 1, &eax, &ebx, &ecx, &edx);
- feat_tmp = feat_l3_cat;
- feat_l3_cat = NULL;
- feat_tmp->ops = l3_cat_ops;
+ if ( (ecx & PSR_CAT_CDP_CAPABILITY) && (opt_psr & PSR_CDP) &&
+ !test_bit(PSR_SOCKET_L3_CDP, &info->feat_mask) )
+ {
+ feat_tmp = feat_l3_cdp;
+ feat_l3_cdp = NULL;
+ feat_tmp->ops = l3_cdp_ops;
+ } else {
+ feat_tmp = feat_l3_cat;
+ feat_l3_cat = NULL;
+ feat_tmp->ops = l3_cat_ops;
+ }
feat_tmp->ops.init_feature(eax, ebx, ecx, edx, feat_tmp, info);
}
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v4 14/24] x86: refactor psr: implement get hw info flow for CDP.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (12 preceding siblings ...)
2016-12-14 4:07 ` [PATCH v4 13/24] x86: refactor psr: implement CPU init and free flow for CDP Yi Sun
@ 2016-12-14 4:07 ` Yi Sun
2016-12-14 4:07 ` [PATCH v4 15/24] x86: refactor psr: implement get value " Yi Sun
` (9 subsequent siblings)
23 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:07 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements get HW info flow for CDP including L3 CDP callback
function.
It also changes sysctl function to make it work for CDP.
With this patch, 'psr-hwinfo' can work for L3 CDP.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 16 ++++++++++++++++
xen/arch/x86/sysctl.c | 24 +++++++++++++++++++++---
2 files changed, 37 insertions(+), 3 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 09da12c..99add7a 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -548,9 +548,25 @@ static unsigned int l3_cdp_get_max_cos_max(const struct feat_node *feat)
return feat->info.l3_cdp_info.cos_max;
}
+static bool l3_cdp_get_feat_info(const struct feat_node *feat,
+ enum cbm_type type,
+ uint32_t dat[], uint32_t array_len)
+{
+ if ( !dat || 3 > array_len ||
+ ( type != PSR_CBM_TYPE_L3_DATA && type != PSR_CBM_TYPE_L3_CODE) )
+ return false;
+
+ dat[CBM_LEN] = feat->info.l3_cdp_info.cbm_len;
+ dat[COS_MAX] = feat->info.l3_cdp_info.cos_max;
+ dat[CDP_FLAG] |= XEN_SYSCTL_PSR_CAT_L3_CDP;
+
+ return true;
+}
+
struct feat_ops l3_cdp_ops = {
.init_feature = l3_cdp_init_feature,
.get_max_cos_max = l3_cdp_get_max_cos_max,
+ .get_feat_info = l3_cdp_get_feat_info,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index 168ed45..846eeaf 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -181,9 +181,27 @@ long arch_do_sysctl(
ret = psr_get_info(sysctl->u.psr_cat_op.target,
PSR_CBM_TYPE_L3, dat, 3);
- sysctl->u.psr_cat_op.u.l3_info.cbm_len = dat[CBM_LEN];
- sysctl->u.psr_cat_op.u.l3_info.cos_max = dat[COS_MAX];
- sysctl->u.psr_cat_op.u.l3_info.flags = dat[CDP_FLAG];
+ if ( !ret )
+ {
+ sysctl->u.psr_cat_op.u.l3_info.cbm_len = dat[CBM_LEN];
+ sysctl->u.psr_cat_op.u.l3_info.cos_max = dat[COS_MAX];
+ sysctl->u.psr_cat_op.u.l3_info.flags = dat[CDP_FLAG];
+ } else {
+ /*
+ * Check if CDP is enabled.
+ *
+ * Per spec, L3 CAT and CDP cannot co-exist. So, we need replace
+ * output values to CDP's if it is enabled.
+ */
+ ret = psr_get_info(sysctl->u.psr_cat_op.target,
+ PSR_CBM_TYPE_L3_CODE, dat, 3);
+ if ( !ret )
+ {
+ sysctl->u.psr_cat_op.u.l3_info.cbm_len = dat[CBM_LEN];
+ sysctl->u.psr_cat_op.u.l3_info.cos_max = dat[COS_MAX];
+ sysctl->u.psr_cat_op.u.l3_info.flags = dat[CDP_FLAG];
+ }
+ }
if ( !ret && __copy_field_to_guest(u_sysctl, sysctl, u.psr_cat_op) )
ret = -EFAULT;
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v4 15/24] x86: refactor psr: implement get value flow for CDP.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (13 preceding siblings ...)
2016-12-14 4:07 ` [PATCH v4 14/24] x86: refactor psr: implement get hw info " Yi Sun
@ 2016-12-14 4:07 ` Yi Sun
2016-12-14 4:07 ` [PATCH v4 16/24] x86: refactor psr: implement set value callback functions " Yi Sun
` (8 subsequent siblings)
23 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:07 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements L3 CDP get value callback function.
With this patch, 'psr-cat-show' can work for L3 CDP.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 99add7a..d0392ca 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -563,10 +563,29 @@ static bool l3_cdp_get_feat_info(const struct feat_node *feat,
return true;
}
+static bool l3_cdp_get_val(const struct feat_node *feat, unsigned int cos,
+ enum cbm_type type, uint64_t *val)
+{
+ if ( type != PSR_CBM_TYPE_L3_DATA && type != PSR_CBM_TYPE_L3_CODE )
+ return false;
+
+ if ( cos > feat->info.l3_cdp_info.cos_max )
+ /* Use default value. */
+ cos = 0;
+
+ if ( type == PSR_CBM_TYPE_L3_DATA )
+ *val = get_cdp_data(feat, cos);
+ else
+ *val = get_cdp_code(feat, cos);
+
+ return true;
+}
+
struct feat_ops l3_cdp_ops = {
.init_feature = l3_cdp_init_feature,
.get_max_cos_max = l3_cdp_get_max_cos_max,
.get_feat_info = l3_cdp_get_feat_info,
+ .get_val = l3_cdp_get_val,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v4 16/24] x86: refactor psr: implement set value callback functions for CDP.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (14 preceding siblings ...)
2016-12-14 4:07 ` [PATCH v4 15/24] x86: refactor psr: implement get value " Yi Sun
@ 2016-12-14 4:07 ` Yi Sun
2016-12-14 4:07 ` [PATCH v4 17/24] x86: L2 CAT: implement CPU init and free flow Yi Sun
` (7 subsequent siblings)
23 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:07 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements L3 CDP set value related callback functions.
With this patch, 'psr-cat-cbm-set' command can work for L3 CDP.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 138 +++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 138 insertions(+)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index d0392ca..eee1f24 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -581,11 +581,149 @@ static bool l3_cdp_get_val(const struct feat_node *feat, unsigned int cos,
return true;
}
+static unsigned int l3_cdp_get_cos_num(const struct feat_node *feat)
+{
+ return 2;
+}
+
+static int l3_cdp_get_old_val(uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int old_cos)
+{
+ if ( old_cos > feat->info.l3_cdp_info.cos_max )
+ /* Use default value. */
+ old_cos = 0;
+
+ /* Data */
+ val[0] = get_cdp_data(feat, old_cos);
+ /* Code */
+ val[1] = get_cdp_code(feat, old_cos);
+
+ /* CDP uses two COSs, one for data, one for code. */
+ return 2;
+}
+
+static int l3_cdp_set_new_val(uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int old_cos,
+ enum cbm_type type,
+ uint64_t m)
+{
+ if ( type != PSR_CBM_TYPE_L3_DATA && type != PSR_CBM_TYPE_L3_CODE )
+ /* CDP uses two COSs. Skip them outside. */
+ return 2;
+
+ if ( !psr_check_cbm(feat->info.l3_cdp_info.cbm_len, m) )
+ return -EINVAL;
+
+ if ( type == PSR_CBM_TYPE_L3_DATA )
+ val[0] = m;
+ else
+ val[1] = m;
+
+ /* CDP uses two COS, one for data, one for code. */
+ return 2;
+}
+
+static unsigned int l3_cdp_get_cos_max_from_type(const struct feat_node *feat,
+ enum cbm_type type)
+{
+ if ( type != PSR_CBM_TYPE_L3_DATA && type != PSR_CBM_TYPE_L3_CODE )
+ return 0;
+
+ return feat->info.l3_cdp_info.cos_max;
+}
+
+static int l3_cdp_compare_val(const uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int cos, bool *found)
+{
+ uint64_t l3_def_cbm;
+
+ l3_def_cbm = (1ull << feat->info.l3_cdp_info.cbm_len) - 1;
+
+ /*
+ * Different features' cos_max are different. If cos id of the feature
+ * being set exceeds other feature's cos_max, the val of other feature
+ * must be default value. HW supports such case.
+ */
+ if ( cos > feat->info.l3_cdp_info.cos_max )
+ {
+ if ( val[0] != l3_def_cbm ||
+ val[1] != l3_def_cbm )
+ {
+ *found = false;
+ return -ENOENT;
+ }
+ *found = true;
+ }
+ else
+ *found = (val[0] == get_cdp_data(feat, cos) &&
+ val[1] == get_cdp_code(feat, cos));
+
+ /* CDP uses two COS, one for data, one for code. */
+ return 2;
+}
+
+static unsigned int l3_cdp_exceeds_cos_max(const uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int cos)
+{
+ uint64_t l3_def_cbm;
+
+ l3_def_cbm = (1ull << feat->info.l3_cdp_info.cbm_len) - 1;
+
+ if ( cos > feat->info.l3_cdp_info.cos_max &&
+ (val[0] != l3_def_cbm || val[1] != l3_def_cbm) )
+ /*
+ * Exceed cos_max and value to set is not default,
+ * return error.
+ */
+ return 0;
+
+ /* CDP uses two COS, one for data, one for code. */
+ return 2;
+}
+
+static int l3_cdp_write_msr(unsigned int cos, const uint64_t val[],
+ struct feat_node *feat)
+{
+ /*
+ * If input cos is more than the cos_max of the feature, we should
+ * not set the value.
+ */
+ if ( cos > feat->info.l3_cdp_info.cos_max )
+ /* CDP uses two COS, one for data, one for code. */
+ return 2;
+
+ /* Data */
+ if ( get_cdp_data(feat, cos) != val[0] )
+ {
+ get_cdp_data(feat, cos) = val[0];
+ wrmsrl(MSR_IA32_PSR_L3_MASK_DATA(cos), val[0]);
+ }
+ /* Code */
+ if ( get_cdp_code(feat, cos) != val[1] )
+ {
+ get_cdp_code(feat, cos) = val[1];
+ wrmsrl(MSR_IA32_PSR_L3_MASK_CODE(cos), val[1]);
+ }
+ /* CDP uses two COS, one for data, one for code. */
+ return 2;
+}
+
struct feat_ops l3_cdp_ops = {
.init_feature = l3_cdp_init_feature,
.get_max_cos_max = l3_cdp_get_max_cos_max,
.get_feat_info = l3_cdp_get_feat_info,
.get_val = l3_cdp_get_val,
+ .get_cos_num = l3_cdp_get_cos_num,
+ .get_old_val = l3_cdp_get_old_val,
+ .set_new_val = l3_cdp_set_new_val,
+ .get_cos_max_from_type = l3_cdp_get_cos_max_from_type,
+ .compare_val = l3_cdp_compare_val,
+ .exceeds_cos_max = l3_cdp_exceeds_cos_max,
+ .write_msr = l3_cdp_write_msr,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v4 17/24] x86: L2 CAT: implement CPU init and free flow.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (15 preceding siblings ...)
2016-12-14 4:07 ` [PATCH v4 16/24] x86: refactor psr: implement set value callback functions " Yi Sun
@ 2016-12-14 4:07 ` Yi Sun
2016-12-14 4:07 ` [PATCH v4 18/24] x86: L2 CAT: implement get hw info flow Yi Sun
` (6 subsequent siblings)
23 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:07 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements the CPU init and free flow for L2 CAT including
L2 CAT initialization callback function.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++
xen/include/asm-x86/psr.h | 1 +
2 files changed, 77 insertions(+)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index eee1f24..b890b64 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -92,6 +92,7 @@ struct feat_hw_info {
union {
struct psr_cat_hw_info l3_cat_info;
struct psr_cat_hw_info l3_cdp_info;
+ struct psr_cat_hw_info l2_cat_info;
};
};
@@ -248,6 +249,7 @@ static DEFINE_PER_CPU(struct psr_assoc, psr_assoc);
/* Declare feature list entry. */
static struct feat_node *feat_l3_cat;
static struct feat_node *feat_l3_cdp;
+static struct feat_node *feat_l2_cat;
/* Common functions. */
static void free_feature(struct psr_socket_info *info)
@@ -276,6 +278,12 @@ static void free_feature(struct psr_socket_info *info)
xfree(feat_l3_cdp);
feat_l3_cdp = NULL;
}
+
+ if ( feat_l2_cat )
+ {
+ xfree(feat_l2_cat);
+ feat_l2_cat = NULL;
+ }
}
static bool_t psr_check_cbm(unsigned int cbm_len, uint64_t cbm)
@@ -726,6 +734,51 @@ struct feat_ops l3_cdp_ops = {
.write_msr = l3_cdp_write_msr,
};
+/* L2 CAT callback functions implementation. */
+static void l2_cat_init_feature(unsigned int eax, unsigned int ebx,
+ unsigned int ecx, unsigned int edx,
+ struct feat_node *feat,
+ struct psr_socket_info *info)
+{
+ struct psr_cat_hw_info l2_cat;
+ unsigned int socket;
+
+ /* No valid values so do not enable the feature. */
+ if ( !eax || !edx )
+ return;
+
+ l2_cat.cbm_len = (eax & CAT_CBM_LEN_MASK) + 1;
+ l2_cat.cos_max = min(opt_cos_max, edx & CAT_COS_MAX_MASK);
+
+ /* cos=0 is reserved as default cbm(all ones). */
+ feat->cos_reg_val[0] = (1ull << l2_cat.cbm_len) - 1;
+
+ feat->feature = PSR_SOCKET_L2_CAT;
+ __set_bit(PSR_SOCKET_L2_CAT, &info->feat_mask);
+
+ feat->info.l2_cat_info = l2_cat;
+
+ info->nr_feat++;
+
+ /* Add this feature into list. */
+ list_add_tail(&feat->list, &info->feat_list);
+
+ socket = cpu_to_socket(smp_processor_id());
+ printk(XENLOG_INFO "L2 CAT: enabled on socket %u, cos_max:%u, cbm_len:%u.\n",
+ socket, feat->info.l2_cat_info.cos_max,
+ feat->info.l2_cat_info.cbm_len);
+}
+
+static unsigned int l2_cat_get_max_cos_max(const struct feat_node *feat)
+{
+ return feat->info.l2_cat_info.cos_max;
+}
+
+struct feat_ops l2_cat_ops = {
+ .init_feature = l2_cat_init_feature,
+ .get_max_cos_max = l2_cat_get_max_cos_max,
+};
+
static void __init parse_psr_bool(char *s, char *value, char *feature,
unsigned int mask)
{
@@ -1419,6 +1472,17 @@ static int cpu_prepare_work(unsigned int cpu)
return -ENOMEM;
}
+ if ( feat_l2_cat == NULL &&
+ (feat_l2_cat = xzalloc(struct feat_node)) == NULL )
+ {
+ xfree(feat_l3_cat);
+ feat_l3_cat = NULL;
+
+ xfree(feat_l3_cdp);
+ feat_l3_cdp = NULL;
+ return -ENOMEM;
+ }
+
return 0;
}
@@ -1460,6 +1524,18 @@ static void cpu_init_work(void)
feat_tmp->ops.init_feature(eax, ebx, ecx, edx, feat_tmp, info);
}
+
+ cpuid_count(PSR_CPUID_LEVEL_CAT, 0, &eax, &ebx, &ecx, &edx);
+ if ( ebx & PSR_RESOURCE_TYPE_L2 )
+ {
+ feat_tmp = feat_l2_cat;
+ feat_l2_cat = NULL;
+
+ /* Initialize L2 CAT according to CPUID. */
+ cpuid_count(PSR_CPUID_LEVEL_CAT, 2, &eax, &ebx, &ecx, &edx);
+ feat_tmp->ops = l2_cat_ops;
+ feat_tmp->ops.init_feature(eax, ebx, ecx, edx, feat_tmp, info);
+ }
}
static void cpu_fini_work(unsigned int cpu)
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index d245de3..a7fc13d 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -23,6 +23,7 @@
/* Resource Type Enumeration */
#define PSR_RESOURCE_TYPE_L3 0x2
+#define PSR_RESOURCE_TYPE_L2 0x4
/* L3 Monitoring Features */
#define PSR_CMT_L3_OCCUPANCY 0x1
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v4 18/24] x86: L2 CAT: implement get hw info flow.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (16 preceding siblings ...)
2016-12-14 4:07 ` [PATCH v4 17/24] x86: L2 CAT: implement CPU init and free flow Yi Sun
@ 2016-12-14 4:07 ` Yi Sun
2016-12-14 4:07 ` [PATCH v4 19/24] x86: L2 CAT: implement get value flow Yi Sun
` (5 subsequent siblings)
23 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:07 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements get HW info flow for L2 CAT including L2 CAT callback
function.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/psr.c | 14 ++++++++++++++
xen/arch/x86/sysctl.c | 15 +++++++++++++++
xen/include/asm-x86/psr.h | 1 +
xen/include/public/sysctl.h | 6 ++++++
4 files changed, 36 insertions(+)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index b890b64..6de0abf 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -774,9 +774,23 @@ static unsigned int l2_cat_get_max_cos_max(const struct feat_node *feat)
return feat->info.l2_cat_info.cos_max;
}
+static bool l2_cat_get_feat_info(const struct feat_node *feat,
+ enum cbm_type type,
+ uint32_t dat[], uint32_t array_len)
+{
+ if ( !dat || 2 > array_len || type != PSR_CBM_TYPE_L2 )
+ return false;
+
+ dat[CBM_LEN] = feat->info.l2_cat_info.cbm_len;
+ dat[COS_MAX] = feat->info.l2_cat_info.cos_max;
+
+ return true;
+}
+
struct feat_ops l2_cat_ops = {
.init_feature = l2_cat_init_feature,
.get_max_cos_max = l2_cat_get_max_cos_max,
+ .get_feat_info = l2_cat_get_feat_info,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index 846eeaf..dede62c 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -207,6 +207,21 @@ long arch_do_sysctl(
ret = -EFAULT;
break;
}
+ case XEN_SYSCTL_PSR_CAT_get_l2_info:
+ {
+ uint32_t dat[2];
+ ret = psr_get_info(sysctl->u.psr_cat_op.target,
+ PSR_CBM_TYPE_L2, dat, 2);
+ if ( ret )
+ break;
+
+ sysctl->u.psr_cat_op.u.l2_info.cbm_len = dat[CBM_LEN];
+ sysctl->u.psr_cat_op.u.l2_info.cos_max = dat[COS_MAX];
+
+ if ( !ret && __copy_field_to_guest(u_sysctl, sysctl, u.psr_cat_op) )
+ ret = -EFAULT;
+ break;
+ }
default:
ret = -EOPNOTSUPP;
break;
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index a7fc13d..ee65fe5 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -56,6 +56,7 @@ enum cbm_type {
PSR_CBM_TYPE_L3,
PSR_CBM_TYPE_L3_CODE,
PSR_CBM_TYPE_L3_DATA,
+ PSR_CBM_TYPE_L2,
};
extern struct psr_cmt *psr_cmt;
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index 418f4bb..0a0c9f7 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -745,6 +745,7 @@ typedef struct xen_sysctl_pcitopoinfo xen_sysctl_pcitopoinfo_t;
DEFINE_XEN_GUEST_HANDLE(xen_sysctl_pcitopoinfo_t);
#define XEN_SYSCTL_PSR_CAT_get_l3_info 0
+#define XEN_SYSCTL_PSR_CAT_get_l2_info 1
struct xen_sysctl_psr_cat_op {
uint32_t cmd; /* IN: XEN_SYSCTL_PSR_CAT_* */
uint32_t target; /* IN */
@@ -755,6 +756,11 @@ struct xen_sysctl_psr_cat_op {
#define XEN_SYSCTL_PSR_CAT_L3_CDP (1u << 0)
uint32_t flags; /* OUT: CAT flags */
} l3_info;
+
+ struct {
+ uint32_t cbm_len; /* OUT: CBM length */
+ uint32_t cos_max; /* OUT: Maximum COS */
+ } l2_info;
} u;
};
typedef struct xen_sysctl_psr_cat_op xen_sysctl_psr_cat_op_t;
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v4 19/24] x86: L2 CAT: implement get value flow.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (17 preceding siblings ...)
2016-12-14 4:07 ` [PATCH v4 18/24] x86: L2 CAT: implement get hw info flow Yi Sun
@ 2016-12-14 4:07 ` Yi Sun
2016-12-14 4:08 ` [PATCH v4 20/24] x86: L2 CAT: implement set " Yi Sun
` (4 subsequent siblings)
23 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:07 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements L2 CAT get value callback function and
interface in domctl.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/domctl.c | 7 +++++++
xen/arch/x86/psr.c | 16 ++++++++++++++++
xen/include/public/domctl.h | 1 +
3 files changed, 24 insertions(+)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 8e5502a..e5aa932 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1424,6 +1424,13 @@ long arch_do_domctl(
copyback = 1;
break;
+ case XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM:
+ ret = psr_get_val(d, domctl->u.psr_cat_op.target,
+ &domctl->u.psr_cat_op.data,
+ PSR_CBM_TYPE_L2);
+ copyback = 1;
+ break;
+
default:
ret = -EOPNOTSUPP;
break;
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 6de0abf..0886a01 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -787,10 +787,26 @@ static bool l2_cat_get_feat_info(const struct feat_node *feat,
return true;
}
+static bool l2_cat_get_val(const struct feat_node *feat, unsigned int cos,
+ enum cbm_type type, uint64_t *val)
+{
+ if ( type != PSR_CBM_TYPE_L2 )
+ return false;
+
+ if ( cos > feat->info.l2_cat_info.cos_max )
+ cos = 0;
+
+ /* L2 CAT */
+ *val = feat->cos_reg_val[cos];
+
+ return true;
+}
+
struct feat_ops l2_cat_ops = {
.init_feature = l2_cat_init_feature,
.get_max_cos_max = l2_cat_get_max_cos_max,
.get_feat_info = l2_cat_get_feat_info,
+ .get_val = l2_cat_get_val,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 177319d..d04e318 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1137,6 +1137,7 @@ struct xen_domctl_psr_cat_op {
#define XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA 3
#define XEN_DOMCTL_PSR_CAT_OP_GET_L3_CODE 4
#define XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA 5
+#define XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM 7
uint32_t cmd; /* IN: XEN_DOMCTL_PSR_CAT_OP_* */
uint32_t target; /* IN */
uint64_t data; /* IN/OUT */
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v4 20/24] x86: L2 CAT: implement set value flow.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (18 preceding siblings ...)
2016-12-14 4:07 ` [PATCH v4 19/24] x86: L2 CAT: implement get value flow Yi Sun
@ 2016-12-14 4:08 ` Yi Sun
2016-12-14 4:08 ` [PATCH v4 21/24] tools: L2 CAT: support get HW info for L2 CAT Yi Sun
` (3 subsequent siblings)
23 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:08 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements L2 CAT set value related callback functions
and domctl interface.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/domctl.c | 6 ++
xen/arch/x86/psr.c | 132 +++++++++++++++++++++++++++++++++++++++-
xen/include/asm-x86/msr-index.h | 1 +
xen/include/public/domctl.h | 1 +
4 files changed, 139 insertions(+), 1 deletion(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index e5aa932..69aa4d3 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1403,6 +1403,12 @@ long arch_do_domctl(
PSR_CBM_TYPE_L3_DATA);
break;
+ case XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM:
+ ret = psr_set_val(d, domctl->u.psr_cat_op.target,
+ domctl->u.psr_cat_op.data,
+ PSR_CBM_TYPE_L2);
+ break;
+
case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM:
ret = psr_get_val(d, domctl->u.psr_cat_op.target,
&domctl->u.psr_cat_op.data,
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 0886a01..ed1ec2f 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -796,17 +796,127 @@ static bool l2_cat_get_val(const struct feat_node *feat, unsigned int cos,
if ( cos > feat->info.l2_cat_info.cos_max )
cos = 0;
- /* L2 CAT */
*val = feat->cos_reg_val[cos];
return true;
}
+static unsigned int l2_cat_get_cos_num(const struct feat_node *feat)
+{
+ /* L2 CAT uses one COS. */
+ return 1;
+}
+
+static int l2_cat_get_old_val(uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int old_cos)
+{
+ if ( old_cos > feat->info.l2_cat_info.cos_max )
+ /* Use default value. */
+ old_cos = 0;
+
+ val[0] = feat->cos_reg_val[old_cos];
+
+ /* L2 CAT uses one COS. */
+ return 1;
+}
+
+static int l2_cat_set_new_val(uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int old_cos,
+ enum cbm_type type,
+ uint64_t m)
+{
+ if ( type != PSR_CBM_TYPE_L2 )
+ return 1;
+
+ if ( !psr_check_cbm(feat->info.l2_cat_info.cbm_len, m) )
+ return -EINVAL;
+
+ val[0] = m;
+
+ /* L2 CAT uses one COS. */
+ return 1;
+}
+
+static unsigned int l2_cat_get_cos_max_from_type(const struct feat_node *feat,
+ enum cbm_type type)
+{
+ if ( type != PSR_CBM_TYPE_L2 )
+ return 0;
+
+ return feat->info.l2_cat_info.cos_max;
+}
+
+static int l2_cat_compare_val(const uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int cos, bool *found)
+{
+ uint64_t l2_def_cbm;
+
+ l2_def_cbm = (1ull << feat->info.l2_cat_info.cbm_len) - 1;
+
+ if ( cos > feat->info.l2_cat_info.cos_max )
+ {
+ if ( val[0] != l2_def_cbm )
+ {
+ *found = false;
+ return -ENOENT;
+ }
+ *found = true;
+ }
+ else
+ *found = (val[0] == feat->cos_reg_val[cos]);
+
+ /* L2 CAT uses one COS. */
+ return 1;
+}
+
+static unsigned int l2_cat_exceeds_cos_max(const uint64_t val[],
+ const struct feat_node *feat,
+ unsigned int cos)
+{
+ uint64_t l2_def_cbm;
+
+ l2_def_cbm = (1ull << feat->info.l2_cat_info.cbm_len) - 1;
+
+ if ( cos > feat->info.l2_cat_info.cos_max &&
+ val[0] != l2_def_cbm )
+ /*
+ * Exceed cos_max and value to set is not default,
+ * return error.
+ */
+ return 0;
+
+ /* L2 CAT uses one COS. */
+ return 1;
+}
+
+static int l2_cat_write_msr(unsigned int cos, const uint64_t val[],
+ struct feat_node *feat)
+{
+ if ( cos > feat->info.l2_cat_info.cos_max )
+ return 1;
+
+ feat->cos_reg_val[cos] = val[0];
+ wrmsrl(MSR_IA32_PSR_L2_MASK(cos), val[0]);
+
+ /* L2 CAT uses one COS. */
+ return 1;
+}
+
struct feat_ops l2_cat_ops = {
.init_feature = l2_cat_init_feature,
.get_max_cos_max = l2_cat_get_max_cos_max,
.get_feat_info = l2_cat_get_feat_info,
.get_val = l2_cat_get_val,
+ .get_cos_num = l2_cat_get_cos_num,
+ .get_old_val = l2_cat_get_old_val,
+ .set_new_val = l2_cat_set_new_val,
+ .get_cos_max_from_type = l2_cat_get_cos_max_from_type,
+ .compare_val = l2_cat_compare_val,
+ .exceeds_cos_max = l2_cat_exceeds_cos_max,
+ .write_msr = l2_cat_write_msr,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
@@ -1317,10 +1427,30 @@ int psr_set_val(struct domain *d, unsigned int socket,
uint64_t *val_array;
struct psr_socket_info *info = get_socket_info(socket);
uint32_t array_len;
+ uint32_t flag;
if ( IS_ERR(info) )
return PTR_ERR(info);
+ /* Judge if feature is enabled. */
+ switch ( type ) {
+ case PSR_CBM_TYPE_L3:
+ flag = PSR_SOCKET_L3_CAT;
+ break;
+ case PSR_CBM_TYPE_L3_DATA:
+ case PSR_CBM_TYPE_L3_CODE:
+ flag = PSR_SOCKET_L3_CDP;
+ break;
+ case PSR_CBM_TYPE_L2:
+ flag = PSR_SOCKET_L2_CAT;
+ break;
+ default:
+ flag = 0xFFFF;
+ break;
+ }
+ if ( !test_bit(flag, &info->feat_mask) )
+ return -ENODEV;
+
/*
* Step 0:
* old_cos means the COS ID current domain is using. By default, it is 0.
diff --git a/xen/include/asm-x86/msr-index.h b/xen/include/asm-x86/msr-index.h
index 98dbff1..a41e63a 100644
--- a/xen/include/asm-x86/msr-index.h
+++ b/xen/include/asm-x86/msr-index.h
@@ -343,6 +343,7 @@
#define MSR_IA32_PSR_L3_MASK(n) (0x00000c90 + (n))
#define MSR_IA32_PSR_L3_MASK_CODE(n) (0x00000c90 + (n) * 2 + 1)
#define MSR_IA32_PSR_L3_MASK_DATA(n) (0x00000c90 + (n) * 2)
+#define MSR_IA32_PSR_L2_MASK(n) (0x00000d10 + (n))
/* Intel Model 6 */
#define MSR_P6_PERFCTR(n) (0x000000c1 + (n))
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index d04e318..17c4ab4 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1137,6 +1137,7 @@ struct xen_domctl_psr_cat_op {
#define XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA 3
#define XEN_DOMCTL_PSR_CAT_OP_GET_L3_CODE 4
#define XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA 5
+#define XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM 6
#define XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM 7
uint32_t cmd; /* IN: XEN_DOMCTL_PSR_CAT_OP_* */
uint32_t target; /* IN */
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v4 21/24] tools: L2 CAT: support get HW info for L2 CAT.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (19 preceding siblings ...)
2016-12-14 4:08 ` [PATCH v4 20/24] x86: L2 CAT: implement set " Yi Sun
@ 2016-12-14 4:08 ` Yi Sun
2017-01-06 12:04 ` Wei Liu
2016-12-14 4:08 ` [PATCH v4 22/24] tools: L2 CAT: support show cbm " Yi Sun
` (2 subsequent siblings)
23 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:08 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements xl/xc changes to support get HW info
for L2 CAT.
'xl psr-hwinfo' is updated to show both L3 CAT and L2 CAT
info.
Example(on machine which only supports L2 CAT):
Cache Monitoring Technology (CMT):
Enabled : 0
Cache Allocation Technology (CAT): L3
libxl: error: libxl_psr.c:100:libxl__psr_cat_log_err_msg: CAT is not enabled on the socket: No such file or directory
Failed to get l3 cat info
Cache Allocation Technology (CAT): L2
Socket ID : 0
Maximum COS : 3
CBM length : 8
Default CBM : 0xff
Signed-off-by: He Chen <he.chen@linux.intel.com>
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
tools/libxc/include/xenctrl.h | 6 +++---
tools/libxc/xc_psr.c | 40 +++++++++++++++++++++++-----------
tools/libxl/libxl.h | 11 ++++++++--
tools/libxl/libxl_psr.c | 8 +++----
tools/libxl/xl_cmdimpl.c | 50 +++++++++++++++++++++++++++++++++++--------
5 files changed, 85 insertions(+), 30 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 2c83544..5b7df22 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2626,9 +2626,9 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_cat_type type, uint32_t target,
uint64_t *data);
-int xc_psr_cat_get_l3_info(xc_interface *xch, uint32_t socket,
- uint32_t *cos_max, uint32_t *cbm_len,
- bool *cdp_enabled);
+int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
+ uint32_t *cos_max, uint32_t *cbm_len,
+ bool *cdp_enabled);
int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 43b3286..7af17d9 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -317,24 +317,40 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
return rc;
}
-int xc_psr_cat_get_l3_info(xc_interface *xch, uint32_t socket,
- uint32_t *cos_max, uint32_t *cbm_len,
- bool *cdp_enabled)
+int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
+ uint32_t *cos_max, uint32_t *cbm_len, bool *cdp_enabled)
{
- int rc;
+ int rc = -1;
DECLARE_SYSCTL;
sysctl.cmd = XEN_SYSCTL_psr_cat_op;
- sysctl.u.psr_cat_op.cmd = XEN_SYSCTL_PSR_CAT_get_l3_info;
sysctl.u.psr_cat_op.target = socket;
- rc = xc_sysctl(xch, &sysctl);
- if ( !rc )
- {
- *cos_max = sysctl.u.psr_cat_op.u.l3_info.cos_max;
- *cbm_len = sysctl.u.psr_cat_op.u.l3_info.cbm_len;
- *cdp_enabled = sysctl.u.psr_cat_op.u.l3_info.flags &
- XEN_SYSCTL_PSR_CAT_L3_CDP;
+ switch ( lvl ) {
+ case 2:
+ sysctl.u.psr_cat_op.cmd = XEN_SYSCTL_PSR_CAT_get_l2_info;
+ rc = xc_sysctl(xch, &sysctl);
+ if ( !rc )
+ {
+ *cos_max = sysctl.u.psr_cat_op.u.l2_info.cos_max;
+ *cbm_len = sysctl.u.psr_cat_op.u.l2_info.cbm_len;
+ *cdp_enabled = false;
+ }
+ break;
+ case 3:
+ sysctl.u.psr_cat_op.cmd = XEN_SYSCTL_PSR_CAT_get_l3_info;
+ rc = xc_sysctl(xch, &sysctl);
+ if ( !rc )
+ {
+ *cos_max = sysctl.u.psr_cat_op.u.l3_info.cos_max;
+ *cbm_len = sysctl.u.psr_cat_op.u.l3_info.cbm_len;
+ *cdp_enabled = sysctl.u.psr_cat_op.u.l3_info.flags &
+ XEN_SYSCTL_PSR_CAT_L3_CDP;
+ }
+ break;
+ default:
+ errno = EOPNOTSUPP;
+ return rc;
}
return rc;
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index acbf476..9233b0f 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -904,6 +904,13 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const libxl_mac *src);
* If this is defined, the Code and Data Prioritization feature is supported.
*/
#define LIBXL_HAVE_PSR_CDP 1
+
+/*
+ * LIBXL_HAVE_PSR_L2_CAT
+ *
+ * If this is defined, the L2 Cache Allocation Technology feature is supported.
+ */
+#define LIBXL_HAVE_PSR_L2_CAT 1
#endif
/*
@@ -2159,8 +2166,8 @@ int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
* On success, the function returns an array of elements in 'info',
* and the length in 'nr'.
*/
-int libxl_psr_cat_get_l3_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
- int *nr);
+int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
+ int *nr, unsigned int lvl);
void libxl_psr_cat_info_list_free(libxl_psr_cat_info *list, int nr);
#endif
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index 786183c..f238f71 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -352,8 +352,8 @@ int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
return rc;
}
-int libxl_psr_cat_get_l3_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
- int *nr)
+int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
+ int *nr, unsigned int lvl)
{
GC_INIT(ctx);
int rc;
@@ -380,8 +380,8 @@ int libxl_psr_cat_get_l3_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
libxl_for_each_set_bit(socketid, socketmap) {
ptr[i].id = socketid;
- if (xc_psr_cat_get_l3_info(ctx->xch, socketid, &ptr[i].cos_max,
- &ptr[i].cbm_len, &ptr[i].cdp_enabled)) {
+ if (xc_psr_cat_get_info(ctx->xch, socketid, lvl, &ptr[i].cos_max,
+ &ptr[i].cbm_len, &ptr[i].cdp_enabled)) {
libxl__psr_cat_log_err_msg(gc, errno);
rc = ERROR_FAIL;
free(ptr);
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 7e8a8ae..c210b05 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -9331,19 +9331,19 @@ int main_psr_cmt_show(int argc, char **argv)
}
#endif
-#ifdef LIBXL_HAVE_PSR_CAT
-static int psr_cat_hwinfo(void)
+#if defined(LIBXL_HAVE_PSR_CAT) || defined(LIBXL_HAVE_PSR_L2_CAT)
+static int psr_l3_cat_hwinfo(void)
{
- int rc;
- int i, nr;
+ int rc, nr;
+ unsigned int i;
uint32_t l3_cache_size;
libxl_psr_cat_info *info;
- printf("Cache Allocation Technology (CAT):\n");
+ printf("Cache Allocation Technology (CAT): L3\n");
- rc = libxl_psr_cat_get_l3_info(ctx, &info, &nr);
+ rc = libxl_psr_cat_get_info(ctx, &info, &nr, 3);
if (rc) {
- fprintf(stderr, "Failed to get cat info\n");
+ fprintf(stderr, "Failed to get l3 cat info\n");
return rc;
}
@@ -9454,7 +9454,7 @@ static int psr_cat_show(uint32_t domid)
int rc;
libxl_psr_cat_info *info;
- rc = libxl_psr_cat_get_l3_info(ctx, &info, &nr);
+ rc = libxl_psr_cat_get_info(ctx, &info, &nr, 3);
if (rc) {
fprintf(stderr, "Failed to get cat info\n");
return rc;
@@ -9471,6 +9471,34 @@ out:
return rc;
}
+static int psr_l2_cat_hwinfo(void)
+{
+ int rc;
+ unsigned int i;
+ int nr;
+ libxl_psr_cat_info *info;
+
+ printf("Cache Allocation Technology (CAT): L2\n");
+
+ rc = libxl_psr_cat_get_info(ctx, &info, &nr, 2);
+ if (rc) {
+ fprintf(stderr, "Failed to get l2 cat info\n");
+ return rc;
+ }
+
+ for (i = 0; i < nr; i++) {
+ /* There is no CMT on L2 cache so far. */
+ printf("%-16s: %u\n", "Socket ID", info[i].id);
+ printf("%-16s: %u\n", "Maximum COS", info[i].cos_max);
+ printf("%-16s: %u\n", "CBM length", info[i].cbm_len);
+ printf("%-16s: %#llx\n", "Default CBM",
+ (1ull << info[i].cbm_len) - 1);
+ }
+
+ libxl_psr_cat_info_list_free(info, nr);
+ return rc;
+}
+
int main_psr_cat_cbm_set(int argc, char **argv)
{
uint32_t domid;
@@ -9588,7 +9616,11 @@ int main_psr_hwinfo(int argc, char **argv)
ret = psr_cmt_hwinfo();
if (!ret && (all || cat))
- ret = psr_cat_hwinfo();
+ ret = psr_l3_cat_hwinfo();
+
+ /* L2 CAT is independent of CMT and L3 CAT */
+ if (all || cat)
+ ret = psr_l2_cat_hwinfo();
return ret;
}
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* Re: [PATCH v4 21/24] tools: L2 CAT: support get HW info for L2 CAT.
2016-12-14 4:08 ` [PATCH v4 21/24] tools: L2 CAT: support get HW info for L2 CAT Yi Sun
@ 2017-01-06 12:04 ` Wei Liu
2017-01-09 1:19 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Wei Liu @ 2017-01-06 12:04 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
mengxu, jbeulich, chao.p.peng, xen-devel
On Wed, Dec 14, 2016 at 12:08:01PM +0800, Yi Sun wrote:
> This patch implements xl/xc changes to support get HW info
> for L2 CAT.
>
> 'xl psr-hwinfo' is updated to show both L3 CAT and L2 CAT
> info.
>
> Example(on machine which only supports L2 CAT):
> Cache Monitoring Technology (CMT):
> Enabled : 0
> Cache Allocation Technology (CAT): L3
> libxl: error: libxl_psr.c:100:libxl__psr_cat_log_err_msg: CAT is not enabled on the socket: No such file or directory
> Failed to get l3 cat info
What is this?
> Cache Allocation Technology (CAT): L2
> Socket ID : 0
> Maximum COS : 3
> CBM length : 8
> Default CBM : 0xff
>
> Signed-off-by: He Chen <he.chen@linux.intel.com>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> ---
> tools/libxc/include/xenctrl.h | 6 +++---
> tools/libxc/xc_psr.c | 40 +++++++++++++++++++++++-----------
> tools/libxl/libxl.h | 11 ++++++++--
> tools/libxl/libxl_psr.c | 8 +++----
> tools/libxl/xl_cmdimpl.c | 50 +++++++++++++++++++++++++++++++++++--------
> 5 files changed, 85 insertions(+), 30 deletions(-)
>
> diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
> index 2c83544..5b7df22 100644
> --- a/tools/libxc/include/xenctrl.h
> +++ b/tools/libxc/include/xenctrl.h
> @@ -2626,9 +2626,9 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
> int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
> xc_psr_cat_type type, uint32_t target,
> uint64_t *data);
> -int xc_psr_cat_get_l3_info(xc_interface *xch, uint32_t socket,
> - uint32_t *cos_max, uint32_t *cbm_len,
> - bool *cdp_enabled);
> +int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
> + uint32_t *cos_max, uint32_t *cbm_len,
> + bool *cdp_enabled);
>
> int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
> int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
> diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
> index 43b3286..7af17d9 100644
> --- a/tools/libxc/xc_psr.c
> +++ b/tools/libxc/xc_psr.c
> @@ -317,24 +317,40 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
> return rc;
> }
>
> -int xc_psr_cat_get_l3_info(xc_interface *xch, uint32_t socket,
> - uint32_t *cos_max, uint32_t *cbm_len,
> - bool *cdp_enabled)
> +int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
> + uint32_t *cos_max, uint32_t *cbm_len, bool *cdp_enabled)
> {
> - int rc;
> + int rc = -1;
> DECLARE_SYSCTL;
>
> sysctl.cmd = XEN_SYSCTL_psr_cat_op;
> - sysctl.u.psr_cat_op.cmd = XEN_SYSCTL_PSR_CAT_get_l3_info;
> sysctl.u.psr_cat_op.target = socket;
>
> - rc = xc_sysctl(xch, &sysctl);
> - if ( !rc )
> - {
> - *cos_max = sysctl.u.psr_cat_op.u.l3_info.cos_max;
> - *cbm_len = sysctl.u.psr_cat_op.u.l3_info.cbm_len;
> - *cdp_enabled = sysctl.u.psr_cat_op.u.l3_info.flags &
> - XEN_SYSCTL_PSR_CAT_L3_CDP;
> + switch ( lvl ) {
> + case 2:
> + sysctl.u.psr_cat_op.cmd = XEN_SYSCTL_PSR_CAT_get_l2_info;
> + rc = xc_sysctl(xch, &sysctl);
> + if ( !rc )
> + {
> + *cos_max = sysctl.u.psr_cat_op.u.l2_info.cos_max;
> + *cbm_len = sysctl.u.psr_cat_op.u.l2_info.cbm_len;
> + *cdp_enabled = false;
> + }
> + break;
> + case 3:
> + sysctl.u.psr_cat_op.cmd = XEN_SYSCTL_PSR_CAT_get_l3_info;
> + rc = xc_sysctl(xch, &sysctl);
> + if ( !rc )
> + {
> + *cos_max = sysctl.u.psr_cat_op.u.l3_info.cos_max;
> + *cbm_len = sysctl.u.psr_cat_op.u.l3_info.cbm_len;
> + *cdp_enabled = sysctl.u.psr_cat_op.u.l3_info.flags &
> + XEN_SYSCTL_PSR_CAT_L3_CDP;
> + }
> + break;
> + default:
> + errno = EOPNOTSUPP;
> + return rc;
There is no need for "return rc".
> }
>
> return rc;
> diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> index acbf476..9233b0f 100644
> --- a/tools/libxl/libxl.h
> +++ b/tools/libxl/libxl.h
> @@ -904,6 +904,13 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const libxl_mac *src);
> * If this is defined, the Code and Data Prioritization feature is supported.
> */
> #define LIBXL_HAVE_PSR_CDP 1
> +
> +/*
> + * LIBXL_HAVE_PSR_L2_CAT
> + *
> + * If this is defined, the L2 Cache Allocation Technology feature is supported.
> + */
> +#define LIBXL_HAVE_PSR_L2_CAT 1
> #endif
>
> /*
> @@ -2159,8 +2166,8 @@ int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
> * On success, the function returns an array of elements in 'info',
> * and the length in 'nr'.
> */
> -int libxl_psr_cat_get_l3_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
> - int *nr);
> +int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
> + int *nr, unsigned int lvl);
You can't delete old public functions. They will basically stay there
forever.
Please refactor the code to support both the new functions and the old.
In this particular case, it would be easy to implement _get_l3_info with
_get_info.
Wei.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 21/24] tools: L2 CAT: support get HW info for L2 CAT.
2017-01-06 12:04 ` Wei Liu
@ 2017-01-09 1:19 ` Yi Sun
2017-01-09 8:31 ` Jan Beulich
0 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2017-01-09 1:19 UTC (permalink / raw)
To: Wei Liu
Cc: andrew.cooper3, dario.faggioli, he.chen, ian.jackson, mengxu,
jbeulich, chao.p.peng, xen-devel
On 17-01-06 12:04:36, Wei Liu wrote:
> On Wed, Dec 14, 2016 at 12:08:01PM +0800, Yi Sun wrote:
> > This patch implements xl/xc changes to support get HW info
> > for L2 CAT.
> >
> > 'xl psr-hwinfo' is updated to show both L3 CAT and L2 CAT
> > info.
> >
> > Example(on machine which only supports L2 CAT):
> > Cache Monitoring Technology (CMT):
> > Enabled : 0
> > Cache Allocation Technology (CAT): L3
> > libxl: error: libxl_psr.c:100:libxl__psr_cat_log_err_msg: CAT is not enabled on the socket: No such file or directory
> > Failed to get l3 cat info
>
> What is this?
>
The psr-hwinfo prints all features out by design. This test was executed on a
machine which only supports L2 CAT. So, the L3 CAT not supported info will be
printed out.
> > Signed-off-by: He Chen <he.chen@linux.intel.com>
> > Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> > ---
> > tools/libxc/include/xenctrl.h | 6 +++---
> > tools/libxc/xc_psr.c | 40 +++++++++++++++++++++++-----------
> > tools/libxl/libxl.h | 11 ++++++++--
> > tools/libxl/libxl_psr.c | 8 +++----
> > tools/libxl/xl_cmdimpl.c | 50 +++++++++++++++++++++++++++++++++++--------
> > 5 files changed, 85 insertions(+), 30 deletions(-)
> >
> > diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
> > index 43b3286..7af17d9 100644
> > --- a/tools/libxc/xc_psr.c
> > +++ b/tools/libxc/xc_psr.c
> > + if ( !rc )
> > + {
> > + *cos_max = sysctl.u.psr_cat_op.u.l3_info.cos_max;
> > + *cbm_len = sysctl.u.psr_cat_op.u.l3_info.cbm_len;
> > + *cdp_enabled = sysctl.u.psr_cat_op.u.l3_info.flags &
> > + XEN_SYSCTL_PSR_CAT_L3_CDP;
> > + }
> > + break;
> > + default:
> > + errno = EOPNOTSUPP;
> > + return rc;
>
> There is no need for "return rc".
>
Thanks!
> > }
> >
> > return rc;
> > diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> > index acbf476..9233b0f 100644
> > --- a/tools/libxl/libxl.h
> > +++ b/tools/libxl/libxl.h
> > @@ -904,6 +904,13 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const libxl_mac *src);
> > * If this is defined, the Code and Data Prioritization feature is supported.
> > */
> > #define LIBXL_HAVE_PSR_CDP 1
> > +
> > +/*
> > + * LIBXL_HAVE_PSR_L2_CAT
> > + *
> > + * If this is defined, the L2 Cache Allocation Technology feature is supported.
> > + */
> > +#define LIBXL_HAVE_PSR_L2_CAT 1
> > #endif
> >
> > /*
> > @@ -2159,8 +2166,8 @@ int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
> > * On success, the function returns an array of elements in 'info',
> > * and the length in 'nr'.
> > */
> > -int libxl_psr_cat_get_l3_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
> > - int *nr);
> > +int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
> > + int *nr, unsigned int lvl);
>
> You can't delete old public functions. They will basically stay there
> forever.
>
> Please refactor the code to support both the new functions and the old.
> In this particular case, it would be easy to implement _get_l3_info with
> _get_info.
>
> Wei.
Got it. I will refactor codes to keep the old interfaces. Thank you!
Sun Yi
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 21/24] tools: L2 CAT: support get HW info for L2 CAT.
2017-01-09 1:19 ` Yi Sun
@ 2017-01-09 8:31 ` Jan Beulich
2017-01-09 9:26 ` Wei Liu
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2017-01-09 8:31 UTC (permalink / raw)
To: Yi Sun
Cc: Wei Liu, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 09.01.17 at 02:19, <yi.y.sun@linux.intel.com> wrote:
> On 17-01-06 12:04:36, Wei Liu wrote:
>> On Wed, Dec 14, 2016 at 12:08:01PM +0800, Yi Sun wrote:
>> > This patch implements xl/xc changes to support get HW info
>> > for L2 CAT.
>> >
>> > 'xl psr-hwinfo' is updated to show both L3 CAT and L2 CAT
>> > info.
>> >
>> > Example(on machine which only supports L2 CAT):
>> > Cache Monitoring Technology (CMT):
>> > Enabled : 0
>> > Cache Allocation Technology (CAT): L3
>> > libxl: error: libxl_psr.c:100:libxl__psr_cat_log_err_msg: CAT is not enabled on the socket: No such file or directory
>> > Failed to get l3 cat info
>>
>> What is this?
>>
> The psr-hwinfo prints all features out by design. This test was executed on a
> machine which only supports L2 CAT. So, the L3 CAT not supported info will be
> printed out.
But there is an indication of error(s), which shouldn't be the case just
because of non-present hardware. The hypercall may return failure
in such a case of course, but this shouldn't propagate to error/failure
messages being printed (but at the same time you need to make sure
you don't hide actual errors). Perhaps there's a call missing querying
host capabilities (based on the result of which further info queries
would be issued)?
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* Re: [PATCH v4 21/24] tools: L2 CAT: support get HW info for L2 CAT.
2017-01-09 8:31 ` Jan Beulich
@ 2017-01-09 9:26 ` Wei Liu
2017-01-10 8:00 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Wei Liu @ 2017-01-09 9:26 UTC (permalink / raw)
To: Jan Beulich
Cc: Yi Sun, xen-devel, he.chen, andrew.cooper3, dario.faggioli,
ian.jackson, mengxu, chao.p.peng, Wei Liu
On Mon, Jan 09, 2017 at 01:31:35AM -0700, Jan Beulich wrote:
> >>> On 09.01.17 at 02:19, <yi.y.sun@linux.intel.com> wrote:
> > On 17-01-06 12:04:36, Wei Liu wrote:
> >> On Wed, Dec 14, 2016 at 12:08:01PM +0800, Yi Sun wrote:
> >> > This patch implements xl/xc changes to support get HW info
> >> > for L2 CAT.
> >> >
> >> > 'xl psr-hwinfo' is updated to show both L3 CAT and L2 CAT
> >> > info.
> >> >
> >> > Example(on machine which only supports L2 CAT):
> >> > Cache Monitoring Technology (CMT):
> >> > Enabled : 0
> >> > Cache Allocation Technology (CAT): L3
> >> > libxl: error: libxl_psr.c:100:libxl__psr_cat_log_err_msg: CAT is not enabled on the socket: No such file or directory
> >> > Failed to get l3 cat info
> >>
> >> What is this?
> >>
> > The psr-hwinfo prints all features out by design. This test was executed on a
> > machine which only supports L2 CAT. So, the L3 CAT not supported info will be
> > printed out.
>
> But there is an indication of error(s), which shouldn't be the case just
> because of non-present hardware. The hypercall may return failure
> in such a case of course, but this shouldn't propagate to error/failure
> messages being printed (but at the same time you need to make sure
> you don't hide actual errors). Perhaps there's a call missing querying
> host capabilities (based on the result of which further info queries
> would be issued)?
>
This is what I thought as well.
> Jan
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* Re: [PATCH v4 21/24] tools: L2 CAT: support get HW info for L2 CAT.
2017-01-09 9:26 ` Wei Liu
@ 2017-01-10 8:00 ` Yi Sun
2017-01-10 8:46 ` Jan Beulich
0 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2017-01-10 8:00 UTC (permalink / raw)
To: Wei Liu
Cc: he.chen, andrew.cooper3, dario.faggioli, ian.jackson, mengxu,
Jan Beulich, xen-devel, chao.p.peng
On 17-01-09 09:26:17, Wei Liu wrote:
> On Mon, Jan 09, 2017 at 01:31:35AM -0700, Jan Beulich wrote:
> > >>> On 09.01.17 at 02:19, <yi.y.sun@linux.intel.com> wrote:
> > > On 17-01-06 12:04:36, Wei Liu wrote:
> > >> On Wed, Dec 14, 2016 at 12:08:01PM +0800, Yi Sun wrote:
> > >> > This patch implements xl/xc changes to support get HW info
> > >> > for L2 CAT.
> > >> >
> > >> > 'xl psr-hwinfo' is updated to show both L3 CAT and L2 CAT
> > >> > info.
> > >> >
> > >> > Example(on machine which only supports L2 CAT):
> > >> > Cache Monitoring Technology (CMT):
> > >> > Enabled : 0
> > >> > Cache Allocation Technology (CAT): L3
> > >> > libxl: error: libxl_psr.c:100:libxl__psr_cat_log_err_msg: CAT is not enabled on the socket: No such file or directory
> > >> > Failed to get l3 cat info
> > >>
> > >> What is this?
> > >>
> > > The psr-hwinfo prints all features out by design. This test was executed on a
> > > machine which only supports L2 CAT. So, the L3 CAT not supported info will be
> > > printed out.
> >
> > But there is an indication of error(s), which shouldn't be the case just
> > because of non-present hardware. The hypercall may return failure
> > in such a case of course, but this shouldn't propagate to error/failure
> > messages being printed (but at the same time you need to make sure
> > you don't hide actual errors). Perhaps there's a call missing querying
> > host capabilities (based on the result of which further info queries
> > would be issued)?
> >
>
> This is what I thought as well.
>
> > Jan
> >
I considered to add a sysctl interface to get feature mask back before.
Then, the application can print enabled feature list out in psr-hwinfo.
But current psr-hwinfo command implementation can have same effect. It
gets all features HW info back. For not enabled feature, such error info
is printed. Although it is not user friendly, it reduces one hypercall.
So, I have below two options:
1) Create a new sysctl interface to get feature mask and get HW info
according to it.
2) Modify current psr-hwinfo codes only to show enabled feature HW info
but not show error logs. Of course the libxl library interface still
returns error to application.
Which one do you prefer? Thanks!
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* Re: [PATCH v4 21/24] tools: L2 CAT: support get HW info for L2 CAT.
2017-01-10 8:00 ` Yi Sun
@ 2017-01-10 8:46 ` Jan Beulich
2017-01-10 9:01 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Jan Beulich @ 2017-01-10 8:46 UTC (permalink / raw)
To: Yi Sun
Cc: Wei Liu, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, xen-devel, chao.p.peng
>>> On 10.01.17 at 09:00, <yi.y.sun@linux.intel.com> wrote:
> On 17-01-09 09:26:17, Wei Liu wrote:
>> On Mon, Jan 09, 2017 at 01:31:35AM -0700, Jan Beulich wrote:
>> > >>> On 09.01.17 at 02:19, <yi.y.sun@linux.intel.com> wrote:
>> > > On 17-01-06 12:04:36, Wei Liu wrote:
>> > >> On Wed, Dec 14, 2016 at 12:08:01PM +0800, Yi Sun wrote:
>> > >> > This patch implements xl/xc changes to support get HW info
>> > >> > for L2 CAT.
>> > >> >
>> > >> > 'xl psr-hwinfo' is updated to show both L3 CAT and L2 CAT
>> > >> > info.
>> > >> >
>> > >> > Example(on machine which only supports L2 CAT):
>> > >> > Cache Monitoring Technology (CMT):
>> > >> > Enabled : 0
>> > >> > Cache Allocation Technology (CAT): L3
>> > >> > libxl: error: libxl_psr.c:100:libxl__psr_cat_log_err_msg: CAT is not enabled on the socket: No such file or directory
>> > >> > Failed to get l3 cat info
>> > >>
>> > >> What is this?
>> > >>
>> > > The psr-hwinfo prints all features out by design. This test was executed on a
>> > > machine which only supports L2 CAT. So, the L3 CAT not supported info will be
>> > > printed out.
>> >
>> > But there is an indication of error(s), which shouldn't be the case just
>> > because of non-present hardware. The hypercall may return failure
>> > in such a case of course, but this shouldn't propagate to error/failure
>> > messages being printed (but at the same time you need to make sure
>> > you don't hide actual errors). Perhaps there's a call missing querying
>> > host capabilities (based on the result of which further info queries
>> > would be issued)?
>> >
>>
>> This is what I thought as well.
>>
> I considered to add a sysctl interface to get feature mask back before.
> Then, the application can print enabled feature list out in psr-hwinfo.
>
> But current psr-hwinfo command implementation can have same effect. It
> gets all features HW info back. For not enabled feature, such error info
> is printed. Although it is not user friendly, it reduces one hypercall.
>
> So, I have below two options:
> 1) Create a new sysctl interface to get feature mask and get HW info
> according to it.
> 2) Modify current psr-hwinfo codes only to show enabled feature HW info
> but not show error logs. Of course the libxl library interface still
> returns error to application.
>
> Which one do you prefer? Thanks!
Well, afaic I'd prefer any variant without new hypercall, as long as
the bogus error messages go away without hiding any actual errors.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* Re: [PATCH v4 21/24] tools: L2 CAT: support get HW info for L2 CAT.
2017-01-10 8:46 ` Jan Beulich
@ 2017-01-10 9:01 ` Yi Sun
0 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2017-01-10 9:01 UTC (permalink / raw)
To: Jan Beulich
Cc: Wei Liu, he.chen, andrew.cooper3, dario.faggioli, ian.jackson,
mengxu, chao.p.peng, xen-devel
On 17-01-10 01:46:17, Jan Beulich wrote:
> >>> On 10.01.17 at 09:00, <yi.y.sun@linux.intel.com> wrote:
> > On 17-01-09 09:26:17, Wei Liu wrote:
> >> On Mon, Jan 09, 2017 at 01:31:35AM -0700, Jan Beulich wrote:
> >> > >>> On 09.01.17 at 02:19, <yi.y.sun@linux.intel.com> wrote:
> >> > > On 17-01-06 12:04:36, Wei Liu wrote:
> >> > >> On Wed, Dec 14, 2016 at 12:08:01PM +0800, Yi Sun wrote:
> >> > >> > This patch implements xl/xc changes to support get HW info
> >> > >> > for L2 CAT.
> >> > >> >
> >> > >> > 'xl psr-hwinfo' is updated to show both L3 CAT and L2 CAT
> >> > >> > info.
> >> > >> >
> >> > >> > Example(on machine which only supports L2 CAT):
> >> > >> > Cache Monitoring Technology (CMT):
> >> > >> > Enabled : 0
> >> > >> > Cache Allocation Technology (CAT): L3
> >> > >> > libxl: error: libxl_psr.c:100:libxl__psr_cat_log_err_msg: CAT is not enabled on the socket: No such file or directory
> >> > >> > Failed to get l3 cat info
> >> > >>
> >> > >> What is this?
> >> > >>
> >> > > The psr-hwinfo prints all features out by design. This test was executed on a
> >> > > machine which only supports L2 CAT. So, the L3 CAT not supported info will be
> >> > > printed out.
> >> >
> >> > But there is an indication of error(s), which shouldn't be the case just
> >> > because of non-present hardware. The hypercall may return failure
> >> > in such a case of course, but this shouldn't propagate to error/failure
> >> > messages being printed (but at the same time you need to make sure
> >> > you don't hide actual errors). Perhaps there's a call missing querying
> >> > host capabilities (based on the result of which further info queries
> >> > would be issued)?
> >> >
> >>
> >> This is what I thought as well.
> >>
> > I considered to add a sysctl interface to get feature mask back before.
> > Then, the application can print enabled feature list out in psr-hwinfo.
> >
> > But current psr-hwinfo command implementation can have same effect. It
> > gets all features HW info back. For not enabled feature, such error info
> > is printed. Although it is not user friendly, it reduces one hypercall.
> >
> > So, I have below two options:
> > 1) Create a new sysctl interface to get feature mask and get HW info
> > according to it.
> > 2) Modify current psr-hwinfo codes only to show enabled feature HW info
> > but not show error logs. Of course the libxl library interface still
> > returns error to application.
> >
> > Which one do you prefer? Thanks!
>
> Well, afaic I'd prefer any variant without new hypercall, as long as
> the bogus error messages go away without hiding any actual errors.
>
> Jan
Ok, thank you!
Sun Yi
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v4 22/24] tools: L2 CAT: support show cbm for L2 CAT.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (20 preceding siblings ...)
2016-12-14 4:08 ` [PATCH v4 21/24] tools: L2 CAT: support get HW info for L2 CAT Yi Sun
@ 2016-12-14 4:08 ` Yi Sun
2017-01-06 12:04 ` Wei Liu
2016-12-14 4:08 ` [PATCH v4 23/24] tools: L2 CAT: support set " Yi Sun
2016-12-14 4:08 ` [PATCH v4 24/24] docs: add L2 CAT description in docs Yi Sun
23 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:08 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements changes in xl/xc changes to support
showing CBM of L2 CAT.
The new level option is introduced to original CAT showing
command in order to show CBM for specified level CAT.
- 'xl psr-cat-show' is updated to show CBM of a domain
according to input cache level.
Examples:
root@:~$ xl psr-cat-show -l2 1
Socket ID : 0
Default CBM : 0xff
ID NAME CBM
1 ubuntu14 0x7f
Signed-off-by: He Chen <he.chen@linux.intel.com>
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
tools/libxc/include/xenctrl.h | 1 +
tools/libxc/xc_psr.c | 3 ++
tools/libxl/libxl_types.idl | 1 +
tools/libxl/xl_cmdimpl.c | 81 ++++++++++++++++++++++++++++---------------
tools/libxl/xl_cmdtable.c | 3 +-
5 files changed, 60 insertions(+), 29 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 5b7df22..c1675d8 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2602,6 +2602,7 @@ enum xc_psr_cat_type {
XC_PSR_CAT_L3_CBM = 1,
XC_PSR_CAT_L3_CBM_CODE = 2,
XC_PSR_CAT_L3_CBM_DATA = 3,
+ XC_PSR_CAT_L2_CBM = 4,
};
typedef enum xc_psr_cat_type xc_psr_cat_type;
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 7af17d9..fe494b0 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -299,6 +299,9 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
case XC_PSR_CAT_L3_CBM_DATA:
cmd = XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA;
break;
+ case XC_PSR_CAT_L2_CBM:
+ cmd = XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM;
+ break;
default:
errno = EINVAL;
return -1;
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index a32c751..944b7d4 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -898,6 +898,7 @@ libxl_psr_cbm_type = Enumeration("psr_cbm_type", [
(1, "L3_CBM"),
(2, "L3_CBM_CODE"),
(3, "L3_CBM_DATA"),
+ (4, "L2_CBM"),
])
libxl_psr_cat_info = Struct("psr_cat_info", [
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index c210b05..4d6f32a 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -9381,7 +9381,7 @@ static void psr_cat_print_one_domain_cbm_type(uint32_t domid, uint32_t socketid,
}
static void psr_cat_print_one_domain_cbm(uint32_t domid, uint32_t socketid,
- bool cdp_enabled)
+ bool cdp_enabled, unsigned int lvl)
{
char *domain_name;
@@ -9389,27 +9389,38 @@ static void psr_cat_print_one_domain_cbm(uint32_t domid, uint32_t socketid,
printf("%5d%25s", domid, domain_name);
free(domain_name);
- if (!cdp_enabled) {
- psr_cat_print_one_domain_cbm_type(domid, socketid,
- LIBXL_PSR_CBM_TYPE_L3_CBM);
- } else {
- psr_cat_print_one_domain_cbm_type(domid, socketid,
- LIBXL_PSR_CBM_TYPE_L3_CBM_CODE);
+ switch (lvl) {
+ case 3:
+ if (!cdp_enabled) {
+ psr_cat_print_one_domain_cbm_type(domid, socketid,
+ LIBXL_PSR_CBM_TYPE_L3_CBM);
+ } else {
+ psr_cat_print_one_domain_cbm_type(domid, socketid,
+ LIBXL_PSR_CBM_TYPE_L3_CBM_CODE);
+ psr_cat_print_one_domain_cbm_type(domid, socketid,
+ LIBXL_PSR_CBM_TYPE_L3_CBM_DATA);
+ }
+ break;
+ case 2:
psr_cat_print_one_domain_cbm_type(domid, socketid,
- LIBXL_PSR_CBM_TYPE_L3_CBM_DATA);
+ LIBXL_PSR_CBM_TYPE_L2_CBM);
+ break;
+ default:
+ printf("Input lvl %d is wrong!", lvl);
+ break;
}
printf("\n");
}
static int psr_cat_print_domain_cbm(uint32_t domid, uint32_t socketid,
- bool cdp_enabled)
+ bool cdp_enabled, unsigned int lvl)
{
int i, nr_domains;
libxl_dominfo *list;
if (domid != INVALID_DOMID) {
- psr_cat_print_one_domain_cbm(domid, socketid, cdp_enabled);
+ psr_cat_print_one_domain_cbm(domid, socketid, cdp_enabled, lvl);
return 0;
}
@@ -9419,49 +9430,55 @@ static int psr_cat_print_domain_cbm(uint32_t domid, uint32_t socketid,
}
for (i = 0; i < nr_domains; i++)
- psr_cat_print_one_domain_cbm(list[i].domid, socketid, cdp_enabled);
+ psr_cat_print_one_domain_cbm(list[i].domid, socketid, cdp_enabled, lvl);
libxl_dominfo_list_free(list, nr_domains);
return 0;
}
-static int psr_cat_print_socket(uint32_t domid, libxl_psr_cat_info *info)
+static int psr_cat_print_socket(uint32_t domid, libxl_psr_cat_info *info,
+ unsigned int lvl)
{
int rc;
uint32_t l3_cache_size;
- rc = libxl_psr_cmt_get_l3_cache_size(ctx, info->id, &l3_cache_size);
- if (rc) {
- fprintf(stderr, "Failed to get l3 cache size for socket:%d\n",
- info->id);
- return -1;
+ printf("%-16s: %u\n", "Socket ID", info->id);
+
+ /* So far, CMT only supports L3 cache. */
+ if (lvl == 3)
+ {
+ rc = libxl_psr_cmt_get_l3_cache_size(ctx, info->id, &l3_cache_size);
+ if (rc) {
+ fprintf(stderr, "Failed to get l3 cache size for socket:%d\n",
+ info->id);
+ return -1;
+ }
+ printf("%-16s: %uKB\n", "L3 Cache", l3_cache_size);
}
- printf("%-16s: %u\n", "Socket ID", info->id);
- printf("%-16s: %uKB\n", "L3 Cache", l3_cache_size);
printf("%-16s: %#llx\n", "Default CBM", (1ull << info->cbm_len) - 1);
if (info->cdp_enabled)
printf("%5s%25s%16s%16s\n", "ID", "NAME", "CBM (code)", "CBM (data)");
else
printf("%5s%25s%16s\n", "ID", "NAME", "CBM");
- return psr_cat_print_domain_cbm(domid, info->id, info->cdp_enabled);
+ return psr_cat_print_domain_cbm(domid, info->id, info->cdp_enabled, lvl);
}
-static int psr_cat_show(uint32_t domid)
+static int psr_cat_show(uint32_t domid, unsigned int lvl)
{
int i, nr;
int rc;
libxl_psr_cat_info *info;
- rc = libxl_psr_cat_get_info(ctx, &info, &nr, 3);
+ rc = libxl_psr_cat_get_info(ctx, &info, &nr, lvl);
if (rc) {
- fprintf(stderr, "Failed to get cat info\n");
+ fprintf(stderr, "Failed to get %s cat info\n", (lvl == 3)?"L3":"L2");
return rc;
}
for (i = 0; i < nr; i++) {
- rc = psr_cat_print_socket(domid, info + i);
+ rc = psr_cat_print_socket(domid, info + i, lvl);
if (rc)
goto out;
}
@@ -9574,11 +9591,19 @@ int main_psr_cat_cbm_set(int argc, char **argv)
int main_psr_cat_show(int argc, char **argv)
{
- int opt;
+ int opt = 0;
uint32_t domid;
+ unsigned int lvl = 3;
- SWITCH_FOREACH_OPT(opt, "", NULL, "psr-cat-show", 0) {
- /* No options */
+ static struct option opts[] = {
+ {"level", 1, 0, 'l'},
+ COMMON_LONG_OPTS
+ };
+
+ SWITCH_FOREACH_OPT(opt, "l:", opts, "psr-cat-show", 0) {
+ case 'l':
+ lvl = atoi(optarg);
+ break;
}
if (optind >= argc)
@@ -9590,7 +9615,7 @@ int main_psr_cat_show(int argc, char **argv)
return 2;
}
- return psr_cat_show(domid);
+ return psr_cat_show(domid, lvl);
}
int main_psr_hwinfo(int argc, char **argv)
diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c
index 588d5d9..c5fbad4 100644
--- a/tools/libxl/xl_cmdtable.c
+++ b/tools/libxl/xl_cmdtable.c
@@ -556,7 +556,8 @@ struct cmd_spec cmd_table[] = {
{ "psr-cat-show",
&main_psr_cat_show, 0, 1,
"Show Cache Allocation Technology information",
- "<Domain>",
+ "[options] <Domain>",
+ "-l <level> Specify the cache level to process, otherwise L3 cache is processed\n"
},
#endif
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* Re: [PATCH v4 22/24] tools: L2 CAT: support show cbm for L2 CAT.
2016-12-14 4:08 ` [PATCH v4 22/24] tools: L2 CAT: support show cbm " Yi Sun
@ 2017-01-06 12:04 ` Wei Liu
2017-01-09 1:24 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Wei Liu @ 2017-01-06 12:04 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
mengxu, jbeulich, chao.p.peng, xen-devel
On Wed, Dec 14, 2016 at 12:08:02PM +0800, Yi Sun wrote:
> This patch implements changes in xl/xc changes to support
> showing CBM of L2 CAT.
>
> The new level option is introduced to original CAT showing
> command in order to show CBM for specified level CAT.
> - 'xl psr-cat-show' is updated to show CBM of a domain
> according to input cache level.
>
> Examples:
> root@:~$ xl psr-cat-show -l2 1
> Socket ID : 0
> Default CBM : 0xff
> ID NAME CBM
> 1 ubuntu14 0x7f
>
> Signed-off-by: He Chen <he.chen@linux.intel.com>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> ---
> tools/libxc/include/xenctrl.h | 1 +
> tools/libxc/xc_psr.c | 3 ++
> tools/libxl/libxl_types.idl | 1 +
> tools/libxl/xl_cmdimpl.c | 81 ++++++++++++++++++++++++++++---------------
> tools/libxl/xl_cmdtable.c | 3 +-
> 5 files changed, 60 insertions(+), 29 deletions(-)
>
> diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
> index 5b7df22..c1675d8 100644
> --- a/tools/libxc/include/xenctrl.h
> +++ b/tools/libxc/include/xenctrl.h
> @@ -2602,6 +2602,7 @@ enum xc_psr_cat_type {
> XC_PSR_CAT_L3_CBM = 1,
> XC_PSR_CAT_L3_CBM_CODE = 2,
> XC_PSR_CAT_L3_CBM_DATA = 3,
> + XC_PSR_CAT_L2_CBM = 4,
> };
> typedef enum xc_psr_cat_type xc_psr_cat_type;
>
> diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
> index 7af17d9..fe494b0 100644
> --- a/tools/libxc/xc_psr.c
> +++ b/tools/libxc/xc_psr.c
> @@ -299,6 +299,9 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
> case XC_PSR_CAT_L3_CBM_DATA:
> cmd = XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA;
> break;
> + case XC_PSR_CAT_L2_CBM:
> + cmd = XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM;
> + break;
> default:
> errno = EINVAL;
> return -1;
> diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> index a32c751..944b7d4 100644
> --- a/tools/libxl/libxl_types.idl
> +++ b/tools/libxl/libxl_types.idl
> @@ -898,6 +898,7 @@ libxl_psr_cbm_type = Enumeration("psr_cbm_type", [
> (1, "L3_CBM"),
> (2, "L3_CBM_CODE"),
> (3, "L3_CBM_DATA"),
> + (4, "L2_CBM"),
Need a LIBXL_HAVE macro.
And a very stupid question: where is the libxl code to use this?
Wei.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 22/24] tools: L2 CAT: support show cbm for L2 CAT.
2017-01-06 12:04 ` Wei Liu
@ 2017-01-09 1:24 ` Yi Sun
2017-01-09 10:08 ` Wei Liu
0 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2017-01-09 1:24 UTC (permalink / raw)
To: Wei Liu
Cc: andrew.cooper3, dario.faggioli, he.chen, ian.jackson, mengxu,
jbeulich, chao.p.peng, xen-devel
On 17-01-06 12:04:43, Wei Liu wrote:
> On Wed, Dec 14, 2016 at 12:08:02PM +0800, Yi Sun wrote:
> > - 'xl psr-cat-show' is updated to show CBM of a domain
> > according to input cache level.
> >
> > Examples:
> > root@:~$ xl psr-cat-show -l2 1
> > Socket ID : 0
> > Default CBM : 0xff
> > ID NAME CBM
> > 1 ubuntu14 0x7f
> >
> > Signed-off-by: He Chen <he.chen@linux.intel.com>
> > Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> > ---
> > tools/libxc/include/xenctrl.h | 1 +
> > tools/libxc/xc_psr.c | 3 ++
> > tools/libxl/libxl_types.idl | 1 +
> > tools/libxl/xl_cmdimpl.c | 81 ++++++++++++++++++++++++++++---------------
> > tools/libxl/xl_cmdtable.c | 3 +-
> > 5 files changed, 60 insertions(+), 29 deletions(-)
> >
> > diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> > index a32c751..944b7d4 100644
> > --- a/tools/libxl/libxl_types.idl
> > +++ b/tools/libxl/libxl_types.idl
> > @@ -898,6 +898,7 @@ libxl_psr_cbm_type = Enumeration("psr_cbm_type", [
> > (1, "L3_CBM"),
> > (2, "L3_CBM_CODE"),
> > (3, "L3_CBM_DATA"),
> > + (4, "L2_CBM"),
>
> Need a LIBXL_HAVE macro.
>
It has been defined in pathc 21, like below.
tools/libxl/libxl.h
#define LIBXL_HAVE_PSR_L2_CAT 1
> And a very stupid question: where is the libxl code to use this?
>
> Wei.
L2_CBM is used in xl_cmdimpl.c, like below.
static void psr_cat_print_one_domain_cbm(uint32_t domid, uint32_t socketid,
bool cdp_enabled, unsigned int lvl)
{
......
case 2:
psr_cat_print_one_domain_cbm_type(domid, socketid,
LIBXL_PSR_CBM_TYPE_L2_CBM);
......
}
int main_psr_cat_cbm_set(int argc, char **argv)
{
......
if (lvl == 2)
type = LIBXL_PSR_CBM_TYPE_L2_CBM;
......
}
Thanks,
Sun Yi
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 22/24] tools: L2 CAT: support show cbm for L2 CAT.
2017-01-09 1:24 ` Yi Sun
@ 2017-01-09 10:08 ` Wei Liu
2017-01-10 7:47 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Wei Liu @ 2017-01-09 10:08 UTC (permalink / raw)
To: Yi Sun
Cc: Wei Liu, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
mengxu, jbeulich, chao.p.peng, xen-devel
On Mon, Jan 09, 2017 at 09:24:54AM +0800, Yi Sun wrote:
> On 17-01-06 12:04:43, Wei Liu wrote:
> > On Wed, Dec 14, 2016 at 12:08:02PM +0800, Yi Sun wrote:
> > > - 'xl psr-cat-show' is updated to show CBM of a domain
> > > according to input cache level.
> > >
> > > Examples:
> > > root@:~$ xl psr-cat-show -l2 1
> > > Socket ID : 0
> > > Default CBM : 0xff
> > > ID NAME CBM
> > > 1 ubuntu14 0x7f
> > >
> > > Signed-off-by: He Chen <he.chen@linux.intel.com>
> > > Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> > > ---
> > > tools/libxc/include/xenctrl.h | 1 +
> > > tools/libxc/xc_psr.c | 3 ++
> > > tools/libxl/libxl_types.idl | 1 +
> > > tools/libxl/xl_cmdimpl.c | 81 ++++++++++++++++++++++++++++---------------
> > > tools/libxl/xl_cmdtable.c | 3 +-
> > > 5 files changed, 60 insertions(+), 29 deletions(-)
> > >
> > > diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> > > index a32c751..944b7d4 100644
> > > --- a/tools/libxl/libxl_types.idl
> > > +++ b/tools/libxl/libxl_types.idl
> > > @@ -898,6 +898,7 @@ libxl_psr_cbm_type = Enumeration("psr_cbm_type", [
> > > (1, "L3_CBM"),
> > > (2, "L3_CBM_CODE"),
> > > (3, "L3_CBM_DATA"),
> > > + (4, "L2_CBM"),
> >
> > Need a LIBXL_HAVE macro.
> >
> It has been defined in pathc 21, like below.
> tools/libxl/libxl.h
> #define LIBXL_HAVE_PSR_L2_CAT 1
>
In that case please move this to patch #21.
Wei.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread* Re: [PATCH v4 22/24] tools: L2 CAT: support show cbm for L2 CAT.
2017-01-09 10:08 ` Wei Liu
@ 2017-01-10 7:47 ` Yi Sun
0 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2017-01-10 7:47 UTC (permalink / raw)
To: Wei Liu
Cc: andrew.cooper3, dario.faggioli, he.chen, ian.jackson, mengxu,
jbeulich, chao.p.peng, xen-devel
On 17-01-09 10:08:04, Wei Liu wrote:
> On Mon, Jan 09, 2017 at 09:24:54AM +0800, Yi Sun wrote:
> > On 17-01-06 12:04:43, Wei Liu wrote:
> > > On Wed, Dec 14, 2016 at 12:08:02PM +0800, Yi Sun wrote:
> > > > diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> > > > index a32c751..944b7d4 100644
> > > > --- a/tools/libxl/libxl_types.idl
> > > > +++ b/tools/libxl/libxl_types.idl
> > > > @@ -898,6 +898,7 @@ libxl_psr_cbm_type = Enumeration("psr_cbm_type", [
> > > > (1, "L3_CBM"),
> > > > (2, "L3_CBM_CODE"),
> > > > (3, "L3_CBM_DATA"),
> > > > + (4, "L2_CBM"),
> > >
> > > Need a LIBXL_HAVE macro.
> > >
> > It has been defined in pathc 21, like below.
> > tools/libxl/libxl.h
> > #define LIBXL_HAVE_PSR_L2_CAT 1
> >
>
> In that case please move this to patch #21.
>
> Wei.
Thank you! Will move it to previous patch with the macro together.
BRs,
Sun Yi
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v4 23/24] tools: L2 CAT: support set cbm for L2 CAT.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (21 preceding siblings ...)
2016-12-14 4:08 ` [PATCH v4 22/24] tools: L2 CAT: support show cbm " Yi Sun
@ 2016-12-14 4:08 ` Yi Sun
2017-01-06 12:04 ` Wei Liu
2016-12-14 4:08 ` [PATCH v4 24/24] docs: add L2 CAT description in docs Yi Sun
23 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:08 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
This patch implements the xl/xc changes to support set CBM
for L2 CAT.
The new level option is introduced to original CAT setting
command in order to set CBM for specified level CAT.
- 'xl psr-cat-cbm-set' is updated to set cache capacity
bitmasks(CBM) for a domain according to input cache level.
root@:~$ xl psr-cat-cbm-set -l2 1 0x7f
root@:~$ xl psr-cat-show -l2 1
Socket ID : 0
Default CBM : 0xff
ID NAME CBM
1 ubuntu14 0x7f
Signed-off-by: He Chen <he.chen@linux.intel.com>
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
tools/libxc/xc_psr.c | 3 +++
tools/libxl/xl_cmdimpl.c | 31 ++++++++++++++++++++-----------
tools/libxl/xl_cmdtable.c | 1 +
3 files changed, 24 insertions(+), 11 deletions(-)
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index fe494b0..31c99da 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -266,6 +266,9 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
case XC_PSR_CAT_L3_CBM_DATA:
cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA;
break;
+ case XC_PSR_CAT_L2_CBM:
+ cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM;
+ break;
default:
errno = EINVAL;
return -1;
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 4d6f32a..58b4eef 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -9527,19 +9527,21 @@ int main_psr_cat_cbm_set(int argc, char **argv)
char *value;
libxl_string_list socket_list;
unsigned long start, end;
- int i, j, len;
+ unsigned int i, j, len;
+ unsigned int lvl = 3;
static struct option opts[] = {
{"socket", 1, 0, 's'},
{"data", 0, 0, 'd'},
{"code", 0, 0, 'c'},
+ {"level", 1, 0, 'l'},
COMMON_LONG_OPTS
};
libxl_socket_bitmap_alloc(ctx, &target_map, 0);
libxl_bitmap_set_none(&target_map);
- SWITCH_FOREACH_OPT(opt, "s:cd", opts, "psr-cat-cbm-set", 2) {
+ SWITCH_FOREACH_OPT(opt, "s:l:cd", opts, "psr-cat-cbm-set", 2) {
case 's':
trim(isspace, optarg, &value);
split_string_into_string_list(value, ",", &socket_list);
@@ -9559,17 +9561,24 @@ int main_psr_cat_cbm_set(int argc, char **argv)
case 'c':
opt_code = 1;
break;
+ case 'l':
+ lvl = atoi(optarg);
+ break;
}
- if (opt_data && opt_code) {
- fprintf(stderr, "Cannot handle -c and -d at the same time\n");
- return -1;
- } else if (opt_data) {
- type = LIBXL_PSR_CBM_TYPE_L3_CBM_DATA;
- } else if (opt_code) {
- type = LIBXL_PSR_CBM_TYPE_L3_CBM_CODE;
- } else {
- type = LIBXL_PSR_CBM_TYPE_L3_CBM;
+ if (lvl == 2)
+ type = LIBXL_PSR_CBM_TYPE_L2_CBM;
+ else if (lvl == 3) {
+ if (opt_data && opt_code) {
+ fprintf(stderr, "Cannot handle -c and -d at the same time\n");
+ return ERROR_FAIL;
+ } else if (opt_data) {
+ type = LIBXL_PSR_CBM_TYPE_L3_CBM_DATA;
+ } else if (opt_code) {
+ type = LIBXL_PSR_CBM_TYPE_L3_CBM_CODE;
+ } else {
+ type = LIBXL_PSR_CBM_TYPE_L3_CBM;
+ }
}
if (libxl_bitmap_is_empty(&target_map))
diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c
index c5fbad4..32c3ee5 100644
--- a/tools/libxl/xl_cmdtable.c
+++ b/tools/libxl/xl_cmdtable.c
@@ -550,6 +550,7 @@ struct cmd_spec cmd_table[] = {
"Set cache capacity bitmasks(CBM) for a domain",
"[options] <Domain> <CBM>",
"-s <socket> Specify the socket to process, otherwise all sockets are processed\n"
+ "-l <level> Specify the cache level to process, otherwise L3 cache is processed\n"
"-c Set code CBM if CDP is supported\n"
"-d Set data CBM if CDP is supported\n"
},
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* Re: [PATCH v4 23/24] tools: L2 CAT: support set cbm for L2 CAT.
2016-12-14 4:08 ` [PATCH v4 23/24] tools: L2 CAT: support set " Yi Sun
@ 2017-01-06 12:04 ` Wei Liu
2017-01-09 1:14 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Wei Liu @ 2017-01-06 12:04 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
mengxu, jbeulich, chao.p.peng, xen-devel
On Wed, Dec 14, 2016 at 12:08:03PM +0800, Yi Sun wrote:
> This patch implements the xl/xc changes to support set CBM
> for L2 CAT.
>
> The new level option is introduced to original CAT setting
> command in order to set CBM for specified level CAT.
> - 'xl psr-cat-cbm-set' is updated to set cache capacity
> bitmasks(CBM) for a domain according to input cache level.
>
> root@:~$ xl psr-cat-cbm-set -l2 1 0x7f
>
> root@:~$ xl psr-cat-show -l2 1
> Socket ID : 0
> Default CBM : 0xff
> ID NAME CBM
> 1 ubuntu14 0x7f
>
> Signed-off-by: He Chen <he.chen@linux.intel.com>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
The code looks reasonable to me.
Wei.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* Re: [PATCH v4 23/24] tools: L2 CAT: support set cbm for L2 CAT.
2017-01-06 12:04 ` Wei Liu
@ 2017-01-09 1:14 ` Yi Sun
0 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2017-01-09 1:14 UTC (permalink / raw)
To: Wei Liu
Cc: andrew.cooper3, dario.faggioli, he.chen, ian.jackson, mengxu,
jbeulich, chao.p.peng, xen-devel
On 17-01-06 12:04:46, Wei Liu wrote:
> On Wed, Dec 14, 2016 at 12:08:03PM +0800, Yi Sun wrote:
> > This patch implements the xl/xc changes to support set CBM
> > for L2 CAT.
> >
> > The new level option is introduced to original CAT setting
> > command in order to set CBM for specified level CAT.
> > - 'xl psr-cat-cbm-set' is updated to set cache capacity
> > bitmasks(CBM) for a domain according to input cache level.
> >
> > root@:~$ xl psr-cat-cbm-set -l2 1 0x7f
> >
> > root@:~$ xl psr-cat-show -l2 1
> > Socket ID : 0
> > Default CBM : 0xff
> > ID NAME CBM
> > 1 ubuntu14 0x7f
> >
> > Signed-off-by: He Chen <he.chen@linux.intel.com>
> > Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
>
> The code looks reasonable to me.
>
> Wei.
Thanks a lot for your review!
Sun Yi
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v4 24/24] docs: add L2 CAT description in docs.
2016-12-14 4:07 [PATCH v4 00/24] Enable L2 Cache Allocation Technology & Refactor psr.c Yi Sun
` (22 preceding siblings ...)
2016-12-14 4:08 ` [PATCH v4 23/24] tools: L2 CAT: support set " Yi Sun
@ 2016-12-14 4:08 ` Yi Sun
2017-01-06 12:04 ` Wei Liu
23 siblings, 1 reply; 79+ messages in thread
From: Yi Sun @ 2016-12-14 4:08 UTC (permalink / raw)
To: xen-devel
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
Yi Sun, mengxu, jbeulich, chao.p.peng
This patch adds L2 CAT description in related documents.
Signed-off-by: He Chen <he.chen@linux.intel.com>
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
docs/man/xl.pod.1.in | 25 ++++++++++++++++++++++---
docs/misc/xl-psr.markdown | 10 ++++++++--
2 files changed, 30 insertions(+), 5 deletions(-)
diff --git a/docs/man/xl.pod.1.in b/docs/man/xl.pod.1.in
index 8e2aa5b..2c41ea7 100644
--- a/docs/man/xl.pod.1.in
+++ b/docs/man/xl.pod.1.in
@@ -1701,6 +1701,9 @@ occupancy monitoring share the same set of underlying monitoring service. Once
a domain is attached to the monitoring service, monitoring data can be shown
for any of these monitoring types.
+There is no cache monitoring and memory bandwidth monitoring on L2 cache so
+far.
+
=over 4
=item B<psr-cmt-attach> [I<domain-id>]
@@ -1725,7 +1728,7 @@ monitor types are:
Intel Broadwell and later server platforms offer capabilities to configure and
make use of the Cache Allocation Technology (CAT) mechanisms, which enable more
-cache resources (i.e. L3 cache) to be made available for high priority
+cache resources (i.e. L3/L2 cache) to be made available for high priority
applications. In the Xen implementation, CAT is used to control cache allocation
on VM basis. To enforce cache on a specific domain, just set capacity bitmasks
(CBM) for the domain.
@@ -1735,7 +1738,7 @@ Intel Broadwell and later server platforms also offer Code/Data Prioritization
applications. CDP is used on a per VM basis in the Xen implementation. To
specify code or data CBM for the domain, CDP feature must be enabled and CBM
type options need to be specified when setting CBM, and the type options (code
-and data) are mutually exclusive.
+and data) are mutually exclusive. There is no CDP support on L2 so far.
=over 4
@@ -1752,6 +1755,11 @@ B<OPTIONS>
Specify the socket to process, otherwise all sockets are processed.
+=item B<-l LEVEL>, B<--level=LEVEL>
+
+Specify the cache level to process, otherwise the last level cache (L3) is
+processed.
+
=item B<-c>, B<--code>
Set code CBM when CDP is enabled.
@@ -1762,10 +1770,21 @@ Set data CBM when CDP is enabled.
=back
-=item B<psr-cat-show> [I<domain-id>]
+=item B<psr-cat-show> [I<OPTIONS>] [I<domain-id>]
Show CAT settings for a certain domain or all domains.
+B<OPTIONS>
+
+=over 4
+
+=item B<-l LEVEL>, B<--level=LEVEL>
+
+Specify the cache level to process, otherwise the last level cache (L3) is
+processed.
+
+=back
+
=back
=head1 IGNORED FOR COMPATIBILITY WITH XM
diff --git a/docs/misc/xl-psr.markdown b/docs/misc/xl-psr.markdown
index c3c1e8e..bd2b6bd 100644
--- a/docs/misc/xl-psr.markdown
+++ b/docs/misc/xl-psr.markdown
@@ -70,7 +70,7 @@ total-mem-bandwidth instead of cache-occupancy). E.g. after a `xl psr-cmt-attach
Cache Allocation Technology (CAT) is a new feature available on Intel
Broadwell and later server platforms that allows an OS or Hypervisor/VMM to
-partition cache allocation (i.e. L3 cache) based on application priority or
+partition cache allocation (i.e. L3/L2 cache) based on application priority or
Class of Service (COS). Each COS is configured using capacity bitmasks (CBM)
which represent cache capacity and indicate the degree of overlap and
isolation between classes. System cache resource is divided into numbers of
@@ -119,13 +119,19 @@ A cbm is valid only when:
In a multi-socket system, the same cbm will be set on each socket by default.
Per socket cbm can be specified with the `--socket SOCKET` option.
+In different systems, the different cache level is supported, e.g. L3 cache or
+L2 cache. Per cache level cbm can be specified with the `--level LEVEL` option.
+
Setting the CBM may not be successful if insufficient COS is available. In
such case unused COS(es) may be freed by setting CBM of all related domains to
its default value(all-ones).
Per domain CBM settings can be shown by:
-`xl psr-cat-show`
+`xl psr-cat-show [OPTIONS] <domid>`
+
+In different systems, the different cache level is supported, e.g. L3 cache or
+L2 cache. Per cache level cbm can be specified with the `--level LEVEL` option.
## Code and Data Prioritization (CDP)
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 79+ messages in thread* Re: [PATCH v4 24/24] docs: add L2 CAT description in docs.
2016-12-14 4:08 ` [PATCH v4 24/24] docs: add L2 CAT description in docs Yi Sun
@ 2017-01-06 12:04 ` Wei Liu
2017-01-09 1:25 ` Yi Sun
0 siblings, 1 reply; 79+ messages in thread
From: Wei Liu @ 2017-01-06 12:04 UTC (permalink / raw)
To: Yi Sun
Cc: wei.liu2, andrew.cooper3, dario.faggioli, he.chen, ian.jackson,
mengxu, jbeulich, chao.p.peng, xen-devel
On Wed, Dec 14, 2016 at 12:08:04PM +0800, Yi Sun wrote:
> This patch adds L2 CAT description in related documents.
>
> Signed-off-by: He Chen <he.chen@linux.intel.com>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
This looks reasonable, too.
Wei.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread
* Re: [PATCH v4 24/24] docs: add L2 CAT description in docs.
2017-01-06 12:04 ` Wei Liu
@ 2017-01-09 1:25 ` Yi Sun
0 siblings, 0 replies; 79+ messages in thread
From: Yi Sun @ 2017-01-09 1:25 UTC (permalink / raw)
To: Wei Liu
Cc: andrew.cooper3, dario.faggioli, he.chen, ian.jackson, mengxu,
jbeulich, chao.p.peng, xen-devel
On 17-01-06 12:04:50, Wei Liu wrote:
> On Wed, Dec 14, 2016 at 12:08:04PM +0800, Yi Sun wrote:
> > This patch adds L2 CAT description in related documents.
> >
> > Signed-off-by: He Chen <he.chen@linux.intel.com>
> > Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
>
> This looks reasonable, too.
>
> Wei.
Thank you very much for reviewing the codes!
BRs,
Sun Yi
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 79+ messages in thread