* [PATCH v2 00/15] Enable Memory Bandwidth Allocation in Xen
@ 2017-08-24 1:14 Yi Sun
2017-08-24 1:14 ` [PATCH v2 01/15] docs: create Memory Bandwidth Allocation (MBA) feature document Yi Sun
` (14 more replies)
0 siblings, 15 replies; 64+ messages in thread
From: Yi Sun @ 2017-08-24 1:14 UTC (permalink / raw)
To: xen-devel
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, Yi Sun, julien.grall, mengxu, jbeulich, chao.p.peng,
roger.pau
Hi, all,
We plan to bring a new PSR (Platform Shared Resource) feature called
Intel Memory Bandwidth Allocation (MBA) to Xen.
Besides the MBA enabling, we change some interfaces to make them more
general but not only for CAT.
Any comments are welcome!
You can find this patch set at:
https://github.com/yisun-git/xen_mba mba_v2-2
---
Acked and Reviewed list before V2:
a - Acked-by
r - Reviewed-by
ar patch 2 - Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
r patch 3 - x86: rename 'cbm_type' to 'psr_val_type' to make it general
ar patch 12 - tools: rename 'xc_psr_cat_type' to 'xc_psr_val_type'
a patch 13 - tools: implement new generic get value interface and MBA get value command
a patch 15 - docs: add MBA description in docs
---
V2 change list:
Patch 1:
- declare 'HW' in Terminology.
(suggested by Chao Peng)
- replace 'COS ID of VCPU' to 'COS ID of domain'.
(suggested by Chao Peng)
- replace 'COS register' to 'Thrtl MSR'.
(suggested by Chao Peng)
- add description for 'psr-mba-show' to state that the decimal value is
shown for linear mode but hexadecimal value is shown for non-linear mode.
(suggested by Chao Peng)
- remove content in 'Areas for improvement'.
(suggested by Chao Peng)
- use '<>' to specify mandatory argument to a command.
(suggested by Wei Liu)
Patch3:
- replace 'PSR_VAL_TYPE_{L3, L2}' to 'PSR_VAL_TYPE_{L3, L2}_CBM'.
(suggested by Chao Peng)
Patch 4:
- modify commit message to replace 'cos register' to 'thrtl register' to
make it accurate.
(suggested by Chao Peng)
- restore the place of the sentence to assign value to 'feat->cbm_len'
because the MBA init flow is splitted out as a separate function in v1.
(suggested by Chao Peng)
- add comment to explain what the MBA thrtl defaul value '0' stands for.
(suggested by Chao Peng)
- check 'thrtl_max' under linear mode. It could not be euqal or larger than
100.
(suggested by Chao Peng)
Patch 5:
- use 'XEN_SYSCTL_PSR_MBA_LINEAR' to set MBA feature HW info.
(suggested by Chao Peng)
Patch 7:
- remove linear mode 'thrtl_max' check in 'mba_check_thrtl' because it has
been checked in 'mba_init_feature'.
(suggested by Chao Peng)
- for non-linear mode, check if '*thrtl' is not 0 in 'mba_check_thrtl'. If
it is 0, we do not need to change it.
(suggested by Chao Peng)
- move comments to explain changes of 'cos_write_info' from psr.c to commit
message.
(suggested by Chao Peng)
Patch 8:
- remove '_INFO' in 'libxl_psr_feat_type' and make corresponding
changes in 'libxl_psr_hw_info'.
(suggested by Chao Peng)
Patch 9:
- split this patch out from a big patch in v1.
(suggested by Wei Liu)
- change 'CAT_INFO' and 'MBA_INFO' to 'CAT' and 'MBA'.
(suggested by Chao Peng)
Patch 10:
- split this patch out from a big patch in v1.
(suggested by Wei Liu)
- change 'CAT_INFO'/'MBA_INFO' to 'CAT' and 'MBA. Also the libxl structure
name 'cat_info'/'mba_info' is changed to 'cat'/'mba'.
(suggested by Chao Peng)
- call 'libxl_psr_hw_info_list_free' in 'libxl_psr_cat_get_info' to free
allocated resources.
(suggested by Chao Peng)
Patch 11:
- split out this patch from a big patch in v1.
(suggested by Wei Liu)
- change 'MBA_INFO' to 'MBA'. Also, change 'mba_info' to 'mba'.
(suggested by Chao Peng)
Patch 13:
- change 'CAT_INFO'/'MBA_INFO' to 'CAT'/'MBA'. The related structure names
are changed too.
(suggested by Chao Peng)
Patch 15:
- state the value type shown by 'psr-mba-show'. For linear mode,
it shows decimal value. For non-linear mode, it shows hexadecimal
value.
(suggested by Chao Peng)
Yi Sun (15):
docs: create Memory Bandwidth Allocation (MBA) feature document
Rename PSR sysctl/domctl interfaces and xsm policy to make them be
general
x86: rename 'cbm_type' to 'psr_val_type' to make it general
x86: implement data structure and CPU init flow for MBA
x86: implement get hw info flow for MBA
x86: implement get value interface for MBA
x86: implement set value flow for MBA
tools: create general interfaces to support psr allocation features
tools: implement the new libxc get hw info interface
tools: implement the new libxl get hw info interface
tools: implement the new xl get hw info interface
tools: rename 'xc_psr_cat_type' to 'xc_psr_val_type'
tools: implement new generic get value interface and MBA get value
command
tools: implement new generic set value interface and MBA set value
command
docs: add MBA description in docs
docs/features/intel_psr_mba.pandoc | 256 +++++++++++++++++++++++++
docs/man/xl.pod.1.in | 34 ++++
docs/misc/xl-psr.markdown | 63 +++++++
tools/flask/policy/modules/dom0.te | 4 +-
tools/libxc/include/xenctrl.h | 47 +++--
tools/libxc/xc_psr.c | 88 ++++++---
tools/libxl/libxl.h | 35 +++-
tools/libxl/libxl_psr.c | 229 ++++++++++++++++++----
tools/libxl/libxl_types.idl | 22 +++
tools/xl/xl.h | 2 +
tools/xl/xl_cmdtable.c | 12 ++
tools/xl/xl_psr.c | 279 +++++++++++++++++++++------
xen/arch/x86/domctl.c | 65 ++++---
xen/arch/x86/psr.c | 366 +++++++++++++++++++++++++++---------
xen/arch/x86/sysctl.c | 47 +++--
xen/include/asm-x86/msr-index.h | 1 +
xen/include/asm-x86/psr.h | 22 ++-
xen/include/public/domctl.h | 14 +-
xen/include/public/sysctl.h | 20 +-
xen/xsm/flask/hooks.c | 8 +-
xen/xsm/flask/policy/access_vectors | 8 +-
21 files changed, 1318 insertions(+), 304 deletions(-)
create mode 100644 docs/features/intel_psr_mba.pandoc
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* [PATCH v2 01/15] docs: create Memory Bandwidth Allocation (MBA) feature document
2017-08-24 1:14 [PATCH v2 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
@ 2017-08-24 1:14 ` Yi Sun
2017-08-29 11:46 ` Roger Pau Monné
2017-08-24 1:14 ` [PATCH v2 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general Yi Sun
` (13 subsequent siblings)
14 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-24 1:14 UTC (permalink / raw)
To: xen-devel
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, Yi Sun, julien.grall, mengxu, jbeulich, chao.p.peng,
roger.pau
This patch creates MBA feature document in doc/features/. It describes
key points to implement MBA which is described in details in Intel SDM
"Introduction to Memory Bandwidth Allocation".
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
v2:
- declare 'HW' in Terminology.
(suggested by Chao Peng)
- replace 'COS ID of VCPU' to 'COS ID of domain'.
(suggested by Chao Peng)
- replace 'COS register' to 'Thrtl MSR'.
(suggested by Chao Peng)
- add description for 'psr-mba-show' to state that the decimal value is
shown for linear mode but hexadecimal value is shown for non-linear mode.
(suggested by Chao Peng)
- remove content in 'Areas for improvement'.
(suggested by Chao Peng)
- use '<>' to specify mandatory argument to a command.
(suggested by Wei Liu)
v1:
- remove a special character to avoid the error when building pandoc.
---
docs/features/intel_psr_mba.pandoc | 256 +++++++++++++++++++++++++++++++++++++
1 file changed, 256 insertions(+)
create mode 100644 docs/features/intel_psr_mba.pandoc
diff --git a/docs/features/intel_psr_mba.pandoc b/docs/features/intel_psr_mba.pandoc
new file mode 100644
index 0000000..21592e8
--- /dev/null
+++ b/docs/features/intel_psr_mba.pandoc
@@ -0,0 +1,256 @@
+% Intel Memory Bandwidth Allocation (MBA) Feature
+% Revision 1.4
+
+\clearpage
+
+# Basics
+
+---------------- ----------------------------------------------------
+ Status: **Tech Preview**
+
+Architecture(s): Intel x86
+
+ Component(s): Hypervisor, toolstack
+
+ Hardware: MBA is supported on Skylake Server and beyond
+---------------- ----------------------------------------------------
+
+# Terminology
+
+* CAT Cache Allocation Technology
+* CBM Capacity BitMasks
+* CDP Code and Data Prioritization
+* COS/CLOS Class of Service
+* HW Hardware
+* MBA Memory Bandwidth Allocation
+* MSRs Machine Specific Registers
+* PSR Intel Platform Shared Resource
+* THRTL Throttle value or delay value
+
+# Overview
+
+The Memory Bandwidth Allocation (MBA) feature provides indirect and approximate
+control over memory bandwidth available per-core. This feature provides OS/
+hypervisor the ability to slow misbehaving apps/domains or create advanced
+closed-loop control system via exposing control over a credit-based throttling
+mechanism.
+
+# User details
+
+* Feature Enabling:
+
+ Add "psr=mba" to boot line parameter to enable MBA feature.
+
+* xl interfaces:
+
+ 1. `psr-mba-show [domain-id]`:
+
+ Show memory bandwidth throttling for domain. For linear mode, it shows the
+ decimal value. For non-linear mode, it shows hexadecimal value.
+
+ 2. `psr-mba-set [OPTIONS] <domain-id> <throttling>`:
+
+ Set memory bandwidth throttling for domain.
+
+ Options:
+ '-s': Specify the socket to process, otherwise all sockets are processed.
+
+ Throttling value set in register implies memory bandwidth blocked, i.e.
+ higher throttling value results in lower bandwidth. The max throttling
+ value can be got through CPUID.
+
+ The response of the throttling value could be linear mode or non-linear
+ mode.
+
+ Linear mode: the input precision is defined as 100-(MBA_MAX). For instance,
+ if the MBA_MAX value is 90, the input precision is 10%. Values not an even
+ multiple of the precision (e.g., 12%) will be rounded down (e.g., to 10%
+ delay applied) by HW automatically.
+
+ Non-linear mode: input delay values are powers-of-two from zero to the
+ MBA_MAX value from CPUID. In this case any values not a power of two will
+ be rounded down the next nearest power of two by HW automatically.
+
+# Technical details
+
+MBA is a member of Intel PSR features, it shares the base PSR infrastructure
+in Xen.
+
+## Hardware perspective
+
+ MBA defines a range of MSRs to support specifying a delay value (Thrtl) per
+ COS, with details below.
+
+ ```
+ +----------------------------+----------------+
+ | MSR (per socket) | Address |
+ +----------------------------+----------------+
+ | IA32_L2_QOS_Ext_BW_Thrtl_0 | 0xD50 |
+ +----------------------------+----------------+
+ | ... | ... |
+ +----------------------------+----------------+
+ | IA32_L2_QOS_Ext_BW_Thrtl_n | 0xD50+n (n<64) |
+ +----------------------------+----------------+
+ ```
+
+ When context switch happens, the COS ID of domain is written to per-thread MSR
+ `IA32_PQR_ASSOC`, and then hardware enforces bandwidth allocation according
+ to the throttling value stored in the Thrtl MSR register.
+
+## The relationship between MBA and CAT/CDP
+
+ Generally speaking, MBA is completely independent of CAT/CDP, and any
+ combination may be applied at any time, e.g. enabling MBA with CAT
+ disabled.
+
+ But it needs to be noticed that MBA shares COS infrastructure with CAT,
+ although MBA is enumerated by different CPUID leaf from CAT (which
+ indicates that the max COS of MBA may be different from CAT). In some
+ cases, a domain is permitted to have a COS that is beyond one (or more)
+ of PSR features but within the others. For instance, let's assume the max
+ COS of MBA is 8 but the max COS of L3 CAT is 16, when a domain is assigned
+ 9 as COS, the L3 CAT CBM associated to COS 9 would be enforced, but for MBA,
+ the HW works as default value is set since COS 9 is beyond the max COS (8)
+ of MBA.
+
+## Design Overview
+
+* Core COS/Thrtl association
+
+ When enforcing Memory Bandwidth Allocation, all cores of domains have
+ the same default Thrtl MSR (COS0) which stores the same Thrtl (0). The
+ default Thrtl MSR is used only in hypervisor and is transparent to tool stack
+ and user.
+
+ System administrator can change PSR allocation policy at runtime by
+ tool stack. Since MBA shares COS ID with CAT/CDP, a COS ID corresponds to a
+ 2-tuple, like [CBM, Thrtl] with only-CAT enalbed, when CDP is enabled,
+ the COS ID corresponds to a 3-tuple, like [Code_CBM, Data_CBM, Thrtl]. If
+ neither CAT nor CDP is enabled, things would be easier, one COS ID corresponds
+ to one Thrtl.
+
+* VCPU schedule
+
+ This part reuses CAT COS infrastructure.
+
+* Multi-sockets
+
+ Different sockets may have different MBA ability (like max COS)
+ although it is consistent on the same socket. So the capability
+ of per-socket MBA is specified.
+
+ This part reuses CAT COS infrastructure.
+
+## Implementation Description
+
+* Hypervisor interfaces:
+
+ 1. Boot line param: "psr=mba" to enable the feature.
+
+ 2. SYSCTL:
+ - XEN_SYSCTL_PSR_MBA_get_info: Get system MBA information.
+
+ 3. DOMCTL:
+ - XEN_DOMCTL_PSR_MBA_OP_GET_THRTL: Get throttling for a domain.
+ - XEN_DOMCTL_PSR_MBA_OP_SET_THRTL: Set throttling for a domain.
+
+* xl interfaces:
+
+ 1. psr-mba-show [domain-id]
+ Show system/domain runtime MBA throttling value. For linear mode,
+ it shows the decimal value. For non-linear mode, it shows hexadecimal
+ value.
+ => XEN_SYSCTL_PSR_MBA_get_info/XEN_DOMCTL_PSR_MBA_OP_GET_THRTL
+
+ 2. psr-mba-set [OPTIONS] <domain-id> <throttling>
+ Set bandwidth throttling for a domain.
+ => XEN_DOMCTL_PSR_MBA_OP_SET_THRTL
+
+ 3. psr-hwinfo
+ Show PSR HW information, including L3 CAT/CDP/L2 CAT/MBA.
+ => XEN_SYSCTL_PSR_MBA_get_info
+
+* Key data structure:
+
+ 1. Feature HW info
+
+ ```
+ struct {
+ unsigned int thrtl_max;
+ unsigned int linear;
+ } mba_info;
+
+ - Member `thrtl_max`
+
+ `thrtl_max` is the max throttling value to be set.
+
+ - Member `linear`
+
+ `linear` means the response of delay value is linear or not.
+
+ As mentioned above, MBA is a member of Intel PSR features, it would
+ share the base PSR infrastructure in Xen. For example, the 'cos_max'
+ is a common HW property for all features. So, for other data structure
+ details, please refer 'intel_psr_cat_cdp.pandoc'.
+
+# Limitations
+
+MBA can only work on HW which enables it (check by CPUID).
+
+# Testing
+
+We can execute these commands to verify MBA on different HWs supporting them.
+
+For example:
+ root@:~$ xl psr-hwinfo --mba
+ Memory Bandwidth Allocation (MBA):
+ Socket ID : 0
+ Linear Mode : Enabled
+ Maximum COS : 7
+ Maximum Throttling Value: 90
+ Default Throttling Value: 0
+
+ root@:~$ xl psr-mba-set 1 0xa
+
+ root@:~$ xl psr-mba-show 1
+ Socket ID : 0
+ Default THRTL : 0
+ ID NAME THRTL
+ 1 ubuntu14 0xa
+
+# Areas for improvement
+
+N/A
+
+# Known issues
+
+N/A
+
+# References
+
+"INTEL RESOURCE DIRECTOR TECHNOLOGY (INTEL RDT) ALLOCATION FEATURES" [Intel 64 and IA-32 Architectures Software Developer Manuals, vol3](http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html)
+
+# History
+
+------------------------------------------------------------------------
+Date Revision Version Notes
+---------- -------- -------- -------------------------------------------
+2017-01-10 1.0 Xen 4.9 Design document written
+2017-07-10 1.1 Xen 4.10 Changes:
+ 1. Modify data structure according to latest
+ codes;
+ 2. Add content for 'Areas for improvement';
+ 3. Other minor changes.
+2017-08-09 1.2 Xen 4.10 Changes:
+ 1. Remove a special character to avoid error when
+ building pandoc.
+2017-08-15 1.3 Xen 4.10 Changes:
+ 1. Add terminology 'HW'.
+ 2. Change 'COS ID of VCPU' to 'COS ID of domain'.
+ 3. Change 'COS register' to 'Thrtl MSR'.
+ 4. Explain the value shown for 'psr-mba-show' under
+ different modes.
+ 5. Remove content in 'Areas for improvement'.
+2017-08-16 1.4 Xen 4.10 Changes:
+ 1. Add '<>' for mandatory argument.
+---------- -------- -------- -------------------------------------------
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
2017-08-24 1:14 [PATCH v2 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
2017-08-24 1:14 ` [PATCH v2 01/15] docs: create Memory Bandwidth Allocation (MBA) feature document Yi Sun
@ 2017-08-24 1:14 ` Yi Sun
2017-08-29 12:00 ` Roger Pau Monné
2017-08-24 1:14 ` [PATCH v2 03/15] x86: rename 'cbm_type' to 'psr_val_type' to make it general Yi Sun
` (12 subsequent siblings)
14 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-24 1:14 UTC (permalink / raw)
To: xen-devel
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, Yi Sun, julien.grall, mengxu, jbeulich, chao.p.peng,
roger.pau
This patch renames PSR sysctl/domctl interfaces and related xsm policy to
make them be general for all resource allocation features but not only
for CAT. Then, we can resuse the interfaces for all allocation features.
Basically, it changes 'cat' to 'alloc'. E.g.:
1. psr_cat_op -> psr_alloc_op
2. XEN_DOMCTL_psr_cat_op -> XEN_DOMCTL_psr_alloc_op
3. XEN_SYSCTL_psr_cat_op -> XEN_SYSCTL_psr_alloc_op
The sysctl/domctl version numbers are bumped.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
---
v1:
- add description about what to be changed in commit message.
(suggested by Wei Liu)
- bump sysctl/domctl version numbers.
(suggested by Wei Liu)
---
tools/flask/policy/modules/dom0.te | 4 ++--
tools/libxc/xc_psr.c | 34 +++++++++++++++++-----------------
xen/arch/x86/domctl.c | 36 ++++++++++++++++++------------------
xen/arch/x86/sysctl.c | 24 ++++++++++++------------
xen/include/public/domctl.h | 12 ++++++------
xen/include/public/sysctl.h | 12 ++++++------
xen/xsm/flask/hooks.c | 8 ++++----
xen/xsm/flask/policy/access_vectors | 8 ++++----
8 files changed, 69 insertions(+), 69 deletions(-)
diff --git a/tools/flask/policy/modules/dom0.te b/tools/flask/policy/modules/dom0.te
index d0a4d91..1a376ac 100644
--- a/tools/flask/policy/modules/dom0.te
+++ b/tools/flask/policy/modules/dom0.te
@@ -14,7 +14,7 @@ allow dom0_t xen_t:xen {
tmem_control getscheduler setscheduler
};
allow dom0_t xen_t:xen2 {
- resource_op psr_cmt_op psr_cat_op pmu_ctrl get_symbol
+ resource_op psr_cmt_op psr_alloc_op pmu_ctrl get_symbol
get_cpu_levelling_caps get_cpu_featureset livepatch_op
gcov_op
};
@@ -39,7 +39,7 @@ allow dom0_t dom0_t:domain {
};
allow dom0_t dom0_t:domain2 {
set_cpuid gettsc settsc setscheduler set_max_evtchn set_vnumainfo
- get_vnumainfo psr_cmt_op psr_cat_op
+ get_vnumainfo psr_cmt_op psr_alloc_op
};
allow dom0_t dom0_t:resource { add remove };
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 039b920..73d05f2 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -274,11 +274,11 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
return -1;
}
- domctl.cmd = XEN_DOMCTL_psr_cat_op;
+ domctl.cmd = XEN_DOMCTL_psr_alloc_op;
domctl.domain = (domid_t)domid;
- domctl.u.psr_cat_op.cmd = cmd;
- domctl.u.psr_cat_op.target = target;
- domctl.u.psr_cat_op.data = data;
+ domctl.u.psr_alloc_op.cmd = cmd;
+ domctl.u.psr_alloc_op.target = target;
+ domctl.u.psr_alloc_op.data = data;
return do_domctl(xch, &domctl);
}
@@ -310,15 +310,15 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
return -1;
}
- domctl.cmd = XEN_DOMCTL_psr_cat_op;
+ domctl.cmd = XEN_DOMCTL_psr_alloc_op;
domctl.domain = (domid_t)domid;
- domctl.u.psr_cat_op.cmd = cmd;
- domctl.u.psr_cat_op.target = target;
+ domctl.u.psr_alloc_op.cmd = cmd;
+ domctl.u.psr_alloc_op.target = target;
rc = do_domctl(xch, &domctl);
if ( !rc )
- *data = domctl.u.psr_cat_op.data;
+ *data = domctl.u.psr_alloc_op.data;
return rc;
}
@@ -329,29 +329,29 @@ int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
int rc = -1;
DECLARE_SYSCTL;
- sysctl.cmd = XEN_SYSCTL_psr_cat_op;
- sysctl.u.psr_cat_op.target = socket;
+ sysctl.cmd = XEN_SYSCTL_psr_alloc_op;
+ sysctl.u.psr_alloc_op.target = socket;
switch ( lvl )
{
case 2:
- sysctl.u.psr_cat_op.cmd = XEN_SYSCTL_PSR_CAT_get_l2_info;
+ sysctl.u.psr_alloc_op.cmd = XEN_SYSCTL_PSR_CAT_get_l2_info;
rc = xc_sysctl(xch, &sysctl);
if ( !rc )
{
- *cos_max = sysctl.u.psr_cat_op.u.cat_info.cos_max;
- *cbm_len = sysctl.u.psr_cat_op.u.cat_info.cbm_len;
+ *cos_max = sysctl.u.psr_alloc_op.u.cat_info.cos_max;
+ *cbm_len = sysctl.u.psr_alloc_op.u.cat_info.cbm_len;
*cdp_enabled = false;
}
break;
case 3:
- sysctl.u.psr_cat_op.cmd = XEN_SYSCTL_PSR_CAT_get_l3_info;
+ sysctl.u.psr_alloc_op.cmd = XEN_SYSCTL_PSR_CAT_get_l3_info;
rc = xc_sysctl(xch, &sysctl);
if ( !rc )
{
- *cos_max = sysctl.u.psr_cat_op.u.cat_info.cos_max;
- *cbm_len = sysctl.u.psr_cat_op.u.cat_info.cbm_len;
- *cdp_enabled = sysctl.u.psr_cat_op.u.cat_info.flags &
+ *cos_max = sysctl.u.psr_alloc_op.u.cat_info.cos_max;
+ *cbm_len = sysctl.u.psr_alloc_op.u.cat_info.cbm_len;
+ *cdp_enabled = sysctl.u.psr_alloc_op.u.cat_info.flags &
XEN_SYSCTL_PSR_CAT_L3_CDP;
}
break;
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 127c84e..24b998b 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1439,60 +1439,60 @@ long arch_do_domctl(
}
break;
- case XEN_DOMCTL_psr_cat_op:
- switch ( domctl->u.psr_cat_op.cmd )
+ case XEN_DOMCTL_psr_alloc_op:
+ switch ( domctl->u.psr_alloc_op.cmd )
{
uint32_t val32;
case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM:
- ret = psr_set_val(d, domctl->u.psr_cat_op.target,
- domctl->u.psr_cat_op.data,
+ ret = psr_set_val(d, domctl->u.psr_alloc_op.target,
+ domctl->u.psr_alloc_op.data,
PSR_CBM_TYPE_L3);
break;
case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CODE:
- ret = psr_set_val(d, domctl->u.psr_cat_op.target,
- domctl->u.psr_cat_op.data,
+ ret = psr_set_val(d, domctl->u.psr_alloc_op.target,
+ domctl->u.psr_alloc_op.data,
PSR_CBM_TYPE_L3_CODE);
break;
case XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA:
- ret = psr_set_val(d, domctl->u.psr_cat_op.target,
- domctl->u.psr_cat_op.data,
+ ret = psr_set_val(d, domctl->u.psr_alloc_op.target,
+ domctl->u.psr_alloc_op.data,
PSR_CBM_TYPE_L3_DATA);
break;
case XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM:
- ret = psr_set_val(d, domctl->u.psr_cat_op.target,
- domctl->u.psr_cat_op.data,
+ ret = psr_set_val(d, domctl->u.psr_alloc_op.target,
+ domctl->u.psr_alloc_op.data,
PSR_CBM_TYPE_L2);
break;
case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM:
- ret = psr_get_val(d, domctl->u.psr_cat_op.target,
+ ret = psr_get_val(d, domctl->u.psr_alloc_op.target,
&val32, PSR_CBM_TYPE_L3);
- domctl->u.psr_cat_op.data = val32;
+ domctl->u.psr_alloc_op.data = val32;
copyback = true;
break;
case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CODE:
- ret = psr_get_val(d, domctl->u.psr_cat_op.target,
+ ret = psr_get_val(d, domctl->u.psr_alloc_op.target,
&val32, PSR_CBM_TYPE_L3_CODE);
- domctl->u.psr_cat_op.data = val32;
+ domctl->u.psr_alloc_op.data = val32;
copyback = true;
break;
case XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA:
- ret = psr_get_val(d, domctl->u.psr_cat_op.target,
+ ret = psr_get_val(d, domctl->u.psr_alloc_op.target,
&val32, PSR_CBM_TYPE_L3_DATA);
- domctl->u.psr_cat_op.data = val32;
+ domctl->u.psr_alloc_op.data = val32;
copyback = true;
break;
case XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM:
- ret = psr_get_val(d, domctl->u.psr_cat_op.target,
+ ret = psr_get_val(d, domctl->u.psr_alloc_op.target,
&val32, PSR_CBM_TYPE_L2);
- domctl->u.psr_cat_op.data = val32;
+ domctl->u.psr_alloc_op.data = val32;
copyback = true;
break;
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index 2bde99c..e7e390e 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -171,45 +171,45 @@ long arch_do_sysctl(
break;
- case XEN_SYSCTL_psr_cat_op:
- switch ( sysctl->u.psr_cat_op.cmd )
+ case XEN_SYSCTL_psr_alloc_op:
+ switch ( sysctl->u.psr_alloc_op.cmd )
{
uint32_t data[PSR_INFO_ARRAY_SIZE];
case XEN_SYSCTL_PSR_CAT_get_l3_info:
{
- ret = psr_get_info(sysctl->u.psr_cat_op.target,
+ ret = psr_get_info(sysctl->u.psr_alloc_op.target,
PSR_CBM_TYPE_L3, data, ARRAY_SIZE(data));
if ( ret )
break;
- sysctl->u.psr_cat_op.u.cat_info.cos_max =
+ sysctl->u.psr_alloc_op.u.cat_info.cos_max =
data[PSR_INFO_IDX_COS_MAX];
- sysctl->u.psr_cat_op.u.cat_info.cbm_len =
+ sysctl->u.psr_alloc_op.u.cat_info.cbm_len =
data[PSR_INFO_IDX_CAT_CBM_LEN];
- sysctl->u.psr_cat_op.u.cat_info.flags =
+ sysctl->u.psr_alloc_op.u.cat_info.flags =
data[PSR_INFO_IDX_CAT_FLAG];
- if ( !ret && __copy_field_to_guest(u_sysctl, sysctl, u.psr_cat_op) )
+ if ( !ret && __copy_field_to_guest(u_sysctl, sysctl, u.psr_alloc_op) )
ret = -EFAULT;
break;
}
case XEN_SYSCTL_PSR_CAT_get_l2_info:
{
- ret = psr_get_info(sysctl->u.psr_cat_op.target,
+ ret = psr_get_info(sysctl->u.psr_alloc_op.target,
PSR_CBM_TYPE_L2, data, ARRAY_SIZE(data));
if ( ret )
break;
- sysctl->u.psr_cat_op.u.cat_info.cos_max =
+ sysctl->u.psr_alloc_op.u.cat_info.cos_max =
data[PSR_INFO_IDX_COS_MAX];
- sysctl->u.psr_cat_op.u.cat_info.cbm_len =
+ sysctl->u.psr_alloc_op.u.cat_info.cbm_len =
data[PSR_INFO_IDX_CAT_CBM_LEN];
- sysctl->u.psr_cat_op.u.cat_info.flags =
+ sysctl->u.psr_alloc_op.u.cat_info.flags =
data[PSR_INFO_IDX_CAT_FLAG];
- if ( !ret && __copy_field_to_guest(u_sysctl, sysctl, u.psr_cat_op) )
+ if ( !ret && __copy_field_to_guest(u_sysctl, sysctl, u.psr_alloc_op) )
ret = -EFAULT;
break;
}
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 0669c31..4bb4399 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -37,7 +37,7 @@
#include "hvm/save.h"
#include "memory.h"
-#define XEN_DOMCTL_INTERFACE_VERSION 0x0000000e
+#define XEN_DOMCTL_INTERFACE_VERSION 0x0000000f
/*
* NB. xen_domctl.domain is an IN/OUT parameter for this operation.
@@ -1135,7 +1135,7 @@ struct xen_domctl_monitor_op {
typedef struct xen_domctl_monitor_op xen_domctl_monitor_op_t;
DEFINE_XEN_GUEST_HANDLE(xen_domctl_monitor_op_t);
-struct xen_domctl_psr_cat_op {
+struct xen_domctl_psr_alloc_op {
#define XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM 0
#define XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM 1
#define XEN_DOMCTL_PSR_CAT_OP_SET_L3_CODE 2
@@ -1148,8 +1148,8 @@ struct xen_domctl_psr_cat_op {
uint32_t target; /* IN */
uint64_t data; /* IN/OUT */
};
-typedef struct xen_domctl_psr_cat_op xen_domctl_psr_cat_op_t;
-DEFINE_XEN_GUEST_HANDLE(xen_domctl_psr_cat_op_t);
+typedef struct xen_domctl_psr_alloc_op xen_domctl_psr_alloc_op_t;
+DEFINE_XEN_GUEST_HANDLE(xen_domctl_psr_alloc_op_t);
struct xen_domctl {
uint32_t cmd;
@@ -1226,7 +1226,7 @@ struct xen_domctl {
#define XEN_DOMCTL_setvnumainfo 74
#define XEN_DOMCTL_psr_cmt_op 75
#define XEN_DOMCTL_monitor_op 77
-#define XEN_DOMCTL_psr_cat_op 78
+#define XEN_DOMCTL_psr_alloc_op 78
#define XEN_DOMCTL_soft_reset 79
#define XEN_DOMCTL_gdbsx_guestmemio 1000
#define XEN_DOMCTL_gdbsx_pausevcpu 1001
@@ -1289,7 +1289,7 @@ struct xen_domctl {
struct xen_domctl_vnuma vnuma;
struct xen_domctl_psr_cmt_op psr_cmt_op;
struct xen_domctl_monitor_op monitor_op;
- struct xen_domctl_psr_cat_op psr_cat_op;
+ struct xen_domctl_psr_alloc_op psr_alloc_op;
uint8_t pad[128];
} u;
};
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index 9e51af6..fcebab1 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -36,7 +36,7 @@
#include "physdev.h"
#include "tmem.h"
-#define XEN_SYSCTL_INTERFACE_VERSION 0x0000000F
+#define XEN_SYSCTL_INTERFACE_VERSION 0x00000010
/*
* Read console content from Xen buffer ring.
@@ -745,7 +745,7 @@ DEFINE_XEN_GUEST_HANDLE(xen_sysctl_pcitopoinfo_t);
#define XEN_SYSCTL_PSR_CAT_get_l3_info 0
#define XEN_SYSCTL_PSR_CAT_get_l2_info 1
-struct xen_sysctl_psr_cat_op {
+struct xen_sysctl_psr_alloc_op {
uint32_t cmd; /* IN: XEN_SYSCTL_PSR_CAT_* */
uint32_t target; /* IN */
union {
@@ -757,8 +757,8 @@ struct xen_sysctl_psr_cat_op {
} cat_info;
} u;
};
-typedef struct xen_sysctl_psr_cat_op xen_sysctl_psr_cat_op_t;
-DEFINE_XEN_GUEST_HANDLE(xen_sysctl_psr_cat_op_t);
+typedef struct xen_sysctl_psr_alloc_op xen_sysctl_psr_alloc_op_t;
+DEFINE_XEN_GUEST_HANDLE(xen_sysctl_psr_alloc_op_t);
#define XEN_SYSCTL_TMEM_OP_ALL_CLIENTS 0xFFFFU
@@ -1119,7 +1119,7 @@ struct xen_sysctl {
#define XEN_SYSCTL_gcov_op 20
#define XEN_SYSCTL_psr_cmt_op 21
#define XEN_SYSCTL_pcitopoinfo 22
-#define XEN_SYSCTL_psr_cat_op 23
+#define XEN_SYSCTL_psr_alloc_op 23
#define XEN_SYSCTL_tmem_op 24
#define XEN_SYSCTL_get_cpu_levelling_caps 25
#define XEN_SYSCTL_get_cpu_featureset 26
@@ -1147,7 +1147,7 @@ struct xen_sysctl {
struct xen_sysctl_scheduler_op scheduler_op;
struct xen_sysctl_gcov_op gcov_op;
struct xen_sysctl_psr_cmt_op psr_cmt_op;
- struct xen_sysctl_psr_cat_op psr_cat_op;
+ struct xen_sysctl_psr_alloc_op psr_alloc_op;
struct xen_sysctl_tmem_op tmem_op;
struct xen_sysctl_cpu_levelling_caps cpu_levelling_caps;
struct xen_sysctl_cpu_featureset cpu_featureset;
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index fd84ac0..95aff21 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -742,8 +742,8 @@ static int flask_domctl(struct domain *d, int cmd)
case XEN_DOMCTL_psr_cmt_op:
return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__PSR_CMT_OP);
- case XEN_DOMCTL_psr_cat_op:
- return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__PSR_CAT_OP);
+ case XEN_DOMCTL_psr_alloc_op:
+ return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__PSR_ALLOC_OP);
case XEN_DOMCTL_soft_reset:
return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__SOFT_RESET);
@@ -806,9 +806,9 @@ static int flask_sysctl(int cmd)
case XEN_SYSCTL_psr_cmt_op:
return avc_current_has_perm(SECINITSID_XEN, SECCLASS_XEN2,
XEN2__PSR_CMT_OP, NULL);
- case XEN_SYSCTL_psr_cat_op:
+ case XEN_SYSCTL_psr_alloc_op:
return avc_current_has_perm(SECINITSID_XEN, SECCLASS_XEN2,
- XEN2__PSR_CAT_OP, NULL);
+ XEN2__PSR_ALLOC_OP, NULL);
case XEN_SYSCTL_tmem_op:
return domain_has_xen(current->domain, XEN__TMEM_CONTROL);
diff --git a/xen/xsm/flask/policy/access_vectors b/xen/xsm/flask/policy/access_vectors
index 1f7eb35..9d81e41 100644
--- a/xen/xsm/flask/policy/access_vectors
+++ b/xen/xsm/flask/policy/access_vectors
@@ -85,8 +85,8 @@ class xen2
resource_op
# XEN_SYSCTL_psr_cmt_op
psr_cmt_op
-# XEN_SYSCTL_psr_cat_op
- psr_cat_op
+# XEN_SYSCTL_psr_alloc_op
+ psr_alloc_op
# XENPF_get_symbol
get_symbol
# PMU control
@@ -244,8 +244,8 @@ class domain2
mem_paging
# XENMEM_sharing_op
mem_sharing
-# XEN_DOMCTL_psr_cat_op
- psr_cat_op
+# XEN_DOMCTL_psr_alloc_op
+ psr_alloc_op
}
# Similar to class domain, but primarily contains domctls related to HVM domains
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 03/15] x86: rename 'cbm_type' to 'psr_val_type' to make it general
2017-08-24 1:14 [PATCH v2 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
2017-08-24 1:14 ` [PATCH v2 01/15] docs: create Memory Bandwidth Allocation (MBA) feature document Yi Sun
2017-08-24 1:14 ` [PATCH v2 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general Yi Sun
@ 2017-08-24 1:14 ` Yi Sun
2017-08-29 12:15 ` Roger Pau Monné
2017-08-24 1:14 ` [PATCH v2 04/15] x86: implement data structure and CPU init flow for MBA Yi Sun
` (11 subsequent siblings)
14 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-24 1:14 UTC (permalink / raw)
To: xen-devel
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, Yi Sun, julien.grall, mengxu, jbeulich, chao.p.peng,
roger.pau
This patch renames 'cbm_type' to 'psr_val_type' to make it be general.
Then, we can reuse this for all psr allocation features.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
---
v2:
- replace 'PSR_VAL_TYPE_{L3, L2}' to 'PSR_VAL_TYPE_{L3, L2}_CBM'.
(suggested by Chao Peng)
---
xen/arch/x86/domctl.c | 16 ++++++------
xen/arch/x86/psr.c | 62 +++++++++++++++++++++++++----------------------
xen/arch/x86/sysctl.c | 4 +--
xen/include/asm-x86/psr.h | 18 +++++++-------
4 files changed, 52 insertions(+), 48 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 24b998b..e1c7c70 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1447,51 +1447,51 @@ long arch_do_domctl(
case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM:
ret = psr_set_val(d, domctl->u.psr_alloc_op.target,
domctl->u.psr_alloc_op.data,
- PSR_CBM_TYPE_L3);
+ PSR_VAL_TYPE_L3_CBM);
break;
case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CODE:
ret = psr_set_val(d, domctl->u.psr_alloc_op.target,
domctl->u.psr_alloc_op.data,
- PSR_CBM_TYPE_L3_CODE);
+ PSR_VAL_TYPE_L3_CODE);
break;
case XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA:
ret = psr_set_val(d, domctl->u.psr_alloc_op.target,
domctl->u.psr_alloc_op.data,
- PSR_CBM_TYPE_L3_DATA);
+ PSR_VAL_TYPE_L3_DATA);
break;
case XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM:
ret = psr_set_val(d, domctl->u.psr_alloc_op.target,
domctl->u.psr_alloc_op.data,
- PSR_CBM_TYPE_L2);
+ PSR_VAL_TYPE_L2_CBM);
break;
case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM:
ret = psr_get_val(d, domctl->u.psr_alloc_op.target,
- &val32, PSR_CBM_TYPE_L3);
+ &val32, PSR_VAL_TYPE_L3_CBM);
domctl->u.psr_alloc_op.data = val32;
copyback = true;
break;
case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CODE:
ret = psr_get_val(d, domctl->u.psr_alloc_op.target,
- &val32, PSR_CBM_TYPE_L3_CODE);
+ &val32, PSR_VAL_TYPE_L3_CODE);
domctl->u.psr_alloc_op.data = val32;
copyback = true;
break;
case XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA:
ret = psr_get_val(d, domctl->u.psr_alloc_op.target,
- &val32, PSR_CBM_TYPE_L3_DATA);
+ &val32, PSR_VAL_TYPE_L3_DATA);
domctl->u.psr_alloc_op.data = val32;
copyback = true;
break;
case XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM:
ret = psr_get_val(d, domctl->u.psr_alloc_op.target,
- &val32, PSR_CBM_TYPE_L2);
+ &val32, PSR_VAL_TYPE_L2_CBM);
domctl->u.psr_alloc_op.data = val32;
copyback = true;
break;
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 3622de0..da62f81 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -100,24 +100,24 @@ static const struct feat_props {
unsigned int cos_num;
/*
- * An array to save all 'enum cbm_type' values of the feature. It is
+ * An array to save all 'enum psr_val_type' values of the feature. It is
* used with cos_num together to get/write a feature's COS registers
* values one by one.
*/
- enum cbm_type type[MAX_COS_NUM];
+ enum psr_val_type type[MAX_COS_NUM];
/*
* alt_type is 'alternative type'. When this 'alt_type' is input, the
* feature does some special operations.
*/
- enum cbm_type alt_type;
+ enum psr_val_type alt_type;
/* get_feat_info is used to return feature HW info through sysctl. */
bool (*get_feat_info)(const struct feat_node *feat,
uint32_t data[], unsigned int array_len);
/* write_msr is used to write out feature MSR register. */
- void (*write_msr)(unsigned int cos, uint32_t val, enum cbm_type type);
+ void (*write_msr)(unsigned int cos, uint32_t val, enum psr_val_type type);
} *feat_props[FEAT_TYPE_NUM];
/*
@@ -216,13 +216,13 @@ static void free_socket_resources(unsigned int socket)
bitmap_zero(info->dom_set, DOMID_IDLE + 1);
}
-static enum psr_feat_type psr_cbm_type_to_feat_type(enum cbm_type type)
+static enum psr_feat_type psr_val_type_to_feat_type(enum psr_val_type type)
{
enum psr_feat_type feat_type = FEAT_TYPE_UNKNOWN;
switch ( type )
{
- case PSR_CBM_TYPE_L3:
+ case PSR_VAL_TYPE_L3_CBM:
feat_type = FEAT_TYPE_L3_CAT;
/*
@@ -234,12 +234,12 @@ static enum psr_feat_type psr_cbm_type_to_feat_type(enum cbm_type type)
break;
- case PSR_CBM_TYPE_L3_DATA:
- case PSR_CBM_TYPE_L3_CODE:
+ case PSR_VAL_TYPE_L3_DATA:
+ case PSR_VAL_TYPE_L3_CODE:
feat_type = FEAT_TYPE_L3_CDP;
break;
- case PSR_CBM_TYPE_L2:
+ case PSR_VAL_TYPE_L2_CBM:
feat_type = FEAT_TYPE_L2_CAT;
break;
@@ -363,15 +363,16 @@ static bool cat_get_feat_info(const struct feat_node *feat,
}
/* L3 CAT props */
-static void l3_cat_write_msr(unsigned int cos, uint32_t val, enum cbm_type type)
+static void l3_cat_write_msr(unsigned int cos, uint32_t val,
+ enum psr_val_type type)
{
wrmsrl(MSR_IA32_PSR_L3_MASK(cos), val);
}
static const struct feat_props l3_cat_props = {
.cos_num = 1,
- .type[0] = PSR_CBM_TYPE_L3,
- .alt_type = PSR_CBM_TYPE_UNKNOWN,
+ .type[0] = PSR_VAL_TYPE_L3_CBM,
+ .alt_type = PSR_VAL_TYPE_UNKNOWN,
.get_feat_info = cat_get_feat_info,
.write_msr = l3_cat_write_msr,
};
@@ -388,9 +389,10 @@ static bool l3_cdp_get_feat_info(const struct feat_node *feat,
return true;
}
-static void l3_cdp_write_msr(unsigned int cos, uint32_t val, enum cbm_type type)
+static void l3_cdp_write_msr(unsigned int cos, uint32_t val,
+ enum psr_val_type type)
{
- wrmsrl(((type == PSR_CBM_TYPE_L3_DATA) ?
+ wrmsrl(((type == PSR_VAL_TYPE_L3_DATA) ?
MSR_IA32_PSR_L3_MASK_DATA(cos) :
MSR_IA32_PSR_L3_MASK_CODE(cos)),
val);
@@ -398,23 +400,24 @@ static void l3_cdp_write_msr(unsigned int cos, uint32_t val, enum cbm_type type)
static const struct feat_props l3_cdp_props = {
.cos_num = 2,
- .type[0] = PSR_CBM_TYPE_L3_DATA,
- .type[1] = PSR_CBM_TYPE_L3_CODE,
- .alt_type = PSR_CBM_TYPE_L3,
+ .type[0] = PSR_VAL_TYPE_L3_DATA,
+ .type[1] = PSR_VAL_TYPE_L3_CODE,
+ .alt_type = PSR_VAL_TYPE_L3_CBM,
.get_feat_info = l3_cdp_get_feat_info,
.write_msr = l3_cdp_write_msr,
};
/* L2 CAT props */
-static void l2_cat_write_msr(unsigned int cos, uint32_t val, enum cbm_type type)
+static void l2_cat_write_msr(unsigned int cos, uint32_t val,
+ enum psr_val_type type)
{
wrmsrl(MSR_IA32_PSR_L2_MASK(cos), val);
}
static const struct feat_props l2_cat_props = {
.cos_num = 1,
- .type[0] = PSR_CBM_TYPE_L2,
- .alt_type = PSR_CBM_TYPE_UNKNOWN,
+ .type[0] = PSR_VAL_TYPE_L2_CBM,
+ .alt_type = PSR_VAL_TYPE_UNKNOWN,
.get_feat_info = cat_get_feat_info,
.write_msr = l2_cat_write_msr,
};
@@ -655,7 +658,7 @@ static struct psr_socket_info *get_socket_info(unsigned int socket)
return socket_info + socket;
}
-int psr_get_info(unsigned int socket, enum cbm_type type,
+int psr_get_info(unsigned int socket, enum psr_val_type type,
uint32_t data[], unsigned int array_len)
{
const struct psr_socket_info *info = get_socket_info(socket);
@@ -667,7 +670,7 @@ int psr_get_info(unsigned int socket, enum cbm_type type,
if ( IS_ERR(info) )
return PTR_ERR(info);
- feat_type = psr_cbm_type_to_feat_type(type);
+ feat_type = psr_val_type_to_feat_type(type);
if ( feat_type >= ARRAY_SIZE(info->features) )
return -ENOENT;
@@ -688,7 +691,7 @@ int psr_get_info(unsigned int socket, enum cbm_type type,
}
int psr_get_val(struct domain *d, unsigned int socket,
- uint32_t *val, enum cbm_type type)
+ uint32_t *val, enum psr_val_type type)
{
const struct psr_socket_info *info = get_socket_info(socket);
const struct feat_node *feat;
@@ -700,7 +703,7 @@ int psr_get_val(struct domain *d, unsigned int socket,
if ( IS_ERR(info) )
return PTR_ERR(info);
- feat_type = psr_cbm_type_to_feat_type(type);
+ feat_type = psr_val_type_to_feat_type(type);
if ( feat_type >= ARRAY_SIZE(info->features) )
return -ENOENT;
@@ -830,7 +833,7 @@ static int insert_val_into_array(uint32_t val[],
unsigned int array_len,
const struct psr_socket_info *info,
enum psr_feat_type feat_type,
- enum cbm_type type,
+ enum psr_val_type type,
uint32_t new_val)
{
const struct feat_node *feat;
@@ -866,8 +869,9 @@ static int insert_val_into_array(uint32_t val[],
/*
* Value setting position is same as feature array.
* For CDP, user may set both DATA and CODE to same value. For such case,
- * user input 'PSR_CBM_TYPE_L3' as type. The alternative type of CDP is same
- * as it. So we should set new_val to both of DATA and CODE under such case.
+ * user input 'PSR_VAL_TYPE_L3_CBM' as type. The alternative type of CDP is
+ * same as it. So we should set new_val to both of DATA and CODE under such
+ * case.
*/
for ( i = 0; i < props->cos_num; i++ )
{
@@ -1156,7 +1160,7 @@ static int write_psr_msrs(unsigned int socket, unsigned int cos,
}
int psr_set_val(struct domain *d, unsigned int socket,
- uint64_t new_val, enum cbm_type type)
+ uint64_t new_val, enum psr_val_type type)
{
unsigned int old_cos, array_len;
int cos, ret;
@@ -1172,7 +1176,7 @@ int psr_set_val(struct domain *d, unsigned int socket,
if ( new_val != val )
return -EINVAL;
- feat_type = psr_cbm_type_to_feat_type(type);
+ feat_type = psr_val_type_to_feat_type(type);
if ( feat_type >= ARRAY_SIZE(info->features) ||
!info->features[feat_type] )
return -ENOENT;
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index e7e390e..df54d38 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -179,7 +179,7 @@ long arch_do_sysctl(
case XEN_SYSCTL_PSR_CAT_get_l3_info:
{
ret = psr_get_info(sysctl->u.psr_alloc_op.target,
- PSR_CBM_TYPE_L3, data, ARRAY_SIZE(data));
+ PSR_VAL_TYPE_L3_CBM, data, ARRAY_SIZE(data));
if ( ret )
break;
@@ -198,7 +198,7 @@ long arch_do_sysctl(
case XEN_SYSCTL_PSR_CAT_get_l2_info:
{
ret = psr_get_info(sysctl->u.psr_alloc_op.target,
- PSR_CBM_TYPE_L2, data, ARRAY_SIZE(data));
+ PSR_VAL_TYPE_L2_CBM, data, ARRAY_SIZE(data));
if ( ret )
break;
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index 18a42f3..fb4a20a 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -53,12 +53,12 @@ struct psr_cmt {
struct psr_cmt_l3 l3;
};
-enum cbm_type {
- PSR_CBM_TYPE_L3,
- PSR_CBM_TYPE_L3_CODE,
- PSR_CBM_TYPE_L3_DATA,
- PSR_CBM_TYPE_L2,
- PSR_CBM_TYPE_UNKNOWN,
+enum psr_val_type {
+ PSR_VAL_TYPE_L3_CBM,
+ PSR_VAL_TYPE_L3_CODE,
+ PSR_VAL_TYPE_L3_DATA,
+ PSR_VAL_TYPE_L2_CBM,
+ PSR_VAL_TYPE_UNKNOWN,
};
extern struct psr_cmt *psr_cmt;
@@ -72,12 +72,12 @@ int psr_alloc_rmid(struct domain *d);
void psr_free_rmid(struct domain *d);
void psr_ctxt_switch_to(struct domain *d);
-int psr_get_info(unsigned int socket, enum cbm_type type,
+int psr_get_info(unsigned int socket, enum psr_val_type type,
uint32_t data[], unsigned int array_len);
int psr_get_val(struct domain *d, unsigned int socket,
- uint32_t *val, enum cbm_type type);
+ uint32_t *val, enum psr_val_type type);
int psr_set_val(struct domain *d, unsigned int socket,
- uint64_t val, enum cbm_type type);
+ uint64_t val, enum psr_val_type type);
void psr_domain_init(struct domain *d);
void psr_domain_free(struct domain *d);
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 04/15] x86: implement data structure and CPU init flow for MBA
2017-08-24 1:14 [PATCH v2 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (2 preceding siblings ...)
2017-08-24 1:14 ` [PATCH v2 03/15] x86: rename 'cbm_type' to 'psr_val_type' to make it general Yi Sun
@ 2017-08-24 1:14 ` Yi Sun
2017-08-29 13:44 ` Roger Pau Monné
2017-08-24 1:14 ` [PATCH v2 05/15] x86: implement get hw info " Yi Sun
` (10 subsequent siblings)
14 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-24 1:14 UTC (permalink / raw)
To: xen-devel
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, Yi Sun, julien.grall, mengxu, jbeulich, chao.p.peng,
roger.pau
This patch implements main data structures of MBA.
Like CAT features, MBA HW info has cos_max which means the max thrtl
register number, and thrtl_max which means the max throttle value
(delay value). It also has a flag to represent if the throttle
value is linear or not.
One thrtl register of MBA stores a throttle value for one or more
domains. The throttle value means the transaction time between L2
cache and next level memory to be delayed.
This patch also implements init flow for MBA and register stub
callback functions.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
v2:
- modify commit message to replace 'cos register' to 'thrtl register' to
make it accurate.
(suggested by Chao Peng)
- restore the place of the sentence to assign value to 'feat->cbm_len'
because the MBA init flow is splitted out as a separate function in v1.
(suggested by Chao Peng)
- add comment to explain what the MBA thrtl defaul value '0' stands for.
(suggested by Chao Peng)
- check 'thrtl_max' under linear mode. It could not be euqal or larger than
100.
(suggested by Chao Peng)
v1:
- rebase codes onto L2 CAT v15.
- move comment to appropriate place.
(suggested by Chao Peng)
- implement 'mba_init_feature' and keep 'cat_init_feature'.
(suggested by Chao Peng)
- keep 'regs.b' into a local variable to avoid reading CPUID every time.
(suggested by Chao Peng)
---
xen/arch/x86/psr.c | 141 +++++++++++++++++++++++++++++++++++-----
xen/include/asm-x86/msr-index.h | 1 +
xen/include/asm-x86/psr.h | 2 +
3 files changed, 126 insertions(+), 18 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index da62f81..f5e99ce 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -27,13 +27,16 @@
* - CMT Cache Monitoring Technology
* - COS/CLOS Class of Service. Also mean COS registers.
* - COS_MAX Max number of COS for the feature (minus 1)
+ * - MBA Memory Bandwidth Allocation
* - MSRs Machine Specific Registers
* - PSR Intel Platform Shared Resource
+ * - THRTL_MAX Max throttle value (delay value) of MBA
*/
#define PSR_CMT (1u << 0)
#define PSR_CAT (1u << 1)
#define PSR_CDP (1u << 2)
+#define PSR_MBA (1u << 3)
#define CAT_CBM_LEN_MASK 0x1f
#define CAT_COS_MAX_MASK 0xffff
@@ -60,10 +63,14 @@
*/
#define MAX_COS_NUM 2
+#define MBA_LINEAR (1u << 2)
+#define MBA_THRTL_MAX_MASK 0xfff
+
enum psr_feat_type {
FEAT_TYPE_L3_CAT,
FEAT_TYPE_L3_CDP,
FEAT_TYPE_L2_CAT,
+ FEAT_TYPE_MBA,
FEAT_TYPE_NUM,
FEAT_TYPE_UNKNOWN,
};
@@ -71,7 +78,6 @@ enum psr_feat_type {
/*
* This structure represents one feature.
* cos_max - The max COS registers number got through CPUID.
- * cbm_len - The length of CBM got through CPUID.
* cos_reg_val - Array to store the values of COS registers. One entry stores
* the value of one COS register.
* For L3 CAT and L2 CAT, one entry corresponds to one COS_ID.
@@ -80,9 +86,23 @@ enum psr_feat_type {
* cos_reg_val[1] (Code).
*/
struct feat_node {
- /* cos_max and cbm_len are common values for all features so far. */
+ /* cos_max is common values for all features so far. */
unsigned int cos_max;
- unsigned int cbm_len;
+
+ /* Feature specific HW info. */
+ union {
+ struct {
+ /* The length of CBM got through CPUID. */
+ unsigned int cbm_len;
+ } cat_info;
+
+ struct {
+ /* The max throttling value got through CPUID. */
+ unsigned int thrtl_max;
+ unsigned int linear;
+ } mba_info;
+ };
+
uint32_t cos_reg_val[MAX_COS_REG_CNT];
};
@@ -161,6 +181,7 @@ static DEFINE_PER_CPU(struct psr_assoc, psr_assoc);
*/
static struct feat_node *feat_l3;
static struct feat_node *feat_l2_cat;
+static struct feat_node *feat_mba;
/* Common functions */
#define cat_default_val(len) (0xffffffff >> (32 - (len)))
@@ -273,7 +294,7 @@ static bool psr_check_cbm(unsigned int cbm_len, unsigned long cbm)
return true;
}
-/* CAT common functions implementation. */
+/* Implementation of allocation features' functions. */
static int cat_init_feature(const struct cpuid_leaf *regs,
struct feat_node *feat,
struct psr_socket_info *info,
@@ -289,8 +310,8 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
if ( !regs->a || !regs->d )
return -ENOENT;
- feat->cbm_len = (regs->a & CAT_CBM_LEN_MASK) + 1;
feat->cos_max = min(opt_cos_max, regs->d & CAT_COS_MAX_MASK);
+ feat->cat_info.cbm_len = (regs->a & CAT_CBM_LEN_MASK) + 1;
switch ( type )
{
@@ -300,12 +321,12 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
return -ENOENT;
/* We reserve cos=0 as default cbm (all bits within cbm_len are 1). */
- feat->cos_reg_val[0] = cat_default_val(feat->cbm_len);
+ feat->cos_reg_val[0] = cat_default_val(feat->cat_info.cbm_len);
wrmsrl((type == FEAT_TYPE_L3_CAT ?
MSR_IA32_PSR_L3_MASK(0) :
MSR_IA32_PSR_L2_MASK(0)),
- cat_default_val(feat->cbm_len));
+ cat_default_val(feat->cat_info.cbm_len));
break;
@@ -320,11 +341,13 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
feat->cos_max = (feat->cos_max - 1) >> 1;
/* We reserve cos=0 as default cbm (all bits within cbm_len are 1). */
- get_cdp_code(feat, 0) = cat_default_val(feat->cbm_len);
- get_cdp_data(feat, 0) = cat_default_val(feat->cbm_len);
+ get_cdp_code(feat, 0) = cat_default_val(feat->cat_info.cbm_len);
+ get_cdp_data(feat, 0) = cat_default_val(feat->cat_info.cbm_len);
- wrmsrl(MSR_IA32_PSR_L3_MASK(0), cat_default_val(feat->cbm_len));
- wrmsrl(MSR_IA32_PSR_L3_MASK(1), cat_default_val(feat->cbm_len));
+ wrmsrl(MSR_IA32_PSR_L3_MASK(0),
+ cat_default_val(feat->cat_info.cbm_len));
+ wrmsrl(MSR_IA32_PSR_L3_MASK(1),
+ cat_default_val(feat->cat_info.cbm_len));
rdmsrl(MSR_IA32_PSR_L3_QOS_CFG, val);
wrmsrl(MSR_IA32_PSR_L3_QOS_CFG,
val | (1ull << PSR_L3_QOS_CDP_ENABLE_BIT));
@@ -344,7 +367,51 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
printk(XENLOG_INFO "%s: enabled on socket %u, cos_max:%u, cbm_len:%u\n",
cat_feat_name[type], cpu_to_socket(smp_processor_id()),
- feat->cos_max, feat->cbm_len);
+ feat->cos_max, feat->cat_info.cbm_len);
+
+ return 0;
+}
+
+static int mba_init_feature(const struct cpuid_leaf *regs,
+ struct feat_node *feat,
+ struct psr_socket_info *info,
+ enum psr_feat_type type)
+{
+ /* No valid value so do not enable feature. */
+ if ( !regs->a || !regs->d )
+ return -ENOENT;
+
+ if ( type != FEAT_TYPE_MBA )
+ return -ENOENT;
+
+ feat->cos_max = min(opt_cos_max, regs->d & CAT_COS_MAX_MASK);
+ if ( feat->cos_max < 1 )
+ return -ENOENT;
+
+ feat->mba_info.thrtl_max = (regs->a & MBA_THRTL_MAX_MASK) + 1;
+
+ if ( regs->c & MBA_LINEAR )
+ {
+ feat->mba_info.linear = 1;
+
+ if ( feat->mba_info.thrtl_max >= 100 )
+ return -ENOENT;
+ }
+
+ /* We reserve cos=0 as default thrtl (0) which means no delay. */
+ feat->cos_reg_val[0] = 0;
+ wrmsrl(MSR_IA32_PSR_MBA_MASK(0), 0);
+
+ /* Add this feature into array. */
+ info->features[type] = feat;
+
+ if ( !opt_cpu_info )
+ return 0;
+
+ printk(XENLOG_INFO "MBA: enabled on socket %u, cos_max:%u,"
+ "thrtl_max:%u, linear:%u.\n",
+ cpu_to_socket(smp_processor_id()),
+ feat->cos_max, feat->mba_info.thrtl_max, feat->mba_info.linear);
return 0;
}
@@ -356,7 +423,7 @@ static bool cat_get_feat_info(const struct feat_node *feat,
return false;
data[PSR_INFO_IDX_COS_MAX] = feat->cos_max;
- data[PSR_INFO_IDX_CAT_CBM_LEN] = feat->cbm_len;
+ data[PSR_INFO_IDX_CAT_CBM_LEN] = feat->cat_info.cbm_len;
data[PSR_INFO_IDX_CAT_FLAG] = 0;
return true;
@@ -422,6 +489,26 @@ static const struct feat_props l2_cat_props = {
.write_msr = l2_cat_write_msr,
};
+/* MBA props */
+static bool mba_get_feat_info(const struct feat_node *feat,
+ uint32_t data[], unsigned int array_len)
+{
+ return false;
+}
+
+static void mba_write_msr(unsigned int cos, uint32_t val,
+ enum psr_val_type type)
+{
+}
+
+static const struct feat_props mba_props = {
+ .cos_num = 1,
+ .type[0] = PSR_VAL_TYPE_MBA,
+ .alt_type = PSR_VAL_TYPE_UNKNOWN,
+ .get_feat_info = mba_get_feat_info,
+ .write_msr = mba_write_msr,
+};
+
static void __init parse_psr_bool(char *s, char *value, char *feature,
unsigned int mask)
{
@@ -457,6 +544,7 @@ static void __init parse_psr_param(char *s)
parse_psr_bool(s, val_str, "cmt", PSR_CMT);
parse_psr_bool(s, val_str, "cat", PSR_CAT);
parse_psr_bool(s, val_str, "cdp", PSR_CDP);
+ parse_psr_bool(s, val_str, "mba", PSR_MBA);
if ( val_str && !strcmp(s, "rmid_max") )
opt_rmid_max = simple_strtoul(val_str, NULL, 0);
@@ -863,7 +951,7 @@ static int insert_val_into_array(uint32_t val[],
if ( array_len < props->cos_num )
return -ENOSPC;
- if ( !psr_check_cbm(feat->cbm_len, new_val) )
+ if ( !psr_check_cbm(feat->cat_info.cbm_len, new_val) )
return -EINVAL;
/*
@@ -1381,6 +1469,10 @@ static int psr_cpu_prepare(void)
(feat_l2_cat = xzalloc(struct feat_node)) == NULL )
return -ENOMEM;
+ if ( feat_mba == NULL &&
+ (feat_mba = xzalloc(struct feat_node)) == NULL )
+ return -ENOMEM;
+
return 0;
}
@@ -1390,6 +1482,7 @@ static void psr_cpu_init(void)
unsigned int socket, cpu = smp_processor_id();
struct feat_node *feat;
struct cpuid_leaf regs;
+ uint32_t reg_b;
if ( !psr_alloc_feat_enabled() || !boot_cpu_has(X86_FEATURE_PQE) )
goto assoc_init;
@@ -1408,7 +1501,8 @@ static void psr_cpu_init(void)
spin_lock_init(&info->ref_lock);
cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 0, ®s);
- if ( regs.b & PSR_RESOURCE_TYPE_L3 )
+ reg_b = regs.b;
+ if ( reg_b & PSR_RESOURCE_TYPE_L3 )
{
cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 1, ®s);
@@ -1429,8 +1523,7 @@ static void psr_cpu_init(void)
}
}
- cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 0, ®s);
- if ( regs.b & PSR_RESOURCE_TYPE_L2 )
+ if ( reg_b & PSR_RESOURCE_TYPE_L2 )
{
cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 2, ®s);
@@ -1442,6 +1535,18 @@ static void psr_cpu_init(void)
feat_l2_cat = feat;
}
+ if ( reg_b & PSR_RESOURCE_TYPE_MBA )
+ {
+ cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 3, ®s);
+
+ feat = feat_mba;
+ feat_mba = NULL;
+ if ( !mba_init_feature(®s, feat, info, FEAT_TYPE_MBA) )
+ feat_props[FEAT_TYPE_MBA] = &mba_props;
+ else
+ feat_mba = feat;
+ }
+
info->feat_init = true;
assoc_init:
@@ -1501,7 +1606,7 @@ static int __init psr_presmp_init(void)
if ( (opt_psr & PSR_CMT) && opt_rmid_max )
init_psr_cmt(opt_rmid_max);
- if ( opt_psr & (PSR_CAT | PSR_CDP) )
+ if ( opt_psr & (PSR_CAT | PSR_CDP | PSR_MBA) )
init_psr();
if ( psr_cpu_prepare() )
diff --git a/xen/include/asm-x86/msr-index.h b/xen/include/asm-x86/msr-index.h
index 4e08de6..41f1677 100644
--- a/xen/include/asm-x86/msr-index.h
+++ b/xen/include/asm-x86/msr-index.h
@@ -348,6 +348,7 @@
#define MSR_IA32_PSR_L3_MASK_CODE(n) (0x00000c90 + (n) * 2 + 1)
#define MSR_IA32_PSR_L3_MASK_DATA(n) (0x00000c90 + (n) * 2)
#define MSR_IA32_PSR_L2_MASK(n) (0x00000d10 + (n))
+#define MSR_IA32_PSR_MBA_MASK(n) (0x00000d50 + (n))
/* Intel Model 6 */
#define MSR_P6_PERFCTR(n) (0x000000c1 + (n))
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index fb4a20a..10043ac 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -24,6 +24,7 @@
/* Resource Type Enumeration */
#define PSR_RESOURCE_TYPE_L3 0x2
#define PSR_RESOURCE_TYPE_L2 0x4
+#define PSR_RESOURCE_TYPE_MBA 0x8
/* L3 Monitoring Features */
#define PSR_CMT_L3_OCCUPANCY 0x1
@@ -58,6 +59,7 @@ enum psr_val_type {
PSR_VAL_TYPE_L3_CODE,
PSR_VAL_TYPE_L3_DATA,
PSR_VAL_TYPE_L2_CBM,
+ PSR_VAL_TYPE_MBA,
PSR_VAL_TYPE_UNKNOWN,
};
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 05/15] x86: implement get hw info flow for MBA
2017-08-24 1:14 [PATCH v2 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (3 preceding siblings ...)
2017-08-24 1:14 ` [PATCH v2 04/15] x86: implement data structure and CPU init flow for MBA Yi Sun
@ 2017-08-24 1:14 ` Yi Sun
2017-08-29 15:01 ` Roger Pau Monné
2017-08-24 1:14 ` [PATCH v2 06/15] x86: implement get value interface " Yi Sun
` (9 subsequent siblings)
14 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-24 1:14 UTC (permalink / raw)
To: xen-devel
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, Yi Sun, julien.grall, mengxu, jbeulich, chao.p.peng,
roger.pau
This patch implements get HW info flow for MBA including its callback
function and sysctl interface.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
v2:
- use 'XEN_SYSCTL_PSR_MBA_LINEAR' to set MBA feature HW info.
(suggested by Chao Peng)
v1:
- sort 'PSR_INFO_IDX_' macros as feature.
(suggested by Chao Peng)
- rename 'PSR_INFO_IDX_MBA_LINEAR' to 'PSR_INFO_IDX_MBA_FLAG'.
- rename 'linear' in 'struct mba_info' to 'flags' for future extension.
(suggested by Chao Peng)
---
xen/arch/x86/psr.c | 17 ++++++++++++++++-
xen/arch/x86/sysctl.c | 19 +++++++++++++++++++
xen/include/asm-x86/psr.h | 2 ++
xen/include/public/sysctl.h | 8 ++++++++
4 files changed, 45 insertions(+), 1 deletion(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index f5e99ce..4a0c982 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -264,6 +264,10 @@ static enum psr_feat_type psr_val_type_to_feat_type(enum psr_val_type type)
feat_type = FEAT_TYPE_L2_CAT;
break;
+ case PSR_VAL_TYPE_MBA:
+ feat_type = FEAT_TYPE_MBA;
+ break;
+
default:
ASSERT_UNREACHABLE();
}
@@ -493,7 +497,18 @@ static const struct feat_props l2_cat_props = {
static bool mba_get_feat_info(const struct feat_node *feat,
uint32_t data[], unsigned int array_len)
{
- return false;
+ if ( array_len != PSR_INFO_ARRAY_SIZE )
+ return false;
+
+ data[PSR_INFO_IDX_COS_MAX] = feat->cos_max;
+ data[PSR_INFO_IDX_MBA_THRTL_MAX] = feat->mba_info.thrtl_max;
+
+ if ( feat->mba_info.linear )
+ data[PSR_INFO_IDX_MBA_FLAG] |= XEN_SYSCTL_PSR_MBA_LINEAR;
+ else
+ data[PSR_INFO_IDX_MBA_FLAG] &= ~XEN_SYSCTL_PSR_MBA_LINEAR;
+
+ return true;
}
static void mba_write_msr(unsigned int cos, uint32_t val,
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index df54d38..cbab85a 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -214,6 +214,25 @@ long arch_do_sysctl(
break;
}
+ case XEN_SYSCTL_PSR_MBA_get_info:
+ {
+ ret = psr_get_info(sysctl->u.psr_alloc_op.target,
+ PSR_VAL_TYPE_MBA, data, ARRAY_SIZE(data));
+ if ( ret )
+ break;
+
+ sysctl->u.psr_alloc_op.u.mba_info.cos_max =
+ data[PSR_INFO_IDX_COS_MAX];
+ sysctl->u.psr_alloc_op.u.mba_info.thrtl_max =
+ data[PSR_INFO_IDX_MBA_THRTL_MAX];
+ sysctl->u.psr_alloc_op.u.mba_info.flags =
+ data[PSR_INFO_IDX_MBA_FLAG];
+
+ if ( !ret && __copy_field_to_guest(u_sysctl, sysctl, u.psr_alloc_op) )
+ ret = -EFAULT;
+ break;
+ }
+
default:
ret = -EOPNOTSUPP;
break;
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index 10043ac..a7fb625 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -39,6 +39,8 @@
#define PSR_INFO_IDX_COS_MAX 0
#define PSR_INFO_IDX_CAT_CBM_LEN 1
#define PSR_INFO_IDX_CAT_FLAG 2
+#define PSR_INFO_IDX_MBA_THRTL_MAX 1
+#define PSR_INFO_IDX_MBA_FLAG 2
#define PSR_INFO_ARRAY_SIZE 3
struct psr_cmt_l3 {
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index fcebab1..169e7e6 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -745,6 +745,7 @@ DEFINE_XEN_GUEST_HANDLE(xen_sysctl_pcitopoinfo_t);
#define XEN_SYSCTL_PSR_CAT_get_l3_info 0
#define XEN_SYSCTL_PSR_CAT_get_l2_info 1
+#define XEN_SYSCTL_PSR_MBA_get_info 2
struct xen_sysctl_psr_alloc_op {
uint32_t cmd; /* IN: XEN_SYSCTL_PSR_CAT_* */
uint32_t target; /* IN */
@@ -755,6 +756,13 @@ struct xen_sysctl_psr_alloc_op {
#define XEN_SYSCTL_PSR_CAT_L3_CDP (1u << 0)
uint32_t flags; /* OUT: CAT flags */
} cat_info;
+
+ struct {
+ uint32_t thrtl_max; /* OUT: Maximum throttle */
+ uint32_t cos_max; /* OUT: Maximum COS */
+#define XEN_SYSCTL_PSR_MBA_LINEAR (1u << 0)
+ uint32_t flags; /* OUT: MBA flags */
+ } mba_info;
} u;
};
typedef struct xen_sysctl_psr_alloc_op xen_sysctl_psr_alloc_op_t;
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 06/15] x86: implement get value interface for MBA
2017-08-24 1:14 [PATCH v2 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (4 preceding siblings ...)
2017-08-24 1:14 ` [PATCH v2 05/15] x86: implement get hw info " Yi Sun
@ 2017-08-24 1:14 ` Yi Sun
2017-08-29 15:04 ` Roger Pau Monné
2017-08-24 1:14 ` [PATCH v2 07/15] x86: implement set value flow " Yi Sun
` (8 subsequent siblings)
14 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-24 1:14 UTC (permalink / raw)
To: xen-devel
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, Yi Sun, julien.grall, mengxu, jbeulich, chao.p.peng,
roger.pau
This patch implements get value domctl interface for MBA.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
xen/arch/x86/domctl.c | 7 +++++++
xen/include/public/domctl.h | 1 +
2 files changed, 8 insertions(+)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index e1c7c70..4936bcb 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1496,6 +1496,13 @@ long arch_do_domctl(
copyback = true;
break;
+ case XEN_DOMCTL_PSR_MBA_OP_GET_THRTL:
+ ret = psr_get_val(d, domctl->u.psr_alloc_op.target,
+ &val32, PSR_VAL_TYPE_MBA);
+ domctl->u.psr_alloc_op.data = val32;
+ copyback = true;
+ break;
+
default:
ret = -EOPNOTSUPP;
break;
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 4bb4399..a577a3e 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1144,6 +1144,7 @@ struct xen_domctl_psr_alloc_op {
#define XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA 5
#define XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM 6
#define XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM 7
+#define XEN_DOMCTL_PSR_MBA_OP_GET_THRTL 9
uint32_t cmd; /* IN: XEN_DOMCTL_PSR_CAT_OP_* */
uint32_t target; /* IN */
uint64_t data; /* IN/OUT */
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 07/15] x86: implement set value flow for MBA
2017-08-24 1:14 [PATCH v2 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (5 preceding siblings ...)
2017-08-24 1:14 ` [PATCH v2 06/15] x86: implement get value interface " Yi Sun
@ 2017-08-24 1:14 ` Yi Sun
2017-08-30 8:31 ` Roger Pau Monné
2017-08-24 1:14 ` [PATCH v2 08/15] tools: create general interfaces to support psr allocation features Yi Sun
` (7 subsequent siblings)
14 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-24 1:14 UTC (permalink / raw)
To: xen-devel
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, Yi Sun, julien.grall, mengxu, jbeulich, chao.p.peng,
roger.pau
This patch implements set value flow for MBA including its callback
function and domctl interface.
It also changes the memebers in 'cos_write_info' to transfer the
feature array, feature properties array and value array. Then, we
can write all features values on the cos id into MSRs.
Because multiple features may co-exist, we need handle all features to write
values of them into a COS register with new COS ID. E.g:
1. L3 CAT and MBA co-exist.
2. Dom1 and Dom2 share a same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff,
the MBA Thrtle of Dom1 is 0xa.
3. User wants to change MBA Thrtl of Dom1 to be 0x14. Because COS ID 2 is
used by Dom2 too, we have to pick a new COS ID 3. The original values of
Dom1 on COS ID 3 may be below:
---------
| COS 3 |
---------
L3 CAT | 0x7ff |
---------
MBA | 0x0 |
---------
4. After setting, the L3 CAT CBM value of Dom1 should be kept and the new MBA
Thrtl is set. So, the values on COS ID 3 should be below.
---------
| COS 3 |
---------
L3 CAT | 0x1ff |
---------
MBA | 0x14 |
---------
So, we should write all features values into their MSRs. That requires the
feature array, feature properties array and value array are input.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
v2:
- remove linear mode 'thrtl_max' check in 'mba_check_thrtl' because it has
been checked in 'mba_init_feature'.
(suggested by Chao Peng)
- for non-linear mode, check if '*thrtl' is not 0 in 'mba_check_thrtl'. If
it is 0, we do not need to change it.
(suggested by Chao Peng)
- move comments to explain changes of 'cos_write_info' from psr.c to commit
message.
(suggested by Chao Peng)
---
xen/arch/x86/domctl.c | 6 ++
xen/arch/x86/psr.c | 150 ++++++++++++++++++++++++++++++--------------
xen/include/public/domctl.h | 1 +
3 files changed, 109 insertions(+), 48 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 4936bcb..0ae4799 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1468,6 +1468,12 @@ long arch_do_domctl(
PSR_VAL_TYPE_L2_CBM);
break;
+ case XEN_DOMCTL_PSR_MBA_OP_SET_THRTL:
+ ret = psr_set_val(d, domctl->u.psr_alloc_op.target,
+ domctl->u.psr_alloc_op.data,
+ PSR_VAL_TYPE_MBA);
+ break;
+
case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM:
ret = psr_get_val(d, domctl->u.psr_alloc_op.target,
&val32, PSR_VAL_TYPE_L3_CBM);
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 4a0c982..ce82975 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -138,6 +138,12 @@ static const struct feat_props {
/* write_msr is used to write out feature MSR register. */
void (*write_msr)(unsigned int cos, uint32_t val, enum psr_val_type type);
+
+ /*
+ * check_val is used to check if input val fulfills SDM requirement.
+ * Change it to valid value if SDM allows.
+ */
+ bool (*check_val)(const struct feat_node *feat, unsigned long *val);
} *feat_props[FEAT_TYPE_NUM];
/*
@@ -275,29 +281,6 @@ static enum psr_feat_type psr_val_type_to_feat_type(enum psr_val_type type)
return feat_type;
}
-static bool psr_check_cbm(unsigned int cbm_len, unsigned long cbm)
-{
- unsigned int first_bit, zero_bit;
-
- /* Set bits should only in the range of [0, cbm_len]. */
- if ( cbm & (~0ul << cbm_len) )
- return false;
-
- /* At least one bit need to be set. */
- if ( cbm == 0 )
- return false;
-
- first_bit = find_first_bit(&cbm, cbm_len);
- zero_bit = find_next_zero_bit(&cbm, cbm_len, first_bit);
-
- /* Set bits should be contiguous. */
- if ( zero_bit < cbm_len &&
- find_next_bit(&cbm, cbm_len, zero_bit) < cbm_len )
- return false;
-
- return true;
-}
-
/* Implementation of allocation features' functions. */
static int cat_init_feature(const struct cpuid_leaf *regs,
struct feat_node *feat,
@@ -433,6 +416,30 @@ static bool cat_get_feat_info(const struct feat_node *feat,
return true;
}
+static bool cat_check_cbm(const struct feat_node *feat, unsigned long *cbm)
+{
+ unsigned int first_bit, zero_bit;
+ unsigned int cbm_len = feat->cat_info.cbm_len;
+
+ /* Set bits should only in the range of [0, cbm_len]. */
+ if ( *cbm & (~0ul << cbm_len) )
+ return false;
+
+ /* At least one bit need to be set. */
+ if ( *cbm == 0 )
+ return false;
+
+ first_bit = find_first_bit(cbm, cbm_len);
+ zero_bit = find_next_zero_bit(cbm, cbm_len, first_bit);
+
+ /* Set bits should be contiguous. */
+ if ( zero_bit < cbm_len &&
+ find_next_bit(cbm, cbm_len, zero_bit) < cbm_len )
+ return false;
+
+ return true;
+}
+
/* L3 CAT props */
static void l3_cat_write_msr(unsigned int cos, uint32_t val,
enum psr_val_type type)
@@ -446,6 +453,7 @@ static const struct feat_props l3_cat_props = {
.alt_type = PSR_VAL_TYPE_UNKNOWN,
.get_feat_info = cat_get_feat_info,
.write_msr = l3_cat_write_msr,
+ .check_val = cat_check_cbm,
};
/* L3 CDP props */
@@ -476,6 +484,7 @@ static const struct feat_props l3_cdp_props = {
.alt_type = PSR_VAL_TYPE_L3_CBM,
.get_feat_info = l3_cdp_get_feat_info,
.write_msr = l3_cdp_write_msr,
+ .check_val = cat_check_cbm,
};
/* L2 CAT props */
@@ -491,6 +500,7 @@ static const struct feat_props l2_cat_props = {
.alt_type = PSR_VAL_TYPE_UNKNOWN,
.get_feat_info = cat_get_feat_info,
.write_msr = l2_cat_write_msr,
+ .check_val = cat_check_cbm,
};
/* MBA props */
@@ -514,6 +524,40 @@ static bool mba_get_feat_info(const struct feat_node *feat,
static void mba_write_msr(unsigned int cos, uint32_t val,
enum psr_val_type type)
{
+ wrmsrl(MSR_IA32_PSR_MBA_MASK(cos), val);
+}
+
+static bool mba_check_thrtl(const struct feat_node *feat, unsigned long *thrtl)
+{
+ if ( *thrtl > feat->mba_info.thrtl_max )
+ return false;
+
+ /*
+ * Per SDM (chapter "Memory Bandwidth Allocation Configuration"):
+ * 1. Linear mode: In the linear mode the input precision is defined
+ * as 100-(MBA_MAX). For instance, if the MBA_MAX value is 90, the
+ * input precision is 10%. Values not an even multiple of the
+ * precision (e.g., 12%) will be rounded down (e.g., to 10% delay
+ * applied).
+ * 2. Non-linear mode: Input delay values are powers-of-two from zero
+ * to the MBA_MAX value from CPUID. In this case any values not a
+ * power of two will be rounded down the next nearest power of two.
+ */
+ if ( feat->mba_info.linear )
+ {
+ unsigned int mod;
+
+ mod = *thrtl % (100 - feat->mba_info.thrtl_max);
+ *thrtl -= mod;
+ }
+ else
+ {
+ /* Not power of 2. */
+ if ( *thrtl && (*thrtl & (*thrtl - 1)) )
+ *thrtl = *thrtl & (1 << (flsl(*thrtl) - 1));
+ }
+
+ return true;
}
static const struct feat_props mba_props = {
@@ -522,6 +566,7 @@ static const struct feat_props mba_props = {
.alt_type = PSR_VAL_TYPE_UNKNOWN,
.get_feat_info = mba_get_feat_info,
.write_msr = mba_write_msr,
+ .check_val = mba_check_thrtl,
};
static void __init parse_psr_bool(char *s, char *value, char *feature,
@@ -942,6 +987,7 @@ static int insert_val_into_array(uint32_t val[],
const struct feat_node *feat;
const struct feat_props *props;
unsigned int i;
+ unsigned long check_val = new_val;
int ret;
ASSERT(feat_type < FEAT_TYPE_NUM);
@@ -966,9 +1012,11 @@ static int insert_val_into_array(uint32_t val[],
if ( array_len < props->cos_num )
return -ENOSPC;
- if ( !psr_check_cbm(feat->cat_info.cbm_len, new_val) )
+ if ( !props->check_val(feat, &check_val) )
return -EINVAL;
+ new_val = check_val;
+
/*
* Value setting position is same as feature array.
* For CDP, user may set both DATA and CODE to same value. For such case,
@@ -1198,25 +1246,42 @@ static unsigned int get_socket_cpu(unsigned int socket)
struct cos_write_info
{
unsigned int cos;
- struct feat_node *feature;
+ struct feat_node **features;
const uint32_t *val;
- const struct feat_props *props;
+ unsigned int array_len;
+ const struct feat_props **props;
};
static void do_write_psr_msrs(void *data)
{
const struct cos_write_info *info = data;
- struct feat_node *feat = info->feature;
- const struct feat_props *props = info->props;
- unsigned int i, cos = info->cos, cos_num = props->cos_num;
+ unsigned int i, j, index = 0, array_len = info->array_len, cos = info->cos;
+ const uint32_t *val_array = info->val;
- for ( i = 0; i < cos_num; i++ )
+ for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
{
- if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
+ struct feat_node *feat = info->features[i];
+ const struct feat_props *props = info->props[i];
+ unsigned int cos_num;
+
+ if ( !feat || !props )
+ continue;
+
+ cos_num = props->cos_num;
+ if ( array_len < cos_num )
+ return;
+
+ for ( j = 0; j < cos_num; j++ )
{
- feat->cos_reg_val[cos * cos_num + i] = info->val[i];
- props->write_msr(cos, info->val[i], props->type[i]);
+ if ( feat->cos_reg_val[cos * cos_num + j] != val_array[index + j] )
+ {
+ feat->cos_reg_val[cos * cos_num + j] = val_array[index + j];
+ props->write_msr(cos, val_array[index + j], props->type[j]);
+ }
}
+
+ array_len -= cos_num;
+ index += cos_num;
}
}
@@ -1224,30 +1289,19 @@ static int write_psr_msrs(unsigned int socket, unsigned int cos,
const uint32_t val[], unsigned int array_len,
enum psr_feat_type feat_type)
{
- int ret;
struct psr_socket_info *info = get_socket_info(socket);
struct cos_write_info data =
{
.cos = cos,
- .feature = info->features[feat_type],
- .props = feat_props[feat_type],
+ .features = info->features,
+ .val = val,
+ .array_len = array_len,
+ .props = feat_props,
};
if ( cos > info->features[feat_type]->cos_max )
return -EINVAL;
- /* Skip to the feature's value head. */
- ret = skip_prior_features(&array_len, feat_type);
- if ( ret < 0 )
- return ret;
-
- val += ret;
-
- if ( array_len < feat_props[feat_type]->cos_num )
- return -ENOSPC;
-
- data.val = val;
-
if ( socket == cpu_to_socket(smp_processor_id()) )
do_write_psr_msrs(&data);
else
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index a577a3e..8826cfb 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1144,6 +1144,7 @@ struct xen_domctl_psr_alloc_op {
#define XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA 5
#define XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM 6
#define XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM 7
+#define XEN_DOMCTL_PSR_MBA_OP_SET_THRTL 8
#define XEN_DOMCTL_PSR_MBA_OP_GET_THRTL 9
uint32_t cmd; /* IN: XEN_DOMCTL_PSR_CAT_OP_* */
uint32_t target; /* IN */
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 08/15] tools: create general interfaces to support psr allocation features
2017-08-24 1:14 [PATCH v2 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (6 preceding siblings ...)
2017-08-24 1:14 ` [PATCH v2 07/15] x86: implement set value flow " Yi Sun
@ 2017-08-24 1:14 ` Yi Sun
2017-08-30 8:42 ` Roger Pau Monné
2017-08-24 1:14 ` [PATCH v2 09/15] tools: implement the new libxc get hw info interface Yi Sun
` (6 subsequent siblings)
14 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-24 1:14 UTC (permalink / raw)
To: xen-devel
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, Yi Sun, julien.grall, mengxu, jbeulich, chao.p.peng,
roger.pau
This patch creates general interfaces in libxl to support all psr
allocation features.
Add 'LIBXL_HAVE_PSR_MBA' to indicate interface change.
Please note, the functionality cannot work until later patches
are applied.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
v2:
- remove '_INFO' in 'libxl_psr_feat_type' and make corresponding
changes in 'libxl_psr_hw_info'.
(suggested by Chao Peng)
---
tools/libxl/libxl.h | 35 ++++++++++++++++++++++++++++++++++-
tools/libxl/libxl_psr.c | 25 +++++++++++++++++++++++++
tools/libxl/libxl_types.idl | 22 ++++++++++++++++++++++
3 files changed, 81 insertions(+), 1 deletion(-)
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 229e289..c1d804c 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -931,6 +931,13 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const libxl_mac *src);
#define LIBXL_HAVE_PSR_L2_CAT 1
/*
+ * LIBXL_HAVE_PSR_MBA
+ *
+ * If this is defined, the Memory Bandwidth Allocation feature is supported.
+ */
+#define LIBXL_HAVE_PSR_MBA 1
+
+/*
* LIBXL_HAVE_MCA_CAPS
*
* If this is defined, setting MCA capabilities for HVM domain is supported.
@@ -2219,7 +2226,33 @@ int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
int libxl_psr_cat_get_l3_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
int *nr);
void libxl_psr_cat_info_list_free(libxl_psr_cat_info *list, int nr);
-#endif
+
+#ifdef LIBXL_HAVE_PSR_MBA
+/*
+ * Function to set a domain's value. It operates on a single or multiple
+ * target(s) defined in 'target_map'. 'target_map' specifies all the sockets
+ * to be operated on.
+ */
+int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_cbm_type type, libxl_bitmap *target_map,
+ uint64_t val);
+/*
+ * Function to get a domain's cbm. It operates on a single 'target'.
+ * 'target' specifies which socket to be operated on.
+ */
+int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_cbm_type type, unsigned int target,
+ uint64_t *val);
+/*
+ * On success, the function returns an array of elements in 'info',
+ * and the length in 'nr'.
+ */
+int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_hw_info **info,
+ unsigned int *nr, libxl_psr_feat_type type,
+ unsigned int lvl);
+void libxl_psr_hw_info_list_free(libxl_psr_hw_info *list, unsigned int nr);
+#endif /* LIBXL_HAVE_PSR_MBA */
+#endif /* LIBXL_HAVE_PSR_CAT */
/* misc */
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index f55ba1e..cf368ba 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -425,6 +425,31 @@ void libxl_psr_cat_info_list_free(libxl_psr_cat_info *list, int nr)
free(list);
}
+int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_cbm_type type, libxl_bitmap *target_map,
+ uint64_t val)
+{
+ return ERROR_FAIL;
+}
+
+int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_cbm_type type, unsigned int target,
+ uint64_t *val)
+{
+ return ERROR_FAIL;
+}
+
+int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_hw_info **info,
+ unsigned int *nr, libxl_psr_feat_type type,
+ unsigned int lvl)
+{
+ return ERROR_FAIL;
+}
+
+void libxl_psr_hw_info_list_free(libxl_psr_hw_info *list, unsigned int nr)
+{
+}
+
/*
* Local variables:
* mode: C
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 6e80d36..ab847f8 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -977,6 +977,7 @@ libxl_psr_cbm_type = Enumeration("psr_cbm_type", [
(2, "L3_CBM_CODE"),
(3, "L3_CBM_DATA"),
(4, "L2_CBM"),
+ (5, "MBA_THRTL"),
])
libxl_psr_cat_info = Struct("psr_cat_info", [
@@ -985,3 +986,24 @@ libxl_psr_cat_info = Struct("psr_cat_info", [
("cbm_len", uint32),
("cdp_enabled", bool),
])
+
+libxl_psr_feat_type = Enumeration("psr_feat_type", [
+ (1, "CAT"),
+ (2, "MBA"),
+ ])
+
+libxl_psr_hw_info = Struct("psr_hw_info", [
+ ("id", uint32),
+ ("u", KeyedUnion(None, libxl_psr_feat_type, "type",
+ [("cat", Struct(None, [
+ ("cos_max", uint32),
+ ("cbm_len", uint32),
+ ("cdp_enabled", bool),
+ ])),
+ ("mba", Struct(None, [
+ ("cos_max", uint32),
+ ("thrtl_max", uint32),
+ ("linear", bool),
+ ])),
+ ]))
+ ], dir=DIR_OUT)
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 09/15] tools: implement the new libxc get hw info interface
2017-08-24 1:14 [PATCH v2 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (7 preceding siblings ...)
2017-08-24 1:14 ` [PATCH v2 08/15] tools: create general interfaces to support psr allocation features Yi Sun
@ 2017-08-24 1:14 ` Yi Sun
2017-08-30 8:58 ` Roger Pau Monné
2017-08-24 1:14 ` [PATCH v2 10/15] tools: implement the new libxl " Yi Sun
` (5 subsequent siblings)
14 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-24 1:14 UTC (permalink / raw)
To: xen-devel
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, Yi Sun, julien.grall, mengxu, jbeulich, chao.p.peng,
roger.pau
This patch implements a new libxc get hw info interface and corresponding
data structures. It also changes libxl_psr.c to call this new interface.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
v2:
- split this patch out from a big patch in v1.
(suggested by Wei Liu)
- change 'CAT_INFO' and 'MBA_INFO' to 'CAT' and 'MBA'.
(suggested by Chao Peng)
---
tools/libxc/include/xenctrl.h | 30 +++++++++++++++++++++++++---
tools/libxc/xc_psr.c | 46 ++++++++++++++++++++++++++++++++-----------
tools/libxl/libxl_psr.c | 34 ++++++++++++++++++++++++++++++--
3 files changed, 93 insertions(+), 17 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index c7710b8..81a6f48 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2458,6 +2458,31 @@ enum xc_psr_cat_type {
};
typedef enum xc_psr_cat_type xc_psr_cat_type;
+enum xc_psr_feat_type {
+ XC_PSR_FEAT_UNKNOWN,
+ XC_PSR_FEAT_CAT_L3,
+ XC_PSR_FEAT_CAT_L2,
+ XC_PSR_FEAT_MBA,
+};
+typedef enum xc_psr_feat_type xc_psr_feat_type;
+
+struct xc_psr_hw_info {
+ union {
+ struct {
+ uint32_t cos_max;
+ uint32_t cbm_len;
+ bool cdp_enabled;
+ } xc_cat_info;
+
+ struct {
+ uint32_t cos_max;
+ uint32_t thrtl_max;
+ bool linear;
+ } xc_mba_info;
+ } u;
+};
+typedef struct xc_psr_hw_info xc_psr_hw_info;
+
int xc_psr_cmt_attach(xc_interface *xch, uint32_t domid);
int xc_psr_cmt_detach(xc_interface *xch, uint32_t domid);
int xc_psr_cmt_get_domain_rmid(xc_interface *xch, uint32_t domid,
@@ -2479,9 +2504,8 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_cat_type type, uint32_t target,
uint64_t *data);
-int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
- uint32_t *cos_max, uint32_t *cbm_len,
- bool *cdp_enabled);
+int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
+ xc_psr_feat_type type, xc_psr_hw_info *hw_info);
int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 73d05f2..ba412e4 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -323,36 +323,58 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
return rc;
}
-int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
- uint32_t *cos_max, uint32_t *cbm_len, bool *cdp_enabled)
+int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
+ xc_psr_feat_type type, xc_psr_hw_info *hw_info)
{
int rc = -1;
DECLARE_SYSCTL;
+ if ( !hw_info )
+ return rc;
+
sysctl.cmd = XEN_SYSCTL_psr_alloc_op;
sysctl.u.psr_alloc_op.target = socket;
- switch ( lvl )
+ switch ( type )
{
- case 2:
+ case XC_PSR_FEAT_CAT_L2:
sysctl.u.psr_alloc_op.cmd = XEN_SYSCTL_PSR_CAT_get_l2_info;
rc = xc_sysctl(xch, &sysctl);
if ( !rc )
{
- *cos_max = sysctl.u.psr_alloc_op.u.cat_info.cos_max;
- *cbm_len = sysctl.u.psr_alloc_op.u.cat_info.cbm_len;
- *cdp_enabled = false;
+ hw_info->u.xc_cat_info.cos_max =
+ sysctl.u.psr_alloc_op.u.cat_info.cos_max;
+ hw_info->u.xc_cat_info.cbm_len =
+ sysctl.u.psr_alloc_op.u.cat_info.cbm_len;
+ hw_info->u.xc_cat_info.cdp_enabled = false;
}
break;
- case 3:
+ case XC_PSR_FEAT_CAT_L3:
sysctl.u.psr_alloc_op.cmd = XEN_SYSCTL_PSR_CAT_get_l3_info;
rc = xc_sysctl(xch, &sysctl);
if ( !rc )
{
- *cos_max = sysctl.u.psr_alloc_op.u.cat_info.cos_max;
- *cbm_len = sysctl.u.psr_alloc_op.u.cat_info.cbm_len;
- *cdp_enabled = sysctl.u.psr_alloc_op.u.cat_info.flags &
- XEN_SYSCTL_PSR_CAT_L3_CDP;
+ hw_info->u.xc_cat_info.cos_max =
+ sysctl.u.psr_alloc_op.u.cat_info.cos_max;
+ hw_info->u.xc_cat_info.cbm_len =
+ sysctl.u.psr_alloc_op.u.cat_info.cbm_len;
+ hw_info->u.xc_cat_info.cdp_enabled =
+ sysctl.u.psr_alloc_op.u.cat_info.flags &
+ XEN_SYSCTL_PSR_CAT_L3_CDP;
+ }
+ break;
+ case XC_PSR_FEAT_MBA:
+ sysctl.u.psr_alloc_op.cmd = XEN_SYSCTL_PSR_MBA_get_info;
+ rc = xc_sysctl(xch, &sysctl);
+ if ( !rc )
+ {
+ hw_info->u.xc_mba_info.cos_max =
+ sysctl.u.psr_alloc_op.u.mba_info.cos_max;
+ hw_info->u.xc_mba_info.thrtl_max =
+ sysctl.u.psr_alloc_op.u.mba_info.thrtl_max;
+ hw_info->u.xc_mba_info.linear =
+ sysctl.u.psr_alloc_op.u.mba_info.flags &
+ XEN_SYSCTL_PSR_MBA_LINEAR;
}
break;
default:
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index cf368ba..b183305 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -361,6 +361,27 @@ int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
return rc;
}
+static inline xc_psr_feat_type libxl__psr_feat_type_to_libxc_psr_feat_type(
+ libxl_psr_feat_type type, unsigned int lvl)
+{
+ xc_psr_feat_type xc_type = XC_PSR_FEAT_UNKNOWN;
+
+ switch (type) {
+ case LIBXL_PSR_FEAT_TYPE_CAT:
+ if (lvl == 3)
+ xc_type = XC_PSR_FEAT_CAT_L3;
+ if (lvl == 2)
+ xc_type = XC_PSR_FEAT_CAT_L2;
+ break;
+ case LIBXL_PSR_FEAT_TYPE_MBA:
+ xc_type = XC_PSR_FEAT_MBA;
+ default:
+ break;
+ }
+
+ return xc_type;
+}
+
int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
int *nr, unsigned int lvl)
{
@@ -369,6 +390,8 @@ int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
int i = 0, socketid, nr_sockets;
libxl_bitmap socketmap;
libxl_psr_cat_info *ptr;
+ xc_psr_hw_info hw_info;
+ xc_psr_feat_type xc_type;
libxl_bitmap_init(&socketmap);
@@ -385,16 +408,23 @@ int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
goto out;
}
+ xc_type = libxl__psr_feat_type_to_libxc_psr_feat_type(
+ LIBXL_PSR_FEAT_TYPE_CAT, lvl);
+
ptr = libxl__malloc(NOGC, nr_sockets * sizeof(libxl_psr_cat_info));
libxl_for_each_set_bit(socketid, socketmap) {
ptr[i].id = socketid;
- if (xc_psr_cat_get_info(ctx->xch, socketid, lvl, &ptr[i].cos_max,
- &ptr[i].cbm_len, &ptr[i].cdp_enabled)) {
+ if (xc_psr_get_hw_info(ctx->xch, socketid, xc_type, &hw_info)) {
rc = ERROR_FAIL;
free(ptr);
goto out;
}
+
+ ptr[i].cos_max = hw_info.u.xc_cat_info.cos_max;
+ ptr[i].cbm_len = hw_info.u.xc_cat_info.cbm_len;
+ ptr[i].cdp_enabled = hw_info.u.xc_cat_info.cdp_enabled;
+
i++;
}
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 10/15] tools: implement the new libxl get hw info interface
2017-08-24 1:14 [PATCH v2 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (8 preceding siblings ...)
2017-08-24 1:14 ` [PATCH v2 09/15] tools: implement the new libxc get hw info interface Yi Sun
@ 2017-08-24 1:14 ` Yi Sun
2017-08-30 9:15 ` Roger Pau Monné
2017-08-24 1:14 ` [PATCH v2 11/15] tools: implement the new xl " Yi Sun
` (4 subsequent siblings)
14 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-24 1:14 UTC (permalink / raw)
To: xen-devel
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, Yi Sun, julien.grall, mengxu, jbeulich, chao.p.peng,
roger.pau
This patch implements the new libxl get hw info interface,
'libxl_psr_get_hw_info', which is suitable to all psr allocation
features. It also implements corresponding list free function,
'libxl_psr_hw_info_list_free' and make 'libxl_psr_cat_get_info' to call
'libxl_psr_get_hw_info' to avoid redundant codes in libxl_psr.c.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
v2:
- split this patch out from a big patch in v1.
(suggested by Wei Liu)
- change 'CAT_INFO'/'MBA_INFO' to 'CAT' and 'MBA. Also the libxl structure
name 'cat_info'/'mba_info' is changed to 'cat'/'mba'.
(suggested by Chao Peng)
- call 'libxl_psr_hw_info_list_free' in 'libxl_psr_cat_get_info' to free
allocated resources.
(suggested by Chao Peng)
---
tools/libxl/libxl_psr.c | 145 +++++++++++++++++++++++++++++++++++++-----------
1 file changed, 112 insertions(+), 33 deletions(-)
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index b183305..d7da7d7 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -382,56 +382,51 @@ static inline xc_psr_feat_type libxl__psr_feat_type_to_libxc_psr_feat_type(
return xc_type;
}
+static inline int libxl__psr_hw_info_to_libxl_psr_cat_info(
+ libxl_psr_feat_type type, libxl_psr_hw_info *hw_info,
+ libxl_psr_cat_info *cat_info)
+{
+ if (type != LIBXL_PSR_FEAT_TYPE_CAT)
+ return ERROR_INVAL;
+
+ cat_info->id = hw_info->id;
+ cat_info->cos_max = hw_info->u.cat.cos_max;
+ cat_info->cbm_len = hw_info->u.cat.cbm_len;
+ cat_info->cdp_enabled = hw_info->u.cat.cdp_enabled;
+
+ return 0;
+}
+
int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
int *nr, unsigned int lvl)
{
GC_INIT(ctx);
int rc;
- int i = 0, socketid, nr_sockets;
- libxl_bitmap socketmap;
+ unsigned int i;
+ libxl_psr_hw_info *hw_info;
libxl_psr_cat_info *ptr;
- xc_psr_hw_info hw_info;
- xc_psr_feat_type xc_type;
-
- libxl_bitmap_init(&socketmap);
-
- rc = libxl__count_physical_sockets(gc, &nr_sockets);
- if (rc) {
- LOGE(ERROR, "failed to get system socket count");
- goto out;
- }
- libxl_socket_bitmap_alloc(ctx, &socketmap, nr_sockets);
- rc = libxl_get_online_socketmap(ctx, &socketmap);
- if (rc < 0) {
- LOGE(ERROR, "failed to get available sockets");
+ rc = libxl_psr_get_hw_info(ctx, &hw_info, (unsigned int *)nr,
+ LIBXL_PSR_FEAT_TYPE_CAT, lvl);
+ if (rc)
goto out;
- }
- xc_type = libxl__psr_feat_type_to_libxc_psr_feat_type(
- LIBXL_PSR_FEAT_TYPE_CAT, lvl);
+ ptr = libxl__malloc(NOGC, *nr * sizeof(libxl_psr_cat_info));
- ptr = libxl__malloc(NOGC, nr_sockets * sizeof(libxl_psr_cat_info));
-
- libxl_for_each_set_bit(socketid, socketmap) {
- ptr[i].id = socketid;
- if (xc_psr_get_hw_info(ctx->xch, socketid, xc_type, &hw_info)) {
+ for (i = 0; i < *nr; i++) {
+ if (libxl__psr_hw_info_to_libxl_psr_cat_info(
+ LIBXL_PSR_FEAT_TYPE_CAT,
+ &hw_info[i], &ptr[i])) {
+ libxl_psr_hw_info_list_free(hw_info, (unsigned int)*nr);
rc = ERROR_FAIL;
free(ptr);
goto out;
}
-
- ptr[i].cos_max = hw_info.u.xc_cat_info.cos_max;
- ptr[i].cbm_len = hw_info.u.xc_cat_info.cbm_len;
- ptr[i].cdp_enabled = hw_info.u.xc_cat_info.cdp_enabled;
-
- i++;
}
*info = ptr;
- *nr = i;
+ libxl_psr_hw_info_list_free(hw_info, (unsigned int)*nr);
out:
- libxl_bitmap_dispose(&socketmap);
GC_FREE;
return rc;
}
@@ -469,15 +464,99 @@ int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
return ERROR_FAIL;
}
+static inline int libxc__psr_hw_info_to_libxl_psr_hw_info(
+ libxl_psr_feat_type type, xc_psr_hw_info *xc_hw_info,
+ libxl_psr_hw_info *xl_hw_info)
+{
+ switch (type) {
+ case LIBXL_PSR_FEAT_TYPE_CAT:
+ xl_hw_info->u.cat.cos_max = xc_hw_info->u.xc_cat_info.cos_max;
+ xl_hw_info->u.cat.cbm_len = xc_hw_info->u.xc_cat_info.cbm_len;
+ xl_hw_info->u.cat.cdp_enabled =
+ xc_hw_info->u.xc_cat_info.cdp_enabled;
+ break;
+ case LIBXL_PSR_FEAT_TYPE_MBA:
+ xl_hw_info->u.mba.cos_max = xc_hw_info->u.xc_mba_info.cos_max;
+ xl_hw_info->u.mba.thrtl_max = xc_hw_info->u.xc_mba_info.thrtl_max;
+ xl_hw_info->u.mba.linear = xc_hw_info->u.xc_mba_info.linear;
+ break;
+ default:
+ return ERROR_INVAL;
+ }
+
+ return 0;
+}
+
int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_hw_info **info,
unsigned int *nr, libxl_psr_feat_type type,
unsigned int lvl)
{
- return ERROR_FAIL;
+ GC_INIT(ctx);
+ int rc, nr_sockets;
+ unsigned int i = 0, socketid;
+ libxl_bitmap socketmap;
+ libxl_psr_hw_info *ptr;
+ xc_psr_feat_type xc_type;
+ xc_psr_hw_info hw_info;
+
+ libxl_bitmap_init(&socketmap);
+
+ if (type == LIBXL_PSR_FEAT_TYPE_CAT && lvl != 3 && lvl != 2) {
+ LOGE(ERROR, "input lvl %d is wrong!\n", lvl);
+ rc = ERROR_FAIL;
+ goto out;
+ }
+
+ xc_type = libxl__psr_feat_type_to_libxc_psr_feat_type(type, lvl);
+
+ rc = libxl__count_physical_sockets(gc, &nr_sockets);
+ if (rc) {
+ LOGE(ERROR, "failed to get system socket count");
+ goto out;
+ }
+
+ libxl_socket_bitmap_alloc(ctx, &socketmap, nr_sockets);
+ rc = libxl_get_online_socketmap(ctx, &socketmap);
+ if (rc < 0) {
+ LOGE(ERROR, "failed to get available sockets");
+ goto out;
+ }
+
+ ptr = libxl__malloc(NOGC, nr_sockets * sizeof(libxl_psr_hw_info));
+
+ libxl_for_each_set_bit(socketid, socketmap) {
+ ptr[i].id = socketid;
+ if (xc_psr_get_hw_info(ctx->xch, socketid, xc_type, &hw_info)) {
+ rc = ERROR_FAIL;
+ free(ptr);
+ goto out;
+ }
+
+ if (libxc__psr_hw_info_to_libxl_psr_hw_info(type, &hw_info, &ptr[i])) {
+ LOGE(ERROR, "Input type %d is wrong!\n", type);
+ rc = ERROR_FAIL;
+ free(ptr);
+ goto out;
+ }
+
+ i++;
+ }
+
+ *info = ptr;
+ *nr = i;
+out:
+ libxl_bitmap_dispose(&socketmap);
+ GC_FREE;
+ return rc;
}
void libxl_psr_hw_info_list_free(libxl_psr_hw_info *list, unsigned int nr)
{
+ unsigned int i;
+
+ for (i = 0; i < nr; i++)
+ libxl_psr_hw_info_dispose(&list[i]);
+ free(list);
}
/*
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 11/15] tools: implement the new xl get hw info interface
2017-08-24 1:14 [PATCH v2 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (9 preceding siblings ...)
2017-08-24 1:14 ` [PATCH v2 10/15] tools: implement the new libxl " Yi Sun
@ 2017-08-24 1:14 ` Yi Sun
2017-08-30 9:23 ` Roger Pau Monné
2017-08-24 1:14 ` [PATCH v2 12/15] tools: rename 'xc_psr_cat_type' to 'xc_psr_val_type' Yi Sun
` (3 subsequent siblings)
14 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-24 1:14 UTC (permalink / raw)
To: xen-devel
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, Yi Sun, julien.grall, mengxu, jbeulich, chao.p.peng,
roger.pau
This patch implements a new xl get HW info interface. A new argument
is added for psr-hwinfo command to get and show MBA HW info.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
v2:
- split out this patch from a big patch in v1.
(suggested by Wei Liu)
- change 'MBA_INFO' to 'MBA'. Also, change 'mba_info' to 'mba'.
(suggested by Chao Peng)
---
tools/xl/xl_cmdtable.c | 1 +
tools/xl/xl_psr.c | 39 +++++++++++++++++++++++++++++++++++++--
2 files changed, 38 insertions(+), 2 deletions(-)
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 2c71a9f..5ac8a7e 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -524,6 +524,7 @@ struct cmd_spec cmd_table[] = {
"[options]",
"-m, --cmt Show Cache Monitoring Technology (CMT) hardware info\n"
"-a, --cat Show Cache Allocation Technology (CAT) hardware info\n"
+ "-b, --mba Show Memory Bandwidth Allocation (MBA) hardware info\n"
},
{ "psr-cmt-attach",
&main_psr_cmt_attach, 0, 1,
diff --git a/tools/xl/xl_psr.c b/tools/xl/xl_psr.c
index 7309d4f..ea37967 100644
--- a/tools/xl/xl_psr.c
+++ b/tools/xl/xl_psr.c
@@ -479,6 +479,33 @@ static int psr_l2_cat_hwinfo(void)
return rc;
}
+static int psr_mba_hwinfo(void)
+{
+ int rc;
+ unsigned int i, nr;
+ libxl_psr_hw_info *info;
+
+ rc = libxl_psr_get_hw_info(ctx, &info, &nr,
+ LIBXL_PSR_FEAT_TYPE_MBA, 0);
+ if (rc)
+ return rc;
+
+ printf("Memory Bandwidth Allocation (MBA):\n");
+
+ for (i = 0; i < nr; i++) {
+ printf("%-16s: %u\n", "Socket ID", info[i].id);
+ printf("%-16s: %s\n", "Linear Mode",
+ info[i].u.mba.linear ? "Enabled" : "Disabled");
+ printf("%-16s: %u\n", "Maximum COS", info[i].u.mba.cos_max);
+ printf("%-16s: %u\n", "Maximum Throttling Value",
+ info[i].u.mba.thrtl_max);
+ printf("%-16s: %u\n", "Default Throttling Value", 0);
+ }
+
+ libxl_psr_hw_info_list_free(info, nr);
+ return rc;
+}
+
int main_psr_cat_cbm_set(int argc, char **argv)
{
uint32_t domid;
@@ -597,20 +624,24 @@ int main_psr_cat_show(int argc, char **argv)
int main_psr_hwinfo(int argc, char **argv)
{
int opt, ret = 0;
- bool all = true, cmt = false, cat = false;
+ bool all = true, cmt = false, cat = false, mba = false;
static struct option opts[] = {
{"cmt", 0, 0, 'm'},
{"cat", 0, 0, 'a'},
+ {"mba", 0, 0, 'b'},
COMMON_LONG_OPTS
};
- SWITCH_FOREACH_OPT(opt, "ma", opts, "psr-hwinfo", 0) {
+ SWITCH_FOREACH_OPT(opt, "mab", opts, "psr-hwinfo", 0) {
case 'm':
all = false; cmt = true;
break;
case 'a':
all = false; cat = true;
break;
+ case 'b':
+ all = false; mba = true;
+ break;
}
if (!ret && (all || cmt))
@@ -623,6 +654,10 @@ int main_psr_hwinfo(int argc, char **argv)
if (all || cat)
ret = psr_l2_cat_hwinfo();
+ /* MBA is independent of CMT and CAT */
+ if (all || mba)
+ ret = psr_mba_hwinfo();
+
return ret;
}
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 12/15] tools: rename 'xc_psr_cat_type' to 'xc_psr_val_type'
2017-08-24 1:14 [PATCH v2 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (10 preceding siblings ...)
2017-08-24 1:14 ` [PATCH v2 11/15] tools: implement the new xl " Yi Sun
@ 2017-08-24 1:14 ` Yi Sun
2017-08-30 9:24 ` Roger Pau Monné
2017-08-24 1:14 ` [PATCH v2 13/15] tools: implement new generic get value interface and MBA get value command Yi Sun
` (2 subsequent siblings)
14 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-24 1:14 UTC (permalink / raw)
To: xen-devel
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, Yi Sun, julien.grall, mengxu, jbeulich, chao.p.peng,
roger.pau
This patch renames 'xc_psr_cat_type' to 'xc_psr_val_type' so that
the structure name is common for all allocation features.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Chao Peng <chao.p.peng@linux.intel.com>
---
tools/libxc/include/xenctrl.h | 8 ++++----
tools/libxc/xc_psr.c | 4 ++--
tools/libxl/libxl_psr.c | 12 ++++++------
3 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 81a6f48..f1c629e 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2450,13 +2450,13 @@ enum xc_psr_cmt_type {
};
typedef enum xc_psr_cmt_type xc_psr_cmt_type;
-enum xc_psr_cat_type {
+enum xc_psr_val_type {
XC_PSR_CAT_L3_CBM = 1,
XC_PSR_CAT_L3_CBM_CODE = 2,
XC_PSR_CAT_L3_CBM_DATA = 3,
XC_PSR_CAT_L2_CBM = 4,
};
-typedef enum xc_psr_cat_type xc_psr_cat_type;
+typedef enum xc_psr_val_type xc_psr_val_type;
enum xc_psr_feat_type {
XC_PSR_FEAT_UNKNOWN,
@@ -2499,10 +2499,10 @@ int xc_psr_cmt_get_data(xc_interface *xch, uint32_t rmid, uint32_t cpu,
int xc_psr_cmt_enabled(xc_interface *xch);
int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_val_type type, uint32_t target,
uint64_t data);
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_val_type type, uint32_t target,
uint64_t *data);
int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
xc_psr_feat_type type, xc_psr_hw_info *hw_info);
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index ba412e4..c69e943 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -249,7 +249,7 @@ int xc_psr_cmt_enabled(xc_interface *xch)
return 0;
}
int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_val_type type, uint32_t target,
uint64_t data)
{
DECLARE_DOMCTL;
@@ -284,7 +284,7 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
}
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_val_type type, uint32_t target,
uint64_t *data)
{
int rc;
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index d7da7d7..c97dca4 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -303,11 +303,11 @@ out:
return rc;
}
-static inline xc_psr_cat_type libxl__psr_cbm_type_to_libxc_psr_cat_type(
+static inline xc_psr_val_type libxl__psr_cbm_type_to_libxc_psr_val_type(
libxl_psr_cbm_type type)
{
- BUILD_BUG_ON(sizeof(libxl_psr_cbm_type) != sizeof(xc_psr_cat_type));
- return (xc_psr_cat_type)type;
+ BUILD_BUG_ON(sizeof(libxl_psr_cbm_type) != sizeof(xc_psr_val_type));
+ return (xc_psr_val_type)type;
}
int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
@@ -325,12 +325,12 @@ int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
}
libxl_for_each_set_bit(socketid, *target_map) {
- xc_psr_cat_type xc_type;
+ xc_psr_val_type xc_type;
if (socketid >= nr_sockets)
break;
- xc_type = libxl__psr_cbm_type_to_libxc_psr_cat_type(type);
+ xc_type = libxl__psr_cbm_type_to_libxc_psr_val_type(type);
if (xc_psr_cat_set_domain_data(ctx->xch, domid, xc_type,
socketid, cbm)) {
libxl__psr_cat_log_err_msg(gc, errno);
@@ -349,7 +349,7 @@ int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
{
GC_INIT(ctx);
int rc = 0;
- xc_psr_cat_type xc_type = libxl__psr_cbm_type_to_libxc_psr_cat_type(type);
+ xc_psr_val_type xc_type = libxl__psr_cbm_type_to_libxc_psr_val_type(type);
if (xc_psr_cat_get_domain_data(ctx->xch, domid, xc_type,
target, cbm_r)) {
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 13/15] tools: implement new generic get value interface and MBA get value command
2017-08-24 1:14 [PATCH v2 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (11 preceding siblings ...)
2017-08-24 1:14 ` [PATCH v2 12/15] tools: rename 'xc_psr_cat_type' to 'xc_psr_val_type' Yi Sun
@ 2017-08-24 1:14 ` Yi Sun
2017-08-24 1:14 ` [PATCH v2 14/15] tools: implement new generic set value interface and MBA set " Yi Sun
2017-08-24 1:14 ` [PATCH v2 15/15] docs: add MBA description in docs Yi Sun
14 siblings, 0 replies; 64+ messages in thread
From: Yi Sun @ 2017-08-24 1:14 UTC (permalink / raw)
To: xen-devel
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, Yi Sun, julien.grall, mengxu, jbeulich, chao.p.peng,
roger.pau
This patch implements generic get value interfaces in libxc and libxl.
It also refactors the get value flow in xl to make it be suitable for all
allocation features. Based on that, a new MBA get value command is added in xl.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
---
v2:
- change 'CAT_INFO'/'MBA_INFO' to 'CAT'/'MBA'. The related structure names
are changed too.
(suggested by Chao Peng)
---
tools/libxc/include/xenctrl.h | 7 +-
tools/libxc/xc_psr.c | 9 +-
tools/libxl/libxl_psr.c | 51 ++++++++----
tools/xl/xl.h | 1 +
tools/xl/xl_cmdtable.c | 5 ++
tools/xl/xl_psr.c | 185 +++++++++++++++++++++++++++++-------------
6 files changed, 179 insertions(+), 79 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index f1c629e..d7eabfb 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2455,6 +2455,7 @@ enum xc_psr_val_type {
XC_PSR_CAT_L3_CBM_CODE = 2,
XC_PSR_CAT_L3_CBM_DATA = 3,
XC_PSR_CAT_L2_CBM = 4,
+ XC_PSR_MBA_THRTL = 5,
};
typedef enum xc_psr_val_type xc_psr_val_type;
@@ -2501,9 +2502,9 @@ int xc_psr_cmt_enabled(xc_interface *xch);
int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_val_type type, uint32_t target,
uint64_t data);
-int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_val_type type, uint32_t target,
- uint64_t *data);
+int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_val_type type, uint32_t target,
+ uint64_t *data);
int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
xc_psr_feat_type type, xc_psr_hw_info *hw_info);
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index c69e943..de501b0 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -283,9 +283,9 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
return do_domctl(xch, &domctl);
}
-int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_val_type type, uint32_t target,
- uint64_t *data)
+int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_val_type type, uint32_t target,
+ uint64_t *data)
{
int rc;
DECLARE_DOMCTL;
@@ -305,6 +305,9 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
case XC_PSR_CAT_L2_CBM:
cmd = XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM;
break;
+ case XC_PSR_MBA_THRTL:
+ cmd = XEN_DOMCTL_PSR_MBA_OP_GET_THRTL;
+ break;
default:
errno = EINVAL;
return -1;
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index c97dca4..1f2eee2 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -71,16 +71,30 @@ static void libxl__psr_cmt_log_err_msg(libxl__gc *gc, int err)
LOGE(ERROR, "%s", msg);
}
-static void libxl__psr_cat_log_err_msg(libxl__gc *gc, int err)
+static void libxl__psr_alloc_log_err_msg(libxl__gc *gc,
+ int err,
+ libxl_psr_cbm_type type)
{
+ /*
+ * Index is 'libxl_psr_cbm_type' so we set two 'CDP' to correspond to
+ * DATA and CODE.
+ */
+ const char * const feat_name[6] = {
+ "UNKNOWN",
+ "L3 CAT",
+ "CDP",
+ "CDP",
+ "L2 CAT",
+ "MBA",
+ };
char *msg;
switch (err) {
case ENODEV:
- msg = "CAT is not supported in this system";
+ msg = "is not supported in this system";
break;
case ENOENT:
- msg = "CAT is not enabled on the socket";
+ msg = "is not enabled on the socket";
break;
case EOVERFLOW:
msg = "no free COS available";
@@ -106,7 +120,7 @@ static void libxl__psr_cat_log_err_msg(libxl__gc *gc, int err)
return;
}
- LOGE(ERROR, "%s", msg);
+ LOGE(ERROR, "%s: %s", feat_name[type], msg);
}
static int libxl__pick_socket_cpu(libxl__gc *gc, uint32_t socketid)
@@ -333,7 +347,7 @@ int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
xc_type = libxl__psr_cbm_type_to_libxc_psr_val_type(type);
if (xc_psr_cat_set_domain_data(ctx->xch, domid, xc_type,
socketid, cbm)) {
- libxl__psr_cat_log_err_msg(gc, errno);
+ libxl__psr_alloc_log_err_msg(gc, errno, type);
rc = ERROR_FAIL;
}
}
@@ -347,18 +361,7 @@ int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
libxl_psr_cbm_type type, uint32_t target,
uint64_t *cbm_r)
{
- GC_INIT(ctx);
- int rc = 0;
- xc_psr_val_type xc_type = libxl__psr_cbm_type_to_libxc_psr_val_type(type);
-
- if (xc_psr_cat_get_domain_data(ctx->xch, domid, xc_type,
- target, cbm_r)) {
- libxl__psr_cat_log_err_msg(gc, errno);
- rc = ERROR_FAIL;
- }
-
- GC_FREE;
- return rc;
+ return libxl_psr_get_val(ctx, domid, type, target, cbm_r);
}
static inline xc_psr_feat_type libxl__psr_feat_type_to_libxc_psr_feat_type(
@@ -461,7 +464,19 @@ int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
libxl_psr_cbm_type type, unsigned int target,
uint64_t *val)
{
- return ERROR_FAIL;
+ GC_INIT(ctx);
+ int rc = 0;
+
+ xc_psr_val_type xc_type = libxl__psr_cbm_type_to_libxc_psr_val_type(type);
+
+ if (xc_psr_get_domain_data(ctx->xch, domid, xc_type,
+ target, val)) {
+ libxl__psr_alloc_log_err_msg(gc, errno, type);
+ rc = ERROR_FAIL;
+ }
+
+ GC_FREE;
+ return rc;
}
static inline int libxc__psr_hw_info_to_libxl_psr_hw_info(
diff --git a/tools/xl/xl.h b/tools/xl/xl.h
index aa95b77..59065c4 100644
--- a/tools/xl/xl.h
+++ b/tools/xl/xl.h
@@ -206,6 +206,7 @@ int main_psr_cmt_show(int argc, char **argv);
#ifdef LIBXL_HAVE_PSR_CAT
int main_psr_cat_cbm_set(int argc, char **argv);
int main_psr_cat_show(int argc, char **argv);
+int main_psr_mba_show(int argc, char **argv);
#endif
int main_qemu_monitor_command(int argc, char **argv);
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 5ac8a7e..19e5004 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -562,6 +562,11 @@ struct cmd_spec cmd_table[] = {
"[options] <Domain>",
"-l <level> Specify the cache level to process, otherwise L3 cache is processed\n"
},
+ { "psr-mba-show",
+ &main_psr_mba_show, 0, 1,
+ "Show Memory Bandwidth Allocation information",
+ "<Domain>",
+ },
#endif
{ "usbctrl-attach",
diff --git a/tools/xl/xl_psr.c b/tools/xl/xl_psr.c
index ea37967..a97dafa 100644
--- a/tools/xl/xl_psr.c
+++ b/tools/xl/xl_psr.c
@@ -330,19 +330,27 @@ out:
return rc;
}
-static void psr_cat_print_one_domain_cbm_type(uint32_t domid, uint32_t socketid,
- libxl_psr_cbm_type type)
+static void psr_print_one_domain_val_type(uint32_t domid,
+ libxl_psr_hw_info *info,
+ libxl_psr_cbm_type type)
{
- uint64_t cbm;
+ uint64_t val;
- if (!libxl_psr_cat_get_cbm(ctx, domid, type, socketid, &cbm))
- printf("%#16"PRIx64, cbm);
+ if (!libxl_psr_get_val(ctx, domid, type, info->id, &val))
+ {
+ if (type == LIBXL_PSR_CBM_TYPE_MBA_THRTL && info->u.mba.linear)
+ printf("%16"PRIu64, val);
+ else
+ printf("%#16"PRIx64, val);
+ }
else
printf("%16s", "error");
}
-static void psr_cat_print_one_domain_cbm(uint32_t domid, uint32_t socketid,
- bool cdp_enabled, unsigned int lvl)
+static void psr_print_one_domain_val(uint32_t domid,
+ libxl_psr_hw_info *info,
+ libxl_psr_feat_type type,
+ unsigned int lvl)
{
char *domain_name;
@@ -350,106 +358,152 @@ static void psr_cat_print_one_domain_cbm(uint32_t domid, uint32_t socketid,
printf("%5d%25s", domid, domain_name);
free(domain_name);
- switch (lvl) {
- case 3:
- if (!cdp_enabled) {
- psr_cat_print_one_domain_cbm_type(domid, socketid,
+ switch (type) {
+ case LIBXL_PSR_FEAT_TYPE_CAT:
+ switch (lvl) {
+ case 3:
+ if (!info->u.cat.cdp_enabled) {
+ psr_print_one_domain_val_type(domid, info,
LIBXL_PSR_CBM_TYPE_L3_CBM);
- } else {
- psr_cat_print_one_domain_cbm_type(domid, socketid,
+ } else {
+ psr_print_one_domain_val_type(domid, info,
LIBXL_PSR_CBM_TYPE_L3_CBM_CODE);
- psr_cat_print_one_domain_cbm_type(domid, socketid,
+ psr_print_one_domain_val_type(domid, info,
LIBXL_PSR_CBM_TYPE_L3_CBM_DATA);
- }
- break;
- case 2:
- psr_cat_print_one_domain_cbm_type(domid, socketid,
+ }
+ break;
+ case 2:
+ psr_print_one_domain_val_type(domid, info,
LIBXL_PSR_CBM_TYPE_L2_CBM);
+ break;
+ default:
+ printf("Input lvl %d is wrong!", lvl);
+ }
break;
- default:
- printf("Input lvl %d is wrong!", lvl);
+
+ case LIBXL_PSR_FEAT_TYPE_MBA:
+ psr_print_one_domain_val_type(domid, info,
+ LIBXL_PSR_CBM_TYPE_MBA_THRTL);
break;
}
printf("\n");
}
-static int psr_cat_print_domain_cbm(uint32_t domid, uint32_t socketid,
- bool cdp_enabled, unsigned int lvl)
+static int psr_print_domain_val(uint32_t domid,
+ libxl_psr_hw_info *info,
+ libxl_psr_feat_type type,
+ unsigned int lvl)
{
int i, nr_domains;
libxl_dominfo *list;
if (domid != INVALID_DOMID) {
- psr_cat_print_one_domain_cbm(domid, socketid, cdp_enabled, lvl);
+ psr_print_one_domain_val(domid, info, type, lvl);
return 0;
}
if (!(list = libxl_list_domain(ctx, &nr_domains))) {
- fprintf(stderr, "Failed to get domain list for cbm display\n");
- return -1;
+ fprintf(stderr, "Failed to get domain list for value display\n");
+ return EXIT_FAILURE;
}
for (i = 0; i < nr_domains; i++)
- psr_cat_print_one_domain_cbm(list[i].domid, socketid, cdp_enabled, lvl);
+ psr_print_one_domain_val(list[i].domid, info, type, lvl);
libxl_dominfo_list_free(list, nr_domains);
return 0;
}
-static int psr_cat_print_socket(uint32_t domid, libxl_psr_cat_info *info,
- unsigned int lvl)
+static int psr_print_socket(uint32_t domid,
+ libxl_psr_hw_info *info,
+ libxl_psr_feat_type type,
+ unsigned int lvl)
{
- int rc;
- uint32_t l3_cache_size;
-
printf("%-16s: %u\n", "Socket ID", info->id);
- /* So far, CMT only supports L3 cache. */
- if (lvl == 3) {
- rc = libxl_psr_cmt_get_l3_cache_size(ctx, info->id, &l3_cache_size);
- if (rc) {
- fprintf(stderr, "Failed to get l3 cache size for socket:%d\n",
- info->id);
- return -1;
+ switch (type) {
+ case LIBXL_PSR_FEAT_TYPE_CAT:
+ {
+ int rc;
+ uint32_t l3_cache_size;
+
+ /* So far, CMT only supports L3 cache. */
+ if (lvl == 3) {
+ rc = libxl_psr_cmt_get_l3_cache_size(ctx, info->id, &l3_cache_size);
+ if (rc) {
+ fprintf(stderr, "Failed to get l3 cache size for socket:%d\n",
+ info->id);
+ return -1;
+ }
+ printf("%-16s: %uKB\n", "L3 Cache", l3_cache_size);
}
- printf("%-16s: %uKB\n", "L3 Cache", l3_cache_size);
+
+ printf("%-16s: %#llx\n", "Default CBM",
+ (1ull << info->u.cat.cbm_len) - 1);
+ if (info->u.cat.cdp_enabled)
+ printf("%5s%25s%16s%16s\n", "ID", "NAME", "CBM (code)", "CBM (data)");
+ else
+ printf("%5s%25s%16s\n", "ID", "NAME", "CBM");
+
+ break;
}
- printf("%-16s: %#llx\n", "Default CBM", (1ull << info->cbm_len) - 1);
- if (info->cdp_enabled)
- printf("%5s%25s%16s%16s\n", "ID", "NAME", "CBM (code)", "CBM (data)");
- else
- printf("%5s%25s%16s\n", "ID", "NAME", "CBM");
+ case LIBXL_PSR_FEAT_TYPE_MBA:
+ printf("%-16s: %u\n", "Default THRTL", 0);
+ printf("%5s%25s%16s\n", "ID", "NAME", "THRTL");
+ break;
- return psr_cat_print_domain_cbm(domid, info->id, info->cdp_enabled, lvl);
+ default:
+ fprintf(stderr, "Input feature type %d is wrong\n", type);
+ return EXIT_FAILURE;
+ }
+
+ return psr_print_domain_val(domid, info, type, lvl);
}
-static int psr_cat_show(uint32_t domid, unsigned int lvl)
+static int psr_val_show(uint32_t domid,
+ libxl_psr_feat_type type,
+ unsigned int lvl)
{
- int i, nr;
+ unsigned int i, nr;
int rc;
- libxl_psr_cat_info *info;
+ libxl_psr_hw_info *info;
- if (lvl != 2 && lvl != 3) {
- fprintf(stderr, "Input lvl %d is wrong\n", lvl);
+ switch (type) {
+ case LIBXL_PSR_FEAT_TYPE_CAT:
+ if (lvl != 2 && lvl != 3) {
+ fprintf(stderr, "Input lvl %d is wrong\n", lvl);
+ return EXIT_FAILURE;
+ }
+ break;
+
+ case LIBXL_PSR_FEAT_TYPE_MBA:
+ if (lvl) {
+ fprintf(stderr, "Input lvl %d is wrong\n", lvl);
+ return EXIT_FAILURE;
+ }
+ break;
+
+ default:
+ fprintf(stderr, "Input feature type %d is wrong\n", type);
return EXIT_FAILURE;
}
- rc = libxl_psr_cat_get_info(ctx, &info, &nr, lvl);
+ rc = libxl_psr_get_hw_info(ctx, &info, &nr, type, lvl);
if (rc) {
- fprintf(stderr, "Failed to get %s cat info\n", (lvl == 3)?"L3":"L2");
+ fprintf(stderr, "Failed to get info\n");
return rc;
}
for (i = 0; i < nr; i++) {
- rc = psr_cat_print_socket(domid, info + i, lvl);
+ rc = psr_print_socket(domid, info + i, type, lvl);
if (rc)
goto out;
}
out:
- libxl_psr_cat_info_list_free(info, nr);
+ libxl_psr_hw_info_list_free(info, nr);
return rc;
}
@@ -479,6 +533,27 @@ static int psr_l2_cat_hwinfo(void)
return rc;
}
+int main_psr_mba_show(int argc, char **argv)
+{
+ int opt;
+ uint32_t domid;
+
+ SWITCH_FOREACH_OPT(opt, "", NULL, "psr-mba-show", 0) {
+ /* No options */
+ }
+
+ if (optind >= argc)
+ domid = INVALID_DOMID;
+ else if (optind == argc - 1)
+ domid = find_domain(argv[optind]);
+ else {
+ help("psr-mba-show");
+ return 2;
+ }
+
+ return psr_val_show(domid, LIBXL_PSR_FEAT_TYPE_MBA, 0);
+}
+
static int psr_mba_hwinfo(void)
{
int rc;
@@ -618,7 +693,7 @@ int main_psr_cat_show(int argc, char **argv)
return 2;
}
- return psr_cat_show(domid, lvl);
+ return psr_val_show(domid, LIBXL_PSR_FEAT_TYPE_CAT, lvl);
}
int main_psr_hwinfo(int argc, char **argv)
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 14/15] tools: implement new generic set value interface and MBA set value command
2017-08-24 1:14 [PATCH v2 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (12 preceding siblings ...)
2017-08-24 1:14 ` [PATCH v2 13/15] tools: implement new generic get value interface and MBA get value command Yi Sun
@ 2017-08-24 1:14 ` Yi Sun
2017-08-30 9:47 ` Roger Pau Monné
2017-08-24 1:14 ` [PATCH v2 15/15] docs: add MBA description in docs Yi Sun
14 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-24 1:14 UTC (permalink / raw)
To: xen-devel
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, Yi Sun, julien.grall, mengxu, jbeulich, chao.p.peng,
roger.pau
This patch implements new generic set value interfaces in libxc and libxl.
These interfaces are suitable for all allocation features. It also adds a
new MBA set value command in xl.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
tools/libxc/include/xenctrl.h | 6 ++---
tools/libxc/xc_psr.c | 9 ++++---
tools/libxl/libxl_psr.c | 56 +++++++++++++++++++++----------------------
tools/xl/xl.h | 1 +
tools/xl/xl_cmdtable.c | 6 +++++
tools/xl/xl_psr.c | 55 ++++++++++++++++++++++++++++++++++++++++++
6 files changed, 99 insertions(+), 34 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index d7eabfb..3161bf6 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2499,9 +2499,9 @@ int xc_psr_cmt_get_data(xc_interface *xch, uint32_t rmid, uint32_t cpu,
uint64_t *tsc);
int xc_psr_cmt_enabled(xc_interface *xch);
-int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_val_type type, uint32_t target,
- uint64_t data);
+int xc_psr_set_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_val_type type, uint32_t target,
+ uint64_t data);
int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_val_type type, uint32_t target,
uint64_t *data);
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index de501b0..4dde76f 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -248,9 +248,9 @@ int xc_psr_cmt_enabled(xc_interface *xch)
return 0;
}
-int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_val_type type, uint32_t target,
- uint64_t data)
+int xc_psr_set_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_val_type type, uint32_t target,
+ uint64_t data)
{
DECLARE_DOMCTL;
uint32_t cmd;
@@ -269,6 +269,9 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
case XC_PSR_CAT_L2_CBM:
cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM;
break;
+ case XC_PSR_MBA_THRTL:
+ cmd = XEN_DOMCTL_PSR_MBA_OP_SET_THRTL;
+ break;
default:
errno = EINVAL;
return -1;
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index 1f2eee2..bd10e8b 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -328,33 +328,7 @@ int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
libxl_psr_cbm_type type, libxl_bitmap *target_map,
uint64_t cbm)
{
- GC_INIT(ctx);
- int rc;
- int socketid, nr_sockets;
-
- rc = libxl__count_physical_sockets(gc, &nr_sockets);
- if (rc) {
- LOGED(ERROR, domid, "failed to get system socket count");
- goto out;
- }
-
- libxl_for_each_set_bit(socketid, *target_map) {
- xc_psr_val_type xc_type;
-
- if (socketid >= nr_sockets)
- break;
-
- xc_type = libxl__psr_cbm_type_to_libxc_psr_val_type(type);
- if (xc_psr_cat_set_domain_data(ctx->xch, domid, xc_type,
- socketid, cbm)) {
- libxl__psr_alloc_log_err_msg(gc, errno, type);
- rc = ERROR_FAIL;
- }
- }
-
-out:
- GC_FREE;
- return rc;
+ return libxl_psr_set_val(ctx, domid, type, target_map, cbm);
}
int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
@@ -457,7 +431,33 @@ int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
libxl_psr_cbm_type type, libxl_bitmap *target_map,
uint64_t val)
{
- return ERROR_FAIL;
+ GC_INIT(ctx);
+ int rc;
+ int socketid, nr_sockets;
+
+ rc = libxl__count_physical_sockets(gc, &nr_sockets);
+ if (rc) {
+ LOGED(ERROR, domid, "failed to get system socket count");
+ goto out;
+ }
+
+ libxl_for_each_set_bit(socketid, *target_map) {
+ xc_psr_val_type xc_type;
+
+ if (socketid >= nr_sockets)
+ break;
+
+ xc_type = libxl__psr_cbm_type_to_libxc_psr_val_type(type);
+ if (xc_psr_set_domain_data(ctx->xch, domid, xc_type,
+ socketid, val)) {
+ libxl__psr_alloc_log_err_msg(gc, errno, type);
+ rc = ERROR_FAIL;
+ }
+ }
+
+out:
+ GC_FREE;
+ return rc;
}
int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
diff --git a/tools/xl/xl.h b/tools/xl/xl.h
index 59065c4..2cbe89c 100644
--- a/tools/xl/xl.h
+++ b/tools/xl/xl.h
@@ -206,6 +206,7 @@ int main_psr_cmt_show(int argc, char **argv);
#ifdef LIBXL_HAVE_PSR_CAT
int main_psr_cat_cbm_set(int argc, char **argv);
int main_psr_cat_show(int argc, char **argv);
+int main_psr_mba_set(int argc, char **argv);
int main_psr_mba_show(int argc, char **argv);
#endif
int main_qemu_monitor_command(int argc, char **argv);
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 19e5004..5f7122d 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -562,6 +562,12 @@ struct cmd_spec cmd_table[] = {
"[options] <Domain>",
"-l <level> Specify the cache level to process, otherwise L3 cache is processed\n"
},
+ { "psr-mba-set",
+ &main_psr_mba_set, 0, 1,
+ "Set throttling value (THRTL) for a domain",
+ "[options] <Domain> <THRTL>",
+ "-s <socket> Specify the socket to process, otherwise all sockets are processed\n"
+ },
{ "psr-mba-show",
&main_psr_mba_show, 0, 1,
"Show Memory Bandwidth Allocation information",
diff --git a/tools/xl/xl_psr.c b/tools/xl/xl_psr.c
index a97dafa..fc5bfcd 100644
--- a/tools/xl/xl_psr.c
+++ b/tools/xl/xl_psr.c
@@ -554,6 +554,61 @@ int main_psr_mba_show(int argc, char **argv)
return psr_val_show(domid, LIBXL_PSR_FEAT_TYPE_MBA, 0);
}
+int main_psr_mba_set(int argc, char **argv)
+{
+ uint32_t domid;
+ libxl_psr_cbm_type type;
+ uint64_t thrtl;
+ int ret, opt = 0;
+ libxl_bitmap target_map;
+ char *value;
+ libxl_string_list socket_list;
+ unsigned long start, end;
+ unsigned int i, j, len;
+
+ static struct option opts[] = {
+ {"socket", 1, 0, 's'},
+ COMMON_LONG_OPTS
+ };
+
+ libxl_socket_bitmap_alloc(ctx, &target_map, 0);
+ libxl_bitmap_set_none(&target_map);
+
+ SWITCH_FOREACH_OPT(opt, "s:", opts, "psr-mba-set", 0) {
+ case 's':
+ trim(isspace, optarg, &value);
+ split_string_into_string_list(value, ",", &socket_list);
+ len = libxl_string_list_length(&socket_list);
+ for (i = 0; i < len; i++) {
+ parse_range(socket_list[i], &start, &end);
+ for (j = start; j <= end; j++)
+ libxl_bitmap_set(&target_map, j);
+ }
+
+ libxl_string_list_dispose(&socket_list);
+ free(value);
+ break;
+ }
+
+ type = LIBXL_PSR_CBM_TYPE_MBA_THRTL;
+
+ if (libxl_bitmap_is_empty(&target_map))
+ libxl_bitmap_set_any(&target_map);
+
+ if (argc != optind + 2) {
+ help("psr-mba-set");
+ return 2;
+ }
+
+ domid = find_domain(argv[optind]);
+ thrtl = strtoll(argv[optind + 1], NULL , 0);
+
+ ret = libxl_psr_set_val(ctx, domid, type, &target_map, thrtl);
+
+ libxl_bitmap_dispose(&target_map);
+ return ret;
+}
+
static int psr_mba_hwinfo(void)
{
int rc;
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 15/15] docs: add MBA description in docs
2017-08-24 1:14 [PATCH v2 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (13 preceding siblings ...)
2017-08-24 1:14 ` [PATCH v2 14/15] tools: implement new generic set value interface and MBA set " Yi Sun
@ 2017-08-24 1:14 ` Yi Sun
14 siblings, 0 replies; 64+ messages in thread
From: Yi Sun @ 2017-08-24 1:14 UTC (permalink / raw)
To: xen-devel
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, Yi Sun, julien.grall, mengxu, jbeulich, chao.p.peng,
roger.pau
This patch adds MBA description in related documents.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
---
v2:
- state the value type shown by 'psr-mba-show'. For linear mode,
it shows decimal value. For non-linear mode, it shows hexadecimal
value.
(suggested by Chao Peng)
---
docs/man/xl.pod.1.in | 34 +++++++++++++++++++++++++
docs/misc/xl-psr.markdown | 63 +++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 97 insertions(+)
diff --git a/docs/man/xl.pod.1.in b/docs/man/xl.pod.1.in
index 16c8306..e644b19 100644
--- a/docs/man/xl.pod.1.in
+++ b/docs/man/xl.pod.1.in
@@ -1798,6 +1798,40 @@ processed.
=back
+=head2 Memory Bandwidth Allocation
+
+Intel Skylake and later server platforms offer capabilities to configure and
+make use of the Memory Bandwidth Allocation (MBA) mechanisms, which provides
+OS/VMMs the ability to slow misbehaving apps/VMs or create advanced closed-loop
+control system via exposing control over a credit-based throttling mechanism.
+In the Xen implementation, MBA is used to control memory bandwidth on VM basis.
+To enforce bandwidth on a specific domain, just set throttling value (THRTL)
+for the domain.
+
+=over 4
+
+=item B<psr-mba-set> [I<OPTIONS>] I<domain-id> I<thrtl>
+
+Set throttling value (THRTL) for a domain. For how to specify I<thrtl>
+please refer to L<http://xenbits.xen.org/docs/unstable/misc/xl-psr.html>.
+
+B<OPTIONS>
+
+=over 4
+
+=item B<-s SOCKET>, B<--socket=SOCKET>
+
+Specify the socket to process, otherwise all sockets are processed.
+
+=back
+
+=item B<psr-mba-show> [I<domain-id>]
+
+Show MBA settings for a certain domain or all domains. For linear mode, it
+shows the decimal value. For non-linear mode, it shows hexadecimal value.
+
+=back
+
=head1 IGNORED FOR COMPATIBILITY WITH XM
xl is mostly command-line compatible with the old xm utility used with
diff --git a/docs/misc/xl-psr.markdown b/docs/misc/xl-psr.markdown
index 04dd957..39fc801 100644
--- a/docs/misc/xl-psr.markdown
+++ b/docs/misc/xl-psr.markdown
@@ -186,6 +186,69 @@ Setting data CBM for a domain:
Setting the same code and data CBM for a domain:
`xl psr-cat-set <domid> <cbm>`
+## Memory Bandwidth Allocation (MBA)
+
+Memory Bandwidth Allocation (MBA) is a new feature available on Intel
+Skylake and later server platforms that allows an OS or Hypervisor/VMM to
+slow misbehaving apps/VMs or create advanced closed-loop control system via
+exposing control over a credit-based throttling mechanism. To enforce bandwidth
+on a specific domain, just set throttling value (THRTL) into Class of Service
+(COS). MBA provides two THRTL mode. One is linear mode and the other is
+non-linear mode.
+
+In the linear mode the input precision is defined as 100-(THRTL_MAX). Values
+not an even multiple of the precision (e.g., 12%) will be rounded down (e.g.,
+to 10% delay applied).
+
+If linear values are not supported then input delay values are powers-of-two
+from zero to the THRTL_MAX value from CPUID. In this case any values not a power
+of two will be rounded down the next nearest power of two.
+
+For example, assuming a system with 2 domains:
+
+ * A THRTL of 0x0 for every domain means each domain can access the whole cache
+ without any delay. This is the default.
+
+ * Linear mode: Giving one domain a THRTL of 0xC and the other domain's 0 means
+ that the first domain gets 10% delay to access the cache and the other one
+ without any delay.
+
+ * Non-linear mode: Giving one domain a THRTL of 0xC and the other domain's 0
+ means that the first domain gets 8% delay to access the cache and the other
+ one without any delay.
+
+For more detailed information please refer to Intel SDM chapter
+"Introduction to Memory Bandwidth Allocation".
+
+In Xen's implementation, THRTL can be configured with libxl/xl interfaces but
+COS is maintained in hypervisor only. The cache partition granularity is per
+domain, each domain has COS=0 assigned by default, the corresponding THRTL is
+0, which means all the cache resource can be accessed without delay.
+
+### xl interfaces
+
+System MBA information such as maximum COS and maximum THRTL can be obtained by:
+
+`xl psr-hwinfo --mba`
+
+The simplest way to change a domain's THRTL from its default is running:
+
+`xl psr-mba-set [OPTIONS] <domid> <thrtl>`
+
+In a multi-socket system, the same thrtl will be set on each socket by default.
+Per socket thrtl can be specified with the `--socket SOCKET` option.
+
+Setting the THRTL may not be successful if insufficient COS is available. In
+such case unused COS(es) may be freed by setting THRTL of all related domains to
+its default value(0).
+
+Per domain THRTL settings can be shown by:
+
+`xl psr-mba-show [OPTIONS] <domid>`
+
+For linear mode, it shows the decimal value. For non-linear mode, it shows
+hexadecimal value.
+
## Reference
[1] Intel SDM
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* Re: [PATCH v2 01/15] docs: create Memory Bandwidth Allocation (MBA) feature document
2017-08-24 1:14 ` [PATCH v2 01/15] docs: create Memory Bandwidth Allocation (MBA) feature document Yi Sun
@ 2017-08-29 11:46 ` Roger Pau Monné
2017-08-30 5:20 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2017-08-29 11:46 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 24, 2017 at 09:14:35AM +0800, Yi Sun wrote:
> This patch creates MBA feature document in doc/features/. It describes
> key points to implement MBA which is described in details in Intel SDM
> "Introduction to Memory Bandwidth Allocation".
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> ---
> v2:
> - declare 'HW' in Terminology.
> (suggested by Chao Peng)
> - replace 'COS ID of VCPU' to 'COS ID of domain'.
> (suggested by Chao Peng)
> - replace 'COS register' to 'Thrtl MSR'.
> (suggested by Chao Peng)
> - add description for 'psr-mba-show' to state that the decimal value is
> shown for linear mode but hexadecimal value is shown for non-linear mode.
> (suggested by Chao Peng)
> - remove content in 'Areas for improvement'.
> (suggested by Chao Peng)
> - use '<>' to specify mandatory argument to a command.
> (suggested by Wei Liu)
> v1:
> - remove a special character to avoid the error when building pandoc.
> ---
> docs/features/intel_psr_mba.pandoc | 256 +++++++++++++++++++++++++++++++++++++
> 1 file changed, 256 insertions(+)
> create mode 100644 docs/features/intel_psr_mba.pandoc
>
> diff --git a/docs/features/intel_psr_mba.pandoc b/docs/features/intel_psr_mba.pandoc
> new file mode 100644
> index 0000000..21592e8
> --- /dev/null
> +++ b/docs/features/intel_psr_mba.pandoc
> @@ -0,0 +1,256 @@
> +% Intel Memory Bandwidth Allocation (MBA) Feature
> +% Revision 1.4
> +
> +\clearpage
> +
> +# Basics
> +
> +---------------- ----------------------------------------------------
> + Status: **Tech Preview**
> +
> +Architecture(s): Intel x86
> +
> + Component(s): Hypervisor, toolstack
> +
> + Hardware: MBA is supported on Skylake Server and beyond
> +---------------- ----------------------------------------------------
> +
> +# Terminology
> +
> +* CAT Cache Allocation Technology
> +* CBM Capacity BitMasks
> +* CDP Code and Data Prioritization
> +* COS/CLOS Class of Service
> +* HW Hardware
> +* MBA Memory Bandwidth Allocation
> +* MSRs Machine Specific Registers
> +* PSR Intel Platform Shared Resource
> +* THRTL Throttle value or delay value
> +
> +# Overview
> +
> +The Memory Bandwidth Allocation (MBA) feature provides indirect and approximate
> +control over memory bandwidth available per-core. This feature provides OS/
> +hypervisor the ability to slow misbehaving apps/domains or create advanced
> +closed-loop control system via exposing control over a credit-based throttling
> +mechanism.
I don't really understand what "advanced closed-loop control system
via exposing..." means. From my understand it is clear/simpler to
write it as:
"... the ability to slow misbehaving apps/domains by using a
credit-based throttling mechanism".
> +
> +# User details
> +
> +* Feature Enabling:
> +
> + Add "psr=mba" to boot line parameter to enable MBA feature.
> +
> +* xl interfaces:
> +
> + 1. `psr-mba-show [domain-id]`:
> +
> + Show memory bandwidth throttling for domain. For linear mode, it shows the
> + decimal value. For non-linear mode, it shows hexadecimal value.
You should first explain what are the linear and non-linear modes.
> +
> + 2. `psr-mba-set [OPTIONS] <domain-id> <throttling>`:
> +
> + Set memory bandwidth throttling for domain.
> +
> + Options:
> + '-s': Specify the socket to process, otherwise all sockets are processed.
> +
> + Throttling value set in register implies memory bandwidth blocked, i.e.
> + higher throttling value results in lower bandwidth. The max throttling
> + value can be got through CPUID.
This is also hard to understand IMHO. I would rather write it as:
"The throttling value describes the amount of blocked bandwidth".
Although I have to admit I don't really understand this interface,
wouldn't it be easier to specify the memory bandwidth allowed
per-domain, rather the amount of bandwidth removed?
Using your approach the user has to first get the total bandwidth, and
then subtract the removed bandwidth in order to know the remaining
bandwidth for a domain.
Also, IMHO you should provide a command to print the max throttling,
remember that from Xen's PoV Dom0 is just another domain, and the
CPUID values reported to Dom0 don't need to be the same as found on
bare metal.
> +
> + The response of the throttling value could be linear mode or non-linear
> + mode.
> +
> + Linear mode: the input precision is defined as 100-(MBA_MAX). For instance,
What's MBA_MAX? I don't see any reference/description of it above.
> + if the MBA_MAX value is 90, the input precision is 10%. Values not an even
> + multiple of the precision (e.g., 12%) will be rounded down (e.g., to 10%
> + delay applied) by HW automatically.
> +
> + Non-linear mode: input delay values are powers-of-two from zero to the
> + MBA_MAX value from CPUID. In this case any values not a power of two will
> + be rounded down the next nearest power of two by HW automatically.
> +
> +# Technical details
> +
> +MBA is a member of Intel PSR features, it shares the base PSR infrastructure
> +in Xen.
> +
> +## Hardware perspective
> +
> + MBA defines a range of MSRs to support specifying a delay value (Thrtl) per
> + COS, with details below.
> +
> + ```
> + +----------------------------+----------------+
> + | MSR (per socket) | Address |
> + +----------------------------+----------------+
> + | IA32_L2_QOS_Ext_BW_Thrtl_0 | 0xD50 |
> + +----------------------------+----------------+
> + | ... | ... |
> + +----------------------------+----------------+
> + | IA32_L2_QOS_Ext_BW_Thrtl_n | 0xD50+n (n<64) |
> + +----------------------------+----------------+
> + ```
Are you sure you want to hardcode this n<64? Isn't there a chance this
is going to be bumped in newer hardware?
> +
> + When context switch happens, the COS ID of domain is written to per-thread MSR
> + `IA32_PQR_ASSOC`, and then hardware enforces bandwidth allocation according
> + to the throttling value stored in the Thrtl MSR register.
> +
> +## The relationship between MBA and CAT/CDP
> +
> + Generally speaking, MBA is completely independent of CAT/CDP, and any
> + combination may be applied at any time, e.g. enabling MBA with CAT
> + disabled.
> +
> + But it needs to be noticed that MBA shares COS infrastructure with CAT,
> + although MBA is enumerated by different CPUID leaf from CAT (which
> + indicates that the max COS of MBA may be different from CAT). In some
> + cases, a domain is permitted to have a COS that is beyond one (or more)
> + of PSR features but within the others. For instance, let's assume the max
> + COS of MBA is 8 but the max COS of L3 CAT is 16, when a domain is assigned
> + 9 as COS, the L3 CAT CBM associated to COS 9 would be enforced, but for MBA,
> + the HW works as default value is set since COS 9 is beyond the max COS (8)
> + of MBA.
> +
> +## Design Overview
> +
> +* Core COS/Thrtl association
> +
> + When enforcing Memory Bandwidth Allocation, all cores of domains have
> + the same default Thrtl MSR (COS0) which stores the same Thrtl (0). The
> + default Thrtl MSR is used only in hypervisor and is transparent to tool stack
> + and user.
> +
> + System administrator can change PSR allocation policy at runtime by
^s
> + tool stack. Since MBA shares COS ID with CAT/CDP, a COS ID corresponds to a
^ using the tool ...
> + 2-tuple, like [CBM, Thrtl] with only-CAT enalbed, when CDP is enabled,
^ enabled
> + the COS ID corresponds to a 3-tuple, like [Code_CBM, Data_CBM, Thrtl]. If
> + neither CAT nor CDP is enabled, things would be easier, one COS ID corresponds
^ are easier, since one...
> + to one Thrtl.
> +
> +* VCPU schedule
> +
> + This part reuses CAT COS infrastructure.
> +
> +* Multi-sockets
> +
> + Different sockets may have different MBA ability (like max COS)
> + although it is consistent on the same socket. So the capability
> + of per-socket MBA is specified.
> +
> + This part reuses CAT COS infrastructure.
> +
> +## Implementation Description
> +
> +* Hypervisor interfaces:
> +
> + 1. Boot line param: "psr=mba" to enable the feature.
> +
> + 2. SYSCTL:
> + - XEN_SYSCTL_PSR_MBA_get_info: Get system MBA information.
> +
> + 3. DOMCTL:
> + - XEN_DOMCTL_PSR_MBA_OP_GET_THRTL: Get throttling for a domain.
> + - XEN_DOMCTL_PSR_MBA_OP_SET_THRTL: Set throttling for a domain.
> +
> +* xl interfaces:
> +
> + 1. psr-mba-show [domain-id]
> + Show system/domain runtime MBA throttling value. For linear mode,
> + it shows the decimal value. For non-linear mode, it shows hexadecimal
> + value.
> + => XEN_SYSCTL_PSR_MBA_get_info/XEN_DOMCTL_PSR_MBA_OP_GET_THRTL
> +
> + 2. psr-mba-set [OPTIONS] <domain-id> <throttling>
> + Set bandwidth throttling for a domain.
> + => XEN_DOMCTL_PSR_MBA_OP_SET_THRTL
> +
> + 3. psr-hwinfo
> + Show PSR HW information, including L3 CAT/CDP/L2 CAT/MBA.
> + => XEN_SYSCTL_PSR_MBA_get_info
> +
> +* Key data structure:
> +
> + 1. Feature HW info
> +
> + ```
> + struct {
> + unsigned int thrtl_max;
> + unsigned int linear;
> + } mba_info;
Is this a domctl structure? a libxl one?
> +
> + - Member `thrtl_max`
> +
> + `thrtl_max` is the max throttling value to be set.
> +
> + - Member `linear`
> +
> + `linear` means the response of delay value is linear or not.
> +
> + As mentioned above, MBA is a member of Intel PSR features, it would
> + share the base PSR infrastructure in Xen. For example, the 'cos_max'
> + is a common HW property for all features. So, for other data structure
> + details, please refer 'intel_psr_cat_cdp.pandoc'.
> +
> +# Limitations
> +
> +MBA can only work on HW which enables it (check by CPUID).
> +
> +# Testing
> +
> +We can execute these commands to verify MBA on different HWs supporting them.
> +
> +For example:
> + root@:~$ xl psr-hwinfo --mba
> + Memory Bandwidth Allocation (MBA):
> + Socket ID : 0
> + Linear Mode : Enabled
> + Maximum COS : 7
> + Maximum Throttling Value: 90
> + Default Throttling Value: 0
> +
> + root@:~$ xl psr-mba-set 1 0xa
Could you elaborate a little bit on why '0xa' is used here? IMHO The
example should provide some context.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
2017-08-24 1:14 ` [PATCH v2 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general Yi Sun
@ 2017-08-29 12:00 ` Roger Pau Monné
2017-08-30 5:23 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2017-08-29 12:00 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 24, 2017 at 09:14:36AM +0800, Yi Sun wrote:
> --- a/xen/arch/x86/domctl.c
> +++ b/xen/arch/x86/domctl.c
> @@ -1439,60 +1439,60 @@ long arch_do_domctl(
> }
> break;
>
> - case XEN_DOMCTL_psr_cat_op:
> - switch ( domctl->u.psr_cat_op.cmd )
> + case XEN_DOMCTL_psr_alloc_op:
> + switch ( domctl->u.psr_alloc_op.cmd )
> {
> uint32_t val32;
>
> case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM:
In order to match the name of the structure used, shouldn't those
defines be renamed to XEN_DOMCTL_PSR_ALLOC_* (instead of CAT)?
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 03/15] x86: rename 'cbm_type' to 'psr_val_type' to make it general
2017-08-24 1:14 ` [PATCH v2 03/15] x86: rename 'cbm_type' to 'psr_val_type' to make it general Yi Sun
@ 2017-08-29 12:15 ` Roger Pau Monné
2017-08-30 5:47 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2017-08-29 12:15 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 24, 2017 at 09:14:37AM +0800, Yi Sun wrote:
> This patch renames 'cbm_type' to 'psr_val_type' to make it be general.
I'm unsure whether the '_val_' is really needed here. AFAICT it
doesn't add any meaningful information to the define itself, and only
makes it longer. Apart from that it looks fine.
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 04/15] x86: implement data structure and CPU init flow for MBA
2017-08-24 1:14 ` [PATCH v2 04/15] x86: implement data structure and CPU init flow for MBA Yi Sun
@ 2017-08-29 13:44 ` Roger Pau Monné
2017-08-29 13:58 ` Jan Beulich
2017-08-30 5:31 ` Yi Sun
0 siblings, 2 replies; 64+ messages in thread
From: Roger Pau Monné @ 2017-08-29 13:44 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 24, 2017 at 09:14:38AM +0800, Yi Sun wrote:
> This patch implements main data structures of MBA.
>
> Like CAT features, MBA HW info has cos_max which means the max thrtl
> register number, and thrtl_max which means the max throttle value
> (delay value). It also has a flag to represent if the throttle
> value is linear or not.
>
> One thrtl register of MBA stores a throttle value for one or more
> domains. The throttle value means the transaction time between L2
> cache and next level memory to be delayed.
>
> This patch also implements init flow for MBA and register stub
> callback functions.
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> ---
> v2:
> - modify commit message to replace 'cos register' to 'thrtl register' to
> make it accurate.
> (suggested by Chao Peng)
> - restore the place of the sentence to assign value to 'feat->cbm_len'
> because the MBA init flow is splitted out as a separate function in v1.
> (suggested by Chao Peng)
> - add comment to explain what the MBA thrtl defaul value '0' stands for.
> (suggested by Chao Peng)
> - check 'thrtl_max' under linear mode. It could not be euqal or larger than
> 100.
> (suggested by Chao Peng)
> v1:
> - rebase codes onto L2 CAT v15.
> - move comment to appropriate place.
> (suggested by Chao Peng)
> - implement 'mba_init_feature' and keep 'cat_init_feature'.
> (suggested by Chao Peng)
> - keep 'regs.b' into a local variable to avoid reading CPUID every time.
> (suggested by Chao Peng)
> ---
> xen/arch/x86/psr.c | 141 +++++++++++++++++++++++++++++++++++-----
> xen/include/asm-x86/msr-index.h | 1 +
> xen/include/asm-x86/psr.h | 2 +
> 3 files changed, 126 insertions(+), 18 deletions(-)
>
> diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
> index da62f81..f5e99ce 100644
> --- a/xen/arch/x86/psr.c
> +++ b/xen/arch/x86/psr.c
> @@ -27,13 +27,16 @@
> * - CMT Cache Monitoring Technology
> * - COS/CLOS Class of Service. Also mean COS registers.
> * - COS_MAX Max number of COS for the feature (minus 1)
> + * - MBA Memory Bandwidth Allocation
> * - MSRs Machine Specific Registers
> * - PSR Intel Platform Shared Resource
> + * - THRTL_MAX Max throttle value (delay value) of MBA
> */
>
> #define PSR_CMT (1u << 0)
> #define PSR_CAT (1u << 1)
> #define PSR_CDP (1u << 2)
> +#define PSR_MBA (1u << 3)
>
> #define CAT_CBM_LEN_MASK 0x1f
> #define CAT_COS_MAX_MASK 0xffff
> @@ -60,10 +63,14 @@
> */
> #define MAX_COS_NUM 2
>
> +#define MBA_LINEAR (1u << 2)
Why is this shifted by 2?
> +#define MBA_THRTL_MAX_MASK 0xfff
> +
> enum psr_feat_type {
> FEAT_TYPE_L3_CAT,
> FEAT_TYPE_L3_CDP,
> FEAT_TYPE_L2_CAT,
> + FEAT_TYPE_MBA,
> FEAT_TYPE_NUM,
> FEAT_TYPE_UNKNOWN,
> };
> @@ -71,7 +78,6 @@ enum psr_feat_type {
> /*
> * This structure represents one feature.
> * cos_max - The max COS registers number got through CPUID.
> - * cbm_len - The length of CBM got through CPUID.
> * cos_reg_val - Array to store the values of COS registers. One entry stores
> * the value of one COS register.
> * For L3 CAT and L2 CAT, one entry corresponds to one COS_ID.
> @@ -80,9 +86,23 @@ enum psr_feat_type {
> * cos_reg_val[1] (Code).
> */
> struct feat_node {
> - /* cos_max and cbm_len are common values for all features so far. */
> + /* cos_max is common values for all features so far. */
> unsigned int cos_max;
> - unsigned int cbm_len;
> +
> + /* Feature specific HW info. */
> + union {
> + struct {
> + /* The length of CBM got through CPUID. */
> + unsigned int cbm_len;
> + } cat_info;
> +
> + struct {
> + /* The max throttling value got through CPUID. */
> + unsigned int thrtl_max;
> + unsigned int linear;
This seems like it wants to be a boolean?
> + } mba_info;
Just naming the fields 'cat' and 'mba' would probably be enough IMHO,
but that's just taste I think, and I won't argue if you prefer to
leave them with the _info suffix.
> + };
> +
> uint32_t cos_reg_val[MAX_COS_REG_CNT];
> };
>
> @@ -161,6 +181,7 @@ static DEFINE_PER_CPU(struct psr_assoc, psr_assoc);
> */
> static struct feat_node *feat_l3;
> static struct feat_node *feat_l2_cat;
> +static struct feat_node *feat_mba;
>
> /* Common functions */
> #define cat_default_val(len) (0xffffffff >> (32 - (len)))
> @@ -273,7 +294,7 @@ static bool psr_check_cbm(unsigned int cbm_len, unsigned long cbm)
> return true;
> }
>
> -/* CAT common functions implementation. */
> +/* Implementation of allocation features' functions. */
> static int cat_init_feature(const struct cpuid_leaf *regs,
> struct feat_node *feat,
> struct psr_socket_info *info,
> @@ -289,8 +310,8 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
> if ( !regs->a || !regs->d )
> return -ENOENT;
>
> - feat->cbm_len = (regs->a & CAT_CBM_LEN_MASK) + 1;
> feat->cos_max = min(opt_cos_max, regs->d & CAT_COS_MAX_MASK);
> + feat->cat_info.cbm_len = (regs->a & CAT_CBM_LEN_MASK) + 1;
>
> switch ( type )
> {
> @@ -300,12 +321,12 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
> return -ENOENT;
>
> /* We reserve cos=0 as default cbm (all bits within cbm_len are 1). */
> - feat->cos_reg_val[0] = cat_default_val(feat->cbm_len);
> + feat->cos_reg_val[0] = cat_default_val(feat->cat_info.cbm_len);
>
> wrmsrl((type == FEAT_TYPE_L3_CAT ?
> MSR_IA32_PSR_L3_MASK(0) :
> MSR_IA32_PSR_L2_MASK(0)),
> - cat_default_val(feat->cbm_len));
> + cat_default_val(feat->cat_info.cbm_len));
>
> break;
>
> @@ -320,11 +341,13 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
> feat->cos_max = (feat->cos_max - 1) >> 1;
>
> /* We reserve cos=0 as default cbm (all bits within cbm_len are 1). */
> - get_cdp_code(feat, 0) = cat_default_val(feat->cbm_len);
> - get_cdp_data(feat, 0) = cat_default_val(feat->cbm_len);
> + get_cdp_code(feat, 0) = cat_default_val(feat->cat_info.cbm_len);
> + get_cdp_data(feat, 0) = cat_default_val(feat->cat_info.cbm_len);
>
> - wrmsrl(MSR_IA32_PSR_L3_MASK(0), cat_default_val(feat->cbm_len));
> - wrmsrl(MSR_IA32_PSR_L3_MASK(1), cat_default_val(feat->cbm_len));
> + wrmsrl(MSR_IA32_PSR_L3_MASK(0),
> + cat_default_val(feat->cat_info.cbm_len));
> + wrmsrl(MSR_IA32_PSR_L3_MASK(1),
> + cat_default_val(feat->cat_info.cbm_len));
> rdmsrl(MSR_IA32_PSR_L3_QOS_CFG, val);
> wrmsrl(MSR_IA32_PSR_L3_QOS_CFG,
> val | (1ull << PSR_L3_QOS_CDP_ENABLE_BIT));
> @@ -344,7 +367,51 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
>
> printk(XENLOG_INFO "%s: enabled on socket %u, cos_max:%u, cbm_len:%u\n",
> cat_feat_name[type], cpu_to_socket(smp_processor_id()),
> - feat->cos_max, feat->cbm_len);
> + feat->cos_max, feat->cat_info.cbm_len);
> +
> + return 0;
> +}
> +
> +static int mba_init_feature(const struct cpuid_leaf *regs,
> + struct feat_node *feat,
> + struct psr_socket_info *info,
> + enum psr_feat_type type)
> +{
> + /* No valid value so do not enable feature. */
> + if ( !regs->a || !regs->d )
> + return -ENOENT;
> +
> + if ( type != FEAT_TYPE_MBA )
> + return -ENOENT;
> +
> + feat->cos_max = min(opt_cos_max, regs->d & CAT_COS_MAX_MASK);
> + if ( feat->cos_max < 1 )
> + return -ENOENT;
> +
> + feat->mba_info.thrtl_max = (regs->a & MBA_THRTL_MAX_MASK) + 1;
> +
> + if ( regs->c & MBA_LINEAR )
> + {
> + feat->mba_info.linear = 1;
> +
> + if ( feat->mba_info.thrtl_max >= 100 )
> + return -ENOENT;
> + }
> +
> + /* We reserve cos=0 as default thrtl (0) which means no delay. */
> + feat->cos_reg_val[0] = 0;
> + wrmsrl(MSR_IA32_PSR_MBA_MASK(0), 0);
> +
> + /* Add this feature into array. */
> + info->features[type] = feat;
> +
> + if ( !opt_cpu_info )
> + return 0;
> +
> + printk(XENLOG_INFO "MBA: enabled on socket %u, cos_max:%u,"
> + "thrtl_max:%u, linear:%u.\n",
Please try to avoid splitting messages, it makes it hard to grep for
them afterward.
> + cpu_to_socket(smp_processor_id()),
> + feat->cos_max, feat->mba_info.thrtl_max, feat->mba_info.linear);
>
> return 0;
> }
> @@ -356,7 +423,7 @@ static bool cat_get_feat_info(const struct feat_node *feat,
> return false;
>
> data[PSR_INFO_IDX_COS_MAX] = feat->cos_max;
> - data[PSR_INFO_IDX_CAT_CBM_LEN] = feat->cbm_len;
> + data[PSR_INFO_IDX_CAT_CBM_LEN] = feat->cat_info.cbm_len;
> data[PSR_INFO_IDX_CAT_FLAG] = 0;
>
> return true;
> @@ -422,6 +489,26 @@ static const struct feat_props l2_cat_props = {
> .write_msr = l2_cat_write_msr,
> };
>
> +/* MBA props */
> +static bool mba_get_feat_info(const struct feat_node *feat,
> + uint32_t data[], unsigned int array_len)
> +{
> + return false;
> +}
Shouldn't this return thrtl_max and whether it's linear?
> +
> +static void mba_write_msr(unsigned int cos, uint32_t val,
> + enum psr_val_type type)
> +{
And this perform the MSR write? (MSR_IA32_PSR_MBA_MASK...)
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 04/15] x86: implement data structure and CPU init flow for MBA
2017-08-29 13:44 ` Roger Pau Monné
@ 2017-08-29 13:58 ` Jan Beulich
2017-08-30 6:07 ` Yi Sun
2017-08-30 5:31 ` Yi Sun
1 sibling, 1 reply; 64+ messages in thread
From: Jan Beulich @ 2017-08-29 13:58 UTC (permalink / raw)
To: Roger Pau Monné, Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, xen-devel, chao.p.peng
>>> On 29.08.17 at 15:44, <roger.pau@citrix.com> wrote:
> On Thu, Aug 24, 2017 at 09:14:38AM +0800, Yi Sun wrote:
>> @@ -80,9 +86,23 @@ enum psr_feat_type {
>> * cos_reg_val[1] (Code).
>> */
>> struct feat_node {
>> - /* cos_max and cbm_len are common values for all features so far. */
>> + /* cos_max is common values for all features so far. */
>> unsigned int cos_max;
>> - unsigned int cbm_len;
>> +
>> + /* Feature specific HW info. */
>> + union {
>> + struct {
>> + /* The length of CBM got through CPUID. */
>> + unsigned int cbm_len;
>> + } cat_info;
>> +
>> + struct {
>> + /* The max throttling value got through CPUID. */
>> + unsigned int thrtl_max;
>> + unsigned int linear;
>
> This seems like it wants to be a boolean?
>
>> + } mba_info;
>
> Just naming the fields 'cat' and 'mba' would probably be enough IMHO,
> but that's just taste I think, and I won't argue if you prefer to
> leave them with the _info suffix.
It's not just a matter of taste - longer names are longer to type
and parse, so if a shorter name can be used unambiguously, it
always should be imo.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 05/15] x86: implement get hw info flow for MBA
2017-08-24 1:14 ` [PATCH v2 05/15] x86: implement get hw info " Yi Sun
@ 2017-08-29 15:01 ` Roger Pau Monné
2017-08-30 5:33 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2017-08-29 15:01 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 24, 2017 at 09:14:39AM +0800, Yi Sun wrote:
> diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
> index df54d38..cbab85a 100644
> --- a/xen/arch/x86/sysctl.c
> +++ b/xen/arch/x86/sysctl.c
> @@ -214,6 +214,25 @@ long arch_do_sysctl(
> break;
> }
>
> + case XEN_SYSCTL_PSR_MBA_get_info:
> + {
> + ret = psr_get_info(sysctl->u.psr_alloc_op.target,
> + PSR_VAL_TYPE_MBA, data, ARRAY_SIZE(data));
> + if ( ret )
> + break;
> +
> + sysctl->u.psr_alloc_op.u.mba_info.cos_max =
> + data[PSR_INFO_IDX_COS_MAX];
> + sysctl->u.psr_alloc_op.u.mba_info.thrtl_max =
> + data[PSR_INFO_IDX_MBA_THRTL_MAX];
> + sysctl->u.psr_alloc_op.u.mba_info.flags =
> + data[PSR_INFO_IDX_MBA_FLAG];
> +
> + if ( !ret && __copy_field_to_guest(u_sysctl, sysctl, u.psr_alloc_op) )
> + ret = -EFAULT;
The !ret is not needed here AFAICT. If psr_get_info sets ret to != 0
you will break, so you cannot get here with ret != 0.
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 06/15] x86: implement get value interface for MBA
2017-08-24 1:14 ` [PATCH v2 06/15] x86: implement get value interface " Yi Sun
@ 2017-08-29 15:04 ` Roger Pau Monné
0 siblings, 0 replies; 64+ messages in thread
From: Roger Pau Monné @ 2017-08-29 15:04 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 24, 2017 at 09:14:40AM +0800, Yi Sun wrote:
> This patch implements get value domctl interface for MBA.
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Just one minor comment...
> ---
> xen/arch/x86/domctl.c | 7 +++++++
> xen/include/public/domctl.h | 1 +
> 2 files changed, 8 insertions(+)
>
> diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
> index e1c7c70..4936bcb 100644
> --- a/xen/arch/x86/domctl.c
> +++ b/xen/arch/x86/domctl.c
> @@ -1496,6 +1496,13 @@ long arch_do_domctl(
> copyback = true;
> break;
>
> + case XEN_DOMCTL_PSR_MBA_OP_GET_THRTL:
> + ret = psr_get_val(d, domctl->u.psr_alloc_op.target,
> + &val32, PSR_VAL_TYPE_MBA);
> + domctl->u.psr_alloc_op.data = val32;
> + copyback = true;
> + break;
> +
> default:
> ret = -EOPNOTSUPP;
> break;
> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
> index 4bb4399..a577a3e 100644
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -1144,6 +1144,7 @@ struct xen_domctl_psr_alloc_op {
> #define XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA 5
> #define XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM 6
> #define XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM 7
> +#define XEN_DOMCTL_PSR_MBA_OP_GET_THRTL 9
I think the '_OP_' is not really meaningful here, but given the other
operations already have it...
> uint32_t cmd; /* IN: XEN_DOMCTL_PSR_CAT_OP_* */
> uint32_t target; /* IN */
> uint64_t data; /* IN/OUT */
> --
> 1.9.1
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 01/15] docs: create Memory Bandwidth Allocation (MBA) feature document
2017-08-29 11:46 ` Roger Pau Monné
@ 2017-08-30 5:20 ` Yi Sun
2017-08-30 7:42 ` Roger Pau Monn�
0 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-30 5:20 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
Thanks a lot for the review comments!
On 17-08-29 12:46:49, Roger Pau Monn� wrote:
> On Thu, Aug 24, 2017 at 09:14:35AM +0800, Yi Sun wrote:
> > +# Overview
> > +
> > +The Memory Bandwidth Allocation (MBA) feature provides indirect and approximate
> > +control over memory bandwidth available per-core. This feature provides OS/
> > +hypervisor the ability to slow misbehaving apps/domains or create advanced
> > +closed-loop control system via exposing control over a credit-based throttling
> > +mechanism.
>
> I don't really understand what "advanced closed-loop control system
> via exposing..." means. From my understand it is clear/simpler to
> write it as:
>
> "... the ability to slow misbehaving apps/domains by using a
> credit-based throttling mechanism".
>
Thanks, 'closed-loop' looks redundant, will remove it.
> > +
> > +# User details
> > +
> > +* Feature Enabling:
> > +
> > + Add "psr=mba" to boot line parameter to enable MBA feature.
> > +
> > +* xl interfaces:
> > +
> > + 1. `psr-mba-show [domain-id]`:
> > +
> > + Show memory bandwidth throttling for domain. For linear mode, it shows the
> > + decimal value. For non-linear mode, it shows hexadecimal value.
>
> You should first explain what are the linear and non-linear modes.
>
Ok, will move below linear/non-linear modes explanation to here.
> > +
> > + 2. `psr-mba-set [OPTIONS] <domain-id> <throttling>`:
> > +
> > + Set memory bandwidth throttling for domain.
> > +
> > + Options:
> > + '-s': Specify the socket to process, otherwise all sockets are processed.
> > +
> > + Throttling value set in register implies memory bandwidth blocked, i.e.
> > + higher throttling value results in lower bandwidth. The max throttling
> > + value can be got through CPUID.
>
> This is also hard to understand IMHO. I would rather write it as:
>
> "The throttling value describes the amount of blocked bandwidth".
Thanks! This is not accurate. In fact, throttling value means the approximate
amount of delaying the traffic between core and memory.
> Although I have to admit I don't really understand this interface,
> wouldn't it be easier to specify the memory bandwidth allowed
> per-domain, rather the amount of bandwidth removed?
>
> Using your approach the user has to first get the total bandwidth, and
> then subtract the removed bandwidth in order to know the remaining
> bandwidth for a domain.
>
The HW only provides throttling set method to control the bandwidth. So, I
think it is straightforward to set throttling in tools layer. The 'psr-mba-set'
is designed as a simple command to set what HW needs.
Also, mentioned by SDM, "The throttling values exposed by MBA are approximate,
and are calibrated to specific traffic patterns.". So, it is hard to provide
exact bandwidth control in 'psr-mba-set'.
> Also, IMHO you should provide a command to print the max throttling,
The 'psr-hwinfo' can show the max throttling. Because it is part of MBA HW info.
> remember that from Xen's PoV Dom0 is just another domain, and the
> CPUID values reported to Dom0 don't need to be the same as found on
> bare metal.
>
But the CPUID values got through 'psr' commands should be ones found on bare
metal, right? Because these commands directly get the values from hypervisor
through domctl/sysctl.
> > +
> > + The response of the throttling value could be linear mode or non-linear
> > + mode.
> > +
> > + Linear mode: the input precision is defined as 100-(MBA_MAX). For instance,
>
> What's MBA_MAX? I don't see any reference/description of it above.
>
Sorry, will explain it.
> > + if the MBA_MAX value is 90, the input precision is 10%. Values not an even
> > + multiple of the precision (e.g., 12%) will be rounded down (e.g., to 10%
> > + delay applied) by HW automatically.
> > +
> > + Non-linear mode: input delay values are powers-of-two from zero to the
> > + MBA_MAX value from CPUID. In this case any values not a power of two will
> > + be rounded down the next nearest power of two by HW automatically.
> > +
> > +# Technical details
> > +
> > +MBA is a member of Intel PSR features, it shares the base PSR infrastructure
> > +in Xen.
> > +
> > +## Hardware perspective
> > +
> > + MBA defines a range of MSRs to support specifying a delay value (Thrtl) per
> > + COS, with details below.
> > +
> > + ```
> > + +----------------------------+----------------+
> > + | MSR (per socket) | Address |
> > + +----------------------------+----------------+
> > + | IA32_L2_QOS_Ext_BW_Thrtl_0 | 0xD50 |
> > + +----------------------------+----------------+
> > + | ... | ... |
> > + +----------------------------+----------------+
> > + | IA32_L2_QOS_Ext_BW_Thrtl_n | 0xD50+n (n<64) |
> > + +----------------------------+----------------+
> > + ```
>
> Are you sure you want to hardcode this n<64? Isn't there a chance this
> is going to be bumped in newer hardware?
>
This is just a HW limitation declared in SDM. In fact, there is no such hard
code limitation. Hypervisor side checks the 'cos_max' got through CPUID.
> > +
> > + When context switch happens, the COS ID of domain is written to per-thread MSR
> > + `IA32_PQR_ASSOC`, and then hardware enforces bandwidth allocation according
> > + to the throttling value stored in the Thrtl MSR register.
> > +
> > +## The relationship between MBA and CAT/CDP
> > +
> > + Generally speaking, MBA is completely independent of CAT/CDP, and any
> > + combination may be applied at any time, e.g. enabling MBA with CAT
> > + disabled.
> > +
> > + But it needs to be noticed that MBA shares COS infrastructure with CAT,
> > + although MBA is enumerated by different CPUID leaf from CAT (which
> > + indicates that the max COS of MBA may be different from CAT). In some
> > + cases, a domain is permitted to have a COS that is beyond one (or more)
> > + of PSR features but within the others. For instance, let's assume the max
> > + COS of MBA is 8 but the max COS of L3 CAT is 16, when a domain is assigned
> > + 9 as COS, the L3 CAT CBM associated to COS 9 would be enforced, but for MBA,
> > + the HW works as default value is set since COS 9 is beyond the max COS (8)
> > + of MBA.
> > +
> > +## Design Overview
> > +
> > +* Core COS/Thrtl association
> > +
> > + When enforcing Memory Bandwidth Allocation, all cores of domains have
> > + the same default Thrtl MSR (COS0) which stores the same Thrtl (0). The
> > + default Thrtl MSR is used only in hypervisor and is transparent to tool stack
> > + and user.
> > +
> > + System administrator can change PSR allocation policy at runtime by
> ^s
> > + tool stack. Since MBA shares COS ID with CAT/CDP, a COS ID corresponds to a
> ^ using the tool ...
Will modify this and below words.
> > + 2-tuple, like [CBM, Thrtl] with only-CAT enalbed, when CDP is enabled,
> ^ enabled
> > + the COS ID corresponds to a 3-tuple, like [Code_CBM, Data_CBM, Thrtl]. If
> > + neither CAT nor CDP is enabled, things would be easier, one COS ID corresponds
> ^ are easier, since one...
> > + to one Thrtl.
> > +
> > +* VCPU schedule
> > +
> > + This part reuses CAT COS infrastructure.
> > +
> > +* Multi-sockets
> > +
> > + Different sockets may have different MBA ability (like max COS)
> > + although it is consistent on the same socket. So the capability
> > + of per-socket MBA is specified.
> > +
> > + This part reuses CAT COS infrastructure.
> > +
> > +## Implementation Description
> > +
> > +* Hypervisor interfaces:
> > +
> > + 1. Boot line param: "psr=mba" to enable the feature.
> > +
> > + 2. SYSCTL:
> > + - XEN_SYSCTL_PSR_MBA_get_info: Get system MBA information.
> > +
> > + 3. DOMCTL:
> > + - XEN_DOMCTL_PSR_MBA_OP_GET_THRTL: Get throttling for a domain.
> > + - XEN_DOMCTL_PSR_MBA_OP_SET_THRTL: Set throttling for a domain.
> > +
> > +* xl interfaces:
> > +
> > + 1. psr-mba-show [domain-id]
> > + Show system/domain runtime MBA throttling value. For linear mode,
> > + it shows the decimal value. For non-linear mode, it shows hexadecimal
> > + value.
> > + => XEN_SYSCTL_PSR_MBA_get_info/XEN_DOMCTL_PSR_MBA_OP_GET_THRTL
> > +
> > + 2. psr-mba-set [OPTIONS] <domain-id> <throttling>
> > + Set bandwidth throttling for a domain.
> > + => XEN_DOMCTL_PSR_MBA_OP_SET_THRTL
> > +
> > + 3. psr-hwinfo
> > + Show PSR HW information, including L3 CAT/CDP/L2 CAT/MBA.
> > + => XEN_SYSCTL_PSR_MBA_get_info
> > +
> > +* Key data structure:
> > +
> > + 1. Feature HW info
> > +
> > + ```
> > + struct {
> > + unsigned int thrtl_max;
> > + unsigned int linear;
> > + } mba_info;
>
> Is this a domctl structure? a libxl one?
>
Nope, this is a hypervisor side data structure used in 'psr.c'.
> > +
> > + - Member `thrtl_max`
> > +
> > + `thrtl_max` is the max throttling value to be set.
> > +
> > + - Member `linear`
> > +
> > + `linear` means the response of delay value is linear or not.
> > +
> > + As mentioned above, MBA is a member of Intel PSR features, it would
> > + share the base PSR infrastructure in Xen. For example, the 'cos_max'
> > + is a common HW property for all features. So, for other data structure
> > + details, please refer 'intel_psr_cat_cdp.pandoc'.
> > +
> > +# Limitations
> > +
> > +MBA can only work on HW which enables it (check by CPUID).
> > +
> > +# Testing
> > +
> > +We can execute these commands to verify MBA on different HWs supporting them.
> > +
> > +For example:
> > + root@:~$ xl psr-hwinfo --mba
> > + Memory Bandwidth Allocation (MBA):
> > + Socket ID : 0
> > + Linear Mode : Enabled
> > + Maximum COS : 7
> > + Maximum Throttling Value: 90
> > + Default Throttling Value: 0
> > +
> > + root@:~$ xl psr-mba-set 1 0xa
>
> Could you elaborate a little bit on why '0xa' is used here? IMHO The
> example should provide some context.
>
Sure, I will explain the meaning of '0xa' or '10' here.
> Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
2017-08-29 12:00 ` Roger Pau Monné
@ 2017-08-30 5:23 ` Yi Sun
2017-08-30 7:47 ` Roger Pau Monn�
0 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-30 5:23 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On 17-08-29 13:00:19, Roger Pau Monn� wrote:
> On Thu, Aug 24, 2017 at 09:14:36AM +0800, Yi Sun wrote:
> > --- a/xen/arch/x86/domctl.c
> > +++ b/xen/arch/x86/domctl.c
> > @@ -1439,60 +1439,60 @@ long arch_do_domctl(
> > }
> > break;
> >
> > - case XEN_DOMCTL_psr_cat_op:
> > - switch ( domctl->u.psr_cat_op.cmd )
> > + case XEN_DOMCTL_psr_alloc_op:
> > + switch ( domctl->u.psr_alloc_op.cmd )
> > {
> > uint32_t val32;
> >
> > case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM:
>
> In order to match the name of the structure used, shouldn't those
> defines be renamed to XEN_DOMCTL_PSR_ALLOC_* (instead of CAT)?
>
These names are feature specific. E.g:
XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM - L3 CAT
XEN_DOMCTL_PSR_MBA_OP_SET_THRTL - MBA
Could you accept them? If not, I think I may write a separate patch to rename
them.
> Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 04/15] x86: implement data structure and CPU init flow for MBA
2017-08-29 13:44 ` Roger Pau Monné
2017-08-29 13:58 ` Jan Beulich
@ 2017-08-30 5:31 ` Yi Sun
2017-08-30 7:55 ` Roger Pau Monn�
1 sibling, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-30 5:31 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On 17-08-29 14:44:32, Roger Pau Monn� wrote:
> On Thu, Aug 24, 2017 at 09:14:38AM +0800, Yi Sun wrote:
> > +#define MBA_LINEAR (1u << 2)
>
> Why is this shifted by 2?
>
This is used to mask ECX register to know if MBA linear mode is set or not.
> > + union {
> > + struct {
> > + /* The length of CBM got through CPUID. */
> > + unsigned int cbm_len;
> > + } cat_info;
> > +
> > + struct {
> > + /* The max throttling value got through CPUID. */
> > + unsigned int thrtl_max;
> > + unsigned int linear;
>
> This seems like it wants to be a boolean?
>
Ok, will change it to bool.
> > + } mba_info;
>
> Just naming the fields 'cat' and 'mba' would probably be enough IMHO,
> but that's just taste I think, and I won't argue if you prefer to
> leave them with the _info suffix.
>
Got it, thanks!
> > + /* We reserve cos=0 as default thrtl (0) which means no delay. */
> > + feat->cos_reg_val[0] = 0;
> > + wrmsrl(MSR_IA32_PSR_MBA_MASK(0), 0);
> > +
> > + /* Add this feature into array. */
> > + info->features[type] = feat;
> > +
> > + if ( !opt_cpu_info )
> > + return 0;
> > +
> > + printk(XENLOG_INFO "MBA: enabled on socket %u, cos_max:%u,"
> > + "thrtl_max:%u, linear:%u.\n",
>
> Please try to avoid splitting messages, it makes it hard to grep for
> them afterward.
>
Ok, but that would exceed 80 characters. Is that acceptable?
> > +/* MBA props */
> > +static bool mba_get_feat_info(const struct feat_node *feat,
> > + uint32_t data[], unsigned int array_len)
> > +{
> > + return false;
> > +}
>
> Shouldn't this return thrtl_max and whether it's linear?
>
> > +
> > +static void mba_write_msr(unsigned int cos, uint32_t val,
> > + enum psr_val_type type)
> > +{
>
> And this perform the MSR write? (MSR_IA32_PSR_MBA_MASK...)
>
Because the patches may not be applied one go. I have to implement these
empty functions and register them into 'mba_props' to avoid crash. IIRC, this
was suggested by Jan before.
> Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 05/15] x86: implement get hw info flow for MBA
2017-08-29 15:01 ` Roger Pau Monné
@ 2017-08-30 5:33 ` Yi Sun
0 siblings, 0 replies; 64+ messages in thread
From: Yi Sun @ 2017-08-30 5:33 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On 17-08-29 16:01:00, Roger Pau Monn� wrote:
> On Thu, Aug 24, 2017 at 09:14:39AM +0800, Yi Sun wrote:
> > diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
> > index df54d38..cbab85a 100644
> > --- a/xen/arch/x86/sysctl.c
> > +++ b/xen/arch/x86/sysctl.c
> > @@ -214,6 +214,25 @@ long arch_do_sysctl(
> > break;
> > }
> >
> > + case XEN_SYSCTL_PSR_MBA_get_info:
> > + {
> > + ret = psr_get_info(sysctl->u.psr_alloc_op.target,
> > + PSR_VAL_TYPE_MBA, data, ARRAY_SIZE(data));
> > + if ( ret )
> > + break;
> > +
> > + sysctl->u.psr_alloc_op.u.mba_info.cos_max =
> > + data[PSR_INFO_IDX_COS_MAX];
> > + sysctl->u.psr_alloc_op.u.mba_info.thrtl_max =
> > + data[PSR_INFO_IDX_MBA_THRTL_MAX];
> > + sysctl->u.psr_alloc_op.u.mba_info.flags =
> > + data[PSR_INFO_IDX_MBA_FLAG];
> > +
> > + if ( !ret && __copy_field_to_guest(u_sysctl, sysctl, u.psr_alloc_op) )
> > + ret = -EFAULT;
>
> The !ret is not needed here AFAICT. If psr_get_info sets ret to != 0
> you will break, so you cannot get here with ret != 0.
>
You are right. Sorry for this.
> Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 03/15] x86: rename 'cbm_type' to 'psr_val_type' to make it general
2017-08-29 12:15 ` Roger Pau Monné
@ 2017-08-30 5:47 ` Yi Sun
2017-08-30 7:51 ` Roger Pau Monn�
0 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-30 5:47 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On 17-08-29 13:15:24, Roger Pau Monn� wrote:
> On Thu, Aug 24, 2017 at 09:14:37AM +0800, Yi Sun wrote:
> > This patch renames 'cbm_type' to 'psr_val_type' to make it be general.
>
> I'm unsure whether the '_val_' is really needed here. AFAICT it
> doesn't add any meaningful information to the define itself, and only
> makes it longer. Apart from that it looks fine.
>
The 'cbm_type' means the CAT value (CBM) type. As MBA value type is throttle,
I want to rename 'cbm_type' to a generic value type. Because there are many
'psr' starting names, to make it clear that this structure is for value type,
I name it 'psr_val_type'.
If you still think it is not appropriate, I will remove '_val'.
> Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 04/15] x86: implement data structure and CPU init flow for MBA
2017-08-29 13:58 ` Jan Beulich
@ 2017-08-30 6:07 ` Yi Sun
0 siblings, 0 replies; 64+ messages in thread
From: Yi Sun @ 2017-08-30 6:07 UTC (permalink / raw)
To: Jan Beulich
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, xen-devel, chao.p.peng,
Roger Pau Monné
On 17-08-29 07:58:04, Jan Beulich wrote:
> >>> On 29.08.17 at 15:44, <roger.pau@citrix.com> wrote:
> > On Thu, Aug 24, 2017 at 09:14:38AM +0800, Yi Sun wrote:
> >> @@ -80,9 +86,23 @@ enum psr_feat_type {
> >> * cos_reg_val[1] (Code).
> >> */
> >> struct feat_node {
> >> - /* cos_max and cbm_len are common values for all features so far. */
> >> + /* cos_max is common values for all features so far. */
> >> unsigned int cos_max;
> >> - unsigned int cbm_len;
> >> +
> >> + /* Feature specific HW info. */
> >> + union {
> >> + struct {
> >> + /* The length of CBM got through CPUID. */
> >> + unsigned int cbm_len;
> >> + } cat_info;
> >> +
> >> + struct {
> >> + /* The max throttling value got through CPUID. */
> >> + unsigned int thrtl_max;
> >> + unsigned int linear;
> >
> > This seems like it wants to be a boolean?
> >
> >> + } mba_info;
> >
> > Just naming the fields 'cat' and 'mba' would probably be enough IMHO,
> > but that's just taste I think, and I won't argue if you prefer to
> > leave them with the _info suffix.
>
> It's not just a matter of taste - longer names are longer to type
> and parse, so if a shorter name can be used unambiguously, it
> always should be imo.
>
Thanks! Will notice this rule in the future.
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 01/15] docs: create Memory Bandwidth Allocation (MBA) feature document
2017-08-30 5:20 ` Yi Sun
@ 2017-08-30 7:42 ` Roger Pau Monn�
0 siblings, 0 replies; 64+ messages in thread
From: Roger Pau Monn� @ 2017-08-30 7:42 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Wed, Aug 30, 2017 at 01:20:14PM +0800, Yi Sun wrote:
> Thanks a lot for the review comments!
>
> On 17-08-29 12:46:49, Roger Pau Monn� wrote:
> > On Thu, Aug 24, 2017 at 09:14:35AM +0800, Yi Sun wrote:
> > Although I have to admit I don't really understand this interface,
> > wouldn't it be easier to specify the memory bandwidth allowed
> > per-domain, rather the amount of bandwidth removed?
> >
> > Using your approach the user has to first get the total bandwidth, and
> > then subtract the removed bandwidth in order to know the remaining
> > bandwidth for a domain.
> >
> The HW only provides throttling set method to control the bandwidth. So, I
> think it is straightforward to set throttling in tools layer. The 'psr-mba-set'
> is designed as a simple command to set what HW needs.
>
> Also, mentioned by SDM, "The throttling values exposed by MBA are approximate,
> and are calibrated to specific traffic patterns.". So, it is hard to provide
> exact bandwidth control in 'psr-mba-set'.
OK, I think I will wait until I see the example explained in order to
express my opinion on the proposed toolstack interface.
> > Also, IMHO you should provide a command to print the max throttling,
>
> The 'psr-hwinfo' can show the max throttling. Because it is part of MBA HW info.
>
> > remember that from Xen's PoV Dom0 is just another domain, and the
> > CPUID values reported to Dom0 don't need to be the same as found on
> > bare metal.
> >
> But the CPUID values got through 'psr' commands should be ones found on bare
> metal, right? Because these commands directly get the values from hypervisor
> through domctl/sysctl.
Yes, if they are provided by the hypervisor (ie: cpuid instruction is
executed in Xen, not Dom0).
> > > + if the MBA_MAX value is 90, the input precision is 10%. Values not an even
> > > + multiple of the precision (e.g., 12%) will be rounded down (e.g., to 10%
> > > + delay applied) by HW automatically.
> > > +
> > > + Non-linear mode: input delay values are powers-of-two from zero to the
> > > + MBA_MAX value from CPUID. In this case any values not a power of two will
> > > + be rounded down the next nearest power of two by HW automatically.
> > > +
> > > +# Technical details
> > > +
> > > +MBA is a member of Intel PSR features, it shares the base PSR infrastructure
> > > +in Xen.
> > > +
> > > +## Hardware perspective
> > > +
> > > + MBA defines a range of MSRs to support specifying a delay value (Thrtl) per
> > > + COS, with details below.
> > > +
> > > + ```
> > > + +----------------------------+----------------+
> > > + | MSR (per socket) | Address |
> > > + +----------------------------+----------------+
> > > + | IA32_L2_QOS_Ext_BW_Thrtl_0 | 0xD50 |
> > > + +----------------------------+----------------+
> > > + | ... | ... |
> > > + +----------------------------+----------------+
> > > + | IA32_L2_QOS_Ext_BW_Thrtl_n | 0xD50+n (n<64) |
> > > + +----------------------------+----------------+
> > > + ```
> >
> > Are you sure you want to hardcode this n<64? Isn't there a chance this
> > is going to be bumped in newer hardware?
> >
> This is just a HW limitation declared in SDM. In fact, there is no such hard
> code limitation. Hypervisor side checks the 'cos_max' got through CPUID.
Then I would remove the n<64, or else this will get out-of-sync
without anyone noticing.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
2017-08-30 5:23 ` Yi Sun
@ 2017-08-30 7:47 ` Roger Pau Monn�
2017-08-30 8:14 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monn� @ 2017-08-30 7:47 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Wed, Aug 30, 2017 at 01:23:08PM +0800, Yi Sun wrote:
> On 17-08-29 13:00:19, Roger Pau Monn� wrote:
> > On Thu, Aug 24, 2017 at 09:14:36AM +0800, Yi Sun wrote:
> > > --- a/xen/arch/x86/domctl.c
> > > +++ b/xen/arch/x86/domctl.c
> > > @@ -1439,60 +1439,60 @@ long arch_do_domctl(
> > > }
> > > break;
> > >
> > > - case XEN_DOMCTL_psr_cat_op:
> > > - switch ( domctl->u.psr_cat_op.cmd )
> > > + case XEN_DOMCTL_psr_alloc_op:
> > > + switch ( domctl->u.psr_alloc_op.cmd )
> > > {
> > > uint32_t val32;
> > >
> > > case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM:
> >
> > In order to match the name of the structure used, shouldn't those
> > defines be renamed to XEN_DOMCTL_PSR_ALLOC_* (instead of CAT)?
> >
> These names are feature specific. E.g:
> XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM - L3 CAT
> XEN_DOMCTL_PSR_MBA_OP_SET_THRTL - MBA
>
> Could you accept them? If not, I think I may write a separate patch to rename
> them.
Maybe I'm just lost, but in this patch you are renaming psr_cat_op to
psr_alloc_op, hence I think those defines should also be renamed in
this same patch.
I would also prefer that you don't introduce the '_OP_|_op_'
everywhere, I don't think it's useful/descriptive. Hypercalls are
operations by definition IMHO, hence the 'op' thing doesn't add
anything useful.
Ie: the struct should be psr_alloc, and the name
XEN_DOMCTL_psr_alloc.
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 03/15] x86: rename 'cbm_type' to 'psr_val_type' to make it general
2017-08-30 5:47 ` Yi Sun
@ 2017-08-30 7:51 ` Roger Pau Monn�
2017-08-30 8:14 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monn� @ 2017-08-30 7:51 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Wed, Aug 30, 2017 at 01:47:38PM +0800, Yi Sun wrote:
> On 17-08-29 13:15:24, Roger Pau Monn� wrote:
> > On Thu, Aug 24, 2017 at 09:14:37AM +0800, Yi Sun wrote:
> > > This patch renames 'cbm_type' to 'psr_val_type' to make it be general.
> >
> > I'm unsure whether the '_val_' is really needed here. AFAICT it
> > doesn't add any meaningful information to the define itself, and only
> > makes it longer. Apart from that it looks fine.
> >
> The 'cbm_type' means the CAT value (CBM) type. As MBA value type is throttle,
> I want to rename 'cbm_type' to a generic value type. Because there are many
> 'psr' starting names, to make it clear that this structure is for value type,
> I name it 'psr_val_type'.
But the enum also has the '_val' thing, and it's used by psr_get_info.
IMHO it would be better to just drop the '_val'.
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 04/15] x86: implement data structure and CPU init flow for MBA
2017-08-30 5:31 ` Yi Sun
@ 2017-08-30 7:55 ` Roger Pau Monn�
2017-08-30 8:19 ` Yi Sun
2017-08-30 8:45 ` Jan Beulich
0 siblings, 2 replies; 64+ messages in thread
From: Roger Pau Monn� @ 2017-08-30 7:55 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Wed, Aug 30, 2017 at 01:31:51PM +0800, Yi Sun wrote:
> On 17-08-29 14:44:32, Roger Pau Monn� wrote:
> > On Thu, Aug 24, 2017 at 09:14:38AM +0800, Yi Sun wrote:
> > > +#define MBA_LINEAR (1u << 2)
> >
> > Why is this shifted by 2?
> >
> This is used to mask ECX register to know if MBA linear mode is set or not.
Shouldn't this be in asm-x86/psr.h since it's part of the hardware
interface?
> > > + /* We reserve cos=0 as default thrtl (0) which means no delay. */
> > > + feat->cos_reg_val[0] = 0;
> > > + wrmsrl(MSR_IA32_PSR_MBA_MASK(0), 0);
> > > +
> > > + /* Add this feature into array. */
> > > + info->features[type] = feat;
> > > +
> > > + if ( !opt_cpu_info )
> > > + return 0;
> > > +
> > > + printk(XENLOG_INFO "MBA: enabled on socket %u, cos_max:%u,"
> > > + "thrtl_max:%u, linear:%u.\n",
> >
> > Please try to avoid splitting messages, it makes it hard to grep for
> > them afterward.
> >
> Ok, but that would exceed 80 characters. Is that acceptable?
That's fine for log messages. Please add a newline after XENLOG_INFO
and align the start of the string to the start of XENLOG_INFO.
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
2017-08-30 7:47 ` Roger Pau Monn�
@ 2017-08-30 8:14 ` Yi Sun
0 siblings, 0 replies; 64+ messages in thread
From: Yi Sun @ 2017-08-30 8:14 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On 17-08-30 08:47:42, Roger Pau Monn� wrote:
> On Wed, Aug 30, 2017 at 01:23:08PM +0800, Yi Sun wrote:
> > On 17-08-29 13:00:19, Roger Pau Monn� wrote:
> > > On Thu, Aug 24, 2017 at 09:14:36AM +0800, Yi Sun wrote:
> > > > --- a/xen/arch/x86/domctl.c
> > > > +++ b/xen/arch/x86/domctl.c
> > > > @@ -1439,60 +1439,60 @@ long arch_do_domctl(
> > > > }
> > > > break;
> > > >
> > > > - case XEN_DOMCTL_psr_cat_op:
> > > > - switch ( domctl->u.psr_cat_op.cmd )
> > > > + case XEN_DOMCTL_psr_alloc_op:
> > > > + switch ( domctl->u.psr_alloc_op.cmd )
> > > > {
> > > > uint32_t val32;
> > > >
> > > > case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM:
> > >
> > > In order to match the name of the structure used, shouldn't those
> > > defines be renamed to XEN_DOMCTL_PSR_ALLOC_* (instead of CAT)?
> > >
> > These names are feature specific. E.g:
> > XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM - L3 CAT
> > XEN_DOMCTL_PSR_MBA_OP_SET_THRTL - MBA
> >
> > Could you accept them? If not, I think I may write a separate patch to rename
> > them.
>
> Maybe I'm just lost, but in this patch you are renaming psr_cat_op to
> psr_alloc_op, hence I think those defines should also be renamed in
> this same patch.
>
> I would also prefer that you don't introduce the '_OP_|_op_'
> everywhere, I don't think it's useful/descriptive. Hypercalls are
> operations by definition IMHO, hence the 'op' thing doesn't add
> anything useful.
>
> Ie: the struct should be psr_alloc, and the name
> XEN_DOMCTL_psr_alloc.
>
Ok, I will modify these things per your suggestion.
> Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 03/15] x86: rename 'cbm_type' to 'psr_val_type' to make it general
2017-08-30 7:51 ` Roger Pau Monn�
@ 2017-08-30 8:14 ` Yi Sun
0 siblings, 0 replies; 64+ messages in thread
From: Yi Sun @ 2017-08-30 8:14 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On 17-08-30 08:51:40, Roger Pau Monn� wrote:
> On Wed, Aug 30, 2017 at 01:47:38PM +0800, Yi Sun wrote:
> > On 17-08-29 13:15:24, Roger Pau Monn� wrote:
> > > On Thu, Aug 24, 2017 at 09:14:37AM +0800, Yi Sun wrote:
> > > > This patch renames 'cbm_type' to 'psr_val_type' to make it be general.
> > >
> > > I'm unsure whether the '_val_' is really needed here. AFAICT it
> > > doesn't add any meaningful information to the define itself, and only
> > > makes it longer. Apart from that it looks fine.
> > >
> > The 'cbm_type' means the CAT value (CBM) type. As MBA value type is throttle,
> > I want to rename 'cbm_type' to a generic value type. Because there are many
> > 'psr' starting names, to make it clear that this structure is for value type,
> > I name it 'psr_val_type'.
>
> But the enum also has the '_val' thing, and it's used by psr_get_info.
> IMHO it would be better to just drop the '_val'.
>
Ok, will do it.
> Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 04/15] x86: implement data structure and CPU init flow for MBA
2017-08-30 7:55 ` Roger Pau Monn�
@ 2017-08-30 8:19 ` Yi Sun
2017-08-30 8:45 ` Jan Beulich
1 sibling, 0 replies; 64+ messages in thread
From: Yi Sun @ 2017-08-30 8:19 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On 17-08-30 08:55:42, Roger Pau Monn� wrote:
> On Wed, Aug 30, 2017 at 01:31:51PM +0800, Yi Sun wrote:
> > On 17-08-29 14:44:32, Roger Pau Monn� wrote:
> > > On Thu, Aug 24, 2017 at 09:14:38AM +0800, Yi Sun wrote:
> > > > +#define MBA_LINEAR (1u << 2)
> > >
> > > Why is this shifted by 2?
> > >
> > This is used to mask ECX register to know if MBA linear mode is set or not.
>
> Shouldn't this be in asm-x86/psr.h since it's part of the hardware
> interface?
>
This macro is only used in 'psr.c' so I defined it here.
> > > > + /* We reserve cos=0 as default thrtl (0) which means no delay. */
> > > > + feat->cos_reg_val[0] = 0;
> > > > + wrmsrl(MSR_IA32_PSR_MBA_MASK(0), 0);
> > > > +
> > > > + /* Add this feature into array. */
> > > > + info->features[type] = feat;
> > > > +
> > > > + if ( !opt_cpu_info )
> > > > + return 0;
> > > > +
> > > > + printk(XENLOG_INFO "MBA: enabled on socket %u, cos_max:%u,"
> > > > + "thrtl_max:%u, linear:%u.\n",
> > >
> > > Please try to avoid splitting messages, it makes it hard to grep for
> > > them afterward.
> > >
> > Ok, but that would exceed 80 characters. Is that acceptable?
>
> That's fine for log messages. Please add a newline after XENLOG_INFO
> and align the start of the string to the start of XENLOG_INFO.
>
Got it, thanks!
> Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 07/15] x86: implement set value flow for MBA
2017-08-24 1:14 ` [PATCH v2 07/15] x86: implement set value flow " Yi Sun
@ 2017-08-30 8:31 ` Roger Pau Monné
2017-08-31 2:20 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2017-08-30 8:31 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 24, 2017 at 09:14:41AM +0800, Yi Sun wrote:
> This patch implements set value flow for MBA including its callback
> function and domctl interface.
>
> It also changes the memebers in 'cos_write_info' to transfer the
> feature array, feature properties array and value array. Then, we
> can write all features values on the cos id into MSRs.
>
> Because multiple features may co-exist, we need handle all features to write
> values of them into a COS register with new COS ID. E.g:
> 1. L3 CAT and MBA co-exist.
> 2. Dom1 and Dom2 share a same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff,
> the MBA Thrtle of Dom1 is 0xa.
> 3. User wants to change MBA Thrtl of Dom1 to be 0x14. Because COS ID 2 is
> used by Dom2 too, we have to pick a new COS ID 3. The original values of
> Dom1 on COS ID 3 may be below:
What original values? You said you pick COS ID 3, because I think it's
assumed to be empty? In which case there are no original values in COS
ID 3.
> ---------
> | COS 3 |
> ---------
> L3 CAT | 0x7ff |
> ---------
> MBA | 0x0 |
> ---------
> 4. After setting, the L3 CAT CBM value of Dom1 should be kept and the new MBA
> Thrtl is set. So, the values on COS ID 3 should be below.
> ---------
> | COS 3 |
> ---------
> L3 CAT | 0x1ff |
> ---------
> MBA | 0x14 |
> ---------
>
> So, we should write all features values into their MSRs. That requires the
> feature array, feature properties array and value array are input.
^ as
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> ---
> v2:
> - remove linear mode 'thrtl_max' check in 'mba_check_thrtl' because it has
> been checked in 'mba_init_feature'.
> (suggested by Chao Peng)
> - for non-linear mode, check if '*thrtl' is not 0 in 'mba_check_thrtl'. If
> it is 0, we do not need to change it.
> (suggested by Chao Peng)
> - move comments to explain changes of 'cos_write_info' from psr.c to commit
> message.
> (suggested by Chao Peng)
> ---
> xen/arch/x86/domctl.c | 6 ++
> xen/arch/x86/psr.c | 150 ++++++++++++++++++++++++++++++--------------
> xen/include/public/domctl.h | 1 +
> 3 files changed, 109 insertions(+), 48 deletions(-)
>
> diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
> index 4936bcb..0ae4799 100644
> --- a/xen/arch/x86/domctl.c
> +++ b/xen/arch/x86/domctl.c
> @@ -1468,6 +1468,12 @@ long arch_do_domctl(
> PSR_VAL_TYPE_L2_CBM);
> break;
>
> + case XEN_DOMCTL_PSR_MBA_OP_SET_THRTL:
> + ret = psr_set_val(d, domctl->u.psr_alloc_op.target,
> + domctl->u.psr_alloc_op.data,
> + PSR_VAL_TYPE_MBA);
> + break;
> +
> case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM:
> ret = psr_get_val(d, domctl->u.psr_alloc_op.target,
> &val32, PSR_VAL_TYPE_L3_CBM);
> diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
> index 4a0c982..ce82975 100644
> --- a/xen/arch/x86/psr.c
> +++ b/xen/arch/x86/psr.c
> @@ -138,6 +138,12 @@ static const struct feat_props {
>
> /* write_msr is used to write out feature MSR register. */
> void (*write_msr)(unsigned int cos, uint32_t val, enum psr_val_type type);
> +
> + /*
> + * check_val is used to check if input val fulfills SDM requirement.
> + * Change it to valid value if SDM allows.
I'm not really sure it's a good idea to change the value to a valid
one, IMHO you should just check and print an error if the value is
invalid (and return false of course).
> + */
> + bool (*check_val)(const struct feat_node *feat, unsigned long *val);
> } *feat_props[FEAT_TYPE_NUM];
>
> /*
> @@ -275,29 +281,6 @@ static enum psr_feat_type psr_val_type_to_feat_type(enum psr_val_type type)
> return feat_type;
> }
>
> -static bool psr_check_cbm(unsigned int cbm_len, unsigned long cbm)
> -{
> - unsigned int first_bit, zero_bit;
> -
> - /* Set bits should only in the range of [0, cbm_len]. */
> - if ( cbm & (~0ul << cbm_len) )
> - return false;
> -
> - /* At least one bit need to be set. */
> - if ( cbm == 0 )
> - return false;
> -
> - first_bit = find_first_bit(&cbm, cbm_len);
> - zero_bit = find_next_zero_bit(&cbm, cbm_len, first_bit);
> -
> - /* Set bits should be contiguous. */
> - if ( zero_bit < cbm_len &&
> - find_next_bit(&cbm, cbm_len, zero_bit) < cbm_len )
> - return false;
> -
> - return true;
> -}
> -
> /* Implementation of allocation features' functions. */
> static int cat_init_feature(const struct cpuid_leaf *regs,
> struct feat_node *feat,
> @@ -433,6 +416,30 @@ static bool cat_get_feat_info(const struct feat_node *feat,
> return true;
> }
>
> +static bool cat_check_cbm(const struct feat_node *feat, unsigned long *cbm)
> +{
> + unsigned int first_bit, zero_bit;
> + unsigned int cbm_len = feat->cat_info.cbm_len;
> +
> + /* Set bits should only in the range of [0, cbm_len]. */
> + if ( *cbm & (~0ul << cbm_len) )
> + return false;
> +
> + /* At least one bit need to be set. */
> + if ( *cbm == 0 )
> + return false;
> +
> + first_bit = find_first_bit(cbm, cbm_len);
> + zero_bit = find_next_zero_bit(cbm, cbm_len, first_bit);
> +
> + /* Set bits should be contiguous. */
> + if ( zero_bit < cbm_len &&
> + find_next_bit(cbm, cbm_len, zero_bit) < cbm_len )
> + return false;
> +
> + return true;
> +}
> +
> /* L3 CAT props */
> static void l3_cat_write_msr(unsigned int cos, uint32_t val,
> enum psr_val_type type)
> @@ -446,6 +453,7 @@ static const struct feat_props l3_cat_props = {
> .alt_type = PSR_VAL_TYPE_UNKNOWN,
> .get_feat_info = cat_get_feat_info,
> .write_msr = l3_cat_write_msr,
> + .check_val = cat_check_cbm,
> };
Maybe the introduction of check_val should be a separate patch? It's
mostly code movement and some fixup.
> /* L3 CDP props */
> @@ -476,6 +484,7 @@ static const struct feat_props l3_cdp_props = {
> .alt_type = PSR_VAL_TYPE_L3_CBM,
> .get_feat_info = l3_cdp_get_feat_info,
> .write_msr = l3_cdp_write_msr,
> + .check_val = cat_check_cbm,
> };
>
> /* L2 CAT props */
> @@ -491,6 +500,7 @@ static const struct feat_props l2_cat_props = {
> .alt_type = PSR_VAL_TYPE_UNKNOWN,
> .get_feat_info = cat_get_feat_info,
> .write_msr = l2_cat_write_msr,
> + .check_val = cat_check_cbm,
> };
>
> /* MBA props */
> @@ -514,6 +524,40 @@ static bool mba_get_feat_info(const struct feat_node *feat,
> static void mba_write_msr(unsigned int cos, uint32_t val,
> enum psr_val_type type)
> {
> + wrmsrl(MSR_IA32_PSR_MBA_MASK(cos), val);
> +}
> +
> +static bool mba_check_thrtl(const struct feat_node *feat, unsigned long *thrtl)
> +{
> + if ( *thrtl > feat->mba_info.thrtl_max )
> + return false;
> +
> + /*
> + * Per SDM (chapter "Memory Bandwidth Allocation Configuration"):
> + * 1. Linear mode: In the linear mode the input precision is defined
> + * as 100-(MBA_MAX). For instance, if the MBA_MAX value is 90, the
> + * input precision is 10%. Values not an even multiple of the
> + * precision (e.g., 12%) will be rounded down (e.g., to 10% delay
> + * applied).
> + * 2. Non-linear mode: Input delay values are powers-of-two from zero
> + * to the MBA_MAX value from CPUID. In this case any values not a
> + * power of two will be rounded down the next nearest power of two.
> + */
> + if ( feat->mba_info.linear )
> + {
> + unsigned int mod;
> +
> + mod = *thrtl % (100 - feat->mba_info.thrtl_max);
> + *thrtl -= mod;
> + }
> + else
> + {
> + /* Not power of 2. */
> + if ( *thrtl && (*thrtl & (*thrtl - 1)) )
This can be joined with the else to avoid another indentation level:
else if ( *thrtl && (*thrtl & (*thrtl - 1)) )
...
> + *thrtl = *thrtl & (1 << (flsl(*thrtl) - 1));
> + }
> +
> + return true;
> }
>
> static const struct feat_props mba_props = {
> @@ -522,6 +566,7 @@ static const struct feat_props mba_props = {
> .alt_type = PSR_VAL_TYPE_UNKNOWN,
> .get_feat_info = mba_get_feat_info,
> .write_msr = mba_write_msr,
> + .check_val = mba_check_thrtl,
> };
>
> static void __init parse_psr_bool(char *s, char *value, char *feature,
> @@ -942,6 +987,7 @@ static int insert_val_into_array(uint32_t val[],
> const struct feat_node *feat;
> const struct feat_props *props;
> unsigned int i;
> + unsigned long check_val = new_val;
> int ret;
>
> ASSERT(feat_type < FEAT_TYPE_NUM);
> @@ -966,9 +1012,11 @@ static int insert_val_into_array(uint32_t val[],
> if ( array_len < props->cos_num )
> return -ENOSPC;
>
> - if ( !psr_check_cbm(feat->cat_info.cbm_len, new_val) )
> + if ( !props->check_val(feat, &check_val) )
> return -EINVAL;
>
> + new_val = check_val;
> +
> /*
> * Value setting position is same as feature array.
> * For CDP, user may set both DATA and CODE to same value. For such case,
> @@ -1198,25 +1246,42 @@ static unsigned int get_socket_cpu(unsigned int socket)
> struct cos_write_info
> {
> unsigned int cos;
> - struct feat_node *feature;
> + struct feat_node **features;
> const uint32_t *val;
> - const struct feat_props *props;
> + unsigned int array_len;
> + const struct feat_props **props;
> };
>
> static void do_write_psr_msrs(void *data)
> {
> const struct cos_write_info *info = data;
> - struct feat_node *feat = info->feature;
> - const struct feat_props *props = info->props;
> - unsigned int i, cos = info->cos, cos_num = props->cos_num;
> + unsigned int i, j, index = 0, array_len = info->array_len, cos = info->cos;
> + const uint32_t *val_array = info->val;
>
> - for ( i = 0; i < cos_num; i++ )
> + for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
> {
index and j can be defined here, they are only used inside of this for
loop AFAICT.
> - if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
> + struct feat_node *feat = info->features[i];
> + const struct feat_props *props = info->props[i];
> + unsigned int cos_num;
> +
> + if ( !feat || !props )
> + continue;
> +
> + cos_num = props->cos_num;
> + if ( array_len < cos_num )
> + return;
> +
> + for ( j = 0; j < cos_num; j++ )
> {
> - feat->cos_reg_val[cos * cos_num + i] = info->val[i];
> - props->write_msr(cos, info->val[i], props->type[i]);
> + if ( feat->cos_reg_val[cos * cos_num + j] != val_array[index + j] )
> + {
> + feat->cos_reg_val[cos * cos_num + j] = val_array[index + j];
> + props->write_msr(cos, val_array[index + j], props->type[j]);
> + }
> }
> +
> + array_len -= cos_num;
> + index += cos_num;
> }
> }
>
> @@ -1224,30 +1289,19 @@ static int write_psr_msrs(unsigned int socket, unsigned int cos,
> const uint32_t val[], unsigned int array_len,
> enum psr_feat_type feat_type)
> {
> - int ret;
> struct psr_socket_info *info = get_socket_info(socket);
info should probably be const here.
> struct cos_write_info data =
> {
> .cos = cos,
> - .feature = info->features[feat_type],
> - .props = feat_props[feat_type],
> + .features = info->features,
> + .val = val,
> + .array_len = array_len,
> + .props = feat_props,
> };
AFAICT data should also be const, but I guess this is not going to
work because on_selected_cpus expects a non-const payload?
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 08/15] tools: create general interfaces to support psr allocation features
2017-08-24 1:14 ` [PATCH v2 08/15] tools: create general interfaces to support psr allocation features Yi Sun
@ 2017-08-30 8:42 ` Roger Pau Monné
2017-08-31 2:38 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2017-08-30 8:42 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 24, 2017 at 09:14:42AM +0800, Yi Sun wrote:
> This patch creates general interfaces in libxl to support all psr
> allocation features.
>
> Add 'LIBXL_HAVE_PSR_MBA' to indicate interface change.
I'm not sure this is enough to cover the changes you are doing here:
you are introducing some MBA stuff, plus a kind of generic interface
for PSR.
I think this should be split into two patches, the first one adding
the generic interface, and the second one adding the MBA stuff.
> Please note, the functionality cannot work until later patches
> are applied.
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> ---
> v2:
> - remove '_INFO' in 'libxl_psr_feat_type' and make corresponding
> changes in 'libxl_psr_hw_info'.
> (suggested by Chao Peng)
> ---
> tools/libxl/libxl.h | 35 ++++++++++++++++++++++++++++++++++-
> tools/libxl/libxl_psr.c | 25 +++++++++++++++++++++++++
> tools/libxl/libxl_types.idl | 22 ++++++++++++++++++++++
> 3 files changed, 81 insertions(+), 1 deletion(-)
>
> diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> index 229e289..c1d804c 100644
> --- a/tools/libxl/libxl.h
> +++ b/tools/libxl/libxl.h
> @@ -931,6 +931,13 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const libxl_mac *src);
> #define LIBXL_HAVE_PSR_L2_CAT 1
>
> /*
> + * LIBXL_HAVE_PSR_MBA
> + *
> + * If this is defined, the Memory Bandwidth Allocation feature is supported.
> + */
> +#define LIBXL_HAVE_PSR_MBA 1
> +
> +/*
> * LIBXL_HAVE_MCA_CAPS
> *
> * If this is defined, setting MCA capabilities for HVM domain is supported.
> @@ -2219,7 +2226,33 @@ int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
> int libxl_psr_cat_get_l3_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
> int *nr);
> void libxl_psr_cat_info_list_free(libxl_psr_cat_info *list, int nr);
> -#endif
> +
> +#ifdef LIBXL_HAVE_PSR_MBA
You don't need this, this is only for consumers of libxl. It is
perfectly fine to have the prototypes of the functions, even if
consumers don't use them.
> +/*
> + * Function to set a domain's value. It operates on a single or multiple
> + * target(s) defined in 'target_map'. 'target_map' specifies all the sockets
> + * to be operated on.
> + */
> +int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
> + libxl_psr_cbm_type type, libxl_bitmap *target_map,
> + uint64_t val);
> +/*
> + * Function to get a domain's cbm. It operates on a single 'target'.
> + * 'target' specifies which socket to be operated on.
> + */
> +int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
> + libxl_psr_cbm_type type, unsigned int target,
> + uint64_t *val);
> +/*
> + * On success, the function returns an array of elements in 'info',
> + * and the length in 'nr'.
> + */
> +int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_hw_info **info,
> + unsigned int *nr, libxl_psr_feat_type type,
> + unsigned int lvl);
> +void libxl_psr_hw_info_list_free(libxl_psr_hw_info *list, unsigned int nr);
> +#endif /* LIBXL_HAVE_PSR_MBA */
> +#endif /* LIBXL_HAVE_PSR_CAT */
Please send a patch to remove the already existing ifdef PSR
pollution.
>
> /* misc */
>
> diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
> index f55ba1e..cf368ba 100644
> --- a/tools/libxl/libxl_psr.c
> +++ b/tools/libxl/libxl_psr.c
> @@ -425,6 +425,31 @@ void libxl_psr_cat_info_list_free(libxl_psr_cat_info *list, int nr)
> free(list);
> }
>
> +int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
> + libxl_psr_cbm_type type, libxl_bitmap *target_map,
> + uint64_t val)
> +{
> + return ERROR_FAIL;
> +}
> +
> +int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
> + libxl_psr_cbm_type type, unsigned int target,
> + uint64_t *val)
> +{
> + return ERROR_FAIL;
> +}
> +
> +int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_hw_info **info,
> + unsigned int *nr, libxl_psr_feat_type type,
> + unsigned int lvl)
> +{
> + return ERROR_FAIL;
> +}
> +
> +void libxl_psr_hw_info_list_free(libxl_psr_hw_info *list, unsigned int nr)
> +{
> +}
> +
> /*
> * Local variables:
> * mode: C
> diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> index 6e80d36..ab847f8 100644
> --- a/tools/libxl/libxl_types.idl
> +++ b/tools/libxl/libxl_types.idl
> @@ -977,6 +977,7 @@ libxl_psr_cbm_type = Enumeration("psr_cbm_type", [
> (2, "L3_CBM_CODE"),
> (3, "L3_CBM_DATA"),
> (4, "L2_CBM"),
> + (5, "MBA_THRTL"),
Is this really a CBM type?
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 04/15] x86: implement data structure and CPU init flow for MBA
2017-08-30 7:55 ` Roger Pau Monn�
2017-08-30 8:19 ` Yi Sun
@ 2017-08-30 8:45 ` Jan Beulich
1 sibling, 0 replies; 64+ messages in thread
From: Jan Beulich @ 2017-08-30 8:45 UTC (permalink / raw)
To: Roger Pau Monn�, Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, xen-devel, chao.p.peng
>>> On 30.08.17 at 09:55, <roger.pau@citrix.com> wrote:
> On Wed, Aug 30, 2017 at 01:31:51PM +0800, Yi Sun wrote:
>> On 17-08-29 14:44:32, Roger Pau Monn wrote:
>> > On Thu, Aug 24, 2017 at 09:14:38AM +0800, Yi Sun wrote:
>> > > +#define MBA_LINEAR (1u << 2)
>> >
>> > Why is this shifted by 2?
>> >
>> This is used to mask ECX register to know if MBA linear mode is set or not.
>
> Shouldn't this be in asm-x86/psr.h since it's part of the hardware
> interface?
No, asm-x86/psr.h should have things consumers of the functionality
provided by psr.c need, nothing else. psr.c itself acts as the interface
to the hardware, so any parts of the hardware interface not needing
further exposure should be declared locally to that file.
That said, I think the chosen name here is not descriptive of its
purpose (i.e. this is an example where the name is actually too
short; I'm mentioning this because we had the opposite example
elsewhere in the series).
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 09/15] tools: implement the new libxc get hw info interface
2017-08-24 1:14 ` [PATCH v2 09/15] tools: implement the new libxc get hw info interface Yi Sun
@ 2017-08-30 8:58 ` Roger Pau Monné
2017-08-31 3:05 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2017-08-30 8:58 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 24, 2017 at 09:14:43AM +0800, Yi Sun wrote:
> This patch implements a new libxc get hw info interface and corresponding
> data structures. It also changes libxl_psr.c to call this new interface.
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> ---
> v2:
> - split this patch out from a big patch in v1.
> (suggested by Wei Liu)
> - change 'CAT_INFO' and 'MBA_INFO' to 'CAT' and 'MBA'.
> (suggested by Chao Peng)
> ---
> tools/libxc/include/xenctrl.h | 30 +++++++++++++++++++++++++---
> tools/libxc/xc_psr.c | 46 ++++++++++++++++++++++++++++++++-----------
> tools/libxl/libxl_psr.c | 34 ++++++++++++++++++++++++++++++--
> 3 files changed, 93 insertions(+), 17 deletions(-)
>
> diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
> index c7710b8..81a6f48 100644
> --- a/tools/libxc/include/xenctrl.h
> +++ b/tools/libxc/include/xenctrl.h
> @@ -2458,6 +2458,31 @@ enum xc_psr_cat_type {
> };
> typedef enum xc_psr_cat_type xc_psr_cat_type;
>
> +enum xc_psr_feat_type {
> + XC_PSR_FEAT_UNKNOWN,
> + XC_PSR_FEAT_CAT_L3,
> + XC_PSR_FEAT_CAT_L2,
> + XC_PSR_FEAT_MBA,
> +};
> +typedef enum xc_psr_feat_type xc_psr_feat_type;
> +
> +struct xc_psr_hw_info {
> + union {
> + struct {
> + uint32_t cos_max;
> + uint32_t cbm_len;
> + bool cdp_enabled;
> + } xc_cat_info;
> +
> + struct {
> + uint32_t cos_max;
> + uint32_t thrtl_max;
> + bool linear;
> + } xc_mba_info;
No need for the _info suffix. The type itself already has info in it's
name.
> + } u;
> +};
> +typedef struct xc_psr_hw_info xc_psr_hw_info;
> +
> int xc_psr_cmt_attach(xc_interface *xch, uint32_t domid);
> int xc_psr_cmt_detach(xc_interface *xch, uint32_t domid);
> int xc_psr_cmt_get_domain_rmid(xc_interface *xch, uint32_t domid,
> @@ -2479,9 +2504,8 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
> int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
> xc_psr_cat_type type, uint32_t target,
> uint64_t *data);
> -int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
> - uint32_t *cos_max, uint32_t *cbm_len,
> - bool *cdp_enabled);
> +int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
> + xc_psr_feat_type type, xc_psr_hw_info *hw_info);
>
> int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
> int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
> diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
> index 73d05f2..ba412e4 100644
> --- a/tools/libxc/xc_psr.c
> +++ b/tools/libxc/xc_psr.c
> @@ -323,36 +323,58 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
> return rc;
> }
>
> -int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
> - uint32_t *cos_max, uint32_t *cbm_len, bool *cdp_enabled)
> +int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
> + xc_psr_feat_type type, xc_psr_hw_info *hw_info)
> {
> int rc = -1;
> DECLARE_SYSCTL;
>
> + if ( !hw_info )
> + return rc;
You should set errno = EINVAL here.
> +
> sysctl.cmd = XEN_SYSCTL_psr_alloc_op;
> sysctl.u.psr_alloc_op.target = socket;
>
> - switch ( lvl )
> + switch ( type )
> {
> - case 2:
> + case XC_PSR_FEAT_CAT_L2:
> sysctl.u.psr_alloc_op.cmd = XEN_SYSCTL_PSR_CAT_get_l2_info;
> rc = xc_sysctl(xch, &sysctl);
> if ( !rc )
> {
> - *cos_max = sysctl.u.psr_alloc_op.u.cat_info.cos_max;
> - *cbm_len = sysctl.u.psr_alloc_op.u.cat_info.cbm_len;
> - *cdp_enabled = false;
> + hw_info->u.xc_cat_info.cos_max =
> + sysctl.u.psr_alloc_op.u.cat_info.cos_max;
> + hw_info->u.xc_cat_info.cbm_len =
> + sysctl.u.psr_alloc_op.u.cat_info.cbm_len;
> + hw_info->u.xc_cat_info.cdp_enabled = false;
> }
> break;
> - case 3:
> + case XC_PSR_FEAT_CAT_L3:
> sysctl.u.psr_alloc_op.cmd = XEN_SYSCTL_PSR_CAT_get_l3_info;
> rc = xc_sysctl(xch, &sysctl);
> if ( !rc )
> {
> - *cos_max = sysctl.u.psr_alloc_op.u.cat_info.cos_max;
> - *cbm_len = sysctl.u.psr_alloc_op.u.cat_info.cbm_len;
> - *cdp_enabled = sysctl.u.psr_alloc_op.u.cat_info.flags &
> - XEN_SYSCTL_PSR_CAT_L3_CDP;
> + hw_info->u.xc_cat_info.cos_max =
> + sysctl.u.psr_alloc_op.u.cat_info.cos_max;
> + hw_info->u.xc_cat_info.cbm_len =
> + sysctl.u.psr_alloc_op.u.cat_info.cbm_len;
> + hw_info->u.xc_cat_info.cdp_enabled =
> + sysctl.u.psr_alloc_op.u.cat_info.flags &
> + XEN_SYSCTL_PSR_CAT_L3_CDP;
> + }
> + break;
> + case XC_PSR_FEAT_MBA:
> + sysctl.u.psr_alloc_op.cmd = XEN_SYSCTL_PSR_MBA_get_info;
> + rc = xc_sysctl(xch, &sysctl);
> + if ( !rc )
> + {
> + hw_info->u.xc_mba_info.cos_max =
> + sysctl.u.psr_alloc_op.u.mba_info.cos_max;
> + hw_info->u.xc_mba_info.thrtl_max =
> + sysctl.u.psr_alloc_op.u.mba_info.thrtl_max;
> + hw_info->u.xc_mba_info.linear =
> + sysctl.u.psr_alloc_op.u.mba_info.flags &
> + XEN_SYSCTL_PSR_MBA_LINEAR;
> }
> break;
> default:
> diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
> index cf368ba..b183305 100644
> --- a/tools/libxl/libxl_psr.c
> +++ b/tools/libxl/libxl_psr.c
> @@ -361,6 +361,27 @@ int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
> return rc;
> }
>
> +static inline xc_psr_feat_type libxl__psr_feat_type_to_libxc_psr_feat_type(
No inline please. In any case this is not performance critical code, so
let the compiler decide.
And the function name could be shorter, maybe:
libxl__psr_type_to_libxc_type
Or is this going to clash with some other translation function?
> + libxl_psr_feat_type type, unsigned int lvl)
> +{
> + xc_psr_feat_type xc_type = XC_PSR_FEAT_UNKNOWN;
> +
> + switch (type) {
> + case LIBXL_PSR_FEAT_TYPE_CAT:
> + if (lvl == 3)
> + xc_type = XC_PSR_FEAT_CAT_L3;
> + if (lvl == 2)
> + xc_type = XC_PSR_FEAT_CAT_L2;
> + break;
> + case LIBXL_PSR_FEAT_TYPE_MBA:
> + xc_type = XC_PSR_FEAT_MBA;
> + default:
> + break;
> + }
> +
> + return xc_type;
> +}
> +
> int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
> int *nr, unsigned int lvl)
> {
> @@ -369,6 +390,8 @@ int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
> int i = 0, socketid, nr_sockets;
> libxl_bitmap socketmap;
> libxl_psr_cat_info *ptr;
> + xc_psr_hw_info hw_info;
> + xc_psr_feat_type xc_type;
>
> libxl_bitmap_init(&socketmap);
>
> @@ -385,16 +408,23 @@ int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
> goto out;
> }
>
> + xc_type = libxl__psr_feat_type_to_libxc_psr_feat_type(
> + LIBXL_PSR_FEAT_TYPE_CAT, lvl);
Shouldn't you check that xc_type != XC_PSR_FEAT_UNKNOWN here?
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 10/15] tools: implement the new libxl get hw info interface
2017-08-24 1:14 ` [PATCH v2 10/15] tools: implement the new libxl " Yi Sun
@ 2017-08-30 9:15 ` Roger Pau Monné
2017-08-31 3:16 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2017-08-30 9:15 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 24, 2017 at 09:14:44AM +0800, Yi Sun wrote:
> This patch implements the new libxl get hw info interface,
> 'libxl_psr_get_hw_info', which is suitable to all psr allocation
> features. It also implements corresponding list free function,
> 'libxl_psr_hw_info_list_free' and make 'libxl_psr_cat_get_info' to call
> 'libxl_psr_get_hw_info' to avoid redundant codes in libxl_psr.c.
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> ---
> v2:
> - split this patch out from a big patch in v1.
> (suggested by Wei Liu)
> - change 'CAT_INFO'/'MBA_INFO' to 'CAT' and 'MBA. Also the libxl structure
> name 'cat_info'/'mba_info' is changed to 'cat'/'mba'.
> (suggested by Chao Peng)
> - call 'libxl_psr_hw_info_list_free' in 'libxl_psr_cat_get_info' to free
> allocated resources.
> (suggested by Chao Peng)
> ---
> tools/libxl/libxl_psr.c | 145 +++++++++++++++++++++++++++++++++++++-----------
> 1 file changed, 112 insertions(+), 33 deletions(-)
>
> diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
> index b183305..d7da7d7 100644
> --- a/tools/libxl/libxl_psr.c
> +++ b/tools/libxl/libxl_psr.c
> @@ -382,56 +382,51 @@ static inline xc_psr_feat_type libxl__psr_feat_type_to_libxc_psr_feat_type(
> return xc_type;
> }
>
> +static inline int libxl__psr_hw_info_to_libxl_psr_cat_info(
No inline. Maybe you could try to shorter the name?
> + libxl_psr_feat_type type, libxl_psr_hw_info *hw_info,
> + libxl_psr_cat_info *cat_info)
> +{
> + if (type != LIBXL_PSR_FEAT_TYPE_CAT)
> + return ERROR_INVAL;
> +
> + cat_info->id = hw_info->id;
> + cat_info->cos_max = hw_info->u.cat.cos_max;
> + cat_info->cbm_len = hw_info->u.cat.cbm_len;
> + cat_info->cdp_enabled = hw_info->u.cat.cdp_enabled;
> +
> + return 0;
> +}
> +
> int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
> int *nr, unsigned int lvl)
> {
> GC_INIT(ctx);
> int rc;
> - int i = 0, socketid, nr_sockets;
> - libxl_bitmap socketmap;
> + unsigned int i;
> + libxl_psr_hw_info *hw_info;
> libxl_psr_cat_info *ptr;
> - xc_psr_hw_info hw_info;
> - xc_psr_feat_type xc_type;
> -
> - libxl_bitmap_init(&socketmap);
> -
> - rc = libxl__count_physical_sockets(gc, &nr_sockets);
> - if (rc) {
> - LOGE(ERROR, "failed to get system socket count");
> - goto out;
> - }
>
> - libxl_socket_bitmap_alloc(ctx, &socketmap, nr_sockets);
> - rc = libxl_get_online_socketmap(ctx, &socketmap);
> - if (rc < 0) {
> - LOGE(ERROR, "failed to get available sockets");
> + rc = libxl_psr_get_hw_info(ctx, &hw_info, (unsigned int *)nr,
Is there any reason nr is int instead of unsigned int?
I would rather avoid casting things. Since this interface has not been
present in a release yet, could you please send a separate patch to
fix this if nr has no reason to be signed?
> + LIBXL_PSR_FEAT_TYPE_CAT, lvl);
> + if (rc)
> goto out;
> - }
>
> - xc_type = libxl__psr_feat_type_to_libxc_psr_feat_type(
> - LIBXL_PSR_FEAT_TYPE_CAT, lvl);
> + ptr = libxl__malloc(NOGC, *nr * sizeof(libxl_psr_cat_info));
>
> - ptr = libxl__malloc(NOGC, nr_sockets * sizeof(libxl_psr_cat_info));
> -
> - libxl_for_each_set_bit(socketid, socketmap) {
> - ptr[i].id = socketid;
> - if (xc_psr_get_hw_info(ctx->xch, socketid, xc_type, &hw_info)) {
> + for (i = 0; i < *nr; i++) {
> + if (libxl__psr_hw_info_to_libxl_psr_cat_info(
> + LIBXL_PSR_FEAT_TYPE_CAT,
> + &hw_info[i], &ptr[i])) {
> + libxl_psr_hw_info_list_free(hw_info, (unsigned int)*nr);
> rc = ERROR_FAIL;
> free(ptr);
> goto out;
> }
> -
> - ptr[i].cos_max = hw_info.u.xc_cat_info.cos_max;
> - ptr[i].cbm_len = hw_info.u.xc_cat_info.cbm_len;
> - ptr[i].cdp_enabled = hw_info.u.xc_cat_info.cdp_enabled;
> -
> - i++;
> }
>
> *info = ptr;
> - *nr = i;
> + libxl_psr_hw_info_list_free(hw_info, (unsigned int)*nr);
> out:
> - libxl_bitmap_dispose(&socketmap);
> GC_FREE;
> return rc;
> }
> @@ -469,15 +464,99 @@ int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
> return ERROR_FAIL;
> }
>
> +static inline int libxc__psr_hw_info_to_libxl_psr_hw_info(
No inline. Again shorter names would be better (although I understand
this might not be possible).
Also, why are you adding a libxc__ prefixed function to libxl?
> + libxl_psr_feat_type type, xc_psr_hw_info *xc_hw_info,
> + libxl_psr_hw_info *xl_hw_info)
you could drop the '_hw' in the parameter names, so all the
assignments below would fit on a single line.
> +{
> + switch (type) {
> + case LIBXL_PSR_FEAT_TYPE_CAT:
> + xl_hw_info->u.cat.cos_max = xc_hw_info->u.xc_cat_info.cos_max;
> + xl_hw_info->u.cat.cbm_len = xc_hw_info->u.xc_cat_info.cbm_len;
> + xl_hw_info->u.cat.cdp_enabled =
> + xc_hw_info->u.xc_cat_info.cdp_enabled;
> + break;
> + case LIBXL_PSR_FEAT_TYPE_MBA:
> + xl_hw_info->u.mba.cos_max = xc_hw_info->u.xc_mba_info.cos_max;
> + xl_hw_info->u.mba.thrtl_max = xc_hw_info->u.xc_mba_info.thrtl_max;
> + xl_hw_info->u.mba.linear = xc_hw_info->u.xc_mba_info.linear;
> + break;
> + default:
> + return ERROR_INVAL;
> + }
> +
> + return 0;
> +}
> +
> int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_hw_info **info,
> unsigned int *nr, libxl_psr_feat_type type,
> unsigned int lvl)
> {
> - return ERROR_FAIL;
> + GC_INIT(ctx);
> + int rc, nr_sockets;
> + unsigned int i = 0, socketid;
> + libxl_bitmap socketmap;
> + libxl_psr_hw_info *ptr;
> + xc_psr_feat_type xc_type;
> + xc_psr_hw_info hw_info;
> +
> + libxl_bitmap_init(&socketmap);
> +
> + if (type == LIBXL_PSR_FEAT_TYPE_CAT && lvl != 3 && lvl != 2) {
> + LOGE(ERROR, "input lvl %d is wrong!\n", lvl);
LOGE is used when errno is set, which I don't think it's the case
here. Please use plain LOG.
> + rc = ERROR_FAIL;
> + goto out;
> + }
> +
> + xc_type = libxl__psr_feat_type_to_libxc_psr_feat_type(type, lvl);
Isn't the above function going to return and invalid type if the
feature is CAT and the level is not correct? In which case you could
remove the above if and just check that the returned type here is not
invalid?
> +
> + rc = libxl__count_physical_sockets(gc, &nr_sockets);
> + if (rc) {
> + LOGE(ERROR, "failed to get system socket count");
Again, libxl__ functions don't set errno. In this case this might be
fine, because libxl__count_physical_sockets calls into libxc which
sets errno, but you should only use LOGE when reporting errors from
system calls or libxc functions.
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 11/15] tools: implement the new xl get hw info interface
2017-08-24 1:14 ` [PATCH v2 11/15] tools: implement the new xl " Yi Sun
@ 2017-08-30 9:23 ` Roger Pau Monné
2017-08-31 5:57 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2017-08-30 9:23 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 24, 2017 at 09:14:45AM +0800, Yi Sun wrote:
> This patch implements a new xl get HW info interface. A new argument
> is added for psr-hwinfo command to get and show MBA HW info.
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> ---
> v2:
> - split out this patch from a big patch in v1.
> (suggested by Wei Liu)
> - change 'MBA_INFO' to 'MBA'. Also, change 'mba_info' to 'mba'.
> (suggested by Chao Peng)
> ---
> tools/xl/xl_cmdtable.c | 1 +
> tools/xl/xl_psr.c | 39 +++++++++++++++++++++++++++++++++++++--
> 2 files changed, 38 insertions(+), 2 deletions(-)
>
> diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
> index 2c71a9f..5ac8a7e 100644
> --- a/tools/xl/xl_cmdtable.c
> +++ b/tools/xl/xl_cmdtable.c
> @@ -524,6 +524,7 @@ struct cmd_spec cmd_table[] = {
> "[options]",
> "-m, --cmt Show Cache Monitoring Technology (CMT) hardware info\n"
> "-a, --cat Show Cache Allocation Technology (CAT) hardware info\n"
> + "-b, --mba Show Memory Bandwidth Allocation (MBA) hardware info\n"
> },
> { "psr-cmt-attach",
> &main_psr_cmt_attach, 0, 1,
> diff --git a/tools/xl/xl_psr.c b/tools/xl/xl_psr.c
> index 7309d4f..ea37967 100644
> --- a/tools/xl/xl_psr.c
> +++ b/tools/xl/xl_psr.c
> @@ -479,6 +479,33 @@ static int psr_l2_cat_hwinfo(void)
> return rc;
> }
>
> +static int psr_mba_hwinfo(void)
> +{
> + int rc;
> + unsigned int i, nr;
> + libxl_psr_hw_info *info;
> +
> + rc = libxl_psr_get_hw_info(ctx, &info, &nr,
> + LIBXL_PSR_FEAT_TYPE_MBA, 0);
> + if (rc)
> + return rc;
> +
> + printf("Memory Bandwidth Allocation (MBA):\n");
> +
> + for (i = 0; i < nr; i++) {
> + printf("%-16s: %u\n", "Socket ID", info[i].id);
> + printf("%-16s: %s\n", "Linear Mode",
> + info[i].u.mba.linear ? "Enabled" : "Disabled");
> + printf("%-16s: %u\n", "Maximum COS", info[i].u.mba.cos_max);
> + printf("%-16s: %u\n", "Maximum Throttling Value",
> + info[i].u.mba.thrtl_max);
> + printf("%-16s: %u\n", "Default Throttling Value", 0);
If you really want to left-justify, shouldn't you choose a value that
aligns everything nicely (strlen("Default Throttling Valu") is
already greater than 16).
In fact you can do the alignment manually in the format string, and
avoid passing the name as the first parameter.
> + }
> +
> + libxl_psr_hw_info_list_free(info, nr);
> + return rc;
> +}
> +
> int main_psr_cat_cbm_set(int argc, char **argv)
> {
> uint32_t domid;
> @@ -597,20 +624,24 @@ int main_psr_cat_show(int argc, char **argv)
> int main_psr_hwinfo(int argc, char **argv)
> {
> int opt, ret = 0;
> - bool all = true, cmt = false, cat = false;
> + bool all = true, cmt = false, cat = false, mba = false;
> static struct option opts[] = {
const?
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 12/15] tools: rename 'xc_psr_cat_type' to 'xc_psr_val_type'
2017-08-24 1:14 ` [PATCH v2 12/15] tools: rename 'xc_psr_cat_type' to 'xc_psr_val_type' Yi Sun
@ 2017-08-30 9:24 ` Roger Pau Monné
0 siblings, 0 replies; 64+ messages in thread
From: Roger Pau Monné @ 2017-08-30 9:24 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 24, 2017 at 09:14:46AM +0800, Yi Sun wrote:
> This patch renames 'xc_psr_cat_type' to 'xc_psr_val_type' so that
> the structure name is common for all allocation features.
xc_psr_type would be fine by me.
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 14/15] tools: implement new generic set value interface and MBA set value command
2017-08-24 1:14 ` [PATCH v2 14/15] tools: implement new generic set value interface and MBA set " Yi Sun
@ 2017-08-30 9:47 ` Roger Pau Monné
2017-08-31 5:58 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2017-08-30 9:47 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 24, 2017 at 09:14:48AM +0800, Yi Sun wrote:
> int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
> @@ -457,7 +431,33 @@ int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
> libxl_psr_cbm_type type, libxl_bitmap *target_map,
> uint64_t val)
> {
> - return ERROR_FAIL;
> + GC_INIT(ctx);
> + int rc;
> + int socketid, nr_sockets;
> +
> + rc = libxl__count_physical_sockets(gc, &nr_sockets);
> + if (rc) {
> + LOGED(ERROR, domid, "failed to get system socket count");
> + goto out;
> + }
> +
> + libxl_for_each_set_bit(socketid, *target_map) {
> + xc_psr_val_type xc_type;
> +
> + if (socketid >= nr_sockets)
> + break;
> +
> + xc_type = libxl__psr_cbm_type_to_libxc_psr_val_type(type);
> + if (xc_psr_set_domain_data(ctx->xch, domid, xc_type,
> + socketid, val)) {
> + libxl__psr_alloc_log_err_msg(gc, errno, type);
> + rc = ERROR_FAIL;
> + }
> + }
> +
> +out:
> + GC_FREE;
> + return rc;
Is there no libxl_psr_get_val?
> }
>
> int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
> diff --git a/tools/xl/xl.h b/tools/xl/xl.h
> index 59065c4..2cbe89c 100644
> --- a/tools/xl/xl.h
> +++ b/tools/xl/xl.h
> @@ -206,6 +206,7 @@ int main_psr_cmt_show(int argc, char **argv);
> #ifdef LIBXL_HAVE_PSR_CAT
> int main_psr_cat_cbm_set(int argc, char **argv);
> int main_psr_cat_show(int argc, char **argv);
> +int main_psr_mba_set(int argc, char **argv);
> int main_psr_mba_show(int argc, char **argv);
> #endif
> int main_qemu_monitor_command(int argc, char **argv);
> diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
> index 19e5004..5f7122d 100644
> --- a/tools/xl/xl_cmdtable.c
> +++ b/tools/xl/xl_cmdtable.c
> @@ -562,6 +562,12 @@ struct cmd_spec cmd_table[] = {
> "[options] <Domain>",
> "-l <level> Specify the cache level to process, otherwise L3 cache is processed\n"
> },
> + { "psr-mba-set",
> + &main_psr_mba_set, 0, 1,
> + "Set throttling value (THRTL) for a domain",
> + "[options] <Domain> <THRTL>",
> + "-s <socket> Specify the socket to process, otherwise all sockets are processed\n"
> + },
> { "psr-mba-show",
> &main_psr_mba_show, 0, 1,
> "Show Memory Bandwidth Allocation information",
> diff --git a/tools/xl/xl_psr.c b/tools/xl/xl_psr.c
> index a97dafa..fc5bfcd 100644
> --- a/tools/xl/xl_psr.c
> +++ b/tools/xl/xl_psr.c
> @@ -554,6 +554,61 @@ int main_psr_mba_show(int argc, char **argv)
> return psr_val_show(domid, LIBXL_PSR_FEAT_TYPE_MBA, 0);
> }
>
> +int main_psr_mba_set(int argc, char **argv)
> +{
> + uint32_t domid;
> + libxl_psr_cbm_type type;
> + uint64_t thrtl;
> + int ret, opt = 0;
> + libxl_bitmap target_map;
> + char *value;
> + libxl_string_list socket_list;
> + unsigned long start, end;
> + unsigned int i, j, len;
> +
> + static struct option opts[] = {
const
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 07/15] x86: implement set value flow for MBA
2017-08-30 8:31 ` Roger Pau Monné
@ 2017-08-31 2:20 ` Yi Sun
2017-08-31 8:30 ` Roger Pau Monn�
0 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-31 2:20 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On 17-08-30 09:31:04, Roger Pau Monn� wrote:
> On Thu, Aug 24, 2017 at 09:14:41AM +0800, Yi Sun wrote:
> > It also changes the memebers in 'cos_write_info' to transfer the
> > feature array, feature properties array and value array. Then, we
> > can write all features values on the cos id into MSRs.
> >
> > Because multiple features may co-exist, we need handle all features to write
> > values of them into a COS register with new COS ID. E.g:
> > 1. L3 CAT and MBA co-exist.
> > 2. Dom1 and Dom2 share a same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff,
> > the MBA Thrtle of Dom1 is 0xa.
> > 3. User wants to change MBA Thrtl of Dom1 to be 0x14. Because COS ID 2 is
> > used by Dom2 too, we have to pick a new COS ID 3. The original values of
> > Dom1 on COS ID 3 may be below:
>
> What original values? You said you pick COS ID 3, because I think it's
> assumed to be empty? In which case there are no original values in COS
> ID 3.
>
Sorry for confusion. The original value means the default value. For CAT, it is
0x7ff on my machine which is shown below.
> > ---------
> > | COS 3 |
> > ---------
> > L3 CAT | 0x7ff |
> > ---------
> > MBA | 0x0 |
> > ---------
> > diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
> > index 4a0c982..ce82975 100644
> > --- a/xen/arch/x86/psr.c
> > +++ b/xen/arch/x86/psr.c
> > @@ -138,6 +138,12 @@ static const struct feat_props {
> >
> > /* write_msr is used to write out feature MSR register. */
> > void (*write_msr)(unsigned int cos, uint32_t val, enum psr_val_type type);
> > +
> > + /*
> > + * check_val is used to check if input val fulfills SDM requirement.
> > + * Change it to valid value if SDM allows.
>
> I'm not really sure it's a good idea to change the value to a valid
> one, IMHO you should just check and print an error if the value is
> invalid (and return false of course).
>
Per SDM, the HW has ability to automatically change the input value to what it
wants. E.g:
Linear mode: HW wants the input value be 10/20/30/.../90. But user inputs 15.
Then, HW can automatically change it to 10.
Even user inputs a value that does not fulfill HW requirement, HW can handle it.
So, we do not need return error to user. Otherwise, user needs to know details
of MBA.
But the issue here is how we get the actual value and show it to user. There are
two ways to do that:
1. When setting value, check and change it to valid one and save it to our cache.
2. When getting value, call rdmsr to read the actual value back from HW.
I think option 1 has better performance and the code looks better.
> > + */
> > + bool (*check_val)(const struct feat_node *feat, unsigned long *val);
> > } *feat_props[FEAT_TYPE_NUM];
> >
[...]
> > /* L3 CAT props */
> > static void l3_cat_write_msr(unsigned int cos, uint32_t val,
> > enum psr_val_type type)
> > @@ -446,6 +453,7 @@ static const struct feat_props l3_cat_props = {
> > .alt_type = PSR_VAL_TYPE_UNKNOWN,
> > .get_feat_info = cat_get_feat_info,
> > .write_msr = l3_cat_write_msr,
> > + .check_val = cat_check_cbm,
> > };
>
> Maybe the introduction of check_val should be a separate patch? It's
> mostly code movement and some fixup.
>
Ok, may consider it.
[...]
> > +static bool mba_check_thrtl(const struct feat_node *feat, unsigned long *thrtl)
> > +{
> > + if ( *thrtl > feat->mba_info.thrtl_max )
> > + return false;
> > +
> > + /*
> > + * Per SDM (chapter "Memory Bandwidth Allocation Configuration"):
> > + * 1. Linear mode: In the linear mode the input precision is defined
> > + * as 100-(MBA_MAX). For instance, if the MBA_MAX value is 90, the
> > + * input precision is 10%. Values not an even multiple of the
> > + * precision (e.g., 12%) will be rounded down (e.g., to 10% delay
> > + * applied).
> > + * 2. Non-linear mode: Input delay values are powers-of-two from zero
> > + * to the MBA_MAX value from CPUID. In this case any values not a
> > + * power of two will be rounded down the next nearest power of two.
> > + */
> > + if ( feat->mba_info.linear )
> > + {
> > + unsigned int mod;
> > +
> > + mod = *thrtl % (100 - feat->mba_info.thrtl_max);
> > + *thrtl -= mod;
> > + }
> > + else
> > + {
> > + /* Not power of 2. */
> > + if ( *thrtl && (*thrtl & (*thrtl - 1)) )
>
> This can be joined with the else to avoid another indentation level:
>
> else if ( *thrtl && (*thrtl & (*thrtl - 1)) )
> ...
>
Thanks!
> > + *thrtl = *thrtl & (1 << (flsl(*thrtl) - 1));
> > + }
> > +
> > + return true;
> > }
> >
[...]
> > static void do_write_psr_msrs(void *data)
> > {
> > const struct cos_write_info *info = data;
> > - struct feat_node *feat = info->feature;
> > - const struct feat_props *props = info->props;
> > - unsigned int i, cos = info->cos, cos_num = props->cos_num;
> > + unsigned int i, j, index = 0, array_len = info->array_len, cos = info->cos;
> > + const uint32_t *val_array = info->val;
> >
> > - for ( i = 0; i < cos_num; i++ )
> > + for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
> > {
>
> index and j can be defined here, they are only used inside of this for
> loop AFAICT.
>
I think definition of j can be moved into the loop. But index cannot unless I
declared it to be 'static'. The index is used as a accumulator.
> > - if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
> > + struct feat_node *feat = info->features[i];
> > + const struct feat_props *props = info->props[i];
> > + unsigned int cos_num;
> > +
> > + if ( !feat || !props )
> > + continue;
> > +
> > + cos_num = props->cos_num;
> > + if ( array_len < cos_num )
> > + return;
> > +
> > + for ( j = 0; j < cos_num; j++ )
> > {
> > - feat->cos_reg_val[cos * cos_num + i] = info->val[i];
> > - props->write_msr(cos, info->val[i], props->type[i]);
> > + if ( feat->cos_reg_val[cos * cos_num + j] != val_array[index + j] )
> > + {
> > + feat->cos_reg_val[cos * cos_num + j] = val_array[index + j];
> > + props->write_msr(cos, val_array[index + j], props->type[j]);
> > + }
> > }
> > +
> > + array_len -= cos_num;
> > + index += cos_num;
> > }
> > }
> >
> > @@ -1224,30 +1289,19 @@ static int write_psr_msrs(unsigned int socket, unsigned int cos,
> > const uint32_t val[], unsigned int array_len,
> > enum psr_feat_type feat_type)
> > {
> > - int ret;
> > struct psr_socket_info *info = get_socket_info(socket);
>
> info should probably be const here.
>
info could not be const. Because 'features' in 'cos_write_info' could not be
const.
> > struct cos_write_info data =
> > {
> > .cos = cos,
> > - .feature = info->features[feat_type],
> > - .props = feat_props[feat_type],
> > + .features = info->features,
> > + .val = val,
> > + .array_len = array_len,
> > + .props = feat_props,
> > };
>
> AFAICT data should also be const, but I guess this is not going to
> work because on_selected_cpus expects a non-const payload?
>
Right, data cannot be const per 'on_selected_cpus' requirement.
> Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 08/15] tools: create general interfaces to support psr allocation features
2017-08-30 8:42 ` Roger Pau Monné
@ 2017-08-31 2:38 ` Yi Sun
2017-08-31 8:37 ` Roger Pau Monn�
0 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-31 2:38 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On 17-08-30 09:42:56, Roger Pau Monn� wrote:
> On Thu, Aug 24, 2017 at 09:14:42AM +0800, Yi Sun wrote:
> > This patch creates general interfaces in libxl to support all psr
> > allocation features.
> >
> > Add 'LIBXL_HAVE_PSR_MBA' to indicate interface change.
>
> I'm not sure this is enough to cover the changes you are doing here:
> you are introducing some MBA stuff, plus a kind of generic interface
> for PSR.
>
> I think this should be split into two patches, the first one adding
> the generic interface, and the second one adding the MBA stuff.
>
This patch only introduces the generic interfaces without any MBA stuff.
The 'LIBXL_HAVE_PSR_MBA' is used to indicate the interfaces change here.
Per my understand, we should add a macro to indicate libxl interfaces
change, right?
> > --- a/tools/libxl/libxl.h
> > +++ b/tools/libxl/libxl.h
> > @@ -931,6 +931,13 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const libxl_mac *src);
> > #define LIBXL_HAVE_PSR_L2_CAT 1
> >
> > /*
> > + * LIBXL_HAVE_PSR_MBA
> > + *
> > + * If this is defined, the Memory Bandwidth Allocation feature is supported.
> > + */
> > +#define LIBXL_HAVE_PSR_MBA 1
> > +
> > +/*
> > * LIBXL_HAVE_MCA_CAPS
> > *
> > * If this is defined, setting MCA capabilities for HVM domain is supported.
> > @@ -2219,7 +2226,33 @@ int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
> > int libxl_psr_cat_get_l3_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
> > int *nr);
> > void libxl_psr_cat_info_list_free(libxl_psr_cat_info *list, int nr);
> > -#endif
> > +
> > +#ifdef LIBXL_HAVE_PSR_MBA
>
> You don't need this, this is only for consumers of libxl. It is
> perfectly fine to have the prototypes of the functions, even if
> consumers don't use them.
>
Oh, ok. So the interfaces declaration does not need be included by macro. I see
some other interfaces are done so. So, I follow the convention.
> > +/*
> > + * Function to set a domain's value. It operates on a single or multiple
> > + * target(s) defined in 'target_map'. 'target_map' specifies all the sockets
> > + * to be operated on.
> > + */
> > +int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
> > + libxl_psr_cbm_type type, libxl_bitmap *target_map,
> > + uint64_t val);
> > +/*
> > + * Function to get a domain's cbm. It operates on a single 'target'.
> > + * 'target' specifies which socket to be operated on.
> > + */
> > +int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
> > + libxl_psr_cbm_type type, unsigned int target,
> > + uint64_t *val);
> > +/*
> > + * On success, the function returns an array of elements in 'info',
> > + * and the length in 'nr'.
> > + */
> > +int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_hw_info **info,
> > + unsigned int *nr, libxl_psr_feat_type type,
> > + unsigned int lvl);
> > +void libxl_psr_hw_info_list_free(libxl_psr_hw_info *list, unsigned int nr);
> > +#endif /* LIBXL_HAVE_PSR_MBA */
> > +#endif /* LIBXL_HAVE_PSR_CAT */
>
> Please send a patch to remove the already existing ifdef PSR
> pollution.
>
Ok, I will send a patch to remoev this '#ifdef LIBXL_HAVE_PSR_CAT' in this file.
[...]
> > diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> > index 6e80d36..ab847f8 100644
> > --- a/tools/libxl/libxl_types.idl
> > +++ b/tools/libxl/libxl_types.idl
> > @@ -977,6 +977,7 @@ libxl_psr_cbm_type = Enumeration("psr_cbm_type", [
> > (2, "L3_CBM_CODE"),
> > (3, "L3_CBM_DATA"),
> > (4, "L2_CBM"),
> > + (5, "MBA_THRTL"),
>
> Is this really a CBM type?
>
This is not CBM type. The 'libxl_psr_cbm_type' name is not good enough. But I
have to introduce a new generic interface here if we want to make the name be
generic. I think it is not so valuable. So, I reuse the 'libxl_psr_cbm_type'
to cover MBA. How do you think?
> Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 09/15] tools: implement the new libxc get hw info interface
2017-08-30 8:58 ` Roger Pau Monné
@ 2017-08-31 3:05 ` Yi Sun
0 siblings, 0 replies; 64+ messages in thread
From: Yi Sun @ 2017-08-31 3:05 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, xen-devel,
chao.p.peng
On 17-08-30 09:58:03, Roger Pau Monn� wrote:
> On Thu, Aug 24, 2017 at 09:14:43AM +0800, Yi Sun wrote:
> > diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
> > index c7710b8..81a6f48 100644
> > --- a/tools/libxc/include/xenctrl.h
> > +++ b/tools/libxc/include/xenctrl.h
> > @@ -2458,6 +2458,31 @@ enum xc_psr_cat_type {
> > };
> > typedef enum xc_psr_cat_type xc_psr_cat_type;
> >
> > +enum xc_psr_feat_type {
> > + XC_PSR_FEAT_UNKNOWN,
> > + XC_PSR_FEAT_CAT_L3,
> > + XC_PSR_FEAT_CAT_L2,
> > + XC_PSR_FEAT_MBA,
> > +};
> > +typedef enum xc_psr_feat_type xc_psr_feat_type;
> > +
> > +struct xc_psr_hw_info {
> > + union {
> > + struct {
> > + uint32_t cos_max;
> > + uint32_t cbm_len;
> > + bool cdp_enabled;
> > + } xc_cat_info;
> > +
> > + struct {
> > + uint32_t cos_max;
> > + uint32_t thrtl_max;
> > + bool linear;
> > + } xc_mba_info;
>
> No need for the _info suffix. The type itself already has info in it's
> name.
>
Ok, got it.
> > + } u;
> > +};
> > +typedef struct xc_psr_hw_info xc_psr_hw_info;
[...]
> > diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
> > index 73d05f2..ba412e4 100644
> > --- a/tools/libxc/xc_psr.c
> > +++ b/tools/libxc/xc_psr.c
> > @@ -323,36 +323,58 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
> > return rc;
> > }
> >
> > -int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
> > - uint32_t *cos_max, uint32_t *cbm_len, bool *cdp_enabled)
> > +int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
> > + xc_psr_feat_type type, xc_psr_hw_info *hw_info)
> > {
> > int rc = -1;
> > DECLARE_SYSCTL;
> >
> > + if ( !hw_info )
> > + return rc;
>
> You should set errno = EINVAL here.
>
Sorry for missing it. Thanks!
[...]
> > diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
> > index cf368ba..b183305 100644
> > --- a/tools/libxl/libxl_psr.c
> > +++ b/tools/libxl/libxl_psr.c
> > @@ -361,6 +361,27 @@ int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
> > return rc;
> > }
> >
> > +static inline xc_psr_feat_type libxl__psr_feat_type_to_libxc_psr_feat_type(
>
> No inline please. In any case this is not performance critical code, so
> let the compiler decide.
>
Ok, will remove it.
> And the function name could be shorter, maybe:
>
> libxl__psr_type_to_libxc_type
>
> Or is this going to clash with some other translation function?
>
Yes, there is another similar function which name is:
'libxl__psr_cbm_type_to_libxc_psr_val_type'
I think I can remove the last '_psr' to make the name be shorter.
[...]
> > @@ -385,16 +408,23 @@ int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
> > goto out;
> > }
> >
> > + xc_type = libxl__psr_feat_type_to_libxc_psr_feat_type(
> > + LIBXL_PSR_FEAT_TYPE_CAT, lvl);
>
> Shouldn't you check that xc_type != XC_PSR_FEAT_UNKNOWN here?
>
Yes, I should. Thanks!
> Roger.
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 10/15] tools: implement the new libxl get hw info interface
2017-08-30 9:15 ` Roger Pau Monné
@ 2017-08-31 3:16 ` Yi Sun
2017-08-31 8:40 ` Roger Pau Monn�
0 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-31 3:16 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On 17-08-30 10:15:22, Roger Pau Monn� wrote:
> On Thu, Aug 24, 2017 at 09:14:44AM +0800, Yi Sun wrote:
> > diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
> > index b183305..d7da7d7 100644
> > --- a/tools/libxl/libxl_psr.c
> > +++ b/tools/libxl/libxl_psr.c
> > @@ -382,56 +382,51 @@ static inline xc_psr_feat_type libxl__psr_feat_type_to_libxc_psr_feat_type(
> > return xc_type;
> > }
> >
> > +static inline int libxl__psr_hw_info_to_libxl_psr_cat_info(
>
> No inline. Maybe you could try to shorter the name?
>
Got it. Will remove the last '_psr'.
[...]
> > int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
> > int *nr, unsigned int lvl)
> > {
> > GC_INIT(ctx);
> > int rc;
> > - int i = 0, socketid, nr_sockets;
> > - libxl_bitmap socketmap;
> > + unsigned int i;
> > + libxl_psr_hw_info *hw_info;
> > libxl_psr_cat_info *ptr;
> > - xc_psr_hw_info hw_info;
> > - xc_psr_feat_type xc_type;
> > -
> > - libxl_bitmap_init(&socketmap);
> > -
> > - rc = libxl__count_physical_sockets(gc, &nr_sockets);
> > - if (rc) {
> > - LOGE(ERROR, "failed to get system socket count");
> > - goto out;
> > - }
> >
> > - libxl_socket_bitmap_alloc(ctx, &socketmap, nr_sockets);
> > - rc = libxl_get_online_socketmap(ctx, &socketmap);
> > - if (rc < 0) {
> > - LOGE(ERROR, "failed to get available sockets");
> > + rc = libxl_psr_get_hw_info(ctx, &hw_info, (unsigned int *)nr,
>
> Is there any reason nr is int instead of unsigned int?
>
> I would rather avoid casting things. Since this interface has not been
> present in a release yet, could you please send a separate patch to
> fix this if nr has no reason to be signed?
>
This is a historical issue.
The first version of PSR introduced 'libxl_psr_cat_get_l3_info'. The input
parameter is 'int *nr'.
I think we cannot change the interface which has been merged and used by
others. Right?
> > + LIBXL_PSR_FEAT_TYPE_CAT, lvl);
[...]
> >
> > +static inline int libxc__psr_hw_info_to_libxl_psr_hw_info(
>
> No inline. Again shorter names would be better (although I understand
> this might not be possible).
>
> Also, why are you adding a libxc__ prefixed function to libxl?
>
Because this function is to convert 'xc_psr_hw_info' to 'libxl_psr_hw_info'.
I think I may change the name to 'libxl__xc_psr_info_to_libxl_psr_info'.
> > + libxl_psr_feat_type type, xc_psr_hw_info *xc_hw_info,
> > + libxl_psr_hw_info *xl_hw_info)
>
> you could drop the '_hw' in the parameter names, so all the
> assignments below would fit on a single line.
>
Ok, thanks!
> > +{
> > + switch (type) {
> > + case LIBXL_PSR_FEAT_TYPE_CAT:
> > + xl_hw_info->u.cat.cos_max = xc_hw_info->u.xc_cat_info.cos_max;
> > + xl_hw_info->u.cat.cbm_len = xc_hw_info->u.xc_cat_info.cbm_len;
> > + xl_hw_info->u.cat.cdp_enabled =
> > + xc_hw_info->u.xc_cat_info.cdp_enabled;
> > + break;
> > + case LIBXL_PSR_FEAT_TYPE_MBA:
> > + xl_hw_info->u.mba.cos_max = xc_hw_info->u.xc_mba_info.cos_max;
> > + xl_hw_info->u.mba.thrtl_max = xc_hw_info->u.xc_mba_info.thrtl_max;
> > + xl_hw_info->u.mba.linear = xc_hw_info->u.xc_mba_info.linear;
> > + break;
> > + default:
> > + return ERROR_INVAL;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_hw_info **info,
> > unsigned int *nr, libxl_psr_feat_type type,
> > unsigned int lvl)
> > {
> > - return ERROR_FAIL;
> > + GC_INIT(ctx);
> > + int rc, nr_sockets;
> > + unsigned int i = 0, socketid;
> > + libxl_bitmap socketmap;
> > + libxl_psr_hw_info *ptr;
> > + xc_psr_feat_type xc_type;
> > + xc_psr_hw_info hw_info;
> > +
> > + libxl_bitmap_init(&socketmap);
> > +
> > + if (type == LIBXL_PSR_FEAT_TYPE_CAT && lvl != 3 && lvl != 2) {
> > + LOGE(ERROR, "input lvl %d is wrong!\n", lvl);
>
> LOGE is used when errno is set, which I don't think it's the case
> here. Please use plain LOG.
>
> > + rc = ERROR_FAIL;
> > + goto out;
> > + }
> > +
> > + xc_type = libxl__psr_feat_type_to_libxc_psr_feat_type(type, lvl);
>
> Isn't the above function going to return and invalid type if the
> feature is CAT and the level is not correct? In which case you could
> remove the above if and just check that the returned type here is not
> invalid?
>
Ok, will check remove the first check and check return value here.
> > +
> > + rc = libxl__count_physical_sockets(gc, &nr_sockets);
> > + if (rc) {
> > + LOGE(ERROR, "failed to get system socket count");
>
> Again, libxl__ functions don't set errno. In this case this might be
> fine, because libxl__count_physical_sockets calls into libxc which
> sets errno, but you should only use LOGE when reporting errors from
> system calls or libxc functions.
>
Ok will change it to LOG.
> Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 11/15] tools: implement the new xl get hw info interface
2017-08-30 9:23 ` Roger Pau Monné
@ 2017-08-31 5:57 ` Yi Sun
2017-08-31 8:43 ` Roger Pau Monn�
0 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-31 5:57 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On 17-08-30 10:23:18, Roger Pau Monn� wrote:
> On Thu, Aug 24, 2017 at 09:14:45AM +0800, Yi Sun wrote:
> > +static int psr_mba_hwinfo(void)
> > +{
> > + int rc;
> > + unsigned int i, nr;
> > + libxl_psr_hw_info *info;
> > +
> > + rc = libxl_psr_get_hw_info(ctx, &info, &nr,
> > + LIBXL_PSR_FEAT_TYPE_MBA, 0);
> > + if (rc)
> > + return rc;
> > +
> > + printf("Memory Bandwidth Allocation (MBA):\n");
> > +
> > + for (i = 0; i < nr; i++) {
> > + printf("%-16s: %u\n", "Socket ID", info[i].id);
> > + printf("%-16s: %s\n", "Linear Mode",
> > + info[i].u.mba.linear ? "Enabled" : "Disabled");
> > + printf("%-16s: %u\n", "Maximum COS", info[i].u.mba.cos_max);
> > + printf("%-16s: %u\n", "Maximum Throttling Value",
> > + info[i].u.mba.thrtl_max);
> > + printf("%-16s: %u\n", "Default Throttling Value", 0);
>
> If you really want to left-justify, shouldn't you choose a value that
> aligns everything nicely (strlen("Default Throttling Valu") is
> already greater than 16).
>
Sorry for missing this.
> In fact you can do the alignment manually in the format string, and
> avoid passing the name as the first parameter.
>
DYM a sentence like below?
printf("%-*s: %u\n", 23, "Default Throttling Value", 0);
> > + }
> > +
> > + libxl_psr_hw_info_list_free(info, nr);
> > + return rc;
> > +}
> > +
> > int main_psr_cat_cbm_set(int argc, char **argv)
> > {
> > uint32_t domid;
> > @@ -597,20 +624,24 @@ int main_psr_cat_show(int argc, char **argv)
> > int main_psr_hwinfo(int argc, char **argv)
> > {
> > int opt, ret = 0;
> > - bool all = true, cmt = false, cat = false;
> > + bool all = true, cmt = false, cat = false, mba = false;
> > static struct option opts[] = {
>
> const?
>
Ok, thanks!
> Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 14/15] tools: implement new generic set value interface and MBA set value command
2017-08-30 9:47 ` Roger Pau Monné
@ 2017-08-31 5:58 ` Yi Sun
0 siblings, 0 replies; 64+ messages in thread
From: Yi Sun @ 2017-08-31 5:58 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On 17-08-30 10:47:37, Roger Pau Monn� wrote:
> On Thu, Aug 24, 2017 at 09:14:48AM +0800, Yi Sun wrote:
> > int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
> > @@ -457,7 +431,33 @@ int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
> > libxl_psr_cbm_type type, libxl_bitmap *target_map,
> > uint64_t val)
> > {
> > - return ERROR_FAIL;
> > + GC_INIT(ctx);
> > + int rc;
> > + int socketid, nr_sockets;
> > +
> > + rc = libxl__count_physical_sockets(gc, &nr_sockets);
> > + if (rc) {
> > + LOGED(ERROR, domid, "failed to get system socket count");
> > + goto out;
> > + }
> > +
> > + libxl_for_each_set_bit(socketid, *target_map) {
> > + xc_psr_val_type xc_type;
> > +
> > + if (socketid >= nr_sockets)
> > + break;
> > +
> > + xc_type = libxl__psr_cbm_type_to_libxc_psr_val_type(type);
> > + if (xc_psr_set_domain_data(ctx->xch, domid, xc_type,
> > + socketid, val)) {
> > + libxl__psr_alloc_log_err_msg(gc, errno, type);
> > + rc = ERROR_FAIL;
> > + }
> > + }
> > +
> > +out:
> > + GC_FREE;
> > + return rc;
>
> Is there no libxl_psr_get_val?
>
Yes, patch 13 implements it.
> > }
> >
> > +int main_psr_mba_set(int argc, char **argv)
> > +{
> > + uint32_t domid;
> > + libxl_psr_cbm_type type;
> > + uint64_t thrtl;
> > + int ret, opt = 0;
> > + libxl_bitmap target_map;
> > + char *value;
> > + libxl_string_list socket_list;
> > + unsigned long start, end;
> > + unsigned int i, j, len;
> > +
> > + static struct option opts[] = {
>
> const
>
Got it.
> Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 07/15] x86: implement set value flow for MBA
2017-08-31 2:20 ` Yi Sun
@ 2017-08-31 8:30 ` Roger Pau Monn�
2017-08-31 9:13 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monn� @ 2017-08-31 8:30 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 31, 2017 at 10:20:10AM +0800, Yi Sun wrote:
> On 17-08-30 09:31:04, Roger Pau Monn� wrote:
> > On Thu, Aug 24, 2017 at 09:14:41AM +0800, Yi Sun wrote:
> > > diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
> > > index 4a0c982..ce82975 100644
> > > --- a/xen/arch/x86/psr.c
> > > +++ b/xen/arch/x86/psr.c
> > > @@ -138,6 +138,12 @@ static const struct feat_props {
> > >
> > > /* write_msr is used to write out feature MSR register. */
> > > void (*write_msr)(unsigned int cos, uint32_t val, enum psr_val_type type);
> > > +
> > > + /*
> > > + * check_val is used to check if input val fulfills SDM requirement.
> > > + * Change it to valid value if SDM allows.
> >
> > I'm not really sure it's a good idea to change the value to a valid
> > one, IMHO you should just check and print an error if the value is
> > invalid (and return false of course).
> >
> Per SDM, the HW has ability to automatically change the input value to what it
> wants. E.g:
> Linear mode: HW wants the input value be 10/20/30/.../90. But user inputs 15.
> Then, HW can automatically change it to 10.
This seems like a very, very bad idea IMHO. What if I input 15 (which
is invalid), but I would rather prefer 20 instead of 10?
I would very much prefer an error, so I can get to chose a valid value
myself instead of the hardware deciding on my back.
> Even user inputs a value that does not fulfill HW requirement, HW can handle it.
> So, we do not need return error to user. Otherwise, user needs to know details
> of MBA.
>
> But the issue here is how we get the actual value and show it to user. There are
> two ways to do that:
> 1. When setting value, check and change it to valid one and save it to our cache.
Instead of performing those checks manually, why not simply write the
value and read it back to see the one the hardware has actually
chosen (and cache it)?
Is there any risk in writing an invalid value?
> 2. When getting value, call rdmsr to read the actual value back from HW.
>
> I think option 1 has better performance and the code looks better.
>
> > > + */
> > > + bool (*check_val)(const struct feat_node *feat, unsigned long *val);
> > > } *feat_props[FEAT_TYPE_NUM];
> > >
> [...]
>
> > > /* L3 CAT props */
> > > static void l3_cat_write_msr(unsigned int cos, uint32_t val,
> > > enum psr_val_type type)
> > > @@ -446,6 +453,7 @@ static const struct feat_props l3_cat_props = {
> > > .alt_type = PSR_VAL_TYPE_UNKNOWN,
> > > .get_feat_info = cat_get_feat_info,
> > > .write_msr = l3_cat_write_msr,
> > > + .check_val = cat_check_cbm,
> > > };
> >
> > Maybe the introduction of check_val should be a separate patch? It's
> > mostly code movement and some fixup.
> >
> Ok, may consider it.
>
> [...]
>
> > > +static bool mba_check_thrtl(const struct feat_node *feat, unsigned long *thrtl)
> > > +{
> > > + if ( *thrtl > feat->mba_info.thrtl_max )
> > > + return false;
> > > +
> > > + /*
> > > + * Per SDM (chapter "Memory Bandwidth Allocation Configuration"):
> > > + * 1. Linear mode: In the linear mode the input precision is defined
> > > + * as 100-(MBA_MAX). For instance, if the MBA_MAX value is 90, the
> > > + * input precision is 10%. Values not an even multiple of the
> > > + * precision (e.g., 12%) will be rounded down (e.g., to 10% delay
> > > + * applied).
> > > + * 2. Non-linear mode: Input delay values are powers-of-two from zero
> > > + * to the MBA_MAX value from CPUID. In this case any values not a
> > > + * power of two will be rounded down the next nearest power of two.
> > > + */
> > > + if ( feat->mba_info.linear )
> > > + {
> > > + unsigned int mod;
> > > +
> > > + mod = *thrtl % (100 - feat->mba_info.thrtl_max);
> > > + *thrtl -= mod;
> > > + }
> > > + else
> > > + {
> > > + /* Not power of 2. */
> > > + if ( *thrtl && (*thrtl & (*thrtl - 1)) )
> >
> > This can be joined with the else to avoid another indentation level:
> >
> > else if ( *thrtl && (*thrtl & (*thrtl - 1)) )
> > ...
> >
> Thanks!
>
> > > + *thrtl = *thrtl & (1 << (flsl(*thrtl) - 1));
> > > + }
> > > +
> > > + return true;
> > > }
> > >
> [...]
>
> > > static void do_write_psr_msrs(void *data)
> > > {
> > > const struct cos_write_info *info = data;
> > > - struct feat_node *feat = info->feature;
> > > - const struct feat_props *props = info->props;
> > > - unsigned int i, cos = info->cos, cos_num = props->cos_num;
> > > + unsigned int i, j, index = 0, array_len = info->array_len, cos = info->cos;
> > > + const uint32_t *val_array = info->val;
> > >
> > > - for ( i = 0; i < cos_num; i++ )
> > > + for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
> > > {
> >
> > index and j can be defined here, they are only used inside of this for
> > loop AFAICT.
> >
> I think definition of j can be moved into the loop. But index cannot unless I
> declared it to be 'static'. The index is used as a accumulator.
Right, I've got my indentation messed up. Please move j.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 08/15] tools: create general interfaces to support psr allocation features
2017-08-31 2:38 ` Yi Sun
@ 2017-08-31 8:37 ` Roger Pau Monn�
2017-09-04 2:09 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monn� @ 2017-08-31 8:37 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 31, 2017 at 10:38:46AM +0800, Yi Sun wrote:
> On 17-08-30 09:42:56, Roger Pau Monn� wrote:
> > On Thu, Aug 24, 2017 at 09:14:42AM +0800, Yi Sun wrote:
> > > This patch creates general interfaces in libxl to support all psr
> > > allocation features.
> > >
> > > Add 'LIBXL_HAVE_PSR_MBA' to indicate interface change.
> >
> > I'm not sure this is enough to cover the changes you are doing here:
> > you are introducing some MBA stuff, plus a kind of generic interface
> > for PSR.
> >
> > I think this should be split into two patches, the first one adding
> > the generic interface, and the second one adding the MBA stuff.
> >
> This patch only introduces the generic interfaces without any MBA stuff.
>
> The 'LIBXL_HAVE_PSR_MBA' is used to indicate the interfaces change here.
> Per my understand, we should add a macro to indicate libxl interfaces
> change, right?
So if there's no MBA specific interfaces introduced, why name the
define PSR_MBA? Maybe PSR_GENERIC or whatever you find more suitable.
> > > diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> > > index 6e80d36..ab847f8 100644
> > > --- a/tools/libxl/libxl_types.idl
> > > +++ b/tools/libxl/libxl_types.idl
> > > @@ -977,6 +977,7 @@ libxl_psr_cbm_type = Enumeration("psr_cbm_type", [
> > > (2, "L3_CBM_CODE"),
> > > (3, "L3_CBM_DATA"),
> > > (4, "L2_CBM"),
> > > + (5, "MBA_THRTL"),
> >
> > Is this really a CBM type?
> >
> This is not CBM type. The 'libxl_psr_cbm_type' name is not good enough. But I
> have to introduce a new generic interface here if we want to make the name be
> generic. I think it is not so valuable. So, I reuse the 'libxl_psr_cbm_type'
> to cover MBA. How do you think?
Maybe you could introduce a new typedef, so that old code call still
work, ie:
typedef enum libxl_psr_cbm_type libxl_psr_type;
(Or whatever name you find suitable).
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 10/15] tools: implement the new libxl get hw info interface
2017-08-31 3:16 ` Yi Sun
@ 2017-08-31 8:40 ` Roger Pau Monn�
2017-08-31 9:19 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monn� @ 2017-08-31 8:40 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 31, 2017 at 11:16:34AM +0800, Yi Sun wrote:
> On 17-08-30 10:15:22, Roger Pau Monn� wrote:
> > On Thu, Aug 24, 2017 at 09:14:44AM +0800, Yi Sun wrote:
> > > diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
> > > index b183305..d7da7d7 100644
> > > --- a/tools/libxl/libxl_psr.c
> > > +++ b/tools/libxl/libxl_psr.c
> > > @@ -382,56 +382,51 @@ static inline xc_psr_feat_type libxl__psr_feat_type_to_libxc_psr_feat_type(
> > > return xc_type;
> > > }
> > >
> > > +static inline int libxl__psr_hw_info_to_libxl_psr_cat_info(
> >
> > No inline. Maybe you could try to shorter the name?
> >
> Got it. Will remove the last '_psr'.
>
> [...]
>
> > > int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
> > > int *nr, unsigned int lvl)
> > > {
> > > GC_INIT(ctx);
> > > int rc;
> > > - int i = 0, socketid, nr_sockets;
> > > - libxl_bitmap socketmap;
> > > + unsigned int i;
> > > + libxl_psr_hw_info *hw_info;
> > > libxl_psr_cat_info *ptr;
> > > - xc_psr_hw_info hw_info;
> > > - xc_psr_feat_type xc_type;
> > > -
> > > - libxl_bitmap_init(&socketmap);
> > > -
> > > - rc = libxl__count_physical_sockets(gc, &nr_sockets);
> > > - if (rc) {
> > > - LOGE(ERROR, "failed to get system socket count");
> > > - goto out;
> > > - }
> > >
> > > - libxl_socket_bitmap_alloc(ctx, &socketmap, nr_sockets);
> > > - rc = libxl_get_online_socketmap(ctx, &socketmap);
> > > - if (rc < 0) {
> > > - LOGE(ERROR, "failed to get available sockets");
> > > + rc = libxl_psr_get_hw_info(ctx, &hw_info, (unsigned int *)nr,
> >
> > Is there any reason nr is int instead of unsigned int?
> >
> > I would rather avoid casting things. Since this interface has not been
> > present in a release yet, could you please send a separate patch to
> > fix this if nr has no reason to be signed?
> >
> This is a historical issue.
>
> The first version of PSR introduced 'libxl_psr_cat_get_l3_info'. The input
> parameter is 'int *nr'.
>
> I think we cannot change the interface which has been merged and used by
> others. Right?
Has libxl_psr_cat_get_info been present in any Xen release? (I don't
think so) If it hasn't then you can change the interface without
issues.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 11/15] tools: implement the new xl get hw info interface
2017-08-31 5:57 ` Yi Sun
@ 2017-08-31 8:43 ` Roger Pau Monn�
2017-08-31 9:24 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monn� @ 2017-08-31 8:43 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 31, 2017 at 01:57:09PM +0800, Yi Sun wrote:
> On 17-08-30 10:23:18, Roger Pau Monn� wrote:
> > On Thu, Aug 24, 2017 at 09:14:45AM +0800, Yi Sun wrote:
> > > +static int psr_mba_hwinfo(void)
> > > +{
> > > + int rc;
> > > + unsigned int i, nr;
> > > + libxl_psr_hw_info *info;
> > > +
> > > + rc = libxl_psr_get_hw_info(ctx, &info, &nr,
> > > + LIBXL_PSR_FEAT_TYPE_MBA, 0);
> > > + if (rc)
> > > + return rc;
> > > +
> > > + printf("Memory Bandwidth Allocation (MBA):\n");
> > > +
> > > + for (i = 0; i < nr; i++) {
> > > + printf("%-16s: %u\n", "Socket ID", info[i].id);
> > > + printf("%-16s: %s\n", "Linear Mode",
> > > + info[i].u.mba.linear ? "Enabled" : "Disabled");
> > > + printf("%-16s: %u\n", "Maximum COS", info[i].u.mba.cos_max);
> > > + printf("%-16s: %u\n", "Maximum Throttling Value",
> > > + info[i].u.mba.thrtl_max);
> > > + printf("%-16s: %u\n", "Default Throttling Value", 0);
> >
> > If you really want to left-justify, shouldn't you choose a value that
> > aligns everything nicely (strlen("Default Throttling Valu") is
> > already greater than 16).
> >
> Sorry for missing this.
>
> > In fact you can do the alignment manually in the format string, and
> > avoid passing the name as the first parameter.
> >
> DYM a sentence like below?
> printf("%-*s: %u\n", 23, "Default Throttling Value", 0);
I was thinking more like:
printf(" Maximum COS: %u\n", ...
printf("Maximum Throttling Value: %u\n", ...
printf("Default Throttling Value: %u\n", ...
Or however you wish to align them. Although I don't have a strong
opinion, as long as it's easy to read.
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 07/15] x86: implement set value flow for MBA
2017-08-31 8:30 ` Roger Pau Monn�
@ 2017-08-31 9:13 ` Yi Sun
2017-08-31 9:30 ` Roger Pau Monn�
0 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-31 9:13 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On 17-08-31 09:30:47, Roger Pau Monn� wrote:
> On Thu, Aug 31, 2017 at 10:20:10AM +0800, Yi Sun wrote:
> > On 17-08-30 09:31:04, Roger Pau Monn� wrote:
> > > On Thu, Aug 24, 2017 at 09:14:41AM +0800, Yi Sun wrote:
> > > > diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
> > > > index 4a0c982..ce82975 100644
> > > > --- a/xen/arch/x86/psr.c
> > > > +++ b/xen/arch/x86/psr.c
> > > > @@ -138,6 +138,12 @@ static const struct feat_props {
> > > >
> > > > /* write_msr is used to write out feature MSR register. */
> > > > void (*write_msr)(unsigned int cos, uint32_t val, enum psr_val_type type);
> > > > +
> > > > + /*
> > > > + * check_val is used to check if input val fulfills SDM requirement.
> > > > + * Change it to valid value if SDM allows.
> > >
> > > I'm not really sure it's a good idea to change the value to a valid
> > > one, IMHO you should just check and print an error if the value is
> > > invalid (and return false of course).
> > >
> > Per SDM, the HW has ability to automatically change the input value to what it
> > wants. E.g:
> > Linear mode: HW wants the input value be 10/20/30/.../90. But user inputs 15.
> > Then, HW can automatically change it to 10.
>
> This seems like a very, very bad idea IMHO. What if I input 15 (which
> is invalid), but I would rather prefer 20 instead of 10?
>
> I would very much prefer an error, so I can get to chose a valid value
> myself instead of the hardware deciding on my back.
>
> > Even user inputs a value that does not fulfill HW requirement, HW can handle it.
> > So, we do not need return error to user. Otherwise, user needs to know details
> > of MBA.
> >
> > But the issue here is how we get the actual value and show it to user. There are
> > two ways to do that:
> > 1. When setting value, check and change it to valid one and save it to our cache.
>
> Instead of performing those checks manually, why not simply write the
> value and read it back to see the one the hardware has actually
> chosen (and cache it)?
>
So, I guess you prefer optioin 2, right? If so, do we need check if input value
is valid? I see above comment that you prefer to return an error and leave the
decision to user.
> Is there any risk in writing an invalid value?
If input value is no more than the thrtl_max got through CPUID, it is fine that
HW can handle it.
>
> > 2. When getting value, call rdmsr to read the actual value back from HW.
> >
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 10/15] tools: implement the new libxl get hw info interface
2017-08-31 8:40 ` Roger Pau Monn�
@ 2017-08-31 9:19 ` Yi Sun
2017-08-31 9:32 ` Roger Pau Monn�
0 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-31 9:19 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On 17-08-31 09:40:37, Roger Pau Monn� wrote:
> On Thu, Aug 31, 2017 at 11:16:34AM +0800, Yi Sun wrote:
> > On 17-08-30 10:15:22, Roger Pau Monn� wrote:
> > > On Thu, Aug 24, 2017 at 09:14:44AM +0800, Yi Sun wrote:
> > > > int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
> > > > int *nr, unsigned int lvl)
> > > > {
> > > > GC_INIT(ctx);
> > > > int rc;
> > > > - int i = 0, socketid, nr_sockets;
> > > > - libxl_bitmap socketmap;
> > > > + unsigned int i;
> > > > + libxl_psr_hw_info *hw_info;
> > > > libxl_psr_cat_info *ptr;
> > > > - xc_psr_hw_info hw_info;
> > > > - xc_psr_feat_type xc_type;
> > > > -
> > > > - libxl_bitmap_init(&socketmap);
> > > > -
> > > > - rc = libxl__count_physical_sockets(gc, &nr_sockets);
> > > > - if (rc) {
> > > > - LOGE(ERROR, "failed to get system socket count");
> > > > - goto out;
> > > > - }
> > > >
> > > > - libxl_socket_bitmap_alloc(ctx, &socketmap, nr_sockets);
> > > > - rc = libxl_get_online_socketmap(ctx, &socketmap);
> > > > - if (rc < 0) {
> > > > - LOGE(ERROR, "failed to get available sockets");
> > > > + rc = libxl_psr_get_hw_info(ctx, &hw_info, (unsigned int *)nr,
> > >
> > > Is there any reason nr is int instead of unsigned int?
> > >
> > > I would rather avoid casting things. Since this interface has not been
> > > present in a release yet, could you please send a separate patch to
> > > fix this if nr has no reason to be signed?
> > >
> > This is a historical issue.
> >
> > The first version of PSR introduced 'libxl_psr_cat_get_l3_info'. The input
> > parameter is 'int *nr'.
> >
> > I think we cannot change the interface which has been merged and used by
> > others. Right?
>
> Has libxl_psr_cat_get_info been present in any Xen release? (I don't
> think so) If it hasn't then you can change the interface without
> issues.
>
libxl_psr_cat_get_info is just introduced and not released yet. But
libxl_psr_cat_get_l3_info has been released for long time. It declares
'int *nr' and it calls 'libxl_psr_cat_get_info' to do the work. So, there
must be a casting anyway.
What do you suggest? Shall I submit a patch to fix 'libxl_psr_cat_get_info'?
Or, a patch to fix both 'libxl_psr_cat_get_info' and 'libxl_psr_cat_get_l3_info'?
> Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 11/15] tools: implement the new xl get hw info interface
2017-08-31 8:43 ` Roger Pau Monn�
@ 2017-08-31 9:24 ` Yi Sun
0 siblings, 0 replies; 64+ messages in thread
From: Yi Sun @ 2017-08-31 9:24 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On 17-08-31 09:43:51, Roger Pau Monn� wrote:
> On Thu, Aug 31, 2017 at 01:57:09PM +0800, Yi Sun wrote:
> > On 17-08-30 10:23:18, Roger Pau Monn� wrote:
> > > On Thu, Aug 24, 2017 at 09:14:45AM +0800, Yi Sun wrote:
> > > > +static int psr_mba_hwinfo(void)
> > > > +{
> > > > + int rc;
> > > > + unsigned int i, nr;
> > > > + libxl_psr_hw_info *info;
> > > > +
> > > > + rc = libxl_psr_get_hw_info(ctx, &info, &nr,
> > > > + LIBXL_PSR_FEAT_TYPE_MBA, 0);
> > > > + if (rc)
> > > > + return rc;
> > > > +
> > > > + printf("Memory Bandwidth Allocation (MBA):\n");
> > > > +
> > > > + for (i = 0; i < nr; i++) {
> > > > + printf("%-16s: %u\n", "Socket ID", info[i].id);
> > > > + printf("%-16s: %s\n", "Linear Mode",
> > > > + info[i].u.mba.linear ? "Enabled" : "Disabled");
> > > > + printf("%-16s: %u\n", "Maximum COS", info[i].u.mba.cos_max);
> > > > + printf("%-16s: %u\n", "Maximum Throttling Value",
> > > > + info[i].u.mba.thrtl_max);
> > > > + printf("%-16s: %u\n", "Default Throttling Value", 0);
> > >
> > > If you really want to left-justify, shouldn't you choose a value that
> > > aligns everything nicely (strlen("Default Throttling Valu") is
> > > already greater than 16).
> > >
> > Sorry for missing this.
> >
> > > In fact you can do the alignment manually in the format string, and
> > > avoid passing the name as the first parameter.
> > >
> > DYM a sentence like below?
> > printf("%-*s: %u\n", 23, "Default Throttling Value", 0);
>
> I was thinking more like:
>
> printf(" Maximum COS: %u\n", ...
> printf("Maximum Throttling Value: %u\n", ...
> printf("Default Throttling Value: %u\n", ...
>
> Or however you wish to align them. Although I don't have a strong
> opinion, as long as it's easy to read.
>
Got it. To keep the format be same as CAT, I would like to keep left alignment:
printf("Maximum COS : %u\n", ...
printf("Maximum Throttling Value: %u\n", ...
printf("Default Throttling Value: %u\n", ...
> Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 07/15] x86: implement set value flow for MBA
2017-08-31 9:13 ` Yi Sun
@ 2017-08-31 9:30 ` Roger Pau Monn�
2017-08-31 10:10 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monn� @ 2017-08-31 9:30 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 31, 2017 at 05:13:08PM +0800, Yi Sun wrote:
> On 17-08-31 09:30:47, Roger Pau Monn� wrote:
> > On Thu, Aug 31, 2017 at 10:20:10AM +0800, Yi Sun wrote:
> > > On 17-08-30 09:31:04, Roger Pau Monn� wrote:
> > > > On Thu, Aug 24, 2017 at 09:14:41AM +0800, Yi Sun wrote:
> > > Even user inputs a value that does not fulfill HW requirement, HW can handle it.
> > > So, we do not need return error to user. Otherwise, user needs to know details
> > > of MBA.
> > >
> > > But the issue here is how we get the actual value and show it to user. There are
> > > two ways to do that:
> > > 1. When setting value, check and change it to valid one and save it to our cache.
> >
> > Instead of performing those checks manually, why not simply write the
> > value and read it back to see the one the hardware has actually
> > chosen (and cache it)?
> >
> So, I guess you prefer optioin 2, right? If so, do we need check if input value
> is valid? I see above comment that you prefer to return an error and leave the
> decision to user.
Yes, I think so. Is reading the MSR in any performance critical path?
Or it's just used by the user-space tools?
> > Is there any risk in writing an invalid value?
>
> If input value is no more than the thrtl_max got through CPUID, it is fine that
> HW can handle it.
OK, so I would just check for that then.
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 10/15] tools: implement the new libxl get hw info interface
2017-08-31 9:19 ` Yi Sun
@ 2017-08-31 9:32 ` Roger Pau Monn�
2017-08-31 10:11 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monn� @ 2017-08-31 9:32 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 31, 2017 at 05:19:56PM +0800, Yi Sun wrote:
> On 17-08-31 09:40:37, Roger Pau Monn� wrote:
> > On Thu, Aug 31, 2017 at 11:16:34AM +0800, Yi Sun wrote:
> > > On 17-08-30 10:15:22, Roger Pau Monn� wrote:
> > > > On Thu, Aug 24, 2017 at 09:14:44AM +0800, Yi Sun wrote:
> > Has libxl_psr_cat_get_info been present in any Xen release? (I don't
> > think so) If it hasn't then you can change the interface without
> > issues.
> >
> libxl_psr_cat_get_info is just introduced and not released yet. But
> libxl_psr_cat_get_l3_info has been released for long time. It declares
> 'int *nr' and it calls 'libxl_psr_cat_get_info' to do the work. So, there
> must be a casting anyway.
>
> What do you suggest? Shall I submit a patch to fix 'libxl_psr_cat_get_info'?
> Or, a patch to fix both 'libxl_psr_cat_get_info' and 'libxl_psr_cat_get_l3_info'?
Just fixing libxl_psr_cat_get_info would be fine IMHO, let's try to
limit the int usage as much as possible.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 07/15] x86: implement set value flow for MBA
2017-08-31 9:30 ` Roger Pau Monn�
@ 2017-08-31 10:10 ` Yi Sun
2017-08-31 10:19 ` Roger Pau Monn�
0 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-08-31 10:10 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On 17-08-31 10:30:37, Roger Pau Monn� wrote:
> On Thu, Aug 31, 2017 at 05:13:08PM +0800, Yi Sun wrote:
> > On 17-08-31 09:30:47, Roger Pau Monn� wrote:
> > > On Thu, Aug 31, 2017 at 10:20:10AM +0800, Yi Sun wrote:
> > > > On 17-08-30 09:31:04, Roger Pau Monn� wrote:
> > > > > On Thu, Aug 24, 2017 at 09:14:41AM +0800, Yi Sun wrote:
> > > > Even user inputs a value that does not fulfill HW requirement, HW can handle it.
> > > > So, we do not need return error to user. Otherwise, user needs to know details
> > > > of MBA.
> > > >
> > > > But the issue here is how we get the actual value and show it to user. There are
> > > > two ways to do that:
> > > > 1. When setting value, check and change it to valid one and save it to our cache.
> > >
> > > Instead of performing those checks manually, why not simply write the
> > > value and read it back to see the one the hardware has actually
> > > chosen (and cache it)?
> > >
> > So, I guess you prefer optioin 2, right? If so, do we need check if input value
> > is valid? I see above comment that you prefer to return an error and leave the
> > decision to user.
>
> Yes, I think so. Is reading the MSR in any performance critical path?
> Or it's just used by the user-space tools?
>
Just used by user-space tools.
> > > Is there any risk in writing an invalid value?
> >
> > If input value is no more than the thrtl_max got through CPUID, it is fine that
> > HW can handle it.
>
> OK, so I would just check for that then.
>
Ok, will check this only in 'check_val' function.
> Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 10/15] tools: implement the new libxl get hw info interface
2017-08-31 9:32 ` Roger Pau Monn�
@ 2017-08-31 10:11 ` Yi Sun
0 siblings, 0 replies; 64+ messages in thread
From: Yi Sun @ 2017-08-31 10:11 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On 17-08-31 10:32:09, Roger Pau Monn� wrote:
> On Thu, Aug 31, 2017 at 05:19:56PM +0800, Yi Sun wrote:
> > On 17-08-31 09:40:37, Roger Pau Monn� wrote:
> > > On Thu, Aug 31, 2017 at 11:16:34AM +0800, Yi Sun wrote:
> > > > On 17-08-30 10:15:22, Roger Pau Monn� wrote:
> > > > > On Thu, Aug 24, 2017 at 09:14:44AM +0800, Yi Sun wrote:
> > > Has libxl_psr_cat_get_info been present in any Xen release? (I don't
> > > think so) If it hasn't then you can change the interface without
> > > issues.
> > >
> > libxl_psr_cat_get_info is just introduced and not released yet. But
> > libxl_psr_cat_get_l3_info has been released for long time. It declares
> > 'int *nr' and it calls 'libxl_psr_cat_get_info' to do the work. So, there
> > must be a casting anyway.
> >
> > What do you suggest? Shall I submit a patch to fix 'libxl_psr_cat_get_info'?
> > Or, a patch to fix both 'libxl_psr_cat_get_info' and 'libxl_psr_cat_get_l3_info'?
>
> Just fixing libxl_psr_cat_get_info would be fine IMHO, let's try to
> limit the int usage as much as possible.
>
Ok, then, I think another separate fix patch is needed.
> Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 07/15] x86: implement set value flow for MBA
2017-08-31 10:10 ` Yi Sun
@ 2017-08-31 10:19 ` Roger Pau Monn�
0 siblings, 0 replies; 64+ messages in thread
From: Roger Pau Monn� @ 2017-08-31 10:19 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On Thu, Aug 31, 2017 at 06:10:07PM +0800, Yi Sun wrote:
> On 17-08-31 10:30:37, Roger Pau Monn� wrote:
> > On Thu, Aug 31, 2017 at 05:13:08PM +0800, Yi Sun wrote:
> > > On 17-08-31 09:30:47, Roger Pau Monn� wrote:
> > > > On Thu, Aug 31, 2017 at 10:20:10AM +0800, Yi Sun wrote:
> > > > > On 17-08-30 09:31:04, Roger Pau Monn� wrote:
> > > > > > On Thu, Aug 24, 2017 at 09:14:41AM +0800, Yi Sun wrote:
> > > > > Even user inputs a value that does not fulfill HW requirement, HW can handle it.
> > > > > So, we do not need return error to user. Otherwise, user needs to know details
> > > > > of MBA.
> > > > >
> > > > > But the issue here is how we get the actual value and show it to user. There are
> > > > > two ways to do that:
> > > > > 1. When setting value, check and change it to valid one and save it to our cache.
> > > >
> > > > Instead of performing those checks manually, why not simply write the
> > > > value and read it back to see the one the hardware has actually
> > > > chosen (and cache it)?
> > > >
> > > So, I guess you prefer optioin 2, right? If so, do we need check if input value
> > > is valid? I see above comment that you prefer to return an error and leave the
> > > decision to user.
> >
> > Yes, I think so. Is reading the MSR in any performance critical path?
> > Or it's just used by the user-space tools?
> >
> Just used by user-space tools.
Then doing a MSR read each time doesn't seem that bad IMHO (it's not a
critical path anyway).
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 08/15] tools: create general interfaces to support psr allocation features
2017-08-31 8:37 ` Roger Pau Monn�
@ 2017-09-04 2:09 ` Yi Sun
2017-09-04 8:43 ` Wei Liu
0 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-09-04 2:09 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel
On 17-08-31 09:37:45, Roger Pau Monn� wrote:
> On Thu, Aug 31, 2017 at 10:38:46AM +0800, Yi Sun wrote:
> > On 17-08-30 09:42:56, Roger Pau Monn� wrote:
> > > On Thu, Aug 24, 2017 at 09:14:42AM +0800, Yi Sun wrote:
> > > > diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> > > > index 6e80d36..ab847f8 100644
> > > > --- a/tools/libxl/libxl_types.idl
> > > > +++ b/tools/libxl/libxl_types.idl
> > > > @@ -977,6 +977,7 @@ libxl_psr_cbm_type = Enumeration("psr_cbm_type", [
> > > > (2, "L3_CBM_CODE"),
> > > > (3, "L3_CBM_DATA"),
> > > > (4, "L2_CBM"),
> > > > + (5, "MBA_THRTL"),
> > >
> > > Is this really a CBM type?
> > >
> > This is not CBM type. The 'libxl_psr_cbm_type' name is not good enough. But I
> > have to introduce a new generic interface here if we want to make the name be
> > generic. I think it is not so valuable. So, I reuse the 'libxl_psr_cbm_type'
> > to cover MBA. How do you think?
>
> Maybe you could introduce a new typedef, so that old code call still
> work, ie:
>
> typedef enum libxl_psr_cbm_type libxl_psr_type;
>
Is there a way in '.idl' to make such 'typedef'? Thanks!
> (Or whatever name you find suitable).
>
> Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 08/15] tools: create general interfaces to support psr allocation features
2017-09-04 2:09 ` Yi Sun
@ 2017-09-04 8:43 ` Wei Liu
0 siblings, 0 replies; 64+ messages in thread
From: Wei Liu @ 2017-09-04 8:43 UTC (permalink / raw)
To: Yi Sun
Cc: kevin.tian, wei.liu2, andrew.cooper3, dario.faggioli,
ian.jackson, julien.grall, mengxu, jbeulich, chao.p.peng,
xen-devel, Roger Pau Monn�
On Mon, Sep 04, 2017 at 10:09:48AM +0800, Yi Sun wrote:
> On 17-08-31 09:37:45, Roger Pau Monn� wrote:
> > On Thu, Aug 31, 2017 at 10:38:46AM +0800, Yi Sun wrote:
> > > On 17-08-30 09:42:56, Roger Pau Monn� wrote:
> > > > On Thu, Aug 24, 2017 at 09:14:42AM +0800, Yi Sun wrote:
> > > > > diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> > > > > index 6e80d36..ab847f8 100644
> > > > > --- a/tools/libxl/libxl_types.idl
> > > > > +++ b/tools/libxl/libxl_types.idl
> > > > > @@ -977,6 +977,7 @@ libxl_psr_cbm_type = Enumeration("psr_cbm_type", [
> > > > > (2, "L3_CBM_CODE"),
> > > > > (3, "L3_CBM_DATA"),
> > > > > (4, "L2_CBM"),
> > > > > + (5, "MBA_THRTL"),
> > > >
> > > > Is this really a CBM type?
> > > >
> > > This is not CBM type. The 'libxl_psr_cbm_type' name is not good enough. But I
> > > have to introduce a new generic interface here if we want to make the name be
> > > generic. I think it is not so valuable. So, I reuse the 'libxl_psr_cbm_type'
> > > to cover MBA. How do you think?
> >
> > Maybe you could introduce a new typedef, so that old code call still
> > work, ie:
> >
> > typedef enum libxl_psr_cbm_type libxl_psr_type;
> >
> Is there a way in '.idl' to make such 'typedef'? Thanks!
>
No (or not yet?).
Please do it in libxl.h.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
end of thread, other threads:[~2017-09-04 8:43 UTC | newest]
Thread overview: 64+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-08-24 1:14 [PATCH v2 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
2017-08-24 1:14 ` [PATCH v2 01/15] docs: create Memory Bandwidth Allocation (MBA) feature document Yi Sun
2017-08-29 11:46 ` Roger Pau Monné
2017-08-30 5:20 ` Yi Sun
2017-08-30 7:42 ` Roger Pau Monn�
2017-08-24 1:14 ` [PATCH v2 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general Yi Sun
2017-08-29 12:00 ` Roger Pau Monné
2017-08-30 5:23 ` Yi Sun
2017-08-30 7:47 ` Roger Pau Monn�
2017-08-30 8:14 ` Yi Sun
2017-08-24 1:14 ` [PATCH v2 03/15] x86: rename 'cbm_type' to 'psr_val_type' to make it general Yi Sun
2017-08-29 12:15 ` Roger Pau Monné
2017-08-30 5:47 ` Yi Sun
2017-08-30 7:51 ` Roger Pau Monn�
2017-08-30 8:14 ` Yi Sun
2017-08-24 1:14 ` [PATCH v2 04/15] x86: implement data structure and CPU init flow for MBA Yi Sun
2017-08-29 13:44 ` Roger Pau Monné
2017-08-29 13:58 ` Jan Beulich
2017-08-30 6:07 ` Yi Sun
2017-08-30 5:31 ` Yi Sun
2017-08-30 7:55 ` Roger Pau Monn�
2017-08-30 8:19 ` Yi Sun
2017-08-30 8:45 ` Jan Beulich
2017-08-24 1:14 ` [PATCH v2 05/15] x86: implement get hw info " Yi Sun
2017-08-29 15:01 ` Roger Pau Monné
2017-08-30 5:33 ` Yi Sun
2017-08-24 1:14 ` [PATCH v2 06/15] x86: implement get value interface " Yi Sun
2017-08-29 15:04 ` Roger Pau Monné
2017-08-24 1:14 ` [PATCH v2 07/15] x86: implement set value flow " Yi Sun
2017-08-30 8:31 ` Roger Pau Monné
2017-08-31 2:20 ` Yi Sun
2017-08-31 8:30 ` Roger Pau Monn�
2017-08-31 9:13 ` Yi Sun
2017-08-31 9:30 ` Roger Pau Monn�
2017-08-31 10:10 ` Yi Sun
2017-08-31 10:19 ` Roger Pau Monn�
2017-08-24 1:14 ` [PATCH v2 08/15] tools: create general interfaces to support psr allocation features Yi Sun
2017-08-30 8:42 ` Roger Pau Monné
2017-08-31 2:38 ` Yi Sun
2017-08-31 8:37 ` Roger Pau Monn�
2017-09-04 2:09 ` Yi Sun
2017-09-04 8:43 ` Wei Liu
2017-08-24 1:14 ` [PATCH v2 09/15] tools: implement the new libxc get hw info interface Yi Sun
2017-08-30 8:58 ` Roger Pau Monné
2017-08-31 3:05 ` Yi Sun
2017-08-24 1:14 ` [PATCH v2 10/15] tools: implement the new libxl " Yi Sun
2017-08-30 9:15 ` Roger Pau Monné
2017-08-31 3:16 ` Yi Sun
2017-08-31 8:40 ` Roger Pau Monn�
2017-08-31 9:19 ` Yi Sun
2017-08-31 9:32 ` Roger Pau Monn�
2017-08-31 10:11 ` Yi Sun
2017-08-24 1:14 ` [PATCH v2 11/15] tools: implement the new xl " Yi Sun
2017-08-30 9:23 ` Roger Pau Monné
2017-08-31 5:57 ` Yi Sun
2017-08-31 8:43 ` Roger Pau Monn�
2017-08-31 9:24 ` Yi Sun
2017-08-24 1:14 ` [PATCH v2 12/15] tools: rename 'xc_psr_cat_type' to 'xc_psr_val_type' Yi Sun
2017-08-30 9:24 ` Roger Pau Monné
2017-08-24 1:14 ` [PATCH v2 13/15] tools: implement new generic get value interface and MBA get value command Yi Sun
2017-08-24 1:14 ` [PATCH v2 14/15] tools: implement new generic set value interface and MBA set " Yi Sun
2017-08-30 9:47 ` Roger Pau Monné
2017-08-31 5:58 ` Yi Sun
2017-08-24 1:14 ` [PATCH v2 15/15] docs: add MBA description in docs Yi Sun
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.